The method relies on three components: (1) a set of visual objects with associated situation impact ratings obtained by crowdsourcing, (2) a corresponding set of object detectors for mining users' photos, and (3) a ground truth dataset made of 500 visual user profiles which are manually rated per situation. These components are combined in LERVUP, a method that learns to rate visual user profiles in each situation. LERVUP exploits a new image descriptor that aggregates object ratings and object detections at the user level and an attention mechanism that boosts highly-rated objects to prevent them from being overwhelmed by low-rated ones.