I personally find affection to be one of the most important things to raise on a girl. You get so many extra stats from 100% on 4*+ girls that it's close to another 30 or so levels and then you get to double that on evo!

Always check your stam when near new levels and if you have extra burn it on the 80 stam gift mission, it is worth it.