You use them on a Black Unit/a unit that's hard to Skill Up due to a unit having that skill being scarce IMHO. Like when the Ninja Event started I tried to collect as many Azami as possible, then bought Saki from the Trading Post and used them to Skill Up her. She's a 4/5 and I plan to use a few Spirits on her. If you're 1 (or 2) Skill Points away using it for that last one helps.