The answer to the OP's question depends on the answer to another question: are you willing to spend money on this game?

If yes, then the flower gems have significantly more value for expanding your inventories and refreshing stamina (not raid points since they regenerate fairly quickly).

If no, then the only way you're going to get the better performing/higher stat girls is to play the gacha and yes, you should use FGs for it.

The event girls are fine for a good while when starting out and a few of the event girls can hold their own or better against many gacha 5* (Edelweiss, Lavender for instance), but for the most part, event girls aren't as strong as the gacha 5* if you disregard the equipment slots with one caveat: if you're not spending money on the game, then you're not likely getting any dressblooms which means the only way to build gacha girls skills/equipment slots is to fuse dupes and that is going to take a -long- while to do so.