I don't think it's really worth it to buy flower gems, unless there's a gacha costing FGs with a girl you REALLY want, but generally, the gachas paid with Nutaku Gold are slightly better value for money than buying FGs to play one. (So I'm specifically excluding hoarding free FGs from the game to occasionally play here)

But the game supplies enough Flower Gems to get by: increase Girl capacity, increasing jewelry storage and even occasionally using FGs for stamina (only worth it if you have a high maximum stamina to start with) or pulling a couple of bronze girls from limited gacha.