If we assume that it only happens when the boss is shared with allies only, I would say when the time has gone from the 2 (or 24 for the seed boss) hour mark down to less then 30 minutes left I would say it's okay to go for the killing blow. Because it takes 30 minutes to recharge a raid point anyway, and you don't want the boss to escape. And the one that discovered the boss might need help taking down the boss when time is running out, so yeah, I would say when it's less then 30 minutes left, then it's time. Maybe a little bit earlier, like 40 minutes.
Otherwise you can risk waiting to recharge multiple raid points, spend more then 1 point on a boss, and then only to sit there without any points when you really need them.
There is an option to instantly recharge the raid points for a flower gem, but it doesn't really feel worth it.