Yeah, the timers seem designed to aggressively force you to use gems to clear cooldowns.
Get home ~7pm. ~1 hour to use up energy. ~5 hours to sleep. Energy is back to full (and stats are near empty) right around 1am. If I don't try to force cooldowns, I'm staying up to almost 2 am, or going to bed with stats already near 0 for significant degredation.
(for now, I've just started waking 'em up early. Hopefully there isn't a significant penalty for that.)