I think something is up, whenever the game crashes and I load it up again I'm missing a couple gems. Sometimes it happens when a timer finishes as well I just go down by 2 gems.

and no, I'm not clicking any speed up buttons, I hear the little "X has finished doing whatever" chime and poof my gem count goes down. WTF