The guardian isn't all that important. It'll also be switched around a lot. You mainly care about the union being active.

Story mode-wise, once you've gotten Nike/Amon/Cybele/Baal, you can stop. Maybe work on them during the off days between events if you want, but there's not really anything significant reward-wise outside of the jewels.

During a union event like right now, you normally ought to be using your AP on fighting lilims to farm grails for your union. But you've started recently enough such that I don't think you can clear standard lilims yet? In that case, elemental caves are fine, as you do need to limit break your SR kamihime. But generally speaking, once you are strong enough to contribute to your union during such an event, please do so.