The raid boss gains some HP when people join, and loses HP when they attack. Sometimes if you have a long attack animation such as a full burst and a bunch of other things happened during it, you'll see them all 1 at a time after your turn finishes, such as alternating between hp up from joins and hp down from their attacks.