Each girl has a specialty just like you do. The stat that correlates to their specialty boosts attack, the other two boost defense against that particular attack type. When you switch red battler who is a know-how type for Juliette who is a charm type your charm defense drops because Juliettes charm stat raises attack, not charm defense. This is also why you want your team to have the same specialty as you.