The difference is a 3.5x modifier vs. 4.5x. It's important for Siegfried, but no so much for the other heroic spirits.
SSR weapons equipped as secondary aren't that much stronger than SR weapons, especially before LV85. So comparisons between a single-element team and a rotating opposite-element team come down to tradeoffs between assault skill and the 45-48% elemental advantage (together with 25% lower damage received). Rotating teams compare favorably at low weapon skill levels. But as content increases in difficulty and demands ever-higher damage output, you'll be forced to level assault skill for 6 elements instead of 1 and farm weapons for every element just to keep pace. Not to mention some of your teams won't have the proper balance to clear difficult content like Ultimate (lacking healing, debuffs, utility or all of the above). It's better to invest heavily in one element that has the best team composition. It'll pay off in the long-term, even if at low levels you're still changing teams for every event. The base stats from the weapons will still buff each team.