Buy it if you want, but it's by no means mandatory.

Only 3 buildings are. Two other are even useless to upgrade.

And once you are done with building, the upgrade is wasted (when pulls are not, except if you draw a dupe...)