It's definitely not worth losing sleep over so long as you have core units that need polishing. Typically the best maps for collecting Silver units are the low stamina cost ones anyway. That's fine if we get a Star Event or you consider the special unit to be a bench warmer but otherwise you're stuck making compromises between collecting fodder and getting the special unit at good CR/SU.