How common are SR drops at layer 3/4? Given how uncommon SSR drops seem to be, you may have to resort to leveled up SR accessories as your exp fodder. Sure, 2 lv 1 SRs will give the same exp as 1 lv 1 SSR, but if you want to be gem efficient in leveling an SSR...
lv 27 SR gives 407 exp, lv 1 SSR gives 400 exp; both before same element bonus. And getting an accessory to lv 27 in the first place takes 2305 exp itself. 8 lv 1 SRs of the same element takes care of that. So that's... 9 SRs of one element + 800 gems to approximate 1 SSR as exp fodder. Swap out some of those SRs with R's if you don't mind raising gem cost and if R's are common enough drops. It's basically also 2 R's for 1 SR.