Same here. The only time I go into public Raids is to dump the raid points to "over level" when I'm about to cross over or I'm going to stop playing, otherwise it's super wasteful to jump in to someone else's.
You get +400/500 or 700/800 for Standard and Expert respectively even for just swinging about 6 rounds or so, so it's worth doing for the XP. But if you need the drops it's just kind of pointless.