I used to keep rares if there was some element that my SRs didn't cover for some hero that I used often. But keeping tabs with that is a pain in the ass, to be honest. You can just use them for skill level fodder as well.