If you don't mind paying the gem cost to take it off later in case you find a better accessory, then sure?

+ability damage really is bleh on single hit skills due to the way it works. You'd think that it'd be something like 'skill damage is increased by so and so percentage'. But that's not how it works! What it does is just add to the skill's multiplier per hit.

So, direct damage skills are basically of the form 1 hit at X power (or X range of power), or Y hits at X power (where power is what to multiply damage by). For example, let's say there's someone with a skill that does 1 hit at 2-3x power, as a well as a skill that does 4 hits at 0.7x power.
Now you throw in a +ability damage buff that's listed as +0.15. And add on an assist that gives +0.05, so there's a total of +0.2 ability damage.
Now the first skill becomes 1 hit at 2.2-3.2x power. The second skill becomes 4 hits at 0.9x power.
(I just described Artemis)

Back to accessories, it looks like the +ability damage effect found on accessories ranges from +0.03 to +0.05.