Just full disclosure, my Berith, Pluto and Osiris are all on different accounts, and I don't own any limited edition himes, so the following is based on theory rather than actual experience. People that run real dark teams should correct me if I am getting anything wrong.
You are losing 20% atk down and BP. Amon U covers 15% light atk down, has rage punishing skill and stun extension in exchange, so you may not be losing as much as you think, but that is dependent on what you are fighting.
I personally like Osiris for the team burst gen, but you can potentially access burst gen and/or DATA buffs from eidolons as well. Do note that if you don't run Andromeda as soul, dropping Osiris means full glass cannon, which is fine if you have high defender values.
This is my AAB-bias creeping through. You are correct, if you are not on AAB, having Pluto's buffs up when you FB is definitely worth more than Berith's dmg potential.
It comes largely down to how much hp you have. Hades, especially AW is very good at limiting enemy dmg potential, but not good for dealing dmg.
Now that I have thought about this a little more, I would rank the swaps as follows from least to most aggressive:
1. Swap Dark Amat for Amon U sacrifices the least in defense for a 10% gain in dmg
2. Swap Osiris for Pluto or Berith. You lose heal for dmg cut or taunt/intercept combo, and lose burst gen for higher atk (ok, I know Osiris has other skills, too, but they are not as important imo.) Hard to say whether those are improvements or not without some testing on whatever you are fighting.
3. Swap Hades for Berith/Pluto - you will likely take a noticeable hit in how long your team can survive, especially if Hades is AW. You should see a noticeable increase in dmg as well though, at least while you are alive.