Do your Spellcasters win or lose their duels?

My Spellcasters are at the highest level, have purchased all the perks, completed both Realm of Magic aspirations and STILL lose most duels. What gives? Does the charge make a difference? With all of ...