Two quick ideas:
1) It should ping me when someone wants to fight me in arena. Give me 60 seconds to respond. If I don't accept the challenge in the 60 seconds then it can't use the AI to fight for me.
2) Make it so the person you just fought can't fight you back without spending 50 crystals for the revenge match.
3) or, coming from the opposite angle of idea #2, if someone defeats you, you could get a message saying "_____ just beat you in arena, would you like a rematch?" This would bypass the timer, but still use an arena turn.
I don't know, Just spitballing here.