MUT X factor issue

Tried to switch which players have X factors on them for my MUT team and now when I try to add a 3rd player as a active X factor the game tells me I have to many players with abilities on. I even lowered the players with Superstar AP abilities from 9 to 7 yet I still can't activate a 3 X factor player.