FC24 Loan to buy can be cancelled by parent club?

When I have a player on loan and agreed to buy them at the end, why does the parent club still exercise the right to recall the player? Usually on the charge that I am not playing them enough, when in reality they have maybe missed 2 games out of the 30 played?