My initial guess would be that there are more players playing 128-player sessions, and therefore more than enough people to populate a server locally. They all seem to be AmazonAWS servers, from what I have observed, by the way.
64-player sessions are relatively new with (currently) less players. Therefore these servers will sometimes have to allow players from further afield to connect in order to populate a game session... higher ping... etc., etc.
As time goes on, hopefully more people will join 64-player sessions and it should become less of an issue.
That's my take on it, but there are many variables to online gameplay. I haven't noticed any issues with 64-player sessions so far on PS5. Seems smooth here.
