Can’t join each others party from same network

My brother and I can’t join each other’s party from different ps5 consoles on the same network. This only happens with each other. I’m able to join other peoples party’s and I can join others.  ...