Sims over using hidden passages

I have the hidden passage clock in 8 rooms in my new castle build, and my sims are using them CONSTANTLY! I I have 4 bedrooms right next to each other, and a living space very close. Each room has the clock. This is the 3rd floor of my build. I also have 3 clocks on the ground floor. My sims are autonomously using the passages every time they move rooms, but not going directly to the room they need to go, they're going through multiple to get to the room right next to where they started.

Xbox series X,  most packs.