Why does camera jump up and down by a floor on some houses not others?

So on some houses (but not on others), whether it's in build mode or live mode, the camera when it gets to it seems one of the outside walls of the house, the camera will jump up a whole floor then down again as you move over it. If you move the camera just enough to be over the wall, you jump up a floor's distance and sty there until you rotate or move the camera.

Some houses it does things like this, others it does not. Is there a way to have this no longer happen? (Live mode is the biggest issue not so much a concern about it during build mode). Anyone know what causes this that can be avoided?

Thank you in advance.