Camera centering on non played sim

I generally play one house, and o noticed that the camera would occasionally center itself on one of the sims in the house, even if I want controling her and didn't hit any key to make it do so. I initially figured a mod may be acting weird and could ignore it. However, I was playing a different house, and when the sim from my main group came into the world, the camera instantly shot over to her again. It could still be a mod, but if it's not, I'd like it to be a known issue. It only centers on the one sim, no one else.