This happened to me the other day. Had a romance buff from the wishing well and took my Sim to visit a specific household. Guy was never home.
I saved right before going over and out of curiosity, I re-loaded the save and played the guy instead. Turned out he was off to his job.
So I think it happens when it's right around the time the Sim you're looking for would be going to work. The game thinks they're home when you're on the map screen, but they vanish off to work the second the lot loads.
It's a pain for sure. I would just try again after the workday would be over. (If you haven't played the Goth household, she's probably still in the Secret Agent career and probably works the usual 8-4, M-F schedule.)
Hope this helps!
