Self-hosted Portal servers all break

I've been trying to load into some Portal servers that have increased vehicle spawns so I can practice the air vehicles. But whenever I load into a map, it breaks. The map loads but nothing else does, then myself and all the bots in the lobby spawn in the center of the map, and sometimes underneath it. As the title says, I've been launching the server myself and setting a password on it so I can be solo. 

I know the server is broken when I load in, and instead of the normal minimap view I get a black void (1st attachment). Then the spawning under the map.