I've played with spellcasters in just about every other world and never had a problem.
You can buy the plants needed in the Starter Spellcaster plants seed pack, and after a couple trips to the magic realm (which you can do with the glimmerstone in your sim's inventory. The gate is only required for non-spellcasters to visit the magic realm.) the game has usually generated enough random npc's that you don't need to worry about visiting to have some other spellcasters for your sims to interact with. You'll see them arriving places by broom.