If you only have your Sims live in Glimmerbrook and never, ever have them travel to the Magic Realm by going through the portal, you won't see any magic being practiced anywhere at all. Spellcasters sometimes use brooms for travel (think witch), that's what the above Simmer was talking about. But like I said, you won't see any Sims riding brooms or any other kind of magic related actions as long as they don't go to the Magic Realm.
I have two save files with two individual Spellcasters. They both started out on the empty Glimmerbrook Watch lot as Rags to Riches Sims. Until they'd earned enough simoleons to build a shelter, I didn't send them to the Realm to become Spellcasters. In all the time they were there, before they went through the portal, I didn't see any kind of magic. In fact, I did see the Sages walking around, but if I hadn't known who they were beforehand, you'd never be able to tell at all that they were Sages. They just look and act like any ordinary Sims. I hope that helps! :)
EDIT: Just a heads up in case you didn't know - ALL the resident premades in Glimmerbrook are spellcasters. If your Sim adds them to your family for any reason, then they might start seeing magic. I'm not sure, but just an FYI. If you're worried about that, just delete the Glimmerbrook premade families and you should be good to go.