Several things can happen. If you have many objects that use the pattern, your game may hang and fail to load (that progress bar at the start of the game, ceases to move at a certain point), or your game may crash. In my experience, I had a lovely pattern I got from Exchange. it worked great for a month that I started a new sim. I liked the pattern so much I made it her trademark, so some accent pieces in her house had the pattern and her career outfit and vehicle also had that pattern. It was fun, so far so good.
Then POOF one day the pattern disappeared. My sim was only visible from her bright green plumbob. I couldn't find her vehicle and the couple pieces of furniture that had the pattern reverted to a default color for the EA item.
I was able to find the pattern in the installed items of the launcher. So what I did was go back into the game and changed the pattern of everything I could remember using the pattern for, back to a default. I removed the customized career outfit. I removed the Sims3Pack from the launcher by using the uninstall option then completely deleted the pattern. I turned the pattern into a package and now it works as long as I'm using my mods folder.
Here's a link from the EA support site which has a part near the bottom, that mentions pattern problems (-it mentions the CAP issue which I discussed above).
see:
https://answers.ea.com/t5/The-Sims-3/HOW-TO-Find-and-remove-bad-custom-content-CC/td-p/2759940
Are there any bad patterns out there or are they always safe?
There are a few older patterns that were not made with the EA Pattern Tool that don't seem to work properly anymore. Usually you see long loading times in CaSt, especially when you try to access the Misc patterns.
There is also an issue if CAP patterns are converted into Package files. See this for details: CaST does not load – Materializing Materials
Sometimes a simple reinstall will work, other times you have to get rid of them. If you stick to newer patterns, you should be fine.