Sounds like your old guild plateaued due to lack of leadership. Once a guild gets to a certain level, you either lay down the law and get everyone to carry their weight, or you start hemorrhaging good players looking for something better. Most "leaders" just can't be bothered to put in the effort.
You don't have to leave your fun guild to join something resembling a military platoon of screeching nerdrage on discord, just need your lead to do something about it. The distinction isn't casual/hardcore, it's active/inactive.