Garden Ops glitch, 5 Quest waves instead of 10 waves (Xbox One)

I attempted to play in Garden Ops mode on Xbox Live with friends today in a friend only match. We would start by choosing the options in the backyard at the RV. When we exited the RV at the start of the game there would only be one garden, instead of the usual 3, to choose from. Instead of the normal Garden Op waves we would have a Citron on the Time Park level, facing zombie robots or Dr. Zomboss on the Egypt level and they would be talking/giving quest instructions and all the waves were "quest" waves.The waves would only go to 5 and the "quest" would end and nothing would happen, all you can do is roam around in an enemy free level. The only way to fix it was to get rid of the two friends that had progressed further in the backyard quests than the rest of us. We had two, out of our group of 5, that were the culprits. We never had both of them in our game at the same time. If we created a match and spawned in without them, there would be three gardens. If one of them joined our match before we chose a garden two of the gardens would disappear. The only way around the glitch was too create a match, choose our garden and start the wave. Then invite the affected person(s) to the match and the garden ops game would continue normally. We were able to finish a full match and start another from the lobby with no issues. The only way around this glitch is to have a friend start a garden ops match and join in progress.