Party set to invite only, randoms are still joining

Not sure if this is an oversight but since the season 2 update, the game has consistently been doing odd things regarding party options. The biggest being that my party is set to invite only or close...