Losing fame, but still having negative effect of fame quirk?

I had a 1 star celebrity who got the "vanity" fame quirk, so he'd get tense if he didn't admire himself in a mirror for too long. Then his fame went down, so he "lost" the quirk, but he still gets that tense moodlet. Only now, there's no way for him to admire his "fine self" anymore! Anyone else have something like this happen? Is it a bug, or is that how it's supposed to be?