Townies randomly return to default look after editing in CAS

After editing townies in CAS (ie refreshing their looks with new hair and clothes, face altering), they often randomly change back into their default look when in gameplay. During gameplay, they go back and forth between their old and new look, and there seems to be no way for me to fix this. It usually happens after loading screens.