Toddler/Child Hair Changing Color

Is there a way to stop the game from changing the hair color of toddlers/children when using CC hair colors? I keep setting the color I want in CAS, but when I go back to game mode it's a different color. It also happens when I use more than one outfit per category. I searched for an answer on this but couldn't find one.