Equipping weapon changes appearance

Whenever I equip a new main hand weapon, it changes the appearance to that new weapon, even though I have a different weapon set as my appearance. If I go to my wardrobe and change it to something else, save it, then change it back it's fine again until I switch weapons.