I feel you on this. Many skin tones that are not meant to be fantasy/occult, look very odd. I like the idea/concept of cool/warm/neutral tones (and even maybe add olive tones). But there are some issues with the current set up.
Skin tones are not properly in order (it is not lightest to darkest)—this out of order swatch collection. This can result in to genetic mismatches, sometimes genetics want to assign between the parents skin tones resulting in odd/wrong colours (sometimes children are way darker than both parents, or just a weird colour).
I wish we had light to dark sliders for cool/warm/neutral/olive/etc. tones (like Sims 3) and be able to save those custom tones—instead of these weird swatches—that we can adjust/save after selection.