A Merfolk with high friendship or romance rel will tell a Sim if they are a merfolk. Otherwise, they deny it. This is the intended design.
You said they were dating, so I assume their romance bar was high, but how high was the friendship bar?
I asked a sim if she was a mermaid once we were "good friends" and she immediately confirmed, while at another time I asked a near stranger while swimming next to her and she denied it (which I personally found funny). So I think it's definitely linked to how high your relationship is. Maybe you need to be good friends? It might also depend on the mermaid's mood.
This might be too late to reply, but I found out that you have to be ''best friends'' with someone to have them say that they are a mermaid. I tried it with ''good friends'', but it didn't work for me as it did for you. When it comes to romantic relationships, I haven't tried just yet.
To answer you on the last thing you wrote: I don't think it depends on the mood of the mermaid. I tried asking when the mermaid and my sim was acquaintances (didn't work and she was happy), when they were good friends (still didn't work and she was happy), and then a last time when they had become best friends (it worked this time and she was still happy).
