On PS5 at least, I don't notice any difference between using Game mode and Sport mode. Mind you, my current 4k TV is more on the bargain side of things... will be easy enough to test once in I bring in big boy LG C2. Generally speaking though, yes, Game mode is what we should all be playing.
Your displayed connection could be wrong as well - Been a problem with NHL for years and I doubt it will ever get fixed. Just the other day my screen said HIGH population (no, seriously, it's real) with an average search of about 35 seconds. My buddy on the other hand said his should Low population and an average search of 1 minute+