I have been debugging this issue further. It seems that AMD B550 is faulty and it has inaccurate Real Time Clock. This can be easily checked by using CPU-z Timers feature.
The one way to "fix" this issue is to transfer data to a USB 3.0 dongle and then measure the Real Time Clock by using CPU-z. When data transfer is active, RTC is very accurate but when USB 3.0 memory dongle is disconnected and data is not being transaferred RTC gets back to inaccurate.
This is a bug in AMD's motherboard drivers and / or AMD system as a whole and this must have something to do with AGESA drivers being still faulty (latest version). Considering RMA your AMD B550's asap as there are no known fix for this.
Best regards,
-Salosieppo
Community Manager
