Based on what I found it can be as easy as changing a few settings
https://www.reddit.com/r/firefox/comments/em250h/ssl_error_rx_malformed_server_hello_with_the_new/
As far as I can see frim the comments these settings can mess with server connection.
[code]
in about:config
Changing: security.tls.hello_downgrade_check to false security.osclientcerts.autoload to true security.tls.version.max to 3
[/code]