Versions client:
windows 11 home
viscosity 1.11
opensc 0.25.0
Asking google, the problem seems to be, that current versions of openvpn and openssl use TLS 1.3 and this requires RSA-PSS, which might not be supported by the NitroKey.
On Linux I got the same error, but was able to connect with openvpn client, when downgrading the connection to TLS1.2 in the openssl.cnf:
Setting tls-version-max and tls-version-min should be sufficient to force the TLS version used by OpenVPN, however it won’t necessarily cause OpenSSL to use ciphers/hashes in that TLS suite version it considers insecure.
TLS 1.2 also optionally supports RSA PSS if the client TLS session announces support for it (which will be happening with Viscosity 1.11 as it’s part of OpenSSL 3’s behaviour) and the server supports it (which yours should as it is also using OpenSSL 3). TLS 1.3 requires support for PSS.
Your Linux OpenSSL config file is essentially doing two things: forcing the use of TLS 1.2 (so making RSA PSS optional), and then disabling the use of RSA PSS by not including it in the list of allowed signature algorithms (e.g. “RSA-PSS+SHA256”/“rsa_pss_pss_sha256”).
There are a number of things you can try:
Adjust the OpenSSL configuration file on the server to add the commands in your Linux OpenSSL configuration file above (i.e. force TLS 1.2 and specify a signature algorithms list without the PSS items). This has the added benefit that connecting clients shouldn’t need to make any changes. A quick internet search seems to indicate this is possible on OpnSense.
Use TLS 1.1 instead. You may need to enable this on the server using tls-version-min. PSS isn’t supported by TLS 1.1, so it’ll definitely prevent PSS signing. Obviously there are security considerations to using older TLS versions.
Try lowering the OpenSSL security level to allow algorithms OpenSSL 3 considers insecure. Please note the security implications of doing so. The command for this is "tls-cipher “DEFAULT:@SECLEVEL=0”
It’s unusual for a PKCS#11 token to not support PSS signing considering the age of TLS 1.2 and TLS 1.3. I would recommend seeing if the manufacturer of your token has a newer version available (either firmware or hardware) that has PSS support, or consider some of the alternatives available (Yubikeys seem quite popular among Viscosity’s userbase).
Thank you very much for your detailed explanations and your suggestions.
I ordered a Yubikey for a test and it works immediately without any problems. So in my opinion it is the best solution to switch to this token, since it works without any hacks on the opnsense and uses the state-of-the-art security features.
Regards
Sebastian
SparkLabs Newsletter
Thank you for being interested in keeping up with the latest news from us! Please double-check your email address below and then click the Subscribe button.
