SSL Connection Error with Wildcard Certificates and Underscores

[michaelfletchercgy]

The DotNetCore TLS certificate validation will return an error when accessing a host with an underscore in its name and a wildcard certificate.

For example https://preprod_curve.curvehero.net/ fails with an SSL connection using the .NET Core WebClient. It works nearly everywhere else including ...

• Firefox
• Chrome
• Safari
• Curl
• OpenSSL

This code would repro the problem

var wc = new System.Net.WebClient();
wc.DownloadFile("https://preprod_curve.curvehero.net/", @"foo.txt");

I tested this on Linux and recently reproduced the issues with .net core 5 preview 3. I can work around the issue by setting DOTNET_SYSTEM_NET_HTTP_USESOCKETSHTTPHANDLER="0".

The error would be

 ---> System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
 ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.
   at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
   at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)

Tagging subscribers to this area: @dotnet/ncl
Notify danmosemsft if you want to be subscribed.

[wfurt]

I did quick test with HttpClient on Ubuntu18 and I get no ssl error.

result=StatusCode: 503, ReasonPhrase: 'Service Unavailable: Back-end server is at capacity', Version: 1.1, Content: System.Net.Http.HttpConnectionResponseContent, Headers:
{
  Connection: keep-alive
  Content-Length: 0
}
Connection: keep-alive

cc: @bartonjs

[vcsjones]

I can reproduce it on Ubuntu 20.04. Let me see if I can figure out what is going on specifically.

Underscores have a bit of a weird history in HTTPS, though wildcards should be supported I believe.

[vcsjones]

If I write this unit test in `HostnameMatchTests.Unix.cs:

[Theory]
[InlineData("micro_soft.example.com", false, true)]
public static void MatchSubjectAltName_Wildcard_Underscores(string targetName, bool mixedCase, bool expectedResult)
{
    string[] sanEntries =
    {
        "example.com",
        "*.example.com"
    };

    RunTest(targetName, "SAN Certificate", sanEntries, !mixedCase, expectedResult);
}

If will fail for me, so the issue lies somewhere in CryptoNative_CheckX509Hostname.

[vcsjones]

It appears X509_check_host in OpenSSL simply does not permit wildcards to match labels that contain underscores. However, we shim that for older OpenSSLs, so it seems to be working for older openssl versions that fall back to the shim.

You can reproduce this with the openssl CLI:

I grabbed the leaf certificate from preprod_curve.curvehero.net and saved it to my desktop as cert.cer

~ openssl version 
OpenSSL 1.1.1f  31 Mar 2020
 ~ openssl x509 -in ~/Desktop/cert.cer -checkhost preprod_curve.curvehero.net -noout
Hostname preprod_curve.curvehero.net does NOT match certificate
 ~ openssl x509 -in ~/Desktop/cert.cer -checkhost blahblahblah.curvehero.net -noout
Hostname blahblahblah.curvehero.net does match certificate
 ~ openssl x509 -in ~/Desktop/cert.cer -checkhost blahblah_blah.curvehero.net -noout
Hostname blahblah_blah.curvehero.net does NOT match certificate

It doesn't match the host with the underscore, but it does match the blahblahblah.

It seems to be this code, here: https://github.com/openssl/openssl/blob/278260bfa238aefef5a1abe2043d2f812c3a4bd5/crypto/x509/v3_utl.c#L697-L702

More interestingly, using openssl s_client -connect preprod_curve.curvehero.net:443 does NOT reproduce the issue, so it may not be using X509_check_host.

While RFC 5280 does not permit underscores in SANs, it doesn't seem like there is anything prohibiting a wildcard from matching a DNS label that matches a host with an underscore. Ballot SC12 from the CAB forum alludes to this:

This ballot creates a brief sunset period intended to allow Subscribers who are relying on FQDNs containing underscores to transition away from them, either by changing the name or deploying a wildcard certificate.

[vcsjones]

Looking further in to this, it looks like it came up in the OpenSSL mailing lists a while ago:

When we want to perform a host verification using openssl's APIs that use
X509_check_host, host URL that includes specific characters such as '_' or
'~' will be failing when CN from the certificate contains wildcard
character.

The ultimate conclusion is won't fix.

@bartonjs it appears we either need to accept that OpenSSL's X509_check_host doesn't accept underscores, or, provide an alternative solution.

[karelz]

Thanks @vcsjones for your analysis!
Why does it work on older versions of OpenSSL? Where is it being shimmed to? Into CoreCLR?

[bartonjs]

We explicitly abandoned our version of hostname matching in favor of using the one provided by OpenSSL (largely to avoid problems like this 😄).

Since the CA/Browser forum members can't issue certificates that use underscores in DNS identifiers, I don't see a reason for us to revert back to our previous implementation of wildcard matching.

[vcsjones]

@bartonjs

Since the CA/Browser forum members can't issue certificates that use underscores in DNS identifiers

But wildcard certificates seem like they should match patterns that contain underscores, which is the case here. The CAB says underscores in dnsName is not permitted, but the ballot seems to indicate that wildcards that match such as host are expected to work.

If your certificate is for *.example.com and your host is foo_bar.example.com, the certificate will be rejected, whereas the older shim would not reject this.

[bartonjs]

My point is that the CA/Browser forum acknowledges that it's non-standard. The decision to support it in wildcards, or not, is therefore up to OpenSSL. If the ballot was "permanently allow this, it's totally normal" then we'd be justified in working around the platform.

[vcsjones]

@bartonjs Understood, thanks.

@karelz

Why does it work on older versions of OpenSSL? Where is it being shimmed to? Into CoreCLR?

When openssl functions that we need aren't available for whatever reason, likely version requirements, they get shimmed in apibridge.c.