Verify error num 21 unable to verify the first certificate

I am having trouble with openssl complaining about not being able to validate locally Issued Cert where I have the CA chain as well. I have the locally issued cert in (PEM and CRT) in addition to LOCAL CA Chain Cert (CER,PEM,CRT). The root and issuer are the same server. Comparing text on both certs, both match on «issuer:» field on both. This is redhat Linux server.
I am getting error «verify error:num=20:unable to get local issuer certificate» and «Verify return code: 21 (unable to verify the first certificate)»

I am not sure what to look for anymore., any feedback is appreciated.

Thanks

Jon

Troubleshooting steps

• I did add the Issuer CA cert to linux certs using cerutil certutil -d /etc/pki/nssdb -A -t «C,,» -n DomainA1-Server1CA -i /root/DomainA1-Server1CA.cer

-Ran certutil -d /etc/pki/nssdb -L and I can see the cert there

   Certificate Nickname   Trust Attributes 
                          SSL,S/MIME,JAR/XPI 
   DomainA1-Server1CA     C,,

• Ran openssl s_client -connect ServerA2:443 -CAfile /root/certs/DomainA1-Server1CA.cer,tried with (.CRT and .PEM), Got 2 errors above.
  -Ran openssl s_client -connect ServerA2:443 -CApath /root/certs, tried with (.CRT and .PEM), Got 2 errors above.

• Ran openssl s_client -connect ServerA2:443 . Got 2 errors above.

Issued Cert snippet

    Data:
    Version: 3 (0x2)
    Serial Number:
    54:a9:50:a3:00:01:00:00:14:47
    Signature Algorithm: sha1WithRSAEncryption
    Issuer: DC=com, DC=domainA1, CN=DomainA1-Server1CA
    Validity
    Not Before: April  5 16:45:48 2017 GMT
    Not After : April  5 16:45:48 2019 GMT
    Subject: C=US, ST=NY, L=CityA, O=CompanyNAME, OU=IT,
    CN=ServerB1.DomainA1.com 

CA Chain Cert

    Data: Version: 3 (0x2) 
    Serial Number: 19:11:eb:af:4c:d5:a9:94:49:ka:2f:41:f2:e1:09:g2 
    Signature Algorithm: sha256WithRSAEncryption 
    Issuer: DC=com, DC=domainA1, CN=DomainA1-Server1CA 
    Validity 
    Not Before: Aug 15 18:41:45 2015 GMT 
    Not After : Aug 15 18:41:45 2025 GMT Subject: DC=com, DC=domainA1,                 
    CN=DomainA1-Server1CA Subject Public Key Info: 
    Public Key Algorithm: rsaEncryption Public-Key: (2048 bit)

        ...C.A
    X509v3 Key Usage:
        Digital Signature, Certificate Sign, CRL Sign
    X509v3 Basic Constraints: critical
        CA:TRUE
    X509v3 Subject Key Identifier:

Hipchat Support

Knowledge base

Products

Jira Software

Project and issue tracking

Jira Service Management

Service management and customer support

Jira Work Management

Manage any business project

Confluence

Bitbucket

Git code management

Resources

Documentation

Usage and admin help

Answers, support, and inspiration

Suggestions and bugs

Feature suggestions and bug reports

Marketplace

Billing and licensing

Frequently asked questions

Viewport

Confluence

Hipchat Server SSL Installation and Troubleshooting Guide

Related content

Still need help?

The Atlassian Community is here for you.

Platform notice: Server and Data Center only. This article only applies to Atlassian products on the server and data center platforms .

This guide is intended for use with Hipchat Server and explains SSL common configuration issues and problems. Most SSL certificate issues for Hipchat Server are caused by:

• Root CA certificate included in the certificate chain (which is self signed most of the time).
• Wrong or missing intermediate certificates in the certificate chain.
• Self-signed certificate(s) located in certificate chain.
• Badly formed .PEM files.

This page

To check your installed SSL certificate for errors, run this OpenSSL command from the Hipchat Server console:

This command checks the SSL certificate and chain(s) installed on Hipchat Server, much like a client would request SSL information to start a secure communication.

It details the public certificate chain and any errors that may cause problems. You can run this command either on the server’s terminal / SSH console itself or externally:

• Internal: Use the command provided above
• Externally: Change the $(hostname) section to the hostname and domain of the server that you are running the check on. For example, hipchat.example.com

When using the comprehensive OpenSSL check, a correctly installed SSL certificate looks like this for example:

In this example, there are two certificates listed in the certificate chain. Usually, certificate 0 is the primary certificate and can be easily identified by the CN which should list the fully qualified domain name (FQDN). Certificate 1 is the intermediate certificate.

If a certificate is installed that has the same subject and issuer line (also called a self-signed certificate), then, more than likely, that is a root certificate and needs to be removed from the chain (because the root certificates are normally housed in the client trust-store).

Common OpenSSL Error Codes

verify error:num=20:unable to get local issuer certificate

Error — Unsupported Certificate Purpose

When running an OpenSSL check against a CSR or a CRT file, you may see this error:

Troubleshooting

Run the following command inside Hipchat Server command line interface:

or run a check against your .crt file:

Cause

This error specifically states that the intended purpose of the certificate installed does not include the role to act as a Server certificate thus cannot be used for handshaking when a client attempts to connect.

Error — Unable to verify the first certificate

When running OpenSSL check, an installed SSL cert from trusted CA vendor shows the following error:

Troubleshooting

1. Verify that you have an intermediate certificate installed.
2. If you have an intermediate certificate installed, it’s possible that you may need more than one to establish trust. This is common with vendors such as Entrust. Check with your vendor regarding how many intermediate certificates you need.
3. You can use the GeoCerts SSL Checker tool to visually see your chain and check if any intermediate certs are missing.
4. If you’ve recently installed a chain with an intermediate certificate, first snapshot or back up your instance. (This will disconnect any user that’s currently connected to the service.) Then restart the services ( HipChat service -restart ) or rebooting the Server ( sudo reboot ).

Cause

The primary certificate (the one you purchase from a vendor) cannot establish the chain of trust between itself and the Root certificate stored locally. Usually installing the intermediate certificates fixes this.

Error — Unable to get local issuer certificate

When running OpenSSL check, an installed SSL cert from trusted CA vendor shows:

Troubleshooting

1. Verify that you are using the -CApath argument when running the OpenSSL check.
2. Verify that the path to the local certificate storage is correct (Hipchat Server local certificate trust-store is /etc/ssl ).

Cause

This error code is thrown when a local certificate path is not found during the OpenSSL check is used. This error is specific to the OpenSSL check. Generally, it can be ignored and doesn’t affect the trust of a certificate chain.

Error — Self-signed certificate in certificate chain

When running OpenSSL check, an installed SSL cert from trusted CA vendor shows:

Troubleshooting

1. Install a trusted CA signed primary SSL certificate and an intermediate certificate.
2. Verify that the Root CA is not included in the certificate chain (as this is almost always self-signed).
3. If you’ve removed the Root CA cert, first snapshot or back up your instance. (This will disconnect any user that’s currently connected to the service.) Then restart the services ( HipChat service -restart ) or rebooting the Server ( sudo reboot ).

Cause

This error is thrown whenever there is a self-signed certificate in the chain. When Hipchat Server is booted for the first time, it generates a self-signed certificate for testing purposes. We highly recommend replacing this certificate with a certificate chain from a trusted vendor.

Error — The certificate and key do not match

When uploading your SSL private key and certificate through the Hipchat Server administrative interface or command-line interface, the following error occurs:

Troubleshooting

Use the following commands to generate an MD5 sum for the private key, certificate signing request (CSR), and certificate files:

Verify that the output string from each command is identical to verify that the files are matching. If the outputs are identical but problem persists, try to re-install the SSL certificate you get from CA as cleanly as possible. Start with reconfiguring the server to use self-signed certificate and then re-import the trusted SSL certificate into the server again:

If problem persists even after cleanly importing the certificate, there could be a problem with the certificate that the CA generated. Reach out to the CA to re-issue the SSL certificate followed by a re-installation of the new certificate on the server. On a side note, the wrong order of your SSL certificate could also produced similar error. Double check the SSL certificate format following the recommended format here: Creating or Obtaining an SSL Key and Certificate

Cause

The private key and certificate being uploaded do not match and are incompatible with each other. It will be necessary to re-form the private key, CSR, and/or certificate in such a way that they do match, at which point they can be uploaded to the Hipchat Server.

Error — self signed certificate

Consider purchasing a trusted CA signed SSL certificate to tackle this issue permanently.

SSL Certificate Chain Ideal Setup

As listed in the Creating or Obtaining an SSL Key and Certificate , when installing the SSL certificate chain in Hipchat Server, please make sure that your .pem file includes the RSA private key, the primary certificate and an intermediate certificate. This gives the certificate chain the best chance of minimizing problems with trust in Hipchat Server.

At this time, Hipchat Server is not compatible with ECC (Elliptic Curve Cryptography) or ECDH (Elliptic Curve Diffie–Hellman) generated SSL certificates.

Trust Issues

The chain of trust extends from the root certificate (also called the anchor certificate) to the intermediate certificate then to the primary certificate. If there are any errors or issues with trust throughout the chain, the following issues may happen:

• Unable to install integrations
• SSL warnings in the Hipchat client when it connects
• Mobile apps may not connect with the Require Certificate Authentication option checked
• Push notifications will not work
• Emoticons may not render
• Desktop client failed to reconnect

Hipchat Server generates a self-signed certificate and private key when it is first deployed. This certificate is meant for testing purposes and is not recommended that you use it in a production environment.

To ensure that the functionalities of Hipchat Server are not affected in any way, we always recommend you to obtain a trusted CA signed full certificate chain applied on your instance.

Internally Signed SSL Certificates

If your group is using an SSL certificate signed by an internal CA (opposed to a vendor like DigiCert), the internal CA will need to provide the corresponding RSA private key, primary, and intermediate certs to install in Hipchat Server. Once the SSL chain is installed in Hipchat Server, install the root certificate into the trust-store of the OS running the Hipchat client. This adds the certificate as ‘trusted’ which is needed when connecting to the Hipchat Server certificate chain. If the root certificate is not installed in the OS running the Hipchat client, then the trust will not be established and you may have problems connecting the client.

Источник

Five Essential OpenSSL Troubleshooting Commands

Troubleshooting SSL certificates and connections? Here are five handy openssl commands that every network engineer should be able to use. Bookmark this – you never know when it will come in handy!

1. Check the Connection

This command opens an SSL connection to the specified site and displays the entire certificate chain as well. Here’s an abridged version of the sample output:

I’ve reformatted the lines to make things a little clearer, and be grateful that I removed the blocks of Base64-encoded text that represent the certificates or it would be even more painful to read. There are a couple of things to note, however.

I Only Want to See the Server Certificate

Fine then; remove the -showcerts argument, and your wish will be fulfilled.

error:num=20:unable to get local issuer certificate

There’s an error here. Depth 2 means which certificate in the chain; in this case the third one as they are numbered 0, 1 and 2, and this error means that openssl was unable to find a certificate for the issuer of certificate 2 whose Common Name(CN) is “VeriSign Class 3 Public Primary Certification Authority – G5”. That’s because the issuer is a root certificate and openssl does not know where the root certificates are. This can be fixed by adding the -CAfile option pointing to a file containing all the trusted root certificates, but where to get those? That’s coming soon in another post. For now what we need to know is that we have three certificates in a chain and at least up to certificate 2, things are verifying correctly.

Certificate Subject and Issuer

Each certificate is presented as a Subject and an Issuer. The Subject is the thing the certificate is supposed to represent, and the Issuer is the issuing Certificate Authority. For example here’s certificate 0 (the server certificate) from this chain:

You can see the certificate number (zero) then s: meaning Subject: and i: meaning Issuer. It follows then that the Issuer of certificate 0 should be the Subject of certificate 1, as we want to verify if the Issuer is valid; and so it is:

This of course continues up the chain.

error:num=21:unable to verify the first certificate

If you see this when you run this command, it means exactly what it says … that chain of trust is broken right from the start. Typically it might happen if you fail to include intermediate certificates, or if you supply the wrong intermediate certificate.

This Opens a Connection

Really. It might look like the openssl command has hung, but actually it did exactly what we asked it to and opened a connection. It’s waiting for you to send something now. To quit, either Ctrl-C, or hit Enter a couple of times or – if you’re testing for a response – try typing some basic HTTP commands, e.g.:

You’ll still have to break out after that using Ctrl-C, but meanwhile, what fun! Don’t forget that for most sites (particularly HTTP but usually HTTPS as well) you have to use the Host: directive so that the web server knows which site you were trying to contact. When you think about it, most hosting companies have tens or hundreds of web sites served by a single server and IP. Supplying a Host: is essential.

2. Decoding a Base64 Certificate (e.g. PEM)

The output from the previous command will display the raw certificate data between the “—–BEGIN CERTIFICATE—–” and “—–END CERTIFICATE—–” tags. I removed it from the output above so that I could hit you with one now as an example:

Certificates can be in a variety of formats (yay for standardization), but the output from OpenSSL (like above) will be Base64 encoded and basically unreadable. Thankfully, the openssl command can help you view those in a format that is human readable and formatted nicely. Take the Base64 text (including the BEGIN and END lines) of the certificate you are interested in, and save it to a file. Then run this command (in my case with a file called cert-microsoft.pem ):

This tells openssl to read the file cert-microsoft.pem , display it in a textual format, and not to create any kind of output certificate. The result is exactly what you asked for:

And, since we have the other certificates in the chain, I could check them too.

Does This Work With Binary Certificate Files?

DER is a binary certificate format, but the content is basically the same underneath it all. When discussing the AIA field in a previous post, I casually skipped over the fact that this file in my experience seems to be supplied in DER format rather than PEM (I don’t know if this is a standard, just that it’s what I’ve seen. Maybe it’s to keep the transfer shorter and thus faster?). All openssl asks is that you tell if you want to supply it with a DER instead of a PEM (Base64) certificate. Personally I would have thought that the absence of “—–BEGIN CERTIFICATE” was sufficient clue for openssl to make an educated guess, but apparently that’s not the case. Instead, you have to use the command line option -inform der . For example, to view a binary certificate as text you’d do this:

By the way, -inform is short for “input format”; you’re not really “informing” openssl about anything. If you were wondering, yes, there is an -outform command as well, and on that note:

3. Convert Certificate From DER to PEM Format

In the examples above, we asked openssl not to create an output certificate using the -nout command line argument. However, openssl is very helpful at converting certificates between formats, so let’s try converting DER to PEM:

This command specifies that the input format is DER, the input file is cert_symantec.der and that we want an output format of PEM saved to an output file called cert_symantec.pem . The observant will have noted that the command actually did not specify the output format of PEM. PEM is the default input and output format, so it does not need to be specified. However, if you like to remove ambiguity in a totally harmless and logical fashion, the full command would be:

Easy peasy. The added benefit of understanding how to do this is that you now don’t have to use somebody else’s website to convert you internal certificates between formats.

4. Checking Your Own Chain of Trust

You’re ready to deploy a certificate for a website, and you have been given a ZIP file containing the public server cert and a file purporting to contain the necessary intermediate certificate(s). How can you check that you have the correct certificates without actually installing them? Why, openssl, of course!

What happened here? Error 20 was mentioned above; it means that the intermediate certificate (or at least, the certificate for the Issuer of the server certificate) is missing. Well of course it is; we didn’t supply it! The www.microsoft.com site uses a certificate from Symantec, so let’s use that and tell openssl about it:

That’s progress; we still have an error 20, but now it has moved to “1 depth” – that is, the error is no longer on the server certificate (0 depth) but now we can’t find the issuer certificate for the Symantec cert. In a previous post, we discovered that the Symantec cert was issued by a Verisign entity that is in our trusted root store. So now I’ll add a link to the root store as well to complete the chain:

That’s it! We have confirmed that we have a full chain of trust from a trusted root cert all the way down to the www.microsoft.com server certificate. Even for a Mac user, this is a good thing.

If you have more than a single Intermediate Certificate between the server and a trusted root certificate, you need to make them all available to the client. That’s easily done by creating a certificate bundle, which is a fancy way of saying “add all the certificates together in a single file.” Really. If you have two files each containing an intemediate certificate and need to bundle them, in *nix / OS X you do this:

It’s that easy. In any GUI environment you can just paste them one after another in Notepad and save them out. Remember to include the BEGIN and END lines. Now in your command line just change the argument to -untrusted intermediatebundle.pem and you’re good.

5. Testing for SSLv3 Using OpenSSL

This one is pretty easy. Using the s_client function again, we can ask openssl to try to connect using SSLv3. A site that supports SSLv3 (naughty naughty) will look like this:

A site that does NOT support SSLv3 will look like this:

There are other error codes you may see, but generally speaking if you get dumped back to a prompt with an error, chances are this means that SSLv3 is not allowed.

I thought it was interesting, by the way, that microsoft.com allowed an SSLv3 connection, but www.microsoft.com did not. The former uses a different certificate chain and redirects to the latter, so perhaps it all comes out in the wash. It’s actually a missed opportunity in some ways for Microsoft not to detect SSLv3 in some way, then pop up a web page saying “Hello IE6 user – why not upgrade now?”, but they don’t do that.

My 10 Bits

I have had to use all of these commands recently in order to troubleshoot a few different issues that were popping up with some web sites. I confess to being terrible at remembering commands in detail, so I’m going to bookmark my own page for reference even if you don’t! Openssl does plenty more that can be useful, but this is a great start when it comes to certificates and ciphers.

Источник

I did this https://www.digitalocean.com/community/tutorials/how-to-secure-apache-with-let-s-encrypt-on-ubuntu-14-04

then switch from apache to nginx with the following ***https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-14-04

***all was good. but now facebook debugger giving me: Curl Error : SSL_CACERT SSL certificate problem: unable to get local issuer certificate

***QUESTION: How do I redownload all certs from your certificate authority and make a fresh bundle in my situation?

Alexs-MacBook-Air:~ alex$ openssl s_client -connect goeasysmile.com:443
CONNECTED(00000003)
depth=0 /CN=goeasysmile.com
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 /CN=goeasysmile.com
verify error:num=27:certificate not trusted
verify return:1
depth=0 /CN=goeasysmile.com
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/CN=goeasysmile.com
   i:/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFFjCCA/6gAwIBAgISBPstpF9NiACN8vtDkMYUsZ/fMA0GCSqGSIb3DQEBCwUA
MEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQD
ExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xNzA5MjExMTIwMDBaFw0x
NzEyMjAxMTIwMDBaMBoxGDAWBgNVBAMTD2dvZWFzeXNtaWxlLmNvbTCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBAKxogeMt8i4wigwQX8idtb4NsBsjY8Aa
huTiGBi+99WVFUU8v6yAIQew+Q2Csd0cxF3Iq6I4pajEzqb/tTYXXVGLL7isdwRG
pRbhcrkF4Urxk3BhdP6f/7QCMkp4/H3knvKBa+cugZXQ3lz/73uVQ6F+bC7ZEp2U
mSkMGS1NT5bPiPML8KOLhOpIT0rj03e3T2PA8oy+TheI8pZ4E+LVnS4qzbas6PN/
ijgfG/ev/C62zmlwz+Dfe8UbiZ0sAmOQ/Q7/a2iaxZBiX+ZTMHGtv4cEd+2p8knn
ZQ+ZM/pmzYyU8o+NlykN2/joY7FlRMDEBJdnstf42OSQGtFWI1CCjv0CAwEAAaOC
AiQwggIgMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYB
BQUHAwIwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQUFJk9G1tyeWd/7eHnbYuKRLMn
1Z8wHwYDVR0jBBgwFoAUqEpqYwR93brm0Tm3pkVl7/Oo7KEwbwYIKwYBBQUHAQEE
YzBhMC4GCCsGAQUFBzABhiJodHRwOi8vb2NzcC5pbnQteDMubGV0c2VuY3J5cHQu
b3JnMC8GCCsGAQUFBzAChiNodHRwOi8vY2VydC5pbnQteDMubGV0c2VuY3J5cHQu
b3JnLzAvBgNVHREEKDAmgg9nb2Vhc3lzbWlsZS5jb22CE3d3dy5nb2Vhc3lzbWls
ZS5jb20wgf4GA1UdIASB9jCB8zAIBgZngQwBAgEwgeYGCysGAQQBgt8TAQEBMIHW
MCYGCCsGAQUFBwIBFhpodHRwOi8vY3BzLmxldHNlbmNyeXB0Lm9yZzCBqwYIKwYB
BQUHAgIwgZ4MgZtUaGlzIENlcnRpZmljYXRlIG1heSBvbmx5IGJlIHJlbGllZCB1
cG9uIGJ5IFJlbHlpbmcgUGFydGllcyBhbmQgb25seSBpbiBhY2NvcmRhbmNlIHdp
dGggdGhlIENlcnRpZmljYXRlIFBvbGljeSBmb3VuZCBhdCBodHRwczovL2xldHNl
bmNyeXB0Lm9yZy9yZXBvc2l0b3J5LzANBgkqhkiG9w0BAQsFAAOCAQEAgHbVMCYk
ySK1ivEO2myMfCkwi0GBTqj9/oKW6T7DNJSpcLYYPZgaQai98QgHBT6P3dsiCSI4
FComGitu8M7foip2eHX6JWPBtCAN1QocsgY3UbppGWKY99qIDi3u5DJLheDzsDPo
633B2J1cm5f3quDTpZPRcW7tzx45VQ/YqT7Ydr3kxriAXUf3pedCXHk3SIZ/92qe
tJE0MFe7zlwnSDAb5uNohVAeQSVymQG/afSifNGYOWclcDOrLatEJn+JlJ4oPbbA
y+en2IeIH5Ez63SJDgzqMHvCSAtmCVUWsI2seGOUMzJikeVAx13jE8JCYdmuvzTN
sRb6/GJYbfWcBA==
-----END CERTIFICATE-----
subject=/CN=goeasysmile.com
issuer=/C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
---
No client certificate CA names sent
---
SSL handshake has read 2261 bytes and written 456 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : DHE-RSA-AES256-SHA
    Session-ID: E8704CF999E67354784246C113DCB93BAB0E0C0BF47942FC44B25B95B8655EB4
    Session-ID-ctx: 
    Master-Key: 4E520458361D6EFF58193ECC63A17DAAEC16146D0834D852E7A5284CD114BF02FA9ED939DF97A58B07AB9176A0A72352
    Key-Arg   : None
    Start Time: 1506319952
    Timeout   : 300 (sec)
    Verify return code: 21 (unable to verify the first certificate)
---
closed

Might have something to do with my buggy /var/log/le-renew.log contab

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/www.goeasysmile.com/fullchain.pem (failure)
Upgrading certbot-auto 0.14.0 to 0.14.1...
Replacing certbot-auto...
Creating virtual environment...
Installing Python packages...
Had a problem while installing Python packages.
pip prints the following errors:
=====================================================
Collecting argparse==1.4.0 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line $
/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/u$
  SNIMissingWarning
/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/u$
  InsecurePlatformWarning
  Downloading argparse-1.4.0-py2.py3-none-any.whl
Collecting pycparser==2.14 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line $
  Downloading pycparser-2.14.tar.gz (223kB)
Collecting cffi==1.4.2 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line 21))
  Downloading cffi-1.4.2.tar.gz (365kB)
Collecting ConfigArgParse==0.10.0 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt$
  Downloading ConfigArgParse-0.10.0.tar.gz
Collecting configobj==5.0.6 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line$
  Downloading configobj-5.0.6.tar.gz
Collecting cryptography==1.5.3 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (l$
  Downloading cryptography-1.5.3.tar.gz (400kB)
Collecting enum34==1.1.2 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line 65$
  Downloading enum34-1.1.2.tar.gz (46kB)
Collecting funcsigs==0.4 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line 68$
  Downloading funcsigs-0.4-py2.py3-none-any.whl
Collecting idna==2.0 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line 71))
  Downloading idna-2.0-py2.py3-none-any.whl (61kB)
Collecting ipaddress==1.0.16 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (lin$
  Downloading ipaddress-1.0.16-py27-none-any.whl
Collecting linecache2==1.0.0 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (lin$
  Downloading linecache2-1.0.0-py2.py3-none-any.whl
Collecting ordereddict==1.1 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line$
  Downloading ordereddict-1.1.tar.gz
Collecting parsedatetime==2.1 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (li$
  Downloading parsedatetime-2.1-py2-none-any.whl
Collecting pbr==1.8.1 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line 85))
  Downloading pbr-1.8.1-py2.py3-none-any.whl (89kB)
Collecting pyasn1==0.1.9 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line 88$
  Downloading pyasn1-0.1.9-py2.py3-none-any.whl
Collecting pyOpenSSL==16.2.0 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (lin$
  Downloading pyOpenSSL-16.2.0-py2.py3-none-any.whl (43kB)
Collecting pyparsing==2.1.8 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line$
  Downloading pyparsing-2.1.8-py2.py3-none-any.whl (54kB)
Collecting pyRFC3339==1.0 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line 1$
  Downloading pyRFC3339-1.0-py2.py3-none-any.whl
Collecting python-augeas==0.5.0 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt ($
  Downloading python-augeas-0.5.0.tar.gz (90kB)
Collecting pytz==2015.7 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line 117$
  Downloading pytz-2015.7-py2.py3-none-any.whl (476kB)
Collecting requests==2.12.1 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line$
  Downloading requests-2.12.1-py2.py3-none-any.whl (574kB)
Collecting six==1.10.0 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line 134))
  Downloading six-1.10.0-py2.py3-none-any.whl
Collecting traceback2==1.4.0 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (lin$
  Downloading traceback2-1.4.0-py2.py3-none-any.whl
Collecting unittest2==1.1.0 (from -r /tmp/tmp.AnzE95LVFd/letsencrypt-auto-requirements.txt (line$

...

    running build_ext
    generating cffi module 'build/temp.linux-x86_64-2.7/_padding.c'
    creating build/temp.linux-x86_64-2.7
    generating cffi module 'build/temp.linux-x86_64-2.7/_constant_time.c'
    generating cffi module 'build/temp.linux-x86_64-2.7/_openssl.c'
    building '_openssl' extension
    creating build/temp.linux-x86_64-2.7/build
    creating build/temp.linux-x86_64-2.7/build/temp.linux-x86_64-2.7
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-pr$
    x86_64-linux-gnu-gcc: internal compiler error: Killed (program cc1)
    Please submit a full bug report,
    with preprocessed source if appropriate.
    See <file:///usr/share/doc/gcc-4.8/README.Bugs> for instructions.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 4

    ----------------------------------------
Command "/root/.local/share/letsencrypt/bin/python2.7 -u -c "import setuptools, tokenize;__file_$
', '
'), __file__, 'exec'))" install --record /tmp/pip-SvFhes-record/install-record.txt --single-vers$
/root/.local/share/letsencrypt/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/u$
  InsecurePlatformWarning
You are using pip version 8.0.3, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
=====================================================

Certbot has problem setting up the virtual environment.

Based on your pip output, the problem can likely be fixed by
increasing the available memory.

Consult https://certbot.eff.org/docs/install.html#problems-with-python-virtual-environment
for possible solutions.
You may also find some support resources at https://certbot.eff.org/support/ .

Troubleshooting SSL certificates and connections? Here are five handy openssl commands that every network engineer should be able to use. Bookmark this – you never know when it will come in handy!

1. Check the Connection

 openssl s_client -showcerts -connect www.microsoft.com:443

This command opens an SSL connection to the specified site and displays the entire certificate chain as well. Here’s an abridged version of the sample output:

MBP$ openssl s_client -showcerts -connect www.microsoft.com:443
CONNECTED(00000003)
depth=2 /C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006
 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3 Public
 Primary Certification Authority - G5
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=
     Washington/businessCategory=Private Organization/
     serialNumber=600413485/C=US/postalCode=98052/ST=Washington/
     L=Redmond/street=1 Microsoft Way/O=Microsoft Corporation/
     OU=MSCOM/CN=www.microsoft.com
   i:/C=US/O=Symantec Corporation/OU=Symantec Trust Network/
     CN=Symantec Class 3 EV SSL CA - G3
-----BEGIN CERTIFICATE-----
[...]
-----END CERTIFICATE-----
 1 s:/C=US/O=Symantec Corporation/OU=Symantec Trust Network/
     CN=Symantec Class 3 EV SSL CA - G3
   i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006
     VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3
     Public Primary Certification Authority - G5
-----BEGIN CERTIFICATE-----
[...]
-----END CERTIFICATE-----
 2 s:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006
     VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3
     Public Primary Certification Authority - G5
   i:/C=US/O=VeriSign, Inc./OU=Class 3 Public Primary Certification
      Authority
-----BEGIN CERTIFICATE-----
[...]
-----END CERTIFICATE-----
---
Server certificate
subject=/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=
Washington/businessCategory=Private Organization/
serialNumber=600413485/C=US/postalCode=98052/ST=Washington/
L=Redmond/street=1 Microsoft Way/O=Microsoft Corporation/OU=MSCOM
/CN=www.microsoft.com
issuer=/C=US/O=Symantec Corporation/OU=Symantec Trust Network/
CN=Symantec Class 3 EV SSL CA - G3
---
No client certificate CA names sent
---
SSL handshake has read 4170 bytes and written 456 bytes
---
New, TLSv1/SSLv3, Cipher is AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
    Protocol  : TLSv1
    Cipher    : AES256-SHA
    Session-ID: BF66575AD2B1E71E74796085FBB60BE1A472818DF0
      0BC27378B26AEF9E924F52
    Session-ID-ctx:
    Master-Key: 88A9E8281A5B66DC2E21D4BAA6B85CB0C6F64522FFDFE1AD
      1DAC8BF3186CDBD6824EDD669C9B4CD5EDCA42AC6361FDB4
    Key-Arg   : None
    Start Time: 1425836408
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---

I’ve reformatted the lines to make things a little clearer, and be grateful that I removed the blocks of Base64-encoded text that represent the certificates or it would be even more painful to read. There are a couple of things to note, however.

I Only Want to See the Server Certificate

Fine then; remove the -showcerts argument, and your wish will be fulfilled.

error:num=20:unable to get local issuer certificate

depth=2 /C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c)
2006 VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3
Public Primary Certification Authority - G5
verify error:num=20:unable to get local issuer certificate

There’s an error here. Depth 2 means which certificate in the chain; in this case the third one as they are numbered 0, 1 and 2, and this error means that openssl was unable to find a certificate for the issuer of certificate 2 whose Common Name(CN) is “VeriSign Class 3 Public Primary Certification Authority – G5”. That’s because the issuer is a root certificate and openssl does not know where the root certificates are. This can be fixed by adding the -CAfile option pointing to a file containing all the trusted root certificates, but where to get those? That’s coming soon in another post. For now what we need to know is that we have three certificates in a chain and at least up to certificate 2, things are verifying correctly.

Certificate Subject and Issuer

Each certificate is presented as a Subject and an Issuer. The Subject is the thing the certificate is supposed to represent, and the Issuer is the issuing Certificate Authority. For example here’s certificate 0 (the server certificate) from this chain:

 0 s:/1.3.6.1.4.1.311.60.2.1.3=US/1.3.6.1.4.1.311.60.2.1.2=
     Washington/businessCategory=Private Organization/serialNumber=
     600413485/C=US/postalCode=98052/ST=Washington/L=Redmond/
     street=1 Microsoft Way/O=Microsoft Corporation/OU=MSCOM
     /CN=www.microsoft.com
   i:/C=US/O=Symantec Corporation/OU=Symantec Trust Network
     /CN=Symantec Class 3 EV SSL CA - G3

You can see the certificate number (zero) then s: meaning Subject: and i: meaning Issuer. It follows then that the Issuer of certificate 0 should be the Subject of certificate 1, as we want to verify if the Issuer is valid; and so it is:

 1 s:/C=US/O=Symantec Corporation/OU=Symantec Trust Network
     /CN=Symantec Class 3 EV SSL CA - G3
   i:/C=US/O=VeriSign, Inc./OU=VeriSign Trust Network/OU=(c) 2006
     VeriSign, Inc. - For authorized use only/CN=VeriSign Class 3
     Public Primary Certification Authority - G5

This of course continues up the chain.

error:num=21:unable to verify the first certificate

If you see this when you run this command, it means exactly what it says … that chain of trust is broken right from the start. Typically it might happen if you fail to include intermediate certificates, or if you supply the wrong intermediate certificate.

This Opens a Connection

Really. It might look like the openssl command has hung, but actually it did exactly what we asked it to and opened a connection. It’s waiting for you to send something now. To quit, either Ctrl-C, or hit Enter a couple of times or – if you’re testing for a response – try typing some basic HTTP commands, e.g.:

    [...]
    Start Time: 1425837372
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
GET / HTTP/1.1                        <<< Typed in
Host: www.microsoft.com               <<< Typed in
                                      <<< Typed in
HTTP/1.1 200 OK
[...delivers response...]

You’ll still have to break out after that using Ctrl-C, but meanwhile, what fun! Don’t forget that for most sites (particularly HTTP but usually HTTPS as well) you have to use the Host: directive so that the web server knows which site you were trying to contact. When you think about it, most hosting companies have tens or hundreds of web sites served by a single server and IP. Supplying a Host: is essential.

2. Decoding a Base64 Certificate (e.g. PEM)

The output from the previous command will display the raw certificate data between the “—–BEGIN CERTIFICATE—–” and “—–END CERTIFICATE—–” tags. I removed it from the output above so that I could hit you with one now as an example:

-----BEGIN CERTIFICATE-----
MIIFmjCCBIKgAwIBAgIKNfMBNgABAAB+LzANBgkqhkiG9w0BAQUFADCBgDETMBEG
CgmSJomT8ixkARkWA2NvbTEZMBcGCgmSJomT8ixkARkWCW1pY3Jvc29mdDEUMBIG
CgmSJomT8ixkARkWBGNvcnAxFzAVBgoJkiaJk/IsZAEZFgdyZWRtb25kMR8wHQYD
VQQDExZNU0lUIE1hY2hpbmUgQXV0aCBDQSAyMB4XDTEzMDYyMDIwMjkyOFoXDTE1
MDYyMDIwMjkyOFowGDEWMBQGA1UEAxMNbWljcm9zb2Z0LmNvbTCCASIwDQYJKoZI
hvcNAQEBBQADggEPADCCAQoCggEBANV/NeoVpoco0OnLeGxUEIoXKRNj6T/r8QGa
NvKRVWKR/msN8mPeWstdzKu3c5e44HnSGw74F+pDilvNxURIAVT15Plfs717+2M7
6eCWL0dvg+epNoDxx6ncMZ0U5+yPvv8rSyPldIBq4KACgSLZF4EvOBUmn/JGUwzw
wHc9MI9lbvBoYoMdOm3ugIgSQJojxi5HMu0VjKbRfmnxlWuDJKcxsBc5qrWG322v
mloroq94NAodqxA0mrB2Ktozm8tGvlm3C3nR9F7x53892dl2KbhiiQmtIxsvN/iK
hLJfuUA+wdfE5MbdB2XF/FhACA/ELO02u225eOJ67WKN1jTs22ECAwEAAaOCAnsw
ggJ3MB0GA1UdDgQWBBSJKGUqqRL75TOfAHBohT5I3Wq4yTALBgNVHQ8EBAMCBLAw
HwYDVR0jBBgwFoAU69sRXvgJntjWYpz9Yp3jhEoo4Scwge4GA1UdHwSB5jCB4zCB
4KCB3aCB2oZPaHR0cDovL21zY3JsLm1pY3Jvc29mdC5jb20vcGtpL21zY29ycC9j
cmwvTVNJVCUyME1hY2hpbmUlMjBBdXRoJTIwQ0ElMjAyKDEpLmNybIZNaHR0cDov
L2NybC5taWNyb3NvZnQuY29tL3BraS9tc2NvcnAvY3JsL01TSVQlMjBNYWNoaW5l
JTIwQXV0aCUyMENBJTIwMigxKS5jcmyGOGh0dHA6Ly9jb3JwcGtpL2NybC9NU0lU
JTIwTWFjaGluZSUyMEF1dGglMjBDQSUyMDIoMSkuY3JsMIGtBggrBgEFBQcBAQSB
oDCBnTBVBggrBgEFBQcwAoZJaHR0cDovL3d3dy5taWNyb3NvZnQuY29tL3BraS9t
c2NvcnAvTVNJVCUyME1hY2hpbmUlMjBBdXRoJTIwQ0ElMjAyKDEpLmNydDBEBggr
BgEFBQcwAoY4aHR0cDovL2NvcnBwa2kvYWlhL01TSVQlMjBNYWNoaW5lJTIwQXV0
aCUyMENBJTIwMigxKS5jcnQwPwYJKwYBBAGCNxUHBDIwMAYoKwYBBAGCNxUIg8+J
Ta3yAoWhnwyC+sp9geH7dIFPg8LthQiOqdKFYwIBZAIBCjAdBgNVHSUEFjAUBggr
BgEFBQcDAgYIKwYBBQUHAwEwJwYJKwYBBAGCNxUKBBowGDAKBggrBgEFBQcDAjAK
BggrBgEFBQcDATANBgkqhkiG9w0BAQUFAAOCAQEAtv7pra/09qguqOGjZvsSKHrH
UX/BORRqq8nFGYaE3qMe9NY3l6LVc3+ei8K+sR8ni2XtsuvSMq+7k6fawXITKBd+
ptfoAMw4w7QrmkUFBPb4e+CcN0TOmnqj/eczuw/zo8WmD6OHM3QyLot3iR16fFAb
TG6oYc1w3lQZzc94Fkun0FA4gEEvi9u1XzNbjqO2tU9FxJjuCni6CwPw7maOShWY
DObGVa3ZebPBq+R60eUfNicgEGgX4O2VstrOpPYYIV5ZiJDqljNh9MI2NKvOp1dl
0fN7gzHLr++OCrh4xTBzYHHCepxRcBB8MIVA8f16l/ShhEbUNjHG0lVPFOqzfg==
-----END CERTIFICATE-----

Certificates can be in a variety of formats (yay for standardization), but the output from OpenSSL (like above) will be Base64 encoded and basically unreadable. Thankfully, the openssl command can help you view those in a format that is human readable and formatted nicely. Take the Base64 text (including the BEGIN and END lines) of the certificate you are interested in, and save it to a file. Then run this command (in my case with a file called cert-microsoft.pem):

openssl x509 -noout -text -in cert-microsoft.pem

This tells openssl to read the file cert-microsoft.pem , display it in a textual format, and not to create any kind of output certificate. The result is exactly what you asked for:

MBP$ openssl x509 -noout -text -in cert-microsoft.pem
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            35:f3:01:36:00:01:00:00:7e:2f
        Signature Algorithm: sha1WithRSAEncryption
        Issuer: DC=com, DC=microsoft, DC=corp, DC=redmond, 
          CN=MSIT Machine Auth CA 2
        Validity
            Not Before: Jun 20 20:29:28 2013 GMT
            Not After : Jun 20 20:29:28 2015 GMT
        Subject: CN=microsoft.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
            RSA Public Key: (2048 bit)
                Modulus (2048 bit):
                    [removed for brevity]
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                89:28:65:2A:A9:12:FB:E5:33:9F:00:70:68:85:3E:
                48:DD:6A:B8:C9
            X509v3 Key Usage:
                Digital Signature, Key Encipherment, Data
                Encipherment
            X509v3 Authority Key Identifier:
                keyid:EB:DB:11:5E:F8:09:9E:D8:D6:62:9C:FD:
                62:9D:E3:84:4A:28:E1:27

            X509v3 CRL Distribution Points:
                URI:http://mscrl.microsoft.com/pki/mscorp/crl/
                  MSIT%20Machine%20Auth%20CA%202(1).crl
                URI:http://crl.microsoft.com/pki/mscorp/crl/
                  MSIT%20Machine%20Auth%20CA%202(1).crl
                URI:http://corppki/crl/MSIT%20Machine%20Auth%20
                  CA%202(1).crl

            Authority Information Access:
                CA Issuers - URI:http://www.microsoft.com/pki/
                  mscorp/MSIT%20Machine%20Auth%20CA%202(1).crt
                CA Issuers - URI:http://corppki/aia/MSIT%20Machine
                  %20Auth%20CA%202(1).crt

            1.3.6.1.4.1.311.21.7:
                00.(+.....7.....M..........}...t.O.........c..d..

            X509v3 Extended Key Usage:
                TLS Web Client Authentication, TLS Web Server
                Authentication
            1.3.6.1.4.1.311.21.10:
                0.0
..+.......0
..+.......
    Signature Algorithm: sha1WithRSAEncryption
                    [removed for brevity]

And, since we have the other certificates in the chain, I could check them too.

Does This Work With Binary Certificate Files?

DER is a binary certificate format, but the content is basically the same underneath it all. When discussing the AIA field in a previous post, I casually skipped over the fact that this file in my experience seems to be supplied in DER format rather than PEM (I don’t know if this is a standard, just that it’s what I’ve seen. Maybe it’s to keep the transfer shorter and thus faster?). All openssl asks is that you tell if you want to supply it with a DER instead of a PEM (Base64) certificate. Personally I would have thought that the absence of “—–BEGIN CERTIFICATE” was sufficient clue for openssl to make an educated guess, but apparently that’s not the case. Instead, you have to use the command line option -inform der. For example, to view a binary certificate as text you’d do this:

openssl x509 -noout -text -inform der -in cert_symantec.der

By the way, -inform is short for “input format”; you’re not really “informing” openssl about anything. If you were wondering, yes, there is an -outform command as well, and on that note:

3. Convert Certificate From DER to PEM Format

In the examples above, we asked openssl not to create an output certificate using the -nout command line argument. However, openssl is very helpful at converting certificates between formats, so let’s try converting DER to PEM:

openssl x509 -inform der -in cert_symantec.der -out cert_symantec.pem

This command specifies that the input format is DER, the input file is cert_symantec.der and that we want an output format of PEM saved to an output file called cert_symantec.pem. The observant will have noted that the command actually did not specify the output format of PEM. PEM is the default input and output format, so it does not need to be specified. However, if you like to remove ambiguity in a totally harmless and logical fashion, the full command would be:

openssl x509 -inform der -in cert_symantec.der -outform pem -out cert_symantec.pem

Easy peasy. The added benefit of understanding how to do this is that you now don’t have to use somebody else’s website to convert you internal certificates between formats.

4. Checking Your Own Chain of Trust

You’re ready to deploy a certificate for a website, and you have been given a ZIP file containing the public server cert and a file purporting to contain the necessary intermediate certificate(s). How can you check that you have the correct certificates without actually installing them? Why, openssl, of course!

MBP$ openssl verify -verbose cert-www-microsoft.pem
cert-www-microsoft.pem: /1.3.6.1.4.1.311.60.2.1.3=US/
  1.3.6.1.4.1.311.60.2.1.2=Washington/businessCategory=Private
  Organization/serialNumber=600413485/C=US/postalCode=98052/
  ST=Washington/L=Redmond/street=1 Microsoft Way/O=Microsoft
  Corporation/OU=MSCOM/CN=www.microsoft.com
error 20 at 0 depth lookup:unable to get local issuer certificate

What happened here? Error 20 was mentioned above; it means that the intermediate certificate (or at least, the certificate for the Issuer of the server certificate) is missing. Well of course it is; we didn’t supply it! The www.microsoft.com site uses a certificate from Symantec, so let’s use that and tell openssl about it:

MBP$ openssl verify -untrusted cert-symantec cert-www-microsoft.pem
cert-www-microsoft.pem: /C=US/O=Symantec Corporation/OU=Symantec Trust Network/CN=Symantec Class 3 EV SSL CA - G3
error 20 at 1 depth lookup:unable to get local issuer certificate
OK

That’s progress; we still have an error 20, but now it has moved to “1 depth” – that is, the error is no longer on the server certificate (0 depth) but now we can’t find the issuer certificate for the Symantec cert. In a previous post, we discovered that the Symantec cert was issued by a Verisign entity that is in our trusted root store. So now I’ll add a link to the root store as well to complete the chain:

MBP$ openssl verify -untrusted cert-symantec 
       -CAfile ./RootCerts.pem cert-www-microsoft.pem
cert-www-microsoft.pem: OK

That’s it! We have confirmed that we have a full chain of trust from a trusted root cert all the way down to the www.microsoft.com server certificate. Even for a Mac user, this is a good thing.

If you have more than a single Intermediate Certificate between the server and a trusted root certificate, you need to make them all available to the client. That’s easily done by creating a certificate bundle, which is a fancy way of saying “add all the certificates together in a single file.” Really. If you have two files each containing an intemediate certificate and need to bundle them, in *nix / OS X you do this:

$ cat intermediate1.pem intermediate2.pem > intermediatebundle.pem

It’s that easy. In any GUI environment you can just paste them one after another in Notepad and save them out. Remember to include the BEGIN and END lines. Now in your command line just change the argument to -untrusted intermediatebundle.pem and you’re good.

This one is pretty easy. Using the s_client function again, we can ask openssl to try to connect using SSLv3. A site that supports SSLv3 (naughty naughty) will look like this:

MBP$ openssl s_client -ssl3 -connect microsoft.com:443
CONNECTED(00000003)
[...certificate stuff removed for brevity...]
SSL-Session:
    Protocol  : SSLv3
    Cipher    : RC4-SHA
    Session-ID: 33410000536...
    Session-ID-ctx:
    Master-Key: F88FCD7DF64CFB48...
    Key-Arg   : None
    Start Time: 1425840399
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
---

A site that does NOT support SSLv3 will look like this:

MBP$ openssl s_client -showcerts -ssl3 -connect www.microsoft.com:443
CONNECTED(00000003)
5688:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert
  handshake failure:/SourceCache/OpenSSL098/OpenSSL098-52.10.1/
  src/ssl/s3_pkt.c:1143:SSL alert number 40
5688:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake
  failure:/SourceCache/OpenSSL098/OpenSSL098-52.10.1/src/ssl/
  s3_pkt.c:564:
MBP$

There are other error codes you may see, but generally speaking if you get dumped back to a prompt with an error, chances are this means that SSLv3 is not allowed.

I thought it was interesting, by the way, that microsoft.com allowed an SSLv3 connection, but www.microsoft.com did not. The former uses a different certificate chain and redirects to the latter, so perhaps it all comes out in the wash. It’s actually a missed opportunity in some ways for Microsoft not to detect SSLv3 in some way, then pop up a web page saying “Hello IE6 user – why not upgrade now?”, but they don’t do that.

My 10 Bits

I have had to use all of these commands recently in order to troubleshoot a few different issues that were popping up with some web sites. I confess to being terrible at remembering commands in detail, so I’m going to bookmark my own page for reference even if you don’t! Openssl does plenty more that can be useful, but this is a great start when it comes to certificates and ciphers.

I’ve created a self-signed certificate for foo.localhost using a Let’s Encrypt recommendation using this Makefile:

include ../.env

configuration = csr.cnf
certificate = self-signed.crt
key = self-signed.key

.PHONY: all
all: $(certificate)

$(certificate): $(configuration)
    openssl req -x509 -out $@ -keyout $(key) -newkey rsa:2048 -nodes -sha256 -subj '/CN=$(HOSTNAME)' -extensions EXT -config $(configuration)

$(configuration):
    printf "[dn]nCN=$(HOSTNAME)n[req]ndistinguished_name = dnn[EXT]nsubjectAltName=DNS:$(HOSTNAME)nkeyUsage=digitalSignaturenextendedKeyUsage=serverAuth" > $@

.PHONY: clean
clean:
    $(RM) $(configuration)

I’ve then assigned that to a web server. I’ve verified that the server returns the relevant certificate:

$ openssl s_client -showcerts -connect foo.localhost:8443 < /dev/null
CONNECTED(00000003)
depth=0 CN = foo.localhost
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = foo.localhost
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/CN=foo.localhost
   i:/CN=foo.localhost
-----BEGIN CERTIFICATE-----
[…]
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=foo.localhost
issuer=/CN=foo.localhost
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 1330 bytes and written 269 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES128-GCM-SHA256
    Session-ID: […]
    Session-ID-ctx: 
    Master-Key: […]
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket:
    […]

    Start Time: 1529622990
    Timeout   : 7200 (sec)
    Verify return code: 21 (unable to verify the first certificate)
    Extended master secret: no
---
DONE

How do I make cURL trust it without modifying anything in /etc? --cacert does not work, presumably because there is no CA:

$ curl --cacert tls/foo.localhost.crt 'https://foo.localhost:8443/'
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

The goal is to enable HTTPS during development:

• I can’t have a completely production-like certificate without a lot of work to enable DNS verification in all development environments. Therefore I have to use a self-signed certificate.
• I still obviously want to make my development environment as similar as possible to production, so I can’t simply ignore any and all certificate issues. curl -k is like catch (Exception e) {} in this case — nothing at all like a browser talking to a web server.

In other words, when running curl [something] https://project.local/api/foo I want to be confident that

1. if TLS is configured properly except for having a self-signed certificate the command will succeed and
2. if I have any issues with my TLS configuration except for having a self-signed certificate the command will fail.

Using HTTP or --insecure fails the second criterion.