Category: TLS

Rather that having your step ca issue certificates that only reflect what is in the CSR, you can use certificate templates in order to dynamically add content to the x509 certificates being issued.

CA Configuration

In your /usr/local/etc/step/ca/config/ca.json configuration file you need to add some options to your provisioner. Let’s assume you have an existing ACME provisioner that looks as follows:

                        {
                                "type": "ACME",
                                "name": "24h",
                                "claims": {
                                        "maxTLSCertDuration": "24h0m0s",
                                        "defaultTLSCertDuration": "24h0m0s"
                                }
                        },

After the claims section you need to insert an options block. As usual, don’t forget the comma after the curly braces that end the claims section.

                        {
                                "type": "ACME",
                                "name": "24h",
                                "claims": {
                                        "maxTLSCertDuration": "24h0m0s",
                                        "defaultTLSCertDuration": "24h0m0s"
                                },
                                "options": {
                                        "x509": {
                                                "templateFile": "/usr/local/etc/step/ca/templates/certs/x509/acme.tpl",
                                                "templateData": {
                                                        "TDCountry": "DE",
                                                        "TDStateOrProvince": "Saxony",
                                                        "TDLocality": "Dresden",
                                                        "TDStreetAddress": "Musterstrasse 1, 01234 Dresden, Germany",
                                                        "TDOrganization": "Tinkivity",
                                                        "TDOrganizationalUnit": "web server team"
                                                }
                                        }
                                }
                        },

Line 10 points to a template file that we will look at in just a few seconds. Line 11 introduces a data section that we use to inject some dynamic data. The data items (line 12 until line 17) make more sense when looking at the actual acme.tpl template file as referenced in line 11.

Template file

Below shows our acme.tpl template file. I will not explain the complete template but just some of the most important aspects.

{
    "subject": {
    {{- if .Insecure.CR.Subject.CommonName }}
        "commonName": "{{ .Insecure.CR.Subject.CommonName }}",
    {{- else }}
        "commonName": "{{ (index .SANs 0).Value }}",
    {{- end }}
        "country": "{{ .TDCountry }}",
        "province": "{{ .TDStateOrProvince }}",
        "locality": "{{ .TDLocality }}",
        "streetAddress": "{{ .TDStreetAddress }}",
        "organization": "{{ .TDOrganization }}",
        "organizationalUnit": "{{ .TDOrganizationalUnit }}"
    },
    "sans": {{ toJson .SANs }},
{{- if typeIs "*rsa.PublicKey" .Insecure.CR.PublicKey }}
    "keyUsage": ["keyEncipherment", "digitalSignature"],
{{- else }}
    "keyUsage": ["digitalSignature"],
{{- end }}
    "extKeyUsage": ["serverAuth", "clientAuth"]
}

Lines 3 until 7 check if the CSR contains a common name in the subject. If the common name exists it will be applied, otherwise the first subject alternative name is used for the common name in the certificate subject. The reason for that logic is that certbot in my case seems to sometimes omit the common name ;-(

Lines 8 until 13 contain a reference to the injected template data and set the according subject fields for the certificate that is being issued.

Certificate issued with applied template

Let’s check how a certificate obtained via certbot looks like after the template is applied.

andreas@testserver ➜  ~ sudo openssl x509 -noout -text -in /usr/local/etc/letsencrypt/live/testserver/fullchain.pem
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            9f:95:31:ed:1f:0b:b8:99:39:e1:64:02:73:89:d1:db
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: C = DE, ST = Saxony, O = Tinkivity, OU = Tinkivity Intermediate Certificate Authority, CN = Smallstep Intermediate CA, emailAddress = xxx@xxx.com
        Validity
            Not Before: Dec  7 18:18:07 2020 GMT
            Not After : Dec  8 18:19:07 2020 GMT
        Subject: C = DE, ST = Saxony, L = Dresden, street = "Musterstrasse 1, 01234 Dresden, Germany", O = Tinkivity, OU = web server team, CN = testserver
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:c4:74:01:28:bb:26:20:2f:a1:6b:30:44:9e:9b:
                    ...
                    << REDACTED >>
                    ...
                    04:57
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Subject Key Identifier: 
                81:7C:BB:71:7F:02:52:06:71:CF:E2:87:3B:CA:8A:09:6C:81:65:46
            X509v3 Authority Key Identifier: 
                keyid:87:32:28:49:63:29:06:79:96:13:DE:47:14:9F:EF:C0:DD:EC:4D:C3

            X509v3 Subject Alternative Name: 
                DNS:testserver, DNS:testserver.local
            1.3.6.1.4.1.37476.9000.64.1: 
                0
.....24h..
    Signature Algorithm: sha256WithRSAEncryption
         b7:74:16:a4:a1:5a:fb:df:a0:ea:42:5c:cd:70:fc:16:2d:8b:
         ...
         << REDACTED >>
         ...
         c9:9e:db:00:40:56:61:a1

If you want to entirely restrict access to a web server to only those folks that you deem authorized, using client certificates is the way to go. The common expression for such pattern is mutual TLS, or mTLS for short.

Outline

There are 3 components you need for this recipe:

1. a web server that supports mTLS
2. a certificate authority that issues a client certificate
3. a client that will submit such client certificate to the web server as part of a request

In scope of this blog post

The following steps are being explained as part of this blog post:

• generate a certificate request (CSR) with openssl
• issue a client certificate with step ca
• configure NGINX to require mTLS
• issue an HTML request with curl

Not in scope of this blog post

There is a lot of background knowledge required to fully comprehend how mTLS works in detail. The following topics are not being addressed in this blog post and assumed to be understood to a minimum extent at least.

• Core concepts of a Public Key Infrastructure (PKI)
• x509 Certificates
• OpenSSL
• step ca
• Import of x509 client certificates into the operating system you use

Generate a CSR with OpenSSL

For a Certificate Authority (CA) to issue a certificate to a client, a Certificate Signing Request (CSR) from the client is needed if such client wants to keep its private key as a secret to itself. An easy way to generate such is to use OpenSSL with a configuration file. Below configuration file (assumed name: john-csr.cnf) shows the bare minimum for a CSR that can be used by a CA to issue a client certificate. The only 2 net information contained in the CSR are the user name and the user’s email address.

[req]
prompt             = no
distinguished_name = req_dn
req_extensions     = req_ext

[req_dn]
CN                 = John Doe

[req_ext]
subjectAltName     = @alt_names

[alt_names]
email.1            = john.doe@examplemail.com

While we could have put the email address into the subject (req_dn section), it is important to understand that we deliberately not do this but use the subject alternative name extension to place the email address.

To generate the CSR we use openssl’s req command. We use the existing private key that we have generated beforehand and stored in the key.pem file.

andreas@laptop ➜  ~ openssl req -new -config john-csr.cnf -key key.pem -out johndoe.csr

When visualizing the generated CSR we can see the 2 net information reflected in the CSR.

andreas@laptop ➜  ~ openssl req -noout -text -in johndoe.csr                          
Certificate Request:
    Data:
        Version: 1 (0x0)
        Subject: CN = John Doe
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
                Modulus:
                    00:e4:10:b6:3d:82:fa:ca:4b:b7:61:20:a0:33:ed:
                    ...
                    <<REDACTED>>
                    ...
                    67:ed
                Exponent: 65537 (0x10001)
        Attributes:
        Requested Extensions:
            X509v3 Subject Alternative Name: 
                email:john.doe@examplemail.com
    Signature Algorithm: sha256WithRSAEncryption
         0a:13:8b:4a:16:ee:c4:4f:23:86:f6:d8:b2:d3:7c:d6:70:d1:
         ...
         <<REDACTED>>
         ...
         3f:61:4d:cf

Issue a certificate with step ca

We will copy the CSR onto our step ca server (maybe under the incoming folder) and issue a step ca sign command. If we do have multiple provisioners, the sign command will prompt a list of all provisioners and ask us to interactively select a provisioner. After selecting the provisioner, we need to input the passphrase that protects the private key of the issuing intermediate CA.

andreas@acme ➜  ~ step ca sign --ca-url https://acme.local:8443 --root /etc/ssl/tinkivity.pem incoming/johndoe.csr issued/johndoe.pem
✔ Provisioner: 1year (JWK) [kid: SlOHMD00B8-WIyUqa1zQxP9xwG4UQCvOorMU02xThUc]
✔ Please enter the password to decrypt the provisioner key: 
✔ CA: https://acme.local:8443
✔ Certificate: issued/johndoe.pem

As an alternative we pre-select the provisioner already along with sign command.

andreas@acme ➜  ~ step ca sign --ca-url https://acme.local:8443 --root /etc/ssl/tinkivity.pem --provisioner 1year incoming/johndoe.csr issued/johndoe.pem
✔ Provisioner: 1year (JWK) [kid: SlOHMD00B8-WIyUqa1zQxP9xwG4UQCvOorMU02xThUc]
✔ Please enter the password to decrypt the provisioner key: 
✔ CA: https://acme.local:8443
✔ Certificate: issued/johndoe.pem

Another alternative is to generate a token upfront as it allows us to pass in the password file and thus make the command completely interaction free (not shown in this blog, but can be see in this previous blog post).

Either way, we can inspect the generated certificate with OpenSSL.

andreas@acme ➜  ~ openssl x509 -noout -text -in issued/johndoe.pem 
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            3f:ed:1f:01:b1:ce:90:66:0f:33:b7:31:fa:ce:b9:8a
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: C=DE, ST=Saxony, O=Tinkivity, OU=Tinkivity Intermediate Certificate Authority, CN=Smallstep Intermediate CA/emailAddress=xxx@xxx.com
        Validity
            Not Before: Dec  6 12:58:16 2020 GMT
            Not After : Dec  6 12:59:16 2021 GMT
        Subject: CN=John Doe
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                Public-Key: (2048 bit)
                Modulus:
                    00:e4:10:b6:3d:82:fa:ca:4b:b7:61:20:a0:33:ed:
                    ...
                    <<REDACTED>>
                    ...
                    67:ed
                Exponent: 65537 (0x10001)
        X509v3 extensions:
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage: 
                TLS Web Server Authentication, TLS Web Client Authentication
            X509v3 Subject Key Identifier: 
                7D:A9:C5:44:49:EC:CC:54:37:64:46:CF:A9:99:85:D6:23:18:9D:F8
            X509v3 Authority Key Identifier: 
                keyid:87:32:28:49:63:29:06:79:96:13:DE:47:14:9F:EF:C0:DD:EC:4D:C3

            X509v3 Subject Alternative Name: 
                email:john.doe@examplemail.com
            1.3.6.1.4.1.37476.9000.64.1: 
                07.....1year.+SlOHMD00B8-WIyUqa1zQxP9xwG4UQCvOorMU02xThUc
    Signature Algorithm: sha256WithRSAEncryption
    Signature Algorithm: sha256WithRSAEncryption
         76:46:dc:d0:c7:81:ab:f3:c0:3c:0f:5c:99:d1:12:ca:97:a1:
         ...
         <<REDACTED>>
         ...
         a7:e7:56:13:79:3d:3c:b0

Setup NGINX for mTLS

Assuming we re-use the NGINX setup from this previous blog post, we only have to add a few lines to the site configuration at /usr/local/etc/nginx/sites/testclient.conf from our NGINX server.

server {
#       listen       80;
        server_name  testclient;

        access_log /var/log/nginx/testclient.access.log;
        error_log /var/log/nginx/testclient.error.log;

        # location of our own root certificate
        ssl_client_certificate  /etc/ssl/certs/97efb5b5.0;
        ssl_verify_client       optional;

        location / {
            root   /usr/local/www/sites/testclient/html;
            index  index.html;

            if ($ssl_client_verify != SUCCESS) {
                    return 403;
            }
        }

        listen 443 ssl;
        ssl_certificate /usr/local/etc/letsencrypt/live/testclient/fullchain.pem;
        ssl_certificate_key /usr/local/etc/letsencrypt/live/testclient/privkey.pem;
        include /usr/local/etc/letsencrypt/options-ssl-nginx.conf;
        ssl_dhparam /usr/local/etc/letsencrypt/ssl-dhparams.pem;
}

server {
        if ($host = testclient) {
            return 301 https://$host$request_uri;
        }

        listen       80;
        server_name  testclient;
        return 404;
}

Now we only have NGINX reload the configuration.

andreas@testclient ➜  ~ sudo service nginx reload
Performing sanity check on nginx configuration:
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful

Issue a request with client authentication

Let’s start by trying to issue a regular request from our laptop. We use a simple curl command to callout to https://testclient. We tell curl where to find our root certificate via the –cacert option, so it will not complain about an unknown root CA.

According to the NGINX configuration, we should immediately receive a 403 error from the web server if we lack client authentication…

andreas@laptop ➜  ~ curl --cacert /etc/ssl/certs/97efb5b5.0 https://testclient
<html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.18.0</center>
</body>
</html>

Now, let’s apply our user certificate and the private key we’ve used to generate the CSR.

andreas@laptop ➜  ~ curl --cacert /etc/ssl/certs/97efb5b5.0 --cert johndoe.pem --key key.pem https://testclient
<html>
    <head>
        <title>TESTCLIENT</title>
    </head>
    <body>
        <h1>Hello World!</h1>
    </body>
</html>

Based on the step ca setup as described in Running your own ACME Server, we can add another provisioner that allows us to manually sign CSRs from web servers that do not support Certbot. One example for such use case would be system solutions like TrueNAS or Proxmox, which by now have ACME support but do not support easy customization or override of the ACME server URL. In fact, many system solutions with ACME support assume that only Let’s Encrypt is used when obtaining certificates via ACME protocol.

In this blog post we look at how to add further provisioners for smallstep’s step ca.

Adding a simple provisioner via command line

More or less only one command is needed to add a provisioner. We need to pass in the name of the provisioner (4weeks), the location of the ca config file, the location of the password file and the create command.

andreas@acme ➜  ~ sudo step ca provisioner add 4weeks --ca-config /usr/local/etc/step/ca/config/ca.json --password-file /usr/local/etc/step/password.txt --create

Looking at the /usr/local/etc/step/ca/config/ca.json configuration file we can find the following new block next to our existing ACME provisioner.

                        {
                                "type": "JWK",
                                "name": "4weeks",
                                "key": {
                                        "use": "sig",
                                        "kty": "EC",
                                        "kid": "WsxEssolEVj1TpF-nfXpSuY2jL8pLQgpCtgVj5Qq3Ls",
                                        "crv": "P-256",
                                        "alg": "ES256",
                                        "x": "5b9f1pk6VVM5CCIHUOpbw6SV8lC-rAxEQtiScRZUopE",
                                        "y": "hxRrUPm7M6S7HBm9LZV5JUbBLP7l2aG4CKr1vY20csw"
                                },
                                "encryptedKey": "eyJh... <<REDACTED>> ...no1w"
                        }

We called our provisioner 4weeks for a reason – we want certificates be valid for 4 weeks (672 hours). That said, we need to add a claims section to the provisioner that clarifies the validity. When adding the claims section, do not forget to comply with JSON and make sure to append a comma to the last line before the new section.

                        {
                                "type": "JWK",
                                "name": "4weeks",
                                "key": {
                                        "use": "sig",
                                        "kty": "EC",
                                        "kid": "WsxEssolEVj1TpF-nfXpSuY2jL8pLQgpCtgVj5Qq3Ls",
                                        "crv": "P-256",
                                        "alg": "ES256",
                                        "x": "5b9f1pk6VVM5CCIHUOpbw6SV8lC-rAxEQtiScRZUopE",
                                        "y": "hxRrUPm7M6S7HBm9LZV5JUbBLP7l2aG4CKr1vY20csw"
                                },
                                "encryptedKey": "eyJh... <<REDACTED>> ...no1w",
                                "claims": {
                                        "minTLSCertDuration": "24h0m0s",
                                        "maxTLSCertDuration": "672h0m0s",
                                        "defaultTLSCertDuration": "672h0m0s",
                                        "disableRenewal": false
                                }
                        }

To make the change effective, the service needs to be restarted.

andreas@acme ➜  ~ sudo service step-ca restart                  
Stopping step_ca.
Starting step_ca.
step_ca is running as pid 96773.

Import a CSR

While this blog post will not cover how to create a CSR, we start by copying a CSR onto our step ca server. For the remainder of this blog post we assume request.csr to be the name of that CSR.

Issue a Certificate

The step ca command line tools available allow us to issue a certificate with a token. The first step is to create such token, which we export into an environment variable for later use.

andreas@acme ➜  ~ export TOKEN=`step ca token 'newserver.local' --ca-url https://acme.local:8443 --root /etc/ssl/tinkivity.pem`
✔ Provisioner: 4weeks (JWK) [kid: WsxEssolEVj1TpF-nfXpSuY2jL8pLQgpCtgVj5Qq3Ls]
✔ Please enter the password to decrypt the provisioner key: 

The command is interactive and will first ask us to select one provisioner from the list of all available provisioners. After we select our provisioner (4weeks), we are being asked for the password to decrypt the provisioner key.

As an alternative to the interactive password input, we could add the –pasword-file directive to the command. That way we don’t have to input our password, but we would need to run the command as sudo in order to get read access to the password file.

andreas@acme ➜  ~ export TOKEN=`step ca token 'newserver.local' --ca-url https://acme.local:8443 --root /etc/ssl/tinkivity.pem --password-file /usr/local/etc/step/password.txt`

Either way the command should complete without error and our token should be available.

andreas@acme ➜  ~ echo $TOKEN                                                                                                           
eyJhbGciOiJFUzI1NiIsImtpZCI6IldzeEVzc29sRVZqMVRwRi1uZlhwU3VZMmpMOHBMUWdwQ3RnVmo1UXEzTHMiLCJ0eXAiOiJKV1QifQ.eyJhdWQiOiJodHRwczovL2FjbWUudGlua2l2aXR5LmhvbWU6ODQ0My8xLjAvc2lnbiIsImV4cCI6MTYwNzAxNTA1NywiaWF0IjoxNjA3MDE0NzU3LCJpc3MiOiI0d2Vla3MiLCJqdGkiOiI3ZDRlZjViZWI3YWQ2NzA1ZTkzNDY3NmMwMjhjMmY3OWFjMmRmNzMxODIwZDg3NDc0NWJmYzMyNzIwMmIyOTNjIiwibmJmIjoxNjA3MDE0NzU3LCJzYW5zIjpbIkFuZHJlYXMgU3RyYXVjaCJdLCJzaGEiOiIwMzUxMWI1YzRjZmNlZWJlNjI5YzJjMjQ2YTIzMjMwYjhhYzQxNDQyOTI0MjliOGMzN2ZhM2FjMGE3MmUwZmM5Iiwic3ViIjoiQW5kcmVhcyBTdHJhdWNoIn0.gaOwV7nEVq8cOL4uVvp1Y4-c3NUMs0YKMri0N9Q9MQRAWnvCg8BKuntSxThIeywvM0gMO2QND_9iz9VObFRULg

All that’s left to do now is to sign the certificate with our token.

andreas@acme ➜  ~ step ca sign --token $TOKEN incoming/request.csr issued/certificate.csr       
✔ CA: https://acme.local:8443/1.0/sign
✔ Certificate: issued/certificate.csr

That’s it. The certificate is ready to be deployed.

In the last blog post Running your own ACME Server we have successfully installed our own PKI with an ACME provisioner. In this blog post we want to look at the client side and automatically obtain and renew a client certificate for a web server.

NGINX

From an ACME point of view the type of web server doesn’t matter at all. In this example we will use NGINX as a web server, because it is lightweight and popular.

andreas@testclient ➜  ~ sudo pkg install nginx

As we want NGINX to run as a service we will append one line to our /etc/rc.conf and then start the service.

andreas@testclient ➜  ~ sudo sh -c 'echo nginx_enable=\"YES\" >> /etc/rc.conf'
andreas@testclient ➜  ~ sudo service nginx start                              
Performing sanity check on nginx configuration:
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Starting nginx.

Now we apply a pattern that extracts web server configuration and contents into separate file system locations. Each web server (or virtual domain) will get its own content folder. In our case we want to put all content into a testclient folder.

andreas@testclient ➜  ~ sudo mkdir -p /usr/local/www/sites/testclient/html

We will create a simple html file at /usr/local/www/sites/testclient/html/index.html with the following content.

<html>
    <head>
        <title>TESTCLIENT</title>
    </head>
    <body>
        <h1>Hello World!</h1>
    </body>
</html>

The webserver configuration will be put into a .conf file the follows the same name. Although we only need one webserver in our example we will still create a sites subfolder for good housekeeping.

andreas@testclient ➜  ~ sudo mkdir /usr/local/etc/nginx/sites
andreas@testclient ➜  ~ sudo touch /usr/local/etc/nginx/sites/testclient.conf

Our site configuration at /usr/local/etc/nginx/sites/testclient.conf will have the following content.

server {
        listen       80;
        server_name  testclient;

        access_log /var/log/nginx/testclient.access.log;
        error_log /var/log/nginx/testclient.error.log;

        location / {
            root   /usr/local/www/sites/testclient/html;
            index  index.html;
        }
}

At last we clean up /usr/local/etc/nginx/nginx.conf by removing the complete server section as we don’t need it anymore. Instead we will add an include statement before the last { of the http section. That will make sure our /usr/local/etc/nginx/sites/testclient.conf configuration file will be parsed.

Based on a fresh installation the config file would most likely look like the following.

# This default error log path is compiled-in to make sure configuration parsing
# errors are logged somewhere, especially during unattended boot when stderr
# isn't normally logged anywhere. This path will be touched on every nginx
# start regardless of error log location configured here. See
# https://trac.nginx.org/nginx/ticket/147 for more info. 
#
#error_log  /var/log/nginx/error.log;
#

#pid        logs/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;

    include "sites/*.conf";
}

Now we check the configuration and reload the config.

andreas@testclient ➜  ~ sudo nginx -t                                      
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
andreas@testclient ➜  ~ sudo service nginx reload                          
Performing sanity check on nginx configuration:
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful

Adding our own Root Certificate to the trust store

To make things easier we will add our own root certificate into the trust store of our client. we will copy the certificate into the /usr/share/certs/trusted folder and then apply a rehash operation that is limited to only the one certificate we just copied.

andreas@testclient ➜  ~ openssl x509 -hash -noout -in /usr/share/certs/trusted/tinkivity.pem 
97efb5b5
andreas@testclient ➜  ~ sudo ln -s /usr/share/certs/trusted/tinkivity.pem /etc/ssl/certs/97efb5b5.0

We can see if we have been successful if openssl’s s_client command can verify the certificate from our ACME server.

andreas@testclient ➜  ~ openssl s_client -connect acme.local:8443 --quiet      
depth=1 C = DE, ST = Saxony, O = Tinkivity, OU = Tinkivity Intermediate Certificate Authority, CN = Smallstep Intermediate CA, emailAddress = xxx@xxx.com
verify return:1
depth=0 CN = Step Online CA
verify return:1

Certbot

Now, as we have setup a new web server, we can install Certbot and have it obtain a certificate from our ACME server. The first step is installing the packages for Certbot itself and its NGINX plugin.

andreas@testclient ➜  ~ sudo pkg install py37-certbot py37-certbot-nginx

Before we move on to the next step of registration of our domain at the ACME server we need to find out if python can successfully integrate the trust store. We issue a simple python command to check SSL verification.

andreas@testclient ➜  ~ python3.7 -c "import requests; print(requests.get('https://acme.local:8443').text)"
404 page not found

If we receive real HTML content (above 404 page not found is actually HTML and thus success), we are good for ‘regular’ Certbot usage. If we receive a lengthy exception that somewhere contains a line like below, our python installation doesn’t include the trust store correctly and we will need to operate Certbot with the –no-verify-ssl option for further requests.

requests.exceptions.SSLError: HTTPSConnectionPool(host='acme.local', port=8443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))

Above error happens on FreebSD 12.2-RC3 with python3.7 and seems to be a deeper issue, because python claims to look at the correct trust store location:

andreas@testclient ➜  ~ python3.7 -c "import ssl; print(ssl.get_default_verify_paths())"                                                         
DefaultVerifyPaths(cafile='/etc/ssl/cert.pem', capath='/etc/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/etc/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/etc/ssl/certs')

The next step already is the registration of our domain at the ACME server. We use the following command:

andreas@testclient ➜  ~ sudo certbot --nginx --agree-tos --non-interactive --no-verify-ssl --email xxx@xxx.com --server https://acme.local:8443/acme/acme-smallstep/directory --domain testclient
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
Obtaining a new certificate
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
Performing the following challenges:
http-01 challenge for testclient
Using default address 80 for authentication.
nginx: [warn] conflicting server name "testclient" on 0.0.0.0:80, ignored
Waiting for verification...
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
Cleaning up challenges
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
/usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning,
Could not automatically find a matching server block for testclient. Set the `server_name` directive to use the Nginx installer.

IMPORTANT NOTES:
 - Unable to install the certificate
 - Congratulations! Your certificate and chain have been saved at:
   /usr/local/etc/letsencrypt/live/testclient/fullchain.pem
   Your key file has been saved at:
   /usr/local/etc/letsencrypt/live/testclient/privkey.pem
   Your cert will expire on 2020-12-01. To obtain a new or tweaked
   version of this certificate in the future, simply run certbot again
   with the "certonly" option. To non-interactively renew *all* of
   your certificates, run "certbot renew"
 - Your account credentials have been saved in your Certbot
   configuration directory at /usr/local/etc/letsencrypt. You should
   make a secure backup of this folder now. This configuration
   directory will also contain certificates and private keys obtained
   by Certbot so making regular backups of this folder is ideal.

Although the ACME part worked completely fine, we still get an error from Certbot’s NGINX plugin. It turns out that the plugin cannot locate the server_name directive in our NGINX configuration. That is driven by the fact that we have extracted parts of the NGINX configuration into a separate configuration file (/usr/local/etc/nginx/sites/testclient.conf). We have two options now:

1. go back to a single NGINX configuration file
2. manually enter the Certbot configuration snippets into our separate NGINX configuration file

We will go with the latter option and put in the Certbot configuration snippets ourselves. The configuration at /usr/local/etc/nginx/sites/testclient.conf will now look as follows.

server {
#       listen       80;
        listen       443 ssl;
        server_name  testclient;

        access_log /var/log/nginx/testclient.access.log;
        error_log /var/log/nginx/testclient.error.log;

        location / {
            root   /usr/local/www/sites/testclient/html;
            index  index.html;
        }

        ssl_certificate /usr/local/etc/letsencrypt/live/testclient/fullchain.pem;
        ssl_certificate_key /usr/local/etc/letsencrypt/live/testclient/privkey.pem;
        include /usr/local/etc/letsencrypt/options-ssl-nginx.conf;
        ssl_dhparam /usr/local/etc/letsencrypt/ssl-dhparams.pem;
}

server {
        if ($host = testclient) {
            return 301 https://$host$request_uri;
        }

        listen       80;
        server_name  testclient;
        return 404;
}

1. our existing server block has been updated to not listen on port 80 any more, but on port 443 via SSL instead
2. the locations for the certificate obtained from the ACME server, the private key, ssl options (cipher suite etc.) as well as Diffie-Hellman parameters have been included into the configuration
3. a new server block has with the goal of listening on port 80 and redirection to port 443 SSL has been added

In order to apply the configuration changes, we have to reload the NGINX configuration.

andreas@testclient ➜  ~ sudo service nginx reload                                 
Performing sanity check on nginx configuration:
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful

Auto Renew

Last but not least we need to insert a cron task so that Certbot will automatically renew the certificate on a regular schedule.

andreas@testclient ➜  ~ echo "0       0,12    *       *       *       root    python3.7 -c 'import random; import time; time.sleep(random.random() * 3600)' && certbot renew --no-verify-ssl --quiet" | sudo tee -a /etc/crontab > /dev/null

Above entry will run Certbot’s renew command at midnight and high noon. Without further parameters (i.e. domain) above command will renew all certificates managed by Certbot. If you want to see which certificates are being managed by Certbot you can run the following command.

andreas@testclient ➜  ~ sudo certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Cannot extract OCSP URI from /usr/local/etc/letsencrypt/live/testclient/cert.pem

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Found the following certs:
  Certificate Name: testclient
    Serial Number: 8a98c881442ed7de1d460dee5a97fb6
    Domains: testclient
    Expiry Date: 2020-12-02 18:51:33+00:00 (VALID: 23 hour(s))
    Certificate Path: /usr/local/etc/letsencrypt/live/testclient/fullchain.pem
    Private Key Path: /usr/local/etc/letsencrypt/live/testclient/privkey.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

That’s it. If you put your web browser onto http://testclient in your local network, you should see a Hello World page with a valid certificate.

While some of us might have heard from Let’s Encrypt and how it uses ACME for complete automation of certificate management, a few of us might even ask themselves: ‘Can I also run my own private ACME server in my home network?‘. The basic answer is yes, because ACME is a standardized and open protocol. As in many ‘make vs. buy‘ decisions a more detailed look will reveal that writing your own implementation of ACME is a lot of effort and thus not the right approach for a home project. Luckily there is smallstep – a company from in the bay area that provides an open-source certificate authority & PKI toolkit that we can use.

Installing step-certificates

There are two packages you need to install in order to start working: the step-certificates package provides the certificate authority (server) and the step-cli package provides a command line client.

andreas@acme ➜  ~ sudo pkg install step-certificates step-cli

After installation there will be a service script available.

andreas@acme ➜  ~ ls -lah /usr/local/etc/rc.d/step-ca
-rwxr-xr-x  1 root  wheel   2.5K Oct  5 10:56 /usr/local/etc/rc.d/step-ca

Looking into the service script that will reveal a number of interesting findings:

1. the rcvar we need to add to our /etc/rc.conf for service management has a value of step_ca_enable
2. the directory that will contain all configuration (including the password) defaults to /usr/local/etc/step and after fresh installation this directory is completely empty
3. the actual configuration file defining our step ca will be /usr/local/etc/step/config/ca.json
4. the master password will be stored in plain text under /usr/local/etc/step/password.txt
5. the service script implements a start_precmd that will interact with the command line in order to initialize a template config and password upon service start

First Time (Auto) Setup

We will append the step_ca_enable rcvar into our /etc/rc.conf so that we can use the service command to start and stop the step-ca service.

# Enable Step CA
step_ca_enable="YES"

Now, what we need to understand is that the start_precmd section of the service script (see last finding in above list) will simply call the step ca init command and then interactively collect a password for storing it in the password.txt file. Having said that, we will make use of that mechanism and let the command line guide us through creation of our PKI.

andreas@acme ➜  ~ sudo service step-ca start
No configured Step CA found.
Creating new one....
✔ What would you like to name your new PKI? (e.g. Smallstep): acme
✔ What DNS names or IP addresses would you like to add to your new CA? (e.g. ca.smallstep.com[,1.1.1.1,etc.]): acme.local,192.168.1.2
✔ What address will your new CA listen at? (e.g. :443): :8443
✔ What would you like to name the first provisioner for your new CA? (e.g. you@smallstep.com): firstprovisioner
✔ What do you want your password to be? [leave empty and we'll generate one]: 

Generating root certificate... 
all done!

Generating intermediate certificate... 
all done!

✔ Root certificate: /usr/local/etc/step/ca/certs/root_ca.crt
✔ Root private key: /usr/local/etc/step/ca/secrets/root_ca_key
✔ Root fingerprint: 97f4728d915d001e51ceaab3e7343a60807625ca5d5d588c52b739b202fb0164
✔ Intermediate certificate: /usr/local/etc/step/ca/certs/intermediate_ca.crt
✔ Intermediate private key: /usr/local/etc/step/ca/secrets/intermediate_ca_key
✔ Database folder: /usr/local/etc/step/ca/db
✔ Default configuration: /usr/local/etc/step/ca/config/defaults.json
✔ Certificate Authority configuration: /usr/local/etc/step/ca/config/ca.json

Your PKI is ready to go. To generate certificates for individual services see 'step help ca'.

FEEDBACK 😍 🍻
      The step utility is not instrumented for usage statistics. It does not
      phone home. But your feedback is extremely valuable. Any information you
      can provide regarding how you’re using `step` helps. Please send us a
      sentence or two, good or bad: feedback@smallstep.com or join
      https://gitter.im/smallstep/community.
Step CA Password file for auto-start not found
Creating it....
Please enter the Step CA Password:

Starting step_ca.
step_ca is running as pid 58450.

Obviously a template config that is ready to go has been created and the service already has been started. Let’s have a look at the directory structure in place, so we can better understand what has been done here.

andreas@acme ➜  ~ sudo tree /usr/local/etc/step
/usr/local/etc/step
├── ca
│   ├── certs
│   │   ├── intermediate_ca.crt
│   │   └── root_ca.crt
│   ├── config
│   │   ├── ca.json
│   │   └── defaults.json
│   ├── db
│   │   ├── 000000.vlog
│   │   ├── LOCK
│   │   └── MANIFEST
│   ├── secrets
│   │   ├── intermediate_ca_key
│   │   └── root_ca_key
│   └── templates
└── password.txt

6 directories, 10 files

The certs subfolder contains a root certificate as well as an intermediate certificate, which the keys for both are stored in the secrets subfolder. Both keys are encrypted with the same password that we’ve interactively provided at the command line when running our initial service start. That password has been stored as plain text in the password.txt file.

The config subfolder contains two json files. One file (ca.json) contains a list of all provisioners and the other file (defaults.json) contains some general information as to where the step ca can be reached and where the root certificate is located.

The db folder contains a NoSQL database with meta information on issued certificates.

The secrets folder contains the private keys for at least the intermediate certificate.

The templates folder will be empty upon initial setup but can be filled later on with certificate templates (very useful later on!).

Running a quick test

Of course we want to find out if our PKI is really running and visible from the outside. On a local command line (not the actual server running the PKI) we use openssl’s s_client command to check things out.

andreas@laptop ➜  ~ openssl s_client -connect acme.local:8443 -showcerts
CONNECTED(00000005)
depth=1 CN = myownlittleca Intermediate CA
verify error:num=20:unable to get local issuer certificate
verify return:0
---
Certificate chain
 0 s:/CN=Step Online CA
   i:/CN=myownlittleca Intermediate CA
-----BEGIN CERTIFICATE-----
MIIB2DCCAX+gAwIBAgIRAP9nSxkc+5TzPw9R3mUwtfIwCgYIKoZIzj0EAwIwKDEm
MCQGA1UEAxMdbXlvd25saXR0bGVjYSBJbnRlcm1lZGlhdGUgQ0EwHhcNMjAxMTI2
MTAzNzQzWhcNMjAxMTI3MTAzODQzWjAZMRcwFQYDVQQDEw5TdGVwIE9ubGluZSBD
QTBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABO7yVcVv1KLZ7e1QntLaSqPuFtGf
8aDuvYuoeP3KAsmcSGYbuukdIcXdL5VhRn10lXOIwGDnAxv+EzirHa94X46jgZgw
gZUwDgYDVR0PAQH/BAQDAgeAMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcD
AjAdBgNVHQ4EFgQUtiU+/65AZJE7CAgRDK4QK/F6YgowHwYDVR0jBBgwFoAUQyq5
oSctWu9k7KSnAz2P5rtKz9UwJAYDVR0RBB0wG4ITYWNtZS50aW5raXZpdHkuaG9t
ZYcEwKgcDzAKBggqhkjOPQQDAgNHADBEAiABBBGCV2x2zKm/6ja3inn9/u8QKx+G
BTuCkGcj1XZzEwIgTO+r7KTh2nuaN+uQsJOb51ASqLD2GDfH47CKBfd03Wo=
-----END CERTIFICATE-----
 1 s:/CN=myownlittleca Intermediate CA
   i:/CN=myownlittleca Root CA
-----BEGIN CERTIFICATE-----
MIIBrTCCAVOgAwIBAgIRAKn1KuHAPtPlKVmfI0G8NQMwCgYIKoZIzj0EAwIwIDEe
MBwGA1UEAxMVbXlvd25saXR0bGVjYSBSb290IENBMB4XDTIwMTEyNjEwMzgzMVoX
DTMwMTEyNDEwMzgzMVowKDEmMCQGA1UEAxMdbXlvd25saXR0bGVjYSBJbnRlcm1l
ZGlhdGUgQ0EwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAARYANusH97/11XzMIYf
7pgI1LEY8UpWVBiVF4/1m5rsaFg//kvkFklI7FjZ4nR4Ard7mqlrCDc16lseVMKl
mFNPo2YwZDAOBgNVHQ8BAf8EBAMCAQYwEgYDVR0TAQH/BAgwBgEB/wIBADAdBgNV
HQ4EFgQUQyq5oSctWu9k7KSnAz2P5rtKz9UwHwYDVR0jBBgwFoAUhArGpAX7JUjc
tn/PGaEkJkJ1tOMwCgYIKoZIzj0EAwIDSAAwRQIgbF/kVS7j+TFTZYpIoA3El+ty
rxRsD61qcT/UHEQSNSgCIQDFhRXerzwvQYz4BbpST2NfCdMvJaFVxrU99wTf4eUQ
bA==
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=Step Online CA
issuer=/CN=myownlittleca Intermediate CA
---

...

Next, we could install a server somewhere and use acme.sh or certbot or similar to automatically retrieve SSL certificates. However, at this point we don’t want to do this because the auto generated setup is not exactly what we want (or need).

Custom Setup

As stated above, we do not want to use the auto-generated certificate authorities. We already have our own CA in place that we’d like to use. Also, we will issue an exclusive intermediate CA for our PKI off-band and import that. In addition we want to have multiple provisioners with different policies as to how long certificates issued are valid.

In this article I will not describe what a Root CA is and how it is being created, but just assume that we have setup one already that is ready for import. Still, if you want to learn more about how to setup a CA please read here.

Importing our own Root CA

What we need to do is to import our existing root certificate. The same holds true for the Intermediate CA. We can either put that into the certs folder or have our configuration point to a central location.

In either event we will not need the private key for from our Root CA!

In this example we will copy our root certificate into a central location under /etc/ssl and make it readable for everybody via a quick chmod 444 command.

andreas@acme ➜  ~ sudo ls -lah /etc/ssl/
total 45
drwxr-xr-x   2 root  wheel     5B Nov 26 19:26 .
drwxr-xr-x  27 root  wheel   109B Nov 26 11:35 ..
lrwxr-xr-x   1 root  wheel    43B Oct 17 03:09 cert.pem -> ../../usr/local/share/certs/ca-root-nss.crt
-rw-r--r--   1 root  wheel    11K Jun 12 20:29 openssl.cnf
-r--r--r--   1 root  wheel   2.2K Nov 26 19:26 tinkivity.pem

For the next step, we need the 32-bit fingerprint from our certificate. Obviously the fingerprint below is redacted and you will not get any of the xx values as a reply on your command line.

andreas@acme ➜  ~ openssl x509 -fingerprint -sha256 -noout -in /etc/ssl/tinkivity.pem                       
SHA256 Fingerprint=00:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:99

We need to update /usr/local/etc/step/ca/config/defaults.json configuration file to reflect the fingerprint of our new root certificate. Please make sure to remove all colons (“:”) from the fingerprint in your defaults.json config. Again, below fingerprint is redacted and instead of the 30 pairs of xx you need to put the middle-30 actual bytes from your actual fingerprint. Also, make sure to update the location of the root certificate accordingly.

{
   "ca-url": "https://acme.local:8443",
   "ca-config": "/usr/local/etc/step/ca/config/ca.json",
   "fingerprint": "01xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx99",
   "root": "/etc/ssl/tinkivity.pem"
}

The other configuration we need to update is /usr/local/etc/step/ca/config/ca.json as it also needs to know where our root certificate lives. The attribute for the root certificate location is most likely the first attribute at the top of the json configuration.

{
   "root": "/etc/ssl/tinkivity.pem",
   "federatedRoots": [],
...

Importing our Intermediate CA

Again, we have created our Intermediate CA off-band and only import it into our ACME server environment in this step. As our Intermediate CA will actually be used to issue certificates, we need both the x509 certificate as well as the RSA private key for the Intermediate CA. We will delete possibly existing certificates and keys from the certs and secrets folder and import our Intermediate CA instead.

andreas@acme ➜  ~ sudo tree /usr/local/etc/step            
/usr/local/etc/step
├── ca
│   ├── certs
│   │   └── intermediate.cert.pem
│   ├── config
│   │   ├── ca.json
│   │   └── defaults.json
│   ├── db
│   │   ├── 000000.vlog
│   │   ├── LOCK
│   │   └── MANIFEST
│   ├── secrets
│   │   └── intermediate.key.pem
│   └── templates
└── password.txt

6 directories, 8 files

The x509 certificate (the public part) shall only be readable, but doesn’t need to be restricted. Thus, it is ok if everybody can read the file.

andreas@acme ➜  ~ sudo ls -lah /usr/local/etc/step/ca/certs/intermediate.cert.pem
-r--r--r--  1 step  step   2.2K Nov 28 14:39 /usr/local/etc/step/ca/certs/intermediate.cert.pem

The RSA private key on the other hand should be restricted. Nobody other the our step ca service user shall be allowed to read its contents.

andreas@acme ➜  ~ sudo ls -lah /usr/local/etc/step/ca/secrets/intermediate.key.pem
-r--------  1 step  step   3.2K Nov 28 14:37 /usr/local/etc/step/ca/secrets/intermediate.key.pem

Another and even more important line of defense is the passphrase that encrypts the RSA key. Even though somebody would come into possession of the RSA key file it couldn’t be decrypted without the proper passphrase. At the same time, the step ca service user needs to know that passphrase in order to sign new certificates. We have two options how to provide the passphrase to the step ca service:

1. interactive command line prompt upon service start
2. persistence in a text file

Obviously only the latter option allows unattended service starts (i.e. b/c of reboot) and we will use that option. The location for the password.txt file is manifested in the service script and by default points to the step ca root folder. In any case we must make sure that nobody else but the step ca service user can read the contents of that file.

andreas@acme ➜  ~ sudo ls -lah /usr/local/etc/step/password.txt
-rw-------  1 step  step    12B Nov 29 12:56 /usr/local/etc/step/password.txt

The last step for setup of our Intermediate CA is to configure its location in the /usr/local/etc/step/ca/config/ca.json configuration.

{
   "root": "/etc/ssl/tinkivity.pem",
   "federatedRoots": [],
   "crt": "/usr/local/etc/step/ca/certs/intermediate.cert.pem",
   "key": "/usr/local/etc/step/ca/secrets/intermediate.key.pem",
...

Delete existing provisioners

When running the automatically guided setup in the beginning, we also created a provisioner named firstprovisioner which we actually don’t want to have any more. There is a step command that allows to manage provisioners – including listing of those.

andreas@acme ➜  ~ sudo step ca provisioner list --ca-url https://acme.local:8443 --root /etc/ssl/tinkivity.pem
[
   {
      "type": "JWK",
      "name": "firstprovisioner",
      "key": {
         "use": "sig",
         "kty": "EC",
         "kid": "TRmwwSxlqIBSPDj6K5pAYrbcbCbkKPIWvPwDhuuqeWI",
         "crv": "P-256",
         "alg": "ES256",
         "x": "EgXHqunMX0k3GbPkbCcrCN44wKcYgHaIKx6TZvGwAXk",
         "y": "iGb2ToEVDC6yBgRxZoNa1MG1RAZUDrFokvim8Ugj9fg"
      },
      "encryptedKey": "eyJhbGciOiJQQkVTMi1IUzI1NitBMTI4S1ciLCJjdHkiOiJqd2sranNvbiIsImVuYyI6IkEyNTZHQ00iLCJwMmMiOjEwMDAwMCwicDJzIjoiWTdTU2kxaTJJRGpMQkY2cF9lNkFrQSJ9.6BhnTrakC_yUC1AMwIJ0pVW_spZode1Np8mba3ONk9NwCTErGb8upQ.tBP0pRs8ha6lijLz.pKHgHq6VChULDNvNWvHBYQMBeeGEJSOrVDU-9gA-soETOf4eLqjqy8OATp3pP3_TQ6y00E2ZziEnfJk58f3cbLT1lldas1yP0XYkc3gHitEwTfbFxppyp9ptjRzIPGby5ucVOzj0j9O8QiIetOc6Cri7rq9bpuTMyazAQlKJ84x1CeZz_hqBf3vxwHZHYODPaxG3u2nsWmjhFA8uJXPSHyic_sgZBi-sc5JGPVa2_4rG8EzM1yx2l0mUZLdVprAFZ0ciWvKRdqObXcbO_DiLn3p6aECFnLfEnvi0T8deoHhU0t5F28T4GNV_E9aq9h46A0O4rcLrXi9kgqs2g_k.eItQ0VITv702y3bFFkNnFQ"
   }
]

More or less the command only dumps out the provisioner section of the configuration at /usr/local/etc/step/ca/config/ca.json which doesn’t seem very helpful when listing existing provisioners. However, the command becomes more helpful when modifying provisioners. First we will delete our existing provisioner. We can use the step ca provisioner command to do this.

andreas@acme ➜  ~ sudo step ca provisioner remove firstprovisioner --ca-config /usr/local/etc/step/ca/config/ca.json
Success! Your `step-ca` config has been updated. To pick up the new configuration SIGHUP (kill -1 <pid>) or restart the step-ca process.

As an alternative to the above command we can directly edit the configuration file at /usr/local/etc/step/ca/config/ca.json and replace the provisioners section by a NULL statement.

Below is the complete /usr/local/etc/step/ca/config/ca.json file matching our current progress.

{
        "root": "/etc/ssl/tinkivity.pem",
        "federatedRoots": [],
        "crt": "/usr/local/etc/step/ca/certs/intermediate.cert.pem",
        "key": "/usr/local/etc/step/ca/secrets/intermediate.key.pem",
        "address": ":8443",
        "dnsNames": [
                "acme.local",
                "192.168.1.2"
        ],
        "logger": {
                "format": "text"
        },
        "db": {
                "type": "badger",
                "dataSource": "/usr/local/etc/step/ca/db",
                "badgerFileLoadingMode": ""
        },
        "authority": {
                "provisioners": null
        },
        "tls": {
                "cipherSuites": [
                        "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305",
                        "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
                ],
                "minVersion": 1.2,
                "maxVersion": 1.2,
                "renegotiation": false
        }
}

Configuring a separate log facility

Let’s configure a separate log facility that logs to /var/log/step.log so that we have an easier job in following the log activities (other than to filter /var/log/messages all the time). We start by inserting the following two lines into the /etc/syslog.conf configuration.

...
# !devd
# *.>=notice                                    /var/log/devd.log
!step_ca
*.*                                             /var/log/step.log
!ppp
*.*                                             /var/log/ppp.log
!*
include                                         /etc/syslog.d
include                                         /usr/local/etc/syslog.d

Next we create an empty log file under /var/log/step.log and make sure it has the same ownership and permissions than other log files under /var/log.

andreas@acme ➜  ~ sudo ls -lah /var/log/messages
-rw-r--r--  1 root  wheel    14K Nov 29 14:35 /var/log/messages
andreas@acme ➜  ~ sudo touch /var/log/step.log
andreas@acme ➜  ~ sudo ls -lah /var/log/step.log
-rw-r--r--  1 root  wheel     0B Nov 29 15:46 /var/log/step.log

Now, we restart the syslog daemon so that the new configuration is applied.

andreas@acme ➜  ~ sudo service syslogd restart
Stopping syslogd.
Waiting for PIDS: 38133.
Starting syslogd.

Finally, we can (re)start the step ca service and make sure the newly configured log file is being used. Assuming we have not made any errors in our configuration approach so far, our step ca should start without errors and be responsive at port 8443 already.

andreas@acme ➜  ~ sudo service step-ca restart
Stopping step_ca.
Starting step_ca.
step_ca is running as pid 39809.
andreas@acme ➜  ~ cat /var/log/step.log 
Nov 29 15:48:34 acme step_ca[39809]: 2020/11/29 15:48:34 Serving HTTPS on :8443 ...

Running a quick smoke test

We could now run openssl’s s_client command again (see above) from a remote host or simply point a web browser at https://acme.local:8443. In both cases we should receive a reply that is including a correctly setup certificate chain.

andreas@acme ➜  ~ cat /var/log/step.log
Nov 29 15:48:34 acme step_ca[39809]: 2020/11/29 15:48:34 Serving HTTPS on :8443 ...
Nov 29 15:53:13 acme step_ca[39809]: time="2020-11-29T15:53:13+01:00" level=warning duration="38.366µs" duration-ns=38366 fields.time="2020-11-29T15:53:13+01:00" method=GET name=ca path=/ protocol=HTTP/2.0 referer= remote-address=192.168.1.205 request-id=bv1rbmajnji9n0kqlm10 size=19 status=404 user-agent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.1 Safari/605.1.15" user-id=

Looking at /var/log/step.log again shows our step ca being responsive. Although the client only receives a 404 error in return, the meta data around that HTTPS request contains the proof that our setup works. It becomes even more clear when looking at the reply from openssl’s s_client command that we can run of our local laptop.

andreas@testclient ➜  ~ openssl s_client -connect acme.local:8443 --quiet      
depth=1 C = DE, ST = Saxony, O = Tinkivity, OU = Tinkivity Intermediate Certificate Authority, CN = Smallstep Intermediate CA, emailAddress = xxx@xxx.com
verify return:1
depth=0 CN = Step Online CA
verify return:1

Of course, this is only a somewhat synthetic test, but it will show us that we’re well on track.

Adding a new ACME provisioner

This is a rather easy step because only two commands are involved. The first command adds a new provisioner of type ACME and the second command restarts the service.

andreas@acme ➜  ~ sudo step ca provisioner add acme-smallstep --type acme --ca-config /usr/local/etc/step/ca/config/ca.json
Success! Your `step-ca` config has been updated. To pick up the new configuration SIGHUP (kill -1 <pid>) or restart the step-ca process.
andreas@acme ➜  ~ sudo service step-ca restart
Stopping step_ca.
Starting step_ca.
step_ca is running as pid 41017.

Looking at the provisioners section in /usr/local/etc/step/ca/config/ca.json we can see that not that much has been added actually.

...
                "provisioners": [
                        {
                                "type": "ACME",
                                "name": "acme-smallstep"
                        }
                ]
...

Such default configuration would start to pass out certificates that adhere to smallstep’s default settings. One setting that we want to change is the validity of the certificates being issued. We actually like certificates to be valid as short as possible while still not adding too much stress to the infrastructure. We will thus agree to certificates being valid for 24 hours.

...
                "provisioners": [
                        {
                                "type": "ACME",
                                "name": "acme-smallstep",
                                "claims": {
                                        "maxTLSCertDuration": "24h0m0s",
                                        "defaultTLSCertDuration": "24h0m0s"
                                }
                        }
                ]
...