In the last blog post Running your own ACME Server we have successfully installed our own PKI with an ACME provisioner. In this blog post we want to look at the client side and automatically obtain and renew a client certificate for a web server.
NGINX
From an ACME point of view the type of web server doesn’t matter at all. In this example we will use NGINX as a web server, because it is lightweight and popular.
andreas@testclient ➜ ~ sudo pkg install nginx
As we want NGINX to run as a service we will append one line to our /etc/rc.conf and then start the service.
andreas@testclient ➜ ~ sudo sh -c 'echo nginx_enable=\"YES\" >> /etc/rc.conf' andreas@testclient ➜ ~ sudo service nginx start Performing sanity check on nginx configuration: nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful Starting nginx.
Now we apply a pattern that extracts web server configuration and contents into separate file system locations. Each web server (or virtual domain) will get its own content folder. In our case we want to put all content into a testclient folder.
andreas@testclient ➜ ~ sudo mkdir -p /usr/local/www/sites/testclient/html
We will create a simple html file at /usr/local/www/sites/testclient/html/index.html with the following content.
<html>
<head>
<title>TESTCLIENT</title>
</head>
<body>
<h1>Hello World!</h1>
</body>
</html>
The webserver configuration will be put into a .conf file the follows the same name. Although we only need one webserver in our example we will still create a sites subfolder for good housekeeping.
andreas@testclient ➜ ~ sudo mkdir /usr/local/etc/nginx/sites andreas@testclient ➜ ~ sudo touch /usr/local/etc/nginx/sites/testclient.conf
Our site configuration at /usr/local/etc/nginx/sites/testclient.conf will have the following content.
server {
listen 80;
server_name testclient;
access_log /var/log/nginx/testclient.access.log;
error_log /var/log/nginx/testclient.error.log;
location / {
root /usr/local/www/sites/testclient/html;
index index.html;
}
}
At last we clean up /usr/local/etc/nginx/nginx.conf by removing the complete server section as we don’t need it anymore. Instead we will add an include statement before the last { of the http section. That will make sure our /usr/local/etc/nginx/sites/testclient.conf configuration file will be parsed.
Based on a fresh installation the config file would most likely look like the following.
# This default error log path is compiled-in to make sure configuration parsing
# errors are logged somewhere, especially during unattended boot when stderr
# isn't normally logged anywhere. This path will be touched on every nginx
# start regardless of error log location configured here. See
# https://trac.nginx.org/nginx/ticket/147 for more info.
#
#error_log /var/log/nginx/error.log;
#
#pid logs/nginx.pid;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
include "sites/*.conf";
}
Now we check the configuration and reload the config.
andreas@testclient ➜ ~ sudo nginx -t nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful andreas@testclient ➜ ~ sudo service nginx reload Performing sanity check on nginx configuration: nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Adding our own Root Certificate to the trust store
To make things easier we will add our own root certificate into the trust store of our client. we will copy the certificate into the /usr/share/certs/trusted folder and then apply a rehash operation that is limited to only the one certificate we just copied.
andreas@testclient ➜ ~ openssl x509 -hash -noout -in /usr/share/certs/trusted/tinkivity.pem 97efb5b5 andreas@testclient ➜ ~ sudo ln -s /usr/share/certs/trusted/tinkivity.pem /etc/ssl/certs/97efb5b5.0
We can see if we have been successful if openssl’s s_client command can verify the certificate from our ACME server.
andreas@testclient ➜ ~ openssl s_client -connect acme.local:8443 --quiet depth=1 C = DE, ST = Saxony, O = Tinkivity, OU = Tinkivity Intermediate Certificate Authority, CN = Smallstep Intermediate CA, emailAddress = xxx@xxx.com verify return:1 depth=0 CN = Step Online CA verify return:1
Certbot
Now, as we have setup a new web server, we can install Certbot and have it obtain a certificate from our ACME server. The first step is installing the packages for Certbot itself and its NGINX plugin.
andreas@testclient ➜ ~ sudo pkg install py37-certbot py37-certbot-nginx
Before we move on to the next step of registration of our domain at the ACME server we need to find out if python can successfully integrate the trust store. We issue a simple python command to check SSL verification.
andreas@testclient ➜ ~ python3.7 -c "import requests; print(requests.get('https://acme.local:8443').text)"
404 page not found
If we receive real HTML content (above 404 page not found is actually HTML and thus success), we are good for ‘regular’ Certbot usage. If we receive a lengthy exception that somewhere contains a line like below, our python installation doesn’t include the trust store correctly and we will need to operate Certbot with the –no-verify-ssl option for further requests.
requests.exceptions.SSLError: HTTPSConnectionPool(host='acme.local', port=8443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])")))
Above error happens on FreebSD 12.2-RC3 with python3.7 and seems to be a deeper issue, because python claims to look at the correct trust store location:
andreas@testclient ➜ ~ python3.7 -c "import ssl; print(ssl.get_default_verify_paths())" DefaultVerifyPaths(cafile='/etc/ssl/cert.pem', capath='/etc/ssl/certs', openssl_cafile_env='SSL_CERT_FILE', openssl_cafile='/etc/ssl/cert.pem', openssl_capath_env='SSL_CERT_DIR', openssl_capath='/etc/ssl/certs')
The next step already is the registration of our domain at the ACME server. We use the following command:
andreas@testclient ➜ ~ sudo certbot --nginx --agree-tos --non-interactive --no-verify-ssl --email xxx@xxx.com --server https://acme.local:8443/acme/acme-smallstep/directory --domain testclient Saving debug log to /var/log/letsencrypt/letsencrypt.log Plugins selected: Authenticator nginx, Installer nginx /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning, /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning, /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning, Obtaining a new certificate /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning, /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning, Performing the following challenges: http-01 challenge for testclient Using default address 80 for authentication. nginx: [warn] conflicting server name "testclient" on 0.0.0.0:80, ignored Waiting for verification... /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning, /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning, Cleaning up challenges /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning, /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning, /usr/local/lib/python3.7/site-packages/urllib3/connectionpool.py:988: InsecureRequestWarning: Unverified HTTPS request is being made to host 'acme.local'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings InsecureRequestWarning, Could not automatically find a matching server block for testclient. Set the `server_name` directive to use the Nginx installer. IMPORTANT NOTES: - Unable to install the certificate - Congratulations! Your certificate and chain have been saved at: /usr/local/etc/letsencrypt/live/testclient/fullchain.pem Your key file has been saved at: /usr/local/etc/letsencrypt/live/testclient/privkey.pem Your cert will expire on 2020-12-01. To obtain a new or tweaked version of this certificate in the future, simply run certbot again with the "certonly" option. To non-interactively renew *all* of your certificates, run "certbot renew" - Your account credentials have been saved in your Certbot configuration directory at /usr/local/etc/letsencrypt. You should make a secure backup of this folder now. This configuration directory will also contain certificates and private keys obtained by Certbot so making regular backups of this folder is ideal.
Although the ACME part worked completely fine, we still get an error from Certbot’s NGINX plugin. It turns out that the plugin cannot locate the server_name directive in our NGINX configuration. That is driven by the fact that we have extracted parts of the NGINX configuration into a separate configuration file (/usr/local/etc/nginx/sites/testclient.conf). We have two options now:
- go back to a single NGINX configuration file
- manually enter the Certbot configuration snippets into our separate NGINX configuration file
We will go with the latter option and put in the Certbot configuration snippets ourselves. The configuration at /usr/local/etc/nginx/sites/testclient.conf will now look as follows.
server {
# listen 80;
listen 443 ssl;
server_name testclient;
access_log /var/log/nginx/testclient.access.log;
error_log /var/log/nginx/testclient.error.log;
location / {
root /usr/local/www/sites/testclient/html;
index index.html;
}
ssl_certificate /usr/local/etc/letsencrypt/live/testclient/fullchain.pem;
ssl_certificate_key /usr/local/etc/letsencrypt/live/testclient/privkey.pem;
include /usr/local/etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /usr/local/etc/letsencrypt/ssl-dhparams.pem;
}
server {
if ($host = testclient) {
return 301 https://$host$request_uri;
}
listen 80;
server_name testclient;
return 404;
}
- our existing server block has been updated to not listen on port 80 any more, but on port 443 via SSL instead
- the locations for the certificate obtained from the ACME server, the private key, ssl options (cipher suite etc.) as well as Diffie-Hellman parameters have been included into the configuration
- a new server block has with the goal of listening on port 80 and redirection to port 443 SSL has been added
In order to apply the configuration changes, we have to reload the NGINX configuration.
andreas@testclient ➜ ~ sudo service nginx reload Performing sanity check on nginx configuration: nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok nginx: configuration file /usr/local/etc/nginx/nginx.conf test is successful
Auto Renew
Last but not least we need to insert a cron task so that Certbot will automatically renew the certificate on a regular schedule.
andreas@testclient ➜ ~ echo "0 0,12 * * * root python3.7 -c 'import random; import time; time.sleep(random.random() * 3600)' && certbot renew --no-verify-ssl --quiet" | sudo tee -a /etc/crontab > /dev/null
Above entry will run Certbot’s renew command at midnight and high noon. Without further parameters (i.e. domain) above command will renew all certificates managed by Certbot. If you want to see which certificates are being managed by Certbot you can run the following command.
andreas@testclient ➜ ~ sudo certbot certificates
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Cannot extract OCSP URI from /usr/local/etc/letsencrypt/live/testclient/cert.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Found the following certs:
Certificate Name: testclient
Serial Number: 8a98c881442ed7de1d460dee5a97fb6
Domains: testclient
Expiry Date: 2020-12-02 18:51:33+00:00 (VALID: 23 hour(s))
Certificate Path: /usr/local/etc/letsencrypt/live/testclient/fullchain.pem
Private Key Path: /usr/local/etc/letsencrypt/live/testclient/privkey.pem
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
That’s it. If you put your web browser onto http://testclient in your local network, you should see a Hello World page with a valid certificate.
