GitHub for server automation — SSL certificates

ci cd

Previously, I shared how I had automated the renewal of my SSH server certificates using GitHub Actions. Immediately after that, I turned to automating SSL certificates. At the time of experimenting, half of my domains, including this blog and my company’s main website, had certificates long expired. To be honest, one main reason for me to use SSH certificate authentication and set up the whole infrastructure is to pave the road for auto renewing my SSL certificates.

I use acme.sh to get new certificates. The only reason I chose it over Let’s Encrypt is because it is more battery-included. I do not need to manually install any plugin to use Aliyun DNS API to complete my domain challenge. You can install it and get new certificates without typing a single key, which is crucial for automation. I believe this can also be done for Let’s Encrypt but I have not tried it yet.

So, how is my set up? First of all, I have prepared a list of servers. Each server entry comes with a list of domains currently on that server, and a reload command that reloads nginx among other things. The whole list is authored in JSON and of course committed into my GitHub repo. The file looks like this:

{
"example.com": {
"domains": [
"example.com",
"example.org"
],
"reload": "systemctl reload nginx"
},
}

.

During my certificate renewal workflow, the script first lists all domains with certificates soon to be expired. The script would proceed only if this list is non-empty. It then installs acme.sh and uses the tool to issue certificates. Lastly, the script enumerates the server list: if some server contains a domain where a certificate has just been issued, that certificate will be uploaded; if a server has any certificate updated, the renew command for that server will be executed, only once, after all certificates of that server have been uploaded.

The whole experience is fun. I would like to share a few interesting parts.

First of all, checking the expiration date. Unlike SSH certificate, where each workflow run simply updates every certificate, SSL certificate renewal is more costly and performed only when a certificate is near expiration. To check that, I googled around and found the following snippet:

echo | openssl s_client -showcerts -servername $DOMAIN -connect $DOMAIN:443 2>/dev/null | openssl x509 -inform pem -noout -enddate | cut -d '=' -f 2

. This should produce something like this:

May 26 21:58:36 2023 GMT

. Unfortunately, I was unable to parse this with my shell skillset. So I ended up writing a Swift script, since GitHub has Swift preinstalled:

import Foundation

let formatter = DateFormatter()

formatter.locale = Locale(identifier: "en_US_POSIX")
formatter.dateFormat = "MMM d HH:mm:SS yyyy z"
formatter.isLenient = true

guard
let stdin = try? FileHandle.standardInput.readToEnd(),
let input = String(data: stdin, encoding: .utf8),
let date = formatter.date(from: input.trimmingCharacters(in: .whitespacesAndNewlines))
else {
exit(-1)
}

let now = Date()
let distance = now.distance(to: date)
print(Int(floor(distance)))

Piping the date string to the Swift script would print the number of seconds before this certificate will be expired.

The next thing I learned is that in bash, I can do calculation just using $((3+4*5)). I always thought I would need calc for that, which is not available on Ubuntu, a point not occuring to me until a failed CI run. Before I found this trick though, I was using python -c "print(3+4*5)", which works equally fine, albeit a bit more verbose.

Last but not the least, jq. This is an immensely useful command line utility, yet I have never heard of it during my decade-long programming career. jq can perform simple JSON query. For instance, to get the list of all domains from my JSON dictionary, I can just write

jq -r 'map_values(.domains)[][]' domain_list.json

. map_values map the values of the dictionary, and each [] unflattens the array.

With jq, I no longer need to write complicated line based configuration full of duplicated information and study how to use sed or awk to get what I want. I can simply write a clear JSON tree. Even better, jq comes preinstalled on Ubuntu. You need to brew it on macOS though.