TL;DR:

Use certbot or some other ACME client if you want dynamically generated LetsEncrypt SSL certificates for multiple domains. And if you're going to dynamically serve an SSL certificate based on the domain from which the request originates - use SNI (Server Name Indication).

Introduction

While working on a system for a company that helps many other companies promote their job openings in their region, we encountered an unusual problem that ought to be solved. In this article, I'll try to explain how we solved the problem and what you can do if you find yourself in a similar situation where you need dynamically generated and/or served SSL certificates.

About the problem

Among many services this company provided for their customers, there was a custom page builder we had built for the customers to create and customize their own page easily and import job openings, job application forms, etc., from other services to showcase their career opportunities. One of the main advantages of using this page builder was the fact that you did not have to self-host this page. Instead, you could point a subdomain you owned to their server that would serve your custom-built page. So, for example, you could point https://jobs.your-company.com/ to this server and have your page served on that URL from their server. This was an awesome feature, but it introduced an unusual problem - we suddenly had a requirement to dynamically generate SSL certificates for each domain name and another problem (related to the first problem, and probably even more unusual) - a requirement to dynamically serve the SSL certificate based on the domain name from which the request is coming.

Initial ideas

When we started planning how to resolve this issue, we already had a general idea of how to generate these SSL certificates. So we commenced research on which ACME client would be the easiest to integrate into our system. We were using Node.js and we wanted to store the generated certificates in our PostgreSQL database. So we opted for a library called acme-client which is available on npm registry. This really doesn't matter since most ACME clients work on the same principle:

1. You create an order using the client and provide a domain for which you want to generate a certificate and the client returns a token provided by LetsEncrypt which is used to validate that you are the owner of the domain (you can serve content on that domain).
2. After you've received the token you should start serving this token on the path provided with the token since the client informs LetsEncrypt that you received the token.
3. LetsEncrypt starts something called ACME challenges that have a couple of different types. For simplicity purposes, we'll only talk about HTTP-01 challenges which are most common and de facto standard. In this scenario we also had to use that type of challenge since we do not own the domain, but for all intents and purposes we can say that we own the subdomain. This means that LE sends a request (potentially multiple times) to your domain on a previously mentioned path trying to retrieve the token that you received earlier.
4. The client validates each ACME challenge and if all of them were satisfied (i.e. the correct token was retrieved by LE) you can finalize your order and get your SSL certificate as a result.

Enough talk, let's get to some coding.

Generating certificates

In this article, I am using Sequelize ORM for handling data in the database and Express as a REST API server.

First of all, if you plan on using acme-client library, you'll have to add it as a dependency to our Node.js project:

npm install acme-client

The first step would be storing each domain on which our customers want us to serve content (their pages). We can store that in our database.

import { Model } from 'sequelize';

export default class Domain extends Model {
  static init(sequelize, DataTypes) {
    return super.init({
      domainName: DataTypes.STRING,
      key: DataTypes.STRING(10000),
      cert: DataTypes.STRING(10000),
      acmeChallenge: DataTypes.STRING,
    }, {
      sequelize,
      paranoid: true,
    });
  }
}

We defined a Domain model which has some fields that we'll talk more about later.

router.post('/domain', async (req, res) => {
  await Domain.create({
    domainName: req.body.domainName,
  });
  res.status(200).send('Added a domain');
});

We defined an endpoint which allows customers to add their subdomain (domainName field of our Domain model).

Additionally what we can (and probably should) do is prevent subdomains being added if they are not pointed to our server. We can achieve this by using the DNS module which is built into Node.js. This module's Resolver class provides multiple methods to check if a subdomain is pointed to our server (resolve, resolve4, resolveCname) and which one we should use depends entirely on which type of DNS record our customers are using to point the subdomain to our server.

We can add an afterCreate hook for our Domain model which triggers our acme-client to create a new order for an SSL certificate for our newly created Domain.

hooks: {
  afterCreate: (domain) => {
    domain.generateSSLCertificate();
  },
},

This hook should utilize acme-client library so let's first set up acme-client in our project.

import acme from 'acme-client';

export const newAcmeClient = async () => {
  const accountKey = await acme.forge.createPrivateKey();
  return (
    new acme.Client({
      directoryUrl: acme.directory.letsencrypt[process.env.NODE_ENV === 'production' ? 'production' : 'staging'],
      accountKey,
    })
  );
};

export const createCsr = (commonName, altNames) => acme.forge.createCsr({
  commonName,
  altNames,
});

export const challengeCreateFn = async (authz, challenge, keyAuthorization, domainModelInstance) => {
  if (keyAuthorization) {
    domainModelInstance.set({
      acmeChallenge: keyAuthorization,
    });
    await domainModelInstance.save();
  }
};

export const challengeRemoveFn = async (authz, challenge, keyAuthorization, domainModelInstance) => {
  domainModelInstance.set({
    acmeChallenge: null,
  });
  await domainModelInstance.save();
};

Let's go through this code:

1. newAcmeClient is a function that returns a new acme-client instance with an account key generated using acme's forge module and a directoryUrl based on the environment to which our app is deployed.
2. createCsr is a function which creates CSR (Certificate Signing Request) for our domain name using acme's forge module
3. challengeCreateFn and challengeRemoveFn are functions which save the token that we receive from LetsEncrypt to the database and remove the token after validating ACME challenges

And then we need to also add generateSSLCertificate method to the Domain model and import the functions we defined above.

import {
  newAcmeClient, createCsr, challengeCreateFn, challengeRemoveFn,
} from '../acme-client';
export default class Domain extends Model {
  // ...
  async generateSSLCertificate() {
    const altNames = [this.domainName];
    const acmeClient = await newAcmeClient();
    await acmeClient.createAccount({
      termsOfServiceAgreed: true,
      contact: ['mailto:info@your-company.com'],
    });
    const order = await acmeClient.createOrder({
      identifiers: [
        { type: 'dns', value: this.domainName },
      ],
    });
    const authorizations = await acmeClient.getAuthorizations(order);

    for (const authz of authorizations) {
      const { challenges } = authz;
      const challenge = challenges.find(({ type }) => type.startsWith('http'));
      if (challenge) {
        const keyAuthorization = await acmeClient.getChallengeKeyAuthorization(challenge);
        try {
          await challengeCreateFn(authz, challenge, keyAuthorization, this);
          await acmeClient.verifyChallenge(authz, challenge);
          await acmeClient.completeChallenge(challenge);
          await acmeClient.waitForValidStatus(challenge);
        } catch (e) {
          console.error(e);
        } finally {
          await challengeRemoveFn(authz, challenge, keyAuthorization, this);
        }
      }
    }
    const [key, csr] = await createCsr(this.domainName, altNames);
    await acmeClient.finalizeOrder(order, csr);
    const cert = await acmeClient.getCertificate(order);
    this.key = key;
    this.cert = cert;
    await this.save();
  }
}

I should probably break this method down since it is really unclear what it does if it is the first time you are working with an ACME client.

1. We create an order for our domain name using a newly created account.
2. We retrieve the authorizations from our order. These authorizations contain http-01 challenges.
3. For each http-01 challenge we are calling the challengeCreateFn and telling ACME client that we are ready for LE validating that challenge.
4. After each http-01 challenge is complete we are calling challengeRemoveFn
5. We need to create a CSR and finalize the order with this CSR which is used to sign our certificate.
6. We retrieve the certificate from the order and store in our database these two strings: the key from CSR, as well as the certificate we just retrieved from the order since those two are what we need to provide for each TLS handshake.

Serving certificates

Serving certificates based on the hostname was a bit tricky since we needed to look up the certificate and the CSR key in the database before the TLS handshake. Thankfully, there is a way to accomplish this using something that's called SNI (Server Name Indication). It's an extension to the TLS protocol by which a client indicates which hostname it is attempting to connect to at the start of the handshaking process. When creating a https server in Node, we can specify the options parameter which can include an SNICallback property for handling this specific case. In that callback we can fetch anything from our database and create a secure context using the corret key and certificate. Let's see how that looks in our code.

const options = {
  SNICallback: async (hostname, cb) => {
    const { key, cert } = await Domain.findOne({ where: { domainName: hostname } });
    cb(null, tls.createSecureContext({
      key,
      cert,
    }));
  },
  key: null,
  cert: null,
};

const server = https.createServer(options, app);

Conclusion

Both in life and engineering, some goals may seem unreachable at first, but you just have to keep going and not give up and eventually, you'll reach them. As this article shows, even building your own mini website hosting can be achieved quite easily using only Node and any SQL database if you focus on finding a solution and leave your comfort zone.

Thank you for taking the time to read this article.