Smart Card/HSM backed OpenSSL CA

This article describes how to set up a Smart Card/HSM backed OpenSSL CA using a Smart Card HSM or any PKCS11 enabled device.

Background

Since some years back I use WPA2 Enterprise with EAP-TLS (Certificate authentication) for my wifi at home. Historically I have used certificates from a public CA for this purpose. This is not best practice since you don’t have control over the certificates that are issued.

Also, I recently bought a new switch capable of 802.1X authentication on all ports. For this purpose I want all my machines (even those without wifi) to have certificates. So I decided to go through the hassle of setting up my own private CA.

Setting up CA

For the basic setup of the CA I followed Jamies excellent guide on setting up a CA. So in this post you can assume that all the basic stuff like folders structure and basic commands are the same. I will only show you the differences needed to have the Root CA key stored on a PKCS11 device like a HSM, Smart Card HSM or a Yubikey. I will even try to follow his topic names so you can follow along.

Configure PKCS11 Engine

I will not discuss the operating system part of getting PKCS11 devices to work in this article. But basically you just need to install some packages, you can read about it here.

First of all we need to configure OpenSSL to talk to your PKCS11 device. This can be done from configuration or interactively on the command line.

From conf:

# At beginning of conf (before everything else)
openssl_conf            = openssl_def

# At end of conf (after everything else)
[openssl_def]
engines = engine_section

[engine_section]
pkcs11 = pkcs11_section

[pkcs11_section]
engine_id = pkcs11
dynamic_path = /usr/local/lib/engines/pkcs11.so
MODULE_PATH = /usr/local/lib/opensc-pkcs11.so
init = 0

From cli:

OpenSSL> engine -t dynamic -pre SO_PATH:/usr/local/lib/engines/pkcs11.so -pre ID:pkcs11 -pre LIST_ADD:1 -pre LOAD -pre MODULE_PATH:/usr/local/lib/opensc-
pkcs11.so

Create the root pair

First of all we need to have a RSA key pair on the PKCS11 device:

# pkcs11-tool --module /usr/local/lib/opensc-pkcs11.so -l --keypairgen --key-type rsa:2048 --label "SSL Root CA"
Using slot 0 with a present token (0x0)
Logging in to "HSM 2 (UserPIN)".
Please enter User PIN:
Key pair generated:
Private Key Object; RSA
  label:      SSL Root CA
  ID:         d15c3e9578a612a658bb14e0e147db4f2279cf19
  Usage:      decrypt, sign, unwrap
Public Key Object; RSA 2048 bits
  label:      SSL Root CA
  ID:         d15c3e9578a612a658bb14e0e147db4f2279cf19
  Usage:      encrypt, verify, wrap

Create the root certificate

I will assume that you have configured pkcs11 in openssl.cnf (otherwise you will have to first run the engine command in openssl interactively before any other command).

# openssl req -config openssl.cnf -new -x509 -days 7300 -sha256 -extensions v3_ca -engine pkcs11 -keyform engine -key 0:d15c3e9578a612a658bb14e0e147db4f2279cf19 -out certs/ca.cert.pem
engine "pkcs11" set.
Enter PKCS#11 token PIN for HSM 2 (UserPIN):
0x8018b6000 07:41:35.523 cannot lock memory, sensitive data may be paged to disk
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [SE]:
State or Province Name []:
Locality Name []:
Organization Name [PeanOrg]:
Organizational Unit Name []:PeanOrg Certificate Authority
Common Name []:PeanOrg Root CA
Email Address []:

Create the intermediate pair

For the intermediate key pair I followed jamies guide. I need frequent access to this CA so I have decided to have the intermediate pair on file instead of HSM.

Create the intermediate certificate

I changed one thing in jamies intermediate/openssl.cnf because I dont see the point of having province set in the CAs

stateOrProvinceName     = optional

To use the Root key stored on pkcs11 to sign the intermediate certificate use this command:

# openssl ca -config openssl.cnf -extensions v3_intermediate_ca -days 3650 -notext -md sha256 -engine pkcs11 -keyform engine -keyfile 0:d15c3e9578a612a658bb14e0e147db4f2279cf19 -in intermediate/csr/intermediate.csr.pem -out intermediate/certs/intermediate.cert.pem
Using configuration from openssl.cnf
engine "pkcs11" set.
Enter PKCS#11 token PIN for HSM 2 (UserPIN):
Check that the request matches the signature
Signature ok
Certificate Details:
        Serial Number: 4096 (0x1000)
        Validity
            Not Before: Apr  7 05:54:22 2018 GMT
            Not After : Apr  4 05:54:22 2028 GMT
        Subject:
            countryName               = SE
            organizationName          = PeanOrg
            organizationalUnitName    = PeanOrg Certificate Authority
            commonName                = PeanOrg Intermediate CA 1
        X509v3 extensions:
            X509v3 Subject Key Identifier:
                77:9C:07:23:FD:40:E9:5C:7E:30:73:8F:59:28:25:F5:06:43:B4:70
            X509v3 Authority Key Identifier:
                keyid:A4:F2:DE:15:8E:9E:A8:87:B0:95:D4:21:A2:BD:4C:41:02:93:E0:8D

            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0
            X509v3 Key Usage: critical
                Digital Signature, Certificate Sign, CRL Sign
Certificate is to be certified until Apr  4 05:54:22 2028 GMT (3650 days)
Sign the certificate? [y/n]:y


1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated

We now have all we need to sign certificates. Just follow Jamies guide Sign server and client certificates

References

It took me a few hours to get this going because of sort of a lack of documentation on how to use OpenSSL and PKCS11 together, during my efforts I found these resources helpful

Specify which CAs are allowed to issue certificates using CAA record

A Certification Authority Authorization (CAA) record is used to specify which certificate authorities (CAs) are allowed to issue certificates for your domains.

Background

When well behaved certificate authorities (CAs) issue new certificates they follow certain procedures. These procedures are documented in their Certification Practice Statement (CPS) which should be a public document. For example the CA needs to make sure that the entity that have ordered a new certificate is actually the owner of that domain. There is today different levels of assurance. For example the CA need to be “more” sure that you are actually the owner of a domain in order to issue a certificate that gives you “the green padlock”

There is now a way for domain owners to influence these procedures using a new DNS record type, CCA (Certification Authority Authorization)

The CAA record and CPS

RFC 6844 describes the format of the record and how CAs should handle it if they decide to take it into consideration.

Before issuing a certificate, a compliant CA MUST check for publication of a relevant CAA Resource Record set. If such a record set exists, a CA MUST NOT issue a certificate unless the CA determines that either (1) the certificate request is consistent with the applicable CAA Resource Record set or (2) an exception specified in the relevant Certificate Policy or Certification Practices Statement applies.

https://tools.ietf.org/html/rfc6844


In this way the domain owner can influence the security of the CA infrastructure by publishing these resource records into DNS. If a company exclusively use one certificate provider CAA records will give the CA another tool to verify that they can issue certificates for a specific domain. Today not all big CAs obey these records, but they will pretty soon. Earlier this year The CAB  Forum voted  in favour of making it mandatory for all CAs by passing Ballot 187 – Make CAA Checking Mandatory. The motion states

As part of the issuance process, the CA must check for a CAA record for each dNSName in the subjectAltName extension of the certificate to be issued, according to the procedure in RFC 6844

https://cabforum.org/2017/03/08/ballot-187-make-caa-checking-mandatory/


This change will be in effect on 8 September 2017.

CAA record format

The structure if a CAA record is as follows

<domain>   CAA <flags> <tag> <value>
  • flags – Octet of option bits in decimal form. Only bit 0 is used today
  • tag – An ASCII string that represents the function of the record. The RFC defines three tags
    • issue – Specifies that the record concerns issuance of certificates
    • issuewild – Specifies that the records concern issuance of wilrdcard certificates
    • iodef – url where the certificate authority can report policy violations
  • value – Basically the domain of the CA allowed to issue certificates for the domain.

One example could be

# drill CAA framkant.org
;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 40373
;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 2, ADDITIONAL: 0 
;; QUESTION SECTION:
;; framkant.org.	IN	CAA

;; ANSWER SECTION:
framkant.org.	300	IN	CAA	0 issue "letsencrypt.org"
framkant.org.	300	IN	CAA	0 iodef "mailto:iodef@framkant.org"

This tells the issuing CAs that only letsencrypt are allowed to issue certificates for framkant.org and all its subdomains. The second record tells them that policy violations should be reported by email to iodef@framkant.org.

One big problem right now is that only a very few domains use this feature. To make this really powerful we need more domains to use this feature. But a big first step is to make CCA checking mandatory for all big CAs.

Scalable access control using OpenSSH Certificates

Background

I’ve been using OpenSSH certificates for some time now. They are very handy if you have a bunch of machines you want to trust, or a bunch of machines that shoud trust you.  It’s very effective to trust just one host CA in order to trust all servers with certificates signed by this CA. Or the other way around, have your personal public key signed by a user CA and then be automatically trusted by all servers that trust this CA. But if you working together with alot of people let say within an organisation this becomes problematic pretty soon. Maybe your frontend people should only have access to webservers and database people to the database servers and so on. The solution to this is the little known flags AuthorizedPrincipalsFile and AuthorizedPrincipalsCommand in sshd_config.

OpenSSH Certificates and principals

Lets start with the certificates and the principals within these certificates. In order to make this work I would suggest to use principals within the certificates that are closely tied to the person using it, their company wide username for example. If different keys have different access leves (lets say because on of them are stored on a physical secure element/smart card) it is good to include this kind of information in the principal, according to some standard you make up. Let say I use yubikeys to store my private keys I could have principals like peter_file and peter_physcial. These are easily parsable and  connected to a physical person.

Please note that there could be other access schemes where role is more important that how the key is stored, then role could be a better option for the principal suffix. But please note that if you burn the role into the certificate that person will need to have the current certificate revoked and have a new one issued if the role is ever changed. I will discuss a better way to handle this later.

Another option is to have multiple principals in the same certificate (ie peter,webmaster,root) but this also gets cumbersome when privileges and roles start to change over time.

AuthorizedPrincipalsFile

One solution to this problem is the configuration option AuthorizedPrincipalsFile in sshd_config. With this option you tell sshd where to look for a list of principals valid for a certain user. I looks something like

AuthorizedPrincipalsFile /etc/ssh/%u_principals

When someone tries to log in as peter sshd will check my certificate for validity and then look for valid principals in the file /etc/ssh/peter_principals. sshd expects this file to contain one valid principal per line and optionally preceded by extra options using the same format as the authorized_keys file. (ie from= and command=). This is flexible enough. I can now give multiple principals (or physical persons) access to a specific account by changing a file. I can also restrict access to certain hosts or create force commands for specific principals.

One use case for this could be a webserver where multiple principals(persons) should be able to use the “webmaster” account but at the same time the test/build system should only be allowed to run a certain commands to publish successful builds. Lets say that the account name is www, then the /etc/ssh/www_principals could look something like this:

peter erik from="buildserver.corp.com" command="/bin/publish_website"
buildserver from="guestcomputer.corp.com" guestworker

Please note: If you are really concerned with security, maybe you have given a certificate to a external partner or something, I would suggest to burn the from and command attributes into the certificate. In this way they will never be overridden by some configuration at the server side. The downside is that you will need to produce a new certificate if something changes.

If you have a pretty static setup and/or a decent configuration manager/orchestration tool this could be enough. It gives full flexibility on who should be able to access what, and how. But in the long run it could be tedious to manage all the principal files. This is where the AuthorizedPrincipalsCommand comes in to the picture.

AuthorizedPrincipalsCommand

This works exactly the same as AuthorizedPrincipalsFile but instead of a static file sshd will run a command followed by some options (basically the username that tries to log in) that will generate the principals file dynamically. This gives you a lot of options. Probably the most straight forward one is that you now can have a single ACL file for you whole environment and just let the command read it and produce a host and user specific principal file. One very simple example could look like this:

# Principal(person) user     host
erik                www      webserver.corp.com
peter               www      webserver.corp.com
guestworker         www      webserver.corp.com

peter               db       database.corp.com
erik                db       database.corp.com

Of course this file could be expanded to include more information and more options but this gives a example on how it can be done.

Other backends

But I think the real power in the AuthorizedPrincipalsCommand is that you now can use whatever backend you like and just have the principals command be a wrapper for this backend which could be some Active Directory, LDAP or whatever you might have at your organisation. This makes it possible to use existing infrastructure and still be able to use ssh certificates which I think is a real killer feature in OpenSSH.

If you have some ideas to improve this concept or an questions, please leave a comment.