3 minute read

AAP support using signed SSH certificate as machine credential. Here are example instructions setting it up.

Signing certificates and distrubte CA public key

In this example, we are going to use my SSL certificate. First, we need to extract the public key from the certificate:

$ openssl x509 -pubkey -in automate.nyc.crt -noout > x509-key.pub

Then, we need to convert it to OpenSSH public key

$ ssh-keygen -i -m pkcs8 -f x509-key.pub > automate.nyc.pub
$ ls -l
total 16
-rw-r--r--. 1 echong echong 1927 Feb 13 10:19 automate.nyc.crt
-rw-------. 1 echong echong 1675 Feb 13 10:19 automate.nyc.key
-rw-rw-r--. 1 echong echong  381 Feb 13 10:25 automate.nyc.pub
-rw-rw-r--. 1 echong echong  451 Feb 13 10:21 x509-key.pub
$ cat automate.nyc.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDj9HfV3SAlTWVdDYjts7nF0+16....

Next, we will generate another set of SSH keys and signed it with our certificate key:

$ ssh-keygen -C 'signed with automate.nyc.crt' -f my_ssh
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in my_ssh.
Your public key has been saved in my_ssh.pub.
The key fingerprint is:
SHA256:5IqjQ/lo5zCDaIjkEGsIg7nIYjTU+z72F27a8C0k91g signed with automate.nyc.crt
The key's randomart image is:
+---[RSA 3072]----+
| ..              |
|o. .             |
|*o  .   .        |
|*=..   o         |
|B= ..   S        |
|@.o  o o + E     |
|=++o+ ..= =      |
|. +=o=  +*..     |
| ..+o oo+o..     |

The key ID will be echong_ssh_automate_nyc and valid for 52 weeks for username echong only. To add multiple usernames/principals, use a comma separated list.

$ ssh-keygen -s automate.nyc.key -I 'echong_ssh_automate_nyc' -V +52w -n echong my_ssh
Signed user key my_ssh-cert.pub: id "echong_ssh_automate_nyc" serial 0 for echong valid from 2023-02-13T10:50:00 to 2024-02-12T10:51:05
$ ls -l
total 28
-rw-------. 1 echong echong 1927 Feb 13 10:19 automate.nyc.crt
-rw-------. 1 echong echong 1675 Feb 13 10:19 automate.nyc.key
-rw-rw-r--. 1 echong echong  381 Feb 13 10:27 automate.nyc.pub
-rw-------. 1 echong echong 2675 Feb 13 10:29 my_ssh
-rw-r--r--. 1 echong echong 1711 Feb 13 10:51 my_ssh-cert.pub
-rw-r--r--. 1 echong echong  582 Feb 13 10:29 my_ssh.pub
-rw-rw-r--. 1 echong echong  451 Feb 13 10:21 x509-key.pub
$ ssh-keygen -L -f my_ssh-cert.pub 
        Type: ssh-rsa-cert-v01@openssh.com user certificate
        Public key: RSA-CERT SHA256:5IqjQ/lo5zCDaIjkEGsIg7nIYjTU+z72F27a8C0k91g
        Signing CA: RSA SHA256:Spn7YWfSWItTuSbzp+0zJKxBSTnYLpcRIcvtNrt9NBE (using rsa-sha2-512)
        Key ID: "echong_ssh_automate_nyc"
        Serial: 0
        Valid: from 2023-02-13T10:50:00 to 2024-02-12T10:51:05
        Critical Options: (none)

Now we need to distribute the CA pub key to all the hosts will be accepting this SSH key. Copy the signing SSH public key string, in our case it will be the content of the converted automate.nyc.pub. On the target hosts, update the /etc/ssh/sshd_config with parameter TrustedUserCAKeys point to the public keys file:

TrustedUserCAKeys /etc/ssh/ca.pub

where the ca.pub file holds all the CA pub keys, one per line. Restart sshd to update the configuraiton. This will allow all users/principles listed in the signed SSH key to login the the host. The CA public key can also assined per user by entering into user’s ~/.ssh/authorized_keys file.

Setup AAP machine credential

On AAP machine credential page, paste the content my_ssh private key to the SSH Private Key and the content of my_ssh-cert.pub to the Signed SSH Certificate text boxes. If the my_ssh key is generated with a passphrase, enter the Private Key Passphrase text box too. AAP Machine Credential

When using this credential to connect to the target host, the following will be logged in syslog:

Feb 13 11:26:32 aap-db1.lab.automate.nyc sshd[109844]: Accepted publickey for echong from port 53348 ssh2: RSA-CERT SHA256:5IqjQ/lo5zCDaIjkEGsIg7nIYjTU+z72F27a8C0k91g ID echong_ssh_automate_nyc (serial 0) CA RSA SHA256:Spn7YWfSWItTuSbzp+0zJKxBSTnYLpcRIcvtNrt9NBE

Noticed the Key ID echong_ssh_automate_nyc we used to sign the SSH key, the thumbnails of the ssh key and signing key are logged in the message.


These are where the files go:

File Description Location
automate.nyc.pub The public key generated from signing certificate /etc/ssh/ca.pub or ~/.ssh/authorized_keys
my_ssh My private key Copy the content to AAP SSH Private Key text box
my_ssh-cert.pub My signed public key Copy the content to AAP Signed SSH Certificate text box
my_ssh.pub My public key No use
x509-key.pub The intermediate certificate public key No use


AWX PR for implementing SSH signed key

Red Hat doc on signing SSH key

Using HashiCorp Vault

Extracting SSH pub key from SSL certificate

How Facebook does it

Leave a comment