Sharing SSH Keys with PuppetDB

Introduction

This article will explain how to easily share SSH public keys in a Puppet environment using PuppetDB and the PuppetDB Query module.

Introduction
Exported Resources
- Enabling Exported Resources
SSH Keys and Exported Resources
PuppetDB Query
SSH Keys and PuppetDB
- Public Key Creation and Management
Conclusion

Exported Resources

Exported Resources have been a long-time feature of Puppet. They allow one node to create a resource using information local to that node and then export it. Another node can then import (or collect) and apply the resource.

The most practical use of this is the Nagios example that is explained in the given link.

Enabling Exported Resources

To enable Exported Resources, a database has always been required. Previously, this was accomplished using Puppet's Stored Configuration system. Most recently, though, PuppetDB is the new standard.

Side Note: Along with Exported Resources, Stored Configuration and PuppetDB also store nodes' facts.

SSH Keys and Exported Resources

Another common use of Exported Resources is to help manage and distribute SSH keys. The de facto way of doing this is described in Puppet's wiki.

There are several modules that have been built around this system. The one I used for the longest time was boklm's module.

While this is an excellent module, using it effectively requires multiple steps to happen before a key is actually shared on another node:

Node 1 exports a key creation resource to the Key Master.
The Key Master creates the key.
Node 1 adds the key to a specified user's home.
Node 2 adds the key to a specified authorized_keys file.

Steps 1 and 2 take two Puppet runs to accomplish. Steps 3 and 4 can be done when Node 1 and Node 2 perform another Puppet run asynchronously. So a total of 3 Puppet runs are needed to complete the above steps. If Puppet runs every 30 minutes, it can take up to 90 minutes to complete.

PuppetDB Query

I recently found an excellent Puppet module that makes querying PuppetDB extremely easy.

As the README shows, Puppet manifests can now query PuppetDB. This opens a lot of possibilities for inter-Puppet communication. For example, rather than having all web servers export a Nagios check to the Nagios server, you can now have the Nagios server query for all nodes with Apache installed and then create the Nagios check. This is even more easier if you're using Puppet 3.2 and the new iteration feature:

$nodes = query_nodes('Class[Apache]')
$nodes.each { |$node|
  nagios_service {
    ...
  }
}

SSH Keys and PuppetDB

With the PuppetDB Query module available, I wanted to create an easier way to share SSH public keys between servers. I have a very early solution available here.

It uses a custom Fact to expose public keys found in the .ssh directory of a node:

$ facter -p sshpubkey_root
$ facter -p sshpubkey_jtopjian

With the public key exposed, it'll then be stored in PuppetDB. Once that happens, any other node can add it to an authorized_keys file:

sshkeys::set_authorized_key { 'root@server1 to root@server2':
  local_user  => 'root',
  remote_user => 'root@server1',
  home        => '/root',
}

Internally, sshkeys::set_authorized_key is doing this:

$parts = split($remote_user, '@')
$remote_username = $parts[0]
$remote_node     = $parts[1]

$results = query_facts("fqdn=\"${remote_node}\"", ["sshpubkey_${remote_username}"])

$key = $results[$remote_node]["sshpubkey_${remote_username}"]

ssh_authorized_key {
  ...
}

Public Key Creation and Management

It's important to note that unlike the "standard" Puppet system of managing SSH keys, this method does not store keys on a centralized server. I chose to do this for two reasons:

To reduce the time / Puppet runs when a key can be available.
Having Puppet control SSH keys can interfere with provisioning/bootstrapping/backup/restore processes.

The module I created has a helper function to create a key, but you can just as easily call ssh-keygen yourself or even copy over an existing key.

Conclusion

Discovering the PuppetDB Query module has opened up a lot of new possibilities to how I can use Puppet. Sharing SSH keys, as described, is really only the tip of the iceberg.

As mentioned, my sshkeys module is still very early. If you have any suggestions on how to improve it, please let me know or submit a pull request.

terrarum