This article is the first in a series about testing Puppet manifests. It will cover the basics such as
puppet-lint and Smoke Testing and detailed testing with
- Why Test?
- Testing Puppet
For systems administrators, testing usually involves configuring something on a test server before it's done on a production server. It's understandable, then, why the development world of testing (test driven development, unit tests, behavior driven development, etc) can seem very foreign.
Take, for example, applying the "sysadmin" method of testing to Puppet manifests: one would write a set of Puppet manifests, provision a test server, then apply the manifest and watch for errors. If there were any, the manifests would be fixed and the test deployment would be re-run.
This kind of testing can be very time consuming. Further, it is only verifying that the manifests are valid for one type of server.
By utilizing methods from the development world, not only can the testing cycle be simplified, but also automated and categorized. Multiple tests can even be created to validate the same set of manifests against different types of servers.
Yes, these methods might seem foreign at first, but if you give them a chance, you'll find that your configuration management system is a lot more maintainable.
There are two main ways to test Puppet manifests for errors.
The easiest and most basic way is known as Smoke Testing. I'm sure lots of people have used this method without even knowing it was an official way of testing.
Smoke Testing involves writing a basic manifest, then applying that manifest with the
--noop option. That's it.
As an example, let's say you were writing a manifest to configure Apache on a server. It might look something like this:
Now instead of applying this manifest as-is, try it with
Notice the caught error.
serveralias is not a correct parameter,
Smoke Testing is a very low-cost form of testing and I highly recommend utilizing it.
RSpec is a more advanced form of testing. It provides (yet another) Ruby DSL that describes how code should behave. RSpec takes a lot longer to set up and learn than simple Smoke Testing, but if you're managing a large and complex module, it's very useful and worth the effort.
The rspec-puppet project is an extension of RSpec that makes testing Puppet manifests with RSpec easier.
The easiest way to get started with
rspec-puppet is to follow this blog entry. It covers how to install and configure
rspec-puppet by using the
puppetlabs_spec_helper gem. It also walks through a basic manifest and how to write tests for that manifest.
The rspec-puppet project page also has a helpful tutorial.
I wanted to include an example because when I started learning RSpec (and I'm still very much a beginner), any example I could find was helpful. Just trying to contribute back.
This example uses my puppet-reprepro module.
For this example, tests will be created for an existing manifest. While testing methodologies state that you should write the tests first, in my opinion, this rarely happens. Keeping up with that tradition, this is how I would write an initial set of tests for a manifest. Once the initial set is written, more tests can continue to be added to account for future features.
I'll use the distribution.pp manifest. It's a small manifest that can easily highlight most features of
First, inventory the resources in the manifest. In
distribution.pp, there are the following resources:
Next, check for conditionals. In
distribution.pp, there are two.
When testing conditionals, make sure you test all logical branches.
Next, check for any classes that are included in this manifest. There are two in
Finally, review the parameters, their defaults, and any facts that might need to be set (for example, the Operating System or system architecture).
With everything inventoried, make note of any modules that the manifest uses. These modules need to be added to the
In this case, the
concat module is used, so the following is added to
Once the fixtures are in place, create the actual RSpec test file. This file will called
Next, I create a hash for the default parameters:
The hash is first built by using the default values of the Class Parameters. Next, Required Parameters are added with values that can be globally used for all tests.
There's one caveat to this, though. Note line 53.
$basedir is supposed to pull its default value from the
reprepro::params class. To my knowledge,
rspec-puppet does not support this, so instead, just hard-code the value in the default params:
A context is a group of related tests. For example, tests related to a specific Operating System. In this example, the first context will be tests involving valid results for when all default values are used.
All I usually do is check and see if all resources that were inventoried earlier are successfully applied:
A few things to note about this. The first is
Merging new parameters into the Default Parameters is an easy way for multiple tests to re-use a standard set of parameters and modify them as needed.
In this case,
:codename are being set. I did this here because these parameters would not be known until the title of the
reprepro::distribution is known:
:title could have all been set in the default parameters.
The next area to note is
Remember I mentioned that we'll be testing the different branches of each conditional. By default, the
$update parameter is
'', which should translate to
false and so the resulting catalog will not contain the
A default value of
'' is probably not the best choice. Setting
false would have probably been better. This brings up an interesting point about writing tests for an existing manifest: you're able to find areas in the code that can be done better. This isn't the best time to change this value, though. It'd be better to finish the tests and then go back fix both the manifest and test later.
The Second Context
For the second context, I check to see what happens when
$update is set to
This context only checks for the one resource. This is because everything else in the first context still applies. No sense in repeating code.
A Note About Facts
If you save this file and run:
You'll see an error about
concat_basedir not being set. This is because the manifest is using the
$concat_basedir in the
default_params hash raises another error because
$concat_basedir does not exist as a parameter in
distribution.pp. To get around this error, I did the following at the top of the rspec file:
Maybe there's a better way to do this.
With all of this in place, you should have a
distribution_spec.rb file that looks like this.
You can run the tests by doing:
puppet-lint against your Puppet manifests is a great way to make sure you're writing Puppet with best practices in mind.
Next, add the following to your
Now you can run
puppet-lint by doing:
This concludes Part 1 of Puppet Testing. This part covered the benefits of testing, basic Puppet testing with Smoke Tests, advanced testing with
rspec-puppet, and finally, best practice checking with
I should note that I'm still very new to testing. If there are better ways to do any of the above, I would love to hear them.