Adopt Cloud and DevOps

Writing Efficient Infrastructure Tests with Serverspec

Learn how to use Ruby looping and hash maps to add infrastructure tests to your configuration management repositories, without writing duplicative code

by Dave Tashner

One of the core tenants of infrastructure as code is testability; your infra code should be covered with unit and integration tests just like your application code and those tests should be run early and often. This becomes increasingly important as continuous delivery pipelines begin to take shape. Being able to commit an infrastructure change to source control and have immediate (or near immediate) and automatic validation that the new changes are functional is critical as code changes move closer to production.

Serverspec is a great tool for verifying the end state of your infrastructure and is commonly used in conjunction with Test Kitchen to help automate pipelines by launching a test instance upon each commit and executing your configuration management tool of choice (Chef, Puppet, Ansible etc), and then running tests against the newly configured instance to make sure everything’s kosher. If all tests pass, then the infrastructure code is able to move forward in the pipeline. In this short post, I’m going to provide a few examples of ways to write more concise Serverspec tests. Based on the popular Ruby testing framework RSpec, Serverspec allows you to write tests like so:

example_spec.rb

require 'spec_helper'

# Verify packages
describe package('httpd') do
  it { should be_installed }
end

# Verify Gems
describe package('chef') do
  it { should be_installed_by('gem').with_version('12.5.1') }
end

# Verify files
describe file('/var/www/html/index.html') do
  it { should be_file }
  it { should be_owned_by 'apache' }
  it { should be_grouped_into 'apache' }
  it { should be_mode 755 }
  it { should contain('Hello World!').before(/^end/) }
end

# Verify services
%w{httpd jenkins}.each do |svc|
  describe service(svc) do
    it { should be_running }
  end
end

# Verify ports
%w{80 8080}.each do |ports|
  describe port(ports) do
    it { should be_listening }
  end
end
With the supporting helper file:

 

spec_helper.rb

 


 ```require 'serverspec'

set :backend, :exec

RSpec.configure do |c|
  c.before :all doc.path = '/opt/chef/embedded:/opt/chef/embedded/bin:/sbin:/usr/sbin:/bin:/usr/bin'
  end
end
These tests are great for verifying the end state of your Chef-configured servers and building confidence in your server builds across environments. But what if you want to verify a lot of packages or gems AND their versions? You could do something like this:

 

example_spec.rb

 


 ```require 'spec_helper'

# Verify packages
describe package('git') do
  it { should be_installed.with_version('1.8.3') }
end

describe package('httpd') do
  it { should be_installed.with_version('2.4.6') }
end

describe package('jenkins') do
  it { should be_installed.with_version('1.633') }
end

# Verify Gems
describe package('chef') do
  it { should be_installed_by('gem').with_version('12.5.1') }
end

describe package('bundler') do
  it { should be_installed_by('gem').with_version('1.10.7.depsolverfix.0') }
end
But this just feels inefficient. What if I don't want to duplicate package and gem Serverspec code for every package or gem I might want to verify? Let's try using loops instead:

 

example_spec.rb

 

require 'spec_helper'

# Define packages
packages = {
  'git' => {
    version: '1.8.3'
  },
  'httpd' => {
    version: '2.4.6'
  },
  'jenkins' => {
    version: '1.633'
  }
}

# Verify packages
packages.each do |name, details|
  describe package(name) do
    it { should be_installed.with_version(details[:version]) }
  end
end

# Define gems
gems = {
  'bundler' => {
    type: 'gem',
    version: '1.10.7.depsolverfix.0'
  },
  'chef' => {
    type: 'gem',
    version: '12.5.1'
  }
}

# Verify Gems
gems.each do |name, details|
  describe package(name) do
    it { should be_installed.by(details[:type]).with_version(details[:version]) }
  end
end
Cool! At least now, we're looping through a hash of a certain thing that we want to test (a hash of packages with the local packages variable and a hash of gems with the local gems variable). We can quickly and easily add new gems and packages to our hash without adding any new Serverspec code, but it still feels wrong...it seems like we have too many details about our gems and packages inside our executable test file. Let's split the details out of the tests into the spec_helper:

 

spec_helper.rb

 

require 'serverspec'

set :backend, :exec

RSpec.configure do |c|
  c.before :all doc.path = '/opt/chef/embedded:/opt/chef/embedded/bin:/sbin:/usr/sbin:/bin:/usr/bin'
  end
end

# Define packages
Packages = {
  'git' => {
    version: '1.8.3'
  },
  'httpd' => {
    version: '2.4.6'
  },
  'jenkins' => {
    version: '1.633'
  }
}

# Define gems
Gems = {
  'bundler' => {
    type: 'gem',
    version: '1.10.7.depsolverfix.0'
  },
  'chef' => {
    type: 'gem',
    version: '12.5.1'
  }
}
Notice that the variables can no longer be local Ruby variables: we need these objects to be available across files, so we're setting them as constants instead by capitalizing the variable name (Packages and Gems). These constants will be available to the example_spec.rb file because we are already requiring spec_helper at the top of the test file. Now, our test file can be consolidated like so:

 

example_spec.rb

 

require 'spec_helper'

# Verify packages
Packages.each do |name, details|
  describe package(name) do
    it { should be_installed.with_version(details[:version]) }
  end
end

# Verify Gems
Gems.each do |name, details|
  describe package(name) do
    it { should be_installed.by(details[:type]).with_version(details[:version]) }
  end
end

# Verify files
describe file('/var/www/html/index.html') do
  it { should be_file }
  it { should be_owned_by 'apache' }
  it { should be_grouped_into 'apache' }
  it { should be_mode 755 }
  it { should contain('Hello World!').before(/^end/) }
end

# Verify services
%w{httpd jenkins}.each do |svc|
  describe service(svc) do
    it { should be_running }
  end
end

# Verify ports
%w{80 8080}.each do |ports|
  describe port(ports) do
    it { should be_listening }
  end
end
Notice the references to our constants Packages and Gems on lines 4 and 11. Now, the details about our tests are kept separate from the actual execution. This is clean and shiny, just like we want when whipping up awesome in the Chef's kitchen.

 

To run these tests with Test Kitchen, install Xcode command line tools, Git, Vagrant, VirtualBox, and the test-kitchen and kitchen-vagrant Ruby Gems, then clone my Git repository here: https://github.com/singlestone/cookbooks.git

Git repository

Change directories into cookbooks/serverspec_example and then run kitchen verify:

kitchen verify

After Test Kitchen runs (may take a little while to pull down the CentOS image), you should see an output similar to this:

Test Kitchen output

You can also visit localhost:9000 and localhost:9001 in your browser to see Jenkins and Apache respectively.

Jenkins

To execute these tests without Test Kitchen, I would install the bundler RubyGem on the test instance, then create a simple Gemfile like so:

Gemfile

 

source 'https://rubygems.org'

gem 'serverspec'

 

Then run the command 'bundle install'. Once complete, run 'serverspec-init' and select the *nix and local options. Copy your tests into the spec/localhost directory and copy the helper file into the spec directory, then run 'rake spec'. The output will be similar to this:

output.log

 

/opt/chef/embedded/bin/ruby -I /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/rspec-support-3.1.2/lib:/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/rspec-core-3.1.7/lib 

/opt/chef/embedded/lib/ruby/gems/2.1.0/gems/rspec-core-3.1.7/exe/rspec --pattern spec/localhost/\*_spec.rb

Package "git"
  should be installed

Package "httpd"
  should be installed

Package "jenkins"
  should be installed

Package "bundler"
  should be installed

Package "chef"
  should be installed

File "/var/www/html/index.html"
  should be file
  should be owned by "apache"
  should be grouped into "apache"
  should be mode 755
  should contain "Hello World!"

Service "httpd"
  should be running

Service "jenkins"
  should be running

Port "80"
  should be listening

Port "8080"
  should be listening

Finished in 1.35 seconds (files took 1.26 seconds to load)
14 examples, 0 failures

 

Learn more about our DevOps and Cloud solutions.

Dave Tashner
Dave Tashner
Senior Consultant
Contact Dave

Related Articles