Puppet - our journey from Puppet 3.8 to Puppet 4

the last months we at Globalways managed to upgrade our puppet infrastructure from Puppet 3.8 to Puppet 4.x.

we did that task in small steps:

trusted_node_data = true on Puppet 3.8
stringify_facts = false on Puppet 3.8
set parser = future on Puppet 3.8
add datatypes to our profiles
Puppet 4.6
future plan: strict_variables = true, agent upgrades

we are using a strict profiles and roles layout. our profiles are shared all over the different customer environments.

trusted_node_data = true

this step was easy, we enabled it, and started to use the $facts variable on new code. the $facts variable is also important for our next big step strict_variables = true.

if you have facts that are not available on all your systems you can't use code like this:

if ($foobar == true) {
}

you have to use the has_key from stdlibs:

if (has_key($facts, 'foobar') && $facts['foobar'] == true) {
}

stringify_facts = false

for this setting we updated several modules from the forge to the newest version. also we had to update some of your internal profiles to query boolean facts in the correct way.

to have puppet code that is usable with stringify_facts = false and true, you have to use code like this:

if (str2bool("${::is_virtual}") == true) {
}

parser = future

this step took longer than we expected. we updated several modules from the forge to support the future parser. also we submitted the following PRs:

graylog_collector (#PR7)
keepalived (#PR90)
puppetlabs-mongodb (#PR275)
tac_plus (#tac-PR7)
puppetlabs-tomcat (#tomcat-PR161)
graylog_collector (#PR8)

we had some dependency problems on our internal profiles that was fixed very easy. we enabled the future parser per environment, did our tests and moved to our next environment.

datatypes on our profiles

after we had done the big step forward with the future parser on Puppet 3.8 we really enjoyed the nice possibilities of the future parser.

we started adding data types to all our profiles, this change also ensured that our hiera yamls getting better:

changed 'true' to a boolean true
changed '1337' to a integer value

we removed some of our 'array' defines and replaced it with .each. (That's cool - less code for the same!)

on new written profiles we also heavily using the star operator ( * => $hash) to parse a hash to a Class or Define.

Puppet 4.6

after doing all that work we added a extra server to our Puppet infrastructure with Puppet 4.6 (with Puppet Server) and updated our Git hooks to deploy our manifests to all our Puppet servers (3.8 and 4.x). Now we are able to change every node or environment to the new Puppet 4.x server.

Currently we are using Puppet Agent 3.8 backports on all our servers (mainly Ubuntu 12.04, 14.04, 16.04 and some Jessie boxes).

Update: we are just waiting for Puppet 4.6.1 to fix some variable scoping regressions in Puppet 4.6.0 , to move all servers to Puppet 4.x

Puppet Server

since the Puppet Server is Jruby, we have to manage to get trocla working with JRuby. We are using Sequel with postgres inside trocla. That's why we need to install gems via Puppet Server:

/opt/puppetlabs/bin/puppetserver gem install pg_jruby --no-ri --no-rdoc
/opt/puppetlabs/bin/puppetserver gem install moneta --no-ri --no-rdoc
/opt/puppetlabs/bin/puppetserver gem install sequel --no-ri --no-rdoc
/opt/puppetlabs/bin/puppetserver gem install trocla --no-ri --no-rdoc

we also symlinked /etc/puppetlabs/code/environments to /etc/puppet/environments because our r10k config should work on 3.8 and 4.x. After we switched off the Puppet 3.8 servers we plan to migrate our r10k config to the new location. But thanks to Puppetlabs the symlinks works :-)

Also you should really read the 'Where did everything go in Puppet 4.x?' more than once! :-)

e.g. the troclarc.yaml is now located at /etc/puppetlabs/puppet/troclarc.yaml

strict_variables = true / agent upgrades

Puppet Agent 3.8 can talk to Puppet Server, but not to Puppet 4.x Master (passenger setup). The Passenger setup will be removed at Puppet 5.0, so you have to upgrade to Puppet Server anyway. Do it early it will save you a lot of time.

While using Puppet Agent 3.8 we can't use all the new features of Puppet 4.x because of the older client. We are planing to upgrade to Puppet 4.6 Agents as soon as possible. But we are not going to use the AIO packages, we will use the backports of the Debian Package, because we don't need that bloated package on all our servers. (We don't use PXP or Mcollective, just the Puppet Agent)

After switching the agents to Puppet 4.x we will work on strict_variables = true to ensure that we are ready to upgrade to Puppet 5.0. (We think the update from 4.x to 5.0 will be less pain then 3.x to 4.0)

So folks: DO THE UPGRADE to Puppet 4.x it's worth it!

home