PuppetDB and puppetdbquery offer a lot of power to dynamically generate configuration files. Here I provide an example use case with HAproxy systems running in a geographically dispersed system.

First off, if you use PuppetDB and haven't started using Puppetdbquery then now is the time to check it out. We will be using its functions inside of our Puppet manifest to gather information to act on inside our haproxy.cfg.erb

Data I am working with to make this happen:

  • Consistent FQDNs of global systems
  • Facter values $hostname, $domain,
  • All servers across multiple geographic regions report data back to a centralized PuppetDB

Most of my automation work relies on solid FQDN naming conventions. This is the root of system identification in a lot of cases and it needs to be consistent and straight forward. For our purposes here all web servers will follow this naming convention:

FQDN Structure

<component><numid>.<geolocation>.<provider>.<domain>.com  

Examples:

web1.chicago.linode.autozane.com  
web2.chicago.linode.autozane.com  
web1.nyc.linode.autozane.com  
web2.nyc.linode.autozane.com  
web1.sf.linode.autozane.com  
web2.sf.linode.autozane.com  
web1.london.linode.autozane.com  
web2.london.linode.autozane.com  

As web servers spin up in each region ie, Chicago, NYC, SF, London, the HAproxy systems need to be aware of new or removed web systems and Puppet must update the haproxy.cfg and reload HAproxy accordingly.

Somewhere in the HAproxy .pp mainfests that house the system configuration logic we need to gather an array of systems for that particular region. Here is where Puppetdbquery comes into action.

$domainhosts = query_nodes("domain='$domain'", hostname)

This will query PuppetDB during a puppet catalog compilation and return an array of hostnames for all systems in the domain. For example, during a puppet run on lb1.london.linode.autozane.com I will get an array of all servers in london.linode.autozane.com ($domain facter fact) and the results in the array will be the hostname values for these hosts.

$domainhosts = ['web1', 'web2', 'web3', 'redis1', 'mongo1', 'monitor1', 'mysql1', 'vpn1' ]

With this array I can now use it in an ERB for haproyx.cfg

Here is what I need to add to the 'backend' section of the haproxy.cfg.erb template

backend autozane  
<% @webhosts.sort.each do |host| -%>  
<% if host =~ /^web/ ; host.gsub!(/\-.*/, '')-%>  
        server <%=host%> <%=host%>:8080 maxconn 10 check inter 5s
<% end -%>  
<% end -%>  

If we had 5 web hosts in London then the configuration would generate...

backend autozane  
     server web1 web1:8080 maxconn 10 check inter 5s
     server web2 web2:8080 maxconn 10 check inter 5s
     server web3 web3:8080 maxconn 10 check inter 5s
     server web4 web4:8080 maxconn 10 check inter 5s
     server web5 web5:8080 maxconn 10 check inter 5s

This will use the $webhosts data we collected in the .pp from puppetdbquery and pull out the 'web' hosts using a regular expression.

Drop a 'notify' on the Puppet file resource used to manage the HAproxy service in order to gracefully reload HAproxy upon dynamic configuration file updates.

notify => Service['haproxy'];  

This was a quick example that could be modified or used in other data templating situations where you have PuppetDB information available to leverage.