Or... has Nomad made the Consul-template tool obsolete?

If you employ Consul or Vault to provide service discovery or secrets management to your applications you will love the freshly released 0.5 version of the Nomad workload scheduler: it includes a new 'template' feature to dynamically generate configuration files from Consul and Vault data for the jobs it runs. Bundling Consul-template as a sidecar to your application is no longer necessary.

Nomad, Consul and Consul-template

A year ago Nomad 0.2 added support for automatic registration of jobs in Consul via a service configuration block. However the applications themselves still had to handle reading data from Consul. For this you had the following three options:

  1. adding code to the application to query Consul's HTTP API.
  2. configuring the runtime environment to forward requests for *.consul to Consuls DNS interface.
  3. adding a sidecar tool to the deployment that will query Consul and feed the results to the application via regular configuration files.

Option 1 certainly is effective and light weight, but may not be feasible if source code is unavailable. And even if it is, you may not want to tie the application to Consul.

Option 2 helps existing applications to obtain healthy endpoints without any modification. They can simply query endpoint-name.service.consul via DNS. But this mechanism does not support Consul's Key-Value store. Neither can it react to changes in these values or in service availability.

This has made option 3 a popular alternative, by means of Consul-template.

Consul-template is a daemon that watches for changes in Consul's Key-Value store or Service Catalog. Upon a change it will (re)render configuration files with these values and one or more templates (hence its name). Optionally it will then signal or restart applications. Likewise it can lease passwords and other credentials from Vault and inject them into the configuration too.

Using Consul-template in 2 minutes

So Consul-Template makes it possible to leverage the power of Consul service discovery and health monitoring and the dynamic secrets management of Vault without having to change a single line of code in your (or a third party's) application. Your application doesn't even know it is getting its configuration from Consul and Vault.

As an example, consider a template file /etc/app.conf.ctmpl containing:

{{range service "backend-service"}}
server {{.Name}} {{.Address}}:{{.Port}}{{end}}

And running consul-template as:

$ consul-template -template=/etc/app.conf.ctmpl:/etc/app.conf -exec=/bin/my-app -exec-reload-signal=SIGHUP

Then consul-template could render my-app's confguration as:

server backend1 10.0.0.10:8888
server backend2 10.0.0.20:9999

And it would send my-app a HUP signal every time the configuation changed (e.g. due to new backend instances starting or existing ones becoming unhealthy).

Consul-template's drawback

Great, with Consul-template your application can be fully unaware and independent of Consul or Vault. But now your application deployment has become more complicated: a new tool needs to be installed, together with configuration templates. Luckily with Docker this is solved all too easily:

  1. add Consul-template and the configuration templates to your container image
  2. change the entrypoint to run consul-template instead of your application.
  3. done!

True, but you just have moved the problem from the application to the container: now the Docker image has become dependent on Consul. Can you do better? Can you keep Consul-template out of your container images and likewise use other people's containers without modification?

Sure!

From Consul ignorant applications to Consul ignorant Containers

You already guessed it: just do the move one more time. Instead of putting the Consul integration into the deployment package (read: Docker image), move it into the deployment itself. And with Nomad 0.5 this has become super easy: simply include a template section to the Nomad job specification with the same, familiar templating syntax:

job "my" {
  group "app" {
    task "server" {
      template {
        data = "{{range service \"backend-service\" }}server {{.Name}} {{.Address}}:{{.Port}}\n{{end}}"
        destination   = "local/app.conf"
        change_mode   = "signal"
        change_signal = "SIGHUP"
      }
    }
  }
}

Note that app.conf now resides in the local directory. The reason for that is that Nomad will render the configuration file before the Docker container starts. Therefore the container filesystem does not even exist yet. Nomads Docker driver mounts the local directory as /local, so configuring my-app to read /local/app.conf as config will solve the issue.

Also noteworthy is that the same mechanism works with Exec and Java driver jobs as well. You can run a naked Go binary without needing Docker at all. An example of that can be found here: https://github.com/bastiaanb/devoxx-2016/blob/master/jobs/helloworld.nomad

How does Nomad implement this?

Nomad has been written in the Go language. One of its nice features is easy reuse of source code from other projects. Therefore, rather than reimplementing Consul-Template, Nomad's developers simply have included it. In short: "Consul-Template (the tool) is dead! Long live Consul-Template (the library)"