Canary deployment is a way to test a new release of a software rolling it only for a small sub set of users. In this post I'll show how at CERN, in the Middleware section of Database group, we configure  our HAProxy setup to work as canary deployment. I'll give a brief introduction on what is a canary deployment and later we will see how to configure HAProxy.

Canary Deployments

Let's take into account the example of an application that allows to book a flight and suppose that in the new release the developers want to add the possibility to book also a hotel. Sometimes even the best benchmarks or deep testing are not enought to discover all the bugs because we cannot foresee the users behaviour.  The Canary Deployments helps you with this. You can decide to have two different releases running at the same time but allow just a small part of your users to access to new release. The image below shows how it works.

In this case only the 5% of the users will access the Version 2.0 of your Production Application and it will allow you to get feedback from these users and to decide if the new release is stable enough to be used by your whole user community.

Let's see how we can achieve this in HAProxy.

HAProxy Configuration

The configuration is pretty easy and straightforward. Let's suppose that our users will access the application using the myprodurl.cern.ch and that we have 6 servers that are serving the requests.

One server will answer to 5% of the requests and the other will take care of other 95% left.

frontend myprodurl_https

         bind 10.0.0.1:443 ssl crt /etc/pki/full_cert_host.pem

         ....

         ....

         ....

         default_backend backend_myprodurl

 

backend backend_myprodurl

         balance roundrobin

         server Minion0 10.0.0.2:10000 cookie Minion0 maxconn 256 check inter 2000 rise 2 fall 5 weight 19

         server Minion1 10.0.0.3:10000 cookie Minion1 maxconn 256 check inter 2000 rise 2 fall 5 weight 19

         server Minion2 10.0.0.4:10000 cookie Minion2 maxconn 256 check inter 2000 rise 2 fall 5 weight 19

         server Minion3 10.0.0.5:10000 cookie Minion3 maxconn 256 check inter 2000 rise 2 fall 5 weight 19

         server Minion4 10.0.0.6:10000 cookie Minion4 maxconn 256 check inter 2000 rise 2 fall 5 weight 19

         server Minion5 10.0.0.7:10000 cookie Minion5 maxconn 256 check inter 2000 rise 2 fall 5 weight 5

The definition of weight in the haproxy documentation says :  "The weight parameter is used to adjust the server's weight relative to other servers. All servers will receive a load proportional to their weight relative to the sum of all weights, so the higher the weight, the higher the load. The default weight is 1, and the maximal value is 256. A value of 0 means the server will not participate in load-balancing but will still accept persistent connection"

If we take the example above the sum of the weight is 19*5 + 5 = 100. It means that on 100 requests the first 5 minions will serve 19 requests each and the other 5 requests will be served by the 6th node.