LVS persistence

LVS has a built-in simple IP based persistence mechanism that can be used to keep users on the same real servers for a configurable amount of time. If your web application requires that each user request to be processed by the same real server then you will probably want to enable this mechanism and ensure that requests coming from the same IP will be directed to the same real server. This article will show how you can achieve this by using regular ipvsadm commands but also by using ldirectord configurations.

IPVS is an advanced IP based load balancing application implemented inside the linux kernel. Working at IP level LVS can’t make a decisions based on the content of the packet. Still, it can perform a basic IP affinity, by keeping all connections from the same source IP directed to the same real server for a configurable amount of time. This is achieved with the -p ipvsadm command parameter and takes as a parameter the time in seconds to keep the connections in the persistence table. From the ipvsadm manual page:

• -p, --persistent [timeout] = Specify that a virtual service is persistent. If this option is specified, multiple requests from a client are redirected to the same real server selected for the first request. Optionally, the timeout of persistent sessions may be specified given in seconds, otherwise the default of 300 seconds will be used. This option may be used in conjunction with protocols such as SSL or FTP where it is important that clients consistently connect with the same real server.

Here is a simple ipvsadm example in a setup with one web server load balanced on 2 real servers using LVS-DR:

ipvsadm -A -t VIP:80 -p 3600 -s -wrr
ipvsadm -a -t VIP:80 -R RS1 -g -w 1
ipvsadm -a -t VIP:80 -R RS2 -g -w 1

by using -p 3600 we request to keep the users coming from the same ip on the same real server as their first connection was handled, for a period of 60 minutes. After this timeout expires any new connection from the same user ip will be routed based on the scheduling policy (Weighted Round-Robin in this case) and can be handled by any of the active servers.

The output of the ipvsadm command will show that the service is persistent:

ipvsadm -L -n
...
TCP  VIP:80 wrr persistent 3600
-> RS2:80              Route   1      0          0
-> RS1:80              Route   1      0          0

I am assuming that not many people are using plain ipvsadm commands and will probably use a frontend like ldirector that will handle health checks, automatic reconfiguration and many other features. The same effect is achieved by using the ldirectord persistent config keyword. The same sample in ldirectord.cf terms will look like:

virtual=VIP:80
real=RS1:80 gate 1
real=RS2:80 gate 1
persistent=3600
...

We can increase the network range used in the persistent match, using -M netmask. From the manual page:

• -M, --netmask netmask = Specify the granularity with which clients are grouped for persistent virtual services. The source address of the request is masked with this netmask to direct all clients from a network to the same real server. The default is 255.255.255.255, that is, the persistence granularity is per client host. Less specific netmasks may be used to resolve problems with non-persistent cache clusters on the client side.

In our sample setup this will look like:

ipvsadm -A -t VIP:80 -p 3600 -s -wrr -M 255.255.255.0
ipvsadm -a -t VIP:80 -R RS1 -g -w 1
ipvsadm -a -t VIP:80 -R RS2 -g -w 1

While for ldirectord this will require the usage of the netmask keyword:

virtual=VIP:80
real=RS1:80 gate 1
real=RS2:80 gate 1
netmask=255.255.255.0
persistent=3600
...

Note: obviously persistence will break the scheduling policy resulting in no longer having exact balancing on the real servers. The bigger the timeout and/or network masks for the LVS persistence, the bigger the effect.

For more details you check the LVS FAQ.