Building high-availability cluster using ucarp; two public ips rotated among two machines using private IPs for rotation
Idea:
Lets suppose, there is a service for ex. smtp.abhijeetr.com running and it resolves to two ips. This might be done to load balance among the two ips. But, the issue arises when one of two servers goes down.
We call these machines as a.a.a.a and b.b.b.b. They have private ips as m.m.m.m & n.n.n.n respectively and the public ips on them are p.p.p.p & q.q.q.q.
When a user tries to connect to smtp.abhijeetr.com, it can resolve to either p.p.p.p or q.q.q.q. What if one of the p.p.p.p or q.q.q.q goes down? In that case, all the user that will get the resolved IP as the host which is down will fail to connect to the smtp proxy. And this thing cannot be fixed soon as the DNS is at work and they generally don't have small TTLs. If suppose, q.q.q.q gets down, there should be some way to bind q.q.q.q to the Host A itself. This way, even if the Host B is down, all the users will connect to Host A and there will be no downtime.
Requirements:
Using local IPs, we should be able to rotate public ips as and when a machine goes down or up. Also, we should be able to assign the priority on the binding of public IP.
All the guides that I find on the internet teach on how to float one public ip among the two machines. To do the same thing with two public ips, we just need to run two public IPs.
Our private IPs are running on eth0 and we'll bind the public IPs on eth1 as eth1:0 and eth1:1. We need to make sure that eth1 is up already by issuing two commands.
/sbin/ifconfig eth1 up
/sbin/ifconfig eth1 0.0.0.0
/sbin/ifconfig eth1 promisc
Run these commands on the HOST A,
/usr/sbin/ucarp -v 1 -p love -s m.m.m.m -a p.p.p.p -i eth1:0 -k 1 -P -b 1 --upscript=/etc/vip-up.sh --downscript=/etc/vip-down.sh
/usr/sbin/ucarp -v 2 -p love -s m.m.m.m -a q.q.q.q -i eth1:1 -b 3 --upscript=/etc/vip-up.sh --downscript=/etc/vip-down.sh
The first command when started will make sure that public IP p.p.p.p will be binded to eth0:1 and the second command makes sure that q.q.q.q will be binded to eth1:1.
The option -P makes sure that this will be the preferred master and if there is other machine that is running ucarp and binding the p.p.p.p IP, it will take that p.p.p.p from that machine bind it to itself.
The option -v assigns the vhid which is simply used to identify which ucarp instances will work together. As the two ucarp instances I started are to handle two different IPs, they will have different IDs.
Similarly, on host B, run these commands:
/usr/sbin/ucarp -v 1 -p love -s n.n.n.n -a p.p.p.p -i eth1:0 -b 3 --upscript=/etc/vip-up.sh --downscript=/etc/vip-down.sh
/usr/sbin/ucarp -v 2 -p love -s n.n.n.n -a q.q.q.q -i eth1:1 -k 1 -P -b 1 --upscript=/etc/vip-up.sh --downscript=/etc/vip-down.sh
If there are two instances of ucarp to bind a particular public ip, the agent with lower value of -k + -b will be given the master.
The commands vip-up.sh and vip-down.sh are just the script to up/down the interface.
vip-up.sh:
#! /bin/sh
ifconfig $1 $2 netmask netmaskip
route add default gw gatewayip eth1
vip-down.sh:
#!/bin/bash
/sbin/ifconfig eth1 0.0.0.0
ifconfig $1 | grep 'inet addr'
if [[ $? == 0 ]];then
/sbin/ifconfig $1 down
fi
You can read more about the options on the github page of the project.
By the above setup, if one of the hosts goes down, the public IP corresponding to that host will be automatically binded to the other host so that all the public IPs are attached to the working host machine at all times. Ain't that some good shit ?! :)
Running ucarp as a daemon
I am feeling lazy to write down the conf files for ucarp. If anyone needs that please write a comment and I'll add the corresponding config files.