LVS : configure DSR ( Direct Server Return )

Here are logs when configuring LVS direct server return.

     Client

       |

     L2SW

|eth0      | eth0

LVS      L3SW

|eth1      |eth1

     L2SW

       |

     Server ( httpd )

request packets : client -> LVS -> Server

reply packets : server -> L3SW -> Client

Default GW of http server : L3SW eth1 ( not LVS’s eth1 )

[ L3SW ]

I used CentOS6 as L3SW.

disable rp_filter and enable ip_forward.

# echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter # echo 1 > /proc/sys/net/ipv4/ip_forward

[ LVS ]

install LVS(ipvsadm) over yum

[root@centos7-lvs ~]# cat /etc/centos-release CentOS Linux release 7.1.1503 (Core) [root@centos7-lvs ~]# yum install -y ipvsadm

[root@centos7-lvs ~]# echo 1 > /proc/sys/net/ipv4/ip_forward [root@centos7-lvs ~]# echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter

add TCP 80 load balancing rule.

[root@centos7-lvs ~]# ipvsadm -A -t 192.168.122.43:80 -s rr [root@centos7-lvs ~]# ipvsadm -a -t 192.168.122.43:80 -r 192.168.150.150::80 –g

--

      [packet-forwarding-method]

             -g, --gatewaying  Use gatewaying (direct routing). This  is  the

             default.

--

[root@centos7-lvs ~]# ipvsadm –Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn TCP  192.168.122.43::80 rr  -> 192.168.150.150::80           Route   1      0          0

I would like to add “VIP” as an alias IP, but how can I do that with nmcl?

I will add VIP with ifconfig command.

[root@centos7-lvs ~]# yum install -y net-tools [root@centos7-lvs ~]# ifconfig eth0:10 192.168.122.43

[ Server ]

on the httpd server,

# iptables -t nat -A PREROUTING -d 192.168.122.43 -j REDIRECT nf_conntrack version 0.5.0 (16384 buckets, 65:806 max)

You don’t need to configure LVS’s VIP on the server.

start the apache

# /etc/init.d/httpd start

LVS : Direct Server Return

Here’s an explanation of how to configure LVS DSR.

[ network topology ]

All machines are running as VMs within Ubuntu 12.04 KVM.

client

  | 192.168.100.10

  |

 vSW

  | 192.168.100.254

Router ( Linux box )

  | 192.168.200.254

  |

vSW  -----eth1 LVS ( VIP 192.168.200.100 , real ip 192.168.200.50 )

  |

 Web*2 ( real ip 192.168.200.10 , real ip 192.168.200.20 )

[ LVS ]

# ipvsadm --version ipvsadm v1.25 2008/5/15 (compiled with popt and IPVS v1.2.1) # keepalived --version Keepalived v1.2.2 (12/23,2011) # tail -1 /etc/lsb-release DISTRIB_DESCRIPTION="Ubuntu 12.04.2 LTS" # uname -ri 3.2.0-39-generic x86_64

enable ip_forward and disable rp_filter.

# egrep 'ip_forward|rp_filter' /etc/sysctl.conf | egrep -v ^# net.ipv4.ip_forward=1 net.ipv4.conf.all.rp_filter = 0

assign VIP

# ip addr add 192.168.200.100 label eth1:vip dev eth1 # ip addr show eth1:vip 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether 52:54:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff    inet 192.168.200.50/24 brd 192.168.200.255 scope global eth1    inet 192.168.200.100/32 scope global eth1:vip

keepalived.conf

# cat keepalived.conf virtual_server_group HTTP 100 {        192.168.200.100 80 } virtual_server group HTTP 100 {        delay_loop 3        lvs_sched rr        lvs_method DR        protocol TCP        virtualhost foo.bar        real_server 192.168.200.10 80 {        weight 1        inhibit_on_failure        HTTP_GET {                url {                        path /index.html                        status_code 200                }                connect_timeout 3        } }        real_server 192.168.200.20 80 {        weight 1        inhibit_on_failure        HTTP_GET {                url {                        path /index.html                        status_code 200                }                connect_timeout 3        } } }

[ Web server ]

On both web servers , configure iptables to redirect packets ( src ip : client , dst ip : 192.168.200.100 ) to its loopback.

iptables -t nat -A PREROUTING -d 192.168.200.100 -j REDIRECT

When you configure redirecting packets with iptables , you do no need to assign VIP address as loopback address.

start apache

# apachectl start

[ ifstat info when downloading 1M byte file from the client ]

LVS’s outbound traffic is very low because outbound traffic does not pass through LVS.

LVS

# ifstat -i eth1 KB/s in  KB/s out    2.41      2.16    1.48      1.48    1.48      1.42    2.48      2.28    0.06      0.06    0.00      0.00    1.06      0.74

Web Server

# ifstat -i eth1 KB/s in  KB/s out    1.01     80.49    1.82    117.66    1.29    112.70    1.35    124.23    1.82    118.88    1.29    141.10    1.54    128.43    1.82    130.26    1.16     79.98    0.06      2.03    0.40      0.20