lost and found ( for me ? )

lost and found ( for me ? )

↑ Grab this Headline Animator

Showing posts with label LVS. Show all posts
Showing posts with label LVS. Show all posts

LVS : configure DSR ( Direct Server Return )

Here are logs when configuring LVS direct server return.
     Client
       |
     L2SW
|eth0      | eth0
LVS      L3SW
|eth1      |eth1
     L2SW
       |
     Server ( httpd )
request packets : client -> LVS -> Server
reply packets : server -> L3SW -> Client
Default GW of http server : L3SW eth1 ( not LVS’s eth1 )
[ L3SW ]
I used CentOS6 as L3SW.
disable rp_filter and enable ip_forward.
# echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter
# echo 1 > /proc/sys/net/ipv4/ip_forward
[ LVS ]
install LVS(ipvsadm) over yum
[root@centos7-lvs ~]# cat /etc/centos-release
CentOS Linux release 7.1.1503 (Core)

[root@centos7-lvs ~]# yum install -y ipvsadm
[root@centos7-lvs ~]# echo 1 > /proc/sys/net/ipv4/ip_forward
[root@centos7-lvs ~]# echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter
add TCP 80 load balancing rule.
[root@centos7-lvs ~]# ipvsadm -A -t 192.168.122.43:80 -s rr
[root@centos7-lvs ~]# ipvsadm -a -t 192.168.122.43:80 -r 192.168.150.150::80 –g
--
      [packet-forwarding-method]
             -g, --gatewaying  Use gatewaying (direct routing). This  is  the
             default.
--
[root@centos7-lvs ~]# ipvsadm –Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
 -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.122.43::80 rr
 -> 192.168.150.150::80           Route   1      0          0
I would like to add “VIP” as an alias IP, but how can I do that with nmcl?
I will add VIP with ifconfig command.
[root@centos7-lvs ~]# yum install -y net-tools

[root@centos7-lvs ~]# ifconfig eth0:10 192.168.122.43
[ Server ]
on the httpd server,
# iptables -t nat -A PREROUTING -d 192.168.122.43 -j REDIRECT
nf_conntrack version 0.5.0 (16384 buckets, 65:806 max)
You don’t need to configure LVS’s VIP on the server.
start the apache
# /etc/init.d/httpd start

No comments:
Labels: , , , ,

LVS : Direct Server Return


Here’s an explanation of how to configure LVS DSR.


[ network topology ]

All machines are running as VMs within Ubuntu 12.04 KVM.

client
  | 192.168.100.10
  |
 vSW
  | 192.168.100.254
Router ( Linux box )
  | 192.168.200.254
  |
vSW  -----eth1 LVS ( VIP 192.168.200.100 , real ip 192.168.200.50 )
  |
 Web*2 ( real ip 192.168.200.10 , real ip 192.168.200.20 )

[ LVS ]

# ipvsadm --version
ipvsadm v1.25 2008/5/15 (compiled with popt and IPVS v1.2.1)
# keepalived --version
Keepalived v1.2.2 (12/23,2011)

# tail -1 /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 12.04.2 LTS"

# uname -ri
3.2.0-39-generic x86_64

enable ip_forward and disable rp_filter.
# egrep 'ip_forward|rp_filter' /etc/sysctl.conf | egrep -v ^#
net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter = 0

assign VIP
# ip addr add 192.168.200.100 label eth1:vip dev eth1

# ip addr show eth1:vip
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
   link/ether 52:54:xx:xx:xx:xx brd ff:ff:ff:ff:ff:ff
   inet 192.168.200.50/24 brd 192.168.200.255 scope global eth1
   inet 192.168.200.100/32 scope global eth1:vip

keepalived.conf
# cat keepalived.conf
virtual_server_group HTTP 100 {
       192.168.200.100 80
}

virtual_server group HTTP 100 {
       delay_loop 3
       lvs_sched rr
       lvs_method DR
       protocol TCP
       virtualhost foo.bar
       real_server 192.168.200.10 80 {
       weight 1
       inhibit_on_failure
       HTTP_GET {
               url {
                       path /index.html
                       status_code 200
               }
               connect_timeout 3
       }
}
       real_server 192.168.200.20 80 {
       weight 1
       inhibit_on_failure
       HTTP_GET {
               url {
                       path /index.html
                       status_code 200
               }
               connect_timeout 3
       }
}
}

[ Web server ]

On both web servers , configure iptables to redirect packets ( src ip : client , dst ip : 192.168.200.100 ) to its loopback.
iptables -t nat -A PREROUTING -d 192.168.200.100 -j REDIRECT

When you configure redirecting packets with iptables , you do no need to assign VIP address as loopback address.

start apache
# apachectl start

[ ifstat info when downloading 1M byte file from the client ]

LVS’s outbound traffic is very low because outbound traffic does not pass through LVS.

LVS
# ifstat -i eth1
KB/s in  KB/s out
   2.41      2.16
   1.48      1.48
   1.48      1.42
   2.48      2.28
   0.06      0.06
   0.00      0.00
   1.06      0.74

Web Server
# ifstat -i eth1
KB/s in  KB/s out
   1.01     80.49
   1.82    117.66
   1.29    112.70
   1.35    124.23
   1.82    118.88
   1.29    141.10
   1.54    128.43
   1.82    130.26
   1.16     79.98
   0.06      2.03
   0.40      0.20
No comments:
Labels: , , ,

ubuntu 12.04 : configure LVS ( Linux Virtual Server )



client ( 192.168.10.14 ) -- VIP 192.168.10.112 LVS 192.168.30.111 -- Web*2 192.168.30.10 , 20
                          interface IP 192.168.10.225

install LVS
# apt-get install ipvsadm libnl1

enable IP forwarding
# echo 1 > /proc/sys/net/ipv4/ip_forward
# echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter

reflect above settings permanently
# egrep ip_forward /etc/sysctl.conf
net.ipv4.ip_forward=1

assign VIP
# ip addr add 192.168.10.112 label eth0:vip dev eth0
# ip addr show eth0 label eth0:vip
   inet 192.168.10.112/32 scope global eth0:vip

configure LVS with ipvsadm command.
# ipvsadm -A -t 192.168.10.112:80 -s rr
# ipvsadm -a -t 192.168.10.112:80 -r 192.168.30.10:80 -m -w 1
# ipvsadm -a -t 192.168.10.112:80 -r 192.168.30.20:80 -m -w 1
# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
 -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.10.122:80 rr
 -> 192.168.30.10:80             Masq    1      0          0
 -> 192.168.30.20:80             Masq    1      0          0

# ipvsadm -ln --rate
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port                 CPS    InPPS   OutPPS    InBPS   OutBPS
 -> RemoteAddress:Port
TCP  192.168.10.112:80                  57      342      285    25908    31045
 -> 192.168.30.10:80                   29      171      143    12957    15525
 -> 192.168.30.20:80                   29      171      143    12952    15519

cap data between client and LVS
# tshark -i eth0 port 80
Capturing on eth0
 0.000000 192.168.10.14 -> 192.168.10.112 TCP 74 54695 > http [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=289451261 TSecr=0 WS=128
 0.000591 192.168.10.112 -> 192.168.10.14 TCP 74 http > 54695 [SYN, ACK] Seq=0 Ack=1 Win=14480 Len=0 MSS=1460 SACK_PERM=1 TSval=356120740 TSecr=289451261 WS=32
 0.000825 192.168.10.14 -> 192.168.10.112 TCP 66 54695 > http [ACK] Seq=1 Ack=1 Win=14720 Len=0 TSval=289451261 TSecr=356120740
 0.000883 192.168.10.14 -> 192.168.10.112 HTTP 180 GET / HTTP/1.1
 0.001312 192.168.10.112 -> 192.168.10.14 TCP 66 http > 54695 [ACK] Seq=1 Ack=115 Win=14496 Len=0 TSval=356120740 TSecr=289451261
 0.001646 192.168.10.112 -> 192.168.10.14 HTTP 342 HTTP/1.1 200 OK  (text/html)
 0.001672 192.168.10.112 -> 192.168.10.14 TCP 66 http > 54695 [FIN, ACK] Seq=277 Ack=115 Win=14496 Len=0 TSval=356120741 TSecr=289451261
 0.001878 192.168.10.14 -> 192.168.10.112 TCP 66 54695 > http [ACK] Seq=115 Ack=277 Win=15744 Len=0 TSval=289451262 TSecr=356120741
 0.002299 192.168.10.14 -> 192.168.10.112 TCP 66 54695 > http [FIN, ACK] Seq=115 Ack=278 Win=15744 Len=0 TSval=289451262 TSecr=356120741
 0.002799 192.168.10.112 -> 192.168.10.14 TCP 66 http > 54695 [ACK] Seq=278 Ack=116 Win=14496 Len=0 TSval=356120742 TSecr=289451262

cap data between LVS and Web servers
# tshark -i eth1 port 80
Capturing on eth1
 0.000000 192.168.10.14 -> 192.168.30.10 TCP 74 54696 > http [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=289453429 TSecr=0 WS=128
 0.000559 192.168.30.10 -> 192.168.10.14 TCP 74 http > 54696 [SYN, ACK] Seq=0 Ack=1 Win=14480 Len=0 MSS=1460 SACK_PERM=1 TSval=356177805 TSecr=289453429 WS=32
 0.000816 192.168.10.14 -> 192.168.30.10 TCP 66 54696 > http [ACK] Seq=1 Ack=1 Win=14720 Len=0 TSval=289453429 TSecr=356177805
 0.000869 192.168.10.14 -> 192.168.30.10 HTTP 180 GET / HTTP/1.1
 0.001283 192.168.30.10 -> 192.168.10.14 TCP 66 http > 54696 [ACK] Seq=1 Ack=115 Win=14496 Len=0 TSval=356177805 TSecr=289453429
 0.001685 192.168.30.10 -> 192.168.10.14 HTTP 342 HTTP/1.1 200 OK  (text/html)
 0.001715 192.168.30.10 -> 192.168.10.14 TCP 66 http > 54696 [FIN, ACK] Seq=277 Ack=115 Win=14496 Len=0 TSval=356177806 TSecr=289453429
 0.001922 192.168.10.14 -> 192.168.30.10 TCP 66 54696 > http [ACK] Seq=115 Ack=277 Win=15744 Len=0 TSval=289453430 TSecr=356177806
 0.002273 192.168.10.14 -> 192.168.30.10 TCP 66 54696 > http [FIN, ACK] Seq=115 Ack=278 Win=15744 Len=0 TSval=289453430 TSecr=356177806
 0.002771 192.168.30.10 -> 192.168.10.14 TCP 66 http > 54696 [ACK] Seq=278 Ack=116 Win=14496 Len=0 TSval=356177807 TSecr=289453430
No comments:
Labels: , , ,
Subscribe to: Posts (Atom)