ubuntu 12.04 64bit : install curl-loader

curl-loader is traffic generator. you can generate HTTP , HTTPS , FTP etc with curl-loader.

Add to that , you can simulate multiple clients ( multiple source IP addresses )

curl-loader

http://curl-loader.sourceforge.net/index.html

Here’s an explanation of how to install curl-loader on Ubuntu 12.04 64bit box and generate HTTP , HTTPS traffic.

# uname –r 3.2.0-39-generic # tail -1 /etc/lsb-release DISTRIB_DESCRIPTION="Ubuntu 12.04.2 LTS"

[ install curl-loader ]

download curl-loader from http://curl-loader.sourceforge.net/

I downloaded curl-loader-0.56.tar.bz2.

extract

# bunzip2 curl-loader-0.56.tar.bz2 # tar xvf curl-loader-0.56.tar # cd curl-loader-0.56/

build requirements

Build it using general C development environment with bash, gcc (3 or 4 series), make, etc on a linux machine. Building pre-requirements are: 1. openssl binaries; 2. openssl development package with include files (on debian package libssl-dev);

I have not installed libssl-dev , so I installed libssl-dev before compiling curl-loader.

# apt-get install libssl-dev

compile

# pwd /home/hattori/My_works/curl-loader-0.56 # make # make install mkdir -p /usr/bin mkdir -p /usr/share/man/man1 mkdir -p /usr/share/man/man5 mkdir -p /usr/share/doc/curl-loader/ cp -f curl-loader /usr/bin cp -f doc/curl-loader.1 /usr/share/man/man1/ cp -f doc/curl-loader-config.5 /usr/share/man/man5/ cp -f doc/* /usr/share/doc/curl-loader/ cp -rf conf-examples /usr/share/doc/curl-loader/

If libidns libraly name is not libidn.so , create a symbolic link file “libidn.so” like this:

# locate libidn.so /usr/lib/x86_64-linux-gnu/libidn.so.11 /usr/lib/x86_64-linux-gnu/libidn.so.11.6.6 # cd /usr/lib/x86_64-linux-gnu/ # ln -s libidn.so.11 libidn.so

tuning

# ulimit -n 10000 # echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle # echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse # echo 102286 > /proc/sys/fs/file-max

sample configuration files are in “/usr/share/doc/curl-loader/conf-examples” directory.

# ls 10K.conf                          https.conf 60K.conf                          ipv6.conf bax.conf                          log-hdr-body.conf bax-login-logoff-noncycling.conf  login-uas-cycling.conf bax-logs.conf                     login-uas-logoff-cycling.conf <snip>

[ configuration ]

copy sample conf

# cp /usr/share/doc/curl-loader/conf-examples/bulk.conf ./my_bulk.conf

generate HTTP traffic to LVS VIP

client

 eth1 ( real ip 192.168.100.10 )

 |

vSW

 | 192.168.100.254

Router

 |

Web 192.168.200.100

  

edit my_bulk.conf  for my network environments

# cat my_bulk.conf ########### GENERAL SECTION ################################ BATCH_NAME= bulk CLIENTS_NUM_MAX=200 # Same as CLIENTS_NUM #CLIENTS_NUM_START=10 CLIENTS_RAMPUP_INC=5 #INTERFACE   =eth0 INTERFACE   =eth1 NETMASK=24 IP_ADDR_MIN= 192.168.100.11 IP_ADDR_MAX= 192.168.100.200 IP_SHARED_NUM=100 CYCLES_NUM= -1 URLS_NUM= 1 ########### URL SECTION #################################### #URL=http://localhost/index.html URL=http://192.168.200.100 #URL=http://localhost/apache2-default/ACE-INSTALL.html #URL=http://localhost/ACE-INSTALL.html #URL_SHORT_NAME="local-apache" REQUEST_TYPE=GET TIMER_URL_COMPLETION = 5000 TIMER_AFTER_URL_SLEEP = 500

run curl-loader

# curl-loader -f my_bulk.conf

cap data on the curl-loader box

I can simulate multiple clients with curl-loader.

# tshark -r zzz.pcap | grep -i get Running as user "root" and group "root". This could be dangerous. 16   0.009724 192.168.100.11 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 17   0.009778 192.168.100.15 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 18   0.009815 192.168.100.14 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 19   0.009859 192.168.100.13 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 20   0.009903 192.168.100.12 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 36   0.526556 192.168.100.12 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 37   0.526639 192.168.100.13 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 38   0.526701 192.168.100.15 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 39   0.526798 192.168.100.14 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 40   0.526868 192.168.100.11 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 60   1.012545 192.168.100.16 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 65   1.012701 192.168.100.19 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 66   1.012769 192.168.100.18 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 67   1.012834 192.168.100.17 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 70   1.012958 192.168.100.20 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 86   1.051773 192.168.100.11 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 87   1.051873 192.168.100.12 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 88   1.051969 192.168.100.13 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 89   1.052063 192.168.100.14 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 90   1.052155 192.168.100.15 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1 101   1.537344 192.168.100.18 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1

curl-loader logs

- bulk.log for errors and traces; - bulk.txt for loading statistics; - bulk.ctx for virtual client based statistics. - bulk.ops for operational statistics.

# less bulk.log # msec_offset cycle_no url_no client_no (ip) indic info 12 0 0 1 !! OK 200 12 0 0 4 !! OK 200 12 0 0 5 !! OK 200 12 0 0 3 !! OK 200 12 0 0 2 !! OK 200 529 1 0 4 !! OK 200 529 1 0 5 !! OK 200 <snip>

Next , generate HTTPS traffic

copy sample config

# cp /usr/share/doc/curl-loader/conf-examples/https.conf my_https.conf

edit config for my network topology.

# cat my_https.conf ########### GENERAL SECTION ################################ BATCH_NAME= https CLIENTS_NUM_MAX=100 # Same as CLIENTS_NUM #CLIENTS_NUM_START=1 #CLIENTS_RAMPUP_INC=1 INTERFACE   =eth1 NETMASK=24 IP_ADDR_MIN= 192.168.100.11 IP_ADDR_MAX= 192.168.100.200  #Actually - this is for self-control CYCLES_NUM=-1 URLS_NUM= 1 ########### URL SECTION #################################### URL=https://192.168.200.100 #URL=https://localhost/apache2-default/ACE-INSTALL.html #URL=https://localhost/ACE-INSTALL.html #URL_SHORT_NAME="url-https" REQUEST_TYPE=GET TIMER_URL_COMPLETION = 0 # In msec. When positive, Now it is enforced by cancelling url fetch on timeout TIMER_AFTER_URL_SLEEP =1000

run curl-loader

# curl-loader -f my_https.conf

cap data on the curl-loader box

# tshark -r https.pcap -R '(tcp.port==443)' | grep -i syn | head -10 | grep -iv "syn, ack" Running as user "root" and group "root". This could be dangerous.  1   0.000000 192.168.100.11 -> 192.168.200.100 TCP 74 56998 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836483 TSecr=0 WS=16  2   0.000182 192.168.100.12 -> 192.168.200.100 TCP 74 49495 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836483 TSecr=0 WS=16  3   0.000357 192.168.100.13 -> 192.168.200.100 TCP 74 35063 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836483 TSecr=0 WS=16  4   0.000555 192.168.100.14 -> 192.168.200.100 TCP 74 51862 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836484 TSecr=0 WS=16  5   0.000724 192.168.100.15 -> 192.168.200.100 TCP 74 47794 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836484 TSecr=0 WS=16  6   0.000893 192.168.100.16 -> 192.168.200.100 TCP 74 35510 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836484 TSecr=0 WS=16  9   0.001116 192.168.100.17 -> 192.168.200.100 TCP 74 36015 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836484 TSecr=0 WS=16

IP addresses are added automatically as when starting curl-loader

# ip addr show eth1 3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000    link/ether <MAC address> brd ff:ff:ff:ff:ff:ff    inet 192.168.100.10/24 brd 192.168.100.255 scope global eth1    inet 192.168.200.11/24 scope global eth1    inet 192.168.200.12/24 scope global secondary eth1    inet 192.168.200.13/24 scope global secondary eth1    inet 192.168.100.11/24 scope global secondary eth1    inet 192.168.100.12/24 scope global secondary eth1    inet 192.168.100.13/24 scope global secondary eth1    inet 192.168.100.14/24 scope global secondary eth1 <snip>