curl-loader is traffic generator. you can generate HTTP , HTTPS , FTP etc with curl-loader.
Add to that , you can simulate multiple clients ( multiple source IP addresses )
curl-loader
Here’s an explanation of how to install curl-loader on Ubuntu 12.04 64bit box and generate HTTP , HTTPS traffic.
# uname –r
3.2.0-39-generic
# tail -1 /etc/lsb-release
DISTRIB_DESCRIPTION="Ubuntu 12.04.2 LTS"
|
[ install curl-loader ]
I downloaded curl-loader-0.56.tar.bz2.
extract
# bunzip2 curl-loader-0.56.tar.bz2
# tar xvf curl-loader-0.56.tar
# cd curl-loader-0.56/
|
build requirements
Build it using general C development environment with bash,
gcc (3 or 4 series), make, etc on a linux machine.
Building pre-requirements are:
1. openssl binaries;
2. openssl development package with include files (on debian package libssl-dev);
|
I have not installed libssl-dev , so I installed libssl-dev before compiling curl-loader.
# apt-get install libssl-dev
|
compile
# pwd
/home/hattori/My_works/curl-loader-0.56
# make
# make install
mkdir -p /usr/bin
mkdir -p /usr/share/man/man1
mkdir -p /usr/share/man/man5
mkdir -p /usr/share/doc/curl-loader/
cp -f curl-loader /usr/bin
cp -f doc/curl-loader.1 /usr/share/man/man1/
cp -f doc/curl-loader-config.5 /usr/share/man/man5/
cp -f doc/* /usr/share/doc/curl-loader/
cp -rf conf-examples /usr/share/doc/curl-loader/
|
If libidns libraly name is not libidn.so , create a symbolic link file “libidn.so” like this:
# locate libidn.so
/usr/lib/x86_64-linux-gnu/libidn.so.11
/usr/lib/x86_64-linux-gnu/libidn.so.11.6.6
# cd /usr/lib/x86_64-linux-gnu/
# ln -s libidn.so.11 libidn.so
|
tuning
# ulimit -n 10000
# echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
# echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse
# echo 102286 > /proc/sys/fs/file-max
|
sample configuration files are in “/usr/share/doc/curl-loader/conf-examples” directory.
# ls
10K.conf https.conf
60K.conf ipv6.conf
bax.conf log-hdr-body.conf
bax-login-logoff-noncycling.conf login-uas-cycling.conf
bax-logs.conf login-uas-logoff-cycling.conf
<snip>
|
[ configuration ]
copy sample conf
# cp /usr/share/doc/curl-loader/conf-examples/bulk.conf ./my_bulk.conf
|
generate HTTP traffic to LVS VIP
client
eth1 ( real ip 192.168.100.10 )
|
vSW
| 192.168.100.254
Router
|
Web 192.168.200.100
edit my_bulk.conf for my network environments
# cat my_bulk.conf
########### GENERAL SECTION ################################
BATCH_NAME= bulk
CLIENTS_NUM_MAX=200 # Same as CLIENTS_NUM
#CLIENTS_NUM_START=10
CLIENTS_RAMPUP_INC=5
#INTERFACE =eth0
INTERFACE =eth1
NETMASK=24
IP_ADDR_MIN= 192.168.100.11
IP_ADDR_MAX= 192.168.100.200
IP_SHARED_NUM=100
CYCLES_NUM= -1
URLS_NUM= 1
########### URL SECTION ####################################
#URL=http://localhost/index.html
URL=http://192.168.200.100
#URL=http://localhost/apache2-default/ACE-INSTALL.html
#URL=http://localhost/ACE-INSTALL.html
#URL_SHORT_NAME="local-apache"
REQUEST_TYPE=GET
TIMER_URL_COMPLETION = 5000
TIMER_AFTER_URL_SLEEP = 500
|
run curl-loader
# curl-loader -f my_bulk.conf
|
cap data on the curl-loader box
I can simulate multiple clients with curl-loader.
# tshark -r zzz.pcap | grep -i get
Running as user "root" and group "root". This could be dangerous.
16 0.009724 192.168.100.11 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
17 0.009778 192.168.100.15 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
18 0.009815 192.168.100.14 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
19 0.009859 192.168.100.13 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
20 0.009903 192.168.100.12 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
36 0.526556 192.168.100.12 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
37 0.526639 192.168.100.13 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
38 0.526701 192.168.100.15 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
39 0.526798 192.168.100.14 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
40 0.526868 192.168.100.11 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
60 1.012545 192.168.100.16 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
65 1.012701 192.168.100.19 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
66 1.012769 192.168.100.18 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
67 1.012834 192.168.100.17 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
70 1.012958 192.168.100.20 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
86 1.051773 192.168.100.11 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
87 1.051873 192.168.100.12 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
88 1.051969 192.168.100.13 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
89 1.052063 192.168.100.14 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
90 1.052155 192.168.100.15 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
101 1.537344 192.168.100.18 -> 192.168.200.100 HTTP 184 GET / HTTP/1.1
|
curl-loader logs
- bulk.log for errors and traces;
- bulk.txt for loading statistics;
- bulk.ctx for virtual client based statistics.
- bulk.ops for operational statistics.
|
# less bulk.log
# msec_offset cycle_no url_no client_no (ip) indic info
12 0 0 1 !! OK 200
12 0 0 4 !! OK 200
12 0 0 5 !! OK 200
12 0 0 3 !! OK 200
12 0 0 2 !! OK 200
529 1 0 4 !! OK 200
529 1 0 5 !! OK 200
<snip>
|
Next , generate HTTPS traffic
copy sample config
# cp /usr/share/doc/curl-loader/conf-examples/https.conf my_https.conf
|
edit config for my network topology.
# cat my_https.conf
########### GENERAL SECTION ################################
BATCH_NAME= https
CLIENTS_NUM_MAX=100 # Same as CLIENTS_NUM
#CLIENTS_NUM_START=1
#CLIENTS_RAMPUP_INC=1
INTERFACE =eth1
NETMASK=24
IP_ADDR_MIN= 192.168.100.11
IP_ADDR_MAX= 192.168.100.200 #Actually - this is for self-control
CYCLES_NUM=-1
URLS_NUM= 1
########### URL SECTION ####################################
URL=https://192.168.200.100
#URL=https://localhost/apache2-default/ACE-INSTALL.html
#URL=https://localhost/ACE-INSTALL.html
#URL_SHORT_NAME="url-https"
REQUEST_TYPE=GET
TIMER_URL_COMPLETION = 0 # In msec. When positive, Now it is enforced by cancelling url fetch on timeout
TIMER_AFTER_URL_SLEEP =1000
|
run curl-loader
# curl-loader -f my_https.conf
|
cap data on the curl-loader box
# tshark -r https.pcap -R '(tcp.port==443)' | grep -i syn | head -10 | grep -iv "syn, ack"
Running as user "root" and group "root". This could be dangerous.
1 0.000000 192.168.100.11 -> 192.168.200.100 TCP 74 56998 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836483 TSecr=0 WS=16
2 0.000182 192.168.100.12 -> 192.168.200.100 TCP 74 49495 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836483 TSecr=0 WS=16
3 0.000357 192.168.100.13 -> 192.168.200.100 TCP 74 35063 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836483 TSecr=0 WS=16
4 0.000555 192.168.100.14 -> 192.168.200.100 TCP 74 51862 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836484 TSecr=0 WS=16
5 0.000724 192.168.100.15 -> 192.168.200.100 TCP 74 47794 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836484 TSecr=0 WS=16
6 0.000893 192.168.100.16 -> 192.168.200.100 TCP 74 35510 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836484 TSecr=0 WS=16
9 0.001116 192.168.100.17 -> 192.168.200.100 TCP 74 36015 > https [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=20836484 TSecr=0 WS=16
|
IP addresses are added automatically as when starting curl-loader
# ip addr show eth1
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether <MAC address> brd ff:ff:ff:ff:ff:ff
inet 192.168.100.10/24 brd 192.168.100.255 scope global eth1
inet 192.168.200.11/24 scope global eth1
inet 192.168.200.12/24 scope global secondary eth1
inet 192.168.200.13/24 scope global secondary eth1
inet 192.168.100.11/24 scope global secondary eth1
inet 192.168.100.12/24 scope global secondary eth1
inet 192.168.100.13/24 scope global secondary eth1
inet 192.168.100.14/24 scope global secondary eth1
<snip>
|
