Python : HTTP keepalive connections with urllib3

just referred to https://urllib3.readthedocs.org/en/1.0.2/pools.html#urllib3.connectionpool.HTTPConnectionPool

install python-urllib3

# python --version Python 2.7.3 # apt-get install -y python-urllib3

[ sample usage ]

send 3 HTTP GET packets over the same TCP connection.

# python Python 2.7.3 (default, Sep 26 2012, 21:51:14) [GCC 4.7.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from urllib3 import HTTPConnectionPool >>> pool = HTTPConnectionPool('192.168.1.100',maxsize=1,port=80,timeout=None) >>> r = pool.request('GET','/') >>> r = pool.request('GET','/') >>> r = pool.request('GET','/') >>> pool.num_connections 1 >>> pool.num_requests 3 >>> r.status 200 >>> r.data 'centos6-4\n'

cap data captured on the web server.
send 3 HTTP GET packets with the same TCP connection. ( HTTP keepalive )

0.000195 192.168.1.10 -> 192.168.1.100 TCP 74 33145 > 80 [SYN] Seq=0 Win=14600 Len=0 MSS=1460 SACK_PERM=1 TSval=23628281 TSecr=0 WS=128  0.000217 192.168.1.100 -> 192.168.1.10 TCP 74 80 > 33145 [SYN, ACK] Seq=0 Ack=1 Win=14480 Len=0 MSS=1460 SACK_PERM=1 TSval=23359358 TSecr=23628281 WS=32  0.000722 192.168.1.10 -> 192.168.1.100 TCP 66 33145 > 80 [ACK] Seq=1 Ack=1 Win=14720 Len=0 TSval=23628281 TSecr=23359358  0.000798 192.168.1.10 -> 192.168.1.100 HTTP 133 GET / HTTP/1.1  0.000809 192.168.1.100 -> 192.168.1.10 TCP 66 80 > 33145 [ACK] Seq=1 Ack=68 Win=14496 Len=0 TSval=23359358 TSecr=23628281  0.002719 192.168.1.100 -> 192.168.1.10 HTTP 347 HTTP/1.1 200 OK  (text/html)  0.003446 192.168.1.10 -> 192.168.1.100 TCP 66 33145 > 80 [ACK] Seq=68 Ack=282 Win=15744 Len=0 TSval=23628282 TSecr=23359358  1.151583 192.168.1.10 -> 192.168.1.100 HTTP 133 GET / HTTP/1.1  1.153386 192.168.1.100 -> 192.168.1.10 HTTP 347 HTTP/1.1 200 OK  (text/html)  1.154018 192.168.1.10 -> 192.168.1.100 TCP 66 33145 > 80 [ACK] Seq=135 Ack=563 Win=16768 Len=0 TSval=23628570 TSecr=23359646  2.264500 192.168.1.10 -> 192.168.1.100 HTTP 133 GET / HTTP/1.1  2.266421 192.168.1.100 -> 192.168.1.10 HTTP 347 HTTP/1.1 200 OK  (text/html)  2.267128 192.168.1.10 -> 192.168.1.100 TCP 66 33145 > 80 [ACK] Seq=202 Ack=844 Win=17920 Len=0 TSval=23628848 TSecr=23359924 14.679534 192.168.1.100 -> 192.168.1.10 TCP 66 80 > 33145 [FIN, ACK] Seq=844 Ack=202 Win=14496 Len=0 TSval=23363028 TSecr=23628848 14.717349 192.168.1.10 -> 192.168.1.100 TCP 66 33145 > 80 [ACK] Seq=202 Ack=845 Win=17920 Len=0 TSval=23631961 TSecr=23363028

sends 10 HTTP GETs every seconds with the same connection and counts the number of “Status Code 200”

# cat -n http-get-keepalive.py     1  #!/usr/bin/env python     2     3  from urllib3 import HTTPConnectionPool     4  import time     5     6  i=0     7  num_status_code=0     8  other_status_code=0     9    10  pool = HTTPConnectionPool('192.168.1.100',maxsize=1,port=80,timeout=None)    11    12  while i < 10:    13          r = pool.request('GET','/')    14    15          # wait for 1 sec    16          time.sleep(1)    17    18          i += 1    19    20          # count the status code    21          if r.status == 200:    22                  num_status_code += 1    23          else:    24                  other_status_code += 1    25    26  print "number of 200 OK %s:" % num_status_code    27  print "number of other status codes: %s" % other_status_code

# ./http-get-keepalive.py number of 200 OK 10: number of other status codes: 0