Rough Measurement for HTTP Client Download Speed

Henrik G. Vogel  / pixelio.de
Ever wonder if your website is slow because of the server or because of the clients?
Do you want to know how fast is your clients' connection to the Internet?
Don't want to use external tracking services, injecting JavaScript etc.?

Why not simply measure how long it takes to deliver the content from your webserver to your users? Apache and nginx both support logging the total processing time of a request with a suitably high precision. That gives the time from starting with first byte received from the client and ending after the last byte sent to the client.

To try out this idea I added %D to the log format for access.log of my Apache server and wrote a little Python script to calculate the transfer speeds. With the help of the apachelog Python module parsing the Apache access.log is really simple. This module takes a log format definition as configuration and automatically breaks down a log line into the corresponding values.

The script can be found together with a little explanation on my GitHub page: github.com/schlomo/apacheclientspeed.

It is rather rough, I am now collecting data to see how useful this information actually is. One thing I can already say: Small files have very erratic results, bigger files yield more trustworthy numbers.

Videos are a good example, the output of this script looks like this:

$ grep mp4 /var/log/apache2/access.log | tail -n 20 | python -m apacheclientspeed
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 1409 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 119 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 1936 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 90 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 159 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 83 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 2067 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 43226 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 4 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 33491 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 28 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g2/GANGNAM_320_Rf_28.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
89.204.139.1 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 0 KiloByte/s
217.111.70.208 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 507 KiloByte/s
80.226.1.16 GET /c/g1/GANGNAM_320_Rf_30.mp4 HTTP/1.1 46 KiloByte/s
The many requests with 0 KB/s are made by iOS devices. They do a lot of smaller Range requests when streaming a movie over HTTP.

Comments

Post a Comment

Like this content? You could send me something from my Amazon Wishlist. Need commercial support? Contact me for Consulting Services.

Popular posts from this blog

Overriding / Patching Linux System Serial Number

Image
I'm a big fan of test driven development  (TDD) for infrastructure components. I'm currently working on a hardware-related topic where we also use the system serial number as identifier. To create a proper integration test, we need to be able to start a system and set the serial number to a known value. This can easily be done with the help of virtual machines like in VMware or VirtualBox , but I couldn't find a way for changing the system serial number on hardware boxes, cloud VMs (e.g. on Alibaba Cloud) or other Linux system. Problem Analysis I was thinking: Linux is the operating system where I can potentially do everything . So how hard can this be? After some digging around I found out that there are those main sources for the serial number on Linux: /sys/firmware/dmi/tables/DMI contains a binary blob of Desktop Management Interface data provided by the kernel and the dmidecode utility is commonly used to decode
Read more

The Demise of KaiOS - Alcatel 3088X

Image
I was really excited, when Firefox OS became commercially available as KaiOS for "smart" feature phones. Unfortunately the high hopes I had put into this platform have been utterly shattered. I was recently looking for a button phone that had only three requirements : very good battery run time, at least several days or a week of standby without charging easy backup or even synchronization of the contacts VoLTE support to benefit from LTE coverage  In Theory... On paper, KaiOS phones look like the perfect match for those requirements. PhoneCopy for KaiOS is a cloud synchronization service that comes with an app for KaiOS (and many other platforms) to synchronize contacts and other phone content to the cloud (and back). With batteries of 1500 mAh and more , KaiOS phones have the potential to deliver outstanding battery performance. The Alcatel 3088X , for example, boasts "up to 300h standby and more than 7h talk time" with a 1530 mAh battery: Battery specification
Read more

A Login Security Architecture Without Passwords

Image
Following up on Lifting the Curse of Static Credentials and Eliminating the Password of Shared Accounts , I have many discussions about why we would benefit from removing password prompts for website logins. Let's dig deeper into the details and show why removing password prompts leads to a  safer security architecture . Update 11.03.2022: Added more details about business vs. consumer websites and additional security suggestions surrounding WebAuthn Problem Space For context, imagine a website that needs to identify online users via their email address. We assume that the website in question is not the primary email system of a user but some other website, e.g. an e-commerce shop system or a collaborative productivity tool. As a User As a user of that website I want to easily sign up for an account have an easy way to login into the account be sure that my account is protected from others or attacks be able to easily recover access to my account i
Read more