Building Eruv at Limmud Festival 2013

I helped building the Eruv at the Limmud Festival 2013:

Ganz in Familie
Über 350 Besucher lernen am Werbellinsee
http://www.juedische-allgemeine.de/article/view/id/15950
Jüdische Allgemeine, 10.05.2013

How To Create a Debian/Ubuntu Repository for DEB Packages

Initial Setup

Required Debian packages: reprepro

1. Create a directory for the repository and its configuration:

   $ mkdir -p repo/conf
   

2. Create a conf/distributions configuration file like this:
   
3. Put my putinrepo script into the repo or next to it:
   
4. "Export" the repo to create the metadata files for the empty repo:

   $ repo/putinrepo.sh
   $ tree repo
   repo
   ├── putinrepo.sh
   ├── conf
   │   └── distributions
   ├── db
   │   ├── checksums.db
   │   ├── contents.cache.db
   │   ├── packages.db
   │   ├── references.db
   │   ├── release.caches.db
   │   └── version
   └── dists
       ├── precise
       │   ├── main
       │   │   ├── binary-amd64
       │   │   │   ├── Packages
       │   │   │   ├── Packages.gz
       │   │   │   └── Release
       │   │   ├── binary-armel
       │   │   │   ├── Packages
       │   │   │   ├── Packages.gz
       │   │   │   └── Release
       │   │   └── binary-i386
       │   │       ├── Packages
       │   │       ├── Packages.gz
       │   │       └── Release
       │   └── Release
       ├── quantal
       │   ├── main
       │   │   ├── binary-amd64
       │   │   │   ├── Packages
       │   │   │   ├── Packages.gz
       │   │   │   └── Release
       │   │   ├── binary-armel
       │   │   │   ├── Packages
       │   │   │   ├── Packages.gz
       │   │   │   └── Release
       │   │   └── binary-i386
       │   │       ├── Packages
       │   │       ├── Packages.gz
       │   │       └── Release
       │   └── Release
       └── raring
           ├── main
           │   ├── binary-amd64
           │   │   ├── Packages
           │   │   ├── Packages.gz
           │   │   └── Release
           │   ├── binary-armel
           │   │   ├── Packages
           │   │   ├── Packages.gz
           │   │   └── Release
           │   └── binary-i386
           │       ├── Packages
           │       ├── Packages.gz
           │       └── Release
           └── Release
   
   18 directories, 37 files
   

Now the repo is ready for usage. To add the repo to the local system (assuming you use quantal):

$ sudo tee /etc/apt/sources.list.d/my_repo.list <<EOF
deb file:///$(pwd)/repo quantal main
EOF

Adding Packages

putinrepo.sh <deb-files> is used to add packages to a distribution in the repo. It will automatically add the package to all codenames. Set REPREPRO_CODENAMES to an array of codenames if you want to select the targets.

Signing the Repository

apt complains if you don't sign the repo. Luckily that is really simple: Just add SignWith: yes to each section in the conf/distributions file to sign the repo with your default GPG key.

See Also

These links were useful for me:

• Reprepro: Das eigene Paketarchiv für Debian und Ubuntu
• Reprepro Man Page

Simple Command Line Download Speed Test

I got a server that started to have a bad network connection. To help debugging this I needed to collect some data, e.g. of running a download test every 5 minutes.

A quick search revealed nothing so I found a simple way to do that with curl and Google Drive.

1. Google Drive Form

I set up a simple Google Drive form that accepts two text values:

Next I need to find out the HTML form field names of these two fields. This takes a little bit of digging.   There is always a label attached to each input field. They are connected via the for attribute of the label.
With this information I can now construct a simple curl script.

2. CURL script to collect some data and post

I want curl to try to download a URL and record the download speed. The -w, -y and -Y options come in handy for this purpose:

• -w lets me output only the information I really care about (the speed of the download in Bytes per second)
• -y gives a timeout after which curl will abort the download
• -Y gives a minimum download speed (in B/s). If after -y seconds this is not achieved then the download is aborted. Choosing a very high number ensures that the download will never run longer than 60 seconds.

curl -w "%{speed_download}" -y 60 -Y 100000000000000 -s -o /dev/null "$1"

runs at most 60 seconds and prints the average download speed (e.g. 546746.000) to standard out.

The result is posted to Google Drive with another curl call. I use -s -o /dev/null as I don't care about the output of the download or the data post. The complete script looks like this:

This is it! The relevant part is curl -d <form field id>=<form field data> and the right URL of the form. Filling the form fields with a subshell $(...) is just a short form that does not need extra variables.

Call the script from a cron job and watch how data starts collecting in your Google Drive. I can even run it on different systems (giving me different probes to provide data) and later aggregate or filter the data.

I don't need to send a timestamp as this is added automatically by Google Drive Forms.

3. Google Drive spreadsheet for data analysis

In the Google Drive form one must select a destination spreadsheet for the results. In that spreadsheet I created a Chart, mark the two columns timestamp and speed and set the chart type to Trend and Timeline. The result looks already very useful:

The chart has its own publish button which gives a JavaScript block. In this block I only had to adjust the width: and height: settings to format the chart and replace the range with A:B so that it would take all the data.

Note: The spreadsheet should be set to a US or UK locale so that the XXX.000 numbers created by curl will be parsed correctly. For other locales you might have to strip the .000 before posting the data.

4. Doing this for multiple targets

The whole point of this exercise was to monitor the service quality of a VPS hoster. As the service quality is still below my needs I got another VPS at another hoster so that I had to setup the same monitoring for the other hoster as well. 

It turns out that this is really simple. I can just copy the form in Google Drive and get a new form that uses the same keys for the input fields! That makes it really easy to customize my little script like this:

And the results of the other hoster look like this:

As we can see here, the other hoster is doing much better :-)

German Python Conference 2012

I just came back from PyCon DE 2012 in Leipzig. I went their not knowing what to expect and came back a real fan.

Not that I am really doing that much with Python, but the community feeling at the conference beat all my previous experiences.

Python is sometimes called a glue language, it seems to be also true for the attendants. Of course they all have an interest in the language as a tool, but they come from all areas of IT and bring their own experience to share.

Between physicists from the CERN, web developers and tool developers there was something new for everybody.

Especially noteworthy where the Lightning Talks that happened at the end of every conference day. Not only entertaining but also very informative. The speakers used their 5 minutes mostly to present some tool or idea to the audience and ask for help or git pull requests. That is certainly a strong sign of a vibrant community.

The 2013 PyCon DE will be in Cologne and the 2014 in Berlin. Even if you don't dream in Python, the conference is worth to attend to meet like minded fellows and get input for new ideas.

Dashboards Made Easy

If you work at an agile shop you probably have seen something like this:

You also find a lot about how people create those nice dash boards as a web page, but you almost never find out how to actually build the system that runs the dash board.

Basically this is a task for digital signage, but "standard" solutions like Xibo seem heavily oversized for running an internal dash board. All we need is a Linux box running a browser and the screen staying on.

As my team also wanted to have a dashboard, I  recently hat to build one and decided to put my effort into an Ubuntu package named kiosk-browser that can be found on the ImmobilienScout24 github page. The package turns a regular Ubuntu installation into a web-based dashboard.

Now the effort to setup a new dashboard system has been reduced to these steps:

1. Install a computer with Ubuntu.
2. Build or download and install the kiosk-browser package. 
3. Log on as a user and run this command:
   sudo tee /etc/default/kiosk-browser \
    <<<'KIOSK_BROWSER_START_PAGE="http://your/dashboard/url"'
4. Reboot 

If your system is a laptop or of you have a rotated screen you might want to add some xrandr commands to /etc/default/kiosk-browser to setup your displays properly. For the laptop that runs the rotated display in the picture this is:

xrandr --output VGA1 --auto --rotate left --output LVDS1 --off

sleep 5

I got the general idea from Ubuntu 12.04 Kiosk in 10 Easy Steps which has all the building blocks.

Update 2012-10-16: Now also with multi-monitor support!

Puppet and Chef do only half the job

Going to conferences and meetups is not just for seeing the talks, it is much more for meeting the "right" people. At the recent devopsdays Mountain View I met Ernest Mueller and discovered his theagileadmin.com blog, where I immediately liked his DevOps: It’s Not Chef And Puppet blog post.

After some talking with Ernest and others at the devopsdays I realized that I believe that the Ops job contains more than most people do with Puppet and Chef: Managing the Operating System patches and upgrades.

Read my guest posting Puppet and Chef do only half the job for the full story.

Update:
In the 2012 SysAdvent Blog Miah Johnson suggests in Packages Doing Too Much? to solve this issue by going the opposite way: Deploy the entire operating systems via recipes. She also points to NixOS and Guix as an example of such operationg systems. While I still believe that using the OS packages for custom software and configuration deployment is the cheapest solution, doing that on a recipe-based OS would be also fine because then one would again use the OS tools for everything else which is the main point I am trying to make.

My Open Source Data Center Conference 2013 talk is also about this topic.

Velocity 2012 T-Shirt Boom

The Velocity Web Performance and Operations Conference is always about bringing hard data to make a point.

This year I was overwhelmed by the amount of T-Shirts given out by the exhibitors. Over a coffee I was joking with somebody, that there must be like 20000 shirts at the conference. He was loughing it off, but I actually go curious and asked all exhibitors how many shirts they brought:

But, I was wrong. According to the survey there where a total of 10746 shirts for about 2500 attendees, which makes over 5 shirts per attendee!

So what caused this "overprovisioning"? As not everybody likes to take shirts, there must be people who walked away with 10 or 20 shirts...

Now that we have some hard data we should try to learn something from this data. Unfortunately I could not find any other information about all these companies that might be correlated here. Some companies who brought few shirts told me that they give them only to special customers and not hand them out freely.

Dropbox for sure won the "most shirts" price and people have been observed taking Dropbox shirts by the dozen.


Verisign won the "unusually packaged" prize with this vaccum packed shirt (but they brought only a few):

But the main question unfortunately remains unanswerd: What do these companies get out of this? Why did all these companies spend so much money on T-Shirts? Is it a desperate grab for attention? Or is it a way of saying thank you?

BTW, at the devopsdays Mountain View just after the Velocity there where much less shirts, mostly from splunk and of course the traditional devopsdays shirts.

And, if you have time for just one video from the Velocity, make sure to watch Jesse Robbins talk about "Changing Culture & Being a force for Awesome":

Cardboard Smart Phone Camera Stand

Do you use your smart phone to record videos?
Do you use it to record talks at conferences?
Did you ever try to hold up a smart phone for 45 minutes?

Well, I did and it was no fun at all!

My fix for this problem was to build an improvised camera stand for my phone:

All you need for that is some cardboard and something to cut it:

It takes about 5 minutes to make and greatly improves the video quality: No more jitter, shaking or covering up the mic. With the right light conditions you can even manage to make professional looking videos. Also, if you switch the phone to flight mode you can be sure that no incoming call will interrupt the recording.

Download printable version.

Limmud 2012 in Berlin

Limmud 2012 was in Berlin at the Jüdische Oberschule and that made it quite different from the previous Limmud festivals at Werbellinsee. The feeling was much less that of a closed community and more that of a regular conference. Luckily the next Limmud festival will (most likely) be back at Werbellinsee.

The festival was well attended without feeling crowded. I talked a bit to the org team. They seemed as stressed as in previous years. Seeing how well the festival runs they really deserve a lot of respect and gratitude for doing this job.


Highlights where the talks by Rabbi Cardozo which where approximately the same as the ones he gave at previous Limmud events (see the his YouTube playlists for a choice of videos), but still worthwhile to sit through.

There were also lots of kids programs, like Ruth and Naomi, Ask the Rabi and of course lots of movies.

The unusal venue in the middle of the city allowed for various tours which where also well frequented.

Hope to see you next year!

Volunteering at the LinuxTag 2012

Today I spent almost all the day helping the LinuxTag crew. They needed someone to help with transportation and I own a trailer and had a day of vacation. IT turns out that the LinuxTag - like all trade shows - comes with a lot of stuff and the people who help also need something:

Shopping for the LinuxTag

And this is actually not all of it, just to help the volunteers and project members and booth staff.

Besides lugging lots of boxes and crates I actually learned a lot about the LinuxTag today: It is all completely done by volunteers, most of whom actually take vacation days to organize the event!

If you like the LinuxTag please come and join the effort and sign up to help at the LinuxTag: http://www.linuxtag.org/2012/de/mitmachen.html