2011-11-21

CloudCamp Gent 2011

I am at the CloudCamp Ghent and can share some notes.

VMware Cloud Foundry


http://www.flickr.com/photos/70260750@N08/6376439699/
  • Lode Vermeiren, VMware.
  • Talk on Slideshare
  • Long part about VMware products and history that are cloud related.
  • It got actually interesting once he started to demo stuff.
    • The good part was that it was very authentic (e.g. not all parts where working).
    • The bad part was that there was not so much to see, just some basic "upload my simple Hello World app"
  • The speaker distributed USB Keys with a VM and registration codes for the Cloud Foundry service.
  • Look at http://cloudfoundry.com/micro and use the cloudcampgh code for instant access (till November 26th). Without the code invites might take a couple of days to activate.
Combell Cloud Cases
  • Company is sponsor, this is a short sales pitch
  • Quite refreshing in its brevity
"Devops in the Cloud" is a pleonasm

http://www.flickr.com/photos/70260750@N08/6376521145/
  • Andrew Shafer (Reductive Labs / CloudScaling)
  • Thesis: Clouds exist only due to DevOps
  • On the question "did you solve the configuration management problem" he suggested packaging it and referred to our package deployment :-)
  • As his laptop did not work he started an interesting discussion about clouds, devops and corporate mindset
  • Bottom line:
    • Total automation
    • Short feedback loops between everybody involved
  • I think that this talk about DevOps and Cloud fits the subject: Prying the Cloud Open: Dell Crowbar & OpenStack, it also fits the OpenStack theme of the CloudCamp OpenStack workshop.
Microsoft Cloud Strategy
  • He started by announcing a "commercial break" before the coffee break. +1 for self-humor.
  • Eventually, every regular product should be available as a cloud service.
  • Office365 is just the start
  • Nothing really useful. But thanks to Microsoft for sponsoring this event.
Manage Non-Cloud Dev & Test Environment With a Cloud Hat On
Source: http://www.facebook.com/patrick.debois
  • Patrick Debois, devopsdays.org and others.
  • Download the talk from Slideshare.
  • Telling his story that brought him to the cloud.
  • His presentations was either plain b/w graphics or really nice cloud photos like
Source: http://www.collthings.co.uk/2008/06/10-very-rare-clouds.html
  • Quote: "Buy a new laptop every year" - The cost of a new laptop is much less than the loss in productivity due to ressource limitations. +1 that.
  • Interesting: Pay attention how the focus and understanding changes and evolves when going from local to cloud.
  • Abstraction through cloud libraries: Jclouds, Fog, Boto
  • Trick: Reuse the same workflows internally and in the cloud.
  • One cannot always do everything in the cloud, but you can shape your internal IT as a cloud
  • Here developed a good discussion
  • Challenges: Orchestration - No good and integrated solution available.
Cloud and Security A match made in heaven?
Sorry, it got too dark for my crap phone to take photos.
Challenges of Running and Scaling Cassandra in the Cloud
Source: http://www.flickr.com/photos/acunu_london/5343106364/]andy.ormsby
  • Andy Ormsby
  • Cloud challenges:
    • Performance of single node (significantly) less that physical server
    • Availability, stuff breaks
  • Apache Cassandra tries to answer these challenges
  • OK, this seems to be a Cassandra 101 talk. If you missed this one then try this Slideshare search.
  • As usual, the stories not written on the slides make the talk worthwhile.
I better don't write abou the telenet vendor talk, but I would rather use this opportunity to thank them vor sponsoring the event.

A Practical Introduction to OpenStack
Source: http://www.facebook.com/dysinger

That was the last "official" talk, now off to the break-out sessions. Let's see what we get today. Unfortunately half or more than half of the people left and did not stay for the break out sessions.

The topics where:
  • Private Cloud
  • Identity Management
  • Auto Scaling
  • Orchestration
Interesting discussion, not too many people. Hallway tack is always one of the hightlights of an event.

OpenStack Workshop
I also attended (and this was the main motive for the trip) the OpenStack workshop that occured on Monday before the CloudCamp and on Tuesday. The workshop was held by Tim Dysinger and Andrew Shafer (see above).

The first day focused on building infrastructure, Tim introduced us to vagrant and Chef. The idea was to use vagrant to manage VirtualBox VMs and Chef to configure the VMs to run OpenStack. Tim provided a vagrant project on GitHub which we used to deploy the VMs.

Unfortunately the first workshop day went by with playing around on vagrant and Chef without touching anything OpenStack-related. I must say that I did not know vagrant before and it is a really cool tool. However, this "exposure" to Chef did not convince me to use Chef at work.

The second day shifted focus to actually deploying OpenStack and in the end we just followed the starter guide on the project homepage.

Bottom line is that I learned a lot, met new and interesting people. I hope the next workshop will focus more on OpenStack and less on Chef recipes.

    2011-11-18

    Silicon Berlin


    What do these companies have in common?





    (bought by Nokia in 2006)






    (bought by Ebay)











    Well, they are all in Berlin!

    Berlin is one of the most vibrant places on earth where new web companies settle and thrive. Berlin attracts not only people from Germany but from all over the world. Especially English speakers are welcome in Berlin and find a lot of like-minded people here.

    Investors are flocking to Berlin and almost every day or week we seen new start ups. Even the big players are coming now, as the recent press reports (sorry, mostly in German):
      Also earlier reports mention Berlin as the European Silicon Valley:
      Berlin also happens to have the most startup companies in Berlin, as deutsche-startups.de found out:
      Source: Originally by deutsche-startups.de, now found on internetworld.de and frische-fische.de.
      I was not able to determine about which year this statistics are, so take it for illustration.
      Velocity Conference Linking Silicon Valley and Silicon Berlin

      But the event that really links the Silicon Valley to Silicon Berlin is the Velocity Web Performance and Operations Conference, that leads the WebOps community in the Silicon Valley. In November 2012 the Velocity Europe came to Berlin for the first time with a great success both in numbers and quality content.

      About 40% of the Velocity attendants where from Germany, we don't yet have a number on the participants from Berlin. The conference managed to bring the culture and feeling of the Silicon Valley to Berlin. I never met so many people from the various German communities at a single event as I did at the Velocity!

      In my opinion the Velocity is an important step in linking Berlin to the Silicon Valley to support the development of a vibrant Web community in Berlin. I hope the next Velicity EU will be again in Berlin, everybody I talked to also thinks the same.

      Update 2011-11-22: Added more links to Tagesspiegel and  techcrunch
      Update 2011-12: Silicon Berlin bei ImmobilienScout24
      Update 2012-02: See Silicon Berlin Update  and report about Silicon Berlin talk at Berlin Devops meetup.

      2011-11-15

      Google Apps - Marketplace Addons you really need

      I migrated my domain to Google Apps and would like to share some of my experience.

      One of the great features is IMHO the Enterprise Marketplace where you can find lots of really useful (and even more of useless) Apps to add to your domain. Here is a selection of apps I tried or at least noticed.

      Must Have

      • Doodle integrates really nice into Google Apps and fulfils all dreams about scheduling meetings or asking several people about their opinion on something
      Useful
      • Lucid Charts has a free account which is more than sufficiant for most simple drawing needs. The shape library is rather simple, e.g. no floor plan shapes. Diagrams are limited to 60 objects each.
      • PresentOnlineNow gets the job done with regard to online presentations and comes for free for up to 10 participants in a meeting room. Only downside is that screen sharing does not work from Linux. Positive is that the participants don't need to install anything, it requires only Flash.
      Might Be Interesting
      These apps seem interesting, I did not yet have time to check them out:
      • Organizer is the cure for Email overload
      • Bookmarks Version 2 (FREE) centralizes bookmark management and sharing, free version covers 500 bookmarks per domain.
      • Exxtensions allows to send a Fax from Google Docs (and possibly also local files in the future). Prices seem competitive. BEWARE: THEY SEND AN EMAIL TO ALL DOMAIN USERS! I talked to the support and the promised that the next release (early 2012) would include customizable Sender ID which is currently missing. The support thanked me for voicing my opinion but did not state when they would stop to send unsolicited emails.
      • Sherpa Tools might be interesting to manage shared contacts. Did not try it out because it does not work on Apps For Free (according to the reviews).
      • Shared Groups. Reviews sound good and really affordable at 10€ per year. Did not try because eventually I don't need shared contacts for a family domain (yet).
      • Lend-Items: Keeping track of items on loan. Free plan allows adding 100 items to your catalogue.
      • SurveyAngle: Online surveys, forms, polls, and questionnaires.
      • Gbridge: Free instant VPN for everyone using Windows. With Desktop and File sharing.
      • Gliffy Diagrams and Flowcharts also offers a free account but seems more restricted compared to Lucid Charts. OTOH it has some really nice shapes like for a floor plan. Unfortunately with the floor plan one cannot set the sizes precisely.

      Things That Looked Good but Did Not Work Well
      • Shared contacts. Even if you get them filled in it is no big help.
      • iClippy promises to solve all screen shot problems but fails to support Linux and is limited to 800x600 in the free plan


      2011-11-08

      VEU 2011 Birds of a Feather was great - see you all next year!

      I would like to thank everybody who helped with the Birds of a Feather event yesterday.

      It was really great, at the start we had about 30 people and managed to fill up all the talk slots and also had a few discussions.

      Till the evening some more people arrived so that we where about 70 for the dinner.

      The party was also joined by participants of the CouchConf, which happened to be on the same day.

      I hope that everybody enjoyed it and will be back next year.

      2011-10-27

      Velocity Europe 2011 Birds of Feathers

      The Velocity Europe 2011 is coming closer, only two weeks to go!


      If you have not yet signed up, do so now and use the discount code veu11pcy to get 20% discount off the regular price. The code can be entered on the Velocity registration page.

      We managed to put together a Birds of Feathers event just on the afternoon before the velocity. This will serve as a warm-up to the Velocity and provide the open space that the main Velocity does not have this time. Meet the vibrant Berlin web community at the Betahaus, a well-known location for start up companies.

      Attendance is free for Velocity attendees! As is the dinner & party after the event :-)

      Go to the Velocity Eruope 2011 Birds of Feathers event page at veubof2011.eventbrite.co.uk to see all the details and sign up, the event is limited to 150 participants.

      2011-10-23

      Sukkot 5772

      Sukkot was really great this year. We had lots of guests, lots of food (no wonder when the holidays fall on Thursday and Friday) and, most of all, perfect weather. It was not too cold, mostly even above 10° C during the day and we could sit in the sukkah every day and evening! I don't remember such a good weather since we came to Germany, so this is really remarkable.
      Our small balcony sukkah
      Come and visit us next year!

      2011-10-04

      GPU Computing - Know When to Use It

      I recently attended the first TechTalk of Telekom' DeveloperGarden in Berlin. It was about GPU computing, a topic that relates to GPU-based image scaling as Apache module. The speaker, Guido Juckeland, is a heavy user of GPU computing and works at The Center for Information Services and High Performance Computing at the Dresden Technical University.

      The talk was quite interesting for GPU computing newbies like me. It gave a solid overview and showed the different programming styles that can be used (CUDA and OpenCL). It conveyed a lot of best practice with regard to GPU computing and several important points to consider before starting.

      The most important aspect is to keep the PCIe bottle-neck in mind:  Moving data between the CPU and the GPU is very slow compared to moving it between the main memory and the CPU or the GPU and its memory. Therefore, in most cases it is not efficient to offload a computation to the GPU if the data is used only once. The speedup of the GPU-based calculation always needs to make up for the extra tansfer time needed for copying the data to the GPU and the results back from the GPU.

      The second part of the talk showcased the current project that Guido is working on: A GPU-based simulation of a laser pulse passing a cloud of hydrogen molecules.


      The event was hosted at the C-Base in Berlin and it even was non-smoking for the evening. There where about 50 attendants and it was a good start of the DevelopGarden TechTalk series in Berlin.
      The next TechTalk will be on November, 24th and I will be speaking about DevOps at ImmobilienScout24.

      Additional Links:


      2011-09-23

      Velocity Web Performance and Operations Conference Europe in Berlin: Register Now and Save Money!

      The Velocity EU website is up and my talk is Be fast or stay behind – Building a Continuous Delivery Platform (together with Ingmar Krusch). The conference is on November 8th and 9th in Berlin.

      BTW, this Berlin conference was first announced at the Velocity US in Santa Clara in an interview:

       

      I hope to see you there, it will be a lot of fun! Make sure to order your ticket now while it only costs 849€ (Best Price till September 28th) compared to the 1095€ (full price)

      2011-09-12

      Peer-2-Peer Backup Ideas

      At the Desktop Summit I stood almost 2 hours in the yard talking to Michael Bell and amongst many other things we got an idea about how to solve the problem of Desktop backups in large environments.

      The basic idea is to go away from a centralized approach and solve the problem with a peer-2-peer solution. The main benefit is scale-out vs. scale-up: As the amount of Desktops that require a backup grows, so does the amount of Desktops that provide backup space.

      2011-08-02

      Velocity Conference Europe 2011 - Call for Papers till August 9th!

      The Velocity Conference comes to Europe. But we don't want to have a clone of the US conference, we want to have a real European version of the conference.




      We need your help!

      The Call for Papers is open, please come and submit your paper to the Velocity Conference about:

      • NoSQL
      • Mobile performance
      • Effective cloud computing
      • JavaScript speedups
      • TCP, HTTP, and SSL optimizations
      • Metrics and monitoring
      • Impact on the bottom line
      • DevOps
      • Web Operations Culture
      One of the benefits of submitting a proposal is solid chance at getting a free ticket to the conference!

      The conference will be on November 8th and 9th, 2011 in Berlin. Important dates are:
      • Submission deadline for proposals is 9 August, 2011
      • Early Registration opens in September 2011
      • Standard Registration begins in October 2011

      2011-06-28

      YADT Server released

      My colleagues at ImmobilienScout24 released the main part of our new deployment solution YADT - an Automated Deployment Tool. Check out the code at our Google code homepage.

      Documentation and proper packages will follow soon.

      2011-06-16

      Velocity 2011 - Part 3: Wednesday (2nd day)

      My notes on the second conference day at the Velocity Conference.

      The keynotes where again a highlight, to be topped only by the talk about Automating for Success: Production Begins in Development which happened to confirm all my theories about web operations and package-based deployment  :-)

      Videos are available on the Velocity 2011 Videos page, slides can be found on the Velocity 2011 Speakers Slides and Video page.

      Read also about the Workshops and the first day.

      2011-06-15

      Velocity 2011 - Part 2: Wednesday (1st day)

      My notes on the first conference day at the Velocity Conference.

      The keynotes where really good, especially the first one about the right mentality for web operations and how to build a career in web operations.

      Videos are available on the Velocity 2011 Videos page, slides can be found on the Velocity 2011 Speakers Slides and Video page.

      Read also about the workshops and the second day.

      2011-06-14

      Velocity 2011 - Part 1: Workshops

      My notes on the workshop day at the Velocity Conference.

      A lot of Chef stuff, but of course OpsCode was a sponsor. Real gems where Decisions in the Face of Uncertainity and Advanced post-mortem Fu and Human Error 101.

      Read also about the first and second day.

      2011-06-03

      Limmud Festival 2011: Challah Baking

      Nadia, best of all wives in the world and my wife, gave a Challah baking workshop at the Limmud Festival today:
      Putting it all into 1:15h was not so simple but Nadia managed and gave the participants a fast-paced and rich run-down of our Shabbat preparations.
      The participants where able to take not only their own self-made challah with them but also Nadia's famous bread-machine challah recipe and a handout about hafrashat challah (the separation of challah).

      Continue reading for the full recipe...

      2011-05-30

      Mailserver Konferenz 2011: Mail ist nicht kompliziert, nur sehr komplex

      Am 26. und 27. Mai fand in Berlin die 5. Mailserver Konferenz bei Heinlein Support statt. Eine Konferenz von und für Spezialisten, war es für mich lohnen und hat mal wieder bestätigt, dass das gesamte Thema Email zwar nicht sehr kompliziert ist, in seiner Gesamtmasse and Details aber ziemlich komplex. Da in der iX mein Konferenzbericht erscheint, gibt es hier jetzt nur meinen Mitschrieb in Rohform.

      2011-05-19

      Configuration Management with Subversion and RPM

      Today I released the first part of our new Open Source Project YADT - an Augmented Deployment Tool. The configuration management with the help of RPM packages and a subversion repository is now available via the project repository.

      At the Subversion Day Berlin 2011 I gave a talk about how we do configuration management with RPM packages at ImmobilienScout24:
      Me showing the ImmobilienScout24 recruiting slide (Picture courtesy of H. Klotz)
      Download the slides as PDF for details: Subversion Day Berlin 2011 - Configuration Management with Subversion and RPM.pdf

      Read my conference reports:

      2011-05-11

      Lab Manager Light: Self Service Virtualization as private Cloud

      I just gave my talk at the LinuxTag 2011 about Self Service Virtualization as private Cloud and my new Open Source project Lab Manager Light.

      From the audience of about 80-100 people there where quite a few interesting questions and also several "I wish you had done that last year" comments.

      Download the slides: LinuxTag 2011 - Lab Manager Light - Self Service Virtualization as a Private Cloud.pdf

      2011-05-06

      Who is responsible for sustainability in IT projects?

      After I got very worked up again yesterday about people asking for "quick help" in setting up something and nobody thinking about the long term sustainability of IT operations I would like to point out a few facts about this topic:

      The major part of IT project costs are not related to the initial setup!

      Operations, maintenance, updates, bugfixes, application administration, backup and disaster recovery exercises make up the majority costs and should not be neglected in the initial planning.

      Actually, the more a project is meant to be temporary, the longer it will run and the more it will cost in the end.

      To underline my personal beliefs in this matter I found the following links on the Internet, I am sure that with some time on Google you can find even more:
      Therefore I would like to suggest to alway strive for a simple solution and take the follow-up costs like maintenance, updates, security fixes etc. into account during the planning stage.

      This is also a golden opportunity to include the original requester in the discussion about TCO and about who will actually spend their time on the follow-up tasks.

      2011-04-13

      1st devops Berlin meetup

      On Tuesday, April 5th 2011, there was the first meeting of the devops Berlin XING group at ImmobilienScout24 where I also work.

      About 20 people showed up for an interesting mix of talks and open space. I got some very interesting and promising contacts and was happy to meet several people whom I had not seen for a long time.

      My own talk was about Package Deployment — Building Bridges between Dev and Ops and is available as a PDF download. The technology that we are building is available at the YADT project homepage and currently undergoing some rework before we can release it to the public.

      2011-04-01

      XEN PV VM on VMware ESX

      Seems like a total waste of time without practical use? Well, it is — unless you need to test drive an idea that will be rolled out in your production data center running XEN PV but you have only VMware ESX as a playground.

      Since I spent a little time putting the pieces together I will share my experience based on CentOS 5.5. Basically it all went es expected, I only had some trouble to get the XEN PV VMs into the regular network via bridging versus the NAT which is the default that comes with CentOS.

      Basic Setup:
      • Configure your VMware VM to allow promiscuous networking:
        • Allow promiscuous networking on the vSwitch
        • Add this line of manual configuration to the Extended Configuration of the VM:
          ethernet0.noPromisc=“FALSE”
      • Install regular CentOS and add the group "Virtualization":
        yum groupinstall Virtualization
      • Make sure that there is enough free disk space in /var/lib/xen/images to store your virtual disk images.
      Network Setup: 
      •  Edit /etc/libvirt/qemu/networks/default.xml to look like this:
        <network>
           <name>default</name>
           <uuid>b8d5fcff-599b-48ce-a8fc-fdcf34506711</uuid>
           <bridge name='lan' stp='on' forwardDelay='0' />
        </network>
        lan is the name of our virtual bridge, keep your original UUID in place.
      • Edit /etc/xen/xend-config.sxp:
        • change the network-script line to look like this:
          (network-script /bin/true)
          this disables the network setup by XEN because we let CentOS do it instead.
        • change the vif-script line to look like this:
          (vif-script 'vif-bridge bridge=arc_int')
          this configures XEN to also use our virtual bridge
      • Edit the CentOS network-scripts for bridging:
        • Create /etc/sysconfig/network-scripts/ifcfg-lan like this:
          DEVICE=lan
          TYPE=Bridge
          BOOTPROTO=dhcp
          ONBOOT=yes
          DELAY=0
          NM_CONTROLLED=no
        • Edit /etc/sysconfig/network-scripts/ifcfg-eth0 like this:
          DEVICE=eth0
          HWADDR=00:50:56:93:00:02
          BRIDGE=lan
          ONBOOT=yes
          Leave the MAC address as it was to match your NIC.
      • Reboot system to test if network configuration is correct
      Basically that was all I had to do to use XEN PV VMs on VMware ESX. You can manage the VMs as usual with virt-manager, virsh and xm.

      The only drawback of this setup is the the low performance of the XEN VM, but for testing purposes and to develop scripts and automation this does not matter and I was happy to skip ordering and setting up a hardware server just to test a few scripts.

        2011-03-09

        Ubuntu Berlin Stammtisch

        Sitze gerade beim Ubuntu Berlin Stammtisch, und kann mal was dazu erzählen.

        Hier sind etwa 40 Leute, unklar wie viele zum Stammtisch kamen. Das ganze findet in der C-Base statt, aber das ist eine Thema für sich. Schade, dass da viel geraucht wird, damit ist die Veranstaltung wohl für Leute unter 18 tabu. Und es stinkt weil kaum Lüftung.

        Die erste Stunde war eher langweilig, es haben sich einige Grüppchen gebildet aber nix gemeinsames für alle. Scheinen aber alles echt nette Leute zu sein.

        Hab dann festgestellt, dass der Mensch neben mir quasi Konzernkollege ist und schon gab es ein spannendes Gespräch und einen interessanten Kontakt.

        Also alles in allem ein lohnender Abend und klare Empfehlung, mal wieder zu kommen. Hoffentlich ist der Laden dann auch rauchfrei.

        2011-01-14

        GPU-based image scaling as Apache module

        I just got an idea that it might be interesting to combine mod_dims and GPU accelerated image manipulation libraries like NVidia's NPP. Apparently nobody looked into this so far, even though a google search will show a lot of GPU related image processing stuff, NVidia also provides a nice intro to GPU image processing.

        The idea is that if it would be possible to service scaled images as fast as unscaled images then it would be possible to skip all the solutions that deal with the caching of scaled images and just scale the images always on demand. This depends on the ability to service scaled images really fast, for example with the help of some Tesla boards for the image scalers. It would be an interesting comparison to compare the cost of sufficiently available storage for caching plus the cost for elastic on-demand scaling with the cost of permanent on-demand scaling with GPU acceleration.

        The advantage of the latter would be, if the costs are in the same region, that it would be a way to be independent of external resources and keep everything in our own data centre.

        I am therefore looking for people who like this idea and would like to join in a Proof-Of-Concept implementation of this idea (first with the help of commodity NVidia GPUs in a desktop) to get a first taste of the power of GPU accelerated image scaling. Please contact me if you want to join me in this attempt.

        Update: See also GPU Computing - Know When to Use It.