So how to make it all work. Rather than wait for my new hardware, I thought I might as well start with the hundred or so Raspberry Pi Model Bs we already have. Here’s a recipe for how to get it all running:

Hypriot: The Operating system

Hypriot are a great bunch and they have integrated Raspbian with docker installation. Their latest working system is something called Hector using Docker 0.9. It works great on our old Raspberry Pi Model Bs.
Note. is that docker 1.1 does not run on ARMv6, so you need to stay on docker 0.9 for now.
They have in development their next release. A big change from the previous Hypriot Cluster/Hector I used. You can download it yourself and then run a dd if=foo.img of=/dev/rdisk bs=1M Make sure to edit the occidentalist.txt file to make a different hostname for each raspberry pi sd card. Or you can use their flash utility. Right now that thing see to have a bug. Mac’s want bs=1M and not a lower case m.
The future. There you had to carefully name all the SDs differently. It is super easy to download, just get their flash utility and point it at a url which today is flash https://github.com/hypriot/image-builder-rpi/releases/download/v0.2.5/sd-card-rpi-v0.2.5.img.zip but of course that has lots of issues, right now they recommend staying on docker 1.9 and their older release of cluster lab.

Overclocking

This isn’t so much of an issue for the newer Raspberry Pi 2, but the Model B only runs at 700Mhz, so an overclock to 1GHz is super helpful and a real speed difference. The parameters are pretty easy in the config
 

Camera support

One details is that hypriot does not by default enable the camera or ship with raspi-config, so you have to manually change the /boot/config.txt image. Some important options are gpu_mem if you are running gstreamer and start_x=1 means turn the camera on.
There doesn’t seem much interest, but there are three possible containers I found that seem to try to get the raspberry pi camera working in a container:

neileyde/rpi-webcam

He has the best documentation but didn’t publish the docker file however it does seem to work! We start it on a particular node. At first it didn’t like the node constraints, but this seems like some sort of glitch with consul.
Now to test run:

# assuming rpi-0 is the master and the cameras are rpi-{1-4}
docker run -p 8080:8080 --privileged --env="constraint:node=rpi-1" --name=webcam-1 nieleyde/rpi-webcam

Now try the to open http://rpi-1.local:8080 and you should see a basic status json. and http://localhost:8080/interface will give you the front screen and it will start to stream video. The light should go on after about 20 seconds and you should see the image. It will update about once every few seconds.
One interesting bug is that if you try to run raspistill while another camera app is running  it will hang the GPU. Even if you try a reboot it doesn’t seem to help so you should instead power cycle. This still hasn’t worked, so it could be a hardware problem. Still it is working on 3 of 4 nodes.
Here are the other things to try.

• arijusg/rpi-camera. This has a Dockerfile so we can at least see how he compiles and gets raspistill to work and there is a comment on the github which shows what else needs to get added to the container to get raspistill up and it looks like the main problem is that /opt/vc/bin isn’t in the path or linked into /usr/bin.
• monsendag/rpi-cameraserver. Haven’t tried this one yet.
• beck/rpi-webcam. I can’t tell but could be a pull from neileyde

Building from source

Well the basic conclusion is that these libraries basically do not work. Not enough effort on them. So I guess I’m back to figuring out how to build it from source. There are lots of guides that cover this but thinkrpi seems the most complete. I’m going to just compile on the Raspberry Pi for now as cross compiling is definitely complicated and I never really understood QEMU etc.

1. You need to git clone raspberry-pi/userland and compile it
2. This leaves raspistill and raspivid in /opt/vc/bin which needs to go into your path. Raspbian by the way has a symbolic link from /usr/bin for raspistill,vid,iyuv, etc.

Consul vs Kubernetes: Orchestration

You need this layer to make it easy to start and stop jobs on the system. Specifically it would be nice to have something that runs a job per actual physical node and tells you want is up or down.
Consul via Hypriot Cluster Lab. Their cluster lab is a bit old, but integrated consul with docker swarm so you can do cluster assignment with a simple constraint. Note that when you do a docker pull, it will actually download the same image across the entire cluster. So longer term, you probably want the things to nfs boot so you just have one image for all of them. You have two graphical views as well. Consul shows what nodes are running on port 8500. You can also run hypriot/rpi-dockerui to see graphically what containers you have up on port 9000.
export DOCKER_HOST=0.0.0.0:2378
docker pull arijusg/rpi-camera
docker run --env="constraint=node==rpi-0" hypriot/armhf-run-hello-world