Docker on SmartOS, the harder way
SmartOS supports running Docker containers through Triton (SmartDataCenter). However I don’t have an entire datacenter at my disposal and the recommended specs for Triton are a bit more than the hardware I have available. I just want to run Docker containers on a single machine.
A little bit of background: SmartOS supports running native Linux binaries in LX-branded zones through their Linux emulation. This is the base for regular LX-branded zones (behaves like a normal Solaris zone) and thus also allows for pulling images from the Docker Hub and running them unmodified.
My initial plan was to run docker in an LX-branded zone, well that escalated quickly (output from an Alpine image):
ERRO[0000] 'overlay' not found as a supported filesystem on this host. Please ensure kernel is new enough and has overlay support loaded.
INFO[0000] Graph migration to content-addressability took 0.00 seconds
WARN[0000] Running modprobe bridge br_netfilter failed with message: modprobe: can't change directory to '/lib/modules': No such file or directory , error: exit status 1
WARN[0000] Running modprobe nf_nat failed with message: `modprobe: can't change directory to '/lib/modules': No such file or directory`, error: exit status 1
FATA[0000] Error starting daemon: Error initializing network controller: error obtaining controller instance: Cannot read IP forwarding setup: open /proc/sys/net/ipv4/ip_forward: no such file or directory
However it was pointed out to me that this had been discussed on the mailinglists already, so lets move on.
imgadm + vmadm =~ docker⌗
Then I remembered that imgadm(1M) add an option to support importing images from the Docker Hub, and down the rabbit hole I went.
First add the Docker Hub to the list of available sources for imgadm(1M) to fetch from:
[root@lumiere ~]# imgadm sources --add-docker-hub
imgadm avail doesn’t work against the Hub, so you’ll have to search the Hub manually. In my case it didn’t matter what I chose, as long as it started a service. So redis:3 it was:
[root@lumiere ~]# imgadm import redis:3
Looking at the image manifest shows some data we’ll have to re-use later. First lookup the UUID:
[root@lumiere ~]# imgadm list --docker
UUID REPOSITORY TAG IMAGE_ID CREATED
260f0a8a-4a0f-2452-6a14-083b44f8dd97 redis 3 56b1d94012a9 2016-02-24T23:15:43Z
Then run imgadm get to lookup the docker tags:
[root@lumiere ~]# imgadm get 260f0a8a-4a0f-2452-6a14-083b44f8dd97
{
[...]
"tags": {
"docker:repo": "redis",
"docker:id": "56b1d94012a94090c62b3b607c283737d0f508b0ecc6bbcbf9297b2876b86a95",
"docker:architecture": "amd64",
"docker:tag:3": true,
"docker:tag:3.0": true,
"docker:tag:3.0.7": true,
"docker:tag:latest": true,
"docker:config": {
"Cmd": [
"redis-server"
],
"Entrypoint": [
"/entrypoint.sh"
],
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
"REDIS_VERSION=3.0.7",
"REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-3.0.7.tar.gz",
"REDIS_DOWNLOAD_SHA1=e56b4b7e033ae8dbf311f9191cf6fdf3ae974d1c"
],
"WorkingDir": "/data"
}
},
[...]
}
Now it took me some time to figure out what the manifest had to look like for vmadm(1M), but eventually this did the job:
{
"brand": "lx",
"max_physical_memory": 512,
"image_uuid": "260f0a8a-4a0f-2452-6a14-083b44f8dd97",
"resolvers": [
"8.8.8.8"
],
"alias": "docker-redis",
"docker": true,
"kernel_version": "3.18.0",
"internal_metadata": {
"docker:entrypoint": "[\"/entrypoint.sh\"]",
"docker:cmd": "[\"redis-server\"]",
"docker:env": "[\"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin\", \"REDIS_VERSION=3.0.7\", \"REDIS_DOWNLOAD_URL=http://download.redis.io/releases/redis-3.0.7.tar.gz\", \"REDIS_DOWNLOAD_SHA1=e56b4b7e033ae8dbf311f9191cf6fdf3ae974d1c\"]",
"docker:workingdir": "/data",
"docker:workdir": "/data",
"docker:open_stdin": "true",
"docker:tty": "true"
},
"nics": [
{
"nic_tag": "admin",
"ip": "192.168.178.91",
"netmask": "255.255.255.0",
"gateway": "192.168.178.1"
}
]
}
There’s a bit more information in the repository fordockerinit. Note that most of the values were taken from the image manifest. I had to add open_stdin and tty to get the docker logs-equivalent output in /zones/$UUID/logs/stdio.log. Upon provisioning the zone dockerinit creates /var/log/sdc-dockerinit.log in the zone root which was quite invaluable in figuring out missing values.
I also noticed here that dockerinit looks for docker:workdir, whereas the imgadm manifest uses docker:WorkingDir.
After creating the zone, you can run vmadm update $UUID docker=true add some more docker specific variables.
Then simply vmadm start $UUID to start the zone; lo and behold, stdio.log has:
"log":"1:C 29 Feb 20:06:15.845 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf\r\n","stream":"stdout","time":"2016-02-29T20:06:15.846099000Z"}
{"log":" _._ \r\n _.-``__ ''-._ \r\n _.-`` `. `_. ''-._ Redis 3.0.7 (00000000/0) 64 bit\r\n .-`` .-```. ```\\/ _.,_ ''-._ \r\n ( ' , .-` | `, ) Running in standalone mode\r\n |`-._`-...-` __...-.``-._|'` _.-'| Port: 6379\r\n | `-._ `._ / _.-' | PID: 1\r\n `-._ `-._ `-./ _.-' _.-' \r\n |`-._`-._ `-.__.-' _.-'_.-'| \r\n | `-._`-._ _.-'_.-' | http://redis.io \r\n `-._ `-._`-.__.-'_.-' _.-' \r\n |`-._`-._ `-.__.-' _.-'_.-'| \r\n | `-._`-._ _.-'_.-' | \r\n `-._ `-._`-.__.-'_.-' _.-' \r\n `-._ `-.__.-' _.-' \r\n ","stream":"stdout","time":"2016-02-29T20:06:15.858764000Z"}
{"log":" `-._ _.-' \r\n `-.__.-' \r\n\r\n","stream":"stdout","time":"2016-02-29T20:06:15.858809000Z"}
{"log":"1:M 29 Feb 20:06:15.858 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.\r\n","stream":"stdout","time":"2016-02-29T20:06:15.858887000Z"}
{"log":"1:M 29 Feb 20:06:15.858 # Server started, Redis version 3.0.7\r\n","stream":"stdout","time":"2016-02-29T20:06:15.858938000Z"}
{"log":"1:M 29 Feb 20:06:15.858 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.\r\n","stream":"stdout","time":"2016-02-29T20:06:15.859019000Z"}
{"log":"1:M 29 Feb 20:06:15.859 * The server is now ready to accept connections on port 6379\r\n","stream":"stdout","time":"2016-02-29T20:06:15.859230000Z"}
That vaguely resembles the redis logo that went through a shredded, but it works:
cafayate:9134 ~ % redis-cli -h 192.168.178.91
192.168.178.91:6379> ping
PONG
192.168.178.91:6379>
Finally, you can use dockerexec through zlogin(1):
[root@lumiere ~]# zlogin -i $UUID /native/usr/vm/sbin/dockerexec bash
[dockerexec debug log omitted]
root@d9c1455b-7a65-eaa1-f4d0-8fba1912e52d:/data# ps ax
PID TTY STAT TIME COMMAND
12689 ? Ss 0:00 bash
12585 ? Ssl 0:00 ipmgmtd
12692 ? R 0:00 ps ax
1 ? Sl 0:02 redis-server *:6379
root@d9c1455b-7a65-eaa1-f4d0-8fba1912e52d:/data#
Note there is no docker0 interface onto which the container is hooked, or all the magic that the docker daemon would take care of. The README for dockerinit already states that it’s experimental, but at least one is able to run vanilla images from the Docker Hub on SmartOS. Albeit with a bit of fiddling.