0x8490 Privileged Swarm Services
Some docker images require extra capabilities to work, ie:
- openvpn needs
- anything using USB/i2c will need
Swarmkit does not support that (nor the GIEF IT ALL
There are a lot of issues on their github(s) such as this, this or this.
It seems there is consensus into adding the feature to swarmkit though it won’t be available before 19.06 or 19.09.
Until then, the best solution seems to be spinning up a service that starts the container after, such as:
version: '3.1' services: web: image: docker:19.03 command: sh -c "docker pull fopina/octoprint && docker run --rm --name octoprint_real -v /nfs/path/to/octoprint/config:/root/.octoprint --device=/dev/ttyAMA0 -p 5000:5000 fopina/octoprint" volumes: - /var/run/docker.sock:/var/run/docker.sock - /root/.docker:/root/.docker:ro deploy: mode: replicated replicas: 1 placement: constraints: [node.hostname == printingPi]
Why would you use swarm service for this, since it’s bound to the specific node where the USB cable is connected?
In this case, it’s managing all services the same way, I don’t want to deploy some with
docker-compose and others with
docker swarm deploy…
But in the case of, let’s say openvpn, it could be useful for easily scaling up across different nodes.
We could use a stack like the first:
version: '3.1' services: vpnd: image: docker:19.03 command: docker run --rm --name openvpn_real --cap-add NET_ADMIN -p 443:443/udp -v /nfs/path/to/openvpn/:/etc/openvpn/ fopina/openvpn volumes: - /var/run/docker.sock:/var/run/docker.sock deploy: mode: replicated replicas: 1 placement: constraints: [node.role == manager]
But as openvpn is not running in the actual service container, the port is not published in the swarm VIP / routing mesh. If we can scale it up, it needs to end in different nodes (as there’s an host bound port) and to make use of the replicas, we need to use the node IP (not any node IP as the usual routing mesh)…
My overkill solution:
- adding some proxy - just used socat in my example
- write an entrypoint script that
- starts docker container (without binding port) attached to a common network with the service container
- extract the IP from the new container
- launch proxy using the same port
- attach to container (so that main service process is the container itself, not the proxy)
- just use that port in the service description as if it was any other service
To accomplish this, I’ve created the image fopina/swarm-service-proxy (built from this)
The stack definition using this is
version: '3.1' services: vpnd: image: fopina/swarm-service-proxy:1 command: --rm --cap-add NET_ADMIN -v /nfs/path/to/openvpn/:/etc/openvpn/ fopina/openvpn volumes: - /var/run/docker.sock:/var/run/docker.sock environment: PROXIED_PORT: 443 PROXIED_PROTO: udp PROXIED_NAME: openvpn_real ports: - 443:443/udp deploy: mode: replicated replicas: 1
With this, port is actually published from the service (not to the secondary container) so it makes use of the routing mesh (and it doesn’t lock the port in the docker host where it lands).
Syntax remained as close as possible to the initial version: command is used as if it was
docker run <command>.
PROXIED_PORTis the internal port of the secondary container.
socatshould use the UDP labels (instead of TCP).
PROXIED_NAMEto choose the name prefix of the secondary container - yes, prefix as random string is appended to it to make sure it will not collide if scaled up.
- Do not forget to bind the port you need in the service
- Do not publish any port on the secondary container (as it won’t be used)
So it is now fully managed as it was a service and fully scalable.