chore: add docs re docker inside envbuiler-built-envs

coder · johnstcn · May 15, 2024 · May 15, 2024 · May 15, 2024 · May 15, 2024
commit 502621d289ad883948f852512be503f52cafb20e
diff --git a/README.md b/README.md
@@ -134,6 +134,11 @@ Provide the encoded JSON config to envbuilder:
 DOCKER_CONFIG_BASE64=ewoJImF1dGhzIjogewoJCSJodHRwczovL2luZGV4LmRvY2tlci5pby92MS8iOiB7CgkJCSJhdXRoIjogImJhc2U2NCBlbmNvZGVkIHRva2VuIgoJCX0KCX0KfQo=
 ```
 
+### Docker-in-Docker
+
+See [here](./docs/docker.md) for instructions on running Docker containers inside
+environments built by Envbuilder.
+
 ## Git Authentication
 
 Two methods of authentication are supported:

diff --git a/docs/docker.md b/docs/docker.md
@@ -0,0 +1,109 @@
+# Docker inside Envbuilder
+
+There are a number of approaches you can use to have access to a Docker daemon
+from inside Envbuilder:
+
+## Docker Outside of Docker (DooD)
+
+**Security:** None
+**Convenience:** High
+
+This approach re-uses the host Docker socket and passes it inside the container.
+It is the simplest approach, but offers **no security** -- any process inside the
+container that can connect to the Docker socket will have access to the
+underlying host.
+Only use it if you are the only person using the Docker socket (for example, if
+you are experimenting on your own workstation).
+
+Example:
+
+```shell
+docker run -it --rm \
+    -v /tmp/envbuilder:/workspaces \
+    -e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
+    -e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/01_dood \
+    -e ENVBUILDER_INIT_SCRIPT=bash \
+    ghcr.io/coder/envbuilder:latest
+```
+
+
+## Docker-in-Docker (DinD)
+
+**Security:** Low
+**Convenience:** High
+
+This approach entails running a Docker daemon inside the container.
+This requires a privileged container to run, and therefore has a wide potential
+attack surface.
+
+Example:
+
+> Note that due to a lack of init system, the Docker daemon
+> needs to be started separately inside the container. In this example, we
+> create a custom entrypoint to start the Docker daemon in the background and
+> call this entrypoint via `ENVBUILDER_INIT_SCRIPT`.
+
+```shell
+docker run -it --rm \
+    --privileged \
+    -v /tmp/envbuilder:/workspaces \
+    -e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
+    -e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/02_dind \
+    -e ENVBUILDER_INIT_SCRIPT=/entrypoint.sh \
+    ghcr.io/coder/envbuilder:latest
+```
+
+This can also be accomplished using the [`docker-in-docker` Devcontainer feature](https://github.com/devcontainers/features/tree/main/src/docker-in-docker).
+
+## Rootless DinD
+
+**Security:** Medium
+**Convenience:** Medium
+
+This approach runs a Docker daemon in *rootless* mode.
+While this still requires a privileged container, this allows you to restrict
+usage of the `root` user inside the container, as the Docker daemon will be run
+under a "fake" root user (via `rootlesskit`). The user inside the workspace can
+then be a 'regular' user without root permissions.
+
+> Note: Once again, we use a custom entrypoint via `ENVBUILDER_INIT_SCRIPT` to
+> start the Docker daemon via `rootlesskit`.
+
+Example:
+
+```
+docker run -it --rm \
+    --privileged \
+    -v /tmp/envbuilder:/workspaces \
+    -e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
+    -e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/03_dind_rootless \
+    -e ENVBUILDER_INIT_SCRIPT=/entrypoint.sh \
+    ghcr.io/coder/envbuilder:latest
+```
+
+## Docker-in-Docker using Sysbox
+
+**Security:** High
+**Convenience:** Low for infra admins, high for users
+
+This approach requires installing the [`sysbox-runc` container
+runtime](https://github.com/nestybox/sysbox/blob/master/docs/user-guide/install-package.md).
+This is an alternative container runtime that provides additional benefits,
+including transparently enabling Docker inside workspaces. Most notably, it
+**does not require a privileged container**, so you can allow developers root
+access inside their workspaces, if required.
+
+Example:
+```
+docker run -it --rm \
+    --privileged \
+    -v /tmp/envbuilder:/workspaces \
+    -e ENVBUILDER_GIT_URL=https://github.com/coder/envbuilder \
+    -e ENVBUILDER_DEVCONTAINER_DIR=/workspaces/envbuilder/examples/docker/02_dind \
+    -e ENVBUILDER_INIT_SCRIPT=/entrypoint.sh \
+    --runtime sysbox-runc \
+    ghcr.io/coder/envbuilder:latest
+```
+
+For further information on Sysbox, please consult the [Sysbox
+Documentation](https://github.com/nestybox/sysbox/blob/master/docs/user-guide/README.md).
diff --git a/examples/docker/01_dood/Dockerfile b/examples/docker/01_dood/Dockerfile
@@ -0,0 +1,2 @@
+FROM ubuntu:latest
+RUN apt-get update && apt-get install -y docker.io
diff --git a/examples/docker/01_dood/devcontainer.json b/examples/docker/01_dood/devcontainer.json
@@ -0,0 +1,5 @@
+{
+  "build": {
+    "dockerfile": "Dockerfile"
+  }
+}
diff --git a/examples/docker/02_dind/Dockerfile b/examples/docker/02_dind/Dockerfile
@@ -0,0 +1,6 @@
+FROM ubuntu:latest
+RUN apt-get update && \
+  apt-get install -y curl apt-transport-https && \
+  curl -fsSL https://get.docker.com/ | sh -s -
+ADD entrypoint.sh /entrypoint.sh
+ENTRYPOINT ["/entrypoint.sh"]
diff --git a/examples/docker/02_dind/devcontainer.json b/examples/docker/02_dind/devcontainer.json
@@ -0,0 +1,5 @@
+{
+  "build": {
+    "dockerfile": "Dockerfile"
+  }
+}
diff --git a/examples/docker/02_dind/entrypoint.sh b/examples/docker/02_dind/entrypoint.sh
@@ -0,0 +1,7 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+nohup dockerd > /var/log/docker.log 2>&1 &
+
+exec bash --login
diff --git a/examples/docker/03_dind_rootless/Dockerfile b/examples/docker/03_dind_rootless/Dockerfile
@@ -0,0 +1,22 @@
+FROM ubuntu:latest
+## Install prerequisites
+RUN apt-get update && \
+  apt-get install -y apt-transport-https curl iproute2 uidmap
+# Install Docker daemon
+RUN curl -fsSL https://get.docker.com/ | sh -s -
+# Add the ubuntu user to the Docker group
+RUN usermod -aG docker ubuntu
+# Add our custom entrypoint
+ADD entrypoint.sh /entrypoint.sh
+RUN chmod +x /entrypoint.sh
+# Create the XDG_RUNTIME_DIR for our user and set DOCKER_HOST
+ENV XDG_RUNTIME_DIR /run/user/1000
+RUN mkdir -p ${XDG_RUNTIME_DIR} && chown ubuntu:ubuntu ${XDG_RUNTIME_DIR}
+ENV DOCKER_HOST unix:///${XDG_RUNTIME_DIR}/docker.sock
+# Setup rootless mode as the ubuntu user.
+USER ubuntu
+RUN dockerd-rootless-setuptool.sh install
+# Switch the ubuntu user to use the rootless docker context
+RUN docker context use rootless
+RUN mkdir -p /home/ubuntu/.local/share/docker
+ENTRYPOINT ["/entrypoint.sh"]
diff --git a/examples/docker/03_dind_rootless/devcontainer.json b/examples/docker/03_dind_rootless/devcontainer.json
@@ -0,0 +1,5 @@
+{
+  "build": {
+    "dockerfile": "Dockerfile"
+  }
+}
diff --git a/examples/docker/03_dind_rootless/entrypoint.sh b/examples/docker/03_dind_rootless/entrypoint.sh
@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+# Start the rootless docker daemon as a non-root user
+nohup rootlesskit --net=slirp4netns --mtu=1500 --disable-host-loopback --port-driver=builtin --copy-up=/etc --copy-up=/run dockerd > "/tmp/dockerd-rootless.log" 2>&1 &
+
+exec bash --login
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		FROM ubuntu:latest
		RUN apt-get update && apt-get install -y docker.io