All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot] [PATCH 0/9] Upgrading Go to 1.9 and Docker to v2017.07
@ 2017-09-20  3:02 Christian Stewart
  2017-09-20  3:02 ` [Buildroot] [PATCH 1/9] dumb-init: new package Christian Stewart
                   ` (8 more replies)
  0 siblings, 9 replies; 16+ messages in thread
From: Christian Stewart @ 2017-09-20  3:02 UTC (permalink / raw)
  To: buildroot

This patch series upgrades Go to 1.9, and Docker to v2017.07.

Unfortunately the Go upgrade is a dependency of the new Docker version, as is
the various other patches for Docker included in this series.

I had previously submitted these changes as separate series, but as they were
never merged, I must include them in this series. They are dependencies of
Docker v2017.07 (having an init process, for example).

This series (including the container tiny init processes) has been tested
extensively against Buildroot 2017.02.3 and 2017.05 on the following platforms:

 - Raspberry Pi 3
 - Odroid XU4, HC1, U2, C2
 - Buildroot-in-docker (x86)

A build test has been completed against master, and we will be completing
runtime tests tomorrow.

Note that this series incorporates the change in repository and project
structure of the Docker project as it transitions from Docker Engine to
the "Moby" project. This means that:

 - Docker engine is split into the daemon and CLI
 - These two components are once again merged in docker/docker-ce
 - Docker-engine in Buildroot now targets docker/docker-ce, cli and daemon
 - Ce in this context stands for "community edition" or the open source Docker
 - The Moby repository and the docker/docker repo are no longer touched by BR.

Cheers!
Christian Stewart

--

Christian Stewart (9):
  dumb-init: new package
  tini: new package
  docker-engine: add support for init processes
  docker-engine: select static for tini and dumb-init
  docker-proxy: new package
  docker-engine: add dependency on docker-proxy
  runc: bump to v1.0.0-rc4
  docker-containerd: bump to 3addd84
  docker-engine: bump to v17.07.0-ce

 package/Config.in                                |  3 ++
 package/docker-containerd/docker-containerd.hash |  3 +-
 package/docker-containerd/docker-containerd.mk   | 13 +++---
 package/docker-engine/Config.in                  | 29 ++++++++++++++
 package/docker-engine/docker-engine.hash         |  3 +-
 package/docker-engine/docker-engine.mk           | 50 +++++++++++++++++-------
 package/docker-proxy/Config.in                   | 18 +++++++++
 package/docker-proxy/docker-proxy.hash           |  2 +
 package/docker-proxy/docker-proxy.mk             | 46 ++++++++++++++++++++++
 package/dumb-init/Config.in                      | 16 ++++++++
 package/dumb-init/dumb-init.hash                 |  2 +
 package/dumb-init/dumb-init.mk                   | 27 +++++++++++++
 package/runc/runc.hash                           |  2 +-
 package/runc/runc.mk                             |  5 ++-
 package/tini/Config.in                           | 21 ++++++++++
 package/tini/tini.hash                           |  2 +
 package/tini/tini.mk                             | 41 +++++++++++++++++++
 17 files changed, 257 insertions(+), 26 deletions(-)
 create mode 100644 package/docker-proxy/Config.in
 create mode 100644 package/docker-proxy/docker-proxy.hash
 create mode 100644 package/docker-proxy/docker-proxy.mk
 create mode 100644 package/dumb-init/Config.in
 create mode 100644 package/dumb-init/dumb-init.hash
 create mode 100644 package/dumb-init/dumb-init.mk
 create mode 100644 package/tini/Config.in
 create mode 100644 package/tini/tini.hash
 create mode 100644 package/tini/tini.mk

-- 
2.13.5

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 1/9] dumb-init: new package
  2017-09-20  3:02 [Buildroot] [PATCH 0/9] Upgrading Go to 1.9 and Docker to v2017.07 Christian Stewart
@ 2017-09-20  3:02 ` Christian Stewart
  2017-09-21 21:23   ` Arnout Vandecappelle
  2017-09-20  3:02 ` [Buildroot] [PATCH 2/9] tini: " Christian Stewart
                   ` (7 subsequent siblings)
  8 siblings, 1 reply; 16+ messages in thread
From: Christian Stewart @ 2017-09-20  3:02 UTC (permalink / raw)
  To: buildroot

dumb-init is a minimal init process to act as PID 1 for containers.

dumb-init can be used to satisfy docker-engine's docker-init binary
requirement.

In many cases docker-init backing processes like dumb-init must be static, as
they are executed inside a container in an unknown environment. This
patch includes an option to specifically compile dumb-init statically, even if
the system is compiled dynamically.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/Config.in                |  1 +
 package/dumb-init/Config.in      | 16 ++++++++++++++++
 package/dumb-init/dumb-init.hash |  2 ++
 package/dumb-init/dumb-init.mk   | 27 +++++++++++++++++++++++++++
 4 files changed, 46 insertions(+)
 create mode 100644 package/dumb-init/Config.in
 create mode 100644 package/dumb-init/dumb-init.hash
 create mode 100644 package/dumb-init/dumb-init.mk

diff --git a/package/Config.in b/package/Config.in
index 48d1381921..96f04dec7a 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1835,6 +1835,7 @@ comment "Utilities"
 	source "package/crudini/Config.in"
 	source "package/dialog/Config.in"
 	source "package/dtach/Config.in"
+	source "package/dumb-init/Config.in"
 	source "package/file/Config.in"
 	source "package/easy-rsa/Config.in"
 	source "package/gnupg/Config.in"
diff --git a/package/dumb-init/Config.in b/package/dumb-init/Config.in
new file mode 100644
index 0000000000..aaad119a15
--- /dev/null
+++ b/package/dumb-init/Config.in
@@ -0,0 +1,16 @@
+config BR2_PACKAGE_DUMB_INIT
+	bool "dumb-init"
+	help
+	  dumb-init is a simple but valid init binary to
+	  act as PID 1 for containers.
+
+	  https://github.com/Yelp/dumb-init
+
+if BR2_PACKAGE_DUMB_INIT
+
+config BR2_PACKAGE_DUMB_INIT_STATIC
+	bool "build static"
+	help
+	  Enables static compilation for dumb-init.
+
+endif
diff --git a/package/dumb-init/dumb-init.hash b/package/dumb-init/dumb-init.hash
new file mode 100644
index 0000000000..4705ce4368
--- /dev/null
+++ b/package/dumb-init/dumb-init.hash
@@ -0,0 +1,2 @@
+# Locally computed
+sha256	74486997321bd939cad2ee6af030f481d39751bc9aa0ece84ed55f864e309a3f	v1.2.0.tar.gz
diff --git a/package/dumb-init/dumb-init.mk b/package/dumb-init/dumb-init.mk
new file mode 100644
index 0000000000..cc24ffafa1
--- /dev/null
+++ b/package/dumb-init/dumb-init.mk
@@ -0,0 +1,27 @@
+################################################################################
+#
+# dumb-init
+#
+################################################################################
+
+DUMB_INIT_VERSION = v1.2.0
+DUMB_INIT_SITE = https://github.com/Yelp/dumb-init/archive
+DUMB_INIT_SOURCE = $(DUMB_INIT_VERSION).tar.gz
+DUMB_INIT_LICENSE = MIT
+DUMB_INIT_LICENSE_FILES = LICENSE
+
+ifeq ($(BR2_PACKAGE_DUMB_INIT_STATIC),y)
+DUMB_INIT_CFLAGS += -static
+endif
+
+define DUMB_INIT_BUILD_CMDS
+	$(TARGET_CC) \
+		$(DUMB_INIT_CFLAGS) \
+		-o $(@D)/dumb-init $(@D)/dumb-init.c
+endef
+
+define DUMB_INIT_INSTALL_TARGET_CMDS
+	$(INSTALL) -D -m 0755 $(@D)/dumb-init $(TARGET_DIR)/usr/bin/dumb-init
+endef
+
+$(eval $(generic-package))
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 2/9] tini: new package
  2017-09-20  3:02 [Buildroot] [PATCH 0/9] Upgrading Go to 1.9 and Docker to v2017.07 Christian Stewart
  2017-09-20  3:02 ` [Buildroot] [PATCH 1/9] dumb-init: new package Christian Stewart
@ 2017-09-20  3:02 ` Christian Stewart
  2017-09-20  4:40   ` Baruch Siach
  2017-09-20  3:02 ` [Buildroot] [PATCH 3/9] docker-engine: add support for init processes Christian Stewart
                   ` (6 subsequent siblings)
  8 siblings, 1 reply; 16+ messages in thread
From: Christian Stewart @ 2017-09-20  3:02 UTC (permalink / raw)
  To: buildroot

Tini is a minimal init process to act as PID 1 for containers.

Tini can be used to satisfy docker-engine's docker-init binary
requirement.

In many cases docker-init backing processes like Tini must be static, as
they are executed inside a container in an unknown environment. This
patch adds an option to specifically compile Tini statically, even if
the system is compiled dynamically.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/Config.in      |  1 +
 package/tini/Config.in | 21 +++++++++++++++++++++
 package/tini/tini.hash |  2 ++
 package/tini/tini.mk   | 41 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 65 insertions(+)
 create mode 100644 package/tini/Config.in
 create mode 100644 package/tini/tini.hash
 create mode 100644 package/tini/tini.mk

diff --git a/package/Config.in b/package/Config.in
index 96f04dec7a..b3fb8fa222 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1851,6 +1851,7 @@ comment "Utilities"
 	source "package/sudo/Config.in"
 	source "package/terminology/Config.in"
 	source "package/time/Config.in"
+	source "package/tini/Config.in"
 	source "package/tmux/Config.in"
 	source "package/which/Config.in"
 	source "package/xmlstarlet/Config.in"
diff --git a/package/tini/Config.in b/package/tini/Config.in
new file mode 100644
index 0000000000..323b9d17a1
--- /dev/null
+++ b/package/tini/Config.in
@@ -0,0 +1,21 @@
+config BR2_PACKAGE_TINI
+	bool "tini"
+	help
+	  tini is a simple but valid init binary to
+	  act as PID 1 for containers.
+
+	  https://github.com/krallin/tini
+
+if BR2_PACKAGE_TINI
+
+config BR2_PACKAGE_TINI_MINIMAL
+	bool "build minimal variant"
+	help
+	  Disables argument parsing and verbose output.
+
+config BR2_PACKAGE_TINI_STATIC
+	bool "build static"
+	help
+	  Enables static compilation for tini.
+
+endif
diff --git a/package/tini/tini.hash b/package/tini/tini.hash
new file mode 100644
index 0000000000..a4731bb791
--- /dev/null
+++ b/package/tini/tini.hash
@@ -0,0 +1,2 @@
+# Locally computed
+sha256	d624bb6fba3bc02701977813b7bdac9dfc9bdeedf88aff67066cf8948d2ec6ab	v0.14.0.tar.gz
diff --git a/package/tini/tini.mk b/package/tini/tini.mk
new file mode 100644
index 0000000000..6fa40e7920
--- /dev/null
+++ b/package/tini/tini.mk
@@ -0,0 +1,41 @@
+################################################################################
+#
+# tini
+#
+################################################################################
+
+TINI_VERSION = v0.14.0
+TINI_SITE = https://github.com/krallin/tini/archive
+TINI_SOURCE = $(TINI_VERSION).tar.gz
+TINI_LICENSE = MIT
+TINI_LICENSE_FILES = LICENSE
+TINI_CFLAGS = $(TARGET_CFLAGS) -DTINI_VERSION=\"$(TINI_VERSION)\" -DTINI_GIT=\"\"
+
+ifeq ($(BR2_PACKAGE_TINI_MINIMAL),y)
+TINI_CFLAGS += -DTINI_MINIMAL
+endif
+
+ifeq ($(BR2_PACKAGE_TINI_STATIC),y)
+TINI_CFLAGS += -static
+endif
+
+ifeq ($(BR2_TOOLCHAIN_HAS_SSP),y)
+TINI_CFLAGS += -fstack-protector --param=ssp-buffer-size=4
+endif
+
+define TINI_CONFIGURE_CMDS
+	printf "#pragma once\n" > $(@D)/src/tiniConfig.h
+endef
+
+define TINI_BUILD_CMDS
+	$(TARGET_CC) \
+		$(TINI_CFLAGS) \
+		-o $(@D)/tini $(@D)/src/tini.c
+endef
+
+define TINI_INSTALL_TARGET_CMDS
+	$(INSTALL) -D -m 0755 $(@D)/tini $(TARGET_DIR)/usr/bin/tini
+endef
+
+# Tini's CMakeLists.txt is not suitable for Buildroot.
+$(eval $(generic-package))
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 3/9] docker-engine: add support for init processes
  2017-09-20  3:02 [Buildroot] [PATCH 0/9] Upgrading Go to 1.9 and Docker to v2017.07 Christian Stewart
  2017-09-20  3:02 ` [Buildroot] [PATCH 1/9] dumb-init: new package Christian Stewart
  2017-09-20  3:02 ` [Buildroot] [PATCH 2/9] tini: " Christian Stewart
@ 2017-09-20  3:02 ` Christian Stewart
  2017-09-21 21:37   ` Arnout Vandecappelle
  2017-09-20  3:02 ` [Buildroot] [PATCH 4/9] docker-engine: select static for tini and dumb-init Christian Stewart
                   ` (5 subsequent siblings)
  8 siblings, 1 reply; 16+ messages in thread
From: Christian Stewart @ 2017-09-20  3:02 UTC (permalink / raw)
  To: buildroot

When a docker container is run with the --init flag, the Docker engine
uses the docker-init binary as PID 1 inside the container. This is
necessary in may cases to avoid issues with signal handling, zombie
processes, and other quirks when running as PID 1.

The docker-init binary is backed by tini on default, but optionally can
be changed to docker-init by user preference.

Furthermore, this patch fixes the following Docker error:

  dockerd: level=warning msg="failed to retrieve docker-init version"

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/docker-engine/Config.in        | 26 ++++++++++++++++++++++++++
 package/docker-engine/docker-engine.mk | 11 +++++++++++
 2 files changed, 37 insertions(+)

diff --git a/package/docker-engine/Config.in b/package/docker-engine/Config.in
index 05670a716e..7d9a87f7bb 100644
--- a/package/docker-engine/Config.in
+++ b/package/docker-engine/Config.in
@@ -33,6 +33,32 @@ config BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT
 
 if BR2_PACKAGE_DOCKER_ENGINE_DAEMON
 
+choice
+	bool "docker-init to use"
+	default BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
+	help
+	  The docker engine uses a minimal init process as PID 1
+	  inside containers. There are several implementations
+	  of this init process.
+
+config BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
+	bool "tini"
+	select BR2_PACKAGE_TINI
+	help
+	  Use Tini as the container init process.
+
+	  https://github.com/krallin/tini
+
+config BR2_PACKAGE_DOCKER_ENGINE_INIT_DUMB_INIT
+	bool "dumb-init"
+	select BR2_PACKAGE_DUMB_INIT
+	help
+	  Use dumb-init as the container init process.
+
+	  https://github.com/Yelp/dumb-init
+
+endchoice
+
 config BR2_PACKAGE_DOCKER_ENGINE_DRIVER_BTRFS
 	bool "btrfs filesystem driver"
 	depends on BR2_USE_MMU # btrfs-progs
diff --git a/package/docker-engine/docker-engine.mk b/package/docker-engine/docker-engine.mk
index 8928f072e7..a2776ed78c 100644
--- a/package/docker-engine/docker-engine.mk
+++ b/package/docker-engine/docker-engine.mk
@@ -50,6 +50,13 @@ endif
 ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y)
 DOCKER_ENGINE_BUILD_TAGS += daemon
 DOCKER_ENGINE_BUILD_TARGETS += dockerd
+
+ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_INIT_DUMB_INIT),y)
+DOCKER_ENGINE_INIT = dumb-init
+else
+DOCKER_ENGINE_INIT = tini
+endif
+
 endif
 
 ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_EXPERIMENTAL),y)
@@ -119,6 +126,10 @@ define DOCKER_ENGINE_INSTALL_TARGET_CMDS
 	$(foreach target,$(DOCKER_ENGINE_BUILD_TARGETS), \
 		$(INSTALL) -D -m 0755 $(@D)/bin/$(target) $(TARGET_DIR)/usr/bin/$(target)
 	)
+
+	$(if $(filter $(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y), \
+		ln -fs $(DOCKER_ENGINE_INIT) $(TARGET_DIR)/usr/bin/docker-init
+	)
 endef
 
 $(eval $(generic-package))
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 4/9] docker-engine: select static for tini and dumb-init
  2017-09-20  3:02 [Buildroot] [PATCH 0/9] Upgrading Go to 1.9 and Docker to v2017.07 Christian Stewart
                   ` (2 preceding siblings ...)
  2017-09-20  3:02 ` [Buildroot] [PATCH 3/9] docker-engine: add support for init processes Christian Stewart
@ 2017-09-20  3:02 ` Christian Stewart
  2017-09-20  3:02 ` [Buildroot] [PATCH 5/9] docker-proxy: new package Christian Stewart
                   ` (4 subsequent siblings)
  8 siblings, 0 replies; 16+ messages in thread
From: Christian Stewart @ 2017-09-20  3:02 UTC (permalink / raw)
  To: buildroot

In many cases docker-init backing processes like Tini and dumb-init must
be static, as they are executed inside a container in an unknown
environment. This patch selects the static compilation options on both
packages when used as docker-init backing processes.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/docker-engine/Config.in | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/package/docker-engine/Config.in b/package/docker-engine/Config.in
index 7d9a87f7bb..47ac7f57a3 100644
--- a/package/docker-engine/Config.in
+++ b/package/docker-engine/Config.in
@@ -44,6 +44,7 @@ choice
 config BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
 	bool "tini"
 	select BR2_PACKAGE_TINI
+	select BR2_PACKAGE_TINI_STATIC
 	help
 	  Use Tini as the container init process.
 
@@ -52,6 +53,7 @@ config BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
 config BR2_PACKAGE_DOCKER_ENGINE_INIT_DUMB_INIT
 	bool "dumb-init"
 	select BR2_PACKAGE_DUMB_INIT
+	select BR2_PACKAGE_DUMB_INIT_STATIC
 	help
 	  Use dumb-init as the container init process.
 
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 5/9] docker-proxy: new package
  2017-09-20  3:02 [Buildroot] [PATCH 0/9] Upgrading Go to 1.9 and Docker to v2017.07 Christian Stewart
                   ` (3 preceding siblings ...)
  2017-09-20  3:02 ` [Buildroot] [PATCH 4/9] docker-engine: select static for tini and dumb-init Christian Stewart
@ 2017-09-20  3:02 ` Christian Stewart
  2017-09-20  3:02 ` [Buildroot] [PATCH 6/9] docker-engine: add dependency on docker-proxy Christian Stewart
                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 16+ messages in thread
From: Christian Stewart @ 2017-09-20  3:02 UTC (permalink / raw)
  To: buildroot

Add docker-proxy package to Buildroot. Needed by docker-engine at
runtime, and previously not included. This leads to the following error
when attempting to port-map ports to a container:

  $ docker run -p 8080:8080 nginx
  docker: Error response from daemon: driver failed programming external
  connectivity on endpoint:
  exec: "docker-proxy": executable file not found in $PATH.

Docker expects the docker-proxy binary to exist in the PATH.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/Config.in                      |  1 +
 package/docker-proxy/Config.in         | 18 +++++++++++++
 package/docker-proxy/docker-proxy.hash |  2 ++
 package/docker-proxy/docker-proxy.mk   | 46 ++++++++++++++++++++++++++++++++++
 4 files changed, 67 insertions(+)
 create mode 100644 package/docker-proxy/Config.in
 create mode 100644 package/docker-proxy/docker-proxy.hash
 create mode 100644 package/docker-proxy/docker-proxy.mk

diff --git a/package/Config.in b/package/Config.in
index b3fb8fa222..872bd95829 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1876,6 +1876,7 @@ menu "System tools"
 	source "package/debianutils/Config.in"
 	source "package/docker-containerd/Config.in"
 	source "package/docker-engine/Config.in"
+	source "package/docker-proxy/Config.in"
 	source "package/dsp-tools/Config.in"
 	source "package/efibootmgr/Config.in"
 	source "package/efivar/Config.in"
diff --git a/package/docker-proxy/Config.in b/package/docker-proxy/Config.in
new file mode 100644
index 0000000000..4f414b24b1
--- /dev/null
+++ b/package/docker-proxy/Config.in
@@ -0,0 +1,18 @@
+config BR2_PACKAGE_DOCKER_PROXY
+	bool "docker-proxy"
+	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
+	depends on BR2_TOOLCHAIN_HAS_THREADS
+	help
+	  Libnetwork is a Container Network Model that provides a
+	  consistent programming interface and the required network
+	  abstractions for applications.
+
+	  This package provides docker-proxy, a run-time dependency of Docker.
+
+	  https://github.com/docker/libnetwork
+
+comment "docker-proxy needs a toolchain w/ threads"
+	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
+	depends on !BR2_TOOLCHAIN_HAS_THREADS
diff --git a/package/docker-proxy/docker-proxy.hash b/package/docker-proxy/docker-proxy.hash
new file mode 100644
index 0000000000..3ec184f54f
--- /dev/null
+++ b/package/docker-proxy/docker-proxy.hash
@@ -0,0 +1,2 @@
+# Locally calculated
+sha256	2eee331b6ded567a36e7db708405b34032b93938682cf049025f48b96d755bf6	docker-proxy-7b2b1feb1de4817d522cc372af149ff48d25028e.tar.gz
diff --git a/package/docker-proxy/docker-proxy.mk b/package/docker-proxy/docker-proxy.mk
new file mode 100644
index 0000000000..3bf6546d26
--- /dev/null
+++ b/package/docker-proxy/docker-proxy.mk
@@ -0,0 +1,46 @@
+################################################################################
+#
+# docker-proxy
+#
+################################################################################
+
+DOCKER_PROXY_VERSION = 7b2b1feb1de4817d522cc372af149ff48d25028e
+DOCKER_PROXY_SITE = $(call github,docker,libnetwork,$(DOCKER_PROXY_VERSION))
+
+DOCKER_PROXY_LICENSE = Apache-2.0
+DOCKER_PROXY_LICENSE_FILES = LICENSE
+
+DOCKER_PROXY_DEPENDENCIES = host-go host-pkgconf
+
+DOCKER_PROXY_GOPATH = "$(@D)/gopath"
+DOCKER_PROXY_MAKE_ENV = $(HOST_GO_TARGET_ENV) \
+	CGO_ENABLED=1 \
+	CGO_NO_EMULATION=1 \
+	GOBIN="$(@D)/bin" \
+	GOPATH="$(DOCKER_PROXY_GOPATH)" \
+	PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \
+	$(TARGET_MAKE_ENV)
+
+ifeq ($(BR2_STATIC_LIBS),y)
+DOCKER_PROXY_GLDFLAGS += -extldflags '-static'
+endif
+
+define DOCKER_PROXY_CONFIGURE_CMDS
+	mkdir -p $(DOCKER_PROXY_GOPATH)/src/github.com/docker
+	ln -fs $(@D) $(DOCKER_PROXY_GOPATH)/src/github.com/docker/libnetwork
+endef
+
+define DOCKER_PROXY_BUILD_CMDS
+	cd $(@D)/gopath/src/github.com/docker/libnetwork; \
+	$(DOCKER_PROXY_MAKE_ENV) \
+	$(HOST_DIR)/usr/bin/go build -v \
+		-o $(@D)/bin/docker-proxy \
+		-ldflags "$(DOCKER_PROXY_GLDFLAGS)" \
+		github.com/docker/libnetwork/cmd/proxy
+endef
+
+define DOCKER_PROXY_INSTALL_TARGET_CMDS
+	$(INSTALL) -D -m 0755 $(@D)/bin/docker-proxy $(TARGET_DIR)/usr/bin/docker-proxy
+endef
+
+$(eval $(generic-package))
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 6/9] docker-engine: add dependency on docker-proxy
  2017-09-20  3:02 [Buildroot] [PATCH 0/9] Upgrading Go to 1.9 and Docker to v2017.07 Christian Stewart
                   ` (4 preceding siblings ...)
  2017-09-20  3:02 ` [Buildroot] [PATCH 5/9] docker-proxy: new package Christian Stewart
@ 2017-09-20  3:02 ` Christian Stewart
  2017-09-20  3:02 ` [Buildroot] [PATCH 7/9] runc: bump to v1.0.0-rc4 Christian Stewart
                   ` (2 subsequent siblings)
  8 siblings, 0 replies; 16+ messages in thread
From: Christian Stewart @ 2017-09-20  3:02 UTC (permalink / raw)
  To: buildroot

docker-proxy is needed by docker-engine at runtime, and was
previously not included. This leads to the following error
when attempting to port-map ports to a container:

  $ docker run -p 8080:8080 nginx
  docker: Error response from daemon: driver failed programming external
  connectivity on endpoint:
  exec: "docker-proxy": executable file not found in $PATH.

Docker expects the docker-proxy binary to exist in the PATH.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/docker-engine/Config.in | 1 +
 1 file changed, 1 insertion(+)

diff --git a/package/docker-engine/Config.in b/package/docker-engine/Config.in
index 47ac7f57a3..05e7aca7ab 100644
--- a/package/docker-engine/Config.in
+++ b/package/docker-engine/Config.in
@@ -16,6 +16,7 @@ config BR2_PACKAGE_DOCKER_ENGINE_DAEMON
 	default y
 	depends on BR2_USE_MMU # docker-containerd
 	select BR2_PACKAGE_DOCKER_CONTAINERD # runtime dependency
+	select BR2_PACKAGE_DOCKER_PROXY # runtime dependency
 	select BR2_PACKAGE_IPTABLES # runtime dependency
 	select BR2_PACKAGE_SQLITE # runtime dependency
 	help
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 7/9] runc: bump to v1.0.0-rc4
  2017-09-20  3:02 [Buildroot] [PATCH 0/9] Upgrading Go to 1.9 and Docker to v2017.07 Christian Stewart
                   ` (5 preceding siblings ...)
  2017-09-20  3:02 ` [Buildroot] [PATCH 6/9] docker-engine: add dependency on docker-proxy Christian Stewart
@ 2017-09-20  3:02 ` Christian Stewart
  2017-09-20  3:02 ` [Buildroot] [PATCH 8/9] docker-containerd: bump to 3addd84 Christian Stewart
  2017-09-20  3:02 ` [Buildroot] [PATCH 9/9] docker-engine: bump to v17.07.0-ce Christian Stewart
  8 siblings, 0 replies; 16+ messages in thread
From: Christian Stewart @ 2017-09-20  3:02 UTC (permalink / raw)
  To: buildroot

This is a dependency of docker-engine v2017.07.0-ce.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/runc/runc.hash | 2 +-
 package/runc/runc.mk   | 5 +++--
 2 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/package/runc/runc.hash b/package/runc/runc.hash
index 3e188c5c79..08e306d0dc 100644
--- a/package/runc/runc.hash
+++ b/package/runc/runc.hash
@@ -1,2 +1,2 @@
 # Locally computed
-sha256	b5c50d7d9a5f610d5424f96196efa306ec708fd9299302dd919545099c16b7c1	runc-9c2d8d184e5da67c95d601382adf14862e4f2228.tar.gz
+sha256	3ffefd293932d2ad509701878dd7dd6ef498c3667198a33e8e5d5a3cbce0eeaa	runc-v1.0.0-rc4.tar.gz
diff --git a/package/runc/runc.mk b/package/runc/runc.mk
index fb3fec20f5..0b51e11896 100644
--- a/package/runc/runc.mk
+++ b/package/runc/runc.mk
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-RUNC_VERSION = 9c2d8d184e5da67c95d601382adf14862e4f2228
+RUNC_VERSION = v1.0.0-rc4
 RUNC_SITE = $(call github,opencontainers,runc,$(RUNC_VERSION))
 RUNC_LICENSE = Apache-2.0
 RUNC_LICENSE_FILES = LICENSE
@@ -38,7 +38,8 @@ define RUNC_CONFIGURE_CMDS
 endef
 
 define RUNC_BUILD_CMDS
-	cd $(@D) && $(RUNC_MAKE_ENV) $(HOST_DIR)/bin/go \
+	cd $(RUNC_GOPATH)/src/github.com/opencontainers/runc && \
+		$(RUNC_MAKE_ENV) $(HOST_DIR)/bin/go \
 		build -v -o $(@D)/bin/runc \
 		-tags "$(RUNC_GOTAGS)" -ldflags "$(RUNC_GLDFLAGS)" .
 endef
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 8/9] docker-containerd: bump to 3addd84
  2017-09-20  3:02 [Buildroot] [PATCH 0/9] Upgrading Go to 1.9 and Docker to v2017.07 Christian Stewart
                   ` (6 preceding siblings ...)
  2017-09-20  3:02 ` [Buildroot] [PATCH 7/9] runc: bump to v1.0.0-rc4 Christian Stewart
@ 2017-09-20  3:02 ` Christian Stewart
  2017-09-20  3:02 ` [Buildroot] [PATCH 9/9] docker-engine: bump to v17.07.0-ce Christian Stewart
  8 siblings, 0 replies; 16+ messages in thread
From: Christian Stewart @ 2017-09-20  3:02 UTC (permalink / raw)
  To: buildroot

This commit bumps docker-containerd to:

  3addd840653146c90a254301d6c3a663c7fd6429

This is a dependency of docker-engine v2017.07.0-ce.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/docker-containerd/docker-containerd.hash |  3 ++-
 package/docker-containerd/docker-containerd.mk   | 13 +++++++------
 2 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/package/docker-containerd/docker-containerd.hash b/package/docker-containerd/docker-containerd.hash
index 992eb5b01a..27fe8491ee 100644
--- a/package/docker-containerd/docker-containerd.hash
+++ b/package/docker-containerd/docker-containerd.hash
@@ -1,2 +1,3 @@
 # Computed locally
-sha256	da89086a7c643702a2ddecec67a45d3d8e4b0d610dd19036be63e6bd54cc1a85	docker-containerd-9048e5e50717ea4497b757314bad98ea3763c145.tar.gz
+sha256	3549e9c4ff1f71c6c9c5932b36b209f9b7ecae3662d251a88ce349853f0249fc	docker-containerd-3addd840653146c90a254301d6c3a663c7fd6429.tar.gz
+
diff --git a/package/docker-containerd/docker-containerd.mk b/package/docker-containerd/docker-containerd.mk
index ffbadb0c3f..730689d267 100644
--- a/package/docker-containerd/docker-containerd.mk
+++ b/package/docker-containerd/docker-containerd.mk
@@ -4,8 +4,8 @@
 #
 ################################################################################
 
-DOCKER_CONTAINERD_VERSION = 9048e5e50717ea4497b757314bad98ea3763c145
-DOCKER_CONTAINERD_SITE = $(call github,docker,containerd,$(DOCKER_CONTAINERD_VERSION))
+DOCKER_CONTAINERD_VERSION = 3addd840653146c90a254301d6c3a663c7fd6429
+DOCKER_CONTAINERD_SITE = $(call github,containerd,containerd,$(DOCKER_CONTAINERD_VERSION))
 DOCKER_CONTAINERD_LICENSE = Apache-2.0
 DOCKER_CONTAINERD_LICENSE_FILES = LICENSE.code
 
@@ -18,22 +18,23 @@ DOCKER_CONTAINERD_MAKE_ENV = $(HOST_GO_TARGET_ENV) \
 	GOPATH="$(DOCKER_CONTAINERD_GOPATH)"
 
 DOCKER_CONTAINERD_GLDFLAGS = \
-	-X github.com/docker/containerd.GitCommit=$(DOCKER_CONTAINERD_VERSION)
+	-X github.com/containerd/containerd.GitCommit=$(DOCKER_CONTAINERD_VERSION)
 
 ifeq ($(BR2_STATIC_LIBS),y)
 DOCKER_CONTAINERD_GLDFLAGS += -extldflags '-static'
 endif
 
 define DOCKER_CONTAINERD_CONFIGURE_CMDS
-	mkdir -p $(DOCKER_CONTAINERD_GOPATH)/src/github.com/docker
-	ln -s $(@D) $(DOCKER_CONTAINERD_GOPATH)/src/github.com/docker/containerd
+	mkdir -p $(DOCKER_CONTAINERD_GOPATH)/src/github.com/containerd
+	ln -s $(@D) $(DOCKER_CONTAINERD_GOPATH)/src/github.com/containerd/containerd
 	mkdir -p $(DOCKER_CONTAINERD_GOPATH)/src/github.com/opencontainers
 	ln -s $(RUNC_SRCDIR) $(DOCKER_CONTAINERD_GOPATH)/src/github.com/opencontainers/runc
 endef
 
 define DOCKER_CONTAINERD_BUILD_CMDS
 	$(foreach d,ctr containerd containerd-shim,\
-		cd $(@D); $(DOCKER_CONTAINERD_MAKE_ENV) $(HOST_DIR)/bin/go build \
+		cd $(DOCKER_CONTAINERD_GOPATH)/src/github.com/containerd/containerd; \
+			$(DOCKER_CONTAINERD_MAKE_ENV) $(HOST_DIR)/bin/go build \
 			-v -o $(@D)/bin/$(d) -ldflags "$(DOCKER_CONTAINERD_GLDFLAGS)" ./$(d)$(sep))
 endef
 
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 9/9] docker-engine: bump to v17.07.0-ce
  2017-09-20  3:02 [Buildroot] [PATCH 0/9] Upgrading Go to 1.9 and Docker to v2017.07 Christian Stewart
                   ` (7 preceding siblings ...)
  2017-09-20  3:02 ` [Buildroot] [PATCH 8/9] docker-containerd: bump to 3addd84 Christian Stewart
@ 2017-09-20  3:02 ` Christian Stewart
  8 siblings, 0 replies; 16+ messages in thread
From: Christian Stewart @ 2017-09-20  3:02 UTC (permalink / raw)
  To: buildroot

This commit bumps docker-engine to v17.07.0-ce.

The docker-engine repository has now been moved to:

  https://github.com/docker/docker-ce

The CLI and the daemon have been split into separate repositories:

  - https://github.com/docker/cli
  - https://github.com/moby/moby

The daemon is still in the "moby" repo, but the import paths in Go are:

  github.com/docker/docker

As before. Presumably, the maintainers will eventually split the engine out
to a different repository as well.

Please note that they literally copy the entire docker/cli and moby/moby
repositories into the docker-ce repository, as well as all of their
dependencies in the vendor directories. This means that downloading the
docker-ce source tarball contains:

 - The cli sources
 - The daemon sources
 - Two copies of all of the dependencies of both

At the moment this is unavoidable, but does produce a working build.

The patch was removed, as it is included in this release.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/docker-engine/docker-engine.hash |  3 ++-
 package/docker-engine/docker-engine.mk   | 39 ++++++++++++++++++++------------
 2 files changed, 26 insertions(+), 16 deletions(-)

diff --git a/package/docker-engine/docker-engine.hash b/package/docker-engine/docker-engine.hash
index 6c0ed338a5..90708ae972 100644
--- a/package/docker-engine/docker-engine.hash
+++ b/package/docker-engine/docker-engine.hash
@@ -1,2 +1,3 @@
 # Locally calculated
-sha256	4716df117d867b82ddab2e82395cd40aa3d0925a689eedcec8919729e4c9f121	docker-engine-v17.05.0-ce.tar.gz
+sha256	e3193118306e0d4dc4b9ad329870633d3a78d80c5f00ca6185e14185e717ca7e	docker-engine-v17.07.0-ce.tar.gz
+
diff --git a/package/docker-engine/docker-engine.mk b/package/docker-engine/docker-engine.mk
index a2776ed78c..2117272936 100644
--- a/package/docker-engine/docker-engine.mk
+++ b/package/docker-engine/docker-engine.mk
@@ -4,9 +4,9 @@
 #
 ################################################################################
 
-DOCKER_ENGINE_VERSION = v17.05.0-ce
-DOCKER_ENGINE_COMMIT = 89658bed64c2a8fe05a978e5b87dbec409d57a0f
-DOCKER_ENGINE_SITE = $(call github,docker,docker,$(DOCKER_ENGINE_VERSION))
+DOCKER_ENGINE_VERSION = v17.07.0-ce
+DOCKER_ENGINE_COMMIT = 87847530f7176a48348d196f7c23bbd058052af1
+DOCKER_ENGINE_SITE = $(call github,docker,docker-ce,$(DOCKER_ENGINE_VERSION))
 
 DOCKER_ENGINE_LICENSE = Apache-2.0
 DOCKER_ENGINE_LICENSE_FILES = LICENSE
@@ -24,7 +24,9 @@ DOCKER_ENGINE_MAKE_ENV = $(HOST_GO_TARGET_ENV) \
 
 DOCKER_ENGINE_GLDFLAGS = \
 	-X main.GitCommit=$(DOCKER_ENGINE_VERSION) \
-	-X main.Version=$(DOCKER_ENGINE_VERSION)
+	-X main.Version=$(DOCKER_ENGINE_VERSION) \
+	-X github.com/docker/cli/cli.GitCommit=$(DOCKER_ENGINE_VERSION) \
+	-X github.com/docker/cli/cli.Version=$(DOCKER_ENGINE_VERSION)
 
 ifeq ($(BR2_STATIC_LIBS),y)
 DOCKER_ENGINE_GLDFLAGS += -extldflags '-static'
@@ -35,7 +37,10 @@ endif
 endif
 
 DOCKER_ENGINE_BUILD_TAGS = cgo exclude_graphdriver_zfs autogen
-DOCKER_ENGINE_BUILD_TARGETS = docker
+DOCKER_ENGINE_BUILD_TARGETS = cli:docker
+DOCKER_ENGINE_BUILD_TARGET_PARSE = \
+		export targetpkg=$$(echo $(target) | cut -d: -f1); \
+		export targetbin=$$(echo $(target) | cut -d: -f2)
 
 ifeq ($(BR2_PACKAGE_LIBSECCOMP),y)
 DOCKER_ENGINE_BUILD_TAGS += seccomp
@@ -49,7 +54,7 @@ endif
 
 ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y)
 DOCKER_ENGINE_BUILD_TAGS += daemon
-DOCKER_ENGINE_BUILD_TARGETS += dockerd
+DOCKER_ENGINE_BUILD_TARGETS += docker:dockerd
 
 ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_INIT_DUMB_INIT),y)
 DOCKER_ENGINE_INIT = dumb-init
@@ -83,10 +88,12 @@ endif
 
 define DOCKER_ENGINE_CONFIGURE_CMDS
 	mkdir -p $(DOCKER_ENGINE_GOPATH)/src/github.com/docker
-	ln -fs $(@D) $(DOCKER_ENGINE_GOPATH)/src/github.com/docker/docker
-	cd $(@D) && \
+	ln -fs $(@D)/components/engine $(DOCKER_ENGINE_GOPATH)/src/github.com/docker/docker
+	ln -fs $(@D)/components/cli $(DOCKER_ENGINE_GOPATH)/src/github.com/docker/cli
+	cd $(@D)/components/engine && \
 		GITCOMMIT="$$(echo $(DOCKER_ENGINE_COMMIT) | head -c7)" \
 		BUILDTIME="$$(date)" \
+		IAMSTATIC="true" \
 		VERSION="$(patsubst v%,%,$(DOCKER_ENGINE_VERSION))" \
 		PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" $(TARGET_MAKE_ENV) \
 		bash ./hack/make/.go-autogen
@@ -95,9 +102,9 @@ endef
 ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y)
 
 define DOCKER_ENGINE_INSTALL_INIT_SYSTEMD
-	$(INSTALL) -D -m 0644 $(@D)/contrib/init/systemd/docker.service \
+	$(INSTALL) -D -m 0644 $(@D)/components/engine/contrib/init/systemd/docker.service \
 		$(TARGET_DIR)/usr/lib/systemd/system/docker.service
-	$(INSTALL) -D -m 0644 $(@D)/contrib/init/systemd/docker.socket \
+	$(INSTALL) -D -m 0644 $(@D)/components/engine/contrib/init/systemd/docker.socket \
 		$(TARGET_DIR)/usr/lib/systemd/system/docker.socket
 	mkdir -p $(TARGET_DIR)/etc/systemd/system/multi-user.target.wants/
 	ln -fs ../../../../usr/lib/systemd/system/docker.service \
@@ -112,19 +119,21 @@ endif
 
 define DOCKER_ENGINE_BUILD_CMDS
 	$(foreach target,$(DOCKER_ENGINE_BUILD_TARGETS), \
-		cd $(@D)/gopath/src/github.com/docker/docker; \
+		$(DOCKER_ENGINE_BUILD_TARGET_PARSE); \
+		cd $(@D)/gopath/src/github.com/docker/$${targetpkg}; \
 		$(DOCKER_ENGINE_MAKE_ENV) \
 		$(HOST_DIR)/bin/go build -v \
-			-o $(@D)/bin/$(target) \
+			-o $(@D)/bin/$${targetbin} \
 			-tags "$(DOCKER_ENGINE_BUILD_TAGS)" \
-			-ldflags "$(DOCKER_ENGINE_GLDFLAGS) $(DOCKER_ENGINE_GLDFLAGS_$(call UPPERCASE,$(target)))" \
-			github.com/docker/docker/cmd/$(target)
+			-ldflags "$(DOCKER_ENGINE_GLDFLAGS)" \
+			./cmd/$${targetbin}
 	)
 endef
 
 define DOCKER_ENGINE_INSTALL_TARGET_CMDS
 	$(foreach target,$(DOCKER_ENGINE_BUILD_TARGETS), \
-		$(INSTALL) -D -m 0755 $(@D)/bin/$(target) $(TARGET_DIR)/usr/bin/$(target)
+		$(DOCKER_ENGINE_BUILD_TARGET_PARSE); \
+		$(INSTALL) -D -m 0755 $(@D)/bin/$${targetbin} $(TARGET_DIR)/usr/bin/$${targetbin}
 	)
 
 	$(if $(filter $(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y), \
-- 
2.13.5

^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 2/9] tini: new package
  2017-09-20  3:02 ` [Buildroot] [PATCH 2/9] tini: " Christian Stewart
@ 2017-09-20  4:40   ` Baruch Siach
  2017-09-20 10:32     ` Christian Stewart
  0 siblings, 1 reply; 16+ messages in thread
From: Baruch Siach @ 2017-09-20  4:40 UTC (permalink / raw)
  To: buildroot

Hi Christian,

On Wed, Sep 20, 2017 at 04:02:03AM +0100, Christian Stewart wrote:
> diff --git a/package/tini/tini.mk b/package/tini/tini.mk
> new file mode 100644
> index 0000000000..6fa40e7920
> --- /dev/null
> +++ b/package/tini/tini.mk
> @@ -0,0 +1,41 @@
> +################################################################################
> +#
> +# tini
> +#
> +################################################################################
> +
> +TINI_VERSION = v0.14.0

Any reason not to package the latest 0.16.1 version?

> +TINI_SITE = https://github.com/krallin/tini/archive
> +TINI_SOURCE = $(TINI_VERSION).tar.gz
> +TINI_LICENSE = MIT
> +TINI_LICENSE_FILES = LICENSE
> +TINI_CFLAGS = $(TARGET_CFLAGS) -DTINI_VERSION=\"$(TINI_VERSION)\" -DTINI_GIT=\"\"
> +
> +ifeq ($(BR2_PACKAGE_TINI_MINIMAL),y)
> +TINI_CFLAGS += -DTINI_MINIMAL
> +endif
> +
> +ifeq ($(BR2_PACKAGE_TINI_STATIC),y)
> +TINI_CFLAGS += -static
> +endif
> +
> +ifeq ($(BR2_TOOLCHAIN_HAS_SSP),y)
> +TINI_CFLAGS += -fstack-protector --param=ssp-buffer-size=4
> +endif
> +
> +define TINI_CONFIGURE_CMDS
> +	printf "#pragma once\n" > $(@D)/src/tiniConfig.h
> +endef
> +
> +define TINI_BUILD_CMDS
> +	$(TARGET_CC) \
> +		$(TINI_CFLAGS) \
> +		-o $(@D)/tini $(@D)/src/tini.c
> +endef
> +
> +define TINI_INSTALL_TARGET_CMDS
> +	$(INSTALL) -D -m 0755 $(@D)/tini $(TARGET_DIR)/usr/bin/tini
> +endef
> +
> +# Tini's CMakeLists.txt is not suitable for Buildroot.

Please explain way.

> +$(eval $(generic-package))

baruch

-- 
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 2/9] tini: new package
  2017-09-20  4:40   ` Baruch Siach
@ 2017-09-20 10:32     ` Christian Stewart
  0 siblings, 0 replies; 16+ messages in thread
From: Christian Stewart @ 2017-09-20 10:32 UTC (permalink / raw)
  To: buildroot

Hi Baruch,

On Wed, Sep 20, 2017 at 5:40 AM, Baruch Siach <baruch@tkos.co.il> wrote:
> Any reason not to package the latest 0.16.1 version?


I can change this, the only reason was because I have not updated
these packages since writing them some months ago when they were
originally pushed to Buildroot.

> > +TINI_SITE = https://github.com/krallin/tini/archive

I'm surprised you didn't notice that I forgot to use the GitHub macro here :)

Updated patch series is here:

 - https://github.com/paralin/buildroot/tree/bump-docker-go

I will respin after some days if there are no comments on the other patches.

Best,
Christian

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 1/9] dumb-init: new package
  2017-09-20  3:02 ` [Buildroot] [PATCH 1/9] dumb-init: new package Christian Stewart
@ 2017-09-21 21:23   ` Arnout Vandecappelle
  2017-09-21 23:13     ` Christian Stewart
  0 siblings, 1 reply; 16+ messages in thread
From: Arnout Vandecappelle @ 2017-09-21 21:23 UTC (permalink / raw)
  To: buildroot



On 20-09-17 05:02, Christian Stewart wrote:
> dumb-init is a minimal init process to act as PID 1 for containers.
> 
> dumb-init can be used to satisfy docker-engine's docker-init binary
> requirement.
> 
> In many cases docker-init backing processes like dumb-init must be static, as
> they are executed inside a container in an unknown environment.

 I really don't understand this requirement. The container is supposed to
contain this dumb-init executable as well as the rest of the Buildroot-generated
filesystem, right? So all required libs + dynamic loader will be there. Why does
it need to be static, then? Is it going to be executed outside of the container?

 For docker-client, I can understand, because there the point is that
docker-engine runs outside of the container while within the container you have
some other OS, and docker-client is supposed to run inside the container as well.

 So please improve your explanation :-) Also, add it to the help text, so that
the user gets an idea why they would want to enable the option.

> This
> patch includes an option to specifically compile dumb-init statically, even if
> the system is compiled dynamically.
> 
> Signed-off-by: Christian Stewart <christian@paral.in>

[snip]
> diff --git a/package/dumb-init/Config.in b/package/dumb-init/Config.in
> new file mode 100644
> index 0000000000..aaad119a15
> --- /dev/null
> +++ b/package/dumb-init/Config.in
> @@ -0,0 +1,16 @@
> +config BR2_PACKAGE_DUMB_INIT
> +	bool "dumb-init"
> +	help
> +	  dumb-init is a simple but valid init binary to
> +	  act as PID 1 for containers.

 Please wrap at 72 columns, not less.

> +
> +	  https://github.com/Yelp/dumb-init
> +
> +if BR2_PACKAGE_DUMB_INIT
> +
> +config BR2_PACKAGE_DUMB_INIT_STATIC
> +	bool "build static"
> +	help
> +	  Enables static compilation for dumb-init.
> +
> +endif
> diff --git a/package/dumb-init/dumb-init.hash b/package/dumb-init/dumb-init.hash
> new file mode 100644
> index 0000000000..4705ce4368
> --- /dev/null
> +++ b/package/dumb-init/dumb-init.hash
> @@ -0,0 +1,2 @@
> +# Locally computed
> +sha256	74486997321bd939cad2ee6af030f481d39751bc9aa0ece84ed55f864e309a3f	v1.2.0.tar.gz
> diff --git a/package/dumb-init/dumb-init.mk b/package/dumb-init/dumb-init.mk
> new file mode 100644
> index 0000000000..cc24ffafa1
> --- /dev/null
> +++ b/package/dumb-init/dumb-init.mk
> @@ -0,0 +1,27 @@
> +################################################################################
> +#
> +# dumb-init
> +#
> +################################################################################
> +
> +DUMB_INIT_VERSION = v1.2.0
> +DUMB_INIT_SITE = https://github.com/Yelp/dumb-init/archive
> +DUMB_INIT_SOURCE = $(DUMB_INIT_VERSION).tar.gz
> +DUMB_INIT_LICENSE = MIT
> +DUMB_INIT_LICENSE_FILES = LICENSE
> +
> +ifeq ($(BR2_PACKAGE_DUMB_INIT_STATIC),y)
> +DUMB_INIT_CFLAGS += -static
> +endif
> +
> +define DUMB_INIT_BUILD_CMDS
> +	$(TARGET_CC) \
> +		$(DUMB_INIT_CFLAGS) \

 You also have to pass TARGET_CFLAGS and TARGET_LDFLAGS. Probably the easiest is
to collect those in DUMB_INIT_CFLAGS as well. Well, that's assuming you don't
remove that option :-)

 Regards,
 Arnout

> +		-o $(@D)/dumb-init $(@D)/dumb-init.c
> +endef
> +
> +define DUMB_INIT_INSTALL_TARGET_CMDS
> +	$(INSTALL) -D -m 0755 $(@D)/dumb-init $(TARGET_DIR)/usr/bin/dumb-init
> +endef
> +
> +$(eval $(generic-package))
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 3/9] docker-engine: add support for init processes
  2017-09-20  3:02 ` [Buildroot] [PATCH 3/9] docker-engine: add support for init processes Christian Stewart
@ 2017-09-21 21:37   ` Arnout Vandecappelle
  2017-09-21 23:09     ` Christian Stewart
  0 siblings, 1 reply; 16+ messages in thread
From: Arnout Vandecappelle @ 2017-09-21 21:37 UTC (permalink / raw)
  To: buildroot



On 20-09-17 05:02, Christian Stewart wrote:
> When a docker container is run with the --init flag, the Docker engine
> uses the docker-init binary as PID 1 inside the container. This is
> necessary in may cases to avoid issues with signal handling, zombie
> processes, and other quirks when running as PID 1.
> 
> The docker-init binary is backed by tini on default, but optionally can
> be changed to docker-init by user preference.

 OK, now I get the reason to build dumb-init statically: it actually comes from
outside the container, not inside. Honestly I don't understand why it takes an
executable from outside the container to run inside it, but OK.

> 
> Furthermore, this patch fixes the following Docker error:
> 
>   dockerd: level=warning msg="failed to retrieve docker-init version"

 On my Debian system there is no docker-init but I don't get this warning when
starting docker...


> 
> Signed-off-by: Christian Stewart <christian@paral.in>

Reviewed-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 Small nit below.

> ---
>  package/docker-engine/Config.in        | 26 ++++++++++++++++++++++++++
>  package/docker-engine/docker-engine.mk | 11 +++++++++++
>  2 files changed, 37 insertions(+)
> 
> diff --git a/package/docker-engine/Config.in b/package/docker-engine/Config.in
> index 05670a716e..7d9a87f7bb 100644
> --- a/package/docker-engine/Config.in
> +++ b/package/docker-engine/Config.in
> @@ -33,6 +33,32 @@ config BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT
>  
>  if BR2_PACKAGE_DOCKER_ENGINE_DAEMON
>  
> +choice
> +	bool "docker-init to use"
> +	default BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
> +	help
> +	  The docker engine uses a minimal init process as PID 1
> +	  inside containers. There are several implementations
> +	  of this init process.
> +
> +config BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
> +	bool "tini"
> +	select BR2_PACKAGE_TINI
> +	help
> +	  Use Tini as the container init process.
> +
> +	  https://github.com/krallin/tini

 Upstream URL is not needed here, only in the package itself.

 It might be useful to add some help text that indicates the difference between
tini and dumb-init.


 Regards,
 Arnout


> +
> +config BR2_PACKAGE_DOCKER_ENGINE_INIT_DUMB_INIT
> +	bool "dumb-init"
> +	select BR2_PACKAGE_DUMB_INIT
> +	help
> +	  Use dumb-init as the container init process.
> +
> +	  https://github.com/Yelp/dumb-init
> +
> +endchoice
> +
>  config BR2_PACKAGE_DOCKER_ENGINE_DRIVER_BTRFS
>  	bool "btrfs filesystem driver"
>  	depends on BR2_USE_MMU # btrfs-progs
> diff --git a/package/docker-engine/docker-engine.mk b/package/docker-engine/docker-engine.mk
> index 8928f072e7..a2776ed78c 100644
> --- a/package/docker-engine/docker-engine.mk
> +++ b/package/docker-engine/docker-engine.mk
> @@ -50,6 +50,13 @@ endif
>  ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y)
>  DOCKER_ENGINE_BUILD_TAGS += daemon
>  DOCKER_ENGINE_BUILD_TARGETS += dockerd
> +
> +ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_INIT_DUMB_INIT),y)
> +DOCKER_ENGINE_INIT = dumb-init
> +else
> +DOCKER_ENGINE_INIT = tini
> +endif
> +
>  endif
>  
>  ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_EXPERIMENTAL),y)
> @@ -119,6 +126,10 @@ define DOCKER_ENGINE_INSTALL_TARGET_CMDS
>  	$(foreach target,$(DOCKER_ENGINE_BUILD_TARGETS), \
>  		$(INSTALL) -D -m 0755 $(@D)/bin/$(target) $(TARGET_DIR)/usr/bin/$(target)
>  	)
> +
> +	$(if $(filter $(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y), \
> +		ln -fs $(DOCKER_ENGINE_INIT) $(TARGET_DIR)/usr/bin/docker-init
> +	)
>  endef
>  
>  $(eval $(generic-package))
> 

-- 
Arnout Vandecappelle                          arnout at mind be
Senior Embedded Software Architect            +32-16-286500
Essensium/Mind                                http://www.mind.be
G.Geenslaan 9, 3001 Leuven, Belgium           BE 872 984 063 RPR Leuven
LinkedIn profile: http://www.linkedin.com/in/arnoutvandecappelle
GPG fingerprint:  7493 020B C7E3 8618 8DEC 222C 82EB F404 F9AC 0DDF

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 3/9] docker-engine: add support for init processes
  2017-09-21 21:37   ` Arnout Vandecappelle
@ 2017-09-21 23:09     ` Christian Stewart
  0 siblings, 0 replies; 16+ messages in thread
From: Christian Stewart @ 2017-09-21 23:09 UTC (permalink / raw)
  To: buildroot

Hi Arnout,

I marked this series as RFC on patchwork, just because there are a
couple of fixes I have applied and will respin after people are done
reviewing.

On Thu, Sep 21, 2017 at 10:37 PM, Arnout Vandecappelle <arnout@mind.be> wrote:
>  On my Debian system there is no docker-init but I don't get this warning when
> starting docker...

This is likely because Docker has changed since I wrote this patch
originally, which was against a version 4 versions ago around 6 months
ago.

>  Upstream URL is not needed here, only in the package itself.

Interesting, I put it there because for some reason I remember seeing
the upstream URL in the Config.in for other packages. Will remove.

>  It might be useful to add some help text that indicates the difference between
> tini and dumb-init.

Functionally there is none, but the actual implementations are just
slightly different by different authors. Tini is a bit newer but
backed by a less trusted source while dumb-init is old and backed by
Yelp. For some reason Tini is used by the docker official
distribution.

Best,
Christian

^ permalink raw reply	[flat|nested] 16+ messages in thread

* [Buildroot] [PATCH 1/9] dumb-init: new package
  2017-09-21 21:23   ` Arnout Vandecappelle
@ 2017-09-21 23:13     ` Christian Stewart
  0 siblings, 0 replies; 16+ messages in thread
From: Christian Stewart @ 2017-09-21 23:13 UTC (permalink / raw)
  To: buildroot

Hi Arnout,

On Thu, Sep 21, 2017 at 10:23 PM, Arnout Vandecappelle:
>  I really don't understand this requirement. The container is supposed to
> contain this dumb-init executable as well as the rest of the Buildroot-generated
> filesystem, right? So all required libs + dynamic loader will be there. Why does
> it need to be static, then? Is it going to be executed outside of the container?

The container is NOT supposed to contain this. Putting dumb-init and
tini inside the container was a workaround for when Docker did not
init the containers properly with a pid 1 manager. Docker has fixed
this "issue" by adding the --init option to the run command which uses
the provided docker-init binary as the PID 1. It will be executed
INSIDE the container as the Pid 1, and has no access to the parent
system.


>  So please improve your explanation :-) Also, add it to the help text, so that
> the user gets an idea why they would want to enable the option.

The option is enabled automatically by docker-engine as a dependency
because it will not work at all without building it statically. The
option is there so that if you decide for some reason you want tini or
dumb-init in your target system with the absence of docker, which is
valid in some cases where you might be doing a cgroup isolation / jail
/ other pid 1 situation without Docker and still want the init
process.

Christian

^ permalink raw reply	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2017-09-21 23:13 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-09-20  3:02 [Buildroot] [PATCH 0/9] Upgrading Go to 1.9 and Docker to v2017.07 Christian Stewart
2017-09-20  3:02 ` [Buildroot] [PATCH 1/9] dumb-init: new package Christian Stewart
2017-09-21 21:23   ` Arnout Vandecappelle
2017-09-21 23:13     ` Christian Stewart
2017-09-20  3:02 ` [Buildroot] [PATCH 2/9] tini: " Christian Stewart
2017-09-20  4:40   ` Baruch Siach
2017-09-20 10:32     ` Christian Stewart
2017-09-20  3:02 ` [Buildroot] [PATCH 3/9] docker-engine: add support for init processes Christian Stewart
2017-09-21 21:37   ` Arnout Vandecappelle
2017-09-21 23:09     ` Christian Stewart
2017-09-20  3:02 ` [Buildroot] [PATCH 4/9] docker-engine: select static for tini and dumb-init Christian Stewart
2017-09-20  3:02 ` [Buildroot] [PATCH 5/9] docker-proxy: new package Christian Stewart
2017-09-20  3:02 ` [Buildroot] [PATCH 6/9] docker-engine: add dependency on docker-proxy Christian Stewart
2017-09-20  3:02 ` [Buildroot] [PATCH 7/9] runc: bump to v1.0.0-rc4 Christian Stewart
2017-09-20  3:02 ` [Buildroot] [PATCH 8/9] docker-containerd: bump to 3addd84 Christian Stewart
2017-09-20  3:02 ` [Buildroot] [PATCH 9/9] docker-engine: bump to v17.07.0-ce Christian Stewart

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.