[Buildroot] [PATCH v2 0/9] upgrade docker-engine and associated components

[Buildroot] [PATCH v2 0/9] upgrade docker-engine and associated components

[Christian Stewart]

Hi all,

This series upgrades Docker and associated components:

 - runc to 9f9c96235cc97674e935002fc3d78361b696a69e
 - docker-containerd to v1.0.2
 - docker-engine to v2018.03.0-ce-rc1

Fixes:

 - Btrfs is disabled in containerd if not included in the target system and selected.

Suggested naming changes:

 - docker-containerd -> containerd

This project has been split off from Docker, and is now hosted at:

  https://github.com/containerd/containerd

It is not specifically a docker project.

 - docker-engine -> docker-ce

Docker "Community Edition" is the name for the daemon and CLI bundled
together by the "moby" bundler/project duo.

It's still named the "Engine" but with the CLI as well, it makes more
sense to name it "Docker Community Edition" which also avoids a
collision with the "docker" package.

An old version of this series adding docker-init exists.

 - https://patchwork.ozlabs.org/patch/827886/

Changes v1 -> v2:

 - Make static default for tini and dumb-init
 - Bump package versions (as described above)
 - Added fix to not link against libsystemd without systemd
 - Removed unnecessary stack protector flags from tini
 - Fixed confusing commit messages
 - Upgraded to 18.03.0-rc1, skipping 18.02, to address critical bugs in 2018.02.
 - Tested on Raspberry Pi 3, Odroid XU4, Odroid HC2, and Raspberry Pi 0.

Best,
Christian Stewart

---

Christian Stewart (9):
  tini: new package
  dumb-init: new package
  docker-engine: add support for init processes
  docker-engine: select static for tini and dumb-init
  docker-proxy: new package
  docker-engine: add dependency on docker-proxy
  runc: bump to 9f9c96235cc
  docker-containerd: bump to v1.0.2
  docker-engine: bump to v18.03.0-ce-rc1

 package/Config.in                                |  3 ++
 package/docker-containerd/Config.in              | 12 +++++
 package/docker-containerd/docker-containerd.hash |  2 +-
 package/docker-containerd/docker-containerd.mk   | 38 +++++++++++----
 package/docker-engine/Config.in                  | 29 ++++++++++++
 package/docker-engine/docker-engine.hash         |  2 +-
 package/docker-engine/docker-engine.mk           | 59 ++++++++++++++++--------
 package/docker-proxy/Config.in                   | 18 ++++++++
 package/docker-proxy/docker-proxy.hash           |  2 +
 package/docker-proxy/docker-proxy.mk             | 46 ++++++++++++++++++
 package/dumb-init/Config.in                      |  7 +++
 package/dumb-init/dumb-init.hash                 |  2 +
 package/dumb-init/dumb-init.mk                   | 24 ++++++++++
 package/runc/runc.hash                           |  2 +-
 package/runc/runc.mk                             | 17 ++++---
 package/tini/Config.in                           | 16 +++++++
 package/tini/tini.hash                           |  2 +
 package/tini/tini.mk                             | 35 ++++++++++++++
 18 files changed, 278 insertions(+), 38 deletions(-)
 create mode 100644 package/docker-proxy/Config.in
 create mode 100644 package/docker-proxy/docker-proxy.hash
 create mode 100644 package/docker-proxy/docker-proxy.mk
 create mode 100644 package/dumb-init/Config.in
 create mode 100644 package/dumb-init/dumb-init.hash
 create mode 100644 package/dumb-init/dumb-init.mk
 create mode 100644 package/tini/Config.in
 create mode 100644 package/tini/tini.hash
 create mode 100644 package/tini/tini.mk

-- 
2.13.6

[Buildroot] [PATCH v2 1/9] tini: new package

[Christian Stewart]

Tini is a minimal init process to act as PID 1 for containers.

Tini can be used to satisfy docker-engine's docker-init binary
requirement.

In many cases docker-init backing processes like Tini must be static, as
they are executed inside a container in an unknown environment. This
patch adds an option to specifically compile Tini statically, even if
the system is compiled dynamically.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/Config.in      |  1 +
 package/tini/Config.in | 16 ++++++++++++++++
 package/tini/tini.hash |  2 ++
 package/tini/tini.mk   | 35 +++++++++++++++++++++++++++++++++++
 4 files changed, 54 insertions(+)
 create mode 100644 package/tini/Config.in
 create mode 100644 package/tini/tini.hash
 create mode 100644 package/tini/tini.mk

diff --git a/package/Config.in b/package/Config.in
index 9cd5d9c087..5bdaa24340 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1921,6 +1921,7 @@ comment "Utilities"
 	source "package/sudo/Config.in"
 	source "package/terminology/Config.in"
 	source "package/time/Config.in"
+	source "package/tini/Config.in"
 	source "package/tmux/Config.in"
 	source "package/which/Config.in"
 	source "package/xmlstarlet/Config.in"
diff --git a/package/tini/Config.in b/package/tini/Config.in
new file mode 100644
index 0000000000..235fc1a8fb
--- /dev/null
+++ b/package/tini/Config.in
@@ -0,0 +1,16 @@
+config BR2_PACKAGE_TINI
+	bool "tini"
+	help
+	  tini is a simple but valid init binary to
+	  act as PID 1 for containers.
+
+	  https://github.com/krallin/tini
+
+if BR2_PACKAGE_TINI
+
+config BR2_PACKAGE_TINI_MINIMAL
+	bool "build minimal variant"
+	help
+	  Disables argument parsing and verbose output.
+
+endif
diff --git a/package/tini/tini.hash b/package/tini/tini.hash
new file mode 100644
index 0000000000..f8c8c2a3aa
--- /dev/null
+++ b/package/tini/tini.hash
@@ -0,0 +1,2 @@
+# Locally computed
+sha256	c2b7fe2fb92e71911d06b7571b53d4e7e8985b34efe2a599670b070b000c8dbe	tini-v0.16.1.tar.gz
diff --git a/package/tini/tini.mk b/package/tini/tini.mk
new file mode 100644
index 0000000000..e345acd7b2
--- /dev/null
+++ b/package/tini/tini.mk
@@ -0,0 +1,35 @@
+################################################################################
+#
+# tini
+#
+################################################################################
+
+TINI_VERSION = v0.16.1
+TINI_SITE = $(call github,krallin,tini,$(TINI_VERSION))
+
+TINI_LICENSE = MIT
+TINI_LICENSE_FILES = LICENSE
+
+TINI_CFLAGS = $(TARGET_CFLAGS) -DTINI_VERSION=\"$(TINI_VERSION)\" -DTINI_GIT=\"\"
+TINI_CFLAGS += -static
+
+ifeq ($(BR2_PACKAGE_TINI_MINIMAL),y)
+TINI_CFLAGS += -DTINI_MINIMAL
+endif
+
+define TINI_CONFIGURE_CMDS
+	printf "#pragma once\n" > $(@D)/src/tiniConfig.h
+endef
+
+define TINI_BUILD_CMDS
+	$(TARGET_CC) \
+		$(TINI_CFLAGS) \
+		-o $(@D)/tini $(@D)/src/tini.c
+endef
+
+define TINI_INSTALL_TARGET_CMDS
+	$(INSTALL) -D -m 0755 $(@D)/tini $(TARGET_DIR)/usr/bin/tini
+endef
+
+# Tini's CMakeLists.txt is not suitable for Buildroot.
+$(eval $(generic-package))
-- 
2.13.6

[Buildroot] [PATCH v2 2/9] dumb-init: new package

[Christian Stewart]

dumb-init is a minimal init process to act as PID 1 for containers.

dumb-init can be used to satisfy docker-engine's docker-init binary
requirement.

In many cases docker-init backing processes like dumb-init must be static, as
they are executed inside a container in an unknown environment. This
patch includes an option to specifically compile dumb-init statically, even if
the system is compiled dynamically.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/Config.in                |  1 +
 package/dumb-init/Config.in      |  7 +++++++
 package/dumb-init/dumb-init.hash |  2 ++
 package/dumb-init/dumb-init.mk   | 24 ++++++++++++++++++++++++
 4 files changed, 34 insertions(+)
 create mode 100644 package/dumb-init/Config.in
 create mode 100644 package/dumb-init/dumb-init.hash
 create mode 100644 package/dumb-init/dumb-init.mk

diff --git a/package/Config.in b/package/Config.in
index 5bdaa24340..dd42db79ec 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1906,6 +1906,7 @@ comment "Utilities"
 	source "package/crudini/Config.in"
 	source "package/dialog/Config.in"
 	source "package/dtach/Config.in"
+	source "package/dumb-init/Config.in"
 	source "package/easy-rsa/Config.in"
 	source "package/file/Config.in"
 	source "package/gnupg/Config.in"
diff --git a/package/dumb-init/Config.in b/package/dumb-init/Config.in
new file mode 100644
index 0000000000..e817493605
--- /dev/null
+++ b/package/dumb-init/Config.in
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_DUMB_INIT
+	bool "dumb-init"
+	help
+	  dumb-init is a simple but valid init binary to
+	  act as PID 1 for containers.
+
+	  https://github.com/Yelp/dumb-init
diff --git a/package/dumb-init/dumb-init.hash b/package/dumb-init/dumb-init.hash
new file mode 100644
index 0000000000..4705ce4368
--- /dev/null
+++ b/package/dumb-init/dumb-init.hash
@@ -0,0 +1,2 @@
+# Locally computed
+sha256	74486997321bd939cad2ee6af030f481d39751bc9aa0ece84ed55f864e309a3f	v1.2.0.tar.gz
diff --git a/package/dumb-init/dumb-init.mk b/package/dumb-init/dumb-init.mk
new file mode 100644
index 0000000000..3133384f5a
--- /dev/null
+++ b/package/dumb-init/dumb-init.mk
@@ -0,0 +1,24 @@
+################################################################################
+#
+# dumb-init
+#
+################################################################################
+
+DUMB_INIT_VERSION = v1.2.0
+DUMB_INIT_SITE = $(call github,Yelp,dumb-init,$(DUMB_INIT_VERSION))
+
+DUMB_INIT_LICENSE = MIT
+DUMB_INIT_LICENSE_FILES = LICENSE
+DUMB_INIT_CFLAGS += -static
+
+define DUMB_INIT_BUILD_CMDS
+	$(TARGET_CC) \
+		$(DUMB_INIT_CFLAGS) \
+		-o $(@D)/dumb-init $(@D)/dumb-init.c
+endef
+
+define DUMB_INIT_INSTALL_TARGET_CMDS
+	$(INSTALL) -D -m 0755 $(@D)/dumb-init $(TARGET_DIR)/usr/bin/dumb-init
+endef
+
+$(eval $(generic-package))
-- 
2.13.6

[Buildroot] [PATCH v2 3/9] docker-engine: add support for init processes

[Christian Stewart]

When a docker container is run with the --init flag, the Docker engine
uses the docker-init binary as PID 1 inside the container. This is
necessary in may cases to avoid issues with signal handling, zombie
processes, and other quirks when running as PID 1.

The docker-init binary is backed by tini on default, but optionally can
be changed to docker-init by user preference.

Furthermore, this patch fixes the following Docker error:

  dockerd: level=warning msg="failed to retrieve docker-init version"

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/docker-engine/Config.in        | 26 ++++++++++++++++++++++++++
 package/docker-engine/docker-engine.mk | 11 +++++++++++
 2 files changed, 37 insertions(+)

diff --git a/package/docker-engine/Config.in b/package/docker-engine/Config.in
index 05670a716e..7d9a87f7bb 100644
--- a/package/docker-engine/Config.in
+++ b/package/docker-engine/Config.in
@@ -33,6 +33,32 @@ config BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT
 
 if BR2_PACKAGE_DOCKER_ENGINE_DAEMON
 
+choice
+	bool "docker-init to use"
+	default BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
+	help
+	  The docker engine uses a minimal init process as PID 1
+	  inside containers. There are several implementations
+	  of this init process.
+
+config BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
+	bool "tini"
+	select BR2_PACKAGE_TINI
+	help
+	  Use Tini as the container init process.
+
+	  https://github.com/krallin/tini
+
+config BR2_PACKAGE_DOCKER_ENGINE_INIT_DUMB_INIT
+	bool "dumb-init"
+	select BR2_PACKAGE_DUMB_INIT
+	help
+	  Use dumb-init as the container init process.
+
+	  https://github.com/Yelp/dumb-init
+
+endchoice
+
 config BR2_PACKAGE_DOCKER_ENGINE_DRIVER_BTRFS
 	bool "btrfs filesystem driver"
 	depends on BR2_USE_MMU # btrfs-progs
diff --git a/package/docker-engine/docker-engine.mk b/package/docker-engine/docker-engine.mk
index 8928f072e7..a2776ed78c 100644
--- a/package/docker-engine/docker-engine.mk
+++ b/package/docker-engine/docker-engine.mk
@@ -50,6 +50,13 @@ endif
 ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y)
 DOCKER_ENGINE_BUILD_TAGS += daemon
 DOCKER_ENGINE_BUILD_TARGETS += dockerd
+
+ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_INIT_DUMB_INIT),y)
+DOCKER_ENGINE_INIT = dumb-init
+else
+DOCKER_ENGINE_INIT = tini
+endif
+
 endif
 
 ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_EXPERIMENTAL),y)
@@ -119,6 +126,10 @@ define DOCKER_ENGINE_INSTALL_TARGET_CMDS
 	$(foreach target,$(DOCKER_ENGINE_BUILD_TARGETS), \
 		$(INSTALL) -D -m 0755 $(@D)/bin/$(target) $(TARGET_DIR)/usr/bin/$(target)
 	)
+
+	$(if $(filter $(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y), \
+		ln -fs $(DOCKER_ENGINE_INIT) $(TARGET_DIR)/usr/bin/docker-init
+	)
 endef
 
 $(eval $(generic-package))
-- 
2.13.6

[Buildroot] [PATCH v2 4/9] docker-engine: select static for tini and dumb-init

[Christian Stewart]

In many cases docker-init backing processes like Tini and dumb-init must
be static, as they are executed inside a container in an unknown
environment. This patch selects the static compilation options on both
packages when used as docker-init backing processes.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/docker-engine/Config.in | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/package/docker-engine/Config.in b/package/docker-engine/Config.in
index 7d9a87f7bb..47ac7f57a3 100644
--- a/package/docker-engine/Config.in
+++ b/package/docker-engine/Config.in
@@ -44,6 +44,7 @@ choice
 config BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
 	bool "tini"
 	select BR2_PACKAGE_TINI
+	select BR2_PACKAGE_TINI_STATIC
 	help
 	  Use Tini as the container init process.
 
@@ -52,6 +53,7 @@ config BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
 config BR2_PACKAGE_DOCKER_ENGINE_INIT_DUMB_INIT
 	bool "dumb-init"
 	select BR2_PACKAGE_DUMB_INIT
+	select BR2_PACKAGE_DUMB_INIT_STATIC
 	help
 	  Use dumb-init as the container init process.
 
-- 
2.13.6

[Buildroot] [PATCH v2 5/9] docker-proxy: new package

[Christian Stewart]

Add docker-proxy package to Buildroot. Needed by docker-engine at
runtime, and previously not included. This leads to the following error
when attempting to port-map ports to a container:

  $ docker run -p 8080:8080 nginx
  docker: Error response from daemon: driver failed programming external
  connectivity on endpoint:
  exec: "docker-proxy": executable file not found in $PATH.

Docker expects the docker-proxy binary to exist in the PATH.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/Config.in                      |  1 +
 package/docker-proxy/Config.in         | 18 +++++++++++++
 package/docker-proxy/docker-proxy.hash |  2 ++
 package/docker-proxy/docker-proxy.mk   | 46 ++++++++++++++++++++++++++++++++++
 4 files changed, 67 insertions(+)
 create mode 100644 package/docker-proxy/Config.in
 create mode 100644 package/docker-proxy/docker-proxy.hash
 create mode 100644 package/docker-proxy/docker-proxy.mk

diff --git a/package/Config.in b/package/Config.in
index dd42db79ec..181edfb5b7 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1947,6 +1947,7 @@ menu "System tools"
 	source "package/debianutils/Config.in"
 	source "package/docker-containerd/Config.in"
 	source "package/docker-engine/Config.in"
+	source "package/docker-proxy/Config.in"
 	source "package/dsp-tools/Config.in"
 	source "package/efibootmgr/Config.in"
 	source "package/efivar/Config.in"
diff --git a/package/docker-proxy/Config.in b/package/docker-proxy/Config.in
new file mode 100644
index 0000000000..4f414b24b1
--- /dev/null
+++ b/package/docker-proxy/Config.in
@@ -0,0 +1,18 @@
+config BR2_PACKAGE_DOCKER_PROXY
+	bool "docker-proxy"
+	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
+	depends on BR2_TOOLCHAIN_HAS_THREADS
+	help
+	  Libnetwork is a Container Network Model that provides a
+	  consistent programming interface and the required network
+	  abstractions for applications.
+
+	  This package provides docker-proxy, a run-time dependency of Docker.
+
+	  https://github.com/docker/libnetwork
+
+comment "docker-proxy needs a toolchain w/ threads"
+	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
+	depends on !BR2_TOOLCHAIN_HAS_THREADS
diff --git a/package/docker-proxy/docker-proxy.hash b/package/docker-proxy/docker-proxy.hash
new file mode 100644
index 0000000000..3ec184f54f
--- /dev/null
+++ b/package/docker-proxy/docker-proxy.hash
@@ -0,0 +1,2 @@
+# Locally calculated
+sha256	2eee331b6ded567a36e7db708405b34032b93938682cf049025f48b96d755bf6	docker-proxy-7b2b1feb1de4817d522cc372af149ff48d25028e.tar.gz
diff --git a/package/docker-proxy/docker-proxy.mk b/package/docker-proxy/docker-proxy.mk
new file mode 100644
index 0000000000..3bf6546d26
--- /dev/null
+++ b/package/docker-proxy/docker-proxy.mk
@@ -0,0 +1,46 @@
+################################################################################
+#
+# docker-proxy
+#
+################################################################################
+
+DOCKER_PROXY_VERSION = 7b2b1feb1de4817d522cc372af149ff48d25028e
+DOCKER_PROXY_SITE = $(call github,docker,libnetwork,$(DOCKER_PROXY_VERSION))
+
+DOCKER_PROXY_LICENSE = Apache-2.0
+DOCKER_PROXY_LICENSE_FILES = LICENSE
+
+DOCKER_PROXY_DEPENDENCIES = host-go host-pkgconf
+
+DOCKER_PROXY_GOPATH = "$(@D)/gopath"
+DOCKER_PROXY_MAKE_ENV = $(HOST_GO_TARGET_ENV) \
+	CGO_ENABLED=1 \
+	CGO_NO_EMULATION=1 \
+	GOBIN="$(@D)/bin" \
+	GOPATH="$(DOCKER_PROXY_GOPATH)" \
+	PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \
+	$(TARGET_MAKE_ENV)
+
+ifeq ($(BR2_STATIC_LIBS),y)
+DOCKER_PROXY_GLDFLAGS += -extldflags '-static'
+endif
+
+define DOCKER_PROXY_CONFIGURE_CMDS
+	mkdir -p $(DOCKER_PROXY_GOPATH)/src/github.com/docker
+	ln -fs $(@D) $(DOCKER_PROXY_GOPATH)/src/github.com/docker/libnetwork
+endef
+
+define DOCKER_PROXY_BUILD_CMDS
+	cd $(@D)/gopath/src/github.com/docker/libnetwork; \
+	$(DOCKER_PROXY_MAKE_ENV) \
+	$(HOST_DIR)/usr/bin/go build -v \
+		-o $(@D)/bin/docker-proxy \
+		-ldflags "$(DOCKER_PROXY_GLDFLAGS)" \
+		github.com/docker/libnetwork/cmd/proxy
+endef
+
+define DOCKER_PROXY_INSTALL_TARGET_CMDS
+	$(INSTALL) -D -m 0755 $(@D)/bin/docker-proxy $(TARGET_DIR)/usr/bin/docker-proxy
+endef
+
+$(eval $(generic-package))
-- 
2.13.6

[Buildroot] [PATCH v2 6/9] docker-engine: add dependency on docker-proxy

[Christian Stewart]

docker-proxy is needed by docker-engine at runtime, and was
previously not included. This leads to the following error
when attempting to port-map ports to a container:

  $ docker run -p 8080:8080 nginx
  docker: Error response from daemon: driver failed programming external
  connectivity on endpoint:
  exec: "docker-proxy": executable file not found in $PATH.

Docker expects the docker-proxy binary to exist in the PATH.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/docker-engine/Config.in | 1 +
 1 file changed, 1 insertion(+)

diff --git a/package/docker-engine/Config.in b/package/docker-engine/Config.in
index 47ac7f57a3..05e7aca7ab 100644
--- a/package/docker-engine/Config.in
+++ b/package/docker-engine/Config.in
@@ -16,6 +16,7 @@ config BR2_PACKAGE_DOCKER_ENGINE_DAEMON
 	default y
 	depends on BR2_USE_MMU # docker-containerd
 	select BR2_PACKAGE_DOCKER_CONTAINERD # runtime dependency
+	select BR2_PACKAGE_DOCKER_PROXY # runtime dependency
 	select BR2_PACKAGE_IPTABLES # runtime dependency
 	select BR2_PACKAGE_SQLITE # runtime dependency
 	help
-- 
2.13.6

[Buildroot] [PATCH v2 7/9] runc: bump to 9f9c96235cc

[Christian Stewart]

This commit bumps runc to the requisite version for Docker 2018.03.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/runc/runc.hash |  2 +-
 package/runc/runc.mk   | 17 +++++++++++------
 2 files changed, 12 insertions(+), 7 deletions(-)

diff --git a/package/runc/runc.hash b/package/runc/runc.hash
index 3e188c5c79..f762707869 100644
--- a/package/runc/runc.hash
+++ b/package/runc/runc.hash
@@ -1,2 +1,2 @@
 # Locally computed
-sha256	b5c50d7d9a5f610d5424f96196efa306ec708fd9299302dd919545099c16b7c1	runc-9c2d8d184e5da67c95d601382adf14862e4f2228.tar.gz
+sha256	df8e7cd1d332d97b468ee455164f3959246bb5fc639dd8042d0db3bded621ad7	runc-6c55f98695e902427906eed2c799e566e3d3dfb5.tar.gz
diff --git a/package/runc/runc.mk b/package/runc/runc.mk
index fb3fec20f5..f19fc5f068 100644
--- a/package/runc/runc.mk
+++ b/package/runc/runc.mk
@@ -4,14 +4,14 @@
 #
 ################################################################################
 
-RUNC_VERSION = 9c2d8d184e5da67c95d601382adf14862e4f2228
+RUNC_VERSION = 6c55f98695e902427906eed2c799e566e3d3dfb5
 RUNC_SITE = $(call github,opencontainers,runc,$(RUNC_VERSION))
 RUNC_LICENSE = Apache-2.0
 RUNC_LICENSE_FILES = LICENSE
 
 RUNC_DEPENDENCIES = host-go
 
-RUNC_GOPATH = "$(@D)/Godeps/_workspace"
+RUNC_GOPATH = $(@D)/gopath
 RUNC_MAKE_ENV = $(HOST_GO_TARGET_ENV) \
 	CGO_ENABLED=1 \
 	GOBIN="$(@D)/bin" \
@@ -23,9 +23,10 @@ RUNC_GLDFLAGS = \
 
 ifeq ($(BR2_STATIC_LIBS),y)
 RUNC_GLDFLAGS += -extldflags '-static'
+RUNC_GOTAGS += static_build
 endif
 
-RUNC_GOTAGS = cgo static_build
+RUNC_GOTAGS = cgo
 
 ifeq ($(BR2_PACKAGE_LIBSECCOMP),y)
 RUNC_GOTAGS += seccomp
@@ -38,9 +39,13 @@ define RUNC_CONFIGURE_CMDS
 endef
 
 define RUNC_BUILD_CMDS
-	cd $(@D) && $(RUNC_MAKE_ENV) $(HOST_DIR)/bin/go \
-		build -v -o $(@D)/bin/runc \
-		-tags "$(RUNC_GOTAGS)" -ldflags "$(RUNC_GLDFLAGS)" .
+	cd $(RUNC_GOPATH)/src/github.com/opencontainers/runc; \
+		$(RUNC_MAKE_ENV) \
+		$(HOST_DIR)/bin/go build -v -i \
+		-o $(@D)/bin/runc \
+		-tags "$(RUNC_GOTAGS)" \
+		-ldflags "$(RUNC_GLDFLAGS)" \
+		./
 endef
 
 define RUNC_INSTALL_TARGET_CMDS
-- 
2.13.6

[Buildroot] [PATCH v2 8/9] docker-containerd: bump to v1.0.2

[Christian Stewart]

This commit bumps docker-containerd to v1.0.2.

This is a requisite bump for the upgrade of docker-engine to 2018.03.

Signed-off-by: Christian Stewart <christian@paral.in>
---
 package/docker-containerd/Config.in              | 12 ++++++++
 package/docker-containerd/docker-containerd.hash |  2 +-
 package/docker-containerd/docker-containerd.mk   | 38 ++++++++++++++++++------
 3 files changed, 42 insertions(+), 10 deletions(-)

diff --git a/package/docker-containerd/Config.in b/package/docker-containerd/Config.in
index 2f7bf71af1..851345f73e 100644
--- a/package/docker-containerd/Config.in
+++ b/package/docker-containerd/Config.in
@@ -15,6 +15,18 @@ config BR2_PACKAGE_DOCKER_CONTAINERD
 
 	  https://github.com/docker/containerd
 
+if BR2_PACKAGE_DOCKER_CONTAINERD
+
+config BR2_PACKAGE_DOCKER_CONTAINERD_DRIVER_BTRFS
+	bool "btrfs snapshot driver"
+	depends on BR2_USE_MMU # btrfs-progs
+	depends on BR2_TOOLCHAIN_HAS_THREADS # btrfs-progs
+	select BR2_PACKAGE_BTRFS_PROGS
+	help
+	  Build the btrfs snapshot driver for containerd.
+
+endif
+
 comment "docker-containerd needs a toolchain w/ threads"
 	depends on BR2_PACKAGE_HOST_GO_ARCH_SUPPORTS
 	depends on BR2_PACKAGE_HOST_GO_CGO_LINKING_SUPPORTS
diff --git a/package/docker-containerd/docker-containerd.hash b/package/docker-containerd/docker-containerd.hash
index 992eb5b01a..bc67fe50dc 100644
--- a/package/docker-containerd/docker-containerd.hash
+++ b/package/docker-containerd/docker-containerd.hash
@@ -1,2 +1,2 @@
 # Computed locally
-sha256	da89086a7c643702a2ddecec67a45d3d8e4b0d610dd19036be63e6bd54cc1a85	docker-containerd-9048e5e50717ea4497b757314bad98ea3763c145.tar.gz
+sha256	a31a12846a7a2f697f8fdd82cedd1626fc5f6345e59ae880664196c73e4c2226	docker-containerd-v1.0.2.tar.gz
diff --git a/package/docker-containerd/docker-containerd.mk b/package/docker-containerd/docker-containerd.mk
index ffbadb0c3f..230307d6b4 100644
--- a/package/docker-containerd/docker-containerd.mk
+++ b/package/docker-containerd/docker-containerd.mk
@@ -4,37 +4,57 @@
 #
 ################################################################################
 
-DOCKER_CONTAINERD_VERSION = 9048e5e50717ea4497b757314bad98ea3763c145
-DOCKER_CONTAINERD_SITE = $(call github,docker,containerd,$(DOCKER_CONTAINERD_VERSION))
+DOCKER_CONTAINERD_VERSION = v1.0.2
+DOCKER_CONTAINERD_COMMIT = cfd04396dc68220d1cecbe686a6cc3aa5ce3667c
+DOCKER_CONTAINERD_SITE = $(call github,containerd,containerd,$(DOCKER_CONTAINERD_VERSION))
 DOCKER_CONTAINERD_LICENSE = Apache-2.0
 DOCKER_CONTAINERD_LICENSE_FILES = LICENSE.code
 
 DOCKER_CONTAINERD_DEPENDENCIES = host-go
 
-DOCKER_CONTAINERD_GOPATH = "$(@D)/vendor"
+DOCKER_CONTAINERD_GOPATH = "$(@D)/gopath"
 DOCKER_CONTAINERD_MAKE_ENV = $(HOST_GO_TARGET_ENV) \
 	CGO_ENABLED=1 \
 	GOBIN="$(@D)/bin" \
 	GOPATH="$(DOCKER_CONTAINERD_GOPATH)"
 
 DOCKER_CONTAINERD_GLDFLAGS = \
-	-X github.com/docker/containerd.GitCommit=$(DOCKER_CONTAINERD_VERSION)
+	-X github.com/containerd/containerd.GitCommit=$(DOCKER_CONTAINERD_COMMIT)
+
+ifeq ($(BR2_PACKAGE_LIBSECCOMP),y)
+DOCKER_CONTAINERD_BUILD_TAGS += seccomp
+DOCKER_CONTAINERD_DEPENDENCIES += libseccomp
+endif
+
+ifeq ($(BR2_PACKAGE_DOCKER_CONTAINERD_DRIVER_BTRFS),y)
+DOCKER_CONTAINERD_DEPENDENCIES += btrfs-progs
+else
+DOCKER_CONTAINERD_BUILD_TAGS += no_btrfs
+endif
 
 ifeq ($(BR2_STATIC_LIBS),y)
-DOCKER_CONTAINERD_GLDFLAGS += -extldflags '-static'
+DOCKER_CONTAINERD_GLDFLAGS += -extldflags '-static -fno-PIC'
+DOCKER_CONTAINERD_BUILD_TAGS += static_build
+DOCKER_CONTAINERD_BUILD_FLAGS += -buildmode pie
 endif
 
 define DOCKER_CONTAINERD_CONFIGURE_CMDS
-	mkdir -p $(DOCKER_CONTAINERD_GOPATH)/src/github.com/docker
-	ln -s $(@D) $(DOCKER_CONTAINERD_GOPATH)/src/github.com/docker/containerd
+	mkdir -p $(DOCKER_CONTAINERD_GOPATH)/src/github.com/containerd
+	ln -s $(@D) $(DOCKER_CONTAINERD_GOPATH)/src/github.com/containerd/containerd
 	mkdir -p $(DOCKER_CONTAINERD_GOPATH)/src/github.com/opencontainers
 	ln -s $(RUNC_SRCDIR) $(DOCKER_CONTAINERD_GOPATH)/src/github.com/opencontainers/runc
 endef
 
 define DOCKER_CONTAINERD_BUILD_CMDS
 	$(foreach d,ctr containerd containerd-shim,\
-		cd $(@D); $(DOCKER_CONTAINERD_MAKE_ENV) $(HOST_DIR)/bin/go build \
-			-v -o $(@D)/bin/$(d) -ldflags "$(DOCKER_CONTAINERD_GLDFLAGS)" ./$(d)$(sep))
+		cd $(DOCKER_CONTAINERD_GOPATH)/src/github.com/containerd/containerd; \
+			$(DOCKER_CONTAINERD_MAKE_ENV) $(HOST_DIR)/bin/go build \
+			-v -i -o $(@D)/bin/$(d) \
+			-tags "$(DOCKER_CONTAINERD_BUILD_TAGS)" \
+			-ldflags "$(DOCKER_CONTAINERD_GLDFLAGS)" \
+			$(DOCKER_CONTAINERD_BUILD_FLAGS) \
+			./cmd/$(d)$(sep)
+	)
 endef
 
 define DOCKER_CONTAINERD_INSTALL_TARGET_CMDS
-- 
2.13.6

[Buildroot] [PATCH v2 9/9] docker-engine: bump to v18.03.0-ce-rc1

[Christian Stewart]

This commit bumps docker-engine to v18.03.0-ce-rc1

The docker-engine repository has now been moved to:

  https://github.com/docker/docker-ce

The CLI and the daemon have been split into separate repositories:

  - https://github.com/docker/cli
  - https://github.com/moby/moby

The daemon codebase lives in the "moby" repo, but the import paths in Go are:

  github.com/docker/docker

As before. Presumably, the maintainers will eventually split the engine out
to a different repository as well.

Please note that the entire docker/cli and moby/moby repositories are copied
into the docker-ce repository, as well as all of their dependencies in the
vendor directories. This means that downloading the docker-ce source tarball
contains:

 - The cli sources
 - The daemon sources
 - Two copies of all of the dependencies of both

At the moment this is unavoidable, but does produce a working build.

Signed-off-by: Christian Stewart <christian@paral.in>

---

Changes since initial series:

 - Upgrade to 2018.03-ce-rc1 due to critical bugs in 2018.02.

Recommended merge to next branch.
---
 package/docker-engine/docker-engine.hash |  2 +-
 package/docker-engine/docker-engine.mk   | 48 +++++++++++++++++++-------------
 2 files changed, 29 insertions(+), 21 deletions(-)

diff --git a/package/docker-engine/docker-engine.hash b/package/docker-engine/docker-engine.hash
index 6c0ed338a5..add0b89d39 100644
--- a/package/docker-engine/docker-engine.hash
+++ b/package/docker-engine/docker-engine.hash
@@ -1,2 +1,2 @@
 # Locally calculated
-sha256	4716df117d867b82ddab2e82395cd40aa3d0925a689eedcec8919729e4c9f121	docker-engine-v17.05.0-ce.tar.gz
+sha256	45cda237a2eea4f0415bfb1f755ba64724833ac026328b765ee82aedfcfa36f3	docker-engine-v18.03.0-ce-rc1.tar.gz
diff --git a/package/docker-engine/docker-engine.mk b/package/docker-engine/docker-engine.mk
index a2776ed78c..f9cb4b8879 100644
--- a/package/docker-engine/docker-engine.mk
+++ b/package/docker-engine/docker-engine.mk
@@ -4,9 +4,8 @@
 #
 ################################################################################
 
-DOCKER_ENGINE_VERSION = v17.05.0-ce
-DOCKER_ENGINE_COMMIT = 89658bed64c2a8fe05a978e5b87dbec409d57a0f
-DOCKER_ENGINE_SITE = $(call github,docker,docker,$(DOCKER_ENGINE_VERSION))
+DOCKER_ENGINE_VERSION = v18.03.0-ce-rc1
+DOCKER_ENGINE_SITE = $(call github,docker,docker-ce,$(DOCKER_ENGINE_VERSION))
 
 DOCKER_ENGINE_LICENSE = Apache-2.0
 DOCKER_ENGINE_LICENSE_FILES = LICENSE
@@ -19,37 +18,43 @@ DOCKER_ENGINE_MAKE_ENV = $(HOST_GO_TARGET_ENV) \
 	CGO_NO_EMULATION=1 \
 	GOBIN="$(@D)/bin" \
 	GOPATH="$(DOCKER_ENGINE_GOPATH)" \
-	PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" \
+	PKG_CONFIG="$(PKG_CONFIG)" \
 	$(TARGET_MAKE_ENV)
 
 DOCKER_ENGINE_GLDFLAGS = \
 	-X main.GitCommit=$(DOCKER_ENGINE_VERSION) \
-	-X main.Version=$(DOCKER_ENGINE_VERSION)
+	-X main.Version=$(DOCKER_ENGINE_VERSION) \
+	-X github.com/docker/cli/cli.GitCommit=$(DOCKER_ENGINE_VERSION) \
+	-X github.com/docker/cli/cli.Version=$(DOCKER_ENGINE_VERSION)
+
+DOCKER_ENGINE_BUILD_TAGS = cgo exclude_graphdriver_zfs autogen
+DOCKER_ENGINE_BUILD_TARGETS = cli:docker
+DOCKER_ENGINE_BUILD_TARGET_PARSE = \
+		export targetpkg=$$(echo $(target) | cut -d: -f1); \
+		export targetbin=$$(echo $(target) | cut -d: -f2)
 
 ifeq ($(BR2_STATIC_LIBS),y)
 DOCKER_ENGINE_GLDFLAGS += -extldflags '-static'
+DOCKER_ENGINE_BUILD_TAGS += static_build
 else
 ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_STATIC_CLIENT),y)
 DOCKER_ENGINE_GLDFLAGS_DOCKER += -extldflags '-static'
 endif
 endif
 
-DOCKER_ENGINE_BUILD_TAGS = cgo exclude_graphdriver_zfs autogen
-DOCKER_ENGINE_BUILD_TARGETS = docker
-
 ifeq ($(BR2_PACKAGE_LIBSECCOMP),y)
 DOCKER_ENGINE_BUILD_TAGS += seccomp
 DOCKER_ENGINE_DEPENDENCIES += libseccomp
 endif
 
 ifeq ($(BR2_INIT_SYSTEMD),y)
-DOCKER_ENGINE_BUILD_TAGS += journald
 DOCKER_ENGINE_DEPENDENCIES += systemd
+DOCKER_ENGINE_BUILD_TAGS += systemd journald
 endif
 
 ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y)
 DOCKER_ENGINE_BUILD_TAGS += daemon
-DOCKER_ENGINE_BUILD_TARGETS += dockerd
+DOCKER_ENGINE_BUILD_TARGETS += docker:dockerd
 
 ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_INIT_DUMB_INIT),y)
 DOCKER_ENGINE_INIT = dumb-init
@@ -83,10 +88,11 @@ endif
 
 define DOCKER_ENGINE_CONFIGURE_CMDS
 	mkdir -p $(DOCKER_ENGINE_GOPATH)/src/github.com/docker
-	ln -fs $(@D) $(DOCKER_ENGINE_GOPATH)/src/github.com/docker/docker
-	cd $(@D) && \
-		GITCOMMIT="$$(echo $(DOCKER_ENGINE_COMMIT) | head -c7)" \
+	ln -fs $(@D)/components/engine $(DOCKER_ENGINE_GOPATH)/src/github.com/docker/docker
+	ln -fs $(@D)/components/cli $(DOCKER_ENGINE_GOPATH)/src/github.com/docker/cli
+	cd $(@D)/components/engine && \
 		BUILDTIME="$$(date)" \
+		IAMSTATIC="true" \
 		VERSION="$(patsubst v%,%,$(DOCKER_ENGINE_VERSION))" \
 		PKG_CONFIG="$(PKG_CONFIG_HOST_BINARY)" $(TARGET_MAKE_ENV) \
 		bash ./hack/make/.go-autogen
@@ -95,9 +101,9 @@ endef
 ifeq ($(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y)
 
 define DOCKER_ENGINE_INSTALL_INIT_SYSTEMD
-	$(INSTALL) -D -m 0644 $(@D)/contrib/init/systemd/docker.service \
+	$(INSTALL) -D -m 0644 $(@D)/components/engine/contrib/init/systemd/docker.service \
 		$(TARGET_DIR)/usr/lib/systemd/system/docker.service
-	$(INSTALL) -D -m 0644 $(@D)/contrib/init/systemd/docker.socket \
+	$(INSTALL) -D -m 0644 $(@D)/components/engine/contrib/init/systemd/docker.socket \
 		$(TARGET_DIR)/usr/lib/systemd/system/docker.socket
 	mkdir -p $(TARGET_DIR)/etc/systemd/system/multi-user.target.wants/
 	ln -fs ../../../../usr/lib/systemd/system/docker.service \
@@ -112,19 +118,21 @@ endif
 
 define DOCKER_ENGINE_BUILD_CMDS
 	$(foreach target,$(DOCKER_ENGINE_BUILD_TARGETS), \
-		cd $(@D)/gopath/src/github.com/docker/docker; \
+		$(DOCKER_ENGINE_BUILD_TARGET_PARSE); \
+		cd $(@D)/gopath/src/github.com/docker/$${targetpkg}; \
 		$(DOCKER_ENGINE_MAKE_ENV) \
 		$(HOST_DIR)/bin/go build -v \
-			-o $(@D)/bin/$(target) \
+			-o $(@D)/bin/$${targetbin} \
 			-tags "$(DOCKER_ENGINE_BUILD_TAGS)" \
-			-ldflags "$(DOCKER_ENGINE_GLDFLAGS) $(DOCKER_ENGINE_GLDFLAGS_$(call UPPERCASE,$(target)))" \
-			github.com/docker/docker/cmd/$(target)
+			-ldflags "$(DOCKER_ENGINE_GLDFLAGS)" \
+			./cmd/$${targetbin}
 	)
 endef
 
 define DOCKER_ENGINE_INSTALL_TARGET_CMDS
 	$(foreach target,$(DOCKER_ENGINE_BUILD_TARGETS), \
-		$(INSTALL) -D -m 0755 $(@D)/bin/$(target) $(TARGET_DIR)/usr/bin/$(target)
+		$(DOCKER_ENGINE_BUILD_TARGET_PARSE); \
+		$(INSTALL) -D -m 0755 $(@D)/bin/$${targetbin} $(TARGET_DIR)/usr/bin/$${targetbin}
 	)
 
 	$(if $(filter $(BR2_PACKAGE_DOCKER_ENGINE_DAEMON),y), \
-- 
2.13.6

[Buildroot] [PATCH v2 4/9] docker-engine: select static for tini and dumb-init

[Thomas Petazzoni]

Hello Christian,

On Thu,  1 Mar 2018 02:20:07 -0500, Christian Stewart wrote:
> In many cases docker-init backing processes like Tini and dumb-init must
> be static, as they are executed inside a container in an unknown
> environment. This patch selects the static compilation options on both
> packages when used as docker-init backing processes.
> 
> Signed-off-by: Christian Stewart <christian@paral.in>
> ---
>  package/docker-engine/Config.in | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/package/docker-engine/Config.in b/package/docker-engine/Config.in
> index 7d9a87f7bb..47ac7f57a3 100644
> --- a/package/docker-engine/Config.in
> +++ b/package/docker-engine/Config.in
> @@ -44,6 +44,7 @@ choice
>  config BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
>  	bool "tini"
>  	select BR2_PACKAGE_TINI
> +	select BR2_PACKAGE_TINI_STATIC
>  	help
>  	  Use Tini as the container init process.
>  
> @@ -52,6 +53,7 @@ config BR2_PACKAGE_DOCKER_ENGINE_INIT_TINI
>  config BR2_PACKAGE_DOCKER_ENGINE_INIT_DUMB_INIT
>  	bool "dumb-init"
>  	select BR2_PACKAGE_DUMB_INIT
> +	select BR2_PACKAGE_DUMB_INIT_STATIC
>  	help
>  	  Use dumb-init as the container init process.

Unless I'm missing something, this patch is selecting options that
don't exist anymore.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com

[Buildroot] [PATCH v2 2/9] dumb-init: new package

[Thomas Petazzoni]

Hello Christian,

On Thu,  1 Mar 2018 02:20:05 -0500, Christian Stewart wrote:

> diff --git a/package/dumb-init/Config.in b/package/dumb-init/Config.in
> new file mode 100644
> index 0000000000..e817493605
> --- /dev/null
> +++ b/package/dumb-init/Config.in
> @@ -0,0 +1,7 @@
> +config BR2_PACKAGE_DUMB_INIT
> +	bool "dumb-init"
> +	help
> +	  dumb-init is a simple but valid init binary to
> +	  act as PID 1 for containers.
> +
> +	  https://github.com/Yelp/dumb-init
> diff --git a/package/dumb-init/dumb-init.hash b/package/dumb-init/dumb-init.hash
> new file mode 100644
> index 0000000000..4705ce4368
> --- /dev/null
> +++ b/package/dumb-init/dumb-init.hash
> @@ -0,0 +1,2 @@
> +# Locally computed
> +sha256	74486997321bd939cad2ee6af030f481d39751bc9aa0ece84ed55f864e309a3f	v1.2.0.tar.gz

Argh, what an annoying name for the tarball :-/

Could you add an entry with the hash of the license file ?

> diff --git a/package/dumb-init/dumb-init.mk b/package/dumb-init/dumb-init.mk
> new file mode 100644
> index 0000000000..3133384f5a
> --- /dev/null
> +++ b/package/dumb-init/dumb-init.mk
> @@ -0,0 +1,24 @@
> +################################################################################
> +#
> +# dumb-init
> +#
> +################################################################################
> +
> +DUMB_INIT_VERSION = v1.2.0
> +DUMB_INIT_SITE = $(call github,Yelp,dumb-init,$(DUMB_INIT_VERSION))
> +
> +DUMB_INIT_LICENSE = MIT
> +DUMB_INIT_LICENSE_FILES = LICENSE
> +DUMB_INIT_CFLAGS += -static

This should be:

DUMB_INIT_CFLAGS = $(TARGET_CFLAGS) -static

with a comment above it that explains why we are forcing -static.

However, using -static has a problem: what happens when you're doing a
BR2_SHARED_LIBS=y configuration, i.e when there is only support for
shared libraries, and not static libraries ? And BR2_SHARED_LIBS=y is
our default.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com

[Buildroot] [PATCH v2 0/9] upgrade docker-engine and associated components

[Peter Korsgaard]

>>>>> "Christian" == Christian Stewart <christian@paral.in> writes:

 > Hi all,
 > This series upgrades Docker and associated components:

 >  - runc to 9f9c96235cc97674e935002fc3d78361b696a69e
 >  - docker-containerd to v1.0.2
 >  - docker-engine to v2018.03.0-ce-rc1

I am far from a docker expert, but isn't this an 'edge' release? Given
the Buildroot release schedule, wouldn't it make more sense to use the
'stable' releases with a similar quarterly support/release schedule?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 2/9] dumb-init: new package

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > On Thu,  1 Mar 2018 02:20:05 -0500, Christian Stewart wrote:

 > However, using -static has a problem: what happens when you're doing a
 > BR2_SHARED_LIBS=y configuration, i.e when there is only support for
 > shared libraries, and not static libraries ? And BR2_SHARED_LIBS=y is
 > our default.

dumb-init doesn't depend on any libraries, so the only issue is static
linking with glibc, right?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 4/9] docker-engine: select static for tini and dumb-init

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

Hi,

 >> +	select BR2_PACKAGE_DUMB_INIT_STATIC
 >> help
 >> Use dumb-init as the container init process.

 > Unless I'm missing something, this patch is selecting options that
 > don't exist anymore.

yes, it looks like it can be dropped.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 2/9] dumb-init: new package

[Thomas Petazzoni]

Hello,

On Thu, 01 Mar 2018 16:51:35 +0100, Peter Korsgaard wrote:

>  > On Thu,  1 Mar 2018 02:20:05 -0500, Christian Stewart wrote:  
> 
>  > However, using -static has a problem: what happens when you're doing a
>  > BR2_SHARED_LIBS=y configuration, i.e when there is only support for
>  > shared libraries, and not static libraries ? And BR2_SHARED_LIBS=y is
>  > our default.  
> 
> dumb-init doesn't depend on any libraries, so the only issue is static
> linking with glibc, right?

No, you can also have musl or uclibc configured shared-only, which is
the case by default since BR2_SHARED_LIBS=y.

Basically, I believe that Christian's patch simply doesn't work with
the default Buildroot configuration of BR2_SHARED_LIBS=y.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com

[Buildroot] [PATCH v2 2/9] dumb-init: new package

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

Hi,

 >> dumb-init doesn't depend on any libraries, so the only issue is static
 >> linking with glibc, right?

 > No, you can also have musl or uclibc configured shared-only, which is
 > the case by default since BR2_SHARED_LIBS=y.

Yes, but static linking against libc still works in that case, right?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 0/9] upgrade docker-engine and associated components

[Christian Stewart]

Hi Peter,

Peter Korsgaard <peter@korsgaard.com> writes:
>  >  - runc to 9f9c96235cc97674e935002fc3d78361b696a69e
>  >  - docker-containerd to v1.0.2
>  >  - docker-engine to v2018.03.0-ce-rc1

>
> I am far from a docker expert, but isn't this an 'edge' release? Given
> the Buildroot release schedule, wouldn't it make more sense to use the
> 'stable' releases with a similar quarterly support/release schedule?

Yes, it's intended to be merged into "next" for release not in Buildroot
2018.02 but later. I thought I had mentioned that in the cover letter,
but I guess that was in another revision, lost somewhere months ago.

The reason is because from our testing on various devices with varying
container workloads, the only two stable versions (with no critical
bugs) of Docker that we have been able to find were early 2017 or 2018.03-rc1.

2018.03-rc1 will be released as a full release soon as 2018.02 was
completely unusable due to a "container already exists" error when
trying to start a container.

If you would like me to make another series based around 17.12.1-ce that
was just released, I would be happy to do so.

Best,
Christian Stewart

[Buildroot] [PATCH v2 2/9] dumb-init: new package

[Christian Stewart]

Hi Thomas,

Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:
>> +sha256	74486997321bd939cad2ee6af030f481d39751bc9aa0ece84ed55f864e309a3f	v1.2.0.tar.gz
>
> Argh, what an annoying name for the tarball :-/

You're absolutely right, and I don't know how I didn't see that before.
Why is the github infra in DUMB_INIT_SITE producing that tarball name?
>
> Could you add an entry with the hash of the license file ?

Sure. I didn't realize that was a thing.

> This should be:
>
> DUMB_INIT_CFLAGS = $(TARGET_CFLAGS) -static
>
> with a comment above it that explains why we are forcing -static.

Will do.

> However, using -static has a problem: what happens when you're doing a
> BR2_SHARED_LIBS=y configuration, i.e when there is only support for
> shared libraries, and not static libraries ? And BR2_SHARED_LIBS=y is
> our default.

This is the build that I use (SHARED_LIBS=y) and it works. It's
necessary to static link because the binary will be run inside a
container without access to the host system's binaries.

Best,
Christian

[Buildroot] [PATCH v2 4/9] docker-engine: select static for tini and dumb-init

[Christian Stewart]

Hi Thomas,

Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:
>> In many cases docker-init backing processes like Tini and dumb-init must
>> be static, as they are executed inside a container in an unknown
>> environment. This patch selects the static compilation options on both
>> packages when used as docker-init backing processes.

> Unless I'm missing something, this patch is selecting options that
> don't exist anymore.

Yes, this can be dropped.

Best,
Christian 

[Buildroot] [PATCH v2 0/9] upgrade docker-engine and associated components

[Peter Korsgaard]

>>>>> "Christian" == Christian Stewart <christian@paral.in> writes:

 > Hi Peter,
 > Peter Korsgaard <peter@korsgaard.com> writes:
 >> >  - runc to 9f9c96235cc97674e935002fc3d78361b696a69e
 >> >  - docker-containerd to v1.0.2
 >> >  - docker-engine to v2018.03.0-ce-rc1

 >> 
 >> I am far from a docker expert, but isn't this an 'edge' release? Given
 >> the Buildroot release schedule, wouldn't it make more sense to use the
 >> 'stable' releases with a similar quarterly support/release schedule?

 > Yes, it's intended to be merged into "next" for release not in Buildroot
 > 2018.02 but later. I thought I had mentioned that in the cover letter,
 > but I guess that was in another revision, lost somewhere months ago.

 > The reason is because from our testing on various devices with varying
 > container workloads, the only two stable versions (with no critical
 > bugs) of Docker that we have been able to find were early 2017 or 2018.03-rc1.

That sounds pretty horrible :/


 > 2018.03-rc1 will be released as a full release soon as 2018.02 was
 > completely unusable due to a "container already exists" error when
 > trying to start a container.

 > If you would like me to make another series based around 17.12.1-ce that
 > was just released, I would be happy to do so.

Well, only if it actually works. But if you think there will be a new
(working) stable release before our next release (E.G. by April) then
I'm also fine with going with this 2018.03 version.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 0/9] upgrade docker-engine and associated components

[Christian Stewart]

Hi Peter,

Peter Korsgaard <peter@korsgaard.com> writes:
>>>>>> "Christian" == Christian Stewart <christian@paral.in> writes:
>  > 2018.03-rc1 will be released as a full release soon as 2018.02 was
>  > completely unusable due to a "container already exists" error when
>  > trying to start a container.
>
>  > If you would like me to make another series based around 17.12.1-ce that
>  > was just released, I would be happy to do so.
>
> Well, only if it actually works. But if you think there will be a new
> (working) stable release before our next release (E.G. by April) then
> I'm also fine with going with this 2018.03 version.

It's probably best to merge this to next for now (or wait until 2018.03
is released) and plan for the next stable release (2018.03) to include
in the next Buildroot stable.

I'll test the stability of the other release this weekend as well.

Best,
Christian

[Buildroot] [PATCH v2 2/9] dumb-init: new package

[Thomas Petazzoni]

Hello,

On Thu, 01 Mar 2018 18:21:53 +0100, Peter Korsgaard wrote:

>  >> dumb-init doesn't depend on any libraries, so the only issue is static
>  >> linking with glibc, right?  
> 
>  > No, you can also have musl or uclibc configured shared-only, which is
>  > the case by default since BR2_SHARED_LIBS=y.  
> 
> Yes, but static linking against libc still works in that case, right?

I'm not sure you have libc.a with uClibc or musl if you build
shared-only.

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin (formerly Free Electrons)
Embedded Linux and Kernel engineering
http://bootlin.com

[Buildroot] [PATCH v2 2/9] dumb-init: new package

[Yann E. MORIN]

Thomas, Peter, All,

On 2018-03-01 23:04 +0100, Thomas Petazzoni spake thusly:
> On Thu, 01 Mar 2018 18:21:53 +0100, Peter Korsgaard wrote:
> >  >> dumb-init doesn't depend on any libraries, so the only issue is static
> >  >> linking with glibc, right?  

One can do a static link with glibc, as long as one does not do any name
resolution (username, hostname, etc...). But basically, static linking
with glibc has long been frowned upon...

> >  > No, you can also have musl or uclibc configured shared-only, which is
> >  > the case by default since BR2_SHARED_LIBS=y.  
> > 
> > Yes, but static linking against libc still works in that case, right?
> 
> I'm not sure you have libc.a with uClibc or musl if you build
> shared-only.

I did a quick build here, and indeed, uClibc-ng does still provide a
libc.a when doing a shared-only build. Same goes for musl.

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v2 2/9] dumb-init: new package

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > Hello,
 > On Thu, 01 Mar 2018 18:21:53 +0100, Peter Korsgaard wrote:

 >> >> dumb-init doesn't depend on any libraries, so the only issue is static
 >> >> linking with glibc, right?  
 >> 
 >> > No, you can also have musl or uclibc configured shared-only, which is
 >> > the case by default since BR2_SHARED_LIBS=y.  
 >> 
 >> Yes, but static linking against libc still works in that case, right?

 > I'm not sure you have libc.a with uClibc or musl if you build
 > shared-only.

Today we unconditionally build the static version of musl and uClibc, so
yes.

We may have other situations where we link statically against libc with
BR2_SHARED_LIBS. go is normally statically linked, doesn't that use
libc?

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 2/9] dumb-init: new package

[Baruch Siach]

Hi Yann,

On Thu, Mar 01, 2018 at 11:23:42PM +0100, Yann E. MORIN wrote:
> On 2018-03-01 23:04 +0100, Thomas Petazzoni spake thusly:
> > On Thu, 01 Mar 2018 18:21:53 +0100, Peter Korsgaard wrote:
> > >  >> dumb-init doesn't depend on any libraries, so the only issue is static
> > >  >> linking with glibc, right?  
> 
> One can do a static link with glibc, as long as one does not do any name
> resolution (username, hostname, etc...). But basically, static linking
> with glibc has long been frowned upon...

This has changed a little recently. Toolchain support is still limited though. 
glibc NEWS for 2.27 says:

* The GNU C Library can now be compiled with support for building static
  PIE executables (See --enable-static-pie in INSTALL).  These static PIE
  executables are like static executables but can be loaded at any address
  and provide additional security hardening benefits at the cost of some
  memory and performance.  When the library is built with --enable-static-pie
  the resulting libc.a is usable with GCC 8 and above to create static PIE
  executables using the GCC option '-static-pie'.  This feature is currently
  supported on i386, x86_64 and x32 with binutils 2.29 or later, and on
  aarch64 with binutils 2.30 or later.

baruch

-- 
     http://baruch.siach.name/blog/                  ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch at tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -

[Buildroot] [PATCH v2 2/9] dumb-init: new package

[Yann E. MORIN]

Baruch, All,

On 2018-03-02 01:02 +0200, Baruch Siach spake thusly:
> On Thu, Mar 01, 2018 at 11:23:42PM +0100, Yann E. MORIN wrote:
> > On 2018-03-01 23:04 +0100, Thomas Petazzoni spake thusly:
> > > On Thu, 01 Mar 2018 18:21:53 +0100, Peter Korsgaard wrote:
> > > >  >> dumb-init doesn't depend on any libraries, so the only issue is static
> > > >  >> linking with glibc, right?  
> > 
> > One can do a static link with glibc, as long as one does not do any name
> > resolution (username, hostname, etc...). But basically, static linking
> > with glibc has long been frowned upon...
> 
> This has changed a little recently. Toolchain support is still limited though. 
> glibc NEWS for 2.27 says:
> 
> * The GNU C Library can now be compiled with support for building static
>   PIE executables (See --enable-static-pie in INSTALL).  These static PIE
>   executables are like static executables but can be loaded at any address
>   and provide additional security hardening benefits at the cost of some
>   memory and performance.  When the library is built with --enable-static-pie
>   the resulting libc.a is usable with GCC 8 and above to create static PIE
>   executables using the GCC option '-static-pie'.  This feature is currently
>   supported on i386, x86_64 and x32 with binutils 2.29 or later, and on
>   aarch64 with binutils 2.30 or later.

Ah, interesting, thanks! :-)

I always wondered why pie and static were mutually exclusive...
Afterall, semantically they mean two orthogonal things.

Incidentally, I also always wondered why code was not
position-indpendent by default...

Regards,
Yann E. MORIN.

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

[Buildroot] [PATCH v2 5/9] docker-proxy: new package

[Peter Korsgaard]

>>>>> "Christian" == Christian Stewart <christian@paral.in> writes:

 > Add docker-proxy package to Buildroot. Needed by docker-engine at
 > runtime, and previously not included. This leads to the following error
 > when attempting to port-map ports to a container:

 >   $ docker run -p 8080:8080 nginx
 >   docker: Error response from daemon: driver failed programming external
 >   connectivity on endpoint:
 >   exec: "docker-proxy": executable file not found in $PATH.

 > Docker expects the docker-proxy binary to exist in the PATH.

 > Signed-off-by: Christian Stewart <christian@paral.in>

Committed to 2018.02.x, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH v2 6/9] docker-engine: add dependency on docker-proxy

[Peter Korsgaard]

>>>>> "Christian" == Christian Stewart <christian@paral.in> writes:

 > docker-proxy is needed by docker-engine at runtime, and was
 > previously not included. This leads to the following error
 > when attempting to port-map ports to a container:

 >   $ docker run -p 8080:8080 nginx
 >   docker: Error response from daemon: driver failed programming external
 >   connectivity on endpoint:
 >   exec: "docker-proxy": executable file not found in $PATH.

 > Docker expects the docker-proxy binary to exist in the PATH.

 > Signed-off-by: Christian Stewart <christian@paral.in>

Committed to 2018.02.x, thanks.

-- 
Bye, Peter Korsgaard