All of lore.kernel.org
 help / color / mirror / Atom feed
* [Buildroot]  [PATCH 0/8 RFC] Adding a Go package infrastructure
@ 2016-02-01 23:30 Ludovic Guegan
  2016-02-01 23:30 ` [Buildroot] [PATCH 1/8 RFC] package/golang: create Go 1.5 host compiler package Ludovic Guegan
                   ` (8 more replies)
  0 siblings, 9 replies; 14+ messages in thread
From: Ludovic Guegan @ 2016-02-01 23:30 UTC (permalink / raw)
  To: buildroot

Hi everyone,

I would like to have your feedbacks about this serie of patches which
experiments with Go packages.

There are two related issues when packaging Go programs:

1. The Go community have not come up with a prefered way to package Go
applications. Here is a quick overview of the situation:
https://github.com/golang/go/wiki/PackageManagementTools

2. Often Go packages have dependencies that are not well expressed
with Buildroot packages. For example, the go program
github.com/rapidloop/rtop depends on golang.org/x/crypto/ssh but not
on golang.org/x/crypto/ssh/terminal.

In this serie of patch, I experiment with gb (http://getgb.io) a tool
from Dave Cheney which focuses on reproducable builds.

Dependencies are expressed using a macro (fetch-golang-package) to
fetch them consistently. See patches 6 to 8 for examples.

The general idea is the following: GOPATH is set to $(@D) and the
dependencies are downloaded into $(@D)/vendor.

You may wonder, why using gb? Well, I found gb easy to use and mature
enough for the task. It is an implementation detail which is invisible
to golang-package .mk files.

If the direction is correct, i will continue to experiement with mixed
package that have Go code and Makefiles.

Patch 1 to 2
  Host packages needed to fetch and build Go packages.

Patch 3
  Allow custom extraction location.

Patch 4 and 5
  Introduce a golang-package infrastructure.

Patch 6 to 8
  Examples of golang-packages, just for illustration. 


Your feedback is highly appreciated!


Regards,

Ludovic


Ludovic Guegan (8):
  package/golang: create Go 1.5 host compiler package
  package/golang-gb: create gb package to build and fetch go packages
  package/pkg-generic.mk: allow custom extract directory
  package/pkg-golang.mk: infrastructure for Go packages
  docs/manual: documents the golang-package infrastructure
  package/rtop: add rtop package in Go
  package/embd: add embd package (embedded programming framework)
  package/bolt: add bolt package (persistent key-value store)

 docs/manual/adding-packages-generic.txt |   3 +
 docs/manual/adding-packages-golang.txt  |  73 +++++++++++++++
 package/Config.in                       |   3 +
 package/Config.in.host                  |   2 +
 package/Makefile.in                     |   1 +
 package/bolt/Config.in                  |   7 ++
 package/bolt/bolt.hash                  |   1 +
 package/bolt/bolt.mk                    |  13 +++
 package/embd/Config.in                  |   8 ++
 package/embd/embd.hash                  |   1 +
 package/embd/embd.mk                    |  16 ++++
 package/golang-gb/Config.in.host        |   8 ++
 package/golang-gb/golang-gb.mk          |  30 ++++++
 package/golang/Config.in.host           |   7 ++
 package/golang/golang.hash              |   2 +
 package/golang/golang.mk                |  72 +++++++++++++++
 package/pkg-generic.mk                  |  12 ++-
 package/pkg-golang.mk                   | 156 ++++++++++++++++++++++++++++++++
 package/rtop/Config.in                  |   8 ++
 package/rtop/rtop.hash                  |   1 +
 package/rtop/rtop.mk                    |  16 ++++
 support/scripts/pkg-stats               |  11 +++
 22 files changed, 449 insertions(+), 2 deletions(-)
 create mode 100644 docs/manual/adding-packages-golang.txt
 create mode 100644 package/bolt/Config.in
 create mode 100644 package/bolt/bolt.hash
 create mode 100644 package/bolt/bolt.mk
 create mode 100644 package/embd/Config.in
 create mode 100644 package/embd/embd.hash
 create mode 100644 package/embd/embd.mk
 create mode 100644 package/golang-gb/Config.in.host
 create mode 100644 package/golang-gb/golang-gb.mk
 create mode 100644 package/golang/Config.in.host
 create mode 100644 package/golang/golang.hash
 create mode 100644 package/golang/golang.mk
 create mode 100644 package/pkg-golang.mk
 create mode 100644 package/rtop/Config.in
 create mode 100644 package/rtop/rtop.hash
 create mode 100644 package/rtop/rtop.mk

-- 
2.7.0

^ permalink raw reply	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 1/8 RFC] package/golang: create Go 1.5 host compiler package
  2016-02-01 23:30 [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Ludovic Guegan
@ 2016-02-01 23:30 ` Ludovic Guegan
  2016-02-01 23:30 ` [Buildroot] [PATCH 2/8 RFC] package/golang-gb: create gb package to build and fetch go packages Ludovic Guegan
                   ` (7 subsequent siblings)
  8 siblings, 0 replies; 14+ messages in thread
From: Ludovic Guegan @ 2016-02-01 23:30 UTC (permalink / raw)
  To: buildroot

This compiler is bootstraped using a Go 1.4 compiler.

Signed-off-by: Ludovic Guegan <ludovic.guegan@gmail.com>
---
 package/Config.in.host        |  1 +
 package/golang/Config.in.host |  7 +++++
 package/golang/golang.hash    |  2 ++
 package/golang/golang.mk      | 72 +++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 82 insertions(+)
 create mode 100644 package/golang/Config.in.host
 create mode 100644 package/golang/golang.hash
 create mode 100644 package/golang/golang.mk

diff --git a/package/Config.in.host b/package/Config.in.host
index 8e6b870..0625bd0 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -12,6 +12,7 @@ menu "Host utilities"
 	source "package/genext2fs/Config.in.host"
 	source "package/genimage/Config.in.host"
 	source "package/genpart/Config.in.host"
+	source "package/golang/Config.in.host"
 	source "package/gptfdisk/Config.in.host"
 	source "package/imx-usb-loader/Config.in.host"
 	source "package/jq/Config.in.host"
diff --git a/package/golang/Config.in.host b/package/golang/Config.in.host
new file mode 100644
index 0000000..11ce925
--- /dev/null
+++ b/package/golang/Config.in.host
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_HOST_GOLANG
+	bool "golang"
+	depends on BR2_TOOLCHAIN_HAS_THREADS
+	help
+	  Go build system. It is used to cros-compile Go programs for the target.
+
+	  http://golang.org/
diff --git a/package/golang/golang.hash b/package/golang/golang.hash
new file mode 100644
index 0000000..751e19a
--- /dev/null
+++ b/package/golang/golang.hash
@@ -0,0 +1,2 @@
+sha256 754e06dab1c31ab168fc9db9e32596734015ea9e24bc44cae7f237f417ce4efe  go1.5.3.src.tar.gz
+sha256 299a6fd8f8adfdce15bc06bde926e7b252ae8e24dd5b16b7d8791ed79e7b5e9b  go1.4.2.src.tar.gz
diff --git a/package/golang/golang.mk b/package/golang/golang.mk
new file mode 100644
index 0000000..f02ebde
--- /dev/null
+++ b/package/golang/golang.mk
@@ -0,0 +1,72 @@
+################################################################################
+#
+# golang
+#
+################################################################################
+
+GOLANG_VERSION = 1.5.3
+GOLANG_SOURCE = go$(GOLANG_VERSION).src.tar.gz
+GOLANG_SITE = https://storage.googleapis.com/golang
+GOLANG_LICENSE = BSD-3c
+GOLANG_LICENSE_FILES = LICENSE
+GOLANG_EXTRA_SOURCE = go1.4.2.src.tar.gz
+HOST_GOLANG_EXTRA_DOWNLOADS = https://storage.googleapis.com/golang/$(GOLANG_EXTRA_SOURCE)
+GOLANG_SUBDIR = go
+
+#
+# Environment used to bootstrap
+#
+GOLANG_BOOTSTRAP_DIR = $(@D)/bootstrap
+GOLANG_BOOTSTRAP_GO = $(GOLANG_BOOTSTRAP_DIR)/go
+GOLANG_BOOTSTRAP_BIN = $(GOLANG_BOOTSTRAP_GO)/bin
+GOLANG_BOOTSTRAP_ENV = GOBIN=$(GOLANG_BOOTSTRAP_BIN) PATH=$(BR_PATH)
+
+#
+# Environment used to build
+#
+GOLANG_GOROOT_FINAL = $(HOST_DIR)/usr/lib/go
+GOLANG_BUILD_ENV = PATH=$(GOLANG_BOOTSTRAP_BIN):$(BR_PATH) \
+	GOROOT_FINAL=$(GOLANG_GOROOT_FINAL) \
+	GOROOT_BOOTSTRAP=$(GOLANG_BOOTSTRAP_GO)
+
+#
+# Packages can use GO_HOST_BINARY to call the go program
+#
+GO_HOST_BINARY = $(HOST_DIR)/usr/bin/go
+
+#
+# Extracts bootstraping and host compilers
+#
+define HOST_GOLANG_EXTRACT_CMDS
+	mkdir -p $(GOLANG_BOOTSTRAP_BIN)
+	tar xvaf $(BR2_DL_DIR)/$(GOLANG_EXTRA_SOURCE) -C $(GOLANG_BOOTSTRAP_DIR)
+	tar xvaf $(BR2_DL_DIR)/$(GOLANG_SOURCE) -C $(@D)
+endef
+
+#
+# Build bootstraping compiler before the host compiler.
+#
+define HOST_GOLANG_BOOTSTRAP
+	cd $(GOLANG_BOOTSTRAP_GO)/src; env -i $(GOLANG_BOOTSTRAP_ENV) ./make.bash
+endef
+
+HOST_GOLANG_PRE_BUILD_HOOKS += HOST_GOLANG_BOOTSTRAP
+
+#
+# Compile the host compiler with the envionment pointing to the bootstraping
+# compiler previously built
+#
+define HOST_GOLANG_BUILD_CMDS
+	cd $(HOST_GOLANG_SRCDIR)/src; env -i $(GOLANG_BUILD_ENV) ./make.bash
+endef
+
+#
+# Install Go by simply copying the source and the binary to the final
+# destination
+#
+define HOST_GOLANG_INSTALL_CMDS
+	cp -a $(HOST_GOLANG_SRCDIR) $(GOLANG_GOROOT_FINAL)
+	ln -fs $(GOLANG_GOROOT_FINAL)/bin/go $(GO_HOST_BINARY)
+endef
+
+$(eval $(host-generic-package))
-- 
2.7.0

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 2/8 RFC] package/golang-gb: create gb package to build and fetch go packages
  2016-02-01 23:30 [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Ludovic Guegan
  2016-02-01 23:30 ` [Buildroot] [PATCH 1/8 RFC] package/golang: create Go 1.5 host compiler package Ludovic Guegan
@ 2016-02-01 23:30 ` Ludovic Guegan
  2016-06-01 20:17   ` Thomas Petazzoni
  2016-02-01 23:31 ` [Buildroot] [PATCH 3/8 RFC] package/pkg-generic.mk: allow custom extract directory Ludovic Guegan
                   ` (6 subsequent siblings)
  8 siblings, 1 reply; 14+ messages in thread
From: Ludovic Guegan @ 2016-02-01 23:30 UTC (permalink / raw)
  To: buildroot

Signed-off-by: Ludovic Guegan <ludovic.guegan@gmail.com>
---
 package/Config.in.host           |  1 +
 package/golang-gb/Config.in.host |  8 ++++++++
 package/golang-gb/golang-gb.mk   | 30 ++++++++++++++++++++++++++++++
 3 files changed, 39 insertions(+)
 create mode 100644 package/golang-gb/Config.in.host
 create mode 100644 package/golang-gb/golang-gb.mk

diff --git a/package/Config.in.host b/package/Config.in.host
index 0625bd0..2d99d37 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -13,6 +13,7 @@ menu "Host utilities"
 	source "package/genimage/Config.in.host"
 	source "package/genpart/Config.in.host"
 	source "package/golang/Config.in.host"
+	source "package/golang-gb/Config.in.host"
 	source "package/gptfdisk/Config.in.host"
 	source "package/imx-usb-loader/Config.in.host"
 	source "package/jq/Config.in.host"
diff --git a/package/golang-gb/Config.in.host b/package/golang-gb/Config.in.host
new file mode 100644
index 0000000..b84a83f
--- /dev/null
+++ b/package/golang-gb/Config.in.host
@@ -0,0 +1,8 @@
+config BR2_PACKAGE_HOST_GOLANG_GB
+	bool "golang-gb"
+	depends on BR2_PACKAGE_HOST_GOLANG
+	help
+          gb is an alternative build tool for the Go programming language.
+
+	  https://getgb.io
+
diff --git a/package/golang-gb/golang-gb.mk b/package/golang-gb/golang-gb.mk
new file mode 100644
index 0000000..9ae9da7
--- /dev/null
+++ b/package/golang-gb/golang-gb.mk
@@ -0,0 +1,30 @@
+################################################################################
+#
+# golang-gb
+#
+################################################################################
+
+GOLANG_GB_VERSION = 0.4.0
+GOLANG_GB_TAG = v$(GOLANG_GB_VERSION)
+GOLANG_GB_SITE = $(call github,constabulary,gb,$(GOLANG_GB_TAG))
+GOLANG_GB_LICENSE = MIT
+
+GOLANG_GB_ENV = GOPATH=$(@D)
+
+define HOST_GOLANG_GB_EXTRACT_CMDS
+	mkdir -p $(@D)/src/github.com/constabulary
+	tar xvaf $(DL_DIR)/golang-gb-$(GOLANG_GB_VERSION).tar.gz -C $(@D)
+	mv $(@D)/gb-$(GOLANG_GB_VERSION) $(@D)/src/github.com/constabulary/gb
+endef
+
+define HOST_GOLANG_GB_BUILD_CMDS
+	env -i $(GOLANG_GB_ENV) $(GO_HOST_BINARY) install -v github.com/constabulary/gb/cmd/gb
+	env -i $(GOLANG_GB_ENV) $(GO_HOST_BINARY) install -v github.com/constabulary/gb/cmd/gb-vendor
+endef
+
+define HOST_GOLANG_GB_INSTALL_CMDS
+	$(INSTALL) -m 755 $(@D)/bin/gb $(HOST_DIR)/usr/bin
+	$(INSTALL) -m 755 $(@D)/bin/gb-vendor $(HOST_DIR)/usr/bin
+endef
+
+$(eval $(host-generic-package))
-- 
2.7.0

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 3/8 RFC] package/pkg-generic.mk: allow custom extract directory
  2016-02-01 23:30 [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Ludovic Guegan
  2016-02-01 23:30 ` [Buildroot] [PATCH 1/8 RFC] package/golang: create Go 1.5 host compiler package Ludovic Guegan
  2016-02-01 23:30 ` [Buildroot] [PATCH 2/8 RFC] package/golang-gb: create gb package to build and fetch go packages Ludovic Guegan
@ 2016-02-01 23:31 ` Ludovic Guegan
  2016-06-01 20:19   ` Thomas Petazzoni
  2016-02-01 23:31 ` [Buildroot] [PATCH 4/8 RFC] package/pkg-golang.mk: infrastructure for Go packages Ludovic Guegan
                   ` (5 subsequent siblings)
  8 siblings, 1 reply; 14+ messages in thread
From: Ludovic Guegan @ 2016-02-01 23:31 UTC (permalink / raw)
  To: buildroot

Signed-off-by: Ludovic Guegan <ludovic.guegan@gmail.com>
---
 docs/manual/adding-packages-generic.txt |  3 +++
 package/pkg-generic.mk                  | 12 ++++++++++--
 2 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/docs/manual/adding-packages-generic.txt b/docs/manual/adding-packages-generic.txt
index b7817e3..b7442a7 100644
--- a/docs/manual/adding-packages-generic.txt
+++ b/docs/manual/adding-packages-generic.txt
@@ -444,6 +444,9 @@ different steps of the build process.
   tarball with a non-standard organization, this variable allows to
   override the package infrastructure default behavior.
 
+* +LIBFOO_EXTRACT_DIR+ can be set to change the directory where the
+  package is extracted.
+
 * +LIBFOO_CONFIGURE_CMDS+ lists the actions to be performed to
   configure the package before its compilation.
 
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index 1e024d3..3051551 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -128,7 +128,7 @@ $(BUILD_DIR)/%/.stamp_extracted:
 	@$(call step_start,extract)
 	@$(call MESSAGE,"Extracting")
 	$(foreach hook,$($(PKG)_PRE_EXTRACT_HOOKS),$(call $(hook))$(sep))
-	$(Q)mkdir -p $(@D)
+	$(Q)mkdir -p $(@D) $($(PKG)_EXTRACT_DIR)
 	$($(PKG)_EXTRACT_CMDS)
 # some packages have messed up permissions inside
 	$(Q)chmod -R +rw $(@D)
@@ -396,6 +396,14 @@ $(2)_BASE_NAME	=  $(1)-$$($(2)_VERSION)
 $(2)_DL_DIR	=  $$(DL_DIR)/$$($(2)_BASE_NAME)
 $(2)_DIR	=  $$(BUILD_DIR)/$$($(2)_BASE_NAME)
 
+ifndef $(2)_EXTRACT_DIR
+ ifdef $(3)_EXTRACT_DIR
+  $(2)_EXTRACT_DIR = $$($(3)_EXTRACT_DIR)
+ else
+  $(2)_EXTRACT_DIR ?= $$($(3)_DIR)
+ endif
+endif
+
 ifndef $(2)_SUBDIR
  ifdef $(3)_SUBDIR
   $(2)_SUBDIR = $$($(3)_SUBDIR)
@@ -525,7 +533,7 @@ $(2)_TARGET_DIRCLEAN =		$$($(2)_DIR)/.stamp_dircleaned
 $(2)_EXTRACT_CMDS ?= \
 	$$(if $$($(2)_SOURCE),$$(INFLATE$$(suffix $$($(2)_SOURCE))) $$(DL_DIR)/$$($(2)_SOURCE) | \
 	$$(TAR) --strip-components=$$($(2)_STRIP_COMPONENTS) \
-		-C $$($(2)_DIR) \
+		-C $$($(2)_EXTRACT_DIR) \
 		$$(foreach x,$$($(2)_EXCLUDES),--exclude='$$(x)' ) \
 		$$(TAR_OPTIONS) -)
 
-- 
2.7.0

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 4/8 RFC] package/pkg-golang.mk: infrastructure for Go packages
  2016-02-01 23:30 [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Ludovic Guegan
                   ` (2 preceding siblings ...)
  2016-02-01 23:31 ` [Buildroot] [PATCH 3/8 RFC] package/pkg-generic.mk: allow custom extract directory Ludovic Guegan
@ 2016-02-01 23:31 ` Ludovic Guegan
  2016-06-01 20:26   ` Thomas Petazzoni
  2016-02-01 23:31 ` [Buildroot] [PATCH 5/8 RFC] docs/manual: documents the golang-package infrastructure Ludovic Guegan
                   ` (4 subsequent siblings)
  8 siblings, 1 reply; 14+ messages in thread
From: Ludovic Guegan @ 2016-02-01 23:31 UTC (permalink / raw)
  To: buildroot

Signed-off-by: Ludovic Guegan <ludovic.guegan@gmail.com>
---
 package/Makefile.in       |   1 +
 package/pkg-golang.mk     | 156 ++++++++++++++++++++++++++++++++++++++++++++++
 support/scripts/pkg-stats |  11 ++++
 3 files changed, 168 insertions(+)
 create mode 100644 package/pkg-golang.mk

diff --git a/package/Makefile.in b/package/Makefile.in
index dd595e2..067abb6 100644
--- a/package/Makefile.in
+++ b/package/Makefile.in
@@ -393,6 +393,7 @@ include package/pkg-download.mk
 include package/pkg-autotools.mk
 include package/pkg-cmake.mk
 include package/pkg-luarocks.mk
+include package/pkg-golang.mk
 include package/pkg-perl.mk
 include package/pkg-python.mk
 include package/pkg-virtual.mk
diff --git a/package/pkg-golang.mk b/package/pkg-golang.mk
new file mode 100644
index 0000000..cff4e9f
--- /dev/null
+++ b/package/pkg-golang.mk
@@ -0,0 +1,156 @@
+################################################################################
+# Golang package infrastructure
+#
+# This file implements an infrastructure that eases development of package
+# .mk files for Go packages.
+#
+# See the Buildroot documentation for details on the usage of this
+# infrastructure
+#
+#
+# In terms of implementation, this Go infrastructure requires the .mk file
+# to only specify metadata information about the package: name, version,
+# download URL, etc.
+#
+# We still allow the package .mk file to override what the different steps
+# are doing, if needed. For example, if <PKG>_BUILD_CMDS is already defined,
+# it is used as the list of commands to perform to build the package,
+# instead of the default golang behaviour. The package can also define some
+# post operation hooks.
+#
+################################################################################
+
+GO_ENV = PATH=$(BR_PATH) GOPATH=$($PKGDIR)
+
+#
+# If compiling for the target then set GOARCH accordingly.
+#
+
+ifeq ($(4),target)
+
+ifeq ($(BR2_i386),y)
+GOLANG_ARCH = 386
+endif # i386
+
+ifeq ($(BR2_x86_64),y)
+GOLANG_ARCH = amd64
+endif # x86_64
+
+ifeq ($(BR2_arm),y)
+GOLANG_ARCH = arm
+endif # arm
+
+GO_ENV += GOARCH=$(GOLANG_ARCH)
+
+endif # ($(4),target)
+
+################################################################################
+#
+# Go packages are normally fetch using "go get", unfortunatly this method
+# fetch the HEAD reference of the package. We use golang-gb in order to
+# implement reproducible builds.
+#
+#  argument 1 is the package name to fetch as specified in the source code
+#  argument 2 is the revision reference
+################################################################################
+
+define fetch-golang-package
+	cd $($(PKG)_SRCDIR); \
+		env -i $(GO_ENV) gb vendor fetch -no-recurse -revision $(2) $(1);
+endef
+
+################################################################################
+# inner-golang-package -- defines how the configuration, compilation and
+# installation of a Go package should be done, implements a few hooks to
+# tune the build process for Go specifities and calls the generic package
+# infrastructure to generate the necessary make targets
+#
+#  argument 1 is the lowercase package name
+#  argument 2 is the uppercase package name, including a HOST_ prefix
+#             for host packages
+#  argument 3 is the uppercase package name, without the HOST_ prefix
+#             for host packages
+#  argument 4 is the type (target or host)
+################################################################################
+
+define inner-golang-package
+
+# Target packages need both the Go compiler on the host and the golang-gb
+# build tool on the host (for reproduable builds).
+$(2)_DEPENDENCIES += host-golang host-golang-gb
+
+#
+# Fetch go package if $($(PKG)_DEPS) is defined
+#
+ifdef $(2)_DEPS
+
+define $(2)_FETCH_DEPS
+	mkdir -p $$($$(PKG)_DIR)/{src,vendor}
+	cd $$($$(PKG)_SRCDIR); env -i $(GO_ENV) gb vendor delete --all
+	$$($$(PKG)_DEPS)
+endef
+
+$(2)_POST_DOWNLOAD_HOOKS += $(2)_FETCH_DEPS
+
+endif # $(2)_DEPS
+
+#
+# Build step. Only define it if not already defined by the package .mk file.
+# There is no differences between host and target packages.
+#
+ifndef $(2)_BUILD_CMDS
+define $(2)_BUILD_CMDS
+	$(foreach p,$$($$(PKG)_BUILD_PACKAGES), cd $$($$(PKG)_SRCDIR); \
+		env -i $(GO_ENV) gb build $(p))
+endef
+endif
+
+#
+# The variable $(2)_INSTALL_BINARIES list the binaries to install. Go programs
+# are named after the last part of their package. If $(2)_INSTALL_BINARIES is
+# not define, it is created from $(2)_BUILD_PACKAGES. 
+#
+ifndef $(2)_INSTALL_BINARIES
+ ifdef $(3)_INSTALL_BINARIES
+  $(2)_INSTALL_BINARIES = $$($(3)_INSTALL_BINARIES)
+ else
+  $(2)_INSTALL_BINARIES = $(foreach p, $($(2)_BUILD_PACKAGES), $(lastword $(subst /, ,$(p))))
+ endif
+endif
+
+#
+# Host installation step. Only define it if not already defined by the
+# package .mk file.
+#
+ifndef $(2)_INSTALL_CMDS
+define $(2)_INSTALL_CMDS
+	$(foreach p,$$($$(PKG)_INSTALL_BINARIES), \
+		$(INSTALL) -m 755 $$($$(PKG)_DIR)/bin/$(p) $(HOST_DIR)/usr/bin)
+endef
+endif
+
+#
+# Target installation step. Only define it if not already defined by the
+# package .mk file.
+#
+ifndef $(2)_INSTALL_TARGET_CMDS
+define $(2)_INSTALL_TARGET_CMDS
+	$(foreach p,$$($$(PKG)_INSTALL_BINARIES), \
+		$(INSTALL) -m 755 $$($$(PKG)_DIR)/bin/$(p) $(TARGET_DIR)/usr/bin)
+endef
+endif
+
+# Call the generic package infrastructure to generate the necessary make
+# targets
+$(call inner-generic-package,$(1),$(2),$(3),$(4))
+
+endef # inner-golang-package
+
+################################################################################
+# golang-package -- the target generator macro for Go packages
+################################################################################
+
+golang-package = $(call inner-golang-package,\
+	$(pkgname),$(call UPPERCASE,$(pkgname)),$(call UPPERCASE,$(pkgname)),target)
+host-golang-package = $(call inner-golang-package,host-$(pkgname),\
+	$(call UPPERCASE,host-$(pkgname)),$(call UPPERCASE,$(pkgname)),host)
diff --git a/support/scripts/pkg-stats b/support/scripts/pkg-stats
index fad7ae9..5e6bc08 100755
--- a/support/scripts/pkg-stats
+++ b/support/scripts/pkg-stats
@@ -112,6 +112,7 @@ for i in $(find boot/ linux/ package/ -name '*.mk' | sort) ; do
 	$i = "package/pkg-cmake.mk" -o \
 	$i = "package/pkg-kconfig.mk" -o \
 	$i = "package/pkg-luarocks.mk" -o \
+	$i = "package/pkg-golang.mk" -o \
 	$i = "package/pkg-perl.mk" -o \
 	$i = "package/pkg-python.mk" -o \
 	$i = "package/pkg-rebar.mk" -o \
@@ -145,6 +146,16 @@ for i in $(find boot/ linux/ package/ -name '*.mk' | sort) ; do
 	hastarget=1
     fi
 
+    if grep -E "\(host-golang-package\)" $i > /dev/null ; then
+	infratype="golang"
+	hashost=1
+    fi
+
+    if grep -E "\(golang-package\)" $i > /dev/null ; then
+	infratype="golang"
+	hastarget=1
+    fi
+
     if grep -E "\(host-luarocks-package\)" $i > /dev/null ; then
 	infratype="luarocks"
 	hashost=1
-- 
2.7.0

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 5/8 RFC] docs/manual: documents the golang-package infrastructure
  2016-02-01 23:30 [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Ludovic Guegan
                   ` (3 preceding siblings ...)
  2016-02-01 23:31 ` [Buildroot] [PATCH 4/8 RFC] package/pkg-golang.mk: infrastructure for Go packages Ludovic Guegan
@ 2016-02-01 23:31 ` Ludovic Guegan
  2016-06-01 20:28   ` Thomas Petazzoni
  2016-02-01 23:31 ` [Buildroot] [PATCH 6/8 RFC] package/rtop: add rtop package in Go Ludovic Guegan
                   ` (3 subsequent siblings)
  8 siblings, 1 reply; 14+ messages in thread
From: Ludovic Guegan @ 2016-02-01 23:31 UTC (permalink / raw)
  To: buildroot

Signed-off-by: Ludovic Guegan <ludovic.guegan@gmail.com>
---
 docs/manual/adding-packages-golang.txt | 73 ++++++++++++++++++++++++++++++++++
 1 file changed, 73 insertions(+)
 create mode 100644 docs/manual/adding-packages-golang.txt

diff --git a/docs/manual/adding-packages-golang.txt b/docs/manual/adding-packages-golang.txt
new file mode 100644
index 0000000..0cb2aa1
--- /dev/null
+++ b/docs/manual/adding-packages-golang.txt
@@ -0,0 +1,73 @@
+// -*- mode:doc; -*-
+// vim: set syntax=asciidoc:
+
+=== Infrastructure for Go packages
+
+[[golang-package-tutorial]]
+
+==== +golang-package+ tutorial
+
+First, let's see how to write a +.mk+ file for a Go package, with an
+example :
+
+------------------------
+01: ################################################################################
+02: #
+03: # foo-bar
+04: #
+05: ################################################################################
+06:
+07: FOO_LICENSE = MIT
+08: FOO_VERSION = release_1.0
+09: FOO_SITE = $(call github,foo,bar,$(FOO_VERSION))
+10: FOO_EXTRACT_DIR = $(@D)/src/github.com/foo/bar
+11:
+12: FOO_DEPS += $(call fetch-golang-package, github.com/other/package, \
+13: 		1f22c0103821b9390939b6776727195525381532)
+14:
+16: FOO_BUILD_PACKAGES = github.com/foo/bar/cmd/foo
+17:
+18: $(eval $(golang-package))
+------------------------
+
+On line 7, we declare the license of the package.
+
+On line 8 and 9, we declare the version and the method to download the
+sources.
+
+On line 10, we declare where to extract the sources. The GOPATH variable is
+set to $(@D) when building the package so the Go source files must be placed
+accordingly into $(@D)/src.
+
+On line 11 and 12, we declare the package dependencies to install. All the
+package dependencies should be added this way.
+
+On line 16, we declare the list of packages to build and to install.
+
+Finally, on line 18, we invoke the +golang-package+ macro that
+generates all the Makefile rules that actually allow the package to be
+built.
+
+[[golang-package-reference]]
+
+==== +golang-package+ reference
+
+As a policy, each Go package declares all its dependencies using the
+fetch-golang-package macro in order to produce reproducible builds.
+
+The main macro of the golang package infrastructure is +golang-package+. It
+is similar to the +generic-package+ macro. The ability to have target and
+host packages is also available, with the +host-golang-package+ macro.
+
+Just like the generic infrastructure, the golang infrastructure works by
+defining a number of variables before calling the +golang-package+ macro.
+
+First, all the package metadata information variables that exist in the
+generic infrastructure also exist in the golang infrastructure:
++FOO_FOO_VERSION+, +FOO_FOO_SOURCE+, +FOO_FOO_PATCH+, +FOO_FOO_SITE+,
++FOO_FOO_SUBDIR+, +FOO_FOO_DEPENDENCIES+, +FOO_FOO_INSTALL_TARGET+.
+
+Note that setting +FOO_INSTALL_STAGING+ to +YES+ has no effect unless a
++FOO_INSTALL_STAGING_CMDS+ variable is defined. The golang infrastructure
+doesn't define these commands since Go pakages don't need to be installed to
+the +staging+ directory.
-- 
2.7.0

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [Buildroot]  [PATCH 6/8 RFC] package/rtop: add rtop package in Go
  2016-02-01 23:30 [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Ludovic Guegan
                   ` (4 preceding siblings ...)
  2016-02-01 23:31 ` [Buildroot] [PATCH 5/8 RFC] docs/manual: documents the golang-package infrastructure Ludovic Guegan
@ 2016-02-01 23:31 ` Ludovic Guegan
  2016-02-01 23:31 ` [Buildroot] [PATCH 7/8 RFC] package/embd: add embd package (embedded programming framework) Ludovic Guegan
                   ` (2 subsequent siblings)
  8 siblings, 0 replies; 14+ messages in thread
From: Ludovic Guegan @ 2016-02-01 23:31 UTC (permalink / raw)
  To: buildroot

Signed-off-by: Ludovic Guegan <ludovic.guegan@gmail.com>
---
 package/Config.in      |  1 +
 package/rtop/Config.in |  8 ++++++++
 package/rtop/rtop.hash |  1 +
 package/rtop/rtop.mk   | 16 ++++++++++++++++
 4 files changed, 26 insertions(+)
 create mode 100644 package/rtop/Config.in
 create mode 100644 package/rtop/rtop.hash
 create mode 100644 package/rtop/rtop.mk

diff --git a/package/Config.in b/package/Config.in
index 09c2b40..d0e5e4e 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -483,6 +483,7 @@ menu "Erlang libraries/modules"
 endmenu
 endif
 	source "package/gauche/Config.in"
+	source "package/rtop/Config.in"
 	source "package/guile/Config.in"
 	source "package/haserl/Config.in"
 	source "package/jamvm/Config.in"
diff --git a/package/rtop/Config.in b/package/rtop/Config.in
new file mode 100644
index 0000000..857112a
--- /dev/null
+++ b/package/rtop/Config.in
@@ -0,0 +1,8 @@
+config BR2_PACKAGE_RTOP
+	bool "rtop"
+	depends on BR2_PACKAGE_HOST_GOLANG
+	help
+	  rtop is an interactive, remote system monitoring tool based on SSH
+
+	  https://github.com/rapidloop/rtop
+
diff --git a/package/rtop/rtop.hash b/package/rtop/rtop.hash
new file mode 100644
index 0000000..1407231
--- /dev/null
+++ b/package/rtop/rtop.hash
@@ -0,0 +1 @@
+sha256 44ab02299a204befe49d725bdb76a249db419807e13b7ca451ba14ea5d3de1c2  rtop-release_1.0.tar.gz
diff --git a/package/rtop/rtop.mk b/package/rtop/rtop.mk
new file mode 100644
index 0000000..f233ebe
--- /dev/null
+++ b/package/rtop/rtop.mk
@@ -0,0 +1,16 @@
+################################################################################
+#
+# rtop
+#
+################################################################################
+
+RTOP_LICENSE = MIT
+RTOP_VERSION = release_1.0
+RTOP_SITE = $(call github,rapidloop,rtop,$(RTOP_VERSION))
+RTOP_EXTRACT_DIR = $(@D)/src/github.com/rapidloop/rtop
+RTOP_BUILD_PACKAGES = github.com/rapidloop/rtop
+
+RTOP_DEPS += $(call fetch-golang-package, golang.org/x/crypto/curve25519, 1f22c0103821b9390939b6776727195525381532)
+RTOP_DEPS += $(call fetch-golang-package, golang.org/x/crypto/ssh, 1f22c0103821b9390939b6776727195525381532)
+
+$(eval $(golang-package))
-- 
2.7.0

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 7/8 RFC] package/embd: add embd package (embedded programming framework)
  2016-02-01 23:30 [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Ludovic Guegan
                   ` (5 preceding siblings ...)
  2016-02-01 23:31 ` [Buildroot] [PATCH 6/8 RFC] package/rtop: add rtop package in Go Ludovic Guegan
@ 2016-02-01 23:31 ` Ludovic Guegan
  2016-02-01 23:31 ` [Buildroot] [PATCH 8/8 RFC] package/bolt: add bolt package (persistent key-value store) Ludovic Guegan
  2016-06-01 20:14 ` [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Thomas Petazzoni
  8 siblings, 0 replies; 14+ messages in thread
From: Ludovic Guegan @ 2016-02-01 23:31 UTC (permalink / raw)
  To: buildroot

Signed-off-by: Ludovic Guegan <ludovic.guegan@gmail.com>
---
 package/Config.in      |  1 +
 package/embd/Config.in |  8 ++++++++
 package/embd/embd.hash |  1 +
 package/embd/embd.mk   | 16 ++++++++++++++++
 4 files changed, 26 insertions(+)
 create mode 100644 package/embd/Config.in
 create mode 100644 package/embd/embd.hash
 create mode 100644 package/embd/embd.mk

diff --git a/package/Config.in b/package/Config.in
index d0e5e4e..017ae97 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -1152,6 +1152,7 @@ menu "Other"
 	source "package/ding-libs/Config.in"
 	source "package/eigen/Config.in"
 	source "package/elfutils/Config.in"
+	source "package/embd/Config.in"
 	source "package/fftw/Config.in"
 	source "package/flann/Config.in"
 	source "package/gflags/Config.in"
diff --git a/package/embd/Config.in b/package/embd/Config.in
new file mode 100644
index 0000000..36ff68b
--- /dev/null
+++ b/package/embd/Config.in
@@ -0,0 +1,8 @@
+config BR2_PACKAGE_EMBD
+	bool "embd"
+	depends on BR2_PACKAGE_HOST_GOLANG
+	help
+	  Golang Embedded Programming Framework
+
+	  http://embd.io
+
diff --git a/package/embd/embd.hash b/package/embd/embd.hash
new file mode 100644
index 0000000..cafb76d
--- /dev/null
+++ b/package/embd/embd.hash
@@ -0,0 +1 @@
+sha256 b3a7885855920fc915ee7075996bd064be20e95381dbe9cf5aa1f82b133dbf82 embd-bfcd1345fe4e3d17ca82475c0c2f3d53120f87a9.tar.gz
diff --git a/package/embd/embd.mk b/package/embd/embd.mk
new file mode 100644
index 0000000..4be0935
--- /dev/null
+++ b/package/embd/embd.mk
@@ -0,0 +1,16 @@
+################################################################################
+#
+# embd
+#
+################################################################################
+
+EMBD_LICENSE = MIT
+EMBD_VERSION = bfcd1345fe4e3d17ca82475c0c2f3d53120f87a9
+EMBD_SITE = $(call github,kidoman,embd,$(EMBD_VERSION))
+EMBD_EXTRACT_DIR = $(@D)/src/github.com/kidoman/embd
+EMBD_BUILD_PACKAGES = github.com/kidoman/embd/embd
+
+EMBD_DEPS += $(call fetch-golang-package, github.com/codegangsta/cli, cf1f63a7274872768d4037305d572b70b1199397)
+EMBD_DEPS += $(call fetch-golang-package, github.com/golang/glog, 23def4e6c14b4da8ac2ed8007337bc5eb5007998)
+
+$(eval $(golang-package))
-- 
2.7.0

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 8/8 RFC] package/bolt: add bolt package (persistent key-value store)
  2016-02-01 23:30 [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Ludovic Guegan
                   ` (6 preceding siblings ...)
  2016-02-01 23:31 ` [Buildroot] [PATCH 7/8 RFC] package/embd: add embd package (embedded programming framework) Ludovic Guegan
@ 2016-02-01 23:31 ` Ludovic Guegan
  2016-06-01 20:14 ` [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Thomas Petazzoni
  8 siblings, 0 replies; 14+ messages in thread
From: Ludovic Guegan @ 2016-02-01 23:31 UTC (permalink / raw)
  To: buildroot

Signed-off-by: Ludovic Guegan <ludovic.guegan@gmail.com>
---
 package/Config.in      |  1 +
 package/bolt/Config.in |  7 +++++++
 package/bolt/bolt.hash |  1 +
 package/bolt/bolt.mk   | 13 +++++++++++++
 4 files changed, 22 insertions(+)
 create mode 100644 package/bolt/Config.in
 create mode 100644 package/bolt/bolt.hash
 create mode 100644 package/bolt/bolt.mk

diff --git a/package/Config.in b/package/Config.in
index 017ae97..7065034 100644
--- a/package/Config.in
+++ b/package/Config.in
@@ -815,6 +815,7 @@ endmenu
 
 menu "Database"
 	source "package/berkeleydb/Config.in"
+	source "package/bolt/Config.in"
 	source "package/cppdb/Config.in"
 	source "package/gdbm/Config.in"
 	source "package/kompexsqlite/Config.in"
diff --git a/package/bolt/Config.in b/package/bolt/Config.in
new file mode 100644
index 0000000..eae6b4c
--- /dev/null
+++ b/package/bolt/Config.in
@@ -0,0 +1,7 @@
+config BR2_PACKAGE_BOLT
+	bool "bolt"
+	depends on BR2_PACKAGE_HOST_GOLANG
+	help
+	  An embedded key/value database for Go.
+
+	  https://github.com/boltdb/bolt
diff --git a/package/bolt/bolt.hash b/package/bolt/bolt.hash
new file mode 100644
index 0000000..694cac5
--- /dev/null
+++ b/package/bolt/bolt.hash
@@ -0,0 +1 @@
+sha256 f426e649dd7c0838efe7bedff1010dfd047d0f354b264af6a030440543c42bd1  bolt-v1.1.0.tar.gz
diff --git a/package/bolt/bolt.mk b/package/bolt/bolt.mk
new file mode 100644
index 0000000..01e3243
--- /dev/null
+++ b/package/bolt/bolt.mk
@@ -0,0 +1,13 @@
+################################################################################
+#
+# bolt
+#
+################################################################################
+
+BOLT_LICENSE = MIT
+BOLT_VERSION = v1.1.0
+BOLT_SITE = $(call github,boltdb,bolt,$(BOLT_VERSION))
+BOLT_EXTRACT_DIR = $(@D)/src/github.com/boltdb/bolt
+BOLT_BUILD_PACKAGES = github.com/boltdb/bolt/cmd/bolt
+
+$(eval $(golang-package))
-- 
2.7.0

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure
  2016-02-01 23:30 [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Ludovic Guegan
                   ` (7 preceding siblings ...)
  2016-02-01 23:31 ` [Buildroot] [PATCH 8/8 RFC] package/bolt: add bolt package (persistent key-value store) Ludovic Guegan
@ 2016-06-01 20:14 ` Thomas Petazzoni
  8 siblings, 0 replies; 14+ messages in thread
From: Thomas Petazzoni @ 2016-06-01 20:14 UTC (permalink / raw)
  To: buildroot

Hello Ludovic,

On Tue,  2 Feb 2016 00:30:57 +0100, Ludovic Guegan wrote:

> Ludovic Guegan (8):
>   package/golang: create Go 1.5 host compiler package
>   package/golang-gb: create gb package to build and fetch go packages
>   package/pkg-generic.mk: allow custom extract directory
>   package/pkg-golang.mk: infrastructure for Go packages
>   docs/manual: documents the golang-package infrastructure
>   package/rtop: add rtop package in Go
>   package/embd: add embd package (embedded programming framework)
>   package/bolt: add bolt package (persistent key-value store)

Sorry for the slow feedback on this. We now have Go support in
Buildroot. For the moment, we only have one package in the tree
(flannel), but Christian Stewart has already submitted three more. And
indeed, the build procedure for each of them look very similar, so a
package infrastructure might very well be useful.

Would you be interested in respining your patch series against the
latest master, so that we can move forward with this topic?

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

^ permalink raw reply	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 2/8 RFC] package/golang-gb: create gb package to build and fetch go packages
  2016-02-01 23:30 ` [Buildroot] [PATCH 2/8 RFC] package/golang-gb: create gb package to build and fetch go packages Ludovic Guegan
@ 2016-06-01 20:17   ` Thomas Petazzoni
  0 siblings, 0 replies; 14+ messages in thread
From: Thomas Petazzoni @ 2016-06-01 20:17 UTC (permalink / raw)
  To: buildroot

Hello,

On Tue,  2 Feb 2016 00:30:59 +0100, Ludovic Guegan wrote:
> Signed-off-by: Ludovic Guegan <ludovic.guegan@gmail.com>
> ---
>  package/Config.in.host           |  1 +
>  package/golang-gb/Config.in.host |  8 ++++++++
>  package/golang-gb/golang-gb.mk   | 30 ++++++++++++++++++++++++++++++
>  3 files changed, 39 insertions(+)
>  create mode 100644 package/golang-gb/Config.in.host
>  create mode 100644 package/golang-gb/golang-gb.mk
> 
> diff --git a/package/Config.in.host b/package/Config.in.host
> index 0625bd0..2d99d37 100644
> --- a/package/Config.in.host
> +++ b/package/Config.in.host
> @@ -13,6 +13,7 @@ menu "Host utilities"
>  	source "package/genimage/Config.in.host"
>  	source "package/genpart/Config.in.host"
>  	source "package/golang/Config.in.host"
> +	source "package/golang-gb/Config.in.host"
>  	source "package/gptfdisk/Config.in.host"
>  	source "package/imx-usb-loader/Config.in.host"
>  	source "package/jq/Config.in.host"
> diff --git a/package/golang-gb/Config.in.host b/package/golang-gb/Config.in.host
> new file mode 100644
> index 0000000..b84a83f
> --- /dev/null
> +++ b/package/golang-gb/Config.in.host
> @@ -0,0 +1,8 @@
> +config BR2_PACKAGE_HOST_GOLANG_GB
> +	bool "golang-gb"
> +	depends on BR2_PACKAGE_HOST_GOLANG
> +	help
> +          gb is an alternative build tool for the Go programming language.
> +
> +	  https://getgb.io

I'm completely ignorant about Go. Why do we need this alternative build
tool? The flannel package contributed by Geoff, and the three packages
contributed by Christian don't need this alternative build tool.

What are the benefits of using this as opposed to what Geoff/Christian
are doing in their Go packages?

> +define HOST_GOLANG_GB_EXTRACT_CMDS
> +	mkdir -p $(@D)/src/github.com/constabulary
> +	tar xvaf $(DL_DIR)/golang-gb-$(GOLANG_GB_VERSION).tar.gz -C $(@D)
> +	mv $(@D)/gb-$(GOLANG_GB_VERSION) $(@D)/src/github.com/constabulary/gb

It is not clear why you are doing this here.

> +endef
> +
> +define HOST_GOLANG_GB_BUILD_CMDS
> +	env -i $(GOLANG_GB_ENV) $(GO_HOST_BINARY) install -v github.com/constabulary/gb/cmd/gb
> +	env -i $(GOLANG_GB_ENV) $(GO_HOST_BINARY) install -v github.com/constabulary/gb/cmd/gb-vendor

Why env -i ?

> +endef
> +
> +define HOST_GOLANG_GB_INSTALL_CMDS
> +	$(INSTALL) -m 755 $(@D)/bin/gb $(HOST_DIR)/usr/bin
> +	$(INSTALL) -m 755 $(@D)/bin/gb-vendor $(HOST_DIR)/usr/bin

We normally want:

	$(INSTALL) -D -m 0755 $(@D)/bin/gb $(HOST_DIR)/usr/bin/gb

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

^ permalink raw reply	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 3/8 RFC] package/pkg-generic.mk: allow custom extract directory
  2016-02-01 23:31 ` [Buildroot] [PATCH 3/8 RFC] package/pkg-generic.mk: allow custom extract directory Ludovic Guegan
@ 2016-06-01 20:19   ` Thomas Petazzoni
  0 siblings, 0 replies; 14+ messages in thread
From: Thomas Petazzoni @ 2016-06-01 20:19 UTC (permalink / raw)
  To: buildroot

Hello,

On Tue,  2 Feb 2016 00:31:00 +0100, Ludovic Guegan wrote:
> Signed-off-by: Ludovic Guegan <ludovic.guegan@gmail.com>
> ---
>  docs/manual/adding-packages-generic.txt |  3 +++
>  package/pkg-generic.mk                  | 12 ++++++++++--
>  2 files changed, 13 insertions(+), 2 deletions(-)

Do we need to allow a custom extract directory? The problem I see is
that with this change, the stamp files are no longer with the package
source code, which is a bit annoying.

Geoff and Christian instead create symlink to ensure that the Go
package source code appears at the right place in the "Go workspace". I
think it's a better approach (at least that's my feeling at the moment,
we'll see down the road if it causes problems or not).

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

^ permalink raw reply	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 4/8 RFC] package/pkg-golang.mk: infrastructure for Go packages
  2016-02-01 23:31 ` [Buildroot] [PATCH 4/8 RFC] package/pkg-golang.mk: infrastructure for Go packages Ludovic Guegan
@ 2016-06-01 20:26   ` Thomas Petazzoni
  0 siblings, 0 replies; 14+ messages in thread
From: Thomas Petazzoni @ 2016-06-01 20:26 UTC (permalink / raw)
  To: buildroot

Hello,

On Tue,  2 Feb 2016 00:31:01 +0100, Ludovic Guegan wrote:
> +
> +GO_ENV = PATH=$(BR_PATH) GOPATH=$($PKGDIR)
> +
> +#
> +# If compiling for the target then set GOARCH accordingly.
> +#
> +
> +ifeq ($(4),target)
> +
> +ifeq ($(BR2_i386),y)
> +GOLANG_ARCH = 386
> +endif # i386
> +
> +ifeq ($(BR2_x86_64),y)
> +GOLANG_ARCH = amd64
> +endif # x86_64
> +
> +ifeq ($(BR2_arm),y)
> +GOLANG_ARCH = arm
> +endif # arm
> +
> +GO_ENV += GOARCH=$(GOLANG_ARCH)

This should all be rebased on top of the Go support that has been
merged, which already exposes some environment variables that can be
used to build Go packages.

> +endif # ($(4),target)
> +
> +################################################################################
> +#
> +# Go packages are normally fetch using "go get", unfortunatly this method
> +# fetch the HEAD reference of the package. We use golang-gb in order to
> +# implement reproducible builds.
> +#
> +#  argument 1 is the package name to fetch as specified in the source code
> +#  argument 2 is the revision reference
> +################################################################################
> +
> +define fetch-golang-package
> +	cd $($(PKG)_SRCDIR); \
> +		env -i $(GO_ENV) gb vendor fetch -no-recurse -revision $(2) $(1);
> +endef

This is not good. Fetching additional stuff within a package is not
correct, as it works around the Buildroot download and legal-info
mechanisms. If a Go package has a dependency, then a proper Buildroot
package should be created for it.

> +#
> +# Fetch go package if $($(PKG)_DEPS) is defined
> +#
> +ifdef $(2)_DEPS
> +
> +define $(2)_FETCH_DEPS
> +	mkdir -p $$($$(PKG)_DIR)/{src,vendor}
> +	cd $$($$(PKG)_SRCDIR); env -i $(GO_ENV) gb vendor delete --all
> +	$$($$(PKG)_DEPS)
> +endef
> +
> +$(2)_POST_DOWNLOAD_HOOKS += $(2)_FETCH_DEPS

See my comment above.

> +
> +endif # $(2)_DEPS
> +
> +#
> +# Build step. Only define it if not already defined by the package .mk file.
> +# There is no differences between host and target packages.
> +#
> +ifndef $(2)_BUILD_CMDS
> +define $(2)_BUILD_CMDS
> +	$(foreach p,$$($$(PKG)_BUILD_PACKAGES), cd $$($$(PKG)_SRCDIR); \
> +		env -i $(GO_ENV) gb build $(p))

We don't use "env -i" anywhere else in Buildroot, so I don't see why
you're using it everywhere here.

> +endef
> +endif
> +
> +#
> +# The variable $(2)_INSTALL_BINARIES list the binaries to install. Go programs
> +# are named after the last part of their package. If $(2)_INSTALL_BINARIES is
> +# not define, it is created from $(2)_BUILD_PACKAGES. 
> +#
> +ifndef $(2)_INSTALL_BINARIES
> + ifdef $(3)_INSTALL_BINARIES
> +  $(2)_INSTALL_BINARIES = $$($(3)_INSTALL_BINARIES)

So here, if we are a host package, and HOST_<pkg>_INSTALL_BINARIES is
not defined, we inherit the value of <pkg>_INSTALL_BINARIES (i.e the
value of the target package). Looks good.

> + else
> +  $(2)_INSTALL_BINARIES = $(foreach p, $($(2)_BUILD_PACKAGES), $(lastword $(subst /, ,$(p))))

What if $(2)_BUILD_PACKAGES is empty, but not $(3)_BUILD_PACKAGES?
Shouldn't $(2)_BUILD_PACKAGES inherit from $(3)_BUILD_PACKAGES when
$(2)_BUILD_PACKAGES is not defined?

> + endif
> +endif
> +
> +#
> +# Host installation step. Only define it if not already defined by the
> +# package .mk file.
> +#
> +ifndef $(2)_INSTALL_CMDS
> +define $(2)_INSTALL_CMDS
> +	$(foreach p,$$($$(PKG)_INSTALL_BINARIES), \
> +		$(INSTALL) -m 755 $$($$(PKG)_DIR)/bin/$(p) $(HOST_DIR)/usr/bin)

$(INSTALL) -D -m 0755 <origin> <full destination path>

So Go packages will only ever install programs in $(HOST_DIR)/usr/bin.
They never install icons, data files, configuration files, etc. ?

> +endef
> +endif
> +
> +#
> +# Target installation step. Only define it if not already defined by the
> +# package .mk file.
> +#
> +ifndef $(2)_INSTALL_TARGET_CMDS
> +define $(2)_INSTALL_TARGET_CMDS
> +	$(foreach p,$$($$(PKG)_INSTALL_BINARIES), \
> +		$(INSTALL) -m 755 $$($$(PKG)_DIR)/bin/$(p) $(TARGET_DIR)/usr/bin)

Same comments as above.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

^ permalink raw reply	[flat|nested] 14+ messages in thread
* [Buildroot] [PATCH 5/8 RFC] docs/manual: documents the golang-package infrastructure
  2016-02-01 23:31 ` [Buildroot] [PATCH 5/8 RFC] docs/manual: documents the golang-package infrastructure Ludovic Guegan
@ 2016-06-01 20:28   ` Thomas Petazzoni
  0 siblings, 0 replies; 14+ messages in thread
From: Thomas Petazzoni @ 2016-06-01 20:28 UTC (permalink / raw)
  To: buildroot

Hello,

On Tue,  2 Feb 2016 00:31:02 +0100, Ludovic Guegan wrote:
> +01: ################################################################################
> +02: #
> +03: # foo-bar
> +04: #
> +05: ################################################################################
> +06:
> +07: FOO_LICENSE = MIT
> +08: FOO_VERSION = release_1.0
> +09: FOO_SITE = $(call github,foo,bar,$(FOO_VERSION))
> +10: FOO_EXTRACT_DIR = $(@D)/src/github.com/foo/bar

Shouldn't this be just where the symlink should be created in the Go
workspace ?

FOO_WORKSPACE_LOCATION or something like that?

> +11:
> +12: FOO_DEPS += $(call fetch-golang-package, github.com/other/package, \
> +13: 		1f22c0103821b9390939b6776727195525381532)

As said in my review of patch 4/8, it is not good when one package
fetches another. github.com/other/package should be another package.

> +14:
> +16: FOO_BUILD_PACKAGES = github.com/foo/bar/cmd/foo

Couldn't this be expressed in terms of FOO_WORKSPACE_LOCATION, like:

FOO_BUILD_PACKAGES = $(FOO_WORKSPACE_LOCATION)/cmd/foo

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

^ permalink raw reply	[flat|nested] 14+ messages in thread
end of thread, other threads:[~2016-06-01 20:28 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-01 23:30 [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Ludovic Guegan
2016-02-01 23:30 ` [Buildroot] [PATCH 1/8 RFC] package/golang: create Go 1.5 host compiler package Ludovic Guegan
2016-02-01 23:30 ` [Buildroot] [PATCH 2/8 RFC] package/golang-gb: create gb package to build and fetch go packages Ludovic Guegan
2016-06-01 20:17   ` Thomas Petazzoni
2016-02-01 23:31 ` [Buildroot] [PATCH 3/8 RFC] package/pkg-generic.mk: allow custom extract directory Ludovic Guegan
2016-06-01 20:19   ` Thomas Petazzoni
2016-02-01 23:31 ` [Buildroot] [PATCH 4/8 RFC] package/pkg-golang.mk: infrastructure for Go packages Ludovic Guegan
2016-06-01 20:26   ` Thomas Petazzoni
2016-02-01 23:31 ` [Buildroot] [PATCH 5/8 RFC] docs/manual: documents the golang-package infrastructure Ludovic Guegan
2016-06-01 20:28   ` Thomas Petazzoni
2016-02-01 23:31 ` [Buildroot] [PATCH 6/8 RFC] package/rtop: add rtop package in Go Ludovic Guegan
2016-02-01 23:31 ` [Buildroot] [PATCH 7/8 RFC] package/embd: add embd package (embedded programming framework) Ludovic Guegan
2016-02-01 23:31 ` [Buildroot] [PATCH 8/8 RFC] package/bolt: add bolt package (persistent key-value store) Ludovic Guegan
2016-06-01 20:14 ` [Buildroot] [PATCH 0/8 RFC] Adding a Go package infrastructure Thomas Petazzoni

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.