[Buildroot] [PATCH v3 1/3] package/go-bootstrap: split into two stages: go1.4 and go1.19.5

[Buildroot] [PATCH v3 1/3] package/go-bootstrap: split into two stages: go1.4 and go1.19.5

[Christian Stewart via buildroot]

Go 1.20 requires a minimum version of go 1.17.13 to bootstrap.

https://go.dev/doc/go1.20#bootstrap

As Go 1.4 was the previous version that could be compiled with C, there is now
no way to bootstrap go with a C compiler, unless we use a two-stage bootstrap:

 - build host-go-bootstrap-1.4-20170531
 - build host-go-bootstrap-1.19.5 with host-go-bootstrap-1.4-20170531
 - build host-go-1.20 with host-go-bootstrap-1.19.5

This is implemented in this commit first, before upgrading host-go to 1.20.

Note: the .patch files from package/go version 1.19.x are not necessary for
package/go-bootstrap-stage2 and have not been included there.

Discussion of possible alternate approaches for future use:

https://lore.kernel.org/all/CA+h8R2rtcynkCBsz=_9yANOEguyPCOcQDj8_ns+cv8RS8+8t9A@mail.gmail.com/
https://lore.kernel.org/all/20220525234312.643dfc03@windsurf/T/

Signed-off-by: Christian Stewart <christian@paral.in>

---

v1 -> v2:

 - fix indentation of GOROOT_BOOTSTRAP in stage2 MAKE_ENV
 - adjust commit description
 - remove duplicate MAKE_ENV declaration line
 - reorder depends on lines in config.in.host

Signed-off-by: Christian Stewart <christian@paral.in>
---
 DEVELOPERS                                    |  2 +
 package/Config.in.host                        |  3 +-
 package/go-bootstrap-stage1/Config.in.host    |  6 ++
 .../go-bootstrap-stage1.hash}                 |  0
 .../go-bootstrap-stage1.mk                    | 46 ++++++++++++++++
 package/go-bootstrap-stage2/Config.in.host    |  4 ++
 .../go-bootstrap-stage2.hash                  |  3 +
 .../go-bootstrap-stage2.mk                    | 55 +++++++++++++++++++
 package/go-bootstrap/Config.in.host           |  7 ---
 package/go-bootstrap/go-bootstrap.mk          | 50 -----------------
 package/go/Config.in.host                     |  6 +-
 package/go/go.mk                              |  9 ++-
 12 files changed, 126 insertions(+), 65 deletions(-)
 create mode 100644 package/go-bootstrap-stage1/Config.in.host
 rename package/{go-bootstrap/go-bootstrap.hash => go-bootstrap-stage1/go-bootstrap-stage1.hash} (100%)
 create mode 100644 package/go-bootstrap-stage1/go-bootstrap-stage1.mk
 create mode 100644 package/go-bootstrap-stage2/Config.in.host
 create mode 100644 package/go-bootstrap-stage2/go-bootstrap-stage2.hash
 create mode 100644 package/go-bootstrap-stage2/go-bootstrap-stage2.mk
 delete mode 100644 package/go-bootstrap/Config.in.host
 delete mode 100644 package/go-bootstrap/go-bootstrap.mk

diff --git a/DEVELOPERS b/DEVELOPERS
index df2b98ad53..80305fdf1e 100644
--- a/DEVELOPERS
+++ b/DEVELOPERS
@@ -548,6 +548,8 @@ F:	package/docker-engine/
 F:	package/embiggen-disk/
 F:	package/fuse-overlayfs/
 F:	package/go/
+F:	package/go-bootstrap-stage1/
+F:	package/go-bootstrap-stage2/
 F:	package/gocryptfs/
 F:	package/mbpfan/
 F:	package/moby-buildkit/
diff --git a/package/Config.in.host b/package/Config.in.host
index dcadbfdfc1..fff75cd165 100644
--- a/package/Config.in.host
+++ b/package/Config.in.host
@@ -40,7 +40,8 @@ menu "Host utilities"
 	source "package/genpart/Config.in.host"
 	source "package/gnupg/Config.in.host"
 	source "package/go/Config.in.host"
-	source "package/go-bootstrap/Config.in.host"
+	source "package/go-bootstrap-stage1/Config.in.host"
+	source "package/go-bootstrap-stage2/Config.in.host"
 	source "package/google-breakpad/Config.in.host"
 	source "package/gptfdisk/Config.in.host"
 	source "package/imagemagick/Config.in.host"
diff --git a/package/go-bootstrap-stage1/Config.in.host b/package/go-bootstrap-stage1/Config.in.host
new file mode 100644
index 0000000000..56a743caf9
--- /dev/null
+++ b/package/go-bootstrap-stage1/Config.in.host
@@ -0,0 +1,6 @@
+config BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE1_ARCH_SUPPORTS
+	bool
+	# See src/cmd/dist/unix.c for the list of supported architectures
+	default y if BR2_HOSTARCH = "x86"
+	default y if BR2_HOSTARCH = "x86_64"
+	default y if BR2_HOSTARCH = "arm"
diff --git a/package/go-bootstrap/go-bootstrap.hash b/package/go-bootstrap-stage1/go-bootstrap-stage1.hash
similarity index 100%
rename from package/go-bootstrap/go-bootstrap.hash
rename to package/go-bootstrap-stage1/go-bootstrap-stage1.hash
diff --git a/package/go-bootstrap-stage1/go-bootstrap-stage1.mk b/package/go-bootstrap-stage1/go-bootstrap-stage1.mk
new file mode 100644
index 0000000000..77e7046cd7
--- /dev/null
+++ b/package/go-bootstrap-stage1/go-bootstrap-stage1.mk
@@ -0,0 +1,46 @@
+################################################################################
+#
+# go-bootstrap-stage1
+#
+################################################################################
+
+# Use last C-based Go compiler: v1.4.x
+# See https://golang.org/doc/install/source#bootstrapFromSource
+GO_BOOTSTRAP_STAGE1_VERSION = 1.4-bootstrap-20171003
+GO_BOOTSTRAP_STAGE1_SITE = https://dl.google.com/go
+GO_BOOTSTRAP_STAGE1_SOURCE = go$(GO_BOOTSTRAP_STAGE1_VERSION).tar.gz
+
+GO_BOOTSTRAP_STAGE1_LICENSE = BSD-3-Clause
+GO_BOOTSTRAP_STAGE1_LICENSE_FILES = LICENSE
+
+# The toolchain is needed for HOSTCC_NOCACHE used to compile the Go compiler.
+HOST_GO_BOOTSTRAP_STAGE1_DEPENDENCIES = toolchain
+
+HOST_GO_BOOTSTRAP_STAGE1_ROOT = $(HOST_DIR)/lib/go-$(GO_BOOTSTRAP_STAGE1_VERSION)
+
+# The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE.
+# See https://github.com/golang/go/issues/11685.
+HOST_GO_BOOTSTRAP_STAGE1_MAKE_ENV = \
+	GOOS=linux \
+	GOROOT_FINAL="$(HOST_GO_BOOTSTRAP_STAGE1_ROOT)" \
+	GOROOT="$(@D)" \
+	GOBIN="$(@D)/bin" \
+	CC=$(HOSTCC_NOCCACHE) \
+	CGO_ENABLED=0
+
+define HOST_GO_BOOTSTRAP_STAGE1_BUILD_CMDS
+	cd $(@D)/src && $(HOST_GO_BOOTSTRAP_STAGE1_MAKE_ENV) ./make.bash
+endef
+
+define HOST_GO_BOOTSTRAP_STAGE1_INSTALL_CMDS
+	$(INSTALL) -D -m 0755 $(@D)/bin/go $(HOST_GO_BOOTSTRAP_STAGE1_ROOT)/bin/go
+	$(INSTALL) -D -m 0755 $(@D)/bin/gofmt $(HOST_GO_BOOTSTRAP_STAGE1_ROOT)/bin/gofmt
+
+	cp -a $(@D)/lib $(HOST_GO_BOOTSTRAP_STAGE1_ROOT)/
+	cp -a $(@D)/pkg $(HOST_GO_BOOTSTRAP_STAGE1_ROOT)/
+
+	# https://golang.org/issue/2775
+	cp -a $(@D)/src $(HOST_GO_BOOTSTRAP_STAGE1_ROOT)/
+endef
+
+$(eval $(host-generic-package))
diff --git a/package/go-bootstrap-stage2/Config.in.host b/package/go-bootstrap-stage2/Config.in.host
new file mode 100644
index 0000000000..967ddaed1d
--- /dev/null
+++ b/package/go-bootstrap-stage2/Config.in.host
@@ -0,0 +1,4 @@
+config BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS
+	bool
+	default y
+	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE1_ARCH_SUPPORTS
diff --git a/package/go-bootstrap-stage2/go-bootstrap-stage2.hash b/package/go-bootstrap-stage2/go-bootstrap-stage2.hash
new file mode 100644
index 0000000000..4c22f0f274
--- /dev/null
+++ b/package/go-bootstrap-stage2/go-bootstrap-stage2.hash
@@ -0,0 +1,3 @@
+# From https://go.dev/dl
+sha256  8e486e8e85a281fc5ce3f0bedc5b9d2dbf6276d7db0b25d3ec034f313da0375f  go1.19.5.src.tar.gz
+sha256  2d36597f7117c38b006835ae7f537487207d8ec407aa9d9980794b2030cbc067  LICENSE
diff --git a/package/go-bootstrap-stage2/go-bootstrap-stage2.mk b/package/go-bootstrap-stage2/go-bootstrap-stage2.mk
new file mode 100644
index 0000000000..b623978500
--- /dev/null
+++ b/package/go-bootstrap-stage2/go-bootstrap-stage2.mk
@@ -0,0 +1,55 @@
+################################################################################
+#
+# go-bootstrap-stage2
+#
+################################################################################
+
+GO_BOOTSTRAP_STAGE2_VERSION = 1.19.5
+GO_BOOTSTRAP_STAGE2_SITE = https://storage.googleapis.com/golang
+GO_BOOTSTRAP_STAGE2_SOURCE = go$(GO_BOOTSTRAP_STAGE2_VERSION).src.tar.gz
+
+GO_BOOTSTRAP_STAGE2_LICENSE = BSD-3-Clause
+GO_BOOTSTRAP_STAGE2_LICENSE_FILES = LICENSE
+
+# Use go-bootstrap-stage1 to bootstrap.
+HOST_GO_BOOTSTRAP_STAGE2_DEPENDENCIES = host-go-bootstrap-stage1
+
+HOST_GO_BOOTSTRAP_STAGE2_ROOT = $(HOST_DIR)/lib/go-$(GO_BOOTSTRAP_STAGE2_VERSION)
+
+# The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE.
+# See https://github.com/golang/go/issues/11685.
+HOST_GO_BOOTSTRAP_STAGE2_MAKE_ENV = \
+	GO111MODULE=off \
+	GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_STAGE1_ROOT) \
+	GOROOT_FINAL=$(HOST_GO_BOOTSTRAP_STAGE2_ROOT) \
+	GOROOT="$(@D)" \
+	GOBIN="$(@D)/bin" \
+	GOOS=linux \
+	CC=$(HOSTCC_NOCCACHE) \
+	CXX=$(HOSTCXX_NOCCACHE) \
+	CGO_ENABLED=0
+
+define HOST_GO_BOOTSTRAP_STAGE2_BUILD_CMDS
+	cd $(@D)/src && \
+		$(HOST_GO_BOOTSTRAP_STAGE2_MAKE_ENV) ./make.bash $(if $(VERBOSE),-v)
+endef
+
+define HOST_GO_BOOTSTRAP_STAGE2_INSTALL_CMDS
+	$(INSTALL) -D -m 0755 $(@D)/bin/go $(HOST_GO_BOOTSTRAP_STAGE2_ROOT)/bin/go
+	$(INSTALL) -D -m 0755 $(@D)/bin/gofmt $(HOST_GO_BOOTSTRAP_STAGE2_ROOT)/bin/gofmt
+
+	cp -a $(@D)/lib $(HOST_GO_BOOTSTRAP_STAGE2_ROOT)/
+
+	mkdir -p $(HOST_GO_BOOTSTRAP_STAGE2_ROOT)/pkg
+	cp -a $(@D)/pkg/include $(@D)/pkg/linux_* $(HOST_GO_BOOTSTRAP_STAGE2_ROOT)/pkg/
+	cp -a $(@D)/pkg/tool $(HOST_GO_BOOTSTRAP_STAGE2_ROOT)/pkg/
+
+	# https://golang.org/issue/2775
+	cp -a $(@D)/src $(HOST_GO_BOOTSTRAP_STAGE2_ROOT)/
+
+	# Set all file timestamps to prevent the go compiler from rebuilding any
+	# built in packages when programs are built.
+	find $(HOST_GO_BOOTSTRAP_STAGE2_ROOT) -type f -exec touch -r $(@D)/bin/go {} \;
+endef
+
+$(eval $(host-generic-package))
diff --git a/package/go-bootstrap/Config.in.host b/package/go-bootstrap/Config.in.host
deleted file mode 100644
index fab80d24b4..0000000000
--- a/package/go-bootstrap/Config.in.host
+++ /dev/null
@@ -1,7 +0,0 @@
-config BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS
-	bool
-	# See src/cmd/dist/unix.c for the list of support
-	# architectures
-	default y if BR2_HOSTARCH = "x86"
-	default y if BR2_HOSTARCH = "x86_64"
-	default y if BR2_HOSTARCH = "arm"
diff --git a/package/go-bootstrap/go-bootstrap.mk b/package/go-bootstrap/go-bootstrap.mk
deleted file mode 100644
index 71696a1540..0000000000
--- a/package/go-bootstrap/go-bootstrap.mk
+++ /dev/null
@@ -1,50 +0,0 @@
-################################################################################
-#
-# go-bootstrap
-#
-################################################################################
-
-# Use last C-based Go compiler: v1.4.x
-# See https://golang.org/doc/install/source#bootstrapFromSource
-GO_BOOTSTRAP_VERSION = 1.4-bootstrap-20171003
-GO_BOOTSTRAP_SITE = https://dl.google.com/go
-GO_BOOTSTRAP_SOURCE = go$(GO_BOOTSTRAP_VERSION).tar.gz
-
-GO_BOOTSTRAP_LICENSE = BSD-3-Clause
-GO_BOOTSTRAP_LICENSE_FILES = LICENSE
-
-# To build programs that need cgo support the toolchain needs to be
-# available, so the toolchain is not needed to build host-go-bootstrap
-# itself, but needed by other packages that depend on
-# host-go-bootstrap.
-HOST_GO_BOOTSTRAP_DEPENDENCIES = toolchain
-
-HOST_GO_BOOTSTRAP_ROOT = $(HOST_DIR)/lib/go-$(GO_BOOTSTRAP_VERSION)
-
-# The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE
-# here.  See https://github.com/golang/go/issues/11685.
-HOST_GO_BOOTSTRAP_MAKE_ENV = \
-	GOOS=linux \
-	GOROOT_FINAL="$(HOST_GO_BOOTSTRAP_ROOT)" \
-	GOROOT="$(@D)" \
-	GOBIN="$(@D)/bin" \
-	CC=$(HOSTCC_NOCCACHE) \
-	CGO_ENABLED=0
-
-define HOST_GO_BOOTSTRAP_BUILD_CMDS
-	cd $(@D)/src && $(HOST_GO_BOOTSTRAP_MAKE_ENV) ./make.bash
-endef
-
-define HOST_GO_BOOTSTRAP_INSTALL_CMDS
-	$(INSTALL) -D -m 0755 $(@D)/bin/go $(HOST_GO_BOOTSTRAP_ROOT)/bin/go
-	$(INSTALL) -D -m 0755 $(@D)/bin/gofmt $(HOST_GO_BOOTSTRAP_ROOT)/bin/gofmt
-
-	cp -a $(@D)/lib $(HOST_GO_BOOTSTRAP_ROOT)/
-	cp -a $(@D)/pkg $(HOST_GO_BOOTSTRAP_ROOT)/
-
-	# There is a known issue which requires the go sources to be installed
-	# https://golang.org/issue/2775
-	cp -a $(@D)/src $(HOST_GO_BOOTSTRAP_ROOT)/
-endef
-
-$(eval $(host-generic-package))
diff --git a/package/go/Config.in.host b/package/go/Config.in.host
index ded02d3b3a..b87b862cec 100644
--- a/package/go/Config.in.host
+++ b/package/go/Config.in.host
@@ -2,7 +2,9 @@
 config BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
 	bool
 	default y
-	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS
+	# See https://go.dev/doc/install/source#environment
+	# See src/go/build/syslist.go for the list of supported architectures
 	depends on (BR2_arm && BR2_TOOLCHAIN_SUPPORTS_PIE) || BR2_aarch64 \
 		|| BR2_i386 || BR2_x86_64 || BR2_powerpc64le \
 		|| BR2_mips64 || BR2_mips64el || BR2_riscv || BR2_s390x
@@ -28,4 +30,4 @@ config BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS
 config BR2_PACKAGE_HOST_GO_HOST_ARCH_SUPPORTS
 	bool
 	default y
-	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS
+	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS
diff --git a/package/go/go.mk b/package/go/go.mk
index c38ae0b99c..b0dd002712 100644
--- a/package/go/go.mk
+++ b/package/go/go.mk
@@ -12,7 +12,7 @@ GO_LICENSE = BSD-3-Clause
 GO_LICENSE_FILES = LICENSE
 GO_CPE_ID_VENDOR = golang
 
-HOST_GO_DEPENDENCIES = host-go-bootstrap
+HOST_GO_DEPENDENCIES = host-go-bootstrap-stage2
 HOST_GO_GOPATH = $(HOST_DIR)/share/go-path
 HOST_GO_HOST_CACHE = $(HOST_DIR)/share/host-go-cache
 HOST_GO_ROOT = $(HOST_DIR)/lib/go
@@ -121,12 +121,12 @@ HOST_GO_HOST_ENV = \
 	CGO_CXXFLAGS="$(HOST_CXXFLAGS)" \
 	CGO_LDFLAGS="$(HOST_LDFLAGS)"
 
-# The go build system is not compatible with ccache, so use
-# HOSTCC_NOCCACHE.  See https://github.com/golang/go/issues/11685.
+# The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE.
+# See https://github.com/golang/go/issues/11685.
 HOST_GO_MAKE_ENV = \
 	GO111MODULE=off \
 	GOCACHE=$(HOST_GO_HOST_CACHE) \
-	GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_ROOT) \
+	GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_STAGE2_ROOT) \
 	GOROOT_FINAL=$(HOST_GO_ROOT) \
 	GOROOT="$(@D)" \
 	GOBIN="$(@D)/bin" \
@@ -154,7 +154,6 @@ define HOST_GO_INSTALL_CMDS
 	cp -a $(@D)/pkg/include $(@D)/pkg/linux_* $(HOST_GO_ROOT)/pkg/
 	cp -a $(@D)/pkg/tool $(HOST_GO_ROOT)/pkg/
 
-	# There is a known issue which requires the go sources to be installed
 	# https://golang.org/issue/2775
 	cp -a $(@D)/src $(HOST_GO_ROOT)/
 
-- 
2.39.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 2/3] package/go: bump to version 1.20.1

[Christian Stewart via buildroot]

The latest Go release, version 1.20, arrives six months after Go 1.19. Most of
its changes are in the implementation of the toolchain, runtime, and libraries.

https://go.dev/doc/go1.20

Go 1.20.1 fixes four minor security issues:

    CVE-2022-41725: net/http, mime/multipart: denial of service from excessive resource consumption.
    CVE-2022-41724: crypto/tls: large handshake records may cause panics
    CVE-2022-41723: net/http: avoid quadratic complexity in HPACK decoding

Signed-off-by: Christian Stewart <christian@paral.in>

---

v1 -> v2:

 - drop unnecessary change to config.in.host

v2 -> v3:

 - bump to go 1.20.1
 - add note about fixed cves

package/go: bump to 1.20.1

Signed-off-by: Christian Stewart <christian@paral.in>
---
 ...03-runtime-support-riscv64-SV57-mode.patch | 65 -------------------
 package/go/go.hash                            |  2 +-
 package/go/go.mk                              |  4 +-
 3 files changed, 3 insertions(+), 68 deletions(-)
 delete mode 100644 package/go/0003-runtime-support-riscv64-SV57-mode.patch

diff --git a/package/go/0003-runtime-support-riscv64-SV57-mode.patch b/package/go/0003-runtime-support-riscv64-SV57-mode.patch
deleted file mode 100644
index f51c2ca093..0000000000
--- a/package/go/0003-runtime-support-riscv64-SV57-mode.patch
+++ /dev/null
@@ -1,65 +0,0 @@
-From 6618c7af436488fa12018cdcd31eeedb3a698745 Mon Sep 17 00:00:00 2001
-From: Dmitry Vyukov <dvyukov@google.com>
-Date: Fri, 27 May 2022 18:55:35 +0200
-Subject: [PATCH] runtime: support riscv64 SV57 mode
-
-Riscv64 has SV57 mode when user-space VA is 56 bits.
-Linux kernel recently got support for this mode and Go binaries started crashing as:
-
-runtime: lfstack.push invalid packing: node=0xffffff5908a940 cnt=0x1
-packed=0xffff5908a9400001 -> node=0xffff5908a940
-
-Adjust lfstack code to use only 8 top bits of pointers on riscv64.
-
-For context see:
-https://groups.google.com/g/syzkaller-bugs/c/lU0GQTZoNQQ/m/O_c3vmE3AAAJ
-
-Update #54104
-
-Change-Id: Ib5d3d6a79c0c6eddf11618d73fcc8bc1832a9c25
-Signed-off-by: Christian Stewart <christian@paral.in>
----
-
-Upstream: https://go-review.googlesource.com/c/go/+/409055/4
----
- src/runtime/lfstack_64bit.go | 12 ++++++++++++
- 1 file changed, 12 insertions(+)
-
-diff --git a/src/runtime/lfstack_64bit.go b/src/runtime/lfstack_64bit.go
-index 154130cf63..39fa647b9e 100644
---- a/src/runtime/lfstack_64bit.go
-+++ b/src/runtime/lfstack_64bit.go
-@@ -36,12 +36,21 @@ const (
- 	// We use one bit to distinguish between the two ranges.
- 	aixAddrBits = 57
- 	aixCntBits  = 64 - aixAddrBits + 3
-+
-+	// Riscv64 SV57 mode gives 56 bits of userspace VA.
-+	// lfstack code supports it, but broader support for SV57 mode is incomplete,
-+	// and there may be other issues (see #54104).
-+	riscv64AddrBits = 56
-+	riscv64CntBits  = 64 - riscv64AddrBits + 3
- )
- 
- func lfstackPack(node *lfnode, cnt uintptr) uint64 {
- 	if GOARCH == "ppc64" && GOOS == "aix" {
- 		return uint64(uintptr(unsafe.Pointer(node)))<<(64-aixAddrBits) | uint64(cnt&(1<<aixCntBits-1))
- 	}
-+	if GOARCH == "riscv64" {
-+		return uint64(uintptr(unsafe.Pointer(node)))<<(64-riscv64AddrBits) | uint64(cnt&(1<<riscv64CntBits-1))
-+	}
- 	return uint64(uintptr(unsafe.Pointer(node)))<<(64-addrBits) | uint64(cnt&(1<<cntBits-1))
- }
- 
-@@ -54,5 +63,8 @@ func lfstackUnpack(val uint64) *lfnode {
- 	if GOARCH == "ppc64" && GOOS == "aix" {
- 		return (*lfnode)(unsafe.Pointer(uintptr((val >> aixCntBits << 3) | 0xa<<56)))
- 	}
-+	if GOARCH == "riscv64" {
-+		return (*lfnode)(unsafe.Pointer(uintptr(val >> riscv64CntBits << 3)))
-+	}
- 	return (*lfnode)(unsafe.Pointer(uintptr(val >> cntBits << 3)))
- }
--- 
-2.35.1
-
diff --git a/package/go/go.hash b/package/go/go.hash
index 4c22f0f274..8e5acfe42f 100644
--- a/package/go/go.hash
+++ b/package/go/go.hash
@@ -1,3 +1,3 @@
 # From https://go.dev/dl
-sha256  8e486e8e85a281fc5ce3f0bedc5b9d2dbf6276d7db0b25d3ec034f313da0375f  go1.19.5.src.tar.gz
+sha256  b5c1a3af52c385a6d1c76aed5361cf26459023980d0320de7658bae3915831a2  go1.20.1.src.tar.gz
 sha256  2d36597f7117c38b006835ae7f537487207d8ec407aa9d9980794b2030cbc067  LICENSE
diff --git a/package/go/go.mk b/package/go/go.mk
index b0dd002712..4db703a301 100644
--- a/package/go/go.mk
+++ b/package/go/go.mk
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-GO_VERSION = 1.19.5
+GO_VERSION = 1.20.1
 GO_SITE = https://storage.googleapis.com/golang
 GO_SOURCE = go$(GO_VERSION).src.tar.gz
 
@@ -151,7 +151,7 @@ define HOST_GO_INSTALL_CMDS
 	cp -a $(@D)/lib $(HOST_GO_ROOT)/
 
 	mkdir -p $(HOST_GO_ROOT)/pkg
-	cp -a $(@D)/pkg/include $(@D)/pkg/linux_* $(HOST_GO_ROOT)/pkg/
+	cp -a $(@D)/pkg/include $(HOST_GO_ROOT)/pkg/
 	cp -a $(@D)/pkg/tool $(HOST_GO_ROOT)/pkg/
 
 	# https://golang.org/issue/2775
-- 
2.39.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

[Buildroot] [PATCH v3 3/3] package/go: use host compiler when go-bootstrap unsupported

[Christian Stewart via buildroot]

All Go compiler versions > 1.4.x (old) are written in Go, and require a existing
compiled Go version to use to build from source.

https://golang.org/doc/install/source#bootstrapFromSource

The process for "bootstrapping" the Go compiler in Buildroot is:

1. Compile a C/C++ cross-compiler (gcc) as the host toolchain.
2. Build go-bootstrap-stage1 (which is Go 1.4.x and written in C)
3. Build go-bootstrap-stage2 (which is Go 1.19.x and written in Go)
3. Build go 1.20 (written in Go) using go-bootstrap-stage2.

go-bootstrap-stage1 does not work on 64-bit arm. The Go 1.4.x bootstrap compiler
is compatible with x86, x86_64, and arm (32 bit) only.

This patch adds a fallback to require a host Go compiler to build host-go when
BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS is not set.

Signed-off-by: Christian Stewart <christian@paral.in>

---

changes prior to inclusion in this series:

 - thanks Thomas for the review & suggestions
 - added NEEDS_HOST_GO boolean
 - added dependency checks to support/dependencies/dependencies.sh
 - removed unnecessary changes to go-bootstrap package
 - add dependency on toolchain if Cgo is enabled
 - updates for go1.20
 - updates for go-bootstrap-stage{1,2}

changes from v1 -> v2:

 - remove whitespace fix in bootstrap stage2

Signed-off-by: Christian Stewart <christian@paral.in>
---
 Config.in                            |  4 ++++
 package/go/Config.in.host            |  2 +-
 package/go/go.mk                     | 14 ++++++++++++--
 support/dependencies/dependencies.sh |  4 ++++
 4 files changed, 21 insertions(+), 3 deletions(-)

diff --git a/Config.in b/Config.in
index 0d7641633c..c70ce94d94 100644
--- a/Config.in
+++ b/Config.in
@@ -58,6 +58,10 @@ config BR2_HOST_GCC_AT_LEAST_9
 # When adding new entries above, be sure to update
 # the HOSTCC_MAX_VERSION variable in the Makefile.
 
+# Hidden boolean selected if bootstrapping Go w/ GCC is not supported.
+config BR2_NEEDS_HOST_GO
+	bool
+
 # Hidden boolean selected by packages in need of Java in order to build
 # (example: kodi)
 config BR2_NEEDS_HOST_JAVA
diff --git a/package/go/Config.in.host b/package/go/Config.in.host
index b87b862cec..7f049ff1ae 100644
--- a/package/go/Config.in.host
+++ b/package/go/Config.in.host
@@ -30,4 +30,4 @@ config BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS
 config BR2_PACKAGE_HOST_GO_HOST_ARCH_SUPPORTS
 	bool
 	default y
-	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS
+	select BR2_NEEDS_HOST_GO if !BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS
diff --git a/package/go/go.mk b/package/go/go.mk
index 4db703a301..68f6d9e36b 100644
--- a/package/go/go.mk
+++ b/package/go/go.mk
@@ -12,7 +12,6 @@ GO_LICENSE = BSD-3-Clause
 GO_LICENSE_FILES = LICENSE
 GO_CPE_ID_VENDOR = golang
 
-HOST_GO_DEPENDENCIES = host-go-bootstrap-stage2
 HOST_GO_GOPATH = $(HOST_DIR)/share/go-path
 HOST_GO_HOST_CACHE = $(HOST_DIR)/share/host-go-cache
 HOST_GO_ROOT = $(HOST_DIR)/lib/go
@@ -109,6 +108,11 @@ else # !BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
 HOST_GO_CGO_ENABLED = 1
 endif # BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
 
+ifeq ($(HOST_GO_CGO_ENABLED),1)
+# For cgo support the toolchain needs to be available.
+HOST_GO_DEPENDENCIES += toolchain
+endif
+
 # For the convenience of host golang packages
 HOST_GO_HOST_ENV = \
 	$(HOST_GO_COMMON_ENV) \
@@ -126,7 +130,6 @@ HOST_GO_HOST_ENV = \
 HOST_GO_MAKE_ENV = \
 	GO111MODULE=off \
 	GOCACHE=$(HOST_GO_HOST_CACHE) \
-	GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_STAGE2_ROOT) \
 	GOROOT_FINAL=$(HOST_GO_ROOT) \
 	GOROOT="$(@D)" \
 	GOBIN="$(@D)/bin" \
@@ -136,6 +139,13 @@ HOST_GO_MAKE_ENV = \
 	CGO_ENABLED=$(HOST_GO_CGO_ENABLED) \
 	$(HOST_GO_CROSS_ENV)
 
+# Use the Go compiler bootstrapped by Buildroot if available.
+# Otherwise, use the host Go compiler.
+ifeq ($(BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS),y)
+HOST_GO_DEPENDENCIES += host-go-bootstrap-stage2
+HOST_GO_MAKE_ENV += GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_STAGE2_ROOT)
+endif
+
 define HOST_GO_BUILD_CMDS
 	cd $(@D)/src && \
 		$(HOST_GO_MAKE_ENV) ./make.bash $(if $(VERBOSE),-v)
diff --git a/support/dependencies/dependencies.sh b/support/dependencies/dependencies.sh
index 4353e9585e..943c7d0daf 100755
--- a/support/dependencies/dependencies.sh
+++ b/support/dependencies/dependencies.sh
@@ -217,6 +217,10 @@ if grep -q ^BR2_NEEDS_HOST_UTF8_LOCALE=y $BR2_CONFIG ; then
 	fi
 fi
 
+if grep -q ^BR2_NEEDS_HOST_GO=y $BR2_CONFIG ; then
+	check_prog_host "go"
+fi
+
 if grep -q ^BR2_NEEDS_HOST_JAVA=y $BR2_CONFIG ; then
 	check_prog_host "java"
 	JAVA_GCJ=$(java -version 2>&1 | grep gcj)
-- 
2.39.1

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 1/3] package/go-bootstrap: split into two stages: go1.4 and go1.19.5

[Thomas Petazzoni via buildroot]

Hello Christian,

Thanks for working on this. I think we are not far from a good solution
that can be merged. However, I have a few comments/questions. See below.

On Tue, 14 Feb 2023 23:32:54 -0800
Christian Stewart via buildroot <buildroot@buildroot.org> wrote:

> +# Use last C-based Go compiler: v1.4.x
> +# See https://golang.org/doc/install/source#bootstrapFromSource
> +GO_BOOTSTRAP_STAGE1_VERSION = 1.4-bootstrap-20171003
> +GO_BOOTSTRAP_STAGE1_SITE = https://dl.google.com/go
> +GO_BOOTSTRAP_STAGE1_SOURCE = go$(GO_BOOTSTRAP_STAGE1_VERSION).tar.gz
> +
> +GO_BOOTSTRAP_STAGE1_LICENSE = BSD-3-Clause
> +GO_BOOTSTRAP_STAGE1_LICENSE_FILES = LICENSE
> +
> +# The toolchain is needed for HOSTCC_NOCACHE used to compile the Go compiler.

This doesn't make much sense. Why would the toolchain package be needed
for HOSTCC_NOCCACHE ? We have several packages that use HOSTCC_NOCCACHE
before the toolchain is ready.

The original comment in go-bootstrap.mk was:

# To build programs that need cgo support the toolchain needs to be
# available, so the toolchain is not needed to build host-go-bootstrap
# itself, but needed by other packages that depend on
# host-go-bootstrap.

and this made a bit more sense (even though I'm wondering why it's
host-go-bootstrap that carries this dependency, and not host-go).


> +	# Set all file timestamps to prevent the go compiler from rebuilding any
> +	# built in packages when programs are built.
> +	find $(HOST_GO_BOOTSTRAP_STAGE2_ROOT) -type f -exec touch -r $(@D)/bin/go {} \;

So we have to do this for bootstrap-stage2 but not bootstrap-stage1 ?


> -	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS
> +	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS
> +	# See https://go.dev/doc/install/source#environment
> +	# See src/go/build/syslist.go for the list of supported architectures

This comment looks good, but is unrelated. Separate patch?

>  	depends on (BR2_arm && BR2_TOOLCHAIN_SUPPORTS_PIE) || BR2_aarch64 \
>  		|| BR2_i386 || BR2_x86_64 || BR2_powerpc64le \
>  		|| BR2_mips64 || BR2_mips64el || BR2_riscv || BR2_s390x
> @@ -28,4 +30,4 @@ config BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS
>  config BR2_PACKAGE_HOST_GO_HOST_ARCH_SUPPORTS
>  	bool
>  	default y
> -	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS
> +	depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS
> diff --git a/package/go/go.mk b/package/go/go.mk
> index c38ae0b99c..b0dd002712 100644
> --- a/package/go/go.mk
> +++ b/package/go/go.mk
> @@ -12,7 +12,7 @@ GO_LICENSE = BSD-3-Clause
>  GO_LICENSE_FILES = LICENSE
>  GO_CPE_ID_VENDOR = golang
>  
> -HOST_GO_DEPENDENCIES = host-go-bootstrap
> +HOST_GO_DEPENDENCIES = host-go-bootstrap-stage2
>  HOST_GO_GOPATH = $(HOST_DIR)/share/go-path
>  HOST_GO_HOST_CACHE = $(HOST_DIR)/share/host-go-cache
>  HOST_GO_ROOT = $(HOST_DIR)/lib/go
> @@ -121,12 +121,12 @@ HOST_GO_HOST_ENV = \
>  	CGO_CXXFLAGS="$(HOST_CXXFLAGS)" \
>  	CGO_LDFLAGS="$(HOST_LDFLAGS)"
>  
> -# The go build system is not compatible with ccache, so use
> -# HOSTCC_NOCCACHE.  See https://github.com/golang/go/issues/11685.
> +# The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE.
> +# See https://github.com/golang/go/issues/11685.

Why is this comment being changed, with a typo added?

>  HOST_GO_MAKE_ENV = \
>  	GO111MODULE=off \
>  	GOCACHE=$(HOST_GO_HOST_CACHE) \
> -	GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_ROOT) \
> +	GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_STAGE2_ROOT) \
>  	GOROOT_FINAL=$(HOST_GO_ROOT) \
>  	GOROOT="$(@D)" \
>  	GOBIN="$(@D)/bin" \
> @@ -154,7 +154,6 @@ define HOST_GO_INSTALL_CMDS
>  	cp -a $(@D)/pkg/include $(@D)/pkg/linux_* $(HOST_GO_ROOT)/pkg/
>  	cp -a $(@D)/pkg/tool $(HOST_GO_ROOT)/pkg/
>  
> -	# There is a known issue which requires the go sources to be installed

Why is this comment being removed?

>  	# https://golang.org/issue/2775
>  	cp -a $(@D)/src $(HOST_GO_ROOT)/

Thanks!

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 3/3] package/go: use host compiler when go-bootstrap unsupported

[Thomas Petazzoni via buildroot]

Hello Christian,

On Tue, 14 Feb 2023 23:32:56 -0800
Christian Stewart via buildroot <buildroot@buildroot.org> wrote:

> The process for "bootstrapping" the Go compiler in Buildroot is:
> 
> 1. Compile a C/C++ cross-compiler (gcc) as the host toolchain.

I find "host toolchain" confusing here. I generally understand "host
toolchain" as "native toolchain".

> diff --git a/package/go/go.mk b/package/go/go.mk
> index 4db703a301..68f6d9e36b 100644
> --- a/package/go/go.mk
> +++ b/package/go/go.mk
> @@ -12,7 +12,6 @@ GO_LICENSE = BSD-3-Clause
>  GO_LICENSE_FILES = LICENSE
>  GO_CPE_ID_VENDOR = golang
>  
> -HOST_GO_DEPENDENCIES = host-go-bootstrap-stage2
>  HOST_GO_GOPATH = $(HOST_DIR)/share/go-path
>  HOST_GO_HOST_CACHE = $(HOST_DIR)/share/host-go-cache
>  HOST_GO_ROOT = $(HOST_DIR)/lib/go
> @@ -109,6 +108,11 @@ else # !BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
>  HOST_GO_CGO_ENABLED = 1
>  endif # BR2_PACKAGE_HOST_GO_TARGET_ARCH_SUPPORTS
>  
> +ifeq ($(HOST_GO_CGO_ENABLED),1)
> +# For cgo support the toolchain needs to be available.
> +HOST_GO_DEPENDENCIES += toolchain
> +endif

Why is this added as part of this commit? It seems completely
unrelated. And in fact, shouldn't this be the correct thing to do
instead of what PATCH 1/3 is doing with the toolchain dependency added
on host-go-bootstrap-stage2 ?

> diff --git a/support/dependencies/dependencies.sh b/support/dependencies/dependencies.sh
> index 4353e9585e..943c7d0daf 100755
> --- a/support/dependencies/dependencies.sh
> +++ b/support/dependencies/dependencies.sh
> @@ -217,6 +217,10 @@ if grep -q ^BR2_NEEDS_HOST_UTF8_LOCALE=y $BR2_CONFIG ; then
>  	fi
>  fi
>  
> +if grep -q ^BR2_NEEDS_HOST_GO=y $BR2_CONFIG ; then
> +	check_prog_host "go"
> +fi

I think this is missing a version check on go: you need at least go
1.17.x to be able to build a go >= 1.20. The check as written here does
not check the go version.

Also, I am wondering if we couldn't improve things a bit and use an
already installed (and sufficiently recent) Go compiler to save on
building host-go-bootstrap-stage{1,2} even on architectures where the
full bootstrap build is supported.

We would have the following cases:

 - Architecture supported by bootstrap-stage{1,2}

   - If system-provided Go is available and recent enough => use it and skip bootstrap
   - If not, do bootstrapping

 - Architecture not supported by bootstrap-stage{1,2}

   - If system-provided Go is available and recent enough => use it
   - If not, should not happen due to check

It is of course not mandatory to implement this right now. It would be
a nice to have, but not a must have for sure.

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 1/3] package/go-bootstrap: split into two stages: go1.4 and go1.19.5

[Christian Stewart via buildroot]

Thomas,

On Sun, Mar 12, 2023 at 2:58 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> On Tue, 14 Feb 2023 23:32:54 -0800
> Christian Stewart via buildroot <buildroot@buildroot.org> wrote:
>
> > +# Use last C-based Go compiler: v1.4.x
> > +# See https://golang.org/doc/install/source#bootstrapFromSource
> > +GO_BOOTSTRAP_STAGE1_VERSION = 1.4-bootstrap-20171003
> > +GO_BOOTSTRAP_STAGE1_SITE = https://dl.google.com/go
> > +GO_BOOTSTRAP_STAGE1_SOURCE = go$(GO_BOOTSTRAP_STAGE1_VERSION).tar.gz
> > +
> > +GO_BOOTSTRAP_STAGE1_LICENSE = BSD-3-Clause
> > +GO_BOOTSTRAP_STAGE1_LICENSE_FILES = LICENSE
> > +
> > +# The toolchain is needed for HOSTCC_NOCACHE used to compile the Go compiler.
>
> This doesn't make much sense. Why would the toolchain package be needed
> for HOSTCC_NOCCACHE ? We have several packages that use HOSTCC_NOCCACHE
> before the toolchain is ready.

I suppose HOSTCC_NOCCACHE uses the gcc installed outside of buildroot,
so it should be fine to remove this dependency.


> > +     # Set all file timestamps to prevent the go compiler from rebuilding any
> > +     # built in packages when programs are built.
> > +     find $(HOST_GO_BOOTSTRAP_STAGE2_ROOT) -type f -exec touch -r $(@D)/bin/go {} \;
>
> So we have to do this for bootstrap-stage2 but not bootstrap-stage1 ?

It's not strictly necessary for either one. I removed it for the next
revision of the patch.


> > -     depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS
> > +     depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS
> > +     # See https://go.dev/doc/install/source#environment
> > +     # See src/go/build/syslist.go for the list of supported architectures
>
> This comment looks good, but is unrelated. Separate patch?

Is it really necessary to put in a separate patch? It's a minor comment change.

Feels appropriate to bundle it here with other related changes.

> >       depends on (BR2_arm && BR2_TOOLCHAIN_SUPPORTS_PIE) || BR2_aarch64 \
> >               || BR2_i386 || BR2_x86_64 || BR2_powerpc64le \
> >               || BR2_mips64 || BR2_mips64el || BR2_riscv || BR2_s390x
> > @@ -28,4 +30,4 @@ config BR2_PACKAGE_HOST_GO_TARGET_CGO_LINKING_SUPPORTS
> >  config BR2_PACKAGE_HOST_GO_HOST_ARCH_SUPPORTS
> >       bool
> >       default y
> > -     depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS
> > +     depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS
> > diff --git a/package/go/go.mk b/package/go/go.mk
> > index c38ae0b99c..b0dd002712 100644
> > --- a/package/go/go.mk
> > +++ b/package/go/go.mk
> > @@ -12,7 +12,7 @@ GO_LICENSE = BSD-3-Clause
> >  GO_LICENSE_FILES = LICENSE
> >  GO_CPE_ID_VENDOR = golang
> >
> > -HOST_GO_DEPENDENCIES = host-go-bootstrap
> > +HOST_GO_DEPENDENCIES = host-go-bootstrap-stage2
> >  HOST_GO_GOPATH = $(HOST_DIR)/share/go-path
> >  HOST_GO_HOST_CACHE = $(HOST_DIR)/share/host-go-cache
> >  HOST_GO_ROOT = $(HOST_DIR)/lib/go
> > @@ -121,12 +121,12 @@ HOST_GO_HOST_ENV = \
> >       CGO_CXXFLAGS="$(HOST_CXXFLAGS)" \
> >       CGO_LDFLAGS="$(HOST_LDFLAGS)"
> >
> > -# The go build system is not compatible with ccache, so use
> > -# HOSTCC_NOCCACHE.  See https://github.com/golang/go/issues/11685.
> > +# The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE.
> > +# See https://github.com/golang/go/issues/11685.
>
> Why is this comment being changed, with a typo added?

Fixed the typo. The comment is being changed because it can fit on a
single line in 75 characters.

It just looks cleaner.

I can revert this if necessary.

> >  HOST_GO_MAKE_ENV = \
> >       GO111MODULE=off \
> >       GOCACHE=$(HOST_GO_HOST_CACHE) \
> > -     GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_ROOT) \
> > +     GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_STAGE2_ROOT) \
> >       GOROOT_FINAL=$(HOST_GO_ROOT) \
> >       GOROOT="$(@D)" \
> >       GOBIN="$(@D)/bin" \
> > @@ -154,7 +154,6 @@ define HOST_GO_INSTALL_CMDS
> >       cp -a $(@D)/pkg/include $(@D)/pkg/linux_* $(HOST_GO_ROOT)/pkg/
> >       cp -a $(@D)/pkg/tool $(HOST_GO_ROOT)/pkg/
> >
> > -     # There is a known issue which requires the go sources to be installed
>
> Why is this comment being removed?

It's not an issue, per se. The nature of the Go compiler is that it
needs the sources to work.
That bug report has been closed for years now and is actually
unrelated to needing sources anyway.

Please let me know if:

 - I need to make a separate patch for that minor comment change (I vote not)
 - That comment adjustment should be reverted or not (I vote not)
 - The comment mentioning installing sources needs to be there (I vote not)

Otherwise I applied the other changes and am ready to respin a new revision.

Best regards,
Christian Stewart
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 1/3] package/go-bootstrap: split into two stages: go1.4 and go1.19.5

[Bagas Sanjaya]

On 2/15/23 14:32, Christian Stewart via buildroot wrote:
> Go 1.20 requires a minimum version of go 1.17.13 to bootstrap.
> 
> https://go.dev/doc/go1.20#bootstrap
> 
> As Go 1.4 was the previous version that could be compiled with C, there is now
> no way to bootstrap go with a C compiler, unless we use a two-stage bootstrap:
> 
>  - build host-go-bootstrap-1.4-20170531
>  - build host-go-bootstrap-1.19.5 with host-go-bootstrap-1.4-20170531
>  - build host-go-1.20 with host-go-bootstrap-1.19.5
> 

I prefer using gccgo to bootstrap Go distribution (on my machine
it takes circa 15-30 minutes since the majority of time is spent on bootstrapping
stage 1). To do that, I build GCC with --enable-languages=c,c++,go
and pass LD_LIBRARY_PATH="<gcc prefix>/lib64" to go sources' make.bash.
I haven't tried bootstrapping using gccgo from cross-compiler, though.

So the bootstrapping flow would have been:

1. Check if Go version (either from Go distribution or from gccgo) is at least
   minimum required version (for now 1.17.13). If it is lower, bootstrap
   Go 1.17.3 (I call it bootstrap Go). Otherwise, go to 2.
2. Bootstrap Buildroot's Go package.

Thanks.

-- 
An old man doll... just what I always wanted! - Clara

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 1/3] package/go-bootstrap: split into two stages: go1.4 and go1.19.5

[Bagas Sanjaya]

On 3/25/23 20:34, Bagas Sanjaya wrote:
> 1. Check if Go version (either from Go distribution or from gccgo) is at least
>    minimum required version (for now 1.17.13). If it is lower, bootstrap
>    Go 1.17.3 (I call it bootstrap Go). Otherwise, go to 2.

Oops, I mean bootstrap Go 1.17.3 first then use it to go to 2. Otherwise, skip
this step.

-- 
An old man doll... just what I always wanted! - Clara

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 1/3] package/go-bootstrap: split into two stages: go1.4 and go1.19.5

[Christian Stewart via buildroot]

[-- Attachment #1.1: Type: text/plain, Size: 1673 bytes --]

Hello,

On Sat, Mar 25, 2023, 6:34 AM Bagas Sanjaya <bagasdotme@gmail.com> wrote:

> On 2/15/23 14:32, Christian Stewart via buildroot wrote:
> > Go 1.20 requires a minimum version of go 1.17.13 to bootstrap.
> >
> > https://go.dev/doc/go1.20#bootstrap
> >
> > As Go 1.4 was the previous version that could be compiled with C, there
> is now
> > no way to bootstrap go with a C compiler, unless we use a two-stage
> bootstrap:
> >
> >  - build host-go-bootstrap-1.4-20170531
> >  - build host-go-bootstrap-1.19.5 with host-go-bootstrap-1.4-20170531
> >  - build host-go-1.20 with host-go-bootstrap-1.19.5
> >
>
> I prefer using gccgo to bootstrap Go distribution (on my machine
> it takes circa 15-30 minutes since the majority of time is spent on
> bootstrapping
> stage 1). To do that, I build GCC with --enable-languages=c,c++,go
> and pass LD_LIBRARY_PATH="<gcc prefix>/lib64" to go sources' make.bash.
> I haven't tried bootstrapping using gccgo from cross-compiler, though.
>
> So the bootstrapping flow would have been


> 1. Check if Go version (either from Go distribution or from gccgo) is at
> least
>    minimum required version (for now 1.17.13). If it is lower, bootstrap
>    Go 1.17.3 (I call it bootstrap Go). Otherwise, go to 2.
> 2. Bootstrap Buildroot's Go package.
>


We had a long discussion about this previously, please check there. We
decided to instead use this multi step process for various reasons, or use
a preinstalled host compiler if the bootstrap process is not possible.

Check the original discussions if you'd like to see the reasoning:

https://lore.kernel.org/all/20220525234312.643dfc03@windsurf/T/

Best regards,
Christian Stewart

[-- Attachment #1.2: Type: text/html, Size: 2660 bytes --]

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot

Re: [Buildroot] [PATCH v3 1/3] package/go-bootstrap: split into two stages: go1.4 and go1.19.5

[Thomas Petazzoni via buildroot]

On Thu, 23 Mar 2023 19:45:04 -0700
Christian Stewart <christian@paral.in> wrote:

> > This doesn't make much sense. Why would the toolchain package be needed
> > for HOSTCC_NOCCACHE ? We have several packages that use HOSTCC_NOCCACHE
> > before the toolchain is ready.  
> 
> I suppose HOSTCC_NOCCACHE uses the gcc installed outside of buildroot,
> so it should be fine to remove this dependency.

Yes, the native toolchain must always be provided by the system, it is
not provided by Buildroot. The "toolchain" package only provides the
cross-compilation toolchain, it does nothing about the native toolchain.

> > > +     # Set all file timestamps to prevent the go compiler from rebuilding any
> > > +     # built in packages when programs are built.
> > > +     find $(HOST_GO_BOOTSTRAP_STAGE2_ROOT) -type f -exec touch -r $(@D)/bin/go {} \;  
> >
> > So we have to do this for bootstrap-stage2 but not bootstrap-stage1 ?  
> 
> It's not strictly necessary for either one. I removed it for the next
> revision of the patch.

But your comment above seemed to indicate that it was necessary.

> > > -     depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_ARCH_SUPPORTS
> > > +     depends on BR2_PACKAGE_HOST_GO_BOOTSTRAP_STAGE2_ARCH_SUPPORTS
> > > +     # See https://go.dev/doc/install/source#environment
> > > +     # See src/go/build/syslist.go for the list of supported architectures  
> >
> > This comment looks good, but is unrelated. Separate patch?  
> 
> Is it really necessary to put in a separate patch? It's a minor comment change.
> 
> Feels appropriate to bundle it here with other related changes.

Well, it always makes the review a bit more complicated, as we wonder
why it's there. If you really want to bundle unrelated changes like
this, they should at least be mentioned in the commit log so that the
reviewer knows what's going on.


> > > -# The go build system is not compatible with ccache, so use
> > > -# HOSTCC_NOCCACHE.  See https://github.com/golang/go/issues/11685.
> > > +# The go build system is not compatable with ccache, so use HOSTCC_NOCCACHE.
> > > +# See https://github.com/golang/go/issues/11685.  
> >
> > Why is this comment being changed, with a typo added?  
> 
> Fixed the typo. The comment is being changed because it can fit on a
> single line in 75 characters.
> 
> It just looks cleaner.
> 
> I can revert this if necessary.

Same as above: it's creating some unrelated "noise" in the patch, which
makes things odd when doing a review.

> 
> > >  HOST_GO_MAKE_ENV = \
> > >       GO111MODULE=off \
> > >       GOCACHE=$(HOST_GO_HOST_CACHE) \
> > > -     GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_ROOT) \
> > > +     GOROOT_BOOTSTRAP=$(HOST_GO_BOOTSTRAP_STAGE2_ROOT) \
> > >       GOROOT_FINAL=$(HOST_GO_ROOT) \
> > >       GOROOT="$(@D)" \
> > >       GOBIN="$(@D)/bin" \
> > > @@ -154,7 +154,6 @@ define HOST_GO_INSTALL_CMDS
> > >       cp -a $(@D)/pkg/include $(@D)/pkg/linux_* $(HOST_GO_ROOT)/pkg/
> > >       cp -a $(@D)/pkg/tool $(HOST_GO_ROOT)/pkg/
> > >
> > > -     # There is a known issue which requires the go sources to be installed  
> >
> > Why is this comment being removed?  
> 
> It's not an issue, per se. The nature of the Go compiler is that it
> needs the sources to work.
> That bug report has been closed for years now and is actually
> unrelated to needing sources anyway.

So perhaps rephrase to explain "Yes the Go compiler needs the sources
to work", rather than just dropping the comment? But here again, it's
unrelated to the patch we're discussing so => separate patch.

Best regards,

Thomas
-- 
Thomas Petazzoni, co-owner and CEO, Bootlin
Embedded Linux and Kernel engineering and training
https://bootlin.com
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot