[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[aduskett at gmail.com]

From: Adam Duskett <Aduskett@gmail.com>

Update the following license file hashes due to copyright year changes:
  - vendor/github.com/mendersoftware/mendertesting/LICENSE
  - vendor/github.com/davecgh/go-spew/LICENSE
  - vendor/github.com/stretchr/testify/LICENSE

Add hashes for the the following new files:
  - vendor/github.com/minio/sha256-simd/LICENSE
  - vendor/gopkg.in/yaml.v2/LICENSE
  - vendor/github.com/klauspost/compress/LICENSE
  - vendor/github.com/russross/blackfriday/v2/LICENSE.txt
  - vendor/github.com/klauspost/pgzip/LICENSE
  - vendor/github.com/cpuguy83/go-md2man/v2/LICENSE.md
  - vendor/github.com/shurcooL/sanitized_anchor_name/LICENSE

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
---
 package/mender-artifact/mender-artifact.hash | 17 ++++++++++++-----
 package/mender-artifact/mender-artifact.mk   | 11 +++++++++--
 2 files changed, 21 insertions(+), 7 deletions(-)

diff --git a/package/mender-artifact/mender-artifact.hash b/package/mender-artifact/mender-artifact.hash
index cd178483d1..09e66617bb 100644
--- a/package/mender-artifact/mender-artifact.hash
+++ b/package/mender-artifact/mender-artifact.hash
@@ -1,6 +1,6 @@
 # Locally computed:
-sha256 6b9fe6f427c43ac92e1fc88dc8c6cac323eaf9ea5854078a06c4313e090e83f8 3.3.0.tar.gz
-sha256 be22e200d0826c94bfd71cc5c1d4995449a6e3bd527090c1cfe953bc81427573 LIC_FILES_CHKSUM.sha256
+sha256  c085479a6e2ae598687a15b646bfaaa68e5d76d08382bdcc4a27a9b4021d6540  3.4.0.tar.gz
+sha256  08b0209abb7e87bb1d21aca8a40693b7d29406d1edffd443f17c59d5d68f6467  LIC_FILES_CHKSUM.sha256
 
 # License hash extracted from LIC_FILES_CHKSUM.sha256 using the
 # following command:
@@ -9,7 +9,9 @@ sha256 be22e200d0826c94bfd71cc5c1d4995449a6e3bd527090c1cfe953bc81427573 LIC_FILE
 
 # Apache 2.0 licenses.
 sha256  32714818ad6f98ee0185a52e23a475d89122e3efd2b2c26c733781c28e798c99  LICENSE
-sha256  beb140be4cd64599bedc691a55b2729c9cc611a4b9d6ec44e01270105daf18a2  vendor/github.com/mendersoftware/mendertesting/LICENSE
+sha256  32714818ad6f98ee0185a52e23a475d89122e3efd2b2c26c733781c28e798c99  vendor/github.com/mendersoftware/mendertesting/LICENSE
+sha256  cfc7749b96f63bd31c3c42b5c471bf756814053e847c10f3eb003417bc523d30  vendor/github.com/minio/sha256-simd/LICENSE
+sha256  b40930bbcf80744c86c46a12bc9da056641d722716c378f5659b9e555ef833e1  vendor/gopkg.in/yaml.v2/LICENSE
 #
 # BSD 2 Clause licenses.
 sha256  8d427fd87bc9579ea368fde3d49f9ca22eac857f91a9dec7e3004bdfab7dee86  vendor/github.com/pkg/errors/LICENSE
@@ -19,11 +21,16 @@ sha256  2eb550be6801c1ea434feba53bf6d12e7c71c90253e0a9de4a4f46cf88b56477  vendor
 sha256  2d36597f7117c38b006835ae7f537487207d8ec407aa9d9980794b2030cbc067  vendor/golang.org/x/sys/LICENSE
 sha256  2d36597f7117c38b006835ae7f537487207d8ec407aa9d9980794b2030cbc067  vendor/golang.org/x/crypto/LICENSE
 sha256  a284e538f3e5649ae6a8f32da2bbd18e9daabf2c1ebbfb99ed92e797b159be1a  vendor/github.com/remyoudompheng/go-liblzma/LICENSE
+sha256  16f848582e4b276a7392cd34496b7a33d6f65c0e190c163ff3a056a7c61219ce  vendor/github.com/klauspost/compress/LICENSE
+sha256  75e1ca97a84a9da6051dee0114333388216f2c4a5a028296b882ff3d57274735  vendor/github.com/russross/blackfriday/v2/LICENSE.txt
 #
 # ISC licenses.
-sha256  3525392c6db3b804af76980b2c560ee9ec1abdadd907d76a26091df7c78f3a25  vendor/github.com/davecgh/go-spew/LICENSE
+sha256  1b93a317849ee09d3d7e4f1d20c2b78ddb230b4becb12d7c224c927b9d470251  vendor/github.com/davecgh/go-spew/LICENSE
 #
 # MIT licenses.
-sha256  402f39eed8a1851385d0703999aa9f23d067c2ea3e15c63c074e389cbf8f8f8f  vendor/github.com/stretchr/testify/LICENSE
+sha256  dad2b0b2cc2dbdbf95ad5d800ef7588956e74dc2479014829d42be295125c25d  vendor/github.com/stretchr/testify/LICENSE
 sha256  da277af11b85227490377fbcac6afccc68be560c4fff36ac05ca62de55345fd7  vendor/github.com/urfave/cli/LICENSE
 sha256  51a0c9ec7f8b7634181b8d4c03e5b5d204ac21d6e72f46c313973424664b2e6b  vendor/github.com/sirupsen/logrus/LICENSE
+sha256  7709cc030f078b17809884f92f33a2016944e1180312dc3f1371b02313d313ed  vendor/github.com/klauspost/pgzip/LICENSE
+sha256  a55959c4e3e8917bfa857359bb641115336276a6cc97408fd8197e079fb18470  vendor/github.com/cpuguy83/go-md2man/v2/LICENSE.md
+sha256  c8024e31c1de453fea90f22a221968835cc7af9d520274a2576c9ec9976055b0  vendor/github.com/shurcooL/sanitized_anchor_name/LICENSE
diff --git a/package/mender-artifact/mender-artifact.mk b/package/mender-artifact/mender-artifact.mk
index 14cbe53d4c..c81ec9ba53 100644
--- a/package/mender-artifact/mender-artifact.mk
+++ b/package/mender-artifact/mender-artifact.mk
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-HOST_MENDER_ARTIFACT_VERSION = 3.3.0
+HOST_MENDER_ARTIFACT_VERSION = 3.4.0
 HOST_MENDER_ARTIFACT_SITE = https://github.com/mendersoftware/mender-artifact/archive
 HOST_MENDER_ARTIFACT_SOURCE = $(HOST_MENDER_ARTIFACT_VERSION).tar.gz
 HOST_MENDER_ARTIFACT_LICENSE = Apache2.0, BSD-2-Clause, BSD-3-Clause, ISC, MIT
@@ -12,15 +12,22 @@ HOST_MENDER_ARTIFACT_LICENSE_FILES = \
 	LICENSE \
 	LIC_FILES_CHKSUM.sha256 \
 	vendor/github.com/mendersoftware/mendertesting/LICENSE \
+	vendor/github.com/minio/sha256-simd/LICENSE \
+	vendor/gopkg.in/yaml.v2/LICENSE \
 	vendor/github.com/pkg/errors/LICENSE \
 	vendor/github.com/pmezard/go-difflib/LICENSE \
 	vendor/golang.org/x/sys/LICENSE \
 	vendor/golang.org/x/crypto/LICENSE \
 	vendor/github.com/remyoudompheng/go-liblzma/LICENSE \
+	vendor/github.com/klauspost/compress/LICENSE \
+	vendor/github.com/russross/blackfriday/v2/LICENSE.txt \
 	vendor/github.com/davecgh/go-spew/LICENSE \
 	vendor/github.com/stretchr/testify/LICENSE \
 	vendor/github.com/urfave/cli/LICENSE \
-	vendor/github.com/sirupsen/logrus/LICENSE
+	vendor/github.com/sirupsen/logrus/LICENSE \
+	vendor/github.com/klauspost/pgzip/LICENSE \
+	vendor/github.com/cpuguy83/go-md2man/v2/LICENSE.md \
+	vendor/github.com/shurcooL/sanitized_anchor_name/LICENSE
 
 HOST_MENDER_ARTIFACT_DEPENDENCIES = host-xz
 
-- 
2.26.2

[Buildroot] [PATCH 2/4] package/mender: bump version to 2.3.0

[aduskett at gmail.com]

From: Adam Duskett <Aduskett@gmail.com>

Update the following license file hashes due to copyright year changes:
  - vendor/github.com/mendersoftware/mendertesting/LICENSE
  - vendor/github.com/mendersoftware/mender-artifact/LICENSE

Remove hashes for the the following removed files:
  - vendor/github.com/mendersoftware/log/LICENSE
  - vendor/github.com/mendersoftware/scopestack/LICENSE

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
---
 package/mender/mender.hash | 10 ++++------
 package/mender/mender.mk   |  4 +---
 2 files changed, 5 insertions(+), 9 deletions(-)

diff --git a/package/mender/mender.hash b/package/mender/mender.hash
index 6a258cf206..f1eef656fe 100644
--- a/package/mender/mender.hash
+++ b/package/mender/mender.hash
@@ -1,15 +1,13 @@
 # Locally computed:
-sha256 fe3567107d1dc9b3f3961f3a19d426bd7728fbc1525b72b8204765670234f4b6 2.2.0.tar.gz
+sha256  d7c885a34f7b7f9969970132411225895c136e22be0caaa6dd3e08ca4bb71ff9  2.3.0.tar.gz
 
 # Vendor licenses
 # Generated with sed '/^[A-Za-z0-9_]/s/^/sha256  /' LIC_FILES_CHKSUM.sha256
 
 # Apache-2.0 license.
 sha256  32714818ad6f98ee0185a52e23a475d89122e3efd2b2c26c733781c28e798c99  LICENSE
-sha256  beb140be4cd64599bedc691a55b2729c9cc611a4b9d6ec44e01270105daf18a2  vendor/github.com/mendersoftware/mendertesting/LICENSE
-sha256  ceb1b36ff073bd13d9806d4615b931707768ca9023805620acc32dd1cfc2f680  vendor/github.com/mendersoftware/log/LICENSE
-sha256  ceb1b36ff073bd13d9806d4615b931707768ca9023805620acc32dd1cfc2f680  vendor/github.com/mendersoftware/scopestack/LICENSE
-sha256  beb140be4cd64599bedc691a55b2729c9cc611a4b9d6ec44e01270105daf18a2  vendor/github.com/mendersoftware/mender-artifact/LICENSE
+sha256  32714818ad6f98ee0185a52e23a475d89122e3efd2b2c26c733781c28e798c99  vendor/github.com/mendersoftware/mendertesting/LICENSE
+sha256  32714818ad6f98ee0185a52e23a475d89122e3efd2b2c26c733781c28e798c99  vendor/github.com/mendersoftware/mender-artifact/LICENSE
 #
 # BSD 2 Clause license.
 sha256  8d427fd87bc9579ea368fde3d49f9ca22eac857f91a9dec7e3004bdfab7dee86  vendor/github.com/pkg/errors/LICENSE
@@ -38,4 +36,4 @@ sha256  da277af11b85227490377fbcac6afccc68be560c4fff36ac05ca62de55345fd7  vendor
 sha256  310fe25c858a9515fc8c8d7d1f24a67c9496f84a91e0a0e41ea9975b1371e569  vendor/github.com/bmatsuo/lmdb-go/LICENSE.mdb.md
 
 # sha256 of all the vendor licenses combined
-sha256 8edc8cda9cc06270f44d2d75f7810a6ccc3d71c251cadb5e1a3065eb057a3132 LIC_FILES_CHKSUM.sha256
+sha256  39f8e574fa6d082b650a5d6159045cdec7c4662da6ccbd67271079904353b8d5  LIC_FILES_CHKSUM.sha256
diff --git a/package/mender/mender.mk b/package/mender/mender.mk
index 6543402f07..cd840b1e3f 100644
--- a/package/mender/mender.mk
+++ b/package/mender/mender.mk
@@ -4,7 +4,7 @@
 #
 ################################################################################
 
-MENDER_VERSION = 2.2.0
+MENDER_VERSION = 2.3.0
 MENDER_SITE = https://github.com/mendersoftware/mender/archive
 MENDER_SOURCE = $(MENDER_VERSION).tar.gz
 MENDER_LICENSE = Apache-2.0, BSD-2-Clause, BSD-3-Clause, ISC, MIT, OLDAP-2.8
@@ -15,8 +15,6 @@ MENDER_LICENSE_FILES = \
 	LICENSE \
 	LIC_FILES_CHKSUM.sha256 \
 	vendor/github.com/mendersoftware/mendertesting/LICENSE \
-	vendor/github.com/mendersoftware/log/LICENSE \
-	vendor/github.com/mendersoftware/scopestack/LICENSE \
 	vendor/github.com/mendersoftware/mender-artifact/LICENSE \
 	vendor/github.com/pkg/errors/LICENSE \
 	vendor/github.com/pmezard/go-difflib/LICENSE \
-- 
2.26.2

[Buildroot] [PATCH 3/4] package/mender: set the default script version to 3

[aduskett at gmail.com]

From: Adam Duskett <Aduskett@gmail.com>

Version 3 scripts have been supported since version 2.0 and should be the
default.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
---
 package/mender/mender.mk | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package/mender/mender.mk b/package/mender/mender.mk
index cd840b1e3f..2faf3fe980 100644
--- a/package/mender/mender.mk
+++ b/package/mender/mender.mk
@@ -39,7 +39,7 @@ MENDER_LDFLAGS = -X main.Version=$(MENDER_VERSION)
 
 define MENDER_INSTALL_CONFIG_FILES
 	$(INSTALL) -d -m 755 $(TARGET_DIR)/etc/mender/scripts
-	echo -n "2" > $(TARGET_DIR)/etc/mender/scripts/version
+	echo -n "3" > $(TARGET_DIR)/etc/mender/scripts/version
 
 	$(INSTALL) -D -m 0644 $(MENDER_PKGDIR)/mender.conf \
 		$(TARGET_DIR)/etc/mender/mender.conf
-- 
2.26.2

[Buildroot] [PATCH 4/4] package/mender: add support for update modules

[aduskett at gmail.com]

From: Adam Duskett <Aduskett@gmail.com>

Mender has support for performing other types of updates other than just
overwriting the rootfs partition that isn't currently in use on the target.

Some of the default provided modules provide support for:
  - Updating a docker container.
  - Running a script.
  - Installing an RPM.
  - Overwriting a directory.
  - Updating a single-file.

The single-file update module is used by upstream for onboarding a new device
to a server, and this fails with Buildroot devices because the modules are
currently not installed.

Install the directory, script, and single-file modules by default, and
install the docker or rpm script if their respective packages are selected.

Signed-off-by: Adam Duskett <Aduskett@gmail.com>
---
 package/mender/mender.mk | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/package/mender/mender.mk b/package/mender/mender.mk
index 2faf3fe980..7d3fe877ce 100644
--- a/package/mender/mender.mk
+++ b/package/mender/mender.mk
@@ -37,6 +37,9 @@ MENDER_DEPENDENCIES = xz
 
 MENDER_LDFLAGS = -X main.Version=$(MENDER_VERSION)
 
+MENDER_UPDATE_MODULES_FILES = directory script single-file
+MENDER_UPDATE_MODULES_FILES += $(if $(BR2_PACKAGE_DOCKER_CLI),docker)
+MENDER_UPDATE_MODULES_FILES +=  $(if $(BR2_PACKAGE_RPM),rpm)
 define MENDER_INSTALL_CONFIG_FILES
 	$(INSTALL) -d -m 755 $(TARGET_DIR)/etc/mender/scripts
 	echo -n "3" > $(TARGET_DIR)/etc/mender/scripts/version
@@ -61,6 +64,10 @@ define MENDER_INSTALL_CONFIG_FILES
 
 	mkdir -p $(TARGET_DIR)/var/lib
 	ln -snf /var/run/mender $(TARGET_DIR)/var/lib/mender
+	$(foreach f,$(MENDER_UPDATE_MODULES_FILES), \
+		$(INSTALL) -D -m 0755 $(@D)/support/modules/$(notdir $(f)) \
+			$(TARGET_DIR)/usr/share/mender/modules/v3/$(notdir $(f))
+	)
 endef
 
 MENDER_POST_INSTALL_TARGET_HOOKS += MENDER_INSTALL_CONFIG_FILES
-- 
2.26.2

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Thomas Petazzoni]

On Wed, 15 Jul 2020 10:37:04 -0700
aduskett at gmail.com wrote:

> From: Adam Duskett <Aduskett@gmail.com>
> 
> Update the following license file hashes due to copyright year changes:
>   - vendor/github.com/mendersoftware/mendertesting/LICENSE
>   - vendor/github.com/davecgh/go-spew/LICENSE
>   - vendor/github.com/stretchr/testify/LICENSE
> 
> Add hashes for the the following new files:
>   - vendor/github.com/minio/sha256-simd/LICENSE
>   - vendor/gopkg.in/yaml.v2/LICENSE
>   - vendor/github.com/klauspost/compress/LICENSE
>   - vendor/github.com/russross/blackfriday/v2/LICENSE.txt
>   - vendor/github.com/klauspost/pgzip/LICENSE
>   - vendor/github.com/cpuguy83/go-md2man/v2/LICENSE.md
>   - vendor/github.com/shurcooL/sanitized_anchor_name/LICENSE
> 
> Signed-off-by: Adam Duskett <Aduskett@gmail.com>
> ---
>  package/mender-artifact/mender-artifact.hash | 17 ++++++++++++-----
>  package/mender-artifact/mender-artifact.mk   | 11 +++++++++--
>  2 files changed, 21 insertions(+), 7 deletions(-)

I've applied the entire series to master. I did some minor formatting
tweaks on the last patch, nothing crazy. Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Thomas Petazzoni]

On Wed, 15 Jul 2020 10:37:04 -0700
aduskett at gmail.com wrote:

> From: Adam Duskett <Aduskett@gmail.com>
> 
> Update the following license file hashes due to copyright year changes:
>   - vendor/github.com/mendersoftware/mendertesting/LICENSE
>   - vendor/github.com/davecgh/go-spew/LICENSE
>   - vendor/github.com/stretchr/testify/LICENSE
> 
> Add hashes for the the following new files:
>   - vendor/github.com/minio/sha256-simd/LICENSE
>   - vendor/gopkg.in/yaml.v2/LICENSE
>   - vendor/github.com/klauspost/compress/LICENSE
>   - vendor/github.com/russross/blackfriday/v2/LICENSE.txt
>   - vendor/github.com/klauspost/pgzip/LICENSE
>   - vendor/github.com/cpuguy83/go-md2man/v2/LICENSE.md
>   - vendor/github.com/shurcooL/sanitized_anchor_name/LICENSE
> 
> Signed-off-by: Adam Duskett <Aduskett@gmail.com>

I don't know if this is exactly related to this version bump, but after
this, I'm seeing my autobuilder fails to remove the output directory,
with lots of:

rm: cannot remove '/home/test/autobuild/run/instance-2/output-1/build/host-mender-artifact-3.4.0/_gopath/pkg/mod/github.com/klauspost/compress at v1.10.5/zip/testdata/utf8-osx.zip': Permission denied

This is due to the fact that the folders created in
/home/test/autobuild/run/instance-2/output-1/build/host-mender-artifact-3.4.0/_gopath/pkg/mod/github.com/klauspost/
don't have write permission:

test at build:~/autobuild/run/instance-2/output-1/build/host-mender-artifact-3.4.0/_gopath/pkg/mod/github.com/klauspost$ ls -l
total 8
dr-x------ 12 test test 4096 Jul 17 05:15 compress at v1.10.5
dr-x------  3 test test 4096 Jul 17 05:15 pgzip at v1.2.3

is this expected ? Is there anything we can do to have proper
permissions on this ?

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Mirza Krak]

On Fri, Jul 17, 2020 at 2:51 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> On Wed, 15 Jul 2020 10:37:04 -0700
> aduskett at gmail.com wrote:
>
> > From: Adam Duskett <Aduskett@gmail.com>
> >
> > Update the following license file hashes due to copyright year changes:
> >   - vendor/github.com/mendersoftware/mendertesting/LICENSE
> >   - vendor/github.com/davecgh/go-spew/LICENSE
> >   - vendor/github.com/stretchr/testify/LICENSE
> >
> > Add hashes for the the following new files:
> >   - vendor/github.com/minio/sha256-simd/LICENSE
> >   - vendor/gopkg.in/yaml.v2/LICENSE
> >   - vendor/github.com/klauspost/compress/LICENSE
> >   - vendor/github.com/russross/blackfriday/v2/LICENSE.txt
> >   - vendor/github.com/klauspost/pgzip/LICENSE
> >   - vendor/github.com/cpuguy83/go-md2man/v2/LICENSE.md
> >   - vendor/github.com/shurcooL/sanitized_anchor_name/LICENSE
> >
> > Signed-off-by: Adam Duskett <Aduskett@gmail.com>
>
> I don't know if this is exactly related to this version bump, but after
> this, I'm seeing my autobuilder fails to remove the output directory,
> with lots of:
>
> rm: cannot remove '/home/test/autobuild/run/instance-2/output-1/build/host-mender-artifact-3.4.0/_gopath/pkg/mod/github.com/klauspost/compress at v1.10.5/zip/testdata/utf8-osx.zip': Permission denied

Not sure if anyone else has followed up on this, but could not find a
response so I will try.

This is indeed intentional to the extent that this is how GO works
when you are using "go mod" (which mender-artifact now does). We are
not first ones having these problems and there are several tickets
discussing this, e.g [1]. This should be applicable to any GO package
in Buildroot that uses "go mod"

There seems to be two ways to get around this,

1. Use 'go clean -modcache' command which should remove these files
2. Use "-modcacherw" flag of "go mod" which will make these files R/W
(this is only available after GO 1.14.x [2])

I am not sure how to best incorporate this into Buildroot.

[1]. https://github.com/golang/go/issues/27161
[2]. https://github.com/golang/go/commit/8ba43eaad8263f26e16846e345389146ec234364

-- 
Mirza Krak | Embedded Solutions Architect | https://mender.io

 Northern.tech AS | @northerntechHQ

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Mirza Krak]

On Thu, Aug 6, 2020 at 12:57 PM Mirza Krak <mirza.krak@northern.tech> wrote:
>
> On Fri, Jul 17, 2020 at 2:51 PM Thomas Petazzoni
> <thomas.petazzoni@bootlin.com> wrote:
> >
> > On Wed, 15 Jul 2020 10:37:04 -0700
> > aduskett at gmail.com wrote:
> >
> > > From: Adam Duskett <Aduskett@gmail.com>
> > >
> > > Update the following license file hashes due to copyright year changes:
> > >   - vendor/github.com/mendersoftware/mendertesting/LICENSE
> > >   - vendor/github.com/davecgh/go-spew/LICENSE
> > >   - vendor/github.com/stretchr/testify/LICENSE
> > >
> > > Add hashes for the the following new files:
> > >   - vendor/github.com/minio/sha256-simd/LICENSE
> > >   - vendor/gopkg.in/yaml.v2/LICENSE
> > >   - vendor/github.com/klauspost/compress/LICENSE
> > >   - vendor/github.com/russross/blackfriday/v2/LICENSE.txt
> > >   - vendor/github.com/klauspost/pgzip/LICENSE
> > >   - vendor/github.com/cpuguy83/go-md2man/v2/LICENSE.md
> > >   - vendor/github.com/shurcooL/sanitized_anchor_name/LICENSE
> > >
> > > Signed-off-by: Adam Duskett <Aduskett@gmail.com>
> >
> > I don't know if this is exactly related to this version bump, but after
> > this, I'm seeing my autobuilder fails to remove the output directory,
> > with lots of:
> >
> > rm: cannot remove '/home/test/autobuild/run/instance-2/output-1/build/host-mender-artifact-3.4.0/_gopath/pkg/mod/github.com/klauspost/compress at v1.10.5/zip/testdata/utf8-osx.zip': Permission denied
>
> Not sure if anyone else has followed up on this, but could not find a
> response so I will try.
>
> This is indeed intentional to the extent that this is how GO works
> when you are using "go mod" (which mender-artifact now does). We are
> not first ones having these problems and there are several tickets
> discussing this, e.g [1]. This should be applicable to any GO package
> in Buildroot that uses "go mod"
>
> There seems to be two ways to get around this,
>
> 1. Use 'go clean -modcache' command which should remove these files
> 2. Use "-modcacherw" flag of "go mod" which will make these files R/W
> (this is only available after GO 1.14.x [2])

Actually there is a third option, this works as well,

    $ git diff
    diff --git a/package/mender-artifact/mender-artifact.mk
b/package/mender-artifact/mender-artifact.mk
    index c81ec9ba53..884fcd6eec 100644
    --- a/package/mender-artifact/mender-artifact.mk
    +++ b/package/mender-artifact/mender-artifact.mk
    @@ -33,6 +33,8 @@ HOST_MENDER_ARTIFACT_DEPENDENCIES = host-xz

     HOST_MENDER_ARTIFACT_LDFLAGS = -X
main.Version=$(HOST_MENDER_ARTIFACT_VERSION)

    +HOST_MENDER_ARTIFACT_GO_ENV = GOFLAGS="-mod=vendor"
    +
     HOST_MENDER_ARTIFACT_BUILD_TARGETS = cli/mender-artifact

     HOST_MENDER_ARTIFACT_BIN_NAME = mender-artifact

-- 
Mirza Krak | Embedded Solutions Architect | https://mender.io

 Northern.tech AS | @northerntechHQ

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Thomas Petazzoni]

Hello,

+Matt since his autobuilders are causing issues.
+Christian for some more Go expertise.

On Wed, 15 Jul 2020 10:37:04 -0700
aduskett at gmail.com wrote:

> From: Adam Duskett <Aduskett@gmail.com>
> 
> Update the following license file hashes due to copyright year changes:
>   - vendor/github.com/mendersoftware/mendertesting/LICENSE
>   - vendor/github.com/davecgh/go-spew/LICENSE
>   - vendor/github.com/stretchr/testify/LICENSE
> 
> Add hashes for the the following new files:
>   - vendor/github.com/minio/sha256-simd/LICENSE
>   - vendor/gopkg.in/yaml.v2/LICENSE
>   - vendor/github.com/klauspost/compress/LICENSE
>   - vendor/github.com/russross/blackfriday/v2/LICENSE.txt
>   - vendor/github.com/klauspost/pgzip/LICENSE
>   - vendor/github.com/cpuguy83/go-md2man/v2/LICENSE.md
>   - vendor/github.com/shurcooL/sanitized_anchor_name/LICENSE
> 
> Signed-off-by: Adam Duskett <Aduskett@gmail.com>

This is causing some significant amount of build failures in the
autobuilders:

  http://autobuild.buildroot.net/?reason=host-mender%

It fails only on Matthew Weber autobuilders, which don't have a
"normal" access to Internet, and the build fails with errors such as:

go: github.com/davecgh/go-spew at v1.1.1: Get https://proxy.golang.org/github.com/davecgh/go-spew/@v/v1.1.1.mod: proxyconnect tcp: read tcp 192.168.6.146:34964->128.24.38.25:9091: read: connection reset by peer

go: github.com/davecgh/go-spew at v1.1.1: Get https://proxy.golang.org/github.com/davecgh/go-spew/@v/v1.1.1.mod: proxyconnect tcp: net/http: TLS handshake timeout

Can this be fixed somehow ?

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Christian Stewart]

Hi Thomas,

On Tue, Aug 11, 2020 at 3:02 PM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
> +Matt since his autobuilders are causing issues.
> +Christian for some more Go expertise.

[snip]

> This is causing some significant amount of build failures in the
> autobuilders:
>
>   http://autobuild.buildroot.net/?reason=host-mender%
>
> It fails only on Matthew Weber autobuilders, which don't have a
> "normal" access to Internet, and the build fails with errors such as:
>
> go: github.com/davecgh/go-spew at v1.1.1: Get https://proxy.golang.org/github.com/davecgh/go-spew/@v/v1.1.1.mod: proxyconnect tcp: read tcp 192.168.6.146:34964->128.24.38.25:9091: read: connection reset by peer
>
> go: github.com/davecgh/go-spew at v1.1.1: Get https://proxy.golang.org/github.com/davecgh/go-spew/@v/v1.1.1.mod: proxyconnect tcp: net/http: TLS handshake timeout
>
> Can this be fixed somehow ?

Yes. Please merge this patch series:
https://patchwork.ozlabs.org/project/buildroot/list/?series=161729

Thanks,
Christian

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Thomas Petazzoni]

On Tue, 11 Aug 2020 19:26:42 -0700
Christian Stewart <christian@paral.in> wrote:

> > Can this be fixed somehow ?  
> 
> Yes. Please merge this patch series:
> https://patchwork.ozlabs.org/project/buildroot/list/?series=161729

Hm, ok. The issue is that this series is quite complex, too complex to
merge in master right now as we have already cut -rc1.

Do we have an intermediate solution for 2020.08 ? Would reverting the
mender-artifact bump resolve the issue ?

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Adam Duskett]

Hello;

I submitted a patch based on Mirza's suggestion above which works as expected:
https://patchwork.ozlabs.org/project/buildroot/patch/20200813184947.4042927-1-aduskett at gmail.com/


On Wed, Aug 12, 2020 at 1:22 AM Thomas Petazzoni
<thomas.petazzoni@bootlin.com> wrote:
>
> On Tue, 11 Aug 2020 19:26:42 -0700
> Christian Stewart <christian@paral.in> wrote:
>
> > > Can this be fixed somehow ?
> >
> > Yes. Please merge this patch series:
> > https://patchwork.ozlabs.org/project/buildroot/list/?series=161729
>
> Hm, ok. The issue is that this series is quite complex, too complex to
> merge in master right now as we have already cut -rc1.
>
> Do we have an intermediate solution for 2020.08 ? Would reverting the
> mender-artifact bump resolve the issue ?
>
> Thanks,
>
> Thomas
> --
> Thomas Petazzoni, CTO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Christian Stewart]

Hi,

Don't merge. This will download stuff out-of-band with the main
Buildroot download mechanism.

On Thu, Aug 13, 2020 at 11:52 AM Adam Duskett <aduskett@gmail.com> wrote:
> I submitted a patch based on Mirza's suggestion above which works as expected:
> https://patchwork.ozlabs.org/project/buildroot/patch/20200813184947.4042927-1-aduskett at gmail.com/

> > Do we have an intermediate solution for 2020.08 ? Would reverting the
> > mender-artifact bump resolve the issue ?

Please just roll it back and then consider the full Go modules patch later on.

I went about submitting that series in a particular way to avoid
breaking the Buildroot download system.

Best regards,
Christian Stewart

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Adam Duskett]

Christian;

Just as a note, the go-modules series breaks mender-artifact, and the
.mk file needs to be modified
to add a "./" before cli/mender-artifact.
IE:
HOST_MENDER_ARTIFACT_BUILD_TARGETS = ./cli/mender-artifact

Adam

On Thu, Aug 13, 2020 at 12:39 PM Christian Stewart <christian@paral.in> wrote:
>
> Hi,
>
> Don't merge. This will download stuff out-of-band with the main
> Buildroot download mechanism.
>
> On Thu, Aug 13, 2020 at 11:52 AM Adam Duskett <aduskett@gmail.com> wrote:
> > I submitted a patch based on Mirza's suggestion above which works as expected:
> > https://patchwork.ozlabs.org/project/buildroot/patch/20200813184947.4042927-1-aduskett at gmail.com/
>
> > > Do we have an intermediate solution for 2020.08 ? Would reverting the
> > > mender-artifact bump resolve the issue ?
>
> Please just roll it back and then consider the full Go modules patch later on.
>
> I went about submitting that series in a particular way to avoid
> breaking the Buildroot download system.
>
> Best regards,
> Christian Stewart

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Christian Stewart]

Hi Adam,

On Thu, Aug 13, 2020 at 1:04 PM Adam Duskett <aduskett@gmail.com> wrote:
> Just as a note, the go-modules series breaks mender-artifact, and the
> .mk file needs to be modified
> to add a "./" before cli/mender-artifact.
> IE:
> HOST_MENDER_ARTIFACT_BUILD_TARGETS = ./cli/mender-artifact

The series needs a few adjustments for host-go packages... I'm doing a
small cleanup + testing pass and will submit a new revision today,
including a patch adjusting the host-mender package build target.

Note:

I recommend changing the SITE to

HOST_MENDER_ARTIFACT_SITE = $(call
github,mendersoftware,mender-artifact,$(HOST_MENDER_ARTIFACT_VERSION))

corresponding hash:

sha256  c085479a6e2ae598687a15b646bfaaa68e5d76d08382bdcc4a27a9b4021d6540
 mender-artifact-3.4.0.tar.gz

(vs 3.4.0.tar.gz filename before).

After the new series revision it will be:

HOST_MENDER_ARTIFACT_BUILD_TARGETS =
github.com/mendersoftware/mender-artifact/cli/mender-artifact

Best regards,
Christian

[Buildroot] [PATCH 1/4] package/mender-artifact: bump version to 3.4.0

[Christian Stewart]

Thomas, Adam,

On Thu, Aug 13, 2020 at 1:04 PM Adam Duskett <aduskett@gmail.com> wrote:
> Just as a note, the go-modules series breaks mender-artifact, and the
> .mk file needs to be modified

> > On Thu, Aug 13, 2020 at 11:52 AM Adam Duskett <aduskett@gmail.com> wrote:
> > > I submitted a patch based on Mirza's suggestion above which works as expected:
> > > https://patchwork.ozlabs.org/project/buildroot/patch/20200813184947.4042927-1-aduskett at gmail.com/
> >
> > > > Do we have an intermediate solution for 2020.08 ? Would reverting the
> > > > mender-artifact bump resolve the issue ?

I have submitted a v4 of the "GO111MODULE" patch series which fixes this issue.

While looking at this and testing - this is the first host-go package
I've tested in mainline - I discovered that a number of the Go build
variables for host packages are not set correctly.

The v4 of the series correctly builds all the HOST and TARGET packages
in the mainline tree that we have today.

The series is not as massive of a change as it may seem. It's actually
just a simplification, rather than symlinking a fake GOPATH we instead
just allow the Go compiler to pick up the root import path from the
go.mod file. As I've pointed out in the past, the Go developers have
requested everyone switch to the new GO111MODULE system instead of
GOPATH.

FWIW I've been using this + testing it constantly for the last months.

Best regards,
Christian Stewart