#golang: go fetches dependencies in compile phase

From: "Juergen Landwehr" <juergen.landwehr@gmx.de>
To: yocto@lists.yoctoproject.org
Subject: #golang: go fetches dependencies in compile phase
Date: Mon, 12 Apr 2021 04:47:33 -0700	[thread overview]
Message-ID: <7847.1618228053017471875@lists.yoctoproject.org> (raw)
In-Reply-To: <8f3535e5-42c0-baf7-ae8b-fca906f33d05@gmail.com>

[-- Attachment #1: Type: text/plain, Size: 2198 bytes --]

Hi Robert,

thanks for your thoughts.

I see your point and the last thing I want is "NOT reproducable builds".

But dependency management in go is not that arbitrary as it may seem. Dependencies and their version is stored in "go.mod". To ensure reproducable builds, hashes for each dependency and version are stored in "go.sum". Both files are in git and together with a local golang proxy, this should ensure reproducable builds, right?

To ensure that licences are valid and did not change over time, we developed a little tool, that goes through all dependencies and creates the following output, which we insert into our recipe:

LICENSE = "Apache-2.0 & MIT & MIT & BSD-2-Clause & BSD-3-Clause & ...

LIC_FILES_CHKSUM = " \
file://${S}/src/${GO_IMPORT}/vendor/github.com/coreos/go-oidc/LICENSE;md5=d2794c0df5b907fdace235a619d80314 \
file://${S}/src/${GO_IMPORT}/vendor/github.com/go-playground/locales/LICENSE;md5=3ccbda375ee345400ad1da85ba522301 \
file://${S}/src/${GO_IMPORT}/vendor/github.com/go-playground/universal-translator/LICENSE;md5=2e2b21ef8f61057977d27c727c84bef1 \
file://${S}/src/${GO_IMPORT}/vendor/github.com/godbus/dbus/v5/LICENSE;md5=09042bd5c6c96a2b9e45ddf1bc517eed \
file://${S}/src/${GO_IMPORT}/vendor/github.com/golang/gddo/LICENSE;md5=4c728948788b1a02f33ae4e906546eef \
... This is a manual step and whenever dependencies change we have to create this list again and add it to the recipe, but it contains all the required license information and makes them visible in the recipe.

It might pollute the recipe a bit, but luckily we do not have thousands of dependencies like some npm projects. So I think it is still manageable. And is it much less work, than defining a recipe for each dependency.

So we would
* guarantee reproducable builds
* use the programming language (in our case golang) to handle dependency management
* ensure that licenses are visible and correct

I mean it is not perfect and still a compromise, but it should work without breaking reproducable builds or causing license issues?
What do you think?

Regards,
Jürgen

PS: Thanks for the link. I was not aware of this discussion (it is quite a bit to read:))

[-- Attachment #2: Type: text/html, Size: 2485 bytes --]