Re: [PATCH] libxl: avoid golang building without CONFIG_GOLANG=y

[George Dunlap <George.Dunlap@citrix.com>]

> On Aug 5, 2020, at 10:55 AM, Ian Jackson <ian.jackson@citrix.com> wrote:
> 
> Nick Rosbrook writes ("Re: [PATCH] libxl: avoid golang building without CONFIG_GOLANG=y"):
>> Jan - is the problem specifically that a fresh clone,  or `git
>> checkout`, etc. changes file timestamps in a way that triggers make to
>> rebuild those targets? I have not used the move-if-changed approach
>> before, but AFAICT that would be sufficient.
> 
> I don't think there is, from the point of view of the build system,
> anything different about gengotypes than about any other in-tree
> committed file which is updated using makefile rules based on only
> other in-tree files and common utilities (eg, in this case, Python).
> 
> I guess using move-if-changed will probably fix this.

That’s probably the quickest fix ATM.

> Jan: the reasons why this output file has to be committed are
> complicated.  We've discussed them at length.  Ultimately the reason
> is deliberate deficiencies[1] in golang.  Sadly this is the best of a
> not-very-good set of options.

I think we decided at the Summit to make a separate repo for the generated code, didn’t we?

To expand on this, Jan:

1. A *useful* go package must be able to be downloaded and built by the go build tools from the URL of a git repo

2. The go build process is restricted in what it will do automatically for security reasons.

For 4.14, we chose to check the generated code into xen.git, so that the main Xen repo could fulfill #1.  To make sure that the generated code was kept up to date with changes in libxl_types.idl, we decided to re-generate the code even for systems which don’t have golang installed.

But this is another example of the annoying side effects of this approach.

The other approach, which I thought we’d agreed upon at the summit, is to have a separate repo with a more friendly URL which is programmatically generated on a regular basis.  That would obviate the need to run the generator, except to verify that the generated code still compiled (which wouldn’t be possible without having golang installed anyway).

> [1] This is an extreme phrase, but justified I think.  The golang
> designers have deliberately aimed at what they regard as "simplicity"
> and one of the things they have "simplified" away (in their language
> and in their package management and build tools) is the ability to
> conveniently generate golang code at build time.  Committing the
> generated code is the norm in the golang community.

It’s a bit less unreasonable than this. :-)  There actually is infrastructure for generating files — `go generate`.  It’s just not allowed to run as part of a build.

One of the core things they wanted to be able to do was to download and build dependencies, recursively and automatically, from arbitrary URLs, without the need for any curation.  For safety concerns, they don’t want any such dependencies to be able to run arbitrary commands as part of their build: Go builds can basically call compilers and linkers and that’s it.

Not sure I actually buy that this gives you a whole lot of safety, since there’s not a not of point in compiling something you’re not going to run; and it’s not clear to me that it’s terribly more risky to run code you haven’t vetted than to build code you haven’t vetted.  But at least it’s not obviously wrong.

 -George