[stage1-xen PATCH v1] init: Add `glide.lock`

[stage1-xen PATCH v1] init: Add `glide.lock`

[Rajiv Ranganath]

Hi Stefano,

`build.sh` does `glide init` followed by `glide up -v` in order to pull
in dependencies for `init.go`.

I ran into two issues -

1. `glide init` automatically starts in interactive mode, which pauses
`build.sh`. This could be fixed with `glide init --non-interactive`.

2. `glide up -v` 

`glide update` dependency resolution breaks with the following error -

```
[INFO]  --> Fetching updates for github.com/vishvananda/netns.
[ERROR] Error scanning github.com/containernetworking/cni/pkg/ip: open /root/.glide/cache/src/https-github.com-containernetworking-cni/pkg/ip: no such file or directory
[ERROR] This error means the referenced package was not found.
[ERROR] Missing file or directory errors usually occur when multiple packages
[ERROR] share a common dependency and the first reference encountered by the scanner
[ERROR] sets the version to one that does not contain a subpackage needed required
[ERROR] by another package that uses the shared dependency. Try setting a
[ERROR] version in your glide.yaml that works for all packages that share this
[ERROR] dependency.

[...]

[ERROR] Failed to retrieve a list of dependencies: Error resolving imports
```

In order to resolve `github.com/vishvananda/netns` dependency and build
correctly we need to use `cni#v0.3.0`.

Following patch proposes introducing `glide.lock` with list of nested
dependencies needed to build `init.go`.

Best,
Rajiv

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[stage1-xen PATCH v1] init: Add `glide.lock`

[Rajiv Ranganath]

Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
---
 build.sh        |    5 +--
 init/glide.lock |   89 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 init/glide.yaml |   23 ++++++++++++++
 3 files changed, 114 insertions(+), 3 deletions(-)
 create mode 100644 init/glide.lock
 create mode 100644 init/glide.yaml

diff --git a/build.sh b/build.sh
index ec56093..6c34890 100755
--- a/build.sh
+++ b/build.sh
@@ -83,10 +83,9 @@ if [ -f stage1-xen.aci ]; then
 fi
 
 # Build init
-go get github.com/hashicorp/errwrap
 cd init
-glide init || true
-glide up -v 
+rm -rf vendor
+glide install -v
 cd ..
 go build -o target/rootfs/init init/init.go
 
diff --git a/init/glide.lock b/init/glide.lock
new file mode 100644
index 0000000..f512bc7
--- /dev/null
+++ b/init/glide.lock
@@ -0,0 +1,89 @@
+hash: eb0d5fbb629911862615dfdc4dde5283949af890a06d3ff70662e507385bd14b
+updated: 2017-08-12T09:56:42.779804672Z
+imports:
+- name: github.com/appc/spec
+  version: 210e2995a04148739121566b71b7440512467cc2
+  subpackages:
+  - aci
+  - pkg/device
+  - pkg/tarheader
+  - schema
+  - schema/common
+  - schema/types
+  - schema/types/resource
+- name: github.com/containernetworking/cni
+  version: 5c3c17164270150467498a32c71436c7cd5501be
+  subpackages:
+  - pkg/ip
+  - pkg/ns
+  - pkg/types
+  - pkg/utils
+  - pkg/utils/sysctl
+- name: github.com/coreos/go-iptables
+  version: f2ede9c85e2fac4d72d5a9af0af59c0858d7a3bd
+  subpackages:
+  - iptables
+- name: github.com/coreos/go-semver
+  version: 1817cd4bea52af76542157eeabd74b057d1a199e
+  subpackages:
+  - semver
+- name: github.com/coreos/go-systemd
+  version: d2196463941895ee908e13531a23a39feb9e1243
+  subpackages:
+  - unit
+- name: github.com/d2g/dhcp4
+  version: fcbeb8a548ebd34b55134f2833c5b036a941aa82
+- name: github.com/d2g/dhcp4client
+  version: 8ca8fe2cad1770f068782377ec6be6733c01a96b
+- name: github.com/hashicorp/errwrap
+  version: 7554cd9344cec97297fa6649b055a8c98c2a1e55
+- name: github.com/rkt/rkt
+  version: 142050d1a558ab07f6eeddea55c0f51053a99b05
+  subpackages:
+  - common
+  - common/cgroup
+  - common/cgroup/v1
+  - common/cgroup/v2
+  - common/networking
+  - networking/netinfo
+  - networking/tuntap
+  - pkg/acl
+  - pkg/fileutil
+  - pkg/flag
+  - pkg/fs
+  - pkg/group
+  - pkg/log
+  - pkg/mountinfo
+  - pkg/passwd
+  - pkg/sys
+  - pkg/user
+  - stage1/common
+  - stage1/common/types
+  - stage1/init/common
+- name: github.com/spf13/pflag
+  version: e57e3eeb33f795204c1ca35f56c44f83227c6e66
+- name: github.com/sstabellini/rkt
+  version: 8a57cb8b6682ed8fef054f57efef292781597fde
+  subpackages:
+  - networking
+- name: github.com/syndtr/gocapability
+  version: db04d3cc01c8b54962a58ec7e491717d06cfcc16
+  subpackages:
+  - capability
+- name: github.com/vishvananda/netlink
+  version: f5a6f697a596c788d474984a38a0ac4ba0719e93
+  subpackages:
+  - nl
+- name: github.com/vishvananda/netns
+  version: 86bef332bfc3b59b7624a600bd53009ce91a9829
+- name: go4.org
+  version: 034d17a462f7b2dcd1a4a73553ec5357ff6e6c6e
+  subpackages:
+  - errorutil
+- name: golang.org/x/sys
+  version: e42485b6e20ae7d2304ec72e535b103ed350cc02
+  subpackages:
+  - unix
+- name: gopkg.in/inf.v0
+  version: 3887ee99ecf07df5b447e9b00d9c0b2adaa9f3e4
+testImports: []
diff --git a/init/glide.yaml b/init/glide.yaml
new file mode 100644
index 0000000..3919338
--- /dev/null
+++ b/init/glide.yaml
@@ -0,0 +1,23 @@
+package: github.com/rkt/stage1-xen/init
+import:
+- package: github.com/appc/spec
+  subpackages:
+  - schema/types
+- package: github.com/hashicorp/errwrap
+- package: github.com/rkt/rkt
+  subpackages:
+  - common
+  - common/networking
+  - pkg/flag
+  - pkg/log
+  - pkg/sys
+  - stage1/common
+  - stage1/common/types
+  - stage1/init/common
+- package: github.com/sstabellini/rkt
+  subpackages:
+  - networking
+- package: github.com/containernetworking/cni
+  version: 0.3.0
+- package: github.com/d2g/dhcp4
+- package: github.com/d2g/dhcp4client


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1] init: Add `glide.lock`

[Stefano Stabellini]

Thank you for the patch. Usually the description that you sent in the
previous email is written here.

I like the build.sh changes and I think introducing init/glide.yaml is a
great idea. But I don't think that introducing init/glide.lock is
necessary, is it? We could let glide generate it on the fly based on the
key versioning info already specified in glide.yaml.

For example, this patch already introduces:

  - package: github.com/containernetworking/cni
    version: 0.3.0

to glide.yaml. Are there any other reasons for committing glide.lock to
the repository instead of generating it?


On Sat, 12 Aug 2017, Rajiv Ranganath wrote:
> Signed-off-by: Rajiv Ranganath <rajiv.ranganath@atihita.com>
> ---
>  build.sh        |    5 +--
>  init/glide.lock |   89 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  init/glide.yaml |   23 ++++++++++++++
>  3 files changed, 114 insertions(+), 3 deletions(-)
>  create mode 100644 init/glide.lock
>  create mode 100644 init/glide.yaml
> 
> diff --git a/build.sh b/build.sh
> index ec56093..6c34890 100755
> --- a/build.sh
> +++ b/build.sh
> @@ -83,10 +83,9 @@ if [ -f stage1-xen.aci ]; then
>  fi
>  
>  # Build init
> -go get github.com/hashicorp/errwrap
>  cd init
> -glide init || true
> -glide up -v 
> +rm -rf vendor
> +glide install -v
>  cd ..
>  go build -o target/rootfs/init init/init.go
>  
> diff --git a/init/glide.lock b/init/glide.lock
> new file mode 100644
> index 0000000..f512bc7
> --- /dev/null
> +++ b/init/glide.lock
> @@ -0,0 +1,89 @@
> +hash: eb0d5fbb629911862615dfdc4dde5283949af890a06d3ff70662e507385bd14b
> +updated: 2017-08-12T09:56:42.779804672Z
> +imports:
> +- name: github.com/appc/spec
> +  version: 210e2995a04148739121566b71b7440512467cc2
> +  subpackages:
> +  - aci
> +  - pkg/device
> +  - pkg/tarheader
> +  - schema
> +  - schema/common
> +  - schema/types
> +  - schema/types/resource
> +- name: github.com/containernetworking/cni
> +  version: 5c3c17164270150467498a32c71436c7cd5501be
> +  subpackages:
> +  - pkg/ip
> +  - pkg/ns
> +  - pkg/types
> +  - pkg/utils
> +  - pkg/utils/sysctl
> +- name: github.com/coreos/go-iptables
> +  version: f2ede9c85e2fac4d72d5a9af0af59c0858d7a3bd
> +  subpackages:
> +  - iptables
> +- name: github.com/coreos/go-semver
> +  version: 1817cd4bea52af76542157eeabd74b057d1a199e
> +  subpackages:
> +  - semver
> +- name: github.com/coreos/go-systemd
> +  version: d2196463941895ee908e13531a23a39feb9e1243
> +  subpackages:
> +  - unit
> +- name: github.com/d2g/dhcp4
> +  version: fcbeb8a548ebd34b55134f2833c5b036a941aa82
> +- name: github.com/d2g/dhcp4client
> +  version: 8ca8fe2cad1770f068782377ec6be6733c01a96b
> +- name: github.com/hashicorp/errwrap
> +  version: 7554cd9344cec97297fa6649b055a8c98c2a1e55
> +- name: github.com/rkt/rkt
> +  version: 142050d1a558ab07f6eeddea55c0f51053a99b05
> +  subpackages:
> +  - common
> +  - common/cgroup
> +  - common/cgroup/v1
> +  - common/cgroup/v2
> +  - common/networking
> +  - networking/netinfo
> +  - networking/tuntap
> +  - pkg/acl
> +  - pkg/fileutil
> +  - pkg/flag
> +  - pkg/fs
> +  - pkg/group
> +  - pkg/log
> +  - pkg/mountinfo
> +  - pkg/passwd
> +  - pkg/sys
> +  - pkg/user
> +  - stage1/common
> +  - stage1/common/types
> +  - stage1/init/common
> +- name: github.com/spf13/pflag
> +  version: e57e3eeb33f795204c1ca35f56c44f83227c6e66
> +- name: github.com/sstabellini/rkt
> +  version: 8a57cb8b6682ed8fef054f57efef292781597fde
> +  subpackages:
> +  - networking
> +- name: github.com/syndtr/gocapability
> +  version: db04d3cc01c8b54962a58ec7e491717d06cfcc16
> +  subpackages:
> +  - capability
> +- name: github.com/vishvananda/netlink
> +  version: f5a6f697a596c788d474984a38a0ac4ba0719e93
> +  subpackages:
> +  - nl
> +- name: github.com/vishvananda/netns
> +  version: 86bef332bfc3b59b7624a600bd53009ce91a9829
> +- name: go4.org
> +  version: 034d17a462f7b2dcd1a4a73553ec5357ff6e6c6e
> +  subpackages:
> +  - errorutil
> +- name: golang.org/x/sys
> +  version: e42485b6e20ae7d2304ec72e535b103ed350cc02
> +  subpackages:
> +  - unix
> +- name: gopkg.in/inf.v0
> +  version: 3887ee99ecf07df5b447e9b00d9c0b2adaa9f3e4
> +testImports: []
> diff --git a/init/glide.yaml b/init/glide.yaml
> new file mode 100644
> index 0000000..3919338
> --- /dev/null
> +++ b/init/glide.yaml
> @@ -0,0 +1,23 @@
> +package: github.com/rkt/stage1-xen/init
> +import:
> +- package: github.com/appc/spec
> +  subpackages:
> +  - schema/types
> +- package: github.com/hashicorp/errwrap
> +- package: github.com/rkt/rkt
> +  subpackages:
> +  - common
> +  - common/networking
> +  - pkg/flag
> +  - pkg/log
> +  - pkg/sys
> +  - stage1/common
> +  - stage1/common/types
> +  - stage1/init/common
> +- package: github.com/sstabellini/rkt
> +  subpackages:
> +  - networking
> +- package: github.com/containernetworking/cni
> +  version: 0.3.0
> +- package: github.com/d2g/dhcp4
> +- package: github.com/d2g/dhcp4client
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1] init: Add `glide.lock`

[Rajiv Ranganath]

On Tue, Aug 15 2017 at 06:23:07 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:
> Thank you for the patch. Usually the description that you sent in the
> previous email is written here.
>
> I like the build.sh changes and I think introducing init/glide.yaml is a
> great idea. But I don't think that introducing init/glide.lock is
> necessary, is it? We could let glide generate it on the fly based on the
> key versioning info already specified in glide.yaml.
>
> For example, this patch already introduces:
>
>   - package: github.com/containernetworking/cni
>     version: 0.3.0
>
> to glide.yaml. Are there any other reasons for committing glide.lock to
> the repository instead of generating it?

I think the pattern of using `.lock` files to manage nested library
dependencies and semantic versioning for library APIs was initially
championed in the Ruby on Rails community. The idea has since been
adopted by Go community in Glide, Rust community in Cargo and JavaScript
community in Yarn.

Here is the link to the original discussion on whether `Gemfile.lock`
should be checked into the source tree or not. [1]

If we go by author's line of reasoning, then answer would depend on if
we consider init to be an app or a library.

Personally, I feel `init.go` is an app and it would make sense to check
in `glide.lock`.

If for some reason, in future there is a build failure due to a nested
dependency issue with dependent go libraries, then having a working
`.lock` in the git is always useful.

In anycase after sending `BUILDING.md` Fedora patches, I am also
planning on sending patches to do continuous build of `stage1-xen` in a
Fedora based docker container. That should also catch build failures
early.

Please let me know what you prefer. I can send a v2 of the patch with
just `glide.yaml`

Best,
Rajiv

[1] http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [stage1-xen PATCH v1] init: Add `glide.lock`

[Stefano Stabellini]

On Tue, 15 Aug 2017, Rajiv Ranganath wrote:
> On Tue, Aug 15 2017 at 06:23:07 AM, Stefano Stabellini <sstabellini@kernel.org> wrote:
> > Thank you for the patch. Usually the description that you sent in the
> > previous email is written here.
> >
> > I like the build.sh changes and I think introducing init/glide.yaml is a
> > great idea. But I don't think that introducing init/glide.lock is
> > necessary, is it? We could let glide generate it on the fly based on the
> > key versioning info already specified in glide.yaml.
> >
> > For example, this patch already introduces:
> >
> >   - package: github.com/containernetworking/cni
> >     version: 0.3.0
> >
> > to glide.yaml. Are there any other reasons for committing glide.lock to
> > the repository instead of generating it?
> 
> I think the pattern of using `.lock` files to manage nested library
> dependencies and semantic versioning for library APIs was initially
> championed in the Ruby on Rails community. The idea has since been
> adopted by Go community in Glide, Rust community in Cargo and JavaScript
> community in Yarn.
> 
> Here is the link to the original discussion on whether `Gemfile.lock`
> should be checked into the source tree or not. [1]
> 
> If we go by author's line of reasoning, then answer would depend on if
> we consider init to be an app or a library.
> 
> Personally, I feel `init.go` is an app and it would make sense to check
> in `glide.lock`.
> 
> If for some reason, in future there is a build failure due to a nested
> dependency issue with dependent go libraries, then having a working
> `.lock` in the git is always useful.
> 
> In anycase after sending `BUILDING.md` Fedora patches, I am also
> planning on sending patches to do continuous build of `stage1-xen` in a
> Fedora based docker container. That should also catch build failures
> early.
> 
> Please let me know what you prefer. I can send a v2 of the patch with
> just `glide.yaml`

I read the explanation and I find it convincing. init.go is definitely
an app. I'll check in the patch as is.


> Best,
> Rajiv
> 
> [1] http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel