Re: [meta-virtualization][kirkstone][master][PATCH 2/4] kubernetes: install the binaries in OE standard places

[Bruce Ashfield <bruce.ashfield@gmail.com>]

On Tue, Aug 9, 2022 at 12:36 PM Jose Quaresma <quaresma.jose@gmail.com> wrote:
>
>
>
> Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia terça, 9/08/2022 à(s) 16:05:
>>
>> On Tue, Aug 9, 2022 at 10:21 AM Bruce Ashfield via
>> lists.yoctoproject.org
>> <bruce.ashfield=gmail.com@lists.yoctoproject.org> wrote:
>> >
>> > On Tue, Aug 9, 2022 at 10:02 AM Jose Quaresma <quaresma.jose@gmail.com> wrote:
>> > >
>> > >
>> > >
>> > > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia terça, 9/08/2022 à(s) 14:43:
>> > >>
>> > >> On Tue, Aug 9, 2022 at 9:34 AM Jose Quaresma <quaresma.jose@gmail.com> wrote:
>> > >> >
>> > >> > Hi Bruce,
>> > >> >
>> > >> > Bruce Ashfield <bruce.ashfield@gmail.com> escreveu no dia terça, 9/08/2022 à(s) 14:19:
>> > >> >>
>> > >> >> You must have something different in your environment, I'm not seeing
>> > >> >> any of these same issues.
>> > >> >
>> > >> >
>> > >> > Maybe because you have the BIN_PREFIX defined somewhere in your distro or local.conf
>> > >> > BIN_PREFIX is defined only in k3s recipe BIN_PREFIX ?= "${exec_prefix}/local"
>> > >> >
>> > >>
>> > >> I'm building a plain OE + meta-virt. BIN_PREFIX isn't in the
>> > >> environment anywhere (which is admittedly strange .. but that's
>> > >> consistent with how it has been).
>> > >>
>> > >> Also, there's no other reports of this ever on the mailing list,
>> > >> including demos for conferences, etc. ... that fails the "how can this
>> > >> be working for everyone else ?" test.
>> > >>
>> > >> So there's definitely something different that I'm not seeing. I use
>> > >> OE nodistro or poky, others are using some other distros .. so I need
>> > >> to figure out what is the difference.
>> > >>
>> > >> That being said, even if we tweaked the binaries install, we don't
>> > >> want them changing where they've been installed previously, there
>> > >> could be any number of scripts expecting those locations in layers
>> > >> that I don't maintain.
>> > >>
>> > >> i.e. there's no way we should be patching the .service file, since
>> > >> that indicates the binaries have moved from where they were before.
>> > >>
>> > >> Bruce
>> > >>
>> > >
>> > > By default with BIN_PREFIX="" the binaries seem to be installed on /bin/k8s-init so they will work as this is in the PATH.
>> > > For OE nodistro maybe this QA is disabled (need to confirm that) but for distros that have this QA enabled it will fire up.
>> > >
>> >
>> > The installed but not shipped, isn't inhibited anywhere that I know
>> > of. Certainly not on poky, which is where most of the nightly builds
>> > happen for this.
>> >
>> > > About moving the binaries for another place is mainly because /bin is not the right place for them IMO.
>> > > https://tldp.org/LDP/Linux-Filesystem-Hierarchy/html/bin.html
>> >
>> > But that's not where they are going for all the rest of the builds :)
>> > I'm aware of the various filesystem standards.
>> >
>> > >
>> > > But I forgot to update all the services involved in this patch set so please drop it all and I will update all services as well in V2.
>> > >
>> >
>> > I wouldn't bother with that for v2, since again, they are using
>> > /usr/bin and /usr/local/bin from all the .service files I recall, and
>> > that's where we want the binaries to continue to be.
>> >
>>
>> aha. I see that k3s does set BIN_PREFIX, but it has been lost for
>> kubernetes and nerdctl.
>
>
> right, maybe you don't see the QA because it will only shon for files installed in /bin when usrmerge is enabled.

That could very well be it.

>
> for nerdctl, the BIN_PREFIX is there from the beginning so the recipe allway install it in /bin.
> for kubernetes, BIN_PREFIX is added in [1] and this only install k8s-init in /bin and all the other binaries
> continue where they were installed (in /usr/bin in this case).
>
> [1] - https://git.yoctoproject.org/meta-virtualization/commit/?id=4d0f0a5ca2338e5f6ed3fe3a18c602447cf60eb4
>
>>
>> The easiest / lowest footprint route is to just add the variable back
>> in for kubernetes and nerdctl, to the current default location. That
>> allows folks to override it and/or keep their scripts/service files
>> unmodified.
>
>
> This is what I did but using ${bindir} instead of ${BIN_PREFIX}.
> Adding the BIN_PREFIX ?= "${exec_prefix}/local" will change install location so
> since we have to change, we can use the ${bindir} and put the files in the right place.

I'm really only interested in the BIN_PREFIX being set in those
recipes. The smallest
footprint change.

So as the series currently stands, it isn't something I'll merge.

Cheers,

Bruce


>
> The last patch for k3s is more invasive as it touches many places and the systemd services as well.
> I will send it separate because this one don't fix anything and only change the installation of binaries
> from /usr/local/bin to /usr/bin
>
> Jose
>
>>
>> Bruce
>>
>> > Bruce
>> >
>> > > Jose
>> > >
>> > >>
>> > >>
>> > >> > Jose
>> > >> >
>> > >> >>
>> > >> >>
>> > >> >> Cheers,
>> > >> >>
>> > >> >> Bruce
>> > >> >>
>> > >> >> On Tue, Aug 9, 2022 at 8:16 AM Jose Quaresma <quaresma.jose@gmail.com> wrote:
>> > >> >> >
>> > >> >> > - The env BIN_PREFIX is there from the beginning but there are no references to it,
>> > >> >> > also fix a fatal QA errors installed-vs-shipped.
>> > >> >> >
>> > >> >> > ERROR: kubernetes-1_v1.23.6+gitfbcfa33018159c033aee77b0d5456df6771aa9b5-r0 do_package: QA Issue: kubernetes: Files/directories were installed but not shipped in any package:
>> > >> >> >   /bin
>> > >> >> >   /bin/k8s-init
>> > >> >> > Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
>> > >> >> > kubernetes: 2 installed and not shipped files. [installed-vs-shipped]
>> > >> >> > ERROR: kubernetes-1_v1.23.6+gitfbcfa33018159c033aee77b0d5456df6771aa9b5-r0 do_package: Fatal QA errors were found, failing task.
>> > >> >> >
>> > >> >> > Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
>> > >> >> > ---
>> > >> >> >  recipes-containers/kubernetes/kubernetes_git.bb | 6 +++---
>> > >> >> >  1 file changed, 3 insertions(+), 3 deletions(-)
>> > >> >> >
>> > >> >> > diff --git a/recipes-containers/kubernetes/kubernetes_git.bb b/recipes-containers/kubernetes/kubernetes_git.bb
>> > >> >> > index e9460d4..82b75b1 100644
>> > >> >> > --- a/recipes-containers/kubernetes/kubernetes_git.bb
>> > >> >> > +++ b/recipes-containers/kubernetes/kubernetes_git.bb
>> > >> >> > @@ -103,8 +103,8 @@ do_install() {
>> > >> >> >      install -m 0644 ${WORKDIR}/git/release/cmd/kubepkg/templates/latest/deb/kubeadm/10-kubeadm.conf  ${D}${systemd_unitdir}/system/kubelet.service.d/
>> > >> >> >
>> > >> >> >      if ${@bb.utils.contains('DISTRO_FEATURES','systemd','true','false',d)}; then
>> > >> >> > -       install -d "${D}${BIN_PREFIX}/bin"
>> > >> >> > -       install -m 755 "${WORKDIR}/k8s-init" "${D}${BIN_PREFIX}/bin"
>> > >> >> > +       install -d ${D}${bindir}
>> > >> >> > +       install -m 755 ${WORKDIR}/k8s-init ${D}${bindir}
>> > >> >> >
>> > >> >> >         install -d ${D}${sysconfdir}/sysctl.d
>> > >> >> >         install -m 0644 "${WORKDIR}/99-kubernetes.conf" "${D}${sysconfdir}/sysctl.d"
>> > >> >> > @@ -141,7 +141,7 @@ FILES:kube-proxy = "${bindir}/kube-proxy"
>> > >> >> >  FILES:${PN}-misc = "${bindir} ${sysconfdir}/sysctl.d"
>> > >> >> >
>> > >> >> >  ALLOW_EMPTY:${PN}-host = "1"
>> > >> >> > -FILE:${PN}-host = "${BIN_PREFIX}/bin/k8s-init"
>> > >> >> > +FILE:${PN}-host = "${bindir}/k8s-init"
>> > >> >> >  RDEPENDS:${PN}-host = "${PN}"
>> > >> >> >
>> > >> >> >  RRECOMMENDS:${PN} = "\
>> > >> >> > --
>> > >> >> > 2.37.1
>> > >> >> >
>> > >> >> >
>> > >> >> >
>> > >> >> >
>> > >> >>
>> > >> >>
>> > >> >> --
>> > >> >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> > >> >> thee at its end
>> > >> >> - "Use the force Harry" - Gandalf, Star Trek II
>> > >> >
>> > >> >
>> > >> >
>> > >> > --
>> > >> > Best regards,
>> > >> >
>> > >> > José Quaresma
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> > >> thee at its end
>> > >> - "Use the force Harry" - Gandalf, Star Trek II
>> > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > >
>> > > José Quaresma
>> >
>> >
>> >
>> > --
>> > - Thou shalt not follow the NULL pointer, for chaos and madness await
>> > thee at its end
>> > - "Use the force Harry" - Gandalf, Star Trek II
>> >
>> > -=-=-=-=-=-=-=-=-=-=-=-
>> > Links: You receive all messages sent to this group.
>> > View/Reply Online (#7504): https://lists.yoctoproject.org/g/meta-virtualization/message/7504
>> > Mute This Topic: https://lists.yoctoproject.org/mt/92913426/1050810
>> > Group Owner: meta-virtualization+owner@lists.yoctoproject.org
>> > Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
>> > -=-=-=-=-=-=-=-=-=-=-=-
>> >
>>
>>
>> --
>> - Thou shalt not follow the NULL pointer, for chaos and madness await
>> thee at its end
>> - "Use the force Harry" - Gandalf, Star Trek II
>
>
>
> --
> Best regards,
>
> José Quaresma



--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II