nerdctl

nerdctl

[Joakim Roubert]

Hi!

On KubeCon Europe, there was a lot of talk about nerdctl (https://github.com/containerd/nerdctl) in various places, and it indeed seems like a nice tool when working in containerd-only environments (yes, I am one of those who find crictl a bit cumbersome to cooperate with). Is nerdctl something that would be interesting to get into meta-virtualization? Perhaps somebody is already working on this?

BR,

/Joakim

Re: [meta-virtualization] nerdctl

[Bruce Ashfield]

On Mon, May 10, 2021 at 3:34 AM Joakim Roubert <joakim.roubert@axis.com> wrote:
>
> Hi!
>
> On KubeCon Europe, there was a lot of talk about nerdctl (https://github.com/containerd/nerdctl) in various places, and it indeed seems like a nice tool when working in containerd-only environments (yes, I am one of those who find crictl a bit cumbersome to cooperate with). Is nerdctl something that would be interesting to get into meta-virtualization? Perhaps somebody is already working on this?

I'm actually already working on it, and have been for about a month.

There's a few build issues remaining, but I expect to have it done in
the not to distant future.

Bruce

>
> BR,
>
> /Joakim
>
>
> 
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] nerdctl

[Joakim Roubert]

On 2021-05-10 14:10, Bruce Ashfield wrote:
> 
> I'm actually already working on it, and have been for about a month.

Awesome!!

> There's a few build issues remaining, but I expect to have it done in
> the not to distant future.

Great news, makes me think of that old movie title "Bruce Almighty".
Kudos!

BR,

/J

Re: [meta-virtualization] nerdctl

[Bruce Ashfield]

On Mon, May 10, 2021 at 12:59 PM Joakim Roubert <joakim.roubert@axis.com> wrote:
>
> On 2021-05-10 14:10, Bruce Ashfield wrote:
> >
> > I'm actually already working on it, and have been for about a month.
>
> Awesome!!
>
> > There's a few build issues remaining, but I expect to have it done in
> > the not to distant future.
>
> Great news, makes me think of that old movie title "Bruce Almighty".
> Kudos!

Ha. I wouldn't say that :D

When I was working on k3s I was adding sanity tests for containerd and
was reading about nerctl and the docker-like interface it provided ..
so I thought it was a good idea.

The issue with the recipe, is that it has an insane amount of go
dependencies and is heavily go.mod based.  I had 0.7x nearly working,
with only some go build errors due to may layout of the source not
being quite right .. I attempted to update to 0.8.x recently and the
amount of SRC_URI churn was huge .. so I wrote myself some helper
tools to generate things.

I'm still working through the fallout of that 0.8.x update, but
hopefully I'll get through it shortly.

Bruce

>
> BR,
>
> /J
>
> 
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] nerdctl

[Bruce Ashfield]

On Mon, May 10, 2021 at 3:15 PM Bruce Ashfield via
lists.yoctoproject.org
<bruce.ashfield=gmail.com@lists.yoctoproject.org> wrote:
>
> On Mon, May 10, 2021 at 12:59 PM Joakim Roubert <joakim.roubert@axis.com> wrote:
> >
> > On 2021-05-10 14:10, Bruce Ashfield wrote:
> > >
> > > I'm actually already working on it, and have been for about a month.
> >
> > Awesome!!
> >
> > > There's a few build issues remaining, but I expect to have it done in
> > > the not to distant future.
> >
> > Great news, makes me think of that old movie title "Bruce Almighty".
> > Kudos!
>
> Ha. I wouldn't say that :D
>
> When I was working on k3s I was adding sanity tests for containerd and
> was reading about nerctl and the docker-like interface it provided ..
> so I thought it was a good idea.
>
> The issue with the recipe, is that it has an insane amount of go
> dependencies and is heavily go.mod based.  I had 0.7x nearly working,
> with only some go build errors due to may layout of the source not
> being quite right .. I attempted to update to 0.8.x recently and the
> amount of SRC_URI churn was huge .. so I wrote myself some helper
> tools to generate things.
>
> I'm still working through the fallout of that 0.8.x update, but
> hopefully I'll get through it shortly.

(resending with the list address fixed)

It took several months, but:

root@qemux86-64:~# uname -a
Linux qemux86-64 5.17.0-rc6-yoctodev-standard #1 SMP PREEMPT Sun Feb
27 22:36:33 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
root@qemux86-64:~# nerdctl --version
nerdctl version 0.17.0-10-gf2427ff.m
root@qemux86-64:~#

The recipe isn't available yet, since this is using a huge amount of
fetched go dependencies, using some new scripts/tooling that I created
.. but at least I know the approach is feasible!

Bruce

>
> Bruce
>
> >
> > BR,
> >
> > /J
> >
> >
> >
>
>
> --
> - Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end
> - "Use the force Harry" - Gandalf, Star Trek II
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#6501): https://lists.yoctoproject.org/g/meta-virtualization/message/6501
> Mute This Topic: https://lists.yoctoproject.org/mt/82714461/1050810
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] nerdctl

[ChenQi]

Hi Bruce and Joakim,

I'm recently looking into this nerdctl recipe. I'm kind of shocked by 
the SRC_URI number in this recipe.
Looking at the discussion on the mailing list (go.bbclass: Allow network 
in do_compile), it seems that they are needed for offline build for this 
recipe.

Bruce, how do you get all these contents? Also, how do you generate the 
modules.txt file? You mentioned some helping script in the discussion, 
what's the status of that script?

Joakim, maybe I misunderstood it, but are you still working on an 
upgrade of this recipe? If not, I'm going to try to do it.

Last but not least, Bruce, thank you again for making this recipe work.

Regards,
Qi

On 3/7/22 15:30, Joakim Roubert wrote:
> On 2022-03-04 16:48, Bruce Ashfield wrote:
>>
>> It took several months, but:
>>
>> root@qemux86-64:~# uname -a
>> Linux qemux86-64 5.17.0-rc6-yoctodev-standard #1 SMP PREEMPT Sun Feb
>> 27 22:36:33 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
>> root@qemux86-64:~# nerdctl --version
>> nerdctl version 0.17.0-10-gf2427ff.m
>> root@qemux86-64:~#
>
> Really cool!!!
>
>> The recipe isn't available yet, since this is using a huge amount of
>> fetched go dependencies, using some new scripts/tooling that I created
>> .. but at least I know the approach is feasible!
>
> Excellent news!
>
> BR,
>
> /Joakim
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#7078): https://lists.yoctoproject.org/g/meta-virtualization/message/7078
> Mute This Topic: https://lists.yoctoproject.org/mt/82714461/3618072
> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [Qi.Chen@windriver.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>

Re: [meta-virtualization] nerdctl

[Bruce Ashfield]

On Mon, Apr 18, 2022 at 2:17 AM ChenQi <Qi.Chen@windriver.com> wrote:
>
> Hi Bruce and Joakim,
>
> I'm recently looking into this nerdctl recipe. I'm kind of shocked by
> the SRC_URI number in this recipe.

k3s is much, much worse.

> Looking at the discussion on the mailing list (go.bbclass: Allow network

Right, we need this for licensing, SBOM, reproducibility, etc. As you
say, there's a shocking amount of code/dependencies that are fetched
for a lot of these recipes. The version management and scaling of them
to individual recipes would be nearly impossible .. so this is my
current effort :)

> in do_compile), it seems that they are needed for offline build for this
> recipe.
>

Yes, they are (almost) all required (indirect dependency generation
isn't exact at times, so there may be a few extras that aren't used in
some build configurations).

> Bruce, how do you get all these contents? Also, how do you generate the
> modules.txt file? You mentioned some helping script in the discussion,
> what's the status of that script?

The tool is still a WIP, once we get past the current release, I'm
going to complete the cleanup and make it more widely available. It
currently isn't really usable.

The script generates the SRC_URI, the modules.txt and the vendor/
relocation parts of the recipe.

>
> Joakim, maybe I misunderstood it, but are you still working on an
> upgrade of this recipe? If not, I'm going to try to do it.

I have an in progress update here.

I'm using it to test the script in that mode, I should have the update
available in the next little while, and then the script that generates
it will follow.

Cheers,

Bruce

>
> Last but not least, Bruce, thank you again for making this recipe work.
>
> Regards,
> Qi
>
> On 3/7/22 15:30, Joakim Roubert wrote:
> > On 2022-03-04 16:48, Bruce Ashfield wrote:
> >>
> >> It took several months, but:
> >>
> >> root@qemux86-64:~# uname -a
> >> Linux qemux86-64 5.17.0-rc6-yoctodev-standard #1 SMP PREEMPT Sun Feb
> >> 27 22:36:33 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
> >> root@qemux86-64:~# nerdctl --version
> >> nerdctl version 0.17.0-10-gf2427ff.m
> >> root@qemux86-64:~#
> >
> > Really cool!!!
> >
> >> The recipe isn't available yet, since this is using a huge amount of
> >> fetched go dependencies, using some new scripts/tooling that I created
> >> .. but at least I know the approach is feasible!
> >
> > Excellent news!
> >
> > BR,
> >
> > /Joakim
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#7078): https://lists.yoctoproject.org/g/meta-virtualization/message/7078
> > Mute This Topic: https://lists.yoctoproject.org/mt/82714461/3618072
> > Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> > Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [Qi.Chen@windriver.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II

Re: [meta-virtualization] nerdctl

[ChenQi]

Hi Bruce,

I found that the nerdctl does not work on qemuarm. After tracking down 
the problem, I found that it's caused by CONFIG_OABI_COMPAT being 
enabled for qemuarm. It needs to be disabled for SECCOMP_FILTER to work, 
which is needed by nerdctl.

Looking at the help message of 'CONFIG_OABI_COMPAT', it seems that it's 
pretty safe for us to disable it as we build things from scratch. And it 
suggests that "If in doubt say N.".

Do you think this option should be disabled by default in yocto's kernel 
config?

Regards,
Qi

On 4/18/22 21:42, Bruce Ashfield wrote:
> On Mon, Apr 18, 2022 at 2:17 AM ChenQi <Qi.Chen@windriver.com> wrote:
>> Hi Bruce and Joakim,
>>
>> I'm recently looking into this nerdctl recipe. I'm kind of shocked by
>> the SRC_URI number in this recipe.
> k3s is much, much worse.
>
>> Looking at the discussion on the mailing list (go.bbclass: Allow network
> Right, we need this for licensing, SBOM, reproducibility, etc. As you
> say, there's a shocking amount of code/dependencies that are fetched
> for a lot of these recipes. The version management and scaling of them
> to individual recipes would be nearly impossible .. so this is my
> current effort :)
>
>> in do_compile), it seems that they are needed for offline build for this
>> recipe.
>>
> Yes, they are (almost) all required (indirect dependency generation
> isn't exact at times, so there may be a few extras that aren't used in
> some build configurations).
>
>> Bruce, how do you get all these contents? Also, how do you generate the
>> modules.txt file? You mentioned some helping script in the discussion,
>> what's the status of that script?
> The tool is still a WIP, once we get past the current release, I'm
> going to complete the cleanup and make it more widely available. It
> currently isn't really usable.
>
> The script generates the SRC_URI, the modules.txt and the vendor/
> relocation parts of the recipe.
>
>> Joakim, maybe I misunderstood it, but are you still working on an
>> upgrade of this recipe? If not, I'm going to try to do it.
> I have an in progress update here.
>
> I'm using it to test the script in that mode, I should have the update
> available in the next little while, and then the script that generates
> it will follow.
>
> Cheers,
>
> Bruce
>
>> Last but not least, Bruce, thank you again for making this recipe work.
>>
>> Regards,
>> Qi
>>
>> On 3/7/22 15:30, Joakim Roubert wrote:
>>> On 2022-03-04 16:48, Bruce Ashfield wrote:
>>>> It took several months, but:
>>>>
>>>> root@qemux86-64:~# uname -a
>>>> Linux qemux86-64 5.17.0-rc6-yoctodev-standard #1 SMP PREEMPT Sun Feb
>>>> 27 22:36:33 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
>>>> root@qemux86-64:~# nerdctl --version
>>>> nerdctl version 0.17.0-10-gf2427ff.m
>>>> root@qemux86-64:~#
>>> Really cool!!!
>>>
>>>> The recipe isn't available yet, since this is using a huge amount of
>>>> fetched go dependencies, using some new scripts/tooling that I created
>>>> .. but at least I know the approach is feasible!
>>> Excellent news!
>>>
>>> BR,
>>>
>>> /Joakim
>>>
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>> Links: You receive all messages sent to this group.
>>> View/Reply Online (#7078): https://lists.yoctoproject.org/g/meta-virtualization/message/7078
>>> Mute This Topic: https://lists.yoctoproject.org/mt/82714461/3618072
>>> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
>>> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [Qi.Chen@windriver.com]
>>> -=-=-=-=-=-=-=-=-=-=-=-
>>>
>

Re: [meta-virtualization] nerdctl

[Bruce Ashfield]

On Thu, Apr 21, 2022 at 10:38 PM ChenQi <Qi.Chen@windriver.com> wrote:
>
> Hi Bruce,
>
> I found that the nerdctl does not work on qemuarm. After tracking down
> the problem, I found that it's caused by CONFIG_OABI_COMPAT being
> enabled for qemuarm. It needs to be disabled for SECCOMP_FILTER to work,
> which is needed by nerdctl.
>
> Looking at the help message of 'CONFIG_OABI_COMPAT', it seems that it's
> pretty safe for us to disable it as we build things from scratch. And it
> suggests that "If in doubt say N.".
>
> Do you think this option should be disabled by default in yocto's kernel
> config?

Yes, it can be changed to =n at this point, but we still want to be
explicit and not take any defaults. It was a transitional ABI setting
.. almost 10 years ago. We can safely say that the transition period
is over.

As well as changing it to disabled, we should create a new fragment
with a name like "enable-oabi.scc" and the associated
"enable-oabi.cfg" that could be used via KERNEL_FEATURES to enable the
OABI functionality if anyone really needs it.

Bruce

>
> Regards,
> Qi
>
> On 4/18/22 21:42, Bruce Ashfield wrote:
> > On Mon, Apr 18, 2022 at 2:17 AM ChenQi <Qi.Chen@windriver.com> wrote:
> >> Hi Bruce and Joakim,
> >>
> >> I'm recently looking into this nerdctl recipe. I'm kind of shocked by
> >> the SRC_URI number in this recipe.
> > k3s is much, much worse.
> >
> >> Looking at the discussion on the mailing list (go.bbclass: Allow network
> > Right, we need this for licensing, SBOM, reproducibility, etc. As you
> > say, there's a shocking amount of code/dependencies that are fetched
> > for a lot of these recipes. The version management and scaling of them
> > to individual recipes would be nearly impossible .. so this is my
> > current effort :)
> >
> >> in do_compile), it seems that they are needed for offline build for this
> >> recipe.
> >>
> > Yes, they are (almost) all required (indirect dependency generation
> > isn't exact at times, so there may be a few extras that aren't used in
> > some build configurations).
> >
> >> Bruce, how do you get all these contents? Also, how do you generate the
> >> modules.txt file? You mentioned some helping script in the discussion,
> >> what's the status of that script?
> > The tool is still a WIP, once we get past the current release, I'm
> > going to complete the cleanup and make it more widely available. It
> > currently isn't really usable.
> >
> > The script generates the SRC_URI, the modules.txt and the vendor/
> > relocation parts of the recipe.
> >
> >> Joakim, maybe I misunderstood it, but are you still working on an
> >> upgrade of this recipe? If not, I'm going to try to do it.
> > I have an in progress update here.
> >
> > I'm using it to test the script in that mode, I should have the update
> > available in the next little while, and then the script that generates
> > it will follow.
> >
> > Cheers,
> >
> > Bruce
> >
> >> Last but not least, Bruce, thank you again for making this recipe work.
> >>
> >> Regards,
> >> Qi
> >>
> >> On 3/7/22 15:30, Joakim Roubert wrote:
> >>> On 2022-03-04 16:48, Bruce Ashfield wrote:
> >>>> It took several months, but:
> >>>>
> >>>> root@qemux86-64:~# uname -a
> >>>> Linux qemux86-64 5.17.0-rc6-yoctodev-standard #1 SMP PREEMPT Sun Feb
> >>>> 27 22:36:33 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
> >>>> root@qemux86-64:~# nerdctl --version
> >>>> nerdctl version 0.17.0-10-gf2427ff.m
> >>>> root@qemux86-64:~#
> >>> Really cool!!!
> >>>
> >>>> The recipe isn't available yet, since this is using a huge amount of
> >>>> fetched go dependencies, using some new scripts/tooling that I created
> >>>> .. but at least I know the approach is feasible!
> >>> Excellent news!
> >>>
> >>> BR,
> >>>
> >>> /Joakim
> >>>
> >>> -=-=-=-=-=-=-=-=-=-=-=-
> >>> Links: You receive all messages sent to this group.
> >>> View/Reply Online (#7078): https://lists.yoctoproject.org/g/meta-virtualization/message/7078
> >>> Mute This Topic: https://lists.yoctoproject.org/mt/82714461/3618072
> >>> Group Owner: meta-virtualization+owner@lists.yoctoproject.org
> >>> Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [Qi.Chen@windriver.com]
> >>> -=-=-=-=-=-=-=-=-=-=-=-
> >>>
> >
>


-- 
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II