Re: Improve kernelci setup for RISC-V

Re: Improve kernelci setup for RISC-V

[Kevin Hilman]

[ + kernelci list ]

Hi Atish,

Atish Patra <Atish.Patra@wdc.com> writes:

> I just want to follow up the discussions we had in the risc-v slack
> channel related to improving kernelci setup for RISC-V.
>
> 1. Adding Qemu target for RISC-V: As per out latest conversation, this
> won't be an issue anymore as Qemu supports -bios and -kernel
> separately.

Yes, this looks straight forward now.  We should be able to get this
going for our basic tests without too much more work.

> 2. Running more tests in addition to boot tests on unleashed target:
>
> The main blocker here is the rootfs. As kernelci only supports debian
> based rootfs to actually run tests. I think it is not that difficult to
> bootstrap a debian rootfs for RISC-V but I have not done that in ages.
> I think palmer uses debian rootfs for his testing. I have cc'd him for
> his inputs.

For every other arch, we debootstrap a buster rootfs, and this doesn't
work for riscv.  But, I just discovered[1] that this should work for
unstable using debian-ports, so we'll need to update our rootfs builder
to support unstable/debian-ports.

I've cc'd Lakshmipathi from Collabora who has been doing work on our
rootfs builder tool to see how we can extend this.

> But Alistair & khem (cc'd) expressed interest to add OE to kernelci as
> well. Can you share some pointers on what needs to be done ?

The main tricky/annoying part with yocto is that builds are per-board,
where as we like to build rootfs images per-arch.  Maybe there's a way
to build an arch-generic yocto rootfs using a qemu target or something,
though.

The place to start would be to look at the kci_rootfs tool in the
kernelci-core repo[2] along with the rootfs-config.yaml file and figure
out how to extend that for yocto/OE.

Again, Lakshmipathi may have some ideas there.

OTOH, I'm not quite sure what we would gain by using yocto instead of
debian?  Are there more packages for yocto available than for debian
currently?

> 3. How difficult is to setup kernelci from scratch locally ?

Not really, but it's not well documented. :(

But it depends on which parts you want to setup.  There are several
moving parts:

1) lab / board-farm
2) backend / storage
3) frontend
4) kernel builds (centralized, published publically to
  storage.kernelci.org)

We have some rudimentary docs for 2-4 here[3], but the compute part for
builds is currently centralized.

Kevin


[1] https://wiki.debian.org/RISC-V#debootstrap
[2] https://github.com/kernelci/kernelci-core/
[3] https://github.com/kernelci/kernelci-doc/wiki/Setting-up-a-local-development-instance

Re: Improve kernelci setup for RISC-V

[Atish Patra]

On Wed, 2020-07-01 at 17:17 -0700, Kevin Hilman wrote:
> [ + kernelci list ]
> 
> Hi Atish,
> 
> Atish Patra <Atish.Patra@wdc.com> writes:
> 
> > I just want to follow up the discussions we had in the risc-v slack
> > channel related to improving kernelci setup for RISC-V.
> > 
> > 1. Adding Qemu target for RISC-V: As per out latest conversation,
> > this
> > won't be an issue anymore as Qemu supports -bios and -kernel
> > separately.
> 
> Yes, this looks straight forward now.  We should be able to get this
> going for our basic tests without too much more work.
> 
> > 2. Running more tests in addition to boot tests on unleashed
> > target:
> > 
> > The main blocker here is the rootfs. As kernelci only supports
> > debian
> > based rootfs to actually run tests. I think it is not that
> > difficult to
> > bootstrap a debian rootfs for RISC-V but I have not done that in
> > ages.
> > I think palmer uses debian rootfs for his testing. I have cc'd him
> > for
> > his inputs.
> 
> For every other arch, we debootstrap a buster rootfs, and this
> doesn't
> work for riscv.  But, I just discovered[1] that this should work for
> unstable using debian-ports, so we'll need to update our rootfs
> builder
> to support unstable/debian-ports.
> 
> I've cc'd Lakshmipathi from Collabora who has been doing work on our
> rootfs builder tool to see how we can extend this.
> 

Thanks.

> > But Alistair & khem (cc'd) expressed interest to add OE to kernelci
> > as
> > well. Can you share some pointers on what needs to be done ?
> 
> The main tricky/annoying part with yocto is that builds are per-
> board,
> where as we like to build rootfs images per-arch.  Maybe there's a
> way
> to build an arch-generic yocto rootfs using a qemu target or
> something,
> though.
> 
> The place to start would be to look at the kci_rootfs tool in the
> kernelci-core repo[2] along with the rootfs-config.yaml file and
> figure
> out how to extend that for yocto/OE.
> 
> Again, Lakshmipathi may have some ideas there.
> 
> OTOH, I'm not quite sure what we would gain by using yocto instead of
> debian?  Are there more packages for yocto available than for debian
> currently?
> 

TBH, I am not quite sure. I think Khem proposed adding OE support
assuming that debian rootfs support may not be possible. If debootstrap
can be done from unstable tree, we may not required OE support.

@Alistair/Khem : Any thoughts ?

> > 3. How difficult is to setup kernelci from scratch locally ?
> 
> Not really, but it's not well documented. :(
> 
> But it depends on which parts you want to setup.  There are several
> moving parts:
> 
> 1) lab / board-farm
> 2) backend / storage
> 3) frontend
> 4) kernel builds (centralized, published publically to
>   storage.kernelci.org)
> 
> We have some rudimentary docs for 2-4 here[3], but the compute part
> for
> builds is currently centralized.
> 

Thanks for the links. I will go through them.

> Kevin
> 
> 
> [1] https://wiki.debian.org/RISC-V#debootstrap
> [2] https://github.com/kernelci/kernelci-core/
> [3] 
> https://github.com/kernelci/kernelci-doc/wiki/Setting-up-a-local-development-instance

-- 
Regards,
Atish

Re: Improve kernelci setup for RISC-V

[David Abdurachmanov]

Hi,

I am adding Karsten Merker, Aurelien Jarno and Manuel A. Fernandez who
are working on Debian RISCV.

Recap: KernelCI uses Debian debootstrap for their test infrastructure.
I believe they are using Buster for it now.

On Thu, Jul 2, 2020 at 10:02 AM Atish Patra <Atish.Patra@wdc.com> wrote:
>
> On Wed, 2020-07-01 at 17:17 -0700, Kevin Hilman wrote:
> > [ + kernelci list ]
> >
> > Hi Atish,
> >
> > Atish Patra <Atish.Patra@wdc.com> writes:
> >
> > > I just want to follow up the discussions we had in the risc-v slack
> > > channel related to improving kernelci setup for RISC-V.
> > >
> > > 1. Adding Qemu target for RISC-V: As per out latest conversation,
> > > this
> > > won't be an issue anymore as Qemu supports -bios and -kernel
> > > separately.
> >
> > Yes, this looks straight forward now.  We should be able to get this
> > going for our basic tests without too much more work.
> >
> > > 2. Running more tests in addition to boot tests on unleashed
> > > target:
> > >
> > > The main blocker here is the rootfs. As kernelci only supports
> > > debian
> > > based rootfs to actually run tests. I think it is not that
> > > difficult to
> > > bootstrap a debian rootfs for RISC-V but I have not done that in
> > > ages.
> > > I think palmer uses debian rootfs for his testing. I have cc'd him
> > > for
> > > his inputs.
> >
> > For every other arch, we debootstrap a buster rootfs, and this
> > doesn't
> > work for riscv.  But, I just discovered[1] that this should work for
> > unstable using debian-ports, so we'll need to update our rootfs
> > builder
> > to support unstable/debian-ports.
> >
> > I've cc'd Lakshmipathi from Collabora who has been doing work on our
> > rootfs builder tool to see how we can extend this.
> >
>
> Thanks.
>
> > > But Alistair & khem (cc'd) expressed interest to add OE to kernelci
> > > as
> > > well. Can you share some pointers on what needs to be done ?
> >
> > The main tricky/annoying part with yocto is that builds are per-
> > board,
> > where as we like to build rootfs images per-arch.  Maybe there's a
> > way
> > to build an arch-generic yocto rootfs using a qemu target or
> > something,
> > though.
> >
> > The place to start would be to look at the kci_rootfs tool in the
> > kernelci-core repo[2] along with the rootfs-config.yaml file and
> > figure
> > out how to extend that for yocto/OE.
> >
> > Again, Lakshmipathi may have some ideas there.
> >
> > OTOH, I'm not quite sure what we would gain by using yocto instead of
> > debian?  Are there more packages for yocto available than for debian
> > currently?
> >
>
> TBH, I am not quite sure. I think Khem proposed adding OE support
> assuming that debian rootfs support may not be possible. If debootstrap
> can be done from unstable tree, we may not required OE support.

If we want to test riscv32 you most likely need to go with OE. I don't
think anything else supports RISC-V 32-bit and I don't know if any
other distro (not targeting embedded market) will work on it.

I don't know if there will be a Debian riscv32 port.

Cheers,
david

>
> @Alistair/Khem : Any thoughts ?
>
> > > 3. How difficult is to setup kernelci from scratch locally ?
> >
> > Not really, but it's not well documented. :(
> >
> > But it depends on which parts you want to setup.  There are several
> > moving parts:
> >
> > 1) lab / board-farm
> > 2) backend / storage
> > 3) frontend
> > 4) kernel builds (centralized, published publically to
> >   storage.kernelci.org)
> >
> > We have some rudimentary docs for 2-4 here[3], but the compute part
> > for
> > builds is currently centralized.
> >
>
> Thanks for the links. I will go through them.
>
> > Kevin
> >
> >
> > [1] https://wiki.debian.org/RISC-V#debootstrap
> > [2] https://github.com/kernelci/kernelci-core/
> > [3]
> > https://github.com/kernelci/kernelci-doc/wiki/Setting-up-a-local-development-instance
>
> --
> Regards,
> Atish

Re: Improve kernelci setup for RISC-V

[David Abdurachmanov]

Almost forgot. I suggest we also run stress-ng for CI. It's a single
binary and it already found a number of kernel related issues on
Unleashed which usually result in board hanging.

I have added Colin from Canonical who is the author of the stress tool.

david

On Thu, Jul 2, 2020 at 10:12 AM David Abdurachmanov
<david.abdurachmanov@sifive.com> wrote:
>
> Hi,
>
> I am adding Karsten Merker, Aurelien Jarno and Manuel A. Fernandez who
> are working on Debian RISCV.
>
> Recap: KernelCI uses Debian debootstrap for their test infrastructure.
> I believe they are using Buster for it now.
>
> On Thu, Jul 2, 2020 at 10:02 AM Atish Patra <Atish.Patra@wdc.com> wrote:
> >
> > On Wed, 2020-07-01 at 17:17 -0700, Kevin Hilman wrote:
> > > [ + kernelci list ]
> > >
> > > Hi Atish,
> > >
> > > Atish Patra <Atish.Patra@wdc.com> writes:
> > >
> > > > I just want to follow up the discussions we had in the risc-v slack
> > > > channel related to improving kernelci setup for RISC-V.
> > > >
> > > > 1. Adding Qemu target for RISC-V: As per out latest conversation,
> > > > this
> > > > won't be an issue anymore as Qemu supports -bios and -kernel
> > > > separately.
> > >
> > > Yes, this looks straight forward now.  We should be able to get this
> > > going for our basic tests without too much more work.
> > >
> > > > 2. Running more tests in addition to boot tests on unleashed
> > > > target:
> > > >
> > > > The main blocker here is the rootfs. As kernelci only supports
> > > > debian
> > > > based rootfs to actually run tests. I think it is not that
> > > > difficult to
> > > > bootstrap a debian rootfs for RISC-V but I have not done that in
> > > > ages.
> > > > I think palmer uses debian rootfs for his testing. I have cc'd him
> > > > for
> > > > his inputs.
> > >
> > > For every other arch, we debootstrap a buster rootfs, and this
> > > doesn't
> > > work for riscv.  But, I just discovered[1] that this should work for
> > > unstable using debian-ports, so we'll need to update our rootfs
> > > builder
> > > to support unstable/debian-ports.
> > >
> > > I've cc'd Lakshmipathi from Collabora who has been doing work on our
> > > rootfs builder tool to see how we can extend this.
> > >
> >
> > Thanks.
> >
> > > > But Alistair & khem (cc'd) expressed interest to add OE to kernelci
> > > > as
> > > > well. Can you share some pointers on what needs to be done ?
> > >
> > > The main tricky/annoying part with yocto is that builds are per-
> > > board,
> > > where as we like to build rootfs images per-arch.  Maybe there's a
> > > way
> > > to build an arch-generic yocto rootfs using a qemu target or
> > > something,
> > > though.
> > >
> > > The place to start would be to look at the kci_rootfs tool in the
> > > kernelci-core repo[2] along with the rootfs-config.yaml file and
> > > figure
> > > out how to extend that for yocto/OE.
> > >
> > > Again, Lakshmipathi may have some ideas there.
> > >
> > > OTOH, I'm not quite sure what we would gain by using yocto instead of
> > > debian?  Are there more packages for yocto available than for debian
> > > currently?
> > >
> >
> > TBH, I am not quite sure. I think Khem proposed adding OE support
> > assuming that debian rootfs support may not be possible. If debootstrap
> > can be done from unstable tree, we may not required OE support.
>
> If we want to test riscv32 you most likely need to go with OE. I don't
> think anything else supports RISC-V 32-bit and I don't know if any
> other distro (not targeting embedded market) will work on it.
>
> I don't know if there will be a Debian riscv32 port.
>
> Cheers,
> david
>
> >
> > @Alistair/Khem : Any thoughts ?
> >
> > > > 3. How difficult is to setup kernelci from scratch locally ?
> > >
> > > Not really, but it's not well documented. :(
> > >
> > > But it depends on which parts you want to setup.  There are several
> > > moving parts:
> > >
> > > 1) lab / board-farm
> > > 2) backend / storage
> > > 3) frontend
> > > 4) kernel builds (centralized, published publically to
> > >   storage.kernelci.org)
> > >
> > > We have some rudimentary docs for 2-4 here[3], but the compute part
> > > for
> > > builds is currently centralized.
> > >
> >
> > Thanks for the links. I will go through them.
> >
> > > Kevin
> > >
> > >
> > > [1] https://wiki.debian.org/RISC-V#debootstrap
> > > [2] https://github.com/kernelci/kernelci-core/
> > > [3]
> > > https://github.com/kernelci/kernelci-doc/wiki/Setting-up-a-local-development-instance
> >
> > --
> > Regards,
> > Atish

Re: Improve kernelci setup for RISC-V

[Aurelien Jarno]

On 2020-07-02 16:19, David Abdurachmanov wrote:
> Almost forgot. I suggest we also run stress-ng for CI. It's a single
> binary and it already found a number of kernel related issues on
> Unleashed which usually result in board hanging.
> 
> I have added Colin from Canonical who is the author of the stress tool.

stress-ng is already available as a package in the riscv64 port, so it
should just be a matter of installing it:

https://packages.debian.org/unstable/stress-ng

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: Improve kernelci setup for RISC-V

[Colin Ian King]

On 02/07/2020 15:50, Aurelien Jarno wrote:
> On 2020-07-02 16:19, David Abdurachmanov wrote:
>> Almost forgot. I suggest we also run stress-ng for CI. It's a single
>> binary and it already found a number of kernel related issues on
>> Unleashed which usually result in board hanging.
>>
>> I have added Colin from Canonical who is the author of the stress tool.
> 
> stress-ng is already available as a package in the riscv64 port, so it
> should just be a matter of installing it:
> 
> https://packages.debian.org/unstable/stress-ng


.. I try to keep the debian package relatively fresh and close to the
development tip.  We use stress-ng for regression testing on Ubuntu, we
have some autotest wrapped tests that we regularly use:

Smoke tests - to see if we can break kernels:

https://kernel.ubuntu.com/git/ubuntu/autotest-client-tests.git/tree/ubuntu_stress_smoke_test

Performance regression tests:

https://kernel.ubuntu.com/git/ubuntu/autotest-client-tests.git/tree/ubuntu_performance_stress_ng

These may be a useful starting point for CI testing.

Colin

> 
> Aurelien
> 

Re: Improve kernelci setup for RISC-V

[Aurelien Jarno]

Hi,

On 2020-07-02 10:12, David Abdurachmanov wrote:
> Hi,
> 
> I am adding Karsten Merker, Aurelien Jarno and Manuel A. Fernandez who
> are working on Debian RISCV.
> 
> Recap: KernelCI uses Debian debootstrap for their test infrastructure.
> I believe they are using Buster for it now.
> 
> On Thu, Jul 2, 2020 at 10:02 AM Atish Patra <Atish.Patra@wdc.com> wrote:
> >
> > On Wed, 2020-07-01 at 17:17 -0700, Kevin Hilman wrote:
> > > [ + kernelci list ]
> > >
> > > Hi Atish,
> > >
> > > Atish Patra <Atish.Patra@wdc.com> writes:
> > >
> > > > I just want to follow up the discussions we had in the risc-v slack
> > > > channel related to improving kernelci setup for RISC-V.
> > > >
> > > > 1. Adding Qemu target for RISC-V: As per out latest conversation,
> > > > this
> > > > won't be an issue anymore as Qemu supports -bios and -kernel
> > > > separately.
> > >
> > > Yes, this looks straight forward now.  We should be able to get this
> > > going for our basic tests without too much more work.
> > >
> > > > 2. Running more tests in addition to boot tests on unleashed
> > > > target:
> > > >
> > > > The main blocker here is the rootfs. As kernelci only supports
> > > > debian
> > > > based rootfs to actually run tests. I think it is not that
> > > > difficult to
> > > > bootstrap a debian rootfs for RISC-V but I have not done that in
> > > > ages.
> > > > I think palmer uses debian rootfs for his testing. I have cc'd him
> > > > for
> > > > his inputs.
> > >
> > > For every other arch, we debootstrap a buster rootfs, and this
> > > doesn't
> > > work for riscv.  But, I just discovered[1] that this should work for
> > > unstable using debian-ports, so we'll need to update our rootfs
> > > builder
> > > to support unstable/debian-ports.
> > >
> > > I've cc'd Lakshmipathi from Collabora who has been doing work on our
> > > rootfs builder tool to see how we can extend this.
> > >

Yes, so far the riscv64 port of debian is only available for the sid
distribution. debootstrap should work fine in general (I have just
tested now), but it might break from time to time (I hope rarely). Do
not hesitate to contact us if it happens. In short just use
debian-ports/ in the URL instead of debian/, for example
https://deb.debian.org/debian-ports

Of course, due to the fact it's sid and not buster, the environment
might be slightly different. OTOH, it means you'll be ready for the next
release.

We really hope that the riscv64 port can become integrated to testing
after the next release of bullseye, we are mostly waiting the general
availability of hardware.

> > > > But Alistair & khem (cc'd) expressed interest to add OE to kernelci
> > > > as
> > > > well. Can you share some pointers on what needs to be done ?
> > >
> > > The main tricky/annoying part with yocto is that builds are per-
> > > board,
> > > where as we like to build rootfs images per-arch.  Maybe there's a
> > > way
> > > to build an arch-generic yocto rootfs using a qemu target or
> > > something,
> > > though.
> > >
> > > The place to start would be to look at the kci_rootfs tool in the
> > > kernelci-core repo[2] along with the rootfs-config.yaml file and
> > > figure
> > > out how to extend that for yocto/OE.
> > >
> > > Again, Lakshmipathi may have some ideas there.
> > >
> > > OTOH, I'm not quite sure what we would gain by using yocto instead of
> > > debian?  Are there more packages for yocto available than for debian
> > > currently?
> > >
> >
> > TBH, I am not quite sure. I think Khem proposed adding OE support
> > assuming that debian rootfs support may not be possible. If debootstrap
> > can be done from unstable tree, we may not required OE support.
> 
> If we want to test riscv32 you most likely need to go with OE. I don't
> think anything else supports RISC-V 32-bit and I don't know if any
> other distro (not targeting embedded market) will work on it.
> 
> I don't know if there will be a Debian riscv32 port.

There is currently no real plan for that, although I know there has been
some discussion about that recently. However, even if it get created it
won't be an official port, Debian will not add new 32-bit official
ports.

Aurelien

-- 
Aurelien Jarno                          GPG: 4096R/1DDD8C9B
aurelien@aurel32.net                 http://www.aurel32.net

Re: Improve kernelci setup for RISC-V

[Karsten Merker]

On Thu, Jul 02, 2020 at 04:49:55PM +0200, Aurelien Jarno wrote:
> On 2020-07-02 10:12, David Abdurachmanov wrote:
[...]
> > Recap: KernelCI uses Debian debootstrap for their test infrastructure.
> > I believe they are using Buster for it now.
> > 
> > On Thu, Jul 2, 2020 at 10:02 AM Atish Patra <Atish.Patra@wdc.com> wrote:
> > > On Wed, 2020-07-01 at 17:17 -0700, Kevin Hilman wrote:
> > > > Atish Patra <Atish.Patra@wdc.com> writes:
[...]
> > > > > The main blocker here is the rootfs.  As kernelci only
> > > > > supports debian based rootfs to actually run tests.  I think
> > > > > it is not that difficult to bootstrap a debian rootfs for
> > > > > RISC-V but I have not done that in ages.  I think palmer uses
> > > > > debian rootfs for his testing.  I have cc'd him for his
> > > > > inputs.
> > > >
> > > > For every other arch, we debootstrap a buster rootfs, and this
> > > > doesn't work for riscv.  But, I just discovered[1] that this
> > > > should work for unstable using debian-ports, so we'll need to
> > > > update our rootfs builder to support unstable/debian-ports.
> > > >
> > > > I've cc'd Lakshmipathi from Collabora who has been doing work
> > > > on our rootfs builder tool to see how we can extend this.

> Yes, so far the riscv64 port of debian is only available for the sid
> distribution.  debootstrap should work fine in general (I have just
> tested now), but it might break from time to time (I hope rarely). 
> Do not hesitate to contact us if it happens.  In short just use
> debian-ports/ in the URL instead of debian/, for example
> https://deb.debian.org/debian-ports

Hello,

I don't know how strongly the kernelci infrastructure is bound to using
debootstrap for creating a Debian rootfs, but if you need to make
adjustments anyway for using the debian-ports archive, using mmdebstrap
instead of debootstrap for debian-ports architectures such as riscv64
might be worth a thought.

There are still some riscv64 packages that aren't in the "unstable"
suite but only in the "unreleased" suite and debootstrap can only pull
packages from a single suite, which means that one has to manually add
a sources.list entry for the "unreleased" suite to a chroot created by
debootstrap if one wants to be able to install packages that have
dependencies in the "unreleased" suite.

In contrast to debootstrap, mmdebstrap can handle multiple suites
without problems and automatically takes care of adding the
corresponding sources.list entries in the chroot.  The capability of
creating a chroot from multiple suites makes it also more resilient to
cases where one of the base packages temporarily has to move from
"unstable" to "unreleased".  The latter hasn't happened for a long
time, but it could in principle happen again - when that happens it is
usually only a temporary issue, but using mmdebstrap instead of
debootstrap completely avoids that becoming a problem at all.

Unfortunately there are certain differences between the debootstrap and
the mmdebstrap commandline parameters, so mmdebstrap is not a direct
1:1 replacement for debootstrap, but the two are similar enough that
this shouldn't pose a serious problem.

Examples of how to run debootstrap and mmdebstrap for a debian-ports
architecture are available at

  https://wiki.debian.org/RISC-V#Creating_a_riscv64_chroot

Regards,
Karsten
-- 
Hiermit widerspreche ich ausdrücklich der Nutzung sowie der Weitergabe
meiner personenbezogenen Daten für Zwecke der Werbung sowie der Markt-
oder Meinungsforschung.

Re: Improve kernelci setup for RISC-V

[Kevin Hilman]

Karsten Merker <merker@debian.org> writes:

> On Thu, Jul 02, 2020 at 04:49:55PM +0200, Aurelien Jarno wrote:
>> On 2020-07-02 10:12, David Abdurachmanov wrote:
> [...]
>> > Recap: KernelCI uses Debian debootstrap for their test infrastructure.
>> > I believe they are using Buster for it now.
>> > 
>> > On Thu, Jul 2, 2020 at 10:02 AM Atish Patra <Atish.Patra@wdc.com> wrote:
>> > > On Wed, 2020-07-01 at 17:17 -0700, Kevin Hilman wrote:
>> > > > Atish Patra <Atish.Patra@wdc.com> writes:
> [...]
>> > > > > The main blocker here is the rootfs.  As kernelci only
>> > > > > supports debian based rootfs to actually run tests.  I think
>> > > > > it is not that difficult to bootstrap a debian rootfs for
>> > > > > RISC-V but I have not done that in ages.  I think palmer uses
>> > > > > debian rootfs for his testing.  I have cc'd him for his
>> > > > > inputs.
>> > > >
>> > > > For every other arch, we debootstrap a buster rootfs, and this
>> > > > doesn't work for riscv.  But, I just discovered[1] that this
>> > > > should work for unstable using debian-ports, so we'll need to
>> > > > update our rootfs builder to support unstable/debian-ports.
>> > > >
>> > > > I've cc'd Lakshmipathi from Collabora who has been doing work
>> > > > on our rootfs builder tool to see how we can extend this.
>
>> Yes, so far the riscv64 port of debian is only available for the sid
>> distribution.  debootstrap should work fine in general (I have just
>> tested now), but it might break from time to time (I hope rarely). 
>> Do not hesitate to contact us if it happens.  In short just use
>> debian-ports/ in the URL instead of debian/, for example
>> https://deb.debian.org/debian-ports
>
> Hello,
>
> I don't know how strongly the kernelci infrastructure is bound to using
> debootstrap for creating a Debian rootfs, but if you need to make
> adjustments anyway for using the debian-ports archive, using mmdebstrap
> instead of debootstrap for debian-ports architectures such as riscv64
> might be worth a thought.

We're currently relying on the debos tool[1] which itself uses
debootstrap under the hood.  I've cc'd some developers of debos to see
it they have any thoughts about using mmdepbstrap instead.

Kevin

[1] https://github.com/go-debos/debos

Re: Improve kernelci setup for RISC-V

[Manuel A. Fernandez Montecelo]

Hi,

Em qui., 2 de jul. de 2020 às 20:21, Karsten Merker
<merker@debian.org> escreveu:
> [...]
> In contrast to debootstrap, mmdebstrap can handle multiple suites
> without problems and automatically takes care of adding the
> corresponding sources.list entries in the chroot.  The capability of
> creating a chroot from multiple suites makes it also more resilient to
> cases where one of the base packages temporarily has to move from
> "unstable" to "unreleased".  The latter hasn't happened for a long
> time, but it could in principle happen again - when that happens it is
> usually only a temporary issue, but using mmdebstrap instead of
> debootstrap completely avoids that becoming a problem at all.
>
> Unfortunately there are certain differences between the debootstrap and
> the mmdebstrap commandline parameters, so mmdebstrap is not a direct
> 1:1 replacement for debootstrap, but the two are similar enough that
> this shouldn't pose a serious problem.

Yeah, I think that it depends on the effort.  I'd suggest to start
with the current set-up of debos with debootstrap underneath, it
should work for now and most of the time, and if needed or deemed
useful, add support for mmdebstrap.

Great to see this happening BTW, thanks all working on it for the effort :-)

-- 
Manuel A. Fernandez Montecelo <manuel.montezelo@gmail.com>

Re: Improve kernelci setup for RISC-V

[Manuel A. Fernandez Montecelo]

Hi again,

There's this recipe for debos from Ana, who was working on kernelci
some time ago, using sid (unstable):

  https://github.com/ana/riscv64-image

Maybe it is of some help to set things up.


Cheers.
-- 
Manuel A. Fernandez Montecelo <manuel.montezelo@gmail.com>

Re: Improve kernelci setup for RISC-V

[Lakshmipathi Ganapathi]

Hi,

Here is the riscv64 patch for kci_rootfs created by khilman (https://github.com/kernelci/kernelci-core/pull/414) and the images can be found here: https://storage.staging.kernelci.org/images/rootfs/debian/sid/2020070601/riscv64/ thanks!

Cheers.
Lakshmipathi.G

On Thursday, July 09, 2020 05:01 IST, "Manuel A. Fernandez Montecelo" <manuel.montezelo@gmail.com> wrote:
 Hi again,

There's this recipe for debos from Ana, who was working on kernelci
some time ago, using sid (unstable):

https://github.com/ana/riscv64-image

Maybe it is of some help to set things up.


Cheers.
--
Manuel A. Fernandez Montecelo <manuel.montezelo@gmail.com>


 

Re: Improve kernelci setup for RISC-V

[Alistair Francis]

On Thu, 2020-07-02 at 10:12 +0300, David Abdurachmanov wrote:
> Hi,
> 
> I am adding Karsten Merker, Aurelien Jarno and Manuel A. Fernandez
> who
> are working on Debian RISCV.
> 
> Recap: KernelCI uses Debian debootstrap for their test
> infrastructure.
> I believe they are using Buster for it now.
> 
> On Thu, Jul 2, 2020 at 10:02 AM Atish Patra <Atish.Patra@wdc.com>
> wrote:
> > On Wed, 2020-07-01 at 17:17 -0700, Kevin Hilman wrote:
> > > [ + kernelci list ]
> > > 
> > > Hi Atish,
> > > 
> > > Atish Patra <Atish.Patra@wdc.com> writes:
> > > 
> > > > I just want to follow up the discussions we had in the risc-v
> > > > slack
> > > > channel related to improving kernelci setup for RISC-V.
> > > > 
> > > > 1. Adding Qemu target for RISC-V: As per out latest
> > > > conversation,
> > > > this
> > > > won't be an issue anymore as Qemu supports -bios and -kernel
> > > > separately.
> > > 
> > > Yes, this looks straight forward now.  We should be able to get
> > > this
> > > going for our basic tests without too much more work.
> > > 
> > > > 2. Running more tests in addition to boot tests on unleashed
> > > > target:
> > > > 
> > > > The main blocker here is the rootfs. As kernelci only supports
> > > > debian
> > > > based rootfs to actually run tests. I think it is not that
> > > > difficult to
> > > > bootstrap a debian rootfs for RISC-V but I have not done that
> > > > in
> > > > ages.
> > > > I think palmer uses debian rootfs for his testing. I have cc'd
> > > > him
> > > > for
> > > > his inputs.
> > > 
> > > For every other arch, we debootstrap a buster rootfs, and this
> > > doesn't
> > > work for riscv.  But, I just discovered[1] that this should work
> > > for
> > > unstable using debian-ports, so we'll need to update our rootfs
> > > builder
> > > to support unstable/debian-ports.
> > > 
> > > I've cc'd Lakshmipathi from Collabora who has been doing work on
> > > our
> > > rootfs builder tool to see how we can extend this.
> > > 
> > 
> > Thanks.
> > 
> > > > But Alistair & khem (cc'd) expressed interest to add OE to
> > > > kernelci
> > > > as
> > > > well. Can you share some pointers on what needs to be done ?
> > > 
> > > The main tricky/annoying part with yocto is that builds are per-
> > > board,
> > > where as we like to build rootfs images per-arch.  Maybe there's
> > > a
> > > way
> > > to build an arch-generic yocto rootfs using a qemu target or
> > > something,
> > > though.
> > > 
> > > The place to start would be to look at the kci_rootfs tool in the
> > > kernelci-core repo[2] along with the rootfs-config.yaml file and
> > > figure
> > > out how to extend that for yocto/OE.
> > > 
> > > Again, Lakshmipathi may have some ideas there.
> > > 
> > > OTOH, I'm not quite sure what we would gain by using yocto
> > > instead of
> > > debian?  Are there more packages for yocto available than for
> > > debian
> > > currently?
> > > 
> > 
> > TBH, I am not quite sure. I think Khem proposed adding OE support
> > assuming that debian rootfs support may not be possible. If
> > debootstrap
> > can be done from unstable tree, we may not required OE support.
> 
> If we want to test riscv32 you most likely need to go with OE. I
> don't
> think anything else supports RISC-V 32-bit and I don't know if any
> other distro (not targeting embedded market) will work on it.

I think for a RV32 test, OE is probably the best way to go. That should
come after we have a 64-bit test running and upstream support for RV32
in glibc though.

So let's get RV64 up and running first.

Alistair

> 
> I don't know if there will be a Debian riscv32 port.
> 
> Cheers,
> david
> 
> > @Alistair/Khem : Any thoughts ?
> > 
> > > > 3. How difficult is to setup kernelci from scratch locally ?
> > > 
> > > Not really, but it's not well documented. :(
> > > 
> > > But it depends on which parts you want to setup.  There are
> > > several
> > > moving parts:
> > > 
> > > 1) lab / board-farm
> > > 2) backend / storage
> > > 3) frontend
> > > 4) kernel builds (centralized, published publically to
> > >   storage.kernelci.org)
> > > 
> > > We have some rudimentary docs for 2-4 here[3], but the compute
> > > part
> > > for
> > > builds is currently centralized.
> > > 
> > 
> > Thanks for the links. I will go through them.
> > 
> > > Kevin
> > > 
> > > 
> > > [1] https://wiki.debian.org/RISC-V#debootstrap
> > > [2] https://github.com/kernelci/kernelci-core/
> > > [3]
> > > https://github.com/kernelci/kernelci-doc/wiki/Setting-up-a-local-development-instance
> > 
> > --
> > Regards,
> > Atish

Re: Improve kernelci setup for RISC-V

[Khem Raj]

On Thu, Jul 2, 2020 at 12:02 AM Atish Patra <Atish.Patra@wdc.com> wrote:
>
> On Wed, 2020-07-01 at 17:17 -0700, Kevin Hilman wrote:
> > [ + kernelci list ]
> >
> > Hi Atish,
> >
> > Atish Patra <Atish.Patra@wdc.com> writes:
> >
> > > I just want to follow up the discussions we had in the risc-v slack
> > > channel related to improving kernelci setup for RISC-V.
> > >
> > > 1. Adding Qemu target for RISC-V: As per out latest conversation,
> > > this
> > > won't be an issue anymore as Qemu supports -bios and -kernel
> > > separately.
> >
> > Yes, this looks straight forward now.  We should be able to get this
> > going for our basic tests without too much more work.
> >
> > > 2. Running more tests in addition to boot tests on unleashed
> > > target:
> > >
> > > The main blocker here is the rootfs. As kernelci only supports
> > > debian
> > > based rootfs to actually run tests. I think it is not that
> > > difficult to
> > > bootstrap a debian rootfs for RISC-V but I have not done that in
> > > ages.
> > > I think palmer uses debian rootfs for his testing. I have cc'd him
> > > for
> > > his inputs.
> >
> > For every other arch, we debootstrap a buster rootfs, and this
> > doesn't
> > work for riscv.  But, I just discovered[1] that this should work for
> > unstable using debian-ports, so we'll need to update our rootfs
> > builder
> > to support unstable/debian-ports.
> >
> > I've cc'd Lakshmipathi from Collabora who has been doing work on our
> > rootfs builder tool to see how we can extend this.
> >
>
> Thanks.
>
> > > But Alistair & khem (cc'd) expressed interest to add OE to kernelci
> > > as
> > > well. Can you share some pointers on what needs to be done ?
> >
> > The main tricky/annoying part with yocto is that builds are per-
> > board,
> > where as we like to build rootfs images per-arch.  Maybe there's a
> > way
> > to build an arch-generic yocto rootfs using a qemu target or
> > something,
> > though.
> >
> > The place to start would be to look at the kci_rootfs tool in the
> > kernelci-core repo[2] along with the rootfs-config.yaml file and
> > figure
> > out how to extend that for yocto/OE.
> >
> > Again, Lakshmipathi may have some ideas there.
> >
> > OTOH, I'm not quite sure what we would gain by using yocto instead of
> > debian?  Are there more packages for yocto available than for debian
> > currently?
> >
>
> TBH, I am not quite sure. I think Khem proposed adding OE support
> assuming that debian rootfs support may not be possible. If debootstrap
> can be done from unstable tree, we may not required OE support.
>
> @Alistair/Khem : Any thoughts ?
>

I think building a rootfs from kernel headers extracted from the
kernel under test does
add to test vectors, and OE/Yocto built rootfs can help in that area,
besides that testing
an existing rootfs with different kernel underneath tests another
vector and existing mechanisms
are good with that.

> > > 3. How difficult is to setup kernelci from scratch locally ?
> >
> > Not really, but it's not well documented. :(
> >
> > But it depends on which parts you want to setup.  There are several
> > moving parts:
> >
> > 1) lab / board-farm
> > 2) backend / storage
> > 3) frontend
> > 4) kernel builds (centralized, published publically to
> >   storage.kernelci.org)
> >
> > We have some rudimentary docs for 2-4 here[3], but the compute part
> > for
> > builds is currently centralized.
> >
>
> Thanks for the links. I will go through them.
>
> > Kevin
> >
> >
> > [1] https://wiki.debian.org/RISC-V#debootstrap
> > [2] https://github.com/kernelci/kernelci-core/
> > [3]
> > https://github.com/kernelci/kernelci-doc/wiki/Setting-up-a-local-development-instance
>
> --
> Regards,
> Atish

Re: Improve kernelci setup for RISC-V

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1151 bytes --]

On Wed, Jul 01, 2020 at 05:17:46PM -0700, Kevin Hilman wrote:
> Atish Patra <Atish.Patra@wdc.com> writes:

> > But Alistair & khem (cc'd) expressed interest to add OE to kernelci as
> > well. Can you share some pointers on what needs to be done ?

> The main tricky/annoying part with yocto is that builds are per-board,
> where as we like to build rootfs images per-arch.  Maybe there's a way
> to build an arch-generic yocto rootfs using a qemu target or something,
> though.

That definitely works well, I'm doing a lot of my testing with a setup
like that - just ignore the kernel that yocto builds and use the rest of
the rootfs.  Once you start to get further up the stack it can be an
issue but for most things KernelCI is covering now it's fine.  The only
gotcha I'm aware of is if there's architecture variants that are binary
incompatible (like moving to v7 on arm) you need to pick one that's
suitable for everything that'll use the image.

> OTOH, I'm not quite sure what we would gain by using yocto instead of
> debian?  Are there more packages for yocto available than for debian
> currently?

I'd expect it's the other way around TBH.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]