CIP-dev Archive on lore.kernel.org
 help / color / Atom feed
* [cip-dev] Kselftest implementation
@ 2021-04-13 17:29 Alice Ferrazzi
  2021-04-13 22:27 ` Chris Paterson
  0 siblings, 1 reply; 5+ messages in thread
From: Alice Ferrazzi @ 2021-04-13 17:29 UTC (permalink / raw)
  To: cip-dev

[-- Attachment #1.1: Type: text/plain, Size: 1294 bytes --]

Hi everyone,

I'm Alice Ferrazzi from Cybertrust Japan.
I'm currently working on adding Kselftest to the current CIP CI tests of
linux-cip Kernel

I made and tested kselftest on a testing instance of lava, currently tested
only on qemu x86_64
I implemented a simple skiplist in the kselftest envelope for skipping
tests that are currently not working.
The usage is similar to how spectre-meltdown-checker-test is currently
working.

Raw log:
http://140.211.166.173:10080/scheduler/job/329
Results list:
http://140.211.166.173:10080/results/329/1_kselftest

My idea is to start implementing kselftest tests for at least qemu x86_64
architecture and expand in the future to other architectures.
As Iwamatsu-san pointed, we need also to create kselftest binary.
I'm currently thinking of implementing kselftest binary creation mechanism
on linux-cip-ci kernel build script but I still didn't start to work on
this yet.

Kselftest test code:
https://gitlab.com/alice.ferrazzi/cip-kselftest

linux-cip Kernel Kselftest implementation (currently in draft):
https://gitlab.com/cip-project/cip-testing/linux-cip-ci/-/merge_requests

Thanks,
Alice
-- 
======================================
Cybertrust Japan Co.,Ltd.
Alice Ferrazzi
alice.ferrazzi@miraclelinux.com
======================================

[-- Attachment #1.2: Type: text/html, Size: 3480 bytes --]

[-- Attachment #2: Type: text/plain, Size: 428 bytes --]


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#6374): https://lists.cip-project.org/g/cip-dev/message/6374
Mute This Topic: https://lists.cip-project.org/mt/82071182/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/4520388/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [cip-dev] Kselftest implementation
  2021-04-13 17:29 [cip-dev] Kselftest implementation Alice Ferrazzi
@ 2021-04-13 22:27 ` Chris Paterson
  2021-04-15  2:04   ` Alice Ferrazzi
  0 siblings, 1 reply; 5+ messages in thread
From: Chris Paterson @ 2021-04-13 22:27 UTC (permalink / raw)
  To: cip-dev, alice.ferrazzi


[-- Attachment #1: Type: text/plain, Size: 2513 bytes --]

Hello Alice,

> From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On
> Behalf Of Alice Ferrazzi via lists.cip-project.org
> Sent: 13 April 2021 18:30
> 
> Hi everyone,
> 
> I'm Alice Ferrazzi from Cybertrust Japan.
> 
> I'm currently working on adding Kselftest to the current CIP CI tests of linux-
> cip Kernel

Thank you for your work on this!

> 
> I made and tested kselftest on a testing instance of lava, currently tested
> only on qemu x86_64

If you'd like to test on a wider range of platforms then get in touch and we'll get you access to the CIP LAVA instance.

> I implemented a simple skiplist in the kselftest envelope for skipping tests
> that are currently not working.
> The usage is similar to how spectre-meltdown-checker-test is currently
> working.
> 
> Raw log:
> 
> http://140.211.166.173:10080/scheduler/job/329
> Results list:
> http://140.211.166.173:10080/results/329/1_kselftest

It looks like a great start.

>
> 
> My idea is to start implementing kselftest tests for at least qemu x86_64
> architecture and expand in the future to other architectures.
> As Iwamatsu-san pointed, we need also to create kselftest binary.
> 
> I'm currently thinking of implementing kselftest binary creation mechanism
> on linux-cip-ci kernel build script but I still didn't start to work on this yet.
> 
> Kselftest test code:
> 
> https://gitlab.com/alice.ferrazzi/cip-kselftest

Is there any work we can leverage from the Linaro test-definitions repo?
https://github.com/Linaro/test-definitions/tree/master/automated/linux/kselftest

KernelCI also recently implemented kselftest support if that's useful at all, this'll include the build side of things.
https://github.com/kernelci/kernelci-core/tree/main/config/lava/kselftest

>
> 
> linux-cip Kernel Kselftest implementation (currently in draft):
> https://gitlab.com/cip-project/cip-testing/linux-cip-ci/-/merge_requests

Please assign the MR to me when you're ready for it to be reviewed, or if you need any help.

Also let me know if you want write access to the linux-cip-ci repo.
If you push your branch to our repo directly the CI will be able to run, making it easier for you to test.

Kind regards, Chris

>
> 
> Thanks,
> Alice
> --
> 
> ======================================
> Cybertrust Japan Co.,Ltd.
> Alice Ferrazzi
> alice.ferrazzi@miraclelinux.com <mailto:alice.ferrazzi@miraclelinux.com>
> ======================================


[-- Attachment #2: Type: text/plain, Size: 428 bytes --]


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#6375): https://lists.cip-project.org/g/cip-dev/message/6375
Mute This Topic: https://lists.cip-project.org/mt/82071182/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/4520388/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [cip-dev] Kselftest implementation
  2021-04-13 22:27 ` Chris Paterson
@ 2021-04-15  2:04   ` Alice Ferrazzi
  2021-04-22  7:34     ` Nobuhiro Iwamatsu
  0 siblings, 1 reply; 5+ messages in thread
From: Alice Ferrazzi @ 2021-04-15  2:04 UTC (permalink / raw)
  To: Chris Paterson; +Cc: cip-dev

[-- Attachment #1.1: Type: text/plain, Size: 3285 bytes --]

Hello Chris,

On Wed, Apr 14, 2021 at 7:27 AM Chris Paterson <Chris.Paterson2@renesas.com>
wrote:

> Hello Alice,
>
> > From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On
> > Behalf Of Alice Ferrazzi via lists.cip-project.org
> > Sent: 13 April 2021 18:30
> >
> > Hi everyone,
> >
> > I'm Alice Ferrazzi from Cybertrust Japan.
> >
> > I'm currently working on adding Kselftest to the current CIP CI tests of
> linux-
> > cip Kernel
>
> Thank you for your work on this!
>
> >
> > I made and tested kselftest on a testing instance of lava, currently
> tested
> > only on qemu x86_64
>
> If you'd like to test on a wider range of platforms then get in touch and
> we'll get you access to the CIP LAVA instance.
>

This would be nice!


> > I implemented a simple skiplist in the kselftest envelope for skipping
> tests
> > that are currently not working.
> > The usage is similar to how spectre-meltdown-checker-test is currently
> > working.
> >
> > Raw log:
> >
> > http://140.211.166.173:10080/scheduler/job/329
> > Results list:
> > http://140.211.166.173:10080/results/329/1_kselftest
>
> It looks like a great start.
>
>
Thanks

>
> >
> > My idea is to start implementing kselftest tests for at least qemu x86_64
> > architecture and expand in the future to other architectures.
> > As Iwamatsu-san pointed, we need also to create kselftest binary.
> >
> > I'm currently thinking of implementing kselftest binary creation
> mechanism
> > on linux-cip-ci kernel build script but I still didn't start to work on
> this yet.
> >
> > Kselftest test code:
> >
> > https://gitlab.com/alice.ferrazzi/cip-kselftest
>
>
Is there any work we can leverage from the Linaro test-definitions repo?
>
> https://github.com/Linaro/test-definitions/tree/master/automated/linux/kselftest
>
> That was the first thing I tried out but was not working correctly on the
cip rootfs enviroment


> KernelCI also recently implemented kselftest support if that's useful at
> all, this'll include the build side of things.
> https://github.com/kernelci/kernelci-core/tree/main/config/lava/kselftest
>
>
I know about KernelCI effort for kselftest and I think to use a similar
approach about the kselftest binary creation implementation.

The code I wrote is based on Glava-tests that is part of GKernelCI.
GKernelCI is the current Gentoo kernel CI system that is reporting and
collaborating with KernelCI.
https://github.com/GKernelCI/Glava-tests
Using a different repository currently is helpful for self managing the
skiplist, but it could be changed in the future.

>
> >
> > linux-cip Kernel Kselftest implementation (currently in draft):
> > https://gitlab.com/cip-project/cip-testing/linux-cip-ci/-/merge_requests
>
> Please assign the MR to me when you're ready for it to be reviewed, or if
> you need any help.
>
> Also let me know if you want write access to the linux-cip-ci repo.
> If you push your branch to our repo directly the CI will be able to run,
> making it easier for you to test.
>

Testing the kselftest branch directly on CIP CI test environment from
linux-cip-ci
repo would be really helpful.

Thanks,
Alicef


-- 
======================================
Cybertrust Japan Co.,Ltd.
Alice Ferrazzi
alice.ferrazzi@miraclelinux.com
======================================

[-- Attachment #1.2: Type: text/html, Size: 7000 bytes --]

[-- Attachment #2: Type: text/plain, Size: 428 bytes --]


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#6378): https://lists.cip-project.org/g/cip-dev/message/6378
Mute This Topic: https://lists.cip-project.org/mt/82071182/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/4520388/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [cip-dev] Kselftest implementation
  2021-04-15  2:04   ` Alice Ferrazzi
@ 2021-04-22  7:34     ` Nobuhiro Iwamatsu
  2021-04-23  8:49       ` Alice Ferrazzi
  0 siblings, 1 reply; 5+ messages in thread
From: Nobuhiro Iwamatsu @ 2021-04-22  7:34 UTC (permalink / raw)
  To: cip-dev; +Cc: Chris Paterson


[-- Attachment #1: Type: text/plain, Size: 4122 bytes --]

Hi,

Thanks for your work.

On Thu, Apr 15, 2021 at 11:04:26AM +0900, Alice Ferrazzi wrote:
> Hello Chris,
> 
> On Wed, Apr 14, 2021 at 7:27 AM Chris Paterson <Chris.Paterson2@renesas.com>
> wrote:
> 
> > Hello Alice,
> >
> > > From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org> On
> > > Behalf Of Alice Ferrazzi via lists.cip-project.org
> > > Sent: 13 April 2021 18:30
> > >
> > > Hi everyone,
> > >
> > > I'm Alice Ferrazzi from Cybertrust Japan.
> > >
> > > I'm currently working on adding Kselftest to the current CIP CI tests of
> > linux-
> > > cip Kernel
> >
> > Thank you for your work on this!
> >
> > >
> > > I made and tested kselftest on a testing instance of lava, currently
> > tested
> > > only on qemu x86_64
> >
> > If you'd like to test on a wider range of platforms then get in touch and
> > we'll get you access to the CIP LAVA instance.
> >
> 
> This would be nice!
> 
> 
> > > I implemented a simple skiplist in the kselftest envelope for skipping
> > tests
> > > that are currently not working.
> > > The usage is similar to how spectre-meltdown-checker-test is currently
> > > working.
> > >
> > > Raw log:
> > >
> > > http://140.211.166.173:10080/scheduler/job/329
> > > Results list:
> > > http://140.211.166.173:10080/results/329/1_kselftest
> >
> > It looks like a great start.
> >
> >
> Thanks
> 
> >
> > >
> > > My idea is to start implementing kselftest tests for at least qemu x86_64
> > > architecture and expand in the future to other architectures.
> > > As Iwamatsu-san pointed, we need also to create kselftest binary.
> > >
> > > I'm currently thinking of implementing kselftest binary creation
> > mechanism
> > > on linux-cip-ci kernel build script but I still didn't start to work on
> > this yet.
> > >
> > > Kselftest test code:
> > >
> > > https://gitlab.com/alice.ferrazzi/cip-kselftest
> >
> >
> Is there any work we can leverage from the Linaro test-definitions repo?
> >
> > https://github.com/Linaro/test-definitions/tree/master/automated/linux/kselftest
> >
> > That was the first thing I tried out but was not working correctly on the
> cip rootfs enviroment

As you write, cip rootfs lacks the software to run kselftest. Therefore,
it is necessary to make a separate flavor of rootfs.
There are also two types of cip rootfs, and we need to use either or
both for kernel testing.

> 
> 
> > KernelCI also recently implemented kselftest support if that's useful at
> > all, this'll include the build side of things.
> > https://github.com/kernelci/kernelci-core/tree/main/config/lava/kselftest
> >
> >
> I know about KernelCI effort for kselftest and I think to use a similar
> approach about the kselftest binary creation implementation.
> 
> The code I wrote is based on Glava-tests that is part of GKernelCI.
> GKernelCI is the current Gentoo kernel CI system that is reporting and
> collaborating with KernelCI.
> https://github.com/GKernelCI/Glava-tests
> Using a different repository currently is helpful for self managing the
> skiplist, but it could be changed in the future.


I think this needs to be done first.
I also think there are several approaches to creating kselftest
binaries.

- Continue to use a particular kernel version of ksefltests.
- Use CIP kernel kselftest.
- Use the latest kernel kselftest.

I used to build an environment using the latter.

https://gitlab.com/cip-project/cip-kernel/linux-cip/-/tree/ci/iwamatsu/linux-cip-kselftest

By the way, do you know which kernelci they are using?

> 
> >
> > >
> > > linux-cip Kernel Kselftest implementation (currently in draft):
> > > https://gitlab.com/cip-project/cip-testing/linux-cip-ci/-/merge_requests
> >
> > Please assign the MR to me when you're ready for it to be reviewed, or if
> > you need any help.
> >
> > Also let me know if you want write access to the linux-cip-ci repo.
> > If you push your branch to our repo directly the CI will be able to run,
> > making it easier for you to test.
> >
> 
> Testing the kselftest branch directly on CIP CI test environment from
> linux-cip-ci
> repo would be really helpful.
> 
> Thanks,
> Alicef


Best regards,
  Nobuhiro

[-- Attachment #2: Type: text/plain, Size: 428 bytes --]


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#6383): https://lists.cip-project.org/g/cip-dev/message/6383
Mute This Topic: https://lists.cip-project.org/mt/82071182/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/4520388/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-


^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [cip-dev] Kselftest implementation
  2021-04-22  7:34     ` Nobuhiro Iwamatsu
@ 2021-04-23  8:49       ` Alice Ferrazzi
  0 siblings, 0 replies; 5+ messages in thread
From: Alice Ferrazzi @ 2021-04-23  8:49 UTC (permalink / raw)
  To: cip-dev

[-- Attachment #1.1: Type: text/plain, Size: 4632 bytes --]

Hi,

thanks for the review,


On Thu, Apr 22, 2021 at 4:35 PM Nobuhiro Iwamatsu <
nobuhiro1.iwamatsu@toshiba.co.jp> wrote:

> Hi,
>
> Thanks for your work.
>
> On Thu, Apr 15, 2021 at 11:04:26AM +0900, Alice Ferrazzi wrote:
> > Hello Chris,
> >
> > On Wed, Apr 14, 2021 at 7:27 AM Chris Paterson <
> Chris.Paterson2@renesas.com>
> > wrote:
> >
> > > Hello Alice,
> > >
> > > > From: cip-dev@lists.cip-project.org <cip-dev@lists.cip-project.org>
> On
> > > > Behalf Of Alice Ferrazzi via lists.cip-project.org
> > > > Sent: 13 April 2021 18:30
> > > >
> > > > Hi everyone,
> > > >
> > > > I'm Alice Ferrazzi from Cybertrust Japan.
> > > >
> > > > I'm currently working on adding Kselftest to the current CIP CI
> tests of
> > > linux-
> > > > cip Kernel
> > >
> > > Thank you for your work on this!
> > >
> > > >
> > > > I made and tested kselftest on a testing instance of lava, currently
> > > tested
> > > > only on qemu x86_64
> > >
> > > If you'd like to test on a wider range of platforms then get in touch
> and
> > > we'll get you access to the CIP LAVA instance.
> > >
> >
> > This would be nice!
> >
> >
> > > > I implemented a simple skiplist in the kselftest envelope for
> skipping
> > > tests
> > > > that are currently not working.
> > > > The usage is similar to how spectre-meltdown-checker-test is
> currently
> > > > working.
> > > >
> > > > Raw log:
> > > >
> > > > http://140.211.166.173:10080/scheduler/job/329
> > > > Results list:
> > > > http://140.211.166.173:10080/results/329/1_kselftest
> > >
> > > It looks like a great start.
> > >
> > >
> > Thanks
> >
> > >
> > > >
> > > > My idea is to start implementing kselftest tests for at least qemu
> x86_64
> > > > architecture and expand in the future to other architectures.
> > > > As Iwamatsu-san pointed, we need also to create kselftest binary.
> > > >
> > > > I'm currently thinking of implementing kselftest binary creation
> > > mechanism
> > > > on linux-cip-ci kernel build script but I still didn't start to work
> on
> > > this yet.
> > > >
> > > > Kselftest test code:
> > > >
> > > > https://gitlab.com/alice.ferrazzi/cip-kselftest
> > >
> > >
> > Is there any work we can leverage from the Linaro test-definitions repo?
> > >
> > >
> https://github.com/Linaro/test-definitions/tree/master/automated/linux/kselftest
> > >
> > > That was the first thing I tried out but was not working correctly on
> the
> > cip rootfs enviroment
>
> As you write, cip rootfs lacks the software to run kselftest. Therefore,
> it is necessary to make a separate flavor of rootfs.
> There are also two types of cip rootfs, and we need to use either or
> both for kernel testing.
>


ok.



> >
> >
> > > KernelCI also recently implemented kselftest support if that's useful
> at
> > > all, this'll include the build side of things.
> > >
> https://github.com/kernelci/kernelci-core/tree/main/config/lava/kselftest
> > >
> > >
> > I know about KernelCI effort for kselftest and I think to use a similar
> > approach about the kselftest binary creation implementation.
> >
> > The code I wrote is based on Glava-tests that is part of GKernelCI.
> > GKernelCI is the current Gentoo kernel CI system that is reporting and
> > collaborating with KernelCI.
> > https://github.com/GKernelCI/Glava-tests
> > Using a different repository currently is helpful for self managing the
> > skiplist, but it could be changed in the future.
>
>
> I think this needs to be done first.
> I also think there are several approaches to creating kselftest
> binaries.
>
> - Continue to use a particular kernel version of ksefltests.
> - Use CIP kernel kselftest.
> - Use the latest kernel kselftest.
>
> I used to build an environment using the latter.
>
>
> https://gitlab.com/cip-project/cip-kernel/linux-cip/-/tree/ci/iwamatsu/linux-cip-kselftest
>
> By the way, do you know which kernelci they are using?
>

KernelCI is using a docker container for building kselftest
https://github.com/kernelci/kernelci-core/blob/26924e6b1a90833bcdf8eb28d038fa2fac19885f/config/docker/gcc-8_x86/Dockerfile

KernelCI is building kselftest in production for at least linux-next and
mainline any time a new revision is discovered on the branches. on
staging.kernelci.org kselftest is built for each job.
This was yesterday staging build with kselftest
https://staging.kernelci.org/build/kernelci/branch/staging-mainline/kernel/staging-mainline-20210422.0/

Currently I'm working on a similar implementation for CIP testing.

Thanks,
Alicef



-- 
======================================
Cybertrust Japan Co.,Ltd.
Alice Ferrazzi
alice.ferrazzi@miraclelinux.com
======================================

[-- Attachment #1.2: Type: text/html, Size: 8493 bytes --]

[-- Attachment #2: Type: text/plain, Size: 428 bytes --]


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#6385): https://lists.cip-project.org/g/cip-dev/message/6385
Mute This Topic: https://lists.cip-project.org/mt/82071182/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/4520388/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-


^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-13 17:29 [cip-dev] Kselftest implementation Alice Ferrazzi
2021-04-13 22:27 ` Chris Paterson
2021-04-15  2:04   ` Alice Ferrazzi
2021-04-22  7:34     ` Nobuhiro Iwamatsu
2021-04-23  8:49       ` Alice Ferrazzi

CIP-dev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/cip-dev/0 cip-dev/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 cip-dev cip-dev/ https://lore.kernel.org/cip-dev \
		cip-dev@lists.cip-project.org
	public-inbox-index cip-dev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.cip-project.lists.cip-dev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git