[cip-dev] Linux-cip: Kselftest plans

[cip-dev] Linux-cip: Kselftest plans

[vijai kumar]

Hi All,

Is kselftest maintained in the cip-kernel tree? I do see some
branch[1] maintained by Nobuhiro-san for kselftest, but it's out of
5.6 linux tree.

The reason being to gather collective thoughts about the plan for
kselftest based tests for cip kernel. Are there any existing or future
plans for using kselftest for testing cip kernels? If so are we going
to use the latest tree from upstream or plan to fix/backport test
cases to current cip kernel versions?

[1] https://gitlab.com/cip-project/cip-kernel/linux-cip/-/tree/ci/iwamatsu/linux-cip-kselftest

Thanks,
Vijai Kumar K
_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev

Re: [cip-dev] Linux-cip: Kselftest plans

[Jan Kiszka]

On 06.03.20 09:21, vijai kumar wrote:
> Hi All,
> 
> Is kselftest maintained in the cip-kernel tree? I do see some
> branch[1] maintained by Nobuhiro-san for kselftest, but it's out of
> 5.6 linux tree.
> 
> The reason being to gather collective thoughts about the plan for
> kselftest based tests for cip kernel. Are there any existing or future
> plans for using kselftest for testing cip kernels? If so are we going
> to use the latest tree from upstream or plan to fix/backport test
> cases to current cip kernel versions?
> 
> [1] https://gitlab.com/cip-project/cip-kernel/linux-cip/-/tree/ci/iwamatsu/linux-cip-kselftest
> 
> Thanks,
> Vijai Kumar K

CC'ing our kernel maintainers and Chris in case someone has input on this.

Vijai, maybe you can say a few words about your integration experience 
and options to move forward.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev

Re: [cip-dev] Linux-cip: Kselftest plans

[nobuhiro1.iwamatsu]

Hi all,

> -----Original Message-----
> From: Jan Kiszka [mailto:jan.kiszka@siemens.com]
> Sent: Tuesday, March 17, 2020 12:27 AM
> To: vijai kumar <vijaikumar.kanagarajan@gmail.com>;
> cip-dev@lists.cip-project.org; Chris Paterson
> <Chris.Paterson2@renesas.com>; iwamatsu nobuhiro(岩松 信洋 ○ＳＷＣ□
> ＯＳＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>; Pavel Machek
> <pavel@denx.de>
> Subject: Re: [cip-dev] Linux-cip: Kselftest plans
> 
> On 06.03.20 09:21, vijai kumar wrote:
> > Hi All,
> >
> > Is kselftest maintained in the cip-kernel tree? I do see some
> > branch[1] maintained by Nobuhiro-san for kselftest, but it's out of
> > 5.6 linux tree.
> >
> > The reason being to gather collective thoughts about the plan for
> > kselftest based tests for cip kernel. Are there any existing or future
> > plans for using kselftest for testing cip kernels? If so are we going
> > to use the latest tree from upstream or plan to fix/backport test
> > cases to current cip kernel versions?

I am now implementing LAVA's kselftest. This kselftest uses kseflftest included in the latest kernel.
There are no plans to backport tests to 4.4 or 4.19. I believe that ksefltest included in the CIP kernel
is inadequate in function and difficult to use as it is.
I suggest using a new kernel test that incorporates the latest tests.

Best regards,
  Nobuhiro


_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev

Re: [cip-dev] Linux-cip: Kselftest plans

[vijai kumar]

On Tue, Mar 17, 2020 at 4:19 AM <nobuhiro1.iwamatsu@toshiba.co.jp> wrote:
>
> Hi all,
>
> > -----Original Message-----
> > From: Jan Kiszka [mailto:jan.kiszka@siemens.com]
> > Sent: Tuesday, March 17, 2020 12:27 AM
> > To: vijai kumar <vijaikumar.kanagarajan@gmail.com>;
> > cip-dev@lists.cip-project.org; Chris Paterson
> > <Chris.Paterson2@renesas.com>; iwamatsu nobuhiro(岩松 信洋 ○ＳＷＣ□
> > ＯＳＴ) <nobuhiro1.iwamatsu@toshiba.co.jp>; Pavel Machek
> > <pavel@denx.de>
> > Subject: Re: [cip-dev] Linux-cip: Kselftest plans
> >
> > On 06.03.20 09:21, vijai kumar wrote:
> > > Hi All,
> > >
> > > Is kselftest maintained in the cip-kernel tree? I do see some
> > > branch[1] maintained by Nobuhiro-san for kselftest, but it's out of
> > > 5.6 linux tree.
> > >
> > > The reason being to gather collective thoughts about the plan for
> > > kselftest based tests for cip kernel. Are there any existing or future
> > > plans for using kselftest for testing cip kernels? If so are we going
> > > to use the latest tree from upstream or plan to fix/backport test
> > > cases to current cip kernel versions?
>
> I am now implementing LAVA's kselftest. This kselftest uses kseflftest included in the latest kernel.
> There are no plans to backport tests to 4.4 or 4.19. I believe that ksefltest included in the CIP kernel
> is inadequate in function and difficult to use as it is.
> I suggest using a new kernel test that incorporates the latest tests.
>

Thank you Nobuhiro-San. Our original idea was to use the kselftest
from 4.19/4.4 cip tree. We faced some issues with
out-of-tree compilation while debianizing it. Also we faced some
compilation issues.

If that is the case, that we are going to use kselftest from the
latest kernel, how can we track the kselftest binaries for a
particular
cip kernel release? I am not sure if all the test cases would pass for
4.19/4/4. Do you have any plans to maintain a branch and release
kselftest binaries and/or testcases that have been tested with a
particular cip kernel release?

Thanks,
Vijai Kumar K

> Best regards,
>   Nobuhiro
>
>
_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev

Re: [cip-dev] Linux-cip: Kselftest plans

[Ben Hutchings]

On Tue, 2020-03-17 at 13:30 +0530, vijai kumar wrote:
> On Tue, Mar 17, 2020 at 4:19 AM <nobuhiro1.iwamatsu@toshiba.co.jp> wrote:
[...]
> > I am now implementing LAVA's kselftest. This kselftest uses kseflftest included in the latest kernel.
> > There are no plans to backport tests to 4.4 or 4.19. I believe that ksefltest included in the CIP kernel
> > is inadequate in function and difficult to use as it is.
> > I suggest using a new kernel test that incorporates the latest tests.
> > 
> 
> Thank you Nobuhiro-San. Our original idea was to use the kselftest
> from 4.19/4.4 cip tree. We faced some issues with
> out-of-tree compilation while debianizing it. Also we faced some
> compilation issues.
> 
> If that is the case, that we are going to use kselftest from the
> latest kernel, how can we track the kselftest binaries for a
> particular
> cip kernel release? I am not sure if all the test cases would pass for
> 4.19/4/4. Do you have any plans to maintain a branch and release
> kselftest binaries and/or testcases that have been tested with a
> particular cip kernel release?

Some self-tests will fail on older kernel versions due to missing
features or bugs that might not be practically fixable.  Test cases can
report "skip" rather than "fail" for missing features, but this is not
done consistently.

The way I've used kselftest with stable branches has been to look for
regressions rather than any failures.  The Linaro folks are doing that
(reporting regressions only) when sending test results to the stable
list, but I don't know how or how much they have automated that.

Ben.

-- 
Ben Hutchings, Software Developer                         Codethink Ltd
https://www.codethink.co.uk/                 Dale House, 35 Dale Street
                                     Manchester, M1 2HF, United Kingdom

_______________________________________________
cip-dev mailing list
cip-dev@lists.cip-project.org
https://lists.cip-project.org/mailman/listinfo/cip-dev