kernel - Regarding tools/testing/ktests or selftests

kernel - Regarding tools/testing/ktests or selftests

[Pintu Kumar]

Hi All,

Is anybody familiar with ktests of selftests framework under: tool/testing/ ?
I have some queries.

1) What is the difference between ktests and selftests ? How to choose ?
2) How to invoke a test under ktests?
    For example:
    I have some user space C program, that tests some kernel driver IOCTL calls.
    I have a Makefile to build 2 executable for the tests.
    I invoke those executable from a bash shell script and execute all
the tests.
    How can I do it automatically from ktests ?
3) Similarly, how can I do it using selftests?
    Using selftests looks much better option for me.
    But the problem is I have some sub directories to execute the tests.
    Example:
    Under, tools/testing/selftests , I have created 2 sub folders.
    selftests/folders/folder1/
    My test setup is under: folder1
    How to invoke selftests, for folder1 ?

If anybody have added new tests under ktests of selftests, please
provide some reference.


Thanks,
Pintu

kernel - Regarding tools/testing/ktests or selftests

[Pintu Kumar]

Hi,

I am actually interested in kselftests.
My requirement is:
I have a sub folders for test. Example: tools/testing/selftests/<my
folders>/<folder1>
I have my own Makefile, and some set of C source files under folder1
I also have a test script under folder1, that tests 2 executable
generated out of 7 C files.
I wanted to know about to make use of selftest framework to generate
these 2 executable, and also execute by script to run the test.
I have already includes <my folders> in top level Makefile under selftests.
TARGETS = <my folders>

Do, I need to create another Makefile under <my folders> and invoke
folder1/Makefile from there?
Or is there another way?

If anybody have already done a new addition in selftests, please let me know.


Thanks,
Pintu



On Tue, Oct 10, 2017 at 1:15 PM, Pintu Kumar <pintu.ping@gmail.com> wrote:
> Hi All,
>
> Is anybody familiar with ktests of selftests framework under: tool/testing/ ?
> I have some queries.
>
> 1) What is the difference between ktests and selftests ? How to choose ?
> 2) How to invoke a test under ktests?
>     For example:
>     I have some user space C program, that tests some kernel driver IOCTL calls.
>     I have a Makefile to build 2 executable for the tests.
>     I invoke those executable from a bash shell script and execute all
> the tests.
>     How can I do it automatically from ktests ?
> 3) Similarly, how can I do it using selftests?
>     Using selftests looks much better option for me.
>     But the problem is I have some sub directories to execute the tests.
>     Example:
>     Under, tools/testing/selftests , I have created 2 sub folders.
>     selftests/folders/folder1/
>     My test setup is under: folder1
>     How to invoke selftests, for folder1 ?
>
> If anybody have added new tests under ktests of selftests, please
> provide some reference.
>
>
> Thanks,
> Pintu

kernel - Regarding tools/testing/ktests or selftests

[Greg KH]

On Tue, Oct 10, 2017 at 01:15:11PM +0530, Pintu Kumar wrote:
> Hi All,
> 
> Is anybody familiar with ktests of selftests framework under: tool/testing/ ?
> I have some queries.
> 
> 1) What is the difference between ktests and selftests ? How to choose ?

It depends on what you want to do, but usually new things go in the
selftest section.

> 2) How to invoke a test under ktests?
>     For example:
>     I have some user space C program, that tests some kernel driver IOCTL calls.
>     I have a Makefile to build 2 executable for the tests.
>     I invoke those executable from a bash shell script and execute all
> the tests.
>     How can I do it automatically from ktests ?

Have you read the documentation the kernel has for this:
$ make help | grep test
Kernel selftest:
  kselftest       - Build and run kernel selftest (run as root)
                    running kselftest on it
  kselftest-clean - Remove all generated kselftest files
  kselftest-merge - Merge all the config dependencies of kselftest to existed

> 3) Similarly, how can I do it using selftests?
>     Using selftests looks much better option for me.
>     But the problem is I have some sub directories to execute the tests.
>     Example:
>     Under, tools/testing/selftests , I have created 2 sub folders.
>     selftests/folders/folder1/
>     My test setup is under: folder1
>     How to invoke selftests, for folder1 ?
> 
> If anybody have added new tests under ktests of selftests, please
> provide some reference.

Have you looked at the many patches in the kernel source tree that show
how the existing tests are added?  That's the best thing to use as a
template.

Hope this helps,

greg k-h

kernel - Regarding tools/testing/ktests or selftests

[Pintu Kumar]

On Tue, Oct 10, 2017 at 7:44 PM, Greg KH <greg@kroah.com> wrote:
> On Tue, Oct 10, 2017 at 01:15:11PM +0530, Pintu Kumar wrote:
>> Hi All,
>>
>> Is anybody familiar with ktests of selftests framework under: tool/testing/ ?
>> I have some queries.
>>
>> 1) What is the difference between ktests and selftests ? How to choose ?
>
> It depends on what you want to do, but usually new things go in the
> selftest section.
>
>> 2) How to invoke a test under ktests?
>>     For example:
>>     I have some user space C program, that tests some kernel driver IOCTL calls.
>>     I have a Makefile to build 2 executable for the tests.
>>     I invoke those executable from a bash shell script and execute all
>> the tests.
>>     How can I do it automatically from ktests ?
>
> Have you read the documentation the kernel has for this:
> $ make help | grep test
> Kernel selftest:
>   kselftest       - Build and run kernel selftest (run as root)
>                     running kselftest on it
>   kselftest-clean - Remove all generated kselftest files
>   kselftest-merge - Merge all the config dependencies of kselftest to existed
>
>> 3) Similarly, how can I do it using selftests?
>>     Using selftests looks much better option for me.
>>     But the problem is I have some sub directories to execute the tests.
>>     Example:
>>     Under, tools/testing/selftests , I have created 2 sub folders.
>>     selftests/folders/folder1/
>>     My test setup is under: folder1
>>     How to invoke selftests, for folder1 ?
>>
>> If anybody have added new tests under ktests of selftests, please
>> provide some reference.
>
> Have you looked at the many patches in the kernel source tree that show
> how the existing tests are added?  That's the best thing to use as a
> template.
>
> Hope this helps,

Yes, I have seen many documentation and also the existing repos under selftests.
But my requirement is little different.
I think my requirement is some what similar to futex repo.
Currently I am trying to under it. But still I am facing some issues
in generating my 2 executable using kselftests.
Also I found that current document does not explain much about how to
include new tests.

My requirement is as follows:
I have a sub folders for test. Example: tools/testing/selftests/<my
folders>/<folder1>
I have my own Makefile, and some set of C source files under folder1
I also have a test script under folder1, that tests 2 executable
generated out of 7 C files.
I wanted to know how to make use of selftest framework to generate
these 2 executable, and also execute my script to run the test.
I have already includes <my folders> in top level Makefile under selftests.
TARGETS = <my folders>

Do, I need to create another Makefile under <my folders> and invoke
folder1/Makefile from there?
Or is there another way?


>
> greg k-h

kernel - Regarding tools/testing/ktests or selftests

[Greg KH]

On Wed, Oct 11, 2017 at 08:38:52PM +0530, Pintu Kumar wrote:
> Yes, I have seen many documentation and also the existing repos under selftests.
> But my requirement is little different.
> I think my requirement is some what similar to futex repo.
> Currently I am trying to under it. But still I am facing some issues
> in generating my 2 executable using kselftests.
> Also I found that current document does not explain much about how to
> include new tests.
> 
> My requirement is as follows:
> I have a sub folders for test. Example: tools/testing/selftests/<my
> folders>/<folder1>

Why do you need <folder1>?  Just don't use it, you should not have so
many files for your test that you need to break it up even more, right?

> I have my own Makefile, and some set of C source files under folder1
> I also have a test script under folder1, that tests 2 executable
> generated out of 7 C files.
> I wanted to know how to make use of selftest framework to generate
> these 2 executable, and also execute my script to run the test.
> I have already includes <my folders> in top level Makefile under selftests.
> TARGETS = <my folders>
> 
> Do, I need to create another Makefile under <my folders> and invoke
> folder1/Makefile from there?
> Or is there another way?

Try it and see!

And post your patch to the kselftest mailing list, if you have problems
with it.  Specifics are good, and the developers there can help you out.
Asking vague questions without seeing the code itself lends itself to
vague answers :)

good luck!

greg k-h

kernel - Regarding tools/testing/ktests or selftests

[Pintu Kumar]

On Wed, Oct 11, 2017 at 10:21 PM, Greg KH <greg@kroah.com> wrote:
> On Wed, Oct 11, 2017 at 08:38:52PM +0530, Pintu Kumar wrote:
>> Yes, I have seen many documentation and also the existing repos under selftests.
>> But my requirement is little different.
>> I think my requirement is some what similar to futex repo.
>> Currently I am trying to under it. But still I am facing some issues
>> in generating my 2 executable using kselftests.
>> Also I found that current document does not explain much about how to
>> include new tests.
>>
>> My requirement is as follows:
>> I have a sub folders for test. Example: tools/testing/selftests/<my
>> folders>/<folder1>
>
> Why do you need <folder1>?  Just don't use it, you should not have so
> many files for your test that you need to break it up even more, right?
>
>> I have my own Makefile, and some set of C source files under folder1
>> I also have a test script under folder1, that tests 2 executable
>> generated out of 7 C files.
>> I wanted to know how to make use of selftest framework to generate
>> these 2 executable, and also execute my script to run the test.
>> I have already includes <my folders> in top level Makefile under selftests.
>> TARGETS = <my folders>
>>
>> Do, I need to create another Makefile under <my folders> and invoke
>> folder1/Makefile from there?
>> Or is there another way?
>
> Try it and see!
>
> And post your patch to the kselftest mailing list, if you have problems
> with it.  Specifics are good, and the developers there can help you out.
> Asking vague questions without seeing the code itself lends itself to
> vague answers :)
>
> good luck!

Ok after doing lots of research I could able to make it working now.
Now everything is perfectly working as I expected.
I think the kselftest documentation should be updated with some examples.
I will try to update it with few examples.

Thank you!
Pintu

>
> greg k-h