All of lore.kernel.org
 help / color / mirror / Atom feed
From: zohar at linux.ibm.com (Mimi Zohar)
Subject: [RFC PATCH 0/2] Kselftest shell (or even C) API
Date: Mon, 08 Apr 2019 08:14:45 -0400	[thread overview]
Message-ID: <1554725685.17244.61.camel@linux.ibm.com> (raw)
In-Reply-To: <20190406214915.16914-1-pvorel@suse.cz>

Hi Petr, Shuah,

On Sat, 2019-04-06 at 23:49 +0200, Petr Vorel wrote:
> Hi,
> 
> this is a draft trying to define some API in order to remove some
> redundancy from kselftest shell scripts. Existing kselftest.h already
> defines some sort of API for C, there is none for shell.

Shuah, when the tests were in the selftests/ima directory I was
planning on including them in my pull request; and then they moved to
selftests/kexec.  As they were still IMA related, I was still
shepherding them and planned on including them in my pull request. (Is
this Okay?  Your Review/Ack would be much appreciated.)  This patch
set, however, introduces a set of "common" set of kselftest functions.

Originally, you suggested deferring defining a set of "common"
kselftests functions to prevent delaying upstreaming the tests.  With
these patches, that time is here.  How do you want to handle this?

Thanks,

Mimi

> 
> It's just a small example how things could be. Draft, not meant to be
> really merged. But instead of defining shell library (with more useful
> helpers), I'd rather adopt LTP shell [1] and C [2] API to kselftest.
> LTP API [1] is more like a framework, easy to use with a lot of helpers
> making tests 1) small, concentrating on the problem itself 2) have
> unique output. API is well documented [3] [4], it's creator Cyril Hrubis
> made it after years experience of handling (at the time) quite bad
> quality LTP code.  Rewriting LTP tests to use this API improved tests a
> lot (less buggy, easier to read).
> 
> Some examples of advantages of LTP API:
> * SAFE_*() macros for C, which handles errors inside a library
> * unified messages, unified test status, unified way to exit testing due
> missing functionality, at the end of testing there is summary of passed,
> failed and skipped tests
> * many prepared functionality for both C and shell
> * handling threads, parent-child synchronization
> * setup and cleanup functions
> * "flags" for defining requirements or certain functionality (need root, temporary
> directory, ...)
> * and many other
> 
> kselftest and LTP has a bit different goals and approach. Probably
> not all of LTP API is needed atm, but I guess it's at least worth of
> thinking to adopt it.
> 
> There are of course other options: reinvent a wheel or left kselftest
> code in a state it is now (code quality varies, some of the code is
> really messy, buggy, not even compile).
> 
> [1] https://github.com/linux-test-project/ltp/blob/master/testcases/lib/tst_test.sh
> [2] https://github.com/linux-test-project/ltp/tree/master/lib
> [3] https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#22-writing-a-test-in-c
> [4] https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#23-writing-a-testcase-in-shell
> 
> Petr Vorel (2):
>   selftests: Start shell API
>   selftest/kexec: Use kselftest shell API
> 
>  .../selftests/kexec/kexec_common_lib.sh       | 74 +++++--------------
>  .../selftests/kexec/test_kexec_file_load.sh   | 53 ++++++-------
>  .../selftests/kexec/test_kexec_load.sh        | 20 ++---
>  tools/testing/selftests/kselftest.sh          | 53 +++++++++++++
>  4 files changed, 105 insertions(+), 95 deletions(-)
>  mode change 100755 => 100644 tools/testing/selftests/kexec/kexec_common_lib.sh
>  create mode 100644 tools/testing/selftests/kselftest.sh
> 

WARNING: multiple messages have this Message-ID
From: zohar@linux.ibm.com (Mimi Zohar)
Subject: [RFC PATCH 0/2] Kselftest shell (or even C) API
Date: Mon, 08 Apr 2019 08:14:45 -0400	[thread overview]
Message-ID: <1554725685.17244.61.camel@linux.ibm.com> (raw)
Message-ID: <20190408121445.L9WbpuP78AxdCTzNdNRBoyc8JX7qVvaKcCCfTLoD_Bs@z> (raw)
In-Reply-To: <20190406214915.16914-1-pvorel@suse.cz>

Hi Petr, Shuah,

On Sat, 2019-04-06@23:49 +0200, Petr Vorel wrote:
> Hi,
> 
> this is a draft trying to define some API in order to remove some
> redundancy from kselftest shell scripts. Existing kselftest.h already
> defines some sort of API for C, there is none for shell.

Shuah, when the tests were in the selftests/ima directory I was
planning on including them in my pull request; and then they moved to
selftests/kexec.  As they were still IMA related, I was still
shepherding them and planned on including them in my pull request. (Is
this Okay?  Your Review/Ack would be much appreciated.)  This patch
set, however, introduces a set of "common" set of kselftest functions.

Originally, you suggested deferring defining a set of "common"
kselftests functions to prevent delaying upstreaming the tests.  With
these patches, that time is here.  How do you want to handle this?

Thanks,

Mimi

> 
> It's just a small example how things could be. Draft, not meant to be
> really merged. But instead of defining shell library (with more useful
> helpers), I'd rather adopt LTP shell [1] and C [2] API to kselftest.
> LTP API [1] is more like a framework, easy to use with a lot of helpers
> making tests 1) small, concentrating on the problem itself 2) have
> unique output. API is well documented [3] [4], it's creator Cyril Hrubis
> made it after years experience of handling (at the time) quite bad
> quality LTP code.  Rewriting LTP tests to use this API improved tests a
> lot (less buggy, easier to read).
> 
> Some examples of advantages of LTP API:
> * SAFE_*() macros for C, which handles errors inside a library
> * unified messages, unified test status, unified way to exit testing due
> missing functionality, at the end of testing there is summary of passed,
> failed and skipped tests
> * many prepared functionality for both C and shell
> * handling threads, parent-child synchronization
> * setup and cleanup functions
> * "flags" for defining requirements or certain functionality (need root, temporary
> directory, ...)
> * and many other
> 
> kselftest and LTP has a bit different goals and approach. Probably
> not all of LTP API is needed atm, but I guess it's at least worth of
> thinking to adopt it.
> 
> There are of course other options: reinvent a wheel or left kselftest
> code in a state it is now (code quality varies, some of the code is
> really messy, buggy, not even compile).
> 
> [1] https://github.com/linux-test-project/ltp/blob/master/testcases/lib/tst_test.sh
> [2] https://github.com/linux-test-project/ltp/tree/master/lib
> [3] https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#22-writing-a-test-in-c
> [4] https://github.com/linux-test-project/ltp/wiki/Test-Writing-Guidelines#23-writing-a-testcase-in-shell
> 
> Petr Vorel (2):
>   selftests: Start shell API
>   selftest/kexec: Use kselftest shell API
> 
>  .../selftests/kexec/kexec_common_lib.sh       | 74 +++++--------------
>  .../selftests/kexec/test_kexec_file_load.sh   | 53 ++++++-------
>  .../selftests/kexec/test_kexec_load.sh        | 20 ++---
>  tools/testing/selftests/kselftest.sh          | 53 +++++++++++++
>  4 files changed, 105 insertions(+), 95 deletions(-)
>  mode change 100755 => 100644 tools/testing/selftests/kexec/kexec_common_lib.sh
>  create mode 100644 tools/testing/selftests/kselftest.sh
> 

  parent reply	other threads:[~2019-04-08 12:14 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-04-06 21:49 pvorel
2019-04-06 21:49 ` Petr Vorel
2019-04-06 21:49 ` [RFC PATCH 1/2] selftests: Start shell API pvorel
2019-04-06 21:49   ` Petr Vorel
2019-04-08 11:06   ` zohar
2019-04-08 11:06     ` Mimi Zohar
2019-04-08 12:22     ` pvorel
2019-04-08 12:22       ` Petr Vorel
2019-04-08 11:38   ` chrubis
2019-04-08 11:38     ` Cyril Hrubis
2019-04-08 13:07     ` pvorel
2019-04-08 13:07       ` Petr Vorel
2019-04-06 21:49 ` [RFC PATCH 2/2] selftest/kexec: Use kselftest " pvorel
2019-04-06 21:49   ` Petr Vorel
2019-04-08 11:29   ` zohar
2019-04-08 11:29     ` Mimi Zohar
2019-04-08 11:43 ` [RFC PATCH 0/2] Kselftest shell (or even C) API chrubis
2019-04-08 11:43   ` Cyril Hrubis
2019-04-08 13:25   ` pvorel
2019-04-08 13:25     ` Petr Vorel
2019-04-08 12:14 ` zohar [this message]
2019-04-08 12:14   ` Mimi Zohar
2019-04-08 12:29   ` pvorel
2019-04-08 12:29     ` Petr Vorel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1554725685.17244.61.camel@linux.ibm.com \
    --to=unknown@example.com \
    --subject='Re: [RFC PATCH 0/2] Kselftest shell (or even C) API' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.