[ANNOUNCE] rt-tests-2.0

[ANNOUNCE] rt-tests-2.0

[John Kacur]

I'm pleased to announce rt-tests-2.0

This release contains some interesting new features.

Daniel Wagner has been very busy doing more work to unify the various
programs in the rt-tests suite, and adding a feature to optionally
generate json output. Thanks for this great work Daniel

Jonathan Schwender added a cool new feature to specify the affinity of
the main thread separately from the measurement threads.

We had a few missteps too, we have long required libnuma for building
cyclictest but not at runtime. Unfortunately there were some patches
that broke the ability of cyclictest to run without libnuma, however I
am aware of the situtation and working to fix this.

Finally, the so called stable version branch has been dead a long time,
and the branch name unstable/devel/latest was the version that people
were using. That name was confusing though, "unstable"
meant that we were allowed to change the api, not that the code was
somehow unstable. In any case, in order to remove this confusion, I have
created a new default branch - main. I have deleted the "master" branch
which only contained a README file anyway as part of the community's
conscious language effort.

I decided that this new release was worthy of a bump-up to version v2.0
We could probably use a round of mostly testing and bug fixing for the
next release.

Bug reports, testing, patches are always appreaciated.

Enjoy!

Clone
git://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git
https://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git
https://kernel.googlesource.com/pub/scm/utils/rt-tests/rt-tests.git

Branch: main

Tag: v2.0

Tarballs are available here:
https://kernel.org/pub/linux/utils/rt-tests

Older version tarballs are available here:
https://kernel.org/pub/linux/utils/rt-tests/older


Daniel Wagner (77):
  cyclictest: Always use libnuma
  cyclictest: Use affinity_mask for steering thread placement
  pip_stress: De-constify prio_min
  ptsematest: Return correct exit code when showing help
  cyclictest: Remove libnuma API version 1 support
  cyclicdeadline: Add missing step command line argument
  cyclicdeadline: Remove unused struct thread_param
  cyclictest: Move verbose message into main
  signaltest: Always use libnuma
  rt-numa: Use error message helpers
  oslat: Use cpuset size as upper bound
  rt-tests: Rename error.h to rt-error.h
  rt-utils: Add JSON common header output helper
  cyclictest: Add JSON output feature
  cyclicdeadline: Add JSON output feature
  pmqtest: Add JSON output feature
  ptsematest: Add JSON output feature
  svsematest: Add JSON output feature
  oslat: Add JSON output feature
  rt-migrate-test: Add JSON output feature
  oslat: Add quiet command line option
  signaltest: Add JSON output feature
  sigwaittest: Add JSON output feature
  cyclictest: Fix printf format specifier
  cyclicdeadline.c: Fix printf format specifier
  cyclictest: Remove unused include header
  cyclicdeadline: Remove unused include header
  signaltest: Add missing --output usage info
  rt-util: Add rt_init function
  cyclictest: Initialize rt-util
  oslat: Initialize rt-util
  pmqtest: Initialize rt-util
  ptsematest: Initialize rt-util
  rt-migrate-test: Initialize rt-util
  cyclicdeadline: Initialize rt-util
  signaltest: Initialize rt-util
  sigwaittest: Initialize rt-util
  svematest: Initialize rt-util
  rt-util: Remove superfluous arguments from rt_write_json
  rt-util: Introduce rt_test_start()
  cyclictest: Record start of test execution
  oslat: Record start of test execution
  pmqtest: Record start of test execution
  ptesematest: Record start of test execution
  rt-migrate-test: Record start of test execution
  cyclicdeadline: Record start of test execution
  signaltest: Record start of test execution
  sigwaittest: Record start of test execution
  svsematest: Record start of test execution
  rt-util: Add return_code to common section of JSON output
  pi_stress: Prepare command line parser for long options only
  pi_stress: Add JSON output feature
  ssdd: Add quiet command line option
  ssdd: Add JSON output feature
  cyclicdeadline: Fix JSON output format
  cyclictest: Rename command line option --output to --json
  oslat: Rename command line option --output to --json
  pi_stress: Rename command line option --output to --json
  pmqtest: Rename command line option --output to --json
  ptsematest: Rename command line option --output to --json
  rt-migrate-test: Rename command line option --output to --json
  cyclicdeadline: Rename command line option --output to --json
  signaltest: Rename command line option --output to --json
  sigwaittest: Rename command line option --output to --json
  ssdd: Rename command line option --output to --json
  svsematest: Rename command line option --output to --json
  cyclictest: Add --json to man page
  oslat: Add --json to man page
  pi_stress: Add --json to man page
  pmqtest: Add --json to man page
  ptsematest: Add --json to man page
  rt-migrate-test: Add --json to man page
  cyclicdeadline: Add --json to man page
  signaltest: Add --json to man page
  sigwaittest: Add --json to man page
  ssdd: Add --json to man page
  svematest: Add --json to man page

John Kacur (11):
  rt-tests: rm scripts/do-git-push
  rt-tests: oslat: print version string
  rt-tests: oslat: Allocate memory for cpu_set
  Revert "cyclictest: Use affinity_mask for steering thread placement"
  Revert "cyclictest: Always use libnuma"
  Revert "signaltest: Always use libnuma"
  rt-tests: Don't assume numa is available at runtime
  rt-tests: remove rt_numa_bitmask_count in rt_numa.h
  rt-tests: cyclicdeadline.c: Remove dead code
  rt-tests: cyclictest: Add entry for mainaffinity in the manpage
  rt-tests: Change VERSION to 2.0

Jonathan Schwender (3):
  cyclictest: Fix --affinity when intermediate CPUs are offline
  cyclictest: Move main pid setaffinity handling into a function
  Subject: [PATCH v4 2/2] cyclictest: Add --mainaffinity=[CPUSET]
    option.

Peter Xu (1):
  oslat: Fix --cpu-list won't allow to schedule on all possible cores

 Makefile                              |   4 +-
 scripts/do-git-push                   | 115 --------------------
 src/cyclictest/cyclictest.8           |  10 +-
 src/cyclictest/cyclictest.c           | 110 ++++++++++++++++----
 src/cyclictest/rt_numa.h              |  14 +--
 src/include/pip_stress.h              |   5 +-
 src/include/{error.h => rt-error.h}   |   0
 src/include/rt-utils.h                |   8 ++
 src/lib/{error.c => rt-error.c}       |   2 +-
 src/lib/rt-numa.c                     |  23 ++--
 src/lib/rt-utils.c                    | 114 +++++++++++++++++++-
 src/oslat/oslat.8                     |   8 +-
 src/oslat/oslat.c                     | 143 +++++++++++++++++++------
 src/pi_tests/pi_stress.8              |   8 +-
 src/pi_tests/pi_stress.c              |  68 +++++++++---
 src/pi_tests/pip_stress.c             |   2 +-
 src/pmqtest/pmqtest.8                 |   5 +-
 src/pmqtest/pmqtest.c                 | 144 +++++++++++++++++++++-----
 src/ptsematest/ptsematest.8           |   5 +-
 src/ptsematest/ptsematest.c           | 130 +++++++++++++++++++----
 src/rt-migrate-test/rt-migrate-test.8 |   6 +-
 src/rt-migrate-test/rt-migrate-test.c | 122 ++++++++++++++++++----
 src/sched_deadline/cyclicdeadline.8   |   5 +-
 src/sched_deadline/cyclicdeadline.c   | 102 +++++++++++-------
 src/signaltest/signaltest.8           |   5 +-
 src/signaltest/signaltest.c           | 119 ++++++++++++++++-----
 src/sigwaittest/sigwaittest.8         |   5 +-
 src/sigwaittest/sigwaittest.c         | 121 ++++++++++++++++++----
 src/ssdd/ssdd.8                       |   5 +-
 src/ssdd/ssdd.c                       |  43 ++++++--
 src/svsematest/svsematest.8           |   5 +-
 src/svsematest/svsematest.c           | 129 +++++++++++++++++++----
 32 files changed, 1178 insertions(+), 407 deletions(-)
 delete mode 100755 scripts/do-git-push
 rename src/include/{error.h => rt-error.h} (100%)
 rename src/lib/{error.c => rt-error.c} (98%)

-- 
2.31.1

Re: [ANNOUNCE] rt-tests-2.0

[Mike Galbraith]

On Fri, 2021-06-25 at 12:08 -0400, John Kacur wrote:
> I'm pleased to announce rt-tests-2.0

Greetings,

cyclictest seems to have grown an mlock related regression.

homer:..git/rt-tests # ./cyclictest -Smp99
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 1.01 1.04 0.51 1/838 4411

T: 0 ( 4404) P:99 I:1000 C:   1536 Min:      1 Act:    1 Avg:    3 Max:    3846
T: 1 ( 4405) P:99 I:1500 C:   1021 Min:      1 Act:    1 Avg:    6 Max:    5912
T: 2 ( 4406) P:99 I:2000 C:    765 Min:      1 Act:    1 Avg:    8 Max:    5368
T: 3 ( 4407) P:99 I:2500 C:    611 Min:      1 Act:    1 Avg:    9 Max:    4862
T: 4 ( 4408) P:99 I:3000 C:    508 Min:      1 Act:    1 Avg:    9 Max:    4394
T: 5 ( 4409) P:99 I:3500 C:    436 Min:      1 Act:    1 Avg:    5 Max:    1699
T: 6 ( 4410) P:99 I:4000 C:    381 Min:      1 Act:    1 Avg:    1 Max:       4
T: 7 ( 4411) P:99 I:4500 C:    338 Min:      1 Act:    1 Avg:    1 Max:       4

Leave off -m switch...

homer:..git/rt-tests # ./cyclictest -Sp99
# /dev/cpu_dma_latency set to 0us
policy: fifo: loadavg: 1.19 1.02 0.55 1/833 4447

T: 0 ( 4440) P:99 I:1000 C:   1533 Min:      1 Act:    1 Avg:    1 Max:       4
T: 1 ( 4441) P:99 I:1500 C:   1022 Min:      1 Act:    1 Avg:    1 Max:       4
T: 2 ( 4442) P:99 I:2000 C:    766 Min:      1 Act:    1 Avg:    1 Max:       4
T: 3 ( 4443) P:99 I:2500 C:    613 Min:      1 Act:    1 Avg:    1 Max:       4
T: 4 ( 4444) P:99 I:3000 C:    511 Min:      1 Act:    1 Avg:    1 Max:       4
T: 5 ( 4445) P:99 I:3500 C:    438 Min:      1 Act:    1 Avg:    1 Max:       3
T: 6 ( 4446) P:99 I:4000 C:    383 Min:      1 Act:    1 Avg:    1 Max:       4
T: 7 ( 4447) P:99 I:4500 C:    340 Min:      1 Act:    1 Avg:    1 Max:       4

...and v2.0 regression goes away.

	-Mike

Re: [ANNOUNCE] rt-tests-2.0

[Mike Galbraith]

On Sat, 2021-06-26 at 08:55 +0200, Mike Galbraith wrote:
> On Fri, 2021-06-25 at 12:08 -0400, John Kacur wrote:
> > I'm pleased to announce rt-tests-2.0
>
> Greetings,
>
> cyclictest seems to have grown an mlock related regression.

Ok, chores done, I did a quick bisect/confirm.  I didn't go stare
at rt_test_start() to ponder what the mlockall connection may be.

426c94e53652b565b0da0410dcad47faef4c23c9 is the first bad commit
commit 426c94e53652b565b0da0410dcad47faef4c23c9
Author: Daniel Wagner <dwagner@suse.de>
Date:   Sat Mar 20 18:28:21 2021 +0100

    cyclictest: Record start of test execution

    Use rt_test_start() to record a timestamp when the test starts.

    Signed-off-by: Daniel Wagner <dwagner@suse.de>
    Signed-off-by: John Kacur <jkacur@redhat.com>

 src/cyclictest/cyclictest.c | 1 +
 1 file changed, 1 insertion(+)

Re: [ANNOUNCE] rt-tests-2.0

[Mike Galbraith]

On Sat, 2021-06-26 at 13:47 +0200, Mike Galbraith wrote:
> On Sat, 2021-06-26 at 08:55 +0200, Mike Galbraith wrote:
> > On Fri, 2021-06-25 at 12:08 -0400, John Kacur wrote:
> > > I'm pleased to announce rt-tests-2.0
> >
> > Greetings,
> >
> > cyclictest seems to have grown an mlock related regression.
>
> Ok, chores done, I did a quick bisect/confirm.  I didn't go stare
> at rt_test_start() to ponder what the mlockall connection may be.

Moving the call above "Get current time" stops it mucking things up.

	-Mike

Re: [ANNOUNCE] rt-tests-2.0

[Daniel Wagner]

Hi Mike,

On Sat, Jun 26, 2021 at 02:45:27PM +0200, Mike Galbraith wrote:
> On Sat, 2021-06-26 at 13:47 +0200, Mike Galbraith wrote:
> > On Sat, 2021-06-26 at 08:55 +0200, Mike Galbraith wrote:
> > > On Fri, 2021-06-25 at 12:08 -0400, John Kacur wrote:
> > > > I'm pleased to announce rt-tests-2.0
> > >
> > > Greetings,
> > >
> > > cyclictest seems to have grown an mlock related regression.
> >
> > Ok, chores done, I did a quick bisect/confirm.  I didn't go stare
> > at rt_test_start() to ponder what the mlockall connection may be.
>
> Moving the call above "Get current time" stops it mucking things up.

Thanks a lot for your excellent report. rt_test_start does

  static char ts_start[MAX_TS_SIZE];

  static void get_timestamp(char *tsbuf)
  {
          struct timeval tv;
          struct tm *tm;
          time_t t;

          gettimeofday(&tv, NULL);
          t = tv.tv_sec;
          tm = localtime(&t);
          /* RFC 2822-compliant date format */
          strftime(tsbuf, MAX_TS_SIZE, "%a, %d %b %Y %T %z", tm);
  }

  void rt_test_start(void)
  {
          get_timestamp(ts_start);
  }

I'd say the tsbuf access is the one which triggers a pagefault.

John, I would suggest to move the rt_test_start() into rt_init() and
take the timestamp at the execution start (as my initial version was). I
think the additional pain in slightly more correct start timestamp
(which is also not defined what it actually means in this context) is
just not worth the effort.

Thanks,
Daniel

Re: [ANNOUNCE] rt-tests-2.0

[John Kacur]

On Mon, 28 Jun 2021, Daniel Wagner wrote:

> Hi Mike,
> 
> On Sat, Jun 26, 2021 at 02:45:27PM +0200, Mike Galbraith wrote:
> > On Sat, 2021-06-26 at 13:47 +0200, Mike Galbraith wrote:
> > > On Sat, 2021-06-26 at 08:55 +0200, Mike Galbraith wrote:
> > > > On Fri, 2021-06-25 at 12:08 -0400, John Kacur wrote:
> > > > > I'm pleased to announce rt-tests-2.0
> > > >
> > > > Greetings,
> > > >
> > > > cyclictest seems to have grown an mlock related regression.
> > >
> > > Ok, chores done, I did a quick bisect/confirm.  I didn't go stare
> > > at rt_test_start() to ponder what the mlockall connection may be.
> >
> > Moving the call above "Get current time" stops it mucking things up.
> 
> Thanks a lot for your excellent report. rt_test_start does
> 
>   static char ts_start[MAX_TS_SIZE];
> 
>   static void get_timestamp(char *tsbuf)
>   {
>           struct timeval tv;
>           struct tm *tm;
>           time_t t;
> 
>           gettimeofday(&tv, NULL);
>           t = tv.tv_sec;
>           tm = localtime(&t);
>           /* RFC 2822-compliant date format */
>           strftime(tsbuf, MAX_TS_SIZE, "%a, %d %b %Y %T %z", tm);
>   }
> 
>   void rt_test_start(void)
>   {
>           get_timestamp(ts_start);
>   }
> 
> I'd say the tsbuf access is the one which triggers a pagefault.
> 
> John, I would suggest to move the rt_test_start() into rt_init() and
> take the timestamp at the execution start (as my initial version was). I
> think the additional pain in slightly more correct start timestamp
> (which is also not defined what it actually means in this context) is
> just not worth the effort.
> 
> Thanks,
> Daniel
> 

Send me a patch, and I'll have a look. The timerthread()function is
sensitive anyway, so it's probably a good idea not to put that kind
of thing there.

Note that the the code following the /* Get Current Time */
is getting the time, but you probably want something rougher that
you can call strftime on?

John

Re: [ANNOUNCE] rt-tests-2.0

[Daniel Wagner]

On Mon, Jun 28, 2021 at 09:50:25AM -0400, John Kacur wrote:
> > I'd say the tsbuf access is the one which triggers a pagefault.
> > 
> > John, I would suggest to move the rt_test_start() into rt_init() and
> > take the timestamp at the execution start (as my initial version was). I
> > think the additional pain in slightly more correct start timestamp
> > (which is also not defined what it actually means in this context) is
> > just not worth the effort.
> > 
> > Thanks,
> > Daniel
> > 
> 
> Send me a patch, and I'll have a look. The timerthread()function is
> sensitive anyway, so it's probably a good idea not to put that kind
> of thing there.
> 
> Note that the the code following the /* Get Current Time */
> is getting the time, but you probably want something rougher that
> you can call strftime on?

I think it's just the buffer strftime is writting into it. The page is
probably not mapped and hence the pagefault in the first access.

Anyway, I'll send a patch which moves the rt_test_start() into rt_init().