From: Shivaprasad G Bhat <sbhat@linux.ibm.com>
To: nvdimm@lists.linux.dev, dan.j.williams@intel.com,
vishal.l.verma@intel.com
Cc: aneesh.kumar@linux.ibm.com, sbhat@linux.ibm.com, vaibhav@linux.ibm.com
Subject: [RFC ndctl PATCH 0/9] test: Enable PAPR test family by default
Date: Mon, 18 Apr 2022 12:09:50 -0500 [thread overview]
Message-ID: <165030175745.3224737.6985015146263991065.stgit@lep8c.aus.stglabs.ibm.com> (raw)
It is possible to have tests run over multiple test families if
the respective modules can be loaded/unloaded serially on a test to
test basis. The kernel patch [1] separated out the PAPR specific
ndtest module on its own as "ndtest.ko".
So with [1], we have two family specific modules(nfit_test & ndtest)
which can be loaded one after the other and have the test results
in one go without a need for kernel recompilation with different
kernel configs.
The series ensures all the known failures/unsupported/irrelevant tests
to be skipped on ndtest to avoid all "known" chances of false negatives.
This is achieved using a skip file containing the list of tests to be
skipped on the given platform. The PAPR specific tests to be skipped
are updated in the skip_PAPR.js file.
A meson config option(-Dtest-families=INTEL,PAPR default) is added.
One can just limit the tests to INTEL or PAPR alone if required.
An example run would look like,
===========================================
1/72 ndctl:ndctl+INTEL / libndctl OK 2.79s
2/72 ndctl:ndctl+INTEL / dsm-fail OK 0.70s
3/72 ndctl:ndctl+INTEL / create.sh OK 0.80s
4/72 ndctl:ndctl+INTEL / clear.sh OK 15.04s
5/72 ndctl:ndctl+INTEL / pmem-errors.sh OK 14.97s
6/72 ndctl:dax+INTEL / daxdev-errors.sh OK 12.33s
7/72 ndctl:dax+INTEL / multi-dax.sh OK 0.67s
8/72 ndctl:ndctl+INTEL / btt-check.sh OK 16.20s
9/72 ndctl:ndctl+INTEL / label-compat.sh OK 0.82s
10/72 ndctl:ndctl+INTEL / sector-mode.sh OK 1.01s
11/72 ndctl:ndctl+INTEL / inject-error.sh OK 14.52s
12/72 ndctl:ndctl+INTEL / btt-errors.sh OK 24.01s
13/72 ndctl:ndctl+INTEL / hugetlb OK 0.09s
14/72 ndctl:ndctl+INTEL / btt-pad-compat.sh OK 1.78s
15/72 ndctl:ndctl+INTEL / firmware-update.sh FAIL 10.33s exit status 1
>>> TEST_PATH=/home/sbhat/code/ndctl/build/test NDCTL_TEST_FAMILY=INTEL DAXCTL=/home/sbhat/code/ndctl/build/daxctl/daxctl NDCTL=/home/sbhat/code/ndctl/build/ndctl/ndctl MALLOC_PERTURB_=181 DATA_PATH=/home/sbhat/code/ndctl/test /home/sbhat
/code/ndctl/test/firmware-update.sh
16/72 ndctl:ndctl+INTEL / ack-shutdown-count-set OK 0.26s
17/72 ndctl:ndctl+INTEL / rescan-partitions.sh OK 7.84s
18/72 ndctl:ndctl+INTEL / inject-smart.sh OK 1.89s
19/72 ndctl:ndctl+INTEL / monitor.sh OK 13.22s
20/72 ndctl:ndctl+INTEL / max_extent_ns OK 1.10s
21/72 ndctl:ndctl+INTEL / pfn-meta-errors.sh OK 14.62s
22/72 ndctl:ndctl+INTEL / track-uuid.sh OK 0.78s
23/72 ndctl:ndctl+INTEL / pmem-ns FAIL 0.26s (exit status 237 or signal 109 SIGinvalid)
>>> TEST_PATH=/home/sbhat/code/ndctl/build/test MALLOC_PERTURB_=140 NDCTL_TEST_FAMILY=INTEL DAXCTL=/home/sbhat/code/ndctl/build/daxctl/daxctl NDCTL=/home/sbhat/code/ndctl/build/ndctl/ndctl DATA_PATH=/home/sbhat/code/ndctl/test /home/sbhat
/code/ndctl/build/test/pmem-ns
24/72 ndctl:dax+INTEL / sub-section.sh SKIP 0.15s exit status 77
25/72 ndctl:dax+INTEL / dax-dev SKIP 0.03s exit status 77
26/72 ndctl:dax+INTEL / dax-ext4.sh SKIP 0.05s exit status 77
27/72 ndctl:dax+INTEL / dax-xfs.sh SKIP 0.05s exit status 77
28/72 ndctl:ndctl+INTEL / align.sh SKIP 0.09s exit status 77
29/72 ndctl:dax+INTEL / device-dax SKIP 0.04s exit status 77
30/72 ndctl:dax+INTEL / revoke-devmem OK 0.03s
31/72 ndctl:dax+INTEL / device-dax-fio.sh SKIP 0.06s exit status 77
32/72 ndctl:dax+INTEL / daxctl-devices.sh SKIP 0.07s exit status 77
33/72 ndctl:dax+INTEL / daxctl-create.sh SKIP 0.02s exit status 77
34/72 ndctl:dax+INTEL / dm.sh FAIL 0.12s exit status 1
>>> TEST_PATH=/home/sbhat/code/ndctl/build/test NDCTL_TEST_FAMILY=INTEL DAXCTL=/home/sbhat/code/ndctl/build/daxctl/daxctl NDCTL=/home/sbhat/code/ndctl/build/ndctl/ndctl MALLOC_PERTURB_=115 DATA_PATH=/home/sbhat/code/ndctl/test /home/sbhat
/code/ndctl/test/dm.sh
35/72 ndctl:dax+INTEL / mmap.sh SKIP 0.05s exit status 77
36/72 ndctl:ndctl+INTEL / security.sh SKIP 0.38s exit status 77
37/72 ndctl:ndctl+PAPR / libndctl OK 2.63s
38/72 ndctl:ndctl+PAPR / dsm-fail OK 1.41s
39/72 ndctl:ndctl+PAPR / create.sh OK 1.06s
40/72 ndctl:ndctl+PAPR / clear.sh SKIP 0.12s exit status 77
41/72 ndctl:ndctl+PAPR / pmem-errors.sh SKIP 0.16s exit status 77
42/72 ndctl:dax+PAPR / daxdev-errors.sh SKIP 0.08s exit status 77
43/72 ndctl:dax+PAPR / multi-dax.sh OK 1.03s
44/72 ndctl:ndctl+PAPR / btt-check.sh OK 17.13s
45/72 ndctl:ndctl+PAPR / label-compat.sh SKIP 0.26s exit status 77
46/72 ndctl:ndctl+PAPR / sector-mode.sh OK 1.26s
47/72 ndctl:ndctl+PAPR / inject-error.sh SKIP 0.20s exit status 77
48/72 ndctl:ndctl+PAPR / btt-errors.sh SKIP 1.42s exit status 77
49/72 ndctl:ndctl+PAPR / hugetlb OK 0.16s
50/72 ndctl:ndctl+PAPR / btt-pad-compat.sh OK 3.06s
51/72 ndctl:ndctl+PAPR / firmware-update.sh SKIP 0.90s exit status 77
52/72 ndctl:ndctl+PAPR / ack-shutdown-count-set SKIP 0.09s exit status 77
53/72 ndctl:ndctl+PAPR / rescan-partitions.sh OK 8.11s
54/72 ndctl:ndctl+PAPR / inject-smart.sh OK 1.37s
55/72 ndctl:ndctl+PAPR / monitor.sh OK 7.63s
56/72 ndctl:ndctl+PAPR / max_extent_ns OK 1.41s
57/72 ndctl:ndctl+PAPR / pfn-meta-errors.sh SKIP 0.14s exit status 77
58/72 ndctl:ndctl+PAPR / track-uuid.sh OK 1.07s
59/72 ndctl:ndctl+PAPR / pmem-ns OK 0.54s
60/72 ndctl:dax+PAPR / sub-section.sh SKIP 0.27s exit status 77
61/72 ndctl:dax+PAPR / dax-dev SKIP 0.04s exit status 77
62/72 ndctl:dax+PAPR / dax-ext4.sh SKIP 0.35s exit status 77
63/72 ndctl:dax+PAPR / dax-xfs.sh SKIP 0.39s exit status 77
64/72 ndctl:ndctl+PAPR / align.sh SKIP 0.46s exit status 77
65/72 ndctl:dax+PAPR / device-dax SKIP 0.04s exit status 77
66/72 ndctl:dax+PAPR / revoke-devmem OK 0.04s
67/72 ndctl:dax+PAPR / device-dax-fio.sh SKIP 0.32s exit status 77
68/72 ndctl:dax+PAPR / daxctl-devices.sh SKIP 0.40s exit status 77
69/72 ndctl:dax+PAPR / daxctl-create.sh SKIP 0.48s exit status 77
70/72 ndctl:dax+PAPR / dm.sh SKIP 0.48s exit status 77
71/72 ndctl:dax+PAPR / mmap.sh SKIP 0.50s exit status 77
72/72 ndctl:ndctl+PAPR / security.sh SKIP 0.22s exit status 77
Ok: 37
Expected Fail: 0
Fail: 3 # Not from PAPR family
Unexpected Pass: 0
Skipped: 32
Timeout: 0
===========================================
Note the meson suite string here "ndctl+INTEL|PAPR" indicating the
test family.
The first 6 patches in the series move the code with little/no change
to current behaviour in isolation while fixing existing bugs in
preparation for running the tests for INTEL and PAPR families one
after the other. The last 3 patches fix the known failures on ndtest
and they depend on the kernel patch [2]. These 3 patches are repost of
previously posted patches with rebase. The last patches are included
here in this series for completeness.
The patches are posted on github repos below
Kernel - https://github.com/shivaprasadbhat/linux/tree/ndtest-runall
ndctl - https://github.com/shivaprasadbhat/ndctl/tree/ndctl-test-all
References:
[1] - https://patchwork.kernel.org/project/linux-nvdimm/patch/165025849452.2974989.6131145460098517848.stgit@lep8c.aus.stglabs.ibm.com/
[2] - https://patchwork.kernel.org/project/linux-nvdimm/patch/165027233876.3035289.4353747702027907365.stgit@lep8c.aus.stglabs.ibm.com/
---
Shivaprasad G Bhat (9):
test/common: Ensure to unload test modules
test: core: Fix module unload failures
test: Unload the nfit module during cleanup
test: Introduce skip file to skip irrelevant tests
test: Assign provider name based on the test family
test: Enable PAPR test family tests after INTEL family tests
test/libndctl: Enable libndctl tests on ndtest
test/inject-smart: Enable inject-smart tests on ndtest
test/monitor.sh: Partially skip monitor test on ndtest
meson.build | 10 +++
meson_options.txt | 2 +
ndctl/bat.c | 4 +-
ndctl/test.c | 4 +-
test.h | 10 ++-
test/ack-shutdown-count-set.c | 17 ++--
test/btt-check.sh | 3 +-
test/btt-errors.sh | 3 +-
test/btt-pad-compat.sh | 3 +-
test/clear.sh | 4 +-
test/common | 57 ++++++++++++-
test/core.c | 140 +++++++++++++++++++++++++++++++-
test/create.sh | 4 +-
test/dax-dev.c | 2 +-
test/dax-pmd.c | 2 +-
test/daxdev-errors.sh | 6 +-
test/device-dax.c | 2 +-
test/dsm-fail.c | 33 +++++---
test/firmware-update.sh | 3 +-
test/inject-error.sh | 3 +-
test/inject-smart.sh | 17 ++--
test/label-compat.sh | 4 +-
test/libndctl.c | 60 ++++++++++++--
test/list-smart-dimm.c | 33 +++++++-
test/max_available_extent_ns.sh | 3 +-
test/meson.build | 59 ++++++++------
test/monitor.sh | 14 +++-
test/multi-dax.sh | 4 +-
test/pfn-meta-errors.sh | 3 +-
test/pmem-errors.sh | 3 +-
test/pmem_namespaces.c | 29 +++----
test/rescan-partitions.sh | 3 +-
test/revoke-devmem.c | 2 +-
test/sector-mode.sh | 4 +-
test/security.sh | 3 +-
test/skip_PAPR.js | 32 ++++++++
test/track-uuid.sh | 3 +-
37 files changed, 447 insertions(+), 141 deletions(-)
create mode 100644 test/skip_PAPR.js
--
next reply other threads:[~2022-04-18 17:10 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-04-18 17:09 Shivaprasad G Bhat [this message]
2022-04-18 17:10 ` [RFC ndctl PATCH 1/9] test/common: Ensure to unload test modules Shivaprasad G Bhat
2022-04-18 17:10 ` [RFC ndctl PATCH 2/9] test: core: Fix module unload failures Shivaprasad G Bhat
2022-04-18 17:10 ` [RFC ndctl PATCH 3/9] test: Unload the nfit module during cleanup Shivaprasad G Bhat
2022-04-18 17:10 ` [RFC ndctl PATCH 4/9] test: Introduce skip file to skip irrelevant tests Shivaprasad G Bhat
2022-04-18 17:10 ` [RFC ndctl PATCH 5/9] test: Assign provider name based on the test family Shivaprasad G Bhat
2022-04-18 17:11 ` [RFC ndctl PATCH 6/9] test: Enable PAPR test family tests after INTEL family tests Shivaprasad G Bhat
2022-04-18 17:15 ` [ndctl v3 PATCH 7/9] test/libndctl: Enable libndctl tests on ndtest Shivaprasad G Bhat
2022-04-18 17:17 ` [ndctl v3 PATCH 8/9] test/inject-smart: Enable inject-smart " Shivaprasad G Bhat
2022-04-18 17:19 ` [ndctl v3 PATCH 9/9] test/monitor.sh: Partially skip monitor test " Shivaprasad G Bhat
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=165030175745.3224737.6985015146263991065.stgit@lep8c.aus.stglabs.ibm.com \
--to=sbhat@linux.ibm.com \
--cc=aneesh.kumar@linux.ibm.com \
--cc=dan.j.williams@intel.com \
--cc=nvdimm@lists.linux.dev \
--cc=vaibhav@linux.ibm.com \
--cc=vishal.l.verma@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.