From: Jonathan Derrick <jonathan.derrick@linux.dev>
To: <linux-nvme@lists.infradead.org>, <linux-block@vger.kernel.org>,
"Shin\\'ichiro Kawasaki" <shinichiro.kawasaki@wdc.com>
Cc: Keith Busch <kbusch@kernel.org>,
Omar Sandoval <osandov@osandov.com>,
Jonathan Derrick <jonathan.derrick@linux.dev>
Subject: [PATCH] tests/nvme: Add admin-passthru+reset race test
Date: Mon, 14 Nov 2022 13:34:12 -0700 [thread overview]
Message-ID: <20221114203412.383-1-jonathan.derrick@linux.dev> (raw)
Adds a test which runs many formats and reset_controllers in parallel.
The intent is to expose timing holes in the controller state machine
which will lead to hung task timing and the controller becoming
unavailable.
Reported by https://bugzilla.kernel.org/show_bug.cgi?id=216354
Signed-off-by: Jonathan Derrick <jonathan.derrick@linux.dev>
---
tests/nvme/046 | 85 ++++++++++++++++++++++++++++++++++++++++++++++
tests/nvme/046.out | 2 ++
2 files changed, 87 insertions(+)
create mode 100755 tests/nvme/046
create mode 100644 tests/nvme/046.out
diff --git a/tests/nvme/046 b/tests/nvme/046
new file mode 100755
index 0000000..4b47783
--- /dev/null
+++ b/tests/nvme/046
@@ -0,0 +1,85 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2022 Jonathan Derrick <jonathan.derrick@linux.dev>
+#
+# Test nvme reset controller during admin passthru
+#
+# Regression for issue reported by
+# https://bugzilla.kernel.org/show_bug.cgi?id=216354
+
+. tests/nvme/rc
+
+#restrict test to nvme-pci only
+nvme_trtype=pci
+
+DESCRIPTION="test nvme reset controller during admin passthru"
+QUICK=1
+CAN_BE_ZONED=1
+
+requires() {
+ _nvme_requires
+}
+
+device_requires() {
+ _require_test_dev_is_nvme
+}
+
+test_device() {
+ echo "Running ${TEST_NAME}"
+
+ local sysfs
+ local attr
+ local m
+
+ sysfs="$TEST_DEV_SYSFS/device"
+ timeout=$(($(cat /proc/sys/kernel/hung_task_timeout_secs) / 2))
+
+ sleep 5
+
+ if [[ ! -d "$sysfs" ]]; then
+ echo "$sysfs doesn't exist"
+ fi
+
+ # do reset controller/format loops
+ # don't check status now because a timing race is desired
+ i=0
+ start=0
+ timing_out=false
+ while [[ $i -le 1000 ]]; do
+ start=$SECONDS
+ if [[ -f "$sysfs/reset_controller" ]]; then
+ echo 1 > "$sysfs/reset_controller" 2>/dev/null &
+ i=$((i+1))
+ fi
+ nvme format -l 0 -f $TEST_DEV 2>/dev/null &
+
+ #Assume the controller is hung and unrecoverable
+ if [[ $(($SECONDS - $start)) -gt $timeout ]]; then
+ echo "nvme controller timing out"
+ timing_out=true
+ break
+ fi
+ done
+
+ { kill $!; wait; } &> /dev/null
+
+ # at this point it may have waited hung_task_timeout / 2 already, so
+ # only wait 25% longer for a total of about 75% of allowed timeout
+ m=0
+ while [[ $m -le $((timeout / 2)) ]]; do
+ if [[ $timing_out == true ]]; then
+ break
+ fi
+ if grep -q live "$sysfs/state"; then
+ break
+ fi
+ sleep 1
+ m=$((m+1))
+ done
+ if ! grep -q live "$sysfs/state"; then
+ echo "nvme still not live after $(($SECONDS - $start)) seconds!"
+ fi
+ udevadm settle
+
+ echo "Test complete"
+}
diff --git a/tests/nvme/046.out b/tests/nvme/046.out
new file mode 100644
index 0000000..2b5fa6a
--- /dev/null
+++ b/tests/nvme/046.out
@@ -0,0 +1,2 @@
+Running nvme/046
+Test complete
--
2.31.1
next reply other threads:[~2022-11-14 20:37 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-14 20:34 Jonathan Derrick [this message]
2022-11-15 0:25 ` [PATCH] tests/nvme: Add admin-passthru+reset race test Chaitanya Kulkarni
2022-11-15 8:49 ` Jonathan Derrick
2022-11-15 11:01 ` Shinichiro Kawasaki
2022-11-15 0:33 ` kernel test robot
2022-11-15 23:07 ` Keith Busch
2022-11-16 7:43 ` Christoph Hellwig
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221114203412.383-1-jonathan.derrick@linux.dev \
--to=jonathan.derrick@linux.dev \
--cc=kbusch@kernel.org \
--cc=linux-block@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=osandov@osandov.com \
--cc=shinichiro.kawasaki@wdc.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.