[PATCH V2 blktests] nvme: Add new test case about nvme rescan/reset/remove during IO

[PATCH V2 blktests] nvme: Add new test case about nvme rescan/reset/remove during IO

[Yi Zhang]

Add one test to cover NVMe SSD rescan/reset/remove operation during
IO, the steps found several issues during my previous testing, check
them here:
http://lists.infradead.org/pipermail/linux-nvme/2017-February/008358.html
http://lists.infradead.org/pipermail/linux-nvme/2017-May/010259.html

Signed-off-by: Yi Zhang <yi.zhang@redhat.com>
---

changes from v1:
 - add variable for "/sys/bus/pci/devices/${pdev}"
 - add kill $!; wait; for background fio
 - add rescan/reset/remove sysfs node check
 - add loop checking for nvme reinitialized

---
---
 tests/nvme/031     | 71 ++++++++++++++++++++++++++++++++++++++++++++++
 tests/nvme/031.out |  2 ++
 2 files changed, 73 insertions(+)
 create mode 100755 tests/nvme/031
 create mode 100644 tests/nvme/031.out

diff --git a/tests/nvme/031 b/tests/nvme/031
new file mode 100755
index 0000000..db163a2
--- /dev/null
+++ b/tests/nvme/031
@@ -0,0 +1,71 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-3.0+
+# Copyright (C) 2019 Yi Zhang <yi.zhang@redhat.com>
+#
+# Test nvme pci adapter rescan/reset/remove operation during I/O
+#
+# Regression test for bellow two commits:
+# http://lists.infradead.org/pipermail/linux-nvme/2017-May/010367.html
+# 986f75c876db nvme: avoid to use blk_mq_abort_requeue_list()
+# 806f026f9b90 nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
+
+. tests/nvme/rc
+
+DESCRIPTION="test nvme pci adapter rescan/reset/remove during I/O"
+TIMED=1
+
+requires() {
+	_have_fio
+}
+
+device_requires() {
+	_test_dev_is_nvme
+}
+
+test_device() {
+	echo "Running ${TEST_NAME}"
+
+	pdev="$(_get_pci_dev_from_blkdev)"
+	local sysfs="/sys/bus/pci/devices/${pdev}"
+
+	# start fio job
+	_run_fio_rand_io --filename="$TEST_DEV" --size=1g \
+		--group_reporting  &> /dev/null &
+
+	sleep 5
+
+	# do rescan/reset/remove operation
+	if [[ -f "${sysfs}"/rescan ]]; then
+		echo 1 > "${sysfs}"/rescan
+	else
+		echo "${sysfs}/rescan doesn't exist!"
+	fi
+	# QEMU VM doesn't have the "reset" attribute, skip it
+	if [[ -f "${sysfs}"/reset ]]; then
+		echo 1 > "${sysfs}"/reset
+	fi
+	if [[ -f "${sysfs}"/remove ]]; then
+		echo 1 > "${sysfs}"/remove
+	else
+		echo "${sysfs}/remove doesn't exist!"
+
+	fi
+
+	{ kill $!; wait; } &> /dev/null
+
+	echo 1 > /sys/bus/pci/rescan
+
+	# wait nvme reinitialized
+	local m
+	for ((m = 0; m < 10; m++)); do
+		if [[ -b "${TEST_DEV}" ]]; then
+			break
+		fi
+		sleep 0.5
+	done
+        if (( m > 9 )); then
+                echo "nvme still not reinitialized after 5 seconds!"
+        fi
+
+	echo "Test complete"
+}
diff --git a/tests/nvme/031.out b/tests/nvme/031.out
new file mode 100644
index 0000000..ae902bd
--- /dev/null
+++ b/tests/nvme/031.out
@@ -0,0 +1,2 @@
+Running nvme/031
+Test complete
-- 
2.17.2

Re: [PATCH V2 blktests] nvme: Add new test case about nvme rescan/reset/remove during IO

[Chaitanya Kulkarni]

On 09/09/2019 09:47 AM, Yi Zhang wrote:
> Add one test to cover NVMe SSD rescan/reset/remove operation during
> IO, the steps found several issues during my previous testing, check
> them here:
> http://lists.infradead.org/pipermail/linux-nvme/2017-February/008358.html
> http://lists.infradead.org/pipermail/linux-nvme/2017-May/010259.html
>
> Signed-off-by: Yi Zhang <yi.zhang@redhat.com>
> ---
>
> changes from v1:
>   - add variable for "/sys/bus/pci/devices/${pdev}"
>   - add kill $!; wait; for background fio
>   - add rescan/reset/remove sysfs node check
>   - add loop checking for nvme reinitialized
>
> ---
> ---
>   tests/nvme/031     | 71 ++++++++++++++++++++++++++++++++++++++++++++++
>   tests/nvme/031.out |  2 ++
>   2 files changed, 73 insertions(+)
>   create mode 100755 tests/nvme/031
>   create mode 100644 tests/nvme/031.out
>
> diff --git a/tests/nvme/031 b/tests/nvme/031
> new file mode 100755
> index 0000000..db163a2
> --- /dev/null
> +++ b/tests/nvme/031
> @@ -0,0 +1,71 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-3.0+
> +# Copyright (C) 2019 Yi Zhang <yi.zhang@redhat.com>
> +#
> +# Test nvme pci adapter rescan/reset/remove operation during I/O
> +#
> +# Regression test for bellow two commits:
> +# http://lists.infradead.org/pipermail/linux-nvme/2017-May/010367.html
> +# 986f75c876db nvme: avoid to use blk_mq_abort_requeue_list()
> +# 806f026f9b90 nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
> +
> +. tests/nvme/rc
> +
> +DESCRIPTION="test nvme pci adapter rescan/reset/remove during I/O"
> +TIMED=1
> +
> +requires() {
> +	_have_fio
> +}
> +
> +device_requires() {
> +	_test_dev_is_nvme
> +}
> +
> +test_device() {
> +	echo "Running ${TEST_NAME}"
> +
> +	pdev="$(_get_pci_dev_from_blkdev)"
> +	local sysfs="/sys/bus/pci/devices/${pdev}"
> +
> +	# start fio job
> +	_run_fio_rand_io --filename="$TEST_DEV" --size=1g \
> +		--group_reporting  &> /dev/null &
> +
> +	sleep 5
> +
> +	# do rescan/reset/remove operation
> +	if [[ -f "${sysfs}"/rescan ]]; then
> +		echo 1 > "${sysfs}"/rescan
> +	else
> +		echo "${sysfs}/rescan doesn't exist!"
> +	fi
> +	# QEMU VM doesn't have the "reset" attribute, skip it
> +	if [[ -f "${sysfs}"/reset ]]; then
> +		echo 1 > "${sysfs}"/reset
> +	fi

> +	if [[ -f "${sysfs}"/remove ]]; then
> +		echo 1 > "${sysfs}"/remove
> +	else
> +		echo "${sysfs}/remove doesn't exist!"
> +
> +	fi

This is a lot of code repetition. You should be creating one helper
and passing just file names. (No need to check the return value).
something like this :-

check_sysfs()
{
	local sysfs_attr=$1

	if [[ -f "${sysfs_attr}" ]]; then
		echo 1 > "${sysfs_attr}"
	else
		#TODO : add a check to not print if sysfs_attr is not
                 #reset
		echo "${sysfs_attr] doesn't exist!"
	fi
}
and call above function here :-

	for i in rescan remove reset; do
		check_sysfs_attr $i
	done

> +
> +	{ kill $!; wait; } &> /dev/null
> +
> +	echo 1 > /sys/bus/pci/rescan
> +
> +	# wait nvme reinitialized
> +	local m
Please declare all the local variables at the start of the function.

Do we need to call udevadm settle here ?

> +	for ((m = 0; m < 10; m++)); do
> +		if [[ -b "${TEST_DEV}" ]]; then
> +			break
> +		fi
> +		sleep 0.5
> +	done
> +        if (( m > 9 )); then
> +                echo "nvme still not reinitialized after 5 seconds!"
> +        fi
Please recheck the alignment in the above if.

> +
> +	echo "Test complete"
> +}
> diff --git a/tests/nvme/031.out b/tests/nvme/031.out
> new file mode 100644
> index 0000000..ae902bd
> --- /dev/null
> +++ b/tests/nvme/031.out
> @@ -0,0 +1,2 @@
> +Running nvme/031
> +Test complete
>

Re: [PATCH V2 blktests] nvme: Add new test case about nvme rescan/reset/remove during IO

[Yi Zhang]

Hi Chaitanya
Thanks for your review.

On 9/10/19 1:03 AM, Chaitanya Kulkarni wrote:
> On 09/09/2019 09:47 AM, Yi Zhang wrote:
>> Add one test to cover NVMe SSD rescan/reset/remove operation during
>> IO, the steps found several issues during my previous testing, check
>> them here:
>> http://lists.infradead.org/pipermail/linux-nvme/2017-February/008358.html
>> http://lists.infradead.org/pipermail/linux-nvme/2017-May/010259.html
>>
>> Signed-off-by: Yi Zhang <yi.zhang@redhat.com>
>> ---
>>
>> changes from v1:
>>    - add variable for "/sys/bus/pci/devices/${pdev}"
>>    - add kill $!; wait; for background fio
>>    - add rescan/reset/remove sysfs node check
>>    - add loop checking for nvme reinitialized
>>
>> ---
>> ---
>>    tests/nvme/031     | 71 ++++++++++++++++++++++++++++++++++++++++++++++
>>    tests/nvme/031.out |  2 ++
>>    2 files changed, 73 insertions(+)
>>    create mode 100755 tests/nvme/031
>>    create mode 100644 tests/nvme/031.out
>>
>> diff --git a/tests/nvme/031 b/tests/nvme/031
>> new file mode 100755
>> index 0000000..db163a2
>> --- /dev/null
>> +++ b/tests/nvme/031
>> @@ -0,0 +1,71 @@
>> +#!/bin/bash
>> +# SPDX-License-Identifier: GPL-3.0+
>> +# Copyright (C) 2019 Yi Zhang <yi.zhang@redhat.com>
>> +#
>> +# Test nvme pci adapter rescan/reset/remove operation during I/O
>> +#
>> +# Regression test for bellow two commits:
>> +# http://lists.infradead.org/pipermail/linux-nvme/2017-May/010367.html
>> +# 986f75c876db nvme: avoid to use blk_mq_abort_requeue_list()
>> +# 806f026f9b90 nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
>> +
>> +. tests/nvme/rc
>> +
>> +DESCRIPTION="test nvme pci adapter rescan/reset/remove during I/O"
>> +TIMED=1
>> +
>> +requires() {
>> +	_have_fio
>> +}
>> +
>> +device_requires() {
>> +	_test_dev_is_nvme
>> +}
>> +
>> +test_device() {
>> +	echo "Running ${TEST_NAME}"
>> +
>> +	pdev="$(_get_pci_dev_from_blkdev)"
>> +	local sysfs="/sys/bus/pci/devices/${pdev}"
>> +
>> +	# start fio job
>> +	_run_fio_rand_io --filename="$TEST_DEV" --size=1g \
>> +		--group_reporting  &> /dev/null &
>> +
>> +	sleep 5
>> +
>> +	# do rescan/reset/remove operation
>> +	if [[ -f "${sysfs}"/rescan ]]; then
>> +		echo 1 > "${sysfs}"/rescan
>> +	else
>> +		echo "${sysfs}/rescan doesn't exist!"
>> +	fi
>> +	# QEMU VM doesn't have the "reset" attribute, skip it
>> +	if [[ -f "${sysfs}"/reset ]]; then
>> +		echo 1 > "${sysfs}"/reset
>> +	fi
>> +	if [[ -f "${sysfs}"/remove ]]; then
>> +		echo 1 > "${sysfs}"/remove
>> +	else
>> +		echo "${sysfs}/remove doesn't exist!"
>> +
>> +	fi
> This is a lot of code repetition. You should be creating one helper
> and passing just file names. (No need to check the return value).
> something like this :-
>
> check_sysfs()
> {
> 	local sysfs_attr=$1
>
> 	if [[ -f "${sysfs_attr}" ]]; then
> 		echo 1 > "${sysfs_attr}"
> 	else
> 		#TODO : add a check to not print if sysfs_attr is not
>                   #reset
> 		echo "${sysfs_attr] doesn't exist!"
> 	fi
> }
> and call above function here :-
>
> 	for i in rescan remove reset; do
> 		check_sysfs_attr $i
> 	done
will fix it in next version

>> +
>> +	{ kill $!; wait; } &> /dev/null
>> +
>> +	echo 1 > /sys/bus/pci/rescan
>> +
>> +	# wait nvme reinitialized
>> +	local m
> Please declare all the local variables at the start of the function.
>
> Do we need to call udevadm settle here ?
agree, will add it.

>
>> +	for ((m = 0; m < 10; m++)); do
>> +		if [[ -b "${TEST_DEV}" ]]; then
>> +			break
>> +		fi
>> +		sleep 0.5
>> +	done
>> +        if (( m > 9 )); then
>> +                echo "nvme still not reinitialized after 5 seconds!"
>> +        fi
> Please recheck the alignment in the above if.
Thanks.

>
>> +
>> +	echo "Test complete"
>> +}
>> diff --git a/tests/nvme/031.out b/tests/nvme/031.out
>> new file mode 100644
>> index 0000000..ae902bd
>> --- /dev/null
>> +++ b/tests/nvme/031.out
>> @@ -0,0 +1,2 @@
>> +Running nvme/031
>> +Test complete
>>