* [PATCH V2 blktests 0/5] nvmeof-mp/srp/nvme-rdma misc fix and enhancement
@ 2020-11-25 7:32 Yi Zhang
2020-11-25 7:32 ` [PATCH V2 blktests 1/5] tests/srp/rc: update the ib_srpt module name Yi Zhang
` (4 more replies)
0 siblings, 5 replies; 10+ messages in thread
From: Yi Zhang @ 2020-11-25 7:32 UTC (permalink / raw)
To: osandov, bvanassche; +Cc: linux-block, linux-nvme, sagi
Hi
This patch series addressed some failures when I run nvmeof-mp/srp
test and also add suport to use siw for nvme-rdma/nvmeof-mp/srp testing
from cmdline, like this:
$ use_siw=1 nvme-trtype=rdma ./check nvme
$ use_siw=1 ./check nvmeof-mp
$ use_siw-1 ./check srp
V2:
Update the ib_srpt module path in patch 1, avoid to use ls
Update the SKIP_REASON in patch 2
Introduce get_scheduler_list, fix nvmeof-mp/012 and srp/012 in patch 3
Typo fix in patch 4
Yi Zhang (5):
tests/srp/rc: update the ib_srpt module name
tests/nvmeof-mp/rc: run nvmeof-mp tests if we set multipath=N
nvmeof-mp/012, srp/012: fix the scheduler list
common/rc: _have_iproute2 fix for "ip -V" change
common/multipath-over-rdma: allow to set use_siw
common/multipath-over-rdma | 14 +++++++++++++-
common/rc | 2 +-
tests/nvmeof-mp/012 | 10 ++++++----
tests/nvmeof-mp/rc | 8 +++++---
tests/srp/012 | 10 ++++++----
tests/srp/rc | 4 ++--
6 files changed, 33 insertions(+), 15 deletions(-)
--
2.21.0
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH V2 blktests 1/5] tests/srp/rc: update the ib_srpt module name
2020-11-25 7:32 [PATCH V2 blktests 0/5] nvmeof-mp/srp/nvme-rdma misc fix and enhancement Yi Zhang
@ 2020-11-25 7:32 ` Yi Zhang
2020-11-26 3:53 ` Bart Van Assche
2020-11-25 7:32 ` [PATCH V2 blktests 2/5] tests/nvmeof-mp/rc: run nvmeof-mp tests if we set multipath=N Yi Zhang
` (3 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Yi Zhang @ 2020-11-25 7:32 UTC (permalink / raw)
To: osandov, bvanassche; +Cc: linux-block, linux-nvme, sagi
Fix the ib_srpt module insmod failure as the module in some distros are
end with .xz, like bellow on fedora:
/lib/modules/$(uname -r)/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko.xz
Signed-off-by: Yi Zhang <yi.zhang@redhat.com>
---
tests/srp/rc | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tests/srp/rc b/tests/srp/rc
index 7fc094b..1f665a2 100755
--- a/tests/srp/rc
+++ b/tests/srp/rc
@@ -465,7 +465,7 @@ configure_target_ports() {
# Load LIO and configure the SRP target driver and LUNs.
start_lio_srpt() {
- local b d gid i ini_ids=() opts p target_ids=() vdev
+ local b d gid i ini_ids=() opts p target_ids=()
for gid in $(all_primary_gids); do
if [ "${gid#fe8}" != "$gid" ]; then
@@ -500,7 +500,7 @@ start_lio_srpt() {
if modinfo ib_srpt | grep -q '^parm:[[:blank:]]*rdma_cm_port:'; then
opts+=("rdma_cm_port=${srp_rdma_cm_port}")
fi
- insmod "/lib/modules/$(uname -r)/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko" "${opts[@]}" || return $?
+ insmod "/lib/modules/$(uname -r)/kernel/drivers/infiniband/ulp/srpt/ib_srpt."* "${opts[@]}" || return $?
i=0
for r in "${vdev_path[@]}"; do
if [ -b "$(readlink -f "$r")" ]; then
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH V2 blktests 2/5] tests/nvmeof-mp/rc: run nvmeof-mp tests if we set multipath=N
2020-11-25 7:32 [PATCH V2 blktests 0/5] nvmeof-mp/srp/nvme-rdma misc fix and enhancement Yi Zhang
2020-11-25 7:32 ` [PATCH V2 blktests 1/5] tests/srp/rc: update the ib_srpt module name Yi Zhang
@ 2020-11-25 7:32 ` Yi Zhang
2020-11-26 3:54 ` Bart Van Assche
2020-11-25 7:32 ` [PATCH V2 blktests 3/5] nvmeof-mp/012, srp/012: fix the scheduler list Yi Zhang
` (2 subsequent siblings)
4 siblings, 1 reply; 10+ messages in thread
From: Yi Zhang @ 2020-11-25 7:32 UTC (permalink / raw)
To: osandov, bvanassche; +Cc: linux-block, linux-nvme, sagi
To enable it, just do bellow step before we run it:
$ echo "options nvme_core multipath=N" >/etc/modprobe.d/nvme.conf
Signed-off-by: Yi Zhang <yi.zhang@redhat.com>
---
tests/nvmeof-mp/rc | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/tests/nvmeof-mp/rc b/tests/nvmeof-mp/rc
index d7a7c87..fc06856 100755
--- a/tests/nvmeof-mp/rc
+++ b/tests/nvmeof-mp/rc
@@ -16,9 +16,11 @@ group_requires() {
# Since the nvmeof-mp tests are based on the dm-mpath driver, these
# tests are incompatible with the NVME_MULTIPATH kernel configuration
- # option.
- if _have_kernel_option NVME_MULTIPATH; then
- SKIP_REASON="CONFIG_NVME_MULTIPATH has been set in .config"
+ # option with multipathing enabled in the nvme_core kernel module.
+ if _have_kernel_option NVME_MULTIPATH && \
+ _have_module_param_value nvme_core multipath Y; then
+ SKIP_REASON="CONFIG_NVME_MULTIPATH has been set in .config \
+ and multipathing has been enabled in the nvme_core kernel module"
return
fi
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH V2 blktests 3/5] nvmeof-mp/012, srp/012: fix the scheduler list
2020-11-25 7:32 [PATCH V2 blktests 0/5] nvmeof-mp/srp/nvme-rdma misc fix and enhancement Yi Zhang
2020-11-25 7:32 ` [PATCH V2 blktests 1/5] tests/srp/rc: update the ib_srpt module name Yi Zhang
2020-11-25 7:32 ` [PATCH V2 blktests 2/5] tests/nvmeof-mp/rc: run nvmeof-mp tests if we set multipath=N Yi Zhang
@ 2020-11-25 7:32 ` Yi Zhang
2020-11-26 3:59 ` Bart Van Assche
2020-11-25 7:32 ` [PATCH V2 blktests 4/5] common/rc: _have_iproute2 fix for "ip -V" change Yi Zhang
2020-11-25 7:32 ` [PATCH V2 blktests 5/5] common/multipath-over-rdma: allow to set use_siw Yi Zhang
4 siblings, 1 reply; 10+ messages in thread
From: Yi Zhang @ 2020-11-25 7:32 UTC (permalink / raw)
To: osandov, bvanassche; +Cc: linux-block, linux-nvme, sagi
There is no cfq scheduler and new added kyber scheduler in lastest kernel,
introduce get_scheduler_list and fix nvmeof-mp/012 srp/012
To reproduce it:
$ ./check nvmeof-mp/012
nvmeof-mp/012 (dm-mpath on top of multiple I/O schedulers) [passed]
runtime 5.922s ... 8.804s
$ cat results/nodev/nvmeof-mp/012.full | grep -n "Changing scheduler"
31:Changing scheduler of dm-3 from mq-deadline kyber [bfq] none into cfq failed
47:Changing scheduler of dm-3 from mq-deadline kyber [bfq] none into cfq failed
Signed-off-by: Yi Zhang <yi.zhang@redhat.com>
---
common/multipath-over-rdma | 12 ++++++++++++
tests/nvmeof-mp/012 | 10 ++++++----
tests/srp/012 | 10 ++++++----
3 files changed, 24 insertions(+), 8 deletions(-)
diff --git a/common/multipath-over-rdma b/common/multipath-over-rdma
index ebc5939..1bbefb7 100644
--- a/common/multipath-over-rdma
+++ b/common/multipath-over-rdma
@@ -326,6 +326,18 @@ set_scheduler() {
fi
}
+# Get block dev scheduler list
+get_scheduler_list() {
+ local b=$1 p
+ p=/sys/block/"$b"/queue/scheduler
+ if [ -e "$p" ]; then
+ scheds=$(sed 's/[][]//g' /sys/block/"$b"/queue/scheduler)
+ echo "$scheds"
+ else
+ return 1
+ fi
+}
+
# Get a /dev/... path that points at dm device number $1. Set its I/O scheduler
# to $2 and its timeout to $3. The shell script that includes this file must
# define a function get_bdev_path() that translates device number $1 into a
diff --git a/tests/nvmeof-mp/012 b/tests/nvmeof-mp/012
index ae6a92c..8b2e918 100755
--- a/tests/nvmeof-mp/012
+++ b/tests/nvmeof-mp/012
@@ -8,7 +8,7 @@ DESCRIPTION="dm-mpath on top of multiple I/O schedulers"
QUICK=1
test_io_schedulers() {
- local dev m
+ local dev m dm scheds
# Load all I/O scheduler kernel modules
for m in "/lib/modules/$(uname -r)/kernel/block/"*.ko; do
@@ -17,15 +17,17 @@ test_io_schedulers() {
for mq in y n; do
use_blk_mq ${mq} || return $?
dev=$(get_bdev 0) || return $?
- for sched in noop deadline bfq cfq; do
- set_scheduler "$(basename "$(readlink -f "${dev}")")" $sched \
+ dm=$(basename "$(readlink -f "${dev}")") || return $?
+ scheds="$(get_scheduler_list "$dm")" || return $?
+ for sched in $scheds; do
+ set_scheduler "$dm" "$sched" \
>>"$FULL" 2>&1 || continue
echo "I/O scheduler: $sched; use_blk_mq: $mq" >>"$FULL"
run_fio --verify=md5 --rw=randwrite --bs=4K --size=64K \
--ioengine=libaio --iodepth=64 \
--iodepth_batch=32 --group_reporting --sync=1 \
--direct=1 --filename="$dev" \
- --name=${sched} --thread --numjobs=1 \
+ --name="${sched}" --thread --numjobs=1 \
--output="${RESULTS_DIR}/nvmeof-mp/012-${sched}-${mq}.txt" \
>>"$FULL" ||
return $?
diff --git a/tests/srp/012 b/tests/srp/012
index 20c6daf..ffd56ef 100755
--- a/tests/srp/012
+++ b/tests/srp/012
@@ -8,7 +8,7 @@ DESCRIPTION="dm-mpath on top of multiple I/O schedulers"
QUICK=1
test_io_schedulers() {
- local dev m
+ local dev m dm scheds
# Load all I/O scheduler kernel modules
for m in "/lib/modules/$(uname -r)/kernel/block/"*.ko; do
@@ -18,15 +18,17 @@ test_io_schedulers() {
[ $mq = n ] && ! _have_legacy_dm && continue
use_blk_mq ${mq} ${mq} || return $?
dev=$(get_bdev 0) || return $?
- for sched in noop deadline bfq cfq; do
- set_scheduler "$(basename "$(readlink -f "${dev}")")" $sched \
+ dm=$(basename "$(readlink -f "${dev}")") || return $?
+ scheds="$(get_scheduler_list "$dm")" || return $?
+ for sched in $scheds; do
+ set_scheduler "$dm" "$sched" \
>>"$FULL" 2>&1 || continue
echo "I/O scheduler: $sched; use_blk_mq: $mq" >>"$FULL"
run_fio --verify=md5 --rw=randwrite --bs=4K --size=64K \
--ioengine=libaio --iodepth=64 \
--iodepth_batch=32 --group_reporting --sync=1 \
--direct=1 --filename="$dev" \
- --name=${sched} --thread --numjobs=1 \
+ --name="${sched}" --thread --numjobs=1 \
--output="${RESULTS_DIR}/srp/012-${sched}-${mq}.txt" \
>>"$FULL" ||
return $?
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH V2 blktests 4/5] common/rc: _have_iproute2 fix for "ip -V" change
2020-11-25 7:32 [PATCH V2 blktests 0/5] nvmeof-mp/srp/nvme-rdma misc fix and enhancement Yi Zhang
` (2 preceding siblings ...)
2020-11-25 7:32 ` [PATCH V2 blktests 3/5] nvmeof-mp/012, srp/012: fix the scheduler list Yi Zhang
@ 2020-11-25 7:32 ` Yi Zhang
2020-11-25 7:32 ` [PATCH V2 blktests 5/5] common/multipath-over-rdma: allow to set use_siw Yi Zhang
4 siblings, 0 replies; 10+ messages in thread
From: Yi Zhang @ 2020-11-25 7:32 UTC (permalink / raw)
To: osandov, bvanassche; +Cc: linux-block, linux-nvme, sagi
With below commit, the version will be updated base on the tag
fbef6555 replace SNAPSHOT with auto-generated version string
To reproduce it:
$ ./check srp/015
common/rc: line 98: [: ip utility, iproute2-5.9.0: integer expression expected
Signed-off-by: Yi Zhang <yi.zhang@redhat.com>
---
common/rc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/common/rc b/common/rc
index 0d7a3cd..37eb873 100644
--- a/common/rc
+++ b/common/rc
@@ -95,7 +95,7 @@ _have_iproute2() {
SKIP_REASON="ip utility not found"
return 1
fi
- if [ "$snapshot" -lt "$1" ]; then
+ if [[ "$snapshot" =~ ^[0-9]+$ && "$snapshot" -lt "$1" ]]; then
SKIP_REASON="ip utility too old"
return 1
fi
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH V2 blktests 5/5] common/multipath-over-rdma: allow to set use_siw
2020-11-25 7:32 [PATCH V2 blktests 0/5] nvmeof-mp/srp/nvme-rdma misc fix and enhancement Yi Zhang
` (3 preceding siblings ...)
2020-11-25 7:32 ` [PATCH V2 blktests 4/5] common/rc: _have_iproute2 fix for "ip -V" change Yi Zhang
@ 2020-11-25 7:32 ` Yi Zhang
4 siblings, 0 replies; 10+ messages in thread
From: Yi Zhang @ 2020-11-25 7:32 UTC (permalink / raw)
To: osandov, bvanassche; +Cc: linux-block, linux-nvme, sagi
With this change, we can change to use siw for nvme-rdma/nvmeof-mp/srp
testing from cmdline:
$ use_siw=1 nvme-trtype=rdma ./check nvme/
$ use_siw=1 ./check nvmeof-mp/
$ use_siw=1 ./check srp/
Signed-off-by: Yi Zhang <yi.zhang@redhat.com>
---
common/multipath-over-rdma | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/common/multipath-over-rdma b/common/multipath-over-rdma
index 1bbefb7..d156a12 100644
--- a/common/multipath-over-rdma
+++ b/common/multipath-over-rdma
@@ -12,7 +12,7 @@ filesystem_type=ext4
fio_aux_path=/tmp/fio-state-files
memtotal=$(sed -n 's/^MemTotal:[[:blank:]]*\([0-9]*\)[[:blank:]]*kB$/\1/p' /proc/meminfo)
max_ramdisk_size=$((1<<25))
-use_siw=
+use_siw=${use_siw:-""}
ramdisk_size=$((memtotal*(1024/16))) # in bytes
if [ $ramdisk_size -gt $max_ramdisk_size ]; then
ramdisk_size=$max_ramdisk_size
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH V2 blktests 1/5] tests/srp/rc: update the ib_srpt module name
2020-11-25 7:32 ` [PATCH V2 blktests 1/5] tests/srp/rc: update the ib_srpt module name Yi Zhang
@ 2020-11-26 3:53 ` Bart Van Assche
0 siblings, 0 replies; 10+ messages in thread
From: Bart Van Assche @ 2020-11-26 3:53 UTC (permalink / raw)
To: Yi Zhang, osandov; +Cc: linux-block, linux-nvme, sagi
On 11/24/20 11:32 PM, Yi Zhang wrote:
> Fix the ib_srpt module insmod failure as the module in some distros are
> end with .xz, like bellow on fedora:
> /lib/modules/$(uname -r)/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko.xz
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH V2 blktests 2/5] tests/nvmeof-mp/rc: run nvmeof-mp tests if we set multipath=N
2020-11-25 7:32 ` [PATCH V2 blktests 2/5] tests/nvmeof-mp/rc: run nvmeof-mp tests if we set multipath=N Yi Zhang
@ 2020-11-26 3:54 ` Bart Van Assche
0 siblings, 0 replies; 10+ messages in thread
From: Bart Van Assche @ 2020-11-26 3:54 UTC (permalink / raw)
To: Yi Zhang, osandov; +Cc: linux-block, linux-nvme, sagi
On 11/24/20 11:32 PM, Yi Zhang wrote:
> To enable it, just do bellow step before we run it:
> $ echo "options nvme_core multipath=N" >/etc/modprobe.d/nvme.conf
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH V2 blktests 3/5] nvmeof-mp/012, srp/012: fix the scheduler list
2020-11-25 7:32 ` [PATCH V2 blktests 3/5] nvmeof-mp/012, srp/012: fix the scheduler list Yi Zhang
@ 2020-11-26 3:59 ` Bart Van Assche
2020-11-26 6:16 ` Yi Zhang
0 siblings, 1 reply; 10+ messages in thread
From: Bart Van Assche @ 2020-11-26 3:59 UTC (permalink / raw)
To: Yi Zhang, osandov; +Cc: linux-block, linux-nvme, sagi
On 11/24/20 11:32 PM, Yi Zhang wrote:
> +# Get block dev scheduler list
> +get_scheduler_list() {
> + local b=$1 p
> + p=/sys/block/"$b"/queue/scheduler
> + if [ -e "$p" ]; then
> + scheds=$(sed 's/[][]//g' /sys/block/"$b"/queue/scheduler)
> + echo "$scheds"
> + else
> + return 1
> + fi
> +}
Can the echo statement and the 'scheds' assignment be left out? This is
what I have in mind:
sed 's/[][]//g' /sys/block/"$b"/queue/scheduler
Otherwise this patch looks good to me.
Thanks,
Bart.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH V2 blktests 3/5] nvmeof-mp/012, srp/012: fix the scheduler list
2020-11-26 3:59 ` Bart Van Assche
@ 2020-11-26 6:16 ` Yi Zhang
0 siblings, 0 replies; 10+ messages in thread
From: Yi Zhang @ 2020-11-26 6:16 UTC (permalink / raw)
To: Bart Van Assche, osandov; +Cc: linux-block, linux-nvme, sagi
On 11/26/20 11:59 AM, Bart Van Assche wrote:
> On 11/24/20 11:32 PM, Yi Zhang wrote:
>> +# Get block dev scheduler list
>> +get_scheduler_list() {
>> + local b=$1 p
>> + p=/sys/block/"$b"/queue/scheduler
>> + if [ -e "$p" ]; then
>> + scheds=$(sed 's/[][]//g' /sys/block/"$b"/queue/scheduler)
>> + echo "$scheds"
>> + else
>> + return 1
>> + fi
>> +}
> Can the echo statement and the 'scheds' assignment be left out? This is
> what I have in mind:
Sure, will update with V3.
> sed 's/[][]//g' /sys/block/"$b"/queue/scheduler
>
> Otherwise this patch looks good to me.
>
> Thanks,
>
> Bart.
>
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2020-11-26 6:16 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-25 7:32 [PATCH V2 blktests 0/5] nvmeof-mp/srp/nvme-rdma misc fix and enhancement Yi Zhang
2020-11-25 7:32 ` [PATCH V2 blktests 1/5] tests/srp/rc: update the ib_srpt module name Yi Zhang
2020-11-26 3:53 ` Bart Van Assche
2020-11-25 7:32 ` [PATCH V2 blktests 2/5] tests/nvmeof-mp/rc: run nvmeof-mp tests if we set multipath=N Yi Zhang
2020-11-26 3:54 ` Bart Van Assche
2020-11-25 7:32 ` [PATCH V2 blktests 3/5] nvmeof-mp/012, srp/012: fix the scheduler list Yi Zhang
2020-11-26 3:59 ` Bart Van Assche
2020-11-26 6:16 ` Yi Zhang
2020-11-25 7:32 ` [PATCH V2 blktests 4/5] common/rc: _have_iproute2 fix for "ip -V" change Yi Zhang
2020-11-25 7:32 ` [PATCH V2 blktests 5/5] common/multipath-over-rdma: allow to set use_siw Yi Zhang
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).