From: Chaitanya Kulkarni Hi, The recent commit uses a combination of command id and a gencounter to calculate the command id so that we can call out buggy controllers for spurious completions and that also avoids use after free. This commit adds various if statements and bitwise operations such as &, <<, >>, & along with comparison operations to validate the   gencounter and print out the errors.   This feature is required to catch the buggy controller, but in the   production environment where controller is validated and known to be stable this achieves nothing but adds additional code and runtime CPU   instructions for the PCIe controller. Worst case scenario when NVMeOF setup is using passthru backend then these instructions gets duplicated on the host and on the target side for each controller.   This patch-series makes the gencounter feature configurable by   defining a new KConfig macro CONFIG_NVME_DEBUG_USE_CID_GENCTR and adding a new KConfig section Debug with subsection for enabling the   gencounter debug feature. We move the current tag + gencounter code when CONFIG_NVME_DEBUG_USE_CID_GENCTR is defined and keep the original code where we use tag based command id where CONFIG_NVME_DEBUG_USE_CID_GENCTR is not defined. The newly added Debug section is kept under NVMe Driver section under host side.   In this way we achieve:- 1. Removing the entire gencounter code at the compile-time and keeping the      fastpath as lean as possible when the controller is known to be stable. 2. Retaining the gencounter behavior by enabling the debug feature. 3. Letting user decide which debug features to enable instead of forcing    any. 4. Avoid measurable differences in fio performance numbers. see below. * fio BW/iops/slat cid gencounter on avg IOPS = 75k, BW = 295MiB :- read: IOPS=75.3k, BW=294MiB/s read: IOPS=75.8k, BW=296MiB/s read: IOPS=75.2k, BW=294MiB/s read: IOPS=76.0k, BW=297MiB/s read: IOPS=75.4k, BW=294MiB/s read: IOPS=76.0k, BW=297MiB/s read: IOPS=75.5k, BW=295MiB/s read: IOPS=75.4k, BW=294MiB/s Submission latency avg = 6698 nsec :- slat (nsec): avg=5057.99 slat (nsec): avg=7434.15 slat (nsec): avg=6310.50 slat (nsec): avg=8083.82 slat (nsec): avg=7297.77 slat (nsec): avg=6003.77 slat (nsec): avg=6657.75 slat (nsec): avg=6745.34 * fio BW/iops/slat cid gencounter off IOPS avg = 76k, BW = 297MiB :- read: IOPS=77.7k, BW=304MiB/s read: IOPS=76.5k, BW=299MiB/s read: IOPS=74.1k, BW=290MiB/s read: IOPS=75.8k, BW=296MiB/s read: IOPS=75.9k, BW=297MiB/s read: IOPS=75.2k, BW=294MiB/s read: IOPS=77.6k, BW=303MiB/s read: IOPS=76.8k, BW=300MiB/s Submission latency avg = 5030 nsec :- slat (nsec): avg=3859.20 slat (nsec): avg=4669.78 slat (nsec): avg=3778.51 slat (nsec): avg=8436.18 slat (nsec): avg=5653.83 slat (nsec): avg=5286.60 slat (nsec): avg=4176.05 slat (nsec): avg=4385.19 -ck Chaitanya Kulkarni (3): nvme-core: make cid gencounter configurable nvme-core: move gencounter check into nvme_cid() nvme: add KConfig options for debug features drivers/nvme/host/Kconfig | 10 ++++++++++ drivers/nvme/host/core.c | 3 --- drivers/nvme/host/nvme.h | 31 +++++++++++++++++++++++++++++++ 3 files changed, 41 insertions(+), 3 deletions(-) -- 2.29.0 Test Log :- root@dev nvme (nvme-5.16) # ./compile_nvme.sh + umount /mnt/nvme0n1 + clear_dmesg ./compile_nvme.sh: line 3: clear_dmesg: command not found umount: /mnt/nvme0n1: no mount point specified. + modprobe -r nvme-fabrics + modprobe -r nvme_loop + modprobe -r nvmet + modprobe -r nvme + sleep 1 + modprobe -r nvme-core + lsmod + grep nvme + sleep 1 + git diff + sleep 1 ++ nproc + make -j 48 M=drivers/nvme/ modules MODPOST drivers/nvme/Module.symvers CC [M] drivers/nvme/host/nvme-core.mod.o CC [M] drivers/nvme/host/nvme-fabrics.mod.o CC [M] drivers/nvme/host/nvme-fc.mod.o CC [M] drivers/nvme/host/nvme-rdma.mod.o CC [M] drivers/nvme/host/nvme-tcp.mod.o CC [M] drivers/nvme/host/nvme.mod.o CC [M] drivers/nvme/target/nvme-fcloop.mod.o CC [M] drivers/nvme/target/nvme-loop.mod.o CC [M] drivers/nvme/target/nvmet-fc.mod.o CC [M] drivers/nvme/target/nvmet-rdma.mod.o CC [M] drivers/nvme/target/nvmet-tcp.mod.o CC [M] drivers/nvme/target/nvmet.mod.o LD [M] drivers/nvme/host/nvme-rdma.ko LD [M] drivers/nvme/host/nvme-core.ko LD [M] drivers/nvme/host/nvme-fc.ko LD [M] drivers/nvme/target/nvme-loop.ko LD [M] drivers/nvme/target/nvmet.ko LD [M] drivers/nvme/target/nvmet-rdma.ko LD [M] drivers/nvme/host/nvme-tcp.ko LD [M] drivers/nvme/target/nvme-fcloop.ko LD [M] drivers/nvme/target/nvmet-fc.ko LD [M] drivers/nvme/target/nvmet-tcp.ko LD [M] drivers/nvme/host/nvme-fabrics.ko LD [M] drivers/nvme/host/nvme.ko BTF [M] drivers/nvme/target/nvme-loop.ko BTF [M] drivers/nvme/host/nvme-rdma.ko BTF [M] drivers/nvme/target/nvme-fcloop.ko BTF [M] drivers/nvme/host/nvme-fabrics.ko BTF [M] drivers/nvme/target/nvmet-rdma.ko BTF [M] drivers/nvme/target/nvmet-tcp.ko BTF [M] drivers/nvme/host/nvme.ko BTF [M] drivers/nvme/host/nvme-tcp.ko BTF [M] drivers/nvme/target/nvmet-fc.ko BTF [M] drivers/nvme/host/nvme-fc.ko BTF [M] drivers/nvme/host/nvme-core.ko BTF [M] drivers/nvme/target/nvmet.ko + HOST=drivers/nvme/host + TARGET=drivers/nvme/target ++ uname -r + HOST_DEST=/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/ ++ uname -r + TARGET_DEST=/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target/ + cp drivers/nvme/host/nvme-core.ko drivers/nvme/host/nvme-fabrics.ko drivers/nvme/host/nvme-fc.ko drivers/nvme/host/nvme.ko drivers/nvme/host/nvme-rdma.ko drivers/nvme/host/nvme-tcp.ko /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host// + cp drivers/nvme/target/nvme-fcloop.ko drivers/nvme/target/nvme-loop.ko drivers/nvme/target/nvmet-fc.ko drivers/nvme/target/nvmet.ko drivers/nvme/target/nvmet-rdma.ko drivers/nvme/target/nvmet-tcp.ko /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target// + ls -lrth /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/ /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target// /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/: total 6.6M -rw-r--r--. 1 root root 2.7M Dec 10 01:49 nvme-core.ko -rw-r--r--. 1 root root 446K Dec 10 01:49 nvme-fabrics.ko -rw-r--r--. 1 root root 964K Dec 10 01:49 nvme-fc.ko -rw-r--r--. 1 root root 732K Dec 10 01:49 nvme.ko -rw-r--r--. 1 root root 1019K Dec 10 01:49 nvme-rdma.ko -rw-r--r--. 1 root root 893K Dec 10 01:49 nvme-tcp.ko /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target//: total 6.7M -rw-r--r--. 1 root root 498K Dec 10 01:49 nvme-fcloop.ko -rw-r--r--. 1 root root 435K Dec 10 01:49 nvme-loop.ko -rw-r--r--. 1 root root 767K Dec 10 01:49 nvmet-fc.ko -rw-r--r--. 1 root root 3.3M Dec 10 01:49 nvmet.ko -rw-r--r--. 1 root root 992K Dec 10 01:49 nvmet-rdma.ko -rw-r--r--. 1 root root 768K Dec 10 01:49 nvmet-tcp.ko + modprobe nvme root@dev nvme (nvme-5.16) # grep NVME_DEBUG .config # CONFIG_NVME_DEBUG_USE_CID_GENCTR is not set (reverse-i-search)`for': mv 0001-nvme-code-command_id-with-a-genctr-^Cr-use-after-fre.patch sagi-genctr.patch root@dev nvme (nvme-5.16) # ./passthru_config.sh /dev/nvme0 umount: /mnt/nvme0n1: no mount point specified. + mkdir /sys/kernel/config/nvmet/subsystems/pt-nqn + sleep 1 + echo 'Initializing passthru ctrl path ...' Initializing passthru ctrl path ... + echo -n /dev/nvme0 + sleep 1 + echo 1 + echo 1 + mkdir /sys/kernel/config/nvmet/ports/1/ + sleep 1 + echo -n loop + sleep 1 + echo 'Connecting passthru ctrl to the port ' Connecting passthru ctrl to the port + ln -s /sys/kernel/config/nvmet/subsystems/pt-nqn /sys/kernel/config/nvmet/ports/1/subsystems/ + sleep 1 + nvme connect -t loop -n pt-nqn + sleep 1 + tr -s ' ' ' ' + set +x root@dev nvme (nvme-5.16) # for i in `seq 1 10`; do fio fio/randread.fio --filename=/dev/nvme1n1 --output=no-cid-genctr-${i}.fio; done root@dev nvme (nvme-5.16) # makemnconfig configuration written to .config *** End of the configuration. *** Execute 'make' to start the build or try 'make help'. root@dev nvme (nvme-5.16) # root@dev nvme (nvme-5.16) # root@dev nvme (nvme-5.16) # root@dev nvme (nvme-5.16) # root@dev nvme (nvme-5.16) # root@dev nvme (nvme-5.16) # ./kernel_compile.sh DEPMOD /lib/modules/5.16.0-rc1nvme+ sh ./arch/x86/boot/install.sh 5.16.0-rc1nvme+ \ arch/x86/boot/bzImage System.map "/boot" Generating grub configuration file ... done real 1m33.758s user 4m54.633s sys 1m50.129s root@dev nvme (nvme-5.16) # reboot Connection to 192.168.0.46 closed by remote host. Connection to 192.168.0.46 closed. [neo@fedora ~]$ dev root@192.168.0.46's password: Last login: Fri Dec 10 01:49:40 2021 from 192.168.0.63 root@dev ~ # cd nvme/ root@dev nvme (nvme-5.16) # ./compile_nvme.sh + umount /mnt/nvme0n1 + clear_dmesg ./compile_nvme.sh: line 3: clear_dmesg: command not found umount: /mnt/nvme0n1: no mount point specified. + modprobe -r nvme-fabrics + modprobe -r nvme_loop + modprobe -r nvmet + modprobe -r nvme + sleep 1 + modprobe -r nvme-core + lsmod + grep nvme + sleep 1 ./+ git diff + sleep 1 ++ nproc + make -j 48 M=drivers/nvme/ modules MODPOST drivers/nvme/Module.symvers CC [M] drivers/nvme/host/nvme-core.mod.o CC [M] drivers/nvme/host/nvme-fabrics.mod.o CC [M] drivers/nvme/host/nvme-fc.mod.o CC [M] drivers/nvme/host/nvme-rdma.mod.o CC [M] drivers/nvme/host/nvme-tcp.mod.o CC [M] drivers/nvme/host/nvme.mod.o CC [M] drivers/nvme/target/nvme-fcloop.mod.o CC [M] drivers/nvme/target/nvme-loop.mod.o CC [M] drivers/nvme/target/nvmet-fc.mod.o CC [M] drivers/nvme/target/nvmet-rdma.mod.o CC [M] drivers/nvme/target/nvmet-tcp.mod.o CC [M] drivers/nvme/target/nvmet.mod.o LD [M] drivers/nvme/target/nvme-loop.ko LD [M] drivers/nvme/target/nvmet.ko LD [M] drivers/nvme/host/nvme.ko LD [M] drivers/nvme/target/nvmet-rdma.ko LD [M] drivers/nvme/host/nvme-rdma.ko LD [M] drivers/nvme/host/nvme-tcp.ko LD [M] drivers/nvme/host/nvme-core.ko LD [M] drivers/nvme/target/nvmet-fc.ko LD [M] drivers/nvme/target/nvmet-tcp.ko LD [M] drivers/nvme/host/nvme-fabrics.ko LD [M] drivers/nvme/host/nvme-fc.ko BTF [M] drivers/nvme/target/nvme-loop.ko LD [M] drivers/nvme/target/nvme-fcloop.ko BTF [M] drivers/nvme/target/nvmet-rdma.ko BTF [M] drivers/nvme/host/nvme-fabrics.ko BTF [M] drivers/nvme/target/nvme-fcloop.ko BTF [M] drivers/nvme/host/nvme-rdma.ko BTF [M] drivers/nvme/host/nvme-tcp.ko BTF [M] drivers/nvme/target/nvmet-fc.ko BTF [M] drivers/nvme/target/nvmet-tcp.ko BTF [M] drivers/nvme/host/nvme.ko BTF [M] drivers/nvme/host/nvme-fc.ko BTF [M] drivers/nvme/target/nvmet.ko BTF [M] drivers/nvme/host/nvme-core.ko + HOST=drivers/nvme/host + TARGET=drivers/nvme/target ++ uname -r + HOST_DEST=/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/ ++ uname -r + TARGET_DEST=/lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target/ + cp drivers/nvme/host/nvme-core.ko drivers/nvme/host/nvme-fabrics.ko drivers/nvme/host/nvme-fc.ko drivers/nvme/host/nvme.ko drivers/nvme/host/nvme-rdma.ko drivers/nvme/host/nvme-tcp.ko /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host// + cp drivers/nvme/target/nvme-fcloop.ko drivers/nvme/target/nvme-loop.ko drivers/nvme/target/nvmet-fc.ko drivers/nvme/target/nvmet.ko drivers/nvme/target/nvmet-rdma.ko drivers/nvme/target/nvmet-tcp.ko /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target// + ls -lrth /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/ /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target// /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/host/: total 6.6M -rw-r--r--. 1 root root 2.7M Dec 10 02:17 nvme-core.ko -rw-r--r--. 1 root root 446K Dec 10 02:17 nvme-fabrics.ko -rw-r--r--. 1 root root 964K Dec 10 02:17 nvme-fc.ko -rw-r--r--. 1 root root 732K Dec 10 02:17 nvme.ko -rw-r--r--. 1 root root 1019K Dec 10 02:17 nvme-rdma.ko -rw-r--r--. 1 root root 897K Dec 10 02:17 nvme-tcp.ko /lib/modules/5.16.0-rc1nvme+/kernel/drivers/nvme/target//: total 6.7M -rw-r--r--. 1 root root 498K Dec 10 02:17 nvme-fcloop.ko -rw-r--r--. 1 root root 436K Dec 10 02:17 nvme-loop.ko -rw-r--r--. 1 root root 767K Dec 10 02:17 nvmet-fc.ko -rw-r--r--. 1 root root 3.3M Dec 10 02:17 nvmet.ko -rw-r--r--. 1 root root 992K Dec 10 02:17 nvmet-rdma.ko -rw-r--r--. 1 root root 768K Dec 10 02:17 nvmet-tcp.ko + modprobe nvme root@dev nvme (nvme-5.16) # ./passthru_config.sh /dev/nvme0 umount: /mnt/nvme0n1: no mount point specified. + mkdir /sys/kernel/config/nvmet/subsystems/pt-nqn + sleep 1 + echo 'Initializing passthru ctrl path ...' Initializing passthru ctrl path ... + echo -n /dev/nvme0 + sleep 1 + echo 1 + echo 1 + mkdir /sys/kernel/config/nvmet/ports/1/ + sleep 1 + echo -n loop + sleep 1 + echo 'Connecting passthru ctrl to the port ' Connecting passthru ctrl to the port + ln -s /sys/kernel/config/nvmet/subsystems/pt-nqn /sys/kernel/config/nvmet/ports/1/subsystems/ + sleep 1 + nvme connect -t loop -n pt-nqn + sleep 1 root@dev nvme (nvme-5.16) # grep NVME_DEBUG .config CONFIG_NVME_DEBUG_USE_CID_GENCTR=y root@dev nvme (nvme-5.16) # for i in `seq 1 10`; do fio fio/randread.fio --filename=/dev/nvme1n1 --output=yes-cid-genctr-${i}.fio; done root@dev nvme (nvme-5.16) # .0%][r=297MiB/s][r=76.0k IOPS][eta 00m:00s] root@dev nvme (nvme-5.16) # root@dev nvme (nvme-5.16) # root@dev nvme (nvme-5.16) # root@dev nvme (nvme-5.16) # grep slat yes*genctr* | awk '{print $6}' | cut -f 2 -d '=' | cut -f1 -d ',' 5057.99 7434.15 6310.50 8083.82 7297.77 6003.77 6657.75 6745.34 root@dev nvme (nvme-5.16) # grep slat no*genctr* | awk '{print $6}' | cut -f 2 -d '=' | cut -f1 -d ',' 3859.20 4669.78 3778.51 8436.18 5653.83 5286.60 4176.05 4385.19 root@dev nvme (nvme-5.16) # sum=0;for i in `grep slat yes-cid-genctr-* | awk '{print $6}' | cut -f 2 -d '=' | cut -f1 -d ','| cut -f 1 -d '.'`; do let sum=sum+$i; done ; echo $sum/8|bc 6698 root@dev nvme (nvme-5.16) # sum=0;for i in `grep slat no-cid-genctr-* | awk '{print $6}' | cut -f 2 -d '=' | cut -f1 -d ','| cut -f 1 -d '.'`; do let sum=sum+$i; done ; echo $sum/8|bc 5030 root@dev nvme (nvme-5.16) #