All of lore.kernel.org
 help / color / mirror / Atom feed
* [SPDK] Re: NVMeofTCP SPDK host abort test
@ 2021-07-16 17:31 Harris, James R
  0 siblings, 0 replies; 3+ messages in thread
From: Harris, James R @ 2021-07-16 17:31 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 19431 bytes --]

Hi Gyan,

Thanks for your detailed report.  Could you file a GitHub issue for this?  https://github.com/spdk/spdk/issues

Thanks,

Jim


On 7/16/21, 10:26 AM, "Gyan Prakash" <gyapra2016(a)gmail.com> wrote:

    Hello all,

    I am using  NVMeofTCP SPDK host  and running IO abort test ( build/examples
    folder) to test my NVMeofTCP target.  I am running abort test with
    different queue depth like 4, 8, 16, 32.
    Test completes  fine for queue depth 4, 8, and 16 but it hangs and seems to
    be in some kind of continuous loop  for queue depth 32.

    In network trace, I see that for queue depth 32, host has sent a write
    command, target responded with r2t, host never sends the requested data.
    From the host console message, it looks like the host is in a continuous
    loop.

    I am providing the SPDK abort command console output for q depth (-q 4, -q
    16 and -q 32). Queue Depth 32 has so many errors on the host console and
    same error is being printed over and over. Please see below for more
    details.

    Can you please let me know how can we fix this

    Thanks,
    GP

    *with q depth = 32*
    ./abort -q 32 -s 4096 -w rw -M 50 -o 40960 -r 'trtype:tcp adrfam:IPv4
    traddr:10.10.10.167 trsvcid:4420 subnqn:nqn.2015-09.com.cdw:nvme.1'
    [2021-07-16 09:55:49.649538] Starting SPDK v21.07-pre git sha1 b73d3e689 /
    DPDK 21.02.0 initialization...
    [2021-07-16 09:55:49.649617] [ DPDK EAL parameters: [2021-07-16
    09:55:49.649630] abort [2021-07-16 09:55:49.649641] --no-shconf [2021-07-16
    09:55:49.649652] -c 0x1 [2021-07-16 09:55:49.649660] -m 4096 [2021-07-16
    09:55:49.649669] --no-pci [2021-07-16 09:55:49.649680]
    --log-level=lib.eal:6 [2021-07-16 09:55:49.649691]
    --log-level=lib.cryptodev:5 [2021-07-16 09:55:49.649702]
    --log-level=user1:6 [2021-07-16 09:55:49.649713] --iova-mode=pa [2021-07-16
    09:55:49.649724] --base-virtaddr=0x200000000000 [2021-07-16
    09:55:49.649736] --match-allocations [2021-07-16 09:55:49.649746]
    --file-prefix=spdk_pid126855 [2021-07-16 09:55:49.649758] ]
    EAL: No available 1048576 kB hugepages reported
    EAL: No legacy callbacks, legacy socket not created
    Initializing NVMe Controllers
    Attached to NVMe over Fabrics controller at 10.10.10.167:4420:
    nqn.2015-09.com.cdw:nvme.1
    controller IO queue size 16 less than required
    Consider using lower queue depth or small IO size because IO requests may
    be queued at the NVMe driver.
    Associating TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) NSID
    1 with lcore 0
    Initialization complete. Launching workers.
    [2021-07-16 09:55:52.997426] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997463] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997474] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22110 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997483] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997490] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997497] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997503] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22120 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997510] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997515] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997523] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997529] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22130 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997534] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997542] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997549] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997557] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22140 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997565] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997571] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997577] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997583] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22150 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997590] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997595] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997600] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997611] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22160 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997619] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997627] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997634] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997642] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22170 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997650] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997658] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997666] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997674] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22180 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997681] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997689] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997697] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997704] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22190 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997711] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997719] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997726] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997733] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22200 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997740] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997747] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997754] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997762] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22210 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997768] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997775] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997782] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997789] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22220 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997796] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997803] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997810] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997817] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22230 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997826] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997834] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997840] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997848] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22240 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997855] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997862] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997870] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997877] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22250 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997885] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997892] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997898] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997905] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22260 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997912] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997919] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997926] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997937] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22270 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997945] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997953] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997959] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997967] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22280 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.997974] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.997981] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.997989] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.997996] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22290 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.998003] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.998010] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.998017] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.998024] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22300 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.998032] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.998040] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.998047] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.998055] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22310 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.998062] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.998070] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.998076] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.998083] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22320 len:10 PRP1 0x0 PRP2 0x0
    [2021-07-16 09:55:52.998090] nvme_qpair.c: 455:spdk_nvme_print_completion:
    *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000 p:0 m:0
    dnr:1
    [2021-07-16 09:55:52.998097] nvme_qpair.c:
    594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
    [2021-07-16 09:55:52.998103] nvme_qpair.c:
    536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
    manually:
    [2021-07-16 09:55:52.998110] nvme_qpair.c: 272:nvme_io_qpair_print_command:
    *N


    *with q depth=4*

    ./abort -q 4 -s 4096 -w rw -M 50 -o 40960 -r 'trtype:tcp adrfam:IPv4
    traddr:10.10.10.167 trsvcid:4420 subnqn:nqn.2015-09.com.cdw:nvme.1'
    [2021-07-16 09:55:04.024076] Starting SPDK v21.07-pre git sha1 b73d3e689 /
    DPDK 21.02.0 initialization...
    [2021-07-16 09:55:04.024156] [ DPDK EAL parameters: [2021-07-16
    09:55:04.024169] abort [2021-07-16 09:55:04.024180] --no-shconf [2021-07-16
    09:55:04.024190] -c 0x1 [2021-07-16 09:55:04.024198] -m 4096 [2021-07-16
    09:55:04.024207] --no-pci [2021-07-16 09:55:04.024217]
    --log-level=lib.eal:6 [2021-07-16 09:55:04.024226]
    --log-level=lib.cryptodev:5 [2021-07-16 09:55:04.024235]
    --log-level=user1:6 [2021-07-16 09:55:04.024247] --iova-mode=pa [2021-07-16
    09:55:04.024257] --base-virtaddr=0x200000000000 [2021-07-16
    09:55:04.024268] --match-allocations [2021-07-16 09:55:04.024279]
    --file-prefix=spdk_pid126833 [2021-07-16 09:55:04.024289] ]
    EAL: No available 1048576 kB hugepages reported
    EAL: No legacy callbacks, legacy socket not created
    Initializing NVMe Controllers
    Attached to NVMe over Fabrics controller at 10.10.10.167:4420:
    nqn.2015-09.com.cdw:nvme.1
    Associating TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) NSID
    1 with lcore 0
    Initialization complete. Launching workers.
    NS: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) NSID 1 I/O
    completed: 30102, failed: 15
    CTRLR: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) abort
    submitted 45, failed to submit 30072
             success 15, unsuccess 30, failed 0


    *with q depth= 16*
    ./abort -q 16 -s 4096 -w rw -M 50 -o 40960 -r 'trtype:tcp adrfam:IPv4
    traddr:10.10.10.167 trsvcid:4420 subnqn:nqn.2015-09.com.c:nvme.1'
    [2021-07-16 09:55:32.400777] Starting SPDK v21.07-pre git sha1 b73d3e689 /
    DPDK 21.02.0 initialization...
    [2021-07-16 09:55:32.400855] [ DPDK EAL parameters: [2021-07-16
    09:55:32.400868] abort [2021-07-16 09:55:32.400876] --no-shconf [2021-07-16
    09:55:32.400886] -c 0x1 [2021-07-16 09:55:32.400896] -m 4096 [2021-07-16
    09:55:32.400905] --no-pci [2021-07-16 09:55:32.400915]
    --log-level=lib.eal:6 [2021-07-16 09:55:32.400924]
    --log-level=lib.cryptodev:5 [2021-07-16 09:55:32.400934]
    --log-level=user1:6 [2021-07-16 09:55:32.400944] --iova-mode=pa [2021-07-16
    09:55:32.400953] --base-virtaddr=0x200000000000 [2021-07-16
    09:55:32.400963] --match-allocations [2021-07-16 09:55:32.400971]
    --file-prefix=spdk_pid126846 [2021-07-16 09:55:32.400980] ]
    EAL: No available 1048576 kB hugepages reported
    EAL: No legacy callbacks, legacy socket not created
    Initializing NVMe Controllers
    Attached to NVMe over Fabrics controller at 10.10.10.167:4420:
    nqn.2015-09.com.cdw:nvme.1
    controller IO queue size 16 less than required
    Consider using lower queue depth or small IO size because IO requests may
    be queued at the NVMe driver.
    Associating TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) NSID
    1 with lcore 0
    Initialization complete. Launching workers.
    NS: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) NSID 1 I/O
    completed: 49920, failed: 16
    CTRLR: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) abort
    submitted 43, failed to submit 49893
             success 16, unsuccess 27, failed 0
    _______________________________________________
    SPDK mailing list -- spdk(a)lists.01.org
    To unsubscribe send an email to spdk-leave(a)lists.01.org


^ permalink raw reply	[flat|nested] 3+ messages in thread

* [SPDK] Re: NVMeofTCP SPDK host abort test
@ 2021-07-16 18:40 Gyan Prakash
  0 siblings, 0 replies; 3+ messages in thread
From: Gyan Prakash @ 2021-07-16 18:40 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 21421 bytes --]

Jim,

Have filed a GitHub issue for this, "
https://github.com/spdk/spdk/issues/2048"

It has been labelled as "Sightings" .

Thanks,
GP


On Fri, Jul 16, 2021 at 10:37 AM Gyan Prakash <gyapra2016(a)gmail.com> wrote:

> Jim,
>
> Sure, I will file a GitHub issue for this.
>
> Thanks,
> GP
>
> On Fri, Jul 16, 2021 at 10:32 AM Harris, James R <james.r.harris(a)intel.com>
> wrote:
>
>> Hi Gyan,
>>
>> Thanks for your detailed report.  Could you file a GitHub issue for
>> this?  https://github.com/spdk/spdk/issues
>>
>> Thanks,
>>
>> Jim
>>
>>
>> On 7/16/21, 10:26 AM, "Gyan Prakash" <gyapra2016(a)gmail.com> wrote:
>>
>>     Hello all,
>>
>>     I am using  NVMeofTCP SPDK host  and running IO abort test (
>> build/examples
>>     folder) to test my NVMeofTCP target.  I am running abort test with
>>     different queue depth like 4, 8, 16, 32.
>>     Test completes  fine for queue depth 4, 8, and 16 but it hangs and
>> seems to
>>     be in some kind of continuous loop  for queue depth 32.
>>
>>     In network trace, I see that for queue depth 32, host has sent a write
>>     command, target responded with r2t, host never sends the requested
>> data.
>>     From the host console message, it looks like the host is in a
>> continuous
>>     loop.
>>
>>     I am providing the SPDK abort command console output for q depth (-q
>> 4, -q
>>     16 and -q 32). Queue Depth 32 has so many errors on the host console
>> and
>>     same error is being printed over and over. Please see below for more
>>     details.
>>
>>     Can you please let me know how can we fix this
>>
>>     Thanks,
>>     GP
>>
>>     *with q depth = 32*
>>     ./abort -q 32 -s 4096 -w rw -M 50 -o 40960 -r 'trtype:tcp adrfam:IPv4
>>     traddr:10.10.10.167 trsvcid:4420 subnqn:nqn.2015-09.com.cdw:nvme.1'
>>     [2021-07-16 09:55:49.649538] Starting SPDK v21.07-pre git sha1
>> b73d3e689 /
>>     DPDK 21.02.0 initialization...
>>     [2021-07-16 09:55:49.649617] [ DPDK EAL parameters: [2021-07-16
>>     09:55:49.649630] abort [2021-07-16 09:55:49.649641] --no-shconf
>> [2021-07-16
>>     09:55:49.649652] -c 0x1 [2021-07-16 09:55:49.649660] -m 4096
>> [2021-07-16
>>     09:55:49.649669] --no-pci [2021-07-16 09:55:49.649680]
>>     --log-level=lib.eal:6 [2021-07-16 09:55:49.649691]
>>     --log-level=lib.cryptodev:5 [2021-07-16 09:55:49.649702]
>>     --log-level=user1:6 [2021-07-16 09:55:49.649713] --iova-mode=pa
>> [2021-07-16
>>     09:55:49.649724] --base-virtaddr=0x200000000000 [2021-07-16
>>     09:55:49.649736] --match-allocations [2021-07-16 09:55:49.649746]
>>     --file-prefix=spdk_pid126855 [2021-07-16 09:55:49.649758] ]
>>     EAL: No available 1048576 kB hugepages reported
>>     EAL: No legacy callbacks, legacy socket not created
>>     Initializing NVMe Controllers
>>     Attached to NVMe over Fabrics controller at 10.10.10.167:4420:
>>     nqn.2015-09.com.cdw:nvme.1
>>     controller IO queue size 16 less than required
>>     Consider using lower queue depth or small IO size because IO requests
>> may
>>     be queued at the NVMe driver.
>>     Associating TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1)
>> NSID
>>     1 with lcore 0
>>     Initialization complete. Launching workers.
>>     [2021-07-16 09:55:52.997426] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997463] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997474] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22110 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997483] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997490] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997497] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997503] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22120 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997510] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997515] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997523] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997529] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22130 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997534] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997542] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997549] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997557] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22140 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997565] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997571] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997577] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997583] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22150 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997590] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997595] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997600] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997611] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22160 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997619] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997627] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997634] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997642] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22170 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997650] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997658] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997666] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997674] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22180 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997681] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997689] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997697] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997704] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22190 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997711] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997719] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997726] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997733] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22200 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997740] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997747] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997754] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997762] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22210 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997768] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997775] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997782] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997789] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22220 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997796] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997803] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997810] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997817] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22230 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997826] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997834] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997840] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997848] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22240 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997855] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997862] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997870] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997877] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22250 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997885] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997892] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997898] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997905] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22260 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997912] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997919] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997926] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997937] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22270 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997945] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997953] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997959] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997967] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22280 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.997974] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.997981] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.997989] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.997996] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22290 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.998003] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.998010] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.998017] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.998024] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22300 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.998032] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.998040] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.998047] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.998055] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22310 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.998062] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.998070] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.998076] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.998083] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22320 len:10 PRP1 0x0 PRP2 0x0
>>     [2021-07-16 09:55:52.998090] nvme_qpair.c:
>> 455:spdk_nvme_print_completion:
>>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
>> p:0 m:0
>>     dnr:1
>>     [2021-07-16 09:55:52.998097] nvme_qpair.c:
>>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>>     [2021-07-16 09:55:52.998103] nvme_qpair.c:
>>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>>     manually:
>>     [2021-07-16 09:55:52.998110] nvme_qpair.c:
>> 272:nvme_io_qpair_print_command:
>>     *N
>>
>>
>>     *with q depth=4*
>>
>>     ./abort -q 4 -s 4096 -w rw -M 50 -o 40960 -r 'trtype:tcp adrfam:IPv4
>>     traddr:10.10.10.167 trsvcid:4420 subnqn:nqn.2015-09.com.cdw:nvme.1'
>>     [2021-07-16 09:55:04.024076] Starting SPDK v21.07-pre git sha1
>> b73d3e689 /
>>     DPDK 21.02.0 initialization...
>>     [2021-07-16 09:55:04.024156] [ DPDK EAL parameters: [2021-07-16
>>     09:55:04.024169] abort [2021-07-16 09:55:04.024180] --no-shconf
>> [2021-07-16
>>     09:55:04.024190] -c 0x1 [2021-07-16 09:55:04.024198] -m 4096
>> [2021-07-16
>>     09:55:04.024207] --no-pci [2021-07-16 09:55:04.024217]
>>     --log-level=lib.eal:6 [2021-07-16 09:55:04.024226]
>>     --log-level=lib.cryptodev:5 [2021-07-16 09:55:04.024235]
>>     --log-level=user1:6 [2021-07-16 09:55:04.024247] --iova-mode=pa
>> [2021-07-16
>>     09:55:04.024257] --base-virtaddr=0x200000000000 [2021-07-16
>>     09:55:04.024268] --match-allocations [2021-07-16 09:55:04.024279]
>>     --file-prefix=spdk_pid126833 [2021-07-16 09:55:04.024289] ]
>>     EAL: No available 1048576 kB hugepages reported
>>     EAL: No legacy callbacks, legacy socket not created
>>     Initializing NVMe Controllers
>>     Attached to NVMe over Fabrics controller at 10.10.10.167:4420:
>>     nqn.2015-09.com.cdw:nvme.1
>>     Associating TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1)
>> NSID
>>     1 with lcore 0
>>     Initialization complete. Launching workers.
>>     NS: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) NSID 1
>> I/O
>>     completed: 30102, failed: 15
>>     CTRLR: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) abort
>>     submitted 45, failed to submit 30072
>>              success 15, unsuccess 30, failed 0
>>
>>
>>     *with q depth= 16*
>>     ./abort -q 16 -s 4096 -w rw -M 50 -o 40960 -r 'trtype:tcp adrfam:IPv4
>>     traddr:10.10.10.167 trsvcid:4420 subnqn:nqn.2015-09.com.c:nvme.1'
>>     [2021-07-16 09:55:32.400777] Starting SPDK v21.07-pre git sha1
>> b73d3e689 /
>>     DPDK 21.02.0 initialization...
>>     [2021-07-16 09:55:32.400855] [ DPDK EAL parameters: [2021-07-16
>>     09:55:32.400868] abort [2021-07-16 09:55:32.400876] --no-shconf
>> [2021-07-16
>>     09:55:32.400886] -c 0x1 [2021-07-16 09:55:32.400896] -m 4096
>> [2021-07-16
>>     09:55:32.400905] --no-pci [2021-07-16 09:55:32.400915]
>>     --log-level=lib.eal:6 [2021-07-16 09:55:32.400924]
>>     --log-level=lib.cryptodev:5 [2021-07-16 09:55:32.400934]
>>     --log-level=user1:6 [2021-07-16 09:55:32.400944] --iova-mode=pa
>> [2021-07-16
>>     09:55:32.400953] --base-virtaddr=0x200000000000 [2021-07-16
>>     09:55:32.400963] --match-allocations [2021-07-16 09:55:32.400971]
>>     --file-prefix=spdk_pid126846 [2021-07-16 09:55:32.400980] ]
>>     EAL: No available 1048576 kB hugepages reported
>>     EAL: No legacy callbacks, legacy socket not created
>>     Initializing NVMe Controllers
>>     Attached to NVMe over Fabrics controller at 10.10.10.167:4420:
>>     nqn.2015-09.com.cdw:nvme.1
>>     controller IO queue size 16 less than required
>>     Consider using lower queue depth or small IO size because IO requests
>> may
>>     be queued at the NVMe driver.
>>     Associating TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1)
>> NSID
>>     1 with lcore 0
>>     Initialization complete. Launching workers.
>>     NS: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) NSID 1
>> I/O
>>     completed: 49920, failed: 16
>>     CTRLR: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) abort
>>     submitted 43, failed to submit 49893
>>              success 16, unsuccess 27, failed 0
>>     _______________________________________________
>>     SPDK mailing list -- spdk(a)lists.01.org
>>     To unsubscribe send an email to spdk-leave(a)lists.01.org
>>
>> _______________________________________________
>> SPDK mailing list -- spdk(a)lists.01.org
>> To unsubscribe send an email to spdk-leave(a)lists.01.org
>>
>

^ permalink raw reply	[flat|nested] 3+ messages in thread

* [SPDK] Re: NVMeofTCP SPDK host abort test
@ 2021-07-16 17:37 Gyan Prakash
  0 siblings, 0 replies; 3+ messages in thread
From: Gyan Prakash @ 2021-07-16 17:37 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 20718 bytes --]

Jim,

Sure, I will file a GitHub issue for this.

Thanks,
GP

On Fri, Jul 16, 2021 at 10:32 AM Harris, James R <james.r.harris(a)intel.com>
wrote:

> Hi Gyan,
>
> Thanks for your detailed report.  Could you file a GitHub issue for this?
> https://github.com/spdk/spdk/issues
>
> Thanks,
>
> Jim
>
>
> On 7/16/21, 10:26 AM, "Gyan Prakash" <gyapra2016(a)gmail.com> wrote:
>
>     Hello all,
>
>     I am using  NVMeofTCP SPDK host  and running IO abort test (
> build/examples
>     folder) to test my NVMeofTCP target.  I am running abort test with
>     different queue depth like 4, 8, 16, 32.
>     Test completes  fine for queue depth 4, 8, and 16 but it hangs and
> seems to
>     be in some kind of continuous loop  for queue depth 32.
>
>     In network trace, I see that for queue depth 32, host has sent a write
>     command, target responded with r2t, host never sends the requested
> data.
>     From the host console message, it looks like the host is in a
> continuous
>     loop.
>
>     I am providing the SPDK abort command console output for q depth (-q
> 4, -q
>     16 and -q 32). Queue Depth 32 has so many errors on the host console
> and
>     same error is being printed over and over. Please see below for more
>     details.
>
>     Can you please let me know how can we fix this
>
>     Thanks,
>     GP
>
>     *with q depth = 32*
>     ./abort -q 32 -s 4096 -w rw -M 50 -o 40960 -r 'trtype:tcp adrfam:IPv4
>     traddr:10.10.10.167 trsvcid:4420 subnqn:nqn.2015-09.com.cdw:nvme.1'
>     [2021-07-16 09:55:49.649538] Starting SPDK v21.07-pre git sha1
> b73d3e689 /
>     DPDK 21.02.0 initialization...
>     [2021-07-16 09:55:49.649617] [ DPDK EAL parameters: [2021-07-16
>     09:55:49.649630] abort [2021-07-16 09:55:49.649641] --no-shconf
> [2021-07-16
>     09:55:49.649652] -c 0x1 [2021-07-16 09:55:49.649660] -m 4096
> [2021-07-16
>     09:55:49.649669] --no-pci [2021-07-16 09:55:49.649680]
>     --log-level=lib.eal:6 [2021-07-16 09:55:49.649691]
>     --log-level=lib.cryptodev:5 [2021-07-16 09:55:49.649702]
>     --log-level=user1:6 [2021-07-16 09:55:49.649713] --iova-mode=pa
> [2021-07-16
>     09:55:49.649724] --base-virtaddr=0x200000000000 [2021-07-16
>     09:55:49.649736] --match-allocations [2021-07-16 09:55:49.649746]
>     --file-prefix=spdk_pid126855 [2021-07-16 09:55:49.649758] ]
>     EAL: No available 1048576 kB hugepages reported
>     EAL: No legacy callbacks, legacy socket not created
>     Initializing NVMe Controllers
>     Attached to NVMe over Fabrics controller at 10.10.10.167:4420:
>     nqn.2015-09.com.cdw:nvme.1
>     controller IO queue size 16 less than required
>     Consider using lower queue depth or small IO size because IO requests
> may
>     be queued at the NVMe driver.
>     Associating TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1)
> NSID
>     1 with lcore 0
>     Initialization complete. Launching workers.
>     [2021-07-16 09:55:52.997426] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997463] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997474] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22110 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997483] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997490] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997497] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997503] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22120 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997510] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997515] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997523] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997529] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22130 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997534] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997542] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997549] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997557] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22140 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997565] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997571] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997577] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997583] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22150 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997590] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997595] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997600] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997611] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22160 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997619] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997627] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997634] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997642] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22170 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997650] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997658] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997666] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997674] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22180 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997681] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997689] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997697] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997704] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22190 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997711] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997719] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997726] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997733] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22200 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997740] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997747] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997754] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997762] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22210 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997768] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997775] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997782] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997789] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22220 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997796] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997803] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997810] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997817] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22230 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997826] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997834] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997840] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997848] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22240 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997855] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997862] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997870] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997877] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22250 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997885] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997892] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997898] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997905] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22260 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997912] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997919] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997926] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997937] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22270 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997945] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997953] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997959] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997967] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22280 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.997974] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.997981] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.997989] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.997996] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22290 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.998003] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.998010] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.998017] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.998024] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: READ sqid:1 cid:0 nsid:1 lba:22300 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.998032] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.998040] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.998047] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.998055] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22310 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.998062] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.998070] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.998076] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.998083] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *NOTICE*: WRITE sqid:1 cid:0 nsid:1 lba:22320 len:10 PRP1 0x0 PRP2 0x0
>     [2021-07-16 09:55:52.998090] nvme_qpair.c:
> 455:spdk_nvme_print_completion:
>     *NOTICE*: ABORTED - BY REQUEST (00/07) qid:1 cid:0 cdw0:0 sqhd:0000
> p:0 m:0
>     dnr:1
>     [2021-07-16 09:55:52.998097] nvme_qpair.c:
>     594:nvme_qpair_abort_queued_reqs: *ERROR*: aborting queued i/o
>     [2021-07-16 09:55:52.998103] nvme_qpair.c:
>     536:nvme_qpair_manual_complete_request: *NOTICE*: Command completed
>     manually:
>     [2021-07-16 09:55:52.998110] nvme_qpair.c:
> 272:nvme_io_qpair_print_command:
>     *N
>
>
>     *with q depth=4*
>
>     ./abort -q 4 -s 4096 -w rw -M 50 -o 40960 -r 'trtype:tcp adrfam:IPv4
>     traddr:10.10.10.167 trsvcid:4420 subnqn:nqn.2015-09.com.cdw:nvme.1'
>     [2021-07-16 09:55:04.024076] Starting SPDK v21.07-pre git sha1
> b73d3e689 /
>     DPDK 21.02.0 initialization...
>     [2021-07-16 09:55:04.024156] [ DPDK EAL parameters: [2021-07-16
>     09:55:04.024169] abort [2021-07-16 09:55:04.024180] --no-shconf
> [2021-07-16
>     09:55:04.024190] -c 0x1 [2021-07-16 09:55:04.024198] -m 4096
> [2021-07-16
>     09:55:04.024207] --no-pci [2021-07-16 09:55:04.024217]
>     --log-level=lib.eal:6 [2021-07-16 09:55:04.024226]
>     --log-level=lib.cryptodev:5 [2021-07-16 09:55:04.024235]
>     --log-level=user1:6 [2021-07-16 09:55:04.024247] --iova-mode=pa
> [2021-07-16
>     09:55:04.024257] --base-virtaddr=0x200000000000 [2021-07-16
>     09:55:04.024268] --match-allocations [2021-07-16 09:55:04.024279]
>     --file-prefix=spdk_pid126833 [2021-07-16 09:55:04.024289] ]
>     EAL: No available 1048576 kB hugepages reported
>     EAL: No legacy callbacks, legacy socket not created
>     Initializing NVMe Controllers
>     Attached to NVMe over Fabrics controller at 10.10.10.167:4420:
>     nqn.2015-09.com.cdw:nvme.1
>     Associating TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1)
> NSID
>     1 with lcore 0
>     Initialization complete. Launching workers.
>     NS: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) NSID 1
> I/O
>     completed: 30102, failed: 15
>     CTRLR: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) abort
>     submitted 45, failed to submit 30072
>              success 15, unsuccess 30, failed 0
>
>
>     *with q depth= 16*
>     ./abort -q 16 -s 4096 -w rw -M 50 -o 40960 -r 'trtype:tcp adrfam:IPv4
>     traddr:10.10.10.167 trsvcid:4420 subnqn:nqn.2015-09.com.c:nvme.1'
>     [2021-07-16 09:55:32.400777] Starting SPDK v21.07-pre git sha1
> b73d3e689 /
>     DPDK 21.02.0 initialization...
>     [2021-07-16 09:55:32.400855] [ DPDK EAL parameters: [2021-07-16
>     09:55:32.400868] abort [2021-07-16 09:55:32.400876] --no-shconf
> [2021-07-16
>     09:55:32.400886] -c 0x1 [2021-07-16 09:55:32.400896] -m 4096
> [2021-07-16
>     09:55:32.400905] --no-pci [2021-07-16 09:55:32.400915]
>     --log-level=lib.eal:6 [2021-07-16 09:55:32.400924]
>     --log-level=lib.cryptodev:5 [2021-07-16 09:55:32.400934]
>     --log-level=user1:6 [2021-07-16 09:55:32.400944] --iova-mode=pa
> [2021-07-16
>     09:55:32.400953] --base-virtaddr=0x200000000000 [2021-07-16
>     09:55:32.400963] --match-allocations [2021-07-16 09:55:32.400971]
>     --file-prefix=spdk_pid126846 [2021-07-16 09:55:32.400980] ]
>     EAL: No available 1048576 kB hugepages reported
>     EAL: No legacy callbacks, legacy socket not created
>     Initializing NVMe Controllers
>     Attached to NVMe over Fabrics controller at 10.10.10.167:4420:
>     nqn.2015-09.com.cdw:nvme.1
>     controller IO queue size 16 less than required
>     Consider using lower queue depth or small IO size because IO requests
> may
>     be queued at the NVMe driver.
>     Associating TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1)
> NSID
>     1 with lcore 0
>     Initialization complete. Launching workers.
>     NS: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) NSID 1
> I/O
>     completed: 49920, failed: 16
>     CTRLR: TCP (addr:10.10.10.167 subnqn:nqn.2015-09.com.cdw:nvme.1) abort
>     submitted 43, failed to submit 49893
>              success 16, unsuccess 27, failed 0
>     _______________________________________________
>     SPDK mailing list -- spdk(a)lists.01.org
>     To unsubscribe send an email to spdk-leave(a)lists.01.org
>
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
>

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2021-07-16 18:40 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-16 17:31 [SPDK] Re: NVMeofTCP SPDK host abort test Harris, James R
2021-07-16 17:37 Gyan Prakash
2021-07-16 18:40 Gyan Prakash

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.