[SPDK] הנושא: IO error for application targets

* [SPDK] הנושא:  IO error for application targets
@ 2018-03-19 19:26 Shahar Salzman
  0 siblings, 0 replies; 3+ messages in thread
From: Shahar Salzman @ 2018-03-19 19:26 UTC (permalink / raw)
  To: spdk

[-- Attachment #1: Type: text/plain, Size: 3475 bytes --]

Hi Daniel,

Thanks for your help!
I will see if I can add some additional info that will "convince" the host not to hang on the device on the one hand but on the other hand, to fail the IO instead of returning an invalid buffer.
Some details on the setup: spdk is used as a target for our existing storage appliance (we were scsi only using scst up to now), integration is done via a bdev module which passes IO from spdk to/from the appliance using rings. We would be happy to share this bdev module if there is any interest in it.
Host is a standard centos using Linux nvme initiator, we simply did normal nvme discover/connect, and processes started to hang on nvme list.
Connectivity is 25g on the target going through a 100g switch, initiator is a VM using an sriov allocated port.

Shahar

-------- ����� ������ --------
����: Re: [SPDK] IO error for application targets
���: "Verkamp, Daniel"
��: Storage Performance Development Kit
����:

Hi Shahar,

The IO_STATUS_SCSI_ERROR and IO_STATUS_NVME_ERROR status codes should be set via spdk_bdev_io_complete_scsi_status() and spdk_bdev_io_complete_nvme_status(), respectively.  They can be used to store additional error information that the application can retrieve via spdk_bdev_io_get_scsi_status() and spdk_bdev_io_get_nvme_status().

What setup are you testing with dd? Is this exporting a bdev via iscsi_tgt, nvmf_tgt, or some other scenario?  Depending on the SPDK application, the extra error information you report might or might not be utilized.

Thanks,
-- Daniel

From: SPDK [mailto:spdk-bounces(a)lists.01.org] On Behalf Of Shahar Salzman
Sent: Monday, March 19, 2018 8:17 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] IO error for application targets

Hi,

I am working on some application error scenarios and am wondering what return codes I should use.

I am running my own blockdev which routes IO to a userspace application, I am currently looking at a scenario where the NSID does not exist, but am also thinking about how to handle an IO which is out of bounds, and other such scenarios.

Looking at the blockdev completions, I can see only a limited set of IO errors I can return:
enum spdk_bdev_io_status {
/*
* NOMEM should be returned when a bdev module cannot start an I/O because of
*  some lack of resources.  It may not be returned for RESET I/O.  I/O completed
*  with NOMEM status will be retried after some I/O from the same channel have
*  completed.
*/
SPDK_BDEV_IO_STATUS_NOMEM = -4,
SPDK_BDEV_IO_STATUS_SCSI_ERROR = -3,
SPDK_BDEV_IO_STATUS_NVME_ERROR = -2,
SPDK_BDEV_IO_STATUS_FAILED = -1,
SPDK_BDEV_IO_STATUS_PENDING = 0,
SPDK_BDEV_IO_STATUS_SUCCESS = 1,
};

I tried both FAILED and NVME_ERROR running dd (read) to an invalid device, and each status gave me different results:

FAILED - process hangs in D state for 30 seconds, and eventually returns an IO error

NVME_ERROR - process returns immediately with some data (doesn't seem random, not sure if it what the application had in its buffer or if some transaction had been made)

I obviously prefer the FAILED return code, but I also get a blkid hung in D state, and other operations (e.g. disconnect) take a very long time.

What does the NVME_ERROR mean? Is there a way to give some more error information, e.g. something like enodev?

Thanks,

Shahar

[-- Attachment #2: attachment.html --]
[-- Type: text/html, Size: 9519 bytes --]

^ permalink raw reply	[flat|nested] 3+ messages in thread