Re: [SPDK] A question about Command ID

Re: [SPDK] A question about Command ID

[Oscar.Huang]

[-- Attachment #1: Type: text/plain, Size: 1258 bytes --]

Hi, Jim

Thank you for your prompt response. I understand the design philosophy about hiding low level complexity from users. I'm trying to use SPDK to test SSD firmware, which might not be an intended use case of SPDK. Sometimes I need to know the exact bit pattern of the commands enqueued to SQ for verification. I may also try to create mal-formatted command on purpose to challenge my firmware.
The two "raw" APIs: spdk_nvme_ctrlr_cmd_admin_raw and spdk_nvme_ctrlr_cmd_io_raw should have 1:1 mapping the calls to them and the actual commands sent to the device, right? It'll be great if they can be extended this way:


1.     After the SPDK assigns a unique CID and populates the PRP/SGL fields for the command, populates , copy the complete command back to the original command buffer the caller provided. Thus the caller can know the CID assigned, and what PRP/SGL fields are like.

2.     It is even better if SPDK let me manipulate PRP/SGL fields directly. The caller does this on its own risk. As you pointed out, the caller needs to observe MDTS/NOIOB/PRP restrictions by any means on itself.

I know this is not the normal use case of SPDK. But if you can share any comments or suggestions, it's much appreciated.


Thanks
-Oscar

Re: [SPDK] A question about Command ID

[Oscar.Huang]

[-- Attachment #1: Type: text/plain, Size: 2344 bytes --]

Thanks, Benjamin. I'll try to add a new API and try it in our environment first. If it works well, I'll share it with the community.

Jim's idea regarding CID is good for my use case. I'll try it as well.


Thanks
-Oscar


On Wed, 2019-04-10 at 01:24 +0000, Oscar.Huang at microchip.com wrote:
> Hi, Jim
>
> Thank you for your prompt response. I understand the design philosophy about
> hiding low level complexity from users. I'm trying to use SPDK to test SSD
> firmware, which might not be an intended use case of SPDK. Sometimes I need to
> know the exact bit pattern of the commands enqueued to SQ for verification. I
> may also try to create mal-formatted command on purpose to challenge my
> firmware.
> The two "raw" APIs: spdk_nvme_ctrlr_cmd_admin_raw and
> spdk_nvme_ctrlr_cmd_io_raw should have 1:1 mapping the calls to them and the
> actual commands sent to the device, right? It'll be great if they can be
> extended this way:
>
>
> 1.     After the SPDK assigns a unique CID and populates the PRP/SGL fields
> for the command, populates , copy the complete command back to the original
> command buffer the caller provided. Thus the caller can know the CID assigned,
> and what PRP/SGL fields are like.
>
> 2.     It is even better if SPDK let me manipulate PRP/SGL fields directly.
> The caller does this on its own risk. As you pointed out, the caller needs to
> observe MDTS/NOIOB/PRP restrictions by any means on itself.
>
> I know this is not the normal use case of SPDK. But if you can share any
> comments or suggestions, it's much appreciated.

While using SPDK as a mechanism to test NVMe drives wasn't our initial intention
with SPDK, I think using a user space driver for testing is a fantastic solution
that I hope most device manufacturers adopt. If you were interested in adding a
new API that allows you to submit a raw command including the PRP/SGL, that
would certainly be considered for acceptance.

The CID problem is a bit harder. I spoke with Jim a bit and his most recent idea
was to add a function that attempted to look up a request by CID, so that when
we give you the CID in the timeout/abort callback, you can try to get to the
request. If the original request was split at all, this would fail, which I
think is fine for your use case. What do you think?

Re: [SPDK] A question about Command ID

[Walker, Benjamin]

[-- Attachment #1: Type: text/plain, Size: 2289 bytes --]

On Wed, 2019-04-10 at 01:24 +0000, Oscar.Huang(a)microchip.com wrote:
> Hi, Jim
> 
> Thank you for your prompt response. I understand the design philosophy about
> hiding low level complexity from users. I'm trying to use SPDK to test SSD
> firmware, which might not be an intended use case of SPDK. Sometimes I need to
> know the exact bit pattern of the commands enqueued to SQ for verification. I
> may also try to create mal-formatted command on purpose to challenge my
> firmware.
> The two "raw" APIs: spdk_nvme_ctrlr_cmd_admin_raw and
> spdk_nvme_ctrlr_cmd_io_raw should have 1:1 mapping the calls to them and the
> actual commands sent to the device, right? It'll be great if they can be
> extended this way:
> 
> 
> 1.     After the SPDK assigns a unique CID and populates the PRP/SGL fields
> for the command, populates , copy the complete command back to the original
> command buffer the caller provided. Thus the caller can know the CID assigned,
> and what PRP/SGL fields are like.
> 
> 2.     It is even better if SPDK let me manipulate PRP/SGL fields directly.
> The caller does this on its own risk. As you pointed out, the caller needs to
> observe MDTS/NOIOB/PRP restrictions by any means on itself.
> 
> I know this is not the normal use case of SPDK. But if you can share any
> comments or suggestions, it's much appreciated.

While using SPDK as a mechanism to test NVMe drives wasn't our initial intention
with SPDK, I think using a user space driver for testing is a fantastic solution
that I hope most device manufacturers adopt. If you were interested in adding a
new API that allows you to submit a raw command including the PRP/SGL, that
would certainly be considered for acceptance.

The CID problem is a bit harder. I spoke with Jim a bit and his most recent idea
was to add a function that attempted to look up a request by CID, so that when
we give you the CID in the timeout/abort callback, you can try to get to the
request. If the original request was split at all, this would fail, which I
think is fine for your use case. What do you think?

> 
> 
> Thanks
> -Oscar
> _______________________________________________
> SPDK mailing list
> SPDK(a)lists.01.org
> https://lists.01.org/mailman/listinfo/spdk

Re: [SPDK] A question about Command ID

[Harris, James R]

[-- Attachment #1: Type: text/plain, Size: 2385 bytes --]



On 4/9/19, 9:57 AM, "SPDK on behalf of Oscar.Huang(a)microchip.com" <spdk-bounces(a)lists.01.org on behalf of Oscar.Huang(a)microchip.com> wrote:

    Hi,
    
    I have a question about how to get the command ID of the NVMe command I submitted via APIs spdk_nvme_ctrlr_cmd_xxx, spdk_nvme_ns_cmd_xxx, spdk_nvme_ctrlr_cmd_admin_raw or spdk_nvme_ctrlr_cmd_io_raw. The APIs don't let me specify the command Identifier, instead, the Command Identify is set internally by SPDK. And it seems no way to know the Command Identified assigned by SPDK. Some APIs need a cid as input, for example, spdk_nvme_ctrlr_cmd_abort, if I don't know the cid of the command previously submitted, how could I try to abort it? Another use case is timeout callback, spdk_nvme_timeout_cb, when called, a cid will be passed in. I'd like to exactly what command timed out from the cid. How could I associate the abstract cid with any previously submitted command in it if I don't know the cid of the commands I submitted?


Hi Oscar,

As you've noticed, there is currently no way to specify nor retrieve the command ID (CID).  This largely stems from the following:

* Command IDs must be unique.  Allowing users to specify command IDs opens up a large surface of potential bugs.
* There is not always a 1-to-1 mapping between calls to the spdk_nvme APIs and actual commands sent to a namespace.

We made a conscious choice to hide a lot of complexity around MDTS, NOIOB and PRP from users of the SPDK NVMe driver, resulting in a number of cases where a user request may be split into multiple commands sent to the SSD.  There is also the ability to queue requests inside of the NVMe driver when the submission queue is already full.

For spdk_nvme_ctrlr_cmd_abort(), you need the CID passed from the timeout callback.  There is no other way to get a CID currently.

The SPDK NVMe driver could be extended to provide some additional insight into relationships between callback contexts and their CIDs.  It would be predicated on the caller ensuring it does not violate any MDTS/NOIOB/PRP restrictions, nor overflowing the SQ.  Is this something you'd be interested in tackling?

Thanks,

-Jim

    
    Thanks
    -Oscar
    _______________________________________________
    SPDK mailing list
    SPDK(a)lists.01.org
    https://lists.01.org/mailman/listinfo/spdk
    

[SPDK] A question about Command ID

[Oscar.Huang]

[-- Attachment #1: Type: text/plain, Size: 859 bytes --]

Hi,

I have a question about how to get the command ID of the NVMe command I submitted via APIs spdk_nvme_ctrlr_cmd_xxx, spdk_nvme_ns_cmd_xxx, spdk_nvme_ctrlr_cmd_admin_raw or spdk_nvme_ctrlr_cmd_io_raw. The APIs don't let me specify the command Identifier, instead, the Command Identify is set internally by SPDK. And it seems no way to know the Command Identified assigned by SPDK. Some APIs need a cid as input, for example, spdk_nvme_ctrlr_cmd_abort, if I don't know the cid of the command previously submitted, how could I try to abort it? Another use case is timeout callback, spdk_nvme_timeout_cb, when called, a cid will be passed in. I'd like to exactly what command timed out from the cid. How could I associate the abstract cid with any previously submitted command in it if I don't know the cid of the commands I submitted?


Thanks
-Oscar