Re: [PATCH v2 12/20] nvme: bump supported specification version to 1.3

[Klaus Birkelund <its@irrelevant.dk>]

On Mon, Nov 25, 2019 at 12:13:15PM +0000, Beata Michalska wrote:
> On Mon, 18 Nov 2019 at 09:48, Klaus Birkelund <its@irrelevant.dk> wrote:
> >
> > On Tue, Nov 12, 2019 at 03:05:06PM +0000, Beata Michalska wrote:
> > > Hi Klaus,
> > >
> > > On Tue, 15 Oct 2019 at 11:52, Klaus Jensen <its@irrelevant.dk> wrote:
> > > >
> > > > +static uint16_t nvme_identify_ns_descr_list(NvmeCtrl *n, NvmeCmd *c)
> > > > +{
> > > > +    static const int len = 4096;
> > > > +
> > > > +    struct ns_descr {
> > > > +        uint8_t nidt;
> > > > +        uint8_t nidl;
> > > > +        uint8_t rsvd2[2];
> > > > +        uint8_t nid[16];
> > > > +    };
> > > > +
> > > > +    uint32_t nsid = le32_to_cpu(c->nsid);
> > > > +    uint64_t prp1 = le64_to_cpu(c->prp1);
> > > > +    uint64_t prp2 = le64_to_cpu(c->prp2);
> > > > +
> > > > +    struct ns_descr *list;
> > > > +    uint16_t ret;
> > > > +
> > > > +    trace_nvme_identify_ns_descr_list(nsid);
> > > > +
> > > > +    if (unlikely(nsid == 0 || nsid > n->num_namespaces)) {
> > > > +        trace_nvme_err_invalid_ns(nsid, n->num_namespaces);
> > > > +        return NVME_INVALID_NSID | NVME_DNR;
> > > > +    }
> > > > +
> > > In theory this should abort the command for inactive NSIDs as well.
> > > But I guess this will come later on.
> > >
> >
> > At this point in the series, the device does not support multiple
> > namespaces anyway and num_namespaces is always 1. But this has also been
> > reported seperately in relation the patch adding multiple namespaces and
> > is fixed in v3.
> >
> > > > +    list = g_malloc0(len);
> > > > +    list->nidt = 0x3;
> > > > +    list->nidl = 0x10;
> > > > +    *(uint32_t *) &list->nid[12] = cpu_to_be32(nsid);
> > > > +
> > > Might be worth to add some comment here -> as per the NGUID/EUI64 format.
> > > Also those are not specified currently in the namespace identity data structure.
> > >
> >
> > I'll add a comment for why the Namespace UUID is set to this value here.
> > The NGUID/EUI64 fields are not set in the namespace identity data
> > structure as they are not required. See the descriptions of NGUID and
> > EUI64. Here for NGUID:
> >
> >     "The controller shall specify a globally unique namespace identifier
> >     in this field, the EUI64 field, or a Namespace UUID in the Namespace
> >     Identification Descriptor..."
> >
> > Here, I chose to provide it in the Namespace Identification Descriptor
> > (by setting `list->nidt = 0x3`).
> >
> > > > +    ret = nvme_dma_read_prp(n, (uint8_t *) list, len, prp1, prp2);
> > > > +    g_free(list);
> > > > +    return ret;
> > > > +}
> > > > +
> > > >  static uint16_t nvme_identify(NvmeCtrl *n, NvmeCmd *cmd)
> > > >  {
> > > >      NvmeIdentify *c = (NvmeIdentify *)cmd;
> > > > @@ -934,7 +978,9 @@ static uint16_t nvme_identify(NvmeCtrl *n, NvmeCmd *cmd)
> > > >      case 0x01:
> > > >          return nvme_identify_ctrl(n, c);
> > > >      case 0x02:
> > > > -        return nvme_identify_nslist(n, c);
> > > > +        return nvme_identify_ns_list(n, c);
> > > > +    case 0x03:
> > > > +        return nvme_identify_ns_descr_list(n, cmd);
> > > >      default:
> > > >          trace_nvme_err_invalid_identify_cns(le32_to_cpu(c->cns));
> > > >          return NVME_INVALID_FIELD | NVME_DNR;
> > > > @@ -1101,6 +1147,14 @@ static uint16_t nvme_set_feature(NvmeCtrl *n, NvmeCmd *cmd, NvmeRequest *req)
> > > >          blk_set_enable_write_cache(n->conf.blk, dw11 & 1);
> > > >          break;
> > > >      case NVME_NUMBER_OF_QUEUES:
> > > > +        if (n->qs_created > 2) {
> > > > +            return NVME_CMD_SEQ_ERROR | NVME_DNR;
> > > > +        }
> > > > +
> > > I am not sure this is entirely correct as the spec says:
> > > "if any I/O Submission and/or Completion Queues (...)"
> > > so it might be enough to have a single queue created
> > > for this command to be valid.
> > > Also I think that the condition here is to make sure that the number
> > > of queues requested is being set once at init phase. Currently this will
> > > allow the setting to happen if there is no active queue -> so at any
> > > point of time (provided the condition mentioned). I might be wrong here
> > > but it seems that what we need is a single status saying any queue
> > > has been created prior to the Set Feature command at all
> > >
> >
> > Internally, the admin queue pair is counted in qs_created, which is the
> > reason for checking if is above 2. The admin queues are created when the
> > controller is enabled (mmio write to the EN register in CC).
> >
> > I'll add a comment about that - I see why it is unclear.
> >
> 
> Ok, so indeed I have missed the fact that the admin queues are being tracked by
> 'qs_created'. Still, I might be wrong, but, it is enough to create I/O
> submission queue and delete it and the code will allow the command to proceed
> Whereas the spec says :
> "If a Set Features command is issued for this feature after creation of
> any I/O Submission and/or I/O Completion Queues, then the Set Features
> command shall fail with status code of Command Sequence Error"
> 
> I might be misreading it though but it is not entirely clear to me whether
> this is : at least one queue create ... or created and in use.
> 
> But I guess that is really minor.
> 
 
Ohh! I see. Yes you are right. "This feature shall only be issued during
initialization prior to creation of any I/O Submission and/or Completion
Queues.".

To me it looks like the controller shall not allow the Set Feature
command again until after a controller reset. Then we also don't need
the counter, but just a flag to indicate if any I/O queue has been
configured.

I've fixed that up.