[PATCHv2] nvmet: make MDTS value configurable

[sagi@grimberg.me (Sagi Grimberg)]

> Yes, from what I saw we don't limit the max_io_size in the target since 
> we're using the rw-API.
> 
> In case 1 rdma ctx is not enough (MR page list for example) we use more 
> ctx's in the rw-API. I think we have a bug there since we don't check 
> that nr_ops is not to big for transaction.

What do you mean too big?

> So probably we need to add some limit int nvmet/RDMA for that as 
> NFS/RDMA does (let's agree on some value/module_param/tunable).
> 
> Or we can say that the limit is (if we force MRs for example):
> 
> rdma_rw_fr_page_list_len(device) * device_page_size * 
> max_mrs_per_transaction.

if at all it should probably be:
rdma_rw_fr_page_list_len(device) * device_page_size * 
attr->cap.max_rdma_ctxs

> We also need to calculate mr_pool_size and max_rdma_ctxs more carefully 
> IMO.
> 
> So let's agree on the approach and then implement a solution (Sagi, can 
> you comments on that too please ?)

I don't think we want a modparam for something this low-level...

we reserve a qp space and mr pool, we should limit MDTS based on that
alone.

> Nevertheless, While digging into it I saw some improvements we can do to 
> rw-API, so I'm preparing few patches (add static allocations option 
> instead of allocations in fast-path).

Note that I still don't understand what is the original issue Hannes
was trying to resolve...