NVM Sets supported on Linux?

NVM Sets supported on Linux?

[Sushil]

Hi All,

Are NVM Sets supported on Linux (Endurance, PLM etc)?
I could find the userspace command option "id-nvmset" (and the ioctl
NVME_ID_CNS_NVMSET_LIST) in the nvme-cli,
but nothing in the Linux kernel host and target.

Thanks,
Sushil.

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: NVM Sets supported on Linux?

[Keith Busch]

On Wed, Nov 20, 2019 at 01:58:29PM +0530, Sushil wrote:
> Hi All,
> 
> Are NVM Sets supported on Linux (Endurance, PLM etc)?
> I could find the userspace command option "id-nvmset" (and the ioctl
> NVME_ID_CNS_NVMSET_LIST) in the nvme-cli,
> but nothing in the Linux kernel host and target.

It makes sense for an admin to query this kind of thing, so tooling
has support for that. It could possibly make sense for the target to
implement sets if it can determine that two namespace resources are
independent from each other, but what do you want the host driver to do
with that information?

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: NVM Sets supported on Linux?

[Sushil]

Hi Keith,
Thanks for reply.

Basically, currently  I want to create NVM Sets on an NVMe disk and
export these NVM sets as LVM volumes to VM instances using Openstack.
Later on try things like Endurance, PLM etc. with these NVM sets.
We don't have any physical NVMe controller with NVM sets though.
So I was thinking whether I can use the 'nvmet' Linux target driver to
create kind of logical/virtual  NVM sets
and therefore whether target has/will have support for NVM sets. Its
like how namespaces are created on-the-fly
in Openstack during volume attachment using the volume path -
/dev/vg1/lvol1 for ex. - and the Nova compute node
then nvme connects to these newly created volume specific namespaces
and attaches them to the VM instances.

So I am thinking instead of volume path, I create NVM sets and
corresponding namespaces on-the-fly using the nvmet target
driver and then the Nova compute node nvme connects to these NVM sets
and attaches them to the VM instances.

Thanks.
Sushil.













On Wed, Nov 20, 2019 at 8:59 PM Keith Busch <kbusch@kernel.org> wrote:
>
> On Wed, Nov 20, 2019 at 01:58:29PM +0530, Sushil wrote:
> > Hi All,
> >
> > Are NVM Sets supported on Linux (Endurance, PLM etc)?
> > I could find the userspace command option "id-nvmset" (and the ioctl
> > NVME_ID_CNS_NVMSET_LIST) in the nvme-cli,
> > but nothing in the Linux kernel host and target.
>
> It makes sense for an admin to query this kind of thing, so tooling
> has support for that. It could possibly make sense for the target to
> implement sets if it can determine that two namespace resources are
> independent from each other, but what do you want the host driver to do
> with that information?

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: NVM Sets supported on Linux?

[Keith Busch]

On Wed, Nov 20, 2019 at 11:33:11PM +0530, Sushil wrote:
> Basically, currently  I want to create NVM Sets on an NVMe disk and
> export these NVM sets as LVM volumes to VM instances using Openstack.
> Later on try things like Endurance, PLM etc. with these NVM sets.
> We don't have any physical NVMe controller with NVM sets though.
> So I was thinking whether I can use the 'nvmet' Linux target driver to
> create kind of logical/virtual  NVM sets
> and therefore whether target has/will have support for NVM sets. Its
> like how namespaces are created on-the-fly
> in Openstack during volume attachment using the volume path -
> /dev/vg1/lvol1 for ex. - and the Nova compute node
> then nvme connects to these newly created volume specific namespaces
> and attaches them to the VM instances.
> 
> So I am thinking instead of volume path, I create NVM sets and
> corresponding namespaces on-the-fly using the nvmet target
> driver and then the Nova compute node nvme connects to these NVM sets
> and attaches them to the VM instances.

Okay, so this is only making a case for linux target driver kernel
to implement nvm sets. Can we agree there is nothing special for the
host-side driver to do for enabling nvm sets?

As I mentioned earlier, it might make sense for the linux target to
report set identifiers for its namespaces if it can determine which
ones are isolated from others. Or maybe just add new configfs options,
I'm not really sure about the implementation details off the top of my
head.

In any case, I'm not aware of anyone actively developing support for
NVM Sets in Linux, so it may be an open topic if you want to take a
stab at it.

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: NVM Sets supported on Linux?

[Sushil]

Hi Keith,

For this simple use case, I think there should be nothing special for
the host side driver to do enabling NVM sets - although I must
admit my limited exposure to NVMe so far.

Somehow I've a simplistic view of NVM sets at the target side which is
as follows:

Lets say I have an LVM volume of size 10TB and I create an NVM over it
using the target nvmet driver.
Then as per my understanding of the NVMe Spec 1.4 (of June 10 2019), I
should be able to create say three NVM sets of
1 TB each leaving 7TB of unallocated space in the NVM (Figure 133 in the spec).
In this scenario the target driver knows about the three NVM sets and
that they are isolated and their boundaries because it itself created
them?

Regarding taking a stab at it: Sure :) although I will need to come to
speed with NVMe Linux implementations fast!
Also, NVMe is such a hot topic and with NVM sets being an enabler for
new important features such as Endurance, PLM etc., I am sure
someone must have already jumped at it :)

Thanks,
Sushil.




















On Wed, Nov 20, 2019 at 11:51 PM Keith Busch <kbusch@kernel.org> wrote:
>
> On Wed, Nov 20, 2019 at 11:33:11PM +0530, Sushil wrote:
> > Basically, currently  I want to create NVM Sets on an NVMe disk and
> > export these NVM sets as LVM volumes to VM instances using Openstack.
> > Later on try things like Endurance, PLM etc. with these NVM sets.
> > We don't have any physical NVMe controller with NVM sets though.
> > So I was thinking whether I can use the 'nvmet' Linux target driver to
> > create kind of logical/virtual  NVM sets
> > and therefore whether target has/will have support for NVM sets. Its
> > like how namespaces are created on-the-fly
> > in Openstack during volume attachment using the volume path -
> > /dev/vg1/lvol1 for ex. - and the Nova compute node
> > then nvme connects to these newly created volume specific namespaces
> > and attaches them to the VM instances.
> >
> > So I am thinking instead of volume path, I create NVM sets and
> > corresponding namespaces on-the-fly using the nvmet target
> > driver and then the Nova compute node nvme connects to these NVM sets
> > and attaches them to the VM instances.
>
> Okay, so this is only making a case for linux target driver kernel
> to implement nvm sets. Can we agree there is nothing special for the
> host-side driver to do for enabling nvm sets?
>
> As I mentioned earlier, it might make sense for the linux target to
> report set identifiers for its namespaces if it can determine which
> ones are isolated from others. Or maybe just add new configfs options,
> I'm not really sure about the implementation details off the top of my
> head.
>
> In any case, I'm not aware of anyone actively developing support for
> NVM Sets in Linux, so it may be an open topic if you want to take a
> stab at it.

_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

Re: NVM Sets supported on Linux?

[Chaitanya Kulkarni]

On 11/20/19 11:23 AM, Sushil wrote:

> 
> Regarding taking a stab at it: Sure:)  although I will need to come to
> speed with NVMe Linux implementations fast!
> Also, NVMe is such a hot topic and with NVM sets being an enabler for
> new important features such as Endurance, PLM etc., I am sure
>

Yes I did already when TP was being developed and came to know that we
don't need NVMe Sets for target. But if then since situation changed
and if you have a detailed usecase for it, please let me know.

  someone must have already jumped at it:)
> 
> Thanks,
> Sushil.


_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme