[SPDK] Re: is it possible to add a blacklist for the lvol module?

[SPDK] Re: is it possible to add a blacklist for the lvol module?

[peng yu]

[-- Attachment #1: Type: text/plain, Size: 1940 bytes --]

My project could take care of the ownership, so "claim" is not really
required (although "claim" is still a good to have feature). Currently
does SPDK has any way to disable "claim"?
By the way, I don't write my SPDK application, I use the
"app/spdk_tgt/spdk_tgt" as a part of my project. So I hope I could
operate it via RPC.

Best regards.

On Wed, Apr 1, 2020 at 6:13 AM Andrey Kuzmin <andrey.v.kuzmin(a)gmail.com> wrote:
>
> On Mon, Mar 30, 2020, 21:56 peng yu <yupeng0921(a)gmail.com> wrote:
>
> > I have a below scenario:
> > A disk is controlled by SPDK APP A.
> > SPDK APP A exports the disk as a NVMEoF target
> > SPDK APP B is a NVMEoF initiator, it connects to that disk.
> > SPDk APP B creates a lvs and a lvol on the disk.
> > When I restart the APP A and APP B. The APP A will examine the disk
> > and find there is a lvs, and then the lvol module on APP A will claim
> > this disk, then this disk can't be exported over NVMEoF.
> >
> > Could we add a api to the lvol module? The api could add a dev name
> > pattern to a black list. When the lvol module examine a disk, it will
> > check against the black list, if a dev name matches the black list, it
> > will ignore this device.
> >
>
> There's another option to check out here: is there any need in your
> scenario for the lvol module and nvmf target to claim the lvol? Basically,
> claims ensure single bdev owner which, outside fool-proofing and/or
> hardware with special requirements, isn't strictly necessary (provided your
> apps take care of the ownership).
>
> Regards,
> Andrey
>
> >
> > Best regards.
> > _______________________________________________
> > SPDK mailing list -- spdk(a)lists.01.org
> > To unsubscribe send an email to spdk-leave(a)lists.01.org
> >
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org

[SPDK] Re: is it possible to add a blacklist for the lvol module?

[peng yu]

[-- Attachment #1: Type: text/plain, Size: 4846 bytes --]

I'm wondering whether the blacklist should match the claimer's module
name too, e.g. only deny a specific module to claim a bdev. And maybe
we need a blacklist_exceptions. Then we could set module_pattern="*",
dev_pattern="your_dev_name_or_alias_or_uuid" in the blacklist. And set
module_patter="a_module_name",
dev_pattern="your_dev_name_or_alias_or_uuid" in the
blacklist_exceptions, thus we could only allow a specify module claim
the device.

Best regards.

On Tue, Mar 31, 2020 at 10:14 PM peng yu <yupeng0921(a)gmail.com> wrote:
>
> Hi, Tomek.
>
> When I read this email, the meeting has started. If you'd like to
> discuss this topic in next meeting, I will attend. If there is any
> update in today's meeting, please let me know.
> I agree that this feature is more suitable to be implemented on the
> bdev layer. As you mentioned, the bdev_set_options RPC is registered
> on SPDK_RPC_STARTUP. We may need to call it at run time. I think add
> several 'examine' RPCs for this feature and register them on
> SPDK_RPC_RUNTIME would be better. In my idea, we may add below RPCs:
> (1) add_examine_blacklist: add a string pattern to the blacklist
> (2) delete_examine_blacklist: delete a string pattern from the blacklist
> (3) get_examine_blacklist: get the full blacklist
> In the bdev_start function, we may check the bdev name, alias and uuid
> against the blacklist, if the blacklist matches anything, the bdev
> layer won't call the examine_disk function.
> About how to "match" the bdev name, alias and uuid against the
> blacklist, we may use the fnmatch function, thus we could support
> wildcard, but the performance would be not very good. Instead of using
> fnmatch, we may only check whether their prefix are matched, thus we
> could store the blacklist as a prefix tree, the performance would be
> better but it would be less powerful.
>
> Please let me know whether you have any idea about it. If we could
> point out an idea that the community would like to accept, I'd like to
> implement it.
>
> Best regards.
>
> On Tue, Mar 31, 2020 at 4:58 AM Zawadzki, Tomasz
> <tomasz.zawadzki(a)intel.com> wrote:
> >
> > Hi Peng Yu,
> >
> > Described scenario can occur for any bdev module that examines device being attached for metadata.
> > Lvols, GPT, OCF, Raid5 (in future) and other custom ones.
> > At this time there is no way to disable bdevs from examining the devices, without code changes.
> >
> > The proposed API would have to be on bdev layer, rather than any of the bdev module types.
> > There is bdev subsystem options that could be changed (bdev_set_options() RPC).
> >
> > Another argument could be added that disables all examination, or as suggested blacklists examination on bdev name/alias/uuid.
> > Drawback here is those options can be set only before subsystems are started (see framework_start_init()).
> > So one cannot change mind after that.
> >
> > Bdev API could be extended with explicit 'examine' RPC to allow run-time examination for all or select bdev.
> >
> > Personally I'm not set on an approach, as examine functionality was supposed to be "straightforward" to use.
> > Yet your example shows that in scenarios where SPDK is used on both ends of a connection (NVMe-oF/iSCSI/vhost),
> > needs more thought.
> >
> > I'll add this topic to today's Community meeting agenda, please feel free to attend:
> > https://spdk.io/community/
> >
> > Thanks,
> > Tomek
> >
> > > -----Original Message-----
> > > From: peng yu <yupeng0921(a)gmail.com>
> > > Sent: Monday, March 30, 2020 8:56 PM
> > > To: Storage Performance Development Kit <spdk(a)lists.01.org>
> > > Subject: [SPDK] is it possible to add a blacklist for the lvol module?
> > >
> > > I have a below scenario:
> > > A disk is controlled by SPDK APP A.
> > > SPDK APP A exports the disk as a NVMEoF target SPDK APP B is a NVMEoF
> > > initiator, it connects to that disk.
> > > SPDk APP B creates a lvs and a lvol on the disk.
> > > When I restart the APP A and APP B. The APP A will examine the disk and find
> > > there is a lvs, and then the lvol module on APP A will claim this disk, then this
> > > disk can't be exported over NVMEoF.
> > >
> > > Could we add a api to the lvol module? The api could add a dev name pattern
> > > to a black list. When the lvol module examine a disk, it will check against the
> > > black list, if a dev name matches the black list, it will ignore this device.
> > >
> > > Best regards.
> > > _______________________________________________
> > > SPDK mailing list -- spdk(a)lists.01.org
> > > To unsubscribe send an email to spdk-leave(a)lists.01.org
> > _______________________________________________
> > SPDK mailing list -- spdk(a)lists.01.org
> > To unsubscribe send an email to spdk-leave(a)lists.01.org

[SPDK] Re: is it possible to add a blacklist for the lvol module?

[Andrey Kuzmin]

[-- Attachment #1: Type: text/plain, Size: 1288 bytes --]

On Mon, Mar 30, 2020, 21:56 peng yu <yupeng0921(a)gmail.com> wrote:

> I have a below scenario:
> A disk is controlled by SPDK APP A.
> SPDK APP A exports the disk as a NVMEoF target
> SPDK APP B is a NVMEoF initiator, it connects to that disk.
> SPDk APP B creates a lvs and a lvol on the disk.
> When I restart the APP A and APP B. The APP A will examine the disk
> and find there is a lvs, and then the lvol module on APP A will claim
> this disk, then this disk can't be exported over NVMEoF.
>
> Could we add a api to the lvol module? The api could add a dev name
> pattern to a black list. When the lvol module examine a disk, it will
> check against the black list, if a dev name matches the black list, it
> will ignore this device.
>

There's another option to check out here: is there any need in your
scenario for the lvol module and nvmf target to claim the lvol? Basically,
claims ensure single bdev owner which, outside fool-proofing and/or
hardware with special requirements, isn't strictly necessary (provided your
apps take care of the ownership).

Regards,
Andrey

>
> Best regards.
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
>

[SPDK] Re: is it possible to add a blacklist for the lvol module?

[peng yu]

[-- Attachment #1: Type: text/plain, Size: 4134 bytes --]

Hi, Tomek.

When I read this email, the meeting has started. If you'd like to
discuss this topic in next meeting, I will attend. If there is any
update in today's meeting, please let me know.
I agree that this feature is more suitable to be implemented on the
bdev layer. As you mentioned, the bdev_set_options RPC is registered
on SPDK_RPC_STARTUP. We may need to call it at run time. I think add
several 'examine' RPCs for this feature and register them on
SPDK_RPC_RUNTIME would be better. In my idea, we may add below RPCs:
(1) add_examine_blacklist: add a string pattern to the blacklist
(2) delete_examine_blacklist: delete a string pattern from the blacklist
(3) get_examine_blacklist: get the full blacklist
In the bdev_start function, we may check the bdev name, alias and uuid
against the blacklist, if the blacklist matches anything, the bdev
layer won't call the examine_disk function.
About how to "match" the bdev name, alias and uuid against the
blacklist, we may use the fnmatch function, thus we could support
wildcard, but the performance would be not very good. Instead of using
fnmatch, we may only check whether their prefix are matched, thus we
could store the blacklist as a prefix tree, the performance would be
better but it would be less powerful.

Please let me know whether you have any idea about it. If we could
point out an idea that the community would like to accept, I'd like to
implement it.

Best regards.

On Tue, Mar 31, 2020 at 4:58 AM Zawadzki, Tomasz
<tomasz.zawadzki(a)intel.com> wrote:
>
> Hi Peng Yu,
>
> Described scenario can occur for any bdev module that examines device being attached for metadata.
> Lvols, GPT, OCF, Raid5 (in future) and other custom ones.
> At this time there is no way to disable bdevs from examining the devices, without code changes.
>
> The proposed API would have to be on bdev layer, rather than any of the bdev module types.
> There is bdev subsystem options that could be changed (bdev_set_options() RPC).
>
> Another argument could be added that disables all examination, or as suggested blacklists examination on bdev name/alias/uuid.
> Drawback here is those options can be set only before subsystems are started (see framework_start_init()).
> So one cannot change mind after that.
>
> Bdev API could be extended with explicit 'examine' RPC to allow run-time examination for all or select bdev.
>
> Personally I'm not set on an approach, as examine functionality was supposed to be "straightforward" to use.
> Yet your example shows that in scenarios where SPDK is used on both ends of a connection (NVMe-oF/iSCSI/vhost),
> needs more thought.
>
> I'll add this topic to today's Community meeting agenda, please feel free to attend:
> https://spdk.io/community/
>
> Thanks,
> Tomek
>
> > -----Original Message-----
> > From: peng yu <yupeng0921(a)gmail.com>
> > Sent: Monday, March 30, 2020 8:56 PM
> > To: Storage Performance Development Kit <spdk(a)lists.01.org>
> > Subject: [SPDK] is it possible to add a blacklist for the lvol module?
> >
> > I have a below scenario:
> > A disk is controlled by SPDK APP A.
> > SPDK APP A exports the disk as a NVMEoF target SPDK APP B is a NVMEoF
> > initiator, it connects to that disk.
> > SPDk APP B creates a lvs and a lvol on the disk.
> > When I restart the APP A and APP B. The APP A will examine the disk and find
> > there is a lvs, and then the lvol module on APP A will claim this disk, then this
> > disk can't be exported over NVMEoF.
> >
> > Could we add a api to the lvol module? The api could add a dev name pattern
> > to a black list. When the lvol module examine a disk, it will check against the
> > black list, if a dev name matches the black list, it will ignore this device.
> >
> > Best regards.
> > _______________________________________________
> > SPDK mailing list -- spdk(a)lists.01.org
> > To unsubscribe send an email to spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org

[SPDK] Re: is it possible to add a blacklist for the lvol module?

[Zawadzki, Tomasz]

[-- Attachment #1: Type: text/plain, Size: 2330 bytes --]

Hi Peng Yu,

Described scenario can occur for any bdev module that examines device being attached for metadata.
Lvols, GPT, OCF, Raid5 (in future) and other custom ones.
At this time there is no way to disable bdevs from examining the devices, without code changes.

The proposed API would have to be on bdev layer, rather than any of the bdev module types.
There is bdev subsystem options that could be changed (bdev_set_options() RPC).

Another argument could be added that disables all examination, or as suggested blacklists examination on bdev name/alias/uuid.
Drawback here is those options can be set only before subsystems are started (see framework_start_init()).
So one cannot change mind after that.

Bdev API could be extended with explicit 'examine' RPC to allow run-time examination for all or select bdev.

Personally I'm not set on an approach, as examine functionality was supposed to be "straightforward" to use.
Yet your example shows that in scenarios where SPDK is used on both ends of a connection (NVMe-oF/iSCSI/vhost),
needs more thought.

I'll add this topic to today's Community meeting agenda, please feel free to attend:
https://spdk.io/community/

Thanks,
Tomek

> -----Original Message-----
> From: peng yu <yupeng0921(a)gmail.com>
> Sent: Monday, March 30, 2020 8:56 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>
> Subject: [SPDK] is it possible to add a blacklist for the lvol module?
> 
> I have a below scenario:
> A disk is controlled by SPDK APP A.
> SPDK APP A exports the disk as a NVMEoF target SPDK APP B is a NVMEoF
> initiator, it connects to that disk.
> SPDk APP B creates a lvs and a lvol on the disk.
> When I restart the APP A and APP B. The APP A will examine the disk and find
> there is a lvs, and then the lvol module on APP A will claim this disk, then this
> disk can't be exported over NVMEoF.
> 
> Could we add a api to the lvol module? The api could add a dev name pattern
> to a black list. When the lvol module examine a disk, it will check against the
> black list, if a dev name matches the black list, it will ignore this device.
> 
> Best regards.
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org