[PATCH v2] nvme-fc: don't require user to enter host_traddr

[PATCH v2] nvme-fc: don't require user to enter host_traddr

[Johannes Thumshirn]

One major usability difference between NVMf RDMA and FC is resolving
the default host transport address in RDMA. This is perfectly doable
in FC as well, as we already have all possible lport <-> rport
combinations pre-populated so we can pick the first lport that has a
connection to our desired rport per default or optionally use the user
supplied lport if we have one.

Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
Cc: James Smart <james.smart@broadcom.com>
--
Changes to v1:
- Both laddr.pn and laddr.nn must be present _if_ we take the laddr into
  account (Hannes)
---
 drivers/nvme/host/fc.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 7ab0be55c7d0..063b9c3438ce 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -3328,15 +3328,19 @@ nvme_fc_create_ctrl(struct device *dev, struct nvmf_ctrl_options *opts)
 	if (ret || !raddr.nn || !raddr.pn)
 		return ERR_PTR(-EINVAL);
 
-	ret = nvme_fc_parse_traddr(&laddr, opts->host_traddr, NVMF_TRADDR_SIZE);
-	if (ret || !laddr.nn || !laddr.pn)
-		return ERR_PTR(-EINVAL);
+	if (opts->mask & NVMF_OPT_HOST_TRADDR) {
+		ret = nvme_fc_parse_traddr(&laddr, opts->host_traddr,
+					   NVMF_TRADDR_SIZE);
+		if (ret || !laddr.nn || !laddr.pn)
+			return ERR_PTR(-EINVAL);
+	}
 
 	/* find the host and remote ports to connect together */
 	spin_lock_irqsave(&nvme_fc_lock, flags);
 	list_for_each_entry(lport, &nvme_fc_lport_list, port_list) {
-		if (lport->localport.node_name != laddr.nn ||
-		    lport->localport.port_name != laddr.pn)
+		if (!(laddr.nn && laddr.pn) &&
+		    (lport->localport.node_name != laddr.nn ||
+		     lport->localport.port_name != laddr.pn))
 			continue;
 
 		list_for_each_entry(rport, &lport->endp_list, endp_list) {
@@ -3364,8 +3368,9 @@ nvme_fc_create_ctrl(struct device *dev, struct nvmf_ctrl_options *opts)
 
 static struct nvmf_transport_ops nvme_fc_transport = {
 	.name		= "fc",
-	.required_opts	= NVMF_OPT_TRADDR | NVMF_OPT_HOST_TRADDR,
-	.allowed_opts	= NVMF_OPT_RECONNECT_DELAY | NVMF_OPT_CTRL_LOSS_TMO,
+	.required_opts	= NVMF_OPT_TRADDR,
+	.allowed_opts	= NVMF_OPT_RECONNECT_DELAY | NVMF_OPT_CTRL_LOSS_TMO
+				| NVMF_OPT_HOST_TRADDR,
 	.create_ctrl	= nvme_fc_create_ctrl,
 };
 
-- 
2.13.6

[PATCH v2] nvme-fc: don't require user to enter host_traddr

[Johannes Thumshirn]

One major usability difference between NVMf RDMA and FC is resolving
the default host transport address in RDMA. This is perfectly doable
in FC as well, as we already have all possible lport <-> rport
combinations pre-populated so we can pick the first lport that has a
connection to our desired rport per default or optionally use the user
supplied lport if we have one.

Signed-off-by: Johannes Thumshirn <jthumshirn at suse.de>
Cc: James Smart <james.smart at broadcom.com>
--
Changes to v1:
- Both laddr.pn and laddr.nn must be present _if_ we take the laddr into
  account (Hannes)
---
 drivers/nvme/host/fc.c | 19 ++++++++++++-------
 1 file changed, 12 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 7ab0be55c7d0..063b9c3438ce 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -3328,15 +3328,19 @@ nvme_fc_create_ctrl(struct device *dev, struct nvmf_ctrl_options *opts)
 	if (ret || !raddr.nn || !raddr.pn)
 		return ERR_PTR(-EINVAL);
 
-	ret = nvme_fc_parse_traddr(&laddr, opts->host_traddr, NVMF_TRADDR_SIZE);
-	if (ret || !laddr.nn || !laddr.pn)
-		return ERR_PTR(-EINVAL);
+	if (opts->mask & NVMF_OPT_HOST_TRADDR) {
+		ret = nvme_fc_parse_traddr(&laddr, opts->host_traddr,
+					   NVMF_TRADDR_SIZE);
+		if (ret || !laddr.nn || !laddr.pn)
+			return ERR_PTR(-EINVAL);
+	}
 
 	/* find the host and remote ports to connect together */
 	spin_lock_irqsave(&nvme_fc_lock, flags);
 	list_for_each_entry(lport, &nvme_fc_lport_list, port_list) {
-		if (lport->localport.node_name != laddr.nn ||
-		    lport->localport.port_name != laddr.pn)
+		if (!(laddr.nn && laddr.pn) &&
+		    (lport->localport.node_name != laddr.nn ||
+		     lport->localport.port_name != laddr.pn))
 			continue;
 
 		list_for_each_entry(rport, &lport->endp_list, endp_list) {
@@ -3364,8 +3368,9 @@ nvme_fc_create_ctrl(struct device *dev, struct nvmf_ctrl_options *opts)
 
 static struct nvmf_transport_ops nvme_fc_transport = {
 	.name		= "fc",
-	.required_opts	= NVMF_OPT_TRADDR | NVMF_OPT_HOST_TRADDR,
-	.allowed_opts	= NVMF_OPT_RECONNECT_DELAY | NVMF_OPT_CTRL_LOSS_TMO,
+	.required_opts	= NVMF_OPT_TRADDR,
+	.allowed_opts	= NVMF_OPT_RECONNECT_DELAY | NVMF_OPT_CTRL_LOSS_TMO
+				| NVMF_OPT_HOST_TRADDR,
 	.create_ctrl	= nvme_fc_create_ctrl,
 };
 
-- 
2.13.6

Re: [PATCH v2] nvme-fc: don't require user to enter host_traddr

[James Smart]

On 11/30/2017 7:12 AM, Johannes Thumshirn wrote:
> One major usability difference between NVMf RDMA and FC is resolving
> the default host transport address in RDMA. This is perfectly doable
> in FC as well, as we already have all possible lport <-> rport
> combinations pre-populated so we can pick the first lport that has a
> connection to our desired rport per default or optionally use the user
> supplied lport if we have one.
>
> Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
> Cc: James Smart <james.smart@broadcom.com>

This is unnecessary and can create weird configurations. It assumes 
connections are manually created. The weirdness is: a) an admin has to 
know there are multiple paths in order to connect them and be 
intelligent on how to get the complex name strings and try to know what 
connections are already in existence;  b) if a users has a connectivity 
loss beyond dev_loss_tmo or ctlr_loss_tmo such that the controller is 
terminated, they must manually issue the connec commands again; and c) 
those un-knowledgeable users will unknowingly find that their multiple 
paths aren't connected and the system will gang up on the host adapter 
detected on the system with connectivity. All things unexpected and not 
what occurs with FC and SCSI and which will result in system support calls.

If the system uses the FC auto-connect scripts things will be properly 
connected across all paths connected to the subsystem - automatically, 
including resume after an extended connectivity loss - and the system 
will behave just like FC does with SCSI.

I see no reason to add this patch.  Please move away from manual 
configuration.

-- james

[PATCH v2] nvme-fc: don't require user to enter host_traddr

[James Smart]

On 11/30/2017 7:12 AM, Johannes Thumshirn wrote:
> One major usability difference between NVMf RDMA and FC is resolving
> the default host transport address in RDMA. This is perfectly doable
> in FC as well, as we already have all possible lport <-> rport
> combinations pre-populated so we can pick the first lport that has a
> connection to our desired rport per default or optionally use the user
> supplied lport if we have one.
>
> Signed-off-by: Johannes Thumshirn <jthumshirn at suse.de>
> Cc: James Smart <james.smart at broadcom.com>

This is unnecessary and can create weird configurations. It assumes 
connections are manually created. The weirdness is: a) an admin has to 
know there are multiple paths in order to connect them and be 
intelligent on how to get the complex name strings and try to know what 
connections are already in existence;? b) if a users has a connectivity 
loss beyond dev_loss_tmo or ctlr_loss_tmo such that the controller is 
terminated, they must manually issue the connec commands again; and c) 
those un-knowledgeable users will unknowingly find that their multiple 
paths aren't connected and the system will gang up on the host adapter 
detected on the system with connectivity. All things unexpected and not 
what occurs with FC and SCSI and which will result in system support calls.

If the system uses the FC auto-connect scripts things will be properly 
connected across all paths connected to the subsystem - automatically, 
including resume after an extended connectivity loss - and the system 
will behave just like FC does with SCSI.

I see no reason to add this patch.? Please move away from manual 
configuration.

-- james

Re: [PATCH v2] nvme-fc: don't require user to enter host_traddr

[Johannes Thumshirn]

James Smart <james.smart@broadcom.com> writes:

> On 11/30/2017 7:12 AM, Johannes Thumshirn wrote:
>> One major usability difference between NVMf RDMA and FC is resolving
>> the default host transport address in RDMA. This is perfectly doable
>> in FC as well, as we already have all possible lport <-> rport
>> combinations pre-populated so we can pick the first lport that has a
>> connection to our desired rport per default or optionally use the user
>> supplied lport if we have one.
>>
>> Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
>> Cc: James Smart <james.smart@broadcom.com>
>
> This is unnecessary and can create weird configurations. It assumes
> connections are manually created.

a) connections can (and will) be maually created and for this the users
have to know the topology or connection establishment will fail b) there
is no need that the connections are manually created. Sagi did post an
RFC systemd service which calls nvme connect-all and this is what should
be done regardless if we're running on FC-NVME or NVMe over RDMA any new
transport that may come in the future. What I want is a consistent user
experience within NVMe, as I am the one that has to answer a
documentation team's inquiries on how to configure NVMf, support QA in
testing and fix end user bugs. The least thing I want ot do is tell them
"well if you use rdma you have to use the nvme-connect.service, if you
use FC you have to have some magic udev rules and auto-connect scripts,
if you use $POSSIBLE_NEW_TRANSPORT you have to yada, yada".

I don't really like that we have to manully connect either but this
behaviour was first in NVMe so we should either stick to that or convert
RDMA over to using some sort of udev magic as well (which wont work as
far as I know, and it definitively won't work with TCP if and when it's
there).

> The weirdness is: a) an admin has to
> know there are multiple paths in order to connect them and be
> intelligent on how to get the complex name strings and try to know
> what connections are already in existence;  b) if a users has a
> connectivity loss beyond dev_loss_tmo or ctlr_loss_tmo such that the
> controller is terminated, they must manually issue the connec commands
> again; and c) those un-knowledgeable users will unknowingly find that
> their multiple paths aren't connected and the system will gang up on
> the host adapter detected on the system with connectivity. All things
> unexpected and not what occurs with FC and SCSI and which will result
> in system support calls.
>
> If the system uses the FC auto-connect scripts things will be properly
> connected across all paths connected to the subsystem - automatically,
> including resume after an extended connectivity loss - and the system
> will behave just like FC does with SCSI.
>
> I see no reason to add this patch.  Please move away from manual
> configuration.

OK so the please help with moving NVMe away from manual configuration.

I'm fine with either way I just don't want to have N different ways
because it is a documentation and usability nightmare.

        Johannes
-- 
Johannes Thumshirn                                          Storage
jthumshirn@suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

[PATCH v2] nvme-fc: don't require user to enter host_traddr

[Johannes Thumshirn]

James Smart <james.smart at broadcom.com> writes:

> On 11/30/2017 7:12 AM, Johannes Thumshirn wrote:
>> One major usability difference between NVMf RDMA and FC is resolving
>> the default host transport address in RDMA. This is perfectly doable
>> in FC as well, as we already have all possible lport <-> rport
>> combinations pre-populated so we can pick the first lport that has a
>> connection to our desired rport per default or optionally use the user
>> supplied lport if we have one.
>>
>> Signed-off-by: Johannes Thumshirn <jthumshirn at suse.de>
>> Cc: James Smart <james.smart at broadcom.com>
>
> This is unnecessary and can create weird configurations. It assumes
> connections are manually created.

a) connections can (and will) be maually created and for this the users
have to know the topology or connection establishment will fail b) there
is no need that the connections are manually created. Sagi did post an
RFC systemd service which calls nvme connect-all and this is what should
be done regardless if we're running on FC-NVME or NVMe over RDMA any new
transport that may come in the future. What I want is a consistent user
experience within NVMe, as I am the one that has to answer a
documentation team's inquiries on how to configure NVMf, support QA in
testing and fix end user bugs. The least thing I want ot do is tell them
"well if you use rdma you have to use the nvme-connect.service, if you
use FC you have to have some magic udev rules and auto-connect scripts,
if you use $POSSIBLE_NEW_TRANSPORT you have to yada, yada".

I don't really like that we have to manully connect either but this
behaviour was first in NVMe so we should either stick to that or convert
RDMA over to using some sort of udev magic as well (which wont work as
far as I know, and it definitively won't work with TCP if and when it's
there).

> The weirdness is: a) an admin has to
> know there are multiple paths in order to connect them and be
> intelligent on how to get the complex name strings and try to know
> what connections are already in existence;? b) if a users has a
> connectivity loss beyond dev_loss_tmo or ctlr_loss_tmo such that the
> controller is terminated, they must manually issue the connec commands
> again; and c) those un-knowledgeable users will unknowingly find that
> their multiple paths aren't connected and the system will gang up on
> the host adapter detected on the system with connectivity. All things
> unexpected and not what occurs with FC and SCSI and which will result
> in system support calls.
>
> If the system uses the FC auto-connect scripts things will be properly
> connected across all paths connected to the subsystem - automatically,
> including resume after an extended connectivity loss - and the system
> will behave just like FC does with SCSI.
>
> I see no reason to add this patch.? Please move away from manual
> configuration.

OK so the please help with moving NVMe away from manual configuration.

I'm fine with either way I just don't want to have N different ways
because it is a documentation and usability nightmare.

        Johannes
-- 
Johannes Thumshirn                                          Storage
jthumshirn at suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

Re: [PATCH v2] nvme-fc: don't require user to enter host_traddr

[James Smart]

On 12/1/2017 12:34 AM, Johannes Thumshirn wrote:
> James Smart <james.smart@broadcom.com> writes:
>
>> On 11/30/2017 7:12 AM, Johannes Thumshirn wrote:
>>> One major usability difference between NVMf RDMA and FC is resolving
>>> the default host transport address in RDMA. This is perfectly doable
>>> in FC as well, as we already have all possible lport <-> rport
>>> combinations pre-populated so we can pick the first lport that has a
>>> connection to our desired rport per default or optionally use the user
>>> supplied lport if we have one.
>>>
>>> Signed-off-by: Johannes Thumshirn <jthumshirn@suse.de>
>>> Cc: James Smart <james.smart@broadcom.com>
>> This is unnecessary and can create weird configurations. It assumes
>> connections are manually created.
> a) connections can (and will) be maually created and for this the users
> have to know the topology or connection establishment will fail b) there
> is no need that the connections are manually created. Sagi did post an
> RFC systemd service which calls nvme connect-all and this is what should
> be done regardless if we're running on FC-NVME or NVMe over RDMA any new
> transport that may come in the future. What I want is a consistent user
> experience within NVMe, as I am the one that has to answer a
> documentation team's inquiries on how to configure NVMf, support QA in
> testing and fix end user bugs. The least thing I want ot do is tell them
> "well if you use rdma you have to use the nvme-connect.service, if you
> use FC you have to have some magic udev rules and auto-connect scripts,
> if you use $POSSIBLE_NEW_TRANSPORT you have to yada, yada".
>
> I don't really like that we have to manully connect either but this
> behaviour was first in NVMe so we should either stick to that or convert
> RDMA over to using some sort of udev magic as well (which wont work as
> far as I know, and it definitively won't work with TCP if and when it's
> there).
>

Sagi's RFC is certainly fine to use with FC-NVME as well. But the 
mechanism does require the admin to have apriori knowledge and populate 
the discovery.conf file with the discovery controller information. It 
also requires updates on any dynamic change. All of that is unnecessary 
with the FC-NVME auto-connect scripts. Note: there's nothing wrong with 
using both mechanisms simultaneously with FC-NVME.

I certainly understand the sentiment of not doing something N ways. But, 
in this case, the functionality is so useful and valuable, that 
constraining the solution to the least common denominator is a bad 
idea.  You are also ignoring the user-base coming from SCSI on FC (which 
is much larger than nvmf on RDMA) that have completely different 
expectations on how the system finds targets and connects to storage. 
They will be confused and have lots of questions about why SCSI is so 
automatic yet (even on the same machines/adapters/fabric) they now must 
have all kinds of upfront knowledge and make manual changes for NVME. I 
get bombarded on this and I know you will too.   Although it's not nice 
to state transport-specific methods, I believe the message is still 
rather short and simple.

We seem to have gotten away from the desired patch into a completely 
different area.   In truth, adding the patch isn't a big deal, but to me 
it doesn't provide any value over the existing scripts. You could say it 
adds yet another one off in FC-specific behavior that could complicate 
the documentation.

-- james

[PATCH v2] nvme-fc: don't require user to enter host_traddr

[James Smart]

On 12/1/2017 12:34 AM, Johannes Thumshirn wrote:
> James Smart <james.smart at broadcom.com> writes:
>
>> On 11/30/2017 7:12 AM, Johannes Thumshirn wrote:
>>> One major usability difference between NVMf RDMA and FC is resolving
>>> the default host transport address in RDMA. This is perfectly doable
>>> in FC as well, as we already have all possible lport <-> rport
>>> combinations pre-populated so we can pick the first lport that has a
>>> connection to our desired rport per default or optionally use the user
>>> supplied lport if we have one.
>>>
>>> Signed-off-by: Johannes Thumshirn <jthumshirn at suse.de>
>>> Cc: James Smart <james.smart at broadcom.com>
>> This is unnecessary and can create weird configurations. It assumes
>> connections are manually created.
> a) connections can (and will) be maually created and for this the users
> have to know the topology or connection establishment will fail b) there
> is no need that the connections are manually created. Sagi did post an
> RFC systemd service which calls nvme connect-all and this is what should
> be done regardless if we're running on FC-NVME or NVMe over RDMA any new
> transport that may come in the future. What I want is a consistent user
> experience within NVMe, as I am the one that has to answer a
> documentation team's inquiries on how to configure NVMf, support QA in
> testing and fix end user bugs. The least thing I want ot do is tell them
> "well if you use rdma you have to use the nvme-connect.service, if you
> use FC you have to have some magic udev rules and auto-connect scripts,
> if you use $POSSIBLE_NEW_TRANSPORT you have to yada, yada".
>
> I don't really like that we have to manully connect either but this
> behaviour was first in NVMe so we should either stick to that or convert
> RDMA over to using some sort of udev magic as well (which wont work as
> far as I know, and it definitively won't work with TCP if and when it's
> there).
>

Sagi's RFC is certainly fine to use with FC-NVME as well. But the 
mechanism does require the admin to have apriori knowledge and populate 
the discovery.conf file with the discovery controller information. It 
also requires updates on any dynamic change. All of that is unnecessary 
with the FC-NVME auto-connect scripts. Note: there's nothing wrong with 
using both mechanisms simultaneously with FC-NVME.

I certainly understand the sentiment of not doing something N ways. But, 
in this case, the functionality is so useful and valuable, that 
constraining the solution to the least common denominator is a bad 
idea.? You are also ignoring the user-base coming from SCSI on FC (which 
is much larger than nvmf on RDMA) that have completely different 
expectations on how the system finds targets and connects to storage. 
They will be confused and have lots of questions about why SCSI is so 
automatic yet (even on the same machines/adapters/fabric) they now must 
have all kinds of upfront knowledge and make manual changes for NVME. I 
get bombarded on this and I know you will too. ? Although it's not nice 
to state transport-specific methods, I believe the message is still 
rather short and simple.

We seem to have gotten away from the desired patch into a completely 
different area.?? In truth, adding the patch isn't a big deal, but to me 
it doesn't provide any value over the existing scripts. You could say it 
adds yet another one off in FC-specific behavior that could complicate 
the documentation.

-- james