* [PATCH v2 01/20] lpfc: Fix plogi collision that causes illegal state transition
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 7:49 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 02/20] lpfc: Fix loop mode target discovery James Smart
` (18 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
Message "0271 Illegal State Transition: node" seen in logs, all
luns are unuseable for that target.
A window exists in the rcv_plogi path where if the state is plogi
issue but the driver has not issued a plogi, then two reglogins will
be sent for the same RPI. The first one to complete will advance
the state to prli issue the second one will be detected as an
illegal state, and leave the node in an unusable state.
Correct the completion routine for the PLOGI ACC that detects the
state change when the driver starts discovery on the node again
and drop the REGLOGIN mailbox command.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_els.c | 26 +++++++++++++++++++++++++-
1 file changed, 25 insertions(+), 1 deletion(-)
diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c
index 6d1d6f691df4..9e9aa8bb7001 100644
--- a/drivers/scsi/lpfc/lpfc_els.c
+++ b/drivers/scsi/lpfc/lpfc_els.c
@@ -3930,7 +3930,31 @@ lpfc_cmpl_els_rsp(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
if (mbox) {
if ((rspiocb->iocb.ulpStatus == 0)
&& (ndlp->nlp_flag & NLP_ACC_REGLOGIN)) {
- lpfc_unreg_rpi(vport, ndlp);
+ if (!lpfc_unreg_rpi(vport, ndlp)) {
+ if (ndlp->nlp_state == NLP_STE_PLOGI_ISSUE ||
+ ndlp->nlp_state ==
+ NLP_STE_REG_LOGIN_ISSUE) {
+ lpfc_printf_vlog(vport, KERN_INFO,
+ LOG_DISCOVERY,
+ "0314 PLOGI recov "
+ "DID x%x "
+ "Data: x%x x%x x%x\n",
+ ndlp->nlp_DID,
+ ndlp->nlp_state,
+ ndlp->nlp_rpi,
+ ndlp->nlp_flag);
+ mp = (struct lpfc_dmabuf *)
+ mbox->context1;
+ if (mp) {
+ lpfc_mbuf_free(phba, mp->virt,
+ mp->phys);
+ kfree(mp);
+ }
+ mempool_free(mbox, phba->mbox_mem_pool);
+ goto out;
+ }
+ }
+
/* Increment reference count to ndlp to hold the
* reference to ndlp for the callback function.
*/
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* Re: [PATCH v2 01/20] lpfc: Fix plogi collision that causes illegal state transition
2017-08-05 0:47 ` [PATCH v2 01/20] lpfc: Fix plogi collision that causes illegal state transition James Smart
@ 2017-08-07 7:49 ` Johannes Thumshirn
0 siblings, 0 replies; 42+ messages in thread
From: Johannes Thumshirn @ 2017-08-07 7:49 UTC (permalink / raw)
To: James Smart; +Cc: linux-scsi, Dick Kennedy, James Smart
On Fri, Aug 04, 2017 at 05:47:12PM -0700, James Smart wrote:
> From: Dick Kennedy <dick.kennedy@broadcom.com>
>
> Message "0271 Illegal State Transition: node" seen in logs, all
> luns are unuseable for that target.
>
> A window exists in the rcv_plogi path where if the state is plogi
> issue but the driver has not issued a plogi, then two reglogins will
> be sent for the same RPI. The first one to complete will advance
> the state to prli issue the second one will be detected as an
> illegal state, and leave the node in an unusable state.
>
> Correct the completion routine for the PLOGI ACC that detects the
> state change when the driver starts discovery on the node again
> and drop the REGLOGIN mailbox command.
>
> Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
> Signed-off-by: James Smart <james.smart@broadcom.com>
> ---
> drivers/scsi/lpfc/lpfc_els.c | 26 +++++++++++++++++++++++++-
> 1 file changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c
> index 6d1d6f691df4..9e9aa8bb7001 100644
> --- a/drivers/scsi/lpfc/lpfc_els.c
> +++ b/drivers/scsi/lpfc/lpfc_els.c
> @@ -3930,7 +3930,31 @@ lpfc_cmpl_els_rsp(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
> if (mbox) {
> if ((rspiocb->iocb.ulpStatus == 0)
> && (ndlp->nlp_flag & NLP_ACC_REGLOGIN)) {
> - lpfc_unreg_rpi(vport, ndlp);
> + if (!lpfc_unreg_rpi(vport, ndlp)) {
> + if (ndlp->nlp_state == NLP_STE_PLOGI_ISSUE ||
> + ndlp->nlp_state ==
> + NLP_STE_REG_LOGIN_ISSUE) {
if (!lpfc_unreg_rpi(vport, ndlp) &&
(ndlp->nlp_state == NLP_STE_PLOGI_ISSUE ||
ndlp->nlp_state == NLP_STE_REG_LOGIN_ISSUE)) {
This saves you one level of indention and given that we already have two
before we reach the 'if' it's highly desirable.
> + lpfc_printf_vlog(vport, KERN_INFO,
> + LOG_DISCOVERY,
> + "0314 PLOGI recov "
> + "DID x%x "
> + "Data: x%x x%x x%x\n",
> + ndlp->nlp_DID,
> + ndlp->nlp_state,
> + ndlp->nlp_rpi,
> + ndlp->nlp_flag);
> + mp = (struct lpfc_dmabuf *)
> + mbox->context1;
No need for a cast.
Otherwise,
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 42+ messages in thread
* [PATCH v2 02/20] lpfc: Fix loop mode target discovery
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
2017-08-05 0:47 ` [PATCH v2 01/20] lpfc: Fix plogi collision that causes illegal state transition James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 7:55 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 03/20] lpfc: Fix rediscovery on switch blade pull James Smart
` (17 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
The driver does not discover targets when in loop mode.
The NLP type is correctly getting set when a fabric connection is
detected but, not for loop. The unknown NLP type means that the
driver does not issue a PRLI when in loop topology. Thus target
discovery fails.
Fix by checking the topology during discovery. If it is loop,
set the NLP FC4 type to FCP.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_nportdisc.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/scsi/lpfc/lpfc_nportdisc.c b/drivers/scsi/lpfc/lpfc_nportdisc.c
index f74cb0142fd4..95b2b43ac37d 100644
--- a/drivers/scsi/lpfc/lpfc_nportdisc.c
+++ b/drivers/scsi/lpfc/lpfc_nportdisc.c
@@ -1724,6 +1724,9 @@ lpfc_cmpl_reglogin_reglogin_issue(struct lpfc_vport *vport,
lpfc_nvme_update_localport(vport);
}
+ } else if (phba->fc_topology == LPFC_TOPOLOGY_LOOP) {
+ ndlp->nlp_fc4_type |= NLP_FC4_FCP;
+
} else if (ndlp->nlp_fc4_type == 0) {
rc = lpfc_ns_cmd(vport, SLI_CTNS_GFT_ID,
0, ndlp->nlp_DID);
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 03/20] lpfc: Fix rediscovery on switch blade pull
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
2017-08-05 0:47 ` [PATCH v2 01/20] lpfc: Fix plogi collision that causes illegal state transition James Smart
2017-08-05 0:47 ` [PATCH v2 02/20] lpfc: Fix loop mode target discovery James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 7:56 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 04/20] lpfc: Fix oops when NVME Target is discovered in a nonNVME environment James Smart
` (16 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
When the switch blade is pulled out then plugged back in, the driver
does not issue a PLOGI to the target
When the switch blade is pulled out, it does not reset the link. The
driver ends up issuing a LOGO to the target, and finally sees devloss.
Since the driver believes that a LOGO is outstanding, it does not
issue a PLOGI to the target upon link up
Correct by placing the ndlp in UNUSED state When devloss happens in
LOGO_ISSUE state.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_nportdisc.c | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/drivers/scsi/lpfc/lpfc_nportdisc.c b/drivers/scsi/lpfc/lpfc_nportdisc.c
index 95b2b43ac37d..a4488d6339c1 100644
--- a/drivers/scsi/lpfc/lpfc_nportdisc.c
+++ b/drivers/scsi/lpfc/lpfc_nportdisc.c
@@ -2192,12 +2192,15 @@ lpfc_device_rm_logo_issue(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
void *arg, uint32_t evt)
{
/*
- * Take no action. If a LOGO is outstanding, then possibly DevLoss has
- * timed out and is calling for Device Remove. In this case, the LOGO
- * must be allowed to complete in state LOGO_ISSUE so that the rpi
- * and other NLP flags are correctly cleaned up.
+ * DevLoss has timed out and is calling for Device Remove.
+ * In this case, abort the LOGO and cleanup the ndlp
*/
- return ndlp->nlp_state;
+
+ lpfc_unreg_rpi(vport, ndlp);
+ /* software abort outstanding PLOGI */
+ lpfc_els_abort(vport->phba, ndlp);
+ lpfc_drop_node(vport, ndlp);
+ return NLP_STE_FREED_NODE;
}
static uint32_t
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 04/20] lpfc: Fix oops when NVME Target is discovered in a nonNVME environment
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (2 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 03/20] lpfc: Fix rediscovery on switch blade pull James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 8:00 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 05/20] lpfc: convert info messages to standard messages James Smart
` (15 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
lpfc oops when it discovers a NVME target but is configured for SCSI
only operation. Oops is in lpfc_nvme_register_port+0x33/0x300.
The localport is not valid so it should not have been referenced.
Added validity check for localport
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_nvme.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c
index 0a0a1b92d01d..2f50df6a3dca 100644
--- a/drivers/scsi/lpfc/lpfc_nvme.c
+++ b/drivers/scsi/lpfc/lpfc_nvme.c
@@ -2296,6 +2296,9 @@ lpfc_nvme_register_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp)
ndlp->nlp_DID, ndlp->nlp_type);
localport = vport->localport;
+ if (!localport)
+ return 0;
+
lport = (struct lpfc_nvme_lport *)localport->private;
/* NVME rports are not preserved across devloss.
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* Re: [PATCH v2 04/20] lpfc: Fix oops when NVME Target is discovered in a nonNVME environment
2017-08-05 0:47 ` [PATCH v2 04/20] lpfc: Fix oops when NVME Target is discovered in a nonNVME environment James Smart
@ 2017-08-07 8:00 ` Johannes Thumshirn
2017-08-07 20:34 ` James Smart
0 siblings, 1 reply; 42+ messages in thread
From: Johannes Thumshirn @ 2017-08-07 8:00 UTC (permalink / raw)
To: James Smart; +Cc: linux-scsi, Dick Kennedy, James Smart
On Fri, Aug 04, 2017 at 05:47:15PM -0700, James Smart wrote:
> From: Dick Kennedy <dick.kennedy@broadcom.com>
>
> lpfc oops when it discovers a NVME target but is configured for SCSI
> only operation. Oops is in lpfc_nvme_register_port+0x33/0x300.
Why does it even call lpfc_nvme_register_rport if it isn't configured for
handling NVMe?
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [PATCH v2 04/20] lpfc: Fix oops when NVME Target is discovered in a nonNVME environment
2017-08-07 8:00 ` Johannes Thumshirn
@ 2017-08-07 20:34 ` James Smart
0 siblings, 0 replies; 42+ messages in thread
From: James Smart @ 2017-08-07 20:34 UTC (permalink / raw)
To: Johannes Thumshirn; +Cc: linux-scsi, Dick Kennedy, James Smart
On 8/7/2017 1:00 AM, Johannes Thumshirn wrote:
> On Fri, Aug 04, 2017 at 05:47:15PM -0700, James Smart wrote:
>> From: Dick Kennedy <dick.kennedy@broadcom.com>
>>
>> lpfc oops when it discovers a NVME target but is configured for SCSI
>> only operation. Oops is in lpfc_nvme_register_port+0x33/0x300.
>
> Why does it even call lpfc_nvme_register_rport if it isn't configured for
> handling NVMe?
>
That was the bug.... it got confused about protocol support.
-- james
^ permalink raw reply [flat|nested] 42+ messages in thread
* [PATCH v2 05/20] lpfc: convert info messages to standard messages
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (3 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 04/20] lpfc: Fix oops when NVME Target is discovered in a nonNVME environment James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 8:00 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 06/20] lpfc: Correct return error codes to align with nvme_fc transport James Smart
` (14 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
Transitioned some informational discovery messages to now always
be displayed when log_verbose is set.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_ct.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/scsi/lpfc/lpfc_ct.c b/drivers/scsi/lpfc/lpfc_ct.c
index 9c0c1463057d..33417681f5d4 100644
--- a/drivers/scsi/lpfc/lpfc_ct.c
+++ b/drivers/scsi/lpfc/lpfc_ct.c
@@ -955,7 +955,7 @@ lpfc_cmpl_ct_cmd_gft_id(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
CTrsp = (struct lpfc_sli_ct_request *)outp->virt;
fc4_data_0 = be32_to_cpu(CTrsp->un.gft_acc.fc4_types[0]);
fc4_data_1 = be32_to_cpu(CTrsp->un.gft_acc.fc4_types[1]);
- lpfc_printf_vlog(vport, KERN_ERR, LOG_DISCOVERY,
+ lpfc_printf_vlog(vport, KERN_INFO, LOG_DISCOVERY,
"3062 DID x%06x GFT Wd0 x%08x Wd1 x%08x\n",
did, fc4_data_0, fc4_data_1);
@@ -969,7 +969,7 @@ lpfc_cmpl_ct_cmd_gft_id(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
ndlp->nlp_fc4_type |= NLP_FC4_FCP;
if (fc4_data_1 & LPFC_FC4_TYPE_BITMASK)
ndlp->nlp_fc4_type |= NLP_FC4_NVME;
- lpfc_printf_vlog(vport, KERN_ERR, LOG_DISCOVERY,
+ lpfc_printf_vlog(vport, KERN_INFO, LOG_DISCOVERY,
"3064 Setting ndlp %p, DID x%06x with "
"FC4 x%08x, Data: x%08x x%08x\n",
ndlp, did, ndlp->nlp_fc4_type,
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 06/20] lpfc: Correct return error codes to align with nvme_fc transport
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (4 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 05/20] lpfc: convert info messages to standard messages James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 8:07 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 07/20] lpfc: Fix handling of FCP and NVME FC4 types in Pt2Pt topology James Smart
` (13 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
Modify driver return error codes to align with host nvme transport.
Driver isn't returning Exxx error codes to properly reflect out of
resource or connectivity conditions (-EBUSY), yet there were hard
error conditions returning -EBUSY.
Fix to return:
temporary failures or temporary resource availability: -EBUSY
Connectivity issues: -ENODEV
all others are treated as hard errors and return an -Exxx value
that indicates the type of error.
Also, lpfc_sli4_issue_wqe() was modified to not translate error
from -Exxx to WQE state. This allows lpfc_nvme_fcp_io_submit()
routine to just return whatever -E value was returned from other
routines.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_nvme.c | 9 ++++-----
drivers/scsi/lpfc/lpfc_sli.c | 23 ++++++++++++++---------
2 files changed, 18 insertions(+), 14 deletions(-)
diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c
index 2f50df6a3dca..87e0743c4ace 100644
--- a/drivers/scsi/lpfc/lpfc_nvme.c
+++ b/drivers/scsi/lpfc/lpfc_nvme.c
@@ -364,7 +364,7 @@ lpfc_nvme_gen_req(struct lpfc_vport *vport, struct lpfc_dmabuf *bmp,
genwqe->sli4_xritag, genwqe->iotag, ndlp->nlp_DID);
rc = lpfc_sli4_issue_wqe(phba, LPFC_ELS_RING, genwqe);
- if (rc == WQE_ERROR) {
+ if (rc) {
lpfc_printf_vlog(vport, KERN_ERR, LOG_ELS,
"6045 Issue GEN REQ WQE to NPORT x%x "
"Data: x%x x%x\n",
@@ -1270,7 +1270,7 @@ lpfc_nvme_fcp_io_submit(struct nvme_fc_local_port *pnvme_lport,
* not exceed the programmed depth.
*/
if (atomic_read(&ndlp->cmd_pending) >= ndlp->cmd_qdepth) {
- ret = -EAGAIN;
+ ret = -EBUSY;
goto out_fail;
}
@@ -1279,7 +1279,7 @@ lpfc_nvme_fcp_io_submit(struct nvme_fc_local_port *pnvme_lport,
lpfc_printf_vlog(vport, KERN_INFO, LOG_NVME_IOERR,
"6065 driver's buffer pool is empty, "
"IO failed\n");
- ret = -ENOMEM;
+ ret = -EBUSY;
goto out_fail;
}
#ifdef CONFIG_SCSI_LPFC_DEBUG_FS
@@ -1332,7 +1332,6 @@ lpfc_nvme_fcp_io_submit(struct nvme_fc_local_port *pnvme_lport,
"sid: x%x did: x%x oxid: x%x\n",
ret, vport->fc_myDID, ndlp->nlp_DID,
lpfc_ncmd->cur_iocbq.sli4_xritag);
- ret = -EBUSY;
goto out_free_nvme_buf;
}
@@ -1576,7 +1575,7 @@ lpfc_nvme_fcp_abort(struct nvme_fc_local_port *pnvme_lport,
abts_buf->wqe_cmpl = lpfc_nvme_abort_fcreq_cmpl;
ret_val = lpfc_sli4_issue_wqe(phba, LPFC_FCP_RING, abts_buf);
spin_unlock_irqrestore(&phba->hbalock, flags);
- if (ret_val == IOCB_ERROR) {
+ if (ret_val) {
lpfc_printf_vlog(vport, KERN_ERR, LOG_NVME_ABTS,
"6137 Failed abts issue_wqe with status x%x "
"for nvme_fcreq %p.\n",
diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index e948ea05fd33..d68afec00f57 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -106,7 +106,7 @@ lpfc_get_iocb_from_iocbq(struct lpfc_iocbq *iocbq)
* -ENOMEM.
* The caller is expected to hold the hbalock when calling this routine.
**/
-static uint32_t
+static int
lpfc_sli4_wq_put(struct lpfc_queue *q, union lpfc_wqe *wqe)
{
union lpfc_wqe *temp_wqe;
@@ -123,7 +123,7 @@ lpfc_sli4_wq_put(struct lpfc_queue *q, union lpfc_wqe *wqe)
idx = ((q->host_index + 1) % q->entry_count);
if (idx == q->hba_index) {
q->WQ_overflow++;
- return -ENOMEM;
+ return -EBUSY;
}
q->WQ_posted++;
/* set consumption flag every once in a while */
@@ -10741,7 +10741,7 @@ lpfc_sli4_abort_nvme_io(struct lpfc_hba *phba, struct lpfc_sli_ring *pring,
abtsiocbp->vport = vport;
abtsiocbp->wqe_cmpl = lpfc_nvme_abort_fcreq_cmpl;
retval = lpfc_sli4_issue_wqe(phba, LPFC_FCP_RING, abtsiocbp);
- if (retval == IOCB_ERROR) {
+ if (retval) {
lpfc_printf_vlog(vport, KERN_ERR, LOG_NVME,
"6147 Failed abts issue_wqe with status x%x "
"for oxid x%x\n",
@@ -18888,6 +18888,7 @@ lpfc_sli4_issue_wqe(struct lpfc_hba *phba, uint32_t ring_number,
struct lpfc_sglq *sglq;
struct lpfc_sli_ring *pring;
unsigned long iflags;
+ uint32_t ret = 0;
/* NVME_LS and NVME_LS ABTS requests. */
if (pwqe->iocb_flag & LPFC_IO_NVME_LS) {
@@ -18906,10 +18907,12 @@ lpfc_sli4_issue_wqe(struct lpfc_hba *phba, uint32_t ring_number,
}
bf_set(wqe_xri_tag, &pwqe->wqe.xmit_bls_rsp.wqe_com,
pwqe->sli4_xritag);
- if (lpfc_sli4_wq_put(phba->sli4_hba.nvmels_wq, wqe)) {
+ ret = lpfc_sli4_wq_put(phba->sli4_hba.nvmels_wq, wqe);
+ if (ret) {
spin_unlock_irqrestore(&pring->ring_lock, iflags);
- return WQE_ERROR;
+ return ret;
}
+
lpfc_sli_ringtxcmpl_put(phba, pring, pwqe);
spin_unlock_irqrestore(&pring->ring_lock, iflags);
return 0;
@@ -18924,9 +18927,10 @@ lpfc_sli4_issue_wqe(struct lpfc_hba *phba, uint32_t ring_number,
wq = phba->sli4_hba.nvme_wq[pwqe->hba_wqidx];
bf_set(wqe_cqid, &wqe->generic.wqe_com,
phba->sli4_hba.nvme_cq[pwqe->hba_wqidx]->queue_id);
- if (lpfc_sli4_wq_put(wq, wqe)) {
+ ret = lpfc_sli4_wq_put(wq, wqe);
+ if (ret) {
spin_unlock_irqrestore(&pring->ring_lock, iflags);
- return WQE_ERROR;
+ return ret;
}
lpfc_sli_ringtxcmpl_put(phba, pring, pwqe);
spin_unlock_irqrestore(&pring->ring_lock, iflags);
@@ -18950,9 +18954,10 @@ lpfc_sli4_issue_wqe(struct lpfc_hba *phba, uint32_t ring_number,
wq = phba->sli4_hba.nvme_wq[pwqe->hba_wqidx];
bf_set(wqe_cqid, &wqe->generic.wqe_com,
phba->sli4_hba.nvme_cq[pwqe->hba_wqidx]->queue_id);
- if (lpfc_sli4_wq_put(wq, wqe)) {
+ ret = lpfc_sli4_wq_put(wq, wqe);
+ if (ret) {
spin_unlock_irqrestore(&pring->ring_lock, iflags);
- return WQE_ERROR;
+ return ret;
}
lpfc_sli_ringtxcmpl_put(phba, pring, pwqe);
spin_unlock_irqrestore(&pring->ring_lock, iflags);
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* Re: [PATCH v2 06/20] lpfc: Correct return error codes to align with nvme_fc transport
2017-08-05 0:47 ` [PATCH v2 06/20] lpfc: Correct return error codes to align with nvme_fc transport James Smart
@ 2017-08-07 8:07 ` Johannes Thumshirn
0 siblings, 0 replies; 42+ messages in thread
From: Johannes Thumshirn @ 2017-08-07 8:07 UTC (permalink / raw)
To: James Smart; +Cc: linux-scsi, Dick Kennedy, James Smart
The change itself is OK, but the Changelog is a bit odd to read, IMHO.
Unfortunately I can't come up with a better version either. If you can think
of one I'd be very happy.
Thanks,
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 42+ messages in thread
* [PATCH v2 07/20] lpfc: Fix handling of FCP and NVME FC4 types in Pt2Pt topology
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (5 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 06/20] lpfc: Correct return error codes to align with nvme_fc transport James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 8:18 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 08/20] lpfc: Fix duplicate NVME rport entries and namespaces James Smart
` (12 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
After link bounce in a NVME Pt2Pt config, the driver managed to
map the same nport twice, resulting in multiple device nodes for the
same namespace.
In Pt2Pt, the driver must send PRLI's for both (scsi) FCP and NVME
rather than using fabric aids. The driver was inconsistent on handling
various PRLI completions, especially rejects, which had reject codes
cross the different protocol PRLI completions.
Fixed to perform the following:
if nvmet mode (fc port can only be a nvme target) - rejects all
unsolicitly FCP PRLI's. Never issues a FCP PRLI.
The multiple protocol PRLI's are sent simultaneously. However, driver
will now only state transition after both PRLI's are complete. New
flags were added to aid tracking the responses from the different PRLI's.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_disc.h | 1 +
drivers/scsi/lpfc/lpfc_els.c | 30 ++++++++++++++++++++++++++++++
drivers/scsi/lpfc/lpfc_nportdisc.c | 9 +++++++++
3 files changed, 40 insertions(+)
diff --git a/drivers/scsi/lpfc/lpfc_disc.h b/drivers/scsi/lpfc/lpfc_disc.h
index 094c97b9e5f7..f9a566eaef04 100644
--- a/drivers/scsi/lpfc/lpfc_disc.h
+++ b/drivers/scsi/lpfc/lpfc_disc.h
@@ -159,6 +159,7 @@ struct lpfc_node_rrq {
#define NLP_RNID_SND 0x00000400 /* sent RNID request for this entry */
#define NLP_ELS_SND_MASK 0x000007e0 /* sent ELS request for this entry */
#define NLP_NVMET_RECOV 0x00001000 /* NVMET auditing node for recovery. */
+#define NLP_FCP_PRLI_RJT 0x00002000 /* Rport does not support FCP PRLI. */
#define NLP_DEFER_RM 0x00010000 /* Remove this ndlp if no longer used */
#define NLP_DELAY_TMO 0x00020000 /* delay timeout is running for node */
#define NLP_NPR_2B_DISC 0x00040000 /* node is included in num_disc_nodes */
diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c
index 9e9aa8bb7001..6962e22087c8 100644
--- a/drivers/scsi/lpfc/lpfc_els.c
+++ b/drivers/scsi/lpfc/lpfc_els.c
@@ -1966,6 +1966,7 @@ int
lpfc_issue_els_plogi(struct lpfc_vport *vport, uint32_t did, uint8_t retry)
{
struct lpfc_hba *phba = vport->phba;
+ struct Scsi_Host *shost;
struct serv_parm *sp;
struct lpfc_nodelist *ndlp;
struct lpfc_iocbq *elsiocb;
@@ -1984,6 +1985,11 @@ lpfc_issue_els_plogi(struct lpfc_vport *vport, uint32_t did, uint8_t retry)
if (!elsiocb)
return 1;
+ shost = lpfc_shost_from_vport(vport);
+ spin_lock_irq(shost->host_lock);
+ ndlp->nlp_flag &= ~NLP_FCP_PRLI_RJT;
+ spin_unlock_irq(shost->host_lock);
+
pcmd = (uint8_t *) (((struct lpfc_dmabuf *) elsiocb->context2)->virt);
/* For PLOGI request, remainder of payload is service parameters */
@@ -3442,6 +3448,23 @@ lpfc_els_retry(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
goto out_retry;
}
break;
+ case LSRJT_CMD_UNSUPPORTED:
+ /* lpfc nvmet returns this type of LS_RJT when it
+ * receives an FCP PRLI because lpfc nvmet only
+ * support NVME. ELS request is terminated for FCP4
+ * on this rport.
+ */
+ if (stat.un.b.lsRjtRsnCodeExp ==
+ LSEXP_REQ_UNSUPPORTED) {
+ if (cmd == ELS_CMD_PRLI) {
+ spin_lock_irq(shost->host_lock);
+ ndlp->nlp_flag |= NLP_FCP_PRLI_RJT;
+ spin_unlock_irq(shost->host_lock);
+ retry = 0;
+ goto out_retry;
+ }
+ }
+ break;
}
break;
@@ -8013,6 +8036,13 @@ lpfc_els_unsol_buffer(struct lpfc_hba *phba, struct lpfc_sli_ring *pring,
rjt_exp = LSEXP_NOTHING_MORE;
break;
}
+
+ /* NVMET accepts NVME PRLI only. Reject FCP PRLI */
+ if (cmd == ELS_CMD_PRLI && phba->nvmet_support) {
+ rjt_err = LSRJT_CMD_UNSUPPORTED;
+ rjt_exp = LSEXP_REQ_UNSUPPORTED;
+ break;
+ }
lpfc_disc_state_machine(vport, ndlp, elsiocb, NLP_EVT_RCV_PRLI);
break;
case ELS_CMD_LIRR:
diff --git a/drivers/scsi/lpfc/lpfc_nportdisc.c b/drivers/scsi/lpfc/lpfc_nportdisc.c
index a4488d6339c1..f3ad7cac355d 100644
--- a/drivers/scsi/lpfc/lpfc_nportdisc.c
+++ b/drivers/scsi/lpfc/lpfc_nportdisc.c
@@ -1895,6 +1895,15 @@ lpfc_cmpl_prli_prli_issue(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
goto out;
}
+ /* When the rport rejected the FCP PRLI as unsupported.
+ * This should only happen in Pt2Pt so an NVME PRLI
+ * should be outstanding still.
+ */
+ if (npr && ndlp->nlp_flag & NLP_FCP_PRLI_RJT) {
+ ndlp->nlp_fc4_type &= ~NLP_FC4_FCP;
+ goto out_err;
+ }
+
/* The LS Req had some error. Don't let this be a
* target.
*/
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* Re: [PATCH v2 07/20] lpfc: Fix handling of FCP and NVME FC4 types in Pt2Pt topology
2017-08-05 0:47 ` [PATCH v2 07/20] lpfc: Fix handling of FCP and NVME FC4 types in Pt2Pt topology James Smart
@ 2017-08-07 8:18 ` Johannes Thumshirn
0 siblings, 0 replies; 42+ messages in thread
From: Johannes Thumshirn @ 2017-08-07 8:18 UTC (permalink / raw)
To: James Smart; +Cc: linux-scsi, Dick Kennedy, James Smart
On Fri, Aug 04, 2017 at 05:47:18PM -0700, James Smart wrote:
> + case LSRJT_CMD_UNSUPPORTED:
> + /* lpfc nvmet returns this type of LS_RJT when it
> + * receives an FCP PRLI because lpfc nvmet only
> + * support NVME. ELS request is terminated for FCP4
> + * on this rport.
> + */
> + if (stat.un.b.lsRjtRsnCodeExp ==
> + LSEXP_REQ_UNSUPPORTED) {
> + if (cmd == ELS_CMD_PRLI) {
> + spin_lock_irq(shost->host_lock);
> + ndlp->nlp_flag |= NLP_FCP_PRLI_RJT;
> + spin_unlock_irq(shost->host_lock);
> + retry = 0;
> + goto out_retry;
> + }
> + }
if (stat.un.b.lsRjtRsnCodeExp ==
LSEXP_REQ_UNSUPPORTED && cmd == ELS_CMD_PRLI ) {
spin_lock_irq(shost->host_lock);
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 42+ messages in thread
* [PATCH v2 08/20] lpfc: Fix duplicate NVME rport entries and namespaces.
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (6 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 07/20] lpfc: Fix handling of FCP and NVME FC4 types in Pt2Pt topology James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 8:33 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 09/20] lpfc: Fix crash in lpfc nvmet when fc port is reset James Smart
` (11 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
After lip, the driver sometimes would have two rports for the same
device, allowing the namespaces to be duplicated by nvme.
In lpfc_plogi_confirm_nport() the driver was not swapping the
nrport maintained by the ndlp's undergoing address swapping. This
allowed the 2nd rport to sneak in as it was considered a separate
device.
This patch adds the fixes to Swap the nrport in each ndlp and take
care of the reference counts on the ndlps similar to FCP rports.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_els.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c
index 6962e22087c8..fe0dd42b75af 100644
--- a/drivers/scsi/lpfc/lpfc_els.c
+++ b/drivers/scsi/lpfc/lpfc_els.c
@@ -1527,6 +1527,7 @@ lpfc_plogi_confirm_nport(struct lpfc_hba *phba, uint32_t *prsp,
uint8_t name[sizeof(struct lpfc_name)];
uint32_t rc, keepDID = 0, keep_nlp_flag = 0;
uint16_t keep_nlp_state;
+ struct lpfc_nvme_rport *keep_nrport = NULL;
int put_node;
int put_rport;
unsigned long *active_rrqs_xri_bitmap = NULL;
@@ -1624,6 +1625,10 @@ lpfc_plogi_confirm_nport(struct lpfc_hba *phba, uint32_t *prsp,
keep_nlp_state = new_ndlp->nlp_state;
lpfc_nlp_set_state(vport, new_ndlp, ndlp->nlp_state);
+ /* For nvme, swap the nrport. */
+ keep_nrport = new_ndlp->nrport;
+ new_ndlp->nrport = ndlp->nrport;
+
/* Move this back to NPR state */
if (memcmp(&ndlp->nlp_portname, name, sizeof(struct lpfc_name)) == 0) {
/* The new_ndlp is replacing ndlp totally, so we need
@@ -1646,6 +1651,13 @@ lpfc_plogi_confirm_nport(struct lpfc_hba *phba, uint32_t *prsp,
}
new_ndlp->nlp_type = ndlp->nlp_type;
}
+
+ /* Fix up the nvme rport */
+ if (ndlp->nrport) {
+ ndlp->nrport = NULL;
+ lpfc_nlp_put(ndlp);
+ }
+
/* We shall actually free the ndlp with both nlp_DID and
* nlp_portname fields equals 0 to avoid any ndlp on the
* nodelist never to be used.
@@ -1690,6 +1702,14 @@ lpfc_plogi_confirm_nport(struct lpfc_hba *phba, uint32_t *prsp,
keep_nlp_state = NLP_STE_NPR_NODE;
lpfc_nlp_set_state(vport, ndlp, keep_nlp_state);
+ /* Previous ndlp no longer active with nvme host transport.
+ * Remove reference from earlier registration unless the
+ * nvme host took care of it.
+ */
+ if (ndlp->nrport)
+ lpfc_nlp_put(ndlp);
+ ndlp->nrport = keep_nrport;
+
/* Fix up the rport accordingly */
rport = ndlp->rport;
if (rport) {
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* Re: [PATCH v2 08/20] lpfc: Fix duplicate NVME rport entries and namespaces.
2017-08-05 0:47 ` [PATCH v2 08/20] lpfc: Fix duplicate NVME rport entries and namespaces James Smart
@ 2017-08-07 8:33 ` Johannes Thumshirn
0 siblings, 0 replies; 42+ messages in thread
From: Johannes Thumshirn @ 2017-08-07 8:33 UTC (permalink / raw)
To: James Smart; +Cc: linux-scsi, Dick Kennedy, James Smart
On Fri, Aug 04, 2017 at 05:47:19PM -0700, James Smart wrote:
> + /* For nvme, swap the nrport. */
> + keep_nrport = new_ndlp->nrport;
> + new_ndlp->nrport = ndlp->nrport;
> +
The above comment did trigger a "Please use the swap() macro" warning in my
brain, which isn't applicable here. Can we maybe reprase it to something like:
/* For NVMe, cache new_ndlp->nrport and override with ndlp->nrport */
Otherwise,
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 42+ messages in thread
* [PATCH v2 09/20] lpfc: Fix crash in lpfc nvmet when fc port is reset
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (7 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 08/20] lpfc: Fix duplicate NVME rport entries and namespaces James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 8:34 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 10/20] lpfc: Fix NVME PRLI handling during RSCN James Smart
` (10 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
in adapter reset tests, an oops was seen with a NULL pointer in
lpfc_free_rq_buffer+0x20/0x60
The driver is failing to properly repost the nvmet sgl list when
recovering from the reset. Thus the driver eventually trys to
walk an errant buffer list.
Corrected the sgl buffer recovery as well as strengthening the
initialization of the bufferlist.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_init.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 491aa95eb0f6..5f32a7fe169d 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -3226,6 +3226,13 @@ lpfc_offline(struct lpfc_hba *phba)
/* stop port and all timers associated with this hba */
lpfc_stop_port(phba);
+
+ /* Tear down the local and target port registrations. The
+ * nvme transports need to cleanup.
+ */
+ lpfc_nvmet_destroy_targetport(phba);
+ lpfc_nvme_destroy_localport(phba->pport);
+
vports = lpfc_create_vport_work_array(phba);
if (vports != NULL)
for (i = 0; i <= phba->max_vports && vports[i] != NULL; i++)
@@ -6516,6 +6523,12 @@ lpfc_free_nvmet_sgl_list(struct lpfc_hba *phba)
lpfc_nvmet_buf_free(phba, sglq_entry->virt, sglq_entry->phys);
kfree(sglq_entry);
}
+
+ /* Update the nvmet_xri_cnt to reflect no current sgls.
+ * The next initialization cycle sets the count and allocates
+ * the sgls over again.
+ */
+ phba->sli4_hba.nvmet_xri_cnt = 0;
}
/**
@@ -8301,6 +8314,9 @@ lpfc_sli4_queue_create(struct lpfc_hba *phba)
goto out_error;
}
+ /* Put list in known state in case driver load fails. */
+ INIT_LIST_HEAD(&qdesc->rqbp->rqb_buffer_list);
+
/* Create NVMET Receive Queue for data */
qdesc = lpfc_sli4_queue_alloc(phba,
phba->sli4_hba.rq_esize,
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 10/20] lpfc: Fix NVME PRLI handling during RSCN
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (8 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 09/20] lpfc: Fix crash in lpfc nvmet when fc port is reset James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 8:35 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 11/20] lpfc: Correct issues with FAWWN and FDISCs James Smart
` (9 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
A race condition was found whereby the initiator would receive
the RSCN for a new NVME device before it had a chance to register
its FC4 support with the fabric. Thus, when queried by the initiator,
it would see that the target supported FC-NVME.
Corrected by making the assumption that the target always supports
FC-NVME thus a PRLI is sent. It's ok for the target to reject it.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_els.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c
index fe0dd42b75af..9a8808abe61f 100644
--- a/drivers/scsi/lpfc/lpfc_els.c
+++ b/drivers/scsi/lpfc/lpfc_els.c
@@ -2177,6 +2177,16 @@ lpfc_issue_els_prli(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
uint16_t cmdsize;
u32 local_nlp_type, elscmd;
+ /*
+ * If we are in RSCN mode, the FC4 types supported from a
+ * previous GFT_ID command may not be accurate. So, if we
+ * are a NVME Initiator, always look for the possibility of
+ * the remote NPort beng a NVME Target.
+ */
+ if (phba->sli_rev == LPFC_SLI_REV4 &&
+ vport->fc_flag & FC_RSCN_MODE &&
+ vport->nvmei_support)
+ ndlp->nlp_fc4_type |= NLP_FC4_NVME;
local_nlp_type = ndlp->nlp_fc4_type;
send_next_prli:
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 11/20] lpfc: Correct issues with FAWWN and FDISCs
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (9 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 10/20] lpfc: Fix NVME PRLI handling during RSCN James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-05 0:47 ` [PATCH v2 12/20] lpfc: Limit amount of work processed in IRQ James Smart
` (8 subsequent siblings)
19 siblings, 0 replies; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
When using fabric-assigned WWNs, the switch doesn't like copy of
the FLOGI payload, which includes valid VVL bits, to be used as
the FDISC paylaod.
Rather than wait for corrected switch firmware, ensure the VVL bits
are marked invalid on FDISCs.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_els.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c
index 9a8808abe61f..f97d578ea6bd 100644
--- a/drivers/scsi/lpfc/lpfc_els.c
+++ b/drivers/scsi/lpfc/lpfc_els.c
@@ -8868,6 +8868,7 @@ lpfc_issue_els_fdisc(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp,
pcmd += sizeof(uint32_t); /* Node Name */
pcmd += sizeof(uint32_t); /* Node Name */
memcpy(pcmd, &vport->fc_nodename, 8);
+ sp->cmn.valid_vendor_ver_level = 0;
memset(sp->un.vendorVersion, 0, sizeof(sp->un.vendorVersion));
lpfc_set_disctmo(vport);
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 12/20] lpfc: Limit amount of work processed in IRQ
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (10 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 11/20] lpfc: Correct issues with FAWWN and FDISCs James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 8:43 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 13/20] lpfc: Fix MRQ > 1 context list handling James Smart
` (7 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
Various oops being seen on being in the ISR too long and cpu
lockups, when under heavy load.
The amount of work being posted off of completion queues kept
the ISR running almost all the time
Correct the issue by limiting the amount of work per itteration.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_sli.c | 31 +++++++++++++++++++------------
drivers/scsi/lpfc/lpfc_sli4.h | 1 +
2 files changed, 20 insertions(+), 12 deletions(-)
diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index d68afec00f57..9ccd8fed0bed 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -80,8 +80,8 @@ static int lpfc_sli4_fp_handle_cqe(struct lpfc_hba *, struct lpfc_queue *,
struct lpfc_cqe *);
static int lpfc_sli4_post_sgl_list(struct lpfc_hba *, struct list_head *,
int);
-static void lpfc_sli4_hba_handle_eqe(struct lpfc_hba *, struct lpfc_eqe *,
- uint32_t);
+static int lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba,
+ struct lpfc_eqe *eqe, uint32_t qidx);
static bool lpfc_sli4_mbox_completions_pending(struct lpfc_hba *phba);
static bool lpfc_sli4_process_missed_mbox_completions(struct lpfc_hba *phba);
static int lpfc_sli4_abort_nvme_io(struct lpfc_hba *phba,
@@ -13010,7 +13010,7 @@ lpfc_sli4_sp_handle_cqe(struct lpfc_hba *phba, struct lpfc_queue *cq,
* completion queue, and then return.
*
**/
-static void
+static int
lpfc_sli4_sp_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe,
struct lpfc_queue *speq)
{
@@ -13034,7 +13034,7 @@ lpfc_sli4_sp_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe,
lpfc_printf_log(phba, KERN_ERR, LOG_SLI,
"0365 Slow-path CQ identifier "
"(%d) does not exist\n", cqid);
- return;
+ return 0;
}
/* Save EQ associated with this CQ */
@@ -13071,7 +13071,7 @@ lpfc_sli4_sp_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe,
lpfc_printf_log(phba, KERN_ERR, LOG_SLI,
"0370 Invalid completion queue type (%d)\n",
cq->type);
- return;
+ return 0;
}
/* Catch the no cq entry condition, log an error */
@@ -13086,6 +13086,8 @@ lpfc_sli4_sp_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe,
/* wake up worker thread if there are works to be done */
if (workposted)
lpfc_worker_wake_up(phba);
+
+ return ecount;
}
/**
@@ -13393,7 +13395,7 @@ lpfc_sli4_fp_handle_cqe(struct lpfc_hba *phba, struct lpfc_queue *cq,
* queue and process all the entries on the completion queue, rearm the
* completion queue, and then return.
**/
-static void
+static int
lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe,
uint32_t qidx)
{
@@ -13409,7 +13411,7 @@ lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe,
"event: majorcode=x%x, minorcode=x%x\n",
bf_get_le32(lpfc_eqe_major_code, eqe),
bf_get_le32(lpfc_eqe_minor_code, eqe));
- return;
+ return 0;
}
/* Get the reference to the corresponding CQ */
@@ -13446,8 +13448,9 @@ lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe,
/* Otherwise this is a Slow path event */
if (cq == NULL) {
- lpfc_sli4_sp_handle_eqe(phba, eqe, phba->sli4_hba.hba_eq[qidx]);
- return;
+ ecount = lpfc_sli4_sp_handle_eqe(phba, eqe,
+ phba->sli4_hba.hba_eq[qidx]);
+ return ecount;
}
process_cq:
@@ -13456,7 +13459,7 @@ lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe,
"0368 Miss-matched fast-path completion "
"queue identifier: eqcqid=%d, fcpcqid=%d\n",
cqid, cq->queue_id);
- return;
+ return 0;
}
/* Save EQ associated with this CQ */
@@ -13486,6 +13489,8 @@ lpfc_sli4_hba_handle_eqe(struct lpfc_hba *phba, struct lpfc_eqe *eqe,
/* wake up worker thread if there are works to be done */
if (workposted)
lpfc_worker_wake_up(phba);
+
+ return ecount;
}
static void
@@ -13706,6 +13711,7 @@ lpfc_sli4_hba_intr_handler(int irq, void *dev_id)
struct lpfc_eqe *eqe;
unsigned long iflag;
int ecount = 0;
+ int ccount = 0;
int hba_eqidx;
/* Get the driver's phba structure from the dev_id */
@@ -13757,8 +13763,9 @@ lpfc_sli4_hba_intr_handler(int irq, void *dev_id)
if (eqe == NULL)
break;
- lpfc_sli4_hba_handle_eqe(phba, eqe, hba_eqidx);
- if (!(++ecount % fpeq->entry_repost))
+ ccount += lpfc_sli4_hba_handle_eqe(phba, eqe, hba_eqidx);
+ if (!(++ecount % fpeq->entry_repost) ||
+ ccount > LPFC_MAX_ISR_CQE)
break;
fpeq->EQ_processed++;
}
diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h
index 7a1d74e9e877..540454b03587 100644
--- a/drivers/scsi/lpfc/lpfc_sli4.h
+++ b/drivers/scsi/lpfc/lpfc_sli4.h
@@ -158,6 +158,7 @@ struct lpfc_queue {
#define LPFC_MQ_REPOST 8
#define LPFC_CQ_REPOST 64
#define LPFC_RQ_REPOST 64
+#define LPFC_MAX_ISR_CQE 64
#define LPFC_RELEASE_NOTIFICATION_INTERVAL 32 /* For WQs */
uint32_t queue_id; /* Queue ID assigned by the hardware */
uint32_t assoc_qid; /* Queue ID associated with, for CQ/WQ/MQ */
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* Re: [PATCH v2 12/20] lpfc: Limit amount of work processed in IRQ
2017-08-05 0:47 ` [PATCH v2 12/20] lpfc: Limit amount of work processed in IRQ James Smart
@ 2017-08-07 8:43 ` Johannes Thumshirn
2017-08-07 20:50 ` James Smart
0 siblings, 1 reply; 42+ messages in thread
From: Johannes Thumshirn @ 2017-08-07 8:43 UTC (permalink / raw)
To: James Smart; +Cc: linux-scsi, Dick Kennedy, James Smart
On Fri, Aug 04, 2017 at 05:47:23PM -0700, James Smart wrote:
> From: Dick Kennedy <dick.kennedy@broadcom.com>
>
> Various oops being seen on being in the ISR too long and cpu
> lockups, when under heavy load.
>
> The amount of work being posted off of completion queues kept
> the ISR running almost all the time
>
> Correct the issue by limiting the amount of work per itteration.
Have you considered using the IRQ poll mechanism for this? It deferres the
heavy lifing into the Soft-IRQ layer and this could help you a lot here. IIRC
I've sent James the "Slides" from my LSF/MM session which have the function
protoypes on 'em. See 'lib/irq_poll.c' or the be2iscsi driver for further
details.
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [PATCH v2 12/20] lpfc: Limit amount of work processed in IRQ
2017-08-07 8:43 ` Johannes Thumshirn
@ 2017-08-07 20:50 ` James Smart
0 siblings, 0 replies; 42+ messages in thread
From: James Smart @ 2017-08-07 20:50 UTC (permalink / raw)
To: Johannes Thumshirn; +Cc: linux-scsi, Dick Kennedy, James Smart
On 8/7/2017 1:43 AM, Johannes Thumshirn wrote:
> On Fri, Aug 04, 2017 at 05:47:23PM -0700, James Smart wrote:
>> From: Dick Kennedy <dick.kennedy@broadcom.com>
>>
>> Various oops being seen on being in the ISR too long and cpu
>> lockups, when under heavy load.
>>
>> The amount of work being posted off of completion queues kept
>> the ISR running almost all the time
>>
>> Correct the issue by limiting the amount of work per itteration.
>
> Have you considered using the IRQ poll mechanism for this? It deferres the
> heavy lifing into the Soft-IRQ layer and this could help you a lot here. IIRC
> I've sent James the "Slides" from my LSF/MM session which have the function
> protoypes on 'em. See 'lib/irq_poll.c' or the be2iscsi driver for further
> details.
We considered it, so far declined it - preferring the behavior of the
dynamic hw coalescing we have. That said, we're already considering the
softirq approach, which shouldn't be that large of a delta. However,
we've chosen to delay that change while we harden the overall stability.
-- james
^ permalink raw reply [flat|nested] 42+ messages in thread
* [PATCH v2 13/20] lpfc: Fix MRQ > 1 context list handling
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (11 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 12/20] lpfc: Limit amount of work processed in IRQ James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 9:02 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 14/20] lpfc: Fix relative offset error on large nvmet target ios James Smart
` (6 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
Various oops including cpu LOCKUPs were seen.
For asynchronously received ius where the driver must assign
exchange resources, the resources were on a single get (free) list
and put list (finished, waiting to be put on get list). As all cpus
are sharing the lists, an interrupt for a receive frame may have to
wait for all the other cpus to place their done work onto the put
list before it can acquire the lock to pull from the list.
Fix by breaking the resource lists into per-cpu lists or at least
more than 1 list with cpu's sharing the lists). A cpu would allocate
from the free list for its own cpu, and put its done work on the its own
put list - avoiding the contention. As cpu load may vary, when empty, a
cpu may grab from another cpu, thereby changing resource distribution.
But searching for a resource only occurs on 1 or a few cpus until a
single resource can be allocated. if the condition reoccurs, it starts
looking at a different cpu.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_attr.c | 11 +-
drivers/scsi/lpfc/lpfc_crtn.h | 5 +-
drivers/scsi/lpfc/lpfc_debugfs.c | 11 +-
drivers/scsi/lpfc/lpfc_init.c | 18 ++-
drivers/scsi/lpfc/lpfc_nvmet.c | 265 ++++++++++++++++++++++++++++++---------
drivers/scsi/lpfc/lpfc_nvmet.h | 14 +++
drivers/scsi/lpfc/lpfc_sli.c | 2 +-
drivers/scsi/lpfc/lpfc_sli4.h | 7 +-
8 files changed, 236 insertions(+), 97 deletions(-)
diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c
index 4ed48ed38e79..d3d01ff44423 100644
--- a/drivers/scsi/lpfc/lpfc_attr.c
+++ b/drivers/scsi/lpfc/lpfc_attr.c
@@ -245,13 +245,10 @@ lpfc_nvme_info_show(struct device *dev, struct device_attribute *attr,
atomic_read(&tgtp->xmt_abort_rsp),
atomic_read(&tgtp->xmt_abort_rsp_error));
- spin_lock(&phba->sli4_hba.nvmet_ctx_get_lock);
- spin_lock(&phba->sli4_hba.nvmet_ctx_put_lock);
- tot = phba->sli4_hba.nvmet_xri_cnt -
- (phba->sli4_hba.nvmet_ctx_get_cnt +
- phba->sli4_hba.nvmet_ctx_put_cnt);
- spin_unlock(&phba->sli4_hba.nvmet_ctx_put_lock);
- spin_unlock(&phba->sli4_hba.nvmet_ctx_get_lock);
+ /* Calculate outstanding IOs */
+ tot = atomic_read(&tgtp->rcv_fcp_cmd_drop);
+ tot += atomic_read(&tgtp->xmt_fcp_release);
+ tot = atomic_read(&tgtp->rcv_fcp_cmd_in) - tot;
len += snprintf(buf + len, PAGE_SIZE - len,
"IO_CTX: %08x WAIT: cur %08x tot %08x\n"
diff --git a/drivers/scsi/lpfc/lpfc_crtn.h b/drivers/scsi/lpfc/lpfc_crtn.h
index da669dce12fe..7e300734b345 100644
--- a/drivers/scsi/lpfc/lpfc_crtn.h
+++ b/drivers/scsi/lpfc/lpfc_crtn.h
@@ -556,9 +556,8 @@ int lpfc_nvmet_update_targetport(struct lpfc_hba *phba);
void lpfc_nvmet_destroy_targetport(struct lpfc_hba *phba);
void lpfc_nvmet_unsol_ls_event(struct lpfc_hba *phba,
struct lpfc_sli_ring *pring, struct lpfc_iocbq *piocb);
-void lpfc_nvmet_unsol_fcp_event(struct lpfc_hba *phba,
- struct lpfc_sli_ring *pring,
- struct rqb_dmabuf *nvmebuf, uint64_t isr_ts);
+void lpfc_nvmet_unsol_fcp_event(struct lpfc_hba *phba, uint32_t idx,
+ struct rqb_dmabuf *nvmebuf, uint64_t isr_ts);
void lpfc_nvme_mod_param_dep(struct lpfc_hba *phba);
void lpfc_nvme_abort_fcreq_cmpl(struct lpfc_hba *phba,
struct lpfc_iocbq *cmdiocb,
diff --git a/drivers/scsi/lpfc/lpfc_debugfs.c b/drivers/scsi/lpfc/lpfc_debugfs.c
index 5cc8b0f7d885..c292264aa687 100644
--- a/drivers/scsi/lpfc/lpfc_debugfs.c
+++ b/drivers/scsi/lpfc/lpfc_debugfs.c
@@ -848,13 +848,10 @@ lpfc_debugfs_nvmestat_data(struct lpfc_vport *vport, char *buf, int size)
spin_unlock(&phba->sli4_hba.abts_nvme_buf_list_lock);
}
- spin_lock(&phba->sli4_hba.nvmet_ctx_get_lock);
- spin_lock(&phba->sli4_hba.nvmet_ctx_put_lock);
- tot = phba->sli4_hba.nvmet_xri_cnt -
- (phba->sli4_hba.nvmet_ctx_get_cnt +
- phba->sli4_hba.nvmet_ctx_put_cnt);
- spin_unlock(&phba->sli4_hba.nvmet_ctx_put_lock);
- spin_unlock(&phba->sli4_hba.nvmet_ctx_get_lock);
+ /* Calculate outstanding IOs */
+ tot = atomic_read(&tgtp->rcv_fcp_cmd_drop);
+ tot += atomic_read(&tgtp->xmt_fcp_release);
+ tot = atomic_read(&tgtp->rcv_fcp_cmd_in) - tot;
len += snprintf(buf + len, size - len,
"IO_CTX: %08x WAIT: cur %08x tot %08x\n"
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 5f32a7fe169d..897eb62ed4e6 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -1253,6 +1253,7 @@ lpfc_hb_timeout_handler(struct lpfc_hba *phba)
unsigned long time_elapsed;
uint32_t tick_cqe, max_cqe, val;
uint64_t tot, data1, data2, data3;
+ struct lpfc_nvmet_tgtport *tgtp;
struct lpfc_register reg_data;
void __iomem *eqdreg = phba->sli4_hba.u.if_type2.EQDregaddr;
@@ -1281,13 +1282,12 @@ lpfc_hb_timeout_handler(struct lpfc_hba *phba)
/* Check outstanding IO count */
if (phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME) {
if (phba->nvmet_support) {
- spin_lock(&phba->sli4_hba.nvmet_ctx_get_lock);
- spin_lock(&phba->sli4_hba.nvmet_ctx_put_lock);
- tot = phba->sli4_hba.nvmet_xri_cnt -
- (phba->sli4_hba.nvmet_ctx_get_cnt +
- phba->sli4_hba.nvmet_ctx_put_cnt);
- spin_unlock(&phba->sli4_hba.nvmet_ctx_put_lock);
- spin_unlock(&phba->sli4_hba.nvmet_ctx_get_lock);
+ tgtp = (struct lpfc_nvmet_tgtport *)
+ phba->targetport->private;
+ /* Calculate outstanding IOs */
+ tot = atomic_read(&tgtp->rcv_fcp_cmd_drop);
+ tot += atomic_read(&tgtp->xmt_fcp_release);
+ tot = atomic_read(&tgtp->rcv_fcp_cmd_in) - tot;
} else {
tot = atomic_read(&phba->fc4NvmeIoCmpls);
data1 = atomic_read(
@@ -5937,8 +5937,6 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
spin_lock_init(&phba->sli4_hba.abts_nvme_buf_list_lock);
INIT_LIST_HEAD(&phba->sli4_hba.lpfc_abts_nvme_buf_list);
INIT_LIST_HEAD(&phba->sli4_hba.lpfc_abts_nvmet_ctx_list);
- INIT_LIST_HEAD(&phba->sli4_hba.lpfc_nvmet_ctx_get_list);
- INIT_LIST_HEAD(&phba->sli4_hba.lpfc_nvmet_ctx_put_list);
INIT_LIST_HEAD(&phba->sli4_hba.lpfc_nvmet_io_wait_list);
/* Fast-path XRI aborted CQ Event work queue list */
@@ -5947,8 +5945,6 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba)
/* This abort list used by worker thread */
spin_lock_init(&phba->sli4_hba.sgl_list_lock);
- spin_lock_init(&phba->sli4_hba.nvmet_ctx_get_lock);
- spin_lock_init(&phba->sli4_hba.nvmet_ctx_put_lock);
spin_lock_init(&phba->sli4_hba.nvmet_io_wait_lock);
/*
diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c
index fbeec344c6cc..cd6c93a17760 100644
--- a/drivers/scsi/lpfc/lpfc_nvmet.c
+++ b/drivers/scsi/lpfc/lpfc_nvmet.c
@@ -170,8 +170,10 @@ lpfc_nvmet_ctxbuf_post(struct lpfc_hba *phba, struct lpfc_nvmet_ctxbuf *ctx_buf)
struct lpfc_nvmet_tgtport *tgtp;
struct fc_frame_header *fc_hdr;
struct rqb_dmabuf *nvmebuf;
+ struct lpfc_nvmet_ctx_info *infop;
uint32_t *payload;
uint32_t size, oxid, sid, rc;
+ int cpu;
unsigned long iflag;
if (ctxp->txrdy) {
@@ -267,11 +269,16 @@ lpfc_nvmet_ctxbuf_post(struct lpfc_hba *phba, struct lpfc_nvmet_ctxbuf *ctx_buf)
}
spin_unlock_irqrestore(&phba->sli4_hba.nvmet_io_wait_lock, iflag);
- spin_lock_irqsave(&phba->sli4_hba.nvmet_ctx_put_lock, iflag);
- list_add_tail(&ctx_buf->list,
- &phba->sli4_hba.lpfc_nvmet_ctx_put_list);
- phba->sli4_hba.nvmet_ctx_put_cnt++;
- spin_unlock_irqrestore(&phba->sli4_hba.nvmet_ctx_put_lock, iflag);
+ /*
+ * Use the CPU context list, from the MRQ the IO was received on
+ * (ctxp->idx), to save context structure.
+ */
+ cpu = smp_processor_id();
+ infop = lpfc_get_ctx_list(phba, cpu, ctxp->idx);
+ spin_lock_irqsave(&infop->nvmet_ctx_list_lock, iflag);
+ list_add_tail(&ctx_buf->list, &infop->nvmet_ctx_list);
+ infop->nvmet_ctx_list_cnt++;
+ spin_unlock_irqrestore(&infop->nvmet_ctx_list_lock, iflag);
#endif
}
@@ -860,51 +867,54 @@ static struct nvmet_fc_target_template lpfc_tgttemplate = {
};
static void
-lpfc_nvmet_cleanup_io_context(struct lpfc_hba *phba)
+__lpfc_nvmet_clean_io_for_cpu(struct lpfc_hba *phba,
+ struct lpfc_nvmet_ctx_info *infop)
{
struct lpfc_nvmet_ctxbuf *ctx_buf, *next_ctx_buf;
unsigned long flags;
- spin_lock_irqsave(&phba->sli4_hba.nvmet_ctx_get_lock, flags);
- spin_lock(&phba->sli4_hba.nvmet_ctx_put_lock);
+ spin_lock_irqsave(&infop->nvmet_ctx_list_lock, flags);
list_for_each_entry_safe(ctx_buf, next_ctx_buf,
- &phba->sli4_hba.lpfc_nvmet_ctx_get_list, list) {
+ &infop->nvmet_ctx_list, list) {
spin_lock(&phba->sli4_hba.abts_nvme_buf_list_lock);
list_del_init(&ctx_buf->list);
spin_unlock(&phba->sli4_hba.abts_nvme_buf_list_lock);
- __lpfc_clear_active_sglq(phba,
- ctx_buf->sglq->sli4_lxritag);
+
+ __lpfc_clear_active_sglq(phba, ctx_buf->sglq->sli4_lxritag);
ctx_buf->sglq->state = SGL_FREED;
ctx_buf->sglq->ndlp = NULL;
spin_lock(&phba->sli4_hba.sgl_list_lock);
list_add_tail(&ctx_buf->sglq->list,
- &phba->sli4_hba.lpfc_nvmet_sgl_list);
+ &phba->sli4_hba.lpfc_nvmet_sgl_list);
spin_unlock(&phba->sli4_hba.sgl_list_lock);
lpfc_sli_release_iocbq(phba, ctx_buf->iocbq);
kfree(ctx_buf->context);
}
- list_for_each_entry_safe(ctx_buf, next_ctx_buf,
- &phba->sli4_hba.lpfc_nvmet_ctx_put_list, list) {
- spin_lock(&phba->sli4_hba.abts_nvme_buf_list_lock);
- list_del_init(&ctx_buf->list);
- spin_unlock(&phba->sli4_hba.abts_nvme_buf_list_lock);
- __lpfc_clear_active_sglq(phba,
- ctx_buf->sglq->sli4_lxritag);
- ctx_buf->sglq->state = SGL_FREED;
- ctx_buf->sglq->ndlp = NULL;
+ spin_unlock_irqrestore(&infop->nvmet_ctx_list_lock, flags);
+}
- spin_lock(&phba->sli4_hba.sgl_list_lock);
- list_add_tail(&ctx_buf->sglq->list,
- &phba->sli4_hba.lpfc_nvmet_sgl_list);
- spin_unlock(&phba->sli4_hba.sgl_list_lock);
+static void
+lpfc_nvmet_cleanup_io_context(struct lpfc_hba *phba)
+{
+ struct lpfc_nvmet_ctx_info *infop;
+ int i, j;
- lpfc_sli_release_iocbq(phba, ctx_buf->iocbq);
- kfree(ctx_buf->context);
+ /* The first context list, MRQ 0 CPU 0 */
+ infop = phba->sli4_hba.nvmet_ctx_info;
+ if (!infop)
+ return;
+
+ /* Cycle the the entire CPU context list for every MRQ */
+ for (i = 0; i < phba->cfg_nvmet_mrq; i++) {
+ for (j = 0; j < phba->sli4_hba.num_present_cpu; j++) {
+ __lpfc_nvmet_clean_io_for_cpu(phba, infop);
+ infop++; /* next */
+ }
}
- spin_unlock(&phba->sli4_hba.nvmet_ctx_put_lock);
- spin_unlock_irqrestore(&phba->sli4_hba.nvmet_ctx_get_lock, flags);
+ kfree(phba->sli4_hba.nvmet_ctx_info);
+ phba->sli4_hba.nvmet_ctx_info = NULL;
}
static int
@@ -913,15 +923,71 @@ lpfc_nvmet_setup_io_context(struct lpfc_hba *phba)
struct lpfc_nvmet_ctxbuf *ctx_buf;
struct lpfc_iocbq *nvmewqe;
union lpfc_wqe128 *wqe;
- int i;
+ struct lpfc_nvmet_ctx_info *last_infop;
+ struct lpfc_nvmet_ctx_info *infop;
+ int i, j, idx;
lpfc_printf_log(phba, KERN_INFO, LOG_NVME,
"6403 Allocate NVMET resources for %d XRIs\n",
phba->sli4_hba.nvmet_xri_cnt);
+ phba->sli4_hba.nvmet_ctx_info = kcalloc(
+ phba->sli4_hba.num_present_cpu * phba->cfg_nvmet_mrq,
+ sizeof(struct lpfc_nvmet_ctx_info), GFP_KERNEL);
+ if (!phba->sli4_hba.nvmet_ctx_info) {
+ lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
+ "6419 Failed allocate memory for "
+ "nvmet context lists\n");
+ return -ENOMEM;
+ }
+
+ /*
+ * Assuming X CPUs in the system, and Y MRQs, allocate some
+ * lpfc_nvmet_ctx_info structures as follows:
+ *
+ * cpu0/mrq0 cpu1/mrq0 ... cpuX/mrq0
+ * cpu0/mrq1 cpu1/mrq1 ... cpuX/mrq1
+ * ...
+ * cpuX/mrqY cpuX/mrqY ... cpuX/mrqY
+ *
+ * Each line represents a MRQ "silo" containing an entry for
+ * every CPU.
+ *
+ * MRQ X is initially assumed to be associated with CPU X, thus
+ * contexts are initially distributed across all MRQs using
+ * the MRQ index (N) as follows cpuN/mrqN. When contexts are
+ * freed, the are freed to the MRQ silo based on the CPU number
+ * of the IO completion. Thus a context that was allocated for MRQ A
+ * whose IO completed on CPU B will be freed to cpuB/mrqA.
+ */
+ infop = phba->sli4_hba.nvmet_ctx_info;
+ for (i = 0; i < phba->sli4_hba.num_present_cpu; i++) {
+ for (j = 0; j < phba->cfg_nvmet_mrq; j++) {
+ INIT_LIST_HEAD(&infop->nvmet_ctx_list);
+ spin_lock_init(&infop->nvmet_ctx_list_lock);
+ infop->nvmet_ctx_list_cnt = 0;
+ infop++;
+ }
+ }
+
+ /*
+ * Setup the next CPU context info ptr for each MRQ.
+ * MRQ 0 will cycle thru CPUs 0 - X separately from
+ * MRQ 1 cycling thru CPUs 0 - X, and so on.
+ */
+ for (j = 0; j < phba->cfg_nvmet_mrq; j++) {
+ last_infop = lpfc_get_ctx_list(phba, 0, j);
+ for (i = phba->sli4_hba.num_present_cpu - 1; i >= 0; i--) {
+ infop = lpfc_get_ctx_list(phba, i, j);
+ infop->nvmet_ctx_next_cpu = last_infop;
+ last_infop = infop;
+ }
+ }
+
/* For all nvmet xris, allocate resources needed to process a
* received command on a per xri basis.
*/
+ idx = 0;
for (i = 0; i < phba->sli4_hba.nvmet_xri_cnt; i++) {
ctx_buf = kzalloc(sizeof(*ctx_buf), GFP_KERNEL);
if (!ctx_buf) {
@@ -976,12 +1042,35 @@ lpfc_nvmet_setup_io_context(struct lpfc_hba *phba)
"6407 Ran out of NVMET XRIs\n");
return -ENOMEM;
}
- spin_lock(&phba->sli4_hba.nvmet_ctx_get_lock);
- list_add_tail(&ctx_buf->list,
- &phba->sli4_hba.lpfc_nvmet_ctx_get_list);
- spin_unlock(&phba->sli4_hba.nvmet_ctx_get_lock);
+
+ /*
+ * Add ctx to MRQidx context list. Our initial assumption
+ * is MRQidx will be associated with CPUidx. This association
+ * can change on the fly.
+ */
+ infop = lpfc_get_ctx_list(phba, idx, idx);
+ spin_lock(&infop->nvmet_ctx_list_lock);
+ list_add_tail(&ctx_buf->list, &infop->nvmet_ctx_list);
+ infop->nvmet_ctx_list_cnt++;
+ spin_unlock(&infop->nvmet_ctx_list_lock);
+
+ /* Spread ctx structures evenly across all MRQs */
+ idx++;
+ if (idx >= phba->cfg_nvmet_mrq)
+ idx = 0;
+ }
+
+ infop = phba->sli4_hba.nvmet_ctx_info;
+ for (j = 0; j < phba->cfg_nvmet_mrq; j++) {
+ for (i = 0; i < phba->sli4_hba.num_present_cpu; i++) {
+ lpfc_printf_log(phba, KERN_INFO, LOG_NVME | LOG_INIT,
+ "6408 TOTAL NVMET ctx for CPU %d "
+ "MRQ %d: cnt %d nextcpu %p\n",
+ i, j, infop->nvmet_ctx_list_cnt,
+ infop->nvmet_ctx_next_cpu);
+ infop++;
+ }
}
- phba->sli4_hba.nvmet_ctx_get_cnt = phba->sli4_hba.nvmet_xri_cnt;
return 0;
}
@@ -1346,10 +1435,66 @@ lpfc_nvmet_unsol_ls_buffer(struct lpfc_hba *phba, struct lpfc_sli_ring *pring,
#endif
}
+static struct lpfc_nvmet_ctxbuf *
+lpfc_nvmet_replenish_context(struct lpfc_hba *phba,
+ struct lpfc_nvmet_ctx_info *current_infop)
+{
+ struct lpfc_nvmet_ctxbuf *ctx_buf = NULL;
+ struct lpfc_nvmet_ctx_info *get_infop;
+ int i;
+
+ /*
+ * The current_infop for the MRQ a NVME command IU was received
+ * on is empty. Our goal is to replenish this MRQs context
+ * list from a another CPUs.
+ *
+ * First we need to pick a context list to start looking on.
+ * nvmet_ctx_start_cpu has available context the last time
+ * we needed to replenish this CPU where nvmet_ctx_next_cpu
+ * is just the next sequential CPU for this MRQ.
+ */
+ if (current_infop->nvmet_ctx_start_cpu)
+ get_infop = current_infop->nvmet_ctx_start_cpu;
+ else
+ get_infop = current_infop->nvmet_ctx_next_cpu;
+
+ for (i = 0; i < phba->sli4_hba.num_present_cpu; i++) {
+ if (get_infop == current_infop) {
+ get_infop = get_infop->nvmet_ctx_next_cpu;
+ continue;
+ }
+ spin_lock(&get_infop->nvmet_ctx_list_lock);
+
+ /* Just take the entire context list, if there are any */
+ if (get_infop->nvmet_ctx_list_cnt) {
+ list_splice(&get_infop->nvmet_ctx_list,
+ ¤t_infop->nvmet_ctx_list);
+ INIT_LIST_HEAD(&get_infop->nvmet_ctx_list);
+ current_infop->nvmet_ctx_list_cnt =
+ get_infop->nvmet_ctx_list_cnt - 1;
+ get_infop->nvmet_ctx_list_cnt = 0;
+ spin_unlock(&get_infop->nvmet_ctx_list_lock);
+
+ current_infop->nvmet_ctx_start_cpu = get_infop;
+ list_remove_head(¤t_infop->nvmet_ctx_list,
+ ctx_buf, struct lpfc_nvmet_ctxbuf,
+ list);
+ return ctx_buf;
+ }
+
+ /* Otherwise, move on to the next CPU for this MRQ */
+ spin_unlock(&get_infop->nvmet_ctx_list_lock);
+ get_infop = get_infop->nvmet_ctx_next_cpu;
+ }
+
+ /* Nothing found, all contexts for the MRQ are in-flight */
+ return NULL;
+}
+
/**
* lpfc_nvmet_unsol_fcp_buffer - Process an unsolicited event data buffer
* @phba: pointer to lpfc hba data structure.
- * @pring: pointer to a SLI ring.
+ * @idx: relative index of MRQ vector
* @nvmebuf: pointer to lpfc nvme command HBQ data structure.
*
* This routine is used for processing the WQE associated with a unsolicited
@@ -1361,7 +1506,7 @@ lpfc_nvmet_unsol_ls_buffer(struct lpfc_hba *phba, struct lpfc_sli_ring *pring,
**/
static void
lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba,
- struct lpfc_sli_ring *pring,
+ uint32_t idx,
struct rqb_dmabuf *nvmebuf,
uint64_t isr_timestamp)
{
@@ -1370,9 +1515,11 @@ lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba,
struct lpfc_nvmet_tgtport *tgtp;
struct fc_frame_header *fc_hdr;
struct lpfc_nvmet_ctxbuf *ctx_buf;
+ struct lpfc_nvmet_ctx_info *current_infop;
uint32_t *payload;
uint32_t size, oxid, sid, rc, qno;
unsigned long iflag;
+ int current_cpu;
#ifdef CONFIG_SCSI_LPFC_DEBUG_FS
uint32_t id;
#endif
@@ -1388,31 +1535,24 @@ lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba,
goto dropit;
}
- spin_lock_irqsave(&phba->sli4_hba.nvmet_ctx_get_lock, iflag);
- if (phba->sli4_hba.nvmet_ctx_get_cnt) {
- list_remove_head(&phba->sli4_hba.lpfc_nvmet_ctx_get_list,
+ /*
+ * Get a pointer to the context list for this MRQ based on
+ * the CPU this MRQ IRQ is associated with. If the CPU association
+ * changes from our initial assumption, the context list could
+ * be empty, thus it would need to be replenished with the
+ * context list from another CPU for this MRQ.
+ */
+ current_cpu = smp_processor_id();
+ current_infop = lpfc_get_ctx_list(phba, current_cpu, idx);
+ spin_lock_irqsave(¤t_infop->nvmet_ctx_list_lock, iflag);
+ if (current_infop->nvmet_ctx_list_cnt) {
+ list_remove_head(¤t_infop->nvmet_ctx_list,
ctx_buf, struct lpfc_nvmet_ctxbuf, list);
- phba->sli4_hba.nvmet_ctx_get_cnt--;
+ current_infop->nvmet_ctx_list_cnt--;
} else {
- spin_lock(&phba->sli4_hba.nvmet_ctx_put_lock);
- if (phba->sli4_hba.nvmet_ctx_put_cnt) {
- list_splice(&phba->sli4_hba.lpfc_nvmet_ctx_put_list,
- &phba->sli4_hba.lpfc_nvmet_ctx_get_list);
- INIT_LIST_HEAD(&phba->sli4_hba.lpfc_nvmet_ctx_put_list);
- phba->sli4_hba.nvmet_ctx_get_cnt =
- phba->sli4_hba.nvmet_ctx_put_cnt;
- phba->sli4_hba.nvmet_ctx_put_cnt = 0;
- spin_unlock(&phba->sli4_hba.nvmet_ctx_put_lock);
-
- list_remove_head(
- &phba->sli4_hba.lpfc_nvmet_ctx_get_list,
- ctx_buf, struct lpfc_nvmet_ctxbuf, list);
- phba->sli4_hba.nvmet_ctx_get_cnt--;
- } else {
- spin_unlock(&phba->sli4_hba.nvmet_ctx_put_lock);
- }
+ ctx_buf = lpfc_nvmet_replenish_context(phba, current_infop);
}
- spin_unlock_irqrestore(&phba->sli4_hba.nvmet_ctx_get_lock, iflag);
+ spin_unlock_irqrestore(¤t_infop->nvmet_ctx_list_lock, iflag);
fc_hdr = (struct fc_frame_header *)(nvmebuf->hbuf.virt);
oxid = be16_to_cpu(fc_hdr->fh_ox_id);
@@ -1464,6 +1604,7 @@ lpfc_nvmet_unsol_fcp_buffer(struct lpfc_hba *phba,
ctxp->size = size;
ctxp->oxid = oxid;
ctxp->sid = sid;
+ ctxp->idx = idx;
ctxp->state = LPFC_NVMET_STE_RCV;
ctxp->entry_cnt = 1;
ctxp->flag = 0;
@@ -1561,7 +1702,7 @@ lpfc_nvmet_unsol_ls_event(struct lpfc_hba *phba, struct lpfc_sli_ring *pring,
/**
* lpfc_nvmet_unsol_fcp_event - Process an unsolicited event from an nvme nport
* @phba: pointer to lpfc hba data structure.
- * @pring: pointer to a SLI ring.
+ * @idx: relative index of MRQ vector
* @nvmebuf: pointer to received nvme data structure.
*
* This routine is used to process an unsolicited event received from a SLI
@@ -1572,7 +1713,7 @@ lpfc_nvmet_unsol_ls_event(struct lpfc_hba *phba, struct lpfc_sli_ring *pring,
**/
void
lpfc_nvmet_unsol_fcp_event(struct lpfc_hba *phba,
- struct lpfc_sli_ring *pring,
+ uint32_t idx,
struct rqb_dmabuf *nvmebuf,
uint64_t isr_timestamp)
{
@@ -1580,7 +1721,7 @@ lpfc_nvmet_unsol_fcp_event(struct lpfc_hba *phba,
lpfc_rq_buf_free(phba, &nvmebuf->hbuf);
return;
}
- lpfc_nvmet_unsol_fcp_buffer(phba, pring, nvmebuf,
+ lpfc_nvmet_unsol_fcp_buffer(phba, idx, nvmebuf,
isr_timestamp);
}
diff --git a/drivers/scsi/lpfc/lpfc_nvmet.h b/drivers/scsi/lpfc/lpfc_nvmet.h
index e675ef17be08..88cd62a11c33 100644
--- a/drivers/scsi/lpfc/lpfc_nvmet.h
+++ b/drivers/scsi/lpfc/lpfc_nvmet.h
@@ -73,6 +73,19 @@ struct lpfc_nvmet_tgtport {
atomic_t xmt_abort_rsp_error;
};
+struct lpfc_nvmet_ctx_info {
+ struct list_head nvmet_ctx_list;
+ spinlock_t nvmet_ctx_list_lock; /* lock per CPU */
+ struct lpfc_nvmet_ctx_info *nvmet_ctx_next_cpu;
+ struct lpfc_nvmet_ctx_info *nvmet_ctx_start_cpu;
+ uint16_t nvmet_ctx_list_cnt;
+ char pad[16]; /* pad to a cache-line */
+};
+
+/* This retrieves the context info associated with the specified cpu / mrq */
+#define lpfc_get_ctx_list(phba, cpu, mrq) \
+ (phba->sli4_hba.nvmet_ctx_info + ((cpu * phba->cfg_nvmet_mrq) + mrq))
+
struct lpfc_nvmet_rcv_ctx {
union {
struct nvmefc_tgt_ls_req ls_req;
@@ -91,6 +104,7 @@ struct lpfc_nvmet_rcv_ctx {
uint16_t size;
uint16_t entry_cnt;
uint16_t cpu;
+ uint16_t idx;
uint16_t state;
/* States */
#define LPFC_NVMET_STE_LS_RCV 1
diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c
index 9ccd8fed0bed..c46897c6b875 100644
--- a/drivers/scsi/lpfc/lpfc_sli.c
+++ b/drivers/scsi/lpfc/lpfc_sli.c
@@ -13291,7 +13291,7 @@ lpfc_sli4_nvmet_handle_rcqe(struct lpfc_hba *phba, struct lpfc_queue *cq,
if (fc_hdr->fh_type == FC_TYPE_FCP) {
dma_buf->bytes_recv = bf_get(lpfc_rcqe_length, rcqe);
lpfc_nvmet_unsol_fcp_event(
- phba, phba->sli4_hba.els_wq->pring, dma_buf,
+ phba, idx, dma_buf,
cq->assoc_qp->isr_timestamp);
return false;
}
diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h
index 540454b03587..7dc8c73b5903 100644
--- a/drivers/scsi/lpfc/lpfc_sli4.h
+++ b/drivers/scsi/lpfc/lpfc_sli4.h
@@ -622,8 +622,6 @@ struct lpfc_sli4_hba {
uint16_t scsi_xri_start;
uint16_t els_xri_cnt;
uint16_t nvmet_xri_cnt;
- uint16_t nvmet_ctx_get_cnt;
- uint16_t nvmet_ctx_put_cnt;
uint16_t nvmet_io_wait_cnt;
uint16_t nvmet_io_wait_total;
struct list_head lpfc_els_sgl_list;
@@ -632,9 +630,8 @@ struct lpfc_sli4_hba {
struct list_head lpfc_abts_nvmet_ctx_list;
struct list_head lpfc_abts_scsi_buf_list;
struct list_head lpfc_abts_nvme_buf_list;
- struct list_head lpfc_nvmet_ctx_get_list;
- struct list_head lpfc_nvmet_ctx_put_list;
struct list_head lpfc_nvmet_io_wait_list;
+ struct lpfc_nvmet_ctx_info *nvmet_ctx_info;
struct lpfc_sglq **lpfc_sglq_active_list;
struct list_head lpfc_rpi_hdr_list;
unsigned long *rpi_bmask;
@@ -665,8 +662,6 @@ struct lpfc_sli4_hba {
spinlock_t abts_nvme_buf_list_lock; /* list of aborted SCSI IOs */
spinlock_t abts_scsi_buf_list_lock; /* list of aborted SCSI IOs */
spinlock_t sgl_list_lock; /* list of aborted els IOs */
- spinlock_t nvmet_ctx_get_lock; /* list of avail XRI contexts */
- spinlock_t nvmet_ctx_put_lock; /* list of avail XRI contexts */
spinlock_t nvmet_io_wait_lock; /* IOs waiting for ctx resources */
uint32_t physical_port;
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* Re: [PATCH v2 13/20] lpfc: Fix MRQ > 1 context list handling
2017-08-05 0:47 ` [PATCH v2 13/20] lpfc: Fix MRQ > 1 context list handling James Smart
@ 2017-08-07 9:02 ` Johannes Thumshirn
0 siblings, 0 replies; 42+ messages in thread
From: Johannes Thumshirn @ 2017-08-07 9:02 UTC (permalink / raw)
To: James Smart; +Cc: linux-scsi, Dick Kennedy, James Smart
On Fri, Aug 04, 2017 at 05:47:24PM -0700, James Smart wrote:
> + tgtp = (struct lpfc_nvmet_tgtport *)
> + phba->targetport->private;
Please don't case void pointers.
[...]
> + list_splice(&get_infop->nvmet_ctx_list,
> + ¤t_infop->nvmet_ctx_list);
> + INIT_LIST_HEAD(&get_infop->nvmet_ctx_list);
list_splice_init(&get_infop->nvmet_ctx_list,
¤t_infop->nvmet_ctx_list);
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 42+ messages in thread
* [PATCH v2 14/20] lpfc: Fix relative offset error on large nvmet target ios
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (12 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 13/20] lpfc: Fix MRQ > 1 context list handling James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 9:02 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 15/20] lpfc: Fix nvme target failure after 2nd adapter reset James Smart
` (5 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
if the nvmet_fc transport breaks an io into multiple sequences,
the driver will improperly set the relative offset on the 2nd through
N sequences.
Correct by properly formatting the hw cmd so the relative offset is
picked up from the hw cmd.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_nvmet.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c
index cd6c93a17760..40fa5dd72bfa 100644
--- a/drivers/scsi/lpfc/lpfc_nvmet.c
+++ b/drivers/scsi/lpfc/lpfc_nvmet.c
@@ -559,7 +559,7 @@ lpfc_nvmet_xmt_fcp_op_cmp(struct lpfc_hba *phba, struct lpfc_iocbq *cmdwqe,
/* lpfc_nvmet_xmt_fcp_release() will recycle the context */
} else {
ctxp->entry_cnt++;
- start_clean = offsetof(struct lpfc_iocbq, wqe);
+ start_clean = offsetof(struct lpfc_iocbq, iocb_flag);
memset(((char *)cmdwqe) + start_clean, 0,
(sizeof(struct lpfc_iocbq) - start_clean));
#ifdef CONFIG_SCSI_LPFC_DEBUG_FS
@@ -1024,7 +1024,6 @@ lpfc_nvmet_setup_io_context(struct lpfc_hba *phba)
/* Word 7 */
bf_set(wqe_ct, &wqe->generic.wqe_com, SLI4_CT_RPI);
bf_set(wqe_class, &wqe->generic.wqe_com, CLASS3);
- bf_set(wqe_pu, &wqe->generic.wqe_com, 1);
/* Word 10 */
bf_set(wqe_nvme, &wqe->fcp_tsend.wqe_com, 1);
bf_set(wqe_ebde_cnt, &wqe->generic.wqe_com, 0);
@@ -1974,6 +1973,7 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba,
nvmewqe->sli4_xritag);
/* Word 7 */
+ bf_set(wqe_pu, &wqe->fcp_tsend.wqe_com, 1);
bf_set(wqe_cmnd, &wqe->fcp_tsend.wqe_com, CMD_FCP_TSEND64_WQE);
/* Word 8 */
@@ -2082,6 +2082,7 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba,
nvmewqe->sli4_xritag);
/* Word 7 */
+ bf_set(wqe_pu, &wqe->fcp_treceive.wqe_com, 1);
bf_set(wqe_ar, &wqe->fcp_treceive.wqe_com, 0);
bf_set(wqe_cmnd, &wqe->fcp_treceive.wqe_com,
CMD_FCP_TRECEIVE64_WQE);
@@ -2165,6 +2166,7 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba,
nvmewqe->sli4_xritag);
/* Word 7 */
+ bf_set(wqe_pu, &wqe->fcp_trsp.wqe_com, 0);
bf_set(wqe_ag, &wqe->fcp_trsp.wqe_com, 1);
bf_set(wqe_cmnd, &wqe->fcp_trsp.wqe_com, CMD_FCP_TRSP64_WQE);
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 15/20] lpfc: Fix nvme target failure after 2nd adapter reset
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (13 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 14/20] lpfc: Fix relative offset error on large nvmet target ios James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 9:05 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 16/20] lpfc: Fix bad sgl reposting " James Smart
` (4 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
Fix nvme target failure after 2nd adapter reset
The nonrecovery occurred because the lpfc nvme initiator
function did not reestablish its localport creation with the nvme
host transport in lpfc_oneline. Because of that, an NVME rport
binding could not take place.
Corrected by recreating the localport in the adapter reset recovery
routine.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_init.c | 18 +++++++++++++++++-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 897eb62ed4e6..659662777b65 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -3048,7 +3048,7 @@ lpfc_online(struct lpfc_hba *phba)
{
struct lpfc_vport *vport;
struct lpfc_vport **vports;
- int i;
+ int i, error = 0;
bool vpis_cleared = false;
if (!phba)
@@ -3072,6 +3072,22 @@ lpfc_online(struct lpfc_hba *phba)
if (!phba->sli4_hba.max_cfg_param.vpi_used)
vpis_cleared = true;
spin_unlock_irq(&phba->hbalock);
+
+ /* Reestablish the local initiator port.
+ * The offline process destroyed the previous lport.
+ */
+ if (phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME) {
+ if (!phba->nvmet_support) {
+ error = lpfc_nvme_create_localport(phba->pport);
+ if (error) {
+ lpfc_printf_log(phba, KERN_ERR,
+ LOG_INIT,
+ "6132 NVME restore reg "
+ "failed on nvmei error "
+ "x%x\n", error);
+ }
+ }
+ }
} else {
lpfc_sli_queue_init(phba);
if (lpfc_sli_hba_setup(phba)) { /* Initialize SLI2/SLI3 HBA */
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* Re: [PATCH v2 15/20] lpfc: Fix nvme target failure after 2nd adapter reset
2017-08-05 0:47 ` [PATCH v2 15/20] lpfc: Fix nvme target failure after 2nd adapter reset James Smart
@ 2017-08-07 9:05 ` Johannes Thumshirn
0 siblings, 0 replies; 42+ messages in thread
From: Johannes Thumshirn @ 2017-08-07 9:05 UTC (permalink / raw)
To: James Smart; +Cc: linux-scsi, Dick Kennedy, James Smart
On Fri, Aug 04, 2017 at 05:47:26PM -0700, James Smart wrote:
> +
> + /* Reestablish the local initiator port.
> + * The offline process destroyed the previous lport.
> + */
> + if (phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME) {
> + if (!phba->nvmet_support) {
if (phba->cfg_enable_fc4_type & LPFC_ENABLE_NVME &&
!phba->nvmet_support) {
error = lpfc_nvme_create_localport(phba->pport);
if (error)
lpfc_printf_log(phba, KERN_ERR,
LOG_INIT,
"6132 NVME restore reg "
"failed on nvmei error "
"x%x\n", error);
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 42+ messages in thread
* [PATCH v2 16/20] lpfc: Fix bad sgl reposting after 2nd adapter reset
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (14 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 15/20] lpfc: Fix nvme target failure after 2nd adapter reset James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 9:06 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 17/20] lpfc: remove console log clutter James Smart
` (3 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
Fix bad sgl reposting after 2nd adapter reset
port issue was fixed, the hbacmd reset would take
more than 8 minutes to complete.
There were conflicting NVME SGL posting/reposting responsibilities
between lpfc_online()/lpfc_sli4_hba_setup() and
lpfc_nvme_create_localport(). The lpfc_online() causes a REPOST
on existing NVME SGLs which is not released during the fc port reset.
However, lpfc_nvme_create_localport() wants to allocate new NVME
buffers and post them. Both cancelled out each other which had a side
effect of hosing the mailbox handling that was used to remove the sgl
lists - causing multiple 60s mbx timeouts.
Fix by preserving all SGL lists over the fc port reset.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_nvme.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c
index 87e0743c4ace..8e7cdbd5d7d3 100644
--- a/drivers/scsi/lpfc/lpfc_nvme.c
+++ b/drivers/scsi/lpfc/lpfc_nvme.c
@@ -2181,8 +2181,15 @@ lpfc_nvme_create_localport(struct lpfc_vport *vport)
vport->localport = localport;
lport->vport = vport;
vport->nvmei_support = 1;
- len = lpfc_new_nvme_buf(vport, phba->sli4_hba.nvme_xri_max);
- vport->phba->total_nvme_bufs += len;
+
+ /* Don't post more new bufs if repost already recovered
+ * the nvme sgls.
+ */
+ if (phba->sli4_hba.nvme_xri_cnt == 0) {
+ len = lpfc_new_nvme_buf(vport,
+ phba->sli4_hba.nvme_xri_max);
+ vport->phba->total_nvme_bufs += len;
+ }
}
return ret;
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 17/20] lpfc: remove console log clutter
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (15 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 16/20] lpfc: Fix bad sgl reposting " James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 9:06 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 18/20] lpfc: Add Buffer to Buffer credit recovery support James Smart
` (2 subsequent siblings)
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
Change hw queue binding messages to info - not error.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_nvme.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c
index 8e7cdbd5d7d3..d3c171a72ae0 100644
--- a/drivers/scsi/lpfc/lpfc_nvme.c
+++ b/drivers/scsi/lpfc/lpfc_nvme.c
@@ -110,7 +110,7 @@ lpfc_nvme_create_queue(struct nvme_fc_local_port *pnvme_lport,
qhandle->index = qidx;
}
- lpfc_printf_vlog(vport, KERN_ERR, LOG_NVME,
+ lpfc_printf_vlog(vport, KERN_INFO, LOG_NVME,
"6073 Binding %s HdwQueue %d (cpu %d) to "
"io_channel %d qhandle %p\n", str,
qidx, qhandle->cpu_id, qhandle->index, qhandle);
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 18/20] lpfc: Add Buffer to Buffer credit recovery support
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (16 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 17/20] lpfc: remove console log clutter James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 9:08 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 19/20] scsi: lpfc: fix "integer constant too large" error on 32bit archs James Smart
2017-08-05 0:47 ` [PATCH v2 20/20] lpfc: lpfc version bump 11.4.0.3 James Smart
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
Add Buffer to buffer credit recovery support to the driver.
This is a negotiated feature with the peer that allows for both
sides to detect dropped RRDY's and FC Frames and recover credit.
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc.h | 3 ++-
drivers/scsi/lpfc/lpfc_attr.c | 41 ++++++++++++++++++++++++++++++++++++++++
drivers/scsi/lpfc/lpfc_attr.h | 10 ++++++++++
drivers/scsi/lpfc/lpfc_els.c | 14 +++++++++++++-
drivers/scsi/lpfc/lpfc_hbadisc.c | 12 ++++++++++--
drivers/scsi/lpfc/lpfc_hw.h | 18 +++++++++++++++---
drivers/scsi/lpfc/lpfc_hw4.h | 23 ++++++++++++++++++----
drivers/scsi/lpfc/lpfc_init.c | 5 +++++
drivers/scsi/lpfc/lpfc_mbox.c | 35 +++++++++++++++++++++++++++++++---
drivers/scsi/lpfc/lpfc_sli4.h | 15 +++++++++++++++
10 files changed, 162 insertions(+), 14 deletions(-)
diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h
index 562dc0139735..50aaae3d966e 100644
--- a/drivers/scsi/lpfc/lpfc.h
+++ b/drivers/scsi/lpfc/lpfc.h
@@ -733,7 +733,6 @@ struct lpfc_hba {
uint32_t fc_rttov; /* R_T_TOV timer value */
uint32_t fc_altov; /* AL_TOV timer value */
uint32_t fc_crtov; /* C_R_TOV timer value */
- uint32_t fc_citov; /* C_I_TOV timer value */
struct serv_parm fc_fabparam; /* fabric service parameters buffer */
uint8_t alpa_map[128]; /* AL_PA map from READ_LA */
@@ -757,6 +756,7 @@ struct lpfc_hba {
#define LPFC_NVMET_MAX_PORTS 32
uint8_t mds_diags_support;
uint32_t initial_imax;
+ uint8_t bbcredit_support;
/* HBA Config Parameters */
uint32_t cfg_ack0;
@@ -836,6 +836,7 @@ struct lpfc_hba {
uint32_t cfg_enable_SmartSAN;
uint32_t cfg_enable_mds_diags;
uint32_t cfg_enable_fc4_type;
+ uint32_t cfg_enable_bbcr; /*Enable BB Credit Recovery*/
uint32_t cfg_xri_split;
#define LPFC_ENABLE_FCP 1
#define LPFC_ENABLE_NVME 2
diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c
index d3d01ff44423..0806323829e6 100644
--- a/drivers/scsi/lpfc/lpfc_attr.c
+++ b/drivers/scsi/lpfc/lpfc_attr.c
@@ -1889,6 +1889,36 @@ static inline bool lpfc_rangecheck(uint val, uint min, uint max)
}
/**
+ * lpfc_enable_bbcr_set: Sets an attribute value.
+ * @phba: pointer the the adapter structure.
+ * @val: integer attribute value.
+ *
+ * Description:
+ * Validates the min and max values then sets the
+ * adapter config field if in the valid range. prints error message
+ * and does not set the parameter if invalid.
+ *
+ * Returns:
+ * zero on success
+ * -EINVAL if val is invalid
+ */
+static ssize_t
+lpfc_enable_bbcr_set(struct lpfc_hba *phba, uint val)
+{
+ if (lpfc_rangecheck(val, 0, 1) && phba->sli_rev == LPFC_SLI_REV4) {
+ lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
+ "3068 %s_enable_bbcr changed from %d to %d\n",
+ LPFC_DRIVER_NAME, phba->cfg_enable_bbcr, val);
+ phba->cfg_enable_bbcr = val;
+ return 0;
+ }
+ lpfc_printf_log(phba, KERN_ERR, LOG_INIT,
+ "0451 %s_enable_bbcr cannot set to %d, range is 0, 1\n",
+ LPFC_DRIVER_NAME, val);
+ return -EINVAL;
+}
+
+/**
* lpfc_param_show - Return a cfg attribute value in decimal
*
* Description:
@@ -5111,6 +5141,14 @@ LPFC_ATTR_R(sg_seg_cnt, LPFC_DEFAULT_SG_SEG_CNT, LPFC_DEFAULT_SG_SEG_CNT,
*/
LPFC_ATTR_R(enable_mds_diags, 0, 0, 1, "Enable MDS Diagnostics");
+/*
+ * lpfc_enable_bbcr: Enable BB Credit Recovery
+ * 0 = BB Credit Recovery disabled
+ * 1 = BB Credit Recovery enabled (default)
+ * Value range is [0,1]. Default value is 1.
+ */
+LPFC_BBCR_ATTR_RW(enable_bbcr, 1, 0, 1, "Enable BBC Recovery");
+
struct device_attribute *lpfc_hba_attrs[] = {
&dev_attr_nvme_info,
&dev_attr_bg_info,
@@ -5218,6 +5256,7 @@ struct device_attribute *lpfc_hba_attrs[] = {
&dev_attr_protocol,
&dev_attr_lpfc_xlane_supported,
&dev_attr_lpfc_enable_mds_diags,
+ &dev_attr_lpfc_enable_bbcr,
NULL,
};
@@ -6229,11 +6268,13 @@ lpfc_get_cfgparam(struct lpfc_hba *phba)
lpfc_nvmet_fb_size_init(phba, lpfc_nvmet_fb_size);
lpfc_fcp_io_channel_init(phba, lpfc_fcp_io_channel);
lpfc_nvme_io_channel_init(phba, lpfc_nvme_io_channel);
+ lpfc_enable_bbcr_init(phba, lpfc_enable_bbcr);
if (phba->sli_rev != LPFC_SLI_REV4) {
/* NVME only supported on SLI4 */
phba->nvmet_support = 0;
phba->cfg_enable_fc4_type = LPFC_ENABLE_FCP;
+ phba->cfg_enable_bbcr = 0;
} else {
/* We MUST have FCP support */
if (!(phba->cfg_enable_fc4_type & LPFC_ENABLE_FCP))
diff --git a/drivers/scsi/lpfc/lpfc_attr.h b/drivers/scsi/lpfc/lpfc_attr.h
index d56dafcdd563..931db52692f5 100644
--- a/drivers/scsi/lpfc/lpfc_attr.h
+++ b/drivers/scsi/lpfc/lpfc_attr.h
@@ -46,6 +46,16 @@ lpfc_param_store(name)\
static DEVICE_ATTR(lpfc_##name, S_IRUGO | S_IWUSR,\
lpfc_##name##_show, lpfc_##name##_store)
+#define LPFC_BBCR_ATTR_RW(name, defval, minval, maxval, desc) \
+static uint lpfc_##name = defval;\
+module_param(lpfc_##name, uint, 0444);\
+MODULE_PARM_DESC(lpfc_##name, desc);\
+lpfc_param_show(name)\
+lpfc_param_init(name, defval, minval, maxval)\
+lpfc_param_store(name)\
+static DEVICE_ATTR(lpfc_##name, 0444 | 0644,\
+ lpfc_##name##_show, lpfc_##name##_store)
+
#define LPFC_ATTR_HEX_R(name, defval, minval, maxval, desc) \
static uint lpfc_##name = defval;\
module_param(lpfc_##name, uint, S_IRUGO);\
diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c
index f97d578ea6bd..0193b144599b 100644
--- a/drivers/scsi/lpfc/lpfc_els.c
+++ b/drivers/scsi/lpfc/lpfc_els.c
@@ -2033,6 +2033,7 @@ lpfc_issue_els_plogi(struct lpfc_vport *vport, uint32_t did, uint8_t retry)
sp->cmn.valid_vendor_ver_level = 0;
memset(sp->un.vendorVersion, 0, sizeof(sp->un.vendorVersion));
+ sp->cmn.bbRcvSizeMsb &= 0xF;
lpfc_debugfs_disc_trc(vport, LPFC_DISC_TRC_ELS_CMD,
"Issue PLOGI: did:x%x",
@@ -3456,8 +3457,18 @@ lpfc_els_retry(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb,
maxretry = 3;
delay = 1000;
retry = 1;
- break;
+ } else if (cmd == ELS_CMD_FLOGI &&
+ stat.un.b.lsRjtRsnCodeExp ==
+ LSEXP_NOTHING_MORE) {
+ vport->fc_sparam.cmn.bbRcvSizeMsb &= 0xf;
+ retry = 1;
+ lpfc_printf_vlog(vport, KERN_ERR, LOG_ELS,
+ "0820 FLOGI Failed (x%x). "
+ "BBCredit Not Supported\n",
+ stat.un.lsRjtError);
}
+ break;
+
case LSRJT_PROTOCOL_ERR:
if ((phba->sli3_options & LPFC_SLI3_NPIV_ENABLED) &&
(cmd == ELS_CMD_FDISC) &&
@@ -4209,6 +4220,7 @@ lpfc_els_rsp_acc(struct lpfc_vport *vport, uint32_t flag,
sp->cmn.valid_vendor_ver_level = 0;
memset(sp->un.vendorVersion, 0,
sizeof(sp->un.vendorVersion));
+ sp->cmn.bbRcvSizeMsb &= 0xF;
/* If our firmware supports this feature, convey that
* info to the target using the vendor specific field.
diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c
index aa5e5ff56dfb..20808349a80e 100644
--- a/drivers/scsi/lpfc/lpfc_hbadisc.c
+++ b/drivers/scsi/lpfc/lpfc_hbadisc.c
@@ -1108,6 +1108,7 @@ void
lpfc_mbx_cmpl_local_config_link(struct lpfc_hba *phba, LPFC_MBOXQ_t *pmb)
{
struct lpfc_vport *vport = pmb->vport;
+ uint8_t bbscn = 0;
if (pmb->u.mb.mbxStatus)
goto out;
@@ -1134,10 +1135,17 @@ lpfc_mbx_cmpl_local_config_link(struct lpfc_hba *phba, LPFC_MBOXQ_t *pmb)
/* Start discovery by sending a FLOGI. port_state is identically
* LPFC_FLOGI while waiting for FLOGI cmpl
*/
- if (vport->port_state != LPFC_FLOGI)
+ if (vport->port_state != LPFC_FLOGI) {
+ if (phba->bbcredit_support && phba->cfg_enable_bbcr) {
+ bbscn = bf_get(lpfc_bbscn_def,
+ &phba->sli4_hba.bbscn_params);
+ vport->fc_sparam.cmn.bbRcvSizeMsb &= 0xf;
+ vport->fc_sparam.cmn.bbRcvSizeMsb |= (bbscn << 4);
+ }
lpfc_initial_flogi(vport);
- else if (vport->fc_flag & FC_PT2PT)
+ } else if (vport->fc_flag & FC_PT2PT) {
lpfc_disc_start(vport);
+ }
return;
out:
diff --git a/drivers/scsi/lpfc/lpfc_hw.h b/drivers/scsi/lpfc/lpfc_hw.h
index 26a5647e057e..bdc1f184f67a 100644
--- a/drivers/scsi/lpfc/lpfc_hw.h
+++ b/drivers/scsi/lpfc/lpfc_hw.h
@@ -2293,15 +2293,27 @@ typedef struct {
uint32_t rttov;
uint32_t altov;
uint32_t crtov;
- uint32_t citov;
+
+#ifdef __BIG_ENDIAN_BITFIELD
+ uint32_t rsvd4:19;
+ uint32_t cscn:1;
+ uint32_t bbscn:4;
+ uint32_t rsvd3:8;
+#else /* __LITTLE_ENDIAN_BITFIELD */
+ uint32_t rsvd3:8;
+ uint32_t bbscn:4;
+ uint32_t cscn:1;
+ uint32_t rsvd4:19;
+#endif
+
#ifdef __BIG_ENDIAN_BITFIELD
uint32_t rrq_enable:1;
uint32_t rrq_immed:1;
- uint32_t rsvd4:29;
+ uint32_t rsvd5:29;
uint32_t ack0_enable:1;
#else /* __LITTLE_ENDIAN_BITFIELD */
uint32_t ack0_enable:1;
- uint32_t rsvd4:29;
+ uint32_t rsvd5:29;
uint32_t rrq_immed:1;
uint32_t rrq_enable:1;
#endif
diff --git a/drivers/scsi/lpfc/lpfc_hw4.h b/drivers/scsi/lpfc/lpfc_hw4.h
index bb4715705fa3..1db0a38683f4 100644
--- a/drivers/scsi/lpfc/lpfc_hw4.h
+++ b/drivers/scsi/lpfc/lpfc_hw4.h
@@ -2217,9 +2217,15 @@ struct lpfc_mbx_reg_vfi {
uint32_t e_d_tov;
uint32_t r_a_tov;
uint32_t word10;
-#define lpfc_reg_vfi_nport_id_SHIFT 0
-#define lpfc_reg_vfi_nport_id_MASK 0x00FFFFFF
-#define lpfc_reg_vfi_nport_id_WORD word10
+#define lpfc_reg_vfi_nport_id_SHIFT 0
+#define lpfc_reg_vfi_nport_id_MASK 0x00FFFFFF
+#define lpfc_reg_vfi_nport_id_WORD word10
+#define lpfc_reg_vfi_bbcr_SHIFT 27
+#define lpfc_reg_vfi_bbcr_MASK 0x00000001
+#define lpfc_reg_vfi_bbcr_WORD word10
+#define lpfc_reg_vfi_bbscn_SHIFT 28
+#define lpfc_reg_vfi_bbscn_MASK 0x0000000F
+#define lpfc_reg_vfi_bbscn_WORD word10
};
struct lpfc_mbx_init_vpi {
@@ -2646,7 +2652,16 @@ struct lpfc_mbx_read_config {
#define lpfc_mbx_rd_conf_link_speed_MASK 0x0000FFFF
#define lpfc_mbx_rd_conf_link_speed_WORD word6
uint32_t rsvd_7;
- uint32_t rsvd_8;
+ uint32_t word8;
+#define lpfc_mbx_rd_conf_bbscn_min_SHIFT 0
+#define lpfc_mbx_rd_conf_bbscn_min_MASK 0x0000000F
+#define lpfc_mbx_rd_conf_bbscn_min_WORD word8
+#define lpfc_mbx_rd_conf_bbscn_max_SHIFT 4
+#define lpfc_mbx_rd_conf_bbscn_max_MASK 0x0000000F
+#define lpfc_mbx_rd_conf_bbscn_max_WORD word8
+#define lpfc_mbx_rd_conf_bbscn_def_SHIFT 8
+#define lpfc_mbx_rd_conf_bbscn_def_MASK 0x0000000F
+#define lpfc_mbx_rd_conf_bbscn_def_WORD word8
uint32_t word9;
#define lpfc_mbx_rd_conf_lmt_SHIFT 0
#define lpfc_mbx_rd_conf_lmt_MASK 0x0000FFFF
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 659662777b65..19af7e2c2ac0 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -7630,6 +7630,11 @@ lpfc_sli4_read_config(struct lpfc_hba *phba)
lpfc_printf_log(phba, KERN_WARNING, LOG_SLI,
"3082 Mailbox (x%x) returned ldv:x0\n",
bf_get(lpfc_mqe_command, &pmb->u.mqe));
+ if (bf_get(lpfc_mbx_rd_conf_bbscn_def, rd_config)) {
+ phba->bbcredit_support = 1;
+ phba->sli4_hba.bbscn_params.word0 = rd_config->word8;
+ }
+
phba->sli4_hba.extents_in_use =
bf_get(lpfc_mbx_rd_conf_extnts_inuse, rd_config);
phba->sli4_hba.max_cfg_param.max_xri =
diff --git a/drivers/scsi/lpfc/lpfc_mbox.c b/drivers/scsi/lpfc/lpfc_mbox.c
index ce25a18367b5..81fb92967b11 100644
--- a/drivers/scsi/lpfc/lpfc_mbox.c
+++ b/drivers/scsi/lpfc/lpfc_mbox.c
@@ -376,7 +376,12 @@ lpfc_config_link(struct lpfc_hba * phba, LPFC_MBOXQ_t * pmb)
mb->un.varCfgLnk.rttov = phba->fc_rttov;
mb->un.varCfgLnk.altov = phba->fc_altov;
mb->un.varCfgLnk.crtov = phba->fc_crtov;
- mb->un.varCfgLnk.citov = phba->fc_citov;
+ mb->un.varCfgLnk.cscn = 0;
+ if (phba->bbcredit_support && phba->cfg_enable_bbcr) {
+ mb->un.varCfgLnk.cscn = 1;
+ mb->un.varCfgLnk.bbscn = bf_get(lpfc_bbscn_def,
+ &phba->sli4_hba.bbscn_params);
+ }
if (phba->cfg_ack0 && (phba->sli_rev < LPFC_SLI_REV4))
mb->un.varCfgLnk.ack0_enable = 1;
@@ -2139,6 +2144,7 @@ lpfc_reg_vfi(struct lpfcMboxq *mbox, struct lpfc_vport *vport, dma_addr_t phys)
{
struct lpfc_mbx_reg_vfi *reg_vfi;
struct lpfc_hba *phba = vport->phba;
+ uint8_t bbscn_fabric = 0, bbscn_max = 0, bbscn_def = 0;
memset(mbox, 0, sizeof(*mbox));
reg_vfi = &mbox->u.mqe.un.reg_vfi;
@@ -2168,16 +2174,39 @@ lpfc_reg_vfi(struct lpfcMboxq *mbox, struct lpfc_vport *vport, dma_addr_t phys)
bf_set(lpfc_reg_vfi_vp, reg_vfi, 0);
bf_set(lpfc_reg_vfi_upd, reg_vfi, 1);
}
+
+ bf_set(lpfc_reg_vfi_bbcr, reg_vfi, 0);
+ bf_set(lpfc_reg_vfi_bbscn, reg_vfi, 0);
+ bbscn_fabric = (phba->fc_fabparam.cmn.bbRcvSizeMsb >> 4) & 0xF;
+
+ if (phba->bbcredit_support && phba->cfg_enable_bbcr &&
+ bbscn_fabric != 0) {
+ bbscn_max = bf_get(lpfc_bbscn_max,
+ &phba->sli4_hba.bbscn_params);
+ if (bbscn_fabric <= bbscn_max) {
+ bbscn_def = bf_get(lpfc_bbscn_def,
+ &phba->sli4_hba.bbscn_params);
+
+ if (bbscn_fabric > bbscn_def)
+ bf_set(lpfc_reg_vfi_bbscn, reg_vfi,
+ bbscn_fabric);
+ else
+ bf_set(lpfc_reg_vfi_bbscn, reg_vfi, bbscn_def);
+
+ bf_set(lpfc_reg_vfi_bbcr, reg_vfi, 1);
+ }
+ }
lpfc_printf_vlog(vport, KERN_INFO, LOG_MBOX,
"3134 Register VFI, mydid:x%x, fcfi:%d, "
" vfi:%d, vpi:%d, fc_pname:%x%x fc_flag:x%x"
- " port_state:x%x topology chg:%d\n",
+ " port_state:x%x topology chg:%d bbscn_fabric :%d\n",
vport->fc_myDID,
phba->fcf.fcfi,
phba->sli4_hba.vfi_ids[vport->vfi],
phba->vpi_ids[vport->vpi],
reg_vfi->wwn[0], reg_vfi->wwn[1], vport->fc_flag,
- vport->port_state, phba->fc_topology_changed);
+ vport->port_state, phba->fc_topology_changed,
+ bbscn_fabric);
}
/**
diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h
index 7dc8c73b5903..60200385fe00 100644
--- a/drivers/scsi/lpfc/lpfc_sli4.h
+++ b/drivers/scsi/lpfc/lpfc_sli4.h
@@ -420,6 +420,20 @@ struct lpfc_hba_eq_hdl {
#define LPFC_MULTI_CPU_AFFINITY 0xffffffff
};
+/*BB Credit recovery value*/
+struct lpfc_bbscn_params {
+ uint32_t word0;
+#define lpfc_bbscn_min_SHIFT 0
+#define lpfc_bbscn_min_MASK 0x0000000F
+#define lpfc_bbscn_min_WORD word0
+#define lpfc_bbscn_max_SHIFT 4
+#define lpfc_bbscn_max_MASK 0x0000000F
+#define lpfc_bbscn_max_WORD word0
+#define lpfc_bbscn_def_SHIFT 8
+#define lpfc_bbscn_def_MASK 0x0000000F
+#define lpfc_bbscn_def_WORD word0
+};
+
/* Port Capabilities for SLI4 Parameters */
struct lpfc_pc_sli4_params {
uint32_t supported;
@@ -551,6 +565,7 @@ struct lpfc_sli4_hba {
uint32_t ue_to_rp;
struct lpfc_register sli_intf;
struct lpfc_pc_sli4_params pc_sli4_params;
+ struct lpfc_bbscn_params bbscn_params;
struct lpfc_hba_eq_hdl *hba_eq_hdl; /* HBA per-WQ handle */
/* Pointers to the constructed SLI4 queues */
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 19/20] scsi: lpfc: fix "integer constant too large" error on 32bit archs
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (17 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 18/20] lpfc: Add Buffer to Buffer credit recovery support James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 9:09 ` Johannes Thumshirn
2017-08-05 0:47 ` [PATCH v2 20/20] lpfc: lpfc version bump 11.4.0.3 James Smart
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Maurizio Lombardi, James Smart
From: Maurizio Lombardi <mlombard@redhat.com>
cc1: warnings being treated as errors
drivers/scsi/lpfc/lpfc_init.c: In function 'lpfc_get_wwpn':
drivers/scsi/lpfc/lpfc_init.c:3253: error: integer constant is too large for 'long' type
Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
v2: use rol64()
---
drivers/scsi/lpfc/lpfc_init.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c
index 19af7e2c2ac0..8c512e494353 100644
--- a/drivers/scsi/lpfc/lpfc_init.c
+++ b/drivers/scsi/lpfc/lpfc_init.c
@@ -37,6 +37,7 @@
#include <linux/miscdevice.h>
#include <linux/percpu.h>
#include <linux/msi.h>
+#include <linux/bitops.h>
#include <scsi/scsi.h>
#include <scsi/scsi_device.h>
@@ -3733,9 +3734,7 @@ lpfc_get_wwpn(struct lpfc_hba *phba)
if (phba->sli_rev == LPFC_SLI_REV4)
return be64_to_cpu(wwn);
else
- return (((wwn & 0xffffffff00000000) >> 32) |
- ((wwn & 0x00000000ffffffff) << 32));
-
+ return rol64(wwn, 32);
}
/**
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread* [PATCH v2 20/20] lpfc: lpfc version bump 11.4.0.3
2017-08-05 0:47 [PATCH v2 00/20] lpfc updates for 11.4.0.3 James Smart
` (18 preceding siblings ...)
2017-08-05 0:47 ` [PATCH v2 19/20] scsi: lpfc: fix "integer constant too large" error on 32bit archs James Smart
@ 2017-08-05 0:47 ` James Smart
2017-08-07 9:08 ` Johannes Thumshirn
19 siblings, 1 reply; 42+ messages in thread
From: James Smart @ 2017-08-05 0:47 UTC (permalink / raw)
To: linux-scsi; +Cc: jsmart2021, Dick Kennedy, James Smart
From: Dick Kennedy <dick.kennedy@broadcom.com>
Update driver version to 11.4.0.3
Signed-off-by: Dick Kennedy <dick.kennedy@broadcom.com>
Signed-off-by: James Smart <james.smart@broadcom.com>
---
drivers/scsi/lpfc/lpfc_version.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/scsi/lpfc/lpfc_version.h b/drivers/scsi/lpfc/lpfc_version.h
index c6a24c3e2d5e..6aa192b3e4bf 100644
--- a/drivers/scsi/lpfc/lpfc_version.h
+++ b/drivers/scsi/lpfc/lpfc_version.h
@@ -20,7 +20,7 @@
* included with this package. *
*******************************************************************/
-#define LPFC_DRIVER_VERSION "11.4.0.1"
+#define LPFC_DRIVER_VERSION "11.4.0.3"
#define LPFC_DRIVER_NAME "lpfc"
/* Used for SLI 2/3 */
--
2.13.1
^ permalink raw reply related [flat|nested] 42+ messages in thread