* [PATCH 0/3] hisi_sas: some CQ processing fixes
@ 2017-01-03 12:24 John Garry
2017-01-03 12:24 ` [PATCH 1/3] scsi: hisi_sas: service v2 hw CQ ISR with tasklet John Garry
` (4 more replies)
0 siblings, 5 replies; 9+ messages in thread
From: John Garry @ 2017-01-03 12:24 UTC (permalink / raw)
To: jejb, martin.petersen
Cc: linuxarm, zhangfei.gao, xuwei5, john.garry2, linux-scsi,
linux-kernel, hanjun.guo, John Garry
This patchset fixes some issues related to servicing of the
completion queue interrupt.
The major fix is that sensitive hisi_hba structures need to be
locked when free'ing a slot.
Another modification is that the v2 hw completion queue irq is
now serviced with a tasklet, as too much work was being done in
the ISR.
John Garry (3):
scsi: hisi_sas: service v2 hw CQ ISR with tasklet
scsi: hisi_sas: lock sensitive regions when servicing CQ interrupt
scsi: hisi_sas: lock sensitive region in hisi_sas_slot_abort()
drivers/scsi/hisi_sas/hisi_sas.h | 1 +
drivers/scsi/hisi_sas/hisi_sas_main.c | 3 +++
drivers/scsi/hisi_sas/hisi_sas_v1_hw.c | 2 ++
drivers/scsi/hisi_sas/hisi_sas_v2_hw.c | 26 ++++++++++++++++++++------
4 files changed, 26 insertions(+), 6 deletions(-)
--
1.9.1
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH 1/3] scsi: hisi_sas: service v2 hw CQ ISR with tasklet
2017-01-03 12:24 [PATCH 0/3] hisi_sas: some CQ processing fixes John Garry
@ 2017-01-03 12:24 ` John Garry
2017-01-04 1:18 ` zhangfei
2017-01-03 12:24 ` [PATCH 2/3] scsi: hisi_sas: lock sensitive regions when servicing CQ interrupt John Garry
` (3 subsequent siblings)
4 siblings, 1 reply; 9+ messages in thread
From: John Garry @ 2017-01-03 12:24 UTC (permalink / raw)
To: jejb, martin.petersen
Cc: linuxarm, zhangfei.gao, xuwei5, john.garry2, linux-scsi,
linux-kernel, hanjun.guo, John Garry
Currently the all the slot processing for the completion
queue is done in ISR context. It is judged that the slot
processing can take a long time, especially when a SATA
NCQ completes (upto 32 slots).
So, as a solution, defer the bulk of the ISR processing
to tasklet context. Each CQ will have its down tasklet.
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Xiang Chen <chenxiang66@hisilicon.com>
---
drivers/scsi/hisi_sas/hisi_sas.h | 1 +
drivers/scsi/hisi_sas/hisi_sas_v2_hw.c | 24 ++++++++++++++++++------
2 files changed, 19 insertions(+), 6 deletions(-)
diff --git a/drivers/scsi/hisi_sas/hisi_sas.h b/drivers/scsi/hisi_sas/hisi_sas.h
index c0cd505..9216dea 100644
--- a/drivers/scsi/hisi_sas/hisi_sas.h
+++ b/drivers/scsi/hisi_sas/hisi_sas.h
@@ -95,6 +95,7 @@ struct hisi_sas_port {
struct hisi_sas_cq {
struct hisi_hba *hisi_hba;
+ struct tasklet_struct tasklet;
int rd_point;
int id;
};
diff --git a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c
index b934aec..e506260 100644
--- a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c
+++ b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c
@@ -2481,20 +2481,17 @@ static irqreturn_t fatal_axi_int_v2_hw(int irq_no, void *p)
return IRQ_HANDLED;
}
-static irqreturn_t cq_interrupt_v2_hw(int irq_no, void *p)
+static void cq_tasklet_v2_hw(unsigned long val)
{
- struct hisi_sas_cq *cq = p;
+ struct hisi_sas_cq *cq = (struct hisi_sas_cq *)val;
struct hisi_hba *hisi_hba = cq->hisi_hba;
struct hisi_sas_slot *slot;
struct hisi_sas_itct *itct;
struct hisi_sas_complete_v2_hdr *complete_queue;
- u32 irq_value, rd_point = cq->rd_point, wr_point, dev_id;
+ u32 rd_point = cq->rd_point, wr_point, dev_id;
int queue = cq->id;
complete_queue = hisi_hba->complete_hdr[queue];
- irq_value = hisi_sas_read32(hisi_hba, OQ_INT_SRC);
-
- hisi_sas_write32(hisi_hba, OQ_INT_SRC, 1 << queue);
wr_point = hisi_sas_read32(hisi_hba, COMPL_Q_0_WR_PTR +
(0x14 * queue));
@@ -2545,6 +2542,18 @@ static irqreturn_t cq_interrupt_v2_hw(int irq_no, void *p)
/* update rd_point */
cq->rd_point = rd_point;
hisi_sas_write32(hisi_hba, COMPL_Q_0_RD_PTR + (0x14 * queue), rd_point);
+}
+
+static irqreturn_t cq_interrupt_v2_hw(int irq_no, void *p)
+{
+ struct hisi_sas_cq *cq = p;
+ struct hisi_hba *hisi_hba = cq->hisi_hba;
+ int queue = cq->id;
+
+ hisi_sas_write32(hisi_hba, OQ_INT_SRC, 1 << queue);
+
+ tasklet_schedule(&cq->tasklet);
+
return IRQ_HANDLED;
}
@@ -2726,6 +2735,8 @@ static int interrupt_init_v2_hw(struct hisi_hba *hisi_hba)
for (i = 0; i < hisi_hba->queue_count; i++) {
int idx = i + 96; /* First cq interrupt is irq96 */
+ struct hisi_sas_cq *cq = &hisi_hba->cq[i];
+ struct tasklet_struct *t = &cq->tasklet;
irq = irq_map[idx];
if (!irq) {
@@ -2742,6 +2753,7 @@ static int interrupt_init_v2_hw(struct hisi_hba *hisi_hba)
irq, rc);
return -ENOENT;
}
+ tasklet_init(t, cq_tasklet_v2_hw, (unsigned long)cq);
}
return 0;
--
1.9.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 2/3] scsi: hisi_sas: lock sensitive regions when servicing CQ interrupt
2017-01-03 12:24 [PATCH 0/3] hisi_sas: some CQ processing fixes John Garry
2017-01-03 12:24 ` [PATCH 1/3] scsi: hisi_sas: service v2 hw CQ ISR with tasklet John Garry
@ 2017-01-03 12:24 ` John Garry
2017-01-04 1:19 ` zhangfei
2017-01-03 12:24 ` [PATCH 3/3] scsi: hisi_sas: lock sensitive region in hisi_sas_slot_abort() John Garry
` (2 subsequent siblings)
4 siblings, 1 reply; 9+ messages in thread
From: John Garry @ 2017-01-03 12:24 UTC (permalink / raw)
To: jejb, martin.petersen
Cc: linuxarm, zhangfei.gao, xuwei5, john.garry2, linux-scsi,
linux-kernel, hanjun.guo, John Garry
There is a bug in the current driver in that certain hisi_hba
and port structure elements which we access when servicing
the CQ interrupt do not use thread-safe accesses; these include
hisi_sas_port linked-list of active slots (hisi_sas_port.entry),
bitmap of currently allocated IPTT (in hisi_hba.slot_index_tags),
and completion queue read pointer.
As a solution, lock these elements with the hisi_hba.lock.
Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Xiang Chen <chenxiang66@hisilicon.com>
---
drivers/scsi/hisi_sas/hisi_sas_v1_hw.c | 2 ++
drivers/scsi/hisi_sas/hisi_sas_v2_hw.c | 2 ++
2 files changed, 4 insertions(+)
diff --git a/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c
index 8a1be0b..854fbea 100644
--- a/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c
+++ b/drivers/scsi/hisi_sas/hisi_sas_v1_hw.c
@@ -1596,6 +1596,7 @@ static irqreturn_t cq_interrupt_v1_hw(int irq, void *p)
hisi_hba->complete_hdr[queue];
u32 irq_value, rd_point = cq->rd_point, wr_point;
+ spin_lock(&hisi_hba->lock);
irq_value = hisi_sas_read32(hisi_hba, OQ_INT_SRC);
hisi_sas_write32(hisi_hba, OQ_INT_SRC, 1 << queue);
@@ -1628,6 +1629,7 @@ static irqreturn_t cq_interrupt_v1_hw(int irq, void *p)
/* update rd_point */
cq->rd_point = rd_point;
hisi_sas_write32(hisi_hba, COMPL_Q_0_RD_PTR + (0x14 * queue), rd_point);
+ spin_unlock(&hisi_hba->lock);
return IRQ_HANDLED;
}
diff --git a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c
index e506260..69b0f06 100644
--- a/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c
+++ b/drivers/scsi/hisi_sas/hisi_sas_v2_hw.c
@@ -2493,6 +2493,7 @@ static void cq_tasklet_v2_hw(unsigned long val)
complete_queue = hisi_hba->complete_hdr[queue];
+ spin_lock(&hisi_hba->lock);
wr_point = hisi_sas_read32(hisi_hba, COMPL_Q_0_WR_PTR +
(0x14 * queue));
@@ -2542,6 +2543,7 @@ static void cq_tasklet_v2_hw(unsigned long val)
/* update rd_point */
cq->rd_point = rd_point;
hisi_sas_write32(hisi_hba, COMPL_Q_0_RD_PTR + (0x14 * queue), rd_point);
+ spin_unlock(&hisi_hba->lock);
}
static irqreturn_t cq_interrupt_v2_hw(int irq_no, void *p)
--
1.9.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH 3/3] scsi: hisi_sas: lock sensitive region in hisi_sas_slot_abort()
2017-01-03 12:24 [PATCH 0/3] hisi_sas: some CQ processing fixes John Garry
2017-01-03 12:24 ` [PATCH 1/3] scsi: hisi_sas: service v2 hw CQ ISR with tasklet John Garry
2017-01-03 12:24 ` [PATCH 2/3] scsi: hisi_sas: lock sensitive regions when servicing CQ interrupt John Garry
@ 2017-01-03 12:24 ` John Garry
2017-01-04 1:20 ` zhangfei
2017-01-04 2:14 ` [PATCH 0/3] hisi_sas: some CQ processing fixes Hanjun Guo
2017-01-05 23:22 ` Martin K. Petersen
4 siblings, 1 reply; 9+ messages in thread
From: John Garry @ 2017-01-03 12:24 UTC (permalink / raw)
To: jejb, martin.petersen
Cc: linuxarm, zhangfei.gao, xuwei5, john.garry2, linux-scsi,
linux-kernel, hanjun.guo, John Garry
When we call hisi_sas_slot_task_free() we should grab the
hisi_hba.lock, as hisi_sas_slot_task_free() accesses common
hisi_hba elements.
Function hisi_sas_slot_abort() is missing this, so add it.
Signed-off-by: John Garry <john.garry@huawei.com>
---
drivers/scsi/hisi_sas/hisi_sas_main.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/scsi/hisi_sas/hisi_sas_main.c b/drivers/scsi/hisi_sas/hisi_sas_main.c
index d50e9cf..22dba01 100644
--- a/drivers/scsi/hisi_sas/hisi_sas_main.c
+++ b/drivers/scsi/hisi_sas/hisi_sas_main.c
@@ -146,6 +146,7 @@ static void hisi_sas_slot_abort(struct work_struct *work)
struct scsi_lun lun;
struct device *dev = &hisi_hba->pdev->dev;
int tag = abort_slot->idx;
+ unsigned long flags;
if (!(task->task_proto & SAS_PROTOCOL_SSP)) {
dev_err(dev, "cannot abort slot for non-ssp task\n");
@@ -159,7 +160,9 @@ static void hisi_sas_slot_abort(struct work_struct *work)
hisi_sas_debug_issue_ssp_tmf(task->dev, lun.scsi_lun, &tmf_task);
out:
/* Do cleanup for this task */
+ spin_lock_irqsave(&hisi_hba->lock, flags);
hisi_sas_slot_task_free(hisi_hba, task, abort_slot);
+ spin_unlock_irqrestore(&hisi_hba->lock, flags);
if (task->task_done)
task->task_done(task);
if (sas_dev)
--
1.9.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH 1/3] scsi: hisi_sas: service v2 hw CQ ISR with tasklet
2017-01-03 12:24 ` [PATCH 1/3] scsi: hisi_sas: service v2 hw CQ ISR with tasklet John Garry
@ 2017-01-04 1:18 ` zhangfei
0 siblings, 0 replies; 9+ messages in thread
From: zhangfei @ 2017-01-04 1:18 UTC (permalink / raw)
To: John Garry, jejb, martin.petersen
Cc: linuxarm, xuwei5, john.garry2, linux-scsi, linux-kernel, hanjun.guo
On 2017年01月03日 20:24, John Garry wrote:
> Currently the all the slot processing for the completion
> queue is done in ISR context. It is judged that the slot
> processing can take a long time, especially when a SATA
> NCQ completes (upto 32 slots).
>
> So, as a solution, defer the bulk of the ISR processing
> to tasklet context. Each CQ will have its down tasklet.
>
> Signed-off-by: John Garry <john.garry@huawei.com>
> Reviewed-by: Xiang Chen <chenxiang66@hisilicon.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 2/3] scsi: hisi_sas: lock sensitive regions when servicing CQ interrupt
2017-01-03 12:24 ` [PATCH 2/3] scsi: hisi_sas: lock sensitive regions when servicing CQ interrupt John Garry
@ 2017-01-04 1:19 ` zhangfei
0 siblings, 0 replies; 9+ messages in thread
From: zhangfei @ 2017-01-04 1:19 UTC (permalink / raw)
To: John Garry, jejb, martin.petersen
Cc: linuxarm, xuwei5, john.garry2, linux-scsi, linux-kernel, hanjun.guo
On 2017年01月03日 20:24, John Garry wrote:
> There is a bug in the current driver in that certain hisi_hba
> and port structure elements which we access when servicing
> the CQ interrupt do not use thread-safe accesses; these include
> hisi_sas_port linked-list of active slots (hisi_sas_port.entry),
> bitmap of currently allocated IPTT (in hisi_hba.slot_index_tags),
> and completion queue read pointer.
>
> As a solution, lock these elements with the hisi_hba.lock.
>
> Signed-off-by: John Garry <john.garry@huawei.com>
> Reviewed-by: Xiang Chen <chenxiang66@hisilicon.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 3/3] scsi: hisi_sas: lock sensitive region in hisi_sas_slot_abort()
2017-01-03 12:24 ` [PATCH 3/3] scsi: hisi_sas: lock sensitive region in hisi_sas_slot_abort() John Garry
@ 2017-01-04 1:20 ` zhangfei
0 siblings, 0 replies; 9+ messages in thread
From: zhangfei @ 2017-01-04 1:20 UTC (permalink / raw)
To: John Garry, jejb, martin.petersen
Cc: linuxarm, xuwei5, john.garry2, linux-scsi, linux-kernel, hanjun.guo
On 2017年01月03日 20:24, John Garry wrote:
> When we call hisi_sas_slot_task_free() we should grab the
> hisi_hba.lock, as hisi_sas_slot_task_free() accesses common
> hisi_hba elements.
> Function hisi_sas_slot_abort() is missing this, so add it.
>
> Signed-off-by: John Garry <john.garry@huawei.com>
Reviewed-by: Zhangfei Gao <zhangfei.gao@linaro.org>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 0/3] hisi_sas: some CQ processing fixes
2017-01-03 12:24 [PATCH 0/3] hisi_sas: some CQ processing fixes John Garry
` (2 preceding siblings ...)
2017-01-03 12:24 ` [PATCH 3/3] scsi: hisi_sas: lock sensitive region in hisi_sas_slot_abort() John Garry
@ 2017-01-04 2:14 ` Hanjun Guo
2017-01-05 23:22 ` Martin K. Petersen
4 siblings, 0 replies; 9+ messages in thread
From: Hanjun Guo @ 2017-01-04 2:14 UTC (permalink / raw)
To: John Garry, jejb, martin.petersen
Cc: linuxarm, zhangfei.gao, xuwei5, john.garry2, linux-scsi, linux-kernel
On 2017/1/3 20:24, John Garry wrote:
> This patchset fixes some issues related to servicing of the
> completion queue interrupt.
> The major fix is that sensitive hisi_hba structures need to be
> locked when free'ing a slot.
> Another modification is that the v2 hw completion queue irq is
> now serviced with a tasklet, as too much work was being done in
> the ISR.
Tested on v2 based sas hardware and the crashes are gone,
Tested-by: Hanjun Guo <hanjun.guo@linaro.org>
Thanks
Hanjun
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH 0/3] hisi_sas: some CQ processing fixes
2017-01-03 12:24 [PATCH 0/3] hisi_sas: some CQ processing fixes John Garry
` (3 preceding siblings ...)
2017-01-04 2:14 ` [PATCH 0/3] hisi_sas: some CQ processing fixes Hanjun Guo
@ 2017-01-05 23:22 ` Martin K. Petersen
4 siblings, 0 replies; 9+ messages in thread
From: Martin K. Petersen @ 2017-01-05 23:22 UTC (permalink / raw)
To: John Garry
Cc: jejb, martin.petersen, linuxarm, zhangfei.gao, xuwei5,
john.garry2, linux-scsi, linux-kernel, hanjun.guo
>>>>> "John" == John Garry <john.garry@huawei.com> writes:
John> This patchset fixes some issues related to servicing of the
John> completion queue interrupt. The major fix is that sensitive
John> hisi_hba structures need to be locked when free'ing a slot.
John> Another modification is that the v2 hw completion queue irq is now
John> serviced with a tasklet, as too much work was being done in the
John> ISR.
Applied to 4.11/scsi-queue.
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2017-01-05 23:22 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-01-03 12:24 [PATCH 0/3] hisi_sas: some CQ processing fixes John Garry
2017-01-03 12:24 ` [PATCH 1/3] scsi: hisi_sas: service v2 hw CQ ISR with tasklet John Garry
2017-01-04 1:18 ` zhangfei
2017-01-03 12:24 ` [PATCH 2/3] scsi: hisi_sas: lock sensitive regions when servicing CQ interrupt John Garry
2017-01-04 1:19 ` zhangfei
2017-01-03 12:24 ` [PATCH 3/3] scsi: hisi_sas: lock sensitive region in hisi_sas_slot_abort() John Garry
2017-01-04 1:20 ` zhangfei
2017-01-04 2:14 ` [PATCH 0/3] hisi_sas: some CQ processing fixes Hanjun Guo
2017-01-05 23:22 ` Martin K. Petersen
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).