* [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver
@ 2022-06-25 7:44 Jie Hai
2022-06-25 7:44 ` [PATCH 1/8] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
` (11 more replies)
0 siblings, 12 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-25 7:44 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they share the
same pci device id but different pci revision and register layouts.
The original version supports HiSilicon IP08 but not HiSilicon IP09.
This series support DMA driver for HIP08 and HIP09:
1. Fix bugs for HIP08 DMA driver
- Disable hardware channels when driver detached
- Update cq_head whenever accessed it
- Support multi-thread for one DMA channel
2. Use macros instead of magic number
3. Add support for HIP09 DMA driver
4. Add debugfs for HIP08 and HIP09 DMA driver
5. Add myself as maintainer of hisi_dma.c
Jie Hai (8):
dmaengine: hisilicon: Disable channels when unregister hisi_dma
dmaengine: hisilicon: Fix CQ head update
dmaengine: hisilicon: Add multi-thread support for a DMA channel
dmaengine: hisilicon: Use macros instead of magic number
dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
dmaengine: hisilicon: Add dfx feature for hisi dma driver
Documentation: Add debugfs doc for hisi_dma
MAINTAINERS: Add debugfs files and maintainer for hisi_dma
Documentation/ABI/testing/debugfs-hisi-dma | 9 +
MAINTAINERS | 2 +
drivers/dma/hisi_dma.c | 733 ++++++++++++++++++---
3 files changed, 648 insertions(+), 96 deletions(-)
create mode 100644 Documentation/ABI/testing/debugfs-hisi-dma
--
2.33.0
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH 1/8] dmaengine: hisilicon: Disable channels when unregister hisi_dma
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
@ 2022-06-25 7:44 ` Jie Hai
2022-06-25 7:44 ` [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update Jie Hai
` (10 subsequent siblings)
11 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-25 7:44 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
When hisi_dma is unloaded or unbinded, all of channels should be disabled.
This patch disables DMA channels when driver is unloaded or unbinded.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 43817ced3a3e..98bc488893cc 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -180,7 +180,8 @@ static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
}
-static void hisi_dma_reset_hw_chan(struct hisi_dma_chan *chan)
+static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
+ bool disable)
{
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
u32 index = chan->qp_num, tmp;
@@ -201,8 +202,11 @@ static void hisi_dma_reset_hw_chan(struct hisi_dma_chan *chan)
hisi_dma_do_reset(hdma_dev, index);
hisi_dma_reset_qp_point(hdma_dev, index);
hisi_dma_pause_dma(hdma_dev, index, false);
- hisi_dma_enable_dma(hdma_dev, index, true);
- hisi_dma_unmask_irq(hdma_dev, index);
+
+ if (!disable) {
+ hisi_dma_enable_dma(hdma_dev, index, true);
+ hisi_dma_unmask_irq(hdma_dev, index);
+ }
ret = readl_relaxed_poll_timeout(hdma_dev->base +
HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
@@ -218,7 +222,7 @@ static void hisi_dma_free_chan_resources(struct dma_chan *c)
struct hisi_dma_chan *chan = to_hisi_dma_chan(c);
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
- hisi_dma_reset_hw_chan(chan);
+ hisi_dma_reset_or_disable_hw_chan(chan, false);
vchan_free_chan_resources(&chan->vc);
memset(chan->sq, 0, sizeof(struct hisi_dma_sqe) * hdma_dev->chan_depth);
@@ -394,7 +398,7 @@ static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
static void hisi_dma_disable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- hisi_dma_reset_hw_chan(&hdma_dev->chan[qp_index]);
+ hisi_dma_reset_or_disable_hw_chan(&hdma_dev->chan[qp_index], true);
}
static void hisi_dma_enable_qps(struct hisi_dma_dev *hdma_dev)
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-06-25 7:44 ` [PATCH 1/8] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
@ 2022-06-25 7:44 ` Jie Hai
2022-06-26 13:37 ` kernel test robot
2022-06-27 6:12 ` Vinod Koul
2022-06-25 7:44 ` [PATCH 3/8] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
` (9 subsequent siblings)
11 siblings, 2 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-25 7:44 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
After completion of data transfer of one or multiple descriptors,
the completion status and the current head pointer to submission
queue are written into the CQ and interrupt can be generated to
inform the software. In interrupt process CQ is read and cq_head
is updated.
hisi_dma_irq updates cq_head only when the completion status is
success. When an abnormal interrupt reports, cq_head will not update
which will cause subsequent interrupt processes read the error CQ
and never report the correct status.
This patch updates cq_head whenever CQ is accessed.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 98bc488893cc..0a0f8a4d168a 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -436,12 +436,11 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
desc = chan->desc;
cqe = chan->cq + chan->cq_head;
if (desc) {
+ chan->cq_head = (chan->cq_head + 1) %
+ hdma_dev->chan_depth;
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
+ chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
- chan->cq_head = (chan->cq_head + 1) %
- hdma_dev->chan_depth;
- hisi_dma_chan_write(hdma_dev->base,
- HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
- chan->cq_head);
vchan_cookie_complete(&desc->vd);
} else {
dev_err(&hdma_dev->pdev->dev, "task error!\n");
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 3/8] dmaengine: hisilicon: Add multi-thread support for a DMA channel
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-06-25 7:44 ` [PATCH 1/8] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
2022-06-25 7:44 ` [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update Jie Hai
@ 2022-06-25 7:44 ` Jie Hai
2022-06-27 6:21 ` Vinod Koul
2022-06-25 7:44 ` [PATCH 4/8] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
` (8 subsequent siblings)
11 siblings, 1 reply; 64+ messages in thread
From: Jie Hai @ 2022-06-25 7:44 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
When we get a DMA channel and try to use it in multiple threads it
will cause oops and hanging the system.
% echo 100 > /sys/module/dmatest/parameters/threads_per_chan
% echo 100 > /sys/module/dmatest/parameters/iterations
% echo 1 > /sys/module/dmatest/parameters/run
[383493.327077] Unable to handle kernel paging request at virtual
address dead000000000108
[383493.335103] Mem abort info:
[383493.335103] ESR = 0x96000044
[383493.335105] EC = 0x25: DABT (current EL), IL = 32 bits
[383493.335107] SET = 0, FnV = 0
[383493.335108] EA = 0, S1PTW = 0
[383493.335109] FSC = 0x04: level 0 translation fault
[383493.335110] Data abort info:
[383493.335111] ISV = 0, ISS = 0x00000044
[383493.364739] CM = 0, WnR = 1
[383493.367793] [dead000000000108] address between user and kernel
address ranges
[383493.375021] Internal error: Oops: 96000044 [#1] PREEMPT SMP
[383493.437574] CPU: 63 PID: 27895 Comm: dma0chan0-copy2 Kdump:
loaded Tainted: GO 5.17.0-rc4+ #2
[383493.457851] pstate: 204000c9 (nzCv daIF +PAN -UAO -TCO -DIT
-SSBS BTYPE=--)
[383493.465331] pc : vchan_tx_submit+0x64/0xa0
[383493.469957] lr : vchan_tx_submit+0x34/0xa0
This happens because of data race. Each thread rewrite channels's
descriptor as soon as device_issue_pending is called. It leads to
the situation that the driver thinks that it uses the right
descriptor in interrupt handler while channels's descriptor has
been changed by other thread.
With current fixes channels's descriptor changes it's value only
when it has been used. A new descriptor is acquired from
vc->desc_issued queue that is already filled with descriptors
that are ready to be sent. Threads have no direct access to DMA
channel descriptor. Now it is just possible to queue a descriptor
for further processing.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 0a0f8a4d168a..0385419be8d5 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -271,7 +271,6 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
vd = vchan_next_desc(&chan->vc);
if (!vd) {
- dev_err(&hdma_dev->pdev->dev, "no issued task!\n");
chan->desc = NULL;
return;
}
@@ -303,7 +302,7 @@ static void hisi_dma_issue_pending(struct dma_chan *c)
spin_lock_irqsave(&chan->vc.lock, flags);
- if (vchan_issue_pending(&chan->vc))
+ if (vchan_issue_pending(&chan->vc) && !chan->desc)
hisi_dma_start_transfer(chan);
spin_unlock_irqrestore(&chan->vc.lock, flags);
@@ -442,11 +441,10 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
vchan_cookie_complete(&desc->vd);
+ hisi_dma_start_transfer(chan);
} else {
dev_err(&hdma_dev->pdev->dev, "task error!\n");
}
-
- chan->desc = NULL;
}
spin_unlock(&chan->vc.lock);
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 4/8] dmaengine: hisilicon: Use macros instead of magic number
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (2 preceding siblings ...)
2022-06-25 7:44 ` [PATCH 3/8] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
@ 2022-06-25 7:44 ` Jie Hai
2022-06-25 7:44 ` [PATCH 5/8] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
` (7 subsequent siblings)
11 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-25 7:44 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
readl_relaxed_poll_timeout() uses magic numbers 10 and 1000, which
indicate maximum time to sleep between reads in us and timeout in us,
respectively.
Use macros HISI_DMA_POLL_Q_STS_DELAY_US and
HISI_DMA_POLL_Q_STS_TIME_OUT_US instead of these two numbers.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 0385419be8d5..d69a73272467 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -36,6 +36,9 @@
#define PCI_BAR_2 2
+#define HISI_DMA_POLL_Q_STS_DELAY_US 10
+#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+
enum hisi_dma_mode {
EP = 0,
RC,
@@ -185,15 +188,20 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
{
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
u32 index = chan->qp_num, tmp;
+ void __iomem *addr;
int ret;
hisi_dma_pause_dma(hdma_dev, index, true);
hisi_dma_enable_dma(hdma_dev, index, false);
hisi_dma_mask_irq(hdma_dev, index);
- ret = readl_relaxed_poll_timeout(hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN, 10, 1000);
+ addr = hdma_dev->base +
+ HISI_DMA_Q_FSM_STS + index * HISI_DMA_Q_OFFSET;
+
+ ret = readl_relaxed_poll_timeout(addr, tmp,
+ FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) != RUN,
+ HISI_DMA_POLL_Q_STS_DELAY_US,
+ HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "disable channel timeout!\n");
WARN_ON(1);
@@ -208,9 +216,10 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
hisi_dma_unmask_irq(hdma_dev, index);
}
- ret = readl_relaxed_poll_timeout(hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE, 10, 1000);
+ ret = readl_relaxed_poll_timeout(addr, tmp,
+ FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) == IDLE,
+ HISI_DMA_POLL_Q_STS_DELAY_US,
+ HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "reset channel timeout!\n");
WARN_ON(1);
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 5/8] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (3 preceding siblings ...)
2022-06-25 7:44 ` [PATCH 4/8] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
@ 2022-06-25 7:44 ` Jie Hai
2022-06-25 7:44 ` [PATCH 6/8] dmaengine: hisilicon: Add dfx feature for hisi dma driver Jie Hai
` (6 subsequent siblings)
11 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-25 7:44 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they
have the same pci device id but different pci revision.
Unfortunately, they have different register layouts, so
the origin driver cannot run on HiSilicon IP09 correctly.
This patch enables the driver to adapt to HiSilicon IP09.
HiSilicon IP09 offers 4 channels, each channel has a send
queue, a complete queue and an interrupt to help to do tasks.
This DMA engine can do memory copy between memory blocks.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 341 +++++++++++++++++++++++++++++++----------
1 file changed, 263 insertions(+), 78 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index d69a73272467..e1a5390567bd 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
-/* Copyright(c) 2019 HiSilicon Limited. */
+/* Copyright(c) 2019-2022 HiSilicon Limited. */
+
#include <linux/bitfield.h>
#include <linux/dmaengine.h>
#include <linux/init.h>
@@ -9,35 +10,87 @@
#include <linux/spinlock.h>
#include "virt-dma.h"
-#define HISI_DMA_SQ_BASE_L 0x0
-#define HISI_DMA_SQ_BASE_H 0x4
-#define HISI_DMA_SQ_DEPTH 0x8
-#define HISI_DMA_SQ_TAIL_PTR 0xc
-#define HISI_DMA_CQ_BASE_L 0x10
-#define HISI_DMA_CQ_BASE_H 0x14
-#define HISI_DMA_CQ_DEPTH 0x18
-#define HISI_DMA_CQ_HEAD_PTR 0x1c
-#define HISI_DMA_CTRL0 0x20
-#define HISI_DMA_CTRL0_QUEUE_EN_S 0
-#define HISI_DMA_CTRL0_QUEUE_PAUSE_S 4
-#define HISI_DMA_CTRL1 0x24
-#define HISI_DMA_CTRL1_QUEUE_RESET_S 0
-#define HISI_DMA_Q_FSM_STS 0x30
-#define HISI_DMA_FSM_STS_MASK GENMASK(3, 0)
-#define HISI_DMA_INT_STS 0x40
-#define HISI_DMA_INT_STS_MASK GENMASK(12, 0)
-#define HISI_DMA_INT_MSK 0x44
-#define HISI_DMA_MODE 0x217c
-#define HISI_DMA_OFFSET 0x100
-
-#define HISI_DMA_MSI_NUM 32
-#define HISI_DMA_CHAN_NUM 30
-#define HISI_DMA_Q_DEPTH_VAL 1024
-
-#define PCI_BAR_2 2
-
-#define HISI_DMA_POLL_Q_STS_DELAY_US 10
-#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+/* HiSilicon DMA register common field define */
+#define HISI_DMA_Q_SQ_BASE_L 0x0
+#define HISI_DMA_Q_SQ_BASE_H 0x4
+#define HISI_DMA_Q_SQ_DEPTH 0x8
+#define HISI_DMA_Q_SQ_TAIL_PTR 0xc
+#define HISI_DMA_Q_CQ_BASE_L 0x10
+#define HISI_DMA_Q_CQ_BASE_H 0x14
+#define HISI_DMA_Q_CQ_DEPTH 0x18
+#define HISI_DMA_Q_CQ_HEAD_PTR 0x1c
+#define HISI_DMA_Q_CTRL0 0x20
+#define HISI_DMA_Q_CTRL0_QUEUE_EN 0
+#define HISI_DMA_Q_CTRL0_QUEUE_PAUSE 4
+#define HISI_DMA_Q_CTRL1 0x24
+#define HISI_DMA_Q_CTRL1_QUEUE_RESET 0
+#define HISI_DMA_Q_FSM_STS 0x30
+#define HISI_DMA_Q_FSM_STS_MASK GENMASK(3, 0)
+#define HISI_DMA_Q_SQ_STS 0x34
+#define HISI_DMA_Q_SQ_HEAD_PTR GENMASK(15, 0)
+#define HISI_DMA_Q_CQ_TAIL_PTR 0x3c
+#define HISI_DMA_Q_ERR_INT_NUM0 0x84
+#define HISI_DMA_Q_ERR_INT_NUM1 0x88
+#define HISI_DMA_Q_ERR_INT_NUM2 0x8c
+
+/* HiSilicon IP08 DMA register and field define */
+#define HISI_DMA_HIP08_MODE 0x217C
+#define HISI_DMA_HIP08_Q_BASE 0x0
+#define HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN 2
+#define HISI_DMA_HIP08_Q_INT_STS 0x40
+#define HISI_DMA_HIP08_Q_INT_MSK 0x44
+#define HISI_DMA_HIP08_Q_INT_STS_MASK GENMASK(14, 0)
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM3 0x90
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM4 0x94
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM5 0x98
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM6 0x48
+#define HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT 24
+
+/* HiSilicon IP09 DMA register and field define */
+#define HISI_DMA_HIP09_DMA_FLR_DISABLE 0xA00
+#define HISI_DMA_HIP09_DMA_FLR_DISABLE_B 0
+#define HISI_DMA_HIP09_Q_BASE 0x2000
+#define HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN GENMASK(31, 28)
+#define HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT 26
+#define HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT 27
+#define HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE 2
+#define HISI_DMA_HIP09_Q_INT_STS 0x40
+#define HISI_DMA_HIP09_Q_INT_MSK 0x44
+#define HISI_DMA_HIP09_Q_INT_STS_MASK 0x1
+#define HISI_DMA_HIP09_Q_ERR_INT_STS 0x48
+#define HISI_DMA_HIP09_Q_ERR_INT_MSK 0x4C
+#define HISI_DMA_HIP09_Q_ERR_INT_STS_MASK GENMASK(18, 1)
+#define HISI_DMA_HIP09_PORT_CFG_REG(port_id) (0x800 + \
+ (port_id) * 0x20)
+#define HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B 16
+
+#define HISI_DMA_HIP09_MAX_PORT_NUM 16
+
+#define HISI_DMA_HIP08_MSI_NUM 32
+#define HISI_DMA_HIP08_CHAN_NUM 30
+#define HISI_DMA_HIP09_MSI_NUM 4
+#define HISI_DMA_HIP09_CHAN_NUM 4
+#define HISI_DMA_REVISION_HIP08B 0x21
+#define HISI_DMA_REVISION_HIP09A 0x30
+
+#define HISI_DMA_Q_OFFSET 0x100
+#define HISI_DMA_Q_DEPTH_VAL 1024
+
+#define PCI_BAR_2 2
+
+#define HISI_DMA_POLL_Q_STS_DELAY_US 10
+#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+/**
+ * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
+ * have the same pci device id but different pci revision.
+ * Unfortunately, they have different register layouts, so two layout
+ * enumerations are defined.
+ */
+enum hisi_dma_reg_layout {
+ HISI_DMA_REG_LAYOUT_INVALID = 0,
+ HISI_DMA_REG_LAYOUT_HIP08,
+ HISI_DMA_REG_LAYOUT_HIP09
+};
enum hisi_dma_mode {
EP = 0,
@@ -108,6 +161,8 @@ struct hisi_dma_dev {
struct dma_device dma_dev;
u32 chan_num;
u32 chan_depth;
+ enum hisi_dma_reg_layout reg_layout;
+ void __iomem *queue_base; /* queue region start of register */
struct hisi_dma_chan chan[];
};
@@ -124,7 +179,7 @@ static inline struct hisi_dma_desc *to_hisi_dma_desc(struct virt_dma_desc *vd)
static inline void hisi_dma_chan_write(void __iomem *base, u32 reg, u32 index,
u32 val)
{
- writel_relaxed(val, base + reg + index * HISI_DMA_OFFSET);
+ writel_relaxed(val, base + reg + index * HISI_DMA_Q_OFFSET);
}
static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
@@ -139,48 +194,76 @@ static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
static void hisi_dma_pause_dma(struct hisi_dma_dev *hdma_dev, u32 index,
bool pause)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_PAUSE_S, pause);
+ addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_PAUSE, pause);
}
static void hisi_dma_enable_dma(struct hisi_dma_dev *hdma_dev, u32 index,
bool enable)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_EN_S, enable);
+ addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_EN, enable);
}
static void hisi_dma_mask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_INT_MSK, qp_index,
- HISI_DMA_INT_STS_MASK);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
+ qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
+ else {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
+ qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
+ qp_index,
+ HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
+ }
}
static void hisi_dma_unmask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- void __iomem *base = hdma_dev->base;
-
- hisi_dma_chan_write(base, HISI_DMA_INT_STS, qp_index,
- HISI_DMA_INT_STS_MASK);
- hisi_dma_chan_write(base, HISI_DMA_INT_MSK, qp_index, 0);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_STS,
+ qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
+ qp_index, 0);
+ } else {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_STS,
+ qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_STS,
+ qp_index,
+ HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
+ qp_index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
+ qp_index, 0);
+ }
}
static void hisi_dma_do_reset(struct hisi_dma_dev *hdma_dev, u32 index)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL1 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL1_QUEUE_RESET_S, 1);
+ addr = hdma_dev->queue_base +
+ HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL1_QUEUE_RESET, 1);
}
static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
{
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, index, 0);
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
}
static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
@@ -195,7 +278,7 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
hisi_dma_enable_dma(hdma_dev, index, false);
hisi_dma_mask_irq(hdma_dev, index);
- addr = hdma_dev->base +
+ addr = hdma_dev->queue_base +
HISI_DMA_Q_FSM_STS + index * HISI_DMA_Q_OFFSET;
ret = readl_relaxed_poll_timeout(addr, tmp,
@@ -300,8 +383,8 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
chan->sq_tail = (chan->sq_tail + 1) % hdma_dev->chan_depth;
/* update sq_tail to trigger a new task */
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, chan->qp_num,
- chan->sq_tail);
+ hisi_dma_chan_write(hdma_dev->queue_base, HISI_DMA_Q_SQ_TAIL_PTR,
+ chan->qp_num, chan->sq_tail);
}
static void hisi_dma_issue_pending(struct dma_chan *c)
@@ -375,26 +458,86 @@ static int hisi_dma_alloc_qps_mem(struct hisi_dma_dev *hdma_dev)
static void hisi_dma_init_hw_qp(struct hisi_dma_dev *hdma_dev, u32 index)
{
struct hisi_dma_chan *chan = &hdma_dev->chan[index];
+ void __iomem *q_base = hdma_dev->queue_base;
u32 hw_depth = hdma_dev->chan_depth - 1;
- void __iomem *base = hdma_dev->base;
+ void __iomem *addr;
+ u32 tmp;
/* set sq, cq base */
- hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_L, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_L, index,
lower_32_bits(chan->sq_dma));
- hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_H, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_H, index,
upper_32_bits(chan->sq_dma));
- hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_L, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_L, index,
lower_32_bits(chan->cq_dma));
- hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_H, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_H, index,
upper_32_bits(chan->cq_dma));
/* set sq, cq depth */
- hisi_dma_chan_write(base, HISI_DMA_SQ_DEPTH, index, hw_depth);
- hisi_dma_chan_write(base, HISI_DMA_CQ_DEPTH, index, hw_depth);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_DEPTH, index, hw_depth);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_DEPTH, index, hw_depth);
/* init sq tail and cq head */
- hisi_dma_chan_write(base, HISI_DMA_SQ_TAIL_PTR, index, 0);
- hisi_dma_chan_write(base, HISI_DMA_CQ_HEAD_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
+
+ /* init error interrupt stats */
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM0, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM1, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM2, index, 0);
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM3,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM4,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM5,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM6,
+ index, 0);
+ /**
+ * init SQ/CQ direction selecting register.
+ * "0" is to local side and "1" is to remote side.
+ */
+ addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT, 0);
+
+ /**
+ * 0 - Continue to next descriptor if error occurs.
+ * 1 - Abort the DMA queue if error occurs.
+ */
+ hisi_dma_update_bit(addr,
+ HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN, 0);
+ } else {
+ addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
+
+ /**
+ * init SQ/CQ direction selecting register.
+ * "0" is to local side and "1" is to remote side.
+ */
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT, 0);
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT, 0);
+
+ /**
+ * 0 - Continue to next descriptor if error occurs.
+ * 1 - Abort the DMA queue if error occurs.
+ */
+
+ tmp = readl_relaxed(addr);
+ tmp &= ~HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN;
+ writel_relaxed(tmp, addr);
+
+ /**
+ * 0 - dma should process FLR whith CPU.
+ * 1 - dma not process FLR, only cpu process FLR.
+ */
+ addr = q_base + HISI_DMA_HIP09_DMA_FLR_DISABLE +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_DMA_FLR_DISABLE_B, 0);
+
+ addr = q_base + HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE, 1);
+ }
}
static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
@@ -438,11 +581,13 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
struct hisi_dma_desc *desc;
struct hisi_dma_cqe *cqe;
+ void __iomem *q_base;
spin_lock(&chan->vc.lock);
desc = chan->desc;
cqe = chan->cq + chan->cq_head;
+ q_base = hdma_dev->queue_base;
if (desc) {
chan->cq_head = (chan->cq_head + 1) %
hdma_dev->chan_depth;
@@ -507,16 +652,58 @@ static void hisi_dma_disable_hw_channels(void *data)
static void hisi_dma_set_mode(struct hisi_dma_dev *hdma_dev,
enum hisi_dma_mode mode)
{
- writel_relaxed(mode == RC ? 1 : 0, hdma_dev->base + HISI_DMA_MODE);
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ writel_relaxed(mode == RC ? 1 : 0,
+ hdma_dev->base + HISI_DMA_HIP08_MODE);
+}
+
+static void hisi_dma_init_hw(struct hisi_dma_dev *hdma_dev)
+{
+ void __iomem *addr;
+ int i;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP09) {
+ for (i = 0; i < HISI_DMA_HIP09_MAX_PORT_NUM; i++) {
+ addr = hdma_dev->base + HISI_DMA_HIP09_PORT_CFG_REG(i);
+ hisi_dma_update_bit(addr,
+ HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B, 1);
+ }
+ }
+}
+
+static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
+{
+ struct dma_device *dma_dev;
+
+ dma_dev = &hdma_dev->dma_dev;
+ dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
+ dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
+ dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
+ dma_dev->device_tx_status = hisi_dma_tx_status;
+ dma_dev->device_issue_pending = hisi_dma_issue_pending;
+ dma_dev->device_terminate_all = hisi_dma_terminate_all;
+ dma_dev->device_synchronize = hisi_dma_synchronize;
+ dma_dev->directions = BIT(DMA_MEM_TO_MEM);
+ dma_dev->dev = &hdma_dev->pdev->dev;
+ INIT_LIST_HEAD(&dma_dev->channels);
}
static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
+ enum hisi_dma_reg_layout reg_layout;
struct device *dev = &pdev->dev;
struct hisi_dma_dev *hdma_dev;
struct dma_device *dma_dev;
+ u32 chan_num;
+ u32 msi_num;
int ret;
+ reg_layout = hisi_dma_get_reg_layout(pdev);
+ if (reg_layout == HISI_DMA_REG_LAYOUT_INVALID) {
+ dev_err(dev, "unsupported device!\n");
+ return -EINVAL;
+ }
+
ret = pcim_enable_device(pdev);
if (ret) {
dev_err(dev, "failed to enable device mem!\n");
@@ -533,40 +720,37 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (ret)
return ret;
- hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, HISI_DMA_CHAN_NUM), GFP_KERNEL);
+ chan_num = hisi_dma_get_chan_num(pdev);
+ hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, chan_num),
+ GFP_KERNEL);
if (!hdma_dev)
return -EINVAL;
hdma_dev->base = pcim_iomap_table(pdev)[PCI_BAR_2];
hdma_dev->pdev = pdev;
- hdma_dev->chan_num = HISI_DMA_CHAN_NUM;
hdma_dev->chan_depth = HISI_DMA_Q_DEPTH_VAL;
+ hdma_dev->chan_num = chan_num;
+ hdma_dev->reg_layout = reg_layout;
+ hdma_dev->queue_base = hdma_dev->base + hisi_dma_get_queue_base(pdev);
pci_set_drvdata(pdev, hdma_dev);
pci_set_master(pdev);
+ msi_num = hisi_dma_get_msi_num(pdev);
+
/* This will be freed by 'pcim_release()'. See 'pcim_enable_device()' */
- ret = pci_alloc_irq_vectors(pdev, HISI_DMA_MSI_NUM, HISI_DMA_MSI_NUM,
- PCI_IRQ_MSI);
+ ret = pci_alloc_irq_vectors(pdev, msi_num, msi_num, PCI_IRQ_MSI);
if (ret < 0) {
dev_err(dev, "Failed to allocate MSI vectors!\n");
return ret;
}
- dma_dev = &hdma_dev->dma_dev;
- dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
- dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
- dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
- dma_dev->device_tx_status = hisi_dma_tx_status;
- dma_dev->device_issue_pending = hisi_dma_issue_pending;
- dma_dev->device_terminate_all = hisi_dma_terminate_all;
- dma_dev->device_synchronize = hisi_dma_synchronize;
- dma_dev->directions = BIT(DMA_MEM_TO_MEM);
- dma_dev->dev = dev;
- INIT_LIST_HEAD(&dma_dev->channels);
+ hisi_dma_init_dma_dev(hdma_dev);
hisi_dma_set_mode(hdma_dev, RC);
+ hisi_dma_init_hw(hdma_dev);
+
ret = hisi_dma_enable_hw_channels(hdma_dev);
if (ret < 0) {
dev_err(dev, "failed to enable hw channel!\n");
@@ -578,8 +762,9 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (ret)
return ret;
+ dma_dev = &hdma_dev->dma_dev;
ret = dmaenginem_async_device_register(dma_dev);
- if (ret < 0)
+ if (ret < 0) {
dev_err(dev, "failed to register device!\n");
return ret;
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 6/8] dmaengine: hisilicon: Add dfx feature for hisi dma driver
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (4 preceding siblings ...)
2022-06-25 7:44 ` [PATCH 5/8] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
@ 2022-06-25 7:44 ` Jie Hai
2022-06-25 9:36 ` kernel test robot
2022-06-25 7:44 ` [PATCH 7/8] Documentation: Add debugfs doc for hisi_dma Jie Hai
` (5 subsequent siblings)
11 siblings, 1 reply; 64+ messages in thread
From: Jie Hai @ 2022-06-25 7:44 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
This patch adds dump of registers with debugfs for HIP08
and HIP09 DMA driver.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 350 ++++++++++++++++++++++++++++++++++++++++-
1 file changed, 348 insertions(+), 2 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index e1a5390567bd..21b84c4f4265 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -80,6 +80,9 @@
#define HISI_DMA_POLL_Q_STS_DELAY_US 10
#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+
+#define HISI_DMA_MAX_DIR_NAME_LEN 128
+
/**
* The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
* have the same pci device id but different pci revision.
@@ -163,9 +166,165 @@ struct hisi_dma_dev {
u32 chan_depth;
enum hisi_dma_reg_layout reg_layout;
void __iomem *queue_base; /* queue region start of register */
+#ifdef CONFIG_DEBUG_FS
+ struct dentry *dbg_hdev_root;
+ struct dentry **dbg_chans;
+#endif
struct hisi_dma_chan chan[];
};
+#ifdef CONFIG_DEBUG_FS
+struct dentry *hisi_dma_debugfs_root;
+
+static const struct debugfs_reg32 hisi_dma_comm_chan_regs[] = {
+ {"DMA_QUEUE_SQ_DEPTH ", 0x0008ull},
+ {"DMA_QUEUE_SQ_TAIL_PTR ", 0x000Cull},
+ {"DMA_QUEUE_CQ_DEPTH ", 0x0018ull},
+ {"DMA_QUEUE_CQ_HEAD_PTR ", 0x001Cull},
+ {"DMA_QUEUE_CTRL0 ", 0x0020ull},
+ {"DMA_QUEUE_CTRL1 ", 0x0024ull},
+ {"DMA_QUEUE_FSM_STS ", 0x0030ull},
+ {"DMA_QUEUE_SQ_STS ", 0x0034ull},
+ {"DMA_QUEUE_CQ_TAIL_PTR ", 0x003Cull},
+ {"DMA_QUEUE_INT_STS ", 0x0040ull},
+ {"DMA_QUEUE_INT_MSK ", 0x0044ull},
+ {"DMA_QUEUE_INT_RO ", 0x006Cull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip08_chan_regs[] = {
+ {"DMA_QUEUE_BYTE_CNT ", 0x0038ull},
+ {"DMA_ERR_INT_NUM6 ", 0x0048ull},
+ {"DMA_QUEUE_DESP0 ", 0x0050ull},
+ {"DMA_QUEUE_DESP1 ", 0x0054ull},
+ {"DMA_QUEUE_DESP2 ", 0x0058ull},
+ {"DMA_QUEUE_DESP3 ", 0x005Cull},
+ {"DMA_QUEUE_DESP4 ", 0x0074ull},
+ {"DMA_QUEUE_DESP5 ", 0x0078ull},
+ {"DMA_QUEUE_DESP6 ", 0x007Cull},
+ {"DMA_QUEUE_DESP7 ", 0x0080ull},
+ {"DMA_ERR_INT_NUM0 ", 0x0084ull},
+ {"DMA_ERR_INT_NUM1 ", 0x0088ull},
+ {"DMA_ERR_INT_NUM2 ", 0x008Cull},
+ {"DMA_ERR_INT_NUM3 ", 0x0090ull},
+ {"DMA_ERR_INT_NUM4 ", 0x0094ull},
+ {"DMA_ERR_INT_NUM5 ", 0x0098ull},
+ {"DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip09_chan_regs[] = {
+ {"DMA_QUEUE_ERR_INT_STS ", 0x0048ull},
+ {"DMA_QUEUE_ERR_INT_MSK ", 0x004Cull},
+ {"DFX_SQ_READ_ERR_PTR ", 0x0068ull},
+ {"DFX_DMA_ERR_INT_NUM0 ", 0x0084ull},
+ {"DFX_DMA_ERR_INT_NUM1 ", 0x0088ull},
+ {"DFX_DMA_ERR_INT_NUM2 ", 0x008Cull},
+ {"DFX_DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip08_comm_regs[] = {
+ {"DMA_ECC_ERR_ADDR ", 0x2004ull},
+ {"DMA_ECC_ECC_CNT ", 0x2014ull},
+ {"COMMON_AND_CH_ERR_STS ", 0x2030ull},
+ {"LOCAL_CPL_ID_STS_0 ", 0x20E0ull},
+ {"LOCAL_CPL_ID_STS_1 ", 0x20E4ull},
+ {"LOCAL_CPL_ID_STS_2 ", 0x20E8ull},
+ {"LOCAL_CPL_ID_STS_3 ", 0x20ECull},
+ {"LOCAL_TLP_NUM ", 0x2158ull},
+ {"SQCQ_TLP_NUM ", 0x2164ull},
+ {"CPL_NUM ", 0x2168ull},
+ {"INF_BACK_PRESS_STS ", 0x2170ull},
+ {"DMA_CH_RAS_LEVEL ", 0x2184ull},
+ {"DMA_CM_RAS_LEVEL ", 0x2188ull},
+ {"DMA_CH_ERR_STS ", 0x2190ull},
+ {"DMA_CH_DONE_STS ", 0x2194ull},
+ {"DMA_SQ_TAG_STS_0 ", 0x21A0ull},
+ {"DMA_SQ_TAG_STS_1 ", 0x21A4ull},
+ {"DMA_SQ_TAG_STS_2 ", 0x21A8ull},
+ {"DMA_SQ_TAG_STS_3 ", 0x21ACull},
+ {"LOCAL_P_ID_STS_0 ", 0x21B0ull},
+ {"LOCAL_P_ID_STS_1 ", 0x21B4ull},
+ {"LOCAL_P_ID_STS_2 ", 0x21B8ull},
+ {"LOCAL_P_ID_STS_3 ", 0x21BCull},
+ {"DMA_PREBUFF_INFO_0 ", 0x2200ull},
+ {"DMA_CM_TABLE_INFO_0 ", 0x2220ull},
+ {"DMA_CM_CE_RO ", 0x2244ull},
+ {"DMA_CM_NFE_RO ", 0x2248ull},
+ {"DMA_CM_FE_RO ", 0x224Cull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip09_comm_regs[] = {
+ {"COMMON_AND_CH_ERR_STS ", 0x0030ull},
+ {"DMA_PORT_IDLE_STS ", 0x0150ull},
+ {"DMA_CH_RAS_LEVEL ", 0x0184ull},
+ {"DMA_CM_RAS_LEVEL ", 0x0188ull},
+ {"DMA_CM_CE_RO ", 0x0244ull},
+ {"DMA_CM_NFE_RO ", 0x0248ull},
+ {"DMA_CM_FE_RO ", 0x024Cull},
+ {"DFX_INF_BACK_PRESS_STS0 ", 0x1A40ull},
+ {"DFX_INF_BACK_PRESS_STS1 ", 0x1A44ull},
+ {"DFX_INF_BACK_PRESS_STS2 ", 0x1A48ull},
+ {"DFX_DMA_WRR_DISABLE ", 0x1A4Cull},
+ {"DFX_PA_REQ_TLP_NUM ", 0x1C00ull},
+ {"DFX_PA_BACK_TLP_NUM ", 0x1C04ull},
+ {"DFX_PA_RETRY_TLP_NUM ", 0x1C08ull},
+ {"DFX_LOCAL_NP_TLP_NUM ", 0x1C0Cull},
+ {"DFX_LOCAL_CPL_HEAD_TLP_NUM ", 0x1C10ull},
+ {"DFX_LOCAL_CPL_DATA_TLP_NUM ", 0x1C14ull},
+ {"DFX_LOCAL_CPL_EXT_DATA_TLP_NUM ", 0x1C18ull},
+ {"DFX_LOCAL_P_HEAD_TLP_NUM ", 0x1C1Cull},
+ {"DFX_LOCAL_P_ACK_TLP_NUM ", 0x1C20ull},
+ {"DFX_BUF_ALOC_PORT_REQ_NUM ", 0x1C24ull},
+ {"DFX_BUF_ALOC_PORT_RESULT_NUM ", 0x1C28ull},
+ {"DFX_BUF_FAIL_SIZE_NUM ", 0x1C2Cull},
+ {"DFX_BUF_ALOC_SIZE_NUM ", 0x1C30ull},
+ {"DFX_BUF_NP_RELEASE_SIZE_NUM ", 0x1C34ull},
+ {"DFX_BUF_P_RELEASE_SIZE_NUM ", 0x1C38ull},
+ {"DFX_BUF_PORT_RELEASE_SIZE_NUM ", 0x1C3Cull},
+ {"DFX_DMA_PREBUF_MEM0_ECC_ERR_ADDR ", 0x1CA8ull},
+ {"DFX_DMA_PREBUF_MEM0_ECC_CNT ", 0x1CACull},
+ {"DFX_DMA_LOC_NP_OSTB_ECC_ERR_ADDR ", 0x1CB0ull},
+ {"DFX_DMA_LOC_NP_OSTB_ECC_CNT ", 0x1CB4ull},
+ {"DFX_DMA_PREBUF_MEM1_ECC_ERR_ADDR ", 0x1CC0ull},
+ {"DFX_DMA_PREBUF_MEM1_ECC_CNT ", 0x1CC4ull},
+ {"DMA_CH_DONE_STS ", 0x02E0ull},
+ {"DMA_CH_ERR_STS ", 0x0320ull},
+};
+#endif /* CONFIG_DEBUG_FS*/
+
+static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_REG_LAYOUT_HIP08;
+ else if (pdev->revision >= HISI_DMA_REVISION_HIP09A)
+ return HISI_DMA_REG_LAYOUT_HIP09;
+
+ return HISI_DMA_REG_LAYOUT_INVALID;
+}
+
+static u32 hisi_dma_get_chan_num(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_CHAN_NUM;
+
+ return HISI_DMA_HIP09_CHAN_NUM;
+}
+
+static u32 hisi_dma_get_msi_num(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_MSI_NUM;
+
+ return HISI_DMA_HIP09_MSI_NUM;
+}
+
+static u32 hisi_dma_get_queue_base(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_Q_BASE;
+
+ return HISI_DMA_HIP09_Q_BASE;
+}
+
static inline struct hisi_dma_chan *to_hisi_dma_chan(struct dma_chan *c)
{
return container_of(c, struct hisi_dma_chan, vc.chan);
@@ -688,6 +847,162 @@ static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
INIT_LIST_HEAD(&dma_dev->channels);
}
+/* --- debugfs implementation --- */
+#ifdef CONFIG_DEBUG_FS
+#include <linux/debugfs.h>
+
+static void hisi_dma_debugfs_init(void)
+{
+ if (!debugfs_initialized())
+ return;
+ hisi_dma_debugfs_root = debugfs_create_dir("hisi_dma", NULL);
+}
+
+static void hisi_dma_debugfs_uninit(void)
+{
+ debugfs_remove_recursive(hisi_dma_debugfs_root);
+}
+
+static struct debugfs_reg32 *hisi_dma_get_ch_regs(struct hisi_dma_dev *hdma_dev,
+ u32 *regs_sz)
+{
+ struct device *dev = &hdma_dev->pdev->dev;
+ struct debugfs_reg32 *regs;
+ u32 regs_sz_comm;
+
+ regs_sz_comm = ARRAY_SIZE(hisi_dma_comm_chan_regs);
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip08_chan_regs);
+ else
+ *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip09_chan_regs);
+
+ regs = devm_kcalloc(dev, *regs_sz, sizeof(struct debugfs_reg32),
+ GFP_KERNEL);
+ if (!regs)
+ return NULL;
+ memcpy(regs, hisi_dma_comm_chan_regs, sizeof(hisi_dma_comm_chan_regs));
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ memcpy(regs + regs_sz_comm, hisi_dma_hip08_chan_regs,
+ sizeof(hisi_dma_hip08_chan_regs));
+ else
+ memcpy(regs + regs_sz_comm, hisi_dma_hip09_chan_regs,
+ sizeof(hisi_dma_hip09_chan_regs));
+
+ return regs;
+}
+
+static int hisi_dma_create_chan_dir(struct hisi_dma_dev *hdma_dev)
+{
+ char dir_name[HISI_DMA_MAX_DIR_NAME_LEN];
+ struct debugfs_regset32 *regsets;
+ struct debugfs_reg32 *regs;
+ struct device *dev;
+ u32 regs_sz;
+ int ret;
+ int i;
+
+ dev = &hdma_dev->pdev->dev;
+
+ regsets = devm_kcalloc(dev, hdma_dev->chan_num,
+ sizeof(*regsets), GFP_KERNEL);
+ if (!regsets)
+ return -ENOMEM;
+
+ hdma_dev->dbg_chans = devm_kcalloc(dev, hdma_dev->chan_num,
+ sizeof(struct dentry *),
+ GFP_KERNEL);
+ if (!hdma_dev->dbg_chans)
+ return -ENOMEM;
+
+ regs = hisi_dma_get_ch_regs(hdma_dev, ®s_sz);
+ if (!regs)
+ return -ENOMEM;
+ for (i = 0; i < hdma_dev->chan_num; i++) {
+ regsets[i].regs = regs;
+ regsets[i].nregs = regs_sz;
+ regsets[i].base = hdma_dev->queue_base + i * HISI_DMA_Q_OFFSET;
+ regsets[i].dev = dev;
+
+ memset(dir_name, 0, HISI_DMA_MAX_DIR_NAME_LEN);
+ ret = sprintf(dir_name, "channel%d", i);
+ if (ret < 0)
+ return ret;
+
+ hdma_dev->dbg_chans[i] = debugfs_create_dir(dir_name,
+ hdma_dev->dbg_hdev_root);
+ if (IS_ERR(hdma_dev->dbg_chans[i])) {
+ hdma_dev->dbg_chans[i] = NULL;
+ dev_err(dev, "dbg_chan[%d] create fail!\n", i);
+ return -EINVAL;
+ }
+ debugfs_create_regset32("regs", 0444,
+ hdma_dev->dbg_chans[i], ®sets[i]);
+ }
+
+ return 0;
+}
+
+static int hisi_dma_debug_register(struct hisi_dma_dev *hdma_dev)
+{
+ struct debugfs_regset32 *regset;
+ struct device *dev;
+ int ret;
+
+ dev = &hdma_dev->pdev->dev;
+
+ hdma_dev->dbg_hdev_root = debugfs_create_dir(dev_name(dev),
+ hisi_dma_debugfs_root);
+ regset = devm_kzalloc(dev, sizeof(*regset), GFP_KERNEL);
+ if (!regset) {
+ ret = -ENOMEM;
+ goto hisi_dma_debug_register_fail;
+ }
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ regset->regs = hisi_dma_hip08_comm_regs;
+ regset->nregs = ARRAY_SIZE(hisi_dma_hip08_comm_regs);
+ } else {
+ regset->regs = hisi_dma_hip09_comm_regs;
+ regset->nregs = ARRAY_SIZE(hisi_dma_hip09_comm_regs);
+ }
+ regset->base = hdma_dev->base;
+ regset->dev = dev;
+
+ debugfs_create_regset32("regs", 0444,
+ hdma_dev->dbg_hdev_root, regset);
+
+ ret = hisi_dma_create_chan_dir(hdma_dev);
+ if (ret < 0)
+ goto hisi_dma_debug_register_fail;
+
+ return 0;
+
+hisi_dma_debug_register_fail:
+ debugfs_remove_recursive(hdma_dev->dbg_hdev_root);
+ hdma_dev->dbg_hdev_root = NULL;
+ return ret;
+}
+
+static void hisi_dma_debug_unregister(void *data)
+{
+ struct hisi_dma_dev *hdma_dev = data;
+
+ debugfs_remove_recursive(hdma_dev->dbg_hdev_root);
+ hdma_dev->dbg_hdev_root = NULL;
+}
+#else
+static void hisi_dma_debugfs_init(void) { }
+static void hisi_dma_debugfs_uninit(void) { }
+
+static int hisi_dma_debug_register(struct hisi_dma_dev *hdma_dev)
+{
+ return 0;
+}
+
+static void hisi_dma_debug_unregister(void *data) { }
+#endif /* CONFIG_DEBUG_FS*/
+/* --- debugfs implementation --- */
+
static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
enum hisi_dma_reg_layout reg_layout;
@@ -766,8 +1081,17 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
ret = dmaenginem_async_device_register(dma_dev);
if (ret < 0) {
dev_err(dev, "failed to register device!\n");
+ return ret;
+ }
- return ret;
+ ret = hisi_dma_debug_register(hdma_dev);
+ if (ret < 0) {
+ dev_err(dev, "failed to register debugfs!\n");
+ return ret;
+ }
+
+ return devm_add_action_or_reset(dev, hisi_dma_debug_unregister,
+ hdma_dev);
}
static const struct pci_device_id hisi_dma_pci_tbl[] = {
@@ -781,7 +1105,29 @@ static struct pci_driver hisi_dma_pci_driver = {
.probe = hisi_dma_probe,
};
-module_pci_driver(hisi_dma_pci_driver);
+static int __init hisi_dma_init(void)
+{
+ int ret;
+
+ hisi_dma_debugfs_init();
+
+ ret = pci_register_driver(&hisi_dma_pci_driver);
+ if (ret) {
+ hisi_dma_debugfs_uninit();
+ pr_err("hisi_dma: can't register hisi dma driver.\n");
+ }
+
+ return ret;
+}
+
+static void __exit hisi_dma_exit(void)
+{
+ pci_unregister_driver(&hisi_dma_pci_driver);
+ hisi_dma_debugfs_uninit();
+}
+
+module_init(hisi_dma_init);
+module_exit(hisi_dma_exit);
MODULE_AUTHOR("Zhou Wang <wangzhou1@hisilicon.com>");
MODULE_AUTHOR("Zhenfa Qiu <qiuzhenfa@hisilicon.com>");
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 7/8] Documentation: Add debugfs doc for hisi_dma
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (5 preceding siblings ...)
2022-06-25 7:44 ` [PATCH 6/8] dmaengine: hisilicon: Add dfx feature for hisi dma driver Jie Hai
@ 2022-06-25 7:44 ` Jie Hai
2022-06-27 5:50 ` Vinod Koul
2022-06-25 7:44 ` [PATCH 8/8] MAINTAINERS: Add debugfs files and maintainer " Jie Hai
` (4 subsequent siblings)
11 siblings, 1 reply; 64+ messages in thread
From: Jie Hai @ 2022-06-25 7:44 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
Add debugfs descriptions for HiSilicon DMA driver.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
Documentation/ABI/testing/debugfs-hisi-dma | 9 +++++++++
1 file changed, 9 insertions(+)
create mode 100644 Documentation/ABI/testing/debugfs-hisi-dma
diff --git a/Documentation/ABI/testing/debugfs-hisi-dma b/Documentation/ABI/testing/debugfs-hisi-dma
new file mode 100644
index 000000000000..162c97945748
--- /dev/null
+++ b/Documentation/ABI/testing/debugfs-hisi-dma
@@ -0,0 +1,9 @@
+What: /sys/kernel/debug/hisi_dma/<bdf>/regs
+Date: Mar 2022
+Contact: dmaengine@vger.kernel.org
+Description: Dump the debug registers from the hisi dma.
+
+What: /sys/kernel/debug/hisi_dma/<bdf>/channel[id]/regs
+Date: Mar 2022
+Contact: dmaengine@vger.kernel.org
+Description: Dump the channel related debug registers from the hisi dma.
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH 8/8] MAINTAINERS: Add debugfs files and maintainer for hisi_dma
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (6 preceding siblings ...)
2022-06-25 7:44 ` [PATCH 7/8] Documentation: Add debugfs doc for hisi_dma Jie Hai
@ 2022-06-25 7:44 ` Jie Hai
2022-06-29 3:55 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (3 subsequent siblings)
11 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-25 7:44 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
This patch does followed things:
1. Add debugfs-hisi-dma path to MAINTAINERS.
2. Add myself as a maintainer for hisi_dma.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
MAINTAINERS | 2 ++
1 file changed, 2 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 3cf9842d9233..e30d8256a39e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8879,8 +8879,10 @@ F: net/dsa/tag_hellcreek.c
HISILICON DMA DRIVER
M: Zhou Wang <wangzhou1@hisilicon.com>
+M: Jie Hai <haijie1@hisilicon.com>
L: dmaengine@vger.kernel.org
S: Maintained
+F: Documentation/ABI/testing/debugfs-hisi-dma
F: drivers/dma/hisi_dma.c
HISILICON GPIO DRIVER
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* Re: [PATCH 6/8] dmaengine: hisilicon: Add dfx feature for hisi dma driver
2022-06-25 7:44 ` [PATCH 6/8] dmaengine: hisilicon: Add dfx feature for hisi dma driver Jie Hai
@ 2022-06-25 9:36 ` kernel test robot
2022-06-27 6:55 ` haijie
0 siblings, 1 reply; 64+ messages in thread
From: kernel test robot @ 2022-06-25 9:36 UTC (permalink / raw)
To: Jie Hai, vkoul, wangzhou1; +Cc: kbuild-all, dmaengine, linux-kernel
Hi Jie,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on vkoul-dmaengine/next]
[also build test WARNING on linus/master v5.19-rc3 next-20220624]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/intel-lab-lkp/linux/commits/Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
base: https://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine.git next
config: microblaze-randconfig-r022-20220625
compiler: microblaze-linux-gcc (GCC) 11.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/ffaa89af83c2321f12a2b4d87711c9e7f7e37134
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
git checkout ffaa89af83c2321f12a2b4d87711c9e7f7e37134
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.3.0 make.cross W=1 O=build_dir ARCH=microblaze SHELL=/bin/bash drivers/dma/
If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> drivers/dma/hisi_dma.c:87: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst
* The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
vim +87 drivers/dma/hisi_dma.c
ffaa89af83c232 Jie Hai 2022-06-25 85
ae8a14d7255c1e Jie Hai 2022-06-25 86 /**
ae8a14d7255c1e Jie Hai 2022-06-25 @87 * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
ae8a14d7255c1e Jie Hai 2022-06-25 88 * have the same pci device id but different pci revision.
ae8a14d7255c1e Jie Hai 2022-06-25 89 * Unfortunately, they have different register layouts, so two layout
ae8a14d7255c1e Jie Hai 2022-06-25 90 * enumerations are defined.
ae8a14d7255c1e Jie Hai 2022-06-25 91 */
ae8a14d7255c1e Jie Hai 2022-06-25 92 enum hisi_dma_reg_layout {
ae8a14d7255c1e Jie Hai 2022-06-25 93 HISI_DMA_REG_LAYOUT_INVALID = 0,
ae8a14d7255c1e Jie Hai 2022-06-25 94 HISI_DMA_REG_LAYOUT_HIP08,
ae8a14d7255c1e Jie Hai 2022-06-25 95 HISI_DMA_REG_LAYOUT_HIP09
ae8a14d7255c1e Jie Hai 2022-06-25 96 };
7ddbde084de590 Jie Hai 2022-06-25 97
--
0-DAY CI Kernel Test Service
https://01.org/lkp
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update
2022-06-25 7:44 ` [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update Jie Hai
@ 2022-06-26 13:37 ` kernel test robot
2022-06-27 6:55 ` haijie
2022-06-27 6:12 ` Vinod Koul
1 sibling, 1 reply; 64+ messages in thread
From: kernel test robot @ 2022-06-26 13:37 UTC (permalink / raw)
To: Jie Hai, vkoul, wangzhou1; +Cc: kbuild-all, dmaengine, linux-kernel
Hi Jie,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on vkoul-dmaengine/next]
[also build test ERROR on linus/master v5.19-rc3 next-20220624]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/intel-lab-lkp/linux/commits/Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
base: https://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine.git next
config: arc-allyesconfig
compiler: arceb-elf-gcc (GCC) 11.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/4a79d13d35e4f95c88bc0dfb44923dbd030bb126
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
git checkout 4a79d13d35e4f95c88bc0dfb44923dbd030bb126
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.3.0 make.cross W=1 O=build_dir ARCH=arc SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>
Note: the linux-review/Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524 HEAD e823cc5940ad1d20993113591a7ba26946ae0840 builds fine.
It only hurts bisectability.
All errors (new ones prefixed by >>):
drivers/dma/hisi_dma.c: In function 'hisi_dma_irq':
>> drivers/dma/hisi_dma.c:441:37: error: 'q_base' undeclared (first use in this function)
441 | hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
| ^~~~~~
drivers/dma/hisi_dma.c:441:37: note: each undeclared identifier is reported only once for each function it appears in
>> drivers/dma/hisi_dma.c:441:45: error: 'HISI_DMA_Q_CQ_HEAD_PTR' undeclared (first use in this function); did you mean 'HISI_DMA_CQ_HEAD_PTR'?
441 | hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
| ^~~~~~~~~~~~~~~~~~~~~~
| HISI_DMA_CQ_HEAD_PTR
vim +/q_base +441 drivers/dma/hisi_dma.c
426
427 static irqreturn_t hisi_dma_irq(int irq, void *data)
428 {
429 struct hisi_dma_chan *chan = data;
430 struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
431 struct hisi_dma_desc *desc;
432 struct hisi_dma_cqe *cqe;
433
434 spin_lock(&chan->vc.lock);
435
436 desc = chan->desc;
437 cqe = chan->cq + chan->cq_head;
438 if (desc) {
439 chan->cq_head = (chan->cq_head + 1) %
440 hdma_dev->chan_depth;
> 441 hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
442 chan->qp_num, chan->cq_head);
443 if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
444 vchan_cookie_complete(&desc->vd);
445 } else {
446 dev_err(&hdma_dev->pdev->dev, "task error!\n");
447 }
448
449 chan->desc = NULL;
450 }
451
452 spin_unlock(&chan->vc.lock);
453
454 return IRQ_HANDLED;
455 }
456
--
0-DAY CI Kernel Test Service
https://01.org/lkp
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 7/8] Documentation: Add debugfs doc for hisi_dma
2022-06-25 7:44 ` [PATCH 7/8] Documentation: Add debugfs doc for hisi_dma Jie Hai
@ 2022-06-27 5:50 ` Vinod Koul
0 siblings, 0 replies; 64+ messages in thread
From: Vinod Koul @ 2022-06-27 5:50 UTC (permalink / raw)
To: Jie Hai; +Cc: wangzhou1, dmaengine, linux-kernel
On 25-06-22, 15:44, Jie Hai wrote:
> Add debugfs descriptions for HiSilicon DMA driver.
>
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
> Documentation/ABI/testing/debugfs-hisi-dma | 9 +++++++++
> 1 file changed, 9 insertions(+)
> create mode 100644 Documentation/ABI/testing/debugfs-hisi-dma
>
> diff --git a/Documentation/ABI/testing/debugfs-hisi-dma b/Documentation/ABI/testing/debugfs-hisi-dma
> new file mode 100644
> index 000000000000..162c97945748
> --- /dev/null
> +++ b/Documentation/ABI/testing/debugfs-hisi-dma
debugfs is not an ABI so no need to document this, so pls drop this
> @@ -0,0 +1,9 @@
> +What: /sys/kernel/debug/hisi_dma/<bdf>/regs
> +Date: Mar 2022
> +Contact: dmaengine@vger.kernel.org
> +Description: Dump the debug registers from the hisi dma.
> +
> +What: /sys/kernel/debug/hisi_dma/<bdf>/channel[id]/regs
> +Date: Mar 2022
> +Contact: dmaengine@vger.kernel.org
> +Description: Dump the channel related debug registers from the hisi dma.
> --
> 2.33.0
--
~Vinod
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update
2022-06-25 7:44 ` [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update Jie Hai
2022-06-26 13:37 ` kernel test robot
@ 2022-06-27 6:12 ` Vinod Koul
2022-06-27 7:01 ` haijie
1 sibling, 1 reply; 64+ messages in thread
From: Vinod Koul @ 2022-06-27 6:12 UTC (permalink / raw)
To: Jie Hai; +Cc: wangzhou1, dmaengine, linux-kernel
On 25-06-22, 15:44, Jie Hai wrote:
> After completion of data transfer of one or multiple descriptors,
> the completion status and the current head pointer to submission
> queue are written into the CQ and interrupt can be generated to
> inform the software. In interrupt process CQ is read and cq_head
> is updated.
>
> hisi_dma_irq updates cq_head only when the completion status is
> success. When an abnormal interrupt reports, cq_head will not update
> which will cause subsequent interrupt processes read the error CQ
> and never report the correct status.
>
> This patch updates cq_head whenever CQ is accessed.
>
> Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
>
No need for blank line
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
> drivers/dma/hisi_dma.c | 9 ++++-----
> 1 file changed, 4 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
> index 98bc488893cc..0a0f8a4d168a 100644
> --- a/drivers/dma/hisi_dma.c
> +++ b/drivers/dma/hisi_dma.c
> @@ -436,12 +436,11 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
> desc = chan->desc;
> cqe = chan->cq + chan->cq_head;
> if (desc) {
> + chan->cq_head = (chan->cq_head + 1) %
> + hdma_dev->chan_depth;
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
q_base?
--
~Vinod
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 3/8] dmaengine: hisilicon: Add multi-thread support for a DMA channel
2022-06-25 7:44 ` [PATCH 3/8] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
@ 2022-06-27 6:21 ` Vinod Koul
[not found] ` <eed10a2b5ea745878024ca653158f4de@huawei.com>
2022-06-29 3:44 ` Jie Hai
0 siblings, 2 replies; 64+ messages in thread
From: Vinod Koul @ 2022-06-27 6:21 UTC (permalink / raw)
To: Jie Hai; +Cc: wangzhou1, dmaengine, linux-kernel
On 25-06-22, 15:44, Jie Hai wrote:
> When we get a DMA channel and try to use it in multiple threads it
> will cause oops and hanging the system.
>
> % echo 100 > /sys/module/dmatest/parameters/threads_per_chan
> % echo 100 > /sys/module/dmatest/parameters/iterations
> % echo 1 > /sys/module/dmatest/parameters/run
> [383493.327077] Unable to handle kernel paging request at virtual
> address dead000000000108
> [383493.335103] Mem abort info:
> [383493.335103] ESR = 0x96000044
> [383493.335105] EC = 0x25: DABT (current EL), IL = 32 bits
> [383493.335107] SET = 0, FnV = 0
> [383493.335108] EA = 0, S1PTW = 0
> [383493.335109] FSC = 0x04: level 0 translation fault
> [383493.335110] Data abort info:
> [383493.335111] ISV = 0, ISS = 0x00000044
> [383493.364739] CM = 0, WnR = 1
> [383493.367793] [dead000000000108] address between user and kernel
> address ranges
> [383493.375021] Internal error: Oops: 96000044 [#1] PREEMPT SMP
> [383493.437574] CPU: 63 PID: 27895 Comm: dma0chan0-copy2 Kdump:
> loaded Tainted: GO 5.17.0-rc4+ #2
> [383493.457851] pstate: 204000c9 (nzCv daIF +PAN -UAO -TCO -DIT
> -SSBS BTYPE=--)
> [383493.465331] pc : vchan_tx_submit+0x64/0xa0
> [383493.469957] lr : vchan_tx_submit+0x34/0xa0
>
> This happens because of data race. Each thread rewrite channels's
> descriptor as soon as device_issue_pending is called. It leads to
> the situation that the driver thinks that it uses the right
> descriptor in interrupt handler while channels's descriptor has
> been changed by other thread.
>
> With current fixes channels's descriptor changes it's value only
> when it has been used. A new descriptor is acquired from
> vc->desc_issued queue that is already filled with descriptors
> that are ready to be sent. Threads have no direct access to DMA
> channel descriptor. Now it is just possible to queue a descriptor
> for further processing.
>
> Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
> drivers/dma/hisi_dma.c | 6 ++----
> 1 file changed, 2 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
> index 0a0f8a4d168a..0385419be8d5 100644
> --- a/drivers/dma/hisi_dma.c
> +++ b/drivers/dma/hisi_dma.c
> @@ -271,7 +271,6 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
>
> vd = vchan_next_desc(&chan->vc);
> if (!vd) {
> - dev_err(&hdma_dev->pdev->dev, "no issued task!\n");
how is this a fix?
> chan->desc = NULL;
> return;
> }
> @@ -303,7 +302,7 @@ static void hisi_dma_issue_pending(struct dma_chan *c)
>
> spin_lock_irqsave(&chan->vc.lock, flags);
>
> - if (vchan_issue_pending(&chan->vc))
> + if (vchan_issue_pending(&chan->vc) && !chan->desc)
This looks good
> hisi_dma_start_transfer(chan);
>
> spin_unlock_irqrestore(&chan->vc.lock, flags);
> @@ -442,11 +441,10 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
> chan->qp_num, chan->cq_head);
> if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
> vchan_cookie_complete(&desc->vd);
> + hisi_dma_start_transfer(chan);
Why should this fix the error reported?
> } else {
> dev_err(&hdma_dev->pdev->dev, "task error!\n");
> }
> -
> - chan->desc = NULL;
> }
>
> spin_unlock(&chan->vc.lock);
> --
> 2.33.0
--
~Vinod
^ permalink raw reply [flat|nested] 64+ messages in thread
* RE: [PATCH 6/8] dmaengine: hisilicon: Add dfx feature for hisi dma driver
2022-06-25 9:36 ` kernel test robot
@ 2022-06-27 6:55 ` haijie
0 siblings, 0 replies; 64+ messages in thread
From: haijie @ 2022-06-27 6:55 UTC (permalink / raw)
To: kernel test robot, vkoul, Wangzhou (B)
Cc: kbuild-all, dmaengine, linux-kernel
Hi, kernel test robot,
Thanks and this will be corrected in the next version.
-----Original Message-----
From: kernel test robot [mailto:lkp@intel.com]
Sent: Saturday, June 25, 2022 5:37 PM
To: haijie <haijie1@huawei.com>; vkoul@kernel.org; Wangzhou (B) <wangzhou1@hisilicon.com>
Cc: kbuild-all@lists.01.org; dmaengine@vger.kernel.org; linux-kernel@vger.kernel.org
Subject: Re: [PATCH 6/8] dmaengine: hisilicon: Add dfx feature for hisi dma driver
Hi Jie,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on vkoul-dmaengine/next] [also build test WARNING on linus/master v5.19-rc3 next-20220624] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
url: https://github.com/intel-lab-lkp/linux/commits/Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
base: https://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine.git next
config: microblaze-randconfig-r022-20220625
compiler: microblaze-linux-gcc (GCC) 11.3.0 reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/ffaa89af83c2321f12a2b4d87711c9e7f7e37134
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
git checkout ffaa89af83c2321f12a2b4d87711c9e7f7e37134
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.3.0 make.cross W=1 O=build_dir ARCH=microblaze SHELL=/bin/bash drivers/dma/
If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> drivers/dma/hisi_dma.c:87: warning: This comment starts with '/**',
>> but isn't a kernel-doc comment. Refer
>> Documentation/doc-guide/kernel-doc.rst
* The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
vim +87 drivers/dma/hisi_dma.c
ffaa89af83c232 Jie Hai 2022-06-25 85
ae8a14d7255c1e Jie Hai 2022-06-25 86 /**
ae8a14d7255c1e Jie Hai 2022-06-25 @87 * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
ae8a14d7255c1e Jie Hai 2022-06-25 88 * have the same pci device id but different pci revision.
ae8a14d7255c1e Jie Hai 2022-06-25 89 * Unfortunately, they have different register layouts, so two layout
ae8a14d7255c1e Jie Hai 2022-06-25 90 * enumerations are defined.
ae8a14d7255c1e Jie Hai 2022-06-25 91 */
ae8a14d7255c1e Jie Hai 2022-06-25 92 enum hisi_dma_reg_layout {
ae8a14d7255c1e Jie Hai 2022-06-25 93 HISI_DMA_REG_LAYOUT_INVALID = 0,
ae8a14d7255c1e Jie Hai 2022-06-25 94 HISI_DMA_REG_LAYOUT_HIP08,
ae8a14d7255c1e Jie Hai 2022-06-25 95 HISI_DMA_REG_LAYOUT_HIP09
ae8a14d7255c1e Jie Hai 2022-06-25 96 };
7ddbde084de590 Jie Hai 2022-06-25 97
--
0-DAY CI Kernel Test Service
https://01.org/lkp
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 6/8] dmaengine: hisilicon: Add dfx feature for hisi dma driver
@ 2022-06-27 6:55 ` haijie
0 siblings, 0 replies; 64+ messages in thread
From: haijie @ 2022-06-27 6:55 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 3276 bytes --]
Hi, kernel test robot,
Thanks and this will be corrected in the next version.
-----Original Message-----
From: kernel test robot [mailto:lkp(a)intel.com]
Sent: Saturday, June 25, 2022 5:37 PM
To: haijie <haijie1@huawei.com>; vkoul(a)kernel.org; Wangzhou (B) <wangzhou1@hisilicon.com>
Cc: kbuild-all(a)lists.01.org; dmaengine(a)vger.kernel.org; linux-kernel(a)vger.kernel.org
Subject: Re: [PATCH 6/8] dmaengine: hisilicon: Add dfx feature for hisi dma driver
Hi Jie,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on vkoul-dmaengine/next] [also build test WARNING on linus/master v5.19-rc3 next-20220624] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
url: https://github.com/intel-lab-lkp/linux/commits/Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
base: https://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine.git next
config: microblaze-randconfig-r022-20220625
compiler: microblaze-linux-gcc (GCC) 11.3.0 reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/ffaa89af83c2321f12a2b4d87711c9e7f7e37134
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
git checkout ffaa89af83c2321f12a2b4d87711c9e7f7e37134
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.3.0 make.cross W=1 O=build_dir ARCH=microblaze SHELL=/bin/bash drivers/dma/
If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
>> drivers/dma/hisi_dma.c:87: warning: This comment starts with '/**',
>> but isn't a kernel-doc comment. Refer
>> Documentation/doc-guide/kernel-doc.rst
* The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
vim +87 drivers/dma/hisi_dma.c
ffaa89af83c232 Jie Hai 2022-06-25 85
ae8a14d7255c1e Jie Hai 2022-06-25 86 /**
ae8a14d7255c1e Jie Hai 2022-06-25 @87 * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
ae8a14d7255c1e Jie Hai 2022-06-25 88 * have the same pci device id but different pci revision.
ae8a14d7255c1e Jie Hai 2022-06-25 89 * Unfortunately, they have different register layouts, so two layout
ae8a14d7255c1e Jie Hai 2022-06-25 90 * enumerations are defined.
ae8a14d7255c1e Jie Hai 2022-06-25 91 */
ae8a14d7255c1e Jie Hai 2022-06-25 92 enum hisi_dma_reg_layout {
ae8a14d7255c1e Jie Hai 2022-06-25 93 HISI_DMA_REG_LAYOUT_INVALID = 0,
ae8a14d7255c1e Jie Hai 2022-06-25 94 HISI_DMA_REG_LAYOUT_HIP08,
ae8a14d7255c1e Jie Hai 2022-06-25 95 HISI_DMA_REG_LAYOUT_HIP09
ae8a14d7255c1e Jie Hai 2022-06-25 96 };
7ddbde084de590 Jie Hai 2022-06-25 97
--
0-DAY CI Kernel Test Service
https://01.org/lkp
^ permalink raw reply [flat|nested] 64+ messages in thread
* RE: [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update
2022-06-26 13:37 ` kernel test robot
@ 2022-06-27 6:55 ` haijie
0 siblings, 0 replies; 64+ messages in thread
From: haijie @ 2022-06-27 6:55 UTC (permalink / raw)
To: kernel test robot, vkoul, Wangzhou (B)
Cc: kbuild-all, dmaengine, linux-kernel
Hi, kernel test robot,
Thanks and this will be corrected in the next version.
-----Original Message-----
From: kernel test robot [mailto:lkp@intel.com]
Sent: Sunday, June 26, 2022 9:38 PM
To: haijie <haijie1@huawei.com>; vkoul@kernel.org; Wangzhou (B) <wangzhou1@hisilicon.com>
Cc: kbuild-all@lists.01.org; dmaengine@vger.kernel.org; linux-kernel@vger.kernel.org
Subject: Re: [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update
Hi Jie,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on vkoul-dmaengine/next] [also build test ERROR on linus/master v5.19-rc3 next-20220624] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
url: https://github.com/intel-lab-lkp/linux/commits/Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
base: https://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine.git next
config: arc-allyesconfig
compiler: arceb-elf-gcc (GCC) 11.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/4a79d13d35e4f95c88bc0dfb44923dbd030bb126
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
git checkout 4a79d13d35e4f95c88bc0dfb44923dbd030bb126
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.3.0 make.cross W=1 O=build_dir ARCH=arc SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>
Note: the linux-review/Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524 HEAD e823cc5940ad1d20993113591a7ba26946ae0840 builds fine.
It only hurts bisectability.
All errors (new ones prefixed by >>):
drivers/dma/hisi_dma.c: In function 'hisi_dma_irq':
>> drivers/dma/hisi_dma.c:441:37: error: 'q_base' undeclared (first use
>> in this function)
441 | hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
| ^~~~~~
drivers/dma/hisi_dma.c:441:37: note: each undeclared identifier is reported only once for each function it appears in
>> drivers/dma/hisi_dma.c:441:45: error: 'HISI_DMA_Q_CQ_HEAD_PTR' undeclared (first use in this function); did you mean 'HISI_DMA_CQ_HEAD_PTR'?
441 | hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
| ^~~~~~~~~~~~~~~~~~~~~~
| HISI_DMA_CQ_HEAD_PTR
vim +/q_base +441 drivers/dma/hisi_dma.c
426
427 static irqreturn_t hisi_dma_irq(int irq, void *data)
428 {
429 struct hisi_dma_chan *chan = data;
430 struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
431 struct hisi_dma_desc *desc;
432 struct hisi_dma_cqe *cqe;
433
434 spin_lock(&chan->vc.lock);
435
436 desc = chan->desc;
437 cqe = chan->cq + chan->cq_head;
438 if (desc) {
439 chan->cq_head = (chan->cq_head + 1) %
440 hdma_dev->chan_depth;
> 441 hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
442 chan->qp_num, chan->cq_head);
443 if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
444 vchan_cookie_complete(&desc->vd);
445 } else {
446 dev_err(&hdma_dev->pdev->dev, "task error!\n");
447 }
448
449 chan->desc = NULL;
450 }
451
452 spin_unlock(&chan->vc.lock);
453
454 return IRQ_HANDLED;
455 }
456
--
0-DAY CI Kernel Test Service
https://01.org/lkp
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update
@ 2022-06-27 6:55 ` haijie
0 siblings, 0 replies; 64+ messages in thread
From: haijie @ 2022-06-27 6:55 UTC (permalink / raw)
To: kbuild-all
[-- Attachment #1: Type: text/plain, Size: 4018 bytes --]
Hi, kernel test robot,
Thanks and this will be corrected in the next version.
-----Original Message-----
From: kernel test robot [mailto:lkp(a)intel.com]
Sent: Sunday, June 26, 2022 9:38 PM
To: haijie <haijie1@huawei.com>; vkoul(a)kernel.org; Wangzhou (B) <wangzhou1@hisilicon.com>
Cc: kbuild-all(a)lists.01.org; dmaengine(a)vger.kernel.org; linux-kernel(a)vger.kernel.org
Subject: Re: [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update
Hi Jie,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on vkoul-dmaengine/next] [also build test ERROR on linus/master v5.19-rc3 next-20220624] [If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in https://git-scm.com/docs/git-format-patch]
url: https://github.com/intel-lab-lkp/linux/commits/Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
base: https://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine.git next
config: arc-allyesconfig
compiler: arceb-elf-gcc (GCC) 11.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/4a79d13d35e4f95c88bc0dfb44923dbd030bb126
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524
git checkout 4a79d13d35e4f95c88bc0dfb44923dbd030bb126
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-11.3.0 make.cross W=1 O=build_dir ARCH=arc SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
Reported-by: kernel test robot <lkp@intel.com>
Note: the linux-review/Jie-Hai/dmaengine-hisilicon-Add-support-for-hisi-dma-driver/20220625-154524 HEAD e823cc5940ad1d20993113591a7ba26946ae0840 builds fine.
It only hurts bisectability.
All errors (new ones prefixed by >>):
drivers/dma/hisi_dma.c: In function 'hisi_dma_irq':
>> drivers/dma/hisi_dma.c:441:37: error: 'q_base' undeclared (first use
>> in this function)
441 | hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
| ^~~~~~
drivers/dma/hisi_dma.c:441:37: note: each undeclared identifier is reported only once for each function it appears in
>> drivers/dma/hisi_dma.c:441:45: error: 'HISI_DMA_Q_CQ_HEAD_PTR' undeclared (first use in this function); did you mean 'HISI_DMA_CQ_HEAD_PTR'?
441 | hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
| ^~~~~~~~~~~~~~~~~~~~~~
| HISI_DMA_CQ_HEAD_PTR
vim +/q_base +441 drivers/dma/hisi_dma.c
426
427 static irqreturn_t hisi_dma_irq(int irq, void *data)
428 {
429 struct hisi_dma_chan *chan = data;
430 struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
431 struct hisi_dma_desc *desc;
432 struct hisi_dma_cqe *cqe;
433
434 spin_lock(&chan->vc.lock);
435
436 desc = chan->desc;
437 cqe = chan->cq + chan->cq_head;
438 if (desc) {
439 chan->cq_head = (chan->cq_head + 1) %
440 hdma_dev->chan_depth;
> 441 hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
442 chan->qp_num, chan->cq_head);
443 if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
444 vchan_cookie_complete(&desc->vd);
445 } else {
446 dev_err(&hdma_dev->pdev->dev, "task error!\n");
447 }
448
449 chan->desc = NULL;
450 }
451
452 spin_unlock(&chan->vc.lock);
453
454 return IRQ_HANDLED;
455 }
456
--
0-DAY CI Kernel Test Service
https://01.org/lkp
^ permalink raw reply [flat|nested] 64+ messages in thread
* RE: [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update
2022-06-27 6:12 ` Vinod Koul
@ 2022-06-27 7:01 ` haijie
2022-06-27 17:38 ` Vinod Koul
0 siblings, 1 reply; 64+ messages in thread
From: haijie @ 2022-06-27 7:01 UTC (permalink / raw)
To: Vinod Koul; +Cc: Wangzhou (B), dmaengine, linux-kernel
Hi, Vinod,
This happens bacause I rearranged this patch without checking, it will be corrected in v2.
Thanks.
-----Original Message-----
From: Vinod Koul [mailto:vkoul@kernel.org]
Sent: Monday, June 27, 2022 2:13 PM
To: haijie <haijie1@huawei.com>
Cc: Wangzhou (B) <wangzhou1@hisilicon.com>; dmaengine@vger.kernel.org; linux-kernel@vger.kernel.org
Subject: Re: [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update
On 25-06-22, 15:44, Jie Hai wrote:
> After completion of data transfer of one or multiple descriptors, the
> completion status and the current head pointer to submission queue are
> written into the CQ and interrupt can be generated to inform the
> software. In interrupt process CQ is read and cq_head is updated.
>
> hisi_dma_irq updates cq_head only when the completion status is
> success. When an abnormal interrupt reports, cq_head will not update
> which will cause subsequent interrupt processes read the error CQ and
> never report the correct status.
>
> This patch updates cq_head whenever CQ is accessed.
>
> Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine
> support")
>
No need for blank line
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
> drivers/dma/hisi_dma.c | 9 ++++-----
> 1 file changed, 4 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c index
> 98bc488893cc..0a0f8a4d168a 100644
> --- a/drivers/dma/hisi_dma.c
> +++ b/drivers/dma/hisi_dma.c
> @@ -436,12 +436,11 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
> desc = chan->desc;
> cqe = chan->cq + chan->cq_head;
> if (desc) {
> + chan->cq_head = (chan->cq_head + 1) %
> + hdma_dev->chan_depth;
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
q_base?
--
~Vinod
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 3/8] dmaengine: hisilicon: Add multi-thread support for a DMA channel
[not found] ` <eed10a2b5ea745878024ca653158f4de@huawei.com>
@ 2022-06-27 13:14 ` haijie
0 siblings, 0 replies; 64+ messages in thread
From: haijie @ 2022-06-27 13:14 UTC (permalink / raw)
To: Vinod Koul; +Cc: Wangzhou (B), dmaengine, linux-kernel
在 2022/6/27 14:51, - 写道:
> -----Original Message-----
> From: Vinod Koul [mailto:vkoul@kernel.org]
> Sent: Monday, June 27, 2022 2:21 PM
> To: haijie <haijie1@huawei.com>
> Cc: Wangzhou (B) <wangzhou1@hisilicon.com>; dmaengine@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH 3/8] dmaengine: hisilicon: Add multi-thread support for a DMA channel
>
> On 25-06-22, 15:44, Jie Hai wrote:
>> When we get a DMA channel and try to use it in multiple threads it
>> will cause oops and hanging the system.
>>
>> % echo 100 > /sys/module/dmatest/parameters/threads_per_chan
>> % echo 100 > /sys/module/dmatest/parameters/iterations
>> % echo 1 > /sys/module/dmatest/parameters/run
>> [383493.327077] Unable to handle kernel paging request at virtual
>> address dead000000000108
>> [383493.335103] Mem abort info:
>> [383493.335103] ESR = 0x96000044
>> [383493.335105] EC = 0x25: DABT (current EL), IL = 32 bits
>> [383493.335107] SET = 0, FnV = 0
>> [383493.335108] EA = 0, S1PTW = 0
>> [383493.335109] FSC = 0x04: level 0 translation fault
>> [383493.335110] Data abort info:
>> [383493.335111] ISV = 0, ISS = 0x00000044
>> [383493.364739] CM = 0, WnR = 1
>> [383493.367793] [dead000000000108] address between user and kernel
>> address ranges
>> [383493.375021] Internal error: Oops: 96000044 [#1] PREEMPT SMP
>> [383493.437574] CPU: 63 PID: 27895 Comm: dma0chan0-copy2 Kdump:
>> loaded Tainted: GO 5.17.0-rc4+ #2
>> [383493.457851] pstate: 204000c9 (nzCv daIF +PAN -UAO -TCO -DIT
>> -SSBS BTYPE=--)
>> [383493.465331] pc : vchan_tx_submit+0x64/0xa0 [383493.469957] lr :
>> vchan_tx_submit+0x34/0xa0
>>
>> This happens because of data race. Each thread rewrite channels's
>> descriptor as soon as device_issue_pending is called. It leads to the
>> situation that the driver thinks that it uses the right descriptor in
>> interrupt handler while channels's descriptor has been changed by
>> other thread.
>>
>> With current fixes channels's descriptor changes it's value only when
>> it has been used. A new descriptor is acquired from
>> vc->desc_issued queue that is already filled with descriptors
>> that are ready to be sent. Threads have no direct access to DMA
>> channel descriptor. Now it is just possible to queue a descriptor for
>> further processing.
>>
>> Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine
>> support")
>> Signed-off-by: Jie Hai <haijie1@huawei.com>
>> ---
>> drivers/dma/hisi_dma.c | 6 ++----
>> 1 file changed, 2 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c index
>> 0a0f8a4d168a..0385419be8d5 100644
>> --- a/drivers/dma/hisi_dma.c
>> +++ b/drivers/dma/hisi_dma.c
>> @@ -271,7 +271,6 @@ static void hisi_dma_start_transfer(struct
>> hisi_dma_chan *chan)
>>
>> vd = vchan_next_desc(&chan->vc);
>> if (!vd) {
>> - dev_err(&hdma_dev->pdev->dev, "no issued task!\n");
> how is this a fix?
> With current fixes, hisi_dma_start_transfer may be called twice for one desc.
> If channel's desc is NULL, When hisi_dma_issue_pending
>> chan->desc = NULL;
>> return;
>> }
>> @@ -303,7 +302,7 @@ static void hisi_dma_issue_pending(struct dma_chan
>> *c)
>>
>> spin_lock_irqsave(&chan->vc.lock, flags);
>>
>> - if (vchan_issue_pending(&chan->vc))
>> + if (vchan_issue_pending(&chan->vc) && !chan->desc)
> This looks good
>
>> hisi_dma_start_transfer(chan);
>>
>> spin_unlock_irqrestore(&chan->vc.lock, flags); @@ -442,11 +441,10 @@
>> static irqreturn_t hisi_dma_irq(int irq, void *data)
>> chan->qp_num, chan->cq_head);
>> if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
>> vchan_cookie_complete(&desc->vd);
>> + hisi_dma_start_transfer(chan);
> Why should this fix the error reported?
>
>> } else {
>> dev_err(&hdma_dev->pdev->dev, "task error!\n");
>> }
>> -
>> - chan->desc = NULL;
>> }
>>
>> spin_unlock(&chan->vc.lock);
>> --
>> 2.33.0
> --
> ~Vinod
> .
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update
2022-06-27 7:01 ` haijie
@ 2022-06-27 17:38 ` Vinod Koul
0 siblings, 0 replies; 64+ messages in thread
From: Vinod Koul @ 2022-06-27 17:38 UTC (permalink / raw)
To: haijie; +Cc: Wangzhou (B), dmaengine, linux-kernel
On 27-06-22, 07:01, haijie wrote:
> Hi, Vinod,
>
> This happens bacause I rearranged this patch without checking, it will be corrected in v2.
Please _do_ _not_ top post and reply inline to the queries. Otherwise it
is very difficult to understand...
>
> Thanks.
>
> -----Original Message-----
> From: Vinod Koul [mailto:vkoul@kernel.org]
> Sent: Monday, June 27, 2022 2:13 PM
> To: haijie <haijie1@huawei.com>
> Cc: Wangzhou (B) <wangzhou1@hisilicon.com>; dmaengine@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: Re: [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update
>
> On 25-06-22, 15:44, Jie Hai wrote:
> > After completion of data transfer of one or multiple descriptors, the
> > completion status and the current head pointer to submission queue are
> > written into the CQ and interrupt can be generated to inform the
> > software. In interrupt process CQ is read and cq_head is updated.
> >
> > hisi_dma_irq updates cq_head only when the completion status is
> > success. When an abnormal interrupt reports, cq_head will not update
> > which will cause subsequent interrupt processes read the error CQ and
> > never report the correct status.
> >
> > This patch updates cq_head whenever CQ is accessed.
> >
> > Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine
> > support")
> >
>
> No need for blank line
> > Signed-off-by: Jie Hai <haijie1@huawei.com>
> > ---
> > drivers/dma/hisi_dma.c | 9 ++++-----
> > 1 file changed, 4 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c index
> > 98bc488893cc..0a0f8a4d168a 100644
> > --- a/drivers/dma/hisi_dma.c
> > +++ b/drivers/dma/hisi_dma.c
> > @@ -436,12 +436,11 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
> > desc = chan->desc;
> > cqe = chan->cq + chan->cq_head;
> > if (desc) {
> > + chan->cq_head = (chan->cq_head + 1) %
> > + hdma_dev->chan_depth;
> > + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
>
> q_base?
>
> --
> ~Vinod
--
~Vinod
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH 3/8] dmaengine: hisilicon: Add multi-thread support for a DMA channel
2022-06-27 6:21 ` Vinod Koul
[not found] ` <eed10a2b5ea745878024ca653158f4de@huawei.com>
@ 2022-06-29 3:44 ` Jie Hai
1 sibling, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-29 3:44 UTC (permalink / raw)
To: Vinod Koul; +Cc: wangzhou1, dmaengine, linux-kernel
Hi, Vkoul,
Thank you very much for your review. For a detailed explanation, see below.
On 27-06-22, 14:21, Vinod Koulwrote:
> On 25-06-22, 15:44, Jie Hai wrote:
>> When we get a DMA channel and try to use it in multiple threads it
>> will cause oops and hanging the system.
>>
>> % echo 100 > /sys/module/dmatest/parameters/threads_per_chan
>> % echo 100 > /sys/module/dmatest/parameters/iterations
>> % echo 1 > /sys/module/dmatest/parameters/run
>> [383493.327077] Unable to handle kernel paging request at virtual
>> address dead000000000108
>> [383493.335103] Mem abort info:
>> [383493.335103] ESR = 0x96000044
>> [383493.335105] EC = 0x25: DABT (current EL), IL = 32 bits
>> [383493.335107] SET = 0, FnV = 0
>> [383493.335108] EA = 0, S1PTW = 0
>> [383493.335109] FSC = 0x04: level 0 translation fault
>> [383493.335110] Data abort info:
>> [383493.335111] ISV = 0, ISS = 0x00000044
>> [383493.364739] CM = 0, WnR = 1
>> [383493.367793] [dead000000000108] address between user and kernel
>> address ranges
>> [383493.375021] Internal error: Oops: 96000044 [#1] PREEMPT SMP
>> [383493.437574] CPU: 63 PID: 27895 Comm: dma0chan0-copy2 Kdump:
>> loaded Tainted: GO 5.17.0-rc4+ #2
>> [383493.457851] pstate: 204000c9 (nzCv daIF +PAN -UAO -TCO -DIT
>> -SSBS BTYPE=--)
>> [383493.465331] pc : vchan_tx_submit+0x64/0xa0
>> [383493.469957] lr : vchan_tx_submit+0x34/0xa0
>>
>> This happens because of data race. Each thread rewrite channels's
>> descriptor as soon as device_issue_pending is called. It leads to
>> the situation that the driver thinks that it uses the right
>> descriptor in interrupt handler while channels's descriptor has
>> been changed by other thread.
>>
>> With current fixes channels's descriptor changes it's value only
>> when it has been used. A new descriptor is acquired from
>> vc->desc_issued queue that is already filled with descriptors
>> that are ready to be sent. Threads have no direct access to DMA
>> channel descriptor. Now it is just possible to queue a descriptor
>> for further processing.
>>
>> Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
>> Signed-off-by: Jie Hai <haijie1@huawei.com>
>> ---
>> drivers/dma/hisi_dma.c | 6 ++----
>> 1 file changed, 2 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
>> index 0a0f8a4d168a..0385419be8d5 100644
>> --- a/drivers/dma/hisi_dma.c
>> +++ b/drivers/dma/hisi_dma.c
>> @@ -271,7 +271,6 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
>>
>> vd = vchan_next_desc(&chan->vc);
>> if (!vd) {
>> - dev_err(&hdma_dev->pdev->dev, "no issued task!\n");
>
> how is this a fix?
>
Consider that only one deccriptor is in progress, and it is submitted to
hardware successfully in hisi_dma_issue_pending. When hisi_dma_irq calls
hisi_dma_start_transfer, vd is absolutely NULL. This also occurs in
multi-descriptor transfers. It's not abnormal that vd is NULL. So it's
reasonable to delete the error reporting.
>> chan->desc = NULL;
>> return;
>> }
>> @@ -303,7 +302,7 @@ static void hisi_dma_issue_pending(struct dma_chan *c)
>>
>> spin_lock_irqsave(&chan->vc.lock, flags);
>>
>> - if (vchan_issue_pending(&chan->vc))
>> + if (vchan_issue_pending(&chan->vc) && !chan->desc)
>
> This looks good
>
>> hisi_dma_start_transfer(chan);
>>
>> spin_unlock_irqrestore(&chan->vc.lock, flags);
>> @@ -442,11 +441,10 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
>> chan->qp_num, chan->cq_head);
>> if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
>> vchan_cookie_complete(&desc->vd);
>> + hisi_dma_start_transfer(chan);
>
> Why should this fix the error reported?
>
With current fix in hisi_dma_issue_pending, if chan->desc is not NULL,
this submission to hardware is invalid and it will try again when
chan->desc is completed, which ensures that all descriptors are submitted
to hardware.
As to the error reported, it is transfering timeout that causes the error
handling branch in dmatest, then the error reported. The overwritten of
chan->desc in multi-thread lead to that some descriptors are not be
processed by hisi_dma_irq, not to mention their callback. This fixes
timeout problem of hisi_dma and does not enter the error handling branch
of dmatest.
Dmatest uses dmaengine_terminate_sync to handle abnomal situation. It
calles device_terminate_all, most drivers implement this function with
vchan_get_all_descriptors and a temporary list head. It gets all descriptors
the channel holds in lists desc_* and adds them to head, deletions and
releases of these descriptors are performed on head without lock.
In the multi-thread scenario, a descriptor A which has not been submitted
by tx_submit may be in the following situations:
a). desc_A is in the desc_allocated list.
b). desc_A is in the head list of thread t2.
c). desc_A has been deleted from the head list by t2 but has not been freed.
d). desc_A has been deleted from the head list and freed by t2.
If there is a thread t1 attempting to call tx_submit for desc_A
on the preceding conditions, no error will be reported for a) and b), and
d) will cause use-after-free. Now consider c), s2 and c) are all involved
in removing nodes from the list. When a node is deleted from the list by
__list_del_entry, the previous and next node are assigned the constant
pointer LIST_POISON1 and LIST_POISON2, respectively. Accessing the two
addresses will cause an error. Therefore, if you perform __list_del_entry
on a node twice consecutively, an error will report. This is the case of
c). The preceding calltrace is caused by this.
I don't think it's wise for dmatest to use dmaengine_terminate_sync
to handle errors, but we do have problems with our driver. This patch is
to fix hisi_dma.
>> } else {
>> dev_err(&hdma_dev->pdev->dev, "task error!\n");
>> }
>> -
>> - chan->desc = NULL;
>> }
>>
>> spin_unlock(&chan->vc.lock);
>> --
>> 2.33.0
>
Thanks,
Jie Hai.
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (7 preceding siblings ...)
2022-06-25 7:44 ` [PATCH 8/8] MAINTAINERS: Add debugfs files and maintainer " Jie Hai
@ 2022-06-29 3:55 ` Jie Hai
2022-06-29 3:55 ` [PATCH v2 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
` (7 more replies)
2022-07-26 1:35 ` [PATCH v3 " Jie Hai
` (2 subsequent siblings)
11 siblings, 8 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-29 3:55 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they share the
same pci device id but different pci revision and register layouts.
The original version supports HiSilicon IP08 but not HiSilicon IP09.
This series support DMA driver for HIP08 and HIP09:
1. Fix bugs for HIP08 DMA driver
- Disable hardware channels when driver detached
- Update cq_head whenever accessed it
- Support multi-thread for one DMA channel
2. Use macros instead of magic number
3. Add support for HIP09 DMA driver
4. Dump registers for HIP08 and HIP09 DMA driver with debugfs
5. Add myself as maintainer of hisi_dma.c
Changes since version 1:
- fix compile failure reported by kernel test robot
- fix reduldant "*" in comment
- fix reduldant blank line in commit log
- remove debugfs-hisi-dma doc and path in MAINTAINERS
- add more explanations in patch 3/7
Jie Hai (7):
dmaengine: hisilicon: Disable channels when unregister hisi_dma
dmaengine: hisilicon: Fix CQ head update
dmaengine: hisilicon: Add multi-thread support for a DMA channel
dmaengine: hisilicon: Use macros instead of magic number
dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
dmaengine: hisilicon: Add dfx feature for hisi dma driver
MAINTAINERS: Add myself as maintainer for hisi_dma
MAINTAINERS | 1 +
drivers/dma/hisi_dma.c | 730 +++++++++++++++++++++++++++++++++++------
2 files changed, 635 insertions(+), 96 deletions(-)
--
2.33.0
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v2 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma
2022-06-29 3:55 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
@ 2022-06-29 3:55 ` Jie Hai
2022-06-29 3:55 ` [PATCH v2 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
` (6 subsequent siblings)
7 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-29 3:55 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
When hisi_dma is unloaded or unbinded, all of channels should be
disabled. This patch disables DMA channels when driver is unloaded
or unbinded.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 43817ced3a3e..98bc488893cc 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -180,7 +180,8 @@ static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
}
-static void hisi_dma_reset_hw_chan(struct hisi_dma_chan *chan)
+static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
+ bool disable)
{
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
u32 index = chan->qp_num, tmp;
@@ -201,8 +202,11 @@ static void hisi_dma_reset_hw_chan(struct hisi_dma_chan *chan)
hisi_dma_do_reset(hdma_dev, index);
hisi_dma_reset_qp_point(hdma_dev, index);
hisi_dma_pause_dma(hdma_dev, index, false);
- hisi_dma_enable_dma(hdma_dev, index, true);
- hisi_dma_unmask_irq(hdma_dev, index);
+
+ if (!disable) {
+ hisi_dma_enable_dma(hdma_dev, index, true);
+ hisi_dma_unmask_irq(hdma_dev, index);
+ }
ret = readl_relaxed_poll_timeout(hdma_dev->base +
HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
@@ -218,7 +222,7 @@ static void hisi_dma_free_chan_resources(struct dma_chan *c)
struct hisi_dma_chan *chan = to_hisi_dma_chan(c);
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
- hisi_dma_reset_hw_chan(chan);
+ hisi_dma_reset_or_disable_hw_chan(chan, false);
vchan_free_chan_resources(&chan->vc);
memset(chan->sq, 0, sizeof(struct hisi_dma_sqe) * hdma_dev->chan_depth);
@@ -394,7 +398,7 @@ static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
static void hisi_dma_disable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- hisi_dma_reset_hw_chan(&hdma_dev->chan[qp_index]);
+ hisi_dma_reset_or_disable_hw_chan(&hdma_dev->chan[qp_index], true);
}
static void hisi_dma_enable_qps(struct hisi_dma_dev *hdma_dev)
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 2/7] dmaengine: hisilicon: Fix CQ head update
2022-06-29 3:55 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-06-29 3:55 ` [PATCH v2 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
@ 2022-06-29 3:55 ` Jie Hai
2022-07-21 13:27 ` Vinod Koul
2022-06-29 3:55 ` [PATCH v2 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
` (5 subsequent siblings)
7 siblings, 1 reply; 64+ messages in thread
From: Jie Hai @ 2022-06-29 3:55 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
After completion of data transfer of one or multiple descriptors,
the completion status and the current head pointer to submission
queue are written into the CQ and interrupt can be generated to
inform the software. In interrupt process CQ is read and cq_head
is updated.
hisi_dma_irq updates cq_head only when the completion status is
success. When an abnormal interrupt reports, cq_head will not update
which will cause subsequent interrupt processes read the error CQ
and never report the correct status.
This patch updates cq_head whenever CQ is accessed.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 98bc488893cc..7609e6e7eb37 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -436,12 +436,12 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
desc = chan->desc;
cqe = chan->cq + chan->cq_head;
if (desc) {
+ chan->cq_head = (chan->cq_head + 1) %
+ hdma_dev->chan_depth;
+ hisi_dma_chan_write(hdma_dev->base,
+ HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
+ chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
- chan->cq_head = (chan->cq_head + 1) %
- hdma_dev->chan_depth;
- hisi_dma_chan_write(hdma_dev->base,
- HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
- chan->cq_head);
vchan_cookie_complete(&desc->vd);
} else {
dev_err(&hdma_dev->pdev->dev, "task error!\n");
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel
2022-06-29 3:55 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-06-29 3:55 ` [PATCH v2 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
2022-06-29 3:55 ` [PATCH v2 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
@ 2022-06-29 3:55 ` Jie Hai
2022-06-29 3:55 ` [PATCH v2 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
` (4 subsequent siblings)
7 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-29 3:55 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
When we get a DMA channel and try to use it in multiple threads it
will cause oops and hanging the system.
% echo 100 > /sys/module/dmatest/parameters/threads_per_chan
% echo 100 > /sys/module/dmatest/parameters/iterations
% echo 1 > /sys/module/dmatest/parameters/run
[383493.327077] Unable to handle kernel paging request at virtual
address dead000000000108
[383493.335103] Mem abort info:
[383493.335103] ESR = 0x96000044
[383493.335105] EC = 0x25: DABT (current EL), IL = 32 bits
[383493.335107] SET = 0, FnV = 0
[383493.335108] EA = 0, S1PTW = 0
[383493.335109] FSC = 0x04: level 0 translation fault
[383493.335110] Data abort info:
[383493.335111] ISV = 0, ISS = 0x00000044
[383493.364739] CM = 0, WnR = 1
[383493.367793] [dead000000000108] address between user and kernel
address ranges
[383493.375021] Internal error: Oops: 96000044 [#1] PREEMPT SMP
[383493.437574] CPU: 63 PID: 27895 Comm: dma0chan0-copy2 Kdump:
loaded Tainted: GO 5.17.0-rc4+ #2
[383493.457851] pstate: 204000c9 (nzCv daIF +PAN -UAO -TCO -DIT
-SSBS BTYPE=--)
[383493.465331] pc : vchan_tx_submit+0x64/0xa0
[383493.469957] lr : vchan_tx_submit+0x34/0xa0
This occurs because the transmission timed out, and that's due
to data race. Each thread rewrite channels's descriptor as soon as
device_issue_pending is called. It leads to the situation that
the driver thinks that it uses the right descriptor in interrupt
handler while channels's descriptor has been changed by other
thread. The descriptor which in fact reported interrupt will not
be handled any more, as well as its tx->callback.
That's why timeout reports.
With current fixes channels' descriptor changes it's value only
when it has been used. A new descriptor is acquired from
vc->desc_issued queue that is already filled with descriptors
that are ready to be sent. Threads have no direct access to DMA
channel descriptor. In case of channel's descriptor is busy, try
to submit to HW again when a descriptor is completed. In this case,
vc->desc_issued may be empty when hisi_dma_start_transfer is called,
so delete error reporting on this. Now it is just possible to queue
a descriptor for further processing.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 7609e6e7eb37..a3ccaf58be80 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -271,7 +271,6 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
vd = vchan_next_desc(&chan->vc);
if (!vd) {
- dev_err(&hdma_dev->pdev->dev, "no issued task!\n");
chan->desc = NULL;
return;
}
@@ -303,7 +302,7 @@ static void hisi_dma_issue_pending(struct dma_chan *c)
spin_lock_irqsave(&chan->vc.lock, flags);
- if (vchan_issue_pending(&chan->vc))
+ if (vchan_issue_pending(&chan->vc) && !chan->desc)
hisi_dma_start_transfer(chan);
spin_unlock_irqrestore(&chan->vc.lock, flags);
@@ -443,11 +442,10 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
vchan_cookie_complete(&desc->vd);
+ hisi_dma_start_transfer(chan);
} else {
dev_err(&hdma_dev->pdev->dev, "task error!\n");
}
-
- chan->desc = NULL;
}
spin_unlock(&chan->vc.lock);
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 4/7] dmaengine: hisilicon: Use macros instead of magic number
2022-06-29 3:55 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (2 preceding siblings ...)
2022-06-29 3:55 ` [PATCH v2 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
@ 2022-06-29 3:55 ` Jie Hai
2022-06-29 3:55 ` [PATCH v2 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
` (3 subsequent siblings)
7 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-29 3:55 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
readl_relaxed_poll_timeout() uses magic numbers 10 and 1000, which
indicate maximum time to sleep between reads in us and timeout in us,
respectively.
Use macros HISI_DMA_POLL_Q_STS_DELAY_US and
HISI_DMA_POLL_Q_STS_TIME_OUT_US instead of these two numbers.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 21 +++++++++++++++------
1 file changed, 15 insertions(+), 6 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index a3ccaf58be80..d5f87ef4a5ee 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -36,6 +36,9 @@
#define PCI_BAR_2 2
+#define HISI_DMA_POLL_Q_STS_DELAY_US 10
+#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+
enum hisi_dma_mode {
EP = 0,
RC,
@@ -185,15 +188,20 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
{
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
u32 index = chan->qp_num, tmp;
+ void __iomem *addr;
int ret;
hisi_dma_pause_dma(hdma_dev, index, true);
hisi_dma_enable_dma(hdma_dev, index, false);
hisi_dma_mask_irq(hdma_dev, index);
- ret = readl_relaxed_poll_timeout(hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN, 10, 1000);
+ addr = hdma_dev->base +
+ HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET;
+
+ ret = readl_relaxed_poll_timeout(addr, tmp,
+ FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN,
+ HISI_DMA_POLL_Q_STS_DELAY_US,
+ HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "disable channel timeout!\n");
WARN_ON(1);
@@ -208,9 +216,10 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
hisi_dma_unmask_irq(hdma_dev, index);
}
- ret = readl_relaxed_poll_timeout(hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE, 10, 1000);
+ ret = readl_relaxed_poll_timeout(addr, tmp,
+ FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE,
+ HISI_DMA_POLL_Q_STS_DELAY_US,
+ HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "reset channel timeout!\n");
WARN_ON(1);
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
2022-06-29 3:55 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (3 preceding siblings ...)
2022-06-29 3:55 ` [PATCH v2 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
@ 2022-06-29 3:55 ` Jie Hai
2022-07-21 13:29 ` Vinod Koul
2022-06-29 3:55 ` [PATCH v2 6/7] dmaengine: hisilicon: Add dfx feature for hisi dma driver Jie Hai
` (2 subsequent siblings)
7 siblings, 1 reply; 64+ messages in thread
From: Jie Hai @ 2022-06-29 3:55 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they
have the same pci device id but different pci revision.
Unfortunately, they have different register layouts, so
the origin driver cannot run on HiSilicon IP09 correctly.
This patch enables the driver to adapt to HiSilicon IP09.
HiSilicon IP09 offers 4 channels, each channel has a send
queue, a complete queue and an interrupt to help to do tasks.
This DMA engine can do memory copy between memory blocks.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 382 ++++++++++++++++++++++++++++++++---------
1 file changed, 299 insertions(+), 83 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index d5f87ef4a5ee..dc7c59b21114 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
-/* Copyright(c) 2019 HiSilicon Limited. */
+/* Copyright(c) 2019-2022 HiSilicon Limited. */
+
#include <linux/bitfield.h>
#include <linux/dmaengine.h>
#include <linux/init.h>
@@ -9,35 +10,85 @@
#include <linux/spinlock.h>
#include "virt-dma.h"
-#define HISI_DMA_SQ_BASE_L 0x0
-#define HISI_DMA_SQ_BASE_H 0x4
-#define HISI_DMA_SQ_DEPTH 0x8
-#define HISI_DMA_SQ_TAIL_PTR 0xc
-#define HISI_DMA_CQ_BASE_L 0x10
-#define HISI_DMA_CQ_BASE_H 0x14
-#define HISI_DMA_CQ_DEPTH 0x18
-#define HISI_DMA_CQ_HEAD_PTR 0x1c
-#define HISI_DMA_CTRL0 0x20
-#define HISI_DMA_CTRL0_QUEUE_EN_S 0
-#define HISI_DMA_CTRL0_QUEUE_PAUSE_S 4
-#define HISI_DMA_CTRL1 0x24
-#define HISI_DMA_CTRL1_QUEUE_RESET_S 0
-#define HISI_DMA_Q_FSM_STS 0x30
-#define HISI_DMA_FSM_STS_MASK GENMASK(3, 0)
-#define HISI_DMA_INT_STS 0x40
-#define HISI_DMA_INT_STS_MASK GENMASK(12, 0)
-#define HISI_DMA_INT_MSK 0x44
-#define HISI_DMA_MODE 0x217c
-#define HISI_DMA_OFFSET 0x100
-
-#define HISI_DMA_MSI_NUM 32
-#define HISI_DMA_CHAN_NUM 30
-#define HISI_DMA_Q_DEPTH_VAL 1024
-
-#define PCI_BAR_2 2
-
-#define HISI_DMA_POLL_Q_STS_DELAY_US 10
-#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+/* HiSilicon DMA register common field define */
+#define HISI_DMA_Q_SQ_BASE_L 0x0
+#define HISI_DMA_Q_SQ_BASE_H 0x4
+#define HISI_DMA_Q_SQ_DEPTH 0x8
+#define HISI_DMA_Q_SQ_TAIL_PTR 0xc
+#define HISI_DMA_Q_CQ_BASE_L 0x10
+#define HISI_DMA_Q_CQ_BASE_H 0x14
+#define HISI_DMA_Q_CQ_DEPTH 0x18
+#define HISI_DMA_Q_CQ_HEAD_PTR 0x1c
+#define HISI_DMA_Q_CTRL0 0x20
+#define HISI_DMA_Q_CTRL0_QUEUE_EN 0
+#define HISI_DMA_Q_CTRL0_QUEUE_PAUSE 4
+#define HISI_DMA_Q_CTRL1 0x24
+#define HISI_DMA_Q_CTRL1_QUEUE_RESET 0
+#define HISI_DMA_Q_FSM_STS 0x30
+#define HISI_DMA_Q_FSM_STS_MASK GENMASK(3, 0)
+#define HISI_DMA_Q_ERR_INT_NUM0 0x84
+#define HISI_DMA_Q_ERR_INT_NUM1 0x88
+#define HISI_DMA_Q_ERR_INT_NUM2 0x8c
+
+/* HiSilicon IP08 DMA register and field define */
+#define HISI_DMA_HIP08_MODE 0x217C
+#define HISI_DMA_HIP08_Q_BASE 0x0
+#define HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN 2
+#define HISI_DMA_HIP08_Q_INT_STS 0x40
+#define HISI_DMA_HIP08_Q_INT_MSK 0x44
+#define HISI_DMA_HIP08_Q_INT_STS_MASK GENMASK(14, 0)
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM3 0x90
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM4 0x94
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM5 0x98
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM6 0x48
+#define HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT 24
+
+/* HiSilicon IP09 DMA register and field define */
+#define HISI_DMA_HIP09_DMA_FLR_DISABLE 0xA00
+#define HISI_DMA_HIP09_DMA_FLR_DISABLE_B 0
+#define HISI_DMA_HIP09_Q_BASE 0x2000
+#define HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN GENMASK(31, 28)
+#define HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT 26
+#define HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT 27
+#define HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE 2
+#define HISI_DMA_HIP09_Q_INT_STS 0x40
+#define HISI_DMA_HIP09_Q_INT_MSK 0x44
+#define HISI_DMA_HIP09_Q_INT_STS_MASK 0x1
+#define HISI_DMA_HIP09_Q_ERR_INT_STS 0x48
+#define HISI_DMA_HIP09_Q_ERR_INT_MSK 0x4C
+#define HISI_DMA_HIP09_Q_ERR_INT_STS_MASK GENMASK(18, 1)
+#define HISI_DMA_HIP09_PORT_CFG_REG(port_id) (0x800 + \
+ (port_id) * 0x20)
+#define HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B 16
+
+#define HISI_DMA_HIP09_MAX_PORT_NUM 16
+
+#define HISI_DMA_HIP08_MSI_NUM 32
+#define HISI_DMA_HIP08_CHAN_NUM 30
+#define HISI_DMA_HIP09_MSI_NUM 4
+#define HISI_DMA_HIP09_CHAN_NUM 4
+#define HISI_DMA_REVISION_HIP08B 0x21
+#define HISI_DMA_REVISION_HIP09A 0x30
+
+#define HISI_DMA_Q_OFFSET 0x100
+#define HISI_DMA_Q_DEPTH_VAL 1024
+
+#define PCI_BAR_2 2
+
+#define HISI_DMA_POLL_Q_STS_DELAY_US 10
+#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+
+/*
+ * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
+ * have the same pci device id but different pci revision.
+ * Unfortunately, they have different register layouts, so two layout
+ * enumerations are defined.
+ */
+enum hisi_dma_reg_layout {
+ HISI_DMA_REG_LAYOUT_INVALID = 0,
+ HISI_DMA_REG_LAYOUT_HIP08,
+ HISI_DMA_REG_LAYOUT_HIP09
+};
enum hisi_dma_mode {
EP = 0,
@@ -108,9 +159,45 @@ struct hisi_dma_dev {
struct dma_device dma_dev;
u32 chan_num;
u32 chan_depth;
+ enum hisi_dma_reg_layout reg_layout;
+ void __iomem *queue_base; /* queue region start of register */
struct hisi_dma_chan chan[];
};
+static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_REG_LAYOUT_HIP08;
+ else if (pdev->revision >= HISI_DMA_REVISION_HIP09A)
+ return HISI_DMA_REG_LAYOUT_HIP09;
+
+ return HISI_DMA_REG_LAYOUT_INVALID;
+}
+
+static u32 hisi_dma_get_chan_num(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_CHAN_NUM;
+
+ return HISI_DMA_HIP09_CHAN_NUM;
+}
+
+static u32 hisi_dma_get_msi_num(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_MSI_NUM;
+
+ return HISI_DMA_HIP09_MSI_NUM;
+}
+
+static u32 hisi_dma_get_queue_base(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_Q_BASE;
+
+ return HISI_DMA_HIP09_Q_BASE;
+}
+
static inline struct hisi_dma_chan *to_hisi_dma_chan(struct dma_chan *c)
{
return container_of(c, struct hisi_dma_chan, vc.chan);
@@ -124,7 +211,7 @@ static inline struct hisi_dma_desc *to_hisi_dma_desc(struct virt_dma_desc *vd)
static inline void hisi_dma_chan_write(void __iomem *base, u32 reg, u32 index,
u32 val)
{
- writel_relaxed(val, base + reg + index * HISI_DMA_OFFSET);
+ writel_relaxed(val, base + reg + index * HISI_DMA_Q_OFFSET);
}
static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
@@ -139,48 +226,76 @@ static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
static void hisi_dma_pause_dma(struct hisi_dma_dev *hdma_dev, u32 index,
bool pause)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_PAUSE_S, pause);
+ addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_PAUSE, pause);
}
static void hisi_dma_enable_dma(struct hisi_dma_dev *hdma_dev, u32 index,
bool enable)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_EN_S, enable);
+ addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_EN, enable);
}
static void hisi_dma_mask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_INT_MSK, qp_index,
- HISI_DMA_INT_STS_MASK);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
+ qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
+ else {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
+ qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
+ qp_index,
+ HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
+ }
}
static void hisi_dma_unmask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- void __iomem *base = hdma_dev->base;
-
- hisi_dma_chan_write(base, HISI_DMA_INT_STS, qp_index,
- HISI_DMA_INT_STS_MASK);
- hisi_dma_chan_write(base, HISI_DMA_INT_MSK, qp_index, 0);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_STS,
+ qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
+ qp_index, 0);
+ } else {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_STS,
+ qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_STS,
+ qp_index,
+ HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
+ qp_index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
+ qp_index, 0);
+ }
}
static void hisi_dma_do_reset(struct hisi_dma_dev *hdma_dev, u32 index)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL1 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL1_QUEUE_RESET_S, 1);
+ addr = hdma_dev->queue_base +
+ HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL1_QUEUE_RESET, 1);
}
static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
{
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, index, 0);
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
}
static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
@@ -195,11 +310,11 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
hisi_dma_enable_dma(hdma_dev, index, false);
hisi_dma_mask_irq(hdma_dev, index);
- addr = hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET;
+ addr = hdma_dev->queue_base +
+ HISI_DMA_Q_FSM_STS + index * HISI_DMA_Q_OFFSET;
ret = readl_relaxed_poll_timeout(addr, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN,
+ FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) != RUN,
HISI_DMA_POLL_Q_STS_DELAY_US,
HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
@@ -217,7 +332,7 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
}
ret = readl_relaxed_poll_timeout(addr, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE,
+ FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) == IDLE,
HISI_DMA_POLL_Q_STS_DELAY_US,
HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
@@ -300,8 +415,8 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
chan->sq_tail = (chan->sq_tail + 1) % hdma_dev->chan_depth;
/* update sq_tail to trigger a new task */
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, chan->qp_num,
- chan->sq_tail);
+ hisi_dma_chan_write(hdma_dev->queue_base, HISI_DMA_Q_SQ_TAIL_PTR,
+ chan->qp_num, chan->sq_tail);
}
static void hisi_dma_issue_pending(struct dma_chan *c)
@@ -375,26 +490,86 @@ static int hisi_dma_alloc_qps_mem(struct hisi_dma_dev *hdma_dev)
static void hisi_dma_init_hw_qp(struct hisi_dma_dev *hdma_dev, u32 index)
{
struct hisi_dma_chan *chan = &hdma_dev->chan[index];
+ void __iomem *q_base = hdma_dev->queue_base;
u32 hw_depth = hdma_dev->chan_depth - 1;
- void __iomem *base = hdma_dev->base;
+ void __iomem *addr;
+ u32 tmp;
/* set sq, cq base */
- hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_L, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_L, index,
lower_32_bits(chan->sq_dma));
- hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_H, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_H, index,
upper_32_bits(chan->sq_dma));
- hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_L, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_L, index,
lower_32_bits(chan->cq_dma));
- hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_H, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_H, index,
upper_32_bits(chan->cq_dma));
/* set sq, cq depth */
- hisi_dma_chan_write(base, HISI_DMA_SQ_DEPTH, index, hw_depth);
- hisi_dma_chan_write(base, HISI_DMA_CQ_DEPTH, index, hw_depth);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_DEPTH, index, hw_depth);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_DEPTH, index, hw_depth);
/* init sq tail and cq head */
- hisi_dma_chan_write(base, HISI_DMA_SQ_TAIL_PTR, index, 0);
- hisi_dma_chan_write(base, HISI_DMA_CQ_HEAD_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
+
+ /* init error interrupt stats */
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM0, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM1, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM2, index, 0);
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM3,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM4,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM5,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM6,
+ index, 0);
+ /*
+ * init SQ/CQ direction selecting register.
+ * "0" is to local side and "1" is to remote side.
+ */
+ addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT, 0);
+
+ /*
+ * 0 - Continue to next descriptor if error occurs.
+ * 1 - Abort the DMA queue if error occurs.
+ */
+ hisi_dma_update_bit(addr,
+ HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN, 0);
+ } else {
+ addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
+
+ /*
+ * init SQ/CQ direction selecting register.
+ * "0" is to local side and "1" is to remote side.
+ */
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT, 0);
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT, 0);
+
+ /*
+ * 0 - Continue to next descriptor if error occurs.
+ * 1 - Abort the DMA queue if error occurs.
+ */
+
+ tmp = readl_relaxed(addr);
+ tmp &= ~HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN;
+ writel_relaxed(tmp, addr);
+
+ /*
+ * 0 - dma should process FLR whith CPU.
+ * 1 - dma not process FLR, only cpu process FLR.
+ */
+ addr = q_base + HISI_DMA_HIP09_DMA_FLR_DISABLE +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_DMA_FLR_DISABLE_B, 0);
+
+ addr = q_base + HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE, 1);
+ }
}
static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
@@ -438,17 +613,18 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
struct hisi_dma_desc *desc;
struct hisi_dma_cqe *cqe;
+ void __iomem *q_base;
spin_lock(&chan->vc.lock);
desc = chan->desc;
cqe = chan->cq + chan->cq_head;
+ q_base = hdma_dev->queue_base;
if (desc) {
chan->cq_head = (chan->cq_head + 1) %
hdma_dev->chan_depth;
- hisi_dma_chan_write(hdma_dev->base,
- HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
- chan->cq_head);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
+ chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
vchan_cookie_complete(&desc->vd);
hisi_dma_start_transfer(chan);
@@ -508,16 +684,58 @@ static void hisi_dma_disable_hw_channels(void *data)
static void hisi_dma_set_mode(struct hisi_dma_dev *hdma_dev,
enum hisi_dma_mode mode)
{
- writel_relaxed(mode == RC ? 1 : 0, hdma_dev->base + HISI_DMA_MODE);
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ writel_relaxed(mode == RC ? 1 : 0,
+ hdma_dev->base + HISI_DMA_HIP08_MODE);
+}
+
+static void hisi_dma_init_hw(struct hisi_dma_dev *hdma_dev)
+{
+ void __iomem *addr;
+ int i;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP09) {
+ for (i = 0; i < HISI_DMA_HIP09_MAX_PORT_NUM; i++) {
+ addr = hdma_dev->base + HISI_DMA_HIP09_PORT_CFG_REG(i);
+ hisi_dma_update_bit(addr,
+ HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B, 1);
+ }
+ }
+}
+
+static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
+{
+ struct dma_device *dma_dev;
+
+ dma_dev = &hdma_dev->dma_dev;
+ dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
+ dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
+ dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
+ dma_dev->device_tx_status = hisi_dma_tx_status;
+ dma_dev->device_issue_pending = hisi_dma_issue_pending;
+ dma_dev->device_terminate_all = hisi_dma_terminate_all;
+ dma_dev->device_synchronize = hisi_dma_synchronize;
+ dma_dev->directions = BIT(DMA_MEM_TO_MEM);
+ dma_dev->dev = &hdma_dev->pdev->dev;
+ INIT_LIST_HEAD(&dma_dev->channels);
}
static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
+ enum hisi_dma_reg_layout reg_layout;
struct device *dev = &pdev->dev;
struct hisi_dma_dev *hdma_dev;
struct dma_device *dma_dev;
+ u32 chan_num;
+ u32 msi_num;
int ret;
+ reg_layout = hisi_dma_get_reg_layout(pdev);
+ if (reg_layout == HISI_DMA_REG_LAYOUT_INVALID) {
+ dev_err(dev, "unsupported device!\n");
+ return -EINVAL;
+ }
+
ret = pcim_enable_device(pdev);
if (ret) {
dev_err(dev, "failed to enable device mem!\n");
@@ -534,40 +752,37 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (ret)
return ret;
- hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, HISI_DMA_CHAN_NUM), GFP_KERNEL);
+ chan_num = hisi_dma_get_chan_num(pdev);
+ hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, chan_num),
+ GFP_KERNEL);
if (!hdma_dev)
return -EINVAL;
hdma_dev->base = pcim_iomap_table(pdev)[PCI_BAR_2];
hdma_dev->pdev = pdev;
- hdma_dev->chan_num = HISI_DMA_CHAN_NUM;
hdma_dev->chan_depth = HISI_DMA_Q_DEPTH_VAL;
+ hdma_dev->chan_num = chan_num;
+ hdma_dev->reg_layout = reg_layout;
+ hdma_dev->queue_base = hdma_dev->base + hisi_dma_get_queue_base(pdev);
pci_set_drvdata(pdev, hdma_dev);
pci_set_master(pdev);
+ msi_num = hisi_dma_get_msi_num(pdev);
+
/* This will be freed by 'pcim_release()'. See 'pcim_enable_device()' */
- ret = pci_alloc_irq_vectors(pdev, HISI_DMA_MSI_NUM, HISI_DMA_MSI_NUM,
- PCI_IRQ_MSI);
+ ret = pci_alloc_irq_vectors(pdev, msi_num, msi_num, PCI_IRQ_MSI);
if (ret < 0) {
dev_err(dev, "Failed to allocate MSI vectors!\n");
return ret;
}
- dma_dev = &hdma_dev->dma_dev;
- dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
- dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
- dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
- dma_dev->device_tx_status = hisi_dma_tx_status;
- dma_dev->device_issue_pending = hisi_dma_issue_pending;
- dma_dev->device_terminate_all = hisi_dma_terminate_all;
- dma_dev->device_synchronize = hisi_dma_synchronize;
- dma_dev->directions = BIT(DMA_MEM_TO_MEM);
- dma_dev->dev = dev;
- INIT_LIST_HEAD(&dma_dev->channels);
+ hisi_dma_init_dma_dev(hdma_dev);
hisi_dma_set_mode(hdma_dev, RC);
+ hisi_dma_init_hw(hdma_dev);
+
ret = hisi_dma_enable_hw_channels(hdma_dev);
if (ret < 0) {
dev_err(dev, "failed to enable hw channel!\n");
@@ -579,6 +794,7 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (ret)
return ret;
+ dma_dev = &hdma_dev->dma_dev;
ret = dmaenginem_async_device_register(dma_dev);
if (ret < 0)
dev_err(dev, "failed to register device!\n");
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 6/7] dmaengine: hisilicon: Add dfx feature for hisi dma driver
2022-06-29 3:55 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (4 preceding siblings ...)
2022-06-29 3:55 ` [PATCH v2 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
@ 2022-06-29 3:55 ` Jie Hai
2022-07-21 13:25 ` Vinod Koul
2022-06-29 3:55 ` [PATCH v2 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
2022-07-08 3:16 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Zhou Wang
7 siblings, 1 reply; 64+ messages in thread
From: Jie Hai @ 2022-06-29 3:55 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
This patch adds dump of registers with debugfs for HIP08
and HIP09 DMA driver.
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 317 ++++++++++++++++++++++++++++++++++++++++-
1 file changed, 314 insertions(+), 3 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index dc7c59b21114..b0bc7b18933b 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -78,6 +78,8 @@
#define HISI_DMA_POLL_Q_STS_DELAY_US 10
#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+#define HISI_DMA_MAX_DIR_NAME_LEN 128
+
/*
* The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
* have the same pci device id but different pci revision.
@@ -161,9 +163,131 @@ struct hisi_dma_dev {
u32 chan_depth;
enum hisi_dma_reg_layout reg_layout;
void __iomem *queue_base; /* queue region start of register */
+#ifdef CONFIG_DEBUG_FS
+ struct dentry *dbg_hdev_root;
+ struct dentry **dbg_chans;
+#endif
struct hisi_dma_chan chan[];
};
+#ifdef CONFIG_DEBUG_FS
+struct dentry *hisi_dma_debugfs_root;
+
+static const struct debugfs_reg32 hisi_dma_comm_chan_regs[] = {
+ {"DMA_QUEUE_SQ_DEPTH ", 0x0008ull},
+ {"DMA_QUEUE_SQ_TAIL_PTR ", 0x000Cull},
+ {"DMA_QUEUE_CQ_DEPTH ", 0x0018ull},
+ {"DMA_QUEUE_CQ_HEAD_PTR ", 0x001Cull},
+ {"DMA_QUEUE_CTRL0 ", 0x0020ull},
+ {"DMA_QUEUE_CTRL1 ", 0x0024ull},
+ {"DMA_QUEUE_FSM_STS ", 0x0030ull},
+ {"DMA_QUEUE_SQ_STS ", 0x0034ull},
+ {"DMA_QUEUE_CQ_TAIL_PTR ", 0x003Cull},
+ {"DMA_QUEUE_INT_STS ", 0x0040ull},
+ {"DMA_QUEUE_INT_MSK ", 0x0044ull},
+ {"DMA_QUEUE_INT_RO ", 0x006Cull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip08_chan_regs[] = {
+ {"DMA_QUEUE_BYTE_CNT ", 0x0038ull},
+ {"DMA_ERR_INT_NUM6 ", 0x0048ull},
+ {"DMA_QUEUE_DESP0 ", 0x0050ull},
+ {"DMA_QUEUE_DESP1 ", 0x0054ull},
+ {"DMA_QUEUE_DESP2 ", 0x0058ull},
+ {"DMA_QUEUE_DESP3 ", 0x005Cull},
+ {"DMA_QUEUE_DESP4 ", 0x0074ull},
+ {"DMA_QUEUE_DESP5 ", 0x0078ull},
+ {"DMA_QUEUE_DESP6 ", 0x007Cull},
+ {"DMA_QUEUE_DESP7 ", 0x0080ull},
+ {"DMA_ERR_INT_NUM0 ", 0x0084ull},
+ {"DMA_ERR_INT_NUM1 ", 0x0088ull},
+ {"DMA_ERR_INT_NUM2 ", 0x008Cull},
+ {"DMA_ERR_INT_NUM3 ", 0x0090ull},
+ {"DMA_ERR_INT_NUM4 ", 0x0094ull},
+ {"DMA_ERR_INT_NUM5 ", 0x0098ull},
+ {"DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip09_chan_regs[] = {
+ {"DMA_QUEUE_ERR_INT_STS ", 0x0048ull},
+ {"DMA_QUEUE_ERR_INT_MSK ", 0x004Cull},
+ {"DFX_SQ_READ_ERR_PTR ", 0x0068ull},
+ {"DFX_DMA_ERR_INT_NUM0 ", 0x0084ull},
+ {"DFX_DMA_ERR_INT_NUM1 ", 0x0088ull},
+ {"DFX_DMA_ERR_INT_NUM2 ", 0x008Cull},
+ {"DFX_DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip08_comm_regs[] = {
+ {"DMA_ECC_ERR_ADDR ", 0x2004ull},
+ {"DMA_ECC_ECC_CNT ", 0x2014ull},
+ {"COMMON_AND_CH_ERR_STS ", 0x2030ull},
+ {"LOCAL_CPL_ID_STS_0 ", 0x20E0ull},
+ {"LOCAL_CPL_ID_STS_1 ", 0x20E4ull},
+ {"LOCAL_CPL_ID_STS_2 ", 0x20E8ull},
+ {"LOCAL_CPL_ID_STS_3 ", 0x20ECull},
+ {"LOCAL_TLP_NUM ", 0x2158ull},
+ {"SQCQ_TLP_NUM ", 0x2164ull},
+ {"CPL_NUM ", 0x2168ull},
+ {"INF_BACK_PRESS_STS ", 0x2170ull},
+ {"DMA_CH_RAS_LEVEL ", 0x2184ull},
+ {"DMA_CM_RAS_LEVEL ", 0x2188ull},
+ {"DMA_CH_ERR_STS ", 0x2190ull},
+ {"DMA_CH_DONE_STS ", 0x2194ull},
+ {"DMA_SQ_TAG_STS_0 ", 0x21A0ull},
+ {"DMA_SQ_TAG_STS_1 ", 0x21A4ull},
+ {"DMA_SQ_TAG_STS_2 ", 0x21A8ull},
+ {"DMA_SQ_TAG_STS_3 ", 0x21ACull},
+ {"LOCAL_P_ID_STS_0 ", 0x21B0ull},
+ {"LOCAL_P_ID_STS_1 ", 0x21B4ull},
+ {"LOCAL_P_ID_STS_2 ", 0x21B8ull},
+ {"LOCAL_P_ID_STS_3 ", 0x21BCull},
+ {"DMA_PREBUFF_INFO_0 ", 0x2200ull},
+ {"DMA_CM_TABLE_INFO_0 ", 0x2220ull},
+ {"DMA_CM_CE_RO ", 0x2244ull},
+ {"DMA_CM_NFE_RO ", 0x2248ull},
+ {"DMA_CM_FE_RO ", 0x224Cull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip09_comm_regs[] = {
+ {"COMMON_AND_CH_ERR_STS ", 0x0030ull},
+ {"DMA_PORT_IDLE_STS ", 0x0150ull},
+ {"DMA_CH_RAS_LEVEL ", 0x0184ull},
+ {"DMA_CM_RAS_LEVEL ", 0x0188ull},
+ {"DMA_CM_CE_RO ", 0x0244ull},
+ {"DMA_CM_NFE_RO ", 0x0248ull},
+ {"DMA_CM_FE_RO ", 0x024Cull},
+ {"DFX_INF_BACK_PRESS_STS0 ", 0x1A40ull},
+ {"DFX_INF_BACK_PRESS_STS1 ", 0x1A44ull},
+ {"DFX_INF_BACK_PRESS_STS2 ", 0x1A48ull},
+ {"DFX_DMA_WRR_DISABLE ", 0x1A4Cull},
+ {"DFX_PA_REQ_TLP_NUM ", 0x1C00ull},
+ {"DFX_PA_BACK_TLP_NUM ", 0x1C04ull},
+ {"DFX_PA_RETRY_TLP_NUM ", 0x1C08ull},
+ {"DFX_LOCAL_NP_TLP_NUM ", 0x1C0Cull},
+ {"DFX_LOCAL_CPL_HEAD_TLP_NUM ", 0x1C10ull},
+ {"DFX_LOCAL_CPL_DATA_TLP_NUM ", 0x1C14ull},
+ {"DFX_LOCAL_CPL_EXT_DATA_TLP_NUM ", 0x1C18ull},
+ {"DFX_LOCAL_P_HEAD_TLP_NUM ", 0x1C1Cull},
+ {"DFX_LOCAL_P_ACK_TLP_NUM ", 0x1C20ull},
+ {"DFX_BUF_ALOC_PORT_REQ_NUM ", 0x1C24ull},
+ {"DFX_BUF_ALOC_PORT_RESULT_NUM ", 0x1C28ull},
+ {"DFX_BUF_FAIL_SIZE_NUM ", 0x1C2Cull},
+ {"DFX_BUF_ALOC_SIZE_NUM ", 0x1C30ull},
+ {"DFX_BUF_NP_RELEASE_SIZE_NUM ", 0x1C34ull},
+ {"DFX_BUF_P_RELEASE_SIZE_NUM ", 0x1C38ull},
+ {"DFX_BUF_PORT_RELEASE_SIZE_NUM ", 0x1C3Cull},
+ {"DFX_DMA_PREBUF_MEM0_ECC_ERR_ADDR ", 0x1CA8ull},
+ {"DFX_DMA_PREBUF_MEM0_ECC_CNT ", 0x1CACull},
+ {"DFX_DMA_LOC_NP_OSTB_ECC_ERR_ADDR ", 0x1CB0ull},
+ {"DFX_DMA_LOC_NP_OSTB_ECC_CNT ", 0x1CB4ull},
+ {"DFX_DMA_PREBUF_MEM1_ECC_ERR_ADDR ", 0x1CC0ull},
+ {"DFX_DMA_PREBUF_MEM1_ECC_CNT ", 0x1CC4ull},
+ {"DMA_CH_DONE_STS ", 0x02E0ull},
+ {"DMA_CH_ERR_STS ", 0x0320ull},
+};
+#endif /* CONFIG_DEBUG_FS*/
+
static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
{
if (pdev->revision == HISI_DMA_REVISION_HIP08B)
@@ -720,6 +844,162 @@ static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
INIT_LIST_HEAD(&dma_dev->channels);
}
+/* --- debugfs implementation --- */
+#ifdef CONFIG_DEBUG_FS
+#include <linux/debugfs.h>
+
+static void hisi_dma_debugfs_init(void)
+{
+ if (!debugfs_initialized())
+ return;
+ hisi_dma_debugfs_root = debugfs_create_dir("hisi_dma", NULL);
+}
+
+static void hisi_dma_debugfs_uninit(void)
+{
+ debugfs_remove_recursive(hisi_dma_debugfs_root);
+}
+
+static struct debugfs_reg32 *hisi_dma_get_ch_regs(struct hisi_dma_dev *hdma_dev,
+ u32 *regs_sz)
+{
+ struct device *dev = &hdma_dev->pdev->dev;
+ struct debugfs_reg32 *regs;
+ u32 regs_sz_comm;
+
+ regs_sz_comm = ARRAY_SIZE(hisi_dma_comm_chan_regs);
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip08_chan_regs);
+ else
+ *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip09_chan_regs);
+
+ regs = devm_kcalloc(dev, *regs_sz, sizeof(struct debugfs_reg32),
+ GFP_KERNEL);
+ if (!regs)
+ return NULL;
+ memcpy(regs, hisi_dma_comm_chan_regs, sizeof(hisi_dma_comm_chan_regs));
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ memcpy(regs + regs_sz_comm, hisi_dma_hip08_chan_regs,
+ sizeof(hisi_dma_hip08_chan_regs));
+ else
+ memcpy(regs + regs_sz_comm, hisi_dma_hip09_chan_regs,
+ sizeof(hisi_dma_hip09_chan_regs));
+
+ return regs;
+}
+
+static int hisi_dma_create_chan_dir(struct hisi_dma_dev *hdma_dev)
+{
+ char dir_name[HISI_DMA_MAX_DIR_NAME_LEN];
+ struct debugfs_regset32 *regsets;
+ struct debugfs_reg32 *regs;
+ struct device *dev;
+ u32 regs_sz;
+ int ret;
+ int i;
+
+ dev = &hdma_dev->pdev->dev;
+
+ regsets = devm_kcalloc(dev, hdma_dev->chan_num,
+ sizeof(*regsets), GFP_KERNEL);
+ if (!regsets)
+ return -ENOMEM;
+
+ hdma_dev->dbg_chans = devm_kcalloc(dev, hdma_dev->chan_num,
+ sizeof(struct dentry *),
+ GFP_KERNEL);
+ if (!hdma_dev->dbg_chans)
+ return -ENOMEM;
+
+ regs = hisi_dma_get_ch_regs(hdma_dev, ®s_sz);
+ if (!regs)
+ return -ENOMEM;
+ for (i = 0; i < hdma_dev->chan_num; i++) {
+ regsets[i].regs = regs;
+ regsets[i].nregs = regs_sz;
+ regsets[i].base = hdma_dev->queue_base + i * HISI_DMA_Q_OFFSET;
+ regsets[i].dev = dev;
+
+ memset(dir_name, 0, HISI_DMA_MAX_DIR_NAME_LEN);
+ ret = sprintf(dir_name, "channel%d", i);
+ if (ret < 0)
+ return ret;
+
+ hdma_dev->dbg_chans[i] = debugfs_create_dir(dir_name,
+ hdma_dev->dbg_hdev_root);
+ if (IS_ERR(hdma_dev->dbg_chans[i])) {
+ hdma_dev->dbg_chans[i] = NULL;
+ dev_err(dev, "dbg_chan[%d] create fail!\n", i);
+ return -EINVAL;
+ }
+ debugfs_create_regset32("regs", 0444,
+ hdma_dev->dbg_chans[i], ®sets[i]);
+ }
+
+ return 0;
+}
+
+static int hisi_dma_debug_register(struct hisi_dma_dev *hdma_dev)
+{
+ struct debugfs_regset32 *regset;
+ struct device *dev;
+ int ret;
+
+ dev = &hdma_dev->pdev->dev;
+
+ hdma_dev->dbg_hdev_root = debugfs_create_dir(dev_name(dev),
+ hisi_dma_debugfs_root);
+ regset = devm_kzalloc(dev, sizeof(*regset), GFP_KERNEL);
+ if (!regset) {
+ ret = -ENOMEM;
+ goto hisi_dma_debug_register_fail;
+ }
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ regset->regs = hisi_dma_hip08_comm_regs;
+ regset->nregs = ARRAY_SIZE(hisi_dma_hip08_comm_regs);
+ } else {
+ regset->regs = hisi_dma_hip09_comm_regs;
+ regset->nregs = ARRAY_SIZE(hisi_dma_hip09_comm_regs);
+ }
+ regset->base = hdma_dev->base;
+ regset->dev = dev;
+
+ debugfs_create_regset32("regs", 0444,
+ hdma_dev->dbg_hdev_root, regset);
+
+ ret = hisi_dma_create_chan_dir(hdma_dev);
+ if (ret < 0)
+ goto hisi_dma_debug_register_fail;
+
+ return 0;
+
+hisi_dma_debug_register_fail:
+ debugfs_remove_recursive(hdma_dev->dbg_hdev_root);
+ hdma_dev->dbg_hdev_root = NULL;
+ return ret;
+}
+
+static void hisi_dma_debug_unregister(void *data)
+{
+ struct hisi_dma_dev *hdma_dev = data;
+
+ debugfs_remove_recursive(hdma_dev->dbg_hdev_root);
+ hdma_dev->dbg_hdev_root = NULL;
+}
+#else
+static void hisi_dma_debugfs_init(void) { }
+static void hisi_dma_debugfs_uninit(void) { }
+
+static int hisi_dma_debug_register(struct hisi_dma_dev *hdma_dev)
+{
+ return 0;
+}
+
+static void hisi_dma_debug_unregister(void *data) { }
+#endif /* CONFIG_DEBUG_FS*/
+/* --- debugfs implementation --- */
+
static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
enum hisi_dma_reg_layout reg_layout;
@@ -796,10 +1076,19 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
dma_dev = &hdma_dev->dma_dev;
ret = dmaenginem_async_device_register(dma_dev);
- if (ret < 0)
+ if (ret < 0) {
dev_err(dev, "failed to register device!\n");
+ return ret;
+ }
- return ret;
+ ret = hisi_dma_debug_register(hdma_dev);
+ if (ret < 0) {
+ dev_err(dev, "failed to register debugfs!\n");
+ return ret;
+ }
+
+ return devm_add_action_or_reset(dev, hisi_dma_debug_unregister,
+ hdma_dev);
}
static const struct pci_device_id hisi_dma_pci_tbl[] = {
@@ -813,7 +1102,29 @@ static struct pci_driver hisi_dma_pci_driver = {
.probe = hisi_dma_probe,
};
-module_pci_driver(hisi_dma_pci_driver);
+static int __init hisi_dma_init(void)
+{
+ int ret;
+
+ hisi_dma_debugfs_init();
+
+ ret = pci_register_driver(&hisi_dma_pci_driver);
+ if (ret) {
+ hisi_dma_debugfs_uninit();
+ pr_err("hisi_dma: can't register hisi dma driver.\n");
+ }
+
+ return ret;
+}
+
+static void __exit hisi_dma_exit(void)
+{
+ pci_unregister_driver(&hisi_dma_pci_driver);
+ hisi_dma_debugfs_uninit();
+}
+
+module_init(hisi_dma_init);
+module_exit(hisi_dma_exit);
MODULE_AUTHOR("Zhou Wang <wangzhou1@hisilicon.com>");
MODULE_AUTHOR("Zhenfa Qiu <qiuzhenfa@hisilicon.com>");
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v2 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma
2022-06-29 3:55 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (5 preceding siblings ...)
2022-06-29 3:55 ` [PATCH v2 6/7] dmaengine: hisilicon: Add dfx feature for hisi dma driver Jie Hai
@ 2022-06-29 3:55 ` Jie Hai
2022-07-08 3:16 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Zhou Wang
7 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-06-29 3:55 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: dmaengine, linux-kernel
Add myself as a maintainer for hisi_dma.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
MAINTAINERS | 1 +
1 file changed, 1 insertion(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index cb5aaeb84ac4..e85cdb8f2903 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8899,6 +8899,7 @@ F: net/dsa/tag_hellcreek.c
HISILICON DMA DRIVER
M: Zhou Wang <wangzhou1@hisilicon.com>
+M: Jie Hai <haijie1@hisilicon.com>
L: dmaengine@vger.kernel.org
S: Maintained
F: drivers/dma/hisi_dma.c
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* Re: [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver
2022-06-29 3:55 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (6 preceding siblings ...)
2022-06-29 3:55 ` [PATCH v2 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
@ 2022-07-08 3:16 ` Zhou Wang
7 siblings, 0 replies; 64+ messages in thread
From: Zhou Wang @ 2022-07-08 3:16 UTC (permalink / raw)
To: Jie Hai, vkoul; +Cc: dmaengine, linux-kernel
On 2022/6/29 11:55, Jie Hai wrote:
> The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they share the
> same pci device id but different pci revision and register layouts.
>
> The original version supports HiSilicon IP08 but not HiSilicon IP09.
> This series support DMA driver for HIP08 and HIP09:
> 1. Fix bugs for HIP08 DMA driver
> - Disable hardware channels when driver detached
> - Update cq_head whenever accessed it
> - Support multi-thread for one DMA channel
> 2. Use macros instead of magic number
> 3. Add support for HIP09 DMA driver
> 4. Dump registers for HIP08 and HIP09 DMA driver with debugfs
> 5. Add myself as maintainer of hisi_dma.c
>
> Changes since version 1:
> - fix compile failure reported by kernel test robot
> - fix reduldant "*" in comment
> - fix reduldant blank line in commit log
> - remove debugfs-hisi-dma doc and path in MAINTAINERS
> - add more explanations in patch 3/7
>
> Jie Hai (7):
> dmaengine: hisilicon: Disable channels when unregister hisi_dma
> dmaengine: hisilicon: Fix CQ head update
> dmaengine: hisilicon: Add multi-thread support for a DMA channel
> dmaengine: hisilicon: Use macros instead of magic number
> dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
> dmaengine: hisilicon: Add dfx feature for hisi dma driver
> MAINTAINERS: Add myself as maintainer for hisi_dma
>
> MAINTAINERS | 1 +
> drivers/dma/hisi_dma.c | 730 +++++++++++++++++++++++++++++++++++------
> 2 files changed, 635 insertions(+), 96 deletions(-)
>
For the whole series:
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
Thanks for the bug fixes and supporting for IP09 dma :)
Best,
Zhou
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] dmaengine: hisilicon: Add dfx feature for hisi dma driver
2022-06-29 3:55 ` [PATCH v2 6/7] dmaengine: hisilicon: Add dfx feature for hisi dma driver Jie Hai
@ 2022-07-21 13:25 ` Vinod Koul
2022-07-26 1:43 ` Jie Hai
0 siblings, 1 reply; 64+ messages in thread
From: Vinod Koul @ 2022-07-21 13:25 UTC (permalink / raw)
To: Jie Hai; +Cc: wangzhou1, dmaengine, linux-kernel
On 29-06-22, 11:55, Jie Hai wrote:
What does dfx mean in title?
> This patch adds dump of registers with debugfs for HIP08
> and HIP09 DMA driver.
>
> Reported-by: kernel test robot <lkp@intel.com>
?
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
> drivers/dma/hisi_dma.c | 317 ++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 314 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
> index dc7c59b21114..b0bc7b18933b 100644
> --- a/drivers/dma/hisi_dma.c
> +++ b/drivers/dma/hisi_dma.c
> @@ -78,6 +78,8 @@
> #define HISI_DMA_POLL_Q_STS_DELAY_US 10
> #define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
>
> +#define HISI_DMA_MAX_DIR_NAME_LEN 128
> +
> /*
> * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
> * have the same pci device id but different pci revision.
> @@ -161,9 +163,131 @@ struct hisi_dma_dev {
> u32 chan_depth;
> enum hisi_dma_reg_layout reg_layout;
> void __iomem *queue_base; /* queue region start of register */
> +#ifdef CONFIG_DEBUG_FS
> + struct dentry *dbg_hdev_root;
Please do not create your own root and use dmaengine root instead
> + struct dentry **dbg_chans;
> +#endif
> struct hisi_dma_chan chan[];
> };
>
> +#ifdef CONFIG_DEBUG_FS
> +struct dentry *hisi_dma_debugfs_root;
> +
> +static const struct debugfs_reg32 hisi_dma_comm_chan_regs[] = {
> + {"DMA_QUEUE_SQ_DEPTH ", 0x0008ull},
> + {"DMA_QUEUE_SQ_TAIL_PTR ", 0x000Cull},
> + {"DMA_QUEUE_CQ_DEPTH ", 0x0018ull},
> + {"DMA_QUEUE_CQ_HEAD_PTR ", 0x001Cull},
> + {"DMA_QUEUE_CTRL0 ", 0x0020ull},
> + {"DMA_QUEUE_CTRL1 ", 0x0024ull},
> + {"DMA_QUEUE_FSM_STS ", 0x0030ull},
> + {"DMA_QUEUE_SQ_STS ", 0x0034ull},
> + {"DMA_QUEUE_CQ_TAIL_PTR ", 0x003Cull},
> + {"DMA_QUEUE_INT_STS ", 0x0040ull},
> + {"DMA_QUEUE_INT_MSK ", 0x0044ull},
> + {"DMA_QUEUE_INT_RO ", 0x006Cull},
> +};
> +
> +static const struct debugfs_reg32 hisi_dma_hip08_chan_regs[] = {
> + {"DMA_QUEUE_BYTE_CNT ", 0x0038ull},
> + {"DMA_ERR_INT_NUM6 ", 0x0048ull},
> + {"DMA_QUEUE_DESP0 ", 0x0050ull},
> + {"DMA_QUEUE_DESP1 ", 0x0054ull},
> + {"DMA_QUEUE_DESP2 ", 0x0058ull},
> + {"DMA_QUEUE_DESP3 ", 0x005Cull},
> + {"DMA_QUEUE_DESP4 ", 0x0074ull},
> + {"DMA_QUEUE_DESP5 ", 0x0078ull},
> + {"DMA_QUEUE_DESP6 ", 0x007Cull},
> + {"DMA_QUEUE_DESP7 ", 0x0080ull},
> + {"DMA_ERR_INT_NUM0 ", 0x0084ull},
> + {"DMA_ERR_INT_NUM1 ", 0x0088ull},
> + {"DMA_ERR_INT_NUM2 ", 0x008Cull},
> + {"DMA_ERR_INT_NUM3 ", 0x0090ull},
> + {"DMA_ERR_INT_NUM4 ", 0x0094ull},
> + {"DMA_ERR_INT_NUM5 ", 0x0098ull},
> + {"DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
> +};
> +
> +static const struct debugfs_reg32 hisi_dma_hip09_chan_regs[] = {
> + {"DMA_QUEUE_ERR_INT_STS ", 0x0048ull},
> + {"DMA_QUEUE_ERR_INT_MSK ", 0x004Cull},
> + {"DFX_SQ_READ_ERR_PTR ", 0x0068ull},
> + {"DFX_DMA_ERR_INT_NUM0 ", 0x0084ull},
> + {"DFX_DMA_ERR_INT_NUM1 ", 0x0088ull},
> + {"DFX_DMA_ERR_INT_NUM2 ", 0x008Cull},
> + {"DFX_DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
> +};
> +
> +static const struct debugfs_reg32 hisi_dma_hip08_comm_regs[] = {
> + {"DMA_ECC_ERR_ADDR ", 0x2004ull},
> + {"DMA_ECC_ECC_CNT ", 0x2014ull},
> + {"COMMON_AND_CH_ERR_STS ", 0x2030ull},
> + {"LOCAL_CPL_ID_STS_0 ", 0x20E0ull},
> + {"LOCAL_CPL_ID_STS_1 ", 0x20E4ull},
> + {"LOCAL_CPL_ID_STS_2 ", 0x20E8ull},
> + {"LOCAL_CPL_ID_STS_3 ", 0x20ECull},
> + {"LOCAL_TLP_NUM ", 0x2158ull},
> + {"SQCQ_TLP_NUM ", 0x2164ull},
> + {"CPL_NUM ", 0x2168ull},
> + {"INF_BACK_PRESS_STS ", 0x2170ull},
> + {"DMA_CH_RAS_LEVEL ", 0x2184ull},
> + {"DMA_CM_RAS_LEVEL ", 0x2188ull},
> + {"DMA_CH_ERR_STS ", 0x2190ull},
> + {"DMA_CH_DONE_STS ", 0x2194ull},
> + {"DMA_SQ_TAG_STS_0 ", 0x21A0ull},
> + {"DMA_SQ_TAG_STS_1 ", 0x21A4ull},
> + {"DMA_SQ_TAG_STS_2 ", 0x21A8ull},
> + {"DMA_SQ_TAG_STS_3 ", 0x21ACull},
> + {"LOCAL_P_ID_STS_0 ", 0x21B0ull},
> + {"LOCAL_P_ID_STS_1 ", 0x21B4ull},
> + {"LOCAL_P_ID_STS_2 ", 0x21B8ull},
> + {"LOCAL_P_ID_STS_3 ", 0x21BCull},
> + {"DMA_PREBUFF_INFO_0 ", 0x2200ull},
> + {"DMA_CM_TABLE_INFO_0 ", 0x2220ull},
> + {"DMA_CM_CE_RO ", 0x2244ull},
> + {"DMA_CM_NFE_RO ", 0x2248ull},
> + {"DMA_CM_FE_RO ", 0x224Cull},
> +};
> +
> +static const struct debugfs_reg32 hisi_dma_hip09_comm_regs[] = {
> + {"COMMON_AND_CH_ERR_STS ", 0x0030ull},
> + {"DMA_PORT_IDLE_STS ", 0x0150ull},
> + {"DMA_CH_RAS_LEVEL ", 0x0184ull},
> + {"DMA_CM_RAS_LEVEL ", 0x0188ull},
> + {"DMA_CM_CE_RO ", 0x0244ull},
> + {"DMA_CM_NFE_RO ", 0x0248ull},
> + {"DMA_CM_FE_RO ", 0x024Cull},
> + {"DFX_INF_BACK_PRESS_STS0 ", 0x1A40ull},
> + {"DFX_INF_BACK_PRESS_STS1 ", 0x1A44ull},
> + {"DFX_INF_BACK_PRESS_STS2 ", 0x1A48ull},
> + {"DFX_DMA_WRR_DISABLE ", 0x1A4Cull},
> + {"DFX_PA_REQ_TLP_NUM ", 0x1C00ull},
> + {"DFX_PA_BACK_TLP_NUM ", 0x1C04ull},
> + {"DFX_PA_RETRY_TLP_NUM ", 0x1C08ull},
> + {"DFX_LOCAL_NP_TLP_NUM ", 0x1C0Cull},
> + {"DFX_LOCAL_CPL_HEAD_TLP_NUM ", 0x1C10ull},
> + {"DFX_LOCAL_CPL_DATA_TLP_NUM ", 0x1C14ull},
> + {"DFX_LOCAL_CPL_EXT_DATA_TLP_NUM ", 0x1C18ull},
> + {"DFX_LOCAL_P_HEAD_TLP_NUM ", 0x1C1Cull},
> + {"DFX_LOCAL_P_ACK_TLP_NUM ", 0x1C20ull},
> + {"DFX_BUF_ALOC_PORT_REQ_NUM ", 0x1C24ull},
> + {"DFX_BUF_ALOC_PORT_RESULT_NUM ", 0x1C28ull},
> + {"DFX_BUF_FAIL_SIZE_NUM ", 0x1C2Cull},
> + {"DFX_BUF_ALOC_SIZE_NUM ", 0x1C30ull},
> + {"DFX_BUF_NP_RELEASE_SIZE_NUM ", 0x1C34ull},
> + {"DFX_BUF_P_RELEASE_SIZE_NUM ", 0x1C38ull},
> + {"DFX_BUF_PORT_RELEASE_SIZE_NUM ", 0x1C3Cull},
> + {"DFX_DMA_PREBUF_MEM0_ECC_ERR_ADDR ", 0x1CA8ull},
> + {"DFX_DMA_PREBUF_MEM0_ECC_CNT ", 0x1CACull},
> + {"DFX_DMA_LOC_NP_OSTB_ECC_ERR_ADDR ", 0x1CB0ull},
> + {"DFX_DMA_LOC_NP_OSTB_ECC_CNT ", 0x1CB4ull},
> + {"DFX_DMA_PREBUF_MEM1_ECC_ERR_ADDR ", 0x1CC0ull},
> + {"DFX_DMA_PREBUF_MEM1_ECC_CNT ", 0x1CC4ull},
> + {"DMA_CH_DONE_STS ", 0x02E0ull},
> + {"DMA_CH_ERR_STS ", 0x0320ull},
> +};
> +#endif /* CONFIG_DEBUG_FS*/
> +
> static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
> {
> if (pdev->revision == HISI_DMA_REVISION_HIP08B)
> @@ -720,6 +844,162 @@ static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
> INIT_LIST_HEAD(&dma_dev->channels);
> }
>
> +/* --- debugfs implementation --- */
> +#ifdef CONFIG_DEBUG_FS
> +#include <linux/debugfs.h>
> +
> +static void hisi_dma_debugfs_init(void)
> +{
> + if (!debugfs_initialized())
> + return;
> + hisi_dma_debugfs_root = debugfs_create_dir("hisi_dma", NULL);
> +}
> +
> +static void hisi_dma_debugfs_uninit(void)
> +{
> + debugfs_remove_recursive(hisi_dma_debugfs_root);
> +}
> +
> +static struct debugfs_reg32 *hisi_dma_get_ch_regs(struct hisi_dma_dev *hdma_dev,
> + u32 *regs_sz)
> +{
> + struct device *dev = &hdma_dev->pdev->dev;
> + struct debugfs_reg32 *regs;
> + u32 regs_sz_comm;
> +
> + regs_sz_comm = ARRAY_SIZE(hisi_dma_comm_chan_regs);
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
> + *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip08_chan_regs);
> + else
> + *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip09_chan_regs);
> +
> + regs = devm_kcalloc(dev, *regs_sz, sizeof(struct debugfs_reg32),
> + GFP_KERNEL);
> + if (!regs)
> + return NULL;
> + memcpy(regs, hisi_dma_comm_chan_regs, sizeof(hisi_dma_comm_chan_regs));
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
> + memcpy(regs + regs_sz_comm, hisi_dma_hip08_chan_regs,
> + sizeof(hisi_dma_hip08_chan_regs));
> + else
> + memcpy(regs + regs_sz_comm, hisi_dma_hip09_chan_regs,
> + sizeof(hisi_dma_hip09_chan_regs));
> +
> + return regs;
> +}
> +
> +static int hisi_dma_create_chan_dir(struct hisi_dma_dev *hdma_dev)
> +{
> + char dir_name[HISI_DMA_MAX_DIR_NAME_LEN];
> + struct debugfs_regset32 *regsets;
> + struct debugfs_reg32 *regs;
> + struct device *dev;
> + u32 regs_sz;
> + int ret;
> + int i;
> +
> + dev = &hdma_dev->pdev->dev;
> +
> + regsets = devm_kcalloc(dev, hdma_dev->chan_num,
> + sizeof(*regsets), GFP_KERNEL);
> + if (!regsets)
> + return -ENOMEM;
> +
> + hdma_dev->dbg_chans = devm_kcalloc(dev, hdma_dev->chan_num,
> + sizeof(struct dentry *),
> + GFP_KERNEL);
> + if (!hdma_dev->dbg_chans)
> + return -ENOMEM;
> +
> + regs = hisi_dma_get_ch_regs(hdma_dev, ®s_sz);
> + if (!regs)
> + return -ENOMEM;
> + for (i = 0; i < hdma_dev->chan_num; i++) {
> + regsets[i].regs = regs;
> + regsets[i].nregs = regs_sz;
> + regsets[i].base = hdma_dev->queue_base + i * HISI_DMA_Q_OFFSET;
> + regsets[i].dev = dev;
> +
> + memset(dir_name, 0, HISI_DMA_MAX_DIR_NAME_LEN);
> + ret = sprintf(dir_name, "channel%d", i);
> + if (ret < 0)
> + return ret;
> +
> + hdma_dev->dbg_chans[i] = debugfs_create_dir(dir_name,
> + hdma_dev->dbg_hdev_root);
> + if (IS_ERR(hdma_dev->dbg_chans[i])) {
> + hdma_dev->dbg_chans[i] = NULL;
> + dev_err(dev, "dbg_chan[%d] create fail!\n", i);
> + return -EINVAL;
> + }
> + debugfs_create_regset32("regs", 0444,
> + hdma_dev->dbg_chans[i], ®sets[i]);
> + }
> +
> + return 0;
> +}
> +
> +static int hisi_dma_debug_register(struct hisi_dma_dev *hdma_dev)
> +{
> + struct debugfs_regset32 *regset;
> + struct device *dev;
> + int ret;
> +
> + dev = &hdma_dev->pdev->dev;
> +
> + hdma_dev->dbg_hdev_root = debugfs_create_dir(dev_name(dev),
> + hisi_dma_debugfs_root);
> + regset = devm_kzalloc(dev, sizeof(*regset), GFP_KERNEL);
> + if (!regset) {
> + ret = -ENOMEM;
> + goto hisi_dma_debug_register_fail;
> + }
> +
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
> + regset->regs = hisi_dma_hip08_comm_regs;
> + regset->nregs = ARRAY_SIZE(hisi_dma_hip08_comm_regs);
> + } else {
> + regset->regs = hisi_dma_hip09_comm_regs;
> + regset->nregs = ARRAY_SIZE(hisi_dma_hip09_comm_regs);
> + }
> + regset->base = hdma_dev->base;
> + regset->dev = dev;
> +
> + debugfs_create_regset32("regs", 0444,
> + hdma_dev->dbg_hdev_root, regset);
> +
> + ret = hisi_dma_create_chan_dir(hdma_dev);
> + if (ret < 0)
> + goto hisi_dma_debug_register_fail;
> +
> + return 0;
> +
> +hisi_dma_debug_register_fail:
> + debugfs_remove_recursive(hdma_dev->dbg_hdev_root);
> + hdma_dev->dbg_hdev_root = NULL;
> + return ret;
> +}
> +
> +static void hisi_dma_debug_unregister(void *data)
> +{
> + struct hisi_dma_dev *hdma_dev = data;
> +
> + debugfs_remove_recursive(hdma_dev->dbg_hdev_root);
> + hdma_dev->dbg_hdev_root = NULL;
> +}
> +#else
> +static void hisi_dma_debugfs_init(void) { }
> +static void hisi_dma_debugfs_uninit(void) { }
> +
> +static int hisi_dma_debug_register(struct hisi_dma_dev *hdma_dev)
> +{
> + return 0;
> +}
> +
> +static void hisi_dma_debug_unregister(void *data) { }
> +#endif /* CONFIG_DEBUG_FS*/
> +/* --- debugfs implementation --- */
> +
> static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> {
> enum hisi_dma_reg_layout reg_layout;
> @@ -796,10 +1076,19 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>
> dma_dev = &hdma_dev->dma_dev;
> ret = dmaenginem_async_device_register(dma_dev);
> - if (ret < 0)
> + if (ret < 0) {
> dev_err(dev, "failed to register device!\n");
> + return ret;
> + }
>
> - return ret;
> + ret = hisi_dma_debug_register(hdma_dev);
> + if (ret < 0) {
> + dev_err(dev, "failed to register debugfs!\n");
> + return ret;
> + }
why should registering debugfs fail be a driver failure, it can still
work as epxected, pls ignore these errors and remove the error handling
for this piece of code
> +
> + return devm_add_action_or_reset(dev, hisi_dma_debug_unregister,
> + hdma_dev);
> }
>
> static const struct pci_device_id hisi_dma_pci_tbl[] = {
> @@ -813,7 +1102,29 @@ static struct pci_driver hisi_dma_pci_driver = {
> .probe = hisi_dma_probe,
> };
>
> -module_pci_driver(hisi_dma_pci_driver);
> +static int __init hisi_dma_init(void)
> +{
> + int ret;
> +
> + hisi_dma_debugfs_init();
> +
> + ret = pci_register_driver(&hisi_dma_pci_driver);
> + if (ret) {
> + hisi_dma_debugfs_uninit();
> + pr_err("hisi_dma: can't register hisi dma driver.\n");
> + }
> +
> + return ret;
> +}
> +
> +static void __exit hisi_dma_exit(void)
> +{
> + pci_unregister_driver(&hisi_dma_pci_driver);
> + hisi_dma_debugfs_uninit();
> +}
> +
> +module_init(hisi_dma_init);
> +module_exit(hisi_dma_exit);
>
> MODULE_AUTHOR("Zhou Wang <wangzhou1@hisilicon.com>");
> MODULE_AUTHOR("Zhenfa Qiu <qiuzhenfa@hisilicon.com>");
> --
> 2.33.0
--
~Vinod
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 2/7] dmaengine: hisilicon: Fix CQ head update
2022-06-29 3:55 ` [PATCH v2 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
@ 2022-07-21 13:27 ` Vinod Koul
2022-07-26 1:38 ` Jie Hai
0 siblings, 1 reply; 64+ messages in thread
From: Vinod Koul @ 2022-07-21 13:27 UTC (permalink / raw)
To: Jie Hai; +Cc: wangzhou1, dmaengine, linux-kernel
On 29-06-22, 11:55, Jie Hai wrote:
> After completion of data transfer of one or multiple descriptors,
> the completion status and the current head pointer to submission
> queue are written into the CQ and interrupt can be generated to
> inform the software. In interrupt process CQ is read and cq_head
> is updated.
>
> hisi_dma_irq updates cq_head only when the completion status is
> success. When an abnormal interrupt reports, cq_head will not update
> which will cause subsequent interrupt processes read the error CQ
> and never report the correct status.
>
> This patch updates cq_head whenever CQ is accessed.
>
> Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
> Reported-by: kernel test robot <lkp@intel.com>
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
> drivers/dma/hisi_dma.c | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
> index 98bc488893cc..7609e6e7eb37 100644
> --- a/drivers/dma/hisi_dma.c
> +++ b/drivers/dma/hisi_dma.c
> @@ -436,12 +436,12 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
> desc = chan->desc;
> cqe = chan->cq + chan->cq_head;
> if (desc) {
> + chan->cq_head = (chan->cq_head + 1) %
> + hdma_dev->chan_depth;
This can look better with single line
> + hisi_dma_chan_write(hdma_dev->base,
> + HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
> + chan->cq_head);
maybe two lines?
> if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
> - chan->cq_head = (chan->cq_head + 1) %
> - hdma_dev->chan_depth;
> - hisi_dma_chan_write(hdma_dev->base,
> - HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
> - chan->cq_head);
> vchan_cookie_complete(&desc->vd);
> } else {
> dev_err(&hdma_dev->pdev->dev, "task error!\n");
> --
> 2.33.0
--
~Vinod
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
2022-06-29 3:55 ` [PATCH v2 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
@ 2022-07-21 13:29 ` Vinod Koul
2022-07-26 1:40 ` Jie Hai
0 siblings, 1 reply; 64+ messages in thread
From: Vinod Koul @ 2022-07-21 13:29 UTC (permalink / raw)
To: Jie Hai; +Cc: wangzhou1, dmaengine, linux-kernel
On 29-06-22, 11:55, Jie Hai wrote:
> The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they
> have the same pci device id but different pci revision.
> Unfortunately, they have different register layouts, so
> the origin driver cannot run on HiSilicon IP09 correctly.
>
> This patch enables the driver to adapt to HiSilicon IP09.
> HiSilicon IP09 offers 4 channels, each channel has a send
> queue, a complete queue and an interrupt to help to do tasks.
> This DMA engine can do memory copy between memory blocks.
>
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> ---
> drivers/dma/hisi_dma.c | 382 ++++++++++++++++++++++++++++++++---------
> 1 file changed, 299 insertions(+), 83 deletions(-)
>
> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
> index d5f87ef4a5ee..dc7c59b21114 100644
> --- a/drivers/dma/hisi_dma.c
> +++ b/drivers/dma/hisi_dma.c
> @@ -1,5 +1,6 @@
> // SPDX-License-Identifier: GPL-2.0-only
> -/* Copyright(c) 2019 HiSilicon Limited. */
> +/* Copyright(c) 2019-2022 HiSilicon Limited. */
> +
> #include <linux/bitfield.h>
> #include <linux/dmaengine.h>
> #include <linux/init.h>
> @@ -9,35 +10,85 @@
> #include <linux/spinlock.h>
> #include "virt-dma.h"
>
> -#define HISI_DMA_SQ_BASE_L 0x0
> -#define HISI_DMA_SQ_BASE_H 0x4
> -#define HISI_DMA_SQ_DEPTH 0x8
> -#define HISI_DMA_SQ_TAIL_PTR 0xc
> -#define HISI_DMA_CQ_BASE_L 0x10
> -#define HISI_DMA_CQ_BASE_H 0x14
> -#define HISI_DMA_CQ_DEPTH 0x18
> -#define HISI_DMA_CQ_HEAD_PTR 0x1c
> -#define HISI_DMA_CTRL0 0x20
> -#define HISI_DMA_CTRL0_QUEUE_EN_S 0
> -#define HISI_DMA_CTRL0_QUEUE_PAUSE_S 4
> -#define HISI_DMA_CTRL1 0x24
> -#define HISI_DMA_CTRL1_QUEUE_RESET_S 0
> -#define HISI_DMA_Q_FSM_STS 0x30
> -#define HISI_DMA_FSM_STS_MASK GENMASK(3, 0)
> -#define HISI_DMA_INT_STS 0x40
> -#define HISI_DMA_INT_STS_MASK GENMASK(12, 0)
> -#define HISI_DMA_INT_MSK 0x44
> -#define HISI_DMA_MODE 0x217c
> -#define HISI_DMA_OFFSET 0x100
> -
> -#define HISI_DMA_MSI_NUM 32
> -#define HISI_DMA_CHAN_NUM 30
> -#define HISI_DMA_Q_DEPTH_VAL 1024
> -
> -#define PCI_BAR_2 2
> -
> -#define HISI_DMA_POLL_Q_STS_DELAY_US 10
> -#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
> +/* HiSilicon DMA register common field define */
> +#define HISI_DMA_Q_SQ_BASE_L 0x0
> +#define HISI_DMA_Q_SQ_BASE_H 0x4
> +#define HISI_DMA_Q_SQ_DEPTH 0x8
> +#define HISI_DMA_Q_SQ_TAIL_PTR 0xc
> +#define HISI_DMA_Q_CQ_BASE_L 0x10
> +#define HISI_DMA_Q_CQ_BASE_H 0x14
> +#define HISI_DMA_Q_CQ_DEPTH 0x18
> +#define HISI_DMA_Q_CQ_HEAD_PTR 0x1c
> +#define HISI_DMA_Q_CTRL0 0x20
> +#define HISI_DMA_Q_CTRL0_QUEUE_EN 0
> +#define HISI_DMA_Q_CTRL0_QUEUE_PAUSE 4
Pls use BIT/GENMASK only for register bits
> +#define HISI_DMA_Q_CTRL1 0x24
> +#define HISI_DMA_Q_CTRL1_QUEUE_RESET 0
> +#define HISI_DMA_Q_FSM_STS 0x30
> +#define HISI_DMA_Q_FSM_STS_MASK GENMASK(3, 0)
> +#define HISI_DMA_Q_ERR_INT_NUM0 0x84
> +#define HISI_DMA_Q_ERR_INT_NUM1 0x88
> +#define HISI_DMA_Q_ERR_INT_NUM2 0x8c
> +
> +/* HiSilicon IP08 DMA register and field define */
> +#define HISI_DMA_HIP08_MODE 0x217C
> +#define HISI_DMA_HIP08_Q_BASE 0x0
> +#define HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN 2
> +#define HISI_DMA_HIP08_Q_INT_STS 0x40
> +#define HISI_DMA_HIP08_Q_INT_MSK 0x44
> +#define HISI_DMA_HIP08_Q_INT_STS_MASK GENMASK(14, 0)
> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM3 0x90
> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM4 0x94
> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM5 0x98
> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM6 0x48
> +#define HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT 24
> +
> +/* HiSilicon IP09 DMA register and field define */
> +#define HISI_DMA_HIP09_DMA_FLR_DISABLE 0xA00
> +#define HISI_DMA_HIP09_DMA_FLR_DISABLE_B 0
> +#define HISI_DMA_HIP09_Q_BASE 0x2000
> +#define HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN GENMASK(31, 28)
> +#define HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT 26
> +#define HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT 27
> +#define HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE 2
> +#define HISI_DMA_HIP09_Q_INT_STS 0x40
> +#define HISI_DMA_HIP09_Q_INT_MSK 0x44
> +#define HISI_DMA_HIP09_Q_INT_STS_MASK 0x1
> +#define HISI_DMA_HIP09_Q_ERR_INT_STS 0x48
> +#define HISI_DMA_HIP09_Q_ERR_INT_MSK 0x4C
> +#define HISI_DMA_HIP09_Q_ERR_INT_STS_MASK GENMASK(18, 1)
> +#define HISI_DMA_HIP09_PORT_CFG_REG(port_id) (0x800 + \
> + (port_id) * 0x20)
> +#define HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B 16
> +
> +#define HISI_DMA_HIP09_MAX_PORT_NUM 16
> +
> +#define HISI_DMA_HIP08_MSI_NUM 32
> +#define HISI_DMA_HIP08_CHAN_NUM 30
> +#define HISI_DMA_HIP09_MSI_NUM 4
> +#define HISI_DMA_HIP09_CHAN_NUM 4
> +#define HISI_DMA_REVISION_HIP08B 0x21
> +#define HISI_DMA_REVISION_HIP09A 0x30
> +
> +#define HISI_DMA_Q_OFFSET 0x100
> +#define HISI_DMA_Q_DEPTH_VAL 1024
> +
> +#define PCI_BAR_2 2
> +
> +#define HISI_DMA_POLL_Q_STS_DELAY_US 10
> +#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
> +
> +/*
> + * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
> + * have the same pci device id but different pci revision.
> + * Unfortunately, they have different register layouts, so two layout
> + * enumerations are defined.
> + */
> +enum hisi_dma_reg_layout {
> + HISI_DMA_REG_LAYOUT_INVALID = 0,
> + HISI_DMA_REG_LAYOUT_HIP08,
> + HISI_DMA_REG_LAYOUT_HIP09
> +};
>
> enum hisi_dma_mode {
> EP = 0,
> @@ -108,9 +159,45 @@ struct hisi_dma_dev {
> struct dma_device dma_dev;
> u32 chan_num;
> u32 chan_depth;
> + enum hisi_dma_reg_layout reg_layout;
> + void __iomem *queue_base; /* queue region start of register */
> struct hisi_dma_chan chan[];
> };
>
> +static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
> +{
> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
> + return HISI_DMA_REG_LAYOUT_HIP08;
> + else if (pdev->revision >= HISI_DMA_REVISION_HIP09A)
> + return HISI_DMA_REG_LAYOUT_HIP09;
> +
> + return HISI_DMA_REG_LAYOUT_INVALID;
> +}
> +
> +static u32 hisi_dma_get_chan_num(struct pci_dev *pdev)
> +{
> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
> + return HISI_DMA_HIP08_CHAN_NUM;
> +
> + return HISI_DMA_HIP09_CHAN_NUM;
> +}
> +
> +static u32 hisi_dma_get_msi_num(struct pci_dev *pdev)
> +{
> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
> + return HISI_DMA_HIP08_MSI_NUM;
> +
> + return HISI_DMA_HIP09_MSI_NUM;
> +}
> +
> +static u32 hisi_dma_get_queue_base(struct pci_dev *pdev)
> +{
> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
> + return HISI_DMA_HIP08_Q_BASE;
> +
> + return HISI_DMA_HIP09_Q_BASE;
> +}
> +
> static inline struct hisi_dma_chan *to_hisi_dma_chan(struct dma_chan *c)
> {
> return container_of(c, struct hisi_dma_chan, vc.chan);
> @@ -124,7 +211,7 @@ static inline struct hisi_dma_desc *to_hisi_dma_desc(struct virt_dma_desc *vd)
> static inline void hisi_dma_chan_write(void __iomem *base, u32 reg, u32 index,
> u32 val)
> {
> - writel_relaxed(val, base + reg + index * HISI_DMA_OFFSET);
> + writel_relaxed(val, base + reg + index * HISI_DMA_Q_OFFSET);
> }
>
> static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
> @@ -139,48 +226,76 @@ static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
> static void hisi_dma_pause_dma(struct hisi_dma_dev *hdma_dev, u32 index,
> bool pause)
> {
> - void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
> - HISI_DMA_OFFSET;
> + void __iomem *addr;
>
> - hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_PAUSE_S, pause);
> + addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
> + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_PAUSE, pause);
> }
>
> static void hisi_dma_enable_dma(struct hisi_dma_dev *hdma_dev, u32 index,
> bool enable)
> {
> - void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
> - HISI_DMA_OFFSET;
> + void __iomem *addr;
>
> - hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_EN_S, enable);
> + addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
> + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_EN, enable);
> }
>
> static void hisi_dma_mask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
> {
> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_INT_MSK, qp_index,
> - HISI_DMA_INT_STS_MASK);
> + void __iomem *q_base = hdma_dev->queue_base;
> +
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
> + qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
> + else {
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
> + qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
> + qp_index,
> + HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
> + }
> }
>
> static void hisi_dma_unmask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
> {
> - void __iomem *base = hdma_dev->base;
> -
> - hisi_dma_chan_write(base, HISI_DMA_INT_STS, qp_index,
> - HISI_DMA_INT_STS_MASK);
> - hisi_dma_chan_write(base, HISI_DMA_INT_MSK, qp_index, 0);
> + void __iomem *q_base = hdma_dev->queue_base;
> +
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_STS,
> + qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
> + qp_index, 0);
> + } else {
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_STS,
> + qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_STS,
> + qp_index,
> + HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
> + qp_index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
> + qp_index, 0);
> + }
> }
>
> static void hisi_dma_do_reset(struct hisi_dma_dev *hdma_dev, u32 index)
> {
> - void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL1 + index *
> - HISI_DMA_OFFSET;
> + void __iomem *addr;
>
> - hisi_dma_update_bit(addr, HISI_DMA_CTRL1_QUEUE_RESET_S, 1);
> + addr = hdma_dev->queue_base +
> + HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL1_QUEUE_RESET, 1);
> }
>
> static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
> {
> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, index, 0);
> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
> + void __iomem *q_base = hdma_dev->queue_base;
> +
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
> }
>
> static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
> @@ -195,11 +310,11 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
> hisi_dma_enable_dma(hdma_dev, index, false);
> hisi_dma_mask_irq(hdma_dev, index);
>
> - addr = hdma_dev->base +
> - HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET;
> + addr = hdma_dev->queue_base +
> + HISI_DMA_Q_FSM_STS + index * HISI_DMA_Q_OFFSET;
>
> ret = readl_relaxed_poll_timeout(addr, tmp,
> - FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN,
> + FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) != RUN,
> HISI_DMA_POLL_Q_STS_DELAY_US,
> HISI_DMA_POLL_Q_STS_TIME_OUT_US);
> if (ret) {
> @@ -217,7 +332,7 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
> }
>
> ret = readl_relaxed_poll_timeout(addr, tmp,
> - FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE,
> + FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) == IDLE,
> HISI_DMA_POLL_Q_STS_DELAY_US,
> HISI_DMA_POLL_Q_STS_TIME_OUT_US);
> if (ret) {
> @@ -300,8 +415,8 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
> chan->sq_tail = (chan->sq_tail + 1) % hdma_dev->chan_depth;
>
> /* update sq_tail to trigger a new task */
> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, chan->qp_num,
> - chan->sq_tail);
> + hisi_dma_chan_write(hdma_dev->queue_base, HISI_DMA_Q_SQ_TAIL_PTR,
> + chan->qp_num, chan->sq_tail);
> }
>
> static void hisi_dma_issue_pending(struct dma_chan *c)
> @@ -375,26 +490,86 @@ static int hisi_dma_alloc_qps_mem(struct hisi_dma_dev *hdma_dev)
> static void hisi_dma_init_hw_qp(struct hisi_dma_dev *hdma_dev, u32 index)
> {
> struct hisi_dma_chan *chan = &hdma_dev->chan[index];
> + void __iomem *q_base = hdma_dev->queue_base;
> u32 hw_depth = hdma_dev->chan_depth - 1;
> - void __iomem *base = hdma_dev->base;
> + void __iomem *addr;
> + u32 tmp;
>
> /* set sq, cq base */
> - hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_L, index,
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_L, index,
> lower_32_bits(chan->sq_dma));
> - hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_H, index,
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_H, index,
> upper_32_bits(chan->sq_dma));
> - hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_L, index,
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_L, index,
> lower_32_bits(chan->cq_dma));
> - hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_H, index,
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_H, index,
> upper_32_bits(chan->cq_dma));
>
> /* set sq, cq depth */
> - hisi_dma_chan_write(base, HISI_DMA_SQ_DEPTH, index, hw_depth);
> - hisi_dma_chan_write(base, HISI_DMA_CQ_DEPTH, index, hw_depth);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_DEPTH, index, hw_depth);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_DEPTH, index, hw_depth);
>
> /* init sq tail and cq head */
> - hisi_dma_chan_write(base, HISI_DMA_SQ_TAIL_PTR, index, 0);
> - hisi_dma_chan_write(base, HISI_DMA_CQ_HEAD_PTR, index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
> +
> + /* init error interrupt stats */
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM0, index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM1, index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM2, index, 0);
> +
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM3,
> + index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM4,
> + index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM5,
> + index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM6,
> + index, 0);
> + /*
> + * init SQ/CQ direction selecting register.
> + * "0" is to local side and "1" is to remote side.
> + */
> + addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT, 0);
> +
> + /*
> + * 0 - Continue to next descriptor if error occurs.
> + * 1 - Abort the DMA queue if error occurs.
> + */
> + hisi_dma_update_bit(addr,
> + HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN, 0);
> + } else {
> + addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
> +
> + /*
> + * init SQ/CQ direction selecting register.
> + * "0" is to local side and "1" is to remote side.
> + */
> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT, 0);
> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT, 0);
> +
> + /*
> + * 0 - Continue to next descriptor if error occurs.
> + * 1 - Abort the DMA queue if error occurs.
> + */
> +
> + tmp = readl_relaxed(addr);
> + tmp &= ~HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN;
> + writel_relaxed(tmp, addr);
> +
> + /*
> + * 0 - dma should process FLR whith CPU.
> + * 1 - dma not process FLR, only cpu process FLR.
> + */
> + addr = q_base + HISI_DMA_HIP09_DMA_FLR_DISABLE +
> + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_DMA_FLR_DISABLE_B, 0);
> +
> + addr = q_base + HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE, 1);
> + }
> }
>
> static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
> @@ -438,17 +613,18 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
> struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
> struct hisi_dma_desc *desc;
> struct hisi_dma_cqe *cqe;
> + void __iomem *q_base;
>
> spin_lock(&chan->vc.lock);
>
> desc = chan->desc;
> cqe = chan->cq + chan->cq_head;
> + q_base = hdma_dev->queue_base;
> if (desc) {
> chan->cq_head = (chan->cq_head + 1) %
> hdma_dev->chan_depth;
> - hisi_dma_chan_write(hdma_dev->base,
> - HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
> - chan->cq_head);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
> + chan->qp_num, chan->cq_head);
> if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
> vchan_cookie_complete(&desc->vd);
> hisi_dma_start_transfer(chan);
> @@ -508,16 +684,58 @@ static void hisi_dma_disable_hw_channels(void *data)
> static void hisi_dma_set_mode(struct hisi_dma_dev *hdma_dev,
> enum hisi_dma_mode mode)
> {
> - writel_relaxed(mode == RC ? 1 : 0, hdma_dev->base + HISI_DMA_MODE);
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
> + writel_relaxed(mode == RC ? 1 : 0,
> + hdma_dev->base + HISI_DMA_HIP08_MODE);
> +}
> +
> +static void hisi_dma_init_hw(struct hisi_dma_dev *hdma_dev)
> +{
> + void __iomem *addr;
> + int i;
> +
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP09) {
> + for (i = 0; i < HISI_DMA_HIP09_MAX_PORT_NUM; i++) {
> + addr = hdma_dev->base + HISI_DMA_HIP09_PORT_CFG_REG(i);
> + hisi_dma_update_bit(addr,
> + HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B, 1);
> + }
> + }
> +}
> +
> +static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
> +{
> + struct dma_device *dma_dev;
> +
> + dma_dev = &hdma_dev->dma_dev;
> + dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
> + dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
> + dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
> + dma_dev->device_tx_status = hisi_dma_tx_status;
> + dma_dev->device_issue_pending = hisi_dma_issue_pending;
> + dma_dev->device_terminate_all = hisi_dma_terminate_all;
> + dma_dev->device_synchronize = hisi_dma_synchronize;
> + dma_dev->directions = BIT(DMA_MEM_TO_MEM);
> + dma_dev->dev = &hdma_dev->pdev->dev;
> + INIT_LIST_HEAD(&dma_dev->channels);
> }
>
> static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> {
> + enum hisi_dma_reg_layout reg_layout;
> struct device *dev = &pdev->dev;
> struct hisi_dma_dev *hdma_dev;
> struct dma_device *dma_dev;
> + u32 chan_num;
> + u32 msi_num;
> int ret;
>
> + reg_layout = hisi_dma_get_reg_layout(pdev);
> + if (reg_layout == HISI_DMA_REG_LAYOUT_INVALID) {
> + dev_err(dev, "unsupported device!\n");
> + return -EINVAL;
> + }
> +
> ret = pcim_enable_device(pdev);
> if (ret) {
> dev_err(dev, "failed to enable device mem!\n");
> @@ -534,40 +752,37 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> if (ret)
> return ret;
>
> - hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, HISI_DMA_CHAN_NUM), GFP_KERNEL);
> + chan_num = hisi_dma_get_chan_num(pdev);
> + hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, chan_num),
> + GFP_KERNEL);
> if (!hdma_dev)
> return -EINVAL;
>
> hdma_dev->base = pcim_iomap_table(pdev)[PCI_BAR_2];
> hdma_dev->pdev = pdev;
> - hdma_dev->chan_num = HISI_DMA_CHAN_NUM;
> hdma_dev->chan_depth = HISI_DMA_Q_DEPTH_VAL;
> + hdma_dev->chan_num = chan_num;
> + hdma_dev->reg_layout = reg_layout;
> + hdma_dev->queue_base = hdma_dev->base + hisi_dma_get_queue_base(pdev);
>
> pci_set_drvdata(pdev, hdma_dev);
> pci_set_master(pdev);
>
> + msi_num = hisi_dma_get_msi_num(pdev);
> +
> /* This will be freed by 'pcim_release()'. See 'pcim_enable_device()' */
> - ret = pci_alloc_irq_vectors(pdev, HISI_DMA_MSI_NUM, HISI_DMA_MSI_NUM,
> - PCI_IRQ_MSI);
> + ret = pci_alloc_irq_vectors(pdev, msi_num, msi_num, PCI_IRQ_MSI);
> if (ret < 0) {
> dev_err(dev, "Failed to allocate MSI vectors!\n");
> return ret;
> }
>
> - dma_dev = &hdma_dev->dma_dev;
> - dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
> - dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
> - dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
> - dma_dev->device_tx_status = hisi_dma_tx_status;
> - dma_dev->device_issue_pending = hisi_dma_issue_pending;
> - dma_dev->device_terminate_all = hisi_dma_terminate_all;
> - dma_dev->device_synchronize = hisi_dma_synchronize;
> - dma_dev->directions = BIT(DMA_MEM_TO_MEM);
> - dma_dev->dev = dev;
> - INIT_LIST_HEAD(&dma_dev->channels);
> + hisi_dma_init_dma_dev(hdma_dev);
>
> hisi_dma_set_mode(hdma_dev, RC);
>
> + hisi_dma_init_hw(hdma_dev);
> +
> ret = hisi_dma_enable_hw_channels(hdma_dev);
> if (ret < 0) {
> dev_err(dev, "failed to enable hw channel!\n");
> @@ -579,6 +794,7 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> if (ret)
> return ret;
>
> + dma_dev = &hdma_dev->dma_dev;
> ret = dmaenginem_async_device_register(dma_dev);
> if (ret < 0)
> dev_err(dev, "failed to register device!\n");
> --
> 2.33.0
--
~Vinod
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v3 0/7] dmaengine: hisilicon: Add support for hisi dma driver
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (8 preceding siblings ...)
2022-06-29 3:55 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
@ 2022-07-26 1:35 ` Jie Hai
2022-07-26 1:35 ` [PATCH v3 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
` (6 more replies)
2022-08-02 10:12 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-08-05 8:25 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
11 siblings, 7 replies; 64+ messages in thread
From: Jie Hai @ 2022-07-26 1:35 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they share the
same pci device id but different pci revision and register layouts.
The original version supports HiSilicon IP08 but not HiSilicon IP09.
This series support DMA driver for HIP08 and HIP09:
1. Fix bugs for HIP08 DMA driver
- Disable hardware channels when driver detached
- Update cq_head whenever accessed it
- Support multi-thread for one DMA channel
2. Use macros instead of magic number
3. Add support for HIP09 DMA driver
4. Add debugfs for HIP08 and HIP09 DMA driver
5. Add myself as maintainer of hisi_dma.c
Changes since version 2:
- fix unnecessary line breaks
- fix register bit with BIT/GENMASK and adjust hisi_dma_update_bit to it
- remove "Reported-by" in commit message
- use dmaengine root instead of hisi_dma root
- ignore errors for creating debugfs
Changes since version 1:
- remove error changes casuse compile failure
- remove reduldant "*" in comment
- remove debugfs-hisi-dma doc and path in MAINTAINERS
Jie Hai (7):
dmaengine: hisilicon: Disable channels when unregister hisi_dma
dmaengine: hisilicon: Fix CQ head update
dmaengine: hisilicon: Add multi-thread support for a DMA channel
dmaengine: hisilicon: Use macros instead of magic number
dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
dmaengine: hisilicon: dump regs to debugfs
MAINTAINERS: Add myself as maintainer for hisi_dma
MAINTAINERS | 1 +
drivers/dma/hisi_dma.c | 651 +++++++++++++++++++++++++++++++++++------
2 files changed, 556 insertions(+), 96 deletions(-)
--
2.33.0
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v3 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma
2022-07-26 1:35 ` [PATCH v3 " Jie Hai
@ 2022-07-26 1:35 ` Jie Hai
2022-07-26 1:35 ` [PATCH v3 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
` (5 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-07-26 1:35 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
When hisi_dma is unloaded or unbinded, all of channels should be
disabled. This patch disables DMA channels when driver is unloaded
or unbinded.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 43817ced3a3e..98bc488893cc 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -180,7 +180,8 @@ static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
}
-static void hisi_dma_reset_hw_chan(struct hisi_dma_chan *chan)
+static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
+ bool disable)
{
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
u32 index = chan->qp_num, tmp;
@@ -201,8 +202,11 @@ static void hisi_dma_reset_hw_chan(struct hisi_dma_chan *chan)
hisi_dma_do_reset(hdma_dev, index);
hisi_dma_reset_qp_point(hdma_dev, index);
hisi_dma_pause_dma(hdma_dev, index, false);
- hisi_dma_enable_dma(hdma_dev, index, true);
- hisi_dma_unmask_irq(hdma_dev, index);
+
+ if (!disable) {
+ hisi_dma_enable_dma(hdma_dev, index, true);
+ hisi_dma_unmask_irq(hdma_dev, index);
+ }
ret = readl_relaxed_poll_timeout(hdma_dev->base +
HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
@@ -218,7 +222,7 @@ static void hisi_dma_free_chan_resources(struct dma_chan *c)
struct hisi_dma_chan *chan = to_hisi_dma_chan(c);
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
- hisi_dma_reset_hw_chan(chan);
+ hisi_dma_reset_or_disable_hw_chan(chan, false);
vchan_free_chan_resources(&chan->vc);
memset(chan->sq, 0, sizeof(struct hisi_dma_sqe) * hdma_dev->chan_depth);
@@ -394,7 +398,7 @@ static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
static void hisi_dma_disable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- hisi_dma_reset_hw_chan(&hdma_dev->chan[qp_index]);
+ hisi_dma_reset_or_disable_hw_chan(&hdma_dev->chan[qp_index], true);
}
static void hisi_dma_enable_qps(struct hisi_dma_dev *hdma_dev)
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v3 2/7] dmaengine: hisilicon: Fix CQ head update
2022-07-26 1:35 ` [PATCH v3 " Jie Hai
2022-07-26 1:35 ` [PATCH v3 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
@ 2022-07-26 1:35 ` Jie Hai
2022-07-26 1:35 ` [PATCH v3 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
` (4 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-07-26 1:35 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
After completion of data transfer of one or multiple descriptors,
the completion status and the current head pointer to submission
queue are written into the CQ and interrupt can be generated to
inform the software. In interrupt process CQ is read and cq_head
is updated.
hisi_dma_irq updates cq_head only when the completion status is
success. When an abnormal interrupt reports, cq_head will not update
which will cause subsequent interrupt processes read the error CQ
and never report the correct status.
This patch updates cq_head whenever CQ is accessed.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 98bc488893cc..837f7e4adfa6 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -436,12 +436,10 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
desc = chan->desc;
cqe = chan->cq + chan->cq_head;
if (desc) {
+ chan->cq_head = (chan->cq_head + 1) % hdma_dev->chan_depth;
+ hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR,
+ chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
- chan->cq_head = (chan->cq_head + 1) %
- hdma_dev->chan_depth;
- hisi_dma_chan_write(hdma_dev->base,
- HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
- chan->cq_head);
vchan_cookie_complete(&desc->vd);
} else {
dev_err(&hdma_dev->pdev->dev, "task error!\n");
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v3 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel
2022-07-26 1:35 ` [PATCH v3 " Jie Hai
2022-07-26 1:35 ` [PATCH v3 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
2022-07-26 1:35 ` [PATCH v3 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
@ 2022-07-26 1:35 ` Jie Hai
2022-07-26 1:35 ` [PATCH v3 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
` (3 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-07-26 1:35 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
When we get a DMA channel and try to use it in multiple threads it
will cause oops and hanging the system.
% echo 100 > /sys/module/dmatest/parameters/threads_per_chan
% echo 100 > /sys/module/dmatest/parameters/iterations
% echo 1 > /sys/module/dmatest/parameters/run
[383493.327077] Unable to handle kernel paging request at virtual
address dead000000000108
[383493.335103] Mem abort info:
[383493.335103] ESR = 0x96000044
[383493.335105] EC = 0x25: DABT (current EL), IL = 32 bits
[383493.335107] SET = 0, FnV = 0
[383493.335108] EA = 0, S1PTW = 0
[383493.335109] FSC = 0x04: level 0 translation fault
[383493.335110] Data abort info:
[383493.335111] ISV = 0, ISS = 0x00000044
[383493.364739] CM = 0, WnR = 1
[383493.367793] [dead000000000108] address between user and kernel
address ranges
[383493.375021] Internal error: Oops: 96000044 [#1] PREEMPT SMP
[383493.437574] CPU: 63 PID: 27895 Comm: dma0chan0-copy2 Kdump:
loaded Tainted: GO 5.17.0-rc4+ #2
[383493.457851] pstate: 204000c9 (nzCv daIF +PAN -UAO -TCO -DIT
-SSBS BTYPE=--)
[383493.465331] pc : vchan_tx_submit+0x64/0xa0
[383493.469957] lr : vchan_tx_submit+0x34/0xa0
This occurs because the transmission timed out, and that's due
to data race. Each thread rewrite channels's descriptor as soon as
device_issue_pending is called. It leads to the situation that
the driver thinks that it uses the right descriptor in interrupt
handler while channels's descriptor has been changed by other
thread. The descriptor which in fact reported interrupt will not
be handled any more, as well as its tx->callback.
That's why timeout reports.
With current fixes channels' descriptor changes it's value only
when it has been used. A new descriptor is acquired from
vc->desc_issued queue that is already filled with descriptors
that are ready to be sent. Threads have no direct access to DMA
channel descriptor. In case of channel's descriptor is busy, try
to submit to HW again when a descriptor is completed. In this case,
vc->desc_issued may be empty when hisi_dma_start_transfer is called,
so delete error reporting on this. Now it is just possible to queue
a descriptor for further processing.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 837f7e4adfa6..0233b42143c7 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -271,7 +271,6 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
vd = vchan_next_desc(&chan->vc);
if (!vd) {
- dev_err(&hdma_dev->pdev->dev, "no issued task!\n");
chan->desc = NULL;
return;
}
@@ -303,7 +302,7 @@ static void hisi_dma_issue_pending(struct dma_chan *c)
spin_lock_irqsave(&chan->vc.lock, flags);
- if (vchan_issue_pending(&chan->vc))
+ if (vchan_issue_pending(&chan->vc) && !chan->desc)
hisi_dma_start_transfer(chan);
spin_unlock_irqrestore(&chan->vc.lock, flags);
@@ -441,11 +440,10 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
vchan_cookie_complete(&desc->vd);
+ hisi_dma_start_transfer(chan);
} else {
dev_err(&hdma_dev->pdev->dev, "task error!\n");
}
-
- chan->desc = NULL;
}
spin_unlock(&chan->vc.lock);
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v3 4/7] dmaengine: hisilicon: Use macros instead of magic number
2022-07-26 1:35 ` [PATCH v3 " Jie Hai
` (2 preceding siblings ...)
2022-07-26 1:35 ` [PATCH v3 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
@ 2022-07-26 1:35 ` Jie Hai
2022-07-26 1:35 ` [PATCH v3 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
` (2 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-07-26 1:35 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
readl_relaxed_poll_timeout() uses magic numbers 10 and 1000, which
indicate maximum time to sleep between reads in us and timeout in us,
respectively.
Use macros HISI_DMA_POLL_Q_STS_DELAY_US and
HISI_DMA_POLL_Q_STS_TIME_OUT_US instead of these two numbers.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 19 +++++++++++++------
1 file changed, 13 insertions(+), 6 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 0233b42143c7..5d62fe62ba00 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -36,6 +36,9 @@
#define PCI_BAR_2 2
+#define HISI_DMA_POLL_Q_STS_DELAY_US 10
+#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+
enum hisi_dma_mode {
EP = 0,
RC,
@@ -185,15 +188,19 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
{
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
u32 index = chan->qp_num, tmp;
+ void __iomem *addr;
int ret;
hisi_dma_pause_dma(hdma_dev, index, true);
hisi_dma_enable_dma(hdma_dev, index, false);
hisi_dma_mask_irq(hdma_dev, index);
- ret = readl_relaxed_poll_timeout(hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN, 10, 1000);
+ addr = hdma_dev->base +
+ HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET;
+
+ ret = readl_relaxed_poll_timeout(addr, tmp,
+ FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN,
+ HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "disable channel timeout!\n");
WARN_ON(1);
@@ -208,9 +215,9 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
hisi_dma_unmask_irq(hdma_dev, index);
}
- ret = readl_relaxed_poll_timeout(hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE, 10, 1000);
+ ret = readl_relaxed_poll_timeout(addr, tmp,
+ FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE,
+ HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "reset channel timeout!\n");
WARN_ON(1);
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v3 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
2022-07-26 1:35 ` [PATCH v3 " Jie Hai
` (3 preceding siblings ...)
2022-07-26 1:35 ` [PATCH v3 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
@ 2022-07-26 1:35 ` Jie Hai
2022-07-26 1:35 ` [PATCH v3 6/7] dmaengine: hisilicon: dump regs to debugfs Jie Hai
2022-07-26 1:35 ` [PATCH v3 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-07-26 1:35 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they
have the same pci device id but different pci revision.
Unfortunately, they have different register layouts, so
the origin driver cannot run on HiSilicon IP09 correctly.
This patch enables the driver to adapt to HiSilicon IP09.
HiSilicon IP09 offers 4 channels, each channel has a send
queue, a complete queue and an interrupt to help to do tasks.
This DMA engine can do memory copy between memory blocks.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 379 ++++++++++++++++++++++++++++++++---------
1 file changed, 298 insertions(+), 81 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 5d62fe62ba00..5fa3b6fa0529 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
-/* Copyright(c) 2019 HiSilicon Limited. */
+/* Copyright(c) 2019-2022 HiSilicon Limited. */
+
#include <linux/bitfield.h>
#include <linux/dmaengine.h>
#include <linux/init.h>
@@ -9,35 +10,85 @@
#include <linux/spinlock.h>
#include "virt-dma.h"
-#define HISI_DMA_SQ_BASE_L 0x0
-#define HISI_DMA_SQ_BASE_H 0x4
-#define HISI_DMA_SQ_DEPTH 0x8
-#define HISI_DMA_SQ_TAIL_PTR 0xc
-#define HISI_DMA_CQ_BASE_L 0x10
-#define HISI_DMA_CQ_BASE_H 0x14
-#define HISI_DMA_CQ_DEPTH 0x18
-#define HISI_DMA_CQ_HEAD_PTR 0x1c
-#define HISI_DMA_CTRL0 0x20
-#define HISI_DMA_CTRL0_QUEUE_EN_S 0
-#define HISI_DMA_CTRL0_QUEUE_PAUSE_S 4
-#define HISI_DMA_CTRL1 0x24
-#define HISI_DMA_CTRL1_QUEUE_RESET_S 0
-#define HISI_DMA_Q_FSM_STS 0x30
-#define HISI_DMA_FSM_STS_MASK GENMASK(3, 0)
-#define HISI_DMA_INT_STS 0x40
-#define HISI_DMA_INT_STS_MASK GENMASK(12, 0)
-#define HISI_DMA_INT_MSK 0x44
-#define HISI_DMA_MODE 0x217c
-#define HISI_DMA_OFFSET 0x100
-
-#define HISI_DMA_MSI_NUM 32
-#define HISI_DMA_CHAN_NUM 30
-#define HISI_DMA_Q_DEPTH_VAL 1024
-
-#define PCI_BAR_2 2
-
-#define HISI_DMA_POLL_Q_STS_DELAY_US 10
-#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+/* HiSilicon DMA register common field define */
+#define HISI_DMA_Q_SQ_BASE_L 0x0
+#define HISI_DMA_Q_SQ_BASE_H 0x4
+#define HISI_DMA_Q_SQ_DEPTH 0x8
+#define HISI_DMA_Q_SQ_TAIL_PTR 0xc
+#define HISI_DMA_Q_CQ_BASE_L 0x10
+#define HISI_DMA_Q_CQ_BASE_H 0x14
+#define HISI_DMA_Q_CQ_DEPTH 0x18
+#define HISI_DMA_Q_CQ_HEAD_PTR 0x1c
+#define HISI_DMA_Q_CTRL0 0x20
+#define HISI_DMA_Q_CTRL0_QUEUE_EN BIT(0)
+#define HISI_DMA_Q_CTRL0_QUEUE_PAUSE BIT(4)
+#define HISI_DMA_Q_CTRL1 0x24
+#define HISI_DMA_Q_CTRL1_QUEUE_RESET BIT(0)
+#define HISI_DMA_Q_FSM_STS 0x30
+#define HISI_DMA_Q_FSM_STS_MASK GENMASK(3, 0)
+#define HISI_DMA_Q_ERR_INT_NUM0 0x84
+#define HISI_DMA_Q_ERR_INT_NUM1 0x88
+#define HISI_DMA_Q_ERR_INT_NUM2 0x8c
+
+/* HiSilicon IP08 DMA register and field define */
+#define HISI_DMA_HIP08_MODE 0x217C
+#define HISI_DMA_HIP08_Q_BASE 0x0
+#define HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN BIT(2)
+#define HISI_DMA_HIP08_Q_INT_STS 0x40
+#define HISI_DMA_HIP08_Q_INT_MSK 0x44
+#define HISI_DMA_HIP08_Q_INT_STS_MASK GENMASK(14, 0)
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM3 0x90
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM4 0x94
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM5 0x98
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM6 0x48
+#define HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT BIT(24)
+
+/* HiSilicon IP09 DMA register and field define */
+#define HISI_DMA_HIP09_DMA_FLR_DISABLE 0xA00
+#define HISI_DMA_HIP09_DMA_FLR_DISABLE_B BIT(0)
+#define HISI_DMA_HIP09_Q_BASE 0x2000
+#define HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN GENMASK(31, 28)
+#define HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT BIT(26)
+#define HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT BIT(27)
+#define HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE BIT(2)
+#define HISI_DMA_HIP09_Q_INT_STS 0x40
+#define HISI_DMA_HIP09_Q_INT_MSK 0x44
+#define HISI_DMA_HIP09_Q_INT_STS_MASK 0x1
+#define HISI_DMA_HIP09_Q_ERR_INT_STS 0x48
+#define HISI_DMA_HIP09_Q_ERR_INT_MSK 0x4C
+#define HISI_DMA_HIP09_Q_ERR_INT_STS_MASK GENMASK(18, 1)
+#define HISI_DMA_HIP09_PORT_CFG_REG(port_id) (0x800 + \
+ (port_id) * 0x20)
+#define HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B BIT(16)
+
+#define HISI_DMA_HIP09_MAX_PORT_NUM 16
+
+#define HISI_DMA_HIP08_MSI_NUM 32
+#define HISI_DMA_HIP08_CHAN_NUM 30
+#define HISI_DMA_HIP09_MSI_NUM 4
+#define HISI_DMA_HIP09_CHAN_NUM 4
+#define HISI_DMA_REVISION_HIP08B 0x21
+#define HISI_DMA_REVISION_HIP09A 0x30
+
+#define HISI_DMA_Q_OFFSET 0x100
+#define HISI_DMA_Q_DEPTH_VAL 1024
+
+#define PCI_BAR_2 2
+
+#define HISI_DMA_POLL_Q_STS_DELAY_US 10
+#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+
+/*
+ * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
+ * have the same pci device id but different pci revision.
+ * Unfortunately, they have different register layouts, so two layout
+ * enumerations are defined.
+ */
+enum hisi_dma_reg_layout {
+ HISI_DMA_REG_LAYOUT_INVALID = 0,
+ HISI_DMA_REG_LAYOUT_HIP08,
+ HISI_DMA_REG_LAYOUT_HIP09
+};
enum hisi_dma_mode {
EP = 0,
@@ -108,9 +159,45 @@ struct hisi_dma_dev {
struct dma_device dma_dev;
u32 chan_num;
u32 chan_depth;
+ enum hisi_dma_reg_layout reg_layout;
+ void __iomem *queue_base; /* queue region start of register */
struct hisi_dma_chan chan[];
};
+static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_REG_LAYOUT_HIP08;
+ else if (pdev->revision >= HISI_DMA_REVISION_HIP09A)
+ return HISI_DMA_REG_LAYOUT_HIP09;
+
+ return HISI_DMA_REG_LAYOUT_INVALID;
+}
+
+static u32 hisi_dma_get_chan_num(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_CHAN_NUM;
+
+ return HISI_DMA_HIP09_CHAN_NUM;
+}
+
+static u32 hisi_dma_get_msi_num(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_MSI_NUM;
+
+ return HISI_DMA_HIP09_MSI_NUM;
+}
+
+static u32 hisi_dma_get_queue_base(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_Q_BASE;
+
+ return HISI_DMA_HIP09_Q_BASE;
+}
+
static inline struct hisi_dma_chan *to_hisi_dma_chan(struct dma_chan *c)
{
return container_of(c, struct hisi_dma_chan, vc.chan);
@@ -124,7 +211,7 @@ static inline struct hisi_dma_desc *to_hisi_dma_desc(struct virt_dma_desc *vd)
static inline void hisi_dma_chan_write(void __iomem *base, u32 reg, u32 index,
u32 val)
{
- writel_relaxed(val, base + reg + index * HISI_DMA_OFFSET);
+ writel_relaxed(val, base + reg + index * HISI_DMA_Q_OFFSET);
}
static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
@@ -132,55 +219,83 @@ static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
u32 tmp;
tmp = readl_relaxed(addr);
- tmp = val ? tmp | BIT(pos) : tmp & ~BIT(pos);
+ tmp = val ? tmp | pos : tmp & ~pos;
writel_relaxed(tmp, addr);
}
static void hisi_dma_pause_dma(struct hisi_dma_dev *hdma_dev, u32 index,
bool pause)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_PAUSE_S, pause);
+ addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_PAUSE, pause);
}
static void hisi_dma_enable_dma(struct hisi_dma_dev *hdma_dev, u32 index,
bool enable)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_EN_S, enable);
+ addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_EN, enable);
}
static void hisi_dma_mask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_INT_MSK, qp_index,
- HISI_DMA_INT_STS_MASK);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
+ qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
+ else {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
+ qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
+ qp_index,
+ HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
+ }
}
static void hisi_dma_unmask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- void __iomem *base = hdma_dev->base;
-
- hisi_dma_chan_write(base, HISI_DMA_INT_STS, qp_index,
- HISI_DMA_INT_STS_MASK);
- hisi_dma_chan_write(base, HISI_DMA_INT_MSK, qp_index, 0);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_STS,
+ qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
+ qp_index, 0);
+ } else {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_STS,
+ qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_STS,
+ qp_index,
+ HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
+ qp_index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
+ qp_index, 0);
+ }
}
static void hisi_dma_do_reset(struct hisi_dma_dev *hdma_dev, u32 index)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL1 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL1_QUEUE_RESET_S, 1);
+ addr = hdma_dev->queue_base +
+ HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL1_QUEUE_RESET, 1);
}
static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
{
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, index, 0);
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
}
static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
@@ -196,10 +311,10 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
hisi_dma_mask_irq(hdma_dev, index);
addr = hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET;
+ HISI_DMA_Q_FSM_STS + index * HISI_DMA_Q_OFFSET;
ret = readl_relaxed_poll_timeout(addr, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN,
+ FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) != RUN,
HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "disable channel timeout!\n");
@@ -216,7 +331,7 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
}
ret = readl_relaxed_poll_timeout(addr, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE,
+ FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) == IDLE,
HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "reset channel timeout!\n");
@@ -298,8 +413,8 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
chan->sq_tail = (chan->sq_tail + 1) % hdma_dev->chan_depth;
/* update sq_tail to trigger a new task */
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, chan->qp_num,
- chan->sq_tail);
+ hisi_dma_chan_write(hdma_dev->queue_base, HISI_DMA_Q_SQ_TAIL_PTR,
+ chan->qp_num, chan->sq_tail);
}
static void hisi_dma_issue_pending(struct dma_chan *c)
@@ -373,26 +488,86 @@ static int hisi_dma_alloc_qps_mem(struct hisi_dma_dev *hdma_dev)
static void hisi_dma_init_hw_qp(struct hisi_dma_dev *hdma_dev, u32 index)
{
struct hisi_dma_chan *chan = &hdma_dev->chan[index];
+ void __iomem *q_base = hdma_dev->queue_base;
u32 hw_depth = hdma_dev->chan_depth - 1;
- void __iomem *base = hdma_dev->base;
+ void __iomem *addr;
+ u32 tmp;
/* set sq, cq base */
- hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_L, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_L, index,
lower_32_bits(chan->sq_dma));
- hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_H, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_H, index,
upper_32_bits(chan->sq_dma));
- hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_L, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_L, index,
lower_32_bits(chan->cq_dma));
- hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_H, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_H, index,
upper_32_bits(chan->cq_dma));
/* set sq, cq depth */
- hisi_dma_chan_write(base, HISI_DMA_SQ_DEPTH, index, hw_depth);
- hisi_dma_chan_write(base, HISI_DMA_CQ_DEPTH, index, hw_depth);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_DEPTH, index, hw_depth);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_DEPTH, index, hw_depth);
/* init sq tail and cq head */
- hisi_dma_chan_write(base, HISI_DMA_SQ_TAIL_PTR, index, 0);
- hisi_dma_chan_write(base, HISI_DMA_CQ_HEAD_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
+
+ /* init error interrupt stats */
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM0, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM1, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM2, index, 0);
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM3,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM4,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM5,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM6,
+ index, 0);
+ /*
+ * init SQ/CQ direction selecting register.
+ * "0" is to local side and "1" is to remote side.
+ */
+ addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT, 0);
+
+ /*
+ * 0 - Continue to next descriptor if error occurs.
+ * 1 - Abort the DMA queue if error occurs.
+ */
+ hisi_dma_update_bit(addr,
+ HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN, 0);
+ } else {
+ addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
+
+ /*
+ * init SQ/CQ direction selecting register.
+ * "0" is to local side and "1" is to remote side.
+ */
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT, 0);
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT, 0);
+
+ /*
+ * 0 - Continue to next descriptor if error occurs.
+ * 1 - Abort the DMA queue if error occurs.
+ */
+
+ tmp = readl_relaxed(addr);
+ tmp &= ~HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN;
+ writel_relaxed(tmp, addr);
+
+ /*
+ * 0 - dma should process FLR whith CPU.
+ * 1 - dma not process FLR, only cpu process FLR.
+ */
+ addr = q_base + HISI_DMA_HIP09_DMA_FLR_DISABLE +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_DMA_FLR_DISABLE_B, 0);
+
+ addr = q_base + HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE, 1);
+ }
}
static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
@@ -436,14 +611,16 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
struct hisi_dma_desc *desc;
struct hisi_dma_cqe *cqe;
+ void __iomem *q_base;
spin_lock(&chan->vc.lock);
desc = chan->desc;
cqe = chan->cq + chan->cq_head;
+ q_base = hdma_dev->queue_base;
if (desc) {
chan->cq_head = (chan->cq_head + 1) % hdma_dev->chan_depth;
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
vchan_cookie_complete(&desc->vd);
@@ -504,16 +681,58 @@ static void hisi_dma_disable_hw_channels(void *data)
static void hisi_dma_set_mode(struct hisi_dma_dev *hdma_dev,
enum hisi_dma_mode mode)
{
- writel_relaxed(mode == RC ? 1 : 0, hdma_dev->base + HISI_DMA_MODE);
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ writel_relaxed(mode == RC ? 1 : 0,
+ hdma_dev->base + HISI_DMA_HIP08_MODE);
+}
+
+static void hisi_dma_init_hw(struct hisi_dma_dev *hdma_dev)
+{
+ void __iomem *addr;
+ int i;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP09) {
+ for (i = 0; i < HISI_DMA_HIP09_MAX_PORT_NUM; i++) {
+ addr = hdma_dev->base + HISI_DMA_HIP09_PORT_CFG_REG(i);
+ hisi_dma_update_bit(addr,
+ HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B, 1);
+ }
+ }
+}
+
+static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
+{
+ struct dma_device *dma_dev;
+
+ dma_dev = &hdma_dev->dma_dev;
+ dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
+ dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
+ dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
+ dma_dev->device_tx_status = hisi_dma_tx_status;
+ dma_dev->device_issue_pending = hisi_dma_issue_pending;
+ dma_dev->device_terminate_all = hisi_dma_terminate_all;
+ dma_dev->device_synchronize = hisi_dma_synchronize;
+ dma_dev->directions = BIT(DMA_MEM_TO_MEM);
+ dma_dev->dev = &hdma_dev->pdev->dev;
+ INIT_LIST_HEAD(&dma_dev->channels);
}
static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
+ enum hisi_dma_reg_layout reg_layout;
struct device *dev = &pdev->dev;
struct hisi_dma_dev *hdma_dev;
struct dma_device *dma_dev;
+ u32 chan_num;
+ u32 msi_num;
int ret;
+ reg_layout = hisi_dma_get_reg_layout(pdev);
+ if (reg_layout == HISI_DMA_REG_LAYOUT_INVALID) {
+ dev_err(dev, "unsupported device!\n");
+ return -EINVAL;
+ }
+
ret = pcim_enable_device(pdev);
if (ret) {
dev_err(dev, "failed to enable device mem!\n");
@@ -530,40 +749,37 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (ret)
return ret;
- hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, HISI_DMA_CHAN_NUM), GFP_KERNEL);
+ chan_num = hisi_dma_get_chan_num(pdev);
+ hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, chan_num),
+ GFP_KERNEL);
if (!hdma_dev)
return -EINVAL;
hdma_dev->base = pcim_iomap_table(pdev)[PCI_BAR_2];
hdma_dev->pdev = pdev;
- hdma_dev->chan_num = HISI_DMA_CHAN_NUM;
hdma_dev->chan_depth = HISI_DMA_Q_DEPTH_VAL;
+ hdma_dev->chan_num = chan_num;
+ hdma_dev->reg_layout = reg_layout;
+ hdma_dev->queue_base = hdma_dev->base + hisi_dma_get_queue_base(pdev);
pci_set_drvdata(pdev, hdma_dev);
pci_set_master(pdev);
+ msi_num = hisi_dma_get_msi_num(pdev);
+
/* This will be freed by 'pcim_release()'. See 'pcim_enable_device()' */
- ret = pci_alloc_irq_vectors(pdev, HISI_DMA_MSI_NUM, HISI_DMA_MSI_NUM,
- PCI_IRQ_MSI);
+ ret = pci_alloc_irq_vectors(pdev, msi_num, msi_num, PCI_IRQ_MSI);
if (ret < 0) {
dev_err(dev, "Failed to allocate MSI vectors!\n");
return ret;
}
- dma_dev = &hdma_dev->dma_dev;
- dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
- dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
- dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
- dma_dev->device_tx_status = hisi_dma_tx_status;
- dma_dev->device_issue_pending = hisi_dma_issue_pending;
- dma_dev->device_terminate_all = hisi_dma_terminate_all;
- dma_dev->device_synchronize = hisi_dma_synchronize;
- dma_dev->directions = BIT(DMA_MEM_TO_MEM);
- dma_dev->dev = dev;
- INIT_LIST_HEAD(&dma_dev->channels);
+ hisi_dma_init_dma_dev(hdma_dev);
hisi_dma_set_mode(hdma_dev, RC);
+ hisi_dma_init_hw(hdma_dev);
+
ret = hisi_dma_enable_hw_channels(hdma_dev);
if (ret < 0) {
dev_err(dev, "failed to enable hw channel!\n");
@@ -575,6 +791,7 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (ret)
return ret;
+ dma_dev = &hdma_dev->dma_dev;
ret = dmaenginem_async_device_register(dma_dev);
if (ret < 0)
dev_err(dev, "failed to register device!\n");
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v3 6/7] dmaengine: hisilicon: dump regs to debugfs
2022-07-26 1:35 ` [PATCH v3 " Jie Hai
` (4 preceding siblings ...)
2022-07-26 1:35 ` [PATCH v3 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
@ 2022-07-26 1:35 ` Jie Hai
2022-07-26 1:35 ` [PATCH v3 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-07-26 1:35 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
This patch adds dump of registers with debugfs for HIP08
and HIP09 DMA driver.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
drivers/dma/hisi_dma.c | 239 ++++++++++++++++++++++++++++++++++++++++-
1 file changed, 237 insertions(+), 2 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 5fa3b6fa0529..c59d5f7bc15d 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -78,6 +78,8 @@
#define HISI_DMA_POLL_Q_STS_DELAY_US 10
#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+#define HISI_DMA_MAX_DIR_NAME_LEN 128
+
/*
* The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
* have the same pci device id but different pci revision.
@@ -164,6 +166,123 @@ struct hisi_dma_dev {
struct hisi_dma_chan chan[];
};
+#ifdef CONFIG_DEBUG_FS
+
+static const struct debugfs_reg32 hisi_dma_comm_chan_regs[] = {
+ {"DMA_QUEUE_SQ_DEPTH ", 0x0008ull},
+ {"DMA_QUEUE_SQ_TAIL_PTR ", 0x000Cull},
+ {"DMA_QUEUE_CQ_DEPTH ", 0x0018ull},
+ {"DMA_QUEUE_CQ_HEAD_PTR ", 0x001Cull},
+ {"DMA_QUEUE_CTRL0 ", 0x0020ull},
+ {"DMA_QUEUE_CTRL1 ", 0x0024ull},
+ {"DMA_QUEUE_FSM_STS ", 0x0030ull},
+ {"DMA_QUEUE_SQ_STS ", 0x0034ull},
+ {"DMA_QUEUE_CQ_TAIL_PTR ", 0x003Cull},
+ {"DMA_QUEUE_INT_STS ", 0x0040ull},
+ {"DMA_QUEUE_INT_MSK ", 0x0044ull},
+ {"DMA_QUEUE_INT_RO ", 0x006Cull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip08_chan_regs[] = {
+ {"DMA_QUEUE_BYTE_CNT ", 0x0038ull},
+ {"DMA_ERR_INT_NUM6 ", 0x0048ull},
+ {"DMA_QUEUE_DESP0 ", 0x0050ull},
+ {"DMA_QUEUE_DESP1 ", 0x0054ull},
+ {"DMA_QUEUE_DESP2 ", 0x0058ull},
+ {"DMA_QUEUE_DESP3 ", 0x005Cull},
+ {"DMA_QUEUE_DESP4 ", 0x0074ull},
+ {"DMA_QUEUE_DESP5 ", 0x0078ull},
+ {"DMA_QUEUE_DESP6 ", 0x007Cull},
+ {"DMA_QUEUE_DESP7 ", 0x0080ull},
+ {"DMA_ERR_INT_NUM0 ", 0x0084ull},
+ {"DMA_ERR_INT_NUM1 ", 0x0088ull},
+ {"DMA_ERR_INT_NUM2 ", 0x008Cull},
+ {"DMA_ERR_INT_NUM3 ", 0x0090ull},
+ {"DMA_ERR_INT_NUM4 ", 0x0094ull},
+ {"DMA_ERR_INT_NUM5 ", 0x0098ull},
+ {"DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip09_chan_regs[] = {
+ {"DMA_QUEUE_ERR_INT_STS ", 0x0048ull},
+ {"DMA_QUEUE_ERR_INT_MSK ", 0x004Cull},
+ {"DFX_SQ_READ_ERR_PTR ", 0x0068ull},
+ {"DFX_DMA_ERR_INT_NUM0 ", 0x0084ull},
+ {"DFX_DMA_ERR_INT_NUM1 ", 0x0088ull},
+ {"DFX_DMA_ERR_INT_NUM2 ", 0x008Cull},
+ {"DFX_DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip08_comm_regs[] = {
+ {"DMA_ECC_ERR_ADDR ", 0x2004ull},
+ {"DMA_ECC_ECC_CNT ", 0x2014ull},
+ {"COMMON_AND_CH_ERR_STS ", 0x2030ull},
+ {"LOCAL_CPL_ID_STS_0 ", 0x20E0ull},
+ {"LOCAL_CPL_ID_STS_1 ", 0x20E4ull},
+ {"LOCAL_CPL_ID_STS_2 ", 0x20E8ull},
+ {"LOCAL_CPL_ID_STS_3 ", 0x20ECull},
+ {"LOCAL_TLP_NUM ", 0x2158ull},
+ {"SQCQ_TLP_NUM ", 0x2164ull},
+ {"CPL_NUM ", 0x2168ull},
+ {"INF_BACK_PRESS_STS ", 0x2170ull},
+ {"DMA_CH_RAS_LEVEL ", 0x2184ull},
+ {"DMA_CM_RAS_LEVEL ", 0x2188ull},
+ {"DMA_CH_ERR_STS ", 0x2190ull},
+ {"DMA_CH_DONE_STS ", 0x2194ull},
+ {"DMA_SQ_TAG_STS_0 ", 0x21A0ull},
+ {"DMA_SQ_TAG_STS_1 ", 0x21A4ull},
+ {"DMA_SQ_TAG_STS_2 ", 0x21A8ull},
+ {"DMA_SQ_TAG_STS_3 ", 0x21ACull},
+ {"LOCAL_P_ID_STS_0 ", 0x21B0ull},
+ {"LOCAL_P_ID_STS_1 ", 0x21B4ull},
+ {"LOCAL_P_ID_STS_2 ", 0x21B8ull},
+ {"LOCAL_P_ID_STS_3 ", 0x21BCull},
+ {"DMA_PREBUFF_INFO_0 ", 0x2200ull},
+ {"DMA_CM_TABLE_INFO_0 ", 0x2220ull},
+ {"DMA_CM_CE_RO ", 0x2244ull},
+ {"DMA_CM_NFE_RO ", 0x2248ull},
+ {"DMA_CM_FE_RO ", 0x224Cull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip09_comm_regs[] = {
+ {"COMMON_AND_CH_ERR_STS ", 0x0030ull},
+ {"DMA_PORT_IDLE_STS ", 0x0150ull},
+ {"DMA_CH_RAS_LEVEL ", 0x0184ull},
+ {"DMA_CM_RAS_LEVEL ", 0x0188ull},
+ {"DMA_CM_CE_RO ", 0x0244ull},
+ {"DMA_CM_NFE_RO ", 0x0248ull},
+ {"DMA_CM_FE_RO ", 0x024Cull},
+ {"DFX_INF_BACK_PRESS_STS0 ", 0x1A40ull},
+ {"DFX_INF_BACK_PRESS_STS1 ", 0x1A44ull},
+ {"DFX_INF_BACK_PRESS_STS2 ", 0x1A48ull},
+ {"DFX_DMA_WRR_DISABLE ", 0x1A4Cull},
+ {"DFX_PA_REQ_TLP_NUM ", 0x1C00ull},
+ {"DFX_PA_BACK_TLP_NUM ", 0x1C04ull},
+ {"DFX_PA_RETRY_TLP_NUM ", 0x1C08ull},
+ {"DFX_LOCAL_NP_TLP_NUM ", 0x1C0Cull},
+ {"DFX_LOCAL_CPL_HEAD_TLP_NUM ", 0x1C10ull},
+ {"DFX_LOCAL_CPL_DATA_TLP_NUM ", 0x1C14ull},
+ {"DFX_LOCAL_CPL_EXT_DATA_TLP_NUM ", 0x1C18ull},
+ {"DFX_LOCAL_P_HEAD_TLP_NUM ", 0x1C1Cull},
+ {"DFX_LOCAL_P_ACK_TLP_NUM ", 0x1C20ull},
+ {"DFX_BUF_ALOC_PORT_REQ_NUM ", 0x1C24ull},
+ {"DFX_BUF_ALOC_PORT_RESULT_NUM ", 0x1C28ull},
+ {"DFX_BUF_FAIL_SIZE_NUM ", 0x1C2Cull},
+ {"DFX_BUF_ALOC_SIZE_NUM ", 0x1C30ull},
+ {"DFX_BUF_NP_RELEASE_SIZE_NUM ", 0x1C34ull},
+ {"DFX_BUF_P_RELEASE_SIZE_NUM ", 0x1C38ull},
+ {"DFX_BUF_PORT_RELEASE_SIZE_NUM ", 0x1C3Cull},
+ {"DFX_DMA_PREBUF_MEM0_ECC_ERR_ADDR ", 0x1CA8ull},
+ {"DFX_DMA_PREBUF_MEM0_ECC_CNT ", 0x1CACull},
+ {"DFX_DMA_LOC_NP_OSTB_ECC_ERR_ADDR ", 0x1CB0ull},
+ {"DFX_DMA_LOC_NP_OSTB_ECC_CNT ", 0x1CB4ull},
+ {"DFX_DMA_PREBUF_MEM1_ECC_ERR_ADDR ", 0x1CC0ull},
+ {"DFX_DMA_PREBUF_MEM1_ECC_CNT ", 0x1CC4ull},
+ {"DMA_CH_DONE_STS ", 0x02E0ull},
+ {"DMA_CH_ERR_STS ", 0x0320ull},
+};
+#endif /* CONFIG_DEBUG_FS*/
+
static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
{
if (pdev->revision == HISI_DMA_REVISION_HIP08B)
@@ -717,6 +836,118 @@ static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
INIT_LIST_HEAD(&dma_dev->channels);
}
+/* --- debugfs implementation --- */
+#ifdef CONFIG_DEBUG_FS
+#include <linux/debugfs.h>
+static struct debugfs_reg32 *hisi_dma_get_ch_regs(struct hisi_dma_dev *hdma_dev,
+ u32 *regs_sz)
+{
+ struct device *dev = &hdma_dev->pdev->dev;
+ struct debugfs_reg32 *regs;
+ u32 regs_sz_comm;
+
+ regs_sz_comm = ARRAY_SIZE(hisi_dma_comm_chan_regs);
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip08_chan_regs);
+ else
+ *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip09_chan_regs);
+
+ regs = devm_kcalloc(dev, *regs_sz, sizeof(struct debugfs_reg32),
+ GFP_KERNEL);
+ if (!regs)
+ return NULL;
+ memcpy(regs, hisi_dma_comm_chan_regs, sizeof(hisi_dma_comm_chan_regs));
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ memcpy(regs + regs_sz_comm, hisi_dma_hip08_chan_regs,
+ sizeof(hisi_dma_hip08_chan_regs));
+ else
+ memcpy(regs + regs_sz_comm, hisi_dma_hip09_chan_regs,
+ sizeof(hisi_dma_hip09_chan_regs));
+
+ return regs;
+}
+
+static int hisi_dma_create_chan_dir(struct hisi_dma_dev *hdma_dev)
+{
+ char dir_name[HISI_DMA_MAX_DIR_NAME_LEN];
+ struct debugfs_regset32 *regsets;
+ struct debugfs_reg32 *regs;
+ struct dentry *chan_dir;
+ struct device *dev;
+ u32 regs_sz;
+ int ret;
+ int i;
+
+ dev = &hdma_dev->pdev->dev;
+
+ regsets = devm_kcalloc(dev, hdma_dev->chan_num,
+ sizeof(*regsets), GFP_KERNEL);
+ if (!regsets)
+ return -ENOMEM;
+
+ regs = hisi_dma_get_ch_regs(hdma_dev, ®s_sz);
+ if (!regs)
+ return -ENOMEM;
+
+ for (i = 0; i < hdma_dev->chan_num; i++) {
+ regsets[i].regs = regs;
+ regsets[i].nregs = regs_sz;
+ regsets[i].base = hdma_dev->queue_base + i * HISI_DMA_Q_OFFSET;
+ regsets[i].dev = dev;
+
+ memset(dir_name, 0, HISI_DMA_MAX_DIR_NAME_LEN);
+ ret = sprintf(dir_name, "channel%d", i);
+ if (ret < 0)
+ return ret;
+
+ chan_dir = debugfs_create_dir(dir_name,
+ hdma_dev->dma_dev.dbg_dev_root);
+ debugfs_create_regset32("regs", 0444, chan_dir, ®sets[i]);
+ }
+
+ return 0;
+}
+
+static void hisi_dma_create_debugfs(struct hisi_dma_dev *hdma_dev)
+{
+ struct debugfs_regset32 *regset;
+ struct device *dev;
+ int ret;
+
+ dev = &hdma_dev->pdev->dev;
+
+ if (hdma_dev->dma_dev.dbg_dev_root == NULL)
+ return;
+
+ regset = devm_kzalloc(dev, sizeof(*regset), GFP_KERNEL);
+ if (!regset) {
+ return;
+ }
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ regset->regs = hisi_dma_hip08_comm_regs;
+ regset->nregs = ARRAY_SIZE(hisi_dma_hip08_comm_regs);
+ } else {
+ regset->regs = hisi_dma_hip09_comm_regs;
+ regset->nregs = ARRAY_SIZE(hisi_dma_hip09_comm_regs);
+ }
+ regset->base = hdma_dev->base;
+ regset->dev = dev;
+
+ debugfs_create_regset32("regs", 0444,
+ hdma_dev->dma_dev.dbg_dev_root, regset);
+
+ ret = hisi_dma_create_chan_dir(hdma_dev);
+ if (ret < 0)
+ dev_info(&hdma_dev->pdev->dev, "fail to create debugfs for channels!\n");
+}
+#else
+static void hisi_dma_create_debugfs(struct hisi_dma_dev *hdma_dev) { }
+#endif /* CONFIG_DEBUG_FS*/
+/* --- debugfs implementation --- */
+
static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
enum hisi_dma_reg_layout reg_layout;
@@ -793,10 +1024,14 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
dma_dev = &hdma_dev->dma_dev;
ret = dmaenginem_async_device_register(dma_dev);
- if (ret < 0)
+ if (ret < 0) {
dev_err(dev, "failed to register device!\n");
+ return ret;
+ }
+
+ hisi_dma_create_debugfs(hdma_dev);
- return ret;
+ return 0;
}
static const struct pci_device_id hisi_dma_pci_tbl[] = {
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v3 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma
2022-07-26 1:35 ` [PATCH v3 " Jie Hai
` (5 preceding siblings ...)
2022-07-26 1:35 ` [PATCH v3 6/7] dmaengine: hisilicon: dump regs to debugfs Jie Hai
@ 2022-07-26 1:35 ` Jie Hai
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-07-26 1:35 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
Add myself as a maintainer for hisi_dma.
Signed-off-by: Jie Hai <haijie1@huawei.com>
---
MAINTAINERS | 1 +
1 file changed, 1 insertion(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 64379c699903..b1aeffd74a37 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8971,6 +8971,7 @@ F: net/dsa/tag_hellcreek.c
HISILICON DMA DRIVER
M: Zhou Wang <wangzhou1@hisilicon.com>
+M: Jie Hai <haijie1@hisilicon.com>
L: dmaengine@vger.kernel.org
S: Maintained
F: drivers/dma/hisi_dma.c
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* Re: [PATCH v2 2/7] dmaengine: hisilicon: Fix CQ head update
2022-07-21 13:27 ` Vinod Koul
@ 2022-07-26 1:38 ` Jie Hai
0 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-07-26 1:38 UTC (permalink / raw)
To: Vinod Koul; +Cc: wangzhou1, dmaengine, linux-kernel
On 2022/7/21 21:27, Vinod Koul wrote:
> On 29-06-22, 11:55, Jie Hai wrote:
>> After completion of data transfer of one or multiple descriptors,
>> the completion status and the current head pointer to submission
>> queue are written into the CQ and interrupt can be generated to
>> inform the software. In interrupt process CQ is read and cq_head
>> is updated.
>>
>> hisi_dma_irq updates cq_head only when the completion status is
>> success. When an abnormal interrupt reports, cq_head will not update
>> which will cause subsequent interrupt processes read the error CQ
>> and never report the correct status.
>>
>> This patch updates cq_head whenever CQ is accessed.
>>
>> Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
>> Reported-by: kernel test robot <lkp@intel.com>
>> Signed-off-by: Jie Hai <haijie1@huawei.com>
>> ---
>> drivers/dma/hisi_dma.c | 10 +++++-----
>> 1 file changed, 5 insertions(+), 5 deletions(-)
>>
>> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
>> index 98bc488893cc..7609e6e7eb37 100644
>> --- a/drivers/dma/hisi_dma.c
>> +++ b/drivers/dma/hisi_dma.c
>> @@ -436,12 +436,12 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
>> desc = chan->desc;
>> cqe = chan->cq + chan->cq_head;
>> if (desc) {
>> + chan->cq_head = (chan->cq_head + 1) %
>> + hdma_dev->chan_depth;
>
> This can look better with single line
>
Thanks for your reviewing.
These issues have been fixed in V3.
>> + hisi_dma_chan_write(hdma_dev->base,
>> + HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
>> + chan->cq_head);
>
> maybe two lines?
>
>> if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
>> - chan->cq_head = (chan->cq_head + 1) %
>> - hdma_dev->chan_depth;
>> - hisi_dma_chan_write(hdma_dev->base,
>> - HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
>> - chan->cq_head);
>> vchan_cookie_complete(&desc->vd);
>> } else {
>> dev_err(&hdma_dev->pdev->dev, "task error!\n");
>> --
>> 2.33.0
>
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
2022-07-21 13:29 ` Vinod Koul
@ 2022-07-26 1:40 ` Jie Hai
0 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-07-26 1:40 UTC (permalink / raw)
To: Vinod Koul; +Cc: wangzhou1, dmaengine, linux-kernel
On 2022/7/21 21:29, Vinod Koul wrote:
> On 29-06-22, 11:55, Jie Hai wrote:
>> The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they
>> have the same pci device id but different pci revision.
>> Unfortunately, they have different register layouts, so
>> the origin driver cannot run on HiSilicon IP09 correctly.
>>
>> This patch enables the driver to adapt to HiSilicon IP09.
>> HiSilicon IP09 offers 4 channels, each channel has a send
>> queue, a complete queue and an interrupt to help to do tasks.
>> This DMA engine can do memory copy between memory blocks.
>>
>> Signed-off-by: Jie Hai <haijie1@huawei.com>
>> ---
>> drivers/dma/hisi_dma.c | 382 ++++++++++++++++++++++++++++++++---------
>> 1 file changed, 299 insertions(+), 83 deletions(-)
>>
>> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
>> index d5f87ef4a5ee..dc7c59b21114 100644
>> --- a/drivers/dma/hisi_dma.c
>> +++ b/drivers/dma/hisi_dma.c
>> @@ -1,5 +1,6 @@
>> // SPDX-License-Identifier: GPL-2.0-only
>> -/* Copyright(c) 2019 HiSilicon Limited. */
>> +/* Copyright(c) 2019-2022 HiSilicon Limited. */
>> +
>> #include <linux/bitfield.h>
>> #include <linux/dmaengine.h>
>> #include <linux/init.h>
>> @@ -9,35 +10,85 @@
>> #include <linux/spinlock.h>
>> #include "virt-dma.h"
>>
>> -#define HISI_DMA_SQ_BASE_L 0x0
>> -#define HISI_DMA_SQ_BASE_H 0x4
>> -#define HISI_DMA_SQ_DEPTH 0x8
>> -#define HISI_DMA_SQ_TAIL_PTR 0xc
>> -#define HISI_DMA_CQ_BASE_L 0x10
>> -#define HISI_DMA_CQ_BASE_H 0x14
>> -#define HISI_DMA_CQ_DEPTH 0x18
>> -#define HISI_DMA_CQ_HEAD_PTR 0x1c
>> -#define HISI_DMA_CTRL0 0x20
>> -#define HISI_DMA_CTRL0_QUEUE_EN_S 0
>> -#define HISI_DMA_CTRL0_QUEUE_PAUSE_S 4
>> -#define HISI_DMA_CTRL1 0x24
>> -#define HISI_DMA_CTRL1_QUEUE_RESET_S 0
>> -#define HISI_DMA_Q_FSM_STS 0x30
>> -#define HISI_DMA_FSM_STS_MASK GENMASK(3, 0)
>> -#define HISI_DMA_INT_STS 0x40
>> -#define HISI_DMA_INT_STS_MASK GENMASK(12, 0)
>> -#define HISI_DMA_INT_MSK 0x44
>> -#define HISI_DMA_MODE 0x217c
>> -#define HISI_DMA_OFFSET 0x100
>> -
>> -#define HISI_DMA_MSI_NUM 32
>> -#define HISI_DMA_CHAN_NUM 30
>> -#define HISI_DMA_Q_DEPTH_VAL 1024
>> -
>> -#define PCI_BAR_2 2
>> -
>> -#define HISI_DMA_POLL_Q_STS_DELAY_US 10
>> -#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
>> +/* HiSilicon DMA register common field define */
>> +#define HISI_DMA_Q_SQ_BASE_L 0x0
>> +#define HISI_DMA_Q_SQ_BASE_H 0x4
>> +#define HISI_DMA_Q_SQ_DEPTH 0x8
>> +#define HISI_DMA_Q_SQ_TAIL_PTR 0xc
>> +#define HISI_DMA_Q_CQ_BASE_L 0x10
>> +#define HISI_DMA_Q_CQ_BASE_H 0x14
>> +#define HISI_DMA_Q_CQ_DEPTH 0x18
>> +#define HISI_DMA_Q_CQ_HEAD_PTR 0x1c
>> +#define HISI_DMA_Q_CTRL0 0x20
>> +#define HISI_DMA_Q_CTRL0_QUEUE_EN 0
>> +#define HISI_DMA_Q_CTRL0_QUEUE_PAUSE 4
>
> Pls use BIT/GENMASK only for register bits
>
Thanks for your reviewing.
This issue has been fixed in V3.
>> +#define HISI_DMA_Q_CTRL1 0x24
>> +#define HISI_DMA_Q_CTRL1_QUEUE_RESET 0
>> +#define HISI_DMA_Q_FSM_STS 0x30
>> +#define HISI_DMA_Q_FSM_STS_MASK GENMASK(3, 0)
>> +#define HISI_DMA_Q_ERR_INT_NUM0 0x84
>> +#define HISI_DMA_Q_ERR_INT_NUM1 0x88
>> +#define HISI_DMA_Q_ERR_INT_NUM2 0x8c
>> +
>> +/* HiSilicon IP08 DMA register and field define */
>> +#define HISI_DMA_HIP08_MODE 0x217C
>> +#define HISI_DMA_HIP08_Q_BASE 0x0
>> +#define HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN 2
>> +#define HISI_DMA_HIP08_Q_INT_STS 0x40
>> +#define HISI_DMA_HIP08_Q_INT_MSK 0x44
>> +#define HISI_DMA_HIP08_Q_INT_STS_MASK GENMASK(14, 0)
>> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM3 0x90
>> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM4 0x94
>> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM5 0x98
>> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM6 0x48
>> +#define HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT 24
>> +
>> +/* HiSilicon IP09 DMA register and field define */
>> +#define HISI_DMA_HIP09_DMA_FLR_DISABLE 0xA00
>> +#define HISI_DMA_HIP09_DMA_FLR_DISABLE_B 0
>> +#define HISI_DMA_HIP09_Q_BASE 0x2000
>> +#define HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN GENMASK(31, 28)
>> +#define HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT 26
>> +#define HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT 27
>> +#define HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE 2
>> +#define HISI_DMA_HIP09_Q_INT_STS 0x40
>> +#define HISI_DMA_HIP09_Q_INT_MSK 0x44
>> +#define HISI_DMA_HIP09_Q_INT_STS_MASK 0x1
>> +#define HISI_DMA_HIP09_Q_ERR_INT_STS 0x48
>> +#define HISI_DMA_HIP09_Q_ERR_INT_MSK 0x4C
>> +#define HISI_DMA_HIP09_Q_ERR_INT_STS_MASK GENMASK(18, 1)
>> +#define HISI_DMA_HIP09_PORT_CFG_REG(port_id) (0x800 + \
>> + (port_id) * 0x20)
>> +#define HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B 16
>> +
>> +#define HISI_DMA_HIP09_MAX_PORT_NUM 16
>> +
>> +#define HISI_DMA_HIP08_MSI_NUM 32
>> +#define HISI_DMA_HIP08_CHAN_NUM 30
>> +#define HISI_DMA_HIP09_MSI_NUM 4
>> +#define HISI_DMA_HIP09_CHAN_NUM 4
>> +#define HISI_DMA_REVISION_HIP08B 0x21
>> +#define HISI_DMA_REVISION_HIP09A 0x30
>> +
>> +#define HISI_DMA_Q_OFFSET 0x100
>> +#define HISI_DMA_Q_DEPTH_VAL 1024
>> +
>> +#define PCI_BAR_2 2
>> +
>> +#define HISI_DMA_POLL_Q_STS_DELAY_US 10
>> +#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
>> +
>> +/*
>> + * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
>> + * have the same pci device id but different pci revision.
>> + * Unfortunately, they have different register layouts, so two layout
>> + * enumerations are defined.
>> + */
>> +enum hisi_dma_reg_layout {
>> + HISI_DMA_REG_LAYOUT_INVALID = 0,
>> + HISI_DMA_REG_LAYOUT_HIP08,
>> + HISI_DMA_REG_LAYOUT_HIP09
>> +};
>>
>> enum hisi_dma_mode {
>> EP = 0,
>> @@ -108,9 +159,45 @@ struct hisi_dma_dev {
>> struct dma_device dma_dev;
>> u32 chan_num;
>> u32 chan_depth;
>> + enum hisi_dma_reg_layout reg_layout;
>> + void __iomem *queue_base; /* queue region start of register */
>> struct hisi_dma_chan chan[];
>> };
>>
>> +static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
>> +{
>> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
>> + return HISI_DMA_REG_LAYOUT_HIP08;
>> + else if (pdev->revision >= HISI_DMA_REVISION_HIP09A)
>> + return HISI_DMA_REG_LAYOUT_HIP09;
>> +
>> + return HISI_DMA_REG_LAYOUT_INVALID;
>> +}
>> +
>> +static u32 hisi_dma_get_chan_num(struct pci_dev *pdev)
>> +{
>> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
>> + return HISI_DMA_HIP08_CHAN_NUM;
>> +
>> + return HISI_DMA_HIP09_CHAN_NUM;
>> +}
>> +
>> +static u32 hisi_dma_get_msi_num(struct pci_dev *pdev)
>> +{
>> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
>> + return HISI_DMA_HIP08_MSI_NUM;
>> +
>> + return HISI_DMA_HIP09_MSI_NUM;
>> +}
>> +
>> +static u32 hisi_dma_get_queue_base(struct pci_dev *pdev)
>> +{
>> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
>> + return HISI_DMA_HIP08_Q_BASE;
>> +
>> + return HISI_DMA_HIP09_Q_BASE;
>> +}
>> +
>> static inline struct hisi_dma_chan *to_hisi_dma_chan(struct dma_chan *c)
>> {
>> return container_of(c, struct hisi_dma_chan, vc.chan);
>> @@ -124,7 +211,7 @@ static inline struct hisi_dma_desc *to_hisi_dma_desc(struct virt_dma_desc *vd)
>> static inline void hisi_dma_chan_write(void __iomem *base, u32 reg, u32 index,
>> u32 val)
>> {
>> - writel_relaxed(val, base + reg + index * HISI_DMA_OFFSET);
>> + writel_relaxed(val, base + reg + index * HISI_DMA_Q_OFFSET);
>> }
>>
>> static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
>> @@ -139,48 +226,76 @@ static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
>> static void hisi_dma_pause_dma(struct hisi_dma_dev *hdma_dev, u32 index,
>> bool pause)
>> {
>> - void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
>> - HISI_DMA_OFFSET;
>> + void __iomem *addr;
>>
>> - hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_PAUSE_S, pause);
>> + addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
>> + index * HISI_DMA_Q_OFFSET;
>> + hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_PAUSE, pause);
>> }
>>
>> static void hisi_dma_enable_dma(struct hisi_dma_dev *hdma_dev, u32 index,
>> bool enable)
>> {
>> - void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
>> - HISI_DMA_OFFSET;
>> + void __iomem *addr;
>>
>> - hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_EN_S, enable);
>> + addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
>> + index * HISI_DMA_Q_OFFSET;
>> + hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_EN, enable);
>> }
>>
>> static void hisi_dma_mask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
>> {
>> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_INT_MSK, qp_index,
>> - HISI_DMA_INT_STS_MASK);
>> + void __iomem *q_base = hdma_dev->queue_base;
>> +
>> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
>> + qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
>> + else {
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
>> + qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
>> + qp_index,
>> + HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
>> + }
>> }
>>
>> static void hisi_dma_unmask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
>> {
>> - void __iomem *base = hdma_dev->base;
>> -
>> - hisi_dma_chan_write(base, HISI_DMA_INT_STS, qp_index,
>> - HISI_DMA_INT_STS_MASK);
>> - hisi_dma_chan_write(base, HISI_DMA_INT_MSK, qp_index, 0);
>> + void __iomem *q_base = hdma_dev->queue_base;
>> +
>> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_STS,
>> + qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
>> + qp_index, 0);
>> + } else {
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_STS,
>> + qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_STS,
>> + qp_index,
>> + HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
>> + qp_index, 0);
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
>> + qp_index, 0);
>> + }
>> }
>>
>> static void hisi_dma_do_reset(struct hisi_dma_dev *hdma_dev, u32 index)
>> {
>> - void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL1 + index *
>> - HISI_DMA_OFFSET;
>> + void __iomem *addr;
>>
>> - hisi_dma_update_bit(addr, HISI_DMA_CTRL1_QUEUE_RESET_S, 1);
>> + addr = hdma_dev->queue_base +
>> + HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
>> + hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL1_QUEUE_RESET, 1);
>> }
>>
>> static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
>> {
>> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, index, 0);
>> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
>> + void __iomem *q_base = hdma_dev->queue_base;
>> +
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
>> }
>>
>> static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
>> @@ -195,11 +310,11 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
>> hisi_dma_enable_dma(hdma_dev, index, false);
>> hisi_dma_mask_irq(hdma_dev, index);
>>
>> - addr = hdma_dev->base +
>> - HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET;
>> + addr = hdma_dev->queue_base +
>> + HISI_DMA_Q_FSM_STS + index * HISI_DMA_Q_OFFSET;
>>
>> ret = readl_relaxed_poll_timeout(addr, tmp,
>> - FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN,
>> + FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) != RUN,
>> HISI_DMA_POLL_Q_STS_DELAY_US,
>> HISI_DMA_POLL_Q_STS_TIME_OUT_US);
>> if (ret) {
>> @@ -217,7 +332,7 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
>> }
>>
>> ret = readl_relaxed_poll_timeout(addr, tmp,
>> - FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE,
>> + FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) == IDLE,
>> HISI_DMA_POLL_Q_STS_DELAY_US,
>> HISI_DMA_POLL_Q_STS_TIME_OUT_US);
>> if (ret) {
>> @@ -300,8 +415,8 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
>> chan->sq_tail = (chan->sq_tail + 1) % hdma_dev->chan_depth;
>>
>> /* update sq_tail to trigger a new task */
>> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, chan->qp_num,
>> - chan->sq_tail);
>> + hisi_dma_chan_write(hdma_dev->queue_base, HISI_DMA_Q_SQ_TAIL_PTR,
>> + chan->qp_num, chan->sq_tail);
>> }
>>
>> static void hisi_dma_issue_pending(struct dma_chan *c)
>> @@ -375,26 +490,86 @@ static int hisi_dma_alloc_qps_mem(struct hisi_dma_dev *hdma_dev)
>> static void hisi_dma_init_hw_qp(struct hisi_dma_dev *hdma_dev, u32 index)
>> {
>> struct hisi_dma_chan *chan = &hdma_dev->chan[index];
>> + void __iomem *q_base = hdma_dev->queue_base;
>> u32 hw_depth = hdma_dev->chan_depth - 1;
>> - void __iomem *base = hdma_dev->base;
>> + void __iomem *addr;
>> + u32 tmp;
>>
>> /* set sq, cq base */
>> - hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_L, index,
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_L, index,
>> lower_32_bits(chan->sq_dma));
>> - hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_H, index,
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_H, index,
>> upper_32_bits(chan->sq_dma));
>> - hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_L, index,
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_L, index,
>> lower_32_bits(chan->cq_dma));
>> - hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_H, index,
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_H, index,
>> upper_32_bits(chan->cq_dma));
>>
>> /* set sq, cq depth */
>> - hisi_dma_chan_write(base, HISI_DMA_SQ_DEPTH, index, hw_depth);
>> - hisi_dma_chan_write(base, HISI_DMA_CQ_DEPTH, index, hw_depth);
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_DEPTH, index, hw_depth);
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_DEPTH, index, hw_depth);
>>
>> /* init sq tail and cq head */
>> - hisi_dma_chan_write(base, HISI_DMA_SQ_TAIL_PTR, index, 0);
>> - hisi_dma_chan_write(base, HISI_DMA_CQ_HEAD_PTR, index, 0);
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
>> +
>> + /* init error interrupt stats */
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM0, index, 0);
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM1, index, 0);
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM2, index, 0);
>> +
>> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM3,
>> + index, 0);
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM4,
>> + index, 0);
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM5,
>> + index, 0);
>> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM6,
>> + index, 0);
>> + /*
>> + * init SQ/CQ direction selecting register.
>> + * "0" is to local side and "1" is to remote side.
>> + */
>> + addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
>> + hisi_dma_update_bit(addr, HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT, 0);
>> +
>> + /*
>> + * 0 - Continue to next descriptor if error occurs.
>> + * 1 - Abort the DMA queue if error occurs.
>> + */
>> + hisi_dma_update_bit(addr,
>> + HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN, 0);
>> + } else {
>> + addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
>> +
>> + /*
>> + * init SQ/CQ direction selecting register.
>> + * "0" is to local side and "1" is to remote side.
>> + */
>> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT, 0);
>> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT, 0);
>> +
>> + /*
>> + * 0 - Continue to next descriptor if error occurs.
>> + * 1 - Abort the DMA queue if error occurs.
>> + */
>> +
>> + tmp = readl_relaxed(addr);
>> + tmp &= ~HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN;
>> + writel_relaxed(tmp, addr);
>> +
>> + /*
>> + * 0 - dma should process FLR whith CPU.
>> + * 1 - dma not process FLR, only cpu process FLR.
>> + */
>> + addr = q_base + HISI_DMA_HIP09_DMA_FLR_DISABLE +
>> + index * HISI_DMA_Q_OFFSET;
>> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_DMA_FLR_DISABLE_B, 0);
>> +
>> + addr = q_base + HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
>> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE, 1);
>> + }
>> }
>>
>> static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
>> @@ -438,17 +613,18 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
>> struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
>> struct hisi_dma_desc *desc;
>> struct hisi_dma_cqe *cqe;
>> + void __iomem *q_base;
>>
>> spin_lock(&chan->vc.lock);
>>
>> desc = chan->desc;
>> cqe = chan->cq + chan->cq_head;
>> + q_base = hdma_dev->queue_base;
>> if (desc) {
>> chan->cq_head = (chan->cq_head + 1) %
>> hdma_dev->chan_depth;
>> - hisi_dma_chan_write(hdma_dev->base,
>> - HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
>> - chan->cq_head);
>> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
>> + chan->qp_num, chan->cq_head);
>> if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
>> vchan_cookie_complete(&desc->vd);
>> hisi_dma_start_transfer(chan);
>> @@ -508,16 +684,58 @@ static void hisi_dma_disable_hw_channels(void *data)
>> static void hisi_dma_set_mode(struct hisi_dma_dev *hdma_dev,
>> enum hisi_dma_mode mode)
>> {
>> - writel_relaxed(mode == RC ? 1 : 0, hdma_dev->base + HISI_DMA_MODE);
>> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
>> + writel_relaxed(mode == RC ? 1 : 0,
>> + hdma_dev->base + HISI_DMA_HIP08_MODE);
>> +}
>> +
>> +static void hisi_dma_init_hw(struct hisi_dma_dev *hdma_dev)
>> +{
>> + void __iomem *addr;
>> + int i;
>> +
>> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP09) {
>> + for (i = 0; i < HISI_DMA_HIP09_MAX_PORT_NUM; i++) {
>> + addr = hdma_dev->base + HISI_DMA_HIP09_PORT_CFG_REG(i);
>> + hisi_dma_update_bit(addr,
>> + HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B, 1);
>> + }
>> + }
>> +}
>> +
>> +static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
>> +{
>> + struct dma_device *dma_dev;
>> +
>> + dma_dev = &hdma_dev->dma_dev;
>> + dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
>> + dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
>> + dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
>> + dma_dev->device_tx_status = hisi_dma_tx_status;
>> + dma_dev->device_issue_pending = hisi_dma_issue_pending;
>> + dma_dev->device_terminate_all = hisi_dma_terminate_all;
>> + dma_dev->device_synchronize = hisi_dma_synchronize;
>> + dma_dev->directions = BIT(DMA_MEM_TO_MEM);
>> + dma_dev->dev = &hdma_dev->pdev->dev;
>> + INIT_LIST_HEAD(&dma_dev->channels);
>> }
>>
>> static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>> {
>> + enum hisi_dma_reg_layout reg_layout;
>> struct device *dev = &pdev->dev;
>> struct hisi_dma_dev *hdma_dev;
>> struct dma_device *dma_dev;
>> + u32 chan_num;
>> + u32 msi_num;
>> int ret;
>>
>> + reg_layout = hisi_dma_get_reg_layout(pdev);
>> + if (reg_layout == HISI_DMA_REG_LAYOUT_INVALID) {
>> + dev_err(dev, "unsupported device!\n");
>> + return -EINVAL;
>> + }
>> +
>> ret = pcim_enable_device(pdev);
>> if (ret) {
>> dev_err(dev, "failed to enable device mem!\n");
>> @@ -534,40 +752,37 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>> if (ret)
>> return ret;
>>
>> - hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, HISI_DMA_CHAN_NUM), GFP_KERNEL);
>> + chan_num = hisi_dma_get_chan_num(pdev);
>> + hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, chan_num),
>> + GFP_KERNEL);
>> if (!hdma_dev)
>> return -EINVAL;
>>
>> hdma_dev->base = pcim_iomap_table(pdev)[PCI_BAR_2];
>> hdma_dev->pdev = pdev;
>> - hdma_dev->chan_num = HISI_DMA_CHAN_NUM;
>> hdma_dev->chan_depth = HISI_DMA_Q_DEPTH_VAL;
>> + hdma_dev->chan_num = chan_num;
>> + hdma_dev->reg_layout = reg_layout;
>> + hdma_dev->queue_base = hdma_dev->base + hisi_dma_get_queue_base(pdev);
>>
>> pci_set_drvdata(pdev, hdma_dev);
>> pci_set_master(pdev);
>>
>> + msi_num = hisi_dma_get_msi_num(pdev);
>> +
>> /* This will be freed by 'pcim_release()'. See 'pcim_enable_device()' */
>> - ret = pci_alloc_irq_vectors(pdev, HISI_DMA_MSI_NUM, HISI_DMA_MSI_NUM,
>> - PCI_IRQ_MSI);
>> + ret = pci_alloc_irq_vectors(pdev, msi_num, msi_num, PCI_IRQ_MSI);
>> if (ret < 0) {
>> dev_err(dev, "Failed to allocate MSI vectors!\n");
>> return ret;
>> }
>>
>> - dma_dev = &hdma_dev->dma_dev;
>> - dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
>> - dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
>> - dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
>> - dma_dev->device_tx_status = hisi_dma_tx_status;
>> - dma_dev->device_issue_pending = hisi_dma_issue_pending;
>> - dma_dev->device_terminate_all = hisi_dma_terminate_all;
>> - dma_dev->device_synchronize = hisi_dma_synchronize;
>> - dma_dev->directions = BIT(DMA_MEM_TO_MEM);
>> - dma_dev->dev = dev;
>> - INIT_LIST_HEAD(&dma_dev->channels);
>> + hisi_dma_init_dma_dev(hdma_dev);
>>
>> hisi_dma_set_mode(hdma_dev, RC);
>>
>> + hisi_dma_init_hw(hdma_dev);
>> +
>> ret = hisi_dma_enable_hw_channels(hdma_dev);
>> if (ret < 0) {
>> dev_err(dev, "failed to enable hw channel!\n");
>> @@ -579,6 +794,7 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>> if (ret)
>> return ret;
>>
>> + dma_dev = &hdma_dev->dma_dev;
>> ret = dmaenginem_async_device_register(dma_dev);
>> if (ret < 0)
>> dev_err(dev, "failed to register device!\n");
>> --
>> 2.33.0
>
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] dmaengine: hisilicon: Add dfx feature for hisi dma driver
2022-07-21 13:25 ` Vinod Koul
@ 2022-07-26 1:43 ` Jie Hai
2022-07-26 12:39 ` Vinod Koul
0 siblings, 1 reply; 64+ messages in thread
From: Jie Hai @ 2022-07-26 1:43 UTC (permalink / raw)
To: Vinod Koul; +Cc: wangzhou1, dmaengine, linux-kernel
On 2022/7/21 21:25, Vinod Koul wrote:
> On 29-06-22, 11:55, Jie Hai wrote:
>
> What does dfx mean in title?
>
Thanks for the question.
DFX＝"Design for X", I quote its definition as follows:
The product definition puts forward a list of specific aspects about a
design that designers need to optimize, for example, product usage,
functions and features, cost range, and aesthetics. Modern designers
even need to consider more design objectives because the new design
characteristics may finally determine the success or failure of a
product. Design methodologies that consider these new characteristics
are called design for X (DFX). Specific design methodologies include
design for reliability (DFR), design for serviceability (DFS), design
for supply chain (DFSC), design for scalability (DFSc), design for
energy efficiency and environment (DFEE), design for testability
(DFT),and so on.
For clarity, I've replaced it in v3 with something easier to understand.
>> This patch adds dump of registers with debugfs for HIP08
>> and HIP09 DMA driver.
>>
>> Reported-by: kernel test robot <lkp@intel.com>
>
> ?
>
The kernel test robot raised an issue and asked me to modify it and add
"Reported-by: kernel test robot <lkp@intel.com>" to the commit message,
so I did, and it was removed in v3.
>> Signed-off-by: Jie Hai <haijie1@huawei.com>
>> ---
>> drivers/dma/hisi_dma.c | 317 ++++++++++++++++++++++++++++++++++++++++-
>> 1 file changed, 314 insertions(+), 3 deletions(-)
>>
>> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
>> index dc7c59b21114..b0bc7b18933b 100644
>> --- a/drivers/dma/hisi_dma.c
>> +++ b/drivers/dma/hisi_dma.c
>> @@ -78,6 +78,8 @@
>> #define HISI_DMA_POLL_Q_STS_DELAY_US 10
>> #define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
>>
>> +#define HISI_DMA_MAX_DIR_NAME_LEN 128
>> +
>> /*
>> * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
>> * have the same pci device id but different pci revision.
>> @@ -161,9 +163,131 @@ struct hisi_dma_dev {
>> u32 chan_depth;
>> enum hisi_dma_reg_layout reg_layout;
>> void __iomem *queue_base; /* queue region start of register */
>> +#ifdef CONFIG_DEBUG_FS
>> + struct dentry *dbg_hdev_root;
>
> Please do not create your own root and use dmaengine root instead
>This issue has been fixed in V3.
>> + struct dentry **dbg_chans;
>> +#endif
>> struct hisi_dma_chan chan[];
>> };
>>
>> +#ifdef CONFIG_DEBUG_FS
>> +struct dentry *hisi_dma_debugfs_root;
>> +
>> +static const struct debugfs_reg32 hisi_dma_comm_chan_regs[] = {
>> + {"DMA_QUEUE_SQ_DEPTH ", 0x0008ull},
>> + {"DMA_QUEUE_SQ_TAIL_PTR ", 0x000Cull},
>> + {"DMA_QUEUE_CQ_DEPTH ", 0x0018ull},
>> + {"DMA_QUEUE_CQ_HEAD_PTR ", 0x001Cull},
>> + {"DMA_QUEUE_CTRL0 ", 0x0020ull},
>> + {"DMA_QUEUE_CTRL1 ", 0x0024ull},
>> + {"DMA_QUEUE_FSM_STS ", 0x0030ull},
>> + {"DMA_QUEUE_SQ_STS ", 0x0034ull},
>> + {"DMA_QUEUE_CQ_TAIL_PTR ", 0x003Cull},
>> + {"DMA_QUEUE_INT_STS ", 0x0040ull},
>> + {"DMA_QUEUE_INT_MSK ", 0x0044ull},
>> + {"DMA_QUEUE_INT_RO ", 0x006Cull},
>> +};
>> +
>> +static const struct debugfs_reg32 hisi_dma_hip08_chan_regs[] = {
>> + {"DMA_QUEUE_BYTE_CNT ", 0x0038ull},
>> + {"DMA_ERR_INT_NUM6 ", 0x0048ull},
>> + {"DMA_QUEUE_DESP0 ", 0x0050ull},
>> + {"DMA_QUEUE_DESP1 ", 0x0054ull},
>> + {"DMA_QUEUE_DESP2 ", 0x0058ull},
>> + {"DMA_QUEUE_DESP3 ", 0x005Cull},
>> + {"DMA_QUEUE_DESP4 ", 0x0074ull},
>> + {"DMA_QUEUE_DESP5 ", 0x0078ull},
>> + {"DMA_QUEUE_DESP6 ", 0x007Cull},
>> + {"DMA_QUEUE_DESP7 ", 0x0080ull},
>> + {"DMA_ERR_INT_NUM0 ", 0x0084ull},
>> + {"DMA_ERR_INT_NUM1 ", 0x0088ull},
>> + {"DMA_ERR_INT_NUM2 ", 0x008Cull},
>> + {"DMA_ERR_INT_NUM3 ", 0x0090ull},
>> + {"DMA_ERR_INT_NUM4 ", 0x0094ull},
>> + {"DMA_ERR_INT_NUM5 ", 0x0098ull},
>> + {"DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
>> +};
>> +
>> +static const struct debugfs_reg32 hisi_dma_hip09_chan_regs[] = {
>> + {"DMA_QUEUE_ERR_INT_STS ", 0x0048ull},
>> + {"DMA_QUEUE_ERR_INT_MSK ", 0x004Cull},
>> + {"DFX_SQ_READ_ERR_PTR ", 0x0068ull},
>> + {"DFX_DMA_ERR_INT_NUM0 ", 0x0084ull},
>> + {"DFX_DMA_ERR_INT_NUM1 ", 0x0088ull},
>> + {"DFX_DMA_ERR_INT_NUM2 ", 0x008Cull},
>> + {"DFX_DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
>> +};
>> +
>> +static const struct debugfs_reg32 hisi_dma_hip08_comm_regs[] = {
>> + {"DMA_ECC_ERR_ADDR ", 0x2004ull},
>> + {"DMA_ECC_ECC_CNT ", 0x2014ull},
>> + {"COMMON_AND_CH_ERR_STS ", 0x2030ull},
>> + {"LOCAL_CPL_ID_STS_0 ", 0x20E0ull},
>> + {"LOCAL_CPL_ID_STS_1 ", 0x20E4ull},
>> + {"LOCAL_CPL_ID_STS_2 ", 0x20E8ull},
>> + {"LOCAL_CPL_ID_STS_3 ", 0x20ECull},
>> + {"LOCAL_TLP_NUM ", 0x2158ull},
>> + {"SQCQ_TLP_NUM ", 0x2164ull},
>> + {"CPL_NUM ", 0x2168ull},
>> + {"INF_BACK_PRESS_STS ", 0x2170ull},
>> + {"DMA_CH_RAS_LEVEL ", 0x2184ull},
>> + {"DMA_CM_RAS_LEVEL ", 0x2188ull},
>> + {"DMA_CH_ERR_STS ", 0x2190ull},
>> + {"DMA_CH_DONE_STS ", 0x2194ull},
>> + {"DMA_SQ_TAG_STS_0 ", 0x21A0ull},
>> + {"DMA_SQ_TAG_STS_1 ", 0x21A4ull},
>> + {"DMA_SQ_TAG_STS_2 ", 0x21A8ull},
>> + {"DMA_SQ_TAG_STS_3 ", 0x21ACull},
>> + {"LOCAL_P_ID_STS_0 ", 0x21B0ull},
>> + {"LOCAL_P_ID_STS_1 ", 0x21B4ull},
>> + {"LOCAL_P_ID_STS_2 ", 0x21B8ull},
>> + {"LOCAL_P_ID_STS_3 ", 0x21BCull},
>> + {"DMA_PREBUFF_INFO_0 ", 0x2200ull},
>> + {"DMA_CM_TABLE_INFO_0 ", 0x2220ull},
>> + {"DMA_CM_CE_RO ", 0x2244ull},
>> + {"DMA_CM_NFE_RO ", 0x2248ull},
>> + {"DMA_CM_FE_RO ", 0x224Cull},
>> +};
>> +
>> +static const struct debugfs_reg32 hisi_dma_hip09_comm_regs[] = {
>> + {"COMMON_AND_CH_ERR_STS ", 0x0030ull},
>> + {"DMA_PORT_IDLE_STS ", 0x0150ull},
>> + {"DMA_CH_RAS_LEVEL ", 0x0184ull},
>> + {"DMA_CM_RAS_LEVEL ", 0x0188ull},
>> + {"DMA_CM_CE_RO ", 0x0244ull},
>> + {"DMA_CM_NFE_RO ", 0x0248ull},
>> + {"DMA_CM_FE_RO ", 0x024Cull},
>> + {"DFX_INF_BACK_PRESS_STS0 ", 0x1A40ull},
>> + {"DFX_INF_BACK_PRESS_STS1 ", 0x1A44ull},
>> + {"DFX_INF_BACK_PRESS_STS2 ", 0x1A48ull},
>> + {"DFX_DMA_WRR_DISABLE ", 0x1A4Cull},
>> + {"DFX_PA_REQ_TLP_NUM ", 0x1C00ull},
>> + {"DFX_PA_BACK_TLP_NUM ", 0x1C04ull},
>> + {"DFX_PA_RETRY_TLP_NUM ", 0x1C08ull},
>> + {"DFX_LOCAL_NP_TLP_NUM ", 0x1C0Cull},
>> + {"DFX_LOCAL_CPL_HEAD_TLP_NUM ", 0x1C10ull},
>> + {"DFX_LOCAL_CPL_DATA_TLP_NUM ", 0x1C14ull},
>> + {"DFX_LOCAL_CPL_EXT_DATA_TLP_NUM ", 0x1C18ull},
>> + {"DFX_LOCAL_P_HEAD_TLP_NUM ", 0x1C1Cull},
>> + {"DFX_LOCAL_P_ACK_TLP_NUM ", 0x1C20ull},
>> + {"DFX_BUF_ALOC_PORT_REQ_NUM ", 0x1C24ull},
>> + {"DFX_BUF_ALOC_PORT_RESULT_NUM ", 0x1C28ull},
>> + {"DFX_BUF_FAIL_SIZE_NUM ", 0x1C2Cull},
>> + {"DFX_BUF_ALOC_SIZE_NUM ", 0x1C30ull},
>> + {"DFX_BUF_NP_RELEASE_SIZE_NUM ", 0x1C34ull},
>> + {"DFX_BUF_P_RELEASE_SIZE_NUM ", 0x1C38ull},
>> + {"DFX_BUF_PORT_RELEASE_SIZE_NUM ", 0x1C3Cull},
>> + {"DFX_DMA_PREBUF_MEM0_ECC_ERR_ADDR ", 0x1CA8ull},
>> + {"DFX_DMA_PREBUF_MEM0_ECC_CNT ", 0x1CACull},
>> + {"DFX_DMA_LOC_NP_OSTB_ECC_ERR_ADDR ", 0x1CB0ull},
>> + {"DFX_DMA_LOC_NP_OSTB_ECC_CNT ", 0x1CB4ull},
>> + {"DFX_DMA_PREBUF_MEM1_ECC_ERR_ADDR ", 0x1CC0ull},
>> + {"DFX_DMA_PREBUF_MEM1_ECC_CNT ", 0x1CC4ull},
>> + {"DMA_CH_DONE_STS ", 0x02E0ull},
>> + {"DMA_CH_ERR_STS ", 0x0320ull},
>> +};
>> +#endif /* CONFIG_DEBUG_FS*/
>> +
>> static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
>> {
>> if (pdev->revision == HISI_DMA_REVISION_HIP08B)
>> @@ -720,6 +844,162 @@ static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
>> INIT_LIST_HEAD(&dma_dev->channels);
>> }
>>
>> +/* --- debugfs implementation --- */
>> +#ifdef CONFIG_DEBUG_FS
>> +#include <linux/debugfs.h>
>> +
>> +static void hisi_dma_debugfs_init(void)
>> +{
>> + if (!debugfs_initialized())
>> + return;
>> + hisi_dma_debugfs_root = debugfs_create_dir("hisi_dma", NULL);
>> +}
>> +
>> +static void hisi_dma_debugfs_uninit(void)
>> +{
>> + debugfs_remove_recursive(hisi_dma_debugfs_root);
>> +}
>> +
>> +static struct debugfs_reg32 *hisi_dma_get_ch_regs(struct hisi_dma_dev *hdma_dev,
>> + u32 *regs_sz)
>> +{
>> + struct device *dev = &hdma_dev->pdev->dev;
>> + struct debugfs_reg32 *regs;
>> + u32 regs_sz_comm;
>> +
>> + regs_sz_comm = ARRAY_SIZE(hisi_dma_comm_chan_regs);
>> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
>> + *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip08_chan_regs);
>> + else
>> + *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip09_chan_regs);
>> +
>> + regs = devm_kcalloc(dev, *regs_sz, sizeof(struct debugfs_reg32),
>> + GFP_KERNEL);
>> + if (!regs)
>> + return NULL;
>> + memcpy(regs, hisi_dma_comm_chan_regs, sizeof(hisi_dma_comm_chan_regs));
>> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
>> + memcpy(regs + regs_sz_comm, hisi_dma_hip08_chan_regs,
>> + sizeof(hisi_dma_hip08_chan_regs));
>> + else
>> + memcpy(regs + regs_sz_comm, hisi_dma_hip09_chan_regs,
>> + sizeof(hisi_dma_hip09_chan_regs));
>> +
>> + return regs;
>> +}
>> +
>> +static int hisi_dma_create_chan_dir(struct hisi_dma_dev *hdma_dev)
>> +{
>> + char dir_name[HISI_DMA_MAX_DIR_NAME_LEN];
>> + struct debugfs_regset32 *regsets;
>> + struct debugfs_reg32 *regs;
>> + struct device *dev;
>> + u32 regs_sz;
>> + int ret;
>> + int i;
>> +
>> + dev = &hdma_dev->pdev->dev;
>> +
>> + regsets = devm_kcalloc(dev, hdma_dev->chan_num,
>> + sizeof(*regsets), GFP_KERNEL);
>> + if (!regsets)
>> + return -ENOMEM;
>> +
>> + hdma_dev->dbg_chans = devm_kcalloc(dev, hdma_dev->chan_num,
>> + sizeof(struct dentry *),
>> + GFP_KERNEL);
>> + if (!hdma_dev->dbg_chans)
>> + return -ENOMEM;
>> +
>> + regs = hisi_dma_get_ch_regs(hdma_dev, ®s_sz);
>> + if (!regs)
>> + return -ENOMEM;
>> + for (i = 0; i < hdma_dev->chan_num; i++) {
>> + regsets[i].regs = regs;
>> + regsets[i].nregs = regs_sz;
>> + regsets[i].base = hdma_dev->queue_base + i * HISI_DMA_Q_OFFSET;
>> + regsets[i].dev = dev;
>> +
>> + memset(dir_name, 0, HISI_DMA_MAX_DIR_NAME_LEN);
>> + ret = sprintf(dir_name, "channel%d", i);
>> + if (ret < 0)
>> + return ret;
>> +
>> + hdma_dev->dbg_chans[i] = debugfs_create_dir(dir_name,
>> + hdma_dev->dbg_hdev_root);
>> + if (IS_ERR(hdma_dev->dbg_chans[i])) {
>> + hdma_dev->dbg_chans[i] = NULL;
>> + dev_err(dev, "dbg_chan[%d] create fail!\n", i);
>> + return -EINVAL;
>> + }
>> + debugfs_create_regset32("regs", 0444,
>> + hdma_dev->dbg_chans[i], ®sets[i]);
>> + }
>> +
>> + return 0;
>> +}
>> +
>> +static int hisi_dma_debug_register(struct hisi_dma_dev *hdma_dev)
>> +{
>> + struct debugfs_regset32 *regset;
>> + struct device *dev;
>> + int ret;
>> +
>> + dev = &hdma_dev->pdev->dev;
>> +
>> + hdma_dev->dbg_hdev_root = debugfs_create_dir(dev_name(dev),
>> + hisi_dma_debugfs_root);
>> + regset = devm_kzalloc(dev, sizeof(*regset), GFP_KERNEL);
>> + if (!regset) {
>> + ret = -ENOMEM;
>> + goto hisi_dma_debug_register_fail;
>> + }
>> +
>> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
>> + regset->regs = hisi_dma_hip08_comm_regs;
>> + regset->nregs = ARRAY_SIZE(hisi_dma_hip08_comm_regs);
>> + } else {
>> + regset->regs = hisi_dma_hip09_comm_regs;
>> + regset->nregs = ARRAY_SIZE(hisi_dma_hip09_comm_regs);
>> + }
>> + regset->base = hdma_dev->base;
>> + regset->dev = dev;
>> +
>> + debugfs_create_regset32("regs", 0444,
>> + hdma_dev->dbg_hdev_root, regset);
>> +
>> + ret = hisi_dma_create_chan_dir(hdma_dev);
>> + if (ret < 0)
>> + goto hisi_dma_debug_register_fail;
>> +
>> + return 0;
>> +
>> +hisi_dma_debug_register_fail:
>> + debugfs_remove_recursive(hdma_dev->dbg_hdev_root);
>> + hdma_dev->dbg_hdev_root = NULL;
>> + return ret;
>> +}
>> +
>> +static void hisi_dma_debug_unregister(void *data)
>> +{
>> + struct hisi_dma_dev *hdma_dev = data;
>> +
>> + debugfs_remove_recursive(hdma_dev->dbg_hdev_root);
>> + hdma_dev->dbg_hdev_root = NULL;
>> +}
>> +#else
>> +static void hisi_dma_debugfs_init(void) { }
>> +static void hisi_dma_debugfs_uninit(void) { }
>> +
>> +static int hisi_dma_debug_register(struct hisi_dma_dev *hdma_dev)
>> +{
>> + return 0;
>> +}
>> +
>> +static void hisi_dma_debug_unregister(void *data) { }
>> +#endif /* CONFIG_DEBUG_FS*/
>> +/* --- debugfs implementation --- */
>> +
>> static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>> {
>> enum hisi_dma_reg_layout reg_layout;
>> @@ -796,10 +1076,19 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
>>
>> dma_dev = &hdma_dev->dma_dev;
>> ret = dmaenginem_async_device_register(dma_dev);
>> - if (ret < 0)
>> + if (ret < 0) {
>> dev_err(dev, "failed to register device!\n");
>> + return ret;
>> + }
>>
>> - return ret;
>> + ret = hisi_dma_debug_register(hdma_dev);
>> + if (ret < 0) {
>> + dev_err(dev, "failed to register debugfs!\n");
>> + return ret;
>> + }
>
> why should registering debugfs fail be a driver failure, it can still
> work as epxected, pls ignore these errors and remove the error handling
> for this piece of code
>
This issue has been fixed in V3.
>> +
>> + return devm_add_action_or_reset(dev, hisi_dma_debug_unregister,
>> + hdma_dev);
>> }
>>
>> static const struct pci_device_id hisi_dma_pci_tbl[] = {
>> @@ -813,7 +1102,29 @@ static struct pci_driver hisi_dma_pci_driver = {
>> .probe = hisi_dma_probe,
>> };
>>
>> -module_pci_driver(hisi_dma_pci_driver);
>> +static int __init hisi_dma_init(void)
>> +{
>> + int ret;
>> +
>> + hisi_dma_debugfs_init();
>> +
>> + ret = pci_register_driver(&hisi_dma_pci_driver);
>> + if (ret) {
>> + hisi_dma_debugfs_uninit();
>> + pr_err("hisi_dma: can't register hisi dma driver.\n");
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +static void __exit hisi_dma_exit(void)
>> +{
>> + pci_unregister_driver(&hisi_dma_pci_driver);
>> + hisi_dma_debugfs_uninit();
>> +}
>> +
>> +module_init(hisi_dma_init);
>> +module_exit(hisi_dma_exit);
>>
>> MODULE_AUTHOR("Zhou Wang <wangzhou1@hisilicon.com>");
>> MODULE_AUTHOR("Zhenfa Qiu <qiuzhenfa@hisilicon.com>");
>> --
>> 2.33.0
>
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] dmaengine: hisilicon: Add dfx feature for hisi dma driver
2022-07-26 1:43 ` Jie Hai
@ 2022-07-26 12:39 ` Vinod Koul
2022-07-27 0:58 ` Jie Hai
0 siblings, 1 reply; 64+ messages in thread
From: Vinod Koul @ 2022-07-26 12:39 UTC (permalink / raw)
To: Jie Hai; +Cc: wangzhou1, dmaengine, linux-kernel
On 26-07-22, 09:43, Jie Hai wrote:
> On 2022/7/21 21:25, Vinod Koul wrote:
> > On 29-06-22, 11:55, Jie Hai wrote:
> >
> > What does dfx mean in title?
> >
> Thanks for the question.
>
> DFX＝"Design for X", I quote its definition as follows:
> The product definition puts forward a list of specific aspects about a
> design that designers need to optimize, for example, product usage,
> functions and features, cost range, and aesthetics. Modern designers even
> need to consider more design objectives because the new design
> characteristics may finally determine the success or failure of a product.
> Design methodologies that consider these new characteristics are called
> design for X (DFX). Specific design methodologies include design for
> reliability (DFR), design for serviceability (DFS), design for supply chain
> (DFSC), design for scalability (DFSc), design for energy efficiency and
> environment (DFEE), design for testability (DFT),and so on.
This would be better to explain in changelog, not many people would be
familiar with these terms...
>
> For clarity, I've replaced it in v3 with something easier to understand.
> > > This patch adds dump of registers with debugfs for HIP08
> > > and HIP09 DMA driver.
> > >
> > > Reported-by: kernel test robot <lkp@intel.com>
> >
> > ?
> >
> The kernel test robot raised an issue and asked me to modify it and add
> "Reported-by: kernel test robot <lkp@intel.com>" to the commit message, so I
> did, and it was removed in v3.
ok
--
~Vinod
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [PATCH v2 6/7] dmaengine: hisilicon: Add dfx feature for hisi dma driver
2022-07-26 12:39 ` Vinod Koul
@ 2022-07-27 0:58 ` Jie Hai
0 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-07-27 0:58 UTC (permalink / raw)
To: Vinod Koul; +Cc: wangzhou1, dmaengine, linux-kernel
On 2022/7/26 20:39, Vinod Koul wrote:
> On 26-07-22, 09:43, Jie Hai wrote:
>> On 2022/7/21 21:25, Vinod Koul wrote:
>>> On 29-06-22, 11:55, Jie Hai wrote:
>>>
>>> What does dfx mean in title?
>>>
>> Thanks for the question.
>>
>> DFX＝"Design for X", I quote its definition as follows:
>> The product definition puts forward a list of specific aspects about a
>> design that designers need to optimize, for example, product usage,
>> functions and features, cost range, and aesthetics. Modern designers even
>> need to consider more design objectives because the new design
>> characteristics may finally determine the success or failure of a product.
>> Design methodologies that consider these new characteristics are called
>> design for X (DFX). Specific design methodologies include design for
>> reliability (DFR), design for serviceability (DFS), design for supply chain
>> (DFSC), design for scalability (DFSc), design for energy efficiency and
>> environment (DFEE), design for testability (DFT),and so on.
>
> This would be better to explain in changelog, not many people would be
> familiar with these terms...
>
Thanks. As I wrote below, I've replaced it in v3 with something easier
to understand, which is "dump regs to debugfs".
>>
>> For clarity, I've replaced it in v3 with something easier to understand.
>>>> This patch adds dump of registers with debugfs for HIP08
>>>> and HIP09 DMA driver.
>>>>
>>>> Reported-by: kernel test robot <lkp@intel.com>
>>>
>>> ?
>>>
>> The kernel test robot raised an issue and asked me to modify it and add
>> "Reported-by: kernel test robot <lkp@intel.com>" to the commit message, so I
>> did, and it was removed in v3.
>
> ok
>
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (9 preceding siblings ...)
2022-07-26 1:35 ` [PATCH v3 " Jie Hai
@ 2022-08-02 10:12 ` Jie Hai
2022-08-02 10:12 ` [PATCH v4 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
` (6 more replies)
2022-08-05 8:25 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
11 siblings, 7 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-02 10:12 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they share the
same pci device id but different pci revision and register layouts.
The original version supports HiSilicon IP08 but not HiSilicon IP09.
This series support DMA driver for HIP08 and HIP09:
1. Fix bugs for HIP08 DMA driver
- Disable hardware channels when driver detached
- Update cq_head whenever accessed it
- Support multi-thread for one DMA channel
2. Use macros instead of magic number
3. Add support for HIP09 DMA driver
4. Add debugfs for HIP08 and HIP09 DMA driver
5. Add myself as maintainer of hisi_dma.c
Changes since version 3:
- remove reduldant braces
- add "Acked-by: Zhou Wang <wangzhou1@hisilicon.com>" in commit log
Changes since version 2:
- fix unnecessary line breaks
- fix register bit with BIT/GENMASK and adjust hisi_dma_update_bit to it
- remove "Reported-by" in commit message
- use dmaengine root instead of hisi_dma root
- ignore errors for creating debugfs
Changes since version 1:
- remove error changes casuse compile failure
- remove reduldant "*" in comment
- remove debugfs-hisi-dma doc and path in MAINTAINERS
Jie Hai (7):
dmaengine: hisilicon: Disable channels when unregister hisi_dma
dmaengine: hisilicon: Fix CQ head update
dmaengine: hisilicon: Add multi-thread support for a DMA channel
dmaengine: hisilicon: Use macros instead of magic number
dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
dmaengine: hisilicon: Dump regs to debugfs
MAINTAINERS: Add myself as maintainer for hisi_dma
MAINTAINERS | 1 +
drivers/dma/hisi_dma.c | 650 +++++++++++++++++++++++++++++++++++------
2 files changed, 555 insertions(+), 96 deletions(-)
--
2.33.0
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v4 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma
2022-08-02 10:12 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
@ 2022-08-02 10:12 ` Jie Hai
2022-08-02 10:12 ` [PATCH v4 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
` (5 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-02 10:12 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
When hisi_dma is unloaded or unbinded, all of channels should be
disabled. This patch disables DMA channels when driver is unloaded
or unbinded.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 43817ced3a3e..98bc488893cc 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -180,7 +180,8 @@ static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
}
-static void hisi_dma_reset_hw_chan(struct hisi_dma_chan *chan)
+static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
+ bool disable)
{
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
u32 index = chan->qp_num, tmp;
@@ -201,8 +202,11 @@ static void hisi_dma_reset_hw_chan(struct hisi_dma_chan *chan)
hisi_dma_do_reset(hdma_dev, index);
hisi_dma_reset_qp_point(hdma_dev, index);
hisi_dma_pause_dma(hdma_dev, index, false);
- hisi_dma_enable_dma(hdma_dev, index, true);
- hisi_dma_unmask_irq(hdma_dev, index);
+
+ if (!disable) {
+ hisi_dma_enable_dma(hdma_dev, index, true);
+ hisi_dma_unmask_irq(hdma_dev, index);
+ }
ret = readl_relaxed_poll_timeout(hdma_dev->base +
HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
@@ -218,7 +222,7 @@ static void hisi_dma_free_chan_resources(struct dma_chan *c)
struct hisi_dma_chan *chan = to_hisi_dma_chan(c);
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
- hisi_dma_reset_hw_chan(chan);
+ hisi_dma_reset_or_disable_hw_chan(chan, false);
vchan_free_chan_resources(&chan->vc);
memset(chan->sq, 0, sizeof(struct hisi_dma_sqe) * hdma_dev->chan_depth);
@@ -394,7 +398,7 @@ static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
static void hisi_dma_disable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- hisi_dma_reset_hw_chan(&hdma_dev->chan[qp_index]);
+ hisi_dma_reset_or_disable_hw_chan(&hdma_dev->chan[qp_index], true);
}
static void hisi_dma_enable_qps(struct hisi_dma_dev *hdma_dev)
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 2/7] dmaengine: hisilicon: Fix CQ head update
2022-08-02 10:12 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-08-02 10:12 ` [PATCH v4 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
@ 2022-08-02 10:12 ` Jie Hai
2022-08-02 10:12 ` [PATCH v4 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
` (4 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-02 10:12 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
After completion of data transfer of one or multiple descriptors,
the completion status and the current head pointer to submission
queue are written into the CQ and interrupt can be generated to
inform the software. In interrupt process CQ is read and cq_head
is updated.
hisi_dma_irq updates cq_head only when the completion status is
success. When an abnormal interrupt reports, cq_head will not update
which will cause subsequent interrupt processes read the error CQ
and never report the correct status.
This patch updates cq_head whenever CQ is accessed.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 98bc488893cc..837f7e4adfa6 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -436,12 +436,10 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
desc = chan->desc;
cqe = chan->cq + chan->cq_head;
if (desc) {
+ chan->cq_head = (chan->cq_head + 1) % hdma_dev->chan_depth;
+ hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR,
+ chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
- chan->cq_head = (chan->cq_head + 1) %
- hdma_dev->chan_depth;
- hisi_dma_chan_write(hdma_dev->base,
- HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
- chan->cq_head);
vchan_cookie_complete(&desc->vd);
} else {
dev_err(&hdma_dev->pdev->dev, "task error!\n");
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel
2022-08-02 10:12 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-08-02 10:12 ` [PATCH v4 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
2022-08-02 10:12 ` [PATCH v4 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
@ 2022-08-02 10:12 ` Jie Hai
2022-08-02 10:12 ` [PATCH v4 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
` (3 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-02 10:12 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
When we get a DMA channel and try to use it in multiple threads it
will cause oops and hanging the system.
% echo 100 > /sys/module/dmatest/parameters/threads_per_chan
% echo 100 > /sys/module/dmatest/parameters/iterations
% echo 1 > /sys/module/dmatest/parameters/run
[383493.327077] Unable to handle kernel paging request at virtual
address dead000000000108
[383493.335103] Mem abort info:
[383493.335103] ESR = 0x96000044
[383493.335105] EC = 0x25: DABT (current EL), IL = 32 bits
[383493.335107] SET = 0, FnV = 0
[383493.335108] EA = 0, S1PTW = 0
[383493.335109] FSC = 0x04: level 0 translation fault
[383493.335110] Data abort info:
[383493.335111] ISV = 0, ISS = 0x00000044
[383493.364739] CM = 0, WnR = 1
[383493.367793] [dead000000000108] address between user and kernel
address ranges
[383493.375021] Internal error: Oops: 96000044 [#1] PREEMPT SMP
[383493.437574] CPU: 63 PID: 27895 Comm: dma0chan0-copy2 Kdump:
loaded Tainted: GO 5.17.0-rc4+ #2
[383493.457851] pstate: 204000c9 (nzCv daIF +PAN -UAO -TCO -DIT
-SSBS BTYPE=--)
[383493.465331] pc : vchan_tx_submit+0x64/0xa0
[383493.469957] lr : vchan_tx_submit+0x34/0xa0
This occurs because the transmission timed out, and that's due
to data race. Each thread rewrite channels's descriptor as soon as
device_issue_pending is called. It leads to the situation that
the driver thinks that it uses the right descriptor in interrupt
handler while channels's descriptor has been changed by other
thread. The descriptor which in fact reported interrupt will not
be handled any more, as well as its tx->callback.
That's why timeout reports.
With current fixes channels' descriptor changes it's value only
when it has been used. A new descriptor is acquired from
vc->desc_issued queue that is already filled with descriptors
that are ready to be sent. Threads have no direct access to DMA
channel descriptor. In case of channel's descriptor is busy, try
to submit to HW again when a descriptor is completed. In this case,
vc->desc_issued may be empty when hisi_dma_start_transfer is called,
so delete error reporting on this. Now it is just possible to queue
a descriptor for further processing.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 837f7e4adfa6..0233b42143c7 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -271,7 +271,6 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
vd = vchan_next_desc(&chan->vc);
if (!vd) {
- dev_err(&hdma_dev->pdev->dev, "no issued task!\n");
chan->desc = NULL;
return;
}
@@ -303,7 +302,7 @@ static void hisi_dma_issue_pending(struct dma_chan *c)
spin_lock_irqsave(&chan->vc.lock, flags);
- if (vchan_issue_pending(&chan->vc))
+ if (vchan_issue_pending(&chan->vc) && !chan->desc)
hisi_dma_start_transfer(chan);
spin_unlock_irqrestore(&chan->vc.lock, flags);
@@ -441,11 +440,10 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
vchan_cookie_complete(&desc->vd);
+ hisi_dma_start_transfer(chan);
} else {
dev_err(&hdma_dev->pdev->dev, "task error!\n");
}
-
- chan->desc = NULL;
}
spin_unlock(&chan->vc.lock);
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 4/7] dmaengine: hisilicon: Use macros instead of magic number
2022-08-02 10:12 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (2 preceding siblings ...)
2022-08-02 10:12 ` [PATCH v4 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
@ 2022-08-02 10:12 ` Jie Hai
2022-08-02 10:12 ` [PATCH v4 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
` (2 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-02 10:12 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
readl_relaxed_poll_timeout() uses magic numbers 10 and 1000, which
indicate maximum time to sleep between reads in us and timeout in us,
respectively.
Use macros HISI_DMA_POLL_Q_STS_DELAY_US and
HISI_DMA_POLL_Q_STS_TIME_OUT_US instead of these two numbers.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 19 +++++++++++++------
1 file changed, 13 insertions(+), 6 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 0233b42143c7..5d62fe62ba00 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -36,6 +36,9 @@
#define PCI_BAR_2 2
+#define HISI_DMA_POLL_Q_STS_DELAY_US 10
+#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+
enum hisi_dma_mode {
EP = 0,
RC,
@@ -185,15 +188,19 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
{
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
u32 index = chan->qp_num, tmp;
+ void __iomem *addr;
int ret;
hisi_dma_pause_dma(hdma_dev, index, true);
hisi_dma_enable_dma(hdma_dev, index, false);
hisi_dma_mask_irq(hdma_dev, index);
- ret = readl_relaxed_poll_timeout(hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN, 10, 1000);
+ addr = hdma_dev->base +
+ HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET;
+
+ ret = readl_relaxed_poll_timeout(addr, tmp,
+ FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN,
+ HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "disable channel timeout!\n");
WARN_ON(1);
@@ -208,9 +215,9 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
hisi_dma_unmask_irq(hdma_dev, index);
}
- ret = readl_relaxed_poll_timeout(hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE, 10, 1000);
+ ret = readl_relaxed_poll_timeout(addr, tmp,
+ FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE,
+ HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "reset channel timeout!\n");
WARN_ON(1);
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
2022-08-02 10:12 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (3 preceding siblings ...)
2022-08-02 10:12 ` [PATCH v4 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
@ 2022-08-02 10:12 ` Jie Hai
2022-08-05 7:41 ` Jie Hai
2022-08-02 10:12 ` [PATCH v4 6/7] dmaengine: hisilicon: Dump regs to debugfs Jie Hai
2022-08-02 10:12 ` [PATCH v4 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
6 siblings, 1 reply; 64+ messages in thread
From: Jie Hai @ 2022-08-02 10:12 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they
have the same pci device id but different pci revision.
Unfortunately, they have different register layouts, so
the origin driver cannot run on HiSilicon IP09 correctly.
This patch enables the driver to adapt to HiSilicon IP09.
HiSilicon IP09 offers 4 channels, each channel has a send
queue, a complete queue and an interrupt to help to do tasks.
This DMA engine can do memory copy between memory blocks.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 379 ++++++++++++++++++++++++++++++++---------
1 file changed, 298 insertions(+), 81 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 5d62fe62ba00..5fa3b6fa0529 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
-/* Copyright(c) 2019 HiSilicon Limited. */
+/* Copyright(c) 2019-2022 HiSilicon Limited. */
+
#include <linux/bitfield.h>
#include <linux/dmaengine.h>
#include <linux/init.h>
@@ -9,35 +10,85 @@
#include <linux/spinlock.h>
#include "virt-dma.h"
-#define HISI_DMA_SQ_BASE_L 0x0
-#define HISI_DMA_SQ_BASE_H 0x4
-#define HISI_DMA_SQ_DEPTH 0x8
-#define HISI_DMA_SQ_TAIL_PTR 0xc
-#define HISI_DMA_CQ_BASE_L 0x10
-#define HISI_DMA_CQ_BASE_H 0x14
-#define HISI_DMA_CQ_DEPTH 0x18
-#define HISI_DMA_CQ_HEAD_PTR 0x1c
-#define HISI_DMA_CTRL0 0x20
-#define HISI_DMA_CTRL0_QUEUE_EN_S 0
-#define HISI_DMA_CTRL0_QUEUE_PAUSE_S 4
-#define HISI_DMA_CTRL1 0x24
-#define HISI_DMA_CTRL1_QUEUE_RESET_S 0
-#define HISI_DMA_Q_FSM_STS 0x30
-#define HISI_DMA_FSM_STS_MASK GENMASK(3, 0)
-#define HISI_DMA_INT_STS 0x40
-#define HISI_DMA_INT_STS_MASK GENMASK(12, 0)
-#define HISI_DMA_INT_MSK 0x44
-#define HISI_DMA_MODE 0x217c
-#define HISI_DMA_OFFSET 0x100
-
-#define HISI_DMA_MSI_NUM 32
-#define HISI_DMA_CHAN_NUM 30
-#define HISI_DMA_Q_DEPTH_VAL 1024
-
-#define PCI_BAR_2 2
-
-#define HISI_DMA_POLL_Q_STS_DELAY_US 10
-#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+/* HiSilicon DMA register common field define */
+#define HISI_DMA_Q_SQ_BASE_L 0x0
+#define HISI_DMA_Q_SQ_BASE_H 0x4
+#define HISI_DMA_Q_SQ_DEPTH 0x8
+#define HISI_DMA_Q_SQ_TAIL_PTR 0xc
+#define HISI_DMA_Q_CQ_BASE_L 0x10
+#define HISI_DMA_Q_CQ_BASE_H 0x14
+#define HISI_DMA_Q_CQ_DEPTH 0x18
+#define HISI_DMA_Q_CQ_HEAD_PTR 0x1c
+#define HISI_DMA_Q_CTRL0 0x20
+#define HISI_DMA_Q_CTRL0_QUEUE_EN BIT(0)
+#define HISI_DMA_Q_CTRL0_QUEUE_PAUSE BIT(4)
+#define HISI_DMA_Q_CTRL1 0x24
+#define HISI_DMA_Q_CTRL1_QUEUE_RESET BIT(0)
+#define HISI_DMA_Q_FSM_STS 0x30
+#define HISI_DMA_Q_FSM_STS_MASK GENMASK(3, 0)
+#define HISI_DMA_Q_ERR_INT_NUM0 0x84
+#define HISI_DMA_Q_ERR_INT_NUM1 0x88
+#define HISI_DMA_Q_ERR_INT_NUM2 0x8c
+
+/* HiSilicon IP08 DMA register and field define */
+#define HISI_DMA_HIP08_MODE 0x217C
+#define HISI_DMA_HIP08_Q_BASE 0x0
+#define HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN BIT(2)
+#define HISI_DMA_HIP08_Q_INT_STS 0x40
+#define HISI_DMA_HIP08_Q_INT_MSK 0x44
+#define HISI_DMA_HIP08_Q_INT_STS_MASK GENMASK(14, 0)
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM3 0x90
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM4 0x94
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM5 0x98
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM6 0x48
+#define HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT BIT(24)
+
+/* HiSilicon IP09 DMA register and field define */
+#define HISI_DMA_HIP09_DMA_FLR_DISABLE 0xA00
+#define HISI_DMA_HIP09_DMA_FLR_DISABLE_B BIT(0)
+#define HISI_DMA_HIP09_Q_BASE 0x2000
+#define HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN GENMASK(31, 28)
+#define HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT BIT(26)
+#define HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT BIT(27)
+#define HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE BIT(2)
+#define HISI_DMA_HIP09_Q_INT_STS 0x40
+#define HISI_DMA_HIP09_Q_INT_MSK 0x44
+#define HISI_DMA_HIP09_Q_INT_STS_MASK 0x1
+#define HISI_DMA_HIP09_Q_ERR_INT_STS 0x48
+#define HISI_DMA_HIP09_Q_ERR_INT_MSK 0x4C
+#define HISI_DMA_HIP09_Q_ERR_INT_STS_MASK GENMASK(18, 1)
+#define HISI_DMA_HIP09_PORT_CFG_REG(port_id) (0x800 + \
+ (port_id) * 0x20)
+#define HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B BIT(16)
+
+#define HISI_DMA_HIP09_MAX_PORT_NUM 16
+
+#define HISI_DMA_HIP08_MSI_NUM 32
+#define HISI_DMA_HIP08_CHAN_NUM 30
+#define HISI_DMA_HIP09_MSI_NUM 4
+#define HISI_DMA_HIP09_CHAN_NUM 4
+#define HISI_DMA_REVISION_HIP08B 0x21
+#define HISI_DMA_REVISION_HIP09A 0x30
+
+#define HISI_DMA_Q_OFFSET 0x100
+#define HISI_DMA_Q_DEPTH_VAL 1024
+
+#define PCI_BAR_2 2
+
+#define HISI_DMA_POLL_Q_STS_DELAY_US 10
+#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+
+/*
+ * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
+ * have the same pci device id but different pci revision.
+ * Unfortunately, they have different register layouts, so two layout
+ * enumerations are defined.
+ */
+enum hisi_dma_reg_layout {
+ HISI_DMA_REG_LAYOUT_INVALID = 0,
+ HISI_DMA_REG_LAYOUT_HIP08,
+ HISI_DMA_REG_LAYOUT_HIP09
+};
enum hisi_dma_mode {
EP = 0,
@@ -108,9 +159,45 @@ struct hisi_dma_dev {
struct dma_device dma_dev;
u32 chan_num;
u32 chan_depth;
+ enum hisi_dma_reg_layout reg_layout;
+ void __iomem *queue_base; /* queue region start of register */
struct hisi_dma_chan chan[];
};
+static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_REG_LAYOUT_HIP08;
+ else if (pdev->revision >= HISI_DMA_REVISION_HIP09A)
+ return HISI_DMA_REG_LAYOUT_HIP09;
+
+ return HISI_DMA_REG_LAYOUT_INVALID;
+}
+
+static u32 hisi_dma_get_chan_num(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_CHAN_NUM;
+
+ return HISI_DMA_HIP09_CHAN_NUM;
+}
+
+static u32 hisi_dma_get_msi_num(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_MSI_NUM;
+
+ return HISI_DMA_HIP09_MSI_NUM;
+}
+
+static u32 hisi_dma_get_queue_base(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_Q_BASE;
+
+ return HISI_DMA_HIP09_Q_BASE;
+}
+
static inline struct hisi_dma_chan *to_hisi_dma_chan(struct dma_chan *c)
{
return container_of(c, struct hisi_dma_chan, vc.chan);
@@ -124,7 +211,7 @@ static inline struct hisi_dma_desc *to_hisi_dma_desc(struct virt_dma_desc *vd)
static inline void hisi_dma_chan_write(void __iomem *base, u32 reg, u32 index,
u32 val)
{
- writel_relaxed(val, base + reg + index * HISI_DMA_OFFSET);
+ writel_relaxed(val, base + reg + index * HISI_DMA_Q_OFFSET);
}
static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
@@ -132,55 +219,83 @@ static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
u32 tmp;
tmp = readl_relaxed(addr);
- tmp = val ? tmp | BIT(pos) : tmp & ~BIT(pos);
+ tmp = val ? tmp | pos : tmp & ~pos;
writel_relaxed(tmp, addr);
}
static void hisi_dma_pause_dma(struct hisi_dma_dev *hdma_dev, u32 index,
bool pause)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_PAUSE_S, pause);
+ addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_PAUSE, pause);
}
static void hisi_dma_enable_dma(struct hisi_dma_dev *hdma_dev, u32 index,
bool enable)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_EN_S, enable);
+ addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_EN, enable);
}
static void hisi_dma_mask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_INT_MSK, qp_index,
- HISI_DMA_INT_STS_MASK);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
+ qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
+ else {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
+ qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
+ qp_index,
+ HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
+ }
}
static void hisi_dma_unmask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- void __iomem *base = hdma_dev->base;
-
- hisi_dma_chan_write(base, HISI_DMA_INT_STS, qp_index,
- HISI_DMA_INT_STS_MASK);
- hisi_dma_chan_write(base, HISI_DMA_INT_MSK, qp_index, 0);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_STS,
+ qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
+ qp_index, 0);
+ } else {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_STS,
+ qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_STS,
+ qp_index,
+ HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
+ qp_index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
+ qp_index, 0);
+ }
}
static void hisi_dma_do_reset(struct hisi_dma_dev *hdma_dev, u32 index)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL1 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL1_QUEUE_RESET_S, 1);
+ addr = hdma_dev->queue_base +
+ HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL1_QUEUE_RESET, 1);
}
static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
{
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, index, 0);
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
}
static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
@@ -196,10 +311,10 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
hisi_dma_mask_irq(hdma_dev, index);
addr = hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET;
+ HISI_DMA_Q_FSM_STS + index * HISI_DMA_Q_OFFSET;
ret = readl_relaxed_poll_timeout(addr, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN,
+ FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) != RUN,
HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "disable channel timeout!\n");
@@ -216,7 +331,7 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
}
ret = readl_relaxed_poll_timeout(addr, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE,
+ FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) == IDLE,
HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "reset channel timeout!\n");
@@ -298,8 +413,8 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
chan->sq_tail = (chan->sq_tail + 1) % hdma_dev->chan_depth;
/* update sq_tail to trigger a new task */
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, chan->qp_num,
- chan->sq_tail);
+ hisi_dma_chan_write(hdma_dev->queue_base, HISI_DMA_Q_SQ_TAIL_PTR,
+ chan->qp_num, chan->sq_tail);
}
static void hisi_dma_issue_pending(struct dma_chan *c)
@@ -373,26 +488,86 @@ static int hisi_dma_alloc_qps_mem(struct hisi_dma_dev *hdma_dev)
static void hisi_dma_init_hw_qp(struct hisi_dma_dev *hdma_dev, u32 index)
{
struct hisi_dma_chan *chan = &hdma_dev->chan[index];
+ void __iomem *q_base = hdma_dev->queue_base;
u32 hw_depth = hdma_dev->chan_depth - 1;
- void __iomem *base = hdma_dev->base;
+ void __iomem *addr;
+ u32 tmp;
/* set sq, cq base */
- hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_L, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_L, index,
lower_32_bits(chan->sq_dma));
- hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_H, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_H, index,
upper_32_bits(chan->sq_dma));
- hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_L, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_L, index,
lower_32_bits(chan->cq_dma));
- hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_H, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_H, index,
upper_32_bits(chan->cq_dma));
/* set sq, cq depth */
- hisi_dma_chan_write(base, HISI_DMA_SQ_DEPTH, index, hw_depth);
- hisi_dma_chan_write(base, HISI_DMA_CQ_DEPTH, index, hw_depth);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_DEPTH, index, hw_depth);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_DEPTH, index, hw_depth);
/* init sq tail and cq head */
- hisi_dma_chan_write(base, HISI_DMA_SQ_TAIL_PTR, index, 0);
- hisi_dma_chan_write(base, HISI_DMA_CQ_HEAD_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
+
+ /* init error interrupt stats */
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM0, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM1, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM2, index, 0);
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM3,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM4,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM5,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM6,
+ index, 0);
+ /*
+ * init SQ/CQ direction selecting register.
+ * "0" is to local side and "1" is to remote side.
+ */
+ addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT, 0);
+
+ /*
+ * 0 - Continue to next descriptor if error occurs.
+ * 1 - Abort the DMA queue if error occurs.
+ */
+ hisi_dma_update_bit(addr,
+ HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN, 0);
+ } else {
+ addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
+
+ /*
+ * init SQ/CQ direction selecting register.
+ * "0" is to local side and "1" is to remote side.
+ */
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT, 0);
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT, 0);
+
+ /*
+ * 0 - Continue to next descriptor if error occurs.
+ * 1 - Abort the DMA queue if error occurs.
+ */
+
+ tmp = readl_relaxed(addr);
+ tmp &= ~HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN;
+ writel_relaxed(tmp, addr);
+
+ /*
+ * 0 - dma should process FLR whith CPU.
+ * 1 - dma not process FLR, only cpu process FLR.
+ */
+ addr = q_base + HISI_DMA_HIP09_DMA_FLR_DISABLE +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_DMA_FLR_DISABLE_B, 0);
+
+ addr = q_base + HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE, 1);
+ }
}
static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
@@ -436,14 +611,16 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
struct hisi_dma_desc *desc;
struct hisi_dma_cqe *cqe;
+ void __iomem *q_base;
spin_lock(&chan->vc.lock);
desc = chan->desc;
cqe = chan->cq + chan->cq_head;
+ q_base = hdma_dev->queue_base;
if (desc) {
chan->cq_head = (chan->cq_head + 1) % hdma_dev->chan_depth;
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
vchan_cookie_complete(&desc->vd);
@@ -504,16 +681,58 @@ static void hisi_dma_disable_hw_channels(void *data)
static void hisi_dma_set_mode(struct hisi_dma_dev *hdma_dev,
enum hisi_dma_mode mode)
{
- writel_relaxed(mode == RC ? 1 : 0, hdma_dev->base + HISI_DMA_MODE);
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ writel_relaxed(mode == RC ? 1 : 0,
+ hdma_dev->base + HISI_DMA_HIP08_MODE);
+}
+
+static void hisi_dma_init_hw(struct hisi_dma_dev *hdma_dev)
+{
+ void __iomem *addr;
+ int i;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP09) {
+ for (i = 0; i < HISI_DMA_HIP09_MAX_PORT_NUM; i++) {
+ addr = hdma_dev->base + HISI_DMA_HIP09_PORT_CFG_REG(i);
+ hisi_dma_update_bit(addr,
+ HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B, 1);
+ }
+ }
+}
+
+static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
+{
+ struct dma_device *dma_dev;
+
+ dma_dev = &hdma_dev->dma_dev;
+ dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
+ dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
+ dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
+ dma_dev->device_tx_status = hisi_dma_tx_status;
+ dma_dev->device_issue_pending = hisi_dma_issue_pending;
+ dma_dev->device_terminate_all = hisi_dma_terminate_all;
+ dma_dev->device_synchronize = hisi_dma_synchronize;
+ dma_dev->directions = BIT(DMA_MEM_TO_MEM);
+ dma_dev->dev = &hdma_dev->pdev->dev;
+ INIT_LIST_HEAD(&dma_dev->channels);
}
static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
+ enum hisi_dma_reg_layout reg_layout;
struct device *dev = &pdev->dev;
struct hisi_dma_dev *hdma_dev;
struct dma_device *dma_dev;
+ u32 chan_num;
+ u32 msi_num;
int ret;
+ reg_layout = hisi_dma_get_reg_layout(pdev);
+ if (reg_layout == HISI_DMA_REG_LAYOUT_INVALID) {
+ dev_err(dev, "unsupported device!\n");
+ return -EINVAL;
+ }
+
ret = pcim_enable_device(pdev);
if (ret) {
dev_err(dev, "failed to enable device mem!\n");
@@ -530,40 +749,37 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (ret)
return ret;
- hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, HISI_DMA_CHAN_NUM), GFP_KERNEL);
+ chan_num = hisi_dma_get_chan_num(pdev);
+ hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, chan_num),
+ GFP_KERNEL);
if (!hdma_dev)
return -EINVAL;
hdma_dev->base = pcim_iomap_table(pdev)[PCI_BAR_2];
hdma_dev->pdev = pdev;
- hdma_dev->chan_num = HISI_DMA_CHAN_NUM;
hdma_dev->chan_depth = HISI_DMA_Q_DEPTH_VAL;
+ hdma_dev->chan_num = chan_num;
+ hdma_dev->reg_layout = reg_layout;
+ hdma_dev->queue_base = hdma_dev->base + hisi_dma_get_queue_base(pdev);
pci_set_drvdata(pdev, hdma_dev);
pci_set_master(pdev);
+ msi_num = hisi_dma_get_msi_num(pdev);
+
/* This will be freed by 'pcim_release()'. See 'pcim_enable_device()' */
- ret = pci_alloc_irq_vectors(pdev, HISI_DMA_MSI_NUM, HISI_DMA_MSI_NUM,
- PCI_IRQ_MSI);
+ ret = pci_alloc_irq_vectors(pdev, msi_num, msi_num, PCI_IRQ_MSI);
if (ret < 0) {
dev_err(dev, "Failed to allocate MSI vectors!\n");
return ret;
}
- dma_dev = &hdma_dev->dma_dev;
- dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
- dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
- dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
- dma_dev->device_tx_status = hisi_dma_tx_status;
- dma_dev->device_issue_pending = hisi_dma_issue_pending;
- dma_dev->device_terminate_all = hisi_dma_terminate_all;
- dma_dev->device_synchronize = hisi_dma_synchronize;
- dma_dev->directions = BIT(DMA_MEM_TO_MEM);
- dma_dev->dev = dev;
- INIT_LIST_HEAD(&dma_dev->channels);
+ hisi_dma_init_dma_dev(hdma_dev);
hisi_dma_set_mode(hdma_dev, RC);
+ hisi_dma_init_hw(hdma_dev);
+
ret = hisi_dma_enable_hw_channels(hdma_dev);
if (ret < 0) {
dev_err(dev, "failed to enable hw channel!\n");
@@ -575,6 +791,7 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (ret)
return ret;
+ dma_dev = &hdma_dev->dma_dev;
ret = dmaenginem_async_device_register(dma_dev);
if (ret < 0)
dev_err(dev, "failed to register device!\n");
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 6/7] dmaengine: hisilicon: Dump regs to debugfs
2022-08-02 10:12 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (4 preceding siblings ...)
2022-08-02 10:12 ` [PATCH v4 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
@ 2022-08-02 10:12 ` Jie Hai
2022-08-02 10:12 ` [PATCH v4 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-02 10:12 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
This patch adds dump of registers with debugfs for HIP08
and HIP09 DMA driver.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 238 ++++++++++++++++++++++++++++++++++++++++-
1 file changed, 236 insertions(+), 2 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 5fa3b6fa0529..30a9cf21edf0 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -78,6 +78,8 @@
#define HISI_DMA_POLL_Q_STS_DELAY_US 10
#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+#define HISI_DMA_MAX_DIR_NAME_LEN 128
+
/*
* The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
* have the same pci device id but different pci revision.
@@ -164,6 +166,123 @@ struct hisi_dma_dev {
struct hisi_dma_chan chan[];
};
+#ifdef CONFIG_DEBUG_FS
+
+static const struct debugfs_reg32 hisi_dma_comm_chan_regs[] = {
+ {"DMA_QUEUE_SQ_DEPTH ", 0x0008ull},
+ {"DMA_QUEUE_SQ_TAIL_PTR ", 0x000Cull},
+ {"DMA_QUEUE_CQ_DEPTH ", 0x0018ull},
+ {"DMA_QUEUE_CQ_HEAD_PTR ", 0x001Cull},
+ {"DMA_QUEUE_CTRL0 ", 0x0020ull},
+ {"DMA_QUEUE_CTRL1 ", 0x0024ull},
+ {"DMA_QUEUE_FSM_STS ", 0x0030ull},
+ {"DMA_QUEUE_SQ_STS ", 0x0034ull},
+ {"DMA_QUEUE_CQ_TAIL_PTR ", 0x003Cull},
+ {"DMA_QUEUE_INT_STS ", 0x0040ull},
+ {"DMA_QUEUE_INT_MSK ", 0x0044ull},
+ {"DMA_QUEUE_INT_RO ", 0x006Cull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip08_chan_regs[] = {
+ {"DMA_QUEUE_BYTE_CNT ", 0x0038ull},
+ {"DMA_ERR_INT_NUM6 ", 0x0048ull},
+ {"DMA_QUEUE_DESP0 ", 0x0050ull},
+ {"DMA_QUEUE_DESP1 ", 0x0054ull},
+ {"DMA_QUEUE_DESP2 ", 0x0058ull},
+ {"DMA_QUEUE_DESP3 ", 0x005Cull},
+ {"DMA_QUEUE_DESP4 ", 0x0074ull},
+ {"DMA_QUEUE_DESP5 ", 0x0078ull},
+ {"DMA_QUEUE_DESP6 ", 0x007Cull},
+ {"DMA_QUEUE_DESP7 ", 0x0080ull},
+ {"DMA_ERR_INT_NUM0 ", 0x0084ull},
+ {"DMA_ERR_INT_NUM1 ", 0x0088ull},
+ {"DMA_ERR_INT_NUM2 ", 0x008Cull},
+ {"DMA_ERR_INT_NUM3 ", 0x0090ull},
+ {"DMA_ERR_INT_NUM4 ", 0x0094ull},
+ {"DMA_ERR_INT_NUM5 ", 0x0098ull},
+ {"DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip09_chan_regs[] = {
+ {"DMA_QUEUE_ERR_INT_STS ", 0x0048ull},
+ {"DMA_QUEUE_ERR_INT_MSK ", 0x004Cull},
+ {"DFX_SQ_READ_ERR_PTR ", 0x0068ull},
+ {"DFX_DMA_ERR_INT_NUM0 ", 0x0084ull},
+ {"DFX_DMA_ERR_INT_NUM1 ", 0x0088ull},
+ {"DFX_DMA_ERR_INT_NUM2 ", 0x008Cull},
+ {"DFX_DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip08_comm_regs[] = {
+ {"DMA_ECC_ERR_ADDR ", 0x2004ull},
+ {"DMA_ECC_ECC_CNT ", 0x2014ull},
+ {"COMMON_AND_CH_ERR_STS ", 0x2030ull},
+ {"LOCAL_CPL_ID_STS_0 ", 0x20E0ull},
+ {"LOCAL_CPL_ID_STS_1 ", 0x20E4ull},
+ {"LOCAL_CPL_ID_STS_2 ", 0x20E8ull},
+ {"LOCAL_CPL_ID_STS_3 ", 0x20ECull},
+ {"LOCAL_TLP_NUM ", 0x2158ull},
+ {"SQCQ_TLP_NUM ", 0x2164ull},
+ {"CPL_NUM ", 0x2168ull},
+ {"INF_BACK_PRESS_STS ", 0x2170ull},
+ {"DMA_CH_RAS_LEVEL ", 0x2184ull},
+ {"DMA_CM_RAS_LEVEL ", 0x2188ull},
+ {"DMA_CH_ERR_STS ", 0x2190ull},
+ {"DMA_CH_DONE_STS ", 0x2194ull},
+ {"DMA_SQ_TAG_STS_0 ", 0x21A0ull},
+ {"DMA_SQ_TAG_STS_1 ", 0x21A4ull},
+ {"DMA_SQ_TAG_STS_2 ", 0x21A8ull},
+ {"DMA_SQ_TAG_STS_3 ", 0x21ACull},
+ {"LOCAL_P_ID_STS_0 ", 0x21B0ull},
+ {"LOCAL_P_ID_STS_1 ", 0x21B4ull},
+ {"LOCAL_P_ID_STS_2 ", 0x21B8ull},
+ {"LOCAL_P_ID_STS_3 ", 0x21BCull},
+ {"DMA_PREBUFF_INFO_0 ", 0x2200ull},
+ {"DMA_CM_TABLE_INFO_0 ", 0x2220ull},
+ {"DMA_CM_CE_RO ", 0x2244ull},
+ {"DMA_CM_NFE_RO ", 0x2248ull},
+ {"DMA_CM_FE_RO ", 0x224Cull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip09_comm_regs[] = {
+ {"COMMON_AND_CH_ERR_STS ", 0x0030ull},
+ {"DMA_PORT_IDLE_STS ", 0x0150ull},
+ {"DMA_CH_RAS_LEVEL ", 0x0184ull},
+ {"DMA_CM_RAS_LEVEL ", 0x0188ull},
+ {"DMA_CM_CE_RO ", 0x0244ull},
+ {"DMA_CM_NFE_RO ", 0x0248ull},
+ {"DMA_CM_FE_RO ", 0x024Cull},
+ {"DFX_INF_BACK_PRESS_STS0 ", 0x1A40ull},
+ {"DFX_INF_BACK_PRESS_STS1 ", 0x1A44ull},
+ {"DFX_INF_BACK_PRESS_STS2 ", 0x1A48ull},
+ {"DFX_DMA_WRR_DISABLE ", 0x1A4Cull},
+ {"DFX_PA_REQ_TLP_NUM ", 0x1C00ull},
+ {"DFX_PA_BACK_TLP_NUM ", 0x1C04ull},
+ {"DFX_PA_RETRY_TLP_NUM ", 0x1C08ull},
+ {"DFX_LOCAL_NP_TLP_NUM ", 0x1C0Cull},
+ {"DFX_LOCAL_CPL_HEAD_TLP_NUM ", 0x1C10ull},
+ {"DFX_LOCAL_CPL_DATA_TLP_NUM ", 0x1C14ull},
+ {"DFX_LOCAL_CPL_EXT_DATA_TLP_NUM ", 0x1C18ull},
+ {"DFX_LOCAL_P_HEAD_TLP_NUM ", 0x1C1Cull},
+ {"DFX_LOCAL_P_ACK_TLP_NUM ", 0x1C20ull},
+ {"DFX_BUF_ALOC_PORT_REQ_NUM ", 0x1C24ull},
+ {"DFX_BUF_ALOC_PORT_RESULT_NUM ", 0x1C28ull},
+ {"DFX_BUF_FAIL_SIZE_NUM ", 0x1C2Cull},
+ {"DFX_BUF_ALOC_SIZE_NUM ", 0x1C30ull},
+ {"DFX_BUF_NP_RELEASE_SIZE_NUM ", 0x1C34ull},
+ {"DFX_BUF_P_RELEASE_SIZE_NUM ", 0x1C38ull},
+ {"DFX_BUF_PORT_RELEASE_SIZE_NUM ", 0x1C3Cull},
+ {"DFX_DMA_PREBUF_MEM0_ECC_ERR_ADDR ", 0x1CA8ull},
+ {"DFX_DMA_PREBUF_MEM0_ECC_CNT ", 0x1CACull},
+ {"DFX_DMA_LOC_NP_OSTB_ECC_ERR_ADDR ", 0x1CB0ull},
+ {"DFX_DMA_LOC_NP_OSTB_ECC_CNT ", 0x1CB4ull},
+ {"DFX_DMA_PREBUF_MEM1_ECC_ERR_ADDR ", 0x1CC0ull},
+ {"DFX_DMA_PREBUF_MEM1_ECC_CNT ", 0x1CC4ull},
+ {"DMA_CH_DONE_STS ", 0x02E0ull},
+ {"DMA_CH_ERR_STS ", 0x0320ull},
+};
+#endif /* CONFIG_DEBUG_FS*/
+
static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
{
if (pdev->revision == HISI_DMA_REVISION_HIP08B)
@@ -717,6 +836,117 @@ static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
INIT_LIST_HEAD(&dma_dev->channels);
}
+/* --- debugfs implementation --- */
+#ifdef CONFIG_DEBUG_FS
+#include <linux/debugfs.h>
+static struct debugfs_reg32 *hisi_dma_get_ch_regs(struct hisi_dma_dev *hdma_dev,
+ u32 *regs_sz)
+{
+ struct device *dev = &hdma_dev->pdev->dev;
+ struct debugfs_reg32 *regs;
+ u32 regs_sz_comm;
+
+ regs_sz_comm = ARRAY_SIZE(hisi_dma_comm_chan_regs);
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip08_chan_regs);
+ else
+ *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip09_chan_regs);
+
+ regs = devm_kcalloc(dev, *regs_sz, sizeof(struct debugfs_reg32),
+ GFP_KERNEL);
+ if (!regs)
+ return NULL;
+ memcpy(regs, hisi_dma_comm_chan_regs, sizeof(hisi_dma_comm_chan_regs));
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ memcpy(regs + regs_sz_comm, hisi_dma_hip08_chan_regs,
+ sizeof(hisi_dma_hip08_chan_regs));
+ else
+ memcpy(regs + regs_sz_comm, hisi_dma_hip09_chan_regs,
+ sizeof(hisi_dma_hip09_chan_regs));
+
+ return regs;
+}
+
+static int hisi_dma_create_chan_dir(struct hisi_dma_dev *hdma_dev)
+{
+ char dir_name[HISI_DMA_MAX_DIR_NAME_LEN];
+ struct debugfs_regset32 *regsets;
+ struct debugfs_reg32 *regs;
+ struct dentry *chan_dir;
+ struct device *dev;
+ u32 regs_sz;
+ int ret;
+ int i;
+
+ dev = &hdma_dev->pdev->dev;
+
+ regsets = devm_kcalloc(dev, hdma_dev->chan_num,
+ sizeof(*regsets), GFP_KERNEL);
+ if (!regsets)
+ return -ENOMEM;
+
+ regs = hisi_dma_get_ch_regs(hdma_dev, ®s_sz);
+ if (!regs)
+ return -ENOMEM;
+
+ for (i = 0; i < hdma_dev->chan_num; i++) {
+ regsets[i].regs = regs;
+ regsets[i].nregs = regs_sz;
+ regsets[i].base = hdma_dev->queue_base + i * HISI_DMA_Q_OFFSET;
+ regsets[i].dev = dev;
+
+ memset(dir_name, 0, HISI_DMA_MAX_DIR_NAME_LEN);
+ ret = sprintf(dir_name, "channel%d", i);
+ if (ret < 0)
+ return ret;
+
+ chan_dir = debugfs_create_dir(dir_name,
+ hdma_dev->dma_dev.dbg_dev_root);
+ debugfs_create_regset32("regs", 0444, chan_dir, ®sets[i]);
+ }
+
+ return 0;
+}
+
+static void hisi_dma_create_debugfs(struct hisi_dma_dev *hdma_dev)
+{
+ struct debugfs_regset32 *regset;
+ struct device *dev;
+ int ret;
+
+ dev = &hdma_dev->pdev->dev;
+
+ if (hdma_dev->dma_dev.dbg_dev_root == NULL)
+ return;
+
+ regset = devm_kzalloc(dev, sizeof(*regset), GFP_KERNEL);
+ if (!regset)
+ return;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ regset->regs = hisi_dma_hip08_comm_regs;
+ regset->nregs = ARRAY_SIZE(hisi_dma_hip08_comm_regs);
+ } else {
+ regset->regs = hisi_dma_hip09_comm_regs;
+ regset->nregs = ARRAY_SIZE(hisi_dma_hip09_comm_regs);
+ }
+ regset->base = hdma_dev->base;
+ regset->dev = dev;
+
+ debugfs_create_regset32("regs", 0444,
+ hdma_dev->dma_dev.dbg_dev_root, regset);
+
+ ret = hisi_dma_create_chan_dir(hdma_dev);
+ if (ret < 0)
+ dev_info(&hdma_dev->pdev->dev, "fail to create debugfs for channels!\n");
+}
+#else
+static void hisi_dma_create_debugfs(struct hisi_dma_dev *hdma_dev) { }
+#endif /* CONFIG_DEBUG_FS*/
+/* --- debugfs implementation --- */
+
static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
enum hisi_dma_reg_layout reg_layout;
@@ -793,10 +1023,14 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
dma_dev = &hdma_dev->dma_dev;
ret = dmaenginem_async_device_register(dma_dev);
- if (ret < 0)
+ if (ret < 0) {
dev_err(dev, "failed to register device!\n");
+ return ret;
+ }
+
+ hisi_dma_create_debugfs(hdma_dev);
- return ret;
+ return 0;
}
static const struct pci_device_id hisi_dma_pci_tbl[] = {
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v4 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma
2022-08-02 10:12 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (5 preceding siblings ...)
2022-08-02 10:12 ` [PATCH v4 6/7] dmaengine: hisilicon: Dump regs to debugfs Jie Hai
@ 2022-08-02 10:12 ` Jie Hai
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-02 10:12 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
Add myself as a maintainer for hisi_dma.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
MAINTAINERS | 1 +
1 file changed, 1 insertion(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 64379c699903..b1aeffd74a37 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8971,6 +8971,7 @@ F: net/dsa/tag_hellcreek.c
HISILICON DMA DRIVER
M: Zhou Wang <wangzhou1@hisilicon.com>
+M: Jie Hai <haijie1@hisilicon.com>
L: dmaengine@vger.kernel.org
S: Maintained
F: drivers/dma/hisi_dma.c
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* Re: [PATCH v4 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
2022-08-02 10:12 ` [PATCH v4 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
@ 2022-08-05 7:41 ` Jie Hai
0 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-05 7:41 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: liudongdong3, dmaengine, linux-kernel
On 2022/8/2 18:12, Jie Hai wrote:
> The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they
> have the same pci device id but different pci revision.
> Unfortunately, they have different register layouts, so
> the origin driver cannot run on HiSilicon IP09 correctly.
>
> This patch enables the driver to adapt to HiSilicon IP09.
> HiSilicon IP09 offers 4 channels, each channel has a send
> queue, a complete queue and an interrupt to help to do tasks.
> This DMA engine can do memory copy between memory blocks.
>
> Signed-off-by: Jie Hai <haijie1@huawei.com>
> Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
> ---
> drivers/dma/hisi_dma.c | 379 ++++++++++++++++++++++++++++++++---------
> 1 file changed, 298 insertions(+), 81 deletions(-)
>
> diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
> index 5d62fe62ba00..5fa3b6fa0529 100644
> --- a/drivers/dma/hisi_dma.c
> +++ b/drivers/dma/hisi_dma.c
> @@ -1,5 +1,6 @@
> // SPDX-License-Identifier: GPL-2.0-only
> -/* Copyright(c) 2019 HiSilicon Limited. */
> +/* Copyright(c) 2019-2022 HiSilicon Limited. */
> +
> #include <linux/bitfield.h>
> #include <linux/dmaengine.h>
> #include <linux/init.h>
> @@ -9,35 +10,85 @@
> #include <linux/spinlock.h>
> #include "virt-dma.h"
>
> -#define HISI_DMA_SQ_BASE_L 0x0
> -#define HISI_DMA_SQ_BASE_H 0x4
> -#define HISI_DMA_SQ_DEPTH 0x8
> -#define HISI_DMA_SQ_TAIL_PTR 0xc
> -#define HISI_DMA_CQ_BASE_L 0x10
> -#define HISI_DMA_CQ_BASE_H 0x14
> -#define HISI_DMA_CQ_DEPTH 0x18
> -#define HISI_DMA_CQ_HEAD_PTR 0x1c
> -#define HISI_DMA_CTRL0 0x20
> -#define HISI_DMA_CTRL0_QUEUE_EN_S 0
> -#define HISI_DMA_CTRL0_QUEUE_PAUSE_S 4
> -#define HISI_DMA_CTRL1 0x24
> -#define HISI_DMA_CTRL1_QUEUE_RESET_S 0
> -#define HISI_DMA_Q_FSM_STS 0x30
> -#define HISI_DMA_FSM_STS_MASK GENMASK(3, 0)
> -#define HISI_DMA_INT_STS 0x40
> -#define HISI_DMA_INT_STS_MASK GENMASK(12, 0)
> -#define HISI_DMA_INT_MSK 0x44
> -#define HISI_DMA_MODE 0x217c
> -#define HISI_DMA_OFFSET 0x100
> -
> -#define HISI_DMA_MSI_NUM 32
> -#define HISI_DMA_CHAN_NUM 30
> -#define HISI_DMA_Q_DEPTH_VAL 1024
> -
> -#define PCI_BAR_2 2
> -
> -#define HISI_DMA_POLL_Q_STS_DELAY_US 10
> -#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
> +/* HiSilicon DMA register common field define */
> +#define HISI_DMA_Q_SQ_BASE_L 0x0
> +#define HISI_DMA_Q_SQ_BASE_H 0x4
> +#define HISI_DMA_Q_SQ_DEPTH 0x8
> +#define HISI_DMA_Q_SQ_TAIL_PTR 0xc
> +#define HISI_DMA_Q_CQ_BASE_L 0x10
> +#define HISI_DMA_Q_CQ_BASE_H 0x14
> +#define HISI_DMA_Q_CQ_DEPTH 0x18
> +#define HISI_DMA_Q_CQ_HEAD_PTR 0x1c
> +#define HISI_DMA_Q_CTRL0 0x20
> +#define HISI_DMA_Q_CTRL0_QUEUE_EN BIT(0)
> +#define HISI_DMA_Q_CTRL0_QUEUE_PAUSE BIT(4)
> +#define HISI_DMA_Q_CTRL1 0x24
> +#define HISI_DMA_Q_CTRL1_QUEUE_RESET BIT(0)
> +#define HISI_DMA_Q_FSM_STS 0x30
> +#define HISI_DMA_Q_FSM_STS_MASK GENMASK(3, 0)
> +#define HISI_DMA_Q_ERR_INT_NUM0 0x84
> +#define HISI_DMA_Q_ERR_INT_NUM1 0x88
> +#define HISI_DMA_Q_ERR_INT_NUM2 0x8c
> +
> +/* HiSilicon IP08 DMA register and field define */
> +#define HISI_DMA_HIP08_MODE 0x217C
> +#define HISI_DMA_HIP08_Q_BASE 0x0
> +#define HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN BIT(2)
> +#define HISI_DMA_HIP08_Q_INT_STS 0x40
> +#define HISI_DMA_HIP08_Q_INT_MSK 0x44
> +#define HISI_DMA_HIP08_Q_INT_STS_MASK GENMASK(14, 0)
> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM3 0x90
> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM4 0x94
> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM5 0x98
> +#define HISI_DMA_HIP08_Q_ERR_INT_NUM6 0x48
> +#define HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT BIT(24)
> +
> +/* HiSilicon IP09 DMA register and field define */
> +#define HISI_DMA_HIP09_DMA_FLR_DISABLE 0xA00
> +#define HISI_DMA_HIP09_DMA_FLR_DISABLE_B BIT(0)
> +#define HISI_DMA_HIP09_Q_BASE 0x2000
> +#define HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN GENMASK(31, 28)
> +#define HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT BIT(26)
> +#define HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT BIT(27)
> +#define HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE BIT(2)
> +#define HISI_DMA_HIP09_Q_INT_STS 0x40
> +#define HISI_DMA_HIP09_Q_INT_MSK 0x44
> +#define HISI_DMA_HIP09_Q_INT_STS_MASK 0x1
> +#define HISI_DMA_HIP09_Q_ERR_INT_STS 0x48
> +#define HISI_DMA_HIP09_Q_ERR_INT_MSK 0x4C
> +#define HISI_DMA_HIP09_Q_ERR_INT_STS_MASK GENMASK(18, 1)
> +#define HISI_DMA_HIP09_PORT_CFG_REG(port_id) (0x800 + \
> + (port_id) * 0x20)
> +#define HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B BIT(16)
> +
> +#define HISI_DMA_HIP09_MAX_PORT_NUM 16
> +
> +#define HISI_DMA_HIP08_MSI_NUM 32
> +#define HISI_DMA_HIP08_CHAN_NUM 30
> +#define HISI_DMA_HIP09_MSI_NUM 4
> +#define HISI_DMA_HIP09_CHAN_NUM 4
> +#define HISI_DMA_REVISION_HIP08B 0x21
> +#define HISI_DMA_REVISION_HIP09A 0x30
> +
> +#define HISI_DMA_Q_OFFSET 0x100
> +#define HISI_DMA_Q_DEPTH_VAL 1024
> +
> +#define PCI_BAR_2 2
> +
> +#define HISI_DMA_POLL_Q_STS_DELAY_US 10
> +#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
> +
> +/*
> + * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
> + * have the same pci device id but different pci revision.
> + * Unfortunately, they have different register layouts, so two layout
> + * enumerations are defined.
> + */
> +enum hisi_dma_reg_layout {
> + HISI_DMA_REG_LAYOUT_INVALID = 0,
> + HISI_DMA_REG_LAYOUT_HIP08,
> + HISI_DMA_REG_LAYOUT_HIP09
> +};
>
> enum hisi_dma_mode {
> EP = 0,
> @@ -108,9 +159,45 @@ struct hisi_dma_dev {
> struct dma_device dma_dev;
> u32 chan_num;
> u32 chan_depth;
> + enum hisi_dma_reg_layout reg_layout;
> + void __iomem *queue_base; /* queue region start of register */
> struct hisi_dma_chan chan[];
> };
>
> +static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
> +{
> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
> + return HISI_DMA_REG_LAYOUT_HIP08;
> + else if (pdev->revision >= HISI_DMA_REVISION_HIP09A)
> + return HISI_DMA_REG_LAYOUT_HIP09;
> +
> + return HISI_DMA_REG_LAYOUT_INVALID;
> +}
> +
> +static u32 hisi_dma_get_chan_num(struct pci_dev *pdev)
> +{
> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
> + return HISI_DMA_HIP08_CHAN_NUM;
> +
> + return HISI_DMA_HIP09_CHAN_NUM;
> +}
> +
> +static u32 hisi_dma_get_msi_num(struct pci_dev *pdev)
> +{
> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
> + return HISI_DMA_HIP08_MSI_NUM;
> +
> + return HISI_DMA_HIP09_MSI_NUM;
> +}
> +
> +static u32 hisi_dma_get_queue_base(struct pci_dev *pdev)
> +{
> + if (pdev->revision == HISI_DMA_REVISION_HIP08B)
> + return HISI_DMA_HIP08_Q_BASE;
> +
> + return HISI_DMA_HIP09_Q_BASE;
> +}
> +
> static inline struct hisi_dma_chan *to_hisi_dma_chan(struct dma_chan *c)
> {
> return container_of(c, struct hisi_dma_chan, vc.chan);
> @@ -124,7 +211,7 @@ static inline struct hisi_dma_desc *to_hisi_dma_desc(struct virt_dma_desc *vd)
> static inline void hisi_dma_chan_write(void __iomem *base, u32 reg, u32 index,
> u32 val)
> {
> - writel_relaxed(val, base + reg + index * HISI_DMA_OFFSET);
> + writel_relaxed(val, base + reg + index * HISI_DMA_Q_OFFSET);
> }
>
> static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
> @@ -132,55 +219,83 @@ static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
> u32 tmp;
>
> tmp = readl_relaxed(addr);
> - tmp = val ? tmp | BIT(pos) : tmp & ~BIT(pos);
> + tmp = val ? tmp | pos : tmp & ~pos;
> writel_relaxed(tmp, addr);
> }
>
> static void hisi_dma_pause_dma(struct hisi_dma_dev *hdma_dev, u32 index,
> bool pause)
> {
> - void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
> - HISI_DMA_OFFSET;
> + void __iomem *addr;
>
> - hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_PAUSE_S, pause);
> + addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
> + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_PAUSE, pause);
> }
>
> static void hisi_dma_enable_dma(struct hisi_dma_dev *hdma_dev, u32 index,
> bool enable)
> {
> - void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
> - HISI_DMA_OFFSET;
> + void __iomem *addr;
>
> - hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_EN_S, enable);
> + addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
> + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_EN, enable);
> }
>
> static void hisi_dma_mask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
> {
> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_INT_MSK, qp_index,
> - HISI_DMA_INT_STS_MASK);
> + void __iomem *q_base = hdma_dev->queue_base;
> +
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
> + qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
> + else {
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
> + qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
> + qp_index,
> + HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
> + }
> }
>
> static void hisi_dma_unmask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
> {
> - void __iomem *base = hdma_dev->base;
> -
> - hisi_dma_chan_write(base, HISI_DMA_INT_STS, qp_index,
> - HISI_DMA_INT_STS_MASK);
> - hisi_dma_chan_write(base, HISI_DMA_INT_MSK, qp_index, 0);
> + void __iomem *q_base = hdma_dev->queue_base;
> +
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_STS,
> + qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
> + qp_index, 0);
> + } else {
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_STS,
> + qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_STS,
> + qp_index,
> + HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
> + qp_index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
> + qp_index, 0);
> + }
> }
>
> static void hisi_dma_do_reset(struct hisi_dma_dev *hdma_dev, u32 index)
> {
> - void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL1 + index *
> - HISI_DMA_OFFSET;
> + void __iomem *addr;
>
> - hisi_dma_update_bit(addr, HISI_DMA_CTRL1_QUEUE_RESET_S, 1);
> + addr = hdma_dev->queue_base +
> + HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL1_QUEUE_RESET, 1);
> }
>
> static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
> {
> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, index, 0);
> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
> + void __iomem *q_base = hdma_dev->queue_base;
> +
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
> }
>
> static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
> @@ -196,10 +311,10 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
> hisi_dma_mask_irq(hdma_dev, index);
>
> addr = hdma_dev->base +
> - HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET;
> + HISI_DMA_Q_FSM_STS + index * HISI_DMA_Q_OFFSET;
>
This is a channel related register and the base address should be
hdma_dev->queue_base. This problem was introduced in v3 and will be
fixed in next version.
> ret = readl_relaxed_poll_timeout(addr, tmp,
> - FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN,
> + FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) != RUN,
> HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
> if (ret) {
> dev_err(&hdma_dev->pdev->dev, "disable channel timeout!\n");
> @@ -216,7 +331,7 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
> }
>
> ret = readl_relaxed_poll_timeout(addr, tmp,
> - FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE,
> + FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) == IDLE,
> HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
> if (ret) {
> dev_err(&hdma_dev->pdev->dev, "reset channel timeout!\n");
> @@ -298,8 +413,8 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
> chan->sq_tail = (chan->sq_tail + 1) % hdma_dev->chan_depth;
>
> /* update sq_tail to trigger a new task */
> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, chan->qp_num,
> - chan->sq_tail);
> + hisi_dma_chan_write(hdma_dev->queue_base, HISI_DMA_Q_SQ_TAIL_PTR,
> + chan->qp_num, chan->sq_tail);
> }
>
> static void hisi_dma_issue_pending(struct dma_chan *c)
> @@ -373,26 +488,86 @@ static int hisi_dma_alloc_qps_mem(struct hisi_dma_dev *hdma_dev)
> static void hisi_dma_init_hw_qp(struct hisi_dma_dev *hdma_dev, u32 index)
> {
> struct hisi_dma_chan *chan = &hdma_dev->chan[index];
> + void __iomem *q_base = hdma_dev->queue_base;
> u32 hw_depth = hdma_dev->chan_depth - 1;
> - void __iomem *base = hdma_dev->base;
> + void __iomem *addr;
> + u32 tmp;
>
> /* set sq, cq base */
> - hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_L, index,
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_L, index,
> lower_32_bits(chan->sq_dma));
> - hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_H, index,
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_H, index,
> upper_32_bits(chan->sq_dma));
> - hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_L, index,
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_L, index,
> lower_32_bits(chan->cq_dma));
> - hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_H, index,
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_H, index,
> upper_32_bits(chan->cq_dma));
>
> /* set sq, cq depth */
> - hisi_dma_chan_write(base, HISI_DMA_SQ_DEPTH, index, hw_depth);
> - hisi_dma_chan_write(base, HISI_DMA_CQ_DEPTH, index, hw_depth);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_DEPTH, index, hw_depth);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_DEPTH, index, hw_depth);
>
> /* init sq tail and cq head */
> - hisi_dma_chan_write(base, HISI_DMA_SQ_TAIL_PTR, index, 0);
> - hisi_dma_chan_write(base, HISI_DMA_CQ_HEAD_PTR, index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
> +
> + /* init error interrupt stats */
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM0, index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM1, index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM2, index, 0);
> +
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM3,
> + index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM4,
> + index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM5,
> + index, 0);
> + hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM6,
> + index, 0);
> + /*
> + * init SQ/CQ direction selecting register.
> + * "0" is to local side and "1" is to remote side.
> + */
> + addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT, 0);
> +
> + /*
> + * 0 - Continue to next descriptor if error occurs.
> + * 1 - Abort the DMA queue if error occurs.
> + */
> + hisi_dma_update_bit(addr,
> + HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN, 0);
> + } else {
> + addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
> +
> + /*
> + * init SQ/CQ direction selecting register.
> + * "0" is to local side and "1" is to remote side.
> + */
> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT, 0);
> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT, 0);
> +
> + /*
> + * 0 - Continue to next descriptor if error occurs.
> + * 1 - Abort the DMA queue if error occurs.
> + */
> +
> + tmp = readl_relaxed(addr);
> + tmp &= ~HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN;
> + writel_relaxed(tmp, addr);
> +
> + /*
> + * 0 - dma should process FLR whith CPU.
> + * 1 - dma not process FLR, only cpu process FLR.
> + */
> + addr = q_base + HISI_DMA_HIP09_DMA_FLR_DISABLE +
> + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_DMA_FLR_DISABLE_B, 0);
> +
> + addr = q_base + HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
> + hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE, 1);
> + }
> }
>
> static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
> @@ -436,14 +611,16 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
> struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
> struct hisi_dma_desc *desc;
> struct hisi_dma_cqe *cqe;
> + void __iomem *q_base;
>
> spin_lock(&chan->vc.lock);
>
> desc = chan->desc;
> cqe = chan->cq + chan->cq_head;
> + q_base = hdma_dev->queue_base;
> if (desc) {
> chan->cq_head = (chan->cq_head + 1) % hdma_dev->chan_depth;
> - hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR,
> + hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
> chan->qp_num, chan->cq_head);
> if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
> vchan_cookie_complete(&desc->vd);
> @@ -504,16 +681,58 @@ static void hisi_dma_disable_hw_channels(void *data)
> static void hisi_dma_set_mode(struct hisi_dma_dev *hdma_dev,
> enum hisi_dma_mode mode)
> {
> - writel_relaxed(mode == RC ? 1 : 0, hdma_dev->base + HISI_DMA_MODE);
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
> + writel_relaxed(mode == RC ? 1 : 0,
> + hdma_dev->base + HISI_DMA_HIP08_MODE);
> +}
> +
> +static void hisi_dma_init_hw(struct hisi_dma_dev *hdma_dev)
> +{
> + void __iomem *addr;
> + int i;
> +
> + if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP09) {
> + for (i = 0; i < HISI_DMA_HIP09_MAX_PORT_NUM; i++) {
> + addr = hdma_dev->base + HISI_DMA_HIP09_PORT_CFG_REG(i);
> + hisi_dma_update_bit(addr,
> + HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B, 1);
> + }
> + }
> +}
> +
> +static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
> +{
> + struct dma_device *dma_dev;
> +
> + dma_dev = &hdma_dev->dma_dev;
> + dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
> + dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
> + dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
> + dma_dev->device_tx_status = hisi_dma_tx_status;
> + dma_dev->device_issue_pending = hisi_dma_issue_pending;
> + dma_dev->device_terminate_all = hisi_dma_terminate_all;
> + dma_dev->device_synchronize = hisi_dma_synchronize;
> + dma_dev->directions = BIT(DMA_MEM_TO_MEM);
> + dma_dev->dev = &hdma_dev->pdev->dev;
> + INIT_LIST_HEAD(&dma_dev->channels);
> }
>
> static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> {
> + enum hisi_dma_reg_layout reg_layout;
> struct device *dev = &pdev->dev;
> struct hisi_dma_dev *hdma_dev;
> struct dma_device *dma_dev;
> + u32 chan_num;
> + u32 msi_num;
> int ret;
>
> + reg_layout = hisi_dma_get_reg_layout(pdev);
> + if (reg_layout == HISI_DMA_REG_LAYOUT_INVALID) {
> + dev_err(dev, "unsupported device!\n");
> + return -EINVAL;
> + }
> +
> ret = pcim_enable_device(pdev);
> if (ret) {
> dev_err(dev, "failed to enable device mem!\n");
> @@ -530,40 +749,37 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> if (ret)
> return ret;
>
> - hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, HISI_DMA_CHAN_NUM), GFP_KERNEL);
> + chan_num = hisi_dma_get_chan_num(pdev);
> + hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, chan_num),
> + GFP_KERNEL);
> if (!hdma_dev)
> return -EINVAL;
>
> hdma_dev->base = pcim_iomap_table(pdev)[PCI_BAR_2];
> hdma_dev->pdev = pdev;
> - hdma_dev->chan_num = HISI_DMA_CHAN_NUM;
> hdma_dev->chan_depth = HISI_DMA_Q_DEPTH_VAL;
> + hdma_dev->chan_num = chan_num;
> + hdma_dev->reg_layout = reg_layout;
> + hdma_dev->queue_base = hdma_dev->base + hisi_dma_get_queue_base(pdev);
>
> pci_set_drvdata(pdev, hdma_dev);
> pci_set_master(pdev);
>
> + msi_num = hisi_dma_get_msi_num(pdev);
> +
> /* This will be freed by 'pcim_release()'. See 'pcim_enable_device()' */
> - ret = pci_alloc_irq_vectors(pdev, HISI_DMA_MSI_NUM, HISI_DMA_MSI_NUM,
> - PCI_IRQ_MSI);
> + ret = pci_alloc_irq_vectors(pdev, msi_num, msi_num, PCI_IRQ_MSI);
> if (ret < 0) {
> dev_err(dev, "Failed to allocate MSI vectors!\n");
> return ret;
> }
>
> - dma_dev = &hdma_dev->dma_dev;
> - dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
> - dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
> - dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
> - dma_dev->device_tx_status = hisi_dma_tx_status;
> - dma_dev->device_issue_pending = hisi_dma_issue_pending;
> - dma_dev->device_terminate_all = hisi_dma_terminate_all;
> - dma_dev->device_synchronize = hisi_dma_synchronize;
> - dma_dev->directions = BIT(DMA_MEM_TO_MEM);
> - dma_dev->dev = dev;
> - INIT_LIST_HEAD(&dma_dev->channels);
> + hisi_dma_init_dma_dev(hdma_dev);
>
> hisi_dma_set_mode(hdma_dev, RC);
>
> + hisi_dma_init_hw(hdma_dev);
> +
> ret = hisi_dma_enable_hw_channels(hdma_dev);
> if (ret < 0) {
> dev_err(dev, "failed to enable hw channel!\n");
> @@ -575,6 +791,7 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
> if (ret)
> return ret;
>
> + dma_dev = &hdma_dev->dma_dev;
> ret = dmaenginem_async_device_register(dma_dev);
> if (ret < 0)
> dev_err(dev, "failed to register device!\n");
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (10 preceding siblings ...)
2022-08-02 10:12 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
@ 2022-08-05 8:25 ` Jie Hai
2022-08-05 8:25 ` [PATCH v5 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
` (6 more replies)
11 siblings, 7 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-05 8:25 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they share the
same pci device id but different pci revision and register layouts.
The original version supports HiSilicon IP08 but not HiSilicon IP09.
This series support DMA driver for HIP08 and HIP09:
1. Fix bugs for HIP08 DMA driver
- Disable hardware channels when driver detached
- Update cq_head whenever accessed it
- Support multi-thread for one DMA channel
2. Use macros instead of magic number
3. Add support for HIP09 DMA driver
4. Add debugfs for HIP08 and HIP09 DMA driver
5. Add myself as maintainer of hisi_dma.c
Changes since version 4:
- Fix hdma_dev->base to hdma_dev->queue_base in hisi_dma_reset_or_disable_hw_chan
Changes since version 3:
- remove reduldant braces
- add "Acked-by: Zhou Wang <wangzhou1@hisilicon.com>" in commit log
Changes since version 2:
- fix unnecessary line breaks
- fix register bit with BIT/GENMASK and adjust hisi_dma_update_bit to it
- remove "Reported-by" in commit message
- use dmaengine root instead of hisi_dma root
- ignore errors for creating debugfs
Changes since version 1:
- remove error changes casuse compile failure
- remove reduldant "*" in comment
- remove debugfs-hisi-dma doc and path in MAINTAINERS
Jie Hai (7):
dmaengine: hisilicon: Disable channels when unregister hisi_dma
dmaengine: hisilicon: Fix CQ head update
dmaengine: hisilicon: Add multi-thread support for a DMA channel
dmaengine: hisilicon: Use macros instead of magic number
dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
dmaengine: hisilicon: Dump regs to debugfs
MAINTAINERS: Add myself as maintainer for hisi_dma
MAINTAINERS | 1 +
drivers/dma/hisi_dma.c | 650 +++++++++++++++++++++++++++++++++++------
2 files changed, 555 insertions(+), 96 deletions(-)
--
2.33.0
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v5 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma
2022-08-05 8:25 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
@ 2022-08-05 8:25 ` Jie Hai
2022-08-05 8:25 ` [PATCH v5 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
` (5 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-05 8:25 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
When hisi_dma is unloaded or unbinded, all of channels should be
disabled. This patch disables DMA channels when driver is unloaded
or unbinded.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 43817ced3a3e..98bc488893cc 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -180,7 +180,8 @@ static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
}
-static void hisi_dma_reset_hw_chan(struct hisi_dma_chan *chan)
+static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
+ bool disable)
{
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
u32 index = chan->qp_num, tmp;
@@ -201,8 +202,11 @@ static void hisi_dma_reset_hw_chan(struct hisi_dma_chan *chan)
hisi_dma_do_reset(hdma_dev, index);
hisi_dma_reset_qp_point(hdma_dev, index);
hisi_dma_pause_dma(hdma_dev, index, false);
- hisi_dma_enable_dma(hdma_dev, index, true);
- hisi_dma_unmask_irq(hdma_dev, index);
+
+ if (!disable) {
+ hisi_dma_enable_dma(hdma_dev, index, true);
+ hisi_dma_unmask_irq(hdma_dev, index);
+ }
ret = readl_relaxed_poll_timeout(hdma_dev->base +
HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
@@ -218,7 +222,7 @@ static void hisi_dma_free_chan_resources(struct dma_chan *c)
struct hisi_dma_chan *chan = to_hisi_dma_chan(c);
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
- hisi_dma_reset_hw_chan(chan);
+ hisi_dma_reset_or_disable_hw_chan(chan, false);
vchan_free_chan_resources(&chan->vc);
memset(chan->sq, 0, sizeof(struct hisi_dma_sqe) * hdma_dev->chan_depth);
@@ -394,7 +398,7 @@ static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
static void hisi_dma_disable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- hisi_dma_reset_hw_chan(&hdma_dev->chan[qp_index]);
+ hisi_dma_reset_or_disable_hw_chan(&hdma_dev->chan[qp_index], true);
}
static void hisi_dma_enable_qps(struct hisi_dma_dev *hdma_dev)
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v5 2/7] dmaengine: hisilicon: Fix CQ head update
2022-08-05 8:25 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-08-05 8:25 ` [PATCH v5 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
@ 2022-08-05 8:25 ` Jie Hai
2022-08-05 8:25 ` [PATCH v5 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
` (4 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-05 8:25 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
After completion of data transfer of one or multiple descriptors,
the completion status and the current head pointer to submission
queue are written into the CQ and interrupt can be generated to
inform the software. In interrupt process CQ is read and cq_head
is updated.
hisi_dma_irq updates cq_head only when the completion status is
success. When an abnormal interrupt reports, cq_head will not update
which will cause subsequent interrupt processes read the error CQ
and never report the correct status.
This patch updates cq_head whenever CQ is accessed.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 98bc488893cc..837f7e4adfa6 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -436,12 +436,10 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
desc = chan->desc;
cqe = chan->cq + chan->cq_head;
if (desc) {
+ chan->cq_head = (chan->cq_head + 1) % hdma_dev->chan_depth;
+ hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR,
+ chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
- chan->cq_head = (chan->cq_head + 1) %
- hdma_dev->chan_depth;
- hisi_dma_chan_write(hdma_dev->base,
- HISI_DMA_CQ_HEAD_PTR, chan->qp_num,
- chan->cq_head);
vchan_cookie_complete(&desc->vd);
} else {
dev_err(&hdma_dev->pdev->dev, "task error!\n");
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v5 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel
2022-08-05 8:25 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-08-05 8:25 ` [PATCH v5 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
2022-08-05 8:25 ` [PATCH v5 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
@ 2022-08-05 8:25 ` Jie Hai
2022-08-05 8:25 ` [PATCH v5 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
` (3 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-05 8:25 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
When we get a DMA channel and try to use it in multiple threads it
will cause oops and hanging the system.
% echo 100 > /sys/module/dmatest/parameters/threads_per_chan
% echo 100 > /sys/module/dmatest/parameters/iterations
% echo 1 > /sys/module/dmatest/parameters/run
[383493.327077] Unable to handle kernel paging request at virtual
address dead000000000108
[383493.335103] Mem abort info:
[383493.335103] ESR = 0x96000044
[383493.335105] EC = 0x25: DABT (current EL), IL = 32 bits
[383493.335107] SET = 0, FnV = 0
[383493.335108] EA = 0, S1PTW = 0
[383493.335109] FSC = 0x04: level 0 translation fault
[383493.335110] Data abort info:
[383493.335111] ISV = 0, ISS = 0x00000044
[383493.364739] CM = 0, WnR = 1
[383493.367793] [dead000000000108] address between user and kernel
address ranges
[383493.375021] Internal error: Oops: 96000044 [#1] PREEMPT SMP
[383493.437574] CPU: 63 PID: 27895 Comm: dma0chan0-copy2 Kdump:
loaded Tainted: GO 5.17.0-rc4+ #2
[383493.457851] pstate: 204000c9 (nzCv daIF +PAN -UAO -TCO -DIT
-SSBS BTYPE=--)
[383493.465331] pc : vchan_tx_submit+0x64/0xa0
[383493.469957] lr : vchan_tx_submit+0x34/0xa0
This occurs because the transmission timed out, and that's due
to data race. Each thread rewrite channels's descriptor as soon as
device_issue_pending is called. It leads to the situation that
the driver thinks that it uses the right descriptor in interrupt
handler while channels's descriptor has been changed by other
thread. The descriptor which in fact reported interrupt will not
be handled any more, as well as its tx->callback.
That's why timeout reports.
With current fixes channels' descriptor changes it's value only
when it has been used. A new descriptor is acquired from
vc->desc_issued queue that is already filled with descriptors
that are ready to be sent. Threads have no direct access to DMA
channel descriptor. In case of channel's descriptor is busy, try
to submit to HW again when a descriptor is completed. In this case,
vc->desc_issued may be empty when hisi_dma_start_transfer is called,
so delete error reporting on this. Now it is just possible to queue
a descriptor for further processing.
Fixes: e9f08b65250d ("dmaengine: hisilicon: Add Kunpeng DMA engine support")
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 837f7e4adfa6..0233b42143c7 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -271,7 +271,6 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
vd = vchan_next_desc(&chan->vc);
if (!vd) {
- dev_err(&hdma_dev->pdev->dev, "no issued task!\n");
chan->desc = NULL;
return;
}
@@ -303,7 +302,7 @@ static void hisi_dma_issue_pending(struct dma_chan *c)
spin_lock_irqsave(&chan->vc.lock, flags);
- if (vchan_issue_pending(&chan->vc))
+ if (vchan_issue_pending(&chan->vc) && !chan->desc)
hisi_dma_start_transfer(chan);
spin_unlock_irqrestore(&chan->vc.lock, flags);
@@ -441,11 +440,10 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
vchan_cookie_complete(&desc->vd);
+ hisi_dma_start_transfer(chan);
} else {
dev_err(&hdma_dev->pdev->dev, "task error!\n");
}
-
- chan->desc = NULL;
}
spin_unlock(&chan->vc.lock);
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v5 4/7] dmaengine: hisilicon: Use macros instead of magic number
2022-08-05 8:25 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (2 preceding siblings ...)
2022-08-05 8:25 ` [PATCH v5 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
@ 2022-08-05 8:25 ` Jie Hai
2022-08-05 8:25 ` [PATCH v5 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
` (2 subsequent siblings)
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-05 8:25 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
readl_relaxed_poll_timeout() uses magic numbers 10 and 1000, which
indicate maximum time to sleep between reads in us and timeout in us,
respectively.
Use macros HISI_DMA_POLL_Q_STS_DELAY_US and
HISI_DMA_POLL_Q_STS_TIME_OUT_US instead of these two numbers.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 19 +++++++++++++------
1 file changed, 13 insertions(+), 6 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 0233b42143c7..5d62fe62ba00 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -36,6 +36,9 @@
#define PCI_BAR_2 2
+#define HISI_DMA_POLL_Q_STS_DELAY_US 10
+#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+
enum hisi_dma_mode {
EP = 0,
RC,
@@ -185,15 +188,19 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
{
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
u32 index = chan->qp_num, tmp;
+ void __iomem *addr;
int ret;
hisi_dma_pause_dma(hdma_dev, index, true);
hisi_dma_enable_dma(hdma_dev, index, false);
hisi_dma_mask_irq(hdma_dev, index);
- ret = readl_relaxed_poll_timeout(hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN, 10, 1000);
+ addr = hdma_dev->base +
+ HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET;
+
+ ret = readl_relaxed_poll_timeout(addr, tmp,
+ FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN,
+ HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "disable channel timeout!\n");
WARN_ON(1);
@@ -208,9 +215,9 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
hisi_dma_unmask_irq(hdma_dev, index);
}
- ret = readl_relaxed_poll_timeout(hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE, 10, 1000);
+ ret = readl_relaxed_poll_timeout(addr, tmp,
+ FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE,
+ HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "reset channel timeout!\n");
WARN_ON(1);
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v5 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09
2022-08-05 8:25 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (3 preceding siblings ...)
2022-08-05 8:25 ` [PATCH v5 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
@ 2022-08-05 8:25 ` Jie Hai
2022-08-05 8:25 ` [PATCH v5 6/7] dmaengine: hisilicon: Dump regs to debugfs Jie Hai
2022-08-05 8:25 ` [PATCH v5 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-05 8:25 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
The HiSilicon IP08 and HiSilicon IP09 are DMA iEPs, they
have the same pci device id but different pci revision.
Unfortunately, they have different register layouts, so
the origin driver cannot run on HiSilicon IP09 correctly.
This patch enables the driver to adapt to HiSilicon IP09.
HiSilicon IP09 offers 4 channels, each channel has a send
queue, a complete queue and an interrupt to help to do tasks.
This DMA engine can do memory copy between memory blocks.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 381 ++++++++++++++++++++++++++++++++---------
1 file changed, 299 insertions(+), 82 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index 5d62fe62ba00..da5e49ee95fa 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
-/* Copyright(c) 2019 HiSilicon Limited. */
+/* Copyright(c) 2019-2022 HiSilicon Limited. */
+
#include <linux/bitfield.h>
#include <linux/dmaengine.h>
#include <linux/init.h>
@@ -9,35 +10,85 @@
#include <linux/spinlock.h>
#include "virt-dma.h"
-#define HISI_DMA_SQ_BASE_L 0x0
-#define HISI_DMA_SQ_BASE_H 0x4
-#define HISI_DMA_SQ_DEPTH 0x8
-#define HISI_DMA_SQ_TAIL_PTR 0xc
-#define HISI_DMA_CQ_BASE_L 0x10
-#define HISI_DMA_CQ_BASE_H 0x14
-#define HISI_DMA_CQ_DEPTH 0x18
-#define HISI_DMA_CQ_HEAD_PTR 0x1c
-#define HISI_DMA_CTRL0 0x20
-#define HISI_DMA_CTRL0_QUEUE_EN_S 0
-#define HISI_DMA_CTRL0_QUEUE_PAUSE_S 4
-#define HISI_DMA_CTRL1 0x24
-#define HISI_DMA_CTRL1_QUEUE_RESET_S 0
-#define HISI_DMA_Q_FSM_STS 0x30
-#define HISI_DMA_FSM_STS_MASK GENMASK(3, 0)
-#define HISI_DMA_INT_STS 0x40
-#define HISI_DMA_INT_STS_MASK GENMASK(12, 0)
-#define HISI_DMA_INT_MSK 0x44
-#define HISI_DMA_MODE 0x217c
-#define HISI_DMA_OFFSET 0x100
-
-#define HISI_DMA_MSI_NUM 32
-#define HISI_DMA_CHAN_NUM 30
-#define HISI_DMA_Q_DEPTH_VAL 1024
-
-#define PCI_BAR_2 2
-
-#define HISI_DMA_POLL_Q_STS_DELAY_US 10
-#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+/* HiSilicon DMA register common field define */
+#define HISI_DMA_Q_SQ_BASE_L 0x0
+#define HISI_DMA_Q_SQ_BASE_H 0x4
+#define HISI_DMA_Q_SQ_DEPTH 0x8
+#define HISI_DMA_Q_SQ_TAIL_PTR 0xc
+#define HISI_DMA_Q_CQ_BASE_L 0x10
+#define HISI_DMA_Q_CQ_BASE_H 0x14
+#define HISI_DMA_Q_CQ_DEPTH 0x18
+#define HISI_DMA_Q_CQ_HEAD_PTR 0x1c
+#define HISI_DMA_Q_CTRL0 0x20
+#define HISI_DMA_Q_CTRL0_QUEUE_EN BIT(0)
+#define HISI_DMA_Q_CTRL0_QUEUE_PAUSE BIT(4)
+#define HISI_DMA_Q_CTRL1 0x24
+#define HISI_DMA_Q_CTRL1_QUEUE_RESET BIT(0)
+#define HISI_DMA_Q_FSM_STS 0x30
+#define HISI_DMA_Q_FSM_STS_MASK GENMASK(3, 0)
+#define HISI_DMA_Q_ERR_INT_NUM0 0x84
+#define HISI_DMA_Q_ERR_INT_NUM1 0x88
+#define HISI_DMA_Q_ERR_INT_NUM2 0x8c
+
+/* HiSilicon IP08 DMA register and field define */
+#define HISI_DMA_HIP08_MODE 0x217C
+#define HISI_DMA_HIP08_Q_BASE 0x0
+#define HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN BIT(2)
+#define HISI_DMA_HIP08_Q_INT_STS 0x40
+#define HISI_DMA_HIP08_Q_INT_MSK 0x44
+#define HISI_DMA_HIP08_Q_INT_STS_MASK GENMASK(14, 0)
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM3 0x90
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM4 0x94
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM5 0x98
+#define HISI_DMA_HIP08_Q_ERR_INT_NUM6 0x48
+#define HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT BIT(24)
+
+/* HiSilicon IP09 DMA register and field define */
+#define HISI_DMA_HIP09_DMA_FLR_DISABLE 0xA00
+#define HISI_DMA_HIP09_DMA_FLR_DISABLE_B BIT(0)
+#define HISI_DMA_HIP09_Q_BASE 0x2000
+#define HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN GENMASK(31, 28)
+#define HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT BIT(26)
+#define HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT BIT(27)
+#define HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE BIT(2)
+#define HISI_DMA_HIP09_Q_INT_STS 0x40
+#define HISI_DMA_HIP09_Q_INT_MSK 0x44
+#define HISI_DMA_HIP09_Q_INT_STS_MASK 0x1
+#define HISI_DMA_HIP09_Q_ERR_INT_STS 0x48
+#define HISI_DMA_HIP09_Q_ERR_INT_MSK 0x4C
+#define HISI_DMA_HIP09_Q_ERR_INT_STS_MASK GENMASK(18, 1)
+#define HISI_DMA_HIP09_PORT_CFG_REG(port_id) (0x800 + \
+ (port_id) * 0x20)
+#define HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B BIT(16)
+
+#define HISI_DMA_HIP09_MAX_PORT_NUM 16
+
+#define HISI_DMA_HIP08_MSI_NUM 32
+#define HISI_DMA_HIP08_CHAN_NUM 30
+#define HISI_DMA_HIP09_MSI_NUM 4
+#define HISI_DMA_HIP09_CHAN_NUM 4
+#define HISI_DMA_REVISION_HIP08B 0x21
+#define HISI_DMA_REVISION_HIP09A 0x30
+
+#define HISI_DMA_Q_OFFSET 0x100
+#define HISI_DMA_Q_DEPTH_VAL 1024
+
+#define PCI_BAR_2 2
+
+#define HISI_DMA_POLL_Q_STS_DELAY_US 10
+#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+
+/*
+ * The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
+ * have the same pci device id but different pci revision.
+ * Unfortunately, they have different register layouts, so two layout
+ * enumerations are defined.
+ */
+enum hisi_dma_reg_layout {
+ HISI_DMA_REG_LAYOUT_INVALID = 0,
+ HISI_DMA_REG_LAYOUT_HIP08,
+ HISI_DMA_REG_LAYOUT_HIP09
+};
enum hisi_dma_mode {
EP = 0,
@@ -108,9 +159,45 @@ struct hisi_dma_dev {
struct dma_device dma_dev;
u32 chan_num;
u32 chan_depth;
+ enum hisi_dma_reg_layout reg_layout;
+ void __iomem *queue_base; /* queue region start of register */
struct hisi_dma_chan chan[];
};
+static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_REG_LAYOUT_HIP08;
+ else if (pdev->revision >= HISI_DMA_REVISION_HIP09A)
+ return HISI_DMA_REG_LAYOUT_HIP09;
+
+ return HISI_DMA_REG_LAYOUT_INVALID;
+}
+
+static u32 hisi_dma_get_chan_num(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_CHAN_NUM;
+
+ return HISI_DMA_HIP09_CHAN_NUM;
+}
+
+static u32 hisi_dma_get_msi_num(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_MSI_NUM;
+
+ return HISI_DMA_HIP09_MSI_NUM;
+}
+
+static u32 hisi_dma_get_queue_base(struct pci_dev *pdev)
+{
+ if (pdev->revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_HIP08_Q_BASE;
+
+ return HISI_DMA_HIP09_Q_BASE;
+}
+
static inline struct hisi_dma_chan *to_hisi_dma_chan(struct dma_chan *c)
{
return container_of(c, struct hisi_dma_chan, vc.chan);
@@ -124,7 +211,7 @@ static inline struct hisi_dma_desc *to_hisi_dma_desc(struct virt_dma_desc *vd)
static inline void hisi_dma_chan_write(void __iomem *base, u32 reg, u32 index,
u32 val)
{
- writel_relaxed(val, base + reg + index * HISI_DMA_OFFSET);
+ writel_relaxed(val, base + reg + index * HISI_DMA_Q_OFFSET);
}
static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
@@ -132,55 +219,83 @@ static inline void hisi_dma_update_bit(void __iomem *addr, u32 pos, bool val)
u32 tmp;
tmp = readl_relaxed(addr);
- tmp = val ? tmp | BIT(pos) : tmp & ~BIT(pos);
+ tmp = val ? tmp | pos : tmp & ~pos;
writel_relaxed(tmp, addr);
}
static void hisi_dma_pause_dma(struct hisi_dma_dev *hdma_dev, u32 index,
bool pause)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_PAUSE_S, pause);
+ addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_PAUSE, pause);
}
static void hisi_dma_enable_dma(struct hisi_dma_dev *hdma_dev, u32 index,
bool enable)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL0 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL0_QUEUE_EN_S, enable);
+ addr = hdma_dev->queue_base + HISI_DMA_Q_CTRL0 +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL0_QUEUE_EN, enable);
}
static void hisi_dma_mask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_INT_MSK, qp_index,
- HISI_DMA_INT_STS_MASK);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
+ qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
+ else {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
+ qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
+ qp_index,
+ HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
+ }
}
static void hisi_dma_unmask_irq(struct hisi_dma_dev *hdma_dev, u32 qp_index)
{
- void __iomem *base = hdma_dev->base;
-
- hisi_dma_chan_write(base, HISI_DMA_INT_STS, qp_index,
- HISI_DMA_INT_STS_MASK);
- hisi_dma_chan_write(base, HISI_DMA_INT_MSK, qp_index, 0);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_STS,
+ qp_index, HISI_DMA_HIP08_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_INT_MSK,
+ qp_index, 0);
+ } else {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_STS,
+ qp_index, HISI_DMA_HIP09_Q_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_STS,
+ qp_index,
+ HISI_DMA_HIP09_Q_ERR_INT_STS_MASK);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_INT_MSK,
+ qp_index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP09_Q_ERR_INT_MSK,
+ qp_index, 0);
+ }
}
static void hisi_dma_do_reset(struct hisi_dma_dev *hdma_dev, u32 index)
{
- void __iomem *addr = hdma_dev->base + HISI_DMA_CTRL1 + index *
- HISI_DMA_OFFSET;
+ void __iomem *addr;
- hisi_dma_update_bit(addr, HISI_DMA_CTRL1_QUEUE_RESET_S, 1);
+ addr = hdma_dev->queue_base +
+ HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_Q_CTRL1_QUEUE_RESET, 1);
}
static void hisi_dma_reset_qp_point(struct hisi_dma_dev *hdma_dev, u32 index)
{
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, index, 0);
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR, index, 0);
+ void __iomem *q_base = hdma_dev->queue_base;
+
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
}
static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
@@ -195,11 +310,11 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
hisi_dma_enable_dma(hdma_dev, index, false);
hisi_dma_mask_irq(hdma_dev, index);
- addr = hdma_dev->base +
- HISI_DMA_Q_FSM_STS + index * HISI_DMA_OFFSET;
+ addr = hdma_dev->queue_base +
+ HISI_DMA_Q_FSM_STS + index * HISI_DMA_Q_OFFSET;
ret = readl_relaxed_poll_timeout(addr, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) != RUN,
+ FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) != RUN,
HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "disable channel timeout!\n");
@@ -216,7 +331,7 @@ static void hisi_dma_reset_or_disable_hw_chan(struct hisi_dma_chan *chan,
}
ret = readl_relaxed_poll_timeout(addr, tmp,
- FIELD_GET(HISI_DMA_FSM_STS_MASK, tmp) == IDLE,
+ FIELD_GET(HISI_DMA_Q_FSM_STS_MASK, tmp) == IDLE,
HISI_DMA_POLL_Q_STS_DELAY_US, HISI_DMA_POLL_Q_STS_TIME_OUT_US);
if (ret) {
dev_err(&hdma_dev->pdev->dev, "reset channel timeout!\n");
@@ -298,8 +413,8 @@ static void hisi_dma_start_transfer(struct hisi_dma_chan *chan)
chan->sq_tail = (chan->sq_tail + 1) % hdma_dev->chan_depth;
/* update sq_tail to trigger a new task */
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_SQ_TAIL_PTR, chan->qp_num,
- chan->sq_tail);
+ hisi_dma_chan_write(hdma_dev->queue_base, HISI_DMA_Q_SQ_TAIL_PTR,
+ chan->qp_num, chan->sq_tail);
}
static void hisi_dma_issue_pending(struct dma_chan *c)
@@ -373,26 +488,86 @@ static int hisi_dma_alloc_qps_mem(struct hisi_dma_dev *hdma_dev)
static void hisi_dma_init_hw_qp(struct hisi_dma_dev *hdma_dev, u32 index)
{
struct hisi_dma_chan *chan = &hdma_dev->chan[index];
+ void __iomem *q_base = hdma_dev->queue_base;
u32 hw_depth = hdma_dev->chan_depth - 1;
- void __iomem *base = hdma_dev->base;
+ void __iomem *addr;
+ u32 tmp;
/* set sq, cq base */
- hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_L, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_L, index,
lower_32_bits(chan->sq_dma));
- hisi_dma_chan_write(base, HISI_DMA_SQ_BASE_H, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_BASE_H, index,
upper_32_bits(chan->sq_dma));
- hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_L, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_L, index,
lower_32_bits(chan->cq_dma));
- hisi_dma_chan_write(base, HISI_DMA_CQ_BASE_H, index,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_BASE_H, index,
upper_32_bits(chan->cq_dma));
/* set sq, cq depth */
- hisi_dma_chan_write(base, HISI_DMA_SQ_DEPTH, index, hw_depth);
- hisi_dma_chan_write(base, HISI_DMA_CQ_DEPTH, index, hw_depth);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_DEPTH, index, hw_depth);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_DEPTH, index, hw_depth);
/* init sq tail and cq head */
- hisi_dma_chan_write(base, HISI_DMA_SQ_TAIL_PTR, index, 0);
- hisi_dma_chan_write(base, HISI_DMA_CQ_HEAD_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_SQ_TAIL_PTR, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR, index, 0);
+
+ /* init error interrupt stats */
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM0, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM1, index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_ERR_INT_NUM2, index, 0);
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM3,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM4,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM5,
+ index, 0);
+ hisi_dma_chan_write(q_base, HISI_DMA_HIP08_Q_ERR_INT_NUM6,
+ index, 0);
+ /*
+ * init SQ/CQ direction selecting register.
+ * "0" is to local side and "1" is to remote side.
+ */
+ addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP08_Q_CTRL0_SQCQ_DRCT, 0);
+
+ /*
+ * 0 - Continue to next descriptor if error occurs.
+ * 1 - Abort the DMA queue if error occurs.
+ */
+ hisi_dma_update_bit(addr,
+ HISI_DMA_HIP08_Q_CTRL0_ERR_ABORT_EN, 0);
+ } else {
+ addr = q_base + HISI_DMA_Q_CTRL0 + index * HISI_DMA_Q_OFFSET;
+
+ /*
+ * init SQ/CQ direction selecting register.
+ * "0" is to local side and "1" is to remote side.
+ */
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_SQ_DRCT, 0);
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL0_CQ_DRCT, 0);
+
+ /*
+ * 0 - Continue to next descriptor if error occurs.
+ * 1 - Abort the DMA queue if error occurs.
+ */
+
+ tmp = readl_relaxed(addr);
+ tmp &= ~HISI_DMA_HIP09_Q_CTRL0_ERR_ABORT_EN;
+ writel_relaxed(tmp, addr);
+
+ /*
+ * 0 - dma should process FLR whith CPU.
+ * 1 - dma not process FLR, only cpu process FLR.
+ */
+ addr = q_base + HISI_DMA_HIP09_DMA_FLR_DISABLE +
+ index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_DMA_FLR_DISABLE_B, 0);
+
+ addr = q_base + HISI_DMA_Q_CTRL1 + index * HISI_DMA_Q_OFFSET;
+ hisi_dma_update_bit(addr, HISI_DMA_HIP09_Q_CTRL1_VA_ENABLE, 1);
+ }
}
static void hisi_dma_enable_qp(struct hisi_dma_dev *hdma_dev, u32 qp_index)
@@ -436,14 +611,16 @@ static irqreturn_t hisi_dma_irq(int irq, void *data)
struct hisi_dma_dev *hdma_dev = chan->hdma_dev;
struct hisi_dma_desc *desc;
struct hisi_dma_cqe *cqe;
+ void __iomem *q_base;
spin_lock(&chan->vc.lock);
desc = chan->desc;
cqe = chan->cq + chan->cq_head;
+ q_base = hdma_dev->queue_base;
if (desc) {
chan->cq_head = (chan->cq_head + 1) % hdma_dev->chan_depth;
- hisi_dma_chan_write(hdma_dev->base, HISI_DMA_CQ_HEAD_PTR,
+ hisi_dma_chan_write(q_base, HISI_DMA_Q_CQ_HEAD_PTR,
chan->qp_num, chan->cq_head);
if (FIELD_GET(STATUS_MASK, cqe->w0) == STATUS_SUCC) {
vchan_cookie_complete(&desc->vd);
@@ -504,16 +681,58 @@ static void hisi_dma_disable_hw_channels(void *data)
static void hisi_dma_set_mode(struct hisi_dma_dev *hdma_dev,
enum hisi_dma_mode mode)
{
- writel_relaxed(mode == RC ? 1 : 0, hdma_dev->base + HISI_DMA_MODE);
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ writel_relaxed(mode == RC ? 1 : 0,
+ hdma_dev->base + HISI_DMA_HIP08_MODE);
+}
+
+static void hisi_dma_init_hw(struct hisi_dma_dev *hdma_dev)
+{
+ void __iomem *addr;
+ int i;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP09) {
+ for (i = 0; i < HISI_DMA_HIP09_MAX_PORT_NUM; i++) {
+ addr = hdma_dev->base + HISI_DMA_HIP09_PORT_CFG_REG(i);
+ hisi_dma_update_bit(addr,
+ HISI_DMA_HIP09_PORT_CFG_LINK_DOWN_MASK_B, 1);
+ }
+ }
+}
+
+static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
+{
+ struct dma_device *dma_dev;
+
+ dma_dev = &hdma_dev->dma_dev;
+ dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
+ dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
+ dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
+ dma_dev->device_tx_status = hisi_dma_tx_status;
+ dma_dev->device_issue_pending = hisi_dma_issue_pending;
+ dma_dev->device_terminate_all = hisi_dma_terminate_all;
+ dma_dev->device_synchronize = hisi_dma_synchronize;
+ dma_dev->directions = BIT(DMA_MEM_TO_MEM);
+ dma_dev->dev = &hdma_dev->pdev->dev;
+ INIT_LIST_HEAD(&dma_dev->channels);
}
static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
+ enum hisi_dma_reg_layout reg_layout;
struct device *dev = &pdev->dev;
struct hisi_dma_dev *hdma_dev;
struct dma_device *dma_dev;
+ u32 chan_num;
+ u32 msi_num;
int ret;
+ reg_layout = hisi_dma_get_reg_layout(pdev);
+ if (reg_layout == HISI_DMA_REG_LAYOUT_INVALID) {
+ dev_err(dev, "unsupported device!\n");
+ return -EINVAL;
+ }
+
ret = pcim_enable_device(pdev);
if (ret) {
dev_err(dev, "failed to enable device mem!\n");
@@ -530,40 +749,37 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (ret)
return ret;
- hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, HISI_DMA_CHAN_NUM), GFP_KERNEL);
+ chan_num = hisi_dma_get_chan_num(pdev);
+ hdma_dev = devm_kzalloc(dev, struct_size(hdma_dev, chan, chan_num),
+ GFP_KERNEL);
if (!hdma_dev)
return -EINVAL;
hdma_dev->base = pcim_iomap_table(pdev)[PCI_BAR_2];
hdma_dev->pdev = pdev;
- hdma_dev->chan_num = HISI_DMA_CHAN_NUM;
hdma_dev->chan_depth = HISI_DMA_Q_DEPTH_VAL;
+ hdma_dev->chan_num = chan_num;
+ hdma_dev->reg_layout = reg_layout;
+ hdma_dev->queue_base = hdma_dev->base + hisi_dma_get_queue_base(pdev);
pci_set_drvdata(pdev, hdma_dev);
pci_set_master(pdev);
+ msi_num = hisi_dma_get_msi_num(pdev);
+
/* This will be freed by 'pcim_release()'. See 'pcim_enable_device()' */
- ret = pci_alloc_irq_vectors(pdev, HISI_DMA_MSI_NUM, HISI_DMA_MSI_NUM,
- PCI_IRQ_MSI);
+ ret = pci_alloc_irq_vectors(pdev, msi_num, msi_num, PCI_IRQ_MSI);
if (ret < 0) {
dev_err(dev, "Failed to allocate MSI vectors!\n");
return ret;
}
- dma_dev = &hdma_dev->dma_dev;
- dma_cap_set(DMA_MEMCPY, dma_dev->cap_mask);
- dma_dev->device_free_chan_resources = hisi_dma_free_chan_resources;
- dma_dev->device_prep_dma_memcpy = hisi_dma_prep_dma_memcpy;
- dma_dev->device_tx_status = hisi_dma_tx_status;
- dma_dev->device_issue_pending = hisi_dma_issue_pending;
- dma_dev->device_terminate_all = hisi_dma_terminate_all;
- dma_dev->device_synchronize = hisi_dma_synchronize;
- dma_dev->directions = BIT(DMA_MEM_TO_MEM);
- dma_dev->dev = dev;
- INIT_LIST_HEAD(&dma_dev->channels);
+ hisi_dma_init_dma_dev(hdma_dev);
hisi_dma_set_mode(hdma_dev, RC);
+ hisi_dma_init_hw(hdma_dev);
+
ret = hisi_dma_enable_hw_channels(hdma_dev);
if (ret < 0) {
dev_err(dev, "failed to enable hw channel!\n");
@@ -575,6 +791,7 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (ret)
return ret;
+ dma_dev = &hdma_dev->dma_dev;
ret = dmaenginem_async_device_register(dma_dev);
if (ret < 0)
dev_err(dev, "failed to register device!\n");
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v5 6/7] dmaengine: hisilicon: Dump regs to debugfs
2022-08-05 8:25 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (4 preceding siblings ...)
2022-08-05 8:25 ` [PATCH v5 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
@ 2022-08-05 8:25 ` Jie Hai
2022-08-05 8:25 ` [PATCH v5 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-05 8:25 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
This patch adds dump of registers with debugfs for HIP08
and HIP09 DMA driver.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
drivers/dma/hisi_dma.c | 238 ++++++++++++++++++++++++++++++++++++++++-
1 file changed, 236 insertions(+), 2 deletions(-)
diff --git a/drivers/dma/hisi_dma.c b/drivers/dma/hisi_dma.c
index da5e49ee95fa..c1350a36fddd 100644
--- a/drivers/dma/hisi_dma.c
+++ b/drivers/dma/hisi_dma.c
@@ -78,6 +78,8 @@
#define HISI_DMA_POLL_Q_STS_DELAY_US 10
#define HISI_DMA_POLL_Q_STS_TIME_OUT_US 1000
+#define HISI_DMA_MAX_DIR_NAME_LEN 128
+
/*
* The HIP08B(HiSilicon IP08) and HIP09A(HiSilicon IP09) are DMA iEPs, they
* have the same pci device id but different pci revision.
@@ -164,6 +166,123 @@ struct hisi_dma_dev {
struct hisi_dma_chan chan[];
};
+#ifdef CONFIG_DEBUG_FS
+
+static const struct debugfs_reg32 hisi_dma_comm_chan_regs[] = {
+ {"DMA_QUEUE_SQ_DEPTH ", 0x0008ull},
+ {"DMA_QUEUE_SQ_TAIL_PTR ", 0x000Cull},
+ {"DMA_QUEUE_CQ_DEPTH ", 0x0018ull},
+ {"DMA_QUEUE_CQ_HEAD_PTR ", 0x001Cull},
+ {"DMA_QUEUE_CTRL0 ", 0x0020ull},
+ {"DMA_QUEUE_CTRL1 ", 0x0024ull},
+ {"DMA_QUEUE_FSM_STS ", 0x0030ull},
+ {"DMA_QUEUE_SQ_STS ", 0x0034ull},
+ {"DMA_QUEUE_CQ_TAIL_PTR ", 0x003Cull},
+ {"DMA_QUEUE_INT_STS ", 0x0040ull},
+ {"DMA_QUEUE_INT_MSK ", 0x0044ull},
+ {"DMA_QUEUE_INT_RO ", 0x006Cull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip08_chan_regs[] = {
+ {"DMA_QUEUE_BYTE_CNT ", 0x0038ull},
+ {"DMA_ERR_INT_NUM6 ", 0x0048ull},
+ {"DMA_QUEUE_DESP0 ", 0x0050ull},
+ {"DMA_QUEUE_DESP1 ", 0x0054ull},
+ {"DMA_QUEUE_DESP2 ", 0x0058ull},
+ {"DMA_QUEUE_DESP3 ", 0x005Cull},
+ {"DMA_QUEUE_DESP4 ", 0x0074ull},
+ {"DMA_QUEUE_DESP5 ", 0x0078ull},
+ {"DMA_QUEUE_DESP6 ", 0x007Cull},
+ {"DMA_QUEUE_DESP7 ", 0x0080ull},
+ {"DMA_ERR_INT_NUM0 ", 0x0084ull},
+ {"DMA_ERR_INT_NUM1 ", 0x0088ull},
+ {"DMA_ERR_INT_NUM2 ", 0x008Cull},
+ {"DMA_ERR_INT_NUM3 ", 0x0090ull},
+ {"DMA_ERR_INT_NUM4 ", 0x0094ull},
+ {"DMA_ERR_INT_NUM5 ", 0x0098ull},
+ {"DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip09_chan_regs[] = {
+ {"DMA_QUEUE_ERR_INT_STS ", 0x0048ull},
+ {"DMA_QUEUE_ERR_INT_MSK ", 0x004Cull},
+ {"DFX_SQ_READ_ERR_PTR ", 0x0068ull},
+ {"DFX_DMA_ERR_INT_NUM0 ", 0x0084ull},
+ {"DFX_DMA_ERR_INT_NUM1 ", 0x0088ull},
+ {"DFX_DMA_ERR_INT_NUM2 ", 0x008Cull},
+ {"DFX_DMA_QUEUE_SQ_STS2 ", 0x00A4ull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip08_comm_regs[] = {
+ {"DMA_ECC_ERR_ADDR ", 0x2004ull},
+ {"DMA_ECC_ECC_CNT ", 0x2014ull},
+ {"COMMON_AND_CH_ERR_STS ", 0x2030ull},
+ {"LOCAL_CPL_ID_STS_0 ", 0x20E0ull},
+ {"LOCAL_CPL_ID_STS_1 ", 0x20E4ull},
+ {"LOCAL_CPL_ID_STS_2 ", 0x20E8ull},
+ {"LOCAL_CPL_ID_STS_3 ", 0x20ECull},
+ {"LOCAL_TLP_NUM ", 0x2158ull},
+ {"SQCQ_TLP_NUM ", 0x2164ull},
+ {"CPL_NUM ", 0x2168ull},
+ {"INF_BACK_PRESS_STS ", 0x2170ull},
+ {"DMA_CH_RAS_LEVEL ", 0x2184ull},
+ {"DMA_CM_RAS_LEVEL ", 0x2188ull},
+ {"DMA_CH_ERR_STS ", 0x2190ull},
+ {"DMA_CH_DONE_STS ", 0x2194ull},
+ {"DMA_SQ_TAG_STS_0 ", 0x21A0ull},
+ {"DMA_SQ_TAG_STS_1 ", 0x21A4ull},
+ {"DMA_SQ_TAG_STS_2 ", 0x21A8ull},
+ {"DMA_SQ_TAG_STS_3 ", 0x21ACull},
+ {"LOCAL_P_ID_STS_0 ", 0x21B0ull},
+ {"LOCAL_P_ID_STS_1 ", 0x21B4ull},
+ {"LOCAL_P_ID_STS_2 ", 0x21B8ull},
+ {"LOCAL_P_ID_STS_3 ", 0x21BCull},
+ {"DMA_PREBUFF_INFO_0 ", 0x2200ull},
+ {"DMA_CM_TABLE_INFO_0 ", 0x2220ull},
+ {"DMA_CM_CE_RO ", 0x2244ull},
+ {"DMA_CM_NFE_RO ", 0x2248ull},
+ {"DMA_CM_FE_RO ", 0x224Cull},
+};
+
+static const struct debugfs_reg32 hisi_dma_hip09_comm_regs[] = {
+ {"COMMON_AND_CH_ERR_STS ", 0x0030ull},
+ {"DMA_PORT_IDLE_STS ", 0x0150ull},
+ {"DMA_CH_RAS_LEVEL ", 0x0184ull},
+ {"DMA_CM_RAS_LEVEL ", 0x0188ull},
+ {"DMA_CM_CE_RO ", 0x0244ull},
+ {"DMA_CM_NFE_RO ", 0x0248ull},
+ {"DMA_CM_FE_RO ", 0x024Cull},
+ {"DFX_INF_BACK_PRESS_STS0 ", 0x1A40ull},
+ {"DFX_INF_BACK_PRESS_STS1 ", 0x1A44ull},
+ {"DFX_INF_BACK_PRESS_STS2 ", 0x1A48ull},
+ {"DFX_DMA_WRR_DISABLE ", 0x1A4Cull},
+ {"DFX_PA_REQ_TLP_NUM ", 0x1C00ull},
+ {"DFX_PA_BACK_TLP_NUM ", 0x1C04ull},
+ {"DFX_PA_RETRY_TLP_NUM ", 0x1C08ull},
+ {"DFX_LOCAL_NP_TLP_NUM ", 0x1C0Cull},
+ {"DFX_LOCAL_CPL_HEAD_TLP_NUM ", 0x1C10ull},
+ {"DFX_LOCAL_CPL_DATA_TLP_NUM ", 0x1C14ull},
+ {"DFX_LOCAL_CPL_EXT_DATA_TLP_NUM ", 0x1C18ull},
+ {"DFX_LOCAL_P_HEAD_TLP_NUM ", 0x1C1Cull},
+ {"DFX_LOCAL_P_ACK_TLP_NUM ", 0x1C20ull},
+ {"DFX_BUF_ALOC_PORT_REQ_NUM ", 0x1C24ull},
+ {"DFX_BUF_ALOC_PORT_RESULT_NUM ", 0x1C28ull},
+ {"DFX_BUF_FAIL_SIZE_NUM ", 0x1C2Cull},
+ {"DFX_BUF_ALOC_SIZE_NUM ", 0x1C30ull},
+ {"DFX_BUF_NP_RELEASE_SIZE_NUM ", 0x1C34ull},
+ {"DFX_BUF_P_RELEASE_SIZE_NUM ", 0x1C38ull},
+ {"DFX_BUF_PORT_RELEASE_SIZE_NUM ", 0x1C3Cull},
+ {"DFX_DMA_PREBUF_MEM0_ECC_ERR_ADDR ", 0x1CA8ull},
+ {"DFX_DMA_PREBUF_MEM0_ECC_CNT ", 0x1CACull},
+ {"DFX_DMA_LOC_NP_OSTB_ECC_ERR_ADDR ", 0x1CB0ull},
+ {"DFX_DMA_LOC_NP_OSTB_ECC_CNT ", 0x1CB4ull},
+ {"DFX_DMA_PREBUF_MEM1_ECC_ERR_ADDR ", 0x1CC0ull},
+ {"DFX_DMA_PREBUF_MEM1_ECC_CNT ", 0x1CC4ull},
+ {"DMA_CH_DONE_STS ", 0x02E0ull},
+ {"DMA_CH_ERR_STS ", 0x0320ull},
+};
+#endif /* CONFIG_DEBUG_FS*/
+
static enum hisi_dma_reg_layout hisi_dma_get_reg_layout(struct pci_dev *pdev)
{
if (pdev->revision == HISI_DMA_REVISION_HIP08B)
@@ -717,6 +836,117 @@ static void hisi_dma_init_dma_dev(struct hisi_dma_dev *hdma_dev)
INIT_LIST_HEAD(&dma_dev->channels);
}
+/* --- debugfs implementation --- */
+#ifdef CONFIG_DEBUG_FS
+#include <linux/debugfs.h>
+static struct debugfs_reg32 *hisi_dma_get_ch_regs(struct hisi_dma_dev *hdma_dev,
+ u32 *regs_sz)
+{
+ struct device *dev = &hdma_dev->pdev->dev;
+ struct debugfs_reg32 *regs;
+ u32 regs_sz_comm;
+
+ regs_sz_comm = ARRAY_SIZE(hisi_dma_comm_chan_regs);
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip08_chan_regs);
+ else
+ *regs_sz = regs_sz_comm + ARRAY_SIZE(hisi_dma_hip09_chan_regs);
+
+ regs = devm_kcalloc(dev, *regs_sz, sizeof(struct debugfs_reg32),
+ GFP_KERNEL);
+ if (!regs)
+ return NULL;
+ memcpy(regs, hisi_dma_comm_chan_regs, sizeof(hisi_dma_comm_chan_regs));
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ memcpy(regs + regs_sz_comm, hisi_dma_hip08_chan_regs,
+ sizeof(hisi_dma_hip08_chan_regs));
+ else
+ memcpy(regs + regs_sz_comm, hisi_dma_hip09_chan_regs,
+ sizeof(hisi_dma_hip09_chan_regs));
+
+ return regs;
+}
+
+static int hisi_dma_create_chan_dir(struct hisi_dma_dev *hdma_dev)
+{
+ char dir_name[HISI_DMA_MAX_DIR_NAME_LEN];
+ struct debugfs_regset32 *regsets;
+ struct debugfs_reg32 *regs;
+ struct dentry *chan_dir;
+ struct device *dev;
+ u32 regs_sz;
+ int ret;
+ int i;
+
+ dev = &hdma_dev->pdev->dev;
+
+ regsets = devm_kcalloc(dev, hdma_dev->chan_num,
+ sizeof(*regsets), GFP_KERNEL);
+ if (!regsets)
+ return -ENOMEM;
+
+ regs = hisi_dma_get_ch_regs(hdma_dev, ®s_sz);
+ if (!regs)
+ return -ENOMEM;
+
+ for (i = 0; i < hdma_dev->chan_num; i++) {
+ regsets[i].regs = regs;
+ regsets[i].nregs = regs_sz;
+ regsets[i].base = hdma_dev->queue_base + i * HISI_DMA_Q_OFFSET;
+ regsets[i].dev = dev;
+
+ memset(dir_name, 0, HISI_DMA_MAX_DIR_NAME_LEN);
+ ret = sprintf(dir_name, "channel%d", i);
+ if (ret < 0)
+ return ret;
+
+ chan_dir = debugfs_create_dir(dir_name,
+ hdma_dev->dma_dev.dbg_dev_root);
+ debugfs_create_regset32("regs", 0444, chan_dir, ®sets[i]);
+ }
+
+ return 0;
+}
+
+static void hisi_dma_create_debugfs(struct hisi_dma_dev *hdma_dev)
+{
+ struct debugfs_regset32 *regset;
+ struct device *dev;
+ int ret;
+
+ dev = &hdma_dev->pdev->dev;
+
+ if (hdma_dev->dma_dev.dbg_dev_root == NULL)
+ return;
+
+ regset = devm_kzalloc(dev, sizeof(*regset), GFP_KERNEL);
+ if (!regset)
+ return;
+
+ if (hdma_dev->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ regset->regs = hisi_dma_hip08_comm_regs;
+ regset->nregs = ARRAY_SIZE(hisi_dma_hip08_comm_regs);
+ } else {
+ regset->regs = hisi_dma_hip09_comm_regs;
+ regset->nregs = ARRAY_SIZE(hisi_dma_hip09_comm_regs);
+ }
+ regset->base = hdma_dev->base;
+ regset->dev = dev;
+
+ debugfs_create_regset32("regs", 0444,
+ hdma_dev->dma_dev.dbg_dev_root, regset);
+
+ ret = hisi_dma_create_chan_dir(hdma_dev);
+ if (ret < 0)
+ dev_info(&hdma_dev->pdev->dev, "fail to create debugfs for channels!\n");
+}
+#else
+static void hisi_dma_create_debugfs(struct hisi_dma_dev *hdma_dev) { }
+#endif /* CONFIG_DEBUG_FS*/
+/* --- debugfs implementation --- */
+
static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
enum hisi_dma_reg_layout reg_layout;
@@ -793,10 +1023,14 @@ static int hisi_dma_probe(struct pci_dev *pdev, const struct pci_device_id *id)
dma_dev = &hdma_dev->dma_dev;
ret = dmaenginem_async_device_register(dma_dev);
- if (ret < 0)
+ if (ret < 0) {
dev_err(dev, "failed to register device!\n");
+ return ret;
+ }
+
+ hisi_dma_create_debugfs(hdma_dev);
- return ret;
+ return 0;
}
static const struct pci_device_id hisi_dma_pci_tbl[] = {
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v5 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma
2022-08-05 8:25 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
` (5 preceding siblings ...)
2022-08-05 8:25 ` [PATCH v5 6/7] dmaengine: hisilicon: Dump regs to debugfs Jie Hai
@ 2022-08-05 8:25 ` Jie Hai
6 siblings, 0 replies; 64+ messages in thread
From: Jie Hai @ 2022-08-05 8:25 UTC (permalink / raw)
To: vkoul, wangzhou1; +Cc: haijie1, liudongdong3, dmaengine, linux-kernel
Add myself as a maintainer for hisi_dma.
Signed-off-by: Jie Hai <haijie1@huawei.com>
Acked-by: Zhou Wang <wangzhou1@hisilicon.com>
---
MAINTAINERS | 1 +
1 file changed, 1 insertion(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 64379c699903..b1aeffd74a37 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -8971,6 +8971,7 @@ F: net/dsa/tag_hellcreek.c
HISILICON DMA DRIVER
M: Zhou Wang <wangzhou1@hisilicon.com>
+M: Jie Hai <haijie1@hisilicon.com>
L: dmaengine@vger.kernel.org
S: Maintained
F: drivers/dma/hisi_dma.c
--
2.33.0
^ permalink raw reply related [flat|nested] 64+ messages in thread
end of thread, other threads:[~2022-08-05 8:35 UTC | newest]
Thread overview: 64+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-06-25 7:44 [PATCH 0/8] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-06-25 7:44 ` [PATCH 1/8] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
2022-06-25 7:44 ` [PATCH 2/8] dmaengine: hisilicon: Fix CQ head update Jie Hai
2022-06-26 13:37 ` kernel test robot
2022-06-27 6:55 ` haijie
2022-06-27 6:55 ` haijie
2022-06-27 6:12 ` Vinod Koul
2022-06-27 7:01 ` haijie
2022-06-27 17:38 ` Vinod Koul
2022-06-25 7:44 ` [PATCH 3/8] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
2022-06-27 6:21 ` Vinod Koul
[not found] ` <eed10a2b5ea745878024ca653158f4de@huawei.com>
2022-06-27 13:14 ` haijie
2022-06-29 3:44 ` Jie Hai
2022-06-25 7:44 ` [PATCH 4/8] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
2022-06-25 7:44 ` [PATCH 5/8] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
2022-06-25 7:44 ` [PATCH 6/8] dmaengine: hisilicon: Add dfx feature for hisi dma driver Jie Hai
2022-06-25 9:36 ` kernel test robot
2022-06-27 6:55 ` haijie
2022-06-27 6:55 ` haijie
2022-06-25 7:44 ` [PATCH 7/8] Documentation: Add debugfs doc for hisi_dma Jie Hai
2022-06-27 5:50 ` Vinod Koul
2022-06-25 7:44 ` [PATCH 8/8] MAINTAINERS: Add debugfs files and maintainer " Jie Hai
2022-06-29 3:55 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-06-29 3:55 ` [PATCH v2 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
2022-06-29 3:55 ` [PATCH v2 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
2022-07-21 13:27 ` Vinod Koul
2022-07-26 1:38 ` Jie Hai
2022-06-29 3:55 ` [PATCH v2 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
2022-06-29 3:55 ` [PATCH v2 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
2022-06-29 3:55 ` [PATCH v2 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
2022-07-21 13:29 ` Vinod Koul
2022-07-26 1:40 ` Jie Hai
2022-06-29 3:55 ` [PATCH v2 6/7] dmaengine: hisilicon: Add dfx feature for hisi dma driver Jie Hai
2022-07-21 13:25 ` Vinod Koul
2022-07-26 1:43 ` Jie Hai
2022-07-26 12:39 ` Vinod Koul
2022-07-27 0:58 ` Jie Hai
2022-06-29 3:55 ` [PATCH v2 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
2022-07-08 3:16 ` [PATCH v2 0/7] dmaengine: hisilicon: Add support for hisi dma driver Zhou Wang
2022-07-26 1:35 ` [PATCH v3 " Jie Hai
2022-07-26 1:35 ` [PATCH v3 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
2022-07-26 1:35 ` [PATCH v3 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
2022-07-26 1:35 ` [PATCH v3 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
2022-07-26 1:35 ` [PATCH v3 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
2022-07-26 1:35 ` [PATCH v3 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
2022-07-26 1:35 ` [PATCH v3 6/7] dmaengine: hisilicon: dump regs to debugfs Jie Hai
2022-07-26 1:35 ` [PATCH v3 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
2022-08-02 10:12 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-08-02 10:12 ` [PATCH v4 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
2022-08-02 10:12 ` [PATCH v4 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
2022-08-02 10:12 ` [PATCH v4 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
2022-08-02 10:12 ` [PATCH v4 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
2022-08-02 10:12 ` [PATCH v4 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
2022-08-05 7:41 ` Jie Hai
2022-08-02 10:12 ` [PATCH v4 6/7] dmaengine: hisilicon: Dump regs to debugfs Jie Hai
2022-08-02 10:12 ` [PATCH v4 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
2022-08-05 8:25 ` [PATCH v4 0/7] dmaengine: hisilicon: Add support for hisi dma driver Jie Hai
2022-08-05 8:25 ` [PATCH v5 1/7] dmaengine: hisilicon: Disable channels when unregister hisi_dma Jie Hai
2022-08-05 8:25 ` [PATCH v5 2/7] dmaengine: hisilicon: Fix CQ head update Jie Hai
2022-08-05 8:25 ` [PATCH v5 3/7] dmaengine: hisilicon: Add multi-thread support for a DMA channel Jie Hai
2022-08-05 8:25 ` [PATCH v5 4/7] dmaengine: hisilicon: Use macros instead of magic number Jie Hai
2022-08-05 8:25 ` [PATCH v5 5/7] dmaengine: hisilicon: Adapt DMA driver to HiSilicon IP09 Jie Hai
2022-08-05 8:25 ` [PATCH v5 6/7] dmaengine: hisilicon: Dump regs to debugfs Jie Hai
2022-08-05 8:25 ` [PATCH v5 7/7] MAINTAINERS: Add myself as maintainer for hisi_dma Jie Hai
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.