* [dpdk-dev] [PATCH 0/6] dma: add hisilicon DMA driver
@ 2021-10-30 10:36 Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 1/6] dma/hisilicon: add device probe and remove functions Chengwen Feng
` (6 more replies)
0 siblings, 7 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-10-30 10:36 UTC (permalink / raw)
To: thomas; +Cc: dev
This patch set add hisilicon DMA driver.
Chengwen Feng (6):
dma/hisilicon: add device probe and remove functions
dma/hisilicon: add dmadev instances create and destroy
dma/hisilicon: add control path functions
dma/hisilicon: add data path functions
dma/hisilicon: support multi-process
devbind: add Kunpeng DMA to dmadev category
MAINTAINERS | 5 +
doc/guides/dmadevs/hisilicon.rst | 41 ++
doc/guides/dmadevs/index.rst | 1 +
doc/guides/rel_notes/release_21_11.rst | 4 +
drivers/dma/hisilicon/hisi_dmadev.c | 925 +++++++++++++++++++++++++
drivers/dma/hisilicon/hisi_dmadev.h | 236 +++++++
drivers/dma/hisilicon/meson.build | 7 +
drivers/dma/hisilicon/version.map | 3 +
drivers/dma/meson.build | 1 +
usertools/dpdk-devbind.py | 6 +-
10 files changed, 1228 insertions(+), 1 deletion(-)
create mode 100644 doc/guides/dmadevs/hisilicon.rst
create mode 100644 drivers/dma/hisilicon/hisi_dmadev.c
create mode 100644 drivers/dma/hisilicon/hisi_dmadev.h
create mode 100644 drivers/dma/hisilicon/meson.build
create mode 100644 drivers/dma/hisilicon/version.map
--
2.33.0
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH 1/6] dma/hisilicon: add device probe and remove functions
2021-10-30 10:36 [dpdk-dev] [PATCH 0/6] dma: add hisilicon DMA driver Chengwen Feng
@ 2021-10-30 10:36 ` Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 2/6] dma/hisilicon: add dmadev instances create and destroy Chengwen Feng
` (5 subsequent siblings)
6 siblings, 0 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-10-30 10:36 UTC (permalink / raw)
To: thomas; +Cc: dev
Add the basic device probe and remove functions and initial
documentation for new hisilicon DMA drivers. Maintainers update is also
included in this patch.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
MAINTAINERS | 5 ++
doc/guides/dmadevs/hisilicon.rst | 21 +++++
doc/guides/dmadevs/index.rst | 1 +
doc/guides/rel_notes/release_21_11.rst | 4 +
drivers/dma/hisilicon/hisi_dmadev.c | 119 +++++++++++++++++++++++++
drivers/dma/hisilicon/hisi_dmadev.h | 24 +++++
drivers/dma/hisilicon/meson.build | 7 ++
drivers/dma/hisilicon/version.map | 3 +
drivers/dma/meson.build | 1 +
9 files changed, 185 insertions(+)
create mode 100644 doc/guides/dmadevs/hisilicon.rst
create mode 100644 drivers/dma/hisilicon/hisi_dmadev.c
create mode 100644 drivers/dma/hisilicon/hisi_dmadev.h
create mode 100644 drivers/dma/hisilicon/meson.build
create mode 100644 drivers/dma/hisilicon/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 0e5951f8f1..1567f7b695 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1206,6 +1206,11 @@ M: Conor Walsh <conor.walsh@intel.com>
F: drivers/dma/ioat/
F: doc/guides/dmadevs/ioat.rst
+Hisilicon DMA
+M: Chengwen Feng <fengchengwen@huawei.com>
+F: drivers/dma/hisilicon
+F: doc/guides/dmadevs/hisilicon.rst
+
RegEx Drivers
-------------
diff --git a/doc/guides/dmadevs/hisilicon.rst b/doc/guides/dmadevs/hisilicon.rst
new file mode 100644
index 0000000000..4cbaac4204
--- /dev/null
+++ b/doc/guides/dmadevs/hisilicon.rst
@@ -0,0 +1,21 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2021 HiSilicon Limited.
+
+HISILICON Kunpeng DMA Driver
+============================
+
+Kunpeng SoC has an internal DMA unit which can be used by application to
+accelerate data copies. The DMA PF function supports multiple DMA channels.
+
+
+Supported Kunpeng SoCs
+----------------------
+
+* Kunpeng 920
+
+
+Device Setup
+-------------
+
+Kunpeng DMA devices will need to be bound to a suitable DPDK-supported
+user-space IO driver such as ``vfio-pci`` in order to be used by DPDK.
diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst
index 20476039a5..6b04276524 100644
--- a/doc/guides/dmadevs/index.rst
+++ b/doc/guides/dmadevs/index.rst
@@ -13,3 +13,4 @@ an application through DMA API.
idxd
ioat
+ hisilicon
diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
index 502cc5ceb2..00a45475be 100644
--- a/doc/guides/rel_notes/release_21_11.rst
+++ b/doc/guides/rel_notes/release_21_11.rst
@@ -86,6 +86,10 @@ New Features
driver for Intel IOAT devices such as Crystal Beach DMA (CBDMA) on Ice Lake,
Skylake and Broadwell. This device driver can be used through the generic dmadev API.
+* **Added hisilicon dmadev driver implementation.**
+ The hisilicon dmadev driver provide device drivers for the Kunpeng's DMA devices.
+ This device driver can be used through the generic dmadev API.
+
* **Added support to get all MAC addresses of a device.**
Added ``rte_eth_macaddrs_get`` to allow user to retrieve all Ethernet
diff --git a/drivers/dma/hisilicon/hisi_dmadev.c b/drivers/dma/hisilicon/hisi_dmadev.c
new file mode 100644
index 0000000000..e6fb8a0fc8
--- /dev/null
+++ b/drivers/dma/hisilicon/hisi_dmadev.c
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2021 HiSilicon Limited
+ */
+
+#include <inttypes.h>
+#include <string.h>
+
+#include <rte_bus_pci.h>
+#include <rte_eal.h>
+#include <rte_log.h>
+#include <rte_pci.h>
+#include <rte_dmadev_pmd.h>
+
+#include "hisi_dmadev.h"
+
+RTE_LOG_REGISTER_DEFAULT(hisi_dma_logtype, INFO);
+#define HISI_DMA_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, hisi_dma_logtype, \
+ "%s(): " fmt "\n", __func__, ##args)
+#define HISI_DMA_LOG_RAW(hw, level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, hisi_dma_logtype, \
+ "%s %s(): " fmt "\n", (hw)->data->dev_name, \
+ __func__, ##args)
+#define HISI_DMA_DEBUG(hw, fmt, args...) \
+ HISI_DMA_LOG_RAW(hw, DEBUG, fmt, ## args)
+#define HISI_DMA_INFO(hw, fmt, args...) \
+ HISI_DMA_LOG_RAW(hw, INFO, fmt, ## args)
+#define HISI_DMA_WARN(hw, fmt, args...) \
+ HISI_DMA_LOG_RAW(hw, WARNING, fmt, ## args)
+#define HISI_DMA_ERR(hw, fmt, args...) \
+ HISI_DMA_LOG_RAW(hw, ERR, fmt, ## args)
+
+static uint8_t
+hisi_dma_reg_layout(uint8_t revision)
+{
+ if (revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_REG_LAYOUT_HIP08;
+ else
+ return HISI_DMA_REG_LAYOUT_INVALID;
+}
+
+static void
+hisi_dma_gen_pci_device_name(const struct rte_pci_device *pci_dev,
+ char *name, size_t size)
+{
+ memset(name, 0, size);
+ (void)snprintf(name, size, "%x:%x.%x",
+ pci_dev->addr.bus, pci_dev->addr.devid,
+ pci_dev->addr.function);
+}
+
+static int
+hisi_dma_check_revision(struct rte_pci_device *pci_dev, const char *name,
+ uint8_t *out_revision)
+{
+ uint8_t revision;
+ int ret;
+
+ ret = rte_pci_read_config(pci_dev, &revision, 1,
+ HISI_DMA_PCI_REVISION_ID_REG);
+ if (ret != 1) {
+ HISI_DMA_LOG(ERR, "%s read PCI revision failed!", name);
+ return -EINVAL;
+ }
+ if (hisi_dma_reg_layout(revision) == HISI_DMA_REG_LAYOUT_INVALID) {
+ HISI_DMA_LOG(ERR, "%s revision: 0x%x not supported!",
+ name, revision);
+ return -EINVAL;
+ }
+
+ *out_revision = revision;
+ return 0;
+}
+
+static int
+hisi_dma_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ char name[RTE_DEV_NAME_MAX_LEN] = { 0 };
+ uint8_t revision;
+ int ret;
+
+ hisi_dma_gen_pci_device_name(pci_dev, name, sizeof(name));
+
+ if (pci_dev->mem_resource[2].addr == NULL) {
+ HISI_DMA_LOG(ERR, "%s BAR2 is NULL!\n", name);
+ return -ENODEV;
+ }
+
+ ret = hisi_dma_check_revision(pci_dev, name, &revision);
+ if (ret)
+ return ret;
+ HISI_DMA_LOG(DEBUG, "%s read PCI revision: 0x%x", name, revision);
+
+ return ret;
+}
+
+static int
+hisi_dma_remove(struct rte_pci_device *pci_dev)
+{
+ RTE_SET_USED(pci_dev);
+ return 0;
+}
+
+static const struct rte_pci_id pci_id_hisi_dma_map[] = {
+ { RTE_PCI_DEVICE(PCI_VENDOR_ID_HUAWEI, HISI_DMA_DEVICE_ID) },
+ { .vendor_id = 0, }, /* sentinel */
+};
+
+static struct rte_pci_driver hisi_dma_pmd_drv = {
+ .id_table = pci_id_hisi_dma_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
+ .probe = hisi_dma_probe,
+ .remove = hisi_dma_remove,
+};
+
+RTE_PMD_REGISTER_PCI(dma_hisilicon, hisi_dma_pmd_drv);
+RTE_PMD_REGISTER_PCI_TABLE(dma_hisilicon, pci_id_hisi_dma_map);
+RTE_PMD_REGISTER_KMOD_DEP(dma_hisilicon, "vfio-pci");
diff --git a/drivers/dma/hisilicon/hisi_dmadev.h b/drivers/dma/hisilicon/hisi_dmadev.h
new file mode 100644
index 0000000000..114b9dcb5b
--- /dev/null
+++ b/drivers/dma/hisilicon/hisi_dmadev.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ */
+
+#ifndef HISI_DMADEV_H
+#define HISI_DMADEV_H
+
+#define PCI_VENDOR_ID_HUAWEI 0x19e5
+#define HISI_DMA_DEVICE_ID 0xA122
+#define HISI_DMA_PCI_REVISION_ID_REG 0x08
+#define HISI_DMA_REVISION_HIP08B 0x21
+
+/**
+ * The HIP08B(HiSilicon IP08) and later Chip(e.g. HiSilicon IP09) are DMA iEPs,
+ * they have the same pci device id but with different pci revision.
+ * Unfortunately, they have different register layouts, so the layout
+ * enumerations are defined.
+ */
+enum {
+ HISI_DMA_REG_LAYOUT_INVALID = 0,
+ HISI_DMA_REG_LAYOUT_HIP08
+};
+
+#endif /* HISI_DMADEV_H */
diff --git a/drivers/dma/hisilicon/meson.build b/drivers/dma/hisilicon/meson.build
new file mode 100644
index 0000000000..9e31b47141
--- /dev/null
+++ b/drivers/dma/hisilicon/meson.build
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited
+
+deps += ['bus_pci', 'dmadev']
+sources = files(
+ 'hisi_dmadev.c',
+)
diff --git a/drivers/dma/hisilicon/version.map b/drivers/dma/hisilicon/version.map
new file mode 100644
index 0000000000..c2e0723b4c
--- /dev/null
+++ b/drivers/dma/hisilicon/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build
index a69418ce9b..6d6c054f51 100644
--- a/drivers/dma/meson.build
+++ b/drivers/dma/meson.build
@@ -4,6 +4,7 @@
drivers = [
'idxd',
'ioat',
+ 'hisilicon',
'skeleton',
]
std_deps = ['dmadev']
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH 2/6] dma/hisilicon: add dmadev instances create and destroy
2021-10-30 10:36 [dpdk-dev] [PATCH 0/6] dma: add hisilicon DMA driver Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 1/6] dma/hisilicon: add device probe and remove functions Chengwen Feng
@ 2021-10-30 10:36 ` Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 3/6] dma/hisilicon: add control path functions Chengwen Feng
` (4 subsequent siblings)
6 siblings, 0 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-10-30 10:36 UTC (permalink / raw)
To: thomas; +Cc: dev
This patch add dmadev instances create during the PCI probe, and
destroy them during the PCI remove. Internal structures and HW
definitions was also included.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
doc/guides/dmadevs/hisilicon.rst | 10 ++
drivers/dma/hisilicon/hisi_dmadev.c | 212 +++++++++++++++++++++++++++-
drivers/dma/hisilicon/hisi_dmadev.h | 97 +++++++++++++
3 files changed, 318 insertions(+), 1 deletion(-)
diff --git a/doc/guides/dmadevs/hisilicon.rst b/doc/guides/dmadevs/hisilicon.rst
index 4cbaac4204..65138a8365 100644
--- a/doc/guides/dmadevs/hisilicon.rst
+++ b/doc/guides/dmadevs/hisilicon.rst
@@ -19,3 +19,13 @@ Device Setup
Kunpeng DMA devices will need to be bound to a suitable DPDK-supported
user-space IO driver such as ``vfio-pci`` in order to be used by DPDK.
+
+Device Probing and Initialization
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Once probed successfully, the device will appear as four ``dmadev`` which can be
+accessed using API from the ``rte_dmadev`` library.
+
+The name of the ``dmadev`` created is like "B:D.F-chX", e.g. DMA 0000:7b:00.0
+will create four ``dmadev``, the 1st ``dmadev`` name is "7b:00.0-ch0", and the
+2nd ``dmadev`` name is "7b:00.0-ch1".
diff --git a/drivers/dma/hisilicon/hisi_dmadev.c b/drivers/dma/hisilicon/hisi_dmadev.c
index e6fb8a0fc8..b8369e7e71 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.c
+++ b/drivers/dma/hisilicon/hisi_dmadev.c
@@ -6,7 +6,9 @@
#include <string.h>
#include <rte_bus_pci.h>
+#include <rte_cycles.h>
#include <rte_eal.h>
+#include <rte_io.h>
#include <rte_log.h>
#include <rte_pci.h>
#include <rte_dmadev_pmd.h>
@@ -30,6 +32,141 @@ RTE_LOG_REGISTER_DEFAULT(hisi_dma_logtype, INFO);
#define HISI_DMA_ERR(hw, fmt, args...) \
HISI_DMA_LOG_RAW(hw, ERR, fmt, ## args)
+static uint32_t
+hisi_dma_queue_base(struct hisi_dma_dev *hw)
+{
+ if (hw->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ return HISI_DMA_HIP08_QUEUE_BASE;
+ else
+ return 0;
+}
+
+static void
+hisi_dma_write_reg(void *base, uint32_t off, uint32_t val)
+{
+ rte_write32(rte_cpu_to_le_32(val),
+ (volatile void *)((char *)base + off));
+}
+
+static void
+hisi_dma_write_dev(struct hisi_dma_dev *hw, uint32_t off, uint32_t val)
+{
+ hisi_dma_write_reg(hw->io_base, off, val);
+}
+
+static void
+hisi_dma_write_queue(struct hisi_dma_dev *hw, uint32_t qoff, uint32_t val)
+{
+ uint32_t off = hisi_dma_queue_base(hw) +
+ hw->queue_id * HISI_DMA_QUEUE_REGION_SIZE + qoff;
+ hisi_dma_write_dev(hw, off, val);
+}
+
+static uint32_t
+hisi_dma_read_reg(void *base, uint32_t off)
+{
+ uint32_t val = rte_read32((volatile void *)((char *)base + off));
+ return rte_le_to_cpu_32(val);
+}
+
+static uint32_t
+hisi_dma_read_dev(struct hisi_dma_dev *hw, uint32_t off)
+{
+ return hisi_dma_read_reg(hw->io_base, off);
+}
+
+static uint32_t
+hisi_dma_read_queue(struct hisi_dma_dev *hw, uint32_t qoff)
+{
+ uint32_t off = hisi_dma_queue_base(hw) +
+ hw->queue_id * HISI_DMA_QUEUE_REGION_SIZE + qoff;
+ return hisi_dma_read_dev(hw, off);
+}
+
+static void
+hisi_dma_update_bit(struct hisi_dma_dev *hw, uint32_t off, uint32_t pos,
+ bool set)
+{
+ uint32_t tmp = hisi_dma_read_dev(hw, off);
+ uint32_t mask = 1u << pos;
+ tmp = set ? tmp | mask : tmp & ~mask;
+ hisi_dma_write_dev(hw, off, tmp);
+}
+
+static void
+hisi_dma_update_queue_bit(struct hisi_dma_dev *hw, uint32_t qoff, uint32_t pos,
+ bool set)
+{
+ uint32_t tmp = hisi_dma_read_queue(hw, qoff);
+ uint32_t mask = 1u << pos;
+ tmp = set ? tmp | mask : tmp & ~mask;
+ hisi_dma_write_queue(hw, qoff, tmp);
+}
+
+#define hisi_dma_poll_hw_state(hw, val, cond, sleep_us, timeout_us) ({ \
+ uint32_t timeout = 0; \
+ while (timeout++ <= (timeout_us)) { \
+ (val) = hisi_dma_read_queue(hw, HISI_DMA_QUEUE_FSM_REG); \
+ if (cond) \
+ break; \
+ rte_delay_us(sleep_us); \
+ } \
+ (cond) ? 0 : -ETIME; \
+})
+
+static int
+hisi_dma_reset_hw(struct hisi_dma_dev *hw)
+{
+#define POLL_SLEEP_US 100
+#define POLL_TIMEOUT_US 10000
+
+ uint32_t tmp;
+ int ret;
+
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL0_REG,
+ HISI_DMA_QUEUE_CTRL0_PAUSE_B, true);
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL0_REG,
+ HISI_DMA_QUEUE_CTRL0_EN_B, false);
+
+ ret = hisi_dma_poll_hw_state(hw, tmp,
+ FIELD_GET(HISI_DMA_QUEUE_FSM_STS_M, tmp) != HISI_DMA_STATE_RUN,
+ POLL_SLEEP_US, POLL_TIMEOUT_US);
+ if (ret) {
+ HISI_DMA_ERR(hw, "disable dma timeout!");
+ return ret;
+ }
+
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL1_REG,
+ HISI_DMA_QUEUE_CTRL1_RESET_B, true);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_SQ_TAIL_REG, 0);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_CQ_HEAD_REG, 0);
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL0_REG,
+ HISI_DMA_QUEUE_CTRL0_PAUSE_B, false);
+
+ ret = hisi_dma_poll_hw_state(hw, tmp,
+ FIELD_GET(HISI_DMA_QUEUE_FSM_STS_M, tmp) == HISI_DMA_STATE_IDLE,
+ POLL_SLEEP_US, POLL_TIMEOUT_US);
+ if (ret) {
+ HISI_DMA_ERR(hw, "reset dma timeout!");
+ return ret;
+ }
+
+ return 0;
+}
+
+static void
+hisi_dma_init_gbl(void *pci_bar, uint8_t revision)
+{
+ struct hisi_dma_dev hw;
+
+ memset(&hw, 0, sizeof(hw));
+ hw.io_base = pci_bar;
+
+ if (revision == HISI_DMA_REVISION_HIP08B)
+ hisi_dma_update_bit(&hw, HISI_DMA_HIP08_MODE_REG,
+ HISI_DMA_HIP08_MODE_SEL_B, true);
+}
+
static uint8_t
hisi_dma_reg_layout(uint8_t revision)
{
@@ -49,6 +186,57 @@ hisi_dma_gen_pci_device_name(const struct rte_pci_device *pci_dev,
pci_dev->addr.function);
}
+static void
+hisi_dma_gen_dev_name(const struct rte_pci_device *pci_dev,
+ uint8_t queue_id, char *name, size_t size)
+{
+ memset(name, 0, size);
+ (void)snprintf(name, size, "%x:%x.%x-ch%u",
+ pci_dev->addr.bus, pci_dev->addr.devid,
+ pci_dev->addr.function, queue_id);
+}
+
+static int
+hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
+ uint8_t revision)
+{
+#define REG_PCI_BAR_INDEX 2
+
+ char name[RTE_DEV_NAME_MAX_LEN];
+ struct rte_dma_dev *dev;
+ struct hisi_dma_dev *hw;
+ int ret;
+
+ hisi_dma_gen_dev_name(pci_dev, queue_id, name, sizeof(name));
+ dev = rte_dma_pmd_allocate(name, pci_dev->device.numa_node,
+ sizeof(*hw));
+ if (dev == NULL) {
+ HISI_DMA_LOG(ERR, "%s allocate dmadev fail!", name);
+ return -EINVAL;
+ }
+
+ dev->device = &pci_dev->device;
+
+ hw = dev->data->dev_private;
+ hw->data = dev->data;
+ hw->revision = revision;
+ hw->reg_layout = hisi_dma_reg_layout(revision);
+ hw->io_base = pci_dev->mem_resource[REG_PCI_BAR_INDEX].addr;
+ hw->queue_id = queue_id;
+
+ ret = hisi_dma_reset_hw(hw);
+ if (ret) {
+ HISI_DMA_LOG(ERR, "%s init device fail!", name);
+ (void)rte_dma_pmd_release(name);
+ return -EIO;
+ }
+
+ dev->state = RTE_DMA_DEV_READY;
+ HISI_DMA_LOG(DEBUG, "%s create dmadev success!", name);
+
+ return 0;
+}
+
static int
hisi_dma_check_revision(struct rte_pci_device *pci_dev, const char *name,
uint8_t *out_revision)
@@ -78,6 +266,7 @@ hisi_dma_probe(struct rte_pci_driver *pci_drv __rte_unused,
{
char name[RTE_DEV_NAME_MAX_LEN] = { 0 };
uint8_t revision;
+ uint8_t i;
int ret;
hisi_dma_gen_pci_device_name(pci_dev, name, sizeof(name));
@@ -92,13 +281,34 @@ hisi_dma_probe(struct rte_pci_driver *pci_drv __rte_unused,
return ret;
HISI_DMA_LOG(DEBUG, "%s read PCI revision: 0x%x", name, revision);
+ hisi_dma_init_gbl(pci_dev->mem_resource[2].addr, revision);
+
+ for (i = 0; i < HISI_DMA_MAX_HW_QUEUES; i++) {
+ ret = hisi_dma_create(pci_dev, i, revision);
+ if (ret) {
+ HISI_DMA_LOG(ERR, "%s create dmadev %u failed!",
+ name, i);
+ break;
+ }
+ }
+
return ret;
}
static int
hisi_dma_remove(struct rte_pci_device *pci_dev)
{
- RTE_SET_USED(pci_dev);
+ char name[RTE_DEV_NAME_MAX_LEN];
+ uint8_t i;
+ int ret;
+
+ for (i = 0; i < HISI_DMA_MAX_HW_QUEUES; i++) {
+ hisi_dma_gen_dev_name(pci_dev, i, name, sizeof(name));
+ ret = rte_dma_pmd_release(name);
+ if (ret)
+ return ret;
+ }
+
return 0;
}
diff --git a/drivers/dma/hisilicon/hisi_dmadev.h b/drivers/dma/hisilicon/hisi_dmadev.h
index 114b9dcb5b..50aaa38b72 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.h
+++ b/drivers/dma/hisilicon/hisi_dmadev.h
@@ -5,11 +5,24 @@
#ifndef HISI_DMADEV_H
#define HISI_DMADEV_H
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#define BIT(x) (1ul << (x))
+#define BITS_PER_LONG (__SIZEOF_LONG__ * 8)
+#define GENMASK(h, l) \
+ (((~0UL) << (l)) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
+#define BF_SHF(x) (__builtin_ffsll(x) - 1)
+#define FIELD_GET(mask, reg) \
+ ((typeof(mask))(((reg) & (mask)) >> BF_SHF(mask)))
+
#define PCI_VENDOR_ID_HUAWEI 0x19e5
#define HISI_DMA_DEVICE_ID 0xA122
#define HISI_DMA_PCI_REVISION_ID_REG 0x08
#define HISI_DMA_REVISION_HIP08B 0x21
+#define HISI_DMA_MAX_HW_QUEUES 4
+
/**
* The HIP08B(HiSilicon IP08) and later Chip(e.g. HiSilicon IP09) are DMA iEPs,
* they have the same pci device id but with different pci revision.
@@ -21,4 +34,88 @@ enum {
HISI_DMA_REG_LAYOUT_HIP08
};
+/**
+ * Hardware PCI bar register MAP:
+ *
+ * --------------
+ * | Misc-reg-0 |
+ * | |
+ * -------------- -> Queue base
+ * | |
+ * | Queue-0 |
+ * | |
+ * -------------- ---
+ * | | ^
+ * | Queue-1 | Queue region
+ * | | v
+ * -------------- ---
+ * | ... |
+ * | Queue-x |
+ * | ... |
+ * --------------
+ * | Misc-reg-1 |
+ * --------------
+ *
+ * As described above, a single queue register is continuous and occupies the
+ * length of queue-region. The global offset for a single queue register is
+ * calculated by:
+ * offset = queue-base + (queue-id * queue-region) + reg-offset-in-region.
+ *
+ * The first part of queue region is basically the same for HIP08 and later chip
+ * register layouts, therefore, HISI_QUEUE_* registers are defined for it.
+ */
+#define HISI_DMA_QUEUE_SQ_BASE_L_REG 0x0
+#define HISI_DMA_QUEUE_SQ_BASE_H_REG 0x4
+#define HISI_DMA_QUEUE_SQ_DEPTH_REG 0x8
+#define HISI_DMA_QUEUE_SQ_TAIL_REG 0xC
+#define HISI_DMA_QUEUE_CQ_BASE_L_REG 0x10
+#define HISI_DMA_QUEUE_CQ_BASE_H_REG 0x14
+#define HISI_DMA_QUEUE_CQ_DEPTH_REG 0x18
+#define HISI_DMA_QUEUE_CQ_HEAD_REG 0x1C
+#define HISI_DMA_QUEUE_CTRL0_REG 0x20
+#define HISI_DMA_QUEUE_CTRL0_EN_B 0
+#define HISI_DMA_QUEUE_CTRL0_PAUSE_B 4
+#define HISI_DMA_QUEUE_CTRL1_REG 0x24
+#define HISI_DMA_QUEUE_CTRL1_RESET_B 0
+#define HISI_DMA_QUEUE_FSM_REG 0x30
+#define HISI_DMA_QUEUE_FSM_STS_M GENMASK(3, 0)
+#define HISI_DMA_QUEUE_INT_STATUS_REG 0x40
+#define HISI_DMA_QUEUE_ERR_INT_NUM0_REG 0x84
+#define HISI_DMA_QUEUE_ERR_INT_NUM1_REG 0x88
+#define HISI_DMA_QUEUE_ERR_INT_NUM2_REG 0x8C
+#define HISI_DMA_QUEUE_REGION_SIZE 0x100
+
+/**
+ * HiSilicon IP08 DMA register and field define:
+ */
+#define HISI_DMA_HIP08_QUEUE_BASE 0x0
+#define HISI_DMA_HIP08_QUEUE_CTRL0_ERR_ABORT_B 2
+#define HISI_DMA_HIP08_QUEUE_INT_MASK_REG 0x44
+#define HISI_DMA_HIP08_QUEUE_INT_MASK_M GENMASK(14, 0)
+#define HISI_DMA_HIP08_QUEUE_ERR_INT_NUM3_REG 0x90
+#define HISI_DMA_HIP08_QUEUE_ERR_INT_NUM4_REG 0x94
+#define HISI_DMA_HIP08_QUEUE_ERR_INT_NUM5_REG 0x98
+#define HISI_DMA_HIP08_QUEUE_ERR_INT_NUM6_REG 0x48
+#define HISI_DMA_HIP08_MODE_REG 0x217C
+#define HISI_DMA_HIP08_MODE_SEL_B 0
+#define HISI_DMA_HIP08_DUMP_START_REG 0x2000
+#define HISI_DMA_HIP08_DUMP_END_REG 0x2280
+
+/**
+ * In fact, there are multiple states, but it need to pay attention to
+ * the following two states for the driver:
+ */
+enum {
+ HISI_DMA_STATE_IDLE = 0,
+ HISI_DMA_STATE_RUN,
+};
+
+struct hisi_dma_dev {
+ struct rte_dma_dev_data *data;
+ uint8_t revision; /**< PCI revision. */
+ uint8_t reg_layout; /**< hardware register layout. */
+ void *io_base;
+ uint8_t queue_id; /**< hardware DMA queue index. */
+};
+
#endif /* HISI_DMADEV_H */
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH 3/6] dma/hisilicon: add control path functions
2021-10-30 10:36 [dpdk-dev] [PATCH 0/6] dma: add hisilicon DMA driver Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 1/6] dma/hisilicon: add device probe and remove functions Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 2/6] dma/hisilicon: add dmadev instances create and destroy Chengwen Feng
@ 2021-10-30 10:36 ` Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 4/6] dma/hisilicon: add data " Chengwen Feng
` (3 subsequent siblings)
6 siblings, 0 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-10-30 10:36 UTC (permalink / raw)
To: thomas; +Cc: dev
This patch add control path functions for Kunpeng DMA devices.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
doc/guides/dmadevs/hisilicon.rst | 10 +
drivers/dma/hisilicon/hisi_dmadev.c | 385 ++++++++++++++++++++++++++++
drivers/dma/hisilicon/hisi_dmadev.h | 99 +++++++
3 files changed, 494 insertions(+)
diff --git a/doc/guides/dmadevs/hisilicon.rst b/doc/guides/dmadevs/hisilicon.rst
index 65138a8365..24bae86bdc 100644
--- a/doc/guides/dmadevs/hisilicon.rst
+++ b/doc/guides/dmadevs/hisilicon.rst
@@ -29,3 +29,13 @@ accessed using API from the ``rte_dmadev`` library.
The name of the ``dmadev`` created is like "B:D.F-chX", e.g. DMA 0000:7b:00.0
will create four ``dmadev``, the 1st ``dmadev`` name is "7b:00.0-ch0", and the
2nd ``dmadev`` name is "7b:00.0-ch1".
+
+Device Configuration
+~~~~~~~~~~~~~~~~~~~~~
+
+Kunpeng DMA configuration requirements:
+
+* ``ring_size`` must be a power of two, between 32 and 8192.
+* Only one ``vchan`` is supported per ``dmadev``.
+* Silent mode is not supported.
+* The transfer direction must be set to ``RTE_DMA_DIR_MEM_TO_MEM``.
diff --git a/drivers/dma/hisilicon/hisi_dmadev.c b/drivers/dma/hisilicon/hisi_dmadev.c
index b8369e7e71..bcdcf4de4b 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.c
+++ b/drivers/dma/hisilicon/hisi_dmadev.c
@@ -10,6 +10,8 @@
#include <rte_eal.h>
#include <rte_io.h>
#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
#include <rte_pci.h>
#include <rte_dmadev_pmd.h>
@@ -41,6 +43,14 @@ hisi_dma_queue_base(struct hisi_dma_dev *hw)
return 0;
}
+static volatile void *
+hisi_dma_queue_regaddr(struct hisi_dma_dev *hw, uint32_t qoff)
+{
+ uint32_t off = hisi_dma_queue_base(hw) +
+ hw->queue_id * HISI_DMA_QUEUE_REGION_SIZE + qoff;
+ return (volatile void *)((char *)hw->io_base + off);
+}
+
static void
hisi_dma_write_reg(void *base, uint32_t off, uint32_t val)
{
@@ -103,6 +113,15 @@ hisi_dma_update_queue_bit(struct hisi_dma_dev *hw, uint32_t qoff, uint32_t pos,
hisi_dma_write_queue(hw, qoff, tmp);
}
+static void
+hisi_dma_update_queue_mbit(struct hisi_dma_dev *hw, uint32_t qoff,
+ uint32_t mask, bool set)
+{
+ uint32_t tmp = hisi_dma_read_queue(hw, qoff);
+ tmp = set ? tmp | mask : tmp & ~mask;
+ hisi_dma_write_queue(hw, qoff, tmp);
+}
+
#define hisi_dma_poll_hw_state(hw, val, cond, sleep_us, timeout_us) ({ \
uint32_t timeout = 0; \
while (timeout++ <= (timeout_us)) { \
@@ -154,6 +173,45 @@ hisi_dma_reset_hw(struct hisi_dma_dev *hw)
return 0;
}
+static void
+hisi_dma_init_hw(struct hisi_dma_dev *hw)
+{
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_SQ_BASE_L_REG,
+ lower_32_bits(hw->sqe_iova));
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_SQ_BASE_H_REG,
+ upper_32_bits(hw->sqe_iova));
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_CQ_BASE_L_REG,
+ lower_32_bits(hw->cqe_iova));
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_CQ_BASE_H_REG,
+ upper_32_bits(hw->cqe_iova));
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_SQ_DEPTH_REG,
+ hw->sq_depth_mask);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_CQ_DEPTH_REG, hw->cq_depth - 1);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_SQ_TAIL_REG, 0);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_CQ_HEAD_REG, 0);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_ERR_INT_NUM0_REG, 0);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_ERR_INT_NUM1_REG, 0);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_ERR_INT_NUM2_REG, 0);
+
+ if (hw->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_write_queue(hw, HISI_DMA_HIP08_QUEUE_ERR_INT_NUM3_REG,
+ 0);
+ hisi_dma_write_queue(hw, HISI_DMA_HIP08_QUEUE_ERR_INT_NUM4_REG,
+ 0);
+ hisi_dma_write_queue(hw, HISI_DMA_HIP08_QUEUE_ERR_INT_NUM5_REG,
+ 0);
+ hisi_dma_write_queue(hw, HISI_DMA_HIP08_QUEUE_ERR_INT_NUM6_REG,
+ 0);
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL0_REG,
+ HISI_DMA_HIP08_QUEUE_CTRL0_ERR_ABORT_B, false);
+ hisi_dma_update_queue_mbit(hw, HISI_DMA_QUEUE_INT_STATUS_REG,
+ HISI_DMA_HIP08_QUEUE_INT_MASK_M, true);
+ hisi_dma_update_queue_mbit(hw,
+ HISI_DMA_HIP08_QUEUE_INT_MASK_REG,
+ HISI_DMA_HIP08_QUEUE_INT_MASK_M, true);
+ }
+}
+
static void
hisi_dma_init_gbl(void *pci_bar, uint8_t revision)
{
@@ -176,6 +234,301 @@ hisi_dma_reg_layout(uint8_t revision)
return HISI_DMA_REG_LAYOUT_INVALID;
}
+static void
+hisi_dma_zero_iomem(struct hisi_dma_dev *hw)
+{
+ memset(hw->iomz->addr, 0, hw->iomz_sz);
+}
+
+static int
+hisi_dma_alloc_iomem(struct hisi_dma_dev *hw, uint16_t ring_size,
+ const char *dev_name)
+{
+ uint32_t sq_size = sizeof(struct hisi_dma_sqe) * ring_size;
+ uint32_t cq_size = sizeof(struct hisi_dma_cqe) *
+ (ring_size + HISI_DMA_CQ_RESERVED);
+ uint32_t status_size = sizeof(uint16_t) * ring_size;
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ const struct rte_memzone *iomz;
+ uint32_t total_size;
+
+ sq_size = RTE_CACHE_LINE_ROUNDUP(sq_size);
+ cq_size = RTE_CACHE_LINE_ROUNDUP(cq_size);
+ status_size = RTE_CACHE_LINE_ROUNDUP(status_size);
+ total_size = sq_size + cq_size + status_size;
+
+ (void)snprintf(mz_name, sizeof(mz_name), "hisi_dma:%s", dev_name);
+ iomz = rte_memzone_reserve(mz_name, total_size, hw->data->numa_node,
+ RTE_MEMZONE_IOVA_CONTIG);
+ if (iomz == NULL) {
+ HISI_DMA_ERR(hw, "malloc %s iomem fail!", mz_name);
+ return -ENOMEM;
+ }
+
+ hw->iomz = iomz;
+ hw->iomz_sz = total_size;
+ hw->sqe = iomz->addr;
+ hw->cqe = (void *)((char *)iomz->addr + sq_size);
+ hw->status = (void *)((char *)iomz->addr + sq_size + cq_size);
+ hw->sqe_iova = iomz->iova;
+ hw->cqe_iova = iomz->iova + sq_size;
+ hw->sq_depth_mask = ring_size - 1;
+ hw->cq_depth = ring_size + HISI_DMA_CQ_RESERVED;
+ hisi_dma_zero_iomem(hw);
+
+ return 0;
+}
+
+static void
+hisi_dma_free_iomem(struct hisi_dma_dev *hw)
+{
+ if (hw->iomz != NULL)
+ rte_memzone_free(hw->iomz);
+
+ hw->iomz = NULL;
+ hw->sqe = NULL;
+ hw->cqe = NULL;
+ hw->status = NULL;
+ hw->sqe_iova = 0;
+ hw->cqe_iova = 0;
+ hw->sq_depth_mask = 0;
+ hw->cq_depth = 0;
+}
+
+static int
+hisi_dma_info_get(const struct rte_dma_dev *dev,
+ struct rte_dma_info *dev_info,
+ uint32_t info_sz)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(info_sz);
+
+ dev_info->dev_capa = RTE_DMA_CAPA_MEM_TO_MEM |
+ RTE_DMA_CAPA_OPS_COPY;
+ dev_info->max_vchans = 1;
+ dev_info->max_desc = HISI_DMA_MAX_DESC_NUM;
+ dev_info->min_desc = HISI_DMA_MIN_DESC_NUM;
+
+ return 0;
+}
+
+static int
+hisi_dma_configure(struct rte_dma_dev *dev,
+ const struct rte_dma_conf *conf,
+ uint32_t conf_sz)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(conf);
+ RTE_SET_USED(conf_sz);
+ return 0;
+}
+
+static int
+hisi_dma_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan,
+ const struct rte_dma_vchan_conf *conf,
+ uint32_t conf_sz)
+{
+ struct hisi_dma_dev *hw = dev->data->dev_private;
+ int ret;
+
+ RTE_SET_USED(vchan);
+ RTE_SET_USED(conf_sz);
+
+ if (!rte_is_power_of_2(conf->nb_desc)) {
+ HISI_DMA_ERR(hw, "Number of desc must be power of 2!");
+ return -EINVAL;
+ }
+
+ hisi_dma_free_iomem(hw);
+ ret = hisi_dma_alloc_iomem(hw, conf->nb_desc, dev->data->dev_name);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int
+hisi_dma_start(struct rte_dma_dev *dev)
+{
+ struct hisi_dma_dev *hw = dev->data->dev_private;
+
+ if (hw->iomz == NULL) {
+ HISI_DMA_ERR(hw, "Vchan was not setup, start fail!\n");
+ return -EINVAL;
+ }
+
+ /* Reset the dmadev to a known state, include:
+ * 1) zero iomem, also include status fields.
+ * 2) init hardware register.
+ * 3) init index values to zero.
+ * 4) init running statistics.
+ */
+ hisi_dma_zero_iomem(hw);
+ hisi_dma_init_hw(hw);
+ hw->ridx = 0;
+ hw->cridx = 0;
+ hw->sq_head = 0;
+ hw->sq_tail = 0;
+ hw->cq_sq_head = 0;
+ hw->cq_head = 0;
+ hw->cqs_completed = 0;
+ hw->cqe_vld = 1;
+ hw->submitted = 0;
+ hw->completed = 0;
+ hw->errors = 0;
+
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL0_REG,
+ HISI_DMA_QUEUE_CTRL0_EN_B, true);
+
+ return 0;
+}
+
+static int
+hisi_dma_stop(struct rte_dma_dev *dev)
+{
+ return hisi_dma_reset_hw(dev->data->dev_private);
+}
+
+static int
+hisi_dma_close(struct rte_dma_dev *dev)
+{
+ /* The dmadev already stopped */
+ hisi_dma_free_iomem(dev->data->dev_private);
+ return 0;
+}
+
+static int
+hisi_dma_stats_get(const struct rte_dma_dev *dev, uint16_t vchan,
+ struct rte_dma_stats *stats,
+ uint32_t stats_sz)
+{
+ struct hisi_dma_dev *hw = dev->data->dev_private;
+
+ RTE_SET_USED(vchan);
+ RTE_SET_USED(stats_sz);
+ stats->submitted = hw->submitted;
+ stats->completed = hw->completed;
+ stats->errors = hw->errors;
+
+ return 0;
+}
+
+static int
+hisi_dma_stats_reset(struct rte_dma_dev *dev, uint16_t vchan)
+{
+ struct hisi_dma_dev *hw = dev->data->dev_private;
+
+ RTE_SET_USED(vchan);
+ hw->submitted = 0;
+ hw->completed = 0;
+ hw->errors = 0;
+
+ return 0;
+}
+
+static void
+hisi_dma_get_dump_range(struct hisi_dma_dev *hw, uint32_t *start, uint32_t *end)
+{
+ if (hw->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ *start = HISI_DMA_HIP08_DUMP_START_REG;
+ *end = HISI_DMA_HIP08_DUMP_END_REG;
+ } else {
+ *start = 0;
+ *end = 0;
+ }
+}
+
+static void
+hisi_dma_dump_common(struct hisi_dma_dev *hw, FILE *f)
+{
+#define DUMP_REGNUM_PER_LINE 4
+
+ uint32_t start, end;
+ uint32_t cnt, i;
+
+ hisi_dma_get_dump_range(hw, &start, &end);
+
+ (void)fprintf(f, " common-register:\n");
+
+ cnt = 0;
+ for (i = start; i <= end; i += sizeof(uint32_t)) {
+ if (cnt % DUMP_REGNUM_PER_LINE == 0)
+ (void)fprintf(f, " [%4x]:", i);
+ (void)fprintf(f, " 0x%08x", hisi_dma_read_dev(hw, i));
+ cnt++;
+ if (cnt % DUMP_REGNUM_PER_LINE == 0)
+ (void)fprintf(f, "\n");
+ }
+ if (cnt % DUMP_REGNUM_PER_LINE)
+ (void)fprintf(f, "\n");
+}
+
+static void
+hisi_dma_dump_read_queue(struct hisi_dma_dev *hw, uint32_t qoff,
+ char *buffer, int max_sz)
+{
+ memset(buffer, 0, max_sz);
+
+ /* Address-related registers are not printed for security reasons. */
+ if (qoff == HISI_DMA_QUEUE_SQ_BASE_L_REG ||
+ qoff == HISI_DMA_QUEUE_SQ_BASE_H_REG ||
+ qoff == HISI_DMA_QUEUE_CQ_BASE_L_REG ||
+ qoff == HISI_DMA_QUEUE_CQ_BASE_H_REG) {
+ (void)snprintf(buffer, max_sz, "**********");
+ return;
+ }
+
+ (void)snprintf(buffer, max_sz, "0x%08x", hisi_dma_read_queue(hw, qoff));
+}
+
+static void
+hisi_dma_dump_queue(struct hisi_dma_dev *hw, FILE *f)
+{
+#define REG_FMT_LEN 32
+ char buf[REG_FMT_LEN] = { 0 };
+ uint32_t i;
+
+ (void)fprintf(f, " queue-register:\n");
+ for (i = 0; i < HISI_DMA_QUEUE_REGION_SIZE; ) {
+ hisi_dma_dump_read_queue(hw, i, buf, sizeof(buf));
+ (void)fprintf(f, " [%2x]: %s", i, buf);
+ i += sizeof(uint32_t);
+ hisi_dma_dump_read_queue(hw, i, buf, sizeof(buf));
+ (void)fprintf(f, " %s", buf);
+ i += sizeof(uint32_t);
+ hisi_dma_dump_read_queue(hw, i, buf, sizeof(buf));
+ (void)fprintf(f, " %s", buf);
+ i += sizeof(uint32_t);
+ hisi_dma_dump_read_queue(hw, i, buf, sizeof(buf));
+ (void)fprintf(f, " %s\n", buf);
+ i += sizeof(uint32_t);
+ }
+}
+
+static int
+hisi_dma_dump(const struct rte_dma_dev *dev, FILE *f)
+{
+ struct hisi_dma_dev *hw = dev->data->dev_private;
+
+ (void)fprintf(f,
+ " revision: 0x%x queue_id: %u ring_size: %u\n"
+ " ridx: %u cridx: %u\n"
+ " sq_head: %u sq_tail: %u cq_sq_head: %u\n"
+ " cq_head: %u cqs_completed: %u cqe_vld: %u\n"
+ " submitted: %" PRIu64 " completed: %" PRIu64 " errors %"
+ PRIu64"\n",
+ hw->revision, hw->queue_id,
+ hw->sq_depth_mask > 0 ? hw->sq_depth_mask + 1 : 0,
+ hw->ridx, hw->cridx,
+ hw->sq_head, hw->sq_tail, hw->cq_sq_head,
+ hw->cq_head, hw->cqs_completed, hw->cqe_vld,
+ hw->submitted, hw->completed, hw->errors);
+ hisi_dma_dump_queue(hw, f);
+ hisi_dma_dump_common(hw, f);
+
+ return 0;
+}
+
static void
hisi_dma_gen_pci_device_name(const struct rte_pci_device *pci_dev,
char *name, size_t size)
@@ -196,6 +549,33 @@ hisi_dma_gen_dev_name(const struct rte_pci_device *pci_dev,
pci_dev->addr.function, queue_id);
}
+/**
+ * Hardware queue state machine:
+ *
+ * ----------- dmadev_create ------------------
+ * | Unknown | ---------------> | IDLE |
+ * ----------- ------------------
+ * ^ |
+ * | |dev_start
+ * dev_stop| |
+ * | v
+ * ------------------
+ * | RUN |
+ * ------------------
+ *
+ */
+static const struct rte_dma_dev_ops hisi_dmadev_ops = {
+ .dev_info_get = hisi_dma_info_get,
+ .dev_configure = hisi_dma_configure,
+ .dev_start = hisi_dma_start,
+ .dev_stop = hisi_dma_stop,
+ .dev_close = hisi_dma_close,
+ .vchan_setup = hisi_dma_vchan_setup,
+ .stats_get = hisi_dma_stats_get,
+ .stats_reset = hisi_dma_stats_reset,
+ .dev_dump = hisi_dma_dump,
+};
+
static int
hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
uint8_t revision)
@@ -216,6 +596,7 @@ hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
}
dev->device = &pci_dev->device;
+ dev->dev_ops = &hisi_dmadev_ops;
hw = dev->data->dev_private;
hw->data = dev->data;
@@ -223,6 +604,10 @@ hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
hw->reg_layout = hisi_dma_reg_layout(revision);
hw->io_base = pci_dev->mem_resource[REG_PCI_BAR_INDEX].addr;
hw->queue_id = queue_id;
+ hw->sq_tail_reg = hisi_dma_queue_regaddr(hw,
+ HISI_DMA_QUEUE_SQ_TAIL_REG);
+ hw->cq_head_reg = hisi_dma_queue_regaddr(hw,
+ HISI_DMA_QUEUE_CQ_HEAD_REG);
ret = hisi_dma_reset_hw(hw);
if (ret) {
diff --git a/drivers/dma/hisilicon/hisi_dmadev.h b/drivers/dma/hisilicon/hisi_dmadev.h
index 50aaa38b72..dd0315cd31 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.h
+++ b/drivers/dma/hisilicon/hisi_dmadev.h
@@ -16,12 +16,17 @@
#define FIELD_GET(mask, reg) \
((typeof(mask))(((reg) & (mask)) >> BF_SHF(mask)))
+#define lower_32_bits(x) ((uint32_t)(x))
+#define upper_32_bits(x) ((uint32_t)(((x) >> 16) >> 16))
+
#define PCI_VENDOR_ID_HUAWEI 0x19e5
#define HISI_DMA_DEVICE_ID 0xA122
#define HISI_DMA_PCI_REVISION_ID_REG 0x08
#define HISI_DMA_REVISION_HIP08B 0x21
#define HISI_DMA_MAX_HW_QUEUES 4
+#define HISI_DMA_MAX_DESC_NUM 8192
+#define HISI_DMA_MIN_DESC_NUM 32
/**
* The HIP08B(HiSilicon IP08) and later Chip(e.g. HiSilicon IP09) are DMA iEPs,
@@ -110,12 +115,106 @@ enum {
HISI_DMA_STATE_RUN,
};
+/**
+ * After scanning the CQ array, the CQ head register needs to be updated.
+ * Updating the register involves write memory barrier operations.
+ * Here use the following method to reduce WMB operations:
+ * a) malloc more CQEs, which correspond to the macro HISI_DMA_CQ_RESERVED.
+ * b) update the CQ head register after accumulated number of completed CQs
+ * is greater than or equal to HISI_DMA_CQ_RESERVED.
+ */
+#define HISI_DMA_CQ_RESERVED 64
+
+struct hisi_dma_sqe {
+ uint32_t dw0;
+#define SQE_FENCE_FLAG BIT(10)
+#define SQE_OPCODE_M2M 0x4
+ uint32_t dw1;
+ uint32_t dw2;
+ uint32_t length;
+ uint64_t src_addr;
+ uint64_t dst_addr;
+};
+
+struct hisi_dma_cqe {
+ uint64_t rsv;
+ uint64_t misc;
+#define CQE_SQ_HEAD_MASK GENMASK(15, 0)
+#define CQE_VALID_B BIT(48)
+#define CQE_STATUS_MASK GENMASK(63, 49)
+};
+
struct hisi_dma_dev {
+ struct hisi_dma_sqe *sqe;
+ volatile struct hisi_dma_cqe *cqe;
+ uint16_t *status; /* the completion status array of SQEs. */
+
+ volatile void *sq_tail_reg; /**< register address for doorbell. */
+ volatile void *cq_head_reg; /**< register address for answer CQ. */
+
+ uint16_t sq_depth_mask; /**< SQ depth - 1, the SQ depth is power of 2 */
+ uint16_t cq_depth; /* CQ depth */
+
+ uint16_t ridx; /**< ring index which will assign to the next request. */
+ /** ring index which returned by hisi_dmadev_completed APIs. */
+ uint16_t cridx;
+
+ /**
+ * SQE array management fields:
+ *
+ * -----------------------------------------------------
+ * | SQE0 | SQE1 | SQE2 | ... | SQEx | ... | SQEn-1 |
+ * -----------------------------------------------------
+ * ^ ^ ^
+ * | | |
+ * sq_head cq_sq_head sq_tail
+ *
+ * sq_head: index to the oldest completed request, this filed was
+ * updated by hisi_dmadev_completed* APIs.
+ * sq_tail: index of the next new request, this field was updated by
+ * hisi_dmadev_copy API.
+ * cq_sq_head: next index of index that has been completed by hardware,
+ * this filed was updated by hisi_dmadev_completed* APIs.
+ *
+ * [sq_head, cq_sq_head): the SQEs that hardware already completed.
+ * [cq_sq_head, sq_tail): the SQEs that hardware processing.
+ */
+ uint16_t sq_head;
+ uint16_t sq_tail;
+ uint16_t cq_sq_head;
+ /**
+ * The driver scans the CQE array, if the valid bit changes, the CQE is
+ * considered valid.
+ * Note: One CQE is corresponding to one or several SQEs, e.g. app
+ * submits two copy requests, the hardware processes the two SQEs,
+ * but it may write back only one CQE and the CQE's sq_head field
+ * indicates the index of the second copy request in the SQE
+ * array.
+ */
+ uint16_t cq_head; /**< CQ index for next scans. */
+ /** accumulated number of completed CQs
+ * @see HISI_DMA_CQ_RESERVED
+ */
+ uint16_t cqs_completed;
+ uint8_t cqe_vld; /**< valid bit for CQE, will change for every round. */
+
+ uint64_t submitted;
+ uint64_t completed;
+ uint64_t errors;
+
+ /**
+ * The following fields are not accessed in the I/O path, so they are
+ * placed at the end.
+ */
struct rte_dma_dev_data *data;
uint8_t revision; /**< PCI revision. */
uint8_t reg_layout; /**< hardware register layout. */
void *io_base;
uint8_t queue_id; /**< hardware DMA queue index. */
+ const struct rte_memzone *iomz;
+ uint32_t iomz_sz;
+ rte_iova_t sqe_iova;
+ rte_iova_t cqe_iova;
};
#endif /* HISI_DMADEV_H */
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH 4/6] dma/hisilicon: add data path functions
2021-10-30 10:36 [dpdk-dev] [PATCH 0/6] dma: add hisilicon DMA driver Chengwen Feng
` (2 preceding siblings ...)
2021-10-30 10:36 ` [dpdk-dev] [PATCH 3/6] dma/hisilicon: add control path functions Chengwen Feng
@ 2021-10-30 10:36 ` Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 5/6] dma/hisilicon: support multi-process Chengwen Feng
` (2 subsequent siblings)
6 siblings, 0 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-10-30 10:36 UTC (permalink / raw)
To: thomas; +Cc: dev
This patch add data path functions for Kunpeng DMA devices.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
drivers/dma/hisilicon/hisi_dmadev.c | 206 ++++++++++++++++++++++++++++
drivers/dma/hisilicon/hisi_dmadev.h | 16 +++
2 files changed, 222 insertions(+)
diff --git a/drivers/dma/hisilicon/hisi_dmadev.c b/drivers/dma/hisilicon/hisi_dmadev.c
index bcdcf4de4b..d03967cae3 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.c
+++ b/drivers/dma/hisilicon/hisi_dmadev.c
@@ -529,6 +529,206 @@ hisi_dma_dump(const struct rte_dma_dev *dev, FILE *f)
return 0;
}
+static int
+hisi_dma_copy(void *dev_private, uint16_t vchan,
+ rte_iova_t src, rte_iova_t dst,
+ uint32_t length, uint64_t flags)
+{
+ struct hisi_dma_dev *hw = dev_private;
+ struct hisi_dma_sqe *sqe = &hw->sqe[hw->sq_tail];
+
+ RTE_SET_USED(vchan);
+
+ if (((hw->sq_tail + 1) & hw->sq_depth_mask) == hw->sq_head)
+ return -ENOSPC;
+
+ sqe->dw0 = rte_cpu_to_le_32(SQE_OPCODE_M2M);
+ sqe->dw1 = 0;
+ sqe->dw2 = 0;
+ sqe->length = rte_cpu_to_le_32(length);
+ sqe->src_addr = rte_cpu_to_le_64(src);
+ sqe->dst_addr = rte_cpu_to_le_64(dst);
+ hw->sq_tail = (hw->sq_tail + 1) & hw->sq_depth_mask;
+ hw->submitted++;
+
+ if (flags & RTE_DMA_OP_FLAG_FENCE)
+ sqe->dw0 |= rte_cpu_to_le_32(SQE_FENCE_FLAG);
+ if (flags & RTE_DMA_OP_FLAG_SUBMIT)
+ rte_write32(rte_cpu_to_le_32(hw->sq_tail), hw->sq_tail_reg);
+
+ return hw->ridx++;
+}
+
+static int
+hisi_dma_submit(void *dev_private, uint16_t vchan)
+{
+ struct hisi_dma_dev *hw = dev_private;
+
+ RTE_SET_USED(vchan);
+ rte_write32(rte_cpu_to_le_32(hw->sq_tail), hw->sq_tail_reg);
+
+ return 0;
+}
+
+static inline void
+hisi_dma_scan_cq(struct hisi_dma_dev *hw)
+{
+ volatile struct hisi_dma_cqe *cqe;
+ uint16_t csq_head = hw->cq_sq_head;
+ uint16_t cq_head = hw->cq_head;
+ uint16_t count = 0;
+ uint64_t misc;
+
+ while (true) {
+ cqe = &hw->cqe[cq_head];
+ misc = cqe->misc;
+ misc = rte_le_to_cpu_64(misc);
+ if (FIELD_GET(CQE_VALID_B, misc) != hw->cqe_vld)
+ break;
+
+ csq_head = FIELD_GET(CQE_SQ_HEAD_MASK, misc);
+ if (unlikely(misc & CQE_STATUS_MASK))
+ hw->status[csq_head] = FIELD_GET(CQE_STATUS_MASK,
+ misc);
+
+ count++;
+ cq_head++;
+ if (cq_head == hw->cq_depth) {
+ hw->cqe_vld = !hw->cqe_vld;
+ cq_head = 0;
+ }
+ }
+
+ if (count == 0)
+ return;
+
+ hw->cq_head = cq_head;
+ hw->cq_sq_head = (csq_head + 1) & hw->sq_depth_mask;
+ hw->cqs_completed += count;
+ if (hw->cqs_completed >= HISI_DMA_CQ_RESERVED) {
+ rte_write32(rte_cpu_to_le_32(cq_head), hw->cq_head_reg);
+ hw->cqs_completed = 0;
+ }
+}
+
+static inline uint16_t
+hisi_dma_calc_cpls(struct hisi_dma_dev *hw, const uint16_t nb_cpls)
+{
+ uint16_t cpl_num;
+
+ if (hw->cq_sq_head >= hw->sq_head)
+ cpl_num = hw->cq_sq_head - hw->sq_head;
+ else
+ cpl_num = hw->sq_depth_mask + 1 - hw->sq_head + hw->cq_sq_head;
+
+ if (cpl_num > nb_cpls)
+ cpl_num = nb_cpls;
+
+ return cpl_num;
+}
+
+static uint16_t
+hisi_dma_completed(void *dev_private,
+ uint16_t vchan, const uint16_t nb_cpls,
+ uint16_t *last_idx, bool *has_error)
+{
+ struct hisi_dma_dev *hw = dev_private;
+ uint16_t sq_head = hw->sq_head;
+ uint16_t cpl_num, i;
+
+ RTE_SET_USED(vchan);
+ hisi_dma_scan_cq(hw);
+
+ cpl_num = hisi_dma_calc_cpls(hw, nb_cpls);
+ for (i = 0; i < cpl_num; i++) {
+ if (hw->status[sq_head]) {
+ *has_error = true;
+ break;
+ }
+ sq_head = (sq_head + 1) & hw->sq_depth_mask;
+ }
+ if (i > 0) {
+ hw->cridx += i;
+ *last_idx = hw->cridx - 1;
+ hw->sq_head = sq_head;
+ }
+ hw->completed += i;
+
+ return i;
+}
+
+static enum rte_dma_status_code
+hisi_dma_convert_status(uint16_t status)
+{
+ switch (status) {
+ case HISI_DMA_STATUS_SUCCESS:
+ return RTE_DMA_STATUS_SUCCESSFUL;
+ case HISI_DMA_STATUS_INVALID_OPCODE:
+ return RTE_DMA_STATUS_INVALID_OPCODE;
+ case HISI_DMA_STATUS_INVALID_LENGTH:
+ return RTE_DMA_STATUS_INVALID_LENGTH;
+ case HISI_DMA_STATUS_USER_ABORT:
+ return RTE_DMA_STATUS_USER_ABORT;
+ case HISI_DMA_STATUS_REMOTE_READ_ERROR:
+ case HISI_DMA_STATUS_AXI_READ_ERROR:
+ return RTE_DMA_STATUS_BUS_READ_ERROR;
+ case HISI_DMA_STATUS_AXI_WRITE_ERROR:
+ return RTE_DMA_STATUS_BUS_WRITE_ERROR;
+ case HISI_DMA_STATUS_DATA_POISON:
+ case HISI_DMA_STATUS_REMOTE_DATA_POISION:
+ return RTE_DMA_STATUS_DATA_POISION;
+ case HISI_DMA_STATUS_SQE_READ_ERROR:
+ case HISI_DMA_STATUS_SQE_READ_POISION:
+ return RTE_DMA_STATUS_DESCRIPTOR_READ_ERROR;
+ case HISI_DMA_STATUS_LINK_DOWN_ERROR:
+ return RTE_DMA_STATUS_DEV_LINK_ERROR;
+ default:
+ return RTE_DMA_STATUS_ERROR_UNKNOWN;
+ }
+}
+
+static uint16_t
+hisi_dma_completed_status(void *dev_private,
+ uint16_t vchan, const uint16_t nb_cpls,
+ uint16_t *last_idx, enum rte_dma_status_code *status)
+{
+ struct hisi_dma_dev *hw = dev_private;
+ uint16_t sq_head = hw->sq_head;
+ uint16_t cpl_num, i;
+
+ RTE_SET_USED(vchan);
+ hisi_dma_scan_cq(hw);
+
+ cpl_num = hisi_dma_calc_cpls(hw, nb_cpls);
+ for (i = 0; i < cpl_num; i++) {
+ status[i] = hisi_dma_convert_status(hw->status[sq_head]);
+ hw->errors += !!status[i];
+ hw->status[sq_head] = HISI_DMA_STATUS_SUCCESS;
+ sq_head = (sq_head + 1) & hw->sq_depth_mask;
+ }
+ if (likely(cpl_num > 0)) {
+ hw->cridx += cpl_num;
+ *last_idx = hw->cridx - 1;
+ hw->sq_head = sq_head;
+ }
+ hw->completed += cpl_num;
+
+ return cpl_num;
+}
+
+static uint16_t
+hisi_dma_burst_capacity(const void *dev_private, uint16_t vchan)
+{
+ const struct hisi_dma_dev *hw = dev_private;
+ uint16_t sq_head = hw->sq_head;
+ uint16_t sq_tail = hw->sq_tail;
+
+ RTE_SET_USED(vchan);
+
+ return (sq_tail >= sq_head) ? hw->sq_depth_mask - sq_tail + sq_head :
+ sq_head - 1 - sq_tail;
+}
+
static void
hisi_dma_gen_pci_device_name(const struct rte_pci_device *pci_dev,
char *name, size_t size)
@@ -597,6 +797,12 @@ hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
dev->device = &pci_dev->device;
dev->dev_ops = &hisi_dmadev_ops;
+ dev->fp_obj->dev_private = dev->data->dev_private;
+ dev->fp_obj->copy = hisi_dma_copy;
+ dev->fp_obj->submit = hisi_dma_submit;
+ dev->fp_obj->completed = hisi_dma_completed;
+ dev->fp_obj->completed_status = hisi_dma_completed_status;
+ dev->fp_obj->burst_capacity = hisi_dma_burst_capacity;
hw = dev->data->dev_private;
hw->data = dev->data;
diff --git a/drivers/dma/hisilicon/hisi_dmadev.h b/drivers/dma/hisilicon/hisi_dmadev.h
index dd0315cd31..12e209c86e 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.h
+++ b/drivers/dma/hisilicon/hisi_dmadev.h
@@ -115,6 +115,22 @@ enum {
HISI_DMA_STATE_RUN,
};
+/**
+ * Hardware complete status define:
+ */
+#define HISI_DMA_STATUS_SUCCESS 0x0
+#define HISI_DMA_STATUS_INVALID_OPCODE 0x1
+#define HISI_DMA_STATUS_INVALID_LENGTH 0x2
+#define HISI_DMA_STATUS_USER_ABORT 0x4
+#define HISI_DMA_STATUS_REMOTE_READ_ERROR 0x10
+#define HISI_DMA_STATUS_AXI_READ_ERROR 0x20
+#define HISI_DMA_STATUS_AXI_WRITE_ERROR 0x40
+#define HISI_DMA_STATUS_DATA_POISON 0x80
+#define HISI_DMA_STATUS_SQE_READ_ERROR 0x100
+#define HISI_DMA_STATUS_SQE_READ_POISION 0x200
+#define HISI_DMA_STATUS_REMOTE_DATA_POISION 0x400
+#define HISI_DMA_STATUS_LINK_DOWN_ERROR 0x800
+
/**
* After scanning the CQ array, the CQ head register needs to be updated.
* Updating the register involves write memory barrier operations.
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH 5/6] dma/hisilicon: support multi-process
2021-10-30 10:36 [dpdk-dev] [PATCH 0/6] dma: add hisilicon DMA driver Chengwen Feng
` (3 preceding siblings ...)
2021-10-30 10:36 ` [dpdk-dev] [PATCH 4/6] dma/hisilicon: add data " Chengwen Feng
@ 2021-10-30 10:36 ` Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 6/6] devbind: add Kunpeng DMA to dmadev category Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Chengwen Feng
6 siblings, 0 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-10-30 10:36 UTC (permalink / raw)
To: thomas; +Cc: dev
This patch add multi-process support for Kunpeng DMA devices.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
drivers/dma/hisilicon/hisi_dmadev.c | 21 +++++++++++++--------
1 file changed, 13 insertions(+), 8 deletions(-)
diff --git a/drivers/dma/hisilicon/hisi_dmadev.c b/drivers/dma/hisilicon/hisi_dmadev.c
index d03967cae3..05066b4d0e 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.c
+++ b/drivers/dma/hisilicon/hisi_dmadev.c
@@ -392,8 +392,10 @@ hisi_dma_stop(struct rte_dma_dev *dev)
static int
hisi_dma_close(struct rte_dma_dev *dev)
{
- /* The dmadev already stopped */
- hisi_dma_free_iomem(dev->data->dev_private);
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ /* The dmadev already stopped */
+ hisi_dma_free_iomem(dev->data->dev_private);
+ }
return 0;
}
@@ -815,11 +817,13 @@ hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
hw->cq_head_reg = hisi_dma_queue_regaddr(hw,
HISI_DMA_QUEUE_CQ_HEAD_REG);
- ret = hisi_dma_reset_hw(hw);
- if (ret) {
- HISI_DMA_LOG(ERR, "%s init device fail!", name);
- (void)rte_dma_pmd_release(name);
- return -EIO;
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ ret = hisi_dma_reset_hw(hw);
+ if (ret) {
+ HISI_DMA_LOG(ERR, "%s init device fail!", name);
+ (void)rte_dma_pmd_release(name);
+ return -EIO;
+ }
}
dev->state = RTE_DMA_DEV_READY;
@@ -872,7 +876,8 @@ hisi_dma_probe(struct rte_pci_driver *pci_drv __rte_unused,
return ret;
HISI_DMA_LOG(DEBUG, "%s read PCI revision: 0x%x", name, revision);
- hisi_dma_init_gbl(pci_dev->mem_resource[2].addr, revision);
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+ hisi_dma_init_gbl(pci_dev->mem_resource[2].addr, revision);
for (i = 0; i < HISI_DMA_MAX_HW_QUEUES; i++) {
ret = hisi_dma_create(pci_dev, i, revision);
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH 6/6] devbind: add Kunpeng DMA to dmadev category
2021-10-30 10:36 [dpdk-dev] [PATCH 0/6] dma: add hisilicon DMA driver Chengwen Feng
` (4 preceding siblings ...)
2021-10-30 10:36 ` [dpdk-dev] [PATCH 5/6] dma/hisilicon: support multi-process Chengwen Feng
@ 2021-10-30 10:36 ` Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Chengwen Feng
6 siblings, 0 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-10-30 10:36 UTC (permalink / raw)
To: thomas; +Cc: dev
add Kunpeng DMA device ID to dmadev category.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
usertools/dpdk-devbind.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index bb00f43702..a74a68ed82 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -68,10 +68,14 @@
intel_ntb_icx = {'Class': '06', 'Vendor': '8086', 'Device': '347e',
'SVendor': None, 'SDevice': None}
+hisilicon_dma = {'Class': '08', 'Vendor': '19e5', 'Device': 'a122',
+ 'SVendor': None, 'SDevice': None}
+
network_devices = [network_class, cavium_pkx, avp_vnic, ifpga_class]
baseband_devices = [acceleration_class]
crypto_devices = [encryption_class, intel_processor_class]
-dma_devices = [intel_idxd_spr, intel_ioat_bdw, intel_ioat_icx, intel_ioat_skx]
+dma_devices = [intel_idxd_spr, intel_ioat_bdw, intel_ioat_icx, intel_ioat_skx,
+ hisilicon_dma]
eventdev_devices = [cavium_sso, cavium_tim, intel_dlb, octeontx2_sso]
mempool_devices = [cavium_fpa, octeontx2_npa]
compress_devices = [cavium_zip]
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver
2021-10-30 10:36 [dpdk-dev] [PATCH 0/6] dma: add hisilicon DMA driver Chengwen Feng
` (5 preceding siblings ...)
2021-10-30 10:36 ` [dpdk-dev] [PATCH 6/6] devbind: add Kunpeng DMA to dmadev category Chengwen Feng
@ 2021-11-02 12:37 ` Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 1/6] dma/hisilicon: add device probe and remove functions Chengwen Feng
` (6 more replies)
6 siblings, 7 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-11-02 12:37 UTC (permalink / raw)
To: thomas; +Cc: dev
This patch set add hisilicon DMA driver.
Chengwen Feng (6):
dma/hisilicon: add device probe and remove functions
dma/hisilicon: add dmadev instances create and destroy
dma/hisilicon: add control path functions
dma/hisilicon: add data path functions
dma/hisilicon: support multi-process
devbind: add Kunpeng DMA to dmadev category
---
v2: fix compile error on non-Linux OS.
MAINTAINERS | 5 +
doc/guides/dmadevs/hisilicon.rst | 41 ++
doc/guides/dmadevs/index.rst | 1 +
doc/guides/rel_notes/release_21_11.rst | 4 +
drivers/dma/hisilicon/hisi_dmadev.c | 925 +++++++++++++++++++++++++
drivers/dma/hisilicon/hisi_dmadev.h | 236 +++++++
drivers/dma/hisilicon/meson.build | 19 +
drivers/dma/hisilicon/version.map | 3 +
drivers/dma/meson.build | 1 +
usertools/dpdk-devbind.py | 6 +-
10 files changed, 1240 insertions(+), 1 deletion(-)
create mode 100644 doc/guides/dmadevs/hisilicon.rst
create mode 100644 drivers/dma/hisilicon/hisi_dmadev.c
create mode 100644 drivers/dma/hisilicon/hisi_dmadev.h
create mode 100644 drivers/dma/hisilicon/meson.build
create mode 100644 drivers/dma/hisilicon/version.map
--
2.33.0
^ permalink raw reply [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v2 1/6] dma/hisilicon: add device probe and remove functions
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Chengwen Feng
@ 2021-11-02 12:37 ` Chengwen Feng
2021-11-07 17:28 ` Thomas Monjalon
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 2/6] dma/hisilicon: add dmadev instances create and destroy Chengwen Feng
` (5 subsequent siblings)
6 siblings, 1 reply; 16+ messages in thread
From: Chengwen Feng @ 2021-11-02 12:37 UTC (permalink / raw)
To: thomas; +Cc: dev
Add the basic device probe and remove functions and initial
documentation for new hisilicon DMA drivers. Maintainers update is also
included in this patch.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
MAINTAINERS | 5 ++
doc/guides/dmadevs/hisilicon.rst | 21 +++++
doc/guides/dmadevs/index.rst | 1 +
doc/guides/rel_notes/release_21_11.rst | 4 +
drivers/dma/hisilicon/hisi_dmadev.c | 119 +++++++++++++++++++++++++
drivers/dma/hisilicon/hisi_dmadev.h | 24 +++++
drivers/dma/hisilicon/meson.build | 19 ++++
drivers/dma/hisilicon/version.map | 3 +
drivers/dma/meson.build | 1 +
9 files changed, 197 insertions(+)
create mode 100644 doc/guides/dmadevs/hisilicon.rst
create mode 100644 drivers/dma/hisilicon/hisi_dmadev.c
create mode 100644 drivers/dma/hisilicon/hisi_dmadev.h
create mode 100644 drivers/dma/hisilicon/meson.build
create mode 100644 drivers/dma/hisilicon/version.map
diff --git a/MAINTAINERS b/MAINTAINERS
index 0e5951f8f1..1567f7b695 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1206,6 +1206,11 @@ M: Conor Walsh <conor.walsh@intel.com>
F: drivers/dma/ioat/
F: doc/guides/dmadevs/ioat.rst
+Hisilicon DMA
+M: Chengwen Feng <fengchengwen@huawei.com>
+F: drivers/dma/hisilicon
+F: doc/guides/dmadevs/hisilicon.rst
+
RegEx Drivers
-------------
diff --git a/doc/guides/dmadevs/hisilicon.rst b/doc/guides/dmadevs/hisilicon.rst
new file mode 100644
index 0000000000..4cbaac4204
--- /dev/null
+++ b/doc/guides/dmadevs/hisilicon.rst
@@ -0,0 +1,21 @@
+.. SPDX-License-Identifier: BSD-3-Clause
+ Copyright(c) 2021 HiSilicon Limited.
+
+HISILICON Kunpeng DMA Driver
+============================
+
+Kunpeng SoC has an internal DMA unit which can be used by application to
+accelerate data copies. The DMA PF function supports multiple DMA channels.
+
+
+Supported Kunpeng SoCs
+----------------------
+
+* Kunpeng 920
+
+
+Device Setup
+-------------
+
+Kunpeng DMA devices will need to be bound to a suitable DPDK-supported
+user-space IO driver such as ``vfio-pci`` in order to be used by DPDK.
diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst
index 20476039a5..6b04276524 100644
--- a/doc/guides/dmadevs/index.rst
+++ b/doc/guides/dmadevs/index.rst
@@ -13,3 +13,4 @@ an application through DMA API.
idxd
ioat
+ hisilicon
diff --git a/doc/guides/rel_notes/release_21_11.rst b/doc/guides/rel_notes/release_21_11.rst
index 502cc5ceb2..00a45475be 100644
--- a/doc/guides/rel_notes/release_21_11.rst
+++ b/doc/guides/rel_notes/release_21_11.rst
@@ -86,6 +86,10 @@ New Features
driver for Intel IOAT devices such as Crystal Beach DMA (CBDMA) on Ice Lake,
Skylake and Broadwell. This device driver can be used through the generic dmadev API.
+* **Added hisilicon dmadev driver implementation.**
+ The hisilicon dmadev driver provide device drivers for the Kunpeng's DMA devices.
+ This device driver can be used through the generic dmadev API.
+
* **Added support to get all MAC addresses of a device.**
Added ``rte_eth_macaddrs_get`` to allow user to retrieve all Ethernet
diff --git a/drivers/dma/hisilicon/hisi_dmadev.c b/drivers/dma/hisilicon/hisi_dmadev.c
new file mode 100644
index 0000000000..e6fb8a0fc8
--- /dev/null
+++ b/drivers/dma/hisilicon/hisi_dmadev.c
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(C) 2021 HiSilicon Limited
+ */
+
+#include <inttypes.h>
+#include <string.h>
+
+#include <rte_bus_pci.h>
+#include <rte_eal.h>
+#include <rte_log.h>
+#include <rte_pci.h>
+#include <rte_dmadev_pmd.h>
+
+#include "hisi_dmadev.h"
+
+RTE_LOG_REGISTER_DEFAULT(hisi_dma_logtype, INFO);
+#define HISI_DMA_LOG(level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, hisi_dma_logtype, \
+ "%s(): " fmt "\n", __func__, ##args)
+#define HISI_DMA_LOG_RAW(hw, level, fmt, args...) \
+ rte_log(RTE_LOG_ ## level, hisi_dma_logtype, \
+ "%s %s(): " fmt "\n", (hw)->data->dev_name, \
+ __func__, ##args)
+#define HISI_DMA_DEBUG(hw, fmt, args...) \
+ HISI_DMA_LOG_RAW(hw, DEBUG, fmt, ## args)
+#define HISI_DMA_INFO(hw, fmt, args...) \
+ HISI_DMA_LOG_RAW(hw, INFO, fmt, ## args)
+#define HISI_DMA_WARN(hw, fmt, args...) \
+ HISI_DMA_LOG_RAW(hw, WARNING, fmt, ## args)
+#define HISI_DMA_ERR(hw, fmt, args...) \
+ HISI_DMA_LOG_RAW(hw, ERR, fmt, ## args)
+
+static uint8_t
+hisi_dma_reg_layout(uint8_t revision)
+{
+ if (revision == HISI_DMA_REVISION_HIP08B)
+ return HISI_DMA_REG_LAYOUT_HIP08;
+ else
+ return HISI_DMA_REG_LAYOUT_INVALID;
+}
+
+static void
+hisi_dma_gen_pci_device_name(const struct rte_pci_device *pci_dev,
+ char *name, size_t size)
+{
+ memset(name, 0, size);
+ (void)snprintf(name, size, "%x:%x.%x",
+ pci_dev->addr.bus, pci_dev->addr.devid,
+ pci_dev->addr.function);
+}
+
+static int
+hisi_dma_check_revision(struct rte_pci_device *pci_dev, const char *name,
+ uint8_t *out_revision)
+{
+ uint8_t revision;
+ int ret;
+
+ ret = rte_pci_read_config(pci_dev, &revision, 1,
+ HISI_DMA_PCI_REVISION_ID_REG);
+ if (ret != 1) {
+ HISI_DMA_LOG(ERR, "%s read PCI revision failed!", name);
+ return -EINVAL;
+ }
+ if (hisi_dma_reg_layout(revision) == HISI_DMA_REG_LAYOUT_INVALID) {
+ HISI_DMA_LOG(ERR, "%s revision: 0x%x not supported!",
+ name, revision);
+ return -EINVAL;
+ }
+
+ *out_revision = revision;
+ return 0;
+}
+
+static int
+hisi_dma_probe(struct rte_pci_driver *pci_drv __rte_unused,
+ struct rte_pci_device *pci_dev)
+{
+ char name[RTE_DEV_NAME_MAX_LEN] = { 0 };
+ uint8_t revision;
+ int ret;
+
+ hisi_dma_gen_pci_device_name(pci_dev, name, sizeof(name));
+
+ if (pci_dev->mem_resource[2].addr == NULL) {
+ HISI_DMA_LOG(ERR, "%s BAR2 is NULL!\n", name);
+ return -ENODEV;
+ }
+
+ ret = hisi_dma_check_revision(pci_dev, name, &revision);
+ if (ret)
+ return ret;
+ HISI_DMA_LOG(DEBUG, "%s read PCI revision: 0x%x", name, revision);
+
+ return ret;
+}
+
+static int
+hisi_dma_remove(struct rte_pci_device *pci_dev)
+{
+ RTE_SET_USED(pci_dev);
+ return 0;
+}
+
+static const struct rte_pci_id pci_id_hisi_dma_map[] = {
+ { RTE_PCI_DEVICE(PCI_VENDOR_ID_HUAWEI, HISI_DMA_DEVICE_ID) },
+ { .vendor_id = 0, }, /* sentinel */
+};
+
+static struct rte_pci_driver hisi_dma_pmd_drv = {
+ .id_table = pci_id_hisi_dma_map,
+ .drv_flags = RTE_PCI_DRV_NEED_MAPPING,
+ .probe = hisi_dma_probe,
+ .remove = hisi_dma_remove,
+};
+
+RTE_PMD_REGISTER_PCI(dma_hisilicon, hisi_dma_pmd_drv);
+RTE_PMD_REGISTER_PCI_TABLE(dma_hisilicon, pci_id_hisi_dma_map);
+RTE_PMD_REGISTER_KMOD_DEP(dma_hisilicon, "vfio-pci");
diff --git a/drivers/dma/hisilicon/hisi_dmadev.h b/drivers/dma/hisilicon/hisi_dmadev.h
new file mode 100644
index 0000000000..114b9dcb5b
--- /dev/null
+++ b/drivers/dma/hisilicon/hisi_dmadev.h
@@ -0,0 +1,24 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2021 HiSilicon Limited
+ */
+
+#ifndef HISI_DMADEV_H
+#define HISI_DMADEV_H
+
+#define PCI_VENDOR_ID_HUAWEI 0x19e5
+#define HISI_DMA_DEVICE_ID 0xA122
+#define HISI_DMA_PCI_REVISION_ID_REG 0x08
+#define HISI_DMA_REVISION_HIP08B 0x21
+
+/**
+ * The HIP08B(HiSilicon IP08) and later Chip(e.g. HiSilicon IP09) are DMA iEPs,
+ * they have the same pci device id but with different pci revision.
+ * Unfortunately, they have different register layouts, so the layout
+ * enumerations are defined.
+ */
+enum {
+ HISI_DMA_REG_LAYOUT_INVALID = 0,
+ HISI_DMA_REG_LAYOUT_HIP08
+};
+
+#endif /* HISI_DMADEV_H */
diff --git a/drivers/dma/hisilicon/meson.build b/drivers/dma/hisilicon/meson.build
new file mode 100644
index 0000000000..0786772442
--- /dev/null
+++ b/drivers/dma/hisilicon/meson.build
@@ -0,0 +1,19 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2021 HiSilicon Limited
+
+if not is_linux
+ build = false
+ reason = 'only supported on Linux'
+ subdir_done()
+endif
+
+if (arch_subdir != 'x86' and arch_subdir != 'arm') or (not dpdk_conf.get('RTE_ARCH_64'))
+ build = false
+ reason = 'only supported on x86_64 and aarch64'
+ subdir_done()
+endif
+
+deps += ['bus_pci', 'dmadev']
+sources = files(
+ 'hisi_dmadev.c',
+)
diff --git a/drivers/dma/hisilicon/version.map b/drivers/dma/hisilicon/version.map
new file mode 100644
index 0000000000..c2e0723b4c
--- /dev/null
+++ b/drivers/dma/hisilicon/version.map
@@ -0,0 +1,3 @@
+DPDK_22 {
+ local: *;
+};
diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build
index a69418ce9b..6d6c054f51 100644
--- a/drivers/dma/meson.build
+++ b/drivers/dma/meson.build
@@ -4,6 +4,7 @@
drivers = [
'idxd',
'ioat',
+ 'hisilicon',
'skeleton',
]
std_deps = ['dmadev']
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v2 2/6] dma/hisilicon: add dmadev instances create and destroy
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 1/6] dma/hisilicon: add device probe and remove functions Chengwen Feng
@ 2021-11-02 12:37 ` Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 3/6] dma/hisilicon: add control path functions Chengwen Feng
` (4 subsequent siblings)
6 siblings, 0 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-11-02 12:37 UTC (permalink / raw)
To: thomas; +Cc: dev
This patch add dmadev instances create during the PCI probe, and
destroy them during the PCI remove. Internal structures and HW
definitions was also included.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
doc/guides/dmadevs/hisilicon.rst | 10 ++
drivers/dma/hisilicon/hisi_dmadev.c | 212 +++++++++++++++++++++++++++-
drivers/dma/hisilicon/hisi_dmadev.h | 97 +++++++++++++
3 files changed, 318 insertions(+), 1 deletion(-)
diff --git a/doc/guides/dmadevs/hisilicon.rst b/doc/guides/dmadevs/hisilicon.rst
index 4cbaac4204..65138a8365 100644
--- a/doc/guides/dmadevs/hisilicon.rst
+++ b/doc/guides/dmadevs/hisilicon.rst
@@ -19,3 +19,13 @@ Device Setup
Kunpeng DMA devices will need to be bound to a suitable DPDK-supported
user-space IO driver such as ``vfio-pci`` in order to be used by DPDK.
+
+Device Probing and Initialization
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Once probed successfully, the device will appear as four ``dmadev`` which can be
+accessed using API from the ``rte_dmadev`` library.
+
+The name of the ``dmadev`` created is like "B:D.F-chX", e.g. DMA 0000:7b:00.0
+will create four ``dmadev``, the 1st ``dmadev`` name is "7b:00.0-ch0", and the
+2nd ``dmadev`` name is "7b:00.0-ch1".
diff --git a/drivers/dma/hisilicon/hisi_dmadev.c b/drivers/dma/hisilicon/hisi_dmadev.c
index e6fb8a0fc8..b8369e7e71 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.c
+++ b/drivers/dma/hisilicon/hisi_dmadev.c
@@ -6,7 +6,9 @@
#include <string.h>
#include <rte_bus_pci.h>
+#include <rte_cycles.h>
#include <rte_eal.h>
+#include <rte_io.h>
#include <rte_log.h>
#include <rte_pci.h>
#include <rte_dmadev_pmd.h>
@@ -30,6 +32,141 @@ RTE_LOG_REGISTER_DEFAULT(hisi_dma_logtype, INFO);
#define HISI_DMA_ERR(hw, fmt, args...) \
HISI_DMA_LOG_RAW(hw, ERR, fmt, ## args)
+static uint32_t
+hisi_dma_queue_base(struct hisi_dma_dev *hw)
+{
+ if (hw->reg_layout == HISI_DMA_REG_LAYOUT_HIP08)
+ return HISI_DMA_HIP08_QUEUE_BASE;
+ else
+ return 0;
+}
+
+static void
+hisi_dma_write_reg(void *base, uint32_t off, uint32_t val)
+{
+ rte_write32(rte_cpu_to_le_32(val),
+ (volatile void *)((char *)base + off));
+}
+
+static void
+hisi_dma_write_dev(struct hisi_dma_dev *hw, uint32_t off, uint32_t val)
+{
+ hisi_dma_write_reg(hw->io_base, off, val);
+}
+
+static void
+hisi_dma_write_queue(struct hisi_dma_dev *hw, uint32_t qoff, uint32_t val)
+{
+ uint32_t off = hisi_dma_queue_base(hw) +
+ hw->queue_id * HISI_DMA_QUEUE_REGION_SIZE + qoff;
+ hisi_dma_write_dev(hw, off, val);
+}
+
+static uint32_t
+hisi_dma_read_reg(void *base, uint32_t off)
+{
+ uint32_t val = rte_read32((volatile void *)((char *)base + off));
+ return rte_le_to_cpu_32(val);
+}
+
+static uint32_t
+hisi_dma_read_dev(struct hisi_dma_dev *hw, uint32_t off)
+{
+ return hisi_dma_read_reg(hw->io_base, off);
+}
+
+static uint32_t
+hisi_dma_read_queue(struct hisi_dma_dev *hw, uint32_t qoff)
+{
+ uint32_t off = hisi_dma_queue_base(hw) +
+ hw->queue_id * HISI_DMA_QUEUE_REGION_SIZE + qoff;
+ return hisi_dma_read_dev(hw, off);
+}
+
+static void
+hisi_dma_update_bit(struct hisi_dma_dev *hw, uint32_t off, uint32_t pos,
+ bool set)
+{
+ uint32_t tmp = hisi_dma_read_dev(hw, off);
+ uint32_t mask = 1u << pos;
+ tmp = set ? tmp | mask : tmp & ~mask;
+ hisi_dma_write_dev(hw, off, tmp);
+}
+
+static void
+hisi_dma_update_queue_bit(struct hisi_dma_dev *hw, uint32_t qoff, uint32_t pos,
+ bool set)
+{
+ uint32_t tmp = hisi_dma_read_queue(hw, qoff);
+ uint32_t mask = 1u << pos;
+ tmp = set ? tmp | mask : tmp & ~mask;
+ hisi_dma_write_queue(hw, qoff, tmp);
+}
+
+#define hisi_dma_poll_hw_state(hw, val, cond, sleep_us, timeout_us) ({ \
+ uint32_t timeout = 0; \
+ while (timeout++ <= (timeout_us)) { \
+ (val) = hisi_dma_read_queue(hw, HISI_DMA_QUEUE_FSM_REG); \
+ if (cond) \
+ break; \
+ rte_delay_us(sleep_us); \
+ } \
+ (cond) ? 0 : -ETIME; \
+})
+
+static int
+hisi_dma_reset_hw(struct hisi_dma_dev *hw)
+{
+#define POLL_SLEEP_US 100
+#define POLL_TIMEOUT_US 10000
+
+ uint32_t tmp;
+ int ret;
+
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL0_REG,
+ HISI_DMA_QUEUE_CTRL0_PAUSE_B, true);
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL0_REG,
+ HISI_DMA_QUEUE_CTRL0_EN_B, false);
+
+ ret = hisi_dma_poll_hw_state(hw, tmp,
+ FIELD_GET(HISI_DMA_QUEUE_FSM_STS_M, tmp) != HISI_DMA_STATE_RUN,
+ POLL_SLEEP_US, POLL_TIMEOUT_US);
+ if (ret) {
+ HISI_DMA_ERR(hw, "disable dma timeout!");
+ return ret;
+ }
+
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL1_REG,
+ HISI_DMA_QUEUE_CTRL1_RESET_B, true);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_SQ_TAIL_REG, 0);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_CQ_HEAD_REG, 0);
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL0_REG,
+ HISI_DMA_QUEUE_CTRL0_PAUSE_B, false);
+
+ ret = hisi_dma_poll_hw_state(hw, tmp,
+ FIELD_GET(HISI_DMA_QUEUE_FSM_STS_M, tmp) == HISI_DMA_STATE_IDLE,
+ POLL_SLEEP_US, POLL_TIMEOUT_US);
+ if (ret) {
+ HISI_DMA_ERR(hw, "reset dma timeout!");
+ return ret;
+ }
+
+ return 0;
+}
+
+static void
+hisi_dma_init_gbl(void *pci_bar, uint8_t revision)
+{
+ struct hisi_dma_dev hw;
+
+ memset(&hw, 0, sizeof(hw));
+ hw.io_base = pci_bar;
+
+ if (revision == HISI_DMA_REVISION_HIP08B)
+ hisi_dma_update_bit(&hw, HISI_DMA_HIP08_MODE_REG,
+ HISI_DMA_HIP08_MODE_SEL_B, true);
+}
+
static uint8_t
hisi_dma_reg_layout(uint8_t revision)
{
@@ -49,6 +186,57 @@ hisi_dma_gen_pci_device_name(const struct rte_pci_device *pci_dev,
pci_dev->addr.function);
}
+static void
+hisi_dma_gen_dev_name(const struct rte_pci_device *pci_dev,
+ uint8_t queue_id, char *name, size_t size)
+{
+ memset(name, 0, size);
+ (void)snprintf(name, size, "%x:%x.%x-ch%u",
+ pci_dev->addr.bus, pci_dev->addr.devid,
+ pci_dev->addr.function, queue_id);
+}
+
+static int
+hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
+ uint8_t revision)
+{
+#define REG_PCI_BAR_INDEX 2
+
+ char name[RTE_DEV_NAME_MAX_LEN];
+ struct rte_dma_dev *dev;
+ struct hisi_dma_dev *hw;
+ int ret;
+
+ hisi_dma_gen_dev_name(pci_dev, queue_id, name, sizeof(name));
+ dev = rte_dma_pmd_allocate(name, pci_dev->device.numa_node,
+ sizeof(*hw));
+ if (dev == NULL) {
+ HISI_DMA_LOG(ERR, "%s allocate dmadev fail!", name);
+ return -EINVAL;
+ }
+
+ dev->device = &pci_dev->device;
+
+ hw = dev->data->dev_private;
+ hw->data = dev->data;
+ hw->revision = revision;
+ hw->reg_layout = hisi_dma_reg_layout(revision);
+ hw->io_base = pci_dev->mem_resource[REG_PCI_BAR_INDEX].addr;
+ hw->queue_id = queue_id;
+
+ ret = hisi_dma_reset_hw(hw);
+ if (ret) {
+ HISI_DMA_LOG(ERR, "%s init device fail!", name);
+ (void)rte_dma_pmd_release(name);
+ return -EIO;
+ }
+
+ dev->state = RTE_DMA_DEV_READY;
+ HISI_DMA_LOG(DEBUG, "%s create dmadev success!", name);
+
+ return 0;
+}
+
static int
hisi_dma_check_revision(struct rte_pci_device *pci_dev, const char *name,
uint8_t *out_revision)
@@ -78,6 +266,7 @@ hisi_dma_probe(struct rte_pci_driver *pci_drv __rte_unused,
{
char name[RTE_DEV_NAME_MAX_LEN] = { 0 };
uint8_t revision;
+ uint8_t i;
int ret;
hisi_dma_gen_pci_device_name(pci_dev, name, sizeof(name));
@@ -92,13 +281,34 @@ hisi_dma_probe(struct rte_pci_driver *pci_drv __rte_unused,
return ret;
HISI_DMA_LOG(DEBUG, "%s read PCI revision: 0x%x", name, revision);
+ hisi_dma_init_gbl(pci_dev->mem_resource[2].addr, revision);
+
+ for (i = 0; i < HISI_DMA_MAX_HW_QUEUES; i++) {
+ ret = hisi_dma_create(pci_dev, i, revision);
+ if (ret) {
+ HISI_DMA_LOG(ERR, "%s create dmadev %u failed!",
+ name, i);
+ break;
+ }
+ }
+
return ret;
}
static int
hisi_dma_remove(struct rte_pci_device *pci_dev)
{
- RTE_SET_USED(pci_dev);
+ char name[RTE_DEV_NAME_MAX_LEN];
+ uint8_t i;
+ int ret;
+
+ for (i = 0; i < HISI_DMA_MAX_HW_QUEUES; i++) {
+ hisi_dma_gen_dev_name(pci_dev, i, name, sizeof(name));
+ ret = rte_dma_pmd_release(name);
+ if (ret)
+ return ret;
+ }
+
return 0;
}
diff --git a/drivers/dma/hisilicon/hisi_dmadev.h b/drivers/dma/hisilicon/hisi_dmadev.h
index 114b9dcb5b..50aaa38b72 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.h
+++ b/drivers/dma/hisilicon/hisi_dmadev.h
@@ -5,11 +5,24 @@
#ifndef HISI_DMADEV_H
#define HISI_DMADEV_H
+#include <rte_byteorder.h>
+#include <rte_common.h>
+
+#define BIT(x) (1ul << (x))
+#define BITS_PER_LONG (__SIZEOF_LONG__ * 8)
+#define GENMASK(h, l) \
+ (((~0UL) << (l)) & (~0UL >> (BITS_PER_LONG - 1 - (h))))
+#define BF_SHF(x) (__builtin_ffsll(x) - 1)
+#define FIELD_GET(mask, reg) \
+ ((typeof(mask))(((reg) & (mask)) >> BF_SHF(mask)))
+
#define PCI_VENDOR_ID_HUAWEI 0x19e5
#define HISI_DMA_DEVICE_ID 0xA122
#define HISI_DMA_PCI_REVISION_ID_REG 0x08
#define HISI_DMA_REVISION_HIP08B 0x21
+#define HISI_DMA_MAX_HW_QUEUES 4
+
/**
* The HIP08B(HiSilicon IP08) and later Chip(e.g. HiSilicon IP09) are DMA iEPs,
* they have the same pci device id but with different pci revision.
@@ -21,4 +34,88 @@ enum {
HISI_DMA_REG_LAYOUT_HIP08
};
+/**
+ * Hardware PCI bar register MAP:
+ *
+ * --------------
+ * | Misc-reg-0 |
+ * | |
+ * -------------- -> Queue base
+ * | |
+ * | Queue-0 |
+ * | |
+ * -------------- ---
+ * | | ^
+ * | Queue-1 | Queue region
+ * | | v
+ * -------------- ---
+ * | ... |
+ * | Queue-x |
+ * | ... |
+ * --------------
+ * | Misc-reg-1 |
+ * --------------
+ *
+ * As described above, a single queue register is continuous and occupies the
+ * length of queue-region. The global offset for a single queue register is
+ * calculated by:
+ * offset = queue-base + (queue-id * queue-region) + reg-offset-in-region.
+ *
+ * The first part of queue region is basically the same for HIP08 and later chip
+ * register layouts, therefore, HISI_QUEUE_* registers are defined for it.
+ */
+#define HISI_DMA_QUEUE_SQ_BASE_L_REG 0x0
+#define HISI_DMA_QUEUE_SQ_BASE_H_REG 0x4
+#define HISI_DMA_QUEUE_SQ_DEPTH_REG 0x8
+#define HISI_DMA_QUEUE_SQ_TAIL_REG 0xC
+#define HISI_DMA_QUEUE_CQ_BASE_L_REG 0x10
+#define HISI_DMA_QUEUE_CQ_BASE_H_REG 0x14
+#define HISI_DMA_QUEUE_CQ_DEPTH_REG 0x18
+#define HISI_DMA_QUEUE_CQ_HEAD_REG 0x1C
+#define HISI_DMA_QUEUE_CTRL0_REG 0x20
+#define HISI_DMA_QUEUE_CTRL0_EN_B 0
+#define HISI_DMA_QUEUE_CTRL0_PAUSE_B 4
+#define HISI_DMA_QUEUE_CTRL1_REG 0x24
+#define HISI_DMA_QUEUE_CTRL1_RESET_B 0
+#define HISI_DMA_QUEUE_FSM_REG 0x30
+#define HISI_DMA_QUEUE_FSM_STS_M GENMASK(3, 0)
+#define HISI_DMA_QUEUE_INT_STATUS_REG 0x40
+#define HISI_DMA_QUEUE_ERR_INT_NUM0_REG 0x84
+#define HISI_DMA_QUEUE_ERR_INT_NUM1_REG 0x88
+#define HISI_DMA_QUEUE_ERR_INT_NUM2_REG 0x8C
+#define HISI_DMA_QUEUE_REGION_SIZE 0x100
+
+/**
+ * HiSilicon IP08 DMA register and field define:
+ */
+#define HISI_DMA_HIP08_QUEUE_BASE 0x0
+#define HISI_DMA_HIP08_QUEUE_CTRL0_ERR_ABORT_B 2
+#define HISI_DMA_HIP08_QUEUE_INT_MASK_REG 0x44
+#define HISI_DMA_HIP08_QUEUE_INT_MASK_M GENMASK(14, 0)
+#define HISI_DMA_HIP08_QUEUE_ERR_INT_NUM3_REG 0x90
+#define HISI_DMA_HIP08_QUEUE_ERR_INT_NUM4_REG 0x94
+#define HISI_DMA_HIP08_QUEUE_ERR_INT_NUM5_REG 0x98
+#define HISI_DMA_HIP08_QUEUE_ERR_INT_NUM6_REG 0x48
+#define HISI_DMA_HIP08_MODE_REG 0x217C
+#define HISI_DMA_HIP08_MODE_SEL_B 0
+#define HISI_DMA_HIP08_DUMP_START_REG 0x2000
+#define HISI_DMA_HIP08_DUMP_END_REG 0x2280
+
+/**
+ * In fact, there are multiple states, but it need to pay attention to
+ * the following two states for the driver:
+ */
+enum {
+ HISI_DMA_STATE_IDLE = 0,
+ HISI_DMA_STATE_RUN,
+};
+
+struct hisi_dma_dev {
+ struct rte_dma_dev_data *data;
+ uint8_t revision; /**< PCI revision. */
+ uint8_t reg_layout; /**< hardware register layout. */
+ void *io_base;
+ uint8_t queue_id; /**< hardware DMA queue index. */
+};
+
#endif /* HISI_DMADEV_H */
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v2 3/6] dma/hisilicon: add control path functions
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 1/6] dma/hisilicon: add device probe and remove functions Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 2/6] dma/hisilicon: add dmadev instances create and destroy Chengwen Feng
@ 2021-11-02 12:37 ` Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 4/6] dma/hisilicon: add data " Chengwen Feng
` (3 subsequent siblings)
6 siblings, 0 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-11-02 12:37 UTC (permalink / raw)
To: thomas; +Cc: dev
This patch add control path functions for Kunpeng DMA devices.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
doc/guides/dmadevs/hisilicon.rst | 10 +
drivers/dma/hisilicon/hisi_dmadev.c | 385 ++++++++++++++++++++++++++++
drivers/dma/hisilicon/hisi_dmadev.h | 99 +++++++
3 files changed, 494 insertions(+)
diff --git a/doc/guides/dmadevs/hisilicon.rst b/doc/guides/dmadevs/hisilicon.rst
index 65138a8365..24bae86bdc 100644
--- a/doc/guides/dmadevs/hisilicon.rst
+++ b/doc/guides/dmadevs/hisilicon.rst
@@ -29,3 +29,13 @@ accessed using API from the ``rte_dmadev`` library.
The name of the ``dmadev`` created is like "B:D.F-chX", e.g. DMA 0000:7b:00.0
will create four ``dmadev``, the 1st ``dmadev`` name is "7b:00.0-ch0", and the
2nd ``dmadev`` name is "7b:00.0-ch1".
+
+Device Configuration
+~~~~~~~~~~~~~~~~~~~~~
+
+Kunpeng DMA configuration requirements:
+
+* ``ring_size`` must be a power of two, between 32 and 8192.
+* Only one ``vchan`` is supported per ``dmadev``.
+* Silent mode is not supported.
+* The transfer direction must be set to ``RTE_DMA_DIR_MEM_TO_MEM``.
diff --git a/drivers/dma/hisilicon/hisi_dmadev.c b/drivers/dma/hisilicon/hisi_dmadev.c
index b8369e7e71..bcdcf4de4b 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.c
+++ b/drivers/dma/hisilicon/hisi_dmadev.c
@@ -10,6 +10,8 @@
#include <rte_eal.h>
#include <rte_io.h>
#include <rte_log.h>
+#include <rte_malloc.h>
+#include <rte_memzone.h>
#include <rte_pci.h>
#include <rte_dmadev_pmd.h>
@@ -41,6 +43,14 @@ hisi_dma_queue_base(struct hisi_dma_dev *hw)
return 0;
}
+static volatile void *
+hisi_dma_queue_regaddr(struct hisi_dma_dev *hw, uint32_t qoff)
+{
+ uint32_t off = hisi_dma_queue_base(hw) +
+ hw->queue_id * HISI_DMA_QUEUE_REGION_SIZE + qoff;
+ return (volatile void *)((char *)hw->io_base + off);
+}
+
static void
hisi_dma_write_reg(void *base, uint32_t off, uint32_t val)
{
@@ -103,6 +113,15 @@ hisi_dma_update_queue_bit(struct hisi_dma_dev *hw, uint32_t qoff, uint32_t pos,
hisi_dma_write_queue(hw, qoff, tmp);
}
+static void
+hisi_dma_update_queue_mbit(struct hisi_dma_dev *hw, uint32_t qoff,
+ uint32_t mask, bool set)
+{
+ uint32_t tmp = hisi_dma_read_queue(hw, qoff);
+ tmp = set ? tmp | mask : tmp & ~mask;
+ hisi_dma_write_queue(hw, qoff, tmp);
+}
+
#define hisi_dma_poll_hw_state(hw, val, cond, sleep_us, timeout_us) ({ \
uint32_t timeout = 0; \
while (timeout++ <= (timeout_us)) { \
@@ -154,6 +173,45 @@ hisi_dma_reset_hw(struct hisi_dma_dev *hw)
return 0;
}
+static void
+hisi_dma_init_hw(struct hisi_dma_dev *hw)
+{
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_SQ_BASE_L_REG,
+ lower_32_bits(hw->sqe_iova));
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_SQ_BASE_H_REG,
+ upper_32_bits(hw->sqe_iova));
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_CQ_BASE_L_REG,
+ lower_32_bits(hw->cqe_iova));
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_CQ_BASE_H_REG,
+ upper_32_bits(hw->cqe_iova));
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_SQ_DEPTH_REG,
+ hw->sq_depth_mask);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_CQ_DEPTH_REG, hw->cq_depth - 1);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_SQ_TAIL_REG, 0);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_CQ_HEAD_REG, 0);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_ERR_INT_NUM0_REG, 0);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_ERR_INT_NUM1_REG, 0);
+ hisi_dma_write_queue(hw, HISI_DMA_QUEUE_ERR_INT_NUM2_REG, 0);
+
+ if (hw->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ hisi_dma_write_queue(hw, HISI_DMA_HIP08_QUEUE_ERR_INT_NUM3_REG,
+ 0);
+ hisi_dma_write_queue(hw, HISI_DMA_HIP08_QUEUE_ERR_INT_NUM4_REG,
+ 0);
+ hisi_dma_write_queue(hw, HISI_DMA_HIP08_QUEUE_ERR_INT_NUM5_REG,
+ 0);
+ hisi_dma_write_queue(hw, HISI_DMA_HIP08_QUEUE_ERR_INT_NUM6_REG,
+ 0);
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL0_REG,
+ HISI_DMA_HIP08_QUEUE_CTRL0_ERR_ABORT_B, false);
+ hisi_dma_update_queue_mbit(hw, HISI_DMA_QUEUE_INT_STATUS_REG,
+ HISI_DMA_HIP08_QUEUE_INT_MASK_M, true);
+ hisi_dma_update_queue_mbit(hw,
+ HISI_DMA_HIP08_QUEUE_INT_MASK_REG,
+ HISI_DMA_HIP08_QUEUE_INT_MASK_M, true);
+ }
+}
+
static void
hisi_dma_init_gbl(void *pci_bar, uint8_t revision)
{
@@ -176,6 +234,301 @@ hisi_dma_reg_layout(uint8_t revision)
return HISI_DMA_REG_LAYOUT_INVALID;
}
+static void
+hisi_dma_zero_iomem(struct hisi_dma_dev *hw)
+{
+ memset(hw->iomz->addr, 0, hw->iomz_sz);
+}
+
+static int
+hisi_dma_alloc_iomem(struct hisi_dma_dev *hw, uint16_t ring_size,
+ const char *dev_name)
+{
+ uint32_t sq_size = sizeof(struct hisi_dma_sqe) * ring_size;
+ uint32_t cq_size = sizeof(struct hisi_dma_cqe) *
+ (ring_size + HISI_DMA_CQ_RESERVED);
+ uint32_t status_size = sizeof(uint16_t) * ring_size;
+ char mz_name[RTE_MEMZONE_NAMESIZE];
+ const struct rte_memzone *iomz;
+ uint32_t total_size;
+
+ sq_size = RTE_CACHE_LINE_ROUNDUP(sq_size);
+ cq_size = RTE_CACHE_LINE_ROUNDUP(cq_size);
+ status_size = RTE_CACHE_LINE_ROUNDUP(status_size);
+ total_size = sq_size + cq_size + status_size;
+
+ (void)snprintf(mz_name, sizeof(mz_name), "hisi_dma:%s", dev_name);
+ iomz = rte_memzone_reserve(mz_name, total_size, hw->data->numa_node,
+ RTE_MEMZONE_IOVA_CONTIG);
+ if (iomz == NULL) {
+ HISI_DMA_ERR(hw, "malloc %s iomem fail!", mz_name);
+ return -ENOMEM;
+ }
+
+ hw->iomz = iomz;
+ hw->iomz_sz = total_size;
+ hw->sqe = iomz->addr;
+ hw->cqe = (void *)((char *)iomz->addr + sq_size);
+ hw->status = (void *)((char *)iomz->addr + sq_size + cq_size);
+ hw->sqe_iova = iomz->iova;
+ hw->cqe_iova = iomz->iova + sq_size;
+ hw->sq_depth_mask = ring_size - 1;
+ hw->cq_depth = ring_size + HISI_DMA_CQ_RESERVED;
+ hisi_dma_zero_iomem(hw);
+
+ return 0;
+}
+
+static void
+hisi_dma_free_iomem(struct hisi_dma_dev *hw)
+{
+ if (hw->iomz != NULL)
+ rte_memzone_free(hw->iomz);
+
+ hw->iomz = NULL;
+ hw->sqe = NULL;
+ hw->cqe = NULL;
+ hw->status = NULL;
+ hw->sqe_iova = 0;
+ hw->cqe_iova = 0;
+ hw->sq_depth_mask = 0;
+ hw->cq_depth = 0;
+}
+
+static int
+hisi_dma_info_get(const struct rte_dma_dev *dev,
+ struct rte_dma_info *dev_info,
+ uint32_t info_sz)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(info_sz);
+
+ dev_info->dev_capa = RTE_DMA_CAPA_MEM_TO_MEM |
+ RTE_DMA_CAPA_OPS_COPY;
+ dev_info->max_vchans = 1;
+ dev_info->max_desc = HISI_DMA_MAX_DESC_NUM;
+ dev_info->min_desc = HISI_DMA_MIN_DESC_NUM;
+
+ return 0;
+}
+
+static int
+hisi_dma_configure(struct rte_dma_dev *dev,
+ const struct rte_dma_conf *conf,
+ uint32_t conf_sz)
+{
+ RTE_SET_USED(dev);
+ RTE_SET_USED(conf);
+ RTE_SET_USED(conf_sz);
+ return 0;
+}
+
+static int
+hisi_dma_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan,
+ const struct rte_dma_vchan_conf *conf,
+ uint32_t conf_sz)
+{
+ struct hisi_dma_dev *hw = dev->data->dev_private;
+ int ret;
+
+ RTE_SET_USED(vchan);
+ RTE_SET_USED(conf_sz);
+
+ if (!rte_is_power_of_2(conf->nb_desc)) {
+ HISI_DMA_ERR(hw, "Number of desc must be power of 2!");
+ return -EINVAL;
+ }
+
+ hisi_dma_free_iomem(hw);
+ ret = hisi_dma_alloc_iomem(hw, conf->nb_desc, dev->data->dev_name);
+ if (ret)
+ return ret;
+
+ return 0;
+}
+
+static int
+hisi_dma_start(struct rte_dma_dev *dev)
+{
+ struct hisi_dma_dev *hw = dev->data->dev_private;
+
+ if (hw->iomz == NULL) {
+ HISI_DMA_ERR(hw, "Vchan was not setup, start fail!\n");
+ return -EINVAL;
+ }
+
+ /* Reset the dmadev to a known state, include:
+ * 1) zero iomem, also include status fields.
+ * 2) init hardware register.
+ * 3) init index values to zero.
+ * 4) init running statistics.
+ */
+ hisi_dma_zero_iomem(hw);
+ hisi_dma_init_hw(hw);
+ hw->ridx = 0;
+ hw->cridx = 0;
+ hw->sq_head = 0;
+ hw->sq_tail = 0;
+ hw->cq_sq_head = 0;
+ hw->cq_head = 0;
+ hw->cqs_completed = 0;
+ hw->cqe_vld = 1;
+ hw->submitted = 0;
+ hw->completed = 0;
+ hw->errors = 0;
+
+ hisi_dma_update_queue_bit(hw, HISI_DMA_QUEUE_CTRL0_REG,
+ HISI_DMA_QUEUE_CTRL0_EN_B, true);
+
+ return 0;
+}
+
+static int
+hisi_dma_stop(struct rte_dma_dev *dev)
+{
+ return hisi_dma_reset_hw(dev->data->dev_private);
+}
+
+static int
+hisi_dma_close(struct rte_dma_dev *dev)
+{
+ /* The dmadev already stopped */
+ hisi_dma_free_iomem(dev->data->dev_private);
+ return 0;
+}
+
+static int
+hisi_dma_stats_get(const struct rte_dma_dev *dev, uint16_t vchan,
+ struct rte_dma_stats *stats,
+ uint32_t stats_sz)
+{
+ struct hisi_dma_dev *hw = dev->data->dev_private;
+
+ RTE_SET_USED(vchan);
+ RTE_SET_USED(stats_sz);
+ stats->submitted = hw->submitted;
+ stats->completed = hw->completed;
+ stats->errors = hw->errors;
+
+ return 0;
+}
+
+static int
+hisi_dma_stats_reset(struct rte_dma_dev *dev, uint16_t vchan)
+{
+ struct hisi_dma_dev *hw = dev->data->dev_private;
+
+ RTE_SET_USED(vchan);
+ hw->submitted = 0;
+ hw->completed = 0;
+ hw->errors = 0;
+
+ return 0;
+}
+
+static void
+hisi_dma_get_dump_range(struct hisi_dma_dev *hw, uint32_t *start, uint32_t *end)
+{
+ if (hw->reg_layout == HISI_DMA_REG_LAYOUT_HIP08) {
+ *start = HISI_DMA_HIP08_DUMP_START_REG;
+ *end = HISI_DMA_HIP08_DUMP_END_REG;
+ } else {
+ *start = 0;
+ *end = 0;
+ }
+}
+
+static void
+hisi_dma_dump_common(struct hisi_dma_dev *hw, FILE *f)
+{
+#define DUMP_REGNUM_PER_LINE 4
+
+ uint32_t start, end;
+ uint32_t cnt, i;
+
+ hisi_dma_get_dump_range(hw, &start, &end);
+
+ (void)fprintf(f, " common-register:\n");
+
+ cnt = 0;
+ for (i = start; i <= end; i += sizeof(uint32_t)) {
+ if (cnt % DUMP_REGNUM_PER_LINE == 0)
+ (void)fprintf(f, " [%4x]:", i);
+ (void)fprintf(f, " 0x%08x", hisi_dma_read_dev(hw, i));
+ cnt++;
+ if (cnt % DUMP_REGNUM_PER_LINE == 0)
+ (void)fprintf(f, "\n");
+ }
+ if (cnt % DUMP_REGNUM_PER_LINE)
+ (void)fprintf(f, "\n");
+}
+
+static void
+hisi_dma_dump_read_queue(struct hisi_dma_dev *hw, uint32_t qoff,
+ char *buffer, int max_sz)
+{
+ memset(buffer, 0, max_sz);
+
+ /* Address-related registers are not printed for security reasons. */
+ if (qoff == HISI_DMA_QUEUE_SQ_BASE_L_REG ||
+ qoff == HISI_DMA_QUEUE_SQ_BASE_H_REG ||
+ qoff == HISI_DMA_QUEUE_CQ_BASE_L_REG ||
+ qoff == HISI_DMA_QUEUE_CQ_BASE_H_REG) {
+ (void)snprintf(buffer, max_sz, "**********");
+ return;
+ }
+
+ (void)snprintf(buffer, max_sz, "0x%08x", hisi_dma_read_queue(hw, qoff));
+}
+
+static void
+hisi_dma_dump_queue(struct hisi_dma_dev *hw, FILE *f)
+{
+#define REG_FMT_LEN 32
+ char buf[REG_FMT_LEN] = { 0 };
+ uint32_t i;
+
+ (void)fprintf(f, " queue-register:\n");
+ for (i = 0; i < HISI_DMA_QUEUE_REGION_SIZE; ) {
+ hisi_dma_dump_read_queue(hw, i, buf, sizeof(buf));
+ (void)fprintf(f, " [%2x]: %s", i, buf);
+ i += sizeof(uint32_t);
+ hisi_dma_dump_read_queue(hw, i, buf, sizeof(buf));
+ (void)fprintf(f, " %s", buf);
+ i += sizeof(uint32_t);
+ hisi_dma_dump_read_queue(hw, i, buf, sizeof(buf));
+ (void)fprintf(f, " %s", buf);
+ i += sizeof(uint32_t);
+ hisi_dma_dump_read_queue(hw, i, buf, sizeof(buf));
+ (void)fprintf(f, " %s\n", buf);
+ i += sizeof(uint32_t);
+ }
+}
+
+static int
+hisi_dma_dump(const struct rte_dma_dev *dev, FILE *f)
+{
+ struct hisi_dma_dev *hw = dev->data->dev_private;
+
+ (void)fprintf(f,
+ " revision: 0x%x queue_id: %u ring_size: %u\n"
+ " ridx: %u cridx: %u\n"
+ " sq_head: %u sq_tail: %u cq_sq_head: %u\n"
+ " cq_head: %u cqs_completed: %u cqe_vld: %u\n"
+ " submitted: %" PRIu64 " completed: %" PRIu64 " errors %"
+ PRIu64"\n",
+ hw->revision, hw->queue_id,
+ hw->sq_depth_mask > 0 ? hw->sq_depth_mask + 1 : 0,
+ hw->ridx, hw->cridx,
+ hw->sq_head, hw->sq_tail, hw->cq_sq_head,
+ hw->cq_head, hw->cqs_completed, hw->cqe_vld,
+ hw->submitted, hw->completed, hw->errors);
+ hisi_dma_dump_queue(hw, f);
+ hisi_dma_dump_common(hw, f);
+
+ return 0;
+}
+
static void
hisi_dma_gen_pci_device_name(const struct rte_pci_device *pci_dev,
char *name, size_t size)
@@ -196,6 +549,33 @@ hisi_dma_gen_dev_name(const struct rte_pci_device *pci_dev,
pci_dev->addr.function, queue_id);
}
+/**
+ * Hardware queue state machine:
+ *
+ * ----------- dmadev_create ------------------
+ * | Unknown | ---------------> | IDLE |
+ * ----------- ------------------
+ * ^ |
+ * | |dev_start
+ * dev_stop| |
+ * | v
+ * ------------------
+ * | RUN |
+ * ------------------
+ *
+ */
+static const struct rte_dma_dev_ops hisi_dmadev_ops = {
+ .dev_info_get = hisi_dma_info_get,
+ .dev_configure = hisi_dma_configure,
+ .dev_start = hisi_dma_start,
+ .dev_stop = hisi_dma_stop,
+ .dev_close = hisi_dma_close,
+ .vchan_setup = hisi_dma_vchan_setup,
+ .stats_get = hisi_dma_stats_get,
+ .stats_reset = hisi_dma_stats_reset,
+ .dev_dump = hisi_dma_dump,
+};
+
static int
hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
uint8_t revision)
@@ -216,6 +596,7 @@ hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
}
dev->device = &pci_dev->device;
+ dev->dev_ops = &hisi_dmadev_ops;
hw = dev->data->dev_private;
hw->data = dev->data;
@@ -223,6 +604,10 @@ hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
hw->reg_layout = hisi_dma_reg_layout(revision);
hw->io_base = pci_dev->mem_resource[REG_PCI_BAR_INDEX].addr;
hw->queue_id = queue_id;
+ hw->sq_tail_reg = hisi_dma_queue_regaddr(hw,
+ HISI_DMA_QUEUE_SQ_TAIL_REG);
+ hw->cq_head_reg = hisi_dma_queue_regaddr(hw,
+ HISI_DMA_QUEUE_CQ_HEAD_REG);
ret = hisi_dma_reset_hw(hw);
if (ret) {
diff --git a/drivers/dma/hisilicon/hisi_dmadev.h b/drivers/dma/hisilicon/hisi_dmadev.h
index 50aaa38b72..dd0315cd31 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.h
+++ b/drivers/dma/hisilicon/hisi_dmadev.h
@@ -16,12 +16,17 @@
#define FIELD_GET(mask, reg) \
((typeof(mask))(((reg) & (mask)) >> BF_SHF(mask)))
+#define lower_32_bits(x) ((uint32_t)(x))
+#define upper_32_bits(x) ((uint32_t)(((x) >> 16) >> 16))
+
#define PCI_VENDOR_ID_HUAWEI 0x19e5
#define HISI_DMA_DEVICE_ID 0xA122
#define HISI_DMA_PCI_REVISION_ID_REG 0x08
#define HISI_DMA_REVISION_HIP08B 0x21
#define HISI_DMA_MAX_HW_QUEUES 4
+#define HISI_DMA_MAX_DESC_NUM 8192
+#define HISI_DMA_MIN_DESC_NUM 32
/**
* The HIP08B(HiSilicon IP08) and later Chip(e.g. HiSilicon IP09) are DMA iEPs,
@@ -110,12 +115,106 @@ enum {
HISI_DMA_STATE_RUN,
};
+/**
+ * After scanning the CQ array, the CQ head register needs to be updated.
+ * Updating the register involves write memory barrier operations.
+ * Here use the following method to reduce WMB operations:
+ * a) malloc more CQEs, which correspond to the macro HISI_DMA_CQ_RESERVED.
+ * b) update the CQ head register after accumulated number of completed CQs
+ * is greater than or equal to HISI_DMA_CQ_RESERVED.
+ */
+#define HISI_DMA_CQ_RESERVED 64
+
+struct hisi_dma_sqe {
+ uint32_t dw0;
+#define SQE_FENCE_FLAG BIT(10)
+#define SQE_OPCODE_M2M 0x4
+ uint32_t dw1;
+ uint32_t dw2;
+ uint32_t length;
+ uint64_t src_addr;
+ uint64_t dst_addr;
+};
+
+struct hisi_dma_cqe {
+ uint64_t rsv;
+ uint64_t misc;
+#define CQE_SQ_HEAD_MASK GENMASK(15, 0)
+#define CQE_VALID_B BIT(48)
+#define CQE_STATUS_MASK GENMASK(63, 49)
+};
+
struct hisi_dma_dev {
+ struct hisi_dma_sqe *sqe;
+ volatile struct hisi_dma_cqe *cqe;
+ uint16_t *status; /* the completion status array of SQEs. */
+
+ volatile void *sq_tail_reg; /**< register address for doorbell. */
+ volatile void *cq_head_reg; /**< register address for answer CQ. */
+
+ uint16_t sq_depth_mask; /**< SQ depth - 1, the SQ depth is power of 2 */
+ uint16_t cq_depth; /* CQ depth */
+
+ uint16_t ridx; /**< ring index which will assign to the next request. */
+ /** ring index which returned by hisi_dmadev_completed APIs. */
+ uint16_t cridx;
+
+ /**
+ * SQE array management fields:
+ *
+ * -----------------------------------------------------
+ * | SQE0 | SQE1 | SQE2 | ... | SQEx | ... | SQEn-1 |
+ * -----------------------------------------------------
+ * ^ ^ ^
+ * | | |
+ * sq_head cq_sq_head sq_tail
+ *
+ * sq_head: index to the oldest completed request, this filed was
+ * updated by hisi_dmadev_completed* APIs.
+ * sq_tail: index of the next new request, this field was updated by
+ * hisi_dmadev_copy API.
+ * cq_sq_head: next index of index that has been completed by hardware,
+ * this filed was updated by hisi_dmadev_completed* APIs.
+ *
+ * [sq_head, cq_sq_head): the SQEs that hardware already completed.
+ * [cq_sq_head, sq_tail): the SQEs that hardware processing.
+ */
+ uint16_t sq_head;
+ uint16_t sq_tail;
+ uint16_t cq_sq_head;
+ /**
+ * The driver scans the CQE array, if the valid bit changes, the CQE is
+ * considered valid.
+ * Note: One CQE is corresponding to one or several SQEs, e.g. app
+ * submits two copy requests, the hardware processes the two SQEs,
+ * but it may write back only one CQE and the CQE's sq_head field
+ * indicates the index of the second copy request in the SQE
+ * array.
+ */
+ uint16_t cq_head; /**< CQ index for next scans. */
+ /** accumulated number of completed CQs
+ * @see HISI_DMA_CQ_RESERVED
+ */
+ uint16_t cqs_completed;
+ uint8_t cqe_vld; /**< valid bit for CQE, will change for every round. */
+
+ uint64_t submitted;
+ uint64_t completed;
+ uint64_t errors;
+
+ /**
+ * The following fields are not accessed in the I/O path, so they are
+ * placed at the end.
+ */
struct rte_dma_dev_data *data;
uint8_t revision; /**< PCI revision. */
uint8_t reg_layout; /**< hardware register layout. */
void *io_base;
uint8_t queue_id; /**< hardware DMA queue index. */
+ const struct rte_memzone *iomz;
+ uint32_t iomz_sz;
+ rte_iova_t sqe_iova;
+ rte_iova_t cqe_iova;
};
#endif /* HISI_DMADEV_H */
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v2 4/6] dma/hisilicon: add data path functions
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Chengwen Feng
` (2 preceding siblings ...)
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 3/6] dma/hisilicon: add control path functions Chengwen Feng
@ 2021-11-02 12:37 ` Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 5/6] dma/hisilicon: support multi-process Chengwen Feng
` (2 subsequent siblings)
6 siblings, 0 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-11-02 12:37 UTC (permalink / raw)
To: thomas; +Cc: dev
This patch add data path functions for Kunpeng DMA devices.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
drivers/dma/hisilicon/hisi_dmadev.c | 206 ++++++++++++++++++++++++++++
drivers/dma/hisilicon/hisi_dmadev.h | 16 +++
2 files changed, 222 insertions(+)
diff --git a/drivers/dma/hisilicon/hisi_dmadev.c b/drivers/dma/hisilicon/hisi_dmadev.c
index bcdcf4de4b..d03967cae3 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.c
+++ b/drivers/dma/hisilicon/hisi_dmadev.c
@@ -529,6 +529,206 @@ hisi_dma_dump(const struct rte_dma_dev *dev, FILE *f)
return 0;
}
+static int
+hisi_dma_copy(void *dev_private, uint16_t vchan,
+ rte_iova_t src, rte_iova_t dst,
+ uint32_t length, uint64_t flags)
+{
+ struct hisi_dma_dev *hw = dev_private;
+ struct hisi_dma_sqe *sqe = &hw->sqe[hw->sq_tail];
+
+ RTE_SET_USED(vchan);
+
+ if (((hw->sq_tail + 1) & hw->sq_depth_mask) == hw->sq_head)
+ return -ENOSPC;
+
+ sqe->dw0 = rte_cpu_to_le_32(SQE_OPCODE_M2M);
+ sqe->dw1 = 0;
+ sqe->dw2 = 0;
+ sqe->length = rte_cpu_to_le_32(length);
+ sqe->src_addr = rte_cpu_to_le_64(src);
+ sqe->dst_addr = rte_cpu_to_le_64(dst);
+ hw->sq_tail = (hw->sq_tail + 1) & hw->sq_depth_mask;
+ hw->submitted++;
+
+ if (flags & RTE_DMA_OP_FLAG_FENCE)
+ sqe->dw0 |= rte_cpu_to_le_32(SQE_FENCE_FLAG);
+ if (flags & RTE_DMA_OP_FLAG_SUBMIT)
+ rte_write32(rte_cpu_to_le_32(hw->sq_tail), hw->sq_tail_reg);
+
+ return hw->ridx++;
+}
+
+static int
+hisi_dma_submit(void *dev_private, uint16_t vchan)
+{
+ struct hisi_dma_dev *hw = dev_private;
+
+ RTE_SET_USED(vchan);
+ rte_write32(rte_cpu_to_le_32(hw->sq_tail), hw->sq_tail_reg);
+
+ return 0;
+}
+
+static inline void
+hisi_dma_scan_cq(struct hisi_dma_dev *hw)
+{
+ volatile struct hisi_dma_cqe *cqe;
+ uint16_t csq_head = hw->cq_sq_head;
+ uint16_t cq_head = hw->cq_head;
+ uint16_t count = 0;
+ uint64_t misc;
+
+ while (true) {
+ cqe = &hw->cqe[cq_head];
+ misc = cqe->misc;
+ misc = rte_le_to_cpu_64(misc);
+ if (FIELD_GET(CQE_VALID_B, misc) != hw->cqe_vld)
+ break;
+
+ csq_head = FIELD_GET(CQE_SQ_HEAD_MASK, misc);
+ if (unlikely(misc & CQE_STATUS_MASK))
+ hw->status[csq_head] = FIELD_GET(CQE_STATUS_MASK,
+ misc);
+
+ count++;
+ cq_head++;
+ if (cq_head == hw->cq_depth) {
+ hw->cqe_vld = !hw->cqe_vld;
+ cq_head = 0;
+ }
+ }
+
+ if (count == 0)
+ return;
+
+ hw->cq_head = cq_head;
+ hw->cq_sq_head = (csq_head + 1) & hw->sq_depth_mask;
+ hw->cqs_completed += count;
+ if (hw->cqs_completed >= HISI_DMA_CQ_RESERVED) {
+ rte_write32(rte_cpu_to_le_32(cq_head), hw->cq_head_reg);
+ hw->cqs_completed = 0;
+ }
+}
+
+static inline uint16_t
+hisi_dma_calc_cpls(struct hisi_dma_dev *hw, const uint16_t nb_cpls)
+{
+ uint16_t cpl_num;
+
+ if (hw->cq_sq_head >= hw->sq_head)
+ cpl_num = hw->cq_sq_head - hw->sq_head;
+ else
+ cpl_num = hw->sq_depth_mask + 1 - hw->sq_head + hw->cq_sq_head;
+
+ if (cpl_num > nb_cpls)
+ cpl_num = nb_cpls;
+
+ return cpl_num;
+}
+
+static uint16_t
+hisi_dma_completed(void *dev_private,
+ uint16_t vchan, const uint16_t nb_cpls,
+ uint16_t *last_idx, bool *has_error)
+{
+ struct hisi_dma_dev *hw = dev_private;
+ uint16_t sq_head = hw->sq_head;
+ uint16_t cpl_num, i;
+
+ RTE_SET_USED(vchan);
+ hisi_dma_scan_cq(hw);
+
+ cpl_num = hisi_dma_calc_cpls(hw, nb_cpls);
+ for (i = 0; i < cpl_num; i++) {
+ if (hw->status[sq_head]) {
+ *has_error = true;
+ break;
+ }
+ sq_head = (sq_head + 1) & hw->sq_depth_mask;
+ }
+ if (i > 0) {
+ hw->cridx += i;
+ *last_idx = hw->cridx - 1;
+ hw->sq_head = sq_head;
+ }
+ hw->completed += i;
+
+ return i;
+}
+
+static enum rte_dma_status_code
+hisi_dma_convert_status(uint16_t status)
+{
+ switch (status) {
+ case HISI_DMA_STATUS_SUCCESS:
+ return RTE_DMA_STATUS_SUCCESSFUL;
+ case HISI_DMA_STATUS_INVALID_OPCODE:
+ return RTE_DMA_STATUS_INVALID_OPCODE;
+ case HISI_DMA_STATUS_INVALID_LENGTH:
+ return RTE_DMA_STATUS_INVALID_LENGTH;
+ case HISI_DMA_STATUS_USER_ABORT:
+ return RTE_DMA_STATUS_USER_ABORT;
+ case HISI_DMA_STATUS_REMOTE_READ_ERROR:
+ case HISI_DMA_STATUS_AXI_READ_ERROR:
+ return RTE_DMA_STATUS_BUS_READ_ERROR;
+ case HISI_DMA_STATUS_AXI_WRITE_ERROR:
+ return RTE_DMA_STATUS_BUS_WRITE_ERROR;
+ case HISI_DMA_STATUS_DATA_POISON:
+ case HISI_DMA_STATUS_REMOTE_DATA_POISION:
+ return RTE_DMA_STATUS_DATA_POISION;
+ case HISI_DMA_STATUS_SQE_READ_ERROR:
+ case HISI_DMA_STATUS_SQE_READ_POISION:
+ return RTE_DMA_STATUS_DESCRIPTOR_READ_ERROR;
+ case HISI_DMA_STATUS_LINK_DOWN_ERROR:
+ return RTE_DMA_STATUS_DEV_LINK_ERROR;
+ default:
+ return RTE_DMA_STATUS_ERROR_UNKNOWN;
+ }
+}
+
+static uint16_t
+hisi_dma_completed_status(void *dev_private,
+ uint16_t vchan, const uint16_t nb_cpls,
+ uint16_t *last_idx, enum rte_dma_status_code *status)
+{
+ struct hisi_dma_dev *hw = dev_private;
+ uint16_t sq_head = hw->sq_head;
+ uint16_t cpl_num, i;
+
+ RTE_SET_USED(vchan);
+ hisi_dma_scan_cq(hw);
+
+ cpl_num = hisi_dma_calc_cpls(hw, nb_cpls);
+ for (i = 0; i < cpl_num; i++) {
+ status[i] = hisi_dma_convert_status(hw->status[sq_head]);
+ hw->errors += !!status[i];
+ hw->status[sq_head] = HISI_DMA_STATUS_SUCCESS;
+ sq_head = (sq_head + 1) & hw->sq_depth_mask;
+ }
+ if (likely(cpl_num > 0)) {
+ hw->cridx += cpl_num;
+ *last_idx = hw->cridx - 1;
+ hw->sq_head = sq_head;
+ }
+ hw->completed += cpl_num;
+
+ return cpl_num;
+}
+
+static uint16_t
+hisi_dma_burst_capacity(const void *dev_private, uint16_t vchan)
+{
+ const struct hisi_dma_dev *hw = dev_private;
+ uint16_t sq_head = hw->sq_head;
+ uint16_t sq_tail = hw->sq_tail;
+
+ RTE_SET_USED(vchan);
+
+ return (sq_tail >= sq_head) ? hw->sq_depth_mask - sq_tail + sq_head :
+ sq_head - 1 - sq_tail;
+}
+
static void
hisi_dma_gen_pci_device_name(const struct rte_pci_device *pci_dev,
char *name, size_t size)
@@ -597,6 +797,12 @@ hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
dev->device = &pci_dev->device;
dev->dev_ops = &hisi_dmadev_ops;
+ dev->fp_obj->dev_private = dev->data->dev_private;
+ dev->fp_obj->copy = hisi_dma_copy;
+ dev->fp_obj->submit = hisi_dma_submit;
+ dev->fp_obj->completed = hisi_dma_completed;
+ dev->fp_obj->completed_status = hisi_dma_completed_status;
+ dev->fp_obj->burst_capacity = hisi_dma_burst_capacity;
hw = dev->data->dev_private;
hw->data = dev->data;
diff --git a/drivers/dma/hisilicon/hisi_dmadev.h b/drivers/dma/hisilicon/hisi_dmadev.h
index dd0315cd31..12e209c86e 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.h
+++ b/drivers/dma/hisilicon/hisi_dmadev.h
@@ -115,6 +115,22 @@ enum {
HISI_DMA_STATE_RUN,
};
+/**
+ * Hardware complete status define:
+ */
+#define HISI_DMA_STATUS_SUCCESS 0x0
+#define HISI_DMA_STATUS_INVALID_OPCODE 0x1
+#define HISI_DMA_STATUS_INVALID_LENGTH 0x2
+#define HISI_DMA_STATUS_USER_ABORT 0x4
+#define HISI_DMA_STATUS_REMOTE_READ_ERROR 0x10
+#define HISI_DMA_STATUS_AXI_READ_ERROR 0x20
+#define HISI_DMA_STATUS_AXI_WRITE_ERROR 0x40
+#define HISI_DMA_STATUS_DATA_POISON 0x80
+#define HISI_DMA_STATUS_SQE_READ_ERROR 0x100
+#define HISI_DMA_STATUS_SQE_READ_POISION 0x200
+#define HISI_DMA_STATUS_REMOTE_DATA_POISION 0x400
+#define HISI_DMA_STATUS_LINK_DOWN_ERROR 0x800
+
/**
* After scanning the CQ array, the CQ head register needs to be updated.
* Updating the register involves write memory barrier operations.
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v2 5/6] dma/hisilicon: support multi-process
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Chengwen Feng
` (3 preceding siblings ...)
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 4/6] dma/hisilicon: add data " Chengwen Feng
@ 2021-11-02 12:37 ` Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 6/6] devbind: add Kunpeng DMA to dmadev category Chengwen Feng
2021-11-07 18:59 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Thomas Monjalon
6 siblings, 0 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-11-02 12:37 UTC (permalink / raw)
To: thomas; +Cc: dev
This patch add multi-process support for Kunpeng DMA devices.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
drivers/dma/hisilicon/hisi_dmadev.c | 21 +++++++++++++--------
1 file changed, 13 insertions(+), 8 deletions(-)
diff --git a/drivers/dma/hisilicon/hisi_dmadev.c b/drivers/dma/hisilicon/hisi_dmadev.c
index d03967cae3..05066b4d0e 100644
--- a/drivers/dma/hisilicon/hisi_dmadev.c
+++ b/drivers/dma/hisilicon/hisi_dmadev.c
@@ -392,8 +392,10 @@ hisi_dma_stop(struct rte_dma_dev *dev)
static int
hisi_dma_close(struct rte_dma_dev *dev)
{
- /* The dmadev already stopped */
- hisi_dma_free_iomem(dev->data->dev_private);
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ /* The dmadev already stopped */
+ hisi_dma_free_iomem(dev->data->dev_private);
+ }
return 0;
}
@@ -815,11 +817,13 @@ hisi_dma_create(struct rte_pci_device *pci_dev, uint8_t queue_id,
hw->cq_head_reg = hisi_dma_queue_regaddr(hw,
HISI_DMA_QUEUE_CQ_HEAD_REG);
- ret = hisi_dma_reset_hw(hw);
- if (ret) {
- HISI_DMA_LOG(ERR, "%s init device fail!", name);
- (void)rte_dma_pmd_release(name);
- return -EIO;
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
+ ret = hisi_dma_reset_hw(hw);
+ if (ret) {
+ HISI_DMA_LOG(ERR, "%s init device fail!", name);
+ (void)rte_dma_pmd_release(name);
+ return -EIO;
+ }
}
dev->state = RTE_DMA_DEV_READY;
@@ -872,7 +876,8 @@ hisi_dma_probe(struct rte_pci_driver *pci_drv __rte_unused,
return ret;
HISI_DMA_LOG(DEBUG, "%s read PCI revision: 0x%x", name, revision);
- hisi_dma_init_gbl(pci_dev->mem_resource[2].addr, revision);
+ if (rte_eal_process_type() == RTE_PROC_PRIMARY)
+ hisi_dma_init_gbl(pci_dev->mem_resource[2].addr, revision);
for (i = 0; i < HISI_DMA_MAX_HW_QUEUES; i++) {
ret = hisi_dma_create(pci_dev, i, revision);
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [dpdk-dev] [PATCH v2 6/6] devbind: add Kunpeng DMA to dmadev category
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Chengwen Feng
` (4 preceding siblings ...)
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 5/6] dma/hisilicon: support multi-process Chengwen Feng
@ 2021-11-02 12:37 ` Chengwen Feng
2021-11-07 18:59 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Thomas Monjalon
6 siblings, 0 replies; 16+ messages in thread
From: Chengwen Feng @ 2021-11-02 12:37 UTC (permalink / raw)
To: thomas; +Cc: dev
add Kunpeng DMA device ID to dmadev category.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
---
usertools/dpdk-devbind.py | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index bb00f43702..a74a68ed82 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -68,10 +68,14 @@
intel_ntb_icx = {'Class': '06', 'Vendor': '8086', 'Device': '347e',
'SVendor': None, 'SDevice': None}
+hisilicon_dma = {'Class': '08', 'Vendor': '19e5', 'Device': 'a122',
+ 'SVendor': None, 'SDevice': None}
+
network_devices = [network_class, cavium_pkx, avp_vnic, ifpga_class]
baseband_devices = [acceleration_class]
crypto_devices = [encryption_class, intel_processor_class]
-dma_devices = [intel_idxd_spr, intel_ioat_bdw, intel_ioat_icx, intel_ioat_skx]
+dma_devices = [intel_idxd_spr, intel_ioat_bdw, intel_ioat_icx, intel_ioat_skx,
+ hisilicon_dma]
eventdev_devices = [cavium_sso, cavium_tim, intel_dlb, octeontx2_sso]
mempool_devices = [cavium_fpa, octeontx2_npa]
compress_devices = [cavium_zip]
--
2.33.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [dpdk-dev] [PATCH v2 1/6] dma/hisilicon: add device probe and remove functions
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 1/6] dma/hisilicon: add device probe and remove functions Chengwen Feng
@ 2021-11-07 17:28 ` Thomas Monjalon
0 siblings, 0 replies; 16+ messages in thread
From: Thomas Monjalon @ 2021-11-07 17:28 UTC (permalink / raw)
To: Chengwen Feng; +Cc: dev
02/11/2021 13:37, Chengwen Feng:
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> +Hisilicon DMA
> +M: Chengwen Feng <fengchengwen@huawei.com>
> +F: drivers/dma/hisilicon
Final slash is missing (and it is mandatory for the parsing).
> --- a/doc/guides/dmadevs/index.rst
> +++ b/doc/guides/dmadevs/index.rst
> @@ -13,3 +13,4 @@ an application through DMA API.
>
> idxd
> ioat
> + hisilicon
Let's keep an alphabetical order.
> --- a/doc/guides/rel_notes/release_21_11.rst
> +++ b/doc/guides/rel_notes/release_21_11.rst
> +* **Added hisilicon dmadev driver implementation.**
HiSilicon DMA driver
blank line missing
> + The hisilicon dmadev driver provide device drivers for the Kunpeng's DMA devices.
> + This device driver can be used through the generic dmadev API.
> --- a/drivers/dma/meson.build
> +++ b/drivers/dma/meson.build
> @@ -4,6 +4,7 @@
> drivers = [
> 'idxd',
> 'ioat',
> + 'hisilicon',
> 'skeleton',
> ]
alphabetical order
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Chengwen Feng
` (5 preceding siblings ...)
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 6/6] devbind: add Kunpeng DMA to dmadev category Chengwen Feng
@ 2021-11-07 18:59 ` Thomas Monjalon
6 siblings, 0 replies; 16+ messages in thread
From: Thomas Monjalon @ 2021-11-07 18:59 UTC (permalink / raw)
To: Chengwen Feng; +Cc: dev
02/11/2021 13:37, Chengwen Feng:
> This patch set add hisilicon DMA driver.
>
> Chengwen Feng (6):
> dma/hisilicon: add device probe and remove functions
> dma/hisilicon: add dmadev instances create and destroy
> dma/hisilicon: add control path functions
> dma/hisilicon: add data path functions
> dma/hisilicon: support multi-process
> devbind: add Kunpeng DMA to dmadev category
Applied with few minor fixes, thanks.
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2021-11-07 18:59 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-30 10:36 [dpdk-dev] [PATCH 0/6] dma: add hisilicon DMA driver Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 1/6] dma/hisilicon: add device probe and remove functions Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 2/6] dma/hisilicon: add dmadev instances create and destroy Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 3/6] dma/hisilicon: add control path functions Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 4/6] dma/hisilicon: add data " Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 5/6] dma/hisilicon: support multi-process Chengwen Feng
2021-10-30 10:36 ` [dpdk-dev] [PATCH 6/6] devbind: add Kunpeng DMA to dmadev category Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 1/6] dma/hisilicon: add device probe and remove functions Chengwen Feng
2021-11-07 17:28 ` Thomas Monjalon
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 2/6] dma/hisilicon: add dmadev instances create and destroy Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 3/6] dma/hisilicon: add control path functions Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 4/6] dma/hisilicon: add data " Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 5/6] dma/hisilicon: support multi-process Chengwen Feng
2021-11-02 12:37 ` [dpdk-dev] [PATCH v2 6/6] devbind: add Kunpeng DMA to dmadev category Chengwen Feng
2021-11-07 18:59 ` [dpdk-dev] [PATCH v2 0/6] dma: add hisilicon DMA driver Thomas Monjalon
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.