* [PATCH v2 1/8] compress/mlx5: fix decompress xform validation
2023-02-02 16:25 ` [PATCH v2 0/8] compress/mlx5: add LZ4 support Michael Baum
@ 2023-02-02 16:25 ` Michael Baum
2023-02-02 16:25 ` [PATCH v2 2/8] compress/mlx5: fix wrong output Adler-32 checksum offset Michael Baum
` (7 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-02 16:25 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon, rzidane, stable
In xform creation, it first validate the xform according the
capabilities.
One of validations verifies that given "hash_algo" is
"RTE_COMP_HASH_ALGO_NONE" for both compress and decompress xform
objects.
However, the validation for decompress checks it again for compress
xform object.
This patch changes it to verify decompress xform object.
Fixes: 2efd26544554 ("compress/mlx5: support partial transformation")
Cc: rzidane@nvidia.com
Cc: stable@dpdk.org
Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
drivers/compress/mlx5/mlx5_compress.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index 459e4b5e8a..cadff83f27 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -313,7 +313,7 @@ mlx5_compress_xform_create(struct rte_compressdev *dev,
DRV_LOG(ERR, "Not enough capabilities to support decompress operation, maybe old FW/OFED version?");
return -ENOTSUP;
}
- if (xform->compress.hash_algo != RTE_COMP_HASH_ALGO_NONE) {
+ if (xform->decompress.hash_algo != RTE_COMP_HASH_ALGO_NONE) {
DRV_LOG(ERR, "SHA is not supported.");
return -ENOTSUP;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v2 2/8] compress/mlx5: fix wrong output Adler-32 checksum offset
2023-02-02 16:25 ` [PATCH v2 0/8] compress/mlx5: add LZ4 support Michael Baum
2023-02-02 16:25 ` [PATCH v2 1/8] compress/mlx5: fix decompress xform validation Michael Baum
@ 2023-02-02 16:25 ` Michael Baum
2023-02-02 16:25 ` [PATCH v2 3/8] compress/mlx5: fix QP setup for partial transformations Michael Baum
` (6 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-02 16:25 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon, stable
After de/compress dequeue, the output checksum is copied into the op
structure. The "output_checksum" field in op structure is "uint64_t"
type, and the 32-bit checksums (CRC32, Adler-32) are copied into the
lower 32 bits.
When both CRC32 and Adler-32 are configured, CRC32 is copied into the
lower 32 bits and Adler-32 into the upper 32 bits.
However, in mlx5 PMD Adler-32 without CRC, is mistakenly copied into the
upper 32 bits.
This patch updates Adler-32 output checksun to be copied into the
lower 32 bits.
Fixes: f8c97babc9f4 ("compress/mlx5: add data-path functions")
Cc: matan@nvidia.com
Cc: stable@dpdk.org
Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
drivers/compress/mlx5/mlx5_compress.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index cadff83f27..c46fb4eb89 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -633,7 +633,7 @@ mlx5_compress_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
break;
case RTE_COMP_CHECKSUM_ADLER32:
op->output_chksum = (uint64_t)rte_be_to_cpu_32
- (opaq[idx].adler32) << 32;
+ (opaq[idx].adler32);
break;
case RTE_COMP_CHECKSUM_CRC32_ADLER32:
op->output_chksum = (uint64_t)rte_be_to_cpu_32
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v2 3/8] compress/mlx5: fix QP setup for partial transformations
2023-02-02 16:25 ` [PATCH v2 0/8] compress/mlx5: add LZ4 support Michael Baum
2023-02-02 16:25 ` [PATCH v2 1/8] compress/mlx5: fix decompress xform validation Michael Baum
2023-02-02 16:25 ` [PATCH v2 2/8] compress/mlx5: fix wrong output Adler-32 checksum offset Michael Baum
@ 2023-02-02 16:25 ` Michael Baum
2023-02-02 16:25 ` [PATCH v2 4/8] compress/mlx5: support new metadata layout added in BF3 Michael Baum
` (5 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-02 16:25 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon, rzidane, stable
The mlx5_compress_qp_setup() function creates QP for compress,
decompress and DMA. Thus, the MMO flag is turned on only when all
operations are supported.
However, since partial transformations have been allowed, it should be
turn on for part of them.
This patch removes the compress MMO support requirement.
Fixes: 2efd26544554 ("compress/mlx5: support partial transformation")
Cc: rzidane@nvidia.com
Cc: stable@dpdk.org
Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
drivers/compress/mlx5/mlx5_compress.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index c46fb4eb89..c4bf62ed41 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -243,8 +243,8 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
mlx5_ts_format_conv(priv->cdev->config.hca_attr.qp_ts_format);
qp_attr.num_of_receive_wqes = 0;
qp_attr.num_of_send_wqbbs = RTE_BIT32(log_ops_n);
- qp_attr.mmo = priv->mmo_decomp_qp && priv->mmo_comp_qp
- && priv->mmo_dma_qp;
+ qp_attr.mmo = priv->mmo_decomp_qp || priv->mmo_comp_qp ||
+ priv->mmo_dma_qp;
ret = mlx5_devx_qp_create(priv->cdev->ctx, &qp->qp,
qp_attr.num_of_send_wqbbs *
MLX5_WQE_SIZE, &qp_attr, socket_id);
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v2 4/8] compress/mlx5: support new metadata layout added in BF3
2023-02-02 16:25 ` [PATCH v2 0/8] compress/mlx5: add LZ4 support Michael Baum
` (2 preceding siblings ...)
2023-02-02 16:25 ` [PATCH v2 3/8] compress/mlx5: fix QP setup for partial transformations Michael Baum
@ 2023-02-02 16:25 ` Michael Baum
2023-02-02 16:25 ` [PATCH v2 5/8] compress/mlx5: remove unused variable from priv structure Michael Baum
` (4 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-02 16:25 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon
Commit [1] add support in Bluefield-3, but Bluefield-3 has different
GGA opaque structure than Bluefield-2.
This patch updates the PRM structure to include both versions, and
culculate the relevant offset for each version in control path.
Commit [1] 559014f232b4 ("compress/mlx5: add Bluefield-3 device ID")
Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
drivers/common/mlx5/mlx5_devx_cmds.c | 6 ++--
drivers/common/mlx5/mlx5_devx_cmds.h | 3 +-
drivers/common/mlx5/mlx5_prm.h | 40 +++++++++++++++---------
drivers/compress/mlx5/mlx5_compress.c | 45 +++++++++++++++++----------
4 files changed, 61 insertions(+), 33 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index 59cebb530f..dfec4dcf1b 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -978,8 +978,10 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
attr->mmo_dma_qp_en = MLX5_GET(cmd_hca_cap, hcattr, dma_mmo_qp);
attr->mmo_compress_qp_en = MLX5_GET(cmd_hca_cap, hcattr,
compress_mmo_qp);
- attr->mmo_decompress_qp_en = MLX5_GET(cmd_hca_cap, hcattr,
- decompress_mmo_qp);
+ attr->decomp_deflate_v1_en = MLX5_GET(cmd_hca_cap, hcattr,
+ decompress_deflate_v1);
+ attr->decomp_deflate_v2_en = MLX5_GET(cmd_hca_cap, hcattr,
+ decompress_deflate_v2);
attr->compress_min_block_size = MLX5_GET(cmd_hca_cap, hcattr,
compress_min_block_size);
attr->log_max_mmo_dma = MLX5_GET(cmd_hca_cap, hcattr, log_dma_mmo_size);
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index c94b9eac06..edb387e272 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -259,7 +259,8 @@ struct mlx5_hca_attr {
uint32_t mmo_decompress_sq_en:1;
uint32_t mmo_dma_qp_en:1;
uint32_t mmo_compress_qp_en:1;
- uint32_t mmo_decompress_qp_en:1;
+ uint32_t decomp_deflate_v1_en:1;
+ uint32_t decomp_deflate_v2_en:1;
uint32_t mmo_regex_qp_en:1;
uint32_t mmo_regex_sq_en:1;
uint32_t compress_min_block_size:4;
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 2b5c43ee6e..377cbfab87 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -607,17 +607,27 @@ struct mlx5_gga_wqe {
struct mlx5_wqe_dseg scatter;
} __rte_packed;
-struct mlx5_gga_compress_opaque {
- uint32_t syndrom;
- uint32_t reserved0;
- uint32_t scattered_length;
- uint32_t gathered_length;
- uint64_t scatter_crc;
- uint64_t gather_crc;
- uint32_t crc32;
- uint32_t adler32;
- uint8_t reserved1[216];
-} __rte_packed;
+union mlx5_gga_compress_opaque {
+ struct {
+ uint32_t syndrome;
+ uint32_t reserved0;
+ uint32_t scattered_length;
+ union {
+ struct {
+ uint32_t reserved1[5];
+ uint32_t crc32;
+ uint32_t adler32;
+ } v1 __rte_packed;
+ struct {
+ uint32_t crc32;
+ uint32_t adler32;
+ uint32_t crc32c;
+ uint32_t xxh32;
+ } v2 __rte_packed;
+ };
+ } __rte_packed;
+ uint32_t data[64];
+};
struct mlx5_ifc_regexp_mmo_control_bits {
uint8_t reserved_at_31[0x2];
@@ -1463,7 +1473,7 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 log_max_bsf_list_size[0x6];
u8 umr_extended_translation_offset[0x1];
u8 null_mkey[0x1];
- u8 log_max_klm_list_size[0x6];
+ u8 log_maxklm_list_size[0x6];
u8 non_wire_sq[0x1];
u8 reserved_at_121[0x9];
u8 log_max_ra_req_dc[0x6];
@@ -1749,8 +1759,10 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 dma_mmo_qp[0x1];
u8 regexp_mmo_qp[0x1];
u8 compress_mmo_qp[0x1];
- u8 decompress_mmo_qp[0x1];
- u8 reserved_at_74c[0x14];
+ u8 decompress_deflate_v1[0x1];
+ u8 reserved_at_74c[0x4];
+ u8 decompress_deflate_v2[0x1];
+ u8 reserved_at_751[0xf];
u8 reserved_at_760[0x3];
u8 log_max_num_header_modify_argument[0x5];
u8 log_header_modify_argument_granularity_offset[0x4];
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index c4bf62ed41..11fad72a4f 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -55,6 +55,7 @@ struct mlx5_compress_priv {
uint32_t mmo_dma_sq:1;
uint32_t mmo_dma_qp:1;
uint32_t log_block_sz;
+ uint32_t crc32_opaq_offs;
};
struct mlx5_compress_qp {
@@ -157,7 +158,7 @@ mlx5_compress_init_qp(struct mlx5_compress_qp *qp)
{
volatile struct mlx5_gga_wqe *restrict wqe =
(volatile struct mlx5_gga_wqe *)qp->qp.wqes;
- volatile struct mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
+ volatile union mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
const uint32_t sq_ds = rte_cpu_to_be_32((qp->qp.qp->id << 8) | 4u);
const uint32_t flags = RTE_BE32(MLX5_COMP_ALWAYS <<
MLX5_COMP_MODE_OFFSET);
@@ -211,8 +212,8 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
goto err;
}
opaq_buf = rte_calloc(__func__, (size_t)1 << log_ops_n,
- sizeof(struct mlx5_gga_compress_opaque),
- sizeof(struct mlx5_gga_compress_opaque));
+ sizeof(union mlx5_gga_compress_opaque),
+ sizeof(union mlx5_gga_compress_opaque));
if (opaq_buf == NULL) {
DRV_LOG(ERR, "Failed to allocate opaque memory.");
rte_errno = ENOMEM;
@@ -225,7 +226,7 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
qp->ops = (struct rte_comp_op **)RTE_ALIGN((uintptr_t)(qp + 1),
RTE_CACHE_LINE_SIZE);
if (mlx5_common_verbs_reg_mr(priv->cdev->pd, opaq_buf, qp->entries_n *
- sizeof(struct mlx5_gga_compress_opaque),
+ sizeof(union mlx5_gga_compress_opaque),
&qp->opaque_mr) != 0) {
rte_free(opaq_buf);
DRV_LOG(ERR, "Failed to register opaque MR.");
@@ -544,7 +545,7 @@ mlx5_compress_dump_err_objs(volatile uint32_t *cqe, volatile uint32_t *wqe,
DRV_LOG(ERR, "%08X %08X %08X %08X", wqe[i], wqe[i + 1],
wqe[i + 2], wqe[i + 3]);
DRV_LOG(ERR, "\nError opaq:");
- for (i = 0; i < sizeof(struct mlx5_gga_compress_opaque) >> 2; i += 4)
+ for (i = 0; i < sizeof(union mlx5_gga_compress_opaque) >> 2; i += 4)
DRV_LOG(ERR, "%08X %08X %08X %08X", opaq[i], opaq[i + 1],
opaq[i + 2], opaq[i + 3]);
}
@@ -558,7 +559,7 @@ mlx5_compress_cqe_err_handle(struct mlx5_compress_qp *qp,
&qp->cq.cqes[idx];
volatile struct mlx5_gga_wqe *wqes = (volatile struct mlx5_gga_wqe *)
qp->qp.wqes;
- volatile struct mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
+ volatile union mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
volatile uint32_t *synd_word = RTE_PTR_ADD(cqe, MLX5_ERROR_CQE_SYNDROME_OFFSET);
switch (*synd_word) {
@@ -575,7 +576,7 @@ mlx5_compress_cqe_err_handle(struct mlx5_compress_qp *qp,
op->consumed = 0;
op->produced = 0;
op->output_chksum = 0;
- op->debug_status = rte_be_to_cpu_32(opaq[idx].syndrom) |
+ op->debug_status = rte_be_to_cpu_32(opaq[idx].syndrome) |
((uint64_t)rte_be_to_cpu_32(cqe->syndrome) << 32);
mlx5_compress_dump_err_objs((volatile uint32_t *)cqe,
(volatile uint32_t *)&wqes[idx],
@@ -590,13 +591,14 @@ mlx5_compress_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
struct mlx5_compress_qp *qp = queue_pair;
volatile struct mlx5_compress_xform *restrict xform;
volatile struct mlx5_cqe *restrict cqe;
- volatile struct mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
+ volatile union mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
struct rte_comp_op *restrict op;
const unsigned int cq_size = qp->entries_n;
const unsigned int mask = cq_size - 1;
uint32_t idx;
uint32_t next_idx = qp->ci & mask;
const uint16_t max = RTE_MIN((uint16_t)(qp->pi - qp->ci), nb_ops);
+ uint32_t crc32_idx = qp->priv->crc32_opaq_offs;
uint16_t i = 0;
int ret;
@@ -629,17 +631,17 @@ mlx5_compress_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
switch (xform->csum_type) {
case RTE_COMP_CHECKSUM_CRC32:
op->output_chksum = (uint64_t)rte_be_to_cpu_32
- (opaq[idx].crc32);
+ (opaq[idx].data[crc32_idx]);
break;
case RTE_COMP_CHECKSUM_ADLER32:
op->output_chksum = (uint64_t)rte_be_to_cpu_32
- (opaq[idx].adler32);
+ (opaq[idx].data[crc32_idx + 1]);
break;
case RTE_COMP_CHECKSUM_CRC32_ADLER32:
op->output_chksum = (uint64_t)rte_be_to_cpu_32
- (opaq[idx].crc32) |
+ (opaq[idx].data[crc32_idx]) |
((uint64_t)rte_be_to_cpu_32
- (opaq[idx].adler32) << 32);
+ (opaq[idx].data[crc32_idx + 1]) << 32);
break;
default:
break;
@@ -717,15 +719,17 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
.socket_id = cdev->dev->numa_node,
};
const char *ibdev_name = mlx5_os_get_ctx_device_name(cdev->ctx);
+ uint32_t crc32_opaq_offset;
if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
DRV_LOG(ERR, "Non-primary process type is not supported.");
rte_errno = ENOTSUP;
return -rte_errno;
}
- if (!attr->mmo_decompress_qp_en && !attr->mmo_decompress_sq_en
- && !attr->mmo_compress_qp_en && !attr->mmo_compress_sq_en
- && !attr->mmo_dma_qp_en && !attr->mmo_dma_sq_en) {
+ if (!attr->decomp_deflate_v1_en && !attr->decomp_deflate_v2_en &&
+ !attr->mmo_decompress_sq_en && !attr->mmo_compress_qp_en &&
+ !attr->mmo_compress_sq_en && !attr->mmo_dma_qp_en &&
+ !attr->mmo_dma_sq_en) {
DRV_LOG(ERR, "Not enough capabilities to support compress operations, maybe old FW/OFED version?");
rte_errno = ENOTSUP;
return -ENOTSUP;
@@ -746,11 +750,20 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
priv = compressdev->data->dev_private;
priv->log_block_sz = devarg_prms.log_block_sz;
priv->mmo_decomp_sq = attr->mmo_decompress_sq_en;
- priv->mmo_decomp_qp = attr->mmo_decompress_qp_en;
+ priv->mmo_decomp_qp =
+ attr->decomp_deflate_v1_en | attr->decomp_deflate_v2_en;
priv->mmo_comp_sq = attr->mmo_compress_sq_en;
priv->mmo_comp_qp = attr->mmo_compress_qp_en;
priv->mmo_dma_sq = attr->mmo_dma_sq_en;
priv->mmo_dma_qp = attr->mmo_dma_qp_en;
+ if (attr->decomp_deflate_v2_en)
+ crc32_opaq_offset = offsetof(union mlx5_gga_compress_opaque,
+ v2.crc32);
+ else
+ crc32_opaq_offset = offsetof(union mlx5_gga_compress_opaque,
+ v1.crc32);
+ MLX5_ASSERT((crc32_opaq_offset % 4) == 0);
+ priv->crc32_opaq_offs = crc32_opaq_offset / 4;
priv->cdev = cdev;
priv->compressdev = compressdev;
priv->min_block_size = attr->compress_min_block_size;
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v2 5/8] compress/mlx5: remove unused variable from priv structure
2023-02-02 16:25 ` [PATCH v2 0/8] compress/mlx5: add LZ4 support Michael Baum
` (3 preceding siblings ...)
2023-02-02 16:25 ` [PATCH v2 4/8] compress/mlx5: support new metadata layout added in BF3 Michael Baum
@ 2023-02-02 16:25 ` Michael Baum
2023-02-02 16:25 ` [PATCH v2 6/8] compress/mlx5: add xform validate function Michael Baum
` (3 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-02 16:25 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon
The priv structure has variable named "min_block_size" coming from HCA
capabilities.
This field isn't used and copied into the priv structure for free.
This patch removes this field.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
drivers/compress/mlx5/mlx5_compress.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index 11fad72a4f..d54e49db97 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -42,8 +42,6 @@ struct mlx5_compress_priv {
struct rte_compressdev *compressdev;
struct mlx5_common_device *cdev; /* Backend mlx5 device. */
struct mlx5_uar uar;
- uint8_t min_block_size;
- /* Minimum huffman block size supported by the device. */
struct rte_compressdev_config dev_config;
LIST_HEAD(xform_list, mlx5_compress_xform) xform_list;
rte_spinlock_t xform_sl;
@@ -766,7 +764,6 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
priv->crc32_opaq_offs = crc32_opaq_offset / 4;
priv->cdev = cdev;
priv->compressdev = compressdev;
- priv->min_block_size = attr->compress_min_block_size;
if (mlx5_devx_uar_prepare(cdev, &priv->uar) != 0) {
rte_compressdev_pmd_destroy(priv->compressdev);
return -1;
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v2 6/8] compress/mlx5: add xform validate function
2023-02-02 16:25 ` [PATCH v2 0/8] compress/mlx5: add LZ4 support Michael Baum
` (4 preceding siblings ...)
2023-02-02 16:25 ` [PATCH v2 5/8] compress/mlx5: remove unused variable from priv structure Michael Baum
@ 2023-02-02 16:25 ` Michael Baum
2023-02-02 16:25 ` [PATCH v2 7/8] common/mlx5: add LZ4 capabilities check Michael Baum
` (2 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-02 16:25 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon
Separate the xform validations from create function into new function.
The new function checks caps directly from HCA attr structure, so all
capabilities in priv structure were removed.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
drivers/compress/mlx5/mlx5_compress.c | 75 +++++++++++++++------------
1 file changed, 42 insertions(+), 33 deletions(-)
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index d54e49db97..7841f57b9c 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -45,13 +45,6 @@ struct mlx5_compress_priv {
struct rte_compressdev_config dev_config;
LIST_HEAD(xform_list, mlx5_compress_xform) xform_list;
rte_spinlock_t xform_sl;
- /* HCA caps */
- uint32_t mmo_decomp_sq:1;
- uint32_t mmo_decomp_qp:1;
- uint32_t mmo_comp_sq:1;
- uint32_t mmo_comp_qp:1;
- uint32_t mmo_dma_sq:1;
- uint32_t mmo_dma_qp:1;
uint32_t log_block_sz;
uint32_t crc32_opaq_offs;
};
@@ -178,6 +171,7 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
uint32_t max_inflight_ops, int socket_id)
{
struct mlx5_compress_priv *priv = dev->data->dev_private;
+ struct mlx5_hca_attr *attr = &priv->cdev->config.hca_attr;
struct mlx5_compress_qp *qp;
struct mlx5_devx_cq_attr cq_attr = {
.uar_page_id = mlx5_os_get_devx_uar_page_id(priv->uar.obj),
@@ -238,12 +232,11 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
goto err;
}
qp_attr.cqn = qp->cq.cq->id;
- qp_attr.ts_format =
- mlx5_ts_format_conv(priv->cdev->config.hca_attr.qp_ts_format);
+ qp_attr.ts_format = mlx5_ts_format_conv(attr->qp_ts_format);
qp_attr.num_of_receive_wqes = 0;
qp_attr.num_of_send_wqbbs = RTE_BIT32(log_ops_n);
- qp_attr.mmo = priv->mmo_decomp_qp || priv->mmo_comp_qp ||
- priv->mmo_dma_qp;
+ qp_attr.mmo = attr->mmo_compress_qp_en || attr->mmo_dma_qp_en ||
+ attr->decomp_deflate_v1_en || attr->decomp_deflate_v2_en;
ret = mlx5_devx_qp_create(priv->cdev->ctx, &qp->qp,
qp_attr.num_of_send_wqbbs *
MLX5_WQE_SIZE, &qp_attr, socket_id);
@@ -276,21 +269,17 @@ mlx5_compress_xform_free(struct rte_compressdev *dev, void *xform)
}
static int
-mlx5_compress_xform_create(struct rte_compressdev *dev,
- const struct rte_comp_xform *xform,
- void **private_xform)
+mlx5_compress_xform_validate(const struct rte_comp_xform *xform,
+ const struct mlx5_hca_attr *attr)
{
- struct mlx5_compress_priv *priv = dev->data->dev_private;
- struct mlx5_compress_xform *xfrm;
- uint32_t size;
-
switch (xform->type) {
case RTE_COMP_COMPRESS:
if (xform->compress.algo == RTE_COMP_ALGO_NULL &&
- !priv->mmo_dma_qp && !priv->mmo_dma_sq) {
+ !attr->mmo_dma_qp_en && !attr->mmo_dma_sq_en) {
DRV_LOG(ERR, "Not enough capabilities to support DMA operation, maybe old FW/OFED version?");
return -ENOTSUP;
- } else if (!priv->mmo_comp_qp && !priv->mmo_comp_sq) {
+ } else if (!attr->mmo_compress_qp_en &&
+ !attr->mmo_compress_sq_en) {
DRV_LOG(ERR, "Not enough capabilities to support compress operation, maybe old FW/OFED version?");
return -ENOTSUP;
}
@@ -304,12 +293,24 @@ mlx5_compress_xform_create(struct rte_compressdev *dev,
}
break;
case RTE_COMP_DECOMPRESS:
- if (xform->decompress.algo == RTE_COMP_ALGO_NULL &&
- !priv->mmo_dma_qp && !priv->mmo_dma_sq) {
- DRV_LOG(ERR, "Not enough capabilities to support DMA operation, maybe old FW/OFED version?");
- return -ENOTSUP;
- } else if (!priv->mmo_decomp_qp && !priv->mmo_decomp_sq) {
- DRV_LOG(ERR, "Not enough capabilities to support decompress operation, maybe old FW/OFED version?");
+ switch (xform->decompress.algo) {
+ case RTE_COMP_ALGO_NULL:
+ if (attr->mmo_dma_qp_en && !attr->mmo_dma_sq_en) {
+ DRV_LOG(ERR, "Not enough capabilities to support DMA operation, maybe old FW/OFED version?");
+ return -ENOTSUP;
+ }
+ break;
+ case RTE_COMP_ALGO_DEFLATE:
+ if (!attr->decomp_deflate_v1_en &&
+ !attr->decomp_deflate_v2_en &&
+ !attr->mmo_decompress_sq_en) {
+ DRV_LOG(ERR, "Not enough capabilities to support decompress DEFLATE algorithm, maybe old FW/OFED version?");
+ return -ENOTSUP;
+ }
+ break;
+ default:
+ DRV_LOG(ERR, "Algorithm %u is not supported.",
+ xform->decompress.algo);
return -ENOTSUP;
}
if (xform->decompress.hash_algo != RTE_COMP_HASH_ALGO_NONE) {
@@ -321,7 +322,22 @@ mlx5_compress_xform_create(struct rte_compressdev *dev,
DRV_LOG(ERR, "Xform type should be compress/decompress");
return -ENOTSUP;
}
+ return 0;
+}
+
+static int
+mlx5_compress_xform_create(struct rte_compressdev *dev,
+ const struct rte_comp_xform *xform,
+ void **private_xform)
+{
+ struct mlx5_compress_priv *priv = dev->data->dev_private;
+ struct mlx5_compress_xform *xfrm;
+ uint32_t size;
+ int ret;
+ ret = mlx5_compress_xform_validate(xform, &priv->cdev->config.hca_attr);
+ if (ret < 0)
+ return ret;
xfrm = rte_zmalloc_socket(__func__, sizeof(*xfrm), 0,
priv->dev_config.socket_id);
if (xfrm == NULL)
@@ -747,13 +763,6 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
compressdev->feature_flags = RTE_COMPDEV_FF_HW_ACCELERATED;
priv = compressdev->data->dev_private;
priv->log_block_sz = devarg_prms.log_block_sz;
- priv->mmo_decomp_sq = attr->mmo_decompress_sq_en;
- priv->mmo_decomp_qp =
- attr->decomp_deflate_v1_en | attr->decomp_deflate_v2_en;
- priv->mmo_comp_sq = attr->mmo_compress_sq_en;
- priv->mmo_comp_qp = attr->mmo_compress_qp_en;
- priv->mmo_dma_sq = attr->mmo_dma_sq_en;
- priv->mmo_dma_qp = attr->mmo_dma_qp_en;
if (attr->decomp_deflate_v2_en)
crc32_opaq_offset = offsetof(union mlx5_gga_compress_opaque,
v2.crc32);
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v2 7/8] common/mlx5: add LZ4 capabilities check
2023-02-02 16:25 ` [PATCH v2 0/8] compress/mlx5: add LZ4 support Michael Baum
` (5 preceding siblings ...)
2023-02-02 16:25 ` [PATCH v2 6/8] compress/mlx5: add xform validate function Michael Baum
@ 2023-02-02 16:25 ` Michael Baum
2023-02-02 16:25 ` [PATCH v2 8/8] compress/mlx5: add support for LZ4 algorithm Michael Baum
2023-02-21 7:07 ` [PATCH v3 0/8] compress/mlx5: add LZ4 support Michael Baum
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-02 16:25 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon
Add capabilities check for LZ4 decompression algorithm.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
drivers/common/mlx5/mlx5_devx_cmds.c | 6 ++++++
drivers/common/mlx5/mlx5_devx_cmds.h | 3 +++
drivers/common/mlx5/mlx5_prm.h | 16 ++++++++++++++--
3 files changed, 23 insertions(+), 2 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index dfec4dcf1b..f30daa19c7 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -989,6 +989,12 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
log_compress_mmo_size);
attr->log_max_mmo_decompress = MLX5_GET(cmd_hca_cap, hcattr,
log_decompress_mmo_size);
+ attr->decomp_lz4_data_only_en = MLX5_GET(cmd_hca_cap, hcattr,
+ decompress_lz4_data_only_v2);
+ attr->decomp_lz4_no_checksum_en = MLX5_GET(cmd_hca_cap, hcattr,
+ decompress_lz4_no_checksum_v2);
+ attr->decomp_lz4_checksum_en = MLX5_GET(cmd_hca_cap, hcattr,
+ decompress_lz4_checksum_v2);
attr->cqe_compression = MLX5_GET(cmd_hca_cap, hcattr, cqe_compression);
attr->mini_cqe_resp_flow_tag = MLX5_GET(cmd_hca_cap, hcattr,
mini_cqe_resp_flow_tag);
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index edb387e272..a82af9426d 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -267,6 +267,9 @@ struct mlx5_hca_attr {
uint32_t log_max_mmo_dma:5;
uint32_t log_max_mmo_compress:5;
uint32_t log_max_mmo_decompress:5;
+ uint32_t decomp_lz4_data_only_en:1;
+ uint32_t decomp_lz4_no_checksum_en:1;
+ uint32_t decomp_lz4_checksum_en:1;
uint32_t umr_modify_entity_size_disabled:1;
uint32_t umr_indirect_mkey_disabled:1;
uint32_t log_min_stride_wqe_sz:5;
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 377cbfab87..f89af8b96b 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -578,9 +578,19 @@ struct mlx5_rdma_write_wqe {
#define MLX5_OPC_MOD_MMO_DECOMP 0x3u
#define MLX5_OPC_MOD_MMO_DMA 0x1u
+#define WQE_GGA_DECOMP_DEFLATE 0x0u
+#define WQE_GGA_DECOMP_LZ4 0x2u
+
+#define MLX5_GGA_DECOMP_LZ4_BLOCK_WITHOUT_CHECKSUM 0x1u
+#define MLX5_GGA_DECOMP_LZ4_BLOCK_WITH_CHECKSUM 0x2u
+
#define WQE_GGA_COMP_WIN_SIZE_OFFSET 12u
#define WQE_GGA_COMP_BLOCK_SIZE_OFFSET 16u
#define WQE_GGA_COMP_DYNAMIC_SIZE_OFFSET 20u
+#define WQE_GGA_DECOMP_PARAMS_OFFSET 20u
+#define WQE_GGA_DECOMP_TYPE_OFFSET 8u
+#define WQE_GGA_DECOMP_BLOCK_INDEPENDENT_OFFSET 22u
+
#define MLX5_GGA_COMP_WIN_SIZE_UNITS 1024u
#define MLX5_GGA_COMP_WIN_SIZE_MAX (32u * MLX5_GGA_COMP_WIN_SIZE_UNITS)
#define MLX5_GGA_COMP_LOG_BLOCK_SIZE_MAX 15u
@@ -599,7 +609,7 @@ struct mlx5_gga_wqe {
uint32_t opcode;
uint32_t sq_ds;
uint32_t flags;
- uint32_t gga_ctrl1; /* ws 12-15, bs 16-19, dyns 20-23. */
+ uint32_t gga_ctrl1;
uint32_t gga_ctrl2;
uint32_t opaque_lkey;
uint64_t opaque_vaddr;
@@ -1434,7 +1444,9 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 log_dma_mmo_size[0x5];
u8 reserved_at_70[0x3];
u8 log_compress_mmo_size[0x5];
- u8 reserved_at_78[0x3];
+ u8 decompress_lz4_data_only_v2[0x1];
+ u8 decompress_lz4_no_checksum_v2[0x1];
+ u8 decompress_lz4_checksum_v2[0x1];
u8 log_decompress_mmo_size[0x5];
u8 log_max_srq_sz[0x8];
u8 log_max_qp_sz[0x8];
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v2 8/8] compress/mlx5: add support for LZ4 algorithm
2023-02-02 16:25 ` [PATCH v2 0/8] compress/mlx5: add LZ4 support Michael Baum
` (6 preceding siblings ...)
2023-02-02 16:25 ` [PATCH v2 7/8] common/mlx5: add LZ4 capabilities check Michael Baum
@ 2023-02-02 16:25 ` Michael Baum
2023-02-21 7:07 ` [PATCH v3 0/8] compress/mlx5: add LZ4 support Michael Baum
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-02 16:25 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon
Add support for decompress LZ4 algorithm for mlx5 PMD.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
---
doc/guides/compressdevs/features/mlx5.ini | 18 ++-
doc/guides/compressdevs/mlx5.rst | 49 ++++++-
doc/guides/rel_notes/release_23_03.rst | 4 +
drivers/compress/mlx5/mlx5_compress.c | 150 ++++++++++++++++++----
4 files changed, 180 insertions(+), 41 deletions(-)
diff --git a/doc/guides/compressdevs/features/mlx5.ini b/doc/guides/compressdevs/features/mlx5.ini
index 891ce47936..28b050144a 100644
--- a/doc/guides/compressdevs/features/mlx5.ini
+++ b/doc/guides/compressdevs/features/mlx5.ini
@@ -4,10 +4,14 @@
; Supported features of 'MLX5' compression driver.
;
[Features]
-HW Accelerated = Y
-Deflate = Y
-Adler32 = Y
-Crc32 = Y
-Adler32&Crc32 = Y
-Fixed = Y
-Dynamic = Y
+HW Accelerated = Y
+Deflate = Y
+LZ4 = Y
+Adler32 = Y
+Crc32 = Y
+Adler32&Crc32 = Y
+xxHash32 = Y
+Fixed = Y
+Dynamic = Y
+LZ4 Block Checksum = Y
+LZ4 Block Independence = Y
diff --git a/doc/guides/compressdevs/mlx5.rst b/doc/guides/compressdevs/mlx5.rst
index 37839a59e3..c834025732 100644
--- a/doc/guides/compressdevs/mlx5.rst
+++ b/doc/guides/compressdevs/mlx5.rst
@@ -14,8 +14,8 @@ NVIDIA MLX5 Compress Driver
that are now NVIDIA trademarks.
The mlx5 compress driver library
-(**librte_compress_mlx5**) provides support for **NVIDIA BlueField-2**
-families of 25/50/100/200 Gb/s adapters.
+(**librte_compress_mlx5**) provides support for **NVIDIA BlueField-2** and
+**NVIDIA BlueField-3** families of 25/50/100/200 Gb/s adapters.
Design
------
@@ -39,11 +39,27 @@ Features
Compress mlx5 PMD has support for:
-Compression/Decompression algorithm:
+- Compression
+- Decompression
+- DMA
-* DEFLATE.
+Algorithms
+----------
-NULL algorithm for DMA operations.
+NULL algorithm
+~~~~~~~~~~~~~~
+
+NULL algorithm is the way to perform DMA operations.
+It works through either compress or decompress operation.
+
+Shareable transformation.
+
+Checksum generation:
+
+* CRC32, Adler32 and combined checksum.
+
+DEFLATE algorithm
+~~~~~~~~~~~~~~~~~
Huffman code type:
@@ -60,11 +76,31 @@ Checksum generation:
* CRC32, Adler32 and combined checksum.
+LZ4 algorithm
+~~~~~~~~~~~~~
+
+Support for flags:
+
+* ``RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM``
+* ``RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE``
+
+Window size support:
+
+1KB, 2KB, 4KB, 8KB, 16KB and 32KB.
+
+Shareable transformation.
+
+Checksum generation:
+
+* xxHash-32 checksum.
+
Limitations
-----------
* Scatter-Gather, SHA and Stateful are not supported.
* Non-compressed block is not supported in compress (supported in decompress).
+* Compress operation is not supported by BlueField-3.
+* LZ4 algorithm is not supported by BlueField-2.
Driver options
--------------
@@ -75,7 +111,7 @@ for an additional list of options shared with other mlx5 drivers.
- ``log-block-size`` parameter [int]
Log of the Huffman block size in the Deflate algorithm.
- Values from [4-15]; value x means block size is 2^x.
+ Values from [4-15]; value x means block size is 2\ :sup:`x`.
The default value is 15.
@@ -83,6 +119,7 @@ Supported NICs
--------------
* NVIDIA\ |reg| BlueField-2 SmartNIC
+* NVIDIA\ |reg| BlueField-3 SmartNIC
Prerequisites
-------------
diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
index aedc5767ff..3d97a4611d 100644
--- a/doc/guides/rel_notes/release_23_03.rst
+++ b/doc/guides/rel_notes/release_23_03.rst
@@ -86,6 +86,10 @@ New Features
* Added support for ``RTE_COMP_CHECKSUM_XXHASH32``.
* Added support for ``lz4`` in test-compress-perf algo options.
+* **Updated NVIDIA mlx5 compress PMD.**
+
+ * Added LZ4 algorithm support for decompress operation.
+
* **Allowed test single compress operation in test-compress-perf.**
Enable the application options for testing only compress and only decompress.
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index 7841f57b9c..e33b58ab54 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -24,6 +24,7 @@
#define MLX5_COMPRESS_DRIVER_NAME mlx5_compress
#define MLX5_COMPRESS_MAX_QPS 1024
#define MLX5_COMP_MAX_WIN_SIZE_CONF 6u
+#define MLX5_COMP_NUM_SUP_ALGO 4
struct mlx5_compress_devarg_params {
uint32_t log_block_sz;
@@ -43,6 +44,7 @@ struct mlx5_compress_priv {
struct mlx5_common_device *cdev; /* Backend mlx5 device. */
struct mlx5_uar uar;
struct rte_compressdev_config dev_config;
+ struct rte_compressdev_capabilities caps[MLX5_COMP_NUM_SUP_ALGO];
LIST_HEAD(xform_list, mlx5_compress_xform) xform_list;
rte_spinlock_t xform_sl;
uint32_t log_block_sz;
@@ -70,36 +72,16 @@ static pthread_mutex_t priv_list_lock = PTHREAD_MUTEX_INITIALIZER;
int mlx5_compress_logtype;
-static const struct rte_compressdev_capabilities mlx5_caps[] = {
- {
- .algo = RTE_COMP_ALGO_NULL,
- .comp_feature_flags = RTE_COMP_FF_ADLER32_CHECKSUM |
- RTE_COMP_FF_CRC32_CHECKSUM |
- RTE_COMP_FF_CRC32_ADLER32_CHECKSUM |
- RTE_COMP_FF_SHAREABLE_PRIV_XFORM,
- },
- {
- .algo = RTE_COMP_ALGO_DEFLATE,
- .comp_feature_flags = RTE_COMP_FF_ADLER32_CHECKSUM |
- RTE_COMP_FF_CRC32_CHECKSUM |
- RTE_COMP_FF_CRC32_ADLER32_CHECKSUM |
- RTE_COMP_FF_SHAREABLE_PRIV_XFORM |
- RTE_COMP_FF_HUFFMAN_FIXED |
- RTE_COMP_FF_HUFFMAN_DYNAMIC,
- .window_size = {.min = 10, .max = 15, .increment = 1},
- },
- RTE_COMP_END_OF_CAPABILITIES_LIST()
-};
-
static void
mlx5_compress_dev_info_get(struct rte_compressdev *dev,
struct rte_compressdev_info *info)
{
- RTE_SET_USED(dev);
- if (info != NULL) {
+ if (dev != NULL && info != NULL) {
+ struct mlx5_compress_priv *priv = dev->data->dev_private;
+
info->max_nb_queue_pairs = MLX5_COMPRESS_MAX_QPS;
info->feature_flags = RTE_COMPDEV_FF_HW_ACCELERATED;
- info->capabilities = mlx5_caps;
+ info->capabilities = priv->caps;
}
}
@@ -236,6 +218,8 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
qp_attr.num_of_receive_wqes = 0;
qp_attr.num_of_send_wqbbs = RTE_BIT32(log_ops_n);
qp_attr.mmo = attr->mmo_compress_qp_en || attr->mmo_dma_qp_en ||
+ attr->decomp_lz4_checksum_en ||
+ attr->decomp_lz4_no_checksum_en ||
attr->decomp_deflate_v1_en || attr->decomp_deflate_v2_en;
ret = mlx5_devx_qp_create(priv->cdev->ctx, &qp->qp,
qp_attr.num_of_send_wqbbs *
@@ -280,7 +264,11 @@ mlx5_compress_xform_validate(const struct rte_comp_xform *xform,
return -ENOTSUP;
} else if (!attr->mmo_compress_qp_en &&
!attr->mmo_compress_sq_en) {
- DRV_LOG(ERR, "Not enough capabilities to support compress operation, maybe old FW/OFED version?");
+ DRV_LOG(ERR, "Not enough capabilities to support compress operation.");
+ return -ENOTSUP;
+ }
+ if (xform->compress.algo == RTE_COMP_ALGO_LZ4) {
+ DRV_LOG(ERR, "LZ4 compression is not supported.");
return -ENOTSUP;
}
if (xform->compress.level == RTE_COMP_LEVEL_NONE) {
@@ -291,6 +279,10 @@ mlx5_compress_xform_validate(const struct rte_comp_xform *xform,
DRV_LOG(ERR, "SHA is not supported.");
return -ENOTSUP;
}
+ if (xform->compress.chksum == RTE_COMP_CHECKSUM_XXHASH32) {
+ DRV_LOG(ERR, "xxHash32 checksum isn't supported in compress operation.");
+ return -ENOTSUP;
+ }
break;
case RTE_COMP_DECOMPRESS:
switch (xform->decompress.algo) {
@@ -307,6 +299,37 @@ mlx5_compress_xform_validate(const struct rte_comp_xform *xform,
DRV_LOG(ERR, "Not enough capabilities to support decompress DEFLATE algorithm, maybe old FW/OFED version?");
return -ENOTSUP;
}
+ if (xform->decompress.chksum ==
+ RTE_COMP_CHECKSUM_XXHASH32) {
+ DRV_LOG(ERR, "DEFLATE algorithm doesn't support xxHash32 checksum.");
+ return -ENOTSUP;
+ }
+ break;
+ case RTE_COMP_ALGO_LZ4:
+ if (!attr->decomp_lz4_no_checksum_en &&
+ !attr->decomp_lz4_checksum_en) {
+ DRV_LOG(ERR, "Not enough capabilities to support decompress LZ4 algorithm, maybe old FW/OFED version?");
+ return -ENOTSUP;
+ }
+ if (xform->decompress.lz4.flags &
+ RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM) {
+ if (!attr->decomp_lz4_checksum_en) {
+ DRV_LOG(ERR, "Not enough capabilities to support decompress LZ4 block with checksum param, maybe old FW/OFED version?");
+ return -ENOTSUP;
+ }
+ } else {
+ if (!attr->decomp_lz4_no_checksum_en) {
+ DRV_LOG(ERR, "Not enough capabilities to support decompress LZ4 block without checksum param, maybe old FW/OFED version?");
+ return -ENOTSUP;
+ }
+ }
+ if (xform->decompress.chksum !=
+ RTE_COMP_CHECKSUM_XXHASH32 &&
+ xform->decompress.chksum !=
+ RTE_COMP_CHECKSUM_NONE) {
+ DRV_LOG(ERR, "LZ4 algorithm supports only xxHash32 checksum.");
+ return -ENOTSUP;
+ }
break;
default:
DRV_LOG(ERR, "Algorithm %u is not supported.",
@@ -383,6 +406,27 @@ mlx5_compress_xform_create(struct rte_compressdev *dev,
case RTE_COMP_ALGO_DEFLATE:
xfrm->opcode += MLX5_OPC_MOD_MMO_DECOMP <<
WQE_CSEG_OPC_MOD_OFFSET;
+ xfrm->gga_ctrl1 += WQE_GGA_DECOMP_DEFLATE <<
+ WQE_GGA_DECOMP_TYPE_OFFSET;
+ break;
+ case RTE_COMP_ALGO_LZ4:
+ xfrm->opcode += MLX5_OPC_MOD_MMO_DECOMP <<
+ WQE_CSEG_OPC_MOD_OFFSET;
+ xfrm->gga_ctrl1 += WQE_GGA_DECOMP_LZ4 <<
+ WQE_GGA_DECOMP_TYPE_OFFSET;
+ if (xform->decompress.lz4.flags &
+ RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM)
+ xfrm->gga_ctrl1 +=
+ MLX5_GGA_DECOMP_LZ4_BLOCK_WITH_CHECKSUM <<
+ WQE_GGA_DECOMP_PARAMS_OFFSET;
+ else
+ xfrm->gga_ctrl1 +=
+ MLX5_GGA_DECOMP_LZ4_BLOCK_WITHOUT_CHECKSUM
+ << WQE_GGA_DECOMP_PARAMS_OFFSET;
+ if (xform->decompress.lz4.flags &
+ RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE)
+ xfrm->gga_ctrl1 += 1u <<
+ WQE_GGA_DECOMP_BLOCK_INDEPENDENT_OFFSET;
break;
default:
goto err;
@@ -390,7 +434,7 @@ mlx5_compress_xform_create(struct rte_compressdev *dev,
xfrm->csum_type = xform->decompress.chksum;
break;
default:
- DRV_LOG(ERR, "Algorithm %u is not supported.", xform->type);
+ DRV_LOG(ERR, "Operation %u is not supported.", xform->type);
goto err;
}
DRV_LOG(DEBUG, "New xform: gga ctrl1 = 0x%08X opcode = 0x%08X csum "
@@ -657,6 +701,10 @@ mlx5_compress_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
((uint64_t)rte_be_to_cpu_32
(opaq[idx].data[crc32_idx + 1]) << 32);
break;
+ case RTE_COMP_CHECKSUM_XXHASH32:
+ op->output_chksum = (uint64_t)rte_be_to_cpu_32
+ (opaq[idx].v2.xxh32);
+ break;
default:
break;
}
@@ -720,6 +768,49 @@ mlx5_compress_handle_devargs(struct mlx5_kvargs_ctrl *mkvlist,
return 0;
}
+static void
+mlx5_compress_fill_caps(struct mlx5_compress_priv *priv,
+ const struct mlx5_hca_attr *attr)
+{
+ struct rte_compressdev_capabilities caps[] = {
+ {
+ .algo = RTE_COMP_ALGO_NULL,
+ .comp_feature_flags = RTE_COMP_FF_ADLER32_CHECKSUM |
+ RTE_COMP_FF_CRC32_CHECKSUM |
+ RTE_COMP_FF_CRC32_ADLER32_CHECKSUM |
+ RTE_COMP_FF_SHAREABLE_PRIV_XFORM,
+ },
+ {
+ .algo = RTE_COMP_ALGO_DEFLATE,
+ .comp_feature_flags = RTE_COMP_FF_ADLER32_CHECKSUM |
+ RTE_COMP_FF_CRC32_CHECKSUM |
+ RTE_COMP_FF_CRC32_ADLER32_CHECKSUM |
+ RTE_COMP_FF_SHAREABLE_PRIV_XFORM |
+ RTE_COMP_FF_HUFFMAN_FIXED |
+ RTE_COMP_FF_HUFFMAN_DYNAMIC,
+ .window_size = {.min = 10, .max = 15, .increment = 1},
+ },
+ {
+ .algo = RTE_COMP_ALGO_LZ4,
+ .comp_feature_flags = RTE_COMP_FF_XXHASH32_CHECKSUM |
+ RTE_COMP_FF_SHAREABLE_PRIV_XFORM |
+ RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE,
+ .window_size = {.min = 1, .max = 15, .increment = 1},
+ },
+ RTE_COMP_END_OF_CAPABILITIES_LIST()
+ };
+ priv->caps[0] = caps[0];
+ priv->caps[1] = caps[1];
+ if (attr->decomp_lz4_checksum_en || attr->decomp_lz4_no_checksum_en) {
+ priv->caps[2] = caps[2];
+ if (attr->decomp_lz4_checksum_en)
+ priv->caps[2].comp_feature_flags |=
+ RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM;
+ priv->caps[3] = caps[3];
+ } else
+ priv->caps[2] = caps[3];
+}
+
static int
mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
struct mlx5_kvargs_ctrl *mkvlist)
@@ -740,7 +831,8 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
rte_errno = ENOTSUP;
return -rte_errno;
}
- if (!attr->decomp_deflate_v1_en && !attr->decomp_deflate_v2_en &&
+ if (!attr->decomp_lz4_checksum_en && !attr->decomp_lz4_no_checksum_en &&
+ !attr->decomp_deflate_v1_en && !attr->decomp_deflate_v2_en &&
!attr->mmo_decompress_sq_en && !attr->mmo_compress_qp_en &&
!attr->mmo_compress_sq_en && !attr->mmo_dma_qp_en &&
!attr->mmo_dma_sq_en) {
@@ -763,7 +855,8 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
compressdev->feature_flags = RTE_COMPDEV_FF_HW_ACCELERATED;
priv = compressdev->data->dev_private;
priv->log_block_sz = devarg_prms.log_block_sz;
- if (attr->decomp_deflate_v2_en)
+ if (attr->decomp_deflate_v2_en || attr->decomp_lz4_checksum_en ||
+ attr->decomp_lz4_no_checksum_en)
crc32_opaq_offset = offsetof(union mlx5_gga_compress_opaque,
v2.crc32);
else
@@ -773,6 +866,7 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
priv->crc32_opaq_offs = crc32_opaq_offset / 4;
priv->cdev = cdev;
priv->compressdev = compressdev;
+ mlx5_compress_fill_caps(priv, attr);
if (mlx5_devx_uar_prepare(cdev, &priv->uar) != 0) {
rte_compressdev_pmd_destroy(priv->compressdev);
return -1;
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v3 0/8] compress/mlx5: add LZ4 support
2023-02-02 16:25 ` [PATCH v2 0/8] compress/mlx5: add LZ4 support Michael Baum
` (7 preceding siblings ...)
2023-02-02 16:25 ` [PATCH v2 8/8] compress/mlx5: add support for LZ4 algorithm Michael Baum
@ 2023-02-21 7:07 ` Michael Baum
2023-02-21 7:07 ` [PATCH v3 1/8] compress/mlx5: fix decompress xform validation Michael Baum
` (8 more replies)
8 siblings, 9 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-21 7:07 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon
Add support for LZ4 decompression algorithm for mlx5 compress PMD.
Complete the support of compress PMD for Bluefield-3 smart-nic.
RFC:
https://patchwork.dpdk.org/project/dpdk/patch/20220410182622.8828-1-rzidane@nvidia.com/
v2:
- Rebase.
- Update PRM names.
- Add a new fix at the beginning of the patch-set.
v3:
- Rebase.
- Fix missing "!" in decompress xform validation for null algo.
- Change DEFLATE checksum validation for decompress xform.
Michael Baum (8):
compress/mlx5: fix decompress xform validation
compress/mlx5: fix wrong output Adler-32 checksum offset
compress/mlx5: fix QP setup for partial transformations
compress/mlx5: support new metadata layout added in BF3
compress/mlx5: remove unused variable from priv structure
compress/mlx5: add xform validate function
common/mlx5: add LZ4 capabilities check
compress/mlx5: add support for LZ4 algorithm
doc/guides/compressdevs/features/mlx5.ini | 18 +-
doc/guides/compressdevs/mlx5.rst | 44 +++-
doc/guides/rel_notes/release_23_03.rst | 4 +
drivers/common/mlx5/mlx5_devx_cmds.c | 12 +-
drivers/common/mlx5/mlx5_devx_cmds.h | 6 +-
drivers/common/mlx5/mlx5_prm.h | 56 +++--
drivers/compress/mlx5/mlx5_compress.c | 274 ++++++++++++++++------
7 files changed, 307 insertions(+), 107 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 27+ messages in thread* [PATCH v3 1/8] compress/mlx5: fix decompress xform validation
2023-02-21 7:07 ` [PATCH v3 0/8] compress/mlx5: add LZ4 support Michael Baum
@ 2023-02-21 7:07 ` Michael Baum
2023-02-21 7:07 ` [PATCH v3 2/8] compress/mlx5: fix wrong output Adler-32 checksum offset Michael Baum
` (7 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-21 7:07 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon, rzidane, stable
In xform creation, it first validate the xform according the
capabilities.
One of validations verifies that given "hash_algo" is
"RTE_COMP_HASH_ALGO_NONE" for both compress and decompress xform
objects.
However, the validation for decompress checks it again for compress
xform object.
This patch changes it to verify decompress xform object.
Fixes: 2efd26544554 ("compress/mlx5: support partial transformation")
Cc: rzidane@nvidia.com
Cc: stable@dpdk.org
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
drivers/compress/mlx5/mlx5_compress.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index 430f769d41..06d1ff5b95 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -313,7 +313,7 @@ mlx5_compress_xform_create(struct rte_compressdev *dev,
DRV_LOG(ERR, "Not enough capabilities to support decompress operation, maybe old FW/OFED version?");
return -ENOTSUP;
}
- if (xform->compress.hash_algo != RTE_COMP_HASH_ALGO_NONE) {
+ if (xform->decompress.hash_algo != RTE_COMP_HASH_ALGO_NONE) {
DRV_LOG(ERR, "SHA is not supported.");
return -ENOTSUP;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v3 2/8] compress/mlx5: fix wrong output Adler-32 checksum offset
2023-02-21 7:07 ` [PATCH v3 0/8] compress/mlx5: add LZ4 support Michael Baum
2023-02-21 7:07 ` [PATCH v3 1/8] compress/mlx5: fix decompress xform validation Michael Baum
@ 2023-02-21 7:07 ` Michael Baum
2023-02-21 7:07 ` [PATCH v3 3/8] compress/mlx5: fix QP setup for partial transformations Michael Baum
` (6 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-21 7:07 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon, stable
After de/compress dequeue, the output checksum is copied into the op
structure. The "output_checksum" field in op structure is "uint64_t"
type, and the 32-bit checksums (CRC32, Adler-32) are copied into the
lower 32 bits.
When both CRC32 and Adler-32 are configured, CRC32 is copied into the
lower 32 bits and Adler-32 into the upper 32 bits.
However, in mlx5 PMD Adler-32 without CRC, is mistakenly copied into the
upper 32 bits.
This patch updates Adler-32 output checksun to be copied into the
lower 32 bits.
Fixes: f8c97babc9f4 ("compress/mlx5: add data-path functions")
Cc: matan@nvidia.com
Cc: stable@dpdk.org
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
drivers/compress/mlx5/mlx5_compress.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index 06d1ff5b95..82088a7b8c 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -633,7 +633,7 @@ mlx5_compress_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
break;
case RTE_COMP_CHECKSUM_ADLER32:
op->output_chksum = (uint64_t)rte_be_to_cpu_32
- (opaq[idx].adler32) << 32;
+ (opaq[idx].adler32);
break;
case RTE_COMP_CHECKSUM_CRC32_ADLER32:
op->output_chksum = (uint64_t)rte_be_to_cpu_32
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v3 3/8] compress/mlx5: fix QP setup for partial transformations
2023-02-21 7:07 ` [PATCH v3 0/8] compress/mlx5: add LZ4 support Michael Baum
2023-02-21 7:07 ` [PATCH v3 1/8] compress/mlx5: fix decompress xform validation Michael Baum
2023-02-21 7:07 ` [PATCH v3 2/8] compress/mlx5: fix wrong output Adler-32 checksum offset Michael Baum
@ 2023-02-21 7:07 ` Michael Baum
2023-02-21 7:07 ` [PATCH v3 4/8] compress/mlx5: support new metadata layout added in BF3 Michael Baum
` (5 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-21 7:07 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon, rzidane, stable
The mlx5_compress_qp_setup() function creates QP for compress,
decompress and DMA. Thus, the MMO flag is turned on only when all
operations are supported.
However, since partial transformations have been allowed, it should be
turn on for part of them.
This patch removes the compress MMO support requirement.
Fixes: 2efd26544554 ("compress/mlx5: support partial transformation")
Cc: rzidane@nvidia.com
Cc: stable@dpdk.org
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
drivers/compress/mlx5/mlx5_compress.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index 82088a7b8c..6882bcf6d1 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -243,8 +243,8 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
mlx5_ts_format_conv(priv->cdev->config.hca_attr.qp_ts_format);
qp_attr.num_of_receive_wqes = 0;
qp_attr.num_of_send_wqbbs = RTE_BIT32(log_ops_n);
- qp_attr.mmo = priv->mmo_decomp_qp && priv->mmo_comp_qp
- && priv->mmo_dma_qp;
+ qp_attr.mmo = priv->mmo_decomp_qp || priv->mmo_comp_qp ||
+ priv->mmo_dma_qp;
ret = mlx5_devx_qp_create(priv->cdev->ctx, &qp->qp,
qp_attr.num_of_send_wqbbs *
MLX5_WQE_SIZE, &qp_attr, socket_id);
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v3 4/8] compress/mlx5: support new metadata layout added in BF3
2023-02-21 7:07 ` [PATCH v3 0/8] compress/mlx5: add LZ4 support Michael Baum
` (2 preceding siblings ...)
2023-02-21 7:07 ` [PATCH v3 3/8] compress/mlx5: fix QP setup for partial transformations Michael Baum
@ 2023-02-21 7:07 ` Michael Baum
2023-02-21 7:07 ` [PATCH v3 5/8] compress/mlx5: remove unused variable from priv structure Michael Baum
` (4 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-21 7:07 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon
Commit [1] add support in Bluefield-3, but Bluefield-3 has different
GGA opaque structure than Bluefield-2.
This patch updates the PRM structure to include both versions, and
culculate the relevant offset for each version in control path.
Commit [1] 559014f232b4 ("compress/mlx5: add Bluefield-3 device ID")
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
drivers/common/mlx5/mlx5_devx_cmds.c | 6 ++--
drivers/common/mlx5/mlx5_devx_cmds.h | 3 +-
drivers/common/mlx5/mlx5_prm.h | 40 +++++++++++++++---------
drivers/compress/mlx5/mlx5_compress.c | 45 +++++++++++++++++----------
4 files changed, 61 insertions(+), 33 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index fb670950ef..b801b05872 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -987,8 +987,10 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
attr->mmo_dma_qp_en = MLX5_GET(cmd_hca_cap, hcattr, dma_mmo_qp);
attr->mmo_compress_qp_en = MLX5_GET(cmd_hca_cap, hcattr,
compress_mmo_qp);
- attr->mmo_decompress_qp_en = MLX5_GET(cmd_hca_cap, hcattr,
- decompress_mmo_qp);
+ attr->decomp_deflate_v1_en = MLX5_GET(cmd_hca_cap, hcattr,
+ decompress_deflate_v1);
+ attr->decomp_deflate_v2_en = MLX5_GET(cmd_hca_cap, hcattr,
+ decompress_deflate_v2);
attr->compress_min_block_size = MLX5_GET(cmd_hca_cap, hcattr,
compress_min_block_size);
attr->log_max_mmo_dma = MLX5_GET(cmd_hca_cap, hcattr, log_dma_mmo_size);
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 6b5a6b8667..045ff404e9 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -263,7 +263,8 @@ struct mlx5_hca_attr {
uint32_t mmo_decompress_sq_en:1;
uint32_t mmo_dma_qp_en:1;
uint32_t mmo_compress_qp_en:1;
- uint32_t mmo_decompress_qp_en:1;
+ uint32_t decomp_deflate_v1_en:1;
+ uint32_t decomp_deflate_v2_en:1;
uint32_t mmo_regex_qp_en:1;
uint32_t mmo_regex_sq_en:1;
uint32_t compress_min_block_size:4;
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index 613cc6face..b33c55c257 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -613,17 +613,27 @@ struct mlx5_gga_wqe {
struct mlx5_wqe_dseg scatter;
} __rte_packed;
-struct mlx5_gga_compress_opaque {
- uint32_t syndrom;
- uint32_t reserved0;
- uint32_t scattered_length;
- uint32_t gathered_length;
- uint64_t scatter_crc;
- uint64_t gather_crc;
- uint32_t crc32;
- uint32_t adler32;
- uint8_t reserved1[216];
-} __rte_packed;
+union mlx5_gga_compress_opaque {
+ struct {
+ uint32_t syndrome;
+ uint32_t reserved0;
+ uint32_t scattered_length;
+ union {
+ struct {
+ uint32_t reserved1[5];
+ uint32_t crc32;
+ uint32_t adler32;
+ } v1 __rte_packed;
+ struct {
+ uint32_t crc32;
+ uint32_t adler32;
+ uint32_t crc32c;
+ uint32_t xxh32;
+ } v2 __rte_packed;
+ };
+ } __rte_packed;
+ uint32_t data[64];
+};
struct mlx5_ifc_regexp_mmo_control_bits {
uint8_t reserved_at_31[0x2];
@@ -1467,7 +1477,7 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 log_max_bsf_list_size[0x6];
u8 umr_extended_translation_offset[0x1];
u8 null_mkey[0x1];
- u8 log_max_klm_list_size[0x6];
+ u8 log_maxklm_list_size[0x6];
u8 non_wire_sq[0x1];
u8 reserved_at_121[0x9];
u8 log_max_ra_req_dc[0x6];
@@ -1756,8 +1766,10 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 dma_mmo_qp[0x1];
u8 regexp_mmo_qp[0x1];
u8 compress_mmo_qp[0x1];
- u8 decompress_mmo_qp[0x1];
- u8 reserved_at_74c[0x14];
+ u8 decompress_deflate_v1[0x1];
+ u8 reserved_at_74c[0x4];
+ u8 decompress_deflate_v2[0x1];
+ u8 reserved_at_751[0xf];
u8 reserved_at_760[0x3];
u8 log_max_num_header_modify_argument[0x5];
u8 log_header_modify_argument_granularity_offset[0x4];
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index 6882bcf6d1..93721d6047 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -55,6 +55,7 @@ struct mlx5_compress_priv {
uint32_t mmo_dma_sq:1;
uint32_t mmo_dma_qp:1;
uint32_t log_block_sz;
+ uint32_t crc32_opaq_offs;
};
struct mlx5_compress_qp {
@@ -157,7 +158,7 @@ mlx5_compress_init_qp(struct mlx5_compress_qp *qp)
{
volatile struct mlx5_gga_wqe *restrict wqe =
(volatile struct mlx5_gga_wqe *)qp->qp.wqes;
- volatile struct mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
+ volatile union mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
const uint32_t sq_ds = rte_cpu_to_be_32((qp->qp.qp->id << 8) | 4u);
const uint32_t flags = RTE_BE32(MLX5_COMP_ALWAYS <<
MLX5_COMP_MODE_OFFSET);
@@ -211,8 +212,8 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
goto err;
}
opaq_buf = rte_calloc(__func__, (size_t)1 << log_ops_n,
- sizeof(struct mlx5_gga_compress_opaque),
- sizeof(struct mlx5_gga_compress_opaque));
+ sizeof(union mlx5_gga_compress_opaque),
+ sizeof(union mlx5_gga_compress_opaque));
if (opaq_buf == NULL) {
DRV_LOG(ERR, "Failed to allocate opaque memory.");
rte_errno = ENOMEM;
@@ -225,7 +226,7 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
qp->ops = (struct rte_comp_op **)RTE_ALIGN((uintptr_t)(qp + 1),
RTE_CACHE_LINE_SIZE);
if (mlx5_common_verbs_reg_mr(priv->cdev->pd, opaq_buf, qp->entries_n *
- sizeof(struct mlx5_gga_compress_opaque),
+ sizeof(union mlx5_gga_compress_opaque),
&qp->opaque_mr) != 0) {
rte_free(opaq_buf);
DRV_LOG(ERR, "Failed to register opaque MR.");
@@ -544,7 +545,7 @@ mlx5_compress_dump_err_objs(volatile uint32_t *cqe, volatile uint32_t *wqe,
DRV_LOG(ERR, "%08X %08X %08X %08X", wqe[i], wqe[i + 1],
wqe[i + 2], wqe[i + 3]);
DRV_LOG(ERR, "\nError opaq:");
- for (i = 0; i < sizeof(struct mlx5_gga_compress_opaque) >> 2; i += 4)
+ for (i = 0; i < sizeof(union mlx5_gga_compress_opaque) >> 2; i += 4)
DRV_LOG(ERR, "%08X %08X %08X %08X", opaq[i], opaq[i + 1],
opaq[i + 2], opaq[i + 3]);
}
@@ -558,7 +559,7 @@ mlx5_compress_cqe_err_handle(struct mlx5_compress_qp *qp,
&qp->cq.cqes[idx];
volatile struct mlx5_gga_wqe *wqes = (volatile struct mlx5_gga_wqe *)
qp->qp.wqes;
- volatile struct mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
+ volatile union mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
volatile uint32_t *synd_word = RTE_PTR_ADD(cqe, MLX5_ERROR_CQE_SYNDROME_OFFSET);
switch (*synd_word) {
@@ -575,7 +576,7 @@ mlx5_compress_cqe_err_handle(struct mlx5_compress_qp *qp,
op->consumed = 0;
op->produced = 0;
op->output_chksum = 0;
- op->debug_status = rte_be_to_cpu_32(opaq[idx].syndrom) |
+ op->debug_status = rte_be_to_cpu_32(opaq[idx].syndrome) |
((uint64_t)rte_be_to_cpu_32(cqe->syndrome) << 32);
mlx5_compress_dump_err_objs((volatile uint32_t *)cqe,
(volatile uint32_t *)&wqes[idx],
@@ -590,13 +591,14 @@ mlx5_compress_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
struct mlx5_compress_qp *qp = queue_pair;
volatile struct mlx5_compress_xform *restrict xform;
volatile struct mlx5_cqe *restrict cqe;
- volatile struct mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
+ volatile union mlx5_gga_compress_opaque *opaq = qp->opaque_mr.addr;
struct rte_comp_op *restrict op;
const unsigned int cq_size = qp->entries_n;
const unsigned int mask = cq_size - 1;
uint32_t idx;
uint32_t next_idx = qp->ci & mask;
const uint16_t max = RTE_MIN((uint16_t)(qp->pi - qp->ci), nb_ops);
+ uint32_t crc32_idx = qp->priv->crc32_opaq_offs;
uint16_t i = 0;
int ret;
@@ -629,17 +631,17 @@ mlx5_compress_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
switch (xform->csum_type) {
case RTE_COMP_CHECKSUM_CRC32:
op->output_chksum = (uint64_t)rte_be_to_cpu_32
- (opaq[idx].crc32);
+ (opaq[idx].data[crc32_idx]);
break;
case RTE_COMP_CHECKSUM_ADLER32:
op->output_chksum = (uint64_t)rte_be_to_cpu_32
- (opaq[idx].adler32);
+ (opaq[idx].data[crc32_idx + 1]);
break;
case RTE_COMP_CHECKSUM_CRC32_ADLER32:
op->output_chksum = (uint64_t)rte_be_to_cpu_32
- (opaq[idx].crc32) |
+ (opaq[idx].data[crc32_idx]) |
((uint64_t)rte_be_to_cpu_32
- (opaq[idx].adler32) << 32);
+ (opaq[idx].data[crc32_idx + 1]) << 32);
break;
default:
break;
@@ -717,15 +719,17 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
.socket_id = cdev->dev->numa_node,
};
const char *ibdev_name = mlx5_os_get_ctx_device_name(cdev->ctx);
+ uint32_t crc32_opaq_offset;
if (rte_eal_process_type() != RTE_PROC_PRIMARY) {
DRV_LOG(ERR, "Non-primary process type is not supported.");
rte_errno = ENOTSUP;
return -rte_errno;
}
- if (!attr->mmo_decompress_qp_en && !attr->mmo_decompress_sq_en
- && !attr->mmo_compress_qp_en && !attr->mmo_compress_sq_en
- && !attr->mmo_dma_qp_en && !attr->mmo_dma_sq_en) {
+ if (!attr->decomp_deflate_v1_en && !attr->decomp_deflate_v2_en &&
+ !attr->mmo_decompress_sq_en && !attr->mmo_compress_qp_en &&
+ !attr->mmo_compress_sq_en && !attr->mmo_dma_qp_en &&
+ !attr->mmo_dma_sq_en) {
DRV_LOG(ERR, "Not enough capabilities to support compress operations, maybe old FW/OFED version?");
rte_errno = ENOTSUP;
return -ENOTSUP;
@@ -746,11 +750,20 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
priv = compressdev->data->dev_private;
priv->log_block_sz = devarg_prms.log_block_sz;
priv->mmo_decomp_sq = attr->mmo_decompress_sq_en;
- priv->mmo_decomp_qp = attr->mmo_decompress_qp_en;
+ priv->mmo_decomp_qp =
+ attr->decomp_deflate_v1_en | attr->decomp_deflate_v2_en;
priv->mmo_comp_sq = attr->mmo_compress_sq_en;
priv->mmo_comp_qp = attr->mmo_compress_qp_en;
priv->mmo_dma_sq = attr->mmo_dma_sq_en;
priv->mmo_dma_qp = attr->mmo_dma_qp_en;
+ if (attr->decomp_deflate_v2_en)
+ crc32_opaq_offset = offsetof(union mlx5_gga_compress_opaque,
+ v2.crc32);
+ else
+ crc32_opaq_offset = offsetof(union mlx5_gga_compress_opaque,
+ v1.crc32);
+ MLX5_ASSERT((crc32_opaq_offset % 4) == 0);
+ priv->crc32_opaq_offs = crc32_opaq_offset / 4;
priv->cdev = cdev;
priv->compressdev = compressdev;
priv->min_block_size = attr->compress_min_block_size;
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v3 5/8] compress/mlx5: remove unused variable from priv structure
2023-02-21 7:07 ` [PATCH v3 0/8] compress/mlx5: add LZ4 support Michael Baum
` (3 preceding siblings ...)
2023-02-21 7:07 ` [PATCH v3 4/8] compress/mlx5: support new metadata layout added in BF3 Michael Baum
@ 2023-02-21 7:07 ` Michael Baum
2023-02-21 7:07 ` [PATCH v3 6/8] compress/mlx5: add xform validate function Michael Baum
` (3 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-21 7:07 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon
The priv structure has variable named "min_block_size" coming from HCA
capabilities.
This field isn't used and copied into the priv structure for free.
This patch removes this field.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
drivers/compress/mlx5/mlx5_compress.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index 93721d6047..1802a7ba52 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -42,8 +42,6 @@ struct mlx5_compress_priv {
struct rte_compressdev *compressdev;
struct mlx5_common_device *cdev; /* Backend mlx5 device. */
struct mlx5_uar uar;
- uint8_t min_block_size;
- /* Minimum huffman block size supported by the device. */
struct rte_compressdev_config dev_config;
LIST_HEAD(xform_list, mlx5_compress_xform) xform_list;
rte_spinlock_t xform_sl;
@@ -766,7 +764,6 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
priv->crc32_opaq_offs = crc32_opaq_offset / 4;
priv->cdev = cdev;
priv->compressdev = compressdev;
- priv->min_block_size = attr->compress_min_block_size;
if (mlx5_devx_uar_prepare(cdev, &priv->uar) != 0) {
rte_compressdev_pmd_destroy(priv->compressdev);
return -1;
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v3 6/8] compress/mlx5: add xform validate function
2023-02-21 7:07 ` [PATCH v3 0/8] compress/mlx5: add LZ4 support Michael Baum
` (4 preceding siblings ...)
2023-02-21 7:07 ` [PATCH v3 5/8] compress/mlx5: remove unused variable from priv structure Michael Baum
@ 2023-02-21 7:07 ` Michael Baum
2023-02-21 7:07 ` [PATCH v3 7/8] common/mlx5: add LZ4 capabilities check Michael Baum
` (2 subsequent siblings)
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-21 7:07 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon
Separate the xform validations from create function into new function.
The new function checks caps directly from HCA attr structure, so all
capabilities in priv structure were removed.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
drivers/compress/mlx5/mlx5_compress.c | 75 +++++++++++++++------------
1 file changed, 42 insertions(+), 33 deletions(-)
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index 1802a7ba52..3ea3447f11 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -45,13 +45,6 @@ struct mlx5_compress_priv {
struct rte_compressdev_config dev_config;
LIST_HEAD(xform_list, mlx5_compress_xform) xform_list;
rte_spinlock_t xform_sl;
- /* HCA caps */
- uint32_t mmo_decomp_sq:1;
- uint32_t mmo_decomp_qp:1;
- uint32_t mmo_comp_sq:1;
- uint32_t mmo_comp_qp:1;
- uint32_t mmo_dma_sq:1;
- uint32_t mmo_dma_qp:1;
uint32_t log_block_sz;
uint32_t crc32_opaq_offs;
};
@@ -178,6 +171,7 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
uint32_t max_inflight_ops, int socket_id)
{
struct mlx5_compress_priv *priv = dev->data->dev_private;
+ struct mlx5_hca_attr *attr = &priv->cdev->config.hca_attr;
struct mlx5_compress_qp *qp;
struct mlx5_devx_cq_attr cq_attr = {
.uar_page_id = mlx5_os_get_devx_uar_page_id(priv->uar.obj),
@@ -238,12 +232,11 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
goto err;
}
qp_attr.cqn = qp->cq.cq->id;
- qp_attr.ts_format =
- mlx5_ts_format_conv(priv->cdev->config.hca_attr.qp_ts_format);
+ qp_attr.ts_format = mlx5_ts_format_conv(attr->qp_ts_format);
qp_attr.num_of_receive_wqes = 0;
qp_attr.num_of_send_wqbbs = RTE_BIT32(log_ops_n);
- qp_attr.mmo = priv->mmo_decomp_qp || priv->mmo_comp_qp ||
- priv->mmo_dma_qp;
+ qp_attr.mmo = attr->mmo_compress_qp_en || attr->mmo_dma_qp_en ||
+ attr->decomp_deflate_v1_en || attr->decomp_deflate_v2_en;
ret = mlx5_devx_qp_create(priv->cdev->ctx, &qp->qp,
qp_attr.num_of_send_wqbbs *
MLX5_WQE_SIZE, &qp_attr, socket_id);
@@ -276,21 +269,17 @@ mlx5_compress_xform_free(struct rte_compressdev *dev, void *xform)
}
static int
-mlx5_compress_xform_create(struct rte_compressdev *dev,
- const struct rte_comp_xform *xform,
- void **private_xform)
+mlx5_compress_xform_validate(const struct rte_comp_xform *xform,
+ const struct mlx5_hca_attr *attr)
{
- struct mlx5_compress_priv *priv = dev->data->dev_private;
- struct mlx5_compress_xform *xfrm;
- uint32_t size;
-
switch (xform->type) {
case RTE_COMP_COMPRESS:
if (xform->compress.algo == RTE_COMP_ALGO_NULL &&
- !priv->mmo_dma_qp && !priv->mmo_dma_sq) {
+ !attr->mmo_dma_qp_en && !attr->mmo_dma_sq_en) {
DRV_LOG(ERR, "Not enough capabilities to support DMA operation, maybe old FW/OFED version?");
return -ENOTSUP;
- } else if (!priv->mmo_comp_qp && !priv->mmo_comp_sq) {
+ } else if (!attr->mmo_compress_qp_en &&
+ !attr->mmo_compress_sq_en) {
DRV_LOG(ERR, "Not enough capabilities to support compress operation, maybe old FW/OFED version?");
return -ENOTSUP;
}
@@ -304,12 +293,24 @@ mlx5_compress_xform_create(struct rte_compressdev *dev,
}
break;
case RTE_COMP_DECOMPRESS:
- if (xform->decompress.algo == RTE_COMP_ALGO_NULL &&
- !priv->mmo_dma_qp && !priv->mmo_dma_sq) {
- DRV_LOG(ERR, "Not enough capabilities to support DMA operation, maybe old FW/OFED version?");
- return -ENOTSUP;
- } else if (!priv->mmo_decomp_qp && !priv->mmo_decomp_sq) {
- DRV_LOG(ERR, "Not enough capabilities to support decompress operation, maybe old FW/OFED version?");
+ switch (xform->decompress.algo) {
+ case RTE_COMP_ALGO_NULL:
+ if (!attr->mmo_dma_qp_en && !attr->mmo_dma_sq_en) {
+ DRV_LOG(ERR, "Not enough capabilities to support DMA operation, maybe old FW/OFED version?");
+ return -ENOTSUP;
+ }
+ break;
+ case RTE_COMP_ALGO_DEFLATE:
+ if (!attr->decomp_deflate_v1_en &&
+ !attr->decomp_deflate_v2_en &&
+ !attr->mmo_decompress_sq_en) {
+ DRV_LOG(ERR, "Not enough capabilities to support decompress DEFLATE algorithm, maybe old FW/OFED version?");
+ return -ENOTSUP;
+ }
+ break;
+ default:
+ DRV_LOG(ERR, "Algorithm %u is not supported.",
+ xform->decompress.algo);
return -ENOTSUP;
}
if (xform->decompress.hash_algo != RTE_COMP_HASH_ALGO_NONE) {
@@ -321,7 +322,22 @@ mlx5_compress_xform_create(struct rte_compressdev *dev,
DRV_LOG(ERR, "Xform type should be compress/decompress");
return -ENOTSUP;
}
+ return 0;
+}
+
+static int
+mlx5_compress_xform_create(struct rte_compressdev *dev,
+ const struct rte_comp_xform *xform,
+ void **private_xform)
+{
+ struct mlx5_compress_priv *priv = dev->data->dev_private;
+ struct mlx5_compress_xform *xfrm;
+ uint32_t size;
+ int ret;
+ ret = mlx5_compress_xform_validate(xform, &priv->cdev->config.hca_attr);
+ if (ret < 0)
+ return ret;
xfrm = rte_zmalloc_socket(__func__, sizeof(*xfrm), 0,
priv->dev_config.socket_id);
if (xfrm == NULL)
@@ -747,13 +763,6 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
compressdev->feature_flags = RTE_COMPDEV_FF_HW_ACCELERATED;
priv = compressdev->data->dev_private;
priv->log_block_sz = devarg_prms.log_block_sz;
- priv->mmo_decomp_sq = attr->mmo_decompress_sq_en;
- priv->mmo_decomp_qp =
- attr->decomp_deflate_v1_en | attr->decomp_deflate_v2_en;
- priv->mmo_comp_sq = attr->mmo_compress_sq_en;
- priv->mmo_comp_qp = attr->mmo_compress_qp_en;
- priv->mmo_dma_sq = attr->mmo_dma_sq_en;
- priv->mmo_dma_qp = attr->mmo_dma_qp_en;
if (attr->decomp_deflate_v2_en)
crc32_opaq_offset = offsetof(union mlx5_gga_compress_opaque,
v2.crc32);
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v3 7/8] common/mlx5: add LZ4 capabilities check
2023-02-21 7:07 ` [PATCH v3 0/8] compress/mlx5: add LZ4 support Michael Baum
` (5 preceding siblings ...)
2023-02-21 7:07 ` [PATCH v3 6/8] compress/mlx5: add xform validate function Michael Baum
@ 2023-02-21 7:07 ` Michael Baum
2023-02-21 7:07 ` [PATCH v3 8/8] compress/mlx5: add support for LZ4 algorithm Michael Baum
2023-02-27 18:17 ` [EXT] [PATCH v3 0/8] compress/mlx5: add LZ4 support Akhil Goyal
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-21 7:07 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon
Add capabilities check for LZ4 decompression algorithm.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
drivers/common/mlx5/mlx5_devx_cmds.c | 6 ++++++
drivers/common/mlx5/mlx5_devx_cmds.h | 3 +++
drivers/common/mlx5/mlx5_prm.h | 16 ++++++++++++++--
3 files changed, 23 insertions(+), 2 deletions(-)
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.c b/drivers/common/mlx5/mlx5_devx_cmds.c
index b801b05872..bd0e1c6398 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.c
+++ b/drivers/common/mlx5/mlx5_devx_cmds.c
@@ -998,6 +998,12 @@ mlx5_devx_cmd_query_hca_attr(void *ctx,
log_compress_mmo_size);
attr->log_max_mmo_decompress = MLX5_GET(cmd_hca_cap, hcattr,
log_decompress_mmo_size);
+ attr->decomp_lz4_data_only_en = MLX5_GET(cmd_hca_cap, hcattr,
+ decompress_lz4_data_only_v2);
+ attr->decomp_lz4_no_checksum_en = MLX5_GET(cmd_hca_cap, hcattr,
+ decompress_lz4_no_checksum_v2);
+ attr->decomp_lz4_checksum_en = MLX5_GET(cmd_hca_cap, hcattr,
+ decompress_lz4_checksum_v2);
attr->cqe_compression = MLX5_GET(cmd_hca_cap, hcattr, cqe_compression);
attr->mini_cqe_resp_flow_tag = MLX5_GET(cmd_hca_cap, hcattr,
mini_cqe_resp_flow_tag);
diff --git a/drivers/common/mlx5/mlx5_devx_cmds.h b/drivers/common/mlx5/mlx5_devx_cmds.h
index 045ff404e9..96c4c1a819 100644
--- a/drivers/common/mlx5/mlx5_devx_cmds.h
+++ b/drivers/common/mlx5/mlx5_devx_cmds.h
@@ -271,6 +271,9 @@ struct mlx5_hca_attr {
uint32_t log_max_mmo_dma:5;
uint32_t log_max_mmo_compress:5;
uint32_t log_max_mmo_decompress:5;
+ uint32_t decomp_lz4_data_only_en:1;
+ uint32_t decomp_lz4_no_checksum_en:1;
+ uint32_t decomp_lz4_checksum_en:1;
uint32_t umr_modify_entity_size_disabled:1;
uint32_t umr_indirect_mkey_disabled:1;
uint32_t log_min_stride_wqe_sz:5;
diff --git a/drivers/common/mlx5/mlx5_prm.h b/drivers/common/mlx5/mlx5_prm.h
index b33c55c257..75af636f59 100644
--- a/drivers/common/mlx5/mlx5_prm.h
+++ b/drivers/common/mlx5/mlx5_prm.h
@@ -584,9 +584,19 @@ struct mlx5_rdma_write_wqe {
#define MLX5_OPC_MOD_MMO_DECOMP 0x3u
#define MLX5_OPC_MOD_MMO_DMA 0x1u
+#define WQE_GGA_DECOMP_DEFLATE 0x0u
+#define WQE_GGA_DECOMP_LZ4 0x2u
+
+#define MLX5_GGA_DECOMP_LZ4_BLOCK_WITHOUT_CHECKSUM 0x1u
+#define MLX5_GGA_DECOMP_LZ4_BLOCK_WITH_CHECKSUM 0x2u
+
#define WQE_GGA_COMP_WIN_SIZE_OFFSET 12u
#define WQE_GGA_COMP_BLOCK_SIZE_OFFSET 16u
#define WQE_GGA_COMP_DYNAMIC_SIZE_OFFSET 20u
+#define WQE_GGA_DECOMP_PARAMS_OFFSET 20u
+#define WQE_GGA_DECOMP_TYPE_OFFSET 8u
+#define WQE_GGA_DECOMP_BLOCK_INDEPENDENT_OFFSET 22u
+
#define MLX5_GGA_COMP_WIN_SIZE_UNITS 1024u
#define MLX5_GGA_COMP_WIN_SIZE_MAX (32u * MLX5_GGA_COMP_WIN_SIZE_UNITS)
#define MLX5_GGA_COMP_LOG_BLOCK_SIZE_MAX 15u
@@ -605,7 +615,7 @@ struct mlx5_gga_wqe {
uint32_t opcode;
uint32_t sq_ds;
uint32_t flags;
- uint32_t gga_ctrl1; /* ws 12-15, bs 16-19, dyns 20-23. */
+ uint32_t gga_ctrl1;
uint32_t gga_ctrl2;
uint32_t opaque_lkey;
uint64_t opaque_vaddr;
@@ -1438,7 +1448,9 @@ struct mlx5_ifc_cmd_hca_cap_bits {
u8 log_dma_mmo_size[0x5];
u8 reserved_at_70[0x3];
u8 log_compress_mmo_size[0x5];
- u8 reserved_at_78[0x3];
+ u8 decompress_lz4_data_only_v2[0x1];
+ u8 decompress_lz4_no_checksum_v2[0x1];
+ u8 decompress_lz4_checksum_v2[0x1];
u8 log_decompress_mmo_size[0x5];
u8 log_max_srq_sz[0x8];
u8 log_max_qp_sz[0x8];
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v3 8/8] compress/mlx5: add support for LZ4 algorithm
2023-02-21 7:07 ` [PATCH v3 0/8] compress/mlx5: add LZ4 support Michael Baum
` (6 preceding siblings ...)
2023-02-21 7:07 ` [PATCH v3 7/8] common/mlx5: add LZ4 capabilities check Michael Baum
@ 2023-02-21 7:07 ` Michael Baum
2023-02-27 18:17 ` [EXT] [PATCH v3 0/8] compress/mlx5: add LZ4 support Akhil Goyal
8 siblings, 0 replies; 27+ messages in thread
From: Michael Baum @ 2023-02-21 7:07 UTC (permalink / raw)
To: dev; +Cc: Matan Azrad, Akhil Goyal, Thomas Monjalon
Add support for decompress LZ4 algorithm for mlx5 PMD.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
---
doc/guides/compressdevs/features/mlx5.ini | 18 ++-
doc/guides/compressdevs/mlx5.rst | 44 +++++-
doc/guides/rel_notes/release_23_03.rst | 4 +
drivers/compress/mlx5/mlx5_compress.c | 157 ++++++++++++++++++----
4 files changed, 184 insertions(+), 39 deletions(-)
diff --git a/doc/guides/compressdevs/features/mlx5.ini b/doc/guides/compressdevs/features/mlx5.ini
index 891ce47936..28b050144a 100644
--- a/doc/guides/compressdevs/features/mlx5.ini
+++ b/doc/guides/compressdevs/features/mlx5.ini
@@ -4,10 +4,14 @@
; Supported features of 'MLX5' compression driver.
;
[Features]
-HW Accelerated = Y
-Deflate = Y
-Adler32 = Y
-Crc32 = Y
-Adler32&Crc32 = Y
-Fixed = Y
-Dynamic = Y
+HW Accelerated = Y
+Deflate = Y
+LZ4 = Y
+Adler32 = Y
+Crc32 = Y
+Adler32&Crc32 = Y
+xxHash32 = Y
+Fixed = Y
+Dynamic = Y
+LZ4 Block Checksum = Y
+LZ4 Block Independence = Y
diff --git a/doc/guides/compressdevs/mlx5.rst b/doc/guides/compressdevs/mlx5.rst
index 8bf4423882..751fae7184 100644
--- a/doc/guides/compressdevs/mlx5.rst
+++ b/doc/guides/compressdevs/mlx5.rst
@@ -39,11 +39,27 @@ Features
Compress mlx5 PMD has support for:
-Compression/Decompression algorithm:
+- Compression
+- Decompression
+- DMA
-* DEFLATE.
+Algorithms
+----------
-NULL algorithm for DMA operations.
+NULL algorithm
+~~~~~~~~~~~~~~
+
+NULL algorithm is the way to perform DMA operations.
+It works through either compress or decompress operation.
+
+Shareable transformation.
+
+Checksum generation:
+
+* CRC32, Adler32 and combined checksum.
+
+DEFLATE algorithm
+~~~~~~~~~~~~~~~~~
Huffman code type:
@@ -60,11 +76,31 @@ Checksum generation:
* CRC32, Adler32 and combined checksum.
+LZ4 algorithm
+~~~~~~~~~~~~~
+
+Support for flags:
+
+* ``RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM``
+* ``RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE``
+
+Window size support:
+
+1KB, 2KB, 4KB, 8KB, 16KB and 32KB.
+
+Shareable transformation.
+
+Checksum generation:
+
+* xxHash-32 checksum.
+
Limitations
-----------
* Scatter-Gather, SHA and Stateful are not supported.
* Non-compressed block is not supported in compress (supported in decompress).
+* Compress operation is not supported by BlueField-3.
+* LZ4 algorithm is not supported by BlueField-2.
Driver options
--------------
@@ -75,7 +111,7 @@ for an additional list of options shared with other mlx5 drivers.
- ``log-block-size`` parameter [int]
Log of the Huffman block size in the Deflate algorithm.
- Values from [4-15]; value x means block size is 2^x.
+ Values from [4-15]; value x means block size is 2\ :sup:`x`.
The default value is 15.
diff --git a/doc/guides/rel_notes/release_23_03.rst b/doc/guides/rel_notes/release_23_03.rst
index 49c18617a5..8b9c47fd63 100644
--- a/doc/guides/rel_notes/release_23_03.rst
+++ b/doc/guides/rel_notes/release_23_03.rst
@@ -183,6 +183,10 @@ New Features
Added LZ4 compression algorithm with xxHash-32 for the checksum.
+* **Updated NVIDIA mlx5 compress PMD.**
+
+ Added LZ4 algorithm support for decompress operation.
+
* **Updated the eventdev reconfigure logic for service based adapters.**
* eventdev reconfig logic is enhanced to increment the
diff --git a/drivers/compress/mlx5/mlx5_compress.c b/drivers/compress/mlx5/mlx5_compress.c
index 3ea3447f11..41d9752833 100644
--- a/drivers/compress/mlx5/mlx5_compress.c
+++ b/drivers/compress/mlx5/mlx5_compress.c
@@ -24,6 +24,7 @@
#define MLX5_COMPRESS_DRIVER_NAME mlx5_compress
#define MLX5_COMPRESS_MAX_QPS 1024
#define MLX5_COMP_MAX_WIN_SIZE_CONF 6u
+#define MLX5_COMP_NUM_SUP_ALGO 4
struct mlx5_compress_devarg_params {
uint32_t log_block_sz;
@@ -43,6 +44,7 @@ struct mlx5_compress_priv {
struct mlx5_common_device *cdev; /* Backend mlx5 device. */
struct mlx5_uar uar;
struct rte_compressdev_config dev_config;
+ struct rte_compressdev_capabilities caps[MLX5_COMP_NUM_SUP_ALGO];
LIST_HEAD(xform_list, mlx5_compress_xform) xform_list;
rte_spinlock_t xform_sl;
uint32_t log_block_sz;
@@ -70,36 +72,16 @@ static pthread_mutex_t priv_list_lock = PTHREAD_MUTEX_INITIALIZER;
int mlx5_compress_logtype;
-static const struct rte_compressdev_capabilities mlx5_caps[] = {
- {
- .algo = RTE_COMP_ALGO_NULL,
- .comp_feature_flags = RTE_COMP_FF_ADLER32_CHECKSUM |
- RTE_COMP_FF_CRC32_CHECKSUM |
- RTE_COMP_FF_CRC32_ADLER32_CHECKSUM |
- RTE_COMP_FF_SHAREABLE_PRIV_XFORM,
- },
- {
- .algo = RTE_COMP_ALGO_DEFLATE,
- .comp_feature_flags = RTE_COMP_FF_ADLER32_CHECKSUM |
- RTE_COMP_FF_CRC32_CHECKSUM |
- RTE_COMP_FF_CRC32_ADLER32_CHECKSUM |
- RTE_COMP_FF_SHAREABLE_PRIV_XFORM |
- RTE_COMP_FF_HUFFMAN_FIXED |
- RTE_COMP_FF_HUFFMAN_DYNAMIC,
- .window_size = {.min = 10, .max = 15, .increment = 1},
- },
- RTE_COMP_END_OF_CAPABILITIES_LIST()
-};
-
static void
mlx5_compress_dev_info_get(struct rte_compressdev *dev,
struct rte_compressdev_info *info)
{
- RTE_SET_USED(dev);
- if (info != NULL) {
+ if (dev != NULL && info != NULL) {
+ struct mlx5_compress_priv *priv = dev->data->dev_private;
+
info->max_nb_queue_pairs = MLX5_COMPRESS_MAX_QPS;
info->feature_flags = RTE_COMPDEV_FF_HW_ACCELERATED;
- info->capabilities = mlx5_caps;
+ info->capabilities = priv->caps;
}
}
@@ -236,6 +218,8 @@ mlx5_compress_qp_setup(struct rte_compressdev *dev, uint16_t qp_id,
qp_attr.num_of_receive_wqes = 0;
qp_attr.num_of_send_wqbbs = RTE_BIT32(log_ops_n);
qp_attr.mmo = attr->mmo_compress_qp_en || attr->mmo_dma_qp_en ||
+ attr->decomp_lz4_checksum_en ||
+ attr->decomp_lz4_no_checksum_en ||
attr->decomp_deflate_v1_en || attr->decomp_deflate_v2_en;
ret = mlx5_devx_qp_create(priv->cdev->ctx, &qp->qp,
qp_attr.num_of_send_wqbbs *
@@ -280,7 +264,11 @@ mlx5_compress_xform_validate(const struct rte_comp_xform *xform,
return -ENOTSUP;
} else if (!attr->mmo_compress_qp_en &&
!attr->mmo_compress_sq_en) {
- DRV_LOG(ERR, "Not enough capabilities to support compress operation, maybe old FW/OFED version?");
+ DRV_LOG(ERR, "Not enough capabilities to support compress operation.");
+ return -ENOTSUP;
+ }
+ if (xform->compress.algo == RTE_COMP_ALGO_LZ4) {
+ DRV_LOG(ERR, "LZ4 compression is not supported.");
return -ENOTSUP;
}
if (xform->compress.level == RTE_COMP_LEVEL_NONE) {
@@ -291,6 +279,10 @@ mlx5_compress_xform_validate(const struct rte_comp_xform *xform,
DRV_LOG(ERR, "SHA is not supported.");
return -ENOTSUP;
}
+ if (xform->compress.chksum == RTE_COMP_CHECKSUM_XXHASH32) {
+ DRV_LOG(ERR, "xxHash32 checksum isn't supported in compress operation.");
+ return -ENOTSUP;
+ }
break;
case RTE_COMP_DECOMPRESS:
switch (xform->decompress.algo) {
@@ -307,6 +299,44 @@ mlx5_compress_xform_validate(const struct rte_comp_xform *xform,
DRV_LOG(ERR, "Not enough capabilities to support decompress DEFLATE algorithm, maybe old FW/OFED version?");
return -ENOTSUP;
}
+ switch (xform->decompress.chksum) {
+ case RTE_COMP_CHECKSUM_NONE:
+ case RTE_COMP_CHECKSUM_CRC32:
+ case RTE_COMP_CHECKSUM_ADLER32:
+ case RTE_COMP_CHECKSUM_CRC32_ADLER32:
+ break;
+ case RTE_COMP_CHECKSUM_XXHASH32:
+ default:
+ DRV_LOG(ERR, "DEFLATE algorithm doesn't support %u checksum.",
+ xform->decompress.chksum);
+ return -ENOTSUP;
+ }
+ break;
+ case RTE_COMP_ALGO_LZ4:
+ if (!attr->decomp_lz4_no_checksum_en &&
+ !attr->decomp_lz4_checksum_en) {
+ DRV_LOG(ERR, "Not enough capabilities to support decompress LZ4 algorithm, maybe old FW/OFED version?");
+ return -ENOTSUP;
+ }
+ if (xform->decompress.lz4.flags &
+ RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM) {
+ if (!attr->decomp_lz4_checksum_en) {
+ DRV_LOG(ERR, "Not enough capabilities to support decompress LZ4 block with checksum param, maybe old FW/OFED version?");
+ return -ENOTSUP;
+ }
+ } else {
+ if (!attr->decomp_lz4_no_checksum_en) {
+ DRV_LOG(ERR, "Not enough capabilities to support decompress LZ4 block without checksum param, maybe old FW/OFED version?");
+ return -ENOTSUP;
+ }
+ }
+ if (xform->decompress.chksum !=
+ RTE_COMP_CHECKSUM_XXHASH32 &&
+ xform->decompress.chksum !=
+ RTE_COMP_CHECKSUM_NONE) {
+ DRV_LOG(ERR, "LZ4 algorithm supports only xxHash32 checksum.");
+ return -ENOTSUP;
+ }
break;
default:
DRV_LOG(ERR, "Algorithm %u is not supported.",
@@ -383,6 +413,27 @@ mlx5_compress_xform_create(struct rte_compressdev *dev,
case RTE_COMP_ALGO_DEFLATE:
xfrm->opcode += MLX5_OPC_MOD_MMO_DECOMP <<
WQE_CSEG_OPC_MOD_OFFSET;
+ xfrm->gga_ctrl1 += WQE_GGA_DECOMP_DEFLATE <<
+ WQE_GGA_DECOMP_TYPE_OFFSET;
+ break;
+ case RTE_COMP_ALGO_LZ4:
+ xfrm->opcode += MLX5_OPC_MOD_MMO_DECOMP <<
+ WQE_CSEG_OPC_MOD_OFFSET;
+ xfrm->gga_ctrl1 += WQE_GGA_DECOMP_LZ4 <<
+ WQE_GGA_DECOMP_TYPE_OFFSET;
+ if (xform->decompress.lz4.flags &
+ RTE_COMP_LZ4_FLAG_BLOCK_CHECKSUM)
+ xfrm->gga_ctrl1 +=
+ MLX5_GGA_DECOMP_LZ4_BLOCK_WITH_CHECKSUM <<
+ WQE_GGA_DECOMP_PARAMS_OFFSET;
+ else
+ xfrm->gga_ctrl1 +=
+ MLX5_GGA_DECOMP_LZ4_BLOCK_WITHOUT_CHECKSUM
+ << WQE_GGA_DECOMP_PARAMS_OFFSET;
+ if (xform->decompress.lz4.flags &
+ RTE_COMP_LZ4_FLAG_BLOCK_INDEPENDENCE)
+ xfrm->gga_ctrl1 += 1u <<
+ WQE_GGA_DECOMP_BLOCK_INDEPENDENT_OFFSET;
break;
default:
goto err;
@@ -390,7 +441,7 @@ mlx5_compress_xform_create(struct rte_compressdev *dev,
xfrm->csum_type = xform->decompress.chksum;
break;
default:
- DRV_LOG(ERR, "Algorithm %u is not supported.", xform->type);
+ DRV_LOG(ERR, "Operation %u is not supported.", xform->type);
goto err;
}
DRV_LOG(DEBUG, "New xform: gga ctrl1 = 0x%08X opcode = 0x%08X csum "
@@ -657,6 +708,10 @@ mlx5_compress_dequeue_burst(void *queue_pair, struct rte_comp_op **ops,
((uint64_t)rte_be_to_cpu_32
(opaq[idx].data[crc32_idx + 1]) << 32);
break;
+ case RTE_COMP_CHECKSUM_XXHASH32:
+ op->output_chksum = (uint64_t)rte_be_to_cpu_32
+ (opaq[idx].v2.xxh32);
+ break;
default:
break;
}
@@ -720,6 +775,49 @@ mlx5_compress_handle_devargs(struct mlx5_kvargs_ctrl *mkvlist,
return 0;
}
+static void
+mlx5_compress_fill_caps(struct mlx5_compress_priv *priv,
+ const struct mlx5_hca_attr *attr)
+{
+ struct rte_compressdev_capabilities caps[] = {
+ {
+ .algo = RTE_COMP_ALGO_NULL,
+ .comp_feature_flags = RTE_COMP_FF_ADLER32_CHECKSUM |
+ RTE_COMP_FF_CRC32_CHECKSUM |
+ RTE_COMP_FF_CRC32_ADLER32_CHECKSUM |
+ RTE_COMP_FF_SHAREABLE_PRIV_XFORM,
+ },
+ {
+ .algo = RTE_COMP_ALGO_DEFLATE,
+ .comp_feature_flags = RTE_COMP_FF_ADLER32_CHECKSUM |
+ RTE_COMP_FF_CRC32_CHECKSUM |
+ RTE_COMP_FF_CRC32_ADLER32_CHECKSUM |
+ RTE_COMP_FF_SHAREABLE_PRIV_XFORM |
+ RTE_COMP_FF_HUFFMAN_FIXED |
+ RTE_COMP_FF_HUFFMAN_DYNAMIC,
+ .window_size = {.min = 10, .max = 15, .increment = 1},
+ },
+ {
+ .algo = RTE_COMP_ALGO_LZ4,
+ .comp_feature_flags = RTE_COMP_FF_XXHASH32_CHECKSUM |
+ RTE_COMP_FF_SHAREABLE_PRIV_XFORM |
+ RTE_COMP_FF_LZ4_BLOCK_INDEPENDENCE,
+ .window_size = {.min = 1, .max = 15, .increment = 1},
+ },
+ RTE_COMP_END_OF_CAPABILITIES_LIST()
+ };
+ priv->caps[0] = caps[0];
+ priv->caps[1] = caps[1];
+ if (attr->decomp_lz4_checksum_en || attr->decomp_lz4_no_checksum_en) {
+ priv->caps[2] = caps[2];
+ if (attr->decomp_lz4_checksum_en)
+ priv->caps[2].comp_feature_flags |=
+ RTE_COMP_FF_LZ4_BLOCK_WITH_CHECKSUM;
+ priv->caps[3] = caps[3];
+ } else
+ priv->caps[2] = caps[3];
+}
+
static int
mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
struct mlx5_kvargs_ctrl *mkvlist)
@@ -740,7 +838,8 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
rte_errno = ENOTSUP;
return -rte_errno;
}
- if (!attr->decomp_deflate_v1_en && !attr->decomp_deflate_v2_en &&
+ if (!attr->decomp_lz4_checksum_en && !attr->decomp_lz4_no_checksum_en &&
+ !attr->decomp_deflate_v1_en && !attr->decomp_deflate_v2_en &&
!attr->mmo_decompress_sq_en && !attr->mmo_compress_qp_en &&
!attr->mmo_compress_sq_en && !attr->mmo_dma_qp_en &&
!attr->mmo_dma_sq_en) {
@@ -763,7 +862,8 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
compressdev->feature_flags = RTE_COMPDEV_FF_HW_ACCELERATED;
priv = compressdev->data->dev_private;
priv->log_block_sz = devarg_prms.log_block_sz;
- if (attr->decomp_deflate_v2_en)
+ if (attr->decomp_deflate_v2_en || attr->decomp_lz4_checksum_en ||
+ attr->decomp_lz4_no_checksum_en)
crc32_opaq_offset = offsetof(union mlx5_gga_compress_opaque,
v2.crc32);
else
@@ -773,6 +873,7 @@ mlx5_compress_dev_probe(struct mlx5_common_device *cdev,
priv->crc32_opaq_offs = crc32_opaq_offset / 4;
priv->cdev = cdev;
priv->compressdev = compressdev;
+ mlx5_compress_fill_caps(priv, attr);
if (mlx5_devx_uar_prepare(cdev, &priv->uar) != 0) {
rte_compressdev_pmd_destroy(priv->compressdev);
return -1;
--
2.25.1
^ permalink raw reply related [flat|nested] 27+ messages in thread* RE: [EXT] [PATCH v3 0/8] compress/mlx5: add LZ4 support
2023-02-21 7:07 ` [PATCH v3 0/8] compress/mlx5: add LZ4 support Michael Baum
` (7 preceding siblings ...)
2023-02-21 7:07 ` [PATCH v3 8/8] compress/mlx5: add support for LZ4 algorithm Michael Baum
@ 2023-02-27 18:17 ` Akhil Goyal
8 siblings, 0 replies; 27+ messages in thread
From: Akhil Goyal @ 2023-02-27 18:17 UTC (permalink / raw)
To: Michael Baum, dev; +Cc: Matan Azrad, Thomas Monjalon
> Add support for LZ4 decompression algorithm for mlx5 compress PMD.
> Complete the support of compress PMD for Bluefield-3 smart-nic.
>
> RFC:
> https://patchwork.dpdk.org/project/dpdk/patch/20220410182622.8828-1-rzidane@nvidia.com/
>
> v2:
> - Rebase.
> - Update PRM names.
> - Add a new fix at the beginning of the patch-set.
>
> v3:
> - Rebase.
> - Fix missing "!" in decompress xform validation for null algo.
> - Change DEFLATE checksum validation for decompress xform.
>
Series applied to dpdk-next-crypto
Thanks.
^ permalink raw reply [flat|nested] 27+ messages in thread