* [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources
@ 2020-03-18 13:48 Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 1/9] mlxsw: spectrum_cnt: Query bank size from FW resources Ido Schimmel
` (10 more replies)
0 siblings, 11 replies; 12+ messages in thread
From: Ido Schimmel @ 2020-03-18 13:48 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, mlxsw, Ido Schimmel
From: Ido Schimmel <idosch@mellanox.com>
Jiri says:
Capacity and utilization of existing flow and RIF counters are currently
unavailable to be seen by the user. Use the existing devlink resources
API to expose the information:
$ sudo devlink resource show pci/0000:00:10.0 -v
pci/0000:00:10.0:
name kvd resource_path /kvd size 524288 unit entry dpipe_tables none
name span_agents resource_path /span_agents size 8 occ 0 unit entry dpipe_tables none
name counters resource_path /counters size 79872 occ 44 unit entry dpipe_tables none
resources:
name flow resource_path /counters/flow size 61440 occ 4 unit entry dpipe_tables none
name rif resource_path /counters/rif size 18432 occ 40 unit entry dpipe_tables none
Jiri Pirko (9):
mlxsw: spectrum_cnt: Query bank size from FW resources
selftests: spectrum-2: Adjust tc_flower_scale limit according to
current counter count
mlxsw: spectrum_cnt: Move sub_pools under per-instance pool struct
mlxsw: spectrum_cnt: Add entry_size_res_id for each subpool and use it
to query entry size
mlxsw: spectrum_cnt: Expose subpool sizes over devlink resources
mlxsw: spectrum_cnt: Move config validation along with resource
register
mlxsw: spectrum_cnt: Consolidate subpools initialization
mlxsw: spectrum_cnt: Expose devlink resource occupancy for counters
selftests: mlxsw: Add tc action hw_stats tests
.../net/ethernet/mellanox/mlxsw/resources.h | 2 +
.../net/ethernet/mellanox/mlxsw/spectrum.c | 10 +
.../net/ethernet/mellanox/mlxsw/spectrum.h | 7 +
.../ethernet/mellanox/mlxsw/spectrum_cnt.c | 243 +++++++++++++-----
.../ethernet/mellanox/mlxsw/spectrum_cnt.h | 2 +
.../net/mlxsw/spectrum-2/tc_flower_scale.sh | 4 +-
.../drivers/net/mlxsw/tc_action_hw_stats.sh | 130 ++++++++++
tools/testing/selftests/net/forwarding/lib.sh | 9 +
8 files changed, 343 insertions(+), 64 deletions(-)
create mode 100755 tools/testing/selftests/drivers/net/mlxsw/tc_action_hw_stats.sh
--
2.24.1
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH net-next 1/9] mlxsw: spectrum_cnt: Query bank size from FW resources
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
@ 2020-03-18 13:48 ` Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 2/9] selftests: spectrum-2: Adjust tc_flower_scale limit according to current counter count Ido Schimmel
` (9 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Ido Schimmel @ 2020-03-18 13:48 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, mlxsw, Ido Schimmel
From: Jiri Pirko <jiri@mellanox.com>
The bank size is different between Spectrum versions. Also it is
a resource that can be queried. So instead of hard coding the value in
code, query it from the firmware.
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: Ido Schimmel <idosch@mellanox.com>
---
drivers/net/ethernet/mellanox/mlxsw/resources.h | 2 ++
.../net/ethernet/mellanox/mlxsw/spectrum_cnt.c | 15 +++++++++------
2 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/resources.h b/drivers/net/ethernet/mellanox/mlxsw/resources.h
index 6534184cb942..d62496ef299c 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/resources.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/resources.h
@@ -18,6 +18,7 @@ enum mlxsw_res_id {
MLXSW_RES_ID_CQE_V1,
MLXSW_RES_ID_CQE_V2,
MLXSW_RES_ID_COUNTER_POOL_SIZE,
+ MLXSW_RES_ID_COUNTER_BANK_SIZE,
MLXSW_RES_ID_MAX_SPAN,
MLXSW_RES_ID_COUNTER_SIZE_PACKETS_BYTES,
MLXSW_RES_ID_COUNTER_SIZE_ROUTER_BASIC,
@@ -75,6 +76,7 @@ static u16 mlxsw_res_ids[] = {
[MLXSW_RES_ID_CQE_V1] = 0x2211,
[MLXSW_RES_ID_CQE_V2] = 0x2212,
[MLXSW_RES_ID_COUNTER_POOL_SIZE] = 0x2410,
+ [MLXSW_RES_ID_COUNTER_BANK_SIZE] = 0x2411,
[MLXSW_RES_ID_MAX_SPAN] = 0x2420,
[MLXSW_RES_ID_COUNTER_SIZE_PACKETS_BYTES] = 0x2443,
[MLXSW_RES_ID_COUNTER_SIZE_ROUTER_BASIC] = 0x2449,
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
index 6a02ef9ec00e..37811181586a 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
@@ -7,8 +7,6 @@
#include "spectrum_cnt.h"
-#define MLXSW_SP_COUNTER_POOL_BANK_SIZE 4096
-
struct mlxsw_sp_counter_sub_pool {
unsigned int base_index;
unsigned int size;
@@ -36,13 +34,15 @@ static int mlxsw_sp_counter_pool_validate(struct mlxsw_sp *mlxsw_sp)
{
unsigned int total_bank_config = 0;
unsigned int pool_size;
+ unsigned int bank_size;
int i;
pool_size = MLXSW_CORE_RES_GET(mlxsw_sp->core, COUNTER_POOL_SIZE);
+ bank_size = MLXSW_CORE_RES_GET(mlxsw_sp->core, COUNTER_BANK_SIZE);
/* Check config is valid, no bank over subscription */
for (i = 0; i < ARRAY_SIZE(mlxsw_sp_counter_sub_pools); i++)
total_bank_config += mlxsw_sp_counter_sub_pools[i].bank_count;
- if (total_bank_config > pool_size / MLXSW_SP_COUNTER_POOL_BANK_SIZE + 1)
+ if (total_bank_config > pool_size / bank_size + 1)
return -EINVAL;
return 0;
}
@@ -71,11 +71,13 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
struct mlxsw_sp_counter_sub_pool *sub_pool;
struct mlxsw_sp_counter_pool *pool;
unsigned int base_index;
+ unsigned int bank_size;
unsigned int map_size;
int i;
int err;
- if (!MLXSW_CORE_RES_VALID(mlxsw_sp->core, COUNTER_POOL_SIZE))
+ if (!MLXSW_CORE_RES_VALID(mlxsw_sp->core, COUNTER_POOL_SIZE) ||
+ !MLXSW_CORE_RES_VALID(mlxsw_sp->core, COUNTER_BANK_SIZE))
return -EIO;
err = mlxsw_sp_counter_pool_validate(mlxsw_sp);
@@ -94,6 +96,8 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
pool->pool_size = MLXSW_CORE_RES_GET(mlxsw_sp->core, COUNTER_POOL_SIZE);
map_size = BITS_TO_LONGS(pool->pool_size) * sizeof(unsigned long);
+ bank_size = MLXSW_CORE_RES_GET(mlxsw_sp->core, COUNTER_BANK_SIZE);
+
pool->usage = kzalloc(map_size, GFP_KERNEL);
if (!pool->usage) {
err = -ENOMEM;
@@ -107,8 +111,7 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
base_index = 0;
for (i = 0; i < ARRAY_SIZE(mlxsw_sp_counter_sub_pools); i++) {
sub_pool = &pool->sub_pools[i];
- sub_pool->size = sub_pool->bank_count *
- MLXSW_SP_COUNTER_POOL_BANK_SIZE;
+ sub_pool->size = sub_pool->bank_count * bank_size;
sub_pool->base_index = base_index;
base_index += sub_pool->size;
/* The last bank can't be fully used */
--
2.24.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH net-next 2/9] selftests: spectrum-2: Adjust tc_flower_scale limit according to current counter count
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 1/9] mlxsw: spectrum_cnt: Query bank size from FW resources Ido Schimmel
@ 2020-03-18 13:48 ` Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 3/9] mlxsw: spectrum_cnt: Move sub_pools under per-instance pool struct Ido Schimmel
` (8 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Ido Schimmel @ 2020-03-18 13:48 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, mlxsw, Ido Schimmel
From: Jiri Pirko <jiri@mellanox.com>
With the change that made the code to query counter bank size from device
instead of using hard-coded value, the number of available counters
changed for Spectrum-2. Adjust the limit in the selftests.
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: Ido Schimmel <idosch@mellanox.com>
---
.../selftests/drivers/net/mlxsw/spectrum-2/tc_flower_scale.sh | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/tc_flower_scale.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/tc_flower_scale.sh
index a0795227216e..efd798a85931 100644
--- a/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/tc_flower_scale.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/tc_flower_scale.sh
@@ -8,9 +8,9 @@ tc_flower_get_target()
# The driver associates a counter with each tc filter, which means the
# number of supported filters is bounded by the number of available
# counters.
- # Currently, the driver supports 12K (12,288) flow counters and six of
+ # Currently, the driver supports 30K (30,720) flow counters and six of
# these are used for multicast routing.
- local target=12282
+ local target=30714
if ((! should_fail)); then
echo $target
--
2.24.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH net-next 3/9] mlxsw: spectrum_cnt: Move sub_pools under per-instance pool struct
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 1/9] mlxsw: spectrum_cnt: Query bank size from FW resources Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 2/9] selftests: spectrum-2: Adjust tc_flower_scale limit according to current counter count Ido Schimmel
@ 2020-03-18 13:48 ` Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 4/9] mlxsw: spectrum_cnt: Add entry_size_res_id for each subpool and use it to query entry size Ido Schimmel
` (7 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Ido Schimmel @ 2020-03-18 13:48 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, mlxsw, Ido Schimmel
From: Jiri Pirko <jiri@mellanox.com>
Currently, the global static array of subpools is used. Make it
per-instance as multiple instances of the mlxsw driver can have
different values.
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: Ido Schimmel <idosch@mellanox.com>
---
.../ethernet/mellanox/mlxsw/spectrum_cnt.c | 45 +++++++++++--------
1 file changed, 27 insertions(+), 18 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
index 37811181586a..4cdabde47dd0 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
@@ -18,10 +18,11 @@ struct mlxsw_sp_counter_pool {
unsigned int pool_size;
unsigned long *usage; /* Usage bitmap */
spinlock_t counter_pool_lock; /* Protects counter pool allocations */
- struct mlxsw_sp_counter_sub_pool *sub_pools;
+ unsigned int sub_pools_count;
+ struct mlxsw_sp_counter_sub_pool sub_pools[];
};
-static struct mlxsw_sp_counter_sub_pool mlxsw_sp_counter_sub_pools[] = {
+static const struct mlxsw_sp_counter_sub_pool mlxsw_sp_counter_sub_pools[] = {
[MLXSW_SP_COUNTER_SUB_POOL_FLOW] = {
.bank_count = 6,
},
@@ -32,6 +33,7 @@ static struct mlxsw_sp_counter_sub_pool mlxsw_sp_counter_sub_pools[] = {
static int mlxsw_sp_counter_pool_validate(struct mlxsw_sp *mlxsw_sp)
{
+ struct mlxsw_sp_counter_pool *pool = mlxsw_sp->counter_pool;
unsigned int total_bank_config = 0;
unsigned int pool_size;
unsigned int bank_size;
@@ -40,8 +42,8 @@ static int mlxsw_sp_counter_pool_validate(struct mlxsw_sp *mlxsw_sp)
pool_size = MLXSW_CORE_RES_GET(mlxsw_sp->core, COUNTER_POOL_SIZE);
bank_size = MLXSW_CORE_RES_GET(mlxsw_sp->core, COUNTER_BANK_SIZE);
/* Check config is valid, no bank over subscription */
- for (i = 0; i < ARRAY_SIZE(mlxsw_sp_counter_sub_pools); i++)
- total_bank_config += mlxsw_sp_counter_sub_pools[i].bank_count;
+ for (i = 0; i < pool->sub_pools_count; i++)
+ total_bank_config += pool->sub_pools[i].bank_count;
if (total_bank_config > pool_size / bank_size + 1)
return -EINVAL;
return 0;
@@ -49,16 +51,17 @@ static int mlxsw_sp_counter_pool_validate(struct mlxsw_sp *mlxsw_sp)
static int mlxsw_sp_counter_sub_pools_prepare(struct mlxsw_sp *mlxsw_sp)
{
+ struct mlxsw_sp_counter_pool *pool = mlxsw_sp->counter_pool;
struct mlxsw_sp_counter_sub_pool *sub_pool;
/* Prepare generic flow pool*/
- sub_pool = &mlxsw_sp_counter_sub_pools[MLXSW_SP_COUNTER_SUB_POOL_FLOW];
+ sub_pool = &pool->sub_pools[MLXSW_SP_COUNTER_SUB_POOL_FLOW];
if (!MLXSW_CORE_RES_VALID(mlxsw_sp->core, COUNTER_SIZE_PACKETS_BYTES))
return -EIO;
sub_pool->entry_size = MLXSW_CORE_RES_GET(mlxsw_sp->core,
COUNTER_SIZE_PACKETS_BYTES);
/* Prepare erif pool*/
- sub_pool = &mlxsw_sp_counter_sub_pools[MLXSW_SP_COUNTER_SUB_POOL_RIF];
+ sub_pool = &pool->sub_pools[MLXSW_SP_COUNTER_SUB_POOL_RIF];
if (!MLXSW_CORE_RES_VALID(mlxsw_sp->core, COUNTER_SIZE_ROUTER_BASIC))
return -EIO;
sub_pool->entry_size = MLXSW_CORE_RES_GET(mlxsw_sp->core,
@@ -68,6 +71,7 @@ static int mlxsw_sp_counter_sub_pools_prepare(struct mlxsw_sp *mlxsw_sp)
int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
{
+ unsigned int sub_pools_count = ARRAY_SIZE(mlxsw_sp_counter_sub_pools);
struct mlxsw_sp_counter_sub_pool *sub_pool;
struct mlxsw_sp_counter_pool *pool;
unsigned int base_index;
@@ -80,18 +84,23 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
!MLXSW_CORE_RES_VALID(mlxsw_sp->core, COUNTER_BANK_SIZE))
return -EIO;
+ pool = kzalloc(struct_size(pool, sub_pools, sub_pools_count),
+ GFP_KERNEL);
+ if (!pool)
+ return -ENOMEM;
+ mlxsw_sp->counter_pool = pool;
+ memcpy(pool->sub_pools, mlxsw_sp_counter_sub_pools,
+ sub_pools_count * sizeof(*sub_pool));
+ pool->sub_pools_count = sub_pools_count;
+ spin_lock_init(&pool->counter_pool_lock);
+
err = mlxsw_sp_counter_pool_validate(mlxsw_sp);
if (err)
- return err;
+ goto err_pool_validate;
err = mlxsw_sp_counter_sub_pools_prepare(mlxsw_sp);
if (err)
- return err;
-
- pool = kzalloc(sizeof(*pool), GFP_KERNEL);
- if (!pool)
- return -ENOMEM;
- spin_lock_init(&pool->counter_pool_lock);
+ goto err_sub_pools_prepare;
pool->pool_size = MLXSW_CORE_RES_GET(mlxsw_sp->core, COUNTER_POOL_SIZE);
map_size = BITS_TO_LONGS(pool->pool_size) * sizeof(unsigned long);
@@ -104,12 +113,11 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
goto err_usage_alloc;
}
- pool->sub_pools = mlxsw_sp_counter_sub_pools;
/* Allocation is based on bank count which should be
* specified for each sub pool statically.
*/
base_index = 0;
- for (i = 0; i < ARRAY_SIZE(mlxsw_sp_counter_sub_pools); i++) {
+ for (i = 0; i < pool->sub_pools_count; i++) {
sub_pool = &pool->sub_pools[i];
sub_pool->size = sub_pool->bank_count * bank_size;
sub_pool->base_index = base_index;
@@ -119,10 +127,11 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
sub_pool->size = pool->pool_size - sub_pool->base_index;
}
- mlxsw_sp->counter_pool = pool;
return 0;
err_usage_alloc:
+err_sub_pools_prepare:
+err_pool_validate:
kfree(pool);
return err;
}
@@ -147,7 +156,7 @@ int mlxsw_sp_counter_alloc(struct mlxsw_sp *mlxsw_sp,
unsigned int stop_index;
int i, err;
- sub_pool = &mlxsw_sp_counter_sub_pools[sub_pool_id];
+ sub_pool = &pool->sub_pools[sub_pool_id];
stop_index = sub_pool->base_index + sub_pool->size;
entry_index = sub_pool->base_index;
@@ -186,7 +195,7 @@ void mlxsw_sp_counter_free(struct mlxsw_sp *mlxsw_sp,
if (WARN_ON(counter_index >= pool->pool_size))
return;
- sub_pool = &mlxsw_sp_counter_sub_pools[sub_pool_id];
+ sub_pool = &pool->sub_pools[sub_pool_id];
spin_lock(&pool->counter_pool_lock);
for (i = 0; i < sub_pool->entry_size; i++)
__clear_bit(counter_index + i, pool->usage);
--
2.24.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH net-next 4/9] mlxsw: spectrum_cnt: Add entry_size_res_id for each subpool and use it to query entry size
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
` (2 preceding siblings ...)
2020-03-18 13:48 ` [PATCH net-next 3/9] mlxsw: spectrum_cnt: Move sub_pools under per-instance pool struct Ido Schimmel
@ 2020-03-18 13:48 ` Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 5/9] mlxsw: spectrum_cnt: Expose subpool sizes over devlink resources Ido Schimmel
` (6 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Ido Schimmel @ 2020-03-18 13:48 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, mlxsw, Ido Schimmel
From: Jiri Pirko <jiri@mellanox.com>
Add new field to subpool struct that would indicate which
resource id should be used to query the entry size for
the subpool from the device.
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: Ido Schimmel <idosch@mellanox.com>
---
.../ethernet/mellanox/mlxsw/spectrum_cnt.c | 26 ++++++++++---------
1 file changed, 14 insertions(+), 12 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
index 4cdabde47dd0..ef2c6c5c8b72 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
@@ -10,6 +10,7 @@
struct mlxsw_sp_counter_sub_pool {
unsigned int base_index;
unsigned int size;
+ enum mlxsw_res_id entry_size_res_id;
unsigned int entry_size;
unsigned int bank_count;
};
@@ -24,9 +25,11 @@ struct mlxsw_sp_counter_pool {
static const struct mlxsw_sp_counter_sub_pool mlxsw_sp_counter_sub_pools[] = {
[MLXSW_SP_COUNTER_SUB_POOL_FLOW] = {
+ .entry_size_res_id = MLXSW_RES_ID_COUNTER_SIZE_PACKETS_BYTES,
.bank_count = 6,
},
[MLXSW_SP_COUNTER_SUB_POOL_RIF] = {
+ .entry_size_res_id = MLXSW_RES_ID_COUNTER_SIZE_ROUTER_BASIC,
.bank_count = 2,
}
};
@@ -53,19 +56,18 @@ static int mlxsw_sp_counter_sub_pools_prepare(struct mlxsw_sp *mlxsw_sp)
{
struct mlxsw_sp_counter_pool *pool = mlxsw_sp->counter_pool;
struct mlxsw_sp_counter_sub_pool *sub_pool;
+ enum mlxsw_res_id res_id;
+ int i;
- /* Prepare generic flow pool*/
- sub_pool = &pool->sub_pools[MLXSW_SP_COUNTER_SUB_POOL_FLOW];
- if (!MLXSW_CORE_RES_VALID(mlxsw_sp->core, COUNTER_SIZE_PACKETS_BYTES))
- return -EIO;
- sub_pool->entry_size = MLXSW_CORE_RES_GET(mlxsw_sp->core,
- COUNTER_SIZE_PACKETS_BYTES);
- /* Prepare erif pool*/
- sub_pool = &pool->sub_pools[MLXSW_SP_COUNTER_SUB_POOL_RIF];
- if (!MLXSW_CORE_RES_VALID(mlxsw_sp->core, COUNTER_SIZE_ROUTER_BASIC))
- return -EIO;
- sub_pool->entry_size = MLXSW_CORE_RES_GET(mlxsw_sp->core,
- COUNTER_SIZE_ROUTER_BASIC);
+ for (i = 0; i < pool->sub_pools_count; i++) {
+ sub_pool = &pool->sub_pools[i];
+ res_id = sub_pool->entry_size_res_id;
+
+ if (!mlxsw_core_res_valid(mlxsw_sp->core, res_id))
+ return -EIO;
+ sub_pool->entry_size = mlxsw_core_res_get(mlxsw_sp->core,
+ res_id);
+ }
return 0;
}
--
2.24.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH net-next 5/9] mlxsw: spectrum_cnt: Expose subpool sizes over devlink resources
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
` (3 preceding siblings ...)
2020-03-18 13:48 ` [PATCH net-next 4/9] mlxsw: spectrum_cnt: Add entry_size_res_id for each subpool and use it to query entry size Ido Schimmel
@ 2020-03-18 13:48 ` Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 6/9] mlxsw: spectrum_cnt: Move config validation along with resource register Ido Schimmel
` (5 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Ido Schimmel @ 2020-03-18 13:48 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, mlxsw, Ido Schimmel
From: Jiri Pirko <jiri@mellanox.com>
Implement devlink resources support for counter pools. Move the subpool
sizes calculations into the new resources register function.
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: Ido Schimmel <idosch@mellanox.com>
---
.../net/ethernet/mellanox/mlxsw/spectrum.c | 10 ++
.../net/ethernet/mellanox/mlxsw/spectrum.h | 7 ++
.../ethernet/mellanox/mlxsw/spectrum_cnt.c | 99 +++++++++++++++----
.../ethernet/mellanox/mlxsw/spectrum_cnt.h | 2 +
4 files changed, 101 insertions(+), 17 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
index 51709012593e..35d3a68ef4fd 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.c
@@ -5421,8 +5421,13 @@ static int mlxsw_sp1_resources_register(struct mlxsw_core *mlxsw_core)
if (err)
goto err_resources_span_register;
+ err = mlxsw_sp_counter_resources_register(mlxsw_core);
+ if (err)
+ goto err_resources_counter_register;
+
return 0;
+err_resources_counter_register:
err_resources_span_register:
devlink_resources_unregister(priv_to_devlink(mlxsw_core), NULL);
return err;
@@ -5440,8 +5445,13 @@ static int mlxsw_sp2_resources_register(struct mlxsw_core *mlxsw_core)
if (err)
goto err_resources_span_register;
+ err = mlxsw_sp_counter_resources_register(mlxsw_core);
+ if (err)
+ goto err_resources_counter_register;
+
return 0;
+err_resources_counter_register:
err_resources_span_register:
devlink_resources_unregister(priv_to_devlink(mlxsw_core), NULL);
return err;
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
index 81801c6fb941..57d8c95e4f9f 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
@@ -46,6 +46,10 @@
#define MLXSW_SP_RESOURCE_NAME_SPAN "span_agents"
+#define MLXSW_SP_RESOURCE_NAME_COUNTERS "counters"
+#define MLXSW_SP_RESOURCE_NAME_COUNTERS_FLOW "flow"
+#define MLXSW_SP_RESOURCE_NAME_COUNTERS_RIF "rif"
+
enum mlxsw_sp_resource_id {
MLXSW_SP_RESOURCE_KVD = 1,
MLXSW_SP_RESOURCE_KVD_LINEAR,
@@ -55,6 +59,9 @@ enum mlxsw_sp_resource_id {
MLXSW_SP_RESOURCE_KVD_LINEAR_CHUNKS,
MLXSW_SP_RESOURCE_KVD_LINEAR_LARGE_CHUNKS,
MLXSW_SP_RESOURCE_SPAN,
+ MLXSW_SP_RESOURCE_COUNTERS,
+ MLXSW_SP_RESOURCE_COUNTERS_FLOW,
+ MLXSW_SP_RESOURCE_COUNTERS_RIF,
};
struct mlxsw_sp_port;
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
index ef2c6c5c8b72..629355cf52a9 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
@@ -8,15 +8,17 @@
#include "spectrum_cnt.h"
struct mlxsw_sp_counter_sub_pool {
+ u64 size;
unsigned int base_index;
- unsigned int size;
enum mlxsw_res_id entry_size_res_id;
+ const char *resource_name; /* devlink resource name */
+ u64 resource_id; /* devlink resource id */
unsigned int entry_size;
unsigned int bank_count;
};
struct mlxsw_sp_counter_pool {
- unsigned int pool_size;
+ u64 pool_size;
unsigned long *usage; /* Usage bitmap */
spinlock_t counter_pool_lock; /* Protects counter pool allocations */
unsigned int sub_pools_count;
@@ -26,10 +28,14 @@ struct mlxsw_sp_counter_pool {
static const struct mlxsw_sp_counter_sub_pool mlxsw_sp_counter_sub_pools[] = {
[MLXSW_SP_COUNTER_SUB_POOL_FLOW] = {
.entry_size_res_id = MLXSW_RES_ID_COUNTER_SIZE_PACKETS_BYTES,
+ .resource_name = MLXSW_SP_RESOURCE_NAME_COUNTERS_FLOW,
+ .resource_id = MLXSW_SP_RESOURCE_COUNTERS_FLOW,
.bank_count = 6,
},
[MLXSW_SP_COUNTER_SUB_POOL_RIF] = {
.entry_size_res_id = MLXSW_RES_ID_COUNTER_SIZE_ROUTER_BASIC,
+ .resource_name = MLXSW_SP_RESOURCE_NAME_COUNTERS_RIF,
+ .resource_id = MLXSW_SP_RESOURCE_COUNTERS_RIF,
.bank_count = 2,
}
};
@@ -74,18 +80,14 @@ static int mlxsw_sp_counter_sub_pools_prepare(struct mlxsw_sp *mlxsw_sp)
int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
{
unsigned int sub_pools_count = ARRAY_SIZE(mlxsw_sp_counter_sub_pools);
+ struct devlink *devlink = priv_to_devlink(mlxsw_sp->core);
struct mlxsw_sp_counter_sub_pool *sub_pool;
struct mlxsw_sp_counter_pool *pool;
unsigned int base_index;
- unsigned int bank_size;
unsigned int map_size;
int i;
int err;
- if (!MLXSW_CORE_RES_VALID(mlxsw_sp->core, COUNTER_POOL_SIZE) ||
- !MLXSW_CORE_RES_VALID(mlxsw_sp->core, COUNTER_BANK_SIZE))
- return -EIO;
-
pool = kzalloc(struct_size(pool, sub_pools, sub_pools_count),
GFP_KERNEL);
if (!pool)
@@ -104,10 +106,12 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
if (err)
goto err_sub_pools_prepare;
- pool->pool_size = MLXSW_CORE_RES_GET(mlxsw_sp->core, COUNTER_POOL_SIZE);
- map_size = BITS_TO_LONGS(pool->pool_size) * sizeof(unsigned long);
+ err = devlink_resource_size_get(devlink, MLXSW_SP_RESOURCE_COUNTERS,
+ &pool->pool_size);
+ if (err)
+ goto err_pool_resource_size_get;
- bank_size = MLXSW_CORE_RES_GET(mlxsw_sp->core, COUNTER_BANK_SIZE);
+ map_size = BITS_TO_LONGS(pool->pool_size) * sizeof(unsigned long);
pool->usage = kzalloc(map_size, GFP_KERNEL);
if (!pool->usage) {
@@ -115,23 +119,26 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
goto err_usage_alloc;
}
- /* Allocation is based on bank count which should be
- * specified for each sub pool statically.
- */
base_index = 0;
for (i = 0; i < pool->sub_pools_count; i++) {
sub_pool = &pool->sub_pools[i];
- sub_pool->size = sub_pool->bank_count * bank_size;
+
+ err = devlink_resource_size_get(devlink,
+ sub_pool->resource_id,
+ &sub_pool->size);
+ if (err)
+ goto err_sub_pool_resource_size_get;
+
sub_pool->base_index = base_index;
base_index += sub_pool->size;
- /* The last bank can't be fully used */
- if (sub_pool->base_index + sub_pool->size > pool->pool_size)
- sub_pool->size = pool->pool_size - sub_pool->base_index;
}
return 0;
+err_sub_pool_resource_size_get:
+ kfree(pool->usage);
err_usage_alloc:
+err_pool_resource_size_get:
err_sub_pools_prepare:
err_pool_validate:
kfree(pool);
@@ -203,3 +210,61 @@ void mlxsw_sp_counter_free(struct mlxsw_sp *mlxsw_sp,
__clear_bit(counter_index + i, pool->usage);
spin_unlock(&pool->counter_pool_lock);
}
+
+int mlxsw_sp_counter_resources_register(struct mlxsw_core *mlxsw_core)
+{
+ static struct devlink_resource_size_params size_params;
+ struct devlink *devlink = priv_to_devlink(mlxsw_core);
+ const struct mlxsw_sp_counter_sub_pool *sub_pool;
+ u64 sub_pool_size;
+ u64 base_index;
+ u64 pool_size;
+ u64 bank_size;
+ int err;
+ int i;
+
+ if (!MLXSW_CORE_RES_VALID(mlxsw_core, COUNTER_POOL_SIZE) ||
+ !MLXSW_CORE_RES_VALID(mlxsw_core, COUNTER_BANK_SIZE))
+ return -EIO;
+
+ pool_size = MLXSW_CORE_RES_GET(mlxsw_core, COUNTER_POOL_SIZE);
+ bank_size = MLXSW_CORE_RES_GET(mlxsw_core, COUNTER_BANK_SIZE);
+
+ devlink_resource_size_params_init(&size_params, pool_size,
+ pool_size, bank_size,
+ DEVLINK_RESOURCE_UNIT_ENTRY);
+ err = devlink_resource_register(devlink,
+ MLXSW_SP_RESOURCE_NAME_COUNTERS,
+ pool_size,
+ MLXSW_SP_RESOURCE_COUNTERS,
+ DEVLINK_RESOURCE_ID_PARENT_TOP,
+ &size_params);
+ if (err)
+ return err;
+
+ /* Allocation is based on bank count which should be
+ * specified for each sub pool statically.
+ */
+ base_index = 0;
+ for (i = 0; i < ARRAY_SIZE(mlxsw_sp_counter_sub_pools); i++) {
+ sub_pool = &mlxsw_sp_counter_sub_pools[i];
+ sub_pool_size = sub_pool->bank_count * bank_size;
+ /* The last bank can't be fully used */
+ if (base_index + sub_pool_size > pool_size)
+ sub_pool_size = pool_size - base_index;
+ base_index += sub_pool_size;
+
+ devlink_resource_size_params_init(&size_params, sub_pool_size,
+ sub_pool_size, bank_size,
+ DEVLINK_RESOURCE_UNIT_ENTRY);
+ err = devlink_resource_register(devlink,
+ sub_pool->resource_name,
+ sub_pool_size,
+ sub_pool->resource_id,
+ MLXSW_SP_RESOURCE_COUNTERS,
+ &size_params);
+ if (err)
+ return err;
+ }
+ return 0;
+}
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.h
index 81465e267b10..a68d931090dd 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.h
@@ -4,6 +4,7 @@
#ifndef _MLXSW_SPECTRUM_CNT_H
#define _MLXSW_SPECTRUM_CNT_H
+#include "core.h"
#include "spectrum.h"
enum mlxsw_sp_counter_sub_pool_id {
@@ -19,5 +20,6 @@ void mlxsw_sp_counter_free(struct mlxsw_sp *mlxsw_sp,
unsigned int counter_index);
int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp);
void mlxsw_sp_counter_pool_fini(struct mlxsw_sp *mlxsw_sp);
+int mlxsw_sp_counter_resources_register(struct mlxsw_core *mlxsw_core);
#endif
--
2.24.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH net-next 6/9] mlxsw: spectrum_cnt: Move config validation along with resource register
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
` (4 preceding siblings ...)
2020-03-18 13:48 ` [PATCH net-next 5/9] mlxsw: spectrum_cnt: Expose subpool sizes over devlink resources Ido Schimmel
@ 2020-03-18 13:48 ` Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 7/9] mlxsw: spectrum_cnt: Consolidate subpools initialization Ido Schimmel
` (4 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Ido Schimmel @ 2020-03-18 13:48 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, mlxsw, Ido Schimmel
From: Jiri Pirko <jiri@mellanox.com>
Move the validation of subpools configuration, to avoid possible over
commitment to resource registration. Add WARN_ON to indicate bug
in the code.
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: Ido Schimmel <idosch@mellanox.com>
---
.../ethernet/mellanox/mlxsw/spectrum_cnt.c | 31 +++++--------------
1 file changed, 8 insertions(+), 23 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
index 629355cf52a9..d36904143b10 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
@@ -40,24 +40,6 @@ static const struct mlxsw_sp_counter_sub_pool mlxsw_sp_counter_sub_pools[] = {
}
};
-static int mlxsw_sp_counter_pool_validate(struct mlxsw_sp *mlxsw_sp)
-{
- struct mlxsw_sp_counter_pool *pool = mlxsw_sp->counter_pool;
- unsigned int total_bank_config = 0;
- unsigned int pool_size;
- unsigned int bank_size;
- int i;
-
- pool_size = MLXSW_CORE_RES_GET(mlxsw_sp->core, COUNTER_POOL_SIZE);
- bank_size = MLXSW_CORE_RES_GET(mlxsw_sp->core, COUNTER_BANK_SIZE);
- /* Check config is valid, no bank over subscription */
- for (i = 0; i < pool->sub_pools_count; i++)
- total_bank_config += pool->sub_pools[i].bank_count;
- if (total_bank_config > pool_size / bank_size + 1)
- return -EINVAL;
- return 0;
-}
-
static int mlxsw_sp_counter_sub_pools_prepare(struct mlxsw_sp *mlxsw_sp)
{
struct mlxsw_sp_counter_pool *pool = mlxsw_sp->counter_pool;
@@ -98,10 +80,6 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
pool->sub_pools_count = sub_pools_count;
spin_lock_init(&pool->counter_pool_lock);
- err = mlxsw_sp_counter_pool_validate(mlxsw_sp);
- if (err)
- goto err_pool_validate;
-
err = mlxsw_sp_counter_sub_pools_prepare(mlxsw_sp);
if (err)
goto err_sub_pools_prepare;
@@ -140,7 +118,6 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
err_usage_alloc:
err_pool_resource_size_get:
err_sub_pools_prepare:
-err_pool_validate:
kfree(pool);
return err;
}
@@ -216,6 +193,7 @@ int mlxsw_sp_counter_resources_register(struct mlxsw_core *mlxsw_core)
static struct devlink_resource_size_params size_params;
struct devlink *devlink = priv_to_devlink(mlxsw_core);
const struct mlxsw_sp_counter_sub_pool *sub_pool;
+ unsigned int total_bank_config;
u64 sub_pool_size;
u64 base_index;
u64 pool_size;
@@ -245,6 +223,7 @@ int mlxsw_sp_counter_resources_register(struct mlxsw_core *mlxsw_core)
/* Allocation is based on bank count which should be
* specified for each sub pool statically.
*/
+ total_bank_config = 0;
base_index = 0;
for (i = 0; i < ARRAY_SIZE(mlxsw_sp_counter_sub_pools); i++) {
sub_pool = &mlxsw_sp_counter_sub_pools[i];
@@ -265,6 +244,12 @@ int mlxsw_sp_counter_resources_register(struct mlxsw_core *mlxsw_core)
&size_params);
if (err)
return err;
+ total_bank_config += sub_pool->bank_count;
}
+
+ /* Check config is valid, no bank over subscription */
+ if (WARN_ON(total_bank_config > pool_size / bank_size + 1))
+ return -EINVAL;
+
return 0;
}
--
2.24.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH net-next 7/9] mlxsw: spectrum_cnt: Consolidate subpools initialization
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
` (5 preceding siblings ...)
2020-03-18 13:48 ` [PATCH net-next 6/9] mlxsw: spectrum_cnt: Move config validation along with resource register Ido Schimmel
@ 2020-03-18 13:48 ` Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 8/9] mlxsw: spectrum_cnt: Expose devlink resource occupancy for counters Ido Schimmel
` (3 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Ido Schimmel @ 2020-03-18 13:48 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, mlxsw, Ido Schimmel
From: Jiri Pirko <jiri@mellanox.com>
Put all init operations related to subpools into
mlxsw_sp_counter_sub_pools_init().
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: Ido Schimmel <idosch@mellanox.com>
---
.../ethernet/mellanox/mlxsw/spectrum_cnt.c | 38 ++++++++-----------
1 file changed, 16 insertions(+), 22 deletions(-)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
index d36904143b10..417c512bc7a2 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
@@ -40,11 +40,14 @@ static const struct mlxsw_sp_counter_sub_pool mlxsw_sp_counter_sub_pools[] = {
}
};
-static int mlxsw_sp_counter_sub_pools_prepare(struct mlxsw_sp *mlxsw_sp)
+static int mlxsw_sp_counter_sub_pools_init(struct mlxsw_sp *mlxsw_sp)
{
struct mlxsw_sp_counter_pool *pool = mlxsw_sp->counter_pool;
+ struct devlink *devlink = priv_to_devlink(mlxsw_sp->core);
struct mlxsw_sp_counter_sub_pool *sub_pool;
+ unsigned int base_index = 0;
enum mlxsw_res_id res_id;
+ int err;
int i;
for (i = 0; i < pool->sub_pools_count; i++) {
@@ -55,6 +58,14 @@ static int mlxsw_sp_counter_sub_pools_prepare(struct mlxsw_sp *mlxsw_sp)
return -EIO;
sub_pool->entry_size = mlxsw_core_res_get(mlxsw_sp->core,
res_id);
+ err = devlink_resource_size_get(devlink,
+ sub_pool->resource_id,
+ &sub_pool->size);
+ if (err)
+ return err;
+
+ sub_pool->base_index = base_index;
+ base_index += sub_pool->size;
}
return 0;
}
@@ -65,9 +76,7 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
struct devlink *devlink = priv_to_devlink(mlxsw_sp->core);
struct mlxsw_sp_counter_sub_pool *sub_pool;
struct mlxsw_sp_counter_pool *pool;
- unsigned int base_index;
unsigned int map_size;
- int i;
int err;
pool = kzalloc(struct_size(pool, sub_pools, sub_pools_count),
@@ -80,10 +89,6 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
pool->sub_pools_count = sub_pools_count;
spin_lock_init(&pool->counter_pool_lock);
- err = mlxsw_sp_counter_sub_pools_prepare(mlxsw_sp);
- if (err)
- goto err_sub_pools_prepare;
-
err = devlink_resource_size_get(devlink, MLXSW_SP_RESOURCE_COUNTERS,
&pool->pool_size);
if (err)
@@ -97,27 +102,16 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
goto err_usage_alloc;
}
- base_index = 0;
- for (i = 0; i < pool->sub_pools_count; i++) {
- sub_pool = &pool->sub_pools[i];
-
- err = devlink_resource_size_get(devlink,
- sub_pool->resource_id,
- &sub_pool->size);
- if (err)
- goto err_sub_pool_resource_size_get;
-
- sub_pool->base_index = base_index;
- base_index += sub_pool->size;
- }
+ err = mlxsw_sp_counter_sub_pools_init(mlxsw_sp);
+ if (err)
+ goto err_sub_pools_init;
return 0;
-err_sub_pool_resource_size_get:
+err_sub_pools_init:
kfree(pool->usage);
err_usage_alloc:
err_pool_resource_size_get:
-err_sub_pools_prepare:
kfree(pool);
return err;
}
--
2.24.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH net-next 8/9] mlxsw: spectrum_cnt: Expose devlink resource occupancy for counters
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
` (6 preceding siblings ...)
2020-03-18 13:48 ` [PATCH net-next 7/9] mlxsw: spectrum_cnt: Consolidate subpools initialization Ido Schimmel
@ 2020-03-18 13:48 ` Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 9/9] selftests: mlxsw: Add tc action hw_stats tests Ido Schimmel
` (2 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Ido Schimmel @ 2020-03-18 13:48 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, mlxsw, Ido Schimmel
From: Jiri Pirko <jiri@mellanox.com>
Implement occupancy counting for counters and expose over devlink
resource API.
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: Ido Schimmel <idosch@mellanox.com>
---
.../ethernet/mellanox/mlxsw/spectrum_cnt.c | 63 ++++++++++++++++++-
1 file changed, 62 insertions(+), 1 deletion(-)
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
index 417c512bc7a2..0268f0a6662a 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum_cnt.c
@@ -15,12 +15,14 @@ struct mlxsw_sp_counter_sub_pool {
u64 resource_id; /* devlink resource id */
unsigned int entry_size;
unsigned int bank_count;
+ atomic_t active_entries_count;
};
struct mlxsw_sp_counter_pool {
u64 pool_size;
unsigned long *usage; /* Usage bitmap */
spinlock_t counter_pool_lock; /* Protects counter pool allocations */
+ atomic_t active_entries_count;
unsigned int sub_pools_count;
struct mlxsw_sp_counter_sub_pool sub_pools[];
};
@@ -40,6 +42,13 @@ static const struct mlxsw_sp_counter_sub_pool mlxsw_sp_counter_sub_pools[] = {
}
};
+static u64 mlxsw_sp_counter_sub_pool_occ_get(void *priv)
+{
+ const struct mlxsw_sp_counter_sub_pool *sub_pool = priv;
+
+ return atomic_read(&sub_pool->active_entries_count);
+}
+
static int mlxsw_sp_counter_sub_pools_init(struct mlxsw_sp *mlxsw_sp)
{
struct mlxsw_sp_counter_pool *pool = mlxsw_sp->counter_pool;
@@ -62,12 +71,50 @@ static int mlxsw_sp_counter_sub_pools_init(struct mlxsw_sp *mlxsw_sp)
sub_pool->resource_id,
&sub_pool->size);
if (err)
- return err;
+ goto err_resource_size_get;
+
+ devlink_resource_occ_get_register(devlink,
+ sub_pool->resource_id,
+ mlxsw_sp_counter_sub_pool_occ_get,
+ sub_pool);
sub_pool->base_index = base_index;
base_index += sub_pool->size;
+ atomic_set(&sub_pool->active_entries_count, 0);
}
return 0;
+
+err_resource_size_get:
+ for (i--; i >= 0; i--) {
+ sub_pool = &pool->sub_pools[i];
+
+ devlink_resource_occ_get_unregister(devlink,
+ sub_pool->resource_id);
+ }
+ return err;
+}
+
+static void mlxsw_sp_counter_sub_pools_fini(struct mlxsw_sp *mlxsw_sp)
+{
+ struct mlxsw_sp_counter_pool *pool = mlxsw_sp->counter_pool;
+ struct devlink *devlink = priv_to_devlink(mlxsw_sp->core);
+ struct mlxsw_sp_counter_sub_pool *sub_pool;
+ int i;
+
+ for (i = 0; i < pool->sub_pools_count; i++) {
+ sub_pool = &pool->sub_pools[i];
+
+ WARN_ON(atomic_read(&sub_pool->active_entries_count));
+ devlink_resource_occ_get_unregister(devlink,
+ sub_pool->resource_id);
+ }
+}
+
+static u64 mlxsw_sp_counter_pool_occ_get(void *priv)
+{
+ const struct mlxsw_sp_counter_pool *pool = priv;
+
+ return atomic_read(&pool->active_entries_count);
}
int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
@@ -88,11 +135,14 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
sub_pools_count * sizeof(*sub_pool));
pool->sub_pools_count = sub_pools_count;
spin_lock_init(&pool->counter_pool_lock);
+ atomic_set(&pool->active_entries_count, 0);
err = devlink_resource_size_get(devlink, MLXSW_SP_RESOURCE_COUNTERS,
&pool->pool_size);
if (err)
goto err_pool_resource_size_get;
+ devlink_resource_occ_get_register(devlink, MLXSW_SP_RESOURCE_COUNTERS,
+ mlxsw_sp_counter_pool_occ_get, pool);
map_size = BITS_TO_LONGS(pool->pool_size) * sizeof(unsigned long);
@@ -111,6 +161,8 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
err_sub_pools_init:
kfree(pool->usage);
err_usage_alloc:
+ devlink_resource_occ_get_unregister(devlink,
+ MLXSW_SP_RESOURCE_COUNTERS);
err_pool_resource_size_get:
kfree(pool);
return err;
@@ -119,10 +171,15 @@ int mlxsw_sp_counter_pool_init(struct mlxsw_sp *mlxsw_sp)
void mlxsw_sp_counter_pool_fini(struct mlxsw_sp *mlxsw_sp)
{
struct mlxsw_sp_counter_pool *pool = mlxsw_sp->counter_pool;
+ struct devlink *devlink = priv_to_devlink(mlxsw_sp->core);
+ mlxsw_sp_counter_sub_pools_fini(mlxsw_sp);
WARN_ON(find_first_bit(pool->usage, pool->pool_size) !=
pool->pool_size);
+ WARN_ON(atomic_read(&pool->active_entries_count));
kfree(pool->usage);
+ devlink_resource_occ_get_unregister(devlink,
+ MLXSW_SP_RESOURCE_COUNTERS);
kfree(pool);
}
@@ -158,6 +215,8 @@ int mlxsw_sp_counter_alloc(struct mlxsw_sp *mlxsw_sp,
spin_unlock(&pool->counter_pool_lock);
*p_counter_index = entry_index;
+ atomic_add(sub_pool->entry_size, &sub_pool->active_entries_count);
+ atomic_add(sub_pool->entry_size, &pool->active_entries_count);
return 0;
err_alloc:
@@ -180,6 +239,8 @@ void mlxsw_sp_counter_free(struct mlxsw_sp *mlxsw_sp,
for (i = 0; i < sub_pool->entry_size; i++)
__clear_bit(counter_index + i, pool->usage);
spin_unlock(&pool->counter_pool_lock);
+ atomic_sub(sub_pool->entry_size, &sub_pool->active_entries_count);
+ atomic_sub(sub_pool->entry_size, &pool->active_entries_count);
}
int mlxsw_sp_counter_resources_register(struct mlxsw_core *mlxsw_core)
--
2.24.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH net-next 9/9] selftests: mlxsw: Add tc action hw_stats tests
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
` (7 preceding siblings ...)
2020-03-18 13:48 ` [PATCH net-next 8/9] mlxsw: spectrum_cnt: Expose devlink resource occupancy for counters Ido Schimmel
@ 2020-03-18 13:48 ` Ido Schimmel
2020-03-18 20:50 ` [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Jakub Kicinski
2020-03-18 23:46 ` David Miller
10 siblings, 0 replies; 12+ messages in thread
From: Ido Schimmel @ 2020-03-18 13:48 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, mlxsw, Ido Schimmel
From: Jiri Pirko <jiri@mellanox.com>
Add tests for mlxsw hw_stats types.
Signed-off-by: Jiri Pirko <jiri@mellanox.com>
Signed-off-by: Ido Schimmel <idosch@mellanox.com>
---
.../drivers/net/mlxsw/tc_action_hw_stats.sh | 130 ++++++++++++++++++
tools/testing/selftests/net/forwarding/lib.sh | 9 ++
2 files changed, 139 insertions(+)
create mode 100755 tools/testing/selftests/drivers/net/mlxsw/tc_action_hw_stats.sh
diff --git a/tools/testing/selftests/drivers/net/mlxsw/tc_action_hw_stats.sh b/tools/testing/selftests/drivers/net/mlxsw/tc_action_hw_stats.sh
new file mode 100755
index 000000000000..20ed98fe5a60
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/mlxsw/tc_action_hw_stats.sh
@@ -0,0 +1,130 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+lib_dir=$(dirname $0)/../../../net/forwarding
+
+ALL_TESTS="
+ default_hw_stats_test
+ immediate_hw_stats_test
+ delayed_hw_stats_test
+ disabled_hw_stats_test
+"
+NUM_NETIFS=2
+
+source $lib_dir/tc_common.sh
+source $lib_dir/lib.sh
+source $lib_dir/devlink_lib.sh
+
+h1_create()
+{
+ simple_if_init $h1 192.0.2.1/24
+}
+
+h1_destroy()
+{
+ simple_if_fini $h1 192.0.2.1/24
+}
+
+switch_create()
+{
+ simple_if_init $swp1 192.0.2.2/24
+ tc qdisc add dev $swp1 clsact
+}
+
+switch_destroy()
+{
+ tc qdisc del dev $swp1 clsact
+ simple_if_fini $swp1 192.0.2.2/24
+}
+
+hw_stats_test()
+{
+ RET=0
+
+ local name=$1
+ local action_hw_stats=$2
+ local occ_delta=$3
+ local expected_packet_count=$4
+
+ local orig_occ=$(devlink_resource_get "counters" "flow" | jq '.["occ"]')
+
+ tc filter add dev $swp1 ingress protocol ip pref 1 handle 101 flower \
+ skip_sw dst_ip 192.0.2.2 action drop $action_hw_stats
+ check_err $? "Failed to add rule with $name hw_stats"
+
+ local new_occ=$(devlink_resource_get "counters" "flow" | jq '.["occ"]')
+ local expected_occ=$((orig_occ + occ_delta))
+ [ "$new_occ" == "$expected_occ" ]
+ check_err $? "Expected occupancy of $expected_occ, got $new_occ"
+
+ $MZ $h1 -c 1 -p 64 -a $h1mac -b $swp1mac -A 192.0.2.1 -B 192.0.2.2 \
+ -t ip -q
+
+ tc_check_packets "dev $swp1 ingress" 101 $expected_packet_count
+ check_err $? "Did not match incoming packet"
+
+ tc filter del dev $swp1 ingress protocol ip pref 1 handle 101 flower
+
+ log_test "$name hw_stats"
+}
+
+default_hw_stats_test()
+{
+ hw_stats_test "default" "" 2 1
+}
+
+immediate_hw_stats_test()
+{
+ hw_stats_test "immediate" "hw_stats immediate" 2 1
+}
+
+delayed_hw_stats_test()
+{
+ RET=0
+
+ tc filter add dev $swp1 ingress protocol ip pref 1 handle 101 flower \
+ skip_sw dst_ip 192.0.2.2 action drop hw_stats delayed
+ check_fail $? "Unexpected success in adding rule with delayed hw_stats"
+
+ log_test "delayed hw_stats"
+}
+
+disabled_hw_stats_test()
+{
+ hw_stats_test "disabled" "hw_stats disabled" 0 0
+}
+
+setup_prepare()
+{
+ h1=${NETIFS[p1]}
+ swp1=${NETIFS[p2]}
+
+ h1mac=$(mac_get $h1)
+ swp1mac=$(mac_get $swp1)
+
+ vrf_prepare
+
+ h1_create
+ switch_create
+}
+
+cleanup()
+{
+ pre_cleanup
+
+ switch_destroy
+ h1_destroy
+
+ vrf_cleanup
+}
+
+check_tc_action_hw_stats_support
+
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+
+tests_run
+
+exit $EXIT_STATUS
diff --git a/tools/testing/selftests/net/forwarding/lib.sh b/tools/testing/selftests/net/forwarding/lib.sh
index a4a7879b3bb9..977fc2b326a2 100644
--- a/tools/testing/selftests/net/forwarding/lib.sh
+++ b/tools/testing/selftests/net/forwarding/lib.sh
@@ -60,6 +60,15 @@ check_tc_chain_support()
fi
}
+check_tc_action_hw_stats_support()
+{
+ tc actions help 2>&1 | grep -q hw_stats
+ if [[ $? -ne 0 ]]; then
+ echo "SKIP: iproute2 too old; tc is missing action hw_stats support"
+ exit 1
+ fi
+}
+
if [[ "$(id -u)" -ne 0 ]]; then
echo "SKIP: need root privileges"
exit 0
--
2.24.1
^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
` (8 preceding siblings ...)
2020-03-18 13:48 ` [PATCH net-next 9/9] selftests: mlxsw: Add tc action hw_stats tests Ido Schimmel
@ 2020-03-18 20:50 ` Jakub Kicinski
2020-03-18 23:46 ` David Miller
10 siblings, 0 replies; 12+ messages in thread
From: Jakub Kicinski @ 2020-03-18 20:50 UTC (permalink / raw)
To: Ido Schimmel; +Cc: netdev, davem, jiri, mlxsw, Ido Schimmel
On Wed, 18 Mar 2020 15:48:48 +0200 Ido Schimmel wrote:
> From: Ido Schimmel <idosch@mellanox.com>
>
> Jiri says:
>
> Capacity and utilization of existing flow and RIF counters are currently
> unavailable to be seen by the user. Use the existing devlink resources
> API to expose the information:
>
> $ sudo devlink resource show pci/0000:00:10.0 -v
> pci/0000:00:10.0:
> name kvd resource_path /kvd size 524288 unit entry dpipe_tables none
> name span_agents resource_path /span_agents size 8 occ 0 unit entry dpipe_tables none
> name counters resource_path /counters size 79872 occ 44 unit entry dpipe_tables none
> resources:
> name flow resource_path /counters/flow size 61440 occ 4 unit entry dpipe_tables none
> name rif resource_path /counters/rif size 18432 occ 40 unit entry dpipe_tables none
Acked-by: Jakub Kicinski <kuba@kernel.org>
Nice!
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
` (9 preceding siblings ...)
2020-03-18 20:50 ` [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Jakub Kicinski
@ 2020-03-18 23:46 ` David Miller
10 siblings, 0 replies; 12+ messages in thread
From: David Miller @ 2020-03-18 23:46 UTC (permalink / raw)
To: idosch; +Cc: netdev, kuba, jiri, mlxsw, idosch
From: Ido Schimmel <idosch@idosch.org>
Date: Wed, 18 Mar 2020 15:48:48 +0200
> From: Ido Schimmel <idosch@mellanox.com>
>
> Jiri says:
>
> Capacity and utilization of existing flow and RIF counters are currently
> unavailable to be seen by the user. Use the existing devlink resources
> API to expose the information:
>
> $ sudo devlink resource show pci/0000:00:10.0 -v
> pci/0000:00:10.0:
> name kvd resource_path /kvd size 524288 unit entry dpipe_tables none
> name span_agents resource_path /span_agents size 8 occ 0 unit entry dpipe_tables none
> name counters resource_path /counters size 79872 occ 44 unit entry dpipe_tables none
> resources:
> name flow resource_path /counters/flow size 61440 occ 4 unit entry dpipe_tables none
> name rif resource_path /counters/rif size 18432 occ 40 unit entry dpipe_tables none
Series applied, thanks everyone.
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2020-03-18 23:46 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-18 13:48 [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 1/9] mlxsw: spectrum_cnt: Query bank size from FW resources Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 2/9] selftests: spectrum-2: Adjust tc_flower_scale limit according to current counter count Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 3/9] mlxsw: spectrum_cnt: Move sub_pools under per-instance pool struct Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 4/9] mlxsw: spectrum_cnt: Add entry_size_res_id for each subpool and use it to query entry size Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 5/9] mlxsw: spectrum_cnt: Expose subpool sizes over devlink resources Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 6/9] mlxsw: spectrum_cnt: Move config validation along with resource register Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 7/9] mlxsw: spectrum_cnt: Consolidate subpools initialization Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 8/9] mlxsw: spectrum_cnt: Expose devlink resource occupancy for counters Ido Schimmel
2020-03-18 13:48 ` [PATCH net-next 9/9] selftests: mlxsw: Add tc action hw_stats tests Ido Schimmel
2020-03-18 20:50 ` [PATCH net-next 0/9] mlxsw: spectrum_cnt: Expose counter resources Jakub Kicinski
2020-03-18 23:46 ` David Miller
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).