* [PATCH net-next v2 0/2] mlxsw: Expose number of physical ports
@ 2021-01-21 13:10 Ido Schimmel
2021-01-21 13:10 ` [PATCH net-next v2 1/2] mlxsw: Register physical ports as a devlink resource Ido Schimmel
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Ido Schimmel @ 2021-01-21 13:10 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, danieller, mlxsw, Ido Schimmel
From: Ido Schimmel <idosch@nvidia.com>
The switch ASIC has a limited capacity of physical ports that it can
support. While each system is brought up with a different number of
ports, this number can be increased via splitting up to the ASIC's
limit.
Expose physical ports as a devlink resource so that user space will have
visibility into the maximum number of ports that can be supported and
the current occupancy. With this resource it is possible, for example,
to write generic (i.e., not platform dependent) tests for port
splitting.
Patch #1 adds the new resource and patch #2 adds a selftest.
v2:
* Add the physical ports resource as a generic devlink resource so that
it could be re-used by other device drivers
Danielle Ratson (2):
mlxsw: Register physical ports as a devlink resource
selftests: mlxsw: Add a scale test for physical ports
.../networking/devlink/devlink-resource.rst | 14 ++++
drivers/net/ethernet/mellanox/mlxsw/core.c | 77 ++++++++++++++++---
drivers/net/ethernet/mellanox/mlxsw/core.h | 5 ++
.../net/ethernet/mellanox/mlxsw/spectrum.h | 2 +-
include/net/devlink.h | 2 +
net/core/devlink.c | 4 +
.../selftests/drivers/net/mlxsw/port_scale.sh | 64 +++++++++++++++
.../net/mlxsw/spectrum-2/port_scale.sh | 16 ++++
.../net/mlxsw/spectrum-2/resource_scale.sh | 2 +-
.../drivers/net/mlxsw/spectrum/port_scale.sh | 16 ++++
.../net/mlxsw/spectrum/resource_scale.sh | 2 +-
11 files changed, 191 insertions(+), 13 deletions(-)
create mode 100644 tools/testing/selftests/drivers/net/mlxsw/port_scale.sh
create mode 100644 tools/testing/selftests/drivers/net/mlxsw/spectrum-2/port_scale.sh
create mode 100644 tools/testing/selftests/drivers/net/mlxsw/spectrum/port_scale.sh
--
2.29.2
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH net-next v2 1/2] mlxsw: Register physical ports as a devlink resource
2021-01-21 13:10 [PATCH net-next v2 0/2] mlxsw: Expose number of physical ports Ido Schimmel
@ 2021-01-21 13:10 ` Ido Schimmel
2021-01-21 13:10 ` [PATCH net-next v2 2/2] selftests: mlxsw: Add a scale test for physical ports Ido Schimmel
2021-01-23 6:10 ` [PATCH net-next v2 0/2] mlxsw: Expose number of " patchwork-bot+netdevbpf
2 siblings, 0 replies; 4+ messages in thread
From: Ido Schimmel @ 2021-01-21 13:10 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, danieller, mlxsw, Ido Schimmel
From: Danielle Ratson <danieller@nvidia.com>
The switch ASIC has a limited capacity of physical ('flavour physical'
in devlink terminology) ports that it can support. While each system is
brought up with a different number of ports, this number can be
increased via splitting up to the ASIC's limit.
Expose physical ports as a devlink resource so that user space will have
visibility to the maximum number of ports that can be supported and the
current occupancy.
In addition, add a "Generic Resources" section in devlink-resource
documentation so the different drivers will be aligned by the same resource
name when exposing to user space.
Signed-off-by: Danielle Ratson <danieller@nvidia.com>
Reviewed-by: Jiri Pirko <jiri@nvidia.com>
Signed-off-by: Ido Schimmel <idosch@nvidia.com>
---
.../networking/devlink/devlink-resource.rst | 14 ++++
drivers/net/ethernet/mellanox/mlxsw/core.c | 77 ++++++++++++++++---
drivers/net/ethernet/mellanox/mlxsw/core.h | 5 ++
.../net/ethernet/mellanox/mlxsw/spectrum.h | 2 +-
include/net/devlink.h | 2 +
net/core/devlink.c | 4 +
6 files changed, 93 insertions(+), 11 deletions(-)
diff --git a/Documentation/networking/devlink/devlink-resource.rst b/Documentation/networking/devlink/devlink-resource.rst
index 93e92d2f0752..3d5ae51e65a2 100644
--- a/Documentation/networking/devlink/devlink-resource.rst
+++ b/Documentation/networking/devlink/devlink-resource.rst
@@ -23,6 +23,20 @@ current size and related sub resources. To access a sub resource, you
specify the path of the resource. For example ``/IPv4/fib`` is the id for
the ``fib`` sub-resource under the ``IPv4`` resource.
+Generic Resources
+=================
+
+Generic resources are used to describe resources that can be shared by multiple
+device drivers and their description must be added to the following table:
+
+.. list-table:: List of Generic Resources
+ :widths: 10 90
+
+ * - Name
+ - Description
+ * - ``physical_ports``
+ - A limited capacity of physical ports that the switch ASIC can support
+
example usage
-------------
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.c b/drivers/net/ethernet/mellanox/mlxsw/core.c
index 685037e052af..52fdc34251ba 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core.c
+++ b/drivers/net/ethernet/mellanox/mlxsw/core.c
@@ -84,6 +84,7 @@ struct mlxsw_core {
struct mlxsw_thermal *thermal;
struct mlxsw_core_port *ports;
unsigned int max_ports;
+ atomic_t active_ports_count;
bool fw_flash_in_progress;
struct {
struct devlink_health_reporter *fw_fatal;
@@ -96,8 +97,36 @@ struct mlxsw_core {
#define MLXSW_PORT_MAX_PORTS_DEFAULT 0x40
-static int mlxsw_ports_init(struct mlxsw_core *mlxsw_core)
+static u64 mlxsw_ports_occ_get(void *priv)
{
+ struct mlxsw_core *mlxsw_core = priv;
+
+ return atomic_read(&mlxsw_core->active_ports_count);
+}
+
+static int mlxsw_core_resources_ports_register(struct mlxsw_core *mlxsw_core)
+{
+ struct devlink *devlink = priv_to_devlink(mlxsw_core);
+ struct devlink_resource_size_params ports_num_params;
+ u32 max_ports;
+
+ max_ports = mlxsw_core->max_ports - 1;
+ devlink_resource_size_params_init(&ports_num_params, max_ports,
+ max_ports, 1,
+ DEVLINK_RESOURCE_UNIT_ENTRY);
+
+ return devlink_resource_register(devlink,
+ DEVLINK_RESOURCE_GENERIC_NAME_PORTS,
+ max_ports, MLXSW_CORE_RESOURCE_PORTS,
+ DEVLINK_RESOURCE_ID_PARENT_TOP,
+ &ports_num_params);
+}
+
+static int mlxsw_ports_init(struct mlxsw_core *mlxsw_core, bool reload)
+{
+ struct devlink *devlink = priv_to_devlink(mlxsw_core);
+ int err;
+
/* Switch ports are numbered from 1 to queried value */
if (MLXSW_CORE_RES_VALID(mlxsw_core, MAX_SYSTEM_PORT))
mlxsw_core->max_ports = MLXSW_CORE_RES_GET(mlxsw_core,
@@ -110,11 +139,30 @@ static int mlxsw_ports_init(struct mlxsw_core *mlxsw_core)
if (!mlxsw_core->ports)
return -ENOMEM;
+ if (!reload) {
+ err = mlxsw_core_resources_ports_register(mlxsw_core);
+ if (err)
+ goto err_resources_ports_register;
+ }
+ atomic_set(&mlxsw_core->active_ports_count, 0);
+ devlink_resource_occ_get_register(devlink, MLXSW_CORE_RESOURCE_PORTS,
+ mlxsw_ports_occ_get, mlxsw_core);
+
return 0;
+
+err_resources_ports_register:
+ kfree(mlxsw_core->ports);
+ return err;
}
-static void mlxsw_ports_fini(struct mlxsw_core *mlxsw_core)
+static void mlxsw_ports_fini(struct mlxsw_core *mlxsw_core, bool reload)
{
+ struct devlink *devlink = priv_to_devlink(mlxsw_core);
+
+ devlink_resource_occ_get_unregister(devlink, MLXSW_CORE_RESOURCE_PORTS);
+ if (!reload)
+ devlink_resources_unregister(priv_to_devlink(mlxsw_core), NULL);
+
kfree(mlxsw_core->ports);
}
@@ -1897,7 +1945,7 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info,
goto err_register_resources;
}
- err = mlxsw_ports_init(mlxsw_core);
+ err = mlxsw_ports_init(mlxsw_core, reload);
if (err)
goto err_ports_init;
@@ -1986,7 +2034,7 @@ __mlxsw_core_bus_device_register(const struct mlxsw_bus_info *mlxsw_bus_info,
err_emad_init:
kfree(mlxsw_core->lag.mapping);
err_alloc_lag_mapping:
- mlxsw_ports_fini(mlxsw_core);
+ mlxsw_ports_fini(mlxsw_core, reload);
err_ports_init:
if (!reload)
devlink_resources_unregister(devlink, NULL);
@@ -2056,7 +2104,7 @@ void mlxsw_core_bus_device_unregister(struct mlxsw_core *mlxsw_core,
devlink_unregister(devlink);
mlxsw_emad_fini(mlxsw_core);
kfree(mlxsw_core->lag.mapping);
- mlxsw_ports_fini(mlxsw_core);
+ mlxsw_ports_fini(mlxsw_core, reload);
if (!reload)
devlink_resources_unregister(devlink, NULL);
mlxsw_core->bus->fini(mlxsw_core->bus_priv);
@@ -2755,16 +2803,25 @@ int mlxsw_core_port_init(struct mlxsw_core *mlxsw_core, u8 local_port,
const unsigned char *switch_id,
unsigned char switch_id_len)
{
- return __mlxsw_core_port_init(mlxsw_core, local_port,
- DEVLINK_PORT_FLAVOUR_PHYSICAL,
- port_number, split, split_port_subnumber,
- splittable, lanes,
- switch_id, switch_id_len);
+ int err;
+
+ err = __mlxsw_core_port_init(mlxsw_core, local_port,
+ DEVLINK_PORT_FLAVOUR_PHYSICAL,
+ port_number, split, split_port_subnumber,
+ splittable, lanes,
+ switch_id, switch_id_len);
+ if (err)
+ return err;
+
+ atomic_inc(&mlxsw_core->active_ports_count);
+ return 0;
}
EXPORT_SYMBOL(mlxsw_core_port_init);
void mlxsw_core_port_fini(struct mlxsw_core *mlxsw_core, u8 local_port)
{
+ atomic_dec(&mlxsw_core->active_ports_count);
+
__mlxsw_core_port_fini(mlxsw_core, local_port);
}
EXPORT_SYMBOL(mlxsw_core_port_fini);
diff --git a/drivers/net/ethernet/mellanox/mlxsw/core.h b/drivers/net/ethernet/mellanox/mlxsw/core.h
index 6b3ccbf6b238..8af7d9d03475 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/core.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/core.h
@@ -19,6 +19,11 @@
#include "cmd.h"
#include "resources.h"
+enum mlxsw_core_resource_id {
+ MLXSW_CORE_RESOURCE_PORTS = 1,
+ MLXSW_CORE_RESOURCE_MAX,
+};
+
struct mlxsw_core;
struct mlxsw_core_port;
struct mlxsw_driver;
diff --git a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
index a6956cfc9cb1..a3769f95a182 100644
--- a/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
+++ b/drivers/net/ethernet/mellanox/mlxsw/spectrum.h
@@ -52,7 +52,7 @@
#define MLXSW_SP_RESOURCE_NAME_COUNTERS_RIF "rif"
enum mlxsw_sp_resource_id {
- MLXSW_SP_RESOURCE_KVD = 1,
+ MLXSW_SP_RESOURCE_KVD = MLXSW_CORE_RESOURCE_MAX,
MLXSW_SP_RESOURCE_KVD_LINEAR,
MLXSW_SP_RESOURCE_KVD_HASH_SINGLE,
MLXSW_SP_RESOURCE_KVD_HASH_DOUBLE,
diff --git a/include/net/devlink.h b/include/net/devlink.h
index f466819cc477..d12ed2854c34 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -380,6 +380,8 @@ struct devlink_resource {
#define DEVLINK_RESOURCE_ID_PARENT_TOP 0
+#define DEVLINK_RESOURCE_GENERIC_NAME_PORTS "physical_ports"
+
#define __DEVLINK_PARAM_MAX_STRING_VALUE 32
enum devlink_param_type {
DEVLINK_PARAM_TYPE_U8,
diff --git a/net/core/devlink.c b/net/core/devlink.c
index 738d4344d679..72ea79879762 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -8617,6 +8617,10 @@ EXPORT_SYMBOL_GPL(devlink_dpipe_table_unregister);
* @resource_id: resource's id
* @parent_resource_id: resource's parent id
* @size_params: size parameters
+ *
+ * Generic resources should reuse the same names across drivers.
+ * Please see the generic resources list at:
+ * Documentation/networking/devlink/devlink-resource.rst
*/
int devlink_resource_register(struct devlink *devlink,
const char *resource_name,
--
2.29.2
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH net-next v2 2/2] selftests: mlxsw: Add a scale test for physical ports
2021-01-21 13:10 [PATCH net-next v2 0/2] mlxsw: Expose number of physical ports Ido Schimmel
2021-01-21 13:10 ` [PATCH net-next v2 1/2] mlxsw: Register physical ports as a devlink resource Ido Schimmel
@ 2021-01-21 13:10 ` Ido Schimmel
2021-01-23 6:10 ` [PATCH net-next v2 0/2] mlxsw: Expose number of " patchwork-bot+netdevbpf
2 siblings, 0 replies; 4+ messages in thread
From: Ido Schimmel @ 2021-01-21 13:10 UTC (permalink / raw)
To: netdev; +Cc: davem, kuba, jiri, danieller, mlxsw, Ido Schimmel
From: Danielle Ratson <danieller@nvidia.com>
Query the maximum number of supported physical ports using devlink-resource
and test that this number can be reached by splitting each of the
splittable ports to its width. Test that an error is returned in case
the maximum number is exceeded.
Signed-off-by: Danielle Ratson <danieller@nvidia.com>
Signed-off-by: Ido Schimmel <idosch@nvidia.com>
---
.../selftests/drivers/net/mlxsw/port_scale.sh | 64 +++++++++++++++++++
.../net/mlxsw/spectrum-2/port_scale.sh | 16 +++++
.../net/mlxsw/spectrum-2/resource_scale.sh | 2 +-
.../drivers/net/mlxsw/spectrum/port_scale.sh | 16 +++++
.../net/mlxsw/spectrum/resource_scale.sh | 2 +-
5 files changed, 98 insertions(+), 2 deletions(-)
create mode 100644 tools/testing/selftests/drivers/net/mlxsw/port_scale.sh
create mode 100644 tools/testing/selftests/drivers/net/mlxsw/spectrum-2/port_scale.sh
create mode 100644 tools/testing/selftests/drivers/net/mlxsw/spectrum/port_scale.sh
diff --git a/tools/testing/selftests/drivers/net/mlxsw/port_scale.sh b/tools/testing/selftests/drivers/net/mlxsw/port_scale.sh
new file mode 100644
index 000000000000..f813ffefc07e
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/mlxsw/port_scale.sh
@@ -0,0 +1,64 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+# Test for physical ports resource. The test splits each splittable port
+# to its width and checks that eventually the number of physical ports equals
+# the maximum number of physical ports.
+
+PORT_NUM_NETIFS=0
+
+port_setup_prepare()
+{
+ :
+}
+
+port_cleanup()
+{
+ pre_cleanup
+
+ for port in "${unsplit[@]}"; do
+ devlink port unsplit $port
+ check_err $? "Did not unsplit $netdev"
+ done
+}
+
+split_all_ports()
+{
+ local should_fail=$1; shift
+ local -a unsplit
+
+ # Loop over the splittable netdevs and create tuples of netdev along
+ # with its width. For example:
+ # '$netdev1 $count1 $netdev2 $count2...', when:
+ # $netdev1-2 are splittable netdevs in the device, and
+ # $count1-2 are the netdevs width respectively.
+ while read netdev count <<<$(
+ devlink -j port show |
+ jq -r '.[][] | select(.splittable==true) | "\(.netdev) \(.lanes)"'
+ )
+ [[ ! -z $netdev ]]
+ do
+ devlink port split $netdev count $count
+ check_err $? "Did not split $netdev into $count"
+ unsplit+=( "${netdev}s0" )
+ done
+}
+
+port_test()
+{
+ local max_ports=$1; shift
+ local should_fail=$1; shift
+
+ split_all_ports $should_fail
+
+ occ=$(devlink -j resource show $DEVLINK_DEV \
+ | jq '.[][][] | select(.name=="physical_ports") |.["occ"]')
+
+ [[ $occ -eq $max_ports ]]
+ if [[ $should_fail -eq 0 ]]; then
+ check_err $? "Mismatch ports number: Expected $max_ports, got $occ."
+ else
+ check_err_fail $should_fail $? "Reached more ports than expected"
+ fi
+
+}
diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/port_scale.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/port_scale.sh
new file mode 100644
index 000000000000..0b71dfbbb447
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/port_scale.sh
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: GPL-2.0
+source ../port_scale.sh
+
+port_get_target()
+{
+ local should_fail=$1
+ local target
+
+ target=$(devlink_resource_size_get physical_ports)
+
+ if ((! should_fail)); then
+ echo $target
+ else
+ echo $((target + 1))
+ fi
+}
diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/resource_scale.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/resource_scale.sh
index d7cf33a3f18d..4a1c9328555f 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/resource_scale.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum-2/resource_scale.sh
@@ -28,7 +28,7 @@ cleanup()
trap cleanup EXIT
-ALL_TESTS="router tc_flower mirror_gre tc_police"
+ALL_TESTS="router tc_flower mirror_gre tc_police port"
for current_test in ${TESTS:-$ALL_TESTS}; do
source ${current_test}_scale.sh
diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum/port_scale.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum/port_scale.sh
new file mode 100644
index 000000000000..0b71dfbbb447
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum/port_scale.sh
@@ -0,0 +1,16 @@
+# SPDX-License-Identifier: GPL-2.0
+source ../port_scale.sh
+
+port_get_target()
+{
+ local should_fail=$1
+ local target
+
+ target=$(devlink_resource_size_get physical_ports)
+
+ if ((! should_fail)); then
+ echo $target
+ else
+ echo $((target + 1))
+ fi
+}
diff --git a/tools/testing/selftests/drivers/net/mlxsw/spectrum/resource_scale.sh b/tools/testing/selftests/drivers/net/mlxsw/spectrum/resource_scale.sh
index 43f662401bc3..087a884f66cd 100755
--- a/tools/testing/selftests/drivers/net/mlxsw/spectrum/resource_scale.sh
+++ b/tools/testing/selftests/drivers/net/mlxsw/spectrum/resource_scale.sh
@@ -22,7 +22,7 @@ cleanup()
devlink_sp_read_kvd_defaults
trap cleanup EXIT
-ALL_TESTS="router tc_flower mirror_gre tc_police"
+ALL_TESTS="router tc_flower mirror_gre tc_police port"
for current_test in ${TESTS:-$ALL_TESTS}; do
source ${current_test}_scale.sh
--
2.29.2
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH net-next v2 0/2] mlxsw: Expose number of physical ports
2021-01-21 13:10 [PATCH net-next v2 0/2] mlxsw: Expose number of physical ports Ido Schimmel
2021-01-21 13:10 ` [PATCH net-next v2 1/2] mlxsw: Register physical ports as a devlink resource Ido Schimmel
2021-01-21 13:10 ` [PATCH net-next v2 2/2] selftests: mlxsw: Add a scale test for physical ports Ido Schimmel
@ 2021-01-23 6:10 ` patchwork-bot+netdevbpf
2 siblings, 0 replies; 4+ messages in thread
From: patchwork-bot+netdevbpf @ 2021-01-23 6:10 UTC (permalink / raw)
To: Ido Schimmel; +Cc: netdev, davem, kuba, jiri, danieller, mlxsw, idosch
Hello:
This series was applied to netdev/net-next.git (refs/heads/master):
On Thu, 21 Jan 2021 15:10:22 +0200 you wrote:
> From: Ido Schimmel <idosch@nvidia.com>
>
> The switch ASIC has a limited capacity of physical ports that it can
> support. While each system is brought up with a different number of
> ports, this number can be increased via splitting up to the ASIC's
> limit.
>
> [...]
Here is the summary with links:
- [net-next,v2,1/2] mlxsw: Register physical ports as a devlink resource
https://git.kernel.org/netdev/net-next/c/321f7ab0d458
- [net-next,v2,2/2] selftests: mlxsw: Add a scale test for physical ports
https://git.kernel.org/netdev/net-next/c/5154b1b826d9
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-01-23 6:11 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-21 13:10 [PATCH net-next v2 0/2] mlxsw: Expose number of physical ports Ido Schimmel
2021-01-21 13:10 ` [PATCH net-next v2 1/2] mlxsw: Register physical ports as a devlink resource Ido Schimmel
2021-01-21 13:10 ` [PATCH net-next v2 2/2] selftests: mlxsw: Add a scale test for physical ports Ido Schimmel
2021-01-23 6:10 ` [PATCH net-next v2 0/2] mlxsw: Expose number of " patchwork-bot+netdevbpf
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).