Netdev Archive on lore.kernel.org
 help / color / Atom feed
* [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW
@ 2020-03-26  3:51 Jacob Keller
  2020-03-26  3:51 ` [net-next v2 01/11] devlink: prepare to support region operations Jacob Keller
                   ` (15 more replies)
  0 siblings, 16 replies; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller

This is a second revision of the previous series to implement the
DEVLINK_CMD_REGION_NEW. The series can be viewed on lore.kernel.org at

https://lore.kernel.org/netdev/20200324223445.2077900-1-jacob.e.keller@intel.com/

This version includes the suggested cleanups from Jakub and Jiri on the
list, including the following changes, broken out by the v1 patch title.

Changes to patches since v1:

 * devlink: prepare to support region operations

   No changes

 * devlink: convert snapshot destructor callback to region op

   No changes

 * devlink: trivial: fix tab in function documentation

   No changes

 * devlink: add function to take snapshot while locked

   Added Jakub's Reviewed-by tag.

 * <NEW> devlink: use -ENOSPC to indicate no more room for snapshots

   New patch added to convert confusing -ENOMEM to -ENOSPC, as suggested by
   Jiri.

 * devlink: extract snapshot id allocation to helper function

   No changes

 * devlink: convert snapshot id getter to return an error

   Changed title to "devlink: report error once U32_MAX snapshot ids have
   been used".

   Refactored this patch to make devlink_region_snapshot_id_get take a
   pointer to u32, so that the error value and id value are separated. This
   means that we can remove the INT_MAX limitation on id values.

 * devlink: track snapshot id usage count using an xarray

   Fixed the xa_init to use xa_init_flags with XA_FLAGS_ALLOC, so that
   xa_alloc can properly be used.

   Changed devlink_region_snapshot_id_get to use an initial count of 1
   instead of 0. Added a new devlink_region_snapshot_id_put function, used
   to release this initial count. This closes the race condition and issues
   caused if the driver either doesn't create a snapshot, or if userspace
   deletes the first snapshot before others are created.

   Used WARN_ON in a few more checks that should not occur, such as if the
   xarray entry is not a value, or when the id isn't yet in the xarray.

   Removed an unnecessary if (err) { return err; } construction.

   Use xa_limit_32b instead of xa_limit_31b now that we don't return the
   snapshot id directly.

   Cleanup the label used in __devlink_region_snapshot_create to indicate the
   failure cause, rather than the cleanup step.

   Removed the unnecessary locking around xa_destroy

 * devlink: implement DEVLINK_CMD_REGION_NEW

   Added a WARN_ON to the check in snapshot_id_insert in case the id already
   exists.

   Removed an unnecessary "if (err) { return err; }" construction

   Use -ENOSPC instead of -ENOMEM when max_snapshots is reached.

   Cleanup label names to match style of the other labels in the file,
   naming after the failure cause rather than the cleanup step. Also fix a
   bug in the label ordering.

   Call the new devlink_region_snapshot_id_put function in the mlx4 and
   netdevsim drivers.

 * netdevsim: support taking immediate snapshot via devlink

   Create a local devlink pointer instead of calling priv_to_devlink
   multiple times.

   Removed previous selftest for devlink region new without a snapshot id,
   as this is no longer supported. Adjusted and verified that the tests pass
   now.

 * ice: add a devlink region for dumping NVM contents

   Use "dev_err" instead of "dev_warn" for a message about failure to create
   the devlink region.

Jacob Keller (11):
  devlink: prepare to support region operations
  devlink: convert snapshot destructor callback to region op
  devlink: trivial: fix tab in function documentation
  devlink: add function to take snapshot while locked
  devlink: use -ENOSPC to indicate no more room for snapshots
  devlink: extract snapshot id allocation to helper function
  devlink: report error once U32_MAX snapshot ids have been used
  devlink: track snapshot id usage count using an xarray
  devlink: implement DEVLINK_CMD_REGION_NEW
  netdevsim: support taking immediate snapshot via devlink
  ice: add a devlink region for dumping NVM contents

 .../networking/devlink/devlink-region.rst     |   8 +
 Documentation/networking/devlink/ice.rst      |  26 ++
 drivers/net/ethernet/intel/ice/ice.h          |   2 +
 drivers/net/ethernet/intel/ice/ice_devlink.c  |  99 +++++
 drivers/net/ethernet/intel/ice/ice_devlink.h  |   3 +
 drivers/net/ethernet/intel/ice/ice_main.c     |   4 +
 drivers/net/ethernet/mellanox/mlx4/crdump.c   |  36 +-
 drivers/net/netdevsim/dev.c                   |  46 ++-
 include/net/devlink.h                         |  33 +-
 net/core/devlink.c                            | 363 +++++++++++++++---
 .../drivers/net/netdevsim/devlink.sh          |  10 +
 11 files changed, 550 insertions(+), 80 deletions(-)

-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [net-next v2 01/11] devlink: prepare to support region operations
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
@ 2020-03-26  3:51 ` Jacob Keller
  2020-03-26  3:51 ` [net-next v2 02/11] devlink: convert snapshot destructor callback to region op Jacob Keller
                   ` (14 subsequent siblings)
  15 siblings, 0 replies; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller, Jiri Pirko

Modify the devlink region code in preparation for adding new operations
on regions.

Create a devlink_region_ops structure, and move the name pointer from
within the devlink_region structure into the ops structure (similar to
the devlink_health_reporter_ops).

This prepares the regions to enable support of additional operations in
the future such as requesting snapshots, or accessing the region
directly without a snapshot.

In order to re-use the constant strings in the mlx4 driver their
declaration must be changed to 'const char * const' to ensure the
compiler realizes that both the data and the pointer cannot change.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
Reviewed-by: Jiri Pirko <jiri@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx4/crdump.c | 16 +++++++++++----
 drivers/net/netdevsim/dev.c                 |  6 +++++-
 include/net/devlink.h                       | 16 +++++++++++----
 net/core/devlink.c                          | 22 ++++++++++-----------
 4 files changed, 40 insertions(+), 20 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/crdump.c b/drivers/net/ethernet/mellanox/mlx4/crdump.c
index 64ed725aec28..cc2bf596c74b 100644
--- a/drivers/net/ethernet/mellanox/mlx4/crdump.c
+++ b/drivers/net/ethernet/mellanox/mlx4/crdump.c
@@ -38,8 +38,16 @@
 #define CR_ENABLE_BIT_OFFSET		0xF3F04
 #define MAX_NUM_OF_DUMPS_TO_STORE	(8)
 
-static const char *region_cr_space_str = "cr-space";
-static const char *region_fw_health_str = "fw-health";
+static const char * const region_cr_space_str = "cr-space";
+static const char * const region_fw_health_str = "fw-health";
+
+static const struct devlink_region_ops region_cr_space_ops = {
+	.name = region_cr_space_str,
+};
+
+static const struct devlink_region_ops region_fw_health_ops = {
+	.name = region_fw_health_str,
+};
 
 /* Set to true in case cr enable bit was set to true before crdump */
 static bool crdump_enbale_bit_set;
@@ -205,7 +213,7 @@ int mlx4_crdump_init(struct mlx4_dev *dev)
 	/* Create cr-space region */
 	crdump->region_crspace =
 		devlink_region_create(devlink,
-				      region_cr_space_str,
+				      &region_cr_space_ops,
 				      MAX_NUM_OF_DUMPS_TO_STORE,
 				      pci_resource_len(pdev, 0));
 	if (IS_ERR(crdump->region_crspace))
@@ -216,7 +224,7 @@ int mlx4_crdump_init(struct mlx4_dev *dev)
 	/* Create fw-health region */
 	crdump->region_fw_health =
 		devlink_region_create(devlink,
-				      region_fw_health_str,
+				      &region_fw_health_ops,
 				      MAX_NUM_OF_DUMPS_TO_STORE,
 				      HEALTH_BUFFER_SIZE);
 	if (IS_ERR(crdump->region_fw_health))
diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c
index 7bfd0622cef1..47a8f8c570c4 100644
--- a/drivers/net/netdevsim/dev.c
+++ b/drivers/net/netdevsim/dev.c
@@ -340,11 +340,15 @@ static void nsim_devlink_param_load_driverinit_values(struct devlink *devlink)
 
 #define NSIM_DEV_DUMMY_REGION_SNAPSHOT_MAX 16
 
+static const struct devlink_region_ops dummy_region_ops = {
+	.name = "dummy",
+};
+
 static int nsim_dev_dummy_region_init(struct nsim_dev *nsim_dev,
 				      struct devlink *devlink)
 {
 	nsim_dev->dummy_region =
-		devlink_region_create(devlink, "dummy",
+		devlink_region_create(devlink, &dummy_region_ops,
 				      NSIM_DEV_DUMMY_REGION_SNAPSHOT_MAX,
 				      NSIM_DEV_DUMMY_REGION_SIZE);
 	return PTR_ERR_OR_ZERO(nsim_dev->dummy_region);
diff --git a/include/net/devlink.h b/include/net/devlink.h
index 37230e23b5b0..85db5dd5184d 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -498,6 +498,14 @@ struct devlink_info_req;
 
 typedef void devlink_snapshot_data_dest_t(const void *data);
 
+/**
+ * struct devlink_region_ops - Region operations
+ * @name: region name
+ */
+struct devlink_region_ops {
+	const char *name;
+};
+
 struct devlink_fmsg;
 struct devlink_health_reporter;
 
@@ -963,10 +971,10 @@ void devlink_port_param_value_changed(struct devlink_port *devlink_port,
 				      u32 param_id);
 void devlink_param_value_str_fill(union devlink_param_value *dst_val,
 				  const char *src);
-struct devlink_region *devlink_region_create(struct devlink *devlink,
-					     const char *region_name,
-					     u32 region_max_snapshots,
-					     u64 region_size);
+struct devlink_region *
+devlink_region_create(struct devlink *devlink,
+		      const struct devlink_region_ops *ops,
+		      u32 region_max_snapshots, u64 region_size);
 void devlink_region_destroy(struct devlink_region *region);
 u32 devlink_region_snapshot_id_get(struct devlink *devlink);
 int devlink_region_snapshot_create(struct devlink_region *region,
diff --git a/net/core/devlink.c b/net/core/devlink.c
index 73bb8fbe3393..ca5362530567 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -344,7 +344,7 @@ devlink_sb_tc_index_get_from_info(struct devlink_sb *devlink_sb,
 struct devlink_region {
 	struct devlink *devlink;
 	struct list_head list;
-	const char *name;
+	const struct devlink_region_ops *ops;
 	struct list_head snapshot_list;
 	u32 max_snapshots;
 	u32 cur_snapshots;
@@ -365,7 +365,7 @@ devlink_region_get_by_name(struct devlink *devlink, const char *region_name)
 	struct devlink_region *region;
 
 	list_for_each_entry(region, &devlink->region_list, list)
-		if (!strcmp(region->name, region_name))
+		if (!strcmp(region->ops->name, region_name))
 			return region;
 
 	return NULL;
@@ -3695,7 +3695,7 @@ static int devlink_nl_region_fill(struct sk_buff *msg, struct devlink *devlink,
 	if (err)
 		goto nla_put_failure;
 
-	err = nla_put_string(msg, DEVLINK_ATTR_REGION_NAME, region->name);
+	err = nla_put_string(msg, DEVLINK_ATTR_REGION_NAME, region->ops->name);
 	if (err)
 		goto nla_put_failure;
 
@@ -3741,7 +3741,7 @@ static void devlink_nl_region_notify(struct devlink_region *region,
 		goto out_cancel_msg;
 
 	err = nla_put_string(msg, DEVLINK_ATTR_REGION_NAME,
-			     region->name);
+			     region->ops->name);
 	if (err)
 		goto out_cancel_msg;
 
@@ -7647,21 +7647,21 @@ EXPORT_SYMBOL_GPL(devlink_param_value_str_fill);
  *	devlink_region_create - create a new address region
  *
  *	@devlink: devlink
- *	@region_name: region name
+ *	@ops: region operations and name
  *	@region_max_snapshots: Maximum supported number of snapshots for region
  *	@region_size: size of region
  */
-struct devlink_region *devlink_region_create(struct devlink *devlink,
-					     const char *region_name,
-					     u32 region_max_snapshots,
-					     u64 region_size)
+struct devlink_region *
+devlink_region_create(struct devlink *devlink,
+		      const struct devlink_region_ops *ops,
+		      u32 region_max_snapshots, u64 region_size)
 {
 	struct devlink_region *region;
 	int err = 0;
 
 	mutex_lock(&devlink->lock);
 
-	if (devlink_region_get_by_name(devlink, region_name)) {
+	if (devlink_region_get_by_name(devlink, ops->name)) {
 		err = -EEXIST;
 		goto unlock;
 	}
@@ -7674,7 +7674,7 @@ struct devlink_region *devlink_region_create(struct devlink *devlink,
 
 	region->devlink = devlink;
 	region->max_snapshots = region_max_snapshots;
-	region->name = region_name;
+	region->ops = ops;
 	region->size = region_size;
 	INIT_LIST_HEAD(&region->snapshot_list);
 	list_add_tail(&region->list, &devlink->region_list);
-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [net-next v2 02/11] devlink: convert snapshot destructor callback to region op
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
  2020-03-26  3:51 ` [net-next v2 01/11] devlink: prepare to support region operations Jacob Keller
@ 2020-03-26  3:51 ` Jacob Keller
  2020-03-26  3:51 ` [net-next v2 03/11] devlink: trivial: fix tab in function documentation Jacob Keller
                   ` (13 subsequent siblings)
  15 siblings, 0 replies; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller, Jiri Pirko

It does not makes sense that two snapshots for a given region would use
different destructors. Simplify snapshot creation by adding
a .destructor op for regions.

This operation will replace the data_destructor for the snapshot
creation, and makes snapshot creation easier.

Noticed-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Jiri Pirko <jiri@mellanox.com>
---
 drivers/net/ethernet/mellanox/mlx4/crdump.c |  6 ++++--
 drivers/net/netdevsim/dev.c                 |  3 ++-
 include/net/devlink.h                       |  7 +++----
 net/core/devlink.c                          | 11 +++++------
 4 files changed, 14 insertions(+), 13 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/crdump.c b/drivers/net/ethernet/mellanox/mlx4/crdump.c
index cc2bf596c74b..c3f90c0f9554 100644
--- a/drivers/net/ethernet/mellanox/mlx4/crdump.c
+++ b/drivers/net/ethernet/mellanox/mlx4/crdump.c
@@ -43,10 +43,12 @@ static const char * const region_fw_health_str = "fw-health";
 
 static const struct devlink_region_ops region_cr_space_ops = {
 	.name = region_cr_space_str,
+	.destructor = &kvfree,
 };
 
 static const struct devlink_region_ops region_fw_health_ops = {
 	.name = region_fw_health_str,
+	.destructor = &kvfree,
 };
 
 /* Set to true in case cr enable bit was set to true before crdump */
@@ -107,7 +109,7 @@ static void mlx4_crdump_collect_crspace(struct mlx4_dev *dev,
 					readl(cr_space + offset);
 
 		err = devlink_region_snapshot_create(crdump->region_crspace,
-						     crspace_data, id, &kvfree);
+						     crspace_data, id);
 		if (err) {
 			kvfree(crspace_data);
 			mlx4_warn(dev, "crdump: devlink create %s snapshot id %d err %d\n",
@@ -146,7 +148,7 @@ static void mlx4_crdump_collect_fw_health(struct mlx4_dev *dev,
 					readl(health_buf_start + offset);
 
 		err = devlink_region_snapshot_create(crdump->region_fw_health,
-						     health_data, id, &kvfree);
+						     health_data, id);
 		if (err) {
 			kvfree(health_data);
 			mlx4_warn(dev, "crdump: devlink create %s snapshot id %d err %d\n",
diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c
index 47a8f8c570c4..f7621ccb7b88 100644
--- a/drivers/net/netdevsim/dev.c
+++ b/drivers/net/netdevsim/dev.c
@@ -56,7 +56,7 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file,
 
 	id = devlink_region_snapshot_id_get(priv_to_devlink(nsim_dev));
 	err = devlink_region_snapshot_create(nsim_dev->dummy_region,
-					     dummy_data, id, kfree);
+					     dummy_data, id);
 	if (err) {
 		pr_err("Failed to create region snapshot\n");
 		kfree(dummy_data);
@@ -342,6 +342,7 @@ static void nsim_devlink_param_load_driverinit_values(struct devlink *devlink)
 
 static const struct devlink_region_ops dummy_region_ops = {
 	.name = "dummy",
+	.destructor = &kfree,
 };
 
 static int nsim_dev_dummy_region_init(struct nsim_dev *nsim_dev,
diff --git a/include/net/devlink.h b/include/net/devlink.h
index 85db5dd5184d..8869ad75b965 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -496,14 +496,14 @@ enum devlink_param_generic_id {
 struct devlink_region;
 struct devlink_info_req;
 
-typedef void devlink_snapshot_data_dest_t(const void *data);
-
 /**
  * struct devlink_region_ops - Region operations
  * @name: region name
+ * @destructor: callback used to free snapshot memory when deleting
  */
 struct devlink_region_ops {
 	const char *name;
+	void (*destructor)(const void *data);
 };
 
 struct devlink_fmsg;
@@ -978,8 +978,7 @@ devlink_region_create(struct devlink *devlink,
 void devlink_region_destroy(struct devlink_region *region);
 u32 devlink_region_snapshot_id_get(struct devlink *devlink);
 int devlink_region_snapshot_create(struct devlink_region *region,
-				   u8 *data, u32 snapshot_id,
-				   devlink_snapshot_data_dest_t *data_destructor);
+				   u8 *data, u32 snapshot_id);
 int devlink_info_serial_number_put(struct devlink_info_req *req,
 				   const char *sn);
 int devlink_info_driver_name_put(struct devlink_info_req *req,
diff --git a/net/core/devlink.c b/net/core/devlink.c
index ca5362530567..84d74fbcff62 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -354,7 +354,6 @@ struct devlink_region {
 struct devlink_snapshot {
 	struct list_head list;
 	struct devlink_region *region;
-	devlink_snapshot_data_dest_t *data_destructor;
 	u8 *data;
 	u32 id;
 };
@@ -3775,7 +3774,7 @@ static void devlink_region_snapshot_del(struct devlink_region *region,
 	devlink_nl_region_notify(region, snapshot, DEVLINK_CMD_REGION_DEL);
 	region->cur_snapshots--;
 	list_del(&snapshot->list);
-	(*snapshot->data_destructor)(snapshot->data);
+	region->ops->destructor(snapshot->data);
 	kfree(snapshot);
 }
 
@@ -7659,6 +7658,9 @@ devlink_region_create(struct devlink *devlink,
 	struct devlink_region *region;
 	int err = 0;
 
+	if (WARN_ON(!ops) || WARN_ON(!ops->destructor))
+		return ERR_PTR(-EINVAL);
+
 	mutex_lock(&devlink->lock);
 
 	if (devlink_region_get_by_name(devlink, ops->name)) {
@@ -7745,11 +7747,9 @@ EXPORT_SYMBOL_GPL(devlink_region_snapshot_id_get);
  *	@region: devlink region of the snapshot
  *	@data: snapshot data
  *	@snapshot_id: snapshot id to be created
- *	@data_destructor: pointer to destructor function to free data
  */
 int devlink_region_snapshot_create(struct devlink_region *region,
-				   u8 *data, u32 snapshot_id,
-				   devlink_snapshot_data_dest_t *data_destructor)
+				   u8 *data, u32 snapshot_id)
 {
 	struct devlink *devlink = region->devlink;
 	struct devlink_snapshot *snapshot;
@@ -7777,7 +7777,6 @@ int devlink_region_snapshot_create(struct devlink_region *region,
 	snapshot->id = snapshot_id;
 	snapshot->region = region;
 	snapshot->data = data;
-	snapshot->data_destructor = data_destructor;
 
 	list_add_tail(&snapshot->list, &region->snapshot_list);
 
-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [net-next v2 03/11] devlink: trivial: fix tab in function documentation
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
  2020-03-26  3:51 ` [net-next v2 01/11] devlink: prepare to support region operations Jacob Keller
  2020-03-26  3:51 ` [net-next v2 02/11] devlink: convert snapshot destructor callback to region op Jacob Keller
@ 2020-03-26  3:51 ` Jacob Keller
  2020-03-26  3:51 ` [net-next v2 04/11] devlink: add function to take snapshot while locked Jacob Keller
                   ` (12 subsequent siblings)
  15 siblings, 0 replies; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller, Jiri Pirko

The function documentation comment for devlink_region_snapshot_create
included a literal tab character between 'future analyses' that was
difficult to spot as it happened to only display as one space wide.

Fix the comment to use a space here instead of a stray tab appearing in
the middle of a sentence.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Jiri Pirko <jiri@mellanox.com>
---
 net/core/devlink.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/core/devlink.c b/net/core/devlink.c
index 84d74fbcff62..73e66a779c13 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -7740,7 +7740,7 @@ EXPORT_SYMBOL_GPL(devlink_region_snapshot_id_get);
  *	devlink_region_snapshot_create - create a new snapshot
  *	This will add a new snapshot of a region. The snapshot
  *	will be stored on the region struct and can be accessed
- *	from devlink. This is useful for future	analyses of snapshots.
+ *	from devlink. This is useful for future analyses of snapshots.
  *	Multiple snapshots can be created on a region.
  *	The @snapshot_id should be obtained using the getter function.
  *
-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [net-next v2 04/11] devlink: add function to take snapshot while locked
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (2 preceding siblings ...)
  2020-03-26  3:51 ` [net-next v2 03/11] devlink: trivial: fix tab in function documentation Jacob Keller
@ 2020-03-26  3:51 ` Jacob Keller
  2020-03-26  3:51 ` [net-next v2 05/11] devlink: use -ENOSPC to indicate no more room for snapshots Jacob Keller
                   ` (11 subsequent siblings)
  15 siblings, 0 replies; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller, Jiri Pirko

A future change is going to add a new devlink command to request
a snapshot on demand. This function will want to call the
devlink_region_snapshot_create function while already holding the
devlink instance lock.

Extract the logic of this function into a static function prefixed by
`__` to indicate that it is an internal helper function. Modify the
original function to be implemented in terms of the new locked
function.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
Reviewed-by: Jiri Pirko <jiri@mellanox.com>
Reviewed-by: Jakub Kicinski <kuba@kernel.org>
---
 net/core/devlink.c | 78 ++++++++++++++++++++++++++++------------------
 1 file changed, 47 insertions(+), 31 deletions(-)

diff --git a/net/core/devlink.c b/net/core/devlink.c
index 73e66a779c13..620e9d07ac85 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -3768,6 +3768,52 @@ static void devlink_nl_region_notify(struct devlink_region *region,
 	nlmsg_free(msg);
 }
 
+/**
+ *	__devlink_region_snapshot_create - create a new snapshot
+ *	This will add a new snapshot of a region. The snapshot
+ *	will be stored on the region struct and can be accessed
+ *	from devlink. This is useful for future analyses of snapshots.
+ *	Multiple snapshots can be created on a region.
+ *	The @snapshot_id should be obtained using the getter function.
+ *
+ *	Must be called only while holding the devlink instance lock.
+ *
+ *	@region: devlink region of the snapshot
+ *	@data: snapshot data
+ *	@snapshot_id: snapshot id to be created
+ */
+static int
+__devlink_region_snapshot_create(struct devlink_region *region,
+				 u8 *data, u32 snapshot_id)
+{
+	struct devlink *devlink = region->devlink;
+	struct devlink_snapshot *snapshot;
+
+	lockdep_assert_held(&devlink->lock);
+
+	/* check if region can hold one more snapshot */
+	if (region->cur_snapshots == region->max_snapshots)
+		return -ENOMEM;
+
+	if (devlink_region_snapshot_get_by_id(region, snapshot_id))
+		return -EEXIST;
+
+	snapshot = kzalloc(sizeof(*snapshot), GFP_KERNEL);
+	if (!snapshot)
+		return -ENOMEM;
+
+	snapshot->id = snapshot_id;
+	snapshot->region = region;
+	snapshot->data = data;
+
+	list_add_tail(&snapshot->list, &region->snapshot_list);
+
+	region->cur_snapshots++;
+
+	devlink_nl_region_notify(region, snapshot, DEVLINK_CMD_REGION_NEW);
+	return 0;
+}
+
 static void devlink_region_snapshot_del(struct devlink_region *region,
 					struct devlink_snapshot *snapshot)
 {
@@ -7752,42 +7798,12 @@ int devlink_region_snapshot_create(struct devlink_region *region,
 				   u8 *data, u32 snapshot_id)
 {
 	struct devlink *devlink = region->devlink;
-	struct devlink_snapshot *snapshot;
 	int err;
 
 	mutex_lock(&devlink->lock);
-
-	/* check if region can hold one more snapshot */
-	if (region->cur_snapshots == region->max_snapshots) {
-		err = -ENOMEM;
-		goto unlock;
-	}
-
-	if (devlink_region_snapshot_get_by_id(region, snapshot_id)) {
-		err = -EEXIST;
-		goto unlock;
-	}
-
-	snapshot = kzalloc(sizeof(*snapshot), GFP_KERNEL);
-	if (!snapshot) {
-		err = -ENOMEM;
-		goto unlock;
-	}
-
-	snapshot->id = snapshot_id;
-	snapshot->region = region;
-	snapshot->data = data;
-
-	list_add_tail(&snapshot->list, &region->snapshot_list);
-
-	region->cur_snapshots++;
-
-	devlink_nl_region_notify(region, snapshot, DEVLINK_CMD_REGION_NEW);
+	err = __devlink_region_snapshot_create(region, data, snapshot_id);
 	mutex_unlock(&devlink->lock);
-	return 0;
 
-unlock:
-	mutex_unlock(&devlink->lock);
 	return err;
 }
 EXPORT_SYMBOL_GPL(devlink_region_snapshot_create);
-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [net-next v2 05/11] devlink: use -ENOSPC to indicate no more room for snapshots
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (3 preceding siblings ...)
  2020-03-26  3:51 ` [net-next v2 04/11] devlink: add function to take snapshot while locked Jacob Keller
@ 2020-03-26  3:51 ` Jacob Keller
  2020-03-26  7:24   ` Jiri Pirko
  2020-03-26  3:51 ` [net-next v2 06/11] devlink: extract snapshot id allocation to helper function Jacob Keller
                   ` (10 subsequent siblings)
  15 siblings, 1 reply; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller

The devlink_region_snapshot_create function returns -ENOMEM when the
maximum number of snapshots has been reached. This is confusing because
it is not an issue of being out of memory. Change this to use -ENOSPC instead.

Reported-by: Jiri Pirko <jiri@resnulli.us>
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 net/core/devlink.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/core/devlink.c b/net/core/devlink.c
index 620e9d07ac85..8c2c4bc009bb 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -3793,7 +3793,7 @@ __devlink_region_snapshot_create(struct devlink_region *region,
 
 	/* check if region can hold one more snapshot */
 	if (region->cur_snapshots == region->max_snapshots)
-		return -ENOMEM;
+		return -ENOSPC;
 
 	if (devlink_region_snapshot_get_by_id(region, snapshot_id))
 		return -EEXIST;
-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [net-next v2 06/11] devlink: extract snapshot id allocation to helper function
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (4 preceding siblings ...)
  2020-03-26  3:51 ` [net-next v2 05/11] devlink: use -ENOSPC to indicate no more room for snapshots Jacob Keller
@ 2020-03-26  3:51 ` Jacob Keller
  2020-03-26  7:25   ` Jiri Pirko
  2020-03-26  3:51 ` [net-next v2 07/11] devlink: report error once U32_MAX snapshot ids have been used Jacob Keller
                   ` (9 subsequent siblings)
  15 siblings, 1 reply; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller

A future change is going to implement a new devlink command to request
a snapshot on demand. As part of this, the logic for handling the
snapshot ids will be refactored. To simplify the snapshot id allocation
function, move it to a separate function prefixed by `__`. This helper
function will assume the lock is held.

While no other callers will exist, it simplifies refactoring the logic
because there is no need to complicate the function with gotos to handle
unlocking on failure.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 net/core/devlink.c | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/net/core/devlink.c b/net/core/devlink.c
index 8c2c4bc009bb..457b8303ae59 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -3768,6 +3768,19 @@ static void devlink_nl_region_notify(struct devlink_region *region,
 	nlmsg_free(msg);
 }
 
+/**
+ *	__devlink_region_snapshot_id_get - get snapshot ID
+ *	@devlink: devlink instance
+ *
+ *	Returns a new snapshot id. Must be called while holding the
+ *	devlink instance lock.
+ */
+static u32 __devlink_region_snapshot_id_get(struct devlink *devlink)
+{
+	lockdep_assert_held(&devlink->lock);
+	return ++devlink->snapshot_id;
+}
+
 /**
  *	__devlink_region_snapshot_create - create a new snapshot
  *	This will add a new snapshot of a region. The snapshot
@@ -7775,7 +7788,7 @@ u32 devlink_region_snapshot_id_get(struct devlink *devlink)
 	u32 id;
 
 	mutex_lock(&devlink->lock);
-	id = ++devlink->snapshot_id;
+	id = __devlink_region_snapshot_id_get(devlink);
 	mutex_unlock(&devlink->lock);
 
 	return id;
-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [net-next v2 07/11] devlink: report error once U32_MAX snapshot ids have been used
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (5 preceding siblings ...)
  2020-03-26  3:51 ` [net-next v2 06/11] devlink: extract snapshot id allocation to helper function Jacob Keller
@ 2020-03-26  3:51 ` Jacob Keller
  2020-03-26  7:30   ` Jiri Pirko
  2020-03-26  3:51 ` [net-next v2 08/11] devlink: track snapshot id usage count using an xarray Jacob Keller
                   ` (8 subsequent siblings)
  15 siblings, 1 reply; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller

The devlink_snapshot_id_get() function returns a snapshot id. The
snapshot id is a u32, so there is no way to indicate an error code.

Indeed, the two current callers of devlink_snapshot_id_get() assume that
a negative value is an error.

A future change is going to possibly add additional cases where this
function could fail. Refactor the function to return the snapshot id in
an argument, so that it can return zero or an error value.

This ensures that snapshot ids cannot be confused with error values, and
aids in the future refactor of snapshot id allocation management.

Because there is no current way to release previously used snapshot ids,
add a simple check ensuring that an error is reported in case the
snapshot_id would over flow.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 drivers/net/ethernet/mellanox/mlx4/crdump.c | 11 ++++++---
 drivers/net/netdevsim/dev.c                 |  6 ++++-
 include/net/devlink.h                       |  2 +-
 net/core/devlink.c                          | 27 +++++++++++++++------
 4 files changed, 33 insertions(+), 13 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/crdump.c b/drivers/net/ethernet/mellanox/mlx4/crdump.c
index c3f90c0f9554..792951f6df0d 100644
--- a/drivers/net/ethernet/mellanox/mlx4/crdump.c
+++ b/drivers/net/ethernet/mellanox/mlx4/crdump.c
@@ -169,6 +169,7 @@ int mlx4_crdump_collect(struct mlx4_dev *dev)
 	struct pci_dev *pdev = dev->persist->pdev;
 	unsigned long cr_res_size;
 	u8 __iomem *cr_space;
+	int err;
 	u32 id;
 
 	if (!dev->caps.health_buffer_addrs) {
@@ -189,10 +190,14 @@ int mlx4_crdump_collect(struct mlx4_dev *dev)
 		return -ENODEV;
 	}
 
-	crdump_enable_crspace_access(dev, cr_space);
-
 	/* Get the available snapshot ID for the dumps */
-	id = devlink_region_snapshot_id_get(devlink);
+	err = devlink_region_snapshot_id_get(devlink, &id);
+	if (err) {
+		mlx4_err(dev, "crdump: devlink get snapshot id err %d\n", err);
+		return err;
+	}
+
+	crdump_enable_crspace_access(dev, cr_space);
 
 	/* Try to capture dumps */
 	mlx4_crdump_collect_crspace(dev, cr_space, id);
diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c
index f7621ccb7b88..b851fe63a75d 100644
--- a/drivers/net/netdevsim/dev.c
+++ b/drivers/net/netdevsim/dev.c
@@ -54,7 +54,11 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file,
 
 	get_random_bytes(dummy_data, NSIM_DEV_DUMMY_REGION_SIZE);
 
-	id = devlink_region_snapshot_id_get(priv_to_devlink(nsim_dev));
+	err = devlink_region_snapshot_id_get(priv_to_devlink(nsim_dev), &id);
+	if (err)
+		pr_err("Failed to get snapshot id\n");
+		return err;
+	}
 	err = devlink_region_snapshot_create(nsim_dev->dummy_region,
 					     dummy_data, id);
 	if (err) {
diff --git a/include/net/devlink.h b/include/net/devlink.h
index 8869ad75b965..9a46bc7fed90 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -976,7 +976,7 @@ devlink_region_create(struct devlink *devlink,
 		      const struct devlink_region_ops *ops,
 		      u32 region_max_snapshots, u64 region_size);
 void devlink_region_destroy(struct devlink_region *region);
-u32 devlink_region_snapshot_id_get(struct devlink *devlink);
+int devlink_region_snapshot_id_get(struct devlink *devlink, u32 *id);
 int devlink_region_snapshot_create(struct devlink_region *region,
 				   u8 *data, u32 snapshot_id);
 int devlink_info_serial_number_put(struct devlink_info_req *req,
diff --git a/net/core/devlink.c b/net/core/devlink.c
index 457b8303ae59..77341c65868f 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -3771,14 +3771,22 @@ static void devlink_nl_region_notify(struct devlink_region *region,
 /**
  *	__devlink_region_snapshot_id_get - get snapshot ID
  *	@devlink: devlink instance
+ *	@id: storage to return snapshot id
  *
- *	Returns a new snapshot id. Must be called while holding the
- *	devlink instance lock.
+ *	Allocates a new snapshot id. Returns zero on success, or a negative
+ *	error on failure. Must be called while holding the devlink instance
+ *	lock.
  */
-static u32 __devlink_region_snapshot_id_get(struct devlink *devlink)
+static int __devlink_region_snapshot_id_get(struct devlink *devlink, u32 *id)
 {
 	lockdep_assert_held(&devlink->lock);
-	return ++devlink->snapshot_id;
+
+	if (devlink->snapshot_id >= U32_MAX)
+		return -ENOSPC;
+
+	*id = ++devlink->snapshot_id;
+
+	return 0;
 }
 
 /**
@@ -7781,17 +7789,20 @@ EXPORT_SYMBOL_GPL(devlink_region_destroy);
  *	Driver should use the same id for multiple snapshots taken
  *	on multiple regions at the same time/by the same trigger.
  *
+ *	Returns zero on success, or a negative error code on failure.
+ *
  *	@devlink: devlink
+ *	@id: storage to return id
  */
-u32 devlink_region_snapshot_id_get(struct devlink *devlink)
+int devlink_region_snapshot_id_get(struct devlink *devlink, u32 *id)
 {
-	u32 id;
+	int err;
 
 	mutex_lock(&devlink->lock);
-	id = __devlink_region_snapshot_id_get(devlink);
+	err = __devlink_region_snapshot_id_get(devlink, id);
 	mutex_unlock(&devlink->lock);
 
-	return id;
+	return err;
 }
 EXPORT_SYMBOL_GPL(devlink_region_snapshot_id_get);
 
-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [net-next v2 08/11] devlink: track snapshot id usage count using an xarray
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (6 preceding siblings ...)
  2020-03-26  3:51 ` [net-next v2 07/11] devlink: report error once U32_MAX snapshot ids have been used Jacob Keller
@ 2020-03-26  3:51 ` Jacob Keller
  2020-03-26  8:35   ` Jiri Pirko
  2020-03-26  3:51 ` [net-next v2 09/11] devlink: implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (7 subsequent siblings)
  15 siblings, 1 reply; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller

Each snapshot created for a devlink region must have an id. These ids
are supposed to be unique per "event" that caused the snapshot to be
created. Drivers call devlink_region_snapshot_id_get to obtain a new id
to use for a new event trigger. The id values are tracked per devlink,
so that the same id number can be used if a triggering event creates
multiple snapshots on different regions.

There is no mechanism for snapshot ids to ever be reused. Introduce an
xarray to store the count of how many snapshots are using a given id,
replacing the snapshot_id field previously used for picking the next id.

The devlink_region_snapshot_id_get() function will use xa_alloc to
insert an initial value of 1 value at an available slot between 0 and
U32_MAX.

The new __devlink_snapshot_id_increment() and
__devlink_snapshot_id_decrement() functions will be used to track how
many snapshots currently use an id.

Drivers must now call devlink_snapshot_id_put() in order to release
their reference of the snapshot id after adding region snapshots.

By tracking the total number of snapshots using a given id, it is
possible for the decrement() function to erase the id from the xarray
when it is not in use.

With this method, a snapshot id can become reused again once all
snapshots that referred to it have been deleted via
DEVLINK_CMD_REGION_DEL, and the driver has finished adding snapshots.

This work also paves the way to introduce a mechanism for userspace to
request a snapshot.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 drivers/net/ethernet/mellanox/mlx4/crdump.c |   3 +
 drivers/net/netdevsim/dev.c                 |   5 +-
 include/net/devlink.h                       |   4 +-
 net/core/devlink.c                          | 130 +++++++++++++++++++-
 4 files changed, 134 insertions(+), 8 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/crdump.c b/drivers/net/ethernet/mellanox/mlx4/crdump.c
index 792951f6df0d..2700628f1689 100644
--- a/drivers/net/ethernet/mellanox/mlx4/crdump.c
+++ b/drivers/net/ethernet/mellanox/mlx4/crdump.c
@@ -203,6 +203,9 @@ int mlx4_crdump_collect(struct mlx4_dev *dev)
 	mlx4_crdump_collect_crspace(dev, cr_space, id);
 	mlx4_crdump_collect_fw_health(dev, cr_space, id);
 
+	/* Release reference on the snapshot id */
+	devlink_region_snapshot_id_put(devlink, id);
+
 	crdump_disable_crspace_access(dev, cr_space);
 
 	iounmap(cr_space);
diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c
index b851fe63a75d..53ec891659eb 100644
--- a/drivers/net/netdevsim/dev.c
+++ b/drivers/net/netdevsim/dev.c
@@ -44,6 +44,7 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file,
 					    size_t count, loff_t *ppos)
 {
 	struct nsim_dev *nsim_dev = file->private_data;
+	struct devlink *devlink = priv_to_devlink(nsim_dev);
 	void *dummy_data;
 	int err;
 	u32 id;
@@ -54,13 +55,15 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file,
 
 	get_random_bytes(dummy_data, NSIM_DEV_DUMMY_REGION_SIZE);
 
-	err = devlink_region_snapshot_id_get(priv_to_devlink(nsim_dev), &id);
+	err = devlink_region_snapshot_id_get(devlink, &id);
 	if (err)
 		pr_err("Failed to get snapshot id\n");
 		return err;
 	}
 	err = devlink_region_snapshot_create(nsim_dev->dummy_region,
 					     dummy_data, id);
+	/* release the snapshot id regardless of errors */
+	devlink_region_snapshot_id_put(devlink, id);
 	if (err) {
 		pr_err("Failed to create region snapshot\n");
 		kfree(dummy_data);
diff --git a/include/net/devlink.h b/include/net/devlink.h
index 9a46bc7fed90..fb9154060e6e 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -18,6 +18,7 @@
 #include <net/net_namespace.h>
 #include <net/flow_offload.h>
 #include <uapi/linux/devlink.h>
+#include <linux/xarray.h>
 
 struct devlink_ops;
 
@@ -29,13 +30,13 @@ struct devlink {
 	struct list_head resource_list;
 	struct list_head param_list;
 	struct list_head region_list;
-	u32 snapshot_id;
 	struct list_head reporter_list;
 	struct mutex reporters_lock; /* protects reporter_list */
 	struct devlink_dpipe_headers *dpipe_headers;
 	struct list_head trap_list;
 	struct list_head trap_group_list;
 	const struct devlink_ops *ops;
+	struct xarray snapshot_ids;
 	struct device *dev;
 	possible_net_t _net;
 	struct mutex lock;
@@ -977,6 +978,7 @@ devlink_region_create(struct devlink *devlink,
 		      u32 region_max_snapshots, u64 region_size);
 void devlink_region_destroy(struct devlink_region *region);
 int devlink_region_snapshot_id_get(struct devlink *devlink, u32 *id);
+void devlink_region_snapshot_id_put(struct devlink *devlink, u32 id);
 int devlink_region_snapshot_create(struct devlink_region *region,
 				   u8 *data, u32 snapshot_id);
 int devlink_info_serial_number_put(struct devlink_info_req *req,
diff --git a/net/core/devlink.c b/net/core/devlink.c
index 77341c65868f..d89eaac0a101 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -3768,6 +3768,83 @@ static void devlink_nl_region_notify(struct devlink_region *region,
 	nlmsg_free(msg);
 }
 
+/**
+ * __devlink_snapshot_id_increment - Increment number of snapshots using an id
+ *	@devlink: devlink instance
+ *	@id: the snapshot id
+ *
+ *	Track when a new snapshot begins using an id. Load the count for the
+ *	given id from the snapshot xarray, increment it, and store it back.
+ *
+ *	Called when a new snapshot is created with the given id.
+ *
+ *	The id *must* have been previously allocated by
+ *	devlink_region_snapshot_id_get().
+ *
+ *	Returns 0 on success, or an error on failure.
+ */
+static int __devlink_snapshot_id_increment(struct devlink *devlink, u32 id)
+{
+	unsigned long count;
+	void *p;
+
+	lockdep_assert_held(&devlink->lock);
+
+	p = xa_load(&devlink->snapshot_ids, id);
+	if (WARN_ON(!p))
+		return -EINVAL;
+
+	if (WARN_ON(!xa_is_value(p)))
+		return -EINVAL;
+
+	count = xa_to_value(p);
+	count++;
+
+	return xa_err(xa_store(&devlink->snapshot_ids, id, xa_mk_value(count),
+			       GFP_KERNEL));
+}
+
+/**
+ * __devlink_snapshot_id_decrement - Decrease number of snapshots using an id
+ *	@devlink: devlink instance
+ *	@id: the snapshot id
+ *
+ *	Track when a snapshot is deleted and stops using an id. Load the count
+ *	for the given id from the snapshot xarray, decrement it, and store it
+ *	back.
+ *
+ *	If the count reaches zero, erase this id from the xarray, freeing it
+ *	up for future re-use by devlink_region_snapshot_id_get().
+ *
+ *	Called when a snapshot using the given id is deleted, and when the
+ *	initial allocator of the id is finished using it.
+ */
+static void __devlink_snapshot_id_decrement(struct devlink *devlink, u32 id)
+{
+	unsigned long count;
+	void *p;
+
+	lockdep_assert_held(&devlink->lock);
+
+	p = xa_load(&devlink->snapshot_ids, id);
+	if (WARN_ON(!p))
+		return;
+
+	if (WARN_ON(!xa_is_value(p)))
+		return;
+
+	count = xa_to_value(p);
+
+	if (count > 1) {
+		count--;
+		xa_store(&devlink->snapshot_ids, id, xa_mk_value(count),
+			 GFP_KERNEL);
+	} else {
+		/* If this was the last user, we can erase this id */
+		xa_erase(&devlink->snapshot_ids, id);
+	}
+}
+
 /**
  *	__devlink_region_snapshot_id_get - get snapshot ID
  *	@devlink: devlink instance
@@ -3776,17 +3853,20 @@ static void devlink_nl_region_notify(struct devlink_region *region,
  *	Allocates a new snapshot id. Returns zero on success, or a negative
  *	error on failure. Must be called while holding the devlink instance
  *	lock.
+ *
+ *	Snapshot IDs are tracked using an xarray which stores the number of
+ *	users of the snapshot id.
+ *
+ *	Note that the caller of this function counts as a 'user', in order to
+ *	avoid race conditions. The caller must release its hold on the
+ *	snapshot by using devlink_region_snapshot_id_put.
  */
 static int __devlink_region_snapshot_id_get(struct devlink *devlink, u32 *id)
 {
 	lockdep_assert_held(&devlink->lock);
 
-	if (devlink->snapshot_id >= U32_MAX)
-		return -ENOSPC;
-
-	*id = ++devlink->snapshot_id;
-
-	return 0;
+	return xa_alloc(&devlink->snapshot_ids, id, xa_mk_value(1),
+			xa_limit_32b, GFP_KERNEL);
 }
 
 /**
@@ -3809,6 +3889,7 @@ __devlink_region_snapshot_create(struct devlink_region *region,
 {
 	struct devlink *devlink = region->devlink;
 	struct devlink_snapshot *snapshot;
+	int err;
 
 	lockdep_assert_held(&devlink->lock);
 
@@ -3823,6 +3904,10 @@ __devlink_region_snapshot_create(struct devlink_region *region,
 	if (!snapshot)
 		return -ENOMEM;
 
+	err = __devlink_snapshot_id_increment(devlink, snapshot_id);
+	if (err)
+		goto id_increment_failure;
+
 	snapshot->id = snapshot_id;
 	snapshot->region = region;
 	snapshot->data = data;
@@ -3833,15 +3918,24 @@ __devlink_region_snapshot_create(struct devlink_region *region,
 
 	devlink_nl_region_notify(region, snapshot, DEVLINK_CMD_REGION_NEW);
 	return 0;
+
+id_increment_failure:
+	kfree(snapshot);
+	return err;
 }
 
 static void devlink_region_snapshot_del(struct devlink_region *region,
 					struct devlink_snapshot *snapshot)
 {
+	struct devlink *devlink = region->devlink;
+
+	lockdep_assert_held(&devlink->lock);
+
 	devlink_nl_region_notify(region, snapshot, DEVLINK_CMD_REGION_DEL);
 	region->cur_snapshots--;
 	list_del(&snapshot->list);
 	region->ops->destructor(snapshot->data);
+	__devlink_snapshot_id_decrement(devlink, snapshot->id);
 	kfree(snapshot);
 }
 
@@ -6494,6 +6588,7 @@ struct devlink *devlink_alloc(const struct devlink_ops *ops, size_t priv_size)
 	if (!devlink)
 		return NULL;
 	devlink->ops = ops;
+	xa_init_flags(&devlink->snapshot_ids, XA_FLAGS_ALLOC);
 	__devlink_net_set(devlink, &init_net);
 	INIT_LIST_HEAD(&devlink->port_list);
 	INIT_LIST_HEAD(&devlink->sb_list);
@@ -6598,6 +6693,8 @@ void devlink_free(struct devlink *devlink)
 	WARN_ON(!list_empty(&devlink->sb_list));
 	WARN_ON(!list_empty(&devlink->port_list));
 
+	xa_destroy(&devlink->snapshot_ids);
+
 	kfree(devlink);
 }
 EXPORT_SYMBOL_GPL(devlink_free);
@@ -7789,6 +7886,9 @@ EXPORT_SYMBOL_GPL(devlink_region_destroy);
  *	Driver should use the same id for multiple snapshots taken
  *	on multiple regions at the same time/by the same trigger.
  *
+ *	The caller of this function must use devlink_region_snapshot_id_put
+ *	when finished creating regions using this id.
+ *
  *	Returns zero on success, or a negative error code on failure.
  *
  *	@devlink: devlink
@@ -7806,6 +7906,24 @@ int devlink_region_snapshot_id_get(struct devlink *devlink, u32 *id)
 }
 EXPORT_SYMBOL_GPL(devlink_region_snapshot_id_get);
 
+/**
+ *	devlink_region_snapshot_id_put - put snapshot ID reference
+ *
+ *	This should be called by a driver after finishing creating snapshots
+ *	with an id. Doing so ensures that the ID can later be released in the
+ *	event that all snapshots using it have been destroyed.
+ *
+ *	@devlink: devlink
+ *	@id: id to release reference on
+ */
+void devlink_region_snapshot_id_put(struct devlink *devlink, u32 id)
+{
+	mutex_lock(&devlink->lock);
+	__devlink_snapshot_id_decrement(devlink, id);
+	mutex_unlock(&devlink->lock);
+}
+EXPORT_SYMBOL_GPL(devlink_region_snapshot_id_put);
+
 /**
  *	devlink_region_snapshot_create - create a new snapshot
  *	This will add a new snapshot of a region. The snapshot
-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [net-next v2 09/11] devlink: implement DEVLINK_CMD_REGION_NEW
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (7 preceding siblings ...)
  2020-03-26  3:51 ` [net-next v2 08/11] devlink: track snapshot id usage count using an xarray Jacob Keller
@ 2020-03-26  3:51 ` Jacob Keller
  2020-03-26  8:51   ` Jiri Pirko
  2020-03-26  8:52   ` Jiri Pirko
  2020-03-26  3:51 ` [net-next v2 10/11] netdevsim: support taking immediate snapshot via devlink Jacob Keller
                   ` (6 subsequent siblings)
  15 siblings, 2 replies; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller

Implement support for the DEVLINK_CMD_REGION_NEW command for creating
snapshots. This new command parallels the existing
DEVLINK_CMD_REGION_DEL.

In order for DEVLINK_CMD_REGION_NEW to work for a region, the new
".snapshot" operation must be implemented in the region's ops structure.

The desired snapshot id must be provided. This helps avoid confusion on
the purpose of DEVLINK_CMD_REGION_NEW, and keeps the API simpler.

The requested id will be inserted into the xarray tracking the number of
snapshots using each id. If this id is already used by another snapshot
on any region, an error will be returned.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 .../networking/devlink/devlink-region.rst     |   8 ++
 include/net/devlink.h                         |   6 ++
 net/core/devlink.c                            | 100 ++++++++++++++++++
 3 files changed, 114 insertions(+)

diff --git a/Documentation/networking/devlink/devlink-region.rst b/Documentation/networking/devlink/devlink-region.rst
index 8b46e8591fe0..9d2d4c95a5c4 100644
--- a/Documentation/networking/devlink/devlink-region.rst
+++ b/Documentation/networking/devlink/devlink-region.rst
@@ -20,6 +20,11 @@ address regions that are otherwise inaccessible to the user.
 Regions may also be used to provide an additional way to debug complex error
 states, but see also :doc:`devlink-health`
 
+Regions may optionally support capturing a snapshot on demand via the
+``DEVLINK_CMD_REGION_NEW`` netlink message. A driver wishing to allow
+requested snapshots must implement the ``.snapshot`` callback for the region
+in its ``devlink_region_ops`` structure.
+
 example usage
 -------------
 
@@ -40,6 +45,9 @@ example usage
     # Delete a snapshot using:
     $ devlink region del pci/0000:00:05.0/cr-space snapshot 1
 
+    # Request an immediate snapshot, if supported by the region
+    $ devlink region new pci/0000:00:05.0/cr-space snapshot 5
+
     # Dump a snapshot:
     $ devlink region dump pci/0000:00:05.0/fw-health snapshot 1
     0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
diff --git a/include/net/devlink.h b/include/net/devlink.h
index fb9154060e6e..a1a02cd5890b 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -501,10 +501,16 @@ struct devlink_info_req;
  * struct devlink_region_ops - Region operations
  * @name: region name
  * @destructor: callback used to free snapshot memory when deleting
+ * @snapshot: callback to request an immediate snapshot. On success,
+ *            the data variable must be updated to point to the snapshot data.
+ *            The function will be called while the devlink instance lock is
+ *            held.
  */
 struct devlink_region_ops {
 	const char *name;
 	void (*destructor)(const void *data);
+	int (*snapshot)(struct devlink *devlink, struct netlink_ext_ack *extack,
+			u8 **data);
 };
 
 struct devlink_fmsg;
diff --git a/net/core/devlink.c b/net/core/devlink.c
index d89eaac0a101..a0960ba139ec 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -3845,6 +3845,33 @@ static void __devlink_snapshot_id_decrement(struct devlink *devlink, u32 id)
 	}
 }
 
+/**
+ *	__devlink_snapshot_id_insert - Insert a specific snapshot ID
+ *	@devlink: devlink instance
+ *	@id: the snapshot id
+ *
+ *	Mark the given snapshot id as used by inserting a zero value into the
+ *	snapshot xarray.
+ *
+ *	This must be called while holding the devlink instance lock. Unlike
+ *	devlink_snapshot_id_get, the initial reference count is zero, not one.
+ *	It is expected that the id will immediately be used before
+ *	releasing the devlink instance lock.
+ *
+ *	Returns zero on success, or an error code if the snapshot id could not
+ *	be inserted.
+ */
+static int __devlink_snapshot_id_insert(struct devlink *devlink, u32 id)
+{
+	lockdep_assert_held(&devlink->lock);
+
+	if (WARN_ON(xa_load(&devlink->snapshot_ids, id)))
+		return -EEXIST;
+
+	return xa_err(xa_store(&devlink->snapshot_ids, id, xa_mk_value(0),
+			       GFP_KERNEL));
+}
+
 /**
  *	__devlink_region_snapshot_id_get - get snapshot ID
  *	@devlink: devlink instance
@@ -4038,6 +4065,72 @@ static int devlink_nl_cmd_region_del(struct sk_buff *skb,
 	return 0;
 }
 
+static int
+devlink_nl_cmd_region_new(struct sk_buff *skb, struct genl_info *info)
+{
+	struct devlink *devlink = info->user_ptr[0];
+	struct devlink_region *region;
+	const char *region_name;
+	u32 snapshot_id;
+	u8 *data;
+	int err;
+
+	if (!info->attrs[DEVLINK_ATTR_REGION_NAME]) {
+		NL_SET_ERR_MSG_MOD(info->extack, "No region name provided");
+		return -EINVAL;
+	}
+
+	if (!info->attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
+		NL_SET_ERR_MSG_MOD(info->extack, "No snapshot id provided");
+		return -EINVAL;
+	}
+
+	region_name = nla_data(info->attrs[DEVLINK_ATTR_REGION_NAME]);
+	region = devlink_region_get_by_name(devlink, region_name);
+	if (!region) {
+		NL_SET_ERR_MSG_MOD(info->extack, "The requested region does not exist");
+		return -EINVAL;
+	}
+
+	if (!region->ops->snapshot) {
+		NL_SET_ERR_MSG_MOD(info->extack, "The requested region does not support taking an immediate snapshot");
+		return -EOPNOTSUPP;
+	}
+
+	if (region->cur_snapshots == region->max_snapshots) {
+		NL_SET_ERR_MSG_MOD(info->extack, "The region has reached the maximum number of stored snapshots");
+		return -ENOSPC;
+	}
+
+	snapshot_id = nla_get_u32(info->attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]);
+
+	if (devlink_region_snapshot_get_by_id(region, snapshot_id)) {
+		NL_SET_ERR_MSG_MOD(info->extack, "The requested snapshot id is already in use");
+		return -EEXIST;
+	}
+
+	err = __devlink_snapshot_id_insert(devlink, snapshot_id);
+	if (err) {
+		return err;
+	}
+
+	err = region->ops->snapshot(devlink, info->extack, &data);
+	if (err)
+		goto snapshot_capture_failure;
+
+	err = __devlink_region_snapshot_create(region, data, snapshot_id);
+	if (err)
+		goto snapshot_create_failure;
+
+	return 0;
+
+snapshot_create_failure:
+	region->ops->destructor(data);
+snapshot_capture_failure:
+	__devlink_snapshot_id_decrement(devlink, snapshot_id);
+	return err;
+}
+
 static int devlink_nl_cmd_region_read_chunk_fill(struct sk_buff *msg,
 						 struct devlink *devlink,
 						 u8 *chunk, u32 chunk_size,
@@ -6445,6 +6538,13 @@ static const struct genl_ops devlink_nl_ops[] = {
 		.flags = GENL_ADMIN_PERM,
 		.internal_flags = DEVLINK_NL_FLAG_NEED_DEVLINK,
 	},
+	{
+		.cmd = DEVLINK_CMD_REGION_NEW,
+		.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
+		.doit = devlink_nl_cmd_region_new,
+		.flags = GENL_ADMIN_PERM,
+		.internal_flags = DEVLINK_NL_FLAG_NEED_DEVLINK,
+	},
 	{
 		.cmd = DEVLINK_CMD_REGION_DEL,
 		.validate = GENL_DONT_VALIDATE_STRICT | GENL_DONT_VALIDATE_DUMP,
-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [net-next v2 10/11] netdevsim: support taking immediate snapshot via devlink
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (8 preceding siblings ...)
  2020-03-26  3:51 ` [net-next v2 09/11] devlink: implement DEVLINK_CMD_REGION_NEW Jacob Keller
@ 2020-03-26  3:51 ` Jacob Keller
  2020-03-26  9:00   ` Jiri Pirko
  2020-03-26  3:51 ` [net-next v2 11/11] ice: add a devlink region for dumping NVM contents Jacob Keller
                   ` (5 subsequent siblings)
  15 siblings, 1 reply; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller

Implement the .snapshot region operation for the dummy data region. This
enables a region snapshot to be taken upon request via the new
DEVLINK_CMD_REGION_SNAPSHOT command.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 drivers/net/netdevsim/dev.c                   | 30 ++++++++++++++-----
 .../drivers/net/netdevsim/devlink.sh          | 10 +++++++
 2 files changed, 33 insertions(+), 7 deletions(-)

diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c
index 53ec891659eb..ffc295c7653e 100644
--- a/drivers/net/netdevsim/dev.c
+++ b/drivers/net/netdevsim/dev.c
@@ -39,24 +39,39 @@ static struct dentry *nsim_dev_ddir;
 
 #define NSIM_DEV_DUMMY_REGION_SIZE (1024 * 32)
 
+static int
+nsim_dev_take_snapshot(struct devlink *devlink, struct netlink_ext_ack *extack,
+		       u8 **data)
+{
+	void *dummy_data;
+
+	dummy_data = kmalloc(NSIM_DEV_DUMMY_REGION_SIZE, GFP_KERNEL);
+	if (!dummy_data)
+		return -ENOMEM;
+
+	get_random_bytes(dummy_data, NSIM_DEV_DUMMY_REGION_SIZE);
+
+	*data = dummy_data;
+
+	return 0;
+}
+
 static ssize_t nsim_dev_take_snapshot_write(struct file *file,
 					    const char __user *data,
 					    size_t count, loff_t *ppos)
 {
 	struct nsim_dev *nsim_dev = file->private_data;
 	struct devlink *devlink = priv_to_devlink(nsim_dev);
-	void *dummy_data;
+	u8 *dummy_data;
 	int err;
 	u32 id;
 
-	dummy_data = kmalloc(NSIM_DEV_DUMMY_REGION_SIZE, GFP_KERNEL);
-	if (!dummy_data)
-		return -ENOMEM;
-
-	get_random_bytes(dummy_data, NSIM_DEV_DUMMY_REGION_SIZE);
+	err = nsim_dev_take_snapshot(devlink, NULL, &dummy_data);
+	if (err)
+		return err;
 
 	err = devlink_region_snapshot_id_get(devlink, &id);
-	if (err)
+	if (err) {
 		pr_err("Failed to get snapshot id\n");
 		return err;
 	}
@@ -350,6 +365,7 @@ static void nsim_devlink_param_load_driverinit_values(struct devlink *devlink)
 static const struct devlink_region_ops dummy_region_ops = {
 	.name = "dummy",
 	.destructor = &kfree,
+	.snapshot = nsim_dev_take_snapshot,
 };
 
 static int nsim_dev_dummy_region_init(struct nsim_dev *nsim_dev,
diff --git a/tools/testing/selftests/drivers/net/netdevsim/devlink.sh b/tools/testing/selftests/drivers/net/netdevsim/devlink.sh
index 025a84c2ab5a..32cb2a159c70 100755
--- a/tools/testing/selftests/drivers/net/netdevsim/devlink.sh
+++ b/tools/testing/selftests/drivers/net/netdevsim/devlink.sh
@@ -141,6 +141,16 @@ regions_test()
 
 	check_region_snapshot_count dummy post-first-delete 2
 
+	devlink region new $DL_HANDLE/dummy snapshot 25
+	check_err $? "Failed to create a new snapshot with id 25"
+
+	check_region_snapshot_count dummy post-first-request 3
+
+	devlink region del $DL_HANDLE/dummy snapshot 25
+	check_err $? "Failed to delete snapshot with id 25"
+
+	check_region_snapshot_count dummy post-second-delete 2
+
 	log_test "regions test"
 }
 
-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* [net-next v2 11/11] ice: add a devlink region for dumping NVM contents
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (9 preceding siblings ...)
  2020-03-26  3:51 ` [net-next v2 10/11] netdevsim: support taking immediate snapshot via devlink Jacob Keller
@ 2020-03-26  3:51 ` Jacob Keller
  2020-03-26  9:02   ` Jiri Pirko
  2020-03-26  7:23 ` [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jiri Pirko
                   ` (4 subsequent siblings)
  15 siblings, 1 reply; 33+ messages in thread
From: Jacob Keller @ 2020-03-26  3:51 UTC (permalink / raw)
  To: netdev; +Cc: Jakub Kicinski, Jiri Pirko, Jacob Keller

Add a devlink region for exposing the device's Non Volatime Memory flash
contents.

Support the recently added .snapshot operation, enabling userspace to
request a snapshot of the NVM contents via DEVLINK_CMD_REGION_NEW.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 Documentation/networking/devlink/ice.rst     | 26 +++++
 drivers/net/ethernet/intel/ice/ice.h         |  2 +
 drivers/net/ethernet/intel/ice/ice_devlink.c | 99 ++++++++++++++++++++
 drivers/net/ethernet/intel/ice/ice_devlink.h |  3 +
 drivers/net/ethernet/intel/ice/ice_main.c    |  4 +
 5 files changed, 134 insertions(+)

diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
index 37fbbd40a5e5..f3d6a3b50342 100644
--- a/Documentation/networking/devlink/ice.rst
+++ b/Documentation/networking/devlink/ice.rst
@@ -69,3 +69,29 @@ The ``ice`` driver reports the following versions
       - The version of the DDP package that is active in the device. Note
         that both the name (as reported by ``fw.app.name``) and version are
         required to uniquely identify the package.
+
+Regions
+=======
+
+The ``ice`` driver enables access to the contents of the Non Volatile Memory
+flash chip via the ``nvm-flash`` region.
+
+Users can request an immediate capture of a snapshot via the
+``DEVLINK_CMD_REGION_NEW``
+
+.. code:: shell
+
+    $ devlink region new pci/0000:01:00.0/nvm-flash snapshot 1
+    $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1
+
+    $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1
+    0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
+    0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8
+    0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc
+    0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5
+
+    $ devlink region read pci/0000:01:00.0/nvm-flash snapshot 1 address 0
+        length 16
+    0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
+
+    $ devlink region delete pci/0000:01:00.0/nvm-flash snapshot 1
diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
index 8ce3afcfeca0..5c11448bfbb3 100644
--- a/drivers/net/ethernet/intel/ice/ice.h
+++ b/drivers/net/ethernet/intel/ice/ice.h
@@ -351,6 +351,8 @@ struct ice_pf {
 	/* devlink port data */
 	struct devlink_port devlink_port;
 
+	struct devlink_region *nvm_region;
+
 	/* OS reserved IRQ details */
 	struct msix_entry *msix_entries;
 	struct ice_res_tracker *irq_tracker;
diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.c b/drivers/net/ethernet/intel/ice/ice_devlink.c
index 27c5034c039a..91edeffd73b1 100644
--- a/drivers/net/ethernet/intel/ice/ice_devlink.c
+++ b/drivers/net/ethernet/intel/ice/ice_devlink.c
@@ -318,3 +318,102 @@ void ice_devlink_destroy_port(struct ice_pf *pf)
 	devlink_port_type_clear(&pf->devlink_port);
 	devlink_port_unregister(&pf->devlink_port);
 }
+
+/**
+ * ice_devlink_nvm_snapshot - Capture a snapshot of the Shadow RAM contents
+ * @devlink: the devlink instance
+ * @extack: extended ACK response structure
+ * @data: on exit points to snapshot data buffer
+ *
+ * This function is called in response to the DEVLINK_CMD_REGION_TRIGGER for
+ * the shadow-ram devlink region. It captures a snapshot of the shadow ram
+ * contents. This snapshot can later be viewed via the devlink-region
+ * interface.
+ *
+ * @returns zero on success, and updates the data pointer. Returns a non-zero
+ * error code on failure.
+ */
+static int
+ice_devlink_nvm_snapshot(struct devlink *devlink, struct netlink_ext_ack *extack,
+			u8 **data)
+{
+	struct ice_pf *pf = devlink_priv(devlink);
+	struct device *dev = ice_pf_to_dev(pf);
+	struct ice_hw *hw = &pf->hw;
+	enum ice_status status;
+	void *nvm_data;
+	u32 nvm_size;
+
+	nvm_size = hw->nvm.flash_size;
+	nvm_data = vzalloc(nvm_size);
+	if (!nvm_data) {
+		NL_SET_ERR_MSG_MOD(extack, "Out of memory");
+		return -ENOMEM;
+	}
+
+	status = ice_acquire_nvm(hw, ICE_RES_READ);
+	if (status) {
+		dev_dbg(dev, "ice_acquire_nvm failed, err %d aq_err %d\n",
+			status, hw->adminq.sq_last_status);
+		NL_SET_ERR_MSG_MOD(extack, "Failed to acquire NVM semaphore");
+		vfree(nvm_data);
+		return -EIO;
+	}
+
+	status = ice_read_flat_nvm(hw, 0, &nvm_size, nvm_data, false);
+	if (status) {
+		dev_dbg(dev, "ice_read_flat_nvm failed after reading %u bytes, err %d aq_err %d\n",
+			nvm_size, status, hw->adminq.sq_last_status);
+		NL_SET_ERR_MSG_MOD(extack, "Failed to read NVM contents");
+		ice_release_nvm(hw);
+		vfree(nvm_data);
+		return -EIO;
+	}
+
+	ice_release_nvm(hw);
+
+	*data = nvm_data;
+
+	return 0;
+}
+
+static const struct devlink_region_ops ice_nvm_region_ops = {
+	.name = "nvm-flash",
+	.destructor = vfree,
+	.snapshot = ice_devlink_nvm_snapshot,
+};
+
+/**
+ * ice_devlink_init_regions - Initialize devlink regions
+ * @pf: the PF device structure
+ *
+ * Create devlink regions used to enable access to dump the contents of the
+ * flash memory on the device.
+ */
+void ice_devlink_init_regions(struct ice_pf *pf)
+{
+	struct devlink *devlink = priv_to_devlink(pf);
+	struct device *dev = ice_pf_to_dev(pf);
+	u64 nvm_size;
+
+	nvm_size = pf->hw.nvm.flash_size;
+	pf->nvm_region = devlink_region_create(devlink, &ice_nvm_region_ops, 1,
+					       nvm_size);
+	if (IS_ERR(pf->nvm_region)) {
+		dev_err(dev, "failed to create NVM devlink region, err %ld\n",
+			PTR_ERR(pf->nvm_region));
+		pf->nvm_region = NULL;
+	}
+}
+
+/**
+ * ice_devlink_destroy_regions - Destroy devlink regions
+ * @pf: the PF device structure
+ *
+ * Remove previously created regions for this PF.
+ */
+void ice_devlink_destroy_regions(struct ice_pf *pf)
+{
+	if (pf->nvm_region)
+		devlink_region_destroy(pf->nvm_region);
+}
diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.h b/drivers/net/ethernet/intel/ice/ice_devlink.h
index f94dc93c24c5..6e806a08dc23 100644
--- a/drivers/net/ethernet/intel/ice/ice_devlink.h
+++ b/drivers/net/ethernet/intel/ice/ice_devlink.h
@@ -11,4 +11,7 @@ void ice_devlink_unregister(struct ice_pf *pf);
 int ice_devlink_create_port(struct ice_pf *pf);
 void ice_devlink_destroy_port(struct ice_pf *pf);
 
+void ice_devlink_init_regions(struct ice_pf *pf);
+void ice_devlink_destroy_regions(struct ice_pf *pf);
+
 #endif /* _ICE_DEVLINK_H_ */
diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
index 359ff8544773..306a4e5b2320 100644
--- a/drivers/net/ethernet/intel/ice/ice_main.c
+++ b/drivers/net/ethernet/intel/ice/ice_main.c
@@ -3276,6 +3276,8 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
 		goto err_init_pf_unroll;
 	}
 
+	ice_devlink_init_regions(pf);
+
 	pf->num_alloc_vsi = hw->func_caps.guar_num_vsi;
 	if (!pf->num_alloc_vsi) {
 		err = -EIO;
@@ -3385,6 +3387,7 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
 	devm_kfree(dev, pf->vsi);
 err_init_pf_unroll:
 	ice_deinit_pf(pf);
+	ice_devlink_destroy_regions(pf);
 	ice_deinit_hw(hw);
 err_exit_unroll:
 	ice_devlink_unregister(pf);
@@ -3427,6 +3430,7 @@ static void ice_remove(struct pci_dev *pdev)
 		ice_vsi_free_q_vectors(pf->vsi[i]);
 	}
 	ice_deinit_pf(pf);
+	ice_devlink_destroy_regions(pf);
 	ice_deinit_hw(&pf->hw);
 	ice_devlink_unregister(pf);
 
-- 
2.24.1


^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (10 preceding siblings ...)
  2020-03-26  3:51 ` [net-next v2 11/11] ice: add a devlink region for dumping NVM contents Jacob Keller
@ 2020-03-26  7:23 ` Jiri Pirko
  2020-03-26  7:51 ` Jiri Pirko
                   ` (3 subsequent siblings)
  15 siblings, 0 replies; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  7:23 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 04:51:46AM CET, jacob.e.keller@intel.com wrote:
>This is a second revision of the previous series to implement the
>DEVLINK_CMD_REGION_NEW. The series can be viewed on lore.kernel.org at
>
>https://lore.kernel.org/netdev/20200324223445.2077900-1-jacob.e.keller@intel.com/
>
>This version includes the suggested cleanups from Jakub and Jiri on the
>list, including the following changes, broken out by the v1 patch title.

Jakob. You are missing the cover letter.

You need to include the cover letter in the same way as for v1. The only
thing that changes is that you add a changelog. Or you can just have
changelog per patch.

Please, keep the changelog complete. For v1 you had changelog from RFC.
Keep it.


I suggest to repost to include the cover letter.


>
>Changes to patches since v1:
>
> * devlink: prepare to support region operations
>
>   No changes
>
> * devlink: convert snapshot destructor callback to region op
>
>   No changes
>
> * devlink: trivial: fix tab in function documentation
>
>   No changes
>
> * devlink: add function to take snapshot while locked
>
>   Added Jakub's Reviewed-by tag.
>
> * <NEW> devlink: use -ENOSPC to indicate no more room for snapshots
>
>   New patch added to convert confusing -ENOMEM to -ENOSPC, as suggested by
>   Jiri.
>
> * devlink: extract snapshot id allocation to helper function
>
>   No changes
>
> * devlink: convert snapshot id getter to return an error
>
>   Changed title to "devlink: report error once U32_MAX snapshot ids have
>   been used".
>
>   Refactored this patch to make devlink_region_snapshot_id_get take a
>   pointer to u32, so that the error value and id value are separated. This
>   means that we can remove the INT_MAX limitation on id values.
>
> * devlink: track snapshot id usage count using an xarray
>
>   Fixed the xa_init to use xa_init_flags with XA_FLAGS_ALLOC, so that
>   xa_alloc can properly be used.
>
>   Changed devlink_region_snapshot_id_get to use an initial count of 1
>   instead of 0. Added a new devlink_region_snapshot_id_put function, used
>   to release this initial count. This closes the race condition and issues
>   caused if the driver either doesn't create a snapshot, or if userspace
>   deletes the first snapshot before others are created.
>
>   Used WARN_ON in a few more checks that should not occur, such as if the
>   xarray entry is not a value, or when the id isn't yet in the xarray.
>
>   Removed an unnecessary if (err) { return err; } construction.
>
>   Use xa_limit_32b instead of xa_limit_31b now that we don't return the
>   snapshot id directly.
>
>   Cleanup the label used in __devlink_region_snapshot_create to indicate the
>   failure cause, rather than the cleanup step.
>
>   Removed the unnecessary locking around xa_destroy
>
> * devlink: implement DEVLINK_CMD_REGION_NEW
>
>   Added a WARN_ON to the check in snapshot_id_insert in case the id already
>   exists.
>
>   Removed an unnecessary "if (err) { return err; }" construction
>
>   Use -ENOSPC instead of -ENOMEM when max_snapshots is reached.
>
>   Cleanup label names to match style of the other labels in the file,
>   naming after the failure cause rather than the cleanup step. Also fix a
>   bug in the label ordering.
>
>   Call the new devlink_region_snapshot_id_put function in the mlx4 and
>   netdevsim drivers.
>
> * netdevsim: support taking immediate snapshot via devlink
>
>   Create a local devlink pointer instead of calling priv_to_devlink
>   multiple times.
>
>   Removed previous selftest for devlink region new without a snapshot id,
>   as this is no longer supported. Adjusted and verified that the tests pass
>   now.
>
> * ice: add a devlink region for dumping NVM contents
>
>   Use "dev_err" instead of "dev_warn" for a message about failure to create
>   the devlink region.
>
>Jacob Keller (11):
>  devlink: prepare to support region operations
>  devlink: convert snapshot destructor callback to region op
>  devlink: trivial: fix tab in function documentation
>  devlink: add function to take snapshot while locked
>  devlink: use -ENOSPC to indicate no more room for snapshots
>  devlink: extract snapshot id allocation to helper function
>  devlink: report error once U32_MAX snapshot ids have been used
>  devlink: track snapshot id usage count using an xarray
>  devlink: implement DEVLINK_CMD_REGION_NEW
>  netdevsim: support taking immediate snapshot via devlink
>  ice: add a devlink region for dumping NVM contents
>
> .../networking/devlink/devlink-region.rst     |   8 +
> Documentation/networking/devlink/ice.rst      |  26 ++
> drivers/net/ethernet/intel/ice/ice.h          |   2 +
> drivers/net/ethernet/intel/ice/ice_devlink.c  |  99 +++++
> drivers/net/ethernet/intel/ice/ice_devlink.h  |   3 +
> drivers/net/ethernet/intel/ice/ice_main.c     |   4 +
> drivers/net/ethernet/mellanox/mlx4/crdump.c   |  36 +-
> drivers/net/netdevsim/dev.c                   |  46 ++-
> include/net/devlink.h                         |  33 +-
> net/core/devlink.c                            | 363 +++++++++++++++---
> .../drivers/net/netdevsim/devlink.sh          |  10 +
> 11 files changed, 550 insertions(+), 80 deletions(-)
>
>-- 
>2.24.1
>

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 05/11] devlink: use -ENOSPC to indicate no more room for snapshots
  2020-03-26  3:51 ` [net-next v2 05/11] devlink: use -ENOSPC to indicate no more room for snapshots Jacob Keller
@ 2020-03-26  7:24   ` Jiri Pirko
  0 siblings, 0 replies; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  7:24 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 04:51:51AM CET, jacob.e.keller@intel.com wrote:
>The devlink_region_snapshot_create function returns -ENOMEM when the
>maximum number of snapshots has been reached. This is confusing because
>it is not an issue of being out of memory. Change this to use -ENOSPC instead.
>
>Reported-by: Jiri Pirko <jiri@resnulli.us>
>Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>

Reviewed-by: Jiri Pirko <jiri@mellanox.com>

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 06/11] devlink: extract snapshot id allocation to helper function
  2020-03-26  3:51 ` [net-next v2 06/11] devlink: extract snapshot id allocation to helper function Jacob Keller
@ 2020-03-26  7:25   ` Jiri Pirko
  0 siblings, 0 replies; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  7:25 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 04:51:52AM CET, jacob.e.keller@intel.com wrote:
>A future change is going to implement a new devlink command to request
>a snapshot on demand. As part of this, the logic for handling the
>snapshot ids will be refactored. To simplify the snapshot id allocation
>function, move it to a separate function prefixed by `__`. This helper
>function will assume the lock is held.
>
>While no other callers will exist, it simplifies refactoring the logic
>because there is no need to complicate the function with gotos to handle
>unlocking on failure.
>
>Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>

I already provided reviewed by tag for this. Please carry those around.

Reviewed-by: Jiri Pirko <jiri@mellanox.com>

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 07/11] devlink: report error once U32_MAX snapshot ids have been used
  2020-03-26  3:51 ` [net-next v2 07/11] devlink: report error once U32_MAX snapshot ids have been used Jacob Keller
@ 2020-03-26  7:30   ` Jiri Pirko
  2020-03-26 16:09     ` Keller, Jacob E
  0 siblings, 1 reply; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  7:30 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 04:51:53AM CET, jacob.e.keller@intel.com wrote:
>The devlink_snapshot_id_get() function returns a snapshot id. The
>snapshot id is a u32, so there is no way to indicate an error code.
>
>Indeed, the two current callers of devlink_snapshot_id_get() assume that
>a negative value is an error.

I don't see they do.


>
>A future change is going to possibly add additional cases where this
>function could fail. Refactor the function to return the snapshot id in
>an argument, so that it can return zero or an error value.
>
>This ensures that snapshot ids cannot be confused with error values, and
>aids in the future refactor of snapshot id allocation management.
>
>Because there is no current way to release previously used snapshot ids,
>add a simple check ensuring that an error is reported in case the
>snapshot_id would over flow.
>
>Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>

The code looks good.

Reviewed-by: Jiri Pirko <jiri@mellanox.com>

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (11 preceding siblings ...)
  2020-03-26  7:23 ` [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jiri Pirko
@ 2020-03-26  7:51 ` Jiri Pirko
  2020-03-26 16:15   ` Jacob Keller
  2020-03-26  8:36 ` Jiri Pirko
                   ` (2 subsequent siblings)
  15 siblings, 1 reply; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  7:51 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 04:51:46AM CET, jacob.e.keller@intel.com wrote:
>This is a second revision of the previous series to implement the
>DEVLINK_CMD_REGION_NEW. The series can be viewed on lore.kernel.org at
>
>https://lore.kernel.org/netdev/20200324223445.2077900-1-jacob.e.keller@intel.com/
>
>This version includes the suggested cleanups from Jakub and Jiri on the
>list, including the following changes, broken out by the v1 patch title.
>
>Changes to patches since v1:
>
> * devlink: prepare to support region operations
>
>   No changes
>
> * devlink: convert snapshot destructor callback to region op
>
>   No changes
>
> * devlink: trivial: fix tab in function documentation
>
>   No changes
>
> * devlink: add function to take snapshot while locked
>
>   Added Jakub's Reviewed-by tag.
>
> * <NEW> devlink: use -ENOSPC to indicate no more room for snapshots
>
>   New patch added to convert confusing -ENOMEM to -ENOSPC, as suggested by
>   Jiri.
>
> * devlink: extract snapshot id allocation to helper function
>
>   No changes
>
> * devlink: convert snapshot id getter to return an error
>
>   Changed title to "devlink: report error once U32_MAX snapshot ids have
>   been used".
>
>   Refactored this patch to make devlink_region_snapshot_id_get take a
>   pointer to u32, so that the error value and id value are separated. This
>   means that we can remove the INT_MAX limitation on id values.
>
> * devlink: track snapshot id usage count using an xarray
>
>   Fixed the xa_init to use xa_init_flags with XA_FLAGS_ALLOC, so that
>   xa_alloc can properly be used.
>
>   Changed devlink_region_snapshot_id_get to use an initial count of 1
>   instead of 0. Added a new devlink_region_snapshot_id_put function, used
>   to release this initial count. This closes the race condition and issues
>   caused if the driver either doesn't create a snapshot, or if userspace
>   deletes the first snapshot before others are created.
>
>   Used WARN_ON in a few more checks that should not occur, such as if the
>   xarray entry is not a value, or when the id isn't yet in the xarray.
>
>   Removed an unnecessary if (err) { return err; } construction.
>
>   Use xa_limit_32b instead of xa_limit_31b now that we don't return the
>   snapshot id directly.
>
>   Cleanup the label used in __devlink_region_snapshot_create to indicate the
>   failure cause, rather than the cleanup step.
>
>   Removed the unnecessary locking around xa_destroy
>
> * devlink: implement DEVLINK_CMD_REGION_NEW
>
>   Added a WARN_ON to the check in snapshot_id_insert in case the id already
>   exists.
>
>   Removed an unnecessary "if (err) { return err; }" construction
>
>   Use -ENOSPC instead of -ENOMEM when max_snapshots is reached.
>
>   Cleanup label names to match style of the other labels in the file,
>   naming after the failure cause rather than the cleanup step. Also fix a
>   bug in the label ordering.
>
>   Call the new devlink_region_snapshot_id_put function in the mlx4 and
>   netdevsim drivers.
>
> * netdevsim: support taking immediate snapshot via devlink
>
>   Create a local devlink pointer instead of calling priv_to_devlink
>   multiple times.
>
>   Removed previous selftest for devlink region new without a snapshot id,
>   as this is no longer supported. Adjusted and verified that the tests pass
>   now.
>
> * ice: add a devlink region for dumping NVM contents
>
>   Use "dev_err" instead of "dev_warn" for a message about failure to create
>   the devlink region.

Could you please have the changelog per-patch, as I suggested for v1?
Much easier to review then.

Also, please omit the "no changes" notes.


>
>Jacob Keller (11):
>  devlink: prepare to support region operations
>  devlink: convert snapshot destructor callback to region op
>  devlink: trivial: fix tab in function documentation
>  devlink: add function to take snapshot while locked
>  devlink: use -ENOSPC to indicate no more room for snapshots
>  devlink: extract snapshot id allocation to helper function
>  devlink: report error once U32_MAX snapshot ids have been used
>  devlink: track snapshot id usage count using an xarray
>  devlink: implement DEVLINK_CMD_REGION_NEW
>  netdevsim: support taking immediate snapshot via devlink
>  ice: add a devlink region for dumping NVM contents
>
> .../networking/devlink/devlink-region.rst     |   8 +
> Documentation/networking/devlink/ice.rst      |  26 ++
> drivers/net/ethernet/intel/ice/ice.h          |   2 +
> drivers/net/ethernet/intel/ice/ice_devlink.c  |  99 +++++
> drivers/net/ethernet/intel/ice/ice_devlink.h  |   3 +
> drivers/net/ethernet/intel/ice/ice_main.c     |   4 +
> drivers/net/ethernet/mellanox/mlx4/crdump.c   |  36 +-
> drivers/net/netdevsim/dev.c                   |  46 ++-
> include/net/devlink.h                         |  33 +-
> net/core/devlink.c                            | 363 +++++++++++++++---
> .../drivers/net/netdevsim/devlink.sh          |  10 +
> 11 files changed, 550 insertions(+), 80 deletions(-)
>
>-- 
>2.24.1
>

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 08/11] devlink: track snapshot id usage count using an xarray
  2020-03-26  3:51 ` [net-next v2 08/11] devlink: track snapshot id usage count using an xarray Jacob Keller
@ 2020-03-26  8:35   ` Jiri Pirko
  0 siblings, 0 replies; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  8:35 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 04:51:54AM CET, jacob.e.keller@intel.com wrote:
>Each snapshot created for a devlink region must have an id. These ids
>are supposed to be unique per "event" that caused the snapshot to be
>created. Drivers call devlink_region_snapshot_id_get to obtain a new id
>to use for a new event trigger. The id values are tracked per devlink,
>so that the same id number can be used if a triggering event creates
>multiple snapshots on different regions.
>
>There is no mechanism for snapshot ids to ever be reused. Introduce an
>xarray to store the count of how many snapshots are using a given id,
>replacing the snapshot_id field previously used for picking the next id.
>
>The devlink_region_snapshot_id_get() function will use xa_alloc to
>insert an initial value of 1 value at an available slot between 0 and
>U32_MAX.
>
>The new __devlink_snapshot_id_increment() and
>__devlink_snapshot_id_decrement() functions will be used to track how
>many snapshots currently use an id.
>
>Drivers must now call devlink_snapshot_id_put() in order to release
>their reference of the snapshot id after adding region snapshots.
>
>By tracking the total number of snapshots using a given id, it is
>possible for the decrement() function to erase the id from the xarray
>when it is not in use.
>
>With this method, a snapshot id can become reused again once all
>snapshots that referred to it have been deleted via
>DEVLINK_CMD_REGION_DEL, and the driver has finished adding snapshots.
>
>This work also paves the way to introduce a mechanism for userspace to
>request a snapshot.
>
>Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>

This looks good to me. 2 small nits below.
With or without changing them:
Reviewed-by: Jiri Pirko <jiri@mellanox.com>


>---
> drivers/net/ethernet/mellanox/mlx4/crdump.c |   3 +
> drivers/net/netdevsim/dev.c                 |   5 +-
> include/net/devlink.h                       |   4 +-
> net/core/devlink.c                          | 130 +++++++++++++++++++-
> 4 files changed, 134 insertions(+), 8 deletions(-)
>
>diff --git a/drivers/net/ethernet/mellanox/mlx4/crdump.c b/drivers/net/ethernet/mellanox/mlx4/crdump.c
>index 792951f6df0d..2700628f1689 100644
>--- a/drivers/net/ethernet/mellanox/mlx4/crdump.c
>+++ b/drivers/net/ethernet/mellanox/mlx4/crdump.c
>@@ -203,6 +203,9 @@ int mlx4_crdump_collect(struct mlx4_dev *dev)
> 	mlx4_crdump_collect_crspace(dev, cr_space, id);
> 	mlx4_crdump_collect_fw_health(dev, cr_space, id);
> 
>+	/* Release reference on the snapshot id */
>+	devlink_region_snapshot_id_put(devlink, id);
>+
> 	crdump_disable_crspace_access(dev, cr_space);
> 
> 	iounmap(cr_space);
>diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c
>index b851fe63a75d..53ec891659eb 100644
>--- a/drivers/net/netdevsim/dev.c
>+++ b/drivers/net/netdevsim/dev.c
>@@ -44,6 +44,7 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file,
> 					    size_t count, loff_t *ppos)
> {
> 	struct nsim_dev *nsim_dev = file->private_data;
>+	struct devlink *devlink = priv_to_devlink(nsim_dev);

DaveM seems to frown upon breaking the reverse christmas tree like this.

Don't assign devlink here and do it later on instead to avoid the tree
breakage.


> 	void *dummy_data;
> 	int err;
> 	u32 id;
>@@ -54,13 +55,15 @@ static ssize_t nsim_dev_take_snapshot_write(struct file *file,
> 
> 	get_random_bytes(dummy_data, NSIM_DEV_DUMMY_REGION_SIZE);
> 
>-	err = devlink_region_snapshot_id_get(priv_to_devlink(nsim_dev), &id);
>+	err = devlink_region_snapshot_id_get(devlink, &id);
> 	if (err)
> 		pr_err("Failed to get snapshot id\n");
> 		return err;
> 	}
> 	err = devlink_region_snapshot_create(nsim_dev->dummy_region,
> 					     dummy_data, id);
>+	/* release the snapshot id regardless of errors */

I don't think this comment is necessary. Up to you.


>+	devlink_region_snapshot_id_put(devlink, id);
> 	if (err) {
> 		pr_err("Failed to create region snapshot\n");
> 		kfree(dummy_data);

[...]

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (12 preceding siblings ...)
  2020-03-26  7:51 ` Jiri Pirko
@ 2020-03-26  8:36 ` Jiri Pirko
  2020-03-26  8:54 ` Jiri Pirko
  2020-03-26 18:27 ` David Miller
  15 siblings, 0 replies; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  8:36 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Re subject. Should be:
[patch net-next v2 00/11]

You are missing "patch".

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 09/11] devlink: implement DEVLINK_CMD_REGION_NEW
  2020-03-26  3:51 ` [net-next v2 09/11] devlink: implement DEVLINK_CMD_REGION_NEW Jacob Keller
@ 2020-03-26  8:51   ` Jiri Pirko
  2020-03-26  8:52   ` Jiri Pirko
  1 sibling, 0 replies; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  8:51 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 04:51:55AM CET, jacob.e.keller@intel.com wrote:
>Implement support for the DEVLINK_CMD_REGION_NEW command for creating
>snapshots. This new command parallels the existing
>DEVLINK_CMD_REGION_DEL.
>
>In order for DEVLINK_CMD_REGION_NEW to work for a region, the new
>".snapshot" operation must be implemented in the region's ops structure.
>
>The desired snapshot id must be provided. This helps avoid confusion on
>the purpose of DEVLINK_CMD_REGION_NEW, and keeps the API simpler.
>
>The requested id will be inserted into the xarray tracking the number of
>snapshots using each id. If this id is already used by another snapshot
>on any region, an error will be returned.
>
>Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>

Reviewed-by: Jiri Pirko <jiri@mellanox.com>

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 09/11] devlink: implement DEVLINK_CMD_REGION_NEW
  2020-03-26  3:51 ` [net-next v2 09/11] devlink: implement DEVLINK_CMD_REGION_NEW Jacob Keller
  2020-03-26  8:51   ` Jiri Pirko
@ 2020-03-26  8:52   ` Jiri Pirko
  2020-03-26 16:19     ` Jacob Keller
  1 sibling, 1 reply; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  8:52 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 04:51:55AM CET, jacob.e.keller@intel.com wrote:

[...]

>+	err = __devlink_snapshot_id_insert(devlink, snapshot_id);
>+	if (err) {
>+		return err;
>+	}
>+
>+	err = region->ops->snapshot(devlink, info->extack, &data);
>+	if (err)
>+		goto snapshot_capture_failure;
>+
>+	err = __devlink_region_snapshot_create(region, data, snapshot_id);
>+	if (err)
>+		goto snapshot_create_failure;
>+
>+	return 0;
>+
>+snapshot_create_failure:
>+	region->ops->destructor(data);
>+snapshot_capture_failure:

Eh, this actually should be "err_snapshot_capture" and
"err_snapshot_create"


>+	__devlink_snapshot_id_decrement(devlink, snapshot_id);
>+	return err;
>+}
>+

[...]

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (13 preceding siblings ...)
  2020-03-26  8:36 ` Jiri Pirko
@ 2020-03-26  8:54 ` Jiri Pirko
  2020-03-26 18:27 ` David Miller
  15 siblings, 0 replies; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  8:54 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 04:51:46AM CET, jacob.e.keller@intel.com wrote:

[...]

> * devlink: implement DEVLINK_CMD_REGION_NEW
>
>   Added a WARN_ON to the check in snapshot_id_insert in case the id already
>   exists.
>
>   Removed an unnecessary "if (err) { return err; }" construction
>
>   Use -ENOSPC instead of -ENOMEM when max_snapshots is reached.
>
>   Cleanup label names to match style of the other labels in the file,
>   naming after the failure cause rather than the cleanup step. Also fix a
>   bug in the label ordering.
>
>   Call the new devlink_region_snapshot_id_put function in the mlx4 and
>   netdevsim drivers.

This belongs to a different patch. This would not happen if you'd have
per-patch changelog :)

[...]

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 10/11] netdevsim: support taking immediate snapshot via devlink
  2020-03-26  3:51 ` [net-next v2 10/11] netdevsim: support taking immediate snapshot via devlink Jacob Keller
@ 2020-03-26  9:00   ` Jiri Pirko
  0 siblings, 0 replies; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  9:00 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 04:51:56AM CET, jacob.e.keller@intel.com wrote:
>Implement the .snapshot region operation for the dummy data region. This
>enables a region snapshot to be taken upon request via the new
>DEVLINK_CMD_REGION_SNAPSHOT command.
>
>Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
>---
> drivers/net/netdevsim/dev.c                   | 30 ++++++++++++++-----
> .../drivers/net/netdevsim/devlink.sh          | 10 +++++++
> 2 files changed, 33 insertions(+), 7 deletions(-)
>
>diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c
>index 53ec891659eb..ffc295c7653e 100644
>--- a/drivers/net/netdevsim/dev.c
>+++ b/drivers/net/netdevsim/dev.c
>@@ -39,24 +39,39 @@ static struct dentry *nsim_dev_ddir;
> 
> #define NSIM_DEV_DUMMY_REGION_SIZE (1024 * 32)
> 
>+static int
>+nsim_dev_take_snapshot(struct devlink *devlink, struct netlink_ext_ack *extack,
>+		       u8 **data)
>+{
>+	void *dummy_data;
>+
>+	dummy_data = kmalloc(NSIM_DEV_DUMMY_REGION_SIZE, GFP_KERNEL);
>+	if (!dummy_data)
>+		return -ENOMEM;
>+
>+	get_random_bytes(dummy_data, NSIM_DEV_DUMMY_REGION_SIZE);
>+
>+	*data = dummy_data;
>+
>+	return 0;
>+}
>+
> static ssize_t nsim_dev_take_snapshot_write(struct file *file,
> 					    const char __user *data,
> 					    size_t count, loff_t *ppos)
> {
> 	struct nsim_dev *nsim_dev = file->private_data;
> 	struct devlink *devlink = priv_to_devlink(nsim_dev);
>-	void *dummy_data;
>+	u8 *dummy_data;
> 	int err;
> 	u32 id;
> 
>-	dummy_data = kmalloc(NSIM_DEV_DUMMY_REGION_SIZE, GFP_KERNEL);
>-	if (!dummy_data)
>-		return -ENOMEM;
>-
>-	get_random_bytes(dummy_data, NSIM_DEV_DUMMY_REGION_SIZE);
>+	err = nsim_dev_take_snapshot(devlink, NULL, &dummy_data);
>+	if (err)
>+		return err;
> 
> 	err = devlink_region_snapshot_id_get(devlink, &id);
>-	if (err)
>+	if (err) {

Hmm, this looks odd. How does the code compile before this patch is
applied?


> 		pr_err("Failed to get snapshot id\n");
> 		return err;
> 	}
>@@ -350,6 +365,7 @@ static void nsim_devlink_param_load_driverinit_values(struct devlink *devlink)
> static const struct devlink_region_ops dummy_region_ops = {
> 	.name = "dummy",
> 	.destructor = &kfree,
>+	.snapshot = nsim_dev_take_snapshot,
> };
> 
> static int nsim_dev_dummy_region_init(struct nsim_dev *nsim_dev,
>diff --git a/tools/testing/selftests/drivers/net/netdevsim/devlink.sh b/tools/testing/selftests/drivers/net/netdevsim/devlink.sh
>index 025a84c2ab5a..32cb2a159c70 100755
>--- a/tools/testing/selftests/drivers/net/netdevsim/devlink.sh
>+++ b/tools/testing/selftests/drivers/net/netdevsim/devlink.sh
>@@ -141,6 +141,16 @@ regions_test()
> 
> 	check_region_snapshot_count dummy post-first-delete 2
> 
>+	devlink region new $DL_HANDLE/dummy snapshot 25
>+	check_err $? "Failed to create a new snapshot with id 25"
>+
>+	check_region_snapshot_count dummy post-first-request 3
>+
>+	devlink region del $DL_HANDLE/dummy snapshot 25
>+	check_err $? "Failed to delete snapshot with id 25"
>+
>+	check_region_snapshot_count dummy post-second-delete 2
>+
> 	log_test "regions test"
> }
> 
>-- 
>2.24.1
>

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 11/11] ice: add a devlink region for dumping NVM contents
  2020-03-26  3:51 ` [net-next v2 11/11] ice: add a devlink region for dumping NVM contents Jacob Keller
@ 2020-03-26  9:02   ` Jiri Pirko
  2020-03-26 16:23     ` Jacob Keller
  0 siblings, 1 reply; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26  9:02 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 04:51:57AM CET, jacob.e.keller@intel.com wrote:
>Add a devlink region for exposing the device's Non Volatime Memory flash
>contents.
>
>Support the recently added .snapshot operation, enabling userspace to
>request a snapshot of the NVM contents via DEVLINK_CMD_REGION_NEW.
>
>Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
>---
> Documentation/networking/devlink/ice.rst     | 26 +++++
> drivers/net/ethernet/intel/ice/ice.h         |  2 +
> drivers/net/ethernet/intel/ice/ice_devlink.c | 99 ++++++++++++++++++++
> drivers/net/ethernet/intel/ice/ice_devlink.h |  3 +
> drivers/net/ethernet/intel/ice/ice_main.c    |  4 +
> 5 files changed, 134 insertions(+)
>
>diff --git a/Documentation/networking/devlink/ice.rst b/Documentation/networking/devlink/ice.rst
>index 37fbbd40a5e5..f3d6a3b50342 100644
>--- a/Documentation/networking/devlink/ice.rst
>+++ b/Documentation/networking/devlink/ice.rst
>@@ -69,3 +69,29 @@ The ``ice`` driver reports the following versions
>       - The version of the DDP package that is active in the device. Note
>         that both the name (as reported by ``fw.app.name``) and version are
>         required to uniquely identify the package.
>+
>+Regions
>+=======
>+
>+The ``ice`` driver enables access to the contents of the Non Volatile Memory
>+flash chip via the ``nvm-flash`` region.
>+
>+Users can request an immediate capture of a snapshot via the
>+``DEVLINK_CMD_REGION_NEW``
>+
>+.. code:: shell
>+
>+    $ devlink region new pci/0000:01:00.0/nvm-flash snapshot 1
>+    $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1
>+
>+    $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1
>+    0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
>+    0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8
>+    0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc
>+    0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5
>+
>+    $ devlink region read pci/0000:01:00.0/nvm-flash snapshot 1 address 0
>+        length 16

Don't wrap the cmdline.


>+    0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
>+
>+    $ devlink region delete pci/0000:01:00.0/nvm-flash snapshot 1
>diff --git a/drivers/net/ethernet/intel/ice/ice.h b/drivers/net/ethernet/intel/ice/ice.h
>index 8ce3afcfeca0..5c11448bfbb3 100644
>--- a/drivers/net/ethernet/intel/ice/ice.h
>+++ b/drivers/net/ethernet/intel/ice/ice.h
>@@ -351,6 +351,8 @@ struct ice_pf {
> 	/* devlink port data */
> 	struct devlink_port devlink_port;
> 
>+	struct devlink_region *nvm_region;
>+
> 	/* OS reserved IRQ details */
> 	struct msix_entry *msix_entries;
> 	struct ice_res_tracker *irq_tracker;
>diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.c b/drivers/net/ethernet/intel/ice/ice_devlink.c
>index 27c5034c039a..91edeffd73b1 100644
>--- a/drivers/net/ethernet/intel/ice/ice_devlink.c
>+++ b/drivers/net/ethernet/intel/ice/ice_devlink.c
>@@ -318,3 +318,102 @@ void ice_devlink_destroy_port(struct ice_pf *pf)
> 	devlink_port_type_clear(&pf->devlink_port);
> 	devlink_port_unregister(&pf->devlink_port);
> }
>+
>+/**
>+ * ice_devlink_nvm_snapshot - Capture a snapshot of the Shadow RAM contents
>+ * @devlink: the devlink instance
>+ * @extack: extended ACK response structure
>+ * @data: on exit points to snapshot data buffer
>+ *
>+ * This function is called in response to the DEVLINK_CMD_REGION_TRIGGER for
>+ * the shadow-ram devlink region. It captures a snapshot of the shadow ram
>+ * contents. This snapshot can later be viewed via the devlink-region
>+ * interface.
>+ *
>+ * @returns zero on success, and updates the data pointer. Returns a non-zero
>+ * error code on failure.
>+ */
>+static int
>+ice_devlink_nvm_snapshot(struct devlink *devlink, struct netlink_ext_ack *extack,
>+			u8 **data)
>+{
>+	struct ice_pf *pf = devlink_priv(devlink);
>+	struct device *dev = ice_pf_to_dev(pf);
>+	struct ice_hw *hw = &pf->hw;
>+	enum ice_status status;
>+	void *nvm_data;
>+	u32 nvm_size;
>+
>+	nvm_size = hw->nvm.flash_size;
>+	nvm_data = vzalloc(nvm_size);
>+	if (!nvm_data) {
>+		NL_SET_ERR_MSG_MOD(extack, "Out of memory");

Pointless extack message. It is obvious from -ENOMEM. Please remove.


>+		return -ENOMEM;
>+	}
>+
>+	status = ice_acquire_nvm(hw, ICE_RES_READ);
>+	if (status) {
>+		dev_dbg(dev, "ice_acquire_nvm failed, err %d aq_err %d\n",
>+			status, hw->adminq.sq_last_status);
>+		NL_SET_ERR_MSG_MOD(extack, "Failed to acquire NVM semaphore");
>+		vfree(nvm_data);
>+		return -EIO;
>+	}
>+
>+	status = ice_read_flat_nvm(hw, 0, &nvm_size, nvm_data, false);
>+	if (status) {
>+		dev_dbg(dev, "ice_read_flat_nvm failed after reading %u bytes, err %d aq_err %d\n",
>+			nvm_size, status, hw->adminq.sq_last_status);
>+		NL_SET_ERR_MSG_MOD(extack, "Failed to read NVM contents");
>+		ice_release_nvm(hw);
>+		vfree(nvm_data);
>+		return -EIO;
>+	}
>+
>+	ice_release_nvm(hw);
>+
>+	*data = nvm_data;
>+
>+	return 0;
>+}
>+
>+static const struct devlink_region_ops ice_nvm_region_ops = {
>+	.name = "nvm-flash",
>+	.destructor = vfree,
>+	.snapshot = ice_devlink_nvm_snapshot,
>+};
>+
>+/**
>+ * ice_devlink_init_regions - Initialize devlink regions
>+ * @pf: the PF device structure
>+ *
>+ * Create devlink regions used to enable access to dump the contents of the
>+ * flash memory on the device.
>+ */
>+void ice_devlink_init_regions(struct ice_pf *pf)
>+{
>+	struct devlink *devlink = priv_to_devlink(pf);
>+	struct device *dev = ice_pf_to_dev(pf);
>+	u64 nvm_size;
>+
>+	nvm_size = pf->hw.nvm.flash_size;
>+	pf->nvm_region = devlink_region_create(devlink, &ice_nvm_region_ops, 1,
>+					       nvm_size);
>+	if (IS_ERR(pf->nvm_region)) {
>+		dev_err(dev, "failed to create NVM devlink region, err %ld\n",
>+			PTR_ERR(pf->nvm_region));
>+		pf->nvm_region = NULL;
>+	}
>+}
>+
>+/**
>+ * ice_devlink_destroy_regions - Destroy devlink regions
>+ * @pf: the PF device structure
>+ *
>+ * Remove previously created regions for this PF.
>+ */
>+void ice_devlink_destroy_regions(struct ice_pf *pf)
>+{
>+	if (pf->nvm_region)
>+		devlink_region_destroy(pf->nvm_region);
>+}
>diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.h b/drivers/net/ethernet/intel/ice/ice_devlink.h
>index f94dc93c24c5..6e806a08dc23 100644
>--- a/drivers/net/ethernet/intel/ice/ice_devlink.h
>+++ b/drivers/net/ethernet/intel/ice/ice_devlink.h
>@@ -11,4 +11,7 @@ void ice_devlink_unregister(struct ice_pf *pf);
> int ice_devlink_create_port(struct ice_pf *pf);
> void ice_devlink_destroy_port(struct ice_pf *pf);
> 
>+void ice_devlink_init_regions(struct ice_pf *pf);
>+void ice_devlink_destroy_regions(struct ice_pf *pf);
>+
> #endif /* _ICE_DEVLINK_H_ */
>diff --git a/drivers/net/ethernet/intel/ice/ice_main.c b/drivers/net/ethernet/intel/ice/ice_main.c
>index 359ff8544773..306a4e5b2320 100644
>--- a/drivers/net/ethernet/intel/ice/ice_main.c
>+++ b/drivers/net/ethernet/intel/ice/ice_main.c
>@@ -3276,6 +3276,8 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
> 		goto err_init_pf_unroll;
> 	}
> 
>+	ice_devlink_init_regions(pf);
>+
> 	pf->num_alloc_vsi = hw->func_caps.guar_num_vsi;
> 	if (!pf->num_alloc_vsi) {
> 		err = -EIO;
>@@ -3385,6 +3387,7 @@ ice_probe(struct pci_dev *pdev, const struct pci_device_id __always_unused *ent)
> 	devm_kfree(dev, pf->vsi);
> err_init_pf_unroll:
> 	ice_deinit_pf(pf);
>+	ice_devlink_destroy_regions(pf);
> 	ice_deinit_hw(hw);
> err_exit_unroll:
> 	ice_devlink_unregister(pf);
>@@ -3427,6 +3430,7 @@ static void ice_remove(struct pci_dev *pdev)
> 		ice_vsi_free_q_vectors(pf->vsi[i]);
> 	}
> 	ice_deinit_pf(pf);
>+	ice_devlink_destroy_regions(pf);
> 	ice_deinit_hw(&pf->hw);
> 	ice_devlink_unregister(pf);
> 
>-- 
>2.24.1
>

^ permalink raw reply	[flat|nested] 33+ messages in thread

* RE: [net-next v2 07/11] devlink: report error once U32_MAX snapshot ids have been used
  2020-03-26  7:30   ` Jiri Pirko
@ 2020-03-26 16:09     ` Keller, Jacob E
  0 siblings, 0 replies; 33+ messages in thread
From: Keller, Jacob E @ 2020-03-26 16:09 UTC (permalink / raw)
  To: Jiri Pirko; +Cc: netdev, Jakub Kicinski



> -----Original Message-----
> From: netdev-owner@vger.kernel.org <netdev-owner@vger.kernel.org> On
> Behalf Of Jiri Pirko
> Sent: Thursday, March 26, 2020 12:31 AM
> To: Keller, Jacob E <jacob.e.keller@intel.com>
> Cc: netdev@vger.kernel.org; Jakub Kicinski <kuba@kernel.org>
> Subject: Re: [net-next v2 07/11] devlink: report error once U32_MAX snapshot
> ids have been used
> 
> Thu, Mar 26, 2020 at 04:51:53AM CET, jacob.e.keller@intel.com wrote:
> >The devlink_snapshot_id_get() function returns a snapshot id. The
> >snapshot id is a u32, so there is no way to indicate an error code.
> >
> >Indeed, the two current callers of devlink_snapshot_id_get() assume that
> >a negative value is an error.
> 
> I don't see they do.
> 

You're right, they don't. I was confused when I wrote this because the previous version of this patch did change them to check for error, and then I realized we didn't need that patch and removed it, but forgot to update this.

Can fix this if we need a respin for another reason.

> 
> >
> >A future change is going to possibly add additional cases where this
> >function could fail. Refactor the function to return the snapshot id in
> >an argument, so that it can return zero or an error value.
> >
> >This ensures that snapshot ids cannot be confused with error values, and
> >aids in the future refactor of snapshot id allocation management.
> >
> >Because there is no current way to release previously used snapshot ids,
> >add a simple check ensuring that an error is reported in case the
> >snapshot_id would over flow.
> >
> >Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> 
> The code looks good.
> 
> Reviewed-by: Jiri Pirko <jiri@mellanox.com>

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW
  2020-03-26  7:51 ` Jiri Pirko
@ 2020-03-26 16:15   ` Jacob Keller
  2020-03-26 21:12     ` Jiri Pirko
  0 siblings, 1 reply; 33+ messages in thread
From: Jacob Keller @ 2020-03-26 16:15 UTC (permalink / raw)
  To: Jiri Pirko; +Cc: netdev, Jakub Kicinski



On 3/26/2020 12:51 AM, Jiri Pirko wrote:
>> Changes to patches since v1:
>>
>> * devlink: prepare to support region operations
>>
>>   No changes
>>
>> * devlink: convert snapshot destructor callback to region op
>>
>>   No changes
>>
>> * devlink: trivial: fix tab in function documentation
>>
>>   No changes
>>
>> * devlink: add function to take snapshot while locked
>>
>>   Added Jakub's Reviewed-by tag.
>>
>> * <NEW> devlink: use -ENOSPC to indicate no more room for snapshots
>>
>>   New patch added to convert confusing -ENOMEM to -ENOSPC, as suggested by
>>   Jiri.
>>
>> * devlink: extract snapshot id allocation to helper function
>>
>>   No changes
>>
>> * devlink: convert snapshot id getter to return an error
>>
>>   Changed title to "devlink: report error once U32_MAX snapshot ids have
>>   been used".
>>
>>   Refactored this patch to make devlink_region_snapshot_id_get take a
>>   pointer to u32, so that the error value and id value are separated. This
>>   means that we can remove the INT_MAX limitation on id values.
>>
>> * devlink: track snapshot id usage count using an xarray
>>
>>   Fixed the xa_init to use xa_init_flags with XA_FLAGS_ALLOC, so that
>>   xa_alloc can properly be used.
>>
>>   Changed devlink_region_snapshot_id_get to use an initial count of 1
>>   instead of 0. Added a new devlink_region_snapshot_id_put function, used
>>   to release this initial count. This closes the race condition and issues
>>   caused if the driver either doesn't create a snapshot, or if userspace
>>   deletes the first snapshot before others are created.
>>
>>   Used WARN_ON in a few more checks that should not occur, such as if the
>>   xarray entry is not a value, or when the id isn't yet in the xarray.
>>
>>   Removed an unnecessary if (err) { return err; } construction.
>>
>>   Use xa_limit_32b instead of xa_limit_31b now that we don't return the
>>   snapshot id directly.
>>
>>   Cleanup the label used in __devlink_region_snapshot_create to indicate the
>>   failure cause, rather than the cleanup step.
>>
>>   Removed the unnecessary locking around xa_destroy
>>
>> * devlink: implement DEVLINK_CMD_REGION_NEW
>>
>>   Added a WARN_ON to the check in snapshot_id_insert in case the id already
>>   exists.
>>
>>   Removed an unnecessary "if (err) { return err; }" construction
>>
>>   Use -ENOSPC instead of -ENOMEM when max_snapshots is reached.
>>
>>   Cleanup label names to match style of the other labels in the file,
>>   naming after the failure cause rather than the cleanup step. Also fix a
>>   bug in the label ordering.
>>
>>   Call the new devlink_region_snapshot_id_put function in the mlx4 and
>>   netdevsim drivers.
>>
>> * netdevsim: support taking immediate snapshot via devlink
>>
>>   Create a local devlink pointer instead of calling priv_to_devlink
>>   multiple times.
>>
>>   Removed previous selftest for devlink region new without a snapshot id,
>>   as this is no longer supported. Adjusted and verified that the tests pass
>>   now.
>>
>> * ice: add a devlink region for dumping NVM contents
>>
>>   Use "dev_err" instead of "dev_warn" for a message about failure to create
>>   the devlink region.
> 
> Could you please have the changelog per-patch, as I suggested for v1?
> Much easier to review then.

What do you mean? I already broke this into a description of the change
for each patch, I thought that's what you wanted..

Do you want me to move this into the individual commit emails?

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 09/11] devlink: implement DEVLINK_CMD_REGION_NEW
  2020-03-26  8:52   ` Jiri Pirko
@ 2020-03-26 16:19     ` Jacob Keller
  2020-03-26 21:10       ` Jiri Pirko
  0 siblings, 1 reply; 33+ messages in thread
From: Jacob Keller @ 2020-03-26 16:19 UTC (permalink / raw)
  To: Jiri Pirko; +Cc: netdev, Jakub Kicinski



On 3/26/2020 1:52 AM, Jiri Pirko wrote:
> Thu, Mar 26, 2020 at 04:51:55AM CET, jacob.e.keller@intel.com wrote:
> 
> [...]
> 
>> +	err = __devlink_snapshot_id_insert(devlink, snapshot_id);
>> +	if (err) {
>> +		return err;
>> +	}
>> +
>> +	err = region->ops->snapshot(devlink, info->extack, &data);
>> +	if (err)
>> +		goto snapshot_capture_failure;
>> +
>> +	err = __devlink_region_snapshot_create(region, data, snapshot_id);
>> +	if (err)
>> +		goto snapshot_create_failure;
>> +
>> +	return 0;
>> +
>> +snapshot_create_failure:
>> +	region->ops->destructor(data);
>> +snapshot_capture_failure:
> 
> Eh, this actually should be "err_snapshot_capture" and
> "err_snapshot_create"
> 

Sure. It seems a lot of functions use "out" or "nla_put_failure", or
other styles.

$grep "^[A-Za-z0-9_]*:" net/core/devlink.c | sort | uniq -c | less
      1 dump_err:
      1 err_action_value_put:
      1 err_action_values_put:
      1 err_cancel_msg:
      1 err_group_init:
      1 err_group_link:
      1 err_match_value_put:
      1 err_match_values_put:
      1 err_resource_put:
      2 err_stats_alloc:
      2 err_table_put:
      1 err_trap_fill:
      1 err_trap_group_fill:
      1 err_trap_group_register:
      1 err_trap_group_verify:
      1 err_trap_init:
      1 err_trap_register:
      1 err_trap_verify:
      1 free_msg:
      2 genlmsg_cancel:
      1 id_increment_failure:
     34 nla_put_failure:
      1 nla_put_failure_type_locked:
     27 out:
      1 out_cancel_msg:
      1 out_dev:
      2 out_free_msg:
      1 out_unlock:
      1 param_nest_cancel:
      1 reporter_nest_cancel:
      1 resource_put_failure:
      1 rollback:
      4 send_done:
      1 snapshot_capture_failure:
      1 snapshot_create_failure:
      3 start_again:
      9 unlock:
      1 value_nest_cancel:
      1 values_list_nest_cancel:

But I'll change these.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 11/11] ice: add a devlink region for dumping NVM contents
  2020-03-26  9:02   ` Jiri Pirko
@ 2020-03-26 16:23     ` Jacob Keller
  2020-03-26 21:11       ` Jiri Pirko
  0 siblings, 1 reply; 33+ messages in thread
From: Jacob Keller @ 2020-03-26 16:23 UTC (permalink / raw)
  To: Jiri Pirko; +Cc: netdev, Jakub Kicinski



On 3/26/2020 2:02 AM, Jiri Pirko wrote:
> Thu, Mar 26, 2020 at 04:51:57AM CET, jacob.e.keller@intel.com wrote:
>> +
>> +Regions
>> +=======
>> +
>> +The ``ice`` driver enables access to the contents of the Non Volatile Memory
>> +flash chip via the ``nvm-flash`` region.
>> +
>> +Users can request an immediate capture of a snapshot via the
>> +``DEVLINK_CMD_REGION_NEW``
>> +
>> +.. code:: shell
>> +
>> +    $ devlink region new pci/0000:01:00.0/nvm-flash snapshot 1
>> +    $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1
>> +
>> +    $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1
>> +    0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
>> +    0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8
>> +    0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc
>> +    0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5
>> +
>> +    $ devlink region read pci/0000:01:00.0/nvm-flash snapshot 1 address 0
>> +        length 16
> 
> Don't wrap the cmdline.

The devlink-region.rst file wrapped it like this..

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW
  2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
                   ` (14 preceding siblings ...)
  2020-03-26  8:54 ` Jiri Pirko
@ 2020-03-26 18:27 ` David Miller
  2020-03-26 18:29   ` Jacob Keller
  15 siblings, 1 reply; 33+ messages in thread
From: David Miller @ 2020-03-26 18:27 UTC (permalink / raw)
  To: jacob.e.keller; +Cc: netdev, kuba, jiri

From: Jacob Keller <jacob.e.keller@intel.com>
Date: Wed, 25 Mar 2020 20:51:46 -0700

> This is a second revision of the previous series to implement the
> DEVLINK_CMD_REGION_NEW. The series can be viewed on lore.kernel.org at
> 
> https://lore.kernel.org/netdev/20200324223445.2077900-1-jacob.e.keller@intel.com/
> 
> This version includes the suggested cleanups from Jakub and Jiri on the
> list, including the following changes, broken out by the v1 patch title.
 ...

Based upon the review so far I'm expecting one more respin of this.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW
  2020-03-26 18:27 ` David Miller
@ 2020-03-26 18:29   ` Jacob Keller
  0 siblings, 0 replies; 33+ messages in thread
From: Jacob Keller @ 2020-03-26 18:29 UTC (permalink / raw)
  To: David Miller; +Cc: netdev, kuba, jiri



On 3/26/2020 11:27 AM, David Miller wrote:
> From: Jacob Keller <jacob.e.keller@intel.com>
> Date: Wed, 25 Mar 2020 20:51:46 -0700
> 
>> This is a second revision of the previous series to implement the
>> DEVLINK_CMD_REGION_NEW. The series can be viewed on lore.kernel.org at
>>
>> https://lore.kernel.org/netdev/20200324223445.2077900-1-jacob.e.keller@intel.com/
>>
>> This version includes the suggested cleanups from Jakub and Jiri on the
>> list, including the following changes, broken out by the v1 patch title.
>  ...
> 
> Based upon the review so far I'm expecting one more respin of this.
> 

Yep, I'm about to send a v3.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 09/11] devlink: implement DEVLINK_CMD_REGION_NEW
  2020-03-26 16:19     ` Jacob Keller
@ 2020-03-26 21:10       ` Jiri Pirko
  0 siblings, 0 replies; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26 21:10 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 05:19:30PM CET, jacob.e.keller@intel.com wrote:
>
>
>On 3/26/2020 1:52 AM, Jiri Pirko wrote:
>> Thu, Mar 26, 2020 at 04:51:55AM CET, jacob.e.keller@intel.com wrote:
>> 
>> [...]
>> 
>>> +	err = __devlink_snapshot_id_insert(devlink, snapshot_id);
>>> +	if (err) {
>>> +		return err;
>>> +	}
>>> +
>>> +	err = region->ops->snapshot(devlink, info->extack, &data);
>>> +	if (err)
>>> +		goto snapshot_capture_failure;
>>> +
>>> +	err = __devlink_region_snapshot_create(region, data, snapshot_id);
>>> +	if (err)
>>> +		goto snapshot_create_failure;
>>> +
>>> +	return 0;
>>> +
>>> +snapshot_create_failure:
>>> +	region->ops->destructor(data);
>>> +snapshot_capture_failure:
>> 
>> Eh, this actually should be "err_snapshot_capture" and
>> "err_snapshot_create"
>> 
>
>Sure. It seems a lot of functions use "out" or "nla_put_failure", or

yeah, "nla_put_failure" is a special kind that is used through the net
code.


>other styles.
>
>$grep "^[A-Za-z0-9_]*:" net/core/devlink.c | sort | uniq -c | less
>      1 dump_err:
>      1 err_action_value_put:
>      1 err_action_values_put:
>      1 err_cancel_msg:
>      1 err_group_init:
>      1 err_group_link:
>      1 err_match_value_put:
>      1 err_match_values_put:
>      1 err_resource_put:
>      2 err_stats_alloc:
>      2 err_table_put:
>      1 err_trap_fill:
>      1 err_trap_group_fill:
>      1 err_trap_group_register:
>      1 err_trap_group_verify:
>      1 err_trap_init:
>      1 err_trap_register:
>      1 err_trap_verify:
>      1 free_msg:
>      2 genlmsg_cancel:
>      1 id_increment_failure:
>     34 nla_put_failure:
>      1 nla_put_failure_type_locked:
>     27 out:
>      1 out_cancel_msg:
>      1 out_dev:
>      2 out_free_msg:
>      1 out_unlock:
>      1 param_nest_cancel:
>      1 reporter_nest_cancel:
>      1 resource_put_failure:
>      1 rollback:
>      4 send_done:
>      1 snapshot_capture_failure:
>      1 snapshot_create_failure:
>      3 start_again:
>      9 unlock:
>      1 value_nest_cancel:
>      1 values_list_nest_cancel:
>
>But I'll change these.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 11/11] ice: add a devlink region for dumping NVM contents
  2020-03-26 16:23     ` Jacob Keller
@ 2020-03-26 21:11       ` Jiri Pirko
  0 siblings, 0 replies; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26 21:11 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 05:23:39PM CET, jacob.e.keller@intel.com wrote:
>
>
>On 3/26/2020 2:02 AM, Jiri Pirko wrote:
>> Thu, Mar 26, 2020 at 04:51:57AM CET, jacob.e.keller@intel.com wrote:
>>> +
>>> +Regions
>>> +=======
>>> +
>>> +The ``ice`` driver enables access to the contents of the Non Volatile Memory
>>> +flash chip via the ``nvm-flash`` region.
>>> +
>>> +Users can request an immediate capture of a snapshot via the
>>> +``DEVLINK_CMD_REGION_NEW``
>>> +
>>> +.. code:: shell
>>> +
>>> +    $ devlink region new pci/0000:01:00.0/nvm-flash snapshot 1
>>> +    $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1
>>> +
>>> +    $ devlink region dump pci/0000:01:00.0/nvm-flash snapshot 1
>>> +    0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
>>> +    0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8
>>> +    0000000000000020 0016 0bb8 0016 1720 0000 0000 c00f 3ffc
>>> +    0000000000000030 bada cce5 bada cce5 bada cce5 bada cce5
>>> +
>>> +    $ devlink region read pci/0000:01:00.0/nvm-flash snapshot 1 address 0
>>> +        length 16
>> 
>> Don't wrap the cmdline.
>
>The devlink-region.rst file wrapped it like this..

That is quite odd. I don't think it is correct.

^ permalink raw reply	[flat|nested] 33+ messages in thread

* Re: [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW
  2020-03-26 16:15   ` Jacob Keller
@ 2020-03-26 21:12     ` Jiri Pirko
  0 siblings, 0 replies; 33+ messages in thread
From: Jiri Pirko @ 2020-03-26 21:12 UTC (permalink / raw)
  To: Jacob Keller; +Cc: netdev, Jakub Kicinski

Thu, Mar 26, 2020 at 05:15:05PM CET, jacob.e.keller@intel.com wrote:
>
>
>On 3/26/2020 12:51 AM, Jiri Pirko wrote:
>>> Changes to patches since v1:
>>>
>>> * devlink: prepare to support region operations
>>>
>>>   No changes
>>>
>>> * devlink: convert snapshot destructor callback to region op
>>>
>>>   No changes
>>>
>>> * devlink: trivial: fix tab in function documentation
>>>
>>>   No changes
>>>
>>> * devlink: add function to take snapshot while locked
>>>
>>>   Added Jakub's Reviewed-by tag.
>>>
>>> * <NEW> devlink: use -ENOSPC to indicate no more room for snapshots
>>>
>>>   New patch added to convert confusing -ENOMEM to -ENOSPC, as suggested by
>>>   Jiri.
>>>
>>> * devlink: extract snapshot id allocation to helper function
>>>
>>>   No changes
>>>
>>> * devlink: convert snapshot id getter to return an error
>>>
>>>   Changed title to "devlink: report error once U32_MAX snapshot ids have
>>>   been used".
>>>
>>>   Refactored this patch to make devlink_region_snapshot_id_get take a
>>>   pointer to u32, so that the error value and id value are separated. This
>>>   means that we can remove the INT_MAX limitation on id values.
>>>
>>> * devlink: track snapshot id usage count using an xarray
>>>
>>>   Fixed the xa_init to use xa_init_flags with XA_FLAGS_ALLOC, so that
>>>   xa_alloc can properly be used.
>>>
>>>   Changed devlink_region_snapshot_id_get to use an initial count of 1
>>>   instead of 0. Added a new devlink_region_snapshot_id_put function, used
>>>   to release this initial count. This closes the race condition and issues
>>>   caused if the driver either doesn't create a snapshot, or if userspace
>>>   deletes the first snapshot before others are created.
>>>
>>>   Used WARN_ON in a few more checks that should not occur, such as if the
>>>   xarray entry is not a value, or when the id isn't yet in the xarray.
>>>
>>>   Removed an unnecessary if (err) { return err; } construction.
>>>
>>>   Use xa_limit_32b instead of xa_limit_31b now that we don't return the
>>>   snapshot id directly.
>>>
>>>   Cleanup the label used in __devlink_region_snapshot_create to indicate the
>>>   failure cause, rather than the cleanup step.
>>>
>>>   Removed the unnecessary locking around xa_destroy
>>>
>>> * devlink: implement DEVLINK_CMD_REGION_NEW
>>>
>>>   Added a WARN_ON to the check in snapshot_id_insert in case the id already
>>>   exists.
>>>
>>>   Removed an unnecessary "if (err) { return err; }" construction
>>>
>>>   Use -ENOSPC instead of -ENOMEM when max_snapshots is reached.
>>>
>>>   Cleanup label names to match style of the other labels in the file,
>>>   naming after the failure cause rather than the cleanup step. Also fix a
>>>   bug in the label ordering.
>>>
>>>   Call the new devlink_region_snapshot_id_put function in the mlx4 and
>>>   netdevsim drivers.
>>>
>>> * netdevsim: support taking immediate snapshot via devlink
>>>
>>>   Create a local devlink pointer instead of calling priv_to_devlink
>>>   multiple times.
>>>
>>>   Removed previous selftest for devlink region new without a snapshot id,
>>>   as this is no longer supported. Adjusted and verified that the tests pass
>>>   now.
>>>
>>> * ice: add a devlink region for dumping NVM contents
>>>
>>>   Use "dev_err" instead of "dev_warn" for a message about failure to create
>>>   the devlink region.
>> 
>> Could you please have the changelog per-patch, as I suggested for v1?
>> Much easier to review then.
>
>What do you mean? I already broke this into a description of the change
>for each patch, I thought that's what you wanted..
>
>Do you want me to move this into the individual commit emails?

Yes, that is what I mean. Much more convenient in my opinion.

^ permalink raw reply	[flat|nested] 33+ messages in thread

end of thread, back to index

Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-26  3:51 [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jacob Keller
2020-03-26  3:51 ` [net-next v2 01/11] devlink: prepare to support region operations Jacob Keller
2020-03-26  3:51 ` [net-next v2 02/11] devlink: convert snapshot destructor callback to region op Jacob Keller
2020-03-26  3:51 ` [net-next v2 03/11] devlink: trivial: fix tab in function documentation Jacob Keller
2020-03-26  3:51 ` [net-next v2 04/11] devlink: add function to take snapshot while locked Jacob Keller
2020-03-26  3:51 ` [net-next v2 05/11] devlink: use -ENOSPC to indicate no more room for snapshots Jacob Keller
2020-03-26  7:24   ` Jiri Pirko
2020-03-26  3:51 ` [net-next v2 06/11] devlink: extract snapshot id allocation to helper function Jacob Keller
2020-03-26  7:25   ` Jiri Pirko
2020-03-26  3:51 ` [net-next v2 07/11] devlink: report error once U32_MAX snapshot ids have been used Jacob Keller
2020-03-26  7:30   ` Jiri Pirko
2020-03-26 16:09     ` Keller, Jacob E
2020-03-26  3:51 ` [net-next v2 08/11] devlink: track snapshot id usage count using an xarray Jacob Keller
2020-03-26  8:35   ` Jiri Pirko
2020-03-26  3:51 ` [net-next v2 09/11] devlink: implement DEVLINK_CMD_REGION_NEW Jacob Keller
2020-03-26  8:51   ` Jiri Pirko
2020-03-26  8:52   ` Jiri Pirko
2020-03-26 16:19     ` Jacob Keller
2020-03-26 21:10       ` Jiri Pirko
2020-03-26  3:51 ` [net-next v2 10/11] netdevsim: support taking immediate snapshot via devlink Jacob Keller
2020-03-26  9:00   ` Jiri Pirko
2020-03-26  3:51 ` [net-next v2 11/11] ice: add a devlink region for dumping NVM contents Jacob Keller
2020-03-26  9:02   ` Jiri Pirko
2020-03-26 16:23     ` Jacob Keller
2020-03-26 21:11       ` Jiri Pirko
2020-03-26  7:23 ` [net-next v2 00/11] implement DEVLINK_CMD_REGION_NEW Jiri Pirko
2020-03-26  7:51 ` Jiri Pirko
2020-03-26 16:15   ` Jacob Keller
2020-03-26 21:12     ` Jiri Pirko
2020-03-26  8:36 ` Jiri Pirko
2020-03-26  8:54 ` Jiri Pirko
2020-03-26 18:27 ` David Miller
2020-03-26 18:29   ` Jacob Keller

Netdev Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/netdev/0 netdev/git/0.git
	git clone --mirror https://lore.kernel.org/netdev/1 netdev/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 netdev netdev/ https://lore.kernel.org/netdev \
		netdev@vger.kernel.org
	public-inbox-index netdev

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.netdev


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git