[PATCH net-next 0/8] devlink: support direct read from region

[PATCH net-next 0/8] devlink: support direct read from region

[Jacob Keller]

A long time ago when initially implementing devlink regions in ice I
proposed the ability to allow reading from a region without taking a
snapshot [1]. I eventually dropped this work from the original series due to
size. Then I eventually lost track of submitting this follow up.

This can be useful when interacting with some region that has some
definitive "contents" from which snapshots are made. For example the ice
driver has regions representing the contents of the device flash.

If userspace wants to read the contents today, it must first take a snapshot
and then read from that snapshot. This makes sense if you want to read a
large portion of data or you want to be sure reads are consistently from the
same recording of the flash.

However if user space only wants to read a small chunk, it must first
generate a snapshot of the entire contents, perform a read from the
snapshot, and then delete the snapshot after reading.

For such a use case, a direct read from the region makes more sense. This
can be achieved by allowing the devlink region read command to work without
a snapshot. Instead the portion to be read can be forwarded directly to the
driver via a new .read callback.

This avoids the need to read the entire region contents into memory first
and avoids the software overhead of creating a snapshot and then deleting
it.

This series implements such behavior and hooks up the ice NVM and shadow RAM
regions to allow it.

[1] https://lore.kernel.org/netdev/20200130225913.1671982-1-jacob.e.keller@intel.com/

Cc: Jiri Pirko <jiri@nvidia.com>
Cc: Jakub Kicinski <kuba@kernel.org>

Jacob Keller (8):
  devlink: find snapshot in devlink_nl_cmd_region_read_dumpit
  devlink: use min_t to calculate data_size
  devlink: report extended error message in region_read_dumpit
  devlink: remove unnecessary parameter from chunk_fill function
  devlink: refactor region_read_snapshot_fill to use a callback function
  devlink: support directly reading from region memory
  ice: use same function to snapshot both NVM and Shadow RAM
  ice: implement direct read for NVM and Shadow RAM regions

 .../networking/devlink/devlink-region.rst     |   8 ++
 drivers/net/ethernet/intel/ice/ice_devlink.c  | 112 +++++++++-------
 include/net/devlink.h                         |  16 +++
 net/core/devlink.c                            | 121 +++++++++++++-----
 4 files changed, 180 insertions(+), 77 deletions(-)


base-commit: b4b221bd79a1c698d9653e3ae2c3cb61cdc9aee7
-- 
2.38.1.420.g319605f8f00e

[PATCH net-next 1/8] devlink: find snapshot in devlink_nl_cmd_region_read_dumpit

[Jacob Keller]

The snapshot pointer is obtained inside of the function
devlink_nl_region_read_snapshot_fill. Simplify this function by locating
the snapshot upfront in devlink_nl_cmd_region_read_dumpit instead. This
aligns with how other netlink attributes are handled, and allows us to
exit slightly earlier if an invalid snapshot ID is provided.

It also allows us to pass the snapshot pointer directly to the
devlink_nl_region_read_snapshot_fill, and remove the now unused attrs
parameter.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 net/core/devlink.c | 21 +++++++++++----------
 1 file changed, 11 insertions(+), 10 deletions(-)

diff --git a/net/core/devlink.c b/net/core/devlink.c
index 7f789bbcbbd7..96afc7013959 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -6399,24 +6399,16 @@ static int devlink_nl_cmd_region_read_chunk_fill(struct sk_buff *msg,
 
 static int devlink_nl_region_read_snapshot_fill(struct sk_buff *skb,
 						struct devlink *devlink,
-						struct devlink_region *region,
-						struct nlattr **attrs,
+						struct devlink_snapshot *snapshot,
 						u64 start_offset,
 						u64 end_offset,
 						u64 *new_offset)
 {
-	struct devlink_snapshot *snapshot;
 	u64 curr_offset = start_offset;
-	u32 snapshot_id;
 	int err = 0;
 
 	*new_offset = start_offset;
 
-	snapshot_id = nla_get_u32(attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]);
-	snapshot = devlink_region_snapshot_get_by_id(region, snapshot_id);
-	if (!snapshot)
-		return -EINVAL;
-
 	while (curr_offset < end_offset) {
 		u32 data_size;
 		u8 *data;
@@ -6447,11 +6439,13 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 	u64 ret_offset, start_offset, end_offset = U64_MAX;
 	struct nlattr **attrs = info->attrs;
 	struct devlink_port *port = NULL;
+	struct devlink_snapshot *snapshot;
 	struct devlink_region *region;
 	struct nlattr *chunks_attr;
 	const char *region_name;
 	struct devlink *devlink;
 	unsigned int index;
+	u32 snapshot_id;
 	void *hdr;
 	int err;
 
@@ -6491,6 +6485,13 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 		goto out_unlock;
 	}
 
+	snapshot_id = nla_get_u32(attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]);
+	snapshot = devlink_region_snapshot_get_by_id(region, snapshot_id);
+	if (!snapshot) {
+		err = -EINVAL;
+		goto out_unlock;
+	}
+
 	if (attrs[DEVLINK_ATTR_REGION_CHUNK_ADDR] &&
 	    attrs[DEVLINK_ATTR_REGION_CHUNK_LEN]) {
 		if (!start_offset)
@@ -6540,7 +6541,7 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 	}
 
 	err = devlink_nl_region_read_snapshot_fill(skb, devlink,
-						   region, attrs,
+						   snapshot,
 						   start_offset,
 						   end_offset, &ret_offset);
 
-- 
2.38.1.420.g319605f8f00e

[PATCH net-next 2/8] devlink: use min_t to calculate data_size

[Jacob Keller]

The calculation for the data_size in the devlink_nl_read_snapshot_fill
function uses an if statement that is better expressed using the min_t
macro.

Noticed-by: Jakub Kicinski <kuba@kernel.org>
Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 net/core/devlink.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/net/core/devlink.c b/net/core/devlink.c
index 96afc7013959..932476956d7e 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -6410,14 +6410,10 @@ static int devlink_nl_region_read_snapshot_fill(struct sk_buff *skb,
 	*new_offset = start_offset;
 
 	while (curr_offset < end_offset) {
-		u32 data_size;
+		u32 data_size = min_t(u32, end_offset - curr_offset,
+				      DEVLINK_REGION_READ_CHUNK_SIZE);
 		u8 *data;
 
-		if (end_offset - curr_offset < DEVLINK_REGION_READ_CHUNK_SIZE)
-			data_size = end_offset - curr_offset;
-		else
-			data_size = DEVLINK_REGION_READ_CHUNK_SIZE;
-
 		data = &snapshot->data[curr_offset];
 		err = devlink_nl_cmd_region_read_chunk_fill(skb, devlink,
 							    data, data_size,
-- 
2.38.1.420.g319605f8f00e

Re: [PATCH net-next 2/8] devlink: use min_t to calculate data_size

[Jakub Kicinski]

On Thu, 17 Nov 2022 14:07:57 -0800 Jacob Keller wrote:
> The calculation for the data_size in the devlink_nl_read_snapshot_fill
> function uses an if statement that is better expressed using the min_t
> macro.
> 
> Noticed-by: Jakub Kicinski <kuba@kernel.org>

I'm afraid that's not a real tag. You can just drop it, 
I get sufficient credits.

> diff --git a/net/core/devlink.c b/net/core/devlink.c
> index 96afc7013959..932476956d7e 100644
> --- a/net/core/devlink.c
> +++ b/net/core/devlink.c
> @@ -6410,14 +6410,10 @@ static int devlink_nl_region_read_snapshot_fill(struct sk_buff *skb,
>  	*new_offset = start_offset;
>  
>  	while (curr_offset < end_offset) {
> -		u32 data_size;
> +		u32 data_size = min_t(u32, end_offset - curr_offset,
> +				      DEVLINK_REGION_READ_CHUNK_SIZE);

nit: don't put multi-line statements on the declaration line if it's 
not the only variable.

>  		u8 *data;
>  
> -		if (end_offset - curr_offset < DEVLINK_REGION_READ_CHUNK_SIZE)
> -			data_size = end_offset - curr_offset;
> -		else
> -			data_size = DEVLINK_REGION_READ_CHUNK_SIZE;

Re: [PATCH net-next 2/8] devlink: use min_t to calculate data_size

[Jacob Keller]

On 11/18/2022 5:36 PM, Jakub Kicinski wrote:
> On Thu, 17 Nov 2022 14:07:57 -0800 Jacob Keller wrote:
>> The calculation for the data_size in the devlink_nl_read_snapshot_fill
>> function uses an if statement that is better expressed using the min_t
>> macro.
>>
>> Noticed-by: Jakub Kicinski <kuba@kernel.org>
> 
> I'm afraid that's not a real tag. You can just drop it,
> I get sufficient credits.
> 

Sure. I pulled this forward from some time ago, not sure why I had added 
it back then. A grep through the log shows its been used a handful of 
times, but it likely just slipped in without review in the past. Will 
remove.

>> diff --git a/net/core/devlink.c b/net/core/devlink.c
>> index 96afc7013959..932476956d7e 100644
>> --- a/net/core/devlink.c
>> +++ b/net/core/devlink.c
>> @@ -6410,14 +6410,10 @@ static int devlink_nl_region_read_snapshot_fill(struct sk_buff *skb,
>>   	*new_offset = start_offset;
>>   
>>   	while (curr_offset < end_offset) {
>> -		u32 data_size;
>> +		u32 data_size = min_t(u32, end_offset - curr_offset,
>> +				      DEVLINK_REGION_READ_CHUNK_SIZE);
> 
> nit: don't put multi-line statements on the declaration line if it's
> not the only variable.
> 

Sure, that makes sense.

>>   		u8 *data;
>>   
>> -		if (end_offset - curr_offset < DEVLINK_REGION_READ_CHUNK_SIZE)
>> -			data_size = end_offset - curr_offset;
>> -		else
>> -			data_size = DEVLINK_REGION_READ_CHUNK_SIZE;

Re: [PATCH net-next 2/8] devlink: use min_t to calculate data_size

[Jacob Keller]

On 11/21/2022 9:51 AM, Jacob Keller wrote:
> On 11/18/2022 5:36 PM, Jakub Kicinski wrote:
>>> diff --git a/net/core/devlink.c b/net/core/devlink.c
>>> index 96afc7013959..932476956d7e 100644
>>> --- a/net/core/devlink.c
>>> +++ b/net/core/devlink.c
>>> @@ -6410,14 +6410,10 @@ static int 
>>> devlink_nl_region_read_snapshot_fill(struct sk_buff *skb,
>>>       *new_offset = start_offset;
>>>       while (curr_offset < end_offset) {
>>> -        u32 data_size;
>>> +        u32 data_size = min_t(u32, end_offset - curr_offset,
>>> +                      DEVLINK_REGION_READ_CHUNK_SIZE);
>>
>> nit: don't put multi-line statements on the declaration line if it's
>> not the only variable.
>>
> 
> Sure, that makes sense.
> 

This becomes the only variable in patch 5 of 8. It ends up making the 
diff look more complicated if I change it back to a combined 
declare+assign in that patch.

Re: [PATCH net-next 2/8] devlink: use min_t to calculate data_size

[Jakub Kicinski]

On Mon, 21 Nov 2022 10:35:34 -0800 Jacob Keller wrote:
> > Sure, that makes sense.
> 
> This becomes the only variable in patch 5 of 8. It ends up making the 
> diff look more complicated if I change it back to a combined 
> declare+assign in that patch.

Don't change it back to declare+assign, then? :)
In general declare+assign should be used sparingly IMO.
My eyes are trained to skip right past the variable declarations,
the goal is to make the code clear, not short :S

BTW you can probably make DEVLINK_REGION_READ_CHUNK_SIZE a ULL to switch
from min_t() to min() ?

Re: [PATCH net-next 2/8] devlink: use min_t to calculate data_size

[Jacob Keller]

On 11/21/2022 11:06 AM, Jakub Kicinski wrote:
> On Mon, 21 Nov 2022 10:35:34 -0800 Jacob Keller wrote:
>>> Sure, that makes sense.
>>
>> This becomes the only variable in patch 5 of 8. It ends up making the
>> diff look more complicated if I change it back to a combined
>> declare+assign in that patch.
> 
> Don't change it back to declare+assign, then? :)
> In general declare+assign should be used sparingly IMO.
> My eyes are trained to skip right past the variable declarations,
> the goal is to make the code clear, not short :S
> 
> BTW you can probably make DEVLINK_REGION_READ_CHUNK_SIZE a ULL to switch
> from min_t() to min() ?

Fair enough. It looked a bit weird when it was:

u32 data_size;

data_size = ...

But I think thats ok.

Thanks,
Jake

[PATCH net-next 3/8] devlink: report extended error message in region_read_dumpit

[Jacob Keller]

Report extended error details in the devlink_nl_cmd_region_read_dumpit
function, by using the extack structure from the netlink_callback.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 net/core/devlink.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/net/core/devlink.c b/net/core/devlink.c
index 932476956d7e..f2ee1da5283c 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -6453,8 +6453,14 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 
 	devl_lock(devlink);
 
-	if (!attrs[DEVLINK_ATTR_REGION_NAME] ||
-	    !attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
+	if (!attrs[DEVLINK_ATTR_REGION_NAME]) {
+		NL_SET_ERR_MSG_MOD(cb->extack, "No region name provided");
+		err = -EINVAL;
+		goto out_unlock;
+	}
+
+	if (!attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
+		NL_SET_ERR_MSG_MOD(cb->extack, "No snapshot id provided");
 		err = -EINVAL;
 		goto out_unlock;
 	}
@@ -6477,6 +6483,8 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 		region = devlink_region_get_by_name(devlink, region_name);
 
 	if (!region) {
+		NL_SET_ERR_MSG_MOD(cb->extack,
+				   "The requested region does not exist");
 		err = -EINVAL;
 		goto out_unlock;
 	}
@@ -6484,6 +6492,8 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 	snapshot_id = nla_get_u32(attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]);
 	snapshot = devlink_region_snapshot_get_by_id(region, snapshot_id);
 	if (!snapshot) {
+		NL_SET_ERR_MSG_MOD(cb->extack,
+				   "The requested snapshot id does not exist");
 		err = -EINVAL;
 		goto out_unlock;
 	}
-- 
2.38.1.420.g319605f8f00e

Re: [PATCH net-next 3/8] devlink: report extended error message in region_read_dumpit

[Jakub Kicinski]

On Thu, 17 Nov 2022 14:07:58 -0800 Jacob Keller wrote:
> Report extended error details in the devlink_nl_cmd_region_read_dumpit
> function, by using the extack structure from the netlink_callback.
> 
> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
> ---
>  net/core/devlink.c | 14 ++++++++++++--
>  1 file changed, 12 insertions(+), 2 deletions(-)
> 
> diff --git a/net/core/devlink.c b/net/core/devlink.c
> index 932476956d7e..f2ee1da5283c 100644
> --- a/net/core/devlink.c
> +++ b/net/core/devlink.c
> @@ -6453,8 +6453,14 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
>  
>  	devl_lock(devlink);
>  
> -	if (!attrs[DEVLINK_ATTR_REGION_NAME] ||
> -	    !attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
> +	if (!attrs[DEVLINK_ATTR_REGION_NAME]) {
> +		NL_SET_ERR_MSG_MOD(cb->extack, "No region name provided");
> +		err = -EINVAL;
> +		goto out_unlock;
> +	}
> +
> +	if (!attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {

Please use GENL_REQ_ATTR_CHECK() instead of adding strings.

> +		NL_SET_ERR_MSG_MOD(cb->extack, "No snapshot id provided");
>  		err = -EINVAL;
>  		goto out_unlock;
>  	}
> @@ -6477,6 +6483,8 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
>  		region = devlink_region_get_by_name(devlink, region_name);
>  
>  	if (!region) {
> +		NL_SET_ERR_MSG_MOD(cb->extack,
> +				   "The requested region does not exist");

NL_SET_ERR_MSG_ATTR()

>  		err = -EINVAL;
>  		goto out_unlock;
>  	}
> @@ -6484,6 +6492,8 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
>  	snapshot_id = nla_get_u32(attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]);
>  	snapshot = devlink_region_snapshot_get_by_id(region, snapshot_id);
>  	if (!snapshot) {
> +		NL_SET_ERR_MSG_MOD(cb->extack,
> +				   "The requested snapshot id does not exist");

ditto

>  		err = -EINVAL;
>  		goto out_unlock;
>  	}

Re: [PATCH net-next 3/8] devlink: report extended error message in region_read_dumpit

[Jacob Keller]

On 11/18/2022 5:40 PM, Jakub Kicinski wrote:
> On Thu, 17 Nov 2022 14:07:58 -0800 Jacob Keller wrote:
>> Report extended error details in the devlink_nl_cmd_region_read_dumpit
>> function, by using the extack structure from the netlink_callback.
>>
>> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
>> ---
>>   net/core/devlink.c | 14 ++++++++++++--
>>   1 file changed, 12 insertions(+), 2 deletions(-)
>>
>> diff --git a/net/core/devlink.c b/net/core/devlink.c
>> index 932476956d7e..f2ee1da5283c 100644
>> --- a/net/core/devlink.c
>> +++ b/net/core/devlink.c
>> @@ -6453,8 +6453,14 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
>>   
>>   	devl_lock(devlink);
>>   
>> -	if (!attrs[DEVLINK_ATTR_REGION_NAME] ||
>> -	    !attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
>> +	if (!attrs[DEVLINK_ATTR_REGION_NAME]) {
>> +		NL_SET_ERR_MSG_MOD(cb->extack, "No region name provided");
>> +		err = -EINVAL;
>> +		goto out_unlock;
>> +	}
>> +
>> +	if (!attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
> 
> Please use GENL_REQ_ATTR_CHECK() instead of adding strings.
> 

Wasn't aware of this, nice!

>> +		NL_SET_ERR_MSG_MOD(cb->extack, "No snapshot id provided");
>>   		err = -EINVAL;
>>   		goto out_unlock;
>>   	}
>> @@ -6477,6 +6483,8 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
>>   		region = devlink_region_get_by_name(devlink, region_name);
>>   
>>   	if (!region) {
>> +		NL_SET_ERR_MSG_MOD(cb->extack,
>> +				   "The requested region does not exist");
> 
> NL_SET_ERR_MSG_ATTR()

Yep, should have noticed that myself. Thanks.

> 
>>   		err = -EINVAL;
>>   		goto out_unlock;
>>   	}
>> @@ -6484,6 +6492,8 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
>>   	snapshot_id = nla_get_u32(attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]);
>>   	snapshot = devlink_region_snapshot_get_by_id(region, snapshot_id);
>>   	if (!snapshot) {
>> +		NL_SET_ERR_MSG_MOD(cb->extack,
>> +				   "The requested snapshot id does not exist");
> 
> ditto
> 
>>   		err = -EINVAL;
>>   		goto out_unlock;
>>   	}
> 

Re: [PATCH net-next 3/8] devlink: report extended error message in region_read_dumpit

[Jacob Keller]

On 11/18/2022 5:40 PM, Jakub Kicinski wrote:
> On Thu, 17 Nov 2022 14:07:58 -0800 Jacob Keller wrote:
>> Report extended error details in the devlink_nl_cmd_region_read_dumpit
>> function, by using the extack structure from the netlink_callback.
>>
>> Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
>> ---
>>   net/core/devlink.c | 14 ++++++++++++--
>>   1 file changed, 12 insertions(+), 2 deletions(-)
>>
>> diff --git a/net/core/devlink.c b/net/core/devlink.c
>> index 932476956d7e..f2ee1da5283c 100644
>> --- a/net/core/devlink.c
>> +++ b/net/core/devlink.c
>> @@ -6453,8 +6453,14 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
>>   
>>   	devl_lock(devlink);
>>   
>> -	if (!attrs[DEVLINK_ATTR_REGION_NAME] ||
>> -	    !attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
>> +	if (!attrs[DEVLINK_ATTR_REGION_NAME]) {
>> +		NL_SET_ERR_MSG_MOD(cb->extack, "No region name provided");
>> +		err = -EINVAL;
>> +		goto out_unlock;
>> +	}
>> +
>> +	if (!attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
> 
> Please use GENL_REQ_ATTR_CHECK() instead of adding strings.
> 

Ahhh. Figured out why GENL_REQ_ATTR_CHECK wasn't used here already. It 
happens because the dumpit functions don't get a genl_info * struct, 
they get a netlink_cb and a genl_dumpit_info.

I can look at improving this.

Thanks,
Jake

Re: [PATCH net-next 3/8] devlink: report extended error message in region_read_dumpit

[Jakub Kicinski]

On Mon, 21 Nov 2022 11:10:37 -0800 Jacob Keller wrote:
> >> @@ -6453,8 +6453,14 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
> >>   
> >>   	devl_lock(devlink);
> >>   
> >> -	if (!attrs[DEVLINK_ATTR_REGION_NAME] ||
> >> -	    !attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
> >> +	if (!attrs[DEVLINK_ATTR_REGION_NAME]) {
> >> +		NL_SET_ERR_MSG_MOD(cb->extack, "No region name provided");
> >> +		err = -EINVAL;
> >> +		goto out_unlock;
> >> +	}
> >> +
> >> +	if (!attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {  
> > 
> > Please use GENL_REQ_ATTR_CHECK() instead of adding strings.
> >   
> 
> Ahhh. Figured out why GENL_REQ_ATTR_CHECK wasn't used here already. It 
> happens because the dumpit functions don't get a genl_info * struct, 
> they get a netlink_cb and a genl_dumpit_info.
> 
> I can look at improving this.

Ah damn, you're right, I thought I just missed it because it wasn't 
at the top of the function.

Re: [PATCH net-next 3/8] devlink: report extended error message in region_read_dumpit

[Jacob Keller]

On 11/21/2022 11:23 AM, Jakub Kicinski wrote:
> On Mon, 21 Nov 2022 11:10:37 -0800 Jacob Keller wrote:
>>>> @@ -6453,8 +6453,14 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
>>>>    
>>>>    	devl_lock(devlink);
>>>>    
>>>> -	if (!attrs[DEVLINK_ATTR_REGION_NAME] ||
>>>> -	    !attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
>>>> +	if (!attrs[DEVLINK_ATTR_REGION_NAME]) {
>>>> +		NL_SET_ERR_MSG_MOD(cb->extack, "No region name provided");
>>>> +		err = -EINVAL;
>>>> +		goto out_unlock;
>>>> +	}
>>>> +
>>>> +	if (!attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
>>>
>>> Please use GENL_REQ_ATTR_CHECK() instead of adding strings.
>>>    
>>
>> Ahhh. Figured out why GENL_REQ_ATTR_CHECK wasn't used here already. It
>> happens because the dumpit functions don't get a genl_info * struct,
>> they get a netlink_cb and a genl_dumpit_info.
>>
>> I can look at improving this.
> 
> Ah damn, you're right, I thought I just missed it because it wasn't
> at the top of the function.

I also saw a few other cases where it might make sense to use a 
GENL_CB_REQ_ATTR_CHECK or similar.

Unfortunately there's at least one area where we check for attributes 
inside a function that is used in both flows which would get a bit 
problematic :( Will see what I can come up with.

Thanks,
Jake

Re: [PATCH net-next 3/8] devlink: report extended error message in region_read_dumpit

[Jakub Kicinski]

On Mon, 21 Nov 2022 13:18:37 -0800 Jacob Keller wrote:
> > Ah damn, you're right, I thought I just missed it because it wasn't
> > at the top of the function.  
> 
> I also saw a few other cases where it might make sense to use a 
> GENL_CB_REQ_ATTR_CHECK or similar.
> 
> Unfortunately there's at least one area where we check for attributes 
> inside a function that is used in both flows which would get a bit 
> problematic :( Will see what I can come up with.

Perhaps this series is not the right place to worry about the missing
attr ext_ack for dumps. Go forward with v2, we can solve that later.

I think the info discrepancy falls under a larger problem of message
building code between doit and dump.

[PATCH net-next 4/8] devlink: remove unnecessary parameter from chunk_fill function

[Jacob Keller]

The devlink parameter of the devlink_nl_cmd_region_read_chunk_fill
function is not used. Remove it, to simplify the function signature.

Once removed, it is also obvious that the devlink parameter is not
necessary for the devlink_nl_region_read_snapshot_fill either.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 net/core/devlink.c | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/net/core/devlink.c b/net/core/devlink.c
index f2ee1da5283c..c28c3f2bb6e4 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -6367,7 +6367,6 @@ devlink_nl_cmd_region_new(struct sk_buff *skb, struct genl_info *info)
 }
 
 static int devlink_nl_cmd_region_read_chunk_fill(struct sk_buff *msg,
-						 struct devlink *devlink,
 						 u8 *chunk, u32 chunk_size,
 						 u64 addr)
 {
@@ -6398,7 +6397,6 @@ static int devlink_nl_cmd_region_read_chunk_fill(struct sk_buff *msg,
 #define DEVLINK_REGION_READ_CHUNK_SIZE 256
 
 static int devlink_nl_region_read_snapshot_fill(struct sk_buff *skb,
-						struct devlink *devlink,
 						struct devlink_snapshot *snapshot,
 						u64 start_offset,
 						u64 end_offset,
@@ -6415,9 +6413,7 @@ static int devlink_nl_region_read_snapshot_fill(struct sk_buff *skb,
 		u8 *data;
 
 		data = &snapshot->data[curr_offset];
-		err = devlink_nl_cmd_region_read_chunk_fill(skb, devlink,
-							    data, data_size,
-							    curr_offset);
+		err = devlink_nl_cmd_region_read_chunk_fill(skb, data, data_size, curr_offset);
 		if (err)
 			break;
 
@@ -6546,9 +6542,7 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 		goto nla_put_failure;
 	}
 
-	err = devlink_nl_region_read_snapshot_fill(skb, devlink,
-						   snapshot,
-						   start_offset,
+	err = devlink_nl_region_read_snapshot_fill(skb, snapshot, start_offset,
 						   end_offset, &ret_offset);
 
 	if (err && err != -EMSGSIZE)
-- 
2.38.1.420.g319605f8f00e

[PATCH net-next 5/8] devlink: refactor region_read_snapshot_fill to use a callback function

[Jacob Keller]

The devlink_nl_region_read_snapshot_fill is used to copy the contents of
a snapshot into a message for reporting to userspace via the
DEVLINK_CMG_REGION_READ netlink message.

A future change is going to add support for directly reading from
a region. Almost all of the logic for this new capability is identical.

To help reduce code duplication and make this logic more generic,
refactor the function to take a cb and cb_priv pointer for doing the
actual copy.

Add a devlink_region_snapshot_fill implementation that will simply copy
the relevant chunk of the region. This does require allocating some
storage for the chunk as opposed to simply passing the correct address
forward to the devlink_nl_cmg_region_read_chunk_fill function.

A future change to implement support for directly reading from a region
without a snapshot will provide a separate implementation that calls the
newly added devlink region operation.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 net/core/devlink.c | 44 +++++++++++++++++++++++++++++++++++---------
 1 file changed, 35 insertions(+), 9 deletions(-)

diff --git a/net/core/devlink.c b/net/core/devlink.c
index c28c3f2bb6e4..97e3a7158788 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -6396,23 +6396,34 @@ static int devlink_nl_cmd_region_read_chunk_fill(struct sk_buff *msg,
 
 #define DEVLINK_REGION_READ_CHUNK_SIZE 256
 
-static int devlink_nl_region_read_snapshot_fill(struct sk_buff *skb,
-						struct devlink_snapshot *snapshot,
-						u64 start_offset,
-						u64 end_offset,
-						u64 *new_offset)
+typedef int devlink_chunk_fill_t(void *cb_priv, u8 *chunk, u32 chunk_size,
+				 u64 curr_offset,
+				 struct netlink_ext_ack *extack);
+
+static int
+devlink_nl_region_read_fill(struct sk_buff *skb, devlink_chunk_fill_t *cb,
+			    void *cb_priv, u64 start_offset, u64 end_offset,
+			    u64 *new_offset, struct netlink_ext_ack *extack)
 {
 	u64 curr_offset = start_offset;
 	int err = 0;
+	u8 *data;
+
+	/* Allocate and re-use a single buffer */
+	data = kzalloc(DEVLINK_REGION_READ_CHUNK_SIZE, GFP_KERNEL);
+	if (!data)
+		return -ENOMEM;
 
 	*new_offset = start_offset;
 
 	while (curr_offset < end_offset) {
 		u32 data_size = min_t(u32, end_offset - curr_offset,
 				      DEVLINK_REGION_READ_CHUNK_SIZE);
-		u8 *data;
 
-		data = &snapshot->data[curr_offset];
+		err = cb(cb_priv, data, data_size, curr_offset, extack);
+		if (err)
+			break;
+
 		err = devlink_nl_cmd_region_read_chunk_fill(skb, data, data_size, curr_offset);
 		if (err)
 			break;
@@ -6421,9 +6432,23 @@ static int devlink_nl_region_read_snapshot_fill(struct sk_buff *skb,
 	}
 	*new_offset = curr_offset;
 
+	kfree(data);
+
 	return err;
 }
 
+static int
+devlink_region_snapshot_fill(void *cb_priv, u8 *chunk, u32 chunk_size,
+			     u64 curr_offset,
+			     struct __always_unused netlink_ext_ack *extack)
+{
+	struct devlink_snapshot *snapshot = cb_priv;
+
+	memcpy(chunk, &snapshot->data[curr_offset], chunk_size);
+
+	return 0;
+}
+
 static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 					     struct netlink_callback *cb)
 {
@@ -6542,8 +6567,9 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 		goto nla_put_failure;
 	}
 
-	err = devlink_nl_region_read_snapshot_fill(skb, snapshot, start_offset,
-						   end_offset, &ret_offset);
+	err = devlink_nl_region_read_fill(skb, &devlink_region_snapshot_fill,
+					  snapshot, start_offset, end_offset,
+					  &ret_offset, cb->extack);
 
 	if (err && err != -EMSGSIZE)
 		goto nla_put_failure;
-- 
2.38.1.420.g319605f8f00e

Re: [PATCH net-next 5/8] devlink: refactor region_read_snapshot_fill to use a callback function

[Jakub Kicinski]

On Thu, 17 Nov 2022 14:08:00 -0800 Jacob Keller wrote:
> +			     struct __always_unused netlink_ext_ack *extack)

clang points out this is not a great placement for __always_unused

Re: [PATCH net-next 5/8] devlink: refactor region_read_snapshot_fill to use a callback function

[Jakub Kicinski]

On Thu, 17 Nov 2022 14:08:00 -0800 Jacob Keller wrote:
> +	/* Allocate and re-use a single buffer */
> +	data = kzalloc(DEVLINK_REGION_READ_CHUNK_SIZE, GFP_KERNEL);
> +	if (!data)
> +		return -ENOMEM;

Why zalloc? If we expect drivers may underfill we should let them
return actual length.

Re: [PATCH net-next 5/8] devlink: refactor region_read_snapshot_fill to use a callback function

[Jacob Keller]

On 11/18/2022 5:48 PM, Jakub Kicinski wrote:
> On Thu, 17 Nov 2022 14:08:00 -0800 Jacob Keller wrote:
>> +	/* Allocate and re-use a single buffer */
>> +	data = kzalloc(DEVLINK_REGION_READ_CHUNK_SIZE, GFP_KERNEL);
>> +	if (!data)
>> +		return -ENOMEM;
> 
> Why zalloc? If we expect drivers may underfill we should let them
> return actual length.
> 

Probably just habit. Let me check on this one. I don't think there's a 
particular reason that drivers would under fill since they're in control 
of the region size.

Thanks,
Jake

[PATCH net-next 6/8] devlink: support directly reading from region memory

[Jacob Keller]

To read from a region, user space must currently request a new snapshot of
the region and then read from that snapshot. This can sometimes be overkill
if user space only reads a tiny portion. They first create the snapshot,
then request a read, then destroy the snapshot.

For regions which have a single underlying "contents", it makes sense to
allow supporting direct reading of the region data.

Extend the DEVLINK_CMD_REGION_READ to allow direct reading from a region if
supported. Instead of reporting a missing snapshot id as invalid, check if
the region supports direct read and if so switch to the direct access
method for reading the region data.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 .../networking/devlink/devlink-region.rst     |  8 +++
 include/net/devlink.h                         | 16 +++++
 net/core/devlink.c                            | 68 ++++++++++++++-----
 3 files changed, 74 insertions(+), 18 deletions(-)

diff --git a/Documentation/networking/devlink/devlink-region.rst b/Documentation/networking/devlink/devlink-region.rst
index f06dca9a1eb6..5770ecde179e 100644
--- a/Documentation/networking/devlink/devlink-region.rst
+++ b/Documentation/networking/devlink/devlink-region.rst
@@ -31,6 +31,10 @@ in its ``devlink_region_ops`` structure. If snapshot id is not set in
 the ``DEVLINK_CMD_REGION_NEW`` request kernel will allocate one and send
 the snapshot information to user space.
 
+Regions may optionally allow directly reading from their contents without a
+snapshot. A driver wishing to enable this for a region should implement the
+``.read`` callback in the ``devlink_region_ops`` structure.
+
 example usage
 -------------
 
@@ -65,6 +69,10 @@ example usage
     $ devlink region read pci/0000:00:05.0/fw-health snapshot 1 address 0 length 16
     0000000000000000 0014 95dc 0014 9514 0035 1670 0034 db30
 
+    # Read from the region without a snapshot
+    $ devlink region read pci/0000:00:05.0/fw-health address 16 length 16
+    0000000000000010 0000 0000 ffff ff04 0029 8c00 0028 8cc8
+
 As regions are likely very device or driver specific, no generic regions are
 defined. See the driver-specific documentation files for information on the
 specific regions a driver supports.
diff --git a/include/net/devlink.h b/include/net/devlink.h
index 611a23a3deb2..74547ebe08e7 100644
--- a/include/net/devlink.h
+++ b/include/net/devlink.h
@@ -647,6 +647,10 @@ struct devlink_info_req;
  *            the data variable must be updated to point to the snapshot data.
  *            The function will be called while the devlink instance lock is
  *            held.
+ * @read: callback to directly read a portion of the region. On success,
+ *        the data pointer will be updated with the contents of the
+ *        requested portion of the region. The function will be called
+ *        while the devlink instance lock is held.
  * @priv: Pointer to driver private data for the region operation
  */
 struct devlink_region_ops {
@@ -656,6 +660,10 @@ struct devlink_region_ops {
 			const struct devlink_region_ops *ops,
 			struct netlink_ext_ack *extack,
 			u8 **data);
+	int (*read)(struct devlink *devlink,
+		    const struct devlink_region_ops *ops,
+		    struct netlink_ext_ack *extack,
+		    u64 offset, u32 size, u8 *data);
 	void *priv;
 };
 
@@ -667,6 +675,10 @@ struct devlink_region_ops {
  *            the data variable must be updated to point to the snapshot data.
  *            The function will be called while the devlink instance lock is
  *            held.
+ * @read: callback to directly read a portion of the region. On success,
+ *        the data pointer will be updated with the contents of the
+ *        requested portion of the region. The function will be called
+ *        while the devlink instance lock is held.
  * @priv: Pointer to driver private data for the region operation
  */
 struct devlink_port_region_ops {
@@ -676,6 +688,10 @@ struct devlink_port_region_ops {
 			const struct devlink_port_region_ops *ops,
 			struct netlink_ext_ack *extack,
 			u8 **data);
+	int (*read)(struct devlink_port *port,
+		    const struct devlink_port_region_ops *ops,
+		    struct netlink_ext_ack *extack,
+		    u64 offset, u32 size, u8 *data);
 	void *priv;
 };
 
diff --git a/net/core/devlink.c b/net/core/devlink.c
index 97e3a7158788..cdbcfdb96727 100644
--- a/net/core/devlink.c
+++ b/net/core/devlink.c
@@ -6449,6 +6449,26 @@ devlink_region_snapshot_fill(void *cb_priv, u8 *chunk, u32 chunk_size,
 	return 0;
 }
 
+static int
+devlink_region_port_direct_fill(void *cb_priv, u8 *chunk, u32 chunk_size,
+				u64 curr_offset, struct netlink_ext_ack *extack)
+{
+	struct devlink_region *region = cb_priv;
+
+	return region->port_ops->read(region->port, region->port_ops, extack,
+				      curr_offset, chunk_size, chunk);
+}
+
+static int
+devlink_region_direct_fill(void *cb_priv, u8 *chunk, u32 chunk_size,
+			   u64 curr_offset, struct netlink_ext_ack *extack)
+{
+	struct devlink_region *region = cb_priv;
+
+	return region->ops->read(region->devlink, region->ops, extack,
+				 curr_offset, chunk_size, chunk);
+}
+
 static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 					     struct netlink_callback *cb)
 {
@@ -6456,13 +6476,13 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 	u64 ret_offset, start_offset, end_offset = U64_MAX;
 	struct nlattr **attrs = info->attrs;
 	struct devlink_port *port = NULL;
-	struct devlink_snapshot *snapshot;
+	devlink_chunk_fill_t *region_cb;
 	struct devlink_region *region;
 	struct nlattr *chunks_attr;
 	const char *region_name;
 	struct devlink *devlink;
 	unsigned int index;
-	u32 snapshot_id;
+	void *region_cb_priv;
 	void *hdr;
 	int err;
 
@@ -6480,12 +6500,6 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 		goto out_unlock;
 	}
 
-	if (!attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
-		NL_SET_ERR_MSG_MOD(cb->extack, "No snapshot id provided");
-		err = -EINVAL;
-		goto out_unlock;
-	}
-
 	if (info->attrs[DEVLINK_ATTR_PORT_INDEX]) {
 		index = nla_get_u32(info->attrs[DEVLINK_ATTR_PORT_INDEX]);
 
@@ -6510,13 +6524,31 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 		goto out_unlock;
 	}
 
-	snapshot_id = nla_get_u32(attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]);
-	snapshot = devlink_region_snapshot_get_by_id(region, snapshot_id);
-	if (!snapshot) {
-		NL_SET_ERR_MSG_MOD(cb->extack,
-				   "The requested snapshot id does not exist");
-		err = -EINVAL;
-		goto out_unlock;
+	if (!attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]) {
+		if (!region->ops->read) {
+			NL_SET_ERR_MSG_MOD(cb->extack,
+					   "The requested region does not support direct read");
+			err = -EOPNOTSUPP;
+			goto out_unlock;
+		}
+		if (port)
+			region_cb = &devlink_region_port_direct_fill;
+		else
+			region_cb = &devlink_region_direct_fill;
+		region_cb_priv = region;
+	} else {
+		u32 snapshot_id = nla_get_u32(attrs[DEVLINK_ATTR_REGION_SNAPSHOT_ID]);
+		struct devlink_snapshot *snapshot;
+
+		snapshot = devlink_region_snapshot_get_by_id(region, snapshot_id);
+		if (!snapshot) {
+			NL_SET_ERR_MSG_MOD(cb->extack,
+					   "The requested snapshot id does not exist");
+			err = -EINVAL;
+			goto out_unlock;
+		}
+		region_cb = &devlink_region_snapshot_fill;
+		region_cb_priv = snapshot;
 	}
 
 	if (attrs[DEVLINK_ATTR_REGION_CHUNK_ADDR] &&
@@ -6567,9 +6599,9 @@ static int devlink_nl_cmd_region_read_dumpit(struct sk_buff *skb,
 		goto nla_put_failure;
 	}
 
-	err = devlink_nl_region_read_fill(skb, &devlink_region_snapshot_fill,
-					  snapshot, start_offset, end_offset,
-					  &ret_offset, cb->extack);
+	err = devlink_nl_region_read_fill(skb, region_cb, region_cb_priv,
+					  start_offset, end_offset, &ret_offset,
+					  cb->extack);
 
 	if (err && err != -EMSGSIZE)
 		goto nla_put_failure;
-- 
2.38.1.420.g319605f8f00e

Re: [PATCH net-next 6/8] devlink: support directly reading from region memory

[Jakub Kicinski]

On Thu, 17 Nov 2022 14:08:01 -0800 Jacob Keller wrote:
> +Regions may optionally allow directly reading from their contents without a
> +snapshot. A driver wishing to enable this for a region should implement the
> +``.read`` callback in the ``devlink_region_ops`` structure.

Perhaps worth adding that direct read has weaker atomicity guarantees
than snapshot? User at the CLI level may not expect the read request
to be broken up into smaller chunks.

Re: [PATCH net-next 6/8] devlink: support directly reading from region memory

[Jacob Keller]

On 11/18/2022 5:49 PM, Jakub Kicinski wrote:
> On Thu, 17 Nov 2022 14:08:01 -0800 Jacob Keller wrote:
>> +Regions may optionally allow directly reading from their contents without a
>> +snapshot. A driver wishing to enable this for a region should implement the
>> +``.read`` callback in the ``devlink_region_ops`` structure.
> 
> Perhaps worth adding that direct read has weaker atomicity guarantees
> than snapshot? User at the CLI level may not expect the read request
> to be broken up into smaller chunks.

Sure. I can expand on that.

[PATCH net-next 7/8] ice: use same function to snapshot both NVM and Shadow RAM

[Jacob Keller]

The ice driver supports a region for both the flat NVM contents as well as
the Shadow RAM which is a layer built on top of the flash during device
initialization.

These regions use an almost identical read function, except that the NVM
needs to set the direct flag when reading, while Shadow RAM needs to read
without the direct flag set. They each call ice_read_flat_nvm with the only
difference being whether to set the direct flash flag.

The NVM region read function also was fixed to read the NVM in blocks to
avoid a situation where the firmware reclaims the lock due to taking too
long.

Note that the region snapshot function takes the ops pointer so the
function can easily determine which region to read. Make use of this and
re-use the NVM snapshot function for both the NVM and Shadow RAM regions.
This makes Shadow RAM benefit from the same block approach as the NVM
region. It also reduces code in the ice driver.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 drivers/net/ethernet/intel/ice/ice_devlink.c | 95 +++++---------------
 1 file changed, 23 insertions(+), 72 deletions(-)

diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.c b/drivers/net/ethernet/intel/ice/ice_devlink.c
index 455489e9457d..82680417b02e 100644
--- a/drivers/net/ethernet/intel/ice/ice_devlink.c
+++ b/drivers/net/ethernet/intel/ice/ice_devlink.c
@@ -1094,21 +1094,22 @@ void ice_devlink_destroy_vf_port(struct ice_vf *vf)
 
 #define ICE_DEVLINK_READ_BLK_SIZE (1024 * 1024)
 
+static const struct devlink_region_ops ice_nvm_region_ops;
+static const struct devlink_region_ops ice_sram_region_ops;
+
 /**
  * ice_devlink_nvm_snapshot - Capture a snapshot of the NVM flash contents
  * @devlink: the devlink instance
- * @ops: the devlink region being snapshotted
+ * @ops: the devlink region to snapshot
  * @extack: extended ACK response structure
  * @data: on exit points to snapshot data buffer
  *
- * This function is called in response to the DEVLINK_CMD_REGION_TRIGGER for
- * the nvm-flash devlink region. It captures a snapshot of the full NVM flash
- * contents, including both banks of flash. This snapshot can later be viewed
- * via the devlink-region interface.
+ * This function is called in response to a DEVLINK_CMD_REGION_NEW for either
+ * the nvm-flash or shadow-ram region.
  *
- * It captures the flash using the FLASH_ONLY bit set when reading via
- * firmware, so it does not read the current Shadow RAM contents. For that,
- * use the shadow-ram region.
+ * It captures a snapshot of the NVM or Shadow RAM flash contents. This
+ * snapshot can then later be viewed via the DEVLINK_CMD_REGION_READ netlink
+ * interface.
  *
  * @returns zero on success, and updates the data pointer. Returns a non-zero
  * error code on failure.
@@ -1120,17 +1121,27 @@ static int ice_devlink_nvm_snapshot(struct devlink *devlink,
 	struct ice_pf *pf = devlink_priv(devlink);
 	struct device *dev = ice_pf_to_dev(pf);
 	struct ice_hw *hw = &pf->hw;
+	bool read_shadow_ram;
 	u8 *nvm_data, *tmp, i;
 	u32 nvm_size, left;
 	s8 num_blks;
 	int status;
 
-	nvm_size = hw->flash.flash_size;
+	if (ops == &ice_nvm_region_ops) {
+		read_shadow_ram = false;
+		nvm_size = hw->flash.flash_size;
+	} else if (ops == &ice_sram_region_ops) {
+		read_shadow_ram = true;
+		nvm_size = hw->flash.sr_words * 2u;
+	} else {
+		NL_SET_ERR_MSG_MOD(extack, "Unexpected region in snapshot function");
+		return -EOPNOTSUPP;
+	}
+
 	nvm_data = vzalloc(nvm_size);
 	if (!nvm_data)
 		return -ENOMEM;
 
-
 	num_blks = DIV_ROUND_UP(nvm_size, ICE_DEVLINK_READ_BLK_SIZE);
 	tmp = nvm_data;
 	left = nvm_size;
@@ -1154,7 +1165,7 @@ static int ice_devlink_nvm_snapshot(struct devlink *devlink,
 		}
 
 		status = ice_read_flat_nvm(hw, i * ICE_DEVLINK_READ_BLK_SIZE,
-					   &read_sz, tmp, false);
+					   &read_sz, tmp, read_shadow_ram);
 		if (status) {
 			dev_dbg(dev, "ice_read_flat_nvm failed after reading %u bytes, err %d aq_err %d\n",
 				read_sz, status, hw->adminq.sq_last_status);
@@ -1174,66 +1185,6 @@ static int ice_devlink_nvm_snapshot(struct devlink *devlink,
 	return 0;
 }
 
-/**
- * ice_devlink_sram_snapshot - Capture a snapshot of the Shadow RAM contents
- * @devlink: the devlink instance
- * @ops: the devlink region being snapshotted
- * @extack: extended ACK response structure
- * @data: on exit points to snapshot data buffer
- *
- * This function is called in response to the DEVLINK_CMD_REGION_TRIGGER for
- * the shadow-ram devlink region. It captures a snapshot of the shadow ram
- * contents. This snapshot can later be viewed via the devlink-region
- * interface.
- *
- * @returns zero on success, and updates the data pointer. Returns a non-zero
- * error code on failure.
- */
-static int
-ice_devlink_sram_snapshot(struct devlink *devlink,
-			  const struct devlink_region_ops __always_unused *ops,
-			  struct netlink_ext_ack *extack, u8 **data)
-{
-	struct ice_pf *pf = devlink_priv(devlink);
-	struct device *dev = ice_pf_to_dev(pf);
-	struct ice_hw *hw = &pf->hw;
-	u8 *sram_data;
-	u32 sram_size;
-	int err;
-
-	sram_size = hw->flash.sr_words * 2u;
-	sram_data = vzalloc(sram_size);
-	if (!sram_data)
-		return -ENOMEM;
-
-	err = ice_acquire_nvm(hw, ICE_RES_READ);
-	if (err) {
-		dev_dbg(dev, "ice_acquire_nvm failed, err %d aq_err %d\n",
-			err, hw->adminq.sq_last_status);
-		NL_SET_ERR_MSG_MOD(extack, "Failed to acquire NVM semaphore");
-		vfree(sram_data);
-		return err;
-	}
-
-	/* Read from the Shadow RAM, rather than directly from NVM */
-	err = ice_read_flat_nvm(hw, 0, &sram_size, sram_data, true);
-	if (err) {
-		dev_dbg(dev, "ice_read_flat_nvm failed after reading %u bytes, err %d aq_err %d\n",
-			sram_size, err, hw->adminq.sq_last_status);
-		NL_SET_ERR_MSG_MOD(extack,
-				   "Failed to read Shadow RAM contents");
-		ice_release_nvm(hw);
-		vfree(sram_data);
-		return err;
-	}
-
-	ice_release_nvm(hw);
-
-	*data = sram_data;
-
-	return 0;
-}
-
 /**
  * ice_devlink_devcaps_snapshot - Capture snapshot of device capabilities
  * @devlink: the devlink instance
@@ -1287,7 +1238,7 @@ static const struct devlink_region_ops ice_nvm_region_ops = {
 static const struct devlink_region_ops ice_sram_region_ops = {
 	.name = "shadow-ram",
 	.destructor = vfree,
-	.snapshot = ice_devlink_sram_snapshot,
+	.snapshot = ice_devlink_nvm_snapshot,
 };
 
 static const struct devlink_region_ops ice_devcaps_region_ops = {
-- 
2.38.1.420.g319605f8f00e

[PATCH net-next 8/8] ice: implement direct read for NVM and Shadow RAM regions

[Jacob Keller]

Implement the .read handler for the NVM and Shadow RAM regions. This
enables user space to read a small chunk of the flash without needing the
overhead of creating a full snapshot.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 drivers/net/ethernet/intel/ice/ice_devlink.c | 69 ++++++++++++++++++++
 1 file changed, 69 insertions(+)

diff --git a/drivers/net/ethernet/intel/ice/ice_devlink.c b/drivers/net/ethernet/intel/ice/ice_devlink.c
index 82680417b02e..8aec2f5fdf4a 100644
--- a/drivers/net/ethernet/intel/ice/ice_devlink.c
+++ b/drivers/net/ethernet/intel/ice/ice_devlink.c
@@ -1185,6 +1185,73 @@ static int ice_devlink_nvm_snapshot(struct devlink *devlink,
 	return 0;
 }
 
+/**
+ * ice_devlink_nvm_read - Read a portion of NVM flash contents
+ * @devlink: the devlink instance
+ * @ops: the devlink region to snapshot
+ * @extack: extended ACK response structure
+ * @offset: the offset to start at
+ * @size: the amount to read
+ * @data: the data buffer to read into
+ *
+ * This function is called in response to DEVLINK_CMD_REGION_READ to directly
+ * read a section of the NVM contents.
+ *
+ * It reads from either the nvm-flash or shadow-ram region contents.
+ *
+ * @returns zero on success, and updates the data pointer. Returns a non-zero
+ * error code on failure.
+ */
+static int ice_devlink_nvm_read(struct devlink *devlink,
+				const struct devlink_region_ops *ops,
+				struct netlink_ext_ack *extack,
+				u64 offset, u32 size, u8 *data)
+{
+	struct ice_pf *pf = devlink_priv(devlink);
+	struct device *dev = ice_pf_to_dev(pf);
+	struct ice_hw *hw = &pf->hw;
+	bool read_shadow_ram;
+	u64 nvm_size;
+	int status;
+
+	if (ops == &ice_nvm_region_ops) {
+		read_shadow_ram = false;
+		nvm_size = hw->flash.flash_size;
+	} else if (ops == &ice_sram_region_ops) {
+		read_shadow_ram = true;
+		nvm_size = hw->flash.sr_words * 2u;
+	} else {
+		NL_SET_ERR_MSG_MOD(extack, "Unexpected region in snapshot function");
+		return -EOPNOTSUPP;
+	}
+
+	if (offset + size >= nvm_size) {
+		NL_SET_ERR_MSG_MOD(extack, "Cannot read beyond the region size");
+		return -ERANGE;
+	}
+
+	status = ice_acquire_nvm(hw, ICE_RES_READ);
+	if (status) {
+		dev_dbg(dev, "ice_acquire_nvm failed, err %d aq_err %d\n",
+			status, hw->adminq.sq_last_status);
+		NL_SET_ERR_MSG_MOD(extack, "Failed to acquire NVM semaphore");
+		return -EIO;
+	}
+
+	status = ice_read_flat_nvm(hw, (u32)offset, &size, data,
+				   read_shadow_ram);
+	if (status) {
+		dev_dbg(dev, "ice_read_flat_nvm failed after reading %u bytes, err %d aq_err %d\n",
+			size, status, hw->adminq.sq_last_status);
+		NL_SET_ERR_MSG_MOD(extack, "Failed to read NVM contents");
+		ice_release_nvm(hw);
+		return -EIO;
+	}
+	ice_release_nvm(hw);
+
+	return 0;
+}
+
 /**
  * ice_devlink_devcaps_snapshot - Capture snapshot of device capabilities
  * @devlink: the devlink instance
@@ -1233,12 +1300,14 @@ static const struct devlink_region_ops ice_nvm_region_ops = {
 	.name = "nvm-flash",
 	.destructor = vfree,
 	.snapshot = ice_devlink_nvm_snapshot,
+	.read = ice_devlink_nvm_read,
 };
 
 static const struct devlink_region_ops ice_sram_region_ops = {
 	.name = "shadow-ram",
 	.destructor = vfree,
 	.snapshot = ice_devlink_nvm_snapshot,
+	.read = ice_devlink_nvm_read,
 };
 
 static const struct devlink_region_ops ice_devcaps_region_ops = {
-- 
2.38.1.420.g319605f8f00e

Re: [PATCH net-next 0/8] devlink: support direct read from region

[Jacob Keller]

On 11/17/2022 2:07 PM, Jacob Keller wrote:
> A long time ago when initially implementing devlink regions in ice I
> proposed the ability to allow reading from a region without taking a
> snapshot [1]. I eventually dropped this work from the original series due to
> size. Then I eventually lost track of submitting this follow up.
> 
> This can be useful when interacting with some region that has some
> definitive "contents" from which snapshots are made. For example the ice
> driver has regions representing the contents of the device flash.
> 
> If userspace wants to read the contents today, it must first take a snapshot
> and then read from that snapshot. This makes sense if you want to read a
> large portion of data or you want to be sure reads are consistently from the
> same recording of the flash.
> 
> However if user space only wants to read a small chunk, it must first
> generate a snapshot of the entire contents, perform a read from the
> snapshot, and then delete the snapshot after reading.
> 
> For such a use case, a direct read from the region makes more sense. This
> can be achieved by allowing the devlink region read command to work without
> a snapshot. Instead the portion to be read can be forwarded directly to the
> driver via a new .read callback.
> 
> This avoids the need to read the entire region contents into memory first
> and avoids the software overhead of creating a snapshot and then deleting
> it.
> 
> This series implements such behavior and hooks up the ice NVM and shadow RAM
> regions to allow it.
> 
> [1] https://lore.kernel.org/netdev/20200130225913.1671982-1-jacob.e.keller@intel.com/
> 
> Cc: Jiri Pirko <jiri@nvidia.com>
> Cc: Jakub Kicinski <kuba@kernel.org>
> 

There was some discussion about GENL_REQ_ATTR_CHECK, but this doesn't 
work for dumpits. While I was investigating this, I also saw a bunch of 
uses of NL_SET_ERR_MSG_MOD in net/core/devlink.c, so I have a separate 
patch to switch those to NL_SET_ERR_MSG, which I'll probably include in 
a series cleaning up the extended error messages.

> Jacob Keller (8):
>    devlink: find snapshot in devlink_nl_cmd_region_read_dumpit
>    devlink: use min_t to calculate data_size
>    devlink: report extended error message in region_read_dumpit
>    devlink: remove unnecessary parameter from chunk_fill function
>    devlink: refactor region_read_snapshot_fill to use a callback function
>    devlink: support directly reading from region memory
>    ice: use same function to snapshot both NVM and Shadow RAM
>    ice: implement direct read for NVM and Shadow RAM regions
> 
>   .../networking/devlink/devlink-region.rst     |   8 ++
>   drivers/net/ethernet/intel/ice/ice_devlink.c  | 112 +++++++++-------
>   include/net/devlink.h                         |  16 +++
>   net/core/devlink.c                            | 121 +++++++++++++-----
>   4 files changed, 180 insertions(+), 77 deletions(-)
> 
> 
> base-commit: b4b221bd79a1c698d9653e3ae2c3cb61cdc9aee7