linux-rdma.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v3] Fix one error in mthca_alloc
@ 2021-08-27  0:52 kangning
  2021-08-31  9:52 ` Leon Romanovsky
  0 siblings, 1 reply; 7+ messages in thread
From: kangning @ 2021-08-27  0:52 UTC (permalink / raw)
  To: haakon.bugge; +Cc: linux-rdma, kangning

drivers/infiniband/hw/mthca/mthca_allocator.c: alloc->last left unchanged in mthca_alloc, which
has impact on performance of function find_next_zero_bit in mthca_alloc.

Signed-off-by: kangning <kangning18z@ict.ac.cn>
---
 
 I squashed two commits into one in this version.
 
 drivers/infiniband/hw/mthca/mthca_allocator.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/infiniband/hw/mthca/mthca_allocator.c b/drivers/infiniband/hw/mthca/mthca_allocator.c
index aef1d274a14e..1141695093e7 100644
--- a/drivers/infiniband/hw/mthca/mthca_allocator.c
+++ b/drivers/infiniband/hw/mthca/mthca_allocator.c
@@ -51,6 +51,9 @@ u32 mthca_alloc(struct mthca_alloc *alloc)
 	}
 
 	if (obj < alloc->max) {
+		alloc->last = obj + 1;
+		if (alloc->last == alloc->max)
+			alloc->last = 0;
 		set_bit(obj, alloc->table);
 		obj |= alloc->top;
 	} else
-- 
2.17.1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH v3] Fix one error in mthca_alloc
  2021-08-27  0:52 [PATCH v3] Fix one error in mthca_alloc kangning
@ 2021-08-31  9:52 ` Leon Romanovsky
  2021-08-31 10:40   ` 康宁
                     ` (2 more replies)
  0 siblings, 3 replies; 7+ messages in thread
From: Leon Romanovsky @ 2021-08-31  9:52 UTC (permalink / raw)
  To: kangning; +Cc: haakon.bugge, linux-rdma

On Fri, Aug 27, 2021 at 08:52:28AM +0800, kangning wrote:
> drivers/infiniband/hw/mthca/mthca_allocator.c: alloc->last left unchanged in mthca_alloc, which
> has impact on performance of function find_next_zero_bit in mthca_alloc.

I don't know what the sentence above means, but the change is unlikely
to be correct.

When alloc->last starts to be equal to alloc->max, the
find_next_zero_bit() will always return alloc->max. which will ensure
that the following code is executed.

   48         if (obj >= alloc->max) {
   49                 alloc->top = (alloc->top + alloc->max) & alloc->mask;
   50                 obj = find_first_zero_bit(alloc->table, alloc->max);
   51         }


However the mthca_alloc() function has other error, it returns -1 while
based on its declaration it needs to be unsigned,

Thanks

> 
> Signed-off-by: kangning <kangning18z@ict.ac.cn>
> ---
>  
>  I squashed two commits into one in this version.
>  
>  drivers/infiniband/hw/mthca/mthca_allocator.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/infiniband/hw/mthca/mthca_allocator.c b/drivers/infiniband/hw/mthca/mthca_allocator.c
> index aef1d274a14e..1141695093e7 100644
> --- a/drivers/infiniband/hw/mthca/mthca_allocator.c
> +++ b/drivers/infiniband/hw/mthca/mthca_allocator.c
> @@ -51,6 +51,9 @@ u32 mthca_alloc(struct mthca_alloc *alloc)
>  	}
>  
>  	if (obj < alloc->max) {
> +		alloc->last = obj + 1;
> +		if (alloc->last == alloc->max)
> +			alloc->last = 0;
>  		set_bit(obj, alloc->table);
>  		obj |= alloc->top;
>  	} else
> -- 
> 2.17.1
> 

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Re: [PATCH v3] Fix one error in mthca_alloc
  2021-08-31  9:52 ` Leon Romanovsky
@ 2021-08-31 10:40   ` 康宁
  2021-08-31 12:00     ` Leon Romanovsky
  2021-08-31 10:47   ` 康宁
       [not found]   ` <202109011027459851120@ict.ac.cn>
  2 siblings, 1 reply; 7+ messages in thread
From: 康宁 @ 2021-08-31 10:40 UTC (permalink / raw)
  To: Leon Romanovsky; +Cc: haakon.bugge, linux-rdma

&gt; 
&gt; On Fri, Aug 27, 2021 at 08:52:28AM +0800, kangning wrote:
&gt; &gt; drivers/infiniband/hw/mthca/mthca_allocator.c: alloc-&gt;last left unchanged in mthca_alloc, which
&gt; &gt; has impact on performance of function find_next_zero_bit in mthca_alloc.
&gt; 
&gt; I don't know what the sentence above means, but the change is unlikely
&gt; to be correct.
&gt; 
&gt; When alloc-&gt;last starts to be equal to alloc-&gt;max, the
&gt; find_next_zero_bit() will always return alloc-&gt;max. which will ensure
&gt; that the following code is executed.
&gt; 
&gt;    48         if (obj &gt;= alloc-&gt;max) {
&gt;    49                 alloc-&gt;top = (alloc-&gt;top + alloc-&gt;max) &amp; alloc-&gt;mask;
&gt;    50                 obj = find_first_zero_bit(alloc-&gt;table, alloc-&gt;max);
&gt;    51         }
&gt; 

Thanks for your review.
Yes, your analysis is right. However, this is a bitmap allocator for resource id allocation (like QP, and CQ). 
We first look at the situation where no resource id is released.
When the value of alloc-&gt;last starts to reaches alloc-&gt;max, the bitmap will 
be full. In this case, it's normal to return an invalid value.

Now, let's add resource releasing into consideration. 
The following code is part of mthca_free:

70	spin_lock_irqsave(&amp;alloc-&gt;lock, flags);
71
72	clear_bit(obj, alloc-&gt;table);
73	alloc-&gt;last = min(alloc-&gt;last, obj);
74	alloc-&gt;top = (alloc-&gt;top + alloc-&gt;max) &amp; alloc-&gt;mask;

mthca_free() is used to release the allocated resource id. It will update alloc-&gt;last when obj is freed. 
So, if the bitmap has space for allocation, my modification can continuously work.

&gt; 
&gt; However the mthca_alloc() function has other error, it returns -1 while
&gt; based on its declaration it needs to be unsigned,

I think you are right about this. But obj is the return value of u32 type, which is the requirement of resource id 
(though it may not fully use 32 bits). I have no idea of how to fix it.

Thanks.

&gt; 
&gt; Thanks
&gt; 
&gt; &gt; 
&gt; &gt; Signed-off-by: kangning <kangning18z@ict.ac.cn>
&gt; &gt; ---
&gt; &gt;  
&gt; &gt;  I squashed two commits into one in this version.
&gt; &gt;  
&gt; &gt;  drivers/infiniband/hw/mthca/mthca_allocator.c | 3 +++
&gt; &gt;  1 file changed, 3 insertions(+)
&gt; &gt; 
&gt; &gt; diff --git a/drivers/infiniband/hw/mthca/mthca_allocator.c b/drivers/infiniband/hw/mthca/mthca_allocator.c
&gt; &gt; index aef1d274a14e..1141695093e7 100644
&gt; &gt; --- a/drivers/infiniband/hw/mthca/mthca_allocator.c
&gt; &gt; +++ b/drivers/infiniband/hw/mthca/mthca_allocator.c
&gt; &gt; @@ -51,6 +51,9 @@ u32 mthca_alloc(struct mthca_alloc *alloc)
&gt; &gt;  	}
&gt; &gt;  
&gt; &gt;  	if (obj &lt; alloc-&gt;max) {
&gt; &gt; +		alloc-&gt;last = obj + 1;
&gt; &gt; +		if (alloc-&gt;last == alloc-&gt;max)
&gt; &gt; +			alloc-&gt;last = 0;
&gt; &gt;  		set_bit(obj, alloc-&gt;table);
&gt; &gt;  		obj |= alloc-&gt;top;
&gt; &gt;  	} else
&gt; &gt; -- 
&gt; &gt; 2.17.1
&gt; &gt; 
</kangning18z@ict.ac.cn>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Re: [PATCH v3] Fix one error in mthca_alloc
  2021-08-31  9:52 ` Leon Romanovsky
  2021-08-31 10:40   ` 康宁
@ 2021-08-31 10:47   ` 康宁
       [not found]   ` <202109011027459851120@ict.ac.cn>
  2 siblings, 0 replies; 7+ messages in thread
From: 康宁 @ 2021-08-31 10:47 UTC (permalink / raw)
  To: Leon Romanovsky; +Cc: haakon.bugge, linux-rdma


&gt;
&gt; On Fri, Aug 27, 2021 at 08:52:28AM +0800, kangning wrote:
&gt; &gt; drivers/infiniband/hw/mthca/mthca_allocator.c: alloc-&gt;last left unchanged in mthca_alloc, which
&gt; &gt; has impact on performance of function find_next_zero_bit in mthca_alloc.
&gt; 
&gt; I don't know what the sentence above means, but the change is unlikely
&gt; to be correct.
&gt; 
&gt; When alloc-&gt;last starts to be equal to alloc-&gt;max, the
&gt; find_next_zero_bit() will always return alloc-&gt;max. which will ensure
&gt; that the following code is executed.
&gt; 
&gt;    48         if (obj &gt;= alloc-&gt;max) {
&gt;    49                 alloc-&gt;top = (alloc-&gt;top + alloc-&gt;max) &amp; alloc-&gt;mask;
&gt;    50                 obj = find_first_zero_bit(alloc-&gt;table, alloc-&gt;max);
&gt;    51         }
&gt; 

Thanks for your review.
Your analysis is right. However, this is a bitmap allocator for resource id allocation (like QP, and CQ). 
We first look at the situation where no resource id is released.
When the value of alloc-&gt;last starts to reaches alloc-&gt;max, the bitmap will 
be full. In this case, it's normal to return an invalid value.

Now, let's add resource releasing into consideration. 
The following code is part of mthca_free():

70	spin_lock_irqsave(&amp;alloc-&gt;lock, flags);
71
72	clear_bit(obj, alloc-&gt;table);
73	alloc-&gt;last = min(alloc-&gt;last, obj);
74	alloc-&gt;top = (alloc-&gt;top + alloc-&gt;max) &amp; alloc-&gt;mask;

mthca_free() is used to release the allocated resource id. It will update alloc-&gt;last when obj is freed. 
So, if the bitmap has space for allocation, my modification can continuously work.

&gt; 
&gt; However the mthca_alloc() function has other error, it returns -1 while
&gt; based on its declaration it needs to be unsigned,

I think you are right about this. But obj is the return value of u32 type, which is the requirement of resource id 
(though it may not fully use 32 bits). I have no idea of how to fix it.

Thanks.

&gt; 
&gt; Thanks
&gt; 
&gt; &gt; 
&gt; &gt; Signed-off-by: kangning <kangning18z@ict.ac.cn>
&gt; &gt; ---
&gt; &gt;  
&gt; &gt;  I squashed two commits into one in this version.
&gt; &gt;  
&gt; &gt;  drivers/infiniband/hw/mthca/mthca_allocator.c | 3 +++
&gt; &gt;  1 file changed, 3 insertions(+)
&gt; &gt; 
&gt; &gt; diff --git a/drivers/infiniband/hw/mthca/mthca_allocator.c b/drivers/infiniband/hw/mthca/mthca_allocator.c
&gt; &gt; index aef1d274a14e..1141695093e7 100644
&gt; &gt; --- a/drivers/infiniband/hw/mthca/mthca_allocator.c
&gt; &gt; +++ b/drivers/infiniband/hw/mthca/mthca_allocator.c
&gt; &gt; @@ -51,6 +51,9 @@ u32 mthca_alloc(struct mthca_alloc *alloc)
&gt; &gt;  	}
&gt; &gt;  
&gt; &gt;  	if (obj &lt; alloc-&gt;max) {
&gt; &gt; +		alloc-&gt;last = obj + 1;
&gt; &gt; +		if (alloc-&gt;last == alloc-&gt;max)
&gt; &gt; +			alloc-&gt;last = 0;
&gt; &gt;  		set_bit(obj, alloc-&gt;table);
&gt; &gt;  		obj |= alloc-&gt;top;
&gt; &gt;  	} else
&gt; &gt; -- 
&gt; &gt; 2.17.1
&gt; &gt; 
</kangning18z@ict.ac.cn>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Re: [PATCH v3] Fix one error in mthca_alloc
  2021-08-31 10:40   ` 康宁
@ 2021-08-31 12:00     ` Leon Romanovsky
  2021-08-31 12:16       ` kangning18z
  0 siblings, 1 reply; 7+ messages in thread
From: Leon Romanovsky @ 2021-08-31 12:00 UTC (permalink / raw)
  To: 康宁; +Cc: haakon.bugge, linux-rdma

On Tue, Aug 31, 2021 at 06:40:38PM +0800, 康宁 wrote:
> &gt; 
> &gt; On Fri, Aug 27, 2021 at 08:52:28AM +0800, kangning wrote:
> &gt; &gt; drivers/infiniband/hw/mthca/mthca_allocator.c: alloc-&gt;last left unchanged in mthca_alloc, which
> &gt; &gt; has impact on performance of function find_next_zero_bit in mthca_alloc.
> &gt; 
> &gt; I don't know what the sentence above means, but the change is unlikely
> &gt; to be correct.
> &gt; 
> &gt; When alloc-&gt;last starts to be equal to alloc-&gt;max, the
> &gt; find_next_zero_bit() will always return alloc-&gt;max. which will ensure
> &gt; that the following code is executed.
> &gt; 
> &gt;    48         if (obj &gt;= alloc-&gt;max) {
> &gt;    49                 alloc-&gt;top = (alloc-&gt;top + alloc-&gt;max) &amp; alloc-&gt;mask;
> &gt;    50                 obj = find_first_zero_bit(alloc-&gt;table, alloc-&gt;max);
> &gt;    51         }
> &gt; 
> 
> Thanks for your review.
> Yes, your analysis is right. However, this is a bitmap allocator for resource id allocation (like QP, and CQ). 
> We first look at the situation where no resource id is released.
> When the value of alloc-&gt;last starts to reaches alloc-&gt;max, the bitmap will 
> be full. In this case, it's normal to return an invalid value.
> 
> Now, let's add resource releasing into consideration. 
> The following code is part of mthca_free:
> 
> 70	spin_lock_irqsave(&amp;alloc-&gt;lock, flags);
> 71
> 72	clear_bit(obj, alloc-&gt;table);
> 73	alloc-&gt;last = min(alloc-&gt;last, obj);
> 74	alloc-&gt;top = (alloc-&gt;top + alloc-&gt;max) &amp; alloc-&gt;mask;
> 
> mthca_free() is used to release the allocated resource id. It will update alloc-&gt;last when obj is freed. 
> So, if the bitmap has space for allocation, my modification can continuously work.

After alloc->last starts to be equal to alloc->max, the bitmap is
searches with find_first_zero_bit() call. That will ensure that we look
for any bit between 0 and alloc->max.

So no, your change can't be right.

Thanks

> 
> &gt; 
> &gt; However the mthca_alloc() function has other error, it returns -1 while
> &gt; based on its declaration it needs to be unsigned,
> 
> I think you are right about this. But obj is the return value of u32 type, which is the requirement of resource id 
> (though it may not fully use 32 bits). I have no idea of how to fix it.
> 
> Thanks.
> 
> &gt; 
> &gt; Thanks
> &gt; 
> &gt; &gt; 
> &gt; &gt; Signed-off-by: kangning <kangning18z@ict.ac.cn>
> &gt; &gt; ---
> &gt; &gt;  
> &gt; &gt;  I squashed two commits into one in this version.
> &gt; &gt;  
> &gt; &gt;  drivers/infiniband/hw/mthca/mthca_allocator.c | 3 +++
> &gt; &gt;  1 file changed, 3 insertions(+)
> &gt; &gt; 
> &gt; &gt; diff --git a/drivers/infiniband/hw/mthca/mthca_allocator.c b/drivers/infiniband/hw/mthca/mthca_allocator.c
> &gt; &gt; index aef1d274a14e..1141695093e7 100644
> &gt; &gt; --- a/drivers/infiniband/hw/mthca/mthca_allocator.c
> &gt; &gt; +++ b/drivers/infiniband/hw/mthca/mthca_allocator.c
> &gt; &gt; @@ -51,6 +51,9 @@ u32 mthca_alloc(struct mthca_alloc *alloc)
> &gt; &gt;  	}
> &gt; &gt;  
> &gt; &gt;  	if (obj &lt; alloc-&gt;max) {
> &gt; &gt; +		alloc-&gt;last = obj + 1;
> &gt; &gt; +		if (alloc-&gt;last == alloc-&gt;max)
> &gt; &gt; +			alloc-&gt;last = 0;
> &gt; &gt;  		set_bit(obj, alloc-&gt;table);
> &gt; &gt;  		obj |= alloc-&gt;top;
> &gt; &gt;  	} else
> &gt; &gt; -- 
> &gt; &gt; 2.17.1
> &gt; &gt; 
> </kangning18z@ict.ac.cn>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Re: [PATCH v3] Fix one error in mthca_alloc
  2021-08-31 12:00     ` Leon Romanovsky
@ 2021-08-31 12:16       ` kangning18z
  0 siblings, 0 replies; 7+ messages in thread
From: kangning18z @ 2021-08-31 12:16 UTC (permalink / raw)
  To: leon; +Cc: linux-rdma

> On Tue, Aug 31, 2021 at 06:40:38PM +0800, 康宁 wrote:
> > &gt;
> > &gt; On Fri, Aug 27, 2021 at 08:52:28AM +0800, kangning wrote:
> > &gt; &gt; drivers/infiniband/hw/mthca/mthca_allocator.c: alloc-&gt;last left unchanged in mthca_alloc, which
> > &gt; &gt; has impact on performance of function find_next_zero_bit in mthca_alloc.
> > &gt;
> > &gt; I don't know what the sentence above means, but the change is unlikely
> > &gt; to be correct.
> > &gt;
> > &gt; When alloc-&gt;last starts to be equal to alloc-&gt;max, the
> > &gt; find_next_zero_bit() will always return alloc-&gt;max. which will ensure
> > &gt; that the following code is executed.
> > &gt;
> > &gt;    48         if (obj &gt;= alloc-&gt;max) {
> > &gt;    49                 alloc-&gt;top = (alloc-&gt;top + alloc-&gt;max) &amp; alloc-&gt;mask;
> > &gt;    50                 obj = find_first_zero_bit(alloc-&gt;table, alloc-&gt;max);
> > &gt;    51         }
> > &gt;
> >
> > Thanks for your review.
> > Yes, your analysis is right. However, this is a bitmap allocator for resource id allocation (like QP, and CQ).
> > We first look at the situation where no resource id is released.
> > When the value of alloc-&gt;last starts to reaches alloc-&gt;max, the bitmap will
> > be full. In this case, it's normal to return an invalid value.
> >
> > Now, let's add resource releasing into consideration.
> > The following code is part of mthca_free:
> >
> > 70	spin_lock_irqsave(&amp;alloc-&gt;lock, flags);
> > 71
> > 72	clear_bit(obj, alloc-&gt;table);
> > 73	alloc-&gt;last = min(alloc-&gt;last, obj);
> > 74	alloc-&gt;top = (alloc-&gt;top + alloc-&gt;max) &amp; alloc-&gt;mask;
> >
> > mthca_free() is used to release the allocated resource id. It will update alloc-&gt;last when obj is freed.
> > So, if the bitmap has space for allocation, my modification can continuously work.
> 
> After alloc->last starts to be equal to alloc->max, the bitmap is
> searches with find_first_zero_bit() call. That will ensure that we look
> for any bit between 0 and alloc->max.
> 
> So no, your change can't be right.

If you never call function mthca_free(), you will never find a valid bit  in [0, alloc->max]
with find_first_zero_bit() call. Cause every bits between 0 and alloc->max are alloacted.

But if you called mthca_free() before, the freed obj is assigned to alloc->last, and we 
still can allocate the bitmap.

My modification is correct. 

> 
> Thanks
> 
> >
> > &gt;
> > &gt; However the mthca_alloc() function has other error, it returns -1 while
> > &gt; based on its declaration it needs to be unsigned,
> >
> > I think you are right about this. But obj is the return value of u32 type, which is the requirement of resource id
> > (though it may not fully use 32 bits). I have no idea of how to fix it.
> >
> > Thanks.
> >
> > &gt;
> > &gt; Thanks
> > &gt;
> > &gt; &gt;
> > &gt; &gt; Signed-off-by: kangning <kangning18z@ict.ac.cn>
> > &gt; &gt; ---
> > &gt; &gt; 
> > &gt; &gt;  I squashed two commits into one in this version.
> > &gt; &gt; 
> > &gt; &gt;  drivers/infiniband/hw/mthca/mthca_allocator.c | 3 +++
> > &gt; &gt;  1 file changed, 3 insertions(+)
> > &gt; &gt;
> > &gt; &gt; diff --git a/drivers/infiniband/hw/mthca/mthca_allocator.c b/drivers/infiniband/hw/mthca/mthca_allocator.c
> > &gt; &gt; index aef1d274a14e..1141695093e7 100644
> > &gt; &gt; --- a/drivers/infiniband/hw/mthca/mthca_allocator.c
> > &gt; &gt; +++ b/drivers/infiniband/hw/mthca/mthca_allocator.c
> > &gt; &gt; @@ -51,6 +51,9 @@ u32 mthca_alloc(struct mthca_alloc *alloc)
> > &gt; &gt;  }
> > &gt; &gt; 
> > &gt; &gt;  if (obj &lt; alloc-&gt;max) {
> > &gt; &gt; +	alloc-&gt;last = obj + 1;
> > &gt; &gt; +	if (alloc-&gt;last == alloc-&gt;max)
> > &gt; &gt; +	alloc-&gt;last = 0;
> > &gt; &gt;  set_bit(obj, alloc-&gt;table);
> > &gt; &gt;  obj |= alloc-&gt;top;
> > &gt; &gt;  } else
> > &gt; &gt; --
> > &gt; &gt; 2.17.1
> > &gt; &gt;
> > </kangning18z@ict.ac.cn>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: Re: [PATCH v3] Fix one error in mthca_alloc
       [not found]   ` <202109011027459851120@ict.ac.cn>
@ 2021-09-01  7:51     ` Leon Romanovsky
  0 siblings, 0 replies; 7+ messages in thread
From: Leon Romanovsky @ 2021-09-01  7:51 UTC (permalink / raw)
  To: kangning18z; +Cc: haakon.bugge, linux-rdma

On Wed, Sep 01, 2021 at 10:27:46AM +0800, kangning18z@ict.ac.cn wrote:
> 

<...>

> Thank you for your review.

Do you have mthca card in hand to test your changes?

This driver didn't get any changes for a long time and we
don't even know if it works or not.

I would prefer to minimize changes to that driver.

Thanks

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2021-09-01  7:51 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-27  0:52 [PATCH v3] Fix one error in mthca_alloc kangning
2021-08-31  9:52 ` Leon Romanovsky
2021-08-31 10:40   ` 康宁
2021-08-31 12:00     ` Leon Romanovsky
2021-08-31 12:16       ` kangning18z
2021-08-31 10:47   ` 康宁
     [not found]   ` <202109011027459851120@ict.ac.cn>
2021-09-01  7:51     ` Leon Romanovsky

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).