[PATCH] async: Let kfree() out of the critical area of the lock

[PATCH] async: Let kfree() out of the critical area of the lock

[Yunfeng Ye]

It's not necessary to put kfree() in the critical area of the lock, so
let it out.

Signed-off-by: Yunfeng Ye <yeyunfeng@huawei.com>
---
 kernel/async.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/kernel/async.c b/kernel/async.c
index 4f9c1d6..1de270d 100644
--- a/kernel/async.c
+++ b/kernel/async.c
@@ -135,12 +135,12 @@ static void async_run_entry_fn(struct work_struct *work)
 	list_del_init(&entry->domain_list);
 	list_del_init(&entry->global_list);

-	/* 3) free the entry */
-	kfree(entry);
 	atomic_dec(&entry_count);
-
 	spin_unlock_irqrestore(&async_lock, flags);

+	/* 3) free the entry */
+	kfree(entry);
+
 	/* 4) wake up any waiters */
 	wake_up(&async_done);
 }
-- 
2.7.4

Re: [PATCH] async: Let kfree() out of the critical area of the lock

[gregkh]

On Wed, Sep 25, 2019 at 08:52:26PM +0800, Yunfeng Ye wrote:
> It's not necessary to put kfree() in the critical area of the lock, so
> let it out.
> 
> Signed-off-by: Yunfeng Ye <yeyunfeng@huawei.com>
> ---
>  kernel/async.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/kernel/async.c b/kernel/async.c
> index 4f9c1d6..1de270d 100644
> --- a/kernel/async.c
> +++ b/kernel/async.c
> @@ -135,12 +135,12 @@ static void async_run_entry_fn(struct work_struct *work)
>  	list_del_init(&entry->domain_list);
>  	list_del_init(&entry->global_list);
> 
> -	/* 3) free the entry */
> -	kfree(entry);
>  	atomic_dec(&entry_count);
> -
>  	spin_unlock_irqrestore(&async_lock, flags);
> 
> +	/* 3) free the entry */
> +	kfree(entry);
> +
>  	/* 4) wake up any waiters */
>  	wake_up(&async_done);
>  }
> -- 
> 2.7.4
> 

Does this result any any measurable performance changes?

thanks,

greg k-h

Re: [PATCH] async: Let kfree() out of the critical area of the lock

[Yunfeng Ye]

On 2019/9/25 21:38, gregkh@linuxfoundation.org wrote:
> On Wed, Sep 25, 2019 at 08:52:26PM +0800, Yunfeng Ye wrote:
>> It's not necessary to put kfree() in the critical area of the lock, so
>> let it out.
>>
>> Signed-off-by: Yunfeng Ye <yeyunfeng@huawei.com>
>> ---
>>  kernel/async.c | 6 +++---
>>  1 file changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/kernel/async.c b/kernel/async.c
>> index 4f9c1d6..1de270d 100644
>> --- a/kernel/async.c
>> +++ b/kernel/async.c
>> @@ -135,12 +135,12 @@ static void async_run_entry_fn(struct work_struct *work)
>>  	list_del_init(&entry->domain_list);
>>  	list_del_init(&entry->global_list);
>>
>> -	/* 3) free the entry */
>> -	kfree(entry);
>>  	atomic_dec(&entry_count);
>> -
>>  	spin_unlock_irqrestore(&async_lock, flags);
>>
>> +	/* 3) free the entry */
>> +	kfree(entry);
>> +
>>  	/* 4) wake up any waiters */
>>  	wake_up(&async_done);
>>  }
>> -- 
>> 2.7.4
>>
> 
> Does this result any any measurable performance changes?
> 
No performance has been Measured at present, I just want the critical area
to be as short as possible. I think it's good to put it outside.

thanks

> thanks,
> 
> greg k-h
> 
> .
> 

Re: [PATCH] async: Let kfree() out of the critical area of the lock

[Alexander Duyck]

On Wed, 2019-09-25 at 20:52 +0800, Yunfeng Ye wrote:
> It's not necessary to put kfree() in the critical area of the lock, so
> let it out.
> 
> Signed-off-by: Yunfeng Ye <yeyunfeng@huawei.com>
> ---
>  kernel/async.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/kernel/async.c b/kernel/async.c
> index 4f9c1d6..1de270d 100644
> --- a/kernel/async.c
> +++ b/kernel/async.c
> @@ -135,12 +135,12 @@ static void async_run_entry_fn(struct work_struct *work)
>  	list_del_init(&entry->domain_list);
>  	list_del_init(&entry->global_list);
> 
> -	/* 3) free the entry */
> -	kfree(entry);
>  	atomic_dec(&entry_count);
> -
>  	spin_unlock_irqrestore(&async_lock, flags);
> 
> +	/* 3) free the entry */
> +	kfree(entry);
> +
>  	/* 4) wake up any waiters */
>  	wake_up(&async_done);
>  }

It probably wouldn't hurt to update the patch description to mention that
async_schedule_node_domain does the allocation outside of the lock, then
takes the lock and does the list addition and entry_count increment inside
the critical section so this is just updating the code to match that it
seems.

Otherwise the change itself looks safe to me, though I am not sure there
is a performance gain to be had so this is mostly just a cosmetic patch.

Reviewed-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>

Re: [PATCH] async: Let kfree() out of the critical area of the lock

[Yunfeng Ye]

On 2019/9/25 23:20, Alexander Duyck wrote:
> On Wed, 2019-09-25 at 20:52 +0800, Yunfeng Ye wrote:
>> It's not necessary to put kfree() in the critical area of the lock, so
>> let it out.
>>
>> Signed-off-by: Yunfeng Ye <yeyunfeng@huawei.com>
>> ---
>>  kernel/async.c | 6 +++---
>>  1 file changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/kernel/async.c b/kernel/async.c
>> index 4f9c1d6..1de270d 100644
>> --- a/kernel/async.c
>> +++ b/kernel/async.c
>> @@ -135,12 +135,12 @@ static void async_run_entry_fn(struct work_struct *work)
>>  	list_del_init(&entry->domain_list);
>>  	list_del_init(&entry->global_list);
>>
>> -	/* 3) free the entry */
>> -	kfree(entry);
>>  	atomic_dec(&entry_count);
>> -
>>  	spin_unlock_irqrestore(&async_lock, flags);
>>
>> +	/* 3) free the entry */
>> +	kfree(entry);
>> +
>>  	/* 4) wake up any waiters */
>>  	wake_up(&async_done);
>>  }
> 
> It probably wouldn't hurt to update the patch description to mention that
> async_schedule_node_domain does the allocation outside of the lock, then
> takes the lock and does the list addition and entry_count increment inside
> the critical section so this is just updating the code to match that it
> seems.
> 
> Otherwise the change itself looks safe to me, though I am not sure there
> is a performance gain to be had so this is mostly just a cosmetic patch.
> 
The async_lock is big global lock, I think it's good to put kfree() outside
to keep the critical area as short as possible.

thanks.

> Reviewed-by: Alexander Duyck <alexander.h.duyck@linux.intel.com>
> 
> 
> .
> 

Re: [PATCH] async: Let kfree() out of the critical area of the lock

[David Sterba]

On Thu, Sep 26, 2019 at 03:58:36PM +0800, Yunfeng Ye wrote:
> On 2019/9/25 23:20, Alexander Duyck wrote:
> > On Wed, 2019-09-25 at 20:52 +0800, Yunfeng Ye wrote:
> > It probably wouldn't hurt to update the patch description to mention that
> > async_schedule_node_domain does the allocation outside of the lock, then
> > takes the lock and does the list addition and entry_count increment inside
> > the critical section so this is just updating the code to match that it
> > seems.
> > 
> > Otherwise the change itself looks safe to me, though I am not sure there
> > is a performance gain to be had so this is mostly just a cosmetic patch.
> > 
> The async_lock is big global lock, I think it's good to put kfree() outside
> to keep the critical area as short as possible.

Agreed, kfree is not always cheap. We had patches in btrfs moving kfree
out of critical section(s) after causing softlockups due to increased lock
contention.

Re: [PATCH] async: Let kfree() out of the critical area of the lock

[Bart Van Assche]

On 9/26/19 4:06 AM, David Sterba wrote:
> On Thu, Sep 26, 2019 at 03:58:36PM +0800, Yunfeng Ye wrote:
>> The async_lock is big global lock, I think it's good to put kfree() outside
>> to keep the critical area as short as possible.
> 
> Agreed, kfree is not always cheap. We had patches in btrfs moving kfree
> out of critical section(s) after causing softlockups due to increased lock
> contention.

The above would be a great addition for the commit description. Anyway:

Reviewed-by: Bart Van Assche <bvanassche@acm.org>

Re: [PATCH] async: Let kfree() out of the critical area of the lock

[Yunfeng Ye]

On 2019/9/26 23:18, Bart Van Assche wrote:
> On 9/26/19 4:06 AM, David Sterba wrote:
>> On Thu, Sep 26, 2019 at 03:58:36PM +0800, Yunfeng Ye wrote:
>>> The async_lock is big global lock, I think it's good to put kfree() outside
>>> to keep the critical area as short as possible.
>>
>> Agreed, kfree is not always cheap. We had patches in btrfs moving kfree
>> out of critical section(s) after causing softlockups due to increased lock
>> contention.
> 
> The above would be a great addition for the commit description. Anyway:
> 
ok, I will update the description, thanks.

> Reviewed-by: Bart Van Assche <bvanassche@acm.org>
> 
> .
>