* [PATCH] powerpc/mm: Add cond_resched() while removing hpte mappings
@ 2021-03-10 7:59 Vaibhav Jain
2021-03-10 9:14 ` Christophe Leroy
2021-03-10 9:17 ` Michael Ellerman
0 siblings, 2 replies; 4+ messages in thread
From: Vaibhav Jain @ 2021-03-10 7:59 UTC (permalink / raw)
To: linuxppc-dev; +Cc: Vaibhav Jain, Aneesh Kumar K . V
While removing large number of mappings from hash page tables for
large memory systems as soft-lockup is reported because of the time
spent inside htap_remove_mapping() like one below:
watchdog: BUG: soft lockup - CPU#8 stuck for 23s!
<snip>
NIP plpar_hcall+0x38/0x58
LR pSeries_lpar_hpte_invalidate+0x68/0xb0
Call Trace:
0x1fffffffffff000 (unreliable)
pSeries_lpar_hpte_removebolted+0x9c/0x230
hash__remove_section_mapping+0xec/0x1c0
remove_section_mapping+0x28/0x3c
arch_remove_memory+0xfc/0x150
devm_memremap_pages_release+0x180/0x2f0
devm_action_release+0x30/0x50
release_nodes+0x28c/0x300
device_release_driver_internal+0x16c/0x280
unbind_store+0x124/0x170
drv_attr_store+0x44/0x60
sysfs_kf_write+0x64/0x90
kernfs_fop_write+0x1b0/0x290
__vfs_write+0x3c/0x70
vfs_write+0xd4/0x270
ksys_write+0xdc/0x130
system_call+0x5c/0x70
Fix this by adding a cond_resched() to the loop in
htap_remove_mapping() that issues hcall to remove hpte mapping. This
should prevent the soft-lockup from being reported.
Suggested-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
---
arch/powerpc/mm/book3s64/hash_utils.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s64/hash_utils.c
index 581b20a2feaf..ea3945c70b18 100644
--- a/arch/powerpc/mm/book3s64/hash_utils.c
+++ b/arch/powerpc/mm/book3s64/hash_utils.c
@@ -359,6 +359,8 @@ int htab_remove_mapping(unsigned long vstart, unsigned long vend,
}
if (rc < 0)
return rc;
+
+ cond_resched();
}
return ret;
--
2.29.2
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] powerpc/mm: Add cond_resched() while removing hpte mappings
2021-03-10 7:59 [PATCH] powerpc/mm: Add cond_resched() while removing hpte mappings Vaibhav Jain
@ 2021-03-10 9:14 ` Christophe Leroy
2021-03-10 9:17 ` Michael Ellerman
1 sibling, 0 replies; 4+ messages in thread
From: Christophe Leroy @ 2021-03-10 9:14 UTC (permalink / raw)
To: Vaibhav Jain, linuxppc-dev; +Cc: Aneesh Kumar K . V
Le 10/03/2021 à 08:59, Vaibhav Jain a écrit :
> While removing large number of mappings from hash page tables for
> large memory systems as soft-lockup is reported because of the time
> spent inside htap_remove_mapping() like one below:
>
> watchdog: BUG: soft lockup - CPU#8 stuck for 23s!
> <snip>
> NIP plpar_hcall+0x38/0x58
> LR pSeries_lpar_hpte_invalidate+0x68/0xb0
> Call Trace:
> 0x1fffffffffff000 (unreliable)
> pSeries_lpar_hpte_removebolted+0x9c/0x230
> hash__remove_section_mapping+0xec/0x1c0
> remove_section_mapping+0x28/0x3c
> arch_remove_memory+0xfc/0x150
> devm_memremap_pages_release+0x180/0x2f0
> devm_action_release+0x30/0x50
> release_nodes+0x28c/0x300
> device_release_driver_internal+0x16c/0x280
> unbind_store+0x124/0x170
> drv_attr_store+0x44/0x60
> sysfs_kf_write+0x64/0x90
> kernfs_fop_write+0x1b0/0x290
> __vfs_write+0x3c/0x70
> vfs_write+0xd4/0x270
> ksys_write+0xdc/0x130
> system_call+0x5c/0x70
>
> Fix this by adding a cond_resched() to the loop in
> htap_remove_mapping() that issues hcall to remove hpte mapping. This
> should prevent the soft-lockup from being reported.
Isn't it overkill to call is at each iteration ?
Looking at a few other places, there is some mitigation. For instance fadump_free_reserved_memory()
does it based on elapsed time. Another exemple is drmem_lmb_next() doing it every 16 iteration.
>
> Suggested-by: Aneesh Kumar K.V <aneesh.kumar@linux.ibm.com>
> Signed-off-by: Vaibhav Jain <vaibhav@linux.ibm.com>
> ---
> arch/powerpc/mm/book3s64/hash_utils.c | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s64/hash_utils.c
> index 581b20a2feaf..ea3945c70b18 100644
> --- a/arch/powerpc/mm/book3s64/hash_utils.c
> +++ b/arch/powerpc/mm/book3s64/hash_utils.c
> @@ -359,6 +359,8 @@ int htab_remove_mapping(unsigned long vstart, unsigned long vend,
> }
> if (rc < 0)
> return rc;
> +
> + cond_resched();
> }
>
> return ret;
>
Christophe
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] powerpc/mm: Add cond_resched() while removing hpte mappings
2021-03-10 7:59 [PATCH] powerpc/mm: Add cond_resched() while removing hpte mappings Vaibhav Jain
2021-03-10 9:14 ` Christophe Leroy
@ 2021-03-10 9:17 ` Michael Ellerman
2021-04-06 4:30 ` Vaibhav Jain
1 sibling, 1 reply; 4+ messages in thread
From: Michael Ellerman @ 2021-03-10 9:17 UTC (permalink / raw)
To: Vaibhav Jain, linuxppc-dev; +Cc: Vaibhav Jain, Aneesh Kumar K . V
Vaibhav Jain <vaibhav@linux.ibm.com> writes:
> While removing large number of mappings from hash page tables for
> large memory systems as soft-lockup is reported because of the time
> spent inside htap_remove_mapping() like one below:
>
> watchdog: BUG: soft lockup - CPU#8 stuck for 23s!
> <snip>
> NIP plpar_hcall+0x38/0x58
> LR pSeries_lpar_hpte_invalidate+0x68/0xb0
> Call Trace:
> 0x1fffffffffff000 (unreliable)
> pSeries_lpar_hpte_removebolted+0x9c/0x230
> hash__remove_section_mapping+0xec/0x1c0
> remove_section_mapping+0x28/0x3c
> arch_remove_memory+0xfc/0x150
> devm_memremap_pages_release+0x180/0x2f0
> devm_action_release+0x30/0x50
> release_nodes+0x28c/0x300
> device_release_driver_internal+0x16c/0x280
> unbind_store+0x124/0x170
> drv_attr_store+0x44/0x60
> sysfs_kf_write+0x64/0x90
> kernfs_fop_write+0x1b0/0x290
> __vfs_write+0x3c/0x70
> vfs_write+0xd4/0x270
> ksys_write+0xdc/0x130
> system_call+0x5c/0x70
>
> Fix this by adding a cond_resched() to the loop in
> htap_remove_mapping() that issues hcall to remove hpte mapping. This
> should prevent the soft-lockup from being reported.
Can/should we also/instead be using H_BLOCK_REMOVE?
cheers
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] powerpc/mm: Add cond_resched() while removing hpte mappings
2021-03-10 9:17 ` Michael Ellerman
@ 2021-04-06 4:30 ` Vaibhav Jain
0 siblings, 0 replies; 4+ messages in thread
From: Vaibhav Jain @ 2021-04-06 4:30 UTC (permalink / raw)
To: Michael Ellerman, linuxppc-dev; +Cc: Aneesh Kumar K . V
Hi Mpe,
Thanks for looking into this patch.
Michael Ellerman <mpe@ellerman.id.au> writes:
> Vaibhav Jain <vaibhav@linux.ibm.com> writes:
>> While removing large number of mappings from hash page tables for
>> large memory systems as soft-lockup is reported because of the time
>> spent inside htap_remove_mapping() like one below:
>>
>> watchdog: BUG: soft lockup - CPU#8 stuck for 23s!
>> <snip>
>> NIP plpar_hcall+0x38/0x58
>> LR pSeries_lpar_hpte_invalidate+0x68/0xb0
>> Call Trace:
>> 0x1fffffffffff000 (unreliable)
>> pSeries_lpar_hpte_removebolted+0x9c/0x230
>> hash__remove_section_mapping+0xec/0x1c0
>> remove_section_mapping+0x28/0x3c
>> arch_remove_memory+0xfc/0x150
>> devm_memremap_pages_release+0x180/0x2f0
>> devm_action_release+0x30/0x50
>> release_nodes+0x28c/0x300
>> device_release_driver_internal+0x16c/0x280
>> unbind_store+0x124/0x170
>> drv_attr_store+0x44/0x60
>> sysfs_kf_write+0x64/0x90
>> kernfs_fop_write+0x1b0/0x290
>> __vfs_write+0x3c/0x70
>> vfs_write+0xd4/0x270
>> ksys_write+0xdc/0x130
>> system_call+0x5c/0x70
>>
>> Fix this by adding a cond_resched() to the loop in
>> htap_remove_mapping() that issues hcall to remove hpte mapping. This
>> should prevent the soft-lockup from being reported.
>
> Can/should we also/instead be using H_BLOCK_REMOVE?
>
> cheers
Current mmp_ops implementation seems to use H_BULK_REMOVE for hugepages
so for removing mappings for regular pages I am looking into adding a
new mmu_op that can take a range to be unmapped and
I did try implmenting a new mmu_op for this which can reduce the number
of hash_pte lookups needed to invalidate this range. But that would need
some more work so as a stop gap I have sent out a v2 with Christophe's
suggestion to add a cond_resched() every HZ interval.
--
Cheers
~ Vaibhav
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-04-06 4:31 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-10 7:59 [PATCH] powerpc/mm: Add cond_resched() while removing hpte mappings Vaibhav Jain
2021-03-10 9:14 ` Christophe Leroy
2021-03-10 9:17 ` Michael Ellerman
2021-04-06 4:30 ` Vaibhav Jain
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).