All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] PM: hibernate: Fix level3 translation fault in swsusp_save()
@ 2024-02-26  3:42 Yaxiong Tian
  2024-02-26  8:37 ` David Hildenbrand
  0 siblings, 1 reply; 14+ messages in thread
From: Yaxiong Tian @ 2024-02-26  3:42 UTC (permalink / raw)
  To: rafael, pavel, len.brown, keescook, tony.luck, gpiccoli, akpm,
	ardb, rppt, david, wangkefeng.wang, catalin.marinas, will
  Cc: linux-pm, linux-kernel, linux-hardening, Yaxiong Tian, xiongxin

From: Yaxiong Tian <tianyaxiong@kylinos.cn>

On ARM64 machines using UEFI, if the linear map is not set (can_set_direct_map() 
return false), swsusp_save() will fail due to can't finding the map table 
under the nomap memory.such as:

[   48.532162] Unable to handle kernel paging request at virtual address ffffff8000000000
[   48.532162] Mem abort info:
[   48.532162]   ESR = 0x0000000096000007
[   48.532162]   EC = 0x25: DABT (current EL), IL = 32 bits
[   48.532162]   SET = 0, FnV = 0
[   48.532162]   EA = 0, S1PTW = 0
[   48.532162]   FSC = 0x07: level 3 translation fault
[   48.532162] Data abort info:
[   48.532162]   ISV = 0, ISS = 0x00000007, ISS2 = 0x00000000
[   48.532162]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[   48.532162]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[   48.532162] swapper pgtable: 4k pages, 39-bit VAs, pgdp=00000000eeb0b000
[   48.532162] [ffffff8000000000] pgd=180000217fff9803, p4d=180000217fff9803, pud=180000217fff9803, pmd=180000217fff8803, pte=0000000000000000
[   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
[   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
[   48.532162] Modules linked in: xt_multiport ipt_REJECT nf_reject_ipv4 xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter bpfilter rfkill at803x snd_hda_codec_hdmi snd_hda_intel snd_intel_dspcfg dwmac_generic stmmac_platform snd_hda_codec stmmac joydev pcs_xpcs snd_hda_core phylink ppdev lp parport ramoops reed_solomon ip_tables x_tables nls_iso8859_1 vfat multipath linear amdgpu amdxcp drm_exec gpu_sched drm_buddy hid_generic usbhid hid radeon video drm_suballoc_helper drm_ttm_helper ttm i2c_algo_bit drm_display_helper cec drm_kms_helper drm
[   48.532162] CPU: 0 PID: 3663 Comm: systemd-sleep Not tainted 6.6.2+ #76
[   48.532162] Source Version: 4e22ed63a0a48e7a7cff9b98b7806d8d4add7dc0
[   48.532162] Hardware name: Greatwall GW-XXXXXX-XXX/GW-XXXXXX-XXX, BIOS KunLun BIOS V4.0 01/19/2021
[   48.532162] pstate: 600003c5 (nZCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   48.532162] pc : swsusp_save+0x280/0x538
[   48.532162] lr : swsusp_save+0x280/0x538
[   48.532162] sp : ffffffa034a3fa40
[   48.532162] x29: ffffffa034a3fa40 x28: ffffff8000001000 x27: 0000000000000000
[   48.532162] x26: ffffff8001400000 x25: ffffffc08113e248 x24: 0000000000000000
[   48.532162] x23: 0000000000080000 x22: ffffffc08113e280 x21: 00000000000c69f2
[   48.532162] x20: ffffff8000000000 x19: ffffffc081ae2500 x18: 0000000000000000
[   48.532162] x17: 6666662074736420 x16: 3030303030303030 x15: 3038666666666666
[   48.532162] x14: 0000000000000b69 x13: ffffff9f89088530 x12: 00000000ffffffea
[   48.532162] x11: 00000000ffff7fff x10: 00000000ffff7fff x9 : ffffffc08193f0d0
[   48.532162] x8 : 00000000000bffe8 x7 : c0000000ffff7fff x6 : 0000000000000001
[   48.532162] x5 : ffffffa0fff09dc8 x4 : 0000000000000000 x3 : 0000000000000027
[   48.532162] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 000000000000004e
[   48.532162] Call trace:
[   48.532162]  swsusp_save+0x280/0x538
[   48.532162]  swsusp_arch_suspend+0x148/0x190
[   48.532162]  hibernation_snapshot+0x240/0x39c
[   48.532162]  hibernate+0xc4/0x378
[   48.532162]  state_store+0xf0/0x10c
[   48.532162]  kobj_attr_store+0x14/0x24

QEMU ARM64 using UEFI also has the problem by setting can_set_direct_map() 
return false.

Since the NOMAP regions are now marked as PageReserved(), pfn walkers
and the rest of core mm will treat them as unusable memory. So this
regions should not saved in hibernation.

This problem may cause by changes to pfn_valid() logic in commit
a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()").

So to fix it, we add pfn_is_map_memory() check in saveable_page(). It
make such regisons don't save in hibernation.

Fixes: a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()")
Co-developed-by: xiongxin <xiongxin@kylinos.cn>
Signed-off-by: xiongxin <xiongxin@kylinos.cn>
Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>
---
 kernel/power/snapshot.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c
index 0f12e0a97e43..a06e3b1869d2 100644
--- a/kernel/power/snapshot.c
+++ b/kernel/power/snapshot.c
@@ -1400,7 +1400,7 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)
 		return NULL;
 
 	if (PageReserved(page)
-	    && (!kernel_page_present(page) || pfn_is_nosave(pfn)))
+	    && (!kernel_page_present(page) || pfn_is_nosave(pfn) || !pfn_is_map_memory(pfn)))
 		return NULL;
 
 	if (page_is_guard(page))
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* Re: [PATCH] PM: hibernate: Fix level3 translation fault in swsusp_save()
  2024-02-26  3:42 [PATCH] PM: hibernate: Fix level3 translation fault in swsusp_save() Yaxiong Tian
@ 2024-02-26  8:37 ` David Hildenbrand
  2024-02-26  9:14   ` Mike Rapoport
  0 siblings, 1 reply; 14+ messages in thread
From: David Hildenbrand @ 2024-02-26  8:37 UTC (permalink / raw)
  To: Yaxiong Tian, rafael, pavel, len.brown, keescook, tony.luck,
	gpiccoli, akpm, ardb, rppt, wangkefeng.wang, catalin.marinas,
	will
  Cc: linux-pm, linux-kernel, linux-hardening, Yaxiong Tian, xiongxin

On 26.02.24 04:42, Yaxiong Tian wrote:
> From: Yaxiong Tian <tianyaxiong@kylinos.cn>
> 
> On ARM64 machines using UEFI, if the linear map is not set (can_set_direct_map()
> return false), swsusp_save() will fail due to can't finding the map table
> under the nomap memory.such as:
> 
> [   48.532162] Unable to handle kernel paging request at virtual address ffffff8000000000
> [   48.532162] Mem abort info:
> [   48.532162]   ESR = 0x0000000096000007
> [   48.532162]   EC = 0x25: DABT (current EL), IL = 32 bits
> [   48.532162]   SET = 0, FnV = 0
> [   48.532162]   EA = 0, S1PTW = 0
> [   48.532162]   FSC = 0x07: level 3 translation fault
> [   48.532162] Data abort info:
> [   48.532162]   ISV = 0, ISS = 0x00000007, ISS2 = 0x00000000
> [   48.532162]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> [   48.532162]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> [   48.532162] swapper pgtable: 4k pages, 39-bit VAs, pgdp=00000000eeb0b000
> [   48.532162] [ffffff8000000000] pgd=180000217fff9803, p4d=180000217fff9803, pud=180000217fff9803, pmd=180000217fff8803, pte=0000000000000000
> [   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
> [   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
> [   48.532162] Modules linked in: xt_multiport ipt_REJECT nf_reject_ipv4 xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter bpfilter rfkill at803x snd_hda_codec_hdmi snd_hda_intel snd_intel_dspcfg dwmac_generic stmmac_platform snd_hda_codec stmmac joydev pcs_xpcs snd_hda_core phylink ppdev lp parport ramoops reed_solomon ip_tables x_tables nls_iso8859_1 vfat multipath linear amdgpu amdxcp drm_exec gpu_sched drm_buddy hid_generic usbhid hid radeon video drm_suballoc_helper drm_ttm_helper ttm i2c_algo_bit drm_display_helper cec drm_kms_helper drm
> [   48.532162] CPU: 0 PID: 3663 Comm: systemd-sleep Not tainted 6.6.2+ #76
> [   48.532162] Source Version: 4e22ed63a0a48e7a7cff9b98b7806d8d4add7dc0
> [   48.532162] Hardware name: Greatwall GW-XXXXXX-XXX/GW-XXXXXX-XXX, BIOS KunLun BIOS V4.0 01/19/2021
> [   48.532162] pstate: 600003c5 (nZCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [   48.532162] pc : swsusp_save+0x280/0x538
> [   48.532162] lr : swsusp_save+0x280/0x538
> [   48.532162] sp : ffffffa034a3fa40
> [   48.532162] x29: ffffffa034a3fa40 x28: ffffff8000001000 x27: 0000000000000000
> [   48.532162] x26: ffffff8001400000 x25: ffffffc08113e248 x24: 0000000000000000
> [   48.532162] x23: 0000000000080000 x22: ffffffc08113e280 x21: 00000000000c69f2
> [   48.532162] x20: ffffff8000000000 x19: ffffffc081ae2500 x18: 0000000000000000
> [   48.532162] x17: 6666662074736420 x16: 3030303030303030 x15: 3038666666666666
> [   48.532162] x14: 0000000000000b69 x13: ffffff9f89088530 x12: 00000000ffffffea
> [   48.532162] x11: 00000000ffff7fff x10: 00000000ffff7fff x9 : ffffffc08193f0d0
> [   48.532162] x8 : 00000000000bffe8 x7 : c0000000ffff7fff x6 : 0000000000000001
> [   48.532162] x5 : ffffffa0fff09dc8 x4 : 0000000000000000 x3 : 0000000000000027
> [   48.532162] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 000000000000004e
> [   48.532162] Call trace:
> [   48.532162]  swsusp_save+0x280/0x538
> [   48.532162]  swsusp_arch_suspend+0x148/0x190
> [   48.532162]  hibernation_snapshot+0x240/0x39c
> [   48.532162]  hibernate+0xc4/0x378
> [   48.532162]  state_store+0xf0/0x10c
> [   48.532162]  kobj_attr_store+0x14/0x24
> 
> QEMU ARM64 using UEFI also has the problem by setting can_set_direct_map()
> return false.
> 
> Since the NOMAP regions are now marked as PageReserved(), pfn walkers
> and the rest of core mm will treat them as unusable memory. So this
> regions should not saved in hibernation.
> 
> This problem may cause by changes to pfn_valid() logic in commit
> a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()").
> 
> So to fix it, we add pfn_is_map_memory() check in saveable_page(). It
> make such regisons don't save in hibernation.
> 
> Fixes: a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()")
> Co-developed-by: xiongxin <xiongxin@kylinos.cn>
> Signed-off-by: xiongxin <xiongxin@kylinos.cn>
> Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>
> ---
>   kernel/power/snapshot.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c
> index 0f12e0a97e43..a06e3b1869d2 100644
> --- a/kernel/power/snapshot.c
> +++ b/kernel/power/snapshot.c
> @@ -1400,7 +1400,7 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)
>   		return NULL;
>   
>   	if (PageReserved(page)
> -	    && (!kernel_page_present(page) || pfn_is_nosave(pfn)))
> +	    && (!kernel_page_present(page) || pfn_is_nosave(pfn) || !pfn_is_map_memory(pfn)))
>   		return NULL;
>   
>   	if (page_is_guard(page))

On top of which tree does this apply?

All occurrences of pfn_is_map_memory() are in arch/arm64, how does this 
compile on other architectures?

-- 
Cheers,

David / dhildenb


^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH] PM: hibernate: Fix level3 translation fault in swsusp_save()
  2024-02-26  8:37 ` David Hildenbrand
@ 2024-02-26  9:14   ` Mike Rapoport
  2024-02-27  7:51     ` Yaxiong Tian
  0 siblings, 1 reply; 14+ messages in thread
From: Mike Rapoport @ 2024-02-26  9:14 UTC (permalink / raw)
  To: David Hildenbrand, Yaxiong Tian
  Cc: rafael, pavel, len.brown, keescook, tony.luck, gpiccoli, akpm,
	ardb, wangkefeng.wang, catalin.marinas, will, linux-pm,
	linux-kernel, linux-hardening, Yaxiong Tian, xiongxin

On Mon, Feb 26, 2024 at 09:37:06AM +0100, David Hildenbrand wrote:
> On 26.02.24 04:42, Yaxiong Tian wrote:
> > From: Yaxiong Tian <tianyaxiong@kylinos.cn>
> > 
> > On ARM64 machines using UEFI, if the linear map is not set (can_set_direct_map()
> > return false), swsusp_save() will fail due to can't finding the map table
> > under the nomap memory.such as:

can_set_direct_map() has nothing to do with presence or absence of the
linear map.

Do you mean that kernel_page_present() presumes that a page is present when
can_set_direct_map() returns false even for NOMAP ranges?

> > [   48.532162] Unable to handle kernel paging request at virtual address ffffff8000000000
> > [   48.532162] Mem abort info:
> > [   48.532162]   ESR = 0x0000000096000007
> > [   48.532162]   EC = 0x25: DABT (current EL), IL = 32 bits
> > [   48.532162]   SET = 0, FnV = 0
> > [   48.532162]   EA = 0, S1PTW = 0
> > [   48.532162]   FSC = 0x07: level 3 translation fault
> > [   48.532162] Data abort info:
> > [   48.532162]   ISV = 0, ISS = 0x00000007, ISS2 = 0x00000000
> > [   48.532162]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
> > [   48.532162]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
> > [   48.532162] swapper pgtable: 4k pages, 39-bit VAs, pgdp=00000000eeb0b000
> > [   48.532162] [ffffff8000000000] pgd=180000217fff9803, p4d=180000217fff9803, pud=180000217fff9803, pmd=180000217fff8803, pte=0000000000000000
> > [   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
> > [   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
> > [   48.532162] Modules linked in: xt_multiport ipt_REJECT nf_reject_ipv4 xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter bpfilter rfkill at803x snd_hda_codec_hdmi snd_hda_intel snd_intel_dspcfg dwmac_generic stmmac_platform snd_hda_codec stmmac joydev pcs_xpcs snd_hda_core phylink ppdev lp parport ramoops reed_solomon ip_tables x_tables nls_iso8859_1 vfat multipath linear amdgpu amdxcp drm_exec gpu_sched drm_buddy hid_generic usbhid hid radeon video drm_suballoc_helper drm_ttm_helper ttm i2c_algo_bit drm_display_helper cec drm_kms_helper drm
> > [   48.532162] CPU: 0 PID: 3663 Comm: systemd-sleep Not tainted 6.6.2+ #76
> > [   48.532162] Source Version: 4e22ed63a0a48e7a7cff9b98b7806d8d4add7dc0
> > [   48.532162] Hardware name: Greatwall GW-XXXXXX-XXX/GW-XXXXXX-XXX, BIOS KunLun BIOS V4.0 01/19/2021
> > [   48.532162] pstate: 600003c5 (nZCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> > [   48.532162] pc : swsusp_save+0x280/0x538
> > [   48.532162] lr : swsusp_save+0x280/0x538
> > [   48.532162] sp : ffffffa034a3fa40
> > [   48.532162] x29: ffffffa034a3fa40 x28: ffffff8000001000 x27: 0000000000000000
> > [   48.532162] x26: ffffff8001400000 x25: ffffffc08113e248 x24: 0000000000000000
> > [   48.532162] x23: 0000000000080000 x22: ffffffc08113e280 x21: 00000000000c69f2
> > [   48.532162] x20: ffffff8000000000 x19: ffffffc081ae2500 x18: 0000000000000000
> > [   48.532162] x17: 6666662074736420 x16: 3030303030303030 x15: 3038666666666666
> > [   48.532162] x14: 0000000000000b69 x13: ffffff9f89088530 x12: 00000000ffffffea
> > [   48.532162] x11: 00000000ffff7fff x10: 00000000ffff7fff x9 : ffffffc08193f0d0
> > [   48.532162] x8 : 00000000000bffe8 x7 : c0000000ffff7fff x6 : 0000000000000001
> > [   48.532162] x5 : ffffffa0fff09dc8 x4 : 0000000000000000 x3 : 0000000000000027
> > [   48.532162] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 000000000000004e
> > [   48.532162] Call trace:
> > [   48.532162]  swsusp_save+0x280/0x538
> > [   48.532162]  swsusp_arch_suspend+0x148/0x190
> > [   48.532162]  hibernation_snapshot+0x240/0x39c
> > [   48.532162]  hibernate+0xc4/0x378
> > [   48.532162]  state_store+0xf0/0x10c
> > [   48.532162]  kobj_attr_store+0x14/0x24
> > 
> > QEMU ARM64 using UEFI also has the problem by setting can_set_direct_map()
> > return false.

Huh?
Why would you do that?

> > 
> > Since the NOMAP regions are now marked as PageReserved(), pfn walkers
> > and the rest of core mm will treat them as unusable memory. So this
> > regions should not saved in hibernation.
> > 
> > This problem may cause by changes to pfn_valid() logic in commit
> > a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()").
> > 
> > So to fix it, we add pfn_is_map_memory() check in saveable_page(). It
> > make such regisons don't save in hibernation.
> > 
> > Fixes: a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()")
> > Co-developed-by: xiongxin <xiongxin@kylinos.cn>
> > Signed-off-by: xiongxin <xiongxin@kylinos.cn>
> > Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>
> > ---
> >   kernel/power/snapshot.c | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c
> > index 0f12e0a97e43..a06e3b1869d2 100644
> > --- a/kernel/power/snapshot.c
> > +++ b/kernel/power/snapshot.c
> > @@ -1400,7 +1400,7 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)
> >   		return NULL;
> >   	if (PageReserved(page)
> > -	    && (!kernel_page_present(page) || pfn_is_nosave(pfn)))
> > +	    && (!kernel_page_present(page) || pfn_is_nosave(pfn) || !pfn_is_map_memory(pfn)))

I think adding the check for !pfn_is_map_memory() to arm64::pfn_is_nosave()
is the best way to fix this.

> >   		return NULL;
> >   	if (page_is_guard(page))
> 
> On top of which tree does this apply?
> 
> All occurrences of pfn_is_map_memory() are in arch/arm64, how does this
> compile on other architectures?
> 
> -- 
> Cheers,
> 
> David / dhildenb
> 

-- 
Sincerely yours,
Mike.

^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH] PM: hibernate: Fix level3 translation fault in swsusp_save()
  2024-02-26  9:14   ` Mike Rapoport
@ 2024-02-27  7:51     ` Yaxiong Tian
  2024-02-29  9:42       ` Mike Rapoport
  0 siblings, 1 reply; 14+ messages in thread
From: Yaxiong Tian @ 2024-02-27  7:51 UTC (permalink / raw)
  To: Mike Rapoport, David Hildenbrand
  Cc: rafael, pavel, len.brown, keescook, tony.luck, gpiccoli, akpm,
	ardb, wangkefeng.wang, catalin.marinas, will, linux-pm,
	linux-kernel, linux-hardening, Yaxiong Tian, xiongxin


在 2024/2/26 17:14, Mike Rapoport 写道:
> On Mon, Feb 26, 2024 at 09:37:06AM +0100, David Hildenbrand wrote:
>> On 26.02.24 04:42, Yaxiong Tian wrote:
>>> From: Yaxiong Tian <tianyaxiong@kylinos.cn>
>>>
>>> On ARM64 machines using UEFI, if the linear map is not set (can_set_direct_map()
>>> return false), swsusp_save() will fail due to can't finding the map table
>>> under the nomap memory.such as:
> can_set_direct_map() has nothing to do with presence or absence of the
> linear map.
>
> Do you mean that kernel_page_present() presumes that a page is present when
> can_set_direct_map() returns false even for NOMAP ranges?
Yes， in swsusp_save()->copy_data_pages()->page_is_saveable(),
kernel_page_present() presumes that a page is present when 
can_set_direct_map()
returns false even for NOMAP ranges.So NOMAP pages will saved in 
after,and then
cause level3 translation fault in this pages.
>>> [   48.532162] Unable to handle kernel paging request at virtual address ffffff8000000000
>>> [   48.532162] Mem abort info:
>>> [   48.532162]   ESR = 0x0000000096000007
>>> [   48.532162]   EC = 0x25: DABT (current EL), IL = 32 bits
>>> [   48.532162]   SET = 0, FnV = 0
>>> [   48.532162]   EA = 0, S1PTW = 0
>>> [   48.532162]   FSC = 0x07: level 3 translation fault
>>> [   48.532162] Data abort info:
>>> [   48.532162]   ISV = 0, ISS = 0x00000007, ISS2 = 0x00000000
>>> [   48.532162]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
>>> [   48.532162]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
>>> [   48.532162] swapper pgtable: 4k pages, 39-bit VAs, pgdp=00000000eeb0b000
>>> [   48.532162] [ffffff8000000000] pgd=180000217fff9803, p4d=180000217fff9803, pud=180000217fff9803, pmd=180000217fff8803, pte=0000000000000000
>>> [   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
>>> [   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
>>> [   48.532162] Modules linked in: xt_multiport ipt_REJECT nf_reject_ipv4 xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter bpfilter rfkill at803x snd_hda_codec_hdmi snd_hda_intel snd_intel_dspcfg dwmac_generic stmmac_platform snd_hda_codec stmmac joydev pcs_xpcs snd_hda_core phylink ppdev lp parport ramoops reed_solomon ip_tables x_tables nls_iso8859_1 vfat multipath linear amdgpu amdxcp drm_exec gpu_sched drm_buddy hid_generic usbhid hid radeon video drm_suballoc_helper drm_ttm_helper ttm i2c_algo_bit drm_display_helper cec drm_kms_helper drm
>>> [   48.532162] CPU: 0 PID: 3663 Comm: systemd-sleep Not tainted 6.6.2+ #76
>>> [   48.532162] Source Version: 4e22ed63a0a48e7a7cff9b98b7806d8d4add7dc0
>>> [   48.532162] Hardware name: Greatwall GW-XXXXXX-XXX/GW-XXXXXX-XXX, BIOS KunLun BIOS V4.0 01/19/2021
>>> [   48.532162] pstate: 600003c5 (nZCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
>>> [   48.532162] pc : swsusp_save+0x280/0x538
>>> [   48.532162] lr : swsusp_save+0x280/0x538
>>> [   48.532162] sp : ffffffa034a3fa40
>>> [   48.532162] x29: ffffffa034a3fa40 x28: ffffff8000001000 x27: 0000000000000000
>>> [   48.532162] x26: ffffff8001400000 x25: ffffffc08113e248 x24: 0000000000000000
>>> [   48.532162] x23: 0000000000080000 x22: ffffffc08113e280 x21: 00000000000c69f2
>>> [   48.532162] x20: ffffff8000000000 x19: ffffffc081ae2500 x18: 0000000000000000
>>> [   48.532162] x17: 6666662074736420 x16: 3030303030303030 x15: 3038666666666666
>>> [   48.532162] x14: 0000000000000b69 x13: ffffff9f89088530 x12: 00000000ffffffea
>>> [   48.532162] x11: 00000000ffff7fff x10: 00000000ffff7fff x9 : ffffffc08193f0d0
>>> [   48.532162] x8 : 00000000000bffe8 x7 : c0000000ffff7fff x6 : 0000000000000001
>>> [   48.532162] x5 : ffffffa0fff09dc8 x4 : 0000000000000000 x3 : 0000000000000027
>>> [   48.532162] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 000000000000004e
>>> [   48.532162] Call trace:
>>> [   48.532162]  swsusp_save+0x280/0x538
>>> [   48.532162]  swsusp_arch_suspend+0x148/0x190
>>> [   48.532162]  hibernation_snapshot+0x240/0x39c
>>> [   48.532162]  hibernate+0xc4/0x378
>>> [   48.532162]  state_store+0xf0/0x10c
>>> [   48.532162]  kobj_attr_store+0x14/0x24
>>>
>>> QEMU ARM64 using UEFI also has the problem by setting can_set_direct_map()
>>> return false.
> Huh?
> Why would you do that?
I discovered this problem when upgrading from 5.4 to 6.6 using the 5.4 
configuration.
So I using latest linux-next code,find the problem still exist.To rule 
out the effects
of a particular machine，I also use qemu to check it.
>
>>> Since the NOMAP regions are now marked as PageReserved(), pfn walkers
>>> and the rest of core mm will treat them as unusable memory. So this
>>> regions should not saved in hibernation.
>>>
>>> This problem may cause by changes to pfn_valid() logic in commit
>>> a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()").
>>>
>>> So to fix it, we add pfn_is_map_memory() check in saveable_page(). It
>>> make such regisons don't save in hibernation.
>>>
>>> Fixes: a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()")
>>> Co-developed-by: xiongxin <xiongxin@kylinos.cn>
>>> Signed-off-by: xiongxin <xiongxin@kylinos.cn>
>>> Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>
>>> ---
>>>    kernel/power/snapshot.c | 2 +-
>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c
>>> index 0f12e0a97e43..a06e3b1869d2 100644
>>> --- a/kernel/power/snapshot.c
>>> +++ b/kernel/power/snapshot.c
>>> @@ -1400,7 +1400,7 @@ static struct page *saveable_page(struct zone *zone, unsigned long pfn)
>>>    		return NULL;
>>>    	if (PageReserved(page)
>>> -	    && (!kernel_page_present(page) || pfn_is_nosave(pfn)))
>>> +	    && (!kernel_page_present(page) || pfn_is_nosave(pfn) || !pfn_is_map_memory(pfn)))
> I think adding the check for !pfn_is_map_memory() to arm64::pfn_is_nosave()
> is the best way to fix this.
Thinks, I also think this is the best modification.
>>>    		return NULL;
>>>    	if (page_is_guard(page))
>> On top of which tree does this apply?
>>
>> All occurrences of pfn_is_map_memory() are in arch/arm64, how does this
>> compile on other architectures?
>>
>> -- 
>> Cheers,
>>
>> David / dhildenb
>>


^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH] PM: hibernate: Fix level3 translation fault in swsusp_save()
  2024-02-27  7:51     ` Yaxiong Tian
@ 2024-02-29  9:42       ` Mike Rapoport
  2024-03-01  2:19           ` Yaxiong Tian
  2024-03-01  2:55         ` [PATCH] PM: " Yaxiong Tian
  0 siblings, 2 replies; 14+ messages in thread
From: Mike Rapoport @ 2024-02-29  9:42 UTC (permalink / raw)
  To: Yaxiong Tian
  Cc: David Hildenbrand, rafael, pavel, len.brown, keescook, tony.luck,
	gpiccoli, akpm, ardb, wangkefeng.wang, catalin.marinas, will,
	linux-pm, linux-kernel, linux-hardening, Yaxiong Tian, xiongxin

On Tue, Feb 27, 2024 at 03:51:25PM +0800, Yaxiong Tian wrote:
> 
> 在 2024/2/26 17:14, Mike Rapoport 写道:
> > On Mon, Feb 26, 2024 at 09:37:06AM +0100, David Hildenbrand wrote:
> > > On 26.02.24 04:42, Yaxiong Tian wrote:
> > > > From: Yaxiong Tian <tianyaxiong@kylinos.cn>
> > > > 
> > > > On ARM64 machines using UEFI, if the linear map is not set (can_set_direct_map()
> > > > return false), swsusp_save() will fail due to can't finding the map table
> > > > under the nomap memory.such as:
> > can_set_direct_map() has nothing to do with presence or absence of the
> > linear map.
> > 
> > Do you mean that kernel_page_present() presumes that a page is present when
> > can_set_direct_map() returns false even for NOMAP ranges?
> Yes， in swsusp_save()->copy_data_pages()->page_is_saveable(),
> kernel_page_present() presumes that a page is present when
> can_set_direct_map()
> returns false even for NOMAP ranges.So NOMAP pages will saved in after,and
> then
> cause level3 translation fault in this pages.

So this should be the description of the problem in the changelog rather
than saying "if the linear map is not set (can_set_direct_map() return
false)"

> > > > QEMU ARM64 using UEFI also has the problem by setting can_set_direct_map()
> > > > return false.
> >
> > Huh?
> > Why would you do that?
> >
> I discovered this problem when upgrading from 5.4 to 6.6 using the 5.4
> configuration.
> So I using latest linux-next code,find the problem still exist.To rule out
> the effects
> of a particular machine，I also use qemu to check it.

I believe this can be reproduced if you boot with rodata=off and then
a better description would be something like

	This issue can be reproduced in QEMU when booting with rodata=off

-- 
Sincerely yours,
Mike.

^ permalink raw reply	[flat|nested] 14+ messages in thread
* [PATCH v2] arm64: hibernate: Fix level3 translation fault in swsusp_save()
  2024-02-29  9:42       ` Mike Rapoport
@ 2024-03-01  2:19           ` Yaxiong Tian
  2024-03-01  2:55         ` [PATCH] PM: " Yaxiong Tian
  1 sibling, 0 replies; 14+ messages in thread
From: Yaxiong Tian @ 2024-03-01  2:19 UTC (permalink / raw)
  To: catalin.marinas, will, keescook, tianyaxiong, xiongxin, rppt
  Cc: tony.luck, gpiccoli, songshuaishuai, wangkefeng.wang, akpm, ardb,
	david, linux-arm-kernel, linux-kernel, linux-hardening

From: Yaxiong Tian <tianyaxiong@kylinos.cn>

On ARM64 machines using UEFI, if can_set_direct_map() return false by
setting some CONFIGS in kernel build or grub,such as
NO CONFIG_DEBUG_PAGEALLOC_ENABLE_DEFAULT、NO CONFIG_KFENCE
NO CONFIG_RODATA_FULL_DEFAULT_ENABLED.Also with setting rodata=off、
debug_pagealloc=off in grub and NO CONFIG_KFENCE.
swsusp_save() will fail due to can't finding the map table under the 
nomap memory.such as:

[   48.532162] Unable to handle kernel paging request at virtual address ffffff8000000000
[   48.532162] Mem abort info:
[   48.532162]   ESR = 0x0000000096000007
[   48.532162]   EC = 0x25: DABT (current EL), IL = 32 bits
[   48.532162]   SET = 0, FnV = 0
[   48.532162]   EA = 0, S1PTW = 0
[   48.532162]   FSC = 0x07: level 3 translation fault
[   48.532162] Data abort info:
[   48.532162]   ISV = 0, ISS = 0x00000007, ISS2 = 0x00000000
[   48.532162]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[   48.532162]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[   48.532162] swapper pgtable: 4k pages, 39-bit VAs, pgdp=00000000eeb0b000
[   48.532162] [ffffff8000000000] pgd=180000217fff9803, p4d=180000217fff9803, pud=180000217fff9803, pmd=180000217fff8803, pte=0000000000000000
[   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
[   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
[   48.532162] Modules linked in: xt_multiport ipt_REJECT nf_reject_ipv4 xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter bpfilter rfkill at803x snd_hda_codec_hdmi snd_hda_intel snd_intel_dspcfg dwmac_generic stmmac_platform snd_hda_codec stmmac joydev pcs_xpcs snd_hda_core phylink ppdev lp parport ramoops reed_solomon ip_tables x_tables nls_iso8859_1 vfat multipath linear amdgpu amdxcp drm_exec gpu_sched drm_buddy hid_generic usbhid hid radeon video drm_suballoc_helper drm_ttm_helper ttm i2c_algo_bit drm_display_helper cec drm_kms_helper drm
[   48.532162] CPU: 0 PID: 3663 Comm: systemd-sleep Not tainted 6.6.2+ #76
[   48.532162] Source Version: 4e22ed63a0a48e7a7cff9b98b7806d8d4add7dc0
[   48.532162] Hardware name: Greatwall GW-XXXXXX-XXX/GW-XXXXXX-XXX, BIOS KunLun BIOS V4.0 01/19/2021
[   48.532162] pstate: 600003c5 (nZCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   48.532162] pc : swsusp_save+0x280/0x538
[   48.532162] lr : swsusp_save+0x280/0x538
[   48.532162] sp : ffffffa034a3fa40
[   48.532162] x29: ffffffa034a3fa40 x28: ffffff8000001000 x27: 0000000000000000
[   48.532162] x26: ffffff8001400000 x25: ffffffc08113e248 x24: 0000000000000000
[   48.532162] x23: 0000000000080000 x22: ffffffc08113e280 x21: 00000000000c69f2
[   48.532162] x20: ffffff8000000000 x19: ffffffc081ae2500 x18: 0000000000000000
[   48.532162] x17: 6666662074736420 x16: 3030303030303030 x15: 3038666666666666
[   48.532162] x14: 0000000000000b69 x13: ffffff9f89088530 x12: 00000000ffffffea
[   48.532162] x11: 00000000ffff7fff x10: 00000000ffff7fff x9 : ffffffc08193f0d0
[   48.532162] x8 : 00000000000bffe8 x7 : c0000000ffff7fff x6 : 0000000000000001
[   48.532162] x5 : ffffffa0fff09dc8 x4 : 0000000000000000 x3 : 0000000000000027
[   48.532162] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 000000000000004e
[   48.532162] Call trace:
[   48.532162]  swsusp_save+0x280/0x538
[   48.532162]  swsusp_arch_suspend+0x148/0x190
[   48.532162]  hibernation_snapshot+0x240/0x39c
[   48.532162]  hibernate+0xc4/0x378
[   48.532162]  state_store+0xf0/0x10c
[   48.532162]  kobj_attr_store+0x14/0x24

This issue can be reproduced in QEMU using UEFI when booting with 
rodata=off、debug_pagealloc=off in grub and NO CONFIG_KFENCE.

This is because in swsusp_save()->copy_data_pages()->page_is_saveable(),
kernel_page_present() presumes that a page is present when can_set_direct_map()
returns false even for NOMAP ranges.So NOMAP pages will saved in after,and then
cause level3 translation fault in this pages.

Since the NOMAP regions are now marked as PageReserved(), pfn walkers
and the rest of core mm will treat them as unusable memory. So this
regions should not saved in hibernation.

This problem may cause by changes to pfn_valid() logic in commit
a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()").

So to fix it, we add !pfn_is_map_memory() check in arm64::pfn_is_nosave(). It
make such regisons don't save in hibernation.

Fixes: a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()")

Suggested-by: Mike Rapoport <rppt@kernel.org>
Co-developed-by: xiongxin <xiongxin@kylinos.cn>
Signed-off-by: xiongxin <xiongxin@kylinos.cn>
Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>

---
v1->v2: move !pfn_is_map_memory() check from page_is_saveable()
to arm64::pfn_is_nosave(),add opt commit messages.
---
 arch/arm64/kernel/hibernate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
index 02870beb271e..d90005de1d26 100644
--- a/arch/arm64/kernel/hibernate.c
+++ b/arch/arm64/kernel/hibernate.c
@@ -94,7 +94,7 @@ int pfn_is_nosave(unsigned long pfn)
 	unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
 
 	return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn)) ||
-		crash_is_nosave(pfn);
+		crash_is_nosave(pfn) || !pfn_is_map_memory(pfn);
 }
 
 void notrace save_processor_state(void)
-- 
2.34.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH v2] arm64: hibernate: Fix level3 translation fault in swsusp_save()
@ 2024-03-01  2:19           ` Yaxiong Tian
  0 siblings, 0 replies; 14+ messages in thread
From: Yaxiong Tian @ 2024-03-01  2:19 UTC (permalink / raw)
  To: catalin.marinas, will, keescook, tianyaxiong, xiongxin, rppt
  Cc: tony.luck, gpiccoli, songshuaishuai, wangkefeng.wang, akpm, ardb,
	david, linux-arm-kernel, linux-kernel, linux-hardening

From: Yaxiong Tian <tianyaxiong@kylinos.cn>

On ARM64 machines using UEFI, if can_set_direct_map() return false by
setting some CONFIGS in kernel build or grub,such as
NO CONFIG_DEBUG_PAGEALLOC_ENABLE_DEFAULT、NO CONFIG_KFENCE
NO CONFIG_RODATA_FULL_DEFAULT_ENABLED.Also with setting rodata=off、
debug_pagealloc=off in grub and NO CONFIG_KFENCE.
swsusp_save() will fail due to can't finding the map table under the 
nomap memory.such as:

[   48.532162] Unable to handle kernel paging request at virtual address ffffff8000000000
[   48.532162] Mem abort info:
[   48.532162]   ESR = 0x0000000096000007
[   48.532162]   EC = 0x25: DABT (current EL), IL = 32 bits
[   48.532162]   SET = 0, FnV = 0
[   48.532162]   EA = 0, S1PTW = 0
[   48.532162]   FSC = 0x07: level 3 translation fault
[   48.532162] Data abort info:
[   48.532162]   ISV = 0, ISS = 0x00000007, ISS2 = 0x00000000
[   48.532162]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[   48.532162]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[   48.532162] swapper pgtable: 4k pages, 39-bit VAs, pgdp=00000000eeb0b000
[   48.532162] [ffffff8000000000] pgd=180000217fff9803, p4d=180000217fff9803, pud=180000217fff9803, pmd=180000217fff8803, pte=0000000000000000
[   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
[   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
[   48.532162] Modules linked in: xt_multiport ipt_REJECT nf_reject_ipv4 xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter bpfilter rfkill at803x snd_hda_codec_hdmi snd_hda_intel snd_intel_dspcfg dwmac_generic stmmac_platform snd_hda_codec stmmac joydev pcs_xpcs snd_hda_core phylink ppdev lp parport ramoops reed_solomon ip_tables x_tables nls_iso8859_1 vfat multipath linear amdgpu amdxcp drm_exec gpu_sched drm_buddy hid_generic usbhid hid radeon video drm_suballoc_helper drm_ttm_helper ttm i2c_algo_bit drm_display_helper cec drm_kms_helper drm
[   48.532162] CPU: 0 PID: 3663 Comm: systemd-sleep Not tainted 6.6.2+ #76
[   48.532162] Source Version: 4e22ed63a0a48e7a7cff9b98b7806d8d4add7dc0
[   48.532162] Hardware name: Greatwall GW-XXXXXX-XXX/GW-XXXXXX-XXX, BIOS KunLun BIOS V4.0 01/19/2021
[   48.532162] pstate: 600003c5 (nZCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   48.532162] pc : swsusp_save+0x280/0x538
[   48.532162] lr : swsusp_save+0x280/0x538
[   48.532162] sp : ffffffa034a3fa40
[   48.532162] x29: ffffffa034a3fa40 x28: ffffff8000001000 x27: 0000000000000000
[   48.532162] x26: ffffff8001400000 x25: ffffffc08113e248 x24: 0000000000000000
[   48.532162] x23: 0000000000080000 x22: ffffffc08113e280 x21: 00000000000c69f2
[   48.532162] x20: ffffff8000000000 x19: ffffffc081ae2500 x18: 0000000000000000
[   48.532162] x17: 6666662074736420 x16: 3030303030303030 x15: 3038666666666666
[   48.532162] x14: 0000000000000b69 x13: ffffff9f89088530 x12: 00000000ffffffea
[   48.532162] x11: 00000000ffff7fff x10: 00000000ffff7fff x9 : ffffffc08193f0d0
[   48.532162] x8 : 00000000000bffe8 x7 : c0000000ffff7fff x6 : 0000000000000001
[   48.532162] x5 : ffffffa0fff09dc8 x4 : 0000000000000000 x3 : 0000000000000027
[   48.532162] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 000000000000004e
[   48.532162] Call trace:
[   48.532162]  swsusp_save+0x280/0x538
[   48.532162]  swsusp_arch_suspend+0x148/0x190
[   48.532162]  hibernation_snapshot+0x240/0x39c
[   48.532162]  hibernate+0xc4/0x378
[   48.532162]  state_store+0xf0/0x10c
[   48.532162]  kobj_attr_store+0x14/0x24

This issue can be reproduced in QEMU using UEFI when booting with 
rodata=off、debug_pagealloc=off in grub and NO CONFIG_KFENCE.

This is because in swsusp_save()->copy_data_pages()->page_is_saveable(),
kernel_page_present() presumes that a page is present when can_set_direct_map()
returns false even for NOMAP ranges.So NOMAP pages will saved in after,and then
cause level3 translation fault in this pages.

Since the NOMAP regions are now marked as PageReserved(), pfn walkers
and the rest of core mm will treat them as unusable memory. So this
regions should not saved in hibernation.

This problem may cause by changes to pfn_valid() logic in commit
a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()").

So to fix it, we add !pfn_is_map_memory() check in arm64::pfn_is_nosave(). It
make such regisons don't save in hibernation.

Fixes: a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()")

Suggested-by: Mike Rapoport <rppt@kernel.org>
Co-developed-by: xiongxin <xiongxin@kylinos.cn>
Signed-off-by: xiongxin <xiongxin@kylinos.cn>
Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>

---
v1->v2: move !pfn_is_map_memory() check from page_is_saveable()
to arm64::pfn_is_nosave(),add opt commit messages.
---
 arch/arm64/kernel/hibernate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
index 02870beb271e..d90005de1d26 100644
--- a/arch/arm64/kernel/hibernate.c
+++ b/arch/arm64/kernel/hibernate.c
@@ -94,7 +94,7 @@ int pfn_is_nosave(unsigned long pfn)
 	unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
 
 	return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn)) ||
-		crash_is_nosave(pfn);
+		crash_is_nosave(pfn) || !pfn_is_map_memory(pfn);
 }
 
 void notrace save_processor_state(void)
-- 
2.34.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 14+ messages in thread
* Re: [PATCH] PM: hibernate: Fix level3 translation fault in swsusp_save()
  2024-02-29  9:42       ` Mike Rapoport
  2024-03-01  2:19           ` Yaxiong Tian
@ 2024-03-01  2:55         ` Yaxiong Tian
  1 sibling, 0 replies; 14+ messages in thread
From: Yaxiong Tian @ 2024-03-01  2:55 UTC (permalink / raw)
  To: Mike Rapoport
  Cc: David Hildenbrand, rafael, pavel, len.brown, keescook, tony.luck,
	gpiccoli, akpm, ardb, wangkefeng.wang, catalin.marinas, will,
	linux-pm, linux-kernel, linux-hardening, Yaxiong Tian, xiongxin


在 2024/2/29 17:42, Mike Rapoport 写道:
> On Tue, Feb 27, 2024 at 03:51:25PM +0800, Yaxiong Tian wrote:
>> 在 2024/2/26 17:14, Mike Rapoport 写道:
>>> On Mon, Feb 26, 2024 at 09:37:06AM +0100, David Hildenbrand wrote:
>>>> On 26.02.24 04:42, Yaxiong Tian wrote:
>>>>> From: Yaxiong Tian <tianyaxiong@kylinos.cn>
>>>>>
>>>>> On ARM64 machines using UEFI, if the linear map is not set (can_set_direct_map()
>>>>> return false), swsusp_save() will fail due to can't finding the map table
>>>>> under the nomap memory.such as:
>>> can_set_direct_map() has nothing to do with presence or absence of the
>>> linear map.
>>>
>>> Do you mean that kernel_page_present() presumes that a page is present when
>>> can_set_direct_map() returns false even for NOMAP ranges?
>> Yes， in swsusp_save()->copy_data_pages()->page_is_saveable(),
>> kernel_page_present() presumes that a page is present when
>> can_set_direct_map()
>> returns false even for NOMAP ranges.So NOMAP pages will saved in after,and
>> then
>> cause level3 translation fault in this pages.
> So this should be the description of the problem in the changelog rather
> than saying "if the linear map is not set (can_set_direct_map() return
> false)"
>
>>>>> QEMU ARM64 using UEFI also has the problem by setting can_set_direct_map()
>>>>> return false.
>>> Huh?
>>> Why would you do that?
>>>
>> I discovered this problem when upgrading from 5.4 to 6.6 using the 5.4
>> configuration.
>> So I using latest linux-next code,find the problem still exist.To rule out
>> the effects
>> of a particular machine，I also use qemu to check it.
> I believe this can be reproduced if you boot with rodata=off and then
> a better description would be something like
>
> 	This issue can be reproduced in QEMU when booting with rodata=off

Thanks, I opt commit messages,and move !pfn_is_map_memory() check from 
page_is_saveable()
to arm64::pfn_is_nosave().  The patch v2 is in:

https://lore.kernel.org/all/20240301021924.33210-1-13327272236@163.com/

I heard that if it is a single PATCH it should be placed under the 
original mailing list,
so please ignore the February 27 patch mailing.

>


^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH v2] arm64: hibernate: Fix level3 translation fault in swsusp_save()
  2024-03-01  2:19           ` Yaxiong Tian
@ 2024-04-12 17:30             ` Catalin Marinas
  -1 siblings, 0 replies; 14+ messages in thread
From: Catalin Marinas @ 2024-04-12 17:30 UTC (permalink / raw)
  To: Yaxiong Tian
  Cc: will, keescook, tianyaxiong, xiongxin, rppt, tony.luck, gpiccoli,
	songshuaishuai, wangkefeng.wang, akpm, ardb, david,
	linux-arm-kernel, linux-kernel, linux-hardening

For some reason I missed the updated patch.

On Fri, Mar 01, 2024 at 10:19:24AM +0800, Yaxiong Tian wrote:
> From: Yaxiong Tian <tianyaxiong@kylinos.cn>
> 
> On ARM64 machines using UEFI, if can_set_direct_map() return false by
> setting some CONFIGS in kernel build or grub,such as
> NO CONFIG_DEBUG_PAGEALLOC_ENABLE_DEFAULT、NO CONFIG_KFENCE
> NO CONFIG_RODATA_FULL_DEFAULT_ENABLED.Also with setting rodata=off、
> debug_pagealloc=off in grub and NO CONFIG_KFENCE.
> swsusp_save() will fail due to can't finding the map table under the 
> nomap memory.such as:
[...]
> [   48.532162] Call trace:
> [   48.532162]  swsusp_save+0x280/0x538
> [   48.532162]  swsusp_arch_suspend+0x148/0x190
> [   48.532162]  hibernation_snapshot+0x240/0x39c
> [   48.532162]  hibernate+0xc4/0x378
> [   48.532162]  state_store+0xf0/0x10c
> [   48.532162]  kobj_attr_store+0x14/0x24
> 
> This issue can be reproduced in QEMU using UEFI when booting with 
> rodata=off、debug_pagealloc=off in grub and NO CONFIG_KFENCE.
> 
> This is because in swsusp_save()->copy_data_pages()->page_is_saveable(),
> kernel_page_present() presumes that a page is present when can_set_direct_map()
> returns false even for NOMAP ranges.So NOMAP pages will saved in after,and then
> cause level3 translation fault in this pages.

I can see how kernel_page_present() ended up returning true if
!can_set_direct_map(), though based on the function naming only, it
feels a bit unintuitive. Is arm64 the only architecture making use of
MEMBLOCK_NOMAP? Or is it the only one where kernel_page_present() also
returns true if !can_set_direct_map()?

> diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
> index 02870beb271e..d90005de1d26 100644
> --- a/arch/arm64/kernel/hibernate.c
> +++ b/arch/arm64/kernel/hibernate.c
> @@ -94,7 +94,7 @@ int pfn_is_nosave(unsigned long pfn)
>  	unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
>  
>  	return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn)) ||
> -		crash_is_nosave(pfn);
> +		crash_is_nosave(pfn) || !pfn_is_map_memory(pfn);
>  }

This indeed fixes the problem but it looks like an arm64-specific
workaround. I can see at least arm, loongarch and riscv making use of
memblock_is_map_memory() (which is what pfn_is_map_memory() calls). Do
they not have the same problem? On riscv, for example,
kernel_page_present() does not depend on any ARCH_HAS_SET_DIRECT_MAP
related options/conditions (neither does x86 though not sure it cares
about MEMBLOCK_NOMAP). Should we do the same for arm64 and drop the
!can_set_direct_map() condition in kernel_page_present()?

-- 
Catalin

^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH v2] arm64: hibernate: Fix level3 translation fault in swsusp_save()
@ 2024-04-12 17:30             ` Catalin Marinas
  0 siblings, 0 replies; 14+ messages in thread
From: Catalin Marinas @ 2024-04-12 17:30 UTC (permalink / raw)
  To: Yaxiong Tian
  Cc: will, keescook, tianyaxiong, xiongxin, rppt, tony.luck, gpiccoli,
	songshuaishuai, wangkefeng.wang, akpm, ardb, david,
	linux-arm-kernel, linux-kernel, linux-hardening

For some reason I missed the updated patch.

On Fri, Mar 01, 2024 at 10:19:24AM +0800, Yaxiong Tian wrote:
> From: Yaxiong Tian <tianyaxiong@kylinos.cn>
> 
> On ARM64 machines using UEFI, if can_set_direct_map() return false by
> setting some CONFIGS in kernel build or grub,such as
> NO CONFIG_DEBUG_PAGEALLOC_ENABLE_DEFAULT、NO CONFIG_KFENCE
> NO CONFIG_RODATA_FULL_DEFAULT_ENABLED.Also with setting rodata=off、
> debug_pagealloc=off in grub and NO CONFIG_KFENCE.
> swsusp_save() will fail due to can't finding the map table under the 
> nomap memory.such as:
[...]
> [   48.532162] Call trace:
> [   48.532162]  swsusp_save+0x280/0x538
> [   48.532162]  swsusp_arch_suspend+0x148/0x190
> [   48.532162]  hibernation_snapshot+0x240/0x39c
> [   48.532162]  hibernate+0xc4/0x378
> [   48.532162]  state_store+0xf0/0x10c
> [   48.532162]  kobj_attr_store+0x14/0x24
> 
> This issue can be reproduced in QEMU using UEFI when booting with 
> rodata=off、debug_pagealloc=off in grub and NO CONFIG_KFENCE.
> 
> This is because in swsusp_save()->copy_data_pages()->page_is_saveable(),
> kernel_page_present() presumes that a page is present when can_set_direct_map()
> returns false even for NOMAP ranges.So NOMAP pages will saved in after,and then
> cause level3 translation fault in this pages.

I can see how kernel_page_present() ended up returning true if
!can_set_direct_map(), though based on the function naming only, it
feels a bit unintuitive. Is arm64 the only architecture making use of
MEMBLOCK_NOMAP? Or is it the only one where kernel_page_present() also
returns true if !can_set_direct_map()?

> diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
> index 02870beb271e..d90005de1d26 100644
> --- a/arch/arm64/kernel/hibernate.c
> +++ b/arch/arm64/kernel/hibernate.c
> @@ -94,7 +94,7 @@ int pfn_is_nosave(unsigned long pfn)
>  	unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
>  
>  	return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn)) ||
> -		crash_is_nosave(pfn);
> +		crash_is_nosave(pfn) || !pfn_is_map_memory(pfn);
>  }

This indeed fixes the problem but it looks like an arm64-specific
workaround. I can see at least arm, loongarch and riscv making use of
memblock_is_map_memory() (which is what pfn_is_map_memory() calls). Do
they not have the same problem? On riscv, for example,
kernel_page_present() does not depend on any ARCH_HAS_SET_DIRECT_MAP
related options/conditions (neither does x86 though not sure it cares
about MEMBLOCK_NOMAP). Should we do the same for arm64 and drop the
!can_set_direct_map() condition in kernel_page_present()?

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH v2] arm64: hibernate: Fix level3 translation fault in swsusp_save()
  2024-04-12 17:30             ` Catalin Marinas
@ 2024-04-17  2:13               ` Yaxiong Tian
  -1 siblings, 0 replies; 14+ messages in thread
From: Yaxiong Tian @ 2024-04-17  2:13 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: will, keescook, tianyaxiong, xiongxin, rppt, tony.luck, gpiccoli,
	songshuaishuai, wangkefeng.wang, akpm, ardb, david,
	linux-arm-kernel, linux-kernel, linux-hardening


在 2024/4/13 01:30, Catalin Marinas 写道:
> For some reason I missed the updated patch.
>
> On Fri, Mar 01, 2024 at 10:19:24AM +0800, Yaxiong Tian wrote:
>> From: Yaxiong Tian <tianyaxiong@kylinos.cn>
>>
>> On ARM64 machines using UEFI, if can_set_direct_map() return false by
>> setting some CONFIGS in kernel build or grub,such as
>> NO CONFIG_DEBUG_PAGEALLOC_ENABLE_DEFAULT、NO CONFIG_KFENCE
>> NO CONFIG_RODATA_FULL_DEFAULT_ENABLED.Also with setting rodata=off、
>> debug_pagealloc=off in grub and NO CONFIG_KFENCE.
>> swsusp_save() will fail due to can't finding the map table under the
>> nomap memory.such as:
> [...]
>> [   48.532162] Call trace:
>> [   48.532162]  swsusp_save+0x280/0x538
>> [   48.532162]  swsusp_arch_suspend+0x148/0x190
>> [   48.532162]  hibernation_snapshot+0x240/0x39c
>> [   48.532162]  hibernate+0xc4/0x378
>> [   48.532162]  state_store+0xf0/0x10c
>> [   48.532162]  kobj_attr_store+0x14/0x24
>>
>> This issue can be reproduced in QEMU using UEFI when booting with
>> rodata=off、debug_pagealloc=off in grub and NO CONFIG_KFENCE.
>>
>> This is because in swsusp_save()->copy_data_pages()->page_is_saveable(),
>> kernel_page_present() presumes that a page is present when can_set_direct_map()
>> returns false even for NOMAP ranges.So NOMAP pages will saved in after,and then
>> cause level3 translation fault in this pages.
> I can see how kernel_page_present() ended up returning true if
> !can_set_direct_map(), though based on the function naming only, it
> feels a bit unintuitive. Is arm64 the only architecture making use of
> MEMBLOCK_NOMAP? Or is it the only one where kernel_page_present() also
> returns true if !can_set_direct_map()?
It looks like ARM64 is only one where kernel_page_present() also returns
true if  !can_set_direct_map(). I remember that on x86 models  there are
no NOMAP regions, on ARM64 machines these NOMAP regions are set with uefi.
Other details are too old to remember.
>> diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
>> index 02870beb271e..d90005de1d26 100644
>> --- a/arch/arm64/kernel/hibernate.c
>> +++ b/arch/arm64/kernel/hibernate.c
>> @@ -94,7 +94,7 @@ int pfn_is_nosave(unsigned long pfn)
>>   	unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
>>   
>>   	return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn)) ||
>> -		crash_is_nosave(pfn);
>> +		crash_is_nosave(pfn) || !pfn_is_map_memory(pfn);
>>   }
> This indeed fixes the problem but it looks like an arm64-specific
> workaround. I can see at least arm, loongarch and riscv making use of
> memblock_is_map_memory() (which is what pfn_is_map_memory() calls). Do
> they not have the same problem? On riscv, for example,
> kernel_page_present() does not depend on any ARCH_HAS_SET_DIRECT_MAP
> related options/conditions (neither does x86 though not sure it cares
> about MEMBLOCK_NOMAP). Should we do the same for arm64 and drop the
> !can_set_direct_map() condition in kernel_page_present()?
I drop the  !can_set_direct_map() condition in kernel_page_present(). And
test it. The test was passed. I use kretprobe to inspect in 
kernel_page_present()
find that the NOMAP pages  will reture false. So in saveable_page() 
these page
will skip. The final logic for processing these pages is the same as 
before (v5.4).
I think it is good way.


^ permalink raw reply	[flat|nested] 14+ messages in thread
* Re: [PATCH v2] arm64: hibernate: Fix level3 translation fault in swsusp_save()
@ 2024-04-17  2:13               ` Yaxiong Tian
  0 siblings, 0 replies; 14+ messages in thread
From: Yaxiong Tian @ 2024-04-17  2:13 UTC (permalink / raw)
  To: Catalin Marinas
  Cc: will, keescook, tianyaxiong, xiongxin, rppt, tony.luck, gpiccoli,
	songshuaishuai, wangkefeng.wang, akpm, ardb, david,
	linux-arm-kernel, linux-kernel, linux-hardening


在 2024/4/13 01:30, Catalin Marinas 写道:
> For some reason I missed the updated patch.
>
> On Fri, Mar 01, 2024 at 10:19:24AM +0800, Yaxiong Tian wrote:
>> From: Yaxiong Tian <tianyaxiong@kylinos.cn>
>>
>> On ARM64 machines using UEFI, if can_set_direct_map() return false by
>> setting some CONFIGS in kernel build or grub,such as
>> NO CONFIG_DEBUG_PAGEALLOC_ENABLE_DEFAULT、NO CONFIG_KFENCE
>> NO CONFIG_RODATA_FULL_DEFAULT_ENABLED.Also with setting rodata=off、
>> debug_pagealloc=off in grub and NO CONFIG_KFENCE.
>> swsusp_save() will fail due to can't finding the map table under the
>> nomap memory.such as:
> [...]
>> [   48.532162] Call trace:
>> [   48.532162]  swsusp_save+0x280/0x538
>> [   48.532162]  swsusp_arch_suspend+0x148/0x190
>> [   48.532162]  hibernation_snapshot+0x240/0x39c
>> [   48.532162]  hibernate+0xc4/0x378
>> [   48.532162]  state_store+0xf0/0x10c
>> [   48.532162]  kobj_attr_store+0x14/0x24
>>
>> This issue can be reproduced in QEMU using UEFI when booting with
>> rodata=off、debug_pagealloc=off in grub and NO CONFIG_KFENCE.
>>
>> This is because in swsusp_save()->copy_data_pages()->page_is_saveable(),
>> kernel_page_present() presumes that a page is present when can_set_direct_map()
>> returns false even for NOMAP ranges.So NOMAP pages will saved in after,and then
>> cause level3 translation fault in this pages.
> I can see how kernel_page_present() ended up returning true if
> !can_set_direct_map(), though based on the function naming only, it
> feels a bit unintuitive. Is arm64 the only architecture making use of
> MEMBLOCK_NOMAP? Or is it the only one where kernel_page_present() also
> returns true if !can_set_direct_map()?
It looks like ARM64 is only one where kernel_page_present() also returns
true if  !can_set_direct_map(). I remember that on x86 models  there are
no NOMAP regions, on ARM64 machines these NOMAP regions are set with uefi.
Other details are too old to remember.
>> diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
>> index 02870beb271e..d90005de1d26 100644
>> --- a/arch/arm64/kernel/hibernate.c
>> +++ b/arch/arm64/kernel/hibernate.c
>> @@ -94,7 +94,7 @@ int pfn_is_nosave(unsigned long pfn)
>>   	unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
>>   
>>   	return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn)) ||
>> -		crash_is_nosave(pfn);
>> +		crash_is_nosave(pfn) || !pfn_is_map_memory(pfn);
>>   }
> This indeed fixes the problem but it looks like an arm64-specific
> workaround. I can see at least arm, loongarch and riscv making use of
> memblock_is_map_memory() (which is what pfn_is_map_memory() calls). Do
> they not have the same problem? On riscv, for example,
> kernel_page_present() does not depend on any ARCH_HAS_SET_DIRECT_MAP
> related options/conditions (neither does x86 though not sure it cares
> about MEMBLOCK_NOMAP). Should we do the same for arm64 and drop the
> !can_set_direct_map() condition in kernel_page_present()?
I drop the  !can_set_direct_map() condition in kernel_page_present(). And
test it. The test was passed. I use kretprobe to inspect in 
kernel_page_present()
find that the NOMAP pages  will reture false. So in saveable_page() 
these page
will skip. The final logic for processing these pages is the same as 
before (v5.4).
I think it is good way.


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 14+ messages in thread
* [PATCH v2] arm64: hibernate: Fix level3 translation fault in swsusp_save()
@ 2024-02-27  7:54 ` Yaxiong Tian
  0 siblings, 0 replies; 14+ messages in thread
From: Yaxiong Tian @ 2024-02-27  7:54 UTC (permalink / raw)
  To: catalin.marinas, will, keescook, tony.luck, gpiccoli,
	tianyaxiong, songshuaishuai, xiongxin, rppt, wangkefeng.wang,
	akpm, ardb, david
  Cc: linux-arm-kernel, linux-kernel, linux-hardening

From: Yaxiong Tian <tianyaxiong@kylinos.cn>

On ARM64 machines using UEFI, if can_set_direct_map() return false by 
setting some CONFIGS in kernel build,such as 
NO CONFIG_DEBUG_PAGEALLOC_ENABLE_DEFAULT、NO CONFIG_RODATA_FULL_DEFAULT_ENABLED.
swsusp_save() will fail due to can't finding the map table under the nomap memory.
such as:

[   48.532162] Unable to handle kernel paging request at virtual address ffffff8000000000
[   48.532162] Mem abort info:
[   48.532162]   ESR = 0x0000000096000007
[   48.532162]   EC = 0x25: DABT (current EL), IL = 32 bits
[   48.532162]   SET = 0, FnV = 0
[   48.532162]   EA = 0, S1PTW = 0
[   48.532162]   FSC = 0x07: level 3 translation fault
[   48.532162] Data abort info:
[   48.532162]   ISV = 0, ISS = 0x00000007, ISS2 = 0x00000000
[   48.532162]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[   48.532162]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[   48.532162] swapper pgtable: 4k pages, 39-bit VAs, pgdp=00000000eeb0b000
[   48.532162] [ffffff8000000000] pgd=180000217fff9803, p4d=180000217fff9803, pud=180000217fff9803, pmd=180000217fff8803, pte=0000000000000000
[   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
[   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
[   48.532162] Modules linked in: xt_multiport ipt_REJECT nf_reject_ipv4 xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter bpfilter rfkill at803x snd_hda_codec_hdmi snd_hda_intel snd_intel_dspcfg dwmac_generic stmmac_platform snd_hda_codec stmmac joydev pcs_xpcs snd_hda_core phylink ppdev lp parport ramoops reed_solomon ip_tables x_tables nls_iso8859_1 vfat multipath linear amdgpu amdxcp drm_exec gpu_sched drm_buddy hid_generic usbhid hid radeon video drm_suballoc_helper drm_ttm_helper ttm i2c_algo_bit drm_display_helper cec drm_kms_helper drm
[   48.532162] CPU: 0 PID: 3663 Comm: systemd-sleep Not tainted 6.6.2+ #76
[   48.532162] Source Version: 4e22ed63a0a48e7a7cff9b98b7806d8d4add7dc0
[   48.532162] Hardware name: Greatwall GW-XXXXXX-XXX/GW-XXXXXX-XXX, BIOS KunLun BIOS V4.0 01/19/2021
[   48.532162] pstate: 600003c5 (nZCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   48.532162] pc : swsusp_save+0x280/0x538
[   48.532162] lr : swsusp_save+0x280/0x538
[   48.532162] sp : ffffffa034a3fa40
[   48.532162] x29: ffffffa034a3fa40 x28: ffffff8000001000 x27: 0000000000000000
[   48.532162] x26: ffffff8001400000 x25: ffffffc08113e248 x24: 0000000000000000
[   48.532162] x23: 0000000000080000 x22: ffffffc08113e280 x21: 00000000000c69f2
[   48.532162] x20: ffffff8000000000 x19: ffffffc081ae2500 x18: 0000000000000000
[   48.532162] x17: 6666662074736420 x16: 3030303030303030 x15: 3038666666666666
[   48.532162] x14: 0000000000000b69 x13: ffffff9f89088530 x12: 00000000ffffffea
[   48.532162] x11: 00000000ffff7fff x10: 00000000ffff7fff x9 : ffffffc08193f0d0
[   48.532162] x8 : 00000000000bffe8 x7 : c0000000ffff7fff x6 : 0000000000000001
[   48.532162] x5 : ffffffa0fff09dc8 x4 : 0000000000000000 x3 : 0000000000000027
[   48.532162] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 000000000000004e
[   48.532162] Call trace:
[   48.532162]  swsusp_save+0x280/0x538
[   48.532162]  swsusp_arch_suspend+0x148/0x190
[   48.532162]  hibernation_snapshot+0x240/0x39c
[   48.532162]  hibernate+0xc4/0x378
[   48.532162]  state_store+0xf0/0x10c
[   48.532162]  kobj_attr_store+0x14/0x24

QEMU ARM64 using UEFI also has the problem by setting can_set_direct_map()
return false.

This is because in swsusp_save()->copy_data_pages()->page_is_saveable(),
kernel_page_present() presumes that a page is present when can_set_direct_map()
returns false even for NOMAP ranges.So NOMAP pages will saved in after,and then
cause level3 translation fault in this pages.

Since the NOMAP regions are now marked as PageReserved(), pfn walkers
and the rest of core mm will treat them as unusable memory. So this
regions should not saved in hibernation.

This problem may cause by changes to pfn_valid() logic in commit
a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()").

So to fix it, we add !pfn_is_map_memory() check in arm64::pfn_is_nosave(). It
make such regisons don't save in hibernation.

Fixes: a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()")
Co-developed-by: xiongxin <xiongxin@kylinos.cn>
Signed-off-by: xiongxin <xiongxin@kylinos.cn>
Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>

---
v1->v2: move !pfn_is_map_memory() check from page_is_saveable()
to arm64::pfn_is_nosave()
---
 arch/arm64/kernel/hibernate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
index 02870beb271e..d90005de1d26 100644
--- a/arch/arm64/kernel/hibernate.c
+++ b/arch/arm64/kernel/hibernate.c
@@ -94,7 +94,7 @@ int pfn_is_nosave(unsigned long pfn)
 	unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
 
 	return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn)) ||
-		crash_is_nosave(pfn);
+		crash_is_nosave(pfn) || !pfn_is_map_memory(pfn);
 }
 
 void notrace save_processor_state(void)
-- 
2.34.1


^ permalink raw reply related	[flat|nested] 14+ messages in thread
* [PATCH v2] arm64: hibernate: Fix level3 translation fault in swsusp_save()
@ 2024-02-27  7:54 ` Yaxiong Tian
  0 siblings, 0 replies; 14+ messages in thread
From: Yaxiong Tian @ 2024-02-27  7:54 UTC (permalink / raw)
  To: catalin.marinas, will, keescook, tony.luck, gpiccoli,
	tianyaxiong, songshuaishuai, xiongxin, rppt, wangkefeng.wang,
	akpm, ardb, david
  Cc: linux-arm-kernel, linux-kernel, linux-hardening

From: Yaxiong Tian <tianyaxiong@kylinos.cn>

On ARM64 machines using UEFI, if can_set_direct_map() return false by 
setting some CONFIGS in kernel build,such as 
NO CONFIG_DEBUG_PAGEALLOC_ENABLE_DEFAULT、NO CONFIG_RODATA_FULL_DEFAULT_ENABLED.
swsusp_save() will fail due to can't finding the map table under the nomap memory.
such as:

[   48.532162] Unable to handle kernel paging request at virtual address ffffff8000000000
[   48.532162] Mem abort info:
[   48.532162]   ESR = 0x0000000096000007
[   48.532162]   EC = 0x25: DABT (current EL), IL = 32 bits
[   48.532162]   SET = 0, FnV = 0
[   48.532162]   EA = 0, S1PTW = 0
[   48.532162]   FSC = 0x07: level 3 translation fault
[   48.532162] Data abort info:
[   48.532162]   ISV = 0, ISS = 0x00000007, ISS2 = 0x00000000
[   48.532162]   CM = 0, WnR = 0, TnD = 0, TagAccess = 0
[   48.532162]   GCS = 0, Overlay = 0, DirtyBit = 0, Xs = 0
[   48.532162] swapper pgtable: 4k pages, 39-bit VAs, pgdp=00000000eeb0b000
[   48.532162] [ffffff8000000000] pgd=180000217fff9803, p4d=180000217fff9803, pud=180000217fff9803, pmd=180000217fff8803, pte=0000000000000000
[   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
[   48.532162] Internal error: Oops: 0000000096000007 [#1] SMP
[   48.532162] Modules linked in: xt_multiport ipt_REJECT nf_reject_ipv4 xt_conntrack nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c iptable_filter bpfilter rfkill at803x snd_hda_codec_hdmi snd_hda_intel snd_intel_dspcfg dwmac_generic stmmac_platform snd_hda_codec stmmac joydev pcs_xpcs snd_hda_core phylink ppdev lp parport ramoops reed_solomon ip_tables x_tables nls_iso8859_1 vfat multipath linear amdgpu amdxcp drm_exec gpu_sched drm_buddy hid_generic usbhid hid radeon video drm_suballoc_helper drm_ttm_helper ttm i2c_algo_bit drm_display_helper cec drm_kms_helper drm
[   48.532162] CPU: 0 PID: 3663 Comm: systemd-sleep Not tainted 6.6.2+ #76
[   48.532162] Source Version: 4e22ed63a0a48e7a7cff9b98b7806d8d4add7dc0
[   48.532162] Hardware name: Greatwall GW-XXXXXX-XXX/GW-XXXXXX-XXX, BIOS KunLun BIOS V4.0 01/19/2021
[   48.532162] pstate: 600003c5 (nZCv DAIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[   48.532162] pc : swsusp_save+0x280/0x538
[   48.532162] lr : swsusp_save+0x280/0x538
[   48.532162] sp : ffffffa034a3fa40
[   48.532162] x29: ffffffa034a3fa40 x28: ffffff8000001000 x27: 0000000000000000
[   48.532162] x26: ffffff8001400000 x25: ffffffc08113e248 x24: 0000000000000000
[   48.532162] x23: 0000000000080000 x22: ffffffc08113e280 x21: 00000000000c69f2
[   48.532162] x20: ffffff8000000000 x19: ffffffc081ae2500 x18: 0000000000000000
[   48.532162] x17: 6666662074736420 x16: 3030303030303030 x15: 3038666666666666
[   48.532162] x14: 0000000000000b69 x13: ffffff9f89088530 x12: 00000000ffffffea
[   48.532162] x11: 00000000ffff7fff x10: 00000000ffff7fff x9 : ffffffc08193f0d0
[   48.532162] x8 : 00000000000bffe8 x7 : c0000000ffff7fff x6 : 0000000000000001
[   48.532162] x5 : ffffffa0fff09dc8 x4 : 0000000000000000 x3 : 0000000000000027
[   48.532162] x2 : 0000000000000000 x1 : 0000000000000000 x0 : 000000000000004e
[   48.532162] Call trace:
[   48.532162]  swsusp_save+0x280/0x538
[   48.532162]  swsusp_arch_suspend+0x148/0x190
[   48.532162]  hibernation_snapshot+0x240/0x39c
[   48.532162]  hibernate+0xc4/0x378
[   48.532162]  state_store+0xf0/0x10c
[   48.532162]  kobj_attr_store+0x14/0x24

QEMU ARM64 using UEFI also has the problem by setting can_set_direct_map()
return false.

This is because in swsusp_save()->copy_data_pages()->page_is_saveable(),
kernel_page_present() presumes that a page is present when can_set_direct_map()
returns false even for NOMAP ranges.So NOMAP pages will saved in after,and then
cause level3 translation fault in this pages.

Since the NOMAP regions are now marked as PageReserved(), pfn walkers
and the rest of core mm will treat them as unusable memory. So this
regions should not saved in hibernation.

This problem may cause by changes to pfn_valid() logic in commit
a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()").

So to fix it, we add !pfn_is_map_memory() check in arm64::pfn_is_nosave(). It
make such regisons don't save in hibernation.

Fixes: a7d9f306ba70 ("arm64: drop pfn_valid_within() and simplify pfn_valid()")
Co-developed-by: xiongxin <xiongxin@kylinos.cn>
Signed-off-by: xiongxin <xiongxin@kylinos.cn>
Signed-off-by: Yaxiong Tian <tianyaxiong@kylinos.cn>

---
v1->v2: move !pfn_is_map_memory() check from page_is_saveable()
to arm64::pfn_is_nosave()
---
 arch/arm64/kernel/hibernate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/hibernate.c b/arch/arm64/kernel/hibernate.c
index 02870beb271e..d90005de1d26 100644
--- a/arch/arm64/kernel/hibernate.c
+++ b/arch/arm64/kernel/hibernate.c
@@ -94,7 +94,7 @@ int pfn_is_nosave(unsigned long pfn)
 	unsigned long nosave_end_pfn = sym_to_pfn(&__nosave_end - 1);
 
 	return ((pfn >= nosave_begin_pfn) && (pfn <= nosave_end_pfn)) ||
-		crash_is_nosave(pfn);
+		crash_is_nosave(pfn) || !pfn_is_map_memory(pfn);
 }
 
 void notrace save_processor_state(void)
-- 
2.34.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 14+ messages in thread
end of thread, other threads:[~2024-04-17  2:15 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-02-26  3:42 [PATCH] PM: hibernate: Fix level3 translation fault in swsusp_save() Yaxiong Tian
2024-02-26  8:37 ` David Hildenbrand
2024-02-26  9:14   ` Mike Rapoport
2024-02-27  7:51     ` Yaxiong Tian
2024-02-29  9:42       ` Mike Rapoport
2024-03-01  2:19         ` [PATCH v2] arm64: " Yaxiong Tian
2024-03-01  2:19           ` Yaxiong Tian
2024-04-12 17:30           ` Catalin Marinas
2024-04-12 17:30             ` Catalin Marinas
2024-04-17  2:13             ` Yaxiong Tian
2024-04-17  2:13               ` Yaxiong Tian
2024-03-01  2:55         ` [PATCH] PM: " Yaxiong Tian
2024-02-27  7:54 [PATCH v2] arm64: " Yaxiong Tian
2024-02-27  7:54 ` Yaxiong Tian

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.