[PATCH v1] mm: initialize struct page for reserved pages in ZONE_DEVICE

[PATCH v1] mm: initialize struct page for reserved pages in ZONE_DEVICE

[Naoya Horiguchi]

Reading /proc/kpageflags for pfns allocated by pmem namespace triggers
kernel panic with a message like "BUG: unable to handle kernel paging
request at fffffffffffffffe".

The first few pages (controlled by altmap passed to memmap_init_zone())
in the ZONE_DEVICE can skip struct page initialization, which causes
the reported issue.

This patch simply adds some initialization code for them.

Fixes: 4b94ffdc4163 ("x86, mm: introduce vmem_altmap to augment vmemmap_populate()")
Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
---
 mm/page_alloc.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git v4.17-mmotm-2018-06-07-16-59/mm/page_alloc.c v4.17-mmotm-2018-06-07-16-59_patched/mm/page_alloc.c
index 1772513..0b36afe 100644
--- v4.17-mmotm-2018-06-07-16-59/mm/page_alloc.c
+++ v4.17-mmotm-2018-06-07-16-59_patched/mm/page_alloc.c
@@ -5574,8 +5574,16 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
 	 * Honor reservation requested by the driver for this ZONE_DEVICE
 	 * memory
 	 */
-	if (altmap && start_pfn == altmap->base_pfn)
+	if (altmap && start_pfn == altmap->base_pfn) {
+		unsigned long i;
+
+		for (i = 0; i < altmap->reserve; i++) {
+			page = pfn_to_page(start_pfn + i);
+			__init_single_page(page, start_pfn + i, zone, nid);
+			SetPageReserved(page);
+		}
 		start_pfn += altmap->reserve;
+	}
 
 	for (pfn = start_pfn; pfn < end_pfn; pfn++) {
 		/*
-- 
2.7.0

Re: [PATCH v1] mm: initialize struct page for reserved pages in ZONE_DEVICE

[Dan Williams]

On Thu, Jun 21, 2018 at 11:08 PM, Naoya Horiguchi
<n-horiguchi@ah.jp.nec.com> wrote:
> Reading /proc/kpageflags for pfns allocated by pmem namespace triggers
> kernel panic with a message like "BUG: unable to handle kernel paging
> request at fffffffffffffffe".
>
> The first few pages (controlled by altmap passed to memmap_init_zone())
> in the ZONE_DEVICE can skip struct page initialization, which causes
> the reported issue.
>
> This patch simply adds some initialization code for them.
>
> Fixes: 4b94ffdc4163 ("x86, mm: introduce vmem_altmap to augment vmemmap_populate()")
> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> ---
>  mm/page_alloc.c | 10 +++++++++-
>  1 file changed, 9 insertions(+), 1 deletion(-)
>
> diff --git v4.17-mmotm-2018-06-07-16-59/mm/page_alloc.c v4.17-mmotm-2018-06-07-16-59_patched/mm/page_alloc.c
> index 1772513..0b36afe 100644
> --- v4.17-mmotm-2018-06-07-16-59/mm/page_alloc.c
> +++ v4.17-mmotm-2018-06-07-16-59_patched/mm/page_alloc.c
> @@ -5574,8 +5574,16 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
>          * Honor reservation requested by the driver for this ZONE_DEVICE
>          * memory
>          */
> -       if (altmap && start_pfn == altmap->base_pfn)
> +       if (altmap && start_pfn == altmap->base_pfn) {
> +               unsigned long i;
> +
> +               for (i = 0; i < altmap->reserve; i++) {
> +                       page = pfn_to_page(start_pfn + i);
> +                       __init_single_page(page, start_pfn + i, zone, nid);
> +                       SetPageReserved(page);
> +               }
>                 start_pfn += altmap->reserve;
> +       }

No, unfortunately this will clobber metadata that lives in that
reserved area, see __nvdimm_setup_pfn().

Re: [PATCH v1] mm: initialize struct page for reserved pages in ZONE_DEVICE

[Dan Williams]

On Thu, Jun 21, 2018 at 11:12 PM, Dan Williams <dan.j.williams@intel.com> wrote:
> On Thu, Jun 21, 2018 at 11:08 PM, Naoya Horiguchi
> <n-horiguchi@ah.jp.nec.com> wrote:
>> Reading /proc/kpageflags for pfns allocated by pmem namespace triggers
>> kernel panic with a message like "BUG: unable to handle kernel paging
>> request at fffffffffffffffe".
>>
>> The first few pages (controlled by altmap passed to memmap_init_zone())
>> in the ZONE_DEVICE can skip struct page initialization, which causes
>> the reported issue.
>>
>> This patch simply adds some initialization code for them.
>>
>> Fixes: 4b94ffdc4163 ("x86, mm: introduce vmem_altmap to augment vmemmap_populate()")
>> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
>> ---
>>  mm/page_alloc.c | 10 +++++++++-
>>  1 file changed, 9 insertions(+), 1 deletion(-)
>>
>> diff --git v4.17-mmotm-2018-06-07-16-59/mm/page_alloc.c v4.17-mmotm-2018-06-07-16-59_patched/mm/page_alloc.c
>> index 1772513..0b36afe 100644
>> --- v4.17-mmotm-2018-06-07-16-59/mm/page_alloc.c
>> +++ v4.17-mmotm-2018-06-07-16-59_patched/mm/page_alloc.c
>> @@ -5574,8 +5574,16 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
>>          * Honor reservation requested by the driver for this ZONE_DEVICE
>>          * memory
>>          */
>> -       if (altmap && start_pfn == altmap->base_pfn)
>> +       if (altmap && start_pfn == altmap->base_pfn) {
>> +               unsigned long i;
>> +
>> +               for (i = 0; i < altmap->reserve; i++) {
>> +                       page = pfn_to_page(start_pfn + i);
>> +                       __init_single_page(page, start_pfn + i, zone, nid);
>> +                       SetPageReserved(page);
>> +               }
>>                 start_pfn += altmap->reserve;
>> +       }
>
> No, unfortunately this will clobber metadata that lives in that
> reserved area, see __nvdimm_setup_pfn().

I think the kpageflags code needs to lookup the dev_pagemap in the
ZONE_DEVICE case and honor the altmap.

Re: [PATCH v1] mm: initialize struct page for reserved pages in ZONE_DEVICE

[Naoya Horiguchi]

On Thu, Jun 21, 2018 at 11:12:01PM -0700, Dan Williams wrote:
> On Thu, Jun 21, 2018 at 11:08 PM, Naoya Horiguchi
> <n-horiguchi@ah.jp.nec.com> wrote:
> > Reading /proc/kpageflags for pfns allocated by pmem namespace triggers
> > kernel panic with a message like "BUG: unable to handle kernel paging
> > request at fffffffffffffffe".
> >
> > The first few pages (controlled by altmap passed to memmap_init_zone())
> > in the ZONE_DEVICE can skip struct page initialization, which causes
> > the reported issue.
> >
> > This patch simply adds some initialization code for them.
> >
> > Fixes: 4b94ffdc4163 ("x86, mm: introduce vmem_altmap to augment vmemmap_populate()")
> > Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> > ---
> >  mm/page_alloc.c | 10 +++++++++-
> >  1 file changed, 9 insertions(+), 1 deletion(-)
> >
> > diff --git v4.17-mmotm-2018-06-07-16-59/mm/page_alloc.c v4.17-mmotm-2018-06-07-16-59_patched/mm/page_alloc.c
> > index 1772513..0b36afe 100644
> > --- v4.17-mmotm-2018-06-07-16-59/mm/page_alloc.c
> > +++ v4.17-mmotm-2018-06-07-16-59_patched/mm/page_alloc.c
> > @@ -5574,8 +5574,16 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
> >          * Honor reservation requested by the driver for this ZONE_DEVICE
> >          * memory
> >          */
> > -       if (altmap && start_pfn == altmap->base_pfn)
> > +       if (altmap && start_pfn == altmap->base_pfn) {
> > +               unsigned long i;
> > +
> > +               for (i = 0; i < altmap->reserve; i++) {
> > +                       page = pfn_to_page(start_pfn + i);
> > +                       __init_single_page(page, start_pfn + i, zone, nid);
> > +                       SetPageReserved(page);
> > +               }
> >                 start_pfn += altmap->reserve;
> > +       }
> 
> No, unfortunately this will clobber metadata that lives in that
> reserved area, see __nvdimm_setup_pfn().

Hi Dan,

This patch doesn't touch the reserved region itself, but only
struct pages on the region. I'm still not sure why it's necessary
to leave these struct pages uninitialized for pmem operation?

My another related concern is about memory_failure_dev_pagemap().
If a memory error happens on the reserved pfn range, this function
seems to try to access to the uninitialized struct page and maybe
trigger oops. So do we need something to prevent this?

Thanks,
Naoya Horiguchi

Re: [PATCH v1] mm: initialize struct page for reserved pages in ZONE_DEVICE

[Dan Williams]

On Thu, Jun 21, 2018 at 11:55 PM, Naoya Horiguchi
<n-horiguchi@ah.jp.nec.com> wrote:
> On Thu, Jun 21, 2018 at 11:12:01PM -0700, Dan Williams wrote:
>> On Thu, Jun 21, 2018 at 11:08 PM, Naoya Horiguchi
>> <n-horiguchi@ah.jp.nec.com> wrote:
>> > Reading /proc/kpageflags for pfns allocated by pmem namespace triggers
>> > kernel panic with a message like "BUG: unable to handle kernel paging
>> > request at fffffffffffffffe".
>> >
>> > The first few pages (controlled by altmap passed to memmap_init_zone())
>> > in the ZONE_DEVICE can skip struct page initialization, which causes
>> > the reported issue.
>> >
>> > This patch simply adds some initialization code for them.
>> >
>> > Fixes: 4b94ffdc4163 ("x86, mm: introduce vmem_altmap to augment vmemmap_populate()")
>> > Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
>> > ---
>> >  mm/page_alloc.c | 10 +++++++++-
>> >  1 file changed, 9 insertions(+), 1 deletion(-)
>> >
>> > diff --git v4.17-mmotm-2018-06-07-16-59/mm/page_alloc.c v4.17-mmotm-2018-06-07-16-59_patched/mm/page_alloc.c
>> > index 1772513..0b36afe 100644
>> > --- v4.17-mmotm-2018-06-07-16-59/mm/page_alloc.c
>> > +++ v4.17-mmotm-2018-06-07-16-59_patched/mm/page_alloc.c
>> > @@ -5574,8 +5574,16 @@ void __meminit memmap_init_zone(unsigned long size, int nid, unsigned long zone,
>> >          * Honor reservation requested by the driver for this ZONE_DEVICE
>> >          * memory
>> >          */
>> > -       if (altmap && start_pfn == altmap->base_pfn)
>> > +       if (altmap && start_pfn == altmap->base_pfn) {
>> > +               unsigned long i;
>> > +
>> > +               for (i = 0; i < altmap->reserve; i++) {
>> > +                       page = pfn_to_page(start_pfn + i);
>> > +                       __init_single_page(page, start_pfn + i, zone, nid);
>> > +                       SetPageReserved(page);
>> > +               }
>> >                 start_pfn += altmap->reserve;
>> > +       }
>>
>> No, unfortunately this will clobber metadata that lives in that
>> reserved area, see __nvdimm_setup_pfn().
>
> Hi Dan,
>
> This patch doesn't touch the reserved region itself, but only
> struct pages on the region. I'm still not sure why it's necessary
> to leave these struct pages uninitialized for pmem operation?
>
> My another related concern is about memory_failure_dev_pagemap().
> If a memory error happens on the reserved pfn range, this function
> seems to try to access to the uninitialized struct page and maybe
> trigger oops. So do we need something to prevent this?

Those pages are never mapped to userspace, so there is no opportunity
to consume the media error in that space. I'm still not sure it is
safe to initialize the pfns in the reserved range, I'll take a closer
look tomorrow. Otherwise, why not just delete the entire check and let
the main loop initialize the pages?