Re: [PATCH v1] drivers/base/memory.c: indicate all memory blocks as removable

[Dan Williams <dan.j.williams@intel.com>]

On Fri, Mar 27, 2020 at 9:50 AM David Hildenbrand <david@redhat.com> wrote:
>
> On 27.03.20 17:28, Dan Williams wrote:
> > On Fri, Mar 27, 2020 at 2:00 AM David Hildenbrand <david@redhat.com> wrote:
> >>
> >> On 27.03.20 08:47, Michal Hocko wrote:
> >>> On Thu 26-03-20 23:24:08, Dan Williams wrote:
> >>> [...]
> >>>> David, Andrew,
> >>>>
> >>>> I'd like to recommend this patch for -stable as it likely (test
> >>>> underway) solves this crash report from Steve:
> >>>>
> >>>> [  148.796036] page dumped because: VM_BUG_ON_PAGE(PagePoisoned(p))
> >>>> [  148.796074] ------------[ cut here ]------------
> >>>> [  148.796098] kernel BUG at include/linux/mm.h:1087!
> >>>> [  148.796126] invalid opcode: 0000 [#1] SMP NOPTI
> >>>> [  148.796146] CPU: 63 PID: 5471 Comm: lsmem Not tainted 5.5.10-200.fc31.x8=
> >>>> 6_64+debug #1
> >>>> [  148.796173] Hardware name: Intel Corporation S2600WFD/S2600WFD, BIOS SE5=
> >>>> C620.86B.02.01.0010.010620200716 01/06/2020
> >>>> [  148.796212] RIP: 0010:is_mem_section_removable+0x1a4/0x1b0
> >>>> [  148.796561] Call Trace:
> >>>> [  148.796591]  removable_show+0x6e/0xa0
> >>>> [  148.796608]  dev_attr_show+0x19/0x40
> >>>> [  148.796625]  sysfs_kf_seq_show+0xa9/0x100
> >>>> [  148.796640]  seq_read+0xd5/0x450
> >>>> [  148.796657]  vfs_read+0xc5/0x180
> >>>> [  148.796672]  ksys_read+0x68/0xe0
> >>>> [  148.796688]  do_syscall_64+0x5c/0xa0
> >>>> [  148.796704]  entry_SYSCALL_64_after_hwframe+0x49/0xbe
> >>>> [  148.796721] RIP: 0033:0x7f3ab1646412
> >>>>
> >>>> ...on a non-debug kernel it just crashes.
> >>>>
> >>>> In this case lsmem is failing when reading memory96:
> >>>>
> >>>> openat(3, "memory96/removable", O_RDONLY|O_CLOEXEC) = 4
> >>>> fcntl(4, F_GETFL)                       = 0x8000 (flags O_RDONLY|O_LARGEFILE)
> >>>> fstat(4, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0
> >>>> read(4,  <unfinished ...>)              = ?
> >>>> +++ killed by SIGSEGV +++
> >>>> Segmentation fault (core dumped)
> >>>>
> >>>> ...which is phys_index 0x60 => memory address 0x3000000000
> >>>>
> >>>> On this platform that lands us here:
> >>>>
> >>>> 100000000-303fffffff : System RAM
> >>>>   291f000000-291fe00f70 : Kernel code
> >>>>   2920000000-292051efff : Kernel rodata
> >>>>   2920600000-292093b0bf : Kernel data
> >>>>   29214f3000-2922dfffff : Kernel bss
> >>>> 3040000000-305fffffff : Reserved
> >>>> 3060000000-1aa5fffffff : Persistent Memory
> >>>
> >>> OK, 2GB memblocks and that would mean [0x3000000000, 0x3080000000]
> >>>
> >>>> ...where the last memory block of System RAM is shared with persistent
> >>>> memory. I.e. the block is only partially online which means that
> >>>> page_to_nid() in is_mem_section_removable() will assert or crash for
> >>>> some of the offline pages in that block.
> >>>
> >>> Yes, this patch is a simple workaround. Normal memory hotplug will not
> >>> blow up because it should be able to find out that test_pages_in_a_zone
> >>> is false. Who knows how other potential pfn walkers handle that.
> >>
> >> All other pfn walkers now correctly use pfn_to_online_page() - which
> >> will also result in false positives in this scenario and is still to be
> >> fixed by Dan IIRC. [1]
> >
> > Sorry, it's been too long and this fell out of my cache. I also turned
> > away once the major fire in KVM was put out with special consideration
> > for for devmem pages. What's left these days? ...besides
> > removable_show()?
>
> Essentially any pfn_to_online_page() is a candidate.
>
> E.g.,
>
> mm/memory-failure.c:memory_failure()
>
> is obviously broken (could be worked around)

Ooh, the current state looks worse than when I looked previously. I
wasn't copied on commit 96c804a6ae8c ("mm/memory-failure.c: don't
access uninitialized memmaps in memory_failure()"). That commit seems
to ensure the pmem errors in memory sections that overlap with
System-RAM are not handled. So that change looks broken to me.
Previously get_devpagemap() was sufficient protection.

>
> Also
>
> mm/memory-failure.c:soft_offline_page()
>
> is obviously broken.

How exactly? The soft_offline_page() callers seem to already account
for System-RAM vs devmem.

>
>
> Also set_zone_contiguous()->__pageblock_pfn_to_page() is broken, when it
> checks for "page_zone(start_page) != zone" if the memmap contains garbage.
>
> And I only checked a handful of examples.

Ok, but as the first example shows in the absence of a problem report
these pre-emptive changes might make things worse so I don't think
it's as simple as go instrument all the pfn_to_online_page() users.