* [PATCH] x86/mm: Fix boot with some memory above MAXMEM
@ 2020-05-11 16:37 Kirill A. Shutemov
2020-05-11 16:43 ` Dave Hansen
2020-05-11 17:07 ` Kirill A. Shutemov
0 siblings, 2 replies; 11+ messages in thread
From: Kirill A. Shutemov @ 2020-05-11 16:37 UTC (permalink / raw)
To: Hansen, Dave, Dan Williams, Kleen, Andi
Cc: dave.hansen, linux-drivers-review, Kirill A. Shutemov, stable
A 5-level paging capable machine can have memory above 46-bit in the
physical address space. This memory is only addressable in the 5-level
paging mode: we don't have enough virtual address space to create direct
mapping for such memory in the 4-level paging mode.
Currently, we fail boot completely: NULL pointer dereference in
subsection_map_init().
Skip creating a memblock for such memory instead and notify user that
some memory is not addressable.
Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: stable@vger.kernel.org # v4.14
---
Tested with a hacked QEMU: https://gist.github.com/kiryl/d45eb54110944ff95e544972d8bdac1d
---
arch/x86/kernel/e820.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
index c5399e80c59c..022fe1de8940 100644
--- a/arch/x86/kernel/e820.c
+++ b/arch/x86/kernel/e820.c
@@ -1307,7 +1307,14 @@ void __init e820__memblock_setup(void)
if (entry->type != E820_TYPE_RAM && entry->type != E820_TYPE_RESERVED_KERN)
continue;
- memblock_add(entry->addr, entry->size);
+ if (entry->addr >= MAXMEM || end >= MAXMEM)
+ pr_err_once("Some physical memory is not addressable in the paging mode.\n");
+
+ if (entry->addr >= MAXMEM)
+ continue;
+
+ end = min_t(u64, end, MAXMEM - 1);
+ memblock_add(entry->addr, end - entry->addr);
}
/* Throw away partial pages: */
--
2.26.2
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH] x86/mm: Fix boot with some memory above MAXMEM
2020-05-11 16:37 [PATCH] x86/mm: Fix boot with some memory above MAXMEM Kirill A. Shutemov
@ 2020-05-11 16:43 ` Dave Hansen
2020-05-11 17:04 ` [linux-drivers-review] " Kirill A. Shutemov
2020-05-11 17:07 ` Kirill A. Shutemov
1 sibling, 1 reply; 11+ messages in thread
From: Dave Hansen @ 2020-05-11 16:43 UTC (permalink / raw)
To: Kirill A. Shutemov, Dan Williams, Kleen, Andi
Cc: dave.hansen, linux-drivers-review, stable
On 5/11/20 9:37 AM, Kirill A. Shutemov wrote:
> - memblock_add(entry->addr, entry->size);
> + if (entry->addr >= MAXMEM || end >= MAXMEM)
> + pr_err_once("Some physical memory is not addressable in the paging mode.\n");
Hi Kirill,
Thanks for fixing this!
Could we make the pr_err() a bit more informative, though? It would be
nice to print out how much memory (or which addresses at least) are
being thrown away.
I was also thinking that it would be handy to tell folks how to rectify
the situation. Should we perhaps dump out the runtime status of
X86_FEATURE_LA57?
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-drivers-review] [PATCH] x86/mm: Fix boot with some memory above MAXMEM
2020-05-11 16:43 ` Dave Hansen
@ 2020-05-11 17:04 ` Kirill A. Shutemov
2020-05-11 17:09 ` Dave Hansen
0 siblings, 1 reply; 11+ messages in thread
From: Kirill A. Shutemov @ 2020-05-11 17:04 UTC (permalink / raw)
To: Dave Hansen
Cc: Dan Williams, Kleen, Andi, dave.hansen, linux-drivers-review, stable
On Mon, May 11, 2020 at 09:43:30AM -0700, Dave Hansen wrote:
> On 5/11/20 9:37 AM, Kirill A. Shutemov wrote:
> > - memblock_add(entry->addr, entry->size);
> > + if (entry->addr >= MAXMEM || end >= MAXMEM)
> > + pr_err_once("Some physical memory is not addressable in the paging mode.\n");
>
> Hi Kirill,
>
> Thanks for fixing this!
>
> Could we make the pr_err() a bit more informative, though? It would be
> nice to print out how much memory (or which addresses at least) are
> being thrown away.
>
> I was also thinking that it would be handy to tell folks how to rectify
> the situation. Should we perhaps dump out the runtime status of
> X86_FEATURE_LA57?
Something like this (incremental patch)?
diff --git a/arch/x86/kernel/e820.c b/arch/x86/kernel/e820.c
index 022fe1de8940..172b4244069f 100644
--- a/arch/x86/kernel/e820.c
+++ b/arch/x86/kernel/e820.c
@@ -1280,8 +1280,8 @@ void __init e820__memory_setup(void)
void __init e820__memblock_setup(void)
{
+ u64 size, end, not_addressable = 0;
int i;
- u64 end;
/*
* The bootstrap memblock region count maximum is 128 entries
@@ -1307,16 +1307,24 @@ void __init e820__memblock_setup(void)
if (entry->type != E820_TYPE_RAM && entry->type != E820_TYPE_RESERVED_KERN)
continue;
- if (entry->addr >= MAXMEM || end >= MAXMEM)
- pr_err_once("Some physical memory is not addressable in the paging mode.\n");
-
- if (entry->addr >= MAXMEM)
+ if (entry->addr >= MAXMEM) {
+ not_addressable += entry->size;
continue;
+ }
end = min_t(u64, end, MAXMEM - 1);
+ size = end - entry->addr;
+ not_addressable += entry->size - size;
memblock_add(entry->addr, end - entry->addr);
}
+ if (not_addressable) {
+ pr_err("%lldMB of physical memory is not addressable in the paging mode\n",
+ not_addressable >> 20);
+ if (!pgtable_l5_enabled())
+ pr_err("Consider enabling 5-level paging\n");
+ }
+
/* Throw away partial pages: */
memblock_trim_memory(PAGE_SIZE);
--
Kirill A. Shutemov
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [linux-drivers-review] [PATCH] x86/mm: Fix boot with some memory above MAXMEM
2020-05-11 16:37 [PATCH] x86/mm: Fix boot with some memory above MAXMEM Kirill A. Shutemov
2020-05-11 16:43 ` Dave Hansen
@ 2020-05-11 17:07 ` Kirill A. Shutemov
1 sibling, 0 replies; 11+ messages in thread
From: Kirill A. Shutemov @ 2020-05-11 17:07 UTC (permalink / raw)
To: Hansen, Dave, Dan Williams, Kleen, Andi, dave.hansen,
linux-drivers-review, stable
On Mon, May 11, 2020 at 07:37:06PM +0300, Kirill A. Shutemov wrote:
> A 5-level paging capable machine can have memory above 46-bit in the
> physical address space. This memory is only addressable in the 5-level
> paging mode: we don't have enough virtual address space to create direct
> mapping for such memory in the 4-level paging mode.
>
> Currently, we fail boot completely: NULL pointer dereference in
> subsection_map_init().
>
> Skip creating a memblock for such memory instead and notify user that
> some memory is not addressable.
>
> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
> Cc: stable@vger.kernel.org # v4.14
> ---
>
> Tested with a hacked QEMU: https://gist.github.com/kiryl/d45eb54110944ff95e544972d8bdac1d
BTW, I was only able to boot with legacy SeaBIOS, not with OVMF. No idea
why.
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-drivers-review] [PATCH] x86/mm: Fix boot with some memory above MAXMEM
2020-05-11 17:04 ` [linux-drivers-review] " Kirill A. Shutemov
@ 2020-05-11 17:09 ` Dave Hansen
2020-05-11 18:10 ` Luck, Tony
0 siblings, 1 reply; 11+ messages in thread
From: Dave Hansen @ 2020-05-11 17:09 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Dan Williams, Kleen, Andi, dave.hansen, linux-drivers-review, stable
On 5/11/20 10:04 AM, Kirill A. Shutemov wrote:
> + if (not_addressable) {
> + pr_err("%lldMB of physical memory is not addressable in the paging mode\n",
> + not_addressable >> 20);
> + if (!pgtable_l5_enabled())
> + pr_err("Consider enabling 5-level paging\n");
> + }
Looks sane to me. Definitely good enough until we get the first bug
reports from an end user about how they screwed this up in practice.
For the aggregate patch:
Reviewed-by: Dave Hansen <dave.hansen@intel.com>
BTW, it's a shame that 0day and friends can't find stuff like this. I
have the feeling we have more bugs like this coming.
^ permalink raw reply [flat|nested] 11+ messages in thread
* RE: [linux-drivers-review] [PATCH] x86/mm: Fix boot with some memory above MAXMEM
2020-05-11 17:09 ` Dave Hansen
@ 2020-05-11 18:10 ` Luck, Tony
2020-05-11 18:43 ` Kirill A. Shutemov
0 siblings, 1 reply; 11+ messages in thread
From: Luck, Tony @ 2020-05-11 18:10 UTC (permalink / raw)
To: Hansen, Dave, Kirill A. Shutemov
Cc: Williams, Dan J, Kleen, Andi, dave.hansen, linux-drivers-review, stable
> + pr_err("%lldMB of physical memory is not addressable in the paging mode\n",
> + not_addressable >> 20);
Is "MB" the right unit for this. The problem seems to happen for systems with >64TB ... I doubt
the unaddressable memory is just a couple of MBbytes
-Tony
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-drivers-review] [PATCH] x86/mm: Fix boot with some memory above MAXMEM
2020-05-11 18:10 ` Luck, Tony
@ 2020-05-11 18:43 ` Kirill A. Shutemov
2020-05-11 18:58 ` Luck, Tony
0 siblings, 1 reply; 11+ messages in thread
From: Kirill A. Shutemov @ 2020-05-11 18:43 UTC (permalink / raw)
To: Luck, Tony
Cc: Hansen, Dave, Williams, Dan J, Kleen, Andi, dave.hansen,
linux-drivers-review, stable
On Mon, May 11, 2020 at 06:10:12PM +0000, Luck, Tony wrote:
> > + pr_err("%lldMB of physical memory is not addressable in the paging mode\n",
> > + not_addressable >> 20);
>
> Is "MB" the right unit for this. The problem seems to happen for systems with >64TB ... I doubt
> the unaddressable memory is just a couple of MBbytes
Change it to GB?
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 11+ messages in thread
* RE: [linux-drivers-review] [PATCH] x86/mm: Fix boot with some memory above MAXMEM
2020-05-11 18:43 ` Kirill A. Shutemov
@ 2020-05-11 18:58 ` Luck, Tony
2020-05-11 21:19 ` Andy Shevchenko
0 siblings, 1 reply; 11+ messages in thread
From: Luck, Tony @ 2020-05-11 18:58 UTC (permalink / raw)
To: Kirill A. Shutemov
Cc: Hansen, Dave, Williams, Dan J, Kleen, Andi, dave.hansen,
linux-drivers-review, stable
>> > + pr_err("%lldMB of physical memory is not addressable in the paging mode\n",
>> > + not_addressable >> 20);
>>
>> Is "MB" the right unit for this. The problem seems to happen for systems with >64TB ... I doubt
>> the unaddressable memory is just a couple of MBbytes
>
> Change it to GB?
I think it would be more readable.
[Maybe Linux needs a magic %p{something} that does auto-sizing to print in the most appropriate out of KB, MB, GB, TB, PB?]
-Tony
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-drivers-review] [PATCH] x86/mm: Fix boot with some memory above MAXMEM
2020-05-11 18:58 ` Luck, Tony
@ 2020-05-11 21:19 ` Andy Shevchenko
2020-05-12 0:50 ` Luck, Tony
0 siblings, 1 reply; 11+ messages in thread
From: Andy Shevchenko @ 2020-05-11 21:19 UTC (permalink / raw)
To: Luck, Tony
Cc: Kirill A. Shutemov, Hansen, Dave, Williams, Dan J, Kleen, Andi,
dave.hansen, linux-drivers-review, stable
On Mon, May 11, 2020 at 06:58:21PM +0000, Luck, Tony wrote:
> >> > + pr_err("%lldMB of physical memory is not addressable in the paging mode\n",
> >> > + not_addressable >> 20);
> >>
> >> Is "MB" the right unit for this. The problem seems to happen for systems with >64TB ... I doubt
> >> the unaddressable memory is just a couple of MBbytes
> >
> > Change it to GB?
>
> I think it would be more readable.
>
> [Maybe Linux needs a magic %p{something} that does auto-sizing to print in the most appropriate out of KB, MB, GB, TB, PB?]
We have one in string_helpers.c.
--
With Best Regards,
Andy Shevchenko
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-drivers-review] [PATCH] x86/mm: Fix boot with some memory above MAXMEM
2020-05-11 21:19 ` Andy Shevchenko
@ 2020-05-12 0:50 ` Luck, Tony
2020-05-12 10:05 ` Kirill A. Shutemov
0 siblings, 1 reply; 11+ messages in thread
From: Luck, Tony @ 2020-05-12 0:50 UTC (permalink / raw)
To: Andy Shevchenko
Cc: Kirill A. Shutemov, Hansen, Dave, Williams, Dan J, Kleen, Andi,
dave.hansen, linux-drivers-review, stable
On Tue, May 12, 2020 at 12:19:25AM +0300, Andy Shevchenko wrote:
> On Mon, May 11, 2020 at 06:58:21PM +0000, Luck, Tony wrote:
> > >> > + pr_err("%lldMB of physical memory is not addressable in the paging mode\n",
> > >> > + not_addressable >> 20);
> > >>
> > >> Is "MB" the right unit for this. The problem seems to happen for systems with >64TB ... I doubt
> > >> the unaddressable memory is just a couple of MBbytes
> > >
> > > Change it to GB?
> >
> > I think it would be more readable.
> >
> > [Maybe Linux needs a magic %p{something} that does auto-sizing to print in the most appropriate out of KB, MB, GB, TB, PB?]
>
> We have one in string_helpers.c.
Ah, nice. So:
#include <linux/string_helpers.h>
char tmp[10]; /* Bother, no #define for this, just a comment in string_helpers.c */
string_get_size(not_addressable, 1, STRING_UNITS_2, tmp, sizeof(tmp);
pr_err("%s of physical memory is not addressable in the paging mode\n", tmp);
-Tony
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [linux-drivers-review] [PATCH] x86/mm: Fix boot with some memory above MAXMEM
2020-05-12 0:50 ` Luck, Tony
@ 2020-05-12 10:05 ` Kirill A. Shutemov
0 siblings, 0 replies; 11+ messages in thread
From: Kirill A. Shutemov @ 2020-05-12 10:05 UTC (permalink / raw)
To: Luck, Tony
Cc: Andy Shevchenko, Hansen, Dave, Williams, Dan J, Kleen, Andi,
dave.hansen, linux-drivers-review, stable
On Mon, May 11, 2020 at 05:50:01PM -0700, Luck, Tony wrote:
> On Tue, May 12, 2020 at 12:19:25AM +0300, Andy Shevchenko wrote:
> > On Mon, May 11, 2020 at 06:58:21PM +0000, Luck, Tony wrote:
> > > >> > + pr_err("%lldMB of physical memory is not addressable in the paging mode\n",
> > > >> > + not_addressable >> 20);
> > > >>
> > > >> Is "MB" the right unit for this. The problem seems to happen for systems with >64TB ... I doubt
> > > >> the unaddressable memory is just a couple of MBbytes
> > > >
> > > > Change it to GB?
> > >
> > > I think it would be more readable.
> > >
> > > [Maybe Linux needs a magic %p{something} that does auto-sizing to print in the most appropriate out of KB, MB, GB, TB, PB?]
> >
> > We have one in string_helpers.c.
>
> Ah, nice. So:
>
> #include <linux/string_helpers.h>
>
>
> char tmp[10]; /* Bother, no #define for this, just a comment in string_helpers.c */
>
>
> string_get_size(not_addressable, 1, STRING_UNITS_2, tmp, sizeof(tmp);
>
> pr_err("%s of physical memory is not addressable in the paging mode\n", tmp);
I've already submitted the patch upstream. I'll add it if a new revision
is needed.
--
Kirill A. Shutemov
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2020-05-12 10:05 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-11 16:37 [PATCH] x86/mm: Fix boot with some memory above MAXMEM Kirill A. Shutemov
2020-05-11 16:43 ` Dave Hansen
2020-05-11 17:04 ` [linux-drivers-review] " Kirill A. Shutemov
2020-05-11 17:09 ` Dave Hansen
2020-05-11 18:10 ` Luck, Tony
2020-05-11 18:43 ` Kirill A. Shutemov
2020-05-11 18:58 ` Luck, Tony
2020-05-11 21:19 ` Andy Shevchenko
2020-05-12 0:50 ` Luck, Tony
2020-05-12 10:05 ` Kirill A. Shutemov
2020-05-11 17:07 ` Kirill A. Shutemov
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).