Re: [PATCH] x86/mm: Fix boot with some memory above MAXMEM

From: Dave Hansen <dave.hansen@intel.com>
To: "Kirill A. Shutemov" <kirill@shutemov.name>,
	Mike Rapoport <rppt@kernel.org>
Cc: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Andy Lutomirski <luto@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Dan Williams <dan.j.williams@intel.com>,
	Tony Luck <tony.luck@intel.com>,
	x86@kernel.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	stable@vger.kernel.org
Subject: Re: [PATCH] x86/mm: Fix boot with some memory above MAXMEM
Date: Tue, 26 May 2020 07:27:15 -0700	[thread overview]
Message-ID: <24b51944-bfba-a937-484a-5d9ec54fdf01@intel.com> (raw)
In-Reply-To: <20200525150820.zljiamptpzi37ohx@box>

On 5/25/20 8:08 AM, Kirill A. Shutemov wrote:
>>>> +	if (not_addressable) {
>>>> +		pr_err("%lldGB of physical memory is not addressable in the paging mode\n",
>>>> +		       not_addressable >> 30);
>>>> +		if (!pgtable_l5_enabled())
>>>> +			pr_err("Consider enabling 5-level paging\n");
>> Could this happen at all when l5 is enabled?
>> Does it mean we need kmap() for 64-bit?
> It's future-profing. Who knows what paging modes we would have in the
> future.

Future-proofing and firmware-proofing. :)

In any case, are we *really* limited to 52 bits of physical memory with
5-level paging?  Previously, we said we were limited to 46 bits, and now
we're saying that the limit is 52 with 5-level paging:

#define MAX_PHYSMEM_BITS (pgtable_l5_enabled() ? 52 : 46)

The 46 was fine with the 48 bits of address space on 4-level paging
systems since we need 1/2 of the address space for userspace, 1/4 for
the direct map and 1/4 for the vmalloc-and-friends area.  At 46 bits of
address space, we fill up the direct map.

The hardware designers know this and never enumerated a MAXPHYADDR from
CPUID which was higher than what we could cover with 46 bits.  It was
nice and convenient that these two separate things matched:
1. The amount of physical address space addressable in a direct map
   consuming 1/4 of the virtual address space.
2. The CPU-enumerated MAXPHYADDR which among other things dictates how
   much physical address space is addressable in a PTE.

But, with 5-level paging, things are a little different.  The limit in
addressable memory because of running out of the direct map actually
happens at 55 bits (57-2=55, analogous to the 4-level 48-2=46).

So shouldn't it technically be this:

#define MAX_PHYSMEM_BITS (pgtable_l5_enabled() ? 55 : 46)

?