From: Maxim Levitsky <mlevitsk@redhat.com>
To: Joao Martins <joao.m.martins@oracle.com>,
Paolo Bonzini <pbonzini@redhat.com>
Cc: stable@vger.kernel.org, David Matlack <dmatlack@google.com>,
linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Subject: Re: [PATCH] selftests: KVM: avoid failures due to reserved HyperTransport region
Date: Mon, 09 Aug 2021 10:45:27 +0300 [thread overview]
Message-ID: <33d01b8bb31541be7911f95581cdf608c6c79bf6.camel@redhat.com> (raw)
In-Reply-To: <4b530fb6-81cc-be36-aa68-92ec01c65775@oracle.com>
On Fri, 2021-08-06 at 11:57 +0100, Joao Martins wrote:
>
> On 8/5/21 11:54 AM, Paolo Bonzini wrote:
> > Accessing guest physical addresses at 0xFFFD_0000_0000 and above causes
> > a failure on AMD processors because those addresses are reserved by
> > HyperTransport (this is not documented).
>
> Oh, but it's actually documented in the AMD IOMMU manual [0] (and AMD IOMMU in linux do
> mark it as a reserved IOVA region i.e. HT_RANGE_START..HT_RANGE_END). And it's usually
> marked as a reserved type in E820. At least on the machines I've seen.
>
> See manual section '2.1.2 IOMMU Logical Topology':
>
> "Special address controls in Table 3 are interpreted against untranslated guest physical
> addressess (GPA) that lack a PASID TLP prefix."
>
> Base Address Top Address Use
>
> FD_0000_0000h FD_F7FF_FFFFh Reserved interrupt address space
> FD_F800_0000h FD_F8FF_FFFFh Interrupt/EOI IntCtl
> FD_F900_0000h FD_F90F_FFFFh Legacy PIC IACK
> FD_F910_0000h FD_F91F_FFFFh System Management
> FD_F920_0000h FD_FAFF_FFFFh Reserved Page Tables
> FD_FB00_0000h FD_FBFF_FFFFh Address Translation
> FD_FC00_0000h FD_FDFF_FFFFh I/O Space
> FD_FE00_0000h FD_FFFF_FFFFh Configuration
> FE_0000_0000h FE_1FFF_FFFFh Extended Configuration/Device Messages
> FE_2000_0000h FF_FFFF_FFFFh Reserved
>
> It covers the range starting that address you fixed up ... up to 1Tb, fwiw.
>
> You mark it ~1010G as max gfn so shouldn't be a problem.
>
> [0] https://www.amd.com/system/files/TechDocs/48882_IOMMU.pdf
>
> > Avoid selftests failures
> > by reserving those guest physical addresses.
> >
> > Fixes: ef4c9f4f6546 ("KVM: selftests: Fix 32-bit truncation of vm_get_max_gfn()")
> > Cc: stable@vger.kernel.org
> > Cc: David Matlack <dmatlack@google.com>
> > Reported-by: Maxim Levitsky <mlevitsk@redhat.com>
> > Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> > ---
> > tools/testing/selftests/kvm/lib/kvm_util.c | 6 ++++++
> > 1 file changed, 6 insertions(+)
> >
> > diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
> > index 10a8ed691c66..d995cc9836ee 100644
> > --- a/tools/testing/selftests/kvm/lib/kvm_util.c
> > +++ b/tools/testing/selftests/kvm/lib/kvm_util.c
> > @@ -309,6 +309,12 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
> > /* Limit physical addresses to PA-bits. */
> > vm->max_gfn = ((1ULL << vm->pa_bits) >> vm->page_shift) - 1;
> >
> > +#ifdef __x86_64__
> > + /* Avoid reserved HyperTransport region on AMD processors. */
> > + if (vm->pa_bits == 48)
> > + vm->max_gfn = 0xfffcfffff;
> > +#endif
> > +
>
> Not sure if it's worth the trouble having a macro with the same name as AMD iommu like:
>
> #define HT_RANGE_START (0xfd00000000ULL)
> #define MAX_GFN (HT_RANGE_START - 1ULL)
>
> #ifdef __x86_64__
> /* Avoid reserved HyperTransport region on AMD processors. */
> if (vm->pa_bits == 48)
> vm->max_gfn = MAX_GFN;
> #endif
I guess now that we know that it is documented, it is worth it,
to remove '== 48' check and add check for an AMD cpu, and add reference
to this manual.
I am mentioning the 48 bit check because I have seen that AMD just recently
posted 5 level NPT support, so I guess CPUs which > 48 bit max physical address
are also probably on horison.
And long term solution for this I guess is to add these areas to a blacklist
and avoid them.
Best regards,
Maxim Levitsky
>
> It's a detail, but *perhaps* would help people grepping around it.
>
> Also, not sure if checking against AMD cpuid vendor is worth, considering this is
> a limitation only on AMD.
>
>
> > /* Allocate and setup memory for guest. */
> > vm->vpages_mapped = sparsebit_alloc();
> > if (phy_pages != 0)
> >
next prev parent reply other threads:[~2021-08-09 7:45 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-05 10:54 [PATCH] selftests: KVM: avoid failures due to reserved HyperTransport region Paolo Bonzini
2021-08-05 11:23 ` Maxim Levitsky
2021-08-06 10:57 ` Joao Martins
2021-08-09 7:45 ` Maxim Levitsky [this message]
2021-08-09 9:03 ` Paolo Bonzini
2021-08-09 10:00 ` Joao Martins
2021-08-09 10:23 ` Paolo Bonzini
2021-12-08 16:54 ` Sean Christopherson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=33d01b8bb31541be7911f95581cdf608c6c79bf6.camel@redhat.com \
--to=mlevitsk@redhat.com \
--cc=dmatlack@google.com \
--cc=joao.m.martins@oracle.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pbonzini@redhat.com \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).