Re: [PATCH] selftests: KVM: avoid failures due to reserved HyperTransport region

From: Joao Martins <joao.m.martins@oracle.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: stable@vger.kernel.org, David Matlack <dmatlack@google.com>,
	Maxim Levitsky <mlevitsk@redhat.com>,
	linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Subject: Re: [PATCH] selftests: KVM: avoid failures due to reserved HyperTransport region
Date: Fri, 6 Aug 2021 11:57:38 +0100	[thread overview]
Message-ID: <4b530fb6-81cc-be36-aa68-92ec01c65775@oracle.com> (raw)
In-Reply-To: <20210805105423.412878-1-pbonzini@redhat.com>

On 8/5/21 11:54 AM, Paolo Bonzini wrote:
> Accessing guest physical addresses at 0xFFFD_0000_0000 and above causes
> a failure on AMD processors because those addresses are reserved by
> HyperTransport (this is not documented).  

Oh, but it's actually documented in the AMD IOMMU manual [0] (and AMD IOMMU in linux do
mark it as a reserved IOVA region i.e. HT_RANGE_START..HT_RANGE_END). And it's usually
marked as a reserved type in E820. At least on the machines I've seen.

See manual section '2.1.2 IOMMU Logical Topology':

"Special address controls in Table 3 are interpreted against untranslated guest physical
addressess (GPA) that lack a PASID TLP prefix."

 Base Address   Top Address   Use

  FD_0000_0000h FD_F7FF_FFFFh Reserved interrupt address space
  FD_F800_0000h FD_F8FF_FFFFh Interrupt/EOI IntCtl
  FD_F900_0000h FD_F90F_FFFFh Legacy PIC IACK
  FD_F910_0000h FD_F91F_FFFFh System Management
  FD_F920_0000h FD_FAFF_FFFFh Reserved Page Tables
  FD_FB00_0000h FD_FBFF_FFFFh Address Translation
  FD_FC00_0000h FD_FDFF_FFFFh I/O Space
  FD_FE00_0000h FD_FFFF_FFFFh Configuration
  FE_0000_0000h FE_1FFF_FFFFh Extended Configuration/Device Messages
  FE_2000_0000h FF_FFFF_FFFFh Reserved

It covers the range starting that address you fixed up ... up to 1Tb, fwiw.

You mark it ~1010G as max gfn so shouldn't be a problem.

[0] https://www.amd.com/system/files/TechDocs/48882_IOMMU.pdf

> Avoid selftests failures
> by reserving those guest physical addresses.
> 
> Fixes: ef4c9f4f6546 ("KVM: selftests: Fix 32-bit truncation of vm_get_max_gfn()")
> Cc: stable@vger.kernel.org
> Cc: David Matlack <dmatlack@google.com>
> Reported-by: Maxim Levitsky <mlevitsk@redhat.com>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  tools/testing/selftests/kvm/lib/kvm_util.c | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
> index 10a8ed691c66..d995cc9836ee 100644
> --- a/tools/testing/selftests/kvm/lib/kvm_util.c
> +++ b/tools/testing/selftests/kvm/lib/kvm_util.c
> @@ -309,6 +309,12 @@ struct kvm_vm *vm_create(enum vm_guest_mode mode, uint64_t phy_pages, int perm)
>  	/* Limit physical addresses to PA-bits. */
>  	vm->max_gfn = ((1ULL << vm->pa_bits) >> vm->page_shift) - 1;
>  
> +#ifdef __x86_64__
> +	/* Avoid reserved HyperTransport region on AMD processors.  */
> +	if (vm->pa_bits == 48)
> +		vm->max_gfn = 0xfffcfffff;
> +#endif
> +

Not sure if it's worth the trouble having a macro with the same name as AMD iommu like:

#define HT_RANGE_START                (0xfd00000000ULL)
#define MAX_GFN			      (HT_RANGE_START - 1ULL)

#ifdef __x86_64__
	/* Avoid reserved HyperTransport region on AMD processors.  */
	if (vm->pa_bits == 48)
		vm->max_gfn = MAX_GFN;
#endif

It's a detail, but *perhaps* would help people grepping around it.

Also, not sure if checking against AMD cpuid vendor is worth, considering this is
a limitation only on AMD.

>  	/* Allocate and setup memory for guest. */
>  	vm->vpages_mapped = sparsebit_alloc();
>  	if (phy_pages != 0)
> 