Re: Thoughts of AMX KVM support based on latest kernel

From: "Nakajima, Jun" <jun.nakajima@intel.com>
To: Paolo Bonzini <pbonzini@redhat.com>
Cc: "Tian, Kevin" <kevin.tian@intel.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	"Liu, Jing2" <jing2.liu@intel.com>,
	LKML <linux-kernel@vger.kernel.org>,
	"x86@kernel.org" <x86@kernel.org>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Arjan van de Ven <arjan@linux.intel.com>,
	Jing Liu <jing2.liu@linux.intel.com>,
	"seanjc@google.com" <seanjc@google.com>,
	"Cooper, Andrew" <andrew.cooper3@citrix.com>,
	"Bae, Chang Seok" <chang.seok.bae@intel.com>
Subject: Re: Thoughts of AMX KVM support based on latest kernel
Date: Thu, 18 Nov 2021 23:17:42 +0000	[thread overview]
Message-ID: <D93C093C-8420-45DA-99F5-0A5318ADBBEF@intel.com> (raw)
In-Reply-To: <c7d87723-5533-257f-fbf0-ecd3a0b96602@redhat.com>

On Nov 17, 2021, at 4:53 AM, Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
> On 11/17/21 11:15, Tian, Kevin wrote:
>> We are not sure whether such trick is worthwhile, since a sane
>> guest shouldn't set XFD[AMX]=1 before storing the AMX state. This
>> is why we want to seek SDM change to mark out that the software
>> should not assume XTILEDATA is still valid when XFD[AMX]=1.
> 
> Okay, I just don't want it to be called out as virtualization specific.
> 
> It doesn't have to happen in current processors, but it should be architecturally valid behavior to clear the processor's state as soon as a bit in XFD is set to 1.
> 
> Paolo
> 

We recommend that "system software initialize AMX state _before_ doing so" (below). Also, I think what the “creative” guest is doing is "lazy restore”, and "This approach will not operate correctly for a variety of reasons."

https://software.intel.com/sites/default/files/managed/c5/15/architecture-instruction-set-extensions-programming-reference.pdf

3.3 RECOMMENDATIONS FOR SYSTEM SOFTWARE

System software may disable use of Intel AMX by clearing XCR0[18:17], by clearing CR4.OSXSAVE, or by setting
IA32_XFD[18]. It is recommended that system software initialize AMX state (e.g., by executing TILERELEASE)
before doing so. This is because maintaining AMX state in a non-initialized state may have negative power and performance implications.

System software should not use XFD to implement a “lazy restore” approach to management of the XTILEDATA
state component. This approach will not operate correctly for a variety of reasons. One is that the LDTILECFG and TILERELEASE instructions initialize XTILEDATA and do not cause an #NM exception. Another is that an execution of XSAVE by a user thread will save XTILEDATA as initialized instead of the data expected by the user thread.

--- 
Jun