Re: OVMF very slow on AMD

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 28/07/16 20:25, Boris Ostrovsky wrote:
> On 07/28/2016 11:51 AM, Andrew Cooper wrote:
>> On 28/07/16 16:17, Boris Ostrovsky wrote:
>>> On 07/28/2016 06:54 AM, Andrew Cooper wrote:
>>>> On 28/07/16 11:43, George Dunlap wrote:
>>>>> On Thu, Jul 28, 2016 at 11:18 AM, Anthony PERARD
>>>>> <anthony.perard@citrix.com> wrote:
>>>>>> On Wed, Jul 27, 2016 at 03:45:23PM -0400, Boris Ostrovsky wrote:
>>>>>>> On 07/27/2016 07:35 AM, Anthony PERARD wrote:
>>>>>>>> On Wed, Jul 27, 2016 at 12:08:04PM +0100, Anthony PERARD wrote:
>>>>>>>>> I can try to describe how OVMF is setting up the memory.
>>>>>>>> From the start of the day:
>>>>>>>> setup gdt
>>>>>>>> cr0 = 0x40000023
>>>>>>> I think this is slightly odd, with bit 30 (cache disable) set. I'd
>>>>>>> suspect that this would affect both Intel and AMD though.
>>>>>>>
>>>>>>> Can you try clearing this bit?
>>>>>> That works...
>>>>>>
>>>>>> I wonder why it does not appear to affect Intel or KVM.
>>>>> Are those bits hard-coded, or are they set based on the hardware
>>>>> that's available?
>>>>>
>>>>> Is it possible that the particular combination of CPUID bits presented
>>>>> by Xen on AMD are causing a different value to be written?
>>>>>
>>>>> Or is it possible that the cache disable bit is being ignored (by Xen)
>>>>> on Intel and KVM?
>>>> If a guest has no hardware, then it has no reason to actually disable
>>>> caches.  We should have logic to catch this an avoid actually disabling
>>>> caches when the guest asks for it.
>>> Is this really safe to do? Can't a guest decide to disable cache to
>>> avoid having to deal with coherency in SW?
>> What SW coherency issue do you think can be solved with disabling the cache?
>>
>> x86 has strict ordering of writes and reads with respect to each other. 
>> The only case which can be out of order is reads promoted ahead of
>> unaliasing writes.
> Right, that was not a good example.
>
>>> As far as Intel vs AMD implementation in Xen, we have vmx_handle_cd()
>>> but no corresponding SVM code. Could it be that we need to set gPAT, for
>>> example?
>> A better approach would be to find out why ovmf insists on disabling
>> caches at all.  Even if we optimise the non-PCI-device case in the
>> hypervisor, a passthrough case will still run like treacle if caches are
>> disabled.
> True, we should understand why OVMF does this. But I think we also need
> to understand what makes Intel run faster. Or is it already clear from
> vmx_handle_cd()?

Wow this code is hard to follow :(

handle_cd() is only called when an IOMMU is enabled and the domain in
question has access to real ioports or PCI devices.

However, I really can't spot anything that ends up eliding the
cache-disable setting even for Intel.  This clearly needs further
investigation.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel