Re: AMD VMMCALL and VM86 mode

Re: AMD VMMCALL and VM86 mode

[Boris Ostrovsky]

----- andrew.cooper3@citrix.com wrote:

> On 09/12/16 19:55, Andrew Cooper wrote:
> > On 09/12/16 19:55, Boris Ostrovsky wrote:
> >> On 12/09/2016 02:01 PM, Andrew Cooper wrote:
> >>> Hello,
> >>>
> >>> While working on XSA-192, I found a curious thing.  On AMD
> hardware, the
> >>> VMMCALL instruction appears to behave like a nop if executed in
> VM86
> >>> mode.  All other processor modes work fine.
> >>>
> >>> The documentation suggests it should be valid in any situation,
> but I
> >>> never get a #VMEXIT from it. 
> >> And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we
> have in
> >> Xen by default)?
> > Yes, because I have already used hypercalls to get text to the
> console
> > before entering vm86 mode.
> >
> >> What happens if you don't set it?
> > Let me do some hacking and see.
> 
> Outside of vm86 mode, VMMCALL raises #UD, which is expected as it
> wasn't
> intercepted.
> 
> From within vm86 mode, I now get #GP rather than #UD.
> 
> There is certainly an argument to be made that VMMCALL inside vm86
> mode
> should trap to the vm86 monitor and #GP would be the expected way of
> that happening, but this also doesn't match the documentation.


Just curious: why do you think #GP could be a reasonable exception here? It's #UD because if not intercepted it doesn't make sense to execute it.

But either way, I think AMD should clarify this. Suravee, can you find out what the expected behavior is?

-boris

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: AMD VMMCALL and VM86 mode

[Andrew Cooper]

On 11/12/16 17:33, Boris Ostrovsky wrote:
> ----- andrew.cooper3@citrix.com wrote:
>
>> On 09/12/16 19:55, Andrew Cooper wrote:
>>> On 09/12/16 19:55, Boris Ostrovsky wrote:
>>>> On 12/09/2016 02:01 PM, Andrew Cooper wrote:
>>>>> Hello,
>>>>>
>>>>> While working on XSA-192, I found a curious thing.  On AMD
>> hardware, the
>>>>> VMMCALL instruction appears to behave like a nop if executed in
>> VM86
>>>>> mode.  All other processor modes work fine.
>>>>>
>>>>> The documentation suggests it should be valid in any situation,
>> but I
>>>>> never get a #VMEXIT from it. 
>>>> And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we
>> have in
>>>> Xen by default)?
>>> Yes, because I have already used hypercalls to get text to the
>> console
>>> before entering vm86 mode.
>>>
>>>> What happens if you don't set it?
>>> Let me do some hacking and see.
>> Outside of vm86 mode, VMMCALL raises #UD, which is expected as it
>> wasn't
>> intercepted.
>>
>> From within vm86 mode, I now get #GP rather than #UD.
>>
>> There is certainly an argument to be made that VMMCALL inside vm86
>> mode
>> should trap to the vm86 monitor and #GP would be the expected way of
>> that happening, but this also doesn't match the documentation.
>
> Just curious: why do you think #GP could be a reasonable exception here?

In vm86 mode, #GP is raised for privileged instructions which should
break for auditing in the vm86 monitor.  It would be reasonable to
include "talking to the hypervisor" as a privileged action.

> It's #UD because if not intercepted it doesn't make sense to execute it.

I agree with this, if privilege isn't considered an issue.  If a
hypervisor doesn't actually get involved, the instruction shouldn't
complete.

> But either way, I think AMD should clarify this. Suravee, can you find out what the expected behavior is?

IMO, it should either consistently #GP and break to the vm86 monitor, or
#UD/#VMEXIT depending on whether it is intercepted by the hypervisor. 
Either way the documentation should be clarified.

Having said this, given that its behaviour is consistent on any AMD
processor I choose to try, and given that vm86 mode is very legacy these
days, I doubt a reasonable argument can be made to fixing the behaviour.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: AMD VMMCALL and VM86 mode

[Suravee Suthikulpanit]

Hi Andrew/Boris,

On 12/12/16 01:37, Andrew Cooper wrote:
> On 11/12/16 17:33, Boris Ostrovsky wrote:
>> ----- andrew.cooper3@citrix.com wrote:
>>
>>> On 09/12/16 19:55, Andrew Cooper wrote:
>>>> On 09/12/16 19:55, Boris Ostrovsky wrote:
>>>>> On 12/09/2016 02:01 PM, Andrew Cooper wrote:
>>>>>> Hello,
>>>>>>
>>>>>> While working on XSA-192, I found a curious thing.  On AMD
>>> hardware, the
>>>>>> VMMCALL instruction appears to behave like a nop if executed in
>>> VM86
>>>>>> mode.  All other processor modes work fine.
>>>>>>
>>>>>> The documentation suggests it should be valid in any situation,
>>> but I
>>>>>> never get a #VMEXIT from it.
>>>>> And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we
>>> have in
>>>>> Xen by default)?
>>>> Yes, because I have already used hypercalls to get text to the
>>> console
>>>> before entering vm86 mode.
>>>>
>>>>> What happens if you don't set it?
>>>> Let me do some hacking and see.
>>> Outside of vm86 mode, VMMCALL raises #UD, which is expected as it
>>> wasn't
>>> intercepted.
>>>
>>> From within vm86 mode, I now get #GP rather than #UD.
>>>
>>> There is certainly an argument to be made that VMMCALL inside vm86
>>> mode
>>> should trap to the vm86 monitor and #GP would be the expected way of
>>> that happening, but this also doesn't match the documentation.
>>
>> Just curious: why do you think #GP could be a reasonable exception here?
>
> In vm86 mode, #GP is raised for privileged instructions which should
> break for auditing in the vm86 monitor.  It would be reasonable to
> include "talking to the hypervisor" as a privileged action.
>
>> It's #UD because if not intercepted it doesn't make sense to execute it.
>
> I agree with this, if privilege isn't considered an issue.  If a
> hypervisor doesn't actually get involved, the instruction shouldn't
> complete.
>
>> But either way, I think AMD should clarify this. Suravee, can you find out what the expected behavior is?
>
> IMO, it should either consistently #GP and break to the vm86 monitor, or
> #UD/#VMEXIT depending on whether it is intercepted by the hypervisor.
> Either way the documentation should be clarified.
>
> Having said this, given that its behaviour is consistent on any AMD
> processor I choose to try, and given that vm86 mode is very legacy these
> days, I doubt a reasonable argument can be made to fixing the behaviour.
>
> ~Andrew
>

I'm checking with the HW folks. Let me get back to you.

Suravee

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: AMD VMMCALL and VM86 mode

[Suravee Suthikulpanit]

On 12/12/16 01:37, Andrew Cooper wrote:
> On 11/12/16 17:33, Boris Ostrovsky wrote:
>> ----- andrew.cooper3@citrix.com wrote:
>>
>>> On 09/12/16 19:55, Andrew Cooper wrote:
>>>> On 09/12/16 19:55, Boris Ostrovsky wrote:
>>>>> On 12/09/2016 02:01 PM, Andrew Cooper wrote:
>>>>>> Hello,
>>>>>>
>>>>>> While working on XSA-192, I found a curious thing.  On AMD
>>> hardware, the
>>>>>> VMMCALL instruction appears to behave like a nop if executed in
>>> VM86
>>>>>> mode.  All other processor modes work fine.
>>>>>>
>>>>>> The documentation suggests it should be valid in any situation,
>>> but I
>>>>>> never get a #VMEXIT from it.
>>>>> And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we
>>> have in
>>>>> Xen by default)?
>>>> Yes, because I have already used hypercalls to get text to the
>>> console
>>>> before entering vm86 mode.
>>>>
>>>>> What happens if you don't set it?
>>>> Let me do some hacking and see.
>>> Outside of vm86 mode, VMMCALL raises #UD, which is expected as it
>>> wasn't
>>> intercepted.
>>>
>>> From within vm86 mode, I now get #GP rather than #UD.
>>>
>>> There is certainly an argument to be made that VMMCALL inside vm86
>>> mode
>>> should trap to the vm86 monitor and #GP would be the expected way of
>>> that happening, but this also doesn't match the documentation.
>>
>> Just curious: why do you think #GP could be a reasonable exception here?
>
> In vm86 mode, #GP is raised for privileged instructions which should
> break for auditing in the vm86 monitor.  It would be reasonable to
> include "talking to the hypervisor" as a privileged action.
>
>> It's #UD because if not intercepted it doesn't make sense to execute it.
>
> I agree with this, if privilege isn't considered an issue.  If a
> hypervisor doesn't actually get involved, the instruction shouldn't
> complete.
>
>> But either way, I think AMD should clarify this. Suravee, can you find out what the expected behavior is?
>
> IMO, it should either consistently #GP and break to the vm86 monitor, or
> #UD/#VMEXIT depending on whether it is intercepted by the hypervisor.
> Either way the documentation should be clarified.
>
> Having said this, given that its behaviour is consistent on any AMD
> processor I choose to try, and given that vm86 mode is very legacy these
> days, I doubt a reasonable argument can be made to fixing the behaviour.
>
> ~Andrew
>

So, just to confirm your observation. When executing VMMCALL in vm86 
mode with the hypervisor is set to intercept VMMCALL and not-intercept 
VMMCALL, you are seeing #GP in both cases or just in the later?

According to the HW folks, in the vm86 mode, we should observe the same 
behavior as in protected mode. (e.g. #UD in guest if not intercept 
VMMCALL, and #vmexit if intercept)

Could you please describe how you are reproducing this behavior? What 
processors are you using to reproduce this behavior?

Thanks,
Suravee

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: AMD VMMCALL and VM86 mode

[Andrew Cooper]

On 25/12/2016 07:47, Suravee Suthikulpanit wrote:
>
>
> On 12/12/16 01:37, Andrew Cooper wrote:
>> On 11/12/16 17:33, Boris Ostrovsky wrote:
>>> ----- andrew.cooper3@citrix.com wrote:
>>>
>>>> On 09/12/16 19:55, Andrew Cooper wrote:
>>>>> On 09/12/16 19:55, Boris Ostrovsky wrote:
>>>>>> On 12/09/2016 02:01 PM, Andrew Cooper wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> While working on XSA-192, I found a curious thing.  On AMD
>>>> hardware, the
>>>>>>> VMMCALL instruction appears to behave like a nop if executed in
>>>> VM86
>>>>>>> mode.  All other processor modes work fine.
>>>>>>>
>>>>>>> The documentation suggests it should be valid in any situation,
>>>> but I
>>>>>>> never get a #VMEXIT from it.
>>>>>> And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we
>>>> have in
>>>>>> Xen by default)?
>>>>> Yes, because I have already used hypercalls to get text to the
>>>> console
>>>>> before entering vm86 mode.
>>>>>
>>>>>> What happens if you don't set it?
>>>>> Let me do some hacking and see.
>>>> Outside of vm86 mode, VMMCALL raises #UD, which is expected as it
>>>> wasn't
>>>> intercepted.
>>>>
>>>> From within vm86 mode, I now get #GP rather than #UD.
>>>>
>>>> There is certainly an argument to be made that VMMCALL inside vm86
>>>> mode
>>>> should trap to the vm86 monitor and #GP would be the expected way of
>>>> that happening, but this also doesn't match the documentation.
>>>
>>> Just curious: why do you think #GP could be a reasonable exception
>>> here?
>>
>> In vm86 mode, #GP is raised for privileged instructions which should
>> break for auditing in the vm86 monitor.  It would be reasonable to
>> include "talking to the hypervisor" as a privileged action.
>>
>>> It's #UD because if not intercepted it doesn't make sense to execute
>>> it.
>>
>> I agree with this, if privilege isn't considered an issue.  If a
>> hypervisor doesn't actually get involved, the instruction shouldn't
>> complete.
>>
>>> But either way, I think AMD should clarify this. Suravee, can you
>>> find out what the expected behavior is?
>>
>> IMO, it should either consistently #GP and break to the vm86 monitor, or
>> #UD/#VMEXIT depending on whether it is intercepted by the hypervisor.
>> Either way the documentation should be clarified.
>>
>> Having said this, given that its behaviour is consistent on any AMD
>> processor I choose to try, and given that vm86 mode is very legacy these
>> days, I doubt a reasonable argument can be made to fixing the behaviour.
>>
>> ~Andrew
>>
>

Appologies for the delays replying.  To answer your questions out of
order...

> What processors are you using to reproduce this behavior?

I am still out of the office until tomorrow, so haven't been able to do
more thorough testing.

I am observing expected behaviour on:

(XEN) CPU Vendor: AMD, Family 16 (0x10), Model 2 (0x2), Stepping 3 (raw
00100f23)

And observing incorrect behaviour on:

(XEN) CPU Vendor: AMD, Family 21 (0x15), Model 96 (0x60), Stepping 1
(raw 00660f01)

I note from your AVIC series that this machine should be capable, but it
it is an SDP running with microcode version 0x600610b which (now I think
about it) could easily be the source of this issue.

> So, just to confirm your observation. When executing VMMCALL in vm86
> mode with the hypervisor is set to intercept VMMCALL and not-intercept
> VMMCALL, you are seeing #GP in both cases or just in the later?

Seeing #GP when not intercepted, and a something resembling a nop when
intercepted.

> According to the HW folks, in the vm86 mode, we should observe the
> same behavior as in protected mode. (e.g. #UD in guest if not
> intercept VMMCALL, and #vmexit if intercept)

Having checked on the one other processor I have easy external access
to, this does indeed appear to be the case for Fam 0x10 processors.

> Could you please describe how you are reproducing this behavior?

I have a short XTF test which demonstrates the issue, but it would
probably be better to rule out SDP/microcode issues first.  I can find a
BKDG for Fam 15h, Model 60h-6fh processors, but not a revision guide. 
Where can I find and get the latest microcode?

Thanks,

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: AMD VMMCALL and VM86 mode

[Andrew Cooper]

On 09/12/16 19:55, Andrew Cooper wrote:
> On 09/12/16 19:55, Boris Ostrovsky wrote:
>> On 12/09/2016 02:01 PM, Andrew Cooper wrote:
>>> Hello,
>>>
>>> While working on XSA-192, I found a curious thing.  On AMD hardware, the
>>> VMMCALL instruction appears to behave like a nop if executed in VM86
>>> mode.  All other processor modes work fine.
>>>
>>> The documentation suggests it should be valid in any situation, but I
>>> never get a #VMEXIT from it. 
>> And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we have in
>> Xen by default)?
> Yes, because I have already used hypercalls to get text to the console
> before entering vm86 mode.
>
>> What happens if you don't set it?
> Let me do some hacking and see.

Outside of vm86 mode, VMMCALL raises #UD, which is expected as it wasn't
intercepted.

From within vm86 mode, I now get #GP rather than #UD.

There is certainly an argument to be made that VMMCALL inside vm86 mode
should trap to the vm86 monitor and #GP would be the expected way of
that happening, but this also doesn't match the documentation.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: AMD VMMCALL and VM86 mode

[Boris Ostrovsky]

On 12/09/2016 02:01 PM, Andrew Cooper wrote:
> Hello,
>
> While working on XSA-192, I found a curious thing.  On AMD hardware, the
> VMMCALL instruction appears to behave like a nop if executed in VM86
> mode.  All other processor modes work fine.
>
> The documentation suggests it should be valid in any situation, but I
> never get a #VMEXIT from it. 

And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we have in
Xen by default)?

What happens if you don't set it?

-boris


>  Thus, I would have thought it would fall
> into the un-intercepted category and raise a #UD fault, but I don't get
> that either.
>
> Is this behaviour expected?  The documentation would certainly seem to
> indicate not.




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: AMD VMMCALL and VM86 mode

[Andrew Cooper]

On 09/12/16 19:55, Boris Ostrovsky wrote:
> On 12/09/2016 02:01 PM, Andrew Cooper wrote:
>> Hello,
>>
>> While working on XSA-192, I found a curious thing.  On AMD hardware, the
>> VMMCALL instruction appears to behave like a nop if executed in VM86
>> mode.  All other processor modes work fine.
>>
>> The documentation suggests it should be valid in any situation, but I
>> never get a #VMEXIT from it. 
> And I assume GENERAL2_INTERCEPT_VMMCALL is set (which is what we have in
> Xen by default)?

Yes, because I have already used hypercalls to get text to the console
before entering vm86 mode.

> What happens if you don't set it?

Let me do some hacking and see.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

AMD VMMCALL and VM86 mode

[Andrew Cooper]

Hello,

While working on XSA-192, I found a curious thing.  On AMD hardware, the
VMMCALL instruction appears to behave like a nop if executed in VM86
mode.  All other processor modes work fine.

The documentation suggests it should be valid in any situation, but I
never get a #VMEXIT from it.  Thus, I would have thought it would fall
into the un-intercepted category and raise a #UD fault, but I don't get
that either.

Is this behaviour expected?  The documentation would certainly seem to
indicate not.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel