Altp2m/#VE page issue

Altp2m/#VE page issue

[Razvan Cojocaru]

Hello,

We're trying to get #VE to work with a "regular" guest page (that is,
not a page that we get via xc_domain_increase_reservation_exact() /
xc_domain_populate_physmap_exact()).

However, the way Xen's code works now, it doesn't seem to be possible:

http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/x86/hvm/vmx/vmx.c;h=20a8a69fbe412aa928c75b5d7756816bf55178fc;hb=refs/heads/staging#l2150

because get_gfn_query_unlocked() calls ept_get_entry(), which returns
INVALID_MFN if gfn > p2m->max_mapped_pfn.

max_mapped_pfn is only ever updated in ept_set_entry(), and so unless
ept_set_entry() gets called previously to vmx_vcpu_update_vmfunc_ve(),
we will fail to enable #VE for any given (valid) GFN.

I believe that this works with the
xc_domain_increase_reservation_exact() /
xc_domain_populate_physmap_exact() strategy because somewhere at the end
of the callchain, ept_set_entry() _does_ get called for the new GFN.

Forcing max_mapped_pfn should work, but I can't help thinking that
there's a better way. Maybe George and Tamas have a suggestion here?


Thanks,
Razvan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Altp2m/#VE page issue

[Tamas K Lengyel]

On Fri, Jun 29, 2018 at 9:25 AM Razvan Cojocaru
<rcojocaru@bitdefender.com> wrote:
>
> Hello,
>
> We're trying to get #VE to work with a "regular" guest page (that is,
> not a page that we get via xc_domain_increase_reservation_exact() /
> xc_domain_populate_physmap_exact()).
>
> However, the way Xen's code works now, it doesn't seem to be possible:
>
> http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/x86/hvm/vmx/vmx.c;h=20a8a69fbe412aa928c75b5d7756816bf55178fc;hb=refs/heads/staging#l2150
>
> because get_gfn_query_unlocked() calls ept_get_entry(), which returns
> INVALID_MFN if gfn > p2m->max_mapped_pfn.

It's certainly weird to have a "regular" gfn that's also above
max_mapped_pfn. That to me sounds like a bug with how max_mapped_pfn
is set. To be honest, I've ran into issues with max_mapped_gfn in the
past too and it was annoying so I do actually end up just running:

rc = xc_domain_setmaxmem(drakvuf->xen->xc, drakvuf->domID, ~0);

to make it go away. It's fugly but didn't have time yet to investigate
further ¯\_(ツ)_/¯. Maybe it's time for a proper fix.

Tamas

Tamas

>
> max_mapped_pfn is only ever updated in ept_set_entry(), and so unless
> ept_set_entry() gets called previously to vmx_vcpu_update_vmfunc_ve(),
> we will fail to enable #VE for any given (valid) GFN.
>
> I believe that this works with the
> xc_domain_increase_reservation_exact() /
> xc_domain_populate_physmap_exact() strategy because somewhere at the end
> of the callchain, ept_set_entry() _does_ get called for the new GFN.
>
> Forcing max_mapped_pfn should work, but I can't help thinking that
> there's a better way. Maybe George and Tamas have a suggestion here?
>
>
> Thanks,
> Razvan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Altp2m/#VE page issue

[Razvan Cojocaru]

On 06/29/2018 07:52 PM, Tamas K Lengyel wrote:
> On Fri, Jun 29, 2018 at 9:25 AM Razvan Cojocaru
> <rcojocaru@bitdefender.com> wrote:
>>
>> Hello,
>>
>> We're trying to get #VE to work with a "regular" guest page (that is,
>> not a page that we get via xc_domain_increase_reservation_exact() /
>> xc_domain_populate_physmap_exact()).
>>
>> However, the way Xen's code works now, it doesn't seem to be possible:
>>
>> http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/x86/hvm/vmx/vmx.c;h=20a8a69fbe412aa928c75b5d7756816bf55178fc;hb=refs/heads/staging#l2150
>>
>> because get_gfn_query_unlocked() calls ept_get_entry(), which returns
>> INVALID_MFN if gfn > p2m->max_mapped_pfn.
> 
> It's certainly weird to have a "regular" gfn that's also above
> max_mapped_pfn. That to me sounds like a bug with how max_mapped_pfn
> is set. To be honest, I've ran into issues with max_mapped_gfn in the
> past too and it was annoying so I do actually end up just running:
> 
> rc = xc_domain_setmaxmem(drakvuf->xen->xc, drakvuf->domID, ~0);
> 
> to make it go away. It's fugly but didn't have time yet to investigate
> further ¯\_(ツ)_/¯. Maybe it's time for a proper fix.

Those are two problems then, I've just tried that and it doesn't fix
mine. I've logged all ept_set_entry()es, and here are the ones before
the enable_notify() call:

(XEN) [17928.632914] ept_set_entry(): GFN 0x22c8, mfn: 0xb126c8
(XEN) [17928.633175] ept_set_entry(): GFN 0x22ce, mfn: 0xb126ce
(XEN) [17928.634519] ept_set_entry(): GFN 0x10687d, mfn: 0xa2807d
(XEN) [17928.635939] ept_set_entry(): GFN 0x106745, mfn: 0xa28345
(XEN) [17928.636240] ept_set_entry(): GFN 0x234c, mfn: 0xb1274c
(XEN) [17928.636459] ept_set_entry(): GFN 0x1066c2, mfn: 0xa282c2
(XEN) [17928.637163] ept_set_entry(): GFN 0x1066d7, mfn: 0xa282d7
(XEN) [17928.637400] ept_set_entry(): GFN 0x106758, mfn: 0xa28358
(XEN) [17928.639576] ept_set_entry(): GFN 0x1067d9, mfn: 0xa283d9
(XEN) [17928.641908] ept_set_entry(): GFN 0x22b8, mfn: 0xb126b8
(XEN) [17928.642936] ept_set_entry(): GFN 0x2078, mfn: 0xb12878
(XEN) [17928.672682] HVMOP_altp2m_vcpu_enable_notify domain: 5, vcpu: 0,
gfn: 0x1c67000
(XEN) [17928.672684] calling altp2m_vcpu_update_vmfunc_ve()
(XEN) [17928.672690] gfn > p2m->max_mapped_pfn (goto out)
(XEN) [17928.672691] mfn_x(mfn) == INVALID_MFN

I'm just trying to set the #VE page to 0x1c67000 on a 4GB domain, and
it's less than max_mapped_pfn (at least at the time the application is
making the call).


Thanks,
Razvan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Altp2m/#VE page issue

[Razvan Cojocaru]

On 06/29/2018 10:29 PM, Razvan Cojocaru wrote:
> On 06/29/2018 07:52 PM, Tamas K Lengyel wrote:
>> On Fri, Jun 29, 2018 at 9:25 AM Razvan Cojocaru
>> <rcojocaru@bitdefender.com> wrote:
>>>
>>> Hello,
>>>
>>> We're trying to get #VE to work with a "regular" guest page (that is,
>>> not a page that we get via xc_domain_increase_reservation_exact() /
>>> xc_domain_populate_physmap_exact()).
>>>
>>> However, the way Xen's code works now, it doesn't seem to be possible:
>>>
>>> http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/x86/hvm/vmx/vmx.c;h=20a8a69fbe412aa928c75b5d7756816bf55178fc;hb=refs/heads/staging#l2150
>>>
>>> because get_gfn_query_unlocked() calls ept_get_entry(), which returns
>>> INVALID_MFN if gfn > p2m->max_mapped_pfn.
>>
>> It's certainly weird to have a "regular" gfn that's also above
>> max_mapped_pfn. That to me sounds like a bug with how max_mapped_pfn
>> is set. To be honest, I've ran into issues with max_mapped_gfn in the
>> past too and it was annoying so I do actually end up just running:
>>
>> rc = xc_domain_setmaxmem(drakvuf->xen->xc, drakvuf->domID, ~0);
>>
>> to make it go away. It's fugly but didn't have time yet to investigate
>> further ¯\_(ツ)_/¯. Maybe it's time for a proper fix.
> 
> Those are two problems then, I've just tried that and it doesn't fix
> mine. I've logged all ept_set_entry()es, and here are the ones before
> the enable_notify() call:
> 
> (XEN) [17928.632914] ept_set_entry(): GFN 0x22c8, mfn: 0xb126c8
> (XEN) [17928.633175] ept_set_entry(): GFN 0x22ce, mfn: 0xb126ce
> (XEN) [17928.634519] ept_set_entry(): GFN 0x10687d, mfn: 0xa2807d
> (XEN) [17928.635939] ept_set_entry(): GFN 0x106745, mfn: 0xa28345
> (XEN) [17928.636240] ept_set_entry(): GFN 0x234c, mfn: 0xb1274c
> (XEN) [17928.636459] ept_set_entry(): GFN 0x1066c2, mfn: 0xa282c2
> (XEN) [17928.637163] ept_set_entry(): GFN 0x1066d7, mfn: 0xa282d7
> (XEN) [17928.637400] ept_set_entry(): GFN 0x106758, mfn: 0xa28358
> (XEN) [17928.639576] ept_set_entry(): GFN 0x1067d9, mfn: 0xa283d9
> (XEN) [17928.641908] ept_set_entry(): GFN 0x22b8, mfn: 0xb126b8
> (XEN) [17928.642936] ept_set_entry(): GFN 0x2078, mfn: 0xb12878
> (XEN) [17928.672682] HVMOP_altp2m_vcpu_enable_notify domain: 5, vcpu: 0,
> gfn: 0x1c67000
> (XEN) [17928.672684] calling altp2m_vcpu_update_vmfunc_ve()
> (XEN) [17928.672690] gfn > p2m->max_mapped_pfn (goto out)
> (XEN) [17928.672691] mfn_x(mfn) == INVALID_MFN
> 
> I'm just trying to set the #VE page to 0x1c67000 on a 4GB domain, and
> it's less than max_mapped_pfn (at least at the time the application is
> making the call).

And that's clearly a GPA not a GFN, I needed to convert them somewhere
in the code...

It's been a long day, sorry for the noise.


Thanks,
Razvan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel