RE: Regarding page table management changes from Xen v1to Xen v2 (and v3)

RE: Regarding page table management changes from Xen v1to Xen v2 (and v3)

[Ian Pratt]

> In another example, mmapped access to a file, the new 
> approach will incur these two extra faults for every page 
> fault required to bring a file page to page cache, right?

No, these singleton updates on the Linux demand fault path are issued
with the update_va_mapping hypercall. This is possible because the
virtual address that the pte maps is known, so its possible for Xen to
use a linear pagetable to do the update rather than creating an
on-demand mapping to the domain's pagetable page. This is an important
optimization on 32b where Xen address space is tight.

Ian

Re: Regarding page table management changes from Xen v1to Xen v2 (and v3)

[Keir Fraser]

On 26 Apr 2006, at 19:15, Anthony Liguori wrote:

>> No, these singleton updates on the Linux demand fault path are issued
>> with the update_va_mapping hypercall. This is possible because the
>> virtual address that the pte maps is known, so its possible for Xen to
>> use a linear pagetable to do the update rather than creating an
>> on-demand mapping to the domain's pagetable page. This is an important
>> optimization on 32b where Xen address space is tight.
>>
>
> Just to make sure I understand, we do the linear mapping for 
> performance (so we don't have to keep remapping) but not necessarily 
> to save VA space since the linear mapping uses a fixed amount of 
> space.
>
> Is there any reason that we would strictly require a linear mapping 
> (other than it being convenient)?

On 64b we could use the 1:1 mapping of all physmem that Xen has. 
Obviously that is infeasible on 32b. In fact the interface (passing a 
virtual address) pretty much means we have to use a linear mapping in 
both cases.

  -- Keir

Re: Regarding page table management changes from Xen v1to Xen v2 (and v3)

[Anthony Liguori]

Ian Pratt wrote:
>> In another example, mmapped access to a file, the new 
>> approach will incur these two extra faults for every page 
>> fault required to bring a file page to page cache, right?
>>     
>
> No, these singleton updates on the Linux demand fault path are issued
> with the update_va_mapping hypercall. This is possible because the
> virtual address that the pte maps is known, so its possible for Xen to
> use a linear pagetable to do the update rather than creating an
> on-demand mapping to the domain's pagetable page. This is an important
> optimization on 32b where Xen address space is tight.
>   

Just to make sure I understand, we do the linear mapping for performance 
(so we don't have to keep remapping) but not necessarily to save VA 
space since the linear mapping uses a fixed amount of space.

Is there any reason that we would strictly require a linear mapping 
(other than it being convenient)?

Thanks,

Anthony Liguori

> Ian
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   

Re: Regarding page table management changes from Xen v1to Xen v2 (and v3)

[Anthony Liguori]

Keir Fraser wrote:
>
> On 26 Apr 2006, at 19:15, Anthony Liguori wrote:
>
>>> No, these singleton updates on the Linux demand fault path are issued
>>> with the update_va_mapping hypercall. This is possible because the
>>> virtual address that the pte maps is known, so its possible for Xen to
>>> use a linear pagetable to do the update rather than creating an
>>> on-demand mapping to the domain's pagetable page. This is an important
>>> optimization on 32b where Xen address space is tight.
>>>
>>
>> Just to make sure I understand, we do the linear mapping for 
>> performance (so we don't have to keep remapping) but not necessarily 
>> to save VA space since the linear mapping uses a fixed amount of space.
>>
>> Is there any reason that we would strictly require a linear mapping 
>> (other than it being convenient)?
>
> On 64b we could use the 1:1 mapping of all physmem that Xen has. 
> Obviously that is infeasible on 32b. In fact the interface (passing a 
> virtual address) pretty much means we have to use a linear mapping in 
> both cases.

Couldn't we temporarily map the page table page?  Not that we would 
actually want to do that.  Just trying to confirm my understand.

Regards,

Anthony Liguori

>  -- Keir
>

Re: Regarding page table management changes from Xen v1to Xen v2 (and v3)

[Keir Fraser]

On 27 Apr 2006, at 00:33, Anthony Liguori wrote:

>>> Couldn't we temporarily map the page table page?  Not that we would 
>>> actually want to do that.  Just trying to confirm my understand.
>>>
>>
>> Sure, but that's expensive on 32b as map_domain_mem has to do real 
>> work
>> (it's cheap on 64b as we have a 1-to-1 map of machine memory).
>>
>
> Excellent, that's just what I thought.
>
> Thanks Ian!

Where do you get the machine address of the page to map, given a linear 
address at the hypercall interface? You'd still need a linear mapping 
access, but you'd want to read out the pde not the pte.

  -- Keir

Re: Regarding page table management changes from Xen v1to Xen v2 (and v3)

[Anthony Liguori]

Ian Pratt wrote:
>>> On 64b we could use the 1:1 mapping of all physmem that Xen has. 
>>> Obviously that is infeasible on 32b. In fact the interface 
>>>       
>> (passing a 
>>     
>>> virtual address) pretty much means we have to use a linear 
>>>       
>> mapping in 
>>     
>>> both cases.
>>>       
>> Couldn't we temporarily map the page table page?  Not that we 
>> would actually want to do that.  Just trying to confirm my understand.
>>     
>
> Sure, but that's expensive on 32b as map_domain_mem has to do real work
> (it's cheap on 64b as we have a 1-to-1 map of machine memory).
>   

Excellent, that's just what I thought.

Thanks Ian!

Regards,

Anthony Liguori

> Ian
>   

RE: Regarding page table management changes from Xen v1to Xen v2 (and v3)

[Nakajima, Jun]

Himanshu Raj wrote:
>>> Could you tell me another example other than fork when one can use
>>> batched PTE modifications.
>> 
>> fork() is the only one for us these days. All others use
>> update_va_mapping(), act on pagetables that aren't pinned (so the
>> guest can directly update them without faulting) or are infrequent
>> enough we do not care.
> Not sure I understand the last part (aren't pinned ...) - my
> assumptions about 
> page tables are (these apply to both direct mapped and shadow page
> tables): 
> 
> 1. Always pinned (backed) - so a access to them cannot cause a page
> unavailable fault. 
> 2. Always read only to guest - so a read access to them is fine, but
>    a write access will cause a protection fault.

Today we pin once per page table when the root is pinned (rather than
any of the other levels). So whatever page tables populated later, for
example, are not pinned (they have been a data page). 

> 
> Both of these faults are reflected as a PG fault.
> 
> Are you refering to the case when Xen has "detached" the page table
> page and 
> has made it RW for guest?
> 
> Thanks for your answers and patience :-).
> 
> -Himanshu


Jun
---
Intel Open Source Technology Center

RE: Regarding page table management changes from Xen v1to Xen v2 (and v3)

[Ian Pratt]

> > On 64b we could use the 1:1 mapping of all physmem that Xen has. 
> > Obviously that is infeasible on 32b. In fact the interface 
> (passing a 
> > virtual address) pretty much means we have to use a linear 
> mapping in 
> > both cases.
> 
> Couldn't we temporarily map the page table page?  Not that we 
> would actually want to do that.  Just trying to confirm my understand.

Sure, but that's expensive on 32b as map_domain_mem has to do real work
(it's cheap on 64b as we have a 1-to-1 map of machine memory).

Ian

RE: Regarding page table management changes from Xen v1to Xen v2 (and v3)

[Ian Pratt]

> Although, same results can be obtained by doing the v1.2 way, 
> i.e. making one hypercall requesting these 1024 changes, no?

No, not easily -- the batched interface has problems with SMP guests, as
the updates may be expected to be individually atomic.

Ian

> On Wed, Apr 26, 2006 at 05:18:28PM +0100, Keir Fraser wrote:
> > 
> > On 26 Apr 2006, at 16:35, Himanshu Raj wrote:
> > 
> > >I am trying to understand the rationale behind this change. In 
> > >previous case, there would be no page faults due to page table 
> > >updates and only one hypercall.
> > >In the second case, there would be atleast 2 page faults due to PT 
> > >management activity, but no hypercalls. Besides, mapping and 
> > >remapping with different permissions imply removing this 
> entry from 
> > >TLB (which is hopefully being done with invlpg).
> > >Benefit of latter approach only seems to be the removal of xen 
> > >specific linux code. Why was the approach changed? Could someone 
> > >please shed some light on this?
> > 
> > It's useful for batched updates (e.g., fork()) where the 2 
> faults are 
> > amortised across up to 1024 pte changes.
> > 
> >  -- Keir
> 
> --
> --------------------------------------------------------------
> -----------
> Himanshu Raj
> PhD Student, GaTech (www.cc.gatech.edu/~rhim) I prefer to 
> receive attachments in an open, non-proprietary format.
> --------------------------------------------------------------
> -----------
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>