New heap API and scrubbing

New heap API and scrubbing

[Dan Magenheimer]

So in the new (soon-to-be 3.4) heap API, are

alloc_domheap_pages(NULL,order,flags)

and

alloc_xenheap_pages(order,flags)

synonomous on 64-bit hypervisors?  I.e. they
are allocating from the same pool of pages?
And if so are the matching free routines
synonomous?

If not, under what circumstances should one be
used rather than the other?

Are there any cases now where free_XXXheap_pages
might free up pages that could be grabbed by
another domain and those pages have not been
scrubbed?

Thanks,
Dan

Re: New heap API and scrubbing

[Keir Fraser]

They allocate from the same pool of pages, but they differ very slightly. If
you alloc_fooheap_pages() you must free_fooheap_pages(). You could probably
make the distinction based on is_xenheap_page() if you wanted.

 -- Keir

On 10/02/2009 00:24, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:

> So in the new (soon-to-be 3.4) heap API, are
> 
> alloc_domheap_pages(NULL,order,flags)
> 
> and
> 
> alloc_xenheap_pages(order,flags)
> 
> synonomous on 64-bit hypervisors?  I.e. they
> are allocating from the same pool of pages?
> And if so are the matching free routines
> synonomous?
> 
> If not, under what circumstances should one be
> used rather than the other?
> 
> Are there any cases now where free_XXXheap_pages
> might free up pages that could be grabbed by
> another domain and those pages have not been
> scrubbed?
> 
> Thanks,
> Dan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: New heap API and scrubbing

[Jan Beulich]

>>> Dan Magenheimer <dan.magenheimer@oracle.com> 10.02.09 01:24 >>>
>So in the new (soon-to-be 3.4) heap API, are
>
>alloc_domheap_pages(NULL,order,flags)
>
>and
>
>alloc_xenheap_pages(order,flags)
>
>synonomous on 64-bit hypervisors?  I.e. they
>are allocating from the same pool of pages?

I thought thee was a small difference, but in looking at the code it seems
there isn't at present. The difference I though would be there was that
alloc_xenheap_pages() should be allowed to access MEMZONE_XEN,
while alloc_domheap_pages() should only start allocating at MEMZONE_XEN+1.
That way, by default Xen heap allocations would eat from the domain heap,
but if that's empty would be allowed to fall back to the low memory portion.
Keir?

>And if so are the matching free routines
>synonomous?

No, because PGC_xen_heap must be cleared on pages allocated through
alloc_xenheap_pages().

Jan

Re: New heap API and scrubbing

[Keir Fraser]

On 10/02/2009 08:22, "Jan Beulich" <jbeulich@novell.com> wrote:

>> synonomous on 64-bit hypervisors?  I.e. they
>> are allocating from the same pool of pages?
> 
> I thought thee was a small difference, but in looking at the code it seems
> there isn't at present. The difference I though would be there was that
> alloc_xenheap_pages() should be allowed to access MEMZONE_XEN,
> while alloc_domheap_pages() should only start allocating at MEMZONE_XEN+1.
> That way, by default Xen heap allocations would eat from the domain heap,
> but if that's empty would be allowed to fall back to the low memory portion.
> Keir?

MEMZONE_XEN is empty on ia64 and x86_64. I was considering killing it off
completely, but I haven't really thought too hard about alternatives for
i386 so I think I'll just leave it for now, vestigial for all but i386.

I hadn't considered the possibility of it having use as a last-chance
emergency pool. I'm not really sure about the value of that to be honest. I
don't think I'd bother.

 -- Keir

RE: New heap API and scrubbing

[Dan Magenheimer]

> Are there any cases now where free_XXXheap_pages
> might free up pages that could be grabbed by
> another domain and those pages have not been
> scrubbed?

No replies on this part so following up to myself...

For tmem, I'm trying to determine under what circumstances
pages free'd to xenheap or domheap must be scrubbed.
Ideally, I'd like to free directly to the scrub list
so the standard page_scrub_timer mechanism will
scrub them.  I looked but the mechanism doesn't
appear to be easily accessible.

Moreover, it appears that there are MANY calls throughout
Xen to free_XXXheap_page/s() but I don't see much code
that scrubs the pages before freeing them.  Isn't
this a potential security issue?  Perhaps it should
be easier to free+scrub pages?

I'm thinking that free_XXXheap_pages should have
a parameter (or a sister function) that results
in freeing but also putting the pages on the
scrub list.  Something like:

void free_domheap_pages_scrub(x,y,scrub)
{
// existing free_domheap_pages code with
// a few changes to handle scrub param
}

#define free_domheap_pages(x,y) \
     free_domheap_pages_scrub(x,y,0)

(and similar for free_xenheap_pages().)

Then, over time, each call to free_XXXheap_pages
can/should be examined to see whether it should
scrub or not.

Comments?  Any thoughts on how to approach
this problem differently?

Also, I am maintaining a list of pages (using the
new page_list mechanism) that (in some cases) will
need to be "free+scrub".  So I'd like to be able
to pass an entire list to the scrub
list, rather than remove each page from one list
(in tmem) and insert it into the scrub list.
Essentially a list_splice (from list.h).
Is this feasible/reasonable?

Dan

Re: New heap API and scrubbing

[Keir Fraser]

On 10/02/2009 22:53, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:

> Moreover, it appears that there are MANY calls throughout
> Xen to free_XXXheap_page/s() but I don't see much code
> that scrubs the pages before freeing them.  Isn't
> this a potential security issue?  Perhaps it should
> be easier to free+scrub pages?

Pages which are currently not scrubbed are either:
 1. Freed by a domain before it dies, so it has to scrub them.
 2. Xenheap pages or anonymous domheap pages which thus contained no guest
data and no security risk in not scrubbing them.

Feel free to add a free+scrub function.

 -- Keir

RE: New heap API and scrubbing

[Dan Magenheimer]

> > Moreover, it appears that there are MANY calls throughout
> > Xen to free_XXXheap_page/s() but I don't see much code
> > that scrubs the pages before freeing them.  Isn't
> > this a potential security issue?  Perhaps it should
> > be easier to free+scrub pages?
> 
> Pages which are currently not scrubbed are either:
>  1. Freed by a domain before it dies, so it has to scrub them.
>  2. Xenheap pages or anonymous domheap pages which thus 
> contained no guest
> data and no security risk in not scrubbing them.

I realize that's true of "data" pages.  I'm no security
expert, but I think I'm referring to "sideband" attacks.
I.e. if an attacker gets enough "non-data" pages (such
as page-table pages) from another domain, there is eventually
sufficient information to derive something useful.
The security guys get into a tizzy about such things.
 
> Feel free to add a free+scrub function.

OK.  Since tmem has true "data" pages to free, I will
do that.

Thanks,
Dan