Re: Reducing or removing direct map from xen (was Re: Ongoing/future speculative mitigation work)

From: Wei Liu <wei.liu2@citrix.com>
To: Xen-devel <xen-devel@lists.xenproject.org>
Cc: "Martin Pohlack" <mpohlack@amazon.de>,
	"Julien Grall" <julien.grall@arm.com>,
	"Jan Beulich" <JBeulich@suse.com>,
	"Joao Martins" <joao.m.martins@oracle.com>,
	"Stefano Stabellini" <sstabellini@kernel.org>,
	"Daniel Kiper" <daniel.kiper@oracle.com>,
	"Marek Marczykowski" <marmarek@invisiblethingslab.com>,
	"Anthony Liguori" <aliguori@amazon.com>,
	"Dannowski, Uwe" <uwed@amazon.de>,
	"Lars Kurth" <lars.kurth@citrix.com>,
	"Konrad Wilk" <konrad.wilk@oracle.com>,
	"Ross Philipson" <ross.philipson@oracle.com>,
	"Dario Faggioli" <dfaggioli@suse.com>,
	"Matt Wilson" <msw@amazon.com>,
	"Boris Ostrovsky" <boris.ostrovsky@oracle.com>,
	"Juergen Gross" <JGross@suse.com>,
	"Sergey Dyasli" <sergey.dyasli@citrix.com>,
	"Wei Liu" <wei.liu2@citrix.com>,
	"George Dunlap" <george.dunlap@eu.citrix.com>,
	"Andrew Cooper" <andrew.cooper3@citrix.com>,
	"Mihai Donțu" <mdontu@bitdefender.com>,
	"Woodhouse, David" <dwmw@amazon.co.uk>
Subject: Re: Reducing or removing direct map from xen (was Re: Ongoing/future speculative mitigation work)
Date: Wed, 20 Feb 2019 12:29:01 +0000	[thread overview]
Message-ID: <20190220122901.6kvc7ibyjo4uyzke@zion.uk.xensource.com> (raw)
In-Reply-To: <20190124114455.eyke5dvkqsxhklav@zion.uk.xensource.com>

On Thu, Jan 24, 2019 at 11:44:55AM +0000, Wei Liu wrote:
> 3. Implement xenheap using vmap infrastructure
> 
> This helps preserve xenheap's "always mapped" property. At the moment,
> vmap relies on xenheap, we want to turn this relationship around.
> 
> There is a loop what needs breaking in the new world:
> 
>   alloc_xenheap_pages -> vmap -> __vmap -> map_pages_to_xen ->
>     virt_to_xen_l1e -> alloc_xen_pagetable -> alloc_xenheap_page -> vmap ...
> 
> Two options were proposed to break this loop:
> 
>   3.1 Pre-populate all page tables for vmap region

Now that we have this ...

>   3.2 Switch page table allocation to use domheap page
> 
> 
> The other work item is to track page<->virt relationship so that
> conversion functions (_to_virt etc) continue to work. For PoC purpose,
> putting a void * into page_info is good enough. But in the future we
> want to have a separate array for tracking so that page_info stays power
> of two in size.
> 

I started working on some prototyping code for the rest of this major
work item. Conversion functions are a bit messy to deal with (I have no
idea whether my modifications are totally correct at this point), but
the most major issue I see is an optimisation done by xmalloc which
isn't compatible with vmap.

So xmalloc has this optimisation: it will allocate a high-order page
from xenheap when necessary and then attempt to break that up and return
the unused portion.  Vmap uses bitmap to track address space usage, and
it mandates a guard page before every address space allocation. What
xmalloc does is to free a portion of the address space, which isn't
really supported by vmap.

I came up with two options yesterday:

1. Remove the optimisation in xmalloc
2. Make vmap able to break up allocation

Neither looks great to me. The first is simple but potentially wasteful
(how much is wasted?). The second requires non-trivial modification to
vmap, essentially removing the mandatory guard page. In comparison the
first is easier and safer.

I would like to hear people's thought on this. Comments are welcome.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel