Re: [PATCH v5 10/28] xsplice: Implement payload loading

["Jan Beulich" <JBeulich@suse.com>]

>>> On 04.04.16 at 21:44, <konrad.wilk@oracle.com> wrote:
> On Fri, Apr 01, 2016 at 03:18:45AM -0600, Jan Beulich wrote:
>> >>> On 31.03.16 at 23:26, <konrad@darnok.org> wrote:
>> >>  Also - how well will this O(n^2) lookup work once there are enough
>> >> payloads? I think this calls for the alternative vmap() extension I've
>> >> been suggesting earlier.
>> > 
>> > Could you elaborate on the vmap extension a bit please?
>> > 
>> > Your earlier email seems to say: drop the vmap API and just 
>> > allocate the underlaying pages yourself.
>> 
>> Actually I had also said in that earlier mail: "If, otoh, you left that
>> VA management to (an extended version of) vmap(), by e.g.
>> allowing the caller to request allocation from a different VA range
>> (much like iirc x86-64 Linux handles its modules address range
>> allocation), things would be different. After all the VA
>> management is the important part here, while the backing
>> memory allocation is just a trivial auxiliary operation."
>> 
>> I.e. elaboration here really just consists of the referral to the
>> respective Linux approach.
> 
> I am in need here of guidance I am afraid.
> 
> Let me explain (did this in IRC but this will have a broader scope):
> 
> In Linux we have the 'struct vm_area' which internally contains the start
> and end address (amongst other things). The callers usually use 
> __vmalloc_node_range
> to an provide those addresses. Internally the vmalloc API allocates the
> 'struct vm_area' from the normal SLAB allocator. Vmalloc API also has an
> vmap block area (allocated within vmalloc area) which is a red and black 
> tree
> for all the users of its API. When vm_size() is called this tree is searched
> to find the 'vm_area' for the provided virtual address. There is a lot
> of code in this. Copying it and jamming does not look easy so it would be
> better to take concepts of this an implement this.. 
> 
> 
> On Xen we setup a bitmap that covers the full vmalloc area (128MB on my
> 4GB box, but can go up to 64GB) - the 128MB vmalloc area requires about
> 32K bits.
> 
> For every allocation  we "waste" an page (and a bit) so that there is a gap.
> This gap is needed when trying to to determine the size of the allocated
> region - when scanning the bitmap we can easily figure out the cleared
> bit which is akin to a fencepost.
> 
> 
> To make Xen's vmalloc API be generic I need to wholesale make it able
> to deal with virtual addresses that are not part of its space (as in
> not in VMAP_VIRT_START to vm_top). At the start I the input to vm_size()
> needs to get the size of the virtual address (either the ones from
> the vmalloc areas or the ones provided earlier by vmalloc_cb).
> 
> One easy mechanism is to embedded an array of simplified 'struct vm_area' 
> structure:
> 
> struct vm_area {
> 	unsigned long va;
> }
> 
> for every slot in the VMAP_VIRT_START area (that is have 32K entries).
> The vm_size and all the rest can check for this array if the virtual
> address provided is not within the vmalloc virtual addresses. If there
> is a match we just need to consult the vm_bitmap at the same index and
> figure out where the empty bit is set.
> The downside is that I've to walk the full array (32K entries).
> 
> But when you think about it - most of the time we use normal vmalloc 
> addresses
> and only in exceptional cases do we need the alternate ones. And the only 
> reason
> to keep track of it is to know the size.
> 
> The easier way would be to track them via a linked list:
> 
> struct vm_area {
> 	struct list_head list;
> 	unsigned long va;
> 	size_t nr;
> }
> 
> And vm_size, vm_index, etc would consult this list for the virtual address 
> and
> could get the proper information. (See inline patch)
> 
> But if we are doing that this, then why even put it in the vmalloc API? Why 
> not
> track all of this with the user of this? (like it was done in v4 of this 
> patch series?)
> 
> Please advise.

Well, I certainly didn't think of it getting done that way. To me the
most natural generalization would be for an arch to register one or
more secondary ranges (which could even get referred to by an
enum) at boot time (or maybe that could even be arranged for at
compile time, i.e. no active registration necessary), with each such
area getting the same data structures set up as is being done right
now for the "base" VA range.

But if that's too cumbersome for now, I'm certainly fine with
xSplice code dealing with the VA management itself. We can
always add such an extension later (and then make xSplice use it).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel