Re: [Xen-devel] [PATCH 2/2] xen/mm: Introduce PGC_state_uninitialised

[Julien Grall <julien@xen.org>]

Hi David,

On 24/03/2020 17:55, David Woodhouse wrote:
> On Tue, 2020-03-24 at 10:08 +0000, Julien Grall wrote:
>> Hi David,
>>
>> On 23/03/2020 10:55, David Woodhouse wrote:
>>> On Mon, 2020-03-23 at 09:34 +0000, Julien Grall wrote:
>>>> For liveupdate, we will need a way to initialize a page but mark it as
>>>> already inuse (i.e in the same state as they would be if allocated
>>>> normally).
>>>
>>> I am unconvinced of the veracity of this claim.
>>>
>>> We don't want to turn specific details of the current Xen buddy
>>> allocator part into of the implicit ABI of live update. That goes for
>>> the power-of-two zone boundaries, amongst other things.
>>
>> Why would you to do that? Marking the page as already used is no
>> different to "PGC_state_unitialized" except the "struct page_info" and
>> the internal of the buddy allocator would be properly setup for start
>> rather than at free.
> 
> The internals of the buddy allocator *cannot* be set up properly for a
> page which it would not actually give out — like the zero page.
> 
> We *could* do some tricks to allocate the zone structures for zones
> which *do* exist but contain only "pre-allocated" pages so the buddy
> allocator has never seen those zones... yet.
> 
> 
>> I think using "PGC_state_unitialised" for preserved page is an abuse. I
>> understand this is existing in other part of Xen (particularly on x86),
>> but I would rather not try to add more.
> 
> 
> I am perfectly happy to try avoiding PGC_state_uninitialised for pages
> which are "in use" at boot time due to live update.
> 
> All I insist on is that you explicitly describe the ABI constraints
> that it imposes, if any.

Agreed.

> 
> For example, that hack which stops the buddy allocator from giving out
> page zero: Could we have live updated from a Xen without that hack, to
> a Xen which has it? With page zero already given out to a domain?

The buddy allocator could never have given out page 0 on x86 because it 
is part of the first MB of the RAM (see arch_init_memory() in 
arch/x86/mm.c). AFAIK, the first MB cannot be freed..

The change in the buddy allocator was to address the Arm side and also 
make clear this was a problem this is a weakness of the allocator.

> What's yours? How would we cope with a situation like that, over LU?

When do you expect the pages to be carved out from the buddy allocator?

I can see only two situations:
	1) Workaround a bug in the allocator.
         2) A CPU errata requiring to not use memory.

In the case of 1), it is still safe for a domain to run with that page. 
However, we don't want to give it back to the page allocator. A solution 
is to mark them as "offlining/broken". Alternatively, you could remove 
the swap the page (see more below).

In the case of 2), it is not safe for a domain to run with that page. So 
it is probably best to swap the pages with a new one. For HVM domain 
(including the P2M), it should be easy. For PV domain, I am not entirely 
sure if that's feasible.

Cheers,

-- 
Julien Grall