Re: [PATCH 08/10] xen/arm: introduce reserved_page_list

From: Julien Grall <julien@xen.org>
To: Penny Zheng <penny.zheng@arm.com>,
	xen-devel@lists.xenproject.org, sstabellini@kernel.org
Cc: Bertrand.Marquis@arm.com, Wei.Chen@arm.com, nd@arm.com
Subject: Re: [PATCH 08/10] xen/arm: introduce reserved_page_list
Date: Tue, 18 May 2021 12:02:06 +0100	[thread overview]
Message-ID: <c002d9b2-8210-1c03-b374-76e037b65e2f@xen.org> (raw)
In-Reply-To: <20210518052113.725808-9-penny.zheng@arm.com>

Hi Penny,

On 18/05/2021 06:21, Penny Zheng wrote:
> Since page_list under struct domain refers to linked pages as gueast RAM

s/gueast/guest/

> allocated from heap, it should not include reserved pages of static memory.

You already have PGC_reserved to indicate they are "static memory". So 
why do you need yet another list?

> 
> The number of PGC_reserved pages assigned to a domain is tracked in
> a new 'reserved_pages' counter. Also introduce a new reserved_page_list
> to link pages of static memory. Let page_to_list return reserved_page_list,
> when flag is PGC_reserved.
> 
> Later, when domain get destroyed or restarted, those new values will help
> relinquish memory to proper place, not been given back to heap.
> 
> Signed-off-by: Penny Zheng <penny.zheng@arm.com>
> ---
>   xen/common/domain.c     | 1 +
>   xen/common/page_alloc.c | 7 +++++--
>   xen/include/xen/sched.h | 5 +++++
>   3 files changed, 11 insertions(+), 2 deletions(-)
> 
> diff --git a/xen/common/domain.c b/xen/common/domain.c
> index 6b71c6d6a9..c38afd2969 100644
> --- a/xen/common/domain.c
> +++ b/xen/common/domain.c
> @@ -578,6 +578,7 @@ struct domain *domain_create(domid_t domid,
>       INIT_PAGE_LIST_HEAD(&d->page_list);
>       INIT_PAGE_LIST_HEAD(&d->extra_page_list);
>       INIT_PAGE_LIST_HEAD(&d->xenpage_list);
> +    INIT_PAGE_LIST_HEAD(&d->reserved_page_list);
>   
>       spin_lock_init(&d->node_affinity_lock);
>       d->node_affinity = NODE_MASK_ALL;
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index f1f1296a61..e3f07ec6c5 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -2410,7 +2410,7 @@ int assign_pages(
>   
>           for ( i = 0; i < nr_pfns; i++ )
>           {
> -            ASSERT(!(pg[i].count_info & ~PGC_extra));
> +            ASSERT(!(pg[i].count_info & ~(PGC_extra | PGC_reserved)));
I think this change belongs to the previous patch.

>               if ( pg[i].count_info & PGC_extra )
>                   extra_pages++;
>           }
> @@ -2439,6 +2439,9 @@ int assign_pages(
>           }
>       }
>   
> +    if ( pg[0].count_info & PGC_reserved )
> +        d->reserved_pages += nr_pfns;

reserved_pages doesn't seem to be ever read or decremented. So why do 
you need it?

> +
>       if ( !(memflags & MEMF_no_refcount) &&
>            unlikely(domain_adjust_tot_pages(d, nr_pfns) == nr_pfns) )
>           get_knownalive_domain(d);
> @@ -2452,7 +2455,7 @@ int assign_pages(
>               page_set_reserved_owner(&pg[i], d);
>           smp_wmb(); /* Domain pointer must be visible before updating refcnt. */
>           pg[i].count_info =
> -            (pg[i].count_info & PGC_extra) | PGC_allocated | 1;
> +            (pg[i].count_info & (PGC_extra | PGC_reserved)) | PGC_allocated | 1;

Same here.

>           page_list_add_tail(&pg[i], page_to_list(d, &pg[i]));
>       }
>   
> diff --git a/xen/include/xen/sched.h b/xen/include/xen/sched.h
> index 3982167144..b6333ed8bb 100644
> --- a/xen/include/xen/sched.h
> +++ b/xen/include/xen/sched.h
> @@ -368,6 +368,7 @@ struct domain
>       struct page_list_head page_list;  /* linked list */
>       struct page_list_head extra_page_list; /* linked list (size extra_pages) */
>       struct page_list_head xenpage_list; /* linked list (size xenheap_pages) */
> +    struct page_list_head reserved_page_list; /* linked list (size reserved pages) */
>   
>       /*
>        * This field should only be directly accessed by domain_adjust_tot_pages()
> @@ -379,6 +380,7 @@ struct domain
>       unsigned int     outstanding_pages; /* pages claimed but not possessed */
>       unsigned int     max_pages;         /* maximum value for domain_tot_pages() */
>       unsigned int     extra_pages;       /* pages not included in domain_tot_pages() */
> +    unsigned int     reserved_pages;    /* pages of static memory */
>       atomic_t         shr_pages;         /* shared pages */
>       atomic_t         paged_pages;       /* paged-out pages */
>   
> @@ -588,6 +590,9 @@ static inline struct page_list_head *page_to_list(
>       if ( pg->count_info & PGC_extra )
>           return &d->extra_page_list;
>   
> +    if ( pg->count_info & PGC_reserved )
> +        return &d->reserved_page_list;
> +
>       return &d->page_list;
>   }
>   
> 

Cheers,

-- 
Julien Grall