out of memory issue on dom0_hvm start

out of memory issue on dom0_hvm start

[andrei.semenov]

[-- Attachment #1.1: Type: text/plain, Size: 1416 bytes --]

Hi all,

I have some troubles to start xen when using dom0 in HVM mode on X86 Intel nuc7i5bnh 
board. After some investigation it seems that  "domheap" memory allocator is out of memory while 
constructing the p2m mapping  (pvh_setup_p2m).  

This misbehavior, from my understanding, is the result of wrong computation of number of pages 
to "give" to dom0 (dom0_compute_nr_pages function). In fact the pages needed to paging 
(dom0_paging_pages function)  are not reserved in this function if  IOMMU mappings are shareable 
with HAP(EPT) mappings. Moreover this memory (for paging needs) is allocated just after that in 
"pvh_setup_p2m" in subroutine "paging_set_allocation". 

Generally speaking from what I understood it's not the IOMMU driver that offers its mappings to HAP,
but it's rather the HAP that shares its mappings with IOMMU driver. So the obvious patch IMHO would 
be:

--- a/xen/arch/x86/dom0_build.c
+++ b/xen/arch/x86/dom0_build.c
@@ -294,8 +294,7 @@ unsigned long __init dom0_compute_nr_pages(
             avail -= max_pdx >> s;
     }
 
-    need_paging = is_hvm_domain(d) &&
-        (!iommu_hap_pt_share || !paging_mode_hap(d));
+    need_paging = is_hvm_domain(d);
     for ( ; ; need_paging = false )
     {
         nr_pages = dom0_nrpages; 

Did anyone encounter the same problem or has some thoughts on this issue?

Andrei.
1

[-- Attachment #1.2: Type: text/html, Size: 6087 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: out of memory issue on dom0_hvm start

[Roger Pau Monné]

On Wed, Nov 28, 2018 at 12:25:23AM +0100, andrei.semenov@bertin.fr wrote:
> Hi all,
> 
> I have some troubles to start xen when using dom0 in HVM mode on X86 Intel nuc7i5bnh 
> board. After some investigation it seems that  "domheap" memory allocator is out of memory while 
> constructing the p2m mapping  (pvh_setup_p2m).  
> 
> This misbehavior, from my understanding, is the result of wrong computation of number of pages 
> to "give" to dom0 (dom0_compute_nr_pages function). In fact the pages needed to paging 
> (dom0_paging_pages function)  are not reserved in this function if  IOMMU mappings are shareable 
> with HAP(EPT) mappings. Moreover this memory (for paging needs) is allocated just after that in 
> "pvh_setup_p2m" in subroutine "paging_set_allocation". 

All this calculations are inherited from the remnants of the paging PV
Dom0 implementation, and are likely wrong.

For example dom0_paging_pages is calculating the amount of pages
required to run in shadow mode, but not the amount of pages that would
be required to run in HAP mode, or the amount of memory used by the
IOMMU pages tables if they are not shared with HAP.

> Generally speaking from what I understood it's not the IOMMU driver that offers its mappings to HAP,
> but it's rather the HAP that shares its mappings with IOMMU driver. So the obvious patch IMHO would 
> be:
> 
> --- a/xen/arch/x86/dom0_build.c
> +++ b/xen/arch/x86/dom0_build.c
> @@ -294,8 +294,7 @@ unsigned long __init dom0_compute_nr_pages(
>              avail -= max_pdx >> s;
>      }
>  
> -    need_paging = is_hvm_domain(d) &&
> -        (!iommu_hap_pt_share || !paging_mode_hap(d));
> +    need_paging = is_hvm_domain(d);

I think we need a better approach, one that takes the following into
account:

 - Memory used by HAP or shadow.
 - Memory used by the IOMMU page tables if not shared with HAP.

The main issue here is that I don't currently have a way to
approximate the amount of memory used by any of this. Solving this
requires someone to take a look at the memory used by HAP, shadow and
IOMMU page tables and come up with a formula.

>      for ( ; ; need_paging = false )
>      {
>          nr_pages = dom0_nrpages; 
> 
> Did anyone encounter the same problem or has some thoughts on this issue?

Yes, this a known issue that I haven't got around to fixing yet. The
recommended way to workaround this is to set the dom0_mem parameter to
a sensible value. For example you could set dom0_mem=-512M or -1G to
make sure Xen always has at least 512MB or 1GB, which should be more
than enough to allocate the internal structures for Dom0.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel