[PATCH v3 00/19] mini-os: support of auto-ballooning

[PATCH v3 00/19] mini-os: support of auto-ballooning

[Juergen Gross]

Support ballooning Mini-OS automatically up in case of memory shortage.

Do some cleanups, a small correction and add some basic features to
lay groundwork for support of ballooning in Mini-OS (patches 1-14).

The main visible change is the virtual memory layout: to be able to
add memory to the running Mini-OS we need to have some spare areas
especially after the 1:1 mapping of physical memory.

Then add the ballooning functionality: the p2m map must be expanded,
the page allocator's bitmap must  be expanded and we must get new
memory from the hypervisor.

In case of a detected memory shortage the domain will balloon up until
either enough memory is available or the upper limit has been reached.

Ballooning has been tested with a xenstore stubdom.
Regression tests have been done with:
- pure mini-os
- ioemu stubdom
- pvgrub 64 bit

pvgrub 32 bit didn't work before applying the series, it just entered
the grub shell. With the series applied the behavior was exactly the
same. The grub shell however was working (I tried "help" and "reboot").

I tried to modify arm specific files in order not to break the
non-ballooning case, but I haven't tested it to either work or to
compile.

V1 of this series consisted of patches 1-9 only.

Changes in V3:
- some minor adjustments as requested by Samuel Thibault
- added patch 19

Changes in V2:
- added patches 10-18
- some coding style corrections
- patch 7: introduced balloon specific source files
- moved ballooning specific functions/definitions to ballon specific
  files
- patch 9: avoid conflict with hypervisor mapped area on 32 bits

Juergen Gross (19):
  mini-os: correct first free pfn
  mini-os: remove unused alloc_contig_pages() function
  mini-os: remove MM_DEBUG code
  mini-os: add description of x86 memory usage
  mini-os: add nr_free_pages counter
  mini-os: let memory allocation fail if no free page available
  mini-os: add ballooning config item
  mini-os: get maximum memory size from hypervisor
  mini-os: modify virtual memory layout for support of ballooning
  mini-os: remove unused mem_test() function
  mini-os: add checks for out of memory
  mini-os: don't allocate new pages for level 1 p2m tree
  mini-os: add function to map one frame
  mini-os: move p2m related macros to header file
  mini-os: remap p2m list in case of ballooning
  mini-os: map page allocator's bitmap to virtual kernel area for
    ballooning
  mini-os: add support for ballooning up
  mini-os: balloon up in case of oom
  mini-os: repair build system

 Config.mk             |  93 +++++++++++++++
 Makefile              |  43 +------
 arch/arm/balloon.c    |  39 +++++++
 arch/arm/mm.c         |  10 +-
 arch/x86/Makefile     |   3 -
 arch/x86/balloon.c    | 147 +++++++++++++++++++++++
 arch/x86/mm.c         | 314 +++++++-------------------------------------------
 balloon.c             | 160 +++++++++++++++++++++++++
 config/MiniOS.mk      |  10 --
 config/StdGNU.mk      |  47 --------
 config/arm32.mk       |  22 ----
 config/arm64.mk       |  19 ---
 config/x86_32.mk      |  20 ----
 config/x86_64.mk      |  33 ------
 include/arm/arch_mm.h |   2 +
 include/balloon.h     |  59 ++++++++++
 include/mm.h          |  13 ++-
 include/x86/arch_mm.h |  70 +++++++++++
 minios.mk             |   4 +-
 mm.c                  | 131 ++++++++-------------
 20 files changed, 682 insertions(+), 557 deletions(-)
 create mode 100644 arch/arm/balloon.c
 create mode 100644 arch/x86/balloon.c
 create mode 100644 balloon.c
 delete mode 100644 config/MiniOS.mk
 delete mode 100644 config/StdGNU.mk
 delete mode 100644 config/arm32.mk
 delete mode 100644 config/arm64.mk
 delete mode 100644 config/x86_32.mk
 delete mode 100644 config/x86_64.mk
 create mode 100644 include/balloon.h

-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 01/19] mini-os: correct first free pfn

[Juergen Gross]

The first free pfn available for allocation is calculated by adding the
number of page table frames to the pfn of the first page table and
then the magic number 3 to account for start info page et al.

As the start info page, xenstore page and console page are allocated
_before_ the page tables leaving room for these pages behind the page
tables makes no sense.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/mm.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index 51aa966..ae1036e 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -867,9 +867,8 @@ void arch_init_mm(unsigned long* start_pfn_p, unsigned long* max_pfn_p)
     printk("stack start: %p(VA)\n", stack);
     printk("       _end: %p(VA)\n", &_end);
 
-    /* First page follows page table pages and 3 more pages (store page etc) */
-    start_pfn = PFN_UP(to_phys(start_info.pt_base)) + 
-        start_info.nr_pt_frames + 3;
+    /* First page follows page table pages. */
+    start_pfn = PFN_UP(to_phys(start_info.pt_base)) + start_info.nr_pt_frames;
     max_pfn = start_info.nr_pages;
 
     /* We need room for demand mapping and heap, clip available memory */
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 02/19] mini-os: remove unused alloc_contig_pages() function

[Juergen Gross]

alloc_contig_pages() is never used anywhere in mini-os. Remove it.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/mm.c | 142 ----------------------------------------------------------
 include/mm.h  |   1 -
 2 files changed, 143 deletions(-)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index ae1036e..c59a5d3 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -652,148 +652,6 @@ int unmap_frames(unsigned long va, unsigned long num_frames)
 }
 
 /*
- * Allocate pages which are contiguous in machine memory.
- * Returns a VA to where they are mapped or 0 on failure.
- * 
- * addr_bits indicates if the region has restrictions on where it is
- * located. Typical values are 32 (if for example PCI devices can't access
- * 64bit memory) or 0 for no restrictions.
- *
- * Allocated pages can be freed using the page allocators free_pages() 
- * function.
- *
- * based on Linux function xen_create_contiguous_region()
- */
-#define MAX_CONTIG_ORDER 9 /* 2MB */
-unsigned long alloc_contig_pages(int order, unsigned int addr_bits)
-{
-    unsigned long in_va, va;
-    unsigned long in_frames[1UL << order], out_frames, mfn;
-    multicall_entry_t call[1UL << order];
-    unsigned int i, num_pages = 1UL << order;
-    int ret, exch_success;
-
-    /* pass in num_pages 'extends' of size 1 and
-     * request 1 extend of size 'order */
-    struct xen_memory_exchange exchange = {
-        .in = {
-            .nr_extents   = num_pages,
-            .extent_order = 0,
-            .domid        = DOMID_SELF
-        },
-        .out = {
-            .nr_extents   = 1,
-            .extent_order = order,
-            .address_bits = addr_bits,
-            .domid        = DOMID_SELF
-        },
-        .nr_exchanged = 0
-    };
-
-    if ( order > MAX_CONTIG_ORDER )
-    {
-        printk("alloc_contig_pages: order too large 0x%x > 0x%x\n",
-               order, MAX_CONTIG_ORDER);
-        return 0;
-    }
-
-    /* Allocate some potentially discontiguous pages */
-    in_va = alloc_pages(order);
-    if ( !in_va )
-    {
-        printk("alloc_contig_pages: could not get enough pages (order=0x%x\n",
-               order);
-        return 0;
-    }
-
-    /* set up arguments for exchange hyper call */
-    set_xen_guest_handle(exchange.in.extent_start, in_frames);
-    set_xen_guest_handle(exchange.out.extent_start, &out_frames);
-
-    /* unmap current frames, keep a list of MFNs */
-    for ( i = 0; i < num_pages; i++ )
-    {
-        int arg = 0;
-
-        va = in_va + (PAGE_SIZE * i);
-        in_frames[i] = virt_to_mfn(va);
-
-        /* update P2M mapping */
-        phys_to_machine_mapping[virt_to_pfn(va)] = INVALID_P2M_ENTRY;
-
-        /* build multi call */
-        call[i].op = __HYPERVISOR_update_va_mapping;
-        call[i].args[arg++] = va;
-        call[i].args[arg++] = 0;
-#ifdef __i386__
-        call[i].args[arg++] = 0;
-#endif  
-        call[i].args[arg++] = UVMF_INVLPG;
-    }
-
-    ret = HYPERVISOR_multicall(call, i);
-    if ( ret )
-    {
-        printk("Odd, update_va_mapping hypercall failed with rc=%d.\n", ret);
-        return 0;
-    }
-
-    /* try getting a contig range of MFNs */
-    out_frames = virt_to_pfn(in_va); /* PFNs to populate */
-    ret = HYPERVISOR_memory_op(XENMEM_exchange, &exchange);
-    if ( ret ) {
-        printk("mem exchanged order=0x%x failed with rc=%d, nr_exchanged=%lu\n",
-               order, ret, exchange.nr_exchanged);
-        /* we still need to return the allocated pages above to the pool
-         * ie. map them back into the 1:1 mapping etc. so we continue but 
-         * in the end return the pages to the page allocator and return 0. */
-        exch_success = 0;
-    }
-    else
-        exch_success = 1;
-
-    /* map frames into 1:1 and update p2m */
-    for ( i = 0; i < num_pages; i++ )
-    {
-        int arg = 0;
-        pte_t pte;
-
-        va = in_va + (PAGE_SIZE * i);
-        mfn = i < exchange.nr_exchanged ? (out_frames + i) : in_frames[i];
-        pte = __pte(mfn << PAGE_SHIFT | L1_PROT);
-
-        /* update P2M mapping */
-        phys_to_machine_mapping[virt_to_pfn(va)] = mfn;
-
-        /* build multi call */
-        call[i].op = __HYPERVISOR_update_va_mapping;
-        call[i].args[arg++] = va;
-#ifdef __x86_64__
-        call[i].args[arg++] = (pgentry_t)pte.pte;
-#else
-        call[i].args[arg++] = pte.pte_low;
-        call[i].args[arg++] = pte.pte_high;
-#endif  
-        call[i].args[arg++] = UVMF_INVLPG;
-    }
-    ret = HYPERVISOR_multicall(call, i);
-    if ( ret )
-    {
-        printk("update_va_mapping hypercall no. 2 failed with rc=%d.\n", ret);
-        return 0;
-    }
-
-    if ( !exch_success )
-    {
-        /* since the exchanged failed we just free the pages as well */
-        free_pages((void *) in_va, order);
-        return 0;
-    }
-    
-    return in_va;
-}
-
-/*
  * Clear some of the bootstrap memory
  */
 static void clear_bootstrap(void)
diff --git a/include/mm.h b/include/mm.h
index f57d8ab..a48f485 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -71,7 +71,6 @@ void do_map_frames(unsigned long addr,
         const unsigned long *f, unsigned long n, unsigned long stride,
 	unsigned long increment, domid_t id, int *err, unsigned long prot);
 int unmap_frames(unsigned long va, unsigned long num_frames);
-unsigned long alloc_contig_pages(int order, unsigned int addr_bits);
 #ifdef HAVE_LIBC
 extern unsigned long heap, brk, heap_mapped, heap_end;
 #endif
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 03/19] mini-os: remove MM_DEBUG code

[Juergen Gross]

mm.c contains unused code inside #ifdef MM_DEBUG areas. Its usability
is rather questionable and some parts are even wrong (e.g.
print_chunks() called with nr_pages > 1000 will clobber an arbitrary
stack content with a 0 byte).

Remove this code.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 mm.c | 60 ------------------------------------------------------------
 1 file changed, 60 deletions(-)

diff --git a/mm.c b/mm.c
index 31aaf83..0dd4862 100644
--- a/mm.c
+++ b/mm.c
@@ -42,13 +42,6 @@
 #include <mini-os/lib.h>
 #include <mini-os/xmalloc.h>
 
-#ifdef MM_DEBUG
-#define DEBUG(_f, _a...) \
-    printk("MINI_OS(file=mm.c, line=%d) " _f "\n", __LINE__, ## _a)
-#else
-#define DEBUG(_f, _a...)    ((void)0)
-#endif
-
 /*********************
  * ALLOCATION BITMAP
  *  One bit per page of memory. Bit set => page is allocated.
@@ -140,59 +133,6 @@ static chunk_head_t  free_tail[FREELIST_SIZE];
 #define round_pgdown(_p)  ((_p)&PAGE_MASK)
 #define round_pgup(_p)    (((_p)+(PAGE_SIZE-1))&PAGE_MASK)
 
-#ifdef MM_DEBUG
-/*
- * Prints allocation[0/1] for @nr_pages, starting at @start
- * address (virtual).
- */
-USED static void print_allocation(void *start, int nr_pages)
-{
-    unsigned long pfn_start = virt_to_pfn(start);
-    int count;
-    for(count = 0; count < nr_pages; count++)
-        if(allocated_in_map(pfn_start + count)) printk("1");
-        else printk("0");
-        
-    printk("\n");        
-}
-
-/*
- * Prints chunks (making them with letters) for @nr_pages starting
- * at @start (virtual).
- */
-USED static void print_chunks(void *start, int nr_pages)
-{
-    char chunks[1001], current='A';
-    int order, count;
-    chunk_head_t *head;
-    unsigned long pfn_start = virt_to_pfn(start);
-   
-    memset(chunks, (int)'_', 1000);
-    if(nr_pages > 1000) 
-    {
-        DEBUG("Can only pring 1000 pages. Increase buffer size.");
-    }
-    
-    for(order=0; order < FREELIST_SIZE; order++)
-    {
-        head = free_head[order];
-        while(!FREELIST_EMPTY(head))
-        {
-            for(count = 0; count < 1UL<< head->level; count++)
-            {
-                if(count + virt_to_pfn(head) - pfn_start < 1000)
-                    chunks[count + virt_to_pfn(head) - pfn_start] = current;
-            }
-            head = head->next;
-            current++;
-        }
-    }
-    chunks[nr_pages] = '\0';
-    printk("%s\n", chunks);
-}
-#endif
-
-
 /*
  * Initialise allocator, placing addresses [@min,@max] in free pool.
  * @min and @max are PHYSICAL addresses.
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 04/19] mini-os: add description of x86 memory usage

[Juergen Gross]

Add a brief description how the physical and virtual address usage
looks like on x86 to include/x86/arch_mm.h

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 include/x86/arch_mm.h | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index 58f29fc..f756dab 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -36,6 +36,26 @@
 #endif
 #endif
 
+/*
+ * Physical address space usage:
+ *
+ * 0..._edata: kernel text/data
+ * *stack    : kernel stack (thread 0)
+ * hypervisor allocated data: p2m_list, start_info page, xenstore page,
+ *                            console page, initial page tables
+ * bitmap of allocated pages
+ * pages controlled by the page allocator
+ *
+ *
+ * Virtual address space usage:
+ *
+ * 1:1 mapping of physical memory starting at VA(0)
+ * 1 unallocated page
+ * demand map area (32 bits: 2 GB, 64 bits: 128 GB) for virtual allocations
+ * 1 unallocated page
+ * with libc: heap area (32 bits: 1 GB, 64 bits: 128 GB)
+ */
+
 #define L1_FRAME                1
 #define L2_FRAME                2
 #define L3_FRAME                3
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 05/19] mini-os: add nr_free_pages counter

[Juergen Gross]

Add a variable holding the number of available memory pages. This will
aid auto-ballooning later.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 include/mm.h | 1 +
 mm.c         | 6 ++++++
 2 files changed, 7 insertions(+)

diff --git a/include/mm.h b/include/mm.h
index a48f485..b97b43e 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -42,6 +42,7 @@
 #define STACK_SIZE_PAGE_ORDER __STACK_SIZE_PAGE_ORDER
 #define STACK_SIZE __STACK_SIZE
 
+extern unsigned long nr_free_pages;
 
 void init_mm(void);
 unsigned long alloc_pages(int order);
diff --git a/mm.c b/mm.c
index 0dd4862..263a356 100644
--- a/mm.c
+++ b/mm.c
@@ -53,6 +53,8 @@ static unsigned long *alloc_bitmap;
 #define allocated_in_map(_pn) \
 (alloc_bitmap[(_pn)/PAGES_PER_MAPWORD] & (1UL<<((_pn)&(PAGES_PER_MAPWORD-1))))
 
+unsigned long nr_free_pages;
+
 /*
  * Hint regarding bitwise arithmetic in map_{alloc,free}:
  *  -(1<<n)  sets all bits >= n. 
@@ -81,6 +83,8 @@ static void map_alloc(unsigned long first_page, unsigned long nr_pages)
         while ( ++curr_idx < end_idx ) alloc_bitmap[curr_idx] = ~0UL;
         alloc_bitmap[curr_idx] |= (1UL<<end_off)-1;
     }
+
+    nr_free_pages -= nr_pages;
 }
 
 
@@ -93,6 +97,8 @@ static void map_free(unsigned long first_page, unsigned long nr_pages)
     end_idx   = (first_page + nr_pages) / PAGES_PER_MAPWORD;
     end_off   = (first_page + nr_pages) & (PAGES_PER_MAPWORD-1);
 
+    nr_free_pages += nr_pages;
+
     if ( curr_idx == end_idx )
     {
         alloc_bitmap[curr_idx] &= -(1UL<<end_off) | ((1UL<<start_off)-1);
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 06/19] mini-os: let memory allocation fail if no free page available

[Juergen Gross]

Instead of panicing when no page can be allocated try to fail the
memory allocation by returning NULL instead.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
V2: fixed minor style issue
---
 mm.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/mm.c b/mm.c
index 263a356..8cf3210 100644
--- a/mm.c
+++ b/mm.c
@@ -335,6 +335,13 @@ void *sbrk(ptrdiff_t increment)
     
     if (new_brk > heap_mapped) {
         unsigned long n = (new_brk - heap_mapped + PAGE_SIZE - 1) / PAGE_SIZE;
+
+        if ( n > nr_free_pages )
+        {
+            printk("Memory exhausted: want %ld pages, but only %ld are left\n",
+                   n, nr_free_pages);
+            return NULL;
+        }
         do_map_zero(heap_mapped, n);
         heap_mapped += n * PAGE_SIZE;
     }
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 07/19] mini-os: add ballooning config item

[Juergen Gross]

Add CONFIG_BALLOON defaulting to 'n' as a config item to Mini-OS.

Add balloon.c, balloon.h and arch/*/balloon.c for future use.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
V2: Added dummy sources and header
---
 Makefile           |  3 +++
 arch/arm/balloon.c | 28 ++++++++++++++++++++++++++++
 arch/x86/balloon.c | 28 ++++++++++++++++++++++++++++
 balloon.c          | 24 ++++++++++++++++++++++++
 include/balloon.h  | 32 ++++++++++++++++++++++++++++++++
 5 files changed, 115 insertions(+)
 create mode 100644 arch/arm/balloon.c
 create mode 100644 arch/x86/balloon.c
 create mode 100644 balloon.c
 create mode 100644 include/balloon.h

diff --git a/Makefile b/Makefile
index 2e4bdba..f5b7011 100644
--- a/Makefile
+++ b/Makefile
@@ -33,6 +33,7 @@ CONFIG_CONSFRONT ?= y
 CONFIG_XENBUS ?= y
 CONFIG_XC ?=y
 CONFIG_LWIP ?= $(lwip)
+CONFIG_BALLOON ?= n
 
 # Export config items as compiler directives
 flags-$(CONFIG_START_NETWORK) += -DCONFIG_START_NETWORK
@@ -48,6 +49,7 @@ flags-$(CONFIG_KBDFRONT) += -DCONFIG_KBDFRONT
 flags-$(CONFIG_FBFRONT) += -DCONFIG_FBFRONT
 flags-$(CONFIG_CONSFRONT) += -DCONFIG_CONSFRONT
 flags-$(CONFIG_XENBUS) += -DCONFIG_XENBUS
+flags-$(CONFIG_BALLOON) += -DCONFIG_BALLOON
 
 DEF_CFLAGS += $(flags-y)
 
@@ -96,6 +98,7 @@ src-$(CONFIG_NETFRONT) += netfront.c
 src-$(CONFIG_PCIFRONT) += pcifront.c
 src-y += sched.c
 src-$(CONFIG_TEST) += test.c
+src-$(CONFIG_BALLOON) += balloon.c
 
 src-y += lib/ctype.c
 src-y += lib/math.c
diff --git a/arch/arm/balloon.c b/arch/arm/balloon.c
new file mode 100644
index 0000000..28021d6
--- /dev/null
+++ b/arch/arm/balloon.c
@@ -0,0 +1,28 @@
+/* -*-  Mode:C; c-basic-offset:4; tab-width:4 -*-
+ *
+ * (C) 2016 - Juergen Gross, SUSE Linux GmbH
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include <mini-os/balloon.h>
+
+#ifdef CONFIG_BALLOON
+
+#endif
diff --git a/arch/x86/balloon.c b/arch/x86/balloon.c
new file mode 100644
index 0000000..28021d6
--- /dev/null
+++ b/arch/x86/balloon.c
@@ -0,0 +1,28 @@
+/* -*-  Mode:C; c-basic-offset:4; tab-width:4 -*-
+ *
+ * (C) 2016 - Juergen Gross, SUSE Linux GmbH
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include <mini-os/balloon.h>
+
+#ifdef CONFIG_BALLOON
+
+#endif
diff --git a/balloon.c b/balloon.c
new file mode 100644
index 0000000..f9cf23b
--- /dev/null
+++ b/balloon.c
@@ -0,0 +1,24 @@
+/* -*-  Mode:C; c-basic-offset:4; tab-width:4 -*-
+ *
+ * (C) 2016 - Juergen Gross, SUSE Linux GmbH
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#include <mini-os/balloon.h>
diff --git a/include/balloon.h b/include/balloon.h
new file mode 100644
index 0000000..9756a3f
--- /dev/null
+++ b/include/balloon.h
@@ -0,0 +1,32 @@
+/* -*-  Mode:C; c-basic-offset:4; tab-width:4 -*-
+ *
+ * (C) 2016 - Juergen Gross, SUSE Linux GmbH
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to
+ * deal in the Software without restriction, including without limitation the
+ * rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
+ * sell copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+ * DEALINGS IN THE SOFTWARE.
+ */
+
+#ifndef _BALLOON_H_
+#define _BALLOON_H_
+
+#ifdef CONFIG_BALLOON
+
+#else /* CONFIG_BALLOON */
+
+#endif /* CONFIG_BALLOON */
+#endif /* _BALLOON_H_ */
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 08/19] mini-os: get maximum memory size from hypervisor

[Juergen Gross]

Add support for obtaining the maximum memory size from the hypervisor.
This will make it possible to support ballooning.

Signed-off-by: Juergen Gross <jgross@suse.com>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
V2: Moved new stuff to balloon.c
---
 balloon.c         | 22 ++++++++++++++++++++++
 include/balloon.h |  6 ++++++
 mm.c              |  2 ++
 3 files changed, 30 insertions(+)

diff --git a/balloon.c b/balloon.c
index f9cf23b..1ec113d 100644
--- a/balloon.c
+++ b/balloon.c
@@ -21,4 +21,26 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
+#include <mini-os/os.h>
 #include <mini-os/balloon.h>
+#include <mini-os/lib.h>
+#include <xen/xen.h>
+#include <xen/memory.h>
+
+unsigned long nr_max_pages;
+
+void get_max_pages(void)
+{
+    long ret;
+    domid_t domid = DOMID_SELF;
+
+    ret = HYPERVISOR_memory_op(XENMEM_maximum_reservation, &domid);
+    if ( ret < 0 )
+    {
+        printk("Could not get maximum pfn\n");
+        return;
+    }
+
+    nr_max_pages = ret;
+    printk("Maximum memory size: %ld pages\n", nr_max_pages);
+}
diff --git a/include/balloon.h b/include/balloon.h
index 9756a3f..cd79017 100644
--- a/include/balloon.h
+++ b/include/balloon.h
@@ -26,7 +26,13 @@
 
 #ifdef CONFIG_BALLOON
 
+extern unsigned long nr_max_pages;
+
+void get_max_pages(void);
+
 #else /* CONFIG_BALLOON */
 
+static inline void get_max_pages(void) { }
+
 #endif /* CONFIG_BALLOON */
 #endif /* _BALLOON_H_ */
diff --git a/mm.c b/mm.c
index 8cf3210..6d82f2a 100644
--- a/mm.c
+++ b/mm.c
@@ -38,6 +38,7 @@
 #include <mini-os/hypervisor.h>
 #include <xen/memory.h>
 #include <mini-os/mm.h>
+#include <mini-os/balloon.h>
 #include <mini-os/types.h>
 #include <mini-os/lib.h>
 #include <mini-os/xmalloc.h>
@@ -361,6 +362,7 @@ void init_mm(void)
 
     printk("MM: Init\n");
 
+    get_max_pages();
     arch_init_mm(&start_pfn, &max_pfn);
     /*
      * now we can initialise the page allocator
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 09/19] mini-os: modify virtual memory layout for support of ballooning

[Juergen Gross]

In order to be able to support ballooning the virtual memory layout
of Mini-OS has to be modified: instead of a (nearly) consecutive
area used for physical memory mapping, on demand mappings, and heap
we need enough spare place for adding new memory.

So instead of dynamically place the different regions based on found
memory size locate them statically at fixed virtual addresses:

area                           x86-64               x86-32
------------------------------------------------------------
mapped physical memory       00000000             00000000
kernel virtual mappings    8000000000             3f000000
demand mappings          100000000000             40000000
heap                     200000000000             b0000000

This will enable Mini-OS to support up to 512GB of domain memory with
a 64 bit kernel and nearly 1GB with a 32 bit kernel.

For a 32 bit Mini-OS we have to avoid a conflict between heap and
m2p table which the hypervisor maps at f5600000. So the demand mapping
size is reduced by 256MB in order to keep the heap at about 1GB.

The kernel virtual mappings are a new area needed for being able to
grow the p2m list without having to relocate it in physical memory.

Modify the placement of the demand mappings and heap and adjust the
memory layout description.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
V2: avoid conflict with hypervisor mapped area on 32 bits
---
 arch/arm/mm.c         |  2 +-
 arch/x86/mm.c         | 44 +++++++++++---------------------------------
 include/mm.h          |  2 +-
 include/x86/arch_mm.h | 35 ++++++++++++++++++++++++++++++-----
 mm.c                  |  2 +-
 5 files changed, 44 insertions(+), 41 deletions(-)

diff --git a/arch/arm/mm.c b/arch/arm/mm.c
index efecc51..f75888d 100644
--- a/arch/arm/mm.c
+++ b/arch/arm/mm.c
@@ -75,7 +75,7 @@ void arch_init_p2m(unsigned long max_pfn)
 {
 }
 
-void arch_init_demand_mapping_area(unsigned long cur_pfn)
+void arch_init_demand_mapping_area(void)
 {
 }
 
diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index c59a5d3..6aa4468 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -442,37 +442,21 @@ pgentry_t *need_pgt(unsigned long va)
  * Reserve an area of virtual address space for mappings and Heap
  */
 static unsigned long demand_map_area_start;
-#ifdef __x86_64__
-#define DEMAND_MAP_PAGES ((128ULL << 30) / PAGE_SIZE)
-#else
-#define DEMAND_MAP_PAGES ((2ULL << 30) / PAGE_SIZE)
-#endif
-
-#ifndef HAVE_LIBC
-#define HEAP_PAGES 0
-#else
+static unsigned long demand_map_area_end;
+#ifdef HAVE_LIBC
 unsigned long heap, brk, heap_mapped, heap_end;
-#ifdef __x86_64__
-#define HEAP_PAGES ((128ULL << 30) / PAGE_SIZE)
-#else
-#define HEAP_PAGES ((1ULL << 30) / PAGE_SIZE)
-#endif
 #endif
 
-void arch_init_demand_mapping_area(unsigned long cur_pfn)
+void arch_init_demand_mapping_area(void)
 {
-    cur_pfn++;
-
-    demand_map_area_start = (unsigned long) pfn_to_virt(cur_pfn);
-    cur_pfn += DEMAND_MAP_PAGES;
-    printk("Demand map pfns at %lx-%p.\n", 
-           demand_map_area_start, pfn_to_virt(cur_pfn));
+    demand_map_area_start = VIRT_DEMAND_AREA;
+    demand_map_area_end = demand_map_area_start + DEMAND_MAP_PAGES * PAGE_SIZE;
+    printk("Demand map pfns at %lx-%lx.\n", demand_map_area_start,
+           demand_map_area_end);
 
 #ifdef HAVE_LIBC
-    cur_pfn++;
-    heap_mapped = brk = heap = (unsigned long) pfn_to_virt(cur_pfn);
-    cur_pfn += HEAP_PAGES;
-    heap_end = (unsigned long) pfn_to_virt(cur_pfn);
+    heap_mapped = brk = heap = VIRT_HEAP_AREA;
+    heap_end = heap_mapped + HEAP_PAGES * PAGE_SIZE;
     printk("Heap resides at %lx-%lx.\n", brk, heap_end);
 #endif
 }
@@ -729,14 +713,8 @@ void arch_init_mm(unsigned long* start_pfn_p, unsigned long* max_pfn_p)
     start_pfn = PFN_UP(to_phys(start_info.pt_base)) + start_info.nr_pt_frames;
     max_pfn = start_info.nr_pages;
 
-    /* We need room for demand mapping and heap, clip available memory */
-#if defined(__i386__)
-    {
-        unsigned long virt_pfns = 1 + DEMAND_MAP_PAGES + 1 + HEAP_PAGES;
-        if (max_pfn + virt_pfns >= 0x100000)
-            max_pfn = 0x100000 - virt_pfns - 1;
-    }
-#endif
+    if ( max_pfn >= MAX_MEM_SIZE / PAGE_SIZE )
+        max_pfn = MAX_MEM_SIZE / PAGE_SIZE - 1;
 
     printk("  start_pfn: %lx\n", start_pfn);
     printk("    max_pfn: %lx\n", max_pfn);
diff --git a/include/mm.h b/include/mm.h
index b97b43e..a22dcd1 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -59,7 +59,7 @@ static __inline__ int get_order(unsigned long size)
     return order;
 }
 
-void arch_init_demand_mapping_area(unsigned long max_pfn);
+void arch_init_demand_mapping_area(void);
 void arch_init_mm(unsigned long* start_pfn_p, unsigned long* max_pfn_p);
 void arch_init_p2m(unsigned long max_pfn_p);
 
diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index f756dab..d87fe55 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -49,11 +49,13 @@
  *
  * Virtual address space usage:
  *
- * 1:1 mapping of physical memory starting at VA(0)
- * 1 unallocated page
- * demand map area (32 bits: 2 GB, 64 bits: 128 GB) for virtual allocations
- * 1 unallocated page
- * with libc: heap area (32 bits: 1 GB, 64 bits: 128 GB)
+ *  area                           x86-64               x86-32
+ *  ------------------------------------------------------------
+ *  mapped physical memory       00000000             00000000
+ *  kernel virtual mappings    8000000000             3f000000
+ *  demand mappings          100000000000             40000000
+ *  heap (with libc only)    200000000000             b0000000
+ *
  */
 
 #define L1_FRAME                1
@@ -81,6 +83,15 @@
 typedef uint64_t pgentry_t;
 #endif
 
+#define MAX_MEM_SIZE            0x3f000000UL
+#define VIRT_KERNEL_AREA        0x3f000000UL
+#define VIRT_DEMAND_AREA        0x40000000UL
+#define VIRT_HEAP_AREA          0xb0000000UL
+
+#define DEMAND_MAP_PAGES        0x6ffffUL
+#define HEAP_PAGES_MAX          ((HYPERVISOR_VIRT_START - VIRT_HEAP_AREA) / \
+                                 PAGE_SIZE - 1)
+
 #elif defined(__x86_64__)
 
 #define L2_PAGETABLE_SHIFT      21
@@ -106,6 +117,20 @@ typedef uint64_t pgentry_t;
 typedef unsigned long pgentry_t;
 #endif
 
+#define MAX_MEM_SIZE            (512ULL << 30)
+#define VIRT_KERNEL_AREA        0x0000008000000000UL
+#define VIRT_DEMAND_AREA        0x0000100000000000UL
+#define VIRT_HEAP_AREA          0x0000200000000000UL
+
+#define DEMAND_MAP_PAGES        0x8000000UL
+#define HEAP_PAGES_MAX          0x8000000UL
+
+#endif
+
+#ifndef HAVE_LIBC
+#define HEAP_PAGES 0
+#else
+#define HEAP_PAGES   HEAP_PAGES_MAX
 #endif
 
 #define L1_MASK  ((1UL << L2_PAGETABLE_SHIFT) - 1)
diff --git a/mm.c b/mm.c
index 6d82f2a..c53b0ca 100644
--- a/mm.c
+++ b/mm.c
@@ -375,7 +375,7 @@ void init_mm(void)
 
     arch_init_p2m(max_pfn);
     
-    arch_init_demand_mapping_area(max_pfn);
+    arch_init_demand_mapping_area();
 }
 
 void fini_mm(void)
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 10/19] mini-os: remove unused mem_test() function

[Juergen Gross]

mem_test() isn't used anywhere and its value is rather questionable
with mini-os being in a mature state. Remove the function.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/mm.c | 55 -------------------------------------------------------
 1 file changed, 55 deletions(-)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index 6aa4468..e2f026b 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -302,61 +302,6 @@ static void set_readonly(void *text, void *etext)
 }
 
 /*
- * A useful mem testing function. Write the address to every address in the
- * range provided and read back the value. If verbose, print page walk to
- * some VA
- * 
- * If we get MEM_TEST_MAX_ERRORS we might as well stop
- */
-#define MEM_TEST_MAX_ERRORS 10 
-int mem_test(unsigned long *start_va, unsigned long *end_va, int verbose)
-{
-    unsigned long mask = 0x10000;
-    unsigned long *pointer;
-    int error_count = 0;
- 
-    /* write values and print page walks */
-    if ( verbose && (((unsigned long)start_va) & 0xfffff) )
-    {
-        printk("MemTest Start: 0x%p\n", start_va);
-        page_walk((unsigned long)start_va);
-    }
-    for ( pointer = start_va; pointer < end_va; pointer++ )
-    {
-        if ( verbose && !(((unsigned long)pointer) & 0xfffff) )
-        {
-            printk("Writing to %p\n", pointer);
-            page_walk((unsigned long)pointer);
-        }
-        *pointer = (unsigned long)pointer & ~mask;
-    }
-    if ( verbose && (((unsigned long)end_va) & 0xfffff) )
-    {
-        printk("MemTest End: %p\n", end_va-1);
-        page_walk((unsigned long)end_va-1);
-    }
- 
-    /* verify values */
-    for ( pointer = start_va; pointer < end_va; pointer++ )
-    {
-        if ( ((unsigned long)pointer & ~mask) != *pointer )
-        {
-            printk("Read error at 0x%lx. Read: 0x%lx, should read 0x%lx\n",
-                   (unsigned long)pointer, *pointer, 
-                   ((unsigned long)pointer & ~mask));
-            error_count++;
-            if ( error_count >= MEM_TEST_MAX_ERRORS )
-            {
-                printk("mem_test: too many errors\n");
-                return -1;
-            }
-        }
-    }
-    return 0;
-}
-
-
-/*
  * get the PTE for virtual address va if it exists. Otherwise NULL.
  */
 static pgentry_t *get_pgt(unsigned long va)
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 11/19] mini-os: add checks for out of memory

[Juergen Gross]

There are several core functions in Mini-OS not checking for failed
memory allocations. Add such checks.

Add do_map_frames() dummy function to arm architecture as it will be
needed in future for compilations to succeed.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/arm/mm.c |  8 ++++++++
 arch/x86/mm.c | 26 +++++++++++++++++++-------
 include/mm.h  |  2 +-
 3 files changed, 28 insertions(+), 8 deletions(-)

diff --git a/arch/arm/mm.c b/arch/arm/mm.c
index f75888d..fc8d4bc 100644
--- a/arch/arm/mm.c
+++ b/arch/arm/mm.c
@@ -1,6 +1,7 @@
 #include <mini-os/console.h>
 #include <xen/memory.h>
 #include <arch_mm.h>
+#include <mini-os/errno.h>
 #include <mini-os/hypervisor.h>
 #include <libfdt.h>
 #include <lib.h>
@@ -79,6 +80,13 @@ void arch_init_demand_mapping_area(void)
 {
 }
 
+int do_map_frames(unsigned long addr,
+        const unsigned long *f, unsigned long n, unsigned long stride,
+        unsigned long increment, domid_t id, int *err, unsigned long prot)
+{
+    return -ENOSYS;
+}
+
 /* Get Xen's suggested physical page assignments for the grant table. */
 static paddr_t get_gnttab_base(void)
 {
diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index e2f026b..12f7fe4 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -34,6 +34,7 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
+#include <mini-os/errno.h>
 #include <mini-os/os.h>
 #include <mini-os/hypervisor.h>
 #include <mini-os/mm.h>
@@ -354,6 +355,8 @@ pgentry_t *need_pgt(unsigned long va)
     if ( !(tab[offset] & _PAGE_PRESENT) )
     {
         pt_pfn = virt_to_pfn(alloc_page());
+        if ( !pt_pfn )
+            return NULL;
         new_pt_frame(&pt_pfn, pt_mfn, offset, L3_FRAME);
     }
     ASSERT(tab[offset] & _PAGE_PRESENT);
@@ -364,6 +367,8 @@ pgentry_t *need_pgt(unsigned long va)
     if ( !(tab[offset] & _PAGE_PRESENT) ) 
     {
         pt_pfn = virt_to_pfn(alloc_page());
+        if ( !pt_pfn )
+            return NULL;
         new_pt_frame(&pt_pfn, pt_mfn, offset, L2_FRAME);
     }
     ASSERT(tab[offset] & _PAGE_PRESENT);
@@ -373,6 +378,8 @@ pgentry_t *need_pgt(unsigned long va)
     if ( !(tab[offset] & _PAGE_PRESENT) )
     {
         pt_pfn = virt_to_pfn(alloc_page());
+        if ( !pt_pfn )
+            return NULL;
         new_pt_frame(&pt_pfn, pt_mfn, offset, L1_FRAME);
     }
     ASSERT(tab[offset] & _PAGE_PRESENT);
@@ -445,10 +452,10 @@ unsigned long allocate_ondemand(unsigned long n, unsigned long alignment)
  * va. map f[i*stride]+i*increment for i in 0..n-1.
  */
 #define MAP_BATCH ((STACK_SIZE / 2) / sizeof(mmu_update_t))
-void do_map_frames(unsigned long va,
-                   const unsigned long *mfns, unsigned long n, 
-                   unsigned long stride, unsigned long incr, 
-                   domid_t id, int *err, unsigned long prot)
+int do_map_frames(unsigned long va,
+                  const unsigned long *mfns, unsigned long n,
+                  unsigned long stride, unsigned long incr,
+                  domid_t id, int *err, unsigned long prot)
 {
     pgentry_t *pgt = NULL;
     unsigned long done = 0;
@@ -458,7 +465,7 @@ void do_map_frames(unsigned long va,
     if ( !mfns ) 
     {
         printk("do_map_frames: no mfns supplied\n");
-        return;
+        return -EINVAL;
     }
     DEBUG("va=%p n=0x%lx, mfns[0]=0x%lx stride=0x%lx incr=0x%lx prot=0x%lx\n",
           va, n, mfns[0], stride, incr, prot);
@@ -484,7 +491,9 @@ void do_map_frames(unsigned long va,
             {
                 if ( !pgt || !(va & L1_MASK) )
                     pgt = need_pgt(va);
-                
+                if ( !pgt )
+                    return -ENOMEM;
+
                 mmu_updates[i].ptr = virt_to_mach(pgt) | MMU_NORMAL_PT_UPDATE;
                 mmu_updates[i].val = ((pgentry_t)(mfns[(done + i) * stride] +
                                                   (done + i) * incr)
@@ -505,6 +514,8 @@ void do_map_frames(unsigned long va,
         }
         done += todo;
     }
+
+    return 0;
 }
 
 /*
@@ -521,7 +532,8 @@ void *map_frames_ex(const unsigned long *mfns, unsigned long n,
     if ( !va )
         return NULL;
 
-    do_map_frames(va, mfns, n, stride, incr, id, err, prot);
+    if ( do_map_frames(va, mfns, n, stride, incr, id, err, prot) )
+        return NULL;
 
     return (void *)va;
 }
diff --git a/include/mm.h b/include/mm.h
index a22dcd1..9244e26 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -68,7 +68,7 @@ unsigned long allocate_ondemand(unsigned long n, unsigned long alignment);
 void *map_frames_ex(const unsigned long *f, unsigned long n, unsigned long stride,
 	unsigned long increment, unsigned long alignment, domid_t id,
 	int *err, unsigned long prot);
-void do_map_frames(unsigned long addr,
+int do_map_frames(unsigned long addr,
         const unsigned long *f, unsigned long n, unsigned long stride,
 	unsigned long increment, domid_t id, int *err, unsigned long prot);
 int unmap_frames(unsigned long va, unsigned long num_frames);
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 12/19] mini-os: don't allocate new pages for level 1 p2m tree

[Juergen Gross]

When constructing the 3 level p2m tree there is no need to allocate
new pages for the level 1 containing the p2m info for all pages. The
pages from the linear p2m list constructed by the hypervisor can be
used for that purpose.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/mm.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index 12f7fe4..e10c2c5 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -625,11 +625,11 @@ void arch_init_p2m(unsigned long max_pfn)
 #define L2_P2M_MASK     (L2_P2M_ENTRIES - 1)    
 #define L3_P2M_MASK     (L3_P2M_ENTRIES - 1)    
     
-    unsigned long *l1_list = NULL, *l2_list = NULL, *l3_list;
+    unsigned long *l2_list = NULL, *l3_list;
     unsigned long pfn;
     
     l3_list = (unsigned long *)alloc_page(); 
-    for ( pfn=0; pfn<max_pfn; pfn++ )
+    for ( pfn = 0; pfn < max_pfn; pfn += L1_P2M_ENTRIES )
     {
         if ( !(pfn % (L1_P2M_ENTRIES * L2_P2M_ENTRIES)) )
         {
@@ -641,14 +641,8 @@ void arch_init_p2m(unsigned long max_pfn)
             }
             l3_list[(pfn >> L2_P2M_SHIFT)] = virt_to_mfn(l2_list);  
         }
-        if ( !(pfn % (L1_P2M_ENTRIES)) )
-        {
-            l1_list = (unsigned long*)alloc_page();
-            l2_list[(pfn >> L1_P2M_SHIFT) & L2_P2M_MASK] = 
-                virt_to_mfn(l1_list); 
-        }
-
-        l1_list[pfn & L1_P2M_MASK] = pfn_to_mfn(pfn); 
+        l2_list[(pfn >> L1_P2M_SHIFT) & L2_P2M_MASK] =
+            virt_to_mfn(phys_to_machine_mapping + pfn);
     }
     HYPERVISOR_shared_info->arch.pfn_to_mfn_frame_list_list = 
         virt_to_mfn(l3_list);
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 13/19] mini-os: add function to map one frame

[Juergen Gross]

Add a function to map one physical frame to a specified virtual
address as read/write. This will be used later multiple times.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 include/arm/arch_mm.h | 2 ++
 include/mm.h          | 1 +
 mm.c                  | 5 +++++
 3 files changed, 8 insertions(+)

diff --git a/include/arm/arch_mm.h b/include/arm/arch_mm.h
index 085d4e5..f4685d8 100644
--- a/include/arm/arch_mm.h
+++ b/include/arm/arch_mm.h
@@ -14,6 +14,8 @@ extern uint32_t physical_address_offset;	/* Add this to a virtual address to get
 
 #define L1_PAGETABLE_SHIFT      12
 
+#define L1_PROT          0
+
 #define to_phys(x)                 (((paddr_t)(x)+physical_address_offset) & 0xffffffff)
 #define to_virt(x)                 ((void *)(((x)-physical_address_offset) & 0xffffffff))
 
diff --git a/include/mm.h b/include/mm.h
index 9244e26..6add683 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -72,6 +72,7 @@ int do_map_frames(unsigned long addr,
         const unsigned long *f, unsigned long n, unsigned long stride,
 	unsigned long increment, domid_t id, int *err, unsigned long prot);
 int unmap_frames(unsigned long va, unsigned long num_frames);
+int map_frame_rw(unsigned long addr, unsigned long mfn);
 #ifdef HAVE_LIBC
 extern unsigned long heap, brk, heap_mapped, heap_end;
 #endif
diff --git a/mm.c b/mm.c
index c53b0ca..707a3e0 100644
--- a/mm.c
+++ b/mm.c
@@ -319,6 +319,11 @@ int free_physical_pages(xen_pfn_t *mfns, int n)
     return HYPERVISOR_memory_op(XENMEM_decrease_reservation, &reservation);
 }
 
+int map_frame_rw(unsigned long addr, unsigned long mfn)
+{
+    return do_map_frames(addr, &mfn, 1, 1, 1, DOMID_SELF, NULL, L1_PROT);
+}
+
 #ifdef HAVE_LIBC
 void *sbrk(ptrdiff_t increment)
 {
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 14/19] mini-os: move p2m related macros to header file

[Juergen Gross]

In order to be able to use p2m related macros for ballooning move
their definitions to arch/x86/mm.h.

There is no need to define different macros regarding index masks and
number of entries for the different levels, as all levels share the
same entry format (a plain mfn). So reduce the number of macros
accordingly.

Add some macros to get the indices into p2m pages from a pfn and make
use of them in current p2m code.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/mm.c         | 31 +++++--------------------------
 include/x86/arch_mm.h | 21 +++++++++++++++++++++
 2 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index e10c2c5..a5c8959 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -609,40 +609,19 @@ static void clear_bootstrap(void)
 
 void arch_init_p2m(unsigned long max_pfn)
 {
-#ifdef __x86_64__
-#define L1_P2M_SHIFT    9
-#define L2_P2M_SHIFT    18    
-#define L3_P2M_SHIFT    27    
-#else
-#define L1_P2M_SHIFT    10
-#define L2_P2M_SHIFT    20    
-#define L3_P2M_SHIFT    30    
-#endif
-#define L1_P2M_ENTRIES  (1 << L1_P2M_SHIFT)    
-#define L2_P2M_ENTRIES  (1 << (L2_P2M_SHIFT - L1_P2M_SHIFT))    
-#define L3_P2M_ENTRIES  (1 << (L3_P2M_SHIFT - L2_P2M_SHIFT))    
-#define L1_P2M_MASK     (L1_P2M_ENTRIES - 1)    
-#define L2_P2M_MASK     (L2_P2M_ENTRIES - 1)    
-#define L3_P2M_MASK     (L3_P2M_ENTRIES - 1)    
-    
     unsigned long *l2_list = NULL, *l3_list;
     unsigned long pfn;
     
+    p2m_chk_pfn(max_pfn - 1);
     l3_list = (unsigned long *)alloc_page(); 
-    for ( pfn = 0; pfn < max_pfn; pfn += L1_P2M_ENTRIES )
+    for ( pfn = 0; pfn < max_pfn; pfn += P2M_ENTRIES )
     {
-        if ( !(pfn % (L1_P2M_ENTRIES * L2_P2M_ENTRIES)) )
+        if ( !(pfn % (P2M_ENTRIES * P2M_ENTRIES)) )
         {
             l2_list = (unsigned long*)alloc_page();
-            if ( (pfn >> L3_P2M_SHIFT) > 0 )
-            {
-                printk("Error: Too many pfns.\n");
-                do_exit();
-            }
-            l3_list[(pfn >> L2_P2M_SHIFT)] = virt_to_mfn(l2_list);  
+            l3_list[L3_P2M_IDX(pfn)] = virt_to_mfn(l2_list);
         }
-        l2_list[(pfn >> L1_P2M_SHIFT) & L2_P2M_MASK] =
-            virt_to_mfn(phys_to_machine_mapping + pfn);
+        l2_list[L2_P2M_IDX(pfn)] = virt_to_mfn(phys_to_machine_mapping + pfn);
     }
     HYPERVISOR_shared_info->arch.pfn_to_mfn_frame_list_list = 
         virt_to_mfn(l3_list);
diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index d87fe55..7283f64 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -176,7 +176,28 @@ typedef unsigned long pgentry_t;
 #define IO_PROT_NOCACHE (L1_PROT | _PAGE_PCD)
 
 /* for P2M */
+#ifdef __x86_64__
+#define P2M_SHIFT       9
+#else
+#define P2M_SHIFT       10
+#endif
+#define P2M_ENTRIES     (1UL << P2M_SHIFT)
+#define P2M_MASK        (P2M_ENTRIES - 1)
+#define L1_P2M_SHIFT    P2M_SHIFT
+#define L2_P2M_SHIFT    (2 * P2M_SHIFT)
+#define L3_P2M_SHIFT    (3 * P2M_SHIFT)
+#define L1_P2M_IDX(pfn) ((pfn) & P2M_MASK)
+#define L2_P2M_IDX(pfn) (((pfn) >> L1_P2M_SHIFT) & P2M_MASK)
+#define L3_P2M_IDX(pfn) (((pfn) >> L2_P2M_SHIFT) & P2M_MASK)
 #define INVALID_P2M_ENTRY (~0UL)
+static inline void p2m_chk_pfn(unsigned long pfn)
+{
+    if ( (pfn >> L3_P2M_SHIFT) > 0 )
+    {
+        printk("Error: Too many pfns.\n");
+        do_exit();
+    }
+}
 
 #include "arch_limits.h"
 #define PAGE_SIZE       __PAGE_SIZE
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 15/19] mini-os: remap p2m list in case of ballooning

[Juergen Gross]

In case of enabled ballooning we must be prepared for a growing p2m
list. If the maximum memory size of the domain can't be covered by the
actual p2m list remap it to the kernel virtual mapping area and leave
enough space at the end.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
V3: add assertion as requested by Samuel Thibault
---
 arch/arm/balloon.c    |  2 ++
 arch/x86/balloon.c    | 25 +++++++++++++++++++++++++
 arch/x86/mm.c         |  3 +++
 include/balloon.h     |  3 +++
 include/x86/arch_mm.h |  4 ++++
 5 files changed, 37 insertions(+)

diff --git a/arch/arm/balloon.c b/arch/arm/balloon.c
index 28021d6..549e51b 100644
--- a/arch/arm/balloon.c
+++ b/arch/arm/balloon.c
@@ -25,4 +25,6 @@
 
 #ifdef CONFIG_BALLOON
 
+unsigned long virt_kernel_area_end;   /* TODO: find a virtual area */
+
 #endif
diff --git a/arch/x86/balloon.c b/arch/x86/balloon.c
index 28021d6..a7f20e4 100644
--- a/arch/x86/balloon.c
+++ b/arch/x86/balloon.c
@@ -21,8 +21,33 @@
  * DEALINGS IN THE SOFTWARE.
  */
 
+#include <mini-os/os.h>
 #include <mini-os/balloon.h>
+#include <mini-os/lib.h>
+#include <mini-os/mm.h>
 
 #ifdef CONFIG_BALLOON
 
+unsigned long virt_kernel_area_end = VIRT_KERNEL_AREA;
+
+void arch_remap_p2m(unsigned long max_pfn)
+{
+    unsigned long pfn;
+
+    if ( p2m_pages(nr_max_pages) <= p2m_pages(max_pfn) )
+        return;
+
+    for ( pfn = 0; pfn < max_pfn; pfn += P2M_ENTRIES )
+    {
+        map_frame_rw(virt_kernel_area_end + PAGE_SIZE * (pfn / P2M_ENTRIES),
+                     virt_to_mfn(phys_to_machine_mapping + pfn));
+    }
+
+    phys_to_machine_mapping = (unsigned long *)virt_kernel_area_end;
+    printk("remapped p2m list to %p\n", phys_to_machine_mapping);
+
+    virt_kernel_area_end += PAGE_SIZE * p2m_pages(nr_max_pages);
+    ASSERT(virt_kernel_area_end <= VIRT_DEMAND_AREA);
+}
+
 #endif
diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index a5c8959..8fa3b4c 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -37,6 +37,7 @@
 #include <mini-os/errno.h>
 #include <mini-os/os.h>
 #include <mini-os/hypervisor.h>
+#include <mini-os/balloon.h>
 #include <mini-os/mm.h>
 #include <mini-os/types.h>
 #include <mini-os/lib.h>
@@ -626,6 +627,8 @@ void arch_init_p2m(unsigned long max_pfn)
     HYPERVISOR_shared_info->arch.pfn_to_mfn_frame_list_list = 
         virt_to_mfn(l3_list);
     HYPERVISOR_shared_info->arch.max_pfn = max_pfn;
+
+    arch_remap_p2m(max_pfn);
 }
 
 void arch_init_mm(unsigned long* start_pfn_p, unsigned long* max_pfn_p)
diff --git a/include/balloon.h b/include/balloon.h
index cd79017..b0d0ebf 100644
--- a/include/balloon.h
+++ b/include/balloon.h
@@ -27,12 +27,15 @@
 #ifdef CONFIG_BALLOON
 
 extern unsigned long nr_max_pages;
+extern unsigned long virt_kernel_area_end;
 
 void get_max_pages(void);
+void arch_remap_p2m(unsigned long max_pfn);
 
 #else /* CONFIG_BALLOON */
 
 static inline void get_max_pages(void) { }
+static inline void arch_remap_p2m(unsigned long max_pfn) { }
 
 #endif /* CONFIG_BALLOON */
 #endif /* _BALLOON_H_ */
diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index 7283f64..e5d9c57 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -198,6 +198,10 @@ static inline void p2m_chk_pfn(unsigned long pfn)
         do_exit();
     }
 }
+static inline unsigned long p2m_pages(unsigned long pages)
+{
+    return (pages + P2M_ENTRIES - 1) >> L1_P2M_SHIFT;
+}
 
 #include "arch_limits.h"
 #define PAGE_SIZE       __PAGE_SIZE
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 16/19] mini-os: map page allocator's bitmap to virtual kernel area for ballooning

[Juergen Gross]

In case of CONFIG_BALLOON the page allocator's bitmap needs some space
to be able to grow. Remap it to kernel virtual area if the preallocated
area isn't large enough.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
V3: - add assertion as requested by Samuel Thibault
    - rename functions to have mm_ prefix as requested by Samuel Thibault
---
 balloon.c         | 18 ++++++++++++++++++
 include/balloon.h |  2 ++
 include/mm.h      |  6 ++++++
 mm.c              | 43 ++++++++++++++++++++++---------------------
 4 files changed, 48 insertions(+), 21 deletions(-)

diff --git a/balloon.c b/balloon.c
index 1ec113d..78b30af 100644
--- a/balloon.c
+++ b/balloon.c
@@ -44,3 +44,21 @@ void get_max_pages(void)
     nr_max_pages = ret;
     printk("Maximum memory size: %ld pages\n", nr_max_pages);
 }
+
+void mm_alloc_bitmap_remap(void)
+{
+    unsigned long i;
+
+    if ( mm_bitmap_size >= ((nr_max_pages + 1) >> (PAGE_SHIFT + 3)) )
+        return;
+
+    for ( i = 0; i < mm_bitmap_size; i += PAGE_SIZE )
+    {
+        map_frame_rw(virt_kernel_area_end + i,
+                     virt_to_mfn((unsigned long)(mm_bitmap) + i));
+    }
+
+    mm_bitmap = (unsigned long *)virt_kernel_area_end;
+    virt_kernel_area_end += round_pgup((nr_max_pages + 1) >> (PAGE_SHIFT + 3));
+    ASSERT(virt_kernel_area_end <= VIRT_DEMAND_AREA);
+}
diff --git a/include/balloon.h b/include/balloon.h
index b0d0ebf..9154f44 100644
--- a/include/balloon.h
+++ b/include/balloon.h
@@ -31,11 +31,13 @@ extern unsigned long virt_kernel_area_end;
 
 void get_max_pages(void);
 void arch_remap_p2m(unsigned long max_pfn);
+void mm_alloc_bitmap_remap(void);
 
 #else /* CONFIG_BALLOON */
 
 static inline void get_max_pages(void) { }
 static inline void arch_remap_p2m(unsigned long max_pfn) { }
+static inline void mm_alloc_bitmap_remap(void) { }
 
 #endif /* CONFIG_BALLOON */
 #endif /* _BALLOON_H_ */
diff --git a/include/mm.h b/include/mm.h
index 6add683..ab56445 100644
--- a/include/mm.h
+++ b/include/mm.h
@@ -42,8 +42,14 @@
 #define STACK_SIZE_PAGE_ORDER __STACK_SIZE_PAGE_ORDER
 #define STACK_SIZE __STACK_SIZE
 
+#define round_pgdown(_p)  ((_p) & PAGE_MASK)
+#define round_pgup(_p)    (((_p) + (PAGE_SIZE - 1)) & PAGE_MASK)
+
 extern unsigned long nr_free_pages;
 
+extern unsigned long *mm_bitmap;
+extern unsigned long mm_bitmap_size;
+
 void init_mm(void);
 unsigned long alloc_pages(int order);
 #define alloc_page()    alloc_pages(0)
diff --git a/mm.c b/mm.c
index 707a3e0..e2f55af 100644
--- a/mm.c
+++ b/mm.c
@@ -48,11 +48,13 @@
  *  One bit per page of memory. Bit set => page is allocated.
  */
 
-static unsigned long *alloc_bitmap;
+unsigned long *mm_bitmap;
+unsigned long mm_bitmap_size;
+
 #define PAGES_PER_MAPWORD (sizeof(unsigned long) * 8)
 
 #define allocated_in_map(_pn) \
-(alloc_bitmap[(_pn)/PAGES_PER_MAPWORD] & (1UL<<((_pn)&(PAGES_PER_MAPWORD-1))))
+(mm_bitmap[(_pn)/PAGES_PER_MAPWORD] & (1UL<<((_pn)&(PAGES_PER_MAPWORD-1))))
 
 unsigned long nr_free_pages;
 
@@ -61,8 +63,8 @@ unsigned long nr_free_pages;
  *  -(1<<n)  sets all bits >= n. 
  *  (1<<n)-1 sets all bits <  n.
  * Variable names in map_{alloc,free}:
- *  *_idx == Index into `alloc_bitmap' array.
- *  *_off == Bit offset within an element of the `alloc_bitmap' array.
+ *  *_idx == Index into `mm_bitmap' array.
+ *  *_off == Bit offset within an element of the `mm_bitmap' array.
  */
 
 static void map_alloc(unsigned long first_page, unsigned long nr_pages)
@@ -76,13 +78,13 @@ static void map_alloc(unsigned long first_page, unsigned long nr_pages)
 
     if ( curr_idx == end_idx )
     {
-        alloc_bitmap[curr_idx] |= ((1UL<<end_off)-1) & -(1UL<<start_off);
+        mm_bitmap[curr_idx] |= ((1UL<<end_off)-1) & -(1UL<<start_off);
     }
     else 
     {
-        alloc_bitmap[curr_idx] |= -(1UL<<start_off);
-        while ( ++curr_idx < end_idx ) alloc_bitmap[curr_idx] = ~0UL;
-        alloc_bitmap[curr_idx] |= (1UL<<end_off)-1;
+        mm_bitmap[curr_idx] |= -(1UL<<start_off);
+        while ( ++curr_idx < end_idx ) mm_bitmap[curr_idx] = ~0UL;
+        mm_bitmap[curr_idx] |= (1UL<<end_off)-1;
     }
 
     nr_free_pages -= nr_pages;
@@ -102,13 +104,13 @@ static void map_free(unsigned long first_page, unsigned long nr_pages)
 
     if ( curr_idx == end_idx )
     {
-        alloc_bitmap[curr_idx] &= -(1UL<<end_off) | ((1UL<<start_off)-1);
+        mm_bitmap[curr_idx] &= -(1UL<<end_off) | ((1UL<<start_off)-1);
     }
     else 
     {
-        alloc_bitmap[curr_idx] &= (1UL<<start_off)-1;
-        while ( ++curr_idx != end_idx ) alloc_bitmap[curr_idx] = 0;
-        alloc_bitmap[curr_idx] &= -(1UL<<end_off);
+        mm_bitmap[curr_idx] &= (1UL<<start_off)-1;
+        while ( ++curr_idx != end_idx ) mm_bitmap[curr_idx] = 0;
+        mm_bitmap[curr_idx] &= -(1UL<<end_off);
     }
 }
 
@@ -137,9 +139,6 @@ static chunk_head_t *free_head[FREELIST_SIZE];
 static chunk_head_t  free_tail[FREELIST_SIZE];
 #define FREELIST_EMPTY(_l) ((_l)->next == NULL)
 
-#define round_pgdown(_p)  ((_p)&PAGE_MASK)
-#define round_pgup(_p)    (((_p)+(PAGE_SIZE-1))&PAGE_MASK)
-
 /*
  * Initialise allocator, placing addresses [@min,@max] in free pool.
  * @min and @max are PHYSICAL addresses.
@@ -147,7 +146,7 @@ static chunk_head_t  free_tail[FREELIST_SIZE];
 static void init_page_allocator(unsigned long min, unsigned long max)
 {
     int i;
-    unsigned long range, bitmap_size;
+    unsigned long range;
     chunk_head_t *ch;
     chunk_tail_t *ct;
     for ( i = 0; i < FREELIST_SIZE; i++ )
@@ -161,14 +160,14 @@ static void init_page_allocator(unsigned long min, unsigned long max)
     max = round_pgdown(max);
 
     /* Allocate space for the allocation bitmap. */
-    bitmap_size  = (max+1) >> (PAGE_SHIFT+3);
-    bitmap_size  = round_pgup(bitmap_size);
-    alloc_bitmap = (unsigned long *)to_virt(min);
-    min         += bitmap_size;
+    mm_bitmap_size  = (max + 1) >> (PAGE_SHIFT + 3);
+    mm_bitmap_size  = round_pgup(mm_bitmap_size);
+    mm_bitmap = (unsigned long *)to_virt(min);
+    min         += mm_bitmap_size;
     range        = max - min;
 
     /* All allocated by default. */
-    memset(alloc_bitmap, ~0, bitmap_size);
+    memset(mm_bitmap, ~0, mm_bitmap_size);
     /* Free up the memory we've been given to play with. */
     map_free(PHYS_PFN(min), range>>PAGE_SHIFT);
 
@@ -198,6 +197,8 @@ static void init_page_allocator(unsigned long min, unsigned long max)
         free_head[i]    = ch;
         ct->level       = i;
     }
+
+    mm_alloc_bitmap_remap();
 }
 
 
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 16/19] mini-os: map page allocator's bitmap to virtual kernel area for ballooning

[Samuel Thibault]

Juergen Gross, on Thu 11 Aug 2016 11:18:19 +0200, wrote:
> +extern unsigned long *mm_bitmap;
> +extern unsigned long mm_bitmap_size;

Ah I was thinking to have these use mm_malloc_ too. "mm_bitmap" seems
short for namespace pollution.

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 16/19] mini-os: map page allocator's bitmap to virtual kernel area for ballooning

[Juergen Gross]

On 11/08/16 11:40, Samuel Thibault wrote:
> Juergen Gross, on Thu 11 Aug 2016 11:18:19 +0200, wrote:
>> +extern unsigned long *mm_bitmap;
>> +extern unsigned long mm_bitmap_size;
> 
> Ah I was thinking to have these use mm_malloc_ too. "mm_bitmap" seems
> short for namespace pollution.

Sorry, you wrote:

> Ditto, mm_bitmap and mm_bitmap_size.

So I thought you wanted it to be that way.

I can easily change it again.

Wei, do I need to send all patches again, or is it enough to start with
patch 16? Patches 1-15 are already Acked/Reviewed by Samuel.


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 16/19] mini-os: map page allocator's bitmap to virtual kernel area for ballooning

[Wei Liu]

On Thu, Aug 11, 2016 at 12:19:20PM +0200, Juergen Gross wrote:
> On 11/08/16 11:40, Samuel Thibault wrote:
> > Juergen Gross, on Thu 11 Aug 2016 11:18:19 +0200, wrote:
> >> +extern unsigned long *mm_bitmap;
> >> +extern unsigned long mm_bitmap_size;
> > 
> > Ah I was thinking to have these use mm_malloc_ too. "mm_bitmap" seems
> > short for namespace pollution.
> 
> Sorry, you wrote:
> 
> > Ditto, mm_bitmap and mm_bitmap_size.
> 
> So I thought you wanted it to be that way.
> 
> I can easily change it again.
> 
> Wei, do I need to send all patches again, or is it enough to start with
> patch 16? Patches 1-15 are already Acked/Reviewed by Samuel.
> 
> 

No need to resend all of them. Resending this one is ok.

I would in fact even prefer a git branch that I can pull and commit
directly.

Wei.

> Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 16/19] mini-os: map page allocator's bitmap to virtual kernel area for ballooning

[Juergen Gross]

On 11/08/16 12:21, Wei Liu wrote:
> On Thu, Aug 11, 2016 at 12:19:20PM +0200, Juergen Gross wrote:
>> On 11/08/16 11:40, Samuel Thibault wrote:
>>> Juergen Gross, on Thu 11 Aug 2016 11:18:19 +0200, wrote:
>>>> +extern unsigned long *mm_bitmap;
>>>> +extern unsigned long mm_bitmap_size;
>>>
>>> Ah I was thinking to have these use mm_malloc_ too. "mm_bitmap" seems
>>> short for namespace pollution.
>>
>> Sorry, you wrote:
>>
>>> Ditto, mm_bitmap and mm_bitmap_size.
>>
>> So I thought you wanted it to be that way.
>>
>> I can easily change it again.
>>
>> Wei, do I need to send all patches again, or is it enough to start with
>> patch 16? Patches 1-15 are already Acked/Reviewed by Samuel.
>>
>>
> 
> No need to resend all of them. Resending this one is ok.

I think I'll need to modify at least one other patch due to this change.
So I'll resend all modified patches.

> 
> I would in fact even prefer a git branch that I can pull and commit
> directly.

NP.


Juergen


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 16/19] mini-os: map page allocator's bitmap to virtual kernel area for ballooning

[Samuel Thibault]

Juergen Gross, on Thu 11 Aug 2016 12:19:20 +0200, wrote:
> On 11/08/16 11:40, Samuel Thibault wrote:
> > Juergen Gross, on Thu 11 Aug 2016 11:18:19 +0200, wrote:
> >> +extern unsigned long *mm_bitmap;
> >> +extern unsigned long mm_bitmap_size;
> > 
> > Ah I was thinking to have these use mm_malloc_ too. "mm_bitmap" seems
> > short for namespace pollution.
> 
> Sorry, you wrote:
> 
> > Ditto, mm_bitmap and mm_bitmap_size.
> 
> So I thought you wanted it to be that way.

Oops, sorry about my typo.

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 17/19] mini-os: add support for ballooning up

[Juergen Gross]

Add support for ballooning the domain up by a specified amount of
pages. Following steps are performed:

- extending the p2m map
- extending the page allocator's bitmap
- getting new memory pages from the hypervisor
- adding the memory at the current end of guest memory

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
V3: change "if" to "while" in balloon_up() as requested by Samuel Thibault
---
 arch/arm/balloon.c |  9 ++++++
 arch/x86/balloon.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 balloon.c          | 64 +++++++++++++++++++++++++++++++++++++
 include/balloon.h  |  5 +++
 mm.c               |  4 +++
 5 files changed, 176 insertions(+)

diff --git a/arch/arm/balloon.c b/arch/arm/balloon.c
index 549e51b..7f35328 100644
--- a/arch/arm/balloon.c
+++ b/arch/arm/balloon.c
@@ -27,4 +27,13 @@
 
 unsigned long virt_kernel_area_end;   /* TODO: find a virtual area */
 
+int arch_expand_p2m(unsigned long max_pfn)
+{
+    return 0;
+}
+
+void arch_pfn_add(unsigned long pfn, unsigned long mfn)
+{
+}
+
 #endif
diff --git a/arch/x86/balloon.c b/arch/x86/balloon.c
index a7f20e4..42389e4 100644
--- a/arch/x86/balloon.c
+++ b/arch/x86/balloon.c
@@ -23,6 +23,7 @@
 
 #include <mini-os/os.h>
 #include <mini-os/balloon.h>
+#include <mini-os/errno.h>
 #include <mini-os/lib.h>
 #include <mini-os/mm.h>
 
@@ -30,9 +31,36 @@
 
 unsigned long virt_kernel_area_end = VIRT_KERNEL_AREA;
 
+static void p2m_invalidate(unsigned long *list, unsigned long start_idx)
+{
+    unsigned long idx;
+
+    for ( idx = start_idx; idx < P2M_ENTRIES; idx++ )
+        list[idx] = INVALID_P2M_ENTRY;
+}
+
+static inline unsigned long *p2m_l3list(void)
+{
+    return mfn_to_virt(HYPERVISOR_shared_info->arch.pfn_to_mfn_frame_list_list);
+}
+
+static inline unsigned long *p2m_to_virt(unsigned long p2m)
+{
+    return ( p2m == INVALID_P2M_ENTRY ) ? NULL : mfn_to_virt(p2m);
+}
+
 void arch_remap_p2m(unsigned long max_pfn)
 {
     unsigned long pfn;
+    unsigned long *l3_list, *l2_list, *l1_list;
+
+    l3_list = p2m_l3list();
+    l2_list = p2m_to_virt(l3_list[L3_P2M_IDX(max_pfn - 1)]);
+    l1_list = p2m_to_virt(l2_list[L2_P2M_IDX(max_pfn - 1)]);
+
+    p2m_invalidate(l3_list, L3_P2M_IDX(max_pfn - 1) + 1);
+    p2m_invalidate(l2_list, L2_P2M_IDX(max_pfn - 1) + 1);
+    p2m_invalidate(l1_list, L1_P2M_IDX(max_pfn - 1) + 1);
 
     if ( p2m_pages(nr_max_pages) <= p2m_pages(max_pfn) )
         return;
@@ -50,4 +78,70 @@ void arch_remap_p2m(unsigned long max_pfn)
     ASSERT(virt_kernel_area_end <= VIRT_DEMAND_AREA);
 }
 
+int arch_expand_p2m(unsigned long max_pfn)
+{
+    unsigned long pfn;
+    unsigned long *l1_list, *l2_list, *l3_list;
+
+    p2m_chk_pfn(max_pfn - 1);
+    l3_list = p2m_l3list();
+
+    for ( pfn = (HYPERVISOR_shared_info->arch.max_pfn + P2M_MASK) & ~P2M_MASK;
+          pfn < max_pfn; pfn += P2M_ENTRIES )
+    {
+        l2_list = p2m_to_virt(l3_list[L3_P2M_IDX(pfn)]);
+        if ( !l2_list )
+        {
+            l2_list = (unsigned long*)alloc_page();
+            if ( !l2_list )
+                return -ENOMEM;
+            p2m_invalidate(l2_list, 0);
+            l3_list[L3_P2M_IDX(pfn)] = virt_to_mfn(l2_list);
+        }
+        l1_list = p2m_to_virt(l2_list[L2_P2M_IDX(pfn)]);
+        if ( !l1_list )
+        {
+            l1_list = (unsigned long*)alloc_page();
+            if ( !l1_list )
+                return -ENOMEM;
+            p2m_invalidate(l1_list, 0);
+            l2_list[L2_P2M_IDX(pfn)] = virt_to_mfn(l1_list);
+
+            if ( map_frame_rw((unsigned long)(phys_to_machine_mapping + pfn),
+                              l2_list[L2_P2M_IDX(pfn)]) )
+                return -ENOMEM;
+        }
+    }
+
+    HYPERVISOR_shared_info->arch.max_pfn = max_pfn;
+
+    /* Make sure the new last page can be mapped. */
+    if ( !need_pgt((unsigned long)pfn_to_virt(max_pfn - 1)) )
+        return -ENOMEM;
+
+    return 0;
+}
+
+void arch_pfn_add(unsigned long pfn, unsigned long mfn)
+{
+    mmu_update_t mmu_updates[1];
+    pgentry_t *pgt;
+    int rc;
+
+    phys_to_machine_mapping[pfn] = mfn;
+
+    pgt = need_pgt((unsigned long)pfn_to_virt(pfn));
+    ASSERT(pgt);
+    mmu_updates[0].ptr = virt_to_mach(pgt) | MMU_NORMAL_PT_UPDATE;
+    mmu_updates[0].val = (pgentry_t)(mfn << PAGE_SHIFT) |
+                         _PAGE_PRESENT | _PAGE_RW;
+    rc = HYPERVISOR_mmu_update(mmu_updates, 1, NULL, DOMID_SELF);
+    if ( rc < 0 )
+    {
+        printk("ERROR: build_pagetable(): PTE could not be updated\n");
+        printk("       mmu_update failed with rc=%d\n", rc);
+        do_exit();
+    }
+}
+
 #endif
diff --git a/balloon.c b/balloon.c
index 78b30af..07ef532 100644
--- a/balloon.c
+++ b/balloon.c
@@ -23,11 +23,13 @@
 
 #include <mini-os/os.h>
 #include <mini-os/balloon.h>
+#include <mini-os/errno.h>
 #include <mini-os/lib.h>
 #include <xen/xen.h>
 #include <xen/memory.h>
 
 unsigned long nr_max_pages;
+unsigned long nr_mem_pages;
 
 void get_max_pages(void)
 {
@@ -62,3 +64,65 @@ void mm_alloc_bitmap_remap(void)
     virt_kernel_area_end += round_pgup((nr_max_pages + 1) >> (PAGE_SHIFT + 3));
     ASSERT(virt_kernel_area_end <= VIRT_DEMAND_AREA);
 }
+
+#define N_BALLOON_FRAMES 64
+static unsigned long balloon_frames[N_BALLOON_FRAMES];
+
+int balloon_up(unsigned long n_pages)
+{
+    unsigned long page, pfn;
+    int rc;
+    struct xen_memory_reservation reservation = {
+        .address_bits = 0,
+        .extent_order = 0,
+        .domid        = DOMID_SELF
+    };
+
+    if ( n_pages > nr_max_pages - nr_mem_pages )
+        n_pages = nr_max_pages - nr_mem_pages;
+    if ( n_pages > N_BALLOON_FRAMES )
+        n_pages = N_BALLOON_FRAMES;
+
+    /* Resize alloc_bitmap if necessary. */
+    while ( mm_bitmap_size * 8 < nr_mem_pages + n_pages )
+    {
+        page = alloc_page();
+        if ( !page )
+            return -ENOMEM;
+
+        memset((void *)page, ~0, PAGE_SIZE);
+        if ( map_frame_rw((unsigned long)mm_bitmap + mm_bitmap_size,
+                          virt_to_mfn(page)) )
+        {
+            free_page((void *)page);
+            return -ENOMEM;
+        }
+
+        mm_bitmap_size += PAGE_SIZE;
+    }
+
+    rc = arch_expand_p2m(nr_mem_pages + n_pages);
+    if ( rc )
+        return rc;
+
+    /* Get new memory from hypervisor. */
+    for ( pfn = 0; pfn < n_pages; pfn++ )
+    {
+        balloon_frames[pfn] = nr_mem_pages + pfn;
+    }
+    set_xen_guest_handle(reservation.extent_start, balloon_frames);
+    reservation.nr_extents = n_pages;
+    rc = HYPERVISOR_memory_op(XENMEM_populate_physmap, &reservation);
+    if ( rc <= 0 )
+        return rc;
+
+    for ( pfn = 0; pfn < rc; pfn++ )
+    {
+        arch_pfn_add(nr_mem_pages + pfn, balloon_frames[pfn]);
+        free_page(pfn_to_virt(nr_mem_pages + pfn));
+    }
+
+    nr_mem_pages += rc;
+
+    return rc;
+}
diff --git a/include/balloon.h b/include/balloon.h
index 9154f44..5ec1bbb 100644
--- a/include/balloon.h
+++ b/include/balloon.h
@@ -28,10 +28,15 @@
 
 extern unsigned long nr_max_pages;
 extern unsigned long virt_kernel_area_end;
+extern unsigned long nr_mem_pages;
 
 void get_max_pages(void);
+int balloon_up(unsigned long n_pages);
+
 void arch_remap_p2m(unsigned long max_pfn);
 void mm_alloc_bitmap_remap(void);
+int arch_expand_p2m(unsigned long max_pfn);
+void arch_pfn_add(unsigned long pfn, unsigned long mfn);
 
 #else /* CONFIG_BALLOON */
 
diff --git a/mm.c b/mm.c
index e2f55af..5364079 100644
--- a/mm.c
+++ b/mm.c
@@ -382,6 +382,10 @@ void init_mm(void)
     arch_init_p2m(max_pfn);
     
     arch_init_demand_mapping_area();
+
+#ifdef CONFIG_BALLOON
+    nr_mem_pages = max_pfn;
+#endif
 }
 
 void fini_mm(void)
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 18/19] mini-os: balloon up in case of oom

[Juergen Gross]

If a memory shortage is detected balloon up.

Be careful to always leave some pages free as ballooning up might need
some memory, too:

- new p2m frames
- page tables for addressing new p2m frame
- new frame for page allocation bitmap
- page table for addressing new page allocation bitmap frame
- page tables for addressing new 1:1 mapped frames

For the moment we only balloon up synchronously when memory shortage
is detected in allocation routines with irqs on.

Signed-off-by: Juergen Gross <jgross@suse.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
V3: Reverse chk_free_pages() return value as requested by Samuel Thibault
---
 balloon.c         | 32 ++++++++++++++++++++++++++++++++
 include/balloon.h | 11 +++++++++++
 mm.c              |  4 +++-
 3 files changed, 46 insertions(+), 1 deletion(-)

diff --git a/balloon.c b/balloon.c
index 07ef532..afec757 100644
--- a/balloon.c
+++ b/balloon.c
@@ -126,3 +126,35 @@ int balloon_up(unsigned long n_pages)
 
     return rc;
 }
+
+static int in_balloon;
+
+int chk_free_pages(unsigned long needed)
+{
+    unsigned long n_pages;
+
+    /* No need for ballooning if plenty of space available. */
+    if ( needed + BALLOON_EMERGENCY_PAGES <= nr_free_pages )
+        return 1;
+
+    /* If we are already ballooning up just hope for the best. */
+    if ( in_balloon )
+        return 1;
+
+    /* Interrupts disabled can't be handled right now. */
+    if ( irqs_disabled() )
+        return 1;
+
+    in_balloon = 1;
+
+    while ( needed + BALLOON_EMERGENCY_PAGES > nr_free_pages )
+    {
+        n_pages = needed + BALLOON_EMERGENCY_PAGES - nr_free_pages;
+        if ( !balloon_up(n_pages) )
+            break;
+    }
+
+    in_balloon = 0;
+
+    return needed <= nr_free_pages;
+}
diff --git a/include/balloon.h b/include/balloon.h
index 5ec1bbb..d8710ad 100644
--- a/include/balloon.h
+++ b/include/balloon.h
@@ -26,6 +26,12 @@
 
 #ifdef CONFIG_BALLOON
 
+/*
+ * Always keep some pages free for allocations while ballooning or
+ * interrupts disabled.
+ */
+#define BALLOON_EMERGENCY_PAGES   64
+
 extern unsigned long nr_max_pages;
 extern unsigned long virt_kernel_area_end;
 extern unsigned long nr_mem_pages;
@@ -37,12 +43,17 @@ void arch_remap_p2m(unsigned long max_pfn);
 void mm_alloc_bitmap_remap(void);
 int arch_expand_p2m(unsigned long max_pfn);
 void arch_pfn_add(unsigned long pfn, unsigned long mfn);
+int chk_free_pages(unsigned long needed);
 
 #else /* CONFIG_BALLOON */
 
 static inline void get_max_pages(void) { }
 static inline void arch_remap_p2m(unsigned long max_pfn) { }
 static inline void mm_alloc_bitmap_remap(void) { }
+static inline int chk_free_pages(unsigned long needed)
+{
+    return needed <= nr_free_pages;
+}
 
 #endif /* CONFIG_BALLOON */
 #endif /* _BALLOON_H_ */
diff --git a/mm.c b/mm.c
index 5364079..b1f8f34 100644
--- a/mm.c
+++ b/mm.c
@@ -209,6 +209,8 @@ unsigned long alloc_pages(int order)
     chunk_head_t *alloc_ch, *spare_ch;
     chunk_tail_t            *spare_ct;
 
+    if ( !chk_free_pages(1UL << order) )
+        goto no_memory;
 
     /* Find smallest order which can satisfy the request. */
     for ( i = order; i < FREELIST_SIZE; i++ ) {
@@ -343,7 +345,7 @@ void *sbrk(ptrdiff_t increment)
     if (new_brk > heap_mapped) {
         unsigned long n = (new_brk - heap_mapped + PAGE_SIZE - 1) / PAGE_SIZE;
 
-        if ( n > nr_free_pages )
+        if ( !chk_free_pages(n) )
         {
             printk("Memory exhausted: want %ld pages, but only %ld are left\n",
                    n, nr_free_pages);
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[PATCH v3 19/19] mini-os: repair build system

[Juergen Gross]

The build system of Mini-OS is using different settings for arch/*/*
than for the rest of the tree. The main reasons are that Config.mk is
included only conditionally in the top level Makefile, while minios.mk
isn't included by the arch Makefiles.

Repairing this mess enables us to move the CONFIG_* handling to
Config.mk enabling the arch sources to make use of those even if no
MINIOS_CONFIG was specified by the caller.

Most of the files under config were not used. Integrate the used ones
into Config.mk and delete the rest.

The CONFIG_* defines should be set for assembler sources, too.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 Config.mk         | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 Makefile          | 44 --------------------------
 arch/x86/Makefile |  3 --
 config/MiniOS.mk  | 10 ------
 config/StdGNU.mk  | 47 ----------------------------
 config/arm32.mk   | 22 -------------
 config/arm64.mk   | 19 ------------
 config/x86_32.mk  | 20 ------------
 config/x86_64.mk  | 33 --------------------
 minios.mk         |  4 +--
 10 files changed, 95 insertions(+), 200 deletions(-)
 delete mode 100644 config/MiniOS.mk
 delete mode 100644 config/StdGNU.mk
 delete mode 100644 config/arm32.mk
 delete mode 100644 config/arm64.mk
 delete mode 100644 config/x86_32.mk
 delete mode 100644 config/x86_64.mk

diff --git a/Config.mk b/Config.mk
index 9d19cd7..8ab1a7e 100644
--- a/Config.mk
+++ b/Config.mk
@@ -23,6 +23,11 @@ cc-option = $(shell if test -z "`echo 'void*p=1;' | \
               $(1) $(2) -S -o /dev/null -x c - 2>&1 | grep -- $(2) -`"; \
               then echo "$(2)"; else echo "$(3)"; fi ;)
 
+ifneq ($(MINIOS_CONFIG),)
+EXTRA_DEPS += $(MINIOS_CONFIG)
+include $(MINIOS_CONFIG)
+endif
+
 # Compatibility with Xen's stubdom build environment.  If we are building
 # stubdom, some XEN_ variables are set, set MINIOS_ variables accordingly.
 #
@@ -97,3 +102,91 @@ DEF_CPPFLAGS += -DHAVE_LWIP
 DEF_CPPFLAGS += -isystem $(LWIPDIR)/src/include
 DEF_CPPFLAGS += -isystem $(LWIPDIR)/src/include/ipv4
 endif
+
+# Set tools
+AS         = $(CROSS_COMPILE)as
+LD         = $(CROSS_COMPILE)ld
+ifeq ($(clang),y)
+CC         = $(CROSS_COMPILE)clang
+LD_LTO     = $(CROSS_COMPILE)llvm-ld
+else
+CC         = $(CROSS_COMPILE)gcc
+LD_LTO     = $(CROSS_COMPILE)ld
+endif
+CPP        = $(CC) -E
+AR         = $(CROSS_COMPILE)ar
+RANLIB     = $(CROSS_COMPILE)ranlib
+NM         = $(CROSS_COMPILE)nm
+STRIP      = $(CROSS_COMPILE)strip
+OBJCOPY    = $(CROSS_COMPILE)objcopy
+OBJDUMP    = $(CROSS_COMPILE)objdump
+SIZEUTIL   = $(CROSS_COMPILE)size
+
+# Allow git to be wrappered in the environment
+GIT        ?= git
+
+INSTALL      = install
+INSTALL_DIR  = $(INSTALL) -d -m0755 -p
+INSTALL_DATA = $(INSTALL) -m0644 -p
+INSTALL_PROG = $(INSTALL) -m0755 -p
+
+BOOT_DIR ?= /boot
+
+SOCKET_LIBS =
+UTIL_LIBS = -lutil
+DLOPEN_LIBS = -ldl
+
+SONAME_LDFLAG = -soname
+SHLIB_LDFLAGS = -shared
+
+ifneq ($(debug),y)
+CFLAGS += -O2 -fomit-frame-pointer
+else
+# Less than -O1 produces bad code and large stack frames
+CFLAGS += -O1 -fno-omit-frame-pointer
+CFLAGS-$(gcc) += -fno-optimize-sibling-calls
+endif
+
+ifeq ($(lto),y)
+CFLAGS += -flto
+LDFLAGS-$(clang) += -plugin LLVMgold.so
+endif
+
+# Configuration defaults
+CONFIG_START_NETWORK ?= y
+CONFIG_SPARSE_BSS ?= y
+CONFIG_QEMU_XS_ARGS ?= n
+CONFIG_TEST ?= n
+CONFIG_PCIFRONT ?= n
+CONFIG_BLKFRONT ?= y
+CONFIG_TPMFRONT ?= n
+CONFIG_TPM_TIS ?= n
+CONFIG_TPMBACK ?= n
+CONFIG_NETFRONT ?= y
+CONFIG_FBFRONT ?= y
+CONFIG_KBDFRONT ?= y
+CONFIG_CONSFRONT ?= y
+CONFIG_XENBUS ?= y
+CONFIG_XC ?=y
+CONFIG_LWIP ?= $(lwip)
+CONFIG_BALLOON ?= n
+
+# Export config items as compiler directives
+DEFINES-$(CONFIG_START_NETWORK) += -DCONFIG_START_NETWORK
+DEFINES-$(CONFIG_SPARSE_BSS) += -DCONFIG_SPARSE_BSS
+DEFINES-$(CONFIG_QEMU_XS_ARGS) += -DCONFIG_QEMU_XS_ARGS
+DEFINES-$(CONFIG_PCIFRONT) += -DCONFIG_PCIFRONT
+DEFINES-$(CONFIG_BLKFRONT) += -DCONFIG_BLKFRONT
+DEFINES-$(CONFIG_TPMFRONT) += -DCONFIG_TPMFRONT
+DEFINES-$(CONFIG_TPM_TIS) += -DCONFIG_TPM_TIS
+DEFINES-$(CONFIG_TPMBACK) += -DCONFIG_TPMBACK
+DEFINES-$(CONFIG_NETFRONT) += -DCONFIG_NETFRONT
+DEFINES-$(CONFIG_KBDFRONT) += -DCONFIG_KBDFRONT
+DEFINES-$(CONFIG_FBFRONT) += -DCONFIG_FBFRONT
+DEFINES-$(CONFIG_CONSFRONT) += -DCONFIG_CONSFRONT
+DEFINES-$(CONFIG_XENBUS) += -DCONFIG_XENBUS
+DEFINES-$(CONFIG_BALLOON) += -DCONFIG_BALLOON
+
+# Override settings for this OS
+PTHREAD_LIBS =
+nosharedlibs=y
diff --git a/Makefile b/Makefile
index f5b7011..5464e89 100644
--- a/Makefile
+++ b/Makefile
@@ -7,51 +7,7 @@
 OBJ_DIR=$(CURDIR)
 TOPLEVEL_DIR=$(CURDIR)
 
-ifeq ($(MINIOS_CONFIG),)
 include Config.mk
-else
-EXTRA_DEPS += $(MINIOS_CONFIG)
-include $(MINIOS_CONFIG)
-endif
-
-include $(MINIOS_ROOT)/config/MiniOS.mk
-
-# Configuration defaults
-CONFIG_START_NETWORK ?= y
-CONFIG_SPARSE_BSS ?= y
-CONFIG_QEMU_XS_ARGS ?= n
-CONFIG_TEST ?= n
-CONFIG_PCIFRONT ?= n
-CONFIG_BLKFRONT ?= y
-CONFIG_TPMFRONT ?= n
-CONFIG_TPM_TIS ?= n
-CONFIG_TPMBACK ?= n
-CONFIG_NETFRONT ?= y
-CONFIG_FBFRONT ?= y
-CONFIG_KBDFRONT ?= y
-CONFIG_CONSFRONT ?= y
-CONFIG_XENBUS ?= y
-CONFIG_XC ?=y
-CONFIG_LWIP ?= $(lwip)
-CONFIG_BALLOON ?= n
-
-# Export config items as compiler directives
-flags-$(CONFIG_START_NETWORK) += -DCONFIG_START_NETWORK
-flags-$(CONFIG_SPARSE_BSS) += -DCONFIG_SPARSE_BSS
-flags-$(CONFIG_QEMU_XS_ARGS) += -DCONFIG_QEMU_XS_ARGS
-flags-$(CONFIG_PCIFRONT) += -DCONFIG_PCIFRONT
-flags-$(CONFIG_BLKFRONT) += -DCONFIG_BLKFRONT
-flags-$(CONFIG_TPMFRONT) += -DCONFIG_TPMFRONT
-flags-$(CONFIG_TPM_TIS) += -DCONFIG_TPM_TIS
-flags-$(CONFIG_TPMBACK) += -DCONFIG_TPMBACK
-flags-$(CONFIG_NETFRONT) += -DCONFIG_NETFRONT
-flags-$(CONFIG_KBDFRONT) += -DCONFIG_KBDFRONT
-flags-$(CONFIG_FBFRONT) += -DCONFIG_FBFRONT
-flags-$(CONFIG_CONSFRONT) += -DCONFIG_CONSFRONT
-flags-$(CONFIG_XENBUS) += -DCONFIG_XENBUS
-flags-$(CONFIG_BALLOON) += -DCONFIG_BALLOON
-
-DEF_CFLAGS += $(flags-y)
 
 # Symlinks and headers that must be created before building the C files
 GENERATED_HEADERS := include/list.h $(ARCH_LINKS) include/mini-os include/$(TARGET_ARCH_FAM)/mini-os
diff --git a/arch/x86/Makefile b/arch/x86/Makefile
index a46766c..0052b4c 100644
--- a/arch/x86/Makefile
+++ b/arch/x86/Makefile
@@ -6,9 +6,6 @@
 TOPLEVEL_DIR = $(CURDIR)/../..
 include ../../Config.mk
 
-# include arch.mk has to be before mini-os.mk!
-
-include arch.mk
 include ../../minios.mk
 
 # Sources here are all *.c *.S without $(MINIOS_TARGET_ARCH).S
diff --git a/config/MiniOS.mk b/config/MiniOS.mk
deleted file mode 100644
index be542dc..0000000
--- a/config/MiniOS.mk
+++ /dev/null
@@ -1,10 +0,0 @@
-include $(MINIOS_ROOT)/config/StdGNU.mk
-include $(MINIOS_ROOT)/Config.mk
-CFLAGS += $(DEF_CFLAGS) $(ARCH_CFLAGS)
-CPPFLAGS += $(DEF_CPPFLAGS) $(ARCH_CPPFLAGS) $(extra_incl)
-ASFLAGS += $(DEF_ASFLAGS) $(ARCH_ASFLAGS)
-LDFLAGS += $(DEF_LDFLAGS) $(ARCH_LDFLAGS)
-
-# Override settings for this OS
-PTHREAD_LIBS =
-nosharedlibs=y
diff --git a/config/StdGNU.mk b/config/StdGNU.mk
deleted file mode 100644
index 4efebe3..0000000
--- a/config/StdGNU.mk
+++ /dev/null
@@ -1,47 +0,0 @@
-AS         = $(CROSS_COMPILE)as
-LD         = $(CROSS_COMPILE)ld
-ifeq ($(clang),y)
-CC         = $(CROSS_COMPILE)clang
-LD_LTO     = $(CROSS_COMPILE)llvm-ld
-else
-CC         = $(CROSS_COMPILE)gcc
-LD_LTO     = $(CROSS_COMPILE)ld
-endif
-CPP        = $(CC) -E
-AR         = $(CROSS_COMPILE)ar
-RANLIB     = $(CROSS_COMPILE)ranlib
-NM         = $(CROSS_COMPILE)nm
-STRIP      = $(CROSS_COMPILE)strip
-OBJCOPY    = $(CROSS_COMPILE)objcopy
-OBJDUMP    = $(CROSS_COMPILE)objdump
-SIZEUTIL   = $(CROSS_COMPILE)size
-
-# Allow git to be wrappered in the environment
-GIT        ?= git
-
-INSTALL      = install
-INSTALL_DIR  = $(INSTALL) -d -m0755 -p
-INSTALL_DATA = $(INSTALL) -m0644 -p
-INSTALL_PROG = $(INSTALL) -m0755 -p
-
-BOOT_DIR ?= /boot
-
-SOCKET_LIBS =
-UTIL_LIBS = -lutil
-DLOPEN_LIBS = -ldl
-
-SONAME_LDFLAG = -soname
-SHLIB_LDFLAGS = -shared
-
-ifneq ($(debug),y)
-CFLAGS += -O2 -fomit-frame-pointer
-else
-# Less than -O1 produces bad code and large stack frames
-CFLAGS += -O1 -fno-omit-frame-pointer
-CFLAGS-$(gcc) += -fno-optimize-sibling-calls
-endif
-
-ifeq ($(lto),y)
-CFLAGS += -flto
-LDFLAGS-$(clang) += -plugin LLVMgold.so
-endif
diff --git a/config/arm32.mk b/config/arm32.mk
deleted file mode 100644
index 4f83a63..0000000
--- a/config/arm32.mk
+++ /dev/null
@@ -1,22 +0,0 @@
-CONFIG_ARM := y
-CONFIG_ARM_32 := y
-CONFIG_ARM_$(XEN_OS) := y
-
-CONFIG_XEN_INSTALL_SUFFIX :=
-
-# -march= -mcpu=
-
-# Explicitly specifiy 32-bit ARM ISA since toolchain default can be -mthumb:
-CFLAGS += -marm
-
-HAS_PL011 := y
-HAS_EXYNOS4210 := y
-HAS_OMAP := y
-HAS_NS16550 := y
-
-# Use only if calling $(LD) directly.
-LDFLAGS_DIRECT += -EL
-
-CONFIG_LOAD_ADDRESS ?= 0x80000000
-
-IOEMU_CPU_ARCH ?= arm
diff --git a/config/arm64.mk b/config/arm64.mk
deleted file mode 100644
index 6eafda2..0000000
--- a/config/arm64.mk
+++ /dev/null
@@ -1,19 +0,0 @@
-CONFIG_ARM := y
-CONFIG_ARM_64 := y
-CONFIG_ARM_$(XEN_OS) := y
-
-CONFIG_XEN_INSTALL_SUFFIX :=
-
-CFLAGS += #-marm -march= -mcpu= etc
-
-HAS_PL011 := y
-HAS_NS16550 := y
-
-# Use only if calling $(LD) directly.
-LDFLAGS_DIRECT += -EL
-
-CONFIG_LOAD_ADDRESS ?= 0x80000000
-
-IOEMU_CPU_ARCH ?= aarch64
-
-EFI_DIR ?= /usr/lib64/efi
diff --git a/config/x86_32.mk b/config/x86_32.mk
deleted file mode 100644
index c2d785e..0000000
--- a/config/x86_32.mk
+++ /dev/null
@@ -1,20 +0,0 @@
-CONFIG_X86 := y
-CONFIG_X86_32 := y
-CONFIG_X86_$(XEN_OS) := y
-
-CONFIG_HVM := y
-CONFIG_MIGRATE := y
-CONFIG_XCUTILS := y
-
-HAS_MEM_ACCESS := y
-HAS_MEM_PAGING := y
-HAS_MEM_SHARING := y
-
-CFLAGS += -m32 -march=i686
-
-# Use only if calling $(LD) directly.
-LDFLAGS_DIRECT_OpenBSD = _obsd
-LDFLAGS_DIRECT_FreeBSD = _fbsd
-LDFLAGS_DIRECT += -melf_i386$(LDFLAGS_DIRECT_$(XEN_OS))
-
-IOEMU_CPU_ARCH ?= i386
diff --git a/config/x86_64.mk b/config/x86_64.mk
deleted file mode 100644
index bf104c2..0000000
--- a/config/x86_64.mk
+++ /dev/null
@@ -1,33 +0,0 @@
-CONFIG_X86 := y
-CONFIG_X86_64 := y
-CONFIG_X86_$(XEN_OS) := y
-
-CONFIG_COMPAT := y
-CONFIG_HVM := y
-CONFIG_MIGRATE := y
-CONFIG_XCUTILS := y
-
-HAS_MEM_ACCESS := y
-HAS_MEM_PAGING := y
-HAS_MEM_SHARING := y
-
-CONFIG_XEN_INSTALL_SUFFIX := .gz
-
-CFLAGS += -m64
-
-SunOS_LIBDIR = $(SunOS_LIBDIR_x86_64)
-
-EFI_DIR ?= /usr/lib64/efi
-
-# Use only if calling $(LD) directly.
-ifeq ($(XEN_OS),OpenBSD)
-LDFLAGS_DIRECT += -melf_x86_64_obsd
-else
-ifeq ($(XEN_OS),FreeBSD)
-LDFLAGS_DIRECT += -melf_x86_64_fbsd
-else
-LDFLAGS_DIRECT += -melf_x86_64
-endif
-endif
-
-IOEMU_CPU_ARCH ?= x86_64
diff --git a/minios.mk b/minios.mk
index 89534f7..1d1cc99 100644
--- a/minios.mk
+++ b/minios.mk
@@ -32,9 +32,9 @@ DEF_CFLAGS += -D__INSIDE_MINIOS__
 # Build the CFLAGS and ASFLAGS for compiling and assembling.
 # DEF_... flags are the common mini-os flags,
 # ARCH_... flags may be defined in arch/$(TARGET_ARCH_FAM/rules.mk
-CFLAGS := $(DEF_CFLAGS) $(ARCH_CFLAGS)
+CFLAGS := $(DEF_CFLAGS) $(ARCH_CFLAGS) $(DEFINES-y)
 CPPFLAGS := $(DEF_CPPFLAGS) $(ARCH_CPPFLAGS)
-ASFLAGS := $(DEF_ASFLAGS) $(ARCH_ASFLAGS)
+ASFLAGS := $(DEF_ASFLAGS) $(ARCH_ASFLAGS) $(DEFINES-y)
 LDFLAGS := $(DEF_LDFLAGS) $(ARCH_LDFLAGS)
 
 # Special build dependencies.
-- 
2.6.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 19/19] mini-os: repair build system

[Wei Liu]

On Thu, Aug 11, 2016 at 11:18:22AM +0200, Juergen Gross wrote:
> The build system of Mini-OS is using different settings for arch/*/*
> than for the rest of the tree. The main reasons are that Config.mk is
> included only conditionally in the top level Makefile, while minios.mk
> isn't included by the arch Makefiles.
> 
> Repairing this mess enables us to move the CONFIG_* handling to
> Config.mk enabling the arch sources to make use of those even if no
> MINIOS_CONFIG was specified by the caller.
> 
> Most of the files under config were not used. Integrate the used ones
> into Config.mk and delete the rest.
> 
> The CONFIG_* defines should be set for assembler sources, too.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Wei Liu <wei.liu2@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [PATCH v3 19/19] mini-os: repair build system

[Samuel Thibault]

Juergen Gross, on Thu 11 Aug 2016 11:18:22 +0200, wrote:
> The build system of Mini-OS is using different settings for arch/*/*
> than for the rest of the tree. The main reasons are that Config.mk is
> included only conditionally in the top level Makefile, while minios.mk
> isn't included by the arch Makefiles.
> 
> Repairing this mess enables us to move the CONFIG_* handling to
> Config.mk enabling the arch sources to make use of those even if no
> MINIOS_CONFIG was specified by the caller.
> 
> Most of the files under config were not used. Integrate the used ones
> into Config.mk and delete the rest.
> 
> The CONFIG_* defines should be set for assembler sources, too.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

> ---
>  Config.mk         | 93 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  Makefile          | 44 --------------------------
>  arch/x86/Makefile |  3 --
>  config/MiniOS.mk  | 10 ------
>  config/StdGNU.mk  | 47 ----------------------------
>  config/arm32.mk   | 22 -------------
>  config/arm64.mk   | 19 ------------
>  config/x86_32.mk  | 20 ------------
>  config/x86_64.mk  | 33 --------------------
>  minios.mk         |  4 +--
>  10 files changed, 95 insertions(+), 200 deletions(-)
>  delete mode 100644 config/MiniOS.mk
>  delete mode 100644 config/StdGNU.mk
>  delete mode 100644 config/arm32.mk
>  delete mode 100644 config/arm64.mk
>  delete mode 100644 config/x86_32.mk
>  delete mode 100644 config/x86_64.mk
> 
> diff --git a/Config.mk b/Config.mk
> index 9d19cd7..8ab1a7e 100644
> --- a/Config.mk
> +++ b/Config.mk
> @@ -23,6 +23,11 @@ cc-option = $(shell if test -z "`echo 'void*p=1;' | \
>                $(1) $(2) -S -o /dev/null -x c - 2>&1 | grep -- $(2) -`"; \
>                then echo "$(2)"; else echo "$(3)"; fi ;)
>  
> +ifneq ($(MINIOS_CONFIG),)
> +EXTRA_DEPS += $(MINIOS_CONFIG)
> +include $(MINIOS_CONFIG)
> +endif
> +
>  # Compatibility with Xen's stubdom build environment.  If we are building
>  # stubdom, some XEN_ variables are set, set MINIOS_ variables accordingly.
>  #
> @@ -97,3 +102,91 @@ DEF_CPPFLAGS += -DHAVE_LWIP
>  DEF_CPPFLAGS += -isystem $(LWIPDIR)/src/include
>  DEF_CPPFLAGS += -isystem $(LWIPDIR)/src/include/ipv4
>  endif
> +
> +# Set tools
> +AS         = $(CROSS_COMPILE)as
> +LD         = $(CROSS_COMPILE)ld
> +ifeq ($(clang),y)
> +CC         = $(CROSS_COMPILE)clang
> +LD_LTO     = $(CROSS_COMPILE)llvm-ld
> +else
> +CC         = $(CROSS_COMPILE)gcc
> +LD_LTO     = $(CROSS_COMPILE)ld
> +endif
> +CPP        = $(CC) -E
> +AR         = $(CROSS_COMPILE)ar
> +RANLIB     = $(CROSS_COMPILE)ranlib
> +NM         = $(CROSS_COMPILE)nm
> +STRIP      = $(CROSS_COMPILE)strip
> +OBJCOPY    = $(CROSS_COMPILE)objcopy
> +OBJDUMP    = $(CROSS_COMPILE)objdump
> +SIZEUTIL   = $(CROSS_COMPILE)size
> +
> +# Allow git to be wrappered in the environment
> +GIT        ?= git
> +
> +INSTALL      = install
> +INSTALL_DIR  = $(INSTALL) -d -m0755 -p
> +INSTALL_DATA = $(INSTALL) -m0644 -p
> +INSTALL_PROG = $(INSTALL) -m0755 -p
> +
> +BOOT_DIR ?= /boot
> +
> +SOCKET_LIBS =
> +UTIL_LIBS = -lutil
> +DLOPEN_LIBS = -ldl
> +
> +SONAME_LDFLAG = -soname
> +SHLIB_LDFLAGS = -shared
> +
> +ifneq ($(debug),y)
> +CFLAGS += -O2 -fomit-frame-pointer
> +else
> +# Less than -O1 produces bad code and large stack frames
> +CFLAGS += -O1 -fno-omit-frame-pointer
> +CFLAGS-$(gcc) += -fno-optimize-sibling-calls
> +endif
> +
> +ifeq ($(lto),y)
> +CFLAGS += -flto
> +LDFLAGS-$(clang) += -plugin LLVMgold.so
> +endif
> +
> +# Configuration defaults
> +CONFIG_START_NETWORK ?= y
> +CONFIG_SPARSE_BSS ?= y
> +CONFIG_QEMU_XS_ARGS ?= n
> +CONFIG_TEST ?= n
> +CONFIG_PCIFRONT ?= n
> +CONFIG_BLKFRONT ?= y
> +CONFIG_TPMFRONT ?= n
> +CONFIG_TPM_TIS ?= n
> +CONFIG_TPMBACK ?= n
> +CONFIG_NETFRONT ?= y
> +CONFIG_FBFRONT ?= y
> +CONFIG_KBDFRONT ?= y
> +CONFIG_CONSFRONT ?= y
> +CONFIG_XENBUS ?= y
> +CONFIG_XC ?=y
> +CONFIG_LWIP ?= $(lwip)
> +CONFIG_BALLOON ?= n
> +
> +# Export config items as compiler directives
> +DEFINES-$(CONFIG_START_NETWORK) += -DCONFIG_START_NETWORK
> +DEFINES-$(CONFIG_SPARSE_BSS) += -DCONFIG_SPARSE_BSS
> +DEFINES-$(CONFIG_QEMU_XS_ARGS) += -DCONFIG_QEMU_XS_ARGS
> +DEFINES-$(CONFIG_PCIFRONT) += -DCONFIG_PCIFRONT
> +DEFINES-$(CONFIG_BLKFRONT) += -DCONFIG_BLKFRONT
> +DEFINES-$(CONFIG_TPMFRONT) += -DCONFIG_TPMFRONT
> +DEFINES-$(CONFIG_TPM_TIS) += -DCONFIG_TPM_TIS
> +DEFINES-$(CONFIG_TPMBACK) += -DCONFIG_TPMBACK
> +DEFINES-$(CONFIG_NETFRONT) += -DCONFIG_NETFRONT
> +DEFINES-$(CONFIG_KBDFRONT) += -DCONFIG_KBDFRONT
> +DEFINES-$(CONFIG_FBFRONT) += -DCONFIG_FBFRONT
> +DEFINES-$(CONFIG_CONSFRONT) += -DCONFIG_CONSFRONT
> +DEFINES-$(CONFIG_XENBUS) += -DCONFIG_XENBUS
> +DEFINES-$(CONFIG_BALLOON) += -DCONFIG_BALLOON
> +
> +# Override settings for this OS
> +PTHREAD_LIBS =
> +nosharedlibs=y
> diff --git a/Makefile b/Makefile
> index f5b7011..5464e89 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -7,51 +7,7 @@
>  OBJ_DIR=$(CURDIR)
>  TOPLEVEL_DIR=$(CURDIR)
>  
> -ifeq ($(MINIOS_CONFIG),)
>  include Config.mk
> -else
> -EXTRA_DEPS += $(MINIOS_CONFIG)
> -include $(MINIOS_CONFIG)
> -endif
> -
> -include $(MINIOS_ROOT)/config/MiniOS.mk
> -
> -# Configuration defaults
> -CONFIG_START_NETWORK ?= y
> -CONFIG_SPARSE_BSS ?= y
> -CONFIG_QEMU_XS_ARGS ?= n
> -CONFIG_TEST ?= n
> -CONFIG_PCIFRONT ?= n
> -CONFIG_BLKFRONT ?= y
> -CONFIG_TPMFRONT ?= n
> -CONFIG_TPM_TIS ?= n
> -CONFIG_TPMBACK ?= n
> -CONFIG_NETFRONT ?= y
> -CONFIG_FBFRONT ?= y
> -CONFIG_KBDFRONT ?= y
> -CONFIG_CONSFRONT ?= y
> -CONFIG_XENBUS ?= y
> -CONFIG_XC ?=y
> -CONFIG_LWIP ?= $(lwip)
> -CONFIG_BALLOON ?= n
> -
> -# Export config items as compiler directives
> -flags-$(CONFIG_START_NETWORK) += -DCONFIG_START_NETWORK
> -flags-$(CONFIG_SPARSE_BSS) += -DCONFIG_SPARSE_BSS
> -flags-$(CONFIG_QEMU_XS_ARGS) += -DCONFIG_QEMU_XS_ARGS
> -flags-$(CONFIG_PCIFRONT) += -DCONFIG_PCIFRONT
> -flags-$(CONFIG_BLKFRONT) += -DCONFIG_BLKFRONT
> -flags-$(CONFIG_TPMFRONT) += -DCONFIG_TPMFRONT
> -flags-$(CONFIG_TPM_TIS) += -DCONFIG_TPM_TIS
> -flags-$(CONFIG_TPMBACK) += -DCONFIG_TPMBACK
> -flags-$(CONFIG_NETFRONT) += -DCONFIG_NETFRONT
> -flags-$(CONFIG_KBDFRONT) += -DCONFIG_KBDFRONT
> -flags-$(CONFIG_FBFRONT) += -DCONFIG_FBFRONT
> -flags-$(CONFIG_CONSFRONT) += -DCONFIG_CONSFRONT
> -flags-$(CONFIG_XENBUS) += -DCONFIG_XENBUS
> -flags-$(CONFIG_BALLOON) += -DCONFIG_BALLOON
> -
> -DEF_CFLAGS += $(flags-y)
>  
>  # Symlinks and headers that must be created before building the C files
>  GENERATED_HEADERS := include/list.h $(ARCH_LINKS) include/mini-os include/$(TARGET_ARCH_FAM)/mini-os
> diff --git a/arch/x86/Makefile b/arch/x86/Makefile
> index a46766c..0052b4c 100644
> --- a/arch/x86/Makefile
> +++ b/arch/x86/Makefile
> @@ -6,9 +6,6 @@
>  TOPLEVEL_DIR = $(CURDIR)/../..
>  include ../../Config.mk
>  
> -# include arch.mk has to be before mini-os.mk!
> -
> -include arch.mk
>  include ../../minios.mk
>  
>  # Sources here are all *.c *.S without $(MINIOS_TARGET_ARCH).S
> diff --git a/config/MiniOS.mk b/config/MiniOS.mk
> deleted file mode 100644
> index be542dc..0000000
> --- a/config/MiniOS.mk
> +++ /dev/null
> @@ -1,10 +0,0 @@
> -include $(MINIOS_ROOT)/config/StdGNU.mk
> -include $(MINIOS_ROOT)/Config.mk
> -CFLAGS += $(DEF_CFLAGS) $(ARCH_CFLAGS)
> -CPPFLAGS += $(DEF_CPPFLAGS) $(ARCH_CPPFLAGS) $(extra_incl)
> -ASFLAGS += $(DEF_ASFLAGS) $(ARCH_ASFLAGS)
> -LDFLAGS += $(DEF_LDFLAGS) $(ARCH_LDFLAGS)
> -
> -# Override settings for this OS
> -PTHREAD_LIBS =
> -nosharedlibs=y
> diff --git a/config/StdGNU.mk b/config/StdGNU.mk
> deleted file mode 100644
> index 4efebe3..0000000
> --- a/config/StdGNU.mk
> +++ /dev/null
> @@ -1,47 +0,0 @@
> -AS         = $(CROSS_COMPILE)as
> -LD         = $(CROSS_COMPILE)ld
> -ifeq ($(clang),y)
> -CC         = $(CROSS_COMPILE)clang
> -LD_LTO     = $(CROSS_COMPILE)llvm-ld
> -else
> -CC         = $(CROSS_COMPILE)gcc
> -LD_LTO     = $(CROSS_COMPILE)ld
> -endif
> -CPP        = $(CC) -E
> -AR         = $(CROSS_COMPILE)ar
> -RANLIB     = $(CROSS_COMPILE)ranlib
> -NM         = $(CROSS_COMPILE)nm
> -STRIP      = $(CROSS_COMPILE)strip
> -OBJCOPY    = $(CROSS_COMPILE)objcopy
> -OBJDUMP    = $(CROSS_COMPILE)objdump
> -SIZEUTIL   = $(CROSS_COMPILE)size
> -
> -# Allow git to be wrappered in the environment
> -GIT        ?= git
> -
> -INSTALL      = install
> -INSTALL_DIR  = $(INSTALL) -d -m0755 -p
> -INSTALL_DATA = $(INSTALL) -m0644 -p
> -INSTALL_PROG = $(INSTALL) -m0755 -p
> -
> -BOOT_DIR ?= /boot
> -
> -SOCKET_LIBS =
> -UTIL_LIBS = -lutil
> -DLOPEN_LIBS = -ldl
> -
> -SONAME_LDFLAG = -soname
> -SHLIB_LDFLAGS = -shared
> -
> -ifneq ($(debug),y)
> -CFLAGS += -O2 -fomit-frame-pointer
> -else
> -# Less than -O1 produces bad code and large stack frames
> -CFLAGS += -O1 -fno-omit-frame-pointer
> -CFLAGS-$(gcc) += -fno-optimize-sibling-calls
> -endif
> -
> -ifeq ($(lto),y)
> -CFLAGS += -flto
> -LDFLAGS-$(clang) += -plugin LLVMgold.so
> -endif
> diff --git a/config/arm32.mk b/config/arm32.mk
> deleted file mode 100644
> index 4f83a63..0000000
> --- a/config/arm32.mk
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -CONFIG_ARM := y
> -CONFIG_ARM_32 := y
> -CONFIG_ARM_$(XEN_OS) := y
> -
> -CONFIG_XEN_INSTALL_SUFFIX :=
> -
> -# -march= -mcpu=
> -
> -# Explicitly specifiy 32-bit ARM ISA since toolchain default can be -mthumb:
> -CFLAGS += -marm
> -
> -HAS_PL011 := y
> -HAS_EXYNOS4210 := y
> -HAS_OMAP := y
> -HAS_NS16550 := y
> -
> -# Use only if calling $(LD) directly.
> -LDFLAGS_DIRECT += -EL
> -
> -CONFIG_LOAD_ADDRESS ?= 0x80000000
> -
> -IOEMU_CPU_ARCH ?= arm
> diff --git a/config/arm64.mk b/config/arm64.mk
> deleted file mode 100644
> index 6eafda2..0000000
> --- a/config/arm64.mk
> +++ /dev/null
> @@ -1,19 +0,0 @@
> -CONFIG_ARM := y
> -CONFIG_ARM_64 := y
> -CONFIG_ARM_$(XEN_OS) := y
> -
> -CONFIG_XEN_INSTALL_SUFFIX :=
> -
> -CFLAGS += #-marm -march= -mcpu= etc
> -
> -HAS_PL011 := y
> -HAS_NS16550 := y
> -
> -# Use only if calling $(LD) directly.
> -LDFLAGS_DIRECT += -EL
> -
> -CONFIG_LOAD_ADDRESS ?= 0x80000000
> -
> -IOEMU_CPU_ARCH ?= aarch64
> -
> -EFI_DIR ?= /usr/lib64/efi
> diff --git a/config/x86_32.mk b/config/x86_32.mk
> deleted file mode 100644
> index c2d785e..0000000
> --- a/config/x86_32.mk
> +++ /dev/null
> @@ -1,20 +0,0 @@
> -CONFIG_X86 := y
> -CONFIG_X86_32 := y
> -CONFIG_X86_$(XEN_OS) := y
> -
> -CONFIG_HVM := y
> -CONFIG_MIGRATE := y
> -CONFIG_XCUTILS := y
> -
> -HAS_MEM_ACCESS := y
> -HAS_MEM_PAGING := y
> -HAS_MEM_SHARING := y
> -
> -CFLAGS += -m32 -march=i686
> -
> -# Use only if calling $(LD) directly.
> -LDFLAGS_DIRECT_OpenBSD = _obsd
> -LDFLAGS_DIRECT_FreeBSD = _fbsd
> -LDFLAGS_DIRECT += -melf_i386$(LDFLAGS_DIRECT_$(XEN_OS))
> -
> -IOEMU_CPU_ARCH ?= i386
> diff --git a/config/x86_64.mk b/config/x86_64.mk
> deleted file mode 100644
> index bf104c2..0000000
> --- a/config/x86_64.mk
> +++ /dev/null
> @@ -1,33 +0,0 @@
> -CONFIG_X86 := y
> -CONFIG_X86_64 := y
> -CONFIG_X86_$(XEN_OS) := y
> -
> -CONFIG_COMPAT := y
> -CONFIG_HVM := y
> -CONFIG_MIGRATE := y
> -CONFIG_XCUTILS := y
> -
> -HAS_MEM_ACCESS := y
> -HAS_MEM_PAGING := y
> -HAS_MEM_SHARING := y
> -
> -CONFIG_XEN_INSTALL_SUFFIX := .gz
> -
> -CFLAGS += -m64
> -
> -SunOS_LIBDIR = $(SunOS_LIBDIR_x86_64)
> -
> -EFI_DIR ?= /usr/lib64/efi
> -
> -# Use only if calling $(LD) directly.
> -ifeq ($(XEN_OS),OpenBSD)
> -LDFLAGS_DIRECT += -melf_x86_64_obsd
> -else
> -ifeq ($(XEN_OS),FreeBSD)
> -LDFLAGS_DIRECT += -melf_x86_64_fbsd
> -else
> -LDFLAGS_DIRECT += -melf_x86_64
> -endif
> -endif
> -
> -IOEMU_CPU_ARCH ?= x86_64
> diff --git a/minios.mk b/minios.mk
> index 89534f7..1d1cc99 100644
> --- a/minios.mk
> +++ b/minios.mk
> @@ -32,9 +32,9 @@ DEF_CFLAGS += -D__INSIDE_MINIOS__
>  # Build the CFLAGS and ASFLAGS for compiling and assembling.
>  # DEF_... flags are the common mini-os flags,
>  # ARCH_... flags may be defined in arch/$(TARGET_ARCH_FAM/rules.mk
> -CFLAGS := $(DEF_CFLAGS) $(ARCH_CFLAGS)
> +CFLAGS := $(DEF_CFLAGS) $(ARCH_CFLAGS) $(DEFINES-y)
>  CPPFLAGS := $(DEF_CPPFLAGS) $(ARCH_CPPFLAGS)
> -ASFLAGS := $(DEF_ASFLAGS) $(ARCH_ASFLAGS)
> +ASFLAGS := $(DEF_ASFLAGS) $(ARCH_ASFLAGS) $(DEFINES-y)
>  LDFLAGS := $(DEF_LDFLAGS) $(ARCH_LDFLAGS)
>  
>  # Special build dependencies.
> -- 
> 2.6.6
> 

-- 
Samuel
<s> on se croirait en cool : Some browsers close comments on the first ">" character, so to hide script content from such browsers, you can transpose operands for relational and shift operators (e.g., use "y < x" rather than "x > y") or use scripting language-dependent escapes for ">".
 -+- #ens-mim -+-

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Minios-devel] [PATCH v3 00/19] mini-os: support of auto-ballooning

[Wei Liu]

Series pushed, except for the last patch that changed the build system
because it is not yet acked or reviewed.

Note that patch #16 and #17 are resent in another thread.

Thank you both.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Minios-devel] [PATCH v3 00/19] mini-os: support of auto-ballooning

[Wei Liu]

On Thu, Aug 11, 2016 at 11:18:03AM +0200, Juergen Gross wrote:
> Support ballooning Mini-OS automatically up in case of memory shortage.
> 
> Do some cleanups, a small correction and add some basic features to
> lay groundwork for support of ballooning in Mini-OS (patches 1-14).
> 
> The main visible change is the virtual memory layout: to be able to
> add memory to the running Mini-OS we need to have some spare areas
> especially after the 1:1 mapping of physical memory.
> 
> Then add the ballooning functionality: the p2m map must be expanded,
> the page allocator's bitmap must  be expanded and we must get new
> memory from the hypervisor.
> 
> In case of a detected memory shortage the domain will balloon up until
> either enough memory is available or the upper limit has been reached.
> 
> Ballooning has been tested with a xenstore stubdom.
> Regression tests have been done with:
> - pure mini-os
> - ioemu stubdom
> - pvgrub 64 bit
> 

Unfortunately vtmpmgr stubdom build failed. :-/

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Minios-devel] [PATCH v3 00/19] mini-os: support of auto-ballooning

[Wei Liu]

On Fri, Aug 12, 2016 at 12:42:31PM +0100, Wei Liu wrote:
> On Thu, Aug 11, 2016 at 11:18:03AM +0200, Juergen Gross wrote:
> > Support ballooning Mini-OS automatically up in case of memory shortage.
> > 
> > Do some cleanups, a small correction and add some basic features to
> > lay groundwork for support of ballooning in Mini-OS (patches 1-14).
> > 
> > The main visible change is the virtual memory layout: to be able to
> > add memory to the running Mini-OS we need to have some spare areas
> > especially after the 1:1 mapping of physical memory.
> > 
> > Then add the ballooning functionality: the p2m map must be expanded,
> > the page allocator's bitmap must  be expanded and we must get new
> > memory from the hypervisor.
> > 
> > In case of a detected memory shortage the domain will balloon up until
> > either enough memory is available or the upper limit has been reached.
> > 
> > Ballooning has been tested with a xenstore stubdom.
> > Regression tests have been done with:
> > - pure mini-os
> > - ioemu stubdom
> > - pvgrub 64 bit
> > 
> 
> Unfortunately vtmpmgr stubdom build failed. :-/
> 


And that's probably because vtpmmgr uses printk or whatever...

Here is a patch that seems to fix the build:

---8<---
From d72510368cdc3c73af3c8918a404a8137f40bd9c Mon Sep 17 00:00:00 2001
From: Wei Liu <wei.liu2@citrix.com>
Date: Fri, 12 Aug 2016 11:32:57 +0100
Subject: [PATCH] x86/arch_mm.h: move p2m_chk_pfn to x86/mm.c

Making that function inlined won't buy us much and is causing error in
vtpm manager stubdom build.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 arch/x86/mm.c         |  9 +++++++++
 include/x86/arch_mm.h | 11 +++--------
 2 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index 8fa3b4c..88a928d 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -608,6 +608,15 @@ static void clear_bootstrap(void)
         printk("Unable to unmap NULL page. rc=%d\n", rc);
 }
 
+void p2m_chk_pfn(unsigned long pfn)
+{
+    if ( (pfn >> L3_P2M_SHIFT) > 0 )
+    {
+        printk("Error: Too many pfns.\n");
+        do_exit();
+    }
+}
+
 void arch_init_p2m(unsigned long max_pfn)
 {
     unsigned long *l2_list = NULL, *l3_list;
diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index e5d9c57..690a919 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -190,14 +190,9 @@ typedef unsigned long pgentry_t;
 #define L2_P2M_IDX(pfn) (((pfn) >> L1_P2M_SHIFT) & P2M_MASK)
 #define L3_P2M_IDX(pfn) (((pfn) >> L2_P2M_SHIFT) & P2M_MASK)
 #define INVALID_P2M_ENTRY (~0UL)
-static inline void p2m_chk_pfn(unsigned long pfn)
-{
-    if ( (pfn >> L3_P2M_SHIFT) > 0 )
-    {
-        printk("Error: Too many pfns.\n");
-        do_exit();
-    }
-}
+
+void p2m_chk_pfn(unsigned long pfn);
+
 static inline unsigned long p2m_pages(unsigned long pages)
 {
     return (pages + P2M_ENTRIES - 1) >> L1_P2M_SHIFT;
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Minios-devel] [PATCH v3 00/19] mini-os: support of auto-ballooning

[Samuel Thibault]

Wei Liu, on Fri 12 Aug 2016 13:48:09 +0100, wrote:
> ---8<---
> From d72510368cdc3c73af3c8918a404a8137f40bd9c Mon Sep 17 00:00:00 2001
> From: Wei Liu <wei.liu2@citrix.com>
> Date: Fri, 12 Aug 2016 11:32:57 +0100
> Subject: [PATCH] x86/arch_mm.h: move p2m_chk_pfn to x86/mm.c
> 
> Making that function inlined won't buy us much and is causing error in
> vtpm manager stubdom build.
> 
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

> ---
>  arch/x86/mm.c         |  9 +++++++++
>  include/x86/arch_mm.h | 11 +++--------
>  2 files changed, 12 insertions(+), 8 deletions(-)
> 
> diff --git a/arch/x86/mm.c b/arch/x86/mm.c
> index 8fa3b4c..88a928d 100644
> --- a/arch/x86/mm.c
> +++ b/arch/x86/mm.c
> @@ -608,6 +608,15 @@ static void clear_bootstrap(void)
>          printk("Unable to unmap NULL page. rc=%d\n", rc);
>  }
>  
> +void p2m_chk_pfn(unsigned long pfn)
> +{
> +    if ( (pfn >> L3_P2M_SHIFT) > 0 )
> +    {
> +        printk("Error: Too many pfns.\n");
> +        do_exit();
> +    }
> +}
> +
>  void arch_init_p2m(unsigned long max_pfn)
>  {
>      unsigned long *l2_list = NULL, *l3_list;
> diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
> index e5d9c57..690a919 100644
> --- a/include/x86/arch_mm.h
> +++ b/include/x86/arch_mm.h
> @@ -190,14 +190,9 @@ typedef unsigned long pgentry_t;
>  #define L2_P2M_IDX(pfn) (((pfn) >> L1_P2M_SHIFT) & P2M_MASK)
>  #define L3_P2M_IDX(pfn) (((pfn) >> L2_P2M_SHIFT) & P2M_MASK)
>  #define INVALID_P2M_ENTRY (~0UL)
> -static inline void p2m_chk_pfn(unsigned long pfn)
> -{
> -    if ( (pfn >> L3_P2M_SHIFT) > 0 )
> -    {
> -        printk("Error: Too many pfns.\n");
> -        do_exit();
> -    }
> -}
> +
> +void p2m_chk_pfn(unsigned long pfn);
> +
>  static inline unsigned long p2m_pages(unsigned long pages)
>  {
>      return (pages + P2M_ENTRIES - 1) >> L1_P2M_SHIFT;
> -- 
> 2.1.4
> 

-- 
Samuel
"How should I know if it works?  That's what beta testers are for.  I only
coded it."
(Attributed to Linus Torvalds, somewhere in a posting)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel