[Xen-devel] [RFC PATCH v2 0/14] Live update: boot memory management, data stream handling

[Xen-devel] [RFC PATCH v2 0/14] Live update: boot memory management, data stream handling

[David Woodhouse]

[-- Attachment #1.1: Type: text/plain, Size: 2016 bytes --]

Expanding on the previous WIP patch set.

As before, reserve a contiguous region of memory which can be safely
used by the boot allocator in the new Xen, before the live update data
stream has been processed and thus before the locations of all the
other pages which contain live domain data are known.

As the last gasp of kexec_reloc(), leave a 'breadcrumb' in the first
words of the reserved bootmem region, for Xen to find the live update
data. Which is mostly a guest-transparent live migration data stream,
except the guest memory is left in-place.

The breadcrumb has a magic value, the physical address of an MFN array
referencing the pages with actual data, and the number of such pages.
All of these are allocated in arbitrary heap pages (and not in the
reserved bootmem region) by the original Xen.

Provide functions on the "save" side for appending to the LU data
stream (designed to cope with the way that hvm_save_size() and
hvm_save() work), and on the "receive" side for detecting and mapping it.

On the way to excluding "already in use" pages from being added to the
heap at start up, also fix the long-standing bug that pages marked bad
with 'badpage=' on the command line weren't being eschewed if they were
above HYPERVISOR_VIRT_END and added directly to the heap; only
init_boot_pages() was doing that filtering.

This is now handled by setting either PGC_broken (for bad pages) or
PGC_allocated (for those containing live update data) in the
corresponding page_info, at a time when the frametable is expected to
be initialised to zero. When init_heap_pages() sees such a page it
knows not to use it. Bad pages thus get completely ignored as they
should be (and put on the pointless page_broken_list that nobody ever
uses AFAICT).

The "in use" pages will need some rehabilitation (of refcount,
ownership etc.) before the system is in a correct state. That will come
shortly, as we start passing real domain data across this mechanism and
processing it.

[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5174 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 01/14] x86/setup: Don't skip 2MiB underneath relocated Xen image

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

Set 'e' correctly to reflect the location that Xen is actually relocated
to from its default 2MiB location. Not 2MiB below that.

This is only vaguely a bug fix. The "missing" 2MiB would have been used
in the end, and fed to the allocator. It's just that other things don't
get to sit right up *next* to the Xen image, and it isn't very tidy.

For live update, I'd quite like a single contiguous region for the
reserved bootmem and Xen, allowing the 'slack' in the former to be used
when Xen itself grows larger. Let's not allow 2MiB of random heap pages
to get in the way...

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 xen/arch/x86/setup.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 1b6ca4a47d..3c6d0a5b35 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1080,9 +1080,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)
             unsigned long pte_update_limit;
 
             /* Select relocation address. */
-            e = end - reloc_size;
-            xen_phys_start = e;
-            bootsym(trampoline_xen_phys_start) = e;
+            xen_phys_start = end - reloc_size;
+            e = xen_phys_start + XEN_IMG_OFFSET;
+            bootsym(trampoline_xen_phys_start) = xen_phys_start;
 
             /*
              * No PTEs pointing above this address are candidates for relocation.
@@ -1090,7 +1090,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
              * and the beginning of region for destination image some PTEs may
              * point to addresses in range [e, e + XEN_IMG_OFFSET).
              */
-            pte_update_limit = PFN_DOWN(e + XEN_IMG_OFFSET);
+            pte_update_limit = PFN_DOWN(e);
 
             /*
              * Perform relocation to new physical address.
@@ -1099,7 +1099,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
              * data until after we have switched to the relocated pagetables!
              */
             barrier();
-            move_memory(e + XEN_IMG_OFFSET, XEN_IMG_OFFSET, _end - _start, 1);
+            move_memory(e, XEN_IMG_OFFSET, _end - _start, 1);
 
             /* Walk initial pagetables, relocating page directory entries. */
             pl4e = __va(__pa(idle_pg_table));
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 02/14] x86/boot: Reserve live update boot memory

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

For live update to work, it will need a region of memory that can be
given to the boot allocator while it parses the state information from
the previous Xen and works out which of the other pages of memory it
can consume.

Reserve that like the crashdump region, and accept it on the command
line. Use only that region for early boot, and register the remaining
RAM (all of it for now, until the real live update happens) later.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 xen/arch/x86/setup.c         | 114 ++++++++++++++++++++++++++++++++---
 xen/include/asm-x86/config.h |   1 +
 2 files changed, 108 insertions(+), 7 deletions(-)

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 3c6d0a5b35..420c38456b 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -681,6 +681,41 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li
 /* How much of the directmap is prebuilt at compile time. */
 #define PREBUILT_MAP_LIMIT (1 << L2_PAGETABLE_SHIFT)
 
+static unsigned long lu_bootmem_start, lu_bootmem_size, lu_data;
+
+static int __init parse_liveupdate(const char *str)
+{
+    const char *cur;
+    lu_bootmem_size = parse_size_and_unit(cur = str, &str);
+    if (!lu_bootmem_size || cur == str)
+        return -EINVAL;
+
+    if (!*str) {
+        printk("Live update size 0x%lx\n", lu_bootmem_size);
+        return 0;
+    }
+    if (*str != '@')
+        return -EINVAL;
+    lu_bootmem_start = parse_size_and_unit(cur = str + 1, &str);
+    if (!lu_bootmem_start || cur == str)
+        return -EINVAL;
+
+    printk("Live update area 0x%lx-0x%lx (0x%lx)\n", lu_bootmem_start,
+           lu_bootmem_start + lu_bootmem_size, lu_bootmem_size);
+
+    if (!*str)
+        return 0;
+    if (*str != ':')
+        return -EINVAL;
+    lu_data = simple_strtoull(cur = str + 1, &str, 0);
+    if (!lu_data || cur == str)
+        return -EINVAL;
+
+    printk("Live update data at 0x%lx\n", lu_data);
+    return 0;
+}
+custom_param("liveupdate", parse_liveupdate);
+
 void __init noreturn __start_xen(unsigned long mbi_p)
 {
     char *memmap_type = NULL;
@@ -690,7 +725,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
     module_t *mod;
     unsigned long nr_pages, raw_max_page, modules_headroom, module_map[1];
     int i, j, e820_warn = 0, bytes = 0;
-    bool acpi_boot_table_init_done = false, relocated = false;
+    bool acpi_boot_table_init_done = false, relocated = false, lu_reserved = false;
     int ret;
     struct ns16550_defaults ns16550 = {
         .data_bits = 8,
@@ -980,6 +1015,22 @@ void __init noreturn __start_xen(unsigned long mbi_p)
     set_kexec_crash_area_size((u64)nr_pages << PAGE_SHIFT);
     kexec_reserve_area(&boot_e820);
 
+    if ( lu_bootmem_start )
+    {
+        /* XX: Check it's in usable memory first */
+        reserve_e820_ram(&boot_e820, lu_bootmem_start, lu_bootmem_start + lu_bootmem_size);
+
+        /* Since it will already be out of the e820 map by the time the first
+         * loop over physical memory, map it manually already. */
+        set_pdx_range(lu_bootmem_start >> PAGE_SHIFT,
+                      (lu_bootmem_start + lu_bootmem_size) >> PAGE_SHIFT);
+        map_pages_to_xen((unsigned long)__va(lu_bootmem_start),
+                         maddr_to_mfn(lu_bootmem_start),
+                         PFN_DOWN(lu_bootmem_size), PAGE_HYPERVISOR);
+
+        lu_reserved = true;
+    }
+
     initial_images = mod;
     nr_initial_images = mbi->mods_count;
 
@@ -1207,6 +1258,16 @@ void __init noreturn __start_xen(unsigned long mbi_p)
             printk("New Xen image base address: %#lx\n", xen_phys_start);
         }
 
+        /* Is the region suitable for the live update bootmem region? */
+        if ( lu_bootmem_size && ! lu_bootmem_start && e < limit )
+        {
+            end = consider_modules(s, e, lu_bootmem_size, mod, mbi->mods_count + relocated, -1);
+            if ( end )
+            {
+                e = lu_bootmem_start = end - lu_bootmem_size;
+            }
+        }
+
         /* Is the region suitable for relocating the multiboot modules? */
         for ( j = mbi->mods_count - 1; j >= 0; j-- )
         {
@@ -1270,6 +1331,15 @@ void __init noreturn __start_xen(unsigned long mbi_p)
     if ( !xen_phys_start )
         panic("Not enough memory to relocate Xen\n");
 
+    if ( lu_bootmem_start )
+    {
+        if ( !lu_reserved )
+            reserve_e820_ram(&boot_e820, lu_bootmem_start, lu_bootmem_start + lu_bootmem_size);
+        printk("LU bootmem: 0x%lx - 0x%lx\n", lu_bootmem_start, lu_bootmem_start + lu_bootmem_size);
+        init_boot_pages(lu_bootmem_start, lu_bootmem_start + lu_bootmem_size);
+        lu_reserved = true;
+    }
+
     /* This needs to remain in sync with xen_in_range(). */
     reserve_e820_ram(&boot_e820, __pa(_stext), __pa(__2M_rwdata_end));
 
@@ -1281,8 +1351,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
         xenheap_max_mfn(PFN_DOWN(highmem_start - 1));
 
     /*
-     * Walk every RAM region and map it in its entirety (on x86/64, at least)
-     * and notify it to the boot allocator.
+     * Walk every RAM region and map it in its entirety and (unless in
+     * live update mode) notify it to the boot allocator.
      */
     for ( i = 0; i < boot_e820.nr_map; i++ )
     {
@@ -1335,6 +1405,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
                 printk(XENLOG_WARNING "Ignoring inaccessible memory range"
                                       " %013"PRIx64"-%013"PRIx64"\n",
                        s, e);
+                reserve_e820_ram(&boot_e820, s, e);
                 continue;
             }
             map_e = e;
@@ -1342,6 +1413,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
             printk(XENLOG_WARNING "Ignoring inaccessible memory range"
                                   " %013"PRIx64"-%013"PRIx64"\n",
                    e, map_e);
+            reserve_e820_ram(&boot_e820, e, map_e);
         }
 
         set_pdx_range(s >> PAGE_SHIFT, e >> PAGE_SHIFT);
@@ -1352,7 +1424,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)
                       ARRAY_SIZE(l2_identmap) << L2_PAGETABLE_SHIFT);
 
         /* Pass mapped memory to allocator /before/ creating new mappings. */
-        init_boot_pages(s, min(map_s, e));
+        if ( !lu_reserved)
+            init_boot_pages(s, min(map_s, e));
+
         s = map_s;
         if ( s < map_e )
         {
@@ -1360,7 +1434,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 
             map_s = (s + mask) & ~mask;
             map_e &= ~mask;
-            init_boot_pages(map_s, map_e);
+            if ( !lu_reserved)
+                init_boot_pages(map_s, map_e);
         }
 
         if ( map_s > map_e )
@@ -1376,7 +1451,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
             {
                 map_pages_to_xen((unsigned long)__va(map_e), maddr_to_mfn(map_e),
                                  PFN_DOWN(end - map_e), PAGE_HYPERVISOR);
-                init_boot_pages(map_e, end);
+                if ( !lu_reserved)
+                    init_boot_pages(map_e, end);
                 map_e = end;
             }
         }
@@ -1391,7 +1467,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
         {
             map_pages_to_xen((unsigned long)__va(s), maddr_to_mfn(s),
                              PFN_DOWN(map_s - s), PAGE_HYPERVISOR);
-            init_boot_pages(s, map_s);
+            if ( !lu_reserved)
+                init_boot_pages(s, map_s);
         }
     }
 
@@ -1489,6 +1566,29 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 
     numa_initmem_init(0, raw_max_page);
 
+    if ( lu_bootmem_start )
+    {
+        unsigned long limit = virt_to_mfn(HYPERVISOR_VIRT_END - 1);
+        uint64_t mask = PAGE_SIZE - 1;
+
+        for ( i = 0; i < boot_e820.nr_map; i++ )
+        {
+            uint64_t s, e;
+
+            if ( boot_e820.map[i].type != E820_RAM )
+                continue;
+            s = (boot_e820.map[i].addr + mask) & ~mask;
+            e = (boot_e820.map[i].addr + boot_e820.map[i].size) & ~mask;
+            s = max_t(uint64_t, s, 1<<20);
+            if ( PFN_DOWN(s) > limit )
+                continue;
+            if ( PFN_DOWN(e) > limit )
+                e = pfn_to_paddr(limit);
+
+            init_boot_pages(s, e);
+        }
+    }
+
     if ( max_page - 1 > virt_to_mfn(HYPERVISOR_VIRT_END - 1) )
     {
         unsigned long limit = virt_to_mfn(HYPERVISOR_VIRT_END - 1);
diff --git a/xen/include/asm-x86/config.h b/xen/include/asm-x86/config.h
index dab7d7bade..55f327db70 100644
--- a/xen/include/asm-x86/config.h
+++ b/xen/include/asm-x86/config.h
@@ -278,6 +278,7 @@ extern unsigned char boot_edid_info[128];
 
 #ifndef __ASSEMBLY__
 extern unsigned long xen_phys_start;
+extern unsigned long lu_bootmem_start, lu_bootmem_size;
 #endif
 
 /* GDT/LDT shadow mapping area. The first per-domain-mapping sub-area. */
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC PATCH v2 02/14] x86/boot: Reserve live update boot memory

[Wei Liu]

On Wed, Jan 22, 2020 at 08:53:45AM +0000, David Woodhouse wrote:
> From: David Woodhouse <dwmw@amazon.co.uk>
> 
> For live update to work, it will need a region of memory that can be
> given to the boot allocator while it parses the state information from
> the previous Xen and works out which of the other pages of memory it
> can consume.
> 
> Reserve that like the crashdump region, and accept it on the command
> line. Use only that region for early boot, and register the remaining
> RAM (all of it for now, until the real live update happens) later.
> 
> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
> ---
>  xen/arch/x86/setup.c         | 114 ++++++++++++++++++++++++++++++++---
>  xen/include/asm-x86/config.h |   1 +

You also need to patch docs/misc/xen-command-line.pandoc for the new
command line option.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 03/14] Don't add unused parts of live update reserved bootmem to heap

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

We need to promise that we won't use it for anything that will need to
persist across live update. Until we can achieve that (which will build
on of some of the work Hongyan is doing for secret hiding), the easy
answer is just not to add that region to the main heap at all. If
it was sized appropriately there isn't even much wastage.
---
 xen/arch/x86/setup.c    | 10 ++++++++++
 xen/common/page_alloc.c |  5 +++++
 xen/include/xen/mm.h    |  1 +
 3 files changed, 16 insertions(+)

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 420c38456b..4296447d9f 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1571,6 +1571,16 @@ void __init noreturn __start_xen(unsigned long mbi_p)
         unsigned long limit = virt_to_mfn(HYPERVISOR_VIRT_END - 1);
         uint64_t mask = PAGE_SIZE - 1;
 
+        /*
+         * Pages in the reserved LU region must not be used for anything which
+         * will need to persist across a live update. There is ongoing work to
+         * eliminate or limit the use of share_xen_page_with_guest() and get
+         * to a point where we can actually honour that promise, but for now
+         * just *don't* add those pages to the heap. Clear the boot allocator
+         * out completely, before adding the non-reserved ranges.
+         */
+        clear_boot_allocator();
+
         for ( i = 0; i < boot_e820.nr_map; i++ )
         {
             uint64_t s, e;
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 919a270587..20ef25d45a 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1879,6 +1879,11 @@ void __init end_boot_allocator(void)
     printk("\n");
 }
 
+void __init clear_boot_allocator(void)
+{
+    nr_bootmem_regions = 0;
+}
+
 static void __init smp_scrub_heap_pages(void *data)
 {
     unsigned long mfn, start, end;
diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
index 8d0ddfb60c..5bce174a3e 100644
--- a/xen/include/xen/mm.h
+++ b/xen/include/xen/mm.h
@@ -158,6 +158,7 @@ struct domain *__must_check page_get_owner_and_reference(struct page_info *);
 void init_boot_pages(paddr_t ps, paddr_t pe);
 mfn_t alloc_boot_pages(unsigned long nr_pfns, unsigned long pfn_align);
 void end_boot_allocator(void);
+void clear_boot_allocator(void);
 
 /* Xen suballocator. These functions are interrupt-safe. */
 void init_xenheap_pages(paddr_t ps, paddr_t pe);
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 04/14] Add KEXEC_RANGE_MA_LIVEUPDATE

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

This allows kexec userspace to tell the next Xen where the range is,
on its command line.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 xen/arch/x86/machine_kexec.c | 13 ++++++++++---
 xen/arch/x86/setup.c         |  2 +-
 xen/include/public/kexec.h   |  1 +
 3 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/xen/arch/x86/machine_kexec.c b/xen/arch/x86/machine_kexec.c
index b70d5a6a86..273bc20664 100644
--- a/xen/arch/x86/machine_kexec.c
+++ b/xen/arch/x86/machine_kexec.c
@@ -186,9 +186,16 @@ void machine_kexec(struct kexec_image *image)
 
 int machine_kexec_get(xen_kexec_range_t *range)
 {
-	if (range->range != KEXEC_RANGE_MA_XEN)
-		return -EINVAL;
-	return machine_kexec_get_xen(range);
+    switch (range->range) {
+    case KEXEC_RANGE_MA_XEN:
+        return machine_kexec_get_xen(range);
+    case KEXEC_RANGE_MA_LIVEUPDATE:
+        range->start = lu_bootmem_start;
+        range->size = lu_bootmem_size;
+        return 0;
+    default:
+        return -EINVAL;
+    }
 }
 
 void arch_crash_save_vmcoreinfo(void)
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 4296447d9f..521946dc27 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -681,7 +681,7 @@ static unsigned int __init copy_bios_e820(struct e820entry *map, unsigned int li
 /* How much of the directmap is prebuilt at compile time. */
 #define PREBUILT_MAP_LIMIT (1 << L2_PAGETABLE_SHIFT)
 
-static unsigned long lu_bootmem_start, lu_bootmem_size, lu_data;
+unsigned long lu_bootmem_start, lu_bootmem_size, lu_data;
 
 static int __init parse_liveupdate(const char *str)
 {
diff --git a/xen/include/public/kexec.h b/xen/include/public/kexec.h
index 3f2a118381..298381af8d 100644
--- a/xen/include/public/kexec.h
+++ b/xen/include/public/kexec.h
@@ -150,6 +150,7 @@ typedef struct xen_kexec_load_v1 {
 #define KEXEC_RANGE_MA_EFI_MEMMAP 5 /* machine address and size of
                                      * of the EFI Memory Map */
 #define KEXEC_RANGE_MA_VMCOREINFO 6 /* machine address and size of vmcoreinfo */
+#define KEXEC_RANGE_MA_LIVEUPDATE 7 /* Boot mem for live update */
 
 /*
  * Find the address and size of certain memory areas
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 05/14] Add KEXEC_TYPE_LIVE_UPDATE

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

This is identical to the default case... for now.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 xen/common/kexec.c         | 18 ++++++++++++++++++
 xen/include/public/kexec.h | 12 ++++++++----
 2 files changed, 26 insertions(+), 4 deletions(-)

diff --git a/xen/common/kexec.c b/xen/common/kexec.c
index a262cc5a18..a78aa4f5b0 100644
--- a/xen/common/kexec.c
+++ b/xen/common/kexec.c
@@ -404,6 +404,19 @@ static long kexec_reboot(void *_image)
     return 0;
 }
 
+static long kexec_live_update(void *_image)
+{
+    struct kexec_image *image = _image;
+
+    kexecing = TRUE;
+
+    kexec_common_shutdown();
+    machine_reboot_kexec(image);
+
+    BUG();
+    return 0;
+}
+
 static void do_crashdump_trigger(unsigned char key)
 {
     printk("'%c' pressed -> triggering crashdump\n", key);
@@ -736,6 +749,7 @@ static int kexec_load_get_bits(int type, int *base, int *bit)
     switch ( type )
     {
     case KEXEC_TYPE_DEFAULT:
+    case KEXEC_TYPE_LIVE_UPDATE:
         *base = KEXEC_IMAGE_DEFAULT_BASE;
         *bit = KEXEC_FLAG_DEFAULT_POS;
         break;
@@ -837,6 +851,10 @@ static int kexec_exec(XEN_GUEST_HANDLE_PARAM(void) uarg)
         image = kexec_image[base + pos];
         ret = continue_hypercall_on_cpu(0, kexec_reboot, image);
         break;
+    case KEXEC_TYPE_LIVE_UPDATE:
+        image = kexec_image[base + pos];
+        ret = continue_hypercall_on_cpu(0, kexec_live_update, image);
+        break;
     case KEXEC_TYPE_CRASH:
         kexec_crash(); /* Does not return */
         break;
diff --git a/xen/include/public/kexec.h b/xen/include/public/kexec.h
index 298381af8d..f5230286d3 100644
--- a/xen/include/public/kexec.h
+++ b/xen/include/public/kexec.h
@@ -71,18 +71,22 @@
  */
 
 /*
- * Kexec supports two types of operation:
+ * Kexec supports three types of operation:
  * - kexec into a regular kernel, very similar to a standard reboot
  *   - KEXEC_TYPE_DEFAULT is used to specify this type
  * - kexec into a special "crash kernel", aka kexec-on-panic
  *   - KEXEC_TYPE_CRASH is used to specify this type
  *   - parts of our system may be broken at kexec-on-panic time
  *     - the code should be kept as simple and self-contained as possible
+ * - Live update into a new Xen, preserving all running domains
+ *   - KEXEC_TYPE_LIVE_UPDATE is used to specify this type
+ *   - Xen performs guest-transparent live migration and stores live
+ *     update state in memory, passing it to the new Xen.
  */
 
-#define KEXEC_TYPE_DEFAULT 0
-#define KEXEC_TYPE_CRASH   1
-
+#define KEXEC_TYPE_DEFAULT          0
+#define KEXEC_TYPE_CRASH            1
+#define KEXEC_TYPE_LIVE_UPDATE      2
 
 /* The kexec implementation for Xen allows the user to load two
  * types of kernels, KEXEC_TYPE_DEFAULT and KEXEC_TYPE_CRASH.
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 06/14] Add IND_WRITE64 primitive to kexec kimage

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

This allows a single page-aligned physical address to be written to
the current destination, intended to pass the location of the live
update data stream from one Xen to the next.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 xen/arch/x86/x86_64/kexec_reloc.S | 9 ++++++++-
 xen/include/xen/kimage.h          | 1 +
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/x86_64/kexec_reloc.S b/xen/arch/x86/x86_64/kexec_reloc.S
index d488d127cf..a75f989926 100644
--- a/xen/arch/x86/x86_64/kexec_reloc.S
+++ b/xen/arch/x86/x86_64/kexec_reloc.S
@@ -131,11 +131,18 @@ is_source:
         jmp     next_entry
 is_zero:
         testb   $IND_ZERO, %cl
-        jz      next_entry
+        jz      is_write64
         movl    $(PAGE_SIZE / 8), %ecx  /* Zero the destination page. */
         xorl    %eax, %eax
         rep stosq
         jmp     next_entry
+is_write64:
+        testb   $IND_WRITE64, %cl
+        jz      next_entry
+        andq    $PAGE_MASK, %rcx
+        movq    %rcx, %rax
+        stosq
+        jmp     next_entry
 done:
         popq    %rbx
         ret
diff --git a/xen/include/xen/kimage.h b/xen/include/xen/kimage.h
index cbfb9e9054..e94839d7c3 100644
--- a/xen/include/xen/kimage.h
+++ b/xen/include/xen/kimage.h
@@ -6,6 +6,7 @@
 #define IND_DONE         0x4
 #define IND_SOURCE       0x8
 #define IND_ZERO        0x10
+#define IND_WRITE64     0x20
 
 #ifndef __ASSEMBLY__
 
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 07/14] Add kimage_add_live_update_data()

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 xen/common/kimage.c          | 33 +++++++++++++++++++++++++++++++++
 xen/include/asm-x86/config.h |  2 ++
 xen/include/xen/kimage.h     |  3 +++
 3 files changed, 38 insertions(+)

diff --git a/xen/common/kimage.c b/xen/common/kimage.c
index 210241dfb7..eafa6a9c26 100644
--- a/xen/common/kimage.c
+++ b/xen/common/kimage.c
@@ -938,6 +938,39 @@ done:
     return ret;
 }
 
+int kimage_add_live_update_data(struct kexec_image *image, mfn_t data, int nr_mfns)
+{
+    int ret;
+
+    /*
+     * For live update, we place the physical location of 'data'
+     * into the first 64 bits of the reserved live update bootmem
+     * region. At 'data' is an MFN list of pages containing the
+     * actual live update stream, which the new Xen can vmap().
+     *
+     * Append IND_WRITE64 operations to the end of the kimage stream
+     * to store the live update magic and the address of 'data' for
+     * the new Xen to see.
+     */
+    if (!lu_bootmem_start || kimage_dst_used(image, lu_bootmem_start))
+        return -EINVAL;
+
+    ret = machine_kexec_add_page(image, lu_bootmem_start, lu_bootmem_start);
+    if ( ret < 0 )
+        return ret;
+
+    ret = kimage_set_destination(image, lu_bootmem_start);
+    if (!ret)
+        ret = kimage_add_entry(image, LIVE_UPDATE_MAGIC | IND_WRITE64);
+    if (!ret)
+        ret = kimage_add_entry(image, mfn_to_maddr(data) | IND_WRITE64);
+    if (!ret)
+        ret = kimage_add_entry(image, (nr_mfns << PAGE_SHIFT) | IND_WRITE64);
+
+    kimage_terminate(image);
+
+    return ret;
+}
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-x86/config.h b/xen/include/asm-x86/config.h
index 55f327db70..e80cfd54f0 100644
--- a/xen/include/asm-x86/config.h
+++ b/xen/include/asm-x86/config.h
@@ -281,6 +281,8 @@ extern unsigned long xen_phys_start;
 extern unsigned long lu_bootmem_start, lu_bootmem_size;
 #endif
 
+#define LIVE_UPDATE_MAGIC        (0x4c69766555700000UL)
+
 /* GDT/LDT shadow mapping area. The first per-domain-mapping sub-area. */
 #define GDT_LDT_VCPU_SHIFT       5
 #define GDT_LDT_VCPU_VA_SHIFT    (GDT_LDT_VCPU_SHIFT + PAGE_SHIFT)
diff --git a/xen/include/xen/kimage.h b/xen/include/xen/kimage.h
index e94839d7c3..1e0e378afd 100644
--- a/xen/include/xen/kimage.h
+++ b/xen/include/xen/kimage.h
@@ -54,6 +54,9 @@ unsigned long kimage_entry_ind(kimage_entry_t *entry, bool_t compat);
 int kimage_build_ind(struct kexec_image *image, mfn_t ind_mfn,
                      bool_t compat);
 
+int kimage_add_live_update_data(struct kexec_image *image, mfn_t data,
+                                int nr_mfns);
+
 #endif /* __ASSEMBLY__ */
 
 #endif /* __XEN_KIMAGE_H__ */
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 08/14] Add basic live update stream creation

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 xen/common/Makefile    |   1 +
 xen/common/lu/Makefile |   1 +
 xen/common/lu/stream.c | 108 +++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/lu.h   |  12 +++++
 4 files changed, 122 insertions(+)
 create mode 100644 xen/common/lu/Makefile
 create mode 100644 xen/common/lu/stream.c
 create mode 100644 xen/include/xen/lu.h

diff --git a/xen/common/Makefile b/xen/common/Makefile
index 62b34e69e9..36c824a99a 100644
--- a/xen/common/Makefile
+++ b/xen/common/Makefile
@@ -78,3 +78,4 @@ subdir-$(CONFIG_UBSAN) += ubsan
 
 subdir-$(CONFIG_NEEDS_LIBELF) += libelf
 subdir-$(CONFIG_HAS_DEVICE_TREE) += libfdt
+subdir-y += lu
diff --git a/xen/common/lu/Makefile b/xen/common/lu/Makefile
new file mode 100644
index 0000000000..68991b3ca4
--- /dev/null
+++ b/xen/common/lu/Makefile
@@ -0,0 +1 @@
+obj-y += stream.o
diff --git a/xen/common/lu/stream.c b/xen/common/lu/stream.c
new file mode 100644
index 0000000000..52030e8b2e
--- /dev/null
+++ b/xen/common/lu/stream.c
@@ -0,0 +1,108 @@
+/*
+ * Live update data stream handling.
+ *
+ * During live update, one version of Xen (Xen#1) performs a kexec into
+ * a new version of Xen (Xen#2), performing guest-transparent live
+ * migration of all existing domains.
+ *
+ * Xen#2 must avoid scribbling on any pages which may belong to existing
+ * domains. In order to achieve this, we reserve a contiguous area of
+ * physical memory to be used by the boot allocator in Xen#2. Xen must
+ * not allocate pages from that region which are later shared with
+ * guests or need to persist across live update.
+ *
+ * The live update bootmem region is reserved by the first Xen to boot,
+ * and userspace can obtain its address using KEXEC_CMD_kexec_get_range
+ * with the new KEXEC_RANGE_MA_LIVEUPDATE type. Userspace kexec(8)
+ * appends the appropriate 'liveupdate=' parameter to the command line
+ * of Xen#2 when setting up the kexec image.
+ *
+ * At the time of kexec, Xen#1 serialises the domain state into buffers
+ * allocated from its own heap., then creates a single physically
+ * contiguous scatter-gather list containing the MFNs of those data
+ * pages (which Xen#2 can then trivially vmap()). In a system with
+ * 4KiB pages, the MFN list for the live update data stream will fit
+ * into a single page until the total size of the live update data
+ * exceeds 2MiB.
+ *
+ * The physical address of the MFN list is passed to Xen#2 by placing
+ * it at the start of the reserved live update bootmem region, with a
+ * magic number to avoid false positives.
+ */
+
+#include <xen/types.h>
+#include <xen/vmap.h>
+#include <xen/lu.h>
+
+static int lu_stream_extend(struct lu_stream *stream, int nr_pages)
+{
+	int order = get_order_from_bytes((nr_pages + 1) * sizeof(mfn_t));
+	int old_order = get_order_from_bytes((stream->nr_pages + 1) * sizeof(mfn_t));
+
+	if (!stream->nr_pages || order > old_order) {
+		mfn_t *new_pglist = alloc_xenheap_pages(order, 0);
+		if (!new_pglist)
+			return -ENOMEM;
+
+		if (stream->nr_pages) {
+			memcpy(new_pglist, stream->pagelist,
+			       stream->nr_pages * sizeof(mfn_t));
+			free_xenheap_pages(stream->pagelist, old_order);
+		}
+		stream->pagelist = new_pglist;
+	}
+	while (stream->nr_pages < nr_pages) {
+		struct page_info *pg = alloc_domheap_page(NULL, MEMF_no_owner);
+		if (!pg) {
+			/* Ensure the cleanup frees the correct order of pagelist */
+			stream->nr_pages++;
+			return -ENOMEM;
+		}
+		stream->pagelist[stream->nr_pages++] = page_to_mfn(pg);
+		stream->pagelist[stream->nr_pages] = 0;
+	}
+
+	if (stream->data)
+		vunmap(stream->data);
+	stream->data = vmap(stream->pagelist, stream->nr_pages);
+	if (!stream->data)
+		return -ENOMEM;
+
+	return 0;
+}
+
+void *lu_stream_reserve(struct lu_stream *stream, size_t size)
+{
+	int nr_pages = (stream->len + size + PAGE_SIZE - 1) >> PAGE_SHIFT;
+
+	if (stream->nr_pages < nr_pages && lu_stream_extend(stream, nr_pages))
+		return NULL;
+
+	return stream->data + stream->len;
+}
+
+void lu_stream_end_reservation(struct lu_stream *stream, size_t size)
+{
+	stream->len += size;
+}
+
+int lu_stream_append(struct lu_stream *stream, void *data, size_t size)
+{
+	void *p = lu_stream_reserve(stream, size);
+	if (!p)
+		return -ENOMEM;
+	memcpy(p, data, size);
+	lu_stream_end_reservation(stream, size);
+
+	return 0;
+}
+
+void lu_stream_free(struct lu_stream *stream)
+{
+	int order = get_order_from_bytes((stream->nr_pages + 1) * sizeof(mfn_t));
+
+	if (stream->pagelist)
+		free_xenheap_pages(stream->pagelist, order);
+	if (stream->data)
+		vunmap(stream->data);
+}
diff --git a/xen/include/xen/lu.h b/xen/include/xen/lu.h
new file mode 100644
index 0000000000..cb2f1dbe06
--- /dev/null
+++ b/xen/include/xen/lu.h
@@ -0,0 +1,12 @@
+
+struct lu_stream {
+	mfn_t *pagelist;
+	size_t len;
+	int nr_pages;
+	char *data;
+};
+
+void *lu_stream_reserve(struct lu_stream *stream, size_t size);
+void lu_stream_end_reservation(struct lu_stream *stream, size_t size);
+int lu_stream_append(struct lu_stream *stream, void *data, size_t size);
+void lu_stream_free(struct lu_stream *stream);
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 09/14] Add basic lu_save_all() shell

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

---
 xen/common/kexec.c     |  6 ++++++
 xen/common/lu/Makefile |  2 +-
 xen/common/lu/save.c   | 45 ++++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/lu.h   |  3 +++
 4 files changed, 55 insertions(+), 1 deletion(-)
 create mode 100644 xen/common/lu/save.c

diff --git a/xen/common/kexec.c b/xen/common/kexec.c
index a78aa4f5b0..658fe3d3d4 100644
--- a/xen/common/kexec.c
+++ b/xen/common/kexec.c
@@ -29,6 +29,7 @@
 #include <public/elfnote.h>
 #include <xsm/xsm.h>
 #include <xen/cpu.h>
+#include <xen/lu.h>
 #ifdef CONFIG_COMPAT
 #include <compat/kexec.h>
 #endif
@@ -407,6 +408,11 @@ static long kexec_reboot(void *_image)
 static long kexec_live_update(void *_image)
 {
     struct kexec_image *image = _image;
+    int ret;
+
+    ret = lu_save_all(image);
+    if (ret)
+        return ret;
 
     kexecing = TRUE;
 
diff --git a/xen/common/lu/Makefile b/xen/common/lu/Makefile
index 68991b3ca4..7b7d975f65 100644
--- a/xen/common/lu/Makefile
+++ b/xen/common/lu/Makefile
@@ -1 +1 @@
-obj-y += stream.o
+obj-y += stream.o save.o
diff --git a/xen/common/lu/save.c b/xen/common/lu/save.c
new file mode 100644
index 0000000000..c767abd8f4
--- /dev/null
+++ b/xen/common/lu/save.c
@@ -0,0 +1,45 @@
+
+#include <xen/types.h>
+#include <xen/vmap.h>
+#include <xen/lu.h>
+#include <xen/kimage.h>
+#include <xen/sched.h>
+
+int lu_save_global(struct lu_stream *stream)
+{
+	return 0;
+}
+
+
+int lu_save_domain(struct lu_stream *stream, struct domain *d)
+{
+	return 0;
+}
+
+int lu_save_all(struct kexec_image *image)
+{
+	struct lu_stream stream;
+	struct domain *d;
+	int ret;
+
+	memset(&stream, 0, sizeof(stream));
+
+	ret = lu_save_global(&stream);
+
+	for_each_domain ( d ) {
+		if (ret)
+			break;
+
+		ret = lu_save_domain(&stream, d);
+	}
+
+	if (!ret)
+		ret = kimage_add_live_update_data(image,
+						  virt_to_mfn(stream.pagelist),
+						  stream.nr_pages);
+
+	if (ret)
+		lu_stream_free(&stream);
+
+	return ret;
+}
diff --git a/xen/include/xen/lu.h b/xen/include/xen/lu.h
index cb2f1dbe06..6e8377b6c3 100644
--- a/xen/include/xen/lu.h
+++ b/xen/include/xen/lu.h
@@ -10,3 +10,6 @@ void *lu_stream_reserve(struct lu_stream *stream, size_t size);
 void lu_stream_end_reservation(struct lu_stream *stream, size_t size);
 int lu_stream_append(struct lu_stream *stream, void *data, size_t size);
 void lu_stream_free(struct lu_stream *stream);
+
+struct kexec_image;
+int lu_save_all(struct kexec_image *image);
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 10/14] Put *something* into LU data...

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 xen/common/lu/save.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/common/lu/save.c b/xen/common/lu/save.c
index c767abd8f4..24d727ff47 100644
--- a/xen/common/lu/save.c
+++ b/xen/common/lu/save.c
@@ -7,7 +7,7 @@
 
 int lu_save_global(struct lu_stream *stream)
 {
-	return 0;
+	return lu_stream_append(stream, "Hello World!", 12);
 }
 
 
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 11/14] Don't panic if no multiboot modules are provided on live update boot

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 xen/arch/x86/setup.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 521946dc27..bd65d6bf5d 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -883,9 +883,14 @@ void __init noreturn __start_xen(unsigned long mbi_p)
     printk(" Found %d EDD information structures\n",
            bootsym(boot_edd_info_nr));
 
-    /* Check that we have at least one Multiboot module. */
-    if ( !(mbi->flags & MBI_MODULES) || (mbi->mods_count == 0) )
-        panic("dom0 kernel not specified. Check bootloader configuration\n");
+    if ( lu_data ) {
+        if ( (mbi->flags & MBI_MODULES) && (mbi->mods_count > 0) )
+            panic("Multiboot modules provided with live update data\n");
+    } else {
+        /* Check that we have at least one Multiboot module. */
+        if ( !(mbi->flags & MBI_MODULES) || (mbi->mods_count == 0) )
+            panic("dom0 kernel not specified. Check bootloader configuration\n");
+    }
 
     /* Check that we don't have a silly number of modules. */
     if ( mbi->mods_count > sizeof(module_map) * 8 )
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 12/14] Don't add bad pages above HYPERVISOR_VIRT_END to the domheap

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 xen/common/page_alloc.c | 83 +++++++++++++++++++++++++++++++++++++++--
 1 file changed, 80 insertions(+), 3 deletions(-)

diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 20ef25d45a..2a20c12abb 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1758,6 +1758,18 @@ int query_page_offline(mfn_t mfn, uint32_t *status)
     return 0;
 }
 
+static unsigned long contig_avail_pages(struct page_info *pg, unsigned long max_pages)
+{
+    unsigned long i;
+
+    for ( i = 0 ; i < max_pages; i++)
+    {
+        if ( pg[i].count_info & (PGC_broken | PGC_allocated) )
+            break;
+    }
+    return i;
+}
+
 /*
  * Hand the specified arbitrary page range to the specified heap zone
  * checking the node_id of the previous page.  If they differ and the
@@ -1799,18 +1811,24 @@ static void init_heap_pages(
     {
         unsigned int nid = phys_to_nid(page_to_maddr(pg+i));
 
+        /* If the (first) page is already marked bad, or allocated in advance
+         * due to live update, don't add it to the heap. */
+        if (pg[i].count_info & (PGC_broken | PGC_allocated))
+            continue;
+
         if ( unlikely(!avail[nid]) )
         {
+            unsigned long contig_nr_pages = contig_avail_pages(pg + i, nr_pages);
             unsigned long s = mfn_x(page_to_mfn(pg + i));
-            unsigned long e = mfn_x(mfn_add(page_to_mfn(pg + nr_pages - 1), 1));
+            unsigned long e = mfn_x(mfn_add(page_to_mfn(pg + contig_nr_pages - 1), 1));
             bool use_tail = (nid == phys_to_nid(pfn_to_paddr(e - 1))) &&
                             !(s & ((1UL << MAX_ORDER) - 1)) &&
                             (find_first_set_bit(e) <= find_first_set_bit(s));
             unsigned long n;
 
-            n = init_node_heap(nid, mfn_x(page_to_mfn(pg + i)), nr_pages - i,
+            n = init_node_heap(nid, mfn_x(page_to_mfn(pg + i)), contig_nr_pages - i,
                                &use_tail);
-            BUG_ON(i + n > nr_pages);
+            BUG_ON(i + n > contig_nr_pages);
             if ( n && !use_tail )
             {
                 i += n - 1;
@@ -1846,6 +1864,63 @@ static unsigned long avail_heap_pages(
     return free_pages;
 }
 
+static void mark_bad_pages(void)
+{
+    unsigned long bad_spfn, bad_epfn;
+    const char *p;
+    struct page_info *pg;
+#ifdef CONFIG_X86
+    const struct platform_bad_page *badpage;
+    unsigned int i, j, array_size;
+
+    badpage = get_platform_badpages(&array_size);
+    if ( badpage )
+    {
+        for ( i = 0; i < array_size; i++ )
+        {
+            for ( j = 0; j < 1UL << badpage->order; j++ )
+            {
+                if ( mfn_valid(badpage->mfn + j) )
+                {
+                    pg = mfn_to_page(badpage->mfn + j);
+                    pg->count_info |= PGC_broken;
+                    page_list_add_tail(pg, &page_broken_list);
+                }
+            }
+        }
+    }
+#endif
+
+    /* Check new pages against the bad-page list. */
+    p = opt_badpage;
+    while ( *p != '\0' )
+    {
+        bad_spfn = simple_strtoul(p, &p, 0);
+        bad_epfn = bad_spfn;
+
+        if ( *p == '-' )
+        {
+            p++;
+            bad_epfn = simple_strtoul(p, &p, 0);
+            if ( bad_epfn < bad_spfn )
+                bad_epfn = bad_spfn;
+        }
+
+        if ( *p == ',' )
+            p++;
+        else if ( *p != '\0' )
+            break;
+
+        while ( mfn_valid(_mfn(bad_spfn)) && bad_spfn < bad_epfn )
+        {
+            pg = mfn_to_page(_mfn(bad_spfn));
+            pg->count_info |= PGC_broken;
+            page_list_add_tail(pg, &page_broken_list);
+            bad_spfn++;
+        }
+    }
+}
+
 void __init end_boot_allocator(void)
 {
     unsigned int i;
@@ -1870,6 +1945,8 @@ void __init end_boot_allocator(void)
     }
     nr_bootmem_regions = 0;
 
+    mark_bad_pages();
+
     if ( !dma_bitsize && (num_online_nodes() > 1) )
         dma_bitsize = arch_get_dma_bitsize();
 
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 13/14] Basic shell of lu_reserve_all() from breadcrumb at boot

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

This mostly just reserves pages, hence the name. The rest of the actual
restoration will happen later, after the heap allocator works.

Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
---
 xen/arch/x86/setup.c    | 20 +++++++++++++++++++
 xen/common/lu/Makefile  |  2 +-
 xen/common/lu/restore.c | 44 +++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/lu.h    |  1 +
 4 files changed, 66 insertions(+), 1 deletion(-)
 create mode 100644 xen/common/lu/restore.c

diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index bd65d6bf5d..af874ee8b5 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -25,6 +25,7 @@
 #include <xen/nodemask.h>
 #include <xen/virtual_region.h>
 #include <xen/watchdog.h>
+#include <xen/lu.h>
 #include <public/version.h>
 #include <compat/platform.h>
 #include <compat/xen.h>
@@ -739,6 +740,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
         .max_maptrack_frames = -1,
     };
     const char *hypervisor_name;
+    uint64_t lu_mfnlist_phys = 0, lu_nr_pages = 0;
 
     /* Critical region without IDT or TSS.  Any fault is deadly! */
 
@@ -1336,6 +1338,21 @@ void __init noreturn __start_xen(unsigned long mbi_p)
     if ( !xen_phys_start )
         panic("Not enough memory to relocate Xen\n");
 
+    if ( lu_data )
+    {
+        uint64_t *breadcrumb = maddr_to_virt(lu_data);
+
+        lu_mfnlist_phys = breadcrumb[1];
+        lu_nr_pages = breadcrumb[2] >> PAGE_SHIFT;
+
+        if ( breadcrumb[0] == LIVE_UPDATE_MAGIC && lu_nr_pages) {
+            printk("%ld pages of live update data at 0x%lx\n", lu_nr_pages, lu_mfnlist_phys);
+        } else {
+            panic("Live update breadcrumb not found: %lx %lx %lx at %lx\n",
+                  breadcrumb[0], breadcrumb[1], breadcrumb[2], lu_data);
+        }
+    }
+
     if ( lu_bootmem_start )
     {
         if ( !lu_reserved )
@@ -1571,6 +1588,9 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 
     numa_initmem_init(0, raw_max_page);
 
+    if ( lu_data )
+        lu_reserve_pages(lu_mfnlist_phys, lu_nr_pages);
+
     if ( lu_bootmem_start )
     {
         unsigned long limit = virt_to_mfn(HYPERVISOR_VIRT_END - 1);
diff --git a/xen/common/lu/Makefile b/xen/common/lu/Makefile
index 7b7d975f65..592c72e1ec 100644
--- a/xen/common/lu/Makefile
+++ b/xen/common/lu/Makefile
@@ -1 +1 @@
-obj-y += stream.o save.o
+obj-y += stream.o save.o restore.o
diff --git a/xen/common/lu/restore.c b/xen/common/lu/restore.c
new file mode 100644
index 0000000000..7f40513ef9
--- /dev/null
+++ b/xen/common/lu/restore.c
@@ -0,0 +1,44 @@
+#include <xen/types.h>
+#include <xen/vmap.h>
+#include <xen/lu.h>
+#include <xen/sched.h>
+
+#define MFNS_PER_PAGE (PAGE_SIZE / sizeof(mfn_t))
+
+/* Returns void. There's not a lot we can do if this fails except panic. */
+void lu_reserve_pages(paddr_t mfns_p, int nr_pages)
+{
+    unsigned int index_pages = (nr_pages + MFNS_PER_PAGE - 1) / MFNS_PER_PAGE;
+    unsigned char *p;
+    mfn_t *mfns;
+    int i;
+
+    /* Would like to vmap(). Too early. */
+    mfns = __va(mfns_p);
+    map_pages_to_xen((unsigned long)mfns, maddr_to_mfn(mfns_p), index_pages, PAGE_HYPERVISOR);
+
+    for ( i = 0; i < index_pages; i++ )
+    {
+        maddr_to_page(mfns_p + (i << PAGE_SHIFT))->count_info |= PGC_allocated;
+    }
+
+    for ( i = 0; i < nr_pages; i++ )
+    {
+        if (!mfn_valid(mfns[i]))
+            panic("Invalid MFN %lx in live update stream\n", mfn_x(mfns[i]));
+
+        mfn_to_page(mfns[i])->count_info |= PGC_allocated;
+	}
+
+    /* XX: Consume the stream of data and mark all domain pages as allocated too. */
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/xen/lu.h b/xen/include/xen/lu.h
index 6e8377b6c3..496db9bfe1 100644
--- a/xen/include/xen/lu.h
+++ b/xen/include/xen/lu.h
@@ -13,3 +13,4 @@ void lu_stream_free(struct lu_stream *stream);
 
 struct kexec_image;
 int lu_save_all(struct kexec_image *image);
+void lu_reserve_pages(paddr_t mfnlist, int nr_pages);
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC PATCH v2 14/14] debug hacks

[David Woodhouse]

From: David Woodhouse <dwmw@amazon.co.uk>

---
 xen/common/lu/restore.c | 7 +++++++
 xen/common/lu/stream.c  | 2 ++
 xen/common/page_alloc.c | 6 ++++++
 3 files changed, 15 insertions(+)

diff --git a/xen/common/lu/restore.c b/xen/common/lu/restore.c
index 7f40513ef9..fa168bdac1 100644
--- a/xen/common/lu/restore.c
+++ b/xen/common/lu/restore.c
@@ -31,6 +31,13 @@ void lu_reserve_pages(paddr_t mfns_p, int nr_pages)
 	}
 
     /* XX: Consume the stream of data and mark all domain pages as allocated too. */
+
+    printk("First live update data page at MFN %lx:\n", mfns[0]);
+    p = mfn_to_virt(mfns[0]);
+    map_pages_to_xen((unsigned long)p, mfns[0], 1, PAGE_HYPERVISOR);
+    for (i = 0; i < 16; i++)
+        printk(" %02x", p[i]);
+    printk("\n");
 }
 
 /*
diff --git a/xen/common/lu/stream.c b/xen/common/lu/stream.c
index 52030e8b2e..327c0396ff 100644
--- a/xen/common/lu/stream.c
+++ b/xen/common/lu/stream.c
@@ -65,6 +65,7 @@ static int lu_stream_extend(struct lu_stream *stream, int nr_pages)
 	if (stream->data)
 		vunmap(stream->data);
 	stream->data = vmap(stream->pagelist, stream->nr_pages);
+	printk("Mapped %d MFNs at %lx to %p\n", stream->nr_pages, stream->pagelist[0], stream->data);
 	if (!stream->data)
 		return -ENOMEM;
 
@@ -91,6 +92,7 @@ int lu_stream_append(struct lu_stream *stream, void *data, size_t size)
 	void *p = lu_stream_reserve(stream, size);
 	if (!p)
 		return -ENOMEM;
+	printk("Copy %ld starting %02x to %p\n", size, *(unsigned char *)data, p);
 	memcpy(p, data, size);
 	lu_stream_end_reservation(stream, size);
 
diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
index 2a20c12abb..a3668a517f 100644
--- a/xen/common/page_alloc.c
+++ b/xen/common/page_alloc.c
@@ -1765,7 +1765,10 @@ static unsigned long contig_avail_pages(struct page_info *pg, unsigned long max_
     for ( i = 0 ; i < max_pages; i++)
     {
         if ( pg[i].count_info & (PGC_broken | PGC_allocated) )
+        {
+            printk("Skipping reserved MFN %lx\n", mfn_x(page_to_mfn(&pg[i])));
             break;
+        }
     }
     return i;
 }
@@ -1814,7 +1817,10 @@ static void init_heap_pages(
         /* If the (first) page is already marked bad, or allocated in advance
          * due to live update, don't add it to the heap. */
         if (pg[i].count_info & (PGC_broken | PGC_allocated))
+        {
+            printk("Skipping reserved MFN %lx\n", mfn_x(page_to_mfn(&pg[i])));
             continue;
+        }
 
         if ( unlikely(!avail[nid]) )
         {
-- 
2.21.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel