[PATCH v2 00/16] Save/Restore Support for mini-OS PVH

[PATCH v2 00/16] Save/Restore Support for mini-OS PVH

[Bruno Alvisio]

Hi all,

I am sending the second revision for supporting save/restore in Mini-OS PVH. The 
branch can be found at: 

https://github.com/balvisio/mini-os/tree/feature/mini-os-suspend-support-submission-2

Feedback would be greatly appreciated.

Cheers,

Bruno

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 01/16] Save/Restore Support: Refactor HYPERVISOR_suspend hypercall

[Bruno Alvisio]

Directly using the SHUTDOWN_suspend macro as a parameter for the schedop
hypercall causes an error in the Xen hypercall handler. Also for consistency,
the SHUTDOWN_suspend param is wrapped in the sched_shutdown struct.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 include/x86/x86_32/hypercall-x86_32.h | 4 ++--
 include/x86/x86_64/hypercall-x86_64.h | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/include/x86/x86_32/hypercall-x86_32.h b/include/x86/x86_32/hypercall-x86_32.h
index 5c93464..70505a4 100644
--- a/include/x86/x86_32/hypercall-x86_32.h
+++ b/include/x86/x86_32/hypercall-x86_32.h
@@ -298,8 +298,8 @@ static inline int
 HYPERVISOR_suspend(
 	unsigned long srec)
 {
-	return _hypercall3(int, sched_op, SCHEDOP_shutdown,
-			   SHUTDOWN_suspend, srec);
+	struct sched_shutdown shutdown = { .reason = SHUTDOWN_suspend };
+	return _hypercall3(int, sched_op, SCHEDOP_shutdown, &shutdown, srec);
 }
 
 static inline int
diff --git a/include/x86/x86_64/hypercall-x86_64.h b/include/x86/x86_64/hypercall-x86_64.h
index 6171812..95f8ade 100644
--- a/include/x86/x86_64/hypercall-x86_64.h
+++ b/include/x86/x86_64/hypercall-x86_64.h
@@ -305,8 +305,8 @@ static inline int
 HYPERVISOR_suspend(
 	unsigned long srec)
 {
-	return _hypercall3(int, sched_op, SCHEDOP_shutdown,
-			   SHUTDOWN_suspend, srec);
+	struct sched_shutdown shutdown = { .reason = SHUTDOWN_suspend };
+	return _hypercall3(int, sched_op, SCHEDOP_shutdown, &shutdown, srec);
 }
 
 static inline int
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 02/16] Save/Restore Support: Refactor trap_init() and setup vector callbacks

[Bruno Alvisio]

Currently the setup of the IDT and the request to set the HVM vector callbacks
are performed both in the trap_init function.

As part of the post-suspend operation, the HVM vector callback needs to be setup
again while the IDT does not. Thus, the trap_init function is split into two
separate functions: trap_init (sets up IDT) and xen_callback_vector (sets the
HVM vector callback). During the post-suspend operations the xen_callback_vector
function will be invoked.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/traps.c | 17 +++++++++++------
 include/x86/os.h |  3 +++
 2 files changed, 14 insertions(+), 6 deletions(-)

diff --git a/arch/x86/traps.c b/arch/x86/traps.c
index aa17da3..a7388a5 100644
--- a/arch/x86/traps.c
+++ b/arch/x86/traps.c
@@ -389,6 +389,16 @@ static void setup_gate(unsigned int entry, void *addr, unsigned int dpl)
 #endif
 }
 
+void xen_callback_vector(void)
+{
+    if (hvm_set_parameter(HVM_PARAM_CALLBACK_IRQ,
+                         (2ULL << 56) | TRAP_xen_callback))
+    {
+        xprintk("Request for Xen HVM callback vector failed\n");
+        do_exit();
+    }
+}
+
 void trap_init(void)
 {
     setup_gate(TRAP_divide_error, &divide_error, 0);
@@ -415,12 +425,7 @@ void trap_init(void)
     gdt[GDTE_TSS] = (typeof(*gdt))INIT_GDTE((unsigned long)&tss, 0x67, 0x89);
     asm volatile ("ltr %w0" :: "rm" (GDTE_TSS * 8));
 
-    if ( hvm_set_parameter(HVM_PARAM_CALLBACK_IRQ,
-                           (2ULL << 56) | TRAP_xen_callback) )
-    {
-        xprintk("Request for Xen HVM callback vector failed\n");
-        do_exit();
-    }
+    xen_callback_vector();
 }
 
 void trap_fini(void)
diff --git a/include/x86/os.h b/include/x86/os.h
index fbc2eeb..d155914 100644
--- a/include/x86/os.h
+++ b/include/x86/os.h
@@ -67,6 +67,9 @@ extern shared_info_t *HYPERVISOR_shared_info;
 
 void trap_init(void);
 void trap_fini(void);
+#ifndef CONFIG_PARAVIRT
+void xen_callback_vector(void);
+#endif
 
 void arch_fini(void);
 
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 03/16] Save/Restore Support: Declare kernel and arch pre/post suspend functions

[Bruno Alvisio]

For mini-OS to support suspend and restore, the kernel will have to suspend
different modules such as xenbus, console, irq, etc. During save/restore the
kernel and arch pre_suspend and post_suspend functions will be invoked to
suspend/resume each of the modules.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/setup.c | 10 ++++++++++
 include/kernel.h |  2 ++
 include/x86/os.h |  4 ++--
 kernel.c         | 10 ++++++++++
 4 files changed, 24 insertions(+), 2 deletions(-)

diff --git a/arch/x86/setup.c b/arch/x86/setup.c
index 5278227..3dd86f9 100644
--- a/arch/x86/setup.c
+++ b/arch/x86/setup.c
@@ -204,6 +204,16 @@ arch_init(void *par)
 	start_kernel();
 }
 
+void arch_pre_suspend(void)
+{
+
+}
+
+void arch_post_suspend(int canceled)
+{
+
+}
+
 void
 arch_fini(void)
 {
diff --git a/include/kernel.h b/include/kernel.h
index d37ddda..161d757 100644
--- a/include/kernel.h
+++ b/include/kernel.h
@@ -5,6 +5,8 @@
 extern char cmdline[MAX_CMDLINE_SIZE];
 
 void start_kernel(void);
+void pre_suspend(void);
+void post_suspend(int canceled);
 void do_exit(void) __attribute__((noreturn));
 void arch_do_exit(void);
 void stop_kernel(void);
diff --git a/include/x86/os.h b/include/x86/os.h
index d155914..a73b63e 100644
--- a/include/x86/os.h
+++ b/include/x86/os.h
@@ -71,10 +71,10 @@ void trap_fini(void);
 void xen_callback_vector(void);
 #endif
 
+void arch_pre_suspend(void);
+void arch_post_suspend(int canceled);
 void arch_fini(void);
 
-
-
 #ifdef CONFIG_PARAVIRT
 
 /* 
diff --git a/kernel.c b/kernel.c
index 0d84a9b..90c865a 100644
--- a/kernel.c
+++ b/kernel.c
@@ -155,6 +155,16 @@ void start_kernel(void)
     run_idle_thread();
 }
 
+void pre_suspend(void)
+{
+
+}
+
+void post_suspend(int canceled)
+{
+
+}
+
 void stop_kernel(void)
 {
     /* TODO: fs import */
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 04/16] Save/Restore Support: Add xenbus_release_wait_for_watch

[Bruno Alvisio]

xenbus_release_wait_for_watch generates a fake event to trigger make
xenbus_wait_for_watch return. This is necessary to wake up waiting threads.

release_xenbus_id additionally checks if the number of requests == 0 to wake
up the 'waiting' suspend xenbus thread.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
Changed since v1:
  * Added doc for change in release_xenbus_id
---
 include/xenbus.h |  1 +
 xenbus/xenbus.c  | 10 +++++++++-
 2 files changed, 10 insertions(+), 1 deletion(-)

diff --git a/include/xenbus.h b/include/xenbus.h
index 12391b9..b2d5072 100644
--- a/include/xenbus.h
+++ b/include/xenbus.h
@@ -42,6 +42,7 @@ char *xenbus_unwatch_path_token(xenbus_transaction_t xbt, const char *path, cons
 extern struct wait_queue_head xenbus_watch_queue;
 void xenbus_wait_for_watch(xenbus_event_queue *queue);
 char **xenbus_wait_for_watch_return(xenbus_event_queue *queue);
+void xenbus_release_wait_for_watch(xenbus_event_queue *queue);
 char* xenbus_wait_for_value(const char *path, const char *value, xenbus_event_queue *queue);
 char *xenbus_wait_for_state_change(const char* path, XenbusState *state, xenbus_event_queue *queue);
 char *xenbus_switch_state(xenbus_transaction_t xbt, const char* path, XenbusState state);
diff --git a/xenbus/xenbus.c b/xenbus/xenbus.c
index 636786c..c2d2bd1 100644
--- a/xenbus/xenbus.c
+++ b/xenbus/xenbus.c
@@ -129,6 +129,14 @@ void xenbus_wait_for_watch(xenbus_event_queue *queue)
         printk("unexpected path returned by watch\n");
 }
 
+void xenbus_release_wait_for_watch(xenbus_event_queue *queue)
+{
+    struct xenbus_event *event = malloc(sizeof(*event));
+    event->next = *queue;
+    *queue = event;
+    wake_up(&xenbus_watch_queue);
+}
+
 char* xenbus_wait_for_value(const char* path, const char* value, xenbus_event_queue *queue)
 {
     if (!queue)
@@ -318,7 +326,7 @@ static void release_xenbus_id(int id)
     req_info[id].in_use = 0;
     nr_live_reqs--;
     req_info[id].in_use = 0;
-    if (nr_live_reqs == NR_REQS - 1)
+    if (nr_live_reqs == 0 || nr_live_reqs == NR_REQS - 1)
         wake_up(&req_wq);
     spin_unlock(&req_lock);
 }
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 05/16] Save/Restore Support: Add kernel shutdown logic to shutdown.c

[Bruno Alvisio]

Created shutdown.c for the shutdown thread and all the shutdown related
functions.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
---
Changesd since v1:
   * Updated license to a BSD 3-clause. This license was taken
from the updated original file. (Repo: sysml/mini-os)
---
 Makefile           |   1 +
 include/shutdown.h |  11 ++++
 shutdown.c         | 188 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 200 insertions(+)
 create mode 100644 include/shutdown.h
 create mode 100644 shutdown.c

diff --git a/Makefile b/Makefile
index 88315c4..6a05de6 100644
--- a/Makefile
+++ b/Makefile
@@ -53,6 +53,7 @@ src-y += mm.c
 src-$(CONFIG_NETFRONT) += netfront.c
 src-$(CONFIG_PCIFRONT) += pcifront.c
 src-y += sched.c
+src-y += shutdown.c
 src-$(CONFIG_TEST) += test.c
 src-$(CONFIG_BALLOON) += balloon.c
 
diff --git a/include/shutdown.h b/include/shutdown.h
new file mode 100644
index 0000000..a5ec019
--- /dev/null
+++ b/include/shutdown.h
@@ -0,0 +1,11 @@
+#ifndef _SHUTDOWN_H_
+#define _SHUTDOWN_H_
+
+#include <mini-os/hypervisor.h>
+
+void init_shutdown(start_info_t *si);
+
+void kernel_shutdown(int reason) __attribute__((noreturn));
+void kernel_suspend(void);
+
+#endif
diff --git a/shutdown.c b/shutdown.c
new file mode 100644
index 0000000..aba146e
--- /dev/null
+++ b/shutdown.c
@@ -0,0 +1,188 @@
+/*
+ *          MiniOS
+ *
+ *   file: fromdevice.cc
+ *
+ * Authors: Joao Martins <joao.martins@neclab.eu>
+ *
+ *
+ * Copyright (c) 2014, NEC Europe Ltd., NEC Corporation. All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the copyright holder nor the names of its
+ *    contributors may be used to endorse or promote products derived from
+ *    this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+ * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+ * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+ * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+ * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+ * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+ * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+ * POSSIBILITY OF SUCH DAMAGE.
+ *
+ * THIS HEADER MAY NOT BE EXTRACTED OR MODIFIED IN ANY WAY.
+ */
+
+#include <mini-os/os.h>
+#include <mini-os/events.h>
+#include <mini-os/kernel.h>
+#include <mini-os/sched.h>
+#include <mini-os/shutdown.h>
+#include <mini-os/lib.h>
+#include <mini-os/xenbus.h>
+#include <mini-os/xmalloc.h>
+
+
+static start_info_t *start_info_ptr;
+
+static const char *path = "control/shutdown";
+static const char *token = "control/shutdown";
+static xenbus_event_queue events = NULL;
+static int end_shutdown_thread = 0;
+
+#ifdef CONFIG_XENBUS
+/* This should be overridden by the application we are linked against. */
+__attribute__((weak)) void app_shutdown(unsigned reason)
+{
+    printk("Shutdown requested: %d\n", reason);
+    if (reason == SHUTDOWN_suspend) {
+        kernel_suspend();
+    } else {
+        struct sched_shutdown sched_shutdown = { .reason = reason };
+        HYPERVISOR_sched_op(SCHEDOP_shutdown, &sched_shutdown);
+    }
+}
+
+static void shutdown_thread(void *p)
+{
+    char *shutdown, *err;
+    unsigned int shutdown_reason;
+
+    xenbus_watch_path_token(XBT_NIL, path, token, &events);
+
+    for ( ;; ) {
+        xenbus_wait_for_watch(&events);
+        if ((err = xenbus_read(XBT_NIL, path, &shutdown))) {
+            free(err);
+            do_exit();
+        }
+
+        if (end_shutdown_thread)
+            break;
+
+        if (!strcmp(shutdown, "")) {
+            /* Avoid spurious event on xenbus */
+            /* FIXME: investigate the reason of the spurious event */
+            free(shutdown);
+            continue;
+        } else if (!strcmp(shutdown, "poweroff")) {
+            shutdown_reason = SHUTDOWN_poweroff;
+        } else if (!strcmp(shutdown, "reboot")) {
+            shutdown_reason = SHUTDOWN_reboot;
+        } else if (!strcmp(shutdown, "suspend")) {
+            shutdown_reason = SHUTDOWN_suspend;
+        } else {
+            shutdown_reason = SHUTDOWN_crash;
+        }
+        free(shutdown);
+
+        /* Acknowledge shutdown request */
+        if ((err = xenbus_write(XBT_NIL, path, ""))) {
+            free(err);
+            do_exit();
+        }
+
+        app_shutdown(shutdown_reason);
+    }
+}
+#endif
+
+static void fini_shutdown(void)
+{
+    char *err;
+
+    end_shutdown_thread = 1;
+    xenbus_release_wait_for_watch(&events);
+    err = xenbus_unwatch_path_token(XBT_NIL, path, token);
+    if (err) {
+        free(err);
+        do_exit();
+    }
+}
+
+void init_shutdown(start_info_t *si)
+{
+    start_info_ptr = si;
+
+    end_shutdown_thread = 0;
+    create_thread("shutdown", shutdown_thread, NULL);
+}
+
+void kernel_shutdown(int reason)
+{
+    char* reason_str = NULL;
+
+    switch(reason) {
+        case SHUTDOWN_poweroff:
+            reason_str = "poweroff";
+            break;
+        case SHUTDOWN_reboot:
+            reason_str = "reboot";
+            break;
+        case SHUTDOWN_crash:
+            reason_str = "crash";
+            break;
+        default:
+            do_exit();
+            break;
+    }
+
+    printk("MiniOS will shutdown (reason = %s) ...\n", reason_str);
+
+    fini_shutdown();
+
+    stop_kernel();
+
+    for ( ;; ) {
+        struct sched_shutdown sched_shutdown = { .reason = reason };
+        HYPERVISOR_sched_op(SCHEDOP_shutdown, &sched_shutdown);
+    }
+}
+
+void kernel_suspend(void)
+{
+    int rc;
+
+    printk("MiniOS will suspend ...\n");
+
+    pre_suspend();
+    arch_pre_suspend();
+
+    /*
+     * This hypercall returns 1 if the suspend
+     * was cancelled and 0 if resuming in a new domain
+     */
+    rc = HYPERVISOR_suspend(virt_to_mfn(start_info_ptr));
+
+    arch_post_suspend(rc);
+    post_suspend(rc);
+
+    if (rc) {
+        printk("MiniOS suspend canceled!");
+    } else {
+        printk("MiniOS resumed from suspend!\n");
+    }
+}
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 06/16] Save/Restore Support: Moved shutdown thread to shutdown.c

[Bruno Alvisio]

The shutdown thread present in kernel.c was removed and now the thread in
shutdown.c is created instead.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/setup.c |  2 +-
 include/kernel.h |  2 +-
 kernel.c         | 50 ++++++--------------------------------------------
 3 files changed, 8 insertions(+), 46 deletions(-)

diff --git a/arch/x86/setup.c b/arch/x86/setup.c
index 3dd86f9..31fa2c6 100644
--- a/arch/x86/setup.c
+++ b/arch/x86/setup.c
@@ -201,7 +201,7 @@ arch_init(void *par)
 	memcpy(&start_info, par, sizeof(start_info));
 #endif
 
-	start_kernel();
+	start_kernel((start_info_t *)par);
 }
 
 void arch_pre_suspend(void)
diff --git a/include/kernel.h b/include/kernel.h
index 161d757..742abf5 100644
--- a/include/kernel.h
+++ b/include/kernel.h
@@ -4,7 +4,7 @@
 #define MAX_CMDLINE_SIZE 1024
 extern char cmdline[MAX_CMDLINE_SIZE];
 
-void start_kernel(void);
+void start_kernel(void* par);
 void pre_suspend(void);
 void post_suspend(int canceled);
 void do_exit(void) __attribute__((noreturn));
diff --git a/kernel.c b/kernel.c
index 90c865a..1cd40e8 100644
--- a/kernel.c
+++ b/kernel.c
@@ -42,6 +42,9 @@
 #include <mini-os/blkfront.h>
 #include <mini-os/fbfront.h>
 #include <mini-os/pcifront.h>
+#ifdef CONFIG_XENBUS
+#include <mini-os/shutdown.h>
+#endif
 #include <mini-os/xmalloc.h>
 #include <fcntl.h>
 #include <xen/features.h>
@@ -66,48 +69,6 @@ void setup_xen_features(void)
     }
 }
 
-#ifdef CONFIG_XENBUS
-/* This should be overridden by the application we are linked against. */
-__attribute__((weak)) void app_shutdown(unsigned reason)
-{
-    struct sched_shutdown sched_shutdown = { .reason = reason };
-    printk("Shutdown requested: %d\n", reason);
-    HYPERVISOR_sched_op(SCHEDOP_shutdown, &sched_shutdown);
-}
-
-static void shutdown_thread(void *p)
-{
-    const char *path = "control/shutdown";
-    const char *token = path;
-    xenbus_event_queue events = NULL;
-    char *shutdown = NULL, *err;
-    unsigned int shutdown_reason;
-    xenbus_watch_path_token(XBT_NIL, path, token, &events);
-    while ((err = xenbus_read(XBT_NIL, path, &shutdown)) != NULL || !strcmp(shutdown, ""))
-    {
-        free(err);
-        free(shutdown);
-        shutdown = NULL;
-        xenbus_wait_for_watch(&events);
-    }
-    err = xenbus_unwatch_path_token(XBT_NIL, path, token);
-    free(err);
-    err = xenbus_write(XBT_NIL, path, "");
-    free(err);
-    printk("Shutting down (%s)\n", shutdown);
-
-    if (!strcmp(shutdown, "poweroff"))
-        shutdown_reason = SHUTDOWN_poweroff;
-    else if (!strcmp(shutdown, "reboot"))
-        shutdown_reason = SHUTDOWN_reboot;
-    else
-        /* Unknown */
-        shutdown_reason = SHUTDOWN_crash;
-    app_shutdown(shutdown_reason);
-    free(shutdown);
-}
-#endif
-
 
 /* This should be overridden by the application we are linked against. */
 __attribute__((weak)) int app_main(void *p)
@@ -116,7 +77,7 @@ __attribute__((weak)) int app_main(void *p)
     return 0;
 }
 
-void start_kernel(void)
+void start_kernel(void* par)
 {
     /* Set up events. */
     init_events();
@@ -145,7 +106,8 @@ void start_kernel(void)
     init_xenbus();
 
 #ifdef CONFIG_XENBUS
-    create_thread("shutdown", shutdown_thread, NULL);
+    /* Init shutdown thread */
+    init_shutdown((start_info_t *)par);
 #endif
 
     /* Call (possibly overridden) app_main() */
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 07/16] Save/Restore Support: Add unmap_shared_info

[Bruno Alvisio]

This function is necessary as part of the pre-suspend operation.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
Changed since v1:
 * Changed HYPERVISOR_shared_info for shared_info
---
 arch/x86/setup.c     | 12 ++++++++++++
 hypervisor.c         | 12 ++++++++++++
 include/hypervisor.h |  1 +
 3 files changed, 25 insertions(+)

diff --git a/arch/x86/setup.c b/arch/x86/setup.c
index 31fa2c6..b6e0541 100644
--- a/arch/x86/setup.c
+++ b/arch/x86/setup.c
@@ -93,6 +93,18 @@ shared_info_t *map_shared_info(void *p)
     return (shared_info_t *)shared_info;
 }
 
+void unmap_shared_info(void)
+{
+    int rc;
+
+    if ( (rc = HYPERVISOR_update_va_mapping((unsigned long)shared_info,
+            __pte((virt_to_mfn(shared_info)<<L1_PAGETABLE_SHIFT)| L1_PROT), UVMF_INVLPG)) )
+    {
+        printk("Failed to unmap shared_info page!! rc=%d\n", rc);
+        do_exit();
+    }
+}
+
 static void get_cmdline(void *p)
 {
     start_info_t *si = p;
diff --git a/hypervisor.c b/hypervisor.c
index 1647121..d3857e7 100644
--- a/hypervisor.c
+++ b/hypervisor.c
@@ -78,6 +78,18 @@ shared_info_t *map_shared_info(void *p)
 
     return &shared_info;
 }
+
+void unmap_shared_info(void)
+{
+    struct xen_remove_from_physmap xrtp;
+
+    xrtp.domid = DOMID_SELF;
+    xrtp.gpfn = virt_to_pfn(&shared_info);
+    if ( HYPERVISOR_memory_op(XENMEM_remove_from_physmap, &xrtp) != 0 )
+        BUG();
+
+    return;
+}
 #endif
 
 void do_hypervisor_callback(struct pt_regs *regs)
diff --git a/include/hypervisor.h b/include/hypervisor.h
index f3b1f3c..1d09271 100644
--- a/include/hypervisor.h
+++ b/include/hypervisor.h
@@ -43,6 +43,7 @@ int hvm_get_parameter(int idx, uint64_t *value);
 int hvm_set_parameter(int idx, uint64_t value);
 #endif
 shared_info_t *map_shared_info(void *p);
+void unmap_shared_info(void);
 void force_evtchn_callback(void);
 void do_hypervisor_callback(struct pt_regs *regs);
 void mask_evtchn(uint32_t port);
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 08/16] Save/Restore Support: Add arch_mm_pre|post_suspend

[Bruno Alvisio]

For PV guests the pagetables reference the real MFNs rather than PFNs, so when
the guest is resumed into a different area of a hosts memory, these will need to
be rewritten. Thus for PV guests the MFNs need to be replaced with PFNs:
canonicalization.

PVH guests are auto-translated so no memory operation is needed.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 arch/x86/mm.c         | 14 ++++++++++++++
 include/x86/arch_mm.h |  3 +++
 2 files changed, 17 insertions(+)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index 05ad029..1b163ac 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -848,6 +848,20 @@ void arch_init_p2m(unsigned long max_pfn)
 
     arch_remap_p2m(max_pfn);
 }
+
+void arch_mm_pre_suspend(void)
+{
+    //TODO: Canonicalize pagetables
+}
+
+void arch_mm_post_suspend(int canceled)
+{
+    //TODO: Locate pagetables and 'uncanonicalize' them
+}
+#else
+void arch_mm_pre_suspend(void){ }
+
+void arch_mm_post_suspend(int canceled){ }
 #endif
 
 void arch_init_mm(unsigned long* start_pfn_p, unsigned long* max_pfn_p)
diff --git a/include/x86/arch_mm.h b/include/x86/arch_mm.h
index ab8a53e..cbbeb21 100644
--- a/include/x86/arch_mm.h
+++ b/include/x86/arch_mm.h
@@ -279,6 +279,9 @@ pgentry_t *need_pgt(unsigned long addr);
 void arch_mm_preinit(void *p);
 unsigned long alloc_virt_kernel(unsigned n_pages);
 
+void arch_mm_pre_suspend(void);
+void arch_mm_post_suspend(int canceled);
+
 #ifndef CONFIG_PARAVIRT
 void arch_print_memmap(void);
 #endif
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 09/16] Save/Restore Support: Disable/enable IRQs during suspend/restore

[Bruno Alvisio]

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 kernel.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/kernel.c b/kernel.c
index 1cd40e8..782eb79 100644
--- a/kernel.c
+++ b/kernel.c
@@ -119,12 +119,12 @@ void start_kernel(void* par)
 
 void pre_suspend(void)
 {
-
+    local_irq_disable();
 }
 
 void post_suspend(int canceled)
 {
-
+    local_irq_enable();
 }
 
 void stop_kernel(void)
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 10/16] Save/Restore Support: Add suspend/resume support for timers

[Bruno Alvisio]

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
---
Changed since v1:
   * Removed resume/suspend_time() and used init/fini_time() instead
---
 arch/x86/time.c | 1 -
 kernel.c        | 4 ++++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/x86/time.c b/arch/x86/time.c
index 3658142..8077c80 100644
--- a/arch/x86/time.c
+++ b/arch/x86/time.c
@@ -233,7 +233,6 @@ static void timer_handler(evtchn_port_t ev, struct pt_regs *regs, void *ign)
 static evtchn_port_t port;
 void init_time(void)
 {
-    printk("Initialising timer interface\n");
     port = bind_virq(VIRQ_TIMER, &timer_handler, NULL);
     unmask_evtchn(port);
 }
diff --git a/kernel.c b/kernel.c
index 782eb79..3564af3 100644
--- a/kernel.c
+++ b/kernel.c
@@ -120,10 +120,14 @@ void start_kernel(void* par)
 void pre_suspend(void)
 {
     local_irq_disable();
+
+    fini_time();
 }
 
 void post_suspend(int canceled)
 {
+    init_time();
+
     local_irq_enable();
 }
 
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 11/16] Save/Restore Support: Add suspend/restore support for console

[Bruno Alvisio]

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 console/console.c      | 15 ++++++++-
 console/xenbus.c       |  3 +-
 console/xencons_ring.c | 83 +++++++++++++++++++++++++++++++-------------------
 include/console.h      |  6 +++-
 kernel.c               |  4 +++
 lib/sys.c              |  2 +-
 6 files changed, 77 insertions(+), 36 deletions(-)

diff --git a/console/console.c b/console/console.c
index 2e04552..9814506 100644
--- a/console/console.c
+++ b/console/console.c
@@ -52,6 +52,7 @@
 
 /* If console not initialised the printk will be sent to xen serial line 
    NOTE: you need to enable verbose in xen/Rules.mk for it to work. */
+static struct consfront_dev* xen_console = NULL;
 static int console_initialised = 0;
 
 __attribute__((weak)) void console_input(char * buf, unsigned len)
@@ -162,8 +163,20 @@ void xprintk(const char *fmt, ...)
 void init_console(void)
 {   
     printk("Initialising console ... ");
-    xencons_ring_init();    
+    xen_console = xencons_ring_init();
     console_initialised = 1;
     /* This is also required to notify the daemon */
     printk("done.\n");
 }
+
+void suspend_console(void)
+{
+    console_initialised = 0;
+    xencons_ring_fini(xen_console);
+}
+
+void resume_console(void)
+{
+    xencons_ring_resume(xen_console);
+    console_initialised = 1;
+}
\ No newline at end of file
diff --git a/console/xenbus.c b/console/xenbus.c
index 1c9a590..654b469 100644
--- a/console/xenbus.c
+++ b/console/xenbus.c
@@ -188,8 +188,7 @@ error:
     return NULL;
 }
 
-void fini_console(struct consfront_dev *dev)
+void fini_consfront(struct consfront_dev *dev)
 {
     if (dev) free_consfront(dev);
 }
-
diff --git a/console/xencons_ring.c b/console/xencons_ring.c
index dd64a41..b6db74e 100644
--- a/console/xencons_ring.c
+++ b/console/xencons_ring.c
@@ -19,6 +19,8 @@ DECLARE_WAIT_QUEUE_HEAD(console_queue);
 static struct xencons_interface *console_ring;
 uint32_t console_evtchn;
 
+static struct consfront_dev* resume_xen_console(struct consfront_dev* dev);
+
 #ifdef CONFIG_PARAVIRT
 void get_console(void *p)
 {
@@ -32,10 +34,12 @@ void get_console(void *p)
 {
     uint64_t v = -1;
 
-    hvm_get_parameter(HVM_PARAM_CONSOLE_EVTCHN, &v);
+    if (hvm_get_parameter(HVM_PARAM_CONSOLE_EVTCHN, &v))
+        BUG();
     console_evtchn = v;
 
-    hvm_get_parameter(HVM_PARAM_CONSOLE_PFN, &v);
+    if (hvm_get_parameter(HVM_PARAM_CONSOLE_PFN, &v))
+        BUG();
     console_ring = (struct xencons_interface *)map_frame_virt(v);
 }
 #endif
@@ -89,9 +93,7 @@ int xencons_ring_send(struct consfront_dev *dev, const char *data, unsigned len)
     notify_daemon(dev);
 
     return sent;
-}	
-
-
+}
 
 void console_handle_input(evtchn_port_t port, struct pt_regs *regs, void *data)
 {
@@ -177,41 +179,60 @@ int xencons_ring_recv(struct consfront_dev *dev, char *data, unsigned len)
 
 struct consfront_dev *xencons_ring_init(void)
 {
-	int err;
-	struct consfront_dev *dev;
+    struct consfront_dev *dev;
 
-	if (!console_evtchn)
-		return 0;
+    if (!console_evtchn)
+        return 0;
 
-	dev = malloc(sizeof(struct consfront_dev));
-	memset(dev, 0, sizeof(struct consfront_dev));
-	dev->nodename = "device/console";
-	dev->dom = 0;
-	dev->backend = 0;
-	dev->ring_ref = 0;
+    dev = malloc(sizeof(struct consfront_dev));
+    memset(dev, 0, sizeof(struct consfront_dev));
+    dev->nodename = "device/console";
+    dev->dom = 0;
+    dev->backend = 0;
+    dev->ring_ref = 0;
 
 #ifdef HAVE_LIBC
-	dev->fd = -1;
+    dev->fd = -1;
 #endif
-	dev->evtchn = console_evtchn;
-	dev->ring = xencons_interface();
-
-	err = bind_evtchn(dev->evtchn, console_handle_input, dev);
-	if (err <= 0) {
-		printk("XEN console request chn bind failed %i\n", err);
-                free(dev);
-		return NULL;
-	}
-        unmask_evtchn(dev->evtchn);
 
-	/* In case we have in-flight data after save/restore... */
-	notify_daemon(dev);
+    return resume_xen_console(dev);
+}
+
+static struct consfront_dev* resume_xen_console(struct consfront_dev* dev)
+{
+    int err;
 
-	return dev;
+    dev->evtchn = console_evtchn;
+    dev->ring = xencons_interface();
+
+    err = bind_evtchn(dev->evtchn, console_handle_input, dev);
+    if (err <= 0) {
+        printk("XEN console request chn bind failed %i\n", err);
+        free(dev);
+        return NULL;
+    }
+    unmask_evtchn(dev->evtchn);
+
+    /* In case we have in-flight data after save/restore... */
+    notify_daemon(dev);
+
+    return dev;
 }
 
-void xencons_resume(void)
+void xencons_ring_fini(struct consfront_dev* dev)
 {
-	(void)xencons_ring_init();
+    if (dev)
+        mask_evtchn(dev->evtchn);
 }
 
+void xencons_ring_resume(struct consfront_dev* dev)
+{
+    if (dev) {
+#if CONFIG_PARAVIRT
+        get_console(&start_info);
+#else
+        get_console(0);
+#endif
+        resume_xen_console(dev);
+    }
+}
diff --git a/include/console.h b/include/console.h
index 539cccd..0d7bf07 100644
--- a/include/console.h
+++ b/include/console.h
@@ -78,11 +78,15 @@ void xencons_tx(void);
 void get_console(void *p);
 void init_console(void);
 void console_print(struct consfront_dev *dev, char *data, int length);
-void fini_console(struct consfront_dev *dev);
+void fini_consfront(struct consfront_dev *dev);
+void suspend_console(void);
+void resume_console(void);
 
 /* Low level functions defined in xencons_ring.c */
 extern struct wait_queue_head console_queue;
 struct consfront_dev *xencons_ring_init(void);
+void xencons_ring_fini(struct consfront_dev* dev);
+void xencons_ring_resume(struct consfront_dev* dev);
 struct consfront_dev *init_consfront(char *_nodename);
 int xencons_ring_send(struct consfront_dev *dev, const char *data, unsigned len);
 int xencons_ring_send_no_notify(struct consfront_dev *dev, const char *data, unsigned len);
diff --git a/kernel.c b/kernel.c
index 3564af3..2fb69bf 100644
--- a/kernel.c
+++ b/kernel.c
@@ -122,10 +122,14 @@ void pre_suspend(void)
     local_irq_disable();
 
     fini_time();
+
+    suspend_console();
 }
 
 void post_suspend(int canceled)
 {
+    resume_console();
+
     init_time();
 
     local_irq_enable();
diff --git a/lib/sys.c b/lib/sys.c
index 23dc2a5..da434fc 100644
--- a/lib/sys.c
+++ b/lib/sys.c
@@ -487,7 +487,7 @@ int close(int fd)
 #ifdef CONFIG_CONSFRONT
         case FTYPE_SAVEFILE:
         case FTYPE_CONSOLE:
-            fini_console(files[fd].cons.dev);
+            fini_consfront(files[fd].cons.dev);
             files[fd].type = FTYPE_NONE;
             return 0;
 #endif
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 12/16] Save/Restore Support: Add support for suspend/restore events.

[Bruno Alvisio]

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 events.c         | 5 +++++
 include/events.h | 1 +
 kernel.c         | 2 ++
 3 files changed, 8 insertions(+)

diff --git a/events.c b/events.c
index e8ef8aa..342aead 100644
--- a/events.c
+++ b/events.c
@@ -183,6 +183,11 @@ void fini_events(void)
     arch_fini_events();
 }
 
+void suspend_events(void)
+{
+    unbind_all_ports();
+}
+
 void default_handler(evtchn_port_t port, struct pt_regs *regs, void *ignore)
 {
     printk("[Port %d] - event received\n", port);
diff --git a/include/events.h b/include/events.h
index 89b5997..705ad93 100644
--- a/include/events.h
+++ b/include/events.h
@@ -55,5 +55,6 @@ static inline int notify_remote_via_evtchn(evtchn_port_t port)
 }
 
 void fini_events(void);
+void suspend_events(void);
 
 #endif /* _EVENTS_H_ */
diff --git a/kernel.c b/kernel.c
index 2fb69bf..d078e0a 100644
--- a/kernel.c
+++ b/kernel.c
@@ -124,6 +124,8 @@ void pre_suspend(void)
     fini_time();
 
     suspend_console();
+
+    suspend_events();
 }
 
 void post_suspend(int canceled)
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 13/16] Save/Restore Support: Add suspend/restore support for Grant Tables.

[Bruno Alvisio]

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
---
Changed since v1:
    - Moved suspend/resume _gnttab to arch specific files
---
 arch/x86/mm.c    | 34 ++++++++++++++++++++++++++++++++++
 gnttab.c         | 10 ++++++++++
 include/gnttab.h |  4 ++++
 kernel.c         |  4 ++++
 4 files changed, 52 insertions(+)

diff --git a/arch/x86/mm.c b/arch/x86/mm.c
index 1b163ac..2597c5b 100644
--- a/arch/x86/mm.c
+++ b/arch/x86/mm.c
@@ -917,6 +917,40 @@ grant_entry_v1_t *arch_init_gnttab(int nr_grant_frames)
     return map_frames(frames, nr_grant_frames);
 }
 
+void arch_suspend_gnttab(grant_entry_v1_t *gnttab_table, int nr_grant_frames)
+{
+#ifdef CONFIG_PARAVIRT
+    int i;
+
+    for (i = 0; i < nr_grant_frames; i++) {
+        HYPERVISOR_update_va_mapping((unsigned long)(((char *)gnttab_table) + PAGE_SIZE*i),
+                (pte_t){0x0<<PAGE_SHIFT}, UVMF_INVLPG);
+    }
+#endif
+    return;
+}
+
+void arch_resume_gnttab(grant_entry_v1_t *gnttab_table, int nr_grant_frames)
+{
+    struct gnttab_setup_table setup;
+    unsigned long frames[nr_grant_frames];
+#ifdef CONFIG_PARAVIRT
+    int i;
+#endif
+    setup.dom = DOMID_SELF;
+    setup.nr_frames = nr_grant_frames;
+    set_xen_guest_handle(setup.frame_list, frames);
+
+    HYPERVISOR_grant_table_op(GNTTABOP_setup_table, &setup, 1);
+
+#ifdef CONFIG_PARAVIRT
+    for (i = 0; i < nr_grant_frames; i++) {
+        HYPERVISOR_update_va_mapping((unsigned long)(((char *)gnttab_table) + PAGE_SIZE*i),
+                (pte_t){(frames[i] << PAGE_SHIFT) | L1_PROT}, UVMF_INVLPG);
+    }
+#endif
+}
+
 unsigned long alloc_virt_kernel(unsigned n_pages)
 {
     unsigned long addr;
diff --git a/gnttab.c b/gnttab.c
index 3f0e35f..6978a9b 100644
--- a/gnttab.c
+++ b/gnttab.c
@@ -194,3 +194,13 @@ fini_gnttab(void)
 
     HYPERVISOR_grant_table_op(GNTTABOP_setup_table, &setup, 1);
 }
+
+void suspend_gnttab(void)
+{
+    arch_suspend_gnttab(gnttab_table, NR_GRANT_FRAMES);
+}
+
+void resume_gnttab(void)
+{
+    arch_resume_gnttab(gnttab_table, NR_GRANT_FRAMES);
+}
diff --git a/include/gnttab.h b/include/gnttab.h
index a9d8e09..974cb89 100644
--- a/include/gnttab.h
+++ b/include/gnttab.h
@@ -12,6 +12,10 @@ unsigned long gnttab_end_transfer(grant_ref_t gref);
 int gnttab_end_access(grant_ref_t ref);
 const char *gnttabop_error(int16_t status);
 void fini_gnttab(void);
+void suspend_gnttab(void);
+void resume_gnttab(void);
 grant_entry_v1_t *arch_init_gnttab(int nr_grant_frames);
+void arch_suspend_gnttab(grant_entry_v1_t *gnttab_table, int nr_grant_frames);
+void arch_resume_gnttab(grant_entry_v1_t *gnttab_table, int nr_grant_frames);
 
 #endif /* !__GNTTAB_H__ */
diff --git a/kernel.c b/kernel.c
index d078e0a..933cbcd 100644
--- a/kernel.c
+++ b/kernel.c
@@ -121,6 +121,8 @@ void pre_suspend(void)
 {
     local_irq_disable();
 
+    suspend_gnttab();
+
     fini_time();
 
     suspend_console();
@@ -134,6 +136,8 @@ void post_suspend(int canceled)
 
     init_time();
 
+    resume_gnttab();
+
     local_irq_enable();
 }
 
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 14/16] Save/Restore Support: Add suspend/restore support for xenbus

[Bruno Alvisio]

Currently the watch path is not saved in the watch struct when it is registered.
During xenbus resume the path is needed so that the watches can be registered again.
Thus, 'path' field is added to struct watch so that watches can be re-registered
during xenbus resume.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 include/xenbus.h |   2 ++
 kernel.c         |   8 +++++
 xenbus/xenbus.c  | 106 +++++++++++++++++++++++++++++++++++++++----------------
 3 files changed, 85 insertions(+), 31 deletions(-)

diff --git a/include/xenbus.h b/include/xenbus.h
index b2d5072..3871f35 100644
--- a/include/xenbus.h
+++ b/include/xenbus.h
@@ -120,6 +120,8 @@ domid_t xenbus_get_self_id(void);
 #ifdef CONFIG_XENBUS
 /* Reset the XenBus system. */
 void fini_xenbus(void);
+void suspend_xenbus(void);
+void resume_xenbus(int canceled);
 #else
 static inline void fini_xenbus(void)
 {
diff --git a/kernel.c b/kernel.c
index 933cbcd..1393d15 100644
--- a/kernel.c
+++ b/kernel.c
@@ -119,6 +119,10 @@ void start_kernel(void* par)
 
 void pre_suspend(void)
 {
+#ifdef CONFIG_XENBUS
+    suspend_xenbus();
+#endif
+
     local_irq_disable();
 
     suspend_gnttab();
@@ -139,6 +143,10 @@ void post_suspend(int canceled)
     resume_gnttab();
 
     local_irq_enable();
+
+#ifdef CONFIG_XENBUS
+    resume_xenbus(canceled);
+#endif
 }
 
 void stop_kernel(void)
diff --git a/xenbus/xenbus.c b/xenbus/xenbus.c
index c2d2bd1..d72dc3a 100644
--- a/xenbus/xenbus.c
+++ b/xenbus/xenbus.c
@@ -50,6 +50,7 @@ DECLARE_WAIT_QUEUE_HEAD(xenbus_watch_queue);
 xenbus_event_queue xenbus_events;
 static struct watch {
     char *token;
+    char *path;
     xenbus_event_queue *events;
     struct watch *next;
 } *watches;
@@ -63,6 +64,8 @@ struct xenbus_req_info
 #define NR_REQS 32
 static struct xenbus_req_info req_info[NR_REQS];
 
+static char *errmsg(struct xsd_sockmsg *rep);
+
 uint32_t xenbus_evtchn;
 
 #ifdef CONFIG_PARAVIRT
@@ -231,45 +234,39 @@ static void xenbus_thread_func(void *ign)
     struct xsd_sockmsg msg;
     unsigned prod = xenstore_buf->rsp_prod;
 
-    for (;;) 
-    {
+    for (;;) {
         wait_event(xb_waitq, prod != xenstore_buf->rsp_prod);
-        while (1) 
-        {
+        while (1) {
             prod = xenstore_buf->rsp_prod;
             DEBUG("Rsp_cons %d, rsp_prod %d.\n", xenstore_buf->rsp_cons,
-                    xenstore_buf->rsp_prod);
+                  xenstore_buf->rsp_prod);
             if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons < sizeof(msg))
                 break;
             rmb();
-            memcpy_from_ring(xenstore_buf->rsp,
-                    &msg,
-                    MASK_XENSTORE_IDX(xenstore_buf->rsp_cons),
-                    sizeof(msg));
-            DEBUG("Msg len %d, %d avail, id %d.\n",
-                    msg.len + sizeof(msg),
-                    xenstore_buf->rsp_prod - xenstore_buf->rsp_cons,
-                    msg.req_id);
+            memcpy_from_ring(xenstore_buf->rsp, &msg,
+                             MASK_XENSTORE_IDX(xenstore_buf->rsp_cons),
+                             sizeof(msg));
+            DEBUG("Msg len %d, %d avail, id %d.\n", msg.len + sizeof(msg),
+                  xenstore_buf->rsp_prod - xenstore_buf->rsp_cons, msg.req_id);
+
             if (xenstore_buf->rsp_prod - xenstore_buf->rsp_cons <
-                    sizeof(msg) + msg.len)
+                sizeof(msg) + msg.len)
                 break;
 
             DEBUG("Message is good.\n");
 
-            if(msg.type == XS_WATCH_EVENT)
-            {
-		struct xenbus_event *event = malloc(sizeof(*event) + msg.len);
+            if (msg.type == XS_WATCH_EVENT) {
+                struct xenbus_event *event = malloc(sizeof(*event) + msg.len);
                 xenbus_event_queue *events = NULL;
-		char *data = (char*)event + sizeof(*event);
+                char *data = (char*)event + sizeof(*event);
                 struct watch *watch;
 
-                memcpy_from_ring(xenstore_buf->rsp,
-		    data,
+                memcpy_from_ring(xenstore_buf->rsp, data,
                     MASK_XENSTORE_IDX(xenstore_buf->rsp_cons + sizeof(msg)),
                     msg.len);
 
-		event->path = data;
-		event->token = event->path + strlen(event->path) + 1;
+                event->path = data;
+                event->token = event->path + strlen(event->path) + 1;
 
                 mb();
                 xenstore_buf->rsp_cons += msg.len + sizeof(msg);
@@ -288,15 +285,11 @@ static void xenbus_thread_func(void *ign)
                     printk("unexpected watch token %s\n", event->token);
                     free(event);
                 }
-            }
-
-            else
-            {
+            } else {
                 req_info[msg.req_id].reply = malloc(sizeof(msg) + msg.len);
-                memcpy_from_ring(xenstore_buf->rsp,
-                    req_info[msg.req_id].reply,
-                    MASK_XENSTORE_IDX(xenstore_buf->rsp_cons),
-                    msg.len + sizeof(msg));
+                memcpy_from_ring(xenstore_buf->rsp, req_info[msg.req_id].reply,
+                                 MASK_XENSTORE_IDX(xenstore_buf->rsp_cons),
+                                 msg.len + sizeof(msg));
                 mb();
                 xenstore_buf->rsp_cons += msg.len + sizeof(msg);
                 wake_up(&req_info[msg.req_id].waitq);
@@ -380,6 +373,55 @@ void fini_xenbus(void)
 {
 }
 
+void suspend_xenbus(void)
+{
+    /* Check for live requests and wait until they finish */
+    while (1)
+    {
+        spin_lock(&req_lock);
+        if (nr_live_reqs == 0)
+            break;
+        spin_unlock(&req_lock);
+        wait_event(req_wq, (nr_live_reqs == 0));
+    }
+
+    mask_evtchn(xenbus_evtchn);
+    xenstore_buf = NULL;
+    spin_unlock(&req_lock);
+}
+
+void resume_xenbus(int canceled)
+{
+    char *msg;
+    struct watch *watch;
+    struct write_req req[2];
+    struct xsd_sockmsg *rep;
+
+#ifdef CONFIG_PARAVIRT
+    get_xenbus(&start_info);
+#else
+    get_xenbus(0);
+#endif
+    unmask_evtchn(xenbus_evtchn);
+
+    if (!canceled) {
+        for (watch = watches; watch; watch = watch->next) {
+            req[0].data = watch->path;
+            req[0].len = strlen(watch->path) + 1;
+            req[1].data = watch->token;
+            req[1].len = strlen(watch->token) + 1;
+
+            rep = xenbus_msg_reply(XS_WATCH, XBT_NIL, req, ARRAY_SIZE(req));
+            msg = errmsg(rep);
+            if (msg)
+                xprintk("error on XS_WATCH: %s\n", msg);
+            free(rep);
+        }
+    }
+
+    notify_remote_via_evtchn(xenbus_evtchn);
+}
+
 /* Send data to xenbus.  This can block.  All of the requests are seen
    by xenbus as if sent atomically.  The header is added
    automatically, using type %type, req_id %req_id, and trans_id
@@ -501,7 +543,7 @@ static char *errmsg(struct xsd_sockmsg *rep)
     res[rep->len] = 0;
     free(rep);
     return res;
-}	
+}
 
 /* Send a debug message to xenbus.  Can block. */
 static void xenbus_debug_msg(const char *msg)
@@ -601,6 +643,7 @@ char* xenbus_watch_path_token( xenbus_transaction_t xbt, const char *path, const
         events = &xenbus_events;
 
     watch->token = strdup(token);
+    watch->path = strdup(path);
     watch->events = events;
     watch->next = watches;
     watches = watch;
@@ -636,6 +679,7 @@ char* xenbus_unwatch_path_token( xenbus_transaction_t xbt, const char *path, con
     for (prev = &watches, watch = *prev; watch; prev = &watch->next, watch = *prev)
         if (!strcmp(watch->token, token)) {
             free(watch->token);
+            free(watch->path);
             *prev = watch->next;
             free(watch);
             break;
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 15/16] Save/Restore Support: Add suspend/restore support for netfront

[Bruno Alvisio]

Performed an additional cleanup to make the file more syntactically consistent.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 include/netfront.h |   8 +-
 kernel.c           |   8 ++
 netfront.c         | 309 ++++++++++++++++++++++++++++++++++++++---------------
 3 files changed, 236 insertions(+), 89 deletions(-)

diff --git a/include/netfront.h b/include/netfront.h
index 2b95da9..1164d50 100644
--- a/include/netfront.h
+++ b/include/netfront.h
@@ -3,9 +3,15 @@
 #include <lwip/netif.h>
 #endif
 struct netfront_dev;
-struct netfront_dev *init_netfront(char *nodename, void (*netif_rx)(unsigned char *data, int len), unsigned char rawmac[6], char **ip);
+struct netfront_dev *init_netfront(char *nodename,
+                                   void (*netif_rx)(unsigned char *data,
+                                                    int len, void* arg),
+                                   unsigned char rawmac[6],
+                                   char **ip);
 void netfront_xmit(struct netfront_dev *dev, unsigned char* data,int len);
 void shutdown_netfront(struct netfront_dev *dev);
+void suspend_netfront(void);
+void resume_netfront(void);
 #ifdef HAVE_LIBC
 int netfront_tap_open(char *nodename);
 ssize_t netfront_receive(struct netfront_dev *dev, unsigned char *data, size_t len);
diff --git a/kernel.c b/kernel.c
index 1393d15..301273d 100644
--- a/kernel.c
+++ b/kernel.c
@@ -119,6 +119,10 @@ void start_kernel(void* par)
 
 void pre_suspend(void)
 {
+#ifdef CONFIG_NETFRONT
+    suspend_netfront();
+#endif
+
 #ifdef CONFIG_XENBUS
     suspend_xenbus();
 #endif
@@ -147,6 +151,10 @@ void post_suspend(int canceled)
 #ifdef CONFIG_XENBUS
     resume_xenbus(canceled);
 #endif
+
+#ifdef CONFIG_NETFRONT
+    resume_netfront();
+#endif
 }
 
 void stop_kernel(void)
diff --git a/netfront.c b/netfront.c
index b8fac62..50b3a57 100644
--- a/netfront.c
+++ b/netfront.c
@@ -63,10 +63,30 @@ struct netfront_dev {
     size_t rlen;
 #endif
 
-    void (*netif_rx)(unsigned char* data, int len);
+    void (*netif_rx)(unsigned char* data, int len, void* arg);
+    void *netif_rx_arg;
 };
 
+struct netfront_dev_list {
+    struct netfront_dev *dev;
+    unsigned char rawmac[6];
+    char *ip;
+
+    int refcount;
+
+    struct netfront_dev_list *next;
+};
+
+static struct netfront_dev_list *dev_list = NULL;
+
 void init_rx_buffers(struct netfront_dev *dev);
+static struct netfront_dev *_init_netfront(struct netfront_dev *dev,
+                                           unsigned char rawmac[6], char **ip);
+static void _shutdown_netfront(struct netfront_dev *dev);
+void netfront_set_rx_handler(struct netfront_dev *dev,
+                             void (*thenetif_rx)(unsigned char *data, int len,
+                                                 void *arg),
+                             void *arg);
 
 static inline void add_id_to_freelist(unsigned int id,unsigned short* freelist)
 {
@@ -81,7 +101,7 @@ static inline unsigned short get_id_from_freelist(unsigned short* freelist)
     return id;
 }
 
-__attribute__((weak)) void netif_rx(unsigned char* data,int len)
+__attribute__((weak)) void netif_rx(unsigned char* data, int len, void *arg)
 {
     printk("%d bytes incoming at %p\n",len,data);
 }
@@ -120,21 +140,20 @@ moretodo:
         page = (unsigned char*)buf->page;
         gnttab_end_access(buf->gref);
 
-        if (rx->status > NETIF_RSP_NULL)
-        {
+        if (rx->status > NETIF_RSP_NULL) {
 #ifdef HAVE_LIBC
-	    if (dev->netif_rx == NETIF_SELECT_RX) {
-		int len = rx->status;
-		ASSERT(current == main_thread);
-		if (len > dev->len)
-		    len = dev->len;
-		memcpy(dev->data, page+rx->offset, len);
-		dev->rlen = len;
-		/* No need to receive the rest for now */
-		dobreak = 1;
-	    } else
+            if (dev->netif_rx == NETIF_SELECT_RX) {
+                int len = rx->status;
+                ASSERT(current == main_thread);
+                if (len > dev->len)
+                    len = dev->len;
+                memcpy(dev->data, page+rx->offset, len);
+                dev->rlen = len;
+                /* No need to receive the rest for now */
+                dobreak = 1;
+            } else
 #endif
-		dev->netif_rx(page+rx->offset,rx->status);
+		        dev->netif_rx(page+rx->offset, rx->status, dev->netif_rx_arg);
         }
     }
     dev->rx.rsp_cons=cons;
@@ -144,17 +163,16 @@ moretodo:
 
     req_prod = dev->rx.req_prod_pvt;
 
-    for(i=0; i<nr_consumed; i++)
-    {
+    for (i = 0; i < nr_consumed; i++) {
         int id = xennet_rxidx(req_prod + i);
         netif_rx_request_t *req = RING_GET_REQUEST(&dev->rx, req_prod + i);
         struct net_buffer* buf = &dev->rx_buffers[id];
         void* page = buf->page;
 
         /* We are sure to have free gnttab entries since they got released above */
-        buf->gref = req->gref = 
-            gnttab_grant_access(dev->dom,virt_to_mfn(page),0);
-
+        buf->gref = req->gref = gnttab_grant_access(dev->dom,
+                                                    virt_to_mfn(page),
+                                                    0);
         req->id = id;
     }
 
@@ -165,13 +183,10 @@ moretodo:
     RING_PUSH_REQUESTS_AND_CHECK_NOTIFY(&dev->rx, notify);
     if (notify)
         notify_remote_via_evtchn(dev->evtchn);
-
 }
 
 void network_tx_buf_gc(struct netfront_dev *dev)
 {
-
-
     RING_IDX cons, prod;
     unsigned short id;
 
@@ -197,8 +212,8 @@ void network_tx_buf_gc(struct netfront_dev *dev)
             gnttab_end_access(buf->gref);
             buf->gref=GRANT_INVALID_REF;
 
-	    add_id_to_freelist(id,dev->tx_freelist);
-	    up(&dev->tx_sem);
+            add_id_to_freelist(id,dev->tx_freelist);
+            up(&dev->tx_sem);
         }
 
         dev->tx.rsp_cons = prod;
@@ -215,8 +230,6 @@ void network_tx_buf_gc(struct netfront_dev *dev)
             prod + ((dev->tx.sring->req_prod - prod) >> 1) + 1;
         mb();
     } while ((cons == prod) && (prod != dev->tx.sring->rsp_prod));
-
-
 }
 
 void netfront_handler(evtchn_port_t port, struct pt_regs *regs, void *data)
@@ -253,8 +266,8 @@ static void free_netfront(struct netfront_dev *dev)
 {
     int i;
 
-    for(i=0;i<NET_TX_RING_SIZE;i++)
-	down(&dev->tx_sem);
+    for(i = 0; i < NET_TX_RING_SIZE; i++)
+        down(&dev->tx_sem);
 
     mask_evtchn(dev->evtchn);
 
@@ -269,32 +282,31 @@ static void free_netfront(struct netfront_dev *dev)
 
     unbind_evtchn(dev->evtchn);
 
-    for(i=0;i<NET_RX_RING_SIZE;i++) {
-	gnttab_end_access(dev->rx_buffers[i].gref);
-	free_page(dev->rx_buffers[i].page);
+    for (i = 0; i < NET_RX_RING_SIZE; i++) {
+        if (dev->rx_buffers[i].page) {
+            gnttab_end_access(dev->rx_buffers[i].gref);
+            free_page(dev->rx_buffers[i].page);
+        }
     }
 
-    for(i=0;i<NET_TX_RING_SIZE;i++)
-	if (dev->tx_buffers[i].page)
-	    free_page(dev->tx_buffers[i].page);
+    for (i = 0; i < NET_TX_RING_SIZE; i++)
+        if (dev->tx_buffers[i].page)
+            free_page(dev->tx_buffers[i].page);
 
     free(dev->nodename);
     free(dev);
 }
 
-struct netfront_dev *init_netfront(char *_nodename, void (*thenetif_rx)(unsigned char* data, int len), unsigned char rawmac[6], char **ip)
+struct netfront_dev *init_netfront(char *_nodename,
+                                   void (*thenetif_rx)(unsigned char* data,
+                                                       int len, void* arg),
+                                   unsigned char rawmac[6],
+                                   char **ip)
 {
-    xenbus_transaction_t xbt;
-    char* err;
-    char* message=NULL;
-    struct netif_tx_sring *txs;
-    struct netif_rx_sring *rxs;
-    int retry=0;
-    int i;
-    char* msg = NULL;
     char nodename[256];
-    char path[256];
     struct netfront_dev *dev;
+    struct netfront_dev_list *ldev = NULL;
+    struct netfront_dev_list *list = NULL;
     static int netfrontends = 0;
 
     if (!_nodename)
@@ -303,10 +315,20 @@ struct netfront_dev *init_netfront(char *_nodename, void (*thenetif_rx)(unsigned
         strncpy(nodename, _nodename, sizeof(nodename) - 1);
         nodename[sizeof(nodename) - 1] = 0;
     }
-    netfrontends++;
+
+    /* Check if the device is already initialized */
+    for (list = dev_list; list != NULL; list = list->next) {
+        if (strcmp(nodename, list->dev->nodename) == 0) {
+            list->refcount++;
+            dev = list->dev;
+            if (thenetif_rx)
+                netfront_set_rx_handler(dev, thenetif_rx, NULL);
+            goto out;
+        }
+    }
 
     if (!thenetif_rx)
-	thenetif_rx = netif_rx;
+        thenetif_rx = netif_rx;
 
     printk("************************ NETFRONT for %s **********\n\n\n", nodename);
 
@@ -316,26 +338,79 @@ struct netfront_dev *init_netfront(char *_nodename, void (*thenetif_rx)(unsigned
 #ifdef HAVE_LIBC
     dev->fd = -1;
 #endif
+    dev->netif_rx = thenetif_rx;
+    dev->netif_rx_arg = NULL;
+
+    ldev = malloc(sizeof(struct netfront_dev_list));
+    memset(ldev, 0, sizeof(struct netfront_dev_list));
+
+    if (_init_netfront(dev, ldev->rawmac, &(ldev->ip))) {
+        ldev->dev = dev;
+        ldev->refcount = 1;
+        ldev->next = NULL;
+
+        if (!dev_list) {
+            dev_list = ldev;
+        } else {
+            for (list = dev_list; list->next != NULL; list = list->next)
+                ;
+            list->next = ldev;
+		}
+        netfrontends++;
+    } else {
+        free(ldev);
+        dev = NULL;
+        goto err;
+    }
+
+out:
+    if (rawmac) {
+        rawmac[0] = ldev->rawmac[0];
+        rawmac[1] = ldev->rawmac[1];
+        rawmac[2] = ldev->rawmac[2];
+        rawmac[3] = ldev->rawmac[3];
+        rawmac[4] = ldev->rawmac[4];
+        rawmac[5] = ldev->rawmac[5];
+	}
+    if (ip)
+        *ip = strdup(ldev->ip);
+
+err:
+    return dev;
+}
+
+static struct netfront_dev *_init_netfront(struct netfront_dev *dev,
+					   unsigned char rawmac[6],
+					   char **ip)
+{
+    xenbus_transaction_t xbt;
+    char* err = NULL;
+    char* message=NULL;
+    struct netif_tx_sring *txs;
+    struct netif_rx_sring *rxs;
+    char* msg = NULL;
+    int retry=0;
+    int i;
+    char path[256];
 
     printk("net TX ring size %lu\n", (unsigned long) NET_TX_RING_SIZE);
     printk("net RX ring size %lu\n", (unsigned long) NET_RX_RING_SIZE);
     init_SEMAPHORE(&dev->tx_sem, NET_TX_RING_SIZE);
-    for(i=0;i<NET_TX_RING_SIZE;i++)
-    {
-	add_id_to_freelist(i,dev->tx_freelist);
+    for (i = 0; i < NET_TX_RING_SIZE; i++) {
+        add_id_to_freelist(i, dev->tx_freelist);
         dev->tx_buffers[i].page = NULL;
     }
 
-    for(i=0;i<NET_RX_RING_SIZE;i++)
-    {
-	/* TODO: that's a lot of memory */
+    for (i = 0; i < NET_RX_RING_SIZE; i++) {
+        /* TODO: that's a lot of memory */
         dev->rx_buffers[i].page = (char*)alloc_page();
+        BUG_ON(dev->rx_buffers[i].page == NULL);
     }
 
-    snprintf(path, sizeof(path), "%s/backend-id", nodename);
+    snprintf(path, sizeof(path), "%s/backend-id", dev->nodename);
     dev->dom = xenbus_read_integer(path);
 #ifdef HAVE_LIBC
-    if (thenetif_rx == NETIF_SELECT_RX)
+    if (dev->netif_rx == NETIF_SELECT_RX)
         evtchn_alloc_unbound(dev->dom, netfront_select_handler, dev, &dev->evtchn);
     else
 #endif
@@ -343,22 +418,19 @@ struct netfront_dev *init_netfront(char *_nodename, void (*thenetif_rx)(unsigned
 
     txs = (struct netif_tx_sring *) alloc_page();
     rxs = (struct netif_rx_sring *) alloc_page();
-    memset(txs,0,PAGE_SIZE);
-    memset(rxs,0,PAGE_SIZE);
-
+    memset(txs, 0, PAGE_SIZE);
+    memset(rxs, 0, PAGE_SIZE);
 
     SHARED_RING_INIT(txs);
     SHARED_RING_INIT(rxs);
     FRONT_RING_INIT(&dev->tx, txs, PAGE_SIZE);
     FRONT_RING_INIT(&dev->rx, rxs, PAGE_SIZE);
 
-    dev->tx_ring_ref = gnttab_grant_access(dev->dom,virt_to_mfn(txs),0);
-    dev->rx_ring_ref = gnttab_grant_access(dev->dom,virt_to_mfn(rxs),0);
+    dev->tx_ring_ref = gnttab_grant_access(dev->dom, virt_to_mfn(txs), 0);
+    dev->rx_ring_ref = gnttab_grant_access(dev->dom, virt_to_mfn(rxs), 0);
 
     init_rx_buffers(dev);
 
-    dev->netif_rx = thenetif_rx;
-
     dev->events = NULL;
 
 again:
@@ -368,33 +440,31 @@ again:
         free(err);
     }
 
-    err = xenbus_printf(xbt, nodename, "tx-ring-ref","%u",
-                dev->tx_ring_ref);
+    err = xenbus_printf(xbt, dev->nodename, "tx-ring-ref","%u",
+                        dev->tx_ring_ref);
     if (err) {
         message = "writing tx ring-ref";
         goto abort_transaction;
     }
-    err = xenbus_printf(xbt, nodename, "rx-ring-ref","%u",
-                dev->rx_ring_ref);
+    err = xenbus_printf(xbt, dev->nodename, "rx-ring-ref","%u",
+                        dev->rx_ring_ref);
     if (err) {
         message = "writing rx ring-ref";
         goto abort_transaction;
     }
-    err = xenbus_printf(xbt, nodename,
-                "event-channel", "%u", dev->evtchn);
+    err = xenbus_printf(xbt, dev->nodename, "event-channel", "%u", dev->evtchn);
     if (err) {
         message = "writing event-channel";
         goto abort_transaction;
     }
 
-    err = xenbus_printf(xbt, nodename, "request-rx-copy", "%u", 1);
-
+    err = xenbus_printf(xbt, dev->nodename, "request-rx-copy", "%u", 1);
     if (err) {
         message = "writing request-rx-copy";
         goto abort_transaction;
     }
 
-    snprintf(path, sizeof(path), "%s/state", nodename);
+    snprintf(path, sizeof(path), "%s/state", dev->nodename);
     err = xenbus_switch_state(xbt, path, XenbusStateConnected);
     if (err) {
         message = "switching state";
@@ -404,7 +474,7 @@ again:
     err = xenbus_transaction_end(xbt, 0, &retry);
     free(err);
     if (retry) {
-            goto again;
+        goto again;
         printk("completing transaction\n");
     }
 
@@ -417,10 +487,9 @@ abort_transaction:
     goto error;
 
 done:
-
-    snprintf(path, sizeof(path), "%s/backend", nodename);
+    snprintf(path, sizeof(path), "%s/backend", dev->nodename);
     msg = xenbus_read(XBT_NIL, path, &dev->backend);
-    snprintf(path, sizeof(path), "%s/mac", nodename);
+    snprintf(path, sizeof(path), "%s/mac", dev->nodename);
     msg = xenbus_read(XBT_NIL, path, &dev->mac);
 
     if ((dev->backend == NULL) || (dev->mac == NULL)) {
@@ -428,8 +497,8 @@ done:
         goto error;
     }
 
-    printk("backend at %s\n",dev->backend);
-    printk("mac is %s\n",dev->mac);
+    printk("backend at %s\n", dev->backend);
+    printk("mac is %s\n", dev->mac);
 
     {
         XenbusState state;
@@ -458,18 +527,20 @@ done:
 
     unmask_evtchn(dev->evtchn);
 
-        /* Special conversion specifier 'hh' needed for __ia64__. Without
-           this mini-os panics with 'Unaligned reference'. */
+    /* Special conversion specifier 'hh' needed for __ia64__. Without
+     * this mini-os panics with 'Unaligned reference'.
+     */
     if (rawmac)
-	sscanf(dev->mac,"%hhx:%hhx:%hhx:%hhx:%hhx:%hhx",
-            &rawmac[0],
-            &rawmac[1],
-            &rawmac[2],
-            &rawmac[3],
-            &rawmac[4],
-            &rawmac[5]);
+        sscanf(dev->mac,"%hhx:%hhx:%hhx:%hhx:%hhx:%hhx",
+               &rawmac[0],
+               &rawmac[1],
+               &rawmac[2],
+               &rawmac[3],
+               &rawmac[4],
+               &rawmac[5]);
 
     return dev;
+
 error:
     free(msg);
     free(err);
@@ -496,6 +567,41 @@ int netfront_tap_open(char *nodename) {
 
 void shutdown_netfront(struct netfront_dev *dev)
 {
+    struct netfront_dev_list *list = NULL;
+    struct netfront_dev_list *to_del = NULL;
+
+    /* Check this is a valid device */
+    for (list = dev_list; list != NULL; list = list->next) {
+        if (list->dev == dev)
+            break;
+    }
+
+    if (!list) {
+        printk("Trying to shutdown an invalid netfront device (%p)\n", dev);
+        return;
+    }
+
+    list->refcount--;
+    if (list->refcount == 0) {
+        _shutdown_netfront(dev);
+        free(dev->nodename);
+        free(dev);
+
+        to_del = list;
+        if (to_del == dev_list) {
+            free(to_del);
+			dev_list = NULL;
+        } else {
+            for (list = dev_list; list->next != to_del; list = list->next)
+                ;
+            list->next = to_del->next;
+            free(to_del);
+        }
+    }
+}
+
+static void _shutdown_netfront(struct netfront_dev *dev)
+{
     char* err = NULL, *err2;
     XenbusState state;
 
@@ -559,6 +665,21 @@ close:
         free_netfront(dev);
 }
 
+void suspend_netfront(void)
+{
+    struct netfront_dev_list *list;
+
+    for (list = dev_list; list != NULL; list = list->next)
+        _shutdown_netfront(list->dev);
+}
+
+void resume_netfront(void)
+{
+    struct netfront_dev_list *list;
+
+    for (list = dev_list; list != NULL; list = list->next)
+        _init_netfront(list->dev, NULL, NULL);
+}
 
 void init_rx_buffers(struct netfront_dev *dev)
 {
@@ -653,8 +774,8 @@ ssize_t netfront_receive(struct netfront_dev *dev, unsigned char *data, size_t l
     local_irq_save(flags);
     network_rx(dev);
     if (!dev->rlen && fd != -1)
-	/* No data for us, make select stop returning */
-	files[fd].read = 0;
+        /* No data for us, make select stop returning */
+        files[fd].read = 0;
     /* Before re-enabling the interrupts, in case a packet just arrived in the
      * meanwhile. */
     local_irq_restore(flags);
@@ -665,3 +786,15 @@ ssize_t netfront_receive(struct netfront_dev *dev, unsigned char *data, size_t l
     return dev->rlen;
 }
 #endif
+
+void netfront_set_rx_handler(struct netfront_dev *dev,
+                             void (*thenetif_rx)(unsigned char *data, int len,
+                                                 void *arg),
+                             void *arg)
+{
+    if (dev->netif_rx && dev->netif_rx != netif_rx)
+        printk("Replacing netif_rx handler for dev %s\n", dev->nodename);
+
+    dev->netif_rx = thenetif_rx;
+    dev->netif_rx_arg = arg;
+}
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 16/16] Save/Restore Support: Implement code for arch suspend/resume

[Bruno Alvisio]

Before suspending the domain the shared_info_page is unmapped and for PVs the
pagetables should be canonicalized. After resume the shared_info_page should be
mapped again.

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
Reviewed-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
Changed since v1:
  * Fixed comment
---
 arch/x86/setup.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 51 insertions(+)

diff --git a/arch/x86/setup.c b/arch/x86/setup.c
index b6e0541..b5ed1c8 100644
--- a/arch/x86/setup.c
+++ b/arch/x86/setup.c
@@ -32,6 +32,7 @@
 #include <xen/xen.h>
 #include <xen/arch-x86/cpuid.h>
 #include <xen/arch-x86/hvm/start_info.h>
+#include <xen/hvm/params.h>
 
 #ifdef CONFIG_PARAVIRT
 /*
@@ -42,6 +43,11 @@ union start_info_union start_info_union;
 #endif
 
 /*
+ * This pointer holds a reference to the copy of the start_info struct.
+ */
+static start_info_t *start_info_ptr;
+
+/*
  * Shared page for communicating with the hypervisor.
  * Events flags go here, for example.
  */
@@ -212,18 +218,63 @@ arch_init(void *par)
 #ifdef CONFIG_PARAVIRT
 	memcpy(&start_info, par, sizeof(start_info));
 #endif
+	start_info_ptr = (start_info_t *)par;
 
 	start_kernel((start_info_t *)par);
 }
 
 void arch_pre_suspend(void)
 {
+#ifdef CONFIG_PARAVIRT
+   /* Replace xenstore and console mfns with the correspondent pfns */
+    start_info_ptr->store_mfn =
+        virt_to_pfn(mfn_to_virt(start_info_ptr->store_mfn));
+    start_info_ptr->console.domU.mfn =
+        virt_to_pfn(mfn_to_virt(start_info_ptr->console.domU.mfn));
+#else
+    uint64_t store_v;
+    uint64_t console_v;
+
+    if( hvm_get_parameter(HVM_PARAM_STORE_PFN, &store_v) )
+        BUG();
+    start_info_ptr->store_mfn = store_v;
+
+    if( hvm_get_parameter(HVM_PARAM_CONSOLE_PFN, &console_v) )
+        BUG();
+    start_info_ptr->console.domU.mfn = console_v;
+#endif
+    unmap_shared_info();
 
+    arch_mm_pre_suspend();
 }
 
 void arch_post_suspend(int canceled)
 {
+#if CONFIG_PARAVIRT
+    if (canceled) {
+        start_info_ptr->store_mfn = pfn_to_mfn(start_info_ptr->store_mfn);
+        start_info_ptr->console.domU.mfn = pfn_to_mfn(start_info_ptr->console.domU.mfn);
+    } else {
+        memcpy(&start_info, start_info_ptr, sizeof(start_info_t));
+    }
+#else
+    uint64_t store_v;
+    uint64_t console_v;
+
+    if (hvm_get_parameter(HVM_PARAM_STORE_PFN, &store_v))
+        BUG();
+    start_info_ptr->store_mfn = pfn_to_mfn(store_v);
 
+    if (hvm_get_parameter(HVM_PARAM_CONSOLE_PFN, &console_v))
+        BUG();
+    start_info_ptr->console.domU.mfn = pfn_to_mfn(console_v);
+#endif
+
+    HYPERVISOR_shared_info = map_shared_info((void*) start_info_ptr->shared_info);
+#ifndef CONFIG_PARAVIRT
+    xen_callback_vector();
+#endif
+    arch_mm_post_suspend(canceled);
 }
 
 void
-- 
2.3.2 (Apple Git-55)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 05/16] Save/Restore Support: Add kernel shutdown logic to shutdown.c

[Juergen Gross]

On 14/02/18 03:27, Bruno Alvisio wrote:
> Created shutdown.c for the shutdown thread and all the shutdown related
> functions.
> 
> Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
> ---
> Changesd since v1:
>    * Updated license to a BSD 3-clause. This license was taken
> from the updated original file. (Repo: sysml/mini-os)
> ---
>  Makefile           |   1 +
>  include/shutdown.h |  11 ++++
>  shutdown.c         | 188 +++++++++++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 200 insertions(+)
>  create mode 100644 include/shutdown.h
>  create mode 100644 shutdown.c
> 
> diff --git a/Makefile b/Makefile
> index 88315c4..6a05de6 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -53,6 +53,7 @@ src-y += mm.c
>  src-$(CONFIG_NETFRONT) += netfront.c
>  src-$(CONFIG_PCIFRONT) += pcifront.c
>  src-y += sched.c
> +src-y += shutdown.c
>  src-$(CONFIG_TEST) += test.c
>  src-$(CONFIG_BALLOON) += balloon.c
>  
> diff --git a/include/shutdown.h b/include/shutdown.h
> new file mode 100644
> index 0000000..a5ec019
> --- /dev/null
> +++ b/include/shutdown.h
> @@ -0,0 +1,11 @@
> +#ifndef _SHUTDOWN_H_
> +#define _SHUTDOWN_H_
> +
> +#include <mini-os/hypervisor.h>
> +
> +void init_shutdown(start_info_t *si);
> +
> +void kernel_shutdown(int reason) __attribute__((noreturn));
> +void kernel_suspend(void);
> +
> +#endif
> diff --git a/shutdown.c b/shutdown.c
> new file mode 100644
> index 0000000..aba146e
> --- /dev/null
> +++ b/shutdown.c
> @@ -0,0 +1,188 @@
> +/*
> + *          MiniOS
> + *
> + *   file: fromdevice.cc

shutdown.c?

> + *
> + * Authors: Joao Martins <joao.martins@neclab.eu>
> + *
> + *
> + * Copyright (c) 2014, NEC Europe Ltd., NEC Corporation. All rights reserved.
> + *
> + * Redistribution and use in source and binary forms, with or without
> + * modification, are permitted provided that the following conditions
> + * are met:
> + *
> + * 1. Redistributions of source code must retain the above copyright
> + *    notice, this list of conditions and the following disclaimer.
> + * 2. Redistributions in binary form must reproduce the above copyright
> + *    notice, this list of conditions and the following disclaimer in the
> + *    documentation and/or other materials provided with the distribution.
> + * 3. Neither the name of the copyright holder nor the names of its
> + *    contributors may be used to endorse or promote products derived from
> + *    this software without specific prior written permission.
> + *
> + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
> + * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
> + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
> + * ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
> + * LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
> + * CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
> + * SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
> + * INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
> + * CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
> + * ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
> + * POSSIBILITY OF SUCH DAMAGE.
> + *
> + * THIS HEADER MAY NOT BE EXTRACTED OR MODIFIED IN ANY WAY.
> + */
> +
> +#include <mini-os/os.h>
> +#include <mini-os/events.h>
> +#include <mini-os/kernel.h>
> +#include <mini-os/sched.h>
> +#include <mini-os/shutdown.h>
> +#include <mini-os/lib.h>
> +#include <mini-os/xenbus.h>
> +#include <mini-os/xmalloc.h>
> +
> +
> +static start_info_t *start_info_ptr;
> +
> +static const char *path = "control/shutdown";
> +static const char *token = "control/shutdown";
> +static xenbus_event_queue events = NULL;
> +static int end_shutdown_thread = 0;
> +
> +#ifdef CONFIG_XENBUS
> +/* This should be overridden by the application we are linked against. */
> +__attribute__((weak)) void app_shutdown(unsigned reason)
> +{
> +    printk("Shutdown requested: %d\n", reason);
> +    if (reason == SHUTDOWN_suspend) {
> +        kernel_suspend();
> +    } else {
> +        struct sched_shutdown sched_shutdown = { .reason = reason };
> +        HYPERVISOR_sched_op(SCHEDOP_shutdown, &sched_shutdown);
> +    }
> +}
> +
> +static void shutdown_thread(void *p)
> +{
> +    char *shutdown, *err;
> +    unsigned int shutdown_reason;
> +
> +    xenbus_watch_path_token(XBT_NIL, path, token, &events);
> +
> +    for ( ;; ) {
> +        xenbus_wait_for_watch(&events);
> +        if ((err = xenbus_read(XBT_NIL, path, &shutdown))) {
> +            free(err);
> +            do_exit();
> +        }
> +
> +        if (end_shutdown_thread)
> +            break;
> +
> +        if (!strcmp(shutdown, "")) {
> +            /* Avoid spurious event on xenbus */
> +            /* FIXME: investigate the reason of the spurious event */

Remove the FIXME, please.

Watches will fire e.g. directly after setting them up once.


With above issues fixed you can add my:

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 10/16] Save/Restore Support: Add suspend/resume support for timers

[Juergen Gross]

On 14/02/18 03:27, Bruno Alvisio wrote:
> Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 13/16] Save/Restore Support: Add suspend/restore support for Grant Tables.

[Juergen Gross]

On 14/02/18 03:27, Bruno Alvisio wrote:
> Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>
> ---
> Changed since v1:
>     - Moved suspend/resume _gnttab to arch specific files
> ---
>  arch/x86/mm.c    | 34 ++++++++++++++++++++++++++++++++++
>  gnttab.c         | 10 ++++++++++
>  include/gnttab.h |  4 ++++
>  kernel.c         |  4 ++++
>  4 files changed, 52 insertions(+)
> 
> diff --git a/arch/x86/mm.c b/arch/x86/mm.c
> index 1b163ac..2597c5b 100644
> --- a/arch/x86/mm.c
> +++ b/arch/x86/mm.c
> @@ -917,6 +917,40 @@ grant_entry_v1_t *arch_init_gnttab(int nr_grant_frames)
>      return map_frames(frames, nr_grant_frames);
>  }
>  
> +void arch_suspend_gnttab(grant_entry_v1_t *gnttab_table, int nr_grant_frames)
> +{
> +#ifdef CONFIG_PARAVIRT
> +    int i;
> +
> +    for (i = 0; i < nr_grant_frames; i++) {
> +        HYPERVISOR_update_va_mapping((unsigned long)(((char *)gnttab_table) + PAGE_SIZE*i),
> +                (pte_t){0x0<<PAGE_SHIFT}, UVMF_INVLPG);

Coding style (blanks around operators, once again below).


Other than that:

Reviewed-by: Juergen Gross <jgross@suse.com>


Juergen

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 00/16] Save/Restore Support for mini-OS PVH

[Bruno Alvisio]

Hi all,

I am sending the second revision for supporting save/restore in Mini-OS PVH. The 
branch can be found at: 

https://github.com/balvisio/mini-os/tree/feature/mini-os-suspend-support-submission-2

Feedback would be greatly appreciated.

Cheers,

Bruno

Signed-off-by: Bruno Alvisio <bruno.alvisio@gmail.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel