[PATCH 0/14][user-cr] Enable linking with LIBLXC

[PATCH 0/14][user-cr] Enable linking with LIBLXC

[Sukadev Bhattiprolu]

Patches to enable linking LIBLXC with USERCR.

Patches 1 through 9 are simply rebased patches due to the dropping of one
of the earlier patches (Oren had nacked one patch in my earlier set. See

https://lists.linux-foundation.org/pipermail/containers/2010-March/023564.html

Patches 10 through 14 are additional changes needed to get LXC checkpoint
and restart to work.

	[PATCH 01/14] Add app_restart_args->debug
	[PATCH 02/14] Add app_restart_args->verbose
	[PATCH 03/14] Add app_restart_args->ulogfd
	[PATCH 04/14] Add app_restart_args->uerrfd
	[PATCH 05/14] Define INIT_SIGNAL_ARRAY
	[PATCH 06/14] Create common.h
	[PATCH 07/14] Create app-checkpoint.h
	[PATCH 08/14] restart: Move main() to restart-main.c
	[PATCH 09/14] checkpoint: Move main() to checkpoint-main.c

	[PATCH 10/14] Have app_restart() return pid
	[PATCH 11/14] restart: Define process_args()
	[PATCH 12/14] app_restart(): mnt-pty implies mntns
	[PATCH 13/14] restart: Move args checking to app_restart()
	[PATCH 14/14] Minimize unshare() calls

With these patches, USERCR exports app_checkpoint() and app_restart() 
functions in checkpoint.o and restart.o. LXC links directly with these
.o files to checkpoint/restart its containers.

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>

[PATCH 01/14][user-cr] Add app_restart_args->debug

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Thu, 4 Mar 2010 17:46:31 -0800
Subject: [PATCH 01/14][user-cr] Add app_restart_args->debug

Make 'debug' a field in struct app_restart_args so other callers of
app_restart() can also control the debug output of app_restart().

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 restart.c |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/restart.c b/restart.c
index 0c74bb6..365b7ad 100644
--- a/restart.c
+++ b/restart.c
@@ -386,6 +386,7 @@ struct app_restart_args {
 	int infd;
 	int klogfd;
 	long warn;
+	int debug;
 	long fail;
 	int keep_lsm;
 };
@@ -581,7 +582,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 			args->copy_status = 1;
 			break;
 		case 'd':
-			global_debug = 1;
+			global_debug = args->debug = 1;
 			break;
 		case 'F':
 			args->freezer = optarg;
@@ -815,6 +816,8 @@ int app_restart(struct app_restart_args *args)
 	memset(&ctx, 0, sizeof(ctx));
 	ctx.args = args;
 
+	global_debug = args->debug;
+
 	/* input file descriptor (default: stdin) */
 	if (args->infd >= 0) {
 		if (dup2(args->infd, STDIN_FILENO) < 0) {
-- 
1.6.0.4

[PATCH 02/14][user-cr] Add app_restart_args->verbose

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Thu, 4 Mar 2010 17:53:27 -0800
Subject: [PATCH 02/14][user-cr] Add app_restart_args->verbose

Make 'verbose' a field in struct app_restart_args so other callers of
app_restart() can also control the verbose output of app_restart().

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 restart.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletions(-)

diff --git a/restart.c b/restart.c
index 365b7ad..9d16a4d 100644
--- a/restart.c
+++ b/restart.c
@@ -387,6 +387,7 @@ struct app_restart_args {
 	int klogfd;
 	long warn;
 	int debug;
+	int verbose;
 	long fail;
 	int keep_lsm;
 };
@@ -513,7 +514,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 		case 'h':
 			usage(usage_str);
 		case 'v':
-			global_verbose = 1;
+			global_verbose = args->verbose = 1;
 			break;
 		case 5:  /* --inspect */
 			args->inspect = 1;
@@ -817,6 +818,7 @@ int app_restart(struct app_restart_args *args)
 	ctx.args = args;
 
 	global_debug = args->debug;
+	global_verbose = args->verbose;
 
 	/* input file descriptor (default: stdin) */
 	if (args->infd >= 0) {
-- 
1.6.0.4

[PATCH 03/14][user-cr] Add app_restart_args->ulogfd

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Thu, 4 Mar 2010 22:33:34 -0800
Subject: [PATCH 03/14][user-cr] Add app_restart_args->ulogfd

Add a 'ulogfd' field in struct app_restart_args so other callers of
app_restart() can also choose a file descriptor for the log messages
from app_restart().

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 restart.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/restart.c b/restart.c
index 9d16a4d..5ae5ec4 100644
--- a/restart.c
+++ b/restart.c
@@ -385,6 +385,7 @@ struct app_restart_args {
 	char *freezer;
 	int infd;
 	int klogfd;
+	int ulogfd;
 	long warn;
 	int debug;
 	int verbose;
@@ -497,6 +498,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 	args->wait = 1;
 	args->infd = -1;
 	args->klogfd = -1;
+	args->ulogfd = fileno(stdout);
 	args->warn = CKPT_COND_WARN;
 	args->fail = CKPT_COND_FAIL;
 	no_pidns = 0;
@@ -819,6 +821,7 @@ int app_restart(struct app_restart_args *args)
 
 	global_debug = args->debug;
 	global_verbose = args->verbose;
+	global_ulogfd = args->ulogfd;
 
 	/* input file descriptor (default: stdin) */
 	if (args->infd >= 0) {
-- 
1.6.0.4

[PATCH 04/14][user-cr] Add app_restart_args->uerrfd

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Thu, 4 Mar 2010 22:36:31 -0800
Subject: [PATCH 04/14][user-cr] Add app_restart_args->uerrfd

Add a 'uerrfd' field in struct app_restart_args so other callers of
app_restart() can also choose a file descriptor for the error messages
from app_restart().

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 restart.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/restart.c b/restart.c
index 5ae5ec4..b526bbd 100644
--- a/restart.c
+++ b/restart.c
@@ -386,6 +386,7 @@ struct app_restart_args {
 	int infd;
 	int klogfd;
 	int ulogfd;
+	int uerrfd;
 	long warn;
 	int debug;
 	int verbose;
@@ -499,6 +500,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 	args->infd = -1;
 	args->klogfd = -1;
 	args->ulogfd = fileno(stdout);
+	args->uerrfd = fileno(stderr);
 	args->warn = CKPT_COND_WARN;
 	args->fail = CKPT_COND_FAIL;
 	no_pidns = 0;
@@ -822,6 +824,7 @@ int app_restart(struct app_restart_args *args)
 	global_debug = args->debug;
 	global_verbose = args->verbose;
 	global_ulogfd = args->ulogfd;
+	global_uerrfd = args->uerrfd;
 
 	/* input file descriptor (default: stdin) */
 	if (args->infd >= 0) {
-- 
1.6.0.4

[PATCH 05/14][user-cr] Define INIT_SIGNAL_ARRAY

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Thu, 4 Mar 2010 21:15:25 -0800
Subject: [PATCH 05/14][user-cr] Define INIT_SIGNAL_ARRAY

Code to initialize the signal_array[] table will need to be shared by more
than one file. To enable this sharing with, define INIT_SIGNAL_ARRAY which
can later be used in many places.

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 restart.c |   76 ++++++++++++++++++++++++++++++++-----------------------------
 1 files changed, 40 insertions(+), 36 deletions(-)

diff --git a/restart.c b/restart.c
index b526bbd..4cd0bad 100644
--- a/restart.c
+++ b/restart.c
@@ -157,46 +157,50 @@ static inline void ckpt_msg(int fd, char *format, ...)
 			ckpt_msg(global_ulogfd, __VA_ARGS__);	\
 	} while(0)
 
-#define SIGNAL_ENTRY(signal)  { SIG ## signal, #signal }
-
-struct {
+struct signal_array {
 	int signum;
 	char *sigstr;
-} signal_array[] = {
-	{ 0, "NONE" },
-	SIGNAL_ENTRY(ALRM),
-	SIGNAL_ENTRY(HUP),
-	SIGNAL_ENTRY(INT),
-	SIGNAL_ENTRY(KILL),
-	SIGNAL_ENTRY(PIPE),
-	SIGNAL_ENTRY(POLL),
-	SIGNAL_ENTRY(PROF),
-	SIGNAL_ENTRY(TERM),
-	SIGNAL_ENTRY(USR1),
-	SIGNAL_ENTRY(USR2),
-	SIGNAL_ENTRY(VTALRM),
-	SIGNAL_ENTRY(STKFLT),
-	SIGNAL_ENTRY(PWR),
-	SIGNAL_ENTRY(WINCH),
-	SIGNAL_ENTRY(CHLD),
-	SIGNAL_ENTRY(URG),
-	SIGNAL_ENTRY(TTIN),
-	SIGNAL_ENTRY(TTOU),
-	SIGNAL_ENTRY(STOP),
-	SIGNAL_ENTRY(CONT),
-	SIGNAL_ENTRY(ABRT),
-	SIGNAL_ENTRY(FPE),
-	SIGNAL_ENTRY(ILL),
-	SIGNAL_ENTRY(QUIT),
-	SIGNAL_ENTRY(SEGV),
-	SIGNAL_ENTRY(TRAP),
-	SIGNAL_ENTRY(SYS),
-	SIGNAL_ENTRY(BUS),
-	SIGNAL_ENTRY(XCPU),
-	SIGNAL_ENTRY(XFSZ),
-	{ -1, "LAST" },
 };
 
+#define SIGNAL_ENTRY(signal)  { SIG ## signal, #signal }
+
+#define INIT_SIGNAL_ARRAY { 	\
+	{ 0, "NONE" },		\
+	SIGNAL_ENTRY(ALRM),	\
+	SIGNAL_ENTRY(HUP),	\
+	SIGNAL_ENTRY(INT),	\
+	SIGNAL_ENTRY(KILL),	\
+	SIGNAL_ENTRY(PIPE),	\
+	SIGNAL_ENTRY(POLL),	\
+	SIGNAL_ENTRY(PROF),	\
+	SIGNAL_ENTRY(TERM),	\
+	SIGNAL_ENTRY(USR1),	\
+	SIGNAL_ENTRY(USR2),	\
+	SIGNAL_ENTRY(VTALRM),	\
+	SIGNAL_ENTRY(STKFLT),	\
+	SIGNAL_ENTRY(PWR),	\
+	SIGNAL_ENTRY(WINCH),	\
+	SIGNAL_ENTRY(CHLD),	\
+	SIGNAL_ENTRY(URG),	\
+	SIGNAL_ENTRY(TTIN),	\
+	SIGNAL_ENTRY(TTOU),	\
+	SIGNAL_ENTRY(STOP),	\
+	SIGNAL_ENTRY(CONT),	\
+	SIGNAL_ENTRY(ABRT),	\
+	SIGNAL_ENTRY(FPE),	\
+	SIGNAL_ENTRY(ILL),	\
+	SIGNAL_ENTRY(QUIT),	\
+	SIGNAL_ENTRY(SEGV),	\
+	SIGNAL_ENTRY(TRAP),	\
+	SIGNAL_ENTRY(SYS),	\
+	SIGNAL_ENTRY(BUS),	\
+	SIGNAL_ENTRY(XCPU),	\
+	SIGNAL_ENTRY(XFSZ),	\
+	{ -1, "LAST" },		\
+}
+
+static struct signal_array signal_array[] = INIT_SIGNAL_ARRAY;
+
 static char *sig2str(int sig)
 {
 	int i = 0;
-- 
1.6.0.4

[PATCH 06/14][user-cr] Create common.h

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Thu, 4 Mar 2010 21:30:50 -0800
Subject: [PATCH 06/14][user-cr] Create common.h

Code in common.h can be shared by checkpoint.c and restart.c for now.

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 Makefile     |    4 ++
 checkpoint.c |   31 +---------------
 common.h     |  112 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 restart.c    |  110 +--------------------------------------------------------
 4 files changed, 119 insertions(+), 138 deletions(-)
 create mode 100644 common.h

diff --git a/Makefile b/Makefile
index 64b5f73..406f685 100644
--- a/Makefile
+++ b/Makefile
@@ -13,6 +13,8 @@ CKPT_HEADERS = include/linux/checkpoint.h \
 		include/linux/checkpoint_hdr.h \
 		include/asm/checkpoint_hdr.h
 
+CR_OBJS = checkpoint.o restart.o
+
 # detect architecture (for eclone)
 SUBARCH ?= $(patsubst i%86,x86_32,$(shell uname -m))
 
@@ -50,6 +52,8 @@ $(LIB_ECLONE):
 # restart needs to be thread-safe
 restart: CFLAGS += -D__REENTRANT -pthread
 
+$(CR_OBJS): common.h
+
 # eclone() is architecture specific
 ifneq ($(SUBARCH),)
 $(ECLONE_PROGS): $(LIB_ECLONE) 
diff --git a/checkpoint.c b/checkpoint.c
index 86608b1..6e91149 100644
--- a/checkpoint.c
+++ b/checkpoint.c
@@ -21,6 +21,8 @@
 
 #include <linux/checkpoint.h>
 
+#include "common.h"
+
 static char usage_str[] =
 "usage: ckpt [opts] PID\n"
 "  'checkpoint' takes a checkpoint of the task indicated by PID, and all\n"
@@ -55,35 +57,6 @@ inline static int checkpoint(pid_t pid, int fd, unsigned long flags, int logfd)
 	return syscall(__NR_checkpoint, pid, fd, flags, logfd);
 }
 
-#define BUFSIZE  (4 * 4096)
-static inline void ckpt_msg(int fd, char *format, ...)
-{
-	va_list ap;
-	char *bufp;
-	if (fd < 0)
-		return;
-
-	va_start(ap, format);
-
-	bufp = malloc(BUFSIZE);
-	if(bufp) {
-		vsnprintf(bufp, BUFSIZE, format, ap);
-		write(fd, bufp, strlen(bufp));
-	}
-	free(bufp);
-
-	va_end(ap);
-}
-
-#define ckpt_err(...)				\
-	ckpt_msg(global_uerrfd, __VA_ARGS__)
-
-#define ckpt_perror(s)                                                  \
-	do {                                                            \
-		ckpt_msg(global_uerrfd, s);                             \
-		ckpt_msg(global_uerrfd, ": %s\n", strerror(errno));     \
-	} while (0)
-
 static void usage(char *str)
 {
 	ckpt_err("%s", str);
diff --git a/common.h b/common.h
new file mode 100644
index 0000000..99b224d
--- /dev/null
+++ b/common.h
@@ -0,0 +1,112 @@
+#include <stdio.h>
+#include <signal.h>
+
+#define BUFSIZE  (4 * 4096)
+
+static inline void ckpt_msg(int fd, char *format, ...)
+{
+	va_list ap;
+	char *bufp;
+	if (fd < 0)
+		return;
+
+	va_start(ap, format);
+
+	bufp = malloc(BUFSIZE);
+	if(bufp) {
+		vsnprintf(bufp, BUFSIZE, format, ap);
+		write(fd, bufp, strlen(bufp));
+	}
+	free(bufp);
+
+	va_end(ap);
+}
+
+#define ckpt_perror(s) 							\
+	do {								\
+		ckpt_msg(global_uerrfd, s);				\
+		ckpt_msg(global_uerrfd, ": %s\n", strerror(errno));	\
+	} while (0)
+
+#ifdef CHECKPOINT_DEBUG
+#define ckpt_dbg(_format, _args...)					\
+	do {								\
+		if (global_debug)					\
+			ckpt_msg(global_uerrfd, "<%d>" _format, 	\
+					_gettid(), ##_args); 		\
+	} while (0)
+#define ckpt_dbg_cont(_format, _args...)				\
+	do {								\
+		if (global_debug)					\
+			ckpt_msg(global_uerrfd, _format, ##_args);	\
+	} while (0)
+#else
+#define ckpt_dbg(_format, _args...)  \
+	do { } while (0)
+#define ckpt_dbg_cont(_format, _args...)  \
+	do { } while (0)
+#endif
+
+#define ckpt_err(...)  \
+	ckpt_msg(global_uerrfd, __VA_ARGS__)
+
+#define ckpt_verbose(...)					\
+	do {							\
+		if (global_verbose)				\
+			ckpt_msg(global_ulogfd, __VA_ARGS__);	\
+	} while(0)
+
+struct signal_array {
+	int signum;
+	char *sigstr;
+};
+
+#define SIGNAL_ENTRY(signal)  { SIG ## signal, #signal }
+
+#define INIT_SIGNAL_ARRAY { 	\
+	{ 0, "NONE" },		\
+	SIGNAL_ENTRY(ALRM),	\
+	SIGNAL_ENTRY(HUP),	\
+	SIGNAL_ENTRY(INT),	\
+	SIGNAL_ENTRY(KILL),	\
+	SIGNAL_ENTRY(PIPE),	\
+	SIGNAL_ENTRY(POLL),	\
+	SIGNAL_ENTRY(PROF),	\
+	SIGNAL_ENTRY(TERM),	\
+	SIGNAL_ENTRY(USR1),	\
+	SIGNAL_ENTRY(USR2),	\
+	SIGNAL_ENTRY(VTALRM),	\
+	SIGNAL_ENTRY(STKFLT),	\
+	SIGNAL_ENTRY(PWR),	\
+	SIGNAL_ENTRY(WINCH),	\
+	SIGNAL_ENTRY(CHLD),	\
+	SIGNAL_ENTRY(URG),	\
+	SIGNAL_ENTRY(TTIN),	\
+	SIGNAL_ENTRY(TTOU),	\
+	SIGNAL_ENTRY(STOP),	\
+	SIGNAL_ENTRY(CONT),	\
+	SIGNAL_ENTRY(ABRT),	\
+	SIGNAL_ENTRY(FPE),	\
+	SIGNAL_ENTRY(ILL),	\
+	SIGNAL_ENTRY(QUIT),	\
+	SIGNAL_ENTRY(SEGV),	\
+	SIGNAL_ENTRY(TRAP),	\
+	SIGNAL_ENTRY(SYS),	\
+	SIGNAL_ENTRY(BUS),	\
+	SIGNAL_ENTRY(XCPU),	\
+	SIGNAL_ENTRY(XFSZ),	\
+	{ -1, "LAST" },		\
+}
+
+#define CKPT_COND_PIDZERO  0x1
+#define CKPT_COND_MNTPROC  0x2
+#define CKPT_COND_MNTPTY   0x4
+
+#define CKPT_COND_NONE     0
+#define CKPT_COND_ANY      ULONG_MAX
+
+/* default for skip/warn/fail */
+#define CKPT_COND_WARN     (CKPT_COND_MNTPROC | \
+			    CKPT_COND_MNTPTY)
+#define CKPT_COND_FAIL     (CKPT_COND_NONE)
+
diff --git a/restart.c b/restart.c
index 4cd0bad..7c9db80 100644
--- a/restart.c
+++ b/restart.c
@@ -40,6 +40,7 @@
 #include "eclone.h"
 #include "genstack.h"
 #include "compat.h"
+#include "common.h"
 
 static char usage_str[] =
 "usage: restart [opts]\n"
@@ -102,103 +103,6 @@ static char usage_str[] =
  * of the checkpoint image stream.
  */
 
-#define BUFSIZE  (4 * 4096)
-
-static inline void ckpt_msg(int fd, char *format, ...)
-{
-	va_list ap;
-	char *bufp;
-	if (fd < 0)
-		return;
-
-	va_start(ap, format);
-
-	bufp = malloc(BUFSIZE);
-	if(bufp) {
-		vsnprintf(bufp, BUFSIZE, format, ap);
-		write(fd, bufp, strlen(bufp));
-	}
-	free(bufp);
-
-	va_end(ap);
-}
-
-#define ckpt_perror(s) 							\
-	do {								\
-		ckpt_msg(global_uerrfd, s);				\
-		ckpt_msg(global_uerrfd, ": %s\n", strerror(errno));	\
-	} while (0)
-
-#ifdef CHECKPOINT_DEBUG
-#define ckpt_dbg(_format, _args...)					\
-	do {								\
-		if (global_debug)					\
-			ckpt_msg(global_uerrfd, "<%d>" _format, 	\
-					_gettid(), ##_args); 		\
-	} while (0)
-#define ckpt_dbg_cont(_format, _args...)				\
-	do {								\
-		if (global_debug)					\
-			ckpt_msg(global_uerrfd, _format, ##_args);	\
-	} while (0)
-#else
-#define ckpt_dbg(_format, _args...)  \
-	do { } while (0)
-#define ckpt_dbg_cont(_format, _args...)  \
-	do { } while (0)
-#endif
-
-#define ckpt_err(...)  \
-	ckpt_msg(global_uerrfd, __VA_ARGS__)
-
-#define ckpt_verbose(...)					\
-	do {							\
-		if (global_verbose)				\
-			ckpt_msg(global_ulogfd, __VA_ARGS__);	\
-	} while(0)
-
-struct signal_array {
-	int signum;
-	char *sigstr;
-};
-
-#define SIGNAL_ENTRY(signal)  { SIG ## signal, #signal }
-
-#define INIT_SIGNAL_ARRAY { 	\
-	{ 0, "NONE" },		\
-	SIGNAL_ENTRY(ALRM),	\
-	SIGNAL_ENTRY(HUP),	\
-	SIGNAL_ENTRY(INT),	\
-	SIGNAL_ENTRY(KILL),	\
-	SIGNAL_ENTRY(PIPE),	\
-	SIGNAL_ENTRY(POLL),	\
-	SIGNAL_ENTRY(PROF),	\
-	SIGNAL_ENTRY(TERM),	\
-	SIGNAL_ENTRY(USR1),	\
-	SIGNAL_ENTRY(USR2),	\
-	SIGNAL_ENTRY(VTALRM),	\
-	SIGNAL_ENTRY(STKFLT),	\
-	SIGNAL_ENTRY(PWR),	\
-	SIGNAL_ENTRY(WINCH),	\
-	SIGNAL_ENTRY(CHLD),	\
-	SIGNAL_ENTRY(URG),	\
-	SIGNAL_ENTRY(TTIN),	\
-	SIGNAL_ENTRY(TTOU),	\
-	SIGNAL_ENTRY(STOP),	\
-	SIGNAL_ENTRY(CONT),	\
-	SIGNAL_ENTRY(ABRT),	\
-	SIGNAL_ENTRY(FPE),	\
-	SIGNAL_ENTRY(ILL),	\
-	SIGNAL_ENTRY(QUIT),	\
-	SIGNAL_ENTRY(SEGV),	\
-	SIGNAL_ENTRY(TRAP),	\
-	SIGNAL_ENTRY(SYS),	\
-	SIGNAL_ENTRY(BUS),	\
-	SIGNAL_ENTRY(XCPU),	\
-	SIGNAL_ENTRY(XFSZ),	\
-	{ -1, "LAST" },		\
-}
-
 static struct signal_array signal_array[] = INIT_SIGNAL_ARRAY;
 
 static char *sig2str(int sig)
@@ -398,18 +302,6 @@ struct app_restart_args {
 	int keep_lsm;
 };
 
-#define CKPT_COND_PIDZERO  0x1
-#define CKPT_COND_MNTPROC  0x2
-#define CKPT_COND_MNTPTY   0x4
-
-#define CKPT_COND_NONE     0
-#define CKPT_COND_ANY      ULONG_MAX
-
-/* default for skip/warn/fail */
-#define CKPT_COND_WARN     (CKPT_COND_MNTPROC | \
-			    CKPT_COND_MNTPTY)
-#define CKPT_COND_FAIL     (CKPT_COND_NONE)
-
 static void usage(char *str)
 {
 	ckpt_err("%s", str);
-- 
1.6.0.4

[PATCH 07/14][user-cr] Create app-checkpoint.h

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Thu, 4 Mar 2010 21:51:27 -0800
Subject: [PATCH 07/14][user-cr] Create app-checkpoint.h

Create a new header file, app-checkpoint.h that can be used to export
the app_checkpoint() and app_restart() functionality to other users
(such as LXC).

For now, leave 'global_send_sigint' as an external variable. We should
eventually implement an API that gives callers better control of how
to handle signals during restart.

Changelog[v2]:
	- (In response to Oren Laadan's comment): Leave 'global_send_sigint'
	  as an extern and add a TODO to remove later.

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 Makefile         |    2 +-
 app-checkpoint.h |   42 ++++++++++++++++++++++++++++++++++++++++++
 checkpoint.c     |    9 +--------
 restart.c        |   31 +++++++------------------------
 4 files changed, 51 insertions(+), 33 deletions(-)
 create mode 100644 app-checkpoint.h

diff --git a/Makefile b/Makefile
index 406f685..639d42f 100644
--- a/Makefile
+++ b/Makefile
@@ -52,7 +52,7 @@ $(LIB_ECLONE):
 # restart needs to be thread-safe
 restart: CFLAGS += -D__REENTRANT -pthread
 
-$(CR_OBJS): common.h
+$(CR_OBJS): common.h app-checkpoint.h
 
 # eclone() is architecture specific
 ifneq ($(SUBARCH),)
diff --git a/app-checkpoint.h b/app-checkpoint.h
new file mode 100644
index 0000000..f740085
--- /dev/null
+++ b/app-checkpoint.h
@@ -0,0 +1,42 @@
+
+struct app_checkpoint_args {
+	int outfd;
+	int logfd;
+	int uerrfd;
+	int container;
+	int verbose;
+};
+
+struct app_restart_args {
+	int self;
+	int pids;
+	int pidns;
+	int inspect;
+	char *root;
+	int wait;
+	int mntns;
+	int mnt_pty;
+	int show_status;
+	int copy_status;
+	char *freezer;
+	int infd;
+	int klogfd;
+	int ulogfd;
+	int uerrfd;
+	long warn;
+	int debug;
+	int verbose;
+	long fail;
+	int keep_lsm;
+};
+
+/*
+ * TODO: Implement an API to let callers choose if/how an interrupt be sent
+ *  	 and remove global_send_sigint.
+ */
+extern int global_send_sigint;
+
+extern int app_checkpoint(int pid, unsigned long flags,
+				struct app_checkpoint_args *args);
+
+extern int app_restart(struct app_restart_args *args);
diff --git a/checkpoint.c b/checkpoint.c
index 6e91149..291cb36 100644
--- a/checkpoint.c
+++ b/checkpoint.c
@@ -21,6 +21,7 @@
 
 #include <linux/checkpoint.h>
 
+#include "app-checkpoint.h"
 #include "common.h"
 
 static char usage_str[] =
@@ -44,14 +45,6 @@ static char usage_str[] =
 
 static int global_uerrfd = -1;
 
-struct app_checkpoint_args {
-	int outfd;
-	int logfd;
-	int uerrfd;
-	int container;
-	int verbose;
-};
-
 inline static int checkpoint(pid_t pid, int fd, unsigned long flags, int logfd)
 {
 	return syscall(__NR_checkpoint, pid, fd, flags, logfd);
diff --git a/restart.c b/restart.c
index 7c9db80..bbc5ef3 100644
--- a/restart.c
+++ b/restart.c
@@ -40,6 +40,7 @@
 #include "eclone.h"
 #include "genstack.h"
 #include "compat.h"
+#include "app-checkpoint.h"
 #include "common.h"
 
 static char usage_str[] =
@@ -216,9 +217,14 @@ static int global_verbose;
 static pid_t global_child_pid;
 static int global_child_status;
 static int global_child_collected;
-static int global_send_sigint = -1;
 static int global_sent_sigint;
 
+/*
+ * TODO: Implement an API to let callers choose if/how an interrupt be sent
+ * 	 and remove global_send_sigint.
+ */
+int global_send_sigint = -1;
+
 static int ckpt_remount_proc(struct ckpt_ctx *ctx);
 static int ckpt_remount_devpts(struct ckpt_ctx *ctx);
 
@@ -279,29 +285,6 @@ struct pid_swap {
 	pid_t new;
 };
 
-struct app_restart_args {
-	int self;
-	int pids;
-	int pidns;
-	int inspect;
-	char *root;
-	int wait;
-	int mntns;
-	int mnt_pty;
-	int show_status;
-	int copy_status;
-	char *freezer;
-	int infd;
-	int klogfd;
-	int ulogfd;
-	int uerrfd;
-	long warn;
-	int debug;
-	int verbose;
-	long fail;
-	int keep_lsm;
-};
-
 static void usage(char *str)
 {
 	ckpt_err("%s", str);
-- 
1.6.0.4

[PATCH 08/14][user-cr] restart: Move main() to restart-main.c

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Thu, 4 Mar 2010 21:46:14 -0800
Subject: [PATCH 08/14][user-cr] restart: Move main() to restart-main.c

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 Makefile       |    4 +-
 restart-main.c |  350 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 restart.c      |  331 -----------------------------------------------------
 3 files changed, 353 insertions(+), 332 deletions(-)
 create mode 100644 restart-main.c

diff --git a/Makefile b/Makefile
index 639d42f..7e1d701 100644
--- a/Makefile
+++ b/Makefile
@@ -13,7 +13,7 @@ CKPT_HEADERS = include/linux/checkpoint.h \
 		include/linux/checkpoint_hdr.h \
 		include/asm/checkpoint_hdr.h
 
-CR_OBJS = checkpoint.o restart.o
+CR_OBJS = checkpoint.o restart.o restart-main.o
 
 # detect architecture (for eclone)
 SUBARCH ?= $(patsubst i%86,x86_32,$(shell uname -m))
@@ -54,6 +54,8 @@ restart: CFLAGS += -D__REENTRANT -pthread
 
 $(CR_OBJS): common.h app-checkpoint.h
 
+restart: restart.o restart-main.o
+
 # eclone() is architecture specific
 ifneq ($(SUBARCH),)
 $(ECLONE_PROGS): $(LIB_ECLONE) 
diff --git a/restart-main.c b/restart-main.c
new file mode 100644
index 0000000..d3a1782
--- /dev/null
+++ b/restart-main.c
@@ -0,0 +1,350 @@
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <errno.h>
+#include <string.h>
+#include <fcntl.h>
+#include <stdarg.h>
+#include <limits.h>
+#include <getopt.h>
+
+#include "app-checkpoint.h"
+#include "common.h"
+
+static int global_ulogfd;
+static int global_uerrfd;
+static int global_debug;
+static int global_verbose;
+static struct signal_array signal_array[] = INIT_SIGNAL_ARRAY;
+
+static char usage_str[] =
+"usage: restart [opts]\n"
+"  restart restores from a checkpoint image by first creating in userspace\n"
+"  the original tasks tree, and then calling sys_restart by each task.\n"
+"Options:\n"
+"  -h,--help             print this help message\n"
+"  -p,--pidns            create a new pid namspace (default with --pids)\n"
+"  -P,--no-pidns         do not create a new pid namespace (default)\n"
+"     --pids             restore original pids (default with --pidns)\n"
+"     --self             restart a single task, usually from self-checkpoint\n"
+"  -r,--root=ROOT        restart under the directory ROOT instead of current\n"
+"     --signal=SIG       send SIG to root task on SIGINT (default: SIGKILL\n"
+"                        to container root, SIGINT otherwise)\n"
+"     --mntns            restart under a private mounts namespace\n"
+"     --mount-pty        start in a new devpts namespace to supprt ptys\n"
+"  -w,--wait             wait for root task to termiate (default)\n"
+"     --show-status      show exit status of root task (implies -w)\n"
+"     --copy-status      imitate exit status of root task (implies -w)\n"
+"  -W,--no-wait          do not wait for root task to terminate\n"
+"  -k,--keeplsm          try to recreate original LSM labels on all objects\n"
+"  -F,--freezer=CGROUP   freeze tasks in freezer group CGROUP on success\n"
+"  -i,--input=FILE       read data from FILE instead of standard input\n"
+"     --input-fd=FD      read data from file descriptor FD (instead of stdin)\n"
+"  -l,--logfile=FILE     write error and debug data to FILE (default=none)\n"
+"     --logfile-fd=FD    write error and debug data to file desctiptor FD\n"
+"     --inspect          inspect image on-the-fly for error records\n"
+"  -v,--verbose          verbose output\n"
+"  -d,--debug            debugging output\n"
+"     --skip-COND        skip condition COND, and proceed anyway\n"
+"     --warn-COND        warn on condition COND, but proceed anyway\n"
+"     --fail-COND        warn on condition COND, and abort operation\n"
+"  	  COND=any:        any condition\n"
+"  	  COND=pidzero:    task with sid/pgid zero in a --no-pidns restart\n"
+"  	  COND=mntproc:    /proc isn't already mounted at restart (def: warn)\n"
+"";
+
+static void usage(char *str)
+{
+	ckpt_err("%s", str);
+	exit(1);
+}
+
+/* negative retval means error */
+static int str2num(char *str)
+{
+	char *nptr;
+	int num;
+
+	num = strtol(str, &nptr, 10);
+	if (nptr - str != strlen(str))
+		num = -1;
+	return num;
+}
+
+static long cond_to_mask(const char *cond)
+{
+	static struct {
+		char *cond;
+		long mask;
+	} conditions[] = {
+		{"pidzero", CKPT_COND_PIDZERO},
+		{"mntproc", CKPT_COND_MNTPROC},
+		{"any", CKPT_COND_ANY},
+		{NULL, 0}
+	};
+
+	int i;
+
+	for (i = 0; conditions[i].cond; i++)
+		if (!strcmp(cond, conditions[i].cond))
+			return conditions[i].mask;
+
+	ckpt_err("restart: invalid warn/fail condition '%s'\n", cond);
+	exit(1);
+}
+
+static int str2sig(char *str)
+{
+	int sig = 0;
+
+	do {
+		if (!strcmp(signal_array[sig].sigstr, str))
+			return signal_array[sig].signum;
+	} while (signal_array[++sig].signum >= 0);
+
+	return -1;
+}
+
+static void parse_args(struct app_restart_args *args, int argc, char *argv[])
+{
+	static struct option opts[] = {
+		{ "help",	no_argument,		NULL, 'h' },
+		{ "pidns",	no_argument,		NULL, 'p' },
+		{ "no-pidns",	no_argument,		NULL, 'P' },
+		{ "pids",	no_argument,		NULL, 3 },
+		{ "self",	no_argument,		NULL, 6},
+		{ "signal",	required_argument,	NULL, 4 },
+		{ "inspect",	no_argument,		NULL, 5 },
+		{ "keeplsm",	no_argument,		NULL, 'k' },
+		{ "input",	required_argument,	NULL, 'i' },
+		{ "input-fd",	required_argument,	NULL, 7 },
+		{ "logfile",	required_argument,	NULL, 'l' },
+		{ "logfile-fd",	required_argument,	NULL, 8 },
+		{ "root",	required_argument,	NULL, 'r' },
+		{ "mntns",	no_argument,		NULL, 11 },
+		{ "wait",	no_argument,		NULL, 'w' },
+		{ "show-status",	no_argument,	NULL, 1 },
+		{ "copy-status",	no_argument,	NULL, 2 },
+		{ "no-wait",	no_argument,		NULL, 'W' },
+		{ "freezer",	required_argument,	NULL, 'F' },
+		{ "verbose",	no_argument,		NULL, 'v' },
+		{ "debug",	no_argument,		NULL, 'd' },
+		{ "warn-pidzero",	no_argument,	NULL, 9 },
+		{ "fail-pidzero",	no_argument,	NULL, 10 },
+		{ "mount-pty",	no_argument,		NULL, 12 },
+		{ NULL,		0,			NULL, 0 }
+	};
+	static char optc[] = "hdvkpPwWF:r:i:l:";
+
+	int optind;
+	int sig;
+	int no_pidns;
+
+	char *klogfile;
+	char *input;
+
+	/* defaults */
+	memset(args, 0, sizeof(*args));
+	args->wait = 1;
+	args->infd = -1;
+	args->klogfd = -1;
+	args->ulogfd = fileno(stdout);
+	args->uerrfd = fileno(stderr);
+	args->warn = CKPT_COND_WARN;
+	args->fail = CKPT_COND_FAIL;
+	no_pidns = 0;
+
+	klogfile = NULL;
+	input = NULL;
+
+	while (1) {
+		int c = getopt_long(argc, argv, optc, opts, &optind);
+		if (c == -1)
+			break;
+		switch (c) {
+		case '?':
+			exit(1);
+		case 'h':
+			usage(usage_str);
+		case 'v':
+			global_verbose = args->verbose = 1;
+			break;
+		case 5:  /* --inspect */
+			args->inspect = 1;
+			break;
+		case 'i':
+			input = optarg;
+			break;
+		case 7:
+			args->infd = str2num(optarg);
+			if (args->infd < 0) {
+				ckpt_err("restart: invalid file descriptor\n");
+				exit(1);
+			}
+			break;
+		case 'l':
+			klogfile = optarg;
+			break;
+		case 8:
+			args->klogfd = str2num(optarg);
+			if (args->klogfd < 0) {
+				ckpt_err("restart: invalid file descriptor\n");
+				exit(1);
+			}
+			break;
+		case 'p':
+			args->pidns = 1;
+			break;
+		case 'P':
+			no_pidns = 1;
+			break;
+		case 6:  /* --self */
+			args->self = 1;
+			break;
+		case 4:  /* --signal */
+			sig = str2sig(optarg);
+			if (sig < 0)
+				sig = str2num(optarg);
+			if (sig < 0 || sig >= NSIG) {
+				ckpt_err("restart: invalid signal\n");
+				exit(1);
+			}
+			global_send_sigint = sig;
+			break;
+		case 3:  /* --pids */
+			args->pids = 1;
+			args->pidns = 1;  /* implied */
+			break;
+		case 'r':
+			args->root = optarg;
+			break;
+		case 'w':
+			args->wait = 1;
+			break;
+		case 'W':
+			args->wait = 0;
+			break;
+		case 'k':
+			args->keep_lsm = 1;
+			break;
+		case 1:  /* --show-status */
+			args->wait = 1;
+			args->show_status = 1;
+			break;
+		case 2: /* --copy-status */
+			args->wait = 1;
+			args->copy_status = 1;
+			break;
+		case 'd':
+			global_debug = args->debug = 1;
+			break;
+		case 'F':
+			args->freezer = optarg;
+			break;
+		case 9:
+			args->warn |= cond_to_mask(&opts[optind].name[5]);
+			break;
+		case 10:
+			args->fail |= cond_to_mask(&opts[optind].name[5]);
+			break;
+		case 11:
+			args->mntns = 1;
+			break;
+		case 12:
+			args->mnt_pty = 1;
+			break;
+		default:
+			usage(usage_str);
+		}
+	}
+
+	if (no_pidns)
+		args->pidns = 0;
+
+#ifndef CLONE_NEWPID
+	if (args->pidns) {
+		ckpt_err("This version of restart was compiled without "
+		       "support for --pidns.\n");
+		exit(1);
+	}
+#endif
+
+#ifndef CHECKPOINT_DEBUG
+	if (global_debug) {
+		ckpt_err("This version of restart was compiled without "
+		       "support for --debug.\n");
+		exit(1);
+	}
+#endif
+
+	if (args->pidns)
+		args->pids = 1;
+
+#if 0   /* Defered until __NR_eclone makes it to standard headers */
+#ifndef __NR_eclone
+	if (args->pids) {
+		ckpt_err("This version of restart was compiled without "
+		       "support for --pids.\n");
+		exit(1);
+	}
+#endif
+#endif
+
+	if (args->self &&
+	    (args->pids || args->pidns || no_pidns ||
+	     args->show_status || args->copy_status || args->freezer)) {
+		ckpt_err("Invalid mix of --self with multiprocess options\n");
+		exit(1);
+	}
+
+	if (input && args->infd >= 0) {
+		ckpt_err("Invalid use of both -i/--input and --input-fd\n");
+		exit(1);
+	}
+
+	/* input file ? */
+	if (input) {
+		args->infd = open(input, O_RDONLY, 0);
+		if (args->infd < 0) {
+			ckpt_perror("open input file");
+			exit(1);
+		}
+	}
+
+	if (klogfile && args->klogfd >= 0) {
+		ckpt_err("Invalid use of both -l/--logfile and --logfile-fd\n");
+		exit(1);
+	}
+
+	/* (optional) log file */
+	if (klogfile) {
+		args->klogfd = open(klogfile, O_RDWR | O_CREAT | O_EXCL, 0644);
+		if (args->klogfd < 0) {
+			ckpt_perror("open log file");
+			exit(1);
+		}
+	}
+
+
+	if (args->mnt_pty)
+		args->mntns = 1;
+}
+
+int main(int argc, char *argv[])
+{
+	struct app_restart_args args;
+
+	/*
+	 * Initialize the log/error fds early so even parse_args() errors
+	 * are redirected here. Even if we later implement command line options
+	 * that override these, any errors/messages that occur before those
+	 * new options are parsed still go to stdout/stderr
+	 */
+	global_ulogfd = fileno(stdout);
+	global_uerrfd = fileno(stderr);
+
+	parse_args(&args, argc, argv);
+
+	return app_restart(&args);
+}
+
diff --git a/restart.c b/restart.c
index bbc5ef3..0118ad9 100644
--- a/restart.c
+++ b/restart.c
@@ -43,42 +43,6 @@
 #include "app-checkpoint.h"
 #include "common.h"
 
-static char usage_str[] =
-"usage: restart [opts]\n"
-"  restart restores from a checkpoint image by first creating in userspace\n"
-"  the original tasks tree, and then calling sys_restart by each task.\n"
-"Options:\n"
-"  -h,--help             print this help message\n"
-"  -p,--pidns            create a new pid namspace (default with --pids)\n"
-"  -P,--no-pidns         do not create a new pid namespace (default)\n"
-"     --pids             restore original pids (default with --pidns)\n"
-"     --self             restart a single task, usually from self-checkpoint\n"
-"  -r,--root=ROOT        restart under the directory ROOT instead of current\n"
-"     --signal=SIG       send SIG to root task on SIGINT (default: SIGKILL\n"
-"                        to container root, SIGINT otherwise)\n"
-"     --mntns            restart under a private mounts namespace\n"
-"     --mount-pty        start in a new devpts namespace to supprt ptys\n"
-"  -w,--wait             wait for root task to termiate (default)\n"
-"     --show-status      show exit status of root task (implies -w)\n"
-"     --copy-status      imitate exit status of root task (implies -w)\n"
-"  -W,--no-wait          do not wait for root task to terminate\n"
-"  -k,--keeplsm          try to recreate original LSM labels on all objects\n"
-"  -F,--freezer=CGROUP   freeze tasks in freezer group CGROUP on success\n"
-"  -i,--input=FILE       read data from FILE instead of standard input\n"
-"     --input-fd=FD      read data from file descriptor FD (instead of stdin)\n"
-"  -l,--logfile=FILE     write error and debug data to FILE (default=none)\n"
-"     --logfile-fd=FD    write error and debug data to file desctiptor FD\n"
-"     --inspect          inspect image on-the-fly for error records\n"
-"  -v,--verbose          verbose output\n"
-"  -d,--debug            debugging output\n"
-"     --skip-COND        skip condition COND, and proceed anyway\n"
-"     --warn-COND        warn on condition COND, but proceed anyway\n"
-"     --fail-COND        warn on condition COND, and abort operation\n"
-"  	  COND=any:        any condition\n"
-"  	  COND=pidzero:    task with sid/pgid zero in a --no-pidns restart\n"
-"  	  COND=mntproc:    /proc isn't already mounted at restart (def: warn)\n"
-"";
-
 /*
  * By default, 'restart' creates a new pid namespace in which the
  * restart takes place, using the original pids from the time of the
@@ -117,18 +81,6 @@ static char *sig2str(int sig)
 	return "UNKNOWN SIGNAL";
 }
 
-static int str2sig(char *str)
-{
-	int sig = 0;
-
-	do {
-		if (!strcmp(signal_array[sig].sigstr, str))
-			return signal_array[sig].signum;
-	} while (signal_array[++sig].signum >= 0);
-
-	return -1;
-}
-
 inline static int restart(pid_t pid, int fd, unsigned long flags, int klogfd)
 {
 	return syscall(__NR_restart, pid, fd, flags, klogfd);
@@ -285,46 +237,6 @@ struct pid_swap {
 	pid_t new;
 };
 
-static void usage(char *str)
-{
-	ckpt_err("%s", str);
-	exit(1);
-}
-
-/* negative retval means error */
-static int str2num(char *str)
-{
-	char *nptr;
-	int num;
-
-	num = strtol(str, &nptr, 10);
-	if (nptr - str != strlen(str))
-		num = -1;
-	return num;
-}
-
-static long cond_to_mask(const char *cond)
-{
-	static struct {
-		char *cond;
-		long mask;
-	} conditions[] = {
-		{"pidzero", CKPT_COND_PIDZERO},
-		{"mntproc", CKPT_COND_MNTPROC},
-		{"any", CKPT_COND_ANY},
-		{NULL, 0}
-	};
-
-	int i;
-
-	for (i = 0; conditions[i].cond; i++)
-		if (!strcmp(cond, conditions[i].cond))
-			return conditions[i].mask;
-
-	ckpt_err("restart: invalid warn/fail condition '%s'\n", cond);
-	exit(1);
-}
-
 static inline int ckpt_cond_warn(struct ckpt_ctx *ctx, long mask)
 {
 	return (ctx->args->warn & mask);
@@ -335,231 +247,6 @@ static inline int ckpt_cond_fail(struct ckpt_ctx *ctx, long mask)
 	return (ctx->args->fail & mask);
 }
 
-static void parse_args(struct app_restart_args *args, int argc, char *argv[])
-{
-	static struct option opts[] = {
-		{ "help",	no_argument,		NULL, 'h' },
-		{ "pidns",	no_argument,		NULL, 'p' },
-		{ "no-pidns",	no_argument,		NULL, 'P' },
-		{ "pids",	no_argument,		NULL, 3 },
-		{ "self",	no_argument,		NULL, 6},
-		{ "signal",	required_argument,	NULL, 4 },
-		{ "inspect",	no_argument,		NULL, 5 },
-		{ "keeplsm",	no_argument,		NULL, 'k' },
-		{ "input",	required_argument,	NULL, 'i' },
-		{ "input-fd",	required_argument,	NULL, 7 },
-		{ "logfile",	required_argument,	NULL, 'l' },
-		{ "logfile-fd",	required_argument,	NULL, 8 },
-		{ "root",	required_argument,	NULL, 'r' },
-		{ "mntns",	no_argument,		NULL, 11 },
-		{ "wait",	no_argument,		NULL, 'w' },
-		{ "show-status",	no_argument,	NULL, 1 },
-		{ "copy-status",	no_argument,	NULL, 2 },
-		{ "no-wait",	no_argument,		NULL, 'W' },
-		{ "freezer",	required_argument,	NULL, 'F' },
-		{ "verbose",	no_argument,		NULL, 'v' },
-		{ "debug",	no_argument,		NULL, 'd' },
-		{ "warn-pidzero",	no_argument,	NULL, 9 },
-		{ "fail-pidzero",	no_argument,	NULL, 10 },
-		{ "mount-pty",	no_argument,		NULL, 12 },
-		{ NULL,		0,			NULL, 0 }
-	};
-	static char optc[] = "hdvkpPwWF:r:i:l:";
-
-	int optind;
-	int sig;
-	int no_pidns;
-
-	char *klogfile;
-	char *input;
-
-	/* defaults */
-	memset(args, 0, sizeof(*args));
-	args->wait = 1;
-	args->infd = -1;
-	args->klogfd = -1;
-	args->ulogfd = fileno(stdout);
-	args->uerrfd = fileno(stderr);
-	args->warn = CKPT_COND_WARN;
-	args->fail = CKPT_COND_FAIL;
-	no_pidns = 0;
-
-	klogfile = NULL;
-	input = NULL;
-
-	while (1) {
-		int c = getopt_long(argc, argv, optc, opts, &optind);
-		if (c == -1)
-			break;
-		switch (c) {
-		case '?':
-			exit(1);
-		case 'h':
-			usage(usage_str);
-		case 'v':
-			global_verbose = args->verbose = 1;
-			break;
-		case 5:  /* --inspect */
-			args->inspect = 1;
-			break;
-		case 'i':
-			input = optarg;
-			break;
-		case 7:
-			args->infd = str2num(optarg);
-			if (args->infd < 0) {
-				ckpt_err("restart: invalid file descriptor\n");
-				exit(1);
-			}
-			break;
-		case 'l':
-			klogfile = optarg;
-			break;
-		case 8:
-			args->klogfd = str2num(optarg);
-			if (args->klogfd < 0) {
-				ckpt_err("restart: invalid file descriptor\n");
-				exit(1);
-			}
-			break;
-		case 'p':
-			args->pidns = 1;
-			break;
-		case 'P':
-			no_pidns = 1;
-			break;
-		case 6:  /* --self */
-			args->self = 1;
-			break;
-		case 4:  /* --signal */
-			sig = str2sig(optarg);
-			if (sig < 0)
-				sig = str2num(optarg);
-			if (sig < 0 || sig >= NSIG) {
-				ckpt_err("restart: invalid signal\n");
-				exit(1);
-			}
-			global_send_sigint = sig;
-			break;
-		case 3:  /* --pids */
-			args->pids = 1;
-			args->pidns = 1;  /* implied */
-			break;
-		case 'r':
-			args->root = optarg;
-			break;
-		case 'w':
-			args->wait = 1;
-			break;
-		case 'W':
-			args->wait = 0;
-			break;
-		case 'k':
-			args->keep_lsm = 1;
-			break;
-		case 1:  /* --show-status */
-			args->wait = 1;
-			args->show_status = 1;
-			break;
-		case 2: /* --copy-status */
-			args->wait = 1;
-			args->copy_status = 1;
-			break;
-		case 'd':
-			global_debug = args->debug = 1;
-			break;
-		case 'F':
-			args->freezer = optarg;
-			break;
-		case 9:
-			args->warn |= cond_to_mask(&opts[optind].name[5]);
-			break;
-		case 10:
-			args->fail |= cond_to_mask(&opts[optind].name[5]);
-			break;
-		case 11:
-			args->mntns = 1;
-			break;
-		case 12:
-			args->mnt_pty = 1;
-			break;
-		default:
-			usage(usage_str);
-		}
-	}
-
-	if (no_pidns)
-		args->pidns = 0;
-
-#ifndef CLONE_NEWPID
-	if (args->pidns) {
-		ckpt_err("This version of restart was compiled without "
-		       "support for --pidns.\n");
-		exit(1);
-	}
-#endif
-
-#ifndef CHECKPOINT_DEBUG
-	if (global_debug) {
-		ckpt_err("This version of restart was compiled without "
-		       "support for --debug.\n");
-		exit(1);
-	}
-#endif
-
-	if (args->pidns)
-		args->pids = 1;
-
-#if 0   /* Defered until __NR_eclone makes it to standard headers */
-#ifndef __NR_eclone
-	if (args->pids) {
-		ckpt_err("This version of restart was compiled without "
-		       "support for --pids.\n");
-		exit(1);
-	}
-#endif
-#endif
-
-	if (args->self &&
-	    (args->pids || args->pidns || no_pidns ||
-	     args->show_status || args->copy_status || args->freezer)) {
-		ckpt_err("Invalid mix of --self with multiprocess options\n");
-		exit(1);
-	}
-
-	if (input && args->infd >= 0) {
-		ckpt_err("Invalid use of both -i/--input and --input-fd\n");
-		exit(1);
-	}
-
-	/* input file ? */
-	if (input) {
-		args->infd = open(input, O_RDONLY, 0);
-		if (args->infd < 0) {
-			ckpt_perror("open input file");
-			exit(1);
-		}
-	}
-
-	if (klogfile && args->klogfd >= 0) {
-		ckpt_err("Invalid use of both -l/--logfile and --logfile-fd\n");
-		exit(1);
-	}
-
-	/* (optional) log file */
-	if (klogfile) {
-		args->klogfd = open(klogfile, O_RDWR | O_CREAT | O_EXCL, 0644);
-		if (args->klogfd < 0) {
-			ckpt_perror("open log file");
-			exit(1);
-		}
-	}
-
-
-	if (args->mnt_pty)
-		args->mntns = 1;
-}
-
 static void report_exit_status(int status, char *str, int debug)
 {
 	char msg[64];
@@ -806,24 +493,6 @@ int app_restart(struct app_restart_args *args)
 	return ret;
 }
 
-int main(int argc, char *argv[])
-{
-	struct app_restart_args args;
-
-	/*
-	 * Initialize the log/error fds early so even parse_args() errors
-	 * are redirected here. Even if we later implement command line options
-	 * that override these, any errors/messages that occur before those
-	 * new options are parsed still go to stdout/stderr
-	 */
-	global_ulogfd = fileno(stdout);
-	global_uerrfd = fileno(stderr);
-
-	parse_args(&args, argc, argv);
-
-	return app_restart(&args);
-}
-
 static int ckpt_parse_status(int status, int mimic, int verbose)
 {
 	int sig = 0;
-- 
1.6.0.4

[PATCH 09/14][user-cr] checkpoint: Move main() to checkpoint-main.c

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Thu, 4 Mar 2010 22:28:34 -0800
Subject: [PATCH 09/14][user-cr] checkpoint: Move main() to checkpoint-main.c

---
 Makefile          |    4 +-
 checkpoint-main.c |  171 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 checkpoint.c      |  157 ------------------------------------------------
 3 files changed, 174 insertions(+), 158 deletions(-)
 create mode 100644 checkpoint-main.c

diff --git a/Makefile b/Makefile
index 7e1d701..6c9ff93 100644
--- a/Makefile
+++ b/Makefile
@@ -13,7 +13,7 @@ CKPT_HEADERS = include/linux/checkpoint.h \
 		include/linux/checkpoint_hdr.h \
 		include/asm/checkpoint_hdr.h
 
-CR_OBJS = checkpoint.o restart.o restart-main.o
+CR_OBJS = checkpoint.o checkpoint-main.o restart.o restart-main.o
 
 # detect architecture (for eclone)
 SUBARCH ?= $(patsubst i%86,x86_32,$(shell uname -m))
@@ -56,6 +56,8 @@ $(CR_OBJS): common.h app-checkpoint.h
 
 restart: restart.o restart-main.o
 
+checkpoint: checkpoint.o checkpoint-main.o
+
 # eclone() is architecture specific
 ifneq ($(SUBARCH),)
 $(ECLONE_PROGS): $(LIB_ECLONE) 
diff --git a/checkpoint-main.c b/checkpoint-main.c
new file mode 100644
index 0000000..f6faa32
--- /dev/null
+++ b/checkpoint-main.c
@@ -0,0 +1,171 @@
+#include <stdio.h>
+#include <unistd.h>
+#include <stdarg.h>
+#include <stdlib.h>
+#include <string.h>
+#include <getopt.h>
+#include <fcntl.h>
+#include <errno.h>
+
+#include <linux/checkpoint.h>
+
+#include "app-checkpoint.h"
+#include "common.h"
+
+static int global_uerrfd = -1;
+
+static char usage_str[] =
+"usage: ckpt [opts] PID\n"
+"  'checkpoint' takes a checkpoint of the task indicated by PID, and all\n"
+"  its descendents, and outputs the checkpoint image. If the task is the\n"
+"  init(1) process of a container, it checkpoints the entire container.\n"
+"  By default 'checkpoint' allows to checkpoint any subtree of tasks. The\n"
+"  user can override this feature and request that only whole containers\n"
+"  be considered.\n"
+"\n"
+"\tOptions:\n"
+"  -h,--help             print this help message\n"
+"  -o,--output=FILE      write data to FILE instead of standard output\n"
+"     --output-fd=FD     write data to file descriptor FD instead of stdout\n"
+"  -l,--logfile=FILE     write error and debug data to FILE (default=none)\n"
+"     --logile-fd=FD     write error and debug data to file descriptor FD\n"
+"  -c,--container        require the PID is a container-init\n"
+"  -v,--verbose          verbose output\n"
+"";
+
+static void usage(char *str)
+{
+	ckpt_err("%s", str);
+	exit(1);
+}
+
+/* negative retval means error */
+static int str2num(char *str)
+{
+	char *nptr;
+	int num;
+
+	num = strtol(str, &nptr, 10);
+	if (nptr - str != strlen(str))
+		num = -1;
+	return num;
+}
+
+static void parse_args(struct app_checkpoint_args *args, int argc, char *argv[])
+{
+	static struct option opts[] = {
+		{ "help",	no_argument,		NULL, 'h' },
+		{ "output",	required_argument,	NULL, 'o' },
+		{ "output-fd",	required_argument,	NULL, 1 },
+		{ "logfile",	required_argument,	NULL, 'l' },
+		{ "logfile-fd",	required_argument,	NULL, 2 },
+		{ "container",	no_argument,		NULL, 'c' },
+		{ "verbose",	no_argument,		NULL, 'v' },
+		{ NULL,		0,			NULL, 0 }
+	};
+	static char optc[] = "hvco:l:";
+	char *output;
+	char *logfile;
+
+	/* defaults */
+	args->outfd = -1;
+	args->logfd = -1;
+	args->uerrfd = fileno(stderr);
+	output = NULL;
+	logfile = NULL;
+
+	while (1) {
+		int c = getopt_long(argc, argv, optc, opts, NULL);
+		if (c == -1)
+			break;
+		switch (c) {
+		case '?':
+			exit(1);
+		case 'h':
+			usage(usage_str);
+		case 'o':
+			output = optarg;
+			break;
+		case 1:
+			args->outfd = str2num(optarg);
+			if (args->outfd < 0) {
+				ckpt_err("checkpoint: invalid file descriptor\n");
+				exit(1);
+			}
+			break;
+		case 'l':
+			logfile = optarg;
+			break;
+		case 2:
+			args->logfd = str2num(optarg);
+			if (args->logfd < 0) {
+				ckpt_err("checkpoint: invalid file descriptor\n");
+				exit(1);
+			}
+			break;
+		case 'c':
+			args->container = 1;
+			break;
+		case 'v':
+			args->verbose = 1;
+			break;
+		default:
+			usage(usage_str);
+		}
+	}
+
+	if (output && args->outfd >= 0) {
+		ckpt_err("Invalid use of both -o/--output and --output-fd\n");
+		exit(1);
+	}
+
+	/* output file */
+	if (output) {
+		args->outfd = open(output, O_RDWR | O_CREAT | O_EXCL, 0644);
+		if (args->outfd < 0) {
+			ckpt_perror("open output file");
+			exit(1);
+		}
+	}
+
+	if (logfile && args->logfd >= 0) {
+		ckpt_err("Invalid use of both -l/--logfile and --logfile-fd\n");
+		exit(1);
+	}
+
+	/* (optional) log file */
+	if (logfile) {
+		args->logfd = open(logfile, O_RDWR | O_CREAT | O_EXCL, 0644);
+		if (args->logfd < 0) {
+			ckpt_perror("open log file");
+			exit(1);
+		}
+	}
+}
+
+int main(int argc, char *argv[])
+{
+	struct app_checkpoint_args args;
+	unsigned long flags = 0;
+	pid_t pid;
+
+	global_uerrfd = fileno(stderr);
+
+	memset(&args, 0, sizeof(args));
+	parse_args(&args, argc, argv);
+
+	argc -= optind;
+	if (argc != 1)
+		usage(usage_str);
+
+	pid = atoi(argv[optind]);
+	if (pid <= 0) {
+		ckpt_err("invalid pid\n");
+		exit(1);
+	}
+
+	if (!args.container)
+		flags |= CHECKPOINT_SUBTREE;
+
+	return app_checkpoint(pid, flags, &args);
+}
diff --git a/checkpoint.c b/checkpoint.c
index 291cb36..e3a1ce8 100644
--- a/checkpoint.c
+++ b/checkpoint.c
@@ -24,25 +24,6 @@
 #include "app-checkpoint.h"
 #include "common.h"
 
-static char usage_str[] =
-"usage: ckpt [opts] PID\n"
-"  'checkpoint' takes a checkpoint of the task indicated by PID, and all\n"
-"  its descendents, and outputs the checkpoint image. If the task is the\n"
-"  init(1) process of a container, it checkpoints the entire container.\n"
-"  By default 'checkpoint' allows to checkpoint any subtree of tasks. The\n"
-"  user can override this feature and request that only whole containers\n"
-"  be considered.\n"
-"\n"
-"\tOptions:\n"
-"  -h,--help             print this help message\n"
-"  -o,--output=FILE      write data to FILE instead of standard output\n"
-"     --output-fd=FD     write data to file descriptor FD instead of stdout\n"
-"  -l,--logfile=FILE     write error and debug data to FILE (default=none)\n"
-"     --logile-fd=FD     write error and debug data to file descriptor FD\n"
-"  -c,--container        require the PID is a container-init\n"
-"  -v,--verbose          verbose output\n"
-"";
-
 static int global_uerrfd = -1;
 
 inline static int checkpoint(pid_t pid, int fd, unsigned long flags, int logfd)
@@ -50,116 +31,6 @@ inline static int checkpoint(pid_t pid, int fd, unsigned long flags, int logfd)
 	return syscall(__NR_checkpoint, pid, fd, flags, logfd);
 }
 
-static void usage(char *str)
-{
-	ckpt_err("%s", str);
-	exit(1);
-}
-
-/* negative retval means error */
-static int str2num(char *str)
-{
-	char *nptr;
-	int num;
-
-	num = strtol(str, &nptr, 10);
-	if (nptr - str != strlen(str))
-		num = -1;
-	return num;
-}
-
-static void parse_args(struct app_checkpoint_args *args, int argc, char *argv[])
-{
-	static struct option opts[] = {
-		{ "help",	no_argument,		NULL, 'h' },
-		{ "output",	required_argument,	NULL, 'o' },
-		{ "output-fd",	required_argument,	NULL, 1 },
-		{ "logfile",	required_argument,	NULL, 'l' },
-		{ "logfile-fd",	required_argument,	NULL, 2 },
-		{ "container",	no_argument,		NULL, 'c' },
-		{ "verbose",	no_argument,		NULL, 'v' },
-		{ NULL,		0,			NULL, 0 }
-	};
-	static char optc[] = "hvco:l:";
-	char *output;
-	char *logfile;
-
-	/* defaults */
-	args->outfd = -1;
-	args->logfd = -1;
-	args->uerrfd = fileno(stderr);
-	output = NULL;
-	logfile = NULL;
-
-	while (1) {
-		int c = getopt_long(argc, argv, optc, opts, NULL);
-		if (c == -1)
-			break;
-		switch (c) {
-		case '?':
-			exit(1);
-		case 'h':
-			usage(usage_str);
-		case 'o':
-			output = optarg;
-			break;
-		case 1:
-			args->outfd = str2num(optarg);
-			if (args->outfd < 0) {
-				ckpt_err("checkpoint: invalid file descriptor\n");
-				exit(1);
-			}
-			break;
-		case 'l':
-			logfile = optarg;
-			break;
-		case 2:
-			args->logfd = str2num(optarg);
-			if (args->logfd < 0) {
-				ckpt_err("checkpoint: invalid file descriptor\n");
-				exit(1);
-			}
-			break;
-		case 'c':
-			args->container = 1;
-			break;
-		case 'v':
-			args->verbose = 1;
-			break;
-		default:
-			usage(usage_str);
-		}
-	}
-
-	if (output && args->outfd >= 0) {
-		ckpt_err("Invalid use of both -o/--output and --output-fd\n");
-		exit(1);
-	}
-
-	/* output file */
-	if (output) {
-		args->outfd = open(output, O_RDWR | O_CREAT | O_EXCL, 0644);
-		if (args->outfd < 0) {
-			ckpt_perror("open output file");
-			exit(1);
-		}
-	}
-
-	if (logfile && args->logfd >= 0) {
-		ckpt_err("Invalid use of both -l/--logfile and --logfile-fd\n");
-		exit(1);
-	}
-
-	/* (optional) log file */
-	if (logfile) {
-		args->logfd = open(logfile, O_RDWR | O_CREAT | O_EXCL, 0644);
-		if (args->logfd < 0) {
-			ckpt_perror("open log file");
-			exit(1);
-		}
-	}
-}
-
 int app_checkpoint(int pid, unsigned long flags,
 				struct app_checkpoint_args *args)
 {
@@ -186,31 +57,3 @@ int app_checkpoint(int pid, unsigned long flags,
 
 	return (ret > 0 ? 0 : 1);
 }
-
-int main(int argc, char *argv[])
-{
-	struct app_checkpoint_args args;
-	unsigned long flags = 0;
-	pid_t pid;
-
-	global_uerrfd = fileno(stderr);
-
-	memset(&args, 0, sizeof(args));
-	parse_args(&args, argc, argv);
-
-	argc -= optind;
-	if (argc != 1)
-		usage(usage_str);
-
-	pid = atoi(argv[optind]);
-	if (pid <= 0) {
-		ckpt_err("invalid pid\n");
-		exit(1);
-	}
-
-	if (!args.container)
-		flags |= CHECKPOINT_SUBTREE;
-
-	return app_checkpoint(pid, flags, &args);
-}
-
-- 
1.6.0.4

[PATCH 10/14][user-cr] Have app_restart() return pid

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Fri, 5 Mar 2010 10:24:53 -0800
Subject: [PATCH 10/14][user-cr] Have app_restart() return pid

On success, return the pid of a the root of the restarted process tree.
This would enable callers to manage the restarted process-tree as a cgroup.

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 restart-main.c |    7 ++++++-
 restart.c      |    6 ++++++
 2 files changed, 12 insertions(+), 1 deletions(-)

diff --git a/restart-main.c b/restart-main.c
index d3a1782..237742e 100644
--- a/restart-main.c
+++ b/restart-main.c
@@ -332,6 +332,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 
 int main(int argc, char *argv[])
 {
+	int ret;
 	struct app_restart_args args;
 
 	/*
@@ -345,6 +346,10 @@ int main(int argc, char *argv[])
 
 	parse_args(&args, argc, argv);
 
-	return app_restart(&args);
+	ret = app_restart(&args);
+	if (ret > 0)
+		ret = 0;
+
+	return ret;
 }
 
diff --git a/restart.c b/restart.c
index 0118ad9..b6ee23c 100644
--- a/restart.c
+++ b/restart.c
@@ -490,6 +490,12 @@ int app_restart(struct app_restart_args *args)
 		ret = ckpt_coordinator(&ctx);
 	}
 
+	/*
+	 * On success, return pid of root of the restart process tree.
+	 */
+	if (ret >= 0)
+		ret = global_child_pid;
+
 	return ret;
 }
 
-- 
1.6.0.4

[PATCH 11/14][user-cr] restart: Define process_args()

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Sat, 6 Mar 2010 13:14:02 -0800
Subject: [PATCH 11/14][user-cr] restart: Define process_args()

Move the code that validates/processes the arguments into a separate
function.

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 restart.c |   27 ++++++++++++++++++++-------
 1 files changed, 20 insertions(+), 7 deletions(-)

diff --git a/restart.c b/restart.c
index b6ee23c..139f6db 100644
--- a/restart.c
+++ b/restart.c
@@ -379,14 +379,12 @@ static int freezer_register(struct ckpt_ctx *ctx, pid_t pid)
 	return ret;
 }
 
-int app_restart(struct app_restart_args *args)
+/*
+ * Validate the specified arguments and initialize globals based on the
+ * arguments. Return 0 on success.
+ */
+int process_args(struct app_restart_args *args)
 {
-	struct ckpt_ctx ctx;
-	int ret;
-
-	memset(&ctx, 0, sizeof(ctx));
-	ctx.args = args;
-
 	global_debug = args->debug;
 	global_verbose = args->verbose;
 	global_ulogfd = args->ulogfd;
@@ -406,6 +404,21 @@ int app_restart(struct app_restart_args *args)
 	if (args->klogfd < 0)
 		args->klogfd = CHECKPOINT_FD_NONE;
 
+	return 0;
+}
+
+int app_restart(struct app_restart_args *args)
+{
+	struct ckpt_ctx ctx;
+	int ret;
+
+	memset(&ctx, 0, sizeof(ctx));
+	ctx.args = args;
+
+	ret = process_args(args);
+	if (ret < 0)
+		return ret;
+
 	/* freezer preparation */
 	if (args->freezer && freezer_prepare(&ctx) < 0)
 		exit(1);
-- 
1.6.0.4

[PATCH 12/14][user-cr] app_restart(): mnt-pty implies mntns

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Sat, 6 Mar 2010 12:49:40 -0800
Subject: [PATCH 12/14][user-cr] app_restart(): mnt-pty implies mntns

If the mount_pty is set, mntns must be implied, otherwise the newinstance
mount of devpts will make the ptys in the original pts instance inaccessible.

parse_args() does this correctly, but move the code into app_restart() so
other callers of app_restart() also get this behavior.

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 restart-main.c |    4 ----
 restart.c      |    3 +++
 2 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/restart-main.c b/restart-main.c
index 237742e..36318a4 100644
--- a/restart-main.c
+++ b/restart-main.c
@@ -324,10 +324,6 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 			exit(1);
 		}
 	}
-
-
-	if (args->mnt_pty)
-		args->mntns = 1;
 }
 
 int main(int argc, char *argv[])
diff --git a/restart.c b/restart.c
index 139f6db..d7b5b72 100644
--- a/restart.c
+++ b/restart.c
@@ -404,6 +404,9 @@ int process_args(struct app_restart_args *args)
 	if (args->klogfd < 0)
 		args->klogfd = CHECKPOINT_FD_NONE;
 
+	if (args->mnt_pty)
+		args->mntns = 1;
+
 	return 0;
 }
 
-- 
1.6.0.4

[PATCH 13/14][user-cr] restart: Move args checking to app_restart()

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Sat, 6 Mar 2010 13:06:35 -0800
Subject: [PATCH 13/14][user-cr] restart: Move args checking to app_restart()

parse_args() validates the specified arguments to ensure they are consistent.
Move this code into app_restart() so that these checks apply to other callers
of app_restart() also.

Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 restart-main.c |   33 +--------------------------------
 restart.c      |   36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+), 32 deletions(-)

diff --git a/restart-main.c b/restart-main.c
index 36318a4..681ff8e 100644
--- a/restart-main.c
+++ b/restart-main.c
@@ -261,38 +261,7 @@ static void parse_args(struct app_restart_args *args, int argc, char *argv[])
 	if (no_pidns)
 		args->pidns = 0;
 
-#ifndef CLONE_NEWPID
-	if (args->pidns) {
-		ckpt_err("This version of restart was compiled without "
-		       "support for --pidns.\n");
-		exit(1);
-	}
-#endif
-
-#ifndef CHECKPOINT_DEBUG
-	if (global_debug) {
-		ckpt_err("This version of restart was compiled without "
-		       "support for --debug.\n");
-		exit(1);
-	}
-#endif
-
-	if (args->pidns)
-		args->pids = 1;
-
-#if 0   /* Defered until __NR_eclone makes it to standard headers */
-#ifndef __NR_eclone
-	if (args->pids) {
-		ckpt_err("This version of restart was compiled without "
-		       "support for --pids.\n");
-		exit(1);
-	}
-#endif
-#endif
-
-	if (args->self &&
-	    (args->pids || args->pidns || no_pidns ||
-	     args->show_status || args->copy_status || args->freezer)) {
+	if (args->self && no_pidns) {
 		ckpt_err("Invalid mix of --self with multiprocess options\n");
 		exit(1);
 	}
diff --git a/restart.c b/restart.c
index d7b5b72..b2708c3 100644
--- a/restart.c
+++ b/restart.c
@@ -407,6 +407,42 @@ int process_args(struct app_restart_args *args)
 	if (args->mnt_pty)
 		args->mntns = 1;
 
+#ifndef CLONE_NEWPID
+	if (args->pidns) {
+		ckpt_err("This version of restart was compiled without "
+		       "support for --pidns.\n");
+		exit(1);
+	}
+#endif
+
+#ifndef CHECKPOINT_DEBUG
+	if (global_debug) {
+		ckpt_err("This version of restart was compiled without "
+		       "support for --debug.\n");
+		exit(1);
+	}
+#endif
+
+	if (args->pidns)
+		args->pids = 1;
+
+#if 0   /* Defered until __NR_eclone makes it to standard headers */
+#ifndef __NR_eclone
+	if (args->pids) {
+		ckpt_err("This version of restart was compiled without "
+		       "support for --pids.\n");
+		exit(1);
+	}
+#endif
+#endif
+
+	if (args->self &&
+	    (args->pids || args->pidns || args->show_status ||
+	     args->copy_status || args->freezer)) {
+		ckpt_err("Invalid mix of --self with multiprocess options\n");
+		exit(1);
+	}
+
 	return 0;
 }
 
-- 
1.6.0.4

[PATCH 14/14][user-cr] Minimize unshare() calls

[Sukadev Bhattiprolu]

From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
Date: Mon, 8 Mar 2010 12:03:46 -0800
Subject: [PATCH 14/14][user-cr] Minimize unshare() calls

We currently have a few unshare() calls at different points in the
code. While these don't affect the restart application itself, the
excess calls create additional levels in the cgroup hierarchy, which
can surprise the administrator (or other users of the hierarchy
such as LXC.

Rather than several unshare() calls, can we instead specify the
appropriate clone_flags while creating the coordinator/root process
of the application tree ? When this root process is created it can
remount /proc, remount devpts, chroot() etc if necessary.

Note that for "new-container with init" and "subtree restart", the
first process is also the root of the application process tree.

In the case of "new-container without init", the coordinator process
which acts as the container-init can do the setup.

In case of self-restart, the main process itself can do the unshare.

This patch has been very gently tested :-) but wanted to get more feedback
on the direction and see if there is an easier way.

Changelog[v2]:
	- Remove rather than '#ifdef 0' some unshare() calls
---
 restart.c |   80 +++++++++++++++++++++++++++++++++++++++++++++---------------
 1 files changed, 60 insertions(+), 20 deletions(-)

diff --git a/restart.c b/restart.c
index b2708c3..484668a 100644
--- a/restart.c
+++ b/restart.c
@@ -127,6 +127,9 @@ struct task zero_task;
 #define TASK_SESSION	0x10	/* inherits creator's original sid */
 #define TASK_NEWPID	0x20	/* starts a new pid namespace */
 #define TASK_DEAD	0x40	/* dead task (dummy) */
+#define TASK_NEWROOT	0x80	/* task must chroot() */
+#define TASK_NEWPTS	0x100	/* remount devpts */
+#define TASK_NEWNS	0x200	/* unshare namespace/file-system */
 
 struct ckpt_ctx {
 	pid_t root_pid;
@@ -462,24 +465,24 @@ int app_restart(struct app_restart_args *args)
 	if (args->freezer && freezer_prepare(&ctx) < 0)
 		exit(1);
 
-	/* private mounts namespace ? */
-	if (args->mntns && unshare(CLONE_NEWNS | CLONE_FS) < 0) {
-		ckpt_perror("unshare");
-		exit(1);
-	}
+	/* self-restart ends here: */
+	if (args->self) {
+		/* private mounts namespace ? */
+		if (args->mntns && unshare(CLONE_NEWNS | CLONE_FS) < 0) {
+			ckpt_perror("unshare");
+			exit(1);
+		}
 
-	/* chroot ? */
-	if (args->root && chroot(args->root) < 0) {
-		ckpt_perror("chroot");
-		exit(1);
-	}
+		/* chroot ? */
+		if (args->root && chroot(args->root) < 0) {
+			ckpt_perror("chroot");
+			exit(1);
+		}
 
-	/* remount /dev/pts ? */
-	if (args->mnt_pty && ckpt_remount_devpts(&ctx) < 0)
-		exit(1);
+		/* remount /dev/pts ? */
+		if (args->mnt_pty && ckpt_remount_devpts(&ctx) < 0)
+			exit(1);
 
-	/* self-restart ends here: */
-	if (args->self) {
 		restart(getpid(), STDIN_FILENO, RESTART_TASKSELF, args->klogfd);
 		/* reach here if restart(2) failed ! */
 		ckpt_perror("restart");
@@ -524,10 +527,30 @@ int app_restart(struct app_restart_args *args)
 	if (ret < 0)
 		exit(1);
 
+	/*
+	 * Have the first child in the restarted process tree
+	 * setup devpts, root-dir and /proc if necessary, ...
+	 */
+	if (ctx.args->mnt_pty)
+		ctx.tasks_arr[0].flags |= TASK_NEWPTS;
+
+	if (ctx.args->mntns)
+		ctx.tasks_arr[0].flags |= TASK_NEWNS;
+
+	if (ctx.args->root)
+		ctx.tasks_arr[0].flags |= TASK_NEWROOT;
+
 	if (ctx.args->pidns && ctx.tasks_arr[0].pid != 1) {
 		ckpt_dbg("new pidns without init\n");
 		if (global_send_sigint == -1)
 			global_send_sigint = SIGINT;
+		/*
+		 * ...unless we have an explicit coordinator, in which case
+		 * the coordinator should set up the filesystems and
+		 * not the first process in the application process tree.
+		 */
+		ctx.tasks_arr[0].flags &= ~(TASK_NEWPTS | TASK_NEWROOT | \
+				TASK_NEWNS);
 		ret = ckpt_coordinator_pidns(&ctx);
 	} else if (ctx.args->pidns) {
 		ckpt_dbg("new pidns with init\n");
@@ -721,10 +744,6 @@ static int ckpt_probe_child(pid_t pid, char *str)
  */
 static int ckpt_remount_proc(struct ckpt_ctx *ctx)
 {
-	if (unshare(CLONE_NEWNS | CLONE_FS) < 0) {
-		ckpt_perror("unshare");
-		return -1;
-	}
 	/* this is unlikely, but we don't want to fail */
 	if (umount2("/proc", MNT_DETACH) < 0) {
 		if (ckpt_cond_fail(ctx, CKPT_COND_MNTPROC)) {
@@ -747,9 +766,18 @@ static int __ckpt_coordinator(void *arg)
 {
 	struct ckpt_ctx *ctx = (struct ckpt_ctx *) arg;
 
+	/* chroot ? */
+	if (ctx->args->root && chroot(ctx->args->root) < 0) {
+		ckpt_perror("chroot");
+		exit(1);
+	}
+
 	if (ckpt_remount_proc(ctx) < 0)
 		return -1;
 
+	if (ctx->args->mnt_pty && ckpt_remount_devpts(ctx) < 0)
+		return -1;
+
 	if (!ctx->args->wait)
 		close(ctx->pipe_coord[0]);
 
@@ -782,6 +810,7 @@ static int ckpt_coordinator_pidns(struct ckpt_ctx *ctx)
 	int copy, ret;
 	genstack stk;
 	void *sp;
+	unsigned long flags = SIGCHLD;
 
 	ckpt_dbg("forking coordinator in new pidns\n");
 
@@ -806,7 +835,9 @@ static int ckpt_coordinator_pidns(struct ckpt_ctx *ctx)
 	copy = ctx->args->copy_status;
 	ctx->args->copy_status = 1;
 
-	coord_pid = clone(__ckpt_coordinator, sp, CLONE_NEWPID|SIGCHLD, ctx);
+	flags |= CLONE_NEWPID|CLONE_NEWNS;
+
+	coord_pid = clone(__ckpt_coordinator, sp, flags, ctx);
 	genstack_release(stk);
 	if (coord_pid < 0) {
 		ckpt_perror("clone coordinator");
@@ -1614,10 +1645,16 @@ int ckpt_fork_stub(void *data)
 	struct task *task = (struct task *) data;
 	struct ckpt_ctx *ctx = task->ctx;
 
+	if ((task->flags & TASK_NEWROOT) && chroot(ctx->args->root) < 0)
+		return -1;
+
 	/* tasks with new pid-ns need new /proc mount */
 	if ((task->flags & TASK_NEWPID) && ckpt_remount_proc(ctx) < 0)
 		return -1;
 
+	if ((task->flags & TASK_NEWPTS) && ckpt_remount_devpts(ctx) < 0)
+		return -1;
+
 	/*
 	 * In restart into a new pid namespace (--pidns), coordinator
 	 * is the container init, hence if it terminated permatutely
@@ -1695,6 +1732,9 @@ static pid_t ckpt_fork_child(struct ckpt_ctx *ctx, struct task *child)
 		pid = 0;
 	}
 #endif
+	if (child->flags & TASK_NEWNS) {
+		flags |= CLONE_NEWNS;
+	}
 
 	if (child->flags & (TASK_SIBLING | TASK_THREAD))
 		child->real_parent = getppid();
-- 
1.6.0.4

Re: [PATCH 14/14][user-cr] Minimize unshare() calls

[Serge E. Hallyn]

Quoting Sukadev Bhattiprolu (sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org):
> 
> From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
> Date: Mon, 8 Mar 2010 12:03:46 -0800
> Subject: [PATCH 14/14][user-cr] Minimize unshare() calls
> 
> We currently have a few unshare() calls at different points in the
> code. While these don't affect the restart application itself, the
> excess calls create additional levels in the cgroup hierarchy, which
> can surprise the administrator (or other users of the hierarchy
> such as LXC.
> 
> Rather than several unshare() calls, can we instead specify the
> appropriate clone_flags while creating the coordinator/root process
> of the application tree ? When this root process is created it can
> remount /proc, remount devpts, chroot() etc if necessary.
> 
> Note that for "new-container with init" and "subtree restart", the
> first process is also the root of the application process tree.
> 
> In the case of "new-container without init", the coordinator process
> which acts as the container-init can do the setup.
> 
> In case of self-restart, the main process itself can do the unshare.
> 
> This patch has been very gently tested :-) but wanted to get more feedback
> on the direction and see if there is an easier way.

No objection from me on the direction or the code.

Acked-by: Serge Hallyn <serue-r/Jw6+rmf7HQT0dZR+AlfA@public.gmane.org>

Re: [PATCH 14/14][user-cr] Minimize unshare() calls

[Serge E. Hallyn]

Quoting Sukadev Bhattiprolu (sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org):
> 
> From: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
> Date: Mon, 8 Mar 2010 12:03:46 -0800
> Subject: [PATCH 14/14][user-cr] Minimize unshare() calls

Hmm, and yet, when I run cr_tests/simple.sh, restart apparently
does not use CLONE_NEWNS, bc I then have to
	umount /proc
	mount -t proc proc /proc
to restore sanity.

Not sure why...  restart -vd gives me

Using output dir ./cr_simple_N3wEpal
about to clone with 20020000
<3793>number of tasks: 1
<3793>total tasks (including ghosts): 1
<3793>====== TASKS
<3793>  [0] pid 1 ppid 0 sid 0 creator 0       
<3793>............
<3793>new pidns with init
<3793>forking child vpid 1 flags 0x21
<3793>forked child vpid 3795 (asked 1)
<1>root task pid 1
<1>pid 1: pid 1 sid 0 parent 0
<1>about to call sys_restart(), flags 0
<3794>====== PIDS ARRAY
<3794>[0] pid 1 ppid 0 sid 0 pgid 0
<3794>............
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 16384
<3794>c/r read input 11161
<3794>c/r read input 0
Success
<3793>restart succeeded
<3793>SIGCHLD: already collected
<3793>task exited with status 0
<3793>c/r succeeded
PASS

Oh, there you go - I suppose restart.c:557 should have TASK_NEWNS added?
Indeed that fixes that.

-serge

Re: [PATCH 0/14][user-cr] Enable linking with LIBLXC

[Oren Laadan]

Suka,

Thanks for the patches - applied on ckpt-v20-dev.

I think it's time to look for a naming convention for the
library and API ...

Oren

Sukadev Bhattiprolu wrote:
> Patches to enable linking LIBLXC with USERCR.
> 
> Patches 1 through 9 are simply rebased patches due to the dropping of one
> of the earlier patches (Oren had nacked one patch in my earlier set. See
> 
> https://lists.linux-foundation.org/pipermail/containers/2010-March/023564.html
> 
> Patches 10 through 14 are additional changes needed to get LXC checkpoint
> and restart to work.
> 
> 	[PATCH 01/14] Add app_restart_args->debug
> 	[PATCH 02/14] Add app_restart_args->verbose
> 	[PATCH 03/14] Add app_restart_args->ulogfd
> 	[PATCH 04/14] Add app_restart_args->uerrfd
> 	[PATCH 05/14] Define INIT_SIGNAL_ARRAY
> 	[PATCH 06/14] Create common.h
> 	[PATCH 07/14] Create app-checkpoint.h
> 	[PATCH 08/14] restart: Move main() to restart-main.c
> 	[PATCH 09/14] checkpoint: Move main() to checkpoint-main.c
> 
> 	[PATCH 10/14] Have app_restart() return pid
> 	[PATCH 11/14] restart: Define process_args()
> 	[PATCH 12/14] app_restart(): mnt-pty implies mntns
> 	[PATCH 13/14] restart: Move args checking to app_restart()
> 	[PATCH 14/14] Minimize unshare() calls
> 
> With these patches, USERCR exports app_checkpoint() and app_restart() 
> functions in checkpoint.o and restart.o. LXC links directly with these
> .o files to checkpoint/restart its containers.
> 
> Signed-off-by: Sukadev Bhattiprolu <sukadev-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
>