[PATCH 0/8] Change default MSGMNI tunable to scale with lowmem (v3)

[PATCH 0/8] Change default MSGMNI tunable to scale with lowmem (v3)

[Nadia.Derbey]

Resending the set of patches after Yasunori's remark about being able to
turn on/off automatic recomputing.
(see message at http://lkml.org/lkml/2008/2/5/149).
I actually introduced an intermediate solution: when msgmni is set by hand,
it is uneregistered from the ipcns notifier chain (i.e. automatic recomputing
is disabled). This corresponds to an implicit turn off. Setting it to a
negative value makes it registered back in the notifier chain (which
corresponds to the turn on proposed by Yasunaori).

Only patch # 8 introduces new stuff compared to the already sent patch sets.

Also ported the patchset to 2.6.24-mm1.

-------------------

On large systems we'd like to allow a larger number of message queues.
In some cases up to 32K. However simply setting MSGMNI to a larger value may
cause problems for smaller systems.

The first patch of this series introduces a default maximum number of message
queue ids that scales with the amount of lowmem.

Since msgmni is per namespace and there is no amount of memory dedicated to
each namespace so far, the second patch of this series scales msgmni to
the number of ipc namespaces too.

Since msgmni depends on the amount of memory, it becomes necessary to
recompute it upon memory add/remove.
In the 4th patch, memory hotplug management is added: a notifier block is
registered into the memory hotplug notifier chain for the ipc subsystem.
Since the ipc namespaces are not linked together, they have their own
notification chain: one notifier_block is defined per ipc namespace.
Each time an ipc namespace is created (removed) it registers (unregisters)
its notifier block in (from) the ipcns chain.
The callback routine registered in the memory chain invokes the ipcns
notifier chain with the IPCNS_MEMCHANGE event.
Each callback routine registered in the ipcns namespace, in turn, recomputes
msgmni for the owning namespace.

The 5th patch makes it possible to keep the memory hotplug notifier chain's
lock for a lesser amount of time: instead of directly notifying the ipcns
notifier chain upon memory add/remove, a work item is added to the global
workqueue. When activated, this work item is the one who notifies the ipcns
notifier chain.

Since msgmni depends on the number of ipc namespaces, it becomes necessary
to recompute it upon ipc namespace creation / removal.
The 6th patch uses the ipc namespace notifier chain for that purpose: that
chain is notified each time an ipc namespace is created or removed. This makes
it possible to recompute msgmni for all the namespaces each time one of them
is created or removed.

When msgmni is explicitely set from userspace, we should avoid recomputing it
upon memory add/remove or ipcns creation/removal.
This is what the 7th patch does: it simply unregisters the ipcns callback
routine as soon as msgmni has been changed from procfs or sysctl().

Even if msgmni is set by hand, it should be possible to make it back
automatically recomputed upon memory add/remove or ipcns creation/removal.
This what is achieved in patch 8: if set to a negative value, msgmni is added
back to the ipcns notifier chain, making it automatically recomputed again.


These patches should be applied to 2.6.24-mm1, in the following order:

[PATCH 1/8]: ipc_scale_msgmni_with_lowmem.patch
[PATCH 2/8]: ipc_scale_msgmni_with_namespaces.patch
[PATCH 3/8]: ipc_slab_memory_callback_prio_to_const.patch
[PATCH 4/8]: ipc_recompute_msgmni_on_memory_hotplug.patch
[PATCH 5/8]: ipc_ipcns_notification_workqueue.patch
[PATCH 6/8]: ipc_recompute_msgmni_on_ipcns_create_remove.patch
[PATCH 7/8]: ipc_unregister_callback_on_msgmni_manual_change.patch
[PATCH 8/8]: ipc_register_callback_on_negative_msgmni.patch


Andrew, do you think this patchset could be considered for inclusion in -mm
(or at least patches 1 to 6)?


Regards,
Nadia

--

[PATCH 1/8] Scaling msgmni to the amount of lowmem

[Nadia.Derbey]

[-- Attachment #1: ipc_scale_msgmni_with_lowmem.patch --]
[-- Type: text/plain, Size: 3661 bytes --]

[PATCH 01/08]

This patch computes msg_ctlmni to make it scale with the amount of lowmem.
msg_ctlmni is now set to make the message queues occupy 1/32 of the available
lowmem.

Some cleaning has also been done for the MSGPOOL constant: the msgctl man page
says it's not used, but it also defines it as a size in bytes (the code
expresses it in Kbytes).

Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 include/linux/msg.h |   14 ++++++++++++--
 ipc/msg.c           |   37 ++++++++++++++++++++++++++++++++++++-
 2 files changed, 48 insertions(+), 3 deletions(-)

Index: linux-2.6.24-mm1/include/linux/msg.h
===================================================================
--- linux-2.6.24-mm1.orig/include/linux/msg.h	2008-02-07 15:01:38.000000000 +0100
+++ linux-2.6.24-mm1/include/linux/msg.h	2008-02-07 15:23:17.000000000 +0100
@@ -49,16 +49,26 @@ struct msginfo {
 	unsigned short  msgseg; 
 };
 
+/*
+ * Scaling factor to compute msgmni:
+ * the memory dedicated to msg queues (msgmni * msgmnb) should occupy
+ * at most 1/MSG_MEM_SCALE of the lowmem (see the formula in ipc/msg.c):
+ * up to 8MB       : msgmni = 16 (MSGMNI)
+ * 4 GB            : msgmni = 8K
+ * more than 16 GB : msgmni = 32K (IPCMNI)
+ */
+#define MSG_MEM_SCALE 32
+
 #define MSGMNI    16   /* <= IPCMNI */     /* max # of msg queue identifiers */
 #define MSGMAX  8192   /* <= INT_MAX */   /* max size of message (bytes) */
 #define MSGMNB 16384   /* <= INT_MAX */   /* default max size of a message queue */
 
 /* unused */
-#define MSGPOOL (MSGMNI*MSGMNB/1024)  /* size in kilobytes of message pool */
+#define MSGPOOL (MSGMNI * MSGMNB) /* size in bytes of message pool */
 #define MSGTQL  MSGMNB            /* number of system message headers */
 #define MSGMAP  MSGMNB            /* number of entries in message map */
 #define MSGSSZ  16                /* message segment size */
-#define __MSGSEG ((MSGPOOL*1024)/ MSGSSZ) /* max no. of segments */
+#define __MSGSEG (MSGPOOL / MSGSSZ) /* max no. of segments */
 #define MSGSEG (__MSGSEG <= 0xffff ? __MSGSEG : 0xffff)
 
 #ifdef __KERNEL__
Index: linux-2.6.24-mm1/ipc/msg.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/msg.c	2008-02-07 15:02:29.000000000 +0100
+++ linux-2.6.24-mm1/ipc/msg.c	2008-02-07 15:24:19.000000000 +0100
@@ -27,6 +27,7 @@
 #include <linux/msg.h>
 #include <linux/spinlock.h>
 #include <linux/init.h>
+#include <linux/mm.h>
 #include <linux/proc_fs.h>
 #include <linux/list.h>
 #include <linux/security.h>
@@ -78,11 +79,45 @@ static int newque(struct ipc_namespace *
 static int sysvipc_msg_proc_show(struct seq_file *s, void *it);
 #endif
 
+/*
+ * Scale msgmni with the available lowmem size: the memory dedicated to msg
+ * queues should occupy at most 1/MSG_MEM_SCALE of lowmem.
+ * This should be done staying within the (MSGMNI , IPCMNI) range.
+ */
+static void recompute_msgmni(struct ipc_namespace *ns)
+{
+	struct sysinfo i;
+	unsigned long allowed;
+
+	si_meminfo(&i);
+	allowed = (((i.totalram - i.totalhigh) / MSG_MEM_SCALE) * i.mem_unit)
+		/ MSGMNB;
+
+	if (allowed < MSGMNI) {
+		ns->msg_ctlmni = MSGMNI;
+		goto out_callback;
+	}
+
+	if (allowed > IPCMNI) {
+		ns->msg_ctlmni = IPCMNI;
+		goto out_callback;
+	}
+
+	ns->msg_ctlmni = allowed;
+
+out_callback:
+
+	printk(KERN_INFO "msgmni has been set to %d for ipc namespace %p\n",
+		ns->msg_ctlmni, ns);
+}
+
 void msg_init_ns(struct ipc_namespace *ns)
 {
 	ns->msg_ctlmax = MSGMAX;
 	ns->msg_ctlmnb = MSGMNB;
-	ns->msg_ctlmni = MSGMNI;
+
+	recompute_msgmni(ns);
+
 	atomic_set(&ns->msg_bytes, 0);
 	atomic_set(&ns->msg_hdrs, 0);
 	ipc_init_ids(&ns->ids[IPC_MSG_IDS]);

--

[PATCH 2/8] Scaling msgmni to the number of ipc namespaces

[Nadia.Derbey]

[-- Attachment #1: ipc_scale_msgmni_with_namespaces.patch --]
[-- Type: text/plain, Size: 3131 bytes --]

[PATCH 02/08]

Since all the namespaces see the same amount of memory (the total one)
this patch introduces a new variable that counts the ipc namespaces and
divides msg_ctlmni by this counter.

Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 include/linux/ipc_namespace.h |    1 +
 ipc/msg.c                     |   10 +++++++---
 ipc/namespace.c               |    3 +++
 ipc/util.c                    |    3 +++
 4 files changed, 14 insertions(+), 3 deletions(-)

Index: linux-2.6.24-mm1/include/linux/ipc_namespace.h
===================================================================
--- linux-2.6.24-mm1.orig/include/linux/ipc_namespace.h	2008-02-07 15:00:43.000000000 +0100
+++ linux-2.6.24-mm1/include/linux/ipc_namespace.h	2008-02-07 15:26:53.000000000 +0100
@@ -33,6 +33,7 @@ struct ipc_namespace {
 };
 
 extern struct ipc_namespace init_ipc_ns;
+extern atomic_t nr_ipc_ns;
 
 #ifdef CONFIG_SYSVIPC
 #define INIT_IPC_NS(ns)		.ns		= &init_ipc_ns,
Index: linux-2.6.24-mm1/ipc/util.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/util.c	2008-02-07 13:41:07.000000000 +0100
+++ linux-2.6.24-mm1/ipc/util.c	2008-02-07 15:36:22.000000000 +0100
@@ -52,6 +52,9 @@ struct ipc_namespace init_ipc_ns = {
 	},
 };
 
+atomic_t nr_ipc_ns = ATOMIC_INIT(1);
+
+
 /**
  *	ipc_init	-	initialise IPC subsystem
  *
Index: linux-2.6.24-mm1/ipc/namespace.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/namespace.c	2008-02-07 13:41:07.000000000 +0100
+++ linux-2.6.24-mm1/ipc/namespace.c	2008-02-07 15:40:19.000000000 +0100
@@ -20,6 +20,8 @@ static struct ipc_namespace *clone_ipc_n
 	if (ns == NULL)
 		return ERR_PTR(-ENOMEM);
 
+	atomic_inc(&nr_ipc_ns);
+
 	sem_init_ns(ns);
 	msg_init_ns(ns);
 	shm_init_ns(ns);
@@ -83,4 +85,5 @@ void free_ipc_ns(struct kref *kref)
 	msg_exit_ns(ns);
 	shm_exit_ns(ns);
 	kfree(ns);
+	atomic_dec(&nr_ipc_ns);
 }
Index: linux-2.6.24-mm1/ipc/msg.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/msg.c	2008-02-07 15:24:19.000000000 +0100
+++ linux-2.6.24-mm1/ipc/msg.c	2008-02-07 15:43:51.000000000 +0100
@@ -82,24 +82,28 @@ static int sysvipc_msg_proc_show(struct 
 /*
  * Scale msgmni with the available lowmem size: the memory dedicated to msg
  * queues should occupy at most 1/MSG_MEM_SCALE of lowmem.
- * This should be done staying within the (MSGMNI , IPCMNI) range.
+ * Also take into account the number of nsproxies created so far.
+ * This should be done staying within the (MSGMNI , IPCMNI/nr_ipc_ns) range.
  */
 static void recompute_msgmni(struct ipc_namespace *ns)
 {
 	struct sysinfo i;
 	unsigned long allowed;
+	int nb_ns;
 
 	si_meminfo(&i);
 	allowed = (((i.totalram - i.totalhigh) / MSG_MEM_SCALE) * i.mem_unit)
 		/ MSGMNB;
+	nb_ns = atomic_read(&nr_ipc_ns);
+	allowed /= nb_ns;
 
 	if (allowed < MSGMNI) {
 		ns->msg_ctlmni = MSGMNI;
 		goto out_callback;
 	}
 
-	if (allowed > IPCMNI) {
-		ns->msg_ctlmni = IPCMNI;
+	if (allowed > IPCMNI / nb_ns) {
+		ns->msg_ctlmni = IPCMNI / nb_ns;
 		goto out_callback;
 	}
 

--

[PATCH 3/8] Defining the slab_memory_callback priority as a constant

[Nadia.Derbey]

[-- Attachment #1: ipc_slab_memory_callback_prio_to_const.patch --]
[-- Type: text/plain, Size: 1492 bytes --]

[PATCH 03/08]

This is a trivial patch that defines the priority of slab_memory_callback in
the callback chain as a constant.
This is to prepare for next patch in the series.

Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 include/linux/memory.h |    6 ++++++
 mm/slub.c              |    2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)

Index: linux-2.6.24-mm1/include/linux/memory.h
===================================================================
--- linux-2.6.24-mm1.orig/include/linux/memory.h	2008-02-07 13:40:35.000000000 +0100
+++ linux-2.6.24-mm1/include/linux/memory.h	2008-02-07 17:10:07.000000000 +0100
@@ -53,6 +53,12 @@ struct memory_notify {
 struct notifier_block;
 struct mem_section;
 
+/*
+ * Priorities for the hotplug memory callback routines (stored in decreasing
+ * order in the callback chain)
+ */
+#define SLAB_CALLBACK_PRI       1
+
 #ifndef CONFIG_MEMORY_HOTPLUG_SPARSE
 static inline int memory_dev_init(void)
 {
Index: linux-2.6.24-mm1/mm/slub.c
===================================================================
--- linux-2.6.24-mm1.orig/mm/slub.c	2008-02-07 13:40:46.000000000 +0100
+++ linux-2.6.24-mm1/mm/slub.c	2008-02-07 17:16:09.000000000 +0100
@@ -3001,7 +3001,7 @@ void __init kmem_cache_init(void)
 	kmalloc_caches[0].refcount = -1;
 	caches++;
 
-	hotplug_memory_notifier(slab_memory_callback, 1);
+	hotplug_memory_notifier(slab_memory_callback, SLAB_CALLBACK_PRI);
 #endif
 
 	/* Able to allocate the per node structures */

--

[PATCH 4/8] Recomputing msgmni on memory add / remove

[Nadia.Derbey]

[-- Attachment #1: ipc_recompute_msgmni_on_memory_hotplug.patch --]
[-- Type: text/plain, Size: 10306 bytes --]

[PATCH 04/08]

This patch introduces the registration of a callback routine that recomputes
msg_ctlmni upon memory add / remove.

A single notifier block is registered in the hotplug memory chain for all the
ipc namespaces.

Since the ipc namespaces are not linked together, they have their own
notification chain: one notifier_block is defined per ipc namespace.

Each time an ipc namespace is created (removed) it registers (unregisters)
its notifier block in (from) the ipcns chain.
The callback routine registered in the memory chain invokes the ipcns
notifier chain with the IPCNS_LOWMEM event.
Each callback routine registered in the ipcns namespace, in turn, recomputes
msgmni for the owning namespace.


Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 include/linux/ipc_namespace.h |   43 ++++++++++++++++++++++++-
 include/linux/memory.h        |    1 
 ipc/Makefile                  |    3 +
 ipc/ipcns_notifier.c          |   71 ++++++++++++++++++++++++++++++++++++++++++
 ipc/msg.c                     |    2 -
 ipc/namespace.c               |   11 ++++++
 ipc/util.c                    |   33 +++++++++++++++++++
 ipc/util.h                    |    2 +
 8 files changed, 162 insertions(+), 4 deletions(-)

Index: linux-2.6.24-mm1/include/linux/ipc_namespace.h
===================================================================
--- linux-2.6.24-mm1.orig/include/linux/ipc_namespace.h	2008-02-07 15:26:53.000000000 +0100
+++ linux-2.6.24-mm1/include/linux/ipc_namespace.h	2008-02-08 08:29:21.000000000 +0100
@@ -4,6 +4,17 @@
 #include <linux/err.h>
 #include <linux/idr.h>
 #include <linux/rwsem.h>
+#ifdef CONFIG_MEMORY_HOTPLUG
+#include <linux/notifier.h>
+#endif /* CONFIG_MEMORY_HOTPLUG */
+
+/*
+ * ipc namespace events
+ */
+#define IPCNS_MEMCHANGED   0x00000001   /* Notify lowmem size changed */
+
+#define IPCNS_CALLBACK_PRI 0
+
 
 struct ipc_ids {
 	int in_use;
@@ -30,6 +41,10 @@ struct ipc_namespace {
 	size_t		shm_ctlall;
 	int		shm_ctlmni;
 	int		shm_tot;
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+	struct notifier_block ipcns_nb;
+#endif
 };
 
 extern struct ipc_namespace init_ipc_ns;
@@ -37,9 +52,33 @@ extern atomic_t nr_ipc_ns;
 
 #ifdef CONFIG_SYSVIPC
 #define INIT_IPC_NS(ns)		.ns		= &init_ipc_ns,
-#else
+
+#ifdef CONFIG_MEMORY_HOTPLUG
+
+extern int register_ipcns_notifier(struct ipc_namespace *);
+extern int unregister_ipcns_notifier(struct ipc_namespace *);
+extern int ipcns_notify(unsigned long);
+
+#else /* CONFIG_MEMORY_HOTPLUG */
+
+static inline int register_ipcns_notifier(struct ipc_namespace *ipcns)
+{
+	return 0;
+}
+static inline int unregister_ipcns_notifier(struct ipc_namespace *ipcns)
+{
+	return 0;
+}
+static inline int ipcns_notify(unsigned long ev)
+{
+	return 0;
+}
+
+#endif /* CONFIG_MEMORY_HOTPLUG */
+
+#else /* CONFIG_SYSVIPC */
 #define INIT_IPC_NS(ns)
-#endif
+#endif /* CONFIG_SYSVIPC */
 
 #if defined(CONFIG_SYSVIPC) && defined(CONFIG_IPC_NS)
 extern void free_ipc_ns(struct kref *kref);
Index: linux-2.6.24-mm1/include/linux/memory.h
===================================================================
--- linux-2.6.24-mm1.orig/include/linux/memory.h	2008-02-07 17:10:07.000000000 +0100
+++ linux-2.6.24-mm1/include/linux/memory.h	2008-02-08 08:04:36.000000000 +0100
@@ -58,6 +58,7 @@ struct mem_section;
  * order in the callback chain)
  */
 #define SLAB_CALLBACK_PRI       1
+#define IPC_CALLBACK_PRI        10
 
 #ifndef CONFIG_MEMORY_HOTPLUG_SPARSE
 static inline int memory_dev_init(void)
Index: linux-2.6.24-mm1/ipc/ipcns_notifier.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ linux-2.6.24-mm1/ipc/ipcns_notifier.c	2008-02-08 13:59:26.000000000 +0100
@@ -0,0 +1,71 @@
+/*
+ * linux/ipc/ipcns_notifier.c
+ * Copyright (C) 2007 BULL SA. Nadia Derbey
+ *
+ * Notification mechanism for ipc namespaces:
+ * The callback routine registered in the memory chain invokes the ipcns
+ * notifier chain with the IPCNS_MEMCHANGED event.
+ * Each callback routine registered in the ipcns namespace recomputes msgmni
+ * for the owning namespace.
+ */
+
+#include <linux/msg.h>
+#include <linux/rcupdate.h>
+#include <linux/notifier.h>
+#include <linux/nsproxy.h>
+#include <linux/ipc_namespace.h>
+
+#include "util.h"
+
+
+
+static BLOCKING_NOTIFIER_HEAD(ipcns_chain);
+
+
+static int ipcns_callback(struct notifier_block *self,
+				unsigned long action, void *arg)
+{
+	struct ipc_namespace *ns;
+
+	switch (action) {
+	case IPCNS_MEMCHANGED:   /* amount of lowmem has changed */
+		/*
+		 * It's time to recompute msgmni
+		 */
+		ns = container_of(self, struct ipc_namespace, ipcns_nb);
+		/*
+		 * No need to get a reference on the ns: the 1st job of
+		 * free_ipc_ns() is to unregister the callback routine.
+		 * blocking_notifier_chain_unregister takes the wr lock to do
+		 * it.
+		 * When this callback routine is called the rd lock is held by
+		 * blocking_notifier_call_chain.
+		 * So the ipc ns cannot be freed while we are here.
+		 */
+		recompute_msgmni(ns);
+		break;
+	default:
+		break;
+	}
+
+	return NOTIFY_OK;
+}
+
+int register_ipcns_notifier(struct ipc_namespace *ns)
+{
+	memset(&ns->ipcns_nb, 0, sizeof(ns->ipcns_nb));
+	ns->ipcns_nb.notifier_call = ipcns_callback;
+	ns->ipcns_nb.priority = IPCNS_CALLBACK_PRI;
+	return blocking_notifier_chain_register(&ipcns_chain, &ns->ipcns_nb);
+}
+
+int unregister_ipcns_notifier(struct ipc_namespace *ns)
+{
+	return blocking_notifier_chain_unregister(&ipcns_chain,
+						&ns->ipcns_nb);
+}
+
+int ipcns_notify(unsigned long val)
+{
+	return blocking_notifier_call_chain(&ipcns_chain, val, NULL);
+}
Index: linux-2.6.24-mm1/ipc/Makefile
===================================================================
--- linux-2.6.24-mm1.orig/ipc/Makefile	2008-02-07 13:41:07.000000000 +0100
+++ linux-2.6.24-mm1/ipc/Makefile	2008-02-08 08:10:13.000000000 +0100
@@ -3,7 +3,8 @@
 #
 
 obj-$(CONFIG_SYSVIPC_COMPAT) += compat.o
-obj-$(CONFIG_SYSVIPC) += util.o msgutil.o msg.o sem.o shm.o
+obj_mem-$(CONFIG_MEMORY_HOTPLUG) += ipcns_notifier.o
+obj-$(CONFIG_SYSVIPC) += util.o msgutil.o msg.o sem.o shm.o $(obj_mem-y)
 obj-$(CONFIG_SYSVIPC_SYSCTL) += ipc_sysctl.o
 obj_mq-$(CONFIG_COMPAT) += compat_mq.o
 obj-$(CONFIG_POSIX_MQUEUE) += mqueue.o msgutil.o $(obj_mq-y)
Index: linux-2.6.24-mm1/ipc/util.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/util.c	2008-02-07 15:36:22.000000000 +0100
+++ linux-2.6.24-mm1/ipc/util.c	2008-02-08 08:15:35.000000000 +0100
@@ -33,6 +33,7 @@
 #include <linux/audit.h>
 #include <linux/nsproxy.h>
 #include <linux/rwsem.h>
+#include <linux/memory.h>
 #include <linux/ipc_namespace.h>
 
 #include <asm/unistd.h>
@@ -55,11 +56,41 @@ struct ipc_namespace init_ipc_ns = {
 atomic_t nr_ipc_ns = ATOMIC_INIT(1);
 
 
+#ifdef CONFIG_MEMORY_HOTPLUG
+
+static int ipc_memory_callback(struct notifier_block *self,
+				unsigned long action, void *arg)
+{
+	switch (action) {
+	case MEM_ONLINE:    /* memory successfully brought online */
+	case MEM_OFFLINE:   /* or offline: it's time to recompute msgmni */
+		/*
+		 * This is done by invoking the ipcns notifier chain with the
+		 * IPC_MEMCHANGED event.
+		 */
+		ipcns_notify(IPCNS_MEMCHANGED);
+		break;
+	case MEM_GOING_ONLINE:
+	case MEM_GOING_OFFLINE:
+	case MEM_CANCEL_ONLINE:
+	case MEM_CANCEL_OFFLINE:
+	default:
+		break;
+	}
+
+	return NOTIFY_OK;
+}
+
+#endif /* CONFIG_MEMORY_HOTPLUG */
+
 /**
  *	ipc_init	-	initialise IPC subsystem
  *
  *	The various system5 IPC resources (semaphores, messages and shared
  *	memory) are initialised
+ *	A callback routine is registered into the memory hotplug notifier
+ *	chain: since msgmni scales to lowmem this callback routine will be
+ *	called upon successful memory add / remove to recompute msmgni.
  */
  
 static int __init ipc_init(void)
@@ -67,6 +98,8 @@ static int __init ipc_init(void)
 	sem_init();
 	msg_init();
 	shm_init();
+	hotplug_memory_notifier(ipc_memory_callback, IPC_CALLBACK_PRI);
+	register_ipcns_notifier(&init_ipc_ns);
 	return 0;
 }
 __initcall(ipc_init);
Index: linux-2.6.24-mm1/ipc/namespace.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/namespace.c	2008-02-07 15:40:19.000000000 +0100
+++ linux-2.6.24-mm1/ipc/namespace.c	2008-02-08 08:18:35.000000000 +0100
@@ -26,6 +26,8 @@ static struct ipc_namespace *clone_ipc_n
 	msg_init_ns(ns);
 	shm_init_ns(ns);
 
+	register_ipcns_notifier(ns);
+
 	kref_init(&ns->kref);
 	return ns;
 }
@@ -81,6 +83,15 @@ void free_ipc_ns(struct kref *kref)
 	struct ipc_namespace *ns;
 
 	ns = container_of(kref, struct ipc_namespace, kref);
+	/*
+	 * Unregistering the hotplug notifier at the beginning guarantees
+	 * that the ipc namespace won't be freed while we are inside the
+	 * callback routine. Since the blocking_notifier_chain_XXX routines
+	 * hold a rw lock on the notifier list, unregister_ipcns_notifier()
+	 * won't take the rw lock before blocking_notifier_call_chain() has
+	 * released the rd lock.
+	 */
+	unregister_ipcns_notifier(ns);
 	sem_exit_ns(ns);
 	msg_exit_ns(ns);
 	shm_exit_ns(ns);
Index: linux-2.6.24-mm1/ipc/msg.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/msg.c	2008-02-07 15:43:51.000000000 +0100
+++ linux-2.6.24-mm1/ipc/msg.c	2008-02-08 08:19:36.000000000 +0100
@@ -85,7 +85,7 @@ static int sysvipc_msg_proc_show(struct 
  * Also take into account the number of nsproxies created so far.
  * This should be done staying within the (MSGMNI , IPCMNI/nr_ipc_ns) range.
  */
-static void recompute_msgmni(struct ipc_namespace *ns)
+void recompute_msgmni(struct ipc_namespace *ns)
 {
 	struct sysinfo i;
 	unsigned long allowed;
Index: linux-2.6.24-mm1/ipc/util.h
===================================================================
--- linux-2.6.24-mm1.orig/ipc/util.h	2008-02-07 13:41:07.000000000 +0100
+++ linux-2.6.24-mm1/ipc/util.h	2008-02-08 08:21:23.000000000 +0100
@@ -124,6 +124,8 @@ extern void free_msg(struct msg_msg *msg
 extern struct msg_msg *load_msg(const void __user *src, int len);
 extern int store_msg(void __user *dest, struct msg_msg *msg, int len);
 
+extern void recompute_msgmni(struct ipc_namespace *);
+
 static inline int ipc_buildid(int id, int seq)
 {
 	return SEQ_MULTIPLIER * seq + id;

--

[PATCH 5/8] Invoke the ipcns notifier chain as a work item

[Nadia.Derbey]

[-- Attachment #1: ipc_ipcns_notification_workqueue.patch --]
[-- Type: text/plain, Size: 1581 bytes --]

[PATCH 05/08]

This patch makes the memory hotplug chain's mutex held for a shorter time:
when memory is offlined or onlined a work item is added to the global
workqueue.
When the work item is run, it notifies the ipcns notifier chain with the
IPCNS_MEMCHANGED event.

Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 ipc/util.c |   15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

Index: linux-2.6.24-mm1/ipc/util.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/util.c	2008-02-08 08:15:35.000000000 +0100
+++ linux-2.6.24-mm1/ipc/util.c	2008-02-08 14:14:03.000000000 +0100
@@ -58,6 +58,14 @@ atomic_t nr_ipc_ns = ATOMIC_INIT(1);
 
 #ifdef CONFIG_MEMORY_HOTPLUG
 
+static void ipc_memory_notifier(struct work_struct *work)
+{
+	ipcns_notify(IPCNS_MEMCHANGED);
+}
+
+static DECLARE_WORK(ipc_memory_wq, ipc_memory_notifier);
+
+
 static int ipc_memory_callback(struct notifier_block *self,
 				unsigned long action, void *arg)
 {
@@ -67,8 +75,13 @@ static int ipc_memory_callback(struct no
 		/*
 		 * This is done by invoking the ipcns notifier chain with the
 		 * IPC_MEMCHANGED event.
+		 * In order not to keep the lock on the hotplug memory chain
+		 * for too long, queue a work item that will, when waken up,
+		 * activate the ipcns notification chain.
+		 * No need to keep several ipc work items on the queue.
 		 */
-		ipcns_notify(IPCNS_MEMCHANGED);
+		if (!work_pending(&ipc_memory_wq))
+			schedule_work(&ipc_memory_wq);
 		break;
 	case MEM_GOING_ONLINE:
 	case MEM_GOING_OFFLINE:

--

[PATCH 6/8] Recomputing msgmni on ipc namespace creation/removal

[Nadia.Derbey]

[-- Attachment #1: ipc_recompute_msgmni_on_ipcns_create_remove.patch --]
[-- Type: text/plain, Size: 4604 bytes --]

[PATCH 06/08]

This patch introduces a notification mechanism that aims at recomputing
msgmni each time an ipc namespace is created or removed.

The ipc namespace notifier chain already defined for memory hotplug management
is used for that purpose too.

Each time a new ipc namespace is allocated or an existing ipc namespace is
removed, the ipcns notifier chain is notified. The callback routine for each
registered ipc namespace is then activated in order to recompute msgmni for
that namespace.  


Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 include/linux/ipc_namespace.h |   25 ++-----------------------
 ipc/Makefile                  |    3 +--
 ipc/ipcns_notifier.c          |    2 ++
 ipc/namespace.c               |   12 ++++++++++++
 4 files changed, 17 insertions(+), 25 deletions(-)

Index: linux-2.6.24-mm1/include/linux/ipc_namespace.h
===================================================================
--- linux-2.6.24-mm1.orig/include/linux/ipc_namespace.h	2008-02-08 08:29:21.000000000 +0100
+++ linux-2.6.24-mm1/include/linux/ipc_namespace.h	2008-02-08 14:35:08.000000000 +0100
@@ -4,14 +4,14 @@
 #include <linux/err.h>
 #include <linux/idr.h>
 #include <linux/rwsem.h>
-#ifdef CONFIG_MEMORY_HOTPLUG
 #include <linux/notifier.h>
-#endif /* CONFIG_MEMORY_HOTPLUG */
 
 /*
  * ipc namespace events
  */
 #define IPCNS_MEMCHANGED   0x00000001   /* Notify lowmem size changed */
+#define IPCNS_CREATED  0x00000002   /* Notify new ipc namespace created */
+#define IPCNS_REMOVED  0x00000003   /* Notify ipc namespace removed */
 
 #define IPCNS_CALLBACK_PRI 0
 
@@ -42,9 +42,7 @@ struct ipc_namespace {
 	int		shm_ctlmni;
 	int		shm_tot;
 
-#ifdef CONFIG_MEMORY_HOTPLUG
 	struct notifier_block ipcns_nb;
-#endif
 };
 
 extern struct ipc_namespace init_ipc_ns;
@@ -53,29 +51,10 @@ extern atomic_t nr_ipc_ns;
 #ifdef CONFIG_SYSVIPC
 #define INIT_IPC_NS(ns)		.ns		= &init_ipc_ns,
 
-#ifdef CONFIG_MEMORY_HOTPLUG
-
 extern int register_ipcns_notifier(struct ipc_namespace *);
 extern int unregister_ipcns_notifier(struct ipc_namespace *);
 extern int ipcns_notify(unsigned long);
 
-#else /* CONFIG_MEMORY_HOTPLUG */
-
-static inline int register_ipcns_notifier(struct ipc_namespace *ipcns)
-{
-	return 0;
-}
-static inline int unregister_ipcns_notifier(struct ipc_namespace *ipcns)
-{
-	return 0;
-}
-static inline int ipcns_notify(unsigned long ev)
-{
-	return 0;
-}
-
-#endif /* CONFIG_MEMORY_HOTPLUG */
-
 #else /* CONFIG_SYSVIPC */
 #define INIT_IPC_NS(ns)
 #endif /* CONFIG_SYSVIPC */
Index: linux-2.6.24-mm1/ipc/ipcns_notifier.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/ipcns_notifier.c	2008-02-08 13:59:26.000000000 +0100
+++ linux-2.6.24-mm1/ipc/ipcns_notifier.c	2008-02-08 14:36:05.000000000 +0100
@@ -29,6 +29,8 @@ static int ipcns_callback(struct notifie
 
 	switch (action) {
 	case IPCNS_MEMCHANGED:   /* amount of lowmem has changed */
+	case IPCNS_CREATED:
+	case IPCNS_REMOVED:
 		/*
 		 * It's time to recompute msgmni
 		 */
Index: linux-2.6.24-mm1/ipc/Makefile
===================================================================
--- linux-2.6.24-mm1.orig/ipc/Makefile	2008-02-08 08:10:13.000000000 +0100
+++ linux-2.6.24-mm1/ipc/Makefile	2008-02-08 14:36:52.000000000 +0100
@@ -3,8 +3,7 @@
 #
 
 obj-$(CONFIG_SYSVIPC_COMPAT) += compat.o
-obj_mem-$(CONFIG_MEMORY_HOTPLUG) += ipcns_notifier.o
-obj-$(CONFIG_SYSVIPC) += util.o msgutil.o msg.o sem.o shm.o $(obj_mem-y)
+obj-$(CONFIG_SYSVIPC) += util.o msgutil.o msg.o sem.o shm.o ipcns_notifier.o
 obj-$(CONFIG_SYSVIPC_SYSCTL) += ipc_sysctl.o
 obj_mq-$(CONFIG_COMPAT) += compat_mq.o
 obj-$(CONFIG_POSIX_MQUEUE) += mqueue.o msgutil.o $(obj_mq-y)
Index: linux-2.6.24-mm1/ipc/namespace.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/namespace.c	2008-02-08 08:18:35.000000000 +0100
+++ linux-2.6.24-mm1/ipc/namespace.c	2008-02-08 14:41:37.000000000 +0100
@@ -26,6 +26,12 @@ static struct ipc_namespace *clone_ipc_n
 	msg_init_ns(ns);
 	shm_init_ns(ns);
 
+	/*
+	 * msgmni has already been computed for the new ipc ns.
+	 * Thus, do the ipcns creation notification before registering that
+	 * new ipcns in the chain.
+	 */
+	ipcns_notify(IPCNS_CREATED);
 	register_ipcns_notifier(ns);
 
 	kref_init(&ns->kref);
@@ -97,4 +103,10 @@ void free_ipc_ns(struct kref *kref)
 	shm_exit_ns(ns);
 	kfree(ns);
 	atomic_dec(&nr_ipc_ns);
+
+	/*
+	 * Do the ipcns removal notification after decrementing nr_ipc_ns in
+	 * order to have a correct value when recomputing msgmni.
+	 */
+	ipcns_notify(IPCNS_REMOVED);
 }

--

[PATCH 7/8] Do not recompute msgmni anymore if explicitely set by user

[Nadia.Derbey]

[-- Attachment #1: ipc_unregister_callback_on_msgmni_manual_change.patch --]
[-- Type: text/plain, Size: 2799 bytes --]

[PATCH 07/08]

This patch makes msgmni not recomputed anymore upon ipc namespace creation /
removal or memory add/remove, as soon as it has been set from userland.

As soon as msgmni is explicitely set via procfs or sysctl(), the associated
callback routine is unregistered from the ipc namespace notifier chain.


Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 ipc/ipc_sysctl.c |   43 +++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 2 deletions(-)

Index: linux-2.6.24-mm1/ipc/ipc_sysctl.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/ipc_sysctl.c	2008-02-08 16:07:15.000000000 +0100
+++ linux-2.6.24-mm1/ipc/ipc_sysctl.c	2008-02-08 16:08:32.000000000 +0100
@@ -35,6 +35,24 @@ static int proc_ipc_dointvec(ctl_table *
 	return proc_dointvec(&ipc_table, write, filp, buffer, lenp, ppos);
 }
 
+static int proc_ipc_callback_dointvec(ctl_table *table, int write,
+	struct file *filp, void __user *buffer, size_t *lenp, loff_t *ppos)
+{
+	size_t lenp_bef = *lenp;
+	int rc;
+
+	rc = proc_ipc_dointvec(table, write, filp, buffer, lenp, ppos);
+
+	if (write && !rc && lenp_bef == *lenp)
+		/*
+		 * Tunable has successfully been changed from userland:
+		 * disable its automatic recomputing.
+		 */
+		unregister_ipcns_notifier(current->nsproxy->ipc_ns);
+
+	return rc;
+}
+
 static int proc_ipc_doulongvec_minmax(ctl_table *table, int write,
 	struct file *filp, void __user *buffer, size_t *lenp, loff_t *ppos)
 {
@@ -49,6 +67,7 @@ static int proc_ipc_doulongvec_minmax(ct
 #else
 #define proc_ipc_doulongvec_minmax NULL
 #define proc_ipc_dointvec	   NULL
+#define proc_ipc_callback_dointvec NULL
 #endif
 
 #ifdef CONFIG_SYSCTL_SYSCALL
@@ -90,8 +109,28 @@ static int sysctl_ipc_data(ctl_table *ta
 	}
 	return 1;
 }
+
+static int sysctl_ipc_registered_data(ctl_table *table, int __user *name,
+		int nlen, void __user *oldval, size_t __user *oldlenp,
+		void __user *newval, size_t newlen)
+{
+	int rc;
+
+	rc = sysctl_ipc_data(table, name, nlen, oldval, oldlenp, newval,
+		newlen);
+
+	if (newval && newlen && rc > 0)
+		/*
+		 * Tunable has successfully been changed from userland:
+		 * disable its automatic recomputing.
+		 */
+		unregister_ipcns_notifier(current->nsproxy->ipc_ns);
+
+	return rc;
+}
 #else
 #define sysctl_ipc_data NULL
+#define sysctl_ipc_registered_data NULL
 #endif
 
 static struct ctl_table ipc_kern_table[] = {
@@ -137,8 +176,8 @@ static struct ctl_table ipc_kern_table[]
 		.data		= &init_ipc_ns.msg_ctlmni,
 		.maxlen		= sizeof (init_ipc_ns.msg_ctlmni),
 		.mode		= 0644,
-		.proc_handler	= proc_ipc_dointvec,
-		.strategy	= sysctl_ipc_data,
+		.proc_handler	= proc_ipc_callback_dointvec,
+		.strategy	= sysctl_ipc_registered_data,
 	},
 	{
 		.ctl_name	= KERN_MSGMNB,

--

[PATCH 8/8] Re-enable msgmni automatic recomputing msgmni if set to negative

[Nadia.Derbey]

[-- Attachment #1: ipc_register_callback_on_negative_msgmni.patch --]
[-- Type: text/plain, Size: 7412 bytes --]

[PATCH 08/08]

This patch is the enhancement as asked for by Yasunori: if msgmni is set to
a negative value, register it back into the ipcns notifier chain.

A new interface has been added to the notification mechanism:
notifier_chain_cond_register() registers a notifier block only if not already
registered. With that new interface we avoid taking care of the states changes
in procfs.

Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 include/linux/ipc_namespace.h |    1 
 include/linux/notifier.h      |    4 +++
 ipc/ipc_sysctl.c              |   45 ++++++++++++++++++++++++++++++++----------
 ipc/ipcns_notifier.c          |    9 ++++++++
 kernel/notifier.c             |   38 +++++++++++++++++++++++++++++++++++
 5 files changed, 87 insertions(+), 10 deletions(-)

Index: linux-2.6.24-mm1/ipc/ipc_sysctl.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/ipc_sysctl.c	2008-02-08 16:08:32.000000000 +0100
+++ linux-2.6.24-mm1/ipc/ipc_sysctl.c	2008-02-11 14:32:09.000000000 +0100
@@ -15,6 +15,8 @@
 #include <linux/sysctl.h>
 #include <linux/uaccess.h>
 #include <linux/ipc_namespace.h>
+#include <linux/msg.h>
+#include "util.h"
 
 static void *get_ipc(ctl_table *table)
 {
@@ -24,6 +26,27 @@ static void *get_ipc(ctl_table *table)
 	return which;
 }
 
+/*
+ * Routine that is called when a tunable has successfully been changed by
+ * hand and it has a callback routine registered on the ipc namespace notifier
+ * chain: we don't want such tunables to be recomputed anymore upon memory
+ * add/remove or ipc namespace creation/removal.
+ * They can come back to a recomputable state by being set to a <0 value.
+ */
+static void tunable_set_callback(int val)
+{
+	if (val >= 0)
+		unregister_ipcns_notifier(current->nsproxy->ipc_ns);
+	else {
+		/*
+		 * Re-enable automatic recomputing only if not already
+		 * enabled.
+		 */
+		recompute_msgmni(current->nsproxy->ipc_ns);
+		cond_register_ipcns_notifier(current->nsproxy->ipc_ns);
+	}
+}
+
 #ifdef CONFIG_PROC_FS
 static int proc_ipc_dointvec(ctl_table *table, int write, struct file *filp,
 	void __user *buffer, size_t *lenp, loff_t *ppos)
@@ -38,17 +61,17 @@ static int proc_ipc_dointvec(ctl_table *
 static int proc_ipc_callback_dointvec(ctl_table *table, int write,
 	struct file *filp, void __user *buffer, size_t *lenp, loff_t *ppos)
 {
+	struct ctl_table ipc_table;
 	size_t lenp_bef = *lenp;
 	int rc;
 
-	rc = proc_ipc_dointvec(table, write, filp, buffer, lenp, ppos);
+	memcpy(&ipc_table, table, sizeof(ipc_table));
+	ipc_table.data = get_ipc(table);
+
+	rc = proc_dointvec(&ipc_table, write, filp, buffer, lenp, ppos);
 
 	if (write && !rc && lenp_bef == *lenp)
-		/*
-		 * Tunable has successfully been changed from userland:
-		 * disable its automatic recomputing.
-		 */
-		unregister_ipcns_notifier(current->nsproxy->ipc_ns);
+		tunable_set_callback(*((int *)(ipc_table.data)));
 
 	return rc;
 }
@@ -119,12 +142,14 @@ static int sysctl_ipc_registered_data(ct
 	rc = sysctl_ipc_data(table, name, nlen, oldval, oldlenp, newval,
 		newlen);
 
-	if (newval && newlen && rc > 0)
+	if (newval && newlen && rc > 0) {
 		/*
-		 * Tunable has successfully been changed from userland:
-		 * disable its automatic recomputing.
+		 * Tunable has successfully been changed from userland
 		 */
-		unregister_ipcns_notifier(current->nsproxy->ipc_ns);
+		int *data = get_ipc(table);
+
+		tunable_set_callback(*data);
+	}
 
 	return rc;
 }
Index: linux-2.6.24-mm1/kernel/notifier.c
===================================================================
--- linux-2.6.24-mm1.orig/kernel/notifier.c	2008-02-07 13:40:46.000000000 +0100
+++ linux-2.6.24-mm1/kernel/notifier.c	2008-02-11 14:54:34.000000000 +0100
@@ -31,6 +31,21 @@ static int notifier_chain_register(struc
 	return 0;
 }
 
+static int notifier_chain_cond_register(struct notifier_block **nl,
+		struct notifier_block *n)
+{
+	while ((*nl) != NULL) {
+		if ((*nl) == n)
+			return 0;
+		if (n->priority > (*nl)->priority)
+			break;
+		nl = &((*nl)->next);
+	}
+	n->next = *nl;
+	rcu_assign_pointer(*nl, n);
+	return 0;
+}
+
 static int notifier_chain_unregister(struct notifier_block **nl,
 		struct notifier_block *n)
 {
@@ -205,6 +220,29 @@ int blocking_notifier_chain_register(str
 EXPORT_SYMBOL_GPL(blocking_notifier_chain_register);
 
 /**
+ *	blocking_notifier_chain_cond_register - Cond add notifier to a blocking notifier chain
+ *	@nh: Pointer to head of the blocking notifier chain
+ *	@n: New entry in notifier chain
+ *
+ *	Adds a notifier to a blocking notifier chain, only if not already
+ *	present in the chain.
+ *	Must be called in process context.
+ *
+ *	Currently always returns zero.
+ */
+int blocking_notifier_chain_cond_register(struct blocking_notifier_head *nh,
+		struct notifier_block *n)
+{
+	int ret;
+
+	down_write(&nh->rwsem);
+	ret = notifier_chain_cond_register(&nh->head, n);
+	up_write(&nh->rwsem);
+	return ret;
+}
+EXPORT_SYMBOL_GPL(blocking_notifier_chain_cond_register);
+
+/**
  *	blocking_notifier_chain_unregister - Remove notifier from a blocking notifier chain
  *	@nh: Pointer to head of the blocking notifier chain
  *	@n: Entry to remove from notifier chain
Index: linux-2.6.24-mm1/include/linux/notifier.h
===================================================================
--- linux-2.6.24-mm1.orig/include/linux/notifier.h	2008-02-07 13:40:35.000000000 +0100
+++ linux-2.6.24-mm1/include/linux/notifier.h	2008-02-11 14:55:06.000000000 +0100
@@ -121,6 +121,10 @@ extern int raw_notifier_chain_register(s
 extern int srcu_notifier_chain_register(struct srcu_notifier_head *nh,
 		struct notifier_block *nb);
 
+extern int blocking_notifier_chain_cond_register(
+		struct blocking_notifier_head *nh,
+		struct notifier_block *nb);
+
 extern int atomic_notifier_chain_unregister(struct atomic_notifier_head *nh,
 		struct notifier_block *nb);
 extern int blocking_notifier_chain_unregister(struct blocking_notifier_head *nh,
Index: linux-2.6.24-mm1/include/linux/ipc_namespace.h
===================================================================
--- linux-2.6.24-mm1.orig/include/linux/ipc_namespace.h	2008-02-08 14:35:08.000000000 +0100
+++ linux-2.6.24-mm1/include/linux/ipc_namespace.h	2008-02-11 11:19:31.000000000 +0100
@@ -52,6 +52,7 @@ extern atomic_t nr_ipc_ns;
 #define INIT_IPC_NS(ns)		.ns		= &init_ipc_ns,
 
 extern int register_ipcns_notifier(struct ipc_namespace *);
+extern int cond_register_ipcns_notifier(struct ipc_namespace *);
 extern int unregister_ipcns_notifier(struct ipc_namespace *);
 extern int ipcns_notify(unsigned long);
 
Index: linux-2.6.24-mm1/ipc/ipcns_notifier.c
===================================================================
--- linux-2.6.24-mm1.orig/ipc/ipcns_notifier.c	2008-02-08 14:36:05.000000000 +0100
+++ linux-2.6.24-mm1/ipc/ipcns_notifier.c	2008-02-11 12:24:39.000000000 +0100
@@ -61,6 +61,15 @@ int register_ipcns_notifier(struct ipc_n
 	return blocking_notifier_chain_register(&ipcns_chain, &ns->ipcns_nb);
 }
 
+int cond_register_ipcns_notifier(struct ipc_namespace *ns)
+{
+	memset(&ns->ipcns_nb, 0, sizeof(ns->ipcns_nb));
+	ns->ipcns_nb.notifier_call = ipcns_callback;
+	ns->ipcns_nb.priority = IPCNS_CALLBACK_PRI;
+	return blocking_notifier_chain_cond_register(&ipcns_chain,
+							&ns->ipcns_nb);
+}
+
 int unregister_ipcns_notifier(struct ipc_namespace *ns)
 {
 	return blocking_notifier_chain_unregister(&ipcns_chain,

--

Re: [PATCH 7/8] Do not recompute msgmni anymore if explicitely set by user

[Andrew Morton]

On Mon, 11 Feb 2008 15:16:53 +0100
Nadia.Derbey@bull.net wrote:

> [PATCH 07/08]
> 
> This patch makes msgmni not recomputed anymore upon ipc namespace creation /
> removal or memory add/remove, as soon as it has been set from userland.
> 
> As soon as msgmni is explicitely set via procfs or sysctl(), the associated
> callback routine is unregistered from the ipc namespace notifier chain.
> 

The patch series looks pretty good.

> ===================================================================
> --- linux-2.6.24-mm1.orig/ipc/ipc_sysctl.c	2008-02-08 16:07:15.000000000 +0100
> +++ linux-2.6.24-mm1/ipc/ipc_sysctl.c	2008-02-08 16:08:32.000000000 +0100
> @@ -35,6 +35,24 @@ static int proc_ipc_dointvec(ctl_table *
>  	return proc_dointvec(&ipc_table, write, filp, buffer, lenp, ppos);
>  }
>  
> +static int proc_ipc_callback_dointvec(ctl_table *table, int write,
> +	struct file *filp, void __user *buffer, size_t *lenp, loff_t *ppos)
> +{
> +	size_t lenp_bef = *lenp;
> +	int rc;
> +
> +	rc = proc_ipc_dointvec(table, write, filp, buffer, lenp, ppos);
> +
> +	if (write && !rc && lenp_bef == *lenp)
> +		/*
> +		 * Tunable has successfully been changed from userland:
> +		 * disable its automatic recomputing.
> +		 */
> +		unregister_ipcns_notifier(current->nsproxy->ipc_ns);
> +
> +	return rc;
> +}

If you haven't done so, could you please check that it all builds cleanly
with CONFIG_PROCFS=n, and that all code which isn't needed if procfs is
disabled is not present in the final binary?

Re: [PATCH 8/8] Re-enable msgmni automatic recomputing msgmni if set to negative

[Andrew Morton]

On Mon, 11 Feb 2008 15:16:54 +0100
Nadia.Derbey@bull.net wrote:

> [PATCH 08/08]
> 
> This patch is the enhancement as asked for by Yasunori: if msgmni is set to
> a negative value, register it back into the ipcns notifier chain.
> 
> A new interface has been added to the notification mechanism:
> notifier_chain_cond_register() registers a notifier block only if not already
> registered. With that new interface we avoid taking care of the states changes
> in procfs.
> 
> ...
>
>  static int proc_ipc_callback_dointvec(ctl_table *table, int write,
>  	struct file *filp, void __user *buffer, size_t *lenp, loff_t *ppos)
>  {
> +	struct ctl_table ipc_table;
>  	size_t lenp_bef = *lenp;
>  	int rc;
>  
> -	rc = proc_ipc_dointvec(table, write, filp, buffer, lenp, ppos);
> +	memcpy(&ipc_table, table, sizeof(ipc_table));
> +	ipc_table.data = get_ipc(table);
> +
> +	rc = proc_dointvec(&ipc_table, write, filp, buffer, lenp, ppos);
>  
>  	if (write && !rc && lenp_bef == *lenp)
> -		/*
> -		 * Tunable has successfully been changed from userland:
> -		 * disable its automatic recomputing.
> -		 */
> -		unregister_ipcns_notifier(current->nsproxy->ipc_ns);
> +		tunable_set_callback(*((int *)(ipc_table.data)));
>  
>  	return rc;
>  }
> @@ -119,12 +142,14 @@ static int sysctl_ipc_registered_data(ct
>  	rc = sysctl_ipc_data(table, name, nlen, oldval, oldlenp, newval,
>  		newlen);
>  
> -	if (newval && newlen && rc > 0)
> +	if (newval && newlen && rc > 0) {
>  		/*
> -		 * Tunable has successfully been changed from userland:
> -		 * disable its automatic recomputing.
> +		 * Tunable has successfully been changed from userland
>  		 */
> -		unregister_ipcns_notifier(current->nsproxy->ipc_ns);
> +		int *data = get_ipc(table);
> +
> +		tunable_set_callback(*data);
> +	}
>  
>  	return rc;
>  }

hm, what's happening here?  We take a local copy of the caller's ctl_table
and then pass that into proc_dointvec().  Is that as hacky as it seems??

Re: [PATCH 7/8] Do not recompute msgmni anymore if explicitely set by user

[Nadia Derbey]

[-- Attachment #1: Type: text/plain, Size: 1706 bytes --]

Andrew Morton wrote:
> On Mon, 11 Feb 2008 15:16:53 +0100
> Nadia.Derbey@bull.net wrote:
> 
> 
>>[PATCH 07/08]
>>
>>This patch makes msgmni not recomputed anymore upon ipc namespace creation /
>>removal or memory add/remove, as soon as it has been set from userland.
>>
>>As soon as msgmni is explicitely set via procfs or sysctl(), the associated
>>callback routine is unregistered from the ipc namespace notifier chain.
>>
> 
> 
> The patch series looks pretty good.
> 
> 
>>===================================================================
>>--- linux-2.6.24-mm1.orig/ipc/ipc_sysctl.c	2008-02-08 16:07:15.000000000 +0100
>>+++ linux-2.6.24-mm1/ipc/ipc_sysctl.c	2008-02-08 16:08:32.000000000 +0100
>>@@ -35,6 +35,24 @@ static int proc_ipc_dointvec(ctl_table *
>> 	return proc_dointvec(&ipc_table, write, filp, buffer, lenp, ppos);
>> }
>> 
>>+static int proc_ipc_callback_dointvec(ctl_table *table, int write,
>>+	struct file *filp, void __user *buffer, size_t *lenp, loff_t *ppos)
>>+{
>>+	size_t lenp_bef = *lenp;
>>+	int rc;
>>+
>>+	rc = proc_ipc_dointvec(table, write, filp, buffer, lenp, ppos);
>>+
>>+	if (write && !rc && lenp_bef == *lenp)
>>+		/*
>>+		 * Tunable has successfully been changed from userland:
>>+		 * disable its automatic recomputing.
>>+		 */
>>+		unregister_ipcns_notifier(current->nsproxy->ipc_ns);
>>+
>>+	return rc;
>>+}
> 
> 
> If you haven't done so, could you please check that it all builds cleanly
> with CONFIG_PROCFS=n, and that all code which isn't needed if procfs is
> disabled is not present in the final binary?
> 
> 
> 
> 

Andrew,

it builds fine, modulo some changes in ipv4 and ipv6 (see attached patch 
- didn't find it in the hot fixes).

Regards,
Nadia


[-- Attachment #2: ip_v4_v6_procfs.patch --]
[-- Type: text/x-patch, Size: 1957 bytes --]

Fix header files to let IPV4 and IPV6 build if CONFIG_PROC_FS=n

Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 include/net/ip_fib.h |   13 ++++++++++++-
 include/net/ipv6.h   |    6 +++---
 2 files changed, 15 insertions(+), 4 deletions(-)

Index: linux-2.6.24-mm1/include/net/ip_fib.h
===================================================================
--- linux-2.6.24-mm1.orig/include/net/ip_fib.h	2008-02-12 11:03:40.000000000 +0100
+++ linux-2.6.24-mm1/include/net/ip_fib.h	2008-02-12 11:09:40.000000000 +0100
@@ -266,6 +266,17 @@ static inline void fib_res_put(struct fi
 #ifdef CONFIG_PROC_FS
 extern int __net_init  fib_proc_init(struct net *net);
 extern void __net_exit fib_proc_exit(struct net *net);
-#endif
+#else /* CONFIG_PROC_FS */
+static inline int fib_proc_init(struct net *net)
+{
+	return 0;
+}
+
+static inline int fib_proc_exit(struct net *net)
+{
+	return 0;
+}
+
+#endif /* CONFIG_PROC_FS */
 
 #endif  /* _NET_FIB_H */
Index: linux-2.6.24-mm1/include/net/ipv6.h
===================================================================
--- linux-2.6.24-mm1.orig/include/net/ipv6.h	2008-02-07 13:40:38.000000000 +0100
+++ linux-2.6.24-mm1/include/net/ipv6.h	2008-02-12 11:16:27.000000000 +0100
@@ -586,9 +586,6 @@ extern int ip6_mc_msfget(struct sock *sk
 			 int __user *optlen);
 
 #ifdef CONFIG_PROC_FS
-extern struct ctl_table *ipv6_icmp_sysctl_init(struct net *net);
-extern struct ctl_table *ipv6_route_sysctl_init(struct net *net);
-
 extern int  ac6_proc_init(void);
 extern void ac6_proc_exit(void);
 extern int  raw6_proc_init(void);
@@ -621,6 +618,9 @@ static inline int snmp6_unregister_dev(s
 extern ctl_table ipv6_route_table_template[];
 extern ctl_table ipv6_icmp_table_template[];
 
+extern struct ctl_table *ipv6_icmp_sysctl_init(struct net *net);
+extern struct ctl_table *ipv6_route_sysctl_init(struct net *net);
+
 extern int ipv6_sysctl_register(void);
 extern void ipv6_sysctl_unregister(void);
 #endif

Re: [PATCH 7/8] Do not recompute msgmni anymore if explicitely set by user

[Andrew Morton]

On Tue, 12 Feb 2008 10:32:31 +0100 Nadia Derbey <Nadia.Derbey@bull.net> wrote:

> it builds fine, modulo some changes in ipv4 and ipv6 (see attached patch 
> - didn't find it in the hot fixes).

OK, thanks for checking.  Did you confirm that we don't have unneeded code
in vmlinux when CONFIG_PROCFS=n?  I guess before-and-after comparison of
the size(1) output would tell us.

Those networking build problems appear to have already been fixed.

In future, please quote the compiler error output in the changelog when
sending build fixes or warning fixes, thanks.

Re: [PATCH 7/8] Do not recompute msgmni anymore if explicitely set by user

[Nadia Derbey]

[-- Attachment #1: Type: text/plain, Size: 1937 bytes --]

Nadia Derbey wrote:
> Andrew Morton wrote:
> 
>> On Mon, 11 Feb 2008 15:16:53 +0100
>> Nadia.Derbey@bull.net wrote:
>>
>>
>>> [PATCH 07/08]
>>>
>>> This patch makes msgmni not recomputed anymore upon ipc namespace 
>>> creation /
>>> removal or memory add/remove, as soon as it has been set from userland.
>>>
>>> As soon as msgmni is explicitely set via procfs or sysctl(), the 
>>> associated
>>> callback routine is unregistered from the ipc namespace notifier chain.
>>>
>>
>>
>> The patch series looks pretty good.
>>
>>
>>> ===================================================================
>>> --- linux-2.6.24-mm1.orig/ipc/ipc_sysctl.c    2008-02-08 
>>> 16:07:15.000000000 +0100
>>> +++ linux-2.6.24-mm1/ipc/ipc_sysctl.c    2008-02-08 
>>> 16:08:32.000000000 +0100
>>> @@ -35,6 +35,24 @@ static int proc_ipc_dointvec(ctl_table *
>>>     return proc_dointvec(&ipc_table, write, filp, buffer, lenp, ppos);
>>> }
>>>
>>> +static int proc_ipc_callback_dointvec(ctl_table *table, int write,
>>> +    struct file *filp, void __user *buffer, size_t *lenp, loff_t *ppos)
>>> +{
>>> +    size_t lenp_bef = *lenp;
>>> +    int rc;
>>> +
>>> +    rc = proc_ipc_dointvec(table, write, filp, buffer, lenp, ppos);
>>> +
>>> +    if (write && !rc && lenp_bef == *lenp)
>>> +        /*
>>> +         * Tunable has successfully been changed from userland:
>>> +         * disable its automatic recomputing.
>>> +         */
>>> +        unregister_ipcns_notifier(current->nsproxy->ipc_ns);
>>> +
>>> +    return rc;
>>> +}
>>
>>
>>
>> If you haven't done so, could you please check that it all builds cleanly
>> with CONFIG_PROCFS=n, and that all code which isn't needed if procfs is
>> disabled is not present in the final binary?
>>
>>
>>
>>
> 
> Andrew,
> 
> it builds fine, modulo some changes in ipv4 and ipv6 (see attached patch 
> - didn't find it in the hot fixes).
> 
> Regards,
> Nadia
> 
> 

Oops, forgot the patch. Thx Benjamin!




[-- Attachment #2: ip_v4_v6_procfs.patch --]
[-- Type: text/x-patch, Size: 1957 bytes --]

Fix header files to let IPV4 and IPV6 build if CONFIG_PROC_FS=n

Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 include/net/ip_fib.h |   13 ++++++++++++-
 include/net/ipv6.h   |    6 +++---
 2 files changed, 15 insertions(+), 4 deletions(-)

Index: linux-2.6.24-mm1/include/net/ip_fib.h
===================================================================
--- linux-2.6.24-mm1.orig/include/net/ip_fib.h	2008-02-12 11:03:40.000000000 +0100
+++ linux-2.6.24-mm1/include/net/ip_fib.h	2008-02-12 11:09:40.000000000 +0100
@@ -266,6 +266,17 @@ static inline void fib_res_put(struct fi
 #ifdef CONFIG_PROC_FS
 extern int __net_init  fib_proc_init(struct net *net);
 extern void __net_exit fib_proc_exit(struct net *net);
-#endif
+#else /* CONFIG_PROC_FS */
+static inline int fib_proc_init(struct net *net)
+{
+	return 0;
+}
+
+static inline int fib_proc_exit(struct net *net)
+{
+	return 0;
+}
+
+#endif /* CONFIG_PROC_FS */
 
 #endif  /* _NET_FIB_H */
Index: linux-2.6.24-mm1/include/net/ipv6.h
===================================================================
--- linux-2.6.24-mm1.orig/include/net/ipv6.h	2008-02-07 13:40:38.000000000 +0100
+++ linux-2.6.24-mm1/include/net/ipv6.h	2008-02-12 11:16:27.000000000 +0100
@@ -586,9 +586,6 @@ extern int ip6_mc_msfget(struct sock *sk
 			 int __user *optlen);
 
 #ifdef CONFIG_PROC_FS
-extern struct ctl_table *ipv6_icmp_sysctl_init(struct net *net);
-extern struct ctl_table *ipv6_route_sysctl_init(struct net *net);
-
 extern int  ac6_proc_init(void);
 extern void ac6_proc_exit(void);
 extern int  raw6_proc_init(void);
@@ -621,6 +618,9 @@ static inline int snmp6_unregister_dev(s
 extern ctl_table ipv6_route_table_template[];
 extern ctl_table ipv6_icmp_table_template[];
 
+extern struct ctl_table *ipv6_icmp_sysctl_init(struct net *net);
+extern struct ctl_table *ipv6_route_sysctl_init(struct net *net);
+
 extern int ipv6_sysctl_register(void);
 extern void ipv6_sysctl_unregister(void);
 #endif

Re: [PATCH 8/8] Re-enable msgmni automatic recomputing msgmni if set to negative

[Nadia Derbey]

Andrew Morton wrote:
> On Mon, 11 Feb 2008 15:16:54 +0100
> Nadia.Derbey@bull.net wrote:
> 
> 
>>[PATCH 08/08]
>>
>>This patch is the enhancement as asked for by Yasunori: if msgmni is set to
>>a negative value, register it back into the ipcns notifier chain.
>>
>>A new interface has been added to the notification mechanism:
>>notifier_chain_cond_register() registers a notifier block only if not already
>>registered. With that new interface we avoid taking care of the states changes
>>in procfs.
>>
>>...
>>
>> static int proc_ipc_callback_dointvec(ctl_table *table, int write,
>> 	struct file *filp, void __user *buffer, size_t *lenp, loff_t *ppos)
>> {
>>+	struct ctl_table ipc_table;
>> 	size_t lenp_bef = *lenp;
>> 	int rc;
>> 
>>-	rc = proc_ipc_dointvec(table, write, filp, buffer, lenp, ppos);
>>+	memcpy(&ipc_table, table, sizeof(ipc_table));
>>+	ipc_table.data = get_ipc(table);
>>+
>>+	rc = proc_dointvec(&ipc_table, write, filp, buffer, lenp, ppos);
>> 
>> 	if (write && !rc && lenp_bef == *lenp)
>>-		/*
>>-		 * Tunable has successfully been changed from userland:
>>-		 * disable its automatic recomputing.
>>-		 */
>>-		unregister_ipcns_notifier(current->nsproxy->ipc_ns);
>>+		tunable_set_callback(*((int *)(ipc_table.data)));
>> 
>> 	return rc;
>> }
>>@@ -119,12 +142,14 @@ static int sysctl_ipc_registered_data(ct
>> 	rc = sysctl_ipc_data(table, name, nlen, oldval, oldlenp, newval,
>> 		newlen);
>> 
>>-	if (newval && newlen && rc > 0)
>>+	if (newval && newlen && rc > 0) {
>> 		/*
>>-		 * Tunable has successfully been changed from userland:
>>-		 * disable its automatic recomputing.
>>+		 * Tunable has successfully been changed from userland
>> 		 */
>>-		unregister_ipcns_notifier(current->nsproxy->ipc_ns);
>>+		int *data = get_ipc(table);
>>+
>>+		tunable_set_callback(*data);
>>+	}
>> 
>> 	return rc;
>> }
> 
> 
> hm, what's happening here?  We take a local copy of the caller's ctl_table
> and then pass that into proc_dointvec().  Is that as hacky as it seems??
> 
> 

Well, the caller's ctl_table contains the tunables addresses for the 
init namspeace in its .data fields. While what needs to be passed in to 
proc_dointvec() is the tunable address in the caller's namespace.
Since all the fields in ipc_kern_table[] are ok but the .data one, imho 
it's correct to store it in a local copy and change the data field to 
the appropirate one, before passing it to proc_dointvec().

Regards,
Nadia

Re: [PATCH 7/8] Do not recompute msgmni anymore if explicitely set by user

[Nadia Derbey]

Andrew Morton wrote:
> On Tue, 12 Feb 2008 10:32:31 +0100 Nadia Derbey <Nadia.Derbey@bull.net> wrote:
> 
> 
>>it builds fine, modulo some changes in ipv4 and ipv6 (see attached patch 
>>- didn't find it in the hot fixes).
> 
> 
> OK, thanks for checking.  Did you confirm that we don't have unneeded code
> in vmlinux when CONFIG_PROCFS=n?  I guess before-and-after comparison of
> the size(1) output would tell us.
> 
> Those networking build problems appear to have already been fixed.
> 
> In future, please quote the compiler error output in the changelog when
> sending build fixes or warning fixes, thanks.
> 
> 
> 

BEFORE:

lkernel@akt$ size vmlinux
    text    data     bss     dec     hex filename
4318525  454484  462848 5235857  4fe491 vmlinux


AFTER:

lkernel@akt$ size vmlinux
    text    data     bss     dec     hex filename
4323161  454484  462848 5240493  4ff6ad vmlinux

which makes it +4636 = +0.11%

I've got the details for */built-in.o if needed.


Regards,
Nadia

Re: [PATCH 7/8] Do not recompute msgmni anymore if explicitely set by user

[Andrew Morton]

On Tue, 12 Feb 2008 16:15:00 +0100
Nadia Derbey <Nadia.Derbey@bull.net> wrote:

> Andrew Morton wrote:
> > On Tue, 12 Feb 2008 10:32:31 +0100 Nadia Derbey <Nadia.Derbey@bull.net> wrote:
> > 
> > 
> >>it builds fine, modulo some changes in ipv4 and ipv6 (see attached patch 
> >>- didn't find it in the hot fixes).
> > 
> > 
> > OK, thanks for checking.  Did you confirm that we don't have unneeded code
> > in vmlinux when CONFIG_PROCFS=n?  I guess before-and-after comparison of
> > the size(1) output would tell us.
> > 
> > Those networking build problems appear to have already been fixed.
> > 
> > In future, please quote the compiler error output in the changelog when
> > sending build fixes or warning fixes, thanks.
> > 
> > 
> > 
> 
> BEFORE:
> 
> lkernel@akt$ size vmlinux
>     text    data     bss     dec     hex filename
> 4318525  454484  462848 5235857  4fe491 vmlinux
> 
> 
> AFTER:
> 
> lkernel@akt$ size vmlinux
>     text    data     bss     dec     hex filename
> 4323161  454484  462848 5240493  4ff6ad vmlinux
> 
> which makes it +4636 = +0.11%
> 
> I've got the details for */built-in.o if needed.
> 

That seems to be a lot of increase.  Are you sure you had CONFIG_PROCFS=n
in both cases?  If so, the patch must have added a lot of code which will
never be executed?

Re: [PATCH 7/8] Do not recompute msgmni anymore if explicitely set by user

[Nadia Derbey]

Andrew Morton wrote:
> On Tue, 12 Feb 2008 16:15:00 +0100
> Nadia Derbey <Nadia.Derbey@bull.net> wrote:
> 
> 
>>Andrew Morton wrote:
>>
>>>On Tue, 12 Feb 2008 10:32:31 +0100 Nadia Derbey <Nadia.Derbey@bull.net> wrote:
>>>
>>>
>>>
>>>>it builds fine, modulo some changes in ipv4 and ipv6 (see attached patch 
>>>>- didn't find it in the hot fixes).
>>>
>>>
>>>OK, thanks for checking.  Did you confirm that we don't have unneeded code
>>>in vmlinux when CONFIG_PROCFS=n?  I guess before-and-after comparison of
>>>the size(1) output would tell us.
>>>
>>>Those networking build problems appear to have already been fixed.
>>>
>>>In future, please quote the compiler error output in the changelog when
>>>sending build fixes or warning fixes, thanks.
>>>
>>>
>>>
>>
>>BEFORE:
>>
>>lkernel@akt$ size vmlinux
>>    text    data     bss     dec     hex filename
>>4318525  454484  462848 5235857  4fe491 vmlinux
>>
>>
>>AFTER:
>>
>>lkernel@akt$ size vmlinux
>>    text    data     bss     dec     hex filename
>>4323161  454484  462848 5240493  4ff6ad vmlinux
>>
>>which makes it +4636 = +0.11%
>>
>>I've got the details for */built-in.o if needed.
>>
> 
> 
> That seems to be a lot of increase.  Are you sure you had CONFIG_PROCFS=n
> in both cases?  If so, the patch must have added a lot of code which will
> never be executed?
> 
> 
> 
Well, the patches that are impacted by procfs being configured or not 
are #7 and #8. While the sizes I've sent you are before all patches 
applied vs after all patches applied :-(

So here are the "interesting sizes" - all with CONFIG_PROC_FS unset:

before patch 7:
    text    data     bss     dec     hex filename
4318757  454484  462848 5236089  4fe579 vmlinux

before patch 8:
    text    data     bss     dec     hex filename
4318853  454484  462848 5236185  4fe5d9 vmlinux    +96

after patch 8:
4319055  454484  462848 5236387  4fe6a3 vmlinux    +202

The higher difference after patch 8 is because I'm adding the new 
interface blocking_notifier_chain_cond_register() even if CONFIG_PROC_FS 
is not defined. This is to cover the case where msgmni is set through 
the sysctl() syscall (CONFIG_SYSCTL_SYSCALL).

Regards,
Nadia

Re: [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Andrew Morton]

On Mon, 11 Feb 2008 15:16:47 +0100 Nadia.Derbey@bull.net wrote:

> [PATCH 01/08]
> 
> This patch computes msg_ctlmni to make it scale with the amount of lowmem.
> msg_ctlmni is now set to make the message queues occupy 1/32 of the available
> lowmem.
> 
> Some cleaning has also been done for the MSGPOOL constant: the msgctl man page
> says it's not used, but it also defines it as a size in bytes (the code
> expresses it in Kbytes).
> 

Something's wrong here.  Running LTP's msgctl08 (specifically:
ltp-full-20070228) cripples the machine.  It's a 4-way 4GB x86_64.

http://userweb.kernel.org/~akpm/config-x.txt
http://userweb.kernel.org/~akpm/dmesg-x.txt

Normally msgctl08 will complete in a second or two.  With this patch I
don't know how long it will take to complete, and the machine is horridly
bogged down.  It does recover if you manage to kill msgctl08.  Feels like
a terrible memory shortage, but there's plenty of memory free and it isn't
swapping.

Re: [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Nadia Derbey]

Andrew Morton wrote:
> On Mon, 11 Feb 2008 15:16:47 +0100 Nadia.Derbey@bull.net wrote:
> 
> 
>>[PATCH 01/08]
>>
>>This patch computes msg_ctlmni to make it scale with the amount of lowmem.
>>msg_ctlmni is now set to make the message queues occupy 1/32 of the available
>>lowmem.
>>
>>Some cleaning has also been done for the MSGPOOL constant: the msgctl man page
>>says it's not used, but it also defines it as a size in bytes (the code
>>expresses it in Kbytes).
>>
> 
> 
> Something's wrong here.  Running LTP's msgctl08 (specifically:
> ltp-full-20070228) cripples the machine.  It's a 4-way 4GB x86_64.
> 
> http://userweb.kernel.org/~akpm/config-x.txt
> http://userweb.kernel.org/~akpm/dmesg-x.txt
> 
> Normally msgctl08 will complete in a second or two.  With this patch I
> don't know how long it will take to complete, and the machine is horridly
> bogged down.  It does recover if you manage to kill msgctl08.  Feels like
> a terrible memory shortage, but there's plenty of memory free and it isn't
> swapping.
> 
> 
> 

Before the patchset, msgctl08 used to be run with the old msgmni value: 
16. Now it is run with a much higher msgmni value (1746 in my case), 
since it scales to the memory size.
When I call "msgctl08 100000 16" it completes fast.

Doing the follwing on the ref kernel:
echo 1746 > /proc/sys/kernel/msgmni
msgctl08 100000 1746

makes th test block too :-(

Will check to see where the problem comes from.

Rgards,
Nadia

Re: [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Nadia Derbey]

Nadia Derbey wrote:
> Andrew Morton wrote:
> 
>> On Mon, 11 Feb 2008 15:16:47 +0100 Nadia.Derbey@bull.net wrote:
>>
>>
>>> [PATCH 01/08]
>>>
>>> This patch computes msg_ctlmni to make it scale with the amount of 
>>> lowmem.
>>> msg_ctlmni is now set to make the message queues occupy 1/32 of the 
>>> available
>>> lowmem.
>>>
>>> Some cleaning has also been done for the MSGPOOL constant: the msgctl 
>>> man page
>>> says it's not used, but it also defines it as a size in bytes (the code
>>> expresses it in Kbytes).
>>>
>>
>>
>> Something's wrong here.  Running LTP's msgctl08 (specifically:
>> ltp-full-20070228) cripples the machine.  It's a 4-way 4GB x86_64.
>>
>> http://userweb.kernel.org/~akpm/config-x.txt
>> http://userweb.kernel.org/~akpm/dmesg-x.txt
>>
>> Normally msgctl08 will complete in a second or two.  With this patch I
>> don't know how long it will take to complete, and the machine is horridly
>> bogged down.  It does recover if you manage to kill msgctl08.  Feels like
>> a terrible memory shortage, but there's plenty of memory free and it 
>> isn't
>> swapping.
>>
>>
>>
> 
> Before the patchset, msgctl08 used to be run with the old msgmni value: 
> 16. Now it is run with a much higher msgmni value (1746 in my case), 
> since it scales to the memory size.
> When I call "msgctl08 100000 16" it completes fast.
> 
> Doing the follwing on the ref kernel:
> echo 1746 > /proc/sys/kernel/msgmni
> msgctl08 100000 1746
> 
> makes th test block too :-(
> 
> Will check to see where the problem comes from.
> 

Well, actually, the test does not block, it only takes much much more 
time to be executed:

doing this:
date; ./msgctl08 100000 XXX; date


gives us the following results:
XXX           16   32   64   128   256   512   1024   1746
time(secs)     2    4    8    16    32    64    132    241

XXX is the # of msg queues to be created = # of processes to be forked 
as readers = # of processes to be created as writers
time is approximative since it is obtained by a "date" before and after.

XXX used to be 16 before the patchset  ---> 1st column
     --> 16 processes forked as reader
     --> + 16 processes forked as writers
     --> + 16 msg queues
XXX = 1746 (on my victim) after the patchset ---> last column
     --> 1746 reader processes forked
     --> + 1746 writers forked
     --> + 1746 msg queues created

The same tests on the ref kernel give approximatly the same results.

So if we don't want this longer time to appear as a regression, the LTP 
should be changed:
1) either by setting the result of get_max_msgqueues() as the MSGMNI 
constant (16) (that would be the best solution in my mind)
2) or by warning the tester that it may take a long time to finish.

There would be 3 tests impacted:

kernel/syscalls/ipc/msgctl/msgctl08.c
kernel/syscalls/ipc/msgctl/msgctl09.c
kernel/syscalls/ipc/msgget/msgget03.c

Cc-ing ltp mailing list ...

Regards,
Nadia

Re: [LTP] [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Subrata Modak]

> Nadia Derbey wrote:
> > Andrew Morton wrote:
> > 
> >> On Mon, 11 Feb 2008 15:16:47 +0100 Nadia.Derbey@bull.net wrote:
> >>
> >>
> >>> [PATCH 01/08]
> >>>
> >>> This patch computes msg_ctlmni to make it scale with the amount of 
> >>> lowmem.
> >>> msg_ctlmni is now set to make the message queues occupy 1/32 of the 
> >>> available
> >>> lowmem.
> >>>
> >>> Some cleaning has also been done for the MSGPOOL constant: the msgctl 
> >>> man page
> >>> says it's not used, but it also defines it as a size in bytes (the code
> >>> expresses it in Kbytes).
> >>>
> >>
> >>
> >> Something's wrong here.  Running LTP's msgctl08 (specifically:
> >> ltp-full-20070228) cripples the machine.  It's a 4-way 4GB x86_64.
> >>
> >> http://userweb.kernel.org/~akpm/config-x.txt
> >> http://userweb.kernel.org/~akpm/dmesg-x.txt
> >>
> >> Normally msgctl08 will complete in a second or two.  With this patch I
> >> don't know how long it will take to complete, and the machine is horridly
> >> bogged down.  It does recover if you manage to kill msgctl08.  Feels like
> >> a terrible memory shortage, but there's plenty of memory free and it 
> >> isn't
> >> swapping.
> >>
> >>
> >>
> > 
> > Before the patchset, msgctl08 used to be run with the old msgmni value: 
> > 16. Now it is run with a much higher msgmni value (1746 in my case), 
> > since it scales to the memory size.
> > When I call "msgctl08 100000 16" it completes fast.
> > 
> > Doing the follwing on the ref kernel:
> > echo 1746 > /proc/sys/kernel/msgmni
> > msgctl08 100000 1746
> > 
> > makes th test block too :-(
> > 
> > Will check to see where the problem comes from.
> > 
> 
> Well, actually, the test does not block, it only takes much much more 
> time to be executed:
> 
> doing this:
> date; ./msgctl08 100000 XXX; date
> 
> 
> gives us the following results:
> XXX           16   32   64   128   256   512   1024   1746
> time(secs)     2    4    8    16    32    64    132    241
> 
> XXX is the # of msg queues to be created = # of processes to be forked 
> as readers = # of processes to be created as writers
> time is approximative since it is obtained by a "date" before and after.
> 
> XXX used to be 16 before the patchset  ---> 1st column
>      --> 16 processes forked as reader
>      --> + 16 processes forked as writers
>      --> + 16 msg queues
> XXX = 1746 (on my victim) after the patchset ---> last column
>      --> 1746 reader processes forked
>      --> + 1746 writers forked
>      --> + 1746 msg queues created
> 
> The same tests on the ref kernel give approximatly the same results.
> 
> So if we don't want this longer time to appear as a regression, the LTP 
> should be changed:
> 1) either by setting the result of get_max_msgqueues() as the MSGMNI 
> constant (16) (that would be the best solution in my mind)
> 2) or by warning the tester that it may take a long time to finish.
> 
> There would be 3 tests impacted:
> 
> kernel/syscalls/ipc/msgctl/msgctl08.c
> kernel/syscalls/ipc/msgctl/msgctl09.c
> kernel/syscalls/ipc/msgget/msgget03.c

We will change the test case if need that be. Nadia, kindly send us the
patch set which will do the necessary changes.

Regards--
Subrata

> 
> Cc-ing ltp mailing list ...
> 
> Regards,
> Nadia
> 
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Ltp-list mailing list
> Ltp-list@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/ltp-list

Re: [LTP] [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Nadia Derbey]

[-- Attachment #1: Type: text/plain, Size: 3362 bytes --]

Subrata Modak wrote:
>>Nadia Derbey wrote:
>>
>>>Andrew Morton wrote:
>>>
>>>
>>>>On Mon, 11 Feb 2008 15:16:47 +0100 Nadia.Derbey@bull.net wrote:
>>>>
>>>>
>>>>
>>>>>[PATCH 01/08]
>>>>>
>>>>>This patch computes msg_ctlmni to make it scale with the amount of 
>>>>>lowmem.
>>>>>msg_ctlmni is now set to make the message queues occupy 1/32 of the 
>>>>>available
>>>>>lowmem.
>>>>>
>>>>>Some cleaning has also been done for the MSGPOOL constant: the msgctl 
>>>>>man page
>>>>>says it's not used, but it also defines it as a size in bytes (the code
>>>>>expresses it in Kbytes).
>>>>>
>>>>
>>>>
>>>>Something's wrong here.  Running LTP's msgctl08 (specifically:
>>>>ltp-full-20070228) cripples the machine.  It's a 4-way 4GB x86_64.
>>>>
>>>>http://userweb.kernel.org/~akpm/config-x.txt
>>>>http://userweb.kernel.org/~akpm/dmesg-x.txt
>>>>
>>>>Normally msgctl08 will complete in a second or two.  With this patch I
>>>>don't know how long it will take to complete, and the machine is horridly
>>>>bogged down.  It does recover if you manage to kill msgctl08.  Feels like
>>>>a terrible memory shortage, but there's plenty of memory free and it 
>>>>isn't
>>>>swapping.
>>>>
>>>>
>>>>
>>>
>>>Before the patchset, msgctl08 used to be run with the old msgmni value: 
>>>16. Now it is run with a much higher msgmni value (1746 in my case), 
>>>since it scales to the memory size.
>>>When I call "msgctl08 100000 16" it completes fast.
>>>
>>>Doing the follwing on the ref kernel:
>>>echo 1746 > /proc/sys/kernel/msgmni
>>>msgctl08 100000 1746
>>>
>>>makes th test block too :-(
>>>
>>>Will check to see where the problem comes from.
>>>
>>
>>Well, actually, the test does not block, it only takes much much more 
>>time to be executed:
>>
>>doing this:
>>date; ./msgctl08 100000 XXX; date
>>
>>
>>gives us the following results:
>>XXX           16   32   64   128   256   512   1024   1746
>>time(secs)     2    4    8    16    32    64    132    241
>>
>>XXX is the # of msg queues to be created = # of processes to be forked 
>>as readers = # of processes to be created as writers
>>time is approximative since it is obtained by a "date" before and after.
>>
>>XXX used to be 16 before the patchset  ---> 1st column
>>     --> 16 processes forked as reader
>>     --> + 16 processes forked as writers
>>     --> + 16 msg queues
>>XXX = 1746 (on my victim) after the patchset ---> last column
>>     --> 1746 reader processes forked
>>     --> + 1746 writers forked
>>     --> + 1746 msg queues created
>>
>>The same tests on the ref kernel give approximatly the same results.
>>
>>So if we don't want this longer time to appear as a regression, the LTP 
>>should be changed:
>>1) either by setting the result of get_max_msgqueues() as the MSGMNI 
>>constant (16) (that would be the best solution in my mind)
>>2) or by warning the tester that it may take a long time to finish.
>>
>>There would be 3 tests impacted:
>>
>>kernel/syscalls/ipc/msgctl/msgctl08.c
>>kernel/syscalls/ipc/msgctl/msgctl09.c
>>kernel/syscalls/ipc/msgget/msgget03.c
> 
> 
> We will change the test case if need that be. Nadia, kindly send us the
> patch set which will do the necessary changes.
> 
> Regards--
> Subrata
> 

Subrata,

You'll find the patch in attachment.
FYI I didn't change msgget03.c since we need to get the actual max value 
in order to generate an error.

Regards,
Nadia


[-- Attachment #2: ipc_ltp_full_20080131.patch --]
[-- Type: text/x-patch, Size: 6296 bytes --]

Since msgmni now scales to the memory size, it may reach big values.
To avoid forking 2*msgmni processes and create msgmni msg queues, do not take
msgmni from procfs anymore.
Just define it as 16 (which is the MSGMNI constant value in linux/msg.h)

Also fixed the Makefiles in ipc/lib and ipc/msgctl: there was no dependency
on the lib/ipc*.h header files.

Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 testcases/kernel/syscalls/ipc/lib/Makefile      |    3 +++
 testcases/kernel/syscalls/ipc/lib/ipcmsg.h      |    2 ++
 testcases/kernel/syscalls/ipc/msgctl/Makefile   |    3 +++
 testcases/kernel/syscalls/ipc/msgctl/msgctl08.c |   23 ++---------------------
 testcases/kernel/syscalls/ipc/msgctl/msgctl09.c |   24 ++----------------------
 5 files changed, 12 insertions(+), 43 deletions(-)

Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c	2006-02-11 05:46:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c	2008-02-19 18:45:27.000000000 +0100
@@ -50,6 +50,7 @@
 #include <sys/msg.h>
 #include "test.h"
 #include "usctest.h"
+#include "ipcmsg.h"
 
 void setup();
 void cleanup();
@@ -479,26 +480,6 @@ static int get_used_msgqueues()
         return used_queues;
 }
 
-/** Get the max number of message queues allowed on system */
-static int get_max_msgqueues()
-{
-        FILE *f;
-        char buff[BUFSIZE];
-
-        /* Get the max number of message queues allowed on system */
-        f = fopen("/proc/sys/kernel/msgmni", "r");
-        if (!f){
-                tst_resm(TBROK,"Could not open /proc/sys/kernel/msgmni");
-                tst_exit();
-        }
-        if (!fgets(buff, BUFSIZE, f)) {
-                tst_resm(TBROK,"Could not read /proc/sys/kernel/msgmni");
-                tst_exit();
-        }
-        fclose(f);
-        return atoi(buff);
-}
-
 /***************************************************************
  * setup() - performs all ONE TIME setup for this test.
  *****************************************************************/
@@ -520,7 +501,7 @@ setup()
 	 */
         TEST_PAUSE;
 
-        MSGMNI = get_max_msgqueues() - get_used_msgqueues();
+        MSGMNI = MAX_MSGQUEUES - get_used_msgqueues();
 	if (MSGMNI <= 0){
 		tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
 		cleanup(); 
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c	2006-02-11 05:46:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c	2008-02-19 18:46:44.000000000 +0100
@@ -49,6 +49,7 @@
 #include <unistd.h>
 #include "test.h"
 #include "usctest.h"
+#include "ipcmsg.h"
 
 #define MAXNREPS	1000
 #ifndef CONFIG_COLDFIRE
@@ -649,26 +650,6 @@ static int get_used_msgqueues()
         return used_queues;
 }
 
-/** Get the max number of message queues allowed on system */
-static int get_max_msgqueues()
-{
-        FILE *f;
-        char buff[BUFSIZE];
-
-        /* Get the max number of message queues allowed on system */
-        f = fopen("/proc/sys/kernel/msgmni", "r");
-        if (!f){
-                tst_resm(TBROK,"Could not open /proc/sys/kernel/msgmni");
-                tst_exit();
-        }
-        if (!fgets(buff, BUFSIZE, f)) {
-                tst_resm(TBROK,"Could not read /proc/sys/kernel/msgmni");
-                tst_exit();
-        }
-        fclose(f);
-        return atoi(buff);
-}
-
 /***************************************************************
  * setup() - performs all ONE TIME setup for this test.
  *****************************************************************/
@@ -689,12 +670,11 @@ setup()
          */
         TEST_PAUSE;
 
-        MSGMNI = get_max_msgqueues() - get_used_msgqueues();
+        MSGMNI = MAX_MSGQUEUES - get_used_msgqueues();
         if (MSGMNI <= 0){
                 tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
                 cleanup();
         }
-
 }
 
 
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/ipcmsg.h
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/lib/ipcmsg.h	2005-12-22 21:18:23.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/ipcmsg.h	2008-02-19 18:51:38.000000000 +0100
@@ -41,6 +41,8 @@ void setup(void);
 #define MSGSIZE	1024		/* a resonable size for a message */
 #define MSGTYPE 1		/* a type ID for a message */
 
+#define MAX_MSGQUEUES	16	/* MSGMNI as defined in linux/msg.h */
+
 typedef struct mbuf {		/* a generic message structure */
 	long mtype;
 	char mtext[MSGSIZE + 1];  /* add 1 here so the message can be 1024   */
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/Makefile
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/lib/Makefile	2006-08-21 08:58:39.000000000 +0200
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/Makefile	2008-02-19 18:50:19.000000000 +0100
@@ -19,6 +19,7 @@
 SRCS   = libipc.c
 OBJS   = $(SRCS:.c=.o)
 LIBIPC = ../libipc.a
+LIBIPC_HEADERS	= ipcmsg.h ipcsem.h
 
 CFLAGS += -I../../../../../include -Wall
 
@@ -27,6 +28,8 @@ all: $(LIBIPC)
 $(LIBIPC): $(OBJS)
 	$(AR) -rc $@ $(OBJS)
 
+$(OBJS): $(LIBIPC_HEADERS)
+
 install:
 
 clean:
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/Makefile
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgctl/Makefile	2006-08-23 07:46:27.000000000 +0200
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/Makefile	2008-02-19 19:02:26.000000000 +0100
@@ -18,12 +18,15 @@
 
 CFLAGS += -I../lib -I../../../../../include -Wall
 LDLIBS += -L../../../../../lib -lltp -L.. -lipc
+LIBIPC_HEADERS	= ../lib/ipcmsg.h
 
 SRCS    = $(wildcard *.c)
 TARGETS = $(patsubst %.c,%,$(SRCS))
 
 all: $(TARGETS)
 
+$(TARGETS): $(LIBIPC_HEADERS)
+
 install:
 	@set -e; for i in $(TARGETS); do ln -f $$i ../../../../bin/$$i ; done
 

Re: [LTP] [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Matt Helsley]

On Tue, 2008-02-19 at 18:16 +0100, Nadia Derbey wrote:

<snip>

> +#define MAX_MSGQUEUES  16      /* MSGMNI as defined in linux/msg.h */
> +

It's not quite the maximum anymore, is it? More like the minumum
maximum ;). A better name might better document what the test is
actually trying to do.

One question I have is whether the unpatched test is still valuable.
Based on my limited knowledge of the test I suspect it's still a correct
test of message queues. If so, perhaps renaming the old test (so it's
not confused with a performance regression) and adding your patched
version is best?

<snip>

Cheers,
	-Matt Helsley

Re: [LTP] [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Subrata Modak]

> Subrata Modak wrote:
> >>Nadia Derbey wrote:
> >>
> >>>Andrew Morton wrote:
> >>>
> >>>
> >>>>On Mon, 11 Feb 2008 15:16:47 +0100 Nadia.Derbey@bull.net wrote:
> >>>>
> >>>>
> >>>>
> >>>>>[PATCH 01/08]
> >>>>>
> >>>>>This patch computes msg_ctlmni to make it scale with the amount of 
> >>>>>lowmem.
> >>>>>msg_ctlmni is now set to make the message queues occupy 1/32 of the 
> >>>>>available
> >>>>>lowmem.
> >>>>>
> >>>>>Some cleaning has also been done for the MSGPOOL constant: the msgctl 
> >>>>>man page
> >>>>>says it's not used, but it also defines it as a size in bytes (the code
> >>>>>expresses it in Kbytes).
> >>>>>
> >>>>
> >>>>
> >>>>Something's wrong here.  Running LTP's msgctl08 (specifically:
> >>>>ltp-full-20070228) cripples the machine.  It's a 4-way 4GB x86_64.
> >>>>
> >>>>http://userweb.kernel.org/~akpm/config-x.txt
> >>>>http://userweb.kernel.org/~akpm/dmesg-x.txt
> >>>>
> >>>>Normally msgctl08 will complete in a second or two.  With this patch I
> >>>>don't know how long it will take to complete, and the machine is horridly
> >>>>bogged down.  It does recover if you manage to kill msgctl08.  Feels like
> >>>>a terrible memory shortage, but there's plenty of memory free and it 
> >>>>isn't
> >>>>swapping.
> >>>>
> >>>>
> >>>>
> >>>
> >>>Before the patchset, msgctl08 used to be run with the old msgmni value: 
> >>>16. Now it is run with a much higher msgmni value (1746 in my case), 
> >>>since it scales to the memory size.
> >>>When I call "msgctl08 100000 16" it completes fast.
> >>>
> >>>Doing the follwing on the ref kernel:
> >>>echo 1746 > /proc/sys/kernel/msgmni
> >>>msgctl08 100000 1746
> >>>
> >>>makes th test block too :-(
> >>>
> >>>Will check to see where the problem comes from.
> >>>
> >>
> >>Well, actually, the test does not block, it only takes much much more 
> >>time to be executed:
> >>
> >>doing this:
> >>date; ./msgctl08 100000 XXX; date
> >>
> >>
> >>gives us the following results:
> >>XXX           16   32   64   128   256   512   1024   1746
> >>time(secs)     2    4    8    16    32    64    132    241
> >>
> >>XXX is the # of msg queues to be created = # of processes to be forked 
> >>as readers = # of processes to be created as writers
> >>time is approximative since it is obtained by a "date" before and after.
> >>
> >>XXX used to be 16 before the patchset  ---> 1st column
> >>     --> 16 processes forked as reader
> >>     --> + 16 processes forked as writers
> >>     --> + 16 msg queues
> >>XXX = 1746 (on my victim) after the patchset ---> last column
> >>     --> 1746 reader processes forked
> >>     --> + 1746 writers forked
> >>     --> + 1746 msg queues created
> >>
> >>The same tests on the ref kernel give approximatly the same results.
> >>
> >>So if we don't want this longer time to appear as a regression, the LTP 
> >>should be changed:
> >>1) either by setting the result of get_max_msgqueues() as the MSGMNI 
> >>constant (16) (that would be the best solution in my mind)
> >>2) or by warning the tester that it may take a long time to finish.
> >>
> >>There would be 3 tests impacted:
> >>
> >>kernel/syscalls/ipc/msgctl/msgctl08.c
> >>kernel/syscalls/ipc/msgctl/msgctl09.c
> >>kernel/syscalls/ipc/msgget/msgget03.c
> > 
> > 
> > We will change the test case if need that be. Nadia, kindly send us the
> > patch set which will do the necessary changes.
> > 
> > Regards--
> > Subrata
> > 
> 
> Subrata,
> 
> You'll find the patch in attachment.
> FYI I didn't change msgget03.c since we need to get the actual max value 
> in order to generate an error.

Thanks. The same has been Merged.

Regards--
Subrata

> 
> Regards,
> Nadia
> 

Re: [LTP] [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Nadia Derbey]

Matt Helsley wrote:
> On Tue, 2008-02-19 at 18:16 +0100, Nadia Derbey wrote:
> 
> <snip>
> 
>>+#define MAX_MSGQUEUES  16      /* MSGMNI as defined in linux/msg.h */
>>+
> 
> 
> It's not quite the maximum anymore, is it? More like the minumum
> maximum ;). A better name might better document what the test is
> actually trying to do.
> 
> One question I have is whether the unpatched test is still valuable.
> Based on my limited knowledge of the test I suspect it's still a correct
> test of message queues. If so, perhaps renaming the old test (so it's
> not confused with a performance regression) and adding your patched
> version is best?
> 

Yes, you're completely right.

I'll resend a patch today.

Regards,
Nadia

Re: [LTP] [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Nadia Derbey]

Matt Helsley wrote:
> On Tue, 2008-02-19 at 18:16 +0100, Nadia Derbey wrote:
> 
> <snip>
> 
>>+#define MAX_MSGQUEUES  16      /* MSGMNI as defined in linux/msg.h */
>>+
> 
> 
> It's not quite the maximum anymore, is it? More like the minumum
> maximum ;). A better name might better document what the test is
> actually trying to do.
> 
> One question I have is whether the unpatched test is still valuable.
> Based on my limited knowledge of the test I suspect it's still a correct
> test of message queues. If so, perhaps renaming the old test (so it's
> not confused with a performance regression) and adding your patched
> version is best?
> 

So, here's the new patch based on Matt's points.

Subrata, it has to be applied on top of the original ltp-full-20080131. 
Please tell me if you'd prefer one based on the merged version you've 
got (i.e. with my Tuesday patch applied).

Regards,
Nadia

Re: [LTP] [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Nadia Derbey]

[-- Attachment #1: Type: text/plain, Size: 1006 bytes --]

Nadia Derbey wrote:
> Matt Helsley wrote:
> 
>> On Tue, 2008-02-19 at 18:16 +0100, Nadia Derbey wrote:
>>
>> <snip>
>>
>>> +#define MAX_MSGQUEUES  16      /* MSGMNI as defined in linux/msg.h */
>>> +
>>
>>
>>
>> It's not quite the maximum anymore, is it? More like the minumum
>> maximum ;). A better name might better document what the test is
>> actually trying to do.
>>
>> One question I have is whether the unpatched test is still valuable.
>> Based on my limited knowledge of the test I suspect it's still a correct
>> test of message queues. If so, perhaps renaming the old test (so it's
>> not confused with a performance regression) and adding your patched
>> version is best?
>>
> 
> So, here's the new patch based on Matt's points.
> 
> Subrata, it has to be applied on top of the original ltp-full-20080131. 
> Please tell me if you'd prefer one based on the merged version you've 
> got (i.e. with my Tuesday patch applied).
> 

Forgot the patch, sorry for that (thx Andrew).

Regards,
Nadia


[-- Attachment #2: ipc_ltp_full_20080131.patch --]
[-- Type: text/x-patch, Size: 40863 bytes --]

Since msgmni now scales to the memory size, it may reach big values.
To avoid forking 2*msgmni processes and create msgmni msg queues, take the min
between the procfs value and MSGMNI (as found in linux/msg.h).

Also fixed the Makefiles in ipc/lib and ipc/msgctl: there was no dependency
on the lib/ipc*.h header files.

Also integrated the following in libipc.a:
  . get_max_msgqueues()
  . get_used_msgqueues()

Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 testcases/kernel/syscalls/ipc/lib/Makefile      |    3 
 testcases/kernel/syscalls/ipc/lib/ipcmsg.h      |    7 
 testcases/kernel/syscalls/ipc/lib/libipc.c      |   54 +
 testcases/kernel/syscalls/ipc/msgctl/Makefile   |    3 
 testcases/kernel/syscalls/ipc/msgctl/msgctl08.c |   63 --
 testcases/kernel/syscalls/ipc/msgctl/msgctl09.c |   62 --
 testcases/kernel/syscalls/ipc/msgctl/msgctl10.c |  527 ++++++++++++++++++
 testcases/kernel/syscalls/ipc/msgctl/msgctl11.c |  696 ++++++++++++++++++++++++
 testcases/kernel/syscalls/ipc/msgget/Makefile   |    3 
 testcases/kernel/syscalls/ipc/msgget/msgget03.c |   22 
 10 files changed, 1326 insertions(+), 114 deletions(-)

Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/ipcmsg.h
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/lib/ipcmsg.h	2008-02-21 10:45:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/ipcmsg.h	2008-02-21 14:09:00.000000000 +0100
@@ -41,6 +41,10 @@ void setup(void);
 #define MSGSIZE	1024		/* a resonable size for a message */
 #define MSGTYPE 1		/* a type ID for a message */
 
+#define NR_MSGQUEUES	16	/* MSGMNI as defined in linux/msg.h */
+
+#define min(a, b)	(((a) < (b)) ? (a) : (b))
+
 typedef struct mbuf {		/* a generic message structure */
 	long mtype;
 	char mtext[MSGSIZE + 1];  /* add 1 here so the message can be 1024   */
@@ -59,4 +63,7 @@ void rm_queue(int);
 int getipckey();
 int getuserid(char *);
 
+int get_max_msgqueues(void);
+int get_used_msgqueues(void);
+
 #endif /* ipcmsg.h */
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/libipc.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/lib/libipc.c	2008-02-21 10:45:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/libipc.c	2008-02-21 13:35:41.000000000 +0100
@@ -201,3 +201,57 @@ rm_shm(int shm_id)
 		tst_resm(TINFO, "id = %d", shm_id);
 	}
 }
+
+#define BUFSIZE 512
+
+/*
+ * Get the number of message queues already in use
+ */
+int
+get_used_msgqueues()
+{
+	FILE *f;
+	int used_queues;
+	char buff[BUFSIZE];
+
+	f = popen("ipcs -q", "r");
+	if (!f) {
+		tst_resm(TBROK, "Could not run 'ipcs' to calculate used "
+			"message queues");
+		tst_exit();
+	}
+	/* FIXME: Start at -4 because ipcs prints four lines of header */
+	for (used_queues = -4; fgets(buff, BUFSIZE, f); used_queues++)
+		;
+	pclose(f);
+	if (used_queues < 0) {
+		tst_resm(TBROK, "Could not read output of 'ipcs' to "
+			"calculate used message queues");
+		tst_exit();
+	}
+	return used_queues;
+}
+
+/*
+ * Get the max number of message queues allowed on system
+ */
+int
+get_max_msgqueues()
+{
+	FILE *f;
+	char buff[BUFSIZE];
+
+	/* Get the max number of message queues allowed on system */
+	f = fopen("/proc/sys/kernel/msgmni", "r");
+	if (!f) {
+		tst_resm(TBROK, "Could not open /proc/sys/kernel/msgmni");
+		return -1;
+	}
+	if (!fgets(buff, BUFSIZE, f)) {
+		fclose(f);
+		tst_resm(TBROK, "Could not read /proc/sys/kernel/msgmni");
+		return -1;
+	}
+	fclose(f);
+	return atoi(buff);
+}
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c	2008-02-21 10:45:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c	2008-02-21 14:18:23.000000000 +0100
@@ -50,6 +50,7 @@
 #include <sys/msg.h>
 #include "test.h"
 #include "usctest.h"
+#include "ipcmsg.h"
 
 void setup();
 void cleanup();
@@ -454,57 +455,14 @@ sig_handler()
 {
 }
 
-#define BUFSIZE 512
-
-/** Get the number of message queues already in use */
-static int get_used_msgqueues()
-{
-        FILE *f;
-        int used_queues;
-        char buff[BUFSIZE];
-
-        f = popen("ipcs -q", "r");
-        if (!f) {
-                tst_resm(TBROK,"Could not run 'ipcs' to calculate used message queues");
-                tst_exit();
-        }
-        /* FIXME: Start at -4 because ipcs prints four lines of header */
-        for (used_queues = -4; fgets(buff, BUFSIZE, f); used_queues++)
-                ;
-        pclose(f);
-        if (used_queues < 0) {
-                tst_resm(TBROK,"Could not read output of 'ipcs' to calculate used message queues");
-                tst_exit();
-        }
-        return used_queues;
-}
-
-/** Get the max number of message queues allowed on system */
-static int get_max_msgqueues()
-{
-        FILE *f;
-        char buff[BUFSIZE];
-
-        /* Get the max number of message queues allowed on system */
-        f = fopen("/proc/sys/kernel/msgmni", "r");
-        if (!f){
-                tst_resm(TBROK,"Could not open /proc/sys/kernel/msgmni");
-                tst_exit();
-        }
-        if (!fgets(buff, BUFSIZE, f)) {
-                tst_resm(TBROK,"Could not read /proc/sys/kernel/msgmni");
-                tst_exit();
-        }
-        fclose(f);
-        return atoi(buff);
-}
-
 /***************************************************************
  * setup() - performs all ONE TIME setup for this test.
  *****************************************************************/
 void
 setup()
 {
+	int nr_msgqs;
+
 	tst_tmpdir();
  
         /* You will want to enable some signal handling so you can capture
@@ -520,11 +478,22 @@ setup()
 	 */
         TEST_PAUSE;
 
-        MSGMNI = get_max_msgqueues() - get_used_msgqueues();
-	if (MSGMNI <= 0){
+	nr_msgqs = get_max_msgqueues();
+	if (nr_msgqs < 0)
+		cleanup();
+
+	nr_msgqs -= get_used_msgqueues();
+	if (nr_msgqs <= 0){
 		tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
 		cleanup(); 
 	}	
+
+	/*
+	 * Since msgmni scales to the memory size, it may reach huge values
+	 * that are not necessary for this test.
+	 * That's why we define NR_MSGQUEUES as a high boundary for it.
+	 */
+	MSGMNI = min(nr_msgqs, NR_MSGQUEUES);
 }
 
 
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c	2008-02-21 10:45:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c	2008-02-21 13:38:42.000000000 +0100
@@ -49,6 +49,7 @@
 #include <unistd.h>
 #include "test.h"
 #include "usctest.h"
+#include "ipcmsg.h"
 
 #define MAXNREPS	1000
 #ifndef CONFIG_COLDFIRE
@@ -624,57 +625,14 @@ term(int sig)
 	}
 }
 
-#define BUFSIZE 512
-
-/** Get the number of message queues already in use */
-static int get_used_msgqueues()
-{
-        FILE *f;
-        int used_queues;
-        char buff[BUFSIZE];
-
-        f = popen("ipcs -q", "r");
-        if (!f) {
-                tst_resm(TBROK,"Could not run 'ipcs' to calculate used message queues");
-                tst_exit();
-        }
-        /* FIXME: Start at -4 because ipcs prints four lines of header */
-        for (used_queues = -4; fgets(buff, BUFSIZE, f); used_queues++)
-                ;
-        pclose(f);
-        if (used_queues < 0) {
-                tst_resm(TBROK,"Could not read output of 'ipcs' to calculate used message queues");
-                tst_exit();
-        }
-        return used_queues;
-}
-
-/** Get the max number of message queues allowed on system */
-static int get_max_msgqueues()
-{
-        FILE *f;
-        char buff[BUFSIZE];
-
-        /* Get the max number of message queues allowed on system */
-        f = fopen("/proc/sys/kernel/msgmni", "r");
-        if (!f){
-                tst_resm(TBROK,"Could not open /proc/sys/kernel/msgmni");
-                tst_exit();
-        }
-        if (!fgets(buff, BUFSIZE, f)) {
-                tst_resm(TBROK,"Could not read /proc/sys/kernel/msgmni");
-                tst_exit();
-        }
-        fclose(f);
-        return atoi(buff);
-}
-
 /***************************************************************
  * setup() - performs all ONE TIME setup for this test.
  *****************************************************************/
 void
 setup()
 {
+	int nr_msgqs;
+
 	tst_tmpdir();
         /* You will want to enable some signal handling so you can capture
          * unexpected signals like SIGSEGV.
@@ -689,12 +647,22 @@ setup()
          */
         TEST_PAUSE;
 
-        MSGMNI = get_max_msgqueues() - get_used_msgqueues();
-        if (MSGMNI <= 0){
+        nr_msgqs = get_max_msgqueues();
+	if (nr_msgqs < 0)
+		cleanup();
+
+	nr_msgqs -= get_used_msgqueues();
+	if (nr_msgqs <= 0) {
                 tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
                 cleanup();
         }
 
+	/*
+	 * Since msgmni scales to the memory size, it may reach huge values
+	 * that are not necessary for this test.
+	 * That's why we define NR_MSGQUEUES as a high boundary for it.
+	 */
+	MSGMNI = min(nr_msgqs, NR_MSGQUEUES);
 }
 
 
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgget/msgget03.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgget/msgget03.c	2008-02-21 10:45:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgget/msgget03.c	2008-02-21 12:10:25.000000000 +0100
@@ -68,8 +68,6 @@ int exp_enos[] = {ENOSPC, 0};	/* 0 termi
 int *msg_q_arr = NULL;		/* hold the id's that we create */
 int num_queue = 0;		/* count the queues created */
 
-static int get_max_msgqueues();
-
 int main(int ac, char **av)
 {
 	int lc;				/* loop counter */
@@ -121,24 +119,6 @@ int main(int ac, char **av)
 	return(0);
 }
 
-/** Get the max number of message queues allowed on system */
-int get_max_msgqueues()
-{
-        FILE *f;
-        char buff[512];
-
-        /* Get the max number of message queues allowed on system */
-        f = fopen("/proc/sys/kernel/msgmni", "r");
-        if (!f){
-                tst_brkm(TBROK, cleanup, "Could not open /proc/sys/kernel/msgmni");
-        }
-        if (!fgets(buff, 512, f)) {
-                tst_brkm(TBROK, cleanup, "Could not read /proc/sys/kernel/msgmni");
-        }
-        fclose(f);
-        return atoi(buff);
-}
-
 /*
  * setup() - performs all the ONE TIME setup for this test.
  */
@@ -166,6 +146,8 @@ setup(void)
 	msgkey = getipckey();
 
 	maxmsgs = get_max_msgqueues();
+	if (maxmsgs < 0)
+		tst_brkm(TBROK, cleanup, "");
 
 	msg_q_arr = (int *)calloc(maxmsgs, sizeof (int));
 	if (msg_q_arr == NULL) {
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/Makefile
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/lib/Makefile	2008-02-21 10:45:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/Makefile	2008-02-21 12:14:39.000000000 +0100
@@ -19,6 +19,7 @@
 SRCS   = libipc.c
 OBJS   = $(SRCS:.c=.o)
 LIBIPC = ../libipc.a
+LIBIPC_HEADERS = ipcmsg.h ipcsem.h
 
 CFLAGS += -I../../../../../include -Wall
 
@@ -27,6 +28,8 @@ all: $(LIBIPC)
 $(LIBIPC): $(OBJS)
 	$(AR) -rc $@ $(OBJS)
 
+$(OBJS): $(LIBIPC_HEADERS)
+
 install:
 
 clean:
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/Makefile
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgctl/Makefile	2008-02-21 10:45:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/Makefile	2008-02-21 14:11:46.000000000 +0100
@@ -18,12 +18,15 @@
 
 CFLAGS += -I../lib -I../../../../../include -Wall
 LDLIBS += -L../../../../../lib -lltp -L.. -lipc
+LIBIPC_HEADERS	= ../lib/ipcmsg.h
 
 SRCS    = $(wildcard *.c)
 TARGETS = $(patsubst %.c,%,$(SRCS))
 
 all: $(TARGETS)
 
+$(TARGETS): $(LIBIPC_HEADERS)
+
 install:
 	@set -e; for i in $(TARGETS); do ln -f $$i ../../../../bin/$$i ; done
 
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl10.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl10.c	2008-02-21 13:56:10.000000000 +0100
@@ -0,0 +1,527 @@
+/*
+ *
+ *   Copyright (c) International Business Machines  Corp., 2002
+ *
+ *   This program is free software;  you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation; either version 2 of the License, or
+ *   (at your option) any later version.
+ *
+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ *   the GNU General Public License for more details.
+ *
+ *   You should have received a copy of the GNU General Public License
+ *   along with this program;  if not, write to the Free Software
+ *   Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+/* 06/30/2001	Port to Linux	nsharoff@us.ibm.com */
+/* 11/06/2002   Port to LTP     dbarrera@us.ibm.com */
+
+/*
+ * NAME
+ *	msgctl10
+ *
+ * CALLS
+ *	msgget(2) msgctl(2)
+ *
+ * ALGORITHM
+ *	Get and manipulate a message queue.
+ *	Same as msgctl08 but gets the actual msgmni value under procfs.
+ *
+ * RESTRICTIONS
+ *
+ */
+
+#define _XOPEN_SOURCE 500
+#include <signal.h>
+#include <errno.h>
+#include <string.h>
+#include <fcntl.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <values.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <sys/stat.h>
+#include <sys/ipc.h>
+#include <sys/msg.h>
+#include "test.h"
+#include "usctest.h"
+#include "ipcmsg.h"
+
+void setup();
+void cleanup();
+/*
+ *  *  *  * These globals must be defined in the test.
+ *   *   *   */
+
+
+char *TCID="msgctl10";           /* Test program identifier.    */
+int TST_TOTAL=1;                /* Total number of test cases. */
+extern int Tst_count;           /* Test Case counter for tst_* routines */
+
+int exp_enos[]={0};     /* List must end with 0 */
+
+#ifndef CONFIG_COLDFIRE
+#define MAXNPROCS	1000000  /* This value is set to an arbitrary high limit. */
+#else
+#define MAXNPROCS	 100000   /* Coldfire can't deal with 1000000 */
+#endif
+#define MAXNREPS	100000
+#define FAIL		1
+#define PASS		0
+
+key_t	keyarray[MAXNPROCS];
+
+struct {
+	long	type;
+	struct {
+		char	len;
+		char	pbytes[99];
+		} data;
+	} buffer;
+
+int	pidarray[MAXNPROCS];
+int tid;
+int MSGMNI,nprocs, nreps;
+int procstat;
+int dotest(key_t key, int child_process);
+int doreader(int id, long key, int child);
+int dowriter(int id,long key, int child);
+int fill_buffer(register char *buf, char val, register int size);
+int verify(register char *buf,char val, register int size,int child);
+void sig_handler();             /* signal catching function */
+int mykid;
+#ifdef UCLINUX
+static char *argv0;
+
+void do_child_1_uclinux();
+static key_t key_uclinux;
+static int i_uclinux;
+
+void do_child_2_uclinux();
+static int id_uclinux;
+static int child_process_uclinux;
+#endif
+
+/*-----------------------------------------------------------------*/
+int main(argc, argv)
+int	argc;
+char	*argv[];
+{
+	register int i, j, ok, pid;
+	int count, status;
+	struct sigaction act;
+
+#ifdef UCLINUX
+	char *msg;			/* message returned from parse_opts */
+
+	argv0 = argv[0];
+
+	/* parse standard options */
+	if ((msg = parse_opts(argc, argv, (option_t *)NULL, NULL)) != (char *)NULL)
+	{
+		tst_brkm(TBROK, cleanup, "OPTION PARSING ERROR - %s", msg);
+	}
+
+	maybe_run_child(&do_child_1_uclinux, "ndd", 1, &key_uclinux, &i_uclinux);
+	maybe_run_child(&do_child_2_uclinux, "nddd", 2, &id_uclinux, &key_uclinux,
+			&child_process_uclinux);
+#endif
+
+	setup();
+
+	if (argc == 1 )
+	{
+		/* Set default parameters */
+		nreps = MAXNREPS;
+		nprocs = MSGMNI;
+	}
+	else if (argc == 3 )
+	{
+		if ( atoi(argv[1]) > MAXNREPS )
+		{
+			tst_resm(TCONF,"Requested number of iterations too large, setting to Max. of %d", MAXNREPS);
+			nreps = MAXNREPS;
+		}
+		else
+		{
+			nreps = atoi(argv[1]);
+		}
+		if (atoi(argv[2]) > MSGMNI )
+		{
+			tst_resm(TCONF,"Requested number of processes too large, setting to Max. of %d", MSGMNI);
+			nprocs = MSGMNI;
+		}
+		else
+		{
+			nprocs = atoi(argv[2]);
+		}
+	}
+	else
+	{
+		tst_resm(TCONF," Usage: %s [ number of iterations  number of processes ]", argv[0]);
+		tst_exit();
+	}
+
+	srand(getpid());
+	tid = -1;
+
+	/* Setup signal handleing routine */
+	memset(&act, 0, sizeof(act));
+	act.sa_handler = sig_handler;
+	sigemptyset(&act.sa_mask);
+	sigaddset(&act.sa_mask, SIGTERM);
+	if (sigaction(SIGTERM, &act, NULL) < 0)
+	{
+		tst_resm(TFAIL, "Sigset SIGTERM failed");
+		tst_exit();
+	}
+	/* Set up array of unique keys for use in allocating message
+	 * queues
+	 */
+	for (i = 0; i < nprocs; i++)
+	{
+		ok = 1;
+		do
+		{
+			/* Get random key */
+			keyarray[i] = (key_t)rand();
+			/* Make sure key is unique and not private */
+			if (keyarray[i] == IPC_PRIVATE)
+			{
+				ok = 0;
+				continue;
+			}
+			for (j = 0; j < i; j++)
+			{
+				if (keyarray[j] == keyarray[i])
+				{
+					ok = 0;
+					break;
+				}
+				ok = 1;
+			}
+		} while (ok == 0);
+	}
+
+	/* Fork a number of processes, each of which will
+	 * create a message queue with one reader/writer
+	 * pair which will read and write a number (iterations)
+	 * of random length messages with specific values.
+	 */
+
+	for (i = 0; i <  nprocs; i++)
+	{
+		fflush(stdout);
+		if ((pid = FORK_OR_VFORK()) < 0)
+		{
+			tst_resm(TFAIL, "\tFork failed (may be OK if under stress)");
+			tst_exit();
+		}
+		/* Child does this */
+		if (pid == 0)
+		{
+#ifdef UCLINUX
+			if (self_exec(argv[0], "ndd", 1, keyarray[i], i) < 0)
+			{
+				tst_resm(TFAIL, "\tself_exec failed");
+				tst_exit();
+			}
+#else
+			procstat = 1;
+			exit( dotest(keyarray[i], i) );
+#endif
+		}
+		pidarray[i] = pid;
+	}
+
+	count = 0;
+	while(1)
+	{
+		if (( wait(&status)) > 0)
+		{
+			if (status>>8 != 0 )
+			{
+				tst_resm(TFAIL, "Child exit status = %d", status>>8);
+				tst_exit();
+			}
+			count++;
+		}
+		else
+		{
+			if (errno != EINTR)
+			{
+				break;
+			}
+#ifdef DEBUG
+			tst_resm(TINFO,"Signal detected during wait");
+#endif
+		}
+	}
+	/* Make sure proper number of children exited */
+	if (count != nprocs)
+	{
+		tst_resm(TFAIL, "Wrong number of children exited, Saw %d, Expected %d", count, nprocs);
+		tst_exit();
+	}
+
+	tst_resm(TPASS,"msgctl10 ran successfully!");
+
+	cleanup();
+	return (0);
+
+}
+/*--------------------------------------------------------------------*/
+
+#ifdef UCLINUX
+void
+do_child_1_uclinux()
+{
+	procstat = 1;
+	exit(dotest(key_uclinux, i_uclinux));
+}
+
+void
+do_child_2_uclinux()
+{
+	exit(doreader(id_uclinux, key_uclinux % 255, child_process_uclinux));
+}
+#endif
+
+int dotest(key, child_process)
+key_t 	key;
+int	child_process;
+{
+	int id, pid;
+
+	sighold(SIGTERM);
+	TEST(msgget(key, IPC_CREAT | S_IRUSR | S_IWUSR));
+	if (TEST_RETURN < 0)
+	{
+		tst_resm(TFAIL, "Msgget error in child %d, errno = %d", child_process, TEST_ERRNO);
+		tst_exit();
+	}
+	tid = id = TEST_RETURN;
+	sigrelse(SIGTERM);
+
+	fflush(stdout);
+	if ((pid = FORK_OR_VFORK()) < 0)
+	{
+		tst_resm(TWARN, "\tFork failed (may be OK if under stress)");
+		TEST(msgctl(tid, IPC_RMID, 0));
+		if (TEST_RETURN < 0)
+		{
+			tst_resm(TFAIL, "Msgctl error in cleanup, errno = %d", errno);
+		}
+		tst_exit();
+	}
+	/* Child does this */
+	if (pid == 0)
+	{
+#ifdef UCLINUX
+		if (self_exec(argv0, "nddd", 2, id, key, child_process) < 0) {
+			tst_resm(TWARN, "self_exec failed");
+			TEST(msgctl(tid, IPC_RMID, 0));
+			if (TEST_RETURN < 0)
+			{
+				tst_resm(TFAIL, "\tMsgctl error in cleanup, "
+					"errno = %d\n", errno);
+			}
+			tst_exit();
+		}
+#else
+		exit( doreader(id, key % 255, child_process) );
+#endif
+	}
+	/* Parent does this */
+	mykid = pid;
+	procstat = 2;
+	dowriter(id, key % 255, child_process);
+	wait(0);
+	TEST(msgctl(id, IPC_RMID, 0));
+	if (TEST_RETURN < 0)
+	{
+		tst_resm(TFAIL, "msgctl errno %d", TEST_ERRNO);
+		tst_exit();
+	}
+	exit(PASS);
+}
+
+int doreader(id, key, child)
+int id, child;
+long key;
+{
+	int i, size;
+
+	for (i = 0; i < nreps; i++)
+	{
+		if ((size = msgrcv(id, &buffer, 100, 0, 0)) < 0)
+		{
+			tst_brkm(TBROK, cleanup, "Msgrcv error in child %d, read # = %d, errno = %d", (i + 1), child, errno);
+			tst_exit();
+		}
+		if (buffer.data.len + 1 != size)
+		{
+			tst_resm(TFAIL, "Size mismatch in child %d, read # = %d", child, (i + 1));
+			tst_resm(TFAIL, "for message size got  %d expected  %d %s",size ,buffer.data.len);
+			tst_exit();
+		}
+		if ( verify(buffer.data.pbytes, key, size - 1, child) )
+		{
+			tst_resm(TFAIL, "in read # = %d,key =  %x", (i + 1), child, key);
+			tst_exit();
+		}
+		key++;
+	}
+	return (0);
+}
+
+int dowriter(id, key, child)
+int id,child;
+long key;
+{
+	int i, size;
+
+	for (i = 0; i < nreps; i++)
+	{
+		do
+		{
+			size = (rand() % 99);
+		} while (size == 0);
+		fill_buffer(buffer.data.pbytes, key, size);
+		buffer.data.len = size;
+		buffer.type = 1;
+		TEST(msgsnd(id, &buffer, size + 1, 0));
+		if (TEST_RETURN < 0)
+		{
+			tst_brkm(TBROK, cleanup, "Msgsnd error in child %d, key =   %x errno  = %d", child, key, TEST_ERRNO);
+		}
+		key++;
+	}
+	return (0);
+}
+
+int fill_buffer(buf, val, size)
+register char *buf;
+char	val;
+register int size;
+{
+	register int i;
+
+	for(i = 0; i < size; i++)
+	{
+		buf[i] = val;
+	}
+
+	return (0);
+}
+
+
+/*
+ * verify()
+ *	Check a buffer for correct values.
+ */
+
+int verify(buf, val, size, child)
+	register char *buf;
+	char	val;
+	register int size;
+	int	child;
+{
+	while(size-- > 0)
+	{
+		if (*buf++ != val)
+		{
+			tst_resm(TWARN, "Verify error in child %d, *buf = %x, val = %x, size = %d", child, *buf, val, size);
+			return(FAIL);
+		}
+	}
+	return(PASS);
+}
+
+/*
+ *  * void
+ *  * sig_handler() - signal catching function for 'SIGUSR1' signal.
+ *  *
+ *  *   This is a null function and used only to catch the above signal
+ *  *   generated in parent process.
+ *  */
+void
+sig_handler()
+{
+}
+
+/***************************************************************
+ * setup() - performs all ONE TIME setup for this test.
+ *****************************************************************/
+void
+setup()
+{
+	int nr_msgqs;
+
+	tst_tmpdir();
+
+	/* You will want to enable some signal handling so you can capture
+	 * unexpected signals like SIGSEGV.
+	 */
+	tst_sig(FORK, DEF_HANDLER, cleanup);
+
+
+	/* Pause if that option was specified */
+	/* One cavet that hasn't been fixed yet.  TEST_PAUSE contains the code to
+	 * fork the test with the -c option.  You want to make sure you do this
+	 * before you create your temporary directory.
+	 */
+	TEST_PAUSE;
+
+	nr_msgqs = get_max_msgqueues();
+	if (nr_msgqs < 0)
+		cleanup();
+
+	MSGMNI = nr_msgqs - get_used_msgqueues();
+	if (MSGMNI <= 0){
+		tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
+		cleanup();
+	}
+}
+
+
+/***************************************************************
+ * cleanup() - performs all ONE TIME cleanup for this test at
+ *             completion or premature exit.
+ ****************************************************************/
+void
+cleanup()
+{
+	int status;
+	/*
+	 *  Remove the message queue from the system
+	 */
+#ifdef DEBUG
+	tst_resm(TINFO,"Removing the message queue");
+#endif
+	fflush (stdout);
+	(void) msgctl(tid, IPC_RMID, (struct msqid_ds *)NULL);
+	if ((status = msgctl(tid, IPC_STAT, (struct msqid_ds *)NULL)) != -1)
+	{
+		(void) msgctl(tid, IPC_RMID, (struct msqid_ds *)NULL);
+		tst_resm(TFAIL, "msgctl(tid, IPC_RMID) failed");
+		tst_exit();
+	}
+
+	fflush (stdout);
+	/*
+	 * print timing stats if that option was specified.
+	 * print errno log if that option was specified.
+	 */
+	TEST_CLEANUP;
+	tst_rmdir();
+	/* exit with return code appropriate for results */
+	tst_exit();
+}
+
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgget/Makefile
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgget/Makefile	2008-02-21 10:45:36.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgget/Makefile	2008-02-21 12:22:43.000000000 +0100
@@ -21,9 +21,12 @@ LDLIBS += -L../../../../../lib -lltp -L.
 
 SRCS    = $(wildcard *.c)
 TARGETS = $(patsubst %.c,%,$(SRCS))
+LIBIPC_HEADERS	= ../lib/ipcmsg.h
 
 all: $(TARGETS)
 
+$(TARGETS):	$(LIBIPC_HEADERS)
+
 install:
 	@set -e; for i in $(TARGETS); do ln -f $$i ../../../../bin/$$i ; done
 
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl11.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl11.c	2008-02-21 14:04:14.000000000 +0100
@@ -0,0 +1,696 @@
+/*
+ *
+ *   Copyright (c) International Business Machines  Corp., 2002
+ *
+ *   This program is free software;  you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation; either version 2 of the License, or
+ *   (at your option) any later version.
+ *
+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ *   the GNU General Public License for more details.
+ *
+ *   You should have received a copy of the GNU General Public License
+ *   along with this program;  if not, write to the Free Software
+ *   Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+/* 06/30/2001	Port to Linux	nsharoff@us.ibm.com */
+/* 11/11/2002   Port to LTP     dbarrera@us.ibm.com */
+
+
+/*
+ * NAME
+ *	msgctl11
+ *
+ * CALLS
+ *	msgget(2) msgctl(2) msgop(2)
+ *
+ * ALGORITHM
+ *	Get and manipulate a message queue.
+ *	Same as msgctl09 but gets the actual msgmni value under procfs.
+ *
+ * RESTRICTIONS
+ *
+ */
+
+#define _XOPEN_SOURCE 500
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/ipc.h>
+#include <sys/msg.h>
+#include <sys/wait.h>
+#include <signal.h>
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include "test.h"
+#include "usctest.h"
+#include "ipcmsg.h"
+
+#define MAXNREPS	1000
+#ifndef CONFIG_COLDFIRE
+#define MAXNPROCS	 1000000  /* This value is set to an arbitrary high limit. */
+#else
+#define MAXNPROCS	 100000   /* Coldfire can't deal with 1000000 */
+#endif
+#define MAXNKIDS	10
+#define FAIL		1
+#define PASS		0
+
+int dotest(key_t,int);
+int doreader(long,int,int);
+int dowriter(long,int,int);
+int fill_buffer(char*,char,int);
+int verify(char*,char,int,int);
+void setup();
+void cleanup();
+
+/*
+ * These globals must be defined in the test.
+ * */
+
+
+char *TCID="msgctl11";           /* Test program identifier.    */
+int TST_TOTAL=1;                /* Total number of test cases. */
+extern int Tst_count;           /* Test Case counter for tst_* routines */
+
+int exp_enos[]={0};     /* List must end with 0 */
+
+
+key_t	keyarray[MAXNPROCS];
+
+struct {
+	long	type;
+	struct {
+		char	len;
+		char	pbytes[99];
+		} data;
+	} buffer;
+
+int	pidarray[MAXNPROCS];
+int	rkidarray[MAXNKIDS];
+int	wkidarray[MAXNKIDS];
+int 	tid;
+int 	nprocs, nreps, nkids, MSGMNI;
+int 	procstat;
+void 	term(int);
+#ifdef UCLINUX
+static char *argv0;
+
+void do_child_1_uclinux();
+static key_t key_uclinux;
+static int i_uclinux;
+
+void do_child_2_uclinux();
+static int pid_uclinux;
+static int child_process_uclinux;
+
+void do_child_3_uclinux();
+static int rkid_uclinux;
+#endif
+void cleanup_msgqueue(int i, int tid);
+
+/*-----------------------------------------------------------------*/
+int main(argc, argv)
+int	argc;
+char	*argv[];
+{
+	register int i, j, ok, pid;
+	int count, status;
+
+#ifdef UCLINUX
+	char *msg;			/* message returned from parse_opts */
+
+	argv0 = argv[0];
+
+	/* parse standard options */
+	if ((msg = parse_opts(argc, argv, (option_t *)NULL, NULL)) != (char *)NULL)
+	{
+		tst_brkm(TBROK, cleanup, "OPTION PARSING ERROR - %s", msg);
+	}
+
+	maybe_run_child(&do_child_1_uclinux, "ndd", 1, &key_uclinux, &i_uclinux);
+	maybe_run_child(&do_child_2_uclinux, "nddd", 2, &key_uclinux,
+			&pid_uclinux, &child_process_uclinux);
+	maybe_run_child(&do_child_3_uclinux, "nddd", 3, &key_uclinux,
+			&rkid_uclinux, &child_process_uclinux);
+#endif
+
+	setup();
+
+	if (argc == 1 )
+	{
+		/* Set default parameters */
+		nreps = MAXNREPS;
+		nprocs = MSGMNI;
+		nkids = MAXNKIDS;
+	}
+	else if (argc == 4 )
+	{
+		if ( atoi(argv[1]) > MAXNREPS )
+		{
+			tst_resm(TCONF,"Requested number of iterations too large, setting to Max. of %d", MAXNREPS);
+			nreps = MAXNREPS;
+		}
+		else
+		{
+			nreps = atoi(argv[1]);
+		}
+		if (atoi(argv[2]) > MSGMNI )
+		{
+			tst_resm(TCONF,"Requested number of processes too large, setting to Max. of %d", MSGMNI);
+			nprocs = MSGMNI;
+		}
+		else
+		{
+			nprocs = atoi(argv[2]);
+		}
+		if (atoi(argv[3]) > MAXNKIDS )
+		{
+			tst_resm(TCONF,"Requested number of read/write pairs too large; setting to Max. of %d", MAXNKIDS);
+			nkids = MAXNKIDS;
+		}
+		else
+		{
+			nkids = atoi(argv[3]);
+		}
+	}
+	else
+	{
+		tst_resm(TCONF," Usage: %s [ number of iterations  number of processes number of read/write pairs ]", argv[0]);
+		tst_exit();
+	}
+
+	procstat = 0;
+	srand48((unsigned)getpid() + (unsigned)(getppid() << 16));
+	tid = -1;
+
+	/* Setup signal handleing routine */
+	if (sigset(SIGTERM, term) == SIG_ERR)
+	{
+		tst_resm(TFAIL, "Sigset SIGTERM failed");
+		tst_exit();
+	}
+	/* Set up array of unique keys for use in allocating message
+	 * queues
+	 */
+	for (i = 0; i < nprocs; i++)
+	{
+		ok = 1;
+		do
+		{
+			/* Get random key */
+			keyarray[i] = (key_t)lrand48();
+			/* Make sure key is unique and not private */
+			if (keyarray[i] == IPC_PRIVATE)
+			{
+				ok = 0;
+				continue;
+			}
+			for (j = 0; j < i; j++)
+			{
+				if (keyarray[j] == keyarray[i])
+				{
+					ok = 0;
+					break;
+				}
+				ok = 1;
+			}
+		} while (ok == 0);
+	}
+/*-----------------------------------------------------------------*/
+	/* Fork a number of processes (nprocs), each of which will
+	 * create a message queue with several (nkids) reader/writer
+	 * pairs which will read and write a number (iterations)
+	 * of random length messages with specific values (keys).
+	 */
+
+	for (i = 0; i <  nprocs; i++)
+	{
+		fflush(stdout);
+		if ((pid = FORK_OR_VFORK()) < 0)
+		{
+			tst_resm(TFAIL, "\tFork failed (may be OK if under stress)");
+			tst_exit();
+		}
+		/* Child does this */
+		if (pid == 0)
+		{
+#ifdef UCLINUX
+			if (self_exec(argv[0], "ndd", 1, keyarray[i], i) < 0)
+			{
+				tst_resm(TFAIL, "\tself_exec failed");
+				tst_exit();
+			}
+#else
+			procstat = 1;
+			exit( dotest(keyarray[i], i) );
+#endif
+		}
+		pidarray[i] = pid;
+	}
+
+	count = 0;
+	while(1)
+	{
+		if (( wait(&status)) > 0)
+		{
+			if (status>>8 != PASS )
+			{
+				tst_resm(TFAIL, "Child exit status = %d", status>>8);
+				tst_exit();
+			}
+			count++;
+		}
+		else
+		{
+			if (errno != EINTR)
+			{
+				break;
+			}
+#ifdef DEBUG
+			tst_resm(TINFO,"Signal detected during wait");
+#endif
+		}
+	}
+	/* Make sure proper number of children exited */
+	if (count != nprocs)
+	{
+		tst_resm(TFAIL, "Wrong number of children exited, Saw %d, Expected %d", count, nprocs);
+		tst_exit();
+	}
+
+	tst_resm(TPASS,"msgctl11 ran successfully!");
+
+	cleanup();
+
+	return (0);
+
+
+
+}
+/*--------------------------------------------------------------------*/
+
+#ifdef UCLINUX
+void
+do_child_1_uclinux()
+{
+	procstat = 1;
+	exit(dotest(key_uclinux, i_uclinux));
+}
+
+void
+do_child_2_uclinux()
+{
+	procstat = 2;
+	exit(doreader(key_uclinux, pid_uclinux, child_process_uclinux));
+}
+
+void
+do_child_3_uclinux()
+{
+	procstat = 2;
+	exit(dowriter(key_uclinux, rkid_uclinux, child_process_uclinux));
+}
+#endif
+
+void
+cleanup_msgqueue(int i, int tid)
+{
+	/*
+	 * Decrease the value of i by 1 because it
+	 * is getting incremented even if the fork
+	 * is failing.
+	 */
+
+	i--;
+	/*
+	 * Kill all children & free message queue.
+	 */
+	for (; i >= 0; i--) {
+		(void)kill(rkidarray[i], SIGKILL);
+		(void)kill(wkidarray[i], SIGKILL);
+	}
+
+	if (msgctl(tid, IPC_RMID, 0) < 0) {
+		tst_resm(TFAIL, "Msgctl error in cleanup, errno = %d", errno);
+		tst_exit();
+	}
+}
+
+int dotest(key, child_process)
+key_t 	key;
+int	child_process;
+{
+	int id, pid;
+	int i, count, status, exit_status;
+
+	sighold(SIGTERM);
+	if ((id = msgget(key, IPC_CREAT | S_IRUSR | S_IWUSR )) < 0)
+	{
+		tst_resm(TFAIL, "Msgget error in child %d, errno = %d", child_process, errno);
+		tst_exit();
+	}
+	tid = id;
+	sigrelse(SIGTERM);
+
+	exit_status = PASS;
+
+	for (i=0; i < nkids; i++)
+	{
+		fflush(stdout);
+		if ((pid = FORK_OR_VFORK()) < 0)
+		{
+			tst_resm(TWARN, "Fork failure in first child of child group %d", child_process);
+			cleanup_msgqueue(i, tid);
+			tst_exit();
+		}
+		/* First child does this */
+		if (pid == 0)
+		{
+#ifdef UCLINUX
+			if (self_exec(argv0, "nddd", 2, key, getpid(),
+							child_process) < 0) {
+				tst_resm(TWARN, "self_exec failed");
+				cleanup_msgqueue(i, tid);
+				tst_exit();
+			}
+#else
+			procstat = 2;
+			exit( doreader( key, getpid(), child_process) );
+#endif
+		}
+		rkidarray[i] = pid;
+		fflush(stdout);
+		if ((pid = FORK_OR_VFORK()) < 0)
+		{
+			tst_resm(TWARN, "Fork failure in first child of child group %d", child_process);
+			/*
+			 * Kill the reader child process
+			 */
+			(void)kill(rkidarray[i], SIGKILL);
+
+			cleanup_msgqueue(i, tid);
+			tst_exit();
+		}
+		/* Second child does this */
+		if (pid == 0)
+		{
+#ifdef UCLINUX
+			if (self_exec(argv0, "nddd", 3, key, rkidarray[i],
+							child_process) < 0) {
+				tst_resm(TWARN, "\tFork failure in first child "
+					"of child group %d \n", child_process);
+				/*
+				 * Kill the reader child process
+				 */
+				(void)kill(rkidarray[i], SIGKILL);
+
+				cleanup_msgqueue(i, tid);
+				tst_exit();
+			}
+#else
+			procstat = 2;
+			exit( dowriter( key, rkidarray[i], child_process) );
+#endif
+		}
+		wkidarray[i] = pid;
+	}
+	/* Parent does this */
+	count = 0;
+	while(1)
+	{
+		if (( wait(&status)) > 0)
+		{
+			if (status>>8 != PASS )
+			{
+				tst_resm(TFAIL, "Child exit status = %d from child group %d", status>>8, child_process);
+				for (i = 0; i < nkids; i++)
+				{
+					kill(rkidarray[i], SIGTERM);
+					kill(wkidarray[i], SIGTERM);
+				}
+				if (msgctl(tid, IPC_RMID, 0) < 0) {
+					tst_resm(TFAIL, "Msgctl error, errno = %d", errno);
+				}
+				tst_exit();
+			}
+			count++;
+		}
+		else
+		{
+			if (errno != EINTR)
+			{
+				break;
+			}
+		}
+	}
+	/* Make sure proper number of children exited */
+	if (count != (nkids * 2))
+	{
+		tst_resm(TFAIL, "Wrong number of children exited in child group %d, Saw %d Expected %d", child_process, count, (nkids * 2));
+		if (msgctl(tid, IPC_RMID, 0) < 0) {
+			tst_resm(TFAIL, "Msgctl error, errno = %d", errno);
+		}
+		tst_exit();
+	}
+	if (msgctl(id, IPC_RMID, 0) < 0)
+	{
+		tst_resm(TFAIL, "Msgctl failure in child group %d, errno %d", child_process, errno);
+		tst_exit();
+	}
+	exit(exit_status);
+}
+
+int doreader( key, type, child)
+int type, child;
+long key;
+{
+	int i, size;
+	int id;
+
+	if ((id = msgget(key, 0)) < 0)
+	{
+		tst_resm(TFAIL, "Msgget error in reader of child group %d, errno = %d", child, errno);
+		tst_exit();
+	}
+	if (id != tid)
+	{
+		tst_resm(TFAIL, "Message queue mismatch in reader of child group %d for message queue id %d", child, id);
+		tst_exit();
+	}
+	for (i = 0; i < nreps; i++)
+	{
+		if ((size = msgrcv(id, &buffer, 100, type, 0)) < 0)
+		{
+			tst_resm(TFAIL, "Msgrcv error in child %d, read # = %d, errno = %d", (i + 1), child, errno);
+			tst_exit();
+		}
+		if (buffer.type != type)
+		{
+			tst_resm(TFAIL, "Size mismatch in child %d, read # = %d", child, (i + 1));
+			tst_resm(TFAIL, "\tfor message size got  %d expected  %d %s",size ,buffer.data.len);
+			tst_exit();
+		}
+		if (buffer.data.len + 1 != size)
+		{
+			tst_resm(TFAIL, "Size mismatch in child %d, read # = %d, size = %d, expected = %d", child, (i + 1), buffer.data.len, size);
+			tst_exit();
+		}
+		if ( verify(buffer.data.pbytes, (key % 255), size - 1, child) )
+		{
+			tst_resm(TFAIL, "in read # = %d,key =  %x", (i + 1), child, key);
+			tst_exit();
+		}
+		key++;
+	}
+	exit(PASS);
+}
+
+int dowriter( key, type, child)
+int type,child;
+long key;
+{
+	int i, size;
+	int id;
+
+	if ((id = msgget(key, 0)) < 0)
+	{
+		tst_resm(TFAIL, "Msgget error in writer of child group %d, errno = %d", child, errno);
+		tst_exit();
+	}
+	if (id != tid)
+	{
+		tst_resm(TFAIL, "Message queue mismatch in writer of child group %d", child);
+		tst_resm(TFAIL, "\tfor message queue id %d expected  %d",id, tid);
+		tst_exit();
+	}
+
+	for (i = 0; i < nreps; i++)
+	{
+		do
+		{
+			size = (lrand48() % 99);
+		} while (size == 0);
+		fill_buffer(buffer.data.pbytes, (key % 255), size);
+		buffer.data.len = size;
+		buffer.type = type;
+		if (msgsnd(id, &buffer, size + 1, 0) < 0)
+		{
+			tst_resm(TFAIL, "Msgsnd error in child %d, key =   %x errno  = %d", child, key, errno);
+			tst_exit();
+		}
+		key++;
+	}
+	exit(PASS);
+}
+
+int fill_buffer(buf, val, size)
+register char *buf;
+char	val;
+register int size;
+{
+	register int i;
+
+	for(i = 0; i < size; i++)
+		buf[i] = val;
+	return(0);
+}
+
+
+/*
+ * verify()
+ *	Check a buffer for correct values.
+ */
+
+int verify(buf, val, size, child)
+	register char *buf;
+	char	val;
+	register int size;
+	int	child;
+{
+	while(size-- > 0)
+		if (*buf++ != val)
+		{
+			tst_resm(TWARN, "Verify error in child %d, *buf = %x, val = %x, size = %d", child, *buf, val, size);
+			return(FAIL);
+		}
+	return(PASS);
+}
+
+/* ARGSUSED */
+void
+term(int sig)
+{
+	int i;
+
+	if (procstat == 0)
+	{
+#ifdef DEBUG
+		tst_resm(TINFO,"SIGTERM signal received, test killing kids");
+#endif
+		for (i = 0; i < nprocs; i++)
+		{
+			if ( pidarray[i] > 0){
+				if ( kill(pidarray[i], SIGTERM) < 0)
+				{
+					tst_resm(TBROK,"Kill failed to kill child %d", i);
+					exit(FAIL);
+				}
+			}
+		}
+		return;
+	}
+
+	if (procstat == 2)
+	{
+		fflush(stdout);
+		exit(PASS);
+	}
+
+	if (tid == -1)
+	{
+		exit(FAIL);
+	}
+	for (i = 0; i < nkids; i++)
+	{
+		if (rkidarray[i] > 0)
+			kill(rkidarray[i], SIGTERM);
+		if (wkidarray[i] > 0)
+			kill(wkidarray[i], SIGTERM);
+	}
+}
+
+/***************************************************************
+ * setup() - performs all ONE TIME setup for this test.
+ *****************************************************************/
+void
+setup()
+{
+	int nr_msgqs;
+
+	tst_tmpdir();
+	/* You will want to enable some signal handling so you can capture
+	 * unexpected signals like SIGSEGV.
+	 */
+	tst_sig(FORK, DEF_HANDLER, cleanup);
+
+
+	/* Pause if that option was specified */
+	/* One cavet that hasn't been fixed yet.  TEST_PAUSE contains the code to
+	 * fork the test with the -c option.  You want to make sure you do this
+	 * before you create your temporary directory.
+	 */
+	TEST_PAUSE;
+
+	nr_msgqs = get_max_msgqueues();
+	if (nr_msgqs < 0)
+		cleanup();
+
+	MSGMNI = nr_msgqs - get_used_msgqueues();
+	if (MSGMNI <= 0){
+		tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
+		cleanup();
+	}
+}
+
+
+/***************************************************************
+ * cleanup() - performs all ONE TIME cleanup for this test at
+ * completion or premature exit.
+ ****************************************************************/
+void
+cleanup()
+{
+	int status;
+	/*
+	 * print timing stats if that option was specified.
+	 * print errno log if that option was specified.
+	 */
+	TEST_CLEANUP;
+
+	/*
+	 * Remove the message queue from the system
+	 */
+#ifdef DEBUG
+	tst_resm(TINFO,"Removing the message queue");
+#endif
+	fflush (stdout);
+	(void) msgctl(tid, IPC_RMID, (struct msqid_ds *)NULL);
+	if ((status = msgctl(tid, IPC_STAT, (struct msqid_ds *)NULL)) != -1) {
+		(void) msgctl(tid, IPC_RMID, (struct msqid_ds *)NULL);
+		tst_resm(TFAIL, "msgctl(tid, IPC_RMID) failed");
+		tst_exit();
+	}
+
+	fflush (stdout);
+	tst_rmdir();
+	/* exit with return code appropriate for results */
+	tst_exit();
+}
+

Re: [LTP] [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Subrata Modak]

> Nadia Derbey wrote:
> > Matt Helsley wrote:
> > 
> >> On Tue, 2008-02-19 at 18:16 +0100, Nadia Derbey wrote:
> >>
> >> <snip>
> >>
> >>> +#define MAX_MSGQUEUES  16      /* MSGMNI as defined in linux/msg.h */
> >>> +
> >>
> >>
> >>
> >> It's not quite the maximum anymore, is it? More like the minumum
> >> maximum ;). A better name might better document what the test is
> >> actually trying to do.
> >>
> >> One question I have is whether the unpatched test is still valuable.
> >> Based on my limited knowledge of the test I suspect it's still a correct
> >> test of message queues. If so, perhaps renaming the old test (so it's
> >> not confused with a performance regression) and adding your patched
> >> version is best?
> >>
> > 
> > So, here's the new patch based on Matt's points.
> > 
> > Subrata, it has to be applied on top of the original ltp-full-20080131. 
> > Please tell me if you'd prefer one based on the merged version you've 
> > got (i.e. with my Tuesday patch applied).

Nadia, I would prefer Patch on the top of the already merged version (on
top of latest CVS snapshot as of today). Anyways, thanks for all these
effort :-)

--Subrata

> > 
> 
> Forgot the patch, sorry for that (thx Andrew).
> 
> Regards,
> Nadia
> 

Re: [LTP] [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Nadia Derbey]

[-- Attachment #1: Type: text/plain, Size: 1294 bytes --]

Subrata Modak wrote:
>>Nadia Derbey wrote:
>>
>>>Matt Helsley wrote:
>>>
>>>
>>>>On Tue, 2008-02-19 at 18:16 +0100, Nadia Derbey wrote:
>>>>
>>>><snip>
>>>>
>>>>>+#define MAX_MSGQUEUES  16      /* MSGMNI as defined in linux/msg.h */
>>>>>+
>>>>
>>>>
>>>>
>>>>It's not quite the maximum anymore, is it? More like the minumum
>>>>maximum ;). A better name might better document what the test is
>>>>actually trying to do.
>>>>
>>>>One question I have is whether the unpatched test is still valuable.
>>>>Based on my limited knowledge of the test I suspect it's still a correct
>>>>test of message queues. If so, perhaps renaming the old test (so it's
>>>>not confused with a performance regression) and adding your patched
>>>>version is best?
>>>>
>>>
>>>So, here's the new patch based on Matt's points.
>>>
>>>Subrata, it has to be applied on top of the original ltp-full-20080131. 
>>>Please tell me if you'd prefer one based on the merged version you've 
>>>got (i.e. with my Tuesday patch applied).
> 
> 
> Nadia, I would prefer Patch on the top of the already merged version (on
> top of latest CVS snapshot as of today). Anyways, thanks for all these
> effort :-)
> 
> --Subrata
> 

In attachment, you'll find a patch to apply on top of the patches I sent 
you on Tuesday.

Regards,
Nadia

[-- Attachment #2: ipc_ltp_full_20080131.2.patch --]
[-- Type: text/x-patch, Size: 37757 bytes --]

Since msgmni now scales to the memory size, it may reach big values.
To avoid forking 2*msgmni processes and create msgmni msg queues, take the min
between the procfs value and MSGMNI (as found in linux/msg.h).

Also integrated the following in libipc.a:
  . get_max_msgqueues()
  . get_used_msgqueues()

Signed-off-by: Nadia Derbey <Nadia.Derbey@bull.net>

---
 testcases/kernel/syscalls/ipc/lib/ipcmsg.h      |    7 
 testcases/kernel/syscalls/ipc/lib/libipc.c      |   54 +
 testcases/kernel/syscalls/ipc/msgctl/msgctl08.c |   42 -
 testcases/kernel/syscalls/ipc/msgctl/msgctl09.c |   42 -
 testcases/kernel/syscalls/ipc/msgctl/msgctl10.c |  527 ++++++++++++++++++
 testcases/kernel/syscalls/ipc/msgctl/msgctl11.c |  696 ++++++++++++++++++++++++
 testcases/kernel/syscalls/ipc/msgget/Makefile   |    3 
 testcases/kernel/syscalls/ipc/msgget/msgget03.c |   22 
 8 files changed, 1318 insertions(+), 75 deletions(-)

Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/libipc.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/lib/libipc.c	2008-02-22 07:57:47.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/libipc.c	2008-02-22 08:02:55.000000000 +0100
@@ -201,3 +201,57 @@ rm_shm(int shm_id)
 		tst_resm(TINFO, "id = %d", shm_id);
 	}
 }
+
+#define BUFSIZE 512
+
+/*
+ * Get the number of message queues already in use
+ */
+int
+get_used_msgqueues()
+{
+	FILE *f;
+	int used_queues;
+	char buff[BUFSIZE];
+
+	f = popen("ipcs -q", "r");
+	if (!f) {
+		tst_resm(TBROK, "Could not run 'ipcs' to calculate used "
+			"message queues");
+		tst_exit();
+	}
+	/* FIXME: Start at -4 because ipcs prints four lines of header */
+	for (used_queues = -4; fgets(buff, BUFSIZE, f); used_queues++)
+		;
+	pclose(f);
+	if (used_queues < 0) {
+		tst_resm(TBROK, "Could not read output of 'ipcs' to "
+			"calculate used message queues");
+		tst_exit();
+	}
+	return used_queues;
+}
+
+/*
+ * Get the max number of message queues allowed on system
+ */
+int
+get_max_msgqueues()
+{
+	FILE *f;
+	char buff[BUFSIZE];
+
+	/* Get the max number of message queues allowed on system */
+	f = fopen("/proc/sys/kernel/msgmni", "r");
+	if (!f) {
+		tst_resm(TBROK, "Could not open /proc/sys/kernel/msgmni");
+		return -1;
+	}
+	if (!fgets(buff, BUFSIZE, f)) {
+		fclose(f);
+		tst_resm(TBROK, "Could not read /proc/sys/kernel/msgmni");
+		return -1;
+	}
+	fclose(f);
+	return atoi(buff);
+}
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/ipcmsg.h
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/lib/ipcmsg.h	2008-02-22 07:57:47.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/lib/ipcmsg.h	2008-02-22 08:04:15.000000000 +0100
@@ -41,7 +41,9 @@ void setup(void);
 #define MSGSIZE	1024		/* a resonable size for a message */
 #define MSGTYPE 1		/* a type ID for a message */
 
-#define MAX_MSGQUEUES	16	/* MSGMNI as defined in linux/msg.h */
+#define NR_MSGQUEUES	16	/* MSGMNI as defined in linux/msg.h */
+
+#define min(a, b)	(((a) < (b)) ? (a) : (b))
 
 typedef struct mbuf {		/* a generic message structure */
 	long mtype;
@@ -61,4 +63,7 @@ void rm_queue(int);
 int getipckey();
 int getuserid(char *);
 
+int get_max_msgqueues(void);
+int get_used_msgqueues(void);
+
 #endif /* ipcmsg.h */
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl10.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl10.c	2008-02-22 08:05:53.000000000 +0100
@@ -0,0 +1,527 @@
+/*
+ *
+ *   Copyright (c) International Business Machines  Corp., 2002
+ *
+ *   This program is free software;  you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation; either version 2 of the License, or
+ *   (at your option) any later version.
+ *
+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ *   the GNU General Public License for more details.
+ *
+ *   You should have received a copy of the GNU General Public License
+ *   along with this program;  if not, write to the Free Software
+ *   Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+/* 06/30/2001	Port to Linux	nsharoff@us.ibm.com */
+/* 11/06/2002   Port to LTP     dbarrera@us.ibm.com */
+
+/*
+ * NAME
+ *	msgctl10
+ *
+ * CALLS
+ *	msgget(2) msgctl(2)
+ *
+ * ALGORITHM
+ *	Get and manipulate a message queue.
+ *	Same as msgctl08 but gets the actual msgmni value under procfs.
+ *
+ * RESTRICTIONS
+ *
+ */
+
+#define _XOPEN_SOURCE 500
+#include <signal.h>
+#include <errno.h>
+#include <string.h>
+#include <fcntl.h>
+#include <stdlib.h>
+#include <stdio.h>
+#include <unistd.h>
+#include <values.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <sys/stat.h>
+#include <sys/ipc.h>
+#include <sys/msg.h>
+#include "test.h"
+#include "usctest.h"
+#include "ipcmsg.h"
+
+void setup();
+void cleanup();
+/*
+ *  *  *  * These globals must be defined in the test.
+ *   *   *   */
+
+
+char *TCID="msgctl10";           /* Test program identifier.    */
+int TST_TOTAL=1;                /* Total number of test cases. */
+extern int Tst_count;           /* Test Case counter for tst_* routines */
+
+int exp_enos[]={0};     /* List must end with 0 */
+
+#ifndef CONFIG_COLDFIRE
+#define MAXNPROCS	1000000  /* This value is set to an arbitrary high limit. */
+#else
+#define MAXNPROCS	 100000   /* Coldfire can't deal with 1000000 */
+#endif
+#define MAXNREPS	100000
+#define FAIL		1
+#define PASS		0
+
+key_t	keyarray[MAXNPROCS];
+
+struct {
+	long	type;
+	struct {
+		char	len;
+		char	pbytes[99];
+		} data;
+	} buffer;
+
+int	pidarray[MAXNPROCS];
+int tid;
+int MSGMNI,nprocs, nreps;
+int procstat;
+int dotest(key_t key, int child_process);
+int doreader(int id, long key, int child);
+int dowriter(int id,long key, int child);
+int fill_buffer(register char *buf, char val, register int size);
+int verify(register char *buf,char val, register int size,int child);
+void sig_handler();             /* signal catching function */
+int mykid;
+#ifdef UCLINUX
+static char *argv0;
+
+void do_child_1_uclinux();
+static key_t key_uclinux;
+static int i_uclinux;
+
+void do_child_2_uclinux();
+static int id_uclinux;
+static int child_process_uclinux;
+#endif
+
+/*-----------------------------------------------------------------*/
+int main(argc, argv)
+int	argc;
+char	*argv[];
+{
+	register int i, j, ok, pid;
+	int count, status;
+	struct sigaction act;
+
+#ifdef UCLINUX
+	char *msg;			/* message returned from parse_opts */
+
+	argv0 = argv[0];
+
+	/* parse standard options */
+	if ((msg = parse_opts(argc, argv, (option_t *)NULL, NULL)) != (char *)NULL)
+	{
+		tst_brkm(TBROK, cleanup, "OPTION PARSING ERROR - %s", msg);
+	}
+
+	maybe_run_child(&do_child_1_uclinux, "ndd", 1, &key_uclinux, &i_uclinux);
+	maybe_run_child(&do_child_2_uclinux, "nddd", 2, &id_uclinux, &key_uclinux,
+			&child_process_uclinux);
+#endif
+
+	setup();
+
+	if (argc == 1 )
+	{
+		/* Set default parameters */
+		nreps = MAXNREPS;
+		nprocs = MSGMNI;
+	}
+	else if (argc == 3 )
+	{
+		if ( atoi(argv[1]) > MAXNREPS )
+		{
+			tst_resm(TCONF,"Requested number of iterations too large, setting to Max. of %d", MAXNREPS);
+			nreps = MAXNREPS;
+		}
+		else
+		{
+			nreps = atoi(argv[1]);
+		}
+		if (atoi(argv[2]) > MSGMNI )
+		{
+			tst_resm(TCONF,"Requested number of processes too large, setting to Max. of %d", MSGMNI);
+			nprocs = MSGMNI;
+		}
+		else
+		{
+			nprocs = atoi(argv[2]);
+		}
+	}
+	else
+	{
+		tst_resm(TCONF," Usage: %s [ number of iterations  number of processes ]", argv[0]);
+		tst_exit();
+	}
+
+	srand(getpid());
+	tid = -1;
+
+	/* Setup signal handleing routine */
+	memset(&act, 0, sizeof(act));
+	act.sa_handler = sig_handler;
+	sigemptyset(&act.sa_mask);
+	sigaddset(&act.sa_mask, SIGTERM);
+	if (sigaction(SIGTERM, &act, NULL) < 0)
+	{
+		tst_resm(TFAIL, "Sigset SIGTERM failed");
+		tst_exit();
+	}
+	/* Set up array of unique keys for use in allocating message
+	 * queues
+	 */
+	for (i = 0; i < nprocs; i++)
+	{
+		ok = 1;
+		do
+		{
+			/* Get random key */
+			keyarray[i] = (key_t)rand();
+			/* Make sure key is unique and not private */
+			if (keyarray[i] == IPC_PRIVATE)
+			{
+				ok = 0;
+				continue;
+			}
+			for (j = 0; j < i; j++)
+			{
+				if (keyarray[j] == keyarray[i])
+				{
+					ok = 0;
+					break;
+				}
+				ok = 1;
+			}
+		} while (ok == 0);
+	}
+
+	/* Fork a number of processes, each of which will
+	 * create a message queue with one reader/writer
+	 * pair which will read and write a number (iterations)
+	 * of random length messages with specific values.
+	 */
+
+	for (i = 0; i <  nprocs; i++)
+	{
+		fflush(stdout);
+		if ((pid = FORK_OR_VFORK()) < 0)
+		{
+			tst_resm(TFAIL, "\tFork failed (may be OK if under stress)");
+			tst_exit();
+		}
+		/* Child does this */
+		if (pid == 0)
+		{
+#ifdef UCLINUX
+			if (self_exec(argv[0], "ndd", 1, keyarray[i], i) < 0)
+			{
+				tst_resm(TFAIL, "\tself_exec failed");
+				tst_exit();
+			}
+#else
+			procstat = 1;
+			exit( dotest(keyarray[i], i) );
+#endif
+		}
+		pidarray[i] = pid;
+	}
+
+	count = 0;
+	while(1)
+	{
+		if (( wait(&status)) > 0)
+		{
+			if (status>>8 != 0 )
+			{
+				tst_resm(TFAIL, "Child exit status = %d", status>>8);
+				tst_exit();
+			}
+			count++;
+		}
+		else
+		{
+			if (errno != EINTR)
+			{
+				break;
+			}
+#ifdef DEBUG
+			tst_resm(TINFO,"Signal detected during wait");
+#endif
+		}
+	}
+	/* Make sure proper number of children exited */
+	if (count != nprocs)
+	{
+		tst_resm(TFAIL, "Wrong number of children exited, Saw %d, Expected %d", count, nprocs);
+		tst_exit();
+	}
+
+	tst_resm(TPASS,"msgctl10 ran successfully!");
+
+	cleanup();
+	return (0);
+
+}
+/*--------------------------------------------------------------------*/
+
+#ifdef UCLINUX
+void
+do_child_1_uclinux()
+{
+	procstat = 1;
+	exit(dotest(key_uclinux, i_uclinux));
+}
+
+void
+do_child_2_uclinux()
+{
+	exit(doreader(id_uclinux, key_uclinux % 255, child_process_uclinux));
+}
+#endif
+
+int dotest(key, child_process)
+key_t 	key;
+int	child_process;
+{
+	int id, pid;
+
+	sighold(SIGTERM);
+	TEST(msgget(key, IPC_CREAT | S_IRUSR | S_IWUSR));
+	if (TEST_RETURN < 0)
+	{
+		tst_resm(TFAIL, "Msgget error in child %d, errno = %d", child_process, TEST_ERRNO);
+		tst_exit();
+	}
+	tid = id = TEST_RETURN;
+	sigrelse(SIGTERM);
+
+	fflush(stdout);
+	if ((pid = FORK_OR_VFORK()) < 0)
+	{
+		tst_resm(TWARN, "\tFork failed (may be OK if under stress)");
+		TEST(msgctl(tid, IPC_RMID, 0));
+		if (TEST_RETURN < 0)
+		{
+			tst_resm(TFAIL, "Msgctl error in cleanup, errno = %d", errno);
+		}
+		tst_exit();
+	}
+	/* Child does this */
+	if (pid == 0)
+	{
+#ifdef UCLINUX
+		if (self_exec(argv0, "nddd", 2, id, key, child_process) < 0) {
+			tst_resm(TWARN, "self_exec failed");
+			TEST(msgctl(tid, IPC_RMID, 0));
+			if (TEST_RETURN < 0)
+			{
+				tst_resm(TFAIL, "\tMsgctl error in cleanup, "
+					"errno = %d\n", errno);
+			}
+			tst_exit();
+		}
+#else
+		exit( doreader(id, key % 255, child_process) );
+#endif
+	}
+	/* Parent does this */
+	mykid = pid;
+	procstat = 2;
+	dowriter(id, key % 255, child_process);
+	wait(0);
+	TEST(msgctl(id, IPC_RMID, 0));
+	if (TEST_RETURN < 0)
+	{
+		tst_resm(TFAIL, "msgctl errno %d", TEST_ERRNO);
+		tst_exit();
+	}
+	exit(PASS);
+}
+
+int doreader(id, key, child)
+int id, child;
+long key;
+{
+	int i, size;
+
+	for (i = 0; i < nreps; i++)
+	{
+		if ((size = msgrcv(id, &buffer, 100, 0, 0)) < 0)
+		{
+			tst_brkm(TBROK, cleanup, "Msgrcv error in child %d, read # = %d, errno = %d", (i + 1), child, errno);
+			tst_exit();
+		}
+		if (buffer.data.len + 1 != size)
+		{
+			tst_resm(TFAIL, "Size mismatch in child %d, read # = %d", child, (i + 1));
+			tst_resm(TFAIL, "for message size got  %d expected  %d %s",size ,buffer.data.len);
+			tst_exit();
+		}
+		if ( verify(buffer.data.pbytes, key, size - 1, child) )
+		{
+			tst_resm(TFAIL, "in read # = %d,key =  %x", (i + 1), child, key);
+			tst_exit();
+		}
+		key++;
+	}
+	return (0);
+}
+
+int dowriter(id, key, child)
+int id,child;
+long key;
+{
+	int i, size;
+
+	for (i = 0; i < nreps; i++)
+	{
+		do
+		{
+			size = (rand() % 99);
+		} while (size == 0);
+		fill_buffer(buffer.data.pbytes, key, size);
+		buffer.data.len = size;
+		buffer.type = 1;
+		TEST(msgsnd(id, &buffer, size + 1, 0));
+		if (TEST_RETURN < 0)
+		{
+			tst_brkm(TBROK, cleanup, "Msgsnd error in child %d, key =   %x errno  = %d", child, key, TEST_ERRNO);
+		}
+		key++;
+	}
+	return (0);
+}
+
+int fill_buffer(buf, val, size)
+register char *buf;
+char	val;
+register int size;
+{
+	register int i;
+
+	for(i = 0; i < size; i++)
+	{
+		buf[i] = val;
+	}
+
+	return (0);
+}
+
+
+/*
+ * verify()
+ *	Check a buffer for correct values.
+ */
+
+int verify(buf, val, size, child)
+	register char *buf;
+	char	val;
+	register int size;
+	int	child;
+{
+	while(size-- > 0)
+	{
+		if (*buf++ != val)
+		{
+			tst_resm(TWARN, "Verify error in child %d, *buf = %x, val = %x, size = %d", child, *buf, val, size);
+			return(FAIL);
+		}
+	}
+	return(PASS);
+}
+
+/*
+ *  * void
+ *  * sig_handler() - signal catching function for 'SIGUSR1' signal.
+ *  *
+ *  *   This is a null function and used only to catch the above signal
+ *  *   generated in parent process.
+ *  */
+void
+sig_handler()
+{
+}
+
+/***************************************************************
+ * setup() - performs all ONE TIME setup for this test.
+ *****************************************************************/
+void
+setup()
+{
+	int nr_msgqs;
+
+	tst_tmpdir();
+
+	/* You will want to enable some signal handling so you can capture
+	 * unexpected signals like SIGSEGV.
+	 */
+	tst_sig(FORK, DEF_HANDLER, cleanup);
+
+
+	/* Pause if that option was specified */
+	/* One cavet that hasn't been fixed yet.  TEST_PAUSE contains the code to
+	 * fork the test with the -c option.  You want to make sure you do this
+	 * before you create your temporary directory.
+	 */
+	TEST_PAUSE;
+
+	nr_msgqs = get_max_msgqueues();
+	if (nr_msgqs < 0)
+		cleanup();
+
+	MSGMNI = nr_msgqs - get_used_msgqueues();
+	if (MSGMNI <= 0){
+		tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
+		cleanup();
+	}
+}
+
+
+/***************************************************************
+ * cleanup() - performs all ONE TIME cleanup for this test at
+ *             completion or premature exit.
+ ****************************************************************/
+void
+cleanup()
+{
+	int status;
+	/*
+	 *  Remove the message queue from the system
+	 */
+#ifdef DEBUG
+	tst_resm(TINFO,"Removing the message queue");
+#endif
+	fflush (stdout);
+	(void) msgctl(tid, IPC_RMID, (struct msqid_ds *)NULL);
+	if ((status = msgctl(tid, IPC_STAT, (struct msqid_ds *)NULL)) != -1)
+	{
+		(void) msgctl(tid, IPC_RMID, (struct msqid_ds *)NULL);
+		tst_resm(TFAIL, "msgctl(tid, IPC_RMID) failed");
+		tst_exit();
+	}
+
+	fflush (stdout);
+	/*
+	 * print timing stats if that option was specified.
+	 * print errno log if that option was specified.
+	 */
+	TEST_CLEANUP;
+	tst_rmdir();
+	/* exit with return code appropriate for results */
+	tst_exit();
+}
+
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl11.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl11.c	2008-02-22 08:06:11.000000000 +0100
@@ -0,0 +1,696 @@
+/*
+ *
+ *   Copyright (c) International Business Machines  Corp., 2002
+ *
+ *   This program is free software;  you can redistribute it and/or modify
+ *   it under the terms of the GNU General Public License as published by
+ *   the Free Software Foundation; either version 2 of the License, or
+ *   (at your option) any later version.
+ *
+ *   This program is distributed in the hope that it will be useful,
+ *   but WITHOUT ANY WARRANTY;  without even the implied warranty of
+ *   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See
+ *   the GNU General Public License for more details.
+ *
+ *   You should have received a copy of the GNU General Public License
+ *   along with this program;  if not, write to the Free Software
+ *   Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+/* 06/30/2001	Port to Linux	nsharoff@us.ibm.com */
+/* 11/11/2002   Port to LTP     dbarrera@us.ibm.com */
+
+
+/*
+ * NAME
+ *	msgctl11
+ *
+ * CALLS
+ *	msgget(2) msgctl(2) msgop(2)
+ *
+ * ALGORITHM
+ *	Get and manipulate a message queue.
+ *	Same as msgctl09 but gets the actual msgmni value under procfs.
+ *
+ * RESTRICTIONS
+ *
+ */
+
+#define _XOPEN_SOURCE 500
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/ipc.h>
+#include <sys/msg.h>
+#include <sys/wait.h>
+#include <signal.h>
+#include <errno.h>
+#include <stdio.h>
+#include <string.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include "test.h"
+#include "usctest.h"
+#include "ipcmsg.h"
+
+#define MAXNREPS	1000
+#ifndef CONFIG_COLDFIRE
+#define MAXNPROCS	 1000000  /* This value is set to an arbitrary high limit. */
+#else
+#define MAXNPROCS	 100000   /* Coldfire can't deal with 1000000 */
+#endif
+#define MAXNKIDS	10
+#define FAIL		1
+#define PASS		0
+
+int dotest(key_t,int);
+int doreader(long,int,int);
+int dowriter(long,int,int);
+int fill_buffer(char*,char,int);
+int verify(char*,char,int,int);
+void setup();
+void cleanup();
+
+/*
+ * These globals must be defined in the test.
+ * */
+
+
+char *TCID="msgctl11";           /* Test program identifier.    */
+int TST_TOTAL=1;                /* Total number of test cases. */
+extern int Tst_count;           /* Test Case counter for tst_* routines */
+
+int exp_enos[]={0};     /* List must end with 0 */
+
+
+key_t	keyarray[MAXNPROCS];
+
+struct {
+	long	type;
+	struct {
+		char	len;
+		char	pbytes[99];
+		} data;
+	} buffer;
+
+int	pidarray[MAXNPROCS];
+int	rkidarray[MAXNKIDS];
+int	wkidarray[MAXNKIDS];
+int 	tid;
+int 	nprocs, nreps, nkids, MSGMNI;
+int 	procstat;
+void 	term(int);
+#ifdef UCLINUX
+static char *argv0;
+
+void do_child_1_uclinux();
+static key_t key_uclinux;
+static int i_uclinux;
+
+void do_child_2_uclinux();
+static int pid_uclinux;
+static int child_process_uclinux;
+
+void do_child_3_uclinux();
+static int rkid_uclinux;
+#endif
+void cleanup_msgqueue(int i, int tid);
+
+/*-----------------------------------------------------------------*/
+int main(argc, argv)
+int	argc;
+char	*argv[];
+{
+	register int i, j, ok, pid;
+	int count, status;
+
+#ifdef UCLINUX
+	char *msg;			/* message returned from parse_opts */
+
+	argv0 = argv[0];
+
+	/* parse standard options */
+	if ((msg = parse_opts(argc, argv, (option_t *)NULL, NULL)) != (char *)NULL)
+	{
+		tst_brkm(TBROK, cleanup, "OPTION PARSING ERROR - %s", msg);
+	}
+
+	maybe_run_child(&do_child_1_uclinux, "ndd", 1, &key_uclinux, &i_uclinux);
+	maybe_run_child(&do_child_2_uclinux, "nddd", 2, &key_uclinux,
+			&pid_uclinux, &child_process_uclinux);
+	maybe_run_child(&do_child_3_uclinux, "nddd", 3, &key_uclinux,
+			&rkid_uclinux, &child_process_uclinux);
+#endif
+
+	setup();
+
+	if (argc == 1 )
+	{
+		/* Set default parameters */
+		nreps = MAXNREPS;
+		nprocs = MSGMNI;
+		nkids = MAXNKIDS;
+	}
+	else if (argc == 4 )
+	{
+		if ( atoi(argv[1]) > MAXNREPS )
+		{
+			tst_resm(TCONF,"Requested number of iterations too large, setting to Max. of %d", MAXNREPS);
+			nreps = MAXNREPS;
+		}
+		else
+		{
+			nreps = atoi(argv[1]);
+		}
+		if (atoi(argv[2]) > MSGMNI )
+		{
+			tst_resm(TCONF,"Requested number of processes too large, setting to Max. of %d", MSGMNI);
+			nprocs = MSGMNI;
+		}
+		else
+		{
+			nprocs = atoi(argv[2]);
+		}
+		if (atoi(argv[3]) > MAXNKIDS )
+		{
+			tst_resm(TCONF,"Requested number of read/write pairs too large; setting to Max. of %d", MAXNKIDS);
+			nkids = MAXNKIDS;
+		}
+		else
+		{
+			nkids = atoi(argv[3]);
+		}
+	}
+	else
+	{
+		tst_resm(TCONF," Usage: %s [ number of iterations  number of processes number of read/write pairs ]", argv[0]);
+		tst_exit();
+	}
+
+	procstat = 0;
+	srand48((unsigned)getpid() + (unsigned)(getppid() << 16));
+	tid = -1;
+
+	/* Setup signal handleing routine */
+	if (sigset(SIGTERM, term) == SIG_ERR)
+	{
+		tst_resm(TFAIL, "Sigset SIGTERM failed");
+		tst_exit();
+	}
+	/* Set up array of unique keys for use in allocating message
+	 * queues
+	 */
+	for (i = 0; i < nprocs; i++)
+	{
+		ok = 1;
+		do
+		{
+			/* Get random key */
+			keyarray[i] = (key_t)lrand48();
+			/* Make sure key is unique and not private */
+			if (keyarray[i] == IPC_PRIVATE)
+			{
+				ok = 0;
+				continue;
+			}
+			for (j = 0; j < i; j++)
+			{
+				if (keyarray[j] == keyarray[i])
+				{
+					ok = 0;
+					break;
+				}
+				ok = 1;
+			}
+		} while (ok == 0);
+	}
+/*-----------------------------------------------------------------*/
+	/* Fork a number of processes (nprocs), each of which will
+	 * create a message queue with several (nkids) reader/writer
+	 * pairs which will read and write a number (iterations)
+	 * of random length messages with specific values (keys).
+	 */
+
+	for (i = 0; i <  nprocs; i++)
+	{
+		fflush(stdout);
+		if ((pid = FORK_OR_VFORK()) < 0)
+		{
+			tst_resm(TFAIL, "\tFork failed (may be OK if under stress)");
+			tst_exit();
+		}
+		/* Child does this */
+		if (pid == 0)
+		{
+#ifdef UCLINUX
+			if (self_exec(argv[0], "ndd", 1, keyarray[i], i) < 0)
+			{
+				tst_resm(TFAIL, "\tself_exec failed");
+				tst_exit();
+			}
+#else
+			procstat = 1;
+			exit( dotest(keyarray[i], i) );
+#endif
+		}
+		pidarray[i] = pid;
+	}
+
+	count = 0;
+	while(1)
+	{
+		if (( wait(&status)) > 0)
+		{
+			if (status>>8 != PASS )
+			{
+				tst_resm(TFAIL, "Child exit status = %d", status>>8);
+				tst_exit();
+			}
+			count++;
+		}
+		else
+		{
+			if (errno != EINTR)
+			{
+				break;
+			}
+#ifdef DEBUG
+			tst_resm(TINFO,"Signal detected during wait");
+#endif
+		}
+	}
+	/* Make sure proper number of children exited */
+	if (count != nprocs)
+	{
+		tst_resm(TFAIL, "Wrong number of children exited, Saw %d, Expected %d", count, nprocs);
+		tst_exit();
+	}
+
+	tst_resm(TPASS,"msgctl11 ran successfully!");
+
+	cleanup();
+
+	return (0);
+
+
+
+}
+/*--------------------------------------------------------------------*/
+
+#ifdef UCLINUX
+void
+do_child_1_uclinux()
+{
+	procstat = 1;
+	exit(dotest(key_uclinux, i_uclinux));
+}
+
+void
+do_child_2_uclinux()
+{
+	procstat = 2;
+	exit(doreader(key_uclinux, pid_uclinux, child_process_uclinux));
+}
+
+void
+do_child_3_uclinux()
+{
+	procstat = 2;
+	exit(dowriter(key_uclinux, rkid_uclinux, child_process_uclinux));
+}
+#endif
+
+void
+cleanup_msgqueue(int i, int tid)
+{
+	/*
+	 * Decrease the value of i by 1 because it
+	 * is getting incremented even if the fork
+	 * is failing.
+	 */
+
+	i--;
+	/*
+	 * Kill all children & free message queue.
+	 */
+	for (; i >= 0; i--) {
+		(void)kill(rkidarray[i], SIGKILL);
+		(void)kill(wkidarray[i], SIGKILL);
+	}
+
+	if (msgctl(tid, IPC_RMID, 0) < 0) {
+		tst_resm(TFAIL, "Msgctl error in cleanup, errno = %d", errno);
+		tst_exit();
+	}
+}
+
+int dotest(key, child_process)
+key_t 	key;
+int	child_process;
+{
+	int id, pid;
+	int i, count, status, exit_status;
+
+	sighold(SIGTERM);
+	if ((id = msgget(key, IPC_CREAT | S_IRUSR | S_IWUSR )) < 0)
+	{
+		tst_resm(TFAIL, "Msgget error in child %d, errno = %d", child_process, errno);
+		tst_exit();
+	}
+	tid = id;
+	sigrelse(SIGTERM);
+
+	exit_status = PASS;
+
+	for (i=0; i < nkids; i++)
+	{
+		fflush(stdout);
+		if ((pid = FORK_OR_VFORK()) < 0)
+		{
+			tst_resm(TWARN, "Fork failure in first child of child group %d", child_process);
+			cleanup_msgqueue(i, tid);
+			tst_exit();
+		}
+		/* First child does this */
+		if (pid == 0)
+		{
+#ifdef UCLINUX
+			if (self_exec(argv0, "nddd", 2, key, getpid(),
+							child_process) < 0) {
+				tst_resm(TWARN, "self_exec failed");
+				cleanup_msgqueue(i, tid);
+				tst_exit();
+			}
+#else
+			procstat = 2;
+			exit( doreader( key, getpid(), child_process) );
+#endif
+		}
+		rkidarray[i] = pid;
+		fflush(stdout);
+		if ((pid = FORK_OR_VFORK()) < 0)
+		{
+			tst_resm(TWARN, "Fork failure in first child of child group %d", child_process);
+			/*
+			 * Kill the reader child process
+			 */
+			(void)kill(rkidarray[i], SIGKILL);
+
+			cleanup_msgqueue(i, tid);
+			tst_exit();
+		}
+		/* Second child does this */
+		if (pid == 0)
+		{
+#ifdef UCLINUX
+			if (self_exec(argv0, "nddd", 3, key, rkidarray[i],
+							child_process) < 0) {
+				tst_resm(TWARN, "\tFork failure in first child "
+					"of child group %d \n", child_process);
+				/*
+				 * Kill the reader child process
+				 */
+				(void)kill(rkidarray[i], SIGKILL);
+
+				cleanup_msgqueue(i, tid);
+				tst_exit();
+			}
+#else
+			procstat = 2;
+			exit( dowriter( key, rkidarray[i], child_process) );
+#endif
+		}
+		wkidarray[i] = pid;
+	}
+	/* Parent does this */
+	count = 0;
+	while(1)
+	{
+		if (( wait(&status)) > 0)
+		{
+			if (status>>8 != PASS )
+			{
+				tst_resm(TFAIL, "Child exit status = %d from child group %d", status>>8, child_process);
+				for (i = 0; i < nkids; i++)
+				{
+					kill(rkidarray[i], SIGTERM);
+					kill(wkidarray[i], SIGTERM);
+				}
+				if (msgctl(tid, IPC_RMID, 0) < 0) {
+					tst_resm(TFAIL, "Msgctl error, errno = %d", errno);
+				}
+				tst_exit();
+			}
+			count++;
+		}
+		else
+		{
+			if (errno != EINTR)
+			{
+				break;
+			}
+		}
+	}
+	/* Make sure proper number of children exited */
+	if (count != (nkids * 2))
+	{
+		tst_resm(TFAIL, "Wrong number of children exited in child group %d, Saw %d Expected %d", child_process, count, (nkids * 2));
+		if (msgctl(tid, IPC_RMID, 0) < 0) {
+			tst_resm(TFAIL, "Msgctl error, errno = %d", errno);
+		}
+		tst_exit();
+	}
+	if (msgctl(id, IPC_RMID, 0) < 0)
+	{
+		tst_resm(TFAIL, "Msgctl failure in child group %d, errno %d", child_process, errno);
+		tst_exit();
+	}
+	exit(exit_status);
+}
+
+int doreader( key, type, child)
+int type, child;
+long key;
+{
+	int i, size;
+	int id;
+
+	if ((id = msgget(key, 0)) < 0)
+	{
+		tst_resm(TFAIL, "Msgget error in reader of child group %d, errno = %d", child, errno);
+		tst_exit();
+	}
+	if (id != tid)
+	{
+		tst_resm(TFAIL, "Message queue mismatch in reader of child group %d for message queue id %d", child, id);
+		tst_exit();
+	}
+	for (i = 0; i < nreps; i++)
+	{
+		if ((size = msgrcv(id, &buffer, 100, type, 0)) < 0)
+		{
+			tst_resm(TFAIL, "Msgrcv error in child %d, read # = %d, errno = %d", (i + 1), child, errno);
+			tst_exit();
+		}
+		if (buffer.type != type)
+		{
+			tst_resm(TFAIL, "Size mismatch in child %d, read # = %d", child, (i + 1));
+			tst_resm(TFAIL, "\tfor message size got  %d expected  %d %s",size ,buffer.data.len);
+			tst_exit();
+		}
+		if (buffer.data.len + 1 != size)
+		{
+			tst_resm(TFAIL, "Size mismatch in child %d, read # = %d, size = %d, expected = %d", child, (i + 1), buffer.data.len, size);
+			tst_exit();
+		}
+		if ( verify(buffer.data.pbytes, (key % 255), size - 1, child) )
+		{
+			tst_resm(TFAIL, "in read # = %d,key =  %x", (i + 1), child, key);
+			tst_exit();
+		}
+		key++;
+	}
+	exit(PASS);
+}
+
+int dowriter( key, type, child)
+int type,child;
+long key;
+{
+	int i, size;
+	int id;
+
+	if ((id = msgget(key, 0)) < 0)
+	{
+		tst_resm(TFAIL, "Msgget error in writer of child group %d, errno = %d", child, errno);
+		tst_exit();
+	}
+	if (id != tid)
+	{
+		tst_resm(TFAIL, "Message queue mismatch in writer of child group %d", child);
+		tst_resm(TFAIL, "\tfor message queue id %d expected  %d",id, tid);
+		tst_exit();
+	}
+
+	for (i = 0; i < nreps; i++)
+	{
+		do
+		{
+			size = (lrand48() % 99);
+		} while (size == 0);
+		fill_buffer(buffer.data.pbytes, (key % 255), size);
+		buffer.data.len = size;
+		buffer.type = type;
+		if (msgsnd(id, &buffer, size + 1, 0) < 0)
+		{
+			tst_resm(TFAIL, "Msgsnd error in child %d, key =   %x errno  = %d", child, key, errno);
+			tst_exit();
+		}
+		key++;
+	}
+	exit(PASS);
+}
+
+int fill_buffer(buf, val, size)
+register char *buf;
+char	val;
+register int size;
+{
+	register int i;
+
+	for(i = 0; i < size; i++)
+		buf[i] = val;
+	return(0);
+}
+
+
+/*
+ * verify()
+ *	Check a buffer for correct values.
+ */
+
+int verify(buf, val, size, child)
+	register char *buf;
+	char	val;
+	register int size;
+	int	child;
+{
+	while(size-- > 0)
+		if (*buf++ != val)
+		{
+			tst_resm(TWARN, "Verify error in child %d, *buf = %x, val = %x, size = %d", child, *buf, val, size);
+			return(FAIL);
+		}
+	return(PASS);
+}
+
+/* ARGSUSED */
+void
+term(int sig)
+{
+	int i;
+
+	if (procstat == 0)
+	{
+#ifdef DEBUG
+		tst_resm(TINFO,"SIGTERM signal received, test killing kids");
+#endif
+		for (i = 0; i < nprocs; i++)
+		{
+			if ( pidarray[i] > 0){
+				if ( kill(pidarray[i], SIGTERM) < 0)
+				{
+					tst_resm(TBROK,"Kill failed to kill child %d", i);
+					exit(FAIL);
+				}
+			}
+		}
+		return;
+	}
+
+	if (procstat == 2)
+	{
+		fflush(stdout);
+		exit(PASS);
+	}
+
+	if (tid == -1)
+	{
+		exit(FAIL);
+	}
+	for (i = 0; i < nkids; i++)
+	{
+		if (rkidarray[i] > 0)
+			kill(rkidarray[i], SIGTERM);
+		if (wkidarray[i] > 0)
+			kill(wkidarray[i], SIGTERM);
+	}
+}
+
+/***************************************************************
+ * setup() - performs all ONE TIME setup for this test.
+ *****************************************************************/
+void
+setup()
+{
+	int nr_msgqs;
+
+	tst_tmpdir();
+	/* You will want to enable some signal handling so you can capture
+	 * unexpected signals like SIGSEGV.
+	 */
+	tst_sig(FORK, DEF_HANDLER, cleanup);
+
+
+	/* Pause if that option was specified */
+	/* One cavet that hasn't been fixed yet.  TEST_PAUSE contains the code to
+	 * fork the test with the -c option.  You want to make sure you do this
+	 * before you create your temporary directory.
+	 */
+	TEST_PAUSE;
+
+	nr_msgqs = get_max_msgqueues();
+	if (nr_msgqs < 0)
+		cleanup();
+
+	MSGMNI = nr_msgqs - get_used_msgqueues();
+	if (MSGMNI <= 0){
+		tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
+		cleanup();
+	}
+}
+
+
+/***************************************************************
+ * cleanup() - performs all ONE TIME cleanup for this test at
+ * completion or premature exit.
+ ****************************************************************/
+void
+cleanup()
+{
+	int status;
+	/*
+	 * print timing stats if that option was specified.
+	 * print errno log if that option was specified.
+	 */
+	TEST_CLEANUP;
+
+	/*
+	 * Remove the message queue from the system
+	 */
+#ifdef DEBUG
+	tst_resm(TINFO,"Removing the message queue");
+#endif
+	fflush (stdout);
+	(void) msgctl(tid, IPC_RMID, (struct msqid_ds *)NULL);
+	if ((status = msgctl(tid, IPC_STAT, (struct msqid_ds *)NULL)) != -1) {
+		(void) msgctl(tid, IPC_RMID, (struct msqid_ds *)NULL);
+		tst_resm(TFAIL, "msgctl(tid, IPC_RMID) failed");
+		tst_exit();
+	}
+
+	fflush (stdout);
+	tst_rmdir();
+	/* exit with return code appropriate for results */
+	tst_exit();
+}
+
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c	2008-02-22 07:57:47.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl08.c	2008-02-22 08:08:12.000000000 +0100
@@ -455,37 +455,14 @@ sig_handler()
 {
 }
 
-#define BUFSIZE 512
-
-/** Get the number of message queues already in use */
-static int get_used_msgqueues()
-{
-        FILE *f;
-        int used_queues;
-        char buff[BUFSIZE];
-
-        f = popen("ipcs -q", "r");
-        if (!f) {
-                tst_resm(TBROK,"Could not run 'ipcs' to calculate used message queues");
-                tst_exit();
-        }
-        /* FIXME: Start at -4 because ipcs prints four lines of header */
-        for (used_queues = -4; fgets(buff, BUFSIZE, f); used_queues++)
-                ;
-        pclose(f);
-        if (used_queues < 0) {
-                tst_resm(TBROK,"Could not read output of 'ipcs' to calculate used message queues");
-                tst_exit();
-        }
-        return used_queues;
-}
-
 /***************************************************************
  * setup() - performs all ONE TIME setup for this test.
  *****************************************************************/
 void
 setup()
 {
+	int nr_msgqs;
+
 	tst_tmpdir();
  
         /* You will want to enable some signal handling so you can capture
@@ -501,11 +478,22 @@ setup()
 	 */
         TEST_PAUSE;
 
-        MSGMNI = MAX_MSGQUEUES - get_used_msgqueues();
-	if (MSGMNI <= 0){
+	nr_msgqs = get_max_msgqueues();
+	if (nr_msgqs < 0)
+		cleanup();
+
+	nr_msgqs -= get_used_msgqueues();
+	if (nr_msgqs <= 0){
 		tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
 		cleanup(); 
 	}	
+
+	/*
+	 * Since msgmni scales to the memory size, it may reach huge values
+	 * that are not necessary for this test.
+	 * That's why we define NR_MSGQUEUES as a high boundary for it.
+	 */
+	MSGMNI = min(nr_msgqs, NR_MSGQUEUES);
 }
 
 
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c	2008-02-22 07:57:47.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgctl/msgctl09.c	2008-02-22 08:09:44.000000000 +0100
@@ -625,37 +625,14 @@ term(int sig)
 	}
 }
 
-#define BUFSIZE 512
-
-/** Get the number of message queues already in use */
-static int get_used_msgqueues()
-{
-        FILE *f;
-        int used_queues;
-        char buff[BUFSIZE];
-
-        f = popen("ipcs -q", "r");
-        if (!f) {
-                tst_resm(TBROK,"Could not run 'ipcs' to calculate used message queues");
-                tst_exit();
-        }
-        /* FIXME: Start at -4 because ipcs prints four lines of header */
-        for (used_queues = -4; fgets(buff, BUFSIZE, f); used_queues++)
-                ;
-        pclose(f);
-        if (used_queues < 0) {
-                tst_resm(TBROK,"Could not read output of 'ipcs' to calculate used message queues");
-                tst_exit();
-        }
-        return used_queues;
-}
-
 /***************************************************************
  * setup() - performs all ONE TIME setup for this test.
  *****************************************************************/
 void
 setup()
 {
+	int nr_msgqs;
+
 	tst_tmpdir();
         /* You will want to enable some signal handling so you can capture
          * unexpected signals like SIGSEGV.
@@ -670,11 +647,22 @@ setup()
          */
         TEST_PAUSE;
 
-        MSGMNI = MAX_MSGQUEUES - get_used_msgqueues();
-        if (MSGMNI <= 0){
+        nr_msgqs = get_max_msgqueues();
+	if (nr_msgqs < 0)
+		cleanup();
+
+	nr_msgqs -= get_used_msgqueues();
+	if (nr_msgqs <= 0) {
                 tst_resm(TBROK,"Max number of message queues already used, cannot create more.");
                 cleanup();
         }
+
+	/*
+	 * Since msgmni scales to the memory size, it may reach huge values
+	 * that are not necessary for this test.
+	 * That's why we define NR_MSGQUEUES as a high boundary for it.
+	 */
+	MSGMNI = min(nr_msgqs, NR_MSGQUEUES);
 }
 
 
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgget/msgget03.c
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgget/msgget03.c	2008-02-22 07:57:47.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgget/msgget03.c	2008-02-22 08:13:22.000000000 +0100
@@ -68,8 +68,6 @@ int exp_enos[] = {ENOSPC, 0};	/* 0 termi
 int *msg_q_arr = NULL;		/* hold the id's that we create */
 int num_queue = 0;		/* count the queues created */
 
-static int get_max_msgqueues();
-
 int main(int ac, char **av)
 {
 	int lc;				/* loop counter */
@@ -121,24 +119,6 @@ int main(int ac, char **av)
 	return(0);
 }
 
-/** Get the max number of message queues allowed on system */
-int get_max_msgqueues()
-{
-        FILE *f;
-        char buff[512];
-
-        /* Get the max number of message queues allowed on system */
-        f = fopen("/proc/sys/kernel/msgmni", "r");
-        if (!f){
-                tst_brkm(TBROK, cleanup, "Could not open /proc/sys/kernel/msgmni");
-        }
-        if (!fgets(buff, 512, f)) {
-                tst_brkm(TBROK, cleanup, "Could not read /proc/sys/kernel/msgmni");
-        }
-        fclose(f);
-        return atoi(buff);
-}
-
 /*
  * setup() - performs all the ONE TIME setup for this test.
  */
@@ -166,6 +146,8 @@ setup(void)
 	msgkey = getipckey();
 
 	maxmsgs = get_max_msgqueues();
+	if (maxmsgs < 0)
+		tst_brkm(TBROK, cleanup, "");
 
 	msg_q_arr = (int *)calloc(maxmsgs, sizeof (int));
 	if (msg_q_arr == NULL) {
Index: ltp-full-20080131/testcases/kernel/syscalls/ipc/msgget/Makefile
===================================================================
--- ltp-full-20080131.orig/testcases/kernel/syscalls/ipc/msgget/Makefile	2008-02-22 07:57:47.000000000 +0100
+++ ltp-full-20080131/testcases/kernel/syscalls/ipc/msgget/Makefile	2008-02-22 08:14:00.000000000 +0100
@@ -21,9 +21,12 @@ LDLIBS += -L../../../../../lib -lltp -L.
 
 SRCS    = $(wildcard *.c)
 TARGETS = $(patsubst %.c,%,$(SRCS))
+LIBIPC_HEADERS	= ../lib/ipcmsg.h
 
 all: $(TARGETS)
 
+$(TARGETS):	$(LIBIPC_HEADERS)
+
 install:
 	@set -e; for i in $(TARGETS); do ln -f $$i ../../../../bin/$$i ; done
 

Re: [LTP] [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Subrata Modak]

On Fri, 2008-02-22 at 07:25 +0100, Nadia Derbey wrote:
> Subrata Modak wrote:
> >>Nadia Derbey wrote:
> >>
> >>>Matt Helsley wrote:
> >>>
> >>>
> >>>>On Tue, 2008-02-19 at 18:16 +0100, Nadia Derbey wrote:
> >>>>
> >>>><snip>
> >>>>
> >>>>>+#define MAX_MSGQUEUES  16      /* MSGMNI as defined in linux/msg.h */
> >>>>>+
> >>>>
> >>>>
> >>>>
> >>>>It's not quite the maximum anymore, is it? More like the minumum
> >>>>maximum ;). A better name might better document what the test is
> >>>>actually trying to do.
> >>>>
> >>>>One question I have is whether the unpatched test is still valuable.
> >>>>Based on my limited knowledge of the test I suspect it's still a correct
> >>>>test of message queues. If so, perhaps renaming the old test (so it's
> >>>>not confused with a performance regression) and adding your patched
> >>>>version is best?
> >>>>
> >>>
> >>>So, here's the new patch based on Matt's points.
> >>>
> >>>Subrata, it has to be applied on top of the original ltp-full-20080131. 
> >>>Please tell me if you'd prefer one based on the merged version you've 
> >>>got (i.e. with my Tuesday patch applied).
> > 
> > 
> > Nadia, I would prefer Patch on the top of the already merged version (on
> > top of latest CVS snapshot as of today). Anyways, thanks for all these
> > effort :-)
> > 
> > --Subrata
> > 
> 
> In attachment, you'll find a patch to apply on top of the patches I sent 
> you on Tuesday.

Nadia,

Thanks a ton for that. The same has been merged.

Regards--
Subrata

> 
> Regards,
> Nadia

Re: [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Tony Luck]

On Mon, Feb 11, 2008 at 7:16 AM,  <Nadia.Derbey@bull.net> wrote:
>  Index: linux-2.6.24-mm1/ipc/msg.c
>  ===================================================================
>  --- linux-2.6.24-mm1.orig/ipc/msg.c     2008-02-07 15:02:29.000000000 +0100
>  +++ linux-2.6.24-mm1/ipc/msg.c  2008-02-07 15:24:19.000000000 +0100
...
>  +out_callback:
>  +
>  +       printk(KERN_INFO "msgmni has been set to %d for ipc namespace %p\n",
>  +               ns->msg_ctlmni, ns);
>  +}

This patch has now made its way to mainline.  I can see how this printk
was really useful to you while developing this patch. But does it add
much value in a production system? It just looks like another piece of
clutter on the console to my uncontainerized eyes.

-Tony

Re: [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Nadia Derbey]

Tony Luck wrote:
> On Mon, Feb 11, 2008 at 7:16 AM,  <Nadia.Derbey@bull.net> wrote:
> 
>> Index: linux-2.6.24-mm1/ipc/msg.c
>> ===================================================================
>> --- linux-2.6.24-mm1.orig/ipc/msg.c     2008-02-07 15:02:29.000000000 +0100
>> +++ linux-2.6.24-mm1/ipc/msg.c  2008-02-07 15:24:19.000000000 +0100
> 
> ...
> 
>> +out_callback:
>> +
>> +       printk(KERN_INFO "msgmni has been set to %d for ipc namespace %p\n",
>> +               ns->msg_ctlmni, ns);
>> +}
> 
> 
> This patch has now made its way to mainline.  I can see how this printk
> was really useful to you while developing this patch. But does it add
> much value in a production system? It just looks like another piece of
> clutter on the console to my uncontainerized eyes.
> 
> -Tony
> 
> 


Well, this printk had been suggested by somebody (sorry I don't remember 
who) when I first submitted the patch. Actually I think it might be 
useful for a sysadmin to be aware of a change in the msgmni value: we 
have the message not only at boot time, but also each time msgmni is 
recomputed because of a change in the amount of memory.
Also, at boot time, I think it's interesting to have the actual msgmni 
value: it used to unconditionally be set to 16. Some applications that 
used to need an initialization script setting msgmni to a higher value 
might not need that script anymore, since the new value might fit their 
needs.

Regards,
Nadia

RE: [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Luck, Tony]

> Well, this printk had been suggested by somebody (sorry I don't remember 
> who) when I first submitted the patch. Actually I think it might be 
> useful for a sysadmin to be aware of a change in the msgmni value: we 
> have the message not only at boot time, but also each time msgmni is 
> recomputed because of a change in the amount of memory.

If the message is directed at the system administrator, then it would
be nice if there were some more meaningful way to show the namespace
that is affected than just printing the hex address of the kernel structure.

As the sysadmin for my test systems, printing the hex address is mildly
annoying ... I now have to add a new case to my scripts that look at
dmesg output for unusual activity.

Is there some better "name for a namespace" than the address? Perhaps
the process id of the process that instantiated the namespace???

-Tony

Re: [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Serge E. Hallyn]

Quoting Luck, Tony (tony.luck@intel.com):
> > Well, this printk had been suggested by somebody (sorry I don't remember 
> > who) when I first submitted the patch. Actually I think it might be 
> > useful for a sysadmin to be aware of a change in the msgmni value: we 
> > have the message not only at boot time, but also each time msgmni is 
> > recomputed because of a change in the amount of memory.
> 
> If the message is directed at the system administrator, then it would
> be nice if there were some more meaningful way to show the namespace
> that is affected than just printing the hex address of the kernel structure.
> 
> As the sysadmin for my test systems, printing the hex address is mildly
> annoying ... I now have to add a new case to my scripts that look at
> dmesg output for unusual activity.
> 
> Is there some better "name for a namespace" than the address? Perhaps
> the process id of the process that instantiated the namespace???

I agree with Tony here.  Aside from the nuisance it is to see that
message on console every time I unshare a namespace, a printk doesn't
seem like the right way to output the info.  At most I'd say an audit
message.

-serge

Re: [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Nadia Derbey]

Luck, Tony wrote:
>>Well, this printk had been suggested by somebody (sorry I don't remember 
>>who) when I first submitted the patch. Actually I think it might be 
>>useful for a sysadmin to be aware of a change in the msgmni value: we 
>>have the message not only at boot time, but also each time msgmni is 
>>recomputed because of a change in the amount of memory.
> 
> 
> If the message is directed at the system administrator, then it would
> be nice if there were some more meaningful way to show the namespace
> that is affected than just printing the hex address of the kernel structure.
> 
> As the sysadmin for my test systems, printing the hex address is mildly
> annoying ... I now have to add a new case to my scripts that look at
> dmesg output for unusual activity.
> 
> Is there some better "name for a namespace" than the address? Perhaps
> the process id of the process that instantiated the namespace???
> 

Unfortunately no when we are inside an ipc namespace, we don't have such 
interesting informations. But I agree with you, an address is not 
readable enough. I'll try to find a solution.

Regards,
Nadia

Re: [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Nadia Derbey]

Serge E. Hallyn wrote:
> Quoting Luck, Tony (tony.luck@intel.com):
> 
>>>Well, this printk had been suggested by somebody (sorry I don't remember 
>>>who) when I first submitted the patch. Actually I think it might be 
>>>useful for a sysadmin to be aware of a change in the msgmni value: we 
>>>have the message not only at boot time, but also each time msgmni is 
>>>recomputed because of a change in the amount of memory.
>>
>>If the message is directed at the system administrator, then it would
>>be nice if there were some more meaningful way to show the namespace
>>that is affected than just printing the hex address of the kernel structure.
>>
>>As the sysadmin for my test systems, printing the hex address is mildly
>>annoying ... I now have to add a new case to my scripts that look at
>>dmesg output for unusual activity.
>>
>>Is there some better "name for a namespace" than the address? Perhaps
>>the process id of the process that instantiated the namespace???
> 
> 
> I agree with Tony here.  Aside from the nuisance it is to see that
> message on console every time I unshare a namespace, a printk doesn't
> seem like the right way to output the info.

But you agree that this is happening only because you're doing tests 
related to namespaces, right?
I don't think that in a "standard" configuration this will happen very 
frequently, but may be I'm wrong.

>  At most I'd say an audit
> message.
> 

That's a good idea. Thanks, Serge. I'll do that.

Regards,
Nadia

Re: [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Serge E. Hallyn]

Quoting Nadia Derbey (Nadia.Derbey@bull.net):
> Serge E. Hallyn wrote:
>> Quoting Luck, Tony (tony.luck@intel.com):
>>>> Well, this printk had been suggested by somebody (sorry I don't remember 
>>>> who) when I first submitted the patch. Actually I think it might be 
>>>> useful for a sysadmin to be aware of a change in the msgmni value: we 
>>>> have the message not only at boot time, but also each time msgmni is 
>>>> recomputed because of a change in the amount of memory.
>>>
>>> If the message is directed at the system administrator, then it would
>>> be nice if there were some more meaningful way to show the namespace
>>> that is affected than just printing the hex address of the kernel 
>>> structure.
>>>
>>> As the sysadmin for my test systems, printing the hex address is mildly
>>> annoying ... I now have to add a new case to my scripts that look at
>>> dmesg output for unusual activity.
>>>
>>> Is there some better "name for a namespace" than the address? Perhaps
>>> the process id of the process that instantiated the namespace???
>> I agree with Tony here.  Aside from the nuisance it is to see that
>> message on console every time I unshare a namespace, a printk doesn't
>> seem like the right way to output the info.
>
> But you agree that this is happening only because you're doing tests 
> related to namespaces, right?

Yup :)

> I don't think that in a "standard" configuration this will happen very 
> frequently, but may be I'm wrong.
>
>>  At most I'd say an audit
>> message.
>
> That's a good idea. Thanks, Serge. I'll do that.

It'll probably still end up a printk for me, but it'll be my own fault
for not setting up audit.

> Regards,
> Nadia

thanks,
-serge

Re: [PATCH 1/8] Scaling msgmni to the amount of lowmem

[Matt Helsley]

On Wed, 2008-05-07 at 08:17 -0500, Serge E. Hallyn wrote:
> Quoting Nadia Derbey (Nadia.Derbey@bull.net):
> > Serge E. Hallyn wrote:
> >> Quoting Luck, Tony (tony.luck@intel.com):
> >>>> Well, this printk had been suggested by somebody (sorry I don't remember 
> >>>> who) when I first submitted the patch. Actually I think it might be 
> >>>> useful for a sysadmin to be aware of a change in the msgmni value: we 
> >>>> have the message not only at boot time, but also each time msgmni is 
> >>>> recomputed because of a change in the amount of memory.
> >>>
> >>> If the message is directed at the system administrator, then it would
> >>> be nice if there were some more meaningful way to show the namespace
> >>> that is affected than just printing the hex address of the kernel 
> >>> structure.
> >>>
> >>> As the sysadmin for my test systems, printing the hex address is mildly
> >>> annoying ... I now have to add a new case to my scripts that look at
> >>> dmesg output for unusual activity.
> >>>
> >>> Is there some better "name for a namespace" than the address? Perhaps
> >>> the process id of the process that instantiated the namespace???
> >> I agree with Tony here.  Aside from the nuisance it is to see that
> >> message on console every time I unshare a namespace, a printk doesn't
> >> seem like the right way to output the info.
> >
> > But you agree that this is happening only because you're doing tests 
> > related to namespaces, right?
> 
> Yup :)
> 
> > I don't think that in a "standard" configuration this will happen very 
> > frequently, but may be I'm wrong.
> >
> >>  At most I'd say an audit
> >> message.
>
> > That's a good idea. Thanks, Serge. I'll do that.

	I'm not familiar with kernel policies regarding audit messages. Are
audit messages treated anything like kernel interfaces when it comes to
removing/changing them?

Cheers,
	-Matt Helsley