linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH v10 0/4] ipc: Increase IPCMNI limit & IPC id generation modes
@ 2018-11-05 15:43 Waiman Long
  2018-11-05 15:43 ` [PATCH v10 1/4] ipc: Allow boot time extension of IPCMNI from 32k to 8M Waiman Long
                   ` (3 more replies)
  0 siblings, 4 replies; 10+ messages in thread
From: Waiman Long @ 2018-11-05 15:43 UTC (permalink / raw)
  To: Luis R. Rodriguez, Kees Cook, Andrew Morton, Jonathan Corbet
  Cc: linux-kernel, linux-fsdevel, linux-doc, Al Viro, Matthew Wilcox,
	Eric W. Biederman, Takashi Iwai, Davidlohr Bueso, Manfred Spraul,
	Waiman Long

v9->v10:
 - Drop v9 patches 1 & 2 because they have been merged upstream.
 - Minor twist to fix checkpatch warnings.
 - Add a new ipcid_mode sysctl parameter (new patches 3 & 4) to control
   how the IPC ids are being generated.

v7 patch: https://lkml.org/lkml/2018/5/7/666
v8 patch: https://lkml.org/lkml/2018/6/18/706
v9 patch: https://lkml.org/lkml/2018/9/7/1141  

There are users out there requesting increase in the IPCMNI value to
more than 32k. This patchset does that by using a boot kernel parameter
"ipcmni_extend" to increase the IPCMNI limit from 32k to 8M when that
boot command line option is specified.

To reduce the chance of id reuse, a new sysctl parameter ipcid_mode is
added for changing the way an IPC id is being generated. Currently,
three modes are supported - legacy, delete and cyclic. They differs
in the way the identifier and sequence number within an id are being
generated. This new mode is per IPC namespace.

The IPC id mode can be changed dynamically at run time, but extending
the IPCMNI number can only be done at boot time.

Patch 1 adds a "ipcmni_extend" boot command line parameter to extend
the IPCMNI limit from 32k to 8M.

Patch 2 changes how the sequence number within an id is being generated
to reduce the chance of id reuse with "ipcmni_extend".

Patch 3 decouples the new sequence generation mode into a new sysctl
parameter "ipcid_mode" so that it can be used even if "ipcmni_extend"
isn't specified.

Patch 4 adds a cyclic mode to "ipcid_mode" to allocate identifier
cyclically as well.

Waiman Long (4):
  ipc: Allow boot time extension of IPCMNI from 32k to 8M
  ipc: Conserve sequence numbers in extended IPCMNI mode
  ipc: Make the new sequence number generation mode available to all
  ipc: Add a cyclic mode for id generation

 Documentation/admin-guide/kernel-parameters.txt |  3 ++
 Documentation/sysctl/kernel.txt                 | 23 ++++++++++++
 include/linux/ipc_namespace.h                   | 14 +++++++
 ipc/ipc_sysctl.c                                | 25 ++++++++++++-
 ipc/msg.c                                       |  3 +-
 ipc/namespace.c                                 |  2 +
 ipc/sem.c                                       |  3 +-
 ipc/shm.c                                       |  3 +-
 ipc/util.c                                      | 41 +++++++++++++++------
 ipc/util.h                                      | 49 ++++++++++++++++++++-----
 10 files changed, 140 insertions(+), 26 deletions(-)

-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 10+ messages in thread

* [PATCH v10 1/4] ipc: Allow boot time extension of IPCMNI from 32k to 8M
  2018-11-05 15:43 [PATCH v10 0/4] ipc: Increase IPCMNI limit & IPC id generation modes Waiman Long
@ 2018-11-05 15:43 ` Waiman Long
  2018-11-06 13:20   ` Matthew Wilcox
  2018-11-05 15:43 ` [PATCH v10 2/4] ipc: Conserve sequence numbers in extended IPCMNI mode Waiman Long
                   ` (2 subsequent siblings)
  3 siblings, 1 reply; 10+ messages in thread
From: Waiman Long @ 2018-11-05 15:43 UTC (permalink / raw)
  To: Luis R. Rodriguez, Kees Cook, Andrew Morton, Jonathan Corbet
  Cc: linux-kernel, linux-fsdevel, linux-doc, Al Viro, Matthew Wilcox,
	Eric W. Biederman, Takashi Iwai, Davidlohr Bueso, Manfred Spraul,
	Waiman Long

The maximum number of unique System V IPC identifiers was limited to
32k.  That limit should be big enough for most use cases.

However, there are some users out there requesting for more, especially
those that are migrating from Solaris which uses 24 bits for unique
identifiers. To satisfy the need of those users, a new boot time kernel
option "ipcmni_extend" is added to extend the IPCMNI value to 8M. This
is a 256X increase which hopefully is big enough for them.

The use of this new option will change the pattern of the IPC identifiers
returned by functions like shmget(2). An application that depends on
such pattern may not work properly.  So it should only be used if the
users really need more than 32k of unique IPC numbers.

This new option does have the side effect of reducing the maximum number
of unique sequence numbers from 64k down to 256. So it is a trade-off.

The computation of a new IPC id is not done in the performance critical
path.  So a little bit of additional overhead shouldn't have any real
performance impact.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 Documentation/admin-guide/kernel-parameters.txt |  3 ++
 ipc/ipc_sysctl.c                                | 12 ++++++-
 ipc/util.c                                      | 10 +++---
 ipc/util.h                                      | 44 ++++++++++++++++++++-----
 4 files changed, 54 insertions(+), 15 deletions(-)

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index b90fe3b..0449e0c 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1795,6 +1795,9 @@
 	ip=		[IP_PNP]
 			See Documentation/filesystems/nfs/nfsroot.txt.
 
+	ipcmni_extend	[KNL] Extend the maximum number of unique System V
+			IPC identifiers from 32,768 to 8,388,608.
+
 	irqaffinity=	[SMP] Set the default irq affinity mask
 			The argument is a cpu list, as described above.
 
diff --git a/ipc/ipc_sysctl.c b/ipc/ipc_sysctl.c
index 49f9bf4..73b7782 100644
--- a/ipc/ipc_sysctl.c
+++ b/ipc/ipc_sysctl.c
@@ -120,7 +120,8 @@ static int proc_ipc_sem_dointvec(struct ctl_table *table, int write,
 static int zero;
 static int one = 1;
 static int int_max = INT_MAX;
-static int ipc_mni = IPCMNI;
+int ipc_mni = IPCMNI;
+int ipc_mni_shift = IPCMNI_SHIFT;
 
 static struct ctl_table ipc_kern_table[] = {
 	{
@@ -246,3 +247,12 @@ static int __init ipc_sysctl_init(void)
 }
 
 device_initcall(ipc_sysctl_init);
+
+static int __init ipc_mni_extend(char *str)
+{
+	ipc_mni = IPCMNI_EXTEND;
+	ipc_mni_shift = IPCMNI_EXTEND_SHIFT;
+	pr_info("IPCMNI extended to %d.\n", ipc_mni);
+	return 0;
+}
+early_param("ipcmni_extend", ipc_mni_extend);
diff --git a/ipc/util.c b/ipc/util.c
index 0af0575..07ae117 100644
--- a/ipc/util.c
+++ b/ipc/util.c
@@ -110,7 +110,7 @@ static int __init ipc_init(void)
  * @ids: ipc identifier set
  *
  * Set up the sequence range to use for the ipc identifier range (limited
- * below IPCMNI) then initialise the keys hashtable and ids idr.
+ * below ipc_mni) then initialise the keys hashtable and ids idr.
  */
 void ipc_init_ids(struct ipc_ids *ids)
 {
@@ -226,7 +226,7 @@ static inline int ipc_idr_alloc(struct ipc_ids *ids, struct kern_ipc_perm *new)
 				0, GFP_NOWAIT);
 	}
 	if (idx >= 0)
-		new->id = SEQ_MULTIPLIER * new->seq + idx;
+		new->id = (new->seq << IPCMNI_SEQ_SHIFT) + idx;
 	return idx;
 }
 
@@ -254,8 +254,8 @@ int ipc_addid(struct ipc_ids *ids, struct kern_ipc_perm *new, int limit)
 	/* 1) Initialize the refcount so that ipc_rcu_putref works */
 	refcount_set(&new->refcount, 1);
 
-	if (limit > IPCMNI)
-		limit = IPCMNI;
+	if (limit > ipc_mni)
+		limit = ipc_mni;
 
 	if (ids->in_use >= limit)
 		return -ENOSPC;
@@ -738,7 +738,7 @@ static struct kern_ipc_perm *sysvipc_find_ipc(struct ipc_ids *ids, loff_t pos,
 	if (total >= ids->in_use)
 		return NULL;
 
-	for (; pos < IPCMNI; pos++) {
+	for (; pos < ipc_mni; pos++) {
 		ipc = idr_find(&ids->ipcs_idr, pos);
 		if (ipc != NULL) {
 			*new_pos = pos + 1;
diff --git a/ipc/util.h b/ipc/util.h
index d768fdb..640f916 100644
--- a/ipc/util.h
+++ b/ipc/util.h
@@ -15,8 +15,34 @@
 #include <linux/err.h>
 #include <linux/ipc_namespace.h>
 
-#define IPCMNI 32768  /* <= MAX_INT limit for ipc arrays (including sysctl changes) */
-#define SEQ_MULTIPLIER	(IPCMNI)
+/*
+ * The IPC ID contains 2 separate numbers - index and sequence number.
+ * By default,
+ *   bits  0-14: index (32k, 15 bits)
+ *   bits 15-30: sequence number (64k, 16 bits)
+ *
+ * When IPCMNI extension mode is turned on, the composition changes:
+ *   bits  0-22: index (8M, 23 bits)
+ *   bits 23-30: sequence number (256, 8 bits)
+ */
+#define IPCMNI_SHIFT		15
+#define IPCMNI_EXTEND_SHIFT	23
+#define IPCMNI			(1 << IPCMNI_SHIFT)
+#define IPCMNI_EXTEND		(1 << IPCMNI_EXTEND_SHIFT)
+
+#ifdef CONFIG_SYSVIPC_SYSCTL
+extern int ipc_mni;
+extern int ipc_mni_shift;
+
+#define IPCMNI_SEQ_SHIFT	ipc_mni_shift
+#define IPCMNI_IDX_MASK		((1 << ipc_mni_shift) - 1)
+
+#else /* CONFIG_SYSVIPC_SYSCTL */
+
+#define ipc_mni		IPCMNI
+#define IPCMNI_SEQ_SHIFT	IPCMNI_SHIFT
+#define IPCMNI_IDX_MASK		((1 << IPCMNI_SHIFT) - 1)
+#endif /* CONFIG_SYSVIPC_SYSCTL */
 
 void sem_init(void);
 void msg_init(void);
@@ -96,9 +122,9 @@ void __init ipc_init_proc_interface(const char *path, const char *header,
 #define IPC_MSG_IDS	1
 #define IPC_SHM_IDS	2
 
-#define ipcid_to_idx(id) ((id) % SEQ_MULTIPLIER)
-#define ipcid_to_seqx(id) ((id) / SEQ_MULTIPLIER)
-#define IPCID_SEQ_MAX min_t(int, INT_MAX/SEQ_MULTIPLIER, USHRT_MAX)
+#define ipcid_to_idx(id)  ((id) & IPCMNI_IDX_MASK)
+#define ipcid_to_seqx(id) ((id) >> IPCMNI_SEQ_SHIFT)
+#define IPCID_SEQ_MAX	  (INT_MAX >> IPCMNI_SEQ_SHIFT)
 
 /* must be called with ids->rwsem acquired for writing */
 int ipc_addid(struct ipc_ids *, struct kern_ipc_perm *, int);
@@ -123,8 +149,8 @@ static inline int ipc_get_maxidx(struct ipc_ids *ids)
 	if (ids->in_use == 0)
 		return -1;
 
-	if (ids->in_use == IPCMNI)
-		return IPCMNI - 1;
+	if (ids->in_use == ipc_mni)
+		return ipc_mni - 1;
 
 	return ids->max_idx;
 }
@@ -219,10 +245,10 @@ void free_ipcs(struct ipc_namespace *ns, struct ipc_ids *ids,
 
 static inline int sem_check_semmni(struct ipc_namespace *ns) {
 	/*
-	 * Check semmni range [0, IPCMNI]
+	 * Check semmni range [0, ipc_mni]
 	 * semmni is the last element of sem_ctls[4] array
 	 */
-	return ((ns->sem_ctls[3] < 0) || (ns->sem_ctls[3] > IPCMNI))
+	return ((ns->sem_ctls[3] < 0) || (ns->sem_ctls[3] > ipc_mni))
 		? -ERANGE : 0;
 }
 
-- 
1.8.3.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH v10 2/4] ipc: Conserve sequence numbers in extended IPCMNI mode
  2018-11-05 15:43 [PATCH v10 0/4] ipc: Increase IPCMNI limit & IPC id generation modes Waiman Long
  2018-11-05 15:43 ` [PATCH v10 1/4] ipc: Allow boot time extension of IPCMNI from 32k to 8M Waiman Long
@ 2018-11-05 15:43 ` Waiman Long
  2018-11-05 15:43 ` [PATCH v10 3/4] ipc: Make the new sequence number generation mode available to all Waiman Long
  2018-11-05 15:43 ` [PATCH v10 4/4] ipc: Add a cyclic mode for id generation Waiman Long
  3 siblings, 0 replies; 10+ messages in thread
From: Waiman Long @ 2018-11-05 15:43 UTC (permalink / raw)
  To: Luis R. Rodriguez, Kees Cook, Andrew Morton, Jonathan Corbet
  Cc: linux-kernel, linux-fsdevel, linux-doc, Al Viro, Matthew Wilcox,
	Eric W. Biederman, Takashi Iwai, Davidlohr Bueso, Manfred Spraul,
	Waiman Long

The mixing in of a sequence number into the IPC IDs is probably to
avoid ID reuse in userspace as much as possible. With extended IPCMNI
mode, the number of usable sequence numbers is greatly reduced leading
to higher chance of ID reuse.

To address this issue, we need to conserve the sequence number space
as much as possible. Right now, the sequence number is incremented
for every new ID created. In reality, we only need to increment the
sequence number when one or more IDs have been removed previously to
make sure that those IDs will not be reused when a new one is built.
This is being done only in the new extended IPCMNI mode.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 include/linux/ipc_namespace.h |  1 +
 ipc/ipc_sysctl.c              |  2 ++
 ipc/util.c                    | 17 ++++++++++++++---
 ipc/util.h                    |  2 ++
 4 files changed, 19 insertions(+), 3 deletions(-)

diff --git a/include/linux/ipc_namespace.h b/include/linux/ipc_namespace.h
index 6ab8c1b..7d5f553 100644
--- a/include/linux/ipc_namespace.h
+++ b/include/linux/ipc_namespace.h
@@ -16,6 +16,7 @@
 struct ipc_ids {
 	int in_use;
 	unsigned short seq;
+	unsigned short deleted;
 	struct rw_semaphore rwsem;
 	struct idr ipcs_idr;
 	int max_idx;
diff --git a/ipc/ipc_sysctl.c b/ipc/ipc_sysctl.c
index 73b7782..d9ac6ca 100644
--- a/ipc/ipc_sysctl.c
+++ b/ipc/ipc_sysctl.c
@@ -122,6 +122,7 @@ static int proc_ipc_sem_dointvec(struct ctl_table *table, int write,
 static int int_max = INT_MAX;
 int ipc_mni = IPCMNI;
 int ipc_mni_shift = IPCMNI_SHIFT;
+bool ipc_mni_extended;
 
 static struct ctl_table ipc_kern_table[] = {
 	{
@@ -252,6 +253,7 @@ static int __init ipc_mni_extend(char *str)
 {
 	ipc_mni = IPCMNI_EXTEND;
 	ipc_mni_shift = IPCMNI_EXTEND_SHIFT;
+	ipc_mni_extended = true;
 	pr_info("IPCMNI extended to %d.\n", ipc_mni);
 	return 0;
 }
diff --git a/ipc/util.c b/ipc/util.c
index 07ae117..6ae0007 100644
--- a/ipc/util.c
+++ b/ipc/util.c
@@ -115,6 +115,7 @@ static int __init ipc_init(void)
 void ipc_init_ids(struct ipc_ids *ids)
 {
 	ids->in_use = 0;
+	ids->deleted = false;
 	ids->seq = 0;
 	init_rwsem(&ids->rwsem);
 	rhashtable_init(&ids->key_ht, &ipc_kht_params);
@@ -198,6 +199,11 @@ static inline int ipc_idr_alloc(struct ipc_ids *ids, struct kern_ipc_perm *new)
 {
 	int idx, next_id = -1;
 
+/*
+ * To conserve sequence number space with extended ipc_mni when new ID
+ * is built, the sequence number is incremented only when one or more
+ * IDs have been removed previously.
+ */
 #ifdef CONFIG_CHECKPOINT_RESTORE
 	next_id = ids->next_id;
 	ids->next_id = -1;
@@ -216,9 +222,13 @@ static inline int ipc_idr_alloc(struct ipc_ids *ids, struct kern_ipc_perm *new)
 	 */
 
 	if (next_id < 0) { /* !CHECKPOINT_RESTORE or next_id is unset */
-		new->seq = ids->seq++;
-		if (ids->seq > IPCID_SEQ_MAX)
-			ids->seq = 0;
+		if (!ipc_mni_extended || ids->deleted) {
+			ids->seq++;
+			if (ids->seq > IPCID_SEQ_MAX)
+				ids->seq = 0;
+			ids->deleted = false;
+		}
+		new->seq = ids->seq;
 		idx = idr_alloc(&ids->ipcs_idr, new, 0, 0, GFP_NOWAIT);
 	} else {
 		new->seq = ipcid_to_seqx(next_id);
@@ -436,6 +446,7 @@ void ipc_rmid(struct ipc_ids *ids, struct kern_ipc_perm *ipcp)
 	idr_remove(&ids->ipcs_idr, idx);
 	ipc_kht_remove(ids, ipcp);
 	ids->in_use--;
+	ids->deleted = true;
 	ipcp->deleted = true;
 
 	if (unlikely(idx == ids->max_idx)) {
diff --git a/ipc/util.h b/ipc/util.h
index 640f916..1f19729 100644
--- a/ipc/util.h
+++ b/ipc/util.h
@@ -33,6 +33,7 @@
 #ifdef CONFIG_SYSVIPC_SYSCTL
 extern int ipc_mni;
 extern int ipc_mni_shift;
+extern bool ipc_mni_extended;
 
 #define IPCMNI_SEQ_SHIFT	ipc_mni_shift
 #define IPCMNI_IDX_MASK		((1 << ipc_mni_shift) - 1)
@@ -40,6 +41,7 @@
 #else /* CONFIG_SYSVIPC_SYSCTL */
 
 #define ipc_mni		IPCMNI
+#define ipc_mni_extended	false
 #define IPCMNI_SEQ_SHIFT	IPCMNI_SHIFT
 #define IPCMNI_IDX_MASK		((1 << IPCMNI_SHIFT) - 1)
 #endif /* CONFIG_SYSVIPC_SYSCTL */
-- 
1.8.3.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH v10 3/4] ipc: Make the new sequence number generation mode available to all
  2018-11-05 15:43 [PATCH v10 0/4] ipc: Increase IPCMNI limit & IPC id generation modes Waiman Long
  2018-11-05 15:43 ` [PATCH v10 1/4] ipc: Allow boot time extension of IPCMNI from 32k to 8M Waiman Long
  2018-11-05 15:43 ` [PATCH v10 2/4] ipc: Conserve sequence numbers in extended IPCMNI mode Waiman Long
@ 2018-11-05 15:43 ` Waiman Long
  2018-11-06  4:53   ` kbuild test robot
  2018-11-05 15:43 ` [PATCH v10 4/4] ipc: Add a cyclic mode for id generation Waiman Long
  3 siblings, 1 reply; 10+ messages in thread
From: Waiman Long @ 2018-11-05 15:43 UTC (permalink / raw)
  To: Luis R. Rodriguez, Kees Cook, Andrew Morton, Jonathan Corbet
  Cc: linux-kernel, linux-fsdevel, linux-doc, Al Viro, Matthew Wilcox,
	Eric W. Biederman, Takashi Iwai, Davidlohr Bueso, Manfred Spraul,
	Waiman Long

It happens that the new IPC id sequence number inrement mode can be
useful to reduce the chance of IPC id reuse even if the ipcmni_extend
boot command line parameter isn't specified. So a new ipcid_mode sysctl
parameter is added to control the sequence number generation mode -
legacy and delete modes.

In the legacy mode, the sequence number is incremented every time a new
ID is generated. In the delete mode, the number is incremented only if
one or more IDs have been previously deleted. The default is legacy for
non-ipcmni_extend and delete for ipcmni_extend. This new ipcid_mode
parameter is specific to each of the IPC namespaces.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 Documentation/sysctl/kernel.txt | 17 +++++++++++++++++
 include/linux/ipc_namespace.h   | 12 ++++++++++++
 ipc/ipc_sysctl.c                | 10 ++++++++++
 ipc/msg.c                       |  3 ++-
 ipc/namespace.c                 |  2 ++
 ipc/sem.c                       |  3 ++-
 ipc/shm.c                       |  3 ++-
 ipc/util.c                      | 10 ++++++----
 ipc/util.h                      |  3 ++-
 9 files changed, 55 insertions(+), 8 deletions(-)

diff --git a/Documentation/sysctl/kernel.txt b/Documentation/sysctl/kernel.txt
index 37a6795..91bada1 100644
--- a/Documentation/sysctl/kernel.txt
+++ b/Documentation/sysctl/kernel.txt
@@ -41,6 +41,7 @@ show up in /proc/sys/kernel:
 - hung_task_check_interval_secs
 - hung_task_warnings
 - hyperv_record_panic_msg
+- ipcid_mode
 - kexec_load_disabled
 - kptr_restrict
 - l2cr                        [ PPC only ]
@@ -398,6 +399,22 @@ Controls whether the panic kmsg data should be reported to Hyper-V.
 
 ==============================================================
 
+ipcid_mode:
+
+Controls how the IPC ids returned by msgget(), semget() and shmget()
+are being generated.
+
+0: legacy mode
+1: delete mode
+
+There are two components in an IPC id - an integer identifier and a
+sequence number. In the legacy mode, the sequence number is incremented
+every time a new id is generated. In the delete mode, the sequence number
+is only incremented if one or more ids have been previously deleted. The
+delete mode reduces the chance that a given id will be reused again.
+
+==============================================================
+
 kexec_load_disabled:
 
 A toggle indicating if the kexec_load syscall has been disabled. This
diff --git a/include/linux/ipc_namespace.h b/include/linux/ipc_namespace.h
index 7d5f553..79d9d50 100644
--- a/include/linux/ipc_namespace.h
+++ b/include/linux/ipc_namespace.h
@@ -26,6 +26,15 @@ struct ipc_ids {
 	struct rhashtable key_ht;
 };
 
+/*
+ * IPC id generation mode for controlling how the IPC id returned by
+ * {msg,sem,shm}get() is being generated.
+ */
+enum ipc_id_mode {
+	ipc_id_legacy,	/* Sequence # incremented on every allocation */
+	ipc_id_delete,	/* Sequence # incremented only if an ID was deleted */
+};
+
 struct ipc_namespace {
 	refcount_t	count;
 	struct ipc_ids	ids[3];
@@ -39,6 +48,9 @@ struct ipc_namespace {
 	atomic_t	msg_bytes;
 	atomic_t	msg_hdrs;
 
+	/* IPC id generation mode */
+	unsigned int	ipcid_mode;
+
 	size_t		shm_ctlmax;
 	size_t		shm_ctlall;
 	unsigned long	shm_tot;
diff --git a/ipc/ipc_sysctl.c b/ipc/ipc_sysctl.c
index d9ac6ca..4c30e62 100644
--- a/ipc/ipc_sysctl.c
+++ b/ipc/ipc_sysctl.c
@@ -200,6 +200,15 @@ static int proc_ipc_sem_dointvec(struct ctl_table *table, int write,
 		.mode		= 0644,
 		.proc_handler	= proc_ipc_sem_dointvec,
 	},
+	{
+		.procname	= "ipcid_mode",
+		.data		= &init_ipc_ns.ipcid_mode,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= proc_ipc_dointvec_minmax,
+		.extra1		= &zero,
+		.extra2		= &one,
+	},
 #ifdef CONFIG_CHECKPOINT_RESTORE
 	{
 		.procname	= "sem_next_id",
@@ -254,6 +263,7 @@ static int __init ipc_mni_extend(char *str)
 	ipc_mni = IPCMNI_EXTEND;
 	ipc_mni_shift = IPCMNI_EXTEND_SHIFT;
 	ipc_mni_extended = true;
+	init_ipc_ns.ipcid_mode = ipc_id_delete;
 	pr_info("IPCMNI extended to %d.\n", ipc_mni);
 	return 0;
 }
diff --git a/ipc/msg.c b/ipc/msg.c
index 0833c64..b401ba2 100644
--- a/ipc/msg.c
+++ b/ipc/msg.c
@@ -161,7 +161,8 @@ static int newque(struct ipc_namespace *ns, struct ipc_params *params)
 	INIT_LIST_HEAD(&msq->q_senders);
 
 	/* ipc_addid() locks msq upon success. */
-	retval = ipc_addid(&msg_ids(ns), &msq->q_perm, ns->msg_ctlmni);
+	retval = ipc_addid(&msg_ids(ns), &msq->q_perm, ns->msg_ctlmni,
+			   ns->ipcid_mode);
 	if (retval < 0) {
 		ipc_rcu_putref(&msq->q_perm, msg_rcu_free);
 		return retval;
diff --git a/ipc/namespace.c b/ipc/namespace.c
index 2160779..8b62cbd0 100644
--- a/ipc/namespace.c
+++ b/ipc/namespace.c
@@ -55,6 +55,8 @@ static struct ipc_namespace *create_ipc_ns(struct user_namespace *user_ns,
 	ns->user_ns = get_user_ns(user_ns);
 	ns->ucounts = ucounts;
 
+	ns->ipcid_mode = ipc_mni_extended ? ipc_id_delete : ipc_id_legacy;
+
 	err = mq_init_ns(ns);
 	if (err)
 		goto fail_put;
diff --git a/ipc/sem.c b/ipc/sem.c
index 745dc61..51af634 100644
--- a/ipc/sem.c
+++ b/ipc/sem.c
@@ -553,7 +553,8 @@ static int newary(struct ipc_namespace *ns, struct ipc_params *params)
 	sma->sem_ctime = ktime_get_real_seconds();
 
 	/* ipc_addid() locks sma upon success. */
-	retval = ipc_addid(&sem_ids(ns), &sma->sem_perm, ns->sc_semmni);
+	retval = ipc_addid(&sem_ids(ns), &sma->sem_perm, ns->sc_semmni,
+			   ns->ipcid_mode);
 	if (retval < 0) {
 		ipc_rcu_putref(&sma->sem_perm, sem_rcu_free);
 		return retval;
diff --git a/ipc/shm.c b/ipc/shm.c
index 0842411..182ae4c 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -676,7 +676,8 @@ static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
 	shp->shm_creator = current;
 
 	/* ipc_addid() locks shp upon success. */
-	error = ipc_addid(&shm_ids(ns), &shp->shm_perm, ns->shm_ctlmni);
+	error = ipc_addid(&shm_ids(ns), &shp->shm_perm, ns->shm_ctlmni,
+			  ns->ipcid_mode);
 	if (error < 0)
 		goto no_id;
 
diff --git a/ipc/util.c b/ipc/util.c
index 6ae0007..04c8e31 100644
--- a/ipc/util.c
+++ b/ipc/util.c
@@ -195,7 +195,8 @@ static struct kern_ipc_perm *ipc_findkey(struct ipc_ids *ids, key_t key)
  * The caller must own kern_ipc_perm.lock.of the new object.
  * On error, the function returns a (negative) error code.
  */
-static inline int ipc_idr_alloc(struct ipc_ids *ids, struct kern_ipc_perm *new)
+static inline int ipc_idr_alloc(struct ipc_ids *ids, struct kern_ipc_perm *new,
+				int idmode)
 {
 	int idx, next_id = -1;
 
@@ -222,7 +223,7 @@ static inline int ipc_idr_alloc(struct ipc_ids *ids, struct kern_ipc_perm *new)
 	 */
 
 	if (next_id < 0) { /* !CHECKPOINT_RESTORE or next_id is unset */
-		if (!ipc_mni_extended || ids->deleted) {
+		if (idmode == ipc_id_legacy || ids->deleted) {
 			ids->seq++;
 			if (ids->seq > IPCID_SEQ_MAX)
 				ids->seq = 0;
@@ -255,7 +256,8 @@ static inline int ipc_idr_alloc(struct ipc_ids *ids, struct kern_ipc_perm *new)
  *
  * Called with writer ipc_ids.rwsem held.
  */
-int ipc_addid(struct ipc_ids *ids, struct kern_ipc_perm *new, int limit)
+int ipc_addid(struct ipc_ids *ids, struct kern_ipc_perm *new, int limit,
+	      int idmode)
 {
 	kuid_t euid;
 	kgid_t egid;
@@ -282,7 +284,7 @@ int ipc_addid(struct ipc_ids *ids, struct kern_ipc_perm *new, int limit)
 
 	new->deleted = false;
 
-	idx = ipc_idr_alloc(ids, new);
+	idx = ipc_idr_alloc(ids, new, idmode);
 	idr_preload_end();
 
 	if (idx >= 0 && new->key != IPC_PRIVATE) {
diff --git a/ipc/util.h b/ipc/util.h
index 1f19729..c8f2d0ed 100644
--- a/ipc/util.h
+++ b/ipc/util.h
@@ -129,7 +129,8 @@ void __init ipc_init_proc_interface(const char *path, const char *header,
 #define IPCID_SEQ_MAX	  (INT_MAX >> IPCMNI_SEQ_SHIFT)
 
 /* must be called with ids->rwsem acquired for writing */
-int ipc_addid(struct ipc_ids *, struct kern_ipc_perm *, int);
+int ipc_addid(struct ipc_ids *ids, struct kern_ipc_perm *new, int limit,
+	      int idmode);
 
 /* must be called with both locks acquired. */
 void ipc_rmid(struct ipc_ids *, struct kern_ipc_perm *);
-- 
1.8.3.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* [PATCH v10 4/4] ipc: Add a cyclic mode for id generation
  2018-11-05 15:43 [PATCH v10 0/4] ipc: Increase IPCMNI limit & IPC id generation modes Waiman Long
                   ` (2 preceding siblings ...)
  2018-11-05 15:43 ` [PATCH v10 3/4] ipc: Make the new sequence number generation mode available to all Waiman Long
@ 2018-11-05 15:43 ` Waiman Long
  2018-11-06 13:02   ` Matthew Wilcox
  3 siblings, 1 reply; 10+ messages in thread
From: Waiman Long @ 2018-11-05 15:43 UTC (permalink / raw)
  To: Luis R. Rodriguez, Kees Cook, Andrew Morton, Jonathan Corbet
  Cc: linux-kernel, linux-fsdevel, linux-doc, Al Viro, Matthew Wilcox,
	Eric W. Biederman, Takashi Iwai, Davidlohr Bueso, Manfred Spraul,
	Waiman Long

The idea of using the cyclic mode to reduce id reuse came from Manfred
Spraul <manfred@colorfullife.com>. There may be a little bit of
additional memory/performance overhead in doing cyclic id allocation,
but it is a slow path anyway and a bit of overhead shouldn't be an issue.

This patch differs from his as the cyclic mode is not the default and
has to be explicitly opted in for users who want that.

Note that it is possible to use an identifier larger than the given
IPC mni number in cyclic mode.

Signed-off-by: Waiman Long <longman@redhat.com>
---
 Documentation/sysctl/kernel.txt | 10 ++++++++--
 include/linux/ipc_namespace.h   |  1 +
 ipc/ipc_sysctl.c                |  3 ++-
 ipc/util.c                      |  6 +++++-
 4 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/Documentation/sysctl/kernel.txt b/Documentation/sysctl/kernel.txt
index 91bada1..6de0679 100644
--- a/Documentation/sysctl/kernel.txt
+++ b/Documentation/sysctl/kernel.txt
@@ -406,12 +406,18 @@ are being generated.
 
 0: legacy mode
 1: delete mode
+2: cyclic mode
 
 There are two components in an IPC id - an integer identifier and a
 sequence number. In the legacy mode, the sequence number is incremented
 every time a new id is generated. In the delete mode, the sequence number
-is only incremented if one or more ids have been previously deleted. The
-delete mode reduces the chance that a given id will be reused again.
+is only incremented if one or more ids have been previously deleted.
+In the cyclic mode, the sequence number increments in the same way as the
+delete mode, but the identifier is allocated cyclically through the whole
+IPC identifier number space instead of using the lowest available number.
+
+The cyclic mode has the lowest chance of IPC id reuse followed by the
+delete mode and the legacy mode.
 
 ==============================================================
 
diff --git a/include/linux/ipc_namespace.h b/include/linux/ipc_namespace.h
index 79d9d50..481dc02 100644
--- a/include/linux/ipc_namespace.h
+++ b/include/linux/ipc_namespace.h
@@ -33,6 +33,7 @@ struct ipc_ids {
 enum ipc_id_mode {
 	ipc_id_legacy,	/* Sequence # incremented on every allocation */
 	ipc_id_delete,	/* Sequence # incremented only if an ID was deleted */
+	ipc_id_cyclic,	/* Identifier is allocated cyclically */
 };
 
 struct ipc_namespace {
diff --git a/ipc/ipc_sysctl.c b/ipc/ipc_sysctl.c
index 4c30e62..8d114d0 100644
--- a/ipc/ipc_sysctl.c
+++ b/ipc/ipc_sysctl.c
@@ -119,6 +119,7 @@ static int proc_ipc_sem_dointvec(struct ctl_table *table, int write,
 
 static int zero;
 static int one = 1;
+static int two = 2;
 static int int_max = INT_MAX;
 int ipc_mni = IPCMNI;
 int ipc_mni_shift = IPCMNI_SHIFT;
@@ -207,7 +208,7 @@ static int proc_ipc_sem_dointvec(struct ctl_table *table, int write,
 		.mode		= 0644,
 		.proc_handler	= proc_ipc_dointvec_minmax,
 		.extra1		= &zero,
-		.extra2		= &one,
+		.extra2		= &two,
 	},
 #ifdef CONFIG_CHECKPOINT_RESTORE
 	{
diff --git a/ipc/util.c b/ipc/util.c
index 04c8e31..8d73d17 100644
--- a/ipc/util.c
+++ b/ipc/util.c
@@ -230,7 +230,11 @@ static inline int ipc_idr_alloc(struct ipc_ids *ids, struct kern_ipc_perm *new,
 			ids->deleted = false;
 		}
 		new->seq = ids->seq;
-		idx = idr_alloc(&ids->ipcs_idr, new, 0, 0, GFP_NOWAIT);
+		if (idmode == ipc_id_cyclic)
+			idx = idr_alloc_cyclic(&ids->ipcs_idr, new, 0, IPCMNI,
+						GFP_NOWAIT);
+		else
+			idx = idr_alloc(&ids->ipcs_idr, new, 0, 0, GFP_NOWAIT);
 	} else {
 		new->seq = ipcid_to_seqx(next_id);
 		idx = idr_alloc(&ids->ipcs_idr, new, ipcid_to_idx(next_id),
-- 
1.8.3.1


^ permalink raw reply related	[flat|nested] 10+ messages in thread

* Re: [PATCH v10 3/4] ipc: Make the new sequence number generation mode available to all
  2018-11-05 15:43 ` [PATCH v10 3/4] ipc: Make the new sequence number generation mode available to all Waiman Long
@ 2018-11-06  4:53   ` kbuild test robot
  0 siblings, 0 replies; 10+ messages in thread
From: kbuild test robot @ 2018-11-06  4:53 UTC (permalink / raw)
  To: Waiman Long
  Cc: kbuild-all, Luis R. Rodriguez, Kees Cook, Andrew Morton,
	Jonathan Corbet, linux-kernel, linux-fsdevel, linux-doc, Al Viro,
	Matthew Wilcox, Eric W. Biederman, Takashi Iwai, Davidlohr Bueso,
	Manfred Spraul, Waiman Long

[-- Attachment #1: Type: text/plain, Size: 17400 bytes --]

Hi Waiman,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on linus/master]
[also build test WARNING on v4.20-rc1]
[cannot apply to next-20181105]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Waiman-Long/ipc-Increase-IPCMNI-limit-IPC-id-generation-modes/20181106-044522
reproduce: make htmldocs

All warnings (new ones prefixed by >>):

   WARNING: convert(1) not found, for SVG to PDF conversion install ImageMagick (https://www.imagemagick.org)
>> ipc/util.c:262: warning: Function parameter or member 'idmode' not described in 'ipc_addid'
   kernel/resource.c:337: warning: Function parameter or member 'start' not described in 'find_next_iomem_res'
   kernel/resource.c:337: warning: Function parameter or member 'end' not described in 'find_next_iomem_res'
   kernel/resource.c:337: warning: Function parameter or member 'flags' not described in 'find_next_iomem_res'
   kernel/resource.c:337: warning: Function parameter or member 'desc' not described in 'find_next_iomem_res'
   kernel/resource.c:337: warning: Function parameter or member 'first_lvl' not described in 'find_next_iomem_res'
   kernel/resource.c:337: warning: Function parameter or member 'res' not described in 'find_next_iomem_res'
   kernel/resource.c:409: warning: Function parameter or member 'arg' not described in 'walk_iomem_res_desc'
   kernel/resource.c:409: warning: Function parameter or member 'func' not described in 'walk_iomem_res_desc'
   kernel/resource.c:409: warning: Function parameter or member 'arg' not described in 'walk_iomem_res_desc'
   kernel/resource.c:409: warning: Function parameter or member 'func' not described in 'walk_iomem_res_desc'
   include/linux/rcutree.h:1: warning: no structured comments found
   kernel/rcu/tree.c:684: warning: Excess function parameter 'irq' description in 'rcu_nmi_exit'
   include/linux/srcu.h:175: warning: Function parameter or member 'p' not described in 'srcu_dereference_notrace'
   include/linux/srcu.h:175: warning: Function parameter or member 'sp' not described in 'srcu_dereference_notrace'
   include/linux/gfp.h:1: warning: no structured comments found
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:4439: warning: Function parameter or member 'wext.ibss' not described in 'wireless_dev'
   include/net/cfg80211.h:4439: warning: Function parameter or member 'wext.connect' not described in 'wireless_dev'
   include/net/cfg80211.h:4439: warning: Function parameter or member 'wext.keys' not described in 'wireless_dev'
   include/net/cfg80211.h:4439: warning: Function parameter or member 'wext.ie' not described in 'wireless_dev'
   include/net/cfg80211.h:4439: warning: Function parameter or member 'wext.ie_len' not described in 'wireless_dev'
   include/net/cfg80211.h:4439: warning: Function parameter or member 'wext.bssid' not described in 'wireless_dev'
   include/net/cfg80211.h:4439: warning: Function parameter or member 'wext.ssid' not described in 'wireless_dev'
   include/net/cfg80211.h:4439: warning: Function parameter or member 'wext.default_key' not described in 'wireless_dev'
   include/net/cfg80211.h:4439: warning: Function parameter or member 'wext.default_mgmt_key' not described in 'wireless_dev'
   include/net/cfg80211.h:4439: warning: Function parameter or member 'wext.prev_bssid_valid' not described in 'wireless_dev'
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '
   include/net/cfg80211.h:2838: warning: cannot understand function prototype: 'struct cfg80211_ftm_responder_stats '

vim +262 ipc/util.c

b8fd998384 Davidlohr Bueso   2017-11-17  243  
^1da177e4c Linus Torvalds    2005-04-16  244  /**
8001c85810 Davidlohr Bueso   2014-01-27  245   * ipc_addid - add an ipc identifier
8001c85810 Davidlohr Bueso   2014-01-27  246   * @ids: ipc identifier set
8001c85810 Davidlohr Bueso   2014-01-27  247   * @new: new ipc permission set
ebf66799ac Davidlohr Bueso   2017-11-17  248   * @limit: limit for the number of used ids
^1da177e4c Linus Torvalds    2005-04-16  249   *
8001c85810 Davidlohr Bueso   2014-01-27  250   * Add an entry 'new' to the ipc ids idr. The permissions object is
27c331a174 Manfred Spraul    2018-08-21  251   * initialised and the first free entry is set up and the index assigned
f4566f0485 Nadia Derbey      2007-10-18  252   * is returned. The 'new' entry is returned in a locked state on success.
39cfffd774 Manfred Spraul    2018-08-21  253   *
283bb7fada Pierre Peiffer    2007-10-18  254   * On failure the entry is not locked and a negative err-code is returned.
39cfffd774 Manfred Spraul    2018-08-21  255   * The caller must use ipc_rcu_putref() to free the identifier.
^1da177e4c Linus Torvalds    2005-04-16  256   *
d9a605e40b Davidlohr Bueso   2013-09-11  257   * Called with writer ipc_ids.rwsem held.
^1da177e4c Linus Torvalds    2005-04-16  258   */
4073296664 Waiman Long       2018-11-05  259  int ipc_addid(struct ipc_ids *ids, struct kern_ipc_perm *new, int limit,
4073296664 Waiman Long       2018-11-05  260  	      int idmode)
^1da177e4c Linus Torvalds    2005-04-16  261  {
1efdb69b0b Eric W. Biederman 2012-02-07 @262  	kuid_t euid;
1efdb69b0b Eric W. Biederman 2012-02-07  263  	kgid_t egid;
e2652ae6bd Manfred Spraul    2018-08-21  264  	int idx, err;
^1da177e4c Linus Torvalds    2005-04-16  265  
39cfffd774 Manfred Spraul    2018-08-21  266  	/* 1) Initialize the refcount so that ipc_rcu_putref works */
39cfffd774 Manfred Spraul    2018-08-21  267  	refcount_set(&new->refcount, 1);
39cfffd774 Manfred Spraul    2018-08-21  268  
fa04270b8b Waiman Long       2018-11-05  269  	if (limit > ipc_mni)
fa04270b8b Waiman Long       2018-11-05  270  		limit = ipc_mni;
7ca7e564e0 Nadia Derbey      2007-10-18  271  
dc2c8c84de Davidlohr Bueso   2018-08-21  272  	if (ids->in_use >= limit)
283bb7fada Pierre Peiffer    2007-10-18  273  		return -ENOSPC;
7ca7e564e0 Nadia Derbey      2007-10-18  274  
54924ea33f Tejun Heo         2013-02-27  275  	idr_preload(GFP_KERNEL);
54924ea33f Tejun Heo         2013-02-27  276  
e00b4ff7eb Nadia Derbey      2008-11-19  277  	spin_lock_init(&new->lock);
e00b4ff7eb Nadia Derbey      2008-11-19  278  	rcu_read_lock();
e00b4ff7eb Nadia Derbey      2008-11-19  279  	spin_lock(&new->lock);
e00b4ff7eb Nadia Derbey      2008-11-19  280  
b9a5322779 Linus Torvalds    2015-09-30  281  	current_euid_egid(&euid, &egid);
b9a5322779 Linus Torvalds    2015-09-30  282  	new->cuid = new->uid = euid;
b9a5322779 Linus Torvalds    2015-09-30  283  	new->gid = new->cgid = egid;
b9a5322779 Linus Torvalds    2015-09-30  284  
39cfffd774 Manfred Spraul    2018-08-21  285  	new->deleted = false;
39cfffd774 Manfred Spraul    2018-08-21  286  
4073296664 Waiman Long       2018-11-05  287  	idx = ipc_idr_alloc(ids, new, idmode);
54924ea33f Tejun Heo         2013-02-27  288  	idr_preload_end();
0cfb6aee70 Guillaume Knispel 2017-09-08  289  
e2652ae6bd Manfred Spraul    2018-08-21  290  	if (idx >= 0 && new->key != IPC_PRIVATE) {
0cfb6aee70 Guillaume Knispel 2017-09-08  291  		err = rhashtable_insert_fast(&ids->key_ht, &new->khtnode,
0cfb6aee70 Guillaume Knispel 2017-09-08  292  					     ipc_kht_params);
0cfb6aee70 Guillaume Knispel 2017-09-08  293  		if (err < 0) {
e2652ae6bd Manfred Spraul    2018-08-21  294  			idr_remove(&ids->ipcs_idr, idx);
e2652ae6bd Manfred Spraul    2018-08-21  295  			idx = err;
0cfb6aee70 Guillaume Knispel 2017-09-08  296  		}
0cfb6aee70 Guillaume Knispel 2017-09-08  297  	}
e2652ae6bd Manfred Spraul    2018-08-21  298  	if (idx < 0) {
39cfffd774 Manfred Spraul    2018-08-21  299  		new->deleted = true;
e00b4ff7eb Nadia Derbey      2008-11-19  300  		spin_unlock(&new->lock);
e00b4ff7eb Nadia Derbey      2008-11-19  301  		rcu_read_unlock();
e2652ae6bd Manfred Spraul    2018-08-21  302  		return idx;
e00b4ff7eb Nadia Derbey      2008-11-19  303  	}
7ca7e564e0 Nadia Derbey      2007-10-18  304  
^1da177e4c Linus Torvalds    2005-04-16  305  	ids->in_use++;
27c331a174 Manfred Spraul    2018-08-21  306  	if (idx > ids->max_idx)
27c331a174 Manfred Spraul    2018-08-21  307  		ids->max_idx = idx;
e2652ae6bd Manfred Spraul    2018-08-21  308  	return idx;
^1da177e4c Linus Torvalds    2005-04-16  309  }
^1da177e4c Linus Torvalds    2005-04-16  310  

:::::: The code at line 262 was first introduced by commit
:::::: 1efdb69b0bb41dec8ee3e2cac0a0f167837d0919 userns: Convert ipc to use kuid and kgid where appropriate

:::::: TO: Eric W. Biederman <ebiederm@xmission.com>
:::::: CC: Eric W. Biederman <ebiederm@xmission.com>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 6597 bytes --]

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v10 4/4] ipc: Add a cyclic mode for id generation
  2018-11-05 15:43 ` [PATCH v10 4/4] ipc: Add a cyclic mode for id generation Waiman Long
@ 2018-11-06 13:02   ` Matthew Wilcox
  2018-11-08 21:38     ` Waiman Long
  0 siblings, 1 reply; 10+ messages in thread
From: Matthew Wilcox @ 2018-11-06 13:02 UTC (permalink / raw)
  To: Waiman Long
  Cc: Luis R. Rodriguez, Kees Cook, Andrew Morton, Jonathan Corbet,
	linux-kernel, linux-fsdevel, linux-doc, Al Viro,
	Eric W. Biederman, Takashi Iwai, Davidlohr Bueso, Manfred Spraul

On Mon, Nov 05, 2018 at 10:43:46AM -0500, Waiman Long wrote:
> The idea of using the cyclic mode to reduce id reuse came from Manfred
> Spraul <manfred@colorfullife.com>. There may be a little bit of
> additional memory/performance overhead in doing cyclic id allocation,
> but it is a slow path anyway and a bit of overhead shouldn't be an issue.
> 
> This patch differs from his as the cyclic mode is not the default and
> has to be explicitly opted in for users who want that.

I really don't like the "make the sysadmin choose" approach.  It's
shifting blame to someone who has no idea what the tradeoffs are.
We should try to get it right rather than force more people to learn
why we're incapable of making the right decision for them.


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v10 1/4] ipc: Allow boot time extension of IPCMNI from 32k to 8M
  2018-11-05 15:43 ` [PATCH v10 1/4] ipc: Allow boot time extension of IPCMNI from 32k to 8M Waiman Long
@ 2018-11-06 13:20   ` Matthew Wilcox
  2018-11-08 21:29     ` Waiman Long
  0 siblings, 1 reply; 10+ messages in thread
From: Matthew Wilcox @ 2018-11-06 13:20 UTC (permalink / raw)
  To: Waiman Long
  Cc: Luis R. Rodriguez, Kees Cook, Andrew Morton, Jonathan Corbet,
	linux-kernel, linux-fsdevel, linux-doc, Al Viro,
	Eric W. Biederman, Takashi Iwai, Davidlohr Bueso, Manfred Spraul

On Mon, Nov 05, 2018 at 10:43:43AM -0500, Waiman Long wrote:
> The maximum number of unique System V IPC identifiers was limited to
> 32k.  That limit should be big enough for most use cases.
> 
> However, there are some users out there requesting for more, especially
> those that are migrating from Solaris which uses 24 bits for unique
> identifiers. To satisfy the need of those users, a new boot time kernel
> option "ipcmni_extend" is added to extend the IPCMNI value to 8M. This
> is a 256X increase which hopefully is big enough for them.

Why go to 23 bits when people are coming from systems with 24 bits?
Let's just go to 24 bits.  This happens to fit well with the underlying
data structure which uses 6 bits per layer of the tree.

> The use of this new option will change the pattern of the IPC identifiers
> returned by functions like shmget(2). An application that depends on
> such pattern may not work properly.  So it should only be used if the
> users really need more than 32k of unique IPC numbers.

Are there applications out there that rely on the internal structure of
the IPC identifiers?!

How about scrapping all this and just doing the following:

Allocate 24 bits of the ID cyclically.  Increment the top 7 bits of the
ID every time the cursor wraps.  That's not going to give us a perfect
progression from 0-2 billion, because it'll skip the ones in use.
But it'll ensure the ID isn't reused particularly quickly unless the
application is really using millions of IDs.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v10 1/4] ipc: Allow boot time extension of IPCMNI from 32k to 8M
  2018-11-06 13:20   ` Matthew Wilcox
@ 2018-11-08 21:29     ` Waiman Long
  0 siblings, 0 replies; 10+ messages in thread
From: Waiman Long @ 2018-11-08 21:29 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: Luis R. Rodriguez, Kees Cook, Andrew Morton, Jonathan Corbet,
	linux-kernel, linux-fsdevel, linux-doc, Al Viro,
	Eric W. Biederman, Takashi Iwai, Davidlohr Bueso, Manfred Spraul

On 11/06/2018 08:20 AM, Matthew Wilcox wrote:
> On Mon, Nov 05, 2018 at 10:43:43AM -0500, Waiman Long wrote:
>> The maximum number of unique System V IPC identifiers was limited to
>> 32k.  That limit should be big enough for most use cases.
>>
>> However, there are some users out there requesting for more, especially
>> those that are migrating from Solaris which uses 24 bits for unique
>> identifiers. To satisfy the need of those users, a new boot time kernel
>> option "ipcmni_extend" is added to extend the IPCMNI value to 8M. This
>> is a 256X increase which hopefully is big enough for them.
> Why go to 23 bits when people are coming from systems with 24 bits?
> Let's just go to 24 bits.  This happens to fit well with the underlying
> data structure which uses 6 bits per layer of the tree.

Sure. I can move it up to 24 bits leave 7 bits for the sequence number.

>
>> The use of this new option will change the pattern of the IPC identifiers
>> returned by functions like shmget(2). An application that depends on
>> such pattern may not work properly.  So it should only be used if the
>> users really need more than 32k of unique IPC numbers.
> Are there applications out there that rely on the internal structure of
> the IPC identifiers?!

That is a question that may not have a clear answer. Most applications
won't do that, but there are always some outliners that do crazy thing.
So you never know for sure.

>
> How about scrapping all this and just doing the following:
>
> Allocate 24 bits of the ID cyclically.  Increment the top 7 bits of the
> ID every time the cursor wraps.  That's not going to give us a perfect
> progression from 0-2 billion, because it'll skip the ones in use.
> But it'll ensure the ID isn't reused particularly quickly unless the
> application is really using millions of IDs.

Eric Biederman had sent out a patch somewhat like that before. Again,
there is a slight chance that it may break existing applications. So the
question is whether we are willing to take the risk. I won't mind if
upstream decide to go this route.

Cheers,
Longman



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH v10 4/4] ipc: Add a cyclic mode for id generation
  2018-11-06 13:02   ` Matthew Wilcox
@ 2018-11-08 21:38     ` Waiman Long
  0 siblings, 0 replies; 10+ messages in thread
From: Waiman Long @ 2018-11-08 21:38 UTC (permalink / raw)
  To: Matthew Wilcox
  Cc: Luis R. Rodriguez, Kees Cook, Andrew Morton, Jonathan Corbet,
	linux-kernel, linux-fsdevel, linux-doc, Al Viro,
	Eric W. Biederman, Takashi Iwai, Davidlohr Bueso, Manfred Spraul

On 11/06/2018 08:02 AM, Matthew Wilcox wrote:
> On Mon, Nov 05, 2018 at 10:43:46AM -0500, Waiman Long wrote:
>> The idea of using the cyclic mode to reduce id reuse came from Manfred
>> Spraul <manfred@colorfullife.com>. There may be a little bit of
>> additional memory/performance overhead in doing cyclic id allocation,
>> but it is a slow path anyway and a bit of overhead shouldn't be an issue.
>>
>> This patch differs from his as the cyclic mode is not the default and
>> has to be explicitly opted in for users who want that.
> I really don't like the "make the sysadmin choose" approach.  It's
> shifting blame to someone who has no idea what the tradeoffs are.
> We should try to get it right rather than force more people to learn
> why we're incapable of making the right decision for them.
>
My patch is conservative as there will be no behavior change by default.
I won't mind changing the default to cyclic if we all agree.

Cheers,
Longman



^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2018-11-08 21:38 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-05 15:43 [PATCH v10 0/4] ipc: Increase IPCMNI limit & IPC id generation modes Waiman Long
2018-11-05 15:43 ` [PATCH v10 1/4] ipc: Allow boot time extension of IPCMNI from 32k to 8M Waiman Long
2018-11-06 13:20   ` Matthew Wilcox
2018-11-08 21:29     ` Waiman Long
2018-11-05 15:43 ` [PATCH v10 2/4] ipc: Conserve sequence numbers in extended IPCMNI mode Waiman Long
2018-11-05 15:43 ` [PATCH v10 3/4] ipc: Make the new sequence number generation mode available to all Waiman Long
2018-11-06  4:53   ` kbuild test robot
2018-11-05 15:43 ` [PATCH v10 4/4] ipc: Add a cyclic mode for id generation Waiman Long
2018-11-06 13:02   ` Matthew Wilcox
2018-11-08 21:38     ` Waiman Long

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).