[PATCH v2] ipc,shm: disable shmmax and shmall by default

[PATCH v2] ipc,shm: disable shmmax and shmall by default

[Davidlohr Bueso]

From: Davidlohr Bueso <davidlohr@hp.com>

The default size for shmmax is, and always has been, 32Mb.
Today, in the XXI century, it seems that this value is rather small,
making users have to increase it via sysctl, which can cause
unnecessary work and userspace application workarounds[1].

Instead of choosing yet another arbitrary value, larger than 32Mb,
this patch disables the use of both shmmax and shmall by default,
allowing users to create segments of unlimited sizes. Users and
applications that already explicitly set these values through sysctl
are left untouched, and thus does not change any of the behavior.

So a value of 0 bytes or pages, for shmmax and shmall, respectively,
implies unlimited memory, as opposed to disabling sysv shared memory.
This is safe as 0 cannot possibly be used previously as SHMMIN is
hardcoded to 1 and cannot be modified.

This change allows Linux to treat shm just as regular anonymous memory.
One important difference between them, though, is handling out-of-memory
conditions: as opposed to regular anon memory, the OOM killer will not
free the memory as it is shm, allowing users to potentially abuse this.
To overcome this situation, the shm_rmid_forced option must be enabled.

[1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html

Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
---
Changes from v1:
 - Respect SHMMIN even when shmmax is 0 (unlimited).
   This fixes the shmget02 test that broke in v1. (per Manfred)

 - Update changelog regarding OOM description (per Kosaki)

 include/linux/shm.h      | 2 +-
 include/uapi/linux/shm.h | 8 ++++----
 ipc/shm.c                | 6 ++++--
 3 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/include/linux/shm.h b/include/linux/shm.h
index 1e2cd2e..0ca06a3 100644
--- a/include/linux/shm.h
+++ b/include/linux/shm.h
@@ -4,7 +4,7 @@
 #include <asm/page.h>
 #include <uapi/linux/shm.h>
 
-#define SHMALL (SHMMAX/PAGE_SIZE*(SHMMNI/16)) /* max shm system wide (pages) */
+#define SHMALL 0 /* max shm system wide (pages) */
 #include <asm/shmparam.h>
 struct shmid_kernel /* private to the kernel */
 {	
diff --git a/include/uapi/linux/shm.h b/include/uapi/linux/shm.h
index 78b6941..5f0ef28 100644
--- a/include/uapi/linux/shm.h
+++ b/include/uapi/linux/shm.h
@@ -9,14 +9,14 @@
 
 /*
  * SHMMAX, SHMMNI and SHMALL are upper limits are defaults which can
- * be increased by sysctl
+ * be increased by sysctl. By default, disable SHMMAX and SHMALL with
+ * 0 bytes, thus allowing processes to have unlimited shared memory.
  */
-
-#define SHMMAX 0x2000000		 /* max shared seg size (bytes) */
+#define SHMMAX 0		         /* max shared seg size (bytes) */
 #define SHMMIN 1			 /* min shared seg size (bytes) */
 #define SHMMNI 4096			 /* max num of segs system wide */
 #ifndef __KERNEL__
-#define SHMALL (SHMMAX/getpagesize()*(SHMMNI/16))
+#define SHMALL 0
 #endif
 #define SHMSEG SHMMNI			 /* max shared segs per process */
 
diff --git a/ipc/shm.c b/ipc/shm.c
index 7645961..8630561 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -490,10 +490,12 @@ static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
 	int id;
 	vm_flags_t acctflag = 0;
 
-	if (size < SHMMIN || size > ns->shm_ctlmax)
+	if (size < SHMMIN ||
+	    (ns->shm_ctlmax && size > ns->shm_ctlmax))
 		return -EINVAL;
 
-	if (ns->shm_tot + numpages > ns->shm_ctlall)
+	if (ns->shm_ctlall &&
+	    ns->shm_tot + numpages > ns->shm_ctlall)
 		return -ENOSPC;
 
 	shp = ipc_rcu_alloc(sizeof(*shp));
-- 
1.8.1.4

[PATCH v2] ipc,shm: disable shmmax and shmall by default

[Davidlohr Bueso]

From: Davidlohr Bueso <davidlohr@hp.com>

The default size for shmmax is, and always has been, 32Mb.
Today, in the XXI century, it seems that this value is rather small,
making users have to increase it via sysctl, which can cause
unnecessary work and userspace application workarounds[1].

Instead of choosing yet another arbitrary value, larger than 32Mb,
this patch disables the use of both shmmax and shmall by default,
allowing users to create segments of unlimited sizes. Users and
applications that already explicitly set these values through sysctl
are left untouched, and thus does not change any of the behavior.

So a value of 0 bytes or pages, for shmmax and shmall, respectively,
implies unlimited memory, as opposed to disabling sysv shared memory.
This is safe as 0 cannot possibly be used previously as SHMMIN is
hardcoded to 1 and cannot be modified.

This change allows Linux to treat shm just as regular anonymous memory.
One important difference between them, though, is handling out-of-memory
conditions: as opposed to regular anon memory, the OOM killer will not
free the memory as it is shm, allowing users to potentially abuse this.
To overcome this situation, the shm_rmid_forced option must be enabled.

[1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html

Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
---
Changes from v1:
 - Respect SHMMIN even when shmmax is 0 (unlimited).
   This fixes the shmget02 test that broke in v1. (per Manfred)

 - Update changelog regarding OOM description (per Kosaki)

 include/linux/shm.h      | 2 +-
 include/uapi/linux/shm.h | 8 ++++----
 ipc/shm.c                | 6 ++++--
 3 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/include/linux/shm.h b/include/linux/shm.h
index 1e2cd2e..0ca06a3 100644
--- a/include/linux/shm.h
+++ b/include/linux/shm.h
@@ -4,7 +4,7 @@
 #include <asm/page.h>
 #include <uapi/linux/shm.h>
 
-#define SHMALL (SHMMAX/PAGE_SIZE*(SHMMNI/16)) /* max shm system wide (pages) */
+#define SHMALL 0 /* max shm system wide (pages) */
 #include <asm/shmparam.h>
 struct shmid_kernel /* private to the kernel */
 {	
diff --git a/include/uapi/linux/shm.h b/include/uapi/linux/shm.h
index 78b6941..5f0ef28 100644
--- a/include/uapi/linux/shm.h
+++ b/include/uapi/linux/shm.h
@@ -9,14 +9,14 @@
 
 /*
  * SHMMAX, SHMMNI and SHMALL are upper limits are defaults which can
- * be increased by sysctl
+ * be increased by sysctl. By default, disable SHMMAX and SHMALL with
+ * 0 bytes, thus allowing processes to have unlimited shared memory.
  */
-
-#define SHMMAX 0x2000000		 /* max shared seg size (bytes) */
+#define SHMMAX 0		         /* max shared seg size (bytes) */
 #define SHMMIN 1			 /* min shared seg size (bytes) */
 #define SHMMNI 4096			 /* max num of segs system wide */
 #ifndef __KERNEL__
-#define SHMALL (SHMMAX/getpagesize()*(SHMMNI/16))
+#define SHMALL 0
 #endif
 #define SHMSEG SHMMNI			 /* max shared segs per process */
 
diff --git a/ipc/shm.c b/ipc/shm.c
index 7645961..8630561 100644
--- a/ipc/shm.c
+++ b/ipc/shm.c
@@ -490,10 +490,12 @@ static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
 	int id;
 	vm_flags_t acctflag = 0;
 
-	if (size < SHMMIN || size > ns->shm_ctlmax)
+	if (size < SHMMIN ||
+	    (ns->shm_ctlmax && size > ns->shm_ctlmax))
 		return -EINVAL;
 
-	if (ns->shm_tot + numpages > ns->shm_ctlall)
+	if (ns->shm_ctlall &&
+	    ns->shm_tot + numpages > ns->shm_ctlall)
 		return -ENOSPC;
 
 	shp = ipc_rcu_alloc(sizeof(*shp));
-- 
1.8.1.4



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Michael Kerrisk]

On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
> From: Davidlohr Bueso <davidlohr@hp.com>
>
> The default size for shmmax is, and always has been, 32Mb.
> Today, in the XXI century, it seems that this value is rather small,
> making users have to increase it via sysctl, which can cause
> unnecessary work and userspace application workarounds[1].
>
> Instead of choosing yet another arbitrary value, larger than 32Mb,
> this patch disables the use of both shmmax and shmall by default,
> allowing users to create segments of unlimited sizes. Users and
> applications that already explicitly set these values through sysctl
> are left untouched, and thus does not change any of the behavior.
>
> So a value of 0 bytes or pages, for shmmax and shmall, respectively,
> implies unlimited memory, as opposed to disabling sysv shared memory.
> This is safe as 0 cannot possibly be used previously as SHMMIN is
> hardcoded to 1 and cannot be modified.
>
> This change allows Linux to treat shm just as regular anonymous memory.
> One important difference between them, though, is handling out-of-memory
> conditions: as opposed to regular anon memory, the OOM killer will not
> free the memory as it is shm, allowing users to potentially abuse this.
> To overcome this situation, the shm_rmid_forced option must be enabled.
>
> [1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html
>
> Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>

Of the two proposed approaches (the other being
marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
me, since it allows strange users to maintain historical behavior
(i.e., the ability to set a limit) if they really want it, so:

Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>

One or two comments below, that you might consider for your v3 patch.

> ---
> Changes from v1:
>  - Respect SHMMIN even when shmmax is 0 (unlimited).
>    This fixes the shmget02 test that broke in v1. (per Manfred)
>
>  - Update changelog regarding OOM description (per Kosaki)
>
>  include/linux/shm.h      | 2 +-
>  include/uapi/linux/shm.h | 8 ++++----
>  ipc/shm.c                | 6 ++++--
>  3 files changed, 9 insertions(+), 7 deletions(-)
>
> diff --git a/include/linux/shm.h b/include/linux/shm.h
> index 1e2cd2e..0ca06a3 100644
> --- a/include/linux/shm.h
> +++ b/include/linux/shm.h
> @@ -4,7 +4,7 @@
>  #include <asm/page.h>
>  #include <uapi/linux/shm.h>
>
> -#define SHMALL (SHMMAX/PAGE_SIZE*(SHMMNI/16)) /* max shm system wide (pages) */
> +#define SHMALL 0 /* max shm system wide (pages) */
>  #include <asm/shmparam.h>
>  struct shmid_kernel /* private to the kernel */
>  {
> diff --git a/include/uapi/linux/shm.h b/include/uapi/linux/shm.h
> index 78b6941..5f0ef28 100644
> --- a/include/uapi/linux/shm.h
> +++ b/include/uapi/linux/shm.h
> @@ -9,14 +9,14 @@
>
>  /*
>   * SHMMAX, SHMMNI and SHMALL are upper limits are defaults which can
> - * be increased by sysctl
> + * be increased by sysctl. By default, disable SHMMAX and SHMALL with

s/increased/modified/

> + * 0 bytes, thus allowing processes to have unlimited shared memory.
>   */
> -
> -#define SHMMAX 0x2000000                /* max shared seg size (bytes) */
> +#define SHMMAX 0                        /* max shared seg size (bytes) */

I suggest: s/(bytes)/(bytes); 0 means "no limit" */

>  #define SHMMIN 1                        /* min shared seg size (bytes) */
>  #define SHMMNI 4096                     /* max num of segs system wide */
>  #ifndef __KERNEL__
> -#define SHMALL (SHMMAX/getpagesize()*(SHMMNI/16))
> +#define SHMALL 0

As long as we're here, let's add a meaningful comment to that one:

/* system-wide limit on number of pages of shared memory; 0 means "no limit" */

Cheers,

Michael


>  #endif
>  #define SHMSEG SHMMNI                   /* max shared segs per process */
>
> diff --git a/ipc/shm.c b/ipc/shm.c
> index 7645961..8630561 100644
> --- a/ipc/shm.c
> +++ b/ipc/shm.c
> @@ -490,10 +490,12 @@ static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
>         int id;
>         vm_flags_t acctflag = 0;
>
> -       if (size < SHMMIN || size > ns->shm_ctlmax)
> +       if (size < SHMMIN ||
> +           (ns->shm_ctlmax && size > ns->shm_ctlmax))
>                 return -EINVAL;
>
> -       if (ns->shm_tot + numpages > ns->shm_ctlall)
> +       if (ns->shm_ctlall &&
> +           ns->shm_tot + numpages > ns->shm_ctlall)
>                 return -ENOSPC;
>
>         shp = ipc_rcu_alloc(sizeof(*shp));
> --
> 1.8.1.4
>
>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>



-- 
Michael Kerrisk Linux man-pages maintainer;
http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface", http://blog.man7.org/

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Michael Kerrisk]

On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
> From: Davidlohr Bueso <davidlohr@hp.com>
>
> The default size for shmmax is, and always has been, 32Mb.
> Today, in the XXI century, it seems that this value is rather small,
> making users have to increase it via sysctl, which can cause
> unnecessary work and userspace application workarounds[1].
>
> Instead of choosing yet another arbitrary value, larger than 32Mb,
> this patch disables the use of both shmmax and shmall by default,
> allowing users to create segments of unlimited sizes. Users and
> applications that already explicitly set these values through sysctl
> are left untouched, and thus does not change any of the behavior.
>
> So a value of 0 bytes or pages, for shmmax and shmall, respectively,
> implies unlimited memory, as opposed to disabling sysv shared memory.
> This is safe as 0 cannot possibly be used previously as SHMMIN is
> hardcoded to 1 and cannot be modified.
>
> This change allows Linux to treat shm just as regular anonymous memory.
> One important difference between them, though, is handling out-of-memory
> conditions: as opposed to regular anon memory, the OOM killer will not
> free the memory as it is shm, allowing users to potentially abuse this.
> To overcome this situation, the shm_rmid_forced option must be enabled.
>
> [1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html
>
> Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>

Of the two proposed approaches (the other being
marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
me, since it allows strange users to maintain historical behavior
(i.e., the ability to set a limit) if they really want it, so:

Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>

One or two comments below, that you might consider for your v3 patch.

> ---
> Changes from v1:
>  - Respect SHMMIN even when shmmax is 0 (unlimited).
>    This fixes the shmget02 test that broke in v1. (per Manfred)
>
>  - Update changelog regarding OOM description (per Kosaki)
>
>  include/linux/shm.h      | 2 +-
>  include/uapi/linux/shm.h | 8 ++++----
>  ipc/shm.c                | 6 ++++--
>  3 files changed, 9 insertions(+), 7 deletions(-)
>
> diff --git a/include/linux/shm.h b/include/linux/shm.h
> index 1e2cd2e..0ca06a3 100644
> --- a/include/linux/shm.h
> +++ b/include/linux/shm.h
> @@ -4,7 +4,7 @@
>  #include <asm/page.h>
>  #include <uapi/linux/shm.h>
>
> -#define SHMALL (SHMMAX/PAGE_SIZE*(SHMMNI/16)) /* max shm system wide (pages) */
> +#define SHMALL 0 /* max shm system wide (pages) */
>  #include <asm/shmparam.h>
>  struct shmid_kernel /* private to the kernel */
>  {
> diff --git a/include/uapi/linux/shm.h b/include/uapi/linux/shm.h
> index 78b6941..5f0ef28 100644
> --- a/include/uapi/linux/shm.h
> +++ b/include/uapi/linux/shm.h
> @@ -9,14 +9,14 @@
>
>  /*
>   * SHMMAX, SHMMNI and SHMALL are upper limits are defaults which can
> - * be increased by sysctl
> + * be increased by sysctl. By default, disable SHMMAX and SHMALL with

s/increased/modified/

> + * 0 bytes, thus allowing processes to have unlimited shared memory.
>   */
> -
> -#define SHMMAX 0x2000000                /* max shared seg size (bytes) */
> +#define SHMMAX 0                        /* max shared seg size (bytes) */

I suggest: s/(bytes)/(bytes); 0 means "no limit" */

>  #define SHMMIN 1                        /* min shared seg size (bytes) */
>  #define SHMMNI 4096                     /* max num of segs system wide */
>  #ifndef __KERNEL__
> -#define SHMALL (SHMMAX/getpagesize()*(SHMMNI/16))
> +#define SHMALL 0

As long as we're here, let's add a meaningful comment to that one:

/* system-wide limit on number of pages of shared memory; 0 means "no limit" */

Cheers,

Michael


>  #endif
>  #define SHMSEG SHMMNI                   /* max shared segs per process */
>
> diff --git a/ipc/shm.c b/ipc/shm.c
> index 7645961..8630561 100644
> --- a/ipc/shm.c
> +++ b/ipc/shm.c
> @@ -490,10 +490,12 @@ static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
>         int id;
>         vm_flags_t acctflag = 0;
>
> -       if (size < SHMMIN || size > ns->shm_ctlmax)
> +       if (size < SHMMIN ||
> +           (ns->shm_ctlmax && size > ns->shm_ctlmax))
>                 return -EINVAL;
>
> -       if (ns->shm_tot + numpages > ns->shm_ctlall)
> +       if (ns->shm_ctlall &&
> +           ns->shm_tot + numpages > ns->shm_ctlall)
>                 return -ENOSPC;
>
>         shp = ipc_rcu_alloc(sizeof(*shp));
> --
> 1.8.1.4
>
>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>



-- 
Michael Kerrisk Linux man-pages maintainer;
http://www.kernel.org/doc/man-pages/
Author of "The Linux Programming Interface", http://blog.man7.org/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Manfred Spraul]

Hi Michael,

On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
>> From: Davidlohr Bueso <davidlohr@hp.com>
>>
>> The default size for shmmax is, and always has been, 32Mb.
>> Today, in the XXI century, it seems that this value is rather small,
>> making users have to increase it via sysctl, which can cause
>> unnecessary work and userspace application workarounds[1].
>>
>> Instead of choosing yet another arbitrary value, larger than 32Mb,
>> this patch disables the use of both shmmax and shmall by default,
>> allowing users to create segments of unlimited sizes. Users and
>> applications that already explicitly set these values through sysctl
>> are left untouched, and thus does not change any of the behavior.
>>
>> So a value of 0 bytes or pages, for shmmax and shmall, respectively,
>> implies unlimited memory, as opposed to disabling sysv shared memory.
>> This is safe as 0 cannot possibly be used previously as SHMMIN is
>> hardcoded to 1 and cannot be modified.
>>
>> This change allows Linux to treat shm just as regular anonymous memory.
>> One important difference between them, though, is handling out-of-memory
>> conditions: as opposed to regular anon memory, the OOM killer will not
>> free the memory as it is shm, allowing users to potentially abuse this.
>> To overcome this situation, the shm_rmid_forced option must be enabled.
>>
>> [1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html
>>
>> Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
>> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
>> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Of the two proposed approaches (the other being
> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
> me, since it allows strange users to maintain historical behavior
> (i.e., the ability to set a limit) if they really want it, so:
>
> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
>
> One or two comments below, that you might consider for your v3 patch.
I don't understand what you mean.

After a
     # echo 33554432 > /proc/sys/kernel/shmmax
     # echo 2097152 > /proc/sys/kernel/shmmax

both patches behave exactly identical.

There are only two differences:
- Davidlohr's patch handles
     # echo <really huge number that doesn't fit into 64-bit> > 
/proc/sys/kernel/shmmax
    With my patch, shmmax would end up as 0 and all allocations fail.

- My patch handles the case if some startup code/installer checks
    shmmax and complains if it is below the requirement of the application.

--
     Manfred

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Manfred Spraul]

Hi Michael,

On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
>> From: Davidlohr Bueso <davidlohr@hp.com>
>>
>> The default size for shmmax is, and always has been, 32Mb.
>> Today, in the XXI century, it seems that this value is rather small,
>> making users have to increase it via sysctl, which can cause
>> unnecessary work and userspace application workarounds[1].
>>
>> Instead of choosing yet another arbitrary value, larger than 32Mb,
>> this patch disables the use of both shmmax and shmall by default,
>> allowing users to create segments of unlimited sizes. Users and
>> applications that already explicitly set these values through sysctl
>> are left untouched, and thus does not change any of the behavior.
>>
>> So a value of 0 bytes or pages, for shmmax and shmall, respectively,
>> implies unlimited memory, as opposed to disabling sysv shared memory.
>> This is safe as 0 cannot possibly be used previously as SHMMIN is
>> hardcoded to 1 and cannot be modified.
>>
>> This change allows Linux to treat shm just as regular anonymous memory.
>> One important difference between them, though, is handling out-of-memory
>> conditions: as opposed to regular anon memory, the OOM killer will not
>> free the memory as it is shm, allowing users to potentially abuse this.
>> To overcome this situation, the shm_rmid_forced option must be enabled.
>>
>> [1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html
>>
>> Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
>> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
>> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> Of the two proposed approaches (the other being
> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
> me, since it allows strange users to maintain historical behavior
> (i.e., the ability to set a limit) if they really want it, so:
>
> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
>
> One or two comments below, that you might consider for your v3 patch.
I don't understand what you mean.

After a
     # echo 33554432 > /proc/sys/kernel/shmmax
     # echo 2097152 > /proc/sys/kernel/shmmax

both patches behave exactly identical.

There are only two differences:
- Davidlohr's patch handles
     # echo <really huge number that doesn't fit into 64-bit> > 
/proc/sys/kernel/shmmax
    With my patch, shmmax would end up as 0 and all allocations fail.

- My patch handles the case if some startup code/installer checks
    shmmax and complains if it is below the requirement of the application.

--
     Manfred

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Michael Kerrisk (man-pages)]

Hi Manfred!

On Thu, Apr 17, 2014 at 6:22 PM, Manfred Spraul
<manfred@colorfullife.com> wrote:
> Hi Michael,
>
>
> On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
>>
>> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
>>>
>>> From: Davidlohr Bueso <davidlohr@hp.com>
>>>
>>> The default size for shmmax is, and always has been, 32Mb.
>>> Today, in the XXI century, it seems that this value is rather small,
>>> making users have to increase it via sysctl, which can cause
>>> unnecessary work and userspace application workarounds[1].
>>>
>>> Instead of choosing yet another arbitrary value, larger than 32Mb,
>>> this patch disables the use of both shmmax and shmall by default,
>>> allowing users to create segments of unlimited sizes. Users and
>>> applications that already explicitly set these values through sysctl
>>> are left untouched, and thus does not change any of the behavior.
>>>
>>> So a value of 0 bytes or pages, for shmmax and shmall, respectively,
>>> implies unlimited memory, as opposed to disabling sysv shared memory.
>>> This is safe as 0 cannot possibly be used previously as SHMMIN is
>>> hardcoded to 1 and cannot be modified.
>>>
>>> This change allows Linux to treat shm just as regular anonymous memory.
>>> One important difference between them, though, is handling out-of-memory
>>> conditions: as opposed to regular anon memory, the OOM killer will not
>>> free the memory as it is shm, allowing users to potentially abuse this.
>>> To overcome this situation, the shm_rmid_forced option must be enabled.
>>>
>>> [1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html
>>>
>>> Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
>>> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
>>> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>>
>> Of the two proposed approaches (the other being
>> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
>> me, since it allows strange users to maintain historical behavior
>> (i.e., the ability to set a limit) if they really want it, so:
>>
>> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
>>
>> One or two comments below, that you might consider for your v3 patch.
>
> I don't understand what you mean.

As noted in the other mail, you don't understand, because I was being
dense (and misled a little by the commit message).

> After a
>     # echo 33554432 > /proc/sys/kernel/shmmax
>     # echo 2097152 > /proc/sys/kernel/shmmax
>
> both patches behave exactly identical.

Yes.

> There are only two differences:
> - Davidlohr's patch handles
>     # echo <really huge number that doesn't fit into 64-bit> >
> /proc/sys/kernel/shmmax
>    With my patch, shmmax would end up as 0 and all allocations fail.
>
> - My patch handles the case if some startup code/installer checks
>    shmmax and complains if it is below the requirement of the application.

Thanks for that clarification. I withdraw my Ack. In fact, maybe I
even like your approach a little more, because of that last point. Did
one of you not yet manage to persuade the other to his point of view
yet?

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Michael Kerrisk (man-pages)]

Hi Manfred!

On Thu, Apr 17, 2014 at 6:22 PM, Manfred Spraul
<manfred@colorfullife.com> wrote:
> Hi Michael,
>
>
> On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
>>
>> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
>>>
>>> From: Davidlohr Bueso <davidlohr@hp.com>
>>>
>>> The default size for shmmax is, and always has been, 32Mb.
>>> Today, in the XXI century, it seems that this value is rather small,
>>> making users have to increase it via sysctl, which can cause
>>> unnecessary work and userspace application workarounds[1].
>>>
>>> Instead of choosing yet another arbitrary value, larger than 32Mb,
>>> this patch disables the use of both shmmax and shmall by default,
>>> allowing users to create segments of unlimited sizes. Users and
>>> applications that already explicitly set these values through sysctl
>>> are left untouched, and thus does not change any of the behavior.
>>>
>>> So a value of 0 bytes or pages, for shmmax and shmall, respectively,
>>> implies unlimited memory, as opposed to disabling sysv shared memory.
>>> This is safe as 0 cannot possibly be used previously as SHMMIN is
>>> hardcoded to 1 and cannot be modified.
>>>
>>> This change allows Linux to treat shm just as regular anonymous memory.
>>> One important difference between them, though, is handling out-of-memory
>>> conditions: as opposed to regular anon memory, the OOM killer will not
>>> free the memory as it is shm, allowing users to potentially abuse this.
>>> To overcome this situation, the shm_rmid_forced option must be enabled.
>>>
>>> [1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html
>>>
>>> Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
>>> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
>>> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
>>
>> Of the two proposed approaches (the other being
>> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
>> me, since it allows strange users to maintain historical behavior
>> (i.e., the ability to set a limit) if they really want it, so:
>>
>> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
>>
>> One or two comments below, that you might consider for your v3 patch.
>
> I don't understand what you mean.

As noted in the other mail, you don't understand, because I was being
dense (and misled a little by the commit message).

> After a
>     # echo 33554432 > /proc/sys/kernel/shmmax
>     # echo 2097152 > /proc/sys/kernel/shmmax
>
> both patches behave exactly identical.

Yes.

> There are only two differences:
> - Davidlohr's patch handles
>     # echo <really huge number that doesn't fit into 64-bit> >
> /proc/sys/kernel/shmmax
>    With my patch, shmmax would end up as 0 and all allocations fail.
>
> - My patch handles the case if some startup code/installer checks
>    shmmax and complains if it is below the requirement of the application.

Thanks for that clarification. I withdraw my Ack. In fact, maybe I
even like your approach a little more, because of that last point. Did
one of you not yet manage to persuade the other to his point of view
yet?

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Davidlohr Bueso]

On Thu, 2014-04-17 at 22:23 +0200, Michael Kerrisk (man-pages) wrote:
> Hi Manfred!
> 
> On Thu, Apr 17, 2014 at 6:22 PM, Manfred Spraul
> <manfred@colorfullife.com> wrote:
> > Hi Michael,
> >
> >
> > On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
> >>
> >> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
> >>>
> >>> From: Davidlohr Bueso <davidlohr@hp.com>
> >>>
> >>> The default size for shmmax is, and always has been, 32Mb.
> >>> Today, in the XXI century, it seems that this value is rather small,
> >>> making users have to increase it via sysctl, which can cause
> >>> unnecessary work and userspace application workarounds[1].
> >>>
> >>> Instead of choosing yet another arbitrary value, larger than 32Mb,
> >>> this patch disables the use of both shmmax and shmall by default,
> >>> allowing users to create segments of unlimited sizes. Users and
> >>> applications that already explicitly set these values through sysctl
> >>> are left untouched, and thus does not change any of the behavior.
> >>>
> >>> So a value of 0 bytes or pages, for shmmax and shmall, respectively,
> >>> implies unlimited memory, as opposed to disabling sysv shared memory.
> >>> This is safe as 0 cannot possibly be used previously as SHMMIN is
> >>> hardcoded to 1 and cannot be modified.
> >>>
> >>> This change allows Linux to treat shm just as regular anonymous memory.
> >>> One important difference between them, though, is handling out-of-memory
> >>> conditions: as opposed to regular anon memory, the OOM killer will not
> >>> free the memory as it is shm, allowing users to potentially abuse this.
> >>> To overcome this situation, the shm_rmid_forced option must be enabled.
> >>>
> >>> [1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html
> >>>
> >>> Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
> >>> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> >>> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> >>
> >> Of the two proposed approaches (the other being
> >> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
> >> me, since it allows strange users to maintain historical behavior
> >> (i.e., the ability to set a limit) if they really want it, so:
> >>
> >> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
> >>
> >> One or two comments below, that you might consider for your v3 patch.
> >
> > I don't understand what you mean.
> 
> As noted in the other mail, you don't understand, because I was being
> dense (and misled a little by the commit message).
> 
> > After a
> >     # echo 33554432 > /proc/sys/kernel/shmmax
> >     # echo 2097152 > /proc/sys/kernel/shmmax
> >
> > both patches behave exactly identical.
> 
> Yes.
> 
> > There are only two differences:
> > - Davidlohr's patch handles
> >     # echo <really huge number that doesn't fit into 64-bit> >
> > /proc/sys/kernel/shmmax
> >    With my patch, shmmax would end up as 0 and all allocations fail.
> >
> > - My patch handles the case if some startup code/installer checks
> >    shmmax and complains if it is below the requirement of the application.
> 
> Thanks for that clarification. I withdraw my Ack. 

:(

> In fact, maybe I
> even like your approach a little more, because of that last point.

And it is a fair point. However, this is my counter argument: if users
are checking shmmax then they sure better be checking shmmin as well! So
if my patch causes shmctl(,IPC_INFO,) to return shminfo.shmmax = 0 and a
user only checks this value and breaks the application, then *he's*
doing it wrong. Checking shmmin is just as important... 0 value is
*bogus*, heck it even says so in shmctl's manpage.

>  Did
> one of you not yet manage to persuade the other to his point of view
> yet?

I think we've left that up to akpm.

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Davidlohr Bueso]

On Thu, 2014-04-17 at 22:23 +0200, Michael Kerrisk (man-pages) wrote:
> Hi Manfred!
> 
> On Thu, Apr 17, 2014 at 6:22 PM, Manfred Spraul
> <manfred@colorfullife.com> wrote:
> > Hi Michael,
> >
> >
> > On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
> >>
> >> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
> >>>
> >>> From: Davidlohr Bueso <davidlohr@hp.com>
> >>>
> >>> The default size for shmmax is, and always has been, 32Mb.
> >>> Today, in the XXI century, it seems that this value is rather small,
> >>> making users have to increase it via sysctl, which can cause
> >>> unnecessary work and userspace application workarounds[1].
> >>>
> >>> Instead of choosing yet another arbitrary value, larger than 32Mb,
> >>> this patch disables the use of both shmmax and shmall by default,
> >>> allowing users to create segments of unlimited sizes. Users and
> >>> applications that already explicitly set these values through sysctl
> >>> are left untouched, and thus does not change any of the behavior.
> >>>
> >>> So a value of 0 bytes or pages, for shmmax and shmall, respectively,
> >>> implies unlimited memory, as opposed to disabling sysv shared memory.
> >>> This is safe as 0 cannot possibly be used previously as SHMMIN is
> >>> hardcoded to 1 and cannot be modified.
> >>>
> >>> This change allows Linux to treat shm just as regular anonymous memory.
> >>> One important difference between them, though, is handling out-of-memory
> >>> conditions: as opposed to regular anon memory, the OOM killer will not
> >>> free the memory as it is shm, allowing users to potentially abuse this.
> >>> To overcome this situation, the shm_rmid_forced option must be enabled.
> >>>
> >>> [1]: http://rhaas.blogspot.com/2012/06/absurd-shared-memory-limits.html
> >>>
> >>> Signed-off-by: Davidlohr Bueso <davidlohr@hp.com>
> >>> Acked-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
> >>> Acked-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
> >>
> >> Of the two proposed approaches (the other being
> >> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
> >> me, since it allows strange users to maintain historical behavior
> >> (i.e., the ability to set a limit) if they really want it, so:
> >>
> >> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
> >>
> >> One or two comments below, that you might consider for your v3 patch.
> >
> > I don't understand what you mean.
> 
> As noted in the other mail, you don't understand, because I was being
> dense (and misled a little by the commit message).
> 
> > After a
> >     # echo 33554432 > /proc/sys/kernel/shmmax
> >     # echo 2097152 > /proc/sys/kernel/shmmax
> >
> > both patches behave exactly identical.
> 
> Yes.
> 
> > There are only two differences:
> > - Davidlohr's patch handles
> >     # echo <really huge number that doesn't fit into 64-bit> >
> > /proc/sys/kernel/shmmax
> >    With my patch, shmmax would end up as 0 and all allocations fail.
> >
> > - My patch handles the case if some startup code/installer checks
> >    shmmax and complains if it is below the requirement of the application.
> 
> Thanks for that clarification. I withdraw my Ack. 

:(

> In fact, maybe I
> even like your approach a little more, because of that last point.

And it is a fair point. However, this is my counter argument: if users
are checking shmmax then they sure better be checking shmmin as well! So
if my patch causes shmctl(,IPC_INFO,) to return shminfo.shmmax = 0 and a
user only checks this value and breaks the application, then *he's*
doing it wrong. Checking shmmin is just as important... 0 value is
*bogus*, heck it even says so in shmctl's manpage.

>  Did
> one of you not yet manage to persuade the other to his point of view
> yet?

I think we've left that up to akpm.



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Michael Kerrisk (man-pages)]

Hello Davidlohr,

On Fri, Apr 18, 2014 at 12:31 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
> On Thu, 2014-04-17 at 22:23 +0200, Michael Kerrisk (man-pages) wrote:
>> Hi Manfred!
>>
>> On Thu, Apr 17, 2014 at 6:22 PM, Manfred Spraul
>> <manfred@colorfullife.com> wrote:
>> > Hi Michael,
>> >
>> >
>> > On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
>> >>
>> >> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:

[...]

>> >> Of the two proposed approaches (the other being
>> >> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
>> >> me, since it allows strange users to maintain historical behavior
>> >> (i.e., the ability to set a limit) if they really want it, so:
>> >>
>> >> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
>> >>
>> >> One or two comments below, that you might consider for your v3 patch.
>> >
>> > I don't understand what you mean.
>>
>> As noted in the other mail, you don't understand, because I was being
>> dense (and misled a little by the commit message).
>>
>> > After a
>> >     # echo 33554432 > /proc/sys/kernel/shmmax
>> >     # echo 2097152 > /proc/sys/kernel/shmmax
>> >
>> > both patches behave exactly identical.
>>
>> Yes.
>>
>> > There are only two differences:
>> > - Davidlohr's patch handles
>> >     # echo <really huge number that doesn't fit into 64-bit> >
>> > /proc/sys/kernel/shmmax
>> >    With my patch, shmmax would end up as 0 and all allocations fail.
>> >
>> > - My patch handles the case if some startup code/installer checks
>> >    shmmax and complains if it is below the requirement of the application.
>>
>> Thanks for that clarification. I withdraw my Ack.
>
> :(
>
>> In fact, maybe I
>> even like your approach a little more, because of that last point.
>
> And it is a fair point. However, this is my counter argument: if users
> are checking shmmax then they sure better be checking shmmin as well! So
> if my patch causes shmctl(,IPC_INFO,) to return shminfo.shmmax = 0 and a
> user only checks this value and breaks the application, then *he's*
> doing it wrong. Checking shmmin is just as important...  0 value is
> *bogus*,

That counter-argument sounds bogus. On all systems that I know/knew
of, SHMIN always defaulted to 1. (Stevens APUE 1e documents this as
the typical default even as far back as 1992.) Furthermore, the limit
was always 1 on Linux, and as far as I know it has always been
immutable. I very much doubt any sysadmin ever changed SHMMIN (why
would they?), even on those systems where it was possible (and both
SHMMIN and SHMMAX seem to have been obsolete on Solaris for some time
now), or that any application ever checked the limit.

Probably the only thing that matters in this discussion is the Linux
behavior (set-up scripts will in any case be tailored to different
OSes): SHMMIN has always been fixed at 1, and so, likely ignored by
apps and install scripts. Thus, it seems difficult to argue that
checking SHMMIN is just as important as checking SHMMAX.

> heck it even says so in shmctl's manpage.

All it says in the man page is that the limit is (always) 1. A 0 value
isn't bogus; it's merely impossible.

>
>>  Did
>> one of you not yet manage to persuade the other to his point of view
>> yet?
>
> I think we've left that up to akpm.

Well, I mean, it's not like Andrew needs the extra work, right? It's a
small thing, but it makes Andrew's life a little easier when you can
give him an agreed solution, rather than asking him to make a
decision.

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Michael Kerrisk (man-pages)]

Hello Davidlohr,

On Fri, Apr 18, 2014 at 12:31 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
> On Thu, 2014-04-17 at 22:23 +0200, Michael Kerrisk (man-pages) wrote:
>> Hi Manfred!
>>
>> On Thu, Apr 17, 2014 at 6:22 PM, Manfred Spraul
>> <manfred@colorfullife.com> wrote:
>> > Hi Michael,
>> >
>> >
>> > On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
>> >>
>> >> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:

[...]

>> >> Of the two proposed approaches (the other being
>> >> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
>> >> me, since it allows strange users to maintain historical behavior
>> >> (i.e., the ability to set a limit) if they really want it, so:
>> >>
>> >> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
>> >>
>> >> One or two comments below, that you might consider for your v3 patch.
>> >
>> > I don't understand what you mean.
>>
>> As noted in the other mail, you don't understand, because I was being
>> dense (and misled a little by the commit message).
>>
>> > After a
>> >     # echo 33554432 > /proc/sys/kernel/shmmax
>> >     # echo 2097152 > /proc/sys/kernel/shmmax
>> >
>> > both patches behave exactly identical.
>>
>> Yes.
>>
>> > There are only two differences:
>> > - Davidlohr's patch handles
>> >     # echo <really huge number that doesn't fit into 64-bit> >
>> > /proc/sys/kernel/shmmax
>> >    With my patch, shmmax would end up as 0 and all allocations fail.
>> >
>> > - My patch handles the case if some startup code/installer checks
>> >    shmmax and complains if it is below the requirement of the application.
>>
>> Thanks for that clarification. I withdraw my Ack.
>
> :(
>
>> In fact, maybe I
>> even like your approach a little more, because of that last point.
>
> And it is a fair point. However, this is my counter argument: if users
> are checking shmmax then they sure better be checking shmmin as well! So
> if my patch causes shmctl(,IPC_INFO,) to return shminfo.shmmax = 0 and a
> user only checks this value and breaks the application, then *he's*
> doing it wrong. Checking shmmin is just as important...  0 value is
> *bogus*,

That counter-argument sounds bogus. On all systems that I know/knew
of, SHMIN always defaulted to 1. (Stevens APUE 1e documents this as
the typical default even as far back as 1992.) Furthermore, the limit
was always 1 on Linux, and as far as I know it has always been
immutable. I very much doubt any sysadmin ever changed SHMMIN (why
would they?), even on those systems where it was possible (and both
SHMMIN and SHMMAX seem to have been obsolete on Solaris for some time
now), or that any application ever checked the limit.

Probably the only thing that matters in this discussion is the Linux
behavior (set-up scripts will in any case be tailored to different
OSes): SHMMIN has always been fixed at 1, and so, likely ignored by
apps and install scripts. Thus, it seems difficult to argue that
checking SHMMIN is just as important as checking SHMMAX.

> heck it even says so in shmctl's manpage.

All it says in the man page is that the limit is (always) 1. A 0 value
isn't bogus; it's merely impossible.

>
>>  Did
>> one of you not yet manage to persuade the other to his point of view
>> yet?
>
> I think we've left that up to akpm.

Well, I mean, it's not like Andrew needs the extra work, right? It's a
small thing, but it makes Andrew's life a little easier when you can
give him an agreed solution, rather than asking him to make a
decision.

Cheers,

Michael

-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Davidlohr Bueso]

On Fri, 2014-04-18 at 07:28 +0200, Michael Kerrisk (man-pages) wrote:
> Hello Davidlohr,
> 
> On Fri, Apr 18, 2014 at 12:31 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
> > On Thu, 2014-04-17 at 22:23 +0200, Michael Kerrisk (man-pages) wrote:
> >> Hi Manfred!
> >>
> >> On Thu, Apr 17, 2014 at 6:22 PM, Manfred Spraul
> >> <manfred@colorfullife.com> wrote:
> >> > Hi Michael,
> >> >
> >> >
> >> > On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
> >> >>
> >> >> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
> 
> [...]
> 
> >> >> Of the two proposed approaches (the other being
> >> >> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
> >> >> me, since it allows strange users to maintain historical behavior
> >> >> (i.e., the ability to set a limit) if they really want it, so:
> >> >>
> >> >> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
> >> >>
> >> >> One or two comments below, that you might consider for your v3 patch.
> >> >
> >> > I don't understand what you mean.
> >>
> >> As noted in the other mail, you don't understand, because I was being
> >> dense (and misled a little by the commit message).
> >>
> >> > After a
> >> >     # echo 33554432 > /proc/sys/kernel/shmmax
> >> >     # echo 2097152 > /proc/sys/kernel/shmmax
> >> >
> >> > both patches behave exactly identical.
> >>
> >> Yes.
> >>
> >> > There are only two differences:
> >> > - Davidlohr's patch handles
> >> >     # echo <really huge number that doesn't fit into 64-bit> >
> >> > /proc/sys/kernel/shmmax
> >> >    With my patch, shmmax would end up as 0 and all allocations fail.
> >> >
> >> > - My patch handles the case if some startup code/installer checks
> >> >    shmmax and complains if it is below the requirement of the application.
> >>
> >> Thanks for that clarification. I withdraw my Ack.
> >
> > :(
> >
> >> In fact, maybe I
> >> even like your approach a little more, because of that last point.
> >
> > And it is a fair point. However, this is my counter argument: if users
> > are checking shmmax then they sure better be checking shmmin as well! So
> > if my patch causes shmctl(,IPC_INFO,) to return shminfo.shmmax = 0 and a
> > user only checks this value and breaks the application, then *he's*
> > doing it wrong. Checking shmmin is just as important...  0 value is
> > *bogus*,
> 
> That counter-argument sounds bogus. On all systems that I know/knew
> of, SHMIN always defaulted to 1. (Stevens APUE 1e documents this as
> the typical default even as far back as 1992.) Furthermore, the limit
> was always 1 on Linux, and as far as I know it has always been
> immutable. I very much doubt any sysadmin ever changed SHMMIN (why
> would they?), even on those systems where it was possible (and both
> SHMMIN and SHMMAX seem to have been obsolete on Solaris for some time
> now), or that any application ever checked the limit.

I'm not talking about *changing* SHMMIN, but checking for the value...
anything less than 1 is of course complete crap. And that's not the
kernel's fault.

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Davidlohr Bueso]

On Fri, 2014-04-18 at 07:28 +0200, Michael Kerrisk (man-pages) wrote:
> Hello Davidlohr,
> 
> On Fri, Apr 18, 2014 at 12:31 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
> > On Thu, 2014-04-17 at 22:23 +0200, Michael Kerrisk (man-pages) wrote:
> >> Hi Manfred!
> >>
> >> On Thu, Apr 17, 2014 at 6:22 PM, Manfred Spraul
> >> <manfred@colorfullife.com> wrote:
> >> > Hi Michael,
> >> >
> >> >
> >> > On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
> >> >>
> >> >> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
> 
> [...]
> 
> >> >> Of the two proposed approaches (the other being
> >> >> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
> >> >> me, since it allows strange users to maintain historical behavior
> >> >> (i.e., the ability to set a limit) if they really want it, so:
> >> >>
> >> >> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
> >> >>
> >> >> One or two comments below, that you might consider for your v3 patch.
> >> >
> >> > I don't understand what you mean.
> >>
> >> As noted in the other mail, you don't understand, because I was being
> >> dense (and misled a little by the commit message).
> >>
> >> > After a
> >> >     # echo 33554432 > /proc/sys/kernel/shmmax
> >> >     # echo 2097152 > /proc/sys/kernel/shmmax
> >> >
> >> > both patches behave exactly identical.
> >>
> >> Yes.
> >>
> >> > There are only two differences:
> >> > - Davidlohr's patch handles
> >> >     # echo <really huge number that doesn't fit into 64-bit> >
> >> > /proc/sys/kernel/shmmax
> >> >    With my patch, shmmax would end up as 0 and all allocations fail.
> >> >
> >> > - My patch handles the case if some startup code/installer checks
> >> >    shmmax and complains if it is below the requirement of the application.
> >>
> >> Thanks for that clarification. I withdraw my Ack.
> >
> > :(
> >
> >> In fact, maybe I
> >> even like your approach a little more, because of that last point.
> >
> > And it is a fair point. However, this is my counter argument: if users
> > are checking shmmax then they sure better be checking shmmin as well! So
> > if my patch causes shmctl(,IPC_INFO,) to return shminfo.shmmax = 0 and a
> > user only checks this value and breaks the application, then *he's*
> > doing it wrong. Checking shmmin is just as important...  0 value is
> > *bogus*,
> 
> That counter-argument sounds bogus. On all systems that I know/knew
> of, SHMIN always defaulted to 1. (Stevens APUE 1e documents this as
> the typical default even as far back as 1992.) Furthermore, the limit
> was always 1 on Linux, and as far as I know it has always been
> immutable. I very much doubt any sysadmin ever changed SHMMIN (why
> would they?), even on those systems where it was possible (and both
> SHMMIN and SHMMAX seem to have been obsolete on Solaris for some time
> now), or that any application ever checked the limit.

I'm not talking about *changing* SHMMIN, but checking for the value...
anything less than 1 is of course complete crap. And that's not the
kernel's fault.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Michael Kerrisk (man-pages)]

On 04/18/2014 06:29 PM, Davidlohr Bueso wrote:
> On Fri, 2014-04-18 at 07:28 +0200, Michael Kerrisk (man-pages) wrote:
>> Hello Davidlohr,
>>
>> On Fri, Apr 18, 2014 at 12:31 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
>>> On Thu, 2014-04-17 at 22:23 +0200, Michael Kerrisk (man-pages) wrote:
>>>> Hi Manfred!
>>>>
>>>> On Thu, Apr 17, 2014 at 6:22 PM, Manfred Spraul
>>>> <manfred@colorfullife.com> wrote:
>>>>> Hi Michael,
>>>>>
>>>>>
>>>>> On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
>>>>>>
>>>>>> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
>>
>> [...]
>>
>>>>>> Of the two proposed approaches (the other being
>>>>>> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
>>>>>> me, since it allows strange users to maintain historical behavior
>>>>>> (i.e., the ability to set a limit) if they really want it, so:
>>>>>>
>>>>>> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
>>>>>>
>>>>>> One or two comments below, that you might consider for your v3 patch.
>>>>>
>>>>> I don't understand what you mean.
>>>>
>>>> As noted in the other mail, you don't understand, because I was being
>>>> dense (and misled a little by the commit message).
>>>>
>>>>> After a
>>>>>     # echo 33554432 > /proc/sys/kernel/shmmax
>>>>>     # echo 2097152 > /proc/sys/kernel/shmmax
>>>>>
>>>>> both patches behave exactly identical.
>>>>
>>>> Yes.
>>>>
>>>>> There are only two differences:
>>>>> - Davidlohr's patch handles
>>>>>     # echo <really huge number that doesn't fit into 64-bit> >
>>>>> /proc/sys/kernel/shmmax
>>>>>    With my patch, shmmax would end up as 0 and all allocations fail.
>>>>>
>>>>> - My patch handles the case if some startup code/installer checks
>>>>>    shmmax and complains if it is below the requirement of the application.
>>>>
>>>> Thanks for that clarification. I withdraw my Ack.
>>>
>>> :(
>>>
>>>> In fact, maybe I
>>>> even like your approach a little more, because of that last point.
>>>
>>> And it is a fair point. However, this is my counter argument: if users
>>> are checking shmmax then they sure better be checking shmmin as well! So
>>> if my patch causes shmctl(,IPC_INFO,) to return shminfo.shmmax = 0 and a
>>> user only checks this value and breaks the application, then *he's*
>>> doing it wrong. Checking shmmin is just as important...  0 value is
>>> *bogus*,
>>
>> That counter-argument sounds bogus. On all systems that I know/knew
>> of, SHMIN always defaulted to 1. (Stevens APUE 1e documents this as
>> the typical default even as far back as 1992.) Furthermore, the limit
>> was always 1 on Linux, and as far as I know it has always been
>> immutable. I very much doubt any sysadmin ever changed SHMMIN (why
>> would they?), even on those systems where it was possible (and both
>> SHMMIN and SHMMAX seem to have been obsolete on Solaris for some time
>> now), or that any application ever checked the limit.
> 
> I'm not talking about *changing* SHMMIN, but checking for the value...
> anything less than 1 is of course complete crap. And that's not the
> kernel's fault.

Okay--I think I must be missing something. If shmmin is immutable, with the
value 1, why would anyone ever need to check its value? How can checking
it be just as important as checking shmmax?


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Re: [PATCH v2] ipc,shm: disable shmmax and shmall by default

[Michael Kerrisk (man-pages)]

On 04/18/2014 06:29 PM, Davidlohr Bueso wrote:
> On Fri, 2014-04-18 at 07:28 +0200, Michael Kerrisk (man-pages) wrote:
>> Hello Davidlohr,
>>
>> On Fri, Apr 18, 2014 at 12:31 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
>>> On Thu, 2014-04-17 at 22:23 +0200, Michael Kerrisk (man-pages) wrote:
>>>> Hi Manfred!
>>>>
>>>> On Thu, Apr 17, 2014 at 6:22 PM, Manfred Spraul
>>>> <manfred@colorfullife.com> wrote:
>>>>> Hi Michael,
>>>>>
>>>>>
>>>>> On 04/17/2014 12:53 PM, Michael Kerrisk wrote:
>>>>>>
>>>>>> On Sat, Apr 12, 2014 at 5:22 AM, Davidlohr Bueso <davidlohr@hp.com> wrote:
>>
>> [...]
>>
>>>>>> Of the two proposed approaches (the other being
>>>>>> marc.info/?l=linux-kernel&m=139730332306185), this looks preferable to
>>>>>> me, since it allows strange users to maintain historical behavior
>>>>>> (i.e., the ability to set a limit) if they really want it, so:
>>>>>>
>>>>>> Acked-by: Michael Kerrisk <mtk.manpages@gmail.com>
>>>>>>
>>>>>> One or two comments below, that you might consider for your v3 patch.
>>>>>
>>>>> I don't understand what you mean.
>>>>
>>>> As noted in the other mail, you don't understand, because I was being
>>>> dense (and misled a little by the commit message).
>>>>
>>>>> After a
>>>>>     # echo 33554432 > /proc/sys/kernel/shmmax
>>>>>     # echo 2097152 > /proc/sys/kernel/shmmax
>>>>>
>>>>> both patches behave exactly identical.
>>>>
>>>> Yes.
>>>>
>>>>> There are only two differences:
>>>>> - Davidlohr's patch handles
>>>>>     # echo <really huge number that doesn't fit into 64-bit> >
>>>>> /proc/sys/kernel/shmmax
>>>>>    With my patch, shmmax would end up as 0 and all allocations fail.
>>>>>
>>>>> - My patch handles the case if some startup code/installer checks
>>>>>    shmmax and complains if it is below the requirement of the application.
>>>>
>>>> Thanks for that clarification. I withdraw my Ack.
>>>
>>> :(
>>>
>>>> In fact, maybe I
>>>> even like your approach a little more, because of that last point.
>>>
>>> And it is a fair point. However, this is my counter argument: if users
>>> are checking shmmax then they sure better be checking shmmin as well! So
>>> if my patch causes shmctl(,IPC_INFO,) to return shminfo.shmmax = 0 and a
>>> user only checks this value and breaks the application, then *he's*
>>> doing it wrong. Checking shmmin is just as important...  0 value is
>>> *bogus*,
>>
>> That counter-argument sounds bogus. On all systems that I know/knew
>> of, SHMIN always defaulted to 1. (Stevens APUE 1e documents this as
>> the typical default even as far back as 1992.) Furthermore, the limit
>> was always 1 on Linux, and as far as I know it has always been
>> immutable. I very much doubt any sysadmin ever changed SHMMIN (why
>> would they?), even on those systems where it was possible (and both
>> SHMMIN and SHMMAX seem to have been obsolete on Solaris for some time
>> now), or that any application ever checked the limit.
> 
> I'm not talking about *changing* SHMMIN, but checking for the value...
> anything less than 1 is of course complete crap. And that's not the
> kernel's fault.

Okay--I think I must be missing something. If shmmin is immutable, with the
value 1, why would anyone ever need to check its value? How can checking
it be just as important as checking shmmax?


-- 
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>