* [PATCH] Sys V shared memory limited to 8TiB.
@ 2013-04-10 2:39 Robin Holt
2013-04-10 23:15 ` Andrew Morton
0 siblings, 1 reply; 5+ messages in thread
From: Robin Holt @ 2013-04-10 2:39 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel, Alex Thorlton
Trying to run an application which was trying to put data into half
of memory using shmget(), we found that having a shmall value below
8EiB-8TiB would prevent us from using anything more than 8TiB. By setting
kernel.shmall greater that 8EiB-8TiB would make the job work.
In the newseg() function, ns->shm_tot which, at 8TiB is INT_MAX.
ipc/shm.c:
458 static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
459 {
...
465 int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
...
474 if (ns->shm_tot + numpages > ns->shm_ctlall)
475 return -ENOSPC;
Signed-off-by: Robin Holt <holt@sgi.com>
Reported-by: Alex Thorlton <athorlton@sgi.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Stable Kernel Maintainers <stable@vger.kernel.org>
Cc: lkml <linux-kernel@vger.kernel.org>
---
include/linux/ipc_namespace.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/include/linux/ipc_namespace.h b/include/linux/ipc_namespace.h
index ae221a7..ca62eca 100644
--- a/include/linux/ipc_namespace.h
+++ b/include/linux/ipc_namespace.h
@@ -43,8 +43,8 @@ struct ipc_namespace {
size_t shm_ctlmax;
size_t shm_ctlall;
+ unsigned long shm_tot;
int shm_ctlmni;
- int shm_tot;
/*
* Defines whether IPC_RMID is forced for _all_ shm segments regardless
* of shmctl()
--
1.8.1.2
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH] Sys V shared memory limited to 8TiB.
2013-04-10 2:39 [PATCH] Sys V shared memory limited to 8TiB Robin Holt
@ 2013-04-10 23:15 ` Andrew Morton
2013-04-11 2:42 ` Robin Holt
0 siblings, 1 reply; 5+ messages in thread
From: Andrew Morton @ 2013-04-10 23:15 UTC (permalink / raw)
To: Robin Holt; +Cc: linux-kernel, Alex Thorlton
On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <holt@sgi.com> wrote:
> Trying to run an application which was trying to put data into half
> of memory using shmget(), we found that having a shmall value below
> 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting
> kernel.shmall greater that 8EiB-8TiB would make the job work.
>
> In the newseg() function, ns->shm_tot which, at 8TiB is INT_MAX.
You have way too much memory.
> ipc/shm.c:
> 458 static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
> 459 {
> ...
> 465 int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
> ...
> 474 if (ns->shm_tot + numpages > ns->shm_ctlall)
> 475 return -ENOSPC;
>
> ...
>
> --- a/include/linux/ipc_namespace.h
> +++ b/include/linux/ipc_namespace.h
> @@ -43,8 +43,8 @@ struct ipc_namespace {
>
> size_t shm_ctlmax;
> size_t shm_ctlall;
> + unsigned long shm_tot;
> int shm_ctlmni;
> - int shm_tot;
> /*
> * Defines whether IPC_RMID is forced for _all_ shm segments regardless
> * of shmctl()
I reviewed everything for fallout from this and don't see any obvious
issues.
I do wonder about the appropriateness of the unsigned long type. Most
(but by no means all) code in this area uses size_t, and the
above-quoted ns->shm_ctlall is size_t.
And the above-quoted num_pages is `int'. Both the size and signedness
of `int' make no sense - what happens if the incoming ipc_params.u.size
is >2G?
So I'll add this, and ask whether ipc_namespace.shm_tot should be size_t?
--- a/ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix
+++ a/ipc/shm.c
@@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *
size_t size = params->u.size;
int error;
struct shmid_kernel *shp;
- int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
+ size_t numpages = (size + PAGE_SIZE - 1) >> PAGE_SHIFT;
struct file * file;
char name[13];
int id;
_
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] Sys V shared memory limited to 8TiB.
2013-04-10 23:15 ` Andrew Morton
@ 2013-04-11 2:42 ` Robin Holt
2013-04-11 21:07 ` Andrew Morton
0 siblings, 1 reply; 5+ messages in thread
From: Robin Holt @ 2013-04-11 2:42 UTC (permalink / raw)
To: Andrew Morton; +Cc: Robin Holt, linux-kernel, Alex Thorlton
On Wed, Apr 10, 2013 at 04:15:07PM -0700, Andrew Morton wrote:
> On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <holt@sgi.com> wrote:
>
> > Trying to run an application which was trying to put data into half
> > of memory using shmget(), we found that having a shmall value below
> > 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting
> > kernel.shmall greater that 8EiB-8TiB would make the job work.
> >
> > In the newseg() function, ns->shm_tot which, at 8TiB is INT_MAX.
>
> You have way too much memory.
>
> > ipc/shm.c:
> > 458 static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
> > 459 {
> > ...
> > 465 int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
> > ...
> > 474 if (ns->shm_tot + numpages > ns->shm_ctlall)
> > 475 return -ENOSPC;
> >
> > ...
> >
> > --- a/include/linux/ipc_namespace.h
> > +++ b/include/linux/ipc_namespace.h
> > @@ -43,8 +43,8 @@ struct ipc_namespace {
> >
> > size_t shm_ctlmax;
> > size_t shm_ctlall;
> > + unsigned long shm_tot;
> > int shm_ctlmni;
> > - int shm_tot;
> > /*
> > * Defines whether IPC_RMID is forced for _all_ shm segments regardless
> > * of shmctl()
>
> I reviewed everything for fallout from this and don't see any obvious
> issues.
>
> I do wonder about the appropriateness of the unsigned long type. Most
> (but by no means all) code in this area uses size_t, and the
> above-quoted ns->shm_ctlall is size_t.
The only reason I went with unsigned long instead of size_t was most
places in the kernel track stuff I recalled that was tracking stuff
in pages used unsigned longs. Also, I found shm_tot field in shm_info
structure was an unsigned long so this felt like a natural fit. I would
happily changed to size_t. Whatever you feel is right.
> And the above-quoted num_pages is `int'. Both the size and signedness
> of `int' make no sense - what happens if the incoming ipc_params.u.size
> is >2G?
>
> So I'll add this, and ask whether ipc_namespace.shm_tot should be size_t?
>
> --- a/ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix
> +++ a/ipc/shm.c
> @@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *
> size_t size = params->u.size;
> int error;
> struct shmid_kernel *shp;
> - int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
> + size_t numpages = (size + PAGE_SIZE - 1) >> PAGE_SHIFT;
I was holding off from that change only because I was asking for this
to go to stable and this doubles the size of the patch. ;)
> struct file * file;
> char name[13];
> int id;
> _
Thank you,
Robin
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] Sys V shared memory limited to 8TiB.
2013-04-11 2:42 ` Robin Holt
@ 2013-04-11 21:07 ` Andrew Morton
2013-04-11 21:10 ` Robin Holt
0 siblings, 1 reply; 5+ messages in thread
From: Andrew Morton @ 2013-04-11 21:07 UTC (permalink / raw)
To: Robin Holt; +Cc: linux-kernel, Alex Thorlton
On Wed, 10 Apr 2013 21:42:23 -0500 Robin Holt <holt@sgi.com> wrote:
> On Wed, Apr 10, 2013 at 04:15:07PM -0700, Andrew Morton wrote:
> > On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <holt@sgi.com> wrote:
> >
> > > Trying to run an application which was trying to put data into half
> > > of memory using shmget(), we found that having a shmall value below
> > > 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting
> > > kernel.shmall greater that 8EiB-8TiB would make the job work.
> > >
> > > In the newseg() function, ns->shm_tot which, at 8TiB is INT_MAX.
> >
> > You have way too much memory.
> >
> > > ipc/shm.c:
> > > 458 static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
> > > 459 {
> > > ...
> > > 465 int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
> > > ...
> > > 474 if (ns->shm_tot + numpages > ns->shm_ctlall)
> > > 475 return -ENOSPC;
> > >
> > > ...
> > >
> > > --- a/include/linux/ipc_namespace.h
> > > +++ b/include/linux/ipc_namespace.h
> > > @@ -43,8 +43,8 @@ struct ipc_namespace {
> > >
> > > size_t shm_ctlmax;
> > > size_t shm_ctlall;
> > > + unsigned long shm_tot;
> > > int shm_ctlmni;
> > > - int shm_tot;
> > > /*
> > > * Defines whether IPC_RMID is forced for _all_ shm segments regardless
> > > * of shmctl()
> >
> > I reviewed everything for fallout from this and don't see any obvious
> > issues.
> >
> > I do wonder about the appropriateness of the unsigned long type. Most
> > (but by no means all) code in this area uses size_t, and the
> > above-quoted ns->shm_ctlall is size_t.
>
> The only reason I went with unsigned long instead of size_t was most
> places in the kernel track stuff I recalled that was tracking stuff
> in pages used unsigned longs. Also, I found shm_tot field in shm_info
> structure was an unsigned long so this felt like a natural fit. I would
> happily changed to size_t. Whatever you feel is right.
I have no really strong feelings, but let's at least put some thought
into it. I do prefer ulong and find size_t to be a PITA. I guess it
doesn't matter much.
> > --- a/ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix
> > +++ a/ipc/shm.c
> > @@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *
> > size_t size = params->u.size;
> > int error;
> > struct shmid_kernel *shp;
> > - int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
> > + size_t numpages = (size + PAGE_SIZE - 1) >> PAGE_SHIFT;
>
> I was holding off from that change only because I was asking for this
> to go to stable and this doubles the size of the patch. ;)
It's a bug, isn't it? Is there anything else which prevents creation
of segments which are >=8TB?
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH] Sys V shared memory limited to 8TiB.
2013-04-11 21:07 ` Andrew Morton
@ 2013-04-11 21:10 ` Robin Holt
0 siblings, 0 replies; 5+ messages in thread
From: Robin Holt @ 2013-04-11 21:10 UTC (permalink / raw)
To: Andrew Morton; +Cc: Robin Holt, linux-kernel, Alex Thorlton
On Thu, Apr 11, 2013 at 02:07:08PM -0700, Andrew Morton wrote:
> On Wed, 10 Apr 2013 21:42:23 -0500 Robin Holt <holt@sgi.com> wrote:
>
> > On Wed, Apr 10, 2013 at 04:15:07PM -0700, Andrew Morton wrote:
> > > On Tue, 9 Apr 2013 21:39:24 -0500 Robin Holt <holt@sgi.com> wrote:
> > >
> > > > Trying to run an application which was trying to put data into half
> > > > of memory using shmget(), we found that having a shmall value below
> > > > 8EiB-8TiB would prevent us from using anything more than 8TiB. By setting
> > > > kernel.shmall greater that 8EiB-8TiB would make the job work.
> > > >
> > > > In the newseg() function, ns->shm_tot which, at 8TiB is INT_MAX.
> > >
> > > You have way too much memory.
> > >
> > > > ipc/shm.c:
> > > > 458 static int newseg(struct ipc_namespace *ns, struct ipc_params *params)
> > > > 459 {
> > > > ...
> > > > 465 int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
> > > > ...
> > > > 474 if (ns->shm_tot + numpages > ns->shm_ctlall)
> > > > 475 return -ENOSPC;
> > > >
> > > > ...
> > > >
> > > > --- a/include/linux/ipc_namespace.h
> > > > +++ b/include/linux/ipc_namespace.h
> > > > @@ -43,8 +43,8 @@ struct ipc_namespace {
> > > >
> > > > size_t shm_ctlmax;
> > > > size_t shm_ctlall;
> > > > + unsigned long shm_tot;
> > > > int shm_ctlmni;
> > > > - int shm_tot;
> > > > /*
> > > > * Defines whether IPC_RMID is forced for _all_ shm segments regardless
> > > > * of shmctl()
> > >
> > > I reviewed everything for fallout from this and don't see any obvious
> > > issues.
> > >
> > > I do wonder about the appropriateness of the unsigned long type. Most
> > > (but by no means all) code in this area uses size_t, and the
> > > above-quoted ns->shm_ctlall is size_t.
> >
> > The only reason I went with unsigned long instead of size_t was most
> > places in the kernel track stuff I recalled that was tracking stuff
> > in pages used unsigned longs. Also, I found shm_tot field in shm_info
> > structure was an unsigned long so this felt like a natural fit. I would
> > happily changed to size_t. Whatever you feel is right.
>
> I have no really strong feelings, but let's at least put some thought
> into it. I do prefer ulong and find size_t to be a PITA. I guess it
> doesn't matter much.
>
> > > --- a/ipc/shm.c~ipc-sysv-shared-memory-limited-to-8tib-fix
> > > +++ a/ipc/shm.c
> > > @@ -462,7 +462,7 @@ static int newseg(struct ipc_namespace *
> > > size_t size = params->u.size;
> > > int error;
> > > struct shmid_kernel *shp;
> > > - int numpages = (size + PAGE_SIZE -1) >> PAGE_SHIFT;
> > > + size_t numpages = (size + PAGE_SIZE - 1) >> PAGE_SHIFT;
> >
> > I was holding off from that change only because I was asking for this
> > to go to stable and this doubles the size of the patch. ;)
>
> It's a bug, isn't it? Is there anything else which prevents creation
> of segments which are >=8TB?
Definitely a bug. Have not tried to create a segment >= 8TiB. I can
give that a try tomorrow morning when I have a machine again to test with.
Thanks,
Robin
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2013-04-11 21:11 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-04-10 2:39 [PATCH] Sys V shared memory limited to 8TiB Robin Holt
2013-04-10 23:15 ` Andrew Morton
2013-04-11 2:42 ` Robin Holt
2013-04-11 21:07 ` Andrew Morton
2013-04-11 21:10 ` Robin Holt
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).