user- vs kernel-level resource sandbox for Linux?

user- vs kernel-level resource sandbox for Linux?

[Marek Habersack]

[-- Attachment #1: Type: text/plain, Size: 1268 bytes --]

Hello,

  I am looking for advice on how to limit resource (memory in particular)
usage on a linux machine (running either kernel v2.4 or2.6) on the per-user
(vs per-process) basis. I am aware that I could use Xen or UML for that
purpose, but I am wondering whether anybody knows any solution that can
implement that entirely in the userland (e.g. a monitor application that
intercepts system calls responsible for resource allocation and controls the
memory usage that way). My problem is apache which spawns a certain process
on which sometimes runs away and causes the kernel to kill apache, the
offending process and cause all fork(2) attempts to fail (which effectively
disables ssh). I've tried limiting resources on the apache startup, but that
isn't of much help since each apache process will get the same resources and
it's enough that several of them allocate too much memory at the same time
and the effect is as described above. I've also played with overcommit on
the 2.6 kernel in hope that it will stop the process from allocating
excessive amounts of memory, but it wasn't of much help either, alas...
  I would appreciate any pointers to the userland solutions for that problem
(if any exist) before I resort to Xen/UML.

  thanks in advance,

marek

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: user- vs kernel-level resource sandbox for Linux?

[Jeff Dike]

grendel@caudium.net said:
>   I would appreciate any pointers to the userland solutions for that
> problem (if any exist) before I resort to Xen/UML. 

UML would be exactly what you're looking for.

				Jeff

Re: user- vs kernel-level resource sandbox for Linux?

[Peter Chubb]

>>>>> "Jeff" == Jeff Dike <jdike@addtoit.com> writes:

Jeff> grendel@caudium.net said:
>> I would appreciate any pointers to the userland solutions for that
>> problem (if any exist) before I resort to Xen/UML.

Jeff> UML would be exactly what you're looking for.

Jeff> 				Jeff

apart from the performance hit :-(

There have been a number of different approaches proposed in the past
to limit real memory usage per-process; search for RSS limit in the
archives.

--
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*

Re: user- vs kernel-level resource sandbox for Linux?

[Marek Habersack]

[-- Attachment #1: Type: text/plain, Size: 971 bytes --]

On Tue, Nov 30, 2004 at 09:13:03AM +1100, Peter Chubb scribbled:
> >>>>> "Jeff" == Jeff Dike <jdike@addtoit.com> writes:
> 
> Jeff> grendel@caudium.net said:
> >> I would appreciate any pointers to the userland solutions for that
> >> problem (if any exist) before I resort to Xen/UML.
> 
> Jeff> UML would be exactly what you're looking for.
> 
> Jeff> 				Jeff
> 
> apart from the performance hit :-(
that's the problem...

> 
> There have been a number of different approaches proposed in the past
> to limit real memory usage per-process; search for RSS limit in the
> archives.
per-process isn't enough. I specifically need something to limit the memory
usage on a more global scale - per user ID or per process group or a similar
way of grouping related processes. That's the only way to tame processes
like apache. At this point the option I'm considering is Xen, unless I can
find a userland solution to the problem...

regards,

marek

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: user- vs kernel-level resource sandbox for Linux?

[Alan Cox]

On Maw, 2004-11-30 at 02:39, Marek Habersack wrote:
> per-process isn't enough. I specifically need something to limit the memory
> usage on a more global scale - per user ID or per process group or a similar
> way of grouping related processes. That's the only way to tame processes
> like apache. At this point the option I'm considering is Xen, unless I can
> find a userland solution to the problem...

I'd suggest playing with Xen - its very efficient and it really does
come close to perfect constraint for resources.

Re: user- vs kernel-level resource sandbox for Linux?

[Alan Cox]

On Maw, 2004-11-30 at 20:47, Marek Habersack wrote:
> That's my current impression. I also considered writing a simple kernel
> module to intercept sys_brk, but that seemed to be a bit clumsy. We have

You have to consider kernel side resources too - page tables, memory
maps
and the like which jails don't really fix. 

Re: user- vs kernel-level resource sandbox for Linux?

[Marek Habersack]

[-- Attachment #1: Type: text/plain, Size: 1266 bytes --]

On Tue, Nov 30, 2004 at 06:48:27PM +0000, Alan Cox scribbled:
> On Maw, 2004-11-30 at 02:39, Marek Habersack wrote:
> > per-process isn't enough. I specifically need something to limit the memory
> > usage on a more global scale - per user ID or per process group or a similar
> > way of grouping related processes. That's the only way to tame processes
> > like apache. At this point the option I'm considering is Xen, unless I can
> > find a userland solution to the problem...
> 
> I'd suggest playing with Xen - its very efficient and it really does
> come close to perfect constraint for resources.
That's my current impression. I also considered writing a simple kernel
module to intercept sys_brk, but that seemed to be a bit clumsy. We have
been running a test installation of Xen with 2 VMs under quite high load and
it performs outstandingly well in "laboratory environment".
Also, I seem to recall there used to be a patch for the linux kernel to implement 
BSD-like jail environment, which would suit my purpose too, do you know what happened
to the project/where it can be found? It would be a great addition to the
kernel, just like the Zones in Solaris 10 are (which are based on the BSD
jail concept as well).

regards,

marek

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: user- vs kernel-level resource sandbox for Linux?

[Herbert Poetzl]

On Tue, Nov 30, 2004 at 09:47:08PM +0100, Marek Habersack wrote:
> On Tue, Nov 30, 2004 at 06:48:27PM +0000, Alan Cox scribbled:
> > On Maw, 2004-11-30 at 02:39, Marek Habersack wrote:
> > > per-process isn't enough. I specifically need something to limit the memory
> > > usage on a more global scale - per user ID or per process group or a similar
> > > way of grouping related processes. That's the only way to tame processes
> > > like apache. At this point the option I'm considering is Xen, unless I can
> > > find a userland solution to the problem...
> > 
> > I'd suggest playing with Xen - its very efficient and it really does
> > come close to perfect constraint for resources.
> That's my current impression. I also considered writing a simple kernel
> module to intercept sys_brk, but that seemed to be a bit clumsy. We have
> been running a test installation of Xen with 2 VMs under quite high load and
> it performs outstandingly well in "laboratory environment".
> Also, I seem to recall there used to be a patch for the linux kernel to implement 
> BSD-like jail environment, which would suit my purpose too, do you know what happened
> to the project/where it can be found? It would be a great addition to the
> kernel, just like the Zones in Solaris 10 are (which are based on the BSD
> jail concept as well).

maybe this might be of interest for you:

  http://linux-vserver.org/
  http://linux-vserver.org/Linux-VServer-Paper

best,
Herbert

> regards,
> 
> marek