Special Kernel Modification

Special Kernel Modification

[Lonnie Cumberland]

Hello All,

I am new to this list and am very interested in any help that you might 
be able to give me for a particular Linux project that I am working on.

I am currently using a Pentium III 500Mhz running Mandrake 8.0 Linux and 
Kernel 2.4.3 as a development system.

The basic problem that I am running into is that I am working on an 
Internet related project and thus need to ensure various types of 
document security for the eventual users of this system, if things go well.

I have look into using things like "chroot" to restrict the users for 
this very special server, but that solution is not what we need.

I am building a special xserver that will allow users to login and then 
the xserver will run a single application such as StarOffice. When the 
user exits from the application then the Xserver will log them out.

My problem is that I need to find a way to prevent the user from 
navigating out of their home directories.

I have also looking the possiblility of writing my own filesystem, but I 
am told that this needs to be done at the VFS level.

Is there someone who might be able to give me some information on how I 
could add a few lines to the VFS filesystem so that I might set some 
type of extended attribute to prevent users from navigating out of the 
locations.

Any help would be greatly appreciated,
Lonnie Cumberland

Re: Special Kernel Modification

[Ryan Cumming]

On November 4, 2001 16:01, Lonnie Cumberland wrote:
> I have look into using things like "chroot" to restrict the users for
> this very special server, but that solution is not what we need.
....
> Is there someone who might be able to give me some information on how I
> could add a few lines to the VFS filesystem so that I might set some
> type of extended attribute to prevent users from navigating out of the
> locations.

I fail to see the difference between "chroot" and "preventing users from 
navigating out of locations". Would you care to clarify what was wrong was 
chroot that you believe you can solve with a different approach?
-Ryan

Re: Special Kernel Modification

[Alan Cox]

> I have look into using things like "chroot" to restrict the users for 
> this very special server, but that solution is not what we need.

It sounds like it is to me

> My problem is that I need to find a way to prevent the user from 
> navigating out of their home directories.

Then you must put al the files in their home directories. Alternatively
with later 2.4 you can use bind mounts to remount the application file
systems below the user.

> Is there someone who might be able to give me some information on how I 
> could add a few lines to the VFS filesystem so that I might set some 
> type of extended attribute to prevent users from navigating out of the 
> locations.

It isnt down to attributes - how you can run a binary or load a shared
library you cant see.

You might also want to see http://www.nsa.gov/selinux, but that would
require a lot of careful policy setup

Re: Special Kernel Modification

[lonnie]

Hello Ryan,

>From what I can see. With chrooting, I have to make a complete "fake" system an
then place the users below that into a home directory, or make a complete "fake"
system for each user.

I was trying to find a simple solution that would allow for:

I was initially thinking about something like this for each user:

/system (real) /dev/hda4 (chrooted also)
      |
      /bin
      /etc
      /lib



/home (each user chrooted)
     |
     /user1
     |     |
     |     /system (mounted /dev/hda4)
     |            |
     |            /bin
     |            /etc
     |            /lib
     |
     /user2
     |     |
     |     /system (mounted /dev/hda4)
     |            |
     |            /bin
     |            /etc
     |            /lib
     |
     /user n
          |
          /system (mounted /dev/hda4)
                 |
                 /bin
                 /etc
                 /lib

The basic problem is that I did not want, for example "user2" to be able to "cd
.." or some thing to go out of user2

I was hoping to be able to accomplish this at the filesystem level somehow, and
possibly without the need to mount the /dev/hda4 onto each /home/user/system, or
without having to make entire copies of the chrooted environment for each user.

Cheers,
Lonnie

Quoting Ryan Cumming <bodnar42@phalynx.dhs.org>:

> On November 4, 2001 16:01, Lonnie Cumberland wrote:
> > I have look into using things like "chroot" to restrict the users
> for
> > this very special server, but that solution is not what we need.
> ....
> > Is there someone who might be able to give me some information on how
> I
> > could add a few lines to the VFS filesystem so that I might set some
> > type of extended attribute to prevent users from navigating out of
> the
> > locations.
>
> I fail to see the difference between "chroot" and "preventing users from
>
> navigating out of locations". Would you care to clarify what was wrong
> was
> chroot that you believe you can solve with a different approach?
> -Ryan
>

Re: Special Kernel Modification

[Rik van Riel]

On Sun, 4 Nov 2001, Lonnie Cumberland wrote:

> I have look into using things like "chroot" to restrict the users for
> this very special server, but that solution is not what we need.

> My problem is that I need to find a way to prevent the user from
> navigating out of their home directories.

chroot() will do that pretty well, but if you want more
you can always take a look at vserver:

http://www.solucorp.qc.ca/miscprj/s_context.hc

Not as sophisticated as NSA's selinux, but that also
means it's much easier to get security because it's
just easier to setup ;)

regards,

Rik
-- 
DMCA, SSSCA, W3C?  Who cares?  http://thefreeworld.net/

http://www.surriel.com/		http://distro.conectiva.com/

Re: Special Kernel Modification

[Phil Sorber]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


can you point me to a site so i can learn more about that binding of mounts? i recently heard something about that in plan 9, and it sounds very interesting. union mounting they call it also, correct?

On Mon, 5 Nov 2001 00:22:27 +0000 (GMT), Alan Cox wrote:
> > I have look into using things like "chroot" to restrict the users for 
> > this very special server, but that solution is not what we need.
> 
> It sounds like it is to me
> 
> > My problem is that I need to find a way to prevent the user from 
> > navigating out of their home directories.
> 
> Then you must put al the files in their home directories. Alternatively
> with later 2.4 you can use bind mounts to remount the application file
> systems below the user.
> 
> > Is there someone who might be able to give me some information on how I 
> > could add a few lines to the VFS filesystem so that I might set some 
> > type of extended attribute to prevent users from navigating out of the 
> > locations.
> 
> It isnt down to attributes - how you can run a binary or load a shared
> library you cant see.
> 
> You might also want to see http://www.nsa.gov/selinux, but that would
> require a lot of careful policy setup
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 
> 

- --
Phil Sorber
AIM: PSUdaemon
IRC: irc.openprojects.net #psulug PSUdaemon
GnuPG: keyserver - pgp.mit.edu
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE75d+9Xm6Gwek+iaQRAgnVAKCBpdA6EzpXoT/SIffK4yuPviHENgCggVq6
+6Dn7AzjrsT+S7GavhNudSI=
=eEGW
-----END PGP SIGNATURE-----

Re: Special Kernel Modification

[Jan-Benedict Glaw]

On Sun, 2001-11-04 19:29:01 -0500, lonnie@outstep.com <lonnie@outstep.com>
wrote in message <1004920141.3be5dd4db68a0@mail.outstep.com>:
> The basic problem is that I did not want, for example "user2" to be able to "cd
> .." or some thing to go out of user2
> 
> I was hoping to be able to accomplish this at the filesystem level somehow, and
> possibly without the need to mount the /dev/hda4 onto each /home/user/system, or
> without having to make entire copies of the chrooted environment for each user.

Isn't it the simplest of all to set all the user's file and directory
modes to sth likt 0700? Then, every user may only access his/her
files. (S)he cannot look at other user's files.

MfG, JBG

-- 
Jan-Benedict Glaw . jbglaw@lug-owl.de . +49-172-7608481

Re: Special Kernel Modification

[Jeremy Jackson]

Um, what about chmod command?  You don't want people to be able to see
directories like /bin?  You would have those anyway in a chroot environment.
Use chmod to turn off read and execute(search) permissions to anything they
don't
need to run (ie /bin, /usr/bin, /lib, etc).  Sorry if that sounds dumb, but
there's not much information to go on.

Also, for a special xserver, all you need to make it run staroffice or
something else, is change the
shell script that starts the x server and/or the user's session.  Xdm has a
bunch of options for that.

There are also kernel "capabilities"...

----- Original Message -----
From: Lonnie Cumberland <lonnie@outstep.com>
To: <linux-kernel@vger.kernel.org>
Sent: Sunday, November 04, 2001 4:01 PM
Subject: Special Kernel Modification


> The basic problem that I am running into is that I am working on an
> Internet related project and thus need to ensure various types of
> document security for the eventual users of this system, if things go
well.
>
> I have look into using things like "chroot" to restrict the users for
> this very special server, but that solution is not what we need.
>
> I am building a special xserver that will allow users to login and then
> the xserver will run a single application such as StarOffice. When the
> user exits from the application then the Xserver will log them out.
>
> My problem is that I need to find a way to prevent the user from
> navigating out of their home directories.
>
> I have also looking the possiblility of writing my own filesystem, but I
> am told that this needs to be done at the VFS level.
>
> Is there someone who might be able to give me some information on how I
> could add a few lines to the VFS filesystem so that I might set some
> type of extended attribute to prevent users from navigating out of the
> locations.
>
> Any help would be greatly appreciated,
> Lonnie Cumberland
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: Special Kernel Modification

[Jeff Dike]

lonnie@outstep.com said:
> My problem is that I need to find a way to prevent the user from
> navigating out of their home directories. 

A virtual machine would be an administratively easy way of doing this.

Let the 'app' be a VM with the real apps installed inside.  The users would
effectively be confined to a *file* on the host, not merely their home
directories.

My (biased :-) recommendation would be User-mode Linux 
(http://user-mode-linux.sourceforge.net), but VMWare would work as
well.

				Jeff

Re: Special Kernel Modification

[Ryan Cumming]

Hey Jeff,
> A virtual machine would be an administratively easy way of doing this.
>
> Let the 'app' be a VM with the real apps installed inside.  The users would
> effectively be confined to a *file* on the host, not merely their home
> directories.
>
> My (biased :-) recommendation would be User-mode Linux
> (http://user-mode-linux.sourceforge.net), but VMWare would work as
> well.

I'm afraid that would scale horribly on anything greater than a very modest 
load:

1) There is a fairly significant overhead due to running inside a VM, which 
would certainly pile up over multiple invocations. Besides the overhead of 
the VM, you have the have the additional overhead caused by such things as 
having multiple running kernels doing exactly the same thing, all running 
seperate copies of the required system daemons.
2) Upgrading/changing the system configuration would require modifying -all- 
of the VMs. This could be partially alleviated by using an NFS root, but then 
the overhead mentioned above becomes even more looming
3) Implicitly shared memory no longer works too well, and this is one of the 
reasons Unix is so strong as a terminal server:
  You have thirty people running StarOffice from /usr/bin, and because        
they've all mmap()ed the same binary, the read only portions are all shared,  
and will definitely stay in memory in the event of memory pressure. The same 
things goes for all the libraries and mmap()'ed data files StarOffice uses, 
including some pretty hefty ones such as glibc. Even many of the writable 
pages will be shared until someone decides to write to one of them, and then 
the writer will get a private copy.
  Now, try 30 people all running from VMs. They all must have a seperate copy 
of the otherwise sharable portions of StarOffice, and all of its 
dependancies. Memory pressure quickly rises, and it will start swapping out 
one copy of StarOffice in favour of an identical portion of another copy (!). 
System quickly becomes unusable. 

So, to summarize, you could use a VM, and it'd work well for containment, but 
it'd be pretty useless as far as performance, ease of administration, and 
scaling go. Comments?

-Ryan

Re: Special Kernel Modification

[Mike Fedyk]

On Sun, Nov 04, 2001 at 07:29:01PM -0500, lonnie@outstep.com wrote:
> Hello Ryan,
> 
> >From what I can see. With chrooting, I have to make a complete "fake" system an
> then place the users below that into a home directory, or make a complete "fake"
> system for each user.

> The basic problem is that I did not want, for example "user2" to be able to "cd
> .." or some thing to go out of user2
> 
> I was hoping to be able to accomplish this at the filesystem level somehow, and
> possibly without the need to mount the /dev/hda4 onto each /home/user/system, or
> without having to make entire copies of the chrooted environment for each user.
> 

Chroot will allow you to keel a user within a certain directory tree, and as
long as you use hard links on the same FS, you won't waste much space for
each chroot...

Also, why don't you want the users to be able to see the executable
directories?  They're only writable by root, right?

If you set each user's home directory to mode 0700 no other user will be
able to cd into that dir...

The real question is why do you want to split each user so much?

Mike

Re: Special Kernel Modification

[Ryan Cumming]

Jeff, 
>  Mounting it synchronous will  disable caching in the VM. 
Who told you that? Synchronous mounting turns off write buffering. Even with 
"-o sync" writes will still end up in the page cache, they'll just be 
commited immediately. In case you don't believe me, here's a trivial test on 
a block device mounted -o sync (ext3):

bodnar42:/mnt# time cat linux-2.4.13.tar.bz2 > /dev/null

real    0m2.837s
user    0m0.040s
sys     0m0.350s
bodnar42:/mnt# time cat linux-2.4.13.tar.bz2 > /dev/null

real    0m0.328s
user    0m0.010s
sys     0m0.240s
bodnar42:/mnt#

Trust me, the factor of 8 performace improvement the second time is not due 
to lucky head positioning or anything, that's coming straight out of cache.

> Another possibility is hostfs.  You can directly mount a host directory
> inside UML.  That can obviously be shared between UMLs, so you again
> eliminate all the duplication.
Er, it will be shared in the host's context, but each VM will still have 
their own copy in the page cache. This is no better than a COW'ed block device

-Ryan

Re: Special Kernel Modification

[Jeff Dike]

bodnar42@phalynx.dhs.org said:
> So, to summarize, you could use a VM, and it'd work well for
> containment, but  it'd be pretty useless as far as performance, ease
> of administration, and  scaling go. 

Yeah, that's mostly wrong.

> Comments?

They follow...

> 1) There is a fairly significant overhead due to running inside a VM,
> which  would certainly pile up over multiple invocations. Besides the
> overhead of  the VM, you have the have the additional overhead caused
> by such things as  having multiple running kernels doing exactly the
> same thing, all running  seperate copies of the required system
> daemons.

This can be minimized.  What the original poster asked for is a one-application
jail.  So, you boot up a filesystem that runs only that app, perhaps even
as the init process.  No system daemons, nothing but the app and whatever it 
runs.

> Now, try 30 people all running from VMs. They all must have a seperate
> copy  of the otherwise sharable portions of StarOffice, and all of its
>  dependancies. Memory pressure quickly rises, and it will start
> swapping out  one copy of StarOffice in favour of an identical portion
> of another copy (!).  System quickly becomes unusable. 

Nope.  Duplicate block caching can be eliminated by making the root filesystem
a COW device (http://user-mode-linux.sourceforge.net/shared_fs.html) and 
mounting it sychronously.  COWing it will allow it to be shared among all VMs, 
with any changes being stored in the private COW file.  Mounting it synchronous
will disable caching in the VM.  So, for a number of virtual machines, there
will be, in total, slightly more than one copy of their data in the host's
memory.

Another possibility is hostfs.  You can directly mount a host directory
inside UML.  That can obviously be shared between UMLs, so you again 
eliminate all the duplication.

> 2) Upgrading/changing the system configuration would require modifying
> -all-  of the VMs. This could be partially alleviated by using an NFS
> root, but then  the overhead mentioned above becomes even more looming

This is a problem for a COWed block device, but it's trivial for a shared
hostfs directory.  Just dump the new bits in.

				Jeff

Re: Special Kernel Modification

[Ryan Cumming]

On November 4, 2001 21:52, you wrote:
> bodnar42@phalynx.dhs.org said:
> > >  Mounting it synchronous will  disable caching in the VM.
> >
> > Who told
> > you that? Synchronous mounting turns off write buffering. Even with
> > "-o sync" writes will still end up in the page cache, they'll just be
> > commited immediately.
>
> Ummm, how about O_DIRECT instead of O_SYNC (or maybe as well, my googling
> hasn't been clear on whether O_DIRECT bypasses the cache on writes as
> well)?
>

Nope, last I checked O_DIRECT enforces buffer and file offset alignment. 
Normal apps wouldn't work very well at all. Maybe UML needs some hacks around 
the whole caching issue?

-Ryan

Re: Special Kernel Modification

[Jeff Dike]

bodnar42@phalynx.dhs.org said:
> >  Mounting it synchronous will  disable caching in the VM.  
> Who told
> you that? Synchronous mounting turns off write buffering. Even with
> "-o sync" writes will still end up in the page cache, they'll just be
> commited immediately.

Ummm, how about O_DIRECT instead of O_SYNC (or maybe as well, my googling
hasn't been clear on whether O_DIRECT bypasses the cache on writes as well)?

All IO bypasses the virtual machine cache, so there's no multiple caching.

> Er, it will be shared in the host's context, but each VM will still
> have  their own copy in the page cache. This is no better than a
> COW'ed block device 

Nope.  You O_DIRECT here as well.

				Jeff

Re: Special Kernel Modification

[Jeff Dike]

bodnar42@phalynx.dhs.org said:
> Nope, last I checked O_DIRECT enforces buffer and file offset
> alignment.  Normal apps wouldn't work very well at all. 

Except that the apps won't be the ones doing O_DIRECT IO.  It'll be UML, and
it can presumably provide whatever alignment is required.

> Maybe UML needs some hacks around  the whole caching issue?

Not that I can see...

				Jeff

Re: Special Kernel Modification

[Andrea Arcangeli]

On Mon, Nov 05, 2001 at 12:52:51AM -0500, Jeff Dike wrote:
> bodnar42@phalynx.dhs.org said:
> > >  Mounting it synchronous will  disable caching in the VM.  
> > Who told
> > you that? Synchronous mounting turns off write buffering. Even with
> > "-o sync" writes will still end up in the page cache, they'll just be
> > commited immediately.
> 
> Ummm, how about O_DIRECT instead of O_SYNC (or maybe as well, my googling
> hasn't been clear on whether O_DIRECT bypasses the cache on writes as well)?

O_DIRECT is synchronous but only in terms of data, if you want the
metadata to be synchronous as well you need to open with
O_SYNC|O_DIRECT, and even in such case all the metadata reads will came
from cache.

For example if you only care about being able to reach the data after a
crash (not about the inode info) in a file with all its logical blocks
mapped to physical blcoks (no holes) and then you fsync, later you can
as well avoid O_SYNC and you still don't risk to lose data after a crash
because the block mappings never changes, if you grow/shrink the file
you definitely need O_SYNC to be sure the O_DIRECT data is still there
after a crash instead.

Andrea

Re: Special Kernel Modification

[Andrea Arcangeli]

On Mon, Nov 05, 2001 at 03:18:42PM -0500, Jeff Dike wrote:
> eliminating caching as much as possible.  If the metadata/data ratio is
> small, then the metadata caching probably won't be noticable.

yes, of course the metadata/data ratio is very small, O_DIRECT isn't
slower than rawio infact (assuming the file isn't fragmented).

Andrea

Re: Special Kernel Modification

[Jeff Dike]

andrea@suse.de said:
> O_DIRECT is synchronous but only in terms of data, if you want the
> metadata to be synchronous as well you need to open with
> O_SYNC|O_DIRECT, and even in such case all the metadata reads will
> came from cache. 

That's OK.  The goal is to cram as many UMLs onto a host as possible by
eliminating caching as much as possible.  If the metadata/data ratio is
small, then the metadata caching probably won't be noticable.

				Jeff

Re: Special Kernel Modification

[Jorgen Cederlof]

On Sun, Nov 04, 2001 at 19:29:01 -0500, lonnie@outstep.com wrote:

> From what I can see. With chrooting, I have to make a complete
> "fake" system an then place the users below that into a home
> directory, or make a complete "fake" system for each user.
> 
> I was trying to find a simple solution that would allow for:
> 
> I was initially thinking about something like this for each user:
> 
> /system (real) /dev/hda4 (chrooted also)
>       |
>       /bin
>       /etc
>       /lib

chtrunk (http://noid.sf.net/chtrunk.html) can set up the namespace
dynamically for you. Instead of creating a complete system by hand and
run chroot, just run (you don't need to be root):

   chtrunk -s /bin /etc /lib /home/user -c program_to_run

This will give that program access to /bin, /etc, /lib and the home
directory, but nothing more.

You can use

   chtrunk -s /bin /etc /lib /home/user /tmp=/home/user/tmp -c program

to give every user their own private /tmp.

As a bonus, the suid/sgid bits will have no effect for these users,
which will prevent them from becoming root through buggy suid
programs.

    Jörgen

Re: Special Kernel Modification

[lonnie]

Thanks Jorgen,

I am sure that this will help as it looks like what I might need....

Thanks again,
Lonnie

Quoting Jorgen Cederlof <jc@lysator.liu.se>:

> 
> On Sun, Nov 04, 2001 at 19:29:01 -0500, lonnie@outstep.com wrote:
> 
> > From what I can see. With chrooting, I have to make a complete
> > "fake" system an then place the users below that into a home
> > directory, or make a complete "fake" system for each user.
> > 
> > I was trying to find a simple solution that would allow for:
> > 
> > I was initially thinking about something like this for each user:
> > 
> > /system (real) /dev/hda4 (chrooted also)
> >       |
> >       /bin
> >       /etc
> >       /lib
> 
> chtrunk (http://noid.sf.net/chtrunk.html) can set up the namespace
> dynamically for you. Instead of creating a complete system by hand and
> run chroot, just run (you don't need to be root):
> 
>    chtrunk -s /bin /etc /lib /home/user -c program_to_run
> 
> This will give that program access to /bin, /etc, /lib and the home
> directory, but nothing more.
> 
> You can use
> 
>    chtrunk -s /bin /etc /lib /home/user /tmp=/home/user/tmp -c program
> 
> to give every user their own private /tmp.
> 
> As a bonus, the suid/sgid bits will have no effect for these users,
> which will prevent them from becoming root through buggy suid
> programs.
> 
>     Jörgen
> 

Re: Special Kernel Modification

[Andi Kleen]

Hi Jeff,

Jeff Dike <jdike@karaya.com> writes:

> Ummm, how about O_DIRECT instead of O_SYNC (or maybe as well, my googling
> hasn't been clear on whether O_DIRECT bypasses the cache on writes as well)?

It does, but it has been deimplemented in the Linus tree and never put into
the -ac tree. You would need -aa for working O_DIRECT or use a raw device.

Also they both have some restrictions on buffer alignment, but these would be 
already fulfilled if UML internally writes through the page cache (which 
it does of course)

-Andi

Re: Special Kernel Modification

[John Weber]

There is a Linux implementation of private namespaces (based on Plan 9,
I believe).
Don't know if this is really what you need...

Check out issue #331 (latest issue) of Dr. Dobb's Journal...