Re: encrypted swap [was: The disappearing sys_call_table export.] (fwd)

Re: encrypted swap [was: The disappearing sys_call_table export.] (fwd)

[Ahmed Masud]

On Fri, 16 May 2003, Yoav Weiss wrote:

> Hi,
>
> I got the below from some guy, off the list.
> He may has a point, at least when writable pages are shared between
> processes.
>
> What do you think ?
>
> 	Yoav
>
>  My apologies; I was unclear.
>
>  I think that you need to associate swap encryption keys with memory
> spaces, not with processes, precisely because you need to be able to
> swap out and swap in from any process using that memory space. And
> correspondingly the key can't die on a per-process basis, it has to
> die if and only if the associated memory space is torn down (which
> may be long after the PID that originally creates it goes away).

Hi Yoav,

After sort of thinking about it at this early friday hour (well late
thursday for me), it occurs to me that we may want to maintain keys
either in the vm_area_struct (vma) or for a vma group.

We want to decrypt mostlikely after a page-fault, which triggers a vma
nopage (code here?), has loaded the page so vma key, and swapping out of
course is still in vma domain.

Since we can always go from process to vma to page and back again i think
it is not going to cause any tracking issues.

Further, we have different vma's for shared and other interesting pages
so various optimizations are also doable on a case to case basis.

Does this make any sense? or am I off the cuckoo train at this hour :)

Please comment.

Cheers,

Ahmed.

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Yoav Weiss]

> Hi Yoav,
>
> After sort of thinking about it at this early friday hour (well late
> thursday for me), it occurs to me that we may want to maintain keys
> either in the vm_area_struct (vma) or for a vma group.
>
> We want to decrypt mostlikely after a page-fault, which triggers a vma
> nopage (code here?), has loaded the page so vma key, and swapping out of
> course is still in vma domain.
>
> Since we can always go from process to vma to page and back again i think
> it is not going to cause any tracking issues.

Right.  The only drawback is that we have another level of indirection for
some operations, but its negligible because it'll only be used on swap
operations or core dumps, both of which rely on disk access times anyway.

I guess a refcount and a key can be added to vm_area_struct, to provide
that.  As long as this vma is in use, key remains valid.

There's still something I'm unsure of.  I'm not familiar with the mm
subsystem.  Do you know whether vma structs are shared among processes
with shared mapping ?  I'd think the answer is yes, in which case the
above is the right way, but if it works that way, how come vm_area_struct
doesn't have a refcount yet ?  Who keeps track of it ?  Who frees it when
the last mapper unmaps it ?  Is the vma->shared in charge of all that ?
If so, what lock protects it ?

>
> Further, we have different vma's for shared and other interesting pages
> so various optimizations are also doable on a case to case basis.
>

Yes, I seems so, but we need to dig into mm and find some answers to be
sure about all cases.

> Does this make any sense? or am I off the cuckoo train at this hour :)

It does, unless I'm also off the cuckoo train at this hour.  Time to get
some sleep, I guess :)

	Yoav

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Yoav Weiss]

> There's still something I'm unsure of.  I'm not familiar with the mm
> subsystem.  Do you know whether vma structs are shared among processes
> with shared mapping ?  I'd think the answer is yes, in which case the
> above is the right way, but if it works that way, how come vm_area_struct
> doesn't have a refcount yet ?  Who keeps track of it ?  Who frees it when
> the last mapper unmaps it ?  Is the vma->shared in charge of all that ?
> If so, what lock protects it ?
>

Answering myself here.  Sanity-check me :)

According to mm/mmap.c, vma's are indeed shared among processes.
vma->shared is the list, and is maintained by __remove_shared_vm_struct()
and __vma_link_file().

The semaphore we should probably grab when messing with the key of a
shared area is actually its inode's sem, since vma doesn't have one.
inode->i_mapping->i_shared_sem, in the inode pointed by vma->vm_file.
If it doesn't exist, it probably means this vma has only one user.

Invalidating the vma key should probably occur in any place that
calls vma->vm_ops->close(), after this call.  We may want to add our own
refcount in vma, in case vm_file isn't available for extracting the
inode.

Just a thought: If we encrypt per-area, it might make more sense to go
down another level, to mm_struct.  It already has its refcount and
backpoints its users.  The key is valid iff the mm_struct is valid anyway,
so we may not have to track so many things ourselves.  Just allocate a
random key whenever mm_struct is allocated, and overwrite the key before
mm_struct is freed.  Any mm experts care to comment ?

	Yoav

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Hugh Dickins]

On Sat, 17 May 2003, Yoav Weiss wrote:
> 
> Just a thought: If we encrypt per-area, it might make more sense to go
> down another level, to mm_struct.  It already has its refcount and
> backpoints its users.  The key is valid iff the mm_struct is valid anyway,
> so we may not have to track so many things ourselves.  Just allocate a
> random key whenever mm_struct is allocated, and overwrite the key before
> mm_struct is freed.  Any mm experts care to comment ?

A page of swap may belong to different mms (see fork's copy_page_range
calling swap_duplicate).  That's a difficulty for your approaches,
and one reason why cryptoloop of block device is used instead.

Hugh

Re: The disappearing sys_call_table export.

[Yoav Weiss]

On Tue, 13 May 2003, Ahmed Masud wrote:

> This definitely sounds very interesting.  I can start looking at this
> problem seriously and see if i can put something together for 2.5.x
> since crypto subsystem routines are largely in place.

Yes, it sounds like an interesting project.  Check out openbsd's paper
about this: http://www.openbsd.org/papers/swapencrypt.ps
Let me know when you get it rolling.  I'll try to help where I can.
I just hope it has a chance to be included.

> > Another fun project is encrypted hibernation (suspend-to-disk).  Once the
> > kernel contains a stable hibernation option, I'm certainly going to
> > encrypt it.
> >
>
> Yes that too could be a fun thing to do.

Actually, I forgot that swsusp is now included.  I haven't tried it in a
while.  Anyone knows if its stable enough to start playing with encrypting
it ?

	Yoav Weiss

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Ahmed Masud]

On Wed, 14 May 2003, Yoav Weiss wrote:

> On Tue, 13 May 2003, Ahmed Masud wrote:
>
> Yes, it sounds like an interesting project.  Check out openbsd's paper
> about this: http://www.openbsd.org/papers/swapencrypt.ps

Thank you for this paper, it is a fun read. I do think however that a
few implementation differences should take place:

1. We should not enforce Rijndael as the only choice.

2. Every page should be encrypted iff it marked with some flag. This gives
a generic enough hook to create a swap_encrypt_policy type function to
determine whether it is desirable to encrypt a particular page or not.

2a. The above flag may also be set or cleared by the page-owner process on
a page-to-page basis (something a-kin to mlock()).

3. A slightly more sophisticated timeout framework should be created with
the ability to enforce expiry or request expiry extensions (upto some type
of a system hard limit?) on a per page.

Please comment.

This is an aside: should do we do anything about core dumps?

> Let me know when you get it rolling.  I'll try to help where I can.
> I just hope it has a chance to be included.

I will start looking at it seriously within next couple of days actually.
I looked at the swap stuff in mm code yesterday for the first time and it
seems (eeriely)  straightforward, and i know i am going to run into an
unseen brick wall :-).

I would suspect that somewhere between the io requst generated by
swap_readpage and swap_writepage cypto can be hooked in...  haven't yet
determined where/when the key generations should take place.

Cheers,

Ahmed Masud.

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Yoav Weiss]

On Wed, 14 May 2003, Ahmed Masud wrote:

> Thank you for this paper, it is a fun read. I do think however that a
> few implementation differences should take place:
>
> 1. We should not enforce Rijndael as the only choice.

I agree.

>
> 2. Every page should be encrypted iff it marked with some flag. This gives
> a generic enough hook to create a swap_encrypt_policy type function to
> determine whether it is desirable to encrypt a particular page or not.

Good idea.  Of course, a policy that only encrypts the secret stuff may
have an impact on security, but it makes sense to let the user decide on a
policy.

>
> 2a. The above flag may also be set or cleared by the page-owner process on
> a page-to-page basis (something a-kin to mlock()).

And just like mlock(), only root will be able to call it.

>
> 3. A slightly more sophisticated timeout framework should be created with
> the ability to enforce expiry or request expiry extensions (upto some type
> of a system hard limit?) on a per page.
>

Why is this one needed ?

> This is an aside: should do we do anything about core dumps?
>

Thats a good question.  I see three options:
1. Dump the core plaintext.  (sucks but convenient for users).
2. In the core, zero the pages that would be encrypted when swapped out.
   On some policies where only things like keys are encrypted, the core
   will be usable.  On others it won't.  (Not sure its really an option).
3. If the core contains pages that should be encrypted, dump it encrypted
   with some system-wide (or per-uid) key generated on the first core
   dump.  The key will be available to the user via some /proc interface.
   Its up to the user to be smart and take the core to another host and
   decrypt_core(1) it there (or just decrypt_core(1) it to an encrypted
   filesystem).  In any case, the decrypted core or the system-wide key
   are never written to disk.
4. Refuse to dump core of a process that has pages that should be
   encrypted.

Do you see more options ?
Anyway, it should probably be policy controlled.

> I looked at the swap stuff in mm code yesterday for the first time and it
> seems (eeriely)  straightforward, and i know i am going to run into an
> unseen brick wall :-).

I'm not familiar with this core either and somehow got the same feeling
when I looked into it :)

>
> I would suspect that somewhere between the io requst generated by
> swap_readpage and swap_writepage cypto can be hooked in...  haven't yet
> determined where/when the key generations should take place.

Probably at process initialization during fork.  The key must be ready
before the process gets its first chance to allocated pages that may be
swapped out.

>
> Cheers,
>
> Ahmed Masud.
>
>

Bye,
	Yoav Weiss

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Ahmed Masud]

On Wed, 14 May 2003, Yoav Weiss wrote:

> On Wed, 14 May 2003, Ahmed Masud wrote:
>
> >
> > 3. A slightly more sophisticated timeout framework should be created with
> > the ability to enforce expiry or request expiry extensions (upto some type
> > of a system hard limit?) on a per page.
> >
>
> Why is this one needed ?
>


Well we definitely need a way to timeout keys.  The other reason is to be
able to "change your mind" about it while the key is being used.  This may
not be a useful thing for now but think of encrypted swaps on the
infamous: oopsies-i-tripped-over-a-wire-and-disconnected-network-file-system

Here we have a situation where we want to not have an expired key with
valid data hanging out there.

Or are we saying that expiration only affects encryption and that the
decryption counterpart sticks around until its reference count goes to
zero? On the surface this seems to be easier, although not sure if it
makes us miss any situation.

Cheers,

Ahmed.

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Yoav Weiss]

On Wed, 14 May 2003, Ahmed Masud wrote:

> Well we definitely need a way to timeout keys.  The other reason is to be

Why ?  We keep a key per-process, and wipe it from memory as soon as the
process dies.  Its not time-related.

> able to "change your mind" about it while the key is being used.  This may
> not be a useful thing for now but think of encrypted swaps on the
> infamous: oopsies-i-tripped-over-a-wire-and-disconnected-network-file-system

Your swapfile is coming from a remote NFS mount ?  If your swap becomes
unavailable due to network problems, the swapped-out processes are
probably doomed anyway.

>
> Here we have a situation where we want to not have an expired key with
> valid data hanging out there.

Data is valid iff the owning process still lives.  When do we need to
expire a key while its process is still alive (or keep a key valid for
pages of a dead process) ?

	Yoav

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Ahmed Masud]

On Wed, 14 May 2003, Yoav Weiss wrote:

>
> Thats a good question.  I see three options:
> 1. Dump the core plaintext.  (sucks but convenient for users).
> 2. In the core, zero the pages that would be encrypted when swapped out.
>    On some policies where only things like keys are encrypted, the core
>    will be usable.  On others it won't.  (Not sure its really an option).
> 3. If the core contains pages that should be encrypted, dump it encrypted
>    with some system-wide (or per-uid) key generated on the first core
>    dump.  The key will be available to the user via some /proc interface.
>    Its up to the user to be smart and take the core to another host and
>    decrypt_core(1) it there (or just decrypt_core(1) it to an encrypted
>    filesystem).  In any case, the decrypted core or the system-wide key
>    are never written to disk.
> 4. Refuse to dump core of a process that has pages that should be
>    encrypted.
>
> Do you see more options ?
> Anyway, it should probably be policy controlled.

These are all very good options, ofcourse things get hairy don't they :)
Perhaps in the beginning either 1, 2 and 4 as per a system wide dump
policy. May be even a setrlimit extension and use that as a jump point to
make a per user policy?

Cheers,

Ahmed.

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Yoav Weiss]

> > Do you see more options ?
> > Anyway, it should probably be policy controlled.
>
> These are all very good options, ofcourse things get hairy don't they :)

Certainly.  Option 3 certainly doesn't have to be implemented in the first
version :)
In fact, the first version could ignore the core dump issue and setrlimit
will be used to avoid core dumps of sensitive processes.  In the future,
it can be handled more gracefully.

> Perhaps in the beginning either 1, 2 and 4 as per a system wide dump
> policy. May be even a setrlimit extension and use that as a jump point to
> make a per user policy?

Makes sense.  Only when 3 is implemented, a special /proc interface is
required.  For everything else, setrlimit will suffice.

>
> Cheers,
>
> Ahmed.
>

Bye,
	Yoav

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Jörn Engel]

On Wed, 14 May 2003 06:06:56 -0400, Ahmed Masud wrote:
> On Wed, 14 May 2003, Yoav Weiss wrote:
> > On Tue, 13 May 2003, Ahmed Masud wrote:
> >
> > Yes, it sounds like an interesting project.  Check out openbsd's paper
> > about this: http://www.openbsd.org/papers/swapencrypt.ps
> 
> Thank you for this paper, it is a fun read. I do think however that a
> few implementation differences should take place:
> 
> 1. We should not enforce Rijndael as the only choice.
> 
> 2. Every page should be encrypted iff it marked with some flag. This gives
> a generic enough hook to create a swap_encrypt_policy type function to
> determine whether it is desirable to encrypt a particular page or not.
>
> [...]

Just browsed across the white paper, but this doesn't make a lot of
sense to me.

1. Instead of cryptographic filesystems, you could just encrypt the
   block device.
2. The only reason not to do so it security.  An attacker could use
   known-plaintext attacks, since some parts of the metadata can be
   reconstructed or guessed easily.
3. Instead of encrypted swap, you could just encrypt the block device.
4. The only reason reason not to do so is what?

Sorry, beats me, I cannot see any reason. Is there a possible
known-plaintext attack possible, that is not obvious to everyone, at
least not to me?

Jörn

-- 
A defeated army first battles and then seeks victory.
-- Sun Tzu

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Ahmed Masud]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=X-UNKNOWN, Size: 757 bytes --]



On Wed, 14 May 2003, [iso-8859-1] Jörn Engel wrote:
> sense to me.
>
> 1. Instead of cryptographic filesystems, you could just encrypt the
>    block device.
> 2. The only reason not to do so it security.  An attacker could use
>    known-plaintext attacks, since some parts of the metadata can be
>    reconstructed or guessed easily.
> 3. Instead of encrypted swap, you could just encrypt the block device.
> 4. The only reason reason not to do so is what?
>

The idea is to have encryption keys for the pages to be unique on a
per-uid per-process basis. So one user on the system cannot access (even
if they are root) parts of another's private data.  To achieve this,
different parts of swap device need to be encrypted with different keys.

Ahmed.

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Jörn Engel]

On Wed, 14 May 2003 12:13:03 -0400, Ahmed Masud wrote:
> 
> The idea is to have encryption keys for the pages to be unique on a
> per-uid per-process basis. So one user on the system cannot access (even
> if they are root) parts of another's private data.  To achieve this,
> different parts of swap device need to be encrypted with different keys.

How do user *know* that root cannot simply bypass this security?

Root, god, what's the difference? ;-)

Jörn

-- 
"Error protection by error detection and correction."
-- from a university class

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Ahmed Masud]

Jörn Engel wrote:
> On Wed, 14 May 2003 12:13:03 -0400, Ahmed Masud wrote:
> 
> 
> How do user *know* that root cannot simply bypass this security?
> 
> Root, god, what's the difference? ;-)

Hahah so true so true ... but all gods fall if their worshipers stop 
worshiping them ;-)

So ... there are ways. Root is only given this power because it is 
allowed that by the operating system kernel. It is always possible to 
shunt root out if the kernel chooses to do so.   We can actually 
construct environments where security officers != system administrators.


Ahmed.

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Ahmed Masud]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: TEXT/PLAIN; charset=X-UNKNOWN, Size: 855 bytes --]



On Wed, 14 May 2003, [iso-8859-1] Jörn Engel wrote:

> On Wed, 14 May 2003 12:13:03 -0400, Ahmed Masud wrote:
> >
> > The idea is to have encryption keys for the pages to be unique on a
> > per-uid per-process basis. So one user on the system cannot access (even
> > if they are root) parts of another's private data.  To achieve this,
> > different parts of swap device need to be encrypted with different keys.
>
> How do user *know* that root cannot simply bypass this security?
>
> Root, god, what's the difference? ;-)
>
> Jörn

Well :-) that's sorta true. In the new world the old gods will fall to
give rise to new ones.  worshippers of root will fade in the echos of the
past ... Rootshunting is possible if the kernel so chooses. Trusted Linux,
which is my perosnal and favourite focus for linux would be an environment
without root.


Ahmed.

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Yoav Weiss]

On Wed, 14 May 2003, Jörn Engel wrote:

> On Wed, 14 May 2003 12:13:03 -0400, Ahmed Masud wrote:
> >
> > The idea is to have encryption keys for the pages to be unique on a
> > per-uid per-process basis. So one user on the system cannot access (even
> > if they are root) parts of another's private data.  To achieve this,
> > different parts of swap device need to be encrypted with different keys.
>
> How do user *know* that root cannot simply bypass this security?
>
> Root, god, what's the difference? ;-)

Aside from what Ahmed said about about rootless systems, the per-process
encryption reduces the window of opportunity for attackers who gain root
(or physical access).

Try strings(1) on your swap device.  You'll be surprised at what you find.
You'll probably recognize passwords you haven't useds for a long time, and
a lot of other stuff you didn't expect.  Sometimes you can find whole ssh
sessions there, plaintext.  (think xterm scroll buffer).

With per-process encryption, even if root decides to read the swap at some
point (evil admin or an attacker who 0wn3d the box), the leakage is
limited to processes currently running.

	Yoav

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Jörn Engel]

On Wed, 14 May 2003 21:59:47 +0300, Yoav Weiss wrote:
> On Wed, 14 May 2003, Jörn Engel wrote:
> > On Wed, 14 May 2003 12:13:03 -0400, Ahmed Masud wrote:
> > >
> > > The idea is to have encryption keys for the pages to be unique on a
> > > per-uid per-process basis. So one user on the system cannot access (even
> > > if they are root) parts of another's private data.  To achieve this,
> > > different parts of swap device need to be encrypted with different keys.
> >
> > How do user *know* that root cannot simply bypass this security?
> >
> > Root, god, what's the difference? ;-)
> 
> Aside from what Ahmed said about about rootless systems, the per-process
> encryption reduces the window of opportunity for attackers who gain root
> (or physical access).
> 
> Try strings(1) on your swap device.  You'll be surprised at what you find.
> You'll probably recognize passwords you haven't useds for a long time, and
> a lot of other stuff you didn't expect.  Sometimes you can find whole ssh
> sessions there, plaintext.  (think xterm scroll buffer).
> 
> With per-process encryption, even if root decides to read the swap at some
> point (evil admin or an attacker who 0wn3d the box), the leakage is
> limited to processes currently running.

s/currently running/running now or in the future/

But apart from that, it does really reduce the window, agreed.

An alternative approach would simply zero all freed memory in the
system, with almost identical effects. Almost means you are missing
memory (that isn't cleared on reboot on all systems, ...) and this is
missing hard disk recovery that can read data already overwritten.

Arguments against this simpler approach?

Jörn

-- 
Rules of Optimization:
Rule 1: Don't do it.
Rule 2 (for experts only): Don't do it yet.
-- M.A. Jackson 

Re: encrypted swap [was: The disappearing sys_call_table export.]

[Yoav Weiss]

> s/currently running/running now or in the future/

Hopefully near future only, assuming you have a proper log/alert system
that sends logs to another machine.
(and if you don't and your box stays owned, you have bigger problems
than the swap sniffing).

>
> But apart from that, it does really reduce the window, agreed.
>
> An alternative approach would simply zero all freed memory in the
> system, with almost identical effects. Almost means you are missing
> memory (that isn't cleared on reboot on all systems, ...) and this is
> missing hard disk recovery that can read data already overwritten.
>
> Arguments against this simpler approach?

The performance impact for one.  My systems often has processes taking
hundreds of megabytes in swap.  If we'd have to wipe all this space on the
disk whenever such process dies, the system would be unusable.

Second, see previous posts on this thread re hardware issues when writing
zero blocks to some disks.  In short, you're never sure its really clean.

Third, in case of crash (i.e. when someone pulls the plug and steals the
server), the system had no chance to clean the swap.

Encrypted swap solves all that.

>
> Jörn
>
> --
> Rules of Optimization:
> Rule 1: Don't do it.
> Rule 2 (for experts only): Don't do it yet.
> -- M.A. Jackson
>