[dm-crypt] Integrating luksSuspend in suspend to memory

[dm-crypt] Integrating luksSuspend in suspend to memory

[Jonas Meurer]

[-- Attachment #1.1: Type: text/plain, Size: 2293 bytes --]

Hello dm-crypt list,

in the Debian cryptsetup team, we recently started to work on a solution
to luksSuspend LUKS devices before suspending a machine to memory. That
way, the storage data would be protected in suspended state and only the
data available in memory would be exposed to possible attackers.

Theoretically, that should be no big deal:

(1) Prepare a small chroot in a ramfs with all the necessary binaries,
    libraries and mounted /dev, /proc and /sys inside.
(2) luksSuspend the device(s) from the chroot
(3) immediately afterwards suspend the machine from the chroot
(4) After machine is resumed, luksResume the device(s) from the chroot.

We have a (hackish) proof of concept that works in most cases. But
unfortunately, in some cases, we seem to hit race conditions between (2)
and (3).

Our current take is the following:

To limit race conditions, we wrote a small C program[1] that merely does
the following:
(1) renice to '-20' in order to prevent race conditions
(2) run sync() in order to flush any pending fs writes
(3) run crypt_suspend() on the LUKS device
(4) write 'mem' to /sys/power/state in order to suspend the machine

Additionally, we have a shell script[2] that prepares the chroot, runs
the C program from inside the chroot and runs luksResume after the
machine has been resumed.

Unfortunately, sometimes we still discover races. It seems like in
between luksSuspend and the machine suspend, some other process wants to
perform I/O from the suspended device, and since the kernel wants to
sync(2) before suspend, the pending I/O causes the machine suspend
('mem' >/sys/power/state) to fail with:

    "Freezing of tasks failed after 20.010 seconds (1 tasks refusing to
    freeze, wq_busy=0)"

So we wonder whether there's a good solution to really lock down any
writes to the device in between luksSuspend and machine suspend in order
to phase out possible race conditions. Or do we miss anything else?

Looking forward to any pointers or hints.

Cheers
 jonas

[1]
https://salsa.debian.org/cryptsetup-team/cryptsetup/blob/suspend/debian/cryptroot-suspend/cryptroot-suspend.c
[2]
https://salsa.debian.org/cryptsetup-team/cryptsetup/blob/suspend/debian/cryptroot-suspend/cryptroot-suspend-wrapper


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [dm-crypt] Integrating luksSuspend in suspend to memory

[Jordan Glover]

On June 16, 2018 4:50 PM, Jonas Meurer <jonas@freesources.org> wrote:

> Hello dm-crypt list,
> 
> in the Debian cryptsetup team, we recently started to work on a solution
> 
> to luksSuspend LUKS devices before suspending a machine to memory. That
> 
> way, the storage data would be protected in suspended state and only the
> 
> data available in memory would be exposed to possible attackers.
> 
> Theoretically, that should be no big deal:
> 
> (1) Prepare a small chroot in a ramfs with all the necessary binaries,
> 
> libraries and mounted /dev, /proc and /sys inside.
> 
> (2) luksSuspend the device(s) from the chroot
> 
> (3) immediately afterwards suspend the machine from the chroot
> 
> (4) After machine is resumed, luksResume the device(s) from the chroot.
> 
> We have a (hackish) proof of concept that works in most cases. But
> 
> unfortunately, in some cases, we seem to hit race conditions between (2)
> 
> and (3).
> 
> Our current take is the following:
> 
> To limit race conditions, we wrote a small C program[1] that merely does
> 
> the following:
> 
> (1) renice to '-20' in order to prevent race conditions
> 
> (2) run sync() in order to flush any pending fs writes
> 
> (3) run crypt_suspend() on the LUKS device
> 
> (4) write 'mem' to /sys/power/state in order to suspend the machine
> 
> Additionally, we have a shell script[2] that prepares the chroot, runs
> 
> the C program from inside the chroot and runs luksResume after the
> 
> machine has been resumed.
> 
> Unfortunately, sometimes we still discover races. It seems like in
> 
> between luksSuspend and the machine suspend, some other process wants to
> 
> perform I/O from the suspended device, and since the kernel wants to
> 
> sync(2) before suspend, the pending I/O causes the machine suspend
> 
> ('mem' >/sys/power/state) to fail with:
> 
> "Freezing of tasks failed after 20.010 seconds (1 tasks refusing to
> 
> freeze, wq_busy=0)"
> 
> So we wonder whether there's a good solution to really lock down any
> 
> writes to the device in between luksSuspend and machine suspend in order
> 
> to phase out possible race conditions. Or do we miss anything else?
> 
> Looking forward to any pointers or hints.
> 
> Cheers
> 
> jonas
> 
> [1]
> 
> https://salsa.debian.org/cryptsetup-team/cryptsetup/blob/suspend/debian/cryptroot-suspend/cryptroot-suspend.c
> 
> [2]
> 
> https://salsa.debian.org/cryptsetup-team/cryptsetup/blob/suspend/debian/cryptroot-suspend/cryptroot-suspend-wrapper
> 

FYI: You may already know this but there are existing projects which try to handle
similar case like yours so you may want to borrow something from them:

https://github.com/zhongfu/ubuntu-luks-suspend

https://github.com/nailfarmer/debian-luks-suspend

https://github.com/guns/go-luks-suspend

https://github.com/cornelinux/yubikey-luks/blob/master/yubikey-luks-suspend

There are even reports with the same issues (unfortunately without solution):
https://github.com/guns/go-luks-suspend/issues/6#issuecomment-360902724
https://github.com/guns/go-luks-suspend/issues/9

I face it often myself so I'm interested if luks devs have any ideas.

Jordan

Re: [dm-crypt] Integrating luksSuspend in suspend to memory

[Jordan Glover]

On June 17, 2018 2:22 AM, Jordan Glover <Golden_Miller83@protonmail.ch> wrote:
> 
> I face it often myself so I'm interested if luks devs have any ideas.
> 

I also noted that it seems hardware dependent. On some machines it never occurred.
On others I can reproduce it in certain circumstances (i.e. it always hangs on first
suspend but not when I suspend normally at first, wakeup then do encrypted suspend).

Jordan