fscrypt: in-place decrypt vs. out-of-place encrypt?

fscrypt: in-place decrypt vs. out-of-place encrypt?

[Richard Weinberger]

Hi!

While reading the fscrypt code I noticed that some functions use the bounce pages
and some not.
fscrypt_decrypt_page() and fscrypt_decrypt_bio_pages() work in-place while
fscrypt_encrypt_page() and fscrypt_zeroout_range() use a bounce page.

So, both ext4 and f2fs encrypt data using an extra buffer but decrypt mostly
in-place without the need of an extra buffer.
Why that? I'd assume when decryption can be done in-place also encryption is possible
that way.

I'm working on fscrypt for UBIFS and would like to avoid an extra buffer since memory
is low on embedded systems.

Thanks,
//richard

fscrypt: in-place decrypt vs. out-of-place encrypt?

[Richard Weinberger]

Hi!

While reading the fscrypt code I noticed that some functions use the bounce pages
and some not.
fscrypt_decrypt_page() and fscrypt_decrypt_bio_pages() work in-place while
fscrypt_encrypt_page() and fscrypt_zeroout_range() use a bounce page.

So, both ext4 and f2fs encrypt data using an extra buffer but decrypt mostly
in-place without the need of an extra buffer.
Why that? I'd assume when decryption can be done in-place also encryption is possible
that way.

I'm working on fscrypt for UBIFS and would like to avoid an extra buffer since memory
is low on embedded systems.

Thanks,
//richard

------------------------------------------------------------------------------

Re: fscrypt: in-place decrypt vs. out-of-place encrypt?

[Theodore Ts'o]

On Thu, Sep 22, 2016 at 10:49:41AM +0200, Richard Weinberger wrote:
> 
> While reading the fscrypt code I noticed that some functions use the bounce pages
> and some not.
> fscrypt_decrypt_page() and fscrypt_decrypt_bio_pages() work in-place while
> fscrypt_encrypt_page() and fscrypt_zeroout_range() use a bounce page.
> 
> So, both ext4 and f2fs encrypt data using an extra buffer but decrypt mostly
> in-place without the need of an extra buffer.
> Why that? I'd assume when decryption can be done in-place also encryption is possible
> that way.

The problem is that the page may be mapped into some process's (or
multiple processes') address space.  The only way we can encrypt in
place is if we make sure that it is not mapped into any processes'
page tables, and prohibit it from being mmapped, *and* prevent any
simultaneous reads from accessing the page for which we are in the
process of destroying its original contents because we are doing an
in-plance encrypt.

So it could be done, but you would have to remove the page from the
page cache first (meaning making sure it's not in any page tables) so
it can't be found by subsequent reads and mmaps, and then sure you
have a lock out which prevents anyone else from trying to read or mmap
the page(s) in question, since the version on the storage device is
out-of-date, because it hasn't been written yet.

So basically, it's a mess, and if you do need the page again after
it's been written, the fact that you have to read it back in from the
storage device would also be a performance hit in some cases.

> I'm working on fscrypt for UBIFS and would like to avoid an extra
> buffer since memory is low on embedded systems.

The amount of memory needed for the bounce pages is relative to the
speed of your storage device.  It should be possible throttle how much
memory gets used based on how many I/O's you allow to be in flight at
any one time, and hence, how much memory is needed for bounce buffers.

In the long run, I suspect that the real answer for embedded devices
will be hardware support in the form of in-line encryption engines,
since embedded devices tend to have pathetically slow CPU support for
encryption anyway.  It also solves the problem where you have an
extremely unbalanced system (i.e., expensive PCIe attached flash that
can do thousands of IOPS per second, and a tiny wee bit of memory :-)

       		    	     	     	 - Ted

Re: fscrypt: in-place decrypt vs. out-of-place encrypt?

[Theodore Ts'o]

On Thu, Sep 22, 2016 at 10:49:41AM +0200, Richard Weinberger wrote:
> 
> While reading the fscrypt code I noticed that some functions use the bounce pages
> and some not.
> fscrypt_decrypt_page() and fscrypt_decrypt_bio_pages() work in-place while
> fscrypt_encrypt_page() and fscrypt_zeroout_range() use a bounce page.
> 
> So, both ext4 and f2fs encrypt data using an extra buffer but decrypt mostly
> in-place without the need of an extra buffer.
> Why that? I'd assume when decryption can be done in-place also encryption is possible
> that way.

The problem is that the page may be mapped into some process's (or
multiple processes') address space.  The only way we can encrypt in
place is if we make sure that it is not mapped into any processes'
page tables, and prohibit it from being mmapped, *and* prevent any
simultaneous reads from accessing the page for which we are in the
process of destroying its original contents because we are doing an
in-plance encrypt.

So it could be done, but you would have to remove the page from the
page cache first (meaning making sure it's not in any page tables) so
it can't be found by subsequent reads and mmaps, and then sure you
have a lock out which prevents anyone else from trying to read or mmap
the page(s) in question, since the version on the storage device is
out-of-date, because it hasn't been written yet.

So basically, it's a mess, and if you do need the page again after
it's been written, the fact that you have to read it back in from the
storage device would also be a performance hit in some cases.

> I'm working on fscrypt for UBIFS and would like to avoid an extra
> buffer since memory is low on embedded systems.

The amount of memory needed for the bounce pages is relative to the
speed of your storage device.  It should be possible throttle how much
memory gets used based on how many I/O's you allow to be in flight at
any one time, and hence, how much memory is needed for bounce buffers.

In the long run, I suspect that the real answer for embedded devices
will be hardware support in the form of in-line encryption engines,
since embedded devices tend to have pathetically slow CPU support for
encryption anyway.  It also solves the problem where you have an
extremely unbalanced system (i.e., expensive PCIe attached flash that
can do thousands of IOPS per second, and a tiny wee bit of memory :-)

       		    	     	     	 - Ted

------------------------------------------------------------------------------

Re: fscrypt: in-place decrypt vs. out-of-place encrypt?

[Richard Weinberger]

Ted,

On 22.09.2016 15:14, Theodore Ts'o wrote:
> On Thu, Sep 22, 2016 at 10:49:41AM +0200, Richard Weinberger wrote:
>>
>> While reading the fscrypt code I noticed that some functions use the bounce pages
>> and some not.
>> fscrypt_decrypt_page() and fscrypt_decrypt_bio_pages() work in-place while
>> fscrypt_encrypt_page() and fscrypt_zeroout_range() use a bounce page.
>>
>> So, both ext4 and f2fs encrypt data using an extra buffer but decrypt mostly
>> in-place without the need of an extra buffer.
>> Why that? I'd assume when decryption can be done in-place also encryption is possible
>> that way.
> 
> The problem is that the page may be mapped into some process's (or
> multiple processes') address space.  The only way we can encrypt in
> place is if we make sure that it is not mapped into any processes'
> page tables, and prohibit it from being mmapped, *and* prevent any
> simultaneous reads from accessing the page for which we are in the
> process of destroying its original contents because we are doing an
> in-plance encrypt.
> 
> So it could be done, but you would have to remove the page from the
> page cache first (meaning making sure it's not in any page tables) so
> it can't be found by subsequent reads and mmaps, and then sure you
> have a lock out which prevents anyone else from trying to read or mmap
> the page(s) in question, since the version on the storage device is
> out-of-date, because it hasn't been written yet.
> 
> So basically, it's a mess, and if you do need the page again after
> it's been written, the fact that you have to read it back in from the
> storage device would also be a performance hit in some cases.

For UBIFS the story is a bit different. UBIFS operates on a kmap()'ed
page and has already a temporary buffer where it prepares and compresses data.
So, I can use in-place encryption. All I have to do is massaging the fscrypt
interface to allow the filesystem to select when a bouncing page (or buffer in
my case) is needed or not.
I've extended the fscrypt API already to work on buffers too.
In MTD world we don't have struct bio and other fancy stuff. :-)

Thanks,
//richard