Linux-BTRFS Archive on lore.kernel.org
 help / Atom feed
* btrfs development - question about crypto api integration
@ 2018-11-29 16:43 Jean Fobe
  2018-11-29 17:37 ` Nikolay Borisov
  0 siblings, 1 reply; 5+ messages in thread
From: Jean Fobe @ 2018-11-29 16:43 UTC (permalink / raw)
  To: linux-btrfs

Hi all,
    I've been studying LZ4 and other compression algorithms on the
kernel, and seen other projects such as zram and ubifs using the
crypto api. Is there a technical reason for not using the crypto api
for compression (and possibly for encryption) in btrfs?
    I did not find any design/technical implementation choices in
btrfs development in the developer's FAQ on the wiki. If I completely
missed it, could someone point me in the right direction?
    Lastly, if there is no technical reason for this, would it be
something interesting to have implemented?

Best regards

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs development - question about crypto api integration
  2018-11-29 16:43 btrfs development - question about crypto api integration Jean Fobe
@ 2018-11-29 17:37 ` Nikolay Borisov
  2018-11-30 15:22   ` Chris Mason
  0 siblings, 1 reply; 5+ messages in thread
From: Nikolay Borisov @ 2018-11-29 17:37 UTC (permalink / raw)
  To: Jean Fobe, linux-btrfs



On 29.11.18 г. 18:43 ч., Jean Fobe wrote:
> Hi all,
>     I've been studying LZ4 and other compression algorithms on the
> kernel, and seen other projects such as zram and ubifs using the
> crypto api. Is there a technical reason for not using the crypto api
> for compression (and possibly for encryption) in btrfs?
>     I did not find any design/technical implementation choices in
> btrfs development in the developer's FAQ on the wiki. If I completely
> missed it, could someone point me in the right direction?
>     Lastly, if there is no technical reason for this, would it be
> something interesting to have implemented?

I personally think it would be better if btrfs' exploited the generic
framework. And in fact when you look at zstd, btrfs does use the
generic, low-level ZSTD routines but not the crypto library wrappers. If
I were I'd try and convert zstd (since it's the most recently added
algorithm) to using the crypto layer to see if there are any lurking
problems.

> 
> Best regards
> 

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs development - question about crypto api integration
  2018-11-29 17:37 ` Nikolay Borisov
@ 2018-11-30 15:22   ` Chris Mason
  2018-11-30 16:27     ` Nikolay Borisov
  0 siblings, 1 reply; 5+ messages in thread
From: Chris Mason @ 2018-11-30 15:22 UTC (permalink / raw)
  To: Nikolay Borisov; +Cc: Jean Fobe, linux-btrfs

On 29 Nov 2018, at 12:37, Nikolay Borisov wrote:

> On 29.11.18 г. 18:43 ч., Jean Fobe wrote:
>> Hi all,
>>     I've been studying LZ4 and other compression algorithms on the
>> kernel, and seen other projects such as zram and ubifs using the
>> crypto api. Is there a technical reason for not using the crypto api
>> for compression (and possibly for encryption) in btrfs?
>>     I did not find any design/technical implementation choices in
>> btrfs development in the developer's FAQ on the wiki. If I completely
>> missed it, could someone point me in the right direction?
>>     Lastly, if there is no technical reason for this, would it be
>> something interesting to have implemented?
>
> I personally think it would be better if btrfs' exploited the generic
> framework. And in fact when you look at zstd, btrfs does use the
> generic, low-level ZSTD routines but not the crypto library wrappers. 
> If
> I were I'd try and convert zstd (since it's the most recently added
> algorithm) to using the crypto layer to see if there are any lurking
> problems.

Back when I first added the zlib support, the zlib API was both easier 
to use and a better fit for our async worker threads.  That doesn't mean 
we shouldn't switch, it's just how we got to step one ;)

-chris

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs development - question about crypto api integration
  2018-11-30 15:22   ` Chris Mason
@ 2018-11-30 16:27     ` Nikolay Borisov
  2018-12-04 13:29       ` David Sterba
  0 siblings, 1 reply; 5+ messages in thread
From: Nikolay Borisov @ 2018-11-30 16:27 UTC (permalink / raw)
  To: Chris Mason; +Cc: Jean Fobe, linux-btrfs



On 30.11.18 г. 17:22 ч., Chris Mason wrote:
> On 29 Nov 2018, at 12:37, Nikolay Borisov wrote:
> 
>> On 29.11.18 г. 18:43 ч., Jean Fobe wrote:
>>> Hi all,
>>>     I've been studying LZ4 and other compression algorithms on the
>>> kernel, and seen other projects such as zram and ubifs using the
>>> crypto api. Is there a technical reason for not using the crypto api
>>> for compression (and possibly for encryption) in btrfs?
>>>     I did not find any design/technical implementation choices in
>>> btrfs development in the developer's FAQ on the wiki. If I completely
>>> missed it, could someone point me in the right direction?
>>>     Lastly, if there is no technical reason for this, would it be
>>> something interesting to have implemented?
>>
>> I personally think it would be better if btrfs' exploited the generic
>> framework. And in fact when you look at zstd, btrfs does use the
>> generic, low-level ZSTD routines but not the crypto library wrappers. 
>> If
>> I were I'd try and convert zstd (since it's the most recently added
>> algorithm) to using the crypto layer to see if there are any lurking
>> problems.
> 
> Back when I first added the zlib support, the zlib API was both easier 
> to use and a better fit for our async worker threads.  That doesn't mean 
> we shouldn't switch, it's just how we got to step one ;)

And what about zstd? WHy is it also using the low level api and not the
crypto wrappers?

> 
> -chris
> 

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: btrfs development - question about crypto api integration
  2018-11-30 16:27     ` Nikolay Borisov
@ 2018-12-04 13:29       ` David Sterba
  0 siblings, 0 replies; 5+ messages in thread
From: David Sterba @ 2018-12-04 13:29 UTC (permalink / raw)
  To: Nikolay Borisov; +Cc: Chris Mason, Jean Fobe, linux-btrfs

On Fri, Nov 30, 2018 at 06:27:58PM +0200, Nikolay Borisov wrote:
> 
> 
> On 30.11.18 г. 17:22 ч., Chris Mason wrote:
> > On 29 Nov 2018, at 12:37, Nikolay Borisov wrote:
> > 
> >> On 29.11.18 г. 18:43 ч., Jean Fobe wrote:
> >>> Hi all,
> >>>     I've been studying LZ4 and other compression algorithms on the
> >>> kernel, and seen other projects such as zram and ubifs using the
> >>> crypto api. Is there a technical reason for not using the crypto api
> >>> for compression (and possibly for encryption) in btrfs?
> >>>     I did not find any design/technical implementation choices in
> >>> btrfs development in the developer's FAQ on the wiki. If I completely
> >>> missed it, could someone point me in the right direction?
> >>>     Lastly, if there is no technical reason for this, would it be
> >>> something interesting to have implemented?
> >>
> >> I personally think it would be better if btrfs' exploited the generic
> >> framework. And in fact when you look at zstd, btrfs does use the
> >> generic, low-level ZSTD routines but not the crypto library wrappers. 
> >> If
> >> I were I'd try and convert zstd (since it's the most recently added
> >> algorithm) to using the crypto layer to see if there are any lurking
> >> problems.
> > 
> > Back when I first added the zlib support, the zlib API was both easier 
> > to use and a better fit for our async worker threads.  That doesn't mean 
> > we shouldn't switch, it's just how we got to step one ;)
> 
> And what about zstd? WHy is it also using the low level api and not the
> crypto wrappers?

I think beacuse it just copied the way things already were.

Here's fs/ubifs/compress.c as an example use of the API in a filesystem:

ubifs_compress
  crypto_comp_compress
    crypto_comp_crt -- address of, dereference
    ->cot_commpress -- takes another address of something, indirect
		       function call
      with value 'crypto_compress'
	-- this does 2 pointer dereferences and indirect call to
	   coa_compress
	   coa_compress = lzo_compress
	     lzo_compress -- pointer dereferences for crypto context
	       lzo1x_1_compress -- through static __lzo_compress (no overhead)

while in btrfs:

btrfs_compress_pages
  ->compress_pages
    lzo1x_1_compress

The crypto API adds several pointer and function call indirectinos, I'm
not sure I want to get rid of the direct calls just yet.

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-29 16:43 btrfs development - question about crypto api integration Jean Fobe
2018-11-29 17:37 ` Nikolay Borisov
2018-11-30 15:22   ` Chris Mason
2018-11-30 16:27     ` Nikolay Borisov
2018-12-04 13:29       ` David Sterba

Linux-BTRFS Archive on lore.kernel.org

Archives are clonable: git clone --mirror https://lore.kernel.org/linux-btrfs/0 linux-btrfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-btrfs linux-btrfs/ https://lore.kernel.org/linux-btrfs \
		linux-btrfs@vger.kernel.org linux-btrfs@archiver.kernel.org
	public-inbox-index linux-btrfs


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-btrfs


AGPL code for this site: git clone https://public-inbox.org/ public-inbox