Cryptographically verifying a btrfs subvolume

Cryptographically verifying a btrfs subvolume

[Leonid Bloch]

Hi List,

Can you suggest a way of cryptographically verifying the content of a 
btrfs subvolume, besides the naïve approach, of running a cryptographic 
hash function on the output of btrfs send?

Back in 2014, an RFC patch was sent to allow using sha256 instead of 
crc32c for checksumming.
(https://patchwork.kernel.org/patch/5363311)
It was not merged. Had it been merged, one could just check the return 
value of btrfs scrub, instead of checksumming the whole btrfs send 
output, correct?


Regards,
Leonid.
___

Re: Cryptographically verifying a btrfs subvolume

[Austin S. Hemmelgarn]

On 2019-04-08 07:27, Leonid Bloch wrote:
> Hi List,
> 
> Can you suggest a way of cryptographically verifying the content of a
> btrfs subvolume, besides the naïve approach, of running a cryptographic
> hash function on the output of btrfs send?
Running BTRFS on top of dm-integrity and dm-crypt with them set up to 
provide AEAD-style encryption comes to mind as an option, and would 
actually provide a much higher level of verification than just verifying 
the content of a subvolume (it will verify the entire filesystem).
> 
> Back in 2014, an RFC patch was sent to allow using sha256 instead of
> crc32c for checksumming.
> (https://patchwork.kernel.org/patch/5363311)
> It was not merged. Had it been merged, one could just check the return
> value of btrfs scrub, instead of checksumming the whole btrfs send
> output, correct?
In theory yes, provided you just want hashes and not an HMAC.

Re: Cryptographically verifying a btrfs subvolume

[Johannes Thumshirn]

On Mon, Apr 08, 2019 at 11:27:51AM +0000, Leonid Bloch wrote:
> Hi List,
> 
> Can you suggest a way of cryptographically verifying the content of a 
> btrfs subvolume, besides the naïve approach, of running a cryptographic 
> hash function on the output of btrfs send?
> 
> Back in 2014, an RFC patch was sent to allow using sha256 instead of 
> crc32c for checksumming.
> (https://patchwork.kernel.org/patch/5363311)
> It was not merged. Had it been merged, one could just check the return 
> value of btrfs scrub, instead of checksumming the whole btrfs send 
> output, correct?

Hi Leonid,

I'm working on this feature currently but unfortunately get distracted with
other things all too often.

Thanks for your patience.

Byte,
	Johannes
-- 
Johannes Thumshirn                            SUSE Labs Filesystems
jthumshirn@suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

Re: Cryptographically verifying a btrfs subvolume

[Leonid Bloch]

On 4/8/19 3:44 PM, Austin S. Hemmelgarn wrote:
> On 2019-04-08 07:27, Leonid Bloch wrote:
>> Hi List,
>>
>> Can you suggest a way of cryptographically verifying the content of a
>> btrfs subvolume, besides the naïve approach, of running a cryptographic
>> hash function on the output of btrfs send?
> Running BTRFS on top of dm-integrity and dm-crypt with them set up to 
> provide AEAD-style encryption comes to mind as an option, and would 
> actually provide a much higher level of verification than just verifying 
> the content of a subvolume (it will verify the entire filesystem).

Thanks! That's actually a good point, I would like to verify a specific 
subvolume(s), while on others the content can change.

That's a good point cause it shows that I was wrong assuming that btrfs 
scrub would help - it will scrub the entire filesystem as well, and 
compare checksums internally, which is not what I want - I want to 
compare to some external checksum. Sorry for the confusion there.

>>
>> Back in 2014, an RFC patch was sent to allow using sha256 instead of
>> crc32c for checksumming.
>> (https://patchwork.kernel.org/patch/5363311)
>> It was not merged. Had it been merged, one could just check the return
>> value of btrfs scrub, instead of checksumming the whole btrfs send
>> output, correct?
> In theory yes, provided you just want hashes and not an HMAC.

Re: Cryptographically verifying a btrfs subvolume

[Leonid Bloch]

On 4/8/19 4:10 PM, Johannes Thumshirn wrote:
> On Mon, Apr 08, 2019 at 11:27:51AM +0000, Leonid Bloch wrote:
>> Hi List,
>>
>> Can you suggest a way of cryptographically verifying the content of a
>> btrfs subvolume, besides the naïve approach, of running a cryptographic
>> hash function on the output of btrfs send?
>>
>> Back in 2014, an RFC patch was sent to allow using sha256 instead of
>> crc32c for checksumming.
>> (https://patchwork.kernel.org/patch/5363311)
>> It was not merged. Had it been merged, one could just check the return
>> value of btrfs scrub, instead of checksumming the whole btrfs send
>> output, correct?
> 
> Hi Leonid,
> 
> I'm working on this feature currently but unfortunately get distracted with
> other things all too often.
> 
> Thanks for your patience.
> 
> Byte,
> 	Johannes

Hi Johannes,

Thanks for letting know! Btrfs inspect-internal csum-dump indeed sounds 
awesome! Recursively, on a subvolume that would be perfect. Let me know 
if help is needed.

Thanks,
Leonid.
___

Re: Cryptographically verifying a btrfs subvolume

[Johannes Thumshirn]

On Mon, Apr 08, 2019 at 01:49:49PM +0000, Leonid Bloch wrote:
> Thanks for letting know! Btrfs inspect-internal csum-dump indeed sounds 
> awesome! Recursively, on a subvolume that would be perfect. Let me know 
> if help is needed.

Well actually the intention of this part was mostly to have a proper way of
verifying the checksums we write in the checksum tree.

I have a patchset in development that adds SHA-256 in the mix (it's actually
very similar to the patch you've referenced).

Apart from SHA-256 I want to have a not cryptographically secure checksum as
well which would help detecting duplicate extents and can be used for
de-duplication.

But this patchset is not anywhere near I can (or better want to) show to
someone.

Thanks for your patience.
	Johannes
-- 
Johannes Thumshirn                            SUSE Labs Filesystems
jthumshirn@suse.de                                +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Mary Higgins, Sri Rasiah
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850

Re: Cryptographically verifying a btrfs subvolume

[Austin S. Hemmelgarn]

On 2019-04-08 09:30, Leonid Bloch wrote:
> On 4/8/19 3:44 PM, Austin S. Hemmelgarn wrote:
>> On 2019-04-08 07:27, Leonid Bloch wrote:
>>> Hi List,
>>>
>>> Can you suggest a way of cryptographically verifying the content of a
>>> btrfs subvolume, besides the naïve approach, of running a cryptographic
>>> hash function on the output of btrfs send?
>> Running BTRFS on top of dm-integrity and dm-crypt with them set up to
>> provide AEAD-style encryption comes to mind as an option, and would
>> actually provide a much higher level of verification than just verifying
>> the content of a subvolume (it will verify the entire filesystem).
> 
> Thanks! That's actually a good point, I would like to verify a specific
> subvolume(s), while on others the content can change.
> 
> That's a good point cause it shows that I was wrong assuming that btrfs
> scrub would help - it will scrub the entire filesystem as well, and
> compare checksums internally, which is not what I want - I want to
> compare to some external checksum. Sorry for the confusion there.
If your primary goal is to just verify that nobody has changed the 
contents of the subvolume, then even hashing the send stream may be 
problematic for your use case. Certain differences in the on-disk layout 
of the data (which could be caused by deduplication or defragmentation) 
result in changes in the send stream, but don't actually change anything 
from the perspective of userspace. Similarly, the send stream protocol 
has changed slightly over time in backwards compatible ways, and such 
changes may happen again in the future, so even if the on-disk layout is 
identical, the hash may visibly change.