inline deduplication security issues

inline deduplication security issues

[Andrei Bacs]

[-- Attachment #1: Type: text/plain, Size: 443 bytes --]

Hi,

We have found security issues with inline deduplication in storage
systems, using ZFS and Btrfs and running examples. See the attached
paper for details.

The paper will be presented at FAST'22 and thus our findings will be
public by 23rd of February 2022.

Please let us know if you plan to deploy any defenses (such as the one
we describe in the paper) by 23rd of February 2022. If not, we can
make our findings public earlier.

Andrei

[-- Attachment #2: paper.pdf --]
[-- Type: application/pdf, Size: 269987 bytes --]

Re: inline deduplication security issues

[Diego Calleja]

El sábado, 22 de enero de 2022 19:42:47 (CET) Andrei Bacs escribió:
> We have found security issues with inline deduplication in storage
> systems, using ZFS and Btrfs and running examples. See the attached
> paper for details.

(Not actually a btrfs developer here)

I am confused, Btrfs does not support inline deduplication. The inline 
deduplication implementation used in that paper is pretty old and as far as I 
know it's not maintained (people seem to be happy with out of band 
deduplication).

You might want to contact the developer on the inline implementation: https://
lore.kernel.org/linux-btrfs/20181106064122.6154-1-lufq.fnst@cn.fujitsu.com/

Also, this is a public mailing list, so there is no point in waiting until 
23rd of February 2022 to make things public.

Kind regards.

Re: inline deduplication security issues

[Su Yue]

On Sun 23 Jan 2022 at 20:18, Diego Calleja <diegocg@gmail.com> 
wrote:

> El sábado, 22 de enero de 2022 19:42:47 (CET) Andrei Bacs 
> escribió:
>> We have found security issues with inline deduplication in 
>> storage
>> systems, using ZFS and Btrfs and running examples. See the 
>> attached
>> paper for details.
>
> (Not actually a btrfs developer here)
>
> I am confused, Btrfs does not support inline deduplication. The 
> inline
> deduplication implementation used in that paper is pretty old 
> and as far as I
> know it's not maintained (people seem to be happy with out of 
> band
> deduplication).
>
> You might want to contact the developer on the inline 
> implementation: https://
> lore.kernel.org/linux-btrfs/20181106064122.6154-1-lufq.fnst@cn.fujitsu.com/
>
AFAK, Fujitsu has no more plans about btrfs.
So there is no follow-up version of the inline deduplication 
feature.

--
Su

> Also, this is a public mailing list, so there is no point in 
> waiting until
> 23rd of February 2022 to make things public.
>
> Kind regards.

Re: inline deduplication security issues

[Qu Wenruo]

On 2022/1/24 09:38, Su Yue wrote:
>
> On Sun 23 Jan 2022 at 20:18, Diego Calleja <diegocg@gmail.com> wrote:
>
>> El sábado, 22 de enero de 2022 19:42:47 (CET) Andrei Bacs escribió:
>>> We have found security issues with inline deduplication in storage
>>> systems, using ZFS and Btrfs and running examples. See the attached
>>> paper for details.
>>
>> (Not actually a btrfs developer here)
>>
>> I am confused, Btrfs does not support inline deduplication. The inline
>> deduplication implementation used in that paper is pretty old and as
>> far as I
>> know it's not maintained (people seem to be happy with out of band
>> deduplication).
>>
>> You might want to contact the developer on the inline implementation:
>> https://
>> lore.kernel.org/linux-btrfs/20181106064122.6154-1-lufq.fnst@cn.fujitsu.com/
>>
>>
> AFAK, Fujitsu has no more plans about btrfs.
> So there is no follow-up version of the inline deduplication feature.

To add more, as one of the original authors, I and my employer have no
interest to push write-time dedupe any more.

Furthermore, the original implementation has one limit, one extent must
be written to disk, before it can be utilized by write-time dedupe.

Thanks,
Qu
>
> --
> Su
>
>> Also, this is a public mailing list, so there is no point in waiting
>> until
>> 23rd of February 2022 to make things public.
>>
>> Kind regards.

Re: inline deduplication security issues

[David Sterba]

On Mon, Jan 24, 2022 at 07:45:20PM +0800, Qu Wenruo wrote:
> 
> 
> On 2022/1/24 09:38, Su Yue wrote:
> >
> > On Sun 23 Jan 2022 at 20:18, Diego Calleja <diegocg@gmail.com> wrote:
> >
> >> El sábado, 22 de enero de 2022 19:42:47 (CET) Andrei Bacs escribió:
> >>> We have found security issues with inline deduplication in storage
> >>> systems, using ZFS and Btrfs and running examples. See the attached
> >>> paper for details.
> >>
> >> (Not actually a btrfs developer here)
> >>
> >> I am confused, Btrfs does not support inline deduplication. The inline
> >> deduplication implementation used in that paper is pretty old and as
> >> far as I
> >> know it's not maintained (people seem to be happy with out of band
> >> deduplication).
> >>
> >> You might want to contact the developer on the inline implementation:
> >> https://
> >> lore.kernel.org/linux-btrfs/20181106064122.6154-1-lufq.fnst@cn.fujitsu.com/
> >>
> > AFAK, Fujitsu has no more plans about btrfs.
> > So there is no follow-up version of the inline deduplication feature.
> 
> To add more, as one of the original authors, I and my employer have no
> interest to push write-time dedupe any more.
> 
> Furthermore, the original implementation has one limit, one extent must
> be written to disk, before it can be utilized by write-time dedupe.

The inline deduplication feature has been abandoned, the patches were
incomplete and overall the feature is very complex to be implemented in
kernel. The replacement should be a user space tool like BEES and the
kernel module provides ioctls to do the actual deduplication (or help in
another way). Many things are much easier in userspace, also with the
clear separation of kernel/userspace when in it comes to the complexity.