* What if TRIM issued a wipe on devices that don't TRIM?
@ 2018-12-06 6:11 Robert White
2018-12-06 6:26 ` Roman Mamedov
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Robert White @ 2018-12-06 6:11 UTC (permalink / raw)
To: Btrfs BTRFS
(1) Automatic and selective wiping of unused and previously used disk
blocks is a good security measure, particularly when there is an
encryption layer beneath the file system.
(2) USB attached devices _never_ support TRIM and they are the most
likely to fall into strangers hands.
(3) I vaguely recall that some flash chips will take bulk writhes of
full sectors of 0x00 or 0xFF (I don't remember which) were second-best
to TRIM for letting the flash controllers defragment their internals.
So it would be dog-slow, but it would be neat if BTRFS had a mount
option to convert any TRIM command from above into the write of a zero,
0xFF, or trash block to the device below if that device doesn't support
TRIM. Real TRIM support would override the block write.
Obviously doing an fstrim would involve a lot of slow device writes but
only for people likely to do that sort of thing.
For testing purposes the destruction of unused pages in this manner
might catch file system failures or coding errors.
(The other layer where this might be most appropriate is in cryptsetup
et al, where it could lie about TRIM support, but that sort of stealth
lag might be bad for filesystem-level operations. Doing it there would
also loose the simpler USB use cases.)
...Just a thought...
--Rob White.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: What if TRIM issued a wipe on devices that don't TRIM?
2018-12-06 6:11 What if TRIM issued a wipe on devices that don't TRIM? Robert White
@ 2018-12-06 6:26 ` Roman Mamedov
2018-12-06 10:04 ` ronnie sahlberg
2018-12-06 13:04 ` Austin S. Hemmelgarn
2 siblings, 0 replies; 6+ messages in thread
From: Roman Mamedov @ 2018-12-06 6:26 UTC (permalink / raw)
To: Robert White; +Cc: Btrfs BTRFS
On Thu, 6 Dec 2018 06:11:46 +0000
Robert White <rwhite@pobox.com> wrote:
> So it would be dog-slow, but it would be neat if BTRFS had a mount
> option to convert any TRIM command from above into the write of a zero,
> 0xFF, or trash block to the device below if that device doesn't support
> TRIM. Real TRIM support would override the block write.
There is such a project:
"dm-linear like target which provides discard, but replaces it with write of
random data to a discarded region. Thus, discarded data is securely deleted."
https://github.com/vt-alt/dm-secdel
--
With respect,
Roman
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: What if TRIM issued a wipe on devices that don't TRIM?
2018-12-06 6:11 What if TRIM issued a wipe on devices that don't TRIM? Robert White
2018-12-06 6:26 ` Roman Mamedov
@ 2018-12-06 10:04 ` ronnie sahlberg
2018-12-06 13:04 ` Austin S. Hemmelgarn
2 siblings, 0 replies; 6+ messages in thread
From: ronnie sahlberg @ 2018-12-06 10:04 UTC (permalink / raw)
To: Robert White; +Cc: Btrfs BTRFS
Hi,
I am more of a SCSI guy than ATA so forgive where I am ignorant.
The SCSI equivalent to TRIM is called UNMAP.
UNMAP is unfortunately only a "hint" to the device so if the device
for any reason
is busy, it can just do a NO-OP, leave the data as is and still
return status SUCCESS.
That is not good if you want to wipe data for confidentiality reasons :-)
As UNMAP and TRIM are related make sure that TRIM actually provides a
guarantee to wipe the
data. I do not know ATA enough to know if it does or not.
In SCSI, instead of using UNMAP/TRIM you can use WRITESAME10/16 which
can be used
and which does providde an overwrite/wipe guarantee. Maybe ATA has
something equivalent to
WRITESAME10/16.
I just want to say: be careful, sometimes these commands do not
provide a guarantee that they will actually
make the data overwritten/unretrievable.
ronnie sahlberg
On Thu, Dec 6, 2018 at 4:24 PM Robert White <rwhite@pobox.com> wrote:
>
> (1) Automatic and selective wiping of unused and previously used disk
> blocks is a good security measure, particularly when there is an
> encryption layer beneath the file system.
>
> (2) USB attached devices _never_ support TRIM and they are the most
> likely to fall into strangers hands.
>
> (3) I vaguely recall that some flash chips will take bulk writhes of
> full sectors of 0x00 or 0xFF (I don't remember which) were second-best
> to TRIM for letting the flash controllers defragment their internals.
>
> So it would be dog-slow, but it would be neat if BTRFS had a mount
> option to convert any TRIM command from above into the write of a zero,
> 0xFF, or trash block to the device below if that device doesn't support
> TRIM. Real TRIM support would override the block write.
>
> Obviously doing an fstrim would involve a lot of slow device writes but
> only for people likely to do that sort of thing.
>
> For testing purposes the destruction of unused pages in this manner
> might catch file system failures or coding errors.
>
> (The other layer where this might be most appropriate is in cryptsetup
> et al, where it could lie about TRIM support, but that sort of stealth
> lag might be bad for filesystem-level operations. Doing it there would
> also loose the simpler USB use cases.)
>
> ...Just a thought...
>
> --Rob White.
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: What if TRIM issued a wipe on devices that don't TRIM?
2018-12-06 6:11 What if TRIM issued a wipe on devices that don't TRIM? Robert White
2018-12-06 6:26 ` Roman Mamedov
2018-12-06 10:04 ` ronnie sahlberg
@ 2018-12-06 13:04 ` Austin S. Hemmelgarn
2018-12-07 4:09 ` Andrei Borzenkov
2 siblings, 1 reply; 6+ messages in thread
From: Austin S. Hemmelgarn @ 2018-12-06 13:04 UTC (permalink / raw)
To: Robert White, Btrfs BTRFS
On 2018-12-06 01:11, Robert White wrote:
> (1) Automatic and selective wiping of unused and previously used disk
> blocks is a good security measure, particularly when there is an
> encryption layer beneath the file system.
>
> (2) USB attached devices _never_ support TRIM and they are the most
> likely to fall into strangers hands.
Not true on the first count. Some really nice UAS devices do support
SCSI UNMAP and WRITESAME commands.
>
> (3) I vaguely recall that some flash chips will take bulk writhes of
> full sectors of 0x00 or 0xFF (I don't remember which) were second-best
> to TRIM for letting the flash controllers defragment their internals.
>
> So it would be dog-slow, but it would be neat if BTRFS had a mount
> option to convert any TRIM command from above into the write of a zero,
> 0xFF, or trash block to the device below if that device doesn't support
> TRIM. Real TRIM support would override the block write.
>
> Obviously doing an fstrim would involve a lot of slow device writes but
> only for people likely to do that sort of thing.
>
> For testing purposes the destruction of unused pages in this manner
> might catch file system failures or coding errors.
>
> (The other layer where this might be most appropriate is in cryptsetup
> et al, where it could lie about TRIM support, but that sort of stealth
> lag might be bad for filesystem-level operations. Doing it there would
> also loose the simpler USB use cases.)
>
> ...Just a thought...
First off, TRIM is an ATA command, not the kernel term. `fstrim`
inherited the ATA name, but in kernel it's called a discard operation,
and it's kind of important to understand here that a discard operation
can result in a number of different behaviors.
In particular, you have at least the following implementations:
* On SCSI devices, a discard operation translates to a SCSI UNMAP
command. As pointed out by Ronnie Sahlberg in his reply, this command
is purely advisory, may not result in any actual state change on the
target device, and is not guaranteed to wipe the data. To actually wipe
things, you have to explicitly write bogus data to the given regions
(using either regular writes, or a WRITESAME command with the desired
pattern), and _then_ call UNMAP on them.
* On dm-thinp devices, a discard operation results in simply unmapping
the blocks in the region it covers. The underlying blocks themselves
are not wiped until they get reallocated (which may not happen when you
write to that region of the dm-thinp device again), and may not even be
wiped then (depending on how the dm-thinp device is configured). Thus,
the same behavior as for SCSI is required here.
* On SD/MMC devices, a discard operation results in an SD ERASE command
being issued. This one is non-advisory (that is, it's guaranteed to
happen), and is supposed to guarantee an overwrite of the region with
zeroes or ones.
* eMMC devices additionally define a discard operation independent of
the SD ERASE command which unmaps the region in the translation layer,
but does not wipe the blocks either on issuing the command or on
re-allocating the low-level blocks. Essentially, it's just a hint for
the wear-leveling algorithm.
* NVMe provides two different discard operations, and I'm not sure which
the kernel uses for NVMe block emulation. They correspond almost
exactly to the SCSI UNMAP and SD ERASE commands in terms of behavior.
* For ATA devices, a discard operation translates to an ATA TRIM
command. This command doesn't even require that the data read back from
a region the command has been issued against be consistent between
reads, let alone that it actually returns zeroes, and it is completely
silent on how the device should actually implement the operation. In
practice, most drives that implement it actually behave like dm-thinp
devices, unmapping the low-level blocks in the region and only clearing
them when they get reallocated, while returning any data they want on
subsequent reads to that logical region until a write happens.
* The MTD subsystem has support for discard operations in the various
FTL's, and they appear from a cursory look at the code to behave like a
non-advisory version of the SCSI UNMAP command (FWIW, MTD's are what the
concept of a discard operation was originally implemented in Linux for).
Notice that the only implementations that are actually guaranteed to
clear out the low-level physical blocks are the SD ERASE and one of the
two NVMe options, and all others require you to manually wipe the data
before issuing the discard operation to guarantee that no data is retained.
Given this, I don't think this should be done as a mechanism of
intercepting or translating discard operations, but as something else
entirely. Perhaps as a block-layer that wipes the region then issues a
discard for it to the lower level device if the device supports it?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: What if TRIM issued a wipe on devices that don't TRIM?
2018-12-06 13:04 ` Austin S. Hemmelgarn
@ 2018-12-07 4:09 ` Andrei Borzenkov
2018-12-07 12:18 ` Austin S. Hemmelgarn
0 siblings, 1 reply; 6+ messages in thread
From: Andrei Borzenkov @ 2018-12-07 4:09 UTC (permalink / raw)
To: Austin S. Hemmelgarn, Robert White, Btrfs BTRFS
06.12.2018 16:04, Austin S. Hemmelgarn пишет:
>
> * On SCSI devices, a discard operation translates to a SCSI UNMAP
> command. As pointed out by Ronnie Sahlberg in his reply, this command
> is purely advisory, may not result in any actual state change on the
> target device, and is not guaranteed to wipe the data. To actually wipe
> things, you have to explicitly write bogus data to the given regions
> (using either regular writes, or a WRITESAME command with the desired
> pattern), and _then_ call UNMAP on them.
WRITE SAME command has UNMAP bit and depending on device and kernel
version kernel may actually issue either UNMAP or WRITE SAME with UNMAP
bit set when doing discard.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: What if TRIM issued a wipe on devices that don't TRIM?
2018-12-07 4:09 ` Andrei Borzenkov
@ 2018-12-07 12:18 ` Austin S. Hemmelgarn
0 siblings, 0 replies; 6+ messages in thread
From: Austin S. Hemmelgarn @ 2018-12-07 12:18 UTC (permalink / raw)
To: Andrei Borzenkov, Robert White, Btrfs BTRFS
On 2018-12-06 23:09, Andrei Borzenkov wrote:
> 06.12.2018 16:04, Austin S. Hemmelgarn пишет:
>>
>> * On SCSI devices, a discard operation translates to a SCSI UNMAP
>> command. As pointed out by Ronnie Sahlberg in his reply, this command
>> is purely advisory, may not result in any actual state change on the
>> target device, and is not guaranteed to wipe the data. To actually wipe
>> things, you have to explicitly write bogus data to the given regions
>> (using either regular writes, or a WRITESAME command with the desired
>> pattern), and _then_ call UNMAP on them.
>
> WRITE SAME command has UNMAP bit and depending on device and kernel
> version kernel may actually issue either UNMAP or WRITE SAME with UNMAP
> bit set when doing discard.
>
Good to know. I've not looked at the SCSI code much, and actually
didn't know about the UNMAP bit for the WRITE SAME command, so I just
assumed that the kernel only used the UNMAP command.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-12-07 12:18 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-12-06 6:11 What if TRIM issued a wipe on devices that don't TRIM? Robert White
2018-12-06 6:26 ` Roman Mamedov
2018-12-06 10:04 ` ronnie sahlberg
2018-12-06 13:04 ` Austin S. Hemmelgarn
2018-12-07 4:09 ` Andrei Borzenkov
2018-12-07 12:18 ` Austin S. Hemmelgarn
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).