Creating detached dm-integrity hashes for existing disk

Creating detached dm-integrity hashes for existing disk

[Ciprian Craciun]

Hello all!  (Please keep me in CC, as I'm not subscribed to the mailing list.)


I already have a large disk containing important data (backup data)
and I would like to use `dm-integrity` (with meta-data on a separate
device) to make sure that when reading from that disk I actually get
the original, non-corrupted, data.  (It seems that just the SATA
provided CRC isn't enough sometime with a problematic USB-to-SATA
adapter, and I get successful reads with corrupted data.)

Thus, I've tried using `integritysetup format` with `--no-wipe`,
`--data-device` and `--integrity-bitmap-mode`, and then
`integritysetup open` with `--integrity-recalculate`.  However upon
subsequent `integritysetup open` without the recalculate flag, the
`dump` command still shows the recalculating flag.  Searching on the
internet about this I've found only a few issues about this, but no
actual solution or more insight.

As such my question:  is there a way to actually force `dm-integrity`
to compute the hashes without wiping my original data?




Here are the snippets I've tried (on blank files via loop devices):


# create empty files
truncate --size 4GiB -- /tmp/dm-integrity-tests-data
truncate --size 512MiB -- /tmp/dm-integrity-tests-meta

# fill the data with something to make sure dm-integrity doesn't touch
the original data
dd if=/dev/urandom of=/tmp/dm-integrity-tests-data bs=1M count=128
iflag=fullblock status=progress

# compute a hash of the original data
md5sum -b -- /tmp/dm-integrity-tests-data /tmp/dm-integrity-tests-meta


# the data is loop0, and the meta-data is loop1
losetup --show --sector-size 4096 --find /tmp/dm-integrity-tests-data
losetup --show --sector-size 4096 --find /tmp/dm-integrity-tests-meta


# format, but DO NOT TOUCH the original data
# both the bitmap and hash should use 4096 bytes
integritysetup format \
        --batch-mode \
        --verbose \
        --no-wipe \
        --sector-size 4096 \
        --integrity sha256 \
        --integrity-bitmap-mode \
        --bitmap-sectors-per-bit 8 \
        --data-device /dev/loop0 \
        -- \
        /dev/loop1 \
#


# open, requesting a recalculation of the hashes
integritysetup open \
        --verbose \
        --integrity sha256 \
        --data-device /dev/loop0 \
        --integrity-recalculate \
        -- \
        /dev/loop1 \
        dm-integrity-tests \
#

# read the target, checking if we get our original data
dd if=/dev/mapper/dm-integrity-tests bs=1M iflag=fullblock status=progress \
| md5sum -b

integritysetup close dm-integrity-tests


# open, without the recalculation flag
integritysetup open \
        --verbose \
        --integrity sha256 \
        --data-device /dev/loop0 \
        -- \
        /dev/loop1 \
        dm-integrity-tests \
#

integritysetup close dm-integrity-tests


# recalculation still shows up
integritysetup dump -- /tmp/dm-integrity-tests-meta


Am I missing something?  Shouldn't the recalculation flag disappear?

Thanks,
Ciprian.

Re: Creating detached dm-integrity hashes for existing disk

[Ciprian Craciun]

Hello all!  (Please keep me in CC, as I'm not subscribed to the mailing list.)

I've sent the following email two months ago, but I assume that either
it slipped through the cracks or nobody has any experience with the
use case I'm mentioning.  However, I'll try to submit this question
one more time, perhaps with more luck this time.  Thanks for the time.


~~~~


I already have a large disk containing important data (backup data)
and I would like to use `dm-integrity` (with meta-data on a separate
device) to make sure that when reading from that disk I actually get
the original, non-corrupted, data.  (It seems that just the SATA
provided CRC isn't enough sometime with a problematic USB-to-SATA
adapter, and I get successful reads with corrupted data.)

Thus, I've tried using `integritysetup format` with `--no-wipe`,
`--data-device` and `--integrity-bitmap-mode`, and then
`integritysetup open` with `--integrity-recalculate`.  However upon
subsequent `integritysetup open` without the recalculate flag, the
`dump` command still shows the recalculating flag.  Searching on the
internet about this I've found only a few issues about this, but no
actual solution or more insight.

As such my question:  is there a way to actually force `dm-integrity`
to compute the hashes without wiping my original data?




Here are the snippets I've tried (on blank files via loop devices):


# create empty files
truncate --size 4GiB -- /tmp/dm-integrity-tests-data
truncate --size 512MiB -- /tmp/dm-integrity-tests-meta

# fill the data with something to make sure dm-integrity doesn't touch
the original data
dd if=/dev/urandom of=/tmp/dm-integrity-tests-dat
a bs=1M count=128
iflag=fullblock status=progress

# compute a hash of the original data
md5sum -b -- /tmp/dm-integrity-tests-data /tmp/dm-integrity-tests-meta


# the data is loop0, and the meta-data is loop1
losetup --show --sector-size 4096 --find /tmp/dm-integrity-tests-data
losetup --show --sector-size 4096 --find /tmp/dm-integrity-tests-meta


# format, but DO NOT TOUCH the original data
# both the bitmap and hash should use 4096 bytes
integritysetup format \
        --batch-mode \
        --verbose \
        --no-wipe \
        --sector-size 4096 \
        --integrity sha256 \
        --integrity-bitmap-mode \
        --bitmap-sectors-per-bit 8 \
        --data-device /dev/loop0 \
        -- \
        /dev/loop1 \
#


# open, requesting a recalculation of the hashes
integritysetup open \
        --verbose \
        --integrity sha256 \
        --data-device /dev/loop0 \
        --integrity-recalculate \
        -- \
        /dev/loop1 \
        dm-integrity-tests \
#

# read the target, checking if we get our original data
dd if=/dev/mapper/dm-integrity-tests bs=1M iflag=fullblock status=progress \
| md5sum -b

integritysetup close dm-integrity-tests


# open, without the recalculation flag
integritysetup open \
        --verbose \
        --integrity sha256 \
        --data-device /dev/loop0 \
        -- \
        /dev/loop1 \
        dm-integrity-tests \
#

integritysetup close dm-integrity-tests


# recalculation still shows up
integritysetup dump -- /tmp/dm-integrity-tests-meta


Am I missing something?  Shouldn't the recalculation flag disappear?

Thanks,
Ciprian.

Re: Creating detached dm-integrity hashes for existing disk

[Milan Broz]

On 11/21/22 15:50, Ciprian Craciun wrote:
> Hello all!  (Please keep me in CC, as I'm not subscribed to the mailing list.)
> 
> I've sent the following email two months ago, but I assume that either
> it slipped through the cracks or nobody has any experience with the
> use case I'm mentioning.  However, I'll try to submit this question
> one more time, perhaps with more luck this time.  Thanks for the time.

:-)

Actually I planned to respond, just had not time to test it.

TL;DR: the recalculation flag is persistent, it is ok that it remains set.


It is there because if the data device is resized later (and as it is separate,
it can happen), you need to recalculate the rest of the device (also automatically).

Check "integrity dump <metadata_device>", you should see recalculation offset,
and also the recalc sector pointer e.g.:

# integritysetup dump /tmp/dm-integrity-tests-meta
Info for integrity device /tmp/dm-integrity-tests-meta.
superblock_version 5
log2_interleave_sectors 0
integrity_tag_size 32
journal_sections 7
provided_data_sectors 262144
sector_size 4096
recalc_sector 262144
log2_blocks_per_bitmap 12
flags recalculating fix_hmac

Here you can see, that recalc_sector is the same as data sectors, so the recalculation
is finished. If you need to recalculater everything from the start, use
--integrity-recalculate-reset option.

There still can be some bugs (mainly for bitmap mode that is very rarely used;
better test it in internal hash mode).

Some more notes inline:

> I already have a large disk containing important data (backup data)
> and I would like to use `dm-integrity` (with meta-data on a separate
> device) to make sure that when reading from that disk I actually get
> the original, non-corrupted, data.  (It seems that just the SATA
> provided CRC isn't enough sometime with a problematic USB-to-SATA
> adapter, and I get successful reads with corrupted data.)
> 
> Thus, I've tried using `integritysetup format` with `--no-wipe`,
> `--data-device` and `--integrity-bitmap-mode`, and then
> `integritysetup open` with `--integrity-recalculate`.  However upon
> subsequent `integritysetup open` without the recalculate flag, the
> `dump` command still shows the recalculating flag.  Searching on the
> internet about this I've found only a few issues about this, but no
> actual solution or more insight.
> 
> As such my question:  is there a way to actually force `dm-integrity`
> to compute the hashes without wiping my original data?

I think you used it in this mode already.

> Here are the snippets I've tried (on blank files via loop devices):
> 
> 
> # create empty files
> truncate --size 4GiB -- /tmp/dm-integrity-tests-data
> truncate --size 512MiB -- /tmp/dm-integrity-tests-meta
> 
> # fill the data with something to make sure dm-integrity doesn't touch
> the original data
> dd if=/dev/urandom of=/tmp/dm-integrity-tests-dat
> a bs=1M count=128
> iflag=fullblock status=progress

here you decreased the data size (you perhaps want add conv=notrunc)

> 
> # compute a hash of the original data
> md5sum -b -- /tmp/dm-integrity-tests-data /tmp/dm-integrity-tests-meta
> 
> 
> # the data is loop0, and the meta-data is loop1
> losetup --show --sector-size 4096 --find /tmp/dm-integrity-tests-data
> losetup --show --sector-size 4096 --find /tmp/dm-integrity-tests-meta

Do not setup loop devices, integritysetup will do it for you automatically
(and removes them on deactivation). Just use images file paths.

m.

Re: Creating detached dm-integrity hashes for existing disk

[Ciprian Craciun]

(Please keep me in CC, as I'm not subscribed to the mailing list.)


On Tue, Nov 22, 2022 at 3:34 PM Milan Broz <gmazyland@gmail.com> wrote:
> There still can be some bugs (mainly for bitmap mode that is very rarely used;
> better test it in internal hash mode).


I am interested in the bitmap mode because I want the drive to behave
as "write-through", and I care more about writes reaching the data
disk, than computing the integrity.  In my case the integrity is only
an afterthought, or optional, and I want to be able to just remove it
and use the underlying data disk directly without losing any writes.

If I understand correctly, with journaled mode the data is first
written to the journal, hashed, and then written to the data disk.
Thus if something happens with the metadata disk, the data disk could
have some writes missing.

Are there any serious disadvantages in using bitmap mode vs journaled mode?

Thanks,
Ciprian.