ignore/update integrity checksums

ignore/update integrity checksums

[Erich Eckner]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

I have multiple disks with LUKS+integrity created by

cryptsetup luksFormat /dev/sde --key-file /mnt/key/key --integrity hmac-sha256

which are part of a raid6. Details of the device:

/dev/mapper/leg0 is active.
   type:    LUKS2
   cipher:  aes-xts-plain64
   keysize: 768 bits
   key location: keyring
   integrity: hmac(sha256)
   integrity keysize: 256 bits
   device:  /dev/sdb
   sector size:  512
   offset:  0 sectors
   size:    11031354576 sectors
   mode:    read/write


Recently, I rebooted this box and apparently, I missed to cleanly sync the 
disks, so they now report integrity errors, when mdadm probes (during 
assemble) for the raid superblock:

device-mapper: crypt: dm-1: INTEGRITY AEAD ERROR, sector 11031354368

There was no write activity on the raid before the reboot except for a 
running

mdadm /dev/md0 --replace /dev/dm-0 --with /dev/dm-1

which of course might have written a lot to all superblocks.

Since I believe, the superblocks should be mostly in-sync (except for 
event-counters?): Is there a way to ignore or re-calculate the integrity 
checks?

Also: What is the correct way to assure, that data has been synced to the 
disk(s) before switching off power? (If that matters, there is a 
raid-controller underneath: "06:00.0 RAID bus controller: Hewlett-Packard 
Company Smart Array G6 controllers (rev 01)" - but it
does not actually handle the raid, it only feeds the disks through to the 
os) I can execute any command after closing the luks-integrity device, my 
question aims at: what should I execute?

regards,
Erich


-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEE3p92iMrPBP64GmxZCu7JB1Xae1oFAl3XpUQACgkQCu7JB1Xa
e1ry/hAAsWBT+D4UgVEbLNQj8s2cpFP8z4iTfT2vTG9UeEilxjpxrU5tPcRGKANv
qewyTwC5JrH6Mst3AE6P+ToSUWisMOcUXSg663dh4q8lmJjL/IdBrTU6cMqrFJV/
QXx3crf3j6soZ9AXSQgGRJm0wIV24WKg5vgTTDug56TR0ifUK+KJrWsqtforhtYu
hynMYlMHOcYUX4aPRHL6M56K9dSt92/9i5oRcm+jmKKSnTceJHduSD4RPMs93Lb9
JYIef1Qg+0wZqPZr1JD5AR+IU4J4dvysiqP9H3XYNIZaGYo2dGI6q6+6ksuGDxDu
sCeMFNlOdftptC8HlVUm29sERNEXqN+cWZ5X9OdEPKPuY+fQtF4vugy9rRcw+Q41
FHLGs5Aip/jGbXxh0vtOflcpZF4O4j4mu5r0KWEh1YgMIOb+JeWueGamSCdvb0yP
p/ZwfCM0a3sncOw5YRIIS6YXshCl57xzgRv326Z199Y9uROAd0pGvW2B91pKS2QM
W++AaEozRAT5X/5PSSJpw9bx2iprS6YwkygCeFuIswcrMfvbaiwBdXeIVvPZRJjO
lTVMnxWIqt3zkDWaZdwJzQCFycLtlGGXlei4Y3TCBfwRRGBWj6iLRY6t7SlKX9ST
7KHoha1dp2EheKJV6eZXo0mxrF7bt6gcPeVpPZIKKGJG5XkgHMA=
=VedA
-----END PGP SIGNATURE-----

Re: ignore/update integrity checksums

[Erich Eckner]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Fri, 22 Nov 2019, Erich Eckner wrote:

> Hi,
>
> I have multiple disks with LUKS+integrity created by
>
> cryptsetup luksFormat /dev/sde --key-file /mnt/key/key --integrity 
> hmac-sha256
>
> which are part of a raid6. Details of the device:
>
> /dev/mapper/leg0 is active.
>  type:    LUKS2
>  cipher:  aes-xts-plain64
>  keysize: 768 bits
>  key location: keyring
>  integrity: hmac(sha256)
>  integrity keysize: 256 bits
>  device:  /dev/sdb
>  sector size:  512
>  offset:  0 sectors
>  size:    11031354576 sectors
>  mode:    read/write
>
>
> Recently, I rebooted this box and apparently, I missed to cleanly sync the 
> disks, so they now report integrity errors, when mdadm probes (during 
> assemble) for the raid superblock:
>
> device-mapper: crypt: dm-1: INTEGRITY AEAD ERROR, sector 11031354368
>
> There was no write activity on the raid before the reboot except for a 
> running
>
> mdadm /dev/md0 --replace /dev/dm-0 --with /dev/dm-1
>
> which of course might have written a lot to all superblocks.
>
> Since I believe, the superblocks should be mostly in-sync (except for 
> event-counters?): Is there a way to ignore or re-calculate the integrity 
> checks?
>
> Also: What is the correct way to assure, that data has been synced to the 
> disk(s) before switching off power? (If that matters, there is a 
> raid-controller underneath: "06:00.0 RAID bus controller: Hewlett-Packard 
> Company Smart Array G6 controllers (rev 01)" - but it
> does not actually handle the raid, it only feeds the disks through to the os) 
> I can execute any command after closing the luks-integrity device, my 
> question aims at: what should I execute?
>
> regards,
> Erich
>
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>

just a follow-up experimentation with the broken disks:

I noticed /sys/block/dm-0/integrity/read_verify and similar which should 
manipulate verification upon read and updating upon write, according to

https://github.com/ibuildthecloud/ubuntu-kernel/blob/master/Documentation/block/data-integrity.txt#L169

However, changing /sys/block/dm-0/integrity/read_verify to 0 (it was at 1 
before) does not change the behaviour: `mdadm --examine` still generates 
read errors and cannot find its superblock for the corresponding crypt 
device.

Oh, I just see, that I forgot all the details of my system in the first 
email - sry! - here it comes:

This box is running arch linux with up-to-date packages.

# uname -a
Linux backup 5.3.12-arch1-1 #1 SMP PREEMPT Wed, 20 Nov 2019 19:45:16 +0000 
x86_64 GNU/Linux

# pacman -Q cryptsetup mdadm
cryptsetup 2.2.2-1
mdadm 4.1-2

regards,
Erich


-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEE3p92iMrPBP64GmxZCu7JB1Xae1oFAl3Xs8gACgkQCu7JB1Xa
e1qykQ//cTcBJ1NGt2R3Cs23AGwdsUzsqyrf3ahPPWryFvN53GbYT6LOrjCqpF8n
DWNFQ/xBqDwpuQh3l3MatEoaIewGMgkjt2gcpG1aVPnZWJo1hgJDhpXW8locBpN+
h/q8bxcD9nOkMApDwu9bQxLIJcqBZISTgE8QmAvKvXFiPSuh22Gbth6NtSki2G3U
95TPvNbdMJCSG32SEQ1F3/4nky20CMkd31sam44pL//Mr9a6sSeKeaUeQ9rX13Oe
58kmkKp3d8ZIp89ruWgca2wEE5SZA3qdewWs5dfkCnLxzIs8tjL9jfrp1wR+yKNb
DiDj2i203IPxxIgcwu+aDPxboCrJDu3zEE1OcQf6ZDd7nGqsdaf2aGp/F/SkqQ9s
d20+DZSi+2NSQ8n782/zXuroVmy1of4eDn2flFSZFOKfuqgtWcQFnN+JZsZT/bIQ
xNKuD8eIY5lvO58Rpkg38JjIMfVuQiGdjfh1CK/ZUeYOXC6YnGgmOaZqdIWMH4ya
JknY7oXkCluGRNKn3wObufRMa3LkAjoRTmHRw4JpLWJgBxXaj7vsc/thrWy+Uw+3
cmwTMQBR3YttEAYsMRp+s6QNvN6ny3FlNAupnrSTiujjSnO7RZKTV5VeBOcAeoL4
4dTi0FTpehWWA6U3UlmUOafyZjGCHuXDVHKAbVUHiHjQJIpSztw=
=9Xhi
-----END PGP SIGNATURE-----

ignore/update integrity checksums

[Erich Eckner]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi,

I have multiple disks with LUKS+integrity created by

cryptsetup luksFormat /dev/sde --key-file /mnt/key/key --integrity hmac-sha256

which are part of a raid6. Details of the device:

/dev/mapper/leg0 is active.
   type:    LUKS2
   cipher:  aes-xts-plain64
   keysize: 768 bits
   key location: keyring
   integrity: hmac(sha256)
   integrity keysize: 256 bits
   device:  /dev/sdb
   sector size:  512
   offset:  0 sectors
   size:    11031354576 sectors
   mode:    read/write


Recently, I rebooted this box and apparently, I missed to cleanly sync the 
disks, so they now report integrity errors, when mdadm probes (during 
assemble) for the raid superblock:

device-mapper: crypt: dm-1: INTEGRITY AEAD ERROR, sector 11031354368

There was no write activity on the raid before the reboot except for a 
running

mdadm /dev/md0 --replace /dev/dm-0 --with /dev/dm-1

which of course might have written a lot to all superblocks.

Since I believe, the superblocks should be mostly in-sync (except for 
event-counters?): Is there a way to ignore or re-calculate the integrity 
checks?

Also: What is the correct way to assure, that data has been synced to the 
disk(s) before switching off power? (If that matters, there is a 
raid-controller underneath: "06:00.0 RAID bus controller: Hewlett-Packard 
Company Smart Array G6 controllers (rev 01)" - but it does not actually 
handle the raid, it only feeds the disks through to the os) I can execute 
any command after closing the luks-integrity device, my question aims at: 
what should I execute?

regards,
Erich

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEE3p92iMrPBP64GmxZCu7JB1Xae1oFAl3XojcACgkQCu7JB1Xa
e1q17g//TCKyH5US3c5j+FHH/90gsAoW8LuMzeIV3KnswtJm1fA/lWhbJMbKG/8r
eo1P48wOKLRTNT5RJV+M3+we/80la/x6PGl2IM/WXTGT8TAdOISKksifzJ7/wXhd
UxIjzhenc2mFnpYjrMABY3BYGvJf7AkrmrmliGe/BO9sMKej6R1WlE17ZOD+sGFN
/FWLPtgcBcfVFNk+tn2aUa6oovLxpzzhYUcktuqpI0WQOOydQwRsb3sIM0bYHpNZ
99ieIHyIDY7p0V/Ym0Msm/ic1PIzsPtWl6MiRmzjXf8v0UVVFKb23kuPxXHKEBLV
5u/DOzbrBpT+w5hI8eUk5me/qUPWEnrY/4AodZEJxwOpbeqOEbLPa15cPxwT5s3b
+45mpf0raREAI4WANNym5FmAxVukaQKaEGJ7Q8AyVJtN76oGaPNhxEQ4BDxQXOsV
RituTR17MULV80jpVS2atwRNkged9H6aRaDtcmrkx2qyaAHSLghXTLySID9XumbW
pkH45nWvsU1z8t8Lvxy81CrMnB8cmy6Iud3yH1WVKvCnjqmbSZQ4FMXw0fiAas+C
Q1ipkMdKg2mgLUQCRgwWpbvtVR14kZnQjZoPAhOTzbGaYTRbaEPkOMoKqCRStUkb
3l+Hg+AGeP9CR19sFYoR2HSnJGdeK/kYUEBuNfHrOjop2yi+lZY=
=aG59
-----END PGP SIGNATURE-----