* [BUG] Oops caused by FEC in 4.10.0
@ 2017-03-15 20:22 michal virgovic
[not found] ` <20170315221223.GA116381@samitolvanen.mtv.corp.google.com>
0 siblings, 1 reply; 5+ messages in thread
From: michal virgovic @ 2017-03-15 20:22 UTC (permalink / raw)
To: dm-devel; +Cc: samitolvanen, agk, snitzer, Milan Broz
[-- Attachment #1: Type: text/plain, Size: 11933 bytes --]
Hi,
This oops keeps appearing.
Attachment file is the reproducer. Fec,hash devices, salt and root
digest are not correct, if required i can send a script, where
everything is computed properly.
Output of crash:
Message from syslogd@debian8 at Mar 15 20:08:54 ...
kernel:[ 731.046277] BUG: stack guard page was hit at
ffffc90001233ff8 (stack is ffffc90001234000..ffffc90001237fff)
[ 731.046277] BUG: stack guard page was hit at ffffc90001233ff8
(stack is ffffc90001234000..ffffc90001237fff)
[ 731.046284] kernel stack overflow (double-fault): 0000 [#1] SMP
[ 731.046285] Modules linked in: sha256_ssse3 sha256_generic
dm_verity dm_bufio reed_solomon dm_mod loop nls_utf8 isofs udf
crc_itu_t binfmt_misc cfg80211 rfkill nfsd auth_rpcgss oid_registry
nfs_acl nfs lockd grace fscache sunrpc joydev crct10dif_pclmul
crc32_pclmul ghash_clmulni_intel pcbc snd_intel8x0 aesni_intel
snd_ac97_codec aes_x86_64 crypto_simd snd_pcm glue_helper cryptd
i2c_piix4 acpi_cpufreq i2c_core video snd_timer evdev ac97_bus snd
serio_raw soundcore button tpm_tis tpm_tis_core pcspkr battery ac tpm
fuse parport_pc ppdev lp parport autofs4 ext4 crc16 jbd2 mbcache sg
sr_mod cdrom sd_mod ata_generic hid_generic usbhid hid ohci_pci
ehci_pci ahci ohci_hcd ata_piix libahci ehci_hcd crc32c_intel usbcore
psmouse libata usb_common scsi_mod e1000
[ 731.046329] CPU: 2 PID: 123 Comm: kworker/u8:2 Not tainted 4.10.0 #1
[ 731.046330] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS
VirtualBox 12/01/2006
[ 731.046340] Workqueue: kverityd verity_work [dm_verity]
[ 731.046341] task: ffff88012fda0180 task.stack: ffffc90001234000
[ 731.046345] RIP: 0010:__flush_write_list+0x18/0x80 [dm_bufio]
[ 731.046346] RSP: 0018:ffffc90001234000 EFLAGS: 00010286
[ 731.046347] RAX: ffff88012fda0180 RBX: ffffc90001234058 RCX: 0000000000000000
[ 731.046348] RDX: ffff88012fda0180 RSI: ffff880152557820 RDI: ffffc90001234000
[ 731.046349] RBP: ffffc90001234040 R08: ffffc90001234058 R09: ffffc90001234148
[ 731.046349] R10: 000000000000a000 R11: 0000000000000000 R12: ffff880152557800
[ 731.046350] R13: ffffc900012340b0 R14: 0000000000000001 R15: ffff88015228d0c0
[ 731.046352] FS: 0000000000000000(0000) GS:ffff880159d00000(0000)
knlGS:0000000000000000
[ 731.046352] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 731.046353] CR2: ffffc90001233ff8 CR3: 000000012ec61000 CR4: 00000000000406e0
[ 731.046359] Call Trace:
[ 731.046365] ? __bufio_new+0x8e/0x2e0 [dm_bufio]
[ 731.046366] ? new_read+0x5a/0xe0 [dm_bufio]
[ 731.046368] ? verity_verify_level+0x72/0x180 [dm_verity]
[ 731.046369] ? verity_hash_for_block+0x3c/0xe0 [dm_verity]
[ 731.046370] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046380] ? __kernel_fpu_end+0x20/0xa0
[ 731.046381] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046383] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046384] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046386] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046387] ? __kernel_fpu_end+0x20/0xa0
[ 731.046388] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046390] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046391] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046392] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046394] ? __kernel_fpu_end+0x20/0xa0
[ 731.046395] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046396] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046398] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046399] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046400] ? __kernel_fpu_end+0x20/0xa0
[ 731.046402] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046403] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046404] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046406] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046407] ? __kernel_fpu_end+0x20/0xa0
[ 731.046408] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046410] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046411] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046412] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046414] ? __kernel_fpu_end+0x20/0xa0
[ 731.046415] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046417] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046418] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046419] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046421] ? __kernel_fpu_end+0x20/0xa0
[ 731.046422] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046423] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046425] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046426] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046427] ? __kernel_fpu_end+0x20/0xa0
[ 731.046429] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046430] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046431] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046432] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046434] ? __kernel_fpu_end+0x20/0xa0
[ 731.046435] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046437] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046438] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046439] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046441] ? __kernel_fpu_end+0x20/0xa0
[ 731.046442] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046443] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046445] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046454] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046456] ? __kernel_fpu_end+0x20/0xa0
[ 731.046457] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046459] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046460] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046461] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046463] ? __kernel_fpu_end+0x20/0xa0
[ 731.046464] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046465] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046467] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046468] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046469] ? __kernel_fpu_end+0x20/0xa0
[ 731.046471] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046472] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046474] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046475] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046476] ? __kernel_fpu_end+0x20/0xa0
[ 731.046478] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046479] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046480] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046482] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046483] ? __kernel_fpu_end+0x20/0xa0
[ 731.046484] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046486] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046487] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046488] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046490] ? __kernel_fpu_end+0x20/0xa0
[ 731.046491] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046492] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046494] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046495] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046497] ? __kernel_fpu_end+0x20/0xa0
[ 731.046498] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046499] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046501] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046502] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046503] ? __kernel_fpu_end+0x20/0xa0
[ 731.046505] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046506] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046507] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046509] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046510] ? __kernel_fpu_end+0x20/0xa0
[ 731.046511] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046513] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046514] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046515] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046517] ? __kernel_fpu_end+0x20/0xa0
[ 731.046518] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046520] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046521] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046522] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046524] ? __kernel_fpu_end+0x20/0xa0
[ 731.046525] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046526] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046528] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046529] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046530] ? __kernel_fpu_end+0x20/0xa0
[ 731.046532] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046533] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046534] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046536] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046537] ? __kernel_fpu_end+0x20/0xa0
[ 731.046538] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046540] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046541] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046542] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046544] ? __kernel_fpu_end+0x20/0xa0
[ 731.046545] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046547] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046548] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046549] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046551] ? __kernel_fpu_end+0x20/0xa0
[ 731.046552] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046553] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046555] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046556] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046559] ? set_next_entity+0x70/0x9e0
[ 731.046560] ? __kernel_fpu_end+0x20/0xa0
[ 731.046561] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046563] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046564] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046565] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046567] ? __kernel_fpu_end+0x20/0xa0
[ 731.046568] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046569] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046571] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046572] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046585] ? __blk_run_queue+0x2f/0x40
[ 731.046586] ? __kernel_fpu_end+0x20/0xa0
[ 731.046588] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046589] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046591] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046592] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046593] ? __kernel_fpu_end+0x20/0xa0
[ 731.046595] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046596] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046597] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046599] ? fec_decode_rsb+0x6a9/0x9c0 [dm_verity]
[ 731.046600] ? __kernel_fpu_end+0x20/0xa0
[ 731.046601] ? verity_fec_decode+0x8e/0x160 [dm_verity]
[ 731.046603] ? verity_verify_level+0x151/0x180 [dm_verity]
[ 731.046604] ? verity_hash_for_block+0xa8/0xe0 [dm_verity]
[ 731.046605] ? verity_work+0x5e/0x189 [dm_verity]
[ 731.046609] ? process_one_work+0x14d/0x410
[ 731.046611] ? worker_thread+0x62/0x490
[ 731.046612] ? rescuer_thread+0x320/0x320
[ 731.046614] ? kthread+0x104/0x140
[ 731.046615] ? kthread_park+0x80/0x80
[ 731.046621] ? ret_from_fork+0x26/0x40
[ 731.046621] Code: 41 5d 41 5e 41 5f e9 78 f0 d4 e0 0f 0b 66 0f 1f
44 00 00 0f 1f 44 00 00 55 48 89 e5 53 48 89 fb 48 83 e4 f0 48 83 ec
30 48 89 e7 <e8> 93 a1 d4 e0 48 8b 03 48 39 c3 74 4c 48 8b 03 48 be 00
01 00
[ 731.046638] RIP: __flush_write_list+0x18/0x80 [dm_bufio] RSP:
ffffc90001234000
[ 731.046639] ---[ end trace 55a62936c3c55e6b ]---
[-- Attachment #2: fec_crash --]
[-- Type: application/octet-stream, Size: 964 bytes --]
#!/bin/bash -x
# fails on kernel 4.10.0
DEV_NAME=test_device
DEV_DATA=test_data
DEV_DATA_L=/dev/loop0
DEV_HASH=hash_dev
DEV_HASH_L=/dev/loop1
DEV_FEC=fec_dev
DEV_FEC_L=/dev/loop2
# prepare devices
dd if=/dev/zero of=$DEV_DATA bs=4096 count=10
dd if=/dev/zero of=$DEV_HASH bs=4096 count=10
dd if=/dev/zero of=$DEV_FEC bs=4096 count=50
losetup $DEV_DATA_L $DEV_DATA
losetup $DEV_HASH_L $DEV_HASH
losetup $DEV_FEC_L $DEV_FEC
# create device with dmsetup with specified table
dmsetup -r create $DEV_NAME --table "0 80 verity 1 $DEV_DATA_L $DEV_HASH_L 4096 4096 10 0 sha256 890941fb54f49d9ec59872d279faa5ff93f1f9ded66632000dec18f3f444b7ec aea32343144f02cf6e44a50343386eab250848412e3ece1880394814ac4aaa54 8 use_fec_from_device $DEV_FEC_L fec_roots 10 fec_blocks 20 fec_start 0"
udevadm settle
dmsetup table $DEV_NAME
dmsetup ls
# clean
dmsetup remove $DEV_NAME
losetup -d $DEV_DATA_L
losetup -d $DEV_HASH_L
losetup -d $DEV_FEC_L
rm $DEV_HASH $DEV_FEC $DEV_DATA
[-- Attachment #3: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BUG] Oops caused by FEC in 4.10.0
[not found] ` <20170315221223.GA116381@samitolvanen.mtv.corp.google.com>
@ 2017-03-16 7:15 ` Milan Broz
2017-03-17 11:45 ` Milan Broz
1 sibling, 0 replies; 5+ messages in thread
From: Milan Broz @ 2017-03-16 7:15 UTC (permalink / raw)
To: Sami Tolvanen, michal virgovic; +Cc: dm-devel, agk, snitzer
On 03/15/2017 11:12 PM, Sami Tolvanen wrote:
> On Wed, Mar 15, 2017 at 09:22:02PM +0100, michal virgovic wrote:
>> This oops keeps appearing.
>
> This can happen if dm-verity is set up with an invalid root hash, for
> example. Please test the attached patch, which limits recursion and should
> allow it to fail more gracefully.
>
> Sami
Thanks Sami!
I see these patches are already in Android, Why it is not upstream?
https://android.googlesource.com/kernel/goldfish/+/21c0fe9f24b7707d2b49401f8c740c3e35c580ea%5E%21/
We tried to finally implement FEC support in veritysetup and it would
be nice that upstream kernel contains all patches so we do not
need to reinvent the wheel...
Thanks,
Milan
>
> ---
>
> dm verity fec: limit error correction recursion
>
> If the hash tree itself is sufficiently corrupt in addition to data blocks,
> it's possible for error correction to end up in a deep recursive loop,
> which eventually causes a kernel panic. This change limits the recursion to
> a reasonable level during a single I/O operation.
>
> Fixes: a739ff3f543a ("dm verity: add support for forward error correction")
> Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
> ---
> drivers/md/dm-verity-fec.c | 12 +++++++++++-
> drivers/md/dm-verity-fec.h | 4 ++++
> 2 files changed, 15 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/md/dm-verity-fec.c b/drivers/md/dm-verity-fec.c
> index 0f0eb8a3d922..c3cc04d89524 100644
> --- a/drivers/md/dm-verity-fec.c
> +++ b/drivers/md/dm-verity-fec.c
> @@ -439,6 +439,13 @@ int verity_fec_decode(struct dm_verity *v, struct dm_verity_io *io,
> if (!verity_fec_is_enabled(v))
> return -EOPNOTSUPP;
>
> + if (fio->level >= DM_VERITY_FEC_MAX_RECURSION) {
> + DMWARN_LIMIT("%s: FEC: recursion too deep", v->data_dev->name);
> + return -EIO;
> + }
> +
> + fio->level++;
> +
> if (type == DM_VERITY_BLOCK_TYPE_METADATA)
> block += v->data_blocks;
>
> @@ -470,7 +477,7 @@ int verity_fec_decode(struct dm_verity *v, struct dm_verity_io *io,
> if (r < 0) {
> r = fec_decode_rsb(v, io, fio, rsb, offset, true);
> if (r < 0)
> - return r;
> + goto done;
> }
>
> if (dest)
> @@ -480,6 +487,8 @@ int verity_fec_decode(struct dm_verity *v, struct dm_verity_io *io,
> r = verity_for_bv_block(v, io, iter, fec_bv_copy);
> }
>
> +done:
> + fio->level--;
> return r;
> }
>
> @@ -520,6 +529,7 @@ void verity_fec_init_io(struct dm_verity_io *io)
> memset(fio->bufs, 0, sizeof(fio->bufs));
> fio->nbufs = 0;
> fio->output = NULL;
> + fio->level = 0;
> }
>
> /*
> diff --git a/drivers/md/dm-verity-fec.h b/drivers/md/dm-verity-fec.h
> index 7fa0298b995e..bb31ce87a933 100644
> --- a/drivers/md/dm-verity-fec.h
> +++ b/drivers/md/dm-verity-fec.h
> @@ -27,6 +27,9 @@
> #define DM_VERITY_FEC_BUF_MAX \
> (1 << (PAGE_SHIFT - DM_VERITY_FEC_BUF_RS_BITS))
>
> +/* maximum recursion level for verity_fec_decode */
> +#define DM_VERITY_FEC_MAX_RECURSION 4
> +
> #define DM_VERITY_OPT_FEC_DEV "use_fec_from_device"
> #define DM_VERITY_OPT_FEC_BLOCKS "fec_blocks"
> #define DM_VERITY_OPT_FEC_START "fec_start"
> @@ -58,6 +61,7 @@ struct dm_verity_fec_io {
> unsigned nbufs; /* number of buffers allocated */
> u8 *output; /* buffer for corrected output */
> size_t output_pos;
> + unsigned level; /* recursion level */
> };
>
> #ifdef CONFIG_DM_VERITY_FEC
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BUG] Oops caused by FEC in 4.10.0
[not found] ` <20170315221223.GA116381@samitolvanen.mtv.corp.google.com>
2017-03-16 7:15 ` Milan Broz
@ 2017-03-17 11:45 ` Milan Broz
2017-03-17 12:27 ` michal virgovic
[not found] ` <20170317153503.GA52633@samitolvanen.mtv.corp.google.com>
1 sibling, 2 replies; 5+ messages in thread
From: Milan Broz @ 2017-03-17 11:45 UTC (permalink / raw)
To: Sami Tolvanen; +Cc: michal virgovic, dm-devel, agk, snitzer
On 03/15/2017 11:12 PM, Sami Tolvanen wrote:
> On Wed, Mar 15, 2017 at 09:22:02PM +0100, michal virgovic wrote:
>> This oops keeps appearing.
>
> This can happen if dm-verity is set up with an invalid root hash, for
> example. Please test the attached patch, which limits recursion and should
> allow it to fail more gracefully.
I tried it and it still does take at least two minutes and tons of logs until even
udev scan finishes on a device with incorrect FEC attributes.
And once it is finished, dm-verity crashes on dm-device removal...
Could we please have all crucial fixes upstream? It seems that
FEC is really not usable for now.
Milan
This is what I run (32bit VM, Linus' git + your fix), dmesg below.
DEV_NAME=tst12414
DEV_DATA=/dev/sdb
DEV_HASH=/dev/sdc
DEV_FEC=/dev/sdd
dmsetup -r create $DEV_NAME --table "0 8 verity 1 $DEV_DATA $DEV_HASH 4096 4096 1 1 sha256 0000000000000000000000000000000000000000000000000000000000000000 0000000000000000000000000000000000000000000000000000000000000000 8 use_fec_from_device $DEV_FEC fec_roots 16 fec_blocks 1 fec_start 1"
udevadm settle
dd if=/dev/mapper/$DEV_NAME of=/dev/null bs=4k count=1 iflag=direct
dmsetup remove $DEV_NAME
...
Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:34:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:34:59 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:00 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:01 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:02 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:03 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : fec_decode_bufs: 320 callbacks suppressed
Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : buffer_io_error: 66 callbacks suppressed
Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:04 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:05 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:05 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:05 : device-mapper: verity: 8:16: data block 0 is corrupted
Mar 17 12:35:05 : device-mapper: verity: 8:16: reached maximum errors
Mar 17 12:35:09 : fec_decode_bufs: 328 callbacks suppressed
Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:09 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:09 : buffer_io_error: 67 callbacks suppressed
Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:09 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:14 : fec_decode_bufs: 332 callbacks suppressed
Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:14 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:14 : buffer_io_error: 68 callbacks suppressed
Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:14 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:19 : fec_decode_bufs: 332 callbacks suppressed
Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:19 : buffer_io_error: 67 callbacks suppressed
Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:19 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:19 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:24 : fec_decode_bufs: 320 callbacks suppressed
Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:24 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:24 : buffer_io_error: 65 callbacks suppressed
Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:24 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:29 : fec_decode_bufs: 324 callbacks suppressed
Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:29 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:29 : buffer_io_error: 68 callbacks suppressed
Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:29 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:30 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:34 : fec_decode_bufs: 330 callbacks suppressed
Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:34 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:34 : buffer_io_error: 68 callbacks suppressed
Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:34 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:35 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:35 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:39 : fec_decode_bufs: 330 callbacks suppressed
Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:39 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:39 : buffer_io_error: 67 callbacks suppressed
Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:39 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:40 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:40 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:40 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:44 : fec_decode_bufs: 330 callbacks suppressed
Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:44 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:44 : buffer_io_error: 65 callbacks suppressed
Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:44 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:45 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:45 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:49 : fec_decode_bufs: 320 callbacks suppressed
Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:49 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:49 : buffer_io_error: 67 callbacks suppressed
Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:49 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:50 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:50 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:50 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:54 : fec_decode_bufs: 326 callbacks suppressed
Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:54 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:54 : buffer_io_error: 66 callbacks suppressed
Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:54 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:55 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:55 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:55 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:55 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:59 : fec_decode_bufs: 328 callbacks suppressed
Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:35:59 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:35:59 : buffer_io_error: 66 callbacks suppressed
Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:35:59 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:00 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:00 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:00 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:04 : fec_decode_bufs: 326 callbacks suppressed
Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (1 erasures)
Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: corrected 14 errors
Mar 17 12:36:04 : device-mapper: verity-fec: 8:16: FEC 0: failed to correct (0 erasures)
Mar 17 12:36:04 : buffer_io_error: 69 callbacks suppressed
Mar 17 12:36:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:04 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:05 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:05 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:05 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:05 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:05 : Buffer I/O error on dev dm-0, logical block 0, async page read
Mar 17 12:36:05 : ------------[ cut here ]------------
Mar 17 12:36:05 : WARNING: CPU: 0 PID: 1895 at drivers/md/dm-bufio.c:1505 dm_bufio_client_destroy+0x204/0x210 [dm_bufio]
Mar 17 12:36:05 : Modules linked in: dm_verity reed_solomon dm_bufio loop rpcsec_gss_krb5 dm_mod crc32_pclmul crc32c_intel pcbc aesni_intel aes_i586 crypto_simd cryptd ata_piix
Mar 17 12:36:05 : CPU: 0 PID: 1895 Comm: dmsetup Tainted: G W 4.11.0-rc2+ #163
Mar 17 12:36:05 : Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Mar 17 12:36:05 : Call Trace:
Mar 17 12:36:05 : dump_stack+0x60/0x94
Mar 17 12:36:05 : __warn+0xc1/0xe0
Mar 17 12:36:05 : warn_slowpath_null+0x11/0x20
Mar 17 12:36:05 : dm_bufio_client_destroy+0x204/0x210 [dm_bufio]
Mar 17 12:36:05 : verity_fec_dtr+0x4f/0x80 [dm_verity]
Mar 17 12:36:05 : verity_dtr+0x73/0x80 [dm_verity]
Mar 17 12:36:05 : dm_table_destroy+0x51/0xf0 [dm_mod]
Mar 17 12:36:05 : __dm_destroy+0x132/0x220 [dm_mod]
Mar 17 12:36:05 : dm_destroy+0xd/0x10 [dm_mod]
Mar 17 12:36:05 : dev_remove+0xf9/0x150 [dm_mod]
Mar 17 12:36:05 : ? dev_suspend+0x220/0x220 [dm_mod]
Mar 17 12:36:05 : ctl_ioctl+0x1f3/0x4c0 [dm_mod]
Mar 17 12:36:05 : ? dev_suspend+0x220/0x220 [dm_mod]
Mar 17 12:36:05 : ? ctl_ioctl+0x4c0/0x4c0 [dm_mod]
Mar 17 12:36:05 : dm_ctl_ioctl+0xc/0x10 [dm_mod]
Mar 17 12:36:05 : do_vfs_ioctl+0x7d/0x770
Mar 17 12:36:05 : ? mntput_no_expire+0x6e/0x380
Mar 17 12:36:05 : ? SyS_semctl+0x230/0x4b0
Mar 17 12:36:05 : SyS_ioctl+0x28/0x50
Mar 17 12:36:05 : do_int80_syscall_32+0x3e/0x110
Mar 17 12:36:05 : ? quirk_via_cx700_pci_parking_caching+0xb4/0x100
Mar 17 12:36:05 : entry_INT80_32+0x2f/0x2f
Mar 17 12:36:05 : EIP: 0xb770c9f2
Mar 17 12:36:05 : EFLAGS: 00000286 CPU: 0
Mar 17 12:36:05 : EAX: ffffffda EBX: 00000003 ECX: c138fd04 EDX: 800cc910
Mar 17 12:36:05 : ESI: 00000000 EDI: b7701500 EBP: b76f5c0f ESP: bfcb9a34
Mar 17 12:36:05 : DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
Mar 17 12:36:05 : ? quirk_via_cx700_pci_parking_caching+0xb4/0x100
Mar 17 12:36:05 : ---[ end trace f97eb78ad904f683 ]---
Mar 17 12:36:05 : device-mapper: bufio: leaked buffer 10, hold count 2224, list 0
Mar 17 12:36:05 : ------------[ cut here ]------------
Mar 17 12:36:05 : kernel BUG at drivers/md/dm-bufio.c:1521!
Mar 17 12:36:05 : invalid opcode: 0000 [#1] PREEMPT SMP
Mar 17 12:36:05 : Modules linked in: dm_verity reed_solomon dm_bufio loop rpcsec_gss_krb5 dm_mod crc32_pclmul crc32c_intel pcbc aesni_intel aes_i586 crypto_simd cryptd ata_piix
Mar 17 12:36:05 : CPU: 0 PID: 1895 Comm: dmsetup Tainted: G W 4.11.0-rc2+ #163
Mar 17 12:36:05 : Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/02/2015
Mar 17 12:36:05 : task: f5e201c0 task.stack: f3a34000
Mar 17 12:36:05 : EIP: dm_bufio_client_destroy+0x170/0x210 [dm_bufio]
Mar 17 12:36:05 : EFLAGS: 00010297 CPU: 0
Mar 17 12:36:05 : EAX: f5ed0c0c EBX: f5ec0e00 ECX: f5ec0e5c EDX: 00000001
Mar 17 12:36:05 : ESI: f5ec0e64 EDI: 00000002 EBP: f3a35d28 ESP: f3a35d14
Mar 17 12:36:05 : DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Mar 17 12:36:05 : CR0: 80050033 CR2: b74ff4d0 CR3: 346c3000 CR4: 001406d0
Mar 17 12:36:05 : Call Trace:
Mar 17 12:36:05 : verity_fec_dtr+0x4f/0x80 [dm_verity]
Mar 17 12:36:05 : verity_dtr+0x73/0x80 [dm_verity]
Mar 17 12:36:05 : dm_table_destroy+0x51/0xf0 [dm_mod]
Mar 17 12:36:05 : __dm_destroy+0x132/0x220 [dm_mod]
Mar 17 12:36:05 : dm_destroy+0xd/0x10 [dm_mod]
Mar 17 12:36:05 : dev_remove+0xf9/0x150 [dm_mod]
Mar 17 12:36:05 : ? dev_suspend+0x220/0x220 [dm_mod]
Mar 17 12:36:05 : ctl_ioctl+0x1f3/0x4c0 [dm_mod]
Mar 17 12:36:05 : ? dev_suspend+0x220/0x220 [dm_mod]
Mar 17 12:36:05 : ? ctl_ioctl+0x4c0/0x4c0 [dm_mod]
Mar 17 12:36:05 : dm_ctl_ioctl+0xc/0x10 [dm_mod]
Mar 17 12:36:05 : do_vfs_ioctl+0x7d/0x770
Mar 17 12:36:05 : ? mntput_no_expire+0x6e/0x380
Mar 17 12:36:05 : ? SyS_semctl+0x230/0x4b0
Mar 17 12:36:05 : SyS_ioctl+0x28/0x50
Mar 17 12:36:05 : do_int80_syscall_32+0x3e/0x110
Mar 17 12:36:05 : ? quirk_via_cx700_pci_parking_caching+0xb4/0x100
Mar 17 12:36:05 : entry_INT80_32+0x2f/0x2f
Mar 17 12:36:05 : EIP: 0xb770c9f2
Mar 17 12:36:05 : EFLAGS: 00000286 CPU: 0
Mar 17 12:36:05 : EAX: ffffffda EBX: 00000003 ECX: c138fd04 EDX: 800cc910
Mar 17 12:36:05 : ESI: 00000000 EDI: b7701500 EBP: b76f5c0f ESP: bfcb9a34
Mar 17 12:36:05 : DS: 007b ES: 007b FS: 0000 GS: 0033 SS: 007b
Mar 17 12:36:05 : ? quirk_via_cx700_pci_parking_caching+0xb4/0x100
Mar 17 12:36:05 : Code: 0f 85 84 00 00 00 8b 43 68 85 c0 75 5e 8b 83 84 00 00 00 e8 43 18 02 00 89 d8 e8 bc 7e 0e c9 8d 65 f4 5b 5e 5f 5d c3 8d 74 26 00 <0f> 0b 0f 0b 0f 0b 0f 0b 84 c0 74 76 83 e9 0c 57 ff 71 28 89 4d
Mar 17 12:36:05 : EIP: dm_bufio_client_destroy+0x170/0x210 [dm_bufio] SS:ESP: 0068:f3a35d14
Mar 17 12:36:05 : ---[ end trace f97eb78ad904f684 ]---
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BUG] Oops caused by FEC in 4.10.0
2017-03-17 11:45 ` Milan Broz
@ 2017-03-17 12:27 ` michal virgovic
[not found] ` <20170317153503.GA52633@samitolvanen.mtv.corp.google.com>
1 sibling, 0 replies; 5+ messages in thread
From: michal virgovic @ 2017-03-17 12:27 UTC (permalink / raw)
To: Milan Broz; +Cc: dm-devel, agk, snitzer, Sami Tolvanen
[-- Attachment #1: Type: text/plain, Size: 449 bytes --]
2017-03-17 12:45 GMT+01:00 Milan Broz <mbroz@redhat.com>:
> I tried it and it still does take at least two minutes and tons of logs until even
> udev scan finishes on a device with incorrect FEC attributes.
>
> And once it is finished, dm-verity crashes on dm-device removal...
I tried it too, but with correctly computed root-hash, salt,
hash-device, fec-device (using veritysetup) and the outcome is same.
Attachment file is the script i used.
[-- Attachment #2: test_FEC --]
[-- Type: application/octet-stream, Size: 1512 bytes --]
#!/bin/bash -x
# fails on kernel 4.10.0
DEV_NAME=test_device
DEV_DATA=test_data
DEV_DATA_L=/dev/loop0
DEV_HASH=hash_dev
DEV_HASH_L=/dev/loop1
DEV_FEC=fec_dev
DEV_FEC_L=/dev/loop2
# clone cryptsetup
# checkout wip-fec branch and build it
VS_LOCATION=crypts-fec/src/veritysetup
# create data with dd (size=40960000 B)
dd if=/dev/zero of=$DEV_DATA bs=4096 count=10000
# calculate (with veritysetup format) correct root digest and salt, so RS won't repair erasures + hash device will be created + get salt and root hash from output
ARRAY=(`$VS_LOCATION --fec-roots=10 --fec-device=$DEV_FEC format $DEV_DATA $DEV_HASH`)
SALT=${ARRAY[25]}
ROOT_HASH=${ARRAY[28]}
DATA_BLOCKS=${ARRAY[12]}
ARRAY=(`ls -l $DEV_DATA $DEV_HASH`)
TOTAL_BLOCKS=$((((${ARRAY[4]}+${ARRAY[13]})/4096)-1))
losetup $DEV_DATA_L $DEV_DATA
losetup $DEV_HASH_L $DEV_HASH
losetup $DEV_FEC_L $DEV_FEC
# create device with dmsetup with specified table
dmsetup -r create $DEV_NAME --table "0 $(($DATA_BLOCKS*8)) verity 1 $DEV_DATA_L $DEV_HASH_L 4096 4096 $DATA_BLOCKS 0 sha256 $ROOT_HASH $SALT 8 use_fec_from_device $DEV_FEC_L fec_roots 10 fec_blocks $TOTAL_BLOCKS fec_start 0"
###################################################################
dmsetup table $DEV_NAME
dmsetup ls
# wait until udev blkid finishes - thisshould be quick but apparently verity is trying recover more than it should
udevadm settle
# clean
dmsetup remove $DEV_NAME
losetup -d $DEV_DATA_L
losetup -d $DEV_HASH_L
losetup -d $DEV_FEC_L
rm $DEV_HASH $DEV_FEC $DEV_DATA
[-- Attachment #3: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [BUG] Oops caused by FEC in 4.10.0
[not found] ` <20170317153503.GA52633@samitolvanen.mtv.corp.google.com>
@ 2017-03-17 16:29 ` Milan Broz
0 siblings, 0 replies; 5+ messages in thread
From: Milan Broz @ 2017-03-17 16:29 UTC (permalink / raw)
To: Sami Tolvanen; +Cc: michal virgovic, dm-devel, agk, snitzer
On 03/17/2017 04:35 PM, Sami Tolvanen wrote:
> On Fri, Mar 17, 2017 at 12:45:00PM +0100, Milan Broz wrote:
>> it still does take at least two minutes and tons of logs until even
>> udev scan finishes on a device with incorrect FEC attributes.
>
> If dm-verity is set up with invalid arguments so that the entire partition
> is corrupt, it does take a while to determine this with FEC as it attempts
> to correct both metadata and data blocks before giving up. It also depends
> on the random read performance of the underlying hardware obviously.
Sure, no problem with that. Just this testing device was the smallest
possible size (4k) so almost 2 minutes for FEC repairs was quite unexpected :)
>> And once it is finished, dm-verity crashes on dm-device removal...
>
> That's interesting, I haven't seen this before. Looks like something goes
> wrong at the very end of dm_bufio_client_destroy. I'm not familiar enough
> with dm_bufio internals to immediately tell why though.
>
>> Could we please have all crucial fixes upstream?
>
> Sorry, there are no other pending fixes remaining in our tree.
Ok, so it is a new bug then. Sorry for being grumpy, I was just trying
to setup trivial verity device to test activation code in veritysetup
(this part was not in initial commit at all) and every attempt to test
table format ended in kernel OOPs.
We should fix all this problems because it smells like a DoS problem
(someone modifies on-disk FEC data to intentionally crash kernel).
Anyway, Michal (our student intern) should help with testing
(and veritysetup code as well), so if you have any patches, please send
it to the dm-devel list and cc him.
Thanks!
Milan
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2017-03-17 16:29 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-15 20:22 [BUG] Oops caused by FEC in 4.10.0 michal virgovic
[not found] ` <20170315221223.GA116381@samitolvanen.mtv.corp.google.com>
2017-03-16 7:15 ` Milan Broz
2017-03-17 11:45 ` Milan Broz
2017-03-17 12:27 ` michal virgovic
[not found] ` <20170317153503.GA52633@samitolvanen.mtv.corp.google.com>
2017-03-17 16:29 ` Milan Broz
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.