From: Daniel Black <daniel@mariadb.org>
To: io-uring@vger.kernel.org, linux-btrfs@vger.kernel.org
Subject: btrfs / io-uring corrupting reads
Date: Mon, 21 Feb 2022 16:15:57 +1100 [thread overview]
Message-ID: <CABVffEM0eEWho+206m470rtM0d9J8ue85TtR-A_oVTuGLWFicA@mail.gmail.com> (raw)
Per references at the bottom btfs + iouring manage to corrupt the
reading of a file.
Using podman here however docker or another container runtime will
probably work. As will any MariaDB-10.6 on a distro with a native
liburing userspace. Apologies for the older and bloated container
image.
Reproduction of bug:
using a btrfs:
$ dd if=/dev/zero of=../btrfs.blk bs=1M count=2K
$ sudo losetup --direct-io=on -f ../btrfs.blk
$ sudo mkfs.btrfs /dev/loop6
$ sudo mount /dev/loop/6 /mnt/btrfstest
$ sudo mkdir /mnt/btrfstest/noaio
$ sudo chown dan: /mnt/btrfstest/noaio
Initialize database on directory:
$ podman run --name mdbinit --rm -v
/mnt/btrfstest/noaio/:/var/lib/mysql:Z -e
MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1
quay.io/danielgblack/mariadb-test:10.6-impish-sysbench
--innodb_use_native_aio=0
$ podman kill mdbinit
Switch to using uring to read:
$ podman run --rm -v /mnt/btrfstest/noaio/:/var/lib/mysql:Z -e
MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1
quay.io/danielgblack/mariadb-test:10.6-impish-sysbench
--innodb_use_native_aio=1
Failure observed on startup:
2022-02-21 14:43:31 0 [ERROR] InnoDB: Database page corruption on disk
or a failed read of file './ibdata1' page [page id: space=0, page
number=9]. You may have to recover from a backup.
2022-02-21 14:43:31 0 [Note] InnoDB: Page dump in ascii and hex (16384 bytes):
len 16384; hex 00000000000000092022-02-21 14:43:31 0 [ERROR] InnoDB:
Database page corruption on disk or a failed read of file './ibdata1'
page [page id: space=0, page number=243]. You may have to recover from
a backup.
ffffffffffffff2022-02-21 14:43:31 0 [Note] InnoDB: Page dump in ascii
and hex (16384 bytes):
len 16384; hex
00000000000000f3ffffff0000000000009c2045bf00ffffff0000000000ffffff00000000000000020171000900000000015500020006000700000000000000090000000000000000000000000500000000000000020572000000000000000204b208010000030085690000006e66696d756d000908a0d3000300000803000073757072656d756d000000000013080000100500a000000000000000000000000000000b5359535f464f524549474e18080000180500c0000000000000000c5359ffffff535f464fff0000524549474e5f434f4c531308ffffff0000200500ff0000db000000000000000d5359535f56495254550000000000000056414c00000000000020080000280501030000ffffffff0000ffffffff0000000000000000000e6d7973000000716c2f696e6e6f64625f7461626c655f7300ffffffff0000ffffffff0000000000746174732000ffff080000300501ffff0000ffffffff000005d6692b000000000000000f6d7973716c2f696e6e6f64625f696e6465785fd2007374617473220800003805015500000000000000106d7973716c2f7472616e73616374696f6e5f72656769737472791c0800004005000000f4ffffff74000000000000ffffff00116d79ffffffffffffffffffffffffffffffffffff73716c2f67ffffffffffffffffff7469645f736c6176655fffffffffffffffffff706f7300ffffffffffffff2022-02-21
14:43:31 0 [ERROR] InnoDB: Database page corruption on disk or a
failed read of file './mysql/innodb_table_stats.ibd' page [page id:
space=1, page number=0]. You may have to recover from a backup.
002022-02-21 14:43:31 0 [Note] InnoDB: Page dump in ascii and hex (16384 bytes):
Without --innodb_use_native_aio=0 as a container argument this starts
without error.
$ sudo losetup --direct-io=off -f btrfs.blk also exhibits the failure
Observed failures in:
* 5.17.0-0.rc4.96.fc36.x86_64
* 5.16.8 (on nixos)
* 5.15.6
No observed failure:
* 5.15.14-200.fc35.x86_64
* 5.10
references:
* https://jira.mariadb.org/browse/MDEV-27900
* https://github.com/NixOS/nixpkgs/issues/160516
* https://jira.mariadb.org/browse/MDEV-27449
next reply other threads:[~2022-02-21 5:16 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-02-21 5:15 Daniel Black [this message]
2022-02-21 21:53 ` Fwd: btrfs / io-uring corrupting reads Daniel Black
2022-02-22 11:41 ` Filipe Manana
2022-02-22 12:22 ` Daniel Black
2022-02-22 12:46 ` Filipe Manana
2022-02-22 12:54 ` Filipe Manana
2022-03-01 21:51 ` Daniel Black
2022-03-02 11:28 ` Filipe Manana
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CABVffEM0eEWho+206m470rtM0d9J8ue85TtR-A_oVTuGLWFicA@mail.gmail.com \
--to=daniel@mariadb.org \
--cc=io-uring@vger.kernel.org \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).