Re: [REGRESSION] v5.17-rc1+: FIFREEZE ioctl system call hangs

From: Song Liu <song@kernel.org>
To: Thomas Deutschmann <whissi@whissi.de>
Cc: Vishal Verma <vverma@digitalocean.com>,
	Thorsten Leemhuis <regressions@leemhuis.info>,
	 "stable@vger.kernel.org" <stable@vger.kernel.org>,
	 "regressions@lists.linux.dev" <regressions@lists.linux.dev>,
	Jens Axboe <axboe@kernel.dk>
Subject: Re: [REGRESSION] v5.17-rc1+: FIFREEZE ioctl system call hangs
Date: Mon, 22 Aug 2022 14:52:28 -0700	[thread overview]
Message-ID: <CAPhsuW6Bq8rkiCzsWW7bkrCbYYEwFWtKaswOZcXsyk8tu3C5Dg@mail.gmail.com> (raw)
In-Reply-To: <4f69659f-7160-7854-0ed5-6867e3eb2edb@whissi.de>

On Mon, Aug 22, 2022 at 9:30 AM Thomas Deutschmann <whissi@whissi.de> wrote:
>
> Hi,
>
> I can now reproduce using fio:
>
> I looked around in MariaDB issue tracker and found
> https://jira.mariadb.org/browse/MDEV-26674 which lead me to
> https://github.com/MariaDB/server/commit/de7db5517de11a58d57d2a41d0bc6f38b6f92dd8
> -- it's a conditional based on $KV and I hit that kernel regression
> during one of my bisect attempts (see
> https://lore.kernel.org/all/701f3fc0-2f0c-a32c-0d41-b489a9a59b99@whissi.de/).
>
> Setting innodb_use_native_aio=OFF will prevent the problem.
>
> This helped me to find https://github.com/axboe/fio/issues/1195 so I now
> have a working reproducer for fio.
>
>    $ cat reproducer.fio
>    [global]
>    direct=1
>    thread=1
>    norandommap=1
>    group_reporting=1
>    time_based=1
>    ioengine=io_uring
>
>    rw=randwrite
>    bs=4096
>    runtime=20
>    numjobs=1
>    fixedbufs=1
>    hipri=1
>    registerfiles=1
>    sqthread_poll=1
>
>
>    [filename0]
>    directory=/srv/machines/fio
>    size=200M
>    iodepth=1
>    cpus_allowed=20
>
>
> ...now call fio like "fio reproducer.fio". After one successful fio run,
> fsfreeze will already hang for me.

Hmm.. I still cannot repro the hang in my test. I have:

[root@eth50-1 ~]# mount | grep mnt
/dev/md0 on /root/mnt type ext4 (rw,relatime,stripe=384)
[root@eth50-1 ~]# lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sr0      11:0    1 1024M  0 rom
vda     253:0    0   32G  0 disk
├─vda1  253:1    0    2G  0 part  /boot
└─vda2  253:2    0   30G  0 part  /
nvme0n1 259:0    0    4G  0 disk
└─md0     9:0    0   12G  0 raid5 /root/mnt
nvme2n1 259:1    0    4G  0 disk
└─md0     9:0    0   12G  0 raid5 /root/mnt
nvme3n1 259:2    0    4G  0 disk
└─md0     9:0    0   12G  0 raid5 /root/mnt
nvme1n1 259:3    0    4G  0 disk
└─md0     9:0    0   12G  0 raid5 /root/mnt

[root@eth50-1 ~]# history
  381  fio iou/repro.fio
  382  fsfreeze --freeze /root/mnt
  383  fsfreeze --unfreeze /root/mnt
  384  fio iou/repro.fio
  385  fsfreeze --freeze /root/mnt
  386  fsfreeze --unfreeze /root/mnt
^^^^^^^^^^^^^^ all works fine.

Did I miss something?

Thanks,
Song