From: Song Liu <song@kernel.org>
To: Vishal Verma <vverma@digitalocean.com>
Cc: Thorsten Leemhuis <regressions@leemhuis.info>,
stable@vger.kernel.org, regressions@lists.linux.dev,
Thomas Deutschmann <whissi@whissi.de>,
Jens Axboe <axboe@kernel.dk>
Subject: Re: [REGRESSION] v5.17-rc1+: FIFREEZE ioctl system call hangs
Date: Tue, 16 Aug 2022 23:19:02 -0700 [thread overview]
Message-ID: <CAPhsuW5f9QD+gzJ9eBhn5irsHvrsvkWjSnA4MPaHsQjjLMypXg@mail.gmail.com> (raw)
In-Reply-To: <0FBCAB10-545E-45E2-A0C8-D7620817651D@digitalocean.com>
On Mon, Aug 15, 2022 at 8:46 AM Vishal Verma <vverma@digitalocean.com> wrote:
>
> Just saw this. I’m trying to understand whether this happens only on md array or individual nvme drives (without any raid) too?
> The commit you pointed added REQ_NOWAIT for md based arrays, but if it is happening on individual nvme drives then that could point to something with REQ_NOWAIT I think.
Agreed with this analysis.
>
> > On Aug 15, 2022, at 3:58 AM, Thorsten Leemhuis <regressions@leemhuis.info> wrote:
> >
> > Hi, this is your Linux kernel regression tracker. Top-posting for once,
> > to make this easily accessible to everyone.
> >
> > [CCing Jens, as the top-level maintainer who in this case also reviewed
> > the patch that causes this regression.]
> >
> > Vishal, Song, what up here? Could you please look into this and at least
> > comment on the issue, as it's a regression that was reported more than
> > 10 days ago already. Ideally at this point it would be good if the
> > regression was fixed already, as explained by "Prioritize work on fixing
> > regressions" here:
> > https://docs.kernel.org/process/handling-regressions.html#prioritize-work-on-fixing-regressions
I am sorry for the delay.
[...]
> >>
> >> Hi,
> >>
> >> any news on this? Is there anything else you need from me or I can help
> >> with?
> >>
> >> Thanks.
> >>
> >>
> >> -- Regards, Thomas -----Original Message----- From: Thomas Deutschmann
> >> <whissi@whissi.de> Sent: Wednesday, August 3, 2022 4:35 PM To:
> >> vverma@digitalocean.com; song@kernel.org Cc: stable@vger.kernel.org;
> >> regressions@lists.linux.dev Subject: [REGRESSION] v5.17-rc1+: FIFREEZE
> >> ioctl system call hangs Hi, while trying to backup a Dell R7525 system
> >> running Debian bookworm/testing using LVM snapshots I noticed that the
> >> system will 'freeze' sometimes (not all the times) when creating the
> >> snapshot. First I thought this was related to LVM so I created
> >> https://listman.redhat.com/archives/linux-lvm/2022-July/026228.html
> >> (continued at
> >> https://listman.redhat.com/archives/linux-lvm/2022-August/thread.html#26229) Long story short: I was even able to reproduce with fsfreeze, see last strace lines
> >>> [...]
> >>> 14471 1659449870.984635 openat(AT_FDCWD, "/var/lib/machines", O_RDONLY) =3
> >>> 14471 1659449870.984658 newfstatat(3, "",
> >> {st_mode=S_IFDIR|0700,st_size=4096, ...}, AT_EMPTY_PATH) = 0
> >>> 14471 1659449870.984678 ioctl(3, FIFREEZE
> >> so I started to bisect kernel and found the following bad commit:
I am not able to reproduce this on 5.19+ kernel. I have:
[root@eth50-1 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
vda 253:0 0 32G 0 disk
├─vda1 253:1 0 2G 0 part /boot
└─vda2 253:2 0 30G 0 part /
nvme0n1 259:0 0 4G 0 disk
└─md0 9:0 0 12G 0 raid5 /root/mnt
nvme2n1 259:1 0 4G 0 disk
└─md0 9:0 0 12G 0 raid5 /root/mnt
nvme3n1 259:2 0 4G 0 disk
└─md0 9:0 0 12G 0 raid5 /root/mnt
nvme1n1 259:3 0 4G 0 disk
└─md0 9:0 0 12G 0 raid5 /root/mnt
[root@eth50-1 ~]# for x in {1..100} ; do fsfreeze --unfreeze /root/mnt
; fsfreeze --freeze /root/mnt ; done
Did I miss something?
Thanks,
Song
[...]
next prev parent reply other threads:[~2022-08-17 6:19 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-03 14:35 [REGRESSION] v5.17-rc1+: FIFREEZE ioctl system call hangs Thomas Deutschmann
2022-08-11 12:34 ` Thomas Deutschmann
2022-08-15 10:58 ` Thorsten Leemhuis
2022-08-15 15:46 ` Vishal Verma
2022-08-17 6:19 ` Song Liu [this message]
2022-08-17 6:53 ` Thomas Deutschmann
2022-08-17 18:29 ` Thomas Deutschmann
2022-08-19 2:46 ` Thomas Deutschmann
2022-08-20 1:04 ` Song Liu
2022-08-22 15:29 ` Thomas Deutschmann
2022-08-22 16:30 ` Thomas Deutschmann
2022-08-22 21:52 ` Song Liu
2022-08-22 22:44 ` Thomas Deutschmann
2022-08-22 22:59 ` Song Liu
2022-08-23 1:37 ` Song Liu
2022-08-23 3:15 ` Thomas Deutschmann
2022-08-23 17:13 ` Song Liu
2022-08-25 16:47 ` Song Liu
2022-08-25 19:12 ` Jens Axboe
2022-08-25 22:24 ` Song Liu
2022-08-26 20:10 ` Thomas Deutschmann
2022-09-08 13:25 ` [REGRESSION] v5.17-rc1+: FIFREEZE ioctl system call hangs #forregzbot Thorsten Leemhuis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAPhsuW5f9QD+gzJ9eBhn5irsHvrsvkWjSnA4MPaHsQjjLMypXg@mail.gmail.com \
--to=song@kernel.org \
--cc=axboe@kernel.dk \
--cc=regressions@leemhuis.info \
--cc=regressions@lists.linux.dev \
--cc=stable@vger.kernel.org \
--cc=vverma@digitalocean.com \
--cc=whissi@whissi.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).