Re: generic/269 hangs on lastest upstream kernel

[Jan Kara <jack@suse.cz>]

On Wed 19-02-20 18:42:36, Yang Xu wrote:
> on 2020/02/19 18:09, Yang Xu wrote:
> > > > > 1) If you mount ext4 with barrier=0 mount option, does the
> > > > > problem go away?
> > > > Yes. Use barrier=0, this case doesn't hang,
> > > 
> > > OK, so there's some problem with how the block layer is handling flush
> > > bios...
> > > 
> > > > > 2) Can you run the test and at the same time run 'blktrace
> > > > > -d /dev/sdc' to
> > > > > gather traces? Once the machine is stuck, abort blktrace, process the
> > > > > resulting files with 'blkparse -i sdc' and send here
> > > > > compressed blkparse
> > > > > output. We should be able to see what was happening with the
> > > > > stuck request
> > > > > in the trace and maybe that will tell us something.
> > > > The log size is too big(58M) and our emali limit is 5M.
> > > 
> > > OK, can you put the log somewhere for download? Alternatively you could
> > > provide only last say 20s of the trace which should hopefully fit
> > > into the
> > > limit...
> > Ok. I will use split command and send you in private to avoid much noise.
> log as attach.

Thanks for the log. So the reason for the hang is clearly visible at the
end of the log:

  8,32   2   104324   164.814457402   995  Q FWS [fsstress]
  8,32   2   104325   164.814458088   995  G FWS [fsstress]
  8,32   2   104326   164.814460957   739  D  FN [kworker/2:1H]

This means, fsstress command has queued cache flush request (from
blkdev_issue_flush()), this has been dispatched to the driver ('D' event)
but it has never been completed by the driver and so blkdev_issue_flush()
never returns.

To debug this further, you probably need to start looking into what happens
with the request inside QEMU. There's not much I can help you with at this
point since I'm not an expert there. Do you use image file as a backing store
or a raw partition?

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR