From: Mikulas Patocka <mikulas@artax.karlin.mff.cuni.cz>
To: linux-kernel@vger.kernel.org
Subject: Temporary lockup on loopback block device
Date: Sat, 10 Nov 2007 20:51:31 +0100 (CET) [thread overview]
Message-ID: <Pine.LNX.4.64.0711102025440.17691@artax.karlin.mff.cuni.cz> (raw)
Hi
I am experiencing a transient lockup in 'D' state with loopback device. It
happens when process writes to a filesystem in loopback with command like
dd if=/dev/zero of=/s/fill bs=4k
CPU is idle, disk is idle too, yet the dd process is waiting in 'D' in
congestion_wait called from balance_dirty_pages.
After about 30 seconds, the lockup is gone and dd resumes, but it locks up
soon again.
I added a printk to the balance_dirty_pages
printk("wait: nr_reclaimable %d, nr_writeback %d, dirty_thresh %d,
pages_written %d, write_chunk %d\n", nr_reclaimable,
global_page_state(NR_WRITEBACK), dirty_thresh, pages_written,
write_chunk);
and it shows this during the lockup:
wait: nr_reclaimable 3099, nr_writeback 0, dirty_thresh 2985,
pages_written 1021, write_chunk 1522
wait: nr_reclaimable 3099, nr_writeback 0, dirty_thresh 2985,
pages_written 1021, write_chunk 1522
wait: nr_reclaimable 3099, nr_writeback 0, dirty_thresh 2985,
pages_written 1021, write_chunk 1522
What apparently happens:
writeback_inodes syncs inodes only on the given wbc->bdi, however
balance_dirty_pages checks against global counts of dirty pages. So if
there's nothing to sync on a given device, but there are other dirty pages
so that the counts are over the limit, it will loop without doing any
work.
To reproduce it, you need totally idle machine (no GUI, etc.) -- if
something writes to the backing device, it flushes the dirty pages
generated by the loopback and the lockup is gone. If you add printk, don't
forget to stop klogd, otherwise logging would end the lockup.
The hotfix (that I verified to work) is to not set wbc->bdi, so that all
devices are flushed ... but the code probably needs some redesign (i.e.
either account per-device and flush per-device, or account-global and
flush-global).
Mikulas
diff -u -r ../x/linux-2.6.23.1/mm/page-writeback.c mm/page-writeback.c
--- ../x/linux-2.6.23.1/mm/page-writeback.c 2007-10-12 18:43:44.000000000 +0200
+++ mm/page-writeback.c 2007-11-10 20:32:43.000000000 +0100
@@ -214,7 +214,6 @@
for (;;) {
struct writeback_control wbc = {
- .bdi = bdi,
.sync_mode = WB_SYNC_NONE,
.older_than_this = NULL,
.nr_to_write = write_chunk,
next reply other threads:[~2007-11-10 19:51 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-11-10 19:51 Mikulas Patocka [this message]
2007-11-10 22:54 ` Temporary lockup on loopback block device Andrew Morton
2007-11-10 23:02 ` Peter Zijlstra
2007-11-11 0:38 ` Mikulas Patocka
2007-11-11 7:50 ` Miklos Szeredi
2007-11-11 18:29 ` Mikulas Patocka
2007-11-12 13:32 ` Miklos Szeredi
2007-11-15 22:35 ` Mikulas Patocka
2007-11-11 0:33 ` Mikulas Patocka
2007-11-11 3:56 ` Mikulas Patocka
2007-11-11 5:33 ` Mikulas Patocka
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Pine.LNX.4.64.0711102025440.17691@artax.karlin.mff.cuni.cz \
--to=mikulas@artax.karlin.mff.cuni.cz \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).