From: Jan Kara <jack@suse.cz>
To: Mingming Cao <cmm@us.ibm.com>
Cc: Jan Kara <jack@suse.cz>, Badari Pulavarty <pbadari@us.ibm.com>,
akpm@linux-foundation.org, linux-ext4@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH] jbd_commit_transaction() races with journal_try_to_drop_buffers() causing DIO failures
Date: Wed, 14 May 2008 19:08:56 +0200 [thread overview]
Message-ID: <20080514170856.GH24363@duck.suse.cz> (raw)
In-Reply-To: <1210717389.3638.24.camel@localhost.localdomain>
On Tue 13-05-08 15:23:09, Mingming Cao wrote:
> On Tue, 2008-05-13 at 16:54 +0200, Jan Kara wrote:
> > On Mon 12-05-08 17:39:43, Mingming Cao wrote:
> > > On Mon, 2008-05-12 at 17:54 +0200, Jan Kara wrote:
> > > Does this match what you are thinking? It certainly slow down the DIO
> > > path, but the positive side is it doesn't disturb the other code path...
> > > thanks for your feedback!
> > >
> > > --------------------------------------------
> > >
> > > An unexpected EIO error gets returned when writing to a file
> > > using buffered writes and DIO writes at the same time.
> > >
> > > We found there are a number of places where journal_try_to_free_buffers()
> > > could race with journal_commit_transaction(), the later still
> > > helds the reference to the buffers on the t_syncdata_list or t_locked_list
> > > , while journal_try_to_free_buffers() tries to free them, which resulting an EIO
> > > error returns back to the dio caller.
> > >
> > > The logic fix is to retry freeing if journal_try_to_free_buffers() to failed
> > > to free those data buffers while journal_commit_transaction() is still
> > > reference those buffers.
> > > This is done via implement ext3 launder_page() callback, instead of inside
> > > journal_try_to_free_buffers() itself, so that it doesn't affecting other code
> > > path calling journal_try_to_free_buffers and only dio path get affected.
> > >
> > > Signed-off-by: Mingming Cao <cmm@us.ibm.com>
> > > Index: linux-2.6.26-rc1/fs/ext3/inode.c
> > > ===================================================================
> > > --- linux-2.6.26-rc1.orig/fs/ext3/inode.c 2008-05-03 11:59:44.000000000 -0700
> > > +++ linux-2.6.26-rc1/fs/ext3/inode.c 2008-05-12 12:41:27.000000000 -0700
> > > @@ -1766,6 +1766,23 @@ static int ext3_journalled_set_page_dirt
> > > return __set_page_dirty_nobuffers(page);
> > > }
> > >
> > > +static int ext3_launder_page(struct page *page)
> > > +{
> > > + int ret;
> > > + int retry = 5;
> > > +
> > > + while (retry --) {
> > > + ret = ext3_releasepage(page, GFP_KERNEL);
> > > + if (ret == 1)
> > > + break;
> > > + else
> > > + schedule();
> > > + }
> > > +
> > > + return ret;
> > > +}
> > > +
> > > +
> > Yes, I meant something like this. We could be more clever and do:
> >
> > head = bh = page_buffers(page);
> > do {
> > wait_on_buffer(bh);
> > bh = bh->b_this_page;
> > } while (bh != head);
> > /*
> > * Now commit code should have been able to proceed and release
> > * those buffers
> > */
> > schedule();
> >
>
> Bummer, we can't free buffers in ext3_launder_page() before calling
> try_to_free_page, as later
> invalidate_complete_page2()->try_to_free_page() expecting the page
> buffers are still here, and will return EIO if it launder_page() has
> already freed those buffers.:(
Are you sure? Because if bufferes are released in ext3_launder_page(),
PagePrivate() has been set to 0 and we should directly fall through to
releasing the page without ever calling try_to_release_page()... So I'd
want to find out why PagePrivate is still set in
invalidate_complete_page2().
> Doing wait_on_buffer() alone in launder_page() is not enough as it
> doesn't wait for buffer reference drop to 0.
Yes, this would not be enough.
Honza
--
Jan Kara <jack@suse.cz>
SUSE Labs, CR
next prev parent reply other threads:[~2008-05-14 17:09 UTC|newest]
Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-03-06 17:42 [RFC] JBD ordered mode rewrite Jan Kara
2008-03-06 19:05 ` Josef Bacik
2008-03-10 16:30 ` Jan Kara
2008-03-06 23:53 ` Andrew Morton
2008-03-10 17:38 ` Jan Kara
2008-03-07 1:34 ` Mark Fasheh
2008-03-10 18:00 ` Jan Kara
2008-03-07 10:55 ` Mingming Cao
2008-03-10 18:29 ` Jan Kara
2008-03-07 23:52 ` Andreas Dilger
2008-03-08 0:08 ` Mingming Cao
2008-03-08 12:14 ` Christoph Hellwig
2008-03-10 19:54 ` Jan Kara
2008-03-10 21:37 ` Andreas Dilger
2008-04-25 23:38 ` Possible race between direct IO and JBD? Mingming Cao
2008-04-26 10:41 ` Andrew Morton
2008-04-28 12:26 ` Jan Kara
2008-04-28 17:11 ` Badari Pulavarty
2008-04-28 18:09 ` Jan Kara
2008-04-28 19:09 ` Mingming Cao
2008-04-29 12:43 ` Jan Kara
2008-04-29 17:49 ` Mingming Cao
2008-05-01 15:16 ` [PATCH] jbd_commit_transaction() races with journal_try_to_drop_buffers() causing DIO failures Badari Pulavarty
2008-05-01 22:08 ` Mingming Cao
2008-05-05 17:06 ` Jan Kara
2008-05-05 17:53 ` Mingming Cao
2008-05-06 0:10 ` Badari Pulavarty
2008-05-09 22:27 ` Mingming Cao
2008-05-12 15:54 ` Jan Kara
2008-05-12 19:23 ` Mingming Cao
2008-05-13 14:20 ` Jan Kara
2008-05-13 0:39 ` Mingming Cao
2008-05-13 14:54 ` Jan Kara
2008-05-13 16:37 ` Mingming Cao
2008-05-13 22:23 ` Mingming Cao
2008-05-14 17:08 ` Jan Kara [this message]
2008-05-14 17:41 ` Mingming Cao
2008-05-14 18:14 ` Jan Kara
2008-05-16 14:13 ` Mingming Cao
2008-05-16 14:14 ` [PATCH] Fix DIO EIO error caused by race between jbd_commit_transaction() and journal_try_to_drop_buffers() Mingming Cao
2008-05-16 15:01 ` Josef Bacik
2008-05-16 17:11 ` Mingming Cao
2008-05-16 17:17 ` Badari Pulavarty
2008-05-16 17:30 ` Mingming Cao
2008-05-16 17:12 ` Badari Pulavarty
2008-05-16 21:01 ` [PATCH] JBD: Fix DIO EIO error caused by race between free buffer and commit trasanction Mingming Cao
2008-05-18 22:37 ` Jan Kara
2008-05-19 19:59 ` Mingming Cao
2008-05-19 20:25 ` Andrew Morton
2008-05-19 22:07 ` Mingming Cao
2008-05-20 9:30 ` Jens Axboe
2008-05-20 17:47 ` Mingming Cao
2008-05-20 18:02 ` [PATCH-v2] JBD: Fix " Mingming Cao
2008-05-20 23:53 ` Jan Kara
2008-05-21 17:14 ` Mingming
2008-05-24 22:44 ` Jan Kara
2008-05-28 18:18 ` Mingming Cao
2008-05-28 18:55 ` Jan Kara
2008-05-29 0:15 ` Mingming Cao
2008-05-29 0:16 ` [PATCH][take 5] " Mingming Cao
2008-05-29 0:18 ` [PATCH][take 5] JBD2: " Mingming Cao
2008-05-30 6:24 ` Aneesh Kumar K.V
2008-05-30 15:17 ` Mingming Cao
2008-05-21 23:38 ` [PATCH 1/2][TAKE3] JBD: " Mingming
2008-05-22 5:57 ` Andrew Morton
2008-05-21 23:39 ` [PATCH 2/2][TAKE3] JBD2: " Mingming
2008-05-20 18:03 ` [PATCH -v2] JBD2: Fix race between journal " Mingming Cao
2008-05-16 21:01 ` [PATCH] JBD2: Fix DIO EIO error caused by race between " Mingming Cao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20080514170856.GH24363@duck.suse.cz \
--to=jack@suse.cz \
--cc=akpm@linux-foundation.org \
--cc=cmm@us.ibm.com \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=pbadari@us.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).