From: Jeff Moyer <jmoyer@redhat.com>
To: Christoph Hellwig <hch@infradead.org>
Cc: "Darrick J. Wong" <darrick.wong@oracle.com>,
axboe@kernel.dk, tytso@mit.edu, david@fromorbit.com, bpm@sgi.com,
viro@zeniv.linux.org.uk, jack@suse.cz,
linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org,
linux-kernel@vger.kernel.org, xfs@oss.sgi.com
Subject: Re: [PATCH 1/9] vfs: Handle O_SYNC AIO DIO in generic code properly
Date: Wed, 21 Nov 2012 11:58:05 -0500 [thread overview]
Message-ID: <x497gpevqoy.fsf@segfault.boston.devel.redhat.com> (raw)
In-Reply-To: <20121121100809.GE23339@infradead.org> (Christoph Hellwig's message of "Wed, 21 Nov 2012 05:08:09 -0500")
Christoph Hellwig <hch@infradead.org> writes:
> On Mon, Nov 19, 2012 at 11:41:23PM -0800, Darrick J. Wong wrote:
>> Provide VFS helpers for handling O_SYNC AIO DIO writes. Filesystems wanting to
>> use the helpers have to pass DIO_SYNC_WRITES to __blockdev_direct_IO. If the
>> filesystem doesn't provide its own direct IO end_io handler, the generic code
>> will take care of issuing the flush. Otherwise, the filesystem's custom end_io
>> handler is passed struct dio_sync_io_work pointer as 'private' argument, and it
>> must call generic_dio_end_io() to finish the AIO DIO. The generic code then
>> takes care to call generic_write_sync() from a workqueue context when AIO DIO
>> is complete.
>>
>> Since all filesystems using blockdev_direct_IO() need O_SYNC aio dio handling
>> and the generic suffices for them, make blockdev_direct_IO() pass the new
>> DIO_SYNC_WRITES flag.
>
> I'd like to use this as a vehicle to revisit how dio completions work.
I don't like the sound of that. ;-) It sounds like this bugfix may get
further delayed by the desire for unrelated code cleanup.
> Now that the generic code has a reason to defer aio completions to a
> workqueue can we maybe take the whole offload to a workqueue code into
> the direct-io code instead of reimplementing it in ext4 and xfs?
On the surface, I don't see a problem with that.
> From a simplicity point of view I'd love to do it unconditionally, but I
> also remember that this was causing performance regressions on important
> workload. So maybe we just need a flag in the dio structure, with a way
> that the get_blocks callback can communicate that it's needed.
Yeah, adding context switches to the normal io completion path is a
non-starter.
> For the specific case of O_(D)SYNC aio this would allos allow to call
> ->fsync from generic code instead of the filesystems having to
> reimplement this.
This is the only reason I'd even consider such a cleanup for this
series. Alas, I don't find it compelling enough to do the work.
>> + if (dio->sync_work)
>> + private = dio->sync_work;
>> + else
>> + private = dio->private;
>> +
>> dio->end_io(dio->iocb, offset, transferred,
>> - dio->private, ret, is_async);
>> + private, ret, is_async);
>
> Eww. I'd be much happier to add a new argument than having two
> different members passed as the private argument.
OK.
> Maybe it's even time to bite the bullet and make struct dio public
> and pass that to the end_io argument as well as generic_dio_end_io.
But I don't agree with that. Really, nothing needs to know about the
struct dio outside of fs/direct-io.c.
>> + /* No IO submitted? Skip syncing... */
>> + if (!dio->result && dio->sync_work) {
>> + kfree(dio->sync_work);
>> + dio->sync_work = NULL;
>> + }
>> + generic_dio_end_io(dio->iocb, offset, transferred,
>> + dio->sync_work, ret, is_async);
>
>
> Any reason the check above isn't done inside of generic_dio_end_io?
Jan? It does seem as though it might make more sense to do the check in
generic_dio_end_io.
>> +static noinline int dio_create_flush_wq(struct super_block *sb)
>> +{
>> + struct workqueue_struct *wq =
>> + alloc_workqueue("dio-sync", WQ_UNBOUND, 1);
>> +
>> + if (!wq)
>> + return -ENOMEM;
>> + /*
>> + * Atomically put workqueue in place. Release our one in case someone
>> + * else won the race and attached workqueue to superblock.
>> + */
>> + if (cmpxchg(&sb->s_dio_flush_wq, NULL, wq))
>> + destroy_workqueue(wq);
>> + return 0;
>
> Eww. Workqueues are cheap, just create it on bootup instead of this
> uglyness. Also I don't really see any reason to make it per-fs instead
> of global.
I would prefer to keep it per-fs. Consider the possibility for sync
work on your database device being backed up behind sync work for your
root file system. So, given my preference to keep it per-fs, would you
rather the workqueues get created at mount time?
Cheers,
Jeff
next prev parent reply other threads:[~2012-11-21 16:58 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-11-20 7:41 [PATCH v1 0/9] fs: fix up AIO+DIO+O_SYNC to actually do the sync part Darrick J. Wong
2012-11-20 7:41 ` [PATCH 1/9] vfs: Handle O_SYNC AIO DIO in generic code properly Darrick J. Wong
2012-11-21 10:08 ` Christoph Hellwig
2012-11-21 16:58 ` Jeff Moyer [this message]
2012-11-21 18:29 ` Christoph Hellwig
2012-11-21 18:38 ` Jeff Moyer
2012-11-21 21:37 ` Jan Kara
2012-11-21 23:09 ` Jeffrey Ellis
2012-11-20 7:41 ` [PATCH 2/9] ext4: honor the O_SYNC flag for aysnchronous direct I/O requests Darrick J. Wong
2012-11-20 10:07 ` Jan Kara
2012-11-20 20:02 ` Jeff Moyer
2012-11-21 0:56 ` Jan Kara
2012-11-21 14:09 ` Jeff Moyer
2012-11-21 16:54 ` Jan Kara
2012-11-20 7:41 ` [PATCH 3/9] xfs: factor out everything but the filemap_write_and_wait from xfs_file_fsync Darrick J. Wong
2012-11-20 10:47 ` Dave Chinner
2012-11-21 10:09 ` Christoph Hellwig
2012-11-21 14:10 ` Jeff Moyer
2012-11-20 7:51 ` [PATCH 7/9] ocfs2: Use generic handlers of O_SYNC AIO DIO Darrick J. Wong
2012-11-21 19:32 ` Joel Becker
2012-11-20 7:51 ` [PATCH 5/9] btrfs: " Darrick J. Wong
2012-11-20 7:51 ` [PATCH 6/9] gfs2: " Darrick J. Wong
2012-11-20 7:51 ` [PATCH 4/9] xfs: honor the O_SYNC flag for aysnchronous direct I/O requests Darrick J. Wong
2012-11-20 10:24 ` Jan Kara
2012-11-20 11:20 ` Dave Chinner
2012-11-20 19:42 ` Jeff Moyer
2012-11-20 20:08 ` Dave Chinner
2012-11-20 7:51 ` [PATCH 8/9] filemap: don't call generic_write_sync for -EIOCBQUEUED Darrick J. Wong
2012-11-20 7:51 ` [PATCH 9/9] blkdev: Fix up AIO+DIO+O_SYNC to do the sync part correctly Darrick J. Wong
2012-11-20 10:15 ` Jan Kara
2012-11-20 20:47 ` Jeff Moyer
2012-11-21 0:57 ` Jan Kara
2012-11-20 8:38 ` [PATCH v1 0/9] fs: fix up AIO+DIO+O_SYNC to actually do the sync part Darrick J. Wong
2012-11-20 14:23 ` Jeff Moyer
2012-11-20 18:57 ` Darrick J. Wong
2012-11-20 19:05 ` Jeff Moyer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=x497gpevqoy.fsf@segfault.boston.devel.redhat.com \
--to=jmoyer@redhat.com \
--cc=axboe@kernel.dk \
--cc=bpm@sgi.com \
--cc=darrick.wong@oracle.com \
--cc=david@fromorbit.com \
--cc=hch@infradead.org \
--cc=jack@suse.cz \
--cc=linux-ext4@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=tytso@mit.edu \
--cc=viro@zeniv.linux.org.uk \
--cc=xfs@oss.sgi.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).