From: Dave Chinner <david@fromorbit.com>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Dave Kleikamp <dave.kleikamp@oracle.com>,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
"Maxim V. Patlasov" <mpatlasov@parallels.com>,
Zach Brown <zab@zabbo.net>
Subject: Re: [PATCH V8 00/33] loop: Issue O_DIRECT aio using bio_vec
Date: Wed, 31 Jul 2013 10:43:03 +1000 [thread overview]
Message-ID: <20130731004303.GA7118@dastard> (raw)
In-Reply-To: <20130730142820.313f1af6827276df31913106@linux-foundation.org>
On Tue, Jul 30, 2013 at 02:28:20PM -0700, Andrew Morton wrote:
> On Thu, 25 Jul 2013 12:50:26 -0500 Dave Kleikamp <dave.kleikamp@oracle.com> wrote:
>
> > This patch series adds a kernel interface to fs/aio.c so that kernel code can
> > issue concurrent asynchronous IO to file systems. It adds an aio command and
> > file system methods which specify io memory with pages instead of userspace
> > addresses.
> >
> > This series was written to reduce the current overhead loop imposes by
> > performing synchronus buffered file system IO from a kernel thread. These
> > patches turn loop into a light weight layer that translates bios into iocbs.
>
> Do you have any performance numbers?
>
> Does anyone care much about loop performance? What's the value here?
Yes. Anyone using loopback devices for file-backed devices exposed
to containers and VMs cares about the memory and CPU overhead
the double caching the existing loop device has.
Or those of us how use loopback devices to examine metadumps from
broken filesystems and run repair/fsck on the filesystem image via
loopback devices. When I'm dealing with images containing tens to
hundreds of gigabytes of metadata, caching a second time in the
backing file is a significant overhead. It's especially annoying
when the application is already using direct IO because the kernel
based block device caching isn't at all efficient and just consumes
needless amount of memory holding on to pages that are never going
to be read again.
And on small memory machines xfstests can trigger OOM killer when it
uses loopback devices in certain tests. Thats generally caused by
the writeback of a dirty page causing a new page to be allocated and
dirtied and so writeback of dirty memory under memory pressure can
a) increase memory usage, and b) increase the percentage of dirty
memory that can't be reclaimed immediately....
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
next prev parent reply other threads:[~2013-07-31 0:43 UTC|newest]
Thread overview: 92+ messages / expand[flat|nested] mbox.gz Atom feed top
2013-07-25 17:50 [PATCH V8 00/33] loop: Issue O_DIRECT aio using bio_vec Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 01/33] iov_iter: move into its own file Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 02/33] iov_iter: iov_iter_copy_from_user() should use non-atomic copy Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 03/33] iov_iter: add copy_to_user support Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 04/33] iov_iter: add __iovec_copy_to_user() Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 05/33] fuse: convert fuse to use iov_iter_copy_[to|from]_user Dave Kleikamp
2013-07-25 17:50 ` Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 06/33] iov_iter: hide iovec details behind ops function pointers Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 07/33] iov_iter: ii_iovec_copy_to_user should pre-fault user pages Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 08/33] iov_iter: add bvec support Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 09/33] iov_iter: add a shorten call Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 10/33] iov_iter: let callers extract iovecs and bio_vecs Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 11/33] dio: Convert direct_IO to use iov_iter Dave Kleikamp
2013-08-23 15:48 ` Geert Uytterhoeven
2013-08-23 15:48 ` Geert Uytterhoeven
2013-07-25 17:50 ` [PATCH V8 12/33] dio: add bio_vec support to __blockdev_direct_IO() Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 13/33] fs: pull iov_iter use higher up the stack Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 14/33] aio: add aio_kernel_() interface Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 15/33] aio: add aio support for iov_iter arguments Dave Kleikamp
2013-08-21 13:55 ` Benjamin LaHaise
2013-08-30 20:05 ` Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 16/33] bio: add bvec_length(), like iov_length() Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 17/33] loop: use aio to perform io on the underlying file Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 18/33] fs: create file_readable() and file_writable() functions Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 19/33] fs: use read_iter and write_iter rather than aio_read and aio_write Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 20/33] fs: add read_iter and write_iter to several file systems Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 21/33] ocfs2: add support for read_iter and write_iter Dave Kleikamp
2013-07-25 17:50 ` [Ocfs2-devel] " Dave Kleikamp
2013-07-25 17:50 ` Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 22/33] ext4: " Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 23/33] nfs: add support for read_iter, write_iter Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 24/33] nfs: simplify swap Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 25/33] btrfs: add support for read_iter and write_iter Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 26/33] block_dev: add support for read_iter, write_iter Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 27/33] xfs: add support for read_iter and write_iter Dave Kleikamp
2013-07-25 17:50 ` Dave Kleikamp
2013-07-26 11:51 ` Dave Chinner
2013-07-26 11:51 ` Dave Chinner
2013-07-25 17:50 ` [PATCH V8 28/33] gfs2: Convert aio_read/write ops to read/write_iter Dave Kleikamp
2013-07-25 17:50 ` [Cluster-devel] " Dave Kleikamp
2013-07-25 17:50 ` Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 29/33] udf: convert file ops from aio_read/write " Dave Kleikamp
2013-07-25 21:34 ` Jan Kara
2013-07-25 17:50 ` [PATCH V8 30/33] afs: add support for read_iter and write_iter Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 31/33] ecrpytfs: Convert aio_read/write ops to read/write_iter Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 32/33] ubifs: convert file ops from aio_read/write " Dave Kleikamp
2013-07-25 17:50 ` Dave Kleikamp
2013-07-25 17:50 ` Dave Kleikamp
2013-07-25 17:50 ` [PATCH V8 33/33] tmpfs: add support for read_iter and write_iter Dave Kleikamp
2013-07-30 21:28 ` [PATCH V8 00/33] loop: Issue O_DIRECT aio using bio_vec Andrew Morton
2013-07-31 0:43 ` Dave Chinner [this message]
2013-07-31 6:40 ` Sedat Dilek
2013-07-31 8:41 ` Sedat Dilek
2013-07-31 11:22 ` Sedat Dilek
2013-07-31 9:51 ` Maxim Patlasov
2013-08-01 8:58 ` Christoph Hellwig
2013-08-01 13:04 ` Dave Kleikamp
2013-08-02 10:48 ` Christoph Hellwig
2013-08-20 13:00 ` Christoph Hellwig
2013-08-20 19:13 ` Dave Kleikamp
2013-08-21 0:14 ` Stephen Rothwell
2013-08-21 5:35 ` Sedat Dilek
2013-08-20 22:46 ` Andrew Morton
2013-08-21 13:02 ` Benjamin LaHaise
2013-08-21 13:02 ` Benjamin LaHaise
2013-08-21 16:30 ` Dave Kleikamp
2013-08-21 16:30 ` Dave Kleikamp
2013-08-21 16:39 ` Benjamin LaHaise
2013-08-21 16:39 ` Benjamin LaHaise
2013-08-21 17:12 ` Dave Kleikamp
2013-08-21 17:12 ` Dave Kleikamp
2013-08-21 19:30 ` Andrew Morton
2013-08-21 19:30 ` Andrew Morton
2013-08-21 20:24 ` Benjamin LaHaise
2013-08-21 20:24 ` Benjamin LaHaise
2013-10-14 15:07 ` Christoph Hellwig
2013-10-14 21:29 ` Benjamin LaHaise
2013-10-14 21:29 ` Benjamin LaHaise
2013-10-15 16:55 ` Christoph Hellwig
2013-10-15 17:14 ` Benjamin LaHaise
2013-10-15 17:14 ` Benjamin LaHaise
2013-10-15 17:18 ` Christoph Hellwig
2013-10-15 17:18 ` Christoph Hellwig
2013-10-15 17:53 ` Dave Kleikamp
2013-10-15 17:53 ` Dave Kleikamp
2014-12-31 20:38 ` Sedat Dilek
2014-12-31 21:52 ` Dave Kleikamp
2014-12-31 22:35 ` Sedat Dilek
2015-01-01 0:52 ` Ming Lei
2015-01-05 19:24 ` Maxim Patlasov
2015-01-06 13:18 ` Ming Lei
2015-01-10 16:51 ` Ming Lei
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20130731004303.GA7118@dastard \
--to=david@fromorbit.com \
--cc=akpm@linux-foundation.org \
--cc=dave.kleikamp@oracle.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mpatlasov@parallels.com \
--cc=zab@zabbo.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.