linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Andrew Morton <akpm@linux-foundation.org>
To: Matthew Wilcox <willy@infradead.org>
Cc: Dave Chinner <david@fromorbit.com>,
	Mel Gorman <mgorman@techsingularity.net>, Jan Kara <jack@suse.cz>,
	Michal Hocko <mhocko@kernel.org>,
	lsf-pc@lists.linux-foundation.org, linux-fsdevel@vger.kernel.org,
	linux-mm@kvack.org, Mel Gorman <mgorman@suse.de>
Subject: Re: [Lsf-pc] [LSF/MM TOPIC] Congestion
Date: Wed, 12 Feb 2020 19:18:10 -0800	[thread overview]
Message-ID: <20200212191810.0b991dcf08138c4170453d6b@linux-foundation.org> (raw)
In-Reply-To: <20200207000853.GD8731@bombadil.infradead.org>

On Thu, 6 Feb 2020 16:08:53 -0800 Matthew Wilcox <willy@infradead.org> wrote:

> On Fri, Feb 07, 2020 at 10:19:28AM +1100, Dave Chinner wrote:
> > But detecting an abundance dirty pages/inodes on the LRU doesn't
> > really solve the problem of determining if and/or how long we should
> > wait for IO before we try to free more objects. There is no problem
> > with having lots of dirty pages/inodes on the LRU as long as the IO
> > subsystem keeps up with the rate at which reclaim is asking them to
> > be written back via async mechanisms (bdi writeback, metadata
> > writeback, etc).
> > 
> > The problem comes when we cannot make efficient progress cleaning
> > pages/inodes on the LRU because the IO subsystem is overloaded and
> > cannot clean pages/inodes any faster. At this point, we have to wait
> > for the IO subsystem to make progress and without feedback from the
> > IO subsystem, we have no idea how fast that progress is made. Hence
> > we have no idea how long we need to wait before trying to reclaim
> > again. i.e. the answer can be different depending on hardware
> > behaviour, not just the current instantaneous reclaim and IO state.
> > 
> > That's the fundamental problem we need to solve, and realistically
> > it can only be done with some level of feedback from the IO
> > subsystem.
> 
> That triggered a memory for me.  Jeremy Kerr presented a paper at LCA2006
> on a different model where the device driver pulls dirty things from the VM
> rather than having the VM push dirty things to the device driver.  It was
> prototyped in K42 rather than Linux, but the idea might be useful.
> 
> http://jk.ozlabs.org/projects/k42/
> http://jk.ozlabs.org/projects/k42/device-driven-IO-lca06.pdf

Fun.  Device drivers says "I have spare bandwidth so send me some stuff".

But if device drivers could do that, we wouldn't have broken congestion
in the first place ;)


      reply	other threads:[~2020-02-13  3:18 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-12-31 12:59 [LSF/MM TOPIC] Congestion Matthew Wilcox
2020-01-04  9:09 ` Dave Chinner
2020-01-06 11:55 ` [Lsf-pc] " Michal Hocko
2020-01-06 23:21   ` Dave Chinner
2020-01-07  8:23     ` Chris Murphy
2020-01-07 11:53       ` Michal Hocko
2020-01-07 20:12         ` Chris Murphy
2020-01-07 11:53     ` Michal Hocko
2020-01-09 11:07     ` Jan Kara
2020-01-09 23:00       ` Dave Chinner
2020-02-05 16:05         ` Mel Gorman
2020-02-06 23:19           ` Dave Chinner
2020-02-07  0:08             ` Matthew Wilcox
2020-02-13  3:18               ` Andrew Morton [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200212191810.0b991dcf08138c4170453d6b@linux-foundation.org \
    --to=akpm@linux-foundation.org \
    --cc=david@fromorbit.com \
    --cc=jack@suse.cz \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=lsf-pc@lists.linux-foundation.org \
    --cc=mgorman@suse.de \
    --cc=mgorman@techsingularity.net \
    --cc=mhocko@kernel.org \
    --cc=willy@infradead.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).