linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Hillf Danton <dhillf@gmail.com>
To: Mel Gorman <mgorman@suse.de>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Linux-MM <linux-mm@kvack.org>,
	Linux-Netdev <netdev@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	David Miller <davem@davemloft.net>, Neil Brown <neilb@suse.de>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>
Subject: Re: [PATCH 00/15] Swap-over-NBD without deadlocking V8
Date: Tue, 7 Feb 2012 20:45:18 +0800	[thread overview]
Message-ID: <CAJd=RBAvvzK=TXwDaEjq2t+uEuP2PSi6zaUj7EW4UbL_AUsJAg@mail.gmail.com> (raw)
In-Reply-To: <1328568978-17553-1-git-send-email-mgorman@suse.de>

On Tue, Feb 7, 2012 at 6:56 AM, Mel Gorman <mgorman@suse.de> wrote:
>
> The core issue is that network block devices do not use mempools like normal
> block devices do. As the host cannot control where they receive packets from,
> they cannot reliably work out in advance how much memory they might need.
>
>
> Patch 1 serialises access to min_free_kbytes. It's not strictly needed
>        by this series but as the series cares about watermarks in
>        general, it's a harmless fix. It could be merged independently.
>
>
Any light shed on tuning min_free_kbytes for every day work?


> Patch 2 adds knowledge of the PFMEMALLOC reserves to SLAB and SLUB to
>        preserve access to pages allocated under low memory situations
>        to callers that are freeing memory.
>
> Patch 3 introduces __GFP_MEMALLOC to allow access to the PFMEMALLOC
>        reserves without setting PFMEMALLOC.
>
> Patch 4 opens the possibility for softirqs to use PFMEMALLOC reserves
>        for later use by network packet processing.
>
> Patch 5 ignores memory policies when ALLOC_NO_WATERMARKS is set.
>
> Patches 6-11 allows network processing to use PFMEMALLOC reserves when
>        the socket has been marked as being used by the VM to clean
>        pages. If packets are received and stored in pages that were
>        allocated under low-memory situations and are unrelated to
>        the VM, the packets are dropped.
>
> Patch 12 is a micro-optimisation to avoid a function call in the
>        common case.
>
> Patch 13 tags NBD sockets as being SOCK_MEMALLOC so they can use
>        PFMEMALLOC if necessary.
>
If it is feasible to bypass hang by tuning min_mem_kbytes, things may
become simpler if NICs are also tagged. Sock buffers, pre-allocated if
necessary just after NICs are turned on, are not handed back to kmem
cache but queued on local lists which are maintained by NIC driver, based
the on the info of min_mem_kbytes or similar, for tagged NICs.
Upside is no changes in VM core. Downsides?


> Patch 14 notes that it is still possible for the PFMEMALLOC reserve
>        to be depleted. To prevent this, direct reclaimers get
>        throttled on a waitqueue if 50% of the PFMEMALLOC reserves are
>        depleted.  It is expected that kswapd and the direct reclaimers
>        already running will clean enough pages for the low watermark
>        to be reached and the throttled processes are woken up.
>
> Patch 15 adds a statistic to track how often processes get throttled
>
>
> For testing swap-over-NBD, a machine was booted with 2G of RAM with a
> swapfile backed by NBD. 8*NUM_CPU processes were started that create
> anonymous memory mappings and read them linearly in a loop. The total
> size of the mappings were 4*PHYSICAL_MEMORY to use swap heavily under
> memory pressure. Without the patches, the machine locks up within
> minutes and runs to completion with them applied.
>
>
While testing, what happens if the network wire is plugged off over
three minutes?

Thanks
Hillf

  parent reply	other threads:[~2012-02-07 12:45 UTC|newest]

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-02-06 22:56 [PATCH 00/15] Swap-over-NBD without deadlocking V8 Mel Gorman
2012-02-06 22:56 ` [PATCH 01/15] mm: Serialize access to min_free_kbytes Mel Gorman
2012-02-08 18:47   ` Rik van Riel
2012-02-06 22:56 ` [PATCH 02/15] mm: sl[au]b: Add knowledge of PFMEMALLOC reserve pages Mel Gorman
2012-02-07 16:27   ` Christoph Lameter
2012-02-08 14:45     ` Mel Gorman
2012-02-08 15:14       ` Christoph Lameter
2012-02-08 16:34         ` Mel Gorman
2012-02-08 19:49           ` Christoph Lameter
2012-02-08 21:23             ` Mel Gorman
2012-02-08 22:13               ` Christoph Lameter
2012-02-09 12:50                 ` Mel Gorman
2012-02-09 19:53                   ` Christoph Lameter
2012-02-10 10:26                     ` Mel Gorman
2012-02-10 21:01                       ` Christoph Lameter
2012-02-10 22:07                         ` Christoph Lameter
2012-02-13 10:12                           ` Mel Gorman
2012-02-13 11:10                         ` Mel Gorman
2012-02-06 22:56 ` [PATCH 03/15] mm: Introduce __GFP_MEMALLOC to allow access to emergency reserves Mel Gorman
2012-02-06 22:56 ` [PATCH 04/15] mm: allow PF_MEMALLOC from softirq context Mel Gorman
2012-02-06 22:56 ` [PATCH 05/15] mm: Ignore mempolicies when using ALLOC_NO_WATERMARK Mel Gorman
2012-02-06 22:56 ` [PATCH 06/15] net: Introduce sk_allocation() to allow addition of GFP flags depending on the individual socket Mel Gorman
2012-02-06 22:56 ` [PATCH 07/15] netvm: Allow the use of __GFP_MEMALLOC by specific sockets Mel Gorman
2012-02-06 22:56 ` [PATCH 08/15] netvm: Allow skb allocation to use PFMEMALLOC reserves Mel Gorman
2012-02-06 22:56 ` [PATCH 09/15] netvm: Propagate page->pfmemalloc to skb Mel Gorman
2012-02-06 22:56 ` [PATCH 10/15] netvm: Propagate page->pfmemalloc from netdev_alloc_page " Mel Gorman
2012-02-07 23:38   ` Alexander Duyck
2012-02-08 15:23     ` Mel Gorman
2012-02-06 22:56 ` [PATCH 11/15] netvm: Set PF_MEMALLOC as appropriate during SKB processing Mel Gorman
2012-02-06 22:56 ` [PATCH 12/15] mm: Micro-optimise slab to avoid a function call Mel Gorman
2012-02-06 22:56 ` [PATCH 13/15] nbd: Set SOCK_MEMALLOC for access to PFMEMALLOC reserves Mel Gorman
2012-02-06 22:56 ` [PATCH 14/15] mm: Throttle direct reclaimers if PF_MEMALLOC reserves are low and swap is backed by network storage Mel Gorman
2012-02-06 22:56 ` [PATCH 15/15] mm: Account for the number of times direct reclaimers get throttled Mel Gorman
2012-02-07 12:45 ` Hillf Danton [this message]
2012-02-07 13:27   ` [PATCH 00/15] Swap-over-NBD without deadlocking V8 Mel Gorman
2012-02-08 12:51     ` Hillf Danton
2012-02-08 15:26       ` Mel Gorman

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAJd=RBAvvzK=TXwDaEjq2t+uEuP2PSi6zaUj7EW4UbL_AUsJAg@mail.gmail.com' \
    --to=dhillf@gmail.com \
    --cc=a.p.zijlstra@chello.nl \
    --cc=akpm@linux-foundation.org \
    --cc=davem@davemloft.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=neilb@suse.de \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).