All of lore.kernel.org
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@kernel.org>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Vlastimil Babka <vbabka@suse.cz>,
	David Rientjes <rientjes@google.com>,
	Mel Gorman <mgorman@suse.de>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Al Viro <viro@zeniv.linux.org.uk>,
	linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>,
	Alexei Starovoitov <ast@kernel.org>,
	Anatoly Stepanov <astepanov@cloudlinux.com>,
	Andreas Dilger <adilger@dilger.ca>,
	Andreas Dilger <andreas.dilger@intel.com>,
	Anton Vorontsov <anton@enomsg.org>,
	Ben Skeggs <bskeggs@redhat.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Colin Cross <ccross@android.com>,
	Dan Williams <dan.j.williams@intel.com>,
	David Sterba <dsterba@suse.com>,
	Eric Dumazet <edumazet@google.com>,
	Eric Dumazet <eric.dumazet@gmail.com>,
	Hariprasad S <hariprasad@chelsio.com>,
	Heiko Carstens <heiko.carstens@de.ibm.com>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	Ilya Dryomov <idryomov@gmail.com>,
	Kees Cook <keescook@chromium.org>,
	Kent Overstreet <kent.overstreet@gmail.com>,
	Martin Schwidefsky <schwidefsky@de.ibm.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Mike Snitzer <snitzer@redhat.com>,
	Oleg Drokin <oleg.drokin@intel.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	Santosh Raspatur <santosh@chelsio.com>,
	Tariq Toukan <tariqt@mellanox.com>,
	"Theodore Ts'o" <tytso@mit.edu>,
	Tom Herbert <tom@herbertland.com>,
	Tony Luck <tony.luck@intel.com>, "Yan, Zheng" <zyan@redhat.com>,
	Yishai Hadas <yishaih@mellanox.com>,
	Daniel Borkmann <daniel@iogearbox.net>
Subject: Re: [PATCH 0/6 v3] kvmalloc
Date: Wed, 25 Jan 2017 14:10:07 +0100	[thread overview]
Message-ID: <20170125131006.GQ32377@dhcp22.suse.cz> (raw)
In-Reply-To: <20170124191716.GA23114@ast-mbp.thefacebook.com>

On Tue 24-01-17 11:17:21, Alexei Starovoitov wrote:
> On Tue, Jan 24, 2017 at 04:17:52PM +0100, Michal Hocko wrote:
> > On Thu 12-01-17 16:37:11, Michal Hocko wrote:
> > > Hi,
> > > this has been previously posted as a single patch [1] but later on more
> > > built on top. It turned out that there are users who would like to have
> > > __GFP_REPEAT semantic. This is currently implemented for costly >64B
> > > requests. Doing the same for smaller requests would require to redefine
> > > __GFP_REPEAT semantic in the page allocator which is out of scope of
> > > this series.
> > > 
> > > There are many open coded kmalloc with vmalloc fallback instances in
> > > the tree.  Most of them are not careful enough or simply do not care
> > > about the underlying semantic of the kmalloc/page allocator which means
> > > that a) some vmalloc fallbacks are basically unreachable because the
> > > kmalloc part will keep retrying until it succeeds b) the page allocator
> > > can invoke a really disruptive steps like the OOM killer to move forward
> > > which doesn't sound appropriate when we consider that the vmalloc
> > > fallback is available.
> > > 
> > > As it can be seen implementing kvmalloc requires quite an intimate
> > > knowledge if the page allocator and the memory reclaim internals which
> > > strongly suggests that a helper should be implemented in the memory
> > > subsystem proper.
> > > 
> > > Most callers I could find have been converted to use the helper instead.
> > > This is patch 5. There are some more relying on __GFP_REPEAT in the
> > > networking stack which I have converted as well but considering we do
> > > not have a support for __GFP_REPEAT for requests smaller than 64kB I
> > > have marked it RFC.
> > 
> > Are there any more comments? I would really appreciate to hear from
> > networking folks before I resubmit the series.
> 
> while this patchset was baking the bpf side switched to use bpf_map_area_alloc()
> which fixes the issue with missing __GFP_NORETRY that we had to fix quickly.
> See commit d407bd25a204 ("bpf: don't trigger OOM killer under pressure with map alloc")
> it covers all kmalloc/vmalloc pairs instead of just one place as in this set.
> So please rebase and switch bpf_map_area_alloc() to use kvmalloc().

OK, will do. Thanks for the heads up.


-- 
Michal Hocko
SUSE Labs

WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko@kernel.org>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	Vlastimil Babka <vbabka@suse.cz>,
	David Rientjes <rientjes@google.com>,
	Mel Gorman <mgorman@suse.de>,
	Johannes Weiner <hannes@cmpxchg.org>,
	Al Viro <viro@zeniv.linux.org.uk>,
	linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>,
	Alexei Starovoitov <ast@kernel.org>,
	Anatoly Stepanov <astepanov@cloudlinux.com>,
	Andreas Dilger <adilger@dilger.ca>,
	Andreas Dilger <andreas.dilger@intel.com>,
	Anton Vorontsov <anton@enomsg.org>,
	Ben Skeggs <bskeggs@redhat.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Colin Cross <ccross@android.com>,
	Dan Williams <dan.j.williams@intel.com>,
	David Sterba <dsterba@suse.com>,
	Eric Dumazet <edumazet@google.com>,
	Eric Dumazet <eric.dumazet@gmail.com>,
	Hariprasad S <hariprasad@chelsio.com>,
	Heiko Carstens <heiko.carstens@de.ibm.com>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	Ilya Dryomov <idryomov@gmail.com>,
	Kees Cook <keescook@chromium.org>,
	Kent Overstreet <kent.overstreet@gmail.com>,
	Martin Schwidefsky <schwidefsky@de.ibm.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Mike Snitzer <snitzer@redhat.com>,
	Oleg Drokin <oleg.drokin@intel.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	"Rafael J. Wysocki" <rjw@rjwysocki.net>,
	Santosh Raspatur <santosh@chelsio.com>,
	Tariq Toukan <tariqt@mellanox.com>, Theodore Ts'o <tytso@mit.edu>,
	Tom Herbert <tom@herbertland.com>,
	Tony Luck <tony.luck@intel.com>, "Yan, Zheng" <zyan@redhat.com>,
	Yishai Hadas <yishaih@mellanox.com>,
	Daniel Borkmann <daniel@iogearbox.net>
Subject: Re: [PATCH 0/6 v3] kvmalloc
Date: Wed, 25 Jan 2017 14:10:07 +0100	[thread overview]
Message-ID: <20170125131006.GQ32377@dhcp22.suse.cz> (raw)
In-Reply-To: <20170124191716.GA23114@ast-mbp.thefacebook.com>

On Tue 24-01-17 11:17:21, Alexei Starovoitov wrote:
> On Tue, Jan 24, 2017 at 04:17:52PM +0100, Michal Hocko wrote:
> > On Thu 12-01-17 16:37:11, Michal Hocko wrote:
> > > Hi,
> > > this has been previously posted as a single patch [1] but later on more
> > > built on top. It turned out that there are users who would like to have
> > > __GFP_REPEAT semantic. This is currently implemented for costly >64B
> > > requests. Doing the same for smaller requests would require to redefine
> > > __GFP_REPEAT semantic in the page allocator which is out of scope of
> > > this series.
> > > 
> > > There are many open coded kmalloc with vmalloc fallback instances in
> > > the tree.  Most of them are not careful enough or simply do not care
> > > about the underlying semantic of the kmalloc/page allocator which means
> > > that a) some vmalloc fallbacks are basically unreachable because the
> > > kmalloc part will keep retrying until it succeeds b) the page allocator
> > > can invoke a really disruptive steps like the OOM killer to move forward
> > > which doesn't sound appropriate when we consider that the vmalloc
> > > fallback is available.
> > > 
> > > As it can be seen implementing kvmalloc requires quite an intimate
> > > knowledge if the page allocator and the memory reclaim internals which
> > > strongly suggests that a helper should be implemented in the memory
> > > subsystem proper.
> > > 
> > > Most callers I could find have been converted to use the helper instead.
> > > This is patch 5. There are some more relying on __GFP_REPEAT in the
> > > networking stack which I have converted as well but considering we do
> > > not have a support for __GFP_REPEAT for requests smaller than 64kB I
> > > have marked it RFC.
> > 
> > Are there any more comments? I would really appreciate to hear from
> > networking folks before I resubmit the series.
> 
> while this patchset was baking the bpf side switched to use bpf_map_area_alloc()
> which fixes the issue with missing __GFP_NORETRY that we had to fix quickly.
> See commit d407bd25a204 ("bpf: don't trigger OOM killer under pressure with map alloc")
> it covers all kmalloc/vmalloc pairs instead of just one place as in this set.
> So please rebase and switch bpf_map_area_alloc() to use kvmalloc().

OK, will do. Thanks for the heads up.


-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2017-01-25 13:10 UTC|newest]

Thread overview: 180+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-01-12 15:37 [PATCH 0/6 v3] kvmalloc Michal Hocko
2017-01-12 15:37 ` Michal Hocko
2017-01-12 15:37 ` [PATCH 1/6] mm: introduce kv[mz]alloc helpers Michal Hocko
2017-01-12 15:37   ` Michal Hocko
2017-01-16  4:34   ` John Hubbard
2017-01-16  4:34     ` John Hubbard
2017-01-16  8:47     ` Michal Hocko
2017-01-16  8:47       ` Michal Hocko
2017-01-16 19:09       ` John Hubbard
2017-01-16 19:09         ` John Hubbard
2017-01-16 19:40         ` Michal Hocko
2017-01-16 19:40           ` Michal Hocko
2017-01-16 21:15           ` John Hubbard
2017-01-16 21:15             ` John Hubbard
2017-01-16 21:48             ` Michal Hocko
2017-01-16 21:48               ` Michal Hocko
2017-01-16 21:57               ` John Hubbard
2017-01-16 21:57                 ` John Hubbard
2017-01-17  7:51                 ` Michal Hocko
2017-01-17  7:51                   ` Michal Hocko
2017-01-18  5:59                   ` John Hubbard
2017-01-18  5:59                     ` John Hubbard
2017-01-18  8:21                     ` Michal Hocko
2017-01-18  8:21                       ` Michal Hocko
2017-01-19  8:37                       ` John Hubbard
2017-01-19  8:37                         ` John Hubbard
2017-01-19  8:45                         ` Michal Hocko
2017-01-19  8:45                           ` Michal Hocko
2017-01-19  9:09                           ` John Hubbard
2017-01-19  9:09                             ` John Hubbard
2017-01-19  9:56                             ` Michal Hocko
2017-01-19  9:56                               ` Michal Hocko
2017-01-19 21:28                               ` John Hubbard
2017-01-19 21:28                                 ` John Hubbard
2017-01-26 12:09   ` Michal Hocko
2017-01-26 12:09     ` Michal Hocko
2017-01-30  8:42     ` Vlastimil Babka
2017-01-30  8:42       ` Vlastimil Babka
2017-01-12 15:37 ` [PATCH 2/6] mm: support __GFP_REPEAT in kvmalloc_node for >=64kB Michal Hocko
2017-01-12 15:37   ` Michal Hocko
2017-01-12 16:12   ` Michael S. Tsirkin
2017-01-12 16:12     ` Michael S. Tsirkin
2017-01-14  2:42   ` Tetsuo Handa
2017-01-14  2:42     ` Tetsuo Handa
2017-01-14  8:45     ` Michal Hocko
2017-01-14  8:45       ` Michal Hocko
2017-01-24 15:40   ` Michael S. Tsirkin
2017-01-24 15:40     ` Michael S. Tsirkin
2017-01-12 15:37 ` [PATCH 3/6] rhashtable: simplify a strange allocation pattern Michal Hocko
2017-01-12 15:37   ` Michal Hocko
2017-01-12 15:37 ` [PATCH 4/6] ila: " Michal Hocko
2017-01-12 15:37   ` Michal Hocko
2017-01-12 15:37 ` [PATCH 5/6] treewide: use kv[mz]alloc* rather than opencoded variants Michal Hocko
2017-01-12 15:37   ` Michal Hocko
2017-01-12 15:37   ` Michal Hocko
2017-01-12 15:57   ` David Sterba
2017-01-12 15:57     ` David Sterba
2017-01-12 15:57     ` David Sterba
2017-01-12 16:05   ` Christian Borntraeger
2017-01-12 16:05     ` Christian Borntraeger
2017-01-12 16:05     ` Christian Borntraeger
2017-01-12 16:54   ` Ilya Dryomov
2017-01-12 16:54     ` Ilya Dryomov
2017-01-12 16:54     ` Ilya Dryomov
2017-01-12 17:18     ` Michal Hocko
2017-01-12 17:18       ` Michal Hocko
2017-01-12 17:18       ` Michal Hocko
2017-01-12 17:00   ` Dan Williams
2017-01-12 17:00     ` Dan Williams
2017-01-12 17:00     ` Dan Williams
2017-01-12 17:26   ` Kees Cook
2017-01-12 17:26     ` Kees Cook
2017-01-12 17:26     ` Kees Cook
2017-01-12 17:37     ` Michal Hocko
2017-01-12 17:37       ` Michal Hocko
2017-01-12 17:37       ` Michal Hocko
2017-01-20 13:41       ` Vlastimil Babka
2017-01-20 13:41         ` Vlastimil Babka
2017-01-20 13:41         ` Vlastimil Babka
2017-01-24 15:00         ` Michal Hocko
2017-01-24 15:00           ` Michal Hocko
2017-01-24 15:00           ` Michal Hocko
2017-01-25 11:15           ` Vlastimil Babka
2017-01-25 11:15             ` Vlastimil Babka
2017-01-25 11:15             ` Vlastimil Babka
2017-01-25 13:09             ` Michal Hocko
2017-01-25 13:09               ` Michal Hocko
2017-01-25 13:09               ` Michal Hocko
2017-01-25 13:40               ` Ilya Dryomov
2017-01-25 13:40                 ` Ilya Dryomov
2017-01-25 13:40                 ` Ilya Dryomov
2017-01-12 17:29   ` Michal Hocko
2017-01-12 17:29     ` Michal Hocko
2017-01-12 17:29     ` Michal Hocko
2017-01-14  3:01     ` Tetsuo Handa
2017-01-14  3:01       ` Tetsuo Handa
2017-01-14  8:49       ` Michal Hocko
2017-01-14  8:49         ` Michal Hocko
2017-01-12 20:14   ` Boris Ostrovsky
2017-01-12 20:14     ` Boris Ostrovsky
2017-01-12 20:14     ` Boris Ostrovsky
2017-01-13  1:11   ` Dilger, Andreas
2017-01-13  1:11     ` Dilger, Andreas
2017-01-13  1:11     ` Dilger, Andreas
2017-01-14 10:56   ` Leon Romanovsky
2017-01-14 10:56     ` Leon Romanovsky
2017-01-16  7:33     ` Michal Hocko
2017-01-16  7:33       ` Michal Hocko
2017-01-16  7:33       ` Michal Hocko
2017-01-16  8:28       ` Leon Romanovsky
2017-01-16  8:28         ` Leon Romanovsky
2017-01-16  8:18   ` Tariq Toukan
2017-01-16  8:18     ` Tariq Toukan
2017-01-16  8:18     ` Tariq Toukan
2017-01-12 15:37 ` [RFC PATCH 6/6] net: use kvmalloc with __GFP_REPEAT rather than open coded variant Michal Hocko
2017-01-12 15:37   ` Michal Hocko
2017-01-12 15:37   ` Michal Hocko
2017-01-24 15:17 ` [PATCH 0/6 v3] kvmalloc Michal Hocko
2017-01-24 15:17   ` Michal Hocko
2017-01-24 16:00   ` Eric Dumazet
2017-01-24 16:00     ` Eric Dumazet
2017-01-25 13:10     ` Michal Hocko
2017-01-25 13:10       ` Michal Hocko
2017-01-24 19:17   ` Alexei Starovoitov
2017-01-24 19:17     ` Alexei Starovoitov
2017-01-25 13:10     ` Michal Hocko [this message]
2017-01-25 13:10       ` Michal Hocko
2017-01-25 13:21       ` Michal Hocko
2017-01-25 13:21         ` Michal Hocko
2017-01-25 18:14 Alexei Starovoitov
2017-01-25 18:14 ` Alexei Starovoitov
2017-01-25 20:16 ` Daniel Borkmann
2017-01-25 20:16   ` Daniel Borkmann
2017-01-26  7:43   ` Michal Hocko
2017-01-26  7:43     ` Michal Hocko
2017-01-26  9:36     ` Daniel Borkmann
2017-01-26  9:36       ` Daniel Borkmann
2017-01-26  9:48       ` David Laight
2017-01-26  9:48         ` David Laight
2017-01-26 10:08       ` Michal Hocko
2017-01-26 10:08         ` Michal Hocko
2017-01-26 10:32         ` Michal Hocko
2017-01-26 10:32           ` Michal Hocko
2017-01-26 11:04           ` Daniel Borkmann
2017-01-26 11:04             ` Daniel Borkmann
2017-01-26 11:49             ` Michal Hocko
2017-01-26 11:49               ` Michal Hocko
2017-01-26 12:14           ` Joe Perches
2017-01-26 12:14             ` Joe Perches
2017-01-26 12:27             ` Michal Hocko
2017-01-26 12:27               ` Michal Hocko
2017-01-26 11:33         ` Daniel Borkmann
2017-01-26 11:33           ` Daniel Borkmann
2017-01-26 11:58           ` Michal Hocko
2017-01-26 11:58             ` Michal Hocko
2017-01-26 13:10             ` Daniel Borkmann
2017-01-26 13:10               ` Daniel Borkmann
2017-01-26 13:40               ` Michal Hocko
2017-01-26 13:40                 ` Michal Hocko
2017-01-26 14:13                 ` Michal Hocko
2017-01-26 14:13                   ` Michal Hocko
2017-01-26 14:13                   ` Michal Hocko
2017-01-26 20:34                 ` Daniel Borkmann
2017-01-26 20:34                   ` Daniel Borkmann
2017-01-27 10:05                   ` Michal Hocko
2017-01-27 10:05                     ` Michal Hocko
2017-01-27 20:12                     ` Daniel Borkmann
2017-01-27 20:12                       ` Daniel Borkmann
2017-01-30  7:56                       ` Michal Hocko
2017-01-30  7:56                         ` Michal Hocko
2017-01-30 16:15                         ` Daniel Borkmann
2017-01-30 16:15                           ` Daniel Borkmann
2017-01-30 16:28                           ` Michal Hocko
2017-01-30 16:28                             ` Michal Hocko
2017-01-30 16:45                             ` Daniel Borkmann
2017-01-30 16:45                               ` Daniel Borkmann
2017-01-30  9:49 Michal Hocko
2017-01-30  9:49 ` Michal Hocko
2017-02-05 10:23 ` Michal Hocko
2017-02-05 10:23   ` Michal Hocko

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170125131006.GQ32377@dhcp22.suse.cz \
    --to=mhocko@kernel.org \
    --cc=adilger@dilger.ca \
    --cc=akpm@linux-foundation.org \
    --cc=alexei.starovoitov@gmail.com \
    --cc=andreas.dilger@intel.com \
    --cc=anton@enomsg.org \
    --cc=ast@kernel.org \
    --cc=astepanov@cloudlinux.com \
    --cc=boris.ostrovsky@oracle.com \
    --cc=bskeggs@redhat.com \
    --cc=ccross@android.com \
    --cc=dan.j.williams@intel.com \
    --cc=daniel@iogearbox.net \
    --cc=dsterba@suse.com \
    --cc=edumazet@google.com \
    --cc=eric.dumazet@gmail.com \
    --cc=hannes@cmpxchg.org \
    --cc=hariprasad@chelsio.com \
    --cc=heiko.carstens@de.ibm.com \
    --cc=herbert@gondor.apana.org.au \
    --cc=idryomov@gmail.com \
    --cc=keescook@chromium.org \
    --cc=kent.overstreet@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=mst@redhat.com \
    --cc=oleg.drokin@intel.com \
    --cc=pbonzini@redhat.com \
    --cc=rientjes@google.com \
    --cc=rjw@rjwysocki.net \
    --cc=santosh@chelsio.com \
    --cc=schwidefsky@de.ibm.com \
    --cc=snitzer@redhat.com \
    --cc=tariqt@mellanox.com \
    --cc=tom@herbertland.com \
    --cc=tony.luck@intel.com \
    --cc=tytso@mit.edu \
    --cc=vbabka@suse.cz \
    --cc=viro@zeniv.linux.org.uk \
    --cc=yishaih@mellanox.com \
    --cc=zyan@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.