All of lore.kernel.org
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@kernel.org>
To: Roman Gushchin <guro@fb.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org,
	Christian.Koenig@amd.com
Subject: Re: [RFC] Per file OOM badness
Date: Tue, 23 Jan 2018 16:36:31 +0100	[thread overview]
Message-ID: <20180123153631.GR1526@dhcp22.suse.cz> (raw)
In-Reply-To: <20180123152659.GA21817@castle.DHCP.thefacebook.com>

On Tue 23-01-18 15:27:00, Roman Gushchin wrote:
> On Thu, Jan 18, 2018 at 06:00:06PM +0100, Michal Hocko wrote:
> > On Thu 18-01-18 11:47:48, Andrey Grodzovsky wrote:
> > > Hi, this series is a revised version of an RFC sent by Christian König
> > > a few years ago. The original RFC can be found at 
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_archives_dri-2Ddevel_2015-2DSeptember_089778.html&d=DwIDAw&c=5VD0RTtNlTh3ycd41b3MUw&r=jJYgtDM7QT-W-Fz_d29HYQ&m=R-JIQjy8rqmH5qD581_VYL0Q7cpWSITKOnBCE-3LI8U&s=QZGqKpKuJ2BtioFGSy8_721owcWJ0J6c6d4jywOwN4w&
> > Here is the origin cover letter text
> > : I'm currently working on the issue that when device drivers allocate memory on
> > : behalf of an application the OOM killer usually doesn't knew about that unless
> > : the application also get this memory mapped into their address space.
> > : 
> > : This is especially annoying for graphics drivers where a lot of the VRAM
> > : usually isn't CPU accessible and so doesn't make sense to map into the
> > : address space of the process using it.
> > : 
> > : The problem now is that when an application starts to use a lot of VRAM those
> > : buffers objects sooner or later get swapped out to system memory, but when we
> > : now run into an out of memory situation the OOM killer obviously doesn't knew
> > : anything about that memory and so usually kills the wrong process.
> > : 
> > : The following set of patches tries to address this problem by introducing a per
> > : file OOM badness score, which device drivers can use to give the OOM killer a
> > : hint how many resources are bound to a file descriptor so that it can make
> > : better decisions which process to kill.
> > : 
> > : So question at every one: What do you think about this approach?
> > : 
> > : My biggest concern right now is the patches are messing with a core kernel
> > : structure (adding a field to struct file). Any better idea? I'm considering
> > : to put a callback into file_ops instead.
> 
> Hello!
> 
> I wonder if groupoom (aka cgroup-aware OOM killer) can work for you?

I do not think so. The problem is that the allocating context is not
identical with the end consumer.
-- 
Michal Hocko
SUSE Labs

WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko@kernel.org>
To: Roman Gushchin <guro@fb.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org,
	Christian.Koenig@amd.com
Subject: Re: [RFC] Per file OOM badness
Date: Tue, 23 Jan 2018 16:36:31 +0100	[thread overview]
Message-ID: <20180123153631.GR1526@dhcp22.suse.cz> (raw)
In-Reply-To: <20180123152659.GA21817@castle.DHCP.thefacebook.com>

On Tue 23-01-18 15:27:00, Roman Gushchin wrote:
> On Thu, Jan 18, 2018 at 06:00:06PM +0100, Michal Hocko wrote:
> > On Thu 18-01-18 11:47:48, Andrey Grodzovsky wrote:
> > > Hi, this series is a revised version of an RFC sent by Christian Konig
> > > a few years ago. The original RFC can be found at 
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_archives_dri-2Ddevel_2015-2DSeptember_089778.html&d=DwIDAw&c=5VD0RTtNlTh3ycd41b3MUw&r=jJYgtDM7QT-W-Fz_d29HYQ&m=R-JIQjy8rqmH5qD581_VYL0Q7cpWSITKOnBCE-3LI8U&s=QZGqKpKuJ2BtioFGSy8_721owcWJ0J6c6d4jywOwN4w&
> > Here is the origin cover letter text
> > : I'm currently working on the issue that when device drivers allocate memory on
> > : behalf of an application the OOM killer usually doesn't knew about that unless
> > : the application also get this memory mapped into their address space.
> > : 
> > : This is especially annoying for graphics drivers where a lot of the VRAM
> > : usually isn't CPU accessible and so doesn't make sense to map into the
> > : address space of the process using it.
> > : 
> > : The problem now is that when an application starts to use a lot of VRAM those
> > : buffers objects sooner or later get swapped out to system memory, but when we
> > : now run into an out of memory situation the OOM killer obviously doesn't knew
> > : anything about that memory and so usually kills the wrong process.
> > : 
> > : The following set of patches tries to address this problem by introducing a per
> > : file OOM badness score, which device drivers can use to give the OOM killer a
> > : hint how many resources are bound to a file descriptor so that it can make
> > : better decisions which process to kill.
> > : 
> > : So question at every one: What do you think about this approach?
> > : 
> > : My biggest concern right now is the patches are messing with a core kernel
> > : structure (adding a field to struct file). Any better idea? I'm considering
> > : to put a callback into file_ops instead.
> 
> Hello!
> 
> I wonder if groupoom (aka cgroup-aware OOM killer) can work for you?

I do not think so. The problem is that the allocating context is not
identical with the end consumer.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko@kernel.org>
To: Roman Gushchin <guro@fb.com>
Cc: Andrey Grodzovsky <andrey.grodzovsky@amd.com>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org,
	Christian.Koenig@amd.com
Subject: Re: [RFC] Per file OOM badness
Date: Tue, 23 Jan 2018 16:36:31 +0100	[thread overview]
Message-ID: <20180123153631.GR1526@dhcp22.suse.cz> (raw)
In-Reply-To: <20180123152659.GA21817@castle.DHCP.thefacebook.com>

On Tue 23-01-18 15:27:00, Roman Gushchin wrote:
> On Thu, Jan 18, 2018 at 06:00:06PM +0100, Michal Hocko wrote:
> > On Thu 18-01-18 11:47:48, Andrey Grodzovsky wrote:
> > > Hi, this series is a revised version of an RFC sent by Christian König
> > > a few years ago. The original RFC can be found at 
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.freedesktop.org_archives_dri-2Ddevel_2015-2DSeptember_089778.html&d=DwIDAw&c=5VD0RTtNlTh3ycd41b3MUw&r=jJYgtDM7QT-W-Fz_d29HYQ&m=R-JIQjy8rqmH5qD581_VYL0Q7cpWSITKOnBCE-3LI8U&s=QZGqKpKuJ2BtioFGSy8_721owcWJ0J6c6d4jywOwN4w&
> > Here is the origin cover letter text
> > : I'm currently working on the issue that when device drivers allocate memory on
> > : behalf of an application the OOM killer usually doesn't knew about that unless
> > : the application also get this memory mapped into their address space.
> > : 
> > : This is especially annoying for graphics drivers where a lot of the VRAM
> > : usually isn't CPU accessible and so doesn't make sense to map into the
> > : address space of the process using it.
> > : 
> > : The problem now is that when an application starts to use a lot of VRAM those
> > : buffers objects sooner or later get swapped out to system memory, but when we
> > : now run into an out of memory situation the OOM killer obviously doesn't knew
> > : anything about that memory and so usually kills the wrong process.
> > : 
> > : The following set of patches tries to address this problem by introducing a per
> > : file OOM badness score, which device drivers can use to give the OOM killer a
> > : hint how many resources are bound to a file descriptor so that it can make
> > : better decisions which process to kill.
> > : 
> > : So question at every one: What do you think about this approach?
> > : 
> > : My biggest concern right now is the patches are messing with a core kernel
> > : structure (adding a field to struct file). Any better idea? I'm considering
> > : to put a callback into file_ops instead.
> 
> Hello!
> 
> I wonder if groupoom (aka cgroup-aware OOM killer) can work for you?

I do not think so. The problem is that the allocating context is not
identical with the end consumer.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2018-01-23 15:36 UTC|newest]

Thread overview: 170+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-01-18 16:47 [RFC] Per file OOM badness Andrey Grodzovsky
2018-01-18 16:47 ` Andrey Grodzovsky
2018-01-18 16:47 ` Andrey Grodzovsky
2018-01-18 16:47 ` [PATCH 1/4] fs: add OOM badness callback in file_operatrations struct Andrey Grodzovsky
2018-01-18 16:47   ` Andrey Grodzovsky
2018-01-18 16:47   ` Andrey Grodzovsky
2018-01-18 16:47 ` [PATCH 2/4] oom: take per file badness into account Andrey Grodzovsky
2018-01-18 16:47   ` Andrey Grodzovsky
2018-01-18 16:47   ` Andrey Grodzovsky
2018-01-18 16:47 ` [PATCH 3/4] drm/gem: adjust per file OOM badness on handling buffers Andrey Grodzovsky
2018-01-18 16:47   ` Andrey Grodzovsky
2018-01-18 16:47   ` Andrey Grodzovsky
2018-01-19  6:01   ` Chunming Zhou
2018-01-19  6:01     ` Chunming Zhou
2018-01-19  6:01     ` Chunming Zhou
2018-01-18 16:47 ` [PATCH 4/4] drm/amdgpu: Use drm_oom_badness for amdgpu Andrey Grodzovsky
2018-01-18 16:47   ` Andrey Grodzovsky
2018-01-18 16:47   ` Andrey Grodzovsky
2018-01-30  9:24   ` Daniel Vetter
2018-01-30  9:24     ` Daniel Vetter
2018-01-30 12:42     ` Andrey Grodzovsky
2018-01-30 12:42       ` Andrey Grodzovsky
2018-01-30 12:42       ` Andrey Grodzovsky
2018-01-18 17:00 ` [RFC] Per file OOM badness Michal Hocko
2018-01-18 17:00   ` Michal Hocko
2018-01-18 17:00   ` Michal Hocko
2018-01-18 17:13   ` Michal Hocko
2018-01-18 17:13     ` Michal Hocko
2018-01-18 20:01     ` Eric Anholt
2018-01-19  8:20       ` Michal Hocko
2018-01-19  8:20         ` Michal Hocko
2018-01-19  8:20         ` Michal Hocko
2018-01-19  8:39         ` Christian König
2018-01-19  8:39           ` Christian König
2018-01-19  8:39           ` Christian König
2018-01-19  9:32           ` Michel Dänzer
2018-01-19  9:32             ` Michel Dänzer
2018-01-19  9:32             ` Michel Dänzer
2018-01-19  9:58             ` Christian König
2018-01-19  9:58               ` Christian König
2018-01-19  9:58               ` Christian König
2018-01-19 10:02               ` Michel Dänzer
2018-01-19 10:02                 ` Michel Dänzer
2018-01-19 10:02                 ` Michel Dänzer
2018-01-19 15:07                 ` Michel Dänzer
2018-01-19 15:07                   ` Michel Dänzer
2018-01-21  6:50                   ` Eric Anholt
2018-01-19 10:40           ` Michal Hocko
2018-01-19 10:40             ` Michal Hocko
2018-01-19 10:40             ` Michal Hocko
2018-01-19 11:37             ` Christian König
2018-01-19 11:37               ` Christian König
2018-01-19 11:37               ` Christian König
2018-01-19 12:13               ` Michal Hocko
2018-01-19 12:13                 ` Michal Hocko
2018-01-19 12:13                 ` Michal Hocko
2018-01-19 12:20                 ` Michal Hocko
2018-01-19 12:20                   ` Michal Hocko
2018-01-19 12:20                   ` Michal Hocko
2018-01-19 16:54                   ` Christian König
2018-01-19 16:54                     ` Christian König
2018-01-19 16:54                     ` Christian König
2018-01-23 11:39                     ` Michal Hocko
2018-01-23 11:39                       ` Michal Hocko
2018-01-23 11:39                       ` Michal Hocko
2018-01-19 16:48               ` Michel Dänzer
2018-01-19 16:48                 ` Michel Dänzer
2018-01-19 16:48                 ` Michel Dänzer
2018-01-19  8:35       ` Christian König
2018-01-19  8:35         ` Christian König
2018-01-19  6:01     ` He, Roger
2018-01-19  8:25       ` Michal Hocko
2018-01-19  8:25         ` Michal Hocko
2018-01-19 10:02         ` roger
2018-01-19 10:02           ` roger
2018-01-23 15:27   ` Roman Gushchin
2018-01-23 15:27     ` Roman Gushchin
2018-01-23 15:27     ` Roman Gushchin
2018-01-23 15:36     ` Michal Hocko [this message]
2018-01-23 15:36       ` Michal Hocko
2018-01-23 15:36       ` Michal Hocko
2018-01-23 16:39       ` Michel Dänzer
2018-01-23 16:39         ` Michel Dänzer
2018-01-23 16:39         ` Michel Dänzer
2018-01-24  9:28         ` Michal Hocko
2018-01-24  9:28           ` Michal Hocko
2018-01-24  9:28           ` Michal Hocko
2018-01-24 10:27           ` Michel Dänzer
2018-01-24 10:27             ` Michel Dänzer
2018-01-24 10:27             ` Michel Dänzer
2018-01-24 11:01             ` Michal Hocko
2018-01-24 11:01               ` Michal Hocko
2018-01-24 11:01               ` Michal Hocko
2018-01-24 11:23               ` Michel Dänzer
2018-01-24 11:23                 ` Michel Dänzer
2018-01-24 11:23                 ` Michel Dänzer
2018-01-24 11:50                 ` Michal Hocko
2018-01-24 11:50                   ` Michal Hocko
2018-01-24 11:50                   ` Michal Hocko
2018-01-24 12:11                   ` Christian König
2018-01-24 12:11                     ` Christian König
2018-01-30  9:31                     ` Daniel Vetter
2018-01-30  9:31                       ` Daniel Vetter
2018-01-30  9:31                       ` Daniel Vetter
2018-01-30  9:43                       ` Michel Dänzer
2018-01-30  9:43                         ` Michel Dänzer
2018-01-30  9:43                         ` Michel Dänzer
2018-01-30 10:40                         ` Christian König
2018-01-30 10:40                           ` Christian König
2018-01-30 10:40                           ` Christian König
2018-01-30 11:02                           ` Michel Dänzer
2018-01-30 11:02                             ` Michel Dänzer
2018-01-30 11:02                             ` Michel Dänzer
2018-01-30 11:28                             ` Christian König
2018-01-30 11:28                               ` Christian König
2018-01-30 11:34                               ` Michel Dänzer
2018-01-30 11:34                                 ` Michel Dänzer
2018-01-30 11:34                                 ` Michel Dänzer
2018-01-30 11:36                                 ` Nicolai Hähnle
2018-01-30 11:36                                   ` Nicolai Hähnle
2018-01-30 11:36                                   ` Nicolai Hähnle
2018-01-30 11:42                                   ` Michel Dänzer
2018-01-30 11:42                                     ` Michel Dänzer
2018-01-30 11:42                                     ` Michel Dänzer
2018-01-30 11:56                                     ` Christian König
2018-01-30 11:56                                       ` Christian König
2018-01-30 11:56                                       ` Christian König
2018-01-30 15:52                                       ` Michel Dänzer
2018-01-30 15:52                                         ` Michel Dänzer
2018-01-30 15:52                                         ` Michel Dänzer
2018-01-30 10:42                         ` Daniel Vetter
2018-01-30 10:42                           ` Daniel Vetter
2018-01-30 10:42                           ` Daniel Vetter
2018-01-30 10:48                           ` Michel Dänzer
2018-01-30 10:48                             ` Michel Dänzer
2018-01-30 10:48                             ` Michel Dänzer
2018-01-30 11:35                             ` Nicolai Hähnle
2018-01-30 11:35                               ` Nicolai Hähnle
2018-01-30 11:35                               ` Nicolai Hähnle
2018-01-24 14:31                   ` Michel Dänzer
2018-01-24 14:31                     ` Michel Dänzer
2018-01-24 14:31                     ` Michel Dänzer
2018-01-30  9:29                   ` Michel Dänzer
2018-01-30  9:29                     ` Michel Dänzer
2018-01-30  9:29                     ` Michel Dänzer
2018-01-30 10:28                     ` Michal Hocko
2018-01-30 10:28                       ` Michal Hocko
2018-01-30 10:28                       ` Michal Hocko
2018-03-26 14:36                       ` Lucas Stach
2018-03-26 14:36                         ` Lucas Stach
2018-03-26 14:36                         ` Lucas Stach
2018-04-04  9:09                         ` Michel Dänzer
2018-04-04  9:09                           ` Michel Dänzer
2018-04-04  9:09                           ` Michel Dänzer
2018-04-04  9:36                           ` Lucas Stach
2018-04-04  9:36                             ` Lucas Stach
2018-04-04  9:36                             ` Lucas Stach
2018-04-04  9:46                             ` Michel Dänzer
2018-04-04  9:46                               ` Michel Dänzer
2018-04-04  9:46                               ` Michel Dänzer
2018-01-19  5:39 ` He, Roger
2018-01-19  8:17   ` Christian König
2018-01-19  8:17     ` Christian König
2018-01-19  8:17     ` Christian König
2018-01-22 23:23 ` Andrew Morton
2018-01-22 23:23   ` Andrew Morton
2018-01-22 23:23   ` Andrew Morton
2018-01-23  1:59   ` Andrey Grodzovsky
2018-01-23  1:59     ` Andrey Grodzovsky
  -- strict thread matches above, loose matches on Subject: below --
2015-09-04 12:53 Christian König

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180123153631.GR1526@dhcp22.suse.cz \
    --to=mhocko@kernel.org \
    --cc=Christian.Koenig@amd.com \
    --cc=amd-gfx@lists.freedesktop.org \
    --cc=andrey.grodzovsky@amd.com \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=guro@fb.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.