From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-f197.google.com (mail-io0-f197.google.com [209.85.223.197]) by kanga.kvack.org (Postfix) with ESMTP id CFC6D800D8 for ; Mon, 22 Jan 2018 20:59:27 -0500 (EST) Received: by mail-io0-f197.google.com with SMTP id s9so11714629ioa.20 for ; Mon, 22 Jan 2018 17:59:27 -0800 (PST) Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-cys01nam02on0076.outbound.protection.outlook.com. [104.47.37.76]) by mx.google.com with ESMTPS id 83si6973330itb.92.2018.01.22.17.59.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 22 Jan 2018 17:59:26 -0800 (PST) Subject: Re: [RFC] Per file OOM badness References: <1516294072-17841-1-git-send-email-andrey.grodzovsky@amd.com> <20180122152315.749d88f3c91ffce4d70ac450@linux-foundation.org> From: Andrey Grodzovsky Message-ID: Date: Mon, 22 Jan 2018 20:59:19 -0500 MIME-Version: 1.0 In-Reply-To: <20180122152315.749d88f3c91ffce4d70ac450@linux-foundation.org> Content-Type: multipart/alternative; boundary="------------6A6E3A4A1D18107554EA7ECB" Content-Language: en-US Sender: owner-linux-mm@kvack.org List-ID: To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Christian.Koenig@amd.com This is a multi-part message in MIME format. --------------6A6E3A4A1D18107554EA7ECB Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 01/22/2018 06:23 PM, Andrew Morton wrote: > On Thu, 18 Jan 2018 11:47:48 -0500 Andrey Grodzovsky wrote: > >> Hi, this series is a revised version of an RFC sent by Christian KA?nig >> a few years ago. The original RFC can be found at >> https://lists.freedesktop.org/archives/dri-devel/2015-September/089778.html >> >> This is the same idea and I've just adressed his concern from the original RFC >> and switched to a callback into file_ops instead of a new member in struct file. > Should be in address_space_operations, I suspect. If an application > opens a file twice, we only want to count it once? Makes sense > > But we're putting the cart ahead of the horse here. Please provide us > with a detailed description of the problem which you are addressing so > that the MM developers can better consider how to address your > requirements. I will just reiterate the problem statement from the original RFC, should have put it in the body of the RFC and not just a link, as already commented by Michal. Bellow is the quoted RFC. Thanks, Andrey P.S You can also check the follow up discussion after this first email. " I'm currently working on the issue that when device drivers allocate memory on behalf of an application the OOM killer usually doesn't knew about that unless the application also get this memory mapped into their address space. This is especially annoying for graphics drivers where a lot of the VRAM usually isn't CPU accessible and so doesn't make sense to map into the address space of the process using it. The problem now is that when an application starts to use a lot of VRAM those buffers objects sooner or later get swapped out to system memory, but when we now run into an out of memory situation the OOM killer obviously doesn't knew anything about that memory and so usually kills the wrong process " --------------6A6E3A4A1D18107554EA7ECB Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit



On 01/22/2018 06:23 PM, Andrew Morton wrote:
On Thu, 18 Jan 2018 11:47:48 -0500 Andrey Grodzovsky <andrey.grodzovsky@amd.com> wrote:

Hi, this series is a revised version of an RFC sent by Christian KA?nig
a few years ago. The original RFC can be found at 
https://lists.freedesktop.org/archives/dri-devel/2015-September/089778.html

This is the same idea and I've just adressed his concern from the original RFC 
and switched to a callback into file_ops instead of a new member in struct file.
Should be in address_space_operations, I suspect.  If an application
opens a file twice, we only want to count it once?

Makes sense


But we're putting the cart ahead of the horse here.  Please provide us
with a detailed description of the problem which you are addressing so
that the MM developers can better consider how to address your
requirements.

I will just reiterate the problem statement from the original RFC, should have
put it in the body of the RFC and not just a link, as already commented by Michal.
Bellow is the quoted RFC.

Thanks,
Andrey

P.S You can also check the follow up discussion after this first email.

"
I'm currently working on the issue that when device drivers allocate memory on
behalf of an application the OOM killer usually doesn't knew about that unless
the application also get this memory mapped into their address space.

This is especially annoying for graphics drivers where a lot of the VRAM
usually isn't CPU accessible and so doesn't make sense to map into the
address space of the process using it.

The problem now is that when an application starts to use a lot of VRAM those
buffers objects sooner or later get swapped out to system memory, but when we
now run into an out of memory situation the OOM killer obviously doesn't knew
anything about that memory and so usually kills the wrong process

"



    

--------------6A6E3A4A1D18107554EA7ECB-- -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrey Grodzovsky Subject: Re: [RFC] Per file OOM badness Date: Mon, 22 Jan 2018 20:59:19 -0500 Message-ID: References: <1516294072-17841-1-git-send-email-andrey.grodzovsky@amd.com> <20180122152315.749d88f3c91ffce4d70ac450@linux-foundation.org> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============0992958000==" Return-path: In-Reply-To: <20180122152315.749d88f3c91ffce4d70ac450-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "amd-gfx" To: Andrew Morton Cc: linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, Christian.Koenig-5C7GfCeVMHo@public.gmane.org List-Id: dri-devel@lists.freedesktop.org This is a multi-part message in MIME format. --===============0992958000== Content-Type: multipart/alternative; boundary="------------6A6E3A4A1D18107554EA7ECB" Content-Language: en-US This is a multi-part message in MIME format. --------------6A6E3A4A1D18107554EA7ECB Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit On 01/22/2018 06:23 PM, Andrew Morton wrote: > On Thu, 18 Jan 2018 11:47:48 -0500 Andrey Grodzovsky wrote: > >> Hi, this series is a revised version of an RFC sent by Christian König >> a few years ago. The original RFC can be found at >> https://lists.freedesktop.org/archives/dri-devel/2015-September/089778.html >> >> This is the same idea and I've just adressed his concern from the original RFC >> and switched to a callback into file_ops instead of a new member in struct file. > Should be in address_space_operations, I suspect. If an application > opens a file twice, we only want to count it once? Makes sense > > But we're putting the cart ahead of the horse here. Please provide us > with a detailed description of the problem which you are addressing so > that the MM developers can better consider how to address your > requirements. I will just reiterate the problem statement from the original RFC, should have put it in the body of the RFC and not just a link, as already commented by Michal. Bellow is the quoted RFC. Thanks, Andrey P.S You can also check the follow up discussion after this first email. " I'm currently working on the issue that when device drivers allocate memory on behalf of an application the OOM killer usually doesn't knew about that unless the application also get this memory mapped into their address space. This is especially annoying for graphics drivers where a lot of the VRAM usually isn't CPU accessible and so doesn't make sense to map into the address space of the process using it. The problem now is that when an application starts to use a lot of VRAM those buffers objects sooner or later get swapped out to system memory, but when we now run into an out of memory situation the OOM killer obviously doesn't knew anything about that memory and so usually kills the wrong process " --------------6A6E3A4A1D18107554EA7ECB Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit



On 01/22/2018 06:23 PM, Andrew Morton wrote:
On Thu, 18 Jan 2018 11:47:48 -0500 Andrey Grodzovsky <andrey.grodzovsky-5C7GfCeVMHo@public.gmane.org> wrote:

Hi, this series is a revised version of an RFC sent by Christian König
a few years ago. The original RFC can be found at 
https://lists.freedesktop.org/archives/dri-devel/2015-September/089778.html

This is the same idea and I've just adressed his concern from the original RFC 
and switched to a callback into file_ops instead of a new member in struct file.
Should be in address_space_operations, I suspect.  If an application
opens a file twice, we only want to count it once?

Makes sense


But we're putting the cart ahead of the horse here.  Please provide us
with a detailed description of the problem which you are addressing so
that the MM developers can better consider how to address your
requirements.

I will just reiterate the problem statement from the original RFC, should have
put it in the body of the RFC and not just a link, as already commented by Michal.
Bellow is the quoted RFC.

Thanks,
Andrey

P.S You can also check the follow up discussion after this first email.

"
I'm currently working on the issue that when device drivers allocate memory on
behalf of an application the OOM killer usually doesn't knew about that unless
the application also get this memory mapped into their address space.

This is especially annoying for graphics drivers where a lot of the VRAM
usually isn't CPU accessible and so doesn't make sense to map into the
address space of the process using it.

The problem now is that when an application starts to use a lot of VRAM those
buffers objects sooner or later get swapped out to system memory, but when we
now run into an out of memory situation the OOM killer obviously doesn't knew
anything about that memory and so usually kills the wrong process

"



    

--------------6A6E3A4A1D18107554EA7ECB-- --===============0992958000== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Content-Disposition: inline X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KYW1kLWdmeCBt YWlsaW5nIGxpc3QKYW1kLWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5m cmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9hbWQtZ2Z4Cg== --===============0992958000==--