From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6E2C8C433EF for ; Tue, 5 Apr 2022 12:12:51 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B835210E5A6; Tue, 5 Apr 2022 12:12:50 +0000 (UTC) Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9886F10E2E2 for ; Tue, 5 Apr 2022 12:12:49 +0000 (UTC) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 2FE681F745; Tue, 5 Apr 2022 12:12:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649160767; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YGolTQlWegdqsJgfgfOCgFdotBM/n5qIvgfFfvEjxd0=; b=BK6a6qL8o7p/MJgbNqqctnSF5bmv5BGKuuqBTv2USITbTpPoKI1f15ogW871KoTo/RdheN 2ofE8y9/q59bmAhdL1seIYTAGQd263xP4FVoKwCcc/FYdRc3tUwqP3pmlISrihOeY4NF+U B5AcViJAx3CYaDGctEIJbO7LJzMoxwU= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id AA5C913A04; Tue, 5 Apr 2022 12:12:46 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id s8TPKD4yTGIlCAAAMHmgww (envelope-from ); Tue, 05 Apr 2022 12:12:46 +0000 Date: Tue, 5 Apr 2022 14:12:45 +0200 From: Michal =?iso-8859-1?Q?Koutn=FD?= To: "T.J. Mercier" Subject: Re: [RFC v4 5/8] dmabuf: Add gpu cgroup charge transfer function Message-ID: <20220405121245.GA30368@blackbody.suse.cz> References: <20220328035951.1817417-1-tjmercier@google.com> <20220328035951.1817417-6-tjmercier@google.com> <20220329152142.GA15794@blackbody.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Zefan Li , linux-doc@vger.kernel.org, David Airlie , dri-devel@lists.freedesktop.org, Benjamin Gaignard , Kalesh Singh , Joel Fernandes , Shuah Khan , Sumit Semwal , Kenny.Ho@amd.com, Jonathan Corbet , Martijn Coenen , Laura Abbott , linux-media@vger.kernel.org, linux-kselftest@vger.kernel.org, Todd Kjos , Thomas Zimmermann , linaro-mm-sig@lists.linaro.org, Tejun Heo , Shuah Khan , cgroups@vger.kernel.org, Suren Baghdasaryan , Christian Brauner , Greg Kroah-Hartman , linux-kernel@vger.kernel.org, Liam Mark , Christian =?iso-8859-1?Q?K=F6nig?= , Arve =?iso-8859-1?B?SGr4bm5lduVn?= , Johannes Weiner , Hridya Valsaraju Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Fri, Apr 01, 2022 at 11:41:36AM -0700, "T.J. Mercier" wrote: > This link doesn't work for me, but I think you're referring to the > discussion about your "RAM_backed_buffers" comment from March 23rd. (Oops, it's a non-public message. But yes, you guessed it right ;-)) > Anyway the test I did goes like this: enable memcg and gpu cgoups > tracking and run a process that allocates 100MiB of dmabufs. Observe > memcg and gpu accounting values before and after the allocation. Thanks for this measurement/dem/demoo. > Before > # cat memory.current gpu.memory.current > 14909440 > system 0 > > > > After > # cat memory.current gpu.memory.current > 48025600 > system 104857600 > > So the memcg value increases by about 30 MiB while the gpu values > increases by 100 MiB. > This is with kmem enabled, and the /proc/maps > file for this process indicates that the majority of that 30 MiB is > kernel memory. > I think this result shows that neither the kernel nor process memory > overlap with the gpu cgroup tracking of these allocations. It depends how the semantics of the 'system' entry is defined, no? As I grasped from other thread, the 'total' is going to be removed, so 'system' represents exclusively device memory? > So despite the fact that these buffers are in main memory, they are > allocated in a way that does not result in memcg attribution. (It > looks to me like __GFP_ACCOUNT is not set for these.) (I thought you knew what dmabufs your program used :-p) So, the goal is to do the tracking and migrations only via the gpu cg layer, regardless how memcg charges it (or not). (I have no opinion on that, I'm just summing it so that we're on the same page.) Michal From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2CF03C433FE for ; Wed, 6 Apr 2022 00:51:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1838243AbiDFAuy (ORCPT ); Tue, 5 Apr 2022 20:50:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36446 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1377013AbiDENMq (ORCPT ); Tue, 5 Apr 2022 09:12:46 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BFF7D120D9D; Tue, 5 Apr 2022 05:12:48 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 2FE681F745; Tue, 5 Apr 2022 12:12:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649160767; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YGolTQlWegdqsJgfgfOCgFdotBM/n5qIvgfFfvEjxd0=; b=BK6a6qL8o7p/MJgbNqqctnSF5bmv5BGKuuqBTv2USITbTpPoKI1f15ogW871KoTo/RdheN 2ofE8y9/q59bmAhdL1seIYTAGQd263xP4FVoKwCcc/FYdRc3tUwqP3pmlISrihOeY4NF+U B5AcViJAx3CYaDGctEIJbO7LJzMoxwU= Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id AA5C913A04; Tue, 5 Apr 2022 12:12:46 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id s8TPKD4yTGIlCAAAMHmgww (envelope-from ); Tue, 05 Apr 2022 12:12:46 +0000 Date: Tue, 5 Apr 2022 14:12:45 +0200 From: Michal =?iso-8859-1?Q?Koutn=FD?= To: "T.J. Mercier" Cc: David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet , Greg Kroah-Hartman , Arve =?iso-8859-1?B?SGr4bm5lduVn?= , Todd Kjos , Martijn Coenen , Joel Fernandes , Christian Brauner , Hridya Valsaraju , Suren Baghdasaryan , Sumit Semwal , Christian =?iso-8859-1?Q?K=F6nig?= , Benjamin Gaignard , Liam Mark , Laura Abbott , Brian Starkey , John Stultz , Tejun Heo , Zefan Li , Johannes Weiner , Shuah Khan , Kalesh Singh , Kenny.Ho@amd.com, Shuah Khan , dri-devel@lists.freedesktop.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, linaro-mm-sig@lists.linaro.org, cgroups@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: Re: [RFC v4 5/8] dmabuf: Add gpu cgroup charge transfer function Message-ID: <20220405121245.GA30368@blackbody.suse.cz> References: <20220328035951.1817417-1-tjmercier@google.com> <20220328035951.1817417-6-tjmercier@google.com> <20220329152142.GA15794@blackbody.suse.cz> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Apr 01, 2022 at 11:41:36AM -0700, "T.J. Mercier" wrote: > This link doesn't work for me, but I think you're referring to the > discussion about your "RAM_backed_buffers" comment from March 23rd. (Oops, it's a non-public message. But yes, you guessed it right ;-)) > Anyway the test I did goes like this: enable memcg and gpu cgoups > tracking and run a process that allocates 100MiB of dmabufs. Observe > memcg and gpu accounting values before and after the allocation. Thanks for this measurement/dem/demoo. > Before > # cat memory.current gpu.memory.current > 14909440 > system 0 > > > > After > # cat memory.current gpu.memory.current > 48025600 > system 104857600 > > So the memcg value increases by about 30 MiB while the gpu values > increases by 100 MiB. > This is with kmem enabled, and the /proc/maps > file for this process indicates that the majority of that 30 MiB is > kernel memory. > I think this result shows that neither the kernel nor process memory > overlap with the gpu cgroup tracking of these allocations. It depends how the semantics of the 'system' entry is defined, no? As I grasped from other thread, the 'total' is going to be removed, so 'system' represents exclusively device memory? > So despite the fact that these buffers are in main memory, they are > allocated in a way that does not result in memcg attribution. (It > looks to me like __GFP_ACCOUNT is not set for these.) (I thought you knew what dmabufs your program used :-p) So, the goal is to do the tracking and migrations only via the gpu cg layer, regardless how memcg charges it (or not). (I have no opinion on that, I'm just summing it so that we're on the same page.) Michal From mboxrd@z Thu Jan 1 00:00:00 1970 From: Michal =?iso-8859-1?Q?Koutn=FD?= Subject: Re: [RFC v4 5/8] dmabuf: Add gpu cgroup charge transfer function Date: Tue, 5 Apr 2022 14:12:45 +0200 Message-ID: <20220405121245.GA30368@blackbody.suse.cz> References: <20220328035951.1817417-1-tjmercier@google.com> <20220328035951.1817417-6-tjmercier@google.com> <20220329152142.GA15794@blackbody.suse.cz> Mime-Version: 1.0 Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.com; s=susede1; t=1649160767; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YGolTQlWegdqsJgfgfOCgFdotBM/n5qIvgfFfvEjxd0=; b=BK6a6qL8o7p/MJgbNqqctnSF5bmv5BGKuuqBTv2USITbTpPoKI1f15ogW871KoTo/RdheN 2ofE8y9/q59bmAhdL1seIYTAGQd263xP4FVoKwCcc/FYdRc3tUwqP3pmlISrihOeY4NF+U B5AcViJAx3CYaDGctEIJbO7LJzMoxwU= Content-Disposition: inline In-Reply-To: List-ID: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: "T.J. Mercier" Cc: David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet , Greg Kroah-Hartman , Arve =?iso-8859-1?B?SGr4bm5lduVn?= , Todd Kjos , Martijn Coenen , Joel Fernandes , Christian Brauner , Hridya Valsaraju , Suren Baghdasaryan , Sumit Semwal , Christian =?iso-8859-1?Q?K=F6nig?= , Benjamin Gaignard , Liam Mark , Laura On Fri, Apr 01, 2022 at 11:41:36AM -0700, "T.J. Mercier" wrote: > This link doesn't work for me, but I think you're referring to the > discussion about your "RAM_backed_buffers" comment from March 23rd. (Oops, it's a non-public message. But yes, you guessed it right ;-)) > Anyway the test I did goes like this: enable memcg and gpu cgoups > tracking and run a process that allocates 100MiB of dmabufs. Observe > memcg and gpu accounting values before and after the allocation. Thanks for this measurement/dem/demoo. > Before > # cat memory.current gpu.memory.current > 14909440 > system 0 > > > > After > # cat memory.current gpu.memory.current > 48025600 > system 104857600 > > So the memcg value increases by about 30 MiB while the gpu values > increases by 100 MiB. > This is with kmem enabled, and the /proc/maps > file for this process indicates that the majority of that 30 MiB is > kernel memory. > I think this result shows that neither the kernel nor process memory > overlap with the gpu cgroup tracking of these allocations. It depends how the semantics of the 'system' entry is defined, no? As I grasped from other thread, the 'total' is going to be removed, so 'system' represents exclusively device memory? > So despite the fact that these buffers are in main memory, they are > allocated in a way that does not result in memcg attribution. (It > looks to me like __GFP_ACCOUNT is not set for these.) (I thought you knew what dmabufs your program used :-p) So, the goal is to do the tracking and migrations only via the gpu cg layer, regardless how memcg charges it (or not). (I have no opinion on that, I'm just summing it so that we're on the same page.) Michal