From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B42AC433DB for ; Wed, 3 Feb 2021 19:53:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 9BCD864F60 for ; Wed, 3 Feb 2021 19:53:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9BCD864F60 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 21D686B0005; Wed, 3 Feb 2021 14:53:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A8436B006C; Wed, 3 Feb 2021 14:53:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 048E56B006E; Wed, 3 Feb 2021 14:53:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0247.hostedemail.com [216.40.44.247]) by kanga.kvack.org (Postfix) with ESMTP id DC6CD6B0005 for ; Wed, 3 Feb 2021 14:53:52 -0500 (EST) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id A724A1EE6 for ; Wed, 3 Feb 2021 19:53:52 +0000 (UTC) X-FDA: 77778007104.08.sound29_5801200275d6 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin08.hostedemail.com (Postfix) with ESMTP id 8B1631819E76B for ; Wed, 3 Feb 2021 19:53:52 +0000 (UTC) X-HE-Tag: sound29_5801200275d6 X-Filterd-Recvd-Size: 8742 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Wed, 3 Feb 2021 19:53:51 +0000 (UTC) Received: by mail-wm1-f44.google.com with SMTP id e15so854432wme.0 for ; Wed, 03 Feb 2021 11:53:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=yeYVRDirmQQ5wl5j/KtVwzs1oXNB1UpSXJMPqylroD8=; b=PrG7fsvod/NdW0x6EITygzcKs0maUOZab7byusEFWwMulxeL9s4s45eFmyLgbahgZK L8MQvn//WnpvVobRbcm1gY5btgOSiWMwe+YvJBv/jkZMmI7NmoJKXj1Svk9y8N082iX9 qmAAE/VJmbAkgGci5vuK3MazRNomaM2AwQgFjf38G+kIvYVksJ5TLCQDxhwY40obK+5c AN+mVmCvgBV5oaV7ql4n8iMPiKphRmRpt9VDn1EEpw74FRxmLNZ8ecxEiyTvJsPSvrkR jGVPqUPY+39WNA7bC4kzm3fo65JAgxZPJqi8Q6g9BniWJtMWBTdOTcZoDibsW94F5ySd HJ5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=yeYVRDirmQQ5wl5j/KtVwzs1oXNB1UpSXJMPqylroD8=; b=dxaevRo6k54fp3uBpte6a6U15Hz4/Gk3R1VT2JLN1LfP9YFimfbnWXalSBGo5lyrsy K7bk33RzQyForhCtDU86VxXc4TF/SM5cEY0fxqoSbNMtLzfXaVrYBKi/mO0ysxQ+dr5v feNNo+5YwEb0oHvklgqFozsCJJsgwmZl6A55MKpT0FjIi5N8wuy8D5djq6hLm7BKtJYP 09arR8+RJ/tuQ6TjPD+mf4M2iXdZ4GqdZFFHIe/7vifQMKEkiRdebXG5jUuKtsdH9HZB ISmjLxXNjZrZ5BSsauHZsnfhgeeyXc8OFYILpavriRKZk05vMkqVz59bVx+bg6b4odX0 W/Qw== X-Gm-Message-State: AOAM533csA89BDWY0gDrDcBtHAEaH/SWLCDNnQh6fObNpmqYmP7/33/G 6m4OPVhPCdiy9io5jGONOHC9JvAJJdBJuEWB+jvNSA== X-Google-Smtp-Source: ABdhPJwHAkjE7WwvDKsGEeDVaItvRVpZmXyiUkxEOFNWYdPypIMM2i5ivMMV3z16KeF/g3UAP5Kq9CLgXvL+gCzihgQ= X-Received: by 2002:a05:600c:4f93:: with SMTP id n19mr4218419wmq.163.1612382029968; Wed, 03 Feb 2021 11:53:49 -0800 (PST) MIME-Version: 1.0 References: <20210203003134.2422308-1-surenb@google.com> <20210203003134.2422308-2-surenb@google.com> <1ea3d79a-2413-bba5-147e-e24df3f91ce0@amd.com> In-Reply-To: <1ea3d79a-2413-bba5-147e-e24df3f91ce0@amd.com> From: Suren Baghdasaryan Date: Wed, 3 Feb 2021 11:53:38 -0800 Message-ID: Subject: Re: [PATCH v2 2/2] dma-buf: heaps: Map system heap pages as managed by linux vm To: =?UTF-8?Q?Christian_K=C3=B6nig?= Cc: Minchan Kim , Sumit Semwal , Andrew Morton , Christoph Hellwig , Liam Mark , Laura Abbott , Brian Starkey , John Stultz , Chris Goldsworthy , =?UTF-8?Q?=C3=98rjan_Eide?= , Robin Murphy , James Jones , Hridya Valsaraju , Sandeep Patil , linux-media , DRI mailing list , "moderated list:DMA BUFFER SHARING FRAMEWORK" , linux-mm , LKML , kernel-team Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Wed, Feb 3, 2021 at 12:06 AM Christian K=C3=B6nig wrote: > > Am 03.02.21 um 03:02 schrieb Suren Baghdasaryan: > > On Tue, Feb 2, 2021 at 5:39 PM Minchan Kim wrote: > >> On Tue, Feb 02, 2021 at 04:31:34PM -0800, Suren Baghdasaryan wrote: > >>> Currently system heap maps its buffers with VM_PFNMAP flag using > >>> remap_pfn_range. This results in such buffers not being accounted > >>> for in PSS calculations because vm treats this memory as having no > >>> page structs. Without page structs there are no counters representing > >>> how many processes are mapping a page and therefore PSS calculation > >>> is impossible. > >>> Historically, ION driver used to map its buffers as VM_PFNMAP areas > >>> due to memory carveouts that did not have page structs [1]. That > >>> is not the case anymore and it seems there was desire to move away > >>> from remap_pfn_range [2]. > >>> Dmabuf system heap design inherits this ION behavior and maps its > >>> pages using remap_pfn_range even though allocated pages are backed > >>> by page structs. > >>> Replace remap_pfn_range with vm_insert_page, following Laura's sugges= tion > >>> in [1]. This would allow correct PSS calculation for dmabufs. > >>> > >>> [1] https://nam11.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F= %2Fdriverdev-devel.linuxdriverproject.narkive.com%2Fv0fJGpaD%2Fusing-ion-me= mory-for-direct-io&data=3D04%7C01%7Cchristian.koenig%40amd.com%7Cb4c145= b86dd0472c943c08d8c7e7ba4b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637= 479145389160353%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI= iLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DW1N%2B%2BlcFDaRSvXdSPe5h= PNMRByHfGkU7Uc3cmM3FCTU%3D&reserved=3D0 > >>> [2] https://nam11.safelinks.protection.outlook.com/?url=3Dhttp%3A%2F%= 2Fdriverdev.linuxdriverproject.org%2Fpipermail%2Fdriverdev-devel%2F2018-Oct= ober%2F127519.html&data=3D04%7C01%7Cchristian.koenig%40amd.com%7Cb4c145= b86dd0472c943c08d8c7e7ba4b%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637= 479145389160353%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzI= iLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DjQxSzKEr52lUcAIx%2FuBHMJ= 7yOgof%2FVMlW9%2BB2f%2FoS%2FE%3D&reserved=3D0 > >>> (sorry, could not find lore links for these discussions) > >>> > >>> Suggested-by: Laura Abbott > >>> Signed-off-by: Suren Baghdasaryan > >> Reviewed-by: Minchan Kim > >> > >> A note: This patch makes dmabuf system heap accounted as PSS so > >> if someone has relies on the size, they will see the bloat. > >> IIRC, there was some debate whether PSS accounting for their > >> buffer is correct or not. If it'd be a problem, we need to > >> discuss how to solve it(maybe, vma->vm_flags and reintroduce > >> remap_pfn_range for them to be respected). > > I did not see debates about not including *mapped* dmabufs into PSS > > calculation. I remember people were discussing how to account dmabufs > > referred only by the FD but that is a different discussion. If the > > buffer is mapped into the address space of a process then IMHO > > including it into PSS of that process is not controversial. > > Well, I think it is. And to be honest this doesn't looks like a good > idea to me since it will eventually lead to double accounting of system > heap DMA-bufs. Thanks for the comment! Could you please expand on this double accounting issue? Do you mean userspace could double account dmabufs because it expects dmabufs not to be part of PSS or is there some in-kernel accounting mechanism that would be broken by this? > > As discussed multiple times it is illegal to use the struct page of a > DMA-buf. This case here is a bit special since it is the owner of the > pages which does that, but I'm not sure if this won't cause problems > elsewhere as well. I would be happy to keep things as they are but calculating dmabuf contribution to PSS without struct pages is extremely inefficient and becomes a real pain when we consider the possibilities of partial mappings, when not the entire dmabuf is being mapped. Calculating this would require parsing /proc/pid/maps for the process, finding dmabuf mappings and the size for each one, then parsing /proc/pid/maps for ALL processes in the system to see if the same dmabufs are used by other processes and only then calculating the PSS. I hope that explains the desire to use already existing struct pages to obtain PSS in a much more efficient way. > > A more appropriate solution would be to held processes accountable for > resources they have allocated through device drivers. Are you suggesting some new kernel mechanism to account resources allocated by a process via a driver? If so, any details? > > Regards, > Christian. > > -- > To unsubscribe from this group and stop receiving emails from it, send an= email to kernel-team+unsubscribe@android.com. >