From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,HK_RANDOM_FROM,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA19FC4727D for ; Tue, 22 Sep 2020 16:14:00 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 413B523A1E for ; Tue, 22 Sep 2020 16:14:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 413B523A1E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6ECFA900061; Tue, 22 Sep 2020 12:13:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 69C5E900059; Tue, 22 Sep 2020 12:13:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 58B85900061; Tue, 22 Sep 2020 12:13:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0130.hostedemail.com [216.40.44.130]) by kanga.kvack.org (Postfix) with ESMTP id 3B133900059 for ; Tue, 22 Sep 2020 12:13:59 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id E7BB082499B9 for ; Tue, 22 Sep 2020 16:13:58 +0000 (UTC) X-FDA: 77291193756.27.foot25_0b021302714f Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin27.hostedemail.com (Postfix) with ESMTP id C02413D663 for ; Tue, 22 Sep 2020 16:13:58 +0000 (UTC) X-HE-Tag: foot25_0b021302714f X-Filterd-Recvd-Size: 7687 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf13.hostedemail.com (Postfix) with ESMTP for ; Tue, 22 Sep 2020 16:13:57 +0000 (UTC) IronPort-SDR: H8LdSeiFjZV6Lc7RkckRrDrit/KCoA1TpgvYPCAkhi37+SGcSaZIZBLx7hia16Q9N6xQtS1GwB JDlhEmdvb8Ww== X-IronPort-AV: E=McAfee;i="6000,8403,9752"; a="158020759" X-IronPort-AV: E=Sophos;i="5.77,291,1596524400"; d="scan'208";a="158020759" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Sep 2020 09:13:55 -0700 IronPort-SDR: W+rmWqa0xT/ewOCnu96ywZFFN7sCksDJXZ0B8m2eSjjHf9ai3YA0OAm303ILetoqVH5E8M1i2E QP+Qxj+EsOlg== X-IronPort-AV: E=Sophos;i="5.77,291,1596524400"; d="scan'208";a="454544739" Received: from atroib-mobl2.ger.corp.intel.com (HELO [10.214.238.184]) ([10.214.238.184]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Sep 2020 09:13:49 -0700 Subject: Re: [Intel-gfx] [PATCH 3/6] drm/i915: use vmap in shmem_pin_map To: Christoph Hellwig Cc: Matthew Wilcox , Juergen Gross , Stefano Stabellini , linux-mm@kvack.org, Peter Zijlstra , Boris Ostrovsky , x86@kernel.org, linux-kernel@vger.kernel.org, Minchan Kim , dri-devel@lists.freedesktop.org, xen-devel@lists.xenproject.org, Andrew Morton , intel-gfx@lists.freedesktop.org, Nitin Gupta , Chris Wilson , Matthew Auld References: <20200918163724.2511-1-hch@lst.de> <20200918163724.2511-4-hch@lst.de> <20200921191157.GX32101@casper.infradead.org> <20200922062249.GA30831@lst.de> <43d10588-2033-038b-14e4-9f41cd622d7b@linux.intel.com> <20200922143141.GA26637@lst.de> From: Tvrtko Ursulin Organization: Intel Corporation UK Plc Message-ID: Date: Tue, 22 Sep 2020 17:13:45 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <20200922143141.GA26637@lst.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On 22/09/2020 15:31, Christoph Hellwig wrote: > On Tue, Sep 22, 2020 at 09:23:59AM +0100, Tvrtko Ursulin wrote: >> If I understood this sub-thread correctly, iterating and freeing the pages >> via the vmapped ptes, so no need for a >> shmem_read_mapping_page_gfp loop in shmem_unpin_map looks plausible to me. >> >> I did not get the reference to kernel/dma/remap.c though, > > What I mean is the code in dma_common_find_pages, which returns the > page array for freeing. Got it. >> and also not sure >> how to do the error unwind path in shmem_pin_map at which point the >> allocated vm area hasn't been fully populated yet. Hand-roll the loop >> walking vm area struct in there? > > Yes. What I originally did (re-created as I didn't save it) would be > something like this: > > --- >>>From 5605e77cda246df6dd7ded99ec22cb3f341ef5d5 Mon Sep 17 00:00:00 2001 > From: Christoph Hellwig > Date: Wed, 16 Sep 2020 13:54:04 +0200 > Subject: drm/i915: use vmap in shmem_pin_map > > shmem_pin_map somewhat awkwardly reimplements vmap using > alloc_vm_area and manual pte setup. The only practical difference > is that alloc_vm_area prefeaults the vmalloc area PTEs, which doesn't > seem to be required here (and could be added to vmap using a flag > if actually required). > > Signed-off-by: Christoph Hellwig > --- > drivers/gpu/drm/i915/gt/shmem_utils.c | 81 +++++++++------------------ > 1 file changed, 27 insertions(+), 54 deletions(-) > > diff --git a/drivers/gpu/drm/i915/gt/shmem_utils.c b/drivers/gpu/drm/i915/gt/shmem_utils.c > index 43c7acbdc79dea..7ec6ba4c1065b2 100644 > --- a/drivers/gpu/drm/i915/gt/shmem_utils.c > +++ b/drivers/gpu/drm/i915/gt/shmem_utils.c > @@ -49,80 +49,53 @@ struct file *shmem_create_from_object(struct drm_i915_gem_object *obj) > return file; > } > > -static size_t shmem_npte(struct file *file) > +static size_t shmem_npages(struct file *file) > { > return file->f_mapping->host->i_size >> PAGE_SHIFT; > } > > -static void __shmem_unpin_map(struct file *file, void *ptr, size_t n_pte) > -{ > - unsigned long pfn; > - > - vunmap(ptr); > - > - for (pfn = 0; pfn < n_pte; pfn++) { > - struct page *page; > - > - page = shmem_read_mapping_page_gfp(file->f_mapping, pfn, > - GFP_KERNEL); > - if (!WARN_ON(IS_ERR(page))) { > - put_page(page); > - put_page(page); > - } > - } > -} > - > void *shmem_pin_map(struct file *file) > { > - const size_t n_pte = shmem_npte(file); > - pte_t *stack[32], **ptes, **mem; Chris can comment how much he'd miss the 32 page stack shortcut. > - struct vm_struct *area; > - unsigned long pfn; > - > - mem = stack; > - if (n_pte > ARRAY_SIZE(stack)) { > - mem = kvmalloc_array(n_pte, sizeof(*mem), GFP_KERNEL); > - if (!mem) > - return NULL; > - } > + size_t n_pages = shmem_npages(file), i; > + struct page **pages; > + void *vaddr; > > - area = alloc_vm_area(n_pte << PAGE_SHIFT, mem); > - if (!area) { > - if (mem != stack) > - kvfree(mem); > + pages = kvmalloc_array(n_pages, sizeof(*pages), GFP_KERNEL); > + if (!pages) > return NULL; > - } > - > - ptes = mem; > - for (pfn = 0; pfn < n_pte; pfn++) { > - struct page *page; > > - page = shmem_read_mapping_page_gfp(file->f_mapping, pfn, > - GFP_KERNEL); > - if (IS_ERR(page)) > + for (i = 0; i < n_pages; i++) { > + pages[i] = shmem_read_mapping_page_gfp(file->f_mapping, i, > + GFP_KERNEL); > + if (IS_ERR(pages[i])) > goto err_page; > - > - **ptes++ = mk_pte(page, PAGE_KERNEL); > } > > - if (mem != stack) > - kvfree(mem); > - > + vaddr = vmap(pages, n_pages, 0, PAGE_KERNEL); > + if (!vaddr) > + goto err_page; > mapping_set_unevictable(file->f_mapping); > - return area->addr; > - > + return vaddr; Is there something in vmap() preventing us from freeing the pages array here? I can't spot anything that is holding on to the pointer. Or it was just a sketch before you realized we could walk the vm_area? Also, I may be totally misunderstanding something, but I think you need to assign area->pages manually so shmem_unpin_map can access it below. > err_page: > - if (mem != stack) > - kvfree(mem); > - > - __shmem_unpin_map(file, area->addr, pfn); > + while (--i >= 0) > + put_page(pages[i]); > + kvfree(pages); > return NULL; > } > > void shmem_unpin_map(struct file *file, void *ptr) > { > + struct vm_struct *area = find_vm_area(ptr); > + size_t i = shmem_npages(file); > + > + if (WARN_ON_ONCE(!area || !area->pages)) > + return; > + > mapping_clear_unevictable(file->f_mapping); > - __shmem_unpin_map(file, ptr, shmem_npte(file)); > + for (i = 0; i < shmem_npages(file); i++) > + put_page(area->pages[i]); > + kvfree(area->pages); > + vunmap(ptr); Is the verdict from mm experts that we can't use vfree due __free_pages vs put_page differences? Could we get from ptes to pages, so that we don't have to keep the area->pages array allocated for the duration of the pin? Regards, Tvrtko > } > > static int __shmem_rw(struct file *file, loff_t off, > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,HK_RANDOM_FROM,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7937CC4363D for ; Tue, 22 Sep 2020 16:14:09 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2876C22206 for ; Tue, 22 Sep 2020 16:14:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2876C22206 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7098888FAE; Tue, 22 Sep 2020 16:14:05 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id 922B589DFA; Tue, 22 Sep 2020 16:13:56 +0000 (UTC) IronPort-SDR: qkr6VoB/egwcfCaNcsnXR7d/qcZmES8OyUZzODJUsOeT0is+x4xx6auq/uq1DUdhzwBHYkKJRS qrd8JFcAiu2w== X-IronPort-AV: E=McAfee;i="6000,8403,9752"; a="148304471" X-IronPort-AV: E=Sophos;i="5.77,291,1596524400"; d="scan'208";a="148304471" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Sep 2020 09:13:55 -0700 IronPort-SDR: W+rmWqa0xT/ewOCnu96ywZFFN7sCksDJXZ0B8m2eSjjHf9ai3YA0OAm303ILetoqVH5E8M1i2E QP+Qxj+EsOlg== X-IronPort-AV: E=Sophos;i="5.77,291,1596524400"; d="scan'208";a="454544739" Received: from atroib-mobl2.ger.corp.intel.com (HELO [10.214.238.184]) ([10.214.238.184]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Sep 2020 09:13:49 -0700 Subject: Re: [Intel-gfx] [PATCH 3/6] drm/i915: use vmap in shmem_pin_map To: Christoph Hellwig References: <20200918163724.2511-1-hch@lst.de> <20200918163724.2511-4-hch@lst.de> <20200921191157.GX32101@casper.infradead.org> <20200922062249.GA30831@lst.de> <43d10588-2033-038b-14e4-9f41cd622d7b@linux.intel.com> <20200922143141.GA26637@lst.de> From: Tvrtko Ursulin Organization: Intel Corporation UK Plc Message-ID: Date: Tue, 22 Sep 2020 17:13:45 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <20200922143141.GA26637@lst.de> Content-Language: en-US X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Minchan Kim , Peter Zijlstra , intel-gfx@lists.freedesktop.org, x86@kernel.org, linux-kernel@vger.kernel.org, Matthew Wilcox , Chris Wilson , linux-mm@kvack.org, dri-devel@lists.freedesktop.org, xen-devel@lists.xenproject.org, Boris Ostrovsky , Andrew Morton , Nitin Gupta , Matthew Auld Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On 22/09/2020 15:31, Christoph Hellwig wrote: > On Tue, Sep 22, 2020 at 09:23:59AM +0100, Tvrtko Ursulin wrote: >> If I understood this sub-thread correctly, iterating and freeing the pages >> via the vmapped ptes, so no need for a >> shmem_read_mapping_page_gfp loop in shmem_unpin_map looks plausible to me. >> >> I did not get the reference to kernel/dma/remap.c though, > > What I mean is the code in dma_common_find_pages, which returns the > page array for freeing. Got it. >> and also not sure >> how to do the error unwind path in shmem_pin_map at which point the >> allocated vm area hasn't been fully populated yet. Hand-roll the loop >> walking vm area struct in there? > > Yes. What I originally did (re-created as I didn't save it) would be > something like this: > > --- >>>From 5605e77cda246df6dd7ded99ec22cb3f341ef5d5 Mon Sep 17 00:00:00 2001 > From: Christoph Hellwig > Date: Wed, 16 Sep 2020 13:54:04 +0200 > Subject: drm/i915: use vmap in shmem_pin_map > > shmem_pin_map somewhat awkwardly reimplements vmap using > alloc_vm_area and manual pte setup. The only practical difference > is that alloc_vm_area prefeaults the vmalloc area PTEs, which doesn't > seem to be required here (and could be added to vmap using a flag > if actually required). > > Signed-off-by: Christoph Hellwig > --- > drivers/gpu/drm/i915/gt/shmem_utils.c | 81 +++++++++------------------ > 1 file changed, 27 insertions(+), 54 deletions(-) > > diff --git a/drivers/gpu/drm/i915/gt/shmem_utils.c b/drivers/gpu/drm/i915/gt/shmem_utils.c > index 43c7acbdc79dea..7ec6ba4c1065b2 100644 > --- a/drivers/gpu/drm/i915/gt/shmem_utils.c > +++ b/drivers/gpu/drm/i915/gt/shmem_utils.c > @@ -49,80 +49,53 @@ struct file *shmem_create_from_object(struct drm_i915_gem_object *obj) > return file; > } > > -static size_t shmem_npte(struct file *file) > +static size_t shmem_npages(struct file *file) > { > return file->f_mapping->host->i_size >> PAGE_SHIFT; > } > > -static void __shmem_unpin_map(struct file *file, void *ptr, size_t n_pte) > -{ > - unsigned long pfn; > - > - vunmap(ptr); > - > - for (pfn = 0; pfn < n_pte; pfn++) { > - struct page *page; > - > - page = shmem_read_mapping_page_gfp(file->f_mapping, pfn, > - GFP_KERNEL); > - if (!WARN_ON(IS_ERR(page))) { > - put_page(page); > - put_page(page); > - } > - } > -} > - > void *shmem_pin_map(struct file *file) > { > - const size_t n_pte = shmem_npte(file); > - pte_t *stack[32], **ptes, **mem; Chris can comment how much he'd miss the 32 page stack shortcut. > - struct vm_struct *area; > - unsigned long pfn; > - > - mem = stack; > - if (n_pte > ARRAY_SIZE(stack)) { > - mem = kvmalloc_array(n_pte, sizeof(*mem), GFP_KERNEL); > - if (!mem) > - return NULL; > - } > + size_t n_pages = shmem_npages(file), i; > + struct page **pages; > + void *vaddr; > > - area = alloc_vm_area(n_pte << PAGE_SHIFT, mem); > - if (!area) { > - if (mem != stack) > - kvfree(mem); > + pages = kvmalloc_array(n_pages, sizeof(*pages), GFP_KERNEL); > + if (!pages) > return NULL; > - } > - > - ptes = mem; > - for (pfn = 0; pfn < n_pte; pfn++) { > - struct page *page; > > - page = shmem_read_mapping_page_gfp(file->f_mapping, pfn, > - GFP_KERNEL); > - if (IS_ERR(page)) > + for (i = 0; i < n_pages; i++) { > + pages[i] = shmem_read_mapping_page_gfp(file->f_mapping, i, > + GFP_KERNEL); > + if (IS_ERR(pages[i])) > goto err_page; > - > - **ptes++ = mk_pte(page, PAGE_KERNEL); > } > > - if (mem != stack) > - kvfree(mem); > - > + vaddr = vmap(pages, n_pages, 0, PAGE_KERNEL); > + if (!vaddr) > + goto err_page; > mapping_set_unevictable(file->f_mapping); > - return area->addr; > - > + return vaddr; Is there something in vmap() preventing us from freeing the pages array here? I can't spot anything that is holding on to the pointer. Or it was just a sketch before you realized we could walk the vm_area? Also, I may be totally misunderstanding something, but I think you need to assign area->pages manually so shmem_unpin_map can access it below. > err_page: > - if (mem != stack) > - kvfree(mem); > - > - __shmem_unpin_map(file, area->addr, pfn); > + while (--i >= 0) > + put_page(pages[i]); > + kvfree(pages); > return NULL; > } > > void shmem_unpin_map(struct file *file, void *ptr) > { > + struct vm_struct *area = find_vm_area(ptr); > + size_t i = shmem_npages(file); > + > + if (WARN_ON_ONCE(!area || !area->pages)) > + return; > + > mapping_clear_unevictable(file->f_mapping); > - __shmem_unpin_map(file, ptr, shmem_npte(file)); > + for (i = 0; i < shmem_npages(file); i++) > + put_page(area->pages[i]); > + kvfree(area->pages); > + vunmap(ptr); Is the verdict from mm experts that we can't use vfree due __free_pages vs put_page differences? Could we get from ptes to pages, so that we don't have to keep the area->pages array allocated for the duration of the pin? Regards, Tvrtko > } > > static int __shmem_rw(struct file *file, loff_t off, > _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,HK_RANDOM_FROM,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD420C4741F for ; Tue, 22 Sep 2020 16:14:05 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5D3EB22206 for ; Tue, 22 Sep 2020 16:14:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5D3EB22206 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3B1B96E84B; Tue, 22 Sep 2020 16:14:03 +0000 (UTC) Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by gabe.freedesktop.org (Postfix) with ESMTPS id 922B589DFA; Tue, 22 Sep 2020 16:13:56 +0000 (UTC) IronPort-SDR: qkr6VoB/egwcfCaNcsnXR7d/qcZmES8OyUZzODJUsOeT0is+x4xx6auq/uq1DUdhzwBHYkKJRS qrd8JFcAiu2w== X-IronPort-AV: E=McAfee;i="6000,8403,9752"; a="148304471" X-IronPort-AV: E=Sophos;i="5.77,291,1596524400"; d="scan'208";a="148304471" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Sep 2020 09:13:55 -0700 IronPort-SDR: W+rmWqa0xT/ewOCnu96ywZFFN7sCksDJXZ0B8m2eSjjHf9ai3YA0OAm303ILetoqVH5E8M1i2E QP+Qxj+EsOlg== X-IronPort-AV: E=Sophos;i="5.77,291,1596524400"; d="scan'208";a="454544739" Received: from atroib-mobl2.ger.corp.intel.com (HELO [10.214.238.184]) ([10.214.238.184]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Sep 2020 09:13:49 -0700 To: Christoph Hellwig References: <20200918163724.2511-1-hch@lst.de> <20200918163724.2511-4-hch@lst.de> <20200921191157.GX32101@casper.infradead.org> <20200922062249.GA30831@lst.de> <43d10588-2033-038b-14e4-9f41cd622d7b@linux.intel.com> <20200922143141.GA26637@lst.de> From: Tvrtko Ursulin Organization: Intel Corporation UK Plc Message-ID: Date: Tue, 22 Sep 2020 17:13:45 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <20200922143141.GA26637@lst.de> Content-Language: en-US Subject: Re: [Intel-gfx] [PATCH 3/6] drm/i915: use vmap in shmem_pin_map X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Juergen Gross , Stefano Stabellini , Minchan Kim , Peter Zijlstra , intel-gfx@lists.freedesktop.org, x86@kernel.org, linux-kernel@vger.kernel.org, Matthew Wilcox , Chris Wilson , linux-mm@kvack.org, dri-devel@lists.freedesktop.org, xen-devel@lists.xenproject.org, Boris Ostrovsky , Andrew Morton , Nitin Gupta , Matthew Auld Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" On 22/09/2020 15:31, Christoph Hellwig wrote: > On Tue, Sep 22, 2020 at 09:23:59AM +0100, Tvrtko Ursulin wrote: >> If I understood this sub-thread correctly, iterating and freeing the pages >> via the vmapped ptes, so no need for a >> shmem_read_mapping_page_gfp loop in shmem_unpin_map looks plausible to me. >> >> I did not get the reference to kernel/dma/remap.c though, > > What I mean is the code in dma_common_find_pages, which returns the > page array for freeing. Got it. >> and also not sure >> how to do the error unwind path in shmem_pin_map at which point the >> allocated vm area hasn't been fully populated yet. Hand-roll the loop >> walking vm area struct in there? > > Yes. What I originally did (re-created as I didn't save it) would be > something like this: > > --- >>>From 5605e77cda246df6dd7ded99ec22cb3f341ef5d5 Mon Sep 17 00:00:00 2001 > From: Christoph Hellwig > Date: Wed, 16 Sep 2020 13:54:04 +0200 > Subject: drm/i915: use vmap in shmem_pin_map > > shmem_pin_map somewhat awkwardly reimplements vmap using > alloc_vm_area and manual pte setup. The only practical difference > is that alloc_vm_area prefeaults the vmalloc area PTEs, which doesn't > seem to be required here (and could be added to vmap using a flag > if actually required). > > Signed-off-by: Christoph Hellwig > --- > drivers/gpu/drm/i915/gt/shmem_utils.c | 81 +++++++++------------------ > 1 file changed, 27 insertions(+), 54 deletions(-) > > diff --git a/drivers/gpu/drm/i915/gt/shmem_utils.c b/drivers/gpu/drm/i915/gt/shmem_utils.c > index 43c7acbdc79dea..7ec6ba4c1065b2 100644 > --- a/drivers/gpu/drm/i915/gt/shmem_utils.c > +++ b/drivers/gpu/drm/i915/gt/shmem_utils.c > @@ -49,80 +49,53 @@ struct file *shmem_create_from_object(struct drm_i915_gem_object *obj) > return file; > } > > -static size_t shmem_npte(struct file *file) > +static size_t shmem_npages(struct file *file) > { > return file->f_mapping->host->i_size >> PAGE_SHIFT; > } > > -static void __shmem_unpin_map(struct file *file, void *ptr, size_t n_pte) > -{ > - unsigned long pfn; > - > - vunmap(ptr); > - > - for (pfn = 0; pfn < n_pte; pfn++) { > - struct page *page; > - > - page = shmem_read_mapping_page_gfp(file->f_mapping, pfn, > - GFP_KERNEL); > - if (!WARN_ON(IS_ERR(page))) { > - put_page(page); > - put_page(page); > - } > - } > -} > - > void *shmem_pin_map(struct file *file) > { > - const size_t n_pte = shmem_npte(file); > - pte_t *stack[32], **ptes, **mem; Chris can comment how much he'd miss the 32 page stack shortcut. > - struct vm_struct *area; > - unsigned long pfn; > - > - mem = stack; > - if (n_pte > ARRAY_SIZE(stack)) { > - mem = kvmalloc_array(n_pte, sizeof(*mem), GFP_KERNEL); > - if (!mem) > - return NULL; > - } > + size_t n_pages = shmem_npages(file), i; > + struct page **pages; > + void *vaddr; > > - area = alloc_vm_area(n_pte << PAGE_SHIFT, mem); > - if (!area) { > - if (mem != stack) > - kvfree(mem); > + pages = kvmalloc_array(n_pages, sizeof(*pages), GFP_KERNEL); > + if (!pages) > return NULL; > - } > - > - ptes = mem; > - for (pfn = 0; pfn < n_pte; pfn++) { > - struct page *page; > > - page = shmem_read_mapping_page_gfp(file->f_mapping, pfn, > - GFP_KERNEL); > - if (IS_ERR(page)) > + for (i = 0; i < n_pages; i++) { > + pages[i] = shmem_read_mapping_page_gfp(file->f_mapping, i, > + GFP_KERNEL); > + if (IS_ERR(pages[i])) > goto err_page; > - > - **ptes++ = mk_pte(page, PAGE_KERNEL); > } > > - if (mem != stack) > - kvfree(mem); > - > + vaddr = vmap(pages, n_pages, 0, PAGE_KERNEL); > + if (!vaddr) > + goto err_page; > mapping_set_unevictable(file->f_mapping); > - return area->addr; > - > + return vaddr; Is there something in vmap() preventing us from freeing the pages array here? I can't spot anything that is holding on to the pointer. Or it was just a sketch before you realized we could walk the vm_area? Also, I may be totally misunderstanding something, but I think you need to assign area->pages manually so shmem_unpin_map can access it below. > err_page: > - if (mem != stack) > - kvfree(mem); > - > - __shmem_unpin_map(file, area->addr, pfn); > + while (--i >= 0) > + put_page(pages[i]); > + kvfree(pages); > return NULL; > } > > void shmem_unpin_map(struct file *file, void *ptr) > { > + struct vm_struct *area = find_vm_area(ptr); > + size_t i = shmem_npages(file); > + > + if (WARN_ON_ONCE(!area || !area->pages)) > + return; > + > mapping_clear_unevictable(file->f_mapping); > - __shmem_unpin_map(file, ptr, shmem_npte(file)); > + for (i = 0; i < shmem_npages(file); i++) > + put_page(area->pages[i]); > + kvfree(area->pages); > + vunmap(ptr); Is the verdict from mm experts that we can't use vfree due __free_pages vs put_page differences? Could we get from ptes to pages, so that we don't have to keep the area->pages array allocated for the duration of the pin? Regards, Tvrtko > } > > static int __shmem_rw(struct file *file, loff_t off, > _______________________________________________ Intel-gfx mailing list Intel-gfx@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/intel-gfx