From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3198C2D0BF for ; Mon, 16 Dec 2019 22:29:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9D776206E0 for ; Mon, 16 Dec 2019 22:29:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="JXf9WyI8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726664AbfLPW32 (ORCPT ); Mon, 16 Dec 2019 17:29:28 -0500 Received: from hqnvemgate26.nvidia.com ([216.228.121.65]:11979 "EHLO hqnvemgate26.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727801AbfLPWZt (ORCPT ); Mon, 16 Dec 2019 17:25:49 -0500 Received: from hqpgpgate102.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Mon, 16 Dec 2019 14:25:31 -0800 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate102.nvidia.com (PGP Universal service); Mon, 16 Dec 2019 14:25:40 -0800 X-PGP-Universal: processed; by hqpgpgate102.nvidia.com on Mon, 16 Dec 2019 14:25:40 -0800 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 16 Dec 2019 22:25:39 +0000 Received: from HQMAIL107.nvidia.com (172.20.187.13) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 16 Dec 2019 22:25:39 +0000 Received: from hqnvemgw03.nvidia.com (10.124.88.68) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Mon, 16 Dec 2019 22:25:39 +0000 Received: from blueforge.nvidia.com (Not Verified[10.110.48.28]) by hqnvemgw03.nvidia.com with Trustwave SEG (v7,5,8,10121) id ; Mon, 16 Dec 2019 14:25:39 -0800 From: John Hubbard To: Andrew Morton CC: Al Viro , Alex Williamson , Benjamin Herrenschmidt , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Christoph Hellwig , Dan Williams , Daniel Vetter , Dave Chinner , David Airlie , "David S . Miller" , Ira Weiny , Jan Kara , Jason Gunthorpe , Jens Axboe , Jonathan Corbet , =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Magnus Karlsson , "Mauro Carvalho Chehab" , Michael Ellerman , Michal Hocko , Mike Kravetz , "Paul Mackerras" , Shuah Khan , Vlastimil Babka , , , , , , , , , , , , , LKML , John Hubbard , "Christoph Hellwig" , "Aneesh Kumar K . V" Subject: [PATCH v11 01/25] mm/gup: factor out duplicate code from four routines Date: Mon, 16 Dec 2019 14:25:13 -0800 Message-ID: <20191216222537.491123-2-jhubbard@nvidia.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191216222537.491123-1-jhubbard@nvidia.com> References: <20191216222537.491123-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1576535131; bh=HWyNjs6ywGvdqZPFXTnFG85letSRuTmGeFnSWXnBMB0=; h=X-PGP-Universal:From:To:CC:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:MIME-Version:X-NVConfidentiality: Content-Type:Content-Transfer-Encoding; b=JXf9WyI84yLWm1KspnvQLa5nDdAOS6RuWp1sE0TvmP1FtTCq2PQxdNrB19hslAYa/ NIsH/oLIq/YgsSflPBxJ/2530usBGdFv0BrfVUWaPVWxeNnWcWwDifcQlXGSiHyOFf ZW27GWt0ji60B6wfH4TfJ55oeagB+3WKWcw1kQ8P5Jtwre220oYwf3/qMVbLwWFL3Z UNmg5YpbxjZpyGozuGfQzxe4wyOm8tDVs5A9vsedBOg4KAEK/wxfief6tmAyrjRzbN bZ3IjOBm03aIJ/nyS+rDYvR5t0iZsx++4h1lgWxcWFfYomIts7kuBaRihCxUbseFlP uMogSAUw7CkxQ== Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org There are four locations in gup.c that have a fair amount of code duplication. This means that changing one requires making the same changes in four places, not to mention reading the same code four times, and wondering if there are subtle differences. Factor out the common code into static functions, thus reducing the overall line count and the code's complexity. Also, take the opportunity to slightly improve the efficiency of the error cases, by doing a mass subtraction of the refcount, surrounded by get_page()/put_page(). Also, further simplify (slightly), by waiting until the the successful end of each routine, to increment *nr. Reviewed-by: Christoph Hellwig Reviewed-by: J=C3=A9r=C3=B4me Glisse Reviewed-by: Jan Kara Cc: Ira Weiny Cc: Christoph Hellwig Cc: Aneesh Kumar K.V Signed-off-by: John Hubbard --- mm/gup.c | 91 ++++++++++++++++++++++---------------------------------- 1 file changed, 36 insertions(+), 55 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 7646bf993b25..f764432914c4 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1978,6 +1978,25 @@ static int __gup_device_huge_pud(pud_t pud, pud_t *p= udp, unsigned long addr, } #endif =20 +static int record_subpages(struct page *page, unsigned long addr, + unsigned long end, struct page **pages) +{ + int nr; + + for (nr =3D 0; addr !=3D end; addr +=3D PAGE_SIZE) + pages[nr++] =3D page++; + + return nr; +} + +static void put_compound_head(struct page *page, int refs) +{ + /* Do a get_page() first, in case refs =3D=3D page->_refcount */ + get_page(page); + page_ref_sub(page, refs); + put_page(page); +} + #ifdef CONFIG_ARCH_HAS_HUGEPD static unsigned long hugepte_addr_end(unsigned long addr, unsigned long en= d, unsigned long sz) @@ -2007,32 +2026,20 @@ static int gup_hugepte(pte_t *ptep, unsigned long s= z, unsigned long addr, /* hugepages are never "special" */ VM_BUG_ON(!pfn_valid(pte_pfn(pte))); =20 - refs =3D 0; head =3D pte_page(pte); - page =3D head + ((addr & (sz-1)) >> PAGE_SHIFT); - do { - VM_BUG_ON(compound_head(page) !=3D head); - pages[*nr] =3D page; - (*nr)++; - page++; - refs++; - } while (addr +=3D PAGE_SIZE, addr !=3D end); + refs =3D record_subpages(page, addr, end, pages + *nr); =20 head =3D try_get_compound_head(head, refs); - if (!head) { - *nr -=3D refs; + if (!head) return 0; - } =20 if (unlikely(pte_val(pte) !=3D pte_val(*ptep))) { - /* Could be optimized better */ - *nr -=3D refs; - while (refs--) - put_page(head); + put_compound_head(head, refs); return 0; } =20 + *nr +=3D refs; SetPageReferenced(head); return 1; } @@ -2079,28 +2086,19 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, un= signed long addr, return __gup_device_huge_pmd(orig, pmdp, addr, end, pages, nr); } =20 - refs =3D 0; page =3D pmd_page(orig) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); - do { - pages[*nr] =3D page; - (*nr)++; - page++; - refs++; - } while (addr +=3D PAGE_SIZE, addr !=3D end); + refs =3D record_subpages(page, addr, end, pages + *nr); =20 head =3D try_get_compound_head(pmd_page(orig), refs); - if (!head) { - *nr -=3D refs; + if (!head) return 0; - } =20 if (unlikely(pmd_val(orig) !=3D pmd_val(*pmdp))) { - *nr -=3D refs; - while (refs--) - put_page(head); + put_compound_head(head, refs); return 0; } =20 + *nr +=3D refs; SetPageReferenced(head); return 1; } @@ -2120,28 +2118,19 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, un= signed long addr, return __gup_device_huge_pud(orig, pudp, addr, end, pages, nr); } =20 - refs =3D 0; page =3D pud_page(orig) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); - do { - pages[*nr] =3D page; - (*nr)++; - page++; - refs++; - } while (addr +=3D PAGE_SIZE, addr !=3D end); + refs =3D record_subpages(page, addr, end, pages + *nr); =20 head =3D try_get_compound_head(pud_page(orig), refs); - if (!head) { - *nr -=3D refs; + if (!head) return 0; - } =20 if (unlikely(pud_val(orig) !=3D pud_val(*pudp))) { - *nr -=3D refs; - while (refs--) - put_page(head); + put_compound_head(head, refs); return 0; } =20 + *nr +=3D refs; SetPageReferenced(head); return 1; } @@ -2157,28 +2146,20 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, un= signed long addr, return 0; =20 BUILD_BUG_ON(pgd_devmap(orig)); - refs =3D 0; + page =3D pgd_page(orig) + ((addr & ~PGDIR_MASK) >> PAGE_SHIFT); - do { - pages[*nr] =3D page; - (*nr)++; - page++; - refs++; - } while (addr +=3D PAGE_SIZE, addr !=3D end); + refs =3D record_subpages(page, addr, end, pages + *nr); =20 head =3D try_get_compound_head(pgd_page(orig), refs); - if (!head) { - *nr -=3D refs; + if (!head) return 0; - } =20 if (unlikely(pgd_val(orig) !=3D pgd_val(*pgdp))) { - *nr -=3D refs; - while (refs--) - put_page(head); + put_compound_head(head, refs); return 0; } =20 + *nr +=3D refs; SetPageReferenced(head); return 1; } --=20 2.24.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4BC97C43603 for ; Mon, 16 Dec 2019 22:42:34 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EE24E22464 for ; Mon, 16 Dec 2019 22:42:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="JXf9WyI8" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EE24E22464 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 47cGVv55mGzDqSZ for ; Tue, 17 Dec 2019 09:42:31 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nvidia.com (client-ip=216.228.121.65; helo=hqnvemgate26.nvidia.com; envelope-from=jhubbard@nvidia.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=nvidia.com Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=nvidia.com header.i=@nvidia.com header.b="JXf9WyI8"; dkim-atps=neutral Received: from hqnvemgate26.nvidia.com (hqnvemgate26.nvidia.com [216.228.121.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 47cG7Z5g4SzDqF0 for ; Tue, 17 Dec 2019 09:25:46 +1100 (AEDT) Received: from hqpgpgate102.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Mon, 16 Dec 2019 14:25:31 -0800 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate102.nvidia.com (PGP Universal service); Mon, 16 Dec 2019 14:25:40 -0800 X-PGP-Universal: processed; by hqpgpgate102.nvidia.com on Mon, 16 Dec 2019 14:25:40 -0800 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 16 Dec 2019 22:25:39 +0000 Received: from HQMAIL107.nvidia.com (172.20.187.13) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 16 Dec 2019 22:25:39 +0000 Received: from hqnvemgw03.nvidia.com (10.124.88.68) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Mon, 16 Dec 2019 22:25:39 +0000 Received: from blueforge.nvidia.com (Not Verified[10.110.48.28]) by hqnvemgw03.nvidia.com with Trustwave SEG (v7, 5, 8, 10121) id ; Mon, 16 Dec 2019 14:25:39 -0800 From: John Hubbard To: Andrew Morton Subject: [PATCH v11 01/25] mm/gup: factor out duplicate code from four routines Date: Mon, 16 Dec 2019 14:25:13 -0800 Message-ID: <20191216222537.491123-2-jhubbard@nvidia.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191216222537.491123-1-jhubbard@nvidia.com> References: <20191216222537.491123-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1576535131; bh=HWyNjs6ywGvdqZPFXTnFG85letSRuTmGeFnSWXnBMB0=; h=X-PGP-Universal:From:To:CC:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:MIME-Version:X-NVConfidentiality: Content-Type:Content-Transfer-Encoding; b=JXf9WyI84yLWm1KspnvQLa5nDdAOS6RuWp1sE0TvmP1FtTCq2PQxdNrB19hslAYa/ NIsH/oLIq/YgsSflPBxJ/2530usBGdFv0BrfVUWaPVWxeNnWcWwDifcQlXGSiHyOFf ZW27GWt0ji60B6wfH4TfJ55oeagB+3WKWcw1kQ8P5Jtwre220oYwf3/qMVbLwWFL3Z UNmg5YpbxjZpyGozuGfQzxe4wyOm8tDVs5A9vsedBOg4KAEK/wxfief6tmAyrjRzbN bZ3IjOBm03aIJ/nyS+rDYvR5t0iZsx++4h1lgWxcWFfYomIts7kuBaRihCxUbseFlP uMogSAUw7CkxQ== X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michal Hocko , Jan Kara , kvm@vger.kernel.org, linux-doc@vger.kernel.org, David Airlie , Dave Chinner , dri-devel@lists.freedesktop.org, LKML , linux-mm@kvack.org, Paul Mackerras , linux-kselftest@vger.kernel.org, Ira Weiny , Christoph Hellwig , Jonathan Corbet , linux-rdma@vger.kernel.org, Christoph Hellwig , Jason Gunthorpe , Vlastimil Babka , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , linux-media@vger.kernel.org, Shuah Khan , John Hubbard , linux-block@vger.kernel.org, =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Al Viro , Dan Williams , Mauro Carvalho Chehab , Magnus Karlsson , Jens Axboe , netdev@vger.kernel.org, Alex Williamson , Daniel Vetter , "Aneesh Kumar K . V" , linux-fsdevel@vger.kernel.org, bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, "David S . Miller" , Mike Kravetz Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" There are four locations in gup.c that have a fair amount of code duplication. This means that changing one requires making the same changes in four places, not to mention reading the same code four times, and wondering if there are subtle differences. Factor out the common code into static functions, thus reducing the overall line count and the code's complexity. Also, take the opportunity to slightly improve the efficiency of the error cases, by doing a mass subtraction of the refcount, surrounded by get_page()/put_page(). Also, further simplify (slightly), by waiting until the the successful end of each routine, to increment *nr. Reviewed-by: Christoph Hellwig Reviewed-by: J=C3=A9r=C3=B4me Glisse Reviewed-by: Jan Kara Cc: Ira Weiny Cc: Christoph Hellwig Cc: Aneesh Kumar K.V Signed-off-by: John Hubbard --- mm/gup.c | 91 ++++++++++++++++++++++---------------------------------- 1 file changed, 36 insertions(+), 55 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index 7646bf993b25..f764432914c4 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -1978,6 +1978,25 @@ static int __gup_device_huge_pud(pud_t pud, pud_t *p= udp, unsigned long addr, } #endif =20 +static int record_subpages(struct page *page, unsigned long addr, + unsigned long end, struct page **pages) +{ + int nr; + + for (nr =3D 0; addr !=3D end; addr +=3D PAGE_SIZE) + pages[nr++] =3D page++; + + return nr; +} + +static void put_compound_head(struct page *page, int refs) +{ + /* Do a get_page() first, in case refs =3D=3D page->_refcount */ + get_page(page); + page_ref_sub(page, refs); + put_page(page); +} + #ifdef CONFIG_ARCH_HAS_HUGEPD static unsigned long hugepte_addr_end(unsigned long addr, unsigned long en= d, unsigned long sz) @@ -2007,32 +2026,20 @@ static int gup_hugepte(pte_t *ptep, unsigned long s= z, unsigned long addr, /* hugepages are never "special" */ VM_BUG_ON(!pfn_valid(pte_pfn(pte))); =20 - refs =3D 0; head =3D pte_page(pte); - page =3D head + ((addr & (sz-1)) >> PAGE_SHIFT); - do { - VM_BUG_ON(compound_head(page) !=3D head); - pages[*nr] =3D page; - (*nr)++; - page++; - refs++; - } while (addr +=3D PAGE_SIZE, addr !=3D end); + refs =3D record_subpages(page, addr, end, pages + *nr); =20 head =3D try_get_compound_head(head, refs); - if (!head) { - *nr -=3D refs; + if (!head) return 0; - } =20 if (unlikely(pte_val(pte) !=3D pte_val(*ptep))) { - /* Could be optimized better */ - *nr -=3D refs; - while (refs--) - put_page(head); + put_compound_head(head, refs); return 0; } =20 + *nr +=3D refs; SetPageReferenced(head); return 1; } @@ -2079,28 +2086,19 @@ static int gup_huge_pmd(pmd_t orig, pmd_t *pmdp, un= signed long addr, return __gup_device_huge_pmd(orig, pmdp, addr, end, pages, nr); } =20 - refs =3D 0; page =3D pmd_page(orig) + ((addr & ~PMD_MASK) >> PAGE_SHIFT); - do { - pages[*nr] =3D page; - (*nr)++; - page++; - refs++; - } while (addr +=3D PAGE_SIZE, addr !=3D end); + refs =3D record_subpages(page, addr, end, pages + *nr); =20 head =3D try_get_compound_head(pmd_page(orig), refs); - if (!head) { - *nr -=3D refs; + if (!head) return 0; - } =20 if (unlikely(pmd_val(orig) !=3D pmd_val(*pmdp))) { - *nr -=3D refs; - while (refs--) - put_page(head); + put_compound_head(head, refs); return 0; } =20 + *nr +=3D refs; SetPageReferenced(head); return 1; } @@ -2120,28 +2118,19 @@ static int gup_huge_pud(pud_t orig, pud_t *pudp, un= signed long addr, return __gup_device_huge_pud(orig, pudp, addr, end, pages, nr); } =20 - refs =3D 0; page =3D pud_page(orig) + ((addr & ~PUD_MASK) >> PAGE_SHIFT); - do { - pages[*nr] =3D page; - (*nr)++; - page++; - refs++; - } while (addr +=3D PAGE_SIZE, addr !=3D end); + refs =3D record_subpages(page, addr, end, pages + *nr); =20 head =3D try_get_compound_head(pud_page(orig), refs); - if (!head) { - *nr -=3D refs; + if (!head) return 0; - } =20 if (unlikely(pud_val(orig) !=3D pud_val(*pudp))) { - *nr -=3D refs; - while (refs--) - put_page(head); + put_compound_head(head, refs); return 0; } =20 + *nr +=3D refs; SetPageReferenced(head); return 1; } @@ -2157,28 +2146,20 @@ static int gup_huge_pgd(pgd_t orig, pgd_t *pgdp, un= signed long addr, return 0; =20 BUILD_BUG_ON(pgd_devmap(orig)); - refs =3D 0; + page =3D pgd_page(orig) + ((addr & ~PGDIR_MASK) >> PAGE_SHIFT); - do { - pages[*nr] =3D page; - (*nr)++; - page++; - refs++; - } while (addr +=3D PAGE_SIZE, addr !=3D end); + refs =3D record_subpages(page, addr, end, pages + *nr); =20 head =3D try_get_compound_head(pgd_page(orig), refs); - if (!head) { - *nr -=3D refs; + if (!head) return 0; - } =20 if (unlikely(pgd_val(orig) !=3D pgd_val(*pgdp))) { - *nr -=3D refs; - while (refs--) - put_page(head); + put_compound_head(head, refs); return 0; } =20 + *nr +=3D refs; SetPageReferenced(head); return 1; } --=20 2.24.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52613C2D0D3 for ; Mon, 16 Dec 2019 22:25:51 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 248AD21835 for ; Mon, 16 Dec 2019 22:25:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (2048-bit key) header.d=nvidia.com header.i=@nvidia.com header.b="jyMh2uZ9" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 248AD21835 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nvidia.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 468816E8C5; Mon, 16 Dec 2019 22:25:42 +0000 (UTC) Received: from hqnvemgate26.nvidia.com (hqnvemgate26.nvidia.com [216.228.121.65]) by gabe.freedesktop.org (Postfix) with ESMTPS id 013606E8C0 for ; Mon, 16 Dec 2019 22:25:40 +0000 (UTC) Received: from hqpgpgate102.nvidia.com (Not Verified[216.228.121.13]) by hqnvemgate26.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Mon, 16 Dec 2019 14:25:31 -0800 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate102.nvidia.com (PGP Universal service); Mon, 16 Dec 2019 14:25:40 -0800 X-PGP-Universal: processed; by hqpgpgate102.nvidia.com on Mon, 16 Dec 2019 14:25:40 -0800 Received: from HQMAIL109.nvidia.com (172.20.187.15) by HQMAIL101.nvidia.com (172.20.187.10) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 16 Dec 2019 22:25:39 +0000 Received: from HQMAIL107.nvidia.com (172.20.187.13) by HQMAIL109.nvidia.com (172.20.187.15) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Mon, 16 Dec 2019 22:25:39 +0000 Received: from hqnvemgw03.nvidia.com (10.124.88.68) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3 via Frontend Transport; Mon, 16 Dec 2019 22:25:39 +0000 Received: from blueforge.nvidia.com (Not Verified[10.110.48.28]) by hqnvemgw03.nvidia.com with Trustwave SEG (v7, 5, 8, 10121) id ; Mon, 16 Dec 2019 14:25:39 -0800 From: John Hubbard To: Andrew Morton Subject: [PATCH v11 01/25] mm/gup: factor out duplicate code from four routines Date: Mon, 16 Dec 2019 14:25:13 -0800 Message-ID: <20191216222537.491123-2-jhubbard@nvidia.com> X-Mailer: git-send-email 2.24.1 In-Reply-To: <20191216222537.491123-1-jhubbard@nvidia.com> References: <20191216222537.491123-1-jhubbard@nvidia.com> MIME-Version: 1.0 X-NVConfidentiality: public DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1576535132; bh=HWyNjs6ywGvdqZPFXTnFG85letSRuTmGeFnSWXnBMB0=; h=X-PGP-Universal:From:To:CC:Subject:Date:Message-ID:X-Mailer: In-Reply-To:References:MIME-Version:X-NVConfidentiality: Content-Type:Content-Transfer-Encoding; b=jyMh2uZ9DlLbmgf/hFXACBgJky8gUF15EUU0ckvOLpJ3PY+P8ESRv6RR73lI5W7L9 1PPPr3nG8d3+imdOwWybdn7nP0qyznDHLsP2ZO1TevAlqPHRqADEE+MTmGtoaTePpa jLbl/hf5tnMu1SfkRZVDLQ+JeUuhb/CgzH/bzkPW5YPIAY6Y+7oVkVtw329jSZFlQI e4Qg6Vp66fdhSULbEcKBKmmxnd6i8dWuDDIQoMlk8YqiOF91oKfMhb25c4lMcNIQMJ lbcy1BRDCWqRZFcaV9a1urI4sC+BeDUETZ69bHvno8lpaKjFgxcEjKLNbs6AumfxI1 3a1KQhm8Dq+qw== X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michal Hocko , Jan Kara , kvm@vger.kernel.org, linux-doc@vger.kernel.org, David Airlie , Dave Chinner , dri-devel@lists.freedesktop.org, LKML , linux-mm@kvack.org, Paul Mackerras , linux-kselftest@vger.kernel.org, Ira Weiny , Christoph Hellwig , Jonathan Corbet , linux-rdma@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Jason Gunthorpe , Vlastimil Babka , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , linux-media@vger.kernel.org, Shuah Khan , John Hubbard , linux-block@vger.kernel.org, =?UTF-8?q?J=C3=A9r=C3=B4me=20Glisse?= , Al Viro , Dan Williams , Mauro Carvalho Chehab , Magnus Karlsson , Jens Axboe , netdev@vger.kernel.org, Alex Williamson , "Aneesh Kumar K . V" , linux-fsdevel@vger.kernel.org, bpf@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, "David S . Miller" , Mike Kravetz Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" VGhlcmUgYXJlIGZvdXIgbG9jYXRpb25zIGluIGd1cC5jIHRoYXQgaGF2ZSBhIGZhaXIgYW1vdW50 IG9mIGNvZGUKZHVwbGljYXRpb24uIFRoaXMgbWVhbnMgdGhhdCBjaGFuZ2luZyBvbmUgcmVxdWly ZXMgbWFraW5nIHRoZSBzYW1lCmNoYW5nZXMgaW4gZm91ciBwbGFjZXMsIG5vdCB0byBtZW50aW9u IHJlYWRpbmcgdGhlIHNhbWUgY29kZSBmb3VyCnRpbWVzLCBhbmQgd29uZGVyaW5nIGlmIHRoZXJl IGFyZSBzdWJ0bGUgZGlmZmVyZW5jZXMuCgpGYWN0b3Igb3V0IHRoZSBjb21tb24gY29kZSBpbnRv IHN0YXRpYyBmdW5jdGlvbnMsIHRodXMgcmVkdWNpbmcgdGhlCm92ZXJhbGwgbGluZSBjb3VudCBh bmQgdGhlIGNvZGUncyBjb21wbGV4aXR5LgoKQWxzbywgdGFrZSB0aGUgb3Bwb3J0dW5pdHkgdG8g c2xpZ2h0bHkgaW1wcm92ZSB0aGUgZWZmaWNpZW5jeSBvZiB0aGUKZXJyb3IgY2FzZXMsIGJ5IGRv aW5nIGEgbWFzcyBzdWJ0cmFjdGlvbiBvZiB0aGUgcmVmY291bnQsIHN1cnJvdW5kZWQKYnkgZ2V0 X3BhZ2UoKS9wdXRfcGFnZSgpLgoKQWxzbywgZnVydGhlciBzaW1wbGlmeSAoc2xpZ2h0bHkpLCBi eSB3YWl0aW5nIHVudGlsIHRoZSB0aGUgc3VjY2Vzc2Z1bAplbmQgb2YgZWFjaCByb3V0aW5lLCB0 byBpbmNyZW1lbnQgKm5yLgoKUmV2aWV3ZWQtYnk6IENocmlzdG9waCBIZWxsd2lnIDxoY2hAbHN0 LmRlPgpSZXZpZXdlZC1ieTogSsOpcsO0bWUgR2xpc3NlIDxqZ2xpc3NlQHJlZGhhdC5jb20+ClJl dmlld2VkLWJ5OiBKYW4gS2FyYSA8amFja0BzdXNlLmN6PgpDYzogSXJhIFdlaW55IDxpcmEud2Vp bnlAaW50ZWwuY29tPgpDYzogQ2hyaXN0b3BoIEhlbGx3aWcgPGhjaEBsc3QuZGU+CkNjOiBBbmVl c2ggS3VtYXIgSy5WIDxhbmVlc2gua3VtYXJAbGludXguaWJtLmNvbT4KU2lnbmVkLW9mZi1ieTog Sm9obiBIdWJiYXJkIDxqaHViYmFyZEBudmlkaWEuY29tPgotLS0KIG1tL2d1cC5jIHwgOTEgKysr KysrKysrKysrKysrKysrKysrKy0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0KIDEg ZmlsZSBjaGFuZ2VkLCAzNiBpbnNlcnRpb25zKCspLCA1NSBkZWxldGlvbnMoLSkKCmRpZmYgLS1n aXQgYS9tbS9ndXAuYyBiL21tL2d1cC5jCmluZGV4IDc2NDZiZjk5M2IyNS4uZjc2NDQzMjkxNGM0 IDEwMDY0NAotLS0gYS9tbS9ndXAuYworKysgYi9tbS9ndXAuYwpAQCAtMTk3OCw2ICsxOTc4LDI1 IEBAIHN0YXRpYyBpbnQgX19ndXBfZGV2aWNlX2h1Z2VfcHVkKHB1ZF90IHB1ZCwgcHVkX3QgKnB1 ZHAsIHVuc2lnbmVkIGxvbmcgYWRkciwKIH0KICNlbmRpZgogCitzdGF0aWMgaW50IHJlY29yZF9z dWJwYWdlcyhzdHJ1Y3QgcGFnZSAqcGFnZSwgdW5zaWduZWQgbG9uZyBhZGRyLAorCQkJICAgdW5z aWduZWQgbG9uZyBlbmQsIHN0cnVjdCBwYWdlICoqcGFnZXMpCit7CisJaW50IG5yOworCisJZm9y IChuciA9IDA7IGFkZHIgIT0gZW5kOyBhZGRyICs9IFBBR0VfU0laRSkKKwkJcGFnZXNbbnIrK10g PSBwYWdlKys7CisKKwlyZXR1cm4gbnI7Cit9CisKK3N0YXRpYyB2b2lkIHB1dF9jb21wb3VuZF9o ZWFkKHN0cnVjdCBwYWdlICpwYWdlLCBpbnQgcmVmcykKK3sKKwkvKiBEbyBhIGdldF9wYWdlKCkg Zmlyc3QsIGluIGNhc2UgcmVmcyA9PSBwYWdlLT5fcmVmY291bnQgKi8KKwlnZXRfcGFnZShwYWdl KTsKKwlwYWdlX3JlZl9zdWIocGFnZSwgcmVmcyk7CisJcHV0X3BhZ2UocGFnZSk7Cit9CisKICNp ZmRlZiBDT05GSUdfQVJDSF9IQVNfSFVHRVBECiBzdGF0aWMgdW5zaWduZWQgbG9uZyBodWdlcHRl X2FkZHJfZW5kKHVuc2lnbmVkIGxvbmcgYWRkciwgdW5zaWduZWQgbG9uZyBlbmQsCiAJCQkJICAg ICAgdW5zaWduZWQgbG9uZyBzeikKQEAgLTIwMDcsMzIgKzIwMjYsMjAgQEAgc3RhdGljIGludCBn dXBfaHVnZXB0ZShwdGVfdCAqcHRlcCwgdW5zaWduZWQgbG9uZyBzeiwgdW5zaWduZWQgbG9uZyBh ZGRyLAogCS8qIGh1Z2VwYWdlcyBhcmUgbmV2ZXIgInNwZWNpYWwiICovCiAJVk1fQlVHX09OKCFw Zm5fdmFsaWQocHRlX3BmbihwdGUpKSk7CiAKLQlyZWZzID0gMDsKIAloZWFkID0gcHRlX3BhZ2Uo cHRlKTsKLQogCXBhZ2UgPSBoZWFkICsgKChhZGRyICYgKHN6LTEpKSA+PiBQQUdFX1NISUZUKTsK LQlkbyB7Ci0JCVZNX0JVR19PTihjb21wb3VuZF9oZWFkKHBhZ2UpICE9IGhlYWQpOwotCQlwYWdl c1sqbnJdID0gcGFnZTsKLQkJKCpucikrKzsKLQkJcGFnZSsrOwotCQlyZWZzKys7Ci0JfSB3aGls ZSAoYWRkciArPSBQQUdFX1NJWkUsIGFkZHIgIT0gZW5kKTsKKwlyZWZzID0gcmVjb3JkX3N1YnBh Z2VzKHBhZ2UsIGFkZHIsIGVuZCwgcGFnZXMgKyAqbnIpOwogCiAJaGVhZCA9IHRyeV9nZXRfY29t cG91bmRfaGVhZChoZWFkLCByZWZzKTsKLQlpZiAoIWhlYWQpIHsKLQkJKm5yIC09IHJlZnM7CisJ aWYgKCFoZWFkKQogCQlyZXR1cm4gMDsKLQl9CiAKIAlpZiAodW5saWtlbHkocHRlX3ZhbChwdGUp ICE9IHB0ZV92YWwoKnB0ZXApKSkgewotCQkvKiBDb3VsZCBiZSBvcHRpbWl6ZWQgYmV0dGVyICov Ci0JCSpuciAtPSByZWZzOwotCQl3aGlsZSAocmVmcy0tKQotCQkJcHV0X3BhZ2UoaGVhZCk7CisJ CXB1dF9jb21wb3VuZF9oZWFkKGhlYWQsIHJlZnMpOwogCQlyZXR1cm4gMDsKIAl9CiAKKwkqbnIg Kz0gcmVmczsKIAlTZXRQYWdlUmVmZXJlbmNlZChoZWFkKTsKIAlyZXR1cm4gMTsKIH0KQEAgLTIw NzksMjggKzIwODYsMTkgQEAgc3RhdGljIGludCBndXBfaHVnZV9wbWQocG1kX3Qgb3JpZywgcG1k X3QgKnBtZHAsIHVuc2lnbmVkIGxvbmcgYWRkciwKIAkJcmV0dXJuIF9fZ3VwX2RldmljZV9odWdl X3BtZChvcmlnLCBwbWRwLCBhZGRyLCBlbmQsIHBhZ2VzLCBucik7CiAJfQogCi0JcmVmcyA9IDA7 CiAJcGFnZSA9IHBtZF9wYWdlKG9yaWcpICsgKChhZGRyICYgflBNRF9NQVNLKSA+PiBQQUdFX1NI SUZUKTsKLQlkbyB7Ci0JCXBhZ2VzWypucl0gPSBwYWdlOwotCQkoKm5yKSsrOwotCQlwYWdlKys7 Ci0JCXJlZnMrKzsKLQl9IHdoaWxlIChhZGRyICs9IFBBR0VfU0laRSwgYWRkciAhPSBlbmQpOwor CXJlZnMgPSByZWNvcmRfc3VicGFnZXMocGFnZSwgYWRkciwgZW5kLCBwYWdlcyArICpucik7CiAK IAloZWFkID0gdHJ5X2dldF9jb21wb3VuZF9oZWFkKHBtZF9wYWdlKG9yaWcpLCByZWZzKTsKLQlp ZiAoIWhlYWQpIHsKLQkJKm5yIC09IHJlZnM7CisJaWYgKCFoZWFkKQogCQlyZXR1cm4gMDsKLQl9 CiAKIAlpZiAodW5saWtlbHkocG1kX3ZhbChvcmlnKSAhPSBwbWRfdmFsKCpwbWRwKSkpIHsKLQkJ Km5yIC09IHJlZnM7Ci0JCXdoaWxlIChyZWZzLS0pCi0JCQlwdXRfcGFnZShoZWFkKTsKKwkJcHV0 X2NvbXBvdW5kX2hlYWQoaGVhZCwgcmVmcyk7CiAJCXJldHVybiAwOwogCX0KIAorCSpuciArPSBy ZWZzOwogCVNldFBhZ2VSZWZlcmVuY2VkKGhlYWQpOwogCXJldHVybiAxOwogfQpAQCAtMjEyMCwy OCArMjExOCwxOSBAQCBzdGF0aWMgaW50IGd1cF9odWdlX3B1ZChwdWRfdCBvcmlnLCBwdWRfdCAq cHVkcCwgdW5zaWduZWQgbG9uZyBhZGRyLAogCQlyZXR1cm4gX19ndXBfZGV2aWNlX2h1Z2VfcHVk KG9yaWcsIHB1ZHAsIGFkZHIsIGVuZCwgcGFnZXMsIG5yKTsKIAl9CiAKLQlyZWZzID0gMDsKIAlw YWdlID0gcHVkX3BhZ2Uob3JpZykgKyAoKGFkZHIgJiB+UFVEX01BU0spID4+IFBBR0VfU0hJRlQp OwotCWRvIHsKLQkJcGFnZXNbKm5yXSA9IHBhZ2U7Ci0JCSgqbnIpKys7Ci0JCXBhZ2UrKzsKLQkJ cmVmcysrOwotCX0gd2hpbGUgKGFkZHIgKz0gUEFHRV9TSVpFLCBhZGRyICE9IGVuZCk7CisJcmVm cyA9IHJlY29yZF9zdWJwYWdlcyhwYWdlLCBhZGRyLCBlbmQsIHBhZ2VzICsgKm5yKTsKIAogCWhl YWQgPSB0cnlfZ2V0X2NvbXBvdW5kX2hlYWQocHVkX3BhZ2Uob3JpZyksIHJlZnMpOwotCWlmICgh aGVhZCkgewotCQkqbnIgLT0gcmVmczsKKwlpZiAoIWhlYWQpCiAJCXJldHVybiAwOwotCX0KIAog CWlmICh1bmxpa2VseShwdWRfdmFsKG9yaWcpICE9IHB1ZF92YWwoKnB1ZHApKSkgewotCQkqbnIg LT0gcmVmczsKLQkJd2hpbGUgKHJlZnMtLSkKLQkJCXB1dF9wYWdlKGhlYWQpOworCQlwdXRfY29t cG91bmRfaGVhZChoZWFkLCByZWZzKTsKIAkJcmV0dXJuIDA7CiAJfQogCisJKm5yICs9IHJlZnM7 CiAJU2V0UGFnZVJlZmVyZW5jZWQoaGVhZCk7CiAJcmV0dXJuIDE7CiB9CkBAIC0yMTU3LDI4ICsy MTQ2LDIwIEBAIHN0YXRpYyBpbnQgZ3VwX2h1Z2VfcGdkKHBnZF90IG9yaWcsIHBnZF90ICpwZ2Rw LCB1bnNpZ25lZCBsb25nIGFkZHIsCiAJCXJldHVybiAwOwogCiAJQlVJTERfQlVHX09OKHBnZF9k ZXZtYXAob3JpZykpOwotCXJlZnMgPSAwOworCiAJcGFnZSA9IHBnZF9wYWdlKG9yaWcpICsgKChh ZGRyICYgflBHRElSX01BU0spID4+IFBBR0VfU0hJRlQpOwotCWRvIHsKLQkJcGFnZXNbKm5yXSA9 IHBhZ2U7Ci0JCSgqbnIpKys7Ci0JCXBhZ2UrKzsKLQkJcmVmcysrOwotCX0gd2hpbGUgKGFkZHIg Kz0gUEFHRV9TSVpFLCBhZGRyICE9IGVuZCk7CisJcmVmcyA9IHJlY29yZF9zdWJwYWdlcyhwYWdl LCBhZGRyLCBlbmQsIHBhZ2VzICsgKm5yKTsKIAogCWhlYWQgPSB0cnlfZ2V0X2NvbXBvdW5kX2hl YWQocGdkX3BhZ2Uob3JpZyksIHJlZnMpOwotCWlmICghaGVhZCkgewotCQkqbnIgLT0gcmVmczsK KwlpZiAoIWhlYWQpCiAJCXJldHVybiAwOwotCX0KIAogCWlmICh1bmxpa2VseShwZ2RfdmFsKG9y aWcpICE9IHBnZF92YWwoKnBnZHApKSkgewotCQkqbnIgLT0gcmVmczsKLQkJd2hpbGUgKHJlZnMt LSkKLQkJCXB1dF9wYWdlKGhlYWQpOworCQlwdXRfY29tcG91bmRfaGVhZChoZWFkLCByZWZzKTsK IAkJcmV0dXJuIDA7CiAJfQogCisJKm5yICs9IHJlZnM7CiAJU2V0UGFnZVJlZmVyZW5jZWQoaGVh ZCk7CiAJcmV0dXJuIDE7CiB9Ci0tIAoyLjI0LjEKCl9fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fCmRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKZHJpLWRldmVsQGxp c3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFu L2xpc3RpbmZvL2RyaS1kZXZlbAo=