From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA00AEB64DA for ; Fri, 7 Jul 2023 09:52:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232782AbjGGJwz (ORCPT ); Fri, 7 Jul 2023 05:52:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60000 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232853AbjGGJwr (ORCPT ); Fri, 7 Jul 2023 05:52:47 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id EB0BF170F for ; Fri, 7 Jul 2023 02:52:43 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 59F45D75; Fri, 7 Jul 2023 02:53:25 -0700 (PDT) Received: from [10.57.77.63] (unknown [10.57.77.63]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BEF063F740; Fri, 7 Jul 2023 02:52:40 -0700 (PDT) Message-ID: <44e60630-5e9d-c8df-ab79-cb0767de680e@arm.com> Date: Fri, 7 Jul 2023 10:52:38 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH v2 4/5] mm: FLEXIBLE_THP for improved performance To: "Huang, Ying" Cc: Andrew Morton , Matthew Wilcox , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Yu Zhao , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20230703135330.1865927-1-ryan.roberts@arm.com> <20230703135330.1865927-5-ryan.roberts@arm.com> <87edlkgnfa.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Ryan Roberts In-Reply-To: <87edlkgnfa.fsf@yhuang6-desk2.ccr.corp.intel.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 07/07/2023 09:01, Huang, Ying wrote: > Ryan Roberts writes: > >> Introduce FLEXIBLE_THP feature, which allows anonymous memory to be >> allocated in large folios of a specified order. All pages of the large >> folio are pte-mapped during the same page fault, significantly reducing >> the number of page faults. The number of per-page operations (e.g. ref >> counting, rmap management lru list management) are also significantly >> reduced since those ops now become per-folio. > > I likes the idea to share as much code as possible between large > (anonymous) folio and THP. Finally, THP becomes just a special kind of > large folio. > > Although we can use smaller page order for FLEXIBLE_THP, it's hard to > avoid internal fragmentation completely. So, I think that finally we > will need to provide a mechanism for the users to opt out, e.g., > something like "always madvise never" via > /sys/kernel/mm/transparent_hugepage/enabled. I'm not sure whether it's > a good idea to reuse the existing interface of THP. I wouldn't want to tie this to the existing interface, simply because that implies that we would want to follow the "always" and "madvise" advice too; That means that on a thp=madvise system (which is certainly the case for android and other client systems) we would have to disable large anon folios for VMAs that haven't explicitly opted in. That breaks the intention that this should be an invisible performance boost. I think it's important to set the policy for use of THP separately to use of large anon folios. I could be persuaded on the merrits of a new runtime enable/disable interface if there is concensus. > > Best Regards, > Huang, Ying From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 288DBEB64DA for ; Fri, 7 Jul 2023 09:53:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To: Subject:MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=vcoj/+hulF4RmOJssSNuJpfGfOI7YXZCBo3TynoHzw8=; b=gcMN9Nr/Og9Q5X vbtBOtYy6IxB5nqqReq6LiAS2/djh3y/Gz/6BpWFr/IynKMPLbq5biIiaPwWupjcladWobpqYBdKX lK0CuhAaoYCp2k5G1O9qkG3RKQRJTP2dMBL0s7hcqhEAHIHlYZLhZh+cIM6/5hQgDSMLXMUk//6uj 3Zhpu4PSSiPBadtowCUSPNZQUbFPUz1Srk2CV5ZIgaYZDSAkhCDK6oQDimIlCv0eWlI0na4RWfTNS KtABIQv2xo3obhgilp3CqGDRsFLkYPx2uvap8ZxRUaiHjMb5JCnAeVFnQM3j2vVmR1qpiNCbdVTvk AyKmqsF8knWLEy7PbNdQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qHi8w-004BKA-06; Fri, 07 Jul 2023 09:52:50 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qHi8s-004BIR-17 for linux-arm-kernel@lists.infradead.org; Fri, 07 Jul 2023 09:52:48 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 59F45D75; Fri, 7 Jul 2023 02:53:25 -0700 (PDT) Received: from [10.57.77.63] (unknown [10.57.77.63]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id BEF063F740; Fri, 7 Jul 2023 02:52:40 -0700 (PDT) Message-ID: <44e60630-5e9d-c8df-ab79-cb0767de680e@arm.com> Date: Fri, 7 Jul 2023 10:52:38 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [PATCH v2 4/5] mm: FLEXIBLE_THP for improved performance To: "Huang, Ying" Cc: Andrew Morton , Matthew Wilcox , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Yu Zhao , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org References: <20230703135330.1865927-1-ryan.roberts@arm.com> <20230703135330.1865927-5-ryan.roberts@arm.com> <87edlkgnfa.fsf@yhuang6-desk2.ccr.corp.intel.com> From: Ryan Roberts In-Reply-To: <87edlkgnfa.fsf@yhuang6-desk2.ccr.corp.intel.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230707_025246_433153_BE80E92C X-CRM114-Status: GOOD ( 17.57 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 07/07/2023 09:01, Huang, Ying wrote: > Ryan Roberts writes: > >> Introduce FLEXIBLE_THP feature, which allows anonymous memory to be >> allocated in large folios of a specified order. All pages of the large >> folio are pte-mapped during the same page fault, significantly reducing >> the number of page faults. The number of per-page operations (e.g. ref >> counting, rmap management lru list management) are also significantly >> reduced since those ops now become per-folio. > > I likes the idea to share as much code as possible between large > (anonymous) folio and THP. Finally, THP becomes just a special kind of > large folio. > > Although we can use smaller page order for FLEXIBLE_THP, it's hard to > avoid internal fragmentation completely. So, I think that finally we > will need to provide a mechanism for the users to opt out, e.g., > something like "always madvise never" via > /sys/kernel/mm/transparent_hugepage/enabled. I'm not sure whether it's > a good idea to reuse the existing interface of THP. I wouldn't want to tie this to the existing interface, simply because that implies that we would want to follow the "always" and "madvise" advice too; That means that on a thp=madvise system (which is certainly the case for android and other client systems) we would have to disable large anon folios for VMAs that haven't explicitly opted in. That breaks the intention that this should be an invisible performance boost. I think it's important to set the policy for use of THP separately to use of large anon folios. I could be persuaded on the merrits of a new runtime enable/disable interface if there is concensus. > > Best Regards, > Huang, Ying _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel