From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 561AFEB64DC for ; Thu, 6 Jul 2023 19:33:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231245AbjGFTdm (ORCPT ); Thu, 6 Jul 2023 15:33:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46948 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229538AbjGFTdl (ORCPT ); Thu, 6 Jul 2023 15:33:41 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1E9641BFF for ; Thu, 6 Jul 2023 12:33:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=0wxmw1h/+sc7ZkGjLi6C59U1J1wukIjy+nrYU+ZKh7s=; b=kdCQhoZRlP+51y/fm7OjLLXVUU MgnblWPjJeWegHqTxc2bW0g5qPTx6NwuwQCIsIJvqXFS5nAz/OseFxEljm4ZSBVx8ZlGklmk3j+yG paqL/h+GOK0chawhP5ibXNRWszLaLGTJyhS9KS4VaS0kGn47IFSlMOdqcxAbPvjLFkao/7Loxweav zMDn7iZYkB5P64QhmKHix+FMnDemyQSDfb0d1ZqkosI0J7H/lbDLI1aJtQ7gsetgEhHDK/r3Cdsu2 RKI9G5Uq2Ymp6A4iOE5hCZOGQCfwbDECKx7oEJ4/xr55VGX0rlMmt/CEdFA3HhBJvEKIA8HxXwpja SxnltGVw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qHUj7-00BLN7-QK; Thu, 06 Jul 2023 19:33:17 +0000 Date: Thu, 6 Jul 2023 20:33:17 +0100 From: Matthew Wilcox To: Yu Zhao Cc: Ryan Roberts , Andrew Morton , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v2 3/5] mm: Default implementation of arch_wants_pte_order() Message-ID: References: <20230703135330.1865927-1-ryan.roberts@arm.com> <20230703135330.1865927-4-ryan.roberts@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jul 04, 2023 at 08:07:19PM -0600, Yu Zhao wrote: > > - On arm64 when the process has marked the VMA for THP (or when > > transparent_hugepage=always) but the VMA does not meet the requirements for a > > PMD-sized mapping (or we failed to allocate, ...) then I'd like to map using > > contpte. For 4K base pages this is 64K (order-4), for 16K this is 2M (order-7) > > and for 64K this is 2M (order-5). The 64K base page case is very important since > > the PMD size for that base page is 512MB which is almost impossible to allocate > > in practice. > > Which case (server or client) are you focusing on here? For our client > devices, I can confidently say that 64KB has to be after 16KB, if it > happens at all. For servers in general, I don't know of any major > memory-intensive workloads that are not THP-aware, i.e., I don't think > "VMA does not meet the requirements" is a concern. It sounds like you've done some measurements, and I'd like to understand those a bit better. There are a number of factors involved: - A larger page size shrinks the length of the LRU list, so systems which see heavy LRU lock contention benefit more - A larger page size has more internal fragmentation, so we run out of memory and have to do reclaim more often (and maybe workload which used to fit in DRAM now do not) (probably others; i'm not at 100% right now) I think concerns about "allocating lots of order-2 folios makes it harder to allocate order-4 folios" are _probably_ not warranted (without data to prove otherwise). All anonymous memory is movable, so our compaction code should be able to create larger order folios. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E9B56EB64D9 for ; Thu, 6 Jul 2023 19:33:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2ZI1fdK289OMM/zPVm80JMTP8U4NfNwqiMl4byOXvW0=; b=4dbMncZgP2HpZE WZOvQF+EmDUjJy1Q1cwmFzV4TufdYnBIU9W3wRnhTpcN8Z+MsyXughs+eqZdsi/jX8BV6vUsPNaqq qVMa8oOnUMdQuChuyLGVO1MFkiyMRA3aNmb1jyxTWt5NG3JsQ2yo5uOPN8jVf0KZdEq2bXkGByiZm 4M/5jAR0ZhRFegA6Ngk+qdYgYiiN7JWTn9MAeyAOw+MCi5ZZHNGQrwcO6IZ94/xQsLIFt69Wd7eto ndnUh/4QuHZU4IsHgakPnnMZqjYodYDN6YLIJ8nsR1CyXwqjnUMo6iHPQAxcNhY5SYZzO7Yiw90NB 06M/Vnjc2Z5FfOe/bdKw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qHUjE-002Vtd-2g; Thu, 06 Jul 2023 19:33:24 +0000 Received: from casper.infradead.org ([2001:8b0:10b:1236::1]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qHUjD-002Vt8-0m for linux-arm-kernel@bombadil.infradead.org; Thu, 06 Jul 2023 19:33:23 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=0wxmw1h/+sc7ZkGjLi6C59U1J1wukIjy+nrYU+ZKh7s=; b=kdCQhoZRlP+51y/fm7OjLLXVUU MgnblWPjJeWegHqTxc2bW0g5qPTx6NwuwQCIsIJvqXFS5nAz/OseFxEljm4ZSBVx8ZlGklmk3j+yG paqL/h+GOK0chawhP5ibXNRWszLaLGTJyhS9KS4VaS0kGn47IFSlMOdqcxAbPvjLFkao/7Loxweav zMDn7iZYkB5P64QhmKHix+FMnDemyQSDfb0d1ZqkosI0J7H/lbDLI1aJtQ7gsetgEhHDK/r3Cdsu2 RKI9G5Uq2Ymp6A4iOE5hCZOGQCfwbDECKx7oEJ4/xr55VGX0rlMmt/CEdFA3HhBJvEKIA8HxXwpja SxnltGVw==; Received: from willy by casper.infradead.org with local (Exim 4.94.2 #2 (Red Hat Linux)) id 1qHUj7-00BLN7-QK; Thu, 06 Jul 2023 19:33:17 +0000 Date: Thu, 6 Jul 2023 20:33:17 +0100 From: Matthew Wilcox To: Yu Zhao Cc: Ryan Roberts , Andrew Morton , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: Re: [PATCH v2 3/5] mm: Default implementation of arch_wants_pte_order() Message-ID: References: <20230703135330.1865927-1-ryan.roberts@arm.com> <20230703135330.1865927-4-ryan.roberts@arm.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Jul 04, 2023 at 08:07:19PM -0600, Yu Zhao wrote: > > - On arm64 when the process has marked the VMA for THP (or when > > transparent_hugepage=always) but the VMA does not meet the requirements for a > > PMD-sized mapping (or we failed to allocate, ...) then I'd like to map using > > contpte. For 4K base pages this is 64K (order-4), for 16K this is 2M (order-7) > > and for 64K this is 2M (order-5). The 64K base page case is very important since > > the PMD size for that base page is 512MB which is almost impossible to allocate > > in practice. > > Which case (server or client) are you focusing on here? For our client > devices, I can confidently say that 64KB has to be after 16KB, if it > happens at all. For servers in general, I don't know of any major > memory-intensive workloads that are not THP-aware, i.e., I don't think > "VMA does not meet the requirements" is a concern. It sounds like you've done some measurements, and I'd like to understand those a bit better. There are a number of factors involved: - A larger page size shrinks the length of the LRU list, so systems which see heavy LRU lock contention benefit more - A larger page size has more internal fragmentation, so we run out of memory and have to do reclaim more often (and maybe workload which used to fit in DRAM now do not) (probably others; i'm not at 100% right now) I think concerns about "allocating lots of order-2 folios makes it harder to allocate order-4 folios" are _probably_ not warranted (without data to prove otherwise). All anonymous memory is movable, so our compaction code should be able to create larger order folios. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel