From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 44228C54EBE for ; Tue, 10 Jan 2023 08:56:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238141AbjAJI4b (ORCPT ); Tue, 10 Jan 2023 03:56:31 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237838AbjAJIz7 (ORCPT ); Tue, 10 Jan 2023 03:55:59 -0500 Received: from mail-yb1-xb32.google.com (mail-yb1-xb32.google.com [IPv6:2607:f8b0:4864:20::b32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A34559FAF for ; Tue, 10 Jan 2023 00:53:11 -0800 (PST) Received: by mail-yb1-xb32.google.com with SMTP id e76so11082807ybh.11 for ; Tue, 10 Jan 2023 00:53:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=19QYjJTilGk/RhxRe6+HhMCtJmIsJ5ED63/iAKX6oJs=; b=lbCmyLed2VvMiWlAcFE73mcQDp5etVdnVHUJB0fOIywFthZ8lU7tD2PGNO2HC3zSAc dWKpNjNnenpRB/dYUu/Bi9UKnr53swijKGeB+DGMJw/CtYP9CyD/TXtTQ84qKtpOqa1B we5SLgjXbaeHfSmWsG3ddc7582JJaXXNPM0ZVcUZvBZvkBf1GjtFGD4jByY9MTuwdTZQ LqlMAkCjgFRNQ11MujIamnA4f5V112faxZSPfRUJ3D5SWQ8SxH5+y7WVLOCty9zTEV4H 9x6AtFdn5fE2ZaudYTtWp4NSrDa4id6ZpWxndwXOdsPPP03SkrZFtgTmG+ErjTOg9i05 S0iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=19QYjJTilGk/RhxRe6+HhMCtJmIsJ5ED63/iAKX6oJs=; b=grnvTEwko7724Ahw6XtgBAKtCQroTHxQEgp9Y9DFa/N9ZvLzvg7kELvLOPN/PVydNU RIetu5Oc5itAUNnceIjCD9+tp4WSX8ofCkgGSkOEsKKy7ELvKsH81X1185leu9HSGdxB btpksYZGiuA3j5n5Q1XA96HDvavFgtcoRdx1YeWBBO3xbOZrzh/P/LgQsSwmH1eQWfMF FzTgScV602Z+k8tgIjZIgH2RXdV5H4IvETcp656F8LG0nkGgQ3EWYBId2Q2UaDQ5ejsp du2k4bxWPKT+JvVsOpSi+EY8NC7yNEjrsUTJLJYKAUdaqrBIeJZP7Tj2eI2eNcjCGIcd Rbwg== X-Gm-Message-State: AFqh2kp2vg2B+BvmhKEk6XCXfXwqih2nel9wRRoj5JmYiVDB41XPqpIC KnmQs1eQFhi/ELZoXnOzyT4S9S64TyOwTp/u0Kr0uA== X-Google-Smtp-Source: AMrXdXu1bywcmD2nBhn/vstGOcfd/CgtS7Mz2N+xJM/cbJTPId17VbthxQeFl64wwFgwjQNUfbOac2B/yqrJaq2AdY4= X-Received: by 2002:a5b:b47:0:b0:6fe:1625:f1f5 with SMTP id b7-20020a5b0b47000000b006fe1625f1f5mr6647952ybr.549.1673340786873; Tue, 10 Jan 2023 00:53:06 -0800 (PST) MIME-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> <20220701142310.2188015-11-glider@google.com> <63b74a6e6a909_c81f0294a5@dwillia2-xfh.jf.intel.com.notmuch> <63bc8fec4744a_5178e29467@dwillia2-xfh.jf.intel.com.notmuch> <63bd0be8945a0_5178e29414@dwillia2-xfh.jf.intel.com.notmuch> In-Reply-To: From: Alexander Potapenko Date: Tue, 10 Jan 2023 09:52:30 +0100 Message-ID: Subject: Re: [PATCH v4 10/45] libnvdimm/pfn_dev: increase MAX_STRUCT_PAGE_SIZE To: Dan Williams Cc: Greg Kroah-Hartman , Marco Elver , Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev , Linux Memory Management List , Linux-Arch , LKML Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > > > > > > > > > > > -- >8 -- > > > > >From 693563817dea3fd8f293f9b69ec78066ab1d96d2 Mon Sep 17 00:00:00 2001 > > > > From: Dan Williams > > > > Date: Thu, 5 Jan 2023 13:27:34 -0800 > > > > Subject: [PATCH] nvdimm: Support sizeof(struct page) > MAX_STRUCT_PAGE_SIZE > > > > > > > > Commit 6e9f05dc66f9 ("libnvdimm/pfn_dev: increase MAX_STRUCT_PAGE_SIZE") > > > > > > > > ...updated MAX_STRUCT_PAGE_SIZE to account for sizeof(struct page) > > > > potentially doubling in the case of CONFIG_KMSAN=y. Unfortunately this > > > > doubles the amount of capacity stolen from user addressable capacity for > > > > everyone, regardless of whether they are using the debug option. Revert > > > > that change, mandate that MAX_STRUCT_PAGE_SIZE never exceed 64, but > > > > allow for debug scenarios to proceed with creating debug sized page maps > > > > with a new 'libnvdimm.page_struct_override' module parameter. > > > > > > > > Note that this only applies to cases where the page map is permanent, > > > > i.e. stored in a reservation of the pmem itself ("--map=dev" in "ndctl > > > > create-namespace" terms). For the "--map=mem" case, since the allocation > > > > is ephemeral for the lifespan of the namespace, there are no explicit > > > > restriction. However, the implicit restriction, of having enough > > > > available "System RAM" to store the page map for the typically large > > > > pmem, still applies. > > > > > > > > Fixes: 6e9f05dc66f9 ("libnvdimm/pfn_dev: increase MAX_STRUCT_PAGE_SIZE") > > > > Cc: > > > > Cc: Alexander Potapenko > > > > Cc: Marco Elver > > > > Reported-by: Jeff Moyer > > > > --- > > > > drivers/nvdimm/nd.h | 2 +- > > > > drivers/nvdimm/pfn_devs.c | 45 ++++++++++++++++++++++++++------------- > > > > 2 files changed, 31 insertions(+), 16 deletions(-) > > > > > > > > diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h > > > > index 85ca5b4da3cf..ec5219680092 100644 > > > > --- a/drivers/nvdimm/nd.h > > > > +++ b/drivers/nvdimm/nd.h > > > > @@ -652,7 +652,7 @@ void devm_namespace_disable(struct device *dev, > > > > struct nd_namespace_common *ndns); > > > > #if IS_ENABLED(CONFIG_ND_CLAIM) > > > > /* max struct page size independent of kernel config */ > > > > -#define MAX_STRUCT_PAGE_SIZE 128 > > > > +#define MAX_STRUCT_PAGE_SIZE 64 > > > > int nvdimm_setup_pfn(struct nd_pfn *nd_pfn, struct dev_pagemap *pgmap); > > > > #else > > > > static inline int nvdimm_setup_pfn(struct nd_pfn *nd_pfn, > > > > diff --git a/drivers/nvdimm/pfn_devs.c b/drivers/nvdimm/pfn_devs.c > > > > index 61af072ac98f..978d63559c0e 100644 > > > > --- a/drivers/nvdimm/pfn_devs.c > > > > +++ b/drivers/nvdimm/pfn_devs.c > > > > @@ -13,6 +13,11 @@ > > > > #include "pfn.h" > > > > #include "nd.h" > > > > > > > > +static bool page_struct_override; > > > > +module_param(page_struct_override, bool, 0644); > > > > +MODULE_PARM_DESC(page_struct_override, > > > > + "Force namespace creation in the presence of mm-debug."); > > > > > > I can't figure out from this description what this is for so perhaps it > > > should be either removed and made dynamic (if you know you want to debug > > > the mm core, why not turn it on then?) or made more obvious what is > > > happening? > > > > I'll kill it and update the KMSAN Documentation that KMSAN has > > interactions with the NVDIMM subsystem that may cause some namespaces to > > fail to enable. That Documentation needs to be a part of this patch > > regardless as that would be the default behavior of this module > > parameter. > > > > Unfortunately, it can not be dynamically enabled because the size of > > 'struct page' is unfortunately recorded in the metadata of the device. > > Recall this is for supporting platform configurations where the capacity > > of the persistent memory exceeds or consumes too much of System RAM. > > Consider 4TB of PMEM consumes 64GB of space just for 'struct page'. So, > > NVDIMM subsystem has a mode to store that page array in a reservation on > > the PMEM device itself. > > Sorry, I might be missing something, but why cannot we have > > #ifdef CONFIG_KMSAN > #define MAX_STRUCT_PAGE_SIZE 128 By the way, KMSAN only adds 16 bytes to struct page - would it help to reduce MAX_STRUCT_PAGE_SIZE to 80 bytes? > #else > #define MAX_STRUCT_PAGE_SIZE 64 > #endif >