From: Pavel Tatashin <pasha.tatashin@soleen.com>
To: Anshuman Khandual <anshuman.khandual@arm.com>
Cc: "James Morris" <jmorris@namei.org>,
"Sasha Levin" <sashal@kernel.org>,
LKML <linux-kernel@vger.kernel.org>,
linux-mm <linux-mm@kvack.org>,
linux-nvdimm <linux-nvdimm@lists.01.org>,
"Andrew Morton" <akpm@linux-foundation.org>,
"Michal Hocko" <mhocko@suse.com>,
"Dave Hansen" <dave.hansen@linux.intel.com>,
"Dan Williams" <dan.j.williams@intel.com>,
"Keith Busch" <keith.busch@intel.com>,
"Vishal L Verma" <vishal.l.verma@intel.com>,
"Dave Jiang" <dave.jiang@intel.com>,
"Ross Zwisler" <zwisler@kernel.org>,
"Tom Lendacky" <thomas.lendacky@amd.com>,
"Huang, Ying" <ying.huang@intel.com>,
"Fengguang Wu" <fengguang.wu@intel.com>,
"Borislav Petkov" <bp@suse.de>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Yaowei Bai" <baiyaowei@cmss.chinamobile.com>,
"Takashi Iwai" <tiwai@suse.de>,
"Jérôme Glisse" <jglisse@redhat.com>,
catalin.marinas@arm.com, "Will Deacon" <will.deacon@arm.com>,
rppt@linux.vnet.ibm.com,
"Ard Biesheuvel" <ard.biesheuvel@linaro.org>,
andrew.murray@arm.com, james.morse@arm.com,
"Marc Zyngier" <marc.zyngier@arm.com>,
sboyd@kernel.org, linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH] arm64: configurable sparsemem section size
Date: Wed, 24 Apr 2019 15:54:26 -0400 [thread overview]
Message-ID: <CA+CK2bB5ahqLrekkTUSdzTE2BPSPbB9nk6nKs+LjTap2oF8X-w@mail.gmail.com> (raw)
In-Reply-To: <CA+CK2bCD11x64pJj5gSnsu5jsUqJyU6o+=J4K8oYAsHqz9ULqQ@mail.gmail.com>
<resending> from original email
On Wed, Apr 24, 2019 at 3:48 PM Pavel Tatashin
<patatash@linux.microsoft.com> wrote:
>
> On Wed, Apr 24, 2019 at 5:07 AM Anshuman Khandual
> <anshuman.khandual@arm.com> wrote:
> >
> > On 04/24/2019 02:08 AM, Pavel Tatashin wrote:
> > > sparsemem section size determines the maximum size and alignment that
> > > is allowed to offline/online memory block. The bigger the size the less
> > > the clutter in /sys/devices/system/memory/*. On the other hand, however,
> > > there is less flexability in what granules of memory can be added and
> > > removed.
> >
> > Is there any scenario where less than a 1GB needs to be added on arm64 ?
>
> Yes, DAX hotplug loses 1G of memory without allowing smaller sections.
> Machines on which we are going to be using this functionality have 8G
> of System RAM, therefore losing 1G is a big problem.
>
> For details about using scenario see this cover letter:
> https://lore.kernel.org/lkml/20190421014429.31206-1-pasha.tatashin@soleen.com/
>
> >
> > >
> > > Recently, it was enabled in Linux to hotadd persistent memory that
> > > can be either real NV device, or reserved from regular System RAM
> > > and has identity of devdax.
> >
> > devdax (even ZONE_DEVICE) support has not been enabled on arm64 yet.
>
> Correct, I use your patches to enable ZONE_DEVICE, and thus devdax on ARM64:
> https://lore.kernel.org/lkml/1554265806-11501-1-git-send-email-anshuman.khandual@arm.com/
>
> >
> > >
> > > The problem is that because ARM64's section size is 1G, and devdax must
> > > have 2M label section, the first 1G is always missed when device is
> > > attached, because it is not 1G aligned.
> >
> > devdax has to be 2M aligned ? Does Linux enforce that right now ?
>
> Unfortunately, there is no way around this. Part of the memory can be
> reserved as persistent memory via device tree.
> memory@40000000 {
> device_type = "memory";
> reg = < 0x00000000 0x40000000
> 0x00000002 0x00000000 >;
> };
>
> pmem@1c0000000 {
> compatible = "pmem-region";
> reg = <0x00000001 0xc0000000
> 0x00000000 0x80000000>;
> volatile;
> numa-node-id = <0>;
> };
>
> So, while pmem is section aligned, as it should be, the dax device is
> going to be pmem start address + label size, which is 2M. The actual
> DAX device starts at:
> 0x1c0000000 + 2M.
>
> Because section size is 1G, the hotplug will able to add only memory
> starting from
> 0x1c0000000 + 1G
>
> > 27 and 28 do not even compile for ARM64_64_PAGES because of MAX_ORDER and
> > SECTION_SIZE mismatch.
>
> Can you please elaborate what configs are you using? I have no
> problems compiling with 27 and 28 bit.
>
> Thank you,
> Pasha
next prev parent reply other threads:[~2019-04-24 19:54 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-04-23 20:38 [PATCH] arm64: configurable sparsemem section size Pavel Tatashin
2019-04-24 9:07 ` Anshuman Khandual
2019-04-24 19:48 ` Pavel Tatashin
2019-04-24 19:54 ` Pavel Tatashin [this message]
2019-04-24 20:24 ` Dan Williams
2019-04-24 20:33 ` Pavel Tatashin
2019-04-25 3:06 ` Anshuman Khandual
2019-04-25 15:25 ` Michal Hocko
2019-04-25 15:31 ` Will Deacon
2019-04-25 15:41 ` Michal Hocko
2019-04-25 17:57 ` Pavel Tatashin
2019-04-26 5:33 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CA+CK2bB5ahqLrekkTUSdzTE2BPSPbB9nk6nKs+LjTap2oF8X-w@mail.gmail.com \
--to=pasha.tatashin@soleen.com \
--cc=akpm@linux-foundation.org \
--cc=andrew.murray@arm.com \
--cc=anshuman.khandual@arm.com \
--cc=ard.biesheuvel@linaro.org \
--cc=baiyaowei@cmss.chinamobile.com \
--cc=bhelgaas@google.com \
--cc=bp@suse.de \
--cc=catalin.marinas@arm.com \
--cc=dan.j.williams@intel.com \
--cc=dave.hansen@linux.intel.com \
--cc=dave.jiang@intel.com \
--cc=fengguang.wu@intel.com \
--cc=james.morse@arm.com \
--cc=jglisse@redhat.com \
--cc=jmorris@namei.org \
--cc=keith.busch@intel.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-nvdimm@lists.01.org \
--cc=marc.zyngier@arm.com \
--cc=mhocko@suse.com \
--cc=rppt@linux.vnet.ibm.com \
--cc=sashal@kernel.org \
--cc=sboyd@kernel.org \
--cc=thomas.lendacky@amd.com \
--cc=tiwai@suse.de \
--cc=vishal.l.verma@intel.com \
--cc=will.deacon@arm.com \
--cc=ying.huang@intel.com \
--cc=zwisler@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).