From: Pavel Tatashin <pasha.tatashin@soleen.com>
To: David Hildenbrand <david@redhat.com>
Cc: Anshuman Khandual <anshuman.khandual@arm.com>,
linux-mm <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>,
Sasha Levin <sashal@kernel.org>,
Tyler Hicks <tyhicks@linux.microsoft.com>,
Andrew Morton <akpm@linux-foundation.org>,
Dan Williams <dan.j.williams@intel.com>,
Michal Hocko <mhocko@suse.com>,
Oscar Salvador <osalvador@suse.de>,
Vlastimil Babka <vbabka@suse.cz>,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Jason Gunthorpe <jgg@ziepe.ca>, Marc Zyngier <maz@kernel.org>,
Linux ARM <linux-arm-kernel@lists.infradead.org>,
Will Deacon <will.deacon@arm.com>,
James Morse <james.morse@arm.com>,
James Morris <jmorris@namei.org>
Subject: Re: dax alignment problem on arm64 (and other achitectures)
Date: Fri, 29 Jan 2021 14:41:47 -0500 [thread overview]
Message-ID: <CA+CK2bCD13JBLMxn2mAuRyVQGKBS5ic2UqYSsxXTccszXCmHkA@mail.gmail.com> (raw)
In-Reply-To: <CA+CK2bD7rXgs9UTX7QNNDAES0z32qV-gq+7TUwzx_zKHhrtUDw@mail.gmail.com>
On Fri, Jan 29, 2021 at 2:12 PM Pavel Tatashin
<pasha.tatashin@soleen.com> wrote:
>
> On Fri, Jan 29, 2021 at 2:06 PM Pavel Tatashin
> <pasha.tatashin@soleen.com> wrote:
> >
> > > > Definitely, but we should try figuring out what's going on here. I
> > > > assume on x86-64 it behaves differently?
> > >
> > > Yes, we should root cause. I highly suspect that there is somewhere
> > > alignment miscalculations happen that cause this memory waste with the
> > > offset 16M. I am also not sure why the 2M label size was increased,
> > > and why 16M is now an alignment requirement.
> >
> > This appears to be because even if we set vmemmap to be outside of the
> > dax device, the alignment calculates the maximum size of vmemmap for
> > this device, and subtracts it from the devdax size.
> > See [1], line 795 is where this offset is calculated.
> >
> > This also explains why with 64K pages, the 16M offset worked: because
> > fewer struct pages were able to fit within 16M - label size.
> >
> > [1] https://soleen.com/source/xref/linux/drivers/nvdimm/pfn_devs.c?r=b7b3c01b&mo=18459&fi=718#795
>
> Actually, strike the previous e-mail. The extra space is when we
> reserve vmemmap from devdax. IFF we do it from mem, the extra space is
> not added. Now, this alignment makes total sense.
commit 2522afb86a8cceba0f67dbf05772d21b76d79f06
Author: Dan Williams <dan.j.williams@intel.com>
Date: Thu Jan 30 12:06:23 2020 -0800
libnvdimm/region: Introduce an 'align' attribute
This is the patch that introduced the 16M alignment.
/*
* PowerPC requires this alignment for memremap_pages(). All other archs
* should be ok with SUBSECTION_SIZE (see memremap_compat_align()).
*/
#define MEMREMAP_COMPAT_ALIGN_MAX SZ_16M
static unsigned long default_align(struct nd_region *nd_region)
{
unsigned long align;
int i, mappings;
u32 remainder;
if (is_nd_blk(&nd_region->dev))
align = PAGE_SIZE;
else
align = MEMREMAP_COMPAT_ALIGN_MAX;
Dan, is this logic correct? Why is_nd_pmem() cannot be set to
SUBSECTION_SIZE alignment?
Thank you,
Pasha
>
> Pasha
next prev parent reply other threads:[~2021-01-29 19:43 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-27 20:43 dax alignment problem on arm64 (and other achitectures) Pavel Tatashin
2021-01-27 21:09 ` David Hildenbrand
2021-01-27 21:49 ` Pavel Tatashin
2021-01-27 22:18 ` David Hildenbrand
2021-01-27 23:33 ` Pavel Tatashin
2021-01-28 15:03 ` David Hildenbrand
2021-01-29 2:06 ` Pavel Tatashin
[not found] ` <92912784-f3a3-b5a5-2d45-4c86ae26315f@redhat.com>
2021-01-29 16:24 ` Pavel Tatashin
2021-01-29 19:06 ` Pavel Tatashin
2021-01-29 19:12 ` Pavel Tatashin
2021-01-29 19:41 ` Pavel Tatashin [this message]
2021-01-29 2:55 ` Dan Williams
[not found] ` <CA+CK2bBJmnTVF8ZfwLyLqgjgo63G-rVQTYwUqgmx8wXFtRH9-g@mail.gmail.com>
[not found] ` <c5fe4330-55e2-48b8-1961-ca9eb879354e@oracle.com>
2021-01-29 16:32 ` Pavel Tatashin
2021-01-29 17:22 ` Joao Martins
2021-01-29 20:26 ` Dan Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CA+CK2bCD13JBLMxn2mAuRyVQGKBS5ic2UqYSsxXTccszXCmHkA@mail.gmail.com \
--to=pasha.tatashin@soleen.com \
--cc=akpm@linux-foundation.org \
--cc=anshuman.khandual@arm.com \
--cc=dan.j.williams@intel.com \
--cc=david@redhat.com \
--cc=iamjoonsoo.kim@lge.com \
--cc=james.morse@arm.com \
--cc=jgg@ziepe.ca \
--cc=jmorris@namei.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=maz@kernel.org \
--cc=mhocko@suse.com \
--cc=osalvador@suse.de \
--cc=sashal@kernel.org \
--cc=tyhicks@linux.microsoft.com \
--cc=vbabka@suse.cz \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).