From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E16C7C433E0 for ; Fri, 29 Jan 2021 19:07:02 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 7AE1564E0B for ; Fri, 29 Jan 2021 19:07:02 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7AE1564E0B Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id DB63C8D0003; Fri, 29 Jan 2021 14:07:01 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D65C58D0001; Fri, 29 Jan 2021 14:07:01 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C06E48D0003; Fri, 29 Jan 2021 14:07:01 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0150.hostedemail.com [216.40.44.150]) by kanga.kvack.org (Postfix) with ESMTP id A74CD8D0001 for ; Fri, 29 Jan 2021 14:07:01 -0500 (EST) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5523B363D for ; Fri, 29 Jan 2021 19:07:01 +0000 (UTC) X-FDA: 77759745042.22.slip46_2916c86275aa Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin22.hostedemail.com (Postfix) with ESMTP id 1C7BF18038C3C for ; Fri, 29 Jan 2021 19:07:01 +0000 (UTC) X-HE-Tag: slip46_2916c86275aa X-Filterd-Recvd-Size: 4560 Received: from mail-ed1-f52.google.com (mail-ed1-f52.google.com [209.85.208.52]) by imf02.hostedemail.com (Postfix) with ESMTP for ; Fri, 29 Jan 2021 19:07:00 +0000 (UTC) Received: by mail-ed1-f52.google.com with SMTP id g1so11787546edu.4 for ; Fri, 29 Jan 2021 11:07:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=vMk9WGzVPpcoDliWw6CE1qXVyF1iQxXPLajClJEgv98=; b=Z6Bk9oGBsdd9aKFm0lhB2VUOgbChso56sK1q34eUOnaYX0wqNSsvQ5K/bvfJdm2owu CsXHCKomyiChcgdeQrKTfW+4drbC2690CBgYZZZZx0mpwrm8Cx1TGmLXkSENJEf1+iOT dXYTRdxmmSomCTejkqVFKN9bz4257beF/X/78A+5gDkQEWTvp8LQSMVhrrPjH3PHVPqY JmFCFc9Q6CvfN1Ch2PGboI5r3XVS1sNow83cRzA9y+kgdr03IVNWwj+aQXYokNnbKbh/ 7h1Sb9xkjG9Q/fa4GB07AcVwQ7tCXh16lM30KTZZ060hTaQOdQNoCEb31ero8qhOWohJ LI8A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=vMk9WGzVPpcoDliWw6CE1qXVyF1iQxXPLajClJEgv98=; b=cQKZzWaryJ3qjNx0aJaf5MiM+BPagpNnMCmQjG3DD39mxNq32ruzYalPxsI2fZET9v KH00trUfoBsaLE9mdMo8rCoFckxKC5ySjKv+2M/zDfbicbD/0vHBCOuASJgtIm8ON5PG b5rMfGTtO4bmEDphQnN2I3SbC2CciK1He0Ahy7hHBCxRCUNWJjalC3aIBy6jYwey7Gkc 0iv/l6UXSZS68YJ4z/xzMBoMR9fJxmprWuMtO+osZbEGlPKIVBEGLJPyeSO7OokHPnN0 2obHuxa90w6lm6Ep/tsKL1UVhpKjT21GgIVLe/uNoJnkQoVp3pZKBiFw686R9gd7gdAO 1hlQ== X-Gm-Message-State: AOAM532eiLNCsRGHxHbvlAISmApPYczTNCxvq/TUmvh+PvSoKGjX5puD iBU17WG6HYy2qvvo+1XlvCLwKuSl2YMg7QeV/axnrQ== X-Google-Smtp-Source: ABdhPJyHRc0vMqmyshCdniPC5IhqwLn4tl4ATEGWnxcs/ZV63yqgy2GfXX9fY2jc64XpsYQckPwlj3ZqVnZQfSJ9M9c= X-Received: by 2002:a05:6402:402:: with SMTP id q2mr6948030edv.116.1611947219186; Fri, 29 Jan 2021 11:06:59 -0800 (PST) MIME-Version: 1.0 References: <8c2b75fe-a3e5-8eff-7f37-5d23c7ad9742@redhat.com> <94797c92-cd90-8a65-b879-0bb5f12b9fc5@redhat.com> <92912784-f3a3-b5a5-2d45-4c86ae26315f@redhat.com> In-Reply-To: From: Pavel Tatashin Date: Fri, 29 Jan 2021 14:06:23 -0500 Message-ID: Subject: Re: dax alignment problem on arm64 (and other achitectures) To: David Hildenbrand Cc: Anshuman Khandual , linux-mm , LKML , Sasha Levin , Tyler Hicks , Andrew Morton , Dan Williams , Michal Hocko , Oscar Salvador , Vlastimil Babka , Joonsoo Kim , Jason Gunthorpe , Marc Zyngier , Linux ARM , Will Deacon , James Morse , James Morris Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: > > Definitely, but we should try figuring out what's going on here. I > > assume on x86-64 it behaves differently? > > Yes, we should root cause. I highly suspect that there is somewhere > alignment miscalculations happen that cause this memory waste with the > offset 16M. I am also not sure why the 2M label size was increased, > and why 16M is now an alignment requirement. This appears to be because even if we set vmemmap to be outside of the dax device, the alignment calculates the maximum size of vmemmap for this device, and subtracts it from the devdax size. See [1], line 795 is where this offset is calculated. This also explains why with 64K pages, the 16M offset worked: because fewer struct pages were able to fit within 16M - label size. [1] https://soleen.com/source/xref/linux/drivers/nvdimm/pfn_devs.c?r=b7b3c01b&mo=18459&fi=718#795