From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03F1EC433E0 for ; Fri, 29 Jan 2021 19:13:04 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 8418664DE3 for ; Fri, 29 Jan 2021 19:13:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8418664DE3 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=soleen.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E69CD8D0002; Fri, 29 Jan 2021 14:13:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E1B118D0001; Fri, 29 Jan 2021 14:13:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D09718D0002; Fri, 29 Jan 2021 14:13:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0002.hostedemail.com [216.40.44.2]) by kanga.kvack.org (Postfix) with ESMTP id BAC208D0001 for ; Fri, 29 Jan 2021 14:13:02 -0500 (EST) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 896598249980 for ; Fri, 29 Jan 2021 19:13:02 +0000 (UTC) X-FDA: 77759760204.23.door37_1207575275aa Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin23.hostedemail.com (Postfix) with ESMTP id 68AB937606 for ; Fri, 29 Jan 2021 19:13:02 +0000 (UTC) X-HE-Tag: door37_1207575275aa X-Filterd-Recvd-Size: 4946 Received: from mail-ed1-f53.google.com (mail-ed1-f53.google.com [209.85.208.53]) by imf07.hostedemail.com (Postfix) with ESMTP for ; Fri, 29 Jan 2021 19:13:01 +0000 (UTC) Received: by mail-ed1-f53.google.com with SMTP id n6so11772400edt.10 for ; Fri, 29 Jan 2021 11:13:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=soleen.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=gaI6OPXn00T7x/fWjtNIEnlv4T36Y5i9DpYyfu2PrY8=; b=A4IKuG8Gg5BkWo/2OBvRWlae087pil//44x54DbKXaoDSZcyD4mBzaUKJh6ERZwZs0 8p+A48Wff73q87hwKQKzT6wwLK9E+b16IBj1BBQxq5Nmh6SaVl3/6C0pWIOLrSEuw+qj zXPqxlrcCV41q0Fm1JoTq+bn+4SwW9kE8m7zvZUEf6zsQHDoXvhfGs4wUy1XEMFteSYp 1ijwRDoY9ABR5pPvO6iqPx+KsbshRJ+oIdabk+XeoPxWE6ZMYCfEVJMCP1RAUOA+dF3R /34AGN0ku1A4QsIZQKh4V+T7XpMYco4FQMsjXOuG7oUNi0ddbiCaPlWehxUU3fxxwT0z uIYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=gaI6OPXn00T7x/fWjtNIEnlv4T36Y5i9DpYyfu2PrY8=; b=KSTaTgP+sJbh8oC87rxaqp26b+ezOGnxM4DVvlPcpYAjWDo+mlF+w8N7z7xDm/uZho s3748D9fcgxcZ+u3/NO9eb8mrKhzzdv4E5egq/lF+EawmtyC3QehV/PE/NOQ6jnEpgsi P4qY+FaiX7ApTiTeCZRjqwiLWMEpx1rewJWp5xqg3eXgfck1a8oYgJV0WgPvcEYL3c9h 90xpReAbqLBwKbr9I55O+YnvIUNymq0aFO7TSAp+FOEitGD+vwvEkttJ6GahvMyGQQWL dAsfWbGmajuHrK3MwVinJJpfWif9PAjYWGogitj9AsQ1Z644goQ2aFNR2kbWL087gnl5 1HCg== X-Gm-Message-State: AOAM531ysAElcO3mLArbzwZqNYftQMd6qY6Eo+MpP5/M+bGObWjy6pdn gDehJzB5bN2xW8RgSxQIRvdam/gx8O+PME4xD+L2tg== X-Google-Smtp-Source: ABdhPJxBMjG1h9nPLTmi7wuVdnc4cEw93n9ss27tjX5NzaWxCUaJoMadPhV+j5KCh8f2SyNC+cRuTxkLw/+pv0o8rEs= X-Received: by 2002:aa7:cd87:: with SMTP id x7mr7216981edv.210.1611947580910; Fri, 29 Jan 2021 11:13:00 -0800 (PST) MIME-Version: 1.0 References: <8c2b75fe-a3e5-8eff-7f37-5d23c7ad9742@redhat.com> <94797c92-cd90-8a65-b879-0bb5f12b9fc5@redhat.com> <92912784-f3a3-b5a5-2d45-4c86ae26315f@redhat.com> In-Reply-To: From: Pavel Tatashin Date: Fri, 29 Jan 2021 14:12:25 -0500 Message-ID: Subject: Re: dax alignment problem on arm64 (and other achitectures) To: David Hildenbrand Cc: Anshuman Khandual , linux-mm , LKML , Sasha Levin , Tyler Hicks , Andrew Morton , Dan Williams , Michal Hocko , Oscar Salvador , Vlastimil Babka , Joonsoo Kim , Jason Gunthorpe , Marc Zyngier , Linux ARM , Will Deacon , James Morse , James Morris Content-Type: text/plain; charset="UTF-8" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: On Fri, Jan 29, 2021 at 2:06 PM Pavel Tatashin wrote: > > > > Definitely, but we should try figuring out what's going on here. I > > > assume on x86-64 it behaves differently? > > > > Yes, we should root cause. I highly suspect that there is somewhere > > alignment miscalculations happen that cause this memory waste with the > > offset 16M. I am also not sure why the 2M label size was increased, > > and why 16M is now an alignment requirement. > > This appears to be because even if we set vmemmap to be outside of the > dax device, the alignment calculates the maximum size of vmemmap for > this device, and subtracts it from the devdax size. > See [1], line 795 is where this offset is calculated. > > This also explains why with 64K pages, the 16M offset worked: because > fewer struct pages were able to fit within 16M - label size. > > [1] https://soleen.com/source/xref/linux/drivers/nvdimm/pfn_devs.c?r=b7b3c01b&mo=18459&fi=718#795 Actually, strike the previous e-mail. The extra space is when we reserve vmemmap from devdax. IFF we do it from mem, the extra space is not added. Now, this alignment makes total sense. Pasha