From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CF00C433B4 for ; Sat, 10 Apr 2021 08:53:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D9282610CB for ; Sat, 10 Apr 2021 08:53:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229494AbhDJIxV (ORCPT ); Sat, 10 Apr 2021 04:53:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40910 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232254AbhDJIxS (ORCPT ); Sat, 10 Apr 2021 04:53:18 -0400 Received: from mail-yb1-xb34.google.com (mail-yb1-xb34.google.com [IPv6:2607:f8b0:4864:20::b34]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43B90C061764 for ; Sat, 10 Apr 2021 01:53:03 -0700 (PDT) Received: by mail-yb1-xb34.google.com with SMTP id z1so9234089ybf.6 for ; Sat, 10 Apr 2021 01:53:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7yHuiMf2mbQHC9INRUFE4f0Q8iL2DCKJY335xtgKT+M=; b=s0sXCo3dvECgpdJR7whscVJ1iWTqthnxVLYQ+cWKngNry1eueNvN2t1KtSQvyAKpin 13En8yn6d+Yw7fzHTql9MiHET60pC0eR48G0sg4Y0BJt25ufPVNEf0X34Jx1xzMIG4Hy Y7EKWXgnYUH4e5mY/YU8c2uHjr8njpZ3dTzJt0iYL4TJPYPHOv7FAshptX+jC9Tzkz3z hfvu7Qt1gZ4d5Qsr3NTxYjx2QW7Fdl/cbwNiQ02u18593AOU/WV8l+R+9uqhoWnAmXr9 /+iF2mseThA+yNVXeIr6fq/WlwKArwfqxhv/83FA2rfwSlB5wOfyw2hsXxWSVYF6fqRM JrGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7yHuiMf2mbQHC9INRUFE4f0Q8iL2DCKJY335xtgKT+M=; b=Z/ZK6qwbVk6kxpsc5BzedCubo2fEdaRT7lYo/FUBhTB4ywSWPrwLUOlJBnXL8WZpw3 UX8GblnfXDE7oOD6TWOsxwcaGcV+ANVRJiqsR/Dh+JGNuGDOJi7EChydDnKfZb6WV1wP qxNd3/JTpAMvfBvXTcrXEIiI+BfIYdmn7OiwXRB9VCCsh5mDLN2kNYcD3jqDcnU0QjFE G6XodzBx8tXOqXVcksiYCLliddzsP5sXMl9B5TLOoQS6mPfgFQfwxCKuWznErLbRfbWP M/ruL2krvWxhGS14LB33yg9wxVSjumo4VjtS5N8OyqsUpDYXX1v7xlmxkLzfnzfUHFXW /w5w== X-Gm-Message-State: AOAM532LUW/QMJ1b5FjoUYcl9inPJMEy19LkFxnJsUpcfqfYx5fX3Af2 SiuWYe+h2E/1PyTuoJE1nwa9PK8DFw0hZ8T6CohwRQ== X-Google-Smtp-Source: ABdhPJy5dr9lQOxU6kvADPui00Fq+MldBcPrkITVyRf0//k/8cK2iDR0U0j3GM3hEL13gRzjdmouFNrBWiWVxuk2Mok= X-Received: by 2002:a25:2bc1:: with SMTP id r184mr25473310ybr.51.1618044782263; Sat, 10 Apr 2021 01:53:02 -0700 (PDT) MIME-Version: 1.0 References: <20210409185105.188284-3-willy@infradead.org> <202104100656.N7EVvkNZ-lkp@intel.com> <20210410024313.GX2531743@casper.infradead.org> <20210410082158.79ad09a6@carbon> In-Reply-To: <20210410082158.79ad09a6@carbon> From: Ilias Apalodimas Date: Sat, 10 Apr 2021 11:52:26 +0300 Message-ID: Subject: Re: Bogus struct page layout on 32-bit To: Jesper Dangaard Brouer Cc: Matthew Wilcox , kernel test robot , Linux-MM , kbuild-all@lists.01.org, clang-built-linux@googlegroups.com, open list , linux-fsdevel@vger.kernel.org, Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , linuxppc-dev@lists.ozlabs.org, Linux ARM , "David S. Miller" , Ivan Khoronzhuk , Matteo Croce , "netdev@vger.kernel.org" , Grygorii Strashko Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org +CC Grygorii for the cpsw part as Ivan's email is not valid anymore Thanks for catching this. Interesting indeed... On Sat, 10 Apr 2021 at 09:22, Jesper Dangaard Brouer wrote: > > On Sat, 10 Apr 2021 03:43:13 +0100 > Matthew Wilcox wrote: > > > On Sat, Apr 10, 2021 at 06:45:35AM +0800, kernel test robot wrote: > > > >> include/linux/mm_types.h:274:1: error: static_assert failed due to requirement '__builtin_offsetof(struct page, lru) == __builtin_offsetof(struct folio, lru)' "offsetof(struct page, lru) == offsetof(struct folio, lru)" > > > FOLIO_MATCH(lru, lru); > > > include/linux/mm_types.h:272:2: note: expanded from macro 'FOLIO_MATCH' > > > static_assert(offsetof(struct page, pg) == offsetof(struct folio, fl)) > > > > Well, this is interesting. pahole reports: > > > > struct page { > > long unsigned int flags; /* 0 4 */ > > /* XXX 4 bytes hole, try to pack */ > > union { > > struct { > > struct list_head lru; /* 8 8 */ > > ... > > struct folio { > > union { > > struct { > > long unsigned int flags; /* 0 4 */ > > struct list_head lru; /* 4 8 */ > > > > so this assert has absolutely done its job. > > > > But why has this assert triggered? Why is struct page layout not what > > we thought it was? Turns out it's the dma_addr added in 2019 by commit > > c25fff7171be ("mm: add dma_addr_t to struct page"). On this particular > > config, it's 64-bit, and ppc32 requires alignment to 64-bit. So > > the whole union gets moved out by 4 bytes. > > Argh, good that you are catching this! > > > Unfortunately, we can't just fix this by putting an 'unsigned long pad' > > in front of it. It still aligns the entire union to 8 bytes, and then > > it skips another 4 bytes after the pad. > > > > We can fix it like this ... > > > > +++ b/include/linux/mm_types.h > > @@ -96,11 +96,12 @@ struct page { > > unsigned long private; > > }; > > struct { /* page_pool used by netstack */ > > + unsigned long _page_pool_pad; > > I'm fine with this pad. Matteo is currently proposing[1] to add a 32-bit > value after @dma_addr, and he could use this area instead. > > [1] https://lore.kernel.org/netdev/20210409223801.104657-3-mcroce@linux.microsoft.com/ > > When adding/changing this, we need to make sure that it doesn't overlap > member @index, because network stack use/check page_is_pfmemalloc(). > As far as my calculations this is safe to add. I always try to keep an > eye out for this, but I wonder if we could have a build check like yours. > > > > /** > > * @dma_addr: might require a 64-bit value even on > > * 32-bit architectures. > > */ > > - dma_addr_t dma_addr; > > + dma_addr_t dma_addr __packed; > > }; > > struct { /* slab, slob and slub */ > > union { > > > > but I don't know if GCC is smart enough to realise that dma_addr is now > > on an 8 byte boundary and it can use a normal instruction to access it, > > or whether it'll do something daft like use byte loads to access it. > > > > We could also do: > > > > + dma_addr_t dma_addr __packed __aligned(sizeof(void *)); > > > > and I see pahole, at least sees this correctly: > > > > struct { > > long unsigned int _page_pool_pad; /* 4 4 */ > > dma_addr_t dma_addr __attribute__((__aligned__(4))); /* 8 8 */ > > } __attribute__((__packed__)) __attribute__((__aligned__(4))); > > > > This presumably affects any 32-bit architecture with a 64-bit phys_addr_t > > / dma_addr_t. Advice, please? > > I'm not sure that the 32-bit behavior is with 64-bit (dma) addrs. > > I don't have any 32-bit boards with 64-bit DMA. Cc. Ivan, wasn't your > board (572x ?) 32-bit with driver 'cpsw' this case (where Ivan added > XDP+page_pool) ? > > -- > Best regards, > Jesper Dangaard Brouer > MSc.CS, Principal Kernel Engineer at Red Hat > LinkedIn: http://www.linkedin.com/in/brouer > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7761BC43460 for ; Sat, 10 Apr 2021 08:53:05 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 038D561184 for ; Sat, 10 Apr 2021 08:53:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 038D561184 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 105B56B006C; Sat, 10 Apr 2021 04:53:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0DD5F6B006E; Sat, 10 Apr 2021 04:53:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EBF936B0070; Sat, 10 Apr 2021 04:53:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0031.hostedemail.com [216.40.44.31]) by kanga.kvack.org (Postfix) with ESMTP id D03C06B006C for ; Sat, 10 Apr 2021 04:53:03 -0400 (EDT) Received: from smtpin35.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 76044ABEF for ; Sat, 10 Apr 2021 08:53:03 +0000 (UTC) X-FDA: 78015842646.35.B29ABFE Received: from mail-yb1-f176.google.com (mail-yb1-f176.google.com [209.85.219.176]) by imf14.hostedemail.com (Postfix) with ESMTP id 1A56CC0007C6 for ; Sat, 10 Apr 2021 08:52:56 +0000 (UTC) Received: by mail-yb1-f176.google.com with SMTP id l14so2996705ybf.11 for ; Sat, 10 Apr 2021 01:53:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7yHuiMf2mbQHC9INRUFE4f0Q8iL2DCKJY335xtgKT+M=; b=s0sXCo3dvECgpdJR7whscVJ1iWTqthnxVLYQ+cWKngNry1eueNvN2t1KtSQvyAKpin 13En8yn6d+Yw7fzHTql9MiHET60pC0eR48G0sg4Y0BJt25ufPVNEf0X34Jx1xzMIG4Hy Y7EKWXgnYUH4e5mY/YU8c2uHjr8njpZ3dTzJt0iYL4TJPYPHOv7FAshptX+jC9Tzkz3z hfvu7Qt1gZ4d5Qsr3NTxYjx2QW7Fdl/cbwNiQ02u18593AOU/WV8l+R+9uqhoWnAmXr9 /+iF2mseThA+yNVXeIr6fq/WlwKArwfqxhv/83FA2rfwSlB5wOfyw2hsXxWSVYF6fqRM JrGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7yHuiMf2mbQHC9INRUFE4f0Q8iL2DCKJY335xtgKT+M=; b=ZCWZi4C5VWjcheQh8f6Q0j86SfH+Jg6RJBTbLHeOPjgqr04h9Rp5tOE09FM7/52Lgb H4/YJyD0mGNELBmNZ38mV4hKNokEzA1aS9bfv4LzB1bNn67U0AD1Yv07/IzAAGvL9NRH dhU0kJCPdHAY2srv0Os3WNE/DTbn2LpF+0gZwpzBBPzG0qn91FLnS1i5VkXQsPJAY0rR yAemz7asmpnUuqZCwlvNEVPsZ0TYbvavMCy8IpVhDVl/LAT0Wl/bNwIZpriVpPh1/dbu IGurdnzV+hPloSY13Gpbs10PW2SIpdsINQ+iItNqAW64vo6cIzM/85LGWiBii+YYpvx0 8r/w== X-Gm-Message-State: AOAM530O40g2xaYk9wEx2KRBdMGRCXA+pE8+xo8e2XG5Db3FT2livts0 u3OJm6bILfK71jk6yV8ihCgbant926l3yhSMr9+uvg== X-Google-Smtp-Source: ABdhPJy5dr9lQOxU6kvADPui00Fq+MldBcPrkITVyRf0//k/8cK2iDR0U0j3GM3hEL13gRzjdmouFNrBWiWVxuk2Mok= X-Received: by 2002:a25:2bc1:: with SMTP id r184mr25473310ybr.51.1618044782263; Sat, 10 Apr 2021 01:53:02 -0700 (PDT) MIME-Version: 1.0 References: <20210409185105.188284-3-willy@infradead.org> <202104100656.N7EVvkNZ-lkp@intel.com> <20210410024313.GX2531743@casper.infradead.org> <20210410082158.79ad09a6@carbon> In-Reply-To: <20210410082158.79ad09a6@carbon> From: Ilias Apalodimas Date: Sat, 10 Apr 2021 11:52:26 +0300 Message-ID: Subject: Re: Bogus struct page layout on 32-bit To: Jesper Dangaard Brouer Cc: Matthew Wilcox , kernel test robot , Linux-MM , kbuild-all@lists.01.org, clang-built-linux@googlegroups.com, open list , linux-fsdevel@vger.kernel.org, Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , linuxppc-dev@lists.ozlabs.org, Linux ARM , "David S. Miller" , Ivan Khoronzhuk , Matteo Croce , "netdev@vger.kernel.org" , Grygorii Strashko Content-Type: text/plain; charset="UTF-8" X-Stat-Signature: eg5drrqqtpe5zu7hbn16bec3cpsqyosk X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 1A56CC0007C6 Received-SPF: none (linaro.org>: No applicable sender policy available) receiver=imf14; identity=mailfrom; envelope-from=""; helo=mail-yb1-f176.google.com; client-ip=209.85.219.176 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618044776-485385 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: +CC Grygorii for the cpsw part as Ivan's email is not valid anymore Thanks for catching this. Interesting indeed... On Sat, 10 Apr 2021 at 09:22, Jesper Dangaard Brouer wrote: > > On Sat, 10 Apr 2021 03:43:13 +0100 > Matthew Wilcox wrote: > > > On Sat, Apr 10, 2021 at 06:45:35AM +0800, kernel test robot wrote: > > > >> include/linux/mm_types.h:274:1: error: static_assert failed due to requirement '__builtin_offsetof(struct page, lru) == __builtin_offsetof(struct folio, lru)' "offsetof(struct page, lru) == offsetof(struct folio, lru)" > > > FOLIO_MATCH(lru, lru); > > > include/linux/mm_types.h:272:2: note: expanded from macro 'FOLIO_MATCH' > > > static_assert(offsetof(struct page, pg) == offsetof(struct folio, fl)) > > > > Well, this is interesting. pahole reports: > > > > struct page { > > long unsigned int flags; /* 0 4 */ > > /* XXX 4 bytes hole, try to pack */ > > union { > > struct { > > struct list_head lru; /* 8 8 */ > > ... > > struct folio { > > union { > > struct { > > long unsigned int flags; /* 0 4 */ > > struct list_head lru; /* 4 8 */ > > > > so this assert has absolutely done its job. > > > > But why has this assert triggered? Why is struct page layout not what > > we thought it was? Turns out it's the dma_addr added in 2019 by commit > > c25fff7171be ("mm: add dma_addr_t to struct page"). On this particular > > config, it's 64-bit, and ppc32 requires alignment to 64-bit. So > > the whole union gets moved out by 4 bytes. > > Argh, good that you are catching this! > > > Unfortunately, we can't just fix this by putting an 'unsigned long pad' > > in front of it. It still aligns the entire union to 8 bytes, and then > > it skips another 4 bytes after the pad. > > > > We can fix it like this ... > > > > +++ b/include/linux/mm_types.h > > @@ -96,11 +96,12 @@ struct page { > > unsigned long private; > > }; > > struct { /* page_pool used by netstack */ > > + unsigned long _page_pool_pad; > > I'm fine with this pad. Matteo is currently proposing[1] to add a 32-bit > value after @dma_addr, and he could use this area instead. > > [1] https://lore.kernel.org/netdev/20210409223801.104657-3-mcroce@linux.microsoft.com/ > > When adding/changing this, we need to make sure that it doesn't overlap > member @index, because network stack use/check page_is_pfmemalloc(). > As far as my calculations this is safe to add. I always try to keep an > eye out for this, but I wonder if we could have a build check like yours. > > > > /** > > * @dma_addr: might require a 64-bit value even on > > * 32-bit architectures. > > */ > > - dma_addr_t dma_addr; > > + dma_addr_t dma_addr __packed; > > }; > > struct { /* slab, slob and slub */ > > union { > > > > but I don't know if GCC is smart enough to realise that dma_addr is now > > on an 8 byte boundary and it can use a normal instruction to access it, > > or whether it'll do something daft like use byte loads to access it. > > > > We could also do: > > > > + dma_addr_t dma_addr __packed __aligned(sizeof(void *)); > > > > and I see pahole, at least sees this correctly: > > > > struct { > > long unsigned int _page_pool_pad; /* 4 4 */ > > dma_addr_t dma_addr __attribute__((__aligned__(4))); /* 8 8 */ > > } __attribute__((__packed__)) __attribute__((__aligned__(4))); > > > > This presumably affects any 32-bit architecture with a 64-bit phys_addr_t > > / dma_addr_t. Advice, please? > > I'm not sure that the 32-bit behavior is with 64-bit (dma) addrs. > > I don't have any 32-bit boards with 64-bit DMA. Cc. Ivan, wasn't your > board (572x ?) 32-bit with driver 'cpsw' this case (where Ivan added > XDP+page_pool) ? > > -- > Best regards, > Jesper Dangaard Brouer > MSc.CS, Principal Kernel Engineer at Red Hat > LinkedIn: http://www.linkedin.com/in/brouer > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BA02CC433B4 for ; Sat, 10 Apr 2021 11:30:35 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F11CE60241 for ; Sat, 10 Apr 2021 11:30:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F11CE60241 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4FHXqY09jVz3c19 for ; Sat, 10 Apr 2021 21:30:33 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=s0sXCo3d; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=linaro.org (client-ip=2607:f8b0:4864:20::b33; helo=mail-yb1-xb33.google.com; envelope-from=ilias.apalodimas@linaro.org; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=linaro.org header.i=@linaro.org header.a=rsa-sha256 header.s=google header.b=s0sXCo3d; dkim-atps=neutral Received: from mail-yb1-xb33.google.com (mail-yb1-xb33.google.com [IPv6:2607:f8b0:4864:20::b33]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4FHTL25Pjdz302Y for ; Sat, 10 Apr 2021 18:53:11 +1000 (AEST) Received: by mail-yb1-xb33.google.com with SMTP id n12so9218477ybf.8 for ; Sat, 10 Apr 2021 01:53:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7yHuiMf2mbQHC9INRUFE4f0Q8iL2DCKJY335xtgKT+M=; b=s0sXCo3dvECgpdJR7whscVJ1iWTqthnxVLYQ+cWKngNry1eueNvN2t1KtSQvyAKpin 13En8yn6d+Yw7fzHTql9MiHET60pC0eR48G0sg4Y0BJt25ufPVNEf0X34Jx1xzMIG4Hy Y7EKWXgnYUH4e5mY/YU8c2uHjr8njpZ3dTzJt0iYL4TJPYPHOv7FAshptX+jC9Tzkz3z hfvu7Qt1gZ4d5Qsr3NTxYjx2QW7Fdl/cbwNiQ02u18593AOU/WV8l+R+9uqhoWnAmXr9 /+iF2mseThA+yNVXeIr6fq/WlwKArwfqxhv/83FA2rfwSlB5wOfyw2hsXxWSVYF6fqRM JrGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7yHuiMf2mbQHC9INRUFE4f0Q8iL2DCKJY335xtgKT+M=; b=DgSciHPWCZFtQh/z8L12y6Hn9oyNW9rhMPvROkq/3NOIrUhGg6XrbpjSUCjndenQ8N ay01MrjVkBf5qrYmU0h+BRTs2k5gZWx4vVtxFN3R0Pd02MQgq884qQWmhgE+k56wu2b1 eYoLJpIVYlA494uQKQKuF3wa0uxh1nM5HxUTv74t0M11QaY36wPqrFi1F8VMunAvtUlo PbYvkhbxg07ynTyM7yhfFMOO25evRZdeFdQ+YgVZ9BC6mg9PY43yzdgqKOKFZ9R/w3C6 GRMr5EF1S+DHsTGrXBHpS1K5vr0vLaIEng+s/dISMqvP/JEDTGed4nCHRFUqy/g84W18 Mvwg== X-Gm-Message-State: AOAM530V9JIzKC0W4pQ0FDQolOQn17UeE0EJa5TAKrFqOiogjk/1dpIL yXTIouqlOmpFTFJel2fp7IZRRHOtXQJl1lfjJx8VWw== X-Google-Smtp-Source: ABdhPJy5dr9lQOxU6kvADPui00Fq+MldBcPrkITVyRf0//k/8cK2iDR0U0j3GM3hEL13gRzjdmouFNrBWiWVxuk2Mok= X-Received: by 2002:a25:2bc1:: with SMTP id r184mr25473310ybr.51.1618044782263; Sat, 10 Apr 2021 01:53:02 -0700 (PDT) MIME-Version: 1.0 References: <20210409185105.188284-3-willy@infradead.org> <202104100656.N7EVvkNZ-lkp@intel.com> <20210410024313.GX2531743@casper.infradead.org> <20210410082158.79ad09a6@carbon> In-Reply-To: <20210410082158.79ad09a6@carbon> From: Ilias Apalodimas Date: Sat, 10 Apr 2021 11:52:26 +0300 Message-ID: Subject: Re: Bogus struct page layout on 32-bit To: Jesper Dangaard Brouer Content-Type: text/plain; charset="UTF-8" X-Mailman-Approved-At: Sat, 10 Apr 2021 21:30:05 +1000 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Grygorii Strashko , kbuild-all@lists.01.org, kernel test robot , Ivan Khoronzhuk , clang-built-linux@googlegroups.com, open list , Matthew Wilcox , Linux-MM , "netdev@vger.kernel.org" , Paul Mackerras , linux-fsdevel@vger.kernel.org, Matteo Croce , linuxppc-dev@lists.ozlabs.org, "David S. Miller" , Linux ARM Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" +CC Grygorii for the cpsw part as Ivan's email is not valid anymore Thanks for catching this. Interesting indeed... On Sat, 10 Apr 2021 at 09:22, Jesper Dangaard Brouer wrote: > > On Sat, 10 Apr 2021 03:43:13 +0100 > Matthew Wilcox wrote: > > > On Sat, Apr 10, 2021 at 06:45:35AM +0800, kernel test robot wrote: > > > >> include/linux/mm_types.h:274:1: error: static_assert failed due to requirement '__builtin_offsetof(struct page, lru) == __builtin_offsetof(struct folio, lru)' "offsetof(struct page, lru) == offsetof(struct folio, lru)" > > > FOLIO_MATCH(lru, lru); > > > include/linux/mm_types.h:272:2: note: expanded from macro 'FOLIO_MATCH' > > > static_assert(offsetof(struct page, pg) == offsetof(struct folio, fl)) > > > > Well, this is interesting. pahole reports: > > > > struct page { > > long unsigned int flags; /* 0 4 */ > > /* XXX 4 bytes hole, try to pack */ > > union { > > struct { > > struct list_head lru; /* 8 8 */ > > ... > > struct folio { > > union { > > struct { > > long unsigned int flags; /* 0 4 */ > > struct list_head lru; /* 4 8 */ > > > > so this assert has absolutely done its job. > > > > But why has this assert triggered? Why is struct page layout not what > > we thought it was? Turns out it's the dma_addr added in 2019 by commit > > c25fff7171be ("mm: add dma_addr_t to struct page"). On this particular > > config, it's 64-bit, and ppc32 requires alignment to 64-bit. So > > the whole union gets moved out by 4 bytes. > > Argh, good that you are catching this! > > > Unfortunately, we can't just fix this by putting an 'unsigned long pad' > > in front of it. It still aligns the entire union to 8 bytes, and then > > it skips another 4 bytes after the pad. > > > > We can fix it like this ... > > > > +++ b/include/linux/mm_types.h > > @@ -96,11 +96,12 @@ struct page { > > unsigned long private; > > }; > > struct { /* page_pool used by netstack */ > > + unsigned long _page_pool_pad; > > I'm fine with this pad. Matteo is currently proposing[1] to add a 32-bit > value after @dma_addr, and he could use this area instead. > > [1] https://lore.kernel.org/netdev/20210409223801.104657-3-mcroce@linux.microsoft.com/ > > When adding/changing this, we need to make sure that it doesn't overlap > member @index, because network stack use/check page_is_pfmemalloc(). > As far as my calculations this is safe to add. I always try to keep an > eye out for this, but I wonder if we could have a build check like yours. > > > > /** > > * @dma_addr: might require a 64-bit value even on > > * 32-bit architectures. > > */ > > - dma_addr_t dma_addr; > > + dma_addr_t dma_addr __packed; > > }; > > struct { /* slab, slob and slub */ > > union { > > > > but I don't know if GCC is smart enough to realise that dma_addr is now > > on an 8 byte boundary and it can use a normal instruction to access it, > > or whether it'll do something daft like use byte loads to access it. > > > > We could also do: > > > > + dma_addr_t dma_addr __packed __aligned(sizeof(void *)); > > > > and I see pahole, at least sees this correctly: > > > > struct { > > long unsigned int _page_pool_pad; /* 4 4 */ > > dma_addr_t dma_addr __attribute__((__aligned__(4))); /* 8 8 */ > > } __attribute__((__packed__)) __attribute__((__aligned__(4))); > > > > This presumably affects any 32-bit architecture with a 64-bit phys_addr_t > > / dma_addr_t. Advice, please? > > I'm not sure that the 32-bit behavior is with 64-bit (dma) addrs. > > I don't have any 32-bit boards with 64-bit DMA. Cc. Ivan, wasn't your > board (572x ?) 32-bit with driver 'cpsw' this case (where Ivan added > XDP+page_pool) ? > > -- > Best regards, > Jesper Dangaard Brouer > MSc.CS, Principal Kernel Engineer at Red Hat > LinkedIn: http://www.linkedin.com/in/brouer > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC0B8C433ED for ; Sat, 10 Apr 2021 08:54:54 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 601C4610CB for ; Sat, 10 Apr 2021 08:54:54 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 601C4610CB Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linaro.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=X3ZoOonBrOcuY4GxLG6cpSbnUw43L5rrqYnbEtvkDYE=; b=lW7MsL2RU1lWSPXnqGqlhYrri f7QfbOXDFl7NHSXkqVVFYGelv8IcJsY5+hGsa7muwxMj0v5D8heoitMATn4mMxE34nX+aUX0Ob6jT LOTOzD5sqSfMjp3hP+1ApeW0RRIuQbRahQ8C19UTtGP8p98r+x+aC/tIluNv/l0IZOC+9o9upaukV QK7Er1yprFYIZONAUO3XrItRxsECMmuU1b4tLpgxN8hBUpxj/sTPP54q4WAJBbBT7I0BYjFRJtEhm 2eG+YeIKlRvZXyAn+a4DuOZCOpgL1V3SAxXXgnsWjm7EEgW3RCiIzthXCIuexxtZUPuQKIqstSZBb KxWbWFPEg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lV9Mc-002Suy-Bt; Sat, 10 Apr 2021 08:53:10 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lV9MZ-002SuJ-0C for linux-arm-kernel@desiato.infradead.org; Sat, 10 Apr 2021 08:53:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Content-Type:Cc:To:Subject:Message-ID :Date:From:In-Reply-To:References:MIME-Version:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=7yHuiMf2mbQHC9INRUFE4f0Q8iL2DCKJY335xtgKT+M=; b=NKBKIm9P63CThT1onPdXBDx9tb pzOJhwOijk1TOrj3MuwVfl6LoIGqdtM17s9lCij8+c3a+O41bHZ7N+MCWDbcXtAlZrJyWn0QbErTC 9EF7jmAWBF6NHHfBBfsrILL3JHeA6vzgb0QaOXDecsxmMTRFhG+O39WVQSxGYyqHeaVc3WYfHwU+K cOod6egS9jezLV1UmQ1w4K8Jbs2IAdyffuYfu3tPoLJekK587cDmYGZleQz/RwwOw7HYBBqUK/HWP PfYYBCY9lq77ckJUpZA7AjYVyTFiX6HMdmJMmG7Ns52BWB68rdhwxO8ononnBDP4k6RlKMeYHldRc pvXXFfng==; Received: from mail-yb1-xb2a.google.com ([2607:f8b0:4864:20::b2a]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lV9MV-004zqa-Lu for linux-arm-kernel@lists.infradead.org; Sat, 10 Apr 2021 08:53:05 +0000 Received: by mail-yb1-xb2a.google.com with SMTP id y2so7132901ybq.13 for ; Sat, 10 Apr 2021 01:53:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7yHuiMf2mbQHC9INRUFE4f0Q8iL2DCKJY335xtgKT+M=; b=s0sXCo3dvECgpdJR7whscVJ1iWTqthnxVLYQ+cWKngNry1eueNvN2t1KtSQvyAKpin 13En8yn6d+Yw7fzHTql9MiHET60pC0eR48G0sg4Y0BJt25ufPVNEf0X34Jx1xzMIG4Hy Y7EKWXgnYUH4e5mY/YU8c2uHjr8njpZ3dTzJt0iYL4TJPYPHOv7FAshptX+jC9Tzkz3z hfvu7Qt1gZ4d5Qsr3NTxYjx2QW7Fdl/cbwNiQ02u18593AOU/WV8l+R+9uqhoWnAmXr9 /+iF2mseThA+yNVXeIr6fq/WlwKArwfqxhv/83FA2rfwSlB5wOfyw2hsXxWSVYF6fqRM JrGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7yHuiMf2mbQHC9INRUFE4f0Q8iL2DCKJY335xtgKT+M=; b=EPB97pG7fMXnDklCBfEBxeUezlONl702WIPgb91croJdTAW/8szFRLf1zeR+fHah1V zDQ9EftWQiF01m7akH90KRMmg36OJG1VblXCYMoTa/TOucefZ8NZmcW5HIahndGnaZ2Z vIiiutsKzKyJ3RecBP93mrnoPGZELR2BWnkpUWwyznRtKPuXAWqu34663xQzLnjpquOO WKYZ1hQzn385ke/pu7sG/OfQ/HQvf4gZ/q5PNtR+Sf5u37845rpUFfkCOEd3wmoD9cdF +2VFeOGNnNfcp/Os9dF5qn4YeR0bwI6FXCbLhGE5FTcwPOuDTo9BD3npZrvyxHoj5f+2 Wklg== X-Gm-Message-State: AOAM530Y245dWsLSORvAndb/6qci2Ai4UD6gsrXvpZJFOPI1UcrCjHFx 0p19f8MJtOZdQhMdpzru4cmMIccpIivK/Iv86vTMKg== X-Google-Smtp-Source: ABdhPJy5dr9lQOxU6kvADPui00Fq+MldBcPrkITVyRf0//k/8cK2iDR0U0j3GM3hEL13gRzjdmouFNrBWiWVxuk2Mok= X-Received: by 2002:a25:2bc1:: with SMTP id r184mr25473310ybr.51.1618044782263; Sat, 10 Apr 2021 01:53:02 -0700 (PDT) MIME-Version: 1.0 References: <20210409185105.188284-3-willy@infradead.org> <202104100656.N7EVvkNZ-lkp@intel.com> <20210410024313.GX2531743@casper.infradead.org> <20210410082158.79ad09a6@carbon> In-Reply-To: <20210410082158.79ad09a6@carbon> From: Ilias Apalodimas Date: Sat, 10 Apr 2021 11:52:26 +0300 Message-ID: Subject: Re: Bogus struct page layout on 32-bit To: Jesper Dangaard Brouer Cc: Matthew Wilcox , kernel test robot , Linux-MM , kbuild-all@lists.01.org, clang-built-linux@googlegroups.com, open list , linux-fsdevel@vger.kernel.org, Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , linuxppc-dev@lists.ozlabs.org, Linux ARM , "David S. Miller" , Ivan Khoronzhuk , Matteo Croce , "netdev@vger.kernel.org" , Grygorii Strashko X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210410_015303_762228_109E7DB4 X-CRM114-Status: GOOD ( 38.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org +CC Grygorii for the cpsw part as Ivan's email is not valid anymore Thanks for catching this. Interesting indeed... On Sat, 10 Apr 2021 at 09:22, Jesper Dangaard Brouer wrote: > > On Sat, 10 Apr 2021 03:43:13 +0100 > Matthew Wilcox wrote: > > > On Sat, Apr 10, 2021 at 06:45:35AM +0800, kernel test robot wrote: > > > >> include/linux/mm_types.h:274:1: error: static_assert failed due to requirement '__builtin_offsetof(struct page, lru) == __builtin_offsetof(struct folio, lru)' "offsetof(struct page, lru) == offsetof(struct folio, lru)" > > > FOLIO_MATCH(lru, lru); > > > include/linux/mm_types.h:272:2: note: expanded from macro 'FOLIO_MATCH' > > > static_assert(offsetof(struct page, pg) == offsetof(struct folio, fl)) > > > > Well, this is interesting. pahole reports: > > > > struct page { > > long unsigned int flags; /* 0 4 */ > > /* XXX 4 bytes hole, try to pack */ > > union { > > struct { > > struct list_head lru; /* 8 8 */ > > ... > > struct folio { > > union { > > struct { > > long unsigned int flags; /* 0 4 */ > > struct list_head lru; /* 4 8 */ > > > > so this assert has absolutely done its job. > > > > But why has this assert triggered? Why is struct page layout not what > > we thought it was? Turns out it's the dma_addr added in 2019 by commit > > c25fff7171be ("mm: add dma_addr_t to struct page"). On this particular > > config, it's 64-bit, and ppc32 requires alignment to 64-bit. So > > the whole union gets moved out by 4 bytes. > > Argh, good that you are catching this! > > > Unfortunately, we can't just fix this by putting an 'unsigned long pad' > > in front of it. It still aligns the entire union to 8 bytes, and then > > it skips another 4 bytes after the pad. > > > > We can fix it like this ... > > > > +++ b/include/linux/mm_types.h > > @@ -96,11 +96,12 @@ struct page { > > unsigned long private; > > }; > > struct { /* page_pool used by netstack */ > > + unsigned long _page_pool_pad; > > I'm fine with this pad. Matteo is currently proposing[1] to add a 32-bit > value after @dma_addr, and he could use this area instead. > > [1] https://lore.kernel.org/netdev/20210409223801.104657-3-mcroce@linux.microsoft.com/ > > When adding/changing this, we need to make sure that it doesn't overlap > member @index, because network stack use/check page_is_pfmemalloc(). > As far as my calculations this is safe to add. I always try to keep an > eye out for this, but I wonder if we could have a build check like yours. > > > > /** > > * @dma_addr: might require a 64-bit value even on > > * 32-bit architectures. > > */ > > - dma_addr_t dma_addr; > > + dma_addr_t dma_addr __packed; > > }; > > struct { /* slab, slob and slub */ > > union { > > > > but I don't know if GCC is smart enough to realise that dma_addr is now > > on an 8 byte boundary and it can use a normal instruction to access it, > > or whether it'll do something daft like use byte loads to access it. > > > > We could also do: > > > > + dma_addr_t dma_addr __packed __aligned(sizeof(void *)); > > > > and I see pahole, at least sees this correctly: > > > > struct { > > long unsigned int _page_pool_pad; /* 4 4 */ > > dma_addr_t dma_addr __attribute__((__aligned__(4))); /* 8 8 */ > > } __attribute__((__packed__)) __attribute__((__aligned__(4))); > > > > This presumably affects any 32-bit architecture with a 64-bit phys_addr_t > > / dma_addr_t. Advice, please? > > I'm not sure that the 32-bit behavior is with 64-bit (dma) addrs. > > I don't have any 32-bit boards with 64-bit DMA. Cc. Ivan, wasn't your > board (572x ?) 32-bit with driver 'cpsw' this case (where Ivan added > XDP+page_pool) ? > > -- > Best regards, > Jesper Dangaard Brouer > MSc.CS, Principal Kernel Engineer at Red Hat > LinkedIn: http://www.linkedin.com/in/brouer > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: multipart/mixed; boundary="===============6719045253504605550==" MIME-Version: 1.0 From: Ilias Apalodimas To: kbuild-all@lists.01.org Subject: Re: Bogus struct page layout on 32-bit Date: Sat, 10 Apr 2021 11:52:26 +0300 Message-ID: In-Reply-To: <20210410082158.79ad09a6@carbon> List-Id: --===============6719045253504605550== Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable +CC Grygorii for the cpsw part as Ivan's email is not valid anymore Thanks for catching this. Interesting indeed... On Sat, 10 Apr 2021 at 09:22, Jesper Dangaard Brouer = wrote: > > On Sat, 10 Apr 2021 03:43:13 +0100 > Matthew Wilcox wrote: > > > On Sat, Apr 10, 2021 at 06:45:35AM +0800, kernel test robot wrote: > > > >> include/linux/mm_types.h:274:1: error: static_assert failed due to= requirement '__builtin_offsetof(struct page, lru) =3D=3D __builtin_offseto= f(struct folio, lru)' "offsetof(struct page, lru) =3D=3D offsetof(struct fo= lio, lru)" > > > FOLIO_MATCH(lru, lru); > > > include/linux/mm_types.h:272:2: note: expanded from macro 'FOLIO_M= ATCH' > > > static_assert(offsetof(struct page, pg) =3D=3D offsetof(st= ruct folio, fl)) > > > > Well, this is interesting. pahole reports: > > > > struct page { > > long unsigned int flags; /* 0 4= */ > > /* XXX 4 bytes hole, try to pack */ > > union { > > struct { > > struct list_head lru; /* 8 8= */ > > ... > > struct folio { > > union { > > struct { > > long unsigned int flags; /* 0 4= */ > > struct list_head lru; /* 4 8= */ > > > > so this assert has absolutely done its job. > > > > But why has this assert triggered? Why is struct page layout not what > > we thought it was? Turns out it's the dma_addr added in 2019 by commit > > c25fff7171be ("mm: add dma_addr_t to struct page"). On this particular > > config, it's 64-bit, and ppc32 requires alignment to 64-bit. So > > the whole union gets moved out by 4 bytes. > > Argh, good that you are catching this! > > > Unfortunately, we can't just fix this by putting an 'unsigned long pad' > > in front of it. It still aligns the entire union to 8 bytes, and then > > it skips another 4 bytes after the pad. > > > > We can fix it like this ... > > > > +++ b/include/linux/mm_types.h > > @@ -96,11 +96,12 @@ struct page { > > unsigned long private; > > }; > > struct { /* page_pool used by netstack */ > > + unsigned long _page_pool_pad; > > I'm fine with this pad. Matteo is currently proposing[1] to add a 32-bit > value after @dma_addr, and he could use this area instead. > > [1] https://lore.kernel.org/netdev/20210409223801.104657-3-mcroce(a)linux= .microsoft.com/ > > When adding/changing this, we need to make sure that it doesn't overlap > member @index, because network stack use/check page_is_pfmemalloc(). > As far as my calculations this is safe to add. I always try to keep an > eye out for this, but I wonder if we could have a build check like yours. > > > > /** > > * @dma_addr: might require a 64-bit value even= on > > * 32-bit architectures. > > */ > > - dma_addr_t dma_addr; > > + dma_addr_t dma_addr __packed; > > }; > > struct { /* slab, slob and slub */ > > union { > > > > but I don't know if GCC is smart enough to realise that dma_addr is now > > on an 8 byte boundary and it can use a normal instruction to access it, > > or whether it'll do something daft like use byte loads to access it. > > > > We could also do: > > > > + dma_addr_t dma_addr __packed __aligned(sizeof(v= oid *)); > > > > and I see pahole, at least sees this correctly: > > > > struct { > > long unsigned int _page_pool_pad; /* 4 = 4 */ > > dma_addr_t dma_addr __attribute__((__aligned__(= 4))); /* 8 8 */ > > } __attribute__((__packed__)) __attribute__((__aligned_= _(4))); > > > > This presumably affects any 32-bit architecture with a 64-bit phys_addr= _t > > / dma_addr_t. Advice, please? > > I'm not sure that the 32-bit behavior is with 64-bit (dma) addrs. > > I don't have any 32-bit boards with 64-bit DMA. Cc. Ivan, wasn't your > board (572x ?) 32-bit with driver 'cpsw' this case (where Ivan added > XDP+page_pool) ? > > -- > Best regards, > Jesper Dangaard Brouer > MSc.CS, Principal Kernel Engineer at Red Hat > LinkedIn: http://www.linkedin.com/in/brouer > --===============6719045253504605550==--