linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Björn Töpel" <bjorn.topel@gmail.com>
To: Jonathan Lemon <jonathan.lemon@gmail.com>
Cc: "Nathan Chancellor" <natechancellor@gmail.com>,
	"Björn Töpel" <bjorn.topel@intel.com>,
	"Magnus Karlsson" <magnus.karlsson@intel.com>,
	"David S. Miller" <davem@davemloft.net>,
	"Alexei Starovoitov" <ast@kernel.org>,
	"Daniel Borkmann" <daniel@iogearbox.net>,
	"Jakub Kicinski" <jakub.kicinski@netronome.com>,
	"Jesper Dangaard Brouer" <hawk@kernel.org>,
	"John Fastabend" <john.fastabend@gmail.com>,
	Netdev <netdev@vger.kernel.org>, bpf <bpf@vger.kernel.org>,
	Xdp <xdp-newbies@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	clang-built-linux@googlegroups.com,
	"Nick Desaulniers" <ndesaulniers@google.com>,
	"Nathan Huckleberry" <nhuck@google.com>
Subject: Re: [PATCH] xsk: Properly terminate assignment in xskq_produce_flush_desc
Date: Tue, 25 Jun 2019 22:49:52 +0200	[thread overview]
Message-ID: <CAJ+HfNjKHG2dmu_juCJE5Xjo4HR4wqfk=yNPSAz8i7YbEWq6uw@mail.gmail.com> (raw)
In-Reply-To: <34F07894-FDE7-44F8-B7F2-E2003D550AD2@gmail.com>

On Tue, 25 Jun 2019 at 22:04, Jonathan Lemon <jonathan.lemon@gmail.com> wrote:
>
>
>
> On 25 Jun 2019, at 11:23, Nathan Chancellor wrote:
>
> > Clang warns:
> >
> > In file included from net/xdp/xsk_queue.c:10:
> > net/xdp/xsk_queue.h:292:2: warning: expression result unused
> > [-Wunused-value]
> >         WRITE_ONCE(q->ring->producer, q->prod_tail);
> >         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > include/linux/compiler.h:284:6: note: expanded from macro 'WRITE_ONCE'
> >         __u.__val;                                      \
> >         ~~~ ^~~~~
> > 1 warning generated.
> >
> > The q->prod_tail assignment has a comma at the end, not a semi-colon.
> > Fix that so clang no longer warns and everything works as expected.
> >
> > Fixes: c497176cb2e4 ("xsk: add Rx receive functions and poll support")
> > Link: https://github.com/ClangBuiltLinux/linux/issues/544
> > Signed-off-by: Nathan Chancellor <natechancellor@gmail.com>
>
> Nice find.
>
> Acked-by: Jonathan Lemon <jonathan.lemon@gmail.com>
>

Yikes. Yes, nice find, indeed.

Acked-by: Björn Töpel <bjorn.topel@intel.com>

The broader question is "Why does it work at all?", which is an "oh no" moment.

The problematic functions are xsk_flush() and xsk_generic_rcv, where
xskq_produce_flush_desc() is inlined. On the test machine, the GCC
version is:

$ gcc --version
gcc (Ubuntu 7.4.0-1ubuntu1~18.04) 7.4.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

I when I diff the output, both .lst and .o:

$ diff -u old.lst new.lst
--- old.lst     2019-06-25 22:10:57.709591605 +0200
+++ new.lst     2019-06-25 22:10:35.301359865 +0200
@@ -2480,7 +2480,7 @@
     1566:      48 8b 87 e0 02 00 00    mov    0x2e0(%rdi),%rax
 {
     156d:      48 89 e5                mov    %rsp,%rbp
-       q->prod_tail = q->prod_head,
+       q->prod_tail = q->prod_head;
     1570:      8b 50 18                mov    0x18(%rax),%edx
     1573:      89 50 1c                mov    %edx,0x1c(%rax)
        WRITE_ONCE(q->ring->producer, q->prod_tail);
@@ -2649,7 +2649,7 @@
     16fb:      83 40 24 01             addl   $0x1,0x24(%rax)
        xskq_produce_flush_desc(xs->rx);
     16ff:      49 8b 86 e0 02 00 00    mov    0x2e0(%r14),%rax
-       q->prod_tail = q->prod_head,
+       q->prod_tail = q->prod_head;
     1706:      8b 50 18                mov    0x18(%rax),%edx
        xs->sk.sk_data_ready(&xs->sk);
     1709:      4c 89 f7                mov    %r14,%rdi

$ diff -u <(gdb -batch -ex 'file old.o' -ex 'disassemble xsk_flush')
<(gdb -batch -ex 'file new.o' -ex 'disassemble xsk_flush') && echo
"Whew"
Whew

$ diff -u <(gdb -batch -ex 'file old.o' -ex 'disassemble
xsk_generic_rcv') <(gdb -batch -ex 'file new.o' -ex 'disassemble
xsk_generic_rcv') && echo "Whew"
Whew

struct xsk_queue {
        u64                        chunk_mask;           /*     0   0x8 */
        u64                        size;                 /*   0x8   0x8 */
        u32                        ring_mask;            /*  0x10   0x4 */
        u32                        nentries;             /*  0x14   0x4 */
        u32                        prod_head;            /*  0x18   0x4 */
        u32                        prod_tail;            /*  0x1c   0x4 */
        u32                        cons_head;            /*  0x20   0x4 */
        u32                        cons_tail;            /*  0x24   0x4 */
        struct xdp_ring *          ring;                 /*  0x28   0x8 */
        u64                        invalid_descs;        /*  0x30   0x8 */

        /* size: 56, cachelines: 1, members: 10 */
        /* last cacheline: 56 bytes */
};

So, it appears that the generated code is equal, both in xsk_flush()
and xsk_generic_rcv() where flush was inlined. I'll be digging into
more GCC versions, and observe the generated code.

Regardless, this was a really good find. Thank you very much! Clang is
added to my kernel build workflow from now on...


Björn



>
> > ---
> >  net/xdp/xsk_queue.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/net/xdp/xsk_queue.h b/net/xdp/xsk_queue.h
> > index 88b9ae24658d..cba4a640d5e8 100644
> > --- a/net/xdp/xsk_queue.h
> > +++ b/net/xdp/xsk_queue.h
> > @@ -288,7 +288,7 @@ static inline void xskq_produce_flush_desc(struct
> > xsk_queue *q)
> >       /* Order producer and data */
> >       smp_wmb(); /* B, matches C */
> >
> > -     q->prod_tail = q->prod_head,
> > +     q->prod_tail = q->prod_head;
> >       WRITE_ONCE(q->ring->producer, q->prod_tail);
> >  }
> >
> > --
> > 2.22.0

  reply	other threads:[~2019-06-25 20:50 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-06-25 18:23 Nathan Chancellor
2019-06-25 18:32 ` Nick Desaulniers
2019-06-25 19:46 ` Jonathan Lemon
2019-06-25 20:49   ` Björn Töpel [this message]
2019-06-26  1:13 ` Song Liu
2019-06-26 14:30 ` Daniel Borkmann

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAJ+HfNjKHG2dmu_juCJE5Xjo4HR4wqfk=yNPSAz8i7YbEWq6uw@mail.gmail.com' \
    --to=bjorn.topel@gmail.com \
    --cc=ast@kernel.org \
    --cc=bjorn.topel@intel.com \
    --cc=bpf@vger.kernel.org \
    --cc=clang-built-linux@googlegroups.com \
    --cc=daniel@iogearbox.net \
    --cc=davem@davemloft.net \
    --cc=hawk@kernel.org \
    --cc=jakub.kicinski@netronome.com \
    --cc=john.fastabend@gmail.com \
    --cc=jonathan.lemon@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=magnus.karlsson@intel.com \
    --cc=natechancellor@gmail.com \
    --cc=ndesaulniers@google.com \
    --cc=netdev@vger.kernel.org \
    --cc=nhuck@google.com \
    --cc=xdp-newbies@vger.kernel.org \
    --subject='Re: [PATCH] xsk: Properly terminate assignment in xskq_produce_flush_desc' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).