From: Bartosz Golaszewski <brgl@bgdev.pl>
To: Arnd Bergmann <arnd@arndb.de>
Cc: Jonathan Corbet <corbet@lwn.net>,
Rob Herring <robh+dt@kernel.org>,
"David S . Miller" <davem@davemloft.net>,
Matthias Brugger <matthias.bgg@gmail.com>,
John Crispin <john@phrozen.org>,
Sean Wang <sean.wang@mediatek.com>,
Mark Lee <Mark-MC.Lee@mediatek.com>,
Jakub Kicinski <kuba@kernel.org>,
Fabien Parent <fparent@baylibre.com>,
Heiner Kallweit <hkallweit1@gmail.com>,
Edwin Peer <edwin.peer@broadcom.com>,
DTML <devicetree@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Networking <netdev@vger.kernel.org>,
Linux ARM <linux-arm-kernel@lists.infradead.org>,
"moderated list:ARM/Mediatek SoC..."
<linux-mediatek@lists.infradead.org>,
Stephane Le Provost <stephane.leprovost@mediatek.com>,
Pedro Tsai <pedro.tsai@mediatek.com>,
Andrew Perepech <andrew.perepech@mediatek.com>,
Bartosz Golaszewski <bgolaszewski@baylibre.com>
Subject: Re: [PATCH v3 10/15] net: ethernet: mtk-eth-mac: new driver
Date: Mon, 18 May 2020 16:07:23 +0200 [thread overview]
Message-ID: <CAMRc=MeVyNzTWw_hk=J9kX1NE9reCE_O4P3wrNpMMc9z4xA_DA@mail.gmail.com> (raw)
In-Reply-To: <CAK8P3a0XgJtZNKePZUUpzADO25-JZKyDiVHFS_yuHRXTjvjDwg@mail.gmail.com>
pt., 15 maj 2020 o 15:32 Arnd Bergmann <arnd@arndb.de> napisał(a):
>
> On Thu, May 14, 2020 at 10:00 AM Bartosz Golaszewski <brgl@bgdev.pl> wrote:
> > +static int mtk_mac_ring_pop_tail(struct mtk_mac_ring *ring,
> > + struct mtk_mac_ring_desc_data *desc_data)
>
> I took another look at this function because of your comment on the locking
> the descriptor updates, which seemed suspicious as the device side does not
> actually use the locks to access them
>
> > +{
> > + struct mtk_mac_ring_desc *desc = &ring->descs[ring->tail];
> > + unsigned int status;
> > +
> > + /* Let the device release the descriptor. */
> > + dma_rmb();
> > + status = desc->status;
> > + if (!(status & MTK_MAC_DESC_BIT_COWN))
> > + return -1;
>
> The dma_rmb() seems odd here, as I don't see which prior read
> is being protected by this.
>
> > + desc_data->len = status & MTK_MAC_DESC_MSK_LEN;
> > + desc_data->flags = status & ~MTK_MAC_DESC_MSK_LEN;
> > + desc_data->dma_addr = ring->dma_addrs[ring->tail];
> > + desc_data->skb = ring->skbs[ring->tail];
> > +
> > + desc->data_ptr = 0;
> > + desc->status = MTK_MAC_DESC_BIT_COWN;
> > + if (status & MTK_MAC_DESC_BIT_EOR)
> > + desc->status |= MTK_MAC_DESC_BIT_EOR;
> > +
> > + /* Flush writes to descriptor memory. */
> > + dma_wmb();
>
> The comment and the barrier here seem odd as well. I would have expected
> a barrier after the update to the data pointer, and only a single store
> but no read of the status flag instead of the read-modify-write,
> something like
>
> desc->data_ptr = 0;
> dma_wmb(); /* make pointer update visible before status update */
> desc->status = MTK_MAC_DESC_BIT_COWN | (status & MTK_MAC_DESC_BIT_EOR);
>
> > + ring->tail = (ring->tail + 1) % MTK_MAC_RING_NUM_DESCS;
> > + ring->count--;
>
> I would get rid of the 'count' here, as it duplicates the information
> that is already known from the difference between head and tail, and you
> can't update it atomically without holding a lock around the access to
> the ring. The way I'd do this is to have the head and tail pointers
> in separate cache lines, and then use READ_ONCE/WRITE_ONCE
> and smp barriers to access them, with each one updated on one
> thread but read by the other.
>
Your previous solution seems much more reliable though. For instance
in the above: when we're doing the TX cleanup (we got the TX ready
irq, we're iterating over descriptors until we know there are no more
packets scheduled (count == 0) or we encounter one that's still owned
by DMA), a parallel TX path can schedule new packets to be sent and I
don't see how we can atomically check the count (understood as a
difference between tail and head) and run a new iteration (where we'd
modify the head or tail) without risking the other path getting in the
way. We'd have to always check the descriptor.
I experimented a bit with this and couldn't come up with anything that
would pass any stress test.
On the other hand: spin_lock_bh() works fine and I like your approach
from the previous e-mail - except for the work for updating stats as
we could potentially lose some stats when we're updating in process
context with RX/TX paths running in parallel in napi context but that
would be rare enough to overlook it.
I hope v4 will be good enough even with spinlocks. :)
Bart
next prev parent reply other threads:[~2020-05-18 14:07 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-14 7:59 [PATCH v3 00/15] mediatek: add support for MediaTek Ethernet MAC Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 01/15] dt-bindings: convert the binding document for mediatek PERICFG to yaml Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 02/15] dt-bindings: add new compatible to mediatek,pericfg Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 03/15] dt-bindings: net: add a binding document for MediaTek Ethernet MAC Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 04/15] net: ethernet: mediatek: rename Kconfig prompt Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 05/15] net: ethernet: mediatek: remove unnecessary spaces from Makefile Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 06/15] Documentation: devres: add a missing section for networking helpers Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 07/15] net: move devres helpers into a separate source file Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 08/15] net: devres: define a separate devres structure for devm_alloc_etherdev() Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 09/15] net: devres: provide devm_register_netdev() Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 10/15] net: ethernet: mtk-eth-mac: new driver Bartosz Golaszewski
2020-05-14 16:19 ` Arnd Bergmann
2020-05-15 7:11 ` Bartosz Golaszewski
2020-05-15 12:04 ` Arnd Bergmann
2020-05-15 12:56 ` Bartosz Golaszewski
2020-05-15 13:11 ` Arnd Bergmann
2020-05-15 13:14 ` Andrew Lunn
2020-05-15 13:32 ` Arnd Bergmann
2020-05-18 14:07 ` Bartosz Golaszewski [this message]
2020-05-18 14:38 ` Arnd Bergmann
2020-05-14 7:59 ` [PATCH v3 11/15] ARM64: dts: mediatek: add pericfg syscon to mt8516.dtsi Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 12/15] ARM64: dts: mediatek: add the ethernet node " Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 13/15] ARM64: dts: mediatek: add an alias for ethernet0 for pumpkin boards Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 14/15] ARM64: dts: mediatek: add ethernet pins " Bartosz Golaszewski
2020-05-14 7:59 ` [PATCH v3 15/15] ARM64: dts: mediatek: enable ethernet on " Bartosz Golaszewski
2020-05-14 19:56 ` [PATCH v3 00/15] mediatek: add support for MediaTek Ethernet MAC David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAMRc=MeVyNzTWw_hk=J9kX1NE9reCE_O4P3wrNpMMc9z4xA_DA@mail.gmail.com' \
--to=brgl@bgdev.pl \
--cc=Mark-MC.Lee@mediatek.com \
--cc=andrew.perepech@mediatek.com \
--cc=arnd@arndb.de \
--cc=bgolaszewski@baylibre.com \
--cc=corbet@lwn.net \
--cc=davem@davemloft.net \
--cc=devicetree@vger.kernel.org \
--cc=edwin.peer@broadcom.com \
--cc=fparent@baylibre.com \
--cc=hkallweit1@gmail.com \
--cc=john@phrozen.org \
--cc=kuba@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mediatek@lists.infradead.org \
--cc=matthias.bgg@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=pedro.tsai@mediatek.com \
--cc=robh+dt@kernel.org \
--cc=sean.wang@mediatek.com \
--cc=stephane.leprovost@mediatek.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).