From: Marcin Wojtas <mw@semihalf.com>
To: "Russell King (Oracle)" <linux@armlinux.org.uk>
Cc: "Marek Behún" <kabel@kernel.org>,
pali@kernel.org, "Christoph Hellwig" <hch@lst.de>,
"Robin Murphy" <robin.murphy@arm.com>,
"Arnd Bergmann" <arnd@kernel.org>,
"Andre Przywara" <andre.przywara@arm.com>,
"Marc Zyngier" <maz@kernel.org>,
"Linus Torvalds" <torvalds@linux-foundation.org>,
"Andrew Lunn" <andrew@lunn.ch>,
"Gregory Clement" <gregory.clement@bootlin.com>,
"Greg Kroah-Hartman" <gregkh@linuxfoundation.org>,
iommu@lists.linux-foundation.org,
linux-arm-kernel@lists.infradead.org
Subject: Re: REGRESSION in 6.0-rc7 caused by patch "ARM/dma-mapping: use dma-direct unconditionally"
Date: Tue, 4 Oct 2022 10:30:40 +0200 [thread overview]
Message-ID: <CAPv3WKeWDWGaMs4VxcLb01tzEz+1pVs5do_MbMdOpvOC7J_DTw@mail.gmail.com> (raw)
In-Reply-To: <Yzvf1TNJebkJic/s@shell.armlinux.org.uk>
wt., 4 paź 2022 o 09:25 Russell King (Oracle) <linux@armlinux.org.uk>
napisał(a):
>
> On Mon, Oct 03, 2022 at 11:30:31PM +0200, Marcin Wojtas wrote:
> > I think the DMA_FROM_DEVICE is used rather properly in the RX path of
> > the driver - the CPU doesn't access the payload afterward.
>
> Please look at the bigger picture rather than concentrating on what
> happens when a packet is received.
>
> The issue is that the CPU writes to the buffer prior to handing the
> buffer over to the hardware, and then the BM reads from the buffer.
> This is DMA _to_ the device no matter how you look at it.
>
> The BM later writes the received packet to the buffer. This is DMA
> _from_ the device.
>
> So, we have DMA in both directions, hence it really is bidirectional,
> and using DMA_FROM_DEVICE in the RX path _is_ incorrect.
>
> Architectures _can_ and _do_ invalidate the data cache when mapping a
> buffer for DMA_FROM_DEVICE and any writes in the data cache for that
> buffer will be discarded. If the writes don't hit the data cache, then
> they will be unaffected by this.
>
> IMHO, having read the docs on the buffer manager, the use of
> DMA_FROM_DEVICE in mvneta in this path is a long-standing bug - it
> should be bidirectional for the reason I state above - the hardware
> both reads and writes the buffer that is passed to it, expecting to
> read data written by the CPU initially.
Thanks for the explanation and I agree with your reasoning. Therefore
the below should be sufficient if we use HW BM and non-coherent
setting:
--- a/drivers/net/ethernet/marvell/mvneta_bm.c
+++ b/drivers/net/ethernet/marvell/mvneta_bm.c
@@ -103,7 +103,7 @@ int mvneta_bm_construct(struct hwbm_pool
*hwbm_pool, void *buf)
*/
*(u32 *)buf = (u32)buf;
phys_addr = dma_map_single(&priv->pdev->dev, buf, bm_pool->buf_size,
- DMA_FROM_DEVICE);
+ DMA_BIDIRECTIONAL);
if (unlikely(dma_mapping_error(&priv->pdev->dev, phys_addr)))
return -ENOMEM;
Marek - can you please confirm that?
>
> > Another thought - when writing to *buf (memory normal) shouldn't we add a dsb()?
>
> That will make no difference to this - this is not about barriers, it's
> about caches.
>
Sure.
> > I have one overall concern here. On all kinds of A38x-based boards I
> > worked on, by default, the firmware set all devices (e.g. network,
> > AHCI, XHCI) on MBUS as fully IO cache coherent - it should be
> > reflected in the MVNETA_WIN_BASE(w) registers attribute field. Bits
> > [15:8] should be set to 0x1D (or 0x1E if there is a second DRAM CS
> > used). Can you please try adding 'dma-coherent;' property under the
> > 'internal-regs' node?
>
> I did notice that there was no dma-coherent markings in DT, which means
> that the DMA API will assume the device is non-coherent, and cache
> maintenance will be performed. If it is dma-coherent, then cache
> maintenance won't be performed, and DT needs to be updated to indicate
> this.
>
> If firmware is making the devices DMA coherent, and it's under firmware
> control, then shouldn't firmware also be updating the kernel's device
> tree to indicate how it's configured the hardware coherency?
>
Imo there are too many boxes out there and updating firmware in the
field is rather no-go. We already handle this in kernel / DT in big
extent, so I think we should stick to that path.
Thanks,
Marcin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2022-10-04 8:32 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-09-30 13:10 REGRESSION in 6.0-rc7 caused by patch "ARM/dma-mapping: use dma-direct unconditionally" Marek Behún
2022-09-30 13:46 ` Robin Murphy
2022-09-30 14:52 ` Marek Behún
2022-09-30 15:02 ` Marek Behún
2022-09-30 16:41 ` Robin Murphy
2022-09-30 18:02 ` Marek Behún
2022-10-03 7:21 ` Christoph Hellwig
2022-10-03 7:30 ` Christoph Hellwig
2022-10-03 14:11 ` Russell King (Oracle)
2022-10-03 15:25 ` Marek Behún
2022-10-03 16:09 ` Pali Rohár
2022-10-03 19:04 ` Marek Behún
2022-10-03 19:08 ` Pali Rohár
2022-10-03 21:30 ` Marcin Wojtas
2022-10-03 21:35 ` Pali Rohár
2022-10-03 22:03 ` Marcin Wojtas
2022-10-04 7:10 ` Christoph Hellwig
2022-10-04 8:15 ` Marek Behún
2022-10-04 8:17 ` [PATCH] ARM: mvebu: select OF_DMA_DEFAULT_COHERENT if MACH_MVEBU_V7 Marek Behún
2022-10-04 8:30 ` Christoph Hellwig
2022-10-04 12:54 ` Marek Behún
2022-10-04 8:30 ` Arnd Bergmann
2022-10-04 9:14 ` Thorsten Leemhuis
2022-10-04 9:22 ` Russell King (Oracle)
2022-10-04 9:56 ` REGRESSION in 6.0-rc7 caused by patch "ARM/dma-mapping: use dma-direct unconditionally" Robin Murphy
2022-10-04 7:25 ` Russell King (Oracle)
2022-10-04 8:30 ` Marcin Wojtas [this message]
2022-10-04 9:08 ` Russell King (Oracle)
2022-10-04 12:36 ` Marek Behún
2022-10-04 12:59 ` Marcin Wojtas
2022-10-04 18:51 ` Pali Rohár
2022-10-04 19:35 ` Marcin Wojtas
2022-10-04 8:26 ` Marek Behún
2022-10-04 8:36 ` Marcin Wojtas
2022-10-20 18:22 ` Russell King (Oracle)
2022-10-20 19:10 ` Marek Behún
2022-10-21 16:25 ` Linus Torvalds
2022-10-21 16:30 ` Christoph Hellwig
2022-10-21 18:21 ` Russell King (Oracle)
2022-10-23 11:58 ` Klaus Kudielka
2022-10-03 18:57 ` Marek Behún
2022-10-01 9:31 ` Thorsten Leemhuis
2022-11-04 12:08 ` REGRESSION in 6.0-rc7 caused by patch "ARM/dma-mapping: use dma-direct unconditionally" #forregzbot Thorsten Leemhuis
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAPv3WKeWDWGaMs4VxcLb01tzEz+1pVs5do_MbMdOpvOC7J_DTw@mail.gmail.com \
--to=mw@semihalf.com \
--cc=andre.przywara@arm.com \
--cc=andrew@lunn.ch \
--cc=arnd@kernel.org \
--cc=gregkh@linuxfoundation.org \
--cc=gregory.clement@bootlin.com \
--cc=hch@lst.de \
--cc=iommu@lists.linux-foundation.org \
--cc=kabel@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux@armlinux.org.uk \
--cc=maz@kernel.org \
--cc=pali@kernel.org \
--cc=robin.murphy@arm.com \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).