Re: [PATCH 00/25] dmaengine: dw-edma: Add RP/EP local DMA controllers support

[Serge Semin <fancer.lancer@gmail.com>]

Hello Frank,

On Fri, Mar 25, 2022 at 02:52:53PM +0000, Frank Li wrote:
> 
> 
> > -----Original Message-----
> > From: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> > Sent: Wednesday, March 23, 2022 8:48 PM
> > To: Gustavo Pimentel <gustavo.pimentel@synopsys.com>; Vinod Koul
> > <vkoul@kernel.org>; Jingoo Han <jingoohan1@gmail.com>; Bjorn Helgaas
> > <bhelgaas@google.com>; Frank Li <frank.li@nxp.com>; Manivannan Sadhasivam
> > <manivannan.sadhasivam@linaro.org>
> > Cc: Serge Semin <Sergey.Semin@baikalelectronics.ru>; Serge Semin
> > <fancer.lancer@gmail.com>; Alexey Malahov
> > <Alexey.Malahov@baikalelectronics.ru>; Pavel Parkhomenko
> > <Pavel.Parkhomenko@baikalelectronics.ru>; Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com>; Rob Herring <robh@kernel.org>; Krzysztof
> > Wilczyński <kw@linux.com>; linux-pci@vger.kernel.org;
> > dmaengine@vger.kernel.org; linux-kernel@vger.kernel.org
> > Subject: [EXT] [PATCH 00/25] dmaengine: dw-edma: Add RP/EP local DMA
> > controllers support
> > 
> > Caution: EXT Email
> > 
> > This is a final patchset in the series created in the framework of
> > my Baikal-T1 PCIe/eDMA-related work:
> > 
> > [1: In-progress] clk: Baikal-T1 DDR/PCIe resets and some xGMAC fixes
> > Link: --submitted--
> > [2: In-progress] PCI: dwc: Various fixes and cleanups
> > Link: --submitted--
> > [3: In-progress] PCI: dwc: Add dma-ranges/YAML-schema/Baikal-T1 support
> > Link: --submitted--
> > [4: In-progress] dmaengine: dw-edma: Add RP/EP local DMA controllers
> > support
> > Link: --you are looking at it--
> > 
> > Note it is recommended to merge the last patchset after the former ones in
> > order to prevent merge conflicts. @Bjorn could you merge in this patchset
> > through your PCIe repo? After getting all the ack'es of course.
> > 
> > Please note originally this series was self content, but due to Frank
> > being a bit faster in his work submission I had to rebase my patchset onto
> > his one. So now this patchset turns to be dependent on the Frank' work:
> > Link:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern
> > el.org%2Fdmaengine%2F20220310192457.3090-1-
> > Frank.Li%40nxp.com%2F&amp;data=04%7C01%7CFrank.Li%40nxp.com%7Cfc099c7e402d4
> > 556176e08da0d386b19%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6378368332
> > 26917527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> > I6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=3wsUdFo2TpBCVe8haYoFMxPyY78D4yABM
> > B%2Bz0QHkE3Q%3D&amp;reserved=0
> > So please review and merge his series first before applying this one.
> > 
> > @Frank, @Manivannan as we agreed here:
> > Link:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern
> > el.org%2Fdmaengine%2F20220309211204.26050-6-
> > Frank.Li%40nxp.com%2F&amp;data=04%7C01%7CFrank.Li%40nxp.com%7Cfc099c7e402d4
> > 556176e08da0d386b19%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6378368332
> > 26917527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> > I6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=UwV79KHJWWhHZ9dYuqrBP8%2B5n6xonLU
> > bK5wqduuSg%2FM%3D&amp;reserved=0
> > this series contains two patches with our joint work. Here they are:
> > [PATCH 1/25] dmaengine: dw-edma: Drop dma_slave_config.direction field
> > usage
> > [PATCH 2/25] dmaengine: dw-edma: Fix eDMA Rd/Wr-channels and DMA-direction
> > semantics
> > @Frank, could you please pick them up and add them to your series in place
> > of the patches:
> > [PATCH v5 6/9] dmaengine: dw-edma: Don't rely on the deprecated "direction"
> > member
> > Link:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern
> > el.org%2Fdmaengine%2F20220310192457.3090-7-
> > Frank.Li%40nxp.com%2F&amp;data=04%7C01%7CFrank.Li%40nxp.com%7Cfc099c7e402d4
> > 556176e08da0d386b19%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6378368332
> > 26917527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> > I6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=G2Q2BTfp4vt0yiyHbPnn9kKBDfOx6QBmE
> > ypOREcWq4g%3D&amp;reserved=0
> > [PATCH v5 5/9] dmaengine: dw-edma: Fix programming the source & dest
> > addresses for ep
> > Link:
> > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kern
> > el.org%2Fdmaengine%2F20220310192457.3090-6-
> > Frank.Li%40nxp.com%2F&amp;data=04%7C01%7CFrank.Li%40nxp.com%7Cfc099c7e402d4
> > 556176e08da0d386b19%7C686ea1d3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C6378368332
> > 26917527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTi
> > I6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=UppQMqEDobyEN2JU05MS6k%2FuXeS6%2B
> > skrr%2BeN%2FeQzPk8%3D&amp;reserved=0
> > respectively?
> > 
> > @Frank please don't forget to fix your series so the chip->dw field is
> > initialized after all the probe() initializations are done. For that sake
> > you also need to make sure that the dw_edma_irq_request(),
> > dw_edma_channel_setup() and dw_edma_v0_core_debugfs_on() methods take
> > dw_edma structure pointer as a parameter.
> > 
> > Here is a short summary regarding this patchset. The series starts with
> > fixes patches. The very first two patches have been modified based on
> > discussion with @Frank and @Manivannan as I noted in the previous
> > paragraph. They concern fixing the Read/Write channels xfer semantics.
> > See the patches description for more details. After that goes the fix of
> > the dma_direct_map_resource() method, which turned out to be not working
> > correctly for the case of having devive.dma_range_map being non-empty
> > (non-empty dma-ranges DT property). Then we discovered that the
> > dw-edma-pcie.c driver incorrectly initializes the LL/DT base addresses for
> > the platforms with not matching CPU and PCIe memory spaces. It is fixed by
> > using the pci_bus_address() method to get a correct base address. After
> > that you can find a series of interleaved xfers fixes. It turned out the
> > interleaved transfers implementation didn't work quite correctly from the
> > very beginning for instance missing src/dst addresses initialization, etc.
> > In the framework of the next two patches we suggest to add a new
> > platform-specific callback - pci_addrees() and use to convert the CPU
> > address to the PCIe space address. It is at least required for the DW eDAM
> > remote End-point setup on the platforms with not-matching address spaces.
> > In case of the DW eDMA local RP/EP setup the conversion will be done
> > automatically by the outbound iATU (if no DMA-bypass flag is specified for
> > the corresponding iATU window). Then we introduce a set of patches to make
> > the DebugFS part of the code supporting the multi-eDMA controllers
> > platforms. It starts with several cleanup patches and is closed joining
> > the Read/Write channels into a single DMA-device as they originally should
> > have been. After that you can find the patches with adding the non-atomic
> > io-64 methods usage, dropping DT-region descriptors allocation, replacing
> > chip IDs with device name. In addition to that in order to have the eDMA
> > embedded into the DW PCIe RP/EP supported we need to bypass the
> > dma-ranges-based memory ranges mapping since in case of the root port DT
> > node it's applicable for the peripheral PCIe devices only. Finally at the
> > series closure we introduce a generic DW eDMA controller support being
> > available in the DW PCIe Host/End-point driver.
> > 
> > Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> > Cc: Alexey Malahov <Alexey.Malahov@baikalelectronics.ru>
> > Cc: Pavel Parkhomenko <Pavel.Parkhomenko@baikalelectronics.ru>
> > Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> > Cc: Rob Herring <robh@kernel.org>
> > Cc: "Krzysztof Wilczyński" <kw@linux.com>
> > Cc: linux-pci@vger.kernel.org
> > Cc: dmaengine@vger.kernel.org
> > Cc: linux-kernel@vger.kernel.org
> > 
> > Serge Semin (25):
> >   dmaengine: dw-edma: Drop dma_slave_config.direction field usage
> >   dmaengine: dw-edma: Fix eDMA Rd/Wr-channels and DMA-direction
> >     semantics
> >   dma-direct: take dma-ranges/offsets into account in resource mapping
> >   dmaengine: Fix dma_slave_config.dst_addr description
> >   dmaengine: dw-edma: Convert ll/dt phys-address to PCIe bus/DMA address
> >   dmaengine: dw-edma: Fix missing src/dst address of the interleaved
> >     xfers
> >   dmaengine: dw-edma: Don't permit non-inc interleaved xfers
> >   dmaengine: dw-edma: Fix invalid interleaved xfers semantics
> >   dmaengine: dw-edma: Add CPU to PCIe bus address translation
> >   dmaengine: dw-edma: Add PCIe bus address getter to the remote EP
> >     glue-driver
> >   dmaengine: dw-edma: Drop chancnt initialization
> >   dmaengine: dw-edma: Fix DebugFS reg entry type
> >   dmaengine: dw-edma: Stop checking debugfs_create_*() return value
> >   dmaengine: dw-edma: Add dw_edma prefix to the DebugFS nodes descriptor
> >   dmaengine: dw-edma: Convert DebugFS descs to being kz-allocated
> >   dmaengine: dw-edma: Simplify the DebugFS context CSRs init procedure
> >   dmaengine: dw-edma: Move eDMA data pointer to DebugFS node descriptor
> >   dmaengine: dw-edma: Join Write/Read channels into a single device
> >   dmaengine: dw-edma: Use DMA-engine device DebugFS subdirectory
> >   dmaengine: dw-edma: Use non-atomic io-64 methods
> >   dmaengine: dw-edma: Drop DT-region allocation
> >   dmaengine: dw-edma: Replace chip ID number with device name
> >   dmaengine: dw-edma: Bypass dma-ranges mapping for the local setup
> >   dmaengine: dw-edma: Skip cleanup procedure if no private data found
> >   PCI: dwc: Add DW eDMA engine support
> 

> Why I can't see your patch in https://www.spinics.net/lists/linux-pci/
> But there are Manivannan's reply in https://www.spinics.net/lists/linux-pci/

That might be connected with the domain. I'll resend the patchsets
from my personal email after getting all the comments
addressed/discussed. Regarding the patches you are the most interested
in I'll speed the discussion up around them and send them out to you
directly so not to block your series any longer.

-Sergey

> 
> Best regards
> Frank Li
> 
> > 
> >  drivers/dma/dw-edma/dw-edma-core.c            | 249 +++++++------
> >  drivers/dma/dw-edma/dw-edma-core.h            |  10 +-
> >  drivers/dma/dw-edma/dw-edma-pcie.c            |  24 +-
> >  drivers/dma/dw-edma/dw-edma-v0-core.c         |  76 ++--
> >  drivers/dma/dw-edma/dw-edma-v0-core.h         |   1 -
> >  drivers/dma/dw-edma/dw-edma-v0-debugfs.c      | 350 ++++++++----------
> >  drivers/dma/dw-edma/dw-edma-v0-debugfs.h      |   5 -
> >  .../pci/controller/dwc/pcie-designware-ep.c   |   4 +
> >  .../pci/controller/dwc/pcie-designware-host.c |  13 +-
> >  drivers/pci/controller/dwc/pcie-designware.c  | 188 ++++++++++
> >  drivers/pci/controller/dwc/pcie-designware.h  |  23 +-
> >  include/linux/dma/edma.h                      |  18 +-
> >  include/linux/dmaengine.h                     |   2 +-
> >  kernel/dma/direct.c                           |   2 +-
> >  14 files changed, 598 insertions(+), 367 deletions(-)
> > 
> > --
> > 2.35.1
>