[PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

Patchset Summary:
  Enhance a PCIe host controller driver.  Because of its unusual design
  we are foced to change dev->dma_pfn_offset into a more general role
  allowing multiple offsets.  See the 'v1' notes below for more info.

v11:
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- Rebased to latest torvalds, Aug 20, 2020.
  -- Minor change in of_dma_get_range() to satisfy the kernel's
     robot tester.
  -- Use of PFN_DOWN(), PFN_PHYS() instead of explicit shifts (Andy)
  -- Eliminate extra return in dma_offset_from_xxx_addr() (Andy)
  -- Change dma_set_offset_range() to correctly handle the case
     of pre-existing DMA map and zero offset.

v10: 
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- change title of commit; "bus core:" => "device-mapping:"
  -- instead of allocating the DMA map with devm, use kcalloc
     and call kfree() during device_release().  (RobH) Also,
     for three cases that want to use the same DMA map, copy
     the dma_range_map using a helper function.
  -- added a missing 'return = 0;' to of_dma_get_range().  (Nicolas)
  -- removed dma_range_overlaps(); instead return error if there
     is an existing DMA map. (Christoph).
  Commit: "PCI: brcmstb: Set additional internal memory DMA ..."
  -- Changed constant 1 to 1ULL. (Nicolas)
  Commit: "ata: ahci_brcm: Fix use of BCM7216 reset controller"
     This commit has been removed from this patchset and will be
     submitted on its own.

v9:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- A number of code improvements were implemented as suggested by
     ChristophH.  Unfortunately, some of these changes reversed the
     implemented suggestions of other reviewers; for example, the new
     macros PFN_DMA_ADDR(), DMA_ADDR_PFN() have been pulled.

v8:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- To satisfy a specific m68 compile configuration, I moved the 'struct
     bus_dma_region; definition out of #ifdef CONFIG_HAS_DMA and also defined
     three inline functions for !CONFIG_HAS_DMA (kernel test robot).
  -- The sunXi drivers -- suc4i_csi, sun6i_csi, cedrus_hw -- set
     a pfn_offset outside of_dma_configure() but the code offers no 
     insight on the size of the translation window.  V7 had me using
     SIZE_MAX as the size.  I have since contacted the sunXi maintainer and
     he said that using a size of SZ_4G would cover sunXi configurations.

v7:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- remove second kcalloc/copy in device.c (AndyS)
  -- use PTR_ERR_OR_ZERO() and PHYS_PFN() (AndyS)
  -- indentation, sizeof(struct ...) => sizeof(*r) (AndyS)
  -- add pfn.h definitions: PFN_DMA_ADDR(), DMA_ADDR_PFN() (AndyS)
  -- Fixed compile error in "sun6i_csi.c" (kernel test robot)
  Commit "ata: ahci_brcm: Fix use of BCM7216 reset controller"
  -- correct name of function in the commit msg (SergeiS)
  
v6:
  Commit "device core: Introduce DMA range map":
  -- of_dma_get_range() now takes a single argument and returns either
     NULL, a valid map, or an ERR_PTR. (Robin)
  -- offsets are no longer a PFN value but an actual address. (Robin)
  -- the bus_dma_region struct stores the range size instead of
     the cpu_end and pci_end values. (Robin)
  -- devices that were setting a single offset with no boundaries
     have been modified to have boundaries; in a few places
     where this information was unavilable a /* FIXME: ... */
     comment was added. (Robin)
  -- dma_attach_offset_range() can be called when an offset
     map already exists; if it's range is already present
     nothing is done and success is returned. (Robin)
  All commits:
  -- Man name/style/corrections/etc changed (Bjorn)
  -- rebase to Torvalds master

v5:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- in of/address.c: "map_size = 0" => "*map_size = 0"
  -- use kcalloc instead of kzalloc (AndyS)
  -- use PHYS_ADDR_MAX instead of "~(phys_addr_t)0"
  Commit "PCI: brcmstb: Set internal memory viewport sizes"
  -- now gives error on missing dma-ranges property.
  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- removed "Allof:" from brcm,scb-sizes definition (RobH)
  All Commits:
  -- indentation style, use max chars 100 (AndyS)
  -- rebased to torvalds master

v4:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- of_dma_get_range() does not take a dev param but instead
     takes two "out" params: map and map_size.  We do this so
     that the code that parses dma-ranges is separate from
     the code that modifies 'dev'.   (Nicolas)
  -- the separate case of having a single pfn offset has
     been removed and is now processed by going through the
     map array. (Nicolas)
  -- move attach_uniform_dma_pfn_offset() from of/address.c to
     dma/mapping.c so that it does not depend on CONFIG_OF. (Nicolas)
  -- devm_kcalloc => devm_kzalloc (DanC)
  -- add/fix assignment to dev->dma_pfn_offset_map for func
     attach_uniform_dma_pfn_offset() (DanC, Nicolas)
  -- s/struct dma_pfn_offset_region/struct bus_dma_region/ (Nicolas)
  -- s/attach_uniform_dma_pfn_offset/dma_attach_uniform_pfn_offset/
  -- s/attach_dma_pfn_offset_map/dma_attach_pfn_offset_map/
  -- More use of PFN_{PHYS,DOWN,UP}. (AndyS)
  Commit "of: Include a dev param in of_dma_get_range()"
  -- this commit was sqaushed with "device core: Introduce ..."

v3:
  Commit "device core: Introduce multiple dma pfn offsets"
  Commit "arm: dma-mapping: Invoke dma offset func if needed"
  -- The above two commits have been squashed.  More importantly,
     the code has been modified so that the functionality for
     multiple pfn offsets subsumes the use of dev->dma_pfn_offset.
     In fact, dma_pfn_offset is removed and supplanted by
     dma_pfn_offset_map, which is a pointer to an array.  The
     more common case of a uniform offset is now handled as
     a map with a single entry, while cases requiring multiple
     pfn offsets use a map with multiple entries.  Code paths
     that used to do this:

         dev->dma_pfn_offset = mydrivers_pfn_offset;

     have been changed to do this:

         attach_uniform_dma_pfn_offset(dev, pfn_offset);

  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- Add if/then clause for required props: resets, reset-names (RobH)
  -- Change compatible list from const to enum (RobH)
  -- Change list of u32-tuples to u64 (RobH)

  Commit "of: Include a dev param in of_dma_get_range()"
  -- modify of/unittests.c to add NULL param in of_dma_get_range() call.

  Commit "device core: Add ability to handle multiple dma offsets"
  -- align comment in device.h (AndyS).
  -- s/cpu_beg/cpu_start/ and s/dma_beg/dma_start/ in struct
     dma_pfn_offset_region (AndyS).

v2:
Commit: "device core: Add ability to handle multiple dma offsets"
  o Added helper func attach_dma_pfn_offset_map() in address.c (Chistoph)
  o Helpers funcs added to __phys_to_dma() & __dma_to_phys() (Christoph)
  o Added warning when multiple offsets are needed and !DMA_PFN_OFFSET_MAP
  o dev->dma_pfn_map => dev->dma_pfn_offset_map
  o s/frm/from/ for dma_pfn_offset_frm_{phys,dma}_addr() (Christoph)
  o In device.h: s/const void */const struct dma_pfn_offset_region */
  o removed 'unlikely' from unlikely(dev->dma_pfn_offset_map) since
    guarded by CONFIG_DMA_PFN_OFFSET_MAP (Christoph)
  o Since dev->dma_pfn_offset is copied in usb/core/{usb,message}.c, now
    dev->dma_pfn_offset_map is copied as well.
  o Merged two of the DMA commits into one (Christoph).

Commit "arm: dma-mapping: Invoke dma offset func if needed":
  o Use helper functions instead of #if CONFIG_DMA_PFN_OFFSET

Other commits' changes:
  o Removed need for carrying of_id var in priv (Nicolas)
  o Commit message rewordings (Bjorn)
  o Commit log messages filled to 75 chars (Bjorn)
  o devm_reset_control_get_shared())
    => devm_reset_control_get_optional_shared (Philipp)
  o Add call to reset_control_assert() in PCIe remove routines (Philipp)

v1:
This patchset expands the usefulness of the Broadcom Settop Box PCIe
controller by building upon the PCIe driver used currently by the
Raspbery Pi.  Other forms of this patchset were submitted by me years
ago and not accepted; the major sticking point was the code required
for the DMA remapping needed for the PCIe driver to work [1].

There have been many changes to the DMA and OF subsystems since that
time, making a cleaner and less intrusive patchset possible.  This
patchset implements a generalization of "dev->dma_pfn_offset", except
that instead of a single scalar offset it provides for multiple
offsets via a function which depends upon the "dma-ranges" property of
the PCIe host controller.  This is required for proper functionality
of the BrcmSTB PCIe controller and possibly some other devices.

[1] https://lore.kernel.org/linux-arm-kernel/1516058925-46522-5-git-send-email-jim2101024@gmail.com/

Jim Quinlan (11):
  PCI: brcmstb: PCIE_BRCMSTB depends on ARCH_BRCMSTB
  dt-bindings: PCI: Add bindings for more Brcmstb chips
  PCI: brcmstb: Add bcm7278 register info
  PCI: brcmstb: Add suspend and resume pm_ops
  PCI: brcmstb: Add bcm7278 PERST# support
  PCI: brcmstb: Add control of rescal reset
  device-mapping: Introduce DMA range map, supplanting dma_pfn_offset
  PCI: brcmstb: Set additional internal memory DMA viewport sizes
  PCI: brcmstb: Accommodate MSI for older chips
  PCI: brcmstb: Set bus max burst size by chip type
  PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list

 .../bindings/pci/brcm,stb-pcie.yaml           |  56 ++-
 arch/arm/include/asm/dma-mapping.h            |  10 +-
 arch/arm/mach-keystone/keystone.c             |  17 +-
 arch/sh/drivers/pci/pcie-sh7786.c             |   9 +-
 arch/x86/pci/sta2x11-fixup.c                  |   7 +-
 drivers/acpi/arm64/iort.c                     |   5 +-
 drivers/base/core.c                           |   2 +
 drivers/gpu/drm/sun4i/sun4i_backend.c         |   5 +-
 drivers/iommu/io-pgtable-arm.c                |   2 +-
 .../platform/sunxi/sun4i-csi/sun4i_csi.c      |   5 +-
 .../platform/sunxi/sun6i-csi/sun6i_csi.c      |   4 +-
 drivers/of/address.c                          |  72 ++-
 drivers/of/device.c                           |  43 +-
 drivers/of/of_private.h                       |  10 +-
 drivers/of/unittest.c                         |  34 +-
 drivers/pci/controller/Kconfig                |   3 +-
 drivers/pci/controller/pcie-brcmstb.c         | 409 +++++++++++++++---
 drivers/remoteproc/remoteproc_core.c          |   8 +-
 .../staging/media/sunxi/cedrus/cedrus_hw.c    |   7 +-
 drivers/usb/core/message.c                    |   9 +-
 drivers/usb/core/usb.c                        |   7 +-
 include/linux/device.h                        |   4 +-
 include/linux/dma-direct.h                    |   8 +-
 include/linux/dma-mapping.h                   |  36 ++
 kernel/dma/coherent.c                         |  10 +-
 kernel/dma/mapping.c                          |  66 +++
 26 files changed, 668 insertions(+), 180 deletions(-)

-- 
2.17.1

[PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

Patchset Summary:
  Enhance a PCIe host controller driver.  Because of its unusual design
  we are foced to change dev->dma_pfn_offset into a more general role
  allowing multiple offsets.  See the 'v1' notes below for more info.

v11:
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- Rebased to latest torvalds, Aug 20, 2020.
  -- Minor change in of_dma_get_range() to satisfy the kernel's
     robot tester.
  -- Use of PFN_DOWN(), PFN_PHYS() instead of explicit shifts (Andy)
  -- Eliminate extra return in dma_offset_from_xxx_addr() (Andy)
  -- Change dma_set_offset_range() to correctly handle the case
     of pre-existing DMA map and zero offset.

v10: 
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- change title of commit; "bus core:" => "device-mapping:"
  -- instead of allocating the DMA map with devm, use kcalloc
     and call kfree() during device_release().  (RobH) Also,
     for three cases that want to use the same DMA map, copy
     the dma_range_map using a helper function.
  -- added a missing 'return = 0;' to of_dma_get_range().  (Nicolas)
  -- removed dma_range_overlaps(); instead return error if there
     is an existing DMA map. (Christoph).
  Commit: "PCI: brcmstb: Set additional internal memory DMA ..."
  -- Changed constant 1 to 1ULL. (Nicolas)
  Commit: "ata: ahci_brcm: Fix use of BCM7216 reset controller"
     This commit has been removed from this patchset and will be
     submitted on its own.

v9:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- A number of code improvements were implemented as suggested by
     ChristophH.  Unfortunately, some of these changes reversed the
     implemented suggestions of other reviewers; for example, the new
     macros PFN_DMA_ADDR(), DMA_ADDR_PFN() have been pulled.

v8:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- To satisfy a specific m68 compile configuration, I moved the 'struct
     bus_dma_region; definition out of #ifdef CONFIG_HAS_DMA and also defined
     three inline functions for !CONFIG_HAS_DMA (kernel test robot).
  -- The sunXi drivers -- suc4i_csi, sun6i_csi, cedrus_hw -- set
     a pfn_offset outside of_dma_configure() but the code offers no 
     insight on the size of the translation window.  V7 had me using
     SIZE_MAX as the size.  I have since contacted the sunXi maintainer and
     he said that using a size of SZ_4G would cover sunXi configurations.

v7:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- remove second kcalloc/copy in device.c (AndyS)
  -- use PTR_ERR_OR_ZERO() and PHYS_PFN() (AndyS)
  -- indentation, sizeof(struct ...) => sizeof(*r) (AndyS)
  -- add pfn.h definitions: PFN_DMA_ADDR(), DMA_ADDR_PFN() (AndyS)
  -- Fixed compile error in "sun6i_csi.c" (kernel test robot)
  Commit "ata: ahci_brcm: Fix use of BCM7216 reset controller"
  -- correct name of function in the commit msg (SergeiS)
  
v6:
  Commit "device core: Introduce DMA range map":
  -- of_dma_get_range() now takes a single argument and returns either
     NULL, a valid map, or an ERR_PTR. (Robin)
  -- offsets are no longer a PFN value but an actual address. (Robin)
  -- the bus_dma_region struct stores the range size instead of
     the cpu_end and pci_end values. (Robin)
  -- devices that were setting a single offset with no boundaries
     have been modified to have boundaries; in a few places
     where this information was unavilable a /* FIXME: ... */
     comment was added. (Robin)
  -- dma_attach_offset_range() can be called when an offset
     map already exists; if it's range is already present
     nothing is done and success is returned. (Robin)
  All commits:
  -- Man name/style/corrections/etc changed (Bjorn)
  -- rebase to Torvalds master

v5:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- in of/address.c: "map_size = 0" => "*map_size = 0"
  -- use kcalloc instead of kzalloc (AndyS)
  -- use PHYS_ADDR_MAX instead of "~(phys_addr_t)0"
  Commit "PCI: brcmstb: Set internal memory viewport sizes"
  -- now gives error on missing dma-ranges property.
  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- removed "Allof:" from brcm,scb-sizes definition (RobH)
  All Commits:
  -- indentation style, use max chars 100 (AndyS)
  -- rebased to torvalds master

v4:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- of_dma_get_range() does not take a dev param but instead
     takes two "out" params: map and map_size.  We do this so
     that the code that parses dma-ranges is separate from
     the code that modifies 'dev'.   (Nicolas)
  -- the separate case of having a single pfn offset has
     been removed and is now processed by going through the
     map array. (Nicolas)
  -- move attach_uniform_dma_pfn_offset() from of/address.c to
     dma/mapping.c so that it does not depend on CONFIG_OF. (Nicolas)
  -- devm_kcalloc => devm_kzalloc (DanC)
  -- add/fix assignment to dev->dma_pfn_offset_map for func
     attach_uniform_dma_pfn_offset() (DanC, Nicolas)
  -- s/struct dma_pfn_offset_region/struct bus_dma_region/ (Nicolas)
  -- s/attach_uniform_dma_pfn_offset/dma_attach_uniform_pfn_offset/
  -- s/attach_dma_pfn_offset_map/dma_attach_pfn_offset_map/
  -- More use of PFN_{PHYS,DOWN,UP}. (AndyS)
  Commit "of: Include a dev param in of_dma_get_range()"
  -- this commit was sqaushed with "device core: Introduce ..."

v3:
  Commit "device core: Introduce multiple dma pfn offsets"
  Commit "arm: dma-mapping: Invoke dma offset func if needed"
  -- The above two commits have been squashed.  More importantly,
     the code has been modified so that the functionality for
     multiple pfn offsets subsumes the use of dev->dma_pfn_offset.
     In fact, dma_pfn_offset is removed and supplanted by
     dma_pfn_offset_map, which is a pointer to an array.  The
     more common case of a uniform offset is now handled as
     a map with a single entry, while cases requiring multiple
     pfn offsets use a map with multiple entries.  Code paths
     that used to do this:

         dev->dma_pfn_offset = mydrivers_pfn_offset;

     have been changed to do this:

         attach_uniform_dma_pfn_offset(dev, pfn_offset);

  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- Add if/then clause for required props: resets, reset-names (RobH)
  -- Change compatible list from const to enum (RobH)
  -- Change list of u32-tuples to u64 (RobH)

  Commit "of: Include a dev param in of_dma_get_range()"
  -- modify of/unittests.c to add NULL param in of_dma_get_range() call.

  Commit "device core: Add ability to handle multiple dma offsets"
  -- align comment in device.h (AndyS).
  -- s/cpu_beg/cpu_start/ and s/dma_beg/dma_start/ in struct
     dma_pfn_offset_region (AndyS).

v2:
Commit: "device core: Add ability to handle multiple dma offsets"
  o Added helper func attach_dma_pfn_offset_map() in address.c (Chistoph)
  o Helpers funcs added to __phys_to_dma() & __dma_to_phys() (Christoph)
  o Added warning when multiple offsets are needed and !DMA_PFN_OFFSET_MAP
  o dev->dma_pfn_map => dev->dma_pfn_offset_map
  o s/frm/from/ for dma_pfn_offset_frm_{phys,dma}_addr() (Christoph)
  o In device.h: s/const void */const struct dma_pfn_offset_region */
  o removed 'unlikely' from unlikely(dev->dma_pfn_offset_map) since
    guarded by CONFIG_DMA_PFN_OFFSET_MAP (Christoph)
  o Since dev->dma_pfn_offset is copied in usb/core/{usb,message}.c, now
    dev->dma_pfn_offset_map is copied as well.
  o Merged two of the DMA commits into one (Christoph).

Commit "arm: dma-mapping: Invoke dma offset func if needed":
  o Use helper functions instead of #if CONFIG_DMA_PFN_OFFSET

Other commits' changes:
  o Removed need for carrying of_id var in priv (Nicolas)
  o Commit message rewordings (Bjorn)
  o Commit log messages filled to 75 chars (Bjorn)
  o devm_reset_control_get_shared())
    => devm_reset_control_get_optional_shared (Philipp)
  o Add call to reset_control_assert() in PCIe remove routines (Philipp)

v1:
This patchset expands the usefulness of the Broadcom Settop Box PCIe
controller by building upon the PCIe driver used currently by the
Raspbery Pi.  Other forms of this patchset were submitted by me years
ago and not accepted; the major sticking point was the code required
for the DMA remapping needed for the PCIe driver to work [1].

There have been many changes to the DMA and OF subsystems since that
time, making a cleaner and less intrusive patchset possible.  This
patchset implements a generalization of "dev->dma_pfn_offset", except
that instead of a single scalar offset it provides for multiple
offsets via a function which depends upon the "dma-ranges" property of
the PCIe host controller.  This is required for proper functionality
of the BrcmSTB PCIe controller and possibly some other devices.

[1] https://lore.kernel.org/linux-arm-kernel/1516058925-46522-5-git-send-email-jim2101024@gmail.com/

Jim Quinlan (11):
  PCI: brcmstb: PCIE_BRCMSTB depends on ARCH_BRCMSTB
  dt-bindings: PCI: Add bindings for more Brcmstb chips
  PCI: brcmstb: Add bcm7278 register info
  PCI: brcmstb: Add suspend and resume pm_ops
  PCI: brcmstb: Add bcm7278 PERST# support
  PCI: brcmstb: Add control of rescal reset
  device-mapping: Introduce DMA range map, supplanting dma_pfn_offset
  PCI: brcmstb: Set additional internal memory DMA viewport sizes
  PCI: brcmstb: Accommodate MSI for older chips
  PCI: brcmstb: Set bus max burst size by chip type
  PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list

 .../bindings/pci/brcm,stb-pcie.yaml           |  56 ++-
 arch/arm/include/asm/dma-mapping.h            |  10 +-
 arch/arm/mach-keystone/keystone.c             |  17 +-
 arch/sh/drivers/pci/pcie-sh7786.c             |   9 +-
 arch/x86/pci/sta2x11-fixup.c                  |   7 +-
 drivers/acpi/arm64/iort.c                     |   5 +-
 drivers/base/core.c                           |   2 +
 drivers/gpu/drm/sun4i/sun4i_backend.c         |   5 +-
 drivers/iommu/io-pgtable-arm.c                |   2 +-
 .../platform/sunxi/sun4i-csi/sun4i_csi.c      |   5 +-
 .../platform/sunxi/sun6i-csi/sun6i_csi.c      |   4 +-
 drivers/of/address.c                          |  72 ++-
 drivers/of/device.c                           |  43 +-
 drivers/of/of_private.h                       |  10 +-
 drivers/of/unittest.c                         |  34 +-
 drivers/pci/controller/Kconfig                |   3 +-
 drivers/pci/controller/pcie-brcmstb.c         | 409 +++++++++++++++---
 drivers/remoteproc/remoteproc_core.c          |   8 +-
 .../staging/media/sunxi/cedrus/cedrus_hw.c    |   7 +-
 drivers/usb/core/message.c                    |   9 +-
 drivers/usb/core/usb.c                        |   7 +-
 include/linux/device.h                        |   4 +-
 include/linux/dma-direct.h                    |   8 +-
 include/linux/dma-mapping.h                   |  36 ++
 kernel/dma/coherent.c                         |  10 +-
 kernel/dma/mapping.c                          |  66 +++
 26 files changed, 668 insertions(+), 180 deletions(-)

-- 
2.17.1

[PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

Patchset Summary:
  Enhance a PCIe host controller driver.  Because of its unusual design
  we are foced to change dev->dma_pfn_offset into a more general role
  allowing multiple offsets.  See the 'v1' notes below for more info.

v11:
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- Rebased to latest torvalds, Aug 20, 2020.
  -- Minor change in of_dma_get_range() to satisfy the kernel's
     robot tester.
  -- Use of PFN_DOWN(), PFN_PHYS() instead of explicit shifts (Andy)
  -- Eliminate extra return in dma_offset_from_xxx_addr() (Andy)
  -- Change dma_set_offset_range() to correctly handle the case
     of pre-existing DMA map and zero offset.

v10: 
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- change title of commit; "bus core:" => "device-mapping:"
  -- instead of allocating the DMA map with devm, use kcalloc
     and call kfree() during device_release().  (RobH) Also,
     for three cases that want to use the same DMA map, copy
     the dma_range_map using a helper function.
  -- added a missing 'return = 0;' to of_dma_get_range().  (Nicolas)
  -- removed dma_range_overlaps(); instead return error if there
     is an existing DMA map. (Christoph).
  Commit: "PCI: brcmstb: Set additional internal memory DMA ..."
  -- Changed constant 1 to 1ULL. (Nicolas)
  Commit: "ata: ahci_brcm: Fix use of BCM7216 reset controller"
     This commit has been removed from this patchset and will be
     submitted on its own.

v9:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- A number of code improvements were implemented as suggested by
     ChristophH.  Unfortunately, some of these changes reversed the
     implemented suggestions of other reviewers; for example, the new
     macros PFN_DMA_ADDR(), DMA_ADDR_PFN() have been pulled.

v8:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- To satisfy a specific m68 compile configuration, I moved the 'struct
     bus_dma_region; definition out of #ifdef CONFIG_HAS_DMA and also defined
     three inline functions for !CONFIG_HAS_DMA (kernel test robot).
  -- The sunXi drivers -- suc4i_csi, sun6i_csi, cedrus_hw -- set
     a pfn_offset outside of_dma_configure() but the code offers no 
     insight on the size of the translation window.  V7 had me using
     SIZE_MAX as the size.  I have since contacted the sunXi maintainer and
     he said that using a size of SZ_4G would cover sunXi configurations.

v7:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- remove second kcalloc/copy in device.c (AndyS)
  -- use PTR_ERR_OR_ZERO() and PHYS_PFN() (AndyS)
  -- indentation, sizeof(struct ...) => sizeof(*r) (AndyS)
  -- add pfn.h definitions: PFN_DMA_ADDR(), DMA_ADDR_PFN() (AndyS)
  -- Fixed compile error in "sun6i_csi.c" (kernel test robot)
  Commit "ata: ahci_brcm: Fix use of BCM7216 reset controller"
  -- correct name of function in the commit msg (SergeiS)
  
v6:
  Commit "device core: Introduce DMA range map":
  -- of_dma_get_range() now takes a single argument and returns either
     NULL, a valid map, or an ERR_PTR. (Robin)
  -- offsets are no longer a PFN value but an actual address. (Robin)
  -- the bus_dma_region struct stores the range size instead of
     the cpu_end and pci_end values. (Robin)
  -- devices that were setting a single offset with no boundaries
     have been modified to have boundaries; in a few places
     where this information was unavilable a /* FIXME: ... */
     comment was added. (Robin)
  -- dma_attach_offset_range() can be called when an offset
     map already exists; if it's range is already present
     nothing is done and success is returned. (Robin)
  All commits:
  -- Man name/style/corrections/etc changed (Bjorn)
  -- rebase to Torvalds master

v5:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- in of/address.c: "map_size = 0" => "*map_size = 0"
  -- use kcalloc instead of kzalloc (AndyS)
  -- use PHYS_ADDR_MAX instead of "~(phys_addr_t)0"
  Commit "PCI: brcmstb: Set internal memory viewport sizes"
  -- now gives error on missing dma-ranges property.
  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- removed "Allof:" from brcm,scb-sizes definition (RobH)
  All Commits:
  -- indentation style, use max chars 100 (AndyS)
  -- rebased to torvalds master

v4:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- of_dma_get_range() does not take a dev param but instead
     takes two "out" params: map and map_size.  We do this so
     that the code that parses dma-ranges is separate from
     the code that modifies 'dev'.   (Nicolas)
  -- the separate case of having a single pfn offset has
     been removed and is now processed by going through the
     map array. (Nicolas)
  -- move attach_uniform_dma_pfn_offset() from of/address.c to
     dma/mapping.c so that it does not depend on CONFIG_OF. (Nicolas)
  -- devm_kcalloc => devm_kzalloc (DanC)
  -- add/fix assignment to dev->dma_pfn_offset_map for func
     attach_uniform_dma_pfn_offset() (DanC, Nicolas)
  -- s/struct dma_pfn_offset_region/struct bus_dma_region/ (Nicolas)
  -- s/attach_uniform_dma_pfn_offset/dma_attach_uniform_pfn_offset/
  -- s/attach_dma_pfn_offset_map/dma_attach_pfn_offset_map/
  -- More use of PFN_{PHYS,DOWN,UP}. (AndyS)
  Commit "of: Include a dev param in of_dma_get_range()"
  -- this commit was sqaushed with "device core: Introduce ..."

v3:
  Commit "device core: Introduce multiple dma pfn offsets"
  Commit "arm: dma-mapping: Invoke dma offset func if needed"
  -- The above two commits have been squashed.  More importantly,
     the code has been modified so that the functionality for
     multiple pfn offsets subsumes the use of dev->dma_pfn_offset.
     In fact, dma_pfn_offset is removed and supplanted by
     dma_pfn_offset_map, which is a pointer to an array.  The
     more common case of a uniform offset is now handled as
     a map with a single entry, while cases requiring multiple
     pfn offsets use a map with multiple entries.  Code paths
     that used to do this:

         dev->dma_pfn_offset = mydrivers_pfn_offset;

     have been changed to do this:

         attach_uniform_dma_pfn_offset(dev, pfn_offset);

  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- Add if/then clause for required props: resets, reset-names (RobH)
  -- Change compatible list from const to enum (RobH)
  -- Change list of u32-tuples to u64 (RobH)

  Commit "of: Include a dev param in of_dma_get_range()"
  -- modify of/unittests.c to add NULL param in of_dma_get_range() call.

  Commit "device core: Add ability to handle multiple dma offsets"
  -- align comment in device.h (AndyS).
  -- s/cpu_beg/cpu_start/ and s/dma_beg/dma_start/ in struct
     dma_pfn_offset_region (AndyS).

v2:
Commit: "device core: Add ability to handle multiple dma offsets"
  o Added helper func attach_dma_pfn_offset_map() in address.c (Chistoph)
  o Helpers funcs added to __phys_to_dma() & __dma_to_phys() (Christoph)
  o Added warning when multiple offsets are needed and !DMA_PFN_OFFSET_MAP
  o dev->dma_pfn_map => dev->dma_pfn_offset_map
  o s/frm/from/ for dma_pfn_offset_frm_{phys,dma}_addr() (Christoph)
  o In device.h: s/const void */const struct dma_pfn_offset_region */
  o removed 'unlikely' from unlikely(dev->dma_pfn_offset_map) since
    guarded by CONFIG_DMA_PFN_OFFSET_MAP (Christoph)
  o Since dev->dma_pfn_offset is copied in usb/core/{usb,message}.c, now
    dev->dma_pfn_offset_map is copied as well.
  o Merged two of the DMA commits into one (Christoph).

Commit "arm: dma-mapping: Invoke dma offset func if needed":
  o Use helper functions instead of #if CONFIG_DMA_PFN_OFFSET

Other commits' changes:
  o Removed need for carrying of_id var in priv (Nicolas)
  o Commit message rewordings (Bjorn)
  o Commit log messages filled to 75 chars (Bjorn)
  o devm_reset_control_get_shared())
    => devm_reset_control_get_optional_shared (Philipp)
  o Add call to reset_control_assert() in PCIe remove routines (Philipp)

v1:
This patchset expands the usefulness of the Broadcom Settop Box PCIe
controller by building upon the PCIe driver used currently by the
Raspbery Pi.  Other forms of this patchset were submitted by me years
ago and not accepted; the major sticking point was the code required
for the DMA remapping needed for the PCIe driver to work [1].

There have been many changes to the DMA and OF subsystems since that
time, making a cleaner and less intrusive patchset possible.  This
patchset implements a generalization of "dev->dma_pfn_offset", except
that instead of a single scalar offset it provides for multiple
offsets via a function which depends upon the "dma-ranges" property of
the PCIe host controller.  This is required for proper functionality
of the BrcmSTB PCIe controller and possibly some other devices.

[1] https://lore.kernel.org/linux-arm-kernel/1516058925-46522-5-git-send-email-jim2101024@gmail.com/

Jim Quinlan (11):
  PCI: brcmstb: PCIE_BRCMSTB depends on ARCH_BRCMSTB
  dt-bindings: PCI: Add bindings for more Brcmstb chips
  PCI: brcmstb: Add bcm7278 register info
  PCI: brcmstb: Add suspend and resume pm_ops
  PCI: brcmstb: Add bcm7278 PERST# support
  PCI: brcmstb: Add control of rescal reset
  device-mapping: Introduce DMA range map, supplanting dma_pfn_offset
  PCI: brcmstb: Set additional internal memory DMA viewport sizes
  PCI: brcmstb: Accommodate MSI for older chips
  PCI: brcmstb: Set bus max burst size by chip type
  PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list

 .../bindings/pci/brcm,stb-pcie.yaml           |  56 ++-
 arch/arm/include/asm/dma-mapping.h            |  10 +-
 arch/arm/mach-keystone/keystone.c             |  17 +-
 arch/sh/drivers/pci/pcie-sh7786.c             |   9 +-
 arch/x86/pci/sta2x11-fixup.c                  |   7 +-
 drivers/acpi/arm64/iort.c                     |   5 +-
 drivers/base/core.c                           |   2 +
 drivers/gpu/drm/sun4i/sun4i_backend.c         |   5 +-
 drivers/iommu/io-pgtable-arm.c                |   2 +-
 .../platform/sunxi/sun4i-csi/sun4i_csi.c      |   5 +-
 .../platform/sunxi/sun6i-csi/sun6i_csi.c      |   4 +-
 drivers/of/address.c                          |  72 ++-
 drivers/of/device.c                           |  43 +-
 drivers/of/of_private.h                       |  10 +-
 drivers/of/unittest.c                         |  34 +-
 drivers/pci/controller/Kconfig                |   3 +-
 drivers/pci/controller/pcie-brcmstb.c         | 409 +++++++++++++++---
 drivers/remoteproc/remoteproc_core.c          |   8 +-
 .../staging/media/sunxi/cedrus/cedrus_hw.c    |   7 +-
 drivers/usb/core/message.c                    |   9 +-
 drivers/usb/core/usb.c                        |   7 +-
 include/linux/device.h                        |   4 +-
 include/linux/dma-direct.h                    |   8 +-
 include/linux/dma-mapping.h                   |  36 ++
 kernel/dma/coherent.c                         |  10 +-
 kernel/dma/mapping.c                          |  66 +++
 26 files changed, 668 insertions(+), 180 deletions(-)

-- 
2.17.1

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

[PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan via iommu]

Patchset Summary:
  Enhance a PCIe host controller driver.  Because of its unusual design
  we are foced to change dev->dma_pfn_offset into a more general role
  allowing multiple offsets.  See the 'v1' notes below for more info.

v11:
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- Rebased to latest torvalds, Aug 20, 2020.
  -- Minor change in of_dma_get_range() to satisfy the kernel's
     robot tester.
  -- Use of PFN_DOWN(), PFN_PHYS() instead of explicit shifts (Andy)
  -- Eliminate extra return in dma_offset_from_xxx_addr() (Andy)
  -- Change dma_set_offset_range() to correctly handle the case
     of pre-existing DMA map and zero offset.

v10: 
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- change title of commit; "bus core:" => "device-mapping:"
  -- instead of allocating the DMA map with devm, use kcalloc
     and call kfree() during device_release().  (RobH) Also,
     for three cases that want to use the same DMA map, copy
     the dma_range_map using a helper function.
  -- added a missing 'return = 0;' to of_dma_get_range().  (Nicolas)
  -- removed dma_range_overlaps(); instead return error if there
     is an existing DMA map. (Christoph).
  Commit: "PCI: brcmstb: Set additional internal memory DMA ..."
  -- Changed constant 1 to 1ULL. (Nicolas)
  Commit: "ata: ahci_brcm: Fix use of BCM7216 reset controller"
     This commit has been removed from this patchset and will be
     submitted on its own.

v9:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- A number of code improvements were implemented as suggested by
     ChristophH.  Unfortunately, some of these changes reversed the
     implemented suggestions of other reviewers; for example, the new
     macros PFN_DMA_ADDR(), DMA_ADDR_PFN() have been pulled.

v8:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- To satisfy a specific m68 compile configuration, I moved the 'struct
     bus_dma_region; definition out of #ifdef CONFIG_HAS_DMA and also defined
     three inline functions for !CONFIG_HAS_DMA (kernel test robot).
  -- The sunXi drivers -- suc4i_csi, sun6i_csi, cedrus_hw -- set
     a pfn_offset outside of_dma_configure() but the code offers no 
     insight on the size of the translation window.  V7 had me using
     SIZE_MAX as the size.  I have since contacted the sunXi maintainer and
     he said that using a size of SZ_4G would cover sunXi configurations.

v7:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- remove second kcalloc/copy in device.c (AndyS)
  -- use PTR_ERR_OR_ZERO() and PHYS_PFN() (AndyS)
  -- indentation, sizeof(struct ...) => sizeof(*r) (AndyS)
  -- add pfn.h definitions: PFN_DMA_ADDR(), DMA_ADDR_PFN() (AndyS)
  -- Fixed compile error in "sun6i_csi.c" (kernel test robot)
  Commit "ata: ahci_brcm: Fix use of BCM7216 reset controller"
  -- correct name of function in the commit msg (SergeiS)
  
v6:
  Commit "device core: Introduce DMA range map":
  -- of_dma_get_range() now takes a single argument and returns either
     NULL, a valid map, or an ERR_PTR. (Robin)
  -- offsets are no longer a PFN value but an actual address. (Robin)
  -- the bus_dma_region struct stores the range size instead of
     the cpu_end and pci_end values. (Robin)
  -- devices that were setting a single offset with no boundaries
     have been modified to have boundaries; in a few places
     where this information was unavilable a /* FIXME: ... */
     comment was added. (Robin)
  -- dma_attach_offset_range() can be called when an offset
     map already exists; if it's range is already present
     nothing is done and success is returned. (Robin)
  All commits:
  -- Man name/style/corrections/etc changed (Bjorn)
  -- rebase to Torvalds master

v5:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- in of/address.c: "map_size = 0" => "*map_size = 0"
  -- use kcalloc instead of kzalloc (AndyS)
  -- use PHYS_ADDR_MAX instead of "~(phys_addr_t)0"
  Commit "PCI: brcmstb: Set internal memory viewport sizes"
  -- now gives error on missing dma-ranges property.
  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- removed "Allof:" from brcm,scb-sizes definition (RobH)
  All Commits:
  -- indentation style, use max chars 100 (AndyS)
  -- rebased to torvalds master

v4:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- of_dma_get_range() does not take a dev param but instead
     takes two "out" params: map and map_size.  We do this so
     that the code that parses dma-ranges is separate from
     the code that modifies 'dev'.   (Nicolas)
  -- the separate case of having a single pfn offset has
     been removed and is now processed by going through the
     map array. (Nicolas)
  -- move attach_uniform_dma_pfn_offset() from of/address.c to
     dma/mapping.c so that it does not depend on CONFIG_OF. (Nicolas)
  -- devm_kcalloc => devm_kzalloc (DanC)
  -- add/fix assignment to dev->dma_pfn_offset_map for func
     attach_uniform_dma_pfn_offset() (DanC, Nicolas)
  -- s/struct dma_pfn_offset_region/struct bus_dma_region/ (Nicolas)
  -- s/attach_uniform_dma_pfn_offset/dma_attach_uniform_pfn_offset/
  -- s/attach_dma_pfn_offset_map/dma_attach_pfn_offset_map/
  -- More use of PFN_{PHYS,DOWN,UP}. (AndyS)
  Commit "of: Include a dev param in of_dma_get_range()"
  -- this commit was sqaushed with "device core: Introduce ..."

v3:
  Commit "device core: Introduce multiple dma pfn offsets"
  Commit "arm: dma-mapping: Invoke dma offset func if needed"
  -- The above two commits have been squashed.  More importantly,
     the code has been modified so that the functionality for
     multiple pfn offsets subsumes the use of dev->dma_pfn_offset.
     In fact, dma_pfn_offset is removed and supplanted by
     dma_pfn_offset_map, which is a pointer to an array.  The
     more common case of a uniform offset is now handled as
     a map with a single entry, while cases requiring multiple
     pfn offsets use a map with multiple entries.  Code paths
     that used to do this:

         dev->dma_pfn_offset = mydrivers_pfn_offset;

     have been changed to do this:

         attach_uniform_dma_pfn_offset(dev, pfn_offset);

  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- Add if/then clause for required props: resets, reset-names (RobH)
  -- Change compatible list from const to enum (RobH)
  -- Change list of u32-tuples to u64 (RobH)

  Commit "of: Include a dev param in of_dma_get_range()"
  -- modify of/unittests.c to add NULL param in of_dma_get_range() call.

  Commit "device core: Add ability to handle multiple dma offsets"
  -- align comment in device.h (AndyS).
  -- s/cpu_beg/cpu_start/ and s/dma_beg/dma_start/ in struct
     dma_pfn_offset_region (AndyS).

v2:
Commit: "device core: Add ability to handle multiple dma offsets"
  o Added helper func attach_dma_pfn_offset_map() in address.c (Chistoph)
  o Helpers funcs added to __phys_to_dma() & __dma_to_phys() (Christoph)
  o Added warning when multiple offsets are needed and !DMA_PFN_OFFSET_MAP
  o dev->dma_pfn_map => dev->dma_pfn_offset_map
  o s/frm/from/ for dma_pfn_offset_frm_{phys,dma}_addr() (Christoph)
  o In device.h: s/const void */const struct dma_pfn_offset_region */
  o removed 'unlikely' from unlikely(dev->dma_pfn_offset_map) since
    guarded by CONFIG_DMA_PFN_OFFSET_MAP (Christoph)
  o Since dev->dma_pfn_offset is copied in usb/core/{usb,message}.c, now
    dev->dma_pfn_offset_map is copied as well.
  o Merged two of the DMA commits into one (Christoph).

Commit "arm: dma-mapping: Invoke dma offset func if needed":
  o Use helper functions instead of #if CONFIG_DMA_PFN_OFFSET

Other commits' changes:
  o Removed need for carrying of_id var in priv (Nicolas)
  o Commit message rewordings (Bjorn)
  o Commit log messages filled to 75 chars (Bjorn)
  o devm_reset_control_get_shared())
    => devm_reset_control_get_optional_shared (Philipp)
  o Add call to reset_control_assert() in PCIe remove routines (Philipp)

v1:
This patchset expands the usefulness of the Broadcom Settop Box PCIe
controller by building upon the PCIe driver used currently by the
Raspbery Pi.  Other forms of this patchset were submitted by me years
ago and not accepted; the major sticking point was the code required
for the DMA remapping needed for the PCIe driver to work [1].

There have been many changes to the DMA and OF subsystems since that
time, making a cleaner and less intrusive patchset possible.  This
patchset implements a generalization of "dev->dma_pfn_offset", except
that instead of a single scalar offset it provides for multiple
offsets via a function which depends upon the "dma-ranges" property of
the PCIe host controller.  This is required for proper functionality
of the BrcmSTB PCIe controller and possibly some other devices.

[1] https://lore.kernel.org/linux-arm-kernel/1516058925-46522-5-git-send-email-jim2101024@gmail.com/

Jim Quinlan (11):
  PCI: brcmstb: PCIE_BRCMSTB depends on ARCH_BRCMSTB
  dt-bindings: PCI: Add bindings for more Brcmstb chips
  PCI: brcmstb: Add bcm7278 register info
  PCI: brcmstb: Add suspend and resume pm_ops
  PCI: brcmstb: Add bcm7278 PERST# support
  PCI: brcmstb: Add control of rescal reset
  device-mapping: Introduce DMA range map, supplanting dma_pfn_offset
  PCI: brcmstb: Set additional internal memory DMA viewport sizes
  PCI: brcmstb: Accommodate MSI for older chips
  PCI: brcmstb: Set bus max burst size by chip type
  PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list

 .../bindings/pci/brcm,stb-pcie.yaml           |  56 ++-
 arch/arm/include/asm/dma-mapping.h            |  10 +-
 arch/arm/mach-keystone/keystone.c             |  17 +-
 arch/sh/drivers/pci/pcie-sh7786.c             |   9 +-
 arch/x86/pci/sta2x11-fixup.c                  |   7 +-
 drivers/acpi/arm64/iort.c                     |   5 +-
 drivers/base/core.c                           |   2 +
 drivers/gpu/drm/sun4i/sun4i_backend.c         |   5 +-
 drivers/iommu/io-pgtable-arm.c                |   2 +-
 .../platform/sunxi/sun4i-csi/sun4i_csi.c      |   5 +-
 .../platform/sunxi/sun6i-csi/sun6i_csi.c      |   4 +-
 drivers/of/address.c                          |  72 ++-
 drivers/of/device.c                           |  43 +-
 drivers/of/of_private.h                       |  10 +-
 drivers/of/unittest.c                         |  34 +-
 drivers/pci/controller/Kconfig                |   3 +-
 drivers/pci/controller/pcie-brcmstb.c         | 409 +++++++++++++++---
 drivers/remoteproc/remoteproc_core.c          |   8 +-
 .../staging/media/sunxi/cedrus/cedrus_hw.c    |   7 +-
 drivers/usb/core/message.c                    |   9 +-
 drivers/usb/core/usb.c                        |   7 +-
 include/linux/device.h                        |   4 +-
 include/linux/dma-direct.h                    |   8 +-
 include/linux/dma-mapping.h                   |  36 ++
 kernel/dma/coherent.c                         |  10 +-
 kernel/dma/mapping.c                          |  66 +++
 26 files changed, 668 insertions(+), 180 deletions(-)

-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

Patchset Summary:
  Enhance a PCIe host controller driver.  Because of its unusual design
  we are foced to change dev->dma_pfn_offset into a more general role
  allowing multiple offsets.  See the 'v1' notes below for more info.

v11:
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- Rebased to latest torvalds, Aug 20, 2020.
  -- Minor change in of_dma_get_range() to satisfy the kernel's
     robot tester.
  -- Use of PFN_DOWN(), PFN_PHYS() instead of explicit shifts (Andy)
  -- Eliminate extra return in dma_offset_from_xxx_addr() (Andy)
  -- Change dma_set_offset_range() to correctly handle the case
     of pre-existing DMA map and zero offset.

v10: 
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- change title of commit; "bus core:" => "device-mapping:"
  -- instead of allocating the DMA map with devm, use kcalloc
     and call kfree() during device_release().  (RobH) Also,
     for three cases that want to use the same DMA map, copy
     the dma_range_map using a helper function.
  -- added a missing 'return = 0;' to of_dma_get_range().  (Nicolas)
  -- removed dma_range_overlaps(); instead return error if there
     is an existing DMA map. (Christoph).
  Commit: "PCI: brcmstb: Set additional internal memory DMA ..."
  -- Changed constant 1 to 1ULL. (Nicolas)
  Commit: "ata: ahci_brcm: Fix use of BCM7216 reset controller"
     This commit has been removed from this patchset and will be
     submitted on its own.

v9:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- A number of code improvements were implemented as suggested by
     ChristophH.  Unfortunately, some of these changes reversed the
     implemented suggestions of other reviewers; for example, the new
     macros PFN_DMA_ADDR(), DMA_ADDR_PFN() have been pulled.

v8:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- To satisfy a specific m68 compile configuration, I moved the 'struct
     bus_dma_region; definition out of #ifdef CONFIG_HAS_DMA and also defined
     three inline functions for !CONFIG_HAS_DMA (kernel test robot).
  -- The sunXi drivers -- suc4i_csi, sun6i_csi, cedrus_hw -- set
     a pfn_offset outside of_dma_configure() but the code offers no 
     insight on the size of the translation window.  V7 had me using
     SIZE_MAX as the size.  I have since contacted the sunXi maintainer and
     he said that using a size of SZ_4G would cover sunXi configurations.

v7:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- remove second kcalloc/copy in device.c (AndyS)
  -- use PTR_ERR_OR_ZERO() and PHYS_PFN() (AndyS)
  -- indentation, sizeof(struct ...) => sizeof(*r) (AndyS)
  -- add pfn.h definitions: PFN_DMA_ADDR(), DMA_ADDR_PFN() (AndyS)
  -- Fixed compile error in "sun6i_csi.c" (kernel test robot)
  Commit "ata: ahci_brcm: Fix use of BCM7216 reset controller"
  -- correct name of function in the commit msg (SergeiS)
  
v6:
  Commit "device core: Introduce DMA range map":
  -- of_dma_get_range() now takes a single argument and returns either
     NULL, a valid map, or an ERR_PTR. (Robin)
  -- offsets are no longer a PFN value but an actual address. (Robin)
  -- the bus_dma_region struct stores the range size instead of
     the cpu_end and pci_end values. (Robin)
  -- devices that were setting a single offset with no boundaries
     have been modified to have boundaries; in a few places
     where this information was unavilable a /* FIXME: ... */
     comment was added. (Robin)
  -- dma_attach_offset_range() can be called when an offset
     map already exists; if it's range is already present
     nothing is done and success is returned. (Robin)
  All commits:
  -- Man name/style/corrections/etc changed (Bjorn)
  -- rebase to Torvalds master

v5:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- in of/address.c: "map_size = 0" => "*map_size = 0"
  -- use kcalloc instead of kzalloc (AndyS)
  -- use PHYS_ADDR_MAX instead of "~(phys_addr_t)0"
  Commit "PCI: brcmstb: Set internal memory viewport sizes"
  -- now gives error on missing dma-ranges property.
  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- removed "Allof:" from brcm,scb-sizes definition (RobH)
  All Commits:
  -- indentation style, use max chars 100 (AndyS)
  -- rebased to torvalds master

v4:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- of_dma_get_range() does not take a dev param but instead
     takes two "out" params: map and map_size.  We do this so
     that the code that parses dma-ranges is separate from
     the code that modifies 'dev'.   (Nicolas)
  -- the separate case of having a single pfn offset has
     been removed and is now processed by going through the
     map array. (Nicolas)
  -- move attach_uniform_dma_pfn_offset() from of/address.c to
     dma/mapping.c so that it does not depend on CONFIG_OF. (Nicolas)
  -- devm_kcalloc => devm_kzalloc (DanC)
  -- add/fix assignment to dev->dma_pfn_offset_map for func
     attach_uniform_dma_pfn_offset() (DanC, Nicolas)
  -- s/struct dma_pfn_offset_region/struct bus_dma_region/ (Nicolas)
  -- s/attach_uniform_dma_pfn_offset/dma_attach_uniform_pfn_offset/
  -- s/attach_dma_pfn_offset_map/dma_attach_pfn_offset_map/
  -- More use of PFN_{PHYS,DOWN,UP}. (AndyS)
  Commit "of: Include a dev param in of_dma_get_range()"
  -- this commit was sqaushed with "device core: Introduce ..."

v3:
  Commit "device core: Introduce multiple dma pfn offsets"
  Commit "arm: dma-mapping: Invoke dma offset func if needed"
  -- The above two commits have been squashed.  More importantly,
     the code has been modified so that the functionality for
     multiple pfn offsets subsumes the use of dev->dma_pfn_offset.
     In fact, dma_pfn_offset is removed and supplanted by
     dma_pfn_offset_map, which is a pointer to an array.  The
     more common case of a uniform offset is now handled as
     a map with a single entry, while cases requiring multiple
     pfn offsets use a map with multiple entries.  Code paths
     that used to do this:

         dev->dma_pfn_offset = mydrivers_pfn_offset;

     have been changed to do this:

         attach_uniform_dma_pfn_offset(dev, pfn_offset);

  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- Add if/then clause for required props: resets, reset-names (RobH)
  -- Change compatible list from const to enum (RobH)
  -- Change list of u32-tuples to u64 (RobH)

  Commit "of: Include a dev param in of_dma_get_range()"
  -- modify of/unittests.c to add NULL param in of_dma_get_range() call.

  Commit "device core: Add ability to handle multiple dma offsets"
  -- align comment in device.h (AndyS).
  -- s/cpu_beg/cpu_start/ and s/dma_beg/dma_start/ in struct
     dma_pfn_offset_region (AndyS).

v2:
Commit: "device core: Add ability to handle multiple dma offsets"
  o Added helper func attach_dma_pfn_offset_map() in address.c (Chistoph)
  o Helpers funcs added to __phys_to_dma() & __dma_to_phys() (Christoph)
  o Added warning when multiple offsets are needed and !DMA_PFN_OFFSET_MAP
  o dev->dma_pfn_map => dev->dma_pfn_offset_map
  o s/frm/from/ for dma_pfn_offset_frm_{phys,dma}_addr() (Christoph)
  o In device.h: s/const void */const struct dma_pfn_offset_region */
  o removed 'unlikely' from unlikely(dev->dma_pfn_offset_map) since
    guarded by CONFIG_DMA_PFN_OFFSET_MAP (Christoph)
  o Since dev->dma_pfn_offset is copied in usb/core/{usb,message}.c, now
    dev->dma_pfn_offset_map is copied as well.
  o Merged two of the DMA commits into one (Christoph).

Commit "arm: dma-mapping: Invoke dma offset func if needed":
  o Use helper functions instead of #if CONFIG_DMA_PFN_OFFSET

Other commits' changes:
  o Removed need for carrying of_id var in priv (Nicolas)
  o Commit message rewordings (Bjorn)
  o Commit log messages filled to 75 chars (Bjorn)
  o devm_reset_control_get_shared())
    => devm_reset_control_get_optional_shared (Philipp)
  o Add call to reset_control_assert() in PCIe remove routines (Philipp)

v1:
This patchset expands the usefulness of the Broadcom Settop Box PCIe
controller by building upon the PCIe driver used currently by the
Raspbery Pi.  Other forms of this patchset were submitted by me years
ago and not accepted; the major sticking point was the code required
for the DMA remapping needed for the PCIe driver to work [1].

There have been many changes to the DMA and OF subsystems since that
time, making a cleaner and less intrusive patchset possible.  This
patchset implements a generalization of "dev->dma_pfn_offset", except
that instead of a single scalar offset it provides for multiple
offsets via a function which depends upon the "dma-ranges" property of
the PCIe host controller.  This is required for proper functionality
of the BrcmSTB PCIe controller and possibly some other devices.

[1] https://lore.kernel.org/linux-arm-kernel/1516058925-46522-5-git-send-email-jim2101024@gmail.com/

Jim Quinlan (11):
  PCI: brcmstb: PCIE_BRCMSTB depends on ARCH_BRCMSTB
  dt-bindings: PCI: Add bindings for more Brcmstb chips
  PCI: brcmstb: Add bcm7278 register info
  PCI: brcmstb: Add suspend and resume pm_ops
  PCI: brcmstb: Add bcm7278 PERST# support
  PCI: brcmstb: Add control of rescal reset
  device-mapping: Introduce DMA range map, supplanting dma_pfn_offset
  PCI: brcmstb: Set additional internal memory DMA viewport sizes
  PCI: brcmstb: Accommodate MSI for older chips
  PCI: brcmstb: Set bus max burst size by chip type
  PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list

 .../bindings/pci/brcm,stb-pcie.yaml           |  56 ++-
 arch/arm/include/asm/dma-mapping.h            |  10 +-
 arch/arm/mach-keystone/keystone.c             |  17 +-
 arch/sh/drivers/pci/pcie-sh7786.c             |   9 +-
 arch/x86/pci/sta2x11-fixup.c                  |   7 +-
 drivers/acpi/arm64/iort.c                     |   5 +-
 drivers/base/core.c                           |   2 +
 drivers/gpu/drm/sun4i/sun4i_backend.c         |   5 +-
 drivers/iommu/io-pgtable-arm.c                |   2 +-
 .../platform/sunxi/sun4i-csi/sun4i_csi.c      |   5 +-
 .../platform/sunxi/sun6i-csi/sun6i_csi.c      |   4 +-
 drivers/of/address.c                          |  72 ++-
 drivers/of/device.c                           |  43 +-
 drivers/of/of_private.h                       |  10 +-
 drivers/of/unittest.c                         |  34 +-
 drivers/pci/controller/Kconfig                |   3 +-
 drivers/pci/controller/pcie-brcmstb.c         | 409 +++++++++++++++---
 drivers/remoteproc/remoteproc_core.c          |   8 +-
 .../staging/media/sunxi/cedrus/cedrus_hw.c    |   7 +-
 drivers/usb/core/message.c                    |   9 +-
 drivers/usb/core/usb.c                        |   7 +-
 include/linux/device.h                        |   4 +-
 include/linux/dma-direct.h                    |   8 +-
 include/linux/dma-mapping.h                   |  36 ++
 kernel/dma/coherent.c                         |  10 +-
 kernel/dma/mapping.c                          |  66 +++
 26 files changed, 668 insertions(+), 180 deletions(-)

-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

Patchset Summary:
  Enhance a PCIe host controller driver.  Because of its unusual design
  we are foced to change dev->dma_pfn_offset into a more general role
  allowing multiple offsets.  See the 'v1' notes below for more info.

v11:
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- Rebased to latest torvalds, Aug 20, 2020.
  -- Minor change in of_dma_get_range() to satisfy the kernel's
     robot tester.
  -- Use of PFN_DOWN(), PFN_PHYS() instead of explicit shifts (Andy)
  -- Eliminate extra return in dma_offset_from_xxx_addr() (Andy)
  -- Change dma_set_offset_range() to correctly handle the case
     of pre-existing DMA map and zero offset.

v10: 
  Commit: "device-mapping: Introduce DMA range map, supplanting ..."
  -- change title of commit; "bus core:" => "device-mapping:"
  -- instead of allocating the DMA map with devm, use kcalloc
     and call kfree() during device_release().  (RobH) Also,
     for three cases that want to use the same DMA map, copy
     the dma_range_map using a helper function.
  -- added a missing 'return = 0;' to of_dma_get_range().  (Nicolas)
  -- removed dma_range_overlaps(); instead return error if there
     is an existing DMA map. (Christoph).
  Commit: "PCI: brcmstb: Set additional internal memory DMA ..."
  -- Changed constant 1 to 1ULL. (Nicolas)
  Commit: "ata: ahci_brcm: Fix use of BCM7216 reset controller"
     This commit has been removed from this patchset and will be
     submitted on its own.

v9:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- A number of code improvements were implemented as suggested by
     ChristophH.  Unfortunately, some of these changes reversed the
     implemented suggestions of other reviewers; for example, the new
     macros PFN_DMA_ADDR(), DMA_ADDR_PFN() have been pulled.

v8:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- To satisfy a specific m68 compile configuration, I moved the 'struct
     bus_dma_region; definition out of #ifdef CONFIG_HAS_DMA and also defined
     three inline functions for !CONFIG_HAS_DMA (kernel test robot).
  -- The sunXi drivers -- suc4i_csi, sun6i_csi, cedrus_hw -- set
     a pfn_offset outside of_dma_configure() but the code offers no 
     insight on the size of the translation window.  V7 had me using
     SIZE_MAX as the size.  I have since contacted the sunXi maintainer and
     he said that using a size of SZ_4G would cover sunXi configurations.

v7:
  Commit: "device core: Introduce DMA range map, supplanting ..."
  -- remove second kcalloc/copy in device.c (AndyS)
  -- use PTR_ERR_OR_ZERO() and PHYS_PFN() (AndyS)
  -- indentation, sizeof(struct ...) => sizeof(*r) (AndyS)
  -- add pfn.h definitions: PFN_DMA_ADDR(), DMA_ADDR_PFN() (AndyS)
  -- Fixed compile error in "sun6i_csi.c" (kernel test robot)
  Commit "ata: ahci_brcm: Fix use of BCM7216 reset controller"
  -- correct name of function in the commit msg (SergeiS)
  
v6:
  Commit "device core: Introduce DMA range map":
  -- of_dma_get_range() now takes a single argument and returns either
     NULL, a valid map, or an ERR_PTR. (Robin)
  -- offsets are no longer a PFN value but an actual address. (Robin)
  -- the bus_dma_region struct stores the range size instead of
     the cpu_end and pci_end values. (Robin)
  -- devices that were setting a single offset with no boundaries
     have been modified to have boundaries; in a few places
     where this information was unavilable a /* FIXME: ... */
     comment was added. (Robin)
  -- dma_attach_offset_range() can be called when an offset
     map already exists; if it's range is already present
     nothing is done and success is returned. (Robin)
  All commits:
  -- Man name/style/corrections/etc changed (Bjorn)
  -- rebase to Torvalds master

v5:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- in of/address.c: "map_size = 0" => "*map_size = 0"
  -- use kcalloc instead of kzalloc (AndyS)
  -- use PHYS_ADDR_MAX instead of "~(phys_addr_t)0"
  Commit "PCI: brcmstb: Set internal memory viewport sizes"
  -- now gives error on missing dma-ranges property.
  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- removed "Allof:" from brcm,scb-sizes definition (RobH)
  All Commits:
  -- indentation style, use max chars 100 (AndyS)
  -- rebased to torvalds master

v4:
  Commit "device core: Introduce multiple dma pfn offsets"
  -- of_dma_get_range() does not take a dev param but instead
     takes two "out" params: map and map_size.  We do this so
     that the code that parses dma-ranges is separate from
     the code that modifies 'dev'.   (Nicolas)
  -- the separate case of having a single pfn offset has
     been removed and is now processed by going through the
     map array. (Nicolas)
  -- move attach_uniform_dma_pfn_offset() from of/address.c to
     dma/mapping.c so that it does not depend on CONFIG_OF. (Nicolas)
  -- devm_kcalloc => devm_kzalloc (DanC)
  -- add/fix assignment to dev->dma_pfn_offset_map for func
     attach_uniform_dma_pfn_offset() (DanC, Nicolas)
  -- s/struct dma_pfn_offset_region/struct bus_dma_region/ (Nicolas)
  -- s/attach_uniform_dma_pfn_offset/dma_attach_uniform_pfn_offset/
  -- s/attach_dma_pfn_offset_map/dma_attach_pfn_offset_map/
  -- More use of PFN_{PHYS,DOWN,UP}. (AndyS)
  Commit "of: Include a dev param in of_dma_get_range()"
  -- this commit was sqaushed with "device core: Introduce ..."

v3:
  Commit "device core: Introduce multiple dma pfn offsets"
  Commit "arm: dma-mapping: Invoke dma offset func if needed"
  -- The above two commits have been squashed.  More importantly,
     the code has been modified so that the functionality for
     multiple pfn offsets subsumes the use of dev->dma_pfn_offset.
     In fact, dma_pfn_offset is removed and supplanted by
     dma_pfn_offset_map, which is a pointer to an array.  The
     more common case of a uniform offset is now handled as
     a map with a single entry, while cases requiring multiple
     pfn offsets use a map with multiple entries.  Code paths
     that used to do this:

         dev->dma_pfn_offset = mydrivers_pfn_offset;

     have been changed to do this:

         attach_uniform_dma_pfn_offset(dev, pfn_offset);

  Commit "dt-bindings: PCI: Add bindings for more Brcmstb chips"
  -- Add if/then clause for required props: resets, reset-names (RobH)
  -- Change compatible list from const to enum (RobH)
  -- Change list of u32-tuples to u64 (RobH)

  Commit "of: Include a dev param in of_dma_get_range()"
  -- modify of/unittests.c to add NULL param in of_dma_get_range() call.

  Commit "device core: Add ability to handle multiple dma offsets"
  -- align comment in device.h (AndyS).
  -- s/cpu_beg/cpu_start/ and s/dma_beg/dma_start/ in struct
     dma_pfn_offset_region (AndyS).

v2:
Commit: "device core: Add ability to handle multiple dma offsets"
  o Added helper func attach_dma_pfn_offset_map() in address.c (Chistoph)
  o Helpers funcs added to __phys_to_dma() & __dma_to_phys() (Christoph)
  o Added warning when multiple offsets are needed and !DMA_PFN_OFFSET_MAP
  o dev->dma_pfn_map => dev->dma_pfn_offset_map
  o s/frm/from/ for dma_pfn_offset_frm_{phys,dma}_addr() (Christoph)
  o In device.h: s/const void */const struct dma_pfn_offset_region */
  o removed 'unlikely' from unlikely(dev->dma_pfn_offset_map) since
    guarded by CONFIG_DMA_PFN_OFFSET_MAP (Christoph)
  o Since dev->dma_pfn_offset is copied in usb/core/{usb,message}.c, now
    dev->dma_pfn_offset_map is copied as well.
  o Merged two of the DMA commits into one (Christoph).

Commit "arm: dma-mapping: Invoke dma offset func if needed":
  o Use helper functions instead of #if CONFIG_DMA_PFN_OFFSET

Other commits' changes:
  o Removed need for carrying of_id var in priv (Nicolas)
  o Commit message rewordings (Bjorn)
  o Commit log messages filled to 75 chars (Bjorn)
  o devm_reset_control_get_shared())
    => devm_reset_control_get_optional_shared (Philipp)
  o Add call to reset_control_assert() in PCIe remove routines (Philipp)

v1:
This patchset expands the usefulness of the Broadcom Settop Box PCIe
controller by building upon the PCIe driver used currently by the
Raspbery Pi.  Other forms of this patchset were submitted by me years
ago and not accepted; the major sticking point was the code required
for the DMA remapping needed for the PCIe driver to work [1].

There have been many changes to the DMA and OF subsystems since that
time, making a cleaner and less intrusive patchset possible.  This
patchset implements a generalization of "dev->dma_pfn_offset", except
that instead of a single scalar offset it provides for multiple
offsets via a function which depends upon the "dma-ranges" property of
the PCIe host controller.  This is required for proper functionality
of the BrcmSTB PCIe controller and possibly some other devices.

[1] https://lore.kernel.org/linux-arm-kernel/1516058925-46522-5-git-send-email-jim2101024@gmail.com/

Jim Quinlan (11):
  PCI: brcmstb: PCIE_BRCMSTB depends on ARCH_BRCMSTB
  dt-bindings: PCI: Add bindings for more Brcmstb chips
  PCI: brcmstb: Add bcm7278 register info
  PCI: brcmstb: Add suspend and resume pm_ops
  PCI: brcmstb: Add bcm7278 PERST# support
  PCI: brcmstb: Add control of rescal reset
  device-mapping: Introduce DMA range map, supplanting dma_pfn_offset
  PCI: brcmstb: Set additional internal memory DMA viewport sizes
  PCI: brcmstb: Accommodate MSI for older chips
  PCI: brcmstb: Set bus max burst size by chip type
  PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list

 .../bindings/pci/brcm,stb-pcie.yaml           |  56 ++-
 arch/arm/include/asm/dma-mapping.h            |  10 +-
 arch/arm/mach-keystone/keystone.c             |  17 +-
 arch/sh/drivers/pci/pcie-sh7786.c             |   9 +-
 arch/x86/pci/sta2x11-fixup.c                  |   7 +-
 drivers/acpi/arm64/iort.c                     |   5 +-
 drivers/base/core.c                           |   2 +
 drivers/gpu/drm/sun4i/sun4i_backend.c         |   5 +-
 drivers/iommu/io-pgtable-arm.c                |   2 +-
 .../platform/sunxi/sun4i-csi/sun4i_csi.c      |   5 +-
 .../platform/sunxi/sun6i-csi/sun6i_csi.c      |   4 +-
 drivers/of/address.c                          |  72 ++-
 drivers/of/device.c                           |  43 +-
 drivers/of/of_private.h                       |  10 +-
 drivers/of/unittest.c                         |  34 +-
 drivers/pci/controller/Kconfig                |   3 +-
 drivers/pci/controller/pcie-brcmstb.c         | 409 +++++++++++++++---
 drivers/remoteproc/remoteproc_core.c          |   8 +-
 .../staging/media/sunxi/cedrus/cedrus_hw.c    |   7 +-
 drivers/usb/core/message.c                    |   9 +-
 drivers/usb/core/usb.c                        |   7 +-
 include/linux/device.h                        |   4 +-
 include/linux/dma-direct.h                    |   8 +-
 include/linux/dma-mapping.h                   |  36 ++
 kernel/dma/coherent.c                         |  10 +-
 kernel/dma/mapping.c                          |  66 +++
 26 files changed, 668 insertions(+), 180 deletions(-)

-- 
2.17.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v11 01/11] PCI: brcmstb: PCIE_BRCMSTB depends on ARCH_BRCMSTB

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

Have PCIE_BRCMSTB depend on ARCH_BRCMSTB.  Also set the default value to
ARCH_BRCMSTB.

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
Reviewed-by: Rob Herring <robh@kernel.org>
---
 drivers/pci/controller/Kconfig | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/controller/Kconfig b/drivers/pci/controller/Kconfig
index f18c3725ef80..624ab986ecb2 100644
--- a/drivers/pci/controller/Kconfig
+++ b/drivers/pci/controller/Kconfig
@@ -270,9 +270,10 @@ config VMD
 
 config PCIE_BRCMSTB
 	tristate "Broadcom Brcmstb PCIe host controller"
-	depends on ARCH_BCM2835 || COMPILE_TEST
+	depends on ARCH_BRCMSTB || ARCH_BCM2835 || COMPILE_TEST
 	depends on OF
 	depends on PCI_MSI_IRQ_DOMAIN
+	default ARCH_BRCMSTB
 	help
 	  Say Y here to enable PCIe host controller support for
 	  Broadcom STB based SoCs, like the Raspberry Pi 4.
-- 
2.17.1

[PATCH v11 02/11] dt-bindings: PCI: Add bindings for more Brcmstb chips

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

- Add compatible strings for three more Broadcom STB chips: 7278, 7216,
  7211 (STB version of RPi4).
- Add new property 'brcm,scb-sizes'.
- Add new property 'resets'.
- Add new property 'reset-names' for 7216 only.
- Allow 'ranges' and 'dma-ranges' to have more than one item and update
  the example to show this.

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Reviewed-by: Rob Herring <robh@kernel.org>
---
 .../bindings/pci/brcm,stb-pcie.yaml           | 56 ++++++++++++++++---
 1 file changed, 49 insertions(+), 7 deletions(-)

diff --git a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
index 8680a0f86c5a..807694b4f41f 100644
--- a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
+++ b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
@@ -9,12 +9,15 @@ title: Brcmstb PCIe Host Controller Device Tree Bindings
 maintainers:
   - Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
 
-allOf:
-  - $ref: /schemas/pci/pci-bus.yaml#
-
 properties:
   compatible:
-    const: brcm,bcm2711-pcie # The Raspberry Pi 4
+    items:
+      - enum:
+          - brcm,bcm2711-pcie # The Raspberry Pi 4
+          - brcm,bcm7211-pcie # Broadcom STB version of RPi4
+          - brcm,bcm7278-pcie # Broadcom 7278 Arm
+          - brcm,bcm7216-pcie # Broadcom 7216 Arm
+          - brcm,bcm7445-pcie # Broadcom 7445 Arm
 
   reg:
     maxItems: 1
@@ -34,10 +37,12 @@ properties:
       - const: msi
 
   ranges:
-    maxItems: 1
+    minItems: 1
+    maxItems: 4
 
   dma-ranges:
-    maxItems: 1
+    minItems: 1
+    maxItems: 6
 
   clocks:
     maxItems: 1
@@ -58,8 +63,31 @@ properties:
 
   aspm-no-l0s: true
 
+  resets:
+    description: for "brcm,bcm7216-pcie", must be a valid reset
+      phandle pointing to the RESCAL reset controller provider node.
+    $ref: "/schemas/types.yaml#/definitions/phandle"
+
+  reset-names:
+    items:
+      - const: rescal
+
+  brcm,scb-sizes:
+    description: u64 giving the 64bit PCIe memory
+      viewport size of a memory controller.  There may be up to
+      three controllers, and each size must be a power of two
+      with a size greater or equal to the amount of memory the
+      controller supports.  Note that each memory controller
+      may have two component regions -- base and extended -- so
+      this information cannot be deduced from the dma-ranges.
+    $ref: /schemas/types.yaml#/definitions/uint64-array
+    items:
+      minItems: 1
+      maxItems: 3
+
 required:
   - reg
+  - ranges
   - dma-ranges
   - "#interrupt-cells"
   - interrupts
@@ -68,6 +96,18 @@ required:
   - interrupt-map
   - msi-controller
 
+allOf:
+  - $ref: /schemas/pci/pci-bus.yaml#
+  - if:
+      properties:
+        compatible:
+          contains:
+            const: brcm,bcm7216-pcie
+    then:
+      required:
+        - resets
+        - reset-names
+
 unevaluatedProperties: false
 
 examples:
@@ -93,7 +133,9 @@ examples:
                     msi-parent = <&pcie0>;
                     msi-controller;
                     ranges = <0x02000000 0x0 0xf8000000 0x6 0x00000000 0x0 0x04000000>;
-                    dma-ranges = <0x02000000 0x0 0x00000000 0x0 0x00000000 0x0 0x80000000>;
+                    dma-ranges = <0x42000000 0x1 0x00000000 0x0 0x40000000 0x0 0x80000000>,
+                                 <0x42000000 0x1 0x80000000 0x3 0x00000000 0x0 0x80000000>;
                     brcm,enable-ssc;
+                    brcm,scb-sizes =  <0x0000000080000000 0x0000000080000000>;
             };
     };
-- 
2.17.1

[PATCH v11 02/11] dt-bindings: PCI: Add bindings for more Brcmstb chips

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

- Add compatible strings for three more Broadcom STB chips: 7278, 7216,
  7211 (STB version of RPi4).
- Add new property 'brcm,scb-sizes'.
- Add new property 'resets'.
- Add new property 'reset-names' for 7216 only.
- Allow 'ranges' and 'dma-ranges' to have more than one item and update
  the example to show this.

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Reviewed-by: Rob Herring <robh@kernel.org>
---
 .../bindings/pci/brcm,stb-pcie.yaml           | 56 ++++++++++++++++---
 1 file changed, 49 insertions(+), 7 deletions(-)

diff --git a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
index 8680a0f86c5a..807694b4f41f 100644
--- a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
+++ b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
@@ -9,12 +9,15 @@ title: Brcmstb PCIe Host Controller Device Tree Bindings
 maintainers:
   - Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
 
-allOf:
-  - $ref: /schemas/pci/pci-bus.yaml#
-
 properties:
   compatible:
-    const: brcm,bcm2711-pcie # The Raspberry Pi 4
+    items:
+      - enum:
+          - brcm,bcm2711-pcie # The Raspberry Pi 4
+          - brcm,bcm7211-pcie # Broadcom STB version of RPi4
+          - brcm,bcm7278-pcie # Broadcom 7278 Arm
+          - brcm,bcm7216-pcie # Broadcom 7216 Arm
+          - brcm,bcm7445-pcie # Broadcom 7445 Arm
 
   reg:
     maxItems: 1
@@ -34,10 +37,12 @@ properties:
       - const: msi
 
   ranges:
-    maxItems: 1
+    minItems: 1
+    maxItems: 4
 
   dma-ranges:
-    maxItems: 1
+    minItems: 1
+    maxItems: 6
 
   clocks:
     maxItems: 1
@@ -58,8 +63,31 @@ properties:
 
   aspm-no-l0s: true
 
+  resets:
+    description: for "brcm,bcm7216-pcie", must be a valid reset
+      phandle pointing to the RESCAL reset controller provider node.
+    $ref: "/schemas/types.yaml#/definitions/phandle"
+
+  reset-names:
+    items:
+      - const: rescal
+
+  brcm,scb-sizes:
+    description: u64 giving the 64bit PCIe memory
+      viewport size of a memory controller.  There may be up to
+      three controllers, and each size must be a power of two
+      with a size greater or equal to the amount of memory the
+      controller supports.  Note that each memory controller
+      may have two component regions -- base and extended -- so
+      this information cannot be deduced from the dma-ranges.
+    $ref: /schemas/types.yaml#/definitions/uint64-array
+    items:
+      minItems: 1
+      maxItems: 3
+
 required:
   - reg
+  - ranges
   - dma-ranges
   - "#interrupt-cells"
   - interrupts
@@ -68,6 +96,18 @@ required:
   - interrupt-map
   - msi-controller
 
+allOf:
+  - $ref: /schemas/pci/pci-bus.yaml#
+  - if:
+      properties:
+        compatible:
+          contains:
+            const: brcm,bcm7216-pcie
+    then:
+      required:
+        - resets
+        - reset-names
+
 unevaluatedProperties: false
 
 examples:
@@ -93,7 +133,9 @@ examples:
                     msi-parent = <&pcie0>;
                     msi-controller;
                     ranges = <0x02000000 0x0 0xf8000000 0x6 0x00000000 0x0 0x04000000>;
-                    dma-ranges = <0x02000000 0x0 0x00000000 0x0 0x00000000 0x0 0x80000000>;
+                    dma-ranges = <0x42000000 0x1 0x00000000 0x0 0x40000000 0x0 0x80000000>,
+                                 <0x42000000 0x1 0x80000000 0x3 0x00000000 0x0 0x80000000>;
                     brcm,enable-ssc;
+                    brcm,scb-sizes =  <0x0000000080000000 0x0000000080000000>;
             };
     };
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v11 03/11] PCI: brcmstb: Add bcm7278 register info

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

Add in compatibility strings and code for three Broadcom STB chips.  Some
of the register locations, shifts, and masks are different for certain
chips, requiring the use of different constants based on of_id.

We would like to add the following at this time to the match list but we
need to wait until the end of this patchset so that everything works.

    { .compatible = "brcm,bcm7211-pcie", .data = &generic_cfg },
    { .compatible = "brcm,bcm7278-pcie", .data = &bcm7278_cfg },
    { .compatible = "brcm,bcm7216-pcie", .data = &bcm7278_cfg },
    { .compatible = "brcm,bcm7445-pcie", .data = &generic_cfg },

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 105 +++++++++++++++++++++++---
 1 file changed, 93 insertions(+), 12 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 85fa7d54f11f..c2b3d2946a36 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -122,9 +122,8 @@
 #define  PCIE_EXT_SLOT_SHIFT				15
 #define  PCIE_EXT_FUNC_SHIFT				12
 
-#define PCIE_RGR1_SW_INIT_1				0x9210
 #define  PCIE_RGR1_SW_INIT_1_PERST_MASK			0x1
-#define  PCIE_RGR1_SW_INIT_1_INIT_MASK			0x2
+#define  PCIE_RGR1_SW_INIT_1_PERST_SHIFT		0x0
 
 /* PCIe parameters */
 #define BRCM_NUM_PCIE_OUT_WINS		0x4
@@ -154,6 +153,73 @@
 #define SSC_STATUS_SSC_MASK		0x400
 #define SSC_STATUS_PLL_LOCK_MASK	0x800
 
+#define IDX_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_INDEX])
+#define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
+#define PCIE_RGR1_SW_INIT_1(pcie)	(pcie->reg_offsets[RGR1_SW_INIT_1])
+
+enum {
+	RGR1_SW_INIT_1,
+	EXT_CFG_INDEX,
+	EXT_CFG_DATA,
+};
+
+enum {
+	RGR1_SW_INIT_1_INIT_MASK,
+	RGR1_SW_INIT_1_INIT_SHIFT,
+};
+
+enum pcie_type {
+	GENERIC,
+	BCM7278,
+	BCM2711,
+};
+
+struct pcie_cfg_data {
+	const int *reg_field_info;
+	const int *offsets;
+	const enum pcie_type type;
+};
+
+static const int pcie_reg_field_info[] = {
+	[RGR1_SW_INIT_1_INIT_MASK] = 0x2,
+	[RGR1_SW_INIT_1_INIT_SHIFT] = 0x1,
+};
+
+static const int pcie_reg_field_info_bcm7278[] = {
+	[RGR1_SW_INIT_1_INIT_MASK] = 0x1,
+	[RGR1_SW_INIT_1_INIT_SHIFT] = 0x0,
+};
+
+static const int pcie_offsets[] = {
+	[RGR1_SW_INIT_1] = 0x9210,
+	[EXT_CFG_INDEX]  = 0x9000,
+	[EXT_CFG_DATA]   = 0x9004,
+};
+
+static const struct pcie_cfg_data generic_cfg = {
+	.reg_field_info	= pcie_reg_field_info,
+	.offsets	= pcie_offsets,
+	.type		= GENERIC,
+};
+
+static const int pcie_offset_bcm7278[] = {
+	[RGR1_SW_INIT_1] = 0xc010,
+	[EXT_CFG_INDEX] = 0x9000,
+	[EXT_CFG_DATA] = 0x9004,
+};
+
+static const struct pcie_cfg_data bcm7278_cfg = {
+	.reg_field_info = pcie_reg_field_info_bcm7278,
+	.offsets	= pcie_offset_bcm7278,
+	.type		= BCM7278,
+};
+
+static const struct pcie_cfg_data bcm2711_cfg = {
+	.reg_field_info	= pcie_reg_field_info,
+	.offsets	= pcie_offsets,
+	.type		= BCM2711,
+};
+
 struct brcm_msi {
 	struct device		*dev;
 	void __iomem		*base;
@@ -177,6 +243,9 @@ struct brcm_pcie {
 	int			gen;
 	u64			msi_target_addr;
 	struct brcm_msi		*msi;
+	const int		*reg_offsets;
+	const int		*reg_field_info;
+	enum pcie_type		type;
 };
 
 /*
@@ -603,20 +672,21 @@ static struct pci_ops brcm_pcie_ops = {
 
 static inline void brcm_pcie_bridge_sw_init_set(struct brcm_pcie *pcie, u32 val)
 {
-	u32 tmp;
+	u32 tmp, mask =  pcie->reg_field_info[RGR1_SW_INIT_1_INIT_MASK];
+	u32 shift = pcie->reg_field_info[RGR1_SW_INIT_1_INIT_SHIFT];
 
-	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1);
-	u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_INIT_MASK);
-	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1);
+	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
+	tmp = (tmp & ~mask) | ((val << shift) & mask);
+	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
 }
 
 static inline void brcm_pcie_perst_set(struct brcm_pcie *pcie, u32 val)
 {
 	u32 tmp;
 
-	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1);
+	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
 	u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_PERST_MASK);
-	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1);
+	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
 }
 
 static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
@@ -927,11 +997,17 @@ static int brcm_pcie_remove(struct platform_device *pdev)
 	return 0;
 }
 
+static const struct of_device_id brcm_pcie_match[] = {
+	{ .compatible = "brcm,bcm2711-pcie", .data = &bcm2711_cfg },
+	{},
+};
+
 static int brcm_pcie_probe(struct platform_device *pdev)
 {
 	struct device_node *np = pdev->dev.of_node, *msi_np;
 	struct pci_host_bridge *bridge;
 	struct device_node *fw_np;
+	const struct pcie_cfg_data *data;
 	struct brcm_pcie *pcie;
 	int ret;
 
@@ -953,9 +1029,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
 	if (!bridge)
 		return -ENOMEM;
 
+	data = of_device_get_match_data(&pdev->dev);
+	if (!data) {
+		pr_err("failed to look up compatible string\n");
+		return -EINVAL;
+	}
+
 	pcie = pci_host_bridge_priv(bridge);
 	pcie->dev = &pdev->dev;
 	pcie->np = np;
+	pcie->reg_offsets = data->offsets;
+	pcie->reg_field_info = data->reg_field_info;
+	pcie->type = data->type;
 
 	pcie->base = devm_platform_ioremap_resource(pdev, 0);
 	if (IS_ERR(pcie->base))
@@ -1000,10 +1085,6 @@ static int brcm_pcie_probe(struct platform_device *pdev)
 	return ret;
 }
 
-static const struct of_device_id brcm_pcie_match[] = {
-	{ .compatible = "brcm,bcm2711-pcie" },
-	{},
-};
 MODULE_DEVICE_TABLE(of, brcm_pcie_match);
 
 static struct platform_driver brcm_pcie_driver = {
-- 
2.17.1

[PATCH v11 03/11] PCI: brcmstb: Add bcm7278 register info

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

Add in compatibility strings and code for three Broadcom STB chips.  Some
of the register locations, shifts, and masks are different for certain
chips, requiring the use of different constants based on of_id.

We would like to add the following at this time to the match list but we
need to wait until the end of this patchset so that everything works.

    { .compatible = "brcm,bcm7211-pcie", .data = &generic_cfg },
    { .compatible = "brcm,bcm7278-pcie", .data = &bcm7278_cfg },
    { .compatible = "brcm,bcm7216-pcie", .data = &bcm7278_cfg },
    { .compatible = "brcm,bcm7445-pcie", .data = &generic_cfg },

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 105 +++++++++++++++++++++++---
 1 file changed, 93 insertions(+), 12 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 85fa7d54f11f..c2b3d2946a36 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -122,9 +122,8 @@
 #define  PCIE_EXT_SLOT_SHIFT				15
 #define  PCIE_EXT_FUNC_SHIFT				12
 
-#define PCIE_RGR1_SW_INIT_1				0x9210
 #define  PCIE_RGR1_SW_INIT_1_PERST_MASK			0x1
-#define  PCIE_RGR1_SW_INIT_1_INIT_MASK			0x2
+#define  PCIE_RGR1_SW_INIT_1_PERST_SHIFT		0x0
 
 /* PCIe parameters */
 #define BRCM_NUM_PCIE_OUT_WINS		0x4
@@ -154,6 +153,73 @@
 #define SSC_STATUS_SSC_MASK		0x400
 #define SSC_STATUS_PLL_LOCK_MASK	0x800
 
+#define IDX_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_INDEX])
+#define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
+#define PCIE_RGR1_SW_INIT_1(pcie)	(pcie->reg_offsets[RGR1_SW_INIT_1])
+
+enum {
+	RGR1_SW_INIT_1,
+	EXT_CFG_INDEX,
+	EXT_CFG_DATA,
+};
+
+enum {
+	RGR1_SW_INIT_1_INIT_MASK,
+	RGR1_SW_INIT_1_INIT_SHIFT,
+};
+
+enum pcie_type {
+	GENERIC,
+	BCM7278,
+	BCM2711,
+};
+
+struct pcie_cfg_data {
+	const int *reg_field_info;
+	const int *offsets;
+	const enum pcie_type type;
+};
+
+static const int pcie_reg_field_info[] = {
+	[RGR1_SW_INIT_1_INIT_MASK] = 0x2,
+	[RGR1_SW_INIT_1_INIT_SHIFT] = 0x1,
+};
+
+static const int pcie_reg_field_info_bcm7278[] = {
+	[RGR1_SW_INIT_1_INIT_MASK] = 0x1,
+	[RGR1_SW_INIT_1_INIT_SHIFT] = 0x0,
+};
+
+static const int pcie_offsets[] = {
+	[RGR1_SW_INIT_1] = 0x9210,
+	[EXT_CFG_INDEX]  = 0x9000,
+	[EXT_CFG_DATA]   = 0x9004,
+};
+
+static const struct pcie_cfg_data generic_cfg = {
+	.reg_field_info	= pcie_reg_field_info,
+	.offsets	= pcie_offsets,
+	.type		= GENERIC,
+};
+
+static const int pcie_offset_bcm7278[] = {
+	[RGR1_SW_INIT_1] = 0xc010,
+	[EXT_CFG_INDEX] = 0x9000,
+	[EXT_CFG_DATA] = 0x9004,
+};
+
+static const struct pcie_cfg_data bcm7278_cfg = {
+	.reg_field_info = pcie_reg_field_info_bcm7278,
+	.offsets	= pcie_offset_bcm7278,
+	.type		= BCM7278,
+};
+
+static const struct pcie_cfg_data bcm2711_cfg = {
+	.reg_field_info	= pcie_reg_field_info,
+	.offsets	= pcie_offsets,
+	.type		= BCM2711,
+};
+
 struct brcm_msi {
 	struct device		*dev;
 	void __iomem		*base;
@@ -177,6 +243,9 @@ struct brcm_pcie {
 	int			gen;
 	u64			msi_target_addr;
 	struct brcm_msi		*msi;
+	const int		*reg_offsets;
+	const int		*reg_field_info;
+	enum pcie_type		type;
 };
 
 /*
@@ -603,20 +672,21 @@ static struct pci_ops brcm_pcie_ops = {
 
 static inline void brcm_pcie_bridge_sw_init_set(struct brcm_pcie *pcie, u32 val)
 {
-	u32 tmp;
+	u32 tmp, mask =  pcie->reg_field_info[RGR1_SW_INIT_1_INIT_MASK];
+	u32 shift = pcie->reg_field_info[RGR1_SW_INIT_1_INIT_SHIFT];
 
-	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1);
-	u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_INIT_MASK);
-	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1);
+	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
+	tmp = (tmp & ~mask) | ((val << shift) & mask);
+	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
 }
 
 static inline void brcm_pcie_perst_set(struct brcm_pcie *pcie, u32 val)
 {
 	u32 tmp;
 
-	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1);
+	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
 	u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_PERST_MASK);
-	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1);
+	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
 }
 
 static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
@@ -927,11 +997,17 @@ static int brcm_pcie_remove(struct platform_device *pdev)
 	return 0;
 }
 
+static const struct of_device_id brcm_pcie_match[] = {
+	{ .compatible = "brcm,bcm2711-pcie", .data = &bcm2711_cfg },
+	{},
+};
+
 static int brcm_pcie_probe(struct platform_device *pdev)
 {
 	struct device_node *np = pdev->dev.of_node, *msi_np;
 	struct pci_host_bridge *bridge;
 	struct device_node *fw_np;
+	const struct pcie_cfg_data *data;
 	struct brcm_pcie *pcie;
 	int ret;
 
@@ -953,9 +1029,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
 	if (!bridge)
 		return -ENOMEM;
 
+	data = of_device_get_match_data(&pdev->dev);
+	if (!data) {
+		pr_err("failed to look up compatible string\n");
+		return -EINVAL;
+	}
+
 	pcie = pci_host_bridge_priv(bridge);
 	pcie->dev = &pdev->dev;
 	pcie->np = np;
+	pcie->reg_offsets = data->offsets;
+	pcie->reg_field_info = data->reg_field_info;
+	pcie->type = data->type;
 
 	pcie->base = devm_platform_ioremap_resource(pdev, 0);
 	if (IS_ERR(pcie->base))
@@ -1000,10 +1085,6 @@ static int brcm_pcie_probe(struct platform_device *pdev)
 	return ret;
 }
 
-static const struct of_device_id brcm_pcie_match[] = {
-	{ .compatible = "brcm,bcm2711-pcie" },
-	{},
-};
 MODULE_DEVICE_TABLE(of, brcm_pcie_match);
 
 static struct platform_driver brcm_pcie_driver = {
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
and resume.  Now the PCIe driver may do so as well.

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index c2b3d2946a36..3d588ab7a6dd 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
 	brcm_pcie_bridge_sw_init_set(pcie, 1);
 }
 
+static int brcm_pcie_suspend(struct device *dev)
+{
+	struct brcm_pcie *pcie = dev_get_drvdata(dev);
+
+	brcm_pcie_turn_off(pcie);
+	clk_disable_unprepare(pcie->clk);
+
+	return 0;
+}
+
+static int brcm_pcie_resume(struct device *dev)
+{
+	struct brcm_pcie *pcie = dev_get_drvdata(dev);
+	void __iomem *base;
+	u32 tmp;
+	int ret;
+
+	base = pcie->base;
+	clk_prepare_enable(pcie->clk);
+
+	/* Take bridge out of reset so we can access the SERDES reg */
+	brcm_pcie_bridge_sw_init_set(pcie, 0);
+
+	/* SERDES_IDDQ = 0 */
+	tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
+	u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
+	writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
+
+	/* wait for serdes to be stable */
+	udelay(100);
+
+	ret = brcm_pcie_setup(pcie);
+	if (ret)
+		return ret;
+
+	if (pcie->msi)
+		brcm_msi_set_regs(pcie->msi);
+
+	return 0;
+}
+
 static void __brcm_pcie_remove(struct brcm_pcie *pcie)
 {
 	brcm_msi_remove(pcie);
@@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
 
 MODULE_DEVICE_TABLE(of, brcm_pcie_match);
 
+static const struct dev_pm_ops brcm_pcie_pm_ops = {
+	.suspend_noirq = brcm_pcie_suspend,
+	.resume_noirq = brcm_pcie_resume,
+};
+
 static struct platform_driver brcm_pcie_driver = {
 	.probe = brcm_pcie_probe,
 	.remove = brcm_pcie_remove,
 	.driver = {
 		.name = "brcm-pcie",
 		.of_match_table = brcm_pcie_match,
+		.pm = &brcm_pcie_pm_ops,
 	},
 };
 module_platform_driver(brcm_pcie_driver);
-- 
2.17.1

[PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
and resume.  Now the PCIe driver may do so as well.

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
 1 file changed, 47 insertions(+)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index c2b3d2946a36..3d588ab7a6dd 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
 	brcm_pcie_bridge_sw_init_set(pcie, 1);
 }
 
+static int brcm_pcie_suspend(struct device *dev)
+{
+	struct brcm_pcie *pcie = dev_get_drvdata(dev);
+
+	brcm_pcie_turn_off(pcie);
+	clk_disable_unprepare(pcie->clk);
+
+	return 0;
+}
+
+static int brcm_pcie_resume(struct device *dev)
+{
+	struct brcm_pcie *pcie = dev_get_drvdata(dev);
+	void __iomem *base;
+	u32 tmp;
+	int ret;
+
+	base = pcie->base;
+	clk_prepare_enable(pcie->clk);
+
+	/* Take bridge out of reset so we can access the SERDES reg */
+	brcm_pcie_bridge_sw_init_set(pcie, 0);
+
+	/* SERDES_IDDQ = 0 */
+	tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
+	u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
+	writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
+
+	/* wait for serdes to be stable */
+	udelay(100);
+
+	ret = brcm_pcie_setup(pcie);
+	if (ret)
+		return ret;
+
+	if (pcie->msi)
+		brcm_msi_set_regs(pcie->msi);
+
+	return 0;
+}
+
 static void __brcm_pcie_remove(struct brcm_pcie *pcie)
 {
 	brcm_msi_remove(pcie);
@@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
 
 MODULE_DEVICE_TABLE(of, brcm_pcie_match);
 
+static const struct dev_pm_ops brcm_pcie_pm_ops = {
+	.suspend_noirq = brcm_pcie_suspend,
+	.resume_noirq = brcm_pcie_resume,
+};
+
 static struct platform_driver brcm_pcie_driver = {
 	.probe = brcm_pcie_probe,
 	.remove = brcm_pcie_remove,
 	.driver = {
 		.name = "brcm-pcie",
 		.of_match_table = brcm_pcie_match,
+		.pm = &brcm_pcie_pm_ops,
 	},
 };
 module_platform_driver(brcm_pcie_driver);
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v11 05/11] PCI: brcmstb: Add bcm7278 PERST# support

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

The PERST# bit was moved to a different register in 7278-type STB chips.
In addition, the polarity of the bit was also changed; for other chips
writing a 1 specified assert; for 7278-type chips, writing a 0 specifies
assert.

Of course, PERST# is a PCIe asserted-low signal.

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 3d588ab7a6dd..acf2239b0251 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -83,6 +83,7 @@
 
 #define PCIE_MISC_PCIE_CTRL				0x4064
 #define  PCIE_MISC_PCIE_CTRL_PCIE_L23_REQUEST_MASK	0x1
+#define PCIE_MISC_PCIE_CTRL_PCIE_PERSTB_MASK		0x4
 
 #define PCIE_MISC_PCIE_STATUS				0x4068
 #define  PCIE_MISC_PCIE_STATUS_PCIE_PORT_MASK		0x80
@@ -684,9 +685,16 @@ static inline void brcm_pcie_perst_set(struct brcm_pcie *pcie, u32 val)
 {
 	u32 tmp;
 
-	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
-	u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_PERST_MASK);
-	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
+	if (pcie->type == BCM7278) {
+		/* Perst bit has moved and assert value is 0 */
+		tmp = readl(pcie->base + PCIE_MISC_PCIE_CTRL);
+		u32p_replace_bits(&tmp, !val, PCIE_MISC_PCIE_CTRL_PCIE_PERSTB_MASK);
+		writel(tmp, pcie->base +  PCIE_MISC_PCIE_CTRL);
+	} else {
+		tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
+		u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_PERST_MASK);
+		writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
+	}
 }
 
 static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
@@ -771,7 +779,10 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 
 	/* Reset the bridge */
 	brcm_pcie_bridge_sw_init_set(pcie, 1);
-	brcm_pcie_perst_set(pcie, 1);
+
+	/* BCM7278 fails when PERST# is set here */
+	if (pcie->type != BCM7278)
+		brcm_pcie_perst_set(pcie, 1);
 
 	usleep_range(100, 200);
 
-- 
2.17.1

[PATCH v11 05/11] PCI: brcmstb: Add bcm7278 PERST# support

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

The PERST# bit was moved to a different register in 7278-type STB chips.
In addition, the polarity of the bit was also changed; for other chips
writing a 1 specified assert; for 7278-type chips, writing a 0 specifies
assert.

Of course, PERST# is a PCIe asserted-low signal.

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 3d588ab7a6dd..acf2239b0251 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -83,6 +83,7 @@
 
 #define PCIE_MISC_PCIE_CTRL				0x4064
 #define  PCIE_MISC_PCIE_CTRL_PCIE_L23_REQUEST_MASK	0x1
+#define PCIE_MISC_PCIE_CTRL_PCIE_PERSTB_MASK		0x4
 
 #define PCIE_MISC_PCIE_STATUS				0x4068
 #define  PCIE_MISC_PCIE_STATUS_PCIE_PORT_MASK		0x80
@@ -684,9 +685,16 @@ static inline void brcm_pcie_perst_set(struct brcm_pcie *pcie, u32 val)
 {
 	u32 tmp;
 
-	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
-	u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_PERST_MASK);
-	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
+	if (pcie->type == BCM7278) {
+		/* Perst bit has moved and assert value is 0 */
+		tmp = readl(pcie->base + PCIE_MISC_PCIE_CTRL);
+		u32p_replace_bits(&tmp, !val, PCIE_MISC_PCIE_CTRL_PCIE_PERSTB_MASK);
+		writel(tmp, pcie->base +  PCIE_MISC_PCIE_CTRL);
+	} else {
+		tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
+		u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_PERST_MASK);
+		writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
+	}
 }
 
 static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
@@ -771,7 +779,10 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 
 	/* Reset the bridge */
 	brcm_pcie_bridge_sw_init_set(pcie, 1);
-	brcm_pcie_perst_set(pcie, 1);
+
+	/* BCM7278 fails when PERST# is set here */
+	if (pcie->type != BCM7278)
+		brcm_pcie_perst_set(pcie, 1);
 
 	usleep_range(100, 200);
 
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v11 06/11] PCI: brcmstb: Add control of rescal reset

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

Some STB chips have a special purpose reset controller named RESCAL (reset
calibration).  The PCIe HW can now control RESCAL to start and stop its
operation.  On probe(), the RESCAL is deasserted and the driver goes
through the sequence of setting registers and reading status in order to
start the internal PHY that is required for the PCIe.

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 82 ++++++++++++++++++++++++++-
 1 file changed, 81 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index acf2239b0251..041b8d109563 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -23,6 +23,7 @@
 #include <linux/of_platform.h>
 #include <linux/pci.h>
 #include <linux/printk.h>
+#include <linux/reset.h>
 #include <linux/sizes.h>
 #include <linux/slab.h>
 #include <linux/string.h>
@@ -158,6 +159,16 @@
 #define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
 #define PCIE_RGR1_SW_INIT_1(pcie)	(pcie->reg_offsets[RGR1_SW_INIT_1])
 
+/* Rescal registers */
+#define PCIE_DVT_PMU_PCIE_PHY_CTRL				0xc700
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS			0x3
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK		0x4
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT	0x2
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK		0x2
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT		0x1
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK		0x1
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT		0x0
+
 enum {
 	RGR1_SW_INIT_1,
 	EXT_CFG_INDEX,
@@ -247,6 +258,7 @@ struct brcm_pcie {
 	const int		*reg_offsets;
 	const int		*reg_field_info;
 	enum pcie_type		type;
+	struct reset_control	*rescal;
 };
 
 /*
@@ -965,6 +977,47 @@ static void brcm_pcie_enter_l23(struct brcm_pcie *pcie)
 		dev_err(pcie->dev, "failed to enter low-power link state\n");
 }
 
+static int brcm_phy_cntl(struct brcm_pcie *pcie, const int start)
+{
+	static const u32 shifts[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT,
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT,
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT,};
+	static const u32 masks[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK,
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK,
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK,};
+	const int beg = start ? 0 : PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS - 1;
+	const int end = start ? PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS : -1;
+	u32 tmp, combined_mask = 0;
+	u32 val = !!start;
+	void __iomem *base = pcie->base;
+	int i;
+
+	for (i = beg; i != end; start ? i++ : i--) {
+		tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
+		tmp = (tmp & ~masks[i]) | ((val << shifts[i]) & masks[i]);
+		writel(tmp, base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
+		usleep_range(50, 200);
+		combined_mask |= masks[i];
+	}
+
+	tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
+	val = start ? combined_mask : 0;
+
+	return (tmp & combined_mask) == val ? 0 : -EIO;
+}
+
+static inline int brcm_phy_start(struct brcm_pcie *pcie)
+{
+	return pcie->rescal ? brcm_phy_cntl(pcie, 1) : 0;
+}
+
+static inline int brcm_phy_stop(struct brcm_pcie *pcie)
+{
+	return pcie->rescal ? brcm_phy_cntl(pcie, 0) : 0;
+}
+
 static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
 {
 	void __iomem *base = pcie->base;
@@ -992,11 +1045,15 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
 static int brcm_pcie_suspend(struct device *dev)
 {
 	struct brcm_pcie *pcie = dev_get_drvdata(dev);
+	int ret;
 
 	brcm_pcie_turn_off(pcie);
+	ret = brcm_phy_stop(pcie);
+	if (ret)
+		dev_err(pcie->dev, "failed to stop phy\n");
 	clk_disable_unprepare(pcie->clk);
 
-	return 0;
+	return ret;
 }
 
 static int brcm_pcie_resume(struct device *dev)
@@ -1009,6 +1066,12 @@ static int brcm_pcie_resume(struct device *dev)
 	base = pcie->base;
 	clk_prepare_enable(pcie->clk);
 
+	ret = brcm_phy_start(pcie);
+	if (ret) {
+		dev_err(pcie->dev, "failed to start phy\n");
+		return ret;
+	}
+
 	/* Take bridge out of reset so we can access the SERDES reg */
 	brcm_pcie_bridge_sw_init_set(pcie, 0);
 
@@ -1034,6 +1097,9 @@ static void __brcm_pcie_remove(struct brcm_pcie *pcie)
 {
 	brcm_msi_remove(pcie);
 	brcm_pcie_turn_off(pcie);
+	if (brcm_phy_stop(pcie))
+		dev_err(pcie->dev, "failed to stop phy\n");
+	reset_control_assert(pcie->rescal);
 	clk_disable_unprepare(pcie->clk);
 }
 
@@ -1112,6 +1178,20 @@ static int brcm_pcie_probe(struct platform_device *pdev)
 		dev_err(&pdev->dev, "could not enable clock\n");
 		return ret;
 	}
+	pcie->rescal = devm_reset_control_get_optional_shared(&pdev->dev, "rescal");
+	if (IS_ERR(pcie->rescal))
+		return PTR_ERR(pcie->rescal);
+
+	ret = reset_control_deassert(pcie->rescal);
+	if (ret)
+		dev_err(&pdev->dev, "failed to deassert 'rescal'\n");
+
+	ret = brcm_phy_start(pcie);
+	if (ret) {
+		dev_err(pcie->dev, "failed to start phy\n");
+		reset_control_assert(pcie->rescal);
+		return ret;
+	}
 
 	ret = brcm_pcie_setup(pcie);
 	if (ret)
-- 
2.17.1

[PATCH v11 06/11] PCI: brcmstb: Add control of rescal reset

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

Some STB chips have a special purpose reset controller named RESCAL (reset
calibration).  The PCIe HW can now control RESCAL to start and stop its
operation.  On probe(), the RESCAL is deasserted and the driver goes
through the sequence of setting registers and reading status in order to
start the internal PHY that is required for the PCIe.

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 82 ++++++++++++++++++++++++++-
 1 file changed, 81 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index acf2239b0251..041b8d109563 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -23,6 +23,7 @@
 #include <linux/of_platform.h>
 #include <linux/pci.h>
 #include <linux/printk.h>
+#include <linux/reset.h>
 #include <linux/sizes.h>
 #include <linux/slab.h>
 #include <linux/string.h>
@@ -158,6 +159,16 @@
 #define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
 #define PCIE_RGR1_SW_INIT_1(pcie)	(pcie->reg_offsets[RGR1_SW_INIT_1])
 
+/* Rescal registers */
+#define PCIE_DVT_PMU_PCIE_PHY_CTRL				0xc700
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS			0x3
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK		0x4
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT	0x2
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK		0x2
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT		0x1
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK		0x1
+#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT		0x0
+
 enum {
 	RGR1_SW_INIT_1,
 	EXT_CFG_INDEX,
@@ -247,6 +258,7 @@ struct brcm_pcie {
 	const int		*reg_offsets;
 	const int		*reg_field_info;
 	enum pcie_type		type;
+	struct reset_control	*rescal;
 };
 
 /*
@@ -965,6 +977,47 @@ static void brcm_pcie_enter_l23(struct brcm_pcie *pcie)
 		dev_err(pcie->dev, "failed to enter low-power link state\n");
 }
 
+static int brcm_phy_cntl(struct brcm_pcie *pcie, const int start)
+{
+	static const u32 shifts[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT,
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT,
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT,};
+	static const u32 masks[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK,
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK,
+		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK,};
+	const int beg = start ? 0 : PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS - 1;
+	const int end = start ? PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS : -1;
+	u32 tmp, combined_mask = 0;
+	u32 val = !!start;
+	void __iomem *base = pcie->base;
+	int i;
+
+	for (i = beg; i != end; start ? i++ : i--) {
+		tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
+		tmp = (tmp & ~masks[i]) | ((val << shifts[i]) & masks[i]);
+		writel(tmp, base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
+		usleep_range(50, 200);
+		combined_mask |= masks[i];
+	}
+
+	tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
+	val = start ? combined_mask : 0;
+
+	return (tmp & combined_mask) == val ? 0 : -EIO;
+}
+
+static inline int brcm_phy_start(struct brcm_pcie *pcie)
+{
+	return pcie->rescal ? brcm_phy_cntl(pcie, 1) : 0;
+}
+
+static inline int brcm_phy_stop(struct brcm_pcie *pcie)
+{
+	return pcie->rescal ? brcm_phy_cntl(pcie, 0) : 0;
+}
+
 static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
 {
 	void __iomem *base = pcie->base;
@@ -992,11 +1045,15 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
 static int brcm_pcie_suspend(struct device *dev)
 {
 	struct brcm_pcie *pcie = dev_get_drvdata(dev);
+	int ret;
 
 	brcm_pcie_turn_off(pcie);
+	ret = brcm_phy_stop(pcie);
+	if (ret)
+		dev_err(pcie->dev, "failed to stop phy\n");
 	clk_disable_unprepare(pcie->clk);
 
-	return 0;
+	return ret;
 }
 
 static int brcm_pcie_resume(struct device *dev)
@@ -1009,6 +1066,12 @@ static int brcm_pcie_resume(struct device *dev)
 	base = pcie->base;
 	clk_prepare_enable(pcie->clk);
 
+	ret = brcm_phy_start(pcie);
+	if (ret) {
+		dev_err(pcie->dev, "failed to start phy\n");
+		return ret;
+	}
+
 	/* Take bridge out of reset so we can access the SERDES reg */
 	brcm_pcie_bridge_sw_init_set(pcie, 0);
 
@@ -1034,6 +1097,9 @@ static void __brcm_pcie_remove(struct brcm_pcie *pcie)
 {
 	brcm_msi_remove(pcie);
 	brcm_pcie_turn_off(pcie);
+	if (brcm_phy_stop(pcie))
+		dev_err(pcie->dev, "failed to stop phy\n");
+	reset_control_assert(pcie->rescal);
 	clk_disable_unprepare(pcie->clk);
 }
 
@@ -1112,6 +1178,20 @@ static int brcm_pcie_probe(struct platform_device *pdev)
 		dev_err(&pdev->dev, "could not enable clock\n");
 		return ret;
 	}
+	pcie->rescal = devm_reset_control_get_optional_shared(&pdev->dev, "rescal");
+	if (IS_ERR(pcie->rescal))
+		return PTR_ERR(pcie->rescal);
+
+	ret = reset_control_deassert(pcie->rescal);
+	if (ret)
+		dev_err(&pdev->dev, "failed to deassert 'rescal'\n");
+
+	ret = brcm_phy_start(pcie);
+	if (ret) {
+		dev_err(pcie->dev, "failed to start phy\n");
+		reset_control_assert(pcie->rescal);
+		return ret;
+	}
 
 	ret = brcm_pcie_setup(pcie);
 	if (ret)
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

The new field 'dma_range_map' in struct device is used to facilitate the
use of single or multiple offsets between mapping regions of cpu addrs and
dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
capable of holding a single uniform offset and had no region bounds
checking.

The function of_dma_get_range() has been modified so that it takes a single
argument -- the device node -- and returns a map, NULL, or an error code.
The map is an array that holds the information regarding the DMA regions.
Each range entry contains the address offset, the cpu_start address, the
dma_start address, and the size of the region.

of_dma_configure() is the typical manner to set range offsets but there are
a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
driver code.  These cases now invoke the function
dma_attach_offset_range(dev, cpu_addr, dma_addr, size).

Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
---
 arch/arm/include/asm/dma-mapping.h            | 10 +--
 arch/arm/mach-keystone/keystone.c             | 17 +++--
 arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
 arch/x86/pci/sta2x11-fixup.c                  |  7 +-
 drivers/acpi/arm64/iort.c                     |  5 +-
 drivers/base/core.c                           |  2 +
 drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
 drivers/iommu/io-pgtable-arm.c                |  2 +-
 .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
 .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
 drivers/of/address.c                          | 72 +++++++++----------
 drivers/of/device.c                           | 43 ++++++-----
 drivers/of/of_private.h                       | 10 +--
 drivers/of/unittest.c                         | 34 ++++++---
 drivers/remoteproc/remoteproc_core.c          |  8 ++-
 .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
 drivers/usb/core/message.c                    |  9 ++-
 drivers/usb/core/usb.c                        |  7 +-
 include/linux/device.h                        |  4 +-
 include/linux/dma-direct.h                    |  8 +--
 include/linux/dma-mapping.h                   | 36 ++++++++++
 kernel/dma/coherent.c                         | 10 +--
 kernel/dma/mapping.c                          | 66 +++++++++++++++++
 23 files changed, 265 insertions(+), 115 deletions(-)

diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index bdd80ddbca34..c21893f683b5 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -35,8 +35,11 @@ static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
 #ifndef __arch_pfn_to_dma
 static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
 {
-	if (dev)
-		pfn -= dev->dma_pfn_offset;
+	if (dev) {
+		phys_addr_t paddr = PFN_PHYS(pfn);
+
+		pfn -= PFN_DOWN(dma_offset_from_phys_addr(dev, paddr));
+	}
 	return (dma_addr_t)__pfn_to_bus(pfn);
 }
 
@@ -45,8 +48,7 @@ static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
 	unsigned long pfn = __bus_to_pfn(addr);
 
 	if (dev)
-		pfn += dev->dma_pfn_offset;
-
+		pfn += PFN_DOWN(dma_offset_from_dma_addr(dev, addr));
 	return pfn;
 }
 
diff --git a/arch/arm/mach-keystone/keystone.c b/arch/arm/mach-keystone/keystone.c
index 638808c4e122..78808942ad1c 100644
--- a/arch/arm/mach-keystone/keystone.c
+++ b/arch/arm/mach-keystone/keystone.c
@@ -8,6 +8,7 @@
  */
 #include <linux/io.h>
 #include <linux/of.h>
+#include <linux/dma-mapping.h>
 #include <linux/init.h>
 #include <linux/of_platform.h>
 #include <linux/of_address.h>
@@ -24,8 +25,6 @@
 
 #include "keystone.h"
 
-static unsigned long keystone_dma_pfn_offset __read_mostly;
-
 static int keystone_platform_notifier(struct notifier_block *nb,
 				      unsigned long event, void *data)
 {
@@ -38,9 +37,12 @@ static int keystone_platform_notifier(struct notifier_block *nb,
 		return NOTIFY_BAD;
 
 	if (!dev->of_node) {
-		dev->dma_pfn_offset = keystone_dma_pfn_offset;
-		dev_err(dev, "set dma_pfn_offset%08lx\n",
-			dev->dma_pfn_offset);
+		int ret = dma_set_offset_range(dev, KEYSTONE_HIGH_PHYS_START,
+					       KEYSTONE_LOW_PHYS_START,
+					       KEYSTONE_HIGH_PHYS_SIZE);
+		dev_err(dev, "set dma_offset%08llx%s\n",
+			KEYSTONE_HIGH_PHYS_START - KEYSTONE_LOW_PHYS_START,
+			ret ? " failed" : "");
 	}
 	return NOTIFY_OK;
 }
@@ -51,11 +53,8 @@ static struct notifier_block platform_nb = {
 
 static void __init keystone_init(void)
 {
-	if (PHYS_OFFSET >= KEYSTONE_HIGH_PHYS_START) {
-		keystone_dma_pfn_offset = PFN_DOWN(KEYSTONE_HIGH_PHYS_START -
-						   KEYSTONE_LOW_PHYS_START);
+	if (PHYS_OFFSET >= KEYSTONE_HIGH_PHYS_START)
 		bus_register_notifier(&platform_bus_type, &platform_nb);
-	}
 	keystone_pm_runtime_init();
 }
 
diff --git a/arch/sh/drivers/pci/pcie-sh7786.c b/arch/sh/drivers/pci/pcie-sh7786.c
index e0b568aaa701..e929f85c5038 100644
--- a/arch/sh/drivers/pci/pcie-sh7786.c
+++ b/arch/sh/drivers/pci/pcie-sh7786.c
@@ -12,6 +12,7 @@
 #include <linux/io.h>
 #include <linux/async.h>
 #include <linux/delay.h>
+#include <linux/dma-mapping.h>
 #include <linux/slab.h>
 #include <linux/clk.h>
 #include <linux/sh_clk.h>
@@ -31,6 +32,8 @@ struct sh7786_pcie_port {
 static struct sh7786_pcie_port *sh7786_pcie_ports;
 static unsigned int nr_ports;
 static unsigned long dma_pfn_offset;
+size_t memsize;
+u64 memstart;
 
 static struct sh7786_pcie_hwops {
 	int (*core_init)(void);
@@ -301,7 +304,6 @@ static int __init pcie_init(struct sh7786_pcie_port *port)
 	struct pci_channel *chan = port->hose;
 	unsigned int data;
 	phys_addr_t memstart, memend;
-	size_t memsize;
 	int ret, i, win;
 
 	/* Begin initialization */
@@ -368,8 +370,6 @@ static int __init pcie_init(struct sh7786_pcie_port *port)
 	memstart = ALIGN_DOWN(memstart, memsize);
 	memsize = roundup_pow_of_two(memend - memstart);
 
-	dma_pfn_offset = memstart >> PAGE_SHIFT;
-
 	/*
 	 * If there's more than 512MB of memory, we need to roll over to
 	 * LAR1/LAMR1.
@@ -487,7 +487,8 @@ int pcibios_map_platform_irq(const struct pci_dev *pdev, u8 slot, u8 pin)
 
 void pcibios_bus_add_device(struct pci_dev *pdev)
 {
-	pdev->dev.dma_pfn_offset = dma_pfn_offset;
+	dma_set_offset_range(&pdev->dev, __pa(memory_start),
+			     __pa(memory_start) - memstart, memsize);
 }
 
 static int __init sh7786_pcie_core_init(void)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index c313d784efab..ea3a58323f81 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -12,6 +12,7 @@
 #include <linux/export.h>
 #include <linux/list.h>
 #include <linux/dma-direct.h>
+#include <linux/dma-mapping.h>
 #include <asm/iommu.h>
 
 #define STA2X11_SWIOTLB_SIZE (4*1024*1024)
@@ -133,7 +134,7 @@ static void sta2x11_map_ep(struct pci_dev *pdev)
 	struct sta2x11_instance *instance = sta2x11_pdev_to_instance(pdev);
 	struct device *dev = &pdev->dev;
 	u32 amba_base, max_amba_addr;
-	int i;
+	int i, ret;
 
 	if (!instance)
 		return;
@@ -141,7 +142,9 @@ static void sta2x11_map_ep(struct pci_dev *pdev)
 	pci_read_config_dword(pdev, AHB_BASE(0), &amba_base);
 	max_amba_addr = amba_base + STA2X11_AMBA_SIZE - 1;
 
-	dev->dma_pfn_offset = PFN_DOWN(-amba_base);
+	ret = dma_set_offset_range(dev, 0, amba_base, STA2X11_AMBA_SIZE);
+	if (ret)
+		dev_err(dev, "sta2x11: could not set DMA offset\n");
 
 	dev->bus_dma_limit = max_amba_addr;
 	pci_set_consistent_dma_mask(pdev, max_amba_addr);
diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index ec782e4a0fe4..8d99e45395bf 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -1184,8 +1184,9 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size)
 	*dma_addr = dmaaddr;
 	*dma_size = size;
 
-	dev->dma_pfn_offset = PFN_DOWN(offset);
-	dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset);
+	ret = dma_set_offset_range(dev, dmaaddr + offset, dmaaddr, size);
+
+	dev_dbg(dev, "dma_offset(%#08llx)%s\n", offset, ret ? " failed!" : "");
 }
 
 static void __init acpi_iort_register_irq(int hwirq, const char *name,
diff --git a/drivers/base/core.c b/drivers/base/core.c
index ac1046a382bc..548520dc6bee 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -1792,6 +1792,8 @@ static void device_release(struct kobject *kobj)
 	 */
 	devres_release_all(dev);
 
+	kfree(dev->dma_range_map);
+
 	if (dev->release)
 		dev->release(dev);
 	else if (dev->type && dev->type->release)
diff --git a/drivers/gpu/drm/sun4i/sun4i_backend.c b/drivers/gpu/drm/sun4i/sun4i_backend.c
index 072ea113e6be..48a4adf1f04e 100644
--- a/drivers/gpu/drm/sun4i/sun4i_backend.c
+++ b/drivers/gpu/drm/sun4i/sun4i_backend.c
@@ -11,6 +11,7 @@
 #include <linux/module.h>
 #include <linux/of_device.h>
 #include <linux/of_graph.h>
+#include <linux/dma-mapping.h>
 #include <linux/platform_device.h>
 #include <linux/reset.h>
 
@@ -812,7 +813,9 @@ static int sun4i_backend_bind(struct device *dev, struct device *master,
 		 * on our device since the RAM mapping is at 0 for the DMA bus,
 		 * unlike the CPU.
 		 */
-		drm->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+		ret = dma_set_offset_range(drm->dev, PHYS_OFFSET, 0, SZ_4G);
+		if (ret)
+			return ret;
 	}
 
 	backend->engine.node = dev->of_node;
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index dc7bcf858b6d..d77e881516a4 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -751,7 +751,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
 	if (cfg->oas > ARM_LPAE_MAX_ADDR_BITS)
 		return NULL;
 
-	if (!selftest_running && cfg->iommu_dev->dma_pfn_offset) {
+	if (!selftest_running && cfg->iommu_dev->dma_range_map) {
 		dev_err(cfg->iommu_dev, "Cannot accommodate DMA offset for IOMMU page tables\n");
 		return NULL;
 	}
diff --git a/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c b/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c
index 5319eb1ab309..c02782f7efc7 100644
--- a/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c
+++ b/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c
@@ -7,6 +7,7 @@
  */
 
 #include <linux/clk.h>
+#include <linux/dma-mapping.h>
 #include <linux/interrupt.h>
 #include <linux/module.h>
 #include <linux/mutex.h>
@@ -183,7 +184,9 @@ static int sun4i_csi_probe(struct platform_device *pdev)
 			return ret;
 	} else {
 #ifdef PHYS_PFN_OFFSET
-		csi->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+		ret = dma_set_offset_range(csi->dev, PHYS_OFFSET, 0, SZ_4G);
+		if (ret)
+			return ret;
 #endif
 	}
 
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index 28e89340fed9..b519d1dd5195 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -900,7 +900,9 @@ static int sun6i_csi_probe(struct platform_device *pdev)
 
 	sdev->dev = &pdev->dev;
 	/* The DMA bus has the memory mapped at 0 */
-	sdev->dev->dma_pfn_offset = PHYS_OFFSET >> PAGE_SHIFT;
+	ret = dma_set_offset_range(sdev->dev, PHYS_OFFSET, 0, SZ_4G);
+	if (ret)
+		return ret;
 
 	ret = sun6i_csi_resource_request(sdev, pdev);
 	if (ret)
diff --git a/drivers/of/address.c b/drivers/of/address.c
index 590493e04b01..50d586c5a858 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -923,33 +923,33 @@ void __iomem *of_io_request_and_map(struct device_node *np, int index,
 }
 EXPORT_SYMBOL(of_io_request_and_map);
 
+#ifdef CONFIG_HAS_DMA
 /**
- * of_dma_get_range - Get DMA range info
+ * of_dma_get_range - Get DMA range info and put it into a map array
  * @np:		device node to get DMA range info
- * @dma_addr:	pointer to store initial DMA address of DMA range
- * @paddr:	pointer to store initial CPU address of DMA range
- * @size:	pointer to store size of DMA range
+ * @map:	dma range structure to return
  *
  * Look in bottom up direction for the first "dma-ranges" property
- * and parse it.
- *  dma-ranges format:
+ * and parse it.  Put the information into a DMA offset map array.
+ *
+ * dma-ranges format:
  *	DMA addr (dma_addr)	: naddr cells
  *	CPU addr (phys_addr_t)	: pna cells
  *	size			: nsize cells
  *
- * It returns -ENODEV if "dma-ranges" property was not found
- * for this device in DT.
+ * It returns -ENODEV if "dma-ranges" property was not found for this
+ * device in the DT.
  */
-int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size)
+int of_dma_get_range(struct device_node *np, const struct bus_dma_region **map)
 {
 	struct device_node *node = of_node_get(np);
 	const __be32 *ranges = NULL;
-	int len;
-	int ret = 0;
 	bool found_dma_ranges = false;
 	struct of_range_parser parser;
 	struct of_range range;
-	u64 dma_start = U64_MAX, dma_end = 0, dma_offset = 0;
+	struct bus_dma_region *r;
+	int len, num_ranges = 0;
+	int ret = 0;
 
 	while (node) {
 		ranges = of_get_property(node, "dma-ranges", &len);
@@ -975,44 +975,36 @@ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *siz
 	}
 
 	of_dma_range_parser_init(&parser, node);
+	for_each_of_range(&parser, &range)
+		num_ranges++;
 
-	for_each_of_range(&parser, &range) {
-		pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
-			 range.bus_addr, range.cpu_addr, range.size);
-
-		if (dma_offset && range.cpu_addr - range.bus_addr != dma_offset) {
-			pr_warn("Can't handle multiple dma-ranges with different offsets on node(%pOF)\n", node);
-			/* Don't error out as we'd break some existing DTs */
-			continue;
-		}
-		dma_offset = range.cpu_addr - range.bus_addr;
-
-		/* Take lower and upper limits */
-		if (range.bus_addr < dma_start)
-			dma_start = range.bus_addr;
-		if (range.bus_addr + range.size > dma_end)
-			dma_end = range.bus_addr + range.size;
-	}
+	of_dma_range_parser_init(&parser, node);
 
-	if (dma_start >= dma_end) {
-		ret = -EINVAL;
-		pr_debug("Invalid DMA ranges configuration on node(%pOF)\n",
-			 node);
+	r = kcalloc(num_ranges + 1, sizeof(*r), GFP_KERNEL);
+	if (!r) {
+		ret = -ENOMEM;
 		goto out;
 	}
 
-	*dma_addr = dma_start;
-	*size = dma_end - dma_start;
-	*paddr = dma_start + dma_offset;
-
-	pr_debug("final: dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
-		 *dma_addr, *paddr, *size);
+	/*
+	 * Record all info in the generic DMA ranges array for struct device.
+	 */
+	*map = r;
+	for_each_of_range(&parser, &range) {
+		pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
+			 range.bus_addr, range.cpu_addr, range.size);
+		r->cpu_start = range.cpu_addr;
+		r->dma_start = range.bus_addr;
+		r->size = range.size;
+		r->offset = (u64)range.cpu_addr - (u64)range.bus_addr;
+		r++;
+	}
 
 out:
 	of_node_put(node);
-
 	return ret;
 }
+#endif
 
 /**
  * of_dma_is_coherent - Check if device is coherent
diff --git a/drivers/of/device.c b/drivers/of/device.c
index b439c1e05434..cd668e7ea7da 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -90,14 +90,14 @@ int of_device_add(struct platform_device *ofdev)
 int of_dma_configure_id(struct device *dev, struct device_node *np,
 			bool force_dma, const u32 *id)
 {
-	u64 dma_addr, paddr, size = 0;
-	int ret;
-	bool coherent;
-	unsigned long offset;
 	const struct iommu_ops *iommu;
-	u64 mask, end;
+	const struct bus_dma_region *map = NULL;
+	dma_addr_t dma_start = 0;
+	u64 mask, end, size = 0;
+	bool coherent;
+	int ret;
 
-	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
+	ret = of_dma_get_range(np, &map);
 	if (ret < 0) {
 		/*
 		 * For legacy reasons, we have to assume some devices need
@@ -106,26 +106,34 @@ int of_dma_configure_id(struct device *dev, struct device_node *np,
 		 */
 		if (!force_dma)
 			return ret == -ENODEV ? 0 : ret;
-
-		dma_addr = offset = 0;
 	} else {
-		offset = PFN_DOWN(paddr - dma_addr);
+		const struct bus_dma_region *r = map;
+		dma_addr_t dma_end = 0;
+
+		/* Determine the overall bounds of all DMA regions */
+		for (dma_start = ~(dma_addr_t)0; r->size; r++) {
+			/* Take lower and upper limits */
+			if (r->dma_start < dma_start)
+				dma_start = r->dma_start;
+			if (r->dma_start + r->size > dma_end)
+				dma_end = r->dma_start + r->size;
+		}
+		size = dma_end - dma_start;
 
 		/*
 		 * Add a work around to treat the size as mask + 1 in case
 		 * it is defined in DT as a mask.
 		 */
 		if (size & 1) {
-			dev_warn(dev, "Invalid size 0x%llx for dma-range\n",
-				 size);
+			dev_warn(dev, "Invalid size 0x%llx for dma-range(s)\n", size);
 			size = size + 1;
 		}
 
 		if (!size) {
 			dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
+			kfree(map);
 			return -EINVAL;
 		}
-		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
 	}
 
 	/*
@@ -144,13 +152,11 @@ int of_dma_configure_id(struct device *dev, struct device_node *np,
 	else if (!size)
 		size = 1ULL << 32;
 
-	dev->dma_pfn_offset = offset;
-
 	/*
 	 * Limit coherent and dma mask based on size and default mask
 	 * set by the driver.
 	 */
-	end = dma_addr + size - 1;
+	end = dma_start + size - 1;
 	mask = DMA_BIT_MASK(ilog2(end) + 1);
 	dev->coherent_dma_mask &= mask;
 	*dev->dma_mask &= mask;
@@ -163,14 +169,17 @@ int of_dma_configure_id(struct device *dev, struct device_node *np,
 		coherent ? " " : " not ");
 
 	iommu = of_iommu_configure(dev, np, id);
-	if (PTR_ERR(iommu) == -EPROBE_DEFER)
+	if (PTR_ERR(iommu) == -EPROBE_DEFER) {
+		kfree(map);
 		return -EPROBE_DEFER;
+	}
 
 	dev_dbg(dev, "device is%sbehind an iommu\n",
 		iommu ? " " : " not ");
 
-	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
+	arch_setup_dma_ops(dev, dma_start, size, iommu, coherent);
 
+	dev->dma_range_map = map;
 	return 0;
 }
 EXPORT_SYMBOL_GPL(of_dma_configure_id);
diff --git a/drivers/of/of_private.h b/drivers/of/of_private.h
index edc682249c00..5c13485a4ad1 100644
--- a/drivers/of/of_private.h
+++ b/drivers/of/of_private.h
@@ -157,12 +157,12 @@ extern void __of_sysfs_remove_bin_file(struct device_node *np,
 extern int of_bus_n_addr_cells(struct device_node *np);
 extern int of_bus_n_size_cells(struct device_node *np);
 
-#ifdef CONFIG_OF_ADDRESS
-extern int of_dma_get_range(struct device_node *np, u64 *dma_addr,
-			    u64 *paddr, u64 *size);
+struct bus_dma_region;
+#if defined(CONFIG_OF_ADDRESS) && defined(CONFIG_HAS_DMA)
+int of_dma_get_range(struct device_node *np, const struct bus_dma_region **map);
 #else
-static inline int of_dma_get_range(struct device_node *np, u64 *dma_addr,
-				   u64 *paddr, u64 *size)
+static inline int of_dma_get_range(struct device_node *np,
+		const struct bus_dma_region **map)
 {
 	return -ENODEV;
 }
diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index 9b7e84bdc7d4..dea68877f3b4 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -7,6 +7,7 @@
 
 #include <linux/memblock.h>
 #include <linux/clk.h>
+#include <linux/dma-mapping.h>
 #include <linux/err.h>
 #include <linux/errno.h>
 #include <linux/hashtable.h>
@@ -869,10 +870,10 @@ static void __init of_unittest_changeset(void)
 }
 
 static void __init of_unittest_dma_ranges_one(const char *path,
-		u64 expect_dma_addr, u64 expect_paddr, u64 expect_size)
+		u64 expect_dma_addr, u64 expect_paddr)
 {
 	struct device_node *np;
-	u64 dma_addr, paddr, size;
+	const struct bus_dma_region *map = NULL;
 	int rc;
 
 	np = of_find_node_by_path(path);
@@ -881,16 +882,29 @@ static void __init of_unittest_dma_ranges_one(const char *path,
 		return;
 	}
 
-	rc = of_dma_get_range(np, &dma_addr, &paddr, &size);
+	rc = of_dma_get_range(np, &map);
 
 	unittest(!rc, "of_dma_get_range failed on node %pOF rc=%i\n", np, rc);
+
 	if (!rc) {
-		unittest(size == expect_size,
-			 "of_dma_get_range wrong size on node %pOF size=%llx\n", np, size);
+		phys_addr_t	paddr;
+		dma_addr_t	dma_addr;
+		struct device	dev_bogus;
+
+		dev_bogus.dma_range_map = map;
+		paddr = (phys_addr_t)expect_dma_addr +
+			dma_offset_from_dma_addr(&dev_bogus, expect_dma_addr);
+		dma_addr = (dma_addr_t)expect_paddr -
+			dma_offset_from_phys_addr(&dev_bogus, expect_paddr);
+
 		unittest(paddr == expect_paddr,
-			 "of_dma_get_range wrong phys addr (%llx) on node %pOF", paddr, np);
+			 "of_dma_get_range: wrong phys addr %llx (expecting %llx) on node %pOF\n",
+			 (u64)paddr, expect_paddr, np);
 		unittest(dma_addr == expect_dma_addr,
-			 "of_dma_get_range wrong DMA addr (%llx) on node %pOF", dma_addr, np);
+			 "of_dma_get_range: wrong DMA addr %llx (expecting %llx) on node %pOF\n",
+			 (u64)dma_addr, expect_dma_addr, np);
+
+		kfree(map);
 	}
 	of_node_put(np);
 }
@@ -898,11 +912,11 @@ static void __init of_unittest_dma_ranges_one(const char *path,
 static void __init of_unittest_parse_dma_ranges(void)
 {
 	of_unittest_dma_ranges_one("/testcase-data/address-tests/device@70000000",
-		0x0, 0x20000000, 0x40000000);
+		0x0, 0x20000000);
 	of_unittest_dma_ranges_one("/testcase-data/address-tests/bus@80000000/device@1000",
-		0x100000000, 0x20000000, 0x2000000000);
+		0x100000000, 0x20000000);
 	of_unittest_dma_ranges_one("/testcase-data/address-tests/pci@90000000",
-		0x80000000, 0x20000000, 0x10000000);
+		0x80000000, 0x20000000);
 }
 
 static void __init of_unittest_pci_dma_ranges(void)
diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 7f90eeea67e2..0946161866e0 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -529,7 +529,13 @@ static int rproc_handle_vdev(struct rproc *rproc, struct fw_rsc_vdev *rsc,
 	/* Initialise vdev subdevice */
 	snprintf(name, sizeof(name), "vdev%dbuffer", rvdev->index);
 	rvdev->dev.parent = &rproc->dev;
-	rvdev->dev.dma_pfn_offset = rproc->dev.parent->dma_pfn_offset;
+	if (rproc->dev.parent->dma_range_map) {
+		const struct bus_dma_region *map =
+			dma_copy_dma_range_map(rproc->dev.parent->dma_range_map);
+		if (!map)
+			return -ENOMEM;
+		rvdev->dev.dma_range_map = map;
+	}
 	rvdev->dev.release = rproc_rvdev_release;
 	dev_set_name(&rvdev->dev, "%s#%s", dev_name(rvdev->dev.parent), name);
 	dev_set_drvdata(&rvdev->dev, rvdev);
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
index 1744e6fcc999..249e4bddaa40 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
@@ -230,8 +230,11 @@ int cedrus_hw_probe(struct cedrus_dev *dev)
 	 */
 
 #ifdef PHYS_PFN_OFFSET
-	if (!(variant->quirks & CEDRUS_QUIRK_NO_DMA_OFFSET))
-		dev->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+	if (!(variant->quirks & CEDRUS_QUIRK_NO_DMA_OFFSET)) {
+		ret = dma_set_offset_range(dev->dev, PHYS_OFFSET, 0, SZ_4G);
+		if (ret)
+			return ret;
+	}
 #endif
 
 	ret = of_reserved_mem_device_init(dev->dev);
diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c
index 6197938dcc2d..c6ccc7b04a7b 100644
--- a/drivers/usb/core/message.c
+++ b/drivers/usb/core/message.c
@@ -1956,10 +1956,15 @@ int usb_set_configuration(struct usb_device *dev, int configuration)
 		intf->dev.groups = usb_interface_groups;
 		/*
 		 * Please refer to usb_alloc_dev() to see why we set
-		 * dma_mask and dma_pfn_offset.
+		 * dma_mask and dma_range_map.
 		 */
 		intf->dev.dma_mask = dev->dev.dma_mask;
-		intf->dev.dma_pfn_offset = dev->dev.dma_pfn_offset;
+		if (dev->dev.dma_range_map) {
+			intf->dev.dma_range_map =
+				dma_copy_dma_range_map(dev->dev.dma_range_map);
+			if (!intf->dev.dma_range_map)
+				dev_err(&dev->dev, "failed to copy DMA map\n");
+		}
 		INIT_WORK(&intf->reset_ws, __usb_queue_reset_device);
 		intf->minor = -1;
 		device_initialize(&intf->dev);
diff --git a/drivers/usb/core/usb.c b/drivers/usb/core/usb.c
index bafc113f2b3e..6e6966704b63 100644
--- a/drivers/usb/core/usb.c
+++ b/drivers/usb/core/usb.c
@@ -610,7 +610,12 @@ struct usb_device *usb_alloc_dev(struct usb_device *parent,
 	 * mask for the entire HCD, so don't do that.
 	 */
 	dev->dev.dma_mask = bus->sysdev->dma_mask;
-	dev->dev.dma_pfn_offset = bus->sysdev->dma_pfn_offset;
+	if (bus->sysdev->dma_range_map) {
+		dev->dev.dma_range_map =
+			dma_copy_dma_range_map(bus->sysdev->dma_range_map);
+		if (!dev->dev.dma_range_map)
+			dev_err(&dev->dev, "failed to copy DMA map\n");
+	}
 	set_dev_node(&dev->dev, dev_to_node(bus->sysdev));
 	dev->state = USB_STATE_ATTACHED;
 	dev->lpm_disable_count = 1;
diff --git a/include/linux/device.h b/include/linux/device.h
index ca18da4768e3..1c78621fc3c0 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -466,7 +466,7 @@ struct dev_links_info {
  * 		such descriptors.
  * @bus_dma_limit: Limit of an upstream bridge or bus which imposes a smaller
  *		DMA limit than the device itself supports.
- * @dma_pfn_offset: offset of DMA memory range relatively of RAM
+ * @dma_range_map: map for DMA memory ranges relative to that of RAM
  * @dma_parms:	A low level driver may set these to teach IOMMU code about
  * 		segment limitations.
  * @dma_pools:	Dma pools (if dma'ble device).
@@ -561,7 +561,7 @@ struct device {
 					     64 bit addresses for consistent
 					     allocations such descriptors. */
 	u64		bus_dma_limit;	/* upstream dma constraint */
-	unsigned long	dma_pfn_offset;
+	const struct bus_dma_region *dma_range_map;
 
 	struct device_dma_parameters *dma_parms;
 
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index 5a3ce2a24794..71acdc3ea87d 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -19,16 +19,12 @@ extern unsigned int zone_dma_bits;
 #else
 static inline dma_addr_t __phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	dma_addr_t dev_addr = (dma_addr_t)paddr;
-
-	return dev_addr - ((dma_addr_t)dev->dma_pfn_offset << PAGE_SHIFT);
+	return (dma_addr_t)paddr - dma_offset_from_phys_addr(dev, paddr);
 }
 
 static inline phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dev_addr)
 {
-	phys_addr_t paddr = (phys_addr_t)dev_addr;
-
-	return paddr + ((phys_addr_t)dev->dma_pfn_offset << PAGE_SHIFT);
+	return (phys_addr_t)dev_addr + dma_offset_from_dma_addr(dev, dev_addr);
 }
 #endif /* !CONFIG_ARCH_HAS_PHYS_TO_DMA */
 
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 016b96b384bd..ebe0b2f419da 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -188,7 +188,38 @@ static inline int dma_mmap_from_global_coherent(struct vm_area_struct *vma,
 }
 #endif /* CONFIG_DMA_DECLARE_COHERENT */
 
+struct bus_dma_region {
+	phys_addr_t	cpu_start;
+	dma_addr_t	dma_start;
+	u64		size;
+	u64		offset;
+};
+
 #ifdef CONFIG_HAS_DMA
+static inline u64 dma_offset_from_dma_addr(struct device *dev, dma_addr_t dma_addr)
+{
+	const struct bus_dma_region *m = dev->dma_range_map;
+
+	if (m)
+		for (; m->size; m++)
+			if (dma_addr >= m->dma_start &&
+			    dma_addr - m->dma_start < m->size)
+				return m->offset;
+	return 0;
+}
+
+static inline u64 dma_offset_from_phys_addr(struct device *dev, phys_addr_t paddr)
+{
+	const struct bus_dma_region *m = dev->dma_range_map;
+
+	if (m)
+		for (; m->size; m++)
+			if (paddr >= m->cpu_start &&
+			    paddr - m->cpu_start < m->size)
+				return m->offset;
+	return 0;
+}
+
 #include <asm/dma-mapping.h>
 
 #ifdef CONFIG_DMA_OPS
@@ -608,6 +639,11 @@ static inline void arch_teardown_dma_ops(struct device *dev)
 }
 #endif /* CONFIG_ARCH_HAS_TEARDOWN_DMA_OPS */
 
+int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
+		dma_addr_t dma_start, u64 size);
+
+void *dma_copy_dma_range_map(const struct bus_dma_region *map);
+
 static inline unsigned int dma_get_max_seg_size(struct device *dev)
 {
 	if (dev->dma_parms && dev->dma_parms->max_segment_size)
diff --git a/kernel/dma/coherent.c b/kernel/dma/coherent.c
index 2a0c4985f38e..5d0d72b3bf0f 100644
--- a/kernel/dma/coherent.c
+++ b/kernel/dma/coherent.c
@@ -31,10 +31,12 @@ static inline struct dma_coherent_mem *dev_get_coherent_memory(struct device *de
 static inline dma_addr_t dma_get_device_base(struct device *dev,
 					     struct dma_coherent_mem * mem)
 {
-	if (mem->use_dev_dma_pfn_offset)
-		return (mem->pfn_base - dev->dma_pfn_offset) << PAGE_SHIFT;
-	else
-		return mem->device_base;
+	if (mem->use_dev_dma_pfn_offset) {
+		u64 base_addr = PFN_PHYS((u64)mem->pfn_base);
+
+		return base_addr - dma_offset_from_phys_addr(dev, base_addr);
+	}
+	return mem->device_base;
 }
 
 static int dma_init_coherent_memory(phys_addr_t phys_addr,
diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c
index 0d129421e75f..5576bb2fde20 100644
--- a/kernel/dma/mapping.c
+++ b/kernel/dma/mapping.c
@@ -11,6 +11,7 @@
 #include <linux/dma-noncoherent.h>
 #include <linux/export.h>
 #include <linux/gfp.h>
+#include <linux/limits.h>
 #include <linux/of_device.h>
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
@@ -611,3 +612,68 @@ unsigned long dma_get_merge_boundary(struct device *dev)
 	return ops->get_merge_boundary(dev);
 }
 EXPORT_SYMBOL_GPL(dma_get_merge_boundary);
+
+/**
+ * dma_set_offset_range - Assign scalar offset for a single DMA range.
+ * @dev:	device pointer; needed to "own" the alloced memory.
+ * @cpu_start:  beginning of memory region covered by this offset.
+ * @dma_start:  beginning of DMA/PCI region covered by this offset.
+ * @size:	size of the region.
+ *
+ * This is for the simple case of a uniform offset which cannot
+ * be discovered by "dma-ranges".
+ *
+ * It returns -ENOMEM if out of memory, -EINVAL if a map
+ * already exists, 0 otherwise.
+ */
+int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
+			 dma_addr_t dma_start, u64 size)
+{
+	struct bus_dma_region *map;
+	u64 offset = (u64)cpu_start - (u64)dma_start;
+
+	if (dev->dma_range_map) {
+		dev_err(dev, "attempt to add DMA range to existing map\n");
+		return -EINVAL;
+	}
+
+	if (!offset)
+		return 0;
+
+	map = kcalloc(2, sizeof(*map), GFP_KERNEL);
+	if (!map)
+		return -ENOMEM;
+	map[0].cpu_start = cpu_start;
+	map[0].dma_start = dma_start;
+	map[0].offset = offset;
+	map[0].size = size;
+	dev->dma_range_map = map;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(dma_set_offset_range);
+
+/**
+ * dma_copy_dma_range_map - Copy the dma_range map using kcalloc.
+ * @map:	source to copy.
+ *
+ * A dma range map array consists of N + 1 elements; N acutal ranges
+ * and a zero element trailer.
+ *
+ * It returns NULL if the kcalloc fails.
+ */
+void *dma_copy_dma_range_map(const struct bus_dma_region *map)
+{
+	int num_ranges;
+	struct bus_dma_region *new_map;
+	const struct bus_dma_region *r = map;
+
+	for (num_ranges = 0; r->size; num_ranges++)
+		r++;
+	new_map = kcalloc(num_ranges + 1, sizeof(*map), GFP_KERNEL);
+	if (new_map)
+		memcpy(new_map, map, sizeof(*map) * num_ranges);
+
+	return new_map;
+}
+EXPORT_SYMBOL_GPL(dma_copy_dma_range_map);
-- 
2.17.1

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

[PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan via iommu]

The new field 'dma_range_map' in struct device is used to facilitate the
use of single or multiple offsets between mapping regions of cpu addrs and
dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
capable of holding a single uniform offset and had no region bounds
checking.

The function of_dma_get_range() has been modified so that it takes a single
argument -- the device node -- and returns a map, NULL, or an error code.
The map is an array that holds the information regarding the DMA regions.
Each range entry contains the address offset, the cpu_start address, the
dma_start address, and the size of the region.

of_dma_configure() is the typical manner to set range offsets but there are
a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
driver code.  These cases now invoke the function
dma_attach_offset_range(dev, cpu_addr, dma_addr, size).

Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
---
 arch/arm/include/asm/dma-mapping.h            | 10 +--
 arch/arm/mach-keystone/keystone.c             | 17 +++--
 arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
 arch/x86/pci/sta2x11-fixup.c                  |  7 +-
 drivers/acpi/arm64/iort.c                     |  5 +-
 drivers/base/core.c                           |  2 +
 drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
 drivers/iommu/io-pgtable-arm.c                |  2 +-
 .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
 .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
 drivers/of/address.c                          | 72 +++++++++----------
 drivers/of/device.c                           | 43 ++++++-----
 drivers/of/of_private.h                       | 10 +--
 drivers/of/unittest.c                         | 34 ++++++---
 drivers/remoteproc/remoteproc_core.c          |  8 ++-
 .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
 drivers/usb/core/message.c                    |  9 ++-
 drivers/usb/core/usb.c                        |  7 +-
 include/linux/device.h                        |  4 +-
 include/linux/dma-direct.h                    |  8 +--
 include/linux/dma-mapping.h                   | 36 ++++++++++
 kernel/dma/coherent.c                         | 10 +--
 kernel/dma/mapping.c                          | 66 +++++++++++++++++
 23 files changed, 265 insertions(+), 115 deletions(-)

diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index bdd80ddbca34..c21893f683b5 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -35,8 +35,11 @@ static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
 #ifndef __arch_pfn_to_dma
 static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
 {
-	if (dev)
-		pfn -= dev->dma_pfn_offset;
+	if (dev) {
+		phys_addr_t paddr = PFN_PHYS(pfn);
+
+		pfn -= PFN_DOWN(dma_offset_from_phys_addr(dev, paddr));
+	}
 	return (dma_addr_t)__pfn_to_bus(pfn);
 }
 
@@ -45,8 +48,7 @@ static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
 	unsigned long pfn = __bus_to_pfn(addr);
 
 	if (dev)
-		pfn += dev->dma_pfn_offset;
-
+		pfn += PFN_DOWN(dma_offset_from_dma_addr(dev, addr));
 	return pfn;
 }
 
diff --git a/arch/arm/mach-keystone/keystone.c b/arch/arm/mach-keystone/keystone.c
index 638808c4e122..78808942ad1c 100644
--- a/arch/arm/mach-keystone/keystone.c
+++ b/arch/arm/mach-keystone/keystone.c
@@ -8,6 +8,7 @@
  */
 #include <linux/io.h>
 #include <linux/of.h>
+#include <linux/dma-mapping.h>
 #include <linux/init.h>
 #include <linux/of_platform.h>
 #include <linux/of_address.h>
@@ -24,8 +25,6 @@
 
 #include "keystone.h"
 
-static unsigned long keystone_dma_pfn_offset __read_mostly;
-
 static int keystone_platform_notifier(struct notifier_block *nb,
 				      unsigned long event, void *data)
 {
@@ -38,9 +37,12 @@ static int keystone_platform_notifier(struct notifier_block *nb,
 		return NOTIFY_BAD;
 
 	if (!dev->of_node) {
-		dev->dma_pfn_offset = keystone_dma_pfn_offset;
-		dev_err(dev, "set dma_pfn_offset%08lx\n",
-			dev->dma_pfn_offset);
+		int ret = dma_set_offset_range(dev, KEYSTONE_HIGH_PHYS_START,
+					       KEYSTONE_LOW_PHYS_START,
+					       KEYSTONE_HIGH_PHYS_SIZE);
+		dev_err(dev, "set dma_offset%08llx%s\n",
+			KEYSTONE_HIGH_PHYS_START - KEYSTONE_LOW_PHYS_START,
+			ret ? " failed" : "");
 	}
 	return NOTIFY_OK;
 }
@@ -51,11 +53,8 @@ static struct notifier_block platform_nb = {
 
 static void __init keystone_init(void)
 {
-	if (PHYS_OFFSET >= KEYSTONE_HIGH_PHYS_START) {
-		keystone_dma_pfn_offset = PFN_DOWN(KEYSTONE_HIGH_PHYS_START -
-						   KEYSTONE_LOW_PHYS_START);
+	if (PHYS_OFFSET >= KEYSTONE_HIGH_PHYS_START)
 		bus_register_notifier(&platform_bus_type, &platform_nb);
-	}
 	keystone_pm_runtime_init();
 }
 
diff --git a/arch/sh/drivers/pci/pcie-sh7786.c b/arch/sh/drivers/pci/pcie-sh7786.c
index e0b568aaa701..e929f85c5038 100644
--- a/arch/sh/drivers/pci/pcie-sh7786.c
+++ b/arch/sh/drivers/pci/pcie-sh7786.c
@@ -12,6 +12,7 @@
 #include <linux/io.h>
 #include <linux/async.h>
 #include <linux/delay.h>
+#include <linux/dma-mapping.h>
 #include <linux/slab.h>
 #include <linux/clk.h>
 #include <linux/sh_clk.h>
@@ -31,6 +32,8 @@ struct sh7786_pcie_port {
 static struct sh7786_pcie_port *sh7786_pcie_ports;
 static unsigned int nr_ports;
 static unsigned long dma_pfn_offset;
+size_t memsize;
+u64 memstart;
 
 static struct sh7786_pcie_hwops {
 	int (*core_init)(void);
@@ -301,7 +304,6 @@ static int __init pcie_init(struct sh7786_pcie_port *port)
 	struct pci_channel *chan = port->hose;
 	unsigned int data;
 	phys_addr_t memstart, memend;
-	size_t memsize;
 	int ret, i, win;
 
 	/* Begin initialization */
@@ -368,8 +370,6 @@ static int __init pcie_init(struct sh7786_pcie_port *port)
 	memstart = ALIGN_DOWN(memstart, memsize);
 	memsize = roundup_pow_of_two(memend - memstart);
 
-	dma_pfn_offset = memstart >> PAGE_SHIFT;
-
 	/*
 	 * If there's more than 512MB of memory, we need to roll over to
 	 * LAR1/LAMR1.
@@ -487,7 +487,8 @@ int pcibios_map_platform_irq(const struct pci_dev *pdev, u8 slot, u8 pin)
 
 void pcibios_bus_add_device(struct pci_dev *pdev)
 {
-	pdev->dev.dma_pfn_offset = dma_pfn_offset;
+	dma_set_offset_range(&pdev->dev, __pa(memory_start),
+			     __pa(memory_start) - memstart, memsize);
 }
 
 static int __init sh7786_pcie_core_init(void)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index c313d784efab..ea3a58323f81 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -12,6 +12,7 @@
 #include <linux/export.h>
 #include <linux/list.h>
 #include <linux/dma-direct.h>
+#include <linux/dma-mapping.h>
 #include <asm/iommu.h>
 
 #define STA2X11_SWIOTLB_SIZE (4*1024*1024)
@@ -133,7 +134,7 @@ static void sta2x11_map_ep(struct pci_dev *pdev)
 	struct sta2x11_instance *instance = sta2x11_pdev_to_instance(pdev);
 	struct device *dev = &pdev->dev;
 	u32 amba_base, max_amba_addr;
-	int i;
+	int i, ret;
 
 	if (!instance)
 		return;
@@ -141,7 +142,9 @@ static void sta2x11_map_ep(struct pci_dev *pdev)
 	pci_read_config_dword(pdev, AHB_BASE(0), &amba_base);
 	max_amba_addr = amba_base + STA2X11_AMBA_SIZE - 1;
 
-	dev->dma_pfn_offset = PFN_DOWN(-amba_base);
+	ret = dma_set_offset_range(dev, 0, amba_base, STA2X11_AMBA_SIZE);
+	if (ret)
+		dev_err(dev, "sta2x11: could not set DMA offset\n");
 
 	dev->bus_dma_limit = max_amba_addr;
 	pci_set_consistent_dma_mask(pdev, max_amba_addr);
diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index ec782e4a0fe4..8d99e45395bf 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -1184,8 +1184,9 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size)
 	*dma_addr = dmaaddr;
 	*dma_size = size;
 
-	dev->dma_pfn_offset = PFN_DOWN(offset);
-	dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset);
+	ret = dma_set_offset_range(dev, dmaaddr + offset, dmaaddr, size);
+
+	dev_dbg(dev, "dma_offset(%#08llx)%s\n", offset, ret ? " failed!" : "");
 }
 
 static void __init acpi_iort_register_irq(int hwirq, const char *name,
diff --git a/drivers/base/core.c b/drivers/base/core.c
index ac1046a382bc..548520dc6bee 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -1792,6 +1792,8 @@ static void device_release(struct kobject *kobj)
 	 */
 	devres_release_all(dev);
 
+	kfree(dev->dma_range_map);
+
 	if (dev->release)
 		dev->release(dev);
 	else if (dev->type && dev->type->release)
diff --git a/drivers/gpu/drm/sun4i/sun4i_backend.c b/drivers/gpu/drm/sun4i/sun4i_backend.c
index 072ea113e6be..48a4adf1f04e 100644
--- a/drivers/gpu/drm/sun4i/sun4i_backend.c
+++ b/drivers/gpu/drm/sun4i/sun4i_backend.c
@@ -11,6 +11,7 @@
 #include <linux/module.h>
 #include <linux/of_device.h>
 #include <linux/of_graph.h>
+#include <linux/dma-mapping.h>
 #include <linux/platform_device.h>
 #include <linux/reset.h>
 
@@ -812,7 +813,9 @@ static int sun4i_backend_bind(struct device *dev, struct device *master,
 		 * on our device since the RAM mapping is at 0 for the DMA bus,
 		 * unlike the CPU.
 		 */
-		drm->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+		ret = dma_set_offset_range(drm->dev, PHYS_OFFSET, 0, SZ_4G);
+		if (ret)
+			return ret;
 	}
 
 	backend->engine.node = dev->of_node;
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index dc7bcf858b6d..d77e881516a4 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -751,7 +751,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
 	if (cfg->oas > ARM_LPAE_MAX_ADDR_BITS)
 		return NULL;
 
-	if (!selftest_running && cfg->iommu_dev->dma_pfn_offset) {
+	if (!selftest_running && cfg->iommu_dev->dma_range_map) {
 		dev_err(cfg->iommu_dev, "Cannot accommodate DMA offset for IOMMU page tables\n");
 		return NULL;
 	}
diff --git a/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c b/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c
index 5319eb1ab309..c02782f7efc7 100644
--- a/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c
+++ b/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c
@@ -7,6 +7,7 @@
  */
 
 #include <linux/clk.h>
+#include <linux/dma-mapping.h>
 #include <linux/interrupt.h>
 #include <linux/module.h>
 #include <linux/mutex.h>
@@ -183,7 +184,9 @@ static int sun4i_csi_probe(struct platform_device *pdev)
 			return ret;
 	} else {
 #ifdef PHYS_PFN_OFFSET
-		csi->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+		ret = dma_set_offset_range(csi->dev, PHYS_OFFSET, 0, SZ_4G);
+		if (ret)
+			return ret;
 #endif
 	}
 
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index 28e89340fed9..b519d1dd5195 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -900,7 +900,9 @@ static int sun6i_csi_probe(struct platform_device *pdev)
 
 	sdev->dev = &pdev->dev;
 	/* The DMA bus has the memory mapped at 0 */
-	sdev->dev->dma_pfn_offset = PHYS_OFFSET >> PAGE_SHIFT;
+	ret = dma_set_offset_range(sdev->dev, PHYS_OFFSET, 0, SZ_4G);
+	if (ret)
+		return ret;
 
 	ret = sun6i_csi_resource_request(sdev, pdev);
 	if (ret)
diff --git a/drivers/of/address.c b/drivers/of/address.c
index 590493e04b01..50d586c5a858 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -923,33 +923,33 @@ void __iomem *of_io_request_and_map(struct device_node *np, int index,
 }
 EXPORT_SYMBOL(of_io_request_and_map);
 
+#ifdef CONFIG_HAS_DMA
 /**
- * of_dma_get_range - Get DMA range info
+ * of_dma_get_range - Get DMA range info and put it into a map array
  * @np:		device node to get DMA range info
- * @dma_addr:	pointer to store initial DMA address of DMA range
- * @paddr:	pointer to store initial CPU address of DMA range
- * @size:	pointer to store size of DMA range
+ * @map:	dma range structure to return
  *
  * Look in bottom up direction for the first "dma-ranges" property
- * and parse it.
- *  dma-ranges format:
+ * and parse it.  Put the information into a DMA offset map array.
+ *
+ * dma-ranges format:
  *	DMA addr (dma_addr)	: naddr cells
  *	CPU addr (phys_addr_t)	: pna cells
  *	size			: nsize cells
  *
- * It returns -ENODEV if "dma-ranges" property was not found
- * for this device in DT.
+ * It returns -ENODEV if "dma-ranges" property was not found for this
+ * device in the DT.
  */
-int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size)
+int of_dma_get_range(struct device_node *np, const struct bus_dma_region **map)
 {
 	struct device_node *node = of_node_get(np);
 	const __be32 *ranges = NULL;
-	int len;
-	int ret = 0;
 	bool found_dma_ranges = false;
 	struct of_range_parser parser;
 	struct of_range range;
-	u64 dma_start = U64_MAX, dma_end = 0, dma_offset = 0;
+	struct bus_dma_region *r;
+	int len, num_ranges = 0;
+	int ret = 0;
 
 	while (node) {
 		ranges = of_get_property(node, "dma-ranges", &len);
@@ -975,44 +975,36 @@ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *siz
 	}
 
 	of_dma_range_parser_init(&parser, node);
+	for_each_of_range(&parser, &range)
+		num_ranges++;
 
-	for_each_of_range(&parser, &range) {
-		pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
-			 range.bus_addr, range.cpu_addr, range.size);
-
-		if (dma_offset && range.cpu_addr - range.bus_addr != dma_offset) {
-			pr_warn("Can't handle multiple dma-ranges with different offsets on node(%pOF)\n", node);
-			/* Don't error out as we'd break some existing DTs */
-			continue;
-		}
-		dma_offset = range.cpu_addr - range.bus_addr;
-
-		/* Take lower and upper limits */
-		if (range.bus_addr < dma_start)
-			dma_start = range.bus_addr;
-		if (range.bus_addr + range.size > dma_end)
-			dma_end = range.bus_addr + range.size;
-	}
+	of_dma_range_parser_init(&parser, node);
 
-	if (dma_start >= dma_end) {
-		ret = -EINVAL;
-		pr_debug("Invalid DMA ranges configuration on node(%pOF)\n",
-			 node);
+	r = kcalloc(num_ranges + 1, sizeof(*r), GFP_KERNEL);
+	if (!r) {
+		ret = -ENOMEM;
 		goto out;
 	}
 
-	*dma_addr = dma_start;
-	*size = dma_end - dma_start;
-	*paddr = dma_start + dma_offset;
-
-	pr_debug("final: dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
-		 *dma_addr, *paddr, *size);
+	/*
+	 * Record all info in the generic DMA ranges array for struct device.
+	 */
+	*map = r;
+	for_each_of_range(&parser, &range) {
+		pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
+			 range.bus_addr, range.cpu_addr, range.size);
+		r->cpu_start = range.cpu_addr;
+		r->dma_start = range.bus_addr;
+		r->size = range.size;
+		r->offset = (u64)range.cpu_addr - (u64)range.bus_addr;
+		r++;
+	}
 
 out:
 	of_node_put(node);
-
 	return ret;
 }
+#endif
 
 /**
  * of_dma_is_coherent - Check if device is coherent
diff --git a/drivers/of/device.c b/drivers/of/device.c
index b439c1e05434..cd668e7ea7da 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -90,14 +90,14 @@ int of_device_add(struct platform_device *ofdev)
 int of_dma_configure_id(struct device *dev, struct device_node *np,
 			bool force_dma, const u32 *id)
 {
-	u64 dma_addr, paddr, size = 0;
-	int ret;
-	bool coherent;
-	unsigned long offset;
 	const struct iommu_ops *iommu;
-	u64 mask, end;
+	const struct bus_dma_region *map = NULL;
+	dma_addr_t dma_start = 0;
+	u64 mask, end, size = 0;
+	bool coherent;
+	int ret;
 
-	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
+	ret = of_dma_get_range(np, &map);
 	if (ret < 0) {
 		/*
 		 * For legacy reasons, we have to assume some devices need
@@ -106,26 +106,34 @@ int of_dma_configure_id(struct device *dev, struct device_node *np,
 		 */
 		if (!force_dma)
 			return ret == -ENODEV ? 0 : ret;
-
-		dma_addr = offset = 0;
 	} else {
-		offset = PFN_DOWN(paddr - dma_addr);
+		const struct bus_dma_region *r = map;
+		dma_addr_t dma_end = 0;
+
+		/* Determine the overall bounds of all DMA regions */
+		for (dma_start = ~(dma_addr_t)0; r->size; r++) {
+			/* Take lower and upper limits */
+			if (r->dma_start < dma_start)
+				dma_start = r->dma_start;
+			if (r->dma_start + r->size > dma_end)
+				dma_end = r->dma_start + r->size;
+		}
+		size = dma_end - dma_start;
 
 		/*
 		 * Add a work around to treat the size as mask + 1 in case
 		 * it is defined in DT as a mask.
 		 */
 		if (size & 1) {
-			dev_warn(dev, "Invalid size 0x%llx for dma-range\n",
-				 size);
+			dev_warn(dev, "Invalid size 0x%llx for dma-range(s)\n", size);
 			size = size + 1;
 		}
 
 		if (!size) {
 			dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
+			kfree(map);
 			return -EINVAL;
 		}
-		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
 	}
 
 	/*
@@ -144,13 +152,11 @@ int of_dma_configure_id(struct device *dev, struct device_node *np,
 	else if (!size)
 		size = 1ULL << 32;
 
-	dev->dma_pfn_offset = offset;
-
 	/*
 	 * Limit coherent and dma mask based on size and default mask
 	 * set by the driver.
 	 */
-	end = dma_addr + size - 1;
+	end = dma_start + size - 1;
 	mask = DMA_BIT_MASK(ilog2(end) + 1);
 	dev->coherent_dma_mask &= mask;
 	*dev->dma_mask &= mask;
@@ -163,14 +169,17 @@ int of_dma_configure_id(struct device *dev, struct device_node *np,
 		coherent ? " " : " not ");
 
 	iommu = of_iommu_configure(dev, np, id);
-	if (PTR_ERR(iommu) == -EPROBE_DEFER)
+	if (PTR_ERR(iommu) == -EPROBE_DEFER) {
+		kfree(map);
 		return -EPROBE_DEFER;
+	}
 
 	dev_dbg(dev, "device is%sbehind an iommu\n",
 		iommu ? " " : " not ");
 
-	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
+	arch_setup_dma_ops(dev, dma_start, size, iommu, coherent);
 
+	dev->dma_range_map = map;
 	return 0;
 }
 EXPORT_SYMBOL_GPL(of_dma_configure_id);
diff --git a/drivers/of/of_private.h b/drivers/of/of_private.h
index edc682249c00..5c13485a4ad1 100644
--- a/drivers/of/of_private.h
+++ b/drivers/of/of_private.h
@@ -157,12 +157,12 @@ extern void __of_sysfs_remove_bin_file(struct device_node *np,
 extern int of_bus_n_addr_cells(struct device_node *np);
 extern int of_bus_n_size_cells(struct device_node *np);
 
-#ifdef CONFIG_OF_ADDRESS
-extern int of_dma_get_range(struct device_node *np, u64 *dma_addr,
-			    u64 *paddr, u64 *size);
+struct bus_dma_region;
+#if defined(CONFIG_OF_ADDRESS) && defined(CONFIG_HAS_DMA)
+int of_dma_get_range(struct device_node *np, const struct bus_dma_region **map);
 #else
-static inline int of_dma_get_range(struct device_node *np, u64 *dma_addr,
-				   u64 *paddr, u64 *size)
+static inline int of_dma_get_range(struct device_node *np,
+		const struct bus_dma_region **map)
 {
 	return -ENODEV;
 }
diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index 9b7e84bdc7d4..dea68877f3b4 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -7,6 +7,7 @@
 
 #include <linux/memblock.h>
 #include <linux/clk.h>
+#include <linux/dma-mapping.h>
 #include <linux/err.h>
 #include <linux/errno.h>
 #include <linux/hashtable.h>
@@ -869,10 +870,10 @@ static void __init of_unittest_changeset(void)
 }
 
 static void __init of_unittest_dma_ranges_one(const char *path,
-		u64 expect_dma_addr, u64 expect_paddr, u64 expect_size)
+		u64 expect_dma_addr, u64 expect_paddr)
 {
 	struct device_node *np;
-	u64 dma_addr, paddr, size;
+	const struct bus_dma_region *map = NULL;
 	int rc;
 
 	np = of_find_node_by_path(path);
@@ -881,16 +882,29 @@ static void __init of_unittest_dma_ranges_one(const char *path,
 		return;
 	}
 
-	rc = of_dma_get_range(np, &dma_addr, &paddr, &size);
+	rc = of_dma_get_range(np, &map);
 
 	unittest(!rc, "of_dma_get_range failed on node %pOF rc=%i\n", np, rc);
+
 	if (!rc) {
-		unittest(size == expect_size,
-			 "of_dma_get_range wrong size on node %pOF size=%llx\n", np, size);
+		phys_addr_t	paddr;
+		dma_addr_t	dma_addr;
+		struct device	dev_bogus;
+
+		dev_bogus.dma_range_map = map;
+		paddr = (phys_addr_t)expect_dma_addr +
+			dma_offset_from_dma_addr(&dev_bogus, expect_dma_addr);
+		dma_addr = (dma_addr_t)expect_paddr -
+			dma_offset_from_phys_addr(&dev_bogus, expect_paddr);
+
 		unittest(paddr == expect_paddr,
-			 "of_dma_get_range wrong phys addr (%llx) on node %pOF", paddr, np);
+			 "of_dma_get_range: wrong phys addr %llx (expecting %llx) on node %pOF\n",
+			 (u64)paddr, expect_paddr, np);
 		unittest(dma_addr == expect_dma_addr,
-			 "of_dma_get_range wrong DMA addr (%llx) on node %pOF", dma_addr, np);
+			 "of_dma_get_range: wrong DMA addr %llx (expecting %llx) on node %pOF\n",
+			 (u64)dma_addr, expect_dma_addr, np);
+
+		kfree(map);
 	}
 	of_node_put(np);
 }
@@ -898,11 +912,11 @@ static void __init of_unittest_dma_ranges_one(const char *path,
 static void __init of_unittest_parse_dma_ranges(void)
 {
 	of_unittest_dma_ranges_one("/testcase-data/address-tests/device@70000000",
-		0x0, 0x20000000, 0x40000000);
+		0x0, 0x20000000);
 	of_unittest_dma_ranges_one("/testcase-data/address-tests/bus@80000000/device@1000",
-		0x100000000, 0x20000000, 0x2000000000);
+		0x100000000, 0x20000000);
 	of_unittest_dma_ranges_one("/testcase-data/address-tests/pci@90000000",
-		0x80000000, 0x20000000, 0x10000000);
+		0x80000000, 0x20000000);
 }
 
 static void __init of_unittest_pci_dma_ranges(void)
diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 7f90eeea67e2..0946161866e0 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -529,7 +529,13 @@ static int rproc_handle_vdev(struct rproc *rproc, struct fw_rsc_vdev *rsc,
 	/* Initialise vdev subdevice */
 	snprintf(name, sizeof(name), "vdev%dbuffer", rvdev->index);
 	rvdev->dev.parent = &rproc->dev;
-	rvdev->dev.dma_pfn_offset = rproc->dev.parent->dma_pfn_offset;
+	if (rproc->dev.parent->dma_range_map) {
+		const struct bus_dma_region *map =
+			dma_copy_dma_range_map(rproc->dev.parent->dma_range_map);
+		if (!map)
+			return -ENOMEM;
+		rvdev->dev.dma_range_map = map;
+	}
 	rvdev->dev.release = rproc_rvdev_release;
 	dev_set_name(&rvdev->dev, "%s#%s", dev_name(rvdev->dev.parent), name);
 	dev_set_drvdata(&rvdev->dev, rvdev);
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
index 1744e6fcc999..249e4bddaa40 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
@@ -230,8 +230,11 @@ int cedrus_hw_probe(struct cedrus_dev *dev)
 	 */
 
 #ifdef PHYS_PFN_OFFSET
-	if (!(variant->quirks & CEDRUS_QUIRK_NO_DMA_OFFSET))
-		dev->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+	if (!(variant->quirks & CEDRUS_QUIRK_NO_DMA_OFFSET)) {
+		ret = dma_set_offset_range(dev->dev, PHYS_OFFSET, 0, SZ_4G);
+		if (ret)
+			return ret;
+	}
 #endif
 
 	ret = of_reserved_mem_device_init(dev->dev);
diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c
index 6197938dcc2d..c6ccc7b04a7b 100644
--- a/drivers/usb/core/message.c
+++ b/drivers/usb/core/message.c
@@ -1956,10 +1956,15 @@ int usb_set_configuration(struct usb_device *dev, int configuration)
 		intf->dev.groups = usb_interface_groups;
 		/*
 		 * Please refer to usb_alloc_dev() to see why we set
-		 * dma_mask and dma_pfn_offset.
+		 * dma_mask and dma_range_map.
 		 */
 		intf->dev.dma_mask = dev->dev.dma_mask;
-		intf->dev.dma_pfn_offset = dev->dev.dma_pfn_offset;
+		if (dev->dev.dma_range_map) {
+			intf->dev.dma_range_map =
+				dma_copy_dma_range_map(dev->dev.dma_range_map);
+			if (!intf->dev.dma_range_map)
+				dev_err(&dev->dev, "failed to copy DMA map\n");
+		}
 		INIT_WORK(&intf->reset_ws, __usb_queue_reset_device);
 		intf->minor = -1;
 		device_initialize(&intf->dev);
diff --git a/drivers/usb/core/usb.c b/drivers/usb/core/usb.c
index bafc113f2b3e..6e6966704b63 100644
--- a/drivers/usb/core/usb.c
+++ b/drivers/usb/core/usb.c
@@ -610,7 +610,12 @@ struct usb_device *usb_alloc_dev(struct usb_device *parent,
 	 * mask for the entire HCD, so don't do that.
 	 */
 	dev->dev.dma_mask = bus->sysdev->dma_mask;
-	dev->dev.dma_pfn_offset = bus->sysdev->dma_pfn_offset;
+	if (bus->sysdev->dma_range_map) {
+		dev->dev.dma_range_map =
+			dma_copy_dma_range_map(bus->sysdev->dma_range_map);
+		if (!dev->dev.dma_range_map)
+			dev_err(&dev->dev, "failed to copy DMA map\n");
+	}
 	set_dev_node(&dev->dev, dev_to_node(bus->sysdev));
 	dev->state = USB_STATE_ATTACHED;
 	dev->lpm_disable_count = 1;
diff --git a/include/linux/device.h b/include/linux/device.h
index ca18da4768e3..1c78621fc3c0 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -466,7 +466,7 @@ struct dev_links_info {
  * 		such descriptors.
  * @bus_dma_limit: Limit of an upstream bridge or bus which imposes a smaller
  *		DMA limit than the device itself supports.
- * @dma_pfn_offset: offset of DMA memory range relatively of RAM
+ * @dma_range_map: map for DMA memory ranges relative to that of RAM
  * @dma_parms:	A low level driver may set these to teach IOMMU code about
  * 		segment limitations.
  * @dma_pools:	Dma pools (if dma'ble device).
@@ -561,7 +561,7 @@ struct device {
 					     64 bit addresses for consistent
 					     allocations such descriptors. */
 	u64		bus_dma_limit;	/* upstream dma constraint */
-	unsigned long	dma_pfn_offset;
+	const struct bus_dma_region *dma_range_map;
 
 	struct device_dma_parameters *dma_parms;
 
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index 5a3ce2a24794..71acdc3ea87d 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -19,16 +19,12 @@ extern unsigned int zone_dma_bits;
 #else
 static inline dma_addr_t __phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	dma_addr_t dev_addr = (dma_addr_t)paddr;
-
-	return dev_addr - ((dma_addr_t)dev->dma_pfn_offset << PAGE_SHIFT);
+	return (dma_addr_t)paddr - dma_offset_from_phys_addr(dev, paddr);
 }
 
 static inline phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dev_addr)
 {
-	phys_addr_t paddr = (phys_addr_t)dev_addr;
-
-	return paddr + ((phys_addr_t)dev->dma_pfn_offset << PAGE_SHIFT);
+	return (phys_addr_t)dev_addr + dma_offset_from_dma_addr(dev, dev_addr);
 }
 #endif /* !CONFIG_ARCH_HAS_PHYS_TO_DMA */
 
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 016b96b384bd..ebe0b2f419da 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -188,7 +188,38 @@ static inline int dma_mmap_from_global_coherent(struct vm_area_struct *vma,
 }
 #endif /* CONFIG_DMA_DECLARE_COHERENT */
 
+struct bus_dma_region {
+	phys_addr_t	cpu_start;
+	dma_addr_t	dma_start;
+	u64		size;
+	u64		offset;
+};
+
 #ifdef CONFIG_HAS_DMA
+static inline u64 dma_offset_from_dma_addr(struct device *dev, dma_addr_t dma_addr)
+{
+	const struct bus_dma_region *m = dev->dma_range_map;
+
+	if (m)
+		for (; m->size; m++)
+			if (dma_addr >= m->dma_start &&
+			    dma_addr - m->dma_start < m->size)
+				return m->offset;
+	return 0;
+}
+
+static inline u64 dma_offset_from_phys_addr(struct device *dev, phys_addr_t paddr)
+{
+	const struct bus_dma_region *m = dev->dma_range_map;
+
+	if (m)
+		for (; m->size; m++)
+			if (paddr >= m->cpu_start &&
+			    paddr - m->cpu_start < m->size)
+				return m->offset;
+	return 0;
+}
+
 #include <asm/dma-mapping.h>
 
 #ifdef CONFIG_DMA_OPS
@@ -608,6 +639,11 @@ static inline void arch_teardown_dma_ops(struct device *dev)
 }
 #endif /* CONFIG_ARCH_HAS_TEARDOWN_DMA_OPS */
 
+int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
+		dma_addr_t dma_start, u64 size);
+
+void *dma_copy_dma_range_map(const struct bus_dma_region *map);
+
 static inline unsigned int dma_get_max_seg_size(struct device *dev)
 {
 	if (dev->dma_parms && dev->dma_parms->max_segment_size)
diff --git a/kernel/dma/coherent.c b/kernel/dma/coherent.c
index 2a0c4985f38e..5d0d72b3bf0f 100644
--- a/kernel/dma/coherent.c
+++ b/kernel/dma/coherent.c
@@ -31,10 +31,12 @@ static inline struct dma_coherent_mem *dev_get_coherent_memory(struct device *de
 static inline dma_addr_t dma_get_device_base(struct device *dev,
 					     struct dma_coherent_mem * mem)
 {
-	if (mem->use_dev_dma_pfn_offset)
-		return (mem->pfn_base - dev->dma_pfn_offset) << PAGE_SHIFT;
-	else
-		return mem->device_base;
+	if (mem->use_dev_dma_pfn_offset) {
+		u64 base_addr = PFN_PHYS((u64)mem->pfn_base);
+
+		return base_addr - dma_offset_from_phys_addr(dev, base_addr);
+	}
+	return mem->device_base;
 }
 
 static int dma_init_coherent_memory(phys_addr_t phys_addr,
diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c
index 0d129421e75f..5576bb2fde20 100644
--- a/kernel/dma/mapping.c
+++ b/kernel/dma/mapping.c
@@ -11,6 +11,7 @@
 #include <linux/dma-noncoherent.h>
 #include <linux/export.h>
 #include <linux/gfp.h>
+#include <linux/limits.h>
 #include <linux/of_device.h>
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
@@ -611,3 +612,68 @@ unsigned long dma_get_merge_boundary(struct device *dev)
 	return ops->get_merge_boundary(dev);
 }
 EXPORT_SYMBOL_GPL(dma_get_merge_boundary);
+
+/**
+ * dma_set_offset_range - Assign scalar offset for a single DMA range.
+ * @dev:	device pointer; needed to "own" the alloced memory.
+ * @cpu_start:  beginning of memory region covered by this offset.
+ * @dma_start:  beginning of DMA/PCI region covered by this offset.
+ * @size:	size of the region.
+ *
+ * This is for the simple case of a uniform offset which cannot
+ * be discovered by "dma-ranges".
+ *
+ * It returns -ENOMEM if out of memory, -EINVAL if a map
+ * already exists, 0 otherwise.
+ */
+int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
+			 dma_addr_t dma_start, u64 size)
+{
+	struct bus_dma_region *map;
+	u64 offset = (u64)cpu_start - (u64)dma_start;
+
+	if (dev->dma_range_map) {
+		dev_err(dev, "attempt to add DMA range to existing map\n");
+		return -EINVAL;
+	}
+
+	if (!offset)
+		return 0;
+
+	map = kcalloc(2, sizeof(*map), GFP_KERNEL);
+	if (!map)
+		return -ENOMEM;
+	map[0].cpu_start = cpu_start;
+	map[0].dma_start = dma_start;
+	map[0].offset = offset;
+	map[0].size = size;
+	dev->dma_range_map = map;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(dma_set_offset_range);
+
+/**
+ * dma_copy_dma_range_map - Copy the dma_range map using kcalloc.
+ * @map:	source to copy.
+ *
+ * A dma range map array consists of N + 1 elements; N acutal ranges
+ * and a zero element trailer.
+ *
+ * It returns NULL if the kcalloc fails.
+ */
+void *dma_copy_dma_range_map(const struct bus_dma_region *map)
+{
+	int num_ranges;
+	struct bus_dma_region *new_map;
+	const struct bus_dma_region *r = map;
+
+	for (num_ranges = 0; r->size; num_ranges++)
+		r++;
+	new_map = kcalloc(num_ranges + 1, sizeof(*map), GFP_KERNEL);
+	if (new_map)
+		memcpy(new_map, map, sizeof(*map) * num_ranges);
+
+	return new_map;
+}
+EXPORT_SYMBOL_GPL(dma_copy_dma_range_map);
-- 
2.17.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

The new field 'dma_range_map' in struct device is used to facilitate the
use of single or multiple offsets between mapping regions of cpu addrs and
dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
capable of holding a single uniform offset and had no region bounds
checking.

The function of_dma_get_range() has been modified so that it takes a single
argument -- the device node -- and returns a map, NULL, or an error code.
The map is an array that holds the information regarding the DMA regions.
Each range entry contains the address offset, the cpu_start address, the
dma_start address, and the size of the region.

of_dma_configure() is the typical manner to set range offsets but there are
a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
driver code.  These cases now invoke the function
dma_attach_offset_range(dev, cpu_addr, dma_addr, size).

Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
---
 arch/arm/include/asm/dma-mapping.h            | 10 +--
 arch/arm/mach-keystone/keystone.c             | 17 +++--
 arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
 arch/x86/pci/sta2x11-fixup.c                  |  7 +-
 drivers/acpi/arm64/iort.c                     |  5 +-
 drivers/base/core.c                           |  2 +
 drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
 drivers/iommu/io-pgtable-arm.c                |  2 +-
 .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
 .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
 drivers/of/address.c                          | 72 +++++++++----------
 drivers/of/device.c                           | 43 ++++++-----
 drivers/of/of_private.h                       | 10 +--
 drivers/of/unittest.c                         | 34 ++++++---
 drivers/remoteproc/remoteproc_core.c          |  8 ++-
 .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
 drivers/usb/core/message.c                    |  9 ++-
 drivers/usb/core/usb.c                        |  7 +-
 include/linux/device.h                        |  4 +-
 include/linux/dma-direct.h                    |  8 +--
 include/linux/dma-mapping.h                   | 36 ++++++++++
 kernel/dma/coherent.c                         | 10 +--
 kernel/dma/mapping.c                          | 66 +++++++++++++++++
 23 files changed, 265 insertions(+), 115 deletions(-)

diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index bdd80ddbca34..c21893f683b5 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -35,8 +35,11 @@ static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
 #ifndef __arch_pfn_to_dma
 static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
 {
-	if (dev)
-		pfn -= dev->dma_pfn_offset;
+	if (dev) {
+		phys_addr_t paddr = PFN_PHYS(pfn);
+
+		pfn -= PFN_DOWN(dma_offset_from_phys_addr(dev, paddr));
+	}
 	return (dma_addr_t)__pfn_to_bus(pfn);
 }
 
@@ -45,8 +48,7 @@ static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
 	unsigned long pfn = __bus_to_pfn(addr);
 
 	if (dev)
-		pfn += dev->dma_pfn_offset;
-
+		pfn += PFN_DOWN(dma_offset_from_dma_addr(dev, addr));
 	return pfn;
 }
 
diff --git a/arch/arm/mach-keystone/keystone.c b/arch/arm/mach-keystone/keystone.c
index 638808c4e122..78808942ad1c 100644
--- a/arch/arm/mach-keystone/keystone.c
+++ b/arch/arm/mach-keystone/keystone.c
@@ -8,6 +8,7 @@
  */
 #include <linux/io.h>
 #include <linux/of.h>
+#include <linux/dma-mapping.h>
 #include <linux/init.h>
 #include <linux/of_platform.h>
 #include <linux/of_address.h>
@@ -24,8 +25,6 @@
 
 #include "keystone.h"
 
-static unsigned long keystone_dma_pfn_offset __read_mostly;
-
 static int keystone_platform_notifier(struct notifier_block *nb,
 				      unsigned long event, void *data)
 {
@@ -38,9 +37,12 @@ static int keystone_platform_notifier(struct notifier_block *nb,
 		return NOTIFY_BAD;
 
 	if (!dev->of_node) {
-		dev->dma_pfn_offset = keystone_dma_pfn_offset;
-		dev_err(dev, "set dma_pfn_offset%08lx\n",
-			dev->dma_pfn_offset);
+		int ret = dma_set_offset_range(dev, KEYSTONE_HIGH_PHYS_START,
+					       KEYSTONE_LOW_PHYS_START,
+					       KEYSTONE_HIGH_PHYS_SIZE);
+		dev_err(dev, "set dma_offset%08llx%s\n",
+			KEYSTONE_HIGH_PHYS_START - KEYSTONE_LOW_PHYS_START,
+			ret ? " failed" : "");
 	}
 	return NOTIFY_OK;
 }
@@ -51,11 +53,8 @@ static struct notifier_block platform_nb = {
 
 static void __init keystone_init(void)
 {
-	if (PHYS_OFFSET >= KEYSTONE_HIGH_PHYS_START) {
-		keystone_dma_pfn_offset = PFN_DOWN(KEYSTONE_HIGH_PHYS_START -
-						   KEYSTONE_LOW_PHYS_START);
+	if (PHYS_OFFSET >= KEYSTONE_HIGH_PHYS_START)
 		bus_register_notifier(&platform_bus_type, &platform_nb);
-	}
 	keystone_pm_runtime_init();
 }
 
diff --git a/arch/sh/drivers/pci/pcie-sh7786.c b/arch/sh/drivers/pci/pcie-sh7786.c
index e0b568aaa701..e929f85c5038 100644
--- a/arch/sh/drivers/pci/pcie-sh7786.c
+++ b/arch/sh/drivers/pci/pcie-sh7786.c
@@ -12,6 +12,7 @@
 #include <linux/io.h>
 #include <linux/async.h>
 #include <linux/delay.h>
+#include <linux/dma-mapping.h>
 #include <linux/slab.h>
 #include <linux/clk.h>
 #include <linux/sh_clk.h>
@@ -31,6 +32,8 @@ struct sh7786_pcie_port {
 static struct sh7786_pcie_port *sh7786_pcie_ports;
 static unsigned int nr_ports;
 static unsigned long dma_pfn_offset;
+size_t memsize;
+u64 memstart;
 
 static struct sh7786_pcie_hwops {
 	int (*core_init)(void);
@@ -301,7 +304,6 @@ static int __init pcie_init(struct sh7786_pcie_port *port)
 	struct pci_channel *chan = port->hose;
 	unsigned int data;
 	phys_addr_t memstart, memend;
-	size_t memsize;
 	int ret, i, win;
 
 	/* Begin initialization */
@@ -368,8 +370,6 @@ static int __init pcie_init(struct sh7786_pcie_port *port)
 	memstart = ALIGN_DOWN(memstart, memsize);
 	memsize = roundup_pow_of_two(memend - memstart);
 
-	dma_pfn_offset = memstart >> PAGE_SHIFT;
-
 	/*
 	 * If there's more than 512MB of memory, we need to roll over to
 	 * LAR1/LAMR1.
@@ -487,7 +487,8 @@ int pcibios_map_platform_irq(const struct pci_dev *pdev, u8 slot, u8 pin)
 
 void pcibios_bus_add_device(struct pci_dev *pdev)
 {
-	pdev->dev.dma_pfn_offset = dma_pfn_offset;
+	dma_set_offset_range(&pdev->dev, __pa(memory_start),
+			     __pa(memory_start) - memstart, memsize);
 }
 
 static int __init sh7786_pcie_core_init(void)
diff --git a/arch/x86/pci/sta2x11-fixup.c b/arch/x86/pci/sta2x11-fixup.c
index c313d784efab..ea3a58323f81 100644
--- a/arch/x86/pci/sta2x11-fixup.c
+++ b/arch/x86/pci/sta2x11-fixup.c
@@ -12,6 +12,7 @@
 #include <linux/export.h>
 #include <linux/list.h>
 #include <linux/dma-direct.h>
+#include <linux/dma-mapping.h>
 #include <asm/iommu.h>
 
 #define STA2X11_SWIOTLB_SIZE (4*1024*1024)
@@ -133,7 +134,7 @@ static void sta2x11_map_ep(struct pci_dev *pdev)
 	struct sta2x11_instance *instance = sta2x11_pdev_to_instance(pdev);
 	struct device *dev = &pdev->dev;
 	u32 amba_base, max_amba_addr;
-	int i;
+	int i, ret;
 
 	if (!instance)
 		return;
@@ -141,7 +142,9 @@ static void sta2x11_map_ep(struct pci_dev *pdev)
 	pci_read_config_dword(pdev, AHB_BASE(0), &amba_base);
 	max_amba_addr = amba_base + STA2X11_AMBA_SIZE - 1;
 
-	dev->dma_pfn_offset = PFN_DOWN(-amba_base);
+	ret = dma_set_offset_range(dev, 0, amba_base, STA2X11_AMBA_SIZE);
+	if (ret)
+		dev_err(dev, "sta2x11: could not set DMA offset\n");
 
 	dev->bus_dma_limit = max_amba_addr;
 	pci_set_consistent_dma_mask(pdev, max_amba_addr);
diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index ec782e4a0fe4..8d99e45395bf 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -1184,8 +1184,9 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *dma_size)
 	*dma_addr = dmaaddr;
 	*dma_size = size;
 
-	dev->dma_pfn_offset = PFN_DOWN(offset);
-	dev_dbg(dev, "dma_pfn_offset(%#08llx)\n", offset);
+	ret = dma_set_offset_range(dev, dmaaddr + offset, dmaaddr, size);
+
+	dev_dbg(dev, "dma_offset(%#08llx)%s\n", offset, ret ? " failed!" : "");
 }
 
 static void __init acpi_iort_register_irq(int hwirq, const char *name,
diff --git a/drivers/base/core.c b/drivers/base/core.c
index ac1046a382bc..548520dc6bee 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -1792,6 +1792,8 @@ static void device_release(struct kobject *kobj)
 	 */
 	devres_release_all(dev);
 
+	kfree(dev->dma_range_map);
+
 	if (dev->release)
 		dev->release(dev);
 	else if (dev->type && dev->type->release)
diff --git a/drivers/gpu/drm/sun4i/sun4i_backend.c b/drivers/gpu/drm/sun4i/sun4i_backend.c
index 072ea113e6be..48a4adf1f04e 100644
--- a/drivers/gpu/drm/sun4i/sun4i_backend.c
+++ b/drivers/gpu/drm/sun4i/sun4i_backend.c
@@ -11,6 +11,7 @@
 #include <linux/module.h>
 #include <linux/of_device.h>
 #include <linux/of_graph.h>
+#include <linux/dma-mapping.h>
 #include <linux/platform_device.h>
 #include <linux/reset.h>
 
@@ -812,7 +813,9 @@ static int sun4i_backend_bind(struct device *dev, struct device *master,
 		 * on our device since the RAM mapping is at 0 for the DMA bus,
 		 * unlike the CPU.
 		 */
-		drm->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+		ret = dma_set_offset_range(drm->dev, PHYS_OFFSET, 0, SZ_4G);
+		if (ret)
+			return ret;
 	}
 
 	backend->engine.node = dev->of_node;
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index dc7bcf858b6d..d77e881516a4 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -751,7 +751,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
 	if (cfg->oas > ARM_LPAE_MAX_ADDR_BITS)
 		return NULL;
 
-	if (!selftest_running && cfg->iommu_dev->dma_pfn_offset) {
+	if (!selftest_running && cfg->iommu_dev->dma_range_map) {
 		dev_err(cfg->iommu_dev, "Cannot accommodate DMA offset for IOMMU page tables\n");
 		return NULL;
 	}
diff --git a/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c b/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c
index 5319eb1ab309..c02782f7efc7 100644
--- a/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c
+++ b/drivers/media/platform/sunxi/sun4i-csi/sun4i_csi.c
@@ -7,6 +7,7 @@
  */
 
 #include <linux/clk.h>
+#include <linux/dma-mapping.h>
 #include <linux/interrupt.h>
 #include <linux/module.h>
 #include <linux/mutex.h>
@@ -183,7 +184,9 @@ static int sun4i_csi_probe(struct platform_device *pdev)
 			return ret;
 	} else {
 #ifdef PHYS_PFN_OFFSET
-		csi->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+		ret = dma_set_offset_range(csi->dev, PHYS_OFFSET, 0, SZ_4G);
+		if (ret)
+			return ret;
 #endif
 	}
 
diff --git a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
index 28e89340fed9..b519d1dd5195 100644
--- a/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
+++ b/drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c
@@ -900,7 +900,9 @@ static int sun6i_csi_probe(struct platform_device *pdev)
 
 	sdev->dev = &pdev->dev;
 	/* The DMA bus has the memory mapped at 0 */
-	sdev->dev->dma_pfn_offset = PHYS_OFFSET >> PAGE_SHIFT;
+	ret = dma_set_offset_range(sdev->dev, PHYS_OFFSET, 0, SZ_4G);
+	if (ret)
+		return ret;
 
 	ret = sun6i_csi_resource_request(sdev, pdev);
 	if (ret)
diff --git a/drivers/of/address.c b/drivers/of/address.c
index 590493e04b01..50d586c5a858 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -923,33 +923,33 @@ void __iomem *of_io_request_and_map(struct device_node *np, int index,
 }
 EXPORT_SYMBOL(of_io_request_and_map);
 
+#ifdef CONFIG_HAS_DMA
 /**
- * of_dma_get_range - Get DMA range info
+ * of_dma_get_range - Get DMA range info and put it into a map array
  * @np:		device node to get DMA range info
- * @dma_addr:	pointer to store initial DMA address of DMA range
- * @paddr:	pointer to store initial CPU address of DMA range
- * @size:	pointer to store size of DMA range
+ * @map:	dma range structure to return
  *
  * Look in bottom up direction for the first "dma-ranges" property
- * and parse it.
- *  dma-ranges format:
+ * and parse it.  Put the information into a DMA offset map array.
+ *
+ * dma-ranges format:
  *	DMA addr (dma_addr)	: naddr cells
  *	CPU addr (phys_addr_t)	: pna cells
  *	size			: nsize cells
  *
- * It returns -ENODEV if "dma-ranges" property was not found
- * for this device in DT.
+ * It returns -ENODEV if "dma-ranges" property was not found for this
+ * device in the DT.
  */
-int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size)
+int of_dma_get_range(struct device_node *np, const struct bus_dma_region **map)
 {
 	struct device_node *node = of_node_get(np);
 	const __be32 *ranges = NULL;
-	int len;
-	int ret = 0;
 	bool found_dma_ranges = false;
 	struct of_range_parser parser;
 	struct of_range range;
-	u64 dma_start = U64_MAX, dma_end = 0, dma_offset = 0;
+	struct bus_dma_region *r;
+	int len, num_ranges = 0;
+	int ret = 0;
 
 	while (node) {
 		ranges = of_get_property(node, "dma-ranges", &len);
@@ -975,44 +975,36 @@ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *siz
 	}
 
 	of_dma_range_parser_init(&parser, node);
+	for_each_of_range(&parser, &range)
+		num_ranges++;
 
-	for_each_of_range(&parser, &range) {
-		pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
-			 range.bus_addr, range.cpu_addr, range.size);
-
-		if (dma_offset && range.cpu_addr - range.bus_addr != dma_offset) {
-			pr_warn("Can't handle multiple dma-ranges with different offsets on node(%pOF)\n", node);
-			/* Don't error out as we'd break some existing DTs */
-			continue;
-		}
-		dma_offset = range.cpu_addr - range.bus_addr;
-
-		/* Take lower and upper limits */
-		if (range.bus_addr < dma_start)
-			dma_start = range.bus_addr;
-		if (range.bus_addr + range.size > dma_end)
-			dma_end = range.bus_addr + range.size;
-	}
+	of_dma_range_parser_init(&parser, node);
 
-	if (dma_start >= dma_end) {
-		ret = -EINVAL;
-		pr_debug("Invalid DMA ranges configuration on node(%pOF)\n",
-			 node);
+	r = kcalloc(num_ranges + 1, sizeof(*r), GFP_KERNEL);
+	if (!r) {
+		ret = -ENOMEM;
 		goto out;
 	}
 
-	*dma_addr = dma_start;
-	*size = dma_end - dma_start;
-	*paddr = dma_start + dma_offset;
-
-	pr_debug("final: dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
-		 *dma_addr, *paddr, *size);
+	/*
+	 * Record all info in the generic DMA ranges array for struct device.
+	 */
+	*map = r;
+	for_each_of_range(&parser, &range) {
+		pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
+			 range.bus_addr, range.cpu_addr, range.size);
+		r->cpu_start = range.cpu_addr;
+		r->dma_start = range.bus_addr;
+		r->size = range.size;
+		r->offset = (u64)range.cpu_addr - (u64)range.bus_addr;
+		r++;
+	}
 
 out:
 	of_node_put(node);
-
 	return ret;
 }
+#endif
 
 /**
  * of_dma_is_coherent - Check if device is coherent
diff --git a/drivers/of/device.c b/drivers/of/device.c
index b439c1e05434..cd668e7ea7da 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -90,14 +90,14 @@ int of_device_add(struct platform_device *ofdev)
 int of_dma_configure_id(struct device *dev, struct device_node *np,
 			bool force_dma, const u32 *id)
 {
-	u64 dma_addr, paddr, size = 0;
-	int ret;
-	bool coherent;
-	unsigned long offset;
 	const struct iommu_ops *iommu;
-	u64 mask, end;
+	const struct bus_dma_region *map = NULL;
+	dma_addr_t dma_start = 0;
+	u64 mask, end, size = 0;
+	bool coherent;
+	int ret;
 
-	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
+	ret = of_dma_get_range(np, &map);
 	if (ret < 0) {
 		/*
 		 * For legacy reasons, we have to assume some devices need
@@ -106,26 +106,34 @@ int of_dma_configure_id(struct device *dev, struct device_node *np,
 		 */
 		if (!force_dma)
 			return ret == -ENODEV ? 0 : ret;
-
-		dma_addr = offset = 0;
 	} else {
-		offset = PFN_DOWN(paddr - dma_addr);
+		const struct bus_dma_region *r = map;
+		dma_addr_t dma_end = 0;
+
+		/* Determine the overall bounds of all DMA regions */
+		for (dma_start = ~(dma_addr_t)0; r->size; r++) {
+			/* Take lower and upper limits */
+			if (r->dma_start < dma_start)
+				dma_start = r->dma_start;
+			if (r->dma_start + r->size > dma_end)
+				dma_end = r->dma_start + r->size;
+		}
+		size = dma_end - dma_start;
 
 		/*
 		 * Add a work around to treat the size as mask + 1 in case
 		 * it is defined in DT as a mask.
 		 */
 		if (size & 1) {
-			dev_warn(dev, "Invalid size 0x%llx for dma-range\n",
-				 size);
+			dev_warn(dev, "Invalid size 0x%llx for dma-range(s)\n", size);
 			size = size + 1;
 		}
 
 		if (!size) {
 			dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
+			kfree(map);
 			return -EINVAL;
 		}
-		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
 	}
 
 	/*
@@ -144,13 +152,11 @@ int of_dma_configure_id(struct device *dev, struct device_node *np,
 	else if (!size)
 		size = 1ULL << 32;
 
-	dev->dma_pfn_offset = offset;
-
 	/*
 	 * Limit coherent and dma mask based on size and default mask
 	 * set by the driver.
 	 */
-	end = dma_addr + size - 1;
+	end = dma_start + size - 1;
 	mask = DMA_BIT_MASK(ilog2(end) + 1);
 	dev->coherent_dma_mask &= mask;
 	*dev->dma_mask &= mask;
@@ -163,14 +169,17 @@ int of_dma_configure_id(struct device *dev, struct device_node *np,
 		coherent ? " " : " not ");
 
 	iommu = of_iommu_configure(dev, np, id);
-	if (PTR_ERR(iommu) == -EPROBE_DEFER)
+	if (PTR_ERR(iommu) == -EPROBE_DEFER) {
+		kfree(map);
 		return -EPROBE_DEFER;
+	}
 
 	dev_dbg(dev, "device is%sbehind an iommu\n",
 		iommu ? " " : " not ");
 
-	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
+	arch_setup_dma_ops(dev, dma_start, size, iommu, coherent);
 
+	dev->dma_range_map = map;
 	return 0;
 }
 EXPORT_SYMBOL_GPL(of_dma_configure_id);
diff --git a/drivers/of/of_private.h b/drivers/of/of_private.h
index edc682249c00..5c13485a4ad1 100644
--- a/drivers/of/of_private.h
+++ b/drivers/of/of_private.h
@@ -157,12 +157,12 @@ extern void __of_sysfs_remove_bin_file(struct device_node *np,
 extern int of_bus_n_addr_cells(struct device_node *np);
 extern int of_bus_n_size_cells(struct device_node *np);
 
-#ifdef CONFIG_OF_ADDRESS
-extern int of_dma_get_range(struct device_node *np, u64 *dma_addr,
-			    u64 *paddr, u64 *size);
+struct bus_dma_region;
+#if defined(CONFIG_OF_ADDRESS) && defined(CONFIG_HAS_DMA)
+int of_dma_get_range(struct device_node *np, const struct bus_dma_region **map);
 #else
-static inline int of_dma_get_range(struct device_node *np, u64 *dma_addr,
-				   u64 *paddr, u64 *size)
+static inline int of_dma_get_range(struct device_node *np,
+		const struct bus_dma_region **map)
 {
 	return -ENODEV;
 }
diff --git a/drivers/of/unittest.c b/drivers/of/unittest.c
index 9b7e84bdc7d4..dea68877f3b4 100644
--- a/drivers/of/unittest.c
+++ b/drivers/of/unittest.c
@@ -7,6 +7,7 @@
 
 #include <linux/memblock.h>
 #include <linux/clk.h>
+#include <linux/dma-mapping.h>
 #include <linux/err.h>
 #include <linux/errno.h>
 #include <linux/hashtable.h>
@@ -869,10 +870,10 @@ static void __init of_unittest_changeset(void)
 }
 
 static void __init of_unittest_dma_ranges_one(const char *path,
-		u64 expect_dma_addr, u64 expect_paddr, u64 expect_size)
+		u64 expect_dma_addr, u64 expect_paddr)
 {
 	struct device_node *np;
-	u64 dma_addr, paddr, size;
+	const struct bus_dma_region *map = NULL;
 	int rc;
 
 	np = of_find_node_by_path(path);
@@ -881,16 +882,29 @@ static void __init of_unittest_dma_ranges_one(const char *path,
 		return;
 	}
 
-	rc = of_dma_get_range(np, &dma_addr, &paddr, &size);
+	rc = of_dma_get_range(np, &map);
 
 	unittest(!rc, "of_dma_get_range failed on node %pOF rc=%i\n", np, rc);
+
 	if (!rc) {
-		unittest(size == expect_size,
-			 "of_dma_get_range wrong size on node %pOF size=%llx\n", np, size);
+		phys_addr_t	paddr;
+		dma_addr_t	dma_addr;
+		struct device	dev_bogus;
+
+		dev_bogus.dma_range_map = map;
+		paddr = (phys_addr_t)expect_dma_addr +
+			dma_offset_from_dma_addr(&dev_bogus, expect_dma_addr);
+		dma_addr = (dma_addr_t)expect_paddr -
+			dma_offset_from_phys_addr(&dev_bogus, expect_paddr);
+
 		unittest(paddr == expect_paddr,
-			 "of_dma_get_range wrong phys addr (%llx) on node %pOF", paddr, np);
+			 "of_dma_get_range: wrong phys addr %llx (expecting %llx) on node %pOF\n",
+			 (u64)paddr, expect_paddr, np);
 		unittest(dma_addr == expect_dma_addr,
-			 "of_dma_get_range wrong DMA addr (%llx) on node %pOF", dma_addr, np);
+			 "of_dma_get_range: wrong DMA addr %llx (expecting %llx) on node %pOF\n",
+			 (u64)dma_addr, expect_dma_addr, np);
+
+		kfree(map);
 	}
 	of_node_put(np);
 }
@@ -898,11 +912,11 @@ static void __init of_unittest_dma_ranges_one(const char *path,
 static void __init of_unittest_parse_dma_ranges(void)
 {
 	of_unittest_dma_ranges_one("/testcase-data/address-tests/device@70000000",
-		0x0, 0x20000000, 0x40000000);
+		0x0, 0x20000000);
 	of_unittest_dma_ranges_one("/testcase-data/address-tests/bus@80000000/device@1000",
-		0x100000000, 0x20000000, 0x2000000000);
+		0x100000000, 0x20000000);
 	of_unittest_dma_ranges_one("/testcase-data/address-tests/pci@90000000",
-		0x80000000, 0x20000000, 0x10000000);
+		0x80000000, 0x20000000);
 }
 
 static void __init of_unittest_pci_dma_ranges(void)
diff --git a/drivers/remoteproc/remoteproc_core.c b/drivers/remoteproc/remoteproc_core.c
index 7f90eeea67e2..0946161866e0 100644
--- a/drivers/remoteproc/remoteproc_core.c
+++ b/drivers/remoteproc/remoteproc_core.c
@@ -529,7 +529,13 @@ static int rproc_handle_vdev(struct rproc *rproc, struct fw_rsc_vdev *rsc,
 	/* Initialise vdev subdevice */
 	snprintf(name, sizeof(name), "vdev%dbuffer", rvdev->index);
 	rvdev->dev.parent = &rproc->dev;
-	rvdev->dev.dma_pfn_offset = rproc->dev.parent->dma_pfn_offset;
+	if (rproc->dev.parent->dma_range_map) {
+		const struct bus_dma_region *map =
+			dma_copy_dma_range_map(rproc->dev.parent->dma_range_map);
+		if (!map)
+			return -ENOMEM;
+		rvdev->dev.dma_range_map = map;
+	}
 	rvdev->dev.release = rproc_rvdev_release;
 	dev_set_name(&rvdev->dev, "%s#%s", dev_name(rvdev->dev.parent), name);
 	dev_set_drvdata(&rvdev->dev, rvdev);
diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
index 1744e6fcc999..249e4bddaa40 100644
--- a/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
+++ b/drivers/staging/media/sunxi/cedrus/cedrus_hw.c
@@ -230,8 +230,11 @@ int cedrus_hw_probe(struct cedrus_dev *dev)
 	 */
 
 #ifdef PHYS_PFN_OFFSET
-	if (!(variant->quirks & CEDRUS_QUIRK_NO_DMA_OFFSET))
-		dev->dev->dma_pfn_offset = PHYS_PFN_OFFSET;
+	if (!(variant->quirks & CEDRUS_QUIRK_NO_DMA_OFFSET)) {
+		ret = dma_set_offset_range(dev->dev, PHYS_OFFSET, 0, SZ_4G);
+		if (ret)
+			return ret;
+	}
 #endif
 
 	ret = of_reserved_mem_device_init(dev->dev);
diff --git a/drivers/usb/core/message.c b/drivers/usb/core/message.c
index 6197938dcc2d..c6ccc7b04a7b 100644
--- a/drivers/usb/core/message.c
+++ b/drivers/usb/core/message.c
@@ -1956,10 +1956,15 @@ int usb_set_configuration(struct usb_device *dev, int configuration)
 		intf->dev.groups = usb_interface_groups;
 		/*
 		 * Please refer to usb_alloc_dev() to see why we set
-		 * dma_mask and dma_pfn_offset.
+		 * dma_mask and dma_range_map.
 		 */
 		intf->dev.dma_mask = dev->dev.dma_mask;
-		intf->dev.dma_pfn_offset = dev->dev.dma_pfn_offset;
+		if (dev->dev.dma_range_map) {
+			intf->dev.dma_range_map =
+				dma_copy_dma_range_map(dev->dev.dma_range_map);
+			if (!intf->dev.dma_range_map)
+				dev_err(&dev->dev, "failed to copy DMA map\n");
+		}
 		INIT_WORK(&intf->reset_ws, __usb_queue_reset_device);
 		intf->minor = -1;
 		device_initialize(&intf->dev);
diff --git a/drivers/usb/core/usb.c b/drivers/usb/core/usb.c
index bafc113f2b3e..6e6966704b63 100644
--- a/drivers/usb/core/usb.c
+++ b/drivers/usb/core/usb.c
@@ -610,7 +610,12 @@ struct usb_device *usb_alloc_dev(struct usb_device *parent,
 	 * mask for the entire HCD, so don't do that.
 	 */
 	dev->dev.dma_mask = bus->sysdev->dma_mask;
-	dev->dev.dma_pfn_offset = bus->sysdev->dma_pfn_offset;
+	if (bus->sysdev->dma_range_map) {
+		dev->dev.dma_range_map =
+			dma_copy_dma_range_map(bus->sysdev->dma_range_map);
+		if (!dev->dev.dma_range_map)
+			dev_err(&dev->dev, "failed to copy DMA map\n");
+	}
 	set_dev_node(&dev->dev, dev_to_node(bus->sysdev));
 	dev->state = USB_STATE_ATTACHED;
 	dev->lpm_disable_count = 1;
diff --git a/include/linux/device.h b/include/linux/device.h
index ca18da4768e3..1c78621fc3c0 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -466,7 +466,7 @@ struct dev_links_info {
  * 		such descriptors.
  * @bus_dma_limit: Limit of an upstream bridge or bus which imposes a smaller
  *		DMA limit than the device itself supports.
- * @dma_pfn_offset: offset of DMA memory range relatively of RAM
+ * @dma_range_map: map for DMA memory ranges relative to that of RAM
  * @dma_parms:	A low level driver may set these to teach IOMMU code about
  * 		segment limitations.
  * @dma_pools:	Dma pools (if dma'ble device).
@@ -561,7 +561,7 @@ struct device {
 					     64 bit addresses for consistent
 					     allocations such descriptors. */
 	u64		bus_dma_limit;	/* upstream dma constraint */
-	unsigned long	dma_pfn_offset;
+	const struct bus_dma_region *dma_range_map;
 
 	struct device_dma_parameters *dma_parms;
 
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index 5a3ce2a24794..71acdc3ea87d 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -19,16 +19,12 @@ extern unsigned int zone_dma_bits;
 #else
 static inline dma_addr_t __phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	dma_addr_t dev_addr = (dma_addr_t)paddr;
-
-	return dev_addr - ((dma_addr_t)dev->dma_pfn_offset << PAGE_SHIFT);
+	return (dma_addr_t)paddr - dma_offset_from_phys_addr(dev, paddr);
 }
 
 static inline phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dev_addr)
 {
-	phys_addr_t paddr = (phys_addr_t)dev_addr;
-
-	return paddr + ((phys_addr_t)dev->dma_pfn_offset << PAGE_SHIFT);
+	return (phys_addr_t)dev_addr + dma_offset_from_dma_addr(dev, dev_addr);
 }
 #endif /* !CONFIG_ARCH_HAS_PHYS_TO_DMA */
 
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 016b96b384bd..ebe0b2f419da 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -188,7 +188,38 @@ static inline int dma_mmap_from_global_coherent(struct vm_area_struct *vma,
 }
 #endif /* CONFIG_DMA_DECLARE_COHERENT */
 
+struct bus_dma_region {
+	phys_addr_t	cpu_start;
+	dma_addr_t	dma_start;
+	u64		size;
+	u64		offset;
+};
+
 #ifdef CONFIG_HAS_DMA
+static inline u64 dma_offset_from_dma_addr(struct device *dev, dma_addr_t dma_addr)
+{
+	const struct bus_dma_region *m = dev->dma_range_map;
+
+	if (m)
+		for (; m->size; m++)
+			if (dma_addr >= m->dma_start &&
+			    dma_addr - m->dma_start < m->size)
+				return m->offset;
+	return 0;
+}
+
+static inline u64 dma_offset_from_phys_addr(struct device *dev, phys_addr_t paddr)
+{
+	const struct bus_dma_region *m = dev->dma_range_map;
+
+	if (m)
+		for (; m->size; m++)
+			if (paddr >= m->cpu_start &&
+			    paddr - m->cpu_start < m->size)
+				return m->offset;
+	return 0;
+}
+
 #include <asm/dma-mapping.h>
 
 #ifdef CONFIG_DMA_OPS
@@ -608,6 +639,11 @@ static inline void arch_teardown_dma_ops(struct device *dev)
 }
 #endif /* CONFIG_ARCH_HAS_TEARDOWN_DMA_OPS */
 
+int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
+		dma_addr_t dma_start, u64 size);
+
+void *dma_copy_dma_range_map(const struct bus_dma_region *map);
+
 static inline unsigned int dma_get_max_seg_size(struct device *dev)
 {
 	if (dev->dma_parms && dev->dma_parms->max_segment_size)
diff --git a/kernel/dma/coherent.c b/kernel/dma/coherent.c
index 2a0c4985f38e..5d0d72b3bf0f 100644
--- a/kernel/dma/coherent.c
+++ b/kernel/dma/coherent.c
@@ -31,10 +31,12 @@ static inline struct dma_coherent_mem *dev_get_coherent_memory(struct device *de
 static inline dma_addr_t dma_get_device_base(struct device *dev,
 					     struct dma_coherent_mem * mem)
 {
-	if (mem->use_dev_dma_pfn_offset)
-		return (mem->pfn_base - dev->dma_pfn_offset) << PAGE_SHIFT;
-	else
-		return mem->device_base;
+	if (mem->use_dev_dma_pfn_offset) {
+		u64 base_addr = PFN_PHYS((u64)mem->pfn_base);
+
+		return base_addr - dma_offset_from_phys_addr(dev, base_addr);
+	}
+	return mem->device_base;
 }
 
 static int dma_init_coherent_memory(phys_addr_t phys_addr,
diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c
index 0d129421e75f..5576bb2fde20 100644
--- a/kernel/dma/mapping.c
+++ b/kernel/dma/mapping.c
@@ -11,6 +11,7 @@
 #include <linux/dma-noncoherent.h>
 #include <linux/export.h>
 #include <linux/gfp.h>
+#include <linux/limits.h>
 #include <linux/of_device.h>
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
@@ -611,3 +612,68 @@ unsigned long dma_get_merge_boundary(struct device *dev)
 	return ops->get_merge_boundary(dev);
 }
 EXPORT_SYMBOL_GPL(dma_get_merge_boundary);
+
+/**
+ * dma_set_offset_range - Assign scalar offset for a single DMA range.
+ * @dev:	device pointer; needed to "own" the alloced memory.
+ * @cpu_start:  beginning of memory region covered by this offset.
+ * @dma_start:  beginning of DMA/PCI region covered by this offset.
+ * @size:	size of the region.
+ *
+ * This is for the simple case of a uniform offset which cannot
+ * be discovered by "dma-ranges".
+ *
+ * It returns -ENOMEM if out of memory, -EINVAL if a map
+ * already exists, 0 otherwise.
+ */
+int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
+			 dma_addr_t dma_start, u64 size)
+{
+	struct bus_dma_region *map;
+	u64 offset = (u64)cpu_start - (u64)dma_start;
+
+	if (dev->dma_range_map) {
+		dev_err(dev, "attempt to add DMA range to existing map\n");
+		return -EINVAL;
+	}
+
+	if (!offset)
+		return 0;
+
+	map = kcalloc(2, sizeof(*map), GFP_KERNEL);
+	if (!map)
+		return -ENOMEM;
+	map[0].cpu_start = cpu_start;
+	map[0].dma_start = dma_start;
+	map[0].offset = offset;
+	map[0].size = size;
+	dev->dma_range_map = map;
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(dma_set_offset_range);
+
+/**
+ * dma_copy_dma_range_map - Copy the dma_range map using kcalloc.
+ * @map:	source to copy.
+ *
+ * A dma range map array consists of N + 1 elements; N acutal ranges
+ * and a zero element trailer.
+ *
+ * It returns NULL if the kcalloc fails.
+ */
+void *dma_copy_dma_range_map(const struct bus_dma_region *map)
+{
+	int num_ranges;
+	struct bus_dma_region *new_map;
+	const struct bus_dma_region *r = map;
+
+	for (num_ranges = 0; r->size; num_ranges++)
+		r++;
+	new_map = kcalloc(num_ranges + 1, sizeof(*map), GFP_KERNEL);
+	if (new_map)
+		memcpy(new_map, map, sizeof(*map) * num_ranges);
+
+	return new_map;
+}
+EXPORT_SYMBOL_GPL(dma_copy_dma_range_map);
-- 
2.17.1

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

[PATCH v11 08/11] PCI: brcmstb: Set additional internal memory DMA viewport sizes

[Jim Quinlan]

The Raspberry Pi (RPI) is currently the only chip using this driver
(pcie-brcmstb.c).  There, only one memory controller is used, without an
extension region, and the SCB0 viewport size is set to the size of the
first and only dma-range region.  Other BrcmSTB SOCs have more complicated
memory configurations that require setting additional viewport sizes.

BrcmSTB PCIe controllers are intimately connected to the memory
controller(s) on the SOC.  The SOC may have one to three memory
controllers; they are indicated by the term SCBi.  Each controller has a
base region and an optional extension region.  In physical memory, the base
and extension regions of a controller are not adjacent, but in PCIe-space
they are.

There is a "viewport" for each memory controller that allows DMA from
endpoint devices.  Each viewport's size must be set to a power of two, and
that size must be equal to or larger than the amount of memory each
controller supports which is the sum of base region and its optional
extension.  Further, the 1-3 viewports are also adjacent in PCIe-space.

Unfortunately the viewport sizes cannot be ascertained from the
"dma-ranges" property so they have their own property, "brcm,scb-sizes".
This is because dma-range information does not indicate what memory
controller it is associated.  For example, consider the following case
where the size of one dma-range is 2GB and the second dma-range is 1GB:

    /* Case 1: SCB0 size set to 4GB */
    dma-range0: 2GB (from memc0-base)
    dma-range1: 1GB (from memc0-extension)

    /* Case 2: SCB0 size set to 2GB, SCB1 size set to 1GB */
    dma-range0: 2GB (from memc0-base)
    dma-range1: 1GB (from memc0-extension)

By just looking at the dma-ranges information, one cannot tell which
situation applies. That is why an additional property is needed.  Its
length indicates the number of memory controllers being used and each value
indicates the viewport size.

Note that the RPI DT does not have a "brcm,scb-sizes" property value,
as it is assumed that it only requires one memory controller and no
extension.  So the optional use of "brcm,scb-sizes" will be backwards
compatible.

One last layer of complexity exists: all of the viewports sizes must be
added and rounded up to a power of two to determine what the "BAR" size is.
Further, an offset must be given that indicates the base PCIe address of
this "BAR".  The use of the term BAR is typically associated with endpoint
devices, and the term is used here because the PCIe HW may be used as an RC
or an EP.  In the former case, all of the system memory appears in a single
"BAR" region in PCIe memory.  As it turns out, BrcmSTB PCIe HW is rarely
used in the EP role and its system of mapping memory is an artifact that
requires multiple dma-ranges regions.

Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++-------
 1 file changed, 50 insertions(+), 18 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 041b8d109563..7150eaa803c2 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -57,6 +57,8 @@
 #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK	0x300000
 #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128		0x0
 #define  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK		0xf8000000
+#define  PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK		0x07c00000
+#define  PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK		0x0000001f
 
 #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO		0x400c
 #define PCIE_MEM_WIN0_LO(win)	\
@@ -154,6 +156,7 @@
 #define SSC_STATUS_OFFSET		0x1
 #define SSC_STATUS_SSC_MASK		0x400
 #define SSC_STATUS_PLL_LOCK_MASK	0x800
+#define PCIE_BRCM_MAX_MEMC		3
 
 #define IDX_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_INDEX])
 #define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
@@ -259,6 +262,8 @@ struct brcm_pcie {
 	const int		*reg_field_info;
 	enum pcie_type		type;
 	struct reset_control	*rescal;
+	int			num_memc;
+	u64			memc_size[PCIE_BRCM_MAX_MEMC];
 };
 
 /*
@@ -714,22 +719,44 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
 							u64 *rc_bar2_offset)
 {
 	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
-	struct device *dev = pcie->dev;
 	struct resource_entry *entry;
+	struct device *dev = pcie->dev;
+	u64 lowest_pcie_addr = ~(u64)0;
+	int ret, i = 0;
+	u64 size = 0;
 
-	entry = resource_list_first_type(&bridge->dma_ranges, IORESOURCE_MEM);
-	if (!entry)
-		return -ENODEV;
+	resource_list_for_each_entry(entry, &bridge->dma_ranges) {
+		u64 pcie_beg = entry->res->start - entry->offset;
 
+		size += entry->res->end - entry->res->start + 1;
+		if (pcie_beg < lowest_pcie_addr)
+			lowest_pcie_addr = pcie_beg;
+	}
 
-	/*
-	 * The controller expects the inbound window offset to be calculated as
-	 * the difference between PCIe's address space and CPU's. The offset
-	 * provided by the firmware is calculated the opposite way, so we
-	 * negate it.
-	 */
-	*rc_bar2_offset = -entry->offset;
-	*rc_bar2_size = 1ULL << fls64(entry->res->end - entry->res->start);
+	if (lowest_pcie_addr == ~(u64)0) {
+		dev_err(dev, "DT node has no dma-ranges\n");
+		return -EINVAL;
+	}
+
+	ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1,
+						  PCIE_BRCM_MAX_MEMC);
+
+	if (ret <= 0) {
+		/* Make an educated guess */
+		pcie->num_memc = 1;
+		pcie->memc_size[0] = 1ULL << fls64(size - 1);
+	} else {
+		pcie->num_memc = ret;
+	}
+
+	/* Each memc is viewed through a "port" that is a power of 2 */
+	for (i = 0, size = 0; i < pcie->num_memc; i++)
+		size += pcie->memc_size[i];
+
+	/* System memory starts at this address in PCIe-space */
+	*rc_bar2_offset = lowest_pcie_addr;
+	/* The sum of all memc views must also be a power of 2 */
+	*rc_bar2_size = 1ULL << fls64(size - 1);
 
 	/*
 	 * We validate the inbound memory view even though we should trust
@@ -781,12 +808,11 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 	void __iomem *base = pcie->base;
 	struct device *dev = pcie->dev;
 	struct resource_entry *entry;
-	unsigned int scb_size_val;
 	bool ssc_good = false;
 	struct resource *res;
 	int num_out_wins = 0;
 	u16 nlw, cls, lnksta;
-	int i, ret;
+	int i, ret, memc;
 	u32 tmp, aspm_support;
 
 	/* Reset the bridge */
@@ -826,11 +852,17 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 	writel(upper_32_bits(rc_bar2_offset),
 	       base + PCIE_MISC_RC_BAR2_CONFIG_HI);
 
-	scb_size_val = rc_bar2_size ?
-		       ilog2(rc_bar2_size) - 15 : 0xf; /* 0xf is 1GB */
 	tmp = readl(base + PCIE_MISC_MISC_CTRL);
-	u32p_replace_bits(&tmp, scb_size_val,
-			  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
+	for (memc = 0; memc < pcie->num_memc; memc++) {
+		u32 scb_size_val = ilog2(pcie->memc_size[memc]) - 15;
+
+		if (memc == 0)
+			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
+		else if (memc == 1)
+			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK);
+		else if (memc == 2)
+			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK);
+	}
 	writel(tmp, base + PCIE_MISC_MISC_CTRL);
 
 	/*
-- 
2.17.1

[PATCH v11 08/11] PCI: brcmstb: Set additional internal memory DMA viewport sizes

[Jim Quinlan]

The Raspberry Pi (RPI) is currently the only chip using this driver
(pcie-brcmstb.c).  There, only one memory controller is used, without an
extension region, and the SCB0 viewport size is set to the size of the
first and only dma-range region.  Other BrcmSTB SOCs have more complicated
memory configurations that require setting additional viewport sizes.

BrcmSTB PCIe controllers are intimately connected to the memory
controller(s) on the SOC.  The SOC may have one to three memory
controllers; they are indicated by the term SCBi.  Each controller has a
base region and an optional extension region.  In physical memory, the base
and extension regions of a controller are not adjacent, but in PCIe-space
they are.

There is a "viewport" for each memory controller that allows DMA from
endpoint devices.  Each viewport's size must be set to a power of two, and
that size must be equal to or larger than the amount of memory each
controller supports which is the sum of base region and its optional
extension.  Further, the 1-3 viewports are also adjacent in PCIe-space.

Unfortunately the viewport sizes cannot be ascertained from the
"dma-ranges" property so they have their own property, "brcm,scb-sizes".
This is because dma-range information does not indicate what memory
controller it is associated.  For example, consider the following case
where the size of one dma-range is 2GB and the second dma-range is 1GB:

    /* Case 1: SCB0 size set to 4GB */
    dma-range0: 2GB (from memc0-base)
    dma-range1: 1GB (from memc0-extension)

    /* Case 2: SCB0 size set to 2GB, SCB1 size set to 1GB */
    dma-range0: 2GB (from memc0-base)
    dma-range1: 1GB (from memc0-extension)

By just looking at the dma-ranges information, one cannot tell which
situation applies. That is why an additional property is needed.  Its
length indicates the number of memory controllers being used and each value
indicates the viewport size.

Note that the RPI DT does not have a "brcm,scb-sizes" property value,
as it is assumed that it only requires one memory controller and no
extension.  So the optional use of "brcm,scb-sizes" will be backwards
compatible.

One last layer of complexity exists: all of the viewports sizes must be
added and rounded up to a power of two to determine what the "BAR" size is.
Further, an offset must be given that indicates the base PCIe address of
this "BAR".  The use of the term BAR is typically associated with endpoint
devices, and the term is used here because the PCIe HW may be used as an RC
or an EP.  In the former case, all of the system memory appears in a single
"BAR" region in PCIe memory.  As it turns out, BrcmSTB PCIe HW is rarely
used in the EP role and its system of mapping memory is an artifact that
requires multiple dma-ranges regions.

Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++-------
 1 file changed, 50 insertions(+), 18 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 041b8d109563..7150eaa803c2 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -57,6 +57,8 @@
 #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK	0x300000
 #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128		0x0
 #define  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK		0xf8000000
+#define  PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK		0x07c00000
+#define  PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK		0x0000001f
 
 #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO		0x400c
 #define PCIE_MEM_WIN0_LO(win)	\
@@ -154,6 +156,7 @@
 #define SSC_STATUS_OFFSET		0x1
 #define SSC_STATUS_SSC_MASK		0x400
 #define SSC_STATUS_PLL_LOCK_MASK	0x800
+#define PCIE_BRCM_MAX_MEMC		3
 
 #define IDX_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_INDEX])
 #define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
@@ -259,6 +262,8 @@ struct brcm_pcie {
 	const int		*reg_field_info;
 	enum pcie_type		type;
 	struct reset_control	*rescal;
+	int			num_memc;
+	u64			memc_size[PCIE_BRCM_MAX_MEMC];
 };
 
 /*
@@ -714,22 +719,44 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
 							u64 *rc_bar2_offset)
 {
 	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
-	struct device *dev = pcie->dev;
 	struct resource_entry *entry;
+	struct device *dev = pcie->dev;
+	u64 lowest_pcie_addr = ~(u64)0;
+	int ret, i = 0;
+	u64 size = 0;
 
-	entry = resource_list_first_type(&bridge->dma_ranges, IORESOURCE_MEM);
-	if (!entry)
-		return -ENODEV;
+	resource_list_for_each_entry(entry, &bridge->dma_ranges) {
+		u64 pcie_beg = entry->res->start - entry->offset;
 
+		size += entry->res->end - entry->res->start + 1;
+		if (pcie_beg < lowest_pcie_addr)
+			lowest_pcie_addr = pcie_beg;
+	}
 
-	/*
-	 * The controller expects the inbound window offset to be calculated as
-	 * the difference between PCIe's address space and CPU's. The offset
-	 * provided by the firmware is calculated the opposite way, so we
-	 * negate it.
-	 */
-	*rc_bar2_offset = -entry->offset;
-	*rc_bar2_size = 1ULL << fls64(entry->res->end - entry->res->start);
+	if (lowest_pcie_addr == ~(u64)0) {
+		dev_err(dev, "DT node has no dma-ranges\n");
+		return -EINVAL;
+	}
+
+	ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1,
+						  PCIE_BRCM_MAX_MEMC);
+
+	if (ret <= 0) {
+		/* Make an educated guess */
+		pcie->num_memc = 1;
+		pcie->memc_size[0] = 1ULL << fls64(size - 1);
+	} else {
+		pcie->num_memc = ret;
+	}
+
+	/* Each memc is viewed through a "port" that is a power of 2 */
+	for (i = 0, size = 0; i < pcie->num_memc; i++)
+		size += pcie->memc_size[i];
+
+	/* System memory starts at this address in PCIe-space */
+	*rc_bar2_offset = lowest_pcie_addr;
+	/* The sum of all memc views must also be a power of 2 */
+	*rc_bar2_size = 1ULL << fls64(size - 1);
 
 	/*
 	 * We validate the inbound memory view even though we should trust
@@ -781,12 +808,11 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 	void __iomem *base = pcie->base;
 	struct device *dev = pcie->dev;
 	struct resource_entry *entry;
-	unsigned int scb_size_val;
 	bool ssc_good = false;
 	struct resource *res;
 	int num_out_wins = 0;
 	u16 nlw, cls, lnksta;
-	int i, ret;
+	int i, ret, memc;
 	u32 tmp, aspm_support;
 
 	/* Reset the bridge */
@@ -826,11 +852,17 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 	writel(upper_32_bits(rc_bar2_offset),
 	       base + PCIE_MISC_RC_BAR2_CONFIG_HI);
 
-	scb_size_val = rc_bar2_size ?
-		       ilog2(rc_bar2_size) - 15 : 0xf; /* 0xf is 1GB */
 	tmp = readl(base + PCIE_MISC_MISC_CTRL);
-	u32p_replace_bits(&tmp, scb_size_val,
-			  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
+	for (memc = 0; memc < pcie->num_memc; memc++) {
+		u32 scb_size_val = ilog2(pcie->memc_size[memc]) - 15;
+
+		if (memc == 0)
+			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
+		else if (memc == 1)
+			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK);
+		else if (memc == 2)
+			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK);
+	}
 	writel(tmp, base + PCIE_MISC_MISC_CTRL);
 
 	/*
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v11 09/11] PCI: brcmstb: Accommodate MSI for older chips

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

Older BrcmSTB chips do not have a separate register for MSI interrupts; the
MSIs are in a register that also contains unrelated interrupts.  In
addition, the interrupts lie in bits [31..24] for these legacy chips.  This
commit provides common code for both legacy and non-legacy MSI interrupt
registers.

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 71 +++++++++++++++++++--------
 1 file changed, 50 insertions(+), 21 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 7150eaa803c2..61c1be766522 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -82,7 +82,8 @@
 #define PCIE_MISC_MSI_BAR_CONFIG_HI			0x4048
 
 #define PCIE_MISC_MSI_DATA_CONFIG			0x404c
-#define  PCIE_MISC_MSI_DATA_CONFIG_VAL			0xffe06540
+#define  PCIE_MISC_MSI_DATA_CONFIG_VAL_32		0xffe06540
+#define  PCIE_MISC_MSI_DATA_CONFIG_VAL_8		0xfff86540
 
 #define PCIE_MISC_PCIE_CTRL				0x4064
 #define  PCIE_MISC_PCIE_CTRL_PCIE_L23_REQUEST_MASK	0x1
@@ -94,6 +95,9 @@
 #define  PCIE_MISC_PCIE_STATUS_PCIE_PHYLINKUP_MASK	0x10
 #define  PCIE_MISC_PCIE_STATUS_PCIE_LINK_IN_L23_MASK	0x40
 
+#define PCIE_MISC_REVISION				0x406c
+#define  BRCM_PCIE_HW_REV_33				0x0303
+
 #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT		0x4070
 #define  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT_LIMIT_MASK	0xfff00000
 #define  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT_BASE_MASK	0xfff0
@@ -114,10 +118,14 @@
 #define  PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK	0x2
 #define  PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK		0x08000000
 
-#define PCIE_MSI_INTR2_STATUS				0x4500
-#define PCIE_MSI_INTR2_CLR				0x4508
-#define PCIE_MSI_INTR2_MASK_SET				0x4510
-#define PCIE_MSI_INTR2_MASK_CLR				0x4514
+
+#define PCIE_INTR2_CPU_BASE		0x4300
+#define PCIE_MSI_INTR2_BASE		0x4500
+/* Offsets from PCIE_INTR2_CPU_BASE and PCIE_MSI_INTR2_BASE */
+#define  MSI_INT_STATUS			0x0
+#define  MSI_INT_CLR			0x8
+#define  MSI_INT_MASK_SET		0x10
+#define  MSI_INT_MASK_CLR		0x14
 
 #define PCIE_EXT_CFG_DATA				0x8000
 
@@ -132,6 +140,8 @@
 /* PCIe parameters */
 #define BRCM_NUM_PCIE_OUT_WINS		0x4
 #define BRCM_INT_PCI_MSI_NR		32
+#define BRCM_INT_PCI_MSI_LEGACY_NR	8
+#define BRCM_INT_PCI_MSI_SHIFT		0
 
 /* MSI target adresses */
 #define BRCM_MSI_TARGET_ADDR_LT_4GB	0x0fffffffcULL
@@ -246,6 +256,12 @@ struct brcm_msi {
 	int			irq;
 	/* used indicates which MSI interrupts have been alloc'd */
 	unsigned long		used;
+	bool			legacy;
+	/* Some chips have MSIs in bits [31..24] of a shared register. */
+	int			legacy_shift;
+	int			nr; /* No. of MSI available, depends on chip */
+	/* This is the base pointer for interrupt status/set/clr regs */
+	void __iomem		*intr_base;
 };
 
 /* Internal PCIe Host Controller Information.*/
@@ -264,6 +280,7 @@ struct brcm_pcie {
 	struct reset_control	*rescal;
 	int			num_memc;
 	u64			memc_size[PCIE_BRCM_MAX_MEMC];
+	u32			hw_rev;
 };
 
 /*
@@ -454,8 +471,10 @@ static void brcm_pcie_msi_isr(struct irq_desc *desc)
 	msi = irq_desc_get_handler_data(desc);
 	dev = msi->dev;
 
-	status = readl(msi->base + PCIE_MSI_INTR2_STATUS);
-	for_each_set_bit(bit, &status, BRCM_INT_PCI_MSI_NR) {
+	status = readl(msi->intr_base + MSI_INT_STATUS);
+	status >>= msi->legacy_shift;
+
+	for_each_set_bit(bit, &status, msi->nr) {
 		virq = irq_find_mapping(msi->inner_domain, bit);
 		if (virq)
 			generic_handle_irq(virq);
@@ -472,7 +491,7 @@ static void brcm_msi_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
 
 	msg->address_lo = lower_32_bits(msi->target_addr);
 	msg->address_hi = upper_32_bits(msi->target_addr);
-	msg->data = (0xffff & PCIE_MISC_MSI_DATA_CONFIG_VAL) | data->hwirq;
+	msg->data = (0xffff & PCIE_MISC_MSI_DATA_CONFIG_VAL_32) | data->hwirq;
 }
 
 static int brcm_msi_set_affinity(struct irq_data *irq_data,
@@ -484,8 +503,9 @@ static int brcm_msi_set_affinity(struct irq_data *irq_data,
 static void brcm_msi_ack_irq(struct irq_data *data)
 {
 	struct brcm_msi *msi = irq_data_get_irq_chip_data(data);
+	const int shift_amt = data->hwirq + msi->legacy_shift;
 
-	writel(1 << data->hwirq, msi->base + PCIE_MSI_INTR2_CLR);
+	writel(1 << shift_amt, msi->intr_base + MSI_INT_CLR);
 }
 
 
@@ -501,7 +521,7 @@ static int brcm_msi_alloc(struct brcm_msi *msi)
 	int hwirq;
 
 	mutex_lock(&msi->lock);
-	hwirq = bitmap_find_free_region(&msi->used, BRCM_INT_PCI_MSI_NR, 0);
+	hwirq = bitmap_find_free_region(&msi->used, msi->nr, 0);
 	mutex_unlock(&msi->lock);
 
 	return hwirq;
@@ -550,8 +570,7 @@ static int brcm_allocate_domains(struct brcm_msi *msi)
 	struct fwnode_handle *fwnode = of_node_to_fwnode(msi->np);
 	struct device *dev = msi->dev;
 
-	msi->inner_domain = irq_domain_add_linear(NULL, BRCM_INT_PCI_MSI_NR,
-						  &msi_domain_ops, msi);
+	msi->inner_domain = irq_domain_add_linear(NULL, msi->nr, &msi_domain_ops, msi);
 	if (!msi->inner_domain) {
 		dev_err(dev, "failed to create IRQ domain\n");
 		return -ENOMEM;
@@ -588,7 +607,10 @@ static void brcm_msi_remove(struct brcm_pcie *pcie)
 
 static void brcm_msi_set_regs(struct brcm_msi *msi)
 {
-	writel(0xffffffff, msi->base + PCIE_MSI_INTR2_MASK_CLR);
+	u32 val = __GENMASK(31, msi->legacy_shift);
+
+	writel(val, msi->intr_base + MSI_INT_MASK_CLR);
+	writel(val, msi->intr_base + MSI_INT_CLR);
 
 	/*
 	 * The 0 bit of PCIE_MISC_MSI_BAR_CONFIG_LO is repurposed to MSI
@@ -599,8 +621,8 @@ static void brcm_msi_set_regs(struct brcm_msi *msi)
 	writel(upper_32_bits(msi->target_addr),
 	       msi->base + PCIE_MISC_MSI_BAR_CONFIG_HI);
 
-	writel(PCIE_MISC_MSI_DATA_CONFIG_VAL,
-	       msi->base + PCIE_MISC_MSI_DATA_CONFIG);
+	val = msi->legacy ? PCIE_MISC_MSI_DATA_CONFIG_VAL_8 : PCIE_MISC_MSI_DATA_CONFIG_VAL_32;
+	writel(val, msi->base + PCIE_MISC_MSI_DATA_CONFIG);
 }
 
 static int brcm_pcie_enable_msi(struct brcm_pcie *pcie)
@@ -625,6 +647,17 @@ static int brcm_pcie_enable_msi(struct brcm_pcie *pcie)
 	msi->np = pcie->np;
 	msi->target_addr = pcie->msi_target_addr;
 	msi->irq = irq;
+	msi->legacy = pcie->hw_rev < BRCM_PCIE_HW_REV_33;
+
+	if (msi->legacy) {
+		msi->intr_base = msi->base + PCIE_INTR2_CPU_BASE;
+		msi->nr = BRCM_INT_PCI_MSI_LEGACY_NR;
+		msi->legacy_shift = 24;
+	} else {
+		msi->intr_base = msi->base + PCIE_MSI_INTR2_BASE;
+		msi->nr = BRCM_INT_PCI_MSI_NR;
+		msi->legacy_shift = 0;
+	}
 
 	ret = brcm_allocate_domains(msi);
 	if (ret)
@@ -887,12 +920,6 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 	tmp &= ~PCIE_MISC_RC_BAR3_CONFIG_LO_SIZE_MASK;
 	writel(tmp, base + PCIE_MISC_RC_BAR3_CONFIG_LO);
 
-	/* Mask all interrupts since we are not handling any yet */
-	writel(0xffffffff, pcie->base + PCIE_MSI_INTR2_MASK_SET);
-
-	/* clear any interrupts we find on boot */
-	writel(0xffffffff, pcie->base + PCIE_MSI_INTR2_CLR);
-
 	if (pcie->gen)
 		brcm_pcie_set_gen(pcie, pcie->gen);
 
@@ -1229,6 +1256,8 @@ static int brcm_pcie_probe(struct platform_device *pdev)
 	if (ret)
 		goto fail;
 
+	pcie->hw_rev = readl(pcie->base + PCIE_MISC_REVISION);
+
 	msi_np = of_parse_phandle(pcie->np, "msi-parent", 0);
 	if (pci_msi_enabled() && msi_np == pcie->np) {
 		ret = brcm_pcie_enable_msi(pcie);
-- 
2.17.1

[PATCH v11 09/11] PCI: brcmstb: Accommodate MSI for older chips

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

Older BrcmSTB chips do not have a separate register for MSI interrupts; the
MSIs are in a register that also contains unrelated interrupts.  In
addition, the interrupts lie in bits [31..24] for these legacy chips.  This
commit provides common code for both legacy and non-legacy MSI interrupt
registers.

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 71 +++++++++++++++++++--------
 1 file changed, 50 insertions(+), 21 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 7150eaa803c2..61c1be766522 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -82,7 +82,8 @@
 #define PCIE_MISC_MSI_BAR_CONFIG_HI			0x4048
 
 #define PCIE_MISC_MSI_DATA_CONFIG			0x404c
-#define  PCIE_MISC_MSI_DATA_CONFIG_VAL			0xffe06540
+#define  PCIE_MISC_MSI_DATA_CONFIG_VAL_32		0xffe06540
+#define  PCIE_MISC_MSI_DATA_CONFIG_VAL_8		0xfff86540
 
 #define PCIE_MISC_PCIE_CTRL				0x4064
 #define  PCIE_MISC_PCIE_CTRL_PCIE_L23_REQUEST_MASK	0x1
@@ -94,6 +95,9 @@
 #define  PCIE_MISC_PCIE_STATUS_PCIE_PHYLINKUP_MASK	0x10
 #define  PCIE_MISC_PCIE_STATUS_PCIE_LINK_IN_L23_MASK	0x40
 
+#define PCIE_MISC_REVISION				0x406c
+#define  BRCM_PCIE_HW_REV_33				0x0303
+
 #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT		0x4070
 #define  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT_LIMIT_MASK	0xfff00000
 #define  PCIE_MISC_CPU_2_PCIE_MEM_WIN0_BASE_LIMIT_BASE_MASK	0xfff0
@@ -114,10 +118,14 @@
 #define  PCIE_MISC_HARD_PCIE_HARD_DEBUG_CLKREQ_DEBUG_ENABLE_MASK	0x2
 #define  PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK		0x08000000
 
-#define PCIE_MSI_INTR2_STATUS				0x4500
-#define PCIE_MSI_INTR2_CLR				0x4508
-#define PCIE_MSI_INTR2_MASK_SET				0x4510
-#define PCIE_MSI_INTR2_MASK_CLR				0x4514
+
+#define PCIE_INTR2_CPU_BASE		0x4300
+#define PCIE_MSI_INTR2_BASE		0x4500
+/* Offsets from PCIE_INTR2_CPU_BASE and PCIE_MSI_INTR2_BASE */
+#define  MSI_INT_STATUS			0x0
+#define  MSI_INT_CLR			0x8
+#define  MSI_INT_MASK_SET		0x10
+#define  MSI_INT_MASK_CLR		0x14
 
 #define PCIE_EXT_CFG_DATA				0x8000
 
@@ -132,6 +140,8 @@
 /* PCIe parameters */
 #define BRCM_NUM_PCIE_OUT_WINS		0x4
 #define BRCM_INT_PCI_MSI_NR		32
+#define BRCM_INT_PCI_MSI_LEGACY_NR	8
+#define BRCM_INT_PCI_MSI_SHIFT		0
 
 /* MSI target adresses */
 #define BRCM_MSI_TARGET_ADDR_LT_4GB	0x0fffffffcULL
@@ -246,6 +256,12 @@ struct brcm_msi {
 	int			irq;
 	/* used indicates which MSI interrupts have been alloc'd */
 	unsigned long		used;
+	bool			legacy;
+	/* Some chips have MSIs in bits [31..24] of a shared register. */
+	int			legacy_shift;
+	int			nr; /* No. of MSI available, depends on chip */
+	/* This is the base pointer for interrupt status/set/clr regs */
+	void __iomem		*intr_base;
 };
 
 /* Internal PCIe Host Controller Information.*/
@@ -264,6 +280,7 @@ struct brcm_pcie {
 	struct reset_control	*rescal;
 	int			num_memc;
 	u64			memc_size[PCIE_BRCM_MAX_MEMC];
+	u32			hw_rev;
 };
 
 /*
@@ -454,8 +471,10 @@ static void brcm_pcie_msi_isr(struct irq_desc *desc)
 	msi = irq_desc_get_handler_data(desc);
 	dev = msi->dev;
 
-	status = readl(msi->base + PCIE_MSI_INTR2_STATUS);
-	for_each_set_bit(bit, &status, BRCM_INT_PCI_MSI_NR) {
+	status = readl(msi->intr_base + MSI_INT_STATUS);
+	status >>= msi->legacy_shift;
+
+	for_each_set_bit(bit, &status, msi->nr) {
 		virq = irq_find_mapping(msi->inner_domain, bit);
 		if (virq)
 			generic_handle_irq(virq);
@@ -472,7 +491,7 @@ static void brcm_msi_compose_msi_msg(struct irq_data *data, struct msi_msg *msg)
 
 	msg->address_lo = lower_32_bits(msi->target_addr);
 	msg->address_hi = upper_32_bits(msi->target_addr);
-	msg->data = (0xffff & PCIE_MISC_MSI_DATA_CONFIG_VAL) | data->hwirq;
+	msg->data = (0xffff & PCIE_MISC_MSI_DATA_CONFIG_VAL_32) | data->hwirq;
 }
 
 static int brcm_msi_set_affinity(struct irq_data *irq_data,
@@ -484,8 +503,9 @@ static int brcm_msi_set_affinity(struct irq_data *irq_data,
 static void brcm_msi_ack_irq(struct irq_data *data)
 {
 	struct brcm_msi *msi = irq_data_get_irq_chip_data(data);
+	const int shift_amt = data->hwirq + msi->legacy_shift;
 
-	writel(1 << data->hwirq, msi->base + PCIE_MSI_INTR2_CLR);
+	writel(1 << shift_amt, msi->intr_base + MSI_INT_CLR);
 }
 
 
@@ -501,7 +521,7 @@ static int brcm_msi_alloc(struct brcm_msi *msi)
 	int hwirq;
 
 	mutex_lock(&msi->lock);
-	hwirq = bitmap_find_free_region(&msi->used, BRCM_INT_PCI_MSI_NR, 0);
+	hwirq = bitmap_find_free_region(&msi->used, msi->nr, 0);
 	mutex_unlock(&msi->lock);
 
 	return hwirq;
@@ -550,8 +570,7 @@ static int brcm_allocate_domains(struct brcm_msi *msi)
 	struct fwnode_handle *fwnode = of_node_to_fwnode(msi->np);
 	struct device *dev = msi->dev;
 
-	msi->inner_domain = irq_domain_add_linear(NULL, BRCM_INT_PCI_MSI_NR,
-						  &msi_domain_ops, msi);
+	msi->inner_domain = irq_domain_add_linear(NULL, msi->nr, &msi_domain_ops, msi);
 	if (!msi->inner_domain) {
 		dev_err(dev, "failed to create IRQ domain\n");
 		return -ENOMEM;
@@ -588,7 +607,10 @@ static void brcm_msi_remove(struct brcm_pcie *pcie)
 
 static void brcm_msi_set_regs(struct brcm_msi *msi)
 {
-	writel(0xffffffff, msi->base + PCIE_MSI_INTR2_MASK_CLR);
+	u32 val = __GENMASK(31, msi->legacy_shift);
+
+	writel(val, msi->intr_base + MSI_INT_MASK_CLR);
+	writel(val, msi->intr_base + MSI_INT_CLR);
 
 	/*
 	 * The 0 bit of PCIE_MISC_MSI_BAR_CONFIG_LO is repurposed to MSI
@@ -599,8 +621,8 @@ static void brcm_msi_set_regs(struct brcm_msi *msi)
 	writel(upper_32_bits(msi->target_addr),
 	       msi->base + PCIE_MISC_MSI_BAR_CONFIG_HI);
 
-	writel(PCIE_MISC_MSI_DATA_CONFIG_VAL,
-	       msi->base + PCIE_MISC_MSI_DATA_CONFIG);
+	val = msi->legacy ? PCIE_MISC_MSI_DATA_CONFIG_VAL_8 : PCIE_MISC_MSI_DATA_CONFIG_VAL_32;
+	writel(val, msi->base + PCIE_MISC_MSI_DATA_CONFIG);
 }
 
 static int brcm_pcie_enable_msi(struct brcm_pcie *pcie)
@@ -625,6 +647,17 @@ static int brcm_pcie_enable_msi(struct brcm_pcie *pcie)
 	msi->np = pcie->np;
 	msi->target_addr = pcie->msi_target_addr;
 	msi->irq = irq;
+	msi->legacy = pcie->hw_rev < BRCM_PCIE_HW_REV_33;
+
+	if (msi->legacy) {
+		msi->intr_base = msi->base + PCIE_INTR2_CPU_BASE;
+		msi->nr = BRCM_INT_PCI_MSI_LEGACY_NR;
+		msi->legacy_shift = 24;
+	} else {
+		msi->intr_base = msi->base + PCIE_MSI_INTR2_BASE;
+		msi->nr = BRCM_INT_PCI_MSI_NR;
+		msi->legacy_shift = 0;
+	}
 
 	ret = brcm_allocate_domains(msi);
 	if (ret)
@@ -887,12 +920,6 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 	tmp &= ~PCIE_MISC_RC_BAR3_CONFIG_LO_SIZE_MASK;
 	writel(tmp, base + PCIE_MISC_RC_BAR3_CONFIG_LO);
 
-	/* Mask all interrupts since we are not handling any yet */
-	writel(0xffffffff, pcie->base + PCIE_MSI_INTR2_MASK_SET);
-
-	/* clear any interrupts we find on boot */
-	writel(0xffffffff, pcie->base + PCIE_MSI_INTR2_CLR);
-
 	if (pcie->gen)
 		brcm_pcie_set_gen(pcie, pcie->gen);
 
@@ -1229,6 +1256,8 @@ static int brcm_pcie_probe(struct platform_device *pdev)
 	if (ret)
 		goto fail;
 
+	pcie->hw_rev = readl(pcie->base + PCIE_MISC_REVISION);
+
 	msi_np = of_parse_phandle(pcie->np, "msi-parent", 0);
 	if (pci_msi_enabled() && msi_np == pcie->np) {
 		ret = brcm_pcie_enable_msi(pcie);
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v11 10/11] PCI: brcmstb: Set bus max burst size by chip type

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

The proper value of the parameter SCB_MAX_BURST_SIZE varies per chip.  The
2711 family requires 128B whereas other devices can employ 512.  The
assignment is complicated by the fact that the values for this two-bit
field have different meanings;

  Value   Type_Generic    Type_7278

     00       Reserved         128B
     01           128B         256B
     10           256B         512B
     11           512B     Reserved

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 61c1be766522..0d8234da519c 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -55,7 +55,7 @@
 #define  PCIE_MISC_MISC_CTRL_SCB_ACCESS_EN_MASK		0x1000
 #define  PCIE_MISC_MISC_CTRL_CFG_READ_UR_MODE_MASK	0x2000
 #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK	0x300000
-#define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128		0x0
+
 #define  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK		0xf8000000
 #define  PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK		0x07c00000
 #define  PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK		0x0000001f
@@ -846,7 +846,7 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 	int num_out_wins = 0;
 	u16 nlw, cls, lnksta;
 	int i, ret, memc;
-	u32 tmp, aspm_support;
+	u32 tmp, burst, aspm_support;
 
 	/* Reset the bridge */
 	brcm_pcie_bridge_sw_init_set(pcie, 1);
@@ -866,11 +866,22 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 	/* Wait for SerDes to be stable */
 	usleep_range(100, 200);
 
+	/*
+	 * SCB_MAX_BURST_SIZE is a two bit field.  For GENERIC chips it
+	 * is encoded as 0=128, 1=256, 2=512, 3=Rsvd, for BCM7278 it
+	 * is encoded as 0=Rsvd, 1=128, 2=256, 3=512.
+	 */
+	if (pcie->type == BCM2711)
+		burst = 0x0; /* 128B */
+	else if (pcie->type == BCM7278)
+		burst = 0x3; /* 512 bytes */
+	else
+		burst = 0x2; /* 512 bytes */
+
 	/* Set SCB_MAX_BURST_SIZE, CFG_READ_UR_MODE, SCB_ACCESS_EN */
 	u32p_replace_bits(&tmp, 1, PCIE_MISC_MISC_CTRL_SCB_ACCESS_EN_MASK);
 	u32p_replace_bits(&tmp, 1, PCIE_MISC_MISC_CTRL_CFG_READ_UR_MODE_MASK);
-	u32p_replace_bits(&tmp, PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128,
-			  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK);
+	u32p_replace_bits(&tmp, burst, PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK);
 	writel(tmp, base + PCIE_MISC_MISC_CTRL);
 
 	ret = brcm_pcie_get_rc_bar2_size_and_offset(pcie, &rc_bar2_size,
-- 
2.17.1

[PATCH v11 10/11] PCI: brcmstb: Set bus max burst size by chip type

[Jim Quinlan]

From: Jim Quinlan <jquinlan@broadcom.com>

The proper value of the parameter SCB_MAX_BURST_SIZE varies per chip.  The
2711 family requires 128B whereas other devices can employ 512.  The
assignment is complicated by the fact that the values for this two-bit
field have different meanings;

  Value   Type_Generic    Type_7278

     00       Reserved         128B
     01           128B         256B
     10           256B         512B
     11           512B     Reserved

Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 61c1be766522..0d8234da519c 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -55,7 +55,7 @@
 #define  PCIE_MISC_MISC_CTRL_SCB_ACCESS_EN_MASK		0x1000
 #define  PCIE_MISC_MISC_CTRL_CFG_READ_UR_MODE_MASK	0x2000
 #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK	0x300000
-#define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128		0x0
+
 #define  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK		0xf8000000
 #define  PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK		0x07c00000
 #define  PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK		0x0000001f
@@ -846,7 +846,7 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 	int num_out_wins = 0;
 	u16 nlw, cls, lnksta;
 	int i, ret, memc;
-	u32 tmp, aspm_support;
+	u32 tmp, burst, aspm_support;
 
 	/* Reset the bridge */
 	brcm_pcie_bridge_sw_init_set(pcie, 1);
@@ -866,11 +866,22 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
 	/* Wait for SerDes to be stable */
 	usleep_range(100, 200);
 
+	/*
+	 * SCB_MAX_BURST_SIZE is a two bit field.  For GENERIC chips it
+	 * is encoded as 0=128, 1=256, 2=512, 3=Rsvd, for BCM7278 it
+	 * is encoded as 0=Rsvd, 1=128, 2=256, 3=512.
+	 */
+	if (pcie->type == BCM2711)
+		burst = 0x0; /* 128B */
+	else if (pcie->type == BCM7278)
+		burst = 0x3; /* 512 bytes */
+	else
+		burst = 0x2; /* 512 bytes */
+
 	/* Set SCB_MAX_BURST_SIZE, CFG_READ_UR_MODE, SCB_ACCESS_EN */
 	u32p_replace_bits(&tmp, 1, PCIE_MISC_MISC_CTRL_SCB_ACCESS_EN_MASK);
 	u32p_replace_bits(&tmp, 1, PCIE_MISC_MISC_CTRL_CFG_READ_UR_MODE_MASK);
-	u32p_replace_bits(&tmp, PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128,
-			  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK);
+	u32p_replace_bits(&tmp, burst, PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK);
 	writel(tmp, base + PCIE_MISC_MISC_CTRL);
 
 	ret = brcm_pcie_get_rc_bar2_size_and_offset(pcie, &rc_bar2_size,
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v11 11/11] PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list

[Jim Quinlan]

Now that the support is in place with previous commits, we add several
chips that use the BrcmSTB driver.

Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 0d8234da519c..ca4023423acb 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -1187,6 +1187,10 @@ static int brcm_pcie_remove(struct platform_device *pdev)
 
 static const struct of_device_id brcm_pcie_match[] = {
 	{ .compatible = "brcm,bcm2711-pcie", .data = &bcm2711_cfg },
+	{ .compatible = "brcm,bcm7211-pcie", .data = &generic_cfg },
+	{ .compatible = "brcm,bcm7278-pcie", .data = &bcm7278_cfg },
+	{ .compatible = "brcm,bcm7216-pcie", .data = &bcm7278_cfg },
+	{ .compatible = "brcm,bcm7445-pcie", .data = &generic_cfg },
 	{},
 };
 
-- 
2.17.1

[PATCH v11 11/11] PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list

[Jim Quinlan]

Now that the support is in place with previous commits, we add several
chips that use the BrcmSTB driver.

Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
Acked-by: Florian Fainelli <f.fainelli@gmail.com>
---
 drivers/pci/controller/pcie-brcmstb.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 0d8234da519c..ca4023423acb 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -1187,6 +1187,10 @@ static int brcm_pcie_remove(struct platform_device *pdev)
 
 static const struct of_device_id brcm_pcie_match[] = {
 	{ .compatible = "brcm,bcm2711-pcie", .data = &bcm2711_cfg },
+	{ .compatible = "brcm,bcm7211-pcie", .data = &generic_cfg },
+	{ .compatible = "brcm,bcm7278-pcie", .data = &bcm7278_cfg },
+	{ .compatible = "brcm,bcm7216-pcie", .data = &bcm7278_cfg },
+	{ .compatible = "brcm,bcm7445-pcie", .data = &generic_cfg },
 	{},
 };
 
-- 
2.17.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Andy Shevchenko]

On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> The new field 'dma_range_map' in struct device is used to facilitate the
> use of single or multiple offsets between mapping regions of cpu addrs and
> dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> capable of holding a single uniform offset and had no region bounds
> checking.
> 
> The function of_dma_get_range() has been modified so that it takes a single
> argument -- the device node -- and returns a map, NULL, or an error code.
> The map is an array that holds the information regarding the DMA regions.
> Each range entry contains the address offset, the cpu_start address, the
> dma_start address, and the size of the region.
> 
> of_dma_configure() is the typical manner to set range offsets but there are
> a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> driver code.  These cases now invoke the function
> dma_attach_offset_range(dev, cpu_addr, dma_addr, size).

...

> +	/*
> +	 * Record all info in the generic DMA ranges array for struct device.
> +	 */
> +	*map = r;
> +	for_each_of_range(&parser, &range) {
> +		pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
> +			 range.bus_addr, range.cpu_addr, range.size);
> +		r->cpu_start = range.cpu_addr;
> +		r->dma_start = range.bus_addr;
> +		r->size = range.size;

> +		r->offset = (u64)range.cpu_addr - (u64)range.bus_addr;

What's the point in explicit castings to the same type?

> +		r++;
> +	}

...

> +		phys_addr_t	paddr;
> +		dma_addr_t	dma_addr;
> +		struct device	dev_bogus;

>  		unittest(paddr == expect_paddr,
> -			 "of_dma_get_range wrong phys addr (%llx) on node %pOF", paddr, np);
> +			 "of_dma_get_range: wrong phys addr %llx (expecting %llx) on node %pOF\n",
> +			 (u64)paddr, expect_paddr, np);

%llx -> %pap

>  		unittest(dma_addr == expect_dma_addr,
> -			 "of_dma_get_range wrong DMA addr (%llx) on node %pOF", dma_addr, np);
> +			 "of_dma_get_range: wrong DMA addr %llx (expecting %llx) on node %pOF\n",
> +			 (u64)dma_addr, expect_dma_addr, np);

%llx -> %pad

...

> +	if (mem->use_dev_dma_pfn_offset) {
> +		u64 base_addr = PFN_PHYS((u64)mem->pfn_base);

Do we need explicit casting here?

> +
> +		return base_addr - dma_offset_from_phys_addr(dev, base_addr);
> +	}

...

> +int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
> +			 dma_addr_t dma_start, u64 size)
> +{
> +	struct bus_dma_region *map;
> +	u64 offset = (u64)cpu_start - (u64)dma_start;
> +
> +	if (dev->dma_range_map) {
> +		dev_err(dev, "attempt to add DMA range to existing map\n");
> +		return -EINVAL;
> +	}

Wouldn't be better to do an assignment of offset here?

> +	if (!offset)
> +		return 0;
> +
> +	map = kcalloc(2, sizeof(*map), GFP_KERNEL);
> +	if (!map)
> +		return -ENOMEM;
> +	map[0].cpu_start = cpu_start;
> +	map[0].dma_start = dma_start;
> +	map[0].offset = offset;
> +	map[0].size = size;
> +	dev->dma_range_map = map;
> +
> +	return 0;
> +}

-- 
With Best Regards,
Andy Shevchenko


_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Andy Shevchenko]

On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> The new field 'dma_range_map' in struct device is used to facilitate the
> use of single or multiple offsets between mapping regions of cpu addrs and
> dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> capable of holding a single uniform offset and had no region bounds
> checking.
> 
> The function of_dma_get_range() has been modified so that it takes a single
> argument -- the device node -- and returns a map, NULL, or an error code.
> The map is an array that holds the information regarding the DMA regions.
> Each range entry contains the address offset, the cpu_start address, the
> dma_start address, and the size of the region.
> 
> of_dma_configure() is the typical manner to set range offsets but there are
> a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> driver code.  These cases now invoke the function
> dma_attach_offset_range(dev, cpu_addr, dma_addr, size).

...

> +	/*
> +	 * Record all info in the generic DMA ranges array for struct device.
> +	 */
> +	*map = r;
> +	for_each_of_range(&parser, &range) {
> +		pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
> +			 range.bus_addr, range.cpu_addr, range.size);
> +		r->cpu_start = range.cpu_addr;
> +		r->dma_start = range.bus_addr;
> +		r->size = range.size;

> +		r->offset = (u64)range.cpu_addr - (u64)range.bus_addr;

What's the point in explicit castings to the same type?

> +		r++;
> +	}

...

> +		phys_addr_t	paddr;
> +		dma_addr_t	dma_addr;
> +		struct device	dev_bogus;

>  		unittest(paddr == expect_paddr,
> -			 "of_dma_get_range wrong phys addr (%llx) on node %pOF", paddr, np);
> +			 "of_dma_get_range: wrong phys addr %llx (expecting %llx) on node %pOF\n",
> +			 (u64)paddr, expect_paddr, np);

%llx -> %pap

>  		unittest(dma_addr == expect_dma_addr,
> -			 "of_dma_get_range wrong DMA addr (%llx) on node %pOF", dma_addr, np);
> +			 "of_dma_get_range: wrong DMA addr %llx (expecting %llx) on node %pOF\n",
> +			 (u64)dma_addr, expect_dma_addr, np);

%llx -> %pad

...

> +	if (mem->use_dev_dma_pfn_offset) {
> +		u64 base_addr = PFN_PHYS((u64)mem->pfn_base);

Do we need explicit casting here?

> +
> +		return base_addr - dma_offset_from_phys_addr(dev, base_addr);
> +	}

...

> +int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
> +			 dma_addr_t dma_start, u64 size)
> +{
> +	struct bus_dma_region *map;
> +	u64 offset = (u64)cpu_start - (u64)dma_start;
> +
> +	if (dev->dma_range_map) {
> +		dev_err(dev, "attempt to add DMA range to existing map\n");
> +		return -EINVAL;
> +	}

Wouldn't be better to do an assignment of offset here?

> +	if (!offset)
> +		return 0;
> +
> +	map = kcalloc(2, sizeof(*map), GFP_KERNEL);
> +	if (!map)
> +		return -ENOMEM;
> +	map[0].cpu_start = cpu_start;
> +	map[0].dma_start = dma_start;
> +	map[0].offset = offset;
> +	map[0].size = size;
> +	dev->dma_range_map = map;
> +
> +	return 0;
> +}

-- 
With Best Regards,
Andy Shevchenko


_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Andy Shevchenko]

On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> The new field 'dma_range_map' in struct device is used to facilitate the
> use of single or multiple offsets between mapping regions of cpu addrs and
> dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> capable of holding a single uniform offset and had no region bounds
> checking.
> 
> The function of_dma_get_range() has been modified so that it takes a single
> argument -- the device node -- and returns a map, NULL, or an error code.
> The map is an array that holds the information regarding the DMA regions.
> Each range entry contains the address offset, the cpu_start address, the
> dma_start address, and the size of the region.
> 
> of_dma_configure() is the typical manner to set range offsets but there are
> a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> driver code.  These cases now invoke the function
> dma_attach_offset_range(dev, cpu_addr, dma_addr, size).

...

> +	/*
> +	 * Record all info in the generic DMA ranges array for struct device.
> +	 */
> +	*map = r;
> +	for_each_of_range(&parser, &range) {
> +		pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
> +			 range.bus_addr, range.cpu_addr, range.size);
> +		r->cpu_start = range.cpu_addr;
> +		r->dma_start = range.bus_addr;
> +		r->size = range.size;

> +		r->offset = (u64)range.cpu_addr - (u64)range.bus_addr;

What's the point in explicit castings to the same type?

> +		r++;
> +	}

...

> +		phys_addr_t	paddr;
> +		dma_addr_t	dma_addr;
> +		struct device	dev_bogus;

>  		unittest(paddr == expect_paddr,
> -			 "of_dma_get_range wrong phys addr (%llx) on node %pOF", paddr, np);
> +			 "of_dma_get_range: wrong phys addr %llx (expecting %llx) on node %pOF\n",
> +			 (u64)paddr, expect_paddr, np);

%llx -> %pap

>  		unittest(dma_addr == expect_dma_addr,
> -			 "of_dma_get_range wrong DMA addr (%llx) on node %pOF", dma_addr, np);
> +			 "of_dma_get_range: wrong DMA addr %llx (expecting %llx) on node %pOF\n",
> +			 (u64)dma_addr, expect_dma_addr, np);

%llx -> %pad

...

> +	if (mem->use_dev_dma_pfn_offset) {
> +		u64 base_addr = PFN_PHYS((u64)mem->pfn_base);

Do we need explicit casting here?

> +
> +		return base_addr - dma_offset_from_phys_addr(dev, base_addr);
> +	}

...

> +int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
> +			 dma_addr_t dma_start, u64 size)
> +{
> +	struct bus_dma_region *map;
> +	u64 offset = (u64)cpu_start - (u64)dma_start;
> +
> +	if (dev->dma_range_map) {
> +		dev_err(dev, "attempt to add DMA range to existing map\n");
> +		return -EINVAL;
> +	}

Wouldn't be better to do an assignment of offset here?

> +	if (!offset)
> +		return 0;
> +
> +	map = kcalloc(2, sizeof(*map), GFP_KERNEL);
> +	if (!map)
> +		return -ENOMEM;
> +	map[0].cpu_start = cpu_start;
> +	map[0].dma_start = dma_start;
> +	map[0].offset = offset;
> +	map[0].size = size;
> +	dev->dma_range_map = map;
> +
> +	return 0;
> +}

-- 
With Best Regards,
Andy Shevchenko


_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

Hi Andy,


On Tue, Aug 25, 2020 at 5:54 AM Andy Shevchenko
<andriy.shevchenko@linux.intel.com> wrote:
>
> On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> > The new field 'dma_range_map' in struct device is used to facilitate the
> > use of single or multiple offsets between mapping regions of cpu addrs and
> > dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> > capable of holding a single uniform offset and had no region bounds
> > checking.
> >
> > The function of_dma_get_range() has been modified so that it takes a single
> > argument -- the device node -- and returns a map, NULL, or an error code.
> > The map is an array that holds the information regarding the DMA regions.
> > Each range entry contains the address offset, the cpu_start address, the
> > dma_start address, and the size of the region.
> >
> > of_dma_configure() is the typical manner to set range offsets but there are
> > a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> > driver code.  These cases now invoke the function
> > dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
>
> ...
>
> > +     /*
> > +      * Record all info in the generic DMA ranges array for struct device.
> > +      */
> > +     *map = r;
> > +     for_each_of_range(&parser, &range) {
> > +             pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
> > +                      range.bus_addr, range.cpu_addr, range.size);
> > +             r->cpu_start = range.cpu_addr;
> > +             r->dma_start = range.bus_addr;
> > +             r->size = range.size;
>
> > +             r->offset = (u64)range.cpu_addr - (u64)range.bus_addr;
>
> What's the point in explicit castings to the same type?
No point.  If I have to send out another version I will fix this.

>
> > +             r++;
> > +     }
>
> ...
>
> > +             phys_addr_t     paddr;
> > +             dma_addr_t      dma_addr;
> > +             struct device   dev_bogus;
>
> >               unittest(paddr == expect_paddr,
> > -                      "of_dma_get_range wrong phys addr (%llx) on node %pOF", paddr, np);
> > +                      "of_dma_get_range: wrong phys addr %llx (expecting %llx) on node %pOF\n",
> > +                      (u64)paddr, expect_paddr, np);
>
> %llx -> %pap
This was intentional -- I'm aware of %pap and %pad.  The problem is
that %pa[pd]  print out a zero-filled 16 character number regardless
of what the number is.  For example, 1 is "0x0000000000000001",
whereas using %llx yields "1".

>
> >               unittest(dma_addr == expect_dma_addr,
> > -                      "of_dma_get_range wrong DMA addr (%llx) on node %pOF", dma_addr, np);
> > +                      "of_dma_get_range: wrong DMA addr %llx (expecting %llx) on node %pOF\n",
> > +                      (u64)dma_addr, expect_dma_addr, np);
>
> %llx -> %pad
>
> ...
>
> > +     if (mem->use_dev_dma_pfn_offset) {
> > +             u64 base_addr = PFN_PHYS((u64)mem->pfn_base);
>
> Do we need explicit casting here?
I don't think it is needed.  However, the "(u64)" is useless though
since the macro recasts it to a phys_addr_t.

If there is another version of this submission I will change this.
>
> > +
> > +             return base_addr - dma_offset_from_phys_addr(dev, base_addr);
> > +     }
>
> ...
>
> > +int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
> > +                      dma_addr_t dma_start, u64 size)
> > +{
> > +     struct bus_dma_region *map;
> > +     u64 offset = (u64)cpu_start - (u64)dma_start;
> > +
> > +     if (dev->dma_range_map) {
> > +             dev_err(dev, "attempt to add DMA range to existing map\n");
> > +             return -EINVAL;
> > +     }
>
> Wouldn't be better to do an assignment of offset here?
IIRC this was what Christoph requested.  It has actually gone back and
forth over the versions of this submission.
>
> > +     if (!offset)
> > +             return 0;
> > +
> > +     map = kcalloc(2, sizeof(*map), GFP_KERNEL);
> > +     if (!map)
> > +             return -ENOMEM;
> > +     map[0].cpu_start = cpu_start;
> > +     map[0].dma_start = dma_start;
> > +     map[0].offset = offset;
> > +     map[0].size = size;
> > +     dev->dma_range_map = map;
> > +
> > +     return 0;
> > +}
>
> --
> With Best Regards,
> Andy Shevchenko
Thanks again,
Jim
>
>
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan via iommu]

Hi Andy,


On Tue, Aug 25, 2020 at 5:54 AM Andy Shevchenko
<andriy.shevchenko@linux.intel.com> wrote:
>
> On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> > The new field 'dma_range_map' in struct device is used to facilitate the
> > use of single or multiple offsets between mapping regions of cpu addrs and
> > dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> > capable of holding a single uniform offset and had no region bounds
> > checking.
> >
> > The function of_dma_get_range() has been modified so that it takes a single
> > argument -- the device node -- and returns a map, NULL, or an error code.
> > The map is an array that holds the information regarding the DMA regions.
> > Each range entry contains the address offset, the cpu_start address, the
> > dma_start address, and the size of the region.
> >
> > of_dma_configure() is the typical manner to set range offsets but there are
> > a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> > driver code.  These cases now invoke the function
> > dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
>
> ...
>
> > +     /*
> > +      * Record all info in the generic DMA ranges array for struct device.
> > +      */
> > +     *map = r;
> > +     for_each_of_range(&parser, &range) {
> > +             pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
> > +                      range.bus_addr, range.cpu_addr, range.size);
> > +             r->cpu_start = range.cpu_addr;
> > +             r->dma_start = range.bus_addr;
> > +             r->size = range.size;
>
> > +             r->offset = (u64)range.cpu_addr - (u64)range.bus_addr;
>
> What's the point in explicit castings to the same type?
No point.  If I have to send out another version I will fix this.

>
> > +             r++;
> > +     }
>
> ...
>
> > +             phys_addr_t     paddr;
> > +             dma_addr_t      dma_addr;
> > +             struct device   dev_bogus;
>
> >               unittest(paddr == expect_paddr,
> > -                      "of_dma_get_range wrong phys addr (%llx) on node %pOF", paddr, np);
> > +                      "of_dma_get_range: wrong phys addr %llx (expecting %llx) on node %pOF\n",
> > +                      (u64)paddr, expect_paddr, np);
>
> %llx -> %pap
This was intentional -- I'm aware of %pap and %pad.  The problem is
that %pa[pd]  print out a zero-filled 16 character number regardless
of what the number is.  For example, 1 is "0x0000000000000001",
whereas using %llx yields "1".

>
> >               unittest(dma_addr == expect_dma_addr,
> > -                      "of_dma_get_range wrong DMA addr (%llx) on node %pOF", dma_addr, np);
> > +                      "of_dma_get_range: wrong DMA addr %llx (expecting %llx) on node %pOF\n",
> > +                      (u64)dma_addr, expect_dma_addr, np);
>
> %llx -> %pad
>
> ...
>
> > +     if (mem->use_dev_dma_pfn_offset) {
> > +             u64 base_addr = PFN_PHYS((u64)mem->pfn_base);
>
> Do we need explicit casting here?
I don't think it is needed.  However, the "(u64)" is useless though
since the macro recasts it to a phys_addr_t.

If there is another version of this submission I will change this.
>
> > +
> > +             return base_addr - dma_offset_from_phys_addr(dev, base_addr);
> > +     }
>
> ...
>
> > +int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
> > +                      dma_addr_t dma_start, u64 size)
> > +{
> > +     struct bus_dma_region *map;
> > +     u64 offset = (u64)cpu_start - (u64)dma_start;
> > +
> > +     if (dev->dma_range_map) {
> > +             dev_err(dev, "attempt to add DMA range to existing map\n");
> > +             return -EINVAL;
> > +     }
>
> Wouldn't be better to do an assignment of offset here?
IIRC this was what Christoph requested.  It has actually gone back and
forth over the versions of this submission.
>
> > +     if (!offset)
> > +             return 0;
> > +
> > +     map = kcalloc(2, sizeof(*map), GFP_KERNEL);
> > +     if (!map)
> > +             return -ENOMEM;
> > +     map[0].cpu_start = cpu_start;
> > +     map[0].dma_start = dma_start;
> > +     map[0].offset = offset;
> > +     map[0].size = size;
> > +     dev->dma_range_map = map;
> > +
> > +     return 0;
> > +}
>
> --
> With Best Regards,
> Andy Shevchenko
Thanks again,
Jim
>
>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

Hi Andy,


On Tue, Aug 25, 2020 at 5:54 AM Andy Shevchenko
<andriy.shevchenko@linux.intel.com> wrote:
>
> On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> > The new field 'dma_range_map' in struct device is used to facilitate the
> > use of single or multiple offsets between mapping regions of cpu addrs and
> > dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> > capable of holding a single uniform offset and had no region bounds
> > checking.
> >
> > The function of_dma_get_range() has been modified so that it takes a single
> > argument -- the device node -- and returns a map, NULL, or an error code.
> > The map is an array that holds the information regarding the DMA regions.
> > Each range entry contains the address offset, the cpu_start address, the
> > dma_start address, and the size of the region.
> >
> > of_dma_configure() is the typical manner to set range offsets but there are
> > a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> > driver code.  These cases now invoke the function
> > dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
>
> ...
>
> > +     /*
> > +      * Record all info in the generic DMA ranges array for struct device.
> > +      */
> > +     *map = r;
> > +     for_each_of_range(&parser, &range) {
> > +             pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
> > +                      range.bus_addr, range.cpu_addr, range.size);
> > +             r->cpu_start = range.cpu_addr;
> > +             r->dma_start = range.bus_addr;
> > +             r->size = range.size;
>
> > +             r->offset = (u64)range.cpu_addr - (u64)range.bus_addr;
>
> What's the point in explicit castings to the same type?
No point.  If I have to send out another version I will fix this.

>
> > +             r++;
> > +     }
>
> ...
>
> > +             phys_addr_t     paddr;
> > +             dma_addr_t      dma_addr;
> > +             struct device   dev_bogus;
>
> >               unittest(paddr == expect_paddr,
> > -                      "of_dma_get_range wrong phys addr (%llx) on node %pOF", paddr, np);
> > +                      "of_dma_get_range: wrong phys addr %llx (expecting %llx) on node %pOF\n",
> > +                      (u64)paddr, expect_paddr, np);
>
> %llx -> %pap
This was intentional -- I'm aware of %pap and %pad.  The problem is
that %pa[pd]  print out a zero-filled 16 character number regardless
of what the number is.  For example, 1 is "0x0000000000000001",
whereas using %llx yields "1".

>
> >               unittest(dma_addr == expect_dma_addr,
> > -                      "of_dma_get_range wrong DMA addr (%llx) on node %pOF", dma_addr, np);
> > +                      "of_dma_get_range: wrong DMA addr %llx (expecting %llx) on node %pOF\n",
> > +                      (u64)dma_addr, expect_dma_addr, np);
>
> %llx -> %pad
>
> ...
>
> > +     if (mem->use_dev_dma_pfn_offset) {
> > +             u64 base_addr = PFN_PHYS((u64)mem->pfn_base);
>
> Do we need explicit casting here?
I don't think it is needed.  However, the "(u64)" is useless though
since the macro recasts it to a phys_addr_t.

If there is another version of this submission I will change this.
>
> > +
> > +             return base_addr - dma_offset_from_phys_addr(dev, base_addr);
> > +     }
>
> ...
>
> > +int dma_set_offset_range(struct device *dev, phys_addr_t cpu_start,
> > +                      dma_addr_t dma_start, u64 size)
> > +{
> > +     struct bus_dma_region *map;
> > +     u64 offset = (u64)cpu_start - (u64)dma_start;
> > +
> > +     if (dev->dma_range_map) {
> > +             dev_err(dev, "attempt to add DMA range to existing map\n");
> > +             return -EINVAL;
> > +     }
>
> Wouldn't be better to do an assignment of offset here?
IIRC this was what Christoph requested.  It has actually gone back and
forth over the versions of this submission.
>
> > +     if (!offset)
> > +             return 0;
> > +
> > +     map = kcalloc(2, sizeof(*map), GFP_KERNEL);
> > +     if (!map)
> > +             return -ENOMEM;
> > +     map[0].cpu_start = cpu_start;
> > +     map[0].dma_start = dma_start;
> > +     map[0].offset = offset;
> > +     map[0].size = size;
> > +     dev->dma_range_map = map;
> > +
> > +     return 0;
> > +}
>
> --
> With Best Regards,
> Andy Shevchenko
Thanks again,
Jim
>
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

Hi,

On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> 
> Patchset Summary:
>    Enhance a PCIe host controller driver.  Because of its unusual design
>    we are foced to change dev->dma_pfn_offset into a more general role
>    allowing multiple offsets.  See the 'v1' notes below for more info.

We are version 11 and counting, and it is not clear to me whether there 
is any chance of getting these patches reviewed and hopefully merged for 
the 5.10 merge window.

There are a lot of different files being touched, so what would be the 
ideal way of routing those changes towards inclusion?

Thanks!
-- 
Florian

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

Hi,

On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> 
> Patchset Summary:
>    Enhance a PCIe host controller driver.  Because of its unusual design
>    we are foced to change dev->dma_pfn_offset into a more general role
>    allowing multiple offsets.  See the 'v1' notes below for more info.

We are version 11 and counting, and it is not clear to me whether there 
is any chance of getting these patches reviewed and hopefully merged for 
the 5.10 merge window.

There are a lot of different files being touched, so what would be the 
ideal way of routing those changes towards inclusion?

Thanks!
-- 
Florian

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

Hi,

On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> 
> Patchset Summary:
>    Enhance a PCIe host controller driver.  Because of its unusual design
>    we are foced to change dev->dma_pfn_offset into a more general role
>    allowing multiple offsets.  See the 'v1' notes below for more info.

We are version 11 and counting, and it is not clear to me whether there 
is any chance of getting these patches reviewed and hopefully merged for 
the 5.10 merge window.

There are a lot of different files being touched, so what would be the 
ideal way of routing those changes towards inclusion?

Thanks!
-- 
Florian
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

Hi,

On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> 
> Patchset Summary:
>    Enhance a PCIe host controller driver.  Because of its unusual design
>    we are foced to change dev->dma_pfn_offset into a more general role
>    allowing multiple offsets.  See the 'v1' notes below for more info.

We are version 11 and counting, and it is not clear to me whether there 
is any chance of getting these patches reviewed and hopefully merged for 
the 5.10 merge window.

There are a lot of different files being touched, so what would be the 
ideal way of routing those changes towards inclusion?

Thanks!
-- 
Florian
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

Hi,

On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> 
> Patchset Summary:
>    Enhance a PCIe host controller driver.  Because of its unusual design
>    we are foced to change dev->dma_pfn_offset into a more general role
>    allowing multiple offsets.  See the 'v1' notes below for more info.

We are version 11 and counting, and it is not clear to me whether there 
is any chance of getting these patches reviewed and hopefully merged for 
the 5.10 merge window.

There are a lot of different files being touched, so what would be the 
ideal way of routing those changes towards inclusion?

Thanks!
-- 
Florian

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

Hi,

On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> 
> Patchset Summary:
>    Enhance a PCIe host controller driver.  Because of its unusual design
>    we are foced to change dev->dma_pfn_offset into a more general role
>    allowing multiple offsets.  See the 'v1' notes below for more info.

We are version 11 and counting, and it is not clear to me whether there 
is any chance of getting these patches reviewed and hopefully merged for 
the 5.10 merge window.

There are a lot of different files being touched, so what would be the 
ideal way of routing those changes towards inclusion?

Thanks!
-- 
Florian
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Christoph Hellwig]

On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> Hi,
>
> On 8/24/2020 12:30 PM, Jim Quinlan wrote:
>>
>> Patchset Summary:
>>    Enhance a PCIe host controller driver.  Because of its unusual design
>>    we are foced to change dev->dma_pfn_offset into a more general role
>>    allowing multiple offsets.  See the 'v1' notes below for more info.
>
> We are version 11 and counting, and it is not clear to me whether there is 
> any chance of getting these patches reviewed and hopefully merged for the 
> 5.10 merge window.
>
> There are a lot of different files being touched, so what would be the 
> ideal way of routing those changes towards inclusion?

FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
I have a bit of a backlog, but plan to review and if Jim is ok with that
apply the current version.

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Christoph Hellwig]

On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> Hi,
>
> On 8/24/2020 12:30 PM, Jim Quinlan wrote:
>>
>> Patchset Summary:
>>    Enhance a PCIe host controller driver.  Because of its unusual design
>>    we are foced to change dev->dma_pfn_offset into a more general role
>>    allowing multiple offsets.  See the 'v1' notes below for more info.
>
> We are version 11 and counting, and it is not clear to me whether there is 
> any chance of getting these patches reviewed and hopefully merged for the 
> 5.10 merge window.
>
> There are a lot of different files being touched, so what would be the 
> ideal way of routing those changes towards inclusion?

FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
I have a bit of a backlog, but plan to review and if Jim is ok with that
apply the current version.

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Christoph Hellwig]

On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> Hi,
>
> On 8/24/2020 12:30 PM, Jim Quinlan wrote:
>>
>> Patchset Summary:
>>    Enhance a PCIe host controller driver.  Because of its unusual design
>>    we are foced to change dev->dma_pfn_offset into a more general role
>>    allowing multiple offsets.  See the 'v1' notes below for more info.
>
> We are version 11 and counting, and it is not clear to me whether there is 
> any chance of getting these patches reviewed and hopefully merged for the 
> 5.10 merge window.
>
> There are a lot of different files being touched, so what would be the 
> ideal way of routing those changes towards inclusion?

FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
I have a bit of a backlog, but plan to review and if Jim is ok with that
apply the current version.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Christoph Hellwig]

On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> Hi,
>
> On 8/24/2020 12:30 PM, Jim Quinlan wrote:
>>
>> Patchset Summary:
>>    Enhance a PCIe host controller driver.  Because of its unusual design
>>    we are foced to change dev->dma_pfn_offset into a more general role
>>    allowing multiple offsets.  See the 'v1' notes below for more info.
>
> We are version 11 and counting, and it is not clear to me whether there is 
> any chance of getting these patches reviewed and hopefully merged for the 
> 5.10 merge window.
>
> There are a lot of different files being touched, so what would be the 
> ideal way of routing those changes towards inclusion?

FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
I have a bit of a backlog, but plan to review and if Jim is ok with that
apply the current version.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Christoph Hellwig]

On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> Hi,
>
> On 8/24/2020 12:30 PM, Jim Quinlan wrote:
>>
>> Patchset Summary:
>>    Enhance a PCIe host controller driver.  Because of its unusual design
>>    we are foced to change dev->dma_pfn_offset into a more general role
>>    allowing multiple offsets.  See the 'v1' notes below for more info.
>
> We are version 11 and counting, and it is not clear to me whether there is 
> any chance of getting these patches reviewed and hopefully merged for the 
> 5.10 merge window.
>
> There are a lot of different files being touched, so what would be the 
> ideal way of routing those changes towards inclusion?

FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
I have a bit of a backlog, but plan to review and if Jim is ok with that
apply the current version.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>
> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > Hi,
> >
> > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> >>
> >> Patchset Summary:
> >>    Enhance a PCIe host controller driver.  Because of its unusual design
> >>    we are foced to change dev->dma_pfn_offset into a more general role
> >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> >
> > We are version 11 and counting, and it is not clear to me whether there is
> > any chance of getting these patches reviewed and hopefully merged for the
> > 5.10 merge window.
> >
> > There are a lot of different files being touched, so what would be the
> > ideal way of routing those changes towards inclusion?
>
> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> I have a bit of a backlog, but plan to review and if Jim is ok with that
> apply the current version.
Sounds good to me.
Thanks, Jim

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>
> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > Hi,
> >
> > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> >>
> >> Patchset Summary:
> >>    Enhance a PCIe host controller driver.  Because of its unusual design
> >>    we are foced to change dev->dma_pfn_offset into a more general role
> >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> >
> > We are version 11 and counting, and it is not clear to me whether there is
> > any chance of getting these patches reviewed and hopefully merged for the
> > 5.10 merge window.
> >
> > There are a lot of different files being touched, so what would be the
> > ideal way of routing those changes towards inclusion?
>
> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> I have a bit of a backlog, but plan to review and if Jim is ok with that
> apply the current version.
Sounds good to me.
Thanks, Jim

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>
> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > Hi,
> >
> > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> >>
> >> Patchset Summary:
> >>    Enhance a PCIe host controller driver.  Because of its unusual design
> >>    we are foced to change dev->dma_pfn_offset into a more general role
> >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> >
> > We are version 11 and counting, and it is not clear to me whether there is
> > any chance of getting these patches reviewed and hopefully merged for the
> > 5.10 merge window.
> >
> > There are a lot of different files being touched, so what would be the
> > ideal way of routing those changes towards inclusion?
>
> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> I have a bit of a backlog, but plan to review and if Jim is ok with that
> apply the current version.
Sounds good to me.
Thanks, Jim
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan via iommu]

On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>
> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > Hi,
> >
> > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> >>
> >> Patchset Summary:
> >>    Enhance a PCIe host controller driver.  Because of its unusual design
> >>    we are foced to change dev->dma_pfn_offset into a more general role
> >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> >
> > We are version 11 and counting, and it is not clear to me whether there is
> > any chance of getting these patches reviewed and hopefully merged for the
> > 5.10 merge window.
> >
> > There are a lot of different files being touched, so what would be the
> > ideal way of routing those changes towards inclusion?
>
> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> I have a bit of a backlog, but plan to review and if Jim is ok with that
> apply the current version.
Sounds good to me.
Thanks, Jim
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>
> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > Hi,
> >
> > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> >>
> >> Patchset Summary:
> >>    Enhance a PCIe host controller driver.  Because of its unusual design
> >>    we are foced to change dev->dma_pfn_offset into a more general role
> >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> >
> > We are version 11 and counting, and it is not clear to me whether there is
> > any chance of getting these patches reviewed and hopefully merged for the
> > 5.10 merge window.
> >
> > There are a lot of different files being touched, so what would be the
> > ideal way of routing those changes towards inclusion?
>
> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> I have a bit of a backlog, but plan to review and if Jim is ok with that
> apply the current version.
Sounds good to me.
Thanks, Jim

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>
> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > Hi,
> >
> > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> >>
> >> Patchset Summary:
> >>    Enhance a PCIe host controller driver.  Because of its unusual design
> >>    we are foced to change dev->dma_pfn_offset into a more general role
> >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> >
> > We are version 11 and counting, and it is not clear to me whether there is
> > any chance of getting these patches reviewed and hopefully merged for the
> > 5.10 merge window.
> >
> > There are a lot of different files being touched, so what would be the
> > ideal way of routing those changes towards inclusion?
>
> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> I have a bit of a backlog, but plan to review and if Jim is ok with that
> apply the current version.
Sounds good to me.
Thanks, Jim
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

I've applied this to the dma-mapping tree.

I had to resolve a conflict in drivers/of/address.c with a recent
mainline commit.  I also applied the minor tweaks Andy pointed out
plus a few more style changes.  A real change is that I changed the
prototype for dma_copy_dma_range_map to require less boilerplate code.

The result is here:

    http://git.infradead.org/users/hch/dma-mapping.git/commitdiff/eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7

please double check that everyting works as expected.

I can cut a stable branch with this if you need it for other trees, but
I'd like to wait a few days to see if there is any fallout first.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

I've applied this to the dma-mapping tree.

I had to resolve a conflict in drivers/of/address.c with a recent
mainline commit.  I also applied the minor tweaks Andy pointed out
plus a few more style changes.  A real change is that I changed the
prototype for dma_copy_dma_range_map to require less boilerplate code.

The result is here:

    http://git.infradead.org/users/hch/dma-mapping.git/commitdiff/eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7

please double check that everyting works as expected.

I can cut a stable branch with this if you need it for other trees, but
I'd like to wait a few days to see if there is any fallout first.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

On Tue, Sep 1, 2020 at 4:24 AM Christoph Hellwig <hch@lst.de> wrote:
>
> I've applied this to the dma-mapping tree.
>
> I had to resolve a conflict in drivers/of/address.c with a recent
> mainline commit.  I also applied the minor tweaks Andy pointed out
> plus a few more style changes.  A real change is that I changed the
> prototype for dma_copy_dma_range_map to require less boilerplate code.
>
> The result is here:
>
>     http://git.infradead.org/users/hch/dma-mapping.git/commitdiff/eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
>
> please double check that everyting works as expected.
Tested-by: Jim Quinlan <james.quinlan@broadcom.com>

Thanks Christoph
Jim
>
> I can cut a stable branch with this if you need it for other trees, but
> I'd like to wait a few days to see if there is any fallout first.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan via iommu]

On Tue, Sep 1, 2020 at 4:24 AM Christoph Hellwig <hch@lst.de> wrote:
>
> I've applied this to the dma-mapping tree.
>
> I had to resolve a conflict in drivers/of/address.c with a recent
> mainline commit.  I also applied the minor tweaks Andy pointed out
> plus a few more style changes.  A real change is that I changed the
> prototype for dma_copy_dma_range_map to require less boilerplate code.
>
> The result is here:
>
>     http://git.infradead.org/users/hch/dma-mapping.git/commitdiff/eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
>
> please double check that everyting works as expected.
Tested-by: Jim Quinlan <james.quinlan@broadcom.com>

Thanks Christoph
Jim
>
> I can cut a stable branch with this if you need it for other trees, but
> I'd like to wait a few days to see if there is any fallout first.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

On Tue, Sep 1, 2020 at 4:24 AM Christoph Hellwig <hch@lst.de> wrote:
>
> I've applied this to the dma-mapping tree.
>
> I had to resolve a conflict in drivers/of/address.c with a recent
> mainline commit.  I also applied the minor tweaks Andy pointed out
> plus a few more style changes.  A real change is that I changed the
> prototype for dma_copy_dma_range_map to require less boilerplate code.
>
> The result is here:
>
>     http://git.infradead.org/users/hch/dma-mapping.git/commitdiff/eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
>
> please double check that everyting works as expected.
Tested-by: Jim Quinlan <james.quinlan@broadcom.com>

Thanks Christoph
Jim
>
> I can cut a stable branch with this if you need it for other trees, but
> I'd like to wait a few days to see if there is any fallout first.
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> The new field 'dma_range_map' in struct device is used to facilitate the
> use of single or multiple offsets between mapping regions of cpu addrs and
> dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> capable of holding a single uniform offset and had no region bounds
> checking.
> 
> The function of_dma_get_range() has been modified so that it takes a single
> argument -- the device node -- and returns a map, NULL, or an error code.
> The map is an array that holds the information regarding the DMA regions.
> Each range entry contains the address offset, the cpu_start address, the
> dma_start address, and the size of the region.
> 
> of_dma_configure() is the typical manner to set range offsets but there are
> a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> driver code.  These cases now invoke the function
> dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
> 
> Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> ---
>  arch/arm/include/asm/dma-mapping.h            | 10 +--
>  arch/arm/mach-keystone/keystone.c             | 17 +++--
>  arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
>  arch/x86/pci/sta2x11-fixup.c                  |  7 +-
>  drivers/acpi/arm64/iort.c                     |  5 +-
>  drivers/base/core.c                           |  2 +
>  drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
>  drivers/iommu/io-pgtable-arm.c                |  2 +-
>  .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
>  .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
>  drivers/of/address.c                          | 72 +++++++++----------
>  drivers/of/device.c                           | 43 ++++++-----
>  drivers/of/of_private.h                       | 10 +--
>  drivers/of/unittest.c                         | 34 ++++++---
>  drivers/remoteproc/remoteproc_core.c          |  8 ++-
>  .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
>  drivers/usb/core/message.c                    |  9 ++-
>  drivers/usb/core/usb.c                        |  7 +-
>  include/linux/device.h                        |  4 +-
>  include/linux/dma-direct.h                    |  8 +--
>  include/linux/dma-mapping.h                   | 36 ++++++++++
>  kernel/dma/coherent.c                         | 10 +--
>  kernel/dma/mapping.c                          | 66 +++++++++++++++++
>  23 files changed, 265 insertions(+), 115 deletions(-)

Apologies if this has already been reported or is known but this commit
is now in next-20200902 and it causes my Raspberry Pi 4 to no longer
make it to userspace, instead spewing mmc errors:

That commit causes my Raspberry Pi 4 to no longer make it to userspace,
instead spewing mmc errors:

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-00004-geef520b232c6-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 13:48:49 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
...
[    1.459752] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.500007] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.507454] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.615547] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.627537] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.665497] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.690601] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.697892] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.705173] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.713788] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.721228] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.732062] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.741828] ALSA device list:
[    1.744885]   No soundcards found.
[    1.748540] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.788865] random: fast init done
[    1.793489] mmc1: unrecognised SCR structure version 4
[    1.798814] mmc1: error -22 whilst initialising SD card
[    1.813969] mmc0: new high speed SDIO card at address 0001
[    1.883178] mmc1: unrecognised SCR structure version 2
[    1.888423] mmc1: error -22 whilst initialising SD card
[    1.964069] mmc1: unrecognised SCR structure version 4
[    1.969314] mmc1: error -22 whilst initialising SD card
[    2.061225] mmc1: unrecognised SCR structure version 4
[    2.066470] mmc1: error -22 whilst initialising SD card
[    3.160476] mmc1: unrecognised SCR structure version 4
[    3.165718] mmc1: error -22 whilst initialising SD card

This is what it looks like before that commit:

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-00003-g2281f797f552-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 14:10:30 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
...
[    1.459871] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.500017] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.507461] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.615565] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.627571] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.665549] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.690225] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.697516] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.704800] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.713412] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.720853] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.731686] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.741484] ALSA device list:
[    1.744517]   No soundcards found.
[    1.748194] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.788643] random: fast init done
[    1.805009] mmc0: new high speed SDIO card at address 0001
[    1.852305] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
[    1.859647] mmcblk1: mmc1:aaaa SK64G 59.5 GiB
[    1.866190]  mmcblk1: p1 p2
[    1.892559] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)

Bisect log in case it is relevant:

# bad: [4442749a203151a319a5bb8d0b983b84253a6931] Add linux-next specific files for 20200902
# good: [b765a32a2e9170702467747e290614be072c4f76] Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
git bisect start '4442749a203151a319a5bb8d0b983b84253a6931' 'b765a32a2e9170702467747e290614be072c4f76'
# bad: [501c4123d9ecf8e38425fa2dae591d87f18a33f9] Merge remote-tracking branch 'wireless-drivers-next/master' into master
git bisect bad 501c4123d9ecf8e38425fa2dae591d87f18a33f9
# bad: [3722ace45f7d9a37631b6f5b9c591ea1e118a077] Merge remote-tracking branch 'pstore/for-next/pstore' into master
git bisect bad 3722ace45f7d9a37631b6f5b9c591ea1e118a077
# bad: [bcd6576526884cbe2f0798f9196eb2fd96fadb2b] Merge remote-tracking branch 'imx-mxs/for-next' into master
git bisect bad bcd6576526884cbe2f0798f9196eb2fd96fadb2b
# good: [f51293edaaf354f6de8836390cd4bd0407380c8d] Merge remote-tracking branch 'staging.current/staging-linus' into master
git bisect good f51293edaaf354f6de8836390cd4bd0407380c8d
# bad: [2b7f5289d46652f343f51e1853326876114f8647] Merge remote-tracking branch 'arm-soc/for-next' into master
git bisect bad 2b7f5289d46652f343f51e1853326876114f8647
# good: [137aded9aad7803fa48e9a9538b359012715e171] Merge remote-tracking branch 'pidfd-fixes/fixes' into master
git bisect good 137aded9aad7803fa48e9a9538b359012715e171
# good: [f5cb1ada6f986019d24037d9329c99445fcdd50a] Merge remote-tracking branch 'kbuild/for-next' into master
git bisect good f5cb1ada6f986019d24037d9329c99445fcdd50a
# good: [2694ca73fa41491762825ae7bba9b14ab0762415] ARM: Document merges
git bisect good 2694ca73fa41491762825ae7bba9b14ab0762415
# bad: [192ec9acc561411c6ec5d35b3ad225d40c8c39bd] Merge remote-tracking branch 'dma-mapping/for-next' into master
git bisect bad 192ec9acc561411c6ec5d35b3ad225d40c8c39bd
# bad: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
git bisect bad eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
# good: [c6303ab9b91e7ca20a49ff494338309259ed7c65] arm64: mm: reserve per-numa CMA to localize coherent dma buffers
git bisect good c6303ab9b91e7ca20a49ff494338309259ed7c65
# good: [2281f797f5524abb8fff66bf8540b4f4687332a2] mm: cma: use CMA_MAX_NAME to define the length of cma name array
git bisect good 2281f797f5524abb8fff66bf8540b4f4687332a2
# first bad commit: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset

I am happy to provide further information for debugging if necessary.

Cheers,
Nathan
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> The new field 'dma_range_map' in struct device is used to facilitate the
> use of single or multiple offsets between mapping regions of cpu addrs and
> dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> capable of holding a single uniform offset and had no region bounds
> checking.
> 
> The function of_dma_get_range() has been modified so that it takes a single
> argument -- the device node -- and returns a map, NULL, or an error code.
> The map is an array that holds the information regarding the DMA regions.
> Each range entry contains the address offset, the cpu_start address, the
> dma_start address, and the size of the region.
> 
> of_dma_configure() is the typical manner to set range offsets but there are
> a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> driver code.  These cases now invoke the function
> dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
> 
> Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> ---
>  arch/arm/include/asm/dma-mapping.h            | 10 +--
>  arch/arm/mach-keystone/keystone.c             | 17 +++--
>  arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
>  arch/x86/pci/sta2x11-fixup.c                  |  7 +-
>  drivers/acpi/arm64/iort.c                     |  5 +-
>  drivers/base/core.c                           |  2 +
>  drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
>  drivers/iommu/io-pgtable-arm.c                |  2 +-
>  .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
>  .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
>  drivers/of/address.c                          | 72 +++++++++----------
>  drivers/of/device.c                           | 43 ++++++-----
>  drivers/of/of_private.h                       | 10 +--
>  drivers/of/unittest.c                         | 34 ++++++---
>  drivers/remoteproc/remoteproc_core.c          |  8 ++-
>  .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
>  drivers/usb/core/message.c                    |  9 ++-
>  drivers/usb/core/usb.c                        |  7 +-
>  include/linux/device.h                        |  4 +-
>  include/linux/dma-direct.h                    |  8 +--
>  include/linux/dma-mapping.h                   | 36 ++++++++++
>  kernel/dma/coherent.c                         | 10 +--
>  kernel/dma/mapping.c                          | 66 +++++++++++++++++
>  23 files changed, 265 insertions(+), 115 deletions(-)

Apologies if this has already been reported or is known but this commit
is now in next-20200902 and it causes my Raspberry Pi 4 to no longer
make it to userspace, instead spewing mmc errors:

That commit causes my Raspberry Pi 4 to no longer make it to userspace,
instead spewing mmc errors:

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-00004-geef520b232c6-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 13:48:49 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
...
[    1.459752] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.500007] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.507454] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.615547] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.627537] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.665497] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.690601] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.697892] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.705173] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.713788] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.721228] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.732062] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.741828] ALSA device list:
[    1.744885]   No soundcards found.
[    1.748540] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.788865] random: fast init done
[    1.793489] mmc1: unrecognised SCR structure version 4
[    1.798814] mmc1: error -22 whilst initialising SD card
[    1.813969] mmc0: new high speed SDIO card at address 0001
[    1.883178] mmc1: unrecognised SCR structure version 2
[    1.888423] mmc1: error -22 whilst initialising SD card
[    1.964069] mmc1: unrecognised SCR structure version 4
[    1.969314] mmc1: error -22 whilst initialising SD card
[    2.061225] mmc1: unrecognised SCR structure version 4
[    2.066470] mmc1: error -22 whilst initialising SD card
[    3.160476] mmc1: unrecognised SCR structure version 4
[    3.165718] mmc1: error -22 whilst initialising SD card

This is what it looks like before that commit:

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-00003-g2281f797f552-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 14:10:30 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
...
[    1.459871] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.500017] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.507461] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.615565] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.627571] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.665549] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.690225] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.697516] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.704800] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.713412] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.720853] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.731686] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.741484] ALSA device list:
[    1.744517]   No soundcards found.
[    1.748194] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.788643] random: fast init done
[    1.805009] mmc0: new high speed SDIO card at address 0001
[    1.852305] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
[    1.859647] mmcblk1: mmc1:aaaa SK64G 59.5 GiB
[    1.866190]  mmcblk1: p1 p2
[    1.892559] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)

Bisect log in case it is relevant:

# bad: [4442749a203151a319a5bb8d0b983b84253a6931] Add linux-next specific files for 20200902
# good: [b765a32a2e9170702467747e290614be072c4f76] Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
git bisect start '4442749a203151a319a5bb8d0b983b84253a6931' 'b765a32a2e9170702467747e290614be072c4f76'
# bad: [501c4123d9ecf8e38425fa2dae591d87f18a33f9] Merge remote-tracking branch 'wireless-drivers-next/master' into master
git bisect bad 501c4123d9ecf8e38425fa2dae591d87f18a33f9
# bad: [3722ace45f7d9a37631b6f5b9c591ea1e118a077] Merge remote-tracking branch 'pstore/for-next/pstore' into master
git bisect bad 3722ace45f7d9a37631b6f5b9c591ea1e118a077
# bad: [bcd6576526884cbe2f0798f9196eb2fd96fadb2b] Merge remote-tracking branch 'imx-mxs/for-next' into master
git bisect bad bcd6576526884cbe2f0798f9196eb2fd96fadb2b
# good: [f51293edaaf354f6de8836390cd4bd0407380c8d] Merge remote-tracking branch 'staging.current/staging-linus' into master
git bisect good f51293edaaf354f6de8836390cd4bd0407380c8d
# bad: [2b7f5289d46652f343f51e1853326876114f8647] Merge remote-tracking branch 'arm-soc/for-next' into master
git bisect bad 2b7f5289d46652f343f51e1853326876114f8647
# good: [137aded9aad7803fa48e9a9538b359012715e171] Merge remote-tracking branch 'pidfd-fixes/fixes' into master
git bisect good 137aded9aad7803fa48e9a9538b359012715e171
# good: [f5cb1ada6f986019d24037d9329c99445fcdd50a] Merge remote-tracking branch 'kbuild/for-next' into master
git bisect good f5cb1ada6f986019d24037d9329c99445fcdd50a
# good: [2694ca73fa41491762825ae7bba9b14ab0762415] ARM: Document merges
git bisect good 2694ca73fa41491762825ae7bba9b14ab0762415
# bad: [192ec9acc561411c6ec5d35b3ad225d40c8c39bd] Merge remote-tracking branch 'dma-mapping/for-next' into master
git bisect bad 192ec9acc561411c6ec5d35b3ad225d40c8c39bd
# bad: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
git bisect bad eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
# good: [c6303ab9b91e7ca20a49ff494338309259ed7c65] arm64: mm: reserve per-numa CMA to localize coherent dma buffers
git bisect good c6303ab9b91e7ca20a49ff494338309259ed7c65
# good: [2281f797f5524abb8fff66bf8540b4f4687332a2] mm: cma: use CMA_MAX_NAME to define the length of cma name array
git bisect good 2281f797f5524abb8fff66bf8540b4f4687332a2
# first bad commit: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset

I am happy to provide further information for debugging if necessary.

Cheers,
Nathan
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> The new field 'dma_range_map' in struct device is used to facilitate the
> use of single or multiple offsets between mapping regions of cpu addrs and
> dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> capable of holding a single uniform offset and had no region bounds
> checking.
> 
> The function of_dma_get_range() has been modified so that it takes a single
> argument -- the device node -- and returns a map, NULL, or an error code.
> The map is an array that holds the information regarding the DMA regions.
> Each range entry contains the address offset, the cpu_start address, the
> dma_start address, and the size of the region.
> 
> of_dma_configure() is the typical manner to set range offsets but there are
> a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> driver code.  These cases now invoke the function
> dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
> 
> Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> ---
>  arch/arm/include/asm/dma-mapping.h            | 10 +--
>  arch/arm/mach-keystone/keystone.c             | 17 +++--
>  arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
>  arch/x86/pci/sta2x11-fixup.c                  |  7 +-
>  drivers/acpi/arm64/iort.c                     |  5 +-
>  drivers/base/core.c                           |  2 +
>  drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
>  drivers/iommu/io-pgtable-arm.c                |  2 +-
>  .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
>  .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
>  drivers/of/address.c                          | 72 +++++++++----------
>  drivers/of/device.c                           | 43 ++++++-----
>  drivers/of/of_private.h                       | 10 +--
>  drivers/of/unittest.c                         | 34 ++++++---
>  drivers/remoteproc/remoteproc_core.c          |  8 ++-
>  .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
>  drivers/usb/core/message.c                    |  9 ++-
>  drivers/usb/core/usb.c                        |  7 +-
>  include/linux/device.h                        |  4 +-
>  include/linux/dma-direct.h                    |  8 +--
>  include/linux/dma-mapping.h                   | 36 ++++++++++
>  kernel/dma/coherent.c                         | 10 +--
>  kernel/dma/mapping.c                          | 66 +++++++++++++++++
>  23 files changed, 265 insertions(+), 115 deletions(-)

Apologies if this has already been reported or is known but this commit
is now in next-20200902 and it causes my Raspberry Pi 4 to no longer
make it to userspace, instead spewing mmc errors:

That commit causes my Raspberry Pi 4 to no longer make it to userspace,
instead spewing mmc errors:

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-00004-geef520b232c6-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 13:48:49 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
...
[    1.459752] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.500007] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.507454] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.615547] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.627537] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.665497] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.690601] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.697892] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.705173] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.713788] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.721228] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.732062] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.741828] ALSA device list:
[    1.744885]   No soundcards found.
[    1.748540] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.788865] random: fast init done
[    1.793489] mmc1: unrecognised SCR structure version 4
[    1.798814] mmc1: error -22 whilst initialising SD card
[    1.813969] mmc0: new high speed SDIO card at address 0001
[    1.883178] mmc1: unrecognised SCR structure version 2
[    1.888423] mmc1: error -22 whilst initialising SD card
[    1.964069] mmc1: unrecognised SCR structure version 4
[    1.969314] mmc1: error -22 whilst initialising SD card
[    2.061225] mmc1: unrecognised SCR structure version 4
[    2.066470] mmc1: error -22 whilst initialising SD card
[    3.160476] mmc1: unrecognised SCR structure version 4
[    3.165718] mmc1: error -22 whilst initialising SD card

This is what it looks like before that commit:

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-00003-g2281f797f552-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 14:10:30 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
...
[    1.459871] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.500017] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.507461] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.615565] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.627571] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.665549] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.690225] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.697516] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.704800] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.713412] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.720853] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.731686] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.741484] ALSA device list:
[    1.744517]   No soundcards found.
[    1.748194] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.788643] random: fast init done
[    1.805009] mmc0: new high speed SDIO card at address 0001
[    1.852305] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
[    1.859647] mmcblk1: mmc1:aaaa SK64G 59.5 GiB
[    1.866190]  mmcblk1: p1 p2
[    1.892559] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)

Bisect log in case it is relevant:

# bad: [4442749a203151a319a5bb8d0b983b84253a6931] Add linux-next specific files for 20200902
# good: [b765a32a2e9170702467747e290614be072c4f76] Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
git bisect start '4442749a203151a319a5bb8d0b983b84253a6931' 'b765a32a2e9170702467747e290614be072c4f76'
# bad: [501c4123d9ecf8e38425fa2dae591d87f18a33f9] Merge remote-tracking branch 'wireless-drivers-next/master' into master
git bisect bad 501c4123d9ecf8e38425fa2dae591d87f18a33f9
# bad: [3722ace45f7d9a37631b6f5b9c591ea1e118a077] Merge remote-tracking branch 'pstore/for-next/pstore' into master
git bisect bad 3722ace45f7d9a37631b6f5b9c591ea1e118a077
# bad: [bcd6576526884cbe2f0798f9196eb2fd96fadb2b] Merge remote-tracking branch 'imx-mxs/for-next' into master
git bisect bad bcd6576526884cbe2f0798f9196eb2fd96fadb2b
# good: [f51293edaaf354f6de8836390cd4bd0407380c8d] Merge remote-tracking branch 'staging.current/staging-linus' into master
git bisect good f51293edaaf354f6de8836390cd4bd0407380c8d
# bad: [2b7f5289d46652f343f51e1853326876114f8647] Merge remote-tracking branch 'arm-soc/for-next' into master
git bisect bad 2b7f5289d46652f343f51e1853326876114f8647
# good: [137aded9aad7803fa48e9a9538b359012715e171] Merge remote-tracking branch 'pidfd-fixes/fixes' into master
git bisect good 137aded9aad7803fa48e9a9538b359012715e171
# good: [f5cb1ada6f986019d24037d9329c99445fcdd50a] Merge remote-tracking branch 'kbuild/for-next' into master
git bisect good f5cb1ada6f986019d24037d9329c99445fcdd50a
# good: [2694ca73fa41491762825ae7bba9b14ab0762415] ARM: Document merges
git bisect good 2694ca73fa41491762825ae7bba9b14ab0762415
# bad: [192ec9acc561411c6ec5d35b3ad225d40c8c39bd] Merge remote-tracking branch 'dma-mapping/for-next' into master
git bisect bad 192ec9acc561411c6ec5d35b3ad225d40c8c39bd
# bad: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
git bisect bad eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
# good: [c6303ab9b91e7ca20a49ff494338309259ed7c65] arm64: mm: reserve per-numa CMA to localize coherent dma buffers
git bisect good c6303ab9b91e7ca20a49ff494338309259ed7c65
# good: [2281f797f5524abb8fff66bf8540b4f4687332a2] mm: cma: use CMA_MAX_NAME to define the length of cma name array
git bisect good 2281f797f5524abb8fff66bf8540b4f4687332a2
# first bad commit: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset

I am happy to provide further information for debugging if necessary.

Cheers,
Nathan
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

On Wed, Sep 2, 2020 at 5:53 PM Nathan Chancellor
<natechancellor@gmail.com> wrote:
>
> On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> > The new field 'dma_range_map' in struct device is used to facilitate the
> > use of single or multiple offsets between mapping regions of cpu addrs and
> > dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> > capable of holding a single uniform offset and had no region bounds
> > checking.
> >
> > The function of_dma_get_range() has been modified so that it takes a single
> > argument -- the device node -- and returns a map, NULL, or an error code.
> > The map is an array that holds the information regarding the DMA regions.
> > Each range entry contains the address offset, the cpu_start address, the
> > dma_start address, and the size of the region.
> >
> > of_dma_configure() is the typical manner to set range offsets but there are
> > a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> > driver code.  These cases now invoke the function
> > dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
> >
> > Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> > ---
> >  arch/arm/include/asm/dma-mapping.h            | 10 +--
> >  arch/arm/mach-keystone/keystone.c             | 17 +++--
> >  arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
> >  arch/x86/pci/sta2x11-fixup.c                  |  7 +-
> >  drivers/acpi/arm64/iort.c                     |  5 +-
> >  drivers/base/core.c                           |  2 +
> >  drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
> >  drivers/iommu/io-pgtable-arm.c                |  2 +-
> >  .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
> >  .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
> >  drivers/of/address.c                          | 72 +++++++++----------
> >  drivers/of/device.c                           | 43 ++++++-----
> >  drivers/of/of_private.h                       | 10 +--
> >  drivers/of/unittest.c                         | 34 ++++++---
> >  drivers/remoteproc/remoteproc_core.c          |  8 ++-
> >  .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
> >  drivers/usb/core/message.c                    |  9 ++-
> >  drivers/usb/core/usb.c                        |  7 +-
> >  include/linux/device.h                        |  4 +-
> >  include/linux/dma-direct.h                    |  8 +--
> >  include/linux/dma-mapping.h                   | 36 ++++++++++
> >  kernel/dma/coherent.c                         | 10 +--
> >  kernel/dma/mapping.c                          | 66 +++++++++++++++++
> >  23 files changed, 265 insertions(+), 115 deletions(-)
>
> Apologies if this has already been reported or is known but this commit
> is now in next-20200902 and it causes my Raspberry Pi 4 to no longer
> make it to userspace, instead spewing mmc errors:
>
> That commit causes my Raspberry Pi 4 to no longer make it to userspace,
> instead spewing mmc errors:
>
> [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> [    0.000000] Linux version 5.9.0-rc3-00004-geef520b232c6-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 13:48:49 MST 2020
> [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> ...
> [    1.459752] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> [    1.500007] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> [    1.507454] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> [    1.615547] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> [    1.627537] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> [    1.665497] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> [    1.690601] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> [    1.697892] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.705173] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.713788] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> [    1.721228] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.732062] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> [    1.741828] ALSA device list:
> [    1.744885]   No soundcards found.
> [    1.748540] Waiting for root device PARTUUID=45a8dd8a-02...
> [    1.788865] random: fast init done
> [    1.793489] mmc1: unrecognised SCR structure version 4
> [    1.798814] mmc1: error -22 whilst initialising SD card
> [    1.813969] mmc0: new high speed SDIO card at address 0001
> [    1.883178] mmc1: unrecognised SCR structure version 2
> [    1.888423] mmc1: error -22 whilst initialising SD card
> [    1.964069] mmc1: unrecognised SCR structure version 4
> [    1.969314] mmc1: error -22 whilst initialising SD card
> [    2.061225] mmc1: unrecognised SCR structure version 4
> [    2.066470] mmc1: error -22 whilst initialising SD card
> [    3.160476] mmc1: unrecognised SCR structure version 4
> [    3.165718] mmc1: error -22 whilst initialising SD card
>
> This is what it looks like before that commit:
>
> [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> [    0.000000] Linux version 5.9.0-rc3-00003-g2281f797f552-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 14:10:30 MST 2020
> [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> ...
> [    1.459871] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> [    1.500017] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> [    1.507461] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> [    1.615565] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> [    1.627571] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> [    1.665549] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> [    1.690225] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> [    1.697516] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.704800] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.713412] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> [    1.720853] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.731686] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> [    1.741484] ALSA device list:
> [    1.744517]   No soundcards found.
> [    1.748194] Waiting for root device PARTUUID=45a8dd8a-02...
> [    1.788643] random: fast init done
> [    1.805009] mmc0: new high speed SDIO card at address 0001
> [    1.852305] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
> [    1.859647] mmcblk1: mmc1:aaaa SK64G 59.5 GiB
> [    1.866190]  mmcblk1: p1 p2
> [    1.892559] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)
>
> Bisect log in case it is relevant:
>
> # bad: [4442749a203151a319a5bb8d0b983b84253a6931] Add linux-next specific files for 20200902
> # good: [b765a32a2e9170702467747e290614be072c4f76] Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
> git bisect start '4442749a203151a319a5bb8d0b983b84253a6931' 'b765a32a2e9170702467747e290614be072c4f76'
> # bad: [501c4123d9ecf8e38425fa2dae591d87f18a33f9] Merge remote-tracking branch 'wireless-drivers-next/master' into master
> git bisect bad 501c4123d9ecf8e38425fa2dae591d87f18a33f9
> # bad: [3722ace45f7d9a37631b6f5b9c591ea1e118a077] Merge remote-tracking branch 'pstore/for-next/pstore' into master
> git bisect bad 3722ace45f7d9a37631b6f5b9c591ea1e118a077
> # bad: [bcd6576526884cbe2f0798f9196eb2fd96fadb2b] Merge remote-tracking branch 'imx-mxs/for-next' into master
> git bisect bad bcd6576526884cbe2f0798f9196eb2fd96fadb2b
> # good: [f51293edaaf354f6de8836390cd4bd0407380c8d] Merge remote-tracking branch 'staging.current/staging-linus' into master
> git bisect good f51293edaaf354f6de8836390cd4bd0407380c8d
> # bad: [2b7f5289d46652f343f51e1853326876114f8647] Merge remote-tracking branch 'arm-soc/for-next' into master
> git bisect bad 2b7f5289d46652f343f51e1853326876114f8647
> # good: [137aded9aad7803fa48e9a9538b359012715e171] Merge remote-tracking branch 'pidfd-fixes/fixes' into master
> git bisect good 137aded9aad7803fa48e9a9538b359012715e171
> # good: [f5cb1ada6f986019d24037d9329c99445fcdd50a] Merge remote-tracking branch 'kbuild/for-next' into master
> git bisect good f5cb1ada6f986019d24037d9329c99445fcdd50a
> # good: [2694ca73fa41491762825ae7bba9b14ab0762415] ARM: Document merges
> git bisect good 2694ca73fa41491762825ae7bba9b14ab0762415
> # bad: [192ec9acc561411c6ec5d35b3ad225d40c8c39bd] Merge remote-tracking branch 'dma-mapping/for-next' into master
> git bisect bad 192ec9acc561411c6ec5d35b3ad225d40c8c39bd
> # bad: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
> git bisect bad eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
> # good: [c6303ab9b91e7ca20a49ff494338309259ed7c65] arm64: mm: reserve per-numa CMA to localize coherent dma buffers
> git bisect good c6303ab9b91e7ca20a49ff494338309259ed7c65
> # good: [2281f797f5524abb8fff66bf8540b4f4687332a2] mm: cma: use CMA_MAX_NAME to define the length of cma name array
> git bisect good 2281f797f5524abb8fff66bf8540b4f4687332a2
> # first bad commit: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
>
> I am happy to provide further information for debugging if necessary.
Hello Nathan,

Can you tell me how much memory your RPI has and if all of it is
accessible by the PCIe device?  Could you also please include the DTS
of the PCIe node?  IIRC, the RPI firmware does some mangling of the
PCIe DT before Linux boots -- could you describe what is going on
there?
Finally, can you attach the text of the full boot log?

Thanks,
Jim
>
> Cheers,
> Nathan
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan via iommu]

On Wed, Sep 2, 2020 at 5:53 PM Nathan Chancellor
<natechancellor@gmail.com> wrote:
>
> On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> > The new field 'dma_range_map' in struct device is used to facilitate the
> > use of single or multiple offsets between mapping regions of cpu addrs and
> > dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> > capable of holding a single uniform offset and had no region bounds
> > checking.
> >
> > The function of_dma_get_range() has been modified so that it takes a single
> > argument -- the device node -- and returns a map, NULL, or an error code.
> > The map is an array that holds the information regarding the DMA regions.
> > Each range entry contains the address offset, the cpu_start address, the
> > dma_start address, and the size of the region.
> >
> > of_dma_configure() is the typical manner to set range offsets but there are
> > a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> > driver code.  These cases now invoke the function
> > dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
> >
> > Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> > ---
> >  arch/arm/include/asm/dma-mapping.h            | 10 +--
> >  arch/arm/mach-keystone/keystone.c             | 17 +++--
> >  arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
> >  arch/x86/pci/sta2x11-fixup.c                  |  7 +-
> >  drivers/acpi/arm64/iort.c                     |  5 +-
> >  drivers/base/core.c                           |  2 +
> >  drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
> >  drivers/iommu/io-pgtable-arm.c                |  2 +-
> >  .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
> >  .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
> >  drivers/of/address.c                          | 72 +++++++++----------
> >  drivers/of/device.c                           | 43 ++++++-----
> >  drivers/of/of_private.h                       | 10 +--
> >  drivers/of/unittest.c                         | 34 ++++++---
> >  drivers/remoteproc/remoteproc_core.c          |  8 ++-
> >  .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
> >  drivers/usb/core/message.c                    |  9 ++-
> >  drivers/usb/core/usb.c                        |  7 +-
> >  include/linux/device.h                        |  4 +-
> >  include/linux/dma-direct.h                    |  8 +--
> >  include/linux/dma-mapping.h                   | 36 ++++++++++
> >  kernel/dma/coherent.c                         | 10 +--
> >  kernel/dma/mapping.c                          | 66 +++++++++++++++++
> >  23 files changed, 265 insertions(+), 115 deletions(-)
>
> Apologies if this has already been reported or is known but this commit
> is now in next-20200902 and it causes my Raspberry Pi 4 to no longer
> make it to userspace, instead spewing mmc errors:
>
> That commit causes my Raspberry Pi 4 to no longer make it to userspace,
> instead spewing mmc errors:
>
> [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> [    0.000000] Linux version 5.9.0-rc3-00004-geef520b232c6-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 13:48:49 MST 2020
> [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> ...
> [    1.459752] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> [    1.500007] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> [    1.507454] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> [    1.615547] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> [    1.627537] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> [    1.665497] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> [    1.690601] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> [    1.697892] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.705173] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.713788] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> [    1.721228] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.732062] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> [    1.741828] ALSA device list:
> [    1.744885]   No soundcards found.
> [    1.748540] Waiting for root device PARTUUID=45a8dd8a-02...
> [    1.788865] random: fast init done
> [    1.793489] mmc1: unrecognised SCR structure version 4
> [    1.798814] mmc1: error -22 whilst initialising SD card
> [    1.813969] mmc0: new high speed SDIO card at address 0001
> [    1.883178] mmc1: unrecognised SCR structure version 2
> [    1.888423] mmc1: error -22 whilst initialising SD card
> [    1.964069] mmc1: unrecognised SCR structure version 4
> [    1.969314] mmc1: error -22 whilst initialising SD card
> [    2.061225] mmc1: unrecognised SCR structure version 4
> [    2.066470] mmc1: error -22 whilst initialising SD card
> [    3.160476] mmc1: unrecognised SCR structure version 4
> [    3.165718] mmc1: error -22 whilst initialising SD card
>
> This is what it looks like before that commit:
>
> [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> [    0.000000] Linux version 5.9.0-rc3-00003-g2281f797f552-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 14:10:30 MST 2020
> [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> ...
> [    1.459871] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> [    1.500017] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> [    1.507461] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> [    1.615565] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> [    1.627571] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> [    1.665549] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> [    1.690225] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> [    1.697516] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.704800] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.713412] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> [    1.720853] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.731686] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> [    1.741484] ALSA device list:
> [    1.744517]   No soundcards found.
> [    1.748194] Waiting for root device PARTUUID=45a8dd8a-02...
> [    1.788643] random: fast init done
> [    1.805009] mmc0: new high speed SDIO card at address 0001
> [    1.852305] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
> [    1.859647] mmcblk1: mmc1:aaaa SK64G 59.5 GiB
> [    1.866190]  mmcblk1: p1 p2
> [    1.892559] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)
>
> Bisect log in case it is relevant:
>
> # bad: [4442749a203151a319a5bb8d0b983b84253a6931] Add linux-next specific files for 20200902
> # good: [b765a32a2e9170702467747e290614be072c4f76] Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
> git bisect start '4442749a203151a319a5bb8d0b983b84253a6931' 'b765a32a2e9170702467747e290614be072c4f76'
> # bad: [501c4123d9ecf8e38425fa2dae591d87f18a33f9] Merge remote-tracking branch 'wireless-drivers-next/master' into master
> git bisect bad 501c4123d9ecf8e38425fa2dae591d87f18a33f9
> # bad: [3722ace45f7d9a37631b6f5b9c591ea1e118a077] Merge remote-tracking branch 'pstore/for-next/pstore' into master
> git bisect bad 3722ace45f7d9a37631b6f5b9c591ea1e118a077
> # bad: [bcd6576526884cbe2f0798f9196eb2fd96fadb2b] Merge remote-tracking branch 'imx-mxs/for-next' into master
> git bisect bad bcd6576526884cbe2f0798f9196eb2fd96fadb2b
> # good: [f51293edaaf354f6de8836390cd4bd0407380c8d] Merge remote-tracking branch 'staging.current/staging-linus' into master
> git bisect good f51293edaaf354f6de8836390cd4bd0407380c8d
> # bad: [2b7f5289d46652f343f51e1853326876114f8647] Merge remote-tracking branch 'arm-soc/for-next' into master
> git bisect bad 2b7f5289d46652f343f51e1853326876114f8647
> # good: [137aded9aad7803fa48e9a9538b359012715e171] Merge remote-tracking branch 'pidfd-fixes/fixes' into master
> git bisect good 137aded9aad7803fa48e9a9538b359012715e171
> # good: [f5cb1ada6f986019d24037d9329c99445fcdd50a] Merge remote-tracking branch 'kbuild/for-next' into master
> git bisect good f5cb1ada6f986019d24037d9329c99445fcdd50a
> # good: [2694ca73fa41491762825ae7bba9b14ab0762415] ARM: Document merges
> git bisect good 2694ca73fa41491762825ae7bba9b14ab0762415
> # bad: [192ec9acc561411c6ec5d35b3ad225d40c8c39bd] Merge remote-tracking branch 'dma-mapping/for-next' into master
> git bisect bad 192ec9acc561411c6ec5d35b3ad225d40c8c39bd
> # bad: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
> git bisect bad eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
> # good: [c6303ab9b91e7ca20a49ff494338309259ed7c65] arm64: mm: reserve per-numa CMA to localize coherent dma buffers
> git bisect good c6303ab9b91e7ca20a49ff494338309259ed7c65
> # good: [2281f797f5524abb8fff66bf8540b4f4687332a2] mm: cma: use CMA_MAX_NAME to define the length of cma name array
> git bisect good 2281f797f5524abb8fff66bf8540b4f4687332a2
> # first bad commit: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
>
> I am happy to provide further information for debugging if necessary.
Hello Nathan,

Can you tell me how much memory your RPI has and if all of it is
accessible by the PCIe device?  Could you also please include the DTS
of the PCIe node?  IIRC, the RPI firmware does some mangling of the
PCIe DT before Linux boots -- could you describe what is going on
there?
Finally, can you attach the text of the full boot log?

Thanks,
Jim
>
> Cheers,
> Nathan
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

On Wed, Sep 2, 2020 at 5:53 PM Nathan Chancellor
<natechancellor@gmail.com> wrote:
>
> On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> > The new field 'dma_range_map' in struct device is used to facilitate the
> > use of single or multiple offsets between mapping regions of cpu addrs and
> > dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> > capable of holding a single uniform offset and had no region bounds
> > checking.
> >
> > The function of_dma_get_range() has been modified so that it takes a single
> > argument -- the device node -- and returns a map, NULL, or an error code.
> > The map is an array that holds the information regarding the DMA regions.
> > Each range entry contains the address offset, the cpu_start address, the
> > dma_start address, and the size of the region.
> >
> > of_dma_configure() is the typical manner to set range offsets but there are
> > a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> > driver code.  These cases now invoke the function
> > dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
> >
> > Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> > ---
> >  arch/arm/include/asm/dma-mapping.h            | 10 +--
> >  arch/arm/mach-keystone/keystone.c             | 17 +++--
> >  arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
> >  arch/x86/pci/sta2x11-fixup.c                  |  7 +-
> >  drivers/acpi/arm64/iort.c                     |  5 +-
> >  drivers/base/core.c                           |  2 +
> >  drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
> >  drivers/iommu/io-pgtable-arm.c                |  2 +-
> >  .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
> >  .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
> >  drivers/of/address.c                          | 72 +++++++++----------
> >  drivers/of/device.c                           | 43 ++++++-----
> >  drivers/of/of_private.h                       | 10 +--
> >  drivers/of/unittest.c                         | 34 ++++++---
> >  drivers/remoteproc/remoteproc_core.c          |  8 ++-
> >  .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
> >  drivers/usb/core/message.c                    |  9 ++-
> >  drivers/usb/core/usb.c                        |  7 +-
> >  include/linux/device.h                        |  4 +-
> >  include/linux/dma-direct.h                    |  8 +--
> >  include/linux/dma-mapping.h                   | 36 ++++++++++
> >  kernel/dma/coherent.c                         | 10 +--
> >  kernel/dma/mapping.c                          | 66 +++++++++++++++++
> >  23 files changed, 265 insertions(+), 115 deletions(-)
>
> Apologies if this has already been reported or is known but this commit
> is now in next-20200902 and it causes my Raspberry Pi 4 to no longer
> make it to userspace, instead spewing mmc errors:
>
> That commit causes my Raspberry Pi 4 to no longer make it to userspace,
> instead spewing mmc errors:
>
> [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> [    0.000000] Linux version 5.9.0-rc3-00004-geef520b232c6-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 13:48:49 MST 2020
> [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> ...
> [    1.459752] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> [    1.500007] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> [    1.507454] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> [    1.615547] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> [    1.627537] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> [    1.665497] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> [    1.690601] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> [    1.697892] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.705173] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.713788] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> [    1.721228] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.732062] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> [    1.741828] ALSA device list:
> [    1.744885]   No soundcards found.
> [    1.748540] Waiting for root device PARTUUID=45a8dd8a-02...
> [    1.788865] random: fast init done
> [    1.793489] mmc1: unrecognised SCR structure version 4
> [    1.798814] mmc1: error -22 whilst initialising SD card
> [    1.813969] mmc0: new high speed SDIO card at address 0001
> [    1.883178] mmc1: unrecognised SCR structure version 2
> [    1.888423] mmc1: error -22 whilst initialising SD card
> [    1.964069] mmc1: unrecognised SCR structure version 4
> [    1.969314] mmc1: error -22 whilst initialising SD card
> [    2.061225] mmc1: unrecognised SCR structure version 4
> [    2.066470] mmc1: error -22 whilst initialising SD card
> [    3.160476] mmc1: unrecognised SCR structure version 4
> [    3.165718] mmc1: error -22 whilst initialising SD card
>
> This is what it looks like before that commit:
>
> [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> [    0.000000] Linux version 5.9.0-rc3-00003-g2281f797f552-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 14:10:30 MST 2020
> [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> ...
> [    1.459871] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> [    1.500017] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> [    1.507461] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> [    1.615565] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> [    1.627571] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> [    1.665549] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> [    1.690225] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> [    1.697516] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.704800] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.713412] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> [    1.720853] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> [    1.731686] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> [    1.741484] ALSA device list:
> [    1.744517]   No soundcards found.
> [    1.748194] Waiting for root device PARTUUID=45a8dd8a-02...
> [    1.788643] random: fast init done
> [    1.805009] mmc0: new high speed SDIO card at address 0001
> [    1.852305] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
> [    1.859647] mmcblk1: mmc1:aaaa SK64G 59.5 GiB
> [    1.866190]  mmcblk1: p1 p2
> [    1.892559] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)
>
> Bisect log in case it is relevant:
>
> # bad: [4442749a203151a319a5bb8d0b983b84253a6931] Add linux-next specific files for 20200902
> # good: [b765a32a2e9170702467747e290614be072c4f76] Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
> git bisect start '4442749a203151a319a5bb8d0b983b84253a6931' 'b765a32a2e9170702467747e290614be072c4f76'
> # bad: [501c4123d9ecf8e38425fa2dae591d87f18a33f9] Merge remote-tracking branch 'wireless-drivers-next/master' into master
> git bisect bad 501c4123d9ecf8e38425fa2dae591d87f18a33f9
> # bad: [3722ace45f7d9a37631b6f5b9c591ea1e118a077] Merge remote-tracking branch 'pstore/for-next/pstore' into master
> git bisect bad 3722ace45f7d9a37631b6f5b9c591ea1e118a077
> # bad: [bcd6576526884cbe2f0798f9196eb2fd96fadb2b] Merge remote-tracking branch 'imx-mxs/for-next' into master
> git bisect bad bcd6576526884cbe2f0798f9196eb2fd96fadb2b
> # good: [f51293edaaf354f6de8836390cd4bd0407380c8d] Merge remote-tracking branch 'staging.current/staging-linus' into master
> git bisect good f51293edaaf354f6de8836390cd4bd0407380c8d
> # bad: [2b7f5289d46652f343f51e1853326876114f8647] Merge remote-tracking branch 'arm-soc/for-next' into master
> git bisect bad 2b7f5289d46652f343f51e1853326876114f8647
> # good: [137aded9aad7803fa48e9a9538b359012715e171] Merge remote-tracking branch 'pidfd-fixes/fixes' into master
> git bisect good 137aded9aad7803fa48e9a9538b359012715e171
> # good: [f5cb1ada6f986019d24037d9329c99445fcdd50a] Merge remote-tracking branch 'kbuild/for-next' into master
> git bisect good f5cb1ada6f986019d24037d9329c99445fcdd50a
> # good: [2694ca73fa41491762825ae7bba9b14ab0762415] ARM: Document merges
> git bisect good 2694ca73fa41491762825ae7bba9b14ab0762415
> # bad: [192ec9acc561411c6ec5d35b3ad225d40c8c39bd] Merge remote-tracking branch 'dma-mapping/for-next' into master
> git bisect bad 192ec9acc561411c6ec5d35b3ad225d40c8c39bd
> # bad: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
> git bisect bad eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
> # good: [c6303ab9b91e7ca20a49ff494338309259ed7c65] arm64: mm: reserve per-numa CMA to localize coherent dma buffers
> git bisect good c6303ab9b91e7ca20a49ff494338309259ed7c65
> # good: [2281f797f5524abb8fff66bf8540b4f4687332a2] mm: cma: use CMA_MAX_NAME to define the length of cma name array
> git bisect good 2281f797f5524abb8fff66bf8540b4f4687332a2
> # first bad commit: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
>
> I am happy to provide further information for debugging if necessary.
Hello Nathan,

Can you tell me how much memory your RPI has and if all of it is
accessible by the PCIe device?  Could you also please include the DTS
of the PCIe node?  IIRC, the RPI firmware does some mangling of the
PCIe DT before Linux boots -- could you describe what is going on
there?
Finally, can you attach the text of the full boot log?

Thanks,
Jim
>
> Cheers,
> Nathan
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

[-- Attachment #1: Type: text/plain, Size: 10977 bytes --]

On Wed, Sep 02, 2020 at 06:11:08PM -0400, Jim Quinlan wrote:
> On Wed, Sep 2, 2020 at 5:53 PM Nathan Chancellor
> <natechancellor@gmail.com> wrote:
> >
> > On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> > > The new field 'dma_range_map' in struct device is used to facilitate the
> > > use of single or multiple offsets between mapping regions of cpu addrs and
> > > dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> > > capable of holding a single uniform offset and had no region bounds
> > > checking.
> > >
> > > The function of_dma_get_range() has been modified so that it takes a single
> > > argument -- the device node -- and returns a map, NULL, or an error code.
> > > The map is an array that holds the information regarding the DMA regions.
> > > Each range entry contains the address offset, the cpu_start address, the
> > > dma_start address, and the size of the region.
> > >
> > > of_dma_configure() is the typical manner to set range offsets but there are
> > > a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> > > driver code.  These cases now invoke the function
> > > dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
> > >
> > > Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> > > ---
> > >  arch/arm/include/asm/dma-mapping.h            | 10 +--
> > >  arch/arm/mach-keystone/keystone.c             | 17 +++--
> > >  arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
> > >  arch/x86/pci/sta2x11-fixup.c                  |  7 +-
> > >  drivers/acpi/arm64/iort.c                     |  5 +-
> > >  drivers/base/core.c                           |  2 +
> > >  drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
> > >  drivers/iommu/io-pgtable-arm.c                |  2 +-
> > >  .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
> > >  .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
> > >  drivers/of/address.c                          | 72 +++++++++----------
> > >  drivers/of/device.c                           | 43 ++++++-----
> > >  drivers/of/of_private.h                       | 10 +--
> > >  drivers/of/unittest.c                         | 34 ++++++---
> > >  drivers/remoteproc/remoteproc_core.c          |  8 ++-
> > >  .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
> > >  drivers/usb/core/message.c                    |  9 ++-
> > >  drivers/usb/core/usb.c                        |  7 +-
> > >  include/linux/device.h                        |  4 +-
> > >  include/linux/dma-direct.h                    |  8 +--
> > >  include/linux/dma-mapping.h                   | 36 ++++++++++
> > >  kernel/dma/coherent.c                         | 10 +--
> > >  kernel/dma/mapping.c                          | 66 +++++++++++++++++
> > >  23 files changed, 265 insertions(+), 115 deletions(-)
> >
> > Apologies if this has already been reported or is known but this commit
> > is now in next-20200902 and it causes my Raspberry Pi 4 to no longer
> > make it to userspace, instead spewing mmc errors:
> >
> > That commit causes my Raspberry Pi 4 to no longer make it to userspace,
> > instead spewing mmc errors:
> >
> > [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> > [    0.000000] Linux version 5.9.0-rc3-00004-geef520b232c6-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 13:48:49 MST 2020
> > [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> > ...
> > [    1.459752] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> > [    1.500007] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> > [    1.507454] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> > [    1.615547] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> > [    1.627537] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> > [    1.665497] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> > [    1.690601] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> > [    1.697892] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.705173] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.713788] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> > [    1.721228] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.732062] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> > [    1.741828] ALSA device list:
> > [    1.744885]   No soundcards found.
> > [    1.748540] Waiting for root device PARTUUID=45a8dd8a-02...
> > [    1.788865] random: fast init done
> > [    1.793489] mmc1: unrecognised SCR structure version 4
> > [    1.798814] mmc1: error -22 whilst initialising SD card
> > [    1.813969] mmc0: new high speed SDIO card at address 0001
> > [    1.883178] mmc1: unrecognised SCR structure version 2
> > [    1.888423] mmc1: error -22 whilst initialising SD card
> > [    1.964069] mmc1: unrecognised SCR structure version 4
> > [    1.969314] mmc1: error -22 whilst initialising SD card
> > [    2.061225] mmc1: unrecognised SCR structure version 4
> > [    2.066470] mmc1: error -22 whilst initialising SD card
> > [    3.160476] mmc1: unrecognised SCR structure version 4
> > [    3.165718] mmc1: error -22 whilst initialising SD card
> >
> > This is what it looks like before that commit:
> >
> > [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> > [    0.000000] Linux version 5.9.0-rc3-00003-g2281f797f552-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 14:10:30 MST 2020
> > [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> > ...
> > [    1.459871] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> > [    1.500017] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> > [    1.507461] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> > [    1.615565] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> > [    1.627571] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> > [    1.665549] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> > [    1.690225] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> > [    1.697516] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.704800] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.713412] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> > [    1.720853] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.731686] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> > [    1.741484] ALSA device list:
> > [    1.744517]   No soundcards found.
> > [    1.748194] Waiting for root device PARTUUID=45a8dd8a-02...
> > [    1.788643] random: fast init done
> > [    1.805009] mmc0: new high speed SDIO card at address 0001
> > [    1.852305] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
> > [    1.859647] mmcblk1: mmc1:aaaa SK64G 59.5 GiB
> > [    1.866190]  mmcblk1: p1 p2
> > [    1.892559] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)
> >
> > Bisect log in case it is relevant:
> >
> > # bad: [4442749a203151a319a5bb8d0b983b84253a6931] Add linux-next specific files for 20200902
> > # good: [b765a32a2e9170702467747e290614be072c4f76] Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
> > git bisect start '4442749a203151a319a5bb8d0b983b84253a6931' 'b765a32a2e9170702467747e290614be072c4f76'
> > # bad: [501c4123d9ecf8e38425fa2dae591d87f18a33f9] Merge remote-tracking branch 'wireless-drivers-next/master' into master
> > git bisect bad 501c4123d9ecf8e38425fa2dae591d87f18a33f9
> > # bad: [3722ace45f7d9a37631b6f5b9c591ea1e118a077] Merge remote-tracking branch 'pstore/for-next/pstore' into master
> > git bisect bad 3722ace45f7d9a37631b6f5b9c591ea1e118a077
> > # bad: [bcd6576526884cbe2f0798f9196eb2fd96fadb2b] Merge remote-tracking branch 'imx-mxs/for-next' into master
> > git bisect bad bcd6576526884cbe2f0798f9196eb2fd96fadb2b
> > # good: [f51293edaaf354f6de8836390cd4bd0407380c8d] Merge remote-tracking branch 'staging.current/staging-linus' into master
> > git bisect good f51293edaaf354f6de8836390cd4bd0407380c8d
> > # bad: [2b7f5289d46652f343f51e1853326876114f8647] Merge remote-tracking branch 'arm-soc/for-next' into master
> > git bisect bad 2b7f5289d46652f343f51e1853326876114f8647
> > # good: [137aded9aad7803fa48e9a9538b359012715e171] Merge remote-tracking branch 'pidfd-fixes/fixes' into master
> > git bisect good 137aded9aad7803fa48e9a9538b359012715e171
> > # good: [f5cb1ada6f986019d24037d9329c99445fcdd50a] Merge remote-tracking branch 'kbuild/for-next' into master
> > git bisect good f5cb1ada6f986019d24037d9329c99445fcdd50a
> > # good: [2694ca73fa41491762825ae7bba9b14ab0762415] ARM: Document merges
> > git bisect good 2694ca73fa41491762825ae7bba9b14ab0762415
> > # bad: [192ec9acc561411c6ec5d35b3ad225d40c8c39bd] Merge remote-tracking branch 'dma-mapping/for-next' into master
> > git bisect bad 192ec9acc561411c6ec5d35b3ad225d40c8c39bd
> > # bad: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
> > git bisect bad eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
> > # good: [c6303ab9b91e7ca20a49ff494338309259ed7c65] arm64: mm: reserve per-numa CMA to localize coherent dma buffers
> > git bisect good c6303ab9b91e7ca20a49ff494338309259ed7c65
> > # good: [2281f797f5524abb8fff66bf8540b4f4687332a2] mm: cma: use CMA_MAX_NAME to define the length of cma name array
> > git bisect good 2281f797f5524abb8fff66bf8540b4f4687332a2
> > # first bad commit: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
> >
> > I am happy to provide further information for debugging if necessary.
> Hello Nathan,
> 
> Can you tell me how much memory your RPI has and if all of it is

This is the 4GB version.

> accessible by the PCIe device?  Could you also please include the DTS
> of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> PCIe DT before Linux boots -- could you describe what is going on
> there?

Unfortunately, I am not familiar with how to get this information. If
you could provide some instructions for how to do so, I am more than
happy to. I am not very knowleagable about the inner working of the Pi,
I mainly use it as a test platform for making sure that LLVM does not
cause problems on real devices.

> Finally, can you attach the text of the full boot log?

I have attached a working and broken boot log. Thank you for the quick
response!

Cheers,
Nathan

[-- Attachment #2: pi-boot-bad.txt --]
[-- Type: text/plain, Size: 24428 bytes --]

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-next-20200902-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 15:17:43 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000037400000, size 64 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] NUMA: NODE_DATA [mem 0xfb81f100-0xfb820fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x000000003fffffff]
[    0.000000]   DMA32    [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000003b3fffff]
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: EL2 vector hardening
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] ARM_SMCCC_ARCH_WORKAROUND_1 missing from firmware
[    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 996912
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:  dma.dmachans=0x71f5 bcm2709.boardrev=0xc03112 bcm2709.serial=0xb78d398 bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:60:6C:87 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=ttyS1,115200 console=tty1 root=PARTUUID=45a8dd8a-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait plymouth.ignore-serial-consoles
[    0.000000] Kernel parameter elevator= does not have any effect anymore.
[    0.000000] Please use sysfs to set IO scheduler for individual devices.
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x33400000-0x37400000] (64MB)
[    0.000000] Memory: 3812260K/4050944K available (14524K kernel code, 2636K rwdata, 7540K rodata, 1600K init, 615K bss, 173148K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] random: get_random_bytes called from start_kernel+0x1cc/0x3f4 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 54.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123 ns
[    0.000006] sched_clock: 56 bits at 54MHz, resolution 18ns, wraps every 4398046511102ns
[    0.000196] Console: colour dummy device 80x25
[    0.000680] printk: console [tty1] enabled
[    0.000779] Calibrating delay loop (skipped), value calculated using timer frequency.. 108.00 BogoMIPS (lpj=216000)
[    0.000809] pid_max: default: 32768 minimum: 301
[    0.000896] LSM: Security Framework initializing
[    0.001029] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.001115] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.002874] rcu: Hierarchical SRCU implementation.
[    0.005100] EFI services will not be available.
[    0.005396] smp: Bringing up secondary CPUs ...
[    0.005904] Detected PIPT I-cache on CPU1
[    0.005963] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[    0.006505] Detected PIPT I-cache on CPU2
[    0.006544] CPU2: Booted secondary processor 0x0000000002 [0x410fd083]
[    0.007060] Detected PIPT I-cache on CPU3
[    0.007098] CPU3: Booted secondary processor 0x0000000003 [0x410fd083]
[    0.007182] smp: Brought up 1 node, 4 CPUs
[    0.007251] SMP: Total of 4 processors activated.
[    0.007267] CPU features: detected: 32-bit EL0 Support
[    0.007283] CPU features: detected: CRC32 instructions
[    0.007299] CPU features: detected: 32-bit EL1 Support
[    0.030649] CPU: All CPU(s) started at EL2
[    0.030710] alternatives: patching kernel code
[    0.032692] devtmpfs: initialized
[    0.037827] KASLR enabled
[    0.038277] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.038312] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.040163] pinctrl core: initialized pinctrl subsystem
[    0.041597] DMI not present or invalid.
[    0.042151] NET: Registered protocol family 16
[    0.043673] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.043826] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.044107] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.044163] audit: initializing netlink subsys (disabled)
[    0.044352] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[    0.045463] thermal_sys: Registered thermal governor 'step_wise'
[    0.045470] thermal_sys: Registered thermal governor 'power_allocator'
[    0.045807] cpuidle: using governor menu
[    0.046145] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.046246] ASID allocator initialised with 32768 entries
[    0.047838] Serial: AMBA PL011 UART driver
[    0.083072] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.083108] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.083126] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.083143] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.084507] cryptd: max_cpu_qlen set to 1000
[    0.087192] ACPI: Interpreter disabled.
[    0.089180] iommu: Default domain type: Translated 
[    0.089450] vgaarb: loaded
[    0.089800] SCSI subsystem initialized
[    0.090203] usbcore: registered new interface driver usbfs
[    0.090263] usbcore: registered new interface driver hub
[    0.090343] usbcore: registered new device driver usb
[    0.090520] usb_phy_generic phy: supply vcc not found, using dummy regulator
[    0.091346] pps_core: LinuxPPS API ver. 1 registered
[    0.091364] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.091394] PTP clock support registered
[    0.091552] EDAC MC: Ver: 3.0.0
[    0.092935] FPGA manager framework
[    0.093050] Advanced Linux Sound Architecture Driver Initialized.
[    0.093880] clocksource: Switched to clocksource arch_sys_counter
[    0.094112] VFS: Disk quotas dquot_6.6.0
[    0.094190] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.094401] pnp: PnP ACPI: disabled
[    0.100876] NET: Registered protocol family 2
[    0.101384] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.101444] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.101652] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.102173] TCP: Hash tables configured (established 32768 bind 32768)
[    0.102384] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102454] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102668] NET: Registered protocol family 1
[    0.103174] RPC: Registered named UNIX socket transport module.
[    0.103193] RPC: Registered udp transport module.
[    0.103206] RPC: Registered tcp transport module.
[    0.103220] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.103241] PCI: CLS 0 bytes, default 64
[    0.104437] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[    0.104591] kvm [1]: IPA Size Limit: 44bits
[    0.105723] kvm [1]: vgic interrupt IRQ1
[    0.105910] kvm [1]: Hyp mode initialized successfully
[    0.107799] Initialise system trusted keyrings
[    0.107975] workingset: timestamp_bits=44 max_order=20 bucket_order=0
[    0.114780] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.115569] NFS: Registering the id_resolver key type
[    0.115614] Key type id_resolver registered
[    0.115629] Key type id_legacy registered
[    0.115747] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.115926] 9p: Installing v9fs 9p2000 file system support
[    0.177452] Key type asymmetric registered
[    0.177472] Asymmetric key parser 'x509' registered
[    0.177527] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.177547] io scheduler mq-deadline registered
[    0.177560] io scheduler kyber registered
[    0.190013] EINJ: ACPI disabled.
[    0.208072] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.210194] printk: console [ttyS1] disabled
[    0.210297] fe215040.serial: ttyS1 at MMIO 0xfe215040 (irq = 18, base_baud = 24999999) is a 16550
[    1.115569] printk: console [ttyS1] enabled
[    1.121263] SuperH (H)SCI(F) driver initialized
[    1.126522] msm_serial: driver initialized
[    1.132301] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    1.146949] loop: module loaded
[    1.151363] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    1.159671] megasas: 07.714.04.00-rc1
[    1.168505] libphy: Fixed MDIO Bus: probed
[    1.174118] tun: Universal TUN/TAP device driver, 1.6
[    1.180090] thunder_xcv, ver 1.0
[    1.183434] thunder_bgx, ver 1.0
[    1.186768] nicpf, ver 1.0
[    1.190746] hclge is initializing
[    1.194178] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    1.201546] hns3: Copyright (c) 2017 Huawei Corporation.
[    1.207017] e1000: Intel(R) PRO/1000 Network Driver
[    1.211992] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    1.217901] e1000e: Intel(R) PRO/1000 Network Driver
[    1.222962] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.229030] igb: Intel(R) Gigabit Ethernet Network Driver
[    1.234535] igb: Copyright (c) 2007-2014 Intel Corporation.
[    1.240248] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    1.246633] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    1.253038] sky2: driver version 1.30
[    1.257975] VFIO - User Level meta-driver version: 0.3
[    1.265257] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.271935] ehci-pci: EHCI PCI platform driver
[    1.276511] ehci-platform: EHCI generic platform driver
[    1.281987] ehci-orion: EHCI orion driver
[    1.286186] ehci-exynos: EHCI Exynos driver
[    1.290552] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.296868] ohci-pci: OHCI PCI platform driver
[    1.301436] ohci-platform: OHCI generic platform driver
[    1.306891] ohci-exynos: OHCI Exynos driver
[    1.311695] usbcore: registered new interface driver usb-storage
[    1.320736] i2c /dev entries driver
[    1.329662] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    1.338167] sdhci: Secure Digital Host Controller Interface driver
[    1.344475] sdhci: Copyright(c) Pierre Ossman
[    1.349524] Synopsys Designware Multimedia Card Interface Driver
[    1.356759] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.375177] ledtrig-cpu: registered to indicate activity on CPUs
[    1.383060] usbcore: registered new interface driver usbhid
[    1.388747] usbhid: USB HID core driver
[    1.393728] bcm2835-mbox fe00b880.mailbox: mailbox enabled
[    1.403725] NET: Registered protocol family 17
[    1.408495] 9pnet: Installing 9P2000 support
[    1.412912] Key type dns_resolver registered
[    1.417632] registered taskstats version 1
[    1.421822] Loading compiled-in X.509 certificates
[    1.431630] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 16, base_baud = 0) is a PL011 rev2
[    1.440890] serial serial0: tty port ttyAMA0 registered
[    1.462581] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.503538] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.510983] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.619098] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.631122] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.669127] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.694102] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.701389] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.708672] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.717293] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.724735] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.735564] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.745322] ALSA device list:
[    1.748390]   No soundcards found.
[    1.752060] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.792429] random: fast init done
[    1.797033] mmc1: unrecognised SCR structure version 4
[    1.802357] mmc1: error -22 whilst initialising SD card
[    1.817686] mmc0: new high speed SDIO card at address 0001
[    1.886941] mmc1: unrecognised SCR structure version 2
[    1.892189] mmc1: error -22 whilst initialising SD card
[    1.967866] mmc1: unrecognised SCR structure version 4
[    1.973113] mmc1: error -22 whilst initialising SD card
[    2.065006] mmc1: unrecognised SCR structure version 4
[    2.070252] mmc1: error -22 whilst initialising SD card
[    3.161820] mmc1: unrecognised SCR structure version 4
[    3.167064] mmc1: error -22 whilst initialising SD card
[    3.237380] mmc1: unrecognised SCR structure version 4
[    3.242625] mmc1: error -22 whilst initialising SD card
[    3.318280] mmc1: unrecognised SCR structure version 4
[    3.323525] mmc1: error -22 whilst initialising SD card
[    3.415425] mmc1: unrecognised SCR structure version 4
[    3.420669] mmc1: error -22 whilst initialising SD card
[    4.505842] mmc1: unrecognised SCR structure version 4
[    4.511090] mmc1: error -22 whilst initialising SD card
[    4.581408] mmc1: unrecognised SCR structure version 4
[    4.586652] mmc1: error -22 whilst initialising SD card
[    4.662344] mmc1: unrecognised SCR structure version 4
[    4.667588] mmc1: error -22 whilst initialising SD card
[    4.759556] mmc1: unrecognised SCR structure version 4
[    4.764800] mmc1: error -22 whilst initialising SD card
[    5.851270] mmc1: unrecognised SCR structure version 4
[    5.856517] mmc1: error -22 whilst initialising SD card
[    5.926793] mmc1: unrecognised SCR structure version 4
[    5.932038] mmc1: error -22 whilst initialising SD card
[    6.007687] mmc1: unrecognised SCR structure version 4
[    6.012930] mmc1: error -22 whilst initialising SD card
[    6.104759] mmc1: unrecognised SCR structure version 4
[    6.110006] mmc1: error -22 whilst initialising SD card
[    7.193797] mmc1: unrecognised SCR structure version 4
[    7.199043] mmc1: error -22 whilst initialising SD card
[    7.269330] mmc1: unrecognised SCR structure version 4
[    7.274574] mmc1: error -22 whilst initialising SD card
[    7.350192] mmc1: unrecognised SCR structure version 4
[    7.355435] mmc1: error -22 whilst initialising SD card
[    7.447355] mmc1: unrecognised SCR structure version 4
[    7.452598] mmc1: error -22 whilst initialising SD card
[    8.537815] mmc1: unrecognised SCR structure version 4
[    8.543060] mmc1: error -22 whilst initialising SD card
[    8.613360] mmc1: unrecognised SCR structure version 4
[    8.618604] mmc1: error -22 whilst initialising SD card
[    8.694255] mmc1: unrecognised SCR structure version 4
[    8.699499] mmc1: error -22 whilst initialising SD card
[    8.791399] mmc1: unrecognised SCR structure version 4
[    8.796642] mmc1: error -22 whilst initialising SD card
[    9.881840] mmc1: unrecognised SCR structure version 4
[    9.887088] mmc1: error -22 whilst initialising SD card
[    9.957402] mmc1: unrecognised SCR structure version 4
[    9.962646] mmc1: error -22 whilst initialising SD card
[   10.038303] mmc1: unrecognised SCR structure version 4
[   10.043547] mmc1: error -22 whilst initialising SD card
[   10.135464] mmc1: unrecognised SCR structure version 4
[   10.140707] mmc1: error -22 whilst initialising SD card
[   11.225815] mmc1: unrecognised SCR structure version 4
[   11.231062] mmc1: error -22 whilst initialising SD card
[   11.301373] mmc1: unrecognised SCR structure version 4
[   11.306617] mmc1: error -22 whilst initialising SD card
[   11.382264] mmc1: unrecognised SCR structure version 4
[   11.387507] mmc1: error -22 whilst initialising SD card
[   11.479430] mmc1: unrecognised SCR structure version 4
[   11.484673] mmc1: error -22 whilst initialising SD card
[   12.569813] mmc1: unrecognised SCR structure version 4
[   12.575059] mmc1: error -22 whilst initialising SD card
[   12.645354] mmc1: unrecognised SCR structure version 4
[   12.650598] mmc1: error -22 whilst initialising SD card
[   12.726254] mmc1: unrecognised SCR structure version 4
[   12.731497] mmc1: error -22 whilst initialising SD card
[   12.823417] mmc1: unrecognised SCR structure version 4
[   12.828661] mmc1: error -22 whilst initialising SD card
[   13.913815] mmc1: unrecognised SCR structure version 4
[   13.919059] mmc1: error -22 whilst initialising SD card
[   13.989375] mmc1: unrecognised SCR structure version 4
[   13.994620] mmc1: error -22 whilst initialising SD card
[   14.070314] mmc1: unrecognised SCR structure version 4
[   14.075557] mmc1: error -22 whilst initialising SD card
[   14.167479] mmc1: unrecognised SCR structure version 4
[   14.172722] mmc1: error -22 whilst initialising SD card
[   15.257839] mmc1: unrecognised SCR structure version 4
[   15.263086] mmc1: error -22 whilst initialising SD card
[   15.333397] mmc1: unrecognised SCR structure version 4
[   15.338642] mmc1: error -22 whilst initialising SD card
[   15.414259] mmc1: unrecognised SCR structure version 4
[   15.419500] mmc1: error -22 whilst initialising SD card
[   15.511418] mmc1: unrecognised SCR structure version 4
[   15.516662] mmc1: error -22 whilst initialising SD card
[   16.601839] mmc1: unrecognised SCR structure version 4
[   16.607086] mmc1: error -22 whilst initialising SD card
[   16.677414] mmc1: unrecognised SCR structure version 4
[   16.682658] mmc1: error -22 whilst initialising SD card
[   16.758280] mmc1: unrecognised SCR structure version 4
[   16.763524] mmc1: error -22 whilst initialising SD card
[   16.855411] mmc1: unrecognised SCR structure version 4
[   16.860657] mmc1: error -22 whilst initialising SD card
[   17.945838] mmc1: unrecognised SCR structure version 4
[   17.951084] mmc1: error -22 whilst initialising SD card
[   18.022951] mmc1: unrecognised SCR structure version 4
[   18.028196] mmc1: error -22 whilst initialising SD card
[   18.103836] mmc1: unrecognised SCR structure version 4
[   18.109079] mmc1: error -22 whilst initialising SD card
[   18.200919] mmc1: unrecognised SCR structure version 4
[   18.206165] mmc1: error -22 whilst initialising SD card
[   19.289820] mmc1: unrecognised SCR structure version 4
[   19.295068] mmc1: error -22 whilst initialising SD card
[   19.365404] mmc1: unrecognised SCR structure version 4
[   19.370648] mmc1: error -22 whilst initialising SD card
[   19.446261] mmc1: unrecognised SCR structure version 4
[   19.451504] mmc1: error -22 whilst initialising SD card
[   19.543415] mmc1: unrecognised SCR structure version 4
[   19.548658] mmc1: error -22 whilst initialising SD card
[   20.633815] mmc1: unrecognised SCR structure version 4
[   20.639065] mmc1: error -22 whilst initialising SD card
[   20.709375] mmc1: unrecognised SCR structure version 4
[   20.714619] mmc1: error -22 whilst initialising SD card
[   20.790313] mmc1: unrecognised SCR structure version 4
[   20.795557] mmc1: error -22 whilst initialising SD card
[   20.887466] mmc1: unrecognised SCR structure version 4
[   20.892710] mmc1: error -22 whilst initialising SD card
[   21.977834] mmc1: unrecognised SCR structure version 4
[   21.983081] mmc1: error -22 whilst initialising SD card
[   22.053392] mmc1: unrecognised SCR structure version 4
[   22.058637] mmc1: error -22 whilst initialising SD card
[   22.134244] mmc1: unrecognised SCR structure version 4
[   22.139488] mmc1: error -22 whilst initialising SD card
[   22.231412] mmc1: unrecognised SCR structure version 4
[   22.236656] mmc1: error -22 whilst initialising SD card
[   23.321810] mmc1: unrecognised SCR structure version 4
[   23.327056] mmc1: error -22 whilst initialising SD card
[   23.397372] mmc1: unrecognised SCR structure version 4
[   23.402617] mmc1: error -22 whilst initialising SD card
[   23.478248] mmc1: unrecognised SCR structure version 4
[   23.483492] mmc1: error -22 whilst initialising SD card
[   23.575406] mmc1: unrecognised SCR structure version 4
[   23.580650] mmc1: error -22 whilst initialising SD card
[   24.665836] mmc1: unrecognised SCR structure version 4
[   24.671080] mmc1: error -22 whilst initialising SD card
[   24.741396] mmc1: unrecognised SCR structure version 4
[   24.746641] mmc1: error -22 whilst initialising SD card
[   24.822248] mmc1: unrecognised SCR structure version 4
[   24.827492] mmc1: error -22 whilst initialising SD card
[   24.919406] mmc1: unrecognised SCR structure version 4
[   24.924650] mmc1: error -22 whilst initialising SD card
[   26.009818] mmc1: unrecognised SCR structure version 4
[   26.015064] mmc1: error -22 whilst initialising SD card
[   26.085377] mmc1: unrecognised SCR structure version 4
[   26.090621] mmc1: error -22 whilst initialising SD card
[   26.166270] mmc1: unrecognised SCR structure version 4
[   26.171514] mmc1: error -22 whilst initialising SD card
[   26.263190] mmc1: unrecognised SCR structure version 4
[   26.268434] mmc1: error -22 whilst initialising SD card
[   27.353799] mmc1: unrecognised SCR structure version 4
[   27.359045] mmc1: error -22 whilst initialising SD card
[   27.429363] mmc1: unrecognised SCR structure version 4
[   27.434607] mmc1: error -22 whilst initialising SD card
[   27.512614] mmc1: unrecognised SCR structure version 4
[   27.517858] mmc1: error -22 whilst initialising SD card
[   27.609855] mmc1: unrecognised SCR structure version 4
[   27.615102] mmc1: error -22 whilst initialising SD card
[   28.697337] mmc1: unrecognised SCR structure version 4
[   28.702583] mmc1: error -22 whilst initialising SD card
[   28.772895] mmc1: unrecognised SCR structure version 4
[   28.778139] mmc1: error -22 whilst initialising SD card
[   28.854654] mmc1: unrecognised SCR structure version 4
[   28.859900] mmc1: error -22 whilst initialising SD card
[   28.951809] mmc1: unrecognised SCR structure version 4
[   28.957049] mmc1: error -22 whilst initialising SD card
[   30.041823] mmc1: unrecognised SCR structure version 4
[   30.047070] mmc1: error -22 whilst initialising SD card
[   30.117387] mmc1: unrecognised SCR structure version 4
[   30.122631] mmc1: error -22 whilst initialising SD card
[   30.198244] mmc1: unrecognised SCR structure version 4
[   30.203488] mmc1: error -22 whilst initialising SD card
[   30.295391] mmc1: unrecognised SCR structure version 4
[   30.300638] mmc1: error -22 whilst initialising SD card

[-- Attachment #3: pi-boot-good.txt --]
[-- Type: text/plain, Size: 15985 bytes --]

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-next-20200902-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 15:28:17 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000037400000, size 64 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] NUMA: NODE_DATA [mem 0xfb81f100-0xfb820fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x000000003fffffff]
[    0.000000]   DMA32    [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000003b3fffff]
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: EL2 vector hardening
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] ARM_SMCCC_ARCH_WORKAROUND_1 missing from firmware
[    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 996912
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:  dma.dmachans=0x71f5 bcm2709.boardrev=0xc03112 bcm2709.serial=0xb78d398 bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:60:6C:87 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=ttyS1,115200 console=tty1 root=PARTUUID=45a8dd8a-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait plymouth.ignore-serial-consoles
[    0.000000] Kernel parameter elevator= does not have any effect anymore.
[    0.000000] Please use sysfs to set IO scheduler for individual devices.
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x33400000-0x37400000] (64MB)
[    0.000000] Memory: 3812260K/4050944K available (14524K kernel code, 2636K rwdata, 7540K rodata, 1600K init, 615K bss, 173148K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] random: get_random_bytes called from start_kernel+0x1cc/0x3f4 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 54.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123 ns
[    0.000006] sched_clock: 56 bits at 54MHz, resolution 18ns, wraps every 4398046511102ns
[    0.000195] Console: colour dummy device 80x25
[    0.000678] printk: console [tty1] enabled
[    0.000778] Calibrating delay loop (skipped), value calculated using timer frequency.. 108.00 BogoMIPS (lpj=216000)
[    0.000808] pid_max: default: 32768 minimum: 301
[    0.000894] LSM: Security Framework initializing
[    0.001030] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.001116] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.002878] rcu: Hierarchical SRCU implementation.
[    0.005048] EFI services will not be available.
[    0.005338] smp: Bringing up secondary CPUs ...
[    0.005844] Detected PIPT I-cache on CPU1
[    0.005903] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[    0.006454] Detected PIPT I-cache on CPU2
[    0.006493] CPU2: Booted secondary processor 0x0000000002 [0x410fd083]
[    0.007012] Detected PIPT I-cache on CPU3
[    0.007051] CPU3: Booted secondary processor 0x0000000003 [0x410fd083]
[    0.007133] smp: Brought up 1 node, 4 CPUs
[    0.007198] SMP: Total of 4 processors activated.
[    0.007214] CPU features: detected: 32-bit EL0 Support
[    0.007229] CPU features: detected: CRC32 instructions
[    0.007245] CPU features: detected: 32-bit EL1 Support
[    0.030492] CPU: All CPU(s) started at EL2
[    0.030550] alternatives: patching kernel code
[    0.032524] devtmpfs: initialized
[    0.037648] KASLR enabled
[    0.038091] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.038125] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.039986] pinctrl core: initialized pinctrl subsystem
[    0.041393] DMI not present or invalid.
[    0.041944] NET: Registered protocol family 16
[    0.043458] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.043610] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.043887] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.043939] audit: initializing netlink subsys (disabled)
[    0.044132] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[    0.045217] thermal_sys: Registered thermal governor 'step_wise'
[    0.045225] thermal_sys: Registered thermal governor 'power_allocator'
[    0.045555] cpuidle: using governor menu
[    0.045877] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.045978] ASID allocator initialised with 32768 entries
[    0.047544] Serial: AMBA PL011 UART driver
[    0.082787] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.082822] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.082840] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.082856] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.084212] cryptd: max_cpu_qlen set to 1000
[    0.086893] ACPI: Interpreter disabled.
[    0.088884] iommu: Default domain type: Translated 
[    0.089151] vgaarb: loaded
[    0.089505] SCSI subsystem initialized
[    0.089896] usbcore: registered new interface driver usbfs
[    0.089955] usbcore: registered new interface driver hub
[    0.090035] usbcore: registered new device driver usb
[    0.090208] usb_phy_generic phy: supply vcc not found, using dummy regulator
[    0.091009] pps_core: LinuxPPS API ver. 1 registered
[    0.091026] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.091055] PTP clock support registered
[    0.091238] EDAC MC: Ver: 3.0.0
[    0.092605] FPGA manager framework
[    0.092719] Advanced Linux Sound Architecture Driver Initialized.
[    0.093558] clocksource: Switched to clocksource arch_sys_counter
[    0.093791] VFS: Disk quotas dquot_6.6.0
[    0.093865] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.094072] pnp: PnP ACPI: disabled
[    0.100547] NET: Registered protocol family 2
[    0.101063] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.101121] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.101328] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.101841] TCP: Hash tables configured (established 32768 bind 32768)
[    0.102055] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102123] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102329] NET: Registered protocol family 1
[    0.102824] RPC: Registered named UNIX socket transport module.
[    0.102842] RPC: Registered udp transport module.
[    0.102855] RPC: Registered tcp transport module.
[    0.102868] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.102888] PCI: CLS 0 bytes, default 64
[    0.104073] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[    0.104224] kvm [1]: IPA Size Limit: 44bits
[    0.105353] kvm [1]: vgic interrupt IRQ1
[    0.105504] kvm [1]: Hyp mode initialized successfully
[    0.107404] Initialise system trusted keyrings
[    0.107581] workingset: timestamp_bits=44 max_order=20 bucket_order=0
[    0.114285] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.115072] NFS: Registering the id_resolver key type
[    0.115111] Key type id_resolver registered
[    0.115125] Key type id_legacy registered
[    0.115242] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.115415] 9p: Installing v9fs 9p2000 file system support
[    0.178130] Key type asymmetric registered
[    0.178149] Asymmetric key parser 'x509' registered
[    0.178205] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.178225] io scheduler mq-deadline registered
[    0.178239] io scheduler kyber registered
[    0.190560] EINJ: ACPI disabled.
[    0.208427] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.210486] printk: console [ttyS1] disabled
[    0.210587] fe215040.serial: ttyS1 at MMIO 0xfe215040 (irq = 18, base_baud = 24999999) is a 16550
[    1.115809] printk: console [ttyS1] enabled
[    1.121521] SuperH (H)SCI(F) driver initialized
[    1.126760] msm_serial: driver initialized
[    1.132523] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    1.147108] loop: module loaded
[    1.151491] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    1.159778] megasas: 07.714.04.00-rc1
[    1.168551] libphy: Fixed MDIO Bus: probed
[    1.174162] tun: Universal TUN/TAP device driver, 1.6
[    1.180136] thunder_xcv, ver 1.0
[    1.183480] thunder_bgx, ver 1.0
[    1.186813] nicpf, ver 1.0
[    1.190780] hclge is initializing
[    1.194215] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    1.201583] hns3: Copyright (c) 2017 Huawei Corporation.
[    1.207048] e1000: Intel(R) PRO/1000 Network Driver
[    1.212023] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    1.217916] e1000e: Intel(R) PRO/1000 Network Driver
[    1.222976] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.229054] igb: Intel(R) Gigabit Ethernet Network Driver
[    1.234559] igb: Copyright (c) 2007-2014 Intel Corporation.
[    1.240268] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    1.246653] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    1.253048] sky2: driver version 1.30
[    1.257983] VFIO - User Level meta-driver version: 0.3
[    1.265221] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.271898] ehci-pci: EHCI PCI platform driver
[    1.276471] ehci-platform: EHCI generic platform driver
[    1.281934] ehci-orion: EHCI orion driver
[    1.286133] ehci-exynos: EHCI Exynos driver
[    1.290505] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.296820] ohci-pci: OHCI PCI platform driver
[    1.301386] ohci-platform: OHCI generic platform driver
[    1.306836] ohci-exynos: OHCI Exynos driver
[    1.311632] usbcore: registered new interface driver usb-storage
[    1.320631] i2c /dev entries driver
[    1.329492] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    1.337993] sdhci: Secure Digital Host Controller Interface driver
[    1.344303] sdhci: Copyright(c) Pierre Ossman
[    1.349337] Synopsys Designware Multimedia Card Interface Driver
[    1.356563] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.374853] ledtrig-cpu: registered to indicate activity on CPUs
[    1.382707] usbcore: registered new interface driver usbhid
[    1.388393] usbhid: USB HID core driver
[    1.393348] bcm2835-mbox fe00b880.mailbox: mailbox enabled
[    1.403299] NET: Registered protocol family 17
[    1.408070] 9pnet: Installing 9P2000 support
[    1.412487] Key type dns_resolver registered
[    1.417194] registered taskstats version 1
[    1.421390] Loading compiled-in X.509 certificates
[    1.431180] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 16, base_baud = 0) is a PL011 rev2
[    1.440427] serial serial0: tty port ttyAMA0 registered
[    1.461855] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.505592] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.513027] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.621089] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.633123] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.671111] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.696128] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.703412] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.710695] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.719313] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.726750] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.737585] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.747325] ALSA device list:
[    1.750387]   No soundcards found.
[    1.754070] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.793779] random: fast init done
[    1.810764] mmc0: new high speed SDIO card at address 0001
[    1.858064] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
[    1.865420] mmcblk1: mmc1:aaaa SK64G 59.5 GiB 
[    1.871966]  mmcblk1: p1 p2
[    1.897160] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)
[    1.905492] VFS: Mounted root (ext4 filesystem) readonly on device 179:2.
[    1.913206] devtmpfs: mounted
[    1.917652] Freeing unused kernel memory: 1600K
[    1.933776] Run /sbin/init as init process
[    2.314733] systemd[1]: System time before build time, advancing clock.
[    2.661797] NET: Registered protocol family 10
[    2.667365] Segment Routing with IPv6
[    2.730477] systemd[1]: systemd 241 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid)
[    2.752685] systemd[1]: Detected architecture arm64.
[    2.792164] systemd[1]: Set hostname to <raspberrypi>.
[    3.017532] systemd[1]: File /lib/systemd/system/systemd-journald.service:12 configures an IP firewall (IPAddressDeny=any), but the local system does not support BPF/cgroup based firewalling.
[    3.034988] systemd[1]: Proceeding WITHOUT firewalling in effect! (This warning is only shown for the first loaded unit using IP firewalling.)
[    3.309831] random: systemd: uninitialized urandom read (16 bytes read)
[    3.321952] random: systemd: uninitialized urandom read (16 bytes read)
[    3.329307] systemd[1]: Listening on Journal Socket (/dev/log).
[    3.335968] random: systemd: uninitialized urandom read (16 bytes read)
[    3.345903] systemd[1]: Created slice system-serial\x2dgetty.slice.
[    3.353269] systemd[1]: Listening on Journal Audit Socket.
[    3.360213] systemd[1]: Created slice system-getty.slice.
[    3.366141] systemd[1]: Reached target Swap.

[-- Attachment #4: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

[-- Attachment #1: Type: text/plain, Size: 10977 bytes --]

On Wed, Sep 02, 2020 at 06:11:08PM -0400, Jim Quinlan wrote:
> On Wed, Sep 2, 2020 at 5:53 PM Nathan Chancellor
> <natechancellor@gmail.com> wrote:
> >
> > On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> > > The new field 'dma_range_map' in struct device is used to facilitate the
> > > use of single or multiple offsets between mapping regions of cpu addrs and
> > > dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> > > capable of holding a single uniform offset and had no region bounds
> > > checking.
> > >
> > > The function of_dma_get_range() has been modified so that it takes a single
> > > argument -- the device node -- and returns a map, NULL, or an error code.
> > > The map is an array that holds the information regarding the DMA regions.
> > > Each range entry contains the address offset, the cpu_start address, the
> > > dma_start address, and the size of the region.
> > >
> > > of_dma_configure() is the typical manner to set range offsets but there are
> > > a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> > > driver code.  These cases now invoke the function
> > > dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
> > >
> > > Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> > > ---
> > >  arch/arm/include/asm/dma-mapping.h            | 10 +--
> > >  arch/arm/mach-keystone/keystone.c             | 17 +++--
> > >  arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
> > >  arch/x86/pci/sta2x11-fixup.c                  |  7 +-
> > >  drivers/acpi/arm64/iort.c                     |  5 +-
> > >  drivers/base/core.c                           |  2 +
> > >  drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
> > >  drivers/iommu/io-pgtable-arm.c                |  2 +-
> > >  .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
> > >  .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
> > >  drivers/of/address.c                          | 72 +++++++++----------
> > >  drivers/of/device.c                           | 43 ++++++-----
> > >  drivers/of/of_private.h                       | 10 +--
> > >  drivers/of/unittest.c                         | 34 ++++++---
> > >  drivers/remoteproc/remoteproc_core.c          |  8 ++-
> > >  .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
> > >  drivers/usb/core/message.c                    |  9 ++-
> > >  drivers/usb/core/usb.c                        |  7 +-
> > >  include/linux/device.h                        |  4 +-
> > >  include/linux/dma-direct.h                    |  8 +--
> > >  include/linux/dma-mapping.h                   | 36 ++++++++++
> > >  kernel/dma/coherent.c                         | 10 +--
> > >  kernel/dma/mapping.c                          | 66 +++++++++++++++++
> > >  23 files changed, 265 insertions(+), 115 deletions(-)
> >
> > Apologies if this has already been reported or is known but this commit
> > is now in next-20200902 and it causes my Raspberry Pi 4 to no longer
> > make it to userspace, instead spewing mmc errors:
> >
> > That commit causes my Raspberry Pi 4 to no longer make it to userspace,
> > instead spewing mmc errors:
> >
> > [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> > [    0.000000] Linux version 5.9.0-rc3-00004-geef520b232c6-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 13:48:49 MST 2020
> > [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> > ...
> > [    1.459752] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> > [    1.500007] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> > [    1.507454] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> > [    1.615547] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> > [    1.627537] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> > [    1.665497] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> > [    1.690601] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> > [    1.697892] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.705173] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.713788] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> > [    1.721228] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.732062] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> > [    1.741828] ALSA device list:
> > [    1.744885]   No soundcards found.
> > [    1.748540] Waiting for root device PARTUUID=45a8dd8a-02...
> > [    1.788865] random: fast init done
> > [    1.793489] mmc1: unrecognised SCR structure version 4
> > [    1.798814] mmc1: error -22 whilst initialising SD card
> > [    1.813969] mmc0: new high speed SDIO card at address 0001
> > [    1.883178] mmc1: unrecognised SCR structure version 2
> > [    1.888423] mmc1: error -22 whilst initialising SD card
> > [    1.964069] mmc1: unrecognised SCR structure version 4
> > [    1.969314] mmc1: error -22 whilst initialising SD card
> > [    2.061225] mmc1: unrecognised SCR structure version 4
> > [    2.066470] mmc1: error -22 whilst initialising SD card
> > [    3.160476] mmc1: unrecognised SCR structure version 4
> > [    3.165718] mmc1: error -22 whilst initialising SD card
> >
> > This is what it looks like before that commit:
> >
> > [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> > [    0.000000] Linux version 5.9.0-rc3-00003-g2281f797f552-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 14:10:30 MST 2020
> > [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> > ...
> > [    1.459871] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> > [    1.500017] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> > [    1.507461] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> > [    1.615565] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> > [    1.627571] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> > [    1.665549] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> > [    1.690225] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> > [    1.697516] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.704800] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.713412] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> > [    1.720853] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.731686] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> > [    1.741484] ALSA device list:
> > [    1.744517]   No soundcards found.
> > [    1.748194] Waiting for root device PARTUUID=45a8dd8a-02...
> > [    1.788643] random: fast init done
> > [    1.805009] mmc0: new high speed SDIO card at address 0001
> > [    1.852305] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
> > [    1.859647] mmcblk1: mmc1:aaaa SK64G 59.5 GiB
> > [    1.866190]  mmcblk1: p1 p2
> > [    1.892559] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)
> >
> > Bisect log in case it is relevant:
> >
> > # bad: [4442749a203151a319a5bb8d0b983b84253a6931] Add linux-next specific files for 20200902
> > # good: [b765a32a2e9170702467747e290614be072c4f76] Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
> > git bisect start '4442749a203151a319a5bb8d0b983b84253a6931' 'b765a32a2e9170702467747e290614be072c4f76'
> > # bad: [501c4123d9ecf8e38425fa2dae591d87f18a33f9] Merge remote-tracking branch 'wireless-drivers-next/master' into master
> > git bisect bad 501c4123d9ecf8e38425fa2dae591d87f18a33f9
> > # bad: [3722ace45f7d9a37631b6f5b9c591ea1e118a077] Merge remote-tracking branch 'pstore/for-next/pstore' into master
> > git bisect bad 3722ace45f7d9a37631b6f5b9c591ea1e118a077
> > # bad: [bcd6576526884cbe2f0798f9196eb2fd96fadb2b] Merge remote-tracking branch 'imx-mxs/for-next' into master
> > git bisect bad bcd6576526884cbe2f0798f9196eb2fd96fadb2b
> > # good: [f51293edaaf354f6de8836390cd4bd0407380c8d] Merge remote-tracking branch 'staging.current/staging-linus' into master
> > git bisect good f51293edaaf354f6de8836390cd4bd0407380c8d
> > # bad: [2b7f5289d46652f343f51e1853326876114f8647] Merge remote-tracking branch 'arm-soc/for-next' into master
> > git bisect bad 2b7f5289d46652f343f51e1853326876114f8647
> > # good: [137aded9aad7803fa48e9a9538b359012715e171] Merge remote-tracking branch 'pidfd-fixes/fixes' into master
> > git bisect good 137aded9aad7803fa48e9a9538b359012715e171
> > # good: [f5cb1ada6f986019d24037d9329c99445fcdd50a] Merge remote-tracking branch 'kbuild/for-next' into master
> > git bisect good f5cb1ada6f986019d24037d9329c99445fcdd50a
> > # good: [2694ca73fa41491762825ae7bba9b14ab0762415] ARM: Document merges
> > git bisect good 2694ca73fa41491762825ae7bba9b14ab0762415
> > # bad: [192ec9acc561411c6ec5d35b3ad225d40c8c39bd] Merge remote-tracking branch 'dma-mapping/for-next' into master
> > git bisect bad 192ec9acc561411c6ec5d35b3ad225d40c8c39bd
> > # bad: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
> > git bisect bad eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
> > # good: [c6303ab9b91e7ca20a49ff494338309259ed7c65] arm64: mm: reserve per-numa CMA to localize coherent dma buffers
> > git bisect good c6303ab9b91e7ca20a49ff494338309259ed7c65
> > # good: [2281f797f5524abb8fff66bf8540b4f4687332a2] mm: cma: use CMA_MAX_NAME to define the length of cma name array
> > git bisect good 2281f797f5524abb8fff66bf8540b4f4687332a2
> > # first bad commit: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
> >
> > I am happy to provide further information for debugging if necessary.
> Hello Nathan,
> 
> Can you tell me how much memory your RPI has and if all of it is

This is the 4GB version.

> accessible by the PCIe device?  Could you also please include the DTS
> of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> PCIe DT before Linux boots -- could you describe what is going on
> there?

Unfortunately, I am not familiar with how to get this information. If
you could provide some instructions for how to do so, I am more than
happy to. I am not very knowleagable about the inner working of the Pi,
I mainly use it as a test platform for making sure that LLVM does not
cause problems on real devices.

> Finally, can you attach the text of the full boot log?

I have attached a working and broken boot log. Thank you for the quick
response!

Cheers,
Nathan

[-- Attachment #2: pi-boot-bad.txt --]
[-- Type: text/plain, Size: 24428 bytes --]

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-next-20200902-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 15:17:43 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000037400000, size 64 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] NUMA: NODE_DATA [mem 0xfb81f100-0xfb820fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x000000003fffffff]
[    0.000000]   DMA32    [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000003b3fffff]
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: EL2 vector hardening
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] ARM_SMCCC_ARCH_WORKAROUND_1 missing from firmware
[    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 996912
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:  dma.dmachans=0x71f5 bcm2709.boardrev=0xc03112 bcm2709.serial=0xb78d398 bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:60:6C:87 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=ttyS1,115200 console=tty1 root=PARTUUID=45a8dd8a-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait plymouth.ignore-serial-consoles
[    0.000000] Kernel parameter elevator= does not have any effect anymore.
[    0.000000] Please use sysfs to set IO scheduler for individual devices.
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x33400000-0x37400000] (64MB)
[    0.000000] Memory: 3812260K/4050944K available (14524K kernel code, 2636K rwdata, 7540K rodata, 1600K init, 615K bss, 173148K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] random: get_random_bytes called from start_kernel+0x1cc/0x3f4 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 54.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123 ns
[    0.000006] sched_clock: 56 bits at 54MHz, resolution 18ns, wraps every 4398046511102ns
[    0.000196] Console: colour dummy device 80x25
[    0.000680] printk: console [tty1] enabled
[    0.000779] Calibrating delay loop (skipped), value calculated using timer frequency.. 108.00 BogoMIPS (lpj=216000)
[    0.000809] pid_max: default: 32768 minimum: 301
[    0.000896] LSM: Security Framework initializing
[    0.001029] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.001115] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.002874] rcu: Hierarchical SRCU implementation.
[    0.005100] EFI services will not be available.
[    0.005396] smp: Bringing up secondary CPUs ...
[    0.005904] Detected PIPT I-cache on CPU1
[    0.005963] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[    0.006505] Detected PIPT I-cache on CPU2
[    0.006544] CPU2: Booted secondary processor 0x0000000002 [0x410fd083]
[    0.007060] Detected PIPT I-cache on CPU3
[    0.007098] CPU3: Booted secondary processor 0x0000000003 [0x410fd083]
[    0.007182] smp: Brought up 1 node, 4 CPUs
[    0.007251] SMP: Total of 4 processors activated.
[    0.007267] CPU features: detected: 32-bit EL0 Support
[    0.007283] CPU features: detected: CRC32 instructions
[    0.007299] CPU features: detected: 32-bit EL1 Support
[    0.030649] CPU: All CPU(s) started at EL2
[    0.030710] alternatives: patching kernel code
[    0.032692] devtmpfs: initialized
[    0.037827] KASLR enabled
[    0.038277] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.038312] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.040163] pinctrl core: initialized pinctrl subsystem
[    0.041597] DMI not present or invalid.
[    0.042151] NET: Registered protocol family 16
[    0.043673] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.043826] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.044107] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.044163] audit: initializing netlink subsys (disabled)
[    0.044352] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[    0.045463] thermal_sys: Registered thermal governor 'step_wise'
[    0.045470] thermal_sys: Registered thermal governor 'power_allocator'
[    0.045807] cpuidle: using governor menu
[    0.046145] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.046246] ASID allocator initialised with 32768 entries
[    0.047838] Serial: AMBA PL011 UART driver
[    0.083072] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.083108] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.083126] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.083143] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.084507] cryptd: max_cpu_qlen set to 1000
[    0.087192] ACPI: Interpreter disabled.
[    0.089180] iommu: Default domain type: Translated 
[    0.089450] vgaarb: loaded
[    0.089800] SCSI subsystem initialized
[    0.090203] usbcore: registered new interface driver usbfs
[    0.090263] usbcore: registered new interface driver hub
[    0.090343] usbcore: registered new device driver usb
[    0.090520] usb_phy_generic phy: supply vcc not found, using dummy regulator
[    0.091346] pps_core: LinuxPPS API ver. 1 registered
[    0.091364] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.091394] PTP clock support registered
[    0.091552] EDAC MC: Ver: 3.0.0
[    0.092935] FPGA manager framework
[    0.093050] Advanced Linux Sound Architecture Driver Initialized.
[    0.093880] clocksource: Switched to clocksource arch_sys_counter
[    0.094112] VFS: Disk quotas dquot_6.6.0
[    0.094190] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.094401] pnp: PnP ACPI: disabled
[    0.100876] NET: Registered protocol family 2
[    0.101384] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.101444] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.101652] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.102173] TCP: Hash tables configured (established 32768 bind 32768)
[    0.102384] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102454] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102668] NET: Registered protocol family 1
[    0.103174] RPC: Registered named UNIX socket transport module.
[    0.103193] RPC: Registered udp transport module.
[    0.103206] RPC: Registered tcp transport module.
[    0.103220] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.103241] PCI: CLS 0 bytes, default 64
[    0.104437] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[    0.104591] kvm [1]: IPA Size Limit: 44bits
[    0.105723] kvm [1]: vgic interrupt IRQ1
[    0.105910] kvm [1]: Hyp mode initialized successfully
[    0.107799] Initialise system trusted keyrings
[    0.107975] workingset: timestamp_bits=44 max_order=20 bucket_order=0
[    0.114780] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.115569] NFS: Registering the id_resolver key type
[    0.115614] Key type id_resolver registered
[    0.115629] Key type id_legacy registered
[    0.115747] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.115926] 9p: Installing v9fs 9p2000 file system support
[    0.177452] Key type asymmetric registered
[    0.177472] Asymmetric key parser 'x509' registered
[    0.177527] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.177547] io scheduler mq-deadline registered
[    0.177560] io scheduler kyber registered
[    0.190013] EINJ: ACPI disabled.
[    0.208072] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.210194] printk: console [ttyS1] disabled
[    0.210297] fe215040.serial: ttyS1 at MMIO 0xfe215040 (irq = 18, base_baud = 24999999) is a 16550
[    1.115569] printk: console [ttyS1] enabled
[    1.121263] SuperH (H)SCI(F) driver initialized
[    1.126522] msm_serial: driver initialized
[    1.132301] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    1.146949] loop: module loaded
[    1.151363] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    1.159671] megasas: 07.714.04.00-rc1
[    1.168505] libphy: Fixed MDIO Bus: probed
[    1.174118] tun: Universal TUN/TAP device driver, 1.6
[    1.180090] thunder_xcv, ver 1.0
[    1.183434] thunder_bgx, ver 1.0
[    1.186768] nicpf, ver 1.0
[    1.190746] hclge is initializing
[    1.194178] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    1.201546] hns3: Copyright (c) 2017 Huawei Corporation.
[    1.207017] e1000: Intel(R) PRO/1000 Network Driver
[    1.211992] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    1.217901] e1000e: Intel(R) PRO/1000 Network Driver
[    1.222962] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.229030] igb: Intel(R) Gigabit Ethernet Network Driver
[    1.234535] igb: Copyright (c) 2007-2014 Intel Corporation.
[    1.240248] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    1.246633] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    1.253038] sky2: driver version 1.30
[    1.257975] VFIO - User Level meta-driver version: 0.3
[    1.265257] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.271935] ehci-pci: EHCI PCI platform driver
[    1.276511] ehci-platform: EHCI generic platform driver
[    1.281987] ehci-orion: EHCI orion driver
[    1.286186] ehci-exynos: EHCI Exynos driver
[    1.290552] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.296868] ohci-pci: OHCI PCI platform driver
[    1.301436] ohci-platform: OHCI generic platform driver
[    1.306891] ohci-exynos: OHCI Exynos driver
[    1.311695] usbcore: registered new interface driver usb-storage
[    1.320736] i2c /dev entries driver
[    1.329662] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    1.338167] sdhci: Secure Digital Host Controller Interface driver
[    1.344475] sdhci: Copyright(c) Pierre Ossman
[    1.349524] Synopsys Designware Multimedia Card Interface Driver
[    1.356759] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.375177] ledtrig-cpu: registered to indicate activity on CPUs
[    1.383060] usbcore: registered new interface driver usbhid
[    1.388747] usbhid: USB HID core driver
[    1.393728] bcm2835-mbox fe00b880.mailbox: mailbox enabled
[    1.403725] NET: Registered protocol family 17
[    1.408495] 9pnet: Installing 9P2000 support
[    1.412912] Key type dns_resolver registered
[    1.417632] registered taskstats version 1
[    1.421822] Loading compiled-in X.509 certificates
[    1.431630] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 16, base_baud = 0) is a PL011 rev2
[    1.440890] serial serial0: tty port ttyAMA0 registered
[    1.462581] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.503538] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.510983] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.619098] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.631122] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.669127] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.694102] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.701389] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.708672] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.717293] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.724735] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.735564] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.745322] ALSA device list:
[    1.748390]   No soundcards found.
[    1.752060] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.792429] random: fast init done
[    1.797033] mmc1: unrecognised SCR structure version 4
[    1.802357] mmc1: error -22 whilst initialising SD card
[    1.817686] mmc0: new high speed SDIO card at address 0001
[    1.886941] mmc1: unrecognised SCR structure version 2
[    1.892189] mmc1: error -22 whilst initialising SD card
[    1.967866] mmc1: unrecognised SCR structure version 4
[    1.973113] mmc1: error -22 whilst initialising SD card
[    2.065006] mmc1: unrecognised SCR structure version 4
[    2.070252] mmc1: error -22 whilst initialising SD card
[    3.161820] mmc1: unrecognised SCR structure version 4
[    3.167064] mmc1: error -22 whilst initialising SD card
[    3.237380] mmc1: unrecognised SCR structure version 4
[    3.242625] mmc1: error -22 whilst initialising SD card
[    3.318280] mmc1: unrecognised SCR structure version 4
[    3.323525] mmc1: error -22 whilst initialising SD card
[    3.415425] mmc1: unrecognised SCR structure version 4
[    3.420669] mmc1: error -22 whilst initialising SD card
[    4.505842] mmc1: unrecognised SCR structure version 4
[    4.511090] mmc1: error -22 whilst initialising SD card
[    4.581408] mmc1: unrecognised SCR structure version 4
[    4.586652] mmc1: error -22 whilst initialising SD card
[    4.662344] mmc1: unrecognised SCR structure version 4
[    4.667588] mmc1: error -22 whilst initialising SD card
[    4.759556] mmc1: unrecognised SCR structure version 4
[    4.764800] mmc1: error -22 whilst initialising SD card
[    5.851270] mmc1: unrecognised SCR structure version 4
[    5.856517] mmc1: error -22 whilst initialising SD card
[    5.926793] mmc1: unrecognised SCR structure version 4
[    5.932038] mmc1: error -22 whilst initialising SD card
[    6.007687] mmc1: unrecognised SCR structure version 4
[    6.012930] mmc1: error -22 whilst initialising SD card
[    6.104759] mmc1: unrecognised SCR structure version 4
[    6.110006] mmc1: error -22 whilst initialising SD card
[    7.193797] mmc1: unrecognised SCR structure version 4
[    7.199043] mmc1: error -22 whilst initialising SD card
[    7.269330] mmc1: unrecognised SCR structure version 4
[    7.274574] mmc1: error -22 whilst initialising SD card
[    7.350192] mmc1: unrecognised SCR structure version 4
[    7.355435] mmc1: error -22 whilst initialising SD card
[    7.447355] mmc1: unrecognised SCR structure version 4
[    7.452598] mmc1: error -22 whilst initialising SD card
[    8.537815] mmc1: unrecognised SCR structure version 4
[    8.543060] mmc1: error -22 whilst initialising SD card
[    8.613360] mmc1: unrecognised SCR structure version 4
[    8.618604] mmc1: error -22 whilst initialising SD card
[    8.694255] mmc1: unrecognised SCR structure version 4
[    8.699499] mmc1: error -22 whilst initialising SD card
[    8.791399] mmc1: unrecognised SCR structure version 4
[    8.796642] mmc1: error -22 whilst initialising SD card
[    9.881840] mmc1: unrecognised SCR structure version 4
[    9.887088] mmc1: error -22 whilst initialising SD card
[    9.957402] mmc1: unrecognised SCR structure version 4
[    9.962646] mmc1: error -22 whilst initialising SD card
[   10.038303] mmc1: unrecognised SCR structure version 4
[   10.043547] mmc1: error -22 whilst initialising SD card
[   10.135464] mmc1: unrecognised SCR structure version 4
[   10.140707] mmc1: error -22 whilst initialising SD card
[   11.225815] mmc1: unrecognised SCR structure version 4
[   11.231062] mmc1: error -22 whilst initialising SD card
[   11.301373] mmc1: unrecognised SCR structure version 4
[   11.306617] mmc1: error -22 whilst initialising SD card
[   11.382264] mmc1: unrecognised SCR structure version 4
[   11.387507] mmc1: error -22 whilst initialising SD card
[   11.479430] mmc1: unrecognised SCR structure version 4
[   11.484673] mmc1: error -22 whilst initialising SD card
[   12.569813] mmc1: unrecognised SCR structure version 4
[   12.575059] mmc1: error -22 whilst initialising SD card
[   12.645354] mmc1: unrecognised SCR structure version 4
[   12.650598] mmc1: error -22 whilst initialising SD card
[   12.726254] mmc1: unrecognised SCR structure version 4
[   12.731497] mmc1: error -22 whilst initialising SD card
[   12.823417] mmc1: unrecognised SCR structure version 4
[   12.828661] mmc1: error -22 whilst initialising SD card
[   13.913815] mmc1: unrecognised SCR structure version 4
[   13.919059] mmc1: error -22 whilst initialising SD card
[   13.989375] mmc1: unrecognised SCR structure version 4
[   13.994620] mmc1: error -22 whilst initialising SD card
[   14.070314] mmc1: unrecognised SCR structure version 4
[   14.075557] mmc1: error -22 whilst initialising SD card
[   14.167479] mmc1: unrecognised SCR structure version 4
[   14.172722] mmc1: error -22 whilst initialising SD card
[   15.257839] mmc1: unrecognised SCR structure version 4
[   15.263086] mmc1: error -22 whilst initialising SD card
[   15.333397] mmc1: unrecognised SCR structure version 4
[   15.338642] mmc1: error -22 whilst initialising SD card
[   15.414259] mmc1: unrecognised SCR structure version 4
[   15.419500] mmc1: error -22 whilst initialising SD card
[   15.511418] mmc1: unrecognised SCR structure version 4
[   15.516662] mmc1: error -22 whilst initialising SD card
[   16.601839] mmc1: unrecognised SCR structure version 4
[   16.607086] mmc1: error -22 whilst initialising SD card
[   16.677414] mmc1: unrecognised SCR structure version 4
[   16.682658] mmc1: error -22 whilst initialising SD card
[   16.758280] mmc1: unrecognised SCR structure version 4
[   16.763524] mmc1: error -22 whilst initialising SD card
[   16.855411] mmc1: unrecognised SCR structure version 4
[   16.860657] mmc1: error -22 whilst initialising SD card
[   17.945838] mmc1: unrecognised SCR structure version 4
[   17.951084] mmc1: error -22 whilst initialising SD card
[   18.022951] mmc1: unrecognised SCR structure version 4
[   18.028196] mmc1: error -22 whilst initialising SD card
[   18.103836] mmc1: unrecognised SCR structure version 4
[   18.109079] mmc1: error -22 whilst initialising SD card
[   18.200919] mmc1: unrecognised SCR structure version 4
[   18.206165] mmc1: error -22 whilst initialising SD card
[   19.289820] mmc1: unrecognised SCR structure version 4
[   19.295068] mmc1: error -22 whilst initialising SD card
[   19.365404] mmc1: unrecognised SCR structure version 4
[   19.370648] mmc1: error -22 whilst initialising SD card
[   19.446261] mmc1: unrecognised SCR structure version 4
[   19.451504] mmc1: error -22 whilst initialising SD card
[   19.543415] mmc1: unrecognised SCR structure version 4
[   19.548658] mmc1: error -22 whilst initialising SD card
[   20.633815] mmc1: unrecognised SCR structure version 4
[   20.639065] mmc1: error -22 whilst initialising SD card
[   20.709375] mmc1: unrecognised SCR structure version 4
[   20.714619] mmc1: error -22 whilst initialising SD card
[   20.790313] mmc1: unrecognised SCR structure version 4
[   20.795557] mmc1: error -22 whilst initialising SD card
[   20.887466] mmc1: unrecognised SCR structure version 4
[   20.892710] mmc1: error -22 whilst initialising SD card
[   21.977834] mmc1: unrecognised SCR structure version 4
[   21.983081] mmc1: error -22 whilst initialising SD card
[   22.053392] mmc1: unrecognised SCR structure version 4
[   22.058637] mmc1: error -22 whilst initialising SD card
[   22.134244] mmc1: unrecognised SCR structure version 4
[   22.139488] mmc1: error -22 whilst initialising SD card
[   22.231412] mmc1: unrecognised SCR structure version 4
[   22.236656] mmc1: error -22 whilst initialising SD card
[   23.321810] mmc1: unrecognised SCR structure version 4
[   23.327056] mmc1: error -22 whilst initialising SD card
[   23.397372] mmc1: unrecognised SCR structure version 4
[   23.402617] mmc1: error -22 whilst initialising SD card
[   23.478248] mmc1: unrecognised SCR structure version 4
[   23.483492] mmc1: error -22 whilst initialising SD card
[   23.575406] mmc1: unrecognised SCR structure version 4
[   23.580650] mmc1: error -22 whilst initialising SD card
[   24.665836] mmc1: unrecognised SCR structure version 4
[   24.671080] mmc1: error -22 whilst initialising SD card
[   24.741396] mmc1: unrecognised SCR structure version 4
[   24.746641] mmc1: error -22 whilst initialising SD card
[   24.822248] mmc1: unrecognised SCR structure version 4
[   24.827492] mmc1: error -22 whilst initialising SD card
[   24.919406] mmc1: unrecognised SCR structure version 4
[   24.924650] mmc1: error -22 whilst initialising SD card
[   26.009818] mmc1: unrecognised SCR structure version 4
[   26.015064] mmc1: error -22 whilst initialising SD card
[   26.085377] mmc1: unrecognised SCR structure version 4
[   26.090621] mmc1: error -22 whilst initialising SD card
[   26.166270] mmc1: unrecognised SCR structure version 4
[   26.171514] mmc1: error -22 whilst initialising SD card
[   26.263190] mmc1: unrecognised SCR structure version 4
[   26.268434] mmc1: error -22 whilst initialising SD card
[   27.353799] mmc1: unrecognised SCR structure version 4
[   27.359045] mmc1: error -22 whilst initialising SD card
[   27.429363] mmc1: unrecognised SCR structure version 4
[   27.434607] mmc1: error -22 whilst initialising SD card
[   27.512614] mmc1: unrecognised SCR structure version 4
[   27.517858] mmc1: error -22 whilst initialising SD card
[   27.609855] mmc1: unrecognised SCR structure version 4
[   27.615102] mmc1: error -22 whilst initialising SD card
[   28.697337] mmc1: unrecognised SCR structure version 4
[   28.702583] mmc1: error -22 whilst initialising SD card
[   28.772895] mmc1: unrecognised SCR structure version 4
[   28.778139] mmc1: error -22 whilst initialising SD card
[   28.854654] mmc1: unrecognised SCR structure version 4
[   28.859900] mmc1: error -22 whilst initialising SD card
[   28.951809] mmc1: unrecognised SCR structure version 4
[   28.957049] mmc1: error -22 whilst initialising SD card
[   30.041823] mmc1: unrecognised SCR structure version 4
[   30.047070] mmc1: error -22 whilst initialising SD card
[   30.117387] mmc1: unrecognised SCR structure version 4
[   30.122631] mmc1: error -22 whilst initialising SD card
[   30.198244] mmc1: unrecognised SCR structure version 4
[   30.203488] mmc1: error -22 whilst initialising SD card
[   30.295391] mmc1: unrecognised SCR structure version 4
[   30.300638] mmc1: error -22 whilst initialising SD card

[-- Attachment #3: pi-boot-good.txt --]
[-- Type: text/plain, Size: 15985 bytes --]

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-next-20200902-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 15:28:17 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000037400000, size 64 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] NUMA: NODE_DATA [mem 0xfb81f100-0xfb820fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x000000003fffffff]
[    0.000000]   DMA32    [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000003b3fffff]
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: EL2 vector hardening
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] ARM_SMCCC_ARCH_WORKAROUND_1 missing from firmware
[    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 996912
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:  dma.dmachans=0x71f5 bcm2709.boardrev=0xc03112 bcm2709.serial=0xb78d398 bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:60:6C:87 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=ttyS1,115200 console=tty1 root=PARTUUID=45a8dd8a-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait plymouth.ignore-serial-consoles
[    0.000000] Kernel parameter elevator= does not have any effect anymore.
[    0.000000] Please use sysfs to set IO scheduler for individual devices.
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x33400000-0x37400000] (64MB)
[    0.000000] Memory: 3812260K/4050944K available (14524K kernel code, 2636K rwdata, 7540K rodata, 1600K init, 615K bss, 173148K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] random: get_random_bytes called from start_kernel+0x1cc/0x3f4 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 54.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123 ns
[    0.000006] sched_clock: 56 bits at 54MHz, resolution 18ns, wraps every 4398046511102ns
[    0.000195] Console: colour dummy device 80x25
[    0.000678] printk: console [tty1] enabled
[    0.000778] Calibrating delay loop (skipped), value calculated using timer frequency.. 108.00 BogoMIPS (lpj=216000)
[    0.000808] pid_max: default: 32768 minimum: 301
[    0.000894] LSM: Security Framework initializing
[    0.001030] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.001116] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.002878] rcu: Hierarchical SRCU implementation.
[    0.005048] EFI services will not be available.
[    0.005338] smp: Bringing up secondary CPUs ...
[    0.005844] Detected PIPT I-cache on CPU1
[    0.005903] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[    0.006454] Detected PIPT I-cache on CPU2
[    0.006493] CPU2: Booted secondary processor 0x0000000002 [0x410fd083]
[    0.007012] Detected PIPT I-cache on CPU3
[    0.007051] CPU3: Booted secondary processor 0x0000000003 [0x410fd083]
[    0.007133] smp: Brought up 1 node, 4 CPUs
[    0.007198] SMP: Total of 4 processors activated.
[    0.007214] CPU features: detected: 32-bit EL0 Support
[    0.007229] CPU features: detected: CRC32 instructions
[    0.007245] CPU features: detected: 32-bit EL1 Support
[    0.030492] CPU: All CPU(s) started at EL2
[    0.030550] alternatives: patching kernel code
[    0.032524] devtmpfs: initialized
[    0.037648] KASLR enabled
[    0.038091] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.038125] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.039986] pinctrl core: initialized pinctrl subsystem
[    0.041393] DMI not present or invalid.
[    0.041944] NET: Registered protocol family 16
[    0.043458] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.043610] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.043887] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.043939] audit: initializing netlink subsys (disabled)
[    0.044132] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[    0.045217] thermal_sys: Registered thermal governor 'step_wise'
[    0.045225] thermal_sys: Registered thermal governor 'power_allocator'
[    0.045555] cpuidle: using governor menu
[    0.045877] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.045978] ASID allocator initialised with 32768 entries
[    0.047544] Serial: AMBA PL011 UART driver
[    0.082787] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.082822] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.082840] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.082856] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.084212] cryptd: max_cpu_qlen set to 1000
[    0.086893] ACPI: Interpreter disabled.
[    0.088884] iommu: Default domain type: Translated 
[    0.089151] vgaarb: loaded
[    0.089505] SCSI subsystem initialized
[    0.089896] usbcore: registered new interface driver usbfs
[    0.089955] usbcore: registered new interface driver hub
[    0.090035] usbcore: registered new device driver usb
[    0.090208] usb_phy_generic phy: supply vcc not found, using dummy regulator
[    0.091009] pps_core: LinuxPPS API ver. 1 registered
[    0.091026] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.091055] PTP clock support registered
[    0.091238] EDAC MC: Ver: 3.0.0
[    0.092605] FPGA manager framework
[    0.092719] Advanced Linux Sound Architecture Driver Initialized.
[    0.093558] clocksource: Switched to clocksource arch_sys_counter
[    0.093791] VFS: Disk quotas dquot_6.6.0
[    0.093865] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.094072] pnp: PnP ACPI: disabled
[    0.100547] NET: Registered protocol family 2
[    0.101063] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.101121] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.101328] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.101841] TCP: Hash tables configured (established 32768 bind 32768)
[    0.102055] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102123] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102329] NET: Registered protocol family 1
[    0.102824] RPC: Registered named UNIX socket transport module.
[    0.102842] RPC: Registered udp transport module.
[    0.102855] RPC: Registered tcp transport module.
[    0.102868] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.102888] PCI: CLS 0 bytes, default 64
[    0.104073] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[    0.104224] kvm [1]: IPA Size Limit: 44bits
[    0.105353] kvm [1]: vgic interrupt IRQ1
[    0.105504] kvm [1]: Hyp mode initialized successfully
[    0.107404] Initialise system trusted keyrings
[    0.107581] workingset: timestamp_bits=44 max_order=20 bucket_order=0
[    0.114285] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.115072] NFS: Registering the id_resolver key type
[    0.115111] Key type id_resolver registered
[    0.115125] Key type id_legacy registered
[    0.115242] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.115415] 9p: Installing v9fs 9p2000 file system support
[    0.178130] Key type asymmetric registered
[    0.178149] Asymmetric key parser 'x509' registered
[    0.178205] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.178225] io scheduler mq-deadline registered
[    0.178239] io scheduler kyber registered
[    0.190560] EINJ: ACPI disabled.
[    0.208427] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.210486] printk: console [ttyS1] disabled
[    0.210587] fe215040.serial: ttyS1 at MMIO 0xfe215040 (irq = 18, base_baud = 24999999) is a 16550
[    1.115809] printk: console [ttyS1] enabled
[    1.121521] SuperH (H)SCI(F) driver initialized
[    1.126760] msm_serial: driver initialized
[    1.132523] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    1.147108] loop: module loaded
[    1.151491] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    1.159778] megasas: 07.714.04.00-rc1
[    1.168551] libphy: Fixed MDIO Bus: probed
[    1.174162] tun: Universal TUN/TAP device driver, 1.6
[    1.180136] thunder_xcv, ver 1.0
[    1.183480] thunder_bgx, ver 1.0
[    1.186813] nicpf, ver 1.0
[    1.190780] hclge is initializing
[    1.194215] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    1.201583] hns3: Copyright (c) 2017 Huawei Corporation.
[    1.207048] e1000: Intel(R) PRO/1000 Network Driver
[    1.212023] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    1.217916] e1000e: Intel(R) PRO/1000 Network Driver
[    1.222976] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.229054] igb: Intel(R) Gigabit Ethernet Network Driver
[    1.234559] igb: Copyright (c) 2007-2014 Intel Corporation.
[    1.240268] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    1.246653] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    1.253048] sky2: driver version 1.30
[    1.257983] VFIO - User Level meta-driver version: 0.3
[    1.265221] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.271898] ehci-pci: EHCI PCI platform driver
[    1.276471] ehci-platform: EHCI generic platform driver
[    1.281934] ehci-orion: EHCI orion driver
[    1.286133] ehci-exynos: EHCI Exynos driver
[    1.290505] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.296820] ohci-pci: OHCI PCI platform driver
[    1.301386] ohci-platform: OHCI generic platform driver
[    1.306836] ohci-exynos: OHCI Exynos driver
[    1.311632] usbcore: registered new interface driver usb-storage
[    1.320631] i2c /dev entries driver
[    1.329492] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    1.337993] sdhci: Secure Digital Host Controller Interface driver
[    1.344303] sdhci: Copyright(c) Pierre Ossman
[    1.349337] Synopsys Designware Multimedia Card Interface Driver
[    1.356563] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.374853] ledtrig-cpu: registered to indicate activity on CPUs
[    1.382707] usbcore: registered new interface driver usbhid
[    1.388393] usbhid: USB HID core driver
[    1.393348] bcm2835-mbox fe00b880.mailbox: mailbox enabled
[    1.403299] NET: Registered protocol family 17
[    1.408070] 9pnet: Installing 9P2000 support
[    1.412487] Key type dns_resolver registered
[    1.417194] registered taskstats version 1
[    1.421390] Loading compiled-in X.509 certificates
[    1.431180] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 16, base_baud = 0) is a PL011 rev2
[    1.440427] serial serial0: tty port ttyAMA0 registered
[    1.461855] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.505592] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.513027] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.621089] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.633123] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.671111] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.696128] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.703412] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.710695] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.719313] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.726750] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.737585] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.747325] ALSA device list:
[    1.750387]   No soundcards found.
[    1.754070] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.793779] random: fast init done
[    1.810764] mmc0: new high speed SDIO card at address 0001
[    1.858064] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
[    1.865420] mmcblk1: mmc1:aaaa SK64G 59.5 GiB 
[    1.871966]  mmcblk1: p1 p2
[    1.897160] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)
[    1.905492] VFS: Mounted root (ext4 filesystem) readonly on device 179:2.
[    1.913206] devtmpfs: mounted
[    1.917652] Freeing unused kernel memory: 1600K
[    1.933776] Run /sbin/init as init process
[    2.314733] systemd[1]: System time before build time, advancing clock.
[    2.661797] NET: Registered protocol family 10
[    2.667365] Segment Routing with IPv6
[    2.730477] systemd[1]: systemd 241 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid)
[    2.752685] systemd[1]: Detected architecture arm64.
[    2.792164] systemd[1]: Set hostname to <raspberrypi>.
[    3.017532] systemd[1]: File /lib/systemd/system/systemd-journald.service:12 configures an IP firewall (IPAddressDeny=any), but the local system does not support BPF/cgroup based firewalling.
[    3.034988] systemd[1]: Proceeding WITHOUT firewalling in effect! (This warning is only shown for the first loaded unit using IP firewalling.)
[    3.309831] random: systemd: uninitialized urandom read (16 bytes read)
[    3.321952] random: systemd: uninitialized urandom read (16 bytes read)
[    3.329307] systemd[1]: Listening on Journal Socket (/dev/log).
[    3.335968] random: systemd: uninitialized urandom read (16 bytes read)
[    3.345903] systemd[1]: Created slice system-serial\x2dgetty.slice.
[    3.353269] systemd[1]: Listening on Journal Audit Socket.
[    3.360213] systemd[1]: Created slice system-getty.slice.
[    3.366141] systemd[1]: Reached target Swap.

[-- Attachment #4: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

[-- Attachment #1: Type: text/plain, Size: 10977 bytes --]

On Wed, Sep 02, 2020 at 06:11:08PM -0400, Jim Quinlan wrote:
> On Wed, Sep 2, 2020 at 5:53 PM Nathan Chancellor
> <natechancellor@gmail.com> wrote:
> >
> > On Mon, Aug 24, 2020 at 03:30:20PM -0400, Jim Quinlan wrote:
> > > The new field 'dma_range_map' in struct device is used to facilitate the
> > > use of single or multiple offsets between mapping regions of cpu addrs and
> > > dma addrs.  It subsumes the role of "dev->dma_pfn_offset" which was only
> > > capable of holding a single uniform offset and had no region bounds
> > > checking.
> > >
> > > The function of_dma_get_range() has been modified so that it takes a single
> > > argument -- the device node -- and returns a map, NULL, or an error code.
> > > The map is an array that holds the information regarding the DMA regions.
> > > Each range entry contains the address offset, the cpu_start address, the
> > > dma_start address, and the size of the region.
> > >
> > > of_dma_configure() is the typical manner to set range offsets but there are
> > > a number of ad hoc assignments to "dev->dma_pfn_offset" in the kernel
> > > driver code.  These cases now invoke the function
> > > dma_attach_offset_range(dev, cpu_addr, dma_addr, size).
> > >
> > > Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> > > ---
> > >  arch/arm/include/asm/dma-mapping.h            | 10 +--
> > >  arch/arm/mach-keystone/keystone.c             | 17 +++--
> > >  arch/sh/drivers/pci/pcie-sh7786.c             |  9 +--
> > >  arch/x86/pci/sta2x11-fixup.c                  |  7 +-
> > >  drivers/acpi/arm64/iort.c                     |  5 +-
> > >  drivers/base/core.c                           |  2 +
> > >  drivers/gpu/drm/sun4i/sun4i_backend.c         |  5 +-
> > >  drivers/iommu/io-pgtable-arm.c                |  2 +-
> > >  .../platform/sunxi/sun4i-csi/sun4i_csi.c      |  5 +-
> > >  .../platform/sunxi/sun6i-csi/sun6i_csi.c      |  4 +-
> > >  drivers/of/address.c                          | 72 +++++++++----------
> > >  drivers/of/device.c                           | 43 ++++++-----
> > >  drivers/of/of_private.h                       | 10 +--
> > >  drivers/of/unittest.c                         | 34 ++++++---
> > >  drivers/remoteproc/remoteproc_core.c          |  8 ++-
> > >  .../staging/media/sunxi/cedrus/cedrus_hw.c    |  7 +-
> > >  drivers/usb/core/message.c                    |  9 ++-
> > >  drivers/usb/core/usb.c                        |  7 +-
> > >  include/linux/device.h                        |  4 +-
> > >  include/linux/dma-direct.h                    |  8 +--
> > >  include/linux/dma-mapping.h                   | 36 ++++++++++
> > >  kernel/dma/coherent.c                         | 10 +--
> > >  kernel/dma/mapping.c                          | 66 +++++++++++++++++
> > >  23 files changed, 265 insertions(+), 115 deletions(-)
> >
> > Apologies if this has already been reported or is known but this commit
> > is now in next-20200902 and it causes my Raspberry Pi 4 to no longer
> > make it to userspace, instead spewing mmc errors:
> >
> > That commit causes my Raspberry Pi 4 to no longer make it to userspace,
> > instead spewing mmc errors:
> >
> > [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> > [    0.000000] Linux version 5.9.0-rc3-00004-geef520b232c6-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 13:48:49 MST 2020
> > [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> > ...
> > [    1.459752] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> > [    1.500007] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> > [    1.507454] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> > [    1.615547] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> > [    1.627537] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> > [    1.665497] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> > [    1.690601] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> > [    1.697892] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.705173] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.713788] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> > [    1.721228] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.732062] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> > [    1.741828] ALSA device list:
> > [    1.744885]   No soundcards found.
> > [    1.748540] Waiting for root device PARTUUID=45a8dd8a-02...
> > [    1.788865] random: fast init done
> > [    1.793489] mmc1: unrecognised SCR structure version 4
> > [    1.798814] mmc1: error -22 whilst initialising SD card
> > [    1.813969] mmc0: new high speed SDIO card at address 0001
> > [    1.883178] mmc1: unrecognised SCR structure version 2
> > [    1.888423] mmc1: error -22 whilst initialising SD card
> > [    1.964069] mmc1: unrecognised SCR structure version 4
> > [    1.969314] mmc1: error -22 whilst initialising SD card
> > [    2.061225] mmc1: unrecognised SCR structure version 4
> > [    2.066470] mmc1: error -22 whilst initialising SD card
> > [    3.160476] mmc1: unrecognised SCR structure version 4
> > [    3.165718] mmc1: error -22 whilst initialising SD card
> >
> > This is what it looks like before that commit:
> >
> > [    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
> > [    0.000000] Linux version 5.9.0-rc3-00003-g2281f797f552-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 14:10:30 MST 2020
> > [    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
> > ...
> > [    1.459871] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
> > [    1.500017] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
> > [    1.507461] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
> > [    1.615565] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
> > [    1.627571] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
> > [    1.665549] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
> > [    1.690225] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
> > [    1.697516] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.704800] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.713412] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
> > [    1.720853] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
> > [    1.731686] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
> > [    1.741484] ALSA device list:
> > [    1.744517]   No soundcards found.
> > [    1.748194] Waiting for root device PARTUUID=45a8dd8a-02...
> > [    1.788643] random: fast init done
> > [    1.805009] mmc0: new high speed SDIO card at address 0001
> > [    1.852305] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
> > [    1.859647] mmcblk1: mmc1:aaaa SK64G 59.5 GiB
> > [    1.866190]  mmcblk1: p1 p2
> > [    1.892559] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)
> >
> > Bisect log in case it is relevant:
> >
> > # bad: [4442749a203151a319a5bb8d0b983b84253a6931] Add linux-next specific files for 20200902
> > # good: [b765a32a2e9170702467747e290614be072c4f76] Merge tag 'scsi-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi
> > git bisect start '4442749a203151a319a5bb8d0b983b84253a6931' 'b765a32a2e9170702467747e290614be072c4f76'
> > # bad: [501c4123d9ecf8e38425fa2dae591d87f18a33f9] Merge remote-tracking branch 'wireless-drivers-next/master' into master
> > git bisect bad 501c4123d9ecf8e38425fa2dae591d87f18a33f9
> > # bad: [3722ace45f7d9a37631b6f5b9c591ea1e118a077] Merge remote-tracking branch 'pstore/for-next/pstore' into master
> > git bisect bad 3722ace45f7d9a37631b6f5b9c591ea1e118a077
> > # bad: [bcd6576526884cbe2f0798f9196eb2fd96fadb2b] Merge remote-tracking branch 'imx-mxs/for-next' into master
> > git bisect bad bcd6576526884cbe2f0798f9196eb2fd96fadb2b
> > # good: [f51293edaaf354f6de8836390cd4bd0407380c8d] Merge remote-tracking branch 'staging.current/staging-linus' into master
> > git bisect good f51293edaaf354f6de8836390cd4bd0407380c8d
> > # bad: [2b7f5289d46652f343f51e1853326876114f8647] Merge remote-tracking branch 'arm-soc/for-next' into master
> > git bisect bad 2b7f5289d46652f343f51e1853326876114f8647
> > # good: [137aded9aad7803fa48e9a9538b359012715e171] Merge remote-tracking branch 'pidfd-fixes/fixes' into master
> > git bisect good 137aded9aad7803fa48e9a9538b359012715e171
> > # good: [f5cb1ada6f986019d24037d9329c99445fcdd50a] Merge remote-tracking branch 'kbuild/for-next' into master
> > git bisect good f5cb1ada6f986019d24037d9329c99445fcdd50a
> > # good: [2694ca73fa41491762825ae7bba9b14ab0762415] ARM: Document merges
> > git bisect good 2694ca73fa41491762825ae7bba9b14ab0762415
> > # bad: [192ec9acc561411c6ec5d35b3ad225d40c8c39bd] Merge remote-tracking branch 'dma-mapping/for-next' into master
> > git bisect bad 192ec9acc561411c6ec5d35b3ad225d40c8c39bd
> > # bad: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
> > git bisect bad eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7
> > # good: [c6303ab9b91e7ca20a49ff494338309259ed7c65] arm64: mm: reserve per-numa CMA to localize coherent dma buffers
> > git bisect good c6303ab9b91e7ca20a49ff494338309259ed7c65
> > # good: [2281f797f5524abb8fff66bf8540b4f4687332a2] mm: cma: use CMA_MAX_NAME to define the length of cma name array
> > git bisect good 2281f797f5524abb8fff66bf8540b4f4687332a2
> > # first bad commit: [eef520b232c60e74eb8b33a5a7863ad8f2b4a5c7] dma-mapping: introduce DMA range map, supplanting dma_pfn_offset
> >
> > I am happy to provide further information for debugging if necessary.
> Hello Nathan,
> 
> Can you tell me how much memory your RPI has and if all of it is

This is the 4GB version.

> accessible by the PCIe device?  Could you also please include the DTS
> of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> PCIe DT before Linux boots -- could you describe what is going on
> there?

Unfortunately, I am not familiar with how to get this information. If
you could provide some instructions for how to do so, I am more than
happy to. I am not very knowleagable about the inner working of the Pi,
I mainly use it as a test platform for making sure that LLVM does not
cause problems on real devices.

> Finally, can you attach the text of the full boot log?

I have attached a working and broken boot log. Thank you for the quick
response!

Cheers,
Nathan

[-- Attachment #2: pi-boot-bad.txt --]
[-- Type: text/plain, Size: 24428 bytes --]

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-next-20200902-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 15:17:43 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000037400000, size 64 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] NUMA: NODE_DATA [mem 0xfb81f100-0xfb820fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x000000003fffffff]
[    0.000000]   DMA32    [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000003b3fffff]
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: EL2 vector hardening
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] ARM_SMCCC_ARCH_WORKAROUND_1 missing from firmware
[    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 996912
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:  dma.dmachans=0x71f5 bcm2709.boardrev=0xc03112 bcm2709.serial=0xb78d398 bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:60:6C:87 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=ttyS1,115200 console=tty1 root=PARTUUID=45a8dd8a-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait plymouth.ignore-serial-consoles
[    0.000000] Kernel parameter elevator= does not have any effect anymore.
[    0.000000] Please use sysfs to set IO scheduler for individual devices.
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x33400000-0x37400000] (64MB)
[    0.000000] Memory: 3812260K/4050944K available (14524K kernel code, 2636K rwdata, 7540K rodata, 1600K init, 615K bss, 173148K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] random: get_random_bytes called from start_kernel+0x1cc/0x3f4 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 54.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123 ns
[    0.000006] sched_clock: 56 bits at 54MHz, resolution 18ns, wraps every 4398046511102ns
[    0.000196] Console: colour dummy device 80x25
[    0.000680] printk: console [tty1] enabled
[    0.000779] Calibrating delay loop (skipped), value calculated using timer frequency.. 108.00 BogoMIPS (lpj=216000)
[    0.000809] pid_max: default: 32768 minimum: 301
[    0.000896] LSM: Security Framework initializing
[    0.001029] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.001115] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.002874] rcu: Hierarchical SRCU implementation.
[    0.005100] EFI services will not be available.
[    0.005396] smp: Bringing up secondary CPUs ...
[    0.005904] Detected PIPT I-cache on CPU1
[    0.005963] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[    0.006505] Detected PIPT I-cache on CPU2
[    0.006544] CPU2: Booted secondary processor 0x0000000002 [0x410fd083]
[    0.007060] Detected PIPT I-cache on CPU3
[    0.007098] CPU3: Booted secondary processor 0x0000000003 [0x410fd083]
[    0.007182] smp: Brought up 1 node, 4 CPUs
[    0.007251] SMP: Total of 4 processors activated.
[    0.007267] CPU features: detected: 32-bit EL0 Support
[    0.007283] CPU features: detected: CRC32 instructions
[    0.007299] CPU features: detected: 32-bit EL1 Support
[    0.030649] CPU: All CPU(s) started at EL2
[    0.030710] alternatives: patching kernel code
[    0.032692] devtmpfs: initialized
[    0.037827] KASLR enabled
[    0.038277] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.038312] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.040163] pinctrl core: initialized pinctrl subsystem
[    0.041597] DMI not present or invalid.
[    0.042151] NET: Registered protocol family 16
[    0.043673] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.043826] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.044107] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.044163] audit: initializing netlink subsys (disabled)
[    0.044352] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[    0.045463] thermal_sys: Registered thermal governor 'step_wise'
[    0.045470] thermal_sys: Registered thermal governor 'power_allocator'
[    0.045807] cpuidle: using governor menu
[    0.046145] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.046246] ASID allocator initialised with 32768 entries
[    0.047838] Serial: AMBA PL011 UART driver
[    0.083072] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.083108] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.083126] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.083143] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.084507] cryptd: max_cpu_qlen set to 1000
[    0.087192] ACPI: Interpreter disabled.
[    0.089180] iommu: Default domain type: Translated 
[    0.089450] vgaarb: loaded
[    0.089800] SCSI subsystem initialized
[    0.090203] usbcore: registered new interface driver usbfs
[    0.090263] usbcore: registered new interface driver hub
[    0.090343] usbcore: registered new device driver usb
[    0.090520] usb_phy_generic phy: supply vcc not found, using dummy regulator
[    0.091346] pps_core: LinuxPPS API ver. 1 registered
[    0.091364] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.091394] PTP clock support registered
[    0.091552] EDAC MC: Ver: 3.0.0
[    0.092935] FPGA manager framework
[    0.093050] Advanced Linux Sound Architecture Driver Initialized.
[    0.093880] clocksource: Switched to clocksource arch_sys_counter
[    0.094112] VFS: Disk quotas dquot_6.6.0
[    0.094190] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.094401] pnp: PnP ACPI: disabled
[    0.100876] NET: Registered protocol family 2
[    0.101384] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.101444] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.101652] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.102173] TCP: Hash tables configured (established 32768 bind 32768)
[    0.102384] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102454] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102668] NET: Registered protocol family 1
[    0.103174] RPC: Registered named UNIX socket transport module.
[    0.103193] RPC: Registered udp transport module.
[    0.103206] RPC: Registered tcp transport module.
[    0.103220] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.103241] PCI: CLS 0 bytes, default 64
[    0.104437] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[    0.104591] kvm [1]: IPA Size Limit: 44bits
[    0.105723] kvm [1]: vgic interrupt IRQ1
[    0.105910] kvm [1]: Hyp mode initialized successfully
[    0.107799] Initialise system trusted keyrings
[    0.107975] workingset: timestamp_bits=44 max_order=20 bucket_order=0
[    0.114780] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.115569] NFS: Registering the id_resolver key type
[    0.115614] Key type id_resolver registered
[    0.115629] Key type id_legacy registered
[    0.115747] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.115926] 9p: Installing v9fs 9p2000 file system support
[    0.177452] Key type asymmetric registered
[    0.177472] Asymmetric key parser 'x509' registered
[    0.177527] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.177547] io scheduler mq-deadline registered
[    0.177560] io scheduler kyber registered
[    0.190013] EINJ: ACPI disabled.
[    0.208072] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.210194] printk: console [ttyS1] disabled
[    0.210297] fe215040.serial: ttyS1 at MMIO 0xfe215040 (irq = 18, base_baud = 24999999) is a 16550
[    1.115569] printk: console [ttyS1] enabled
[    1.121263] SuperH (H)SCI(F) driver initialized
[    1.126522] msm_serial: driver initialized
[    1.132301] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    1.146949] loop: module loaded
[    1.151363] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    1.159671] megasas: 07.714.04.00-rc1
[    1.168505] libphy: Fixed MDIO Bus: probed
[    1.174118] tun: Universal TUN/TAP device driver, 1.6
[    1.180090] thunder_xcv, ver 1.0
[    1.183434] thunder_bgx, ver 1.0
[    1.186768] nicpf, ver 1.0
[    1.190746] hclge is initializing
[    1.194178] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    1.201546] hns3: Copyright (c) 2017 Huawei Corporation.
[    1.207017] e1000: Intel(R) PRO/1000 Network Driver
[    1.211992] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    1.217901] e1000e: Intel(R) PRO/1000 Network Driver
[    1.222962] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.229030] igb: Intel(R) Gigabit Ethernet Network Driver
[    1.234535] igb: Copyright (c) 2007-2014 Intel Corporation.
[    1.240248] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    1.246633] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    1.253038] sky2: driver version 1.30
[    1.257975] VFIO - User Level meta-driver version: 0.3
[    1.265257] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.271935] ehci-pci: EHCI PCI platform driver
[    1.276511] ehci-platform: EHCI generic platform driver
[    1.281987] ehci-orion: EHCI orion driver
[    1.286186] ehci-exynos: EHCI Exynos driver
[    1.290552] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.296868] ohci-pci: OHCI PCI platform driver
[    1.301436] ohci-platform: OHCI generic platform driver
[    1.306891] ohci-exynos: OHCI Exynos driver
[    1.311695] usbcore: registered new interface driver usb-storage
[    1.320736] i2c /dev entries driver
[    1.329662] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    1.338167] sdhci: Secure Digital Host Controller Interface driver
[    1.344475] sdhci: Copyright(c) Pierre Ossman
[    1.349524] Synopsys Designware Multimedia Card Interface Driver
[    1.356759] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.375177] ledtrig-cpu: registered to indicate activity on CPUs
[    1.383060] usbcore: registered new interface driver usbhid
[    1.388747] usbhid: USB HID core driver
[    1.393728] bcm2835-mbox fe00b880.mailbox: mailbox enabled
[    1.403725] NET: Registered protocol family 17
[    1.408495] 9pnet: Installing 9P2000 support
[    1.412912] Key type dns_resolver registered
[    1.417632] registered taskstats version 1
[    1.421822] Loading compiled-in X.509 certificates
[    1.431630] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 16, base_baud = 0) is a PL011 rev2
[    1.440890] serial serial0: tty port ttyAMA0 registered
[    1.462581] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.503538] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.510983] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.619098] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.631122] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.669127] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.694102] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.701389] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.708672] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.717293] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.724735] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.735564] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.745322] ALSA device list:
[    1.748390]   No soundcards found.
[    1.752060] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.792429] random: fast init done
[    1.797033] mmc1: unrecognised SCR structure version 4
[    1.802357] mmc1: error -22 whilst initialising SD card
[    1.817686] mmc0: new high speed SDIO card at address 0001
[    1.886941] mmc1: unrecognised SCR structure version 2
[    1.892189] mmc1: error -22 whilst initialising SD card
[    1.967866] mmc1: unrecognised SCR structure version 4
[    1.973113] mmc1: error -22 whilst initialising SD card
[    2.065006] mmc1: unrecognised SCR structure version 4
[    2.070252] mmc1: error -22 whilst initialising SD card
[    3.161820] mmc1: unrecognised SCR structure version 4
[    3.167064] mmc1: error -22 whilst initialising SD card
[    3.237380] mmc1: unrecognised SCR structure version 4
[    3.242625] mmc1: error -22 whilst initialising SD card
[    3.318280] mmc1: unrecognised SCR structure version 4
[    3.323525] mmc1: error -22 whilst initialising SD card
[    3.415425] mmc1: unrecognised SCR structure version 4
[    3.420669] mmc1: error -22 whilst initialising SD card
[    4.505842] mmc1: unrecognised SCR structure version 4
[    4.511090] mmc1: error -22 whilst initialising SD card
[    4.581408] mmc1: unrecognised SCR structure version 4
[    4.586652] mmc1: error -22 whilst initialising SD card
[    4.662344] mmc1: unrecognised SCR structure version 4
[    4.667588] mmc1: error -22 whilst initialising SD card
[    4.759556] mmc1: unrecognised SCR structure version 4
[    4.764800] mmc1: error -22 whilst initialising SD card
[    5.851270] mmc1: unrecognised SCR structure version 4
[    5.856517] mmc1: error -22 whilst initialising SD card
[    5.926793] mmc1: unrecognised SCR structure version 4
[    5.932038] mmc1: error -22 whilst initialising SD card
[    6.007687] mmc1: unrecognised SCR structure version 4
[    6.012930] mmc1: error -22 whilst initialising SD card
[    6.104759] mmc1: unrecognised SCR structure version 4
[    6.110006] mmc1: error -22 whilst initialising SD card
[    7.193797] mmc1: unrecognised SCR structure version 4
[    7.199043] mmc1: error -22 whilst initialising SD card
[    7.269330] mmc1: unrecognised SCR structure version 4
[    7.274574] mmc1: error -22 whilst initialising SD card
[    7.350192] mmc1: unrecognised SCR structure version 4
[    7.355435] mmc1: error -22 whilst initialising SD card
[    7.447355] mmc1: unrecognised SCR structure version 4
[    7.452598] mmc1: error -22 whilst initialising SD card
[    8.537815] mmc1: unrecognised SCR structure version 4
[    8.543060] mmc1: error -22 whilst initialising SD card
[    8.613360] mmc1: unrecognised SCR structure version 4
[    8.618604] mmc1: error -22 whilst initialising SD card
[    8.694255] mmc1: unrecognised SCR structure version 4
[    8.699499] mmc1: error -22 whilst initialising SD card
[    8.791399] mmc1: unrecognised SCR structure version 4
[    8.796642] mmc1: error -22 whilst initialising SD card
[    9.881840] mmc1: unrecognised SCR structure version 4
[    9.887088] mmc1: error -22 whilst initialising SD card
[    9.957402] mmc1: unrecognised SCR structure version 4
[    9.962646] mmc1: error -22 whilst initialising SD card
[   10.038303] mmc1: unrecognised SCR structure version 4
[   10.043547] mmc1: error -22 whilst initialising SD card
[   10.135464] mmc1: unrecognised SCR structure version 4
[   10.140707] mmc1: error -22 whilst initialising SD card
[   11.225815] mmc1: unrecognised SCR structure version 4
[   11.231062] mmc1: error -22 whilst initialising SD card
[   11.301373] mmc1: unrecognised SCR structure version 4
[   11.306617] mmc1: error -22 whilst initialising SD card
[   11.382264] mmc1: unrecognised SCR structure version 4
[   11.387507] mmc1: error -22 whilst initialising SD card
[   11.479430] mmc1: unrecognised SCR structure version 4
[   11.484673] mmc1: error -22 whilst initialising SD card
[   12.569813] mmc1: unrecognised SCR structure version 4
[   12.575059] mmc1: error -22 whilst initialising SD card
[   12.645354] mmc1: unrecognised SCR structure version 4
[   12.650598] mmc1: error -22 whilst initialising SD card
[   12.726254] mmc1: unrecognised SCR structure version 4
[   12.731497] mmc1: error -22 whilst initialising SD card
[   12.823417] mmc1: unrecognised SCR structure version 4
[   12.828661] mmc1: error -22 whilst initialising SD card
[   13.913815] mmc1: unrecognised SCR structure version 4
[   13.919059] mmc1: error -22 whilst initialising SD card
[   13.989375] mmc1: unrecognised SCR structure version 4
[   13.994620] mmc1: error -22 whilst initialising SD card
[   14.070314] mmc1: unrecognised SCR structure version 4
[   14.075557] mmc1: error -22 whilst initialising SD card
[   14.167479] mmc1: unrecognised SCR structure version 4
[   14.172722] mmc1: error -22 whilst initialising SD card
[   15.257839] mmc1: unrecognised SCR structure version 4
[   15.263086] mmc1: error -22 whilst initialising SD card
[   15.333397] mmc1: unrecognised SCR structure version 4
[   15.338642] mmc1: error -22 whilst initialising SD card
[   15.414259] mmc1: unrecognised SCR structure version 4
[   15.419500] mmc1: error -22 whilst initialising SD card
[   15.511418] mmc1: unrecognised SCR structure version 4
[   15.516662] mmc1: error -22 whilst initialising SD card
[   16.601839] mmc1: unrecognised SCR structure version 4
[   16.607086] mmc1: error -22 whilst initialising SD card
[   16.677414] mmc1: unrecognised SCR structure version 4
[   16.682658] mmc1: error -22 whilst initialising SD card
[   16.758280] mmc1: unrecognised SCR structure version 4
[   16.763524] mmc1: error -22 whilst initialising SD card
[   16.855411] mmc1: unrecognised SCR structure version 4
[   16.860657] mmc1: error -22 whilst initialising SD card
[   17.945838] mmc1: unrecognised SCR structure version 4
[   17.951084] mmc1: error -22 whilst initialising SD card
[   18.022951] mmc1: unrecognised SCR structure version 4
[   18.028196] mmc1: error -22 whilst initialising SD card
[   18.103836] mmc1: unrecognised SCR structure version 4
[   18.109079] mmc1: error -22 whilst initialising SD card
[   18.200919] mmc1: unrecognised SCR structure version 4
[   18.206165] mmc1: error -22 whilst initialising SD card
[   19.289820] mmc1: unrecognised SCR structure version 4
[   19.295068] mmc1: error -22 whilst initialising SD card
[   19.365404] mmc1: unrecognised SCR structure version 4
[   19.370648] mmc1: error -22 whilst initialising SD card
[   19.446261] mmc1: unrecognised SCR structure version 4
[   19.451504] mmc1: error -22 whilst initialising SD card
[   19.543415] mmc1: unrecognised SCR structure version 4
[   19.548658] mmc1: error -22 whilst initialising SD card
[   20.633815] mmc1: unrecognised SCR structure version 4
[   20.639065] mmc1: error -22 whilst initialising SD card
[   20.709375] mmc1: unrecognised SCR structure version 4
[   20.714619] mmc1: error -22 whilst initialising SD card
[   20.790313] mmc1: unrecognised SCR structure version 4
[   20.795557] mmc1: error -22 whilst initialising SD card
[   20.887466] mmc1: unrecognised SCR structure version 4
[   20.892710] mmc1: error -22 whilst initialising SD card
[   21.977834] mmc1: unrecognised SCR structure version 4
[   21.983081] mmc1: error -22 whilst initialising SD card
[   22.053392] mmc1: unrecognised SCR structure version 4
[   22.058637] mmc1: error -22 whilst initialising SD card
[   22.134244] mmc1: unrecognised SCR structure version 4
[   22.139488] mmc1: error -22 whilst initialising SD card
[   22.231412] mmc1: unrecognised SCR structure version 4
[   22.236656] mmc1: error -22 whilst initialising SD card
[   23.321810] mmc1: unrecognised SCR structure version 4
[   23.327056] mmc1: error -22 whilst initialising SD card
[   23.397372] mmc1: unrecognised SCR structure version 4
[   23.402617] mmc1: error -22 whilst initialising SD card
[   23.478248] mmc1: unrecognised SCR structure version 4
[   23.483492] mmc1: error -22 whilst initialising SD card
[   23.575406] mmc1: unrecognised SCR structure version 4
[   23.580650] mmc1: error -22 whilst initialising SD card
[   24.665836] mmc1: unrecognised SCR structure version 4
[   24.671080] mmc1: error -22 whilst initialising SD card
[   24.741396] mmc1: unrecognised SCR structure version 4
[   24.746641] mmc1: error -22 whilst initialising SD card
[   24.822248] mmc1: unrecognised SCR structure version 4
[   24.827492] mmc1: error -22 whilst initialising SD card
[   24.919406] mmc1: unrecognised SCR structure version 4
[   24.924650] mmc1: error -22 whilst initialising SD card
[   26.009818] mmc1: unrecognised SCR structure version 4
[   26.015064] mmc1: error -22 whilst initialising SD card
[   26.085377] mmc1: unrecognised SCR structure version 4
[   26.090621] mmc1: error -22 whilst initialising SD card
[   26.166270] mmc1: unrecognised SCR structure version 4
[   26.171514] mmc1: error -22 whilst initialising SD card
[   26.263190] mmc1: unrecognised SCR structure version 4
[   26.268434] mmc1: error -22 whilst initialising SD card
[   27.353799] mmc1: unrecognised SCR structure version 4
[   27.359045] mmc1: error -22 whilst initialising SD card
[   27.429363] mmc1: unrecognised SCR structure version 4
[   27.434607] mmc1: error -22 whilst initialising SD card
[   27.512614] mmc1: unrecognised SCR structure version 4
[   27.517858] mmc1: error -22 whilst initialising SD card
[   27.609855] mmc1: unrecognised SCR structure version 4
[   27.615102] mmc1: error -22 whilst initialising SD card
[   28.697337] mmc1: unrecognised SCR structure version 4
[   28.702583] mmc1: error -22 whilst initialising SD card
[   28.772895] mmc1: unrecognised SCR structure version 4
[   28.778139] mmc1: error -22 whilst initialising SD card
[   28.854654] mmc1: unrecognised SCR structure version 4
[   28.859900] mmc1: error -22 whilst initialising SD card
[   28.951809] mmc1: unrecognised SCR structure version 4
[   28.957049] mmc1: error -22 whilst initialising SD card
[   30.041823] mmc1: unrecognised SCR structure version 4
[   30.047070] mmc1: error -22 whilst initialising SD card
[   30.117387] mmc1: unrecognised SCR structure version 4
[   30.122631] mmc1: error -22 whilst initialising SD card
[   30.198244] mmc1: unrecognised SCR structure version 4
[   30.203488] mmc1: error -22 whilst initialising SD card
[   30.295391] mmc1: unrecognised SCR structure version 4
[   30.300638] mmc1: error -22 whilst initialising SD card

[-- Attachment #3: pi-boot-good.txt --]
[-- Type: text/plain, Size: 15985 bytes --]

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-next-20200902-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 15:28:17 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000037400000, size 64 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] NUMA: NODE_DATA [mem 0xfb81f100-0xfb820fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x000000003fffffff]
[    0.000000]   DMA32    [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000003b3fffff]
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: EL2 vector hardening
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] ARM_SMCCC_ARCH_WORKAROUND_1 missing from firmware
[    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 996912
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:  dma.dmachans=0x71f5 bcm2709.boardrev=0xc03112 bcm2709.serial=0xb78d398 bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:60:6C:87 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=ttyS1,115200 console=tty1 root=PARTUUID=45a8dd8a-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait plymouth.ignore-serial-consoles
[    0.000000] Kernel parameter elevator= does not have any effect anymore.
[    0.000000] Please use sysfs to set IO scheduler for individual devices.
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x33400000-0x37400000] (64MB)
[    0.000000] Memory: 3812260K/4050944K available (14524K kernel code, 2636K rwdata, 7540K rodata, 1600K init, 615K bss, 173148K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] random: get_random_bytes called from start_kernel+0x1cc/0x3f4 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 54.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123 ns
[    0.000006] sched_clock: 56 bits at 54MHz, resolution 18ns, wraps every 4398046511102ns
[    0.000195] Console: colour dummy device 80x25
[    0.000678] printk: console [tty1] enabled
[    0.000778] Calibrating delay loop (skipped), value calculated using timer frequency.. 108.00 BogoMIPS (lpj=216000)
[    0.000808] pid_max: default: 32768 minimum: 301
[    0.000894] LSM: Security Framework initializing
[    0.001030] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.001116] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.002878] rcu: Hierarchical SRCU implementation.
[    0.005048] EFI services will not be available.
[    0.005338] smp: Bringing up secondary CPUs ...
[    0.005844] Detected PIPT I-cache on CPU1
[    0.005903] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[    0.006454] Detected PIPT I-cache on CPU2
[    0.006493] CPU2: Booted secondary processor 0x0000000002 [0x410fd083]
[    0.007012] Detected PIPT I-cache on CPU3
[    0.007051] CPU3: Booted secondary processor 0x0000000003 [0x410fd083]
[    0.007133] smp: Brought up 1 node, 4 CPUs
[    0.007198] SMP: Total of 4 processors activated.
[    0.007214] CPU features: detected: 32-bit EL0 Support
[    0.007229] CPU features: detected: CRC32 instructions
[    0.007245] CPU features: detected: 32-bit EL1 Support
[    0.030492] CPU: All CPU(s) started at EL2
[    0.030550] alternatives: patching kernel code
[    0.032524] devtmpfs: initialized
[    0.037648] KASLR enabled
[    0.038091] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.038125] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.039986] pinctrl core: initialized pinctrl subsystem
[    0.041393] DMI not present or invalid.
[    0.041944] NET: Registered protocol family 16
[    0.043458] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.043610] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.043887] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.043939] audit: initializing netlink subsys (disabled)
[    0.044132] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[    0.045217] thermal_sys: Registered thermal governor 'step_wise'
[    0.045225] thermal_sys: Registered thermal governor 'power_allocator'
[    0.045555] cpuidle: using governor menu
[    0.045877] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.045978] ASID allocator initialised with 32768 entries
[    0.047544] Serial: AMBA PL011 UART driver
[    0.082787] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.082822] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.082840] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.082856] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.084212] cryptd: max_cpu_qlen set to 1000
[    0.086893] ACPI: Interpreter disabled.
[    0.088884] iommu: Default domain type: Translated 
[    0.089151] vgaarb: loaded
[    0.089505] SCSI subsystem initialized
[    0.089896] usbcore: registered new interface driver usbfs
[    0.089955] usbcore: registered new interface driver hub
[    0.090035] usbcore: registered new device driver usb
[    0.090208] usb_phy_generic phy: supply vcc not found, using dummy regulator
[    0.091009] pps_core: LinuxPPS API ver. 1 registered
[    0.091026] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.091055] PTP clock support registered
[    0.091238] EDAC MC: Ver: 3.0.0
[    0.092605] FPGA manager framework
[    0.092719] Advanced Linux Sound Architecture Driver Initialized.
[    0.093558] clocksource: Switched to clocksource arch_sys_counter
[    0.093791] VFS: Disk quotas dquot_6.6.0
[    0.093865] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.094072] pnp: PnP ACPI: disabled
[    0.100547] NET: Registered protocol family 2
[    0.101063] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.101121] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.101328] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.101841] TCP: Hash tables configured (established 32768 bind 32768)
[    0.102055] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102123] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102329] NET: Registered protocol family 1
[    0.102824] RPC: Registered named UNIX socket transport module.
[    0.102842] RPC: Registered udp transport module.
[    0.102855] RPC: Registered tcp transport module.
[    0.102868] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.102888] PCI: CLS 0 bytes, default 64
[    0.104073] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[    0.104224] kvm [1]: IPA Size Limit: 44bits
[    0.105353] kvm [1]: vgic interrupt IRQ1
[    0.105504] kvm [1]: Hyp mode initialized successfully
[    0.107404] Initialise system trusted keyrings
[    0.107581] workingset: timestamp_bits=44 max_order=20 bucket_order=0
[    0.114285] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.115072] NFS: Registering the id_resolver key type
[    0.115111] Key type id_resolver registered
[    0.115125] Key type id_legacy registered
[    0.115242] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.115415] 9p: Installing v9fs 9p2000 file system support
[    0.178130] Key type asymmetric registered
[    0.178149] Asymmetric key parser 'x509' registered
[    0.178205] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.178225] io scheduler mq-deadline registered
[    0.178239] io scheduler kyber registered
[    0.190560] EINJ: ACPI disabled.
[    0.208427] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.210486] printk: console [ttyS1] disabled
[    0.210587] fe215040.serial: ttyS1 at MMIO 0xfe215040 (irq = 18, base_baud = 24999999) is a 16550
[    1.115809] printk: console [ttyS1] enabled
[    1.121521] SuperH (H)SCI(F) driver initialized
[    1.126760] msm_serial: driver initialized
[    1.132523] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    1.147108] loop: module loaded
[    1.151491] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    1.159778] megasas: 07.714.04.00-rc1
[    1.168551] libphy: Fixed MDIO Bus: probed
[    1.174162] tun: Universal TUN/TAP device driver, 1.6
[    1.180136] thunder_xcv, ver 1.0
[    1.183480] thunder_bgx, ver 1.0
[    1.186813] nicpf, ver 1.0
[    1.190780] hclge is initializing
[    1.194215] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    1.201583] hns3: Copyright (c) 2017 Huawei Corporation.
[    1.207048] e1000: Intel(R) PRO/1000 Network Driver
[    1.212023] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    1.217916] e1000e: Intel(R) PRO/1000 Network Driver
[    1.222976] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.229054] igb: Intel(R) Gigabit Ethernet Network Driver
[    1.234559] igb: Copyright (c) 2007-2014 Intel Corporation.
[    1.240268] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    1.246653] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    1.253048] sky2: driver version 1.30
[    1.257983] VFIO - User Level meta-driver version: 0.3
[    1.265221] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.271898] ehci-pci: EHCI PCI platform driver
[    1.276471] ehci-platform: EHCI generic platform driver
[    1.281934] ehci-orion: EHCI orion driver
[    1.286133] ehci-exynos: EHCI Exynos driver
[    1.290505] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.296820] ohci-pci: OHCI PCI platform driver
[    1.301386] ohci-platform: OHCI generic platform driver
[    1.306836] ohci-exynos: OHCI Exynos driver
[    1.311632] usbcore: registered new interface driver usb-storage
[    1.320631] i2c /dev entries driver
[    1.329492] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    1.337993] sdhci: Secure Digital Host Controller Interface driver
[    1.344303] sdhci: Copyright(c) Pierre Ossman
[    1.349337] Synopsys Designware Multimedia Card Interface Driver
[    1.356563] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.374853] ledtrig-cpu: registered to indicate activity on CPUs
[    1.382707] usbcore: registered new interface driver usbhid
[    1.388393] usbhid: USB HID core driver
[    1.393348] bcm2835-mbox fe00b880.mailbox: mailbox enabled
[    1.403299] NET: Registered protocol family 17
[    1.408070] 9pnet: Installing 9P2000 support
[    1.412487] Key type dns_resolver registered
[    1.417194] registered taskstats version 1
[    1.421390] Loading compiled-in X.509 certificates
[    1.431180] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 16, base_baud = 0) is a PL011 rev2
[    1.440427] serial serial0: tty port ttyAMA0 registered
[    1.461855] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.505592] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.513027] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.621089] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.633123] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.671111] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.696128] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.703412] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.710695] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.719313] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.726750] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.737585] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.747325] ALSA device list:
[    1.750387]   No soundcards found.
[    1.754070] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.793779] random: fast init done
[    1.810764] mmc0: new high speed SDIO card at address 0001
[    1.858064] mmc1: new ultra high speed DDR50 SDXC card at address aaaa
[    1.865420] mmcblk1: mmc1:aaaa SK64G 59.5 GiB 
[    1.871966]  mmcblk1: p1 p2
[    1.897160] EXT4-fs (mmcblk1p2): mounted filesystem with ordered data mode. Opts: (null)
[    1.905492] VFS: Mounted root (ext4 filesystem) readonly on device 179:2.
[    1.913206] devtmpfs: mounted
[    1.917652] Freeing unused kernel memory: 1600K
[    1.933776] Run /sbin/init as init process
[    2.314733] systemd[1]: System time before build time, advancing clock.
[    2.661797] NET: Registered protocol family 10
[    2.667365] Segment Routing with IPv6
[    2.730477] systemd[1]: systemd 241 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN2 +IDN -PCRE2 default-hierarchy=hybrid)
[    2.752685] systemd[1]: Detected architecture arm64.
[    2.792164] systemd[1]: Set hostname to <raspberrypi>.
[    3.017532] systemd[1]: File /lib/systemd/system/systemd-journald.service:12 configures an IP firewall (IPAddressDeny=any), but the local system does not support BPF/cgroup based firewalling.
[    3.034988] systemd[1]: Proceeding WITHOUT firewalling in effect! (This warning is only shown for the first loaded unit using IP firewalling.)
[    3.309831] random: systemd: uninitialized urandom read (16 bytes read)
[    3.321952] random: systemd: uninitialized urandom read (16 bytes read)
[    3.329307] systemd[1]: Listening on Journal Socket (/dev/log).
[    3.335968] random: systemd: uninitialized urandom read (16 bytes read)
[    3.345903] systemd[1]: Created slice system-serial\x2dgetty.slice.
[    3.353269] systemd[1]: Listening on Journal Audit Socket.
[    3.360213] systemd[1]: Created slice system-getty.slice.
[    3.366141] systemd[1]: Reached target Swap.

[-- Attachment #4: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Florian Fainelli]

On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
[snip]
>> Hello Nathan,
>>
>> Can you tell me how much memory your RPI has and if all of it is
> 
> This is the 4GB version.
> 
>> accessible by the PCIe device?  Could you also please include the DTS
>> of the PCIe node?  IIRC, the RPI firmware does some mangling of the
>> PCIe DT before Linux boots -- could you describe what is going on
>> there?
> 
> Unfortunately, I am not familiar with how to get this information. If
> you could provide some instructions for how to do so, I am more than
> happy to. I am not very knowleagable about the inner working of the Pi,
> I mainly use it as a test platform for making sure that LLVM does not
> cause problems on real devices.

Can you bring the dtc application to your Pi root filesystem, and if so, 
can you run the following:

dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb

or cat /sys/firmware/fdt > device.dtb

and attach the resulting file?

> 
>> Finally, can you attach the text of the full boot log?
> 
> I have attached a working and broken boot log. Thank you for the quick
> response!

Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by 
any chance?

I have a suspicion that this part of the DTS for the bcm2711.dtsi 
platform is at fault:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/bcm2711.dtsi#n264

and the resulting dma-ranges parsing is just not working for reasons to 
be determined.
--
Florian
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Florian Fainelli]

On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
[snip]
>> Hello Nathan,
>>
>> Can you tell me how much memory your RPI has and if all of it is
> 
> This is the 4GB version.
> 
>> accessible by the PCIe device?  Could you also please include the DTS
>> of the PCIe node?  IIRC, the RPI firmware does some mangling of the
>> PCIe DT before Linux boots -- could you describe what is going on
>> there?
> 
> Unfortunately, I am not familiar with how to get this information. If
> you could provide some instructions for how to do so, I am more than
> happy to. I am not very knowleagable about the inner working of the Pi,
> I mainly use it as a test platform for making sure that LLVM does not
> cause problems on real devices.

Can you bring the dtc application to your Pi root filesystem, and if so, 
can you run the following:

dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb

or cat /sys/firmware/fdt > device.dtb

and attach the resulting file?

> 
>> Finally, can you attach the text of the full boot log?
> 
> I have attached a working and broken boot log. Thank you for the quick
> response!

Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by 
any chance?

I have a suspicion that this part of the DTS for the bcm2711.dtsi 
platform is at fault:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/bcm2711.dtsi#n264

and the resulting dma-ranges parsing is just not working for reasons to 
be determined.
--
Florian
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Florian Fainelli]

On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
[snip]
>> Hello Nathan,
>>
>> Can you tell me how much memory your RPI has and if all of it is
> 
> This is the 4GB version.
> 
>> accessible by the PCIe device?  Could you also please include the DTS
>> of the PCIe node?  IIRC, the RPI firmware does some mangling of the
>> PCIe DT before Linux boots -- could you describe what is going on
>> there?
> 
> Unfortunately, I am not familiar with how to get this information. If
> you could provide some instructions for how to do so, I am more than
> happy to. I am not very knowleagable about the inner working of the Pi,
> I mainly use it as a test platform for making sure that LLVM does not
> cause problems on real devices.

Can you bring the dtc application to your Pi root filesystem, and if so, 
can you run the following:

dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb

or cat /sys/firmware/fdt > device.dtb

and attach the resulting file?

> 
>> Finally, can you attach the text of the full boot log?
> 
> I have attached a working and broken boot log. Thank you for the quick
> response!

Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by 
any chance?

I have a suspicion that this part of the DTS for the bcm2711.dtsi 
platform is at fault:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/bcm2711.dtsi#n264

and the resulting dma-ranges parsing is just not working for reasons to 
be determined.
--
Florian
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

[-- Attachment #1: Type: text/plain, Size: 1823 bytes --]

On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> 
> 
> On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> [snip]
> > > Hello Nathan,
> > > 
> > > Can you tell me how much memory your RPI has and if all of it is
> > 
> > This is the 4GB version.
> > 
> > > accessible by the PCIe device?  Could you also please include the DTS
> > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > PCIe DT before Linux boots -- could you describe what is going on
> > > there?
> > 
> > Unfortunately, I am not familiar with how to get this information. If
> > you could provide some instructions for how to do so, I am more than
> > happy to. I am not very knowleagable about the inner working of the Pi,
> > I mainly use it as a test platform for making sure that LLVM does not
> > cause problems on real devices.
> 
> Can you bring the dtc application to your Pi root filesystem, and if so, can
> you run the following:
> 
> dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb

Sure, the result is attached.

> or cat /sys/firmware/fdt > device.dtb
> 
> and attach the resulting file?
> 
> > 
> > > Finally, can you attach the text of the full boot log?
> > 
> > I have attached a working and broken boot log. Thank you for the quick
> > response!
> 
> Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> chance?

Of course. A new log is attached with the debug output from that config.

> I have a suspicion that this part of the DTS for the bcm2711.dtsi platform
> is at fault:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/bcm2711.dtsi#n264
> 
> and the resulting dma-ranges parsing is just not working for reasons to be
> determined.
> --
> Florian

Let me know if you need anything else out of me.

Cheers,
Nathan

[-- Attachment #2: device.dtb --]
[-- Type: application/octet-stream, Size: 24746 bytes --]

[-- Attachment #3: broken-pi-boot-debug.txt --]
[-- Type: text/plain, Size: 19118 bytes --]

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-next-20200902-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 17:41:42 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000037400000, size 64 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] NUMA: NODE_DATA [mem 0xfb81f100-0xfb820fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x000000003fffffff]
[    0.000000]   DMA32    [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000003b3fffff]
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: EL2 vector hardening
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] ARM_SMCCC_ARCH_WORKAROUND_1 missing from firmware
[    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 996912
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:  dma.dmachans=0x71f5 bcm2709.boardrev=0xc03112 bcm2709.serial=0xb78d398 bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:60:6C:87 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=ttyS1,115200 console=tty1 root=PARTUUID=45a8dd8a-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait plymouth.ignore-serial-consoles
[    0.000000] Kernel parameter elevator= does not have any effect anymore.
[    0.000000] Please use sysfs to set IO scheduler for individual devices.
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x33400000-0x37400000] (64MB)
[    0.000000] Memory: 3812260K/4050944K available (14524K kernel code, 2636K rwdata, 7540K rodata, 1600K init, 615K bss, 173148K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] random: get_random_bytes called from start_kernel+0x1cc/0x3f4 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 54.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123 ns
[    0.000006] sched_clock: 56 bits at 54MHz, resolution 18ns, wraps every 4398046511102ns
[    0.000195] Console: colour dummy device 80x25
[    0.000652] printk: console [tty1] enabled
[    0.000751] Calibrating delay loop (skipped), value calculated using timer frequency.. 108.00 BogoMIPS (lpj=216000)
[    0.000781] pid_max: default: 32768 minimum: 301
[    0.000866] LSM: Security Framework initializing
[    0.000997] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.001081] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.002840] rcu: Hierarchical SRCU implementation.
[    0.005054] EFI services will not be available.
[    0.005345] smp: Bringing up secondary CPUs ...
[    0.005851] Detected PIPT I-cache on CPU1
[    0.005911] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[    0.006455] Detected PIPT I-cache on CPU2
[    0.006495] CPU2: Booted secondary processor 0x0000000002 [0x410fd083]
[    0.007007] Detected PIPT I-cache on CPU3
[    0.007046] CPU3: Booted secondary processor 0x0000000003 [0x410fd083]
[    0.007129] smp: Brought up 1 node, 4 CPUs
[    0.007193] SMP: Total of 4 processors activated.
[    0.007209] CPU features: detected: 32-bit EL0 Support
[    0.007224] CPU features: detected: CRC32 instructions
[    0.007240] CPU features: detected: 32-bit EL1 Support
[    0.030540] CPU: All CPU(s) started at EL2
[    0.030599] alternatives: patching kernel code
[    0.032564] devtmpfs: initialized
[    0.037682] KASLR enabled
[    0.038172] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.038206] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.040116] pinctrl core: initialized pinctrl subsystem
[    0.041452] DMI not present or invalid.
[    0.042019] NET: Registered protocol family 16
[    0.043558] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.043712] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.043990] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.044050] audit: initializing netlink subsys (disabled)
[    0.044242] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[    0.045301] thermal_sys: Registered thermal governor 'step_wise'
[    0.045309] thermal_sys: Registered thermal governor 'power_allocator'
[    0.045648] cpuidle: using governor menu
[    0.045945] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.046047] ASID allocator initialised with 32768 entries
[    0.047600] Serial: AMBA PL011 UART driver
[    0.083025] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.083061] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.083078] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.083094] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.084421] cryptd: max_cpu_qlen set to 1000
[    0.087088] ACPI: Interpreter disabled.
[    0.089021] iommu: Default domain type: Translated 
[    0.089276] vgaarb: loaded
[    0.089611] SCSI subsystem initialized
[    0.090012] usbcore: registered new interface driver usbfs
[    0.090072] usbcore: registered new interface driver hub
[    0.090151] usbcore: registered new device driver usb
[    0.090330] usb_phy_generic phy: supply vcc not found, using dummy regulator
[    0.091142] pps_core: LinuxPPS API ver. 1 registered
[    0.091160] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.091196] PTP clock support registered
[    0.091357] EDAC MC: Ver: 3.0.0
[    0.092760] FPGA manager framework
[    0.092865] Advanced Linux Sound Architecture Driver Initialized.
[    0.093701] clocksource: Switched to clocksource arch_sys_counter
[    0.093934] VFS: Disk quotas dquot_6.6.0
[    0.094010] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.094227] pnp: PnP ACPI: disabled
[    0.100879] NET: Registered protocol family 2
[    0.101392] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.101450] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.101659] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.102140] TCP: Hash tables configured (established 32768 bind 32768)
[    0.102352] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102422] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102628] NET: Registered protocol family 1
[    0.103133] RPC: Registered named UNIX socket transport module.
[    0.103152] RPC: Registered udp transport module.
[    0.103165] RPC: Registered tcp transport module.
[    0.103178] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.103199] PCI: CLS 0 bytes, default 64
[    0.104385] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[    0.104540] kvm [1]: IPA Size Limit: 44bits
[    0.105680] kvm [1]: vgic interrupt IRQ1
[    0.105867] kvm [1]: Hyp mode initialized successfully
[    0.107728] Initialise system trusted keyrings
[    0.107883] workingset: timestamp_bits=44 max_order=20 bucket_order=0
[    0.114719] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.115521] NFS: Registering the id_resolver key type
[    0.115561] Key type id_resolver registered
[    0.115575] Key type id_legacy registered
[    0.115680] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.115865] 9p: Installing v9fs 9p2000 file system support
[    0.178003] Key type asymmetric registered
[    0.178022] Asymmetric key parser 'x509' registered
[    0.178076] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.178095] io scheduler mq-deadline registered
[    0.178108] io scheduler kyber registered
[    0.190559] EINJ: ACPI disabled.
[    0.208669] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.210741] printk: console [ttyS1] disabled
[    0.210844] fe215040.serial: ttyS1 at MMIO 0xfe215040 (irq = 18, base_baud = 24999999) is a 16550
[    1.116050] printk: console [ttyS1] enabled
[    1.121777] SuperH (H)SCI(F) driver initialized
[    1.127007] msm_serial: driver initialized
[    1.132787] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    1.147454] loop: module loaded
[    1.151881] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    1.160186] megasas: 07.714.04.00-rc1
[    1.168988] libphy: Fixed MDIO Bus: probed
[    1.174593] tun: Universal TUN/TAP device driver, 1.6
[    1.180565] thunder_xcv, ver 1.0
[    1.183908] thunder_bgx, ver 1.0
[    1.187241] nicpf, ver 1.0
[    1.191216] hclge is initializing
[    1.194649] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    1.202014] hns3: Copyright (c) 2017 Huawei Corporation.
[    1.207482] e1000: Intel(R) PRO/1000 Network Driver
[    1.212455] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    1.218347] e1000e: Intel(R) PRO/1000 Network Driver
[    1.223407] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.229481] igb: Intel(R) Gigabit Ethernet Network Driver
[    1.234987] igb: Copyright (c) 2007-2014 Intel Corporation.
[    1.240693] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    1.247076] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    1.253477] sky2: driver version 1.30
[    1.258420] VFIO - User Level meta-driver version: 0.3
[    1.265684] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.272366] ehci-pci: EHCI PCI platform driver
[    1.276933] ehci-platform: EHCI generic platform driver
[    1.282396] ehci-orion: EHCI orion driver
[    1.286594] ehci-exynos: EHCI Exynos driver
[    1.290961] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.297274] ohci-pci: OHCI PCI platform driver
[    1.301842] ohci-platform: OHCI generic platform driver
[    1.307292] ohci-exynos: OHCI Exynos driver
[    1.312095] usbcore: registered new interface driver usb-storage
[    1.321120] i2c /dev entries driver
[    1.330063] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    1.338533] sdhci: Secure Digital Host Controller Interface driver
[    1.344845] sdhci: Copyright(c) Pierre Ossman
[    1.349891] Synopsys Designware Multimedia Card Interface Driver
[    1.357122] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.375006] ledtrig-cpu: registered to indicate activity on CPUs
[    1.382894] usbcore: registered new interface driver usbhid
[    1.388580] usbhid: USB HID core driver
[    1.393564] bcm2835-mbox fe00b880.mailbox: mailbox enabled
[    1.403535] NET: Registered protocol family 17
[    1.408294] 9pnet: Installing 9P2000 support
[    1.412713] Key type dns_resolver registered
[    1.417430] registered taskstats version 1
[    1.421626] Loading compiled-in X.509 certificates
[    1.431456] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 16, base_baud = 0) is a PL011 rev2
[    1.440702] serial serial0: tty port ttyAMA0 registered
[    1.462488] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.503490] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.510942] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.619058] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.631093] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.656482] mmc0: Got data interrupt 0x00100000 even though no data operation was in progress.
[    1.665246] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[    1.671799] mmc0: sdhci: Sys addr:  0x00000000 | Version:  0x00009902
[    1.678349] mmc0: sdhci: Blk size:  0x00000000 | Blk cnt:  0x00000000
[    1.684899] mmc0: sdhci: Argument:  0x00000000 | Trn mode: 0x00000000
[    1.691448] mmc0: sdhci: Present:   0x01ff0000 | Host ctl: 0x00000000
[    1.697998] mmc0: sdhci: Power:     0x0000000f | Blk gap:  0x00000000
[    1.704548] mmc0: sdhci: Wake-up:   0x00000000 | Clock:    0x00003943
[    1.711097] mmc0: sdhci: Timeout:   0x00000000 | Int stat: 0x00000000
[    1.717646] mmc0: sdhci: Int enab:  0x00ff0003 | Sig enab: 0x00ff0003
[    1.724195] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000000
[    1.730745] mmc0: sdhci: Caps:      0x00000000 | Caps_1:   0x00000000
[    1.737293] mmc0: sdhci: Cmd:       0x00000000 | Max curr: 0x00000001
[    1.743842] mmc0: sdhci: Resp[0]:   0x00000000 | Resp[1]:  0x00000000
[    1.750390] mmc0: sdhci: Resp[2]:   0x00000000 | Resp[3]:  0x00000000
[    1.756939] mmc0: sdhci: Host ctl2: 0x00000000
[    1.761456] mmc0: sdhci: ============================================
[    1.780655] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.805771] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.813060] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.820345] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.828965] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.836403] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.845718] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.855465] ALSA device list:
[    1.858526]   No soundcards found.
[    1.862198] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.902568] mmc1: unrecognised SCR structure version 4
[    1.907898] mmc1: error -22 whilst initialising SD card
[    1.913254] random: fast init done
[    1.929590] mmc0: new high speed SDIO card at address 0001
[    1.992339] mmc1: unrecognised SCR structure version 2
[    1.997586] mmc1: error -22 whilst initialising SD card
[    2.073236] mmc1: unrecognised SCR structure version 4
[    2.078483] mmc1: error -22 whilst initialising SD card
[    2.170487] mmc1: unrecognised SCR structure version 4
[    2.175734] mmc1: error -22 whilst initialising SD card
[    3.257662] mmc1: unrecognised SCR structure version 4
[    3.262908] mmc1: error -22 whilst initialising SD card
[    3.333247] mmc1: unrecognised SCR structure version 4
[    3.338491] mmc1: error -22 whilst initialising SD card
[    3.414071] mmc1: unrecognised SCR structure version 4
[    3.419316] mmc1: error -22 whilst initialising SD card
[    3.511237] mmc1: unrecognised SCR structure version 4
[    3.516480] mmc1: error -22 whilst initialising SD card
[    4.601638] mmc1: unrecognised SCR structure version 4
[    4.606884] mmc1: error -22 whilst initialising SD card
[    4.677201] mmc1: unrecognised SCR structure version 4
[    4.682446] mmc1: error -22 whilst initialising SD card
[    4.758107] mmc1: unrecognised SCR structure version 4
[    4.763350] mmc1: error -22 whilst initialising SD card
[    4.855243] mmc1: unrecognised SCR structure version 4
[    4.860490] mmc1: error -22 whilst initialising SD card
[    5.945660] mmc1: unrecognised SCR structure version 4
[    5.950907] mmc1: error -22 whilst initialising SD card
[    6.021221] mmc1: unrecognised SCR structure version 4
[    6.026465] mmc1: error -22 whilst initialising SD card
[    6.102080] mmc1: unrecognised SCR structure version 4
[    6.107324] mmc1: error -22 whilst initialising SD card
[    6.199241] mmc1: unrecognised SCR structure version 4
[    6.204484] mmc1: error -22 whilst initialising SD card
[    7.289614] mmc1: unrecognised SCR structure version 4
[    7.294860] mmc1: error -22 whilst initialising SD card
[    7.365166] mmc1: unrecognised SCR structure version 4
[    7.370409] mmc1: error -22 whilst initialising SD card
[    7.446022] mmc1: unrecognised SCR structure version 4
[    7.451265] mmc1: error -22 whilst initialising SD card
[    7.543170] mmc1: unrecognised SCR structure version 4
[    7.548413] mmc1: error -22 whilst initialising SD card
[    8.636441] mmc1: unrecognised SCR structure version 4
[    8.641697] mmc1: error -22 whilst initialising SD card
[    8.711995] mmc1: unrecognised SCR structure version 4
[    8.717238] mmc1: error -22 whilst initialising SD card
[    8.792928] mmc1: unrecognised SCR structure version 4
[    8.798171] mmc1: error -22 whilst initialising SD card
[    8.890158] mmc1: unrecognised SCR structure version 4
[    8.895402] mmc1: error -22 whilst initialising SD card
[    9.977661] mmc1: unrecognised SCR structure version 4
[    9.982907] mmc1: error -22 whilst initialising SD card
[   10.053223] mmc1: unrecognised SCR structure version 4
[   10.058467] mmc1: error -22 whilst initialising SD card
[   10.134126] mmc1: unrecognised SCR structure version 4
[   10.139370] mmc1: error -22 whilst initialising SD card
[   10.231286] mmc1: unrecognised SCR structure version 4
[   10.236528] mmc1: error -22 whilst initialising SD card
[   11.321615] mmc1: unrecognised SCR structure version 4
[   11.326861] mmc1: error -22 whilst initialising SD card
[   11.397149] mmc1: unrecognised SCR structure version 4
[   11.402394] mmc1: error -22 whilst initialising SD card
[   11.478010] mmc1: unrecognised SCR structure version 4
[   11.483252] mmc1: error -22 whilst initialising SD card
[   11.575174] mmc1: unrecognised SCR structure version 4
[   11.580416] mmc1: error -22 whilst initialising SD card

[-- Attachment #4: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

[-- Attachment #1: Type: text/plain, Size: 1823 bytes --]

On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> 
> 
> On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> [snip]
> > > Hello Nathan,
> > > 
> > > Can you tell me how much memory your RPI has and if all of it is
> > 
> > This is the 4GB version.
> > 
> > > accessible by the PCIe device?  Could you also please include the DTS
> > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > PCIe DT before Linux boots -- could you describe what is going on
> > > there?
> > 
> > Unfortunately, I am not familiar with how to get this information. If
> > you could provide some instructions for how to do so, I am more than
> > happy to. I am not very knowleagable about the inner working of the Pi,
> > I mainly use it as a test platform for making sure that LLVM does not
> > cause problems on real devices.
> 
> Can you bring the dtc application to your Pi root filesystem, and if so, can
> you run the following:
> 
> dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb

Sure, the result is attached.

> or cat /sys/firmware/fdt > device.dtb
> 
> and attach the resulting file?
> 
> > 
> > > Finally, can you attach the text of the full boot log?
> > 
> > I have attached a working and broken boot log. Thank you for the quick
> > response!
> 
> Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> chance?

Of course. A new log is attached with the debug output from that config.

> I have a suspicion that this part of the DTS for the bcm2711.dtsi platform
> is at fault:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/bcm2711.dtsi#n264
> 
> and the resulting dma-ranges parsing is just not working for reasons to be
> determined.
> --
> Florian

Let me know if you need anything else out of me.

Cheers,
Nathan

[-- Attachment #2: device.dtb --]
[-- Type: application/octet-stream, Size: 24746 bytes --]

[-- Attachment #3: broken-pi-boot-debug.txt --]
[-- Type: text/plain, Size: 19118 bytes --]

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-next-20200902-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 17:41:42 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000037400000, size 64 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] NUMA: NODE_DATA [mem 0xfb81f100-0xfb820fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x000000003fffffff]
[    0.000000]   DMA32    [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000003b3fffff]
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: EL2 vector hardening
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] ARM_SMCCC_ARCH_WORKAROUND_1 missing from firmware
[    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 996912
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:  dma.dmachans=0x71f5 bcm2709.boardrev=0xc03112 bcm2709.serial=0xb78d398 bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:60:6C:87 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=ttyS1,115200 console=tty1 root=PARTUUID=45a8dd8a-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait plymouth.ignore-serial-consoles
[    0.000000] Kernel parameter elevator= does not have any effect anymore.
[    0.000000] Please use sysfs to set IO scheduler for individual devices.
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x33400000-0x37400000] (64MB)
[    0.000000] Memory: 3812260K/4050944K available (14524K kernel code, 2636K rwdata, 7540K rodata, 1600K init, 615K bss, 173148K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] random: get_random_bytes called from start_kernel+0x1cc/0x3f4 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 54.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123 ns
[    0.000006] sched_clock: 56 bits at 54MHz, resolution 18ns, wraps every 4398046511102ns
[    0.000195] Console: colour dummy device 80x25
[    0.000652] printk: console [tty1] enabled
[    0.000751] Calibrating delay loop (skipped), value calculated using timer frequency.. 108.00 BogoMIPS (lpj=216000)
[    0.000781] pid_max: default: 32768 minimum: 301
[    0.000866] LSM: Security Framework initializing
[    0.000997] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.001081] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.002840] rcu: Hierarchical SRCU implementation.
[    0.005054] EFI services will not be available.
[    0.005345] smp: Bringing up secondary CPUs ...
[    0.005851] Detected PIPT I-cache on CPU1
[    0.005911] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[    0.006455] Detected PIPT I-cache on CPU2
[    0.006495] CPU2: Booted secondary processor 0x0000000002 [0x410fd083]
[    0.007007] Detected PIPT I-cache on CPU3
[    0.007046] CPU3: Booted secondary processor 0x0000000003 [0x410fd083]
[    0.007129] smp: Brought up 1 node, 4 CPUs
[    0.007193] SMP: Total of 4 processors activated.
[    0.007209] CPU features: detected: 32-bit EL0 Support
[    0.007224] CPU features: detected: CRC32 instructions
[    0.007240] CPU features: detected: 32-bit EL1 Support
[    0.030540] CPU: All CPU(s) started at EL2
[    0.030599] alternatives: patching kernel code
[    0.032564] devtmpfs: initialized
[    0.037682] KASLR enabled
[    0.038172] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.038206] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.040116] pinctrl core: initialized pinctrl subsystem
[    0.041452] DMI not present or invalid.
[    0.042019] NET: Registered protocol family 16
[    0.043558] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.043712] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.043990] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.044050] audit: initializing netlink subsys (disabled)
[    0.044242] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[    0.045301] thermal_sys: Registered thermal governor 'step_wise'
[    0.045309] thermal_sys: Registered thermal governor 'power_allocator'
[    0.045648] cpuidle: using governor menu
[    0.045945] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.046047] ASID allocator initialised with 32768 entries
[    0.047600] Serial: AMBA PL011 UART driver
[    0.083025] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.083061] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.083078] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.083094] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.084421] cryptd: max_cpu_qlen set to 1000
[    0.087088] ACPI: Interpreter disabled.
[    0.089021] iommu: Default domain type: Translated 
[    0.089276] vgaarb: loaded
[    0.089611] SCSI subsystem initialized
[    0.090012] usbcore: registered new interface driver usbfs
[    0.090072] usbcore: registered new interface driver hub
[    0.090151] usbcore: registered new device driver usb
[    0.090330] usb_phy_generic phy: supply vcc not found, using dummy regulator
[    0.091142] pps_core: LinuxPPS API ver. 1 registered
[    0.091160] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.091196] PTP clock support registered
[    0.091357] EDAC MC: Ver: 3.0.0
[    0.092760] FPGA manager framework
[    0.092865] Advanced Linux Sound Architecture Driver Initialized.
[    0.093701] clocksource: Switched to clocksource arch_sys_counter
[    0.093934] VFS: Disk quotas dquot_6.6.0
[    0.094010] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.094227] pnp: PnP ACPI: disabled
[    0.100879] NET: Registered protocol family 2
[    0.101392] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.101450] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.101659] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.102140] TCP: Hash tables configured (established 32768 bind 32768)
[    0.102352] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102422] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102628] NET: Registered protocol family 1
[    0.103133] RPC: Registered named UNIX socket transport module.
[    0.103152] RPC: Registered udp transport module.
[    0.103165] RPC: Registered tcp transport module.
[    0.103178] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.103199] PCI: CLS 0 bytes, default 64
[    0.104385] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[    0.104540] kvm [1]: IPA Size Limit: 44bits
[    0.105680] kvm [1]: vgic interrupt IRQ1
[    0.105867] kvm [1]: Hyp mode initialized successfully
[    0.107728] Initialise system trusted keyrings
[    0.107883] workingset: timestamp_bits=44 max_order=20 bucket_order=0
[    0.114719] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.115521] NFS: Registering the id_resolver key type
[    0.115561] Key type id_resolver registered
[    0.115575] Key type id_legacy registered
[    0.115680] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.115865] 9p: Installing v9fs 9p2000 file system support
[    0.178003] Key type asymmetric registered
[    0.178022] Asymmetric key parser 'x509' registered
[    0.178076] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.178095] io scheduler mq-deadline registered
[    0.178108] io scheduler kyber registered
[    0.190559] EINJ: ACPI disabled.
[    0.208669] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.210741] printk: console [ttyS1] disabled
[    0.210844] fe215040.serial: ttyS1 at MMIO 0xfe215040 (irq = 18, base_baud = 24999999) is a 16550
[    1.116050] printk: console [ttyS1] enabled
[    1.121777] SuperH (H)SCI(F) driver initialized
[    1.127007] msm_serial: driver initialized
[    1.132787] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    1.147454] loop: module loaded
[    1.151881] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    1.160186] megasas: 07.714.04.00-rc1
[    1.168988] libphy: Fixed MDIO Bus: probed
[    1.174593] tun: Universal TUN/TAP device driver, 1.6
[    1.180565] thunder_xcv, ver 1.0
[    1.183908] thunder_bgx, ver 1.0
[    1.187241] nicpf, ver 1.0
[    1.191216] hclge is initializing
[    1.194649] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    1.202014] hns3: Copyright (c) 2017 Huawei Corporation.
[    1.207482] e1000: Intel(R) PRO/1000 Network Driver
[    1.212455] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    1.218347] e1000e: Intel(R) PRO/1000 Network Driver
[    1.223407] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.229481] igb: Intel(R) Gigabit Ethernet Network Driver
[    1.234987] igb: Copyright (c) 2007-2014 Intel Corporation.
[    1.240693] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    1.247076] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    1.253477] sky2: driver version 1.30
[    1.258420] VFIO - User Level meta-driver version: 0.3
[    1.265684] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.272366] ehci-pci: EHCI PCI platform driver
[    1.276933] ehci-platform: EHCI generic platform driver
[    1.282396] ehci-orion: EHCI orion driver
[    1.286594] ehci-exynos: EHCI Exynos driver
[    1.290961] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.297274] ohci-pci: OHCI PCI platform driver
[    1.301842] ohci-platform: OHCI generic platform driver
[    1.307292] ohci-exynos: OHCI Exynos driver
[    1.312095] usbcore: registered new interface driver usb-storage
[    1.321120] i2c /dev entries driver
[    1.330063] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    1.338533] sdhci: Secure Digital Host Controller Interface driver
[    1.344845] sdhci: Copyright(c) Pierre Ossman
[    1.349891] Synopsys Designware Multimedia Card Interface Driver
[    1.357122] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.375006] ledtrig-cpu: registered to indicate activity on CPUs
[    1.382894] usbcore: registered new interface driver usbhid
[    1.388580] usbhid: USB HID core driver
[    1.393564] bcm2835-mbox fe00b880.mailbox: mailbox enabled
[    1.403535] NET: Registered protocol family 17
[    1.408294] 9pnet: Installing 9P2000 support
[    1.412713] Key type dns_resolver registered
[    1.417430] registered taskstats version 1
[    1.421626] Loading compiled-in X.509 certificates
[    1.431456] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 16, base_baud = 0) is a PL011 rev2
[    1.440702] serial serial0: tty port ttyAMA0 registered
[    1.462488] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.503490] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.510942] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.619058] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.631093] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.656482] mmc0: Got data interrupt 0x00100000 even though no data operation was in progress.
[    1.665246] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[    1.671799] mmc0: sdhci: Sys addr:  0x00000000 | Version:  0x00009902
[    1.678349] mmc0: sdhci: Blk size:  0x00000000 | Blk cnt:  0x00000000
[    1.684899] mmc0: sdhci: Argument:  0x00000000 | Trn mode: 0x00000000
[    1.691448] mmc0: sdhci: Present:   0x01ff0000 | Host ctl: 0x00000000
[    1.697998] mmc0: sdhci: Power:     0x0000000f | Blk gap:  0x00000000
[    1.704548] mmc0: sdhci: Wake-up:   0x00000000 | Clock:    0x00003943
[    1.711097] mmc0: sdhci: Timeout:   0x00000000 | Int stat: 0x00000000
[    1.717646] mmc0: sdhci: Int enab:  0x00ff0003 | Sig enab: 0x00ff0003
[    1.724195] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000000
[    1.730745] mmc0: sdhci: Caps:      0x00000000 | Caps_1:   0x00000000
[    1.737293] mmc0: sdhci: Cmd:       0x00000000 | Max curr: 0x00000001
[    1.743842] mmc0: sdhci: Resp[0]:   0x00000000 | Resp[1]:  0x00000000
[    1.750390] mmc0: sdhci: Resp[2]:   0x00000000 | Resp[3]:  0x00000000
[    1.756939] mmc0: sdhci: Host ctl2: 0x00000000
[    1.761456] mmc0: sdhci: ============================================
[    1.780655] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.805771] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.813060] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.820345] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.828965] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.836403] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.845718] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.855465] ALSA device list:
[    1.858526]   No soundcards found.
[    1.862198] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.902568] mmc1: unrecognised SCR structure version 4
[    1.907898] mmc1: error -22 whilst initialising SD card
[    1.913254] random: fast init done
[    1.929590] mmc0: new high speed SDIO card at address 0001
[    1.992339] mmc1: unrecognised SCR structure version 2
[    1.997586] mmc1: error -22 whilst initialising SD card
[    2.073236] mmc1: unrecognised SCR structure version 4
[    2.078483] mmc1: error -22 whilst initialising SD card
[    2.170487] mmc1: unrecognised SCR structure version 4
[    2.175734] mmc1: error -22 whilst initialising SD card
[    3.257662] mmc1: unrecognised SCR structure version 4
[    3.262908] mmc1: error -22 whilst initialising SD card
[    3.333247] mmc1: unrecognised SCR structure version 4
[    3.338491] mmc1: error -22 whilst initialising SD card
[    3.414071] mmc1: unrecognised SCR structure version 4
[    3.419316] mmc1: error -22 whilst initialising SD card
[    3.511237] mmc1: unrecognised SCR structure version 4
[    3.516480] mmc1: error -22 whilst initialising SD card
[    4.601638] mmc1: unrecognised SCR structure version 4
[    4.606884] mmc1: error -22 whilst initialising SD card
[    4.677201] mmc1: unrecognised SCR structure version 4
[    4.682446] mmc1: error -22 whilst initialising SD card
[    4.758107] mmc1: unrecognised SCR structure version 4
[    4.763350] mmc1: error -22 whilst initialising SD card
[    4.855243] mmc1: unrecognised SCR structure version 4
[    4.860490] mmc1: error -22 whilst initialising SD card
[    5.945660] mmc1: unrecognised SCR structure version 4
[    5.950907] mmc1: error -22 whilst initialising SD card
[    6.021221] mmc1: unrecognised SCR structure version 4
[    6.026465] mmc1: error -22 whilst initialising SD card
[    6.102080] mmc1: unrecognised SCR structure version 4
[    6.107324] mmc1: error -22 whilst initialising SD card
[    6.199241] mmc1: unrecognised SCR structure version 4
[    6.204484] mmc1: error -22 whilst initialising SD card
[    7.289614] mmc1: unrecognised SCR structure version 4
[    7.294860] mmc1: error -22 whilst initialising SD card
[    7.365166] mmc1: unrecognised SCR structure version 4
[    7.370409] mmc1: error -22 whilst initialising SD card
[    7.446022] mmc1: unrecognised SCR structure version 4
[    7.451265] mmc1: error -22 whilst initialising SD card
[    7.543170] mmc1: unrecognised SCR structure version 4
[    7.548413] mmc1: error -22 whilst initialising SD card
[    8.636441] mmc1: unrecognised SCR structure version 4
[    8.641697] mmc1: error -22 whilst initialising SD card
[    8.711995] mmc1: unrecognised SCR structure version 4
[    8.717238] mmc1: error -22 whilst initialising SD card
[    8.792928] mmc1: unrecognised SCR structure version 4
[    8.798171] mmc1: error -22 whilst initialising SD card
[    8.890158] mmc1: unrecognised SCR structure version 4
[    8.895402] mmc1: error -22 whilst initialising SD card
[    9.977661] mmc1: unrecognised SCR structure version 4
[    9.982907] mmc1: error -22 whilst initialising SD card
[   10.053223] mmc1: unrecognised SCR structure version 4
[   10.058467] mmc1: error -22 whilst initialising SD card
[   10.134126] mmc1: unrecognised SCR structure version 4
[   10.139370] mmc1: error -22 whilst initialising SD card
[   10.231286] mmc1: unrecognised SCR structure version 4
[   10.236528] mmc1: error -22 whilst initialising SD card
[   11.321615] mmc1: unrecognised SCR structure version 4
[   11.326861] mmc1: error -22 whilst initialising SD card
[   11.397149] mmc1: unrecognised SCR structure version 4
[   11.402394] mmc1: error -22 whilst initialising SD card
[   11.478010] mmc1: unrecognised SCR structure version 4
[   11.483252] mmc1: error -22 whilst initialising SD card
[   11.575174] mmc1: unrecognised SCR structure version 4
[   11.580416] mmc1: error -22 whilst initialising SD card

[-- Attachment #4: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

[-- Attachment #1: Type: text/plain, Size: 1823 bytes --]

On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> 
> 
> On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> [snip]
> > > Hello Nathan,
> > > 
> > > Can you tell me how much memory your RPI has and if all of it is
> > 
> > This is the 4GB version.
> > 
> > > accessible by the PCIe device?  Could you also please include the DTS
> > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > PCIe DT before Linux boots -- could you describe what is going on
> > > there?
> > 
> > Unfortunately, I am not familiar with how to get this information. If
> > you could provide some instructions for how to do so, I am more than
> > happy to. I am not very knowleagable about the inner working of the Pi,
> > I mainly use it as a test platform for making sure that LLVM does not
> > cause problems on real devices.
> 
> Can you bring the dtc application to your Pi root filesystem, and if so, can
> you run the following:
> 
> dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb

Sure, the result is attached.

> or cat /sys/firmware/fdt > device.dtb
> 
> and attach the resulting file?
> 
> > 
> > > Finally, can you attach the text of the full boot log?
> > 
> > I have attached a working and broken boot log. Thank you for the quick
> > response!
> 
> Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> chance?

Of course. A new log is attached with the debug output from that config.

> I have a suspicion that this part of the DTS for the bcm2711.dtsi platform
> is at fault:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/bcm2711.dtsi#n264
> 
> and the resulting dma-ranges parsing is just not working for reasons to be
> determined.
> --
> Florian

Let me know if you need anything else out of me.

Cheers,
Nathan

[-- Attachment #2: device.dtb --]
[-- Type: application/octet-stream, Size: 24746 bytes --]

[-- Attachment #3: broken-pi-boot-debug.txt --]
[-- Type: text/plain, Size: 19118 bytes --]

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]
[    0.000000] Linux version 5.9.0-rc3-next-20200902-dirty (nathan@ubuntu-n2-xlarge-x86) (ClangBuiltLinux clang version 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb), LLD 12.0.0 (https://github.com/llvm/llvm-project.git b21ddded8f04fee925bbf9e6458347104b5b99eb)) #1 SMP PREEMPT Wed Sep 2 17:41:42 MST 2020
[    0.000000] Machine model: Raspberry Pi 4 Model B Rev 1.2
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000037400000, size 64 MiB
[    0.000000] OF: reserved mem: initialized node linux,cma, compatible id shared-dma-pool
[    0.000000] NUMA: No NUMA configuration found
[    0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] NUMA: NODE_DATA [mem 0xfb81f100-0xfb820fff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000000000-0x000000003fffffff]
[    0.000000]   DMA32    [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000003b3fffff]
[    0.000000]   node   0: [mem 0x0000000040000000-0x00000000fbffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x00000000fbffffff]
[    0.000000] percpu: Embedded 23 pages/cpu s54168 r8192 d31848 u94208
[    0.000000] Detected PIPT I-cache on CPU0
[    0.000000] CPU features: detected: EL2 vector hardening
[    0.000000] CPU features: kernel page table isolation forced ON by KASLR
[    0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[    0.000000] ARM_SMCCC_ARCH_WORKAROUND_1 missing from firmware
[    0.000000] CPU features: detected: ARM errata 1165522, 1319367, or 1530923
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 996912
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line:  dma.dmachans=0x71f5 bcm2709.boardrev=0xc03112 bcm2709.serial=0xb78d398 bcm2709.uart_clock=48000000 bcm2709.disk_led_gpio=42 bcm2709.disk_led_active_low=0 smsc95xx.macaddr=DC:A6:32:60:6C:87 vc_mem.mem_base=0x3ec00000 vc_mem.mem_size=0x40000000  console=ttyS1,115200 console=tty1 root=PARTUUID=45a8dd8a-02 rootfstype=ext4 elevator=deadline fsck.repair=yes rootwait plymouth.ignore-serial-consoles
[    0.000000] Kernel parameter elevator= does not have any effect anymore.
[    0.000000] Please use sysfs to set IO scheduler for individual devices.
[    0.000000] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes, linear)
[    0.000000] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] software IO TLB: mapped [mem 0x33400000-0x37400000] (64MB)
[    0.000000] Memory: 3812260K/4050944K available (14524K kernel code, 2636K rwdata, 7540K rodata, 1600K init, 615K bss, 173148K reserved, 65536K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] rcu: Preemptible hierarchical RCU implementation.
[    0.000000] rcu: 	RCU event tracing is enabled.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
[    0.000000] 	Trampoline variant of Tasks RCU enabled.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] GIC: Using split EOI/Deactivate mode
[    0.000000] random: get_random_bytes called from start_kernel+0x1cc/0x3f4 with crng_init=0
[    0.000000] arch_timer: cp15 timer(s) running at 54.00MHz (phys).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xc743ce346, max_idle_ns: 440795203123 ns
[    0.000006] sched_clock: 56 bits at 54MHz, resolution 18ns, wraps every 4398046511102ns
[    0.000195] Console: colour dummy device 80x25
[    0.000652] printk: console [tty1] enabled
[    0.000751] Calibrating delay loop (skipped), value calculated using timer frequency.. 108.00 BogoMIPS (lpj=216000)
[    0.000781] pid_max: default: 32768 minimum: 301
[    0.000866] LSM: Security Framework initializing
[    0.000997] Mount-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.001081] Mountpoint-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.002840] rcu: Hierarchical SRCU implementation.
[    0.005054] EFI services will not be available.
[    0.005345] smp: Bringing up secondary CPUs ...
[    0.005851] Detected PIPT I-cache on CPU1
[    0.005911] CPU1: Booted secondary processor 0x0000000001 [0x410fd083]
[    0.006455] Detected PIPT I-cache on CPU2
[    0.006495] CPU2: Booted secondary processor 0x0000000002 [0x410fd083]
[    0.007007] Detected PIPT I-cache on CPU3
[    0.007046] CPU3: Booted secondary processor 0x0000000003 [0x410fd083]
[    0.007129] smp: Brought up 1 node, 4 CPUs
[    0.007193] SMP: Total of 4 processors activated.
[    0.007209] CPU features: detected: 32-bit EL0 Support
[    0.007224] CPU features: detected: CRC32 instructions
[    0.007240] CPU features: detected: 32-bit EL1 Support
[    0.030540] CPU: All CPU(s) started at EL2
[    0.030599] alternatives: patching kernel code
[    0.032564] devtmpfs: initialized
[    0.037682] KASLR enabled
[    0.038172] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.038206] futex hash table entries: 1024 (order: 4, 65536 bytes, linear)
[    0.040116] pinctrl core: initialized pinctrl subsystem
[    0.041452] DMI not present or invalid.
[    0.042019] NET: Registered protocol family 16
[    0.043558] DMA: preallocated 512 KiB GFP_KERNEL pool for atomic allocations
[    0.043712] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations
[    0.043990] DMA: preallocated 512 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations
[    0.044050] audit: initializing netlink subsys (disabled)
[    0.044242] audit: type=2000 audit(0.040:1): state=initialized audit_enabled=0 res=1
[    0.045301] thermal_sys: Registered thermal governor 'step_wise'
[    0.045309] thermal_sys: Registered thermal governor 'power_allocator'
[    0.045648] cpuidle: using governor menu
[    0.045945] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[    0.046047] ASID allocator initialised with 32768 entries
[    0.047600] Serial: AMBA PL011 UART driver
[    0.083025] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.083061] HugeTLB registered 32.0 MiB page size, pre-allocated 0 pages
[    0.083078] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.083094] HugeTLB registered 64.0 KiB page size, pre-allocated 0 pages
[    0.084421] cryptd: max_cpu_qlen set to 1000
[    0.087088] ACPI: Interpreter disabled.
[    0.089021] iommu: Default domain type: Translated 
[    0.089276] vgaarb: loaded
[    0.089611] SCSI subsystem initialized
[    0.090012] usbcore: registered new interface driver usbfs
[    0.090072] usbcore: registered new interface driver hub
[    0.090151] usbcore: registered new device driver usb
[    0.090330] usb_phy_generic phy: supply vcc not found, using dummy regulator
[    0.091142] pps_core: LinuxPPS API ver. 1 registered
[    0.091160] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.091196] PTP clock support registered
[    0.091357] EDAC MC: Ver: 3.0.0
[    0.092760] FPGA manager framework
[    0.092865] Advanced Linux Sound Architecture Driver Initialized.
[    0.093701] clocksource: Switched to clocksource arch_sys_counter
[    0.093934] VFS: Disk quotas dquot_6.6.0
[    0.094010] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.094227] pnp: PnP ACPI: disabled
[    0.100879] NET: Registered protocol family 2
[    0.101392] tcp_listen_portaddr_hash hash table entries: 2048 (order: 3, 32768 bytes, linear)
[    0.101450] TCP established hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.101659] TCP bind hash table entries: 32768 (order: 7, 524288 bytes, linear)
[    0.102140] TCP: Hash tables configured (established 32768 bind 32768)
[    0.102352] UDP hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102422] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes, linear)
[    0.102628] NET: Registered protocol family 1
[    0.103133] RPC: Registered named UNIX socket transport module.
[    0.103152] RPC: Registered udp transport module.
[    0.103165] RPC: Registered tcp transport module.
[    0.103178] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.103199] PCI: CLS 0 bytes, default 64
[    0.104385] hw perfevents: enabled with armv8_cortex_a72 PMU driver, 7 counters available
[    0.104540] kvm [1]: IPA Size Limit: 44bits
[    0.105680] kvm [1]: vgic interrupt IRQ1
[    0.105867] kvm [1]: Hyp mode initialized successfully
[    0.107728] Initialise system trusted keyrings
[    0.107883] workingset: timestamp_bits=44 max_order=20 bucket_order=0
[    0.114719] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.115521] NFS: Registering the id_resolver key type
[    0.115561] Key type id_resolver registered
[    0.115575] Key type id_legacy registered
[    0.115680] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.115865] 9p: Installing v9fs 9p2000 file system support
[    0.178003] Key type asymmetric registered
[    0.178022] Asymmetric key parser 'x509' registered
[    0.178076] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 245)
[    0.178095] io scheduler mq-deadline registered
[    0.178108] io scheduler kyber registered
[    0.190559] EINJ: ACPI disabled.
[    0.208669] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.210741] printk: console [ttyS1] disabled
[    0.210844] fe215040.serial: ttyS1 at MMIO 0xfe215040 (irq = 18, base_baud = 24999999) is a 16550
[    1.116050] printk: console [ttyS1] enabled
[    1.121777] SuperH (H)SCI(F) driver initialized
[    1.127007] msm_serial: driver initialized
[    1.132787] cacheinfo: Unable to detect cache hierarchy for CPU 0
[    1.147454] loop: module loaded
[    1.151881] bcm2835-power bcm2835-power: Broadcom BCM2835 power domains driver
[    1.160186] megasas: 07.714.04.00-rc1
[    1.168988] libphy: Fixed MDIO Bus: probed
[    1.174593] tun: Universal TUN/TAP device driver, 1.6
[    1.180565] thunder_xcv, ver 1.0
[    1.183908] thunder_bgx, ver 1.0
[    1.187241] nicpf, ver 1.0
[    1.191216] hclge is initializing
[    1.194649] hns3: Hisilicon Ethernet Network Driver for Hip08 Family - version
[    1.202014] hns3: Copyright (c) 2017 Huawei Corporation.
[    1.207482] e1000: Intel(R) PRO/1000 Network Driver
[    1.212455] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    1.218347] e1000e: Intel(R) PRO/1000 Network Driver
[    1.223407] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    1.229481] igb: Intel(R) Gigabit Ethernet Network Driver
[    1.234987] igb: Copyright (c) 2007-2014 Intel Corporation.
[    1.240693] igbvf: Intel(R) Gigabit Virtual Function Network Driver
[    1.247076] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.
[    1.253477] sky2: driver version 1.30
[    1.258420] VFIO - User Level meta-driver version: 0.3
[    1.265684] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    1.272366] ehci-pci: EHCI PCI platform driver
[    1.276933] ehci-platform: EHCI generic platform driver
[    1.282396] ehci-orion: EHCI orion driver
[    1.286594] ehci-exynos: EHCI Exynos driver
[    1.290961] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    1.297274] ohci-pci: OHCI PCI platform driver
[    1.301842] ohci-platform: OHCI generic platform driver
[    1.307292] ohci-exynos: OHCI Exynos driver
[    1.312095] usbcore: registered new interface driver usb-storage
[    1.321120] i2c /dev entries driver
[    1.330063] bcm2835-wdt bcm2835-wdt: Broadcom BCM2835 watchdog timer
[    1.338533] sdhci: Secure Digital Host Controller Interface driver
[    1.344845] sdhci: Copyright(c) Pierre Ossman
[    1.349891] Synopsys Designware Multimedia Card Interface Driver
[    1.357122] sdhci-pltfm: SDHCI platform and OF driver helper
[    1.375006] ledtrig-cpu: registered to indicate activity on CPUs
[    1.382894] usbcore: registered new interface driver usbhid
[    1.388580] usbhid: USB HID core driver
[    1.393564] bcm2835-mbox fe00b880.mailbox: mailbox enabled
[    1.403535] NET: Registered protocol family 17
[    1.408294] 9pnet: Installing 9P2000 support
[    1.412713] Key type dns_resolver registered
[    1.417430] registered taskstats version 1
[    1.421626] Loading compiled-in X.509 certificates
[    1.431456] fe201000.serial: ttyAMA0 at MMIO 0xfe201000 (irq = 16, base_baud = 0) is a PL011 rev2
[    1.440702] serial serial0: tty port ttyAMA0 registered
[    1.462488] raspberrypi-firmware soc:firmware: Attached to firmware from 2020-08-24T18:50:56
[    1.503490] dwc2 fe980000.usb: supply vusb_d not found, using dummy regulator
[    1.510942] dwc2 fe980000.usb: supply vusb_a not found, using dummy regulator
[    1.619058] dwc2 fe980000.usb: EPs: 8, dedicated fifos, 4080 entries in SPRAM
[    1.631093] sdhci-iproc fe300000.sdhci: allocated mmc-pwrseq
[    1.656482] mmc0: Got data interrupt 0x00100000 even though no data operation was in progress.
[    1.665246] mmc0: sdhci: ============ SDHCI REGISTER DUMP ===========
[    1.671799] mmc0: sdhci: Sys addr:  0x00000000 | Version:  0x00009902
[    1.678349] mmc0: sdhci: Blk size:  0x00000000 | Blk cnt:  0x00000000
[    1.684899] mmc0: sdhci: Argument:  0x00000000 | Trn mode: 0x00000000
[    1.691448] mmc0: sdhci: Present:   0x01ff0000 | Host ctl: 0x00000000
[    1.697998] mmc0: sdhci: Power:     0x0000000f | Blk gap:  0x00000000
[    1.704548] mmc0: sdhci: Wake-up:   0x00000000 | Clock:    0x00003943
[    1.711097] mmc0: sdhci: Timeout:   0x00000000 | Int stat: 0x00000000
[    1.717646] mmc0: sdhci: Int enab:  0x00ff0003 | Sig enab: 0x00ff0003
[    1.724195] mmc0: sdhci: ACmd stat: 0x00000000 | Slot int: 0x00000000
[    1.730745] mmc0: sdhci: Caps:      0x00000000 | Caps_1:   0x00000000
[    1.737293] mmc0: sdhci: Cmd:       0x00000000 | Max curr: 0x00000001
[    1.743842] mmc0: sdhci: Resp[0]:   0x00000000 | Resp[1]:  0x00000000
[    1.750390] mmc0: sdhci: Resp[2]:   0x00000000 | Resp[3]:  0x00000000
[    1.756939] mmc0: sdhci: Host ctl2: 0x00000000
[    1.761456] mmc0: sdhci: ============================================
[    1.780655] mmc0: SDHCI controller on fe300000.sdhci [fe300000.sdhci] using PIO
[    1.805771] mmc0: queuing unknown CIS tuple 0x80 (2 bytes)
[    1.813060] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.820345] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.828965] mmc0: queuing unknown CIS tuple 0x80 (7 bytes)
[    1.836403] mmc0: queuing unknown CIS tuple 0x80 (3 bytes)
[    1.845718] mmc1: SDHCI controller on fe340000.emmc2 [fe340000.emmc2] using ADMA
[    1.855465] ALSA device list:
[    1.858526]   No soundcards found.
[    1.862198] Waiting for root device PARTUUID=45a8dd8a-02...
[    1.902568] mmc1: unrecognised SCR structure version 4
[    1.907898] mmc1: error -22 whilst initialising SD card
[    1.913254] random: fast init done
[    1.929590] mmc0: new high speed SDIO card at address 0001
[    1.992339] mmc1: unrecognised SCR structure version 2
[    1.997586] mmc1: error -22 whilst initialising SD card
[    2.073236] mmc1: unrecognised SCR structure version 4
[    2.078483] mmc1: error -22 whilst initialising SD card
[    2.170487] mmc1: unrecognised SCR structure version 4
[    2.175734] mmc1: error -22 whilst initialising SD card
[    3.257662] mmc1: unrecognised SCR structure version 4
[    3.262908] mmc1: error -22 whilst initialising SD card
[    3.333247] mmc1: unrecognised SCR structure version 4
[    3.338491] mmc1: error -22 whilst initialising SD card
[    3.414071] mmc1: unrecognised SCR structure version 4
[    3.419316] mmc1: error -22 whilst initialising SD card
[    3.511237] mmc1: unrecognised SCR structure version 4
[    3.516480] mmc1: error -22 whilst initialising SD card
[    4.601638] mmc1: unrecognised SCR structure version 4
[    4.606884] mmc1: error -22 whilst initialising SD card
[    4.677201] mmc1: unrecognised SCR structure version 4
[    4.682446] mmc1: error -22 whilst initialising SD card
[    4.758107] mmc1: unrecognised SCR structure version 4
[    4.763350] mmc1: error -22 whilst initialising SD card
[    4.855243] mmc1: unrecognised SCR structure version 4
[    4.860490] mmc1: error -22 whilst initialising SD card
[    5.945660] mmc1: unrecognised SCR structure version 4
[    5.950907] mmc1: error -22 whilst initialising SD card
[    6.021221] mmc1: unrecognised SCR structure version 4
[    6.026465] mmc1: error -22 whilst initialising SD card
[    6.102080] mmc1: unrecognised SCR structure version 4
[    6.107324] mmc1: error -22 whilst initialising SD card
[    6.199241] mmc1: unrecognised SCR structure version 4
[    6.204484] mmc1: error -22 whilst initialising SD card
[    7.289614] mmc1: unrecognised SCR structure version 4
[    7.294860] mmc1: error -22 whilst initialising SD card
[    7.365166] mmc1: unrecognised SCR structure version 4
[    7.370409] mmc1: error -22 whilst initialising SD card
[    7.446022] mmc1: unrecognised SCR structure version 4
[    7.451265] mmc1: error -22 whilst initialising SD card
[    7.543170] mmc1: unrecognised SCR structure version 4
[    7.548413] mmc1: error -22 whilst initialising SD card
[    8.636441] mmc1: unrecognised SCR structure version 4
[    8.641697] mmc1: error -22 whilst initialising SD card
[    8.711995] mmc1: unrecognised SCR structure version 4
[    8.717238] mmc1: error -22 whilst initialising SD card
[    8.792928] mmc1: unrecognised SCR structure version 4
[    8.798171] mmc1: error -22 whilst initialising SD card
[    8.890158] mmc1: unrecognised SCR structure version 4
[    8.895402] mmc1: error -22 whilst initialising SD card
[    9.977661] mmc1: unrecognised SCR structure version 4
[    9.982907] mmc1: error -22 whilst initialising SD card
[   10.053223] mmc1: unrecognised SCR structure version 4
[   10.058467] mmc1: error -22 whilst initialising SD card
[   10.134126] mmc1: unrecognised SCR structure version 4
[   10.139370] mmc1: error -22 whilst initialising SD card
[   10.231286] mmc1: unrecognised SCR structure version 4
[   10.236528] mmc1: error -22 whilst initialising SD card
[   11.321615] mmc1: unrecognised SCR structure version 4
[   11.326861] mmc1: error -22 whilst initialising SD card
[   11.397149] mmc1: unrecognised SCR structure version 4
[   11.402394] mmc1: error -22 whilst initialising SD card
[   11.478010] mmc1: unrecognised SCR structure version 4
[   11.483252] mmc1: error -22 whilst initialising SD card
[   11.575174] mmc1: unrecognised SCR structure version 4
[   11.580416] mmc1: error -22 whilst initialising SD card

[-- Attachment #4: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

FYI, I've moved it off for-next and into its own dma-ranges branch
for now until we sort out the regressions.  I still hope to bring it
back soon.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

FYI, I've moved it off for-next and into its own dma-ranges branch
for now until we sort out the regressions.  I still hope to bring it
back soon.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

On Wed, Sep 2, 2020 at 8:52 PM Nathan Chancellor
<natechancellor@gmail.com> wrote:
>
> On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> >
> >
> > On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> > [snip]
> > > > Hello Nathan,
> > > >
> > > > Can you tell me how much memory your RPI has and if all of it is
> > >
> > > This is the 4GB version.
> > >
> > > > accessible by the PCIe device?  Could you also please include the DTS
> > > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > > PCIe DT before Linux boots -- could you describe what is going on
> > > > there?
> > >
> > > Unfortunately, I am not familiar with how to get this information. If
> > > you could provide some instructions for how to do so, I am more than
> > > happy to. I am not very knowleagable about the inner working of the Pi,
> > > I mainly use it as a test platform for making sure that LLVM does not
> > > cause problems on real devices.
> >
> > Can you bring the dtc application to your Pi root filesystem, and if so, can
> > you run the following:
> >
> > dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb
>
> Sure, the result is attached.
>
> > or cat /sys/firmware/fdt > device.dtb
> >
> > and attach the resulting file?
> >
> > >
> > > > Finally, can you attach the text of the full boot log?
> > >
> > > I have attached a working and broken boot log. Thank you for the quick
> > > response!
> >
> > Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> > chance?
>
> Of course. A new log is attached with the debug output from that config.


Hi Nicolas,

Can you please help us out here?  It appears that your commit

3d2cbb644836 "ARM: dts: bcm2711: Move emmc2 into its own bus"

added the following line to the bcm2711.dtsi file:

+ dma-ranges = <0x0 0xc0000000  0x0 0x00000000  0x40000000>;

and for some reason the eMMC is not operating properly w/ my commit..
 Regardless, the only difference that my commit should make is to
enforce the bounds of the dma_window (whereas the previous code
adds/subs the pfn_offset everywhere).

Thanks,
Jim Quinlan

PS If you would like to talk, let me know and we can make arrangements.


>
>
> > I have a suspicion that this part of the DTS for the bcm2711.dtsi platform
> > is at fault:
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/bcm2711.dtsi#n264
> >
> > and the resulting dma-ranges parsing is just not working for reasons to be
> > determined.
> > --
> > Florian
>
> Let me know if you need anything else out of me.
>
> Cheers,
> Nathan
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan via iommu]

On Wed, Sep 2, 2020 at 8:52 PM Nathan Chancellor
<natechancellor@gmail.com> wrote:
>
> On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> >
> >
> > On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> > [snip]
> > > > Hello Nathan,
> > > >
> > > > Can you tell me how much memory your RPI has and if all of it is
> > >
> > > This is the 4GB version.
> > >
> > > > accessible by the PCIe device?  Could you also please include the DTS
> > > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > > PCIe DT before Linux boots -- could you describe what is going on
> > > > there?
> > >
> > > Unfortunately, I am not familiar with how to get this information. If
> > > you could provide some instructions for how to do so, I am more than
> > > happy to. I am not very knowleagable about the inner working of the Pi,
> > > I mainly use it as a test platform for making sure that LLVM does not
> > > cause problems on real devices.
> >
> > Can you bring the dtc application to your Pi root filesystem, and if so, can
> > you run the following:
> >
> > dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb
>
> Sure, the result is attached.
>
> > or cat /sys/firmware/fdt > device.dtb
> >
> > and attach the resulting file?
> >
> > >
> > > > Finally, can you attach the text of the full boot log?
> > >
> > > I have attached a working and broken boot log. Thank you for the quick
> > > response!
> >
> > Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> > chance?
>
> Of course. A new log is attached with the debug output from that config.


Hi Nicolas,

Can you please help us out here?  It appears that your commit

3d2cbb644836 "ARM: dts: bcm2711: Move emmc2 into its own bus"

added the following line to the bcm2711.dtsi file:

+ dma-ranges = <0x0 0xc0000000  0x0 0x00000000  0x40000000>;

and for some reason the eMMC is not operating properly w/ my commit..
 Regardless, the only difference that my commit should make is to
enforce the bounds of the dma_window (whereas the previous code
adds/subs the pfn_offset everywhere).

Thanks,
Jim Quinlan

PS If you would like to talk, let me know and we can make arrangements.


>
>
> > I have a suspicion that this part of the DTS for the bcm2711.dtsi platform
> > is at fault:
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/bcm2711.dtsi#n264
> >
> > and the resulting dma-ranges parsing is just not working for reasons to be
> > determined.
> > --
> > Florian
>
> Let me know if you need anything else out of me.
>
> Cheers,
> Nathan
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

On Wed, Sep 2, 2020 at 8:52 PM Nathan Chancellor
<natechancellor@gmail.com> wrote:
>
> On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> >
> >
> > On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> > [snip]
> > > > Hello Nathan,
> > > >
> > > > Can you tell me how much memory your RPI has and if all of it is
> > >
> > > This is the 4GB version.
> > >
> > > > accessible by the PCIe device?  Could you also please include the DTS
> > > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > > PCIe DT before Linux boots -- could you describe what is going on
> > > > there?
> > >
> > > Unfortunately, I am not familiar with how to get this information. If
> > > you could provide some instructions for how to do so, I am more than
> > > happy to. I am not very knowleagable about the inner working of the Pi,
> > > I mainly use it as a test platform for making sure that LLVM does not
> > > cause problems on real devices.
> >
> > Can you bring the dtc application to your Pi root filesystem, and if so, can
> > you run the following:
> >
> > dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb
>
> Sure, the result is attached.
>
> > or cat /sys/firmware/fdt > device.dtb
> >
> > and attach the resulting file?
> >
> > >
> > > > Finally, can you attach the text of the full boot log?
> > >
> > > I have attached a working and broken boot log. Thank you for the quick
> > > response!
> >
> > Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> > chance?
>
> Of course. A new log is attached with the debug output from that config.


Hi Nicolas,

Can you please help us out here?  It appears that your commit

3d2cbb644836 "ARM: dts: bcm2711: Move emmc2 into its own bus"

added the following line to the bcm2711.dtsi file:

+ dma-ranges = <0x0 0xc0000000  0x0 0x00000000  0x40000000>;

and for some reason the eMMC is not operating properly w/ my commit..
 Regardless, the only difference that my commit should make is to
enforce the bounds of the dma_window (whereas the previous code
adds/subs the pfn_offset everywhere).

Thanks,
Jim Quinlan

PS If you would like to talk, let me know and we can make arrangements.


>
>
> > I have a suspicion that this part of the DTS for the bcm2711.dtsi platform
> > is at fault:
> >
> > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/bcm2711.dtsi#n264
> >
> > and the resulting dma-ranges parsing is just not working for reasons to be
> > determined.
> > --
> > Florian
>
> Let me know if you need anything else out of me.
>
> Cheers,
> Nathan
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> >
> > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > Hi,
> > >
> > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > >>
> > >> Patchset Summary:
> > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > >
> > > We are version 11 and counting, and it is not clear to me whether there is
> > > any chance of getting these patches reviewed and hopefully merged for the
> > > 5.10 merge window.
> > >
> > > There are a lot of different files being touched, so what would be the
> > > ideal way of routing those changes towards inclusion?
> >
> > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > apply the current version.
> Sounds good to me.

Hi Jim,

is the dependency now solved ? Should we review/take this series as
is for v5.10 through the PCI tree ?

Thanks,
Lorenzo

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> >
> > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > Hi,
> > >
> > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > >>
> > >> Patchset Summary:
> > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > >
> > > We are version 11 and counting, and it is not clear to me whether there is
> > > any chance of getting these patches reviewed and hopefully merged for the
> > > 5.10 merge window.
> > >
> > > There are a lot of different files being touched, so what would be the
> > > ideal way of routing those changes towards inclusion?
> >
> > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > apply the current version.
> Sounds good to me.

Hi Jim,

is the dependency now solved ? Should we review/take this series as
is for v5.10 through the PCI tree ?

Thanks,
Lorenzo

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> >
> > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > Hi,
> > >
> > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > >>
> > >> Patchset Summary:
> > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > >
> > > We are version 11 and counting, and it is not clear to me whether there is
> > > any chance of getting these patches reviewed and hopefully merged for the
> > > 5.10 merge window.
> > >
> > > There are a lot of different files being touched, so what would be the
> > > ideal way of routing those changes towards inclusion?
> >
> > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > apply the current version.
> Sounds good to me.

Hi Jim,

is the dependency now solved ? Should we review/take this series as
is for v5.10 through the PCI tree ?

Thanks,
Lorenzo
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> >
> > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > Hi,
> > >
> > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > >>
> > >> Patchset Summary:
> > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > >
> > > We are version 11 and counting, and it is not clear to me whether there is
> > > any chance of getting these patches reviewed and hopefully merged for the
> > > 5.10 merge window.
> > >
> > > There are a lot of different files being touched, so what would be the
> > > ideal way of routing those changes towards inclusion?
> >
> > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > apply the current version.
> Sounds good to me.

Hi Jim,

is the dependency now solved ? Should we review/take this series as
is for v5.10 through the PCI tree ?

Thanks,
Lorenzo
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> >
> > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > Hi,
> > >
> > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > >>
> > >> Patchset Summary:
> > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > >
> > > We are version 11 and counting, and it is not clear to me whether there is
> > > any chance of getting these patches reviewed and hopefully merged for the
> > > 5.10 merge window.
> > >
> > > There are a lot of different files being touched, so what would be the
> > > ideal way of routing those changes towards inclusion?
> >
> > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > apply the current version.
> Sounds good to me.

Hi Jim,

is the dependency now solved ? Should we review/take this series as
is for v5.10 through the PCI tree ?

Thanks,
Lorenzo

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> >
> > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > Hi,
> > >
> > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > >>
> > >> Patchset Summary:
> > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > >
> > > We are version 11 and counting, and it is not clear to me whether there is
> > > any chance of getting these patches reviewed and hopefully merged for the
> > > 5.10 merge window.
> > >
> > > There are a lot of different files being touched, so what would be the
> > > ideal way of routing those changes towards inclusion?
> >
> > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > apply the current version.
> Sounds good to me.

Hi Jim,

is the dependency now solved ? Should we review/take this series as
is for v5.10 through the PCI tree ?

Thanks,
Lorenzo
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 3806 bytes --]

Hi Jim, sorry I'm a little late to the party, but was on vacation.

On Thu, 2020-09-03 at 13:32 -0400, Jim Quinlan wrote:
> On Wed, Sep 2, 2020 at 8:52 PM Nathan Chancellor
> <natechancellor@gmail.com> wrote:
> > On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> > > 
> > > On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> > > [snip]
> > > > > Hello Nathan,
> > > > > 
> > > > > Can you tell me how much memory your RPI has and if all of it is
> > > > 
> > > > This is the 4GB version.
> > > > 
> > > > > accessible by the PCIe device?  Could you also please include the DTS
> > > > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > > > PCIe DT before Linux boots -- could you describe what is going on
> > > > > there?
> > > > 
> > > > Unfortunately, I am not familiar with how to get this information. If
> > > > you could provide some instructions for how to do so, I am more than
> > > > happy to. I am not very knowleagable about the inner working of the Pi,
> > > > I mainly use it as a test platform for making sure that LLVM does not
> > > > cause problems on real devices.
> > > 
> > > Can you bring the dtc application to your Pi root filesystem, and if so, can
> > > you run the following:
> > > 
> > > dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb
> > 
> > Sure, the result is attached.
> > 
> > > or cat /sys/firmware/fdt > device.dtb
> > > 
> > > and attach the resulting file?
> > > 
> > > > > Finally, can you attach the text of the full boot log?
> > > > 
> > > > I have attached a working and broken boot log. Thank you for the quick
> > > > response!
> > > 
> > > Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> > > chance?
> > 
> > Of course. A new log is attached with the debug output from that config.
> 
> Hi Nicolas,
> 
> Can you please help us out here?  It appears that your commit

It's dma_offset_from_dma_addr() that's causing trouble. It goes over all the
dma regions and, if no region matches the phys address range, it returns 0.
This isn't acceptable as is. In the past, we used to pass the offset
nonetheless, which would make the phys address grow out of the dma memory area
and fail the dma_capable() test.

For example, RPi4 devices attached to the old interconnect see phys [0x0
0x3fffffff] at [0xc0000000 0xffffffff]. So say you're trying to convert
physical address 0xfa000000, you'll get 0 from dma_offset_from_phys_addr()
(since your dma range only covers the first GB) to then test if 0xfa000000 is
dma_capable(), which it is, but for the wrong reasons. Causing DMA issues
further down the line.

I don't have a proper suggestion on how to solve this but here's the solution I
used:

diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4c4646761afe..40fe3c97f2be 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -217,11 +217,19 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
 {
        const struct bus_dma_region *m = dev->dma_range_map;
 
-       if (m)
+       if (m) {
                for (; m->size; m++)
                        if (paddr >= m->cpu_start &&
                            paddr - m->cpu_start < m->size)
                                return m->offset;
+
+               /*
+                * The physical address doesn't fit any of the DMA regions,
+                * return an impossible to fulfill offset.
+                */
+               return -(1ULL << 46);
+       }
+
        return 0;
 }

I didn't catch this on my previous tests as I was using a newer bcm2711 SoC
revision which expanded emmc2's DMA capabilities (can do 32 bit DMA as opposed
to 30 bit).

Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 3806 bytes --]

Hi Jim, sorry I'm a little late to the party, but was on vacation.

On Thu, 2020-09-03 at 13:32 -0400, Jim Quinlan wrote:
> On Wed, Sep 2, 2020 at 8:52 PM Nathan Chancellor
> <natechancellor@gmail.com> wrote:
> > On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> > > 
> > > On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> > > [snip]
> > > > > Hello Nathan,
> > > > > 
> > > > > Can you tell me how much memory your RPI has and if all of it is
> > > > 
> > > > This is the 4GB version.
> > > > 
> > > > > accessible by the PCIe device?  Could you also please include the DTS
> > > > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > > > PCIe DT before Linux boots -- could you describe what is going on
> > > > > there?
> > > > 
> > > > Unfortunately, I am not familiar with how to get this information. If
> > > > you could provide some instructions for how to do so, I am more than
> > > > happy to. I am not very knowleagable about the inner working of the Pi,
> > > > I mainly use it as a test platform for making sure that LLVM does not
> > > > cause problems on real devices.
> > > 
> > > Can you bring the dtc application to your Pi root filesystem, and if so, can
> > > you run the following:
> > > 
> > > dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb
> > 
> > Sure, the result is attached.
> > 
> > > or cat /sys/firmware/fdt > device.dtb
> > > 
> > > and attach the resulting file?
> > > 
> > > > > Finally, can you attach the text of the full boot log?
> > > > 
> > > > I have attached a working and broken boot log. Thank you for the quick
> > > > response!
> > > 
> > > Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> > > chance?
> > 
> > Of course. A new log is attached with the debug output from that config.
> 
> Hi Nicolas,
> 
> Can you please help us out here?  It appears that your commit

It's dma_offset_from_dma_addr() that's causing trouble. It goes over all the
dma regions and, if no region matches the phys address range, it returns 0.
This isn't acceptable as is. In the past, we used to pass the offset
nonetheless, which would make the phys address grow out of the dma memory area
and fail the dma_capable() test.

For example, RPi4 devices attached to the old interconnect see phys [0x0
0x3fffffff] at [0xc0000000 0xffffffff]. So say you're trying to convert
physical address 0xfa000000, you'll get 0 from dma_offset_from_phys_addr()
(since your dma range only covers the first GB) to then test if 0xfa000000 is
dma_capable(), which it is, but for the wrong reasons. Causing DMA issues
further down the line.

I don't have a proper suggestion on how to solve this but here's the solution I
used:

diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4c4646761afe..40fe3c97f2be 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -217,11 +217,19 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
 {
        const struct bus_dma_region *m = dev->dma_range_map;
 
-       if (m)
+       if (m) {
                for (; m->size; m++)
                        if (paddr >= m->cpu_start &&
                            paddr - m->cpu_start < m->size)
                                return m->offset;
+
+               /*
+                * The physical address doesn't fit any of the DMA regions,
+                * return an impossible to fulfill offset.
+                */
+               return -(1ULL << 46);
+       }
+
        return 0;
 }

I didn't catch this on my previous tests as I was using a newer bcm2711 SoC
revision which expanded emmc2's DMA capabilities (can do 32 bit DMA as opposed
to 30 bit).

Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 3806 bytes --]

Hi Jim, sorry I'm a little late to the party, but was on vacation.

On Thu, 2020-09-03 at 13:32 -0400, Jim Quinlan wrote:
> On Wed, Sep 2, 2020 at 8:52 PM Nathan Chancellor
> <natechancellor@gmail.com> wrote:
> > On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> > > 
> > > On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> > > [snip]
> > > > > Hello Nathan,
> > > > > 
> > > > > Can you tell me how much memory your RPI has and if all of it is
> > > > 
> > > > This is the 4GB version.
> > > > 
> > > > > accessible by the PCIe device?  Could you also please include the DTS
> > > > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > > > PCIe DT before Linux boots -- could you describe what is going on
> > > > > there?
> > > > 
> > > > Unfortunately, I am not familiar with how to get this information. If
> > > > you could provide some instructions for how to do so, I am more than
> > > > happy to. I am not very knowleagable about the inner working of the Pi,
> > > > I mainly use it as a test platform for making sure that LLVM does not
> > > > cause problems on real devices.
> > > 
> > > Can you bring the dtc application to your Pi root filesystem, and if so, can
> > > you run the following:
> > > 
> > > dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb
> > 
> > Sure, the result is attached.
> > 
> > > or cat /sys/firmware/fdt > device.dtb
> > > 
> > > and attach the resulting file?
> > > 
> > > > > Finally, can you attach the text of the full boot log?
> > > > 
> > > > I have attached a working and broken boot log. Thank you for the quick
> > > > response!
> > > 
> > > Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> > > chance?
> > 
> > Of course. A new log is attached with the debug output from that config.
> 
> Hi Nicolas,
> 
> Can you please help us out here?  It appears that your commit

It's dma_offset_from_dma_addr() that's causing trouble. It goes over all the
dma regions and, if no region matches the phys address range, it returns 0.
This isn't acceptable as is. In the past, we used to pass the offset
nonetheless, which would make the phys address grow out of the dma memory area
and fail the dma_capable() test.

For example, RPi4 devices attached to the old interconnect see phys [0x0
0x3fffffff] at [0xc0000000 0xffffffff]. So say you're trying to convert
physical address 0xfa000000, you'll get 0 from dma_offset_from_phys_addr()
(since your dma range only covers the first GB) to then test if 0xfa000000 is
dma_capable(), which it is, but for the wrong reasons. Causing DMA issues
further down the line.

I don't have a proper suggestion on how to solve this but here's the solution I
used:

diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4c4646761afe..40fe3c97f2be 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -217,11 +217,19 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
 {
        const struct bus_dma_region *m = dev->dma_range_map;
 
-       if (m)
+       if (m) {
                for (; m->size; m++)
                        if (paddr >= m->cpu_start &&
                            paddr - m->cpu_start < m->size)
                                return m->offset;
+
+               /*
+                * The physical address doesn't fit any of the DMA regions,
+                * return an impossible to fulfill offset.
+                */
+               return -(1ULL << 46);
+       }
+
        return 0;
 }

I didn't catch this on my previous tests as I was using a newer bcm2711 SoC
revision which expanded emmc2's DMA capabilities (can do 32 bit DMA as opposed
to 30 bit).

Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 1118 bytes --]

Hi Christoph, a small fix to your fixes:

On Tue, 2020-09-01 at 10:24 +0200, Christoph Hellwig wrote:
> I've applied this to the dma-mapping tree.
> 
> I had to resolve a conflict in drivers/of/address.c with a recent
> mainline commit.  I also applied the minor tweaks Andy pointed out
> plus a few more style changes.  A real change is that I changed the
> prototype for dma_copy_dma_range_map to require less boilerplate code.

diff --git a/drivers/usb/core/usb.c b/drivers/usb/core/usb.c
index 0b5f8d62f251..4cd012817af6 100644
--- a/drivers/usb/core/usb.c
+++ b/drivers/usb/core/usb.c
@@ -610,7 +610,7 @@ struct usb_device *usb_alloc_dev(struct usb_device *parent,
         * mask for the entire HCD, so don't do that.
         */
        dev->dev.dma_mask = bus->sysdev->dma_mask;
-       if (!dma_copy_dma_range_map(&dev->dev, bus->sysdev))
+       if (dma_copy_dma_range_map(&dev->dev, bus->sysdev))
                dev_err(&dev->dev, "failed to copy DMA map\n");
        set_dev_node(&dev->dev, dev_to_node(bus->sysdev));
        dev->state = USB_STATE_ATTACHED;

Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 1118 bytes --]

Hi Christoph, a small fix to your fixes:

On Tue, 2020-09-01 at 10:24 +0200, Christoph Hellwig wrote:
> I've applied this to the dma-mapping tree.
> 
> I had to resolve a conflict in drivers/of/address.c with a recent
> mainline commit.  I also applied the minor tweaks Andy pointed out
> plus a few more style changes.  A real change is that I changed the
> prototype for dma_copy_dma_range_map to require less boilerplate code.

diff --git a/drivers/usb/core/usb.c b/drivers/usb/core/usb.c
index 0b5f8d62f251..4cd012817af6 100644
--- a/drivers/usb/core/usb.c
+++ b/drivers/usb/core/usb.c
@@ -610,7 +610,7 @@ struct usb_device *usb_alloc_dev(struct usb_device *parent,
         * mask for the entire HCD, so don't do that.
         */
        dev->dev.dma_mask = bus->sysdev->dma_mask;
-       if (!dma_copy_dma_range_map(&dev->dev, bus->sysdev))
+       if (dma_copy_dma_range_map(&dev->dev, bus->sysdev))
                dev_err(&dev->dev, "failed to copy DMA map\n");
        set_dev_node(&dev->dev, dev_to_node(bus->sysdev));
        dev->state = USB_STATE_ATTACHED;

Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 1118 bytes --]

Hi Christoph, a small fix to your fixes:

On Tue, 2020-09-01 at 10:24 +0200, Christoph Hellwig wrote:
> I've applied this to the dma-mapping tree.
> 
> I had to resolve a conflict in drivers/of/address.c with a recent
> mainline commit.  I also applied the minor tweaks Andy pointed out
> plus a few more style changes.  A real change is that I changed the
> prototype for dma_copy_dma_range_map to require less boilerplate code.

diff --git a/drivers/usb/core/usb.c b/drivers/usb/core/usb.c
index 0b5f8d62f251..4cd012817af6 100644
--- a/drivers/usb/core/usb.c
+++ b/drivers/usb/core/usb.c
@@ -610,7 +610,7 @@ struct usb_device *usb_alloc_dev(struct usb_device *parent,
         * mask for the entire HCD, so don't do that.
         */
        dev->dev.dma_mask = bus->sysdev->dma_mask;
-       if (!dma_copy_dma_range_map(&dev->dev, bus->sysdev))
+       if (dma_copy_dma_range_map(&dev->dev, bus->sysdev))
                dev_err(&dev->dev, "failed to copy DMA map\n");
        set_dev_node(&dev->dev, dev_to_node(bus->sysdev));
        dev->state = USB_STATE_ATTACHED;

Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

On Mon, Sep 7, 2020 at 11:01 AM Nicolas Saenz Julienne
<nsaenzjulienne@suse.de> wrote:
>
> Hi Jim, sorry I'm a little late to the party, but was on vacation.
>
> On Thu, 2020-09-03 at 13:32 -0400, Jim Quinlan wrote:
> > On Wed, Sep 2, 2020 at 8:52 PM Nathan Chancellor
> > <natechancellor@gmail.com> wrote:
> > > On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> > > >
> > > > On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> > > > [snip]
> > > > > > Hello Nathan,
> > > > > >
> > > > > > Can you tell me how much memory your RPI has and if all of it is
> > > > >
> > > > > This is the 4GB version.
> > > > >
> > > > > > accessible by the PCIe device?  Could you also please include the DTS
> > > > > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > > > > PCIe DT before Linux boots -- could you describe what is going on
> > > > > > there?
> > > > >
> > > > > Unfortunately, I am not familiar with how to get this information. If
> > > > > you could provide some instructions for how to do so, I am more than
> > > > > happy to. I am not very knowleagable about the inner working of the Pi,
> > > > > I mainly use it as a test platform for making sure that LLVM does not
> > > > > cause problems on real devices.
> > > >
> > > > Can you bring the dtc application to your Pi root filesystem, and if so, can
> > > > you run the following:
> > > >
> > > > dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb
> > >
> > > Sure, the result is attached.
> > >
> > > > or cat /sys/firmware/fdt > device.dtb
> > > >
> > > > and attach the resulting file?
> > > >
> > > > > > Finally, can you attach the text of the full boot log?
> > > > >
> > > > > I have attached a working and broken boot log. Thank you for the quick
> > > > > response!
> > > >
> > > > Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> > > > chance?
> > >
> > > Of course. A new log is attached with the debug output from that config.
> >
> > Hi Nicolas,
> >
> > Can you please help us out here?  It appears that your commit
>
> It's dma_offset_from_dma_addr() that's causing trouble. It goes over all the
> dma regions and, if no region matches the phys address range, it returns 0.
> This isn't acceptable as is. In the past, we used to pass the offset
> nonetheless, which would make the phys address grow out of the dma memory area
> and fail the dma_capable() test.
>
> For example, RPi4 devices attached to the old interconnect see phys [0x0
> 0x3fffffff] at [0xc0000000 0xffffffff]. So say you're trying to convert
> physical address 0xfa000000, you'll get 0 from dma_offset_from_phys_addr()
> (since your dma range only covers the first GB) to then test if 0xfa000000 is
> dma_capable(), which it is, but for the wrong reasons. Causing DMA issues
> further down the line.
>
> I don't have a proper suggestion on how to solve this but here's the solution I
> used:
>
> diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> index 4c4646761afe..40fe3c97f2be 100644
> --- a/include/linux/dma-mapping.h
> +++ b/include/linux/dma-mapping.h
> @@ -217,11 +217,19 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
>  {
>         const struct bus_dma_region *m = dev->dma_range_map;
>
> -       if (m)
> +       if (m) {
>                 for (; m->size; m++)
>                         if (paddr >= m->cpu_start &&
>                             paddr - m->cpu_start < m->size)
>                                 return m->offset;
> +
> +               /*
> +                * The physical address doesn't fit any of the DMA regions,
> +                * return an impossible to fulfill offset.
> +                */
> +               return -(1ULL << 46);
> +       }
> +
>         return 0;
>  }
Hi Nicolas,

Thanks for looking into this.  The concern I have with your solution
is that returning an arbitrarily large offset might overlap with an
improbable but valid usage.  AFAIK there is nothing that disallows
mapping a device to anywhere within the 64bit range of PCIe space,
including up to say 0xffffffffffffffff.

As an alternative perhaps changing dma_capable() so that if the
dma_range_map is present then it validates that both ends of the
prospective DMA region get "hits" on one of the offset regions in the
map.  Christoph, if you are okay with this I can quickly post a patch.

Regards,
Jim Quinlan
Broadcom STB


>
> I didn't catch this on my previous tests as I was using a newer bcm2711 SoC
> revision which expanded emmc2's DMA capabilities (can do 32 bit DMA as opposed
> to 30 bit).
>
> Regards,
> Nicolas
>
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan via iommu]

On Mon, Sep 7, 2020 at 11:01 AM Nicolas Saenz Julienne
<nsaenzjulienne@suse.de> wrote:
>
> Hi Jim, sorry I'm a little late to the party, but was on vacation.
>
> On Thu, 2020-09-03 at 13:32 -0400, Jim Quinlan wrote:
> > On Wed, Sep 2, 2020 at 8:52 PM Nathan Chancellor
> > <natechancellor@gmail.com> wrote:
> > > On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> > > >
> > > > On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> > > > [snip]
> > > > > > Hello Nathan,
> > > > > >
> > > > > > Can you tell me how much memory your RPI has and if all of it is
> > > > >
> > > > > This is the 4GB version.
> > > > >
> > > > > > accessible by the PCIe device?  Could you also please include the DTS
> > > > > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > > > > PCIe DT before Linux boots -- could you describe what is going on
> > > > > > there?
> > > > >
> > > > > Unfortunately, I am not familiar with how to get this information. If
> > > > > you could provide some instructions for how to do so, I am more than
> > > > > happy to. I am not very knowleagable about the inner working of the Pi,
> > > > > I mainly use it as a test platform for making sure that LLVM does not
> > > > > cause problems on real devices.
> > > >
> > > > Can you bring the dtc application to your Pi root filesystem, and if so, can
> > > > you run the following:
> > > >
> > > > dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb
> > >
> > > Sure, the result is attached.
> > >
> > > > or cat /sys/firmware/fdt > device.dtb
> > > >
> > > > and attach the resulting file?
> > > >
> > > > > > Finally, can you attach the text of the full boot log?
> > > > >
> > > > > I have attached a working and broken boot log. Thank you for the quick
> > > > > response!
> > > >
> > > > Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> > > > chance?
> > >
> > > Of course. A new log is attached with the debug output from that config.
> >
> > Hi Nicolas,
> >
> > Can you please help us out here?  It appears that your commit
>
> It's dma_offset_from_dma_addr() that's causing trouble. It goes over all the
> dma regions and, if no region matches the phys address range, it returns 0.
> This isn't acceptable as is. In the past, we used to pass the offset
> nonetheless, which would make the phys address grow out of the dma memory area
> and fail the dma_capable() test.
>
> For example, RPi4 devices attached to the old interconnect see phys [0x0
> 0x3fffffff] at [0xc0000000 0xffffffff]. So say you're trying to convert
> physical address 0xfa000000, you'll get 0 from dma_offset_from_phys_addr()
> (since your dma range only covers the first GB) to then test if 0xfa000000 is
> dma_capable(), which it is, but for the wrong reasons. Causing DMA issues
> further down the line.
>
> I don't have a proper suggestion on how to solve this but here's the solution I
> used:
>
> diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> index 4c4646761afe..40fe3c97f2be 100644
> --- a/include/linux/dma-mapping.h
> +++ b/include/linux/dma-mapping.h
> @@ -217,11 +217,19 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
>  {
>         const struct bus_dma_region *m = dev->dma_range_map;
>
> -       if (m)
> +       if (m) {
>                 for (; m->size; m++)
>                         if (paddr >= m->cpu_start &&
>                             paddr - m->cpu_start < m->size)
>                                 return m->offset;
> +
> +               /*
> +                * The physical address doesn't fit any of the DMA regions,
> +                * return an impossible to fulfill offset.
> +                */
> +               return -(1ULL << 46);
> +       }
> +
>         return 0;
>  }
Hi Nicolas,

Thanks for looking into this.  The concern I have with your solution
is that returning an arbitrarily large offset might overlap with an
improbable but valid usage.  AFAIK there is nothing that disallows
mapping a device to anywhere within the 64bit range of PCIe space,
including up to say 0xffffffffffffffff.

As an alternative perhaps changing dma_capable() so that if the
dma_range_map is present then it validates that both ends of the
prospective DMA region get "hits" on one of the offset regions in the
map.  Christoph, if you are okay with this I can quickly post a patch.

Regards,
Jim Quinlan
Broadcom STB


>
> I didn't catch this on my previous tests as I was using a newer bcm2711 SoC
> revision which expanded emmc2's DMA capabilities (can do 32 bit DMA as opposed
> to 30 bit).
>
> Regards,
> Nicolas
>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

On Mon, Sep 7, 2020 at 11:01 AM Nicolas Saenz Julienne
<nsaenzjulienne@suse.de> wrote:
>
> Hi Jim, sorry I'm a little late to the party, but was on vacation.
>
> On Thu, 2020-09-03 at 13:32 -0400, Jim Quinlan wrote:
> > On Wed, Sep 2, 2020 at 8:52 PM Nathan Chancellor
> > <natechancellor@gmail.com> wrote:
> > > On Wed, Sep 02, 2020 at 05:36:29PM -0700, Florian Fainelli wrote:
> > > >
> > > > On 9/2/2020 3:38 PM, Nathan Chancellor wrote:
> > > > [snip]
> > > > > > Hello Nathan,
> > > > > >
> > > > > > Can you tell me how much memory your RPI has and if all of it is
> > > > >
> > > > > This is the 4GB version.
> > > > >
> > > > > > accessible by the PCIe device?  Could you also please include the DTS
> > > > > > of the PCIe node?  IIRC, the RPI firmware does some mangling of the
> > > > > > PCIe DT before Linux boots -- could you describe what is going on
> > > > > > there?
> > > > >
> > > > > Unfortunately, I am not familiar with how to get this information. If
> > > > > you could provide some instructions for how to do so, I am more than
> > > > > happy to. I am not very knowleagable about the inner working of the Pi,
> > > > > I mainly use it as a test platform for making sure that LLVM does not
> > > > > cause problems on real devices.
> > > >
> > > > Can you bring the dtc application to your Pi root filesystem, and if so, can
> > > > you run the following:
> > > >
> > > > dtc -I fs -O dtb /proc/device-tree -f > /tmp/device.dtb
> > >
> > > Sure, the result is attached.
> > >
> > > > or cat /sys/firmware/fdt > device.dtb
> > > >
> > > > and attach the resulting file?
> > > >
> > > > > > Finally, can you attach the text of the full boot log?
> > > > >
> > > > > I have attached a working and broken boot log. Thank you for the quick
> > > > > response!
> > > >
> > > > Is it possible for you to rebuild your kernel with CONFIG_MMC_DEBUG by any
> > > > chance?
> > >
> > > Of course. A new log is attached with the debug output from that config.
> >
> > Hi Nicolas,
> >
> > Can you please help us out here?  It appears that your commit
>
> It's dma_offset_from_dma_addr() that's causing trouble. It goes over all the
> dma regions and, if no region matches the phys address range, it returns 0.
> This isn't acceptable as is. In the past, we used to pass the offset
> nonetheless, which would make the phys address grow out of the dma memory area
> and fail the dma_capable() test.
>
> For example, RPi4 devices attached to the old interconnect see phys [0x0
> 0x3fffffff] at [0xc0000000 0xffffffff]. So say you're trying to convert
> physical address 0xfa000000, you'll get 0 from dma_offset_from_phys_addr()
> (since your dma range only covers the first GB) to then test if 0xfa000000 is
> dma_capable(), which it is, but for the wrong reasons. Causing DMA issues
> further down the line.
>
> I don't have a proper suggestion on how to solve this but here's the solution I
> used:
>
> diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> index 4c4646761afe..40fe3c97f2be 100644
> --- a/include/linux/dma-mapping.h
> +++ b/include/linux/dma-mapping.h
> @@ -217,11 +217,19 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
>  {
>         const struct bus_dma_region *m = dev->dma_range_map;
>
> -       if (m)
> +       if (m) {
>                 for (; m->size; m++)
>                         if (paddr >= m->cpu_start &&
>                             paddr - m->cpu_start < m->size)
>                                 return m->offset;
> +
> +               /*
> +                * The physical address doesn't fit any of the DMA regions,
> +                * return an impossible to fulfill offset.
> +                */
> +               return -(1ULL << 46);
> +       }
> +
>         return 0;
>  }
Hi Nicolas,

Thanks for looking into this.  The concern I have with your solution
is that returning an arbitrarily large offset might overlap with an
improbable but valid usage.  AFAIK there is nothing that disallows
mapping a device to anywhere within the 64bit range of PCIe space,
including up to say 0xffffffffffffffff.

As an alternative perhaps changing dma_capable() so that if the
dma_range_map is present then it validates that both ends of the
prospective DMA region get "hits" on one of the offset regions in the
map.  Christoph, if you are okay with this I can quickly post a patch.

Regards,
Jim Quinlan
Broadcom STB


>
> I didn't catch this on my previous tests as I was using a newer bcm2711 SoC
> revision which expanded emmc2's DMA capabilities (can do 32 bit DMA as opposed
> to 30 bit).
>
> Regards,
> Nicolas
>
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > >
> > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > Hi,
> > > >
> > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > >>
> > > >> Patchset Summary:
> > > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > > >
> > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > 5.10 merge window.
> > > >
> > > > There are a lot of different files being touched, so what would be the
> > > > ideal way of routing those changes towards inclusion?
> > >
> > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > apply the current version.
> > Sounds good to me.
>
> Hi Jim,
>
> is the dependency now solved ? Should we review/take this series as
> is for v5.10 through the PCI tree ?
Hello Lorenzo,

We are still working out a regression with the DMA offset commit on
the RaspberryPi.  Nicolas has found the root cause and we are now
devising a solution.

Thanks,
Jim Quinlan
Broadcom STB

>
> Thanks,
> Lorenzo

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > >
> > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > Hi,
> > > >
> > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > >>
> > > >> Patchset Summary:
> > > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > > >
> > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > 5.10 merge window.
> > > >
> > > > There are a lot of different files being touched, so what would be the
> > > > ideal way of routing those changes towards inclusion?
> > >
> > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > apply the current version.
> > Sounds good to me.
>
> Hi Jim,
>
> is the dependency now solved ? Should we review/take this series as
> is for v5.10 through the PCI tree ?
Hello Lorenzo,

We are still working out a regression with the DMA offset commit on
the RaspberryPi.  Nicolas has found the root cause and we are now
devising a solution.

Thanks,
Jim Quinlan
Broadcom STB

>
> Thanks,
> Lorenzo

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > >
> > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > Hi,
> > > >
> > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > >>
> > > >> Patchset Summary:
> > > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > > >
> > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > 5.10 merge window.
> > > >
> > > > There are a lot of different files being touched, so what would be the
> > > > ideal way of routing those changes towards inclusion?
> > >
> > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > apply the current version.
> > Sounds good to me.
>
> Hi Jim,
>
> is the dependency now solved ? Should we review/take this series as
> is for v5.10 through the PCI tree ?
Hello Lorenzo,

We are still working out a regression with the DMA offset commit on
the RaspberryPi.  Nicolas has found the root cause and we are now
devising a solution.

Thanks,
Jim Quinlan
Broadcom STB

>
> Thanks,
> Lorenzo
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan via iommu]

On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > >
> > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > Hi,
> > > >
> > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > >>
> > > >> Patchset Summary:
> > > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > > >
> > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > 5.10 merge window.
> > > >
> > > > There are a lot of different files being touched, so what would be the
> > > > ideal way of routing those changes towards inclusion?
> > >
> > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > apply the current version.
> > Sounds good to me.
>
> Hi Jim,
>
> is the dependency now solved ? Should we review/take this series as
> is for v5.10 through the PCI tree ?
Hello Lorenzo,

We are still working out a regression with the DMA offset commit on
the RaspberryPi.  Nicolas has found the root cause and we are now
devising a solution.

Thanks,
Jim Quinlan
Broadcom STB

>
> Thanks,
> Lorenzo
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > >
> > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > Hi,
> > > >
> > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > >>
> > > >> Patchset Summary:
> > > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > > >
> > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > 5.10 merge window.
> > > >
> > > > There are a lot of different files being touched, so what would be the
> > > > ideal way of routing those changes towards inclusion?
> > >
> > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > apply the current version.
> > Sounds good to me.
>
> Hi Jim,
>
> is the dependency now solved ? Should we review/take this series as
> is for v5.10 through the PCI tree ?
Hello Lorenzo,

We are still working out a regression with the DMA offset commit on
the RaspberryPi.  Nicolas has found the root cause and we are now
devising a solution.

Thanks,
Jim Quinlan
Broadcom STB

>
> Thanks,
> Lorenzo

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Jim Quinlan]

On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
<lorenzo.pieralisi@arm.com> wrote:
>
> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > >
> > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > Hi,
> > > >
> > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > >>
> > > >> Patchset Summary:
> > > >>    Enhance a PCIe host controller driver.  Because of its unusual design
> > > >>    we are foced to change dev->dma_pfn_offset into a more general role
> > > >>    allowing multiple offsets.  See the 'v1' notes below for more info.
> > > >
> > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > 5.10 merge window.
> > > >
> > > > There are a lot of different files being touched, so what would be the
> > > > ideal way of routing those changes towards inclusion?
> > >
> > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > apply the current version.
> > Sounds good to me.
>
> Hi Jim,
>
> is the dependency now solved ? Should we review/take this series as
> is for v5.10 through the PCI tree ?
Hello Lorenzo,

We are still working out a regression with the DMA offset commit on
the RaspberryPi.  Nicolas has found the root cause and we are now
devising a solution.

Thanks,
Jim Quinlan
Broadcom STB

>
> Thanks,
> Lorenzo
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 3604 bytes --]

On Mon, 2020-09-07 at 13:40 -0400, Jim Quinlan wrote:
> On Mon, Sep 7, 2020 at 11:01 AM Nicolas Saenz Julienne
> <nsaenzjulienne@suse.de> wrote:
> > > 
> > > Hi Nicolas,
> > > 
> > > Can you please help us out here?  It appears that your commit
> > 
> > It's dma_offset_from_dma_addr() that's causing trouble. It goes over all the
> > dma regions and, if no region matches the phys address range, it returns 0.
> > This isn't acceptable as is. In the past, we used to pass the offset
> > nonetheless, which would make the phys address grow out of the dma memory area
> > and fail the dma_capable() test.
> > 
> > For example, RPi4 devices attached to the old interconnect see phys [0x0
> > 0x3fffffff] at [0xc0000000 0xffffffff]. So say you're trying to convert
> > physical address 0xfa000000, you'll get 0 from dma_offset_from_phys_addr()
> > (since your dma range only covers the first GB) to then test if 0xfa000000 is
> > dma_capable(), which it is, but for the wrong reasons. Causing DMA issues
> > further down the line.
> > 
> > I don't have a proper suggestion on how to solve this but here's the solution I
> > used:
> > 
> > diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> > index 4c4646761afe..40fe3c97f2be 100644
> > --- a/include/linux/dma-mapping.h
> > +++ b/include/linux/dma-mapping.h
> > @@ -217,11 +217,19 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
> >  {
> >         const struct bus_dma_region *m = dev->dma_range_map;
> > 
> > -       if (m)
> > +       if (m) {
> >                 for (; m->size; m++)
> >                         if (paddr >= m->cpu_start &&
> >                             paddr - m->cpu_start < m->size)
> >                                 return m->offset;
> > +
> > +               /*
> > +                * The physical address doesn't fit any of the DMA regions,
> > +                * return an impossible to fulfill offset.
> > +                */
> > +               return -(1ULL << 46);
> > +       }
> > +
> >         return 0;
> >  }
> Hi Nicolas,
> 
> Thanks for looking into this.  The concern I have with your solution
> is that returning an arbitrarily large offset might overlap with an
> improbable but valid usage.  AFAIK there is nothing that disallows
> mapping a device to anywhere within the 64bit range of PCIe space,
> including up to say 0xffffffffffffffff.

Indeed, that's why I wasn't all that happy with my solution.

As an alternative, how about returning '-dev->bus_dma_limit' instead of 0? It's
always 0 for the devices without bus_dma_regions, and, I think, an always
unattainable offset for devices that do (I tested it on RPi4 with the 30bit
limitd mmc controller and it seems to work alright).

--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -222,7 +222,8 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
                        if (paddr >= m->cpu_start &&
                            paddr - m->cpu_start < m->size)
                                return m->offset;
-       return 0;
+
+       return -dev->bus_dma_limit;
 }

> As an alternative perhaps changing dma_capable() so that if the
> dma_range_map is present then it validates that both ends of the
> prospective DMA region get "hits" on one of the offset regions in the
> map.  Christoph, if you are okay with this I can quickly post a patch.

IIUC, by the time you enter dma_capable() you already converted the physical
address to DMA address and the damage is done.

Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 484 bytes --]

[-- Attachment #2: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 3604 bytes --]

On Mon, 2020-09-07 at 13:40 -0400, Jim Quinlan wrote:
> On Mon, Sep 7, 2020 at 11:01 AM Nicolas Saenz Julienne
> <nsaenzjulienne@suse.de> wrote:
> > > 
> > > Hi Nicolas,
> > > 
> > > Can you please help us out here?  It appears that your commit
> > 
> > It's dma_offset_from_dma_addr() that's causing trouble. It goes over all the
> > dma regions and, if no region matches the phys address range, it returns 0.
> > This isn't acceptable as is. In the past, we used to pass the offset
> > nonetheless, which would make the phys address grow out of the dma memory area
> > and fail the dma_capable() test.
> > 
> > For example, RPi4 devices attached to the old interconnect see phys [0x0
> > 0x3fffffff] at [0xc0000000 0xffffffff]. So say you're trying to convert
> > physical address 0xfa000000, you'll get 0 from dma_offset_from_phys_addr()
> > (since your dma range only covers the first GB) to then test if 0xfa000000 is
> > dma_capable(), which it is, but for the wrong reasons. Causing DMA issues
> > further down the line.
> > 
> > I don't have a proper suggestion on how to solve this but here's the solution I
> > used:
> > 
> > diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> > index 4c4646761afe..40fe3c97f2be 100644
> > --- a/include/linux/dma-mapping.h
> > +++ b/include/linux/dma-mapping.h
> > @@ -217,11 +217,19 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
> >  {
> >         const struct bus_dma_region *m = dev->dma_range_map;
> > 
> > -       if (m)
> > +       if (m) {
> >                 for (; m->size; m++)
> >                         if (paddr >= m->cpu_start &&
> >                             paddr - m->cpu_start < m->size)
> >                                 return m->offset;
> > +
> > +               /*
> > +                * The physical address doesn't fit any of the DMA regions,
> > +                * return an impossible to fulfill offset.
> > +                */
> > +               return -(1ULL << 46);
> > +       }
> > +
> >         return 0;
> >  }
> Hi Nicolas,
> 
> Thanks for looking into this.  The concern I have with your solution
> is that returning an arbitrarily large offset might overlap with an
> improbable but valid usage.  AFAIK there is nothing that disallows
> mapping a device to anywhere within the 64bit range of PCIe space,
> including up to say 0xffffffffffffffff.

Indeed, that's why I wasn't all that happy with my solution.

As an alternative, how about returning '-dev->bus_dma_limit' instead of 0? It's
always 0 for the devices without bus_dma_regions, and, I think, an always
unattainable offset for devices that do (I tested it on RPi4 with the 30bit
limitd mmc controller and it seems to work alright).

--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -222,7 +222,8 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
                        if (paddr >= m->cpu_start &&
                            paddr - m->cpu_start < m->size)
                                return m->offset;
-       return 0;
+
+       return -dev->bus_dma_limit;
 }

> As an alternative perhaps changing dma_capable() so that if the
> dma_range_map is present then it validates that both ends of the
> prospective DMA region get "hits" on one of the offset regions in the
> map.  Christoph, if you are okay with this I can quickly post a patch.

IIUC, by the time you enter dma_capable() you already converted the physical
address to DMA address and the damage is done.

Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 484 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 3604 bytes --]

On Mon, 2020-09-07 at 13:40 -0400, Jim Quinlan wrote:
> On Mon, Sep 7, 2020 at 11:01 AM Nicolas Saenz Julienne
> <nsaenzjulienne@suse.de> wrote:
> > > 
> > > Hi Nicolas,
> > > 
> > > Can you please help us out here?  It appears that your commit
> > 
> > It's dma_offset_from_dma_addr() that's causing trouble. It goes over all the
> > dma regions and, if no region matches the phys address range, it returns 0.
> > This isn't acceptable as is. In the past, we used to pass the offset
> > nonetheless, which would make the phys address grow out of the dma memory area
> > and fail the dma_capable() test.
> > 
> > For example, RPi4 devices attached to the old interconnect see phys [0x0
> > 0x3fffffff] at [0xc0000000 0xffffffff]. So say you're trying to convert
> > physical address 0xfa000000, you'll get 0 from dma_offset_from_phys_addr()
> > (since your dma range only covers the first GB) to then test if 0xfa000000 is
> > dma_capable(), which it is, but for the wrong reasons. Causing DMA issues
> > further down the line.
> > 
> > I don't have a proper suggestion on how to solve this but here's the solution I
> > used:
> > 
> > diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> > index 4c4646761afe..40fe3c97f2be 100644
> > --- a/include/linux/dma-mapping.h
> > +++ b/include/linux/dma-mapping.h
> > @@ -217,11 +217,19 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
> >  {
> >         const struct bus_dma_region *m = dev->dma_range_map;
> > 
> > -       if (m)
> > +       if (m) {
> >                 for (; m->size; m++)
> >                         if (paddr >= m->cpu_start &&
> >                             paddr - m->cpu_start < m->size)
> >                                 return m->offset;
> > +
> > +               /*
> > +                * The physical address doesn't fit any of the DMA regions,
> > +                * return an impossible to fulfill offset.
> > +                */
> > +               return -(1ULL << 46);
> > +       }
> > +
> >         return 0;
> >  }
> Hi Nicolas,
> 
> Thanks for looking into this.  The concern I have with your solution
> is that returning an arbitrarily large offset might overlap with an
> improbable but valid usage.  AFAIK there is nothing that disallows
> mapping a device to anywhere within the 64bit range of PCIe space,
> including up to say 0xffffffffffffffff.

Indeed, that's why I wasn't all that happy with my solution.

As an alternative, how about returning '-dev->bus_dma_limit' instead of 0? It's
always 0 for the devices without bus_dma_regions, and, I think, an always
unattainable offset for devices that do (I tested it on RPi4 with the 30bit
limitd mmc controller and it seems to work alright).

--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -222,7 +222,8 @@ static inline u64 dma_offset_from_phys_addr(struct device *dev,
                        if (paddr >= m->cpu_start &&
                            paddr - m->cpu_start < m->size)
                                return m->offset;
-       return 0;
+
+       return -dev->bus_dma_limit;
 }

> As an alternative perhaps changing dma_capable() so that if the
> dma_range_map is present then it validates that both ends of the
> prospective DMA region get "hits" on one of the offset regions in the
> map.  Christoph, if you are okay with this I can quickly post a patch.

IIUC, by the time you enter dma_capable() you already converted the physical
address to DMA address and the damage is done.

Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 484 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
>>
>> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
>>> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>>>>
>>>> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
>>>>> Hi,
>>>>>
>>>>> On 8/24/2020 12:30 PM, Jim Quinlan wrote:
>>>>>>
>>>>>> Patchset Summary:
>>>>>>     Enhance a PCIe host controller driver.  Because of its unusual design
>>>>>>     we are foced to change dev->dma_pfn_offset into a more general role
>>>>>>     allowing multiple offsets.  See the 'v1' notes below for more info.
>>>>>
>>>>> We are version 11 and counting, and it is not clear to me whether there is
>>>>> any chance of getting these patches reviewed and hopefully merged for the
>>>>> 5.10 merge window.
>>>>>
>>>>> There are a lot of different files being touched, so what would be the
>>>>> ideal way of routing those changes towards inclusion?
>>>>
>>>> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
>>>> I have a bit of a backlog, but plan to review and if Jim is ok with that
>>>> apply the current version.
>>> Sounds good to me.
>>
>> Hi Jim,
>>
>> is the dependency now solved ? Should we review/take this series as
>> is for v5.10 through the PCI tree ?
> Hello Lorenzo,
> 
> We are still working out a regression with the DMA offset commit on
> the RaspberryPi.  Nicolas has found the root cause and we are now
> devising a solution.

Maybe we can parallelize the PCIe driver review while the DMA changes 
are being worked on in Christoph's branch. Lorenzo, are you fine with 
the PCIe changes proper?
-- 
Florian

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
>>
>> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
>>> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>>>>
>>>> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
>>>>> Hi,
>>>>>
>>>>> On 8/24/2020 12:30 PM, Jim Quinlan wrote:
>>>>>>
>>>>>> Patchset Summary:
>>>>>>     Enhance a PCIe host controller driver.  Because of its unusual design
>>>>>>     we are foced to change dev->dma_pfn_offset into a more general role
>>>>>>     allowing multiple offsets.  See the 'v1' notes below for more info.
>>>>>
>>>>> We are version 11 and counting, and it is not clear to me whether there is
>>>>> any chance of getting these patches reviewed and hopefully merged for the
>>>>> 5.10 merge window.
>>>>>
>>>>> There are a lot of different files being touched, so what would be the
>>>>> ideal way of routing those changes towards inclusion?
>>>>
>>>> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
>>>> I have a bit of a backlog, but plan to review and if Jim is ok with that
>>>> apply the current version.
>>> Sounds good to me.
>>
>> Hi Jim,
>>
>> is the dependency now solved ? Should we review/take this series as
>> is for v5.10 through the PCI tree ?
> Hello Lorenzo,
> 
> We are still working out a regression with the DMA offset commit on
> the RaspberryPi.  Nicolas has found the root cause and we are now
> devising a solution.

Maybe we can parallelize the PCIe driver review while the DMA changes 
are being worked on in Christoph's branch. Lorenzo, are you fine with 
the PCIe changes proper?
-- 
Florian

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
>>
>> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
>>> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>>>>
>>>> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
>>>>> Hi,
>>>>>
>>>>> On 8/24/2020 12:30 PM, Jim Quinlan wrote:
>>>>>>
>>>>>> Patchset Summary:
>>>>>>     Enhance a PCIe host controller driver.  Because of its unusual design
>>>>>>     we are foced to change dev->dma_pfn_offset into a more general role
>>>>>>     allowing multiple offsets.  See the 'v1' notes below for more info.
>>>>>
>>>>> We are version 11 and counting, and it is not clear to me whether there is
>>>>> any chance of getting these patches reviewed and hopefully merged for the
>>>>> 5.10 merge window.
>>>>>
>>>>> There are a lot of different files being touched, so what would be the
>>>>> ideal way of routing those changes towards inclusion?
>>>>
>>>> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
>>>> I have a bit of a backlog, but plan to review and if Jim is ok with that
>>>> apply the current version.
>>> Sounds good to me.
>>
>> Hi Jim,
>>
>> is the dependency now solved ? Should we review/take this series as
>> is for v5.10 through the PCI tree ?
> Hello Lorenzo,
> 
> We are still working out a regression with the DMA offset commit on
> the RaspberryPi.  Nicolas has found the root cause and we are now
> devising a solution.

Maybe we can parallelize the PCIe driver review while the DMA changes 
are being worked on in Christoph's branch. Lorenzo, are you fine with 
the PCIe changes proper?
-- 
Florian
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
>>
>> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
>>> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>>>>
>>>> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
>>>>> Hi,
>>>>>
>>>>> On 8/24/2020 12:30 PM, Jim Quinlan wrote:
>>>>>>
>>>>>> Patchset Summary:
>>>>>>     Enhance a PCIe host controller driver.  Because of its unusual design
>>>>>>     we are foced to change dev->dma_pfn_offset into a more general role
>>>>>>     allowing multiple offsets.  See the 'v1' notes below for more info.
>>>>>
>>>>> We are version 11 and counting, and it is not clear to me whether there is
>>>>> any chance of getting these patches reviewed and hopefully merged for the
>>>>> 5.10 merge window.
>>>>>
>>>>> There are a lot of different files being touched, so what would be the
>>>>> ideal way of routing those changes towards inclusion?
>>>>
>>>> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
>>>> I have a bit of a backlog, but plan to review and if Jim is ok with that
>>>> apply the current version.
>>> Sounds good to me.
>>
>> Hi Jim,
>>
>> is the dependency now solved ? Should we review/take this series as
>> is for v5.10 through the PCI tree ?
> Hello Lorenzo,
> 
> We are still working out a regression with the DMA offset commit on
> the RaspberryPi.  Nicolas has found the root cause and we are now
> devising a solution.

Maybe we can parallelize the PCIe driver review while the DMA changes 
are being worked on in Christoph's branch. Lorenzo, are you fine with 
the PCIe changes proper?
-- 
Florian
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
>>
>> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
>>> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>>>>
>>>> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
>>>>> Hi,
>>>>>
>>>>> On 8/24/2020 12:30 PM, Jim Quinlan wrote:
>>>>>>
>>>>>> Patchset Summary:
>>>>>>     Enhance a PCIe host controller driver.  Because of its unusual design
>>>>>>     we are foced to change dev->dma_pfn_offset into a more general role
>>>>>>     allowing multiple offsets.  See the 'v1' notes below for more info.
>>>>>
>>>>> We are version 11 and counting, and it is not clear to me whether there is
>>>>> any chance of getting these patches reviewed and hopefully merged for the
>>>>> 5.10 merge window.
>>>>>
>>>>> There are a lot of different files being touched, so what would be the
>>>>> ideal way of routing those changes towards inclusion?
>>>>
>>>> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
>>>> I have a bit of a backlog, but plan to review and if Jim is ok with that
>>>> apply the current version.
>>> Sounds good to me.
>>
>> Hi Jim,
>>
>> is the dependency now solved ? Should we review/take this series as
>> is for v5.10 through the PCI tree ?
> Hello Lorenzo,
> 
> We are still working out a regression with the DMA offset commit on
> the RaspberryPi.  Nicolas has found the root cause and we are now
> devising a solution.

Maybe we can parallelize the PCIe driver review while the DMA changes 
are being worked on in Christoph's branch. Lorenzo, are you fine with 
the PCIe changes proper?
-- 
Florian

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Florian Fainelli]

On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com> wrote:
>>
>> On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
>>> On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
>>>>
>>>> On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
>>>>> Hi,
>>>>>
>>>>> On 8/24/2020 12:30 PM, Jim Quinlan wrote:
>>>>>>
>>>>>> Patchset Summary:
>>>>>>     Enhance a PCIe host controller driver.  Because of its unusual design
>>>>>>     we are foced to change dev->dma_pfn_offset into a more general role
>>>>>>     allowing multiple offsets.  See the 'v1' notes below for more info.
>>>>>
>>>>> We are version 11 and counting, and it is not clear to me whether there is
>>>>> any chance of getting these patches reviewed and hopefully merged for the
>>>>> 5.10 merge window.
>>>>>
>>>>> There are a lot of different files being touched, so what would be the
>>>>> ideal way of routing those changes towards inclusion?
>>>>
>>>> FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
>>>> I have a bit of a backlog, but plan to review and if Jim is ok with that
>>>> apply the current version.
>>> Sounds good to me.
>>
>> Hi Jim,
>>
>> is the dependency now solved ? Should we review/take this series as
>> is for v5.10 through the PCI tree ?
> Hello Lorenzo,
> 
> We are still working out a regression with the DMA offset commit on
> the RaspberryPi.  Nicolas has found the root cause and we are now
> devising a solution.

Maybe we can parallelize the PCIe driver review while the DMA changes 
are being worked on in Christoph's branch. Lorenzo, are you fine with 
the PCIe changes proper?
-- 
Florian
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

On Mon, Sep 07, 2020 at 05:18:59PM +0200, Nicolas Saenz Julienne wrote:
> Hi Christoph, a small fix to your fixes:

Thanks,

folded into the patch on the dma-ranges branch.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

On Mon, Sep 07, 2020 at 05:18:59PM +0200, Nicolas Saenz Julienne wrote:
> Hi Christoph, a small fix to your fixes:

Thanks,

folded into the patch on the dma-ranges branch.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

On Mon, Sep 07, 2020 at 01:40:46PM -0400, Jim Quinlan wrote:
> Thanks for looking into this.  The concern I have with your solution
> is that returning an arbitrarily large offset might overlap with an
> improbable but valid usage.  AFAIK there is nothing that disallows
> mapping a device to anywhere within the 64bit range of PCIe space,
> including up to say 0xffffffffffffffff.
> 
> As an alternative perhaps changing dma_capable() so that if the
> dma_range_map is present then it validates that both ends of the
> prospective DMA region get "hits" on one of the offset regions in the
> map.  Christoph, if you are okay with this I can quickly post a patch.

We use a dma_addr of all-Fs as an error code, see the definition of
DMA_MAPPING_ERROR.

The rationale of why we think it is safe:  We don't do single byte
mappings, so the last address of the address space can't effectively
be used for anything.

So I think it would be a good fit here.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

On Mon, Sep 07, 2020 at 01:40:46PM -0400, Jim Quinlan wrote:
> Thanks for looking into this.  The concern I have with your solution
> is that returning an arbitrarily large offset might overlap with an
> improbable but valid usage.  AFAIK there is nothing that disallows
> mapping a device to anywhere within the 64bit range of PCIe space,
> including up to say 0xffffffffffffffff.
> 
> As an alternative perhaps changing dma_capable() so that if the
> dma_range_map is present then it validates that both ends of the
> prospective DMA region get "hits" on one of the offset regions in the
> map.  Christoph, if you are okay with this I can quickly post a patch.

We use a dma_addr of all-Fs as an error code, see the definition of
DMA_MAPPING_ERROR.

The rationale of why we think it is safe:  We don't do single byte
mappings, so the last address of the address space can't effectively
be used for anything.

So I think it would be a good fit here.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

On Mon, Sep 07, 2020 at 08:19:43PM +0200, Nicolas Saenz Julienne wrote:
> Indeed, that's why I wasn't all that happy with my solution.
> 
> As an alternative, how about returning '-dev->bus_dma_limit' instead of 0? It's
> always 0 for the devices without bus_dma_regions, and, I think, an always
> unattainable offset for devices that do (I tested it on RPi4 with the 30bit
> limitd mmc controller and it seems to work alright).

No, bus_dma_limit can be set independent of offsets.  We use it e.g.
to limit old x86 VIA PCI bridges to 32-bit addressing.
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

On Mon, Sep 07, 2020 at 08:19:43PM +0200, Nicolas Saenz Julienne wrote:
> Indeed, that's why I wasn't all that happy with my solution.
> 
> As an alternative, how about returning '-dev->bus_dma_limit' instead of 0? It's
> always 0 for the devices without bus_dma_regions, and, I think, an always
> unattainable offset for devices that do (I tested it on RPi4 with the 30bit
> limitd mmc controller and it seems to work alright).

No, bus_dma_limit can be set independent of offsets.  We use it e.g.
to limit old x86 VIA PCI bridges to 32-bit addressing.
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

FYI, this is what I'd do relative to the patch on the dma-ranges
branch.  In fact realizing this makes me want to refactor things a bit
so that the new code can entirely live in the dma-direct code, but please
test this first:


diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index c21893f683b585..072fc42349874d 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -35,21 +35,16 @@ static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
 #ifndef __arch_pfn_to_dma
 static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
 {
-	if (dev) {
-		phys_addr_t paddr = PFN_PHYS(pfn);
-
-		pfn -= PFN_DOWN(dma_offset_from_phys_addr(dev, paddr));
-	}
-	return (dma_addr_t)__pfn_to_bus(pfn);
+	if (!dev)
+		return (dma_addr_t)__pfn_to_bus(pfn);
+	return translate_phys_to_dma(dev, PFN_PHYS(pfn));
 }
 
 static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
 {
-	unsigned long pfn = __bus_to_pfn(addr);
-
-	if (dev)
-		pfn += PFN_DOWN(dma_offset_from_dma_addr(dev, addr));
-	return pfn;
+	if (!dev)
+		return __bus_to_pfn(addr);
+	return PFN_DOWN(translate_dma_to_phys(dev, addr));
 }
 
 static inline void *dma_to_virt(struct device *dev, dma_addr_t addr)
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index 7831ca5b1b5dd6..e624171c4962ad 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -19,12 +19,16 @@ extern unsigned int zone_dma_bits;
 #else
 static inline dma_addr_t __phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	return (dma_addr_t)paddr - dma_offset_from_phys_addr(dev, paddr);
+	if (dev->dma_range_map)
+		return (dma_addr_t)paddr - translate_phys_to_dma(dev, paddr);
+	return (dma_addr_t)paddr;
 }
 
-static inline phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dev_addr)
+static inline phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dma_addr)
 {
-	return (phys_addr_t)dev_addr + dma_offset_from_dma_addr(dev, dev_addr);
+	if (dev->dma_range_map)
+		return translate_dma_to_phys(dev, dma_addr);
+	return (phys_addr_t)dma_addr;
 }
 #endif /* !CONFIG_ARCH_HAS_PHYS_TO_DMA */
 
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4c4646761afee4..3b1ceebb6f2ad5 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -199,29 +199,28 @@ struct bus_dma_region {
 };
 
 #ifdef CONFIG_HAS_DMA
-static inline u64 dma_offset_from_dma_addr(struct device *dev,
-		dma_addr_t dma_addr)
+static inline dma_addr_t translate_phys_to_dma(struct device *dev,
+		phys_addr_t paddr)
 {
-	const struct bus_dma_region *m = dev->dma_range_map;
+	const struct bus_dma_region *m;
 
-	if (m)
-		for (; m->size; m++)
-			if (dma_addr >= m->dma_start &&
-			    dma_addr - m->dma_start < m->size)
-				return m->offset;
-	return 0;
+	for (m = dev->dma_range_map; m->size; m++)
+		if (paddr >= m->cpu_start && paddr - m->cpu_start < m->size)
+			return (dma_addr_t)paddr - m->offset;
+
+	/* make sure dma_capable fails when no translation is available */
+	return DMA_MAPPING_ERROR; 
 }
 
-static inline u64 dma_offset_from_phys_addr(struct device *dev,
-		phys_addr_t paddr)
+static inline phys_addr_t translate_dma_to_phys(struct device *dev,
+		dma_addr_t dma_addr)
 {
-	const struct bus_dma_region *m = dev->dma_range_map;
+	const struct bus_dma_region *m;
+
+	for (m = dev->dma_range_map; m->size; m++)
+		if (dma_addr >= m->dma_start && dma_addr - m->dma_start < m->size)
+			return (phys_addr_t)dma_addr + m->offset;
 
-	if (m)
-		for (; m->size; m++)
-			if (paddr >= m->cpu_start &&
-			    paddr - m->cpu_start < m->size)
-				return m->offset;
 	return 0;
 }
 
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

FYI, this is what I'd do relative to the patch on the dma-ranges
branch.  In fact realizing this makes me want to refactor things a bit
so that the new code can entirely live in the dma-direct code, but please
test this first:


diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index c21893f683b585..072fc42349874d 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -35,21 +35,16 @@ static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
 #ifndef __arch_pfn_to_dma
 static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
 {
-	if (dev) {
-		phys_addr_t paddr = PFN_PHYS(pfn);
-
-		pfn -= PFN_DOWN(dma_offset_from_phys_addr(dev, paddr));
-	}
-	return (dma_addr_t)__pfn_to_bus(pfn);
+	if (!dev)
+		return (dma_addr_t)__pfn_to_bus(pfn);
+	return translate_phys_to_dma(dev, PFN_PHYS(pfn));
 }
 
 static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
 {
-	unsigned long pfn = __bus_to_pfn(addr);
-
-	if (dev)
-		pfn += PFN_DOWN(dma_offset_from_dma_addr(dev, addr));
-	return pfn;
+	if (!dev)
+		return __bus_to_pfn(addr);
+	return PFN_DOWN(translate_dma_to_phys(dev, addr));
 }
 
 static inline void *dma_to_virt(struct device *dev, dma_addr_t addr)
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index 7831ca5b1b5dd6..e624171c4962ad 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -19,12 +19,16 @@ extern unsigned int zone_dma_bits;
 #else
 static inline dma_addr_t __phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	return (dma_addr_t)paddr - dma_offset_from_phys_addr(dev, paddr);
+	if (dev->dma_range_map)
+		return (dma_addr_t)paddr - translate_phys_to_dma(dev, paddr);
+	return (dma_addr_t)paddr;
 }
 
-static inline phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dev_addr)
+static inline phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dma_addr)
 {
-	return (phys_addr_t)dev_addr + dma_offset_from_dma_addr(dev, dev_addr);
+	if (dev->dma_range_map)
+		return translate_dma_to_phys(dev, dma_addr);
+	return (phys_addr_t)dma_addr;
 }
 #endif /* !CONFIG_ARCH_HAS_PHYS_TO_DMA */
 
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4c4646761afee4..3b1ceebb6f2ad5 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -199,29 +199,28 @@ struct bus_dma_region {
 };
 
 #ifdef CONFIG_HAS_DMA
-static inline u64 dma_offset_from_dma_addr(struct device *dev,
-		dma_addr_t dma_addr)
+static inline dma_addr_t translate_phys_to_dma(struct device *dev,
+		phys_addr_t paddr)
 {
-	const struct bus_dma_region *m = dev->dma_range_map;
+	const struct bus_dma_region *m;
 
-	if (m)
-		for (; m->size; m++)
-			if (dma_addr >= m->dma_start &&
-			    dma_addr - m->dma_start < m->size)
-				return m->offset;
-	return 0;
+	for (m = dev->dma_range_map; m->size; m++)
+		if (paddr >= m->cpu_start && paddr - m->cpu_start < m->size)
+			return (dma_addr_t)paddr - m->offset;
+
+	/* make sure dma_capable fails when no translation is available */
+	return DMA_MAPPING_ERROR; 
 }
 
-static inline u64 dma_offset_from_phys_addr(struct device *dev,
-		phys_addr_t paddr)
+static inline phys_addr_t translate_dma_to_phys(struct device *dev,
+		dma_addr_t dma_addr)
 {
-	const struct bus_dma_region *m = dev->dma_range_map;
+	const struct bus_dma_region *m;
+
+	for (m = dev->dma_range_map; m->size; m++)
+		if (dma_addr >= m->dma_start && dma_addr - m->dma_start < m->size)
+			return (phys_addr_t)dma_addr + m->offset;
 
-	if (m)
-		for (; m->size; m++)
-			if (paddr >= m->cpu_start &&
-			    paddr - m->cpu_start < m->size)
-				return m->offset;
 	return 0;
 }
 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

On Tue, Sep 08, 2020 at 09:29:35AM +0200, Christoph Hellwig wrote:
> FYI, this is what I'd do relative to the patch on the dma-ranges
> branch.  In fact realizing this makes me want to refactor things a bit
> so that the new code can entirely live in the dma-direct code, but please
> test this first:

And of course this isn't going to work for arm devices without any
range, so let's try this instead:

diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index c21893f683b585..e913e04d2be8b9 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -35,21 +35,20 @@ static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
 #ifndef __arch_pfn_to_dma
 static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
 {
-	if (dev) {
-		phys_addr_t paddr = PFN_PHYS(pfn);
-
-		pfn -= PFN_DOWN(dma_offset_from_phys_addr(dev, paddr));
-	}
-	return (dma_addr_t)__pfn_to_bus(pfn);
+	if (!dev)
+		return (dma_addr_t)__pfn_to_bus(pfn);
+	if (dev->dma_range_map)
+		return translate_phys_to_dma(dev, PFN_PHYS(pfn));
+	return (dma_addr_t)PFN_PHYS(pfn);
 }
 
 static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
 {
-	unsigned long pfn = __bus_to_pfn(addr);
-
-	if (dev)
-		pfn += PFN_DOWN(dma_offset_from_dma_addr(dev, addr));
-	return pfn;
+	if (!dev)
+		return __bus_to_pfn(addr);
+	if (dev->dma_range_map)
+		return PFN_DOWN(translate_dma_to_phys(dev, addr));
+	return PFN_DOWN(addr);
 }
 
 static inline void *dma_to_virt(struct device *dev, dma_addr_t addr)
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index 7831ca5b1b5dd6..e624171c4962ad 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -19,12 +19,16 @@ extern unsigned int zone_dma_bits;
 #else
 static inline dma_addr_t __phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	return (dma_addr_t)paddr - dma_offset_from_phys_addr(dev, paddr);
+	if (dev->dma_range_map)
+		return (dma_addr_t)paddr - translate_phys_to_dma(dev, paddr);
+	return (dma_addr_t)paddr;
 }
 
-static inline phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dev_addr)
+static inline phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dma_addr)
 {
-	return (phys_addr_t)dev_addr + dma_offset_from_dma_addr(dev, dev_addr);
+	if (dev->dma_range_map)
+		return translate_dma_to_phys(dev, dma_addr);
+	return (phys_addr_t)dma_addr;
 }
 #endif /* !CONFIG_ARCH_HAS_PHYS_TO_DMA */
 
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4c4646761afee4..3b1ceebb6f2ad5 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -199,29 +199,28 @@ struct bus_dma_region {
 };
 
 #ifdef CONFIG_HAS_DMA
-static inline u64 dma_offset_from_dma_addr(struct device *dev,
-		dma_addr_t dma_addr)
+static inline dma_addr_t translate_phys_to_dma(struct device *dev,
+		phys_addr_t paddr)
 {
-	const struct bus_dma_region *m = dev->dma_range_map;
+	const struct bus_dma_region *m;
 
-	if (m)
-		for (; m->size; m++)
-			if (dma_addr >= m->dma_start &&
-			    dma_addr - m->dma_start < m->size)
-				return m->offset;
-	return 0;
+	for (m = dev->dma_range_map; m->size; m++)
+		if (paddr >= m->cpu_start && paddr - m->cpu_start < m->size)
+			return (dma_addr_t)paddr - m->offset;
+
+	/* make sure dma_capable fails when no translation is available */
+	return DMA_MAPPING_ERROR; 
 }
 
-static inline u64 dma_offset_from_phys_addr(struct device *dev,
-		phys_addr_t paddr)
+static inline phys_addr_t translate_dma_to_phys(struct device *dev,
+		dma_addr_t dma_addr)
 {
-	const struct bus_dma_region *m = dev->dma_range_map;
+	const struct bus_dma_region *m;
+
+	for (m = dev->dma_range_map; m->size; m++)
+		if (dma_addr >= m->dma_start && dma_addr - m->dma_start < m->size)
+			return (phys_addr_t)dma_addr + m->offset;
 
-	if (m)
-		for (; m->size; m++)
-			if (paddr >= m->cpu_start &&
-			    paddr - m->cpu_start < m->size)
-				return m->offset;
 	return 0;
 }
 
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

On Tue, Sep 08, 2020 at 09:29:35AM +0200, Christoph Hellwig wrote:
> FYI, this is what I'd do relative to the patch on the dma-ranges
> branch.  In fact realizing this makes me want to refactor things a bit
> so that the new code can entirely live in the dma-direct code, but please
> test this first:

And of course this isn't going to work for arm devices without any
range, so let's try this instead:

diff --git a/arch/arm/include/asm/dma-mapping.h b/arch/arm/include/asm/dma-mapping.h
index c21893f683b585..e913e04d2be8b9 100644
--- a/arch/arm/include/asm/dma-mapping.h
+++ b/arch/arm/include/asm/dma-mapping.h
@@ -35,21 +35,20 @@ static inline const struct dma_map_ops *get_arch_dma_ops(struct bus_type *bus)
 #ifndef __arch_pfn_to_dma
 static inline dma_addr_t pfn_to_dma(struct device *dev, unsigned long pfn)
 {
-	if (dev) {
-		phys_addr_t paddr = PFN_PHYS(pfn);
-
-		pfn -= PFN_DOWN(dma_offset_from_phys_addr(dev, paddr));
-	}
-	return (dma_addr_t)__pfn_to_bus(pfn);
+	if (!dev)
+		return (dma_addr_t)__pfn_to_bus(pfn);
+	if (dev->dma_range_map)
+		return translate_phys_to_dma(dev, PFN_PHYS(pfn));
+	return (dma_addr_t)PFN_PHYS(pfn);
 }
 
 static inline unsigned long dma_to_pfn(struct device *dev, dma_addr_t addr)
 {
-	unsigned long pfn = __bus_to_pfn(addr);
-
-	if (dev)
-		pfn += PFN_DOWN(dma_offset_from_dma_addr(dev, addr));
-	return pfn;
+	if (!dev)
+		return __bus_to_pfn(addr);
+	if (dev->dma_range_map)
+		return PFN_DOWN(translate_dma_to_phys(dev, addr));
+	return PFN_DOWN(addr);
 }
 
 static inline void *dma_to_virt(struct device *dev, dma_addr_t addr)
diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index 7831ca5b1b5dd6..e624171c4962ad 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -19,12 +19,16 @@ extern unsigned int zone_dma_bits;
 #else
 static inline dma_addr_t __phys_to_dma(struct device *dev, phys_addr_t paddr)
 {
-	return (dma_addr_t)paddr - dma_offset_from_phys_addr(dev, paddr);
+	if (dev->dma_range_map)
+		return (dma_addr_t)paddr - translate_phys_to_dma(dev, paddr);
+	return (dma_addr_t)paddr;
 }
 
-static inline phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dev_addr)
+static inline phys_addr_t __dma_to_phys(struct device *dev, dma_addr_t dma_addr)
 {
-	return (phys_addr_t)dev_addr + dma_offset_from_dma_addr(dev, dev_addr);
+	if (dev->dma_range_map)
+		return translate_dma_to_phys(dev, dma_addr);
+	return (phys_addr_t)dma_addr;
 }
 #endif /* !CONFIG_ARCH_HAS_PHYS_TO_DMA */
 
diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
index 4c4646761afee4..3b1ceebb6f2ad5 100644
--- a/include/linux/dma-mapping.h
+++ b/include/linux/dma-mapping.h
@@ -199,29 +199,28 @@ struct bus_dma_region {
 };
 
 #ifdef CONFIG_HAS_DMA
-static inline u64 dma_offset_from_dma_addr(struct device *dev,
-		dma_addr_t dma_addr)
+static inline dma_addr_t translate_phys_to_dma(struct device *dev,
+		phys_addr_t paddr)
 {
-	const struct bus_dma_region *m = dev->dma_range_map;
+	const struct bus_dma_region *m;
 
-	if (m)
-		for (; m->size; m++)
-			if (dma_addr >= m->dma_start &&
-			    dma_addr - m->dma_start < m->size)
-				return m->offset;
-	return 0;
+	for (m = dev->dma_range_map; m->size; m++)
+		if (paddr >= m->cpu_start && paddr - m->cpu_start < m->size)
+			return (dma_addr_t)paddr - m->offset;
+
+	/* make sure dma_capable fails when no translation is available */
+	return DMA_MAPPING_ERROR; 
 }
 
-static inline u64 dma_offset_from_phys_addr(struct device *dev,
-		phys_addr_t paddr)
+static inline phys_addr_t translate_dma_to_phys(struct device *dev,
+		dma_addr_t dma_addr)
 {
-	const struct bus_dma_region *m = dev->dma_range_map;
+	const struct bus_dma_region *m;
+
+	for (m = dev->dma_range_map; m->size; m++)
+		if (dma_addr >= m->dma_start && dma_addr - m->dma_start < m->size)
+			return (phys_addr_t)dma_addr + m->offset;
 
-	if (m)
-		for (; m->size; m++)
-			if (paddr >= m->cpu_start &&
-			    paddr - m->cpu_start < m->size)
-				return m->offset;
 	return 0;
 }
 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

And because I like replying to myself so much, here is a link to the
version with the arm cleanup patch applied.  Unlike the previous two
attempts this has at least survived very basic sanity testing:

http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2

Note that we'll still need to sort out the arm/keystone warnings from
the original patch.  Do we have anyone on the CC list who knows that
platform a little better to figure out if the ifdef solution would work?
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

And because I like replying to myself so much, here is a link to the
version with the arm cleanup patch applied.  Unlike the previous two
attempts this has at least survived very basic sanity testing:

http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2

Note that we'll still need to sort out the arm/keystone warnings from
the original patch.  Do we have anyone on the CC list who knows that
platform a little better to figure out if the ifdef solution would work?
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Mon, Sep 07, 2020 at 11:29:06AM -0700, Florian Fainelli wrote:
> 
> 
> On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> > On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com> wrote:
> > > 
> > > On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > > > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > > > > 
> > > > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > > > > > 
> > > > > > > Patchset Summary:
> > > > > > >     Enhance a PCIe host controller driver.  Because of its unusual design
> > > > > > >     we are foced to change dev->dma_pfn_offset into a more general role
> > > > > > >     allowing multiple offsets.  See the 'v1' notes below for more info.
> > > > > > 
> > > > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > > > 5.10 merge window.
> > > > > > 
> > > > > > There are a lot of different files being touched, so what would be the
> > > > > > ideal way of routing those changes towards inclusion?
> > > > > 
> > > > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > > > apply the current version.
> > > > Sounds good to me.
> > > 
> > > Hi Jim,
> > > 
> > > is the dependency now solved ? Should we review/take this series as
> > > is for v5.10 through the PCI tree ?
> > Hello Lorenzo,
> > 
> > We are still working out a regression with the DMA offset commit on
> > the RaspberryPi.  Nicolas has found the root cause and we are now
> > devising a solution.
> 
> Maybe we can parallelize the PCIe driver review while the DMA changes
> are being worked on in Christoph's branch. Lorenzo, are you fine with
> the PCIe changes proper?

I will have a look - the main contentious point was about the DMA
changes - if Christoph is happy with them I am OK with them
too - I hope there is not anything controversial in the host
bridge driver itself but I will look into it.

Lorenzo

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Mon, Sep 07, 2020 at 11:29:06AM -0700, Florian Fainelli wrote:
> 
> 
> On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> > On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com> wrote:
> > > 
> > > On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > > > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > > > > 
> > > > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > > > > > 
> > > > > > > Patchset Summary:
> > > > > > >     Enhance a PCIe host controller driver.  Because of its unusual design
> > > > > > >     we are foced to change dev->dma_pfn_offset into a more general role
> > > > > > >     allowing multiple offsets.  See the 'v1' notes below for more info.
> > > > > > 
> > > > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > > > 5.10 merge window.
> > > > > > 
> > > > > > There are a lot of different files being touched, so what would be the
> > > > > > ideal way of routing those changes towards inclusion?
> > > > > 
> > > > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > > > apply the current version.
> > > > Sounds good to me.
> > > 
> > > Hi Jim,
> > > 
> > > is the dependency now solved ? Should we review/take this series as
> > > is for v5.10 through the PCI tree ?
> > Hello Lorenzo,
> > 
> > We are still working out a regression with the DMA offset commit on
> > the RaspberryPi.  Nicolas has found the root cause and we are now
> > devising a solution.
> 
> Maybe we can parallelize the PCIe driver review while the DMA changes
> are being worked on in Christoph's branch. Lorenzo, are you fine with
> the PCIe changes proper?

I will have a look - the main contentious point was about the DMA
changes - if Christoph is happy with them I am OK with them
too - I hope there is not anything controversial in the host
bridge driver itself but I will look into it.

Lorenzo

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Mon, Sep 07, 2020 at 11:29:06AM -0700, Florian Fainelli wrote:
> 
> 
> On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> > On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com> wrote:
> > > 
> > > On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > > > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > > > > 
> > > > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > > > > > 
> > > > > > > Patchset Summary:
> > > > > > >     Enhance a PCIe host controller driver.  Because of its unusual design
> > > > > > >     we are foced to change dev->dma_pfn_offset into a more general role
> > > > > > >     allowing multiple offsets.  See the 'v1' notes below for more info.
> > > > > > 
> > > > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > > > 5.10 merge window.
> > > > > > 
> > > > > > There are a lot of different files being touched, so what would be the
> > > > > > ideal way of routing those changes towards inclusion?
> > > > > 
> > > > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > > > apply the current version.
> > > > Sounds good to me.
> > > 
> > > Hi Jim,
> > > 
> > > is the dependency now solved ? Should we review/take this series as
> > > is for v5.10 through the PCI tree ?
> > Hello Lorenzo,
> > 
> > We are still working out a regression with the DMA offset commit on
> > the RaspberryPi.  Nicolas has found the root cause and we are now
> > devising a solution.
> 
> Maybe we can parallelize the PCIe driver review while the DMA changes
> are being worked on in Christoph's branch. Lorenzo, are you fine with
> the PCIe changes proper?

I will have a look - the main contentious point was about the DMA
changes - if Christoph is happy with them I am OK with them
too - I hope there is not anything controversial in the host
bridge driver itself but I will look into it.

Lorenzo
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Mon, Sep 07, 2020 at 11:29:06AM -0700, Florian Fainelli wrote:
> 
> 
> On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> > On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com> wrote:
> > > 
> > > On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > > > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > > > > 
> > > > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > > > > > 
> > > > > > > Patchset Summary:
> > > > > > >     Enhance a PCIe host controller driver.  Because of its unusual design
> > > > > > >     we are foced to change dev->dma_pfn_offset into a more general role
> > > > > > >     allowing multiple offsets.  See the 'v1' notes below for more info.
> > > > > > 
> > > > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > > > 5.10 merge window.
> > > > > > 
> > > > > > There are a lot of different files being touched, so what would be the
> > > > > > ideal way of routing those changes towards inclusion?
> > > > > 
> > > > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > > > apply the current version.
> > > > Sounds good to me.
> > > 
> > > Hi Jim,
> > > 
> > > is the dependency now solved ? Should we review/take this series as
> > > is for v5.10 through the PCI tree ?
> > Hello Lorenzo,
> > 
> > We are still working out a regression with the DMA offset commit on
> > the RaspberryPi.  Nicolas has found the root cause and we are now
> > devising a solution.
> 
> Maybe we can parallelize the PCIe driver review while the DMA changes
> are being worked on in Christoph's branch. Lorenzo, are you fine with
> the PCIe changes proper?

I will have a look - the main contentious point was about the DMA
changes - if Christoph is happy with them I am OK with them
too - I hope there is not anything controversial in the host
bridge driver itself but I will look into it.

Lorenzo
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Mon, Sep 07, 2020 at 11:29:06AM -0700, Florian Fainelli wrote:
> 
> 
> On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> > On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com> wrote:
> > > 
> > > On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > > > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > > > > 
> > > > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > > > > > 
> > > > > > > Patchset Summary:
> > > > > > >     Enhance a PCIe host controller driver.  Because of its unusual design
> > > > > > >     we are foced to change dev->dma_pfn_offset into a more general role
> > > > > > >     allowing multiple offsets.  See the 'v1' notes below for more info.
> > > > > > 
> > > > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > > > 5.10 merge window.
> > > > > > 
> > > > > > There are a lot of different files being touched, so what would be the
> > > > > > ideal way of routing those changes towards inclusion?
> > > > > 
> > > > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > > > apply the current version.
> > > > Sounds good to me.
> > > 
> > > Hi Jim,
> > > 
> > > is the dependency now solved ? Should we review/take this series as
> > > is for v5.10 through the PCI tree ?
> > Hello Lorenzo,
> > 
> > We are still working out a regression with the DMA offset commit on
> > the RaspberryPi.  Nicolas has found the root cause and we are now
> > devising a solution.
> 
> Maybe we can parallelize the PCIe driver review while the DMA changes
> are being worked on in Christoph's branch. Lorenzo, are you fine with
> the PCIe changes proper?

I will have a look - the main contentious point was about the DMA
changes - if Christoph is happy with them I am OK with them
too - I hope there is not anything controversial in the host
bridge driver itself but I will look into it.

Lorenzo

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Lorenzo Pieralisi]

On Mon, Sep 07, 2020 at 11:29:06AM -0700, Florian Fainelli wrote:
> 
> 
> On 9/7/2020 10:43 AM, Jim Quinlan wrote:
> > On Mon, Sep 7, 2020 at 5:16 AM Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com> wrote:
> > > 
> > > On Thu, Aug 27, 2020 at 09:29:59AM -0400, Jim Quinlan wrote:
> > > > On Thu, Aug 27, 2020 at 2:35 AM Christoph Hellwig <hch@lst.de> wrote:
> > > > > 
> > > > > On Tue, Aug 25, 2020 at 10:40:27AM -0700, Florian Fainelli wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > On 8/24/2020 12:30 PM, Jim Quinlan wrote:
> > > > > > > 
> > > > > > > Patchset Summary:
> > > > > > >     Enhance a PCIe host controller driver.  Because of its unusual design
> > > > > > >     we are foced to change dev->dma_pfn_offset into a more general role
> > > > > > >     allowing multiple offsets.  See the 'v1' notes below for more info.
> > > > > > 
> > > > > > We are version 11 and counting, and it is not clear to me whether there is
> > > > > > any chance of getting these patches reviewed and hopefully merged for the
> > > > > > 5.10 merge window.
> > > > > > 
> > > > > > There are a lot of different files being touched, so what would be the
> > > > > > ideal way of routing those changes towards inclusion?
> > > > > 
> > > > > FYI, I offered to take the dma-mapping bits through the dma-mapping tree.
> > > > > I have a bit of a backlog, but plan to review and if Jim is ok with that
> > > > > apply the current version.
> > > > Sounds good to me.
> > > 
> > > Hi Jim,
> > > 
> > > is the dependency now solved ? Should we review/take this series as
> > > is for v5.10 through the PCI tree ?
> > Hello Lorenzo,
> > 
> > We are still working out a regression with the DMA offset commit on
> > the RaspberryPi.  Nicolas has found the root cause and we are now
> > devising a solution.
> 
> Maybe we can parallelize the PCIe driver review while the DMA changes
> are being worked on in Christoph's branch. Lorenzo, are you fine with
> the PCIe changes proper?

I will have a look - the main contentious point was about the DMA
changes - if Christoph is happy with them I am OK with them
too - I hope there is not anything controversial in the host
bridge driver itself but I will look into it.

Lorenzo
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 1787 bytes --]

On Tue, 2020-09-08 at 11:43 +0200, Christoph Hellwig wrote:
> And because I like replying to myself so much, here is a link to the
> version with the arm cleanup patch applied.  Unlike the previous two
> attempts this has at least survived very basic sanity testing:
> 
> http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2
> 
> Note that we'll still need to sort out the arm/keystone warnings from
> the original patch.  Do we have anyone on the CC list who knows that
> platform a little better to figure out if the ifdef solution would work?

Had to do the following to boot without errors:

diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index ef61a33c47bc..7dd88a0b6d0b 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -97,6 +97,9 @@ static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size,
 {
        dma_addr_t end = addr + size - 1;
 
+       if (addr == DMA_MAPPING_ERROR)
+               return false;
+
        if (!dev->dma_mask)
                return false;
 
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 90f1ecb6baaf..25809703a5bf 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -71,7 +71,12 @@ static gfp_t dma_direct_optimal_gfp_mask(struct device *dev, u64 dma_mask,
 
 static bool dma_coherent_ok(struct device *dev, phys_addr_t phys, size_t size)
 {
-       return phys_to_dma_direct(dev, phys) + size - 1 <=
+       dma_addr_t dma_addr = phys_to_dma_direct(dev, phys);
+
+       if (dma_addr == DMA_MAPPING_ERROR)
+               return false;
+
+       return dma_addr + size - 1 <=
                        min_not_zero(dev->coherent_dma_mask, dev->bus_dma_limit);
 }


Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 169 bytes --]

_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 1787 bytes --]

On Tue, 2020-09-08 at 11:43 +0200, Christoph Hellwig wrote:
> And because I like replying to myself so much, here is a link to the
> version with the arm cleanup patch applied.  Unlike the previous two
> attempts this has at least survived very basic sanity testing:
> 
> http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2
> 
> Note that we'll still need to sort out the arm/keystone warnings from
> the original patch.  Do we have anyone on the CC list who knows that
> platform a little better to figure out if the ifdef solution would work?

Had to do the following to boot without errors:

diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index ef61a33c47bc..7dd88a0b6d0b 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -97,6 +97,9 @@ static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size,
 {
        dma_addr_t end = addr + size - 1;
 
+       if (addr == DMA_MAPPING_ERROR)
+               return false;
+
        if (!dev->dma_mask)
                return false;
 
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 90f1ecb6baaf..25809703a5bf 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -71,7 +71,12 @@ static gfp_t dma_direct_optimal_gfp_mask(struct device *dev, u64 dma_mask,
 
 static bool dma_coherent_ok(struct device *dev, phys_addr_t phys, size_t size)
 {
-       return phys_to_dma_direct(dev, phys) + size - 1 <=
+       dma_addr_t dma_addr = phys_to_dma_direct(dev, phys);
+
+       if (dma_addr == DMA_MAPPING_ERROR)
+               return false;
+
+       return dma_addr + size - 1 <=
                        min_not_zero(dev->coherent_dma_mask, dev->bus_dma_limit);
 }


Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nicolas Saenz Julienne]

[-- Attachment #1.1: Type: text/plain, Size: 1787 bytes --]

On Tue, 2020-09-08 at 11:43 +0200, Christoph Hellwig wrote:
> And because I like replying to myself so much, here is a link to the
> version with the arm cleanup patch applied.  Unlike the previous two
> attempts this has at least survived very basic sanity testing:
> 
> http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2
> 
> Note that we'll still need to sort out the arm/keystone warnings from
> the original patch.  Do we have anyone on the CC list who knows that
> platform a little better to figure out if the ifdef solution would work?

Had to do the following to boot without errors:

diff --git a/include/linux/dma-direct.h b/include/linux/dma-direct.h
index ef61a33c47bc..7dd88a0b6d0b 100644
--- a/include/linux/dma-direct.h
+++ b/include/linux/dma-direct.h
@@ -97,6 +97,9 @@ static inline bool dma_capable(struct device *dev, dma_addr_t addr, size_t size,
 {
        dma_addr_t end = addr + size - 1;
 
+       if (addr == DMA_MAPPING_ERROR)
+               return false;
+
        if (!dev->dma_mask)
                return false;
 
diff --git a/kernel/dma/direct.c b/kernel/dma/direct.c
index 90f1ecb6baaf..25809703a5bf 100644
--- a/kernel/dma/direct.c
+++ b/kernel/dma/direct.c
@@ -71,7 +71,12 @@ static gfp_t dma_direct_optimal_gfp_mask(struct device *dev, u64 dma_mask,
 
 static bool dma_coherent_ok(struct device *dev, phys_addr_t phys, size_t size)
 {
-       return phys_to_dma_direct(dev, phys) + size - 1 <=
+       dma_addr_t dma_addr = phys_to_dma_direct(dev, phys);
+
+       if (dma_addr == DMA_MAPPING_ERROR)
+               return false;
+
+       return dma_addr + size - 1 <=
                        min_not_zero(dev->coherent_dma_mask, dev->bus_dma_limit);
 }


Regards,
Nicolas


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Christoph Hellwig]

On Tue, Sep 08, 2020 at 11:42:26AM +0100, Lorenzo Pieralisi wrote:
> > Maybe we can parallelize the PCIe driver review while the DMA changes
> > are being worked on in Christoph's branch. Lorenzo, are you fine with
> > the PCIe changes proper?
> 
> I will have a look - the main contentious point was about the DMA
> changes - if Christoph is happy with them I am OK with them
> too - I hope there is not anything controversial in the host
> bridge driver itself but I will look into it.

I'm pretty happy with the overall shape.  Now we just need to squeeze
out the regressions..

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Christoph Hellwig]

On Tue, Sep 08, 2020 at 11:42:26AM +0100, Lorenzo Pieralisi wrote:
> > Maybe we can parallelize the PCIe driver review while the DMA changes
> > are being worked on in Christoph's branch. Lorenzo, are you fine with
> > the PCIe changes proper?
> 
> I will have a look - the main contentious point was about the DMA
> changes - if Christoph is happy with them I am OK with them
> too - I hope there is not anything controversial in the host
> bridge driver itself but I will look into it.

I'm pretty happy with the overall shape.  Now we just need to squeeze
out the regressions..

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Christoph Hellwig]

On Tue, Sep 08, 2020 at 11:42:26AM +0100, Lorenzo Pieralisi wrote:
> > Maybe we can parallelize the PCIe driver review while the DMA changes
> > are being worked on in Christoph's branch. Lorenzo, are you fine with
> > the PCIe changes proper?
> 
> I will have a look - the main contentious point was about the DMA
> changes - if Christoph is happy with them I am OK with them
> too - I hope there is not anything controversial in the host
> bridge driver itself but I will look into it.

I'm pretty happy with the overall shape.  Now we just need to squeeze
out the regressions..
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Christoph Hellwig]

On Tue, Sep 08, 2020 at 11:42:26AM +0100, Lorenzo Pieralisi wrote:
> > Maybe we can parallelize the PCIe driver review while the DMA changes
> > are being worked on in Christoph's branch. Lorenzo, are you fine with
> > the PCIe changes proper?
> 
> I will have a look - the main contentious point was about the DMA
> changes - if Christoph is happy with them I am OK with them
> too - I hope there is not anything controversial in the host
> bridge driver itself but I will look into it.

I'm pretty happy with the overall shape.  Now we just need to squeeze
out the regressions..
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 00/11] PCI: brcmstb: enable PCIe for STB chips

[Christoph Hellwig]

On Tue, Sep 08, 2020 at 11:42:26AM +0100, Lorenzo Pieralisi wrote:
> > Maybe we can parallelize the PCIe driver review while the DMA changes
> > are being worked on in Christoph's branch. Lorenzo, are you fine with
> > the PCIe changes proper?
> 
> I will have a look - the main contentious point was about the DMA
> changes - if Christoph is happy with them I am OK with them
> too - I hope there is not anything controversial in the host
> bridge driver itself but I will look into it.

I'm pretty happy with the overall shape.  Now we just need to squeeze
out the regressions..

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

On Tue, Sep 08, 2020 at 01:20:56PM +0200, Nicolas Saenz Julienne wrote:
> On Tue, 2020-09-08 at 11:43 +0200, Christoph Hellwig wrote:
> > And because I like replying to myself so much, here is a link to the
> > version with the arm cleanup patch applied.  Unlike the previous two
> > attempts this has at least survived very basic sanity testing:
> > 
> > http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2
> > 
> > Note that we'll still need to sort out the arm/keystone warnings from
> > the original patch.  Do we have anyone on the CC list who knows that
> > platform a little better to figure out if the ifdef solution would work?
> 
> Had to do the following to boot without errors:

I've folded in.  That being said the whole RPi4 setup confuses the
heck out of me.  I wonder what thing was smoked to come up with it..
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Christoph Hellwig]

On Tue, Sep 08, 2020 at 01:20:56PM +0200, Nicolas Saenz Julienne wrote:
> On Tue, 2020-09-08 at 11:43 +0200, Christoph Hellwig wrote:
> > And because I like replying to myself so much, here is a link to the
> > version with the arm cleanup patch applied.  Unlike the previous two
> > attempts this has at least survived very basic sanity testing:
> > 
> > http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2
> > 
> > Note that we'll still need to sort out the arm/keystone warnings from
> > the original patch.  Do we have anyone on the CC list who knows that
> > platform a little better to figure out if the ifdef solution would work?
> 
> Had to do the following to boot without errors:

I've folded in.  That being said the whole RPi4 setup confuses the
heck out of me.  I wonder what thing was smoked to come up with it..
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 06/11] PCI: brcmstb: Add control of rescal reset

[Lorenzo Pieralisi]

On Mon, Aug 24, 2020 at 03:30:19PM -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> Some STB chips have a special purpose reset controller named RESCAL (reset
> calibration).  The PCIe HW can now control RESCAL to start and stop its
> operation.  On probe(), the RESCAL is deasserted and the driver goes
> through the sequence of setting registers and reading status in order to
> start the internal PHY that is required for the PCIe.
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 82 ++++++++++++++++++++++++++-
>  1 file changed, 81 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index acf2239b0251..041b8d109563 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -23,6 +23,7 @@
>  #include <linux/of_platform.h>
>  #include <linux/pci.h>
>  #include <linux/printk.h>
> +#include <linux/reset.h>
>  #include <linux/sizes.h>
>  #include <linux/slab.h>
>  #include <linux/string.h>
> @@ -158,6 +159,16 @@
>  #define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
>  #define PCIE_RGR1_SW_INIT_1(pcie)	(pcie->reg_offsets[RGR1_SW_INIT_1])
>  
> +/* Rescal registers */
> +#define PCIE_DVT_PMU_PCIE_PHY_CTRL				0xc700
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS			0x3
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK		0x4
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT	0x2
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK		0x2
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT		0x1
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK		0x1
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT		0x0
> +
>  enum {
>  	RGR1_SW_INIT_1,
>  	EXT_CFG_INDEX,
> @@ -247,6 +258,7 @@ struct brcm_pcie {
>  	const int		*reg_offsets;
>  	const int		*reg_field_info;
>  	enum pcie_type		type;
> +	struct reset_control	*rescal;
>  };
>  
>  /*
> @@ -965,6 +977,47 @@ static void brcm_pcie_enter_l23(struct brcm_pcie *pcie)
>  		dev_err(pcie->dev, "failed to enter low-power link state\n");
>  }
>  
> +static int brcm_phy_cntl(struct brcm_pcie *pcie, const int start)
> +{
> +	static const u32 shifts[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT,};
> +	static const u32 masks[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK,};
> +	const int beg = start ? 0 : PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS - 1;
> +	const int end = start ? PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS : -1;
> +	u32 tmp, combined_mask = 0;
> +	u32 val = !!start;

Quick note: I think it would be nicer to init val within the loop
below (IIUC how this works):

val = start ? BIT_MASK(shifts[i]) : 0;

and then use it in the tmp computation but I leave it up to you.

Lorenzo

> +	void __iomem *base = pcie->base;
> +	int i;
> +
> +	for (i = beg; i != end; start ? i++ : i--) {
> +		tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +		tmp = (tmp & ~masks[i]) | ((val << shifts[i]) & masks[i]);
> +		writel(tmp, base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +		usleep_range(50, 200);
> +		combined_mask |= masks[i];
> +	}
> +
> +	tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +	val = start ? combined_mask : 0;
> +
> +	return (tmp & combined_mask) == val ? 0 : -EIO;
> +}
> +
> +static inline int brcm_phy_start(struct brcm_pcie *pcie)
> +{
> +	return pcie->rescal ? brcm_phy_cntl(pcie, 1) : 0;
> +}
> +
> +static inline int brcm_phy_stop(struct brcm_pcie *pcie)
> +{
> +	return pcie->rescal ? brcm_phy_cntl(pcie, 0) : 0;
> +}
> +
>  static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>  {
>  	void __iomem *base = pcie->base;
> @@ -992,11 +1045,15 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>  static int brcm_pcie_suspend(struct device *dev)
>  {
>  	struct brcm_pcie *pcie = dev_get_drvdata(dev);
> +	int ret;
>  
>  	brcm_pcie_turn_off(pcie);
> +	ret = brcm_phy_stop(pcie);
> +	if (ret)
> +		dev_err(pcie->dev, "failed to stop phy\n");
>  	clk_disable_unprepare(pcie->clk);
>  
> -	return 0;
> +	return ret;
>  }
>  
>  static int brcm_pcie_resume(struct device *dev)
> @@ -1009,6 +1066,12 @@ static int brcm_pcie_resume(struct device *dev)
>  	base = pcie->base;
>  	clk_prepare_enable(pcie->clk);
>  
> +	ret = brcm_phy_start(pcie);
> +	if (ret) {
> +		dev_err(pcie->dev, "failed to start phy\n");
> +		return ret;
> +	}
> +
>  	/* Take bridge out of reset so we can access the SERDES reg */
>  	brcm_pcie_bridge_sw_init_set(pcie, 0);
>  
> @@ -1034,6 +1097,9 @@ static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>  {
>  	brcm_msi_remove(pcie);
>  	brcm_pcie_turn_off(pcie);
> +	if (brcm_phy_stop(pcie))
> +		dev_err(pcie->dev, "failed to stop phy\n");
> +	reset_control_assert(pcie->rescal);
>  	clk_disable_unprepare(pcie->clk);
>  }
>  
> @@ -1112,6 +1178,20 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>  		dev_err(&pdev->dev, "could not enable clock\n");
>  		return ret;
>  	}
> +	pcie->rescal = devm_reset_control_get_optional_shared(&pdev->dev, "rescal");
> +	if (IS_ERR(pcie->rescal))
> +		return PTR_ERR(pcie->rescal);
> +
> +	ret = reset_control_deassert(pcie->rescal);
> +	if (ret)
> +		dev_err(&pdev->dev, "failed to deassert 'rescal'\n");
> +
> +	ret = brcm_phy_start(pcie);
> +	if (ret) {
> +		dev_err(pcie->dev, "failed to start phy\n");
> +		reset_control_assert(pcie->rescal);
> +		return ret;
> +	}
>  
>  	ret = brcm_pcie_setup(pcie);
>  	if (ret)
> -- 
> 2.17.1
> 

Re: [PATCH v11 06/11] PCI: brcmstb: Add control of rescal reset

[Lorenzo Pieralisi]

On Mon, Aug 24, 2020 at 03:30:19PM -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> Some STB chips have a special purpose reset controller named RESCAL (reset
> calibration).  The PCIe HW can now control RESCAL to start and stop its
> operation.  On probe(), the RESCAL is deasserted and the driver goes
> through the sequence of setting registers and reading status in order to
> start the internal PHY that is required for the PCIe.
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 82 ++++++++++++++++++++++++++-
>  1 file changed, 81 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index acf2239b0251..041b8d109563 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -23,6 +23,7 @@
>  #include <linux/of_platform.h>
>  #include <linux/pci.h>
>  #include <linux/printk.h>
> +#include <linux/reset.h>
>  #include <linux/sizes.h>
>  #include <linux/slab.h>
>  #include <linux/string.h>
> @@ -158,6 +159,16 @@
>  #define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
>  #define PCIE_RGR1_SW_INIT_1(pcie)	(pcie->reg_offsets[RGR1_SW_INIT_1])
>  
> +/* Rescal registers */
> +#define PCIE_DVT_PMU_PCIE_PHY_CTRL				0xc700
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS			0x3
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK		0x4
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT	0x2
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK		0x2
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT		0x1
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK		0x1
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT		0x0
> +
>  enum {
>  	RGR1_SW_INIT_1,
>  	EXT_CFG_INDEX,
> @@ -247,6 +258,7 @@ struct brcm_pcie {
>  	const int		*reg_offsets;
>  	const int		*reg_field_info;
>  	enum pcie_type		type;
> +	struct reset_control	*rescal;
>  };
>  
>  /*
> @@ -965,6 +977,47 @@ static void brcm_pcie_enter_l23(struct brcm_pcie *pcie)
>  		dev_err(pcie->dev, "failed to enter low-power link state\n");
>  }
>  
> +static int brcm_phy_cntl(struct brcm_pcie *pcie, const int start)
> +{
> +	static const u32 shifts[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT,};
> +	static const u32 masks[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK,};
> +	const int beg = start ? 0 : PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS - 1;
> +	const int end = start ? PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS : -1;
> +	u32 tmp, combined_mask = 0;
> +	u32 val = !!start;

Quick note: I think it would be nicer to init val within the loop
below (IIUC how this works):

val = start ? BIT_MASK(shifts[i]) : 0;

and then use it in the tmp computation but I leave it up to you.

Lorenzo

> +	void __iomem *base = pcie->base;
> +	int i;
> +
> +	for (i = beg; i != end; start ? i++ : i--) {
> +		tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +		tmp = (tmp & ~masks[i]) | ((val << shifts[i]) & masks[i]);
> +		writel(tmp, base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +		usleep_range(50, 200);
> +		combined_mask |= masks[i];
> +	}
> +
> +	tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +	val = start ? combined_mask : 0;
> +
> +	return (tmp & combined_mask) == val ? 0 : -EIO;
> +}
> +
> +static inline int brcm_phy_start(struct brcm_pcie *pcie)
> +{
> +	return pcie->rescal ? brcm_phy_cntl(pcie, 1) : 0;
> +}
> +
> +static inline int brcm_phy_stop(struct brcm_pcie *pcie)
> +{
> +	return pcie->rescal ? brcm_phy_cntl(pcie, 0) : 0;
> +}
> +
>  static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>  {
>  	void __iomem *base = pcie->base;
> @@ -992,11 +1045,15 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>  static int brcm_pcie_suspend(struct device *dev)
>  {
>  	struct brcm_pcie *pcie = dev_get_drvdata(dev);
> +	int ret;
>  
>  	brcm_pcie_turn_off(pcie);
> +	ret = brcm_phy_stop(pcie);
> +	if (ret)
> +		dev_err(pcie->dev, "failed to stop phy\n");
>  	clk_disable_unprepare(pcie->clk);
>  
> -	return 0;
> +	return ret;
>  }
>  
>  static int brcm_pcie_resume(struct device *dev)
> @@ -1009,6 +1066,12 @@ static int brcm_pcie_resume(struct device *dev)
>  	base = pcie->base;
>  	clk_prepare_enable(pcie->clk);
>  
> +	ret = brcm_phy_start(pcie);
> +	if (ret) {
> +		dev_err(pcie->dev, "failed to start phy\n");
> +		return ret;
> +	}
> +
>  	/* Take bridge out of reset so we can access the SERDES reg */
>  	brcm_pcie_bridge_sw_init_set(pcie, 0);
>  
> @@ -1034,6 +1097,9 @@ static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>  {
>  	brcm_msi_remove(pcie);
>  	brcm_pcie_turn_off(pcie);
> +	if (brcm_phy_stop(pcie))
> +		dev_err(pcie->dev, "failed to stop phy\n");
> +	reset_control_assert(pcie->rescal);
>  	clk_disable_unprepare(pcie->clk);
>  }
>  
> @@ -1112,6 +1178,20 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>  		dev_err(&pdev->dev, "could not enable clock\n");
>  		return ret;
>  	}
> +	pcie->rescal = devm_reset_control_get_optional_shared(&pdev->dev, "rescal");
> +	if (IS_ERR(pcie->rescal))
> +		return PTR_ERR(pcie->rescal);
> +
> +	ret = reset_control_deassert(pcie->rescal);
> +	if (ret)
> +		dev_err(&pdev->dev, "failed to deassert 'rescal'\n");
> +
> +	ret = brcm_phy_start(pcie);
> +	if (ret) {
> +		dev_err(pcie->dev, "failed to start phy\n");
> +		reset_control_assert(pcie->rescal);
> +		return ret;
> +	}
>  
>  	ret = brcm_pcie_setup(pcie);
>  	if (ret)
> -- 
> 2.17.1
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

On Tue, Sep 8, 2020 at 5:43 AM Christoph Hellwig <hch@lst.de> wrote:
>
> And because I like replying to myself so much, here is a link to the
> version with the arm cleanup patch applied.  Unlike the previous two
> attempts this has at least survived very basic sanity testing:
>
> http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2
>
Tested-by: Jim Quinlan <james.quinlan@broadcom.com>

> Note that we'll still need to sort out the arm/keystone warnings from
> the original patch.  Do we have anyone on the CC list who knows that
> platform a little better to figure out if the ifdef solution would work?
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan via iommu]

On Tue, Sep 8, 2020 at 5:43 AM Christoph Hellwig <hch@lst.de> wrote:
>
> And because I like replying to myself so much, here is a link to the
> version with the arm cleanup patch applied.  Unlike the previous two
> attempts this has at least survived very basic sanity testing:
>
> http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2
>
Tested-by: Jim Quinlan <james.quinlan@broadcom.com>

> Note that we'll still need to sort out the arm/keystone warnings from
> the original patch.  Do we have anyone on the CC list who knows that
> platform a little better to figure out if the ifdef solution would work?
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Jim Quinlan]

On Tue, Sep 8, 2020 at 5:43 AM Christoph Hellwig <hch@lst.de> wrote:
>
> And because I like replying to myself so much, here is a link to the
> version with the arm cleanup patch applied.  Unlike the previous two
> attempts this has at least survived very basic sanity testing:
>
> http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2
>
Tested-by: Jim Quinlan <james.quinlan@broadcom.com>

> Note that we'll still need to sort out the arm/keystone warnings from
> the original patch.  Do we have anyone on the CC list who knows that
> platform a little better to figure out if the ifdef solution would work?
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

On Tue, Sep 08, 2020 at 11:43:45AM +0200, Christoph Hellwig wrote:
> And because I like replying to myself so much, here is a link to the
> version with the arm cleanup patch applied.  Unlike the previous two
> attempts this has at least survived very basic sanity testing:
> 
> http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2

Tested-by: Nathan Chancellor <natechancellor@gmail.com>

Thank you for fixing this up quickly!
_______________________________________________
devel mailing list
devel@linuxdriverproject.org
http://driverdev.linuxdriverproject.org/mailman/listinfo/driverdev-devel

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

On Tue, Sep 08, 2020 at 11:43:45AM +0200, Christoph Hellwig wrote:
> And because I like replying to myself so much, here is a link to the
> version with the arm cleanup patch applied.  Unlike the previous two
> attempts this has at least survived very basic sanity testing:
> 
> http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2

Tested-by: Nathan Chancellor <natechancellor@gmail.com>

Thank you for fixing this up quickly!
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v11 07/11] device-mapping: Introduce DMA range map, supplanting dma_pfn_offset

[Nathan Chancellor]

On Tue, Sep 08, 2020 at 11:43:45AM +0200, Christoph Hellwig wrote:
> And because I like replying to myself so much, here is a link to the
> version with the arm cleanup patch applied.  Unlike the previous two
> attempts this has at least survived very basic sanity testing:
> 
> http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/dma-ranges.2

Tested-by: Nathan Chancellor <natechancellor@gmail.com>

Thank you for fixing this up quickly!
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [PATCH v11 03/11] PCI: brcmstb: Add bcm7278 register info

[Rob Herring]

On Mon, Aug 24, 2020 at 03:30:16PM -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> Add in compatibility strings and code for three Broadcom STB chips.  Some
> of the register locations, shifts, and masks are different for certain
> chips, requiring the use of different constants based on of_id.
> 
> We would like to add the following at this time to the match list but we
> need to wait until the end of this patchset so that everything works.
> 
>     { .compatible = "brcm,bcm7211-pcie", .data = &generic_cfg },
>     { .compatible = "brcm,bcm7278-pcie", .data = &bcm7278_cfg },
>     { .compatible = "brcm,bcm7216-pcie", .data = &bcm7278_cfg },
>     { .compatible = "brcm,bcm7445-pcie", .data = &generic_cfg },
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 105 +++++++++++++++++++++++---
>  1 file changed, 93 insertions(+), 12 deletions(-)

I think a better abstraction would be to have 2 versions of the 2 
functions that need to be different. But as-is is fine.

Acked-by: Rob Herring <robh@kernel.org>

Re: [PATCH v11 03/11] PCI: brcmstb: Add bcm7278 register info

[Rob Herring]

On Mon, Aug 24, 2020 at 03:30:16PM -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> Add in compatibility strings and code for three Broadcom STB chips.  Some
> of the register locations, shifts, and masks are different for certain
> chips, requiring the use of different constants based on of_id.
> 
> We would like to add the following at this time to the match list but we
> need to wait until the end of this patchset so that everything works.
> 
>     { .compatible = "brcm,bcm7211-pcie", .data = &generic_cfg },
>     { .compatible = "brcm,bcm7278-pcie", .data = &bcm7278_cfg },
>     { .compatible = "brcm,bcm7216-pcie", .data = &bcm7278_cfg },
>     { .compatible = "brcm,bcm7445-pcie", .data = &generic_cfg },
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 105 +++++++++++++++++++++++---
>  1 file changed, 93 insertions(+), 12 deletions(-)

I think a better abstraction would be to have 2 versions of the 2 
functions that need to be different. But as-is is fine.

Acked-by: Rob Herring <robh@kernel.org>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Rob Herring]

On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
> and resume.  Now the PCIe driver may do so as well.
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
>  1 file changed, 47 insertions(+)
> 
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index c2b3d2946a36..3d588ab7a6dd 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>  	brcm_pcie_bridge_sw_init_set(pcie, 1);
>  }
>  
> +static int brcm_pcie_suspend(struct device *dev)
> +{
> +	struct brcm_pcie *pcie = dev_get_drvdata(dev);
> +
> +	brcm_pcie_turn_off(pcie);
> +	clk_disable_unprepare(pcie->clk);
> +
> +	return 0;
> +}
> +
> +static int brcm_pcie_resume(struct device *dev)
> +{
> +	struct brcm_pcie *pcie = dev_get_drvdata(dev);
> +	void __iomem *base;
> +	u32 tmp;
> +	int ret;
> +
> +	base = pcie->base;
> +	clk_prepare_enable(pcie->clk);
> +
> +	/* Take bridge out of reset so we can access the SERDES reg */
> +	brcm_pcie_bridge_sw_init_set(pcie, 0);
> +
> +	/* SERDES_IDDQ = 0 */
> +	tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> +	u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
> +	writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> +
> +	/* wait for serdes to be stable */
> +	udelay(100);

Really needs to be a spinloop?

> +
> +	ret = brcm_pcie_setup(pcie);
> +	if (ret)
> +		return ret;
> +
> +	if (pcie->msi)
> +		brcm_msi_set_regs(pcie->msi);
> +
> +	return 0;
> +}
> +
>  static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>  {
>  	brcm_msi_remove(pcie);
> @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>  
>  MODULE_DEVICE_TABLE(of, brcm_pcie_match);
>  
> +static const struct dev_pm_ops brcm_pcie_pm_ops = {
> +	.suspend_noirq = brcm_pcie_suspend,
> +	.resume_noirq = brcm_pcie_resume,

Why do you need interrupts disabled? There's 39 cases of .suspend_noirq 
and 1352 of .suspend in the tree.

Is doing a clk unprepare even safe in .suspend_noirq? IIRC, 
prepare/unprepare can sleep.

> +};
> +
>  static struct platform_driver brcm_pcie_driver = {
>  	.probe = brcm_pcie_probe,
>  	.remove = brcm_pcie_remove,
>  	.driver = {
>  		.name = "brcm-pcie",
>  		.of_match_table = brcm_pcie_match,
> +		.pm = &brcm_pcie_pm_ops,
>  	},
>  };
>  module_platform_driver(brcm_pcie_driver);
> -- 
> 2.17.1
> 

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Rob Herring]

On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
> and resume.  Now the PCIe driver may do so as well.
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
>  1 file changed, 47 insertions(+)
> 
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index c2b3d2946a36..3d588ab7a6dd 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>  	brcm_pcie_bridge_sw_init_set(pcie, 1);
>  }
>  
> +static int brcm_pcie_suspend(struct device *dev)
> +{
> +	struct brcm_pcie *pcie = dev_get_drvdata(dev);
> +
> +	brcm_pcie_turn_off(pcie);
> +	clk_disable_unprepare(pcie->clk);
> +
> +	return 0;
> +}
> +
> +static int brcm_pcie_resume(struct device *dev)
> +{
> +	struct brcm_pcie *pcie = dev_get_drvdata(dev);
> +	void __iomem *base;
> +	u32 tmp;
> +	int ret;
> +
> +	base = pcie->base;
> +	clk_prepare_enable(pcie->clk);
> +
> +	/* Take bridge out of reset so we can access the SERDES reg */
> +	brcm_pcie_bridge_sw_init_set(pcie, 0);
> +
> +	/* SERDES_IDDQ = 0 */
> +	tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> +	u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
> +	writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> +
> +	/* wait for serdes to be stable */
> +	udelay(100);

Really needs to be a spinloop?

> +
> +	ret = brcm_pcie_setup(pcie);
> +	if (ret)
> +		return ret;
> +
> +	if (pcie->msi)
> +		brcm_msi_set_regs(pcie->msi);
> +
> +	return 0;
> +}
> +
>  static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>  {
>  	brcm_msi_remove(pcie);
> @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>  
>  MODULE_DEVICE_TABLE(of, brcm_pcie_match);
>  
> +static const struct dev_pm_ops brcm_pcie_pm_ops = {
> +	.suspend_noirq = brcm_pcie_suspend,
> +	.resume_noirq = brcm_pcie_resume,

Why do you need interrupts disabled? There's 39 cases of .suspend_noirq 
and 1352 of .suspend in the tree.

Is doing a clk unprepare even safe in .suspend_noirq? IIRC, 
prepare/unprepare can sleep.

> +};
> +
>  static struct platform_driver brcm_pcie_driver = {
>  	.probe = brcm_pcie_probe,
>  	.remove = brcm_pcie_remove,
>  	.driver = {
>  		.name = "brcm-pcie",
>  		.of_match_table = brcm_pcie_match,
> +		.pm = &brcm_pcie_pm_ops,
>  	},
>  };
>  module_platform_driver(brcm_pcie_driver);
> -- 
> 2.17.1
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 05/11] PCI: brcmstb: Add bcm7278 PERST# support

[Rob Herring]

On Mon, Aug 24, 2020 at 03:30:18PM -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> The PERST# bit was moved to a different register in 7278-type STB chips.
> In addition, the polarity of the bit was also changed; for other chips
> writing a 1 specified assert; for 7278-type chips, writing a 0 specifies
> assert.
> 
> Of course, PERST# is a PCIe asserted-low signal.
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 19 +++++++++++++++----
>  1 file changed, 15 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index 3d588ab7a6dd..acf2239b0251 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -83,6 +83,7 @@
>  
>  #define PCIE_MISC_PCIE_CTRL				0x4064
>  #define  PCIE_MISC_PCIE_CTRL_PCIE_L23_REQUEST_MASK	0x1
> +#define PCIE_MISC_PCIE_CTRL_PCIE_PERSTB_MASK		0x4
>  
>  #define PCIE_MISC_PCIE_STATUS				0x4068
>  #define  PCIE_MISC_PCIE_STATUS_PCIE_PORT_MASK		0x80
> @@ -684,9 +685,16 @@ static inline void brcm_pcie_perst_set(struct brcm_pcie *pcie, u32 val)
>  {
>  	u32 tmp;
>  
> -	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
> -	u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_PERST_MASK);
> -	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
> +	if (pcie->type == BCM7278) {
> +		/* Perst bit has moved and assert value is 0 */
> +		tmp = readl(pcie->base + PCIE_MISC_PCIE_CTRL);
> +		u32p_replace_bits(&tmp, !val, PCIE_MISC_PCIE_CTRL_PCIE_PERSTB_MASK);
> +		writel(tmp, pcie->base +  PCIE_MISC_PCIE_CTRL);
> +	} else {
> +		tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
> +		u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_PERST_MASK);
> +		writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));

Humm, now we have a mixture of a code path based on the chip and 
variables to abstract the register details. Just do a function per chip.

I have some notion to abstract out the PERST# handling from the host 
bridges. We have several cases of GPIO based handling and random 
assertion times. So having an ops function here will move in that 
direction.

> +	}
>  }
>  
>  static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
> @@ -771,7 +779,10 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
>  
>  	/* Reset the bridge */
>  	brcm_pcie_bridge_sw_init_set(pcie, 1);
> -	brcm_pcie_perst_set(pcie, 1);

If these 2 functions are always called together, then you just need 1 
per chip function.

> +
> +	/* BCM7278 fails when PERST# is set here */
> +	if (pcie->type != BCM7278)
> +		brcm_pcie_perst_set(pcie, 1);
>  
>  	usleep_range(100, 200);
>  
> -- 
> 2.17.1
> 

Re: [PATCH v11 05/11] PCI: brcmstb: Add bcm7278 PERST# support

[Rob Herring]

On Mon, Aug 24, 2020 at 03:30:18PM -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> The PERST# bit was moved to a different register in 7278-type STB chips.
> In addition, the polarity of the bit was also changed; for other chips
> writing a 1 specified assert; for 7278-type chips, writing a 0 specifies
> assert.
> 
> Of course, PERST# is a PCIe asserted-low signal.
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 19 +++++++++++++++----
>  1 file changed, 15 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index 3d588ab7a6dd..acf2239b0251 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -83,6 +83,7 @@
>  
>  #define PCIE_MISC_PCIE_CTRL				0x4064
>  #define  PCIE_MISC_PCIE_CTRL_PCIE_L23_REQUEST_MASK	0x1
> +#define PCIE_MISC_PCIE_CTRL_PCIE_PERSTB_MASK		0x4
>  
>  #define PCIE_MISC_PCIE_STATUS				0x4068
>  #define  PCIE_MISC_PCIE_STATUS_PCIE_PORT_MASK		0x80
> @@ -684,9 +685,16 @@ static inline void brcm_pcie_perst_set(struct brcm_pcie *pcie, u32 val)
>  {
>  	u32 tmp;
>  
> -	tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
> -	u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_PERST_MASK);
> -	writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
> +	if (pcie->type == BCM7278) {
> +		/* Perst bit has moved and assert value is 0 */
> +		tmp = readl(pcie->base + PCIE_MISC_PCIE_CTRL);
> +		u32p_replace_bits(&tmp, !val, PCIE_MISC_PCIE_CTRL_PCIE_PERSTB_MASK);
> +		writel(tmp, pcie->base +  PCIE_MISC_PCIE_CTRL);
> +	} else {
> +		tmp = readl(pcie->base + PCIE_RGR1_SW_INIT_1(pcie));
> +		u32p_replace_bits(&tmp, val, PCIE_RGR1_SW_INIT_1_PERST_MASK);
> +		writel(tmp, pcie->base + PCIE_RGR1_SW_INIT_1(pcie));

Humm, now we have a mixture of a code path based on the chip and 
variables to abstract the register details. Just do a function per chip.

I have some notion to abstract out the PERST# handling from the host 
bridges. We have several cases of GPIO based handling and random 
assertion times. So having an ops function here will move in that 
direction.

> +	}
>  }
>  
>  static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
> @@ -771,7 +779,10 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
>  
>  	/* Reset the bridge */
>  	brcm_pcie_bridge_sw_init_set(pcie, 1);
> -	brcm_pcie_perst_set(pcie, 1);

If these 2 functions are always called together, then you just need 1 
per chip function.

> +
> +	/* BCM7278 fails when PERST# is set here */
> +	if (pcie->type != BCM7278)
> +		brcm_pcie_perst_set(pcie, 1);
>  
>  	usleep_range(100, 200);
>  
> -- 
> 2.17.1
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 06/11] PCI: brcmstb: Add control of rescal reset

[Rob Herring]

On Mon, Aug 24, 2020 at 03:30:19PM -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> Some STB chips have a special purpose reset controller named RESCAL (reset
> calibration).  The PCIe HW can now control RESCAL to start and stop its
> operation.  On probe(), the RESCAL is deasserted and the driver goes
> through the sequence of setting registers and reading status in order to
> start the internal PHY that is required for the PCIe.
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 82 ++++++++++++++++++++++++++-
>  1 file changed, 81 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index acf2239b0251..041b8d109563 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -23,6 +23,7 @@
>  #include <linux/of_platform.h>
>  #include <linux/pci.h>
>  #include <linux/printk.h>
> +#include <linux/reset.h>
>  #include <linux/sizes.h>
>  #include <linux/slab.h>
>  #include <linux/string.h>
> @@ -158,6 +159,16 @@
>  #define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
>  #define PCIE_RGR1_SW_INIT_1(pcie)	(pcie->reg_offsets[RGR1_SW_INIT_1])
>  
> +/* Rescal registers */
> +#define PCIE_DVT_PMU_PCIE_PHY_CTRL				0xc700
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS			0x3
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK		0x4
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT	0x2
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK		0x2
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT		0x1
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK		0x1
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT		0x0
> +
>  enum {
>  	RGR1_SW_INIT_1,
>  	EXT_CFG_INDEX,
> @@ -247,6 +258,7 @@ struct brcm_pcie {
>  	const int		*reg_offsets;
>  	const int		*reg_field_info;
>  	enum pcie_type		type;
> +	struct reset_control	*rescal;
>  };
>  
>  /*
> @@ -965,6 +977,47 @@ static void brcm_pcie_enter_l23(struct brcm_pcie *pcie)
>  		dev_err(pcie->dev, "failed to enter low-power link state\n");
>  }
>  
> +static int brcm_phy_cntl(struct brcm_pcie *pcie, const int start)
> +{
> +	static const u32 shifts[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT,};
> +	static const u32 masks[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK,};
> +	const int beg = start ? 0 : PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS - 1;
> +	const int end = start ? PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS : -1;
> +	u32 tmp, combined_mask = 0;
> +	u32 val = !!start;
> +	void __iomem *base = pcie->base;
> +	int i;
> +
> +	for (i = beg; i != end; start ? i++ : i--) {
> +		tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +		tmp = (tmp & ~masks[i]) | ((val << shifts[i]) & masks[i]);
> +		writel(tmp, base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +		usleep_range(50, 200);
> +		combined_mask |= masks[i];
> +	}
> +
> +	tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +	val = start ? combined_mask : 0;
> +
> +	return (tmp & combined_mask) == val ? 0 : -EIO;
> +}
> +
> +static inline int brcm_phy_start(struct brcm_pcie *pcie)
> +{
> +	return pcie->rescal ? brcm_phy_cntl(pcie, 1) : 0;
> +}
> +
> +static inline int brcm_phy_stop(struct brcm_pcie *pcie)
> +{
> +	return pcie->rescal ? brcm_phy_cntl(pcie, 0) : 0;
> +}
> +
>  static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>  {
>  	void __iomem *base = pcie->base;
> @@ -992,11 +1045,15 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>  static int brcm_pcie_suspend(struct device *dev)
>  {
>  	struct brcm_pcie *pcie = dev_get_drvdata(dev);
> +	int ret;
>  
>  	brcm_pcie_turn_off(pcie);
> +	ret = brcm_phy_stop(pcie);
> +	if (ret)
> +		dev_err(pcie->dev, "failed to stop phy\n");
>  	clk_disable_unprepare(pcie->clk);
>  
> -	return 0;
> +	return ret;
>  }
>  
>  static int brcm_pcie_resume(struct device *dev)
> @@ -1009,6 +1066,12 @@ static int brcm_pcie_resume(struct device *dev)
>  	base = pcie->base;
>  	clk_prepare_enable(pcie->clk);
>  
> +	ret = brcm_phy_start(pcie);
> +	if (ret) {
> +		dev_err(pcie->dev, "failed to start phy\n");
> +		return ret;
> +	}
> +
>  	/* Take bridge out of reset so we can access the SERDES reg */
>  	brcm_pcie_bridge_sw_init_set(pcie, 0);
>  
> @@ -1034,6 +1097,9 @@ static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>  {
>  	brcm_msi_remove(pcie);
>  	brcm_pcie_turn_off(pcie);
> +	if (brcm_phy_stop(pcie))
> +		dev_err(pcie->dev, "failed to stop phy\n");
> +	reset_control_assert(pcie->rescal);
>  	clk_disable_unprepare(pcie->clk);
>  }
>  
> @@ -1112,6 +1178,20 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>  		dev_err(&pdev->dev, "could not enable clock\n");
>  		return ret;
>  	}
> +	pcie->rescal = devm_reset_control_get_optional_shared(&pdev->dev, "rescal");
> +	if (IS_ERR(pcie->rescal))
> +		return PTR_ERR(pcie->rescal);
> +
> +	ret = reset_control_deassert(pcie->rescal);
> +	if (ret)
> +		dev_err(&pdev->dev, "failed to deassert 'rescal'\n");
> +
> +	ret = brcm_phy_start(pcie);
> +	if (ret) {
> +		dev_err(pcie->dev, "failed to start phy\n");

4 calls to brcm_phy_cntl() and 4 error prints. Move the error print to 
brcm_phy_cntl.

> +		reset_control_assert(pcie->rescal);
> +		return ret;
> +	}
>  
>  	ret = brcm_pcie_setup(pcie);
>  	if (ret)
> -- 
> 2.17.1
> 

Re: [PATCH v11 06/11] PCI: brcmstb: Add control of rescal reset

[Rob Herring]

On Mon, Aug 24, 2020 at 03:30:19PM -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> Some STB chips have a special purpose reset controller named RESCAL (reset
> calibration).  The PCIe HW can now control RESCAL to start and stop its
> operation.  On probe(), the RESCAL is deasserted and the driver goes
> through the sequence of setting registers and reading status in order to
> start the internal PHY that is required for the PCIe.
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 82 ++++++++++++++++++++++++++-
>  1 file changed, 81 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index acf2239b0251..041b8d109563 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -23,6 +23,7 @@
>  #include <linux/of_platform.h>
>  #include <linux/pci.h>
>  #include <linux/printk.h>
> +#include <linux/reset.h>
>  #include <linux/sizes.h>
>  #include <linux/slab.h>
>  #include <linux/string.h>
> @@ -158,6 +159,16 @@
>  #define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
>  #define PCIE_RGR1_SW_INIT_1(pcie)	(pcie->reg_offsets[RGR1_SW_INIT_1])
>  
> +/* Rescal registers */
> +#define PCIE_DVT_PMU_PCIE_PHY_CTRL				0xc700
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS			0x3
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK		0x4
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT	0x2
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK		0x2
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT		0x1
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK		0x1
> +#define  PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT		0x0
> +
>  enum {
>  	RGR1_SW_INIT_1,
>  	EXT_CFG_INDEX,
> @@ -247,6 +258,7 @@ struct brcm_pcie {
>  	const int		*reg_offsets;
>  	const int		*reg_field_info;
>  	enum pcie_type		type;
> +	struct reset_control	*rescal;
>  };
>  
>  /*
> @@ -965,6 +977,47 @@ static void brcm_pcie_enter_l23(struct brcm_pcie *pcie)
>  		dev_err(pcie->dev, "failed to enter low-power link state\n");
>  }
>  
> +static int brcm_phy_cntl(struct brcm_pcie *pcie, const int start)
> +{
> +	static const u32 shifts[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_SHIFT,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_SHIFT,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_SHIFT,};
> +	static const u32 masks[PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS] = {
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_PWRDN_MASK,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_RESET_MASK,
> +		PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_DIG_RESET_MASK,};
> +	const int beg = start ? 0 : PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS - 1;
> +	const int end = start ? PCIE_DVT_PMU_PCIE_PHY_CTRL_DAST_NFLDS : -1;
> +	u32 tmp, combined_mask = 0;
> +	u32 val = !!start;
> +	void __iomem *base = pcie->base;
> +	int i;
> +
> +	for (i = beg; i != end; start ? i++ : i--) {
> +		tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +		tmp = (tmp & ~masks[i]) | ((val << shifts[i]) & masks[i]);
> +		writel(tmp, base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +		usleep_range(50, 200);
> +		combined_mask |= masks[i];
> +	}
> +
> +	tmp = readl(base + PCIE_DVT_PMU_PCIE_PHY_CTRL);
> +	val = start ? combined_mask : 0;
> +
> +	return (tmp & combined_mask) == val ? 0 : -EIO;
> +}
> +
> +static inline int brcm_phy_start(struct brcm_pcie *pcie)
> +{
> +	return pcie->rescal ? brcm_phy_cntl(pcie, 1) : 0;
> +}
> +
> +static inline int brcm_phy_stop(struct brcm_pcie *pcie)
> +{
> +	return pcie->rescal ? brcm_phy_cntl(pcie, 0) : 0;
> +}
> +
>  static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>  {
>  	void __iomem *base = pcie->base;
> @@ -992,11 +1045,15 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>  static int brcm_pcie_suspend(struct device *dev)
>  {
>  	struct brcm_pcie *pcie = dev_get_drvdata(dev);
> +	int ret;
>  
>  	brcm_pcie_turn_off(pcie);
> +	ret = brcm_phy_stop(pcie);
> +	if (ret)
> +		dev_err(pcie->dev, "failed to stop phy\n");
>  	clk_disable_unprepare(pcie->clk);
>  
> -	return 0;
> +	return ret;
>  }
>  
>  static int brcm_pcie_resume(struct device *dev)
> @@ -1009,6 +1066,12 @@ static int brcm_pcie_resume(struct device *dev)
>  	base = pcie->base;
>  	clk_prepare_enable(pcie->clk);
>  
> +	ret = brcm_phy_start(pcie);
> +	if (ret) {
> +		dev_err(pcie->dev, "failed to start phy\n");
> +		return ret;
> +	}
> +
>  	/* Take bridge out of reset so we can access the SERDES reg */
>  	brcm_pcie_bridge_sw_init_set(pcie, 0);
>  
> @@ -1034,6 +1097,9 @@ static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>  {
>  	brcm_msi_remove(pcie);
>  	brcm_pcie_turn_off(pcie);
> +	if (brcm_phy_stop(pcie))
> +		dev_err(pcie->dev, "failed to stop phy\n");
> +	reset_control_assert(pcie->rescal);
>  	clk_disable_unprepare(pcie->clk);
>  }
>  
> @@ -1112,6 +1178,20 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>  		dev_err(&pdev->dev, "could not enable clock\n");
>  		return ret;
>  	}
> +	pcie->rescal = devm_reset_control_get_optional_shared(&pdev->dev, "rescal");
> +	if (IS_ERR(pcie->rescal))
> +		return PTR_ERR(pcie->rescal);
> +
> +	ret = reset_control_deassert(pcie->rescal);
> +	if (ret)
> +		dev_err(&pdev->dev, "failed to deassert 'rescal'\n");
> +
> +	ret = brcm_phy_start(pcie);
> +	if (ret) {
> +		dev_err(pcie->dev, "failed to start phy\n");

4 calls to brcm_phy_cntl() and 4 error prints. Move the error print to 
brcm_phy_cntl.

> +		reset_control_assert(pcie->rescal);
> +		return ret;
> +	}
>  
>  	ret = brcm_pcie_setup(pcie);
>  	if (ret)
> -- 
> 2.17.1
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 08/11] PCI: brcmstb: Set additional internal memory DMA viewport sizes

[Rob Herring]

On Mon, Aug 24, 2020 at 03:30:21PM -0400, Jim Quinlan wrote:
> The Raspberry Pi (RPI) is currently the only chip using this driver
> (pcie-brcmstb.c).  There, only one memory controller is used, without an
> extension region, and the SCB0 viewport size is set to the size of the
> first and only dma-range region.  Other BrcmSTB SOCs have more complicated
> memory configurations that require setting additional viewport sizes.
> 
> BrcmSTB PCIe controllers are intimately connected to the memory
> controller(s) on the SOC.  The SOC may have one to three memory
> controllers; they are indicated by the term SCBi.  Each controller has a
> base region and an optional extension region.  In physical memory, the base
> and extension regions of a controller are not adjacent, but in PCIe-space
> they are.
> 
> There is a "viewport" for each memory controller that allows DMA from
> endpoint devices.  Each viewport's size must be set to a power of two, and
> that size must be equal to or larger than the amount of memory each
> controller supports which is the sum of base region and its optional
> extension.  Further, the 1-3 viewports are also adjacent in PCIe-space.
> 
> Unfortunately the viewport sizes cannot be ascertained from the
> "dma-ranges" property so they have their own property, "brcm,scb-sizes".
> This is because dma-range information does not indicate what memory
> controller it is associated.  For example, consider the following case
> where the size of one dma-range is 2GB and the second dma-range is 1GB:
> 
>     /* Case 1: SCB0 size set to 4GB */
>     dma-range0: 2GB (from memc0-base)
>     dma-range1: 1GB (from memc0-extension)
> 
>     /* Case 2: SCB0 size set to 2GB, SCB1 size set to 1GB */
>     dma-range0: 2GB (from memc0-base)
>     dma-range1: 1GB (from memc0-extension)
> 
> By just looking at the dma-ranges information, one cannot tell which
> situation applies. That is why an additional property is needed.  Its
> length indicates the number of memory controllers being used and each value
> indicates the viewport size.
> 
> Note that the RPI DT does not have a "brcm,scb-sizes" property value,
> as it is assumed that it only requires one memory controller and no
> extension.  So the optional use of "brcm,scb-sizes" will be backwards
> compatible.
> 
> One last layer of complexity exists: all of the viewports sizes must be
> added and rounded up to a power of two to determine what the "BAR" size is.
> Further, an offset must be given that indicates the base PCIe address of
> this "BAR".  The use of the term BAR is typically associated with endpoint
> devices, and the term is used here because the PCIe HW may be used as an RC
> or an EP.  In the former case, all of the system memory appears in a single
> "BAR" region in PCIe memory.  As it turns out, BrcmSTB PCIe HW is rarely
> used in the EP role and its system of mapping memory is an artifact that
> requires multiple dma-ranges regions.
> 
> Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++-------
>  1 file changed, 50 insertions(+), 18 deletions(-)
> 
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index 041b8d109563..7150eaa803c2 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -57,6 +57,8 @@
>  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK	0x300000
>  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128		0x0
>  #define  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK		0xf8000000
> +#define  PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK		0x07c00000
> +#define  PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK		0x0000001f

Perhaps make 0-2 an arg and then you can just do:

u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB_SIZE_MASK(memc))

>  
>  #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO		0x400c
>  #define PCIE_MEM_WIN0_LO(win)	\
> @@ -154,6 +156,7 @@
>  #define SSC_STATUS_OFFSET		0x1
>  #define SSC_STATUS_SSC_MASK		0x400
>  #define SSC_STATUS_PLL_LOCK_MASK	0x800
> +#define PCIE_BRCM_MAX_MEMC		3
>  
>  #define IDX_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_INDEX])
>  #define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
> @@ -259,6 +262,8 @@ struct brcm_pcie {
>  	const int		*reg_field_info;
>  	enum pcie_type		type;
>  	struct reset_control	*rescal;
> +	int			num_memc;
> +	u64			memc_size[PCIE_BRCM_MAX_MEMC];
>  };
>  
>  /*
> @@ -714,22 +719,44 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
>  							u64 *rc_bar2_offset)
>  {
>  	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> -	struct device *dev = pcie->dev;
>  	struct resource_entry *entry;
> +	struct device *dev = pcie->dev;
> +	u64 lowest_pcie_addr = ~(u64)0;
> +	int ret, i = 0;
> +	u64 size = 0;
>  
> -	entry = resource_list_first_type(&bridge->dma_ranges, IORESOURCE_MEM);
> -	if (!entry)
> -		return -ENODEV;
> +	resource_list_for_each_entry(entry, &bridge->dma_ranges) {
> +		u64 pcie_beg = entry->res->start - entry->offset;
>  
> +		size += entry->res->end - entry->res->start + 1;
> +		if (pcie_beg < lowest_pcie_addr)
> +			lowest_pcie_addr = pcie_beg;
> +	}
>  
> -	/*
> -	 * The controller expects the inbound window offset to be calculated as
> -	 * the difference between PCIe's address space and CPU's. The offset
> -	 * provided by the firmware is calculated the opposite way, so we
> -	 * negate it.
> -	 */
> -	*rc_bar2_offset = -entry->offset;
> -	*rc_bar2_size = 1ULL << fls64(entry->res->end - entry->res->start);
> +	if (lowest_pcie_addr == ~(u64)0) {
> +		dev_err(dev, "DT node has no dma-ranges\n");
> +		return -EINVAL;
> +	}
> +
> +	ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1,
> +						  PCIE_BRCM_MAX_MEMC);
> +
> +	if (ret <= 0) {
> +		/* Make an educated guess */
> +		pcie->num_memc = 1;
> +		pcie->memc_size[0] = 1ULL << fls64(size - 1);

Use roundup_pow_of_two()

> +	} else {
> +		pcie->num_memc = ret;
> +	}
> +
> +	/* Each memc is viewed through a "port" that is a power of 2 */
> +	for (i = 0, size = 0; i < pcie->num_memc; i++)
> +		size += pcie->memc_size[i];
> +
> +	/* System memory starts at this address in PCIe-space */
> +	*rc_bar2_offset = lowest_pcie_addr;
> +	/* The sum of all memc views must also be a power of 2 */
> +	*rc_bar2_size = 1ULL << fls64(size - 1);

Use roundup_pow_of_two()

>  
>  	/*
>  	 * We validate the inbound memory view even though we should trust
> @@ -781,12 +808,11 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
>  	void __iomem *base = pcie->base;
>  	struct device *dev = pcie->dev;
>  	struct resource_entry *entry;
> -	unsigned int scb_size_val;
>  	bool ssc_good = false;
>  	struct resource *res;
>  	int num_out_wins = 0;
>  	u16 nlw, cls, lnksta;
> -	int i, ret;
> +	int i, ret, memc;
>  	u32 tmp, aspm_support;
>  
>  	/* Reset the bridge */
> @@ -826,11 +852,17 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
>  	writel(upper_32_bits(rc_bar2_offset),
>  	       base + PCIE_MISC_RC_BAR2_CONFIG_HI);
>  
> -	scb_size_val = rc_bar2_size ?
> -		       ilog2(rc_bar2_size) - 15 : 0xf; /* 0xf is 1GB */
>  	tmp = readl(base + PCIE_MISC_MISC_CTRL);
> -	u32p_replace_bits(&tmp, scb_size_val,
> -			  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
> +	for (memc = 0; memc < pcie->num_memc; memc++) {
> +		u32 scb_size_val = ilog2(pcie->memc_size[memc]) - 15;
> +
> +		if (memc == 0)
> +			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
> +		else if (memc == 1)
> +			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK);
> +		else if (memc == 2)
> +			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK);
> +	}
>  	writel(tmp, base + PCIE_MISC_MISC_CTRL);
>  
>  	/*
> -- 
> 2.17.1
> 

Re: [PATCH v11 08/11] PCI: brcmstb: Set additional internal memory DMA viewport sizes

[Rob Herring]

On Mon, Aug 24, 2020 at 03:30:21PM -0400, Jim Quinlan wrote:
> The Raspberry Pi (RPI) is currently the only chip using this driver
> (pcie-brcmstb.c).  There, only one memory controller is used, without an
> extension region, and the SCB0 viewport size is set to the size of the
> first and only dma-range region.  Other BrcmSTB SOCs have more complicated
> memory configurations that require setting additional viewport sizes.
> 
> BrcmSTB PCIe controllers are intimately connected to the memory
> controller(s) on the SOC.  The SOC may have one to three memory
> controllers; they are indicated by the term SCBi.  Each controller has a
> base region and an optional extension region.  In physical memory, the base
> and extension regions of a controller are not adjacent, but in PCIe-space
> they are.
> 
> There is a "viewport" for each memory controller that allows DMA from
> endpoint devices.  Each viewport's size must be set to a power of two, and
> that size must be equal to or larger than the amount of memory each
> controller supports which is the sum of base region and its optional
> extension.  Further, the 1-3 viewports are also adjacent in PCIe-space.
> 
> Unfortunately the viewport sizes cannot be ascertained from the
> "dma-ranges" property so they have their own property, "brcm,scb-sizes".
> This is because dma-range information does not indicate what memory
> controller it is associated.  For example, consider the following case
> where the size of one dma-range is 2GB and the second dma-range is 1GB:
> 
>     /* Case 1: SCB0 size set to 4GB */
>     dma-range0: 2GB (from memc0-base)
>     dma-range1: 1GB (from memc0-extension)
> 
>     /* Case 2: SCB0 size set to 2GB, SCB1 size set to 1GB */
>     dma-range0: 2GB (from memc0-base)
>     dma-range1: 1GB (from memc0-extension)
> 
> By just looking at the dma-ranges information, one cannot tell which
> situation applies. That is why an additional property is needed.  Its
> length indicates the number of memory controllers being used and each value
> indicates the viewport size.
> 
> Note that the RPI DT does not have a "brcm,scb-sizes" property value,
> as it is assumed that it only requires one memory controller and no
> extension.  So the optional use of "brcm,scb-sizes" will be backwards
> compatible.
> 
> One last layer of complexity exists: all of the viewports sizes must be
> added and rounded up to a power of two to determine what the "BAR" size is.
> Further, an offset must be given that indicates the base PCIe address of
> this "BAR".  The use of the term BAR is typically associated with endpoint
> devices, and the term is used here because the PCIe HW may be used as an RC
> or an EP.  In the former case, all of the system memory appears in a single
> "BAR" region in PCIe memory.  As it turns out, BrcmSTB PCIe HW is rarely
> used in the EP role and its system of mapping memory is an artifact that
> requires multiple dma-ranges regions.
> 
> Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++-------
>  1 file changed, 50 insertions(+), 18 deletions(-)
> 
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index 041b8d109563..7150eaa803c2 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -57,6 +57,8 @@
>  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK	0x300000
>  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128		0x0
>  #define  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK		0xf8000000
> +#define  PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK		0x07c00000
> +#define  PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK		0x0000001f

Perhaps make 0-2 an arg and then you can just do:

u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB_SIZE_MASK(memc))

>  
>  #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO		0x400c
>  #define PCIE_MEM_WIN0_LO(win)	\
> @@ -154,6 +156,7 @@
>  #define SSC_STATUS_OFFSET		0x1
>  #define SSC_STATUS_SSC_MASK		0x400
>  #define SSC_STATUS_PLL_LOCK_MASK	0x800
> +#define PCIE_BRCM_MAX_MEMC		3
>  
>  #define IDX_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_INDEX])
>  #define DATA_ADDR(pcie)			(pcie->reg_offsets[EXT_CFG_DATA])
> @@ -259,6 +262,8 @@ struct brcm_pcie {
>  	const int		*reg_field_info;
>  	enum pcie_type		type;
>  	struct reset_control	*rescal;
> +	int			num_memc;
> +	u64			memc_size[PCIE_BRCM_MAX_MEMC];
>  };
>  
>  /*
> @@ -714,22 +719,44 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
>  							u64 *rc_bar2_offset)
>  {
>  	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> -	struct device *dev = pcie->dev;
>  	struct resource_entry *entry;
> +	struct device *dev = pcie->dev;
> +	u64 lowest_pcie_addr = ~(u64)0;
> +	int ret, i = 0;
> +	u64 size = 0;
>  
> -	entry = resource_list_first_type(&bridge->dma_ranges, IORESOURCE_MEM);
> -	if (!entry)
> -		return -ENODEV;
> +	resource_list_for_each_entry(entry, &bridge->dma_ranges) {
> +		u64 pcie_beg = entry->res->start - entry->offset;
>  
> +		size += entry->res->end - entry->res->start + 1;
> +		if (pcie_beg < lowest_pcie_addr)
> +			lowest_pcie_addr = pcie_beg;
> +	}
>  
> -	/*
> -	 * The controller expects the inbound window offset to be calculated as
> -	 * the difference between PCIe's address space and CPU's. The offset
> -	 * provided by the firmware is calculated the opposite way, so we
> -	 * negate it.
> -	 */
> -	*rc_bar2_offset = -entry->offset;
> -	*rc_bar2_size = 1ULL << fls64(entry->res->end - entry->res->start);
> +	if (lowest_pcie_addr == ~(u64)0) {
> +		dev_err(dev, "DT node has no dma-ranges\n");
> +		return -EINVAL;
> +	}
> +
> +	ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1,
> +						  PCIE_BRCM_MAX_MEMC);
> +
> +	if (ret <= 0) {
> +		/* Make an educated guess */
> +		pcie->num_memc = 1;
> +		pcie->memc_size[0] = 1ULL << fls64(size - 1);

Use roundup_pow_of_two()

> +	} else {
> +		pcie->num_memc = ret;
> +	}
> +
> +	/* Each memc is viewed through a "port" that is a power of 2 */
> +	for (i = 0, size = 0; i < pcie->num_memc; i++)
> +		size += pcie->memc_size[i];
> +
> +	/* System memory starts at this address in PCIe-space */
> +	*rc_bar2_offset = lowest_pcie_addr;
> +	/* The sum of all memc views must also be a power of 2 */
> +	*rc_bar2_size = 1ULL << fls64(size - 1);

Use roundup_pow_of_two()

>  
>  	/*
>  	 * We validate the inbound memory view even though we should trust
> @@ -781,12 +808,11 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
>  	void __iomem *base = pcie->base;
>  	struct device *dev = pcie->dev;
>  	struct resource_entry *entry;
> -	unsigned int scb_size_val;
>  	bool ssc_good = false;
>  	struct resource *res;
>  	int num_out_wins = 0;
>  	u16 nlw, cls, lnksta;
> -	int i, ret;
> +	int i, ret, memc;
>  	u32 tmp, aspm_support;
>  
>  	/* Reset the bridge */
> @@ -826,11 +852,17 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
>  	writel(upper_32_bits(rc_bar2_offset),
>  	       base + PCIE_MISC_RC_BAR2_CONFIG_HI);
>  
> -	scb_size_val = rc_bar2_size ?
> -		       ilog2(rc_bar2_size) - 15 : 0xf; /* 0xf is 1GB */
>  	tmp = readl(base + PCIE_MISC_MISC_CTRL);
> -	u32p_replace_bits(&tmp, scb_size_val,
> -			  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
> +	for (memc = 0; memc < pcie->num_memc; memc++) {
> +		u32 scb_size_val = ilog2(pcie->memc_size[memc]) - 15;
> +
> +		if (memc == 0)
> +			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
> +		else if (memc == 1)
> +			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK);
> +		else if (memc == 2)
> +			u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK);
> +	}
>  	writel(tmp, base + PCIE_MISC_MISC_CTRL);
>  
>  	/*
> -- 
> 2.17.1
> 

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 09/11] PCI: brcmstb: Accommodate MSI for older chips

[Rob Herring]

On Mon, 24 Aug 2020 15:30:22 -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> Older BrcmSTB chips do not have a separate register for MSI interrupts; the
> MSIs are in a register that also contains unrelated interrupts.  In
> addition, the interrupts lie in bits [31..24] for these legacy chips.  This
> commit provides common code for both legacy and non-legacy MSI interrupt
> registers.
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 71 +++++++++++++++++++--------
>  1 file changed, 50 insertions(+), 21 deletions(-)
> 

Reviewed-by: Rob Herring <robh@kernel.org>

Re: [PATCH v11 09/11] PCI: brcmstb: Accommodate MSI for older chips

[Rob Herring]

On Mon, 24 Aug 2020 15:30:22 -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> Older BrcmSTB chips do not have a separate register for MSI interrupts; the
> MSIs are in a register that also contains unrelated interrupts.  In
> addition, the interrupts lie in bits [31..24] for these legacy chips.  This
> commit provides common code for both legacy and non-legacy MSI interrupt
> registers.
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 71 +++++++++++++++++++--------
>  1 file changed, 50 insertions(+), 21 deletions(-)
> 

Reviewed-by: Rob Herring <robh@kernel.org>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 10/11] PCI: brcmstb: Set bus max burst size by chip type

[Rob Herring]

On Mon, 24 Aug 2020 15:30:23 -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> The proper value of the parameter SCB_MAX_BURST_SIZE varies per chip.  The
> 2711 family requires 128B whereas other devices can employ 512.  The
> assignment is complicated by the fact that the values for this two-bit
> field have different meanings;
> 
>   Value   Type_Generic    Type_7278
> 
>      00       Reserved         128B
>      01           128B         256B
>      10           256B         512B
>      11           512B     Reserved
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 19 +++++++++++++++----
>  1 file changed, 15 insertions(+), 4 deletions(-)
> 

Reviewed-by: Rob Herring <robh@kernel.org>

Re: [PATCH v11 10/11] PCI: brcmstb: Set bus max burst size by chip type

[Rob Herring]

On Mon, 24 Aug 2020 15:30:23 -0400, Jim Quinlan wrote:
> From: Jim Quinlan <jquinlan@broadcom.com>
> 
> The proper value of the parameter SCB_MAX_BURST_SIZE varies per chip.  The
> 2711 family requires 128B whereas other devices can employ 512.  The
> assignment is complicated by the fact that the values for this two-bit
> field have different meanings;
> 
>   Value   Type_Generic    Type_7278
> 
>      00       Reserved         128B
>      01           128B         256B
>      10           256B         512B
>      11           512B     Reserved
> 
> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 19 +++++++++++++++----
>  1 file changed, 15 insertions(+), 4 deletions(-)
> 

Reviewed-by: Rob Herring <robh@kernel.org>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 11/11] PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list

[Rob Herring]

On Mon, 24 Aug 2020 15:30:24 -0400, Jim Quinlan wrote:
> Now that the support is in place with previous commits, we add several
> chips that use the BrcmSTB driver.
> 
> Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 

Acked-by: Rob Herring <robh@kernel.org>

Re: [PATCH v11 11/11] PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list

[Rob Herring]

On Mon, 24 Aug 2020 15:30:24 -0400, Jim Quinlan wrote:
> Now that the support is in place with previous commits, we add several
> chips that use the BrcmSTB driver.
> 
> Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 

Acked-by: Rob Herring <robh@kernel.org>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Jim Quinlan]

[-- Attachment #1: Type: text/plain, Size: 3218 bytes --]

On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
>
> On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
> > From: Jim Quinlan <jquinlan@broadcom.com>
> >
> > Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
> > and resume.  Now the PCIe driver may do so as well.
> >
> > Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> > Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> > ---
> >  drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
> >  1 file changed, 47 insertions(+)
> >
> > diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> > index c2b3d2946a36..3d588ab7a6dd 100644
> > --- a/drivers/pci/controller/pcie-brcmstb.c
> > +++ b/drivers/pci/controller/pcie-brcmstb.c
> > @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
> >       brcm_pcie_bridge_sw_init_set(pcie, 1);
> >  }
> >
> > +static int brcm_pcie_suspend(struct device *dev)
> > +{
> > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > +
> > +     brcm_pcie_turn_off(pcie);
> > +     clk_disable_unprepare(pcie->clk);
> > +
> > +     return 0;
> > +}
> > +
> > +static int brcm_pcie_resume(struct device *dev)
> > +{
> > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > +     void __iomem *base;
> > +     u32 tmp;
> > +     int ret;
> > +
> > +     base = pcie->base;
> > +     clk_prepare_enable(pcie->clk);
> > +
> > +     /* Take bridge out of reset so we can access the SERDES reg */
> > +     brcm_pcie_bridge_sw_init_set(pcie, 0);
> > +
> > +     /* SERDES_IDDQ = 0 */
> > +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
> > +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > +
> > +     /* wait for serdes to be stable */
> > +     udelay(100);
>
> Really needs to be a spinloop?
>
> > +
> > +     ret = brcm_pcie_setup(pcie);
> > +     if (ret)
> > +             return ret;
> > +
> > +     if (pcie->msi)
> > +             brcm_msi_set_regs(pcie->msi);
> > +
> > +     return 0;
> > +}
> > +
> >  static void __brcm_pcie_remove(struct brcm_pcie *pcie)
> >  {
> >       brcm_msi_remove(pcie);
> > @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
> >
> >  MODULE_DEVICE_TABLE(of, brcm_pcie_match);
> >
> > +static const struct dev_pm_ops brcm_pcie_pm_ops = {
> > +     .suspend_noirq = brcm_pcie_suspend,
> > +     .resume_noirq = brcm_pcie_resume,
>
> Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
> and 1352 of .suspend in the tree.

I will test switching this to  suspend_late/resume_early.

Thanks,
Jim Quinlan
Broadcom STB
>
> Is doing a clk unprepare even safe in .suspend_noirq? IIRC,
> prepare/unprepare can sleep.
>
> > +};
> > +
> >  static struct platform_driver brcm_pcie_driver = {
> >       .probe = brcm_pcie_probe,
> >       .remove = brcm_pcie_remove,
> >       .driver = {
> >               .name = "brcm-pcie",
> >               .of_match_table = brcm_pcie_match,
> > +             .pm = &brcm_pcie_pm_ops,
> >       },
> >  };
> >  module_platform_driver(brcm_pcie_driver);
> > --
> > 2.17.1
> >

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4167 bytes --]

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Jim Quinlan]

[-- Attachment #1.1: Type: text/plain, Size: 3218 bytes --]

On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
>
> On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
> > From: Jim Quinlan <jquinlan@broadcom.com>
> >
> > Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
> > and resume.  Now the PCIe driver may do so as well.
> >
> > Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> > Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> > ---
> >  drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
> >  1 file changed, 47 insertions(+)
> >
> > diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> > index c2b3d2946a36..3d588ab7a6dd 100644
> > --- a/drivers/pci/controller/pcie-brcmstb.c
> > +++ b/drivers/pci/controller/pcie-brcmstb.c
> > @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
> >       brcm_pcie_bridge_sw_init_set(pcie, 1);
> >  }
> >
> > +static int brcm_pcie_suspend(struct device *dev)
> > +{
> > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > +
> > +     brcm_pcie_turn_off(pcie);
> > +     clk_disable_unprepare(pcie->clk);
> > +
> > +     return 0;
> > +}
> > +
> > +static int brcm_pcie_resume(struct device *dev)
> > +{
> > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > +     void __iomem *base;
> > +     u32 tmp;
> > +     int ret;
> > +
> > +     base = pcie->base;
> > +     clk_prepare_enable(pcie->clk);
> > +
> > +     /* Take bridge out of reset so we can access the SERDES reg */
> > +     brcm_pcie_bridge_sw_init_set(pcie, 0);
> > +
> > +     /* SERDES_IDDQ = 0 */
> > +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
> > +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > +
> > +     /* wait for serdes to be stable */
> > +     udelay(100);
>
> Really needs to be a spinloop?
>
> > +
> > +     ret = brcm_pcie_setup(pcie);
> > +     if (ret)
> > +             return ret;
> > +
> > +     if (pcie->msi)
> > +             brcm_msi_set_regs(pcie->msi);
> > +
> > +     return 0;
> > +}
> > +
> >  static void __brcm_pcie_remove(struct brcm_pcie *pcie)
> >  {
> >       brcm_msi_remove(pcie);
> > @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
> >
> >  MODULE_DEVICE_TABLE(of, brcm_pcie_match);
> >
> > +static const struct dev_pm_ops brcm_pcie_pm_ops = {
> > +     .suspend_noirq = brcm_pcie_suspend,
> > +     .resume_noirq = brcm_pcie_resume,
>
> Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
> and 1352 of .suspend in the tree.

I will test switching this to  suspend_late/resume_early.

Thanks,
Jim Quinlan
Broadcom STB
>
> Is doing a clk unprepare even safe in .suspend_noirq? IIRC,
> prepare/unprepare can sleep.
>
> > +};
> > +
> >  static struct platform_driver brcm_pcie_driver = {
> >       .probe = brcm_pcie_probe,
> >       .remove = brcm_pcie_remove,
> >       .driver = {
> >               .name = "brcm-pcie",
> >               .of_match_table = brcm_pcie_match,
> > +             .pm = &brcm_pcie_pm_ops,
> >       },
> >  };
> >  module_platform_driver(brcm_pcie_driver);
> > --
> > 2.17.1
> >

[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4167 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Florian Fainelli]

On 9/10/2020 8:56 AM, Rob Herring wrote:
> On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
>> From: Jim Quinlan <jquinlan@broadcom.com>
>>
>> Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
>> and resume.  Now the PCIe driver may do so as well.
>>
>> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
>> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
>> ---
>>   drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
>>   1 file changed, 47 insertions(+)
>>
>> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
>> index c2b3d2946a36..3d588ab7a6dd 100644
>> --- a/drivers/pci/controller/pcie-brcmstb.c
>> +++ b/drivers/pci/controller/pcie-brcmstb.c
>> @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>>   	brcm_pcie_bridge_sw_init_set(pcie, 1);
>>   }
>>   
>> +static int brcm_pcie_suspend(struct device *dev)
>> +{
>> +	struct brcm_pcie *pcie = dev_get_drvdata(dev);
>> +
>> +	brcm_pcie_turn_off(pcie);
>> +	clk_disable_unprepare(pcie->clk);
>> +
>> +	return 0;
>> +}
>> +
>> +static int brcm_pcie_resume(struct device *dev)
>> +{
>> +	struct brcm_pcie *pcie = dev_get_drvdata(dev);
>> +	void __iomem *base;
>> +	u32 tmp;
>> +	int ret;
>> +
>> +	base = pcie->base;
>> +	clk_prepare_enable(pcie->clk);
>> +
>> +	/* Take bridge out of reset so we can access the SERDES reg */
>> +	brcm_pcie_bridge_sw_init_set(pcie, 0);
>> +
>> +	/* SERDES_IDDQ = 0 */
>> +	tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>> +	u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
>> +	writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>> +
>> +	/* wait for serdes to be stable */
>> +	udelay(100);
> 
> Really needs to be a spinloop?
> 
>> +
>> +	ret = brcm_pcie_setup(pcie);
>> +	if (ret)
>> +		return ret;
>> +
>> +	if (pcie->msi)
>> +		brcm_msi_set_regs(pcie->msi);
>> +
>> +	return 0;
>> +}
>> +
>>   static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>>   {
>>   	brcm_msi_remove(pcie);
>> @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>>   
>>   MODULE_DEVICE_TABLE(of, brcm_pcie_match);
>>   
>> +static const struct dev_pm_ops brcm_pcie_pm_ops = {
>> +	.suspend_noirq = brcm_pcie_suspend,
>> +	.resume_noirq = brcm_pcie_resume,
> 
> Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
> and 1352 of .suspend in the tree.
> 
> Is doing a clk unprepare even safe in .suspend_noirq? IIRC,
> prepare/unprepare can sleep.

Yes, it is safe, provided that your clock provider (clk-scmi.c in our 
case) supports it, too. In our case the underlying mailbox driver has 
its interrupts flagged with IRQF_NOSUSPEND such that they can still be 
processed at _noirq time.

I think the rationale was to ensure that this would be done much later 
after other subsystem have been made quiescent, but given the Linux 
device driver model, the PCI bridge should be suspended after all 
pci_device child device, so it should be safe not to use _noirq.
-- 
Florian

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Florian Fainelli]

On 9/10/2020 8:56 AM, Rob Herring wrote:
> On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
>> From: Jim Quinlan <jquinlan@broadcom.com>
>>
>> Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
>> and resume.  Now the PCIe driver may do so as well.
>>
>> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
>> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
>> ---
>>   drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
>>   1 file changed, 47 insertions(+)
>>
>> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
>> index c2b3d2946a36..3d588ab7a6dd 100644
>> --- a/drivers/pci/controller/pcie-brcmstb.c
>> +++ b/drivers/pci/controller/pcie-brcmstb.c
>> @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>>   	brcm_pcie_bridge_sw_init_set(pcie, 1);
>>   }
>>   
>> +static int brcm_pcie_suspend(struct device *dev)
>> +{
>> +	struct brcm_pcie *pcie = dev_get_drvdata(dev);
>> +
>> +	brcm_pcie_turn_off(pcie);
>> +	clk_disable_unprepare(pcie->clk);
>> +
>> +	return 0;
>> +}
>> +
>> +static int brcm_pcie_resume(struct device *dev)
>> +{
>> +	struct brcm_pcie *pcie = dev_get_drvdata(dev);
>> +	void __iomem *base;
>> +	u32 tmp;
>> +	int ret;
>> +
>> +	base = pcie->base;
>> +	clk_prepare_enable(pcie->clk);
>> +
>> +	/* Take bridge out of reset so we can access the SERDES reg */
>> +	brcm_pcie_bridge_sw_init_set(pcie, 0);
>> +
>> +	/* SERDES_IDDQ = 0 */
>> +	tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>> +	u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
>> +	writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>> +
>> +	/* wait for serdes to be stable */
>> +	udelay(100);
> 
> Really needs to be a spinloop?
> 
>> +
>> +	ret = brcm_pcie_setup(pcie);
>> +	if (ret)
>> +		return ret;
>> +
>> +	if (pcie->msi)
>> +		brcm_msi_set_regs(pcie->msi);
>> +
>> +	return 0;
>> +}
>> +
>>   static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>>   {
>>   	brcm_msi_remove(pcie);
>> @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>>   
>>   MODULE_DEVICE_TABLE(of, brcm_pcie_match);
>>   
>> +static const struct dev_pm_ops brcm_pcie_pm_ops = {
>> +	.suspend_noirq = brcm_pcie_suspend,
>> +	.resume_noirq = brcm_pcie_resume,
> 
> Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
> and 1352 of .suspend in the tree.
> 
> Is doing a clk unprepare even safe in .suspend_noirq? IIRC,
> prepare/unprepare can sleep.

Yes, it is safe, provided that your clock provider (clk-scmi.c in our 
case) supports it, too. In our case the underlying mailbox driver has 
its interrupts flagged with IRQF_NOSUSPEND such that they can still be 
processed at _noirq time.

I think the rationale was to ensure that this would be done much later 
after other subsystem have been made quiescent, but given the Linux 
device driver model, the PCI bridge should be suspended after all 
pci_device child device, so it should be safe not to use _noirq.
-- 
Florian

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Rob Herring]

On Thu, Sep 10, 2020 at 10:42 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
>
> On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
> >
> > On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
> > > From: Jim Quinlan <jquinlan@broadcom.com>
> > >
> > > Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
> > > and resume.  Now the PCIe driver may do so as well.
> > >
> > > Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> > > Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> > > ---
> > >  drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
> > >  1 file changed, 47 insertions(+)
> > >
> > > diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> > > index c2b3d2946a36..3d588ab7a6dd 100644
> > > --- a/drivers/pci/controller/pcie-brcmstb.c
> > > +++ b/drivers/pci/controller/pcie-brcmstb.c
> > > @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
> > >       brcm_pcie_bridge_sw_init_set(pcie, 1);
> > >  }
> > >
> > > +static int brcm_pcie_suspend(struct device *dev)
> > > +{
> > > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > > +
> > > +     brcm_pcie_turn_off(pcie);
> > > +     clk_disable_unprepare(pcie->clk);
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int brcm_pcie_resume(struct device *dev)
> > > +{
> > > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > > +     void __iomem *base;
> > > +     u32 tmp;
> > > +     int ret;
> > > +
> > > +     base = pcie->base;
> > > +     clk_prepare_enable(pcie->clk);
> > > +
> > > +     /* Take bridge out of reset so we can access the SERDES reg */
> > > +     brcm_pcie_bridge_sw_init_set(pcie, 0);
> > > +
> > > +     /* SERDES_IDDQ = 0 */
> > > +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > > +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
> > > +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > > +
> > > +     /* wait for serdes to be stable */
> > > +     udelay(100);
> >
> > Really needs to be a spinloop?
> >
> > > +
> > > +     ret = brcm_pcie_setup(pcie);
> > > +     if (ret)
> > > +             return ret;
> > > +
> > > +     if (pcie->msi)
> > > +             brcm_msi_set_regs(pcie->msi);
> > > +
> > > +     return 0;
> > > +}
> > > +
> > >  static void __brcm_pcie_remove(struct brcm_pcie *pcie)
> > >  {
> > >       brcm_msi_remove(pcie);
> > > @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
> > >
> > >  MODULE_DEVICE_TABLE(of, brcm_pcie_match);
> > >
> > > +static const struct dev_pm_ops brcm_pcie_pm_ops = {
> > > +     .suspend_noirq = brcm_pcie_suspend,
> > > +     .resume_noirq = brcm_pcie_resume,
> >
> > Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
> > and 1352 of .suspend in the tree.
>
> I will test switching this to  suspend_late/resume_early.

Why not just the 'regular' flavor suspend/resume?

Rob

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Rob Herring]

On Thu, Sep 10, 2020 at 10:42 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
>
> On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
> >
> > On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
> > > From: Jim Quinlan <jquinlan@broadcom.com>
> > >
> > > Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
> > > and resume.  Now the PCIe driver may do so as well.
> > >
> > > Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> > > Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> > > ---
> > >  drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
> > >  1 file changed, 47 insertions(+)
> > >
> > > diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> > > index c2b3d2946a36..3d588ab7a6dd 100644
> > > --- a/drivers/pci/controller/pcie-brcmstb.c
> > > +++ b/drivers/pci/controller/pcie-brcmstb.c
> > > @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
> > >       brcm_pcie_bridge_sw_init_set(pcie, 1);
> > >  }
> > >
> > > +static int brcm_pcie_suspend(struct device *dev)
> > > +{
> > > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > > +
> > > +     brcm_pcie_turn_off(pcie);
> > > +     clk_disable_unprepare(pcie->clk);
> > > +
> > > +     return 0;
> > > +}
> > > +
> > > +static int brcm_pcie_resume(struct device *dev)
> > > +{
> > > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > > +     void __iomem *base;
> > > +     u32 tmp;
> > > +     int ret;
> > > +
> > > +     base = pcie->base;
> > > +     clk_prepare_enable(pcie->clk);
> > > +
> > > +     /* Take bridge out of reset so we can access the SERDES reg */
> > > +     brcm_pcie_bridge_sw_init_set(pcie, 0);
> > > +
> > > +     /* SERDES_IDDQ = 0 */
> > > +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > > +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
> > > +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > > +
> > > +     /* wait for serdes to be stable */
> > > +     udelay(100);
> >
> > Really needs to be a spinloop?
> >
> > > +
> > > +     ret = brcm_pcie_setup(pcie);
> > > +     if (ret)
> > > +             return ret;
> > > +
> > > +     if (pcie->msi)
> > > +             brcm_msi_set_regs(pcie->msi);
> > > +
> > > +     return 0;
> > > +}
> > > +
> > >  static void __brcm_pcie_remove(struct brcm_pcie *pcie)
> > >  {
> > >       brcm_msi_remove(pcie);
> > > @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
> > >
> > >  MODULE_DEVICE_TABLE(of, brcm_pcie_match);
> > >
> > > +static const struct dev_pm_ops brcm_pcie_pm_ops = {
> > > +     .suspend_noirq = brcm_pcie_suspend,
> > > +     .resume_noirq = brcm_pcie_resume,
> >
> > Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
> > and 1352 of .suspend in the tree.
>
> I will test switching this to  suspend_late/resume_early.

Why not just the 'regular' flavor suspend/resume?

Rob

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Florian Fainelli]

On 9/10/2020 11:50 AM, Rob Herring wrote:
> On Thu, Sep 10, 2020 at 10:42 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
>>
>> On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
>>>
>>> On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
>>>> From: Jim Quinlan <jquinlan@broadcom.com>
>>>>
>>>> Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
>>>> and resume.  Now the PCIe driver may do so as well.
>>>>
>>>> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
>>>> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
>>>> ---
>>>>   drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
>>>>   1 file changed, 47 insertions(+)
>>>>
>>>> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
>>>> index c2b3d2946a36..3d588ab7a6dd 100644
>>>> --- a/drivers/pci/controller/pcie-brcmstb.c
>>>> +++ b/drivers/pci/controller/pcie-brcmstb.c
>>>> @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>>>>        brcm_pcie_bridge_sw_init_set(pcie, 1);
>>>>   }
>>>>
>>>> +static int brcm_pcie_suspend(struct device *dev)
>>>> +{
>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
>>>> +
>>>> +     brcm_pcie_turn_off(pcie);
>>>> +     clk_disable_unprepare(pcie->clk);
>>>> +
>>>> +     return 0;
>>>> +}
>>>> +
>>>> +static int brcm_pcie_resume(struct device *dev)
>>>> +{
>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
>>>> +     void __iomem *base;
>>>> +     u32 tmp;
>>>> +     int ret;
>>>> +
>>>> +     base = pcie->base;
>>>> +     clk_prepare_enable(pcie->clk);
>>>> +
>>>> +     /* Take bridge out of reset so we can access the SERDES reg */
>>>> +     brcm_pcie_bridge_sw_init_set(pcie, 0);
>>>> +
>>>> +     /* SERDES_IDDQ = 0 */
>>>> +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>>>> +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
>>>> +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>>>> +
>>>> +     /* wait for serdes to be stable */
>>>> +     udelay(100);
>>>
>>> Really needs to be a spinloop?
>>>
>>>> +
>>>> +     ret = brcm_pcie_setup(pcie);
>>>> +     if (ret)
>>>> +             return ret;
>>>> +
>>>> +     if (pcie->msi)
>>>> +             brcm_msi_set_regs(pcie->msi);
>>>> +
>>>> +     return 0;
>>>> +}
>>>> +
>>>>   static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>>>>   {
>>>>        brcm_msi_remove(pcie);
>>>> @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>>>>
>>>>   MODULE_DEVICE_TABLE(of, brcm_pcie_match);
>>>>
>>>> +static const struct dev_pm_ops brcm_pcie_pm_ops = {
>>>> +     .suspend_noirq = brcm_pcie_suspend,
>>>> +     .resume_noirq = brcm_pcie_resume,
>>>
>>> Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
>>> and 1352 of .suspend in the tree.
>>
>> I will test switching this to  suspend_late/resume_early.
> 
> Why not just the 'regular' flavor suspend/resume?

We must have inherited this from when the driver was not a 
platform_device back in our 3.14 downstream kernel and we used 
syscore_ops to do the system suspend/resume.

Later on, we sort of mechanically made those _noirq() to preserve the 
semantics of syscore_ops, but in hindsight it should not be necessary, 
the regular suspsend/resume should work and the device driver model 
ordering between parent/child should take care of the bridge being 
suspended last within the PCI bus type that is.
-- 
Florian

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Florian Fainelli]

On 9/10/2020 11:50 AM, Rob Herring wrote:
> On Thu, Sep 10, 2020 at 10:42 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
>>
>> On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
>>>
>>> On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
>>>> From: Jim Quinlan <jquinlan@broadcom.com>
>>>>
>>>> Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
>>>> and resume.  Now the PCIe driver may do so as well.
>>>>
>>>> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
>>>> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
>>>> ---
>>>>   drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
>>>>   1 file changed, 47 insertions(+)
>>>>
>>>> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
>>>> index c2b3d2946a36..3d588ab7a6dd 100644
>>>> --- a/drivers/pci/controller/pcie-brcmstb.c
>>>> +++ b/drivers/pci/controller/pcie-brcmstb.c
>>>> @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>>>>        brcm_pcie_bridge_sw_init_set(pcie, 1);
>>>>   }
>>>>
>>>> +static int brcm_pcie_suspend(struct device *dev)
>>>> +{
>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
>>>> +
>>>> +     brcm_pcie_turn_off(pcie);
>>>> +     clk_disable_unprepare(pcie->clk);
>>>> +
>>>> +     return 0;
>>>> +}
>>>> +
>>>> +static int brcm_pcie_resume(struct device *dev)
>>>> +{
>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
>>>> +     void __iomem *base;
>>>> +     u32 tmp;
>>>> +     int ret;
>>>> +
>>>> +     base = pcie->base;
>>>> +     clk_prepare_enable(pcie->clk);
>>>> +
>>>> +     /* Take bridge out of reset so we can access the SERDES reg */
>>>> +     brcm_pcie_bridge_sw_init_set(pcie, 0);
>>>> +
>>>> +     /* SERDES_IDDQ = 0 */
>>>> +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>>>> +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
>>>> +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>>>> +
>>>> +     /* wait for serdes to be stable */
>>>> +     udelay(100);
>>>
>>> Really needs to be a spinloop?
>>>
>>>> +
>>>> +     ret = brcm_pcie_setup(pcie);
>>>> +     if (ret)
>>>> +             return ret;
>>>> +
>>>> +     if (pcie->msi)
>>>> +             brcm_msi_set_regs(pcie->msi);
>>>> +
>>>> +     return 0;
>>>> +}
>>>> +
>>>>   static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>>>>   {
>>>>        brcm_msi_remove(pcie);
>>>> @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>>>>
>>>>   MODULE_DEVICE_TABLE(of, brcm_pcie_match);
>>>>
>>>> +static const struct dev_pm_ops brcm_pcie_pm_ops = {
>>>> +     .suspend_noirq = brcm_pcie_suspend,
>>>> +     .resume_noirq = brcm_pcie_resume,
>>>
>>> Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
>>> and 1352 of .suspend in the tree.
>>
>> I will test switching this to  suspend_late/resume_early.
> 
> Why not just the 'regular' flavor suspend/resume?

We must have inherited this from when the driver was not a 
platform_device back in our 3.14 downstream kernel and we used 
syscore_ops to do the system suspend/resume.

Later on, we sort of mechanically made those _noirq() to preserve the 
semantics of syscore_ops, but in hindsight it should not be necessary, 
the regular suspsend/resume should work and the device driver model 
ordering between parent/child should take care of the bridge being 
suspended last within the PCI bus type that is.
-- 
Florian

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Jim Quinlan]

[-- Attachment #1: Type: text/plain, Size: 3467 bytes --]

On Thu, Sep 10, 2020 at 2:50 PM Rob Herring <robh@kernel.org> wrote:
>
> On Thu, Sep 10, 2020 at 10:42 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
> >
> > On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
> > >
> > > On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
> > > > From: Jim Quinlan <jquinlan@broadcom.com>
> > > >
> > > > Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
> > > > and resume.  Now the PCIe driver may do so as well.
> > > >
> > > > Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> > > > Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> > > > ---
> > > >  drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
> > > >  1 file changed, 47 insertions(+)
> > > >
> > > > diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> > > > index c2b3d2946a36..3d588ab7a6dd 100644
> > > > --- a/drivers/pci/controller/pcie-brcmstb.c
> > > > +++ b/drivers/pci/controller/pcie-brcmstb.c
> > > > @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
> > > >       brcm_pcie_bridge_sw_init_set(pcie, 1);
> > > >  }
> > > >
> > > > +static int brcm_pcie_suspend(struct device *dev)
> > > > +{
> > > > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > > > +
> > > > +     brcm_pcie_turn_off(pcie);
> > > > +     clk_disable_unprepare(pcie->clk);
> > > > +
> > > > +     return 0;
> > > > +}
> > > > +
> > > > +static int brcm_pcie_resume(struct device *dev)
> > > > +{
> > > > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > > > +     void __iomem *base;
> > > > +     u32 tmp;
> > > > +     int ret;
> > > > +
> > > > +     base = pcie->base;
> > > > +     clk_prepare_enable(pcie->clk);
> > > > +
> > > > +     /* Take bridge out of reset so we can access the SERDES reg */
> > > > +     brcm_pcie_bridge_sw_init_set(pcie, 0);
> > > > +
> > > > +     /* SERDES_IDDQ = 0 */
> > > > +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > > > +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
> > > > +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > > > +
> > > > +     /* wait for serdes to be stable */
> > > > +     udelay(100);
> > >
> > > Really needs to be a spinloop?
> > >
> > > > +
> > > > +     ret = brcm_pcie_setup(pcie);
> > > > +     if (ret)
> > > > +             return ret;
> > > > +
> > > > +     if (pcie->msi)
> > > > +             brcm_msi_set_regs(pcie->msi);
> > > > +
> > > > +     return 0;
> > > > +}
> > > > +
> > > >  static void __brcm_pcie_remove(struct brcm_pcie *pcie)
> > > >  {
> > > >       brcm_msi_remove(pcie);
> > > > @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
> > > >
> > > >  MODULE_DEVICE_TABLE(of, brcm_pcie_match);
> > > >
> > > > +static const struct dev_pm_ops brcm_pcie_pm_ops = {
> > > > +     .suspend_noirq = brcm_pcie_suspend,
> > > > +     .resume_noirq = brcm_pcie_resume,
> > >
> > > Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
> > > and 1352 of .suspend in the tree.
> >
> > I will test switching this to  suspend_late/resume_early.
>
> Why not just the 'regular' flavor suspend/resume?
>
> Rob
We must have our PCIe driver suspend last and resume first because our
current driver turns off/on the power for the EPs.  Note that this
code isn't in the driver as we are still figuring out a way to make it
upstreamable.

Jim

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4167 bytes --]

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Jim Quinlan]

[-- Attachment #1.1: Type: text/plain, Size: 3467 bytes --]

On Thu, Sep 10, 2020 at 2:50 PM Rob Herring <robh@kernel.org> wrote:
>
> On Thu, Sep 10, 2020 at 10:42 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
> >
> > On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
> > >
> > > On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
> > > > From: Jim Quinlan <jquinlan@broadcom.com>
> > > >
> > > > Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
> > > > and resume.  Now the PCIe driver may do so as well.
> > > >
> > > > Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> > > > Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> > > > ---
> > > >  drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
> > > >  1 file changed, 47 insertions(+)
> > > >
> > > > diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> > > > index c2b3d2946a36..3d588ab7a6dd 100644
> > > > --- a/drivers/pci/controller/pcie-brcmstb.c
> > > > +++ b/drivers/pci/controller/pcie-brcmstb.c
> > > > @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
> > > >       brcm_pcie_bridge_sw_init_set(pcie, 1);
> > > >  }
> > > >
> > > > +static int brcm_pcie_suspend(struct device *dev)
> > > > +{
> > > > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > > > +
> > > > +     brcm_pcie_turn_off(pcie);
> > > > +     clk_disable_unprepare(pcie->clk);
> > > > +
> > > > +     return 0;
> > > > +}
> > > > +
> > > > +static int brcm_pcie_resume(struct device *dev)
> > > > +{
> > > > +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> > > > +     void __iomem *base;
> > > > +     u32 tmp;
> > > > +     int ret;
> > > > +
> > > > +     base = pcie->base;
> > > > +     clk_prepare_enable(pcie->clk);
> > > > +
> > > > +     /* Take bridge out of reset so we can access the SERDES reg */
> > > > +     brcm_pcie_bridge_sw_init_set(pcie, 0);
> > > > +
> > > > +     /* SERDES_IDDQ = 0 */
> > > > +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > > > +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
> > > > +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> > > > +
> > > > +     /* wait for serdes to be stable */
> > > > +     udelay(100);
> > >
> > > Really needs to be a spinloop?
> > >
> > > > +
> > > > +     ret = brcm_pcie_setup(pcie);
> > > > +     if (ret)
> > > > +             return ret;
> > > > +
> > > > +     if (pcie->msi)
> > > > +             brcm_msi_set_regs(pcie->msi);
> > > > +
> > > > +     return 0;
> > > > +}
> > > > +
> > > >  static void __brcm_pcie_remove(struct brcm_pcie *pcie)
> > > >  {
> > > >       brcm_msi_remove(pcie);
> > > > @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
> > > >
> > > >  MODULE_DEVICE_TABLE(of, brcm_pcie_match);
> > > >
> > > > +static const struct dev_pm_ops brcm_pcie_pm_ops = {
> > > > +     .suspend_noirq = brcm_pcie_suspend,
> > > > +     .resume_noirq = brcm_pcie_resume,
> > >
> > > Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
> > > and 1352 of .suspend in the tree.
> >
> > I will test switching this to  suspend_late/resume_early.
>
> Why not just the 'regular' flavor suspend/resume?
>
> Rob
We must have our PCIe driver suspend last and resume first because our
current driver turns off/on the power for the EPs.  Note that this
code isn't in the driver as we are still figuring out a way to make it
upstreamable.

Jim

[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4167 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Florian Fainelli]

On 9/10/2020 12:05 PM, Jim Quinlan wrote:
> On Thu, Sep 10, 2020 at 2:50 PM Rob Herring <robh@kernel.org> wrote:
>>
>> On Thu, Sep 10, 2020 at 10:42 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
>>>
>>> On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
>>>>
>>>> On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
>>>>> From: Jim Quinlan <jquinlan@broadcom.com>
>>>>>
>>>>> Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
>>>>> and resume.  Now the PCIe driver may do so as well.
>>>>>
>>>>> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
>>>>> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
>>>>> ---
>>>>>   drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
>>>>>   1 file changed, 47 insertions(+)
>>>>>
>>>>> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
>>>>> index c2b3d2946a36..3d588ab7a6dd 100644
>>>>> --- a/drivers/pci/controller/pcie-brcmstb.c
>>>>> +++ b/drivers/pci/controller/pcie-brcmstb.c
>>>>> @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>>>>>        brcm_pcie_bridge_sw_init_set(pcie, 1);
>>>>>   }
>>>>>
>>>>> +static int brcm_pcie_suspend(struct device *dev)
>>>>> +{
>>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
>>>>> +
>>>>> +     brcm_pcie_turn_off(pcie);
>>>>> +     clk_disable_unprepare(pcie->clk);
>>>>> +
>>>>> +     return 0;
>>>>> +}
>>>>> +
>>>>> +static int brcm_pcie_resume(struct device *dev)
>>>>> +{
>>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
>>>>> +     void __iomem *base;
>>>>> +     u32 tmp;
>>>>> +     int ret;
>>>>> +
>>>>> +     base = pcie->base;
>>>>> +     clk_prepare_enable(pcie->clk);
>>>>> +
>>>>> +     /* Take bridge out of reset so we can access the SERDES reg */
>>>>> +     brcm_pcie_bridge_sw_init_set(pcie, 0);
>>>>> +
>>>>> +     /* SERDES_IDDQ = 0 */
>>>>> +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>>>>> +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
>>>>> +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>>>>> +
>>>>> +     /* wait for serdes to be stable */
>>>>> +     udelay(100);
>>>>
>>>> Really needs to be a spinloop?
>>>>
>>>>> +
>>>>> +     ret = brcm_pcie_setup(pcie);
>>>>> +     if (ret)
>>>>> +             return ret;
>>>>> +
>>>>> +     if (pcie->msi)
>>>>> +             brcm_msi_set_regs(pcie->msi);
>>>>> +
>>>>> +     return 0;
>>>>> +}
>>>>> +
>>>>>   static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>>>>>   {
>>>>>        brcm_msi_remove(pcie);
>>>>> @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>>>>>
>>>>>   MODULE_DEVICE_TABLE(of, brcm_pcie_match);
>>>>>
>>>>> +static const struct dev_pm_ops brcm_pcie_pm_ops = {
>>>>> +     .suspend_noirq = brcm_pcie_suspend,
>>>>> +     .resume_noirq = brcm_pcie_resume,
>>>>
>>>> Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
>>>> and 1352 of .suspend in the tree.
>>>
>>> I will test switching this to  suspend_late/resume_early.
>>
>> Why not just the 'regular' flavor suspend/resume?
>>
>> Rob
> We must have our PCIe driver suspend last and resume first because our
> current driver turns off/on the power for the EPs.  Note that this
> code isn't in the driver as we are still figuring out a way to make it
> upstreamable.

The suspend/resume ordering should be guaranteed by the Linux device 
driver model though if not, this is a bug that ought to be fixed. The 
PCI bridge sits at the top of the pci_device list and all EPs should be 
child devices, so the suspend order should be from EPs down to the 
bridge, and the resume the converse.
-- 
Florian

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Florian Fainelli]

On 9/10/2020 12:05 PM, Jim Quinlan wrote:
> On Thu, Sep 10, 2020 at 2:50 PM Rob Herring <robh@kernel.org> wrote:
>>
>> On Thu, Sep 10, 2020 at 10:42 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
>>>
>>> On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
>>>>
>>>> On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
>>>>> From: Jim Quinlan <jquinlan@broadcom.com>
>>>>>
>>>>> Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
>>>>> and resume.  Now the PCIe driver may do so as well.
>>>>>
>>>>> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
>>>>> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
>>>>> ---
>>>>>   drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
>>>>>   1 file changed, 47 insertions(+)
>>>>>
>>>>> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
>>>>> index c2b3d2946a36..3d588ab7a6dd 100644
>>>>> --- a/drivers/pci/controller/pcie-brcmstb.c
>>>>> +++ b/drivers/pci/controller/pcie-brcmstb.c
>>>>> @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
>>>>>        brcm_pcie_bridge_sw_init_set(pcie, 1);
>>>>>   }
>>>>>
>>>>> +static int brcm_pcie_suspend(struct device *dev)
>>>>> +{
>>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
>>>>> +
>>>>> +     brcm_pcie_turn_off(pcie);
>>>>> +     clk_disable_unprepare(pcie->clk);
>>>>> +
>>>>> +     return 0;
>>>>> +}
>>>>> +
>>>>> +static int brcm_pcie_resume(struct device *dev)
>>>>> +{
>>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
>>>>> +     void __iomem *base;
>>>>> +     u32 tmp;
>>>>> +     int ret;
>>>>> +
>>>>> +     base = pcie->base;
>>>>> +     clk_prepare_enable(pcie->clk);
>>>>> +
>>>>> +     /* Take bridge out of reset so we can access the SERDES reg */
>>>>> +     brcm_pcie_bridge_sw_init_set(pcie, 0);
>>>>> +
>>>>> +     /* SERDES_IDDQ = 0 */
>>>>> +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>>>>> +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
>>>>> +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
>>>>> +
>>>>> +     /* wait for serdes to be stable */
>>>>> +     udelay(100);
>>>>
>>>> Really needs to be a spinloop?
>>>>
>>>>> +
>>>>> +     ret = brcm_pcie_setup(pcie);
>>>>> +     if (ret)
>>>>> +             return ret;
>>>>> +
>>>>> +     if (pcie->msi)
>>>>> +             brcm_msi_set_regs(pcie->msi);
>>>>> +
>>>>> +     return 0;
>>>>> +}
>>>>> +
>>>>>   static void __brcm_pcie_remove(struct brcm_pcie *pcie)
>>>>>   {
>>>>>        brcm_msi_remove(pcie);
>>>>> @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
>>>>>
>>>>>   MODULE_DEVICE_TABLE(of, brcm_pcie_match);
>>>>>
>>>>> +static const struct dev_pm_ops brcm_pcie_pm_ops = {
>>>>> +     .suspend_noirq = brcm_pcie_suspend,
>>>>> +     .resume_noirq = brcm_pcie_resume,
>>>>
>>>> Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
>>>> and 1352 of .suspend in the tree.
>>>
>>> I will test switching this to  suspend_late/resume_early.
>>
>> Why not just the 'regular' flavor suspend/resume?
>>
>> Rob
> We must have our PCIe driver suspend last and resume first because our
> current driver turns off/on the power for the EPs.  Note that this
> code isn't in the driver as we are still figuring out a way to make it
> upstreamable.

The suspend/resume ordering should be guaranteed by the Linux device 
driver model though if not, this is a bug that ought to be fixed. The 
PCI bridge sits at the top of the pci_device list and all EPs should be 
child devices, so the suspend order should be from EPs down to the 
bridge, and the resume the converse.
-- 
Florian

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Jim Quinlan]

[-- Attachment #1: Type: text/plain, Size: 4035 bytes --]

On Thu, Sep 10, 2020 at 3:08 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
>
>
>
> On 9/10/2020 12:05 PM, Jim Quinlan wrote:
> > On Thu, Sep 10, 2020 at 2:50 PM Rob Herring <robh@kernel.org> wrote:
> >>
> >> On Thu, Sep 10, 2020 at 10:42 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
> >>>
> >>> On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
> >>>>
> >>>> On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
> >>>>> From: Jim Quinlan <jquinlan@broadcom.com>
> >>>>>
> >>>>> Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
> >>>>> and resume.  Now the PCIe driver may do so as well.
> >>>>>
> >>>>> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> >>>>> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> >>>>> ---
> >>>>>   drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
> >>>>>   1 file changed, 47 insertions(+)
> >>>>>
> >>>>> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> >>>>> index c2b3d2946a36..3d588ab7a6dd 100644
> >>>>> --- a/drivers/pci/controller/pcie-brcmstb.c
> >>>>> +++ b/drivers/pci/controller/pcie-brcmstb.c
> >>>>> @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
> >>>>>        brcm_pcie_bridge_sw_init_set(pcie, 1);
> >>>>>   }
> >>>>>
> >>>>> +static int brcm_pcie_suspend(struct device *dev)
> >>>>> +{
> >>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> >>>>> +
> >>>>> +     brcm_pcie_turn_off(pcie);
> >>>>> +     clk_disable_unprepare(pcie->clk);
> >>>>> +
> >>>>> +     return 0;
> >>>>> +}
> >>>>> +
> >>>>> +static int brcm_pcie_resume(struct device *dev)
> >>>>> +{
> >>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> >>>>> +     void __iomem *base;
> >>>>> +     u32 tmp;
> >>>>> +     int ret;
> >>>>> +
> >>>>> +     base = pcie->base;
> >>>>> +     clk_prepare_enable(pcie->clk);
> >>>>> +
> >>>>> +     /* Take bridge out of reset so we can access the SERDES reg */
> >>>>> +     brcm_pcie_bridge_sw_init_set(pcie, 0);
> >>>>> +
> >>>>> +     /* SERDES_IDDQ = 0 */
> >>>>> +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> >>>>> +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
> >>>>> +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> >>>>> +
> >>>>> +     /* wait for serdes to be stable */
> >>>>> +     udelay(100);
> >>>>
> >>>> Really needs to be a spinloop?
> >>>>
> >>>>> +
> >>>>> +     ret = brcm_pcie_setup(pcie);
> >>>>> +     if (ret)
> >>>>> +             return ret;
> >>>>> +
> >>>>> +     if (pcie->msi)
> >>>>> +             brcm_msi_set_regs(pcie->msi);
> >>>>> +
> >>>>> +     return 0;
> >>>>> +}
> >>>>> +
> >>>>>   static void __brcm_pcie_remove(struct brcm_pcie *pcie)
> >>>>>   {
> >>>>>        brcm_msi_remove(pcie);
> >>>>> @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
> >>>>>
> >>>>>   MODULE_DEVICE_TABLE(of, brcm_pcie_match);
> >>>>>
> >>>>> +static const struct dev_pm_ops brcm_pcie_pm_ops = {
> >>>>> +     .suspend_noirq = brcm_pcie_suspend,
> >>>>> +     .resume_noirq = brcm_pcie_resume,
> >>>>
> >>>> Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
> >>>> and 1352 of .suspend in the tree.
> >>>
> >>> I will test switching this to  suspend_late/resume_early.
> >>
> >> Why not just the 'regular' flavor suspend/resume?
> >>
> >> Rob
> > We must have our PCIe driver suspend last and resume first because our
> > current driver turns off/on the power for the EPs.  Note that this
> > code isn't in the driver as we are still figuring out a way to make it
> > upstreamable.
>
> The suspend/resume ordering should be guaranteed by the Linux device
> driver model though if not, this is a bug that ought to be fixed. The
> PCI bridge sits at the top of the pci_device list and all EPs should be
> child devices, so the suspend order should be from EPs down to the
> bridge, and the resume the converse.
I remembered that after I hit send.
Jim
> --
> Florian

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4167 bytes --]

Re: [PATCH v11 04/11] PCI: brcmstb: Add suspend and resume pm_ops

[Jim Quinlan]

[-- Attachment #1.1: Type: text/plain, Size: 4035 bytes --]

On Thu, Sep 10, 2020 at 3:08 PM Florian Fainelli <f.fainelli@gmail.com> wrote:
>
>
>
> On 9/10/2020 12:05 PM, Jim Quinlan wrote:
> > On Thu, Sep 10, 2020 at 2:50 PM Rob Herring <robh@kernel.org> wrote:
> >>
> >> On Thu, Sep 10, 2020 at 10:42 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
> >>>
> >>> On Thu, Sep 10, 2020 at 11:56 AM Rob Herring <robh@kernel.org> wrote:
> >>>>
> >>>> On Mon, Aug 24, 2020 at 03:30:17PM -0400, Jim Quinlan wrote:
> >>>>> From: Jim Quinlan <jquinlan@broadcom.com>
> >>>>>
> >>>>> Broadcom Set-top (BrcmSTB) boards typically support S2, S3, and S5 suspend
> >>>>> and resume.  Now the PCIe driver may do so as well.
> >>>>>
> >>>>> Signed-off-by: Jim Quinlan <jquinlan@broadcom.com>
> >>>>> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> >>>>> ---
> >>>>>   drivers/pci/controller/pcie-brcmstb.c | 47 +++++++++++++++++++++++++++
> >>>>>   1 file changed, 47 insertions(+)
> >>>>>
> >>>>> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> >>>>> index c2b3d2946a36..3d588ab7a6dd 100644
> >>>>> --- a/drivers/pci/controller/pcie-brcmstb.c
> >>>>> +++ b/drivers/pci/controller/pcie-brcmstb.c
> >>>>> @@ -978,6 +978,47 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
> >>>>>        brcm_pcie_bridge_sw_init_set(pcie, 1);
> >>>>>   }
> >>>>>
> >>>>> +static int brcm_pcie_suspend(struct device *dev)
> >>>>> +{
> >>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> >>>>> +
> >>>>> +     brcm_pcie_turn_off(pcie);
> >>>>> +     clk_disable_unprepare(pcie->clk);
> >>>>> +
> >>>>> +     return 0;
> >>>>> +}
> >>>>> +
> >>>>> +static int brcm_pcie_resume(struct device *dev)
> >>>>> +{
> >>>>> +     struct brcm_pcie *pcie = dev_get_drvdata(dev);
> >>>>> +     void __iomem *base;
> >>>>> +     u32 tmp;
> >>>>> +     int ret;
> >>>>> +
> >>>>> +     base = pcie->base;
> >>>>> +     clk_prepare_enable(pcie->clk);
> >>>>> +
> >>>>> +     /* Take bridge out of reset so we can access the SERDES reg */
> >>>>> +     brcm_pcie_bridge_sw_init_set(pcie, 0);
> >>>>> +
> >>>>> +     /* SERDES_IDDQ = 0 */
> >>>>> +     tmp = readl(base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> >>>>> +     u32p_replace_bits(&tmp, 0, PCIE_MISC_HARD_PCIE_HARD_DEBUG_SERDES_IDDQ_MASK);
> >>>>> +     writel(tmp, base + PCIE_MISC_HARD_PCIE_HARD_DEBUG);
> >>>>> +
> >>>>> +     /* wait for serdes to be stable */
> >>>>> +     udelay(100);
> >>>>
> >>>> Really needs to be a spinloop?
> >>>>
> >>>>> +
> >>>>> +     ret = brcm_pcie_setup(pcie);
> >>>>> +     if (ret)
> >>>>> +             return ret;
> >>>>> +
> >>>>> +     if (pcie->msi)
> >>>>> +             brcm_msi_set_regs(pcie->msi);
> >>>>> +
> >>>>> +     return 0;
> >>>>> +}
> >>>>> +
> >>>>>   static void __brcm_pcie_remove(struct brcm_pcie *pcie)
> >>>>>   {
> >>>>>        brcm_msi_remove(pcie);
> >>>>> @@ -1087,12 +1128,18 @@ static int brcm_pcie_probe(struct platform_device *pdev)
> >>>>>
> >>>>>   MODULE_DEVICE_TABLE(of, brcm_pcie_match);
> >>>>>
> >>>>> +static const struct dev_pm_ops brcm_pcie_pm_ops = {
> >>>>> +     .suspend_noirq = brcm_pcie_suspend,
> >>>>> +     .resume_noirq = brcm_pcie_resume,
> >>>>
> >>>> Why do you need interrupts disabled? There's 39 cases of .suspend_noirq
> >>>> and 1352 of .suspend in the tree.
> >>>
> >>> I will test switching this to  suspend_late/resume_early.
> >>
> >> Why not just the 'regular' flavor suspend/resume?
> >>
> >> Rob
> > We must have our PCIe driver suspend last and resume first because our
> > current driver turns off/on the power for the EPs.  Note that this
> > code isn't in the driver as we are still figuring out a way to make it
> > upstreamable.
>
> The suspend/resume ordering should be guaranteed by the Linux device
> driver model though if not, this is a bug that ought to be fixed. The
> PCI bridge sits at the top of the pci_device list and all EPs should be
> child devices, so the suspend order should be from EPs down to the
> bridge, and the resume the converse.
I remembered that after I hit send.
Jim
> --
> Florian

[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4167 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 08/11] PCI: brcmstb: Set additional internal memory DMA viewport sizes

[Jim Quinlan]

[-- Attachment #1: Type: text/plain, Size: 9370 bytes --]

On Thu, Sep 10, 2020 at 12:17 PM Rob Herring <robh@kernel.org> wrote:
>
> On Mon, Aug 24, 2020 at 03:30:21PM -0400, Jim Quinlan wrote:
> > The Raspberry Pi (RPI) is currently the only chip using this driver
> > (pcie-brcmstb.c).  There, only one memory controller is used, without an
> > extension region, and the SCB0 viewport size is set to the size of the
> > first and only dma-range region.  Other BrcmSTB SOCs have more complicated
> > memory configurations that require setting additional viewport sizes.
> >
> > BrcmSTB PCIe controllers are intimately connected to the memory
> > controller(s) on the SOC.  The SOC may have one to three memory
> > controllers; they are indicated by the term SCBi.  Each controller has a
> > base region and an optional extension region.  In physical memory, the base
> > and extension regions of a controller are not adjacent, but in PCIe-space
> > they are.
> >
> > There is a "viewport" for each memory controller that allows DMA from
> > endpoint devices.  Each viewport's size must be set to a power of two, and
> > that size must be equal to or larger than the amount of memory each
> > controller supports which is the sum of base region and its optional
> > extension.  Further, the 1-3 viewports are also adjacent in PCIe-space.
> >
> > Unfortunately the viewport sizes cannot be ascertained from the
> > "dma-ranges" property so they have their own property, "brcm,scb-sizes".
> > This is because dma-range information does not indicate what memory
> > controller it is associated.  For example, consider the following case
> > where the size of one dma-range is 2GB and the second dma-range is 1GB:
> >
> >     /* Case 1: SCB0 size set to 4GB */
> >     dma-range0: 2GB (from memc0-base)
> >     dma-range1: 1GB (from memc0-extension)
> >
> >     /* Case 2: SCB0 size set to 2GB, SCB1 size set to 1GB */
> >     dma-range0: 2GB (from memc0-base)
> >     dma-range1: 1GB (from memc0-extension)
> >
> > By just looking at the dma-ranges information, one cannot tell which
> > situation applies. That is why an additional property is needed.  Its
> > length indicates the number of memory controllers being used and each value
> > indicates the viewport size.
> >
> > Note that the RPI DT does not have a "brcm,scb-sizes" property value,
> > as it is assumed that it only requires one memory controller and no
> > extension.  So the optional use of "brcm,scb-sizes" will be backwards
> > compatible.
> >
> > One last layer of complexity exists: all of the viewports sizes must be
> > added and rounded up to a power of two to determine what the "BAR" size is.
> > Further, an offset must be given that indicates the base PCIe address of
> > this "BAR".  The use of the term BAR is typically associated with endpoint
> > devices, and the term is used here because the PCIe HW may be used as an RC
> > or an EP.  In the former case, all of the system memory appears in a single
> > "BAR" region in PCIe memory.  As it turns out, BrcmSTB PCIe HW is rarely
> > used in the EP role and its system of mapping memory is an artifact that
> > requires multiple dma-ranges regions.
> >
> > Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> > Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> > ---
> >  drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++-------
> >  1 file changed, 50 insertions(+), 18 deletions(-)
> >
> > diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> > index 041b8d109563..7150eaa803c2 100644
> > --- a/drivers/pci/controller/pcie-brcmstb.c
> > +++ b/drivers/pci/controller/pcie-brcmstb.c
> > @@ -57,6 +57,8 @@
> >  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK     0x300000
> >  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128              0x0
> >  #define  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK          0xf8000000
> > +#define  PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK          0x07c00000
> > +#define  PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK          0x0000001f
>
> Perhaps make 0-2 an arg and then you can just do:
>
> u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB_SIZE_MASK(memc))

I cannot get this to work.  In this case u32p_replace_bits requires
that the mask is a compile-time constant; when "memc" is a variable I
don't see how to do this.
>
> >
> >  #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO             0x400c
> >  #define PCIE_MEM_WIN0_LO(win)        \
> > @@ -154,6 +156,7 @@
> >  #define SSC_STATUS_OFFSET            0x1
> >  #define SSC_STATUS_SSC_MASK          0x400
> >  #define SSC_STATUS_PLL_LOCK_MASK     0x800
> > +#define PCIE_BRCM_MAX_MEMC           3
> >
> >  #define IDX_ADDR(pcie)                       (pcie->reg_offsets[EXT_CFG_INDEX])
> >  #define DATA_ADDR(pcie)                      (pcie->reg_offsets[EXT_CFG_DATA])
> > @@ -259,6 +262,8 @@ struct brcm_pcie {
> >       const int               *reg_field_info;
> >       enum pcie_type          type;
> >       struct reset_control    *rescal;
> > +     int                     num_memc;
> > +     u64                     memc_size[PCIE_BRCM_MAX_MEMC];
> >  };
> >
> >  /*
> > @@ -714,22 +719,44 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
> >                                                       u64 *rc_bar2_offset)
> >  {
> >       struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> > -     struct device *dev = pcie->dev;
> >       struct resource_entry *entry;
> > +     struct device *dev = pcie->dev;
> > +     u64 lowest_pcie_addr = ~(u64)0;
> > +     int ret, i = 0;
> > +     u64 size = 0;
> >
> > -     entry = resource_list_first_type(&bridge->dma_ranges, IORESOURCE_MEM);
> > -     if (!entry)
> > -             return -ENODEV;
> > +     resource_list_for_each_entry(entry, &bridge->dma_ranges) {
> > +             u64 pcie_beg = entry->res->start - entry->offset;
> >
> > +             size += entry->res->end - entry->res->start + 1;
> > +             if (pcie_beg < lowest_pcie_addr)
> > +                     lowest_pcie_addr = pcie_beg;
> > +     }
> >
> > -     /*
> > -      * The controller expects the inbound window offset to be calculated as
> > -      * the difference between PCIe's address space and CPU's. The offset
> > -      * provided by the firmware is calculated the opposite way, so we
> > -      * negate it.
> > -      */
> > -     *rc_bar2_offset = -entry->offset;
> > -     *rc_bar2_size = 1ULL << fls64(entry->res->end - entry->res->start);
> > +     if (lowest_pcie_addr == ~(u64)0) {
> > +             dev_err(dev, "DT node has no dma-ranges\n");
> > +             return -EINVAL;
> > +     }
> > +
> > +     ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1,
> > +                                               PCIE_BRCM_MAX_MEMC);
> > +
> > +     if (ret <= 0) {
> > +             /* Make an educated guess */
> > +             pcie->num_memc = 1;
> > +             pcie->memc_size[0] = 1ULL << fls64(size - 1);
>
> Use roundup_pow_of_two()
The reason I didn't use roundup_pow_of_two() is that it returns a
ulong which on ARM is 32bits and cannot represent  4GB.

>
> > +     } else {
> > +             pcie->num_memc = ret;
> > +     }
> > +
> > +     /* Each memc is viewed through a "port" that is a power of 2 */
> > +     for (i = 0, size = 0; i < pcie->num_memc; i++)
> > +             size += pcie->memc_size[i];
> > +
> > +     /* System memory starts at this address in PCIe-space */
> > +     *rc_bar2_offset = lowest_pcie_addr;
> > +     /* The sum of all memc views must also be a power of 2 */
> > +     *rc_bar2_size = 1ULL << fls64(size - 1);
>
> Use roundup_pow_of_two()
Ditto

Jim Quinlan
Broadcom STB
>
> >
> >       /*
> >        * We validate the inbound memory view even though we should trust
> > @@ -781,12 +808,11 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
> >       void __iomem *base = pcie->base;
> >       struct device *dev = pcie->dev;
> >       struct resource_entry *entry;
> > -     unsigned int scb_size_val;
> >       bool ssc_good = false;
> >       struct resource *res;
> >       int num_out_wins = 0;
> >       u16 nlw, cls, lnksta;
> > -     int i, ret;
> > +     int i, ret, memc;
> >       u32 tmp, aspm_support;
> >
> >       /* Reset the bridge */
> > @@ -826,11 +852,17 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
> >       writel(upper_32_bits(rc_bar2_offset),
> >              base + PCIE_MISC_RC_BAR2_CONFIG_HI);
> >
> > -     scb_size_val = rc_bar2_size ?
> > -                    ilog2(rc_bar2_size) - 15 : 0xf; /* 0xf is 1GB */
> >       tmp = readl(base + PCIE_MISC_MISC_CTRL);
> > -     u32p_replace_bits(&tmp, scb_size_val,
> > -                       PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
> > +     for (memc = 0; memc < pcie->num_memc; memc++) {
> > +             u32 scb_size_val = ilog2(pcie->memc_size[memc]) - 15;
> > +
> > +             if (memc == 0)
> > +                     u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
> > +             else if (memc == 1)
> > +                     u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK);
> > +             else if (memc == 2)
> > +                     u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK);
> > +     }
> >       writel(tmp, base + PCIE_MISC_MISC_CTRL);
> >
> >       /*
> > --
> > 2.17.1
> >

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4167 bytes --]

Re: [PATCH v11 08/11] PCI: brcmstb: Set additional internal memory DMA viewport sizes

[Jim Quinlan]

[-- Attachment #1.1: Type: text/plain, Size: 9370 bytes --]

On Thu, Sep 10, 2020 at 12:17 PM Rob Herring <robh@kernel.org> wrote:
>
> On Mon, Aug 24, 2020 at 03:30:21PM -0400, Jim Quinlan wrote:
> > The Raspberry Pi (RPI) is currently the only chip using this driver
> > (pcie-brcmstb.c).  There, only one memory controller is used, without an
> > extension region, and the SCB0 viewport size is set to the size of the
> > first and only dma-range region.  Other BrcmSTB SOCs have more complicated
> > memory configurations that require setting additional viewport sizes.
> >
> > BrcmSTB PCIe controllers are intimately connected to the memory
> > controller(s) on the SOC.  The SOC may have one to three memory
> > controllers; they are indicated by the term SCBi.  Each controller has a
> > base region and an optional extension region.  In physical memory, the base
> > and extension regions of a controller are not adjacent, but in PCIe-space
> > they are.
> >
> > There is a "viewport" for each memory controller that allows DMA from
> > endpoint devices.  Each viewport's size must be set to a power of two, and
> > that size must be equal to or larger than the amount of memory each
> > controller supports which is the sum of base region and its optional
> > extension.  Further, the 1-3 viewports are also adjacent in PCIe-space.
> >
> > Unfortunately the viewport sizes cannot be ascertained from the
> > "dma-ranges" property so they have their own property, "brcm,scb-sizes".
> > This is because dma-range information does not indicate what memory
> > controller it is associated.  For example, consider the following case
> > where the size of one dma-range is 2GB and the second dma-range is 1GB:
> >
> >     /* Case 1: SCB0 size set to 4GB */
> >     dma-range0: 2GB (from memc0-base)
> >     dma-range1: 1GB (from memc0-extension)
> >
> >     /* Case 2: SCB0 size set to 2GB, SCB1 size set to 1GB */
> >     dma-range0: 2GB (from memc0-base)
> >     dma-range1: 1GB (from memc0-extension)
> >
> > By just looking at the dma-ranges information, one cannot tell which
> > situation applies. That is why an additional property is needed.  Its
> > length indicates the number of memory controllers being used and each value
> > indicates the viewport size.
> >
> > Note that the RPI DT does not have a "brcm,scb-sizes" property value,
> > as it is assumed that it only requires one memory controller and no
> > extension.  So the optional use of "brcm,scb-sizes" will be backwards
> > compatible.
> >
> > One last layer of complexity exists: all of the viewports sizes must be
> > added and rounded up to a power of two to determine what the "BAR" size is.
> > Further, an offset must be given that indicates the base PCIe address of
> > this "BAR".  The use of the term BAR is typically associated with endpoint
> > devices, and the term is used here because the PCIe HW may be used as an RC
> > or an EP.  In the former case, all of the system memory appears in a single
> > "BAR" region in PCIe memory.  As it turns out, BrcmSTB PCIe HW is rarely
> > used in the EP role and its system of mapping memory is an artifact that
> > requires multiple dma-ranges regions.
> >
> > Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> > Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> > ---
> >  drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++-------
> >  1 file changed, 50 insertions(+), 18 deletions(-)
> >
> > diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> > index 041b8d109563..7150eaa803c2 100644
> > --- a/drivers/pci/controller/pcie-brcmstb.c
> > +++ b/drivers/pci/controller/pcie-brcmstb.c
> > @@ -57,6 +57,8 @@
> >  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK     0x300000
> >  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128              0x0
> >  #define  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK          0xf8000000
> > +#define  PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK          0x07c00000
> > +#define  PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK          0x0000001f
>
> Perhaps make 0-2 an arg and then you can just do:
>
> u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB_SIZE_MASK(memc))

I cannot get this to work.  In this case u32p_replace_bits requires
that the mask is a compile-time constant; when "memc" is a variable I
don't see how to do this.
>
> >
> >  #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO             0x400c
> >  #define PCIE_MEM_WIN0_LO(win)        \
> > @@ -154,6 +156,7 @@
> >  #define SSC_STATUS_OFFSET            0x1
> >  #define SSC_STATUS_SSC_MASK          0x400
> >  #define SSC_STATUS_PLL_LOCK_MASK     0x800
> > +#define PCIE_BRCM_MAX_MEMC           3
> >
> >  #define IDX_ADDR(pcie)                       (pcie->reg_offsets[EXT_CFG_INDEX])
> >  #define DATA_ADDR(pcie)                      (pcie->reg_offsets[EXT_CFG_DATA])
> > @@ -259,6 +262,8 @@ struct brcm_pcie {
> >       const int               *reg_field_info;
> >       enum pcie_type          type;
> >       struct reset_control    *rescal;
> > +     int                     num_memc;
> > +     u64                     memc_size[PCIE_BRCM_MAX_MEMC];
> >  };
> >
> >  /*
> > @@ -714,22 +719,44 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
> >                                                       u64 *rc_bar2_offset)
> >  {
> >       struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> > -     struct device *dev = pcie->dev;
> >       struct resource_entry *entry;
> > +     struct device *dev = pcie->dev;
> > +     u64 lowest_pcie_addr = ~(u64)0;
> > +     int ret, i = 0;
> > +     u64 size = 0;
> >
> > -     entry = resource_list_first_type(&bridge->dma_ranges, IORESOURCE_MEM);
> > -     if (!entry)
> > -             return -ENODEV;
> > +     resource_list_for_each_entry(entry, &bridge->dma_ranges) {
> > +             u64 pcie_beg = entry->res->start - entry->offset;
> >
> > +             size += entry->res->end - entry->res->start + 1;
> > +             if (pcie_beg < lowest_pcie_addr)
> > +                     lowest_pcie_addr = pcie_beg;
> > +     }
> >
> > -     /*
> > -      * The controller expects the inbound window offset to be calculated as
> > -      * the difference between PCIe's address space and CPU's. The offset
> > -      * provided by the firmware is calculated the opposite way, so we
> > -      * negate it.
> > -      */
> > -     *rc_bar2_offset = -entry->offset;
> > -     *rc_bar2_size = 1ULL << fls64(entry->res->end - entry->res->start);
> > +     if (lowest_pcie_addr == ~(u64)0) {
> > +             dev_err(dev, "DT node has no dma-ranges\n");
> > +             return -EINVAL;
> > +     }
> > +
> > +     ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1,
> > +                                               PCIE_BRCM_MAX_MEMC);
> > +
> > +     if (ret <= 0) {
> > +             /* Make an educated guess */
> > +             pcie->num_memc = 1;
> > +             pcie->memc_size[0] = 1ULL << fls64(size - 1);
>
> Use roundup_pow_of_two()
The reason I didn't use roundup_pow_of_two() is that it returns a
ulong which on ARM is 32bits and cannot represent  4GB.

>
> > +     } else {
> > +             pcie->num_memc = ret;
> > +     }
> > +
> > +     /* Each memc is viewed through a "port" that is a power of 2 */
> > +     for (i = 0, size = 0; i < pcie->num_memc; i++)
> > +             size += pcie->memc_size[i];
> > +
> > +     /* System memory starts at this address in PCIe-space */
> > +     *rc_bar2_offset = lowest_pcie_addr;
> > +     /* The sum of all memc views must also be a power of 2 */
> > +     *rc_bar2_size = 1ULL << fls64(size - 1);
>
> Use roundup_pow_of_two()
Ditto

Jim Quinlan
Broadcom STB
>
> >
> >       /*
> >        * We validate the inbound memory view even though we should trust
> > @@ -781,12 +808,11 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
> >       void __iomem *base = pcie->base;
> >       struct device *dev = pcie->dev;
> >       struct resource_entry *entry;
> > -     unsigned int scb_size_val;
> >       bool ssc_good = false;
> >       struct resource *res;
> >       int num_out_wins = 0;
> >       u16 nlw, cls, lnksta;
> > -     int i, ret;
> > +     int i, ret, memc;
> >       u32 tmp, aspm_support;
> >
> >       /* Reset the bridge */
> > @@ -826,11 +852,17 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
> >       writel(upper_32_bits(rc_bar2_offset),
> >              base + PCIE_MISC_RC_BAR2_CONFIG_HI);
> >
> > -     scb_size_val = rc_bar2_size ?
> > -                    ilog2(rc_bar2_size) - 15 : 0xf; /* 0xf is 1GB */
> >       tmp = readl(base + PCIE_MISC_MISC_CTRL);
> > -     u32p_replace_bits(&tmp, scb_size_val,
> > -                       PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
> > +     for (memc = 0; memc < pcie->num_memc; memc++) {
> > +             u32 scb_size_val = ilog2(pcie->memc_size[memc]) - 15;
> > +
> > +             if (memc == 0)
> > +                     u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
> > +             else if (memc == 1)
> > +                     u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK);
> > +             else if (memc == 2)
> > +                     u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK);
> > +     }
> >       writel(tmp, base + PCIE_MISC_MISC_CTRL);
> >
> >       /*
> > --
> > 2.17.1
> >

[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 4167 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v11 08/11] PCI: brcmstb: Set additional internal memory DMA viewport sizes

[Rob Herring]

On Fri, Sep 11, 2020 at 9:28 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
>
> On Thu, Sep 10, 2020 at 12:17 PM Rob Herring <robh@kernel.org> wrote:
> >
> > On Mon, Aug 24, 2020 at 03:30:21PM -0400, Jim Quinlan wrote:
> > > The Raspberry Pi (RPI) is currently the only chip using this driver
> > > (pcie-brcmstb.c).  There, only one memory controller is used, without an
> > > extension region, and the SCB0 viewport size is set to the size of the
> > > first and only dma-range region.  Other BrcmSTB SOCs have more complicated
> > > memory configurations that require setting additional viewport sizes.
> > >
> > > BrcmSTB PCIe controllers are intimately connected to the memory
> > > controller(s) on the SOC.  The SOC may have one to three memory
> > > controllers; they are indicated by the term SCBi.  Each controller has a
> > > base region and an optional extension region.  In physical memory, the base
> > > and extension regions of a controller are not adjacent, but in PCIe-space
> > > they are.
> > >
> > > There is a "viewport" for each memory controller that allows DMA from
> > > endpoint devices.  Each viewport's size must be set to a power of two, and
> > > that size must be equal to or larger than the amount of memory each
> > > controller supports which is the sum of base region and its optional
> > > extension.  Further, the 1-3 viewports are also adjacent in PCIe-space.
> > >
> > > Unfortunately the viewport sizes cannot be ascertained from the
> > > "dma-ranges" property so they have their own property, "brcm,scb-sizes".
> > > This is because dma-range information does not indicate what memory
> > > controller it is associated.  For example, consider the following case
> > > where the size of one dma-range is 2GB and the second dma-range is 1GB:
> > >
> > >     /* Case 1: SCB0 size set to 4GB */
> > >     dma-range0: 2GB (from memc0-base)
> > >     dma-range1: 1GB (from memc0-extension)
> > >
> > >     /* Case 2: SCB0 size set to 2GB, SCB1 size set to 1GB */
> > >     dma-range0: 2GB (from memc0-base)
> > >     dma-range1: 1GB (from memc0-extension)
> > >
> > > By just looking at the dma-ranges information, one cannot tell which
> > > situation applies. That is why an additional property is needed.  Its
> > > length indicates the number of memory controllers being used and each value
> > > indicates the viewport size.
> > >
> > > Note that the RPI DT does not have a "brcm,scb-sizes" property value,
> > > as it is assumed that it only requires one memory controller and no
> > > extension.  So the optional use of "brcm,scb-sizes" will be backwards
> > > compatible.
> > >
> > > One last layer of complexity exists: all of the viewports sizes must be
> > > added and rounded up to a power of two to determine what the "BAR" size is.
> > > Further, an offset must be given that indicates the base PCIe address of
> > > this "BAR".  The use of the term BAR is typically associated with endpoint
> > > devices, and the term is used here because the PCIe HW may be used as an RC
> > > or an EP.  In the former case, all of the system memory appears in a single
> > > "BAR" region in PCIe memory.  As it turns out, BrcmSTB PCIe HW is rarely
> > > used in the EP role and its system of mapping memory is an artifact that
> > > requires multiple dma-ranges regions.
> > >
> > > Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> > > Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> > > ---
> > >  drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++-------
> > >  1 file changed, 50 insertions(+), 18 deletions(-)
> > >
> > > diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> > > index 041b8d109563..7150eaa803c2 100644
> > > --- a/drivers/pci/controller/pcie-brcmstb.c
> > > +++ b/drivers/pci/controller/pcie-brcmstb.c
> > > @@ -57,6 +57,8 @@
> > >  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK     0x300000
> > >  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128              0x0
> > >  #define  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK          0xf8000000
> > > +#define  PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK          0x07c00000
> > > +#define  PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK          0x0000001f
> >
> > Perhaps make 0-2 an arg and then you can just do:
> >
> > u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB_SIZE_MASK(memc))
>
> I cannot get this to work.  In this case u32p_replace_bits requires
> that the mask is a compile-time constant; when "memc" is a variable I
> don't see how to do this.
> >
> > >
> > >  #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO             0x400c
> > >  #define PCIE_MEM_WIN0_LO(win)        \
> > > @@ -154,6 +156,7 @@
> > >  #define SSC_STATUS_OFFSET            0x1
> > >  #define SSC_STATUS_SSC_MASK          0x400
> > >  #define SSC_STATUS_PLL_LOCK_MASK     0x800
> > > +#define PCIE_BRCM_MAX_MEMC           3
> > >
> > >  #define IDX_ADDR(pcie)                       (pcie->reg_offsets[EXT_CFG_INDEX])
> > >  #define DATA_ADDR(pcie)                      (pcie->reg_offsets[EXT_CFG_DATA])
> > > @@ -259,6 +262,8 @@ struct brcm_pcie {
> > >       const int               *reg_field_info;
> > >       enum pcie_type          type;
> > >       struct reset_control    *rescal;
> > > +     int                     num_memc;
> > > +     u64                     memc_size[PCIE_BRCM_MAX_MEMC];
> > >  };
> > >
> > >  /*
> > > @@ -714,22 +719,44 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
> > >                                                       u64 *rc_bar2_offset)
> > >  {
> > >       struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> > > -     struct device *dev = pcie->dev;
> > >       struct resource_entry *entry;
> > > +     struct device *dev = pcie->dev;
> > > +     u64 lowest_pcie_addr = ~(u64)0;
> > > +     int ret, i = 0;
> > > +     u64 size = 0;
> > >
> > > -     entry = resource_list_first_type(&bridge->dma_ranges, IORESOURCE_MEM);
> > > -     if (!entry)
> > > -             return -ENODEV;
> > > +     resource_list_for_each_entry(entry, &bridge->dma_ranges) {
> > > +             u64 pcie_beg = entry->res->start - entry->offset;
> > >
> > > +             size += entry->res->end - entry->res->start + 1;
> > > +             if (pcie_beg < lowest_pcie_addr)
> > > +                     lowest_pcie_addr = pcie_beg;
> > > +     }
> > >
> > > -     /*
> > > -      * The controller expects the inbound window offset to be calculated as
> > > -      * the difference between PCIe's address space and CPU's. The offset
> > > -      * provided by the firmware is calculated the opposite way, so we
> > > -      * negate it.
> > > -      */
> > > -     *rc_bar2_offset = -entry->offset;
> > > -     *rc_bar2_size = 1ULL << fls64(entry->res->end - entry->res->start);
> > > +     if (lowest_pcie_addr == ~(u64)0) {
> > > +             dev_err(dev, "DT node has no dma-ranges\n");
> > > +             return -EINVAL;
> > > +     }
> > > +
> > > +     ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1,
> > > +                                               PCIE_BRCM_MAX_MEMC);
> > > +
> > > +     if (ret <= 0) {
> > > +             /* Make an educated guess */
> > > +             pcie->num_memc = 1;
> > > +             pcie->memc_size[0] = 1ULL << fls64(size - 1);
> >
> > Use roundup_pow_of_two()
> The reason I didn't use roundup_pow_of_two() is that it returns a
> ulong which on ARM is 32bits and cannot represent  4GB.

Guess time for a roundup_pow_of_two_64... Anyways, save that for another day.

Reviewed-by: Rob Herring <robh@kernel.org>

Re: [PATCH v11 08/11] PCI: brcmstb: Set additional internal memory DMA viewport sizes

[Rob Herring]

On Fri, Sep 11, 2020 at 9:28 AM Jim Quinlan <james.quinlan@broadcom.com> wrote:
>
> On Thu, Sep 10, 2020 at 12:17 PM Rob Herring <robh@kernel.org> wrote:
> >
> > On Mon, Aug 24, 2020 at 03:30:21PM -0400, Jim Quinlan wrote:
> > > The Raspberry Pi (RPI) is currently the only chip using this driver
> > > (pcie-brcmstb.c).  There, only one memory controller is used, without an
> > > extension region, and the SCB0 viewport size is set to the size of the
> > > first and only dma-range region.  Other BrcmSTB SOCs have more complicated
> > > memory configurations that require setting additional viewport sizes.
> > >
> > > BrcmSTB PCIe controllers are intimately connected to the memory
> > > controller(s) on the SOC.  The SOC may have one to three memory
> > > controllers; they are indicated by the term SCBi.  Each controller has a
> > > base region and an optional extension region.  In physical memory, the base
> > > and extension regions of a controller are not adjacent, but in PCIe-space
> > > they are.
> > >
> > > There is a "viewport" for each memory controller that allows DMA from
> > > endpoint devices.  Each viewport's size must be set to a power of two, and
> > > that size must be equal to or larger than the amount of memory each
> > > controller supports which is the sum of base region and its optional
> > > extension.  Further, the 1-3 viewports are also adjacent in PCIe-space.
> > >
> > > Unfortunately the viewport sizes cannot be ascertained from the
> > > "dma-ranges" property so they have their own property, "brcm,scb-sizes".
> > > This is because dma-range information does not indicate what memory
> > > controller it is associated.  For example, consider the following case
> > > where the size of one dma-range is 2GB and the second dma-range is 1GB:
> > >
> > >     /* Case 1: SCB0 size set to 4GB */
> > >     dma-range0: 2GB (from memc0-base)
> > >     dma-range1: 1GB (from memc0-extension)
> > >
> > >     /* Case 2: SCB0 size set to 2GB, SCB1 size set to 1GB */
> > >     dma-range0: 2GB (from memc0-base)
> > >     dma-range1: 1GB (from memc0-extension)
> > >
> > > By just looking at the dma-ranges information, one cannot tell which
> > > situation applies. That is why an additional property is needed.  Its
> > > length indicates the number of memory controllers being used and each value
> > > indicates the viewport size.
> > >
> > > Note that the RPI DT does not have a "brcm,scb-sizes" property value,
> > > as it is assumed that it only requires one memory controller and no
> > > extension.  So the optional use of "brcm,scb-sizes" will be backwards
> > > compatible.
> > >
> > > One last layer of complexity exists: all of the viewports sizes must be
> > > added and rounded up to a power of two to determine what the "BAR" size is.
> > > Further, an offset must be given that indicates the base PCIe address of
> > > this "BAR".  The use of the term BAR is typically associated with endpoint
> > > devices, and the term is used here because the PCIe HW may be used as an RC
> > > or an EP.  In the former case, all of the system memory appears in a single
> > > "BAR" region in PCIe memory.  As it turns out, BrcmSTB PCIe HW is rarely
> > > used in the EP role and its system of mapping memory is an artifact that
> > > requires multiple dma-ranges regions.
> > >
> > > Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> > > Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> > > ---
> > >  drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++-------
> > >  1 file changed, 50 insertions(+), 18 deletions(-)
> > >
> > > diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> > > index 041b8d109563..7150eaa803c2 100644
> > > --- a/drivers/pci/controller/pcie-brcmstb.c
> > > +++ b/drivers/pci/controller/pcie-brcmstb.c
> > > @@ -57,6 +57,8 @@
> > >  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK     0x300000
> > >  #define  PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128              0x0
> > >  #define  PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK          0xf8000000
> > > +#define  PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK          0x07c00000
> > > +#define  PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK          0x0000001f
> >
> > Perhaps make 0-2 an arg and then you can just do:
> >
> > u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB_SIZE_MASK(memc))
>
> I cannot get this to work.  In this case u32p_replace_bits requires
> that the mask is a compile-time constant; when "memc" is a variable I
> don't see how to do this.
> >
> > >
> > >  #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO             0x400c
> > >  #define PCIE_MEM_WIN0_LO(win)        \
> > > @@ -154,6 +156,7 @@
> > >  #define SSC_STATUS_OFFSET            0x1
> > >  #define SSC_STATUS_SSC_MASK          0x400
> > >  #define SSC_STATUS_PLL_LOCK_MASK     0x800
> > > +#define PCIE_BRCM_MAX_MEMC           3
> > >
> > >  #define IDX_ADDR(pcie)                       (pcie->reg_offsets[EXT_CFG_INDEX])
> > >  #define DATA_ADDR(pcie)                      (pcie->reg_offsets[EXT_CFG_DATA])
> > > @@ -259,6 +262,8 @@ struct brcm_pcie {
> > >       const int               *reg_field_info;
> > >       enum pcie_type          type;
> > >       struct reset_control    *rescal;
> > > +     int                     num_memc;
> > > +     u64                     memc_size[PCIE_BRCM_MAX_MEMC];
> > >  };
> > >
> > >  /*
> > > @@ -714,22 +719,44 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
> > >                                                       u64 *rc_bar2_offset)
> > >  {
> > >       struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> > > -     struct device *dev = pcie->dev;
> > >       struct resource_entry *entry;
> > > +     struct device *dev = pcie->dev;
> > > +     u64 lowest_pcie_addr = ~(u64)0;
> > > +     int ret, i = 0;
> > > +     u64 size = 0;
> > >
> > > -     entry = resource_list_first_type(&bridge->dma_ranges, IORESOURCE_MEM);
> > > -     if (!entry)
> > > -             return -ENODEV;
> > > +     resource_list_for_each_entry(entry, &bridge->dma_ranges) {
> > > +             u64 pcie_beg = entry->res->start - entry->offset;
> > >
> > > +             size += entry->res->end - entry->res->start + 1;
> > > +             if (pcie_beg < lowest_pcie_addr)
> > > +                     lowest_pcie_addr = pcie_beg;
> > > +     }
> > >
> > > -     /*
> > > -      * The controller expects the inbound window offset to be calculated as
> > > -      * the difference between PCIe's address space and CPU's. The offset
> > > -      * provided by the firmware is calculated the opposite way, so we
> > > -      * negate it.
> > > -      */
> > > -     *rc_bar2_offset = -entry->offset;
> > > -     *rc_bar2_size = 1ULL << fls64(entry->res->end - entry->res->start);
> > > +     if (lowest_pcie_addr == ~(u64)0) {
> > > +             dev_err(dev, "DT node has no dma-ranges\n");
> > > +             return -EINVAL;
> > > +     }
> > > +
> > > +     ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1,
> > > +                                               PCIE_BRCM_MAX_MEMC);
> > > +
> > > +     if (ret <= 0) {
> > > +             /* Make an educated guess */
> > > +             pcie->num_memc = 1;
> > > +             pcie->memc_size[0] = 1ULL << fls64(size - 1);
> >
> > Use roundup_pow_of_two()
> The reason I didn't use roundup_pow_of_two() is that it returns a
> ulong which on ARM is 32bits and cannot represent  4GB.

Guess time for a roundup_pow_of_two_64... Anyways, save that for another day.

Reviewed-by: Rob Herring <robh@kernel.org>

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel