[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Thomas Petazzoni]

Hello,

This set of patches bring a number of fixes to the mvebu-mbus and I/O
coherency code:

 - Fix the MBus window n?13, which is a remappable window on Armada
   XP, 375 and 38x (but not 370), and was incorrectly handled as a
   non-remappable window by the mvebu-mbus driver. Due to this,
   whenever this window was used (for example if you have many PCIe
   devices), the device using this window would not work at all due to
   the window being badly configured.

 - Fix for the DRAM CS information exposed to device drivers by the
   mvebu-mbus driver to make sure the listed ranges do not overlap
   with the MBus bridge window. This is not causing any major issue
   right now, but is causing problems for an improved version of the
   mv_cesa crypto driver (which will be submitted in the future).

 - Switch to using automatic I/O synchronization barriers. This
   simultaneously fixes two problems:

    * The explicit I/O sync barriers could not work properly, because
      they are also needed for coherent DMA mappings, while the kernel
      do not expect any maintenance operation to be needed for such
      mappings. Since such barriers were never added for coherent DMA
      mappings, one could get rare but nonetheless existing cases of
      stale data seen by the CPU in the cache instead of the real data
      pushed by the device through DMA.

    * It allows to switch to use the existing arm_coherent_dma_ops
      instead of mvebu specific DMA operations. arm_coherent_dma_ops
      make sure that DMA coherent mappings are mapped cacheable (which
      is possible in a cache-coherent platform), which is a necessary
      to make sure that both CPU-side and device-side accesses are
      done in a cacheable way. Without this, CPU-side accesses to DMA
      coherent mappings were made uncached, while device-side accesses
      were made in a cacheable way, leading to potentially
      unpredictable behavior.

Patches are marked with the (hopefully) appropriate stable tags.

Best regards,

Thomas

Michal Mazur (1):
  bus: mvebu-mbus: fix support of MBus window 13 on Armada XP/375/38x

Thomas Petazzoni (5):
  dt: bindings: update mvebu-mbus DT binding with new compatible
    properties
  ARM: mvebu: fix compatible strings of MBus on Armada 375 and Armada
    38x
  bus: mvebu-mbus: make sure SDRAM CS for DMA don't overlap the MBus
    bridge window
  bus: mvebu-mbus: use automatic I/O synchronization barriers
  ARM: mvebu: use arm_coherent_dma_ops

 .../devicetree/bindings/bus/mvebu-mbus.txt         |   4 +-
 arch/arm/boot/dts/armada-375.dtsi                  |   2 +-
 arch/arm/boot/dts/armada-38x.dtsi                  |   3 +-
 arch/arm/mach-mvebu/coherency.c                    |  51 +---
 drivers/bus/mvebu-mbus.c                           | 275 ++++++++++++++++-----
 5 files changed, 225 insertions(+), 110 deletions(-)

-- 
2.1.0

[PATCH 1/6] dt: bindings: update mvebu-mbus DT binding with new compatible properties

[Thomas Petazzoni]

Contrary to what was originally thought, the Armada 375 and Armada 38x
MBus windows hardware block is *not* compatible with the Armada 370,
due to a difference in how window 13 is handled. It was rather
compatible with the Armada XP MBus hardware block.

However, the DTs for Armada 375 and Armada 38x encode the following
compatible string for MBus:

  compatible = "marvell,armada375-mbus", "marvell,armada370-mbus", "simple-bus";
  compatible = "marvell,armada380-mbus", "marvell,armada370-mbus", "simple-bus";

So, by extending the mvebu-mbus DT binding to also cover the
marvell,armada375-mbus and marvell,armada380-mbus compatible strings,
we can define a new behavior for those SoCs without changing the DT.

Therefore, this commit adds those two new compatible strings to the DT
binding documentation of mvebu-mbus. Note that it re-uses two existing
duplicated lines for the armada370-mbus and armadaxp-mbus compatible
strings.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 Documentation/devicetree/bindings/bus/mvebu-mbus.txt | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/devicetree/bindings/bus/mvebu-mbus.txt b/Documentation/devicetree/bindings/bus/mvebu-mbus.txt
index 5e16c3c..fa6cde4 100644
--- a/Documentation/devicetree/bindings/bus/mvebu-mbus.txt
+++ b/Documentation/devicetree/bindings/bus/mvebu-mbus.txt
@@ -6,8 +6,8 @@ Required properties:
 - compatible:	 Should be set to one of the following:
 		 marvell,armada370-mbus
 		 marvell,armadaxp-mbus
-		 marvell,armada370-mbus
-		 marvell,armadaxp-mbus
+		 marvell,armada375-mbus
+		 marvell,armada380-mbus
 		 marvell,kirkwood-mbus
 		 marvell,dove-mbus
 		 marvell,orion5x-88f5281-mbus
-- 
2.1.0

[PATCH 2/6] bus: mvebu-mbus: fix support of MBus window 13 on Armada XP/375/38x

[Thomas Petazzoni]

From: Michal Mazur <arg@semihalf.com>

On Armada XP, 375 and 38x the MBus window 13 has the remap capability,
like windows 0 to 7. However, the mvebu-mbus driver isn't currently
taking into account this special case, which means that when window 13
is actually used, the remap registers are left to 0, making the device
using this MBus window unavailable.

To make things even more fun, the hardware designers have chosen to
put the window 13 remap registers in a completely custom location,
using a logic that differs from the one used for all other remappable
windows.

To solve this problem, this commit:

 * Adds a SoC specific function to calculate offset of remap registers
   to the mvebu_mbus_soc_data structure. This function,
   ->win_remap_offset(), returns the offset of the remap registers, or
   MVEBU_MBUS_NO_REMAP if the window does not have the remap
   capability. This new function replaces the previous integer field
   num_remappable_wins, which was insufficient to encode the special
   case of window 13.

 * Adds an implementation of the ->win_remap_offset() function for the
   various SoC families. Some have 2 first windows that are remapable,
   some the 4 first, some the 8 first, and then the Armada XP/375/38x
   case where the 8 first are remapable plus the special window
   13. This is implemented in functions
   generic_mbus_win_remap_2_offset(),
   generic_mbus_win_remap_4_offset(),
   generic_mbus_win_remap_8_offset() and
   armada_xp_mbus_win_remap_offset() respectively.

 * Change the code to use the ->win_remap_offset() function when
   accessing the remap registers, and also to use a newly introduced
   mvebu_mbus_window_is_remappable() helper function that tells
   whether a given window is remapable or not.

 * Separate Armada 370 from XP/375/38X because the window 13 of Armada
   370 does not support the remap capability.

[Thomas: adapted for the mainline kernel, minor clarifications in the
code, reword the commit log.]

Fixes: fddddb52a6c ("bus: introduce an Marvell EBU MBus driver")
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: Michal Mazur <arg@semihalf.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/bus/mvebu-mbus.c | 153 +++++++++++++++++++++++++++++++++++------------
 1 file changed, 116 insertions(+), 37 deletions(-)

diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
index eb7682d..b6694df 100644
--- a/drivers/bus/mvebu-mbus.c
+++ b/drivers/bus/mvebu-mbus.c
@@ -106,9 +106,9 @@ struct mvebu_mbus_state;
 
 struct mvebu_mbus_soc_data {
 	unsigned int num_wins;
-	unsigned int num_remappable_wins;
 	bool has_mbus_bridge;
 	unsigned int (*win_cfg_offset)(const int win);
+	unsigned int (*win_remap_offset)(const int win);
 	void (*setup_cpu_target)(struct mvebu_mbus_state *s);
 	int (*save_cpu_target)(struct mvebu_mbus_state *s,
 			       u32 *store_addr);
@@ -154,6 +154,13 @@ const struct mbus_dram_target_info *mv_mbus_dram_info(void)
 }
 EXPORT_SYMBOL_GPL(mv_mbus_dram_info);
 
+/* Checks whether the given window has remap capability */
+static bool mvebu_mbus_window_is_remappable(struct mvebu_mbus_state *mbus,
+					    const int win)
+{
+	return mbus->soc->win_remap_offset(win) != MVEBU_MBUS_NO_REMAP;
+}
+
 /*
  * Functions to manipulate the address decoding windows
  */
@@ -185,9 +192,12 @@ static void mvebu_mbus_read_window(struct mvebu_mbus_state *mbus,
 		*attr = (ctrlreg & WIN_CTRL_ATTR_MASK) >> WIN_CTRL_ATTR_SHIFT;
 
 	if (remap) {
-		if (win < mbus->soc->num_remappable_wins) {
-			u32 remap_low = readl(addr + WIN_REMAP_LO_OFF);
-			u32 remap_hi  = readl(addr + WIN_REMAP_HI_OFF);
+		if (mvebu_mbus_window_is_remappable(mbus, win)) {
+			u32 remap_low, remap_hi;
+			void __iomem *addr_rmp = mbus->mbuswins_base +
+				mbus->soc->win_remap_offset(win);
+			remap_low = readl(addr_rmp + WIN_REMAP_LO_OFF);
+			remap_hi  = readl(addr_rmp + WIN_REMAP_HI_OFF);
 			*remap = ((u64)remap_hi << 32) | remap_low;
 		} else
 			*remap = 0;
@@ -200,10 +210,11 @@ static void mvebu_mbus_disable_window(struct mvebu_mbus_state *mbus,
 	void __iomem *addr;
 
 	addr = mbus->mbuswins_base + mbus->soc->win_cfg_offset(win);
-
 	writel(0, addr + WIN_BASE_OFF);
 	writel(0, addr + WIN_CTRL_OFF);
-	if (win < mbus->soc->num_remappable_wins) {
+
+	if (mvebu_mbus_window_is_remappable(mbus, win)) {
+		addr = mbus->mbuswins_base + mbus->soc->win_remap_offset(win);
 		writel(0, addr + WIN_REMAP_LO_OFF);
 		writel(0, addr + WIN_REMAP_HI_OFF);
 	}
@@ -307,13 +318,17 @@ static int mvebu_mbus_setup_window(struct mvebu_mbus_state *mbus,
 
 	writel(base & WIN_BASE_LOW, addr + WIN_BASE_OFF);
 	writel(ctrl, addr + WIN_CTRL_OFF);
-	if (win < mbus->soc->num_remappable_wins) {
+
+	if (mvebu_mbus_window_is_remappable(mbus, win)) {
+		void __iomem *addr_rmp = mbus->mbuswins_base +
+			mbus->soc->win_remap_offset(win);
+
 		if (remap == MVEBU_MBUS_NO_REMAP)
 			remap_addr = base;
 		else
 			remap_addr = remap;
-		writel(remap_addr & WIN_REMAP_LOW, addr + WIN_REMAP_LO_OFF);
-		writel(0, addr + WIN_REMAP_HI_OFF);
+		writel(remap_addr & WIN_REMAP_LOW, addr_rmp + WIN_REMAP_LO_OFF);
+		writel(0, addr_rmp + WIN_REMAP_HI_OFF);
 	}
 
 	return 0;
@@ -327,19 +342,27 @@ static int mvebu_mbus_alloc_window(struct mvebu_mbus_state *mbus,
 	int win;
 
 	if (remap == MVEBU_MBUS_NO_REMAP) {
-		for (win = mbus->soc->num_remappable_wins;
-		     win < mbus->soc->num_wins; win++)
+		for (win = 0; win < mbus->soc->num_wins; win++) {
+			if (mvebu_mbus_window_is_remappable(mbus, win))
+				continue;
+
 			if (mvebu_mbus_window_is_free(mbus, win))
 				return mvebu_mbus_setup_window(mbus, win, base,
 							       size, remap,
 							       target, attr);
+		}
 	}
 
+	for (win = 0; win < mbus->soc->num_wins; win++) {
+		/* Skip window if need remap but is not supported */
+		if ((remap != MVEBU_MBUS_NO_REMAP) &&
+		    !mvebu_mbus_window_is_remappable(mbus, win))
+			continue;
 
-	for (win = 0; win < mbus->soc->num_wins; win++)
 		if (mvebu_mbus_window_is_free(mbus, win))
 			return mvebu_mbus_setup_window(mbus, win, base, size,
 						       remap, target, attr);
+	}
 
 	return -ENOMEM;
 }
@@ -451,7 +474,7 @@ static int mvebu_devs_debug_show(struct seq_file *seq, void *v)
 		    ((wbase & (u64)(wsize - 1)) != 0))
 			seq_puts(seq, " (Invalid base/size!!)");
 
-		if (win < mbus->soc->num_remappable_wins) {
+		if (mvebu_mbus_window_is_remappable(mbus, win)) {
 			seq_printf(seq, " (remap %016llx)\n",
 				   (unsigned long long)wremap);
 		} else
@@ -477,12 +500,12 @@ static const struct file_operations mvebu_devs_debug_fops = {
  * SoC-specific functions and definitions
  */
 
-static unsigned int orion_mbus_win_offset(int win)
+static unsigned int generic_mbus_win_cfg_offset(int win)
 {
 	return win << 4;
 }
 
-static unsigned int armada_370_xp_mbus_win_offset(int win)
+static unsigned int armada_370_xp_mbus_win_cfg_offset(int win)
 {
 	/* The register layout is a bit annoying and the below code
 	 * tries to cope with it.
@@ -502,7 +525,7 @@ static unsigned int armada_370_xp_mbus_win_offset(int win)
 		return 0x90 + ((win - 8) << 3);
 }
 
-static unsigned int mv78xx0_mbus_win_offset(int win)
+static unsigned int mv78xx0_mbus_win_cfg_offset(int win)
 {
 	if (win < 8)
 		return win << 4;
@@ -510,6 +533,40 @@ static unsigned int mv78xx0_mbus_win_offset(int win)
 		return 0x900 + ((win - 8) << 4);
 }
 
+static unsigned int generic_mbus_win_remap_2_offset(int win)
+{
+	if (win < 2)
+		return generic_mbus_win_cfg_offset(win);
+	else
+		return MVEBU_MBUS_NO_REMAP;
+}
+
+static unsigned int generic_mbus_win_remap_4_offset(int win)
+{
+	if (win < 4)
+		return generic_mbus_win_cfg_offset(win);
+	else
+		return MVEBU_MBUS_NO_REMAP;
+}
+
+static unsigned int generic_mbus_win_remap_8_offset(int win)
+{
+	if (win < 8)
+		return generic_mbus_win_cfg_offset(win);
+	else
+		return MVEBU_MBUS_NO_REMAP;
+}
+
+static unsigned int armada_xp_mbus_win_remap_offset(int win)
+{
+	if (win < 8)
+		return generic_mbus_win_cfg_offset(win);
+	else if (win == 13)
+		return 0xF0 - WIN_REMAP_LO_OFF;
+	else
+		return MVEBU_MBUS_NO_REMAP;
+}
+
 static void __init
 mvebu_mbus_default_setup_cpu_target(struct mvebu_mbus_state *mbus)
 {
@@ -619,30 +676,40 @@ int mvebu_mbus_save_cpu_target(u32 *store_addr)
 	return mbus_state.soc->save_cpu_target(&mbus_state, store_addr);
 }
 
-static const struct mvebu_mbus_soc_data armada_370_xp_mbus_data = {
+static const struct mvebu_mbus_soc_data armada_370_mbus_data = {
 	.num_wins            = 20,
-	.num_remappable_wins = 8,
 	.has_mbus_bridge     = true,
-	.win_cfg_offset      = armada_370_xp_mbus_win_offset,
+	.win_cfg_offset      = armada_370_xp_mbus_win_cfg_offset,
+	.win_remap_offset    = generic_mbus_win_remap_8_offset,
+	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
+	.show_cpu_target     = mvebu_sdram_debug_show_orion,
 	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
+};
+
+static const struct mvebu_mbus_soc_data armada_xp_mbus_data = {
+	.num_wins            = 20,
+	.has_mbus_bridge     = true,
+	.win_cfg_offset      = armada_370_xp_mbus_win_cfg_offset,
+	.win_remap_offset    = armada_xp_mbus_win_remap_offset,
 	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_orion,
+	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
 };
 
 static const struct mvebu_mbus_soc_data kirkwood_mbus_data = {
 	.num_wins            = 8,
-	.num_remappable_wins = 4,
-	.win_cfg_offset      = orion_mbus_win_offset,
+	.win_cfg_offset      = generic_mbus_win_cfg_offset,
 	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
+	.win_remap_offset    = generic_mbus_win_remap_4_offset,
 	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_orion,
 };
 
 static const struct mvebu_mbus_soc_data dove_mbus_data = {
 	.num_wins            = 8,
-	.num_remappable_wins = 4,
-	.win_cfg_offset      = orion_mbus_win_offset,
+	.win_cfg_offset      = generic_mbus_win_cfg_offset,
 	.save_cpu_target     = mvebu_mbus_dove_save_cpu_target,
+	.win_remap_offset    = generic_mbus_win_remap_4_offset,
 	.setup_cpu_target    = mvebu_mbus_dove_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_dove,
 };
@@ -653,36 +720,40 @@ static const struct mvebu_mbus_soc_data dove_mbus_data = {
  */
 static const struct mvebu_mbus_soc_data orion5x_4win_mbus_data = {
 	.num_wins            = 8,
-	.num_remappable_wins = 4,
-	.win_cfg_offset      = orion_mbus_win_offset,
+	.win_cfg_offset      = generic_mbus_win_cfg_offset,
 	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
+	.win_remap_offset    = generic_mbus_win_remap_4_offset,
 	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_orion,
 };
 
 static const struct mvebu_mbus_soc_data orion5x_2win_mbus_data = {
 	.num_wins            = 8,
-	.num_remappable_wins = 2,
-	.win_cfg_offset      = orion_mbus_win_offset,
+	.win_cfg_offset      = generic_mbus_win_cfg_offset,
 	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
+	.win_remap_offset    = generic_mbus_win_remap_2_offset,
 	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_orion,
 };
 
 static const struct mvebu_mbus_soc_data mv78xx0_mbus_data = {
 	.num_wins            = 14,
-	.num_remappable_wins = 8,
-	.win_cfg_offset      = mv78xx0_mbus_win_offset,
+	.win_cfg_offset      = mv78xx0_mbus_win_cfg_offset,
 	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
+	.win_remap_offset    = generic_mbus_win_remap_8_offset,
 	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_orion,
 };
 
 static const struct of_device_id of_mvebu_mbus_ids[] = {
 	{ .compatible = "marvell,armada370-mbus",
-	  .data = &armada_370_xp_mbus_data, },
+	  .data = &armada_370_mbus_data, },
+	{ .compatible = "marvell,armada375-mbus",
+	  .data = &armada_xp_mbus_data, },
+	{ .compatible = "marvell,armada380-mbus",
+	  .data = &armada_xp_mbus_data, },
 	{ .compatible = "marvell,armadaxp-mbus",
-	  .data = &armada_370_xp_mbus_data, },
+	  .data = &armada_xp_mbus_data, },
 	{ .compatible = "marvell,kirkwood-mbus",
 	  .data = &kirkwood_mbus_data, },
 	{ .compatible = "marvell,dove-mbus",
@@ -789,15 +860,19 @@ static int mvebu_mbus_suspend(void)
 	for (win = 0; win < s->soc->num_wins; win++) {
 		void __iomem *addr = s->mbuswins_base +
 			s->soc->win_cfg_offset(win);
+		void __iomem *addr_rmp;
 
 		s->wins[win].base = readl(addr + WIN_BASE_OFF);
 		s->wins[win].ctrl = readl(addr + WIN_CTRL_OFF);
 
-		if (win >= s->soc->num_remappable_wins)
+		if (!mvebu_mbus_window_is_remappable(s, win))
 			continue;
 
-		s->wins[win].remap_lo = readl(addr + WIN_REMAP_LO_OFF);
-		s->wins[win].remap_hi = readl(addr + WIN_REMAP_HI_OFF);
+		addr_rmp = s->mbuswins_base +
+			s->soc->win_remap_offset(win);
+
+		s->wins[win].remap_lo = readl(addr_rmp + WIN_REMAP_LO_OFF);
+		s->wins[win].remap_hi = readl(addr_rmp + WIN_REMAP_HI_OFF);
 	}
 
 	s->mbus_bridge_ctrl = readl(s->mbusbridge_base +
@@ -821,15 +896,19 @@ static void mvebu_mbus_resume(void)
 	for (win = 0; win < s->soc->num_wins; win++) {
 		void __iomem *addr = s->mbuswins_base +
 			s->soc->win_cfg_offset(win);
+		void __iomem *addr_rmp;
 
 		writel(s->wins[win].base, addr + WIN_BASE_OFF);
 		writel(s->wins[win].ctrl, addr + WIN_CTRL_OFF);
 
-		if (win >= s->soc->num_remappable_wins)
+		if (!mvebu_mbus_window_is_remappable(s, win))
 			continue;
 
-		writel(s->wins[win].remap_lo, addr + WIN_REMAP_LO_OFF);
-		writel(s->wins[win].remap_hi, addr + WIN_REMAP_HI_OFF);
+		addr_rmp = s->mbuswins_base +
+			s->soc->win_remap_offset(win);
+
+		writel(s->wins[win].remap_lo, addr_rmp + WIN_REMAP_LO_OFF);
+		writel(s->wins[win].remap_hi, addr_rmp + WIN_REMAP_HI_OFF);
 	}
 }
 
-- 
2.1.0

[PATCH 3/6] ARM: mvebu: fix compatible strings of MBus on Armada 375 and Armada 38x

[Thomas Petazzoni]

Due to the special handling of window 13 on Armada 375 and Armada 38x
(similar to Armada XP), the MBus hardware block is *not* compatible
with the one used on Armada 370. Using the Armada 370 compatible
string on Armada 375 and 38x will lead to a non-working device if
window 13 ends up being used.

Fixes: 4de59085091 ("ARM: mvebu: add Device Tree description of the Armada 375 SoC")
Fixes: 0d3d96ab005 ("ARM: mvebu: add Device Tree description of the Armada 380/385 SoCs")
Cc: <stable@vger.kernel.org> # v3.15+
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/boot/dts/armada-375.dtsi | 2 +-
 arch/arm/boot/dts/armada-38x.dtsi | 3 +--
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/arch/arm/boot/dts/armada-375.dtsi b/arch/arm/boot/dts/armada-375.dtsi
index 50096d3..126bf7e 100644
--- a/arch/arm/boot/dts/armada-375.dtsi
+++ b/arch/arm/boot/dts/armada-375.dtsi
@@ -63,7 +63,7 @@
 	};
 
 	soc {
-		compatible = "marvell,armada375-mbus", "marvell,armada370-mbus", "simple-bus";
+		compatible = "marvell,armada375-mbus", "simple-bus";
 		#address-cells = <2>;
 		#size-cells = <1>;
 		controller = <&mbusc>;
diff --git a/arch/arm/boot/dts/armada-38x.dtsi b/arch/arm/boot/dts/armada-38x.dtsi
index 74391da..7d8295c 100644
--- a/arch/arm/boot/dts/armada-38x.dtsi
+++ b/arch/arm/boot/dts/armada-38x.dtsi
@@ -31,8 +31,7 @@
 	};
 
 	soc {
-		compatible = "marvell,armada380-mbus", "marvell,armada370-mbus",
-			     "simple-bus";
+		compatible = "marvell,armada380-mbus", "simple-bus";
 		#address-cells = <2>;
 		#size-cells = <1>;
 		controller = <&mbusc>;
-- 
2.1.0

[PATCH 4/6] bus: mvebu-mbus: make sure SDRAM CS for DMA don't overlap the MBus bridge window

[Thomas Petazzoni]

The mvebu-mbus driver reads the SDRAM window registers, and make the
information about the DRAM CS configuration available to device
drivers using the mv_mbus_dram_info() API. This information is used by
the DMA-capable device drivers to program their address decoding
windows.

Until now, we were basically providing the SDRAM window register
details as is. However, it turns out that the DMA capability of the
CESA cryptographic engine consists in doing DMA being the DRAM and the
crypto SRAM mapped as a MBus window. For this case, it is very
important that the SDRAM CS information does not overlap with the MBus
bridge window.

Therefore, this commit improves the mvebu-mbus driver to make sure we
adjust the SDRAM CS information so that it doesn't overlap with the
MBus bridge window. This problem was reported by Boris Brezillon,
while working on the mv_cesa driver for Armada 37x/38x/XP. We use the
memblock memory information to know where the usable RAM is located,
as this information is guaranteed to be correct on all SoC variants.

We could have used the MBus bridge window registers on Armada 370/XP,
but they are not really used on Armada 375/38x (Cortex-A9 based),
since the PL310 L2 filtering is used instead to discriminate between
RAM accesses and I/O accesses. Therefore, using the memblock
information is more generic and works accross the different platforms.

Reported-by: Boris Brezillon <boris.brezillon@free-electrons.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
We have decided to not backport this commit to stable. Even though it
is really fixing the DRAM CS information exposed to device drivers,
there is no way in mainline for the current situation to cause any
problem.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/bus/mvebu-mbus.c | 105 +++++++++++++++++++++++++++++++++++++++--------
 1 file changed, 89 insertions(+), 16 deletions(-)

diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
index b6694df..bb0dd21 100644
--- a/drivers/bus/mvebu-mbus.c
+++ b/drivers/bus/mvebu-mbus.c
@@ -58,6 +58,7 @@
 #include <linux/debugfs.h>
 #include <linux/log2.h>
 #include <linux/syscore_ops.h>
+#include <linux/memblock.h>
 
 /*
  * DDR target is the same on all platforms.
@@ -97,7 +98,9 @@
 
 /* Relative to mbusbridge_base */
 #define MBUS_BRIDGE_CTRL_OFF	0x0
+#define  MBUS_BRIDGE_SIZE_MASK  0xffff0000
 #define MBUS_BRIDGE_BASE_OFF	0x4
+#define  MBUS_BRIDGE_BASE_MASK  0xffff0000
 
 /* Maximum number of windows, for all known platforms */
 #define MBUS_WINS_MAX           20
@@ -567,36 +570,106 @@ static unsigned int armada_xp_mbus_win_remap_offset(int win)
 		return MVEBU_MBUS_NO_REMAP;
 }
 
+/*
+ * Use the memblock information to find the MBus bridge hole in the
+ * physical address space.
+ */
+static void __init
+mvebu_mbus_find_bridge_hole(uint64_t *start, uint64_t *end)
+{
+	struct memblock_region *r;
+	uint64_t s = 0;
+
+	for_each_memblock(memory, r) {
+		/*
+		 * This part of the memory is above 4 GB, so we don't
+		 * care for the MBus bridge hole.
+		 */
+		if (r->base >= 0x100000000)
+			continue;
+
+		/*
+		 * The MBus bridge hole is at the end of the RAM under
+		 * the 4 GB limit.
+		 */
+		if (r->base + r->size > s)
+			s = r->base + r->size;
+	}
+
+	*start = s;
+	*end = 0x100000000;
+}
+
 static void __init
 mvebu_mbus_default_setup_cpu_target(struct mvebu_mbus_state *mbus)
 {
 	int i;
 	int cs;
+	uint64_t mbus_bridge_base, mbus_bridge_end;
 
 	mvebu_mbus_dram_info.mbus_dram_target_id = TARGET_DDR;
 
+	mvebu_mbus_find_bridge_hole(&mbus_bridge_base, &mbus_bridge_end);
+
 	for (i = 0, cs = 0; i < 4; i++) {
-		u32 base = readl(mbus->sdramwins_base + DDR_BASE_CS_OFF(i));
-		u32 size = readl(mbus->sdramwins_base + DDR_SIZE_CS_OFF(i));
+		u64 base = readl(mbus->sdramwins_base + DDR_BASE_CS_OFF(i));
+		u64 size = readl(mbus->sdramwins_base + DDR_SIZE_CS_OFF(i));
+		u64 end;
+		struct mbus_dram_window *w;
+
+		/* Ignore entries that are not enabled */
+		if (!(size & DDR_SIZE_ENABLED))
+			continue;
 
 		/*
-		 * We only take care of entries for which the chip
-		 * select is enabled, and that don't have high base
-		 * address bits set (devices can only access the first
-		 * 32 bits of the memory).
+		 * Ignore entries whose base address is above 2^32,
+		 * since devices cannot DMA to such high addresses
 		 */
-		if ((size & DDR_SIZE_ENABLED) &&
-		    !(base & DDR_BASE_CS_HIGH_MASK)) {
-			struct mbus_dram_window *w;
+		if (base & DDR_BASE_CS_HIGH_MASK)
+			continue;
 
-			w = &mvebu_mbus_dram_info.cs[cs++];
-			w->cs_index = i;
-			w->mbus_attr = 0xf & ~(1 << i);
-			if (mbus->hw_io_coherency)
-				w->mbus_attr |= ATTR_HW_COHERENCY;
-			w->base = base & DDR_BASE_CS_LOW_MASK;
-			w->size = (size | ~DDR_SIZE_MASK) + 1;
+		base = base & DDR_BASE_CS_LOW_MASK;
+		size = (size | ~DDR_SIZE_MASK) + 1;
+		end = base + size;
+
+		/*
+		 * Adjust base/size of the current CS to make sure it
+		 * doesn't overlap with the MBus bridge hole. This is
+		 * particularly important for devices that do DMA from
+		 * DRAM to a SRAM mapped in a MBus window, such as the
+		 * CESA cryptographic engine.
+		 */
+
+		/*
+		 * The CS is fully enclosed inside the MBus bridge
+		 * area, so ignore it.
+		 */
+		if (base >= mbus_bridge_base && end <= mbus_bridge_end)
+			continue;
+
+		/*
+		 * Beginning of CS overlaps with end of MBus, raise CS
+		 * base address, and shrink its size.
+		 */
+		if (base >= mbus_bridge_base && end > mbus_bridge_end) {
+			size -= mbus_bridge_end - base;
+			base = mbus_bridge_end;
 		}
+
+		/*
+		 * End of CS overlaps with beginning of MBus, shrink
+		 * CS size.
+		 */
+		if (base < mbus_bridge_base && end > mbus_bridge_base)
+			size -= end - mbus_bridge_base;
+
+		w = &mvebu_mbus_dram_info.cs[cs++];
+		w->cs_index = i;
+		w->mbus_attr = 0xf & ~(1 << i);
+		if (mbus->hw_io_coherency)
+			w->mbus_attr |= ATTR_HW_COHERENCY;
+		w->base = base;
+		w->size = size;
 	}
 	mvebu_mbus_dram_info.num_cs = cs;
 }
-- 
2.1.0

[PATCH 5/6] bus: mvebu-mbus: use automatic I/O synchronization barriers

[Thomas Petazzoni]

Instead of using explicit I/O synchronization barriers shoehorned
inside the streaming DMA mappings API (in
arch/arm/mach-mvebu/coherency.c), we are switching to use automatic
I/O synchronization barrier.

The primary motivation for this change is that explicit I/O
synchronization barriers are not only needed for streaming DMA
mappings (which can easily be done by overriding the dma_map_ops), but
also for coherent DMA mappings (which is a lot less easy to do, since
the kernel assumes such mappings are coherent and don't require any
sort of cache maintenance operation to ensure the consistency of the
buffers).

Switching to automatic I/O synchronization barriers will also allow us
to use the existing arm_coherent_dma_ops instead of our custom
arm_dma_ops.

In order to use automatic I/O synchronization barriers, this commit
changes mvebu-mbus in two ways:

 - It enables automatic I/O synchronization barriers in the 0x84
   register of the MBus bridge, by enabling such barriers for all MBus
   units. This enables automatic barriers for the on-SoC peripherals
   that are doing DMA.

 - It enables the SyncEnable bit in the MBus windows, so that PCIe
   devices also use automatic I/O synchronization barrier.

This automatic synchronization barrier relies on the assumption that
at least one register of a given hardware unit is read before the
driver accesses the DMA mappings modified by this unit. This
assumption is guaranteed for PCI devices by vertue of the PCI
standard, and we can reasonably verify that this assumption is also
true for the limited number of platform drivers doing DMA used on
Marvell EBU platforms.

Fixes: fddddb52a6c4 ("bus: introduce an Marvell EBU MBus driver")
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
Due to the coherent mapping problem, the I/O coherency has in fact
been broken since its introduction, which is why we request the
backport of this patch all the way up to 3.10.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 drivers/bus/mvebu-mbus.c | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
index bb0dd21..649d187 100644
--- a/drivers/bus/mvebu-mbus.c
+++ b/drivers/bus/mvebu-mbus.c
@@ -70,6 +70,7 @@
  */
 #define WIN_CTRL_OFF		0x0000
 #define   WIN_CTRL_ENABLE       BIT(0)
+#define   WIN_CTRL_SYNCBARRIER  BIT(1)
 #define   WIN_CTRL_TGT_MASK     0xf0
 #define   WIN_CTRL_TGT_SHIFT    4
 #define   WIN_CTRL_ATTR_MASK    0xff00
@@ -83,6 +84,9 @@
 #define   WIN_REMAP_LOW         0xffff0000
 #define WIN_REMAP_HI_OFF	0x000c
 
+#define UNIT_SYNC_BARRIER_OFF   0x84
+#define   UNIT_SYNC_BARRIER_ALL 0xFFFF
+
 #define ATTR_HW_COHERENCY	(0x1 << 4)
 
 #define DDR_BASE_CS_OFF(n)	(0x0000 + ((n) << 3))
@@ -317,6 +321,7 @@ static int mvebu_mbus_setup_window(struct mvebu_mbus_state *mbus,
 	ctrl = ((size - 1) & WIN_CTRL_SIZE_MASK) |
 		(attr << WIN_CTRL_ATTR_SHIFT)    |
 		(target << WIN_CTRL_TGT_SHIFT)   |
+		WIN_CTRL_SYNCBARRIER             |
 		WIN_CTRL_ENABLE;
 
 	writel(base & WIN_BASE_LOW, addr + WIN_BASE_OFF);
@@ -996,7 +1001,8 @@ static int __init mvebu_mbus_common_init(struct mvebu_mbus_state *mbus,
 					 phys_addr_t sdramwins_phys_base,
 					 size_t sdramwins_size,
 					 phys_addr_t mbusbridge_phys_base,
-					 size_t mbusbridge_size)
+					 size_t mbusbridge_size,
+					 bool is_coherent)
 {
 	int win;
 
@@ -1028,6 +1034,10 @@ static int __init mvebu_mbus_common_init(struct mvebu_mbus_state *mbus,
 
 	mbus->soc->setup_cpu_target(mbus);
 
+	if (is_coherent)
+		writel(UNIT_SYNC_BARRIER_ALL,
+		       mbus->mbuswins_base + UNIT_SYNC_BARRIER_OFF);
+
 	register_syscore_ops(&mvebu_mbus_syscore_ops);
 
 	return 0;
@@ -1055,7 +1065,7 @@ int __init mvebu_mbus_init(const char *soc, phys_addr_t mbuswins_phys_base,
 			mbuswins_phys_base,
 			mbuswins_size,
 			sdramwins_phys_base,
-			sdramwins_size, 0, 0);
+			sdramwins_size, 0, 0, false);
 }
 
 #ifdef CONFIG_OF
@@ -1257,7 +1267,8 @@ int __init mvebu_mbus_dt_init(bool is_coherent)
 				     sdramwins_res.start,
 				     resource_size(&sdramwins_res),
 				     mbusbridge_res.start,
-				     resource_size(&mbusbridge_res));
+				     resource_size(&mbusbridge_res),
+				     is_coherent);
 	if (ret)
 		return ret;
 
-- 
2.1.0

[PATCH 6/6] ARM: mvebu: use arm_coherent_dma_ops

[Thomas Petazzoni]

Now that we have enabled automatic I/O synchronization barriers, we no
longer need any explicit barriers. We can therefore simplify
arch/arm/mach-mvebu/coherency.c by using the existing
arm_coherent_dma_ops instead of our custom mvebu_hwcc_dma_ops.

Fixes: e60304f8cb7b ("arm: mvebu: Add hardware I/O Coherency support")
Cc: <stable@vger.kernel.org> # v3.8+
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
---
 arch/arm/mach-mvebu/coherency.c | 51 ++---------------------------------------
 1 file changed, 2 insertions(+), 49 deletions(-)

diff --git a/arch/arm/mach-mvebu/coherency.c b/arch/arm/mach-mvebu/coherency.c
index 3585cb3..440799b 100644
--- a/arch/arm/mach-mvebu/coherency.c
+++ b/arch/arm/mach-mvebu/coherency.c
@@ -33,6 +33,7 @@
 #include <asm/smp_plat.h>
 #include <asm/cacheflush.h>
 #include <asm/mach/map.h>
+#include <asm/dma-mapping.h>
 #include "coherency.h"
 #include "mvebu-soc-id.h"
 
@@ -76,54 +77,6 @@ int set_cpu_coherent(void)
 	return ll_enable_coherency();
 }
 
-static inline void mvebu_hwcc_sync_io_barrier(void)
-{
-	writel(0x1, coherency_cpu_base + IO_SYNC_BARRIER_CTL_OFFSET);
-	while (readl(coherency_cpu_base + IO_SYNC_BARRIER_CTL_OFFSET) & 0x1);
-}
-
-static dma_addr_t mvebu_hwcc_dma_map_page(struct device *dev, struct page *page,
-				  unsigned long offset, size_t size,
-				  enum dma_data_direction dir,
-				  struct dma_attrs *attrs)
-{
-	if (dir != DMA_TO_DEVICE)
-		mvebu_hwcc_sync_io_barrier();
-	return pfn_to_dma(dev, page_to_pfn(page)) + offset;
-}
-
-
-static void mvebu_hwcc_dma_unmap_page(struct device *dev, dma_addr_t dma_handle,
-			      size_t size, enum dma_data_direction dir,
-			      struct dma_attrs *attrs)
-{
-	if (dir != DMA_TO_DEVICE)
-		mvebu_hwcc_sync_io_barrier();
-}
-
-static void mvebu_hwcc_dma_sync(struct device *dev, dma_addr_t dma_handle,
-			size_t size, enum dma_data_direction dir)
-{
-	if (dir != DMA_TO_DEVICE)
-		mvebu_hwcc_sync_io_barrier();
-}
-
-static struct dma_map_ops mvebu_hwcc_dma_ops = {
-	.alloc			= arm_dma_alloc,
-	.free			= arm_dma_free,
-	.mmap			= arm_dma_mmap,
-	.map_page		= mvebu_hwcc_dma_map_page,
-	.unmap_page		= mvebu_hwcc_dma_unmap_page,
-	.get_sgtable		= arm_dma_get_sgtable,
-	.map_sg			= arm_dma_map_sg,
-	.unmap_sg		= arm_dma_unmap_sg,
-	.sync_single_for_cpu	= mvebu_hwcc_dma_sync,
-	.sync_single_for_device	= mvebu_hwcc_dma_sync,
-	.sync_sg_for_cpu	= arm_dma_sync_sg_for_cpu,
-	.sync_sg_for_device	= arm_dma_sync_sg_for_device,
-	.set_dma_mask		= arm_dma_set_mask,
-};
-
 static int mvebu_hwcc_notifier(struct notifier_block *nb,
 			       unsigned long event, void *__dev)
 {
@@ -131,7 +84,7 @@ static int mvebu_hwcc_notifier(struct notifier_block *nb,
 
 	if (event != BUS_NOTIFY_ADD_DEVICE)
 		return NOTIFY_DONE;
-	set_dma_ops(dev, &mvebu_hwcc_dma_ops);
+	set_dma_ops(dev, &arm_coherent_dma_ops);
 
 	return NOTIFY_OK;
 }
-- 
2.1.0

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Andrew Lunn]

Hi Thomas

I'm probably getting ahead of myself here, since there has not been
any review comments, but i'm trying to figure out what should be
submitted for -rc3 and which can wait for the next merge window.

[PATCH 1/6] dt: bindings: update mvebu-mbus DT binding with new
branch dt for next merge window, since it is not strictly a fix.

[PATCH 2/6] bus: mvebu-mbus: fix support of MBus window 13
branch fixes for next -rc

[PATCH 3/6] ARM: mvebu: fix compatible strings of MBus on Armada 375 and Armada 38x
branch fixes for next -rc

[PATCH 4/6] bus: mvebu-mbus: make sure SDRAM CS for DMA don't overlap the MBus bridge window
branch drivers for next merge window. I assume it is only a run time dependency for the
mv_cesa patches?

[PATCH 5/6] bus: mvebu-mbus: use automatic I/O synchronization barriers
branch fixes for next -rc

[PATCH 6/6] ARM: mvebu: use arm_coherent_dma_ops
Here i'm unsure. Is this a fix, or a cleanup?

Thanks
	Andrew

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Thomas Petazzoni]

Dear Andrew Lunn,

On Tue, 30 Dec 2014 17:27:16 +0100, Andrew Lunn wrote:

> [PATCH 1/6] dt: bindings: update mvebu-mbus DT binding with new
> branch dt for next merge window, since it is not strictly a fix.

Right.

> [PATCH 2/6] bus: mvebu-mbus: fix support of MBus window 13
> branch fixes for next -rc

Correct.

> [PATCH 3/6] ARM: mvebu: fix compatible strings of MBus on Armada 375 and Armada 38x
> branch fixes for next -rc

Correct, since this is needed for PATCH 2/6 to work properly.

> [PATCH 4/6] bus: mvebu-mbus: make sure SDRAM CS for DMA don't overlap the MBus bridge window
> branch drivers for next merge window. I assume it is only a run time dependency for the
> mv_cesa patches?

Indeed, next merge window is good enough for this one: the mv_cesa
patches have not even been posted yet. I'm not sure whether the mv_cesa
stuff will be ready for 3.20, or 3.21 anyway. But having for sure this
mvebu-mbus improvement already scheduled for 3.20 means one less
dependency to handle when we'll submit the mv_cesa patches.

> [PATCH 5/6] bus: mvebu-mbus: use automatic I/O synchronization barriers
> branch fixes for next -rc

Right.

> [PATCH 6/6] ARM: mvebu: use arm_coherent_dma_ops
> Here i'm unsure. Is this a fix, or a cleanup?

It's a fix, because it is what brings this part of the cover letter:

    * It allows to switch to use the existing arm_coherent_dma_ops
      instead of mvebu specific DMA operations. arm_coherent_dma_ops
      make sure that DMA coherent mappings are mapped cacheable (which
      is possible in a cache-coherent platform), which is a necessary
      to make sure that both CPU-side and device-side accesses are
      done in a cacheable way. Without this, CPU-side accesses to DMA
      coherent mappings were made uncached, while device-side accesses
      were made in a cacheable way, leading to potentially
      unpredictable behavior.

Which is really a fix. So it should go in 3.19-rc.

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Thomas Petazzoni]

Hello Andrew,

On Tue, 30 Dec 2014 13:43:41 +0100, Thomas Petazzoni wrote:

> Michal Mazur (1):
>   bus: mvebu-mbus: fix support of MBus window 13 on Armada XP/375/38x
> 
> Thomas Petazzoni (5):
>   dt: bindings: update mvebu-mbus DT binding with new compatible
>     properties
>   ARM: mvebu: fix compatible strings of MBus on Armada 375 and Armada
>     38x
>   bus: mvebu-mbus: make sure SDRAM CS for DMA don't overlap the MBus
>     bridge window
>   bus: mvebu-mbus: use automatic I/O synchronization barriers
>   ARM: mvebu: use arm_coherent_dma_ops

Would it be possible to get those patches applied?

Simon has started doing some testing with the automatic I/O sync
barriers. It is still not working 100% properly, but it is apparently
not worse than it was. And clearly the current situation is not good,
and those patches make it slightly better.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[PATCH 1/6] dt: bindings: update mvebu-mbus DT binding with new compatible properties

[Andrew Lunn]

On Tue, Dec 30, 2014 at 01:43:42PM +0100, Thomas Petazzoni wrote:
> Contrary to what was originally thought, the Armada 375 and Armada 38x
> MBus windows hardware block is *not* compatible with the Armada 370,
> due to a difference in how window 13 is handled. It was rather
> compatible with the Armada XP MBus hardware block.
> 
> However, the DTs for Armada 375 and Armada 38x encode the following
> compatible string for MBus:
> 
>   compatible = "marvell,armada375-mbus", "marvell,armada370-mbus", "simple-bus";
>   compatible = "marvell,armada380-mbus", "marvell,armada370-mbus", "simple-bus";
> 
> So, by extending the mvebu-mbus DT binding to also cover the
> marvell,armada375-mbus and marvell,armada380-mbus compatible strings,
> we can define a new behavior for those SoCs without changing the DT.
> 
> Therefore, this commit adds those two new compatible strings to the DT
> binding documentation of mvebu-mbus. Note that it re-uses two existing
> duplicated lines for the armada370-mbus and armadaxp-mbus compatible
> strings.

Hi Thomas

Added to mvebu/dt for the next merge window.

      Andrew

> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  Documentation/devicetree/bindings/bus/mvebu-mbus.txt | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/devicetree/bindings/bus/mvebu-mbus.txt b/Documentation/devicetree/bindings/bus/mvebu-mbus.txt
> index 5e16c3c..fa6cde4 100644
> --- a/Documentation/devicetree/bindings/bus/mvebu-mbus.txt
> +++ b/Documentation/devicetree/bindings/bus/mvebu-mbus.txt
> @@ -6,8 +6,8 @@ Required properties:
>  - compatible:	 Should be set to one of the following:
>  		 marvell,armada370-mbus
>  		 marvell,armadaxp-mbus
> -		 marvell,armada370-mbus
> -		 marvell,armadaxp-mbus
> +		 marvell,armada375-mbus
> +		 marvell,armada380-mbus
>  		 marvell,kirkwood-mbus
>  		 marvell,dove-mbus
>  		 marvell,orion5x-88f5281-mbus
> -- 
> 2.1.0
> 

[PATCH 4/6] bus: mvebu-mbus: make sure SDRAM CS for DMA don't overlap the MBus bridge window

[Andrew Lunn]

On Tue, Dec 30, 2014 at 01:43:45PM +0100, Thomas Petazzoni wrote:
> The mvebu-mbus driver reads the SDRAM window registers, and make the
> information about the DRAM CS configuration available to device
> drivers using the mv_mbus_dram_info() API. This information is used by
> the DMA-capable device drivers to program their address decoding
> windows.
> 
> Until now, we were basically providing the SDRAM window register
> details as is. However, it turns out that the DMA capability of the
> CESA cryptographic engine consists in doing DMA being the DRAM and the
> crypto SRAM mapped as a MBus window. For this case, it is very
> important that the SDRAM CS information does not overlap with the MBus
> bridge window.
> 
> Therefore, this commit improves the mvebu-mbus driver to make sure we
> adjust the SDRAM CS information so that it doesn't overlap with the
> MBus bridge window. This problem was reported by Boris Brezillon,
> while working on the mv_cesa driver for Armada 37x/38x/XP. We use the
> memblock memory information to know where the usable RAM is located,
> as this information is guaranteed to be correct on all SoC variants.
> 
> We could have used the MBus bridge window registers on Armada 370/XP,
> but they are not really used on Armada 375/38x (Cortex-A9 based),
> since the PL310 L2 filtering is used instead to discriminate between
> RAM accesses and I/O accesses. Therefore, using the memblock
> information is more generic and works accross the different platforms.
> 
> Reported-by: Boris Brezillon <boris.brezillon@free-electrons.com>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Hi Thomas

Added to mvebu/soc for the next merge window.

      Andrew

> ---
> We have decided to not backport this commit to stable. Even though it
> is really fixing the DRAM CS information exposed to device drivers,
> there is no way in mainline for the current situation to cause any
> problem.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
>  drivers/bus/mvebu-mbus.c | 105 +++++++++++++++++++++++++++++++++++++++--------
>  1 file changed, 89 insertions(+), 16 deletions(-)
> 
> diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
> index b6694df..bb0dd21 100644
> --- a/drivers/bus/mvebu-mbus.c
> +++ b/drivers/bus/mvebu-mbus.c
> @@ -58,6 +58,7 @@
>  #include <linux/debugfs.h>
>  #include <linux/log2.h>
>  #include <linux/syscore_ops.h>
> +#include <linux/memblock.h>
>  
>  /*
>   * DDR target is the same on all platforms.
> @@ -97,7 +98,9 @@
>  
>  /* Relative to mbusbridge_base */
>  #define MBUS_BRIDGE_CTRL_OFF	0x0
> +#define  MBUS_BRIDGE_SIZE_MASK  0xffff0000
>  #define MBUS_BRIDGE_BASE_OFF	0x4
> +#define  MBUS_BRIDGE_BASE_MASK  0xffff0000
>  
>  /* Maximum number of windows, for all known platforms */
>  #define MBUS_WINS_MAX           20
> @@ -567,36 +570,106 @@ static unsigned int armada_xp_mbus_win_remap_offset(int win)
>  		return MVEBU_MBUS_NO_REMAP;
>  }
>  
> +/*
> + * Use the memblock information to find the MBus bridge hole in the
> + * physical address space.
> + */
> +static void __init
> +mvebu_mbus_find_bridge_hole(uint64_t *start, uint64_t *end)
> +{
> +	struct memblock_region *r;
> +	uint64_t s = 0;
> +
> +	for_each_memblock(memory, r) {
> +		/*
> +		 * This part of the memory is above 4 GB, so we don't
> +		 * care for the MBus bridge hole.
> +		 */
> +		if (r->base >= 0x100000000)
> +			continue;
> +
> +		/*
> +		 * The MBus bridge hole is at the end of the RAM under
> +		 * the 4 GB limit.
> +		 */
> +		if (r->base + r->size > s)
> +			s = r->base + r->size;
> +	}
> +
> +	*start = s;
> +	*end = 0x100000000;
> +}
> +
>  static void __init
>  mvebu_mbus_default_setup_cpu_target(struct mvebu_mbus_state *mbus)
>  {
>  	int i;
>  	int cs;
> +	uint64_t mbus_bridge_base, mbus_bridge_end;
>  
>  	mvebu_mbus_dram_info.mbus_dram_target_id = TARGET_DDR;
>  
> +	mvebu_mbus_find_bridge_hole(&mbus_bridge_base, &mbus_bridge_end);
> +
>  	for (i = 0, cs = 0; i < 4; i++) {
> -		u32 base = readl(mbus->sdramwins_base + DDR_BASE_CS_OFF(i));
> -		u32 size = readl(mbus->sdramwins_base + DDR_SIZE_CS_OFF(i));
> +		u64 base = readl(mbus->sdramwins_base + DDR_BASE_CS_OFF(i));
> +		u64 size = readl(mbus->sdramwins_base + DDR_SIZE_CS_OFF(i));
> +		u64 end;
> +		struct mbus_dram_window *w;
> +
> +		/* Ignore entries that are not enabled */
> +		if (!(size & DDR_SIZE_ENABLED))
> +			continue;
>  
>  		/*
> -		 * We only take care of entries for which the chip
> -		 * select is enabled, and that don't have high base
> -		 * address bits set (devices can only access the first
> -		 * 32 bits of the memory).
> +		 * Ignore entries whose base address is above 2^32,
> +		 * since devices cannot DMA to such high addresses
>  		 */
> -		if ((size & DDR_SIZE_ENABLED) &&
> -		    !(base & DDR_BASE_CS_HIGH_MASK)) {
> -			struct mbus_dram_window *w;
> +		if (base & DDR_BASE_CS_HIGH_MASK)
> +			continue;
>  
> -			w = &mvebu_mbus_dram_info.cs[cs++];
> -			w->cs_index = i;
> -			w->mbus_attr = 0xf & ~(1 << i);
> -			if (mbus->hw_io_coherency)
> -				w->mbus_attr |= ATTR_HW_COHERENCY;
> -			w->base = base & DDR_BASE_CS_LOW_MASK;
> -			w->size = (size | ~DDR_SIZE_MASK) + 1;
> +		base = base & DDR_BASE_CS_LOW_MASK;
> +		size = (size | ~DDR_SIZE_MASK) + 1;
> +		end = base + size;
> +
> +		/*
> +		 * Adjust base/size of the current CS to make sure it
> +		 * doesn't overlap with the MBus bridge hole. This is
> +		 * particularly important for devices that do DMA from
> +		 * DRAM to a SRAM mapped in a MBus window, such as the
> +		 * CESA cryptographic engine.
> +		 */
> +
> +		/*
> +		 * The CS is fully enclosed inside the MBus bridge
> +		 * area, so ignore it.
> +		 */
> +		if (base >= mbus_bridge_base && end <= mbus_bridge_end)
> +			continue;
> +
> +		/*
> +		 * Beginning of CS overlaps with end of MBus, raise CS
> +		 * base address, and shrink its size.
> +		 */
> +		if (base >= mbus_bridge_base && end > mbus_bridge_end) {
> +			size -= mbus_bridge_end - base;
> +			base = mbus_bridge_end;
>  		}
> +
> +		/*
> +		 * End of CS overlaps with beginning of MBus, shrink
> +		 * CS size.
> +		 */
> +		if (base < mbus_bridge_base && end > mbus_bridge_base)
> +			size -= end - mbus_bridge_base;
> +
> +		w = &mvebu_mbus_dram_info.cs[cs++];
> +		w->cs_index = i;
> +		w->mbus_attr = 0xf & ~(1 << i);
> +		if (mbus->hw_io_coherency)
> +			w->mbus_attr |= ATTR_HW_COHERENCY;
> +		w->base = base;
> +		w->size = size;
>  	}
>  	mvebu_mbus_dram_info.num_cs = cs;
>  }
> -- 
> 2.1.0
> 

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Andrew Lunn]

On Fri, Jan 09, 2015 at 04:52:04PM +0100, Thomas Petazzoni wrote:
> Hello Andrew,
> 
> On Tue, 30 Dec 2014 13:43:41 +0100, Thomas Petazzoni wrote:
> 
> > Michal Mazur (1):
> >   bus: mvebu-mbus: fix support of MBus window 13 on Armada XP/375/38x
> > 
> > Thomas Petazzoni (5):
> >   dt: bindings: update mvebu-mbus DT binding with new compatible
> >     properties
> >   ARM: mvebu: fix compatible strings of MBus on Armada 375 and Armada
> >     38x
> >   bus: mvebu-mbus: make sure SDRAM CS for DMA don't overlap the MBus
> >     bridge window
> >   bus: mvebu-mbus: use automatic I/O synchronization barriers
> >   ARM: mvebu: use arm_coherent_dma_ops
> 
> Would it be possible to get those patches applied?

Hi Thomas

I added in Arnd, Olof and Jason and would like there comments and
thoughts.

As you have seen, i've taken the "easy" ones, the ones for the next
merge window.

I'm a bit undecided about the others. There are two issues:

Fixing window 13 is a big fix. It is getting late in the -rc cycle,
which is partially my responsibility, since i was waiting for some
tested-by:'s. But these patches are also heading towards
stable. Stable rules state:

 - It must be obviously correct and tested.
 - It cannot be bigger than 100 lines, with context.
 - It must fix only one thing.
 - It must fix a real bug that bothers people (not a, "This could be a
   problem..." type thing).
 - It must fix a problem that causes a build error (but not for things
   marked CONFIG_BROKEN), an oops, a hang, data corruption, a real
   security issue, or some "oh, that's not good" issue.  In short, something
   critical.

The first patch is over 300 lines. Because of its size, i'm also not
able to say it is obviously correct. It does however tick some of the
other boxes, fix only one thing, fixes a real problem.

Is there a more minimal fix? How big an impact is there in just
disabling window 13? How much pressure do we have on windows? Can
Michal Mazur live with one less window?

We could then pushing this proper fix into the next merge window?

For the IO Coherency fixes, obviousness is an issue. This is
especially true since your own comment is "still not working 100%
properly, but it is apparently not worse than it was."

Maybe the correct fix for stable is to simply disable the I/O
coherency hardware. That at least makes mainline stable.  Once we have
a real, well tested, 100% fix, take it via the normal merge window.

Lets discuss this,

     Andrew

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Andrew Lunn]

On Fri, Jan 09, 2015 at 04:52:04PM +0100, Thomas Petazzoni wrote:
> Hello Andrew,
> 
> On Tue, 30 Dec 2014 13:43:41 +0100, Thomas Petazzoni wrote:
> 
> > Michal Mazur (1):
> >   bus: mvebu-mbus: fix support of MBus window 13 on Armada XP/375/38x
> > 
> > Thomas Petazzoni (5):
> >   dt: bindings: update mvebu-mbus DT binding with new compatible
> >     properties
> >   ARM: mvebu: fix compatible strings of MBus on Armada 375 and Armada
> >     38x
> >   bus: mvebu-mbus: make sure SDRAM CS for DMA don't overlap the MBus
> >     bridge window
> >   bus: mvebu-mbus: use automatic I/O synchronization barriers
> >   ARM: mvebu: use arm_coherent_dma_ops
> 
> Would it be possible to get those patches applied?

Hi Thomas

While we are discussing the fate of these patches, i thought i would
throw them into for-next for some testing. But i get a merge
conflict. This is caused by the re-arranged ordering of the patches.

Can you move:

bus: mvebu-mbus: make sure SDRAM CS for DMA don't overlap the MBus
bridge window

to after the i/o sync barriers fixes.

   Thanks
	Andrew

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Thomas Petazzoni]

Dear Andrew Lunn,

On Sat, 10 Jan 2015 17:30:01 +0100, Andrew Lunn wrote:

> Fixing window 13 is a big fix. It is getting late in the -rc cycle,
> which is partially my responsibility, since i was waiting for some
> tested-by:'s. But these patches are also heading towards
> stable. Stable rules state:
> 
>  - It must be obviously correct and tested.
>  - It cannot be bigger than 100 lines, with context.
>  - It must fix only one thing.
>  - It must fix a real bug that bothers people (not a, "This could be a
>    problem..." type thing).
>  - It must fix a problem that causes a build error (but not for things
>    marked CONFIG_BROKEN), an oops, a hang, data corruption, a real
>    security issue, or some "oh, that's not good" issue.  In short, something
>    critical.
> 
> The first patch is over 300 lines. Because of its size, i'm also not
> able to say it is obviously correct. It does however tick some of the
> other boxes, fix only one thing, fixes a real problem.
> 
> Is there a more minimal fix? How big an impact is there in just
> disabling window 13? How much pressure do we have on windows? Can
> Michal Mazur live with one less window?
> 
> We could then pushing this proper fix into the next merge window?

I believe pushing the proposed fix for the next merge window, and
having a simpler fix that consists in simply not using window 13 for
stable is a reasonable approach.

> For the IO Coherency fixes, obviousness is an issue. This is
> especially true since your own comment is "still not working 100%
> properly, but it is apparently not worse than it was."
> 
> Maybe the correct fix for stable is to simply disable the I/O
> coherency hardware. That at least makes mainline stable.  Once we have
> a real, well tested, 100% fix, take it via the normal merge window.

However, I'm not a big fan of this idea. I'd really like to have I/O
coherency progressively improved, and made working.

The patch is actually make the code *simpler* since it removes the
custom dma_map_ops and uses a set of operations that already exists in
the kernel. The changes in the mvebu-mbus driver are minimal.

Best regards,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Andrew Lunn]

On Sat, Jan 10, 2015 at 05:50:30PM +0100, Thomas Petazzoni wrote:
> Dear Andrew Lunn,
> 
> On Sat, 10 Jan 2015 17:30:01 +0100, Andrew Lunn wrote:
> 
> > Fixing window 13 is a big fix. It is getting late in the -rc cycle,
> > which is partially my responsibility, since i was waiting for some
> > tested-by:'s. But these patches are also heading towards
> > stable. Stable rules state:
> > 
> >  - It must be obviously correct and tested.
> >  - It cannot be bigger than 100 lines, with context.
> >  - It must fix only one thing.
> >  - It must fix a real bug that bothers people (not a, "This could be a
> >    problem..." type thing).
> >  - It must fix a problem that causes a build error (but not for things
> >    marked CONFIG_BROKEN), an oops, a hang, data corruption, a real
> >    security issue, or some "oh, that's not good" issue.  In short, something
> >    critical.
> > 
> > The first patch is over 300 lines. Because of its size, i'm also not
> > able to say it is obviously correct. It does however tick some of the
> > other boxes, fix only one thing, fixes a real problem.
> > 
> > Is there a more minimal fix? How big an impact is there in just
> > disabling window 13? How much pressure do we have on windows? Can
> > Michal Mazur live with one less window?
> > 
> > We could then pushing this proper fix into the next merge window?
> 
> I believe pushing the proposed fix for the next merge window, and
> having a simpler fix that consists in simply not using window 13 for
> stable is a reasonable approach.

Hi Thomas

Thanks for this. I will pull the two patches into soc today and into
mvebu/linux-next for some testing.
 
> > For the IO Coherency fixes, obviousness is an issue. This is
> > especially true since your own comment is "still not working 100%
> > properly, but it is apparently not worse than it was."
> > 
> > Maybe the correct fix for stable is to simply disable the I/O
> > coherency hardware. That at least makes mainline stable.  Once we have
> > a real, well tested, 100% fix, take it via the normal merge window.
> 
> However, I'm not a big fan of this idea. I'd really like to have I/O
> coherency progressively improved, and made working.
> 
> The patch is actually make the code *simpler* since it removes the
> custom dma_map_ops and uses a set of operations that already exists in
> the kernel. The changes in the mvebu-mbus driver are minimal.

Yes, the second patch is a lot of removal of code, which is always
nice.

But i would still like comments from others.

    Andrew

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Arnd Bergmann]

On Saturday 10 January 2015 17:50:30 Thomas Petazzoni wrote:
> 
> > For the IO Coherency fixes, obviousness is an issue. This is
> > especially true since your own comment is "still not working 100%
> > properly, but it is apparently not worse than it was."
> > 
> > Maybe the correct fix for stable is to simply disable the I/O
> > coherency hardware. That at least makes mainline stable.  Once we have
> > a real, well tested, 100% fix, take it via the normal merge window.
> 
> However, I'm not a big fan of this idea. I'd really like to have I/O
> coherency progressively improved, and made working.
> 
> The patch is actually make the code *simpler* since it removes the
> custom dma_map_ops and uses a set of operations that already exists in
> the kernel. The changes in the mvebu-mbus driver are minimal.

Despite my earlier analysis that concluded that the just using the
normal coherent dma_ops with this fix was correct on all hardware,
we have now seen a bug in the xor dmaengine driver (I haven't
seen a confirmation for my oneline fix yet, but I still think it's
the one), which gives us less confidence in the other drivers
being correct.

I think that your patch is the only sensible approach going forward
if we want coherent mappings, but we will need both more extensive
testing and more review of each affected device driver before
putting it into stable.

	Arnd

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Thomas Petazzoni]

Dear Arnd Bergmann,

On Sat, 10 Jan 2015 19:56:59 +0100, Arnd Bergmann wrote:

> Despite my earlier analysis that concluded that the just using the
> normal coherent dma_ops with this fix was correct on all hardware,
> we have now seen a bug in the xor dmaengine driver (I haven't
> seen a confirmation for my oneline fix yet, but I still think it's
> the one), which gives us less confidence in the other drivers
> being correct.
> 
> I think that your patch is the only sensible approach going forward
> if we want coherent mappings, but we will need both more extensive
> testing and more review of each affected device driver before
> putting it into stable.

We will indeed need to do more extensive testing and review. However, I
don't agree that this should prevent this patch from going to stable:
the current situation in the kernel (and also past kernels) is known to
be broken: DMA coherent mappings allocated by dma_alloc_coherent() are
*not* coherent in the current situation. Writes made by the device to
the memory are not guaranteed to be immediately visible to the CPU,
unless an explicit I/O sync barrier is done, which obviously is never
done for DMA coherent mappings since those are assumed by Linux to be
coherent, and therefore not require any cache maintenance operation.

So, while I clearly agree that we haven't yet gotten to the bottom of
all problems, this patch is going in the right direction and making
things slightly better.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Arnd Bergmann]

On Saturday 10 January 2015 20:57:21 Thomas Petazzoni wrote:
> Dear Arnd Bergmann,
> 
> On Sat, 10 Jan 2015 19:56:59 +0100, Arnd Bergmann wrote:
> 
> > Despite my earlier analysis that concluded that the just using the
> > normal coherent dma_ops with this fix was correct on all hardware,
> > we have now seen a bug in the xor dmaengine driver (I haven't
> > seen a confirmation for my oneline fix yet, but I still think it's
> > the one), which gives us less confidence in the other drivers
> > being correct.
> > 
> > I think that your patch is the only sensible approach going forward
> > if we want coherent mappings, but we will need both more extensive
> > testing and more review of each affected device driver before
> > putting it into stable.
> 
> We will indeed need to do more extensive testing and review. However, I
> don't agree that this should prevent this patch from going to stable:
> the current situation in the kernel (and also past kernels) is known to
> be broken: DMA coherent mappings allocated by dma_alloc_coherent() are
> *not* coherent in the current situation. Writes made by the device to
> the memory are not guaranteed to be immediately visible to the CPU,
> unless an explicit I/O sync barrier is done, which obviously is never
> done for DMA coherent mappings since those are assumed by Linux to be
> coherent, and therefore not require any cache maintenance operation.
> 
> So, while I clearly agree that we haven't yet gotten to the bottom of
> all problems, this patch is going in the right direction and making
> things slightly better.

Yes, but it may be better to stage this by first putting a patch
into stable that completely disables the broken coherent mapping,
and following up with another patch to bring the cohernet behavior
in sync with mainline once we are reasonably sure it's good.

	Arnd

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Thomas Petazzoni]

Dear Arnd Bergmann,

On Sat, 10 Jan 2015 21:40:07 +0100, Arnd Bergmann wrote:

> Yes, but it may be better to stage this by first putting a patch
> into stable that completely disables the broken coherent mapping,
> and following up with another patch to bring the cohernet behavior
> in sync with mainline once we are reasonably sure it's good.

Fine with me. I'll submit such patches next week. Though for the next
merge window, I'd like the proposed automatic I/O synchronization
barriers to be merged, so that we progressively make progress towards
the solution.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Arnd Bergmann]

On Saturday 10 January 2015 22:36:24 Thomas Petazzoni wrote:
> Dear Arnd Bergmann,
> 
> On Sat, 10 Jan 2015 21:40:07 +0100, Arnd Bergmann wrote:
> 
> > Yes, but it may be better to stage this by first putting a patch
> > into stable that completely disables the broken coherent mapping,
> > and following up with another patch to bring the cohernet behavior
> > in sync with mainline once we are reasonably sure it's good.
> 
> Fine with me. I'll submit such patches next week.

Ok.

> Though for the next
> merge window, I'd like the proposed automatic I/O synchronization
> barriers to be merged, so that we progressively make progress towards
> the solution.

Yes, definitely. I wouldn't mind taking them for 3.19 even if all mvebu
maintainers want that.

	Arnd

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Russell King - ARM Linux]

On Sat, Jan 10, 2015 at 08:57:21PM +0100, Thomas Petazzoni wrote:
> We will indeed need to do more extensive testing and review. However, I
> don't agree that this should prevent this patch from going to stable:
> the current situation in the kernel (and also past kernels) is known to
> be broken: DMA coherent mappings allocated by dma_alloc_coherent() are
> *not* coherent in the current situation. Writes made by the device to
> the memory are not guaranteed to be immediately visible to the CPU,
> unless an explicit I/O sync barrier is done, which obviously is never
> done for DMA coherent mappings since those are assumed by Linux to be
> coherent, and therefore not require any cache maintenance operation.

That's actually an incorrect statement.

On all ARMv6+ where DMA coherent memory is "normal memory, non-cached,
write combine" but because it's "normal memory", memory barriers are
required.  This is why we have the memory barriers in readl() and
writel().

Remember, DMA coherent memory is about avoiding effects from caching,
not about avoiding weakly ordered memory effects.

-- 
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up
according to speedtest.net.

[PATCH 0/6] ARM: mvebu: mvebu-mbus and I/O coherency fixes

[Thomas Petazzoni]

Russell,

On Mon, 12 Jan 2015 12:36:13 +0000, Russell King - ARM Linux wrote:
> On Sat, Jan 10, 2015 at 08:57:21PM +0100, Thomas Petazzoni wrote:
> > We will indeed need to do more extensive testing and review. However, I
> > don't agree that this should prevent this patch from going to stable:
> > the current situation in the kernel (and also past kernels) is known to
> > be broken: DMA coherent mappings allocated by dma_alloc_coherent() are
> > *not* coherent in the current situation. Writes made by the device to
> > the memory are not guaranteed to be immediately visible to the CPU,
> > unless an explicit I/O sync barrier is done, which obviously is never
> > done for DMA coherent mappings since those are assumed by Linux to be
> > coherent, and therefore not require any cache maintenance operation.
> 
> That's actually an incorrect statement.
> 
> On all ARMv6+ where DMA coherent memory is "normal memory, non-cached,
> write combine" but because it's "normal memory", memory barriers are
> required.  This is why we have the memory barriers in readl() and
> writel().

Except that a readl() or writel() do *not* imply the I/O
synchronization barrier that is needed on I/O coherent Marvell CPUs to
ensure that the CPU sees the changes made by a DMA master to the
memory. At least in the current kernel code, at least.

One option would maybe have been to add this I/O synchronization
barrier in the memory barrier implementation, but it seems a lot
simpler to rely on the automatic I/O synchronization mechanism instead.

Thanks,

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[PATCH] bus: mvebu-mbus: fix support of MBus window 13

[Andrew Lunn]

On Armada XP, 375 and 38x the MBus window 13 has the remap capability,
like windows 0 to 7. However, the mvebu-mbus driver isn't currently
taking into account this special case, which means that when window 13
is actually used, the remap registers are left to 0, making the device
using this MBus window unavailable.

As a minimal fix for stable, don't use window 13. A full fix will
follow later.

Fixes: fddddb52a6c ("bus: introduce an Marvell EBU MBus driver")
Cc: <stable@vger.kernel.org> # v3.10+
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
---
 drivers/bus/mvebu-mbus.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
index eb7682dc123b..81bf297f1034 100644
--- a/drivers/bus/mvebu-mbus.c
+++ b/drivers/bus/mvebu-mbus.c
@@ -210,12 +210,25 @@ static void mvebu_mbus_disable_window(struct mvebu_mbus_state *mbus,
 }
 
 /* Checks whether the given window number is available */
+
+/* On Armada XP, 375 and 38x the MBus window 13 has the remap
+ * capability, like windows 0 to 7. However, the mvebu-mbus driver
+ * isn't currently taking into account this special case, which means
+ * that when window 13 is actually used, the remap registers are left
+ * to 0, making the device using this MBus window unavailable. The
+ * quick fix for stable is to not use window 13. A follow up patch
+ * will correctly handle this window.
+*/
 static int mvebu_mbus_window_is_free(struct mvebu_mbus_state *mbus,
 				     const int win)
 {
 	void __iomem *addr = mbus->mbuswins_base +
 		mbus->soc->win_cfg_offset(win);
 	u32 ctrl = readl(addr + WIN_CTRL_OFF);
+
+	if (win == 13)
+		return false;
+
 	return !(ctrl & WIN_CTRL_ENABLE);
 }
 
-- 
2.1.4

[PATCH] bus: mvebu-mbus: fix support of MBus window 13

[Andrew Lunn]

On Sun, Jan 18, 2015 at 04:50:14PM +0100, Andrew Lunn wrote:
> On Armada XP, 375 and 38x the MBus window 13 has the remap capability,
> like windows 0 to 7. However, the mvebu-mbus driver isn't currently
> taking into account this special case, which means that when window 13
> is actually used, the remap registers are left to 0, making the device
> using this MBus window unavailable.
> 
> As a minimal fix for stable, don't use window 13. A full fix will
> follow later.

Hi Gregory, Michal

We need a minimal fix which can be back ported to stable, since the
real fix is large. Thomas is now on vacation for a week, so i've tried
to produce a fix. Please could you test this and let me know if it
works.

Thanks

	Andrew

> 
> Fixes: fddddb52a6c ("bus: introduce an Marvell EBU MBus driver")
> Cc: <stable@vger.kernel.org> # v3.10+
> Signed-off-by: Andrew Lunn <andrew@lunn.ch>
> ---
>  drivers/bus/mvebu-mbus.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
> index eb7682dc123b..81bf297f1034 100644
> --- a/drivers/bus/mvebu-mbus.c
> +++ b/drivers/bus/mvebu-mbus.c
> @@ -210,12 +210,25 @@ static void mvebu_mbus_disable_window(struct mvebu_mbus_state *mbus,
>  }
>  
>  /* Checks whether the given window number is available */
> +
> +/* On Armada XP, 375 and 38x the MBus window 13 has the remap
> + * capability, like windows 0 to 7. However, the mvebu-mbus driver
> + * isn't currently taking into account this special case, which means
> + * that when window 13 is actually used, the remap registers are left
> + * to 0, making the device using this MBus window unavailable. The
> + * quick fix for stable is to not use window 13. A follow up patch
> + * will correctly handle this window.
> +*/
>  static int mvebu_mbus_window_is_free(struct mvebu_mbus_state *mbus,
>  				     const int win)
>  {
>  	void __iomem *addr = mbus->mbuswins_base +
>  		mbus->soc->win_cfg_offset(win);
>  	u32 ctrl = readl(addr + WIN_CTRL_OFF);
> +
> +	if (win == 13)
> +		return false;
> +
>  	return !(ctrl & WIN_CTRL_ENABLE);
>  }
>  
> -- 
> 2.1.4
> 

[PATCH] bus: mvebu-mbus: fix support of MBus window 13

[Thomas Petazzoni]

Dear Andrew Lunn,

On Sun, 18 Jan 2015 16:50:14 +0100, Andrew Lunn wrote:
> On Armada XP, 375 and 38x the MBus window 13 has the remap capability,
> like windows 0 to 7. However, the mvebu-mbus driver isn't currently
> taking into account this special case, which means that when window 13
> is actually used, the remap registers are left to 0, making the device
> using this MBus window unavailable.
> 
> As a minimal fix for stable, don't use window 13. A full fix will
> follow later.
> 
> Fixes: fddddb52a6c ("bus: introduce an Marvell EBU MBus driver")
> Cc: <stable@vger.kernel.org> # v3.10+
> Signed-off-by: Andrew Lunn <andrew@lunn.ch>

Thanks a lot for doing this!

Reviewed-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

[PATCH] bus: mvebu-mbus: fix support of MBus window 13

[Andrew Lunn]

On Sun, Jan 18, 2015 at 04:50:14PM +0100, Andrew Lunn wrote:
> On Armada XP, 375 and 38x the MBus window 13 has the remap capability,
> like windows 0 to 7. However, the mvebu-mbus driver isn't currently
> taking into account this special case, which means that when window 13
> is actually used, the remap registers are left to 0, making the device
> using this MBus window unavailable.
> 
> As a minimal fix for stable, don't use window 13. A full fix will
> follow later.
> 
> Fixes: fddddb52a6c ("bus: introduce an Marvell EBU MBus driver")
> Cc: <stable@vger.kernel.org> # v3.10+
> Signed-off-by: Andrew Lunn <andrew@lunn.ch>

Added to mvebu/fixes-3 with Thomas's Reviewed-by:

      Andrew 

> ---
>  drivers/bus/mvebu-mbus.c | 13 +++++++++++++
>  1 file changed, 13 insertions(+)
> 
> diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
> index eb7682dc123b..81bf297f1034 100644
> --- a/drivers/bus/mvebu-mbus.c
> +++ b/drivers/bus/mvebu-mbus.c
> @@ -210,12 +210,25 @@ static void mvebu_mbus_disable_window(struct mvebu_mbus_state *mbus,
>  }
>  
>  /* Checks whether the given window number is available */
> +
> +/* On Armada XP, 375 and 38x the MBus window 13 has the remap
> + * capability, like windows 0 to 7. However, the mvebu-mbus driver
> + * isn't currently taking into account this special case, which means
> + * that when window 13 is actually used, the remap registers are left
> + * to 0, making the device using this MBus window unavailable. The
> + * quick fix for stable is to not use window 13. A follow up patch
> + * will correctly handle this window.
> +*/
>  static int mvebu_mbus_window_is_free(struct mvebu_mbus_state *mbus,
>  				     const int win)
>  {
>  	void __iomem *addr = mbus->mbuswins_base +
>  		mbus->soc->win_cfg_offset(win);
>  	u32 ctrl = readl(addr + WIN_CTRL_OFF);
> +
> +	if (win == 13)
> +		return false;
> +
>  	return !(ctrl & WIN_CTRL_ENABLE);
>  }
>  
> -- 
> 2.1.4
> 

[PATCH 1/6] dt: bindings: update mvebu-mbus DT binding with new compatible properties

[Andrew Lunn]

On Tue, Dec 30, 2014 at 01:43:42PM +0100, Thomas Petazzoni wrote:
> Contrary to what was originally thought, the Armada 375 and Armada 38x
> MBus windows hardware block is *not* compatible with the Armada 370,
> due to a difference in how window 13 is handled. It was rather
> compatible with the Armada XP MBus hardware block.
> 
> However, the DTs for Armada 375 and Armada 38x encode the following
> compatible string for MBus:
> 
>   compatible = "marvell,armada375-mbus", "marvell,armada370-mbus", "simple-bus";
>   compatible = "marvell,armada380-mbus", "marvell,armada370-mbus", "simple-bus";
> 
> So, by extending the mvebu-mbus DT binding to also cover the
> marvell,armada375-mbus and marvell,armada380-mbus compatible strings,
> we can define a new behavior for those SoCs without changing the DT.
> 
> Therefore, this commit adds those two new compatible strings to the DT
> binding documentation of mvebu-mbus. Note that it re-uses two existing
> duplicated lines for the armada370-mbus and armadaxp-mbus compatible
> strings.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Added to mvebu/dt

      Andrew

> ---
>  Documentation/devicetree/bindings/bus/mvebu-mbus.txt | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/devicetree/bindings/bus/mvebu-mbus.txt b/Documentation/devicetree/bindings/bus/mvebu-mbus.txt
> index 5e16c3c..fa6cde4 100644
> --- a/Documentation/devicetree/bindings/bus/mvebu-mbus.txt
> +++ b/Documentation/devicetree/bindings/bus/mvebu-mbus.txt
> @@ -6,8 +6,8 @@ Required properties:
>  - compatible:	 Should be set to one of the following:
>  		 marvell,armada370-mbus
>  		 marvell,armadaxp-mbus
> -		 marvell,armada370-mbus
> -		 marvell,armadaxp-mbus
> +		 marvell,armada375-mbus
> +		 marvell,armada380-mbus
>  		 marvell,kirkwood-mbus
>  		 marvell,dove-mbus
>  		 marvell,orion5x-88f5281-mbus
> -- 
> 2.1.0
> 

[PATCH 2/6] bus: mvebu-mbus: fix support of MBus window 13 on Armada XP/375/38x

[Andrew Lunn]

On Tue, Dec 30, 2014 at 01:43:43PM +0100, Thomas Petazzoni wrote:
> From: Michal Mazur <arg@semihalf.com>
> 
> On Armada XP, 375 and 38x the MBus window 13 has the remap capability,
> like windows 0 to 7. However, the mvebu-mbus driver isn't currently
> taking into account this special case, which means that when window 13
> is actually used, the remap registers are left to 0, making the device
> using this MBus window unavailable.
> 
> To make things even more fun, the hardware designers have chosen to
> put the window 13 remap registers in a completely custom location,
> using a logic that differs from the one used for all other remappable
> windows.
> 
> To solve this problem, this commit:
> 
>  * Adds a SoC specific function to calculate offset of remap registers
>    to the mvebu_mbus_soc_data structure. This function,
>    ->win_remap_offset(), returns the offset of the remap registers, or
>    MVEBU_MBUS_NO_REMAP if the window does not have the remap
>    capability. This new function replaces the previous integer field
>    num_remappable_wins, which was insufficient to encode the special
>    case of window 13.
> 
>  * Adds an implementation of the ->win_remap_offset() function for the
>    various SoC families. Some have 2 first windows that are remapable,
>    some the 4 first, some the 8 first, and then the Armada XP/375/38x
>    case where the 8 first are remapable plus the special window
>    13. This is implemented in functions
>    generic_mbus_win_remap_2_offset(),
>    generic_mbus_win_remap_4_offset(),
>    generic_mbus_win_remap_8_offset() and
>    armada_xp_mbus_win_remap_offset() respectively.
> 
>  * Change the code to use the ->win_remap_offset() function when
>    accessing the remap registers, and also to use a newly introduced
>    mvebu_mbus_window_is_remappable() helper function that tells
>    whether a given window is remapable or not.
> 
>  * Separate Armada 370 from XP/375/38X because the window 13 of Armada
>    370 does not support the remap capability.
> 
> [Thomas: adapted for the mainline kernel, minor clarifications in the
> code, reword the commit log.]
> 
> Fixes: fddddb52a6c ("bus: introduce an Marvell EBU MBus driver")
> Cc: <stable@vger.kernel.org> # v3.10+
> Signed-off-by: Michal Mazur <arg@semihalf.com>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

I merged in the revert of the simple fix for window 13. So the end
patch is below.

merged into mvebu/soc

      Andrew

>From 7fdf3d8a0316ce31f87513f903addcb8f3b0dfb2 Mon Sep 17 00:00:00 2001
From: Michal Mazur <arg@semihalf.com>
Date: Tue, 30 Dec 2014 13:43:43 +0100
Subject: [PATCH] bus: mvebu-mbus: fix support of MBus window 13 on Armada
 XP/375/38x

On Armada XP, 375 and 38x the MBus window 13 has the remap capability,
like windows 0 to 7. However, the mvebu-mbus driver isn't currently
taking into account this special case, which means that when window 13
is actually used, the remap registers are left to 0, making the device
using this MBus window unavailable.

To make things even more fun, the hardware designers have chosen to
put the window 13 remap registers in a completely custom location,
using a logic that differs from the one used for all other remappable
windows.

To solve this problem, this commit:

 * Adds a SoC specific function to calculate offset of remap registers
   to the mvebu_mbus_soc_data structure. This function,
   ->win_remap_offset(), returns the offset of the remap registers, or
   MVEBU_MBUS_NO_REMAP if the window does not have the remap
   capability. This new function replaces the previous integer field
   num_remappable_wins, which was insufficient to encode the special
   case of window 13.

 * Adds an implementation of the ->win_remap_offset() function for the
   various SoC families. Some have 2 first windows that are remapable,
   some the 4 first, some the 8 first, and then the Armada XP/375/38x
   case where the 8 first are remapable plus the special window
   13. This is implemented in functions
   generic_mbus_win_remap_2_offset(),
   generic_mbus_win_remap_4_offset(),
   generic_mbus_win_remap_8_offset() and
   armada_xp_mbus_win_remap_offset() respectively.

 * Change the code to use the ->win_remap_offset() function when
   accessing the remap registers, and also to use a newly introduced
   mvebu_mbus_window_is_remappable() helper function that tells
   whether a given window is remapable or not.

 * Separate Armada 370 from XP/375/38X because the window 13 of Armada
   370 does not support the remap capability.

[Thomas: adapted for the mainline kernel, minor clarifications in the
code, reword the commit log.]

Signed-off-by: Michal Mazur <arg@semihalf.com>
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
[Andrew Lunn <andrew@lunn.ch>: Undo the simple fix for stable]
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
---
 drivers/bus/mvebu-mbus.c | 164 +++++++++++++++++++++++++++++++++--------------
 1 file changed, 116 insertions(+), 48 deletions(-)

diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
index 061b5cf1d451..a62c8ae253c3 100644
--- a/drivers/bus/mvebu-mbus.c
+++ b/drivers/bus/mvebu-mbus.c
@@ -110,9 +110,9 @@ struct mvebu_mbus_state;
 
 struct mvebu_mbus_soc_data {
 	unsigned int num_wins;
-	unsigned int num_remappable_wins;
 	bool has_mbus_bridge;
 	unsigned int (*win_cfg_offset)(const int win);
+	unsigned int (*win_remap_offset)(const int win);
 	void (*setup_cpu_target)(struct mvebu_mbus_state *s);
 	int (*save_cpu_target)(struct mvebu_mbus_state *s,
 			       u32 *store_addr);
@@ -158,6 +158,13 @@ const struct mbus_dram_target_info *mv_mbus_dram_info(void)
 }
 EXPORT_SYMBOL_GPL(mv_mbus_dram_info);
 
+/* Checks whether the given window has remap capability */
+static bool mvebu_mbus_window_is_remappable(struct mvebu_mbus_state *mbus,
+					    const int win)
+{
+	return mbus->soc->win_remap_offset(win) != MVEBU_MBUS_NO_REMAP;
+}
+
 /*
  * Functions to manipulate the address decoding windows
  */
@@ -189,9 +196,12 @@ static void mvebu_mbus_read_window(struct mvebu_mbus_state *mbus,
 		*attr = (ctrlreg & WIN_CTRL_ATTR_MASK) >> WIN_CTRL_ATTR_SHIFT;
 
 	if (remap) {
-		if (win < mbus->soc->num_remappable_wins) {
-			u32 remap_low = readl(addr + WIN_REMAP_LO_OFF);
-			u32 remap_hi  = readl(addr + WIN_REMAP_HI_OFF);
+		if (mvebu_mbus_window_is_remappable(mbus, win)) {
+			u32 remap_low, remap_hi;
+			void __iomem *addr_rmp = mbus->mbuswins_base +
+				mbus->soc->win_remap_offset(win);
+			remap_low = readl(addr_rmp + WIN_REMAP_LO_OFF);
+			remap_hi  = readl(addr_rmp + WIN_REMAP_HI_OFF);
 			*remap = ((u64)remap_hi << 32) | remap_low;
 		} else
 			*remap = 0;
@@ -204,10 +214,11 @@ static void mvebu_mbus_disable_window(struct mvebu_mbus_state *mbus,
 	void __iomem *addr;
 
 	addr = mbus->mbuswins_base + mbus->soc->win_cfg_offset(win);
-
 	writel(0, addr + WIN_BASE_OFF);
 	writel(0, addr + WIN_CTRL_OFF);
-	if (win < mbus->soc->num_remappable_wins) {
+
+	if (mvebu_mbus_window_is_remappable(mbus, win)) {
+		addr = mbus->mbuswins_base + mbus->soc->win_remap_offset(win);
 		writel(0, addr + WIN_REMAP_LO_OFF);
 		writel(0, addr + WIN_REMAP_HI_OFF);
 	}
@@ -215,14 +226,6 @@ static void mvebu_mbus_disable_window(struct mvebu_mbus_state *mbus,
 
 /* Checks whether the given window number is available */
 
-/* On Armada XP, 375 and 38x the MBus window 13 has the remap
- * capability, like windows 0 to 7. However, the mvebu-mbus driver
- * isn't currently taking into account this special case, which means
- * that when window 13 is actually used, the remap registers are left
- * to 0, making the device using this MBus window unavailable. The
- * quick fix for stable is to not use window 13. A follow up patch
- * will correctly handle this window.
-*/
 static int mvebu_mbus_window_is_free(struct mvebu_mbus_state *mbus,
 				     const int win)
 {
@@ -230,9 +233,6 @@ static int mvebu_mbus_window_is_free(struct mvebu_mbus_state *mbus,
 		mbus->soc->win_cfg_offset(win);
 	u32 ctrl = readl(addr + WIN_CTRL_OFF);
 
-	if (win == 13)
-		return false;
-
 	return !(ctrl & WIN_CTRL_ENABLE);
 }
 
@@ -325,13 +325,17 @@ static int mvebu_mbus_setup_window(struct mvebu_mbus_state *mbus,
 
 	writel(base & WIN_BASE_LOW, addr + WIN_BASE_OFF);
 	writel(ctrl, addr + WIN_CTRL_OFF);
-	if (win < mbus->soc->num_remappable_wins) {
+
+	if (mvebu_mbus_window_is_remappable(mbus, win)) {
+		void __iomem *addr_rmp = mbus->mbuswins_base +
+			mbus->soc->win_remap_offset(win);
+
 		if (remap == MVEBU_MBUS_NO_REMAP)
 			remap_addr = base;
 		else
 			remap_addr = remap;
-		writel(remap_addr & WIN_REMAP_LOW, addr + WIN_REMAP_LO_OFF);
-		writel(0, addr + WIN_REMAP_HI_OFF);
+		writel(remap_addr & WIN_REMAP_LOW, addr_rmp + WIN_REMAP_LO_OFF);
+		writel(0, addr_rmp + WIN_REMAP_HI_OFF);
 	}
 
 	return 0;
@@ -345,19 +349,27 @@ static int mvebu_mbus_alloc_window(struct mvebu_mbus_state *mbus,
 	int win;
 
 	if (remap == MVEBU_MBUS_NO_REMAP) {
-		for (win = mbus->soc->num_remappable_wins;
-		     win < mbus->soc->num_wins; win++)
+		for (win = 0; win < mbus->soc->num_wins; win++) {
+			if (mvebu_mbus_window_is_remappable(mbus, win))
+				continue;
+
 			if (mvebu_mbus_window_is_free(mbus, win))
 				return mvebu_mbus_setup_window(mbus, win, base,
 							       size, remap,
 							       target, attr);
+		}
 	}
 
+	for (win = 0; win < mbus->soc->num_wins; win++) {
+		/* Skip window if need remap but is not supported */
+		if ((remap != MVEBU_MBUS_NO_REMAP) &&
+		    !mvebu_mbus_window_is_remappable(mbus, win))
+			continue;
 
-	for (win = 0; win < mbus->soc->num_wins; win++)
 		if (mvebu_mbus_window_is_free(mbus, win))
 			return mvebu_mbus_setup_window(mbus, win, base, size,
 						       remap, target, attr);
+	}
 
 	return -ENOMEM;
 }
@@ -469,7 +481,7 @@ static int mvebu_devs_debug_show(struct seq_file *seq, void *v)
 		    ((wbase & (u64)(wsize - 1)) != 0))
 			seq_puts(seq, " (Invalid base/size!!)");
 
-		if (win < mbus->soc->num_remappable_wins) {
+		if (mvebu_mbus_window_is_remappable(mbus, win)) {
 			seq_printf(seq, " (remap %016llx)\n",
 				   (unsigned long long)wremap);
 		} else
@@ -495,12 +507,12 @@ static const struct file_operations mvebu_devs_debug_fops = {
  * SoC-specific functions and definitions
  */
 
-static unsigned int orion_mbus_win_offset(int win)
+static unsigned int generic_mbus_win_cfg_offset(int win)
 {
 	return win << 4;
 }
 
-static unsigned int armada_370_xp_mbus_win_offset(int win)
+static unsigned int armada_370_xp_mbus_win_cfg_offset(int win)
 {
 	/* The register layout is a bit annoying and the below code
 	 * tries to cope with it.
@@ -520,7 +532,7 @@ static unsigned int armada_370_xp_mbus_win_offset(int win)
 		return 0x90 + ((win - 8) << 3);
 }
 
-static unsigned int mv78xx0_mbus_win_offset(int win)
+static unsigned int mv78xx0_mbus_win_cfg_offset(int win)
 {
 	if (win < 8)
 		return win << 4;
@@ -528,6 +540,40 @@ static unsigned int mv78xx0_mbus_win_offset(int win)
 		return 0x900 + ((win - 8) << 4);
 }
 
+static unsigned int generic_mbus_win_remap_2_offset(int win)
+{
+	if (win < 2)
+		return generic_mbus_win_cfg_offset(win);
+	else
+		return MVEBU_MBUS_NO_REMAP;
+}
+
+static unsigned int generic_mbus_win_remap_4_offset(int win)
+{
+	if (win < 4)
+		return generic_mbus_win_cfg_offset(win);
+	else
+		return MVEBU_MBUS_NO_REMAP;
+}
+
+static unsigned int generic_mbus_win_remap_8_offset(int win)
+{
+	if (win < 8)
+		return generic_mbus_win_cfg_offset(win);
+	else
+		return MVEBU_MBUS_NO_REMAP;
+}
+
+static unsigned int armada_xp_mbus_win_remap_offset(int win)
+{
+	if (win < 8)
+		return generic_mbus_win_cfg_offset(win);
+	else if (win == 13)
+		return 0xF0 - WIN_REMAP_LO_OFF;
+	else
+		return MVEBU_MBUS_NO_REMAP;
+}
+
 static void __init
 mvebu_mbus_default_setup_cpu_target(struct mvebu_mbus_state *mbus)
 {
@@ -637,30 +683,40 @@ int mvebu_mbus_save_cpu_target(u32 *store_addr)
 	return mbus_state.soc->save_cpu_target(&mbus_state, store_addr);
 }
 
-static const struct mvebu_mbus_soc_data armada_370_xp_mbus_data = {
+static const struct mvebu_mbus_soc_data armada_370_mbus_data = {
 	.num_wins            = 20,
-	.num_remappable_wins = 8,
 	.has_mbus_bridge     = true,
-	.win_cfg_offset      = armada_370_xp_mbus_win_offset,
+	.win_cfg_offset      = armada_370_xp_mbus_win_cfg_offset,
+	.win_remap_offset    = generic_mbus_win_remap_8_offset,
+	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
+	.show_cpu_target     = mvebu_sdram_debug_show_orion,
 	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
+};
+
+static const struct mvebu_mbus_soc_data armada_xp_mbus_data = {
+	.num_wins            = 20,
+	.has_mbus_bridge     = true,
+	.win_cfg_offset      = armada_370_xp_mbus_win_cfg_offset,
+	.win_remap_offset    = armada_xp_mbus_win_remap_offset,
 	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_orion,
+	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
 };
 
 static const struct mvebu_mbus_soc_data kirkwood_mbus_data = {
 	.num_wins            = 8,
-	.num_remappable_wins = 4,
-	.win_cfg_offset      = orion_mbus_win_offset,
+	.win_cfg_offset      = generic_mbus_win_cfg_offset,
 	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
+	.win_remap_offset    = generic_mbus_win_remap_4_offset,
 	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_orion,
 };
 
 static const struct mvebu_mbus_soc_data dove_mbus_data = {
 	.num_wins            = 8,
-	.num_remappable_wins = 4,
-	.win_cfg_offset      = orion_mbus_win_offset,
+	.win_cfg_offset      = generic_mbus_win_cfg_offset,
 	.save_cpu_target     = mvebu_mbus_dove_save_cpu_target,
+	.win_remap_offset    = generic_mbus_win_remap_4_offset,
 	.setup_cpu_target    = mvebu_mbus_dove_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_dove,
 };
@@ -671,36 +727,40 @@ static const struct mvebu_mbus_soc_data dove_mbus_data = {
  */
 static const struct mvebu_mbus_soc_data orion5x_4win_mbus_data = {
 	.num_wins            = 8,
-	.num_remappable_wins = 4,
-	.win_cfg_offset      = orion_mbus_win_offset,
+	.win_cfg_offset      = generic_mbus_win_cfg_offset,
 	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
+	.win_remap_offset    = generic_mbus_win_remap_4_offset,
 	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_orion,
 };
 
 static const struct mvebu_mbus_soc_data orion5x_2win_mbus_data = {
 	.num_wins            = 8,
-	.num_remappable_wins = 2,
-	.win_cfg_offset      = orion_mbus_win_offset,
+	.win_cfg_offset      = generic_mbus_win_cfg_offset,
 	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
+	.win_remap_offset    = generic_mbus_win_remap_2_offset,
 	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_orion,
 };
 
 static const struct mvebu_mbus_soc_data mv78xx0_mbus_data = {
 	.num_wins            = 14,
-	.num_remappable_wins = 8,
-	.win_cfg_offset      = mv78xx0_mbus_win_offset,
+	.win_cfg_offset      = mv78xx0_mbus_win_cfg_offset,
 	.save_cpu_target     = mvebu_mbus_default_save_cpu_target,
+	.win_remap_offset    = generic_mbus_win_remap_8_offset,
 	.setup_cpu_target    = mvebu_mbus_default_setup_cpu_target,
 	.show_cpu_target     = mvebu_sdram_debug_show_orion,
 };
 
 static const struct of_device_id of_mvebu_mbus_ids[] = {
 	{ .compatible = "marvell,armada370-mbus",
-	  .data = &armada_370_xp_mbus_data, },
+	  .data = &armada_370_mbus_data, },
+	{ .compatible = "marvell,armada375-mbus",
+	  .data = &armada_xp_mbus_data, },
+	{ .compatible = "marvell,armada380-mbus",
+	  .data = &armada_xp_mbus_data, },
 	{ .compatible = "marvell,armadaxp-mbus",
-	  .data = &armada_370_xp_mbus_data, },
+	  .data = &armada_xp_mbus_data, },
 	{ .compatible = "marvell,kirkwood-mbus",
 	  .data = &kirkwood_mbus_data, },
 	{ .compatible = "marvell,dove-mbus",
@@ -807,15 +867,19 @@ static int mvebu_mbus_suspend(void)
 	for (win = 0; win < s->soc->num_wins; win++) {
 		void __iomem *addr = s->mbuswins_base +
 			s->soc->win_cfg_offset(win);
+		void __iomem *addr_rmp;
 
 		s->wins[win].base = readl(addr + WIN_BASE_OFF);
 		s->wins[win].ctrl = readl(addr + WIN_CTRL_OFF);
 
-		if (win >= s->soc->num_remappable_wins)
+		if (!mvebu_mbus_window_is_remappable(s, win))
 			continue;
 
-		s->wins[win].remap_lo = readl(addr + WIN_REMAP_LO_OFF);
-		s->wins[win].remap_hi = readl(addr + WIN_REMAP_HI_OFF);
+		addr_rmp = s->mbuswins_base +
+			s->soc->win_remap_offset(win);
+
+		s->wins[win].remap_lo = readl(addr_rmp + WIN_REMAP_LO_OFF);
+		s->wins[win].remap_hi = readl(addr_rmp + WIN_REMAP_HI_OFF);
 	}
 
 	s->mbus_bridge_ctrl = readl(s->mbusbridge_base +
@@ -839,15 +903,19 @@ static void mvebu_mbus_resume(void)
 	for (win = 0; win < s->soc->num_wins; win++) {
 		void __iomem *addr = s->mbuswins_base +
 			s->soc->win_cfg_offset(win);
+		void __iomem *addr_rmp;
 
 		writel(s->wins[win].base, addr + WIN_BASE_OFF);
 		writel(s->wins[win].ctrl, addr + WIN_CTRL_OFF);
 
-		if (win >= s->soc->num_remappable_wins)
+		if (!mvebu_mbus_window_is_remappable(s, win))
 			continue;
 
-		writel(s->wins[win].remap_lo, addr + WIN_REMAP_LO_OFF);
-		writel(s->wins[win].remap_hi, addr + WIN_REMAP_HI_OFF);
+		addr_rmp = s->mbuswins_base +
+			s->soc->win_remap_offset(win);
+
+		writel(s->wins[win].remap_lo, addr_rmp + WIN_REMAP_LO_OFF);
+		writel(s->wins[win].remap_hi, addr_rmp + WIN_REMAP_HI_OFF);
 	}
 }
 
-- 
2.1.4

[PATCH 3/6] ARM: mvebu: fix compatible strings of MBus on Armada 375 and Armada 38x

[Andrew Lunn]

On Tue, Dec 30, 2014 at 01:43:44PM +0100, Thomas Petazzoni wrote:
> Due to the special handling of window 13 on Armada 375 and Armada 38x
> (similar to Armada XP), the MBus hardware block is *not* compatible
> with the one used on Armada 370. Using the Armada 370 compatible
> string on Armada 375 and 38x will lead to a non-working device if
> window 13 ends up being used.
> 
> Fixes: 4de59085091 ("ARM: mvebu: add Device Tree description of the Armada 375 SoC")
> Fixes: 0d3d96ab005 ("ARM: mvebu: add Device Tree description of the Armada 380/385 SoCs")
> Cc: <stable@vger.kernel.org> # v3.15+
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Added to mvebu/dt, without the Fixes: tags.

      Andrew


> ---
>  arch/arm/boot/dts/armada-375.dtsi | 2 +-
>  arch/arm/boot/dts/armada-38x.dtsi | 3 +--
>  2 files changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/arm/boot/dts/armada-375.dtsi b/arch/arm/boot/dts/armada-375.dtsi
> index 50096d3..126bf7e 100644
> --- a/arch/arm/boot/dts/armada-375.dtsi
> +++ b/arch/arm/boot/dts/armada-375.dtsi
> @@ -63,7 +63,7 @@
>  	};
>  
>  	soc {
> -		compatible = "marvell,armada375-mbus", "marvell,armada370-mbus", "simple-bus";
> +		compatible = "marvell,armada375-mbus", "simple-bus";
>  		#address-cells = <2>;
>  		#size-cells = <1>;
>  		controller = <&mbusc>;
> diff --git a/arch/arm/boot/dts/armada-38x.dtsi b/arch/arm/boot/dts/armada-38x.dtsi
> index 74391da..7d8295c 100644
> --- a/arch/arm/boot/dts/armada-38x.dtsi
> +++ b/arch/arm/boot/dts/armada-38x.dtsi
> @@ -31,8 +31,7 @@
>  	};
>  
>  	soc {
> -		compatible = "marvell,armada380-mbus", "marvell,armada370-mbus",
> -			     "simple-bus";
> +		compatible = "marvell,armada380-mbus", "simple-bus";
>  		#address-cells = <2>;
>  		#size-cells = <1>;
>  		controller = <&mbusc>;
> -- 
> 2.1.0
> 

[PATCH 4/6] bus: mvebu-mbus: make sure SDRAM CS for DMA don't overlap the MBus bridge window

[Andrew Lunn]

On Tue, Dec 30, 2014 at 01:43:45PM +0100, Thomas Petazzoni wrote:
> The mvebu-mbus driver reads the SDRAM window registers, and make the
> information about the DRAM CS configuration available to device
> drivers using the mv_mbus_dram_info() API. This information is used by
> the DMA-capable device drivers to program their address decoding
> windows.
> 
> Until now, we were basically providing the SDRAM window register
> details as is. However, it turns out that the DMA capability of the
> CESA cryptographic engine consists in doing DMA being the DRAM and the
> crypto SRAM mapped as a MBus window. For this case, it is very
> important that the SDRAM CS information does not overlap with the MBus
> bridge window.
> 
> Therefore, this commit improves the mvebu-mbus driver to make sure we
> adjust the SDRAM CS information so that it doesn't overlap with the
> MBus bridge window. This problem was reported by Boris Brezillon,
> while working on the mv_cesa driver for Armada 37x/38x/XP. We use the
> memblock memory information to know where the usable RAM is located,
> as this information is guaranteed to be correct on all SoC variants.
> 
> We could have used the MBus bridge window registers on Armada 370/XP,
> but they are not really used on Armada 375/38x (Cortex-A9 based),
> since the PL310 L2 filtering is used instead to discriminate between
> RAM accesses and I/O accesses. Therefore, using the memblock
> information is more generic and works accross the different platforms.
> 
> Reported-by: Boris Brezillon <boris.brezillon@free-electrons.com>
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> ---
> We have decided to not backport this commit to stable. Even though it
> is really fixing the DRAM CS information exposed to device drivers,
> there is no way in mainline for the current situation to cause any
> problem.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>

Added to mvebu/soc

      Andrew

> ---
>  drivers/bus/mvebu-mbus.c | 105 +++++++++++++++++++++++++++++++++++++++--------
>  1 file changed, 89 insertions(+), 16 deletions(-)
> 
> diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
> index b6694df..bb0dd21 100644
> --- a/drivers/bus/mvebu-mbus.c
> +++ b/drivers/bus/mvebu-mbus.c
> @@ -58,6 +58,7 @@
>  #include <linux/debugfs.h>
>  #include <linux/log2.h>
>  #include <linux/syscore_ops.h>
> +#include <linux/memblock.h>
>  
>  /*
>   * DDR target is the same on all platforms.
> @@ -97,7 +98,9 @@
>  
>  /* Relative to mbusbridge_base */
>  #define MBUS_BRIDGE_CTRL_OFF	0x0
> +#define  MBUS_BRIDGE_SIZE_MASK  0xffff0000
>  #define MBUS_BRIDGE_BASE_OFF	0x4
> +#define  MBUS_BRIDGE_BASE_MASK  0xffff0000
>  
>  /* Maximum number of windows, for all known platforms */
>  #define MBUS_WINS_MAX           20
> @@ -567,36 +570,106 @@ static unsigned int armada_xp_mbus_win_remap_offset(int win)
>  		return MVEBU_MBUS_NO_REMAP;
>  }
>  
> +/*
> + * Use the memblock information to find the MBus bridge hole in the
> + * physical address space.
> + */
> +static void __init
> +mvebu_mbus_find_bridge_hole(uint64_t *start, uint64_t *end)
> +{
> +	struct memblock_region *r;
> +	uint64_t s = 0;
> +
> +	for_each_memblock(memory, r) {
> +		/*
> +		 * This part of the memory is above 4 GB, so we don't
> +		 * care for the MBus bridge hole.
> +		 */
> +		if (r->base >= 0x100000000)
> +			continue;
> +
> +		/*
> +		 * The MBus bridge hole is at the end of the RAM under
> +		 * the 4 GB limit.
> +		 */
> +		if (r->base + r->size > s)
> +			s = r->base + r->size;
> +	}
> +
> +	*start = s;
> +	*end = 0x100000000;
> +}
> +
>  static void __init
>  mvebu_mbus_default_setup_cpu_target(struct mvebu_mbus_state *mbus)
>  {
>  	int i;
>  	int cs;
> +	uint64_t mbus_bridge_base, mbus_bridge_end;
>  
>  	mvebu_mbus_dram_info.mbus_dram_target_id = TARGET_DDR;
>  
> +	mvebu_mbus_find_bridge_hole(&mbus_bridge_base, &mbus_bridge_end);
> +
>  	for (i = 0, cs = 0; i < 4; i++) {
> -		u32 base = readl(mbus->sdramwins_base + DDR_BASE_CS_OFF(i));
> -		u32 size = readl(mbus->sdramwins_base + DDR_SIZE_CS_OFF(i));
> +		u64 base = readl(mbus->sdramwins_base + DDR_BASE_CS_OFF(i));
> +		u64 size = readl(mbus->sdramwins_base + DDR_SIZE_CS_OFF(i));
> +		u64 end;
> +		struct mbus_dram_window *w;
> +
> +		/* Ignore entries that are not enabled */
> +		if (!(size & DDR_SIZE_ENABLED))
> +			continue;
>  
>  		/*
> -		 * We only take care of entries for which the chip
> -		 * select is enabled, and that don't have high base
> -		 * address bits set (devices can only access the first
> -		 * 32 bits of the memory).
> +		 * Ignore entries whose base address is above 2^32,
> +		 * since devices cannot DMA to such high addresses
>  		 */
> -		if ((size & DDR_SIZE_ENABLED) &&
> -		    !(base & DDR_BASE_CS_HIGH_MASK)) {
> -			struct mbus_dram_window *w;
> +		if (base & DDR_BASE_CS_HIGH_MASK)
> +			continue;
>  
> -			w = &mvebu_mbus_dram_info.cs[cs++];
> -			w->cs_index = i;
> -			w->mbus_attr = 0xf & ~(1 << i);
> -			if (mbus->hw_io_coherency)
> -				w->mbus_attr |= ATTR_HW_COHERENCY;
> -			w->base = base & DDR_BASE_CS_LOW_MASK;
> -			w->size = (size | ~DDR_SIZE_MASK) + 1;
> +		base = base & DDR_BASE_CS_LOW_MASK;
> +		size = (size | ~DDR_SIZE_MASK) + 1;
> +		end = base + size;
> +
> +		/*
> +		 * Adjust base/size of the current CS to make sure it
> +		 * doesn't overlap with the MBus bridge hole. This is
> +		 * particularly important for devices that do DMA from
> +		 * DRAM to a SRAM mapped in a MBus window, such as the
> +		 * CESA cryptographic engine.
> +		 */
> +
> +		/*
> +		 * The CS is fully enclosed inside the MBus bridge
> +		 * area, so ignore it.
> +		 */
> +		if (base >= mbus_bridge_base && end <= mbus_bridge_end)
> +			continue;
> +
> +		/*
> +		 * Beginning of CS overlaps with end of MBus, raise CS
> +		 * base address, and shrink its size.
> +		 */
> +		if (base >= mbus_bridge_base && end > mbus_bridge_end) {
> +			size -= mbus_bridge_end - base;
> +			base = mbus_bridge_end;
>  		}
> +
> +		/*
> +		 * End of CS overlaps with beginning of MBus, shrink
> +		 * CS size.
> +		 */
> +		if (base < mbus_bridge_base && end > mbus_bridge_base)
> +			size -= end - mbus_bridge_base;
> +
> +		w = &mvebu_mbus_dram_info.cs[cs++];
> +		w->cs_index = i;
> +		w->mbus_attr = 0xf & ~(1 << i);
> +		if (mbus->hw_io_coherency)
> +			w->mbus_attr |= ATTR_HW_COHERENCY;
> +		w->base = base;
> +		w->size = size;
>  	}
>  	mvebu_mbus_dram_info.num_cs = cs;
>  }
> -- 
> 2.1.0
> 

[PATCH 2/6] bus: mvebu-mbus: fix support of MBus window 13 on Armada XP/375/38x

[Thomas Petazzoni]

Dear Andrew Lunn,

On Mon, 19 Jan 2015 23:29:30 +0100, Andrew Lunn wrote:

> > [Thomas: adapted for the mainline kernel, minor clarifications in the
> > code, reword the commit log.]
> > 
> > Fixes: fddddb52a6c ("bus: introduce an Marvell EBU MBus driver")
> > Cc: <stable@vger.kernel.org> # v3.10+
> > Signed-off-by: Michal Mazur <arg@semihalf.com>
> > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@free-electrons.com>
> 
> I merged in the revert of the simple fix for window 13. So the end
> patch is below.
> 
> merged into mvebu/soc

Quickly looked at it, it looks fine to me.

Thanks again for handling this!

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com