[PATCH v5 0/3] Set SMMU s2 input-size based on p2m tables

[PATCH v5 0/3] Set SMMU s2 input-size based on p2m tables

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com>

Hi,

This is a fix for the issue I'm seeing on ZynqMP with missmatched
setup of the SMMU and the shared p2m page-tables with the CPU.

This implements a global p2m_ipa_bits cap for S2 input-size as
discussed in the previous RFC.

Best regards,
Edgar

Changelog:
v4 -> v5:
* Make p2m_ipa_bits __read_mostly.
* Make p2m_ipa_bits const for ARM32.

v3 -> v4:
* Replace ASSERT on supported IPA sizes with returnig error.
* Remove redundant 'addresses' after IPA.

v2 -> v3:
* pfn -> ipa.
* Fix typos in commit msg for 3/3.

v1 -> v2:
* Use a global pfn bitsize instead of a per-domain one.

Edgar E. Iglesias (3):
  xen/arm: Re-order iommu_setup to after setup_virt_paging
  xen/arm: Add p2m_ipa_bits
  xen/iommu: arm: Use p2m_ipa_bits as stage2 input size

 xen/arch/arm/p2m.c                 |  3 +++
 xen/arch/arm/setup.c               |  4 ++--
 xen/drivers/passthrough/arm/smmu.c | 10 ++++++++--
 xen/include/asm-arm/p2m.h          |  3 +++
 4 files changed, 16 insertions(+), 4 deletions(-)

-- 
1.9.1

[PATCH v5 1/3] xen/arm: Re-order iommu_setup to after setup_virt_paging

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com>

This is needed to allow the paging setup to probe for
IPA bit sizes to be used in p2m tables prior to iommu setup.

Acked-by: Ian Campbell <ian.campbell@citrix.com>
Reviewed-by: Julien Grall <julien.grall@citrix.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
---
 xen/arch/arm/setup.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 4ec7c13..797673b 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -800,8 +800,6 @@ void __init start_xen(unsigned long boot_phys_offset,
     local_irq_enable();
     local_abort_enable();
 
-    iommu_setup();
-
     smp_prepare_cpus(cpus);
 
     initialize_keytable();
@@ -825,6 +823,8 @@ void __init start_xen(unsigned long boot_phys_offset,
 
     setup_virt_paging();
 
+    iommu_setup();
+
     do_initcalls();
 
     /* Create initial domain 0. */
-- 
1.9.1

[PATCH v5 2/3] xen/arm: Add p2m_ipa_bits

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com>

Export p2m_ipa_bits holding the bit size of IPAs used in p2m tables.

Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
---
 xen/arch/arm/p2m.c        | 3 +++
 xen/include/asm-arm/p2m.h | 3 +++
 2 files changed, 6 insertions(+)

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 59dd23a..98f1019 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -17,9 +17,11 @@
 #ifdef CONFIG_ARM_64
 static unsigned int __read_mostly p2m_root_order;
 static unsigned int __read_mostly p2m_root_level;
+unsigned int __read_mostly p2m_ipa_bits;
 #define P2M_ROOT_ORDER    p2m_root_order
 #define P2M_ROOT_LEVEL p2m_root_level
 #else
+const unsigned int __read_mostly p2m_ipa_bits = 40;
 /* First level P2M is alway 2 consecutive pages */
 #define P2M_ROOT_LEVEL 1
 #define P2M_ROOT_ORDER    1
@@ -1560,6 +1562,7 @@ void __init setup_virt_paging(void)
 
     p2m_root_order = pa_range_info[pa_range].root_order;
     p2m_root_level = 2 - pa_range_info[pa_range].sl0;
+    p2m_ipa_bits = 64 - pa_range_info[pa_range].t0sz;
 
     printk("P2M: %d-bit IPA with %d-bit PA\n",
            64 - pa_range_info[pa_range].t0sz,
diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
index 341df55..63748ef 100644
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -10,6 +10,9 @@
 
 #define paddr_bits PADDR_BITS
 
+/* Holds the bit size of IPAs in p2m tables.  */
+extern unsigned int p2m_ipa_bits;
+
 struct domain;
 
 extern void memory_type_changed(struct domain *);
-- 
1.9.1

Re: [PATCH v5 2/3] xen/arm: Add p2m_ipa_bits

[Julien Grall]

Hi Edgar,

On 06/05/15 06:28, Edgar E. Iglesias wrote:
> diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
> index 59dd23a..98f1019 100644
> --- a/xen/arch/arm/p2m.c
> +++ b/xen/arch/arm/p2m.c
> @@ -17,9 +17,11 @@

[..]

> +const unsigned int __read_mostly p2m_ipa_bits = 40;

[..]

> diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
> index 341df55..63748ef 100644
> --- a/xen/include/asm-arm/p2m.h
> +++ b/xen/include/asm-arm/p2m.h
> @@ -10,6 +10,9 @@
>  
>  #define paddr_bits PADDR_BITS
>  
> +/* Holds the bit size of IPAs in p2m tables.  */
> +extern unsigned int p2m_ipa_bits;
> +

The two declarations of p2m_ipa_bits are different on ARM32. GCC will
fail to build with:

p2m.c:24:34: error: conflicting type qualifiers for ‘p2m_ipa_bits’
 const unsigned int __read_mostly p2m_ipa_bits = 40;
                                  ^
In file included from works/xen/xen/include/asm/domain.h:8:0,
                 from works/xen/xen/include/xen/domain.h:6,
                 from works/xen/xen/include/xen/sched.h:10,
                 from p2m.c:2:
works/xen/xen/include/asm/p2m.h:14:21: note: previous declaration of
‘p2m_ipa_bits’ was here
 extern unsigned int p2m_ipa_bits;
                     ^
I would either drop the const in p2m.c or use a define for ARM32 in
asm-arm/p2m.h

Regards,

-- 
Julien Grall

Re: [PATCH v5 2/3] xen/arm: Add p2m_ipa_bits

[Edgar E. Iglesias]

On Wed, May 06, 2015 at 02:00:36PM +0100, Julien Grall wrote:
> Hi Edgar,
> 
> On 06/05/15 06:28, Edgar E. Iglesias wrote:
> > diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
> > index 59dd23a..98f1019 100644
> > --- a/xen/arch/arm/p2m.c
> > +++ b/xen/arch/arm/p2m.c
> > @@ -17,9 +17,11 @@
> 
> [..]
> 
> > +const unsigned int __read_mostly p2m_ipa_bits = 40;
> 
> [..]
> 
> > diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
> > index 341df55..63748ef 100644
> > --- a/xen/include/asm-arm/p2m.h
> > +++ b/xen/include/asm-arm/p2m.h
> > @@ -10,6 +10,9 @@
> >  
> >  #define paddr_bits PADDR_BITS
> >  
> > +/* Holds the bit size of IPAs in p2m tables.  */
> > +extern unsigned int p2m_ipa_bits;
> > +
> 
> The two declarations of p2m_ipa_bits are different on ARM32. GCC will
> fail to build with:
> 
> p2m.c:24:34: error: conflicting type qualifiers for ‘p2m_ipa_bits’
>  const unsigned int __read_mostly p2m_ipa_bits = 40;
>                                   ^
> In file included from works/xen/xen/include/asm/domain.h:8:0,
>                  from works/xen/xen/include/xen/domain.h:6,
>                  from works/xen/xen/include/xen/sched.h:10,
>                  from p2m.c:2:
> works/xen/xen/include/asm/p2m.h:14:21: note: previous declaration of
> ‘p2m_ipa_bits’ was here
>  extern unsigned int p2m_ipa_bits;
>                      ^
> I would either drop the const in p2m.c or use a define for ARM32 in
> asm-arm/p2m.h


Thanks for catching that Julien,

I've setup an environment to build arm32 aswell now.

I went back and removed the const and back to a single declaration of
p2m_ipa_bits __read_mostly.

Sending v6 in a second.

Cheers,
Edgar

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v5 3/3] xen/iommu: arm: Use p2m_ipa_bits as stage2 input size

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.iglesias@xilinx.com>

The Stage2 input-size must match what the CPU uses because
the SMMU and the CPU share page-tables.

Test that the SMMU supports the P2M IPA bit size, use it if
supported or bail out if not.

Reviewed-by: Julien Grall <julien.grall@citrix.com>
Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xilinx.com>
---
 xen/drivers/passthrough/arm/smmu.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index 8a9b58b..d9f3931 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -2230,8 +2230,14 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 	size = arm_smmu_id_size_to_bits((id >> ID2_IAS_SHIFT) & ID2_IAS_MASK);
 	smmu->s1_output_size = min_t(unsigned long, PHYS_MASK_SHIFT, size);
 
-	/* Xen: Stage-2 input size is not restricted */
-	smmu->s2_input_size = size;
+	/* Xen: Stage-2 input size has to match p2m_ipa_bits.  */
+	if (size < p2m_ipa_bits) {
+		dev_err(smmu->dev,
+			"P2M IPA size not supported (P2M=%u SMMU=%lu)!\n",
+			p2m_ipa_bits, size);
+		return -ENODEV;
+	}
+	smmu->s2_input_size = p2m_ipa_bits;
 #if 0
 	/* Stage-2 input size limited due to pgd allocation (PTRS_PER_PGD) */
 #ifdef CONFIG_64BIT
-- 
1.9.1