[XEN v5 00/10] Add support for 32-bit physical address

[XEN v5 00/10] Add support for 32-bit physical address

[Ayan Kumar Halder]

Hi All,

Please have a look at https://lists.xenproject.org/archives/html/xen-devel/2022-11/msg01465.html
for the context.

The benefits of using 32 bit physical addresses are as follows :-

1. It helps to use Xen on platforms (for eg R52) which supports 32-bit
physical addresses and has no support for large physical address extension.
On 32-bit MPU systems which supports flat-mapping (for eg R52), it helps
to translate 32 bit VA into 32 bit PA.

2. It also helps in code optimization when the underlying platform does not
use large physical address extension.

The following points are to be noted :-
1. Device tree always use uint64_t for address and size. The caller needs to
translate between uint64_t and uint32_t (when 32 bit physical addressing is used).
2. Currently, we have enabled this option for Arm_32 as the MMU for Arm_64
uses 48-bit physical addressing.
3. https://lists.xenproject.org/archives/html/xen-devel/2022-12/msg00117.html
has been added to this series.

Changes from :

v1 - 1. Reordered the patches such that the first three patches fixes issues in
the existing codebase. These can be applied independent of the remaining patches
in this serie.

2. Dropped translate_dt_address_size() for the address/size translation between
paddr_t and u64 (as parsed from the device tree). Also, dropped the check for
truncation (while converting u64 to paddr_t).
Instead now we have modified device_tree_get_reg() and typecasted the return for
dt_read_number(), to obtain paddr_t. Also, introduced wrappers for
fdt_get_mem_rsv() and dt_device_get_address() for the same purpose. These can be
found in patch 4/11 and patch 6/11.

3. Split "Other adaptations required to support 32bit paddr" into the following
individual patches for each adaptation :
  xen/arm: smmu: Use writeq_relaxed_non_atomic() for writing to
    SMMU_CBn_TTBR0
  xen/arm: guest_walk: LPAE specific bits should be enclosed within
    "ifndef CONFIG_ARM_PA_32"

4. Introduced "xen/arm: p2m: Enable support for 32bit IPA".

v2 - 1. Dropped patches 1/11, 2/11 and 3/11 from the v2 as it has already been
committed (except 2/11 - "[XEN v5] xen/arm: Use the correct format specifier"
which is waiting to be committed).

2. Introduced a new patch "xen/drivers: ns16550: Use paddr_t for io_base/io_size".

v3 - 1. Combined the patches from https://lists.xenproject.org/archives/html/xen-devel/2023-02/msg00656.html in this series.

v4 - 1. Dropped "xen/drivers: ns16550: Use paddr_t for io_base/io_size" from the patch series.

As Jan (jbeulich@suse.com) had pointed out in https://lore.kernel.org/xen-devel/20230321140357.24094-5-ayan.kumar.halder@amd.com/,
ns16550 driver requires some prior cleanup. Also, ns16550 can be ignored for now
for the 32-bit paddr support (which is mainly targeted for Arm). I will send out
separate patches to fix this once the current serie is committed (or in ready to
commit state). I hope that is fine with Jan ?

2. Introduced "xen/arm: domain_build: Check if the address fits the range of physical address".

3. "xen/arm: Use the correct format specifier" has been committed in v4.

Ayan Kumar Halder (10):
  xen/arm: domain_build: Track unallocated pages using the frame number
  xen/arm: Typecast the DT values into paddr_t
  xen/arm: Introduce a wrapper for dt_device_get_address() to handle
    paddr_t
  xen/arm: smmu: Use writeq_relaxed_non_atomic() for writing to
    SMMU_CBn_TTBR0
  xen/arm: Introduce choice to enable 64/32 bit physical addressing
  xen/arm: guest_walk: LPAE specific bits should be enclosed within
    "ifndef CONFIG_PHYS_ADDR_T_32"
  xen/arm: Restrict zeroeth_table_offset for ARM_64
  xen/arm: domain_build: Check if the address fits the range of physical
    address
  xen/arm: p2m: Use the pa_range_info table to support ARM_32 and ARM_64
  xen/arm: p2m: Enable support for 32bit IPA for ARM_32

 xen/arch/Kconfig                           |  3 +
 xen/arch/arm/Kconfig                       | 37 +++++++++++-
 xen/arch/arm/bootfdt.c                     | 48 ++++++++++++----
 xen/arch/arm/domain_build.c                | 67 +++++++++++++++-------
 xen/arch/arm/gic-v2.c                      | 10 ++--
 xen/arch/arm/gic-v3-its.c                  |  4 +-
 xen/arch/arm/gic-v3.c                      | 10 ++--
 xen/arch/arm/guest_walk.c                  |  2 +
 xen/arch/arm/include/asm/lpae.h            |  4 ++
 xen/arch/arm/include/asm/p2m.h             |  8 +--
 xen/arch/arm/include/asm/page-bits.h       |  6 +-
 xen/arch/arm/include/asm/setup.h           |  4 +-
 xen/arch/arm/include/asm/types.h           |  6 ++
 xen/arch/arm/mm.c                          | 12 ++--
 xen/arch/arm/p2m.c                         | 35 +++++------
 xen/arch/arm/pci/pci-host-common.c         |  6 +-
 xen/arch/arm/platforms/brcm-raspberry-pi.c |  2 +-
 xen/arch/arm/platforms/brcm.c              |  6 +-
 xen/arch/arm/platforms/exynos5.c           | 32 +++++------
 xen/arch/arm/platforms/sunxi.c             |  2 +-
 xen/arch/arm/platforms/xgene-storm.c       |  2 +-
 xen/arch/arm/setup.c                       | 18 +++---
 xen/arch/arm/smpboot.c                     |  2 +-
 xen/common/device_tree.c                   | 35 +++++++++++
 xen/drivers/char/cadence-uart.c            |  4 +-
 xen/drivers/char/exynos4210-uart.c         |  4 +-
 xen/drivers/char/imx-lpuart.c              |  4 +-
 xen/drivers/char/meson-uart.c              |  4 +-
 xen/drivers/char/mvebu-uart.c              |  4 +-
 xen/drivers/char/omap-uart.c               |  4 +-
 xen/drivers/char/pl011.c                   |  6 +-
 xen/drivers/char/scif-uart.c               |  4 +-
 xen/drivers/passthrough/arm/ipmmu-vmsa.c   |  8 +--
 xen/drivers/passthrough/arm/smmu-v3.c      |  2 +-
 xen/drivers/passthrough/arm/smmu.c         | 23 ++++----
 xen/include/xen/device_tree.h              | 36 ++++++++++++
 xen/include/xen/libfdt/libfdt-xen.h        | 55 ++++++++++++++++++
 37 files changed, 369 insertions(+), 150 deletions(-)
 create mode 100644 xen/include/xen/libfdt/libfdt-xen.h

-- 
2.17.1

[XEN v5 01/10] xen/arm: domain_build: Track unallocated pages using the frame number

[Ayan Kumar Halder]

rangeset_{xxx}_range() functions are invoked with 'start' and 'size' as
arguments which are either 'uint64_t' or 'paddr_t'. However, the function
accepts 'unsigned long' for 'start' and 'size'. 'unsigned long' is 32 bits for
Arm32. Thus, there is an implicit downcasting from 'uint64_t'/'paddr_t' to
'unsigned long' when invoking rangeset_{xxx}_range().

So, it may seem there is a possibility of lose of data due to truncation.

In reality, 'start' and 'size' are always page aligned. And Arm32 currently
supports 40 bits as the width of physical address.
So if the addresses are page aligned, the last 12 bits contain zeroes.
Thus, we could instead pass page frame number which will contain 28 bits (40-12
on Arm32) and this can be represented using 'unsigned long'.

On Arm64, this change will not induce any adverse side effect as the width of
physical address is 48 bits. Thus, the width of 'gfn' (ie 48 - 12 = 36) can be
represented using 'unsigned long' (which is 64 bits wide).

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
---
Changes from -

v3 - 1. Extracted the patch from https://lists.xenproject.org/archives/html/xen-devel/2023-02/msg00657.html
and added it to this series.
2. Modified add_ext_regions(). This accepts a frame number instead of physical
address.

v4 - 1. Reworded the commit message to use Arm32/Arm64
(32-bit/64-bit Arm architecture).
2. Replaced pfn with gfn to denote guest frame number in add_ext_regions().
3. Use pfn_to_paddr() to return a physical address from the guest frame number.

 xen/arch/arm/domain_build.c | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 4f9d4f9d88..c8f08d8ee2 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1500,10 +1500,13 @@ static int __init make_resv_memory_node(const struct domain *d,
     return res;
 }
 
-static int __init add_ext_regions(unsigned long s, unsigned long e, void *data)
+static int __init add_ext_regions(unsigned long s_gfn, unsigned long e_gfn,
+                                  void *data)
 {
     struct meminfo *ext_regions = data;
     paddr_t start, size;
+    paddr_t s = pfn_to_paddr(s_gfn);
+    paddr_t e = pfn_to_paddr(e_gfn);
 
     if ( ext_regions->nr_banks >= ARRAY_SIZE(ext_regions->bank) )
         return 0;
@@ -1566,7 +1569,8 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo,
     {
         start = bootinfo.mem.bank[i].start;
         end = bootinfo.mem.bank[i].start + bootinfo.mem.bank[i].size;
-        res = rangeset_add_range(unalloc_mem, start, end - 1);
+        res = rangeset_add_range(unalloc_mem, PFN_DOWN(start),
+                                 PFN_DOWN(end - 1));
         if ( res )
         {
             printk(XENLOG_ERR "Failed to add: %#"PRIpaddr"->%#"PRIpaddr"\n",
@@ -1580,7 +1584,8 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo,
     {
         start = assign_mem->bank[i].start;
         end = assign_mem->bank[i].start + assign_mem->bank[i].size;
-        res = rangeset_remove_range(unalloc_mem, start, end - 1);
+        res = rangeset_remove_range(unalloc_mem, PFN_DOWN(start),
+                                    PFN_DOWN(end - 1));
         if ( res )
         {
             printk(XENLOG_ERR "Failed to remove: %#"PRIpaddr"->%#"PRIpaddr"\n",
@@ -1595,7 +1600,8 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo,
         start = bootinfo.reserved_mem.bank[i].start;
         end = bootinfo.reserved_mem.bank[i].start +
             bootinfo.reserved_mem.bank[i].size;
-        res = rangeset_remove_range(unalloc_mem, start, end - 1);
+        res = rangeset_remove_range(unalloc_mem, PFN_DOWN(start),
+                                    PFN_DOWN(end - 1));
         if ( res )
         {
             printk(XENLOG_ERR "Failed to remove: %#"PRIpaddr"->%#"PRIpaddr"\n",
@@ -1607,7 +1613,7 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo,
     /* Remove grant table region */
     start = kinfo->gnttab_start;
     end = kinfo->gnttab_start + kinfo->gnttab_size;
-    res = rangeset_remove_range(unalloc_mem, start, end - 1);
+    res = rangeset_remove_range(unalloc_mem, PFN_DOWN(start), PFN_DOWN(end - 1));
     if ( res )
     {
         printk(XENLOG_ERR "Failed to remove: %#"PRIpaddr"->%#"PRIpaddr"\n",
@@ -1617,7 +1623,7 @@ static int __init find_unallocated_memory(const struct kernel_info *kinfo,
 
     start = 0;
     end = (1ULL << p2m_ipa_bits) - 1;
-    res = rangeset_report_ranges(unalloc_mem, start, end,
+    res = rangeset_report_ranges(unalloc_mem, PFN_DOWN(start), PFN_DOWN(end),
                                  add_ext_regions, ext_regions);
     if ( res )
         ext_regions->nr_banks = 0;
@@ -1639,7 +1645,7 @@ static int __init handle_pci_range(const struct dt_device_node *dev,
 
     start = addr & PAGE_MASK;
     end = PAGE_ALIGN(addr + len);
-    res = rangeset_remove_range(mem_holes, start, end - 1);
+    res = rangeset_remove_range(mem_holes, PFN_DOWN(start), PFN_DOWN(end - 1));
     if ( res )
     {
         printk(XENLOG_ERR "Failed to remove: %#"PRIpaddr"->%#"PRIpaddr"\n",
@@ -1677,7 +1683,7 @@ static int __init find_memory_holes(const struct kernel_info *kinfo,
     /* Start with maximum possible addressable physical memory range */
     start = 0;
     end = (1ULL << p2m_ipa_bits) - 1;
-    res = rangeset_add_range(mem_holes, start, end);
+    res = rangeset_add_range(mem_holes, PFN_DOWN(start), PFN_DOWN(end));
     if ( res )
     {
         printk(XENLOG_ERR "Failed to add: %#"PRIpaddr"->%#"PRIpaddr"\n",
@@ -1708,7 +1714,8 @@ static int __init find_memory_holes(const struct kernel_info *kinfo,
 
             start = addr & PAGE_MASK;
             end = PAGE_ALIGN(addr + size);
-            res = rangeset_remove_range(mem_holes, start, end - 1);
+            res = rangeset_remove_range(mem_holes, PFN_DOWN(start),
+                                        PFN_DOWN(end - 1));
             if ( res )
             {
                 printk(XENLOG_ERR "Failed to remove: %#"PRIpaddr"->%#"PRIpaddr"\n",
@@ -1735,7 +1742,7 @@ static int __init find_memory_holes(const struct kernel_info *kinfo,
 
     start = 0;
     end = (1ULL << p2m_ipa_bits) - 1;
-    res = rangeset_report_ranges(mem_holes, start, end,
+    res = rangeset_report_ranges(mem_holes, PFN_DOWN(start), PFN_DOWN(end),
                                  add_ext_regions,  ext_regions);
     if ( res )
         ext_regions->nr_banks = 0;
-- 
2.17.1

[XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Ayan Kumar Halder]

The DT functions (dt_read_number(), device_tree_get_reg(), fdt_get_mem_rsv())
currently accept or return 64-bit values.

In future when we support 32-bit physical address, these DT functions are
expected to accept/return 32-bit or 64-bit values (depending on the width of
physical address). Also, we wish to detect if any truncation has occurred
(i.e. while parsing 32-bit physical addresses from 64-bit values read from DT).

device_tree_get_reg() should now be able to return paddr_t. This is invoked by
various callers to get DT address and size.

For fdt_get_mem_rsv(), we have introduced a wrapper named
fdt_get_mem_rsv_paddr() which will invoke fdt_get_mem_rsv() and translate
uint64_t to paddr_t. The reason being we cannot modify fdt_get_mem_rsv() as it
has been imported from external source.

For dt_read_number(), we have also introduced a wrapper named dt_read_paddr()
dt_read_paddr() to read physical addresses. We chose not to modify the original
function as it is used in places where it needs to specifically read 64-bit
values from dt (For e.g. dt_property_read_u64()).

Xen prints warning when it detects truncation in cases where it is not able to
return error.

Also, replaced u32/u64 with uint32_t/uint64_t in the functions touched
by the code changes.

Also, initialized variables to fix the warning "-Werror=maybe-uninitialized".

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
---

Changes from

v1 - 1. Dropped "[XEN v1 2/9] xen/arm: Define translate_dt_address_size() for the translation between u64 and paddr_t" and
"[XEN v1 4/9] xen/arm: Use translate_dt_address_size() to translate between device tree addr/size and paddr_t", instead
this approach achieves the same purpose.

2. No need to check for truncation while converting values from u64 to paddr_t.

v2 - 1. Use "( (dt_start >> (PADDR_SHIFT - 1)) > 1 )" to detect truncation.
2. Introduced libfdt_xen.h to implement fdt_get_mem_rsv_paddr
3. Logged error messages in case truncation is detected.

v3 - 1. Renamed libfdt_xen.h to libfdt-xen.h.
2. Replaced u32/u64 with uint32_t/uint64_t
3. Use "(paddr_t)val != val" to check for truncation.
4. Removed the alias "#define PADDR_SHIFT PADDR_BITS". 

v4 - 1. Added a WARN() when truncation is detected.
2. Always check the return value of fdt_get_mem_rsv().

 xen/arch/arm/bootfdt.c              | 48 +++++++++++++++++++------
 xen/arch/arm/domain_build.c         |  2 +-
 xen/arch/arm/include/asm/setup.h    |  4 +--
 xen/arch/arm/setup.c                | 18 +++++-----
 xen/arch/arm/smpboot.c              |  2 +-
 xen/include/xen/device_tree.h       | 23 ++++++++++++
 xen/include/xen/libfdt/libfdt-xen.h | 55 +++++++++++++++++++++++++++++
 7 files changed, 129 insertions(+), 23 deletions(-)
 create mode 100644 xen/include/xen/libfdt/libfdt-xen.h

diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
index 0085c28d74..ac8148da55 100644
--- a/xen/arch/arm/bootfdt.c
+++ b/xen/arch/arm/bootfdt.c
@@ -11,7 +11,7 @@
 #include <xen/efi.h>
 #include <xen/device_tree.h>
 #include <xen/lib.h>
-#include <xen/libfdt/libfdt.h>
+#include <xen/libfdt/libfdt-xen.h>
 #include <xen/sort.h>
 #include <xsm/xsm.h>
 #include <asm/setup.h>
@@ -52,11 +52,37 @@ static bool __init device_tree_node_compatible(const void *fdt, int node,
     return false;
 }
 
-void __init device_tree_get_reg(const __be32 **cell, u32 address_cells,
-                                u32 size_cells, u64 *start, u64 *size)
+void __init device_tree_get_reg(const __be32 **cell, uint32_t address_cells,
+                                uint32_t size_cells, paddr_t *start,
+                                paddr_t *size)
 {
-    *start = dt_next_cell(address_cells, cell);
-    *size = dt_next_cell(size_cells, cell);
+    uint64_t dt_start, dt_size;
+
+    /*
+     * dt_next_cell will return uint64_t whereas paddr_t may not be 64-bit.
+     * Thus, there is an implicit cast from uint64_t to paddr_t.
+     */
+    dt_start = dt_next_cell(address_cells, cell);
+    dt_size = dt_next_cell(size_cells, cell);
+
+    if ( dt_start != (paddr_t)dt_start )
+    {
+        printk("Error: Physical address greater than max width supported\n");
+        WARN();
+    }
+
+    if ( dt_size != (paddr_t)dt_size )
+    {
+        printk("Error: Physical size greater than max width supported\n");
+        WARN();
+    }
+
+    /*
+     * Xen will truncate the address/size if it is greater than the maximum
+     * supported width and it will give an appropriate warning.
+     */
+    *start = dt_start;
+    *size = dt_size;
 }
 
 static int __init device_tree_get_meminfo(const void *fdt, int node,
@@ -326,7 +352,7 @@ static int __init process_chosen_node(const void *fdt, int node,
         printk("linux,initrd-start property has invalid length %d\n", len);
         return -EINVAL;
     }
-    start = dt_read_number((void *)&prop->data, dt_size_to_cells(len));
+    start = dt_read_paddr((void *)&prop->data, dt_size_to_cells(len));
 
     prop = fdt_get_property(fdt, node, "linux,initrd-end", &len);
     if ( !prop )
@@ -339,7 +365,7 @@ static int __init process_chosen_node(const void *fdt, int node,
         printk("linux,initrd-end property has invalid length %d\n", len);
         return -EINVAL;
     }
-    end = dt_read_number((void *)&prop->data, dt_size_to_cells(len));
+    end = dt_read_paddr((void *)&prop->data, dt_size_to_cells(len));
 
     if ( start >= end )
     {
@@ -593,10 +619,12 @@ static void __init early_print_info(void)
     nr_rsvd = fdt_num_mem_rsv(device_tree_flattened);
     for ( i = 0; i < nr_rsvd; i++ )
     {
-        paddr_t s, e;
-        if ( fdt_get_mem_rsv(device_tree_flattened, i, &s, &e) < 0 )
+        paddr_t s = 0, e = 0;
+
+        if ( fdt_get_mem_rsv_paddr(device_tree_flattened, i, &s, &e) < 0 )
             continue;
-        /* fdt_get_mem_rsv returns length */
+
+        /* fdt_get_mem_rsv_paddr returns length */
         e += s;
         printk(" RESVD[%u]: %"PRIpaddr" - %"PRIpaddr"\n", i, s, e);
     }
diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index c8f08d8ee2..15c8bdd9e4 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -949,7 +949,7 @@ static int __init process_shm(struct domain *d, struct kernel_info *kinfo,
         BUG_ON(!prop);
         cells = (const __be32 *)prop->value;
         device_tree_get_reg(&cells, addr_cells, addr_cells, &pbase, &gbase);
-        psize = dt_read_number(cells, size_cells);
+        psize = dt_read_paddr(cells, size_cells);
         if ( !IS_ALIGNED(pbase, PAGE_SIZE) || !IS_ALIGNED(gbase, PAGE_SIZE) )
         {
             printk("%pd: physical address 0x%"PRIpaddr", or guest address 0x%"PRIpaddr" is not suitably aligned.\n",
diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h
index a926f30a2b..7b697d879e 100644
--- a/xen/arch/arm/include/asm/setup.h
+++ b/xen/arch/arm/include/asm/setup.h
@@ -157,8 +157,8 @@ const char *boot_module_kind_as_string(bootmodule_kind kind);
 extern uint32_t hyp_traps_vector[];
 void init_traps(void);
 
-void device_tree_get_reg(const __be32 **cell, u32 address_cells,
-                         u32 size_cells, u64 *start, u64 *size);
+void device_tree_get_reg(const __be32 **cell, uint32_t address_cells,
+                         uint32_t size_cells, paddr_t *start, paddr_t *size);
 
 u32 device_tree_get_u32(const void *fdt, int node,
                         const char *prop_name, u32 dflt);
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 1f26f67b90..d2a3d8c340 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -29,7 +29,7 @@
 #include <xen/virtual_region.h>
 #include <xen/vmap.h>
 #include <xen/trace.h>
-#include <xen/libfdt/libfdt.h>
+#include <xen/libfdt/libfdt-xen.h>
 #include <xen/acpi.h>
 #include <xen/warning.h>
 #include <asm/alternative.h>
@@ -220,13 +220,13 @@ static void __init dt_unreserved_regions(paddr_t s, paddr_t e,
 
     for ( i = first; i < nr ; i++ )
     {
-        paddr_t r_s, r_e;
+        paddr_t r_s = 0, r_e = 0;
 
-        if ( fdt_get_mem_rsv(device_tree_flattened, i, &r_s, &r_e ) < 0 )
+        if ( fdt_get_mem_rsv_paddr(device_tree_flattened, i, &r_s, &r_e ) < 0 )
             /* If we can't read it, pretend it doesn't exist... */
             continue;
 
-        r_e += r_s; /* fdt_get_mem_rsv returns length */
+        r_e += r_s; /* fdt_get_mem_rsv_paddr returns length */
 
         if ( s < r_e && r_s < e )
         {
@@ -500,15 +500,15 @@ static paddr_t __init consider_modules(paddr_t s, paddr_t e,
 
     for ( ; i < mi->nr_mods + nr; i++ )
     {
-        paddr_t mod_s, mod_e;
+        paddr_t mod_s = 0, mod_e = 0;
 
-        if ( fdt_get_mem_rsv(device_tree_flattened,
-                             i - mi->nr_mods,
-                             &mod_s, &mod_e ) < 0 )
+        if ( fdt_get_mem_rsv_paddr(device_tree_flattened,
+                                   i - mi->nr_mods,
+                                   &mod_s, &mod_e ) < 0 )
             /* If we can't read it, pretend it doesn't exist... */
             continue;
 
-        /* fdt_get_mem_rsv returns length */
+        /* fdt_get_mem_rsv_paddr returns length */
         mod_e += mod_s;
 
         if ( s < mod_e && mod_s < e )
diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
index 412ae22869..c15c177487 100644
--- a/xen/arch/arm/smpboot.c
+++ b/xen/arch/arm/smpboot.c
@@ -159,7 +159,7 @@ static void __init dt_smp_init_cpus(void)
             continue;
         }
 
-        addr = dt_read_number(prop, dt_n_addr_cells(cpu));
+        addr = dt_read_paddr(prop, dt_n_addr_cells(cpu));
 
         hwid = addr;
         if ( hwid != addr )
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 19a74909ce..11bda2fd3d 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -241,6 +241,29 @@ static inline u64 dt_read_number(const __be32 *cell, int size)
     return r;
 }
 
+/* Wrapper for dt_read_number() to return paddr_t (instead of uint64_t) */
+static inline paddr_t dt_read_paddr(const __be32 *cell, int size)
+{
+    uint64_t dt_r = 0;
+    paddr_t r;
+
+    dt_r = dt_read_number(cell, size);
+
+    if ( dt_r != (paddr_t)dt_r )
+    {
+        printk("Error: Physical address greater than max width supported\n");
+        WARN();
+    }
+
+    /*
+     * Xen will truncate the address/size if it is greater than the maximum
+     * supported width and it will give an appropriate warning.
+     */
+    r = dt_r;
+
+    return r;
+}
+
 /* Helper to convert a number of cells to bytes */
 static inline int dt_cells_to_size(int size)
 {
diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
new file mode 100644
index 0000000000..3296a368a6
--- /dev/null
+++ b/xen/include/xen/libfdt/libfdt-xen.h
@@ -0,0 +1,55 @@
+/*
+ * SPDX-License-Identifier: GPL-2.0-only
+ *
+ * xen/include/xen/libfdt/libfdt-xen.h
+ *
+ * Wrapper functions for device tree. This helps to convert dt values
+ * between uint64_t and paddr_t.
+ *
+ * Copyright (C) 2023, Advanced Micro Devices, Inc. All Rights Reserved.
+ */
+
+#ifndef LIBFDT_XEN_H
+#define LIBFDT_XEN_H
+
+#include <xen/libfdt/libfdt.h>
+
+static inline int fdt_get_mem_rsv_paddr(const void *fdt, int n,
+                                        paddr_t *address,
+                                        paddr_t *size)
+{
+    uint64_t dt_addr;
+    uint64_t dt_size;
+    int ret = 0;
+
+    ret = fdt_get_mem_rsv(fdt, n, &dt_addr, &dt_size);
+    if ( ret )
+        return ret;
+
+    if ( dt_addr != (paddr_t)dt_addr )
+    {
+        printk("Error: Physical address greater than max width supported\n");
+        return -FDT_ERR_MAX;
+    }
+
+    if ( dt_size != (paddr_t)dt_size )
+    {
+        printk("Error: Physical size greater than max width supported\n");
+        return -FDT_ERR_MAX;
+    }
+
+    *address = dt_addr;
+    *size = dt_size;
+
+    return ret;
+}
+
+#endif /* LIBFDT_XEN_H */
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
-- 
2.17.1

[XEN v5 03/10] xen/arm: Introduce a wrapper for dt_device_get_address() to handle paddr_t

[Ayan Kumar Halder]

dt_device_get_address() can accept uint64_t only for address and size.
However, the address/size denotes physical addresses. Thus, they should
be represented by 'paddr_t'.
Consequently, we introduce a wrapper for dt_device_get_address() ie
dt_device_get_paddr() which accepts address/size as paddr_t and inturn
invokes dt_device_get_address() after converting address/size to
uint64_t.

The reason for introducing this is that in future 'paddr_t' may not
always be 64-bit. Thus, we need an explicit wrapper to do the type
conversion and return an error in case of truncation.

With this, callers can now invoke dt_device_get_paddr().

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
---
Changes from -

v1 - 1. New patch.

v2 - 1. Extracted part of "[XEN v2 05/11] xen/arm: Use paddr_t instead of u64 for address/size"
into this patch.

2. dt_device_get_address() callers now invoke dt_device_get_paddr() instead.

3. Logged error in case of truncation.

v3 - 1. Modified the truncation checks as "dt_addr != (paddr_t)dt_addr".
2. Some sanity fixes.

v4 - 1. Some sanity fixes.
2. Preserved the declaration of dt_device_get_address() in
xen/include/xen/device_tree.h. The reason being it is currently used by
ns16550.c. This driver requires some more changes as pointed by Jan in
https://lore.kernel.org/xen-devel/6196e90f-752e-e61a-45ce-37e46c22b812@suse.com/
which is to be addressed as a separate series.

 xen/arch/arm/domain_build.c                | 10 +++----
 xen/arch/arm/gic-v2.c                      | 10 +++----
 xen/arch/arm/gic-v3-its.c                  |  4 +--
 xen/arch/arm/gic-v3.c                      | 10 +++----
 xen/arch/arm/pci/pci-host-common.c         |  6 ++--
 xen/arch/arm/platforms/brcm-raspberry-pi.c |  2 +-
 xen/arch/arm/platforms/brcm.c              |  6 ++--
 xen/arch/arm/platforms/exynos5.c           | 32 ++++++++++----------
 xen/arch/arm/platforms/sunxi.c             |  2 +-
 xen/arch/arm/platforms/xgene-storm.c       |  2 +-
 xen/common/device_tree.c                   | 35 ++++++++++++++++++++++
 xen/drivers/char/cadence-uart.c            |  4 +--
 xen/drivers/char/exynos4210-uart.c         |  4 +--
 xen/drivers/char/imx-lpuart.c              |  4 +--
 xen/drivers/char/meson-uart.c              |  4 +--
 xen/drivers/char/mvebu-uart.c              |  4 +--
 xen/drivers/char/omap-uart.c               |  4 +--
 xen/drivers/char/pl011.c                   |  6 ++--
 xen/drivers/char/scif-uart.c               |  4 +--
 xen/drivers/passthrough/arm/ipmmu-vmsa.c   |  8 ++---
 xen/drivers/passthrough/arm/smmu-v3.c      |  2 +-
 xen/drivers/passthrough/arm/smmu.c         |  8 ++---
 xen/include/xen/device_tree.h              | 13 ++++++++
 23 files changed, 116 insertions(+), 68 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 15c8bdd9e4..7d28b75517 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1698,13 +1698,13 @@ static int __init find_memory_holes(const struct kernel_info *kinfo,
     dt_for_each_device_node( dt_host, np )
     {
         unsigned int naddr;
-        u64 addr, size;
+        paddr_t addr, size;
 
         naddr = dt_number_of_address(np);
 
         for ( i = 0; i < naddr; i++ )
         {
-            res = dt_device_get_address(np, i, &addr, &size);
+            res = dt_device_get_paddr(np, i, &addr, &size);
             if ( res )
             {
                 printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
@@ -2478,7 +2478,7 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
     unsigned int naddr;
     unsigned int i;
     int res;
-    u64 addr, size;
+    paddr_t addr, size;
     bool own_device = !dt_device_for_passthrough(dev);
     /*
      * We want to avoid mapping the MMIO in dom0 for the following cases:
@@ -2533,7 +2533,7 @@ static int __init handle_device(struct domain *d, struct dt_device_node *dev,
     /* Give permission and map MMIOs */
     for ( i = 0; i < naddr; i++ )
     {
-        res = dt_device_get_address(dev, i, &addr, &size);
+        res = dt_device_get_paddr(dev, i, &addr, &size);
         if ( res )
         {
             printk(XENLOG_ERR "Unable to retrieve address %u for %s\n",
@@ -2964,7 +2964,7 @@ static int __init handle_passthrough_prop(struct kernel_info *kinfo,
         if ( res )
         {
             printk(XENLOG_ERR "Unable to permit to dom%d access to"
-                   " 0x%"PRIx64" - 0x%"PRIx64"\n",
+                   " 0x%"PRIpaddr" - 0x%"PRIpaddr"\n",
                    kinfo->d->domain_id,
                    mstart & PAGE_MASK, PAGE_ALIGN(mstart + size) - 1);
             return res;
diff --git a/xen/arch/arm/gic-v2.c b/xen/arch/arm/gic-v2.c
index 5d4d298b86..6476ff4230 100644
--- a/xen/arch/arm/gic-v2.c
+++ b/xen/arch/arm/gic-v2.c
@@ -993,7 +993,7 @@ static void gicv2_extension_dt_init(const struct dt_device_node *node)
             continue;
 
         /* Get register frame resource from DT. */
-        if ( dt_device_get_address(v2m, 0, &addr, &size) )
+        if ( dt_device_get_paddr(v2m, 0, &addr, &size) )
             panic("GICv2: Cannot find a valid v2m frame address\n");
 
         /*
@@ -1018,19 +1018,19 @@ static void __init gicv2_dt_init(void)
     paddr_t vsize;
     const struct dt_device_node *node = gicv2_info.node;
 
-    res = dt_device_get_address(node, 0, &dbase, NULL);
+    res = dt_device_get_paddr(node, 0, &dbase, NULL);
     if ( res )
         panic("GICv2: Cannot find a valid address for the distributor\n");
 
-    res = dt_device_get_address(node, 1, &cbase, &csize);
+    res = dt_device_get_paddr(node, 1, &cbase, &csize);
     if ( res )
         panic("GICv2: Cannot find a valid address for the CPU\n");
 
-    res = dt_device_get_address(node, 2, &hbase, NULL);
+    res = dt_device_get_paddr(node, 2, &hbase, NULL);
     if ( res )
         panic("GICv2: Cannot find a valid address for the hypervisor\n");
 
-    res = dt_device_get_address(node, 3, &vbase, &vsize);
+    res = dt_device_get_paddr(node, 3, &vbase, &vsize);
     if ( res )
         panic("GICv2: Cannot find a valid address for the virtual CPU\n");
 
diff --git a/xen/arch/arm/gic-v3-its.c b/xen/arch/arm/gic-v3-its.c
index 1ec9934191..3aa4edda10 100644
--- a/xen/arch/arm/gic-v3-its.c
+++ b/xen/arch/arm/gic-v3-its.c
@@ -1004,12 +1004,12 @@ static void gicv3_its_dt_init(const struct dt_device_node *node)
      */
     dt_for_each_child_node(node, its)
     {
-        uint64_t addr, size;
+        paddr_t addr, size;
 
         if ( !dt_device_is_compatible(its, "arm,gic-v3-its") )
             continue;
 
-        if ( dt_device_get_address(its, 0, &addr, &size) )
+        if ( dt_device_get_paddr(its, 0, &addr, &size) )
             panic("GICv3: Cannot find a valid ITS frame address\n");
 
         add_to_host_its_list(addr, size, its);
diff --git a/xen/arch/arm/gic-v3.c b/xen/arch/arm/gic-v3.c
index bb59ea94cd..4e6c98bada 100644
--- a/xen/arch/arm/gic-v3.c
+++ b/xen/arch/arm/gic-v3.c
@@ -1377,7 +1377,7 @@ static void __init gicv3_dt_init(void)
     int res, i;
     const struct dt_device_node *node = gicv3_info.node;
 
-    res = dt_device_get_address(node, 0, &dbase, NULL);
+    res = dt_device_get_paddr(node, 0, &dbase, NULL);
     if ( res )
         panic("GICv3: Cannot find a valid distributor address\n");
 
@@ -1393,9 +1393,9 @@ static void __init gicv3_dt_init(void)
 
     for ( i = 0; i < gicv3.rdist_count; i++ )
     {
-        uint64_t rdist_base, rdist_size;
+        paddr_t rdist_base, rdist_size;
 
-        res = dt_device_get_address(node, 1 + i, &rdist_base, &rdist_size);
+        res = dt_device_get_paddr(node, 1 + i, &rdist_base, &rdist_size);
         if ( res )
             panic("GICv3: No rdist base found for region %d\n", i);
 
@@ -1417,10 +1417,10 @@ static void __init gicv3_dt_init(void)
      * For GICv3 supporting GICv2, GICC and GICV base address will be
      * provided.
      */
-    res = dt_device_get_address(node, 1 + gicv3.rdist_count,
+    res = dt_device_get_paddr(node, 1 + gicv3.rdist_count,
                                 &cbase, &csize);
     if ( !res )
-        dt_device_get_address(node, 1 + gicv3.rdist_count + 2,
+        dt_device_get_paddr(node, 1 + gicv3.rdist_count + 2,
                               &vbase, &vsize);
 }
 
diff --git a/xen/arch/arm/pci/pci-host-common.c b/xen/arch/arm/pci/pci-host-common.c
index a8ece94303..5550f9478d 100644
--- a/xen/arch/arm/pci/pci-host-common.c
+++ b/xen/arch/arm/pci/pci-host-common.c
@@ -93,7 +93,7 @@ gen_pci_init(struct dt_device_node *dev, const struct pci_ecam_ops *ops)
         cfg_reg_idx = 0;
 
     /* Parse our PCI ecam register address */
-    err = dt_device_get_address(dev, cfg_reg_idx, &addr, &size);
+    err = dt_device_get_paddr(dev, cfg_reg_idx, &addr, &size);
     if ( err )
         goto err_exit;
 
@@ -349,10 +349,10 @@ int __init pci_host_bridge_mappings(struct domain *d)
 
         for ( i = 0; i < dt_number_of_address(dev); i++ )
         {
-            uint64_t addr, size;
+            paddr_t addr, size;
             int err;
 
-            err = dt_device_get_address(dev, i, &addr, &size);
+            err = dt_device_get_paddr(dev, i, &addr, &size);
             if ( err )
             {
                 printk(XENLOG_ERR
diff --git a/xen/arch/arm/platforms/brcm-raspberry-pi.c b/xen/arch/arm/platforms/brcm-raspberry-pi.c
index 811b40b1a6..407ec07f63 100644
--- a/xen/arch/arm/platforms/brcm-raspberry-pi.c
+++ b/xen/arch/arm/platforms/brcm-raspberry-pi.c
@@ -64,7 +64,7 @@ static void __iomem *rpi4_map_watchdog(void)
     if ( !node )
         return NULL;
 
-    ret = dt_device_get_address(node, 0, &start, &len);
+    ret = dt_device_get_paddr(node, 0, &start, &len);
     if ( ret )
     {
         printk("Cannot read watchdog register address\n");
diff --git a/xen/arch/arm/platforms/brcm.c b/xen/arch/arm/platforms/brcm.c
index d481b2c60f..951e4d6cc3 100644
--- a/xen/arch/arm/platforms/brcm.c
+++ b/xen/arch/arm/platforms/brcm.c
@@ -40,7 +40,7 @@ static __init int brcm_get_dt_node(char *compat_str,
                                    u32 *reg_base)
 {
     const struct dt_device_node *node;
-    u64 reg_base_64;
+    paddr_t reg_base_paddr;
     int rc;
 
     node = dt_find_compatible_node(NULL, NULL, compat_str);
@@ -50,7 +50,7 @@ static __init int brcm_get_dt_node(char *compat_str,
         return -ENOENT;
     }
 
-    rc = dt_device_get_address(node, 0, &reg_base_64, NULL);
+    rc = dt_device_get_paddr(node, 0, &reg_base_paddr, NULL);
     if ( rc )
     {
         dprintk(XENLOG_ERR, "%s: missing \"reg\" prop\n", __func__);
@@ -61,7 +61,7 @@ static __init int brcm_get_dt_node(char *compat_str,
         *dn = node;
 
     if ( reg_base )
-        *reg_base = reg_base_64;
+        *reg_base = reg_base_paddr;
 
     return 0;
 }
diff --git a/xen/arch/arm/platforms/exynos5.c b/xen/arch/arm/platforms/exynos5.c
index 6560507092..c48093cd4f 100644
--- a/xen/arch/arm/platforms/exynos5.c
+++ b/xen/arch/arm/platforms/exynos5.c
@@ -42,8 +42,8 @@ static int exynos5_init_time(void)
     void __iomem *mct;
     int rc;
     struct dt_device_node *node;
-    u64 mct_base_addr;
-    u64 size;
+    paddr_t mct_base_addr;
+    paddr_t size;
 
     node = dt_find_compatible_node(NULL, NULL, "samsung,exynos4210-mct");
     if ( !node )
@@ -52,14 +52,14 @@ static int exynos5_init_time(void)
         return -ENXIO;
     }
 
-    rc = dt_device_get_address(node, 0, &mct_base_addr, &size);
+    rc = dt_device_get_paddr(node, 0, &mct_base_addr, &size);
     if ( rc )
     {
         dprintk(XENLOG_ERR, "Error in \"samsung,exynos4210-mct\"\n");
         return -ENXIO;
     }
 
-    dprintk(XENLOG_INFO, "mct_base_addr: %016llx size: %016llx\n",
+    dprintk(XENLOG_INFO, "mct_base_addr: 0x%"PRIpaddr" size: 0x%"PRIpaddr"\n",
             mct_base_addr, size);
 
     mct = ioremap_nocache(mct_base_addr, size);
@@ -97,9 +97,9 @@ static int __init exynos5_smp_init(void)
     struct dt_device_node *node;
     void __iomem *sysram;
     char *compatible;
-    u64 sysram_addr;
-    u64 size;
-    u64 sysram_offset;
+    paddr_t sysram_addr;
+    paddr_t size;
+    paddr_t sysram_offset;
     int rc;
 
     node = dt_find_compatible_node(NULL, NULL, "samsung,secure-firmware");
@@ -125,13 +125,13 @@ static int __init exynos5_smp_init(void)
         return -ENXIO;
     }
 
-    rc = dt_device_get_address(node, 0, &sysram_addr, &size);
+    rc = dt_device_get_paddr(node, 0, &sysram_addr, &size);
     if ( rc )
     {
         dprintk(XENLOG_ERR, "Error in %s\n", compatible);
         return -ENXIO;
     }
-    dprintk(XENLOG_INFO, "sysram_addr: %016llx size: %016llx offset: %016llx\n",
+    dprintk(XENLOG_INFO,"sysram_addr: 0x%"PRIpaddr" size: 0x%"PRIpaddr"offset: 0x%"PRIpaddr"\n",
             sysram_addr, size, sysram_offset);
 
     sysram = ioremap_nocache(sysram_addr, size);
@@ -189,7 +189,7 @@ static int exynos5_cpu_power_up(void __iomem *power, int cpu)
     return 0;
 }
 
-static int exynos5_get_pmu_baseandsize(u64 *power_base_addr, u64 *size)
+static int exynos5_get_pmu_baseandsize(paddr_t *power_base_addr, paddr_t *size)
 {
     struct dt_device_node *node;
     int rc;
@@ -208,14 +208,14 @@ static int exynos5_get_pmu_baseandsize(u64 *power_base_addr, u64 *size)
         return -ENXIO;
     }
 
-    rc = dt_device_get_address(node, 0, power_base_addr, size);
+    rc = dt_device_get_paddr(node, 0, power_base_addr, size);
     if ( rc )
     {
         dprintk(XENLOG_ERR, "Error in \"samsung,exynos5XXX-pmu\"\n");
         return -ENXIO;
     }
 
-    dprintk(XENLOG_DEBUG, "power_base_addr: %016llx size: %016llx\n",
+    dprintk(XENLOG_DEBUG, "power_base_addr: 0x%"PRIpaddr" size: 0x%"PRIpaddr"\n",
             *power_base_addr, *size);
 
     return 0;
@@ -223,8 +223,8 @@ static int exynos5_get_pmu_baseandsize(u64 *power_base_addr, u64 *size)
 
 static int exynos5_cpu_up(int cpu)
 {
-    u64 power_base_addr;
-    u64 size;
+    paddr_t power_base_addr;
+    paddr_t size;
     void __iomem *power;
     int rc;
 
@@ -256,8 +256,8 @@ static int exynos5_cpu_up(int cpu)
 
 static void exynos5_reset(void)
 {
-    u64 power_base_addr;
-    u64 size;
+    paddr_t power_base_addr;
+    paddr_t size;
     void __iomem *pmu;
     int rc;
 
diff --git a/xen/arch/arm/platforms/sunxi.c b/xen/arch/arm/platforms/sunxi.c
index e8e4d88bef..2b2c215f20 100644
--- a/xen/arch/arm/platforms/sunxi.c
+++ b/xen/arch/arm/platforms/sunxi.c
@@ -50,7 +50,7 @@ static void __iomem *sunxi_map_watchdog(bool *new_wdt)
         return NULL;
     }
 
-    ret = dt_device_get_address(node, 0, &wdt_start, &wdt_len);
+    ret = dt_device_get_paddr(node, 0, &wdt_start, &wdt_len);
     if ( ret )
     {
         dprintk(XENLOG_ERR, "Cannot read watchdog register address\n");
diff --git a/xen/arch/arm/platforms/xgene-storm.c b/xen/arch/arm/platforms/xgene-storm.c
index befd0c3c2d..6fc2f9679e 100644
--- a/xen/arch/arm/platforms/xgene-storm.c
+++ b/xen/arch/arm/platforms/xgene-storm.c
@@ -50,7 +50,7 @@ static void __init xgene_check_pirq_eoi(void)
     if ( !node )
         panic("%s: Can not find interrupt controller node\n", __func__);
 
-    res = dt_device_get_address(node, 0, &dbase, NULL);
+    res = dt_device_get_paddr(node, 0, &dbase, NULL);
     if ( res )
         panic("%s: Cannot find a valid address for the distributor\n", __func__);
 
diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index 6c9712ab7b..fdef74e7ff 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -955,6 +955,41 @@ int dt_device_get_address(const struct dt_device_node *dev, unsigned int index,
     return 0;
 }
 
+int dt_device_get_paddr(const struct dt_device_node *dev, unsigned int index,
+                        paddr_t *addr, paddr_t *size)
+{
+    uint64_t dt_addr = 0, dt_size = 0;
+    int ret;
+
+    ret = dt_device_get_address(dev, index, &dt_addr, &dt_size);
+    if ( ret )
+        return ret;
+
+    if ( addr )
+    {
+        if ( dt_addr != (paddr_t)dt_addr )
+        {
+            printk("Error: Physical address 0x%"PRIx64" for node=%s is greater than max width (%zu bytes) supported\n",
+                   dt_addr, dev->name, sizeof(paddr_t));
+            return -ERANGE;
+        }
+
+        *addr = dt_addr;
+    }
+
+    if ( size )
+    {
+        if ( dt_size != (paddr_t)dt_size )
+        {
+            printk("Error: Physical size 0x%"PRIx64" for node=%s is greater than max width (%zu bytes) supported\n",
+                   dt_size, dev->name, sizeof(paddr_t));
+            return -ERANGE;
+        }
+        *size = dt_size;
+    }
+
+    return ret;
+}
 
 int dt_for_each_range(const struct dt_device_node *dev,
                       int (*cb)(const struct dt_device_node *,
diff --git a/xen/drivers/char/cadence-uart.c b/xen/drivers/char/cadence-uart.c
index 22905ba66c..c38d7ed143 100644
--- a/xen/drivers/char/cadence-uart.c
+++ b/xen/drivers/char/cadence-uart.c
@@ -158,14 +158,14 @@ static int __init cuart_init(struct dt_device_node *dev, const void *data)
     const char *config = data;
     struct cuart *uart;
     int res;
-    u64 addr, size;
+    paddr_t addr, size;
 
     if ( strcmp(config, "") )
         printk("WARNING: UART configuration is not supported\n");
 
     uart = &cuart_com;
 
-    res = dt_device_get_address(dev, 0, &addr, &size);
+    res = dt_device_get_paddr(dev, 0, &addr, &size);
     if ( res )
     {
         printk("cadence: Unable to retrieve the base"
diff --git a/xen/drivers/char/exynos4210-uart.c b/xen/drivers/char/exynos4210-uart.c
index 43aaf02e18..2503392ccd 100644
--- a/xen/drivers/char/exynos4210-uart.c
+++ b/xen/drivers/char/exynos4210-uart.c
@@ -303,7 +303,7 @@ static int __init exynos4210_uart_init(struct dt_device_node *dev,
     const char *config = data;
     struct exynos4210_uart *uart;
     int res;
-    u64 addr, size;
+    paddr_t addr, size;
 
     if ( strcmp(config, "") )
         printk("WARNING: UART configuration is not supported\n");
@@ -316,7 +316,7 @@ static int __init exynos4210_uart_init(struct dt_device_node *dev,
     uart->parity    = PARITY_NONE;
     uart->stop_bits = 1;
 
-    res = dt_device_get_address(dev, 0, &addr, &size);
+    res = dt_device_get_paddr(dev, 0, &addr, &size);
     if ( res )
     {
         printk("exynos4210: Unable to retrieve the base"
diff --git a/xen/drivers/char/imx-lpuart.c b/xen/drivers/char/imx-lpuart.c
index 9c1f3b71a3..77f70c2719 100644
--- a/xen/drivers/char/imx-lpuart.c
+++ b/xen/drivers/char/imx-lpuart.c
@@ -204,7 +204,7 @@ static int __init imx_lpuart_init(struct dt_device_node *dev,
     const char *config = data;
     struct imx_lpuart *uart;
     int res;
-    u64 addr, size;
+    paddr_t addr, size;
 
     if ( strcmp(config, "") )
         printk("WARNING: UART configuration is not supported\n");
@@ -216,7 +216,7 @@ static int __init imx_lpuart_init(struct dt_device_node *dev,
     uart->parity = 0;
     uart->stop_bits = 1;
 
-    res = dt_device_get_address(dev, 0, &addr, &size);
+    res = dt_device_get_paddr(dev, 0, &addr, &size);
     if ( res )
     {
         printk("imx8-lpuart: Unable to retrieve the base"
diff --git a/xen/drivers/char/meson-uart.c b/xen/drivers/char/meson-uart.c
index b1e25e0468..c627328122 100644
--- a/xen/drivers/char/meson-uart.c
+++ b/xen/drivers/char/meson-uart.c
@@ -209,14 +209,14 @@ static int __init meson_uart_init(struct dt_device_node *dev, const void *data)
     const char *config = data;
     struct meson_uart *uart;
     int res;
-    u64 addr, size;
+    paddr_t addr, size;
 
     if ( strcmp(config, "") )
         printk("WARNING: UART configuration is not supported\n");
 
     uart = &meson_com;
 
-    res = dt_device_get_address(dev, 0, &addr, &size);
+    res = dt_device_get_paddr(dev, 0, &addr, &size);
     if ( res )
     {
         printk("meson: Unable to retrieve the base address of the UART\n");
diff --git a/xen/drivers/char/mvebu-uart.c b/xen/drivers/char/mvebu-uart.c
index a00618b96f..cc55173513 100644
--- a/xen/drivers/char/mvebu-uart.c
+++ b/xen/drivers/char/mvebu-uart.c
@@ -231,14 +231,14 @@ static int __init mvebu_uart_init(struct dt_device_node *dev, const void *data)
     const char *config = data;
     struct mvebu3700_uart *uart;
     int res;
-    u64 addr, size;
+    paddr_t addr, size;
 
     if ( strcmp(config, "") )
         printk("WARNING: UART configuration is not supported\n");
 
     uart = &mvebu3700_com;
 
-    res = dt_device_get_address(dev, 0, &addr, &size);
+    res = dt_device_get_paddr(dev, 0, &addr, &size);
     if ( res )
     {
         printk("mvebu3700: Unable to retrieve the base address of the UART\n");
diff --git a/xen/drivers/char/omap-uart.c b/xen/drivers/char/omap-uart.c
index d6a5d59aa2..8e643cb039 100644
--- a/xen/drivers/char/omap-uart.c
+++ b/xen/drivers/char/omap-uart.c
@@ -324,7 +324,7 @@ static int __init omap_uart_init(struct dt_device_node *dev,
     struct omap_uart *uart;
     u32 clkspec;
     int res;
-    u64 addr, size;
+    paddr_t addr, size;
 
     if ( strcmp(config, "") )
         printk("WARNING: UART configuration is not supported\n");
@@ -344,7 +344,7 @@ static int __init omap_uart_init(struct dt_device_node *dev,
     uart->parity = UART_PARITY_NONE;
     uart->stop_bits = 1;
 
-    res = dt_device_get_address(dev, 0, &addr, &size);
+    res = dt_device_get_paddr(dev, 0, &addr, &size);
     if ( res )
     {
         printk("omap-uart: Unable to retrieve the base"
diff --git a/xen/drivers/char/pl011.c b/xen/drivers/char/pl011.c
index be67242bc0..052a651251 100644
--- a/xen/drivers/char/pl011.c
+++ b/xen/drivers/char/pl011.c
@@ -222,7 +222,7 @@ static struct uart_driver __read_mostly pl011_driver = {
     .vuart_info   = pl011_vuart,
 };
 
-static int __init pl011_uart_init(int irq, u64 addr, u64 size, bool sbsa)
+static int __init pl011_uart_init(int irq, paddr_t addr, paddr_t size, bool sbsa)
 {
     struct pl011 *uart;
 
@@ -258,14 +258,14 @@ static int __init pl011_dt_uart_init(struct dt_device_node *dev,
 {
     const char *config = data;
     int res;
-    u64 addr, size;
+    paddr_t addr, size;
 
     if ( strcmp(config, "") )
     {
         printk("WARNING: UART configuration is not supported\n");
     }
 
-    res = dt_device_get_address(dev, 0, &addr, &size);
+    res = dt_device_get_paddr(dev, 0, &addr, &size);
     if ( res )
     {
         printk("pl011: Unable to retrieve the base"
diff --git a/xen/drivers/char/scif-uart.c b/xen/drivers/char/scif-uart.c
index 2fccafe340..1b28ba90e9 100644
--- a/xen/drivers/char/scif-uart.c
+++ b/xen/drivers/char/scif-uart.c
@@ -311,14 +311,14 @@ static int __init scif_uart_init(struct dt_device_node *dev,
     const char *config = data;
     struct scif_uart *uart;
     int res;
-    u64 addr, size;
+    paddr_t addr, size;
 
     if ( strcmp(config, "") )
         printk("WARNING: UART configuration is not supported\n");
 
     uart = &scif_com;
 
-    res = dt_device_get_address(dev, 0, &addr, &size);
+    res = dt_device_get_paddr(dev, 0, &addr, &size);
     if ( res )
     {
         printk("scif-uart: Unable to retrieve the base"
diff --git a/xen/drivers/passthrough/arm/ipmmu-vmsa.c b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
index 091f09b217..611d9eeba5 100644
--- a/xen/drivers/passthrough/arm/ipmmu-vmsa.c
+++ b/xen/drivers/passthrough/arm/ipmmu-vmsa.c
@@ -794,7 +794,7 @@ static void ipmmu_device_reset(struct ipmmu_vmsa_device *mmu)
 static __init bool ipmmu_stage2_supported(void)
 {
     struct dt_device_node *np;
-    uint64_t addr, size;
+    paddr_t addr, size;
     void __iomem *base;
     uint32_t product, cut;
     bool stage2_supported = false;
@@ -806,7 +806,7 @@ static __init bool ipmmu_stage2_supported(void)
         return false;
     }
 
-    if ( dt_device_get_address(np, 0, &addr, &size) )
+    if ( dt_device_get_paddr(np, 0, &addr, &size) )
     {
         printk(XENLOG_ERR "ipmmu: Failed to get PRR MMIO\n");
         return false;
@@ -884,7 +884,7 @@ static int ipmmu_probe(struct dt_device_node *node)
 {
     const struct dt_device_match *match;
     struct ipmmu_vmsa_device *mmu;
-    uint64_t addr, size;
+    paddr_t addr, size;
     uint32_t reg;
     int irq, ret;
 
@@ -905,7 +905,7 @@ static int ipmmu_probe(struct dt_device_node *node)
     bitmap_zero(mmu->ctx, IPMMU_CTX_MAX);
 
     /* Map I/O memory and request IRQ. */
-    ret = dt_device_get_address(node, 0, &addr, &size);
+    ret = dt_device_get_paddr(node, 0, &addr, &size);
     if ( ret )
     {
         dev_err(&node->dev, "Failed to get MMIO\n");
diff --git a/xen/drivers/passthrough/arm/smmu-v3.c b/xen/drivers/passthrough/arm/smmu-v3.c
index bfdb62b395..b7fa2e90f7 100644
--- a/xen/drivers/passthrough/arm/smmu-v3.c
+++ b/xen/drivers/passthrough/arm/smmu-v3.c
@@ -2428,7 +2428,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
 	}
 
 	/* Base address */
-	ret = dt_device_get_address(np, 0, &ioaddr, &iosize);
+	ret = dt_device_get_paddr(np, 0, &ioaddr, &iosize);
 	if (ret)
 		goto out_free_smmu;
 
diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index 0a514821b3..79281075ba 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -73,8 +73,8 @@
 /* Xen: Helpers to get device MMIO and IRQs */
 struct resource
 {
-	u64 addr;
-	u64 size;
+	paddr_t addr;
+	paddr_t size;
 	unsigned int type;
 };
 
@@ -101,7 +101,7 @@ static struct resource *platform_get_resource(struct platform_device *pdev,
 
 	switch (type) {
 	case IORESOURCE_MEM:
-		ret = dt_device_get_address(pdev, num, &res.addr, &res.size);
+		ret = dt_device_get_paddr(pdev, num, &res.addr, &res.size);
 
 		return ((ret) ? NULL : &res);
 
@@ -169,7 +169,7 @@ static void __iomem *devm_ioremap_resource(struct device *dev,
 	ptr = ioremap_nocache(res->addr, res->size);
 	if (!ptr) {
 		dev_err(dev,
-			"ioremap failed (addr 0x%"PRIx64" size 0x%"PRIx64")\n",
+			"ioremap failed (addr 0x%"PRIpaddr" size 0x%"PRIpaddr")\n",
 			res->addr, res->size);
 		return ERR_PTR(-ENOMEM);
 	}
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index 11bda2fd3d..c1dc5400e1 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -581,6 +581,19 @@ int dt_find_node_by_gpath(XEN_GUEST_HANDLE(char) u_path, uint32_t u_plen,
  */
 const struct dt_device_node *dt_get_parent(const struct dt_device_node *node);
 
+/**
+ * dt_device_get_paddr - Resolve an address for a device
+ * @device: the device whose address is to be resolved
+ * @index: index of the address to resolve
+ * @addr: address filled by this function
+ * @size: size filled by this function
+ *
+ * This function resolves an address, walking the tree, for a give
+ * device-tree node. It returns 0 on success.
+ */
+int dt_device_get_paddr(const struct dt_device_node *dev, unsigned int index,
+                        paddr_t *addr, paddr_t *size);
+
 /**
  * dt_device_get_address - Resolve an address for a device
  * @device: the device whose address is to be resolved
-- 
2.17.1

[XEN v5 04/10] xen/arm: smmu: Use writeq_relaxed_non_atomic() for writing to SMMU_CBn_TTBR0

[Ayan Kumar Halder]

Refer ARM IHI 0062D.c ID070116 (SMMU 2.0 spec), 17-360, 17.3.9,
SMMU_CBn_TTBR0 is a 64 bit register. Thus, one can use
writeq_relaxed_non_atomic() to write to it instead of invoking
writel_relaxed() twice for lower half and upper half of the register.

This also helps us as p2maddr is 'paddr_t' (which may be u32 in future).
Thus, one can assign p2maddr to a 64 bit register and do the bit
manipulations on it, to generate the value for SMMU_CBn_TTBR0.

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
---
Changes from -

v1 - 1. Extracted the patch from "[XEN v1 8/9] xen/arm: Other adaptations required to support 32bit paddr".
Use writeq_relaxed_non_atomic() to write u64 register in a non-atomic
fashion.

v2 - 1. Added R-b.

v3 - 1. No changes.

v4 - 1. Reordered the R-b. No further changes.
(This patch can be committed independent of the series).

 xen/drivers/passthrough/arm/smmu.c | 15 ++++++++-------
 1 file changed, 8 insertions(+), 7 deletions(-)

diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
index 79281075ba..c8ef2a925f 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -499,8 +499,7 @@ enum arm_smmu_s2cr_privcfg {
 #define ARM_SMMU_CB_SCTLR		0x0
 #define ARM_SMMU_CB_RESUME		0x8
 #define ARM_SMMU_CB_TTBCR2		0x10
-#define ARM_SMMU_CB_TTBR0_LO		0x20
-#define ARM_SMMU_CB_TTBR0_HI		0x24
+#define ARM_SMMU_CB_TTBR0		0x20
 #define ARM_SMMU_CB_TTBCR		0x30
 #define ARM_SMMU_CB_S1_MAIR0		0x38
 #define ARM_SMMU_CB_FSR			0x58
@@ -1083,6 +1082,7 @@ static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu, void *addr,
 static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
 {
 	u32 reg;
+	u64 reg64;
 	bool stage1;
 	struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
 	struct arm_smmu_device *smmu = smmu_domain->smmu;
@@ -1177,12 +1177,13 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
 	dev_notice(smmu->dev, "d%u: p2maddr 0x%"PRIpaddr"\n",
 		   smmu_domain->cfg.domain->domain_id, p2maddr);
 
-	reg = (p2maddr & ((1ULL << 32) - 1));
-	writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_LO);
-	reg = (p2maddr >> 32);
+	reg64 = p2maddr;
+
 	if (stage1)
-		reg |= ARM_SMMU_CB_ASID(cfg) << TTBRn_HI_ASID_SHIFT;
-	writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_HI);
+		reg64 |= (((uint64_t) (ARM_SMMU_CB_ASID(cfg) << TTBRn_HI_ASID_SHIFT))
+		         << 32);
+
+	writeq_relaxed_non_atomic(reg64, cb_base + ARM_SMMU_CB_TTBR0);
 
 	/*
 	 * TTBCR
-- 
2.17.1

[XEN v5 05/10] xen/arm: Introduce choice to enable 64/32 bit physical addressing

[Ayan Kumar Halder]

Some Arm based hardware platforms which does not support LPAE
(eg Cortex-R52), uses 32 bit physical addresses.
Also, users may choose to use 32 bits to represent physical addresses
for optimization.

To support the above use cases, we have introduced arch independent
configs to choose if the physical address can be represented using
32 bits (PHYS_ADDR_T_32) or 64 bits (!PHYS_ADDR_T_32).
For now only ARM_32 provides support to enable 32 bit physical
addressing.

When PHYS_ADDR_T_32 is defined, PADDR_BITS is set to 32.
When PHYS_ADDR_T_32 is not defined for ARM_32, PADDR_BITS is set to 40.
When PHYS_ADDR_T_32 is not defined for ARM_64, PADDR_BITS is set to 48.
The last two are same as the current configuration used today on Xen.

PADDR_BITS is also set to 48 when ARM_64 is defined. The reason being
the choice to select ARM_PA_BITS_32/ARM_PA_BITS_40/ARM_PA_BITS_48 is
currently allowed when ARM_32 is defined.

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
---
Changes from -
v1 - 1. Extracted from "[XEN v1 8/9] xen/arm: Other adaptations required to support 32bit paddr".

v2 - 1. Introduced Kconfig choice. ARM_64 can select PHYS_ADDR_64 only whereas
ARM_32 can select PHYS_ADDR_32 or PHYS_ADDR_64.
2. For CONFIG_ARM_PA_32, paddr_t is defined as 'unsigned long'. 

v3 - 1. Allow user to define PADDR_BITS by selecting different config options
ARM_PA_BITS_32, ARM_PA_BITS_40 and ARM_PA_BITS_48.
2. Add the choice under "Architecture Features".

v4 - 1. Removed PHYS_ADDR_T_64 as !PHYS_ADDR_T_32 means PHYS_ADDR_T_32.

 xen/arch/Kconfig                     |  3 +++
 xen/arch/arm/Kconfig                 | 37 ++++++++++++++++++++++++++--
 xen/arch/arm/include/asm/page-bits.h |  6 +----
 xen/arch/arm/include/asm/types.h     |  6 +++++
 xen/arch/arm/mm.c                    |  5 ++++
 5 files changed, 50 insertions(+), 7 deletions(-)

diff --git a/xen/arch/Kconfig b/xen/arch/Kconfig
index 7028f7b74f..67ba38f32f 100644
--- a/xen/arch/Kconfig
+++ b/xen/arch/Kconfig
@@ -1,6 +1,9 @@
 config 64BIT
 	bool
 
+config PHYS_ADDR_T_32
+	bool
+
 config NR_CPUS
 	int "Maximum number of CPUs"
 	range 1 4095
diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
index 239d3aed3c..3f6e13e475 100644
--- a/xen/arch/arm/Kconfig
+++ b/xen/arch/arm/Kconfig
@@ -19,13 +19,46 @@ config ARM
 	select HAS_PMAP
 	select IOMMU_FORCE_PT_SHARE
 
+menu "Architecture Features"
+
+choice
+	prompt "Physical address space size" if ARM_32
+	default ARM_PA_BITS_48 if ARM_64
+	default ARM_PA_BITS_40 if ARM_32
+	help
+	  User can choose to represent the width of physical address. This can
+	  sometimes help in optimizing the size of image when user chooses a
+	  smaller size to represent physical address.
+
+config ARM_PA_BITS_32
+	bool "32-bit"
+	help
+	  On platforms where any physical address can be represented within 32 bits,
+	  user should choose this option. This will help is reduced size of the
+	  binary.
+	select PHYS_ADDR_T_32
+	depends on ARM_32
+
+config ARM_PA_BITS_40
+	bool "40-bit"
+	depends on ARM_32
+
+config ARM_PA_BITS_48
+	bool "40-bit"
+	depends on ARM_48
+endchoice
+
+config PADDR_BITS
+	int
+	default 32 if ARM_PA_BITS_32
+	default 40 if ARM_PA_BITS_40
+	default 48 if ARM_PA_BITS_48 || ARM_64
+
 config ARCH_DEFCONFIG
 	string
 	default "arch/arm/configs/arm32_defconfig" if ARM_32
 	default "arch/arm/configs/arm64_defconfig" if ARM_64
 
-menu "Architecture Features"
-
 source "arch/Kconfig"
 
 config ACPI
diff --git a/xen/arch/arm/include/asm/page-bits.h b/xen/arch/arm/include/asm/page-bits.h
index 5d6477e599..deb381ceeb 100644
--- a/xen/arch/arm/include/asm/page-bits.h
+++ b/xen/arch/arm/include/asm/page-bits.h
@@ -3,10 +3,6 @@
 
 #define PAGE_SHIFT              12
 
-#ifdef CONFIG_ARM_64
-#define PADDR_BITS              48
-#else
-#define PADDR_BITS              40
-#endif
+#define PADDR_BITS              CONFIG_PADDR_BITS
 
 #endif /* __ARM_PAGE_SHIFT_H__ */
diff --git a/xen/arch/arm/include/asm/types.h b/xen/arch/arm/include/asm/types.h
index e218ed77bd..e3cfbbb060 100644
--- a/xen/arch/arm/include/asm/types.h
+++ b/xen/arch/arm/include/asm/types.h
@@ -34,9 +34,15 @@ typedef signed long long s64;
 typedef unsigned long long u64;
 typedef u32 vaddr_t;
 #define PRIvaddr PRIx32
+#if defined(CONFIG_PHYS_ADDR_T_32)
+typedef unsigned long paddr_t;
+#define INVALID_PADDR (~0UL)
+#define PRIpaddr "08lx"
+#else
 typedef u64 paddr_t;
 #define INVALID_PADDR (~0ULL)
 #define PRIpaddr "016llx"
+#endif
 typedef u32 register_t;
 #define PRIregister "08x"
 #elif defined (CONFIG_ARM_64)
diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index b99806af99..6dc37be97e 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -690,6 +690,11 @@ void __init setup_frametable_mappings(paddr_t ps, paddr_t pe)
     const unsigned long mapping_size = frametable_size < MB(32) ? MB(2) : MB(32);
     int rc;
 
+    /*
+     * The size of paddr_t should be sufficient for the complete range of
+     * physical address.
+     */
+    BUILD_BUG_ON((sizeof(paddr_t) * 8) < PADDR_BITS);
     BUILD_BUG_ON(sizeof(struct page_info) != PAGE_INFO_SIZE);
 
     if ( frametable_size > FRAMETABLE_SIZE )
-- 
2.17.1

[XEN v5 06/10] xen/arm: guest_walk: LPAE specific bits should be enclosed within "ifndef CONFIG_PHYS_ADDR_T_32"

[Ayan Kumar Halder]

As the previous patch introduces CONFIG_PHYS_ADDR_T_32 to support 32 bit
physical addresses, the code specific to "Large Physical Address Extension"
(ie LPAE) should be enclosed within "ifndef CONFIG_PHYS_ADDR_T_32".

Refer xen/arch/arm/include/asm/short-desc.h, "short_desc_l1_supersec_t"
unsigned int extbase1:4;    /* Extended base address, PA[35:32] */
unsigned int extbase2:4;    /* Extended base address, PA[39:36] */

Thus, extbase1 and extbase2 are not valid when 32 bit physical addresses
are supported.

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Acked-by: Stefano Stabellini <sstabellini@kernel.org>
---
Changes from -
v1 - 1. Extracted from "[XEN v1 8/9] xen/arm: Other adaptations required to support 32bit paddr".

v2 - 1. Reordered this patch so that it appears after CONFIG_ARM_PA_32 is
introduced (in 6/9).

v3 - 1. Updated the commit message.
2. Added Ack.

v4 - 1. No changes.

 xen/arch/arm/guest_walk.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/arm/guest_walk.c b/xen/arch/arm/guest_walk.c
index 43d3215304..c80a0ce55b 100644
--- a/xen/arch/arm/guest_walk.c
+++ b/xen/arch/arm/guest_walk.c
@@ -154,8 +154,10 @@ static bool guest_walk_sd(const struct vcpu *v,
             mask = (1ULL << L1DESC_SUPERSECTION_SHIFT) - 1;
             *ipa = gva & mask;
             *ipa |= (paddr_t)(pte.supersec.base) << L1DESC_SUPERSECTION_SHIFT;
+#ifndef CONFIG_PHYS_ADDR_T_32
             *ipa |= (paddr_t)(pte.supersec.extbase1) << L1DESC_SUPERSECTION_EXT_BASE1_SHIFT;
             *ipa |= (paddr_t)(pte.supersec.extbase2) << L1DESC_SUPERSECTION_EXT_BASE2_SHIFT;
+#endif /* CONFIG_PHYS_ADDR_T_32 */
         }
 
         /* Set permissions so that the caller can check the flags by herself. */
-- 
2.17.1

[XEN v5 07/10] xen/arm: Restrict zeroeth_table_offset for ARM_64

[Ayan Kumar Halder]

When 32 bit physical addresses are used (ie PHYS_ADDR_T_32=y),
"va >> ZEROETH_SHIFT" causes an overflow.
Also, there is no zeroeth level page table on Arm32.

Also took the opportunity to clean up dump_pt_walk(). One could use
DECLARE_OFFSETS() macro instead of declaring the declaring an array
of page table offsets.

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
---
Changes from -

v1 - Removed the duplicate declaration for DECLARE_OFFSETS.

v2 - 1. Reworded the commit message. 
2. Use CONFIG_ARM_PA_32 to restrict zeroeth_table_offset.

v3 - 1. Added R-b and Ack.

v4 - 1. Removed R-b and Ack as we use CONFIG_PHYS_ADDR_T_32
instead of CONFIG_ARM_PA_BITS_32. This is to be in parity with our earlier
patches where we use CONFIG_PHYS_ADDR_T_32 to denote 32-bit physical addr
support.

 xen/arch/arm/include/asm/lpae.h | 4 ++++
 xen/arch/arm/mm.c               | 7 +------
 2 files changed, 5 insertions(+), 6 deletions(-)

diff --git a/xen/arch/arm/include/asm/lpae.h b/xen/arch/arm/include/asm/lpae.h
index 3fdd5d0de2..7d2f6fd1bd 100644
--- a/xen/arch/arm/include/asm/lpae.h
+++ b/xen/arch/arm/include/asm/lpae.h
@@ -259,7 +259,11 @@ lpae_t mfn_to_xen_entry(mfn_t mfn, unsigned int attr);
 #define first_table_offset(va)  TABLE_OFFSET(first_linear_offset(va))
 #define second_table_offset(va) TABLE_OFFSET(second_linear_offset(va))
 #define third_table_offset(va)  TABLE_OFFSET(third_linear_offset(va))
+#ifdef CONFIG_PHYS_ADDR_T_32
+#define zeroeth_table_offset(va)  0
+#else
 #define zeroeth_table_offset(va)  TABLE_OFFSET(zeroeth_linear_offset(va))
+#endif
 
 /*
  * Macros to define page-tables:
diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
index 6dc37be97e..247510ac57 100644
--- a/xen/arch/arm/mm.c
+++ b/xen/arch/arm/mm.c
@@ -221,12 +221,7 @@ void dump_pt_walk(paddr_t ttbr, paddr_t addr,
 {
     static const char *level_strs[4] = { "0TH", "1ST", "2ND", "3RD" };
     const mfn_t root_mfn = maddr_to_mfn(ttbr);
-    const unsigned int offsets[4] = {
-        zeroeth_table_offset(addr),
-        first_table_offset(addr),
-        second_table_offset(addr),
-        third_table_offset(addr)
-    };
+    DECLARE_OFFSETS(offsets, addr);
     lpae_t pte, *mapping;
     unsigned int level, root_table;
 
-- 
2.17.1

[XEN v5 08/10] xen/arm: domain_build: Check if the address fits the range of physical address

[Ayan Kumar Halder]

handle_pci_range() and map_range_to_domain() take addr and len as uint64_t
parameters. Then frame numbers are obtained from addr and len by right shifting
with PAGE_SHIFT. The page frame numbers are saved using unsigned long.

Now if 64-bit >> PAGE_SHIFT, the result will have 52-bits as valid. On a 32-bit
system, 'unsigned long' is 32-bits. Thus, there is a potential loss of value
when the result is stored as 'unsigned long'.

To mitigate this issue, we check if the starting and end address can be
contained within the range of physical address supported on the system. If not,
then an appropriate error is returned.

Also, the end address is computed once and used when required. And replaced u64
with uint64_t.

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
---

Changes from :-
v1...v4 - NA. New patch introduced in v5.

 xen/arch/arm/domain_build.c | 30 +++++++++++++++++++++++-------
 1 file changed, 23 insertions(+), 7 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 7d28b75517..b98ee506a8 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1637,15 +1637,23 @@ out:
 }
 
 static int __init handle_pci_range(const struct dt_device_node *dev,
-                                   u64 addr, u64 len, void *data)
+                                   uint64_t addr, uint64_t len, void *data)
 {
     struct rangeset *mem_holes = data;
     paddr_t start, end;
     int res;
+    uint64_t end_addr = addr + len - 1;
+
+    if ( addr != (paddr_t)addr || end_addr != (paddr_t)end_addr )
+    {
+        printk(XENLOG_ERR "addr (0x%"PRIx64") or end_addr (0x%"PRIx64") exceeds the maximum allowed width (%d bits) for physical address\n",
+               addr, end_addr, CONFIG_PADDR_BITS);
+        return -ERANGE;
+    }
 
     start = addr & PAGE_MASK;
-    end = PAGE_ALIGN(addr + len);
-    res = rangeset_remove_range(mem_holes, PFN_DOWN(start), PFN_DOWN(end - 1));
+    end = PAGE_ALIGN(end_addr);
+    res = rangeset_remove_range(mem_holes, PFN_DOWN(start), PFN_DOWN(end));
     if ( res )
     {
         printk(XENLOG_ERR "Failed to remove: %#"PRIpaddr"->%#"PRIpaddr"\n",
@@ -2330,11 +2338,19 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
 }
 
 int __init map_range_to_domain(const struct dt_device_node *dev,
-                               u64 addr, u64 len, void *data)
+                               uint64_t addr, uint64_t len, void *data)
 {
     struct map_range_data *mr_data = data;
     struct domain *d = mr_data->d;
     int res;
+    uint64_t end_addr = addr + len - 1;
+
+    if ( addr != (paddr_t)addr || end_addr != (paddr_t)end_addr )
+    {
+        printk(XENLOG_ERR "addr (0x%"PRIx64") or end_addr (0x%"PRIx64") exceeds the maximum allowed width (%d bits) for physical address\n",
+               addr, end_addr, CONFIG_PADDR_BITS);
+        return -ERANGE;
+    }
 
     /*
      * reserved-memory regions are RAM carved out for a special purpose.
@@ -2345,13 +2361,13 @@ int __init map_range_to_domain(const struct dt_device_node *dev,
                      strlen("/reserved-memory/")) != 0 )
     {
         res = iomem_permit_access(d, paddr_to_pfn(addr),
-                paddr_to_pfn(PAGE_ALIGN(addr + len - 1)));
+                paddr_to_pfn(PAGE_ALIGN(end_addr)));
         if ( res )
         {
             printk(XENLOG_ERR "Unable to permit to dom%d access to"
                     " 0x%"PRIx64" - 0x%"PRIx64"\n",
                     d->domain_id,
-                    addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1);
+                    addr & PAGE_MASK, PAGE_ALIGN(end_addr) - 1);
             return res;
         }
     }
@@ -2368,7 +2384,7 @@ int __init map_range_to_domain(const struct dt_device_node *dev,
         {
             printk(XENLOG_ERR "Unable to map 0x%"PRIx64
                    " - 0x%"PRIx64" in domain %d\n",
-                   addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1,
+                   addr & PAGE_MASK, PAGE_ALIGN(end_addr) - 1,
                    d->domain_id);
             return res;
         }
-- 
2.17.1

[XEN v5 09/10] xen/arm: p2m: Use the pa_range_info table to support ARM_32 and ARM_64

[Ayan Kumar Halder]

Restructure the code so that one can use pa_range_info[] table for both
ARM_32 as well as ARM_64.

Also, removed the hardcoding for P2M_ROOT_ORDER and P2M_ROOT_LEVEL as
p2m_root_order can be obtained from the pa_range_info[].root_order and
p2m_root_level can be obtained from pa_range_info[].sl0.

Refer ARM DDI 0406C.d ID040418, B3-1345,
"Use of concatenated first-level translation tables

...However, a 40-bit input address range with a translation granularity of 4KB
requires a total of 28 bits of address resolution. Therefore, a stage 2
translation that supports a 40-bit input address range requires two concatenated
first-level translation tables,..."

Thus, root-order is 1 for 40-bit IPA on ARM_32.

Refer ARM DDI 0406C.d ID040418, B3-1348,

"Determining the required first lookup level for stage 2 translations

For a stage 2 translation, the output address range from the stage 1
translations determines the required input address range for the stage 2
translation. The permitted values of VTCR.SL0 are:

0b00 Stage 2 translation lookup must start at the second level.
0b01 Stage 2 translation lookup must start at the first level.

VTCR.T0SZ must indicate the required input address range. The size of the input
address region is 2^(32-T0SZ) bytes."

Thus VTCR.SL0 = 1 (maximum value) and VTCR.T0SZ = -8 when the size of input
address region is 2^40 bytes.

Thus, pa_range_info[].t0sz = 1 (VTCR.S) | 8 (VTCR.T0SZ) ie 11000b which is 24.

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
---

Changes from -

v3 - 1. New patch introduced in v4.
2. Restructure the code such that pa_range_info[] is used both by ARM_32 as
well as ARM_64.

v4 - 1. Removed the hardcoded definitions of P2M_ROOT_ORDER and P2M_ROOT_LEVEL.
The reason being root_order will not be always 1 (See the next patch).
2. Updated the commit message to explain t0sz, sl0 and root_order values for
32-bit IPA on Arm32.
3. Some sanity fixes.

 xen/arch/arm/include/asm/p2m.h |  8 +-------
 xen/arch/arm/p2m.c             | 34 ++++++++++++++++++----------------
 2 files changed, 19 insertions(+), 23 deletions(-)

diff --git a/xen/arch/arm/include/asm/p2m.h b/xen/arch/arm/include/asm/p2m.h
index 91df922e1c..28c68428d3 100644
--- a/xen/arch/arm/include/asm/p2m.h
+++ b/xen/arch/arm/include/asm/p2m.h
@@ -14,16 +14,10 @@
 /* Holds the bit size of IPAs in p2m tables.  */
 extern unsigned int p2m_ipa_bits;
 
-#ifdef CONFIG_ARM_64
 extern unsigned int p2m_root_order;
 extern unsigned int p2m_root_level;
-#define P2M_ROOT_ORDER    p2m_root_order
+#define P2M_ROOT_ORDER p2m_root_order
 #define P2M_ROOT_LEVEL p2m_root_level
-#else
-/* First level P2M is always 2 consecutive pages */
-#define P2M_ROOT_ORDER    1
-#define P2M_ROOT_LEVEL 1
-#endif
 
 struct domain;
 
diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 948f199d84..4583658f92 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -19,9 +19,9 @@
 
 #define INVALID_VMID 0 /* VMID 0 is reserved */
 
-#ifdef CONFIG_ARM_64
 unsigned int __read_mostly p2m_root_order;
 unsigned int __read_mostly p2m_root_level;
+#ifdef CONFIG_ARM_64
 static unsigned int __read_mostly max_vmid = MAX_VMID_8_BIT;
 /* VMID is by default 8 bit width on AArch64 */
 #define MAX_VMID       max_vmid
@@ -2265,16 +2265,6 @@ void __init setup_virt_paging(void)
     /* Setup Stage 2 address translation */
     register_t val = VTCR_RES1|VTCR_SH0_IS|VTCR_ORGN0_WBWA|VTCR_IRGN0_WBWA;
 
-#ifdef CONFIG_ARM_32
-    if ( p2m_ipa_bits < 40 )
-        panic("P2M: Not able to support %u-bit IPA at the moment\n",
-              p2m_ipa_bits);
-
-    printk("P2M: 40-bit IPA\n");
-    p2m_ipa_bits = 40;
-    val |= VTCR_T0SZ(0x18); /* 40 bit IPA */
-    val |= VTCR_SL0(0x1); /* P2M starts at first level */
-#else /* CONFIG_ARM_64 */
     static const struct {
         unsigned int pabits; /* Physical Address Size */
         unsigned int t0sz;   /* Desired T0SZ, minimum in comment */
@@ -2283,19 +2273,24 @@ void __init setup_virt_paging(void)
     } pa_range_info[] __initconst = {
         /* T0SZ minimum and SL0 maximum from ARM DDI 0487H.a Table D5-6 */
         /*      PA size, t0sz(min), root-order, sl0(max) */
-        [0] = { 32,      32/*32*/,  0,          1 },
-        [1] = { 36,      28/*28*/,  0,          1 },
-        [2] = { 40,      24/*24*/,  1,          1 },
+        [0] = { 40,      24/*24*/,  1,          1 },
+#ifdef CONFIG_ARM_64
+        [1] = { 32,      32/*32*/,  0,          1 },
+        [2] = { 36,      28/*28*/,  0,          1 },
         [3] = { 42,      22/*22*/,  3,          1 },
         [4] = { 44,      20/*20*/,  0,          2 },
         [5] = { 48,      16/*16*/,  0,          2 },
         [6] = { 52,      12/*12*/,  4,          2 },
         [7] = { 0 }  /* Invalid */
+#else
+        [1] = { 0 }  /* Invalid */
+#endif
     };
 
     unsigned int i;
     unsigned int pa_range = 0x10; /* Larger than any possible value */
 
+#ifdef CONFIG_ARM_64
     /*
      * Restrict "p2m_ipa_bits" if needed. As P2M table is always configured
      * with IPA bits == PA bits, compare against "pabits".
@@ -2309,6 +2304,9 @@ void __init setup_virt_paging(void)
      */
     if ( system_cpuinfo.mm64.vmid_bits == MM64_VMID_16_BITS_SUPPORT )
         max_vmid = MAX_VMID_16_BIT;
+#else
+    p2m_ipa_bits = PADDR_BITS;
+#endif
 
     /* Choose suitable "pa_range" according to the resulted "p2m_ipa_bits". */
     for ( i = 0; i < ARRAY_SIZE(pa_range_info); i++ )
@@ -2324,24 +2322,28 @@ void __init setup_virt_paging(void)
     if ( pa_range >= ARRAY_SIZE(pa_range_info) || !pa_range_info[pa_range].pabits )
         panic("Unknown encoding of ID_AA64MMFR0_EL1.PARange %x\n", pa_range);
 
+#ifdef CONFIG_ARM_64
     val |= VTCR_PS(pa_range);
     val |= VTCR_TG0_4K;
 
     /* Set the VS bit only if 16 bit VMID is supported. */
     if ( MAX_VMID == MAX_VMID_16_BIT )
         val |= VTCR_VS;
+
+    p2m_ipa_bits = 64 - pa_range_info[pa_range].t0sz;
+#endif
+
     val |= VTCR_SL0(pa_range_info[pa_range].sl0);
     val |= VTCR_T0SZ(pa_range_info[pa_range].t0sz);
 
     p2m_root_order = pa_range_info[pa_range].root_order;
     p2m_root_level = 2 - pa_range_info[pa_range].sl0;
-    p2m_ipa_bits = 64 - pa_range_info[pa_range].t0sz;
 
     printk("P2M: %d-bit IPA with %d-bit PA and %d-bit VMID\n",
            p2m_ipa_bits,
            pa_range_info[pa_range].pabits,
            ( MAX_VMID == MAX_VMID_16_BIT ) ? 16 : 8);
-#endif
+
     printk("P2M: %d levels with order-%d root, VTCR 0x%"PRIregister"\n",
            4 - P2M_ROOT_LEVEL, P2M_ROOT_ORDER, val);
 
-- 
2.17.1

[XEN v5 10/10] xen/arm: p2m: Enable support for 32bit IPA for ARM_32

[Ayan Kumar Halder]

Refer ARM DDI 0406C.d ID040418, B3-1345,

"A stage 2 translation with an input address range of 31-34 bits can
start the translation either:

- With a first-level lookup, accessing a first-level translation
  table with 2-16 entries.

- With a second-level lookup, accessing a set of concatenated
  second-level translation tables"

Thus, for 32 bit IPA, there will be no concatenated root level tables.
So, the root-order is 0.

Also, Refer ARM DDI 0406C.d ID040418, B3-1348
"Determining the required first lookup level for stage 2 translations

For a stage 2 translation, the output address range from the stage 1
translations determines the required input address range for the stage 2
translation. The permitted values of VTCR.SL0 are:
0b00 Stage 2 translation lookup must start at the second level.
0b01 Stage 2 translation lookup must start at the first level.

VTCR.T0SZ must indicate the required input address range. The size of
the input address region is 2^(32-T0SZ) bytes."

Thus VTCR.SL0 = 1 (maximum value) and VTCR.T0SZ = 0 when the size of
input address region is 2^32 bytes.

Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
---

Changes from -

v1 - New patch.

v2 - 1. Added Ack.

v3 - 1. Dropped Ack. 
2. Rebased the patch based on the previous change.

v4 - 1. t0sz is 0 for 32-bit IPA on Arm32.
2. Updated the commit message to explain t0sz, sl0 and root_order.

 xen/arch/arm/p2m.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 4583658f92..746b6553e5 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -2283,7 +2283,8 @@ void __init setup_virt_paging(void)
         [6] = { 52,      12/*12*/,  4,          2 },
         [7] = { 0 }  /* Invalid */
 #else
-        [1] = { 0 }  /* Invalid */
+        [1] = { 32,      0/*0*/,    0,          1 },
+        [2] = { 0 }  /* Invalid */
 #endif
     };
 
-- 
2.17.1

Re: [XEN v5 00/10] Add support for 32-bit physical address

[Jan Beulich]

On 13.04.2023 19:37, Ayan Kumar Halder wrote:
> v4 - 1. Dropped "xen/drivers: ns16550: Use paddr_t for io_base/io_size" from the patch series.
> 
> As Jan (jbeulich@suse.com) had pointed out in https://lore.kernel.org/xen-devel/20230321140357.24094-5-ayan.kumar.halder@amd.com/,
> ns16550 driver requires some prior cleanup. Also, ns16550 can be ignored for now
> for the 32-bit paddr support (which is mainly targeted for Arm). I will send out
> separate patches to fix this once the current serie is committed (or in ready to
> commit state). I hope that is fine with Jan ?

Sure - if you don't really need the change right here, I'm happy to see
it come separately later on.

Jan

Re: [XEN v5 05/10] xen/arm: Introduce choice to enable 64/32 bit physical addressing

[Jan Beulich]

On 13.04.2023 19:37, Ayan Kumar Halder wrote:
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -19,13 +19,46 @@ config ARM
>  	select HAS_PMAP
>  	select IOMMU_FORCE_PT_SHARE
>  
> +menu "Architecture Features"
> +
> +choice
> +	prompt "Physical address space size" if ARM_32
> +	default ARM_PA_BITS_48 if ARM_64
> +	default ARM_PA_BITS_40 if ARM_32
> +	help
> +	  User can choose to represent the width of physical address. This can
> +	  sometimes help in optimizing the size of image when user chooses a
> +	  smaller size to represent physical address.
> +
> +config ARM_PA_BITS_32
> +	bool "32-bit"
> +	help
> +	  On platforms where any physical address can be represented within 32 bits,
> +	  user should choose this option. This will help is reduced size of the
> +	  binary.
> +	select PHYS_ADDR_T_32
> +	depends on ARM_32

May I suggest that "help" come generally last, and preferably "depends on"
before "select"?

Jan

Re: [XEN v5 01/10] xen/arm: domain_build: Track unallocated pages using the frame number

[Michal Orzel]

Hi Ayan,

On 13/04/2023 19:37, Ayan Kumar Halder wrote:
> 
> 
> rangeset_{xxx}_range() functions are invoked with 'start' and 'size' as
> arguments which are either 'uint64_t' or 'paddr_t'. However, the function
> accepts 'unsigned long' for 'start' and 'size'. 'unsigned long' is 32 bits for
> Arm32. Thus, there is an implicit downcasting from 'uint64_t'/'paddr_t' to
> 'unsigned long' when invoking rangeset_{xxx}_range().
> 
> So, it may seem there is a possibility of lose of data due to truncation.
> 
> In reality, 'start' and 'size' are always page aligned. And Arm32 currently
> supports 40 bits as the width of physical address.
> So if the addresses are page aligned, the last 12 bits contain zeroes.
> Thus, we could instead pass page frame number which will contain 28 bits (40-12
> on Arm32) and this can be represented using 'unsigned long'.
> 
> On Arm64, this change will not induce any adverse side effect as the width of
> physical address is 48 bits. 
NIT: This reads as const, so it would be better to write: "as the max supported width of ..."

Thus, the width of 'gfn' (ie 48 - 12 = 36) can be
> represented using 'unsigned long' (which is 64 bits wide).
> 
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>

With or without (after all this is just a commit msg):
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Michal Orzel]

Hi Ayan,

On 13/04/2023 19:37, Ayan Kumar Halder wrote:
> 
> 
> The DT functions (dt_read_number(), device_tree_get_reg(), fdt_get_mem_rsv())
> currently accept or return 64-bit values.
> 
> In future when we support 32-bit physical address, these DT functions are
> expected to accept/return 32-bit or 64-bit values (depending on the width of
> physical address). Also, we wish to detect if any truncation has occurred
> (i.e. while parsing 32-bit physical addresses from 64-bit values read from DT).
> 
> device_tree_get_reg() should now be able to return paddr_t. This is invoked by
> various callers to get DT address and size.
> 
> For fdt_get_mem_rsv(), we have introduced a wrapper named
> fdt_get_mem_rsv_paddr() which will invoke fdt_get_mem_rsv() and translate
> uint64_t to paddr_t. The reason being we cannot modify fdt_get_mem_rsv() as it
> has been imported from external source.
> 
> For dt_read_number(), we have also introduced a wrapper named dt_read_paddr()
> dt_read_paddr() to read physical addresses. We chose not to modify the original
> function as it is used in places where it needs to specifically read 64-bit
> values from dt (For e.g. dt_property_read_u64()).
> 
> Xen prints warning when it detects truncation in cases where it is not able to
> return error.
> 
> Also, replaced u32/u64 with uint32_t/uint64_t in the functions touched
> by the code changes.
> 
> Also, initialized variables to fix the warning "-Werror=maybe-uninitialized".
I can see that now you explicitly set to 0 variables passed to fdt_get_mem_rsv_paddr()
which haven't been initialized before being passed to fdt_get_mem_rsv(). Is this what
you are reffering to? I cannot reproduce it, hence my question.

> 
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> ---
> 
> Changes from
> 
> v1 - 1. Dropped "[XEN v1 2/9] xen/arm: Define translate_dt_address_size() for the translation between u64 and paddr_t" and
> "[XEN v1 4/9] xen/arm: Use translate_dt_address_size() to translate between device tree addr/size and paddr_t", instead
> this approach achieves the same purpose.
> 
> 2. No need to check for truncation while converting values from u64 to paddr_t.
> 
> v2 - 1. Use "( (dt_start >> (PADDR_SHIFT - 1)) > 1 )" to detect truncation.
> 2. Introduced libfdt_xen.h to implement fdt_get_mem_rsv_paddr
> 3. Logged error messages in case truncation is detected.
> 
> v3 - 1. Renamed libfdt_xen.h to libfdt-xen.h.
> 2. Replaced u32/u64 with uint32_t/uint64_t
> 3. Use "(paddr_t)val != val" to check for truncation.
> 4. Removed the alias "#define PADDR_SHIFT PADDR_BITS".
> 
> v4 - 1. Added a WARN() when truncation is detected.
> 2. Always check the return value of fdt_get_mem_rsv().
> 
>  xen/arch/arm/bootfdt.c              | 48 +++++++++++++++++++------
>  xen/arch/arm/domain_build.c         |  2 +-
>  xen/arch/arm/include/asm/setup.h    |  4 +--
>  xen/arch/arm/setup.c                | 18 +++++-----
>  xen/arch/arm/smpboot.c              |  2 +-
>  xen/include/xen/device_tree.h       | 23 ++++++++++++
>  xen/include/xen/libfdt/libfdt-xen.h | 55 +++++++++++++++++++++++++++++
>  7 files changed, 129 insertions(+), 23 deletions(-)
>  create mode 100644 xen/include/xen/libfdt/libfdt-xen.h
> 
> diff --git a/xen/arch/arm/bootfdt.c b/xen/arch/arm/bootfdt.c
> index 0085c28d74..ac8148da55 100644
> --- a/xen/arch/arm/bootfdt.c
> +++ b/xen/arch/arm/bootfdt.c
> @@ -11,7 +11,7 @@
>  #include <xen/efi.h>
>  #include <xen/device_tree.h>
>  #include <xen/lib.h>
> -#include <xen/libfdt/libfdt.h>
> +#include <xen/libfdt/libfdt-xen.h>
>  #include <xen/sort.h>
>  #include <xsm/xsm.h>
>  #include <asm/setup.h>
> @@ -52,11 +52,37 @@ static bool __init device_tree_node_compatible(const void *fdt, int node,
>      return false;
>  }
> 
> -void __init device_tree_get_reg(const __be32 **cell, u32 address_cells,
> -                                u32 size_cells, u64 *start, u64 *size)
> +void __init device_tree_get_reg(const __be32 **cell, uint32_t address_cells,
> +                                uint32_t size_cells, paddr_t *start,
> +                                paddr_t *size)
>  {
> -    *start = dt_next_cell(address_cells, cell);
> -    *size = dt_next_cell(size_cells, cell);
> +    uint64_t dt_start, dt_size;
> +
> +    /*
> +     * dt_next_cell will return uint64_t whereas paddr_t may not be 64-bit.
> +     * Thus, there is an implicit cast from uint64_t to paddr_t.
> +     */
> +    dt_start = dt_next_cell(address_cells, cell);
> +    dt_size = dt_next_cell(size_cells, cell);
> +
> +    if ( dt_start != (paddr_t)dt_start )
> +    {
> +        printk("Error: Physical address greater than max width supported\n");
I find it a bit odd to see Error and on the next line warning. I would drop "Error" here
but feel free not to as this is just my POV.

> +        WARN();
> +    }
> +
> +    if ( dt_size != (paddr_t)dt_size )
> +    {
> +        printk("Error: Physical size greater than max width supported\n");
> +        WARN();
> +    }
> +
> +    /*
> +     * Xen will truncate the address/size if it is greater than the maximum
> +     * supported width and it will give an appropriate warning.
> +     */
> +    *start = dt_start;
> +    *size = dt_size;
>  }
> 
>  static int __init device_tree_get_meminfo(const void *fdt, int node,
> @@ -326,7 +352,7 @@ static int __init process_chosen_node(const void *fdt, int node,
>          printk("linux,initrd-start property has invalid length %d\n", len);
>          return -EINVAL;
>      }
> -    start = dt_read_number((void *)&prop->data, dt_size_to_cells(len));
> +    start = dt_read_paddr((void *)&prop->data, dt_size_to_cells(len));
> 
>      prop = fdt_get_property(fdt, node, "linux,initrd-end", &len);
>      if ( !prop )
> @@ -339,7 +365,7 @@ static int __init process_chosen_node(const void *fdt, int node,
>          printk("linux,initrd-end property has invalid length %d\n", len);
>          return -EINVAL;
>      }
> -    end = dt_read_number((void *)&prop->data, dt_size_to_cells(len));
> +    end = dt_read_paddr((void *)&prop->data, dt_size_to_cells(len));
> 
>      if ( start >= end )
>      {
> @@ -593,10 +619,12 @@ static void __init early_print_info(void)
>      nr_rsvd = fdt_num_mem_rsv(device_tree_flattened);
>      for ( i = 0; i < nr_rsvd; i++ )
>      {
> -        paddr_t s, e;
> -        if ( fdt_get_mem_rsv(device_tree_flattened, i, &s, &e) < 0 )
> +        paddr_t s = 0, e = 0;
> +
> +        if ( fdt_get_mem_rsv_paddr(device_tree_flattened, i, &s, &e) < 0 )
>              continue;
> -        /* fdt_get_mem_rsv returns length */
> +
> +        /* fdt_get_mem_rsv_paddr returns length */
>          e += s;
>          printk(" RESVD[%u]: %"PRIpaddr" - %"PRIpaddr"\n", i, s, e);
>      }
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index c8f08d8ee2..15c8bdd9e4 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -949,7 +949,7 @@ static int __init process_shm(struct domain *d, struct kernel_info *kinfo,
>          BUG_ON(!prop);
>          cells = (const __be32 *)prop->value;
>          device_tree_get_reg(&cells, addr_cells, addr_cells, &pbase, &gbase);
> -        psize = dt_read_number(cells, size_cells);
> +        psize = dt_read_paddr(cells, size_cells);
>          if ( !IS_ALIGNED(pbase, PAGE_SIZE) || !IS_ALIGNED(gbase, PAGE_SIZE) )
>          {
>              printk("%pd: physical address 0x%"PRIpaddr", or guest address 0x%"PRIpaddr" is not suitably aligned.\n",
> diff --git a/xen/arch/arm/include/asm/setup.h b/xen/arch/arm/include/asm/setup.h
> index a926f30a2b..7b697d879e 100644
> --- a/xen/arch/arm/include/asm/setup.h
> +++ b/xen/arch/arm/include/asm/setup.h
> @@ -157,8 +157,8 @@ const char *boot_module_kind_as_string(bootmodule_kind kind);
>  extern uint32_t hyp_traps_vector[];
>  void init_traps(void);
> 
> -void device_tree_get_reg(const __be32 **cell, u32 address_cells,
> -                         u32 size_cells, u64 *start, u64 *size);
> +void device_tree_get_reg(const __be32 **cell, uint32_t address_cells,
> +                         uint32_t size_cells, paddr_t *start, paddr_t *size);
> 
>  u32 device_tree_get_u32(const void *fdt, int node,
>                          const char *prop_name, u32 dflt);
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 1f26f67b90..d2a3d8c340 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -29,7 +29,7 @@
>  #include <xen/virtual_region.h>
>  #include <xen/vmap.h>
>  #include <xen/trace.h>
> -#include <xen/libfdt/libfdt.h>
> +#include <xen/libfdt/libfdt-xen.h>
>  #include <xen/acpi.h>
>  #include <xen/warning.h>
>  #include <asm/alternative.h>
> @@ -220,13 +220,13 @@ static void __init dt_unreserved_regions(paddr_t s, paddr_t e,
> 
>      for ( i = first; i < nr ; i++ )
>      {
> -        paddr_t r_s, r_e;
> +        paddr_t r_s = 0, r_e = 0;
> 
> -        if ( fdt_get_mem_rsv(device_tree_flattened, i, &r_s, &r_e ) < 0 )
> +        if ( fdt_get_mem_rsv_paddr(device_tree_flattened, i, &r_s, &r_e ) < 0 )
>              /* If we can't read it, pretend it doesn't exist... */
>              continue;
> 
> -        r_e += r_s; /* fdt_get_mem_rsv returns length */
> +        r_e += r_s; /* fdt_get_mem_rsv_paddr returns length */
> 
>          if ( s < r_e && r_s < e )
>          {
> @@ -500,15 +500,15 @@ static paddr_t __init consider_modules(paddr_t s, paddr_t e,
> 
>      for ( ; i < mi->nr_mods + nr; i++ )
>      {
> -        paddr_t mod_s, mod_e;
> +        paddr_t mod_s = 0, mod_e = 0;
> 
> -        if ( fdt_get_mem_rsv(device_tree_flattened,
> -                             i - mi->nr_mods,
> -                             &mod_s, &mod_e ) < 0 )
> +        if ( fdt_get_mem_rsv_paddr(device_tree_flattened,
> +                                   i - mi->nr_mods,
> +                                   &mod_s, &mod_e ) < 0 )
>              /* If we can't read it, pretend it doesn't exist... */
>              continue;
> 
> -        /* fdt_get_mem_rsv returns length */
> +        /* fdt_get_mem_rsv_paddr returns length */
>          mod_e += mod_s;
> 
>          if ( s < mod_e && mod_s < e )
> diff --git a/xen/arch/arm/smpboot.c b/xen/arch/arm/smpboot.c
> index 412ae22869..c15c177487 100644
> --- a/xen/arch/arm/smpboot.c
> +++ b/xen/arch/arm/smpboot.c
> @@ -159,7 +159,7 @@ static void __init dt_smp_init_cpus(void)
>              continue;
>          }
> 
> -        addr = dt_read_number(prop, dt_n_addr_cells(cpu));
> +        addr = dt_read_paddr(prop, dt_n_addr_cells(cpu));
> 
>          hwid = addr;
>          if ( hwid != addr )
> diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
> index 19a74909ce..11bda2fd3d 100644
> --- a/xen/include/xen/device_tree.h
> +++ b/xen/include/xen/device_tree.h
> @@ -241,6 +241,29 @@ static inline u64 dt_read_number(const __be32 *cell, int size)
>      return r;
>  }
> 
> +/* Wrapper for dt_read_number() to return paddr_t (instead of uint64_t) */
> +static inline paddr_t dt_read_paddr(const __be32 *cell, int size)
> +{
> +    uint64_t dt_r = 0;
no need for this assignment

> +    paddr_t r;
> +
> +    dt_r = dt_read_number(cell, size);
In device_tree_get_reg() you added a note about implicit cast and here it is missing.

> +
> +    if ( dt_r != (paddr_t)dt_r )
> +    {
> +        printk("Error: Physical address greater than max width supported\n");
> +        WARN();
> +    }
> +
> +    /*
> +     * Xen will truncate the address/size if it is greater than the maximum
> +     * supported width and it will give an appropriate warning.
> +     */
> +    r = dt_r;
> +
> +    return r;
> +}
> +
>  /* Helper to convert a number of cells to bytes */
>  static inline int dt_cells_to_size(int size)
>  {
> diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
> new file mode 100644
> index 0000000000..3296a368a6
> --- /dev/null
> +++ b/xen/include/xen/libfdt/libfdt-xen.h
> @@ -0,0 +1,55 @@
> +/*
> + * SPDX-License-Identifier: GPL-2.0-only
Our CODING_STYLE says:
New files should start with a single-line SPDX comment, ..., e.g.
/* SPDX-License-Identifier: GPL-2.0 */

For me it would be perfectly fine to do as you did but it is not what our docs state
(i.e. single-line comment). It might be that we need to modify CODING_STYLE instead.

> + *
> + * xen/include/xen/libfdt/libfdt-xen.h
> + *
> + * Wrapper functions for device tree. This helps to convert dt values
> + * between uint64_t and paddr_t.
> + *
> + * Copyright (C) 2023, Advanced Micro Devices, Inc. All Rights Reserved.
> + */
> +
> +#ifndef LIBFDT_XEN_H
> +#define LIBFDT_XEN_H
> +
> +#include <xen/libfdt/libfdt.h>
> +
> +static inline int fdt_get_mem_rsv_paddr(const void *fdt, int n,
> +                                        paddr_t *address,
> +                                        paddr_t *size)
> +{
> +    uint64_t dt_addr;
> +    uint64_t dt_size;
> +    int ret = 0;
no need for this assignment

> +
> +    ret = fdt_get_mem_rsv(fdt, n, &dt_addr, &dt_size);
> +    if ( ret )
> +        return ret;
> +
> +    if ( dt_addr != (paddr_t)dt_addr )
> +    {
> +        printk("Error: Physical address greater than max width supported\n");
> +        return -FDT_ERR_MAX;
> +    }
> +
> +    if ( dt_size != (paddr_t)dt_size )
> +    {
> +        printk("Error: Physical size greater than max width supported\n");
> +        return -FDT_ERR_MAX;
> +    }
> +
> +    *address = dt_addr;
> +    *size = dt_size;
> +
> +    return ret;
> +}
> +
> +#endif /* LIBFDT_XEN_H */
please add an empty line here.

> +/*
> + * Local variables:
> + * mode: C
> + * c-file-style: "BSD"
> + * c-basic-offset: 4
> + * indent-tabs-mode: nil
> + * End:
> + */
> --
> 2.17.1
> 
> 

~Michal

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Julien Grall]

Hi,

>> diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
>> new file mode 100644
>> index 0000000000..3296a368a6
>> --- /dev/null
>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>> @@ -0,0 +1,55 @@
>> +/*
>> + * SPDX-License-Identifier: GPL-2.0-only
> Our CODING_STYLE says:
> New files should start with a single-line SPDX comment, ..., e.g.
> /* SPDX-License-Identifier: GPL-2.0 */
> 
> For me it would be perfectly fine to do as you did but it is not what our docs state
> (i.e. single-line comment). It might be that we need to modify CODING_STYLE instead.

 From my reading of https://spdx.dev/ids/#how, what you suggest would 
not be a valid SPDX-License-Idenfier as everything in needs to be in 
*one* (including the begin/end comment).

Cheers,

-- 
Julien Grall

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Michal Orzel]

Hi Julien,

On 19/04/2023 15:26, Julien Grall wrote:
> 
> 
> Hi,
> 
>>> diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
>>> new file mode 100644
>>> index 0000000000..3296a368a6
>>> --- /dev/null
>>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>>> @@ -0,0 +1,55 @@
>>> +/*
>>> + * SPDX-License-Identifier: GPL-2.0-only
>> Our CODING_STYLE says:
>> New files should start with a single-line SPDX comment, ..., e.g.
>> /* SPDX-License-Identifier: GPL-2.0 */
>>
>> For me it would be perfectly fine to do as you did but it is not what our docs state
>> (i.e. single-line comment). It might be that we need to modify CODING_STYLE instead.
> 
>  From my reading of https://spdx.dev/ids/#how, what you suggest would
> not be a valid SPDX-License-Idenfier as everything in needs to be in
> *one* (including the begin/end comment).
I said what is written in our CODING_STYLE and what we did already for lots of files, i.e. having 2 comments:
/* SPDX-License-Identifier: GPL-2.0 */
/*
 * ...
 */

In the link you provided, the "*one line*" requirement refers only to SPDX short form identifier
which does not contain any other information like author, description, etc..

~Michal

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Michal Orzel]

On 19/04/2023 15:19, Michal Orzel wrote:
> 
> 
> Hi Ayan,
> 
> On 13/04/2023 19:37, Ayan Kumar Halder wrote:
>>
>>
>> The DT functions (dt_read_number(), device_tree_get_reg(), fdt_get_mem_rsv())
>> currently accept or return 64-bit values.
>>
>> In future when we support 32-bit physical address, these DT functions are
>> expected to accept/return 32-bit or 64-bit values (depending on the width of
>> physical address). Also, we wish to detect if any truncation has occurred
>> (i.e. while parsing 32-bit physical addresses from 64-bit values read from DT).
>>
>> device_tree_get_reg() should now be able to return paddr_t. This is invoked by
>> various callers to get DT address and size.
>>
>> For fdt_get_mem_rsv(), we have introduced a wrapper named
>> fdt_get_mem_rsv_paddr() which will invoke fdt_get_mem_rsv() and translate
>> uint64_t to paddr_t. The reason being we cannot modify fdt_get_mem_rsv() as it
>> has been imported from external source.
>>
>> For dt_read_number(), we have also introduced a wrapper named dt_read_paddr()
>> dt_read_paddr() to read physical addresses. We chose not to modify the original
>> function as it is used in places where it needs to specifically read 64-bit
>> values from dt (For e.g. dt_property_read_u64()).
>>
>> Xen prints warning when it detects truncation in cases where it is not able to
>> return error.
>>
>> Also, replaced u32/u64 with uint32_t/uint64_t in the functions touched
>> by the code changes.
>>
>> Also, initialized variables to fix the warning "-Werror=maybe-uninitialized".
> I can see that now you explicitly set to 0 variables passed to fdt_get_mem_rsv_paddr()
> which haven't been initialized before being passed to fdt_get_mem_rsv(). Is this what
> you are reffering to? I cannot reproduce it, hence my question.
I can see why did you get this error.
Before your change, we always checked for an error from fdt_get_mem_rsv() by checking if < 0.
In your wrapper fdt_get_mem_rsv_paddr(), you switched (not sure why) to checking if not zero.
Becasue of this, you got an error and tried to fix it by initializing the variables to 0.

~Michal

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Julien Grall]

On 19/04/2023 14:39, Michal Orzel wrote:
> Hi Julien,
> 
> On 19/04/2023 15:26, Julien Grall wrote:
>>
>>
>> Hi,
>>
>>>> diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
>>>> new file mode 100644
>>>> index 0000000000..3296a368a6
>>>> --- /dev/null
>>>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>>>> @@ -0,0 +1,55 @@
>>>> +/*
>>>> + * SPDX-License-Identifier: GPL-2.0-only
>>> Our CODING_STYLE says:
>>> New files should start with a single-line SPDX comment, ..., e.g.
>>> /* SPDX-License-Identifier: GPL-2.0 */
>>>
>>> For me it would be perfectly fine to do as you did but it is not what our docs state
>>> (i.e. single-line comment). It might be that we need to modify CODING_STYLE instead.
>>
>>   From my reading of https://spdx.dev/ids/#how, what you suggest would
>> not be a valid SPDX-License-Idenfier as everything in needs to be in
>> *one* (including the begin/end comment).
> I said what is written in our CODING_STYLE and what we did already for lots of files, i.e. having 2 comments:
> /* SPDX-License-Identifier: GPL-2.0 */
> /*
>   * ...
>   */

My comment was about your suggestion to update the CODING_STYLE not what 
it is currently written. Sorry if this wasn't clear enough.

>  > In the link you provided, the "*one line*" requirement refers only to 
SPDX short form identifier
> which does not contain any other information like author, description, etc..
Right because the SPDX block is intended to be in own block and there is 
a another block for the author, description, etc.

I am not in favor of changing of the CODING_STYLE and I thought I would 
express it right now to spare the round-trip of writing a patch.

Cheers,

-- 
Julien Grall

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Michal Orzel]

On 19/04/2023 16:20, Julien Grall wrote:
> 
> 
> On 19/04/2023 14:39, Michal Orzel wrote:
>> Hi Julien,
>>
>> On 19/04/2023 15:26, Julien Grall wrote:
>>>
>>>
>>> Hi,
>>>
>>>>> diff --git a/xen/include/xen/libfdt/libfdt-xen.h b/xen/include/xen/libfdt/libfdt-xen.h
>>>>> new file mode 100644
>>>>> index 0000000000..3296a368a6
>>>>> --- /dev/null
>>>>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>>>>> @@ -0,0 +1,55 @@
>>>>> +/*
>>>>> + * SPDX-License-Identifier: GPL-2.0-only
>>>> Our CODING_STYLE says:
>>>> New files should start with a single-line SPDX comment, ..., e.g.
>>>> /* SPDX-License-Identifier: GPL-2.0 */
>>>>
>>>> For me it would be perfectly fine to do as you did but it is not what our docs state
>>>> (i.e. single-line comment). It might be that we need to modify CODING_STYLE instead.
>>>
>>>   From my reading of https://spdx.dev/ids/#how, what you suggest would
>>> not be a valid SPDX-License-Idenfier as everything in needs to be in
>>> *one* (including the begin/end comment).
>> I said what is written in our CODING_STYLE and what we did already for lots of files, i.e. having 2 comments:
>> /* SPDX-License-Identifier: GPL-2.0 */
>> /*
>>   * ...
>>   */
> 
> My comment was about your suggestion to update the CODING_STYLE not what
> it is currently written. Sorry if this wasn't clear enough.
> 
>>  > In the link you provided, the "*one line*" requirement refers only to
> SPDX short form identifier
>> which does not contain any other information like author, description, etc..
> Right because the SPDX block is intended to be in own block and there is
> a another block for the author, description, etc.
> 
> I am not in favor of changing of the CODING_STYLE and I thought I would
> express it right now to spare the round-trip of writing a patch.
Sure thing :)
So, all in all, we agree that SPDX comment must be a single line comment on its own
(which is also stated in our CODING_STYLE) and not like it was placed in this patch, right?

If so, somehow the wrong placement sneaked into recently added RISCV files:
arch/riscv/include/asm/csr.h
arch/riscv/include/asm/riscv_encoding.h

~Michal

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Julien Grall]

On 19/04/2023 15:29, Michal Orzel wrote:
>> I am not in favor of changing of the CODING_STYLE and I thought I would
>> express it right now to spare the round-trip of writing a patch.
> Sure thing :)
> So, all in all, we agree that SPDX comment must be a single line comment on its own
> (which is also stated in our CODING_STYLE) and not like it was placed in this patch, right?

Correct.

Cheers,

-- 
Julien Grall

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Ayan Kumar Halder]

On 19/04/2023 14:54, Michal Orzel wrote:
> On 19/04/2023 15:19, Michal Orzel wrote:
>>
>> Hi Ayan,
>>
>> On 13/04/2023 19:37, Ayan Kumar Halder wrote:
>>>
>>> The DT functions (dt_read_number(), device_tree_get_reg(), fdt_get_mem_rsv())
>>> currently accept or return 64-bit values.
>>>
>>> In future when we support 32-bit physical address, these DT functions are
>>> expected to accept/return 32-bit or 64-bit values (depending on the width of
>>> physical address). Also, we wish to detect if any truncation has occurred
>>> (i.e. while parsing 32-bit physical addresses from 64-bit values read from DT).
>>>
>>> device_tree_get_reg() should now be able to return paddr_t. This is invoked by
>>> various callers to get DT address and size.
>>>
>>> For fdt_get_mem_rsv(), we have introduced a wrapper named
>>> fdt_get_mem_rsv_paddr() which will invoke fdt_get_mem_rsv() and translate
>>> uint64_t to paddr_t. The reason being we cannot modify fdt_get_mem_rsv() as it
>>> has been imported from external source.
>>>
>>> For dt_read_number(), we have also introduced a wrapper named dt_read_paddr()
>>> dt_read_paddr() to read physical addresses. We chose not to modify the original
>>> function as it is used in places where it needs to specifically read 64-bit
>>> values from dt (For e.g. dt_property_read_u64()).
>>>
>>> Xen prints warning when it detects truncation in cases where it is not able to
>>> return error.
>>>
>>> Also, replaced u32/u64 with uint32_t/uint64_t in the functions touched
>>> by the code changes.
>>>
>>> Also, initialized variables to fix the warning "-Werror=maybe-uninitialized".
>> I can see that now you explicitly set to 0 variables passed to fdt_get_mem_rsv_paddr()
>> which haven't been initialized before being passed to fdt_get_mem_rsv(). Is this what
>> you are reffering to? I cannot reproduce it, hence my question.
> I can see why did you get this error.
> Before your change, we always checked for an error from fdt_get_mem_rsv() by checking if < 0.
> In your wrapper fdt_get_mem_rsv_paddr(), you switched (not sure why) to checking if not zero.
> Becasue of this, you got an error and tried to fix it by initializing the variables to 0.

I actually wanted to return the error code obtained from 
fdt_get_mem_rsv() to the caller.

In this case, it returns a single error code. So does this look sane to 
you ?

diff --git a/xen/include/xen/libfdt/libfdt-xen.h 
b/xen/include/xen/libfdt/libfdt-xen.h
index 3296a368a6..1da87d6668 100644
--- a/xen/include/xen/libfdt/libfdt-xen.h
+++ b/xen/include/xen/libfdt/libfdt-xen.h
@@ -22,9 +22,8 @@ static inline int fdt_get_mem_rsv_paddr(const void 
*fdt, int n,
      uint64_t dt_size;
      int ret = 0;

-    ret = fdt_get_mem_rsv(fdt, n, &dt_addr, &dt_size);
-    if ( ret )
-        return ret;
+    if ( fdt_get_mem_rsv(fdt, n, &dt_addr, &dt_size) < 0 )
+        return -FDT_ERR_BADOFFSET;

      if ( dt_addr != (paddr_t)dt_addr )
      {

- Ayan

>
> ~Michal

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Ayan Kumar Halder]

On 19/04/2023 14:19, Michal Orzel wrote:
> Hi Ayan,

Hi Michal,

...

> --- /dev/null
> +++ b/xen/include/xen/libfdt/libfdt-xen.h
> @@ -0,0 +1,55 @@
> +/*
> + * SPDX-License-Identifier: GPL-2.0-only
> Our CODING_STYLE says:
> New files should start with a single-line SPDX comment, ..., e.g.
> /* SPDX-License-Identifier: GPL-2.0 */
>
> For me it would be perfectly fine to do as you did but it is not what our docs state
> (i.e. single-line comment). It might be that we need to modify CODING_STYLE instead.

Just to be clear, this is what we should have (as Julien had earlier 
suggested to use **GPL-2.0-only** )

diff --git a/xen/include/xen/libfdt/libfdt-xen.h 
b/xen/include/xen/libfdt/libfdt-xen.h
index 3296a368a6..cad7ad3bfb 100644
--- a/xen/include/xen/libfdt/libfdt-xen.h
+++ b/xen/include/xen/libfdt/libfdt-xen.h
@@ -1,6 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0-only
  /*
- * SPDX-License-Identifier: GPL-2.0-only
- *
   * xen/include/xen/libfdt/libfdt-xen.h
   *
   * Wrapper functions for device tree. This helps to convert dt values

- Ayan

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Julien Grall]

Hi Ayan,

On 19/04/2023 15:58, Ayan Kumar Halder wrote:
> 
> On 19/04/2023 14:54, Michal Orzel wrote:
>> On 19/04/2023 15:19, Michal Orzel wrote:
>>>
>>> Hi Ayan,
>>>
>>> On 13/04/2023 19:37, Ayan Kumar Halder wrote:
>>>>
>>>> The DT functions (dt_read_number(), device_tree_get_reg(), 
>>>> fdt_get_mem_rsv())
>>>> currently accept or return 64-bit values.
>>>>
>>>> In future when we support 32-bit physical address, these DT 
>>>> functions are
>>>> expected to accept/return 32-bit or 64-bit values (depending on the 
>>>> width of
>>>> physical address). Also, we wish to detect if any truncation has 
>>>> occurred
>>>> (i.e. while parsing 32-bit physical addresses from 64-bit values 
>>>> read from DT).
>>>>
>>>> device_tree_get_reg() should now be able to return paddr_t. This is 
>>>> invoked by
>>>> various callers to get DT address and size.
>>>>
>>>> For fdt_get_mem_rsv(), we have introduced a wrapper named
>>>> fdt_get_mem_rsv_paddr() which will invoke fdt_get_mem_rsv() and 
>>>> translate
>>>> uint64_t to paddr_t. The reason being we cannot modify 
>>>> fdt_get_mem_rsv() as it
>>>> has been imported from external source.
>>>>
>>>> For dt_read_number(), we have also introduced a wrapper named 
>>>> dt_read_paddr()
>>>> dt_read_paddr() to read physical addresses. We chose not to modify 
>>>> the original
>>>> function as it is used in places where it needs to specifically read 
>>>> 64-bit
>>>> values from dt (For e.g. dt_property_read_u64()).
>>>>
>>>> Xen prints warning when it detects truncation in cases where it is 
>>>> not able to
>>>> return error.
>>>>
>>>> Also, replaced u32/u64 with uint32_t/uint64_t in the functions touched
>>>> by the code changes.
>>>>
>>>> Also, initialized variables to fix the warning 
>>>> "-Werror=maybe-uninitialized".
>>> I can see that now you explicitly set to 0 variables passed to 
>>> fdt_get_mem_rsv_paddr()
>>> which haven't been initialized before being passed to 
>>> fdt_get_mem_rsv(). Is this what
>>> you are reffering to? I cannot reproduce it, hence my question.
>> I can see why did you get this error.
>> Before your change, we always checked for an error from 
>> fdt_get_mem_rsv() by checking if < 0.
>> In your wrapper fdt_get_mem_rsv_paddr(), you switched (not sure why) 
>> to checking if not zero.
>> Becasue of this, you got an error and tried to fix it by initializing 
>> the variables to 0.
> 
> I actually wanted to return the error code obtained from 
> fdt_get_mem_rsv() to the caller.
> 
> In this case, it returns a single error code.

I would rather not rely on this.

> So does this look sane to 
> you ?
> 
> diff --git a/xen/include/xen/libfdt/libfdt-xen.h 
> b/xen/include/xen/libfdt/libfdt-xen.h
> index 3296a368a6..1da87d6668 100644
> --- a/xen/include/xen/libfdt/libfdt-xen.h
> +++ b/xen/include/xen/libfdt/libfdt-xen.h
> @@ -22,9 +22,8 @@ static inline int fdt_get_mem_rsv_paddr(const void 
> *fdt, int n,
>       uint64_t dt_size;
>       int ret = 0;
> 
> -    ret = fdt_get_mem_rsv(fdt, n, &dt_addr, &dt_size);
> -    if ( ret )
> -        return ret;
> +    if ( fdt_get_mem_rsv(fdt, n, &dt_addr, &dt_size) < 0 )
> +        return -FDT_ERR_BADOFFSET;

So the problem if you check for ret to be non-zero. But the caller of 
fdt_get_mem_rsv_paddr() check for < 0.

Given that fdt_get_mem_rsv() is not inline, the compiler doesn't know 
that it will not return a positive value (other than 0). Hence why I 
think you get an unitialize value.

The snippet below should work:

if ( ret < 0 )
   return ret;

Cheers,

-- 
Julien Grall

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Ayan Kumar Halder]

On 19/04/2023 16:18, Julien Grall wrote:
> Hi Ayan,
Hi Julien,
>
> On 19/04/2023 15:58, Ayan Kumar Halder wrote:
>>
>> On 19/04/2023 14:54, Michal Orzel wrote:
>>> On 19/04/2023 15:19, Michal Orzel wrote:
>>>>
>>>> Hi Ayan,
>>>>
>>>> On 13/04/2023 19:37, Ayan Kumar Halder wrote:
>>>>>
>>>>> The DT functions (dt_read_number(), device_tree_get_reg(), 
>>>>> fdt_get_mem_rsv())
>>>>> currently accept or return 64-bit values.
>>>>>
>>>>> In future when we support 32-bit physical address, these DT 
>>>>> functions are
>>>>> expected to accept/return 32-bit or 64-bit values (depending on 
>>>>> the width of
>>>>> physical address). Also, we wish to detect if any truncation has 
>>>>> occurred
>>>>> (i.e. while parsing 32-bit physical addresses from 64-bit values 
>>>>> read from DT).
>>>>>
>>>>> device_tree_get_reg() should now be able to return paddr_t. This 
>>>>> is invoked by
>>>>> various callers to get DT address and size.
>>>>>
>>>>> For fdt_get_mem_rsv(), we have introduced a wrapper named
>>>>> fdt_get_mem_rsv_paddr() which will invoke fdt_get_mem_rsv() and 
>>>>> translate
>>>>> uint64_t to paddr_t. The reason being we cannot modify 
>>>>> fdt_get_mem_rsv() as it
>>>>> has been imported from external source.
>>>>>
>>>>> For dt_read_number(), we have also introduced a wrapper named 
>>>>> dt_read_paddr()
>>>>> dt_read_paddr() to read physical addresses. We chose not to modify 
>>>>> the original
>>>>> function as it is used in places where it needs to specifically 
>>>>> read 64-bit
>>>>> values from dt (For e.g. dt_property_read_u64()).
>>>>>
>>>>> Xen prints warning when it detects truncation in cases where it is 
>>>>> not able to
>>>>> return error.
>>>>>
>>>>> Also, replaced u32/u64 with uint32_t/uint64_t in the functions 
>>>>> touched
>>>>> by the code changes.
>>>>>
>>>>> Also, initialized variables to fix the warning 
>>>>> "-Werror=maybe-uninitialized".
>>>> I can see that now you explicitly set to 0 variables passed to 
>>>> fdt_get_mem_rsv_paddr()
>>>> which haven't been initialized before being passed to 
>>>> fdt_get_mem_rsv(). Is this what
>>>> you are reffering to? I cannot reproduce it, hence my question.
>>> I can see why did you get this error.
>>> Before your change, we always checked for an error from 
>>> fdt_get_mem_rsv() by checking if < 0.
>>> In your wrapper fdt_get_mem_rsv_paddr(), you switched (not sure why) 
>>> to checking if not zero.
>>> Becasue of this, you got an error and tried to fix it by 
>>> initializing the variables to 0.
>>
>> I actually wanted to return the error code obtained from 
>> fdt_get_mem_rsv() to the caller.
>>
>> In this case, it returns a single error code.
>
> I would rather not rely on this.
>
>> So does this look sane to you ?
>>
>> diff --git a/xen/include/xen/libfdt/libfdt-xen.h 
>> b/xen/include/xen/libfdt/libfdt-xen.h
>> index 3296a368a6..1da87d6668 100644
>> --- a/xen/include/xen/libfdt/libfdt-xen.h
>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>> @@ -22,9 +22,8 @@ static inline int fdt_get_mem_rsv_paddr(const void 
>> *fdt, int n,
>>       uint64_t dt_size;
>>       int ret = 0;
>>
>> -    ret = fdt_get_mem_rsv(fdt, n, &dt_addr, &dt_size);
>> -    if ( ret )
>> -        return ret;
>> +    if ( fdt_get_mem_rsv(fdt, n, &dt_addr, &dt_size) < 0 )
>> +        return -FDT_ERR_BADOFFSET;
>
> So the problem if you check for ret to be non-zero. But the caller of 
> fdt_get_mem_rsv_paddr() check for < 0.
>
> Given that fdt_get_mem_rsv() is not inline, the compiler doesn't know 
> that it will not return a positive value (other than 0). Hence why I 
> think you get an unitialize value.
>
> The snippet below should work:
>
> if ( ret < 0 )
>   return ret;

Awesome, thanks for the explanation. This works. :)

- Ayan

>
> Cheers,
>

Re: [XEN v5 02/10] xen/arm: Typecast the DT values into paddr_t

[Michal Orzel]

Hi Ayan,

On 19/04/2023 17:12, Ayan Kumar Halder wrote:
> 
> On 19/04/2023 14:19, Michal Orzel wrote:
>> Hi Ayan,
> 
> Hi Michal,
> 
> ...
> 
>> --- /dev/null
>> +++ b/xen/include/xen/libfdt/libfdt-xen.h
>> @@ -0,0 +1,55 @@
>> +/*
>> + * SPDX-License-Identifier: GPL-2.0-only
>> Our CODING_STYLE says:
>> New files should start with a single-line SPDX comment, ..., e.g.
>> /* SPDX-License-Identifier: GPL-2.0 */
>>
>> For me it would be perfectly fine to do as you did but it is not what our docs state
>> (i.e. single-line comment). It might be that we need to modify CODING_STYLE instead.
> 
> Just to be clear, this is what we should have (as Julien had earlier 
> suggested to use **GPL-2.0-only** )
I used GPL-2.0 just for example. We want of course GPL-2.0-only.

> 
> diff --git a/xen/include/xen/libfdt/libfdt-xen.h 
> b/xen/include/xen/libfdt/libfdt-xen.h
> index 3296a368a6..cad7ad3bfb 100644
> --- a/xen/include/xen/libfdt/libfdt-xen.h
> +++ b/xen/include/xen/libfdt/libfdt-xen.h
> @@ -1,6 +1,5 @@
> +// SPDX-License-Identifier: GPL-2.0-only
You should use /* */ style instead of //

>   /*
> - * SPDX-License-Identifier: GPL-2.0-only
> - *
>    * xen/include/xen/libfdt/libfdt-xen.h
>    *
>    * Wrapper functions for device tree. This helps to convert dt values
> 
> - Ayan
> 

~Michal

Re: [XEN v5 03/10] xen/arm: Introduce a wrapper for dt_device_get_address() to handle paddr_t

[Michal Orzel]

Hi Ayan,

On 13/04/2023 19:37, Ayan Kumar Halder wrote:
> 
> 
> dt_device_get_address() can accept uint64_t only for address and size.
> However, the address/size denotes physical addresses. Thus, they should
> be represented by 'paddr_t'.
> Consequently, we introduce a wrapper for dt_device_get_address() ie
> dt_device_get_paddr() which accepts address/size as paddr_t and inturn
> invokes dt_device_get_address() after converting address/size to
> uint64_t.
> 
> The reason for introducing this is that in future 'paddr_t' may not
> always be 64-bit. Thus, we need an explicit wrapper to do the type
> conversion and return an error in case of truncation.
> 
> With this, callers can now invoke dt_device_get_paddr().
> 
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> ---
> Changes from -
> 
> v1 - 1. New patch.
> 
> v2 - 1. Extracted part of "[XEN v2 05/11] xen/arm: Use paddr_t instead of u64 for address/size"
> into this patch.
> 
> 2. dt_device_get_address() callers now invoke dt_device_get_paddr() instead.
> 
> 3. Logged error in case of truncation.
> 
> v3 - 1. Modified the truncation checks as "dt_addr != (paddr_t)dt_addr".
> 2. Some sanity fixes.
> 
> v4 - 1. Some sanity fixes.
> 2. Preserved the declaration of dt_device_get_address() in
> xen/include/xen/device_tree.h. The reason being it is currently used by
> ns16550.c. This driver requires some more changes as pointed by Jan in
> https://lore.kernel.org/xen-devel/6196e90f-752e-e61a-45ce-37e46c22b812@suse.com/
> which is to be addressed as a separate series.
> 
>  xen/arch/arm/domain_build.c                | 10 +++----
>  xen/arch/arm/gic-v2.c                      | 10 +++----
>  xen/arch/arm/gic-v3-its.c                  |  4 +--
>  xen/arch/arm/gic-v3.c                      | 10 +++----
>  xen/arch/arm/pci/pci-host-common.c         |  6 ++--
>  xen/arch/arm/platforms/brcm-raspberry-pi.c |  2 +-
>  xen/arch/arm/platforms/brcm.c              |  6 ++--
>  xen/arch/arm/platforms/exynos5.c           | 32 ++++++++++----------
>  xen/arch/arm/platforms/sunxi.c             |  2 +-
>  xen/arch/arm/platforms/xgene-storm.c       |  2 +-
>  xen/common/device_tree.c                   | 35 ++++++++++++++++++++++
>  xen/drivers/char/cadence-uart.c            |  4 +--
>  xen/drivers/char/exynos4210-uart.c         |  4 +--
>  xen/drivers/char/imx-lpuart.c              |  4 +--
>  xen/drivers/char/meson-uart.c              |  4 +--
>  xen/drivers/char/mvebu-uart.c              |  4 +--
>  xen/drivers/char/omap-uart.c               |  4 +--
>  xen/drivers/char/pl011.c                   |  6 ++--
>  xen/drivers/char/scif-uart.c               |  4 +--
>  xen/drivers/passthrough/arm/ipmmu-vmsa.c   |  8 ++---
>  xen/drivers/passthrough/arm/smmu-v3.c      |  2 +-
>  xen/drivers/passthrough/arm/smmu.c         |  8 ++---
>  xen/include/xen/device_tree.h              | 13 ++++++++
>  23 files changed, 116 insertions(+), 68 deletions(-)
> 
> diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
> index 6c9712ab7b..fdef74e7ff 100644
> --- a/xen/common/device_tree.c
> +++ b/xen/common/device_tree.c
> @@ -955,6 +955,41 @@ int dt_device_get_address(const struct dt_device_node *dev, unsigned int index,
>      return 0;
>  }
> 
> +int dt_device_get_paddr(const struct dt_device_node *dev, unsigned int index,
> +                        paddr_t *addr, paddr_t *size)
> +{
> +    uint64_t dt_addr = 0, dt_size = 0;
no need for these assignments

> +    int ret;
> +
> +    ret = dt_device_get_address(dev, index, &dt_addr, &dt_size);
> +    if ( ret )
> +        return ret;
> +
> +    if ( addr )
Since dt_device_get_paddr() is a wrapper for dt_device_get_address(), I think you should maintain the same
error logic for addr i.e. if no addr was specified (NULL), you return -EINVAL:
if ( !addr )
    return -EINVAL;

> +    {
> +        if ( dt_addr != (paddr_t)dt_addr )
> +        {
> +            printk("Error: Physical address 0x%"PRIx64" for node=%s is greater than max width (%zu bytes) supported\n",
> +                   dt_addr, dev->name, sizeof(paddr_t));
> +            return -ERANGE;
> +        }
> +
> +        *addr = dt_addr;
> +    }
> +
> +    if ( size )
> +    {
> +        if ( dt_size != (paddr_t)dt_size )
> +        {
> +            printk("Error: Physical size 0x%"PRIx64" for node=%s is greater than max width (%zu bytes) supported\n",
> +                   dt_size, dev->name, sizeof(paddr_t));
> +            return -ERANGE;
> +        }
add empty line

~Michal

Re: [XEN v5 04/10] xen/arm: smmu: Use writeq_relaxed_non_atomic() for writing to SMMU_CBn_TTBR0

[Michal Orzel]

Hi Ayan,

On 13/04/2023 19:37, Ayan Kumar Halder wrote:
> 
> 
> Refer ARM IHI 0062D.c ID070116 (SMMU 2.0 spec), 17-360, 17.3.9,
> SMMU_CBn_TTBR0 is a 64 bit register. Thus, one can use
> writeq_relaxed_non_atomic() to write to it instead of invoking
> writel_relaxed() twice for lower half and upper half of the register.
> 
> This also helps us as p2maddr is 'paddr_t' (which may be u32 in future).
> Thus, one can assign p2maddr to a 64 bit register and do the bit
> manipulations on it, to generate the value for SMMU_CBn_TTBR0.
> 
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
> ---
> Changes from -
> 
> v1 - 1. Extracted the patch from "[XEN v1 8/9] xen/arm: Other adaptations required to support 32bit paddr".
> Use writeq_relaxed_non_atomic() to write u64 register in a non-atomic
> fashion.
> 
> v2 - 1. Added R-b.
> 
> v3 - 1. No changes.
> 
> v4 - 1. Reordered the R-b. No further changes.
> (This patch can be committed independent of the series).
> 
>  xen/drivers/passthrough/arm/smmu.c | 15 ++++++++-------
>  1 file changed, 8 insertions(+), 7 deletions(-)
> 
> diff --git a/xen/drivers/passthrough/arm/smmu.c b/xen/drivers/passthrough/arm/smmu.c
> index 79281075ba..c8ef2a925f 100644
> --- a/xen/drivers/passthrough/arm/smmu.c
> +++ b/xen/drivers/passthrough/arm/smmu.c
> @@ -499,8 +499,7 @@ enum arm_smmu_s2cr_privcfg {
>  #define ARM_SMMU_CB_SCTLR              0x0
>  #define ARM_SMMU_CB_RESUME             0x8
>  #define ARM_SMMU_CB_TTBCR2             0x10
> -#define ARM_SMMU_CB_TTBR0_LO           0x20
> -#define ARM_SMMU_CB_TTBR0_HI           0x24
> +#define ARM_SMMU_CB_TTBR0              0x20
>  #define ARM_SMMU_CB_TTBCR              0x30
>  #define ARM_SMMU_CB_S1_MAIR0           0x38
>  #define ARM_SMMU_CB_FSR                        0x58
> @@ -1083,6 +1082,7 @@ static void arm_smmu_flush_pgtable(struct arm_smmu_device *smmu, void *addr,
>  static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
>  {
>         u32 reg;
> +       u64 reg64;
Shouldn't you be using uint64_t? Also ...

>         bool stage1;
>         struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
>         struct arm_smmu_device *smmu = smmu_domain->smmu;
> @@ -1177,12 +1177,13 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
>         dev_notice(smmu->dev, "d%u: p2maddr 0x%"PRIpaddr"\n",
>                    smmu_domain->cfg.domain->domain_id, p2maddr);
> 
> -       reg = (p2maddr & ((1ULL << 32) - 1));
> -       writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_LO);
> -       reg = (p2maddr >> 32);
> +       reg64 = p2maddr;
> +
>         if (stage1)
> -               reg |= ARM_SMMU_CB_ASID(cfg) << TTBRn_HI_ASID_SHIFT;
> -       writel_relaxed(reg, cb_base + ARM_SMMU_CB_TTBR0_HI);
> +               reg64 |= (((uint64_t) (ARM_SMMU_CB_ASID(cfg) << TTBRn_HI_ASID_SHIFT))
... here you use uint64_t for a cast and not u64

~Michal

Re: [XEN v5 07/10] xen/arm: Restrict zeroeth_table_offset for ARM_64

[Michal Orzel]

Hi Ayan,

On 13/04/2023 19:37, Ayan Kumar Halder wrote:
> 
> 
> When 32 bit physical addresses are used (ie PHYS_ADDR_T_32=y),
> "va >> ZEROETH_SHIFT" causes an overflow.
> Also, there is no zeroeth level page table on Arm32.
> 
> Also took the opportunity to clean up dump_pt_walk(). One could use
> DECLARE_OFFSETS() macro instead of declaring the declaring an array
s/declaring the declaring/declaring/

> of page table offsets.
> 
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
Reviewed-by: Michal Orzel <michal.orzel@amd.com>

~Michal

Re: [XEN v5 08/10] xen/arm: domain_build: Check if the address fits the range of physical address

[Michal Orzel]

Hi Ayan,

On 13/04/2023 19:37, Ayan Kumar Halder wrote:
> 
> 
> handle_pci_range() and map_range_to_domain() take addr and len as uint64_t
> parameters. Then frame numbers are obtained from addr and len by right shifting
> with PAGE_SHIFT. The page frame numbers are saved using unsigned long.
Maybe better to say "expressed" rather than "saved"

> 
> Now if 64-bit >> PAGE_SHIFT, the result will have 52-bits as valid. On a 32-bit
> system, 'unsigned long' is 32-bits. Thus, there is a potential loss of value
> when the result is stored as 'unsigned long'.
> 
> To mitigate this issue, we check if the starting and end address can be
> contained within the range of physical address supported on the system. If not,
> then an appropriate error is returned.
> 
> Also, the end address is computed once and used when required. And replaced u64
> with uint64_t.
> 
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> ---
> 
> Changes from :-
> v1...v4 - NA. New patch introduced in v5.
> 
>  xen/arch/arm/domain_build.c | 30 +++++++++++++++++++++++-------
>  1 file changed, 23 insertions(+), 7 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 7d28b75517..b98ee506a8 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1637,15 +1637,23 @@ out:
>  }
> 
>  static int __init handle_pci_range(const struct dt_device_node *dev,
> -                                   u64 addr, u64 len, void *data)
> +                                   uint64_t addr, uint64_t len, void *data)
>  {
>      struct rangeset *mem_holes = data;
>      paddr_t start, end;
>      int res;
> +    uint64_t end_addr = addr + len - 1;
> +
> +    if ( addr != (paddr_t)addr || end_addr != (paddr_t)end_addr )
> +    {
> +        printk(XENLOG_ERR "addr (0x%"PRIx64") or end_addr (0x%"PRIx64") exceeds the maximum allowed width (%d bits) for physical address\n",
I don't think it is wise to print variable names (end_addr) to the user. Better would be to say explicitly: start, end address.
Also to make the message shorter you could write: ... exceeds the maximum allowed PA width (%u bits)

> +               addr, end_addr, CONFIG_PADDR_BITS);
Why CONFIG_PADDR_BITS if you already introduced PADDR_BITS macro

> +        return -ERANGE;
> +    }
> 
>      start = addr & PAGE_MASK;
> -    end = PAGE_ALIGN(addr + len);
> -    res = rangeset_remove_range(mem_holes, PFN_DOWN(start), PFN_DOWN(end - 1));
> +    end = PAGE_ALIGN(end_addr);
> +    res = rangeset_remove_range(mem_holes, PFN_DOWN(start), PFN_DOWN(end));
I doubt PFN_DOWN(end) is the same as PFN_DOWN(end - 1), so I think you should keep the behavior as it was

>      if ( res )
>      {
>          printk(XENLOG_ERR "Failed to remove: %#"PRIpaddr"->%#"PRIpaddr"\n",
> @@ -2330,11 +2338,19 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
>  }
> 
>  int __init map_range_to_domain(const struct dt_device_node *dev,
> -                               u64 addr, u64 len, void *data)
> +                               uint64_t addr, uint64_t len, void *data)
You changed u64 to uint64_t in a definition but not in a prototype. Please fix.

>  {
>      struct map_range_data *mr_data = data;
>      struct domain *d = mr_data->d;
>      int res;
> +    uint64_t end_addr = addr + len - 1;
> +
> +    if ( addr != (paddr_t)addr || end_addr != (paddr_t)end_addr )
> +    {
> +        printk(XENLOG_ERR "addr (0x%"PRIx64") or end_addr (0x%"PRIx64") exceeds the maximum allowed width (%d bits) for physical address\n",
> +               addr, end_addr, CONFIG_PADDR_BITS);
please see the remarks above about this code

> +        return -ERANGE;
> +    }
> 
>      /*
>       * reserved-memory regions are RAM carved out for a special purpose.
> @@ -2345,13 +2361,13 @@ int __init map_range_to_domain(const struct dt_device_node *dev,
>                       strlen("/reserved-memory/")) != 0 )
>      {
>          res = iomem_permit_access(d, paddr_to_pfn(addr),
> -                paddr_to_pfn(PAGE_ALIGN(addr + len - 1)));
> +                paddr_to_pfn(PAGE_ALIGN(end_addr)));
>          if ( res )
>          {
>              printk(XENLOG_ERR "Unable to permit to dom%d access to"
>                      " 0x%"PRIx64" - 0x%"PRIx64"\n",
>                      d->domain_id,
> -                    addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1);
> +                    addr & PAGE_MASK, PAGE_ALIGN(end_addr) - 1);
>              return res;
>          }
>      }
> @@ -2368,7 +2384,7 @@ int __init map_range_to_domain(const struct dt_device_node *dev,
>          {
>              printk(XENLOG_ERR "Unable to map 0x%"PRIx64
>                     " - 0x%"PRIx64" in domain %d\n",
> -                   addr & PAGE_MASK, PAGE_ALIGN(addr + len) - 1,
> +                   addr & PAGE_MASK, PAGE_ALIGN(end_addr) - 1,
>                     d->domain_id);
>              return res;
>          }
> --
> 2.17.1
> 
> 

~Michal

Re: [XEN v5 01/10] xen/arm: domain_build: Track unallocated pages using the frame number

[Julien Grall]

Hi,

On 19/04/2023 13:05, Michal Orzel wrote:
> On 13/04/2023 19:37, Ayan Kumar Halder wrote:
>>
>>
>> rangeset_{xxx}_range() functions are invoked with 'start' and 'size' as
>> arguments which are either 'uint64_t' or 'paddr_t'. However, the function
>> accepts 'unsigned long' for 'start' and 'size'. 'unsigned long' is 32 bits for
>> Arm32. Thus, there is an implicit downcasting from 'uint64_t'/'paddr_t' to
>> 'unsigned long' when invoking rangeset_{xxx}_range().
>>
>> So, it may seem there is a possibility of lose of data due to truncation.
>>
>> In reality, 'start' and 'size' are always page aligned. And Arm32 currently
>> supports 40 bits as the width of physical address.
>> So if the addresses are page aligned, the last 12 bits contain zeroes.
>> Thus, we could instead pass page frame number which will contain 28 bits (40-12
>> on Arm32) and this can be represented using 'unsigned long'.
>>
>> On Arm64, this change will not induce any adverse side effect as the width of
>> physical address is 48 bits.
> NIT: This reads as const, so it would be better to write: "as the max supported width of ..."
> 
> Thus, the width of 'gfn' (ie 48 - 12 = 36) can be
>> represented using 'unsigned long' (which is 64 bits wide).
>>
>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> 
> With or without (after all this is just a commit msg):

It is always good to have accurate commit message for the future reader.

> Reviewed-by: Michal Orzel <michal.orzel@amd.com>

Acked-by: Julien Grall <jgrall@amazon.com>

-- 
Julien Grall

Re: [XEN v5 09/10] xen/arm: p2m: Use the pa_range_info table to support ARM_32 and ARM_64

[Michal Orzel]

Hi Ayan,

On 13/04/2023 19:37, Ayan Kumar Halder wrote:
> 
> 
> Restructure the code so that one can use pa_range_info[] table for both
> ARM_32 as well as ARM_64.
> 
> Also, removed the hardcoding for P2M_ROOT_ORDER and P2M_ROOT_LEVEL as
> p2m_root_order can be obtained from the pa_range_info[].root_order and
> p2m_root_level can be obtained from pa_range_info[].sl0.
> 
> Refer ARM DDI 0406C.d ID040418, B3-1345,
> "Use of concatenated first-level translation tables
> 
> ...However, a 40-bit input address range with a translation granularity of 4KB
> requires a total of 28 bits of address resolution. Therefore, a stage 2
> translation that supports a 40-bit input address range requires two concatenated
> first-level translation tables,..."
> 
> Thus, root-order is 1 for 40-bit IPA on ARM_32.
> 
> Refer ARM DDI 0406C.d ID040418, B3-1348,
> 
> "Determining the required first lookup level for stage 2 translations
> 
> For a stage 2 translation, the output address range from the stage 1
> translations determines the required input address range for the stage 2
> translation. The permitted values of VTCR.SL0 are:
> 
> 0b00 Stage 2 translation lookup must start at the second level.
> 0b01 Stage 2 translation lookup must start at the first level.
> 
> VTCR.T0SZ must indicate the required input address range. The size of the input
> address region is 2^(32-T0SZ) bytes."
> 
> Thus VTCR.SL0 = 1 (maximum value) and VTCR.T0SZ = -8 when the size of input
> address region is 2^40 bytes.
> 
> Thus, pa_range_info[].t0sz = 1 (VTCR.S) | 8 (VTCR.T0SZ) ie 11000b which is 24.
> 
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> ---
> 
> Changes from -
> 
> v3 - 1. New patch introduced in v4.
> 2. Restructure the code such that pa_range_info[] is used both by ARM_32 as
> well as ARM_64.
> 
> v4 - 1. Removed the hardcoded definitions of P2M_ROOT_ORDER and P2M_ROOT_LEVEL.
> The reason being root_order will not be always 1 (See the next patch).
> 2. Updated the commit message to explain t0sz, sl0 and root_order values for
> 32-bit IPA on Arm32.
> 3. Some sanity fixes.
> 
>  xen/arch/arm/include/asm/p2m.h |  8 +-------
>  xen/arch/arm/p2m.c             | 34 ++++++++++++++++++----------------
>  2 files changed, 19 insertions(+), 23 deletions(-)
> 
> diff --git a/xen/arch/arm/include/asm/p2m.h b/xen/arch/arm/include/asm/p2m.h
> index 91df922e1c..28c68428d3 100644
> --- a/xen/arch/arm/include/asm/p2m.h
> +++ b/xen/arch/arm/include/asm/p2m.h
> @@ -14,16 +14,10 @@
>  /* Holds the bit size of IPAs in p2m tables.  */
>  extern unsigned int p2m_ipa_bits;
> 
> -#ifdef CONFIG_ARM_64
>  extern unsigned int p2m_root_order;
>  extern unsigned int p2m_root_level;
> -#define P2M_ROOT_ORDER    p2m_root_order
> +#define P2M_ROOT_ORDER p2m_root_order
>  #define P2M_ROOT_LEVEL p2m_root_level
> -#else
> -/* First level P2M is always 2 consecutive pages */
> -#define P2M_ROOT_ORDER    1
> -#define P2M_ROOT_LEVEL 1
> -#endif
> 
>  struct domain;
> 
> diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
> index 948f199d84..4583658f92 100644
> --- a/xen/arch/arm/p2m.c
> +++ b/xen/arch/arm/p2m.c
> @@ -19,9 +19,9 @@
> 
>  #define INVALID_VMID 0 /* VMID 0 is reserved */
> 
> -#ifdef CONFIG_ARM_64
>  unsigned int __read_mostly p2m_root_order;
>  unsigned int __read_mostly p2m_root_level;
> +#ifdef CONFIG_ARM_64
>  static unsigned int __read_mostly max_vmid = MAX_VMID_8_BIT;
>  /* VMID is by default 8 bit width on AArch64 */
>  #define MAX_VMID       max_vmid
> @@ -2265,16 +2265,6 @@ void __init setup_virt_paging(void)
>      /* Setup Stage 2 address translation */
>      register_t val = VTCR_RES1|VTCR_SH0_IS|VTCR_ORGN0_WBWA|VTCR_IRGN0_WBWA;
> 
> -#ifdef CONFIG_ARM_32
> -    if ( p2m_ipa_bits < 40 )
> -        panic("P2M: Not able to support %u-bit IPA at the moment\n",
> -              p2m_ipa_bits);
> -
> -    printk("P2M: 40-bit IPA\n");
> -    p2m_ipa_bits = 40;
> -    val |= VTCR_T0SZ(0x18); /* 40 bit IPA */
> -    val |= VTCR_SL0(0x1); /* P2M starts at first level */
> -#else /* CONFIG_ARM_64 */
>      static const struct {
>          unsigned int pabits; /* Physical Address Size */
>          unsigned int t0sz;   /* Desired T0SZ, minimum in comment */
> @@ -2283,19 +2273,24 @@ void __init setup_virt_paging(void)
>      } pa_range_info[] __initconst = {
>          /* T0SZ minimum and SL0 maximum from ARM DDI 0487H.a Table D5-6 */
>          /*      PA size, t0sz(min), root-order, sl0(max) */
> -        [0] = { 32,      32/*32*/,  0,          1 },
> -        [1] = { 36,      28/*28*/,  0,          1 },
> -        [2] = { 40,      24/*24*/,  1,          1 },
> +        [0] = { 40,      24/*24*/,  1,          1 },
Something does not add up here.
This table maintains the same order as PARange field of MMFR0 register, so that later on
we can do:
pa_range_info[system_cpuinfo.mm64.pa_range]

When PARange is 0, the PA size is 32, 1 -> 36 and so on.
However, you modified this behavior and now, if PARange is 0, this table will return PA size as 40.
This is wrong.

~Michal

Re: [XEN v5 07/10] xen/arm: Restrict zeroeth_table_offset for ARM_64

[Julien Grall]

Hi,

On 21/04/2023 09:03, Michal Orzel wrote:
> On 13/04/2023 19:37, Ayan Kumar Halder wrote:
>>
>>
>> When 32 bit physical addresses are used (ie PHYS_ADDR_T_32=y),
>> "va >> ZEROETH_SHIFT" causes an overflow.
>> Also, there is no zeroeth level page table on Arm32.
>>
>> Also took the opportunity to clean up dump_pt_walk(). One could use
>> DECLARE_OFFSETS() macro instead of declaring the declaring an array
> s/declaring the declaring/declaring/
> 
>> of page table offsets.
>>
>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> Reviewed-by: Michal Orzel <michal.orzel@amd.com>

Acked-by: Julien Grall <jgrall@amazon.com>

Cheers,

-- 
Julien Grall

Re: [XEN v5 08/10] xen/arm: domain_build: Check if the address fits the range of physical address

[Julien Grall]

Hi,

On 13/04/2023 18:37, Ayan Kumar Halder wrote:
> handle_pci_range() and map_range_to_domain() take addr and len as uint64_t
> parameters. Then frame numbers are obtained from addr and len by right shifting
> with PAGE_SHIFT. The page frame numbers are saved using unsigned long.
> 
> Now if 64-bit >> PAGE_SHIFT, the result will have 52-bits as valid. On a 32-bit
> system, 'unsigned long' is 32-bits. Thus, there is a potential loss of value
> when the result is stored as 'unsigned long'.
> 
> To mitigate this issue, we check if the starting and end address can be
> contained within the range of physical address supported on the system. If not,
> then an appropriate error is returned.
> 
> Also, the end address is computed once and used when required. And replaced u64
> with uint64_t.
> 
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> ---
> 
> Changes from :-
> v1...v4 - NA. New patch introduced in v5.
> 
>   xen/arch/arm/domain_build.c | 30 +++++++++++++++++++++++-------
>   1 file changed, 23 insertions(+), 7 deletions(-)
> 
> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> index 7d28b75517..b98ee506a8 100644
> --- a/xen/arch/arm/domain_build.c
> +++ b/xen/arch/arm/domain_build.c
> @@ -1637,15 +1637,23 @@ out:
>   }
>   
>   static int __init handle_pci_range(const struct dt_device_node *dev,
> -                                   u64 addr, u64 len, void *data)
> +                                   uint64_t addr, uint64_t len, void *data)
>   {
>       struct rangeset *mem_holes = data;
>       paddr_t start, end;
>       int res;
> +    uint64_t end_addr = addr + len - 1;

I find the difference between end and end_addr a bit confusing. How about...

> +
> +    if ( addr != (paddr_t)addr || end_addr != (paddr_t)end_addr )

... replace the second part with (((paddr_t)~0 - addr) > len))

> +    {
> +        printk(XENLOG_ERR "addr (0x%"PRIx64") or end_addr (0x%"PRIx64") exceeds the maximum allowed width (%d bits) for physical address\n",

In addition to what Michal says, I would replace the "addr .... 
end_addr" with "[start, end]" to further reduce the message.

Also, please use %u rather than %d as the number of bits cannot be negative.

> +               addr, end_addr, CONFIG_PADDR_BITS);
> +        return -ERANGE;
> +    }
>   
>       start = addr & PAGE_MASK;
> -    end = PAGE_ALIGN(addr + len);
> -    res = rangeset_remove_range(mem_holes, PFN_DOWN(start), PFN_DOWN(end - 1));
> +    end = PAGE_ALIGN(end_addr);
> +    res = rangeset_remove_range(mem_holes, PFN_DOWN(start), PFN_DOWN(end));

And this will not need to be changed.

>       if ( res )
>       {
>           printk(XENLOG_ERR "Failed to remove: %#"PRIpaddr"->%#"PRIpaddr"\n",
> @@ -2330,11 +2338,19 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
>   }
>   
>   int __init map_range_to_domain(const struct dt_device_node *dev,
> -                               u64 addr, u64 len, void *data)
> +                               uint64_t addr, uint64_t len, void *data)
>   {
My comment on the previous function applies for this one as well.

Cheers,

-- 
Julien Grall

Re: [XEN v5 08/10] xen/arm: domain_build: Check if the address fits the range of physical address

[Julien Grall]

On 24/04/2023 08:44, Michal Orzel wrote:
> Hi Ayan,
> 
> On 13/04/2023 19:37, Ayan Kumar Halder wrote:
>>
>>
>> handle_pci_range() and map_range_to_domain() take addr and len as uint64_t
>> parameters. Then frame numbers are obtained from addr and len by right shifting
>> with PAGE_SHIFT. The page frame numbers are saved using unsigned long.
> Maybe better to say "expressed" rather than "saved"
> 
>>
>> Now if 64-bit >> PAGE_SHIFT, the result will have 52-bits as valid. On a 32-bit
>> system, 'unsigned long' is 32-bits. Thus, there is a potential loss of value
>> when the result is stored as 'unsigned long'.
>>
>> To mitigate this issue, we check if the starting and end address can be
>> contained within the range of physical address supported on the system. If not,
>> then an appropriate error is returned.
>>
>> Also, the end address is computed once and used when required. And replaced u64
>> with uint64_t.
>>
>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
>> ---
>>
>> Changes from :-
>> v1...v4 - NA. New patch introduced in v5.
>>
>>   xen/arch/arm/domain_build.c | 30 +++++++++++++++++++++++-------
>>   1 file changed, 23 insertions(+), 7 deletions(-)
>>
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 7d28b75517..b98ee506a8 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -1637,15 +1637,23 @@ out:
>>   }
>>
>>   static int __init handle_pci_range(const struct dt_device_node *dev,
>> -                                   u64 addr, u64 len, void *data)
>> +                                   uint64_t addr, uint64_t len, void *data)
>>   {
>>       struct rangeset *mem_holes = data;
>>       paddr_t start, end;
>>       int res;
>> +    uint64_t end_addr = addr + len - 1;
>> +
>> +    if ( addr != (paddr_t)addr || end_addr != (paddr_t)end_addr )
>> +    {
>> +        printk(XENLOG_ERR "addr (0x%"PRIx64") or end_addr (0x%"PRIx64") exceeds the maximum allowed width (%d bits) for physical address\n",
> I don't think it is wise to print variable names (end_addr) to the user. Better would be to say explicitly: start, end address.
> Also to make the message shorter you could write: ... exceeds the maximum allowed PA width (%u bits)
> 
>> +               addr, end_addr, CONFIG_PADDR_BITS);
> Why CONFIG_PADDR_BITS if you already introduced PADDR_BITS macro
> 
>> +        return -ERANGE;
>> +    }
>>
>>       start = addr & PAGE_MASK;
>> -    end = PAGE_ALIGN(addr + len);
>> -    res = rangeset_remove_range(mem_holes, PFN_DOWN(start), PFN_DOWN(end - 1));
>> +    end = PAGE_ALIGN(end_addr);
>> +    res = rangeset_remove_range(mem_holes, PFN_DOWN(start), PFN_DOWN(end));
> I doubt PFN_DOWN(end) is the same as PFN_DOWN(end - 1), so I think you should keep the behavior as it was
> 
>>       if ( res )
>>       {
>>           printk(XENLOG_ERR "Failed to remove: %#"PRIpaddr"->%#"PRIpaddr"\n",
>> @@ -2330,11 +2338,19 @@ static int __init map_dt_irq_to_domain(const struct dt_device_node *dev,
>>   }
>>
>>   int __init map_range_to_domain(const struct dt_device_node *dev,
>> -                               u64 addr, u64 len, void *data)
>> +                               uint64_t addr, uint64_t len, void *data)
> You changed u64 to uint64_t in a definition but not in a prototype. Please fix.

This function is used as a callback. So if you want to switch from u64 
to uint64_t then you should also update dt_for_each_range().

Such changes would need to be done in a separate patch.

Cheers,

-- 
Julien Grall

Re: [XEN v5 05/10] xen/arm: Introduce choice to enable 64/32 bit physical addressing

[Michal Orzel]

Hi Ayan,

On 13/04/2023 19:37, Ayan Kumar Halder wrote:
> 
> 
> Some Arm based hardware platforms which does not support LPAE
> (eg Cortex-R52), uses 32 bit physical addresses.
> Also, users may choose to use 32 bits to represent physical addresses
> for optimization.
> 
> To support the above use cases, we have introduced arch independent
> configs to choose if the physical address can be represented using
> 32 bits (PHYS_ADDR_T_32) or 64 bits (!PHYS_ADDR_T_32).
> For now only ARM_32 provides support to enable 32 bit physical
> addressing.
> 
> When PHYS_ADDR_T_32 is defined, PADDR_BITS is set to 32.
> When PHYS_ADDR_T_32 is not defined for ARM_32, PADDR_BITS is set to 40.
> When PHYS_ADDR_T_32 is not defined for ARM_64, PADDR_BITS is set to 48.
> The last two are same as the current configuration used today on Xen.
> 
> PADDR_BITS is also set to 48 when ARM_64 is defined. The reason being
> the choice to select ARM_PA_BITS_32/ARM_PA_BITS_40/ARM_PA_BITS_48 is
> currently allowed when ARM_32 is defined.
> 
> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
> ---
> Changes from -
> v1 - 1. Extracted from "[XEN v1 8/9] xen/arm: Other adaptations required to support 32bit paddr".
> 
> v2 - 1. Introduced Kconfig choice. ARM_64 can select PHYS_ADDR_64 only whereas
> ARM_32 can select PHYS_ADDR_32 or PHYS_ADDR_64.
> 2. For CONFIG_ARM_PA_32, paddr_t is defined as 'unsigned long'.
> 
> v3 - 1. Allow user to define PADDR_BITS by selecting different config options
> ARM_PA_BITS_32, ARM_PA_BITS_40 and ARM_PA_BITS_48.
> 2. Add the choice under "Architecture Features".
> 
> v4 - 1. Removed PHYS_ADDR_T_64 as !PHYS_ADDR_T_32 means PHYS_ADDR_T_32.
> 
>  xen/arch/Kconfig                     |  3 +++
>  xen/arch/arm/Kconfig                 | 37 ++++++++++++++++++++++++++--
>  xen/arch/arm/include/asm/page-bits.h |  6 +----
>  xen/arch/arm/include/asm/types.h     |  6 +++++
>  xen/arch/arm/mm.c                    |  5 ++++
>  5 files changed, 50 insertions(+), 7 deletions(-)
> 
> diff --git a/xen/arch/Kconfig b/xen/arch/Kconfig
> index 7028f7b74f..67ba38f32f 100644
> --- a/xen/arch/Kconfig
> +++ b/xen/arch/Kconfig
> @@ -1,6 +1,9 @@
>  config 64BIT
>         bool
> 
> +config PHYS_ADDR_T_32
> +       bool
> +
>  config NR_CPUS
>         int "Maximum number of CPUs"
>         range 1 4095
> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
> index 239d3aed3c..3f6e13e475 100644
> --- a/xen/arch/arm/Kconfig
> +++ b/xen/arch/arm/Kconfig
> @@ -19,13 +19,46 @@ config ARM
>         select HAS_PMAP
>         select IOMMU_FORCE_PT_SHARE
> 
> +menu "Architecture Features"
> +
> +choice
> +       prompt "Physical address space size" if ARM_32
Why is it protected by ARM_32 but in the next line you add something protected by ARM_64?
This basically means that for arm64, ARM_PA_BITS_XXX is never defined.

> +       default ARM_PA_BITS_48 if ARM_64
> +       default ARM_PA_BITS_40 if ARM_32
> +       help
> +         User can choose to represent the width of physical address. This can
> +         sometimes help in optimizing the size of image when user chooses a
> +         smaller size to represent physical address.
> +
> +config ARM_PA_BITS_32
> +       bool "32-bit"
> +       help
> +         On platforms where any physical address can be represented within 32 bits,
> +         user should choose this option. This will help is reduced size of the
> +         binary.
> +       select PHYS_ADDR_T_32
> +       depends on ARM_32
> +
> +config ARM_PA_BITS_40
> +       bool "40-bit"
> +       depends on ARM_32
> +
> +config ARM_PA_BITS_48
> +       bool "40-bit"
40-bit? I think this should be 48-bit.

> +       depends on ARM_48
What is ARM_48? Shouldn't it be ARM_64?
And if so, why bother defining it given everything here is protected by ARM_32.

> +endchoice
> +
> +config PADDR_BITS
> +       int
> +       default 32 if ARM_PA_BITS_32
> +       default 40 if ARM_PA_BITS_40
> +       default 48 if ARM_PA_BITS_48 || ARM_64
This reads as if on arm32 we could have 48-bit PA space which is not true (LPAE is 40 bit unless I missed something).
You could get rid of || ARM_64 if the choice wasn't protected by ARM_32 and fixing ARM_48 to ARM_64.

> +
>  config ARCH_DEFCONFIG
>         string
>         default "arch/arm/configs/arm32_defconfig" if ARM_32
>         default "arch/arm/configs/arm64_defconfig" if ARM_64
> 
> -menu "Architecture Features"
> -
>  source "arch/Kconfig"
> 
>  config ACPI
> diff --git a/xen/arch/arm/include/asm/page-bits.h b/xen/arch/arm/include/asm/page-bits.h
> index 5d6477e599..deb381ceeb 100644
> --- a/xen/arch/arm/include/asm/page-bits.h
> +++ b/xen/arch/arm/include/asm/page-bits.h
> @@ -3,10 +3,6 @@
> 
>  #define PAGE_SHIFT              12
> 
> -#ifdef CONFIG_ARM_64
> -#define PADDR_BITS              48
> -#else
> -#define PADDR_BITS              40
> -#endif
> +#define PADDR_BITS              CONFIG_PADDR_BITS
> 
>  #endif /* __ARM_PAGE_SHIFT_H__ */
> diff --git a/xen/arch/arm/include/asm/types.h b/xen/arch/arm/include/asm/types.h
> index e218ed77bd..e3cfbbb060 100644
> --- a/xen/arch/arm/include/asm/types.h
> +++ b/xen/arch/arm/include/asm/types.h
> @@ -34,9 +34,15 @@ typedef signed long long s64;
>  typedef unsigned long long u64;
>  typedef u32 vaddr_t;
>  #define PRIvaddr PRIx32
> +#if defined(CONFIG_PHYS_ADDR_T_32)
> +typedef unsigned long paddr_t;
> +#define INVALID_PADDR (~0UL)
> +#define PRIpaddr "08lx"
> +#else
>  typedef u64 paddr_t;
>  #define INVALID_PADDR (~0ULL)
>  #define PRIpaddr "016llx"
> +#endif
>  typedef u32 register_t;
>  #define PRIregister "08x"
>  #elif defined (CONFIG_ARM_64)
> diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
> index b99806af99..6dc37be97e 100644
> --- a/xen/arch/arm/mm.c
> +++ b/xen/arch/arm/mm.c
> @@ -690,6 +690,11 @@ void __init setup_frametable_mappings(paddr_t ps, paddr_t pe)
>      const unsigned long mapping_size = frametable_size < MB(32) ? MB(2) : MB(32);
>      int rc;
> 
> +    /*
> +     * The size of paddr_t should be sufficient for the complete range of
> +     * physical address.
> +     */
> +    BUILD_BUG_ON((sizeof(paddr_t) * 8) < PADDR_BITS);
Just FYI, there is a macro BITS_PER_BYTE defined in bitops.h that you could use instead of 8.
Although I'm not sure if this would be better :)

>      BUILD_BUG_ON(sizeof(struct page_info) != PAGE_INFO_SIZE);
> 
>      if ( frametable_size > FRAMETABLE_SIZE )
> --
> 2.17.1
> 
> 

~Michal

Re: [XEN v5 05/10] xen/arm: Introduce choice to enable 64/32 bit physical addressing

[Ayan Kumar Halder]

On 24/04/2023 13:08, Michal Orzel wrote:
> Hi Ayan,

Hi Michal,

A clarification.

>
> On 13/04/2023 19:37, Ayan Kumar Halder wrote:
>>
>> Some Arm based hardware platforms which does not support LPAE
>> (eg Cortex-R52), uses 32 bit physical addresses.
>> Also, users may choose to use 32 bits to represent physical addresses
>> for optimization.
>>
>> To support the above use cases, we have introduced arch independent
>> configs to choose if the physical address can be represented using
>> 32 bits (PHYS_ADDR_T_32) or 64 bits (!PHYS_ADDR_T_32).
>> For now only ARM_32 provides support to enable 32 bit physical
>> addressing.
>>
>> When PHYS_ADDR_T_32 is defined, PADDR_BITS is set to 32.
>> When PHYS_ADDR_T_32 is not defined for ARM_32, PADDR_BITS is set to 40.
>> When PHYS_ADDR_T_32 is not defined for ARM_64, PADDR_BITS is set to 48.
>> The last two are same as the current configuration used today on Xen.
>>
>> PADDR_BITS is also set to 48 when ARM_64 is defined. The reason being
>> the choice to select ARM_PA_BITS_32/ARM_PA_BITS_40/ARM_PA_BITS_48 is
>> currently allowed when ARM_32 is defined.
>>
>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
>> ---
>> Changes from -
>> v1 - 1. Extracted from "[XEN v1 8/9] xen/arm: Other adaptations required to support 32bit paddr".
>>
>> v2 - 1. Introduced Kconfig choice. ARM_64 can select PHYS_ADDR_64 only whereas
>> ARM_32 can select PHYS_ADDR_32 or PHYS_ADDR_64.
>> 2. For CONFIG_ARM_PA_32, paddr_t is defined as 'unsigned long'.
>>
>> v3 - 1. Allow user to define PADDR_BITS by selecting different config options
>> ARM_PA_BITS_32, ARM_PA_BITS_40 and ARM_PA_BITS_48.
>> 2. Add the choice under "Architecture Features".
>>
>> v4 - 1. Removed PHYS_ADDR_T_64 as !PHYS_ADDR_T_32 means PHYS_ADDR_T_32.
>>
>>   xen/arch/Kconfig                     |  3 +++
>>   xen/arch/arm/Kconfig                 | 37 ++++++++++++++++++++++++++--
>>   xen/arch/arm/include/asm/page-bits.h |  6 +----
>>   xen/arch/arm/include/asm/types.h     |  6 +++++
>>   xen/arch/arm/mm.c                    |  5 ++++
>>   5 files changed, 50 insertions(+), 7 deletions(-)
>>
>> diff --git a/xen/arch/Kconfig b/xen/arch/Kconfig
>> index 7028f7b74f..67ba38f32f 100644
>> --- a/xen/arch/Kconfig
>> +++ b/xen/arch/Kconfig
>> @@ -1,6 +1,9 @@
>>   config 64BIT
>>          bool
>>
>> +config PHYS_ADDR_T_32
>> +       bool
>> +
>>   config NR_CPUS
>>          int "Maximum number of CPUs"
>>          range 1 4095
>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
>> index 239d3aed3c..3f6e13e475 100644
>> --- a/xen/arch/arm/Kconfig
>> +++ b/xen/arch/arm/Kconfig
>> @@ -19,13 +19,46 @@ config ARM
>>          select HAS_PMAP
>>          select IOMMU_FORCE_PT_SHARE
>>
>> +menu "Architecture Features"
>> +
>> +choice
>> +       prompt "Physical address space size" if ARM_32
> Why is it protected by ARM_32 but in the next line you add something protected by ARM_64?
> This basically means that for arm64, ARM_PA_BITS_XXX is never defined.

Currently, I intentend to support 32-bit physical address configuration 
for ARM_32 only. So in case of ARM_32, user will have a choice between 
32-bit and 40-bit PA.

So, if it is ok with you, can I remove the below line and "config 
ARM_PA_BITS_48 ..." ?

Then ...

>
>> +       default ARM_PA_BITS_48 if ARM_64
>> +       default ARM_PA_BITS_40 if ARM_32
>> +       help
>> +         User can choose to represent the width of physical address. This can
>> +         sometimes help in optimizing the size of image when user chooses a
>> +         smaller size to represent physical address.
>> +
>> +config ARM_PA_BITS_32
>> +       bool "32-bit"
>> +       help
>> +         On platforms where any physical address can be represented within 32 bits,
>> +         user should choose this option. This will help is reduced size of the
>> +         binary.
>> +       select PHYS_ADDR_T_32
>> +       depends on ARM_32
>> +
>> +config ARM_PA_BITS_40
>> +       bool "40-bit"
>> +       depends on ARM_32
>> +
>> +config ARM_PA_BITS_48
>> +       bool "40-bit"
> 40-bit? I think this should be 48-bit.
>
>> +       depends on ARM_48
> What is ARM_48? Shouldn't it be ARM_64?
> And if so, why bother defining it given everything here is protected by ARM_32.
>
>> +endchoice
>> +
>> +config PADDR_BITS
>> +       int
>> +       default 32 if ARM_PA_BITS_32
>> +       default 40 if ARM_PA_BITS_40
>> +       default 48 if ARM_PA_BITS_48 || ARM_64

default 48 if ARM_64

- Ayan

> This reads as if on arm32 we could have 48-bit PA space which is not true (LPAE is 40 bit unless I missed something).
> You could get rid of || ARM_64 if the choice wasn't protected by ARM_32 and fixing ARM_48 to ARM_64.
>
>> +
>>   config ARCH_DEFCONFIG
>>          string
>>          default "arch/arm/configs/arm32_defconfig" if ARM_32
>>          default "arch/arm/configs/arm64_defconfig" if ARM_64
>>
>> -menu "Architecture Features"
>> -
>>   source "arch/Kconfig"
>>
>>   config ACPI
>> diff --git a/xen/arch/arm/include/asm/page-bits.h b/xen/arch/arm/include/asm/page-bits.h
>> index 5d6477e599..deb381ceeb 100644
>> --- a/xen/arch/arm/include/asm/page-bits.h
>> +++ b/xen/arch/arm/include/asm/page-bits.h
>> @@ -3,10 +3,6 @@
>>
>>   #define PAGE_SHIFT              12
>>
>> -#ifdef CONFIG_ARM_64
>> -#define PADDR_BITS              48
>> -#else
>> -#define PADDR_BITS              40
>> -#endif
>> +#define PADDR_BITS              CONFIG_PADDR_BITS
>>
>>   #endif /* __ARM_PAGE_SHIFT_H__ */
>> diff --git a/xen/arch/arm/include/asm/types.h b/xen/arch/arm/include/asm/types.h
>> index e218ed77bd..e3cfbbb060 100644
>> --- a/xen/arch/arm/include/asm/types.h
>> +++ b/xen/arch/arm/include/asm/types.h
>> @@ -34,9 +34,15 @@ typedef signed long long s64;
>>   typedef unsigned long long u64;
>>   typedef u32 vaddr_t;
>>   #define PRIvaddr PRIx32
>> +#if defined(CONFIG_PHYS_ADDR_T_32)
>> +typedef unsigned long paddr_t;
>> +#define INVALID_PADDR (~0UL)
>> +#define PRIpaddr "08lx"
>> +#else
>>   typedef u64 paddr_t;
>>   #define INVALID_PADDR (~0ULL)
>>   #define PRIpaddr "016llx"
>> +#endif
>>   typedef u32 register_t;
>>   #define PRIregister "08x"
>>   #elif defined (CONFIG_ARM_64)
>> diff --git a/xen/arch/arm/mm.c b/xen/arch/arm/mm.c
>> index b99806af99..6dc37be97e 100644
>> --- a/xen/arch/arm/mm.c
>> +++ b/xen/arch/arm/mm.c
>> @@ -690,6 +690,11 @@ void __init setup_frametable_mappings(paddr_t ps, paddr_t pe)
>>       const unsigned long mapping_size = frametable_size < MB(32) ? MB(2) : MB(32);
>>       int rc;
>>
>> +    /*
>> +     * The size of paddr_t should be sufficient for the complete range of
>> +     * physical address.
>> +     */
>> +    BUILD_BUG_ON((sizeof(paddr_t) * 8) < PADDR_BITS);
> Just FYI, there is a macro BITS_PER_BYTE defined in bitops.h that you could use instead of 8.
> Although I'm not sure if this would be better :)
>
>>       BUILD_BUG_ON(sizeof(struct page_info) != PAGE_INFO_SIZE);
>>
>>       if ( frametable_size > FRAMETABLE_SIZE )
>> --
>> 2.17.1
>>
>>
> ~Michal

Re: [XEN v5 08/10] xen/arm: domain_build: Check if the address fits the range of physical address

[Ayan Kumar Halder]

On 24/04/2023 09:26, Julien Grall wrote:
> Hi,
Hi Julien,
>
> On 13/04/2023 18:37, Ayan Kumar Halder wrote:
>> handle_pci_range() and map_range_to_domain() take addr and len as 
>> uint64_t
>> parameters. Then frame numbers are obtained from addr and len by 
>> right shifting
>> with PAGE_SHIFT. The page frame numbers are saved using unsigned long.
>>
>> Now if 64-bit >> PAGE_SHIFT, the result will have 52-bits as valid. 
>> On a 32-bit
>> system, 'unsigned long' is 32-bits. Thus, there is a potential loss 
>> of value
>> when the result is stored as 'unsigned long'.
>>
>> To mitigate this issue, we check if the starting and end address can be
>> contained within the range of physical address supported on the 
>> system. If not,
>> then an appropriate error is returned.
>>
>> Also, the end address is computed once and used when required. And 
>> replaced u64
>> with uint64_t.
>>
>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
>> ---
>>
>> Changes from :-
>> v1...v4 - NA. New patch introduced in v5.
>>
>>   xen/arch/arm/domain_build.c | 30 +++++++++++++++++++++++-------
>>   1 file changed, 23 insertions(+), 7 deletions(-)
>>
>> diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
>> index 7d28b75517..b98ee506a8 100644
>> --- a/xen/arch/arm/domain_build.c
>> +++ b/xen/arch/arm/domain_build.c
>> @@ -1637,15 +1637,23 @@ out:
>>   }
>>     static int __init handle_pci_range(const struct dt_device_node *dev,
>> -                                   u64 addr, u64 len, void *data)
>> +                                   uint64_t addr, uint64_t len, void 
>> *data)
>>   {
>>       struct rangeset *mem_holes = data;
>>       paddr_t start, end;
>>       int res;
>> +    uint64_t end_addr = addr + len - 1;
>
> I find the difference between end and end_addr a bit confusing. How 
> about...
>
>> +
>> +    if ( addr != (paddr_t)addr || end_addr != (paddr_t)end_addr )
>
> ... replace the second part with (((paddr_t)~0 - addr) > len))

It should be

if ( ... || (((paddr_t)~0 - addr) < len) )

{

/* print error */

}

>
>> +    {
>> +        printk(XENLOG_ERR "addr (0x%"PRIx64") or end_addr 
>> (0x%"PRIx64") exceeds the maximum allowed width (%d bits) for 
>> physical address\n",
>
> In addition to what Michal says, I would replace the "addr .... 
> end_addr" with "[start, end]" to further reduce the message.
>
> Also, please use %u rather than %d as the number of bits cannot be 
> negative.

I think this should be fine ?

printk(XENLOG_ERR "[%#"PRIx64", "#PRIx64"] exceeds the maximum allowed PA width (%u bits)", start, end, CONFIG_PADDR_BITS);

- Ayan

>
>> +               addr, end_addr, CONFIG_PADDR_BITS);
>> +        return -ERANGE;
>> +    }
>>         start = addr & PAGE_MASK;
>> -    end = PAGE_ALIGN(addr + len);
>> -    res = rangeset_remove_range(mem_holes, PFN_DOWN(start), 
>> PFN_DOWN(end - 1));
>> +    end = PAGE_ALIGN(end_addr);
>> +    res = rangeset_remove_range(mem_holes, PFN_DOWN(start), 
>> PFN_DOWN(end));
>
> And this will not need to be changed.
>
>>       if ( res )
>>       {
>>           printk(XENLOG_ERR "Failed to remove: 
>> %#"PRIpaddr"->%#"PRIpaddr"\n",
>> @@ -2330,11 +2338,19 @@ static int __init map_dt_irq_to_domain(const 
>> struct dt_device_node *dev,
>>   }
>>     int __init map_range_to_domain(const struct dt_device_node *dev,
>> -                               u64 addr, u64 len, void *data)
>> +                               uint64_t addr, uint64_t len, void *data)
>>   {
> My comment on the previous function applies for this one as well.
>
> Cheers,
>

Re: [XEN v5 05/10] xen/arm: Introduce choice to enable 64/32 bit physical addressing

[Michal Orzel]

Hi Ayan,

On 25/04/2023 17:16, Ayan Kumar Halder wrote:
> 
> On 24/04/2023 13:08, Michal Orzel wrote:
>> Hi Ayan,
> 
> Hi Michal,
> 
> A clarification.
> 
>>
>> On 13/04/2023 19:37, Ayan Kumar Halder wrote:
>>>
>>> Some Arm based hardware platforms which does not support LPAE
>>> (eg Cortex-R52), uses 32 bit physical addresses.
>>> Also, users may choose to use 32 bits to represent physical addresses
>>> for optimization.
>>>
>>> To support the above use cases, we have introduced arch independent
>>> configs to choose if the physical address can be represented using
>>> 32 bits (PHYS_ADDR_T_32) or 64 bits (!PHYS_ADDR_T_32).
>>> For now only ARM_32 provides support to enable 32 bit physical
>>> addressing.
>>>
>>> When PHYS_ADDR_T_32 is defined, PADDR_BITS is set to 32.
>>> When PHYS_ADDR_T_32 is not defined for ARM_32, PADDR_BITS is set to 40.
>>> When PHYS_ADDR_T_32 is not defined for ARM_64, PADDR_BITS is set to 48.
>>> The last two are same as the current configuration used today on Xen.
>>>
>>> PADDR_BITS is also set to 48 when ARM_64 is defined. The reason being
>>> the choice to select ARM_PA_BITS_32/ARM_PA_BITS_40/ARM_PA_BITS_48 is
>>> currently allowed when ARM_32 is defined.
>>>
>>> Signed-off-by: Ayan Kumar Halder <ayan.kumar.halder@amd.com>
>>> ---
>>> Changes from -
>>> v1 - 1. Extracted from "[XEN v1 8/9] xen/arm: Other adaptations required to support 32bit paddr".
>>>
>>> v2 - 1. Introduced Kconfig choice. ARM_64 can select PHYS_ADDR_64 only whereas
>>> ARM_32 can select PHYS_ADDR_32 or PHYS_ADDR_64.
>>> 2. For CONFIG_ARM_PA_32, paddr_t is defined as 'unsigned long'.
>>>
>>> v3 - 1. Allow user to define PADDR_BITS by selecting different config options
>>> ARM_PA_BITS_32, ARM_PA_BITS_40 and ARM_PA_BITS_48.
>>> 2. Add the choice under "Architecture Features".
>>>
>>> v4 - 1. Removed PHYS_ADDR_T_64 as !PHYS_ADDR_T_32 means PHYS_ADDR_T_32.
>>>
>>>   xen/arch/Kconfig                     |  3 +++
>>>   xen/arch/arm/Kconfig                 | 37 ++++++++++++++++++++++++++--
>>>   xen/arch/arm/include/asm/page-bits.h |  6 +----
>>>   xen/arch/arm/include/asm/types.h     |  6 +++++
>>>   xen/arch/arm/mm.c                    |  5 ++++
>>>   5 files changed, 50 insertions(+), 7 deletions(-)
>>>
>>> diff --git a/xen/arch/Kconfig b/xen/arch/Kconfig
>>> index 7028f7b74f..67ba38f32f 100644
>>> --- a/xen/arch/Kconfig
>>> +++ b/xen/arch/Kconfig
>>> @@ -1,6 +1,9 @@
>>>   config 64BIT
>>>          bool
>>>
>>> +config PHYS_ADDR_T_32
>>> +       bool
>>> +
>>>   config NR_CPUS
>>>          int "Maximum number of CPUs"
>>>          range 1 4095
>>> diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig
>>> index 239d3aed3c..3f6e13e475 100644
>>> --- a/xen/arch/arm/Kconfig
>>> +++ b/xen/arch/arm/Kconfig
>>> @@ -19,13 +19,46 @@ config ARM
>>>          select HAS_PMAP
>>>          select IOMMU_FORCE_PT_SHARE
>>>
>>> +menu "Architecture Features"
>>> +
>>> +choice
>>> +       prompt "Physical address space size" if ARM_32
>> Why is it protected by ARM_32 but in the next line you add something protected by ARM_64?
>> This basically means that for arm64, ARM_PA_BITS_XXX is never defined.
> 
> Currently, I intentend to support 32-bit physical address configuration 
> for ARM_32 only. So in case of ARM_32, user will have a choice between 
> 32-bit and 40-bit PA.
> 
> So, if it is ok with you, can I remove the below line and "config 
> ARM_PA_BITS_48 ..." ?
I'm ok with that. I'm also ok to keep ARM_PA_BITS_48 but with ARM_32 protection removed to that the option makes sense.
However, since there is no real choice for arm64, I think the former is better.

~Michal