[PATCH 0/3] GICv3: Bounds check redistributor accesses

[PATCH 0/3] GICv3: Bounds check redistributor accesses

[Punit Agrawal]

Hi,

While bringing up linux on a platform with GICv3, I ran into a kernel
crash (data abort) due to incorrectly sized GIC redistributor region
in the device tree.

Even though the firmware is expected to provide incorrect information,
crashing on the kernel's part when that is not the case is also not
ideal.

This series adds bounds checking to redistributor accesses. This
prevents the crash. But we can do one better - when the redistributor
region has been exhausted without encountering the last record the
user is warned of this situation. This can be useful for developers
during board bring-up.

Patch 3 is where the bulk of the checking is introduced. The patch is
a bit invasive but there doesn't seem to be a nicer way to isolate the
change given layout of the redistributor.

Feedback welcome.

Thanks,
Punit

Punit Agrawal (3):
  irqchip/gic-v3: Use resource structure to store redistributor regions
  irqchip/gic-v3: Report firmwware provided address in case of error
  irqchip/gic-v3: Bounds check redistributor accesses

 drivers/irqchip/irq-gic-v3.c | 80 +++++++++++++++++++++++++++++++++-----------
 1 file changed, 61 insertions(+), 19 deletions(-)

-- 
2.14.1

[PATCH 1/3] irqchip/gic-v3: Use resource structure to store redistributor regions

[Punit Agrawal]

We don't store the size of the redistributor region required to prevent
out of bounds accesses to incorrectly sized regions provided by the
firmware.

Instead of only storing the base address, store the redistributor region
as a resource structure. The resource structure will be used in
subsequent commits to add bounds checking to redistributor region
accesses.

Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3.c | 29 +++++++++++++++++++----------
 1 file changed, 19 insertions(+), 10 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index b5df99c6f680..8cb383b6e605 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -43,7 +43,7 @@
 
 struct redist_region {
 	void __iomem		*redist_base;
-	phys_addr_t		phys_base;
+	struct resource		res;
 	bool			single_redist;
 };
 
@@ -481,7 +481,7 @@ static int __gic_populate_rdist(struct redist_region *region, void __iomem *ptr)
 	if ((typer >> 32) == aff) {
 		u64 offset = ptr - region->redist_base;
 		gic_data_rdist_rd_base() = ptr;
-		gic_data_rdist()->phys_base = region->phys_base + offset;
+		gic_data_rdist()->phys_base = region->res.start + offset;
 
 		pr_info("CPU%d: found redistributor %lx region %d:%pa\n",
 			smp_processor_id(), mpidr,
@@ -1206,17 +1206,18 @@ static int __init gic_of_init(struct device_node *node, struct device_node *pare
 	}
 
 	for (i = 0; i < nr_redist_regions; i++) {
-		struct resource res;
+		struct resource *res;
 		int ret;
 
-		ret = of_address_to_resource(node, 1 + i, &res);
-		rdist_regs[i].redist_base = of_iomap(node, 1 + i);
+		res = &rdist_regs[i].res;
+
+		ret = of_address_to_resource(node, 1 + i, res);
+		rdist_regs[i].redist_base = ioremap(res->start, resource_size(res));
 		if (ret || !rdist_regs[i].redist_base) {
 			pr_err("%pOF: couldn't map region %d\n", node, i);
 			err = -ENODEV;
 			goto out_unmap_rdist;
 		}
-		rdist_regs[i].phys_base = res.start;
 	}
 
 	if (of_property_read_u64(node, "redistributor-stride", &redist_stride))
@@ -1256,11 +1257,11 @@ static struct
 } acpi_data __initdata;
 
 static void __init
-gic_acpi_register_redist(phys_addr_t phys_base, void __iomem *redist_base)
+gic_acpi_register_redist(struct resource *res, void __iomem *redist_base)
 {
 	static int count = 0;
 
-	acpi_data.redist_regs[count].phys_base = phys_base;
+	acpi_data.redist_regs[count].res = *res;
 	acpi_data.redist_regs[count].redist_base = redist_base;
 	acpi_data.redist_regs[count].single_redist = acpi_data.single_redist;
 	count++;
@@ -1273,6 +1274,10 @@ gic_acpi_parse_madt_redist(struct acpi_subtable_header *header,
 	struct acpi_madt_generic_redistributor *redist =
 			(struct acpi_madt_generic_redistributor *)header;
 	void __iomem *redist_base;
+	struct resource redist_res;
+
+	redist_res.start = redist->base_address;
+	redist_res.end = redist_res.start + redist->length - 1;
 
 	redist_base = ioremap(redist->base_address, redist->length);
 	if (!redist_base) {
@@ -1280,7 +1285,7 @@ gic_acpi_parse_madt_redist(struct acpi_subtable_header *header,
 		return -ENOMEM;
 	}
 
-	gic_acpi_register_redist(redist->base_address, redist_base);
+	gic_acpi_register_redist(&redist_res, redist_base);
 	return 0;
 }
 
@@ -1293,12 +1298,16 @@ gic_acpi_parse_madt_gicc(struct acpi_subtable_header *header,
 	u32 reg = readl_relaxed(acpi_data.dist_base + GICD_PIDR2) & GIC_PIDR2_ARCH_MASK;
 	u32 size = reg == GIC_PIDR2_ARCH_GICv4 ? SZ_64K * 4 : SZ_64K * 2;
 	void __iomem *redist_base;
+	struct resource res;
 
 	redist_base = ioremap(gicc->gicr_base_address, size);
 	if (!redist_base)
 		return -ENOMEM;
 
-	gic_acpi_register_redist(gicc->gicr_base_address, redist_base);
+	res.start = gicc->gicr_base_address;
+	res.end = res.start + size - 1;
+
+	gic_acpi_register_redist(&res, redist_base);
 	return 0;
 }
 
-- 
2.14.1

[PATCH 2/3] irqchip/gic-v3: Report firmwware provided address in case of error

[Punit Agrawal]

When a redistributor is not found at the address provided by the
firmware, report the firmware provided address to help user identify the
offending firmware data.

Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 8cb383b6e605..881d327c53fa 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -430,13 +430,14 @@ static int gic_iterate_rdists(int (*fn)(struct redist_region *, void __iomem *))
 
 	for (i = 0; i < gic_data.nr_redist_regions; i++) {
 		void __iomem *ptr = gic_data.redist_regions[i].redist_base;
+		struct resource *res = &gic_data.redist_regions[i].res;
 		u64 typer;
 		u32 reg;
 
 		reg = readl_relaxed(ptr + GICR_PIDR2) & GIC_PIDR2_ARCH_MASK;
 		if (reg != GIC_PIDR2_ARCH_GICv3 &&
 		    reg != GIC_PIDR2_ARCH_GICv4) { /* We're in trouble... */
-			pr_warn("No redistributor present @%p\n", ptr);
+			pr_warn("No redistributor present @%llx\n", res->start);
 			break;
 		}
 
-- 
2.14.1

[PATCH 3/3] irqchip/gic-v3: Bounds check redistributor accesses

[Punit Agrawal]

The kernel crashes while iterating over a redistributor that is
in-correctly sized by the platform firmware or doesn't contain the last
record.

Prevent the crash by checking accesses against the size of the region
provided by the firmware. While we are at it, warn the user about
incorrect region size.

Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3.c | 48 ++++++++++++++++++++++++++++++++++++--------
 1 file changed, 40 insertions(+), 8 deletions(-)

diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index 881d327c53fa..754d936c95e5 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -429,11 +429,21 @@ static int gic_iterate_rdists(int (*fn)(struct redist_region *, void __iomem *))
 	int i;
 
 	for (i = 0; i < gic_data.nr_redist_regions; i++) {
-		void __iomem *ptr = gic_data.redist_regions[i].redist_base;
 		struct resource *res = &gic_data.redist_regions[i].res;
-		u64 typer;
+		void __iomem *ptr, *base;
+		u64 typer, size, stride;
 		u32 reg;
 
+		ptr = base = gic_data.redist_regions[i].redist_base;
+		size = resource_size(res);
+
+		stride = gic_data.redist_stride ?: SZ_64K * 2;
+		if (ptr + stride > base + size) {
+			pr_warn("Insufficient size for redistributor region @%llx. Skipping\n",
+				res->start);
+			continue;
+		}
+
 		reg = readl_relaxed(ptr + GICR_PIDR2) & GIC_PIDR2_ARCH_MASK;
 		if (reg != GIC_PIDR2_ARCH_GICv3 &&
 		    reg != GIC_PIDR2_ARCH_GICv4) { /* We're in trouble... */
@@ -442,7 +452,28 @@ static int gic_iterate_rdists(int (*fn)(struct redist_region *, void __iomem *))
 		}
 
 		do {
+			/*
+			 * We can access GICR_TYPER as we have already
+			 * checked that we have atleast 128kB or
+			 * redist_stride
+			 */
 			typer = gic_read_typer(ptr + GICR_TYPER);
+			if (!gic_data.redist_stride &&
+			    (typer & GICR_TYPER_VLPIS)) {
+				/* VLPI_base + reserved page */
+				stride += SZ_64K * 2;
+
+				/*
+				 * We are larger than we thought, do
+				 * we still fit?
+				 */
+				if (ptr + stride > base + size) {
+					pr_warn("No last record found in redistributor region @%llx\n",
+						gic_data.redist_regions[i].res.start);
+					break;
+				}
+			}
+
 			ret = fn(gic_data.redist_regions + i, ptr);
 			if (!ret)
 				return 0;
@@ -450,12 +481,13 @@ static int gic_iterate_rdists(int (*fn)(struct redist_region *, void __iomem *))
 			if (gic_data.redist_regions[i].single_redist)
 				break;
 
-			if (gic_data.redist_stride) {
-				ptr += gic_data.redist_stride;
-			} else {
-				ptr += SZ_64K * 2; /* Skip RD_base + SGI_base */
-				if (typer & GICR_TYPER_VLPIS)
-					ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
+			ptr += stride;
+
+			stride = gic_data.redist_stride ?: SZ_64K * 2;
+			if (ptr + stride > base + size) {
+				pr_warn("No last record found in redistributor region @%llx\n",
+					gic_data.redist_regions[i].res.start);
+				break;
 			}
 		} while (!(typer & GICR_TYPER_LAST));
 	}
-- 
2.14.1

Re: [3/3] irqchip/gic-v3: Bounds check redistributor accesses

[Lokesh Vutla]

Hi All,

On Wednesday 11 October 2017 03:11 PM, Punit Agrawal wrote:
> The kernel crashes while iterating over a redistributor that is
> in-correctly sized by the platform firmware or doesn't contain the last
> record.
> 
> Prevent the crash by checking accesses against the size of the region
> provided by the firmware. While we are at it, warn the user about
> incorrect region size.
> 
> Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>

Sorry to bring up an old thread. Just wanted to check what is the status
on this series.

This will also be useful when we try to boot linux + hypervisor with
less number of cores than the SoC supports. For example:
- SoC has 4 cores and Linux tries to boot with 2 cores.
- then a type-2 hypervisor gets installed.
- Hypervisor tries to boot a VM with linux on core 1.

Now the VM boot will fail while it iterates over all the GICR regions
till GICR_TYPER is found. Hypervisor will trap any accesses to GICR
regions of any invalid cpus(cpu 2, cpu 3 in this case).

If the $patch is not the right approach, can you suggest on how to
handle the above scenario?

Thanks and regards,
Lokesh

> ---
>  drivers/irqchip/irq-gic-v3.c | 48 ++++++++++++++++++++++++++++++++++++--------
>  1 file changed, 40 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
> index 881d327c53fa..754d936c95e5 100644
> --- a/drivers/irqchip/irq-gic-v3.c
> +++ b/drivers/irqchip/irq-gic-v3.c
> @@ -429,11 +429,21 @@ static int gic_iterate_rdists(int (*fn)(struct redist_region *, void __iomem *))
>  	int i;
>  
>  	for (i = 0; i < gic_data.nr_redist_regions; i++) {
> -		void __iomem *ptr = gic_data.redist_regions[i].redist_base;
>  		struct resource *res = &gic_data.redist_regions[i].res;
> -		u64 typer;
> +		void __iomem *ptr, *base;
> +		u64 typer, size, stride;
>  		u32 reg;
>  
> +		ptr = base = gic_data.redist_regions[i].redist_base;
> +		size = resource_size(res);
> +
> +		stride = gic_data.redist_stride ?: SZ_64K * 2;
> +		if (ptr + stride > base + size) {
> +			pr_warn("Insufficient size for redistributor region @%llx. Skipping\n",
> +				res->start);
> +			continue;
> +		}
> +
>  		reg = readl_relaxed(ptr + GICR_PIDR2) & GIC_PIDR2_ARCH_MASK;
>  		if (reg != GIC_PIDR2_ARCH_GICv3 &&
>  		    reg != GIC_PIDR2_ARCH_GICv4) { /* We're in trouble... */
> @@ -442,7 +452,28 @@ static int gic_iterate_rdists(int (*fn)(struct redist_region *, void __iomem *))
>  		}
>  
>  		do {
> +			/*
> +			 * We can access GICR_TYPER as we have already
> +			 * checked that we have atleast 128kB or
> +			 * redist_stride
> +			 */
>  			typer = gic_read_typer(ptr + GICR_TYPER);
> +			if (!gic_data.redist_stride &&
> +			    (typer & GICR_TYPER_VLPIS)) {
> +				/* VLPI_base + reserved page */
> +				stride += SZ_64K * 2;
> +
> +				/*
> +				 * We are larger than we thought, do
> +				 * we still fit?
> +				 */
> +				if (ptr + stride > base + size) {
> +					pr_warn("No last record found in redistributor region @%llx\n",
> +						gic_data.redist_regions[i].res.start);
> +					break;
> +				}
> +			}
> +
>  			ret = fn(gic_data.redist_regions + i, ptr);
>  			if (!ret)
>  				return 0;
> @@ -450,12 +481,13 @@ static int gic_iterate_rdists(int (*fn)(struct redist_region *, void __iomem *))
>  			if (gic_data.redist_regions[i].single_redist)
>  				break;
>  
> -			if (gic_data.redist_stride) {
> -				ptr += gic_data.redist_stride;
> -			} else {
> -				ptr += SZ_64K * 2; /* Skip RD_base + SGI_base */
> -				if (typer & GICR_TYPER_VLPIS)
> -					ptr += SZ_64K * 2; /* Skip VLPI_base + reserved page */
> +			ptr += stride;
> +
> +			stride = gic_data.redist_stride ?: SZ_64K * 2;
> +			if (ptr + stride > base + size) {
> +				pr_warn("No last record found in redistributor region @%llx\n",
> +					gic_data.redist_regions[i].res.start);
> +				break;
>  			}
>  		} while (!(typer & GICR_TYPER_LAST));
>  	}
> 

Re: [3/3] irqchip/gic-v3: Bounds check redistributor accesses

[Marc Zyngier]

Hi Lokesh,

On 13/03/18 13:38, Lokesh Vutla wrote:
> Hi All,
> 
> On Wednesday 11 October 2017 03:11 PM, Punit Agrawal wrote:
>> The kernel crashes while iterating over a redistributor that is
>> in-correctly sized by the platform firmware or doesn't contain the last
>> record.
>>
>> Prevent the crash by checking accesses against the size of the region
>> provided by the firmware. While we are at it, warn the user about
>> incorrect region size.
>>
>> Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
>> Cc: Marc Zyngier <marc.zyngier@arm.com>
> 
> Sorry to bring up an old thread. Just wanted to check what is the status
> on this series.

So far, I wasn't inclined to merge it, as it only allowed you to detect
a broken system, as opposed to help with a working one.

> This will also be useful when we try to boot linux + hypervisor with
> less number of cores than the SoC supports. For example:
> - SoC has 4 cores and Linux tries to boot with 2 cores.
> - then a type-2 hypervisor gets installed.
> - Hypervisor tries to boot a VM with linux on core 1.
> 
> Now the VM boot will fail while it iterates over all the GICR regions
> till GICR_TYPER is found. Hypervisor will trap any accesses to GICR
> regions of any invalid cpus(cpu 2, cpu 3 in this case).

It you're passing the redistributors to a guest, you're doing something
terribly wrong. You're putting the guest in a position to do a DoS on
the hypervisor (disabling its timer interrupt, for example). Not the
greatest move. There is a number of other gotchas with this approach
(virtual interrupts, distributor virtualization...).

> If the $patch is not the right approach, can you suggest on how to
> handle the above scenario?

The proper way to handle this is to virtualize the distributor and
redistributor by trap/emulate. The only thing you can safely pass to a
guest is the CPU interface, either as system registers or in its MMIO
form (if you have the GICv2 compatibility interface).

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

Re: [3/3] irqchip/gic-v3: Bounds check redistributor accesses

[Nishanth Menon]

Marc,

On Tue, Mar 13, 2018 at 9:21 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> Hi Lokesh,
>
> On 13/03/18 13:38, Lokesh Vutla wrote:
>> Hi All,
>>
>> On Wednesday 11 October 2017 03:11 PM, Punit Agrawal wrote:
>>> The kernel crashes while iterating over a redistributor that is
>>> in-correctly sized by the platform firmware or doesn't contain the last
>>> record.
>>>
>>> Prevent the crash by checking accesses against the size of the region
>>> provided by the firmware. While we are at it, warn the user about
>>> incorrect region size.
>>>
>>> Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
>>> Cc: Marc Zyngier <marc.zyngier@arm.com>
>>
>> Sorry to bring up an old thread. Just wanted to check what is the status
>> on this series.
>
> So far, I wasn't inclined to merge it, as it only allowed you to detect
> a broken system, as opposed to help with a working one.

Is'nt that a good reason to have it? Why not help an error in dtb with
a debug helper than an obtuse crash to debug painfully?

>
>> This will also be useful when we try to boot linux + hypervisor with
>> less number of cores than the SoC supports. For example:
>> - SoC has 4 cores and Linux tries to boot with 2 cores.
>> - then a type-2 hypervisor gets installed.
>> - Hypervisor tries to boot a VM with linux on core 1.
>>
>> Now the VM boot will fail while it iterates over all the GICR regions
>> till GICR_TYPER is found. Hypervisor will trap any accesses to GICR
>> regions of any invalid cpus(cpu 2, cpu 3 in this case).
>
> It you're passing the redistributors to a guest, you're doing something
> terribly wrong. You're putting the guest in a position to do a DoS on
> the hypervisor (disabling its timer interrupt, for example). Not the
> greatest move. There is a number of other gotchas with this approach
> (virtual interrupts, distributor virtualization...).
>
>> If the $patch is not the right approach, can you suggest on how to
>> handle the above scenario?
>
> The proper way to handle this is to virtualize the distributor and
> redistributor by trap/emulate. The only thing you can safely pass to a
> guest is the CPU interface, either as system registers or in its MMIO
> form (if you have the GICv2 compatibility interface).
>

Dumb question: Would'nt a trap emulate be real expensive with hyp
context in v8 (all the context save/restore we'd have to do? (in the
context of discussion, GICV2 compat is disabled).


-- 
---
Regards,
Nishanth Menon

Re: [3/3] irqchip/gic-v3: Bounds check redistributor accesses

[Marc Zyngier]

On Tue, 13 Mar 2018 18:49:44 +0000,
Nishanth Menon wrote:
> 
> Marc,
> 
> On Tue, Mar 13, 2018 at 9:21 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > Hi Lokesh,
> >
> > On 13/03/18 13:38, Lokesh Vutla wrote:
> >> Hi All,
> >>
> >> On Wednesday 11 October 2017 03:11 PM, Punit Agrawal wrote:
> >>> The kernel crashes while iterating over a redistributor that is
> >>> in-correctly sized by the platform firmware or doesn't contain the last
> >>> record.
> >>>
> >>> Prevent the crash by checking accesses against the size of the region
> >>> provided by the firmware. While we are at it, warn the user about
> >>> incorrect region size.
> >>>
> >>> Signed-off-by: Punit Agrawal <punit.agrawal@arm.com>
> >>> Cc: Marc Zyngier <marc.zyngier@arm.com>
> >>
> >> Sorry to bring up an old thread. Just wanted to check what is the status
> >> on this series.
> >
> > So far, I wasn't inclined to merge it, as it only allowed you to detect
> > a broken system, as opposed to help with a working one.
> 
> Is'nt that a good reason to have it? Why not help an error in dtb with
> a debug helper than an obtuse crash to debug painfully?

The kernel is not in the business of validating DTBs. And we both know
that if we start papering over firmware bugs, these bugs become ABI,
and we live with them forever. If you want to validate DTBs, you
should use a tool that is actually validating it against your HW, and
not use the kernel as the validation tool.

> 
> >
> >> This will also be useful when we try to boot linux + hypervisor with
> >> less number of cores than the SoC supports. For example:
> >> - SoC has 4 cores and Linux tries to boot with 2 cores.
> >> - then a type-2 hypervisor gets installed.
> >> - Hypervisor tries to boot a VM with linux on core 1.
> >>
> >> Now the VM boot will fail while it iterates over all the GICR regions
> >> till GICR_TYPER is found. Hypervisor will trap any accesses to GICR
> >> regions of any invalid cpus(cpu 2, cpu 3 in this case).
> >
> > It you're passing the redistributors to a guest, you're doing something
> > terribly wrong. You're putting the guest in a position to do a DoS on
> > the hypervisor (disabling its timer interrupt, for example). Not the
> > greatest move. There is a number of other gotchas with this approach
> > (virtual interrupts, distributor virtualization...).
> >
> >> If the $patch is not the right approach, can you suggest on how to
> >> handle the above scenario?
> >
> > The proper way to handle this is to virtualize the distributor and
> > redistributor by trap/emulate. The only thing you can safely pass to a
> > guest is the CPU interface, either as system registers or in its MMIO
> > form (if you have the GICv2 compatibility interface).
> >
> 
> Dumb question: Would'nt a trap emulate be real expensive with hyp
> context in v8 (all the context save/restore we'd have to do? (in the
> context of discussion, GICV2 compat is disabled).

How often do you hit the distributor and redistributors? Why do you
need to context switch a distributor that you emulate (hint: you
don't). I suggest you look at how real life hypervisor works (there's
a pretty good one in the tree).

Thanks,

	M.

-- 
Jazz is not dead, it just smell funny.