[PATCH] irqchip/gic-v3-its: cap lpi_id_bits to reduce memory footprint

[PATCH] irqchip/gic-v3-its: cap lpi_id_bits to reduce memory footprint

[Jia He]

In commit fe8e93504ce8 ("irqchip/gic-v3-its: Use full range of LPIs"),
it removes the cap for lpi_id_bits. But it will cause more pointless
memory footprint.

There is a WARN_ON when my QDF2400 server boots up (pagesize is 4k)
============begin===============
[    0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[    0.000000] GICv3: Distributor has no Range Selector support
[    0.000000] GICv3: VLPI support, no direct LPI support
[    0.000000] ACPI: SRAT not present
[    0.000000] ITS [mem 0xff7efe0000-0xff7effffff]
[    0.000000] ITS@0x000000ff7efe0000: Using ITS number 0
[    0.000000] GIC: enabling workaround for ITS: QDF2400 erratum 0065
[    0.000000] ITS@0x000000ff7efe0000: allocated 524288 Devices
@179f000000 (indirect, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x000000ff7efe0000: allocated 8192 Interrupt
Collections @179f930000 (flat, esz 8, psz 64K, shr 1)
[    0.000000] ITS@0x000000ff7efe0000: allocated 65536 Virtual CPUs
@179f980000 (flat, esz 8, psz 64K, shr 1)
[    0.000000] WARNING: CPU: 0 PID: 0 at mm/page_alloc.c:4066
__alloc_pages_nodemask+0x2d8/0x1188
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.18.0+ #66
[    0.000000] pstate: 20000085 (nzCv daIf -PAN -UAO)
[    0.000000] pc : __alloc_pages_nodemask+0x2d8/0x1188
[    0.000000] lr : __alloc_pages_nodemask+0x134/0x1188
[    0.000000] sp : ffff0000091b3a30
[    0.000000] x29: ffff0000091b3a30 x28: 0000000000000000
[    0.000000] x27: ffff00000a045000 x26: 0000000000000000
[    0.000000] x25: 0000000000000000 x24: ffff0000091c1000
[    0.000000] x23: ffff0000091b3b98 x22: 000000000000000c
[    0.000000] x21: ffff0000082dc130 x20: 0000000000000001
[    0.000000] x19: 0000000000000000 x18: 000000000000003f
[    0.000000] x17: 0000000000000000 x16: 0000000000000000
[    0.000000] x15: ffffffffffffffff x14: 202c74616c662820
[    0.000000] x13: ffff000009c5f9e0 x12: 0000000000000077
[    0.000000] x11: 0000000000000078 x10: 0000000097110f47
[    0.000000] x9 : 0000000000000000 x8 : 00000000009fff3f
[    0.000000] x7 : 000000000000002b x6 : 000000000000000c
[    0.000000] x5 : 0000000000000001 x4 : 0000000000000000
[    0.000000] x3 : ffff000008fd1000 x2 : ffff000008fd1000
[    0.000000] x1 : ffff0000091c1000 x0 : 0000000000000000
[    0.000000] Call trace:
[    0.000000]  __alloc_pages_nodemask+0x2d8/0x1188
[    0.000000]  alloc_pages_current+0x8c/0xd8
[    0.000000]  its_allocate_prop_table+0x5c/0xb8
[    0.000000]  its_init+0x220/0x3c0
[    0.000000]  gic_init_bases+0x250/0x380
[    0.000000]  gic_acpi_init+0x16c/0x2a4
[    0.000000]  acpi_match_madt+0x50/0x88
[    0.000000]  acpi_table_parse_entries_array+0x180/0x204
[    0.000000]  acpi_table_parse_entries+0x60/0x84
[    0.000000]  acpi_table_parse_madt+0x40/0x4c
[    0.000000]  __acpi_probe_device_table+0x94/0xe8
[    0.000000]  irqchip_init+0x38/0x40
[    0.000000]  init_IRQ+0x70/0x9c
[    0.000000]  start_kernel+0x310/0x4c0
[    0.000000] irq event stamp: 0
[    0.000000] hardirqs last  enabled at (0): [<0000000000000000>]
(null)
[    0.000000] hardirqs last disabled at (0): [<0000000000000000>]
(null)
[    0.000000] softirqs last  enabled at (0): [<0000000000000000>]
(null)
[    0.000000] softirqs last disabled at (0): [<0000000000000000>]
(null)
[    0.000000] ---[ end trace 943781056d97862b ]---
============end============

In its_alloc_lpi_tables, lpi_id_bits is 24 in QDF2400. Then
its_allocate_prop_table will try to allocate 16M(order 12 if
pagesize=4k). Thus it causes the WARN_ON.

As said by Marc,
Capping lpi_id_bits at 16 (which is what we had before) is plenty,
will save a some memory, and gives some margin before we need to push
it up again.

This patch re-caps the lpi_id_bits.

Fixes: fe8e93504ce8 ("irqchip/gic-v3-its: Use full range of LPIs")
Signed-off-by: Jia He <jia.he@hxt-semitech.com>
Suggested-by: Marc Zyngier <marc.zyngier@arm.com>
---
 drivers/irqchip/irq-gic-v3-its.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 316a575..c2df341 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -1439,6 +1439,7 @@ static int its_irq_set_vcpu_affinity(struct irq_data *d, void *vcpu_info)
  * The consequence of the above is that allocation is cost is low, but
  * freeing is expensive. We assumes that freeing rarely occurs.
  */
+#define ITS_MAX_LPI_NRBITS	16 /* 64K LPIs */
 
 static DEFINE_MUTEX(lpi_range_lock);
 static LIST_HEAD(lpi_range_list);
@@ -1625,7 +1626,8 @@ static int __init its_alloc_lpi_tables(void)
 {
 	phys_addr_t paddr;
 
-	lpi_id_bits = GICD_TYPER_ID_BITS(gic_rdists->gicd_typer);
+	lpi_id_bits = min_t(u32, GICD_TYPER_ID_BITS(gic_rdists->gicd_typer),
+				ITS_MAX_LPI_NRBITS);
 	gic_rdists->prop_page = its_allocate_prop_table(GFP_NOWAIT);
 	if (!gic_rdists->prop_page) {
 		pr_err("Failed to allocate PROPBASE\n");
-- 
1.8.3.1

Re: [PATCH] irqchip/gic-v3-its: cap lpi_id_bits to reduce memory footprint

[Marc Zyngier]

On Tue, 28 Aug 2018 05:53:26 +0100,
Jia He <hejianet@gmail.com> wrote:
> 
> In commit fe8e93504ce8 ("irqchip/gic-v3-its: Use full range of LPIs"),
> it removes the cap for lpi_id_bits. But it will cause more pointless
> memory footprint.
> 
> There is a WARN_ON when my QDF2400 server boots up (pagesize is 4k)
> ============begin===============

[trimming not-so-useful trace]

> ============end============
> 
> In its_alloc_lpi_tables, lpi_id_bits is 24 in QDF2400. Then
> its_allocate_prop_table will try to allocate 16M(order 12 if
> pagesize=4k). Thus it causes the WARN_ON.
> 
> As said by Marc,
> Capping lpi_id_bits at 16 (which is what we had before) is plenty,
> will save a some memory, and gives some margin before we need to push
> it up again.
> 
> This patch re-caps the lpi_id_bits.
> 
> Fixes: fe8e93504ce8 ("irqchip/gic-v3-its: Use full range of LPIs")
> Signed-off-by: Jia He <jia.he@hxt-semitech.com>
> Suggested-by: Marc Zyngier <marc.zyngier@arm.com>

Thanks for doing this. Small problem with this patch:

The email comes from hejianet@gmail.com, while the sign off is by
jia.he@hxt-semitech.com. Your email should start with a:

	From: Jia He <jia.he@hxt-semitech.com>

Other than that:

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

Thomas, would you mind picking this up so that it gets into the next
convenient -rc?

Thanks,

	M. (/me goes back hiking...)
	

-- 
Jazz is not dead, it just smell funny.

Re: [PATCH] irqchip/gic-v3-its: cap lpi_id_bits to reduce memory footprint

[Jia He]

On 8/28/2018 4:58 PM, Marc Zyngier Wrote:
> On Tue, 28 Aug 2018 05:53:26 +0100,
> Jia He <hejianet@gmail.com> wrote:
>>
>> In commit fe8e93504ce8 ("irqchip/gic-v3-its: Use full range of LPIs"),
>> it removes the cap for lpi_id_bits. But it will cause more pointless
>> memory footprint.
>>
>> There is a WARN_ON when my QDF2400 server boots up (pagesize is 4k)
>> ============begin===============
> 
> [trimming not-so-useful trace]
> 
>> ============end============
>>
>> In its_alloc_lpi_tables, lpi_id_bits is 24 in QDF2400. Then
>> its_allocate_prop_table will try to allocate 16M(order 12 if
>> pagesize=4k). Thus it causes the WARN_ON.
>>
>> As said by Marc,
>> Capping lpi_id_bits at 16 (which is what we had before) is plenty,
>> will save a some memory, and gives some margin before we need to push
>> it up again.
>>
>> This patch re-caps the lpi_id_bits.
>>
>> Fixes: fe8e93504ce8 ("irqchip/gic-v3-its: Use full range of LPIs")
>> Signed-off-by: Jia He <jia.he@hxt-semitech.com>
>> Suggested-by: Marc Zyngier <marc.zyngier@arm.com>
> 
> Thanks for doing this. Small problem with this patch:
> 
> The email comes from hejianet@gmail.com, while the sign off is by
> jia.he@hxt-semitech.com. Your email should start with a:
> 
> 	From: Jia He <jia.he@hxt-semitech.com>
> 

Thanks for the pointing. And sorry for that problem.

---
Cheers,
Jia
> Other than that:
> 
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
> 
> Thomas, would you mind picking this up so that it gets into the next
> convenient -rc?
> 
> Thanks,
> 
> 	M. (/me goes back hiking...)
> 	
> 

Re: [PATCH] irqchip/gic-v3-its: cap lpi_id_bits to reduce memory footprint

[Olof Johansson]

On Tue, Aug 28, 2018 at 1:58 AM, Marc Zyngier <marc.zyngier@arm.com> wrote:
> On Tue, 28 Aug 2018 05:53:26 +0100,
> Jia He <hejianet@gmail.com> wrote:
>>
>> In commit fe8e93504ce8 ("irqchip/gic-v3-its: Use full range of LPIs"),
>> it removes the cap for lpi_id_bits. But it will cause more pointless
>> memory footprint.
>>
>> There is a WARN_ON when my QDF2400 server boots up (pagesize is 4k)
>> ============begin===============
>
> [trimming not-so-useful trace]
>
>> ============end============
>>
>> In its_alloc_lpi_tables, lpi_id_bits is 24 in QDF2400. Then
>> its_allocate_prop_table will try to allocate 16M(order 12 if
>> pagesize=4k). Thus it causes the WARN_ON.
>>
>> As said by Marc,
>> Capping lpi_id_bits at 16 (which is what we had before) is plenty,
>> will save a some memory, and gives some margin before we need to push
>> it up again.
>>
>> This patch re-caps the lpi_id_bits.
>>
>> Fixes: fe8e93504ce8 ("irqchip/gic-v3-its: Use full range of LPIs")
>> Signed-off-by: Jia He <jia.he@hxt-semitech.com>
>> Suggested-by: Marc Zyngier <marc.zyngier@arm.com>
>
> Thanks for doing this. Small problem with this patch:
>
> The email comes from hejianet@gmail.com, while the sign off is by
> jia.he@hxt-semitech.com. Your email should start with a:
>
>         From: Jia He <jia.he@hxt-semitech.com>
>
> Other than that:
>
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>

Tested-by: Olof Johansson <olof@lixom.net>

> Thomas, would you mind picking this up so that it gets into the next
> convenient -rc?

Yeah, it'd be great to see this go into -rc2 if possible. Thanks!


-Olof

[tip:irq/urgent] irqchip/gic-v3-its: Cap lpi_id_bits to reduce memory footprint

[tip-bot for Jia He]

Commit-ID:  4cb205c0c50f613e2de91f0eb19d5247ed003e89
Gitweb:     https://git.kernel.org/tip/4cb205c0c50f613e2de91f0eb19d5247ed003e89
Author:     Jia He <jia.he@hxt-semitech.com>
AuthorDate: Tue, 28 Aug 2018 12:53:26 +0800
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Thu, 6 Sep 2018 20:31:59 +0200

irqchip/gic-v3-its: Cap lpi_id_bits to reduce memory footprint

Commit fe8e93504ce8 ("irqchip/gic-v3-its: Use full range of LPIs"), removes
the cap for lpi_id_bits, which causes the following warning to trigger on a
QDF2400 server:

 WARNING: CPU: 0 PID: 0 at mm/page_alloc.c:4066  __alloc_pages_nodemask
 ... 
 Call trace:
  __alloc_pages_nodemask+0x2d8/0x1188
  alloc_pages_current+0x8c/0xd8
  its_allocate_prop_table+0x5c/0xb8
  its_init+0x220/0x3c0
  gic_init_bases+0x250/0x380
  gic_acpi_init+0x16c/0x2a4

In its_alloc_lpi_tables(), lpi_id_bits is 24 in QDF2400. The allocation in
allocate_prop_table() tries therefore to allocate 16M (order 12 if
pagesize=4k), which triggers the warning.

As said by MarcL

 Capping lpi_id_bits at 16 (which is what we had before) is plenty,
 will save a some memory, and gives some margin before we need to push
 it up again.

Bring the upper limit of lpi_id_bits back to prevent

Fixes: fe8e93504ce8 ("irqchip/gic-v3-its: Use full range of LPIs")
Suggested-by: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Jia He <jia.he@hxt-semitech.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
Tested-by: Olof Johansson <olof@lixom.net>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: linux-arm-kernel@lists.infradead.org
Link: https://lkml.kernel.org/r/1535432006-2304-1-git-send-email-jia.he@hxt-semitech.com

---
 drivers/irqchip/irq-gic-v3-its.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 316a57530f6d..c2df341ff6fa 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -1439,6 +1439,7 @@ static struct irq_chip its_irq_chip = {
  * The consequence of the above is that allocation is cost is low, but
  * freeing is expensive. We assumes that freeing rarely occurs.
  */
+#define ITS_MAX_LPI_NRBITS	16 /* 64K LPIs */
 
 static DEFINE_MUTEX(lpi_range_lock);
 static LIST_HEAD(lpi_range_list);
@@ -1625,7 +1626,8 @@ static int __init its_alloc_lpi_tables(void)
 {
 	phys_addr_t paddr;
 
-	lpi_id_bits = GICD_TYPER_ID_BITS(gic_rdists->gicd_typer);
+	lpi_id_bits = min_t(u32, GICD_TYPER_ID_BITS(gic_rdists->gicd_typer),
+				ITS_MAX_LPI_NRBITS);
 	gic_rdists->prop_page = its_allocate_prop_table(GFP_NOWAIT);
 	if (!gic_rdists->prop_page) {
 		pr_err("Failed to allocate PROPBASE\n");