* [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT
@ 2021-10-27 15:15 Valentin Schneider
2021-10-27 15:15 ` [PATCH v2 1/3] irqchip/gic-v3-its: Give the percpu rdist struct its own flags field Valentin Schneider
` (3 more replies)
0 siblings, 4 replies; 9+ messages in thread
From: Valentin Schneider @ 2021-10-27 15:15 UTC (permalink / raw)
To: linux-kernel, linux-rt-users, linux-arm-kernel
Cc: Will Deacon, Mark Rutland, Marc Zyngier, Thomas Gleixner,
Sebastian Andrzej Siewior, Ard Biesheuvel
Hi folks,
This is my take at fixing [1]. Reading about the LPI tables situation was
entertaining.
Tested against kexec on an Ampere eMAG. Seems to be working fine atop
5.15-rc6. On the other hand, I can only issue one kexec from
5.15-rc6-rt12 - if I then issue another one on the new kernel, I get tasks
hanging. That is true even without my patches and without
CONFIG_PREEMPT_RT.
[1]: http://lore.kernel.org/r/20210810134127.1394269-3-valentin.schneider@arm.com
Revisions
=========
v1 -> v2
++++++++
o Ditched RDIST_FLAGS renaming; moved to "RD_LOCAL" prefix for new flags
(Marc)
o Simplified cpuhp callback logic: body now runs at most once per CPU
o Changed cpuhp callback installation to happen *after* the BP has invoked
its_cpu_init(); the BP cpuhp state is immediately set to CPUHP_ONLINE, so
cpuhp state alone doesn't guarantee its_cpu_init() has been run.
Cheers,
Valentin
Valentin Schneider (3):
irqchip/gic-v3-its: Give the percpu rdist struct its own flags field
irqchip/gic-v3-its: Postpone LPI pending table freeing and memreserve
irqchip/gic-v3-its: Limit memreserve cpuhp state lifetime
drivers/irqchip/irq-gic-v3-its.c | 82 +++++++++++++++++++++++++++---
drivers/irqchip/irq-gic-v3.c | 1 +
include/linux/irqchip/arm-gic-v3.h | 4 +-
3 files changed, 79 insertions(+), 8 deletions(-)
--
2.25.1
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v2 1/3] irqchip/gic-v3-its: Give the percpu rdist struct its own flags field
2021-10-27 15:15 [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT Valentin Schneider
@ 2021-10-27 15:15 ` Valentin Schneider
2021-12-16 13:30 ` [irqchip: irq/irqchip-next] " irqchip-bot for Valentin Schneider
2021-10-27 15:15 ` [PATCH v2 2/3] irqchip/gic-v3-its: Postpone LPI pending table freeing and memreserve Valentin Schneider
` (2 subsequent siblings)
3 siblings, 1 reply; 9+ messages in thread
From: Valentin Schneider @ 2021-10-27 15:15 UTC (permalink / raw)
To: linux-kernel, linux-rt-users, linux-arm-kernel
Cc: Will Deacon, Mark Rutland, Marc Zyngier, Thomas Gleixner,
Sebastian Andrzej Siewior, Ard Biesheuvel
Later patches will require tracking some per-rdist status. Reuse the bytes
"lost" to padding within the __percpu rdist struct as a flags field, and
re-encode ->lpi_enabled within said flags.
No change in functionality intended.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
drivers/irqchip/irq-gic-v3-its.c | 8 +++++---
include/linux/irqchip/arm-gic-v3.h | 2 +-
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index eb0882d15366..74c22741f3ce 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -46,6 +46,8 @@
#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)
#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1)
+#define RD_LOCAL_LPI_ENABLED BIT(0)
+
static u32 lpi_id_bits;
/*
@@ -3044,7 +3046,7 @@ static void its_cpu_init_lpis(void)
phys_addr_t paddr;
u64 val, tmp;
- if (gic_data_rdist()->lpi_enabled)
+ if (gic_data_rdist()->flags & RD_LOCAL_LPI_ENABLED)
return;
val = readl_relaxed(rbase + GICR_CTLR);
@@ -3158,7 +3160,7 @@ static void its_cpu_init_lpis(void)
/* Make sure the GIC has seen the above */
dsb(sy);
out:
- gic_data_rdist()->lpi_enabled = true;
+ gic_data_rdist()->flags |= RD_LOCAL_LPI_ENABLED;
pr_info("GICv3: CPU%d: using %s LPI pending table @%pa\n",
smp_processor_id(),
gic_data_rdist()->pend_page ? "allocated" : "reserved",
@@ -5138,7 +5140,7 @@ static int redist_disable_lpis(void)
*
* If running with preallocated tables, there is nothing to do.
*/
- if (gic_data_rdist()->lpi_enabled ||
+ if ((gic_data_rdist()->flags & RD_LOCAL_LPI_ENABLED) ||
(gic_rdists->flags & RDIST_FLAGS_RD_TABLES_PREALLOCATED))
return 0;
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index 81cbf85f73de..0dc34d7d735a 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -615,7 +615,7 @@ struct rdists {
void __iomem *rd_base;
struct page *pend_page;
phys_addr_t phys_base;
- bool lpi_enabled;
+ u64 flags;
cpumask_t *vpe_table_mask;
void *vpe_l1_base;
} __percpu *rdist;
--
2.25.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v2 2/3] irqchip/gic-v3-its: Postpone LPI pending table freeing and memreserve
2021-10-27 15:15 [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT Valentin Schneider
2021-10-27 15:15 ` [PATCH v2 1/3] irqchip/gic-v3-its: Give the percpu rdist struct its own flags field Valentin Schneider
@ 2021-10-27 15:15 ` Valentin Schneider
2021-12-16 13:30 ` [irqchip: irq/irqchip-next] " irqchip-bot for Valentin Schneider
2021-10-27 15:15 ` [PATCH v2 3/3] irqchip/gic-v3-its: Limit memreserve cpuhp state lifetime Valentin Schneider
2021-11-19 11:14 ` [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT Sebastian Andrzej Siewior
3 siblings, 1 reply; 9+ messages in thread
From: Valentin Schneider @ 2021-10-27 15:15 UTC (permalink / raw)
To: linux-kernel, linux-rt-users, linux-arm-kernel
Cc: Will Deacon, Mark Rutland, Marc Zyngier, Thomas Gleixner,
Sebastian Andrzej Siewior, Ard Biesheuvel
Memory used by the LPI tables have to be made persistent for kexec to have
a chance to work, as explained in [1]. If they have been made persistent
and we are booting into a kexec'd kernel, we also need to free the pages
that were preemptively allocated by the new kernel for those tables.
Both of those operations currently happen during its_cpu_init(), which
happens in a _STARTING (IOW atomic) cpuhp callback for secondary
CPUs. efi_mem_reserve_iomem() issues a GFP_ATOMIC allocation, which
unfortunately doesn't work under PREEMPT_RT (this ends up grabbing a
non-raw spinlock, which can sleep under PREEMPT_RT). Similarly, freeing the
pages ends up grabbing a sleepable spinlock.
Since the memreserve is only required by kexec, it doesn't have to be done
so early in the secondary boot process. Issue the reservation in a new
CPUHP_AP_ONLINE_DYN cpuhp callback, and piggy-back the page freeing on top
of it. A CPU gets to run the body of this new callback exactly once.
As kexec issues a machine_shutdown() prior to machine_kexec(), it will be
serialized vs a CPU being plugged to life by the hotplug machinery - either
the CPU will have been brought up and have had its redistributor's pending
table memreserved, or it never went online and will have its table
allocated by the new kernel.
[1]: https://lore.kernel.org/lkml/20180921195954.21574-1-marc.zyngier@arm.com/
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
drivers/irqchip/irq-gic-v3-its.c | 58 +++++++++++++++++++++++++++---
drivers/irqchip/irq-gic-v3.c | 1 +
include/linux/irqchip/arm-gic-v3.h | 1 +
3 files changed, 56 insertions(+), 4 deletions(-)
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 74c22741f3ce..f860733d3e4e 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -47,6 +47,8 @@
#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1)
#define RD_LOCAL_LPI_ENABLED BIT(0)
+#define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1)
+#define RD_LOCAL_MEMRESERVE_DONE BIT(2)
static u32 lpi_id_bits;
@@ -3065,15 +3067,13 @@ static void its_cpu_init_lpis(void)
paddr &= GENMASK_ULL(51, 16);
WARN_ON(!gic_check_reserved_range(paddr, LPI_PENDBASE_SZ));
- its_free_pending_table(gic_data_rdist()->pend_page);
- gic_data_rdist()->pend_page = NULL;
+ gic_data_rdist()->flags |= RD_LOCAL_PENDTABLE_PREALLOCATED;
goto out;
}
pend_page = gic_data_rdist()->pend_page;
paddr = page_to_phys(pend_page);
- WARN_ON(gic_reserve_range(paddr, LPI_PENDBASE_SZ));
/* set PROPBASE */
val = (gic_rdists->prop_table_pa |
@@ -3163,7 +3163,8 @@ static void its_cpu_init_lpis(void)
gic_data_rdist()->flags |= RD_LOCAL_LPI_ENABLED;
pr_info("GICv3: CPU%d: using %s LPI pending table @%pa\n",
smp_processor_id(),
- gic_data_rdist()->pend_page ? "allocated" : "reserved",
+ gic_data_rdist()->flags & RD_LOCAL_PENDTABLE_PREALLOCATED ?
+ "reserved" : "allocated",
&paddr);
}
@@ -5202,6 +5203,38 @@ int its_cpu_init(void)
return 0;
}
+static int its_cpu_memreserve_lpi(unsigned int cpu)
+{
+ struct page *pend_page;
+ int ret = 0;
+
+ /* This gets to run exactly once per CPU */
+ if (gic_data_rdist()->flags & RD_LOCAL_MEMRESERVE_DONE)
+ return 0;
+
+ pend_page = gic_data_rdist()->pend_page;
+ if (WARN_ON(!pend_page)) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ /*
+ * If the pending table was pre-programmed, free the memory we
+ * preemptively allocated. Otherwise, reserve that memory for
+ * later kexecs.
+ */
+ if (gic_data_rdist()->flags & RD_LOCAL_PENDTABLE_PREALLOCATED) {
+ its_free_pending_table(pend_page);
+ gic_data_rdist()->pend_page = NULL;
+ } else {
+ phys_addr_t paddr = page_to_phys(pend_page);
+ WARN_ON(gic_reserve_range(paddr, LPI_PENDBASE_SZ));
+ }
+
+out:
+ gic_data_rdist()->flags |= RD_LOCAL_MEMRESERVE_DONE;
+ return ret;
+}
+
static const struct of_device_id its_device_id[] = {
{ .compatible = "arm,gic-v3-its", },
{},
@@ -5385,6 +5418,23 @@ static void __init its_acpi_probe(void)
static void __init its_acpi_probe(void) { }
#endif
+int __init its_lpi_memreserve_init(void)
+{
+ int state;
+
+ if (!efi_enabled(EFI_CONFIG_TABLES))
+ return 0;
+
+ state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+ "irqchip/arm/gicv3/memreserve:online",
+ its_cpu_memreserve_lpi,
+ NULL);
+ if (state < 0)
+ return state;
+
+ return 0;
+}
+
int __init its_init(struct fwnode_handle *handle, struct rdists *rdists,
struct irq_domain *parent_domain)
{
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index fd4e9a37fea6..7a962767d835 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -1802,6 +1802,7 @@ static int __init gic_init_bases(void __iomem *dist_base,
if (gic_dist_supports_lpis()) {
its_init(handle, &gic_data.rdists, gic_data.domain);
its_cpu_init();
+ its_lpi_memreserve_init();
} else {
if (IS_ENABLED(CONFIG_ARM_GIC_V2M))
gicv2m_init(handle, gic_data.domain);
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index 0dc34d7d735a..51b85506ae90 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -632,6 +632,7 @@ struct rdists {
struct irq_domain;
struct fwnode_handle;
+int __init its_lpi_memreserve_init(void);
int its_cpu_init(void);
int its_init(struct fwnode_handle *handle, struct rdists *rdists,
struct irq_domain *domain);
--
2.25.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v2 3/3] irqchip/gic-v3-its: Limit memreserve cpuhp state lifetime
2021-10-27 15:15 [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT Valentin Schneider
2021-10-27 15:15 ` [PATCH v2 1/3] irqchip/gic-v3-its: Give the percpu rdist struct its own flags field Valentin Schneider
2021-10-27 15:15 ` [PATCH v2 2/3] irqchip/gic-v3-its: Postpone LPI pending table freeing and memreserve Valentin Schneider
@ 2021-10-27 15:15 ` Valentin Schneider
2021-12-16 13:30 ` [irqchip: irq/irqchip-next] " irqchip-bot for Valentin Schneider
2021-11-19 11:14 ` [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT Sebastian Andrzej Siewior
3 siblings, 1 reply; 9+ messages in thread
From: Valentin Schneider @ 2021-10-27 15:15 UTC (permalink / raw)
To: linux-kernel, linux-rt-users, linux-arm-kernel
Cc: Will Deacon, Mark Rutland, Marc Zyngier, Thomas Gleixner,
Sebastian Andrzej Siewior, Ard Biesheuvel
The new memreserve cpuhp callback only needs to survive up until a point
where every CPU in the system has booted once. Beyond that, it becomes a
no-op and can be put in the bin.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
drivers/irqchip/irq-gic-v3-its.c | 16 ++++++++++++++++
include/linux/irqchip/arm-gic-v3.h | 1 +
2 files changed, 17 insertions(+)
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index f860733d3e4e..ee83eb377d7e 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -5203,6 +5203,15 @@ int its_cpu_init(void)
return 0;
}
+static void rdist_memreserve_cpuhp_cleanup_workfn(struct work_struct *work)
+{
+ cpuhp_remove_state_nocalls(gic_rdists->cpuhp_memreserve_state);
+ gic_rdists->cpuhp_memreserve_state = CPUHP_INVALID;
+}
+
+static DECLARE_WORK(rdist_memreserve_cpuhp_cleanup_work,
+ rdist_memreserve_cpuhp_cleanup_workfn);
+
static int its_cpu_memreserve_lpi(unsigned int cpu)
{
struct page *pend_page;
@@ -5231,6 +5240,10 @@ static int its_cpu_memreserve_lpi(unsigned int cpu)
}
out:
+ /* Last CPU being brought up gets to issue the cleanup */
+ if (cpumask_equal(&cpus_booted_once_mask, cpu_possible_mask))
+ schedule_work(&rdist_memreserve_cpuhp_cleanup_work);
+
gic_data_rdist()->flags |= RD_LOCAL_MEMRESERVE_DONE;
return ret;
}
@@ -5425,6 +5438,7 @@ int __init its_lpi_memreserve_init(void)
if (!efi_enabled(EFI_CONFIG_TABLES))
return 0;
+ gic_rdists->cpuhp_memreserve_state = CPUHP_INVALID;
state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
"irqchip/arm/gicv3/memreserve:online",
its_cpu_memreserve_lpi,
@@ -5432,6 +5446,8 @@ int __init its_lpi_memreserve_init(void)
if (state < 0)
return state;
+ gic_rdists->cpuhp_memreserve_state = state;
+
return 0;
}
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index 51b85506ae90..12d91f0dedf9 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -624,6 +624,7 @@ struct rdists {
u64 flags;
u32 gicd_typer;
u32 gicd_typer2;
+ int cpuhp_memreserve_state;
bool has_vlpis;
bool has_rvpeid;
bool has_direct_lpi;
--
2.25.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT
2021-10-27 15:15 [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT Valentin Schneider
` (2 preceding siblings ...)
2021-10-27 15:15 ` [PATCH v2 3/3] irqchip/gic-v3-its: Limit memreserve cpuhp state lifetime Valentin Schneider
@ 2021-11-19 11:14 ` Sebastian Andrzej Siewior
2021-11-19 12:28 ` Marc Zyngier
3 siblings, 1 reply; 9+ messages in thread
From: Sebastian Andrzej Siewior @ 2021-11-19 11:14 UTC (permalink / raw)
To: Valentin Schneider
Cc: linux-kernel, linux-rt-users, linux-arm-kernel, Will Deacon,
Mark Rutland, Marc Zyngier, Thomas Gleixner, Ard Biesheuvel
On 2021-10-27 16:15:03 [+0100], Valentin Schneider wrote:
> Hi folks,
Hi,
> This is my take at fixing [1]. Reading about the LPI tables situation was
> entertaining.
>
> Tested against kexec on an Ampere eMAG. Seems to be working fine atop
> 5.15-rc6. On the other hand, I can only issue one kexec from
> 5.15-rc6-rt12 - if I then issue another one on the new kernel, I get tasks
> hanging. That is true even without my patches and without
> CONFIG_PREEMPT_RT.
What is the status of this? I haven't seen any replies, is this just
sitting to be merged?
Sebastian
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT
2021-11-19 11:14 ` [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT Sebastian Andrzej Siewior
@ 2021-11-19 12:28 ` Marc Zyngier
0 siblings, 0 replies; 9+ messages in thread
From: Marc Zyngier @ 2021-11-19 12:28 UTC (permalink / raw)
To: Sebastian Andrzej Siewior
Cc: Valentin Schneider, linux-kernel, linux-rt-users,
linux-arm-kernel, Will Deacon, Mark Rutland, Thomas Gleixner,
Ard Biesheuvel
On Fri, 19 Nov 2021 11:14:53 +0000,
Sebastian Andrzej Siewior <bigeasy@linutronix.de> wrote:
>
> On 2021-10-27 16:15:03 [+0100], Valentin Schneider wrote:
> > Hi folks,
> Hi,
>
> > This is my take at fixing [1]. Reading about the LPI tables situation was
> > entertaining.
> >
> > Tested against kexec on an Ampere eMAG. Seems to be working fine atop
> > 5.15-rc6. On the other hand, I can only issue one kexec from
> > 5.15-rc6-rt12 - if I then issue another one on the new kernel, I get tasks
> > hanging. That is true even without my patches and without
> > CONFIG_PREEMPT_RT.
>
> What is the status of this? I haven't seen any replies, is this just
> sitting to be merged?
No. This is in my review queue, and I'm massively behind. Nobody else
seems to risk reviewing GIC stuff (no idea why...).
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply [flat|nested] 9+ messages in thread
* [irqchip: irq/irqchip-next] irqchip/gic-v3-its: Limit memreserve cpuhp state lifetime
2021-10-27 15:15 ` [PATCH v2 3/3] irqchip/gic-v3-its: Limit memreserve cpuhp state lifetime Valentin Schneider
@ 2021-12-16 13:30 ` irqchip-bot for Valentin Schneider
0 siblings, 0 replies; 9+ messages in thread
From: irqchip-bot for Valentin Schneider @ 2021-12-16 13:30 UTC (permalink / raw)
To: linux-kernel; +Cc: Valentin Schneider, Marc Zyngier, tglx
The following commit has been merged into the irq/irqchip-next branch of irqchip:
Commit-ID: 835f442fdbce33a47a6bde356643fd7e3ef7ec1b
Gitweb: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms/835f442fdbce33a47a6bde356643fd7e3ef7ec1b
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Wed, 27 Oct 2021 16:15:06 +01:00
Committer: Marc Zyngier <maz@kernel.org>
CommitterDate: Thu, 16 Dec 2021 13:21:12
irqchip/gic-v3-its: Limit memreserve cpuhp state lifetime
The new memreserve cpuhp callback only needs to survive up until a point
where every CPU in the system has booted once. Beyond that, it becomes a
no-op and can be put in the bin.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211027151506.2085066-4-valentin.schneider@arm.com
---
drivers/irqchip/irq-gic-v3-its.c | 16 ++++++++++++++++
include/linux/irqchip/arm-gic-v3.h | 1 +
2 files changed, 17 insertions(+)
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index f860733..ee83eb3 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -5203,6 +5203,15 @@ int its_cpu_init(void)
return 0;
}
+static void rdist_memreserve_cpuhp_cleanup_workfn(struct work_struct *work)
+{
+ cpuhp_remove_state_nocalls(gic_rdists->cpuhp_memreserve_state);
+ gic_rdists->cpuhp_memreserve_state = CPUHP_INVALID;
+}
+
+static DECLARE_WORK(rdist_memreserve_cpuhp_cleanup_work,
+ rdist_memreserve_cpuhp_cleanup_workfn);
+
static int its_cpu_memreserve_lpi(unsigned int cpu)
{
struct page *pend_page;
@@ -5231,6 +5240,10 @@ static int its_cpu_memreserve_lpi(unsigned int cpu)
}
out:
+ /* Last CPU being brought up gets to issue the cleanup */
+ if (cpumask_equal(&cpus_booted_once_mask, cpu_possible_mask))
+ schedule_work(&rdist_memreserve_cpuhp_cleanup_work);
+
gic_data_rdist()->flags |= RD_LOCAL_MEMRESERVE_DONE;
return ret;
}
@@ -5425,6 +5438,7 @@ int __init its_lpi_memreserve_init(void)
if (!efi_enabled(EFI_CONFIG_TABLES))
return 0;
+ gic_rdists->cpuhp_memreserve_state = CPUHP_INVALID;
state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
"irqchip/arm/gicv3/memreserve:online",
its_cpu_memreserve_lpi,
@@ -5432,6 +5446,8 @@ int __init its_lpi_memreserve_init(void)
if (state < 0)
return state;
+ gic_rdists->cpuhp_memreserve_state = state;
+
return 0;
}
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index 51b8550..12d91f0 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -624,6 +624,7 @@ struct rdists {
u64 flags;
u32 gicd_typer;
u32 gicd_typer2;
+ int cpuhp_memreserve_state;
bool has_vlpis;
bool has_rvpeid;
bool has_direct_lpi;
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [irqchip: irq/irqchip-next] irqchip/gic-v3-its: Postpone LPI pending table freeing and memreserve
2021-10-27 15:15 ` [PATCH v2 2/3] irqchip/gic-v3-its: Postpone LPI pending table freeing and memreserve Valentin Schneider
@ 2021-12-16 13:30 ` irqchip-bot for Valentin Schneider
0 siblings, 0 replies; 9+ messages in thread
From: irqchip-bot for Valentin Schneider @ 2021-12-16 13:30 UTC (permalink / raw)
To: linux-kernel; +Cc: Valentin Schneider, Marc Zyngier, tglx
The following commit has been merged into the irq/irqchip-next branch of irqchip:
Commit-ID: d23bc2bc1d634658d7fa96395419c1c553a784f0
Gitweb: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms/d23bc2bc1d634658d7fa96395419c1c553a784f0
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Wed, 27 Oct 2021 16:15:05 +01:00
Committer: Marc Zyngier <maz@kernel.org>
CommitterDate: Thu, 16 Dec 2021 13:21:11
irqchip/gic-v3-its: Postpone LPI pending table freeing and memreserve
Memory used by the LPI tables have to be made persistent for kexec to have
a chance to work, as explained in [1]. If they have been made persistent
and we are booting into a kexec'd kernel, we also need to free the pages
that were preemptively allocated by the new kernel for those tables.
Both of those operations currently happen during its_cpu_init(), which
happens in a _STARTING (IOW atomic) cpuhp callback for secondary
CPUs. efi_mem_reserve_iomem() issues a GFP_ATOMIC allocation, which
unfortunately doesn't work under PREEMPT_RT (this ends up grabbing a
non-raw spinlock, which can sleep under PREEMPT_RT). Similarly, freeing the
pages ends up grabbing a sleepable spinlock.
Since the memreserve is only required by kexec, it doesn't have to be done
so early in the secondary boot process. Issue the reservation in a new
CPUHP_AP_ONLINE_DYN cpuhp callback, and piggy-back the page freeing on top
of it. A CPU gets to run the body of this new callback exactly once.
As kexec issues a machine_shutdown() prior to machine_kexec(), it will be
serialized vs a CPU being plugged to life by the hotplug machinery - either
the CPU will have been brought up and have had its redistributor's pending
table memreserved, or it never went online and will have its table
allocated by the new kernel.
[1]: https://lore.kernel.org/lkml/20180921195954.21574-1-marc.zyngier@arm.com/
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211027151506.2085066-3-valentin.schneider@arm.com
---
drivers/irqchip/irq-gic-v3-its.c | 58 +++++++++++++++++++++++++++--
drivers/irqchip/irq-gic-v3.c | 1 +-
include/linux/irqchip/arm-gic-v3.h | 1 +-
3 files changed, 56 insertions(+), 4 deletions(-)
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 74c2274..f860733 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -47,6 +47,8 @@
#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1)
#define RD_LOCAL_LPI_ENABLED BIT(0)
+#define RD_LOCAL_PENDTABLE_PREALLOCATED BIT(1)
+#define RD_LOCAL_MEMRESERVE_DONE BIT(2)
static u32 lpi_id_bits;
@@ -3065,15 +3067,13 @@ static void its_cpu_init_lpis(void)
paddr &= GENMASK_ULL(51, 16);
WARN_ON(!gic_check_reserved_range(paddr, LPI_PENDBASE_SZ));
- its_free_pending_table(gic_data_rdist()->pend_page);
- gic_data_rdist()->pend_page = NULL;
+ gic_data_rdist()->flags |= RD_LOCAL_PENDTABLE_PREALLOCATED;
goto out;
}
pend_page = gic_data_rdist()->pend_page;
paddr = page_to_phys(pend_page);
- WARN_ON(gic_reserve_range(paddr, LPI_PENDBASE_SZ));
/* set PROPBASE */
val = (gic_rdists->prop_table_pa |
@@ -3163,7 +3163,8 @@ out:
gic_data_rdist()->flags |= RD_LOCAL_LPI_ENABLED;
pr_info("GICv3: CPU%d: using %s LPI pending table @%pa\n",
smp_processor_id(),
- gic_data_rdist()->pend_page ? "allocated" : "reserved",
+ gic_data_rdist()->flags & RD_LOCAL_PENDTABLE_PREALLOCATED ?
+ "reserved" : "allocated",
&paddr);
}
@@ -5202,6 +5203,38 @@ int its_cpu_init(void)
return 0;
}
+static int its_cpu_memreserve_lpi(unsigned int cpu)
+{
+ struct page *pend_page;
+ int ret = 0;
+
+ /* This gets to run exactly once per CPU */
+ if (gic_data_rdist()->flags & RD_LOCAL_MEMRESERVE_DONE)
+ return 0;
+
+ pend_page = gic_data_rdist()->pend_page;
+ if (WARN_ON(!pend_page)) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ /*
+ * If the pending table was pre-programmed, free the memory we
+ * preemptively allocated. Otherwise, reserve that memory for
+ * later kexecs.
+ */
+ if (gic_data_rdist()->flags & RD_LOCAL_PENDTABLE_PREALLOCATED) {
+ its_free_pending_table(pend_page);
+ gic_data_rdist()->pend_page = NULL;
+ } else {
+ phys_addr_t paddr = page_to_phys(pend_page);
+ WARN_ON(gic_reserve_range(paddr, LPI_PENDBASE_SZ));
+ }
+
+out:
+ gic_data_rdist()->flags |= RD_LOCAL_MEMRESERVE_DONE;
+ return ret;
+}
+
static const struct of_device_id its_device_id[] = {
{ .compatible = "arm,gic-v3-its", },
{},
@@ -5385,6 +5418,23 @@ static void __init its_acpi_probe(void)
static void __init its_acpi_probe(void) { }
#endif
+int __init its_lpi_memreserve_init(void)
+{
+ int state;
+
+ if (!efi_enabled(EFI_CONFIG_TABLES))
+ return 0;
+
+ state = cpuhp_setup_state(CPUHP_AP_ONLINE_DYN,
+ "irqchip/arm/gicv3/memreserve:online",
+ its_cpu_memreserve_lpi,
+ NULL);
+ if (state < 0)
+ return state;
+
+ return 0;
+}
+
int __init its_init(struct fwnode_handle *handle, struct rdists *rdists,
struct irq_domain *parent_domain)
{
diff --git a/drivers/irqchip/irq-gic-v3.c b/drivers/irqchip/irq-gic-v3.c
index daec330..9fa3e1d 100644
--- a/drivers/irqchip/irq-gic-v3.c
+++ b/drivers/irqchip/irq-gic-v3.c
@@ -1802,6 +1802,7 @@ static int __init gic_init_bases(void __iomem *dist_base,
if (gic_dist_supports_lpis()) {
its_init(handle, &gic_data.rdists, gic_data.domain);
its_cpu_init();
+ its_lpi_memreserve_init();
} else {
if (IS_ENABLED(CONFIG_ARM_GIC_V2M))
gicv2m_init(handle, gic_data.domain);
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index 0dc34d7..51b8550 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -632,6 +632,7 @@ struct rdists {
struct irq_domain;
struct fwnode_handle;
+int __init its_lpi_memreserve_init(void);
int its_cpu_init(void);
int its_init(struct fwnode_handle *handle, struct rdists *rdists,
struct irq_domain *domain);
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [irqchip: irq/irqchip-next] irqchip/gic-v3-its: Give the percpu rdist struct its own flags field
2021-10-27 15:15 ` [PATCH v2 1/3] irqchip/gic-v3-its: Give the percpu rdist struct its own flags field Valentin Schneider
@ 2021-12-16 13:30 ` irqchip-bot for Valentin Schneider
0 siblings, 0 replies; 9+ messages in thread
From: irqchip-bot for Valentin Schneider @ 2021-12-16 13:30 UTC (permalink / raw)
To: linux-kernel; +Cc: Valentin Schneider, Marc Zyngier, tglx
The following commit has been merged into the irq/irqchip-next branch of irqchip:
Commit-ID: c0cdc89072a3e1ae3981437f385de14b7bba8fd8
Gitweb: https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms/c0cdc89072a3e1ae3981437f385de14b7bba8fd8
Author: Valentin Schneider <valentin.schneider@arm.com>
AuthorDate: Wed, 27 Oct 2021 16:15:04 +01:00
Committer: Marc Zyngier <maz@kernel.org>
CommitterDate: Thu, 16 Dec 2021 13:21:11
irqchip/gic-v3-its: Give the percpu rdist struct its own flags field
Later patches will require tracking some per-rdist status. Reuse the bytes
"lost" to padding within the __percpu rdist struct as a flags field, and
re-encode ->lpi_enabled within said flags.
No change in functionality intended.
Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
Signed-off-by: Marc Zyngier <maz@kernel.org>
Link: https://lore.kernel.org/r/20211027151506.2085066-2-valentin.schneider@arm.com
---
drivers/irqchip/irq-gic-v3-its.c | 8 +++++---
include/linux/irqchip/arm-gic-v3.h | 2 +-
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index eb0882d..74c2274 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -46,6 +46,8 @@
#define RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING (1 << 0)
#define RDIST_FLAGS_RD_TABLES_PREALLOCATED (1 << 1)
+#define RD_LOCAL_LPI_ENABLED BIT(0)
+
static u32 lpi_id_bits;
/*
@@ -3044,7 +3046,7 @@ static void its_cpu_init_lpis(void)
phys_addr_t paddr;
u64 val, tmp;
- if (gic_data_rdist()->lpi_enabled)
+ if (gic_data_rdist()->flags & RD_LOCAL_LPI_ENABLED)
return;
val = readl_relaxed(rbase + GICR_CTLR);
@@ -3158,7 +3160,7 @@ static void its_cpu_init_lpis(void)
/* Make sure the GIC has seen the above */
dsb(sy);
out:
- gic_data_rdist()->lpi_enabled = true;
+ gic_data_rdist()->flags |= RD_LOCAL_LPI_ENABLED;
pr_info("GICv3: CPU%d: using %s LPI pending table @%pa\n",
smp_processor_id(),
gic_data_rdist()->pend_page ? "allocated" : "reserved",
@@ -5138,7 +5140,7 @@ static int redist_disable_lpis(void)
*
* If running with preallocated tables, there is nothing to do.
*/
- if (gic_data_rdist()->lpi_enabled ||
+ if ((gic_data_rdist()->flags & RD_LOCAL_LPI_ENABLED) ||
(gic_rdists->flags & RDIST_FLAGS_RD_TABLES_PREALLOCATED))
return 0;
diff --git a/include/linux/irqchip/arm-gic-v3.h b/include/linux/irqchip/arm-gic-v3.h
index 81cbf85..0dc34d7 100644
--- a/include/linux/irqchip/arm-gic-v3.h
+++ b/include/linux/irqchip/arm-gic-v3.h
@@ -615,7 +615,7 @@ struct rdists {
void __iomem *rd_base;
struct page *pend_page;
phys_addr_t phys_base;
- bool lpi_enabled;
+ u64 flags;
cpumask_t *vpe_table_mask;
void *vpe_l1_base;
} __percpu *rdist;
^ permalink raw reply related [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-12-16 13:30 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-27 15:15 [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT Valentin Schneider
2021-10-27 15:15 ` [PATCH v2 1/3] irqchip/gic-v3-its: Give the percpu rdist struct its own flags field Valentin Schneider
2021-12-16 13:30 ` [irqchip: irq/irqchip-next] " irqchip-bot for Valentin Schneider
2021-10-27 15:15 ` [PATCH v2 2/3] irqchip/gic-v3-its: Postpone LPI pending table freeing and memreserve Valentin Schneider
2021-12-16 13:30 ` [irqchip: irq/irqchip-next] " irqchip-bot for Valentin Schneider
2021-10-27 15:15 ` [PATCH v2 3/3] irqchip/gic-v3-its: Limit memreserve cpuhp state lifetime Valentin Schneider
2021-12-16 13:30 ` [irqchip: irq/irqchip-next] " irqchip-bot for Valentin Schneider
2021-11-19 11:14 ` [PATCH v2 0/3] irqchip/gic-v3-its: Fix LPI pending table handling vs PREEMPT_RT Sebastian Andrzej Siewior
2021-11-19 12:28 ` Marc Zyngier
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).