* [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs
2017-03-30 19:06 [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs Paul Burton
@ 2017-03-30 19:06 ` Paul Burton
2017-03-30 19:06 ` [PATCH 1/5] irqchip: mips-cpu: Replace magic 0x100 with IE_SW0 Paul Burton
` (6 subsequent siblings)
7 siblings, 0 replies; 15+ messages in thread
From: Paul Burton @ 2017-03-30 19:06 UTC (permalink / raw)
To: Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper, Paul Burton
This series introduces support for IPI IRQ domains to the CPU interrupt
controller driver, allowing IPIs to function in the same way as those
provided by the MIPS GIC as far as platform/board code is concerned.
Doing this allows us to avoid duplicating code across platforms, avoid
having to handle cases where IPI domains are or aren't in use depending
upon the interrupt controller, and strengthen a sanity check for cases
where IPI IRQ domains are supported.
Applies atop v4.11-rc4.
Paul Burton (5):
irqchip: mips-cpu: Replace magic 0x100 with IE_SW0
irqchip: mips-cpu: Prepare for non-legacy IRQ domains
irqchip: mips-cpu: Introduce IPI IRQ domain support
MIPS: smp-mt: Use CPU interrupt controller IPI IRQ domain support
MIPS: Stengthen IPI IRQ domain sanity check
arch/mips/kernel/smp-mt.c | 49 ++------------
arch/mips/kernel/smp.c | 20 +++---
arch/mips/lantiq/irq.c | 52 --------------
arch/mips/mti-malta/malta-int.c | 83 ++---------------------
drivers/irqchip/Kconfig | 2 +
drivers/irqchip/irq-mips-cpu.c | 146 +++++++++++++++++++++++++++++++++++-----
6 files changed, 151 insertions(+), 201 deletions(-)
--
2.12.1
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH 1/5] irqchip: mips-cpu: Replace magic 0x100 with IE_SW0
2017-03-30 19:06 [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs Paul Burton
2017-03-30 19:06 ` Paul Burton
@ 2017-03-30 19:06 ` Paul Burton
2017-03-30 19:06 ` Paul Burton
2017-03-30 19:06 ` [PATCH 2/5] irqchip: mips-cpu: Prepare for non-legacy IRQ domains Paul Burton
` (5 subsequent siblings)
7 siblings, 1 reply; 15+ messages in thread
From: Paul Burton @ 2017-03-30 19:06 UTC (permalink / raw)
To: Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper, Paul Burton
Replace use of the magic number 0x100 (ie. bit 8) with the more
explanatory IE_SW0 (ie. interrupt enable for software interrupt 0) or
C_SW0 (ie. cause bit for software interrupt 0) as appropriate.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
drivers/irqchip/irq-mips-cpu.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/irqchip/irq-mips-cpu.c b/drivers/irqchip/irq-mips-cpu.c
index 8c504f562e9d..e6b413669e57 100644
--- a/drivers/irqchip/irq-mips-cpu.c
+++ b/drivers/irqchip/irq-mips-cpu.c
@@ -41,13 +41,13 @@
static inline void unmask_mips_irq(struct irq_data *d)
{
- set_c0_status(0x100 << (d->irq - MIPS_CPU_IRQ_BASE));
+ set_c0_status(IE_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
irq_enable_hazard();
}
static inline void mask_mips_irq(struct irq_data *d)
{
- clear_c0_status(0x100 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_status(IE_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
irq_disable_hazard();
}
@@ -70,7 +70,7 @@ static unsigned int mips_mt_cpu_irq_startup(struct irq_data *d)
{
unsigned int vpflags = dvpe();
- clear_c0_cause(0x100 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_cause(C_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
evpe(vpflags);
unmask_mips_irq(d);
return 0;
@@ -83,7 +83,7 @@ static unsigned int mips_mt_cpu_irq_startup(struct irq_data *d)
static void mips_mt_cpu_irq_ack(struct irq_data *d)
{
unsigned int vpflags = dvpe();
- clear_c0_cause(0x100 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_cause(C_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
evpe(vpflags);
mask_mips_irq(d);
}
--
2.12.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH 1/5] irqchip: mips-cpu: Replace magic 0x100 with IE_SW0
2017-03-30 19:06 ` [PATCH 1/5] irqchip: mips-cpu: Replace magic 0x100 with IE_SW0 Paul Burton
@ 2017-03-30 19:06 ` Paul Burton
0 siblings, 0 replies; 15+ messages in thread
From: Paul Burton @ 2017-03-30 19:06 UTC (permalink / raw)
To: Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper, Paul Burton
Replace use of the magic number 0x100 (ie. bit 8) with the more
explanatory IE_SW0 (ie. interrupt enable for software interrupt 0) or
C_SW0 (ie. cause bit for software interrupt 0) as appropriate.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
drivers/irqchip/irq-mips-cpu.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/drivers/irqchip/irq-mips-cpu.c b/drivers/irqchip/irq-mips-cpu.c
index 8c504f562e9d..e6b413669e57 100644
--- a/drivers/irqchip/irq-mips-cpu.c
+++ b/drivers/irqchip/irq-mips-cpu.c
@@ -41,13 +41,13 @@
static inline void unmask_mips_irq(struct irq_data *d)
{
- set_c0_status(0x100 << (d->irq - MIPS_CPU_IRQ_BASE));
+ set_c0_status(IE_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
irq_enable_hazard();
}
static inline void mask_mips_irq(struct irq_data *d)
{
- clear_c0_status(0x100 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_status(IE_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
irq_disable_hazard();
}
@@ -70,7 +70,7 @@ static unsigned int mips_mt_cpu_irq_startup(struct irq_data *d)
{
unsigned int vpflags = dvpe();
- clear_c0_cause(0x100 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_cause(C_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
evpe(vpflags);
unmask_mips_irq(d);
return 0;
@@ -83,7 +83,7 @@ static unsigned int mips_mt_cpu_irq_startup(struct irq_data *d)
static void mips_mt_cpu_irq_ack(struct irq_data *d)
{
unsigned int vpflags = dvpe();
- clear_c0_cause(0x100 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_cause(C_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
evpe(vpflags);
mask_mips_irq(d);
}
--
2.12.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 2/5] irqchip: mips-cpu: Prepare for non-legacy IRQ domains
2017-03-30 19:06 [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs Paul Burton
2017-03-30 19:06 ` Paul Burton
2017-03-30 19:06 ` [PATCH 1/5] irqchip: mips-cpu: Replace magic 0x100 with IE_SW0 Paul Burton
@ 2017-03-30 19:06 ` Paul Burton
2017-03-30 19:06 ` Paul Burton
2017-03-30 19:06 ` [PATCH 3/5] irqchip: mips-cpu: Introduce IPI IRQ domain support Paul Burton
` (4 subsequent siblings)
7 siblings, 1 reply; 15+ messages in thread
From: Paul Burton @ 2017-03-30 19:06 UTC (permalink / raw)
To: Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper, Paul Burton
The various struct irq_chip callbacks in the MIPS CPU interrupt
controller driver have been calculating the hardware interrupt number by
subtracting MIPS_CPU_IRQ_BASE from the virq number. This presumes a
linear mapping beginning from MIPS_CPU_IRQ_BASE, and this will not hold
once an IPI IRQ domain is introduced. Switch to using the hwirq field of
struct irq_data which already contains the hardware interrupt number
instead of attempting to calculate it.
Similarly, plat_irq_dispatch calculated the virq number by adding
MIPS_CPU_IRQ_BASE to the hardware interrupt number. Ready this for the
introduction of an IPI IRQ domain by instead using irq_linear_revmap.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
drivers/irqchip/irq-mips-cpu.c | 23 +++++++++++++----------
1 file changed, 13 insertions(+), 10 deletions(-)
diff --git a/drivers/irqchip/irq-mips-cpu.c b/drivers/irqchip/irq-mips-cpu.c
index e6b413669e57..338de924b269 100644
--- a/drivers/irqchip/irq-mips-cpu.c
+++ b/drivers/irqchip/irq-mips-cpu.c
@@ -39,15 +39,17 @@
#include <asm/mipsmtregs.h>
#include <asm/setup.h>
+static struct irq_domain *irq_domain;
+
static inline void unmask_mips_irq(struct irq_data *d)
{
- set_c0_status(IE_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
+ set_c0_status(IE_SW0 << d->hwirq);
irq_enable_hazard();
}
static inline void mask_mips_irq(struct irq_data *d)
{
- clear_c0_status(IE_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_status(IE_SW0 << d->hwirq);
irq_disable_hazard();
}
@@ -70,7 +72,7 @@ static unsigned int mips_mt_cpu_irq_startup(struct irq_data *d)
{
unsigned int vpflags = dvpe();
- clear_c0_cause(C_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_cause(C_SW0 << d->hwirq);
evpe(vpflags);
unmask_mips_irq(d);
return 0;
@@ -83,7 +85,7 @@ static unsigned int mips_mt_cpu_irq_startup(struct irq_data *d)
static void mips_mt_cpu_irq_ack(struct irq_data *d)
{
unsigned int vpflags = dvpe();
- clear_c0_cause(C_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_cause(C_SW0 << d->hwirq);
evpe(vpflags);
mask_mips_irq(d);
}
@@ -103,6 +105,7 @@ static struct irq_chip mips_mt_cpu_irq_controller = {
asmlinkage void __weak plat_irq_dispatch(void)
{
unsigned long pending = read_c0_cause() & read_c0_status() & ST0_IM;
+ unsigned int virq;
int irq;
if (!pending) {
@@ -113,7 +116,8 @@ asmlinkage void __weak plat_irq_dispatch(void)
pending >>= CAUSEB_IP;
while (pending) {
irq = fls(pending) - 1;
- do_IRQ(MIPS_CPU_IRQ_BASE + irq);
+ virq = irq_linear_revmap(irq_domain, irq);
+ do_IRQ(virq);
pending &= ~BIT(irq);
}
}
@@ -145,15 +149,14 @@ static const struct irq_domain_ops mips_cpu_intc_irq_domain_ops = {
static void __init __mips_cpu_irq_init(struct device_node *of_node)
{
- struct irq_domain *domain;
-
/* Mask interrupts. */
clear_c0_status(ST0_IM);
clear_c0_cause(CAUSEF_IP);
- domain = irq_domain_add_legacy(of_node, 8, MIPS_CPU_IRQ_BASE, 0,
- &mips_cpu_intc_irq_domain_ops, NULL);
- if (!domain)
+ irq_domain = irq_domain_add_legacy(of_node, 8, MIPS_CPU_IRQ_BASE, 0,
+ &mips_cpu_intc_irq_domain_ops,
+ NULL);
+ if (!irq_domain)
panic("Failed to add irqdomain for MIPS CPU");
}
--
2.12.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH 2/5] irqchip: mips-cpu: Prepare for non-legacy IRQ domains
2017-03-30 19:06 ` [PATCH 2/5] irqchip: mips-cpu: Prepare for non-legacy IRQ domains Paul Burton
@ 2017-03-30 19:06 ` Paul Burton
0 siblings, 0 replies; 15+ messages in thread
From: Paul Burton @ 2017-03-30 19:06 UTC (permalink / raw)
To: Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper, Paul Burton
The various struct irq_chip callbacks in the MIPS CPU interrupt
controller driver have been calculating the hardware interrupt number by
subtracting MIPS_CPU_IRQ_BASE from the virq number. This presumes a
linear mapping beginning from MIPS_CPU_IRQ_BASE, and this will not hold
once an IPI IRQ domain is introduced. Switch to using the hwirq field of
struct irq_data which already contains the hardware interrupt number
instead of attempting to calculate it.
Similarly, plat_irq_dispatch calculated the virq number by adding
MIPS_CPU_IRQ_BASE to the hardware interrupt number. Ready this for the
introduction of an IPI IRQ domain by instead using irq_linear_revmap.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
drivers/irqchip/irq-mips-cpu.c | 23 +++++++++++++----------
1 file changed, 13 insertions(+), 10 deletions(-)
diff --git a/drivers/irqchip/irq-mips-cpu.c b/drivers/irqchip/irq-mips-cpu.c
index e6b413669e57..338de924b269 100644
--- a/drivers/irqchip/irq-mips-cpu.c
+++ b/drivers/irqchip/irq-mips-cpu.c
@@ -39,15 +39,17 @@
#include <asm/mipsmtregs.h>
#include <asm/setup.h>
+static struct irq_domain *irq_domain;
+
static inline void unmask_mips_irq(struct irq_data *d)
{
- set_c0_status(IE_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
+ set_c0_status(IE_SW0 << d->hwirq);
irq_enable_hazard();
}
static inline void mask_mips_irq(struct irq_data *d)
{
- clear_c0_status(IE_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_status(IE_SW0 << d->hwirq);
irq_disable_hazard();
}
@@ -70,7 +72,7 @@ static unsigned int mips_mt_cpu_irq_startup(struct irq_data *d)
{
unsigned int vpflags = dvpe();
- clear_c0_cause(C_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_cause(C_SW0 << d->hwirq);
evpe(vpflags);
unmask_mips_irq(d);
return 0;
@@ -83,7 +85,7 @@ static unsigned int mips_mt_cpu_irq_startup(struct irq_data *d)
static void mips_mt_cpu_irq_ack(struct irq_data *d)
{
unsigned int vpflags = dvpe();
- clear_c0_cause(C_SW0 << (d->irq - MIPS_CPU_IRQ_BASE));
+ clear_c0_cause(C_SW0 << d->hwirq);
evpe(vpflags);
mask_mips_irq(d);
}
@@ -103,6 +105,7 @@ static struct irq_chip mips_mt_cpu_irq_controller = {
asmlinkage void __weak plat_irq_dispatch(void)
{
unsigned long pending = read_c0_cause() & read_c0_status() & ST0_IM;
+ unsigned int virq;
int irq;
if (!pending) {
@@ -113,7 +116,8 @@ asmlinkage void __weak plat_irq_dispatch(void)
pending >>= CAUSEB_IP;
while (pending) {
irq = fls(pending) - 1;
- do_IRQ(MIPS_CPU_IRQ_BASE + irq);
+ virq = irq_linear_revmap(irq_domain, irq);
+ do_IRQ(virq);
pending &= ~BIT(irq);
}
}
@@ -145,15 +149,14 @@ static const struct irq_domain_ops mips_cpu_intc_irq_domain_ops = {
static void __init __mips_cpu_irq_init(struct device_node *of_node)
{
- struct irq_domain *domain;
-
/* Mask interrupts. */
clear_c0_status(ST0_IM);
clear_c0_cause(CAUSEF_IP);
- domain = irq_domain_add_legacy(of_node, 8, MIPS_CPU_IRQ_BASE, 0,
- &mips_cpu_intc_irq_domain_ops, NULL);
- if (!domain)
+ irq_domain = irq_domain_add_legacy(of_node, 8, MIPS_CPU_IRQ_BASE, 0,
+ &mips_cpu_intc_irq_domain_ops,
+ NULL);
+ if (!irq_domain)
panic("Failed to add irqdomain for MIPS CPU");
}
--
2.12.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 3/5] irqchip: mips-cpu: Introduce IPI IRQ domain support
2017-03-30 19:06 [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs Paul Burton
` (2 preceding siblings ...)
2017-03-30 19:06 ` [PATCH 2/5] irqchip: mips-cpu: Prepare for non-legacy IRQ domains Paul Burton
@ 2017-03-30 19:06 ` Paul Burton
2017-03-30 19:06 ` Paul Burton
2017-03-30 19:06 ` [PATCH 4/5] MIPS: smp-mt: Use CPU interrupt controller " Paul Burton
` (3 subsequent siblings)
7 siblings, 1 reply; 15+ messages in thread
From: Paul Burton @ 2017-03-30 19:06 UTC (permalink / raw)
To: Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper, Paul Burton
Introduce support for registering an IPI IRQ domain suitable for use by
systems using the MIPS MT (multithreading) ASE within a single core.
This will allow for such systems to be supported generically, without
the current kludge of IPI code split between the MIPS arch & the malta
board support code.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
drivers/irqchip/Kconfig | 2 +
drivers/irqchip/irq-mips-cpu.c | 125 ++++++++++++++++++++++++++++++++++++++---
2 files changed, 119 insertions(+), 8 deletions(-)
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 125528f39e92..cfb33a365081 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -128,7 +128,9 @@ config IMGPDC_IRQ
config IRQ_MIPS_CPU
bool
select GENERIC_IRQ_CHIP
+ select GENERIC_IRQ_IPI if SYS_SUPPORTS_MULTITHREADING
select IRQ_DOMAIN
+ select IRQ_DOMAIN_HIERARCHY if GENERIC_IRQ_IPI
config CLPS711X_IRQCHIP
bool
diff --git a/drivers/irqchip/irq-mips-cpu.c b/drivers/irqchip/irq-mips-cpu.c
index 338de924b269..b247f3c743ac 100644
--- a/drivers/irqchip/irq-mips-cpu.c
+++ b/drivers/irqchip/irq-mips-cpu.c
@@ -17,15 +17,14 @@
/*
* Almost all MIPS CPUs define 8 interrupt sources. They are typically
* level triggered (i.e., cannot be cleared from CPU; must be cleared from
- * device). The first two are software interrupts which we don't really
- * use or support. The last one is usually the CPU timer interrupt if
- * counter register is present or, for CPUs with an external FPU, by
- * convention it's the FPU exception interrupt.
+ * device).
*
- * Don't even think about using this on SMP. You have been warned.
+ * The first two are software interrupts (i.e. not exposed as pins) which
+ * may be used for IPIs in multi-threaded single-core systems.
*
- * This file exports one global function:
- * void mips_cpu_irq_init(void);
+ * The last one is usually the CPU timer interrupt if the counter register
+ * is present, or for old CPUs with an external FPU by convention it's the
+ * FPU exception interrupt.
*/
#include <linux/init.h>
#include <linux/interrupt.h>
@@ -40,6 +39,7 @@
#include <asm/setup.h>
static struct irq_domain *irq_domain;
+static struct irq_domain *ipi_domain;
static inline void unmask_mips_irq(struct irq_data *d)
{
@@ -90,6 +90,29 @@ static void mips_mt_cpu_irq_ack(struct irq_data *d)
mask_mips_irq(d);
}
+#ifdef CONFIG_GENERIC_IRQ_IPI
+
+static void mips_mt_send_ipi(struct irq_data *d, unsigned int cpu)
+{
+ irq_hw_number_t hwirq = irqd_to_hwirq(d);
+ unsigned long flags;
+ int vpflags;
+
+ local_irq_save(flags);
+
+ /* We can only send IPIs to VPEs within the local core */
+ WARN_ON(cpu_data[cpu].core != current_cpu_data.core);
+
+ vpflags = dvpe();
+ settc(cpu_vpe_id(&cpu_data[cpu]));
+ write_vpe_c0_cause(read_vpe_c0_cause() | (C_SW0 << hwirq));
+ evpe(vpflags);
+
+ local_irq_restore(flags);
+}
+
+#endif /* CONFIG_GENERIC_IRQ_IPI */
+
static struct irq_chip mips_mt_cpu_irq_controller = {
.name = "MIPS",
.irq_startup = mips_mt_cpu_irq_startup,
@@ -100,6 +123,9 @@ static struct irq_chip mips_mt_cpu_irq_controller = {
.irq_eoi = unmask_mips_irq,
.irq_disable = mask_mips_irq,
.irq_enable = unmask_mips_irq,
+#ifdef CONFIG_GENERIC_IRQ_IPI
+ .ipi_send_single = mips_mt_send_ipi,
+#endif
};
asmlinkage void __weak plat_irq_dispatch(void)
@@ -116,7 +142,10 @@ asmlinkage void __weak plat_irq_dispatch(void)
pending >>= CAUSEB_IP;
while (pending) {
irq = fls(pending) - 1;
- virq = irq_linear_revmap(irq_domain, irq);
+ if (IS_ENABLED(CONFIG_GENERIC_IRQ_IPI) && irq < 2)
+ virq = irq_linear_revmap(ipi_domain, irq);
+ else
+ virq = irq_linear_revmap(irq_domain, irq);
do_IRQ(virq);
pending &= ~BIT(irq);
}
@@ -147,6 +176,79 @@ static const struct irq_domain_ops mips_cpu_intc_irq_domain_ops = {
.xlate = irq_domain_xlate_onecell,
};
+#ifdef CONFIG_GENERIC_IRQ_IPI
+
+struct cpu_ipi_domain_state {
+ DECLARE_BITMAP(allocated, 2);
+};
+
+static int mips_cpu_ipi_alloc(struct irq_domain *domain, unsigned int virq,
+ unsigned int nr_irqs, void *arg)
+{
+ struct cpu_ipi_domain_state *state = domain->host_data;
+ unsigned int i, hwirq;
+ int ret;
+
+ for (i = 0; i < nr_irqs; i++) {
+ hwirq = find_first_zero_bit(state->allocated, 2);
+ if (hwirq == 2)
+ return -EBUSY;
+ bitmap_set(state->allocated, hwirq, 1);
+
+ ret = irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq,
+ &mips_mt_cpu_irq_controller,
+ NULL);
+ if (ret)
+ return ret;
+
+ ret = irq_set_irq_type(virq + i, IRQ_TYPE_LEVEL_HIGH);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int mips_cpu_ipi_match(struct irq_domain *d, struct device_node *node,
+ enum irq_domain_bus_token bus_token)
+{
+ bool is_ipi;
+
+ switch (bus_token) {
+ case DOMAIN_BUS_IPI:
+ is_ipi = d->bus_token == bus_token;
+ return (!node || (to_of_node(d->fwnode) == node)) && is_ipi;
+ default:
+ return 0;
+ }
+}
+
+static const struct irq_domain_ops mips_cpu_ipi_chip_ops = {
+ .alloc = mips_cpu_ipi_alloc,
+ .match = mips_cpu_ipi_match,
+};
+
+static void mips_cpu_register_ipi_domain(struct device_node *of_node)
+{
+ struct cpu_ipi_domain_state *ipi_domain_state;
+
+ ipi_domain_state = kzalloc(sizeof(*ipi_domain_state), GFP_KERNEL);
+ ipi_domain = irq_domain_add_hierarchy(irq_domain,
+ IRQ_DOMAIN_FLAG_IPI_SINGLE,
+ 2, of_node,
+ &mips_cpu_ipi_chip_ops,
+ ipi_domain_state);
+ if (!ipi_domain)
+ panic("Failed to add MIPS CPU IPI domain");
+ ipi_domain->bus_token = DOMAIN_BUS_IPI;
+}
+
+#else /* !CONFIG_GENERIC_IRQ_IPI */
+
+static inline void mips_cpu_register_ipi_domain(struct device_node *of_node) {}
+
+#endif /* !CONFIG_GENERIC_IRQ_IPI */
+
static void __init __mips_cpu_irq_init(struct device_node *of_node)
{
/* Mask interrupts. */
@@ -158,6 +260,13 @@ static void __init __mips_cpu_irq_init(struct device_node *of_node)
NULL);
if (!irq_domain)
panic("Failed to add irqdomain for MIPS CPU");
+
+ /*
+ * Only proceed to register the software interrupt IPI implementation
+ * for CPUs which implement the MIPS MT (multi-threading) ASE.
+ */
+ if (cpu_has_mipsmt)
+ mips_cpu_register_ipi_domain(of_node);
}
void __init mips_cpu_irq_init(void)
--
2.12.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH 3/5] irqchip: mips-cpu: Introduce IPI IRQ domain support
2017-03-30 19:06 ` [PATCH 3/5] irqchip: mips-cpu: Introduce IPI IRQ domain support Paul Burton
@ 2017-03-30 19:06 ` Paul Burton
0 siblings, 0 replies; 15+ messages in thread
From: Paul Burton @ 2017-03-30 19:06 UTC (permalink / raw)
To: Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper, Paul Burton
Introduce support for registering an IPI IRQ domain suitable for use by
systems using the MIPS MT (multithreading) ASE within a single core.
This will allow for such systems to be supported generically, without
the current kludge of IPI code split between the MIPS arch & the malta
board support code.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
drivers/irqchip/Kconfig | 2 +
drivers/irqchip/irq-mips-cpu.c | 125 ++++++++++++++++++++++++++++++++++++++---
2 files changed, 119 insertions(+), 8 deletions(-)
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index 125528f39e92..cfb33a365081 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -128,7 +128,9 @@ config IMGPDC_IRQ
config IRQ_MIPS_CPU
bool
select GENERIC_IRQ_CHIP
+ select GENERIC_IRQ_IPI if SYS_SUPPORTS_MULTITHREADING
select IRQ_DOMAIN
+ select IRQ_DOMAIN_HIERARCHY if GENERIC_IRQ_IPI
config CLPS711X_IRQCHIP
bool
diff --git a/drivers/irqchip/irq-mips-cpu.c b/drivers/irqchip/irq-mips-cpu.c
index 338de924b269..b247f3c743ac 100644
--- a/drivers/irqchip/irq-mips-cpu.c
+++ b/drivers/irqchip/irq-mips-cpu.c
@@ -17,15 +17,14 @@
/*
* Almost all MIPS CPUs define 8 interrupt sources. They are typically
* level triggered (i.e., cannot be cleared from CPU; must be cleared from
- * device). The first two are software interrupts which we don't really
- * use or support. The last one is usually the CPU timer interrupt if
- * counter register is present or, for CPUs with an external FPU, by
- * convention it's the FPU exception interrupt.
+ * device).
*
- * Don't even think about using this on SMP. You have been warned.
+ * The first two are software interrupts (i.e. not exposed as pins) which
+ * may be used for IPIs in multi-threaded single-core systems.
*
- * This file exports one global function:
- * void mips_cpu_irq_init(void);
+ * The last one is usually the CPU timer interrupt if the counter register
+ * is present, or for old CPUs with an external FPU by convention it's the
+ * FPU exception interrupt.
*/
#include <linux/init.h>
#include <linux/interrupt.h>
@@ -40,6 +39,7 @@
#include <asm/setup.h>
static struct irq_domain *irq_domain;
+static struct irq_domain *ipi_domain;
static inline void unmask_mips_irq(struct irq_data *d)
{
@@ -90,6 +90,29 @@ static void mips_mt_cpu_irq_ack(struct irq_data *d)
mask_mips_irq(d);
}
+#ifdef CONFIG_GENERIC_IRQ_IPI
+
+static void mips_mt_send_ipi(struct irq_data *d, unsigned int cpu)
+{
+ irq_hw_number_t hwirq = irqd_to_hwirq(d);
+ unsigned long flags;
+ int vpflags;
+
+ local_irq_save(flags);
+
+ /* We can only send IPIs to VPEs within the local core */
+ WARN_ON(cpu_data[cpu].core != current_cpu_data.core);
+
+ vpflags = dvpe();
+ settc(cpu_vpe_id(&cpu_data[cpu]));
+ write_vpe_c0_cause(read_vpe_c0_cause() | (C_SW0 << hwirq));
+ evpe(vpflags);
+
+ local_irq_restore(flags);
+}
+
+#endif /* CONFIG_GENERIC_IRQ_IPI */
+
static struct irq_chip mips_mt_cpu_irq_controller = {
.name = "MIPS",
.irq_startup = mips_mt_cpu_irq_startup,
@@ -100,6 +123,9 @@ static struct irq_chip mips_mt_cpu_irq_controller = {
.irq_eoi = unmask_mips_irq,
.irq_disable = mask_mips_irq,
.irq_enable = unmask_mips_irq,
+#ifdef CONFIG_GENERIC_IRQ_IPI
+ .ipi_send_single = mips_mt_send_ipi,
+#endif
};
asmlinkage void __weak plat_irq_dispatch(void)
@@ -116,7 +142,10 @@ asmlinkage void __weak plat_irq_dispatch(void)
pending >>= CAUSEB_IP;
while (pending) {
irq = fls(pending) - 1;
- virq = irq_linear_revmap(irq_domain, irq);
+ if (IS_ENABLED(CONFIG_GENERIC_IRQ_IPI) && irq < 2)
+ virq = irq_linear_revmap(ipi_domain, irq);
+ else
+ virq = irq_linear_revmap(irq_domain, irq);
do_IRQ(virq);
pending &= ~BIT(irq);
}
@@ -147,6 +176,79 @@ static const struct irq_domain_ops mips_cpu_intc_irq_domain_ops = {
.xlate = irq_domain_xlate_onecell,
};
+#ifdef CONFIG_GENERIC_IRQ_IPI
+
+struct cpu_ipi_domain_state {
+ DECLARE_BITMAP(allocated, 2);
+};
+
+static int mips_cpu_ipi_alloc(struct irq_domain *domain, unsigned int virq,
+ unsigned int nr_irqs, void *arg)
+{
+ struct cpu_ipi_domain_state *state = domain->host_data;
+ unsigned int i, hwirq;
+ int ret;
+
+ for (i = 0; i < nr_irqs; i++) {
+ hwirq = find_first_zero_bit(state->allocated, 2);
+ if (hwirq == 2)
+ return -EBUSY;
+ bitmap_set(state->allocated, hwirq, 1);
+
+ ret = irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq,
+ &mips_mt_cpu_irq_controller,
+ NULL);
+ if (ret)
+ return ret;
+
+ ret = irq_set_irq_type(virq + i, IRQ_TYPE_LEVEL_HIGH);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
+static int mips_cpu_ipi_match(struct irq_domain *d, struct device_node *node,
+ enum irq_domain_bus_token bus_token)
+{
+ bool is_ipi;
+
+ switch (bus_token) {
+ case DOMAIN_BUS_IPI:
+ is_ipi = d->bus_token == bus_token;
+ return (!node || (to_of_node(d->fwnode) == node)) && is_ipi;
+ default:
+ return 0;
+ }
+}
+
+static const struct irq_domain_ops mips_cpu_ipi_chip_ops = {
+ .alloc = mips_cpu_ipi_alloc,
+ .match = mips_cpu_ipi_match,
+};
+
+static void mips_cpu_register_ipi_domain(struct device_node *of_node)
+{
+ struct cpu_ipi_domain_state *ipi_domain_state;
+
+ ipi_domain_state = kzalloc(sizeof(*ipi_domain_state), GFP_KERNEL);
+ ipi_domain = irq_domain_add_hierarchy(irq_domain,
+ IRQ_DOMAIN_FLAG_IPI_SINGLE,
+ 2, of_node,
+ &mips_cpu_ipi_chip_ops,
+ ipi_domain_state);
+ if (!ipi_domain)
+ panic("Failed to add MIPS CPU IPI domain");
+ ipi_domain->bus_token = DOMAIN_BUS_IPI;
+}
+
+#else /* !CONFIG_GENERIC_IRQ_IPI */
+
+static inline void mips_cpu_register_ipi_domain(struct device_node *of_node) {}
+
+#endif /* !CONFIG_GENERIC_IRQ_IPI */
+
static void __init __mips_cpu_irq_init(struct device_node *of_node)
{
/* Mask interrupts. */
@@ -158,6 +260,13 @@ static void __init __mips_cpu_irq_init(struct device_node *of_node)
NULL);
if (!irq_domain)
panic("Failed to add irqdomain for MIPS CPU");
+
+ /*
+ * Only proceed to register the software interrupt IPI implementation
+ * for CPUs which implement the MIPS MT (multi-threading) ASE.
+ */
+ if (cpu_has_mipsmt)
+ mips_cpu_register_ipi_domain(of_node);
}
void __init mips_cpu_irq_init(void)
--
2.12.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 4/5] MIPS: smp-mt: Use CPU interrupt controller IPI IRQ domain support
2017-03-30 19:06 [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs Paul Burton
` (3 preceding siblings ...)
2017-03-30 19:06 ` [PATCH 3/5] irqchip: mips-cpu: Introduce IPI IRQ domain support Paul Burton
@ 2017-03-30 19:06 ` Paul Burton
2017-03-30 19:06 ` Paul Burton
2017-03-30 19:06 ` [PATCH 5/5] MIPS: Stengthen IPI IRQ domain sanity check Paul Burton
` (2 subsequent siblings)
7 siblings, 1 reply; 15+ messages in thread
From: Paul Burton @ 2017-03-30 19:06 UTC (permalink / raw)
To: Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper, Paul Burton
Remove the smp-mt IPI code that supported single-core multithreaded
systems and instead make use of the IPI IRQ domain support provided by
the MIPS CPU interrupt controller driver. This removes some less than
nice code, the horrible split between arch & board code and the
duplication that led to within board code.
The lantiq portion of this patch has only been compile tested. Malta has
been tested & is functional.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/kernel/smp-mt.c | 49 ++----------------------
arch/mips/lantiq/irq.c | 52 --------------------------
arch/mips/mti-malta/malta-int.c | 83 ++---------------------------------------
3 files changed, 8 insertions(+), 176 deletions(-)
diff --git a/arch/mips/kernel/smp-mt.c b/arch/mips/kernel/smp-mt.c
index e398cbc3d776..ed6b4df583ea 100644
--- a/arch/mips/kernel/smp-mt.c
+++ b/arch/mips/kernel/smp-mt.c
@@ -83,6 +83,8 @@ static unsigned int __init smvp_vpe_init(unsigned int tc, unsigned int mvpconf0,
if (tc != 0)
smvp_copy_vpe_config();
+ cpu_data[ncpu].vpe_id = tc;
+
return ncpu;
}
@@ -114,49 +116,6 @@ static void __init smvp_tc_init(unsigned int tc, unsigned int mvpconf0)
write_tc_c0_tchalt(TCHALT_H);
}
-static void vsmp_send_ipi_single(int cpu, unsigned int action)
-{
- int i;
- unsigned long flags;
- int vpflags;
-
-#ifdef CONFIG_MIPS_GIC
- if (gic_present) {
- mips_smp_send_ipi_single(cpu, action);
- return;
- }
-#endif
- local_irq_save(flags);
-
- vpflags = dvpe(); /* can't access the other CPU's registers whilst MVPE enabled */
-
- switch (action) {
- case SMP_CALL_FUNCTION:
- i = C_SW1;
- break;
-
- case SMP_RESCHEDULE_YOURSELF:
- default:
- i = C_SW0;
- break;
- }
-
- /* 1:1 mapping of vpe and tc... */
- settc(cpu);
- write_vpe_c0_cause(read_vpe_c0_cause() | i);
- evpe(vpflags);
-
- local_irq_restore(flags);
-}
-
-static void vsmp_send_ipi_mask(const struct cpumask *mask, unsigned int action)
-{
- unsigned int i;
-
- for_each_cpu(i, mask)
- vsmp_send_ipi_single(i, action);
-}
-
static void vsmp_init_secondary(void)
{
#ifdef CONFIG_MIPS_GIC
@@ -281,8 +240,8 @@ static void __init vsmp_prepare_cpus(unsigned int max_cpus)
}
struct plat_smp_ops vsmp_smp_ops = {
- .send_ipi_single = vsmp_send_ipi_single,
- .send_ipi_mask = vsmp_send_ipi_mask,
+ .send_ipi_single = mips_smp_send_ipi_single,
+ .send_ipi_mask = mips_smp_send_ipi_mask,
.init_secondary = vsmp_init_secondary,
.smp_finish = vsmp_smp_finish,
.boot_secondary = vsmp_boot_secondary,
diff --git a/arch/mips/lantiq/irq.c b/arch/mips/lantiq/irq.c
index 0ddf3698b85d..33728b7af426 100644
--- a/arch/mips/lantiq/irq.c
+++ b/arch/mips/lantiq/irq.c
@@ -274,47 +274,6 @@ static void ltq_hw_irq_handler(struct irq_desc *desc)
ltq_hw_irqdispatch(irq_desc_get_irq(desc) - 2);
}
-#ifdef CONFIG_MIPS_MT_SMP
-void __init arch_init_ipiirq(int irq, struct irqaction *action)
-{
- setup_irq(irq, action);
- irq_set_handler(irq, handle_percpu_irq);
-}
-
-static void ltq_sw0_irqdispatch(void)
-{
- do_IRQ(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_RESCHED_IRQ);
-}
-
-static void ltq_sw1_irqdispatch(void)
-{
- do_IRQ(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_CALL_IRQ);
-}
-static irqreturn_t ipi_resched_interrupt(int irq, void *dev_id)
-{
- scheduler_ipi();
- return IRQ_HANDLED;
-}
-
-static irqreturn_t ipi_call_interrupt(int irq, void *dev_id)
-{
- generic_smp_call_function_interrupt();
- return IRQ_HANDLED;
-}
-
-static struct irqaction irq_resched = {
- .handler = ipi_resched_interrupt,
- .flags = IRQF_PERCPU,
- .name = "IPI_resched"
-};
-
-static struct irqaction irq_call = {
- .handler = ipi_call_interrupt,
- .flags = IRQF_PERCPU,
- .name = "IPI_call"
-};
-#endif
-
asmlinkage void plat_irq_dispatch(void)
{
unsigned int pending = read_c0_status() & read_c0_cause() & ST0_IM;
@@ -402,17 +361,6 @@ int __init icu_of_init(struct device_node *node, struct device_node *parent)
(MAX_IM * INT_NUM_IM_OFFSET) + MIPS_CPU_IRQ_CASCADE,
&irq_domain_ops, 0);
-#if defined(CONFIG_MIPS_MT_SMP)
- if (cpu_has_vint) {
- pr_info("Setting up IPI vectored interrupts\n");
- set_vi_handler(MIPS_CPU_IPI_RESCHED_IRQ, ltq_sw0_irqdispatch);
- set_vi_handler(MIPS_CPU_IPI_CALL_IRQ, ltq_sw1_irqdispatch);
- }
- arch_init_ipiirq(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_RESCHED_IRQ,
- &irq_resched);
- arch_init_ipiirq(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_CALL_IRQ, &irq_call);
-#endif
-
#ifndef CONFIG_MIPS_MT_SMP
set_c0_status(IE_IRQ0 | IE_IRQ1 | IE_IRQ2 |
IE_IRQ3 | IE_IRQ4 | IE_IRQ5);
diff --git a/arch/mips/mti-malta/malta-int.c b/arch/mips/mti-malta/malta-int.c
index cb675ec6f283..fe9bb479f2a0 100644
--- a/arch/mips/mti-malta/malta-int.c
+++ b/arch/mips/mti-malta/malta-int.c
@@ -145,56 +145,6 @@ static irqreturn_t corehi_handler(int irq, void *dev_id)
return IRQ_HANDLED;
}
-#ifdef CONFIG_MIPS_MT_SMP
-
-#define MIPS_CPU_IPI_RESCHED_IRQ 0 /* SW int 0 for resched */
-#define C_RESCHED C_SW0
-#define MIPS_CPU_IPI_CALL_IRQ 1 /* SW int 1 for resched */
-#define C_CALL C_SW1
-static int cpu_ipi_resched_irq, cpu_ipi_call_irq;
-
-static void ipi_resched_dispatch(void)
-{
- do_IRQ(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_RESCHED_IRQ);
-}
-
-static void ipi_call_dispatch(void)
-{
- do_IRQ(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_CALL_IRQ);
-}
-
-static irqreturn_t ipi_resched_interrupt(int irq, void *dev_id)
-{
-#ifdef CONFIG_MIPS_VPE_APSP_API_CMP
- if (aprp_hook)
- aprp_hook();
-#endif
-
- scheduler_ipi();
-
- return IRQ_HANDLED;
-}
-
-static irqreturn_t ipi_call_interrupt(int irq, void *dev_id)
-{
- generic_smp_call_function_interrupt();
-
- return IRQ_HANDLED;
-}
-
-static struct irqaction irq_resched = {
- .handler = ipi_resched_interrupt,
- .flags = IRQF_PERCPU,
- .name = "IPI_resched"
-};
-
-static struct irqaction irq_call = {
- .handler = ipi_call_interrupt,
- .flags = IRQF_PERCPU,
- .name = "IPI_call"
-};
-#endif /* CONFIG_MIPS_MT_SMP */
-
static struct irqaction corehi_irqaction = {
.handler = corehi_handler,
.name = "CoreHi",
@@ -222,12 +172,6 @@ static msc_irqmap_t msc_eicirqmap[] __initdata = {
static int msc_nr_eicirqs __initdata = ARRAY_SIZE(msc_eicirqmap);
-void __init arch_init_ipiirq(int irq, struct irqaction *action)
-{
- setup_irq(irq, action);
- irq_set_handler(irq, handle_percpu_irq);
-}
-
void __init arch_init_irq(void)
{
int corehi_irq;
@@ -262,30 +206,11 @@ void __init arch_init_irq(void)
if (gic_present) {
corehi_irq = MIPS_CPU_IRQ_BASE + MIPSCPU_INT_COREHI;
+ } else if (cpu_has_veic) {
+ set_vi_handler(MSC01E_INT_COREHI, corehi_irqdispatch);
+ corehi_irq = MSC01E_INT_BASE + MSC01E_INT_COREHI;
} else {
-#if defined(CONFIG_MIPS_MT_SMP)
- /* set up ipi interrupts */
- if (cpu_has_veic) {
- set_vi_handler (MSC01E_INT_SW0, ipi_resched_dispatch);
- set_vi_handler (MSC01E_INT_SW1, ipi_call_dispatch);
- cpu_ipi_resched_irq = MSC01E_INT_SW0;
- cpu_ipi_call_irq = MSC01E_INT_SW1;
- } else {
- cpu_ipi_resched_irq = MIPS_CPU_IRQ_BASE +
- MIPS_CPU_IPI_RESCHED_IRQ;
- cpu_ipi_call_irq = MIPS_CPU_IRQ_BASE +
- MIPS_CPU_IPI_CALL_IRQ;
- }
- arch_init_ipiirq(cpu_ipi_resched_irq, &irq_resched);
- arch_init_ipiirq(cpu_ipi_call_irq, &irq_call);
-#endif
- if (cpu_has_veic) {
- set_vi_handler(MSC01E_INT_COREHI,
- corehi_irqdispatch);
- corehi_irq = MSC01E_INT_BASE + MSC01E_INT_COREHI;
- } else {
- corehi_irq = MIPS_CPU_IRQ_BASE + MIPSCPU_INT_COREHI;
- }
+ corehi_irq = MIPS_CPU_IRQ_BASE + MIPSCPU_INT_COREHI;
}
setup_irq(corehi_irq, &corehi_irqaction);
--
2.12.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH 4/5] MIPS: smp-mt: Use CPU interrupt controller IPI IRQ domain support
2017-03-30 19:06 ` [PATCH 4/5] MIPS: smp-mt: Use CPU interrupt controller " Paul Burton
@ 2017-03-30 19:06 ` Paul Burton
0 siblings, 0 replies; 15+ messages in thread
From: Paul Burton @ 2017-03-30 19:06 UTC (permalink / raw)
To: Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper, Paul Burton
Remove the smp-mt IPI code that supported single-core multithreaded
systems and instead make use of the IPI IRQ domain support provided by
the MIPS CPU interrupt controller driver. This removes some less than
nice code, the horrible split between arch & board code and the
duplication that led to within board code.
The lantiq portion of this patch has only been compile tested. Malta has
been tested & is functional.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/kernel/smp-mt.c | 49 ++----------------------
arch/mips/lantiq/irq.c | 52 --------------------------
arch/mips/mti-malta/malta-int.c | 83 ++---------------------------------------
3 files changed, 8 insertions(+), 176 deletions(-)
diff --git a/arch/mips/kernel/smp-mt.c b/arch/mips/kernel/smp-mt.c
index e398cbc3d776..ed6b4df583ea 100644
--- a/arch/mips/kernel/smp-mt.c
+++ b/arch/mips/kernel/smp-mt.c
@@ -83,6 +83,8 @@ static unsigned int __init smvp_vpe_init(unsigned int tc, unsigned int mvpconf0,
if (tc != 0)
smvp_copy_vpe_config();
+ cpu_data[ncpu].vpe_id = tc;
+
return ncpu;
}
@@ -114,49 +116,6 @@ static void __init smvp_tc_init(unsigned int tc, unsigned int mvpconf0)
write_tc_c0_tchalt(TCHALT_H);
}
-static void vsmp_send_ipi_single(int cpu, unsigned int action)
-{
- int i;
- unsigned long flags;
- int vpflags;
-
-#ifdef CONFIG_MIPS_GIC
- if (gic_present) {
- mips_smp_send_ipi_single(cpu, action);
- return;
- }
-#endif
- local_irq_save(flags);
-
- vpflags = dvpe(); /* can't access the other CPU's registers whilst MVPE enabled */
-
- switch (action) {
- case SMP_CALL_FUNCTION:
- i = C_SW1;
- break;
-
- case SMP_RESCHEDULE_YOURSELF:
- default:
- i = C_SW0;
- break;
- }
-
- /* 1:1 mapping of vpe and tc... */
- settc(cpu);
- write_vpe_c0_cause(read_vpe_c0_cause() | i);
- evpe(vpflags);
-
- local_irq_restore(flags);
-}
-
-static void vsmp_send_ipi_mask(const struct cpumask *mask, unsigned int action)
-{
- unsigned int i;
-
- for_each_cpu(i, mask)
- vsmp_send_ipi_single(i, action);
-}
-
static void vsmp_init_secondary(void)
{
#ifdef CONFIG_MIPS_GIC
@@ -281,8 +240,8 @@ static void __init vsmp_prepare_cpus(unsigned int max_cpus)
}
struct plat_smp_ops vsmp_smp_ops = {
- .send_ipi_single = vsmp_send_ipi_single,
- .send_ipi_mask = vsmp_send_ipi_mask,
+ .send_ipi_single = mips_smp_send_ipi_single,
+ .send_ipi_mask = mips_smp_send_ipi_mask,
.init_secondary = vsmp_init_secondary,
.smp_finish = vsmp_smp_finish,
.boot_secondary = vsmp_boot_secondary,
diff --git a/arch/mips/lantiq/irq.c b/arch/mips/lantiq/irq.c
index 0ddf3698b85d..33728b7af426 100644
--- a/arch/mips/lantiq/irq.c
+++ b/arch/mips/lantiq/irq.c
@@ -274,47 +274,6 @@ static void ltq_hw_irq_handler(struct irq_desc *desc)
ltq_hw_irqdispatch(irq_desc_get_irq(desc) - 2);
}
-#ifdef CONFIG_MIPS_MT_SMP
-void __init arch_init_ipiirq(int irq, struct irqaction *action)
-{
- setup_irq(irq, action);
- irq_set_handler(irq, handle_percpu_irq);
-}
-
-static void ltq_sw0_irqdispatch(void)
-{
- do_IRQ(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_RESCHED_IRQ);
-}
-
-static void ltq_sw1_irqdispatch(void)
-{
- do_IRQ(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_CALL_IRQ);
-}
-static irqreturn_t ipi_resched_interrupt(int irq, void *dev_id)
-{
- scheduler_ipi();
- return IRQ_HANDLED;
-}
-
-static irqreturn_t ipi_call_interrupt(int irq, void *dev_id)
-{
- generic_smp_call_function_interrupt();
- return IRQ_HANDLED;
-}
-
-static struct irqaction irq_resched = {
- .handler = ipi_resched_interrupt,
- .flags = IRQF_PERCPU,
- .name = "IPI_resched"
-};
-
-static struct irqaction irq_call = {
- .handler = ipi_call_interrupt,
- .flags = IRQF_PERCPU,
- .name = "IPI_call"
-};
-#endif
-
asmlinkage void plat_irq_dispatch(void)
{
unsigned int pending = read_c0_status() & read_c0_cause() & ST0_IM;
@@ -402,17 +361,6 @@ int __init icu_of_init(struct device_node *node, struct device_node *parent)
(MAX_IM * INT_NUM_IM_OFFSET) + MIPS_CPU_IRQ_CASCADE,
&irq_domain_ops, 0);
-#if defined(CONFIG_MIPS_MT_SMP)
- if (cpu_has_vint) {
- pr_info("Setting up IPI vectored interrupts\n");
- set_vi_handler(MIPS_CPU_IPI_RESCHED_IRQ, ltq_sw0_irqdispatch);
- set_vi_handler(MIPS_CPU_IPI_CALL_IRQ, ltq_sw1_irqdispatch);
- }
- arch_init_ipiirq(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_RESCHED_IRQ,
- &irq_resched);
- arch_init_ipiirq(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_CALL_IRQ, &irq_call);
-#endif
-
#ifndef CONFIG_MIPS_MT_SMP
set_c0_status(IE_IRQ0 | IE_IRQ1 | IE_IRQ2 |
IE_IRQ3 | IE_IRQ4 | IE_IRQ5);
diff --git a/arch/mips/mti-malta/malta-int.c b/arch/mips/mti-malta/malta-int.c
index cb675ec6f283..fe9bb479f2a0 100644
--- a/arch/mips/mti-malta/malta-int.c
+++ b/arch/mips/mti-malta/malta-int.c
@@ -145,56 +145,6 @@ static irqreturn_t corehi_handler(int irq, void *dev_id)
return IRQ_HANDLED;
}
-#ifdef CONFIG_MIPS_MT_SMP
-
-#define MIPS_CPU_IPI_RESCHED_IRQ 0 /* SW int 0 for resched */
-#define C_RESCHED C_SW0
-#define MIPS_CPU_IPI_CALL_IRQ 1 /* SW int 1 for resched */
-#define C_CALL C_SW1
-static int cpu_ipi_resched_irq, cpu_ipi_call_irq;
-
-static void ipi_resched_dispatch(void)
-{
- do_IRQ(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_RESCHED_IRQ);
-}
-
-static void ipi_call_dispatch(void)
-{
- do_IRQ(MIPS_CPU_IRQ_BASE + MIPS_CPU_IPI_CALL_IRQ);
-}
-
-static irqreturn_t ipi_resched_interrupt(int irq, void *dev_id)
-{
-#ifdef CONFIG_MIPS_VPE_APSP_API_CMP
- if (aprp_hook)
- aprp_hook();
-#endif
-
- scheduler_ipi();
-
- return IRQ_HANDLED;
-}
-
-static irqreturn_t ipi_call_interrupt(int irq, void *dev_id)
-{
- generic_smp_call_function_interrupt();
-
- return IRQ_HANDLED;
-}
-
-static struct irqaction irq_resched = {
- .handler = ipi_resched_interrupt,
- .flags = IRQF_PERCPU,
- .name = "IPI_resched"
-};
-
-static struct irqaction irq_call = {
- .handler = ipi_call_interrupt,
- .flags = IRQF_PERCPU,
- .name = "IPI_call"
-};
-#endif /* CONFIG_MIPS_MT_SMP */
-
static struct irqaction corehi_irqaction = {
.handler = corehi_handler,
.name = "CoreHi",
@@ -222,12 +172,6 @@ static msc_irqmap_t msc_eicirqmap[] __initdata = {
static int msc_nr_eicirqs __initdata = ARRAY_SIZE(msc_eicirqmap);
-void __init arch_init_ipiirq(int irq, struct irqaction *action)
-{
- setup_irq(irq, action);
- irq_set_handler(irq, handle_percpu_irq);
-}
-
void __init arch_init_irq(void)
{
int corehi_irq;
@@ -262,30 +206,11 @@ void __init arch_init_irq(void)
if (gic_present) {
corehi_irq = MIPS_CPU_IRQ_BASE + MIPSCPU_INT_COREHI;
+ } else if (cpu_has_veic) {
+ set_vi_handler(MSC01E_INT_COREHI, corehi_irqdispatch);
+ corehi_irq = MSC01E_INT_BASE + MSC01E_INT_COREHI;
} else {
-#if defined(CONFIG_MIPS_MT_SMP)
- /* set up ipi interrupts */
- if (cpu_has_veic) {
- set_vi_handler (MSC01E_INT_SW0, ipi_resched_dispatch);
- set_vi_handler (MSC01E_INT_SW1, ipi_call_dispatch);
- cpu_ipi_resched_irq = MSC01E_INT_SW0;
- cpu_ipi_call_irq = MSC01E_INT_SW1;
- } else {
- cpu_ipi_resched_irq = MIPS_CPU_IRQ_BASE +
- MIPS_CPU_IPI_RESCHED_IRQ;
- cpu_ipi_call_irq = MIPS_CPU_IRQ_BASE +
- MIPS_CPU_IPI_CALL_IRQ;
- }
- arch_init_ipiirq(cpu_ipi_resched_irq, &irq_resched);
- arch_init_ipiirq(cpu_ipi_call_irq, &irq_call);
-#endif
- if (cpu_has_veic) {
- set_vi_handler(MSC01E_INT_COREHI,
- corehi_irqdispatch);
- corehi_irq = MSC01E_INT_BASE + MSC01E_INT_COREHI;
- } else {
- corehi_irq = MIPS_CPU_IRQ_BASE + MIPSCPU_INT_COREHI;
- }
+ corehi_irq = MIPS_CPU_IRQ_BASE + MIPSCPU_INT_COREHI;
}
setup_irq(corehi_irq, &corehi_irqaction);
--
2.12.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 5/5] MIPS: Stengthen IPI IRQ domain sanity check
2017-03-30 19:06 [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs Paul Burton
` (4 preceding siblings ...)
2017-03-30 19:06 ` [PATCH 4/5] MIPS: smp-mt: Use CPU interrupt controller " Paul Burton
@ 2017-03-30 19:06 ` Paul Burton
2017-03-30 19:06 ` Paul Burton
2017-03-31 9:02 ` [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs Thomas Gleixner
2017-03-31 21:31 ` Joshua Kinard
7 siblings, 1 reply; 15+ messages in thread
From: Paul Burton @ 2017-03-30 19:06 UTC (permalink / raw)
To: Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper, Paul Burton
Commit fbde2d7d8290 ("MIPS: Add generic SMP IPI support") introduced a
sanity check that an IPI IRQ domain can be found during boot, in order
to ensure that IPIs are able to be set up in systems using such domains.
However it was added at a point where systems may have used an IPI IRQ
domain in some situations but not others, and we could not know which
were the case until runtime, so commit 578bffc82ec5 ("MIPS: Don't BUG_ON
when no IPI domain is found") made that check simply skip IPI init if no
domain were found in order to fix the boot for systems such as QEMU
Malta.
We now use IPI IRQ domains for the MIPS CPU interrupt controller, which
means systems which make use of IPI IRQ domains will always do so when
running on multiple CPUs. As a result we now strengthen the sanity check
to ensure that an IPI IRQ domain is found when multiple CPUs are present
in the system.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/kernel/smp.c | 20 ++++++++++++--------
1 file changed, 12 insertions(+), 8 deletions(-)
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 6e71130549ea..aba1afb64b62 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -261,16 +261,20 @@ int mips_smp_ipi_allocate(const struct cpumask *mask)
ipidomain = irq_find_matching_host(NULL, DOMAIN_BUS_IPI);
/*
- * There are systems which only use IPI domains some of the time,
- * depending upon configuration we don't know until runtime. An
- * example is Malta where we may compile in support for GIC & the
- * MT ASE, but run on a system which has multiple VPEs in a single
- * core and doesn't include a GIC. Until all IPI implementations
- * have been converted to use IPI domains the best we can do here
- * is to return & hope some other code sets up the IPIs.
+ * There are systems which use IPI IRQ domains, but only have one
+ * registered when some runtime condition is met. For example a Malta
+ * kernel may include support for GIC & CPU interrupt controller IPI
+ * IRQ domains, but if run on a system with no GIC & no MT ASE then
+ * neither will be supported or registered.
+ *
+ * We only have a problem if we're actually using multiple CPUs so fail
+ * loudly if that is the case. Otherwise simply return, skipping IPI
+ * setup, if we're running with only a single CPU.
*/
- if (!ipidomain)
+ if (!ipidomain) {
+ BUG_ON(num_present_cpus() > 1);
return 0;
+ }
virq = irq_reserve_ipi(ipidomain, mask);
BUG_ON(!virq);
--
2.12.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH 5/5] MIPS: Stengthen IPI IRQ domain sanity check
2017-03-30 19:06 ` [PATCH 5/5] MIPS: Stengthen IPI IRQ domain sanity check Paul Burton
@ 2017-03-30 19:06 ` Paul Burton
0 siblings, 0 replies; 15+ messages in thread
From: Paul Burton @ 2017-03-30 19:06 UTC (permalink / raw)
To: Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper, Paul Burton
Commit fbde2d7d8290 ("MIPS: Add generic SMP IPI support") introduced a
sanity check that an IPI IRQ domain can be found during boot, in order
to ensure that IPIs are able to be set up in systems using such domains.
However it was added at a point where systems may have used an IPI IRQ
domain in some situations but not others, and we could not know which
were the case until runtime, so commit 578bffc82ec5 ("MIPS: Don't BUG_ON
when no IPI domain is found") made that check simply skip IPI init if no
domain were found in order to fix the boot for systems such as QEMU
Malta.
We now use IPI IRQ domains for the MIPS CPU interrupt controller, which
means systems which make use of IPI IRQ domains will always do so when
running on multiple CPUs. As a result we now strengthen the sanity check
to ensure that an IPI IRQ domain is found when multiple CPUs are present
in the system.
Signed-off-by: Paul Burton <paul.burton@imgtec.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Jason Cooper <jason@lakedaemon.net>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: linux-mips@linux-mips.org
---
arch/mips/kernel/smp.c | 20 ++++++++++++--------
1 file changed, 12 insertions(+), 8 deletions(-)
diff --git a/arch/mips/kernel/smp.c b/arch/mips/kernel/smp.c
index 6e71130549ea..aba1afb64b62 100644
--- a/arch/mips/kernel/smp.c
+++ b/arch/mips/kernel/smp.c
@@ -261,16 +261,20 @@ int mips_smp_ipi_allocate(const struct cpumask *mask)
ipidomain = irq_find_matching_host(NULL, DOMAIN_BUS_IPI);
/*
- * There are systems which only use IPI domains some of the time,
- * depending upon configuration we don't know until runtime. An
- * example is Malta where we may compile in support for GIC & the
- * MT ASE, but run on a system which has multiple VPEs in a single
- * core and doesn't include a GIC. Until all IPI implementations
- * have been converted to use IPI domains the best we can do here
- * is to return & hope some other code sets up the IPIs.
+ * There are systems which use IPI IRQ domains, but only have one
+ * registered when some runtime condition is met. For example a Malta
+ * kernel may include support for GIC & CPU interrupt controller IPI
+ * IRQ domains, but if run on a system with no GIC & no MT ASE then
+ * neither will be supported or registered.
+ *
+ * We only have a problem if we're actually using multiple CPUs so fail
+ * loudly if that is the case. Otherwise simply return, skipping IPI
+ * setup, if we're running with only a single CPU.
*/
- if (!ipidomain)
+ if (!ipidomain) {
+ BUG_ON(num_present_cpus() > 1);
return 0;
+ }
virq = irq_reserve_ipi(ipidomain, mask);
BUG_ON(!virq);
--
2.12.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs
2017-03-30 19:06 [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs Paul Burton
` (5 preceding siblings ...)
2017-03-30 19:06 ` [PATCH 5/5] MIPS: Stengthen IPI IRQ domain sanity check Paul Burton
@ 2017-03-31 9:02 ` Thomas Gleixner
2017-04-12 21:14 ` Ralf Baechle
2017-03-31 21:31 ` Joshua Kinard
7 siblings, 1 reply; 15+ messages in thread
From: Thomas Gleixner @ 2017-03-31 9:02 UTC (permalink / raw)
To: Paul Burton; +Cc: Ralf Baechle, linux-mips, Marc Zyngier, Jason Cooper
On Thu, 30 Mar 2017, Paul Burton wrote:
> This series introduces support for IPI IRQ domains to the CPU interrupt
> controller driver, allowing IPIs to function in the same way as those
> provided by the MIPS GIC as far as platform/board code is concerned.
>
> Doing this allows us to avoid duplicating code across platforms, avoid
> having to handle cases where IPI domains are or aren't in use depending
> upon the interrupt controller, and strengthen a sanity check for cases
> where IPI IRQ domains are supported.
For the irqchip parts:
Acked-by: Thomas Gleixner <tglx@linutronix.de>
Ralf, feel free to route the whole lot through your MIPS tree.
Thanks,
tglx
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs
2017-03-31 9:02 ` [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs Thomas Gleixner
@ 2017-04-12 21:14 ` Ralf Baechle
0 siblings, 0 replies; 15+ messages in thread
From: Ralf Baechle @ 2017-04-12 21:14 UTC (permalink / raw)
To: Thomas Gleixner; +Cc: Paul Burton, linux-mips, Marc Zyngier, Jason Cooper
On Fri, Mar 31, 2017 at 11:02:33AM +0200, Thomas Gleixner wrote:
> On Thu, 30 Mar 2017, Paul Burton wrote:
>
> > This series introduces support for IPI IRQ domains to the CPU interrupt
> > controller driver, allowing IPIs to function in the same way as those
> > provided by the MIPS GIC as far as platform/board code is concerned.
> >
> > Doing this allows us to avoid duplicating code across platforms, avoid
> > having to handle cases where IPI domains are or aren't in use depending
> > upon the interrupt controller, and strengthen a sanity check for cases
> > where IPI IRQ domains are supported.
>
> For the irqchip parts:
>
> Acked-by: Thomas Gleixner <tglx@linutronix.de>
>
> Ralf, feel free to route the whole lot through your MIPS tree.
Done.
Nice cleanup, Paul.
Ralf
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs
2017-03-30 19:06 [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs Paul Burton
` (6 preceding siblings ...)
2017-03-31 9:02 ` [PATCH 0/5] MIPS/irqchip: Use IPI IRQ domains for CPU interrupt controller IPIs Thomas Gleixner
@ 2017-03-31 21:31 ` Joshua Kinard
7 siblings, 0 replies; 15+ messages in thread
From: Joshua Kinard @ 2017-03-31 21:31 UTC (permalink / raw)
To: Paul Burton, Ralf Baechle, linux-mips
Cc: Thomas Gleixner, Marc Zyngier, Jason Cooper
On 03/30/2017 15:06, Paul Burton wrote:
> This series introduces support for IPI IRQ domains to the CPU interrupt
> controller driver, allowing IPIs to function in the same way as those
> provided by the MIPS GIC as far as platform/board code is concerned.
>
> Doing this allows us to avoid duplicating code across platforms, avoid
> having to handle cases where IPI domains are or aren't in use depending
> upon the interrupt controller, and strengthen a sanity check for cases
> where IPI IRQ domains are supported.
>
> Applies atop v4.11-rc4.
>
>
> Paul Burton (5):
> irqchip: mips-cpu: Replace magic 0x100 with IE_SW0
> irqchip: mips-cpu: Prepare for non-legacy IRQ domains
> irqchip: mips-cpu: Introduce IPI IRQ domain support
> MIPS: smp-mt: Use CPU interrupt controller IPI IRQ domain support
> MIPS: Stengthen IPI IRQ domain sanity check
>
> arch/mips/kernel/smp-mt.c | 49 ++------------
> arch/mips/kernel/smp.c | 20 +++---
> arch/mips/lantiq/irq.c | 52 --------------
> arch/mips/mti-malta/malta-int.c | 83 ++---------------------
> drivers/irqchip/Kconfig | 2 +
> drivers/irqchip/irq-mips-cpu.c | 146 +++++++++++++++++++++++++++++++++++-----
> 6 files changed, 151 insertions(+), 201 deletions(-)
Out of curiosity, "legacy" systems like SGI IP27 (in-tree) and IP30 (external)
support SMP and the IRQ handling is fairly old for IP27 (IP30 borrows IP27's
logic). Could these systems benefit from using IPI domains? If so, is there
any kind of crash-course or dummies guide to switching from plain irq_chip to
IPI domains? Note, both systems have somewhat unique interrupt controllers
built into their system ASICs, but actual IRQ dispatch happens from the CPU
interrupt pins.
Thanks!,
--
Joshua Kinard
Gentoo/MIPS
kumba@gentoo.org
6144R/F5C6C943 2015-04-27
177C 1972 1FB8 F254 BAD0 3E72 5C63 F4E3 F5C6 C943
"The past tempts us, the present confuses us, the future frightens us. And our
lives slip away, moment by moment, lost in that vast, terrible in-between."
--Emperor Turhan, Centauri Republic
^ permalink raw reply [flat|nested] 15+ messages in thread