[PATCH 0/2] irq,xen: fix event channel masking on suspend/resume

[PATCH 0/2] irq,xen: fix event channel masking on suspend/resume

[Juergen Gross]

Commit bf22ff45bed664aefb5c4e43029057a199b7070c ("genirq: Avoid
unnecessary low level irq function calls") broke Xen suspend/resume
handling as Xen fiddled with masking/unmasking of event channels (or
irqs) without letting the irq subsystem know about it.

Fix this by setting the correct states in irq and remove the masking
from Xen.

Juergen Gross (2):
  irq: adjust state of irq in resume_irq() when IRQF_FORCE_RESUME set
  xen: dont fiddle with event channel masking in suspend/resume

 drivers/xen/events/events_base.c | 13 +++----------
 kernel/irq/chip.c                | 10 ----------
 kernel/irq/internals.h           | 10 ++++++++++
 kernel/irq/pm.c                  |  2 ++
 4 files changed, 15 insertions(+), 20 deletions(-)

-- 
2.12.3

[PATCH 1/2] irq: adjust state of irq in resume_irq() when IRQF_FORCE_RESUME set

[Juergen Gross]

An irq pretending it got disabled should have its state set
accordingly. Otherwise a potential unmask later won't succeed.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 kernel/irq/chip.c      | 10 ----------
 kernel/irq/internals.h | 10 ++++++++++
 kernel/irq/pm.c        |  2 ++
 3 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index d171bc57e1e0..a3cc37c0c85e 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -170,21 +170,11 @@ static void irq_state_clr_disabled(struct irq_desc *desc)
 	irqd_clear(&desc->irq_data, IRQD_IRQ_DISABLED);
 }
 
-static void irq_state_set_disabled(struct irq_desc *desc)
-{
-	irqd_set(&desc->irq_data, IRQD_IRQ_DISABLED);
-}
-
 static void irq_state_clr_masked(struct irq_desc *desc)
 {
 	irqd_clear(&desc->irq_data, IRQD_IRQ_MASKED);
 }
 
-static void irq_state_set_masked(struct irq_desc *desc)
-{
-	irqd_set(&desc->irq_data, IRQD_IRQ_MASKED);
-}
-
 static void irq_state_clr_started(struct irq_desc *desc)
 {
 	irqd_clear(&desc->irq_data, IRQD_IRQ_STARTED);
diff --git a/kernel/irq/internals.h b/kernel/irq/internals.h
index dbfba9933ed2..a2c48058354c 100644
--- a/kernel/irq/internals.h
+++ b/kernel/irq/internals.h
@@ -227,6 +227,16 @@ static inline bool irqd_has_set(struct irq_data *d, unsigned int mask)
 	return __irqd_to_state(d) & mask;
 }
 
+static inline void irq_state_set_disabled(struct irq_desc *desc)
+{
+	irqd_set(&desc->irq_data, IRQD_IRQ_DISABLED);
+}
+
+static inline void irq_state_set_masked(struct irq_desc *desc)
+{
+	irqd_set(&desc->irq_data, IRQD_IRQ_MASKED);
+}
+
 #undef __irqd_to_state
 
 static inline void kstat_incr_irqs_this_cpu(struct irq_desc *desc)
diff --git a/kernel/irq/pm.c b/kernel/irq/pm.c
index cea1de0161f1..6bd9b58429cc 100644
--- a/kernel/irq/pm.c
+++ b/kernel/irq/pm.c
@@ -149,6 +149,8 @@ static void resume_irq(struct irq_desc *desc)
 
 	/* Pretend that it got disabled ! */
 	desc->depth++;
+	irq_state_set_disabled(desc);
+	irq_state_set_masked(desc);
 resume:
 	desc->istate &= ~IRQS_SUSPENDED;
 	__enable_irq(desc);
-- 
2.12.3

[PATCH 2/2] xen: dont fiddle with event channel masking in suspend/resume

[Juergen Gross]

Instead of fiddling with masking the event channels during suspend
and resume handling let do the irq subsystem do its job. It will do
the mask and unmask operations as needed.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 drivers/xen/events/events_base.c | 13 +++----------
 1 file changed, 3 insertions(+), 10 deletions(-)

diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c
index b241bfa529ce..bae1f5d36c26 100644
--- a/drivers/xen/events/events_base.c
+++ b/drivers/xen/events/events_base.c
@@ -343,14 +343,6 @@ static void bind_evtchn_to_cpu(unsigned int chn, unsigned int cpu)
 	info->cpu = cpu;
 }
 
-static void xen_evtchn_mask_all(void)
-{
-	unsigned int evtchn;
-
-	for (evtchn = 0; evtchn < xen_evtchn_nr_channels(); evtchn++)
-		mask_evtchn(evtchn);
-}
-
 /**
  * notify_remote_via_irq - send event to remote end of event channel via irq
  * @irq: irq of event channel to send event to
@@ -1573,7 +1565,6 @@ void xen_irq_resume(void)
 	struct irq_info *info;
 
 	/* New event-channel space is not 'live' yet. */
-	xen_evtchn_mask_all();
 	xen_evtchn_resume();
 
 	/* No IRQ <-> event-channel mappings. */
@@ -1681,6 +1672,7 @@ module_param(fifo_events, bool, 0);
 void __init xen_init_IRQ(void)
 {
 	int ret = -EINVAL;
+	unsigned int evtchn;
 
 	if (fifo_events)
 		ret = xen_evtchn_fifo_init();
@@ -1692,7 +1684,8 @@ void __init xen_init_IRQ(void)
 	BUG_ON(!evtchn_to_irq);
 
 	/* No event channels are 'live' right now. */
-	xen_evtchn_mask_all();
+	for (evtchn = 0; evtchn < xen_evtchn_nr_channels(); evtchn++)
+		mask_evtchn(evtchn);
 
 	pirq_needs_eoi = pirq_needs_eoi_flag;
 
-- 
2.12.3

[tip:irq/urgent] genirq/PM: Properly pretend disabled state when force resuming interrupts

[tip-bot for Juergen Gross]

Commit-ID:  a696712c3dd54eb58d2c5a807b4aaa27782d80d6
Gitweb:     http://git.kernel.org/tip/a696712c3dd54eb58d2c5a807b4aaa27782d80d6
Author:     Juergen Gross <jgross@suse.com>
AuthorDate: Mon, 17 Jul 2017 19:47:02 +0200
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Mon, 17 Jul 2017 22:32:20 +0200

genirq/PM: Properly pretend disabled state when force resuming interrupts

Interrupts with the IRQF_FORCE_RESUME flag set have also the
IRQF_NO_SUSPEND flag set. They are not disabled in the suspend path, but
must be forcefully resumed. That's used by XEN to keep IPIs enabled beyond
the suspension of device irqs. Force resume works by pretending that the
interrupt was disabled and then calling __irq_enable().

Incrementing the disabled depth counter was enough to do that, but with the
recent changes which use state flags to avoid unnecessary hardware access,
this is not longer sufficient. If the state flags are not set, then the
hardware callbacks are not invoked and the interrupt line stays disabled in
"hardware".

Set the disabled and masked state when pretending that an interrupt got
disabled by suspend.

Fixes: bf22ff45bed6 ("genirq: Avoid unnecessary low level irq function calls")
Suggested-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Juergen Gross <jgross@suse.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: xen-devel@lists.xenproject.org
Cc: boris.ostrovsky@oracle.com
Link: http://lkml.kernel.org/r/20170717174703.4603-2-jgross@suse.com
---
 kernel/irq/chip.c      | 10 ----------
 kernel/irq/internals.h | 10 ++++++++++
 kernel/irq/pm.c        |  2 ++
 3 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c
index d171bc5..a3cc37c 100644
--- a/kernel/irq/chip.c
+++ b/kernel/irq/chip.c
@@ -170,21 +170,11 @@ static void irq_state_clr_disabled(struct irq_desc *desc)
 	irqd_clear(&desc->irq_data, IRQD_IRQ_DISABLED);
 }
 
-static void irq_state_set_disabled(struct irq_desc *desc)
-{
-	irqd_set(&desc->irq_data, IRQD_IRQ_DISABLED);
-}
-
 static void irq_state_clr_masked(struct irq_desc *desc)
 {
 	irqd_clear(&desc->irq_data, IRQD_IRQ_MASKED);
 }
 
-static void irq_state_set_masked(struct irq_desc *desc)
-{
-	irqd_set(&desc->irq_data, IRQD_IRQ_MASKED);
-}
-
 static void irq_state_clr_started(struct irq_desc *desc)
 {
 	irqd_clear(&desc->irq_data, IRQD_IRQ_STARTED);
diff --git a/kernel/irq/internals.h b/kernel/irq/internals.h
index dbfba99..a2c4805 100644
--- a/kernel/irq/internals.h
+++ b/kernel/irq/internals.h
@@ -227,6 +227,16 @@ static inline bool irqd_has_set(struct irq_data *d, unsigned int mask)
 	return __irqd_to_state(d) & mask;
 }
 
+static inline void irq_state_set_disabled(struct irq_desc *desc)
+{
+	irqd_set(&desc->irq_data, IRQD_IRQ_DISABLED);
+}
+
+static inline void irq_state_set_masked(struct irq_desc *desc)
+{
+	irqd_set(&desc->irq_data, IRQD_IRQ_MASKED);
+}
+
 #undef __irqd_to_state
 
 static inline void kstat_incr_irqs_this_cpu(struct irq_desc *desc)
diff --git a/kernel/irq/pm.c b/kernel/irq/pm.c
index cea1de0..6bd9b58 100644
--- a/kernel/irq/pm.c
+++ b/kernel/irq/pm.c
@@ -149,6 +149,8 @@ static void resume_irq(struct irq_desc *desc)
 
 	/* Pretend that it got disabled ! */
 	desc->depth++;
+	irq_state_set_disabled(desc);
+	irq_state_set_masked(desc);
 resume:
 	desc->istate &= ~IRQS_SUSPENDED;
 	__enable_irq(desc);

Re: [PATCH 2/2] xen: dont fiddle with event channel masking in suspend/resume

[Thomas Gleixner]

On Mon, 17 Jul 2017, Juergen Gross wrote:

> Instead of fiddling with masking the event channels during suspend
> and resume handling let do the irq subsystem do its job. It will do
> the mask and unmask operations as needed.
> 
> Signed-off-by: Juergen Gross <jgross@suse.com>

Acked-by: Thomas Gleixner <tglx@linutronix.de>

Re: [PATCH 2/2] xen: dont fiddle with event channel masking in suspend/resume

[Boris Ostrovsky]

On 07/17/2017 01:47 PM, Juergen Gross wrote:
> Instead of fiddling with masking the event channels during suspend
> and resume handling let do the irq subsystem do its job. It will do
> the mask and unmask operations as needed.
>
> Signed-off-by: Juergen Gross <jgross@suse.com>

Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>

(I've been running dumpdata with this for the last couple of days so you
can add Tested-by as well if you wish)