Re: [PATCH] xen/events: Mask a moving irq

Re: [PATCH] xen/events: Mask a moving irq

[David Vrabel]

On 17/03/16 12:45, Boris Ostrovsky wrote:
> Moving an unmasked irq may result in irq handler being invoked on both
> source and target CPUs.
> 
> With 2-level this can happen as follows:
> 
> On source CPU:
>         evtchn_2l_handle_events() ->
>             generic_handle_irq() ->
>                 handle_edge_irq() ->
>                    eoi_pirq():
>                        irq_move_irq(data);
> 
>                        /***** WE ARE HERE *****/
> 
>                        if (VALID_EVTCHN(evtchn))
>                            clear_evtchn(evtchn);
> 
> If at this moment target processor is handling an unrelated event in
> evtchn_2l_handle_events()'s loop it may pick up our event since target's
> cpu_evtchn_mask claims that this event belongs to it *and* the event is
> unmasked and still pending. At the same time, source CPU will continue
> executing its own handle_edge_irq().
> 
> With FIFO interrupt the scenario is similar: irq_move_irq() may result
> in a EVTCHNOP_unmask hypercall which, in turn, may make the event
> pending on the target CPU.
> 
> We can avoid this situation by moving and clearing the event while
> keeping event masked.

Can you do:

   if (unlikely(irqd_is_setaffinity_pending(data))) {
       masked = test_and_set_mask()

       clear_evtchn()
       irq_move_masked_irq()

       unmask(masked);
   } else
       clear_evtchn()

David

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] xen/events: Mask a moving irq

[Boris Ostrovsky]

On 03/17/2016 12:03 PM, David Vrabel wrote:
> On 17/03/16 12:45, Boris Ostrovsky wrote:
>> Moving an unmasked irq may result in irq handler being invoked on both
>> source and target CPUs.
>>
>> With 2-level this can happen as follows:
>>
>> On source CPU:
>>          evtchn_2l_handle_events() ->
>>              generic_handle_irq() ->
>>                  handle_edge_irq() ->
>>                     eoi_pirq():
>>                         irq_move_irq(data);
>>
>>                         /***** WE ARE HERE *****/
>>
>>                         if (VALID_EVTCHN(evtchn))
>>                             clear_evtchn(evtchn);
>>
>> If at this moment target processor is handling an unrelated event in
>> evtchn_2l_handle_events()'s loop it may pick up our event since target's
>> cpu_evtchn_mask claims that this event belongs to it *and* the event is
>> unmasked and still pending. At the same time, source CPU will continue
>> executing its own handle_edge_irq().
>>
>> With FIFO interrupt the scenario is similar: irq_move_irq() may result
>> in a EVTCHNOP_unmask hypercall which, in turn, may make the event
>> pending on the target CPU.
>>
>> We can avoid this situation by moving and clearing the event while
>> keeping event masked.
> Can you do:
>
>     if (unlikely(irqd_is_setaffinity_pending(data))) {
>         masked = test_and_set_mask()
>
>         clear_evtchn()
>         irq_move_masked_irq()

I did think about this but then I wasn't sure whether this might open 
some other window for things to sneak in. It shouldn't but these things 
are rather subtle so I'd rather leave the order of how operations are 
done unchanged.

But I should indeed use irq_move_masked_irq() instead of irq_move_irq().

-boris

>
>         unmask(masked);
>     } else
>         clear_evtchn()




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] xen/events: Mask a moving irq

[David Vrabel]

On 17/03/16 16:53, Boris Ostrovsky wrote:
> On 03/17/2016 12:03 PM, David Vrabel wrote:
>> On 17/03/16 12:45, Boris Ostrovsky wrote:
>>> Moving an unmasked irq may result in irq handler being invoked on both
>>> source and target CPUs.
>>>
>>> With 2-level this can happen as follows:
>>>
>>> On source CPU:
>>>          evtchn_2l_handle_events() ->
>>>              generic_handle_irq() ->
>>>                  handle_edge_irq() ->
>>>                     eoi_pirq():
>>>                         irq_move_irq(data);
>>>
>>>                         /***** WE ARE HERE *****/
>>>
>>>                         if (VALID_EVTCHN(evtchn))
>>>                             clear_evtchn(evtchn);
>>>
>>> If at this moment target processor is handling an unrelated event in
>>> evtchn_2l_handle_events()'s loop it may pick up our event since target's
>>> cpu_evtchn_mask claims that this event belongs to it *and* the event is
>>> unmasked and still pending. At the same time, source CPU will continue
>>> executing its own handle_edge_irq().
>>>
>>> With FIFO interrupt the scenario is similar: irq_move_irq() may result
>>> in a EVTCHNOP_unmask hypercall which, in turn, may make the event
>>> pending on the target CPU.
>>>
>>> We can avoid this situation by moving and clearing the event while
>>> keeping event masked.
>> Can you do:
>>
>>     if (unlikely(irqd_is_setaffinity_pending(data))) {
>>         masked = test_and_set_mask()
>>
>>         clear_evtchn()
>>         irq_move_masked_irq()
> 
> I did think about this but then I wasn't sure whether this might open
> some other window for things to sneak in. It shouldn't but these things
> are rather subtle so I'd rather leave the order of how operations are
> done unchanged.

This is the order your patch has though.  I'm confused.

> But I should indeed use irq_move_masked_irq() instead of irq_move_irq().

David

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] xen/events: Mask a moving irq

[Boris Ostrovsky]

On 03/17/2016 01:29 PM, David Vrabel wrote:
> On 17/03/16 16:53, Boris Ostrovsky wrote:
>> On 03/17/2016 12:03 PM, David Vrabel wrote:
>>> On 17/03/16 12:45, Boris Ostrovsky wrote:
>>>> Moving an unmasked irq may result in irq handler being invoked on both
>>>> source and target CPUs.
>>>>
>>>> With 2-level this can happen as follows:
>>>>
>>>> On source CPU:
>>>>           evtchn_2l_handle_events() ->
>>>>               generic_handle_irq() ->
>>>>                   handle_edge_irq() ->
>>>>                      eoi_pirq():
>>>>                          irq_move_irq(data);
>>>>
>>>>                          /***** WE ARE HERE *****/
>>>>
>>>>                          if (VALID_EVTCHN(evtchn))
>>>>                              clear_evtchn(evtchn);
>>>>
>>>> If at this moment target processor is handling an unrelated event in
>>>> evtchn_2l_handle_events()'s loop it may pick up our event since target's
>>>> cpu_evtchn_mask claims that this event belongs to it *and* the event is
>>>> unmasked and still pending. At the same time, source CPU will continue
>>>> executing its own handle_edge_irq().
>>>>
>>>> With FIFO interrupt the scenario is similar: irq_move_irq() may result
>>>> in a EVTCHNOP_unmask hypercall which, in turn, may make the event
>>>> pending on the target CPU.
>>>>
>>>> We can avoid this situation by moving and clearing the event while
>>>> keeping event masked.
>>> Can you do:
>>>
>>>      if (unlikely(irqd_is_setaffinity_pending(data))) {
>>>          masked = test_and_set_mask()
>>>
>>>          clear_evtchn()
>>>          irq_move_masked_irq()
>> I did think about this but then I wasn't sure whether this might open
>> some other window for things to sneak in. It shouldn't but these things
>> are rather subtle so I'd rather leave the order of how operations are
>> done unchanged.
> This is the order your patch has though.  I'm confused.

Ugh, sorry --- I misread what you wrote, I thought you wanted to clear 
before masking. Which wouldn't make any sense.

So yes, what you are suggesting is better.

-borsi

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH] xen/events: Mask a moving irq

[Boris Ostrovsky]

Moving an unmasked irq may result in irq handler being invoked on both
source and target CPUs.

With 2-level this can happen as follows:

On source CPU:
        evtchn_2l_handle_events() ->
            generic_handle_irq() ->
                handle_edge_irq() ->
                   eoi_pirq():
                       irq_move_irq(data);

                       /***** WE ARE HERE *****/

                       if (VALID_EVTCHN(evtchn))
                           clear_evtchn(evtchn);

If at this moment target processor is handling an unrelated event in
evtchn_2l_handle_events()'s loop it may pick up our event since target's
cpu_evtchn_mask claims that this event belongs to it *and* the event is
unmasked and still pending. At the same time, source CPU will continue
executing its own handle_edge_irq().

With FIFO interrupt the scenario is similar: irq_move_irq() may result
in a EVTCHNOP_unmask hypercall which, in turn, may make the event
pending on the target CPU.

We can avoid this situation by moving and clearing the event while
keeping event masked.

Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: stable@vger.kernel.org
---
 drivers/xen/events/events_base.c |   26 ++++++++++++++++++++++++--
 1 files changed, 24 insertions(+), 2 deletions(-)

diff --git a/drivers/xen/events/events_base.c b/drivers/xen/events/events_base.c
index 524c221..c5725ee 100644
--- a/drivers/xen/events/events_base.c
+++ b/drivers/xen/events/events_base.c
@@ -483,12 +483,23 @@ static void eoi_pirq(struct irq_data *data)
 	int evtchn = evtchn_from_irq(data->irq);
 	struct physdev_eoi eoi = { .irq = pirq_from_irq(data->irq) };
 	int rc = 0;
+	int need_unmask = 0;
 
-	irq_move_irq(data);
+	if (unlikely(irqd_is_setaffinity_pending(data))) {
+		if (VALID_EVTCHN(evtchn))
+			need_unmask = !test_and_set_mask(evtchn);
+	}
 
 	if (VALID_EVTCHN(evtchn))
 		clear_evtchn(evtchn);
 
+	irq_move_irq(data);
+
+	if (VALID_EVTCHN(evtchn)) {
+		if (unlikely(need_unmask))
+			unmask_evtchn(evtchn);
+	}
+
 	if (pirq_needs_eoi(data->irq)) {
 		rc = HYPERVISOR_physdev_op(PHYSDEVOP_eoi, &eoi);
 		WARN_ON(rc);
@@ -1356,11 +1367,22 @@ static void disable_dynirq(struct irq_data *data)
 static void ack_dynirq(struct irq_data *data)
 {
 	int evtchn = evtchn_from_irq(data->irq);
+	int need_unmask = 0;
 
-	irq_move_irq(data);
+	if (unlikely(irqd_is_setaffinity_pending(data))) {
+		if (VALID_EVTCHN(evtchn))
+			need_unmask = !test_and_set_mask(evtchn);
+	}
 
 	if (VALID_EVTCHN(evtchn))
 		clear_evtchn(evtchn);
+
+	irq_move_irq(data);
+
+	if (VALID_EVTCHN(evtchn)) {
+		if (unlikely(need_unmask))
+			unmask_evtchn(evtchn);
+	}
 }
 
 static void mask_ack_dynirq(struct irq_data *data)
-- 
1.7.7.6


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel