Re: [PATCH 0/7] xen/arm: CPU hotplug fixes

[Julien Grall <julien.grall@arm.com>]

On 11/04/18 17:37, Mirela Simonovic wrote:
> Hi Julien,

Hi,

May I ask you to configure your mail client to use > for quoting and use 
plain text? Otherwise, this is going to be really difficult to follow 
the discussion after few round (see already below).

> On Wed, Apr 11, 2018 at 6:02 PM, Julien Grall <julien.grall@arm.com 
> <mailto:julien.grall@arm.com>> wrote:
> 
>     Hi,
> 
>     On 11/04/18 16:58, Mirela Simonovic wrote:
> 
>         On 04/11/2018 05:07 PM, Julien Grall wrote:
> 
>             On 11/04/18 14:19, Mirela Simonovic wrote:
> 
>         Migrating interrupts when turning off a CPU already works.
>         However, when a CPU is turned back on there is no interrupt
>         migration back to the hotplugged CPU - all interrupts will
>         remain routed to the CPU#0.
>         Patch 7/7 fixes this
> 
> 
>     What do you mean by all interrupts? Interrupts routed to guest will
>     always follow the vCPU. So are you sure they are going to be
>     migrated when that vCPU is paused/off?
> 
> 
> Just to make sure we're on the same page - this is about hotplugging 
> physical CPUs. Hotplugging vCPUs using virtual PSCI CPU_OFF interface is 
> already implemented and unrelated to this series.

Yes, we are on the same page :). I was just wondering what happen to 
interrupt routed to that pCPU.

> 
> Assuming that system has 2 pCPUs by 'all interrupts' I mean interrupts 
> that were targeted to the pCPU#0 and pCPU#1 prior to doing any hotplug.
> 
> For example, if a guest is pinned to pCPU#1 an interrupt of a device it 
> owns will be targeted to pCPU#1.
> When pCPU#1 is turned off that interrupt will be migrated to pCPU#0. 
> pCPU#0 finalizes the suspend and receives wake-up interrupts. However, 
> when CPU#1 is turned back on that interrupt will remain targeted to the 
> CPU#0, which I assumed is wrong.
> The scenario described here is also how I tested this.
> 
>     Can you give the path in Xen doing that?
> 
> 
> Sure, here is a backtrace (dumped on the CPU being turned off):
>      0  0x2603dc arch_move_irqs(): vgic.c, line 309
>      1  0x22ee58 sched_move_irqs()+20: schedule.c, line 303
>      2  0x2318e8 cpu_disable_scheduler()+1000: schedule.c, line 586
>      3  0x2318e8 cpu_disable_scheduler()+1000: schedule.c, line 586
>      4  0x25aff8 __cpu_disable()+96: smpboot.c, line 386
>      5  0x201608 take_cpu_down()+52: cpu.c, line 75
>      6  0x23426c stopmachine_action()+188: stop_machine.c, line 159
>      7  0x235858 do_tasklet_work()+176: tasklet.c, line 94
>      8  0x235c80 do_tasklet()+104: tasklet.c, line 126
>      9  0x24daec idle_loop()+144: domain.c, line 72
>     10  0x25b1f8 start_secondary()+404: smpboot.c, line 368


So this cover interrupt routed to a virtual CPU. However, this does not 
handle interrupts used by Xen. How do you handle them?

For instance SMMUs IRQ might be routed to other interrupt than CPU #0.

[...]

> 
>     If the former, it would be nice to get the code you used. If the
>     latter, then having a hack patch to test that code would be nice.
>     Ideally, you want to plug that in the SYSCTL interface for
>     out-of-box testing.
> 
> 
> Ok, I have never used that but I'll try to figure it out. I may come up 
> with additional questions.

You might want to have a look at the x86 version XEN_SYSCTL_cpu_hotplug 
in x86/systcl.c.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel