Currently Xen uses default_cpu_present_to_apicid() which will always report BAD_APICID for PV guests since x86_bios_cpu_apic_id is initialised to that value and is never updated. With commit 1f12e32f4cd5 ("x86/topology: Create logical package id"), this op is now called by smp_init_package_map() when deciding whether to call topology_update_package_map() which sets cpu_data(cpu).logical_proc_id. The latter (as topology_logical_package_id(cpu)) may be used, for example, by cpu_to_rapl_pmu() as an array index. Since uninitialized logical_package_id is set to -1, the index will become 64K which is obviously problematic. While RAPL code (and any other users of logical_package_id) should be careful in their assumptions about id's validity, Xen's cpu_present_to_apicid op should still provide value consistent with its own xen_apic_read(APIC_ID). Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> --- arch/x86/xen/apic.c | 12 ++++++++++-- 1 files changed, 10 insertions(+), 2 deletions(-) diff --git a/arch/x86/xen/apic.c b/arch/x86/xen/apic.c index abf4901..db52a7f 100644 --- a/arch/x86/xen/apic.c +++ b/arch/x86/xen/apic.c @@ -66,7 +66,7 @@ static u32 xen_apic_read(u32 reg) ret = HYPERVISOR_platform_op(&op); if (ret) - return 0; + op.u.pcpu_info.apic_id = BAD_APICID; return op.u.pcpu_info.apic_id << 24; } @@ -142,6 +142,14 @@ static void xen_silent_inquire(int apicid) { } +static int xen_cpu_present_to_apicid(int cpu) +{ + if (cpu_present(cpu)) + return xen_get_apic_id(xen_apic_read(APIC_ID)); + else + return BAD_APICID; +} + static struct apic xen_pv_apic = { .name = "Xen PV", .probe = xen_apic_probe_pv, @@ -162,7 +170,7 @@ static struct apic xen_pv_apic = { .ioapic_phys_id_map = default_ioapic_phys_id_map, /* Used on 32-bit */ .setup_apic_routing = NULL, - .cpu_present_to_apicid = default_cpu_present_to_apicid, + .cpu_present_to_apicid = xen_cpu_present_to_apicid, .apicid_to_cpu_present = physid_set_mask_of_physid, /* Used on 32-bit */ .check_phys_apicid_present = default_check_phys_apicid_present, /* smp_sanity_check needs it */ .phys_pkg_id = xen_phys_pkg_id, /* detect_ht */ -- 1.7.1 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
This call has always been missing from xen_play dead() but until recently this was rather benign. With new cpu hotplug framework however this call is required, otherwise a hot-plugged CPU will not be properly brough up (by never calling cpuhp_online_idle()) Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> --- arch/x86/xen/smp.c | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c index 3c6d17f..719cf29 100644 --- a/arch/x86/xen/smp.c +++ b/arch/x86/xen/smp.c @@ -545,6 +545,8 @@ static void xen_play_dead(void) /* used only with HOTPLUG_CPU */ * data back is to call: */ tick_nohz_idle_enter(); + + cpu_startup_entry(CPUHP_AP_ONLINE_IDLE); } #else /* !CONFIG_HOTPLUG_CPU */ -- 1.7.1 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Thu, Mar 17, 2016 at 09:03:24AM -0400, Boris Ostrovsky wrote: > Currently Xen uses default_cpu_present_to_apicid() which will always > report BAD_APICID for PV guests since x86_bios_cpu_apic_id is initialised > to that value and is never updated. > > With commit 1f12e32f4cd5 ("x86/topology: Create logical package id"), this > op is now called by smp_init_package_map() when deciding whether to call > topology_update_package_map() which sets cpu_data(cpu).logical_proc_id. > The latter (as topology_logical_package_id(cpu)) may be used, for example, > by cpu_to_rapl_pmu() as an array index. Since uninitialized > logical_package_id is set to -1, the index will become 64K which is > obviously problematic. > > While RAPL code (and any other users of logical_package_id) should be > careful in their assumptions about id's validity, Xen's > cpu_present_to_apicid op should still provide value consistent with its > own xen_apic_read(APIC_ID). > > Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> Reviewed-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> > --- > arch/x86/xen/apic.c | 12 ++++++++++-- > 1 files changed, 10 insertions(+), 2 deletions(-) > > diff --git a/arch/x86/xen/apic.c b/arch/x86/xen/apic.c > index abf4901..db52a7f 100644 > --- a/arch/x86/xen/apic.c > +++ b/arch/x86/xen/apic.c > @@ -66,7 +66,7 @@ static u32 xen_apic_read(u32 reg) > > ret = HYPERVISOR_platform_op(&op); > if (ret) > - return 0; > + op.u.pcpu_info.apic_id = BAD_APICID; > > return op.u.pcpu_info.apic_id << 24; > } > @@ -142,6 +142,14 @@ static void xen_silent_inquire(int apicid) > { > } > > +static int xen_cpu_present_to_apicid(int cpu) > +{ > + if (cpu_present(cpu)) > + return xen_get_apic_id(xen_apic_read(APIC_ID)); > + else > + return BAD_APICID; > +} > + > static struct apic xen_pv_apic = { > .name = "Xen PV", > .probe = xen_apic_probe_pv, > @@ -162,7 +170,7 @@ static struct apic xen_pv_apic = { > > .ioapic_phys_id_map = default_ioapic_phys_id_map, /* Used on 32-bit */ > .setup_apic_routing = NULL, > - .cpu_present_to_apicid = default_cpu_present_to_apicid, > + .cpu_present_to_apicid = xen_cpu_present_to_apicid, > .apicid_to_cpu_present = physid_set_mask_of_physid, /* Used on 32-bit */ > .check_phys_apicid_present = default_check_phys_apicid_present, /* smp_sanity_check needs it */ > .phys_pkg_id = xen_phys_pkg_id, /* detect_ht */ > -- > 1.7.1 > _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Thu, Mar 17, 2016 at 09:03:25AM -0400, Boris Ostrovsky wrote: > This call has always been missing from xen_play dead() but until > recently this was rather benign. With new cpu hotplug framework > however this call is required, otherwise a hot-plugged CPU will not Could you include the commit id of the 'new cpu hotplug' in case anybody wants to backport this? Thanks! > be properly brough up (by never calling cpuhp_online_idle()) > > Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> > --- > arch/x86/xen/smp.c | 2 ++ > 1 files changed, 2 insertions(+), 0 deletions(-) > > diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c > index 3c6d17f..719cf29 100644 > --- a/arch/x86/xen/smp.c > +++ b/arch/x86/xen/smp.c > @@ -545,6 +545,8 @@ static void xen_play_dead(void) /* used only with HOTPLUG_CPU */ > * data back is to call: > */ > tick_nohz_idle_enter(); > + > + cpu_startup_entry(CPUHP_AP_ONLINE_IDLE); > } > > #else /* !CONFIG_HOTPLUG_CPU */ > -- > 1.7.1 > _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On 03/25/2016 10:53 AM, Konrad Rzeszutek Wilk wrote: > On Thu, Mar 17, 2016 at 09:03:25AM -0400, Boris Ostrovsky wrote: >> This call has always been missing from xen_play dead() but until >> recently this was rather benign. With new cpu hotplug framework >> however this call is required, otherwise a hot-plugged CPU will not > Could you include the commit id of the 'new cpu hotplug' in case > anybody wants to backport this? Sure. It's commit 8df3e07e7f21 ("cpu/hotplug: Let upcoming cpu bring itself fully up"). Do you (or David) want me to re-send it? -boris > > Thanks! >> be properly brough up (by never calling cpuhp_online_idle()) >> >> Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> >> --- >> arch/x86/xen/smp.c | 2 ++ >> 1 files changed, 2 insertions(+), 0 deletions(-) >> >> diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c >> index 3c6d17f..719cf29 100644 >> --- a/arch/x86/xen/smp.c >> +++ b/arch/x86/xen/smp.c >> @@ -545,6 +545,8 @@ static void xen_play_dead(void) /* used only with HOTPLUG_CPU */ >> * data back is to call: >> */ >> tick_nohz_idle_enter(); >> + >> + cpu_startup_entry(CPUHP_AP_ONLINE_IDLE); >> } >> >> #else /* !CONFIG_HOTPLUG_CPU */ >> -- >> 1.7.1 >> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Fri, Mar 25, 2016 at 11:08:32AM -0400, Boris Ostrovsky wrote: > On 03/25/2016 10:53 AM, Konrad Rzeszutek Wilk wrote: > >On Thu, Mar 17, 2016 at 09:03:25AM -0400, Boris Ostrovsky wrote: > >>This call has always been missing from xen_play dead() but until > >>recently this was rather benign. With new cpu hotplug framework > >>however this call is required, otherwise a hot-plugged CPU will not > >Could you include the commit id of the 'new cpu hotplug' in case > >anybody wants to backport this? > > Sure. > > It's commit 8df3e07e7f21 ("cpu/hotplug: Let upcoming cpu bring itself fully > up"). > > Do you (or David) want me to re-send it? That is OK. I've updated the patch and committed both of them in for-linus-4.6. Thanks! > > -boris > > > > > > >Thanks! > >>be properly brough up (by never calling cpuhp_online_idle()) > >> > >>Signed-off-by: Boris Ostrovsky <boris.ostrovsky@oracle.com> > >>--- > >> arch/x86/xen/smp.c | 2 ++ > >> 1 files changed, 2 insertions(+), 0 deletions(-) > >> > >>diff --git a/arch/x86/xen/smp.c b/arch/x86/xen/smp.c > >>index 3c6d17f..719cf29 100644 > >>--- a/arch/x86/xen/smp.c > >>+++ b/arch/x86/xen/smp.c > >>@@ -545,6 +545,8 @@ static void xen_play_dead(void) /* used only with HOTPLUG_CPU */ > >> * data back is to call: > >> */ > >> tick_nohz_idle_enter(); > >>+ > >>+ cpu_startup_entry(CPUHP_AP_ONLINE_IDLE); > >> } > >> #else /* !CONFIG_HOTPLUG_CPU */ > >>-- > >>1.7.1 > >> > _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel