linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] x86, smpboot: allow manual hotplug of CPUs
@ 2012-11-21 18:22 Sasha Levin
  2012-11-21 18:25 ` H. Peter Anvin
  0 siblings, 1 reply; 8+ messages in thread
From: Sasha Levin @ 2012-11-21 18:22 UTC (permalink / raw)
  To: tglx, mingo, hpa; +Cc: x86, linux-kernel, penberg, Sasha Levin

So far CPU hotplug was ignored for mptable implementations which support it by
having the hotpluggable CPUs marked as disabled during boot.

The current kernel code detects that behaviour and actually deals with it
properly:

	[    0.000000] Intel MultiProcessor Specification v1.4
	[    0.000000] MPTABLE: OEM ID: KVMCPU00
	[    0.000000] MPTABLE: Product ID: 0.1
	[    0.000000] MPTABLE: APIC at: 0xFEE00000
	[    0.000000] Processor #0 (Bootup-CPU)
	[    0.000000] Processor #1
	[    0.000000] Processor #2
	[    0.000000] IOAPIC[0]: apic_id 4, version 17, address 0xfec00000, GSI 0-23
	[    0.000000] Processors: 3
	[    0.000000] smpboot: Allowing 3 CPUs, 1 hotplug CPUs

The problem begins when a user might actually want to online such CPU; there
is no interface for him to tell the kernel that the CPU is now present and
can be used.

Luckily, the kernel provides a generic interface in the form of 'probe' and
'release' sysfs files which are used on different architectures exactly for
that - to probe and release CPUs. On x86 however this was unimplemented
until now.

This patch adds code into the x86 implementation of probe and release to allow
adding and removing CPUs. This allows machines that use mptable to hotplug
CPUs:

	sh-4.2# cd /sys/devices/system/cpu/
	sh-4.2# cat possible present online
	0-3
	0-2
	0-2
	sh-4.2# echo "3 0x14" > probe
	sh-4.2# cat possible present online
	0-3
	0-3
	0-2
	sh-4.2# echo 1 > cpu3/online
	[   29.854133] smpboot: Booting Node 0 Processor 3 APIC 0x3
	[    0.001000] kvm-clock: cpu 3, msr 0:1bd929c1, secondary cpu clock
	[   29.872438] KVM setup async PF for cpu 3
	[   29.872790] kvm-stealtime: cpu 3, msr 1bd8d100
	[   29.873276] microcode: CPU3 sig=0x206a7, pf=0x1, revision=0x1
	sh-4.2# cat possible present online
	0-3
	0-3
	0-3
	sh-4.2# echo 0 > cpu3/online
	[  116.146352] Unregister pv shared memory for cpu 3
	[  116.160000] Cannot set affinity for irq 0
	[  116.163068] smpboot: CPU 3 is now offline
	sh-4.2# cat possible present online
	0-3
	0-3
	0-2
	sh-4.2# echo 3 > release
	sh-4.2# cat possible present online
	0-3
	0-2
	0-2

Signed-off-by: Sasha Levin <sasha.levin@oracle.com>
---
 arch/x86/kernel/smpboot.c | 39 +++++++++++++++++++++++++++++++++++++--
 1 file changed, 37 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index 732bf5c..78b3197 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -97,8 +97,43 @@ void cpu_hotplug_driver_unlock(void)
 	mutex_unlock(&x86_cpu_hotplug_driver_mutex);
 }
 
-ssize_t arch_cpu_probe(const char *buf, size_t count) { return -1; }
-ssize_t arch_cpu_release(const char *buf, size_t count) { return -1; }
+ssize_t arch_cpu_probe(const char *buf, size_t count)
+{
+	int cpu, version, r;
+
+	r = sscanf(buf, "%d %x", &cpu, &version);
+	if (r != 2)
+		return -EINVAL;
+
+	if (!cpu_possible(cpu) || cpu_present(cpu))
+		return -EINVAL;
+
+	arch_register_cpu(cpu);
+	generic_processor_info(cpu, version);
+
+	return count;
+}
+
+ssize_t arch_cpu_release(const char *buf, size_t count)
+{
+	int cpu, r;
+
+	r = kstrtoint(buf, 10, &cpu);
+	if (r < 0)
+		return r;
+
+	if (!cpu_present(cpu))
+		return -EINVAL;
+
+	if (cpu_online(cpu))
+		return -EBUSY;
+
+	arch_unregister_cpu(cpu);
+	set_cpu_present(cpu, false);
+
+	return count;
+}
+
 #endif
 
 /* Number of siblings per CPU package */
-- 
1.8.0


^ permalink raw reply related	[flat|nested] 8+ messages in thread

* Re: [PATCH] x86, smpboot: allow manual hotplug of CPUs
  2012-11-21 18:22 [PATCH] x86, smpboot: allow manual hotplug of CPUs Sasha Levin
@ 2012-11-21 18:25 ` H. Peter Anvin
  2012-11-21 18:35   ` Sasha Levin
  0 siblings, 1 reply; 8+ messages in thread
From: H. Peter Anvin @ 2012-11-21 18:25 UTC (permalink / raw)
  To: Sasha Levin; +Cc: tglx, mingo, x86, linux-kernel, penberg

On 11/21/2012 10:22 AM, Sasha Levin wrote:
> So far CPU hotplug was ignored for mptable implementations which support it by
> having the hotpluggable CPUs marked as disabled during boot.
> 
> The current kernel code detects that behaviour and actually deals with it
> properly:
> 
> 	[    0.000000] Intel MultiProcessor Specification v1.4
> 	[    0.000000] MPTABLE: OEM ID: KVMCPU00
> 	[    0.000000] MPTABLE: Product ID: 0.1
> 	[    0.000000] MPTABLE: APIC at: 0xFEE00000
> 	[    0.000000] Processor #0 (Bootup-CPU)
> 	[    0.000000] Processor #1
> 	[    0.000000] Processor #2
> 	[    0.000000] IOAPIC[0]: apic_id 4, version 17, address 0xfec00000, GSI 0-23
> 	[    0.000000] Processors: 3
> 	[    0.000000] smpboot: Allowing 3 CPUs, 1 hotplug CPUs
> 
> The problem begins when a user might actually want to online such CPU; there
> is no interface for him to tell the kernel that the CPU is now present and
> can be used.
> 
> Luckily, the kernel provides a generic interface in the form of 'probe' and
> 'release' sysfs files which are used on different architectures exactly for
> that - to probe and release CPUs. On x86 however this was unimplemented
> until now.
> 
> This patch adds code into the x86 implementation of probe and release to allow
> adding and removing CPUs. This allows machines that use mptable to hotplug
> CPUs:
> 

Reading between the lines, this sounds like would cause a user-visible
difference between mptable platforms and ACPI platforms?  If so, that is
totally unacceptable.  If not, the description is confusing.

	-hpa



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] x86, smpboot: allow manual hotplug of CPUs
  2012-11-21 18:25 ` H. Peter Anvin
@ 2012-11-21 18:35   ` Sasha Levin
  2012-11-21 18:38     ` H. Peter Anvin
  0 siblings, 1 reply; 8+ messages in thread
From: Sasha Levin @ 2012-11-21 18:35 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: tglx, mingo, x86, linux-kernel, penberg

On 11/21/2012 01:25 PM, H. Peter Anvin wrote:
> On 11/21/2012 10:22 AM, Sasha Levin wrote:
>> So far CPU hotplug was ignored for mptable implementations which support it by
>> having the hotpluggable CPUs marked as disabled during boot.
>>
>> The current kernel code detects that behaviour and actually deals with it
>> properly:
>>
>> 	[    0.000000] Intel MultiProcessor Specification v1.4
>> 	[    0.000000] MPTABLE: OEM ID: KVMCPU00
>> 	[    0.000000] MPTABLE: Product ID: 0.1
>> 	[    0.000000] MPTABLE: APIC at: 0xFEE00000
>> 	[    0.000000] Processor #0 (Bootup-CPU)
>> 	[    0.000000] Processor #1
>> 	[    0.000000] Processor #2
>> 	[    0.000000] IOAPIC[0]: apic_id 4, version 17, address 0xfec00000, GSI 0-23
>> 	[    0.000000] Processors: 3
>> 	[    0.000000] smpboot: Allowing 3 CPUs, 1 hotplug CPUs
>>
>> The problem begins when a user might actually want to online such CPU; there
>> is no interface for him to tell the kernel that the CPU is now present and
>> can be used.
>>
>> Luckily, the kernel provides a generic interface in the form of 'probe' and
>> 'release' sysfs files which are used on different architectures exactly for
>> that - to probe and release CPUs. On x86 however this was unimplemented
>> until now.
>>
>> This patch adds code into the x86 implementation of probe and release to allow
>> adding and removing CPUs. This allows machines that use mptable to hotplug
>> CPUs:
>>
> 
> Reading between the lines, this sounds like would cause a user-visible
> difference between mptable platforms and ACPI platforms?  If so, that is
> totally unacceptable.  If not, the description is confusing.

With ACPI platforms you don't need probe/release because the hardware notifies
on CPU insert/eject - this doesn't exist on mptable which is why you have to
do it manually with probe/release.

The difference is already user visible: you can hotplug on ACPI, but can't on
mptables.

Yes, reading back the subject does sound confusing - a better one would probably
be "provide interface for CPU hotplug on mptable platforms" or something similar.


Thanks,
Sasha


^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] x86, smpboot: allow manual hotplug of CPUs
  2012-11-21 18:35   ` Sasha Levin
@ 2012-11-21 18:38     ` H. Peter Anvin
  2012-11-21 19:19       ` Sasha Levin
  0 siblings, 1 reply; 8+ messages in thread
From: H. Peter Anvin @ 2012-11-21 18:38 UTC (permalink / raw)
  To: Sasha Levin; +Cc: tglx, mingo, x86, linux-kernel, penberg

On 11/21/2012 10:35 AM, Sasha Levin wrote:
>>
>> Reading between the lines, this sounds like would cause a user-visible
>> difference between mptable platforms and ACPI platforms?  If so, that is
>> totally unacceptable.  If not, the description is confusing.
> 
> With ACPI platforms you don't need probe/release because the hardware notifies
> on CPU insert/eject - this doesn't exist on mptable which is why you have to
> do it manually with probe/release.
> 
> The difference is already user visible: you can hotplug on ACPI, but can't on
> mptables.
> 
> Yes, reading back the subject does sound confusing - a better one would probably
> be "provide interface for CPU hotplug on mptable platforms" or something similar.
> 

So, are there any mptables platforms which support hotplug?  If the
answer is "KVM" then the answer is that KVM needs to move to ACPI to get
the proper functionality; putting a hack in is really not okay.

	-hpa



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] x86, smpboot: allow manual hotplug of CPUs
  2012-11-21 18:38     ` H. Peter Anvin
@ 2012-11-21 19:19       ` Sasha Levin
  2012-11-21 19:24         ` H. Peter Anvin
  0 siblings, 1 reply; 8+ messages in thread
From: Sasha Levin @ 2012-11-21 19:19 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: tglx, mingo, x86, linux-kernel, penberg

On 11/21/2012 01:38 PM, H. Peter Anvin wrote:
> On 11/21/2012 10:35 AM, Sasha Levin wrote:
>>>
>>> Reading between the lines, this sounds like would cause a user-visible
>>> difference between mptable platforms and ACPI platforms?  If so, that is
>>> totally unacceptable.  If not, the description is confusing.
>>
>> With ACPI platforms you don't need probe/release because the hardware notifies
>> on CPU insert/eject - this doesn't exist on mptable which is why you have to
>> do it manually with probe/release.
>>
>> The difference is already user visible: you can hotplug on ACPI, but can't on
>> mptables.
>>
>> Yes, reading back the subject does sound confusing - a better one would probably
>> be "provide interface for CPU hotplug on mptable platforms" or something similar.
>>
> 
> So, are there any mptables platforms which support hotplug?  If the
> answer is "KVM" then the answer is that KVM needs to move to ACPI to get
> the proper functionality; putting a hack in is really not okay.

There are no platforms which support actual hotplug, but you can still set
existing processors as disabled in the table and without this patch there's
no way enable them.

I'm not sure if it's a "hack" though - the presentation of hotpluggable cpus
is the almost the same between mptable and acpi, and acpi provides a way to
manually probe/release cpus as well. The only difference is that acpi also
provides notifications about such events.

Actually, maybe acpi should start using probe/release as well... hmm...


Thanks,
Sasha

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] x86, smpboot: allow manual hotplug of CPUs
  2012-11-21 19:19       ` Sasha Levin
@ 2012-11-21 19:24         ` H. Peter Anvin
  2012-11-21 19:35           ` Sasha Levin
  0 siblings, 1 reply; 8+ messages in thread
From: H. Peter Anvin @ 2012-11-21 19:24 UTC (permalink / raw)
  To: Sasha Levin; +Cc: tglx, mingo, x86, linux-kernel, penberg

On 11/21/2012 11:19 AM, Sasha Levin wrote:
>>
>> So, are there any mptables platforms which support hotplug?  If the
>> answer is "KVM" then the answer is that KVM needs to move to ACPI to get
>> the proper functionality; putting a hack in is really not okay.
> 
> There are no platforms which support actual hotplug, but you can still set
> existing processors as disabled in the table and without this patch there's
> no way enable them.
> 
> I'm not sure if it's a "hack" though - the presentation of hotpluggable cpus
> is the almost the same between mptable and acpi, and acpi provides a way to
> manually probe/release cpus as well. The only difference is that acpi also
> provides notifications about such events.
> 
> Actually, maybe acpi should start using probe/release as well... hmm...
> 

The bottom line is that I don't want the underlying implementation to
end up with a user-visible difference... therein lies madness and lots
of bugs.

	-hpa



^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] x86, smpboot: allow manual hotplug of CPUs
  2012-11-21 19:24         ` H. Peter Anvin
@ 2012-11-21 19:35           ` Sasha Levin
  2012-11-21 19:39             ` H. Peter Anvin
  0 siblings, 1 reply; 8+ messages in thread
From: Sasha Levin @ 2012-11-21 19:35 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: tglx, mingo, x86, linux-kernel, penberg

On 11/21/2012 02:24 PM, H. Peter Anvin wrote:
> On 11/21/2012 11:19 AM, Sasha Levin wrote:
>>>
>>> So, are there any mptables platforms which support hotplug?  If the
>>> answer is "KVM" then the answer is that KVM needs to move to ACPI to get
>>> the proper functionality; putting a hack in is really not okay.
>>
>> There are no platforms which support actual hotplug, but you can still set
>> existing processors as disabled in the table and without this patch there's
>> no way enable them.
>>
>> I'm not sure if it's a "hack" though - the presentation of hotpluggable cpus
>> is the almost the same between mptable and acpi, and acpi provides a way to
>> manually probe/release cpus as well. The only difference is that acpi also
>> provides notifications about such events.
>>
>> Actually, maybe acpi should start using probe/release as well... hmm...
>>
> 
> The bottom line is that I don't want the underlying implementation to
> end up with a user-visible difference... therein lies madness and lots
> of bugs.

Okay, so if in the case of ACPI, 'probe' will call acpi_processor_add()
and 'release' would call acpi_processor_remove() so the behaviour
would be the same for both ACPI and mptables. Is this okay?


Thanks,
Sasha

^ permalink raw reply	[flat|nested] 8+ messages in thread

* Re: [PATCH] x86, smpboot: allow manual hotplug of CPUs
  2012-11-21 19:35           ` Sasha Levin
@ 2012-11-21 19:39             ` H. Peter Anvin
  0 siblings, 0 replies; 8+ messages in thread
From: H. Peter Anvin @ 2012-11-21 19:39 UTC (permalink / raw)
  To: Sasha Levin; +Cc: tglx, mingo, x86, linux-kernel, penberg, Brown, Len

On 11/21/2012 11:35 AM, Sasha Levin wrote:
> On 11/21/2012 02:24 PM, H. Peter Anvin wrote:
>> On 11/21/2012 11:19 AM, Sasha Levin wrote:
>>>>
>>>> So, are there any mptables platforms which support hotplug?  If the
>>>> answer is "KVM" then the answer is that KVM needs to move to ACPI to get
>>>> the proper functionality; putting a hack in is really not okay.
>>>
>>> There are no platforms which support actual hotplug, but you can still set
>>> existing processors as disabled in the table and without this patch there's
>>> no way enable them.
>>>
>>> I'm not sure if it's a "hack" though - the presentation of hotpluggable cpus
>>> is the almost the same between mptable and acpi, and acpi provides a way to
>>> manually probe/release cpus as well. The only difference is that acpi also
>>> provides notifications about such events.
>>>
>>> Actually, maybe acpi should start using probe/release as well... hmm...
>>>
>>
>> The bottom line is that I don't want the underlying implementation to
>> end up with a user-visible difference... therein lies madness and lots
>> of bugs.
> 
> Okay, so if in the case of ACPI, 'probe' will call acpi_processor_add()
> and 'release' would call acpi_processor_remove() so the behaviour
> would be the same for both ACPI and mptables. Is this okay?
> 

Sounds reasonable to me.  Adding Len to the Cc: list.

	-hpa



^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2012-11-21 19:39 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-11-21 18:22 [PATCH] x86, smpboot: allow manual hotplug of CPUs Sasha Levin
2012-11-21 18:25 ` H. Peter Anvin
2012-11-21 18:35   ` Sasha Levin
2012-11-21 18:38     ` H. Peter Anvin
2012-11-21 19:19       ` Sasha Levin
2012-11-21 19:24         ` H. Peter Anvin
2012-11-21 19:35           ` Sasha Levin
2012-11-21 19:39             ` H. Peter Anvin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).