* smpboot seems to be confused on dual EPYC system
@ 2018-05-20 13:23 Gabriel C
2018-05-27 20:07 ` Borislav Petkov
0 siblings, 1 reply; 7+ messages in thread
From: Gabriel C @ 2018-05-20 13:23 UTC (permalink / raw)
To: LKML; +Cc: X86 ML, Thomas Gleixner
Hello,
I have an H11DSi-NT board with 2 * EPYC 7281 16C/32T CPUs.
On that box for some reason spmboot things '4' logical packages
are possible.
Max CPUs can be 128 ( 2 * 32C/64T ), however only 2 CPUs are possible
not 4 and therefor there is no way to add other 64 hotplug CPUs.
( not even sure cpu hotplug will work here )
$ dmesg | egrep "smpboot|NR_CPUS|CPUs"
[ 0.000000] smpboot: Allowing 128 CPUs, 64 hotplug CPUs
[ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:128
nr_cpu_ids:128 nr_node_ids:8
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=128, Nodes=8
[ 0.000000] RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=128.
[ 0.037000] smpboot: CPU0: AMD EPYC 7281 16-Core Processor (family:
0x17, model: 0x1, stepping: 0x2)
[ 0.048005] smp: Bringing up secondary CPUs ...
[ 0.055042] .... node #0, CPUs: #1 #2 #3
[ 0.079029] .... node #1, CPUs: #4 #5 #6 #7
[ 0.111049] .... node #2, CPUs: #8 #9 #10 #11
[ 0.143050] .... node #3, CPUs: #12 #13 #14 #15
[ 0.175031] .... node #4, CPUs: #16 #17 #18 #19
[ 0.217031] .... node #5, CPUs: #20 #21 #22 #23
[ 0.249062] .... node #6, CPUs: #24 #25 #26 #27
[ 0.281058] .... node #7, CPUs: #28 #29 #30 #31
[ 0.313028] .... node #0, CPUs: #32 #33 #34 #35
[ 0.345028] .... node #1, CPUs: #36 #37 #38 #39
[ 0.377046] .... node #2, CPUs: #40 #41 #42 #43
[ 0.409043] .... node #3, CPUs: #44 #45 #46 #47
[ 0.441028] .... node #4, CPUs: #48 #49 #50 #51
[ 0.473029] .... node #5, CPUs: #52 #53 #54 #55
[ 0.505060] .... node #6, CPUs: #56 #57 #58 #59
[ 0.537052] .... node #7, CPUs: #60 #61 #62 #63
[ 0.562171] smp: Brought up 8 nodes, 64 CPUs
[ 0.562171] smpboot: Max logical packages: 4
[ 0.562171] smpboot: Total of 64 processors activated (268089.53 BogoMIPS)
Can someone have a look ?
Best Regards,
Gabriel C
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: smpboot seems to be confused on dual EPYC system
2018-05-20 13:23 smpboot seems to be confused on dual EPYC system Gabriel C
@ 2018-05-27 20:07 ` Borislav Petkov
[not found] ` <CAEJqkggtUvpfRjUUxQDx8tgV50NHayc=2nRTDNeWq83pZThsUQ@mail.gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: Borislav Petkov @ 2018-05-27 20:07 UTC (permalink / raw)
To: Gabriel C; +Cc: LKML, X86 ML, Thomas Gleixner
On Sun, May 20, 2018 at 03:23:32PM +0200, Gabriel C wrote:
> Hello,
>
> I have an H11DSi-NT board with 2 * EPYC 7281 16C/32T CPUs.
>
> On that box for some reason spmboot things '4' logical packages
> are possible.
First of all, which kernel?
> Max CPUs can be 128 ( 2 * 32C/64T ), however only 2 CPUs are possible
> not 4 and therefor there is no way to add other 64 hotplug CPUs.
> ( not even sure cpu hotplug will work here )
>
> $ dmesg | egrep "smpboot|NR_CPUS|CPUs"
> [ 0.000000] smpboot: Allowing 128 CPUs, 64 hotplug CPUs
Your BIOS is made to support 32-Core CPUs too.
> [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:128
> nr_cpu_ids:128 nr_node_ids:8
> [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=128, Nodes=8
> [ 0.000000] RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=128.
> [ 0.037000] smpboot: CPU0: AMD EPYC 7281 16-Core Processor (family:
> 0x17, model: 0x1, stepping: 0x2)
That says 2 * 16C * 2T = 64T.
> [ 0.048005] smp: Bringing up secondary CPUs ...
> [ 0.055042] .... node #0, CPUs: #1 #2 #3
> [ 0.079029] .... node #1, CPUs: #4 #5 #6 #7
> [ 0.111049] .... node #2, CPUs: #8 #9 #10 #11
> [ 0.143050] .... node #3, CPUs: #12 #13 #14 #15
> [ 0.175031] .... node #4, CPUs: #16 #17 #18 #19
> [ 0.217031] .... node #5, CPUs: #20 #21 #22 #23
> [ 0.249062] .... node #6, CPUs: #24 #25 #26 #27
> [ 0.281058] .... node #7, CPUs: #28 #29 #30 #31
> [ 0.313028] .... node #0, CPUs: #32 #33 #34 #35
> [ 0.345028] .... node #1, CPUs: #36 #37 #38 #39
> [ 0.377046] .... node #2, CPUs: #40 #41 #42 #43
> [ 0.409043] .... node #3, CPUs: #44 #45 #46 #47
> [ 0.441028] .... node #4, CPUs: #48 #49 #50 #51
> [ 0.473029] .... node #5, CPUs: #52 #53 #54 #55
> [ 0.505060] .... node #6, CPUs: #56 #57 #58 #59
> [ 0.537052] .... node #7, CPUs: #60 #61 #62 #63
> [ 0.562171] smp: Brought up 8 nodes, 64 CPUs
> [ 0.562171] smpboot: Max logical packages: 4
> [ 0.562171] smpboot: Total of 64 processors activated (268089.53 BogoMIPS)
You have 64 logical CPUs.
So what is the problem?
--
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: smpboot seems to be confused on dual EPYC system
[not found] ` <CAEJqkggtUvpfRjUUxQDx8tgV50NHayc=2nRTDNeWq83pZThsUQ@mail.gmail.com>
@ 2018-05-27 21:37 ` Gabriel C
2018-05-27 22:51 ` Gabriel C
2018-05-27 23:05 ` Borislav Petkov
2 siblings, 0 replies; 7+ messages in thread
From: Gabriel C @ 2018-05-27 21:37 UTC (permalink / raw)
To: Borislav Petkov; +Cc: LKML, X86 ML, Thomas Gleixner
( sorry I didn't noticed google changed here something , resending to
the list as plain text)
2018-05-27 23:22 GMT+02:00 Gabriel C <nix.or.die@gmail.com>:
>
> 2018-05-27 22:07 GMT+02:00 Borislav Petkov <bp@alien8.de>:
>>
>> On Sun, May 20, 2018 at 03:23:32PM +0200, Gabriel C wrote:
>> > Hello,
>> >
>> > I have an H11DSi-NT board with 2 * EPYC 7281 16C/32T CPUs.
>> >
>> > On that box for some reason spmboot things '4' logical packages
>> > are possible.
>>
>> First of all, which kernel?
>
>
> That box runs right now 4.16.11
>>
>>
>> > Max CPUs can be 128 ( 2 * 32C/64T ), however only 2 CPUs are possible
>> > not 4 and therefor there is no way to add other 64 hotplug CPUs.
>> > ( not even sure cpu hotplug will work here )
>> >
>> > $ dmesg | egrep "smpboot|NR_CPUS|CPUs"
>> > [ 0.000000] smpboot: Allowing 128 CPUs, 64 hotplug CPUs
>>
>> Your BIOS is made to support 32-Core CPUs too.
>
>
> 128 CPUs are maximal possible however there is no way to have
> 2 CPUs fully populated , in this case 64C and have another 64 hotplug CPUs
>
> That would mean that's an s4 system while impossible with EPYCs.
>>
>>
>> > [ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:128
>> > nr_cpu_ids:128 nr_node_ids:8
>> > [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=128,
>> > Nodes=8
>> > [ 0.000000] RCU restricting CPUs from NR_CPUS=8192 to
>> > nr_cpu_ids=128.
>> > [ 0.037000] smpboot: CPU0: AMD EPYC 7281 16-Core Processor (family:
>> > 0x17, model: 0x1, stepping: 0x2)
>>
>> That says 2 * 16C * 2T = 64T.
>
>
> That says whole topology is set to 128 no ?
> Eg to max possible but not to what the system runs now.
>
>>
>>
>> > [ 0.048005] smp: Bringing up secondary CPUs ...
>> > [ 0.055042] .... node #0, CPUs: #1 #2 #3
>> > [ 0.079029] .... node #1, CPUs: #4 #5 #6 #7
>> > [ 0.111049] .... node #2, CPUs: #8 #9 #10 #11
>> > [ 0.143050] .... node #3, CPUs: #12 #13 #14 #15
>> > [ 0.175031] .... node #4, CPUs: #16 #17 #18 #19
>> > [ 0.217031] .... node #5, CPUs: #20 #21 #22 #23
>> > [ 0.249062] .... node #6, CPUs: #24 #25 #26 #27
>> > [ 0.281058] .... node #7, CPUs: #28 #29 #30 #31
>> > [ 0.313028] .... node #0, CPUs: #32 #33 #34 #35
>> > [ 0.345028] .... node #1, CPUs: #36 #37 #38 #39
>> > [ 0.377046] .... node #2, CPUs: #40 #41 #42 #43
>> > [ 0.409043] .... node #3, CPUs: #44 #45 #46 #47
>> > [ 0.441028] .... node #4, CPUs: #48 #49 #50 #51
>> > [ 0.473029] .... node #5, CPUs: #52 #53 #54 #55
>> > [ 0.505060] .... node #6, CPUs: #56 #57 #58 #59
>> > [ 0.537052] .... node #7, CPUs: #60 #61 #62 #63
>> > [ 0.562171] smp: Brought up 8 nodes, 64 CPUs
>> > [ 0.562171] smpboot: Max logical packages: 4
>> > [ 0.562171] smpboot: Total of 64 processors activated (268089.53
>> > BogoMIPS)
>>
>> You have 64 logical CPUs.
>>
>> So what is the problem?
>>
>
> The problem is that being wrong ..
>
> >[ 0.562171] smpboot: Max logical packages: 4
>
> These are max 2 not 4.
>
> Regards
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: smpboot seems to be confused on dual EPYC system
[not found] ` <CAEJqkggtUvpfRjUUxQDx8tgV50NHayc=2nRTDNeWq83pZThsUQ@mail.gmail.com>
2018-05-27 21:37 ` Gabriel C
@ 2018-05-27 22:51 ` Gabriel C
2018-05-27 23:16 ` Borislav Petkov
2018-05-27 23:05 ` Borislav Petkov
2 siblings, 1 reply; 7+ messages in thread
From: Gabriel C @ 2018-05-27 22:51 UTC (permalink / raw)
To: Borislav Petkov; +Cc: LKML, X86 ML, Thomas Gleixner
2018-05-27 23:22 GMT+02:00 Gabriel C <nix.or.die@gmail.com>:
>
> 2018-05-27 22:07 GMT+02:00 Borislav Petkov <bp@alien8.de>:
>>
>> On Sun, May 20, 2018 at 03:23:32PM +0200, Gabriel C wrote:
>> > Hello,
>> >
>> > I have an H11DSi-NT board with 2 * EPYC 7281 16C/32T CPUs.
>> >
>> > On that box for some reason spmboot things '4' logical packages
>> > are possible.
>>
>> First of all, which kernel?
>
>
Here one box with single socket with an AMD EPYC 7551P
on H11SSL-NC mobo running kernel 4.16.10:
...
[ 0.000000] smpboot: Allowing 128 CPUs, 64 hotplug CPUs
[ 0.030000] smpboot: CPU0: AMD EPYC 7551P 32-Core Processor
(family: 0x17, model: 0x1, stepping: 0x2)
[ 0.545084] smpboot: Max logical packages: 2
[ 0.545084] smpboot: Total of 64 processors activated (255989.37 BogoMIPS)
...
128CPUs for sure not possible and for sure no way to have '2' CPUs installed.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: smpboot seems to be confused on dual EPYC system
[not found] ` <CAEJqkggtUvpfRjUUxQDx8tgV50NHayc=2nRTDNeWq83pZThsUQ@mail.gmail.com>
2018-05-27 21:37 ` Gabriel C
2018-05-27 22:51 ` Gabriel C
@ 2018-05-27 23:05 ` Borislav Petkov
2 siblings, 0 replies; 7+ messages in thread
From: Borislav Petkov @ 2018-05-27 23:05 UTC (permalink / raw)
To: Gabriel C; +Cc: LKML, X86 ML, Thomas Gleixner
On Sun, May 27, 2018 at 11:22:18PM +0200, Gabriel C wrote:
> The problem is that being wrong ..
>
> >[ 0.562171] smpboot: Max logical packages: 4
>
> These are max 2 not 4.
So the whole "problem" is that it miscounts max packages. Which means we
waste a couple of KBs of memory, tops.
Any other, real problems you're experiencing?
--
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: smpboot seems to be confused on dual EPYC system
2018-05-27 22:51 ` Gabriel C
@ 2018-05-27 23:16 ` Borislav Petkov
2018-05-27 23:25 ` Gabriel C
0 siblings, 1 reply; 7+ messages in thread
From: Borislav Petkov @ 2018-05-27 23:16 UTC (permalink / raw)
To: Gabriel C; +Cc: LKML, X86 ML, Thomas Gleixner
On Mon, May 28, 2018 at 12:51:17AM +0200, Gabriel C wrote:
> 128CPUs for sure not possible and for sure no way to have '2' CPUs installed.
With that last "'2' CPUs" you mean, two physical processors and thus two
sockets, right?
In any case, this info is what we've gotten from the BIOS so if that
BIOS tells us this way...
Again, you don't see any other issues besides the misrepresenting of max
packages or?
--
Regards/Gruss,
Boris.
Good mailing practices for 400: avoid top-posting and trim the reply.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: smpboot seems to be confused on dual EPYC system
2018-05-27 23:16 ` Borislav Petkov
@ 2018-05-27 23:25 ` Gabriel C
0 siblings, 0 replies; 7+ messages in thread
From: Gabriel C @ 2018-05-27 23:25 UTC (permalink / raw)
To: Borislav Petkov; +Cc: LKML, X86 ML, Thomas Gleixner
2018-05-28 1:16 GMT+02:00 Borislav Petkov <bp@alien8.de>:
> On Mon, May 28, 2018 at 12:51:17AM +0200, Gabriel C wrote:
>> 128CPUs for sure not possible and for sure no way to have '2' CPUs installed.
>
> With that last "'2' CPUs" you mean, two physical processors and thus two
> sockets, right?
Yes.
>
> In any case, this info is what we've gotten from the BIOS so if that
> BIOS tells us this way...
Sure .. not the first time BIOSes lies about things :)
I never said that being an kernel bug I just noticed
something is getting confused.
>
> Again, you don't see any other issues besides the misrepresenting of max
> packages or?
>
Right no other issues.
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-05-27 23:26 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-05-20 13:23 smpboot seems to be confused on dual EPYC system Gabriel C
2018-05-27 20:07 ` Borislav Petkov
[not found] ` <CAEJqkggtUvpfRjUUxQDx8tgV50NHayc=2nRTDNeWq83pZThsUQ@mail.gmail.com>
2018-05-27 21:37 ` Gabriel C
2018-05-27 22:51 ` Gabriel C
2018-05-27 23:16 ` Borislav Petkov
2018-05-27 23:25 ` Gabriel C
2018-05-27 23:05 ` Borislav Petkov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.