linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* Re: 256 apic id for amd64
@ 2005-01-08  2:37 Mikael Pettersson
  2005-01-08 15:46 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: Mikael Pettersson @ 2005-01-08  2:37 UTC (permalink / raw)
  To: YhLu, ak; +Cc: Matt_Domsch, discuss, jamesclv, linux-kernel, suresh.b.siddha

On Fri, 7 Jan 2005 22:12:00 +0100, Andi Kleen wrote:
>On Fri, Jan 07, 2005 at 01:14:24PM -0800, YhLu wrote:
>> After keep the bsp using 0, the jiffies works well. Werid?
>
>Probably a bug somewhere.  But since BSP should be always 
>0 I'm not sure it is worth tracking down.

I hope by "0" you're referring to a Linux kernel defined
software value and _not_ what the HW or BIOS conjured up!

Case in point: I was involved a while ago in tracking down
and fixing a local APIC enumeration bug in the x86-32 (i386)
kernel code, where the kernel failed miserably on some
dual K7 boxes because (a) only one CPU socket was populated,
(b) the BIOS assigned that CPU a non-zero ID, and (c) the
kernel (apic.c) had a bug which triggered when BSP ID != 0.

Never trust a BIOS to DTRT.

/Mikael

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-08  2:37 256 apic id for amd64 Mikael Pettersson
@ 2005-01-08 15:46 ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-08 15:46 UTC (permalink / raw)
  To: Mikael Pettersson
  Cc: YhLu, Matt_Domsch, discuss, jamesclv, linux-kernel, suresh.b.siddha

On Sat, Jan 08, 2005 at 03:37:50AM +0100, Mikael Pettersson wrote:
> On Fri, 7 Jan 2005 22:12:00 +0100, Andi Kleen wrote:
> >On Fri, Jan 07, 2005 at 01:14:24PM -0800, YhLu wrote:
> >> After keep the bsp using 0, the jiffies works well. Werid?
> >
> >Probably a bug somewhere.  But since BSP should be always 
> >0 I'm not sure it is worth tracking down.
> 
> I hope by "0" you're referring to a Linux kernel defined
> software value and _not_ what the HW or BIOS conjured up!

No, I'm refering to the APIC ID configured by the BIOS. 

> Never trust a BIOS to DTRT.

I won't complicate the x86-64 code to work around non existant
theoretical BIOS issues.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-11 19:11 YhLu
  0 siblings, 0 replies; 52+ messages in thread
From: YhLu @ 2005-01-11 19:11 UTC (permalink / raw)
  To: 'Andi Kleen'
  Cc: 'Mikael Pettersson', 'jamesclv@us.ibm.com',
	'Matt_Domsch@dell.com', 'discuss@x86-64.org',
	'linux-kernel@vger.kernel.org',
	'suresh.b.siddha@intel.com'

On 2.4.28 84 bit
~ # cat /proc/interrupts 
           CPU0       CPU1       CPU2       CPU3       
  0:       5302         19          0          0    IO-APIC-edge  timer
  1:          0          0          0          0    IO-APIC-edge  keyboard
  2:          0          0          0          0          XT-PIC  cascade
  4:        175          0          0          0    IO-APIC-edge  serial
 19:          3          0          0          0   IO-APIC-level  ohci1394
 22:          0          0          0          0   IO-APIC-level  usb-ohci
 23:          0          0          0          0   IO-APIC-level  ehci_hcd
NMI:        147        139        110         77 
LOC:       5063       5060       5061       5062 
ERR:        406
MIS:          0
~ # cat /proc/interrupts 
           CPU0       CPU1       CPU2       CPU3       
  0:       5612         19          0          0    IO-APIC-edge  timer
  1:          0          0          0          0    IO-APIC-edge  keyboard
  2:          0          0          0          0          XT-PIC  cascade
  4:        226          0          0          0    IO-APIC-edge  serial
 19:          3          0          0          0   IO-APIC-level  ohci1394
 22:          0          0          0          0   IO-APIC-level  usb-ohci
 23:          0          0          0          0   IO-APIC-level  ehci_hcd
NMI:        147        139        110         77 
LOC:       5373       5370       5371       5372 
ERR:        406
MIS:          0

-----Original Message-----
From: YhLu 
Sent: Tuesday, January 11, 2005 11:05 AM
To: YhLu; Andi Kleen
Cc: 'Mikael Pettersson'; jamesclv@us.ibm.com; Matt_Domsch@dell.com;
discuss@x86-64.org; linux-kernel@vger.kernel.org; suresh.b.siddha@intel.com
Subject: RE: 256 apic id for amd64

On 2.6.10 32 bit, it only crash core1/node0, still can go on but lost one
cpu.

~ # cat /proc/interrupts 
           CPU0       CPU1       CPU2       
  0:      47612          5       7643    IO-APIC-edge  timer
  2:          0          0          0          XT-PIC  cascade
  4:        365          0          1    IO-APIC-edge  serial
  8:          0          0          0    IO-APIC-edge  rtc
 17:          0          0          0   IO-APIC-level  libata
 18:          0          0          0   IO-APIC-level  libata
 19:          2          0          1   IO-APIC-level  ohci1394
 22:          0          0          0   IO-APIC-level  ohci_hcd
 23:          0          0          0   IO-APIC-level  ehci_hcd
NMI:          0          0          0 
LOC:      49259      49254      49253 
ERR:       5729
MIS:          0
~ #

-----Original Message-----
From: Yinghai Lu [mailto:yhlu@tyan.com] 
Sent: Tuesday, January 11, 2005 10:50 AM
To: 'Andi Kleen'
Cc: 'Mikael Pettersson'; 'jamesclv@us.ibm.com'; 'Matt_Domsch@dell.com';
'discuss@x86-64.org'; 'linux-kernel@vger.kernel.org';
'suresh.b.siddha@intel.com'
Subject: RE: 256 apic id for amd64

> It looks like the timer interrupt goes to the second CPU.
Even I don't lift the apic id for bsp, the MB with Nvidia chipset still can
not start core1/node0, from core0/node0 in kernel 2.6. But the 2.4.28 (32
bit, 64 bit) works well. 

How come?

YH

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-11 19:04 YhLu
  0 siblings, 0 replies; 52+ messages in thread
From: YhLu @ 2005-01-11 19:04 UTC (permalink / raw)
  To: YhLu, Andi Kleen
  Cc: 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

On 2.6.10 32 bit, it only crash core1/node0, still can go on but lost one
cpu.

~ # cat /proc/interrupts 
           CPU0       CPU1       CPU2       
  0:      47612          5       7643    IO-APIC-edge  timer
  2:          0          0          0          XT-PIC  cascade
  4:        365          0          1    IO-APIC-edge  serial
  8:          0          0          0    IO-APIC-edge  rtc
 17:          0          0          0   IO-APIC-level  libata
 18:          0          0          0   IO-APIC-level  libata
 19:          2          0          1   IO-APIC-level  ohci1394
 22:          0          0          0   IO-APIC-level  ohci_hcd
 23:          0          0          0   IO-APIC-level  ehci_hcd
NMI:          0          0          0 
LOC:      49259      49254      49253 
ERR:       5729
MIS:          0
~ #

-----Original Message-----
From: Yinghai Lu [mailto:yhlu@tyan.com] 
Sent: Tuesday, January 11, 2005 10:50 AM
To: 'Andi Kleen'
Cc: 'Mikael Pettersson'; 'jamesclv@us.ibm.com'; 'Matt_Domsch@dell.com';
'discuss@x86-64.org'; 'linux-kernel@vger.kernel.org';
'suresh.b.siddha@intel.com'
Subject: RE: 256 apic id for amd64

> It looks like the timer interrupt goes to the second CPU.
Even I don't lift the apic id for bsp, the MB with Nvidia chipset still can
not start core1/node0, from core0/node0 in kernel 2.6. But the 2.4.28 (32
bit, 64 bit) works well. 

How come?

YH

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-11  4:04 ` Siddha, Suresh B
  2005-01-11  4:39   ` Andi Kleen
@ 2005-01-11 17:50   ` Andi Kleen
  1 sibling, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-11 17:50 UTC (permalink / raw)
  To: Yinghai Lu
  Cc: 'Siddha, Suresh B', 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel

On Tue, Jan 11, 2005 at 09:44:07AM -0800, Yinghai Lu wrote:
> Actually It seems the Opteron System with Nvidia chip set can lift the bsp
> (core0/node0) to 0x10, and the jiffies is still changing in for CPU0. I try
> to enable the 8259 PIC in SB, it will cause the jiffies not changing.
> At the same time in the AMD 8111 SB MB, the 8259 PIC is default enabled, So
> after I lift the APIC id, it will stop jiffies. The problem is after I
> disable the 8259 in amd8111, it will cause other booting problem. maybe the
> io-apic is not enabled properly.

There is code that assumes all legacy interrupts (that includes the timer)
are routed to IO-APIC #0.  I still didn't get quite why you want to 
change that anyways. 

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-10 20:46 YhLu
@ 2005-01-11  5:36 ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-11  5:36 UTC (permalink / raw)
  To: YhLu
  Cc: 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

On Mon, Jan 10, 2005 at 12:46:48PM -0800, YhLu wrote:
> With lifting cpu0 apic id to 0x10, on Nvidia chip set, it could pass the

Hmm, does it work with an APIC ID >0 <=8?

> calibrate_delay in bsp. But it can not start core1/node0.

It looks like the timer interrupt goes to the second CPU.

> APIC error on CPU1: 00(08)
> APIC error on CPU1: 08(08)
> ----------- [cut here ] --------- [please bite here ] ---------
> Kernel BUG at timer:416

This is a known race (fallout of the hotplug CPU code, fix is on my todo 
list), but fixing it would probably not help you when the timer goes to the 
wrong CPU.

Actually when it went through calibrate_delay it is likely
broadcast to all CPUs, which is definitely wrong.

-Andi


> invalid operand: 0000 [1] SMP 
> CPU 1 
> Modules linked in:
> Pid: 0, comm: swapper Not tainted 2.6.10-bk13
> RIP: 0010:[<ffffffff8013958d>] <ffffffff8013958d>{cascade+45}
> RSP: 0018:ffff81000221fed8  EFLAGS: 00010007
> RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
> RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff810001e15820
> RBP: ffff810001e16838 R08: 00000000fffffff2 R09: 0000000000000009
> R10: 00000000ffffffff R11: 0000000000000000 R12: ffff810001e15820
> R13: 0000000000000000 R14: ffff81000221ff08 R15: 0000000000000001
> FS:  0000000000000000(0000) GS:ffffffff80579080(0000) knlGS:0000000000000000
> CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
> CR2: 0000000000000000 CR3: 0000000000101000 CR4: 00000000000006a0
> Process swapper (pid: 0, threadinfo ffff810002216000, task ffff810002210dd0)
> Stack: 0000000000000000 0000000000000000 ffffffff8057a510 ffff810001e15820 
>        ffffffff80609900 ffffffff8013970e ffff81000221ff08 ffff81000221ff08 
>        ffffffff8057a764 fffffffffffffffe 
> Call Trace:<IRQ> <ffffffff8013970e>{run_timer_softirq+126}
> <ffffffff801361b1>{__do_softirq+113} 
>        <ffffffff80136265>{do_softirq+53} <ffffffff8010ff3f>{do_IRQ+63} 
>        <ffffffff8010d865>{ret_from_intr+0}  <EOI>
> <ffffffff805880e1>{calibrate_delay+289} 
>        <ffffffff8058d83b>{smp_callin+171}
> <ffffffff8058d92e>{start_secondary+14} 
>        
> 

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-11  3:28   ` Siddha, Suresh B
@ 2005-01-11  4:42     ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-11  4:42 UTC (permalink / raw)
  To: Siddha, Suresh B
  Cc: James Cleverdon, YhLu, 'Mikael Pettersson',
	Matt_Domsch, discuss, linux-kernel

On Mon, Jan 10, 2005 at 07:28:47PM -0800, Siddha, Suresh B wrote:
> Current kernel code which is doing this, 
> can definitely be made more readable. I will do that when ever I get
> a chance.

That would be nice yes.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-11  4:04 ` Siddha, Suresh B
@ 2005-01-11  4:39   ` Andi Kleen
  2005-01-11 17:50   ` Andi Kleen
  1 sibling, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-11  4:39 UTC (permalink / raw)
  To: Siddha, Suresh B
  Cc: Yinghai Lu, 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel

On Mon, Jan 10, 2005 at 08:04:25PM -0800, Siddha, Suresh B wrote:
> Andi, we don't need physical APIC mode just to handle CPU APIC ID's > 7
> when the total number of cpu's in the system is < 8. Right?

Yes, cutoff point is 8, not 7, agreed. 

-Andi


^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-10 18:44 Andi Kleen
@ 2005-01-11  4:04 ` Siddha, Suresh B
  2005-01-11  4:39   ` Andi Kleen
  2005-01-11 17:50   ` Andi Kleen
  0 siblings, 2 replies; 52+ messages in thread
From: Siddha, Suresh B @ 2005-01-11  4:04 UTC (permalink / raw)
  To: Andi Kleen
  Cc: Yinghai Lu, 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

YH, I suggest you root cause the exact issue and if it turns out to
be a platform issue, then we can have a kernel workaround for that specific
platform. As Andi mentioned, we don't have any kernel limitations with
BSP apicid != 0.

On Mon, Jan 10, 2005 at 07:44:37PM +0100, Andi Kleen wrote:
> On Mon, Jan 10, 2005 at 10:32:42AM -0800, Yinghai Lu wrote:
> > Case 1: lift core0/node0 to use 16, and core1/node0 to use 17.....
> > Case 2: core0/node0-->0, core1/node0-->1, core0/node1-->18,
> > core1/node1-->19...
> > Case 3: core0/node0-->0, core1/node0-->17, core0/node1-->18,
> > core1/node1-->19...
> > 
> > Case 1 will make jiffies not happy and it will hang on calibrate_dalay. (
> > jiffies is not changing).
> 
> That's because it needs physical APIC mode to handle CPU IDs >7, and that is
>  not implemented in the flat case. I added it now, but you likely don't need 
> it anyways.

Andi, we don't need physical APIC mode just to handle CPU APIC ID's > 7
when the total number of cpu's in the system is < 8. Right?

thanks,
suresh

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-11  0:42 ` James Cleverdon
@ 2005-01-11  3:28   ` Siddha, Suresh B
  2005-01-11  4:42     ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: Siddha, Suresh B @ 2005-01-11  3:28 UTC (permalink / raw)
  To: James Cleverdon
  Cc: YhLu, Andi Kleen, 'Mikael Pettersson',
	Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

On Mon, Jan 10, 2005 at 04:42:41PM -0800, James Cleverdon wrote:
> Personally, I don't have any problem with replacing the non-power-of-2 
> code with "hweight32(c->x86_num_cores - 1)", but folks at Intel have 
> been very insistent that it may be needed in the future.  Maybe Suresh 
> can speak up about Intel's interests here.

IA32 SDM vol3 section 7.7.5 talks about the recommended way of computing
physical processor package id. Current kernel code which is doing this, 
can definitely be made more readable. I will do that when ever I get
a chance.

thanks,
suresh

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-10 19:41 YhLu
  2005-01-10 19:43 ` Andi Kleen
@ 2005-01-11  0:42 ` James Cleverdon
  2005-01-11  3:28   ` Siddha, Suresh B
  1 sibling, 1 reply; 52+ messages in thread
From: James Cleverdon @ 2005-01-11  0:42 UTC (permalink / raw)
  To: YhLu
  Cc: Andi Kleen, 'Mikael Pettersson',
	Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

Greetings YH,

That patch will cause phys_proc_id[] to contain incorrect information 
for IBM x366 systems.  There is a reason why phys_pkg_id() indirects 
through the subarch table -- it does so because cpuid returns stale 
data when the APIC ID registers are rewritten by the x366 BIOS.  The 
BIOS must rewrite those registers as part of system probing and 
configuration (i.e. Nocona chips can't latch all 8 bits of APIC ID at 
reset time, and we will need all 8 bits real soon).

Personally, I don't have any problem with replacing the non-power-of-2 
code with "hweight32(c->x86_num_cores - 1)", but folks at Intel have 
been very insistent that it may be needed in the future.  Maybe Suresh 
can speak up about Intel's interests here.


On Monday 10 January 2005 11:41 am, YhLu wrote:
> Please refer the patch.
>
> Regards
>
> YH

-- 
James Cleverdon
IBM LTC (xSeries Linux Solutions)
{jamesclv(Unix, preferred), cleverdj(Notes)} at us dot ibm dot comm

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-10 20:48 YhLu
  0 siblings, 0 replies; 52+ messages in thread
From: YhLu @ 2005-01-10 20:48 UTC (permalink / raw)
  To: Andi Kleen
  Cc: 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

Not lifting bsp to 0x10, it fails on TIMER too.

YH

Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Memory: 1019432k/1048576k available (3006k kernel code, 0k reserved, 1317k
data, 548k init)
LYH calibrating 0 jiffies = 4294667566, now=2500693458
LYH calibrating 1 jiffies = 4294667571, now=2511802784
LYH calibrating 3 jiffies = 4294667601, now=2579969051
4341.76 BogoMIPS (lpj=2170880)
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 0(2) -> Node 0
CPU: Physical Processor ID: 0
Using local APIC NMI watchdog using perfctr0
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 0(2) -> Node 0
CPU: Physical Processor ID: 0
CPU0:  stepping 00
per-CPU timeslice cutoff: 1023.91 usecs.
task migration cache decay timeout: 2 msecs.
Booting processor 1/1 rip 6000 rsp ffff810002217f58
Initializing CPU#1
LYH calibrating 0 jiffies = 4294667672, now=2736795282
LYH calibrating 1 jiffies = 4294667672, now=2736800180
Calibrating delay loop... <7>Calibrating delay loop... ----------- [cut here
] --------- [please bite here ] ---------
Kernel BUG at timer:416
invalid operand: 0000 [1] SMP 
CPU 1 
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.10-bk13
RIP: 0010:[<ffffffff8013958d>] <ffffffff8013958d>{cascade+45}
RSP: 0018:ffff81000221fed8  EFLAGS: 00010007
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff810001e15820
RBP: ffff810001e16838 R08: 00000000fffffff2 R09: 0000000000000009
R10: 00000000ffffffff R11: 0000000000000000 R12: fff

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-10 20:46 YhLu
  2005-01-11  5:36 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-10 20:46 UTC (permalink / raw)
  To: Andi Kleen
  Cc: 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

With lifting cpu0 apic id to 0x10, on Nvidia chip set, it could pass the
calibrate_delay in bsp. But it can not start core1/node0.


Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Memory: 1019432k/1048576k available (3006k kernel code, 0k reserved, 1317k
data, 548k init)
LYH calibrating 0 jiffies = 4294667567, now=1622307524
LYH calibrating 1 jiffies = 4294667572, now=1633421978
LYH calibrating 3 jiffies = 4294667601, now=1699336699
4374.52 BogoMIPS (lpj=2187264)
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 0(2) -> Node 0
CPU: Physical Processor ID: 0
Using local APIC NMI watchdog using perfctr0
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 0(2) -> Node 0
CPU: Physical Processor ID: 0
CPU0:  stepping 00
per-CPU timeslice cutoff: 1023.91 usecs.
task migration cache decay timeout: 2 msecs.
Booting processor 1/17 rip 6000 rsp ffff810002217f58
Initializing CPU#1
LYH calibrating 0 jiffies = 4294667674, now=1860098546
LYH calibrating 1 jiffies = 4294667674, now=1860104172
Calibrating delay loop... <7>Calibrating delay loop... <7>spurious 8259A
interrupt: IRQ7.
APIC error on CPU0: 00(08)
APIC error on CPU1: 00(08)
APIC error on CPU1: 08(08)
----------- [cut here ] --------- [please bite here ] ---------
Kernel BUG at timer:416
invalid operand: 0000 [1] SMP 
CPU 1 
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.10-bk13
RIP: 0010:[<ffffffff8013958d>] <ffffffff8013958d>{cascade+45}
RSP: 0018:ffff81000221fed8  EFLAGS: 00010007
RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff810001e15820
RBP: ffff810001e16838 R08: 00000000fffffff2 R09: 0000000000000009
R10: 00000000ffffffff R11: 0000000000000000 R12: ffff810001e15820
R13: 0000000000000000 R14: ffff81000221ff08 R15: 0000000000000001
FS:  0000000000000000(0000) GS:ffffffff80579080(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 0000000000000000 CR3: 0000000000101000 CR4: 00000000000006a0
Process swapper (pid: 0, threadinfo ffff810002216000, task ffff810002210dd0)
Stack: 0000000000000000 0000000000000000 ffffffff8057a510 ffff810001e15820 
       ffffffff80609900 ffffffff8013970e ffff81000221ff08 ffff81000221ff08 
       ffffffff8057a764 fffffffffffffffe 
Call Trace:<IRQ> <ffffffff8013970e>{run_timer_softirq+126}
<ffffffff801361b1>{__do_softirq+113} 
       <ffffffff80136265>{do_softirq+53} <ffffffff8010ff3f>{do_IRQ+63} 
       <ffffffff8010d865>{ret_from_intr+0}  <EOI>
<ffffffff805880e1>{calibrate_delay+289} 
       <ffffffff8058d83b>{smp_callin+171}
<ffffffff8058d92e>{start_secondary+14} 
       


^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-10 20:37 YhLu
  0 siblings, 0 replies; 52+ messages in thread
From: YhLu @ 2005-01-10 20:37 UTC (permalink / raw)
  To: Andi Kleen
  Cc: 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

You are right, it should be separate that to init_amd and init_intel.

I guess for intel dual core initial apic id different to AMD.
It could be 0,7 for node0. 1, 6 for node1......

Amd would be (0, 1) for node 0, (2,3) for node1 ....

YH

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Monday, January 10, 2005 12:19 PM
To: YhLu
Cc: 'Mikael Pettersson'; jamesclv@us.ibm.com; Matt_Domsch@dell.com;
discuss@x86-64.org; linux-kernel@vger.kernel.org; suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

On Mon, Jan 10, 2005 at 12:09:48PM -0800, YhLu wrote:
> Try this one.

I don't think it will work at all on Intel HT systems, since
nobody initializes c->x86_num_cores there. phys_proc_id[] 
is supposed to be the same on two HT siblings.

You'll either need to initialize c->x86_num_cores on Intel too.
But since Intel seems to have dual cores upcomming and you'll
want HyperThreading support too it would need to be a new field.

Alternatively you can split the function for AMD and Intel and
use the new algorithm on AMD only.  Perhaps that's better.
In the later case I would only use it when CMP_LEGACY is set.  

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-10 20:09 YhLu
@ 2005-01-10 20:18 ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-10 20:18 UTC (permalink / raw)
  To: YhLu
  Cc: 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

On Mon, Jan 10, 2005 at 12:09:48PM -0800, YhLu wrote:
> Try this one.

I don't think it will work at all on Intel HT systems, since
nobody initializes c->x86_num_cores there. phys_proc_id[] 
is supposed to be the same on two HT siblings.

You'll either need to initialize c->x86_num_cores on Intel too.
But since Intel seems to have dual cores upcomming and you'll
want HyperThreading support too it would need to be a new field.

Alternatively you can split the function for AMD and Intel and
use the new algorithm on AMD only.  Perhaps that's better.
In the later case I would only use it when CMP_LEGACY is set.  

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-10 20:09 YhLu
  2005-01-10 20:18 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-10 20:09 UTC (permalink / raw)
  To: Andi Kleen
  Cc: 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

[-- Attachment #1: Type: text/plain, Size: 560 bytes --]

Try this one.

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Monday, January 10, 2005 11:44 AM
To: YhLu
Cc: 'Mikael Pettersson'; jamesclv@us.ibm.com; Matt_Domsch@dell.com;
discuss@x86-64.org; linux-kernel@vger.kernel.org; suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

On Mon, Jan 10, 2005 at 11:41:02AM -0800, YhLu wrote:
> Please refer the patch.

That's unreadable. Can you generate against a recent BK snapshot
(2.6.10 + latest from ftp://ftp.kernel.org/pub/linux/kernel/v2.6/snapshots/)

and use diff -u  ? 

-Andi


[-- Attachment #2: x86_64_phy_proc_id.patch --]
[-- Type: application/octet-stream, Size: 1175 bytes --]

--- linux-2.6.10/arch/x86_64/kernel/setup.c	2005-01-10 12:26:21.414190368 -0800
+++ linux-2.6.10.new.x86_64/arch/x86_64/kernel/setup.c	2005-01-10 12:30:53.656803200 -0800
@@ -738,7 +738,6 @@
 {
 #ifdef CONFIG_SMP
 	u32 	eax, ebx, ecx, edx;
-	int 	index_lsb, index_msb, tmp;
 	int 	cpu = smp_processor_id();
 	
 	if (!cpu_has(c, X86_FEATURE_HT))
@@ -750,8 +749,6 @@
 	if (smp_num_siblings == 1) {
 		printk(KERN_INFO  "CPU: Hyper-Threading is disabled\n");
 	} else if (smp_num_siblings > 1) {
-		index_lsb = 0;
-		index_msb = 31;
 		/*
 		 * At this point we only support two siblings per
 		 * processor package.
@@ -761,19 +758,8 @@
 			smp_num_siblings = 1;
 			return;
 		}
-		tmp = smp_num_siblings;
-		while ((tmp & 1) == 0) {
-			tmp >>=1 ;
-			index_lsb++;
-		}
-		tmp = smp_num_siblings;
-		while ((tmp & 0x80000000 ) == 0) {
-			tmp <<=1 ;
-			index_msb--;
-		}
-		if (index_lsb != index_msb )
-			index_msb++;
-		phys_proc_id[cpu] = phys_pkg_id(index_msb);
+
+		phys_proc_id[cpu] = c->x86_apicid >> hweight32(c->x86_num_cores - 1);
 		
 		printk(KERN_INFO  "CPU: Physical Processor ID: %d\n",
 		       phys_proc_id[cpu]);

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-10 19:41 YhLu
@ 2005-01-10 19:43 ` Andi Kleen
  2005-01-11  0:42 ` James Cleverdon
  1 sibling, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-10 19:43 UTC (permalink / raw)
  To: YhLu
  Cc: 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

On Mon, Jan 10, 2005 at 11:41:02AM -0800, YhLu wrote:
> Please refer the patch.

That's unreadable. Can you generate against a recent BK snapshot
(2.6.10 + latest from ftp://ftp.kernel.org/pub/linux/kernel/v2.6/snapshots/) 
and use diff -u  ? 

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-10 19:41 YhLu
  2005-01-10 19:43 ` Andi Kleen
  2005-01-11  0:42 ` James Cleverdon
  0 siblings, 2 replies; 52+ messages in thread
From: YhLu @ 2005-01-10 19:41 UTC (permalink / raw)
  To: Andi Kleen
  Cc: 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

[-- Attachment #1: Type: text/plain, Size: 38 bytes --]

Please refer the patch.

Regards

YH


[-- Attachment #2: x86_64_phy_proc_id.patch --]
[-- Type: application/octet-stream, Size: 1134 bytes --]

664a665
> 	unsigned int n;
687c688,690
< 	if (c->cpuid_level >= 0x80000008) {
---
> 	n = cpuid_eax(0x80000000);
> 
> 	if (n >= 0x80000008) {
698c701
< 			int cpu = c->x86_apicid;
---
> 			int cpu = c->x86_apicid; // that is initial apicid
725,749c728,730
< 		index_lsb = 0;
< 		index_msb = 31;
< 		/*
< 		 * At this point we only support two siblings per
< 		 * processor package.
< 		 */
< 		if (smp_num_siblings > NR_CPUS) {
< 			printk(KERN_WARNING "CPU: Unsupported number of the siblings %d", smp_num_siblings);
< 			smp_num_siblings = 1;
< 			return;
< 		}
< 		tmp = smp_num_siblings;
< 		while ((tmp & 1) == 0) {
< 			tmp >>=1 ;
< 			index_lsb++;
< 		}
< 		tmp = smp_num_siblings;
< 		while ((tmp & 0x80000000 ) == 0) {
< 			tmp <<=1 ;
< 			index_msb--;
< 		}
< 		if (index_lsb != index_msb )
< 			index_msb++;
< 		phys_proc_id[cpu] = phys_pkg_id(index_msb);
< 		
---
> 
> 		phys_proc_id[cpu] =  c->x86_apicid >> hweight32(c->x86_num_cores - 1);
> 
1067c1048
< 		seq_printf(m, "cpu cores\t: %d\n", c->x86_num_cores);
---
> 		seq_printf(m, "\ncpu cores\t: %d\n", c->x86_num_cores);

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-10 18:48 YhLu
  2005-01-10 18:45 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-10 18:48 UTC (permalink / raw)
  To: jamesclv
  Cc: Mikael Pettersson, ak, Matt_Domsch, discuss, linux-kernel,
	suresh.b.siddha

James,

I'm working on add amd dual core LinuxBIOS support to our MB. So I can
change the apic id as I want.

When I lift the apic id for CPUS, if the bsp apicid is changed to 0x10, the
jiffies is not changing, So I have to leave to set BSP using apic id 0 in
LinuxBIOS. And lifting others to use 0x11.....
According to Andi, that would be one bug in kernel .....


Regards

YH

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-10 18:48 YhLu
@ 2005-01-10 18:45 ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-10 18:45 UTC (permalink / raw)
  To: YhLu
  Cc: jamesclv, Mikael Pettersson, Matt_Domsch, discuss, linux-kernel,
	suresh.b.siddha

On Mon, Jan 10, 2005 at 10:48:40AM -0800, YhLu wrote:
> James,
> 
> I'm working on add amd dual core LinuxBIOS support to our MB. So I can
> change the apic id as I want.
> 
> When I lift the apic id for CPUS, if the bsp apicid is changed to 0x10, the
> jiffies is not changing, So I have to leave to set BSP using apic id 0 in
> LinuxBIOS. And lifting others to use 0x11.....

You should not need that.

> According to Andi, that would be one bug in kernel .....

No, after some consideration it's not a bug.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
@ 2005-01-10 18:44 Andi Kleen
  2005-01-11  4:04 ` Siddha, Suresh B
  0 siblings, 1 reply; 52+ messages in thread
From: Andi Kleen @ 2005-01-10 18:44 UTC (permalink / raw)
  To: Yinghai Lu
  Cc: 'Mikael Pettersson',
	jamesclv, Matt_Domsch, discuss, linux-kernel, suresh.b.siddha

On Mon, Jan 10, 2005 at 10:32:42AM -0800, Yinghai Lu wrote:
> Resend
> 
> Andi, 
> 
> I want to repeat the request:
> Please consider to compute the phy_proc_id with x86_apicid (or the initial
> apic id).

Not sure what you want me to do exactly. 

The phys_proc_id[] array? First that isn't really used for anything and then 
it is filled from CPUID 1 anyways. Or phys_pkg_id()? That uses cpuid too.

If you think something is wrong best you send a patch that makes it 
clear what you're talking about.
> 
> The reason is 
> Phy_proc_id will be used in smpboot.c to produce cpu_siblings_map.
> 
> The two systems I mentioned really exist.
> System 1. 8 ways dual core opteron system and with 3 io apic.
> System 2. 4 ways dual core opteron system and with 15 io apic (7 amd 8131
> and 1 amd 8111).
> 
> We have to lift the apicid of cpus to make io apic to use 0-15.

I don't understand why you want to do that. The IO-APIC and the local APIC ID 
space should separate as far as I know, so you could use 0 for both a CPU 
and an IO-APIC. But in theory it should work anyways even with an BSP 
CPU ID != 0.

> Case 1: lift core0/node0 to use 16, and core1/node0 to use 17.....
> Case 2: core0/node0-->0, core1/node0-->1, core0/node1-->18,
> core1/node1-->19...
> Case 3: core0/node0-->0, core1/node0-->17, core0/node1-->18,
> core1/node1-->19...
> 
> Case 1 will make jiffies not happy and it will hang on calibrate_dalay. (
> jiffies is not changing).

That's because it needs physical APIC mode to handle CPU IDs >7, and that is
 not implemented in the flat case. I added it now, but you likely don't need 
it anyways.

-Andi


^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-08  2:53 YhLu
@ 2005-01-09 23:56 ` James Cleverdon
  0 siblings, 0 replies; 52+ messages in thread
From: James Cleverdon @ 2005-01-09 23:56 UTC (permalink / raw)
  To: YhLu
  Cc: Mikael Pettersson, ak, Matt_Domsch, discuss, linux-kernel,
	suresh.b.siddha

On Friday 07 January 2005 06:53 pm, YhLu wrote:
> I mean keep the bsp physical apic id using 0.
>
> YH

If you mean that Linux will require that the boot processor (BSP) will 
always have physical APIC ID == 0, then it is already too late for 
i386.  I've posted a message to the root of this thread on some example 
boxes with weird APIC numbering schemes that are in our lab.

I don't trust every single BIOS writer to _always_ make the BSP physical 
APIC ID 0 for x86-64 either.  Why do you wish to require this anyway?

It is far safer to make no assumptions about APIC numbering and just 
accept whatever the BIOS gives us in the MPS and/or ACPI tables.  A few 
simple arrays indexed by CPU number will reveal the physical and 
logical APIC IDs whenever we need them.

So, why should Linux care about any CPU's physical APIC ID?


> -----Original Message-----
> From: Mikael Pettersson [mailto:mikpe@csd.uu.se]
> Sent: Friday, January 07, 2005 6:38 PM
> To: YhLu; ak@muc.de
> Cc: Matt_Domsch@dell.com; discuss@x86-64.org; jamesclv@us.ibm.com;
> linux-kernel@vger.kernel.org; suresh.b.siddha@intel.com
> Subject: Re: 256 apic id for amd64
>
> On Fri, 7 Jan 2005 22:12:00 +0100, Andi Kleen wrote:
> >On Fri, Jan 07, 2005 at 01:14:24PM -0800, YhLu wrote:
> >> After keep the bsp using 0, the jiffies works well. Werid?
> >
> >Probably a bug somewhere.  But since BSP should be always
> >0 I'm not sure it is worth tracking down.
>
> I hope by "0" you're referring to a Linux kernel defined
> software value and _not_ what the HW or BIOS conjured up!
>
> Case in point: I was involved a while ago in tracking down
> and fixing a local APIC enumeration bug in the x86-32 (i386)
> kernel code, where the kernel failed miserably on some
> dual K7 boxes because (a) only one CPU socket was populated,
> (b) the BIOS assigned that CPU a non-zero ID, and (c) the
> kernel (apic.c) had a bug which triggered when BSP ID != 0.
>
> Never trust a BIOS to DTRT.
>
> /Mikael

-- 
James Cleverdon
IBM LTC (xSeries Linux Solutions)
{jamesclv(Unix, preferred), cleverdj(Notes)} at us dot ibm dot comm

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-08  2:53 YhLu
  2005-01-09 23:56 ` James Cleverdon
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-08  2:53 UTC (permalink / raw)
  To: Mikael Pettersson, ak
  Cc: Matt_Domsch, discuss, jamesclv, linux-kernel, suresh.b.siddha

I mean keep the bsp physical apic id using 0.

YH

-----Original Message-----
From: Mikael Pettersson [mailto:mikpe@csd.uu.se] 
Sent: Friday, January 07, 2005 6:38 PM
To: YhLu; ak@muc.de
Cc: Matt_Domsch@dell.com; discuss@x86-64.org; jamesclv@us.ibm.com;
linux-kernel@vger.kernel.org; suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

On Fri, 7 Jan 2005 22:12:00 +0100, Andi Kleen wrote:
>On Fri, Jan 07, 2005 at 01:14:24PM -0800, YhLu wrote:
>> After keep the bsp using 0, the jiffies works well. Werid?
>
>Probably a bug somewhere.  But since BSP should be always 
>0 I'm not sure it is worth tracking down.

I hope by "0" you're referring to a Linux kernel defined
software value and _not_ what the HW or BIOS conjured up!

Case in point: I was involved a while ago in tracking down
and fixing a local APIC enumeration bug in the x86-32 (i386)
kernel code, where the kernel failed miserably on some
dual K7 boxes because (a) only one CPU socket was populated,
(b) the BIOS assigned that CPU a non-zero ID, and (c) the
kernel (apic.c) had a bug which triggered when BSP ID != 0.

Never trust a BIOS to DTRT.

/Mikael

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-08  1:50 YhLu
  0 siblings, 0 replies; 52+ messages in thread
From: YhLu @ 2005-01-08  1:50 UTC (permalink / raw)
  To: jamesclv, Andi Kleen; +Cc: Matt Domsch, linux-kernel, discuss, suresh.b.siddha

You already get boot_cpu_physical_apic_id and it is a MARCO = boot_cpu_id.
And it always is physical apic id.

Also you already got cpu_to_node array.

Also you already got x86_cpu_to_apicid array. ( physical apic id are stored
according to cpu num).

Here cpu are [0, NR_CPUS-1), and for 8 ways dual core system it should be
16.
And can get in context by smp_processor_id or safe_smp_processor_id ( it
need x86_cpu_to_apicid to be initialized at first)

Cpu_to_node is used for NUMA.

What is the usage for phys_pkg_id?

YH

-----Original Message-----
From: James Cleverdon [mailto:jamesclv@us.ibm.com] 
Sent: Friday, January 07, 2005 5:30 PM
To: Andi Kleen
Cc: YhLu; Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

Andi has already dealt with some of the coding style issues elsewhere in 
the thread.

My comment:  This is playing with fire.  We've gone to considerable 
trouble to make the boot_cpu_id independent of the physical APIC ID 
(which is what hard_smp_processor_id() returns).  Different BIOSes and 
different CPU revisions can cause the boot processor to shift.

Examples:  We have a box where the boot CPU has an APIC ID of 3.  
Another system starts with the BSP == 3, but the BIOS renumbers it to 
zero after first reassigning the original 0 CPU.  So, the APIC IDs end 
up 0, 1, 2, 4.  Yet another system assigns the IDs:  0, 1, 6, 7.

We can expect even stranger numbering schemes in the future, given that 
dual core hyperthreaded CPUs are in the pipeline.  Creating any 
dependency between CPU number and APIC ID is a _bad_ idea.


On Friday 07 January 2005 04:24 am, Andi Kleen wrote:
> On Thu, Jan 06, 2005 at 06:53:11PM -0800, YhLu wrote:
> > static unsigned int phys_pkg_id(int index_msb)
> > {
> >         return hard_smp_processor_id() >> index_msb;
> > }
> >
> > In arch/x86_64/kernel/genapic_cluster.c
> >
> > Should be changed to
> >
> > static unsigned int phys_pkg_id(int index_msb)
> > {
> >         /* physical apicid, so we need to substract offset */
> >         return (hard_smp_processor_id() - boot_cpu_id) >>
> > index_msb; }
>
> Why?
>
> If you want a patch merged you need to supply some more explanation
> please.
>
> Also cc Suresh & James for comment.
>
> -Andi

-- 
James Cleverdon
IBM LTC (xSeries Linux Solutions)
{jamesclv(Unix, preferred), cleverdj(Notes)} at us dot ibm dot comm

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-07 12:24 ` Andi Kleen
@ 2005-01-08  1:30   ` James Cleverdon
  0 siblings, 0 replies; 52+ messages in thread
From: James Cleverdon @ 2005-01-08  1:30 UTC (permalink / raw)
  To: Andi Kleen; +Cc: YhLu, Matt Domsch, linux-kernel, discuss, suresh.b.siddha

Andi has already dealt with some of the coding style issues elsewhere in 
the thread.

My comment:  This is playing with fire.  We've gone to considerable 
trouble to make the boot_cpu_id independent of the physical APIC ID 
(which is what hard_smp_processor_id() returns).  Different BIOSes and 
different CPU revisions can cause the boot processor to shift.

Examples:  We have a box where the boot CPU has an APIC ID of 3.  
Another system starts with the BSP == 3, but the BIOS renumbers it to 
zero after first reassigning the original 0 CPU.  So, the APIC IDs end 
up 0, 1, 2, 4.  Yet another system assigns the IDs:  0, 1, 6, 7.

We can expect even stranger numbering schemes in the future, given that 
dual core hyperthreaded CPUs are in the pipeline.  Creating any 
dependency between CPU number and APIC ID is a _bad_ idea.


On Friday 07 January 2005 04:24 am, Andi Kleen wrote:
> On Thu, Jan 06, 2005 at 06:53:11PM -0800, YhLu wrote:
> > static unsigned int phys_pkg_id(int index_msb)
> > {
> >         return hard_smp_processor_id() >> index_msb;
> > }
> >
> > In arch/x86_64/kernel/genapic_cluster.c
> >
> > Should be changed to
> >
> > static unsigned int phys_pkg_id(int index_msb)
> > {
> >         /* physical apicid, so we need to substract offset */
> >         return (hard_smp_processor_id() - boot_cpu_id) >>
> > index_msb; }
>
> Why?
>
> If you want a patch merged you need to supply some more explanation
> please.
>
> Also cc Suresh & James for comment.
>
> -Andi

-- 
James Cleverdon
IBM LTC (xSeries Linux Solutions)
{jamesclv(Unix, preferred), cleverdj(Notes)} at us dot ibm dot comm

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-08  0:50 YhLu
  2005-01-08  0:42 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-08  0:50 UTC (permalink / raw)
  To: Andi Kleen, James Cleverdon
  Cc: Matt Domsch, linux-kernel, discuss, suresh.b.siddha

But the result looks ugly

I keep core0 and core1 of node0 to use 0/1 got

4407.29 BogoMIPS (lpj=2203648)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 7 -> Node 3
phy_proc_id[0] = 0
phy_proc_id[1] = 0
phy_proc_id[2] = 9
phy_proc_id[3] = 9
phy_proc_id[4] = 10
phy_proc_id[5] = 10
phy_proc_id[6] = 11
phy_proc_id[7] = 11
CPU: Physical Processor ID: 11
 stepping 00
Total of 8 processors activated (35209.21 BogoMIPS).
If only keep core0/node0 to use 0.

Will get
phy_proc_id[0] = 0
phy_proc_id[1] = 8
phy_proc_id[2] = 9
phy_proc_id[3] = 9
phy_proc_id[4] = 10
phy_proc_id[5] = 10
phy_proc_id[6] = 11
phy_proc_id[7] = 11

it separate core0 and core1 of node 1

YH

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Friday, January 07, 2005 4:34 PM
To: James Cleverdon
Cc: YhLu; Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

On Fri, Jan 07, 2005 at 04:26:57PM -0800, James Cleverdon wrote:
> Already done, although not dividing along AMD vs. Intel lines.  
> phys_pkg_id() indirects through the subarch table.  See 
> genapic_cluster.c and genapic_flat.c for details.
> 
> We may need a third subarch for AMD's Extended APIC mode boxes.

I'm not convinced we do. Things seem to work with BSP APIC-ID = 0.
Is there any real reason to not just require that?

> 
> Can you suggest some heuristics for detecting such a system and 
> discerning it from a clustered APIC box?  (Hopefully, without using MPS 
> or ACPI table ID string lookups.)

Early PCI scan would work in the worst case. All Opterons have a builtin
northbridge with a specific ID.  There is already other code that 
checks for these.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-08  0:50 YhLu
@ 2005-01-08  0:42 ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-08  0:42 UTC (permalink / raw)
  To: YhLu; +Cc: James Cleverdon, Matt Domsch, linux-kernel, discuss, suresh.b.siddha

On Fri, Jan 07, 2005 at 04:50:43PM -0800, YhLu wrote:
> But the result looks ugly
> 
> I keep core0 and core1 of node0 to use 0/1 got
> 
> 4407.29 BogoMIPS (lpj=2203648)
> CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
> CPU: L2 Cache: 1024K (64 bytes/line)
> CPU 7 -> Node 3
> phy_proc_id[0] = 0

But that't BSP = 0, isn't it?
Where's the problem with that setup?

> phy_proc_id[1] = 0
> phy_proc_id[2] = 9
> phy_proc_id[3] = 9
> phy_proc_id[4] = 10
> phy_proc_id[5] = 10
> phy_proc_id[6] = 11
> phy_proc_id[7] = 11
> CPU: Physical Processor ID: 11
>  stepping 00
> Total of 8 processors activated (35209.21 BogoMIPS).
> If only keep core0/node0 to use 0.
> 
> Will get
> phy_proc_id[0] = 0
> phy_proc_id[1] = 8
> phy_proc_id[2] = 9
> phy_proc_id[3] = 9
> phy_proc_id[4] = 10
> phy_proc_id[5] = 10
> phy_proc_id[6] = 11
> phy_proc_id[7] = 11
> 
> it separate core0 and core1 of node 1

That's not supported yes. AMD docs specify core 0/1 are always 
consecutive in APIC space.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-08  0:26 ` James Cleverdon
@ 2005-01-08  0:34   ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-08  0:34 UTC (permalink / raw)
  To: James Cleverdon; +Cc: YhLu, Matt Domsch, linux-kernel, discuss, suresh.b.siddha

On Fri, Jan 07, 2005 at 04:26:57PM -0800, James Cleverdon wrote:
> Already done, although not dividing along AMD vs. Intel lines.  
> phys_pkg_id() indirects through the subarch table.  See 
> genapic_cluster.c and genapic_flat.c for details.
> 
> We may need a third subarch for AMD's Extended APIC mode boxes.

I'm not convinced we do. Things seem to work with BSP APIC-ID = 0.
Is there any real reason to not just require that?

> 
> Can you suggest some heuristics for detecting such a system and 
> discerning it from a clustered APIC box?  (Hopefully, without using MPS 
> or ACPI table ID string lookups.)

Early PCI scan would work in the worst case. All Opterons have a builtin
northbridge with a specific ID.  There is already other code that 
checks for these.

-Andi


^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-08  0:28 YhLu
  2005-01-08  0:26 ` James Cleverdon
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-08  0:28 UTC (permalink / raw)
  To: jamesclv; +Cc: Andi Kleen, Matt Domsch, linux-kernel, discuss, suresh.b.siddha

Then sepertate that into init_amd and init_intel.

YH

-----Original Message-----
From: James Cleverdon [mailto:jamesclv@us.ibm.com] 
Sent: Friday, January 07, 2005 4:13 PM
To: YhLu
Cc: Andi Kleen; Matt Domsch; linux-kernel@vger.kernel.org;
discuss@x86-64.org; suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

Clustered APIC systems need the phys_proc_id to come from the APIC ID 
register.  Intel prefers that non-clustered systems get their 
phys_proc_id from the cpuid opcode.

So, using c->x86_apicid is unlikely to satisfy both parties.


On Friday 07 January 2005 04:04 pm, YhLu wrote:
> arch/x86_64/kernel/setup.c
>
> static void __init detect_ht(struct cpuinfo_x86 *c)
>
>                 phys_proc_id[cpu] = phys_pkg_id(index_msb);
>
> --->
> 			  Phy_proc_id[cpu] = cpu_to_node[cpu];
> Or
>    	      // that is initial apicid
>           phys_proc_id[cpu] = c->x86_apicid >>
> hweight32(c->x86_num_cores - 1);
>
> YH
>
> -----Original Message-----
> From: Andi Kleen [mailto:ak@muc.de]
> Sent: Friday, January 07, 2005 2:18 PM
> To: YhLu
> Cc: Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
> jamesclv@us.ibm.com; suresh.b.siddha@intel.com
> Subject: Re: 256 apic id for amd64
>
> On Fri, Jan 07, 2005 at 01:44:19PM -0800, YhLu wrote:
> > Can you consider to use c->x86_apicid to get phy_proc_id, that is
> > initial apicid.?
>
> Where?
>
> -Andi

-- 
James Cleverdon
IBM LTC (xSeries Linux Solutions)
{jamesclv(Unix, preferred), cleverdj(Notes)} at us dot ibm dot comm

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-08  0:28 YhLu
@ 2005-01-08  0:26 ` James Cleverdon
  2005-01-08  0:34   ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: James Cleverdon @ 2005-01-08  0:26 UTC (permalink / raw)
  To: YhLu; +Cc: Andi Kleen, Matt Domsch, linux-kernel, discuss, suresh.b.siddha

Already done, although not dividing along AMD vs. Intel lines.  
phys_pkg_id() indirects through the subarch table.  See 
genapic_cluster.c and genapic_flat.c for details.

We may need a third subarch for AMD's Extended APIC mode boxes.

Can you suggest some heuristics for detecting such a system and 
discerning it from a clustered APIC box?  (Hopefully, without using MPS 
or ACPI table ID string lookups.)


On Friday 07 January 2005 04:28 pm, YhLu wrote:
> Then sepertate that into init_amd and init_intel.
>
> YH
>
> -----Original Message-----
> From: James Cleverdon [mailto:jamesclv@us.ibm.com]
> Sent: Friday, January 07, 2005 4:13 PM
> To: YhLu
> Cc: Andi Kleen; Matt Domsch; linux-kernel@vger.kernel.org;
> discuss@x86-64.org; suresh.b.siddha@intel.com
> Subject: Re: 256 apic id for amd64
>
> Clustered APIC systems need the phys_proc_id to come from the APIC ID
> register.  Intel prefers that non-clustered systems get their
> phys_proc_id from the cpuid opcode.
>
> So, using c->x86_apicid is unlikely to satisfy both parties.
>
> On Friday 07 January 2005 04:04 pm, YhLu wrote:
> > arch/x86_64/kernel/setup.c
> >
> > static void __init detect_ht(struct cpuinfo_x86 *c)
> >
> >                 phys_proc_id[cpu] = phys_pkg_id(index_msb);
> >
> > --->
> > 			  Phy_proc_id[cpu] = cpu_to_node[cpu];
> > Or
> >    	      // that is initial apicid
> >           phys_proc_id[cpu] = c->x86_apicid >>
> > hweight32(c->x86_num_cores - 1);
> >
> > YH
> >
> > -----Original Message-----
> > From: Andi Kleen [mailto:ak@muc.de]
> > Sent: Friday, January 07, 2005 2:18 PM
> > To: YhLu
> > Cc: Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
> > jamesclv@us.ibm.com; suresh.b.siddha@intel.com
> > Subject: Re: 256 apic id for amd64
> >
> > On Fri, Jan 07, 2005 at 01:44:19PM -0800, YhLu wrote:
> > > Can you consider to use c->x86_apicid to get phy_proc_id, that is
> > > initial apicid.?
> >
> > Where?
> >
> > -Andi

-- 
James Cleverdon
IBM LTC (xSeries Linux Solutions)
{jamesclv(Unix, preferred), cleverdj(Notes)} at us dot ibm dot comm

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-08  0:04 YhLu
@ 2005-01-08  0:12 ` James Cleverdon
  0 siblings, 0 replies; 52+ messages in thread
From: James Cleverdon @ 2005-01-08  0:12 UTC (permalink / raw)
  To: YhLu; +Cc: Andi Kleen, Matt Domsch, linux-kernel, discuss, suresh.b.siddha

Clustered APIC systems need the phys_proc_id to come from the APIC ID 
register.  Intel prefers that non-clustered systems get their 
phys_proc_id from the cpuid opcode.

So, using c->x86_apicid is unlikely to satisfy both parties.


On Friday 07 January 2005 04:04 pm, YhLu wrote:
> arch/x86_64/kernel/setup.c
>
> static void __init detect_ht(struct cpuinfo_x86 *c)
>
>                 phys_proc_id[cpu] = phys_pkg_id(index_msb);
>
> --->
> 			  Phy_proc_id[cpu] = cpu_to_node[cpu];
> Or
>    	      // that is initial apicid
>           phys_proc_id[cpu] = c->x86_apicid >>
> hweight32(c->x86_num_cores - 1);
>
> YH
>
> -----Original Message-----
> From: Andi Kleen [mailto:ak@muc.de]
> Sent: Friday, January 07, 2005 2:18 PM
> To: YhLu
> Cc: Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
> jamesclv@us.ibm.com; suresh.b.siddha@intel.com
> Subject: Re: 256 apic id for amd64
>
> On Fri, Jan 07, 2005 at 01:44:19PM -0800, YhLu wrote:
> > Can you consider to use c->x86_apicid to get phy_proc_id, that is
> > initial apicid.?
>
> Where?
>
> -Andi

-- 
James Cleverdon
IBM LTC (xSeries Linux Solutions)
{jamesclv(Unix, preferred), cleverdj(Notes)} at us dot ibm dot comm

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-08  0:04 YhLu
  2005-01-08  0:12 ` James Cleverdon
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-08  0:04 UTC (permalink / raw)
  To: Andi Kleen; +Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

arch/x86_64/kernel/setup.c

static void __init detect_ht(struct cpuinfo_x86 *c)

                phys_proc_id[cpu] = phys_pkg_id(index_msb);

--->
			  Phy_proc_id[cpu] = cpu_to_node[cpu];
Or
   	      // that is initial apicid
          phys_proc_id[cpu] = c->x86_apicid >> hweight32(c->x86_num_cores -
1);

YH

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Friday, January 07, 2005 2:18 PM
To: YhLu
Cc: Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
jamesclv@us.ibm.com; suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

On Fri, Jan 07, 2005 at 01:44:19PM -0800, YhLu wrote:
> Can you consider to use c->x86_apicid to get phy_proc_id, that is initial
> apicid.?

Where? 

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-07 21:44 YhLu
@ 2005-01-07 22:18 ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-07 22:18 UTC (permalink / raw)
  To: YhLu; +Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

On Fri, Jan 07, 2005 at 01:44:19PM -0800, YhLu wrote:
> Can you consider to use c->x86_apicid to get phy_proc_id, that is initial
> apicid.?

Where? 

-Andi


^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-07 21:44 YhLu
  2005-01-07 22:18 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-07 21:44 UTC (permalink / raw)
  To: Andi Kleen; +Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

Can you consider to use c->x86_apicid to get phy_proc_id, that is initial
apicid.?

YH


4407.29 BogoMIPS (lpj=2203648)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 7 -> Node 3
phy_proc_id[0] = 0
phy_proc_id[1] = 0
phy_proc_id[2] = 9
phy_proc_id[3] = 9
phy_proc_id[4] = 10
phy_proc_id[5] = 10
phy_proc_id[6] = 11
phy_proc_id[7] = 11
CPU: Physical Processor ID: 11
 stepping 00
Total of 8 processors activated (35209.21 BogoMIPS).

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Friday, January 07, 2005 1:12 PM
To: YhLu
Cc: Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
jamesclv@us.ibm.com; suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

On Fri, Jan 07, 2005 at 01:14:24PM -0800, YhLu wrote:
> After keep the bsp using 0, the jiffies works well. Werid?

Probably a bug somewhere.  But since BSP should be always 
0 I'm not sure it is worth tracking down.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-07 21:14 YhLu
  2005-01-07 21:12 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-07 21:14 UTC (permalink / raw)
  To: Andi Kleen; +Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

After keep the bsp using 0, the jiffies works well. Werid?

YH

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Friday, January 07, 2005 11:41 AM
To: YhLu
Cc: Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
jamesclv@us.ibm.com; suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

On Fri, Jan 07, 2005 at 11:43:52AM -0800, YhLu wrote:
> Amd 8111 and 8131 only have 4 bit for apcid. So it only can use 0-15.

How broken. Ok. But I still don't like your patch. You should
give the BSP ID 0 and for the others it shouldn't matter anyways
if they use high APICIDs. 

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-07 21:14 YhLu
@ 2005-01-07 21:12 ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-07 21:12 UTC (permalink / raw)
  To: YhLu; +Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

On Fri, Jan 07, 2005 at 01:14:24PM -0800, YhLu wrote:
> After keep the bsp using 0, the jiffies works well. Werid?

Probably a bug somewhere.  But since BSP should be always 
0 I'm not sure it is worth tracking down.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-07 19:43 YhLu
  2005-01-07 19:40 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-07 19:43 UTC (permalink / raw)
  To: Andi Kleen; +Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

Amd 8111 and 8131 only have 4 bit for apcid. So it only can use 0-15.

YH

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Friday, January 07, 2005 11:29 AM
To: YhLu
Cc: Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
jamesclv@us.ibm.com; suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

> Without subtract boot_cpu_id, phys_pkg_id will return 8.
> With that, It will return 0.

Normally this is set up that the CPUs come first and then the IO-APICs.
Why is this not possible with 8111 and 8131?

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-07 19:43 YhLu
@ 2005-01-07 19:40 ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-07 19:40 UTC (permalink / raw)
  To: YhLu; +Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

On Fri, Jan 07, 2005 at 11:43:52AM -0800, YhLu wrote:
> Amd 8111 and 8131 only have 4 bit for apcid. So it only can use 0-15.

How broken. Ok. But I still don't like your patch. You should
give the BSP ID 0 and for the others it shouldn't matter anyways
if they use high APICIDs. 

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-07 18:27 YhLu
@ 2005-01-07 19:29 ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-07 19:29 UTC (permalink / raw)
  To: YhLu; +Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

On Fri, Jan 07, 2005 at 10:27:05AM -0800, YhLu wrote:
> Also in arch/x86_64/kernel/setup.c  init_amd
> 
> 
>         if (c->cpuid_level >= 0x80000008) {
> 
> ---->
> 
>         n = cpuid_eax(0x80000000);
> 
>         if (n >= 0x80000008) {
> 
> for c->cupid_level is get cupid_eax(0) and it always =1

This is already fixed in mainline.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-07 18:19 YhLu
@ 2005-01-07 19:29 ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-07 19:29 UTC (permalink / raw)
  To: YhLu; +Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

> Without subtract boot_cpu_id, phys_pkg_id will return 8.
> With that, It will return 0.

Normally this is set up that the CPUs come first and then the IO-APICs.
Why is this not possible with 8111 and 8131?

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-07 18:27 YhLu
  2005-01-07 19:29 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-07 18:27 UTC (permalink / raw)
  To: YhLu, Andi Kleen
  Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

Also in arch/x86_64/kernel/setup.c  init_amd


        if (c->cpuid_level >= 0x80000008) {

---->

        n = cpuid_eax(0x80000000);

        if (n >= 0x80000008) {

for c->cupid_level is get cupid_eax(0) and it always =1

YH

-----Original Message-----
From: YhLu 
Sent: Friday, January 07, 2005 10:08 AM
To: 'Andi Kleen'
Cc: Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
jamesclv@us.ibm.com; suresh.b.siddha@intel.com
Subject: RE: 256 apic id for amd64

Hard_smp_processor_id is CPU physical apicid.
Boot_cpu_id is boot_cpu_physical_apicid.

There is two configuration that we need to enable APIC_EXT_ID.
1. 8 way + dual core --- 8*2 + 2 +1 = 19, the cpu will use 0-15, and ioapic
need to use 16 above.
2. 4 way + 7 amd 8131 + 1 8111 --- 4+7*2+1=19

After enabling APIC_EXT_ID, the K8 can use 256 apicid. But the io apic
device (amd 8131 and 8111) still need to use 0-15. So We need to use 16
above for cpu apic id.

The BIOS or LinuxBIOS will set the apic id of CPU to 16 later. Or that's to
say
Apicid = initial apic id + apicid_offset.

When dual core is used and nb_cfg_54 is set, node 0 will use initial apicid
0/1 for core0 and core1. after setting apicid_offset. Apicid will be 16/17.

Without subtract boot_cpu_id, phys_pkg_id will return 8.
With that, It will return 0.

The c->x86_apicid is initial apic id and it is by cupid(0x1).

I guess for one core old cpu
nb_cfg_54 can not be set, and node 0 will use initial apidid 0. After
setting apicid_offset. Apicid will be 16.

Without subtract boot_cpu_id, phys_pkg_id will return 16.
With that, It will return 0.

YH

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Friday, January 07, 2005 4:25 AM
To: YhLu
Cc: Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
jamesclv@us.ibm.com; suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

On Thu, Jan 06, 2005 at 06:53:11PM -0800, YhLu wrote:
> static unsigned int phys_pkg_id(int index_msb)
> {
>         return hard_smp_processor_id() >> index_msb;
> }
> 
> In arch/x86_64/kernel/genapic_cluster.c
> 
> Should be changed to 
> 
> static unsigned int phys_pkg_id(int index_msb)
> {
>         /* physical apicid, so we need to substract offset */
>         return (hard_smp_processor_id() - boot_cpu_id) >> index_msb;
> }

Why? 

If you want a patch merged you need to supply some more explanation
please.

Also cc Suresh & James for comment.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-07 18:19 YhLu
  2005-01-07 19:29 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-07 18:19 UTC (permalink / raw)
  To: Andi Kleen; +Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

Hard_smp_processor_id is CPU physical apicid.
Boot_cpu_id is boot_cpu_physical_apicid.

There is two configuration that we need to enable APIC_EXT_ID.
1. 8 way + dual core --- 8*2 + 2 +1 = 19, the cpu will use 0-15, and ioapic
need to use 16 above.
2. 4 way + 7 amd 8131 + 1 8111 --- 4+7*2+1=19

After enabling APIC_EXT_ID, the K8 can use 256 apicid. But the io apic
device (amd 8131 and 8111) still need to use 0-15. So We need to use 16
above for cpu apic id.

The BIOS or LinuxBIOS will set the apic id of CPU to 16 later. Or that's to
say
Apicid = initial apic id + apicid_offset.

When dual core is used and nb_cfg_54 is set, node 0 will use initial apicid
0/1 for core0 and core1. after setting apicid_offset. Apicid will be 16/17.

Without subtract boot_cpu_id, phys_pkg_id will return 8.
With that, It will return 0.

The c->x86_apicid is initial apic id and it is by cupid(0x1).

I guess for one core old cpu
nb_cfg_54 can not be set, and node 0 will use initial apidid 0. After
setting apicid_offset. Apicid will be 16.

Without subtract boot_cpu_id, phys_pkg_id will return 16.
With that, It will return 0.

YH

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Friday, January 07, 2005 4:25 AM
To: YhLu
Cc: Matt Domsch; linux-kernel@vger.kernel.org; discuss@x86-64.org;
jamesclv@us.ibm.com; suresh.b.siddha@intel.com
Subject: Re: 256 apic id for amd64

On Thu, Jan 06, 2005 at 06:53:11PM -0800, YhLu wrote:
> static unsigned int phys_pkg_id(int index_msb)
> {
>         return hard_smp_processor_id() >> index_msb;
> }
> 
> In arch/x86_64/kernel/genapic_cluster.c
> 
> Should be changed to 
> 
> static unsigned int phys_pkg_id(int index_msb)
> {
>         /* physical apicid, so we need to substract offset */
>         return (hard_smp_processor_id() - boot_cpu_id) >> index_msb;
> }

Why? 

If you want a patch merged you need to supply some more explanation
please.

Also cc Suresh & James for comment.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-07  1:06 YhLu
@ 2005-01-07 12:44 ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2005-01-07 12:44 UTC (permalink / raw)
  To: YhLu; +Cc: Matt Domsch, linux-kernel, discuss

On Thu, Jan 06, 2005 at 05:06:35PM -0800, YhLu wrote:
> Andi,
> 
> I made the Opteron using apic id from 16 later in LinuxBIOS.
> 
> So leave 0-15 for ioapic.
> 
> The problem is that the kernel (caliberate_delay) doesn't like that.
> 
> If using lpj=2170880 as the command line for that, it works well.
> 
> What's the jiffies? The TSC is changing but it doesn't.

jiffies is a counter of the timer interrupt. Most likely you broke
the timer somehow.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2005-01-07  2:53 YhLu
@ 2005-01-07 12:24 ` Andi Kleen
  2005-01-08  1:30   ` James Cleverdon
  0 siblings, 1 reply; 52+ messages in thread
From: Andi Kleen @ 2005-01-07 12:24 UTC (permalink / raw)
  To: YhLu; +Cc: Matt Domsch, linux-kernel, discuss, jamesclv, suresh.b.siddha

On Thu, Jan 06, 2005 at 06:53:11PM -0800, YhLu wrote:
> static unsigned int phys_pkg_id(int index_msb)
> {
>         return hard_smp_processor_id() >> index_msb;
> }
> 
> In arch/x86_64/kernel/genapic_cluster.c
> 
> Should be changed to 
> 
> static unsigned int phys_pkg_id(int index_msb)
> {
>         /* physical apicid, so we need to substract offset */
>         return (hard_smp_processor_id() - boot_cpu_id) >> index_msb;
> }

Why? 

If you want a patch merged you need to supply some more explanation
please.

Also cc Suresh & James for comment.

-Andi


^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-07  2:53 YhLu
  2005-01-07 12:24 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-07  2:53 UTC (permalink / raw)
  To: Andi Kleen, Matt Domsch; +Cc: linux-kernel, discuss

static unsigned int phys_pkg_id(int index_msb)
{
        return hard_smp_processor_id() >> index_msb;
}

In arch/x86_64/kernel/genapic_cluster.c

Should be changed to 

static unsigned int phys_pkg_id(int index_msb)
{
        /* physical apicid, so we need to substract offset */
        return (hard_smp_processor_id() - boot_cpu_id) >> index_msb;
}

-----Original Message-----
From: YhLu 
Sent: Thursday, January 06, 2005 5:07 PM
To: Andi Kleen; Matt Domsch
Cc: linux-kernel@vger.kernel.org; discuss@x86-64.org
Subject: RE: 256 apic id for amd64

Andi,

I made the Opteron using apic id from 16 later in LinuxBIOS.

So leave 0-15 for ioapic.

The problem is that the kernel (caliberate_delay) doesn't like that.

If using lpj=2170880 as the command line for that, it works well.

What's the jiffies? The TSC is changing but it doesn't.

YH

Without offset:
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Memory: 1019456k/1048576k available (3011k kernel code, 0k reserved, 1310k
data, 544k init)
LYH calibrating 0 jiffies = 4294667563, now=2922366256
LYH calibrating 1 jiffies = 4294667568, now=2934372713
LYH calibrating 3 jiffies = 4294667600, now=3003581909
4341.76 BogoMIPS (lpj=2170880)
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 0 -> Node 0
CPU: Physical Processor ID: 0

With apic id offset:
Console: colour dummy device 80x25
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Memory: 1019456k/1048576k available (3011k kernel code, 0k reserved, 1310k
data, 544k init)
LYH calibrating 0 jiffies = 4294667296, now=1383947209
LYH calibrating 1 jiffies = 4294667296, now=1395952717


Please refer the print in init/main.c

void __devinit calibrate_delay(void)
{
        unsigned long ticks, loopbit;
        int lps_precision = LPS_PREC;
 +       unsigned long now;
 +
 +       rdtscl(now);
 +
 +        printk("LYH calibrating 0 jiffies = %lu, now=%lu\r\n", jiffies,
now);


        if (preset_lpj) {
                loops_per_jiffy = preset_lpj;
                printk("Calibrating delay loop (skipped)... "
                        "%lu.%02lu BogoMIPS preset\n",
                        loops_per_jiffy/(500000/HZ),
                        (loops_per_jiffy/(5000/HZ)) % 100);
        } else {
                loops_per_jiffy = (1<<12);

 +               rdtscl(now);
 +               printk("LYH calibrating 1 jiffies = %lu, now=%lu\r\n",
jiffies, now);
                printk(KERN_DEBUG "Calibrating delay loop... ");
                while ((loops_per_jiffy <<= 1) != 0) {
                        /* wait for "start of" clock tick */
                        ticks = jiffies;
                        while (ticks == jiffies)
                                /* nothing */;
                        /* Go .. */
                        ticks = jiffies;
                        __delay(loops_per_jiffy);
                        ticks = jiffies - ticks;
                        if (ticks)
                                break;
                }

                /*
                 * Do a binary approximation to get loops_per_jiffy set to
                 * equal one clock (up to lps_precision bits)
                 */
  +              rdtscl(now);
  +              printk("LYH calibrating 2 jiffies = %lu, now=%lu\r\n",
jiffies, now);

                loops_per_jiffy >>= 1;
                loopbit = loops_per_jiffy;
                while (lps_precision-- && (loopbit >>= 1)) {
                        loops_per_jiffy |= loopbit;
                        ticks = jiffies;
                       while (ticks == jiffies)
                                /* nothing */;
                        ticks = jiffies;
                        __delay(loops_per_jiffy);
                        if (jiffies != ticks)   /* longer than 1 tick */
                                loops_per_jiffy &= ~loopbit;
                }

  +              rdtscl(now);
  +              printk("LYH calibrating 3 jiffies = %lu, now=%lu\r\n",
jiffies, now);

                /* Round the value and print it */
                printk("%lu.%02lu BogoMIPS (lpj=%lu)\n",
                        loops_per_jiffy/(500000/HZ),
                        (loops_per_jiffy/(5000/HZ)) % 100,
                        loops_per_jiffy);
        }

}
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2005-01-07  1:06 YhLu
  2005-01-07 12:44 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2005-01-07  1:06 UTC (permalink / raw)
  To: Andi Kleen, Matt Domsch; +Cc: linux-kernel, discuss

Andi,

I made the Opteron using apic id from 16 later in LinuxBIOS.

So leave 0-15 for ioapic.

The problem is that the kernel (caliberate_delay) doesn't like that.

If using lpj=2170880 as the command line for that, it works well.

What's the jiffies? The TSC is changing but it doesn't.

YH

Without offset:
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Memory: 1019456k/1048576k available (3011k kernel code, 0k reserved, 1310k
data, 544k init)
LYH calibrating 0 jiffies = 4294667563, now=2922366256
LYH calibrating 1 jiffies = 4294667568, now=2934372713
LYH calibrating 3 jiffies = 4294667600, now=3003581909
4341.76 BogoMIPS (lpj=2170880)
Mount-cache hash table entries: 256 (order: 0, 4096 bytes)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 0 -> Node 0
CPU: Physical Processor ID: 0

With apic id offset:
Console: colour dummy device 80x25
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Memory: 1019456k/1048576k available (3011k kernel code, 0k reserved, 1310k
data, 544k init)
LYH calibrating 0 jiffies = 4294667296, now=1383947209
LYH calibrating 1 jiffies = 4294667296, now=1395952717


Please refer the print in init/main.c

void __devinit calibrate_delay(void)
{
        unsigned long ticks, loopbit;
        int lps_precision = LPS_PREC;
 +       unsigned long now;
 +
 +       rdtscl(now);
 +
 +        printk("LYH calibrating 0 jiffies = %lu, now=%lu\r\n", jiffies,
now);


        if (preset_lpj) {
                loops_per_jiffy = preset_lpj;
                printk("Calibrating delay loop (skipped)... "
                        "%lu.%02lu BogoMIPS preset\n",
                        loops_per_jiffy/(500000/HZ),
                        (loops_per_jiffy/(5000/HZ)) % 100);
        } else {
                loops_per_jiffy = (1<<12);

 +               rdtscl(now);
 +               printk("LYH calibrating 1 jiffies = %lu, now=%lu\r\n",
jiffies, now);
                printk(KERN_DEBUG "Calibrating delay loop... ");
                while ((loops_per_jiffy <<= 1) != 0) {
                        /* wait for "start of" clock tick */
                        ticks = jiffies;
                        while (ticks == jiffies)
                                /* nothing */;
                        /* Go .. */
                        ticks = jiffies;
                        __delay(loops_per_jiffy);
                        ticks = jiffies - ticks;
                        if (ticks)
                                break;
                }

                /*
                 * Do a binary approximation to get loops_per_jiffy set to
                 * equal one clock (up to lps_precision bits)
                 */
  +              rdtscl(now);
  +              printk("LYH calibrating 2 jiffies = %lu, now=%lu\r\n",
jiffies, now);

                loops_per_jiffy >>= 1;
                loopbit = loops_per_jiffy;
                while (lps_precision-- && (loopbit >>= 1)) {
                        loops_per_jiffy |= loopbit;
                        ticks = jiffies;
                       while (ticks == jiffies)
                                /* nothing */;
                        ticks = jiffies;
                        __delay(loops_per_jiffy);
                        if (jiffies != ticks)   /* longer than 1 tick */
                                loops_per_jiffy &= ~loopbit;
                }

  +              rdtscl(now);
  +              printk("LYH calibrating 3 jiffies = %lu, now=%lu\r\n",
jiffies, now);

                /* Round the value and print it */
                printk("%lu.%02lu BogoMIPS (lpj=%lu)\n",
                        loops_per_jiffy/(500000/HZ),
                        (loops_per_jiffy/(5000/HZ)) % 100,
                        loops_per_jiffy);
        }

}

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2004-12-30 22:56   ` Matt Domsch
@ 2004-12-30 23:26     ` Andi Kleen
  0 siblings, 0 replies; 52+ messages in thread
From: Andi Kleen @ 2004-12-30 23:26 UTC (permalink / raw)
  To: Matt Domsch; +Cc: YhLu, linux-kernel, discuss

On Thu, Dec 30, 2004 at 04:56:02PM -0600, Matt Domsch wrote:
> On Thu, Dec 30, 2004 at 07:45:22PM +0100, Andi Kleen wrote:
> > i386 also has a different (but Intel specific fix) - uses either
> > 0xf or 0xff based on the APIC version. Just dropping it seems
> > better to me though. I suppose Matt (cc'ed) who apparently
> > wrote this code originally used it to work around some BIOS
> > bug, and at least we can hope for now that there are no 
> > EM64T boxes with that particular BIOS bug.
> 
> The MPC spec (I don't have a copy handy anymore) said it's the OS's
> job to program the APIC ID into the processor based on what BIOS put
> in the MP Table for it.  At the time I wrote the patch, the kernel
> didn't do this, so now it does, else all CPUs could wind up with the
> same APIC ID, which messed up interrupt routing IIRC.

Sorry, you lost me. The code my patch removes just checks if the
APIC ID in the mptable looks legal (is within the expected range)
while doing the ID renaming. I didnt remove the full ID renaming
code, just the sanity check.

I suppose this was added because there was some BIOS that put
junk into its mptable and this check caught it, right?
Do you still remember why it was added?

The problem is that the expected range is too small.

-Andi

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2004-12-30 23:19 YhLu
  0 siblings, 0 replies; 52+ messages in thread
From: YhLu @ 2004-12-30 23:19 UTC (permalink / raw)
  To: Andi Kleen; +Cc: linux-kernel, discuss, Matt_Domsch

2.6.9 has a good feature and you remove that in 2.6.10.

That is if the io apic is set to above 0xf, io_apic.c will find one below
0xf unused and update the table.

YH

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Thursday, December 30, 2004 10:45 AM
To: YhLu
Cc: linux-kernel@vger.kernel.org; discuss@x86-64.org; Matt_Domsch@dell.com
Subject: Re: 256 apic id for amd64

YhLu <YhLu@tyan.com> writes:

> Can someone who maintains the x86-64 io_apic.c look at my patch about 256
> apic id for amd64?

First in general if you want patches submitted look up the maintainer
in the MAINTAINERS file in the source tree and send it directly
to the appropiate person and mailing list.

Just curious - how many IO-APICs does your system have?

Then I don't like your patch very much, since it doesnt handle 
Intel systems. The best fix is probably to 

i386 also has a different (but Intel specific fix) - uses either
0xf or 0xff based on the APIC version. Just dropping it seems
better to me though. I suppose Matt (cc'ed) who apparently
wrote this code originally used it to work around some BIOS
bug, and at least we can hope for now that there are no 
EM64T boxes with that particular BIOS bug.

I will add this patch.

-Andi

Remove check that limited max number of IO-APIC to 8.

The original check was apparently to work around some old BIOS
bugs and we just assume x86-64 machines don't have this class of
problems.

Signed-off-by: Andi Kleen <ak@suse.de>

diff -u linux-2.6.10/arch/x86_64/kernel/io_apic.c-o
linux-2.6.10/arch/x86_64/kernel/io_apic.c
--- linux-2.6.10/arch/x86_64/kernel/io_apic.c-o	2004-12-24
22:34:45.000000000 +0100
+++ linux-2.6.10/arch/x86_64/kernel/io_apic.c	2004-12-30
19:41:08.000000000 +0100
@@ -1160,13 +1160,6 @@
 		
 		old_id = mp_ioapics[apic].mpc_apicid;
 
-		if (mp_ioapics[apic].mpc_apicid >= 0xf) {
-			apic_printk(APIC_QUIET,KERN_ERR "BIOS bug,
IO-APIC#%d ID is %d in the MPC table!...\n",
-				apic, mp_ioapics[apic].mpc_apicid);
-			apic_printk(APIC_QUIET,KERN_ERR "... fixing up to
%d. (tell your hw vendor)\n",
-				reg_00.bits.ID);
-			mp_ioapics[apic].mpc_apicid = reg_00.bits.ID;
-		}
 
 		printk(KERN_INFO "Using IO-APIC %d\n",
mp_ioapics[apic].mpc_apicid);
 

^ permalink raw reply	[flat|nested] 52+ messages in thread

* RE: 256 apic id for amd64
@ 2004-12-30 23:16 YhLu
  0 siblings, 0 replies; 52+ messages in thread
From: YhLu @ 2004-12-30 23:16 UTC (permalink / raw)
  To: Andi Kleen; +Cc: linux-kernel, discuss, Matt_Domsch, 'Stefan Reinauer'

For the 8 way dual core Opteron system.
The CPUs will use 16 APIC ID, so BIOS or LinuxBIOS will enable APIC_EXT_ID
to use 256 apic id. So the ioapic will begin from 16.....

Another case : one system has two much 8131 will need to enable it.
For example four way system with single core Opteron, will have 7 8131s and
1 8111. So apic num will be 4+7*2+1=19.

Thanks.

YH

-----Original Message-----
From: Andi Kleen [mailto:ak@muc.de] 
Sent: Thursday, December 30, 2004 10:45 AM
To: YhLu
Cc: linux-kernel@vger.kernel.org; discuss@x86-64.org; Matt_Domsch@dell.com
Subject: Re: 256 apic id for amd64

YhLu <YhLu@tyan.com> writes:

> Can someone who maintains the x86-64 io_apic.c look at my patch about 256
> apic id for amd64?

First in general if you want patches submitted look up the maintainer
in the MAINTAINERS file in the source tree and send it directly
to the appropiate person and mailing list.

Just curious - how many IO-APICs does your system have?

Then I don't like your patch very much, since it doesnt handle 
Intel systems. The best fix is probably to 

i386 also has a different (but Intel specific fix) - uses either
0xf or 0xff based on the APIC version. Just dropping it seems
better to me though. I suppose Matt (cc'ed) who apparently
wrote this code originally used it to work around some BIOS
bug, and at least we can hope for now that there are no 
EM64T boxes with that particular BIOS bug.

I will add this patch.

-Andi

Remove check that limited max number of IO-APIC to 8.

The original check was apparently to work around some old BIOS
bugs and we just assume x86-64 machines don't have this class of
problems.

Signed-off-by: Andi Kleen <ak@suse.de>

diff -u linux-2.6.10/arch/x86_64/kernel/io_apic.c-o
linux-2.6.10/arch/x86_64/kernel/io_apic.c
--- linux-2.6.10/arch/x86_64/kernel/io_apic.c-o	2004-12-24
22:34:45.000000000 +0100
+++ linux-2.6.10/arch/x86_64/kernel/io_apic.c	2004-12-30
19:41:08.000000000 +0100
@@ -1160,13 +1160,6 @@
 		
 		old_id = mp_ioapics[apic].mpc_apicid;
 
-		if (mp_ioapics[apic].mpc_apicid >= 0xf) {
-			apic_printk(APIC_QUIET,KERN_ERR "BIOS bug,
IO-APIC#%d ID is %d in the MPC table!...\n",
-				apic, mp_ioapics[apic].mpc_apicid);
-			apic_printk(APIC_QUIET,KERN_ERR "... fixing up to
%d. (tell your hw vendor)\n",
-				reg_00.bits.ID);
-			mp_ioapics[apic].mpc_apicid = reg_00.bits.ID;
-		}
 
 		printk(KERN_INFO "Using IO-APIC %d\n",
mp_ioapics[apic].mpc_apicid);
 

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2004-12-30 18:45 ` Andi Kleen
@ 2004-12-30 22:56   ` Matt Domsch
  2004-12-30 23:26     ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: Matt Domsch @ 2004-12-30 22:56 UTC (permalink / raw)
  To: Andi Kleen; +Cc: YhLu, linux-kernel, discuss

On Thu, Dec 30, 2004 at 07:45:22PM +0100, Andi Kleen wrote:
> i386 also has a different (but Intel specific fix) - uses either
> 0xf or 0xff based on the APIC version. Just dropping it seems
> better to me though. I suppose Matt (cc'ed) who apparently
> wrote this code originally used it to work around some BIOS
> bug, and at least we can hope for now that there are no 
> EM64T boxes with that particular BIOS bug.

The MPC spec (I don't have a copy handy anymore) said it's the OS's
job to program the APIC ID into the processor based on what BIOS put
in the MP Table for it.  At the time I wrote the patch, the kernel
didn't do this, so now it does, else all CPUs could wind up with the
same APIC ID, which messed up interrupt routing IIRC.

What's differrent about this x86_64 system in this regard please?

Thanks,
Matt

-- 
Matt Domsch
Software Architect
Dell Linux Solutions linux.dell.com & www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com

^ permalink raw reply	[flat|nested] 52+ messages in thread

* Re: 256 apic id for amd64
  2004-12-29  4:43 YhLu
@ 2004-12-30 18:45 ` Andi Kleen
  2004-12-30 22:56   ` Matt Domsch
  0 siblings, 1 reply; 52+ messages in thread
From: Andi Kleen @ 2004-12-30 18:45 UTC (permalink / raw)
  To: YhLu; +Cc: linux-kernel, discuss, Matt_Domsch

YhLu <YhLu@tyan.com> writes:

> Can someone who maintains the x86-64 io_apic.c look at my patch about 256
> apic id for amd64?

First in general if you want patches submitted look up the maintainer
in the MAINTAINERS file in the source tree and send it directly
to the appropiate person and mailing list.

Just curious - how many IO-APICs does your system have?

Then I don't like your patch very much, since it doesnt handle 
Intel systems. The best fix is probably to 

i386 also has a different (but Intel specific fix) - uses either
0xf or 0xff based on the APIC version. Just dropping it seems
better to me though. I suppose Matt (cc'ed) who apparently
wrote this code originally used it to work around some BIOS
bug, and at least we can hope for now that there are no 
EM64T boxes with that particular BIOS bug.

I will add this patch.

-Andi

Remove check that limited max number of IO-APIC to 8.

The original check was apparently to work around some old BIOS
bugs and we just assume x86-64 machines don't have this class of
problems.

Signed-off-by: Andi Kleen <ak@suse.de>

diff -u linux-2.6.10/arch/x86_64/kernel/io_apic.c-o linux-2.6.10/arch/x86_64/kernel/io_apic.c
--- linux-2.6.10/arch/x86_64/kernel/io_apic.c-o	2004-12-24 22:34:45.000000000 +0100
+++ linux-2.6.10/arch/x86_64/kernel/io_apic.c	2004-12-30 19:41:08.000000000 +0100
@@ -1160,13 +1160,6 @@
 		
 		old_id = mp_ioapics[apic].mpc_apicid;
 
-		if (mp_ioapics[apic].mpc_apicid >= 0xf) {
-			apic_printk(APIC_QUIET,KERN_ERR "BIOS bug, IO-APIC#%d ID is %d in the MPC table!...\n",
-				apic, mp_ioapics[apic].mpc_apicid);
-			apic_printk(APIC_QUIET,KERN_ERR "... fixing up to %d. (tell your hw vendor)\n",
-				reg_00.bits.ID);
-			mp_ioapics[apic].mpc_apicid = reg_00.bits.ID;
-		}
 
 		printk(KERN_INFO "Using IO-APIC %d\n", mp_ioapics[apic].mpc_apicid);
 

^ permalink raw reply	[flat|nested] 52+ messages in thread

* 256 apic id for amd64
@ 2004-12-29  4:43 YhLu
  2004-12-30 18:45 ` Andi Kleen
  0 siblings, 1 reply; 52+ messages in thread
From: YhLu @ 2004-12-29  4:43 UTC (permalink / raw)
  To: Linux Kernel Mailing List

[-- Attachment #1: Type: text/plain, Size: 98 bytes --]

Can someone who maintains the x86-64 io_apic.c look at my patch about 256
apic id for amd64?

YH


[-- Attachment #2: x86_64_ioapic.patch --]
[-- Type: application/octet-stream, Size: 1300 bytes --]

diff -uNr linux-2.6.10/arch/x86_64/kernel/io_apic.c linux-2.6.10.new.x86_64/arch/x86_64/kernel/io_apic.c
--- linux-2.6.10/arch/x86_64/kernel/io_apic.c	2004-12-24 13:34:45.000000000 -0800
+++ linux-2.6.10.new.x86_64/arch/x86_64/kernel/io_apic.c	2004-12-28 15:46:35.828076192 -0800
@@ -1148,6 +1148,19 @@
 	unsigned char old_id;
 	unsigned long flags;
 
+        unsigned int max_apic;
+        u32 vendor;
+
+        /* get the max apic */
+        vendor = read_pci_config(0, 0x18, 0, PCI_VENDOR_ID);
+        vendor &= 0xffff;
+        if(vendor == PCI_VENDOR_ID_AMD) { /* AMD */
+                max_apic = (((read_pci_config(0, 0x18, 0, 0x68)>>17) & 3) == 3) ? 0xff : 0xf;
+        }
+        else { /* intel:  how to find out if intel em64t support 256 apic id? */
+                max_apic = 0xf;
+        }
+
 	/*
 	 * Set the IOAPIC ID to the value stored in the MPC table.
 	 */
@@ -1160,7 +1173,7 @@
 		
 		old_id = mp_ioapics[apic].mpc_apicid;
 
-		if (mp_ioapics[apic].mpc_apicid >= 0xf) {
+		if (mp_ioapics[apic].mpc_apicid >= max_apic) {
 			apic_printk(APIC_QUIET,KERN_ERR "BIOS bug, IO-APIC#%d ID is %d in the MPC table!...\n",
 				apic, mp_ioapics[apic].mpc_apicid);
 			apic_printk(APIC_QUIET,KERN_ERR "... fixing up to %d. (tell your hw vendor)\n",

^ permalink raw reply	[flat|nested] 52+ messages in thread

end of thread, other threads:[~2005-01-11 19:00 UTC | newest]

Thread overview: 52+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2005-01-08  2:37 256 apic id for amd64 Mikael Pettersson
2005-01-08 15:46 ` Andi Kleen
  -- strict thread matches above, loose matches on Subject: below --
2005-01-11 19:11 YhLu
2005-01-11 19:04 YhLu
2005-01-10 20:48 YhLu
2005-01-10 20:46 YhLu
2005-01-11  5:36 ` Andi Kleen
2005-01-10 20:37 YhLu
2005-01-10 20:09 YhLu
2005-01-10 20:18 ` Andi Kleen
2005-01-10 19:41 YhLu
2005-01-10 19:43 ` Andi Kleen
2005-01-11  0:42 ` James Cleverdon
2005-01-11  3:28   ` Siddha, Suresh B
2005-01-11  4:42     ` Andi Kleen
2005-01-10 18:48 YhLu
2005-01-10 18:45 ` Andi Kleen
2005-01-10 18:44 Andi Kleen
2005-01-11  4:04 ` Siddha, Suresh B
2005-01-11  4:39   ` Andi Kleen
2005-01-11 17:50   ` Andi Kleen
2005-01-08  2:53 YhLu
2005-01-09 23:56 ` James Cleverdon
2005-01-08  1:50 YhLu
2005-01-08  0:50 YhLu
2005-01-08  0:42 ` Andi Kleen
2005-01-08  0:28 YhLu
2005-01-08  0:26 ` James Cleverdon
2005-01-08  0:34   ` Andi Kleen
2005-01-08  0:04 YhLu
2005-01-08  0:12 ` James Cleverdon
2005-01-07 21:44 YhLu
2005-01-07 22:18 ` Andi Kleen
2005-01-07 21:14 YhLu
2005-01-07 21:12 ` Andi Kleen
2005-01-07 19:43 YhLu
2005-01-07 19:40 ` Andi Kleen
2005-01-07 18:27 YhLu
2005-01-07 19:29 ` Andi Kleen
2005-01-07 18:19 YhLu
2005-01-07 19:29 ` Andi Kleen
2005-01-07  2:53 YhLu
2005-01-07 12:24 ` Andi Kleen
2005-01-08  1:30   ` James Cleverdon
2005-01-07  1:06 YhLu
2005-01-07 12:44 ` Andi Kleen
2004-12-30 23:19 YhLu
2004-12-30 23:16 YhLu
2004-12-29  4:43 YhLu
2004-12-30 18:45 ` Andi Kleen
2004-12-30 22:56   ` Matt Domsch
2004-12-30 23:26     ` Andi Kleen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).