Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic. - Dmitry Eremin-Solenikov

From: Dmitry Eremin-Solenikov <dmitry.baryshkov@siemens.com>
To: Jan Kiszka <jan.kiszka@siemens.com>
Cc: Gleb Natapov <gleb@redhat.com>, kvm@vger.kernel.org
Subject: Re: [PATCH 05/15] Coalesce userspace/kernel irqchip interrupt injection logic.
Date: Wed, 15 Apr 2009 15:53:40 +0400	[thread overview]
Message-ID: <49E5CAC4.3030002@siemens.com> (raw)
In-Reply-To: <49E5C46F.3010805@siemens.com>

[-- Attachment #1: Type: text/plain, Size: 1389 bytes --]

Jan Kiszka пишет:
> Dmitry Eremin-Solenikov wrote:
>> Gleb Natapov wrote:
>>> On Wed, Apr 15, 2009 at 01:30:29PM +0400, Dmitry Eremin-Solenikov wrote:
>>>> qemu-x86_64 version 0.10.2 running on i386
>>>> Due to problems with qemu-x86_64 I have to boot the 'host' kernel
>>>> with 'noapic'.
>>> Do you mean boot 'guest' kernel with noapic? The guest is what runs
>>> inside qemu. So you are able to boot guest with 'noapic'?
>>>
>>> What is the command line you are using.
>> Well, since this caused lot's of questions, here is my setup:
>>
>> Main host: Debian squeeze, kernel 2.6.28 or .29 (doesn't matter),
>> qemu-system-x86_64 version 0.10.2
>>
>> KVM kernel run inside qemu: e3dbe3f408a46a045012f1882e9f62b27b8a616c
>> from Avi's tree (KVM: x86 emulator: fix call near emulation) + these
>> patches. I have to boot the kernels (both this kernel and 2.6.26 from
>> debian) with noapic to w/around APIC problems (I dunno if it's qemu or
>> bochsbios problem).
> 
> And the bios you are using with 0.10.2 is from 0.10.2 (when in doubt,
> specify explicitly with -bios and/or -L)? Then this would be a QEMU
> upstream bug.

Indeed, there seem to be problems with upstream qemu bios. I was using
the image from the debian's bochsbios package. I asked qemu to use the 
bios from 0.10.2 release and got slightly different messages. Attached
the kernel log

-- 
With best wishes
Dmitry

[-- Attachment #2: log --]
[-- Type: text/plain, Size: 8979 bytes --]

Linux version 2.6.29-06626-gb9d7dba (lumag@doriath) (gcc version 4.3.3 (Debian 4.3.3-3) ) #8 SMP Wed Apr 15 15:46:28 MSD 2009
Command line: root=/dev/sda1 ro console=ttyS0 apic=debug debug
KERNEL supported cpus:
  Intel GenuineIntel
  AMD AuthenticAMD
  Centaur CentaurHauls
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
 BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e8000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000000fff0000 (usable)
 BIOS-e820: 000000000fff0000 - 0000000010000000 (ACPI data)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
DMI 2.4 present.
last_pfn = 0xfff0 max_arch_pfn = 0x100000000
x86 PAT enabled: cpu 0, old 0x0, new 0x7010600070106
init_memory_mapping: 0000000000000000-000000000fff0000
 0000000000 - 000fe00000 page 2M
 000fe00000 - 000fff0000 page 4k
kernel direct mapping tables up to fff0000 @ 8000-b000
last_map_addr: fff0000 end: fff0000
ACPI: RSDP 000FBB80, 0014 (r0 QEMU  )
ACPI: RSDT 0FFF0000, 0034 (r1 QEMU   QEMURSDT        1 QEMU        1)
ACPI: FACP 0FFF0034, 0074 (r1 QEMU   QEMUFACP        1 QEMU        1)
FADT: X_PM1a_EVT_BLK.bit_width (16) does not match PM1_EVT_LEN (4)
ACPI: DSDT 0FFF0100, 080D (r1   BXPC   BXDSDT        1 INTL 20061109)
ACPI: FACS 0FFF00C0, 0040
ACPI: APIC 0FFF0948, 004A (r1 QEMU   QEMUAPIC        1 QEMU        1)
ACPI: SSDT 0FFF090D, 0037 (r1 QEMU   QEMUSSDT        1 QEMU        1)
ACPI: HPET 0FFF0998, 0038 (r1 QEMU   QEMUHPET        1 QEMU        1)
ACPI: Local APIC address 0xfee00000
(5 early reservations) ==> bootmem [0000000000 - 000fff0000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0000006000 - 0000008000]       TRAMPOLINE ==> [0000006000 - 0000008000]
  #2 [0000200000 - 0000876c54]    TEXT DATA BSS ==> [0000200000 - 0000876c54]
  #3 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 0000100000]
  #4 [0000008000 - 0000009000]          PGTABLE ==> [0000008000 - 0000009000]
Scan SMP from ffff880000000000 for 1024 bytes.
Scan SMP from ffff88000009fc00 for 1024 bytes.
Scan SMP from ffff8800000f0000 for 65536 bytes.
found SMP MP-table at [ffff8800000fba60] fba60
 [ffffe20000000000-ffffe200003fffff] PMD -> [ffff880001200000-ffff8800015fffff] on node 0
Zone PFN ranges:
  DMA      0x00000000 -> 0x00001000
  DMA32    0x00001000 -> 0x00100000
  Normal   0x00100000 -> 0x00100000
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
    0: 0x00000000 -> 0x0000009f
    0: 0x00000100 -> 0x0000fff0
On node 0 totalpages: 65423
  DMA zone: 56 pages used for memmap
  DMA zone: 1756 pages reserved
  DMA zone: 2187 pages, LIFO batch:0
  DMA32 zone: 840 pages used for memmap
  DMA32 zone: 60584 pages, LIFO batch:15
ACPI: PM-Timer IO Port: 0xb008
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 1, version 0, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Using ACPI (MADT) for SMP configuration information
ACPI: HPET id: 0x8086a201 base: 0xfed00000
SMP: Allowing 1 CPUs, 0 hotplug CPUs
mapped APIC to ffffffffff5fc000 (fee00000)
mapped IOAPIC to ffffffffff5fb000 (fec00000)
nr_irqs_gsi: 24
Allocating PCI resources starting at 20000000 (gap: 10000000:effc0000)
NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:1 nr_node_ids:1
PERCPU: Embedded 25 pages at ffff880001033000, static data 70880 bytes
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 62771
Kernel command line: root=/dev/sda1 ro console=ttyS0 apic=debug debug
Initializing CPU#0
NR_IRQS:512
PID hash table entries: 1024 (order: 10, 8192 bytes)
Fast TSC calibration using PIT
Detected 1828.564 MHz processor.
Console: colour VGA+ 80x25
console [ttyS0] enabled
Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
Inode-cache hash table entries: 16384 (order: 5, 131072 bytes)
Checking aperture...
No AGP bridge found
Memory: 249848k/262080k available (4048k kernel code, 388k absent, 11528k reserved, 1626k data, 436k init)
SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
hpet clockevent registered
Calibrating delay loop (skipped), value calculated using timer frequency.. 3657.12 BogoMIPS (lpj=7314256)
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 512K (64 bytes/line)
SMP alternatives: switching to UP code
Freeing SMP alternatives: 29k freed
ACPI: Core revision 20081204
Setting APIC routing to flat
Getting VERSION: 50011
Getting VERSION: 50011
Getting ID: 0
Getting ID: ff000000
Getting LVT0: 700
Getting LVT1: 10000
enabled ExtINT on CPU#0
ENABLING IO-APIC IRQs
init IO_APIC IRQs
 1-0 (apicid-pin) not connected
IOAPIC[0]: Set routing entry (1-1 -> 0x31 -> IRQ 1 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-2 -> 0x30 -> IRQ 0 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-3 -> 0x33 -> IRQ 3 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-4 -> 0x34 -> IRQ 4 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-5 -> 0x35 -> IRQ 5 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-6 -> 0x36 -> IRQ 6 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-7 -> 0x37 -> IRQ 7 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-8 -> 0x38 -> IRQ 8 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-9 -> 0x39 -> IRQ 9 Mode:1 Active:1)
IOAPIC[0]: Set routing entry (1-10 -> 0x3a -> IRQ 10 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-11 -> 0x3b -> IRQ 11 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-12 -> 0x3c -> IRQ 12 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-13 -> 0x3d -> IRQ 13 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-14 -> 0x3e -> IRQ 14 Mode:0 Active:0)
IOAPIC[0]: Set routing entry (1-15 -> 0x3f -> IRQ 15 Mode:0 Active:0)
 1-16 1-17 1-18 1-19 1-20 1-21 1-22 1-23 (apicid-pin) not connected
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
..MP-BIOS bug: 8254 timer not connected to IO-APIC
...trying to set up timer (IRQ0) through the 8259A ...
..... (found apic 0 pin 2) ...
....... failed.
...trying to set up timer as Virtual Wire IRQ...
..... failed.
...trying to set up timer as ExtINT IRQ...
..... failed :(.
Kernel panic - not syncing: IO-APIC + timer doesn't work!  Boot with apic=debug and send a report.  Then try booting with the 'noapic' option.

Pid: 1, comm: swapper Not tainted 2.6.29-06626-gb9d7dba #8
Call Trace:
 [<ffffffff805ee186>] ? panic+0x86/0x156
 [<ffffffff80297191>] ? default_enable+0x21/0x40
 [<ffffffff805f0bee>] ? _spin_lock_irqsave+0x2e/0x50
 [<ffffffff8024ccc5>] ? default_spin_lock_flags+0x5/0x10
 [<ffffffff805f0bee>] ? _spin_lock_irqsave+0x2e/0x50
 [<ffffffff804119cd>] ? delay_tsc+0x3d/0x70
 [<ffffffff807c149b>] ? setup_IO_APIC+0x85e/0x9fd
 [<ffffffff805f0b26>] ? __down_read+0xa6/0xc2
 [<ffffffff807bcdb2>] ? native_smp_prepare_cpus+0x2c3/0x351
 [<ffffffff807b2541>] ? kernel_init+0x52/0x1ae
 [<ffffffff80232f2a>] ? child_rip+0xa/0x20
 [<ffffffff807b24ef>] ? kernel_init+0x0/0x1ae
 [<ffffffff80232f20>] ? child_rip+0x0/0x20
------------[ cut here ]------------
WARNING: at kernel/smp.c:329 smp_call_function_many+0x1f9/0x260()
Hardware name: 
Modules linked in:
Pid: 1, comm: swapper Not tainted 2.6.29-06626-gb9d7dba #8
Call Trace:
 [<ffffffff8026421a>] ? warn_slowpath+0xea/0x160
 [<ffffffff80232f20>] ? child_rip+0x0/0x20
 [<ffffffff80232f20>] ? child_rip+0x0/0x20
 [<ffffffff80232f20>] ? child_rip+0x0/0x20
 [<ffffffff80232f20>] ? child_rip+0x0/0x20
 [<ffffffff80232f20>] ? child_rip+0x0/0x20
 [<ffffffff805ee2a4>] ? printk+0x4e/0x56
 [<ffffffff802788ef>] ? __kernel_text_address+0x2f/0x60
 [<ffffffff802366a0>] ? print_context_stack+0x70/0xd0
 [<ffffffff80291dad>] ? crash_kexec+0x6d/0x110
 [<ffffffff8024ccc5>] ? default_spin_lock_flags+0x5/0x10
 [<ffffffff805f0bee>] ? _spin_lock_irqsave+0x2e/0x50
 [<ffffffff8028b009>] ? smp_call_function_many+0x1f9/0x260
 [<ffffffff8028b090>] ? smp_call_function+0x20/0x30
 [<ffffffff80244f20>] ? native_smp_send_stop+0x20/0x40
 [<ffffffff805ee19a>] ? panic+0x9a/0x156
 [<ffffffff80297191>] ? default_enable+0x21/0x40
 [<ffffffff805f0bee>] ? _spin_lock_irqsave+0x2e/0x50
 [<ffffffff8024ccc5>] ? default_spin_lock_flags+0x5/0x10
 [<ffffffff805f0bee>] ? _spin_lock_irqsave+0x2e/0x50
 [<ffffffff804119cd>] ? delay_tsc+0x3d/0x70
 [<ffffffff807c149b>] ? setup_IO_APIC+0x85e/0x9fd
 [<ffffffff805f0b26>] ? __down_read+0xa6/0xc2
 [<ffffffff807bcdb2>] ? native_smp_prepare_cpus+0x2c3/0x351
 [<ffffffff807b2541>] ? kernel_init+0x52/0x1ae
 [<ffffffff80232f2a>] ? child_rip+0xa/0x20
 [<ffffffff807b24ef>] ? kernel_init+0x0/0x1ae
 [<ffffffff80232f20>] ? child_rip+0x0/0x20
---[ end trace 4eaa2a86a8e2da22 ]---