From mboxrd@z Thu Jan  1 00:00:00 1970
Received: from [140.186.70.92] (port=60902 helo=eggs.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43) id 1Op4hy-0001Ys-7z
	for qemu-devel@nongnu.org; Fri, 27 Aug 2010 15:35:27 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69)
	(envelope-from <camm@ualberta.ca>) id 1Op4hw-0008KH-C9
	for qemu-devel@nongnu.org; Fri, 27 Aug 2010 15:35:26 -0400
Received: from mail-vw0-f45.google.com ([209.85.212.45]:57027)
	by eggs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <camm@ualberta.ca>) id 1Op4hw-0008KB-92
	for qemu-devel@nongnu.org; Fri, 27 Aug 2010 15:35:24 -0400
Received: by vws19 with SMTP id 19so3342665vws.4
	for <qemu-devel@nongnu.org>; Fri, 27 Aug 2010 12:35:23 -0700 (PDT)
MIME-Version: 1.0
Sender: camm@ualberta.ca
In-Reply-To: <20100825022136.GB7547@valinux.co.jp>
References: <20100630032901.GA19142@valinux.co.jp>
	<AANLkTik54pAZL8oGhWnc6pVotDcLebCjZ4tPvVSItcKz@mail.gmail.com>
	<20100713204157.GI31689@valinux.co.jp>
	<AANLkTileIhU5wSChUV5VBA_uMEkGeTSq1bzpTMaFq9ws@mail.gmail.com>
	<20100714025216.GK31689@valinux.co.jp>
	<AANLkTin9EFs_FwDqHXP8zFTUIAPhTOjTAsf53zZiS3bQ@mail.gmail.com>
	<20100720095223.GC24957@valinux.co.jp>
	<20100721033101.GD11146@redhat.com>
	<20100721034918.GA6285@valinux.co.jp>
	<AANLkTinw-3cErM-ZAfXK_-S4F3oEkn49HJgwAu5=y_YJ@mail.gmail.com>
	<20100825022136.GB7547@valinux.co.jp>
Date: Fri, 27 Aug 2010 13:35:23 -0600
Message-ID: <AANLkTimtuhtPAZBZL9Pj1qtXm3P8-MAotJ8GP+0nJQs_@mail.gmail.com>
Subject: Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR
From: Cam Macdonell <cam@cs.ualberta.ca>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
List-Id: qemu-devel.nongnu.org
List-Unsubscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <http://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <http://lists.nongnu.org/mailman/listinfo/qemu-devel>,
	<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
To: Isaku Yamahata <yamahata@valinux.co.jp>
Cc: "qemu-devel@nongnu.org Developers" <qemu-devel@nongnu.org>, seabios@seabios.org, Avi Kivity <avi@redhat.com>, "Michael S. Tsirkin" <mst@redhat.com>

On Tue, Aug 24, 2010 at 8:21 PM, Isaku Yamahata <yamahata@valinux.co.jp> wr=
ote:
> On Tue, Aug 24, 2010 at 10:52:36AM -0600, Cam Macdonell wrote:
>> Hi, 64-bit BARs still do not seem to be working.
>>
>> When using the latest seabios the guest does not hit a "BUG:"
>> statement, but booting still fails
>>
>> HPET: 1 timers in total, 0 timers will be used for per-cpu timer
>> divide error: 0000 [#1] SMP
>> last sysfs file:
>> CPU 0
>> Modules linked in:
>>
>> Pid: 1, comm: swapper Not tainted 2.6.35+ #299 /Bochs
>> RIP: 0010:[<ffffffff812a9b5b>] =A0[<ffffffff812a9b5b>] hpet_alloc+0x12c/=
0x35b
>> RSP: 0018:ffff88007d7b3d80 =A0EFLAGS: 00010246
>> RAX: 00038d7ea4c68000 RBX: ffff88007d062cc0 RCX: 0000000000000000
>> RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff817bb9b0
>> RBP: ffff88007d7b3dc0 R08: 00000000000080d0 R09: ffffc90000000000
>> R10: ffff88007d72b5a0 R11: 0000000000000000 R12: ffff88007d7b3dd0
>> R13: ffffc90000000000 R14: 0000000000000000 R15: ffffffff817a41c3
>> FS: =A00000000000000000(0000) GS:ffff880002000000(0000) knlGS:0000000000=
000000
>> CS: =A00010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> CR2: 0000000000000000 CR3: 0000000001a42000 CR4: 00000000000006f0
>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> Process swapper (pid: 1, threadinfo ffff88007d7b2000, task ffff88007d7b8=
000)
>> Stack:
>> =A0ffff88007f43ab90 ffff88007f43ab90 ffffffff81ca6174 ffffffff81b1f5e1
>> <0> 0000000000000000 0000000000000100 0000000000000100 0000000000000000
>> <0> ffff88007d7b3e80 ffffffff810294ea 00000000fed00000 ffffc90000000000
>> Call Trace:
>> =A0[<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>> =A0[<ffffffff810294ea>] hpet_reserve_platform_timers+0x10b/0x115
>> =A0[<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>> =A0[<ffffffff81b1f64c>] hpet_late_init+0x6b/0xea
>> =A0[<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>> =A0[<ffffffff81002069>] do_one_initcall+0x5e/0x159
>> =A0[<ffffffff81b0d72a>] kernel_init+0x19a/0x228
>> =A0[<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
>> =A0[<ffffffff81b0d590>] ? kernel_init+0x0/0x228
>> =A0[<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
>> Code: 89 1d ca b2 b3 00 48 c1 ea 21 8b 73 34 49 c7 c7 c3 41 7a 81 48
>> 8d 04 02 4c 89 f2 48 c7 c7 b0 b9 7b 81 48 c1 ea 20 48 89 d1 31 d2 <48>
>> f7 f1 83 7b 30 01 48 c7 c1 86 1c 7d 81 49 0f 46 cf 48 89 43
>> RIP =A0[<ffffffff812a9b5b>] hpet_alloc+0x12c/0x35b
>> =A0RSP <ffff88007d7b3d80>
>> ---[ end trace a7919e7f17c0a725 ]---
>> Kernel panic - not syncing: Attempted to kill init!
>> Pid: 1, comm: swapper Tainted: G =A0 =A0 =A0D =A0 =A0 2.6.35+ #299
>> Call Trace:
>> =A0[<ffffffff81459a85>] panic+0x8b/0x10b
>> =A0[<ffffffff81056a83>] ? exit_ptrace+0x38/0x121
>> =A0[<ffffffff8104f9e8>] do_exit+0x7a/0x722
>> =A0[<ffffffff8104c3bd>] ? spin_unlock_irqrestore+0xe/0x10
>> =A0[<ffffffff8104cfd6>] ? kmsg_dump+0x12b/0x145
>> =A0[<ffffffff8145ccc8>] oops_end+0xbf/0xc7
>> =A0[<ffffffff8100d299>] die+0x5a/0x63
>> =A0[<ffffffff8145c6d2>] do_trap+0x121/0x130
>> =A0[<ffffffff8100b560>] do_divide_error+0x96/0x9f
>> =A0[<ffffffff812a9b5b>] ? hpet_alloc+0x12c/0x35b
>> =A0[<ffffffff8120cf80>] ? radix_tree_preload+0x34/0x88
>> =A0[<ffffffff8100a83b>] divide_error+0x1b/0x20
>> =A0[<ffffffff812a9b5b>] ? hpet_alloc+0x12c/0x35b
>> =A0[<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>> =A0[<ffffffff810294ea>] hpet_reserve_platform_timers+0x10b/0x115
>> =A0[<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>> =A0[<ffffffff81b1f64c>] hpet_late_init+0x6b/0xea
>> =A0[<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>> =A0[<ffffffff81002069>] do_one_initcall+0x5e/0x159
>> =A0[<ffffffff81b0d72a>] kernel_init+0x19a/0x228
>> =A0[<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
>> =A0[<ffffffff81b0d590>] ? kernel_init+0x0/0x228
>> =A0[<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
>>
>> seabios output for the device:
>>
>> PCI: bus=3D0 devfn=3D0x20: vendor_id=3D0x1af4 device_id=3D0x1110
>> region 0: 0xf1020000
>> region 2: 0x00000000
>> init smm
>>
>> Running the latest seabios, the debug output only remaps the BAR
>> twice, once with a potentially correct address of e00000000
>>
>> pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
>
> The upstream seabios lacks overflow check at the moment.
> I haven't found time to address PMM yet.
>
>
>> pci_default_write_config: (val) 0x0 -> 0x18 (addr)
>> IVSHMEM: guest pci addr =3D e0000000, guest h/w addr =3D 2164588544, siz=
e =3D 20000000
>> pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
>> pci_default_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> pci_default_write_config: (val) 0x0 -> 0x1c (addr)
>> IVSHMEM: guest pci addr =3D ffffffff00000000, guest h/w addr =3D
>> 2164588544, size =3D 20000000
>> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
>> pci_default_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0x0 <- 0x20 (addr)
>>
>> the pci writes are all still 0, I can't see how my debug statements
>> are incorrect though. =A0Below is my trivial pci config debugging patch.
>
> The debug out put should be before the for-loop.
>
> =A0 =A0for (i =3D 0; i < l && addr + i < config_size; val >>=3D 8, ++i) {
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 ^^^^^^^^^
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =
=A0 =A0 =A0 =A0 =A0 =A0 Here val becomes 0
> =A0 =A0 =A0 =A0uint8_t wmask =3D d->wmask[addr + i];
> =A0 =A0 =A0 =A0d->config[addr + i] =3D (d->config[addr + i] & ~wmask) | (=
val & wmask);
> =A0 =A0}
>
> --
> yamahata
>
>

Ah, thanks.  I moved the debug statement and now the writes are the
proper values.

In seabios-kvm, it seems the guest is writing the address of c040 to
0x1c which results to the 48-bit address c04000000000.

pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x1110 <- 0x2 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x500 <- 0xa (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x1110 <- 0x2 (addr)
pci_read_config: (val) 0x0 <- 0x10 (addr)
pci_write_config: (val) 0xffffffff -> 0x10 (addr)
pci_read_config: (val) 0xffffff00 <- 0x10 (addr)
pci_write_config: (val) 0x0 -> 0x10 (addr)
pci_read_config: (val) 0x0 <- 0x10 (addr)
pci_write_config: (val) 0xf1020000 -> 0x10 (addr)
pci_read_config: (val) 0x0 <- 0x14 (addr)
pci_write_config: (val) 0xffffffff -> 0x14 (addr)
pci_read_config: (val) 0x0 <- 0x14 (addr)
pci_write_config: (val) 0x0 -> 0x14 (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0xffffffff -> 0x18 (addr)
pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
pci_write_config: (val) 0x4 -> 0x18 (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0xffffffff -> 0x1c (addr)
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0xc040 -> 0x1c (addr)

<snip>

pci_read_config: (val) 0x0 <- 0x4 (addr)
pci_write_config: (val) 0x3 -> 0x4 (addr)
IVSHMEM: guest pci addr =3D c04000000000, guest h/w addr =3D 2164588544,
size =3D 20000000
pci_read_config: (val) 0x1 <- 0x3d (addr)

<snip>

pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0xffffffff -> 0x18 (addr)
IVSHMEM: guest pci addr =3D c040e0000000, guest h/w addr =3D 2164588544,
size =3D 20000000
pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
pci_write_config: (val) 0x4 -> 0x18 (addr)
IVSHMEM: guest pci addr =3D c04000000000, guest h/w addr =3D 2164588544,
size =3D 20000000
pci_read_config: (val) 0xc040 <- 0x1c (addr)
pci_write_config: (val) 0xffffffff -> 0x1c (addr)
IVSHMEM: guest pci addr =3D ffffffff00000000, guest h/w addr =3D
2164588544, size =3D 20000000
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_write_config: (val) 0xc040 -> 0x1c (addr)
IVSHMEM: guest pci addr =3D c04000000000, guest h/w addr =3D 2164588544,
size =3D 20000000
pci_read_config: (val) 0x0 <- 0x20 (addr)
pci_write_config: (val) 0xffffffff -> 0x20 (addr)
pci_read_config: (val) 0x0 <- 0x20 (addr)
pci_write_config: (val) 0x0 -> 0x20 (addr)

In upstream seabios.git, the c040 is not written, but the device
returns ffffffff from 0x1c (only reads and writes to 0x18 and 0x1c are
shown below)

pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0xffffffff -> 0x18 (addr)
pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
pci_write_config: (val) 0x4 -> 0x18 (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0xffffffff -> 0x18 (addr)
IVSHMEM: guest pci addr =3D e0000000, guest h/w addr =3D 2164588544, size =
=3D 20000000
pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
pci_write_config: (val) 0x4 -> 0x18 (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0xffffffff -> 0x1c (addr)
IVSHMEM: guest pci addr =3D ffffffff00000000, guest h/w addr =3D
2164588544, size =3D 20000000
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)

Thanks,
Cam