From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=48862 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OTeuv-0006yK-LR for qemu-devel@nongnu.org; Tue, 29 Jun 2010 13:48:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OTeut-0006z5-Qr for qemu-devel@nongnu.org; Tue, 29 Jun 2010 13:48:17 -0400 Received: from mail-vw0-f45.google.com ([209.85.212.45]:49378) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OTeut-0006yd-L2 for qemu-devel@nongnu.org; Tue, 29 Jun 2010 13:48:15 -0400 Received: by vws18 with SMTP id 18so361554vws.4 for ; Tue, 29 Jun 2010 10:48:14 -0700 (PDT) MIME-Version: 1.0 Sender: camm@ualberta.ca In-Reply-To: <4C2997C5.1020302@redhat.com> References: <4C175E30.2030605@redhat.com> <4C270E25.7070409@redhat.com> <4C2997C5.1020302@redhat.com> Date: Tue, 29 Jun 2010 11:48:13 -0600 Message-ID: From: Cam Macdonell Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Subject: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Avi Kivity Cc: "qemu-devel@nongnu.org Developers" , "Michael S. Tsirkin" On Tue, Jun 29, 2010 at 12:50 AM, Avi Kivity wrote: > On 06/28/2010 11:38 PM, Cam Macdonell wrote: >> >>> >>>>> Is this really the address the guest programmed, or is qemu >>>>> misinterpreting >>>>> it? >>>>> >>>>> >>> >>> Well, what's the answer? >>> >> >> You're going to have to give me a hint on how to determine that. >> >> lspci in the guest shows the following >> >> Memory at c20000000000 (64-bit, non-prefetchable) [size=3D1024M] >> >> does that demonstrate a guest generated address? >> > > That's the result of a round trip: the guest programmed the address and t= hen > read it back. =A0It could have been screwed up in the first place, or per= haps > qemu screwed it up. > > Add a printf() to the config space handlers in qemu (likely in your own > code) on writes and reads, and show the relevant writes (and reads) for t= his > BAR. > > That's the theory of deductive debugging; however browsing the code shows > the guest is at fault: > > =A0 =A0 =A0 =A0for (i =3D 0; i < PCI_NUM_REGIONS; i++) { > =A0 =A0 =A0 =A0 =A0 =A0int ofs; > =A0 =A0 =A0 =A0 =A0 =A0if (i =3D=3D PCI_ROM_SLOT) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ofs =3D PCI_ROM_ADDRESS; > =A0 =A0 =A0 =A0 =A0 =A0else > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0ofs =3D PCI_BASE_ADDRESS_0 + i * 4; > > =A0 =A0 =A0 =A0 =A0 =A0u32 old =3D pci_config_readl(bdf, ofs); > =A0 =A0 =A0 =A0 =A0 =A0u32 mask; > =A0 =A0 =A0 =A0 =A0 =A0if (i =3D=3D PCI_ROM_SLOT) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mask =3D PCI_ROM_ADDRESS_MASK; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pci_config_writel(bdf, ofs, mask); > =A0 =A0 =A0 =A0 =A0 =A0} else { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (old & PCI_BASE_ADDRESS_SPACE_IO) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mask =3D PCI_BASE_ADDRESS_IO_MASK; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0else > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0mask =3D PCI_BASE_ADDRESS_MEM_MASK= ; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pci_config_writel(bdf, ofs, ~0); > =A0 =A0 =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0 =A0 =A0u32 val =3D pci_config_readl(bdf, ofs); > =A0 =A0 =A0 =A0 =A0 =A0pci_config_writel(bdf, ofs, old); > > =A0 =A0 =A0 =A0 =A0 =A0if (val !=3D 0) { > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0u32 size =3D (~(val & mask)) + 1; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0if (val & PCI_BASE_ADDRESS_SPACE_IO) > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0paddr =3D &pci_bios_io_addr; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0else > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0paddr =3D &pci_bios_mem_addr; > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0*paddr =3D ALIGN(*paddr, size); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0pci_set_io_region_addr(bdf, i, *paddr); > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0*paddr +=3D size; > =A0 =A0 =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0} > =A0 =A0 =A0 =A0break; > =A0 =A0} > > Seabios completely ignore the 64-bitness of the BAR. =A0Looks like it als= o > thinks the second half of the BAR is an I/O region instead of memory (hen= ce > the c200, that's part of the pci portio region. > > Do post those reads and writes, I think there's more than one thing wrong > here. Here it is, I added the debug statements to pci_read_config and pci_default_write_config. here are the reads and writes to offsets 0x18 and 0x1c where a 64-bit BAR2 config would be configured pci_read_config: (val) 0x4 <- 0x18 (addr) pci_write_config: (val) 0x0 -> 0x18 (addr) pci_read_config: (val) 0xc0000004 <- 0x18 (addr) pci_write_config: (val) 0x0 -> 0x18 (addr) pci_read_config: (val) 0x4 <- 0x18 (addr) pci_write_config: (val) 0x0 -> 0x18 (addr) pci_read_config: (val) 0x0 <- 0x1c (addr) pci_write_config: (val) 0x0 -> 0x1c (addr) pci_read_config: (val) 0xffffffff <- 0x1c (addr) pci_write_config: (val) 0x0 -> 0x1c (addr) pci_read_config: (val) 0x0 <- 0x1c (addr) pci_write_config: (val) 0x0 -> 0x1c (addr) pci_read_config: (val) 0x4 <- 0x18 (addr) pci_write_config: (val) 0x0 -> 0x18 (addr) pci_read_config: (val) 0xc0000004 <- 0x18 (addr) pci_write_config: (val) 0x0 -> 0x18 (addr) pci_read_config: (val) 0xc040 <- 0x1c (addr) pci_write_config: (val) 0x0 -> 0x1c (addr) pci_read_config: (val) 0xffffffff <- 0x1c (addr) pci_write_config: (val) 0x0 -> 0x1c (addr) the complete read/write profile is below along with debug statements from the map functions for the BARs (prefixed with IVSHMEM) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x1110 <- 0x2 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x500 <- 0xa (addr) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x1110 <- 0x2 (addr) pci_read_config: (val) 0x0 <- 0x10 (addr) pci_write_config: (val) 0x0 -> 0x10 (addr) pci_read_config: (val) 0xffffff00 <- 0x10 (addr) pci_write_config: (val) 0x0 -> 0x10 (addr) pci_read_config: (val) 0x0 <- 0x10 (addr) pci_write_config: (val) 0x0 -> 0x10 (addr) pci_read_config: (val) 0x0 <- 0x14 (addr) pci_write_config: (val) 0x0 -> 0x14 (addr) pci_read_config: (val) 0xfffff000 <- 0x14 (addr) pci_write_config: (val) 0x0 -> 0x14 (addr) pci_read_config: (val) 0x0 <- 0x14 (addr) pci_write_config: (val) 0x0 -> 0x14 (addr) pci_read_config: (val) 0x4 <- 0x18 (addr) pci_write_config: (val) 0x0 -> 0x18 (addr) pci_read_config: (val) 0xc0000004 <- 0x18 (addr) pci_write_config: (val) 0x0 -> 0x18 (addr) pci_read_config: (val) 0x4 <- 0x18 (addr) pci_write_config: (val) 0x0 -> 0x18 (addr) pci_read_config: (val) 0x0 <- 0x1c (addr) pci_write_config: (val) 0x0 -> 0x1c (addr) pci_read_config: (val) 0xffffffff <- 0x1c (addr) pci_write_config: (val) 0x0 -> 0x1c (addr) pci_read_config: (val) 0x0 <- 0x1c (addr) pci_write_config: (val) 0x0 -> 0x1c (addr) pci_read_config: (val) 0x0 <- 0x20 (addr) pci_write_config: (val) 0x0 -> 0x20 (addr) pci_read_config: (val) 0x0 <- 0x20 (addr) pci_write_config: (val) 0x0 -> 0x20 (addr) pci_read_config: (val) 0x0 <- 0x24 (addr) pci_write_config: (val) 0x0 -> 0x24 (addr) pci_read_config: (val) 0x0 <- 0x24 (addr) pci_write_config: (val) 0x0 -> 0x24 (addr) pci_read_config: (val) 0x0 <- 0x30 (addr) pci_write_config: (val) 0x0 -> 0x30 (addr) pci_read_config: (val) 0x0 <- 0x30 (addr) pci_write_config: (val) 0x0 -> 0x30 (addr) pci_read_config: (val) 0x0 <- 0x4 (addr) pci_write_config: (val) 0x0 -> 0x4 (addr) IVSHMEM: guest pci addr =3D c04000000000, guest h/w addr =3D 1090912256, size =3D 40000000 pci_read_config: (val) 0x1 <- 0x3d (addr) pci_write_config: (val) 0x0 -> 0x3c (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x1 <- 0x3d (addr) pci_read_config: (val) 0xb <- 0x3c (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x5000000 <- 0x8 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x500 <- 0xa (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x11101af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x500 <- 0xa (addr) pci_read_config: (val) 0x11101af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x0 <- 0x30 (addr) pci_write_config: (val) 0x0 -> 0x30 (addr) pci_read_config: (val) 0x0 <- 0x30 (addr) pci_write_config: (val) 0x0 -> 0x30 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x5000000 <- 0x8 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x11101af4 <- 0x0 (addr) pci_read_config: (val) 0x500 <- 0xa (addr) pci_read_config: (val) 0x1af4 <- 0x0 (addr) pci_read_config: (val) 0x1110 <- 0x2 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x5000000 <- 0x8 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x11101af4 <- 0x0 (addr) pci_read_config: (val) 0x0 <- 0xe (addr) pci_read_config: (val) 0x10 <- 0x6 (addr) pci_read_config: (val) 0x40 <- 0x34 (addr) pci_read_config: (val) 0x11 <- 0x40 (addr) pci_read_config: (val) 0x0 <- 0x41 (addr) pci_read_config: (val) 0x5000000 <- 0x8 (addr) pci_read_config: (val) 0x10 <- 0x6 (addr) pci_read_config: (val) 0x40 <- 0x34 (addr) pci_read_config: (val) 0x11 <- 0x40 (addr) pci_read_config: (val) 0x0 <- 0x41 (addr) pci_read_config: (val) 0x1 <- 0x3d (addr) pci_read_config: (val) 0xb <- 0x3c (addr) pci_read_config: (val) 0xf2040000 <- 0x10 (addr) pci_write_config: (val) 0x0 -> 0x10 (addr) pci_read_config: (val) 0xffffff00 <- 0x10 (addr) pci_write_config: (val) 0x0 -> 0x10 (addr) pci_read_config: (val) 0xf2041000 <- 0x14 (addr) pci_write_config: (val) 0x0 -> 0x14 (addr) pci_read_config: (val) 0xfffff000 <- 0x14 (addr) pci_write_config: (val) 0x0 -> 0x14 (addr) pci_read_config: (val) 0x4 <- 0x18 (addr) pci_write_config: (val) 0x0 -> 0x18 (addr) IVSHMEM: guest pci addr =3D c040c0000000, guest h/w addr =3D 1090912256, size =3D 40000000 pci_read_config: (val) 0xc0000004 <- 0x18 (addr) pci_write_config: (val) 0x0 -> 0x18 (addr) IVSHMEM: guest pci addr =3D c04000000000, guest h/w addr =3D 1090912256, size =3D 40000000 pci_read_config: (val) 0xc040 <- 0x1c (addr) pci_write_config: (val) 0x0 -> 0x1c (addr) IVSHMEM: guest pci addr =3D ffffffff00000000, guest h/w addr =3D 1090912256, size =3D 40000000 pci_read_config: (val) 0xffffffff <- 0x1c (addr) pci_write_config: (val) 0x0 -> 0x1c (addr) IVSHMEM: guest pci addr =3D c04000000000, guest h/w addr =3D 1090912256, size =3D 40000000 pci_read_config: (val) 0x0 <- 0x20 (addr) pci_write_config: (val) 0x0 -> 0x20 (addr) pci_read_config: (val) 0x0 <- 0x20 (addr) pci_write_config: (val) 0x0 -> 0x20 (addr) pci_read_config: (val) 0x0 <- 0x24 (addr) pci_write_config: (val) 0x0 -> 0x24 (addr) pci_read_config: (val) 0x0 <- 0x24 (addr) pci_write_config: (val) 0x0 -> 0x24 (addr) pci_read_config: (val) 0x0 <- 0x30 (addr) pci_write_config: (val) 0x0 -> 0x30 (addr) pci_read_config: (val) 0x0 <- 0x30 (addr) pci_write_config: (val) 0x0 -> 0x30 (addr) pci_read_config: (val) 0x1af4 <- 0x2c (addr) pci_read_config: (val) 0x1100 <- 0x2e (addr) pci_read_config: (val) 0x10 <- 0x6 (addr) pci_read_config: (val) 0x40 <- 0x34 (addr) pci_read_config: (val) 0x11 <- 0x40 (addr) pci_read_config: (val) 0x0 <- 0x41 (addr) pci_read_config: (val) 0x10 <- 0x6 (addr) pci_read_config: (val) 0x40 <- 0x34 (addr) pci_read_config: (val) 0x11 <- 0x40 (addr) pci_read_config: (val) 0x0 <- 0x41 (addr) pci_read_config: (val) 0x10 <- 0x6 (addr) pci_read_config: (val) 0x40 <- 0x34 (addr) pci_read_config: (val) 0x11 <- 0x40 (addr) pci_read_config: (val) 0x0 <- 0x41 (addr) pci_read_config: (val) 0x10 <- 0x6 (addr) pci_read_config: (val) 0x40 <- 0x34 (addr) pci_read_config: (val) 0x11 <- 0x40 (addr) pci_read_config: (val) 0x0 <- 0x41 (addr) pci_read_config: (val) 0x3 <- 0x4 (addr) pci_read_config: (val) 0x3 <- 0x4 (addr) pci_read_config: (val) 0x0 <- 0xc (addr) pci_read_config: (val) 0x3 <- 0x4 (addr) pci_read_config: (val) 0x3 <- 0x4 (addr) pci_read_config: (val) 0x10 <- 0x6 (addr) pci_read_config: (val) 0x40 <- 0x34 (addr) pci_read_config: (val) 0x11 <- 0x40 (addr) pci_read_config: (val) 0x0 <- 0x41 (addr) > > > -- > I have a truly marvellous patch that fixes the bug which this > signature is too narrow to contain. > >