From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=47470 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OrLgg-0007QG-KR for qemu-devel@nongnu.org; Thu, 02 Sep 2010 22:07:31 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OrLgf-0004lF-83 for qemu-devel@nongnu.org; Thu, 02 Sep 2010 22:07:30 -0400 Received: from mail.valinux.co.jp ([210.128.90.3]:46307) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OrLge-0004kr-OJ for qemu-devel@nongnu.org; Thu, 02 Sep 2010 22:07:29 -0400 Date: Fri, 3 Sep 2010 11:20:12 +0900 From: Isaku Yamahata Subject: Re: [Qemu-devel] Template for developing a Qemu device with PCIe?and MSI-X Message-ID: <20100903022012.GA25721@valinux.co.jp> References: <20100902174242.4cb1fb0f@shadowfax.no-ip.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-2022-jp Content-Disposition: inline In-Reply-To: <20100902174242.4cb1fb0f@shadowfax.no-ip.com> List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Adnan Khaleel Cc: Cam Macdonell , qemu-devel@nongnu.org On Thu, Sep 02, 2010 at 12:42:42PM -0500, Adnan Khaleel wrote: > I've tried everything you mentioned and I still get the same problem. The only > thing that seems to avoid that issue is if I reduce the aperture size from > 0x2000000000ull to 0x2000000ull. I suppose that Cam is seeing the same issue. Right now seabios can't handle too huge BAR due to overflow. There is a rejected patch floating around, but I haven't created a revised patch yet. > > Here is the relevant section of code: > > static const unsigned long long BAR_Regions[6][2] = > { > // len , type > { 0x2000000ull, PCI_BASE_ADDRESS_SPACE_MEMORY | > PCI_BASE_ADDRESS_MEM_TYPE_64} , //BAR0, > { 0, 0} , // BAR1 > { 0x2000000ull, PCI_BASE_ADDRESS_SPACE_IO } , //BAR2, > { 0, 0} , // BAR3 for MSI-X > { 0, 0} , // BAR4 > { 0, 0} , // BAR5 > }; > > static int pcie_msix_initfn(PCIDevice *pci_dev) > { > PCIE_MSIX_DEVState *d = DO_UPCAST(PCIE_MSIX_DEVState, dev, pci_dev); > PCIBridge *br = DO_UPCAST(PCIBridge, dev, pci_dev); > PCIEPort *p = DO_UPCAST(PCIEPort, br, br); > int rc, i; > > PRINT_DEBUG("%s: PCIE MSIX Device init...\n", __FUNCTION__); > > pci_config_set_vendor_id(d->dev.config, PCIE_MSIX_VID); > pci_config_set_device_id(d->dev.config, PCIE_MSIX_DID); > > memcpy(d->dev.config, g_cfg_init, sizeof(g_cfg_init[0x20])); > d->mmio_index = cpu_register_io_memory(pcie_msix_mem_read_fn, > pcie_msix_mem_write_fn, d); > > int msix_mem_bar = 0; // Since its a 64bit BAR, we take up BAR0 & BAR1 > int msix_io_bar = 2; > int msix_mmio_bar = 3; > > pci_register_bar(&d->dev, msix_mem_bar, BAR_Regions[msix_mem_bar][0], > BAR_Regions[msix_mem_bar][1], pcie_msix_mem_map); > pci_register_bar(&d->dev, msix_io_bar, BAR_Regions[msix_io_bar][0], > BAR_Regions[msix_io_bar][1], pcie_msix_io_map); > > rc = msix_init(&d->dev, d->vectors, msix_mmio_bar, 0); > > if (!rc) { > PRINT_DEBUG("%s: Registering Bar %i as I/O BAR\n", __FUNCTION__, > msix_mmio_bar); > pci_register_bar(&d->dev, msix_mmio_bar, msix_bar_size(&d->dev), > PCI_BASE_ADDRESS_SPACE_MEMORY, msix_mmio_map); > PRINT_DEBUG("%s: MSI-X initialized (%d vectors)\n", __FUNCTION__, d-> > vectors); > } > else { > PRINT_DEBUG("%s: MSI-X initialization failed!\n", __FUNCTION__); > return rc; > } > > // Activate the vectors > for (i = 0; i < d->vectors; i++) { > msix_vector_use(&d->dev, i); > } > > rc = pci_pcie_cap_init(&d->dev, PCIE_MSIX_EXP_OFFSET, > PCI_EXP_TYPE_ENDPOINT, p->port); > if (rc < 0) { > return rc; > } > > pcie_cap_flr_init(&d->dev, &pcie_msix_flr); > pcie_cap_deverr_init(&d->dev); > pcie_cap_ari_init(&d->dev); > rc = pcie_aer_init(&d->dev, PCIE_MSIX_AER_OFFSET); > if (rc < 0) { > return rc; > } > > PRINT_DEBUG("%s: Init done\n", __FUNCTION__); > return 0; > } > > Another question I have is why doesn't the device show up when I try a cat / > proc/interrupts. > > linux-an84:~/AriesKernelModules/gni/aries/ghal # cat /proc/interrupts > CPU0 > 0: 694 IO-APIC-edge timer > 1: 6 IO-APIC-edge i8042 > 4: 753 IO-APIC-edge serial > 8: 1 IO-APIC-edge rtc0 > 9: 0 IO-APIC-fasteoi acpi > 12: 89 IO-APIC-edge i8042 > 14: 3522 IO-APIC-edge ata_piix > 15: 785 IO-APIC-edge ata_piix > 16: 162 IO-APIC-fasteoi eth0 > 4344: 0 PCI-MSI-edge aerdrv > 4345: 0 PCI-MSI-edge aerdrv > 4346: 0 PCI-MSI-edge aerdrv > 4347: 0 PCI-MSI-edge aerdrv > 4348: 0 PCI-MSI-edge aerdrv > 4349: 0 PCI-MSI-edge aerdrv > 4350: 0 PCI-MSI-edge aerdrv > 4351: 0 PCI-MSI-edge aerdrv > NMI: 0 Non-maskable interrupts > LOC: 107095 Local timer interrupts > RES: 0 Rescheduling interrupts > CAL: 0 function call interrupts > TLB: 0 TLB shootdowns > TRM: 0 Thermal event interrupts > THR: 0 Threshold APIC interrupts > SPU: 0 Spurious interrupts > ERR: 0 > > Shouldn't there be an entry for the MSI-X device? > > Thanks for all your input. > > AK > > > > ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ > Probably what you want is something like > > { 0x2000000000ull, PCI_BASE_ADDRESS_SPACE_MEMORY | > PCI_BASE_ADDRESS_MEM_TYPE_64} , //BAR0 > { 0, 0} , //BAR1 > // 64bit BAR occupies 2 BAR entries so that BAR1 can't be used. > { 0x2000000ull, PCI_BASE_ADDRESS_SPACE_IO } , //BAR2 > { 0, 0} , //BAR3 > // for MSI-X > { 0, 0} , //BAR4 > { 0, 0} //BAR5 > > > -- yamahata