From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=52690 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Oqset-0002ze-NU for qemu-devel@nongnu.org; Wed, 01 Sep 2010 15:07:45 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1Oqser-0003VE-6w for qemu-devel@nongnu.org; Wed, 01 Sep 2010 15:07:43 -0400 Received: from smtp.mailix.net ([66.11.225.183]:31570) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1Oqseq-0003Uu-Lm for qemu-devel@nongnu.org; Wed, 01 Sep 2010 15:07:41 -0400 From: "Adnan Khaleel" Subject: Re: [Qemu-devel] Template for developing a Qemu device with PCIe and MSI-X In-Reply-To: AANLkTimmReFQzgQKL1b1mrCjvRwyGt2fynib_xbHX-B_@mail.gmail.com Message-ID: <20100901190733.5ea9ca03@shadowfax.no-ip.com> Date: Wed, 01 Sep 2010 14:07:33 -0500 MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-----------b0469c9ed3f58912faffa2921ef5b58d" Reply-To: adnan@khaleel.us List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Cam Macdonell Cc: Isaku Yamahata , qemu-devel@nongnu.org This is a multi-part message in MIME format. -------------b0469c9ed3f58912faffa2921ef5b58d Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Yamahata, Cam, Thank you both very much for pointers about Qemu coding for PCIe and MSI= -X. I'm at a point where I can see my device when I do an lspci -t -v as sho= wn below. linux-an84:~ # lspci -t -v -[0000:00]-+-00.0 Intel Corporation 82G33/G31/P35/P31 Express DRAM Cont= roller +-01.0 Cirrus Logic GD 5446 +-04.0-[0000:20]-- +-18.0-[0000:21]-- +-18.1-[0000:22]-- +-18.2-[0000:23]-- +-18.3-[0000:24]-- +-18.4-[0000:25]-- +-18.5-[0000:26]-- +-19.0-[0000:36-bf]--+-00.0-[0000:37-47]--+-00.0-[0000:38]-- | | +-01.0-[0000:39]-- | | +-02.0-[0000:3a]-- | | +-03.0-[0000:3b]-- | | +-04.0-[0000:3c]-- | | +-05.0-[0000:3d]-- | | +-06.0-[0000:3e]-- | | +-07.0-[0000:3f]-- | | +-08.0-[0000:40]---= -00.0 Cray Inc Device 0301 <- The device that I've included However, I'm having a bit of an issue with the MSI-X. I'm following the code examples in virtio-pci.c and ivshmem.c that Cam p= ointed out to. I've got bar 0&1 already occupied so I assign the msix=5F= mmio=5Fmap to bar 2. However, when I do that, Qemu fails to boot and fai= ls with the following assertion fail: unused outb: port=3D0x00f1 data=3D0x00 qemu: fatal: Trying to execute code outside RAM or ROM at 0x00000000= 00100000 Couple of thins I'm unsure about: 1. Am I registering the 64bit bar addresses correctly=3F pci=5Fregister=5Fbar(&d->dev, i, BAR=5FRegions[i][0], PCI=5FBASE=5FADD= RESS=5FSPACE=5FIO | PCI=5FBASE=5FADDRESS=5FMEM=5FTYPE=5F64, pcie=5Fmsi= x=5Fio=5Fmap); 2. In the function int msix=5Finit(PCIDevice *pdev, unsigned short nentr= ies, unsigned bar=5Fnr, unsigned bar=5Fsize); I'm not sure what bar=5Fnr is, however setting it to 1 (as in code examp= les above) or to 2 (bar that I want to register the msix=5Fmmio=5Fmap to= ) both fail with the same error. rc =3D msix=5Finit(&d->dev, d->vectors, 2, 0); : : pci=5Fregister=5Fbar(&d->dev, 2, msix=5Fbar=5Fsize(&d->dev), PCI=5FBAS= E=5FADDRESS=5FSPACE=5FMEMORY | PCI=5FBASE=5FADDRESS=5FMEM=5FTYPE=5F64, m= six=5Fmmio=5Fmap); Here is my init function code in its entirety: static const unsigned long long BAR=5FRegions[6][2] =3D=20 { // len , type=20 { 0x2000000000ull, PCI=5FBASE=5FADDRESS=5FSPACE=5FMEMORY} , //BAR0 { 0x2000000ull, PCI=5FBASE=5FADDRESS=5FSPACE=5FIO } , //BAR1 { 0, 0} , //BAR2 { 0, 0} , //BAR3 { 0, 0} , //BAR4 { 0, 0} //BAR5=20 }; static int pcie=5Fmsix=5Finitfn(PCIDevice *pci=5Fdev) { PCIE=5FMSIX=5FDEVState *d =3D DO=5FUPCAST(PCIE=5FMSIX=5FDEVState, de= v, pci=5Fdev); PCIBridge *br =3D DO=5FUPCAST(PCIBridge, dev, pci=5Fdev); PCIEPort *p =3D DO=5FUPCAST(PCIEPort, br, br); int rc, i; PRINT=5FDEBUG("%s: PCIE MSIX Device init...\n", =5F=5FFUNCTION=5F=5F= ); pci=5Fconfig=5Fset=5Fvendor=5Fid(d->dev.config, PCIE=5FMSIX=5FVID); pci=5Fconfig=5Fset=5Fdevice=5Fid(d->dev.config, PCIE=5FMSIX=5FDID); d->dev.config[PCI=5FREVISION=5FID] =3D PCIE=5FMSIX=5FVERSION; d->dev.config[PCI=5FSUBSYSTEM=5FVENDOR=5FID] =3D PCIE=5FMSIX=5FVID= & 0xff; d->dev.config[PCI=5FSUBSYSTEM=5FVENDOR=5FID+1] =3D (PCIE=5FMSIX=5FVI= D >> 8) & 0xff; d->dev.config[PCI=5FSUBSYSTEM=5FID] =3D PCIE=5FMSIX=5FSS=5FDID & 0= xff; d->dev.config[PCI=5FSUBSYSTEM=5FID+1] =3D (PCIE=5FMSIX=5FSS=5FDID >>= 8) & 0xff; d->mmio=5Findex =3D cpu=5Fregister=5Fio=5Fmemory(pcie=5Fmsix=5Fmem= =5Fread=5Ffn, pcie=5Fmsix=5Fmem=5Fwrite=5Ffn, d); =20 for(i=3D0; i < PCI=5FNUM=5FREGIONS -1; i++) { //-1 for the Exp ROM B= AR if(BAR=5FRegions[i][0] !=3D 0) {=20 if(BAR=5FRegions[i][1] =3D=3D PCI=5FBASE=5FADDRESS=5FSPACE= =5FIO)=20 { //io region PRINT=5FDEBUG("%s: Registering Bar %i as I/O BAR\n", =5F= =5FFUNCTION=5F=5F, i); pci=5Fregister=5Fbar(&d->dev, i, BAR=5FRegions[i][0], PC= I=5FBASE=5FADDRESS=5FSPACE=5FIO | PCI=5FBASE=5FADDRESS=5FMEM=5FTYPE=5F64= , pcie=5Fmsix=5Fio=5Fmap); } else {=20 //mem region PRINT=5FDEBUG("%s: Registering Bar %i as MEM BAR\n", =5F= =5FFUNCTION=5F=5F, i); pci=5Fregister=5Fbar(&d->dev, i, BAR=5FRegions[i][0], PC= I=5FBASE=5FADDRESS=5FSPACE=5FMEMORY | PCI=5FBASE=5FADDRESS=5FMEM=5FTYPE= =5F64, pcie=5Fmsix=5Fmem=5Fmap); } } } d->dev.config[PCI=5FINTERRUPT=5FPIN] =3D 1; rc =3D msix=5Finit(&d->dev, d->vectors, 2, 0); =20 if (!rc) { PRINT=5FDEBUG("%s: Registering Bar %i as I/O BAR\n", =5F=5FFUNCT= ION=5F=5F, i); pci=5Fregister=5Fbar(&d->dev, 2, msix=5Fbar=5Fsize(&d->dev), PCI= =5FBASE=5FADDRESS=5FSPACE=5FMEMORY | PCI=5FBASE=5FADDRESS=5FMEM=5FTYPE= =5F64, msix=5Fmmio=5Fmap); PRINT=5FDEBUG("%s: MSI-X initialized (%d vectors)\n", =5F=5FFUNC= TION=5F=5F, d->vectors); } else { PRINT=5FDEBUG("%s: MSI-X initialization failed!\n", =5F=5FFUNCTI= ON=5F=5F); exit(1); } =20 // Activate the vectors for (i =3D 0; i < d->vectors; i++) { msix=5Fvector=5Fuse(&d->dev, i); } rc =3D pci=5Fpcie=5Fcap=5Finit(&d->dev, PCIE=5FMSIX=5FEXP=5FOFFSET, = PCI=5FEXP=5FTYPE=5FENDPOINT, p->port); if (rc < 0) { return rc; } pcie=5Fcap=5Fflr=5Finit(&d->dev, &pcie=5Fmsix=5Fflr); pcie=5Fcap=5Fdeverr=5Finit(&d->dev); pcie=5Fcap=5Fari=5Finit(&d->dev); rc =3D pcie=5Faer=5Finit(&d->dev, PCIE=5FMSIX=5FAER=5FOFFSET); if (rc < 0) { return rc; } PRINT=5FDEBUG("%s: Init done\n", =5F=5FFUNCTION=5F=5F); return 0; } Thanks AK =5F=5F=5F=5F=5F =20 From: Cam Macdonell [mailto:cam@cs.ualberta.ca] To: adnan@khaleel.us Cc: Isaku Yamahata [mailto:yamahata@valinux.co.jp], qemu-devel@nongnu.or= g Sent: Fri, 27 Aug 2010 10:48:48 -0500 Subject: Re: [Qemu-devel] Template for developing a Qemu device with PCI= e and MSI-X On Wed, Aug 25, 2010 at 4:39 PM, Adnan Khaleel wrote: > Hi Isaku, > > I've made some progress in coding the device template but its no whe= re near > complete. > > I've created some files and am attaching it to this note. Based on w= hat I > could gather from the pcie source files I've made a stab at creating= a > simple model. I've also attached a file for a simple pci device that= works > under regular Qemu. I would like to duplicate its functionality in y= our pcie > environment for starters. > > Could you please take a look at the files I've created and tell me i= f I've > understood your pcie model correctly. Any help will be truly appreci= ated. > > Adnan =20 Hi Adnan, =20 There is a fairly simple device I've created called "ivshmem" that is in the qemu git tree. It is a regular PCI device that exports a shared memory object via a BAR and supports a few registers and optional MSI-X interrupts (I had to pick through the virtio code to get MSI-X working, so looking at ivshmem might save you some effort). My device is somewhat similar to a graphics card actually which I recall is your goal. The purpose of ivshmem is to support sharing memory between multiple guests running on the same host. It follows the qdev model which you will need to do. =20 Cam =20 > > The five files I've modified from your git repository are as follows > > hw/pci=5Fids.h // Added vendor id defines > hw/pc=5Fq35.c // Device instantiation > hw/pcie=5Fmsix=5Ftemplate.h // Device header file > hw/pcie=5Fmsix=5Ftemplate.c // Device file > Makefile.objs // Added pcie=5Fmsix=5Ftemplate.o to= list of > objects being built > > Everything should compile without any warnings or errors. > > The last file: > sc=5Flink=5Fpci.c > Is the original PCI device that I'm trying to convert into being PCI= e and > MSI-X and is included merely for reference to help you understand wh= at I'd > like to achieve in your environment. > > > =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F > From: Isaku Yamahata [mailto:yamahata@valinux.co.jp] > To: Adnan Khaleel [mailto:adnan@khaleel.us] > Cc: qemu-devel@nongnu.org > Sent: Wed, 18 Aug 2010 22:19:04 -0500 > Subject: Re: [Qemu-devel] Template for developing a Qemu device with= PCIe > and MSI-X > > On Wed, Aug 18, 2010 at 02:10:10PM -0500, Adnan Khaleel wrote: >> Hello Qemu developers, >> >> I'm interested in developing a device model that plugs into Qemu th= at is >> based >> on a PCIe interface and uses MSI-X. My goal is to ultimately attach= a GPU >> simulator to this PCIe interface and use the entire platfom (Qemu += GPU >> simulator) for studying cpu, gpu interactions. >> >> I'm not terribly familiar with the Qemu device model and I'm lookin= g for >> some >> assistance, perhaps a starting template for pcie and msi-x that wou= ld >> offer the >> basic functionality that I could then build upon. >> >> I have looked at the various devices that already modelled that are >> included >> with Qemu (v0.12.5 at least) and I've noticed several a few pci dev= ices, >> eg; >> ne2k and cirrus-pci etc, however only one device truly seems to uti= lize >> both >> the technologies that I'm interested in and that is the virtio-pci.= c >> >> I'm not sure what virtio-pci does so I'm not sure if that is a suit= able >> starting point for me. >> >> Any help, suggestions etc would be extremely helpful and much appre= ciated. > > Qemu doesn't support pcie at the moment. > Only partial patches have been merged, still more patches have to > be merged for pcie to fully work. The following repo is available. > > git clone http://people.valinux.co.jp/~yamahata/qemu/q35/qemu > git clone http://people.valinux.co.jp/~yamahata/qemu/q35/seabios > git clone http://people.valinux.co.jp/~yamahata/qemu/q35/vgabios > > Note: patched seabios and vgabios are needed, you have to pass ACPI = DSDT > for q35. > example: > qemu-system-x86=5F64 -M pc=5Fq35 -acpitable > load=5Fheader,data=3Droms/seabios/src/q35-acpi-dsdt.aml > > This repo is for those who want to try/develop pcie support, > not for upstream merge. So they include patches unsuitable for upstr= eam. > The repo includes pcie port switch emulator which utilize pcie and > MSI(not MSI-X). > > The difference between PCI device and PCIe device is configuration > space size. > By setting PCIDeviceInfo::is=5Fexpress =3D 1, you'll get 4K configur= ation > space. Helper functions for pcie are found in qemu/hw/pcie.c > For msi-x, see qemu/hw/msix.c. > > Thanks, > -- > yamahata > =20 -------------b0469c9ed3f58912faffa2921ef5b58d Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
Yamahata, Cam,

Thank you both ve= ry much for pointers about Qemu coding for PCIe and MSI-X.

I'm at= a point where I can see my device when I do an lspci -t -v as shown bel= ow.

linux-= an84:~ # lspci -t -v
-[0000:00]-+-00.0  Intel Corporation 8= 2G33/G31/P35/P31 Express DRAM Controller
    = ;       +-01.0  Cirrus Logic GD 5446<= /span>
           +-0= 4.0-[0000:20]--
        =    +-18.0-[0000:21]--
     &n= bsp;     +-18.1-[0000:22]--
  &nbs= p;        +-18.2-[0000:23]--=            +-18.3-[000= 0:24]--
         &n= bsp; +-18.4-[0000:25]--
      &nbs= p;    +-18.5-[0000:26]--
    =        +-19.0-[0000:36-bf]--+-00.0-[0000:3= 7-47]--+-00.0-[0000:38]--
      &n= bsp;    |        =             |&nbs= p;           &nbs= p;       +-01.0-[0000:39]--
 =           |  &nbs= p;           &nbs= p;     |       &n= bsp;            += -02.0-[0000:3a]--
       &nb= sp;   |         &= nbsp;          |  = ;            = ;      +-03.0-[0000:3b]--
  &= nbsp;        |    = ;            = ;    |        &nb= sp;           +-04.0-[= 0000:3c]--
         = ;  |          &nb= sp;         |   &= nbsp;           &= nbsp;    +-05.0-[0000:3d]--
   &nb= sp;       |     &= nbsp;           &= nbsp;  |          = ;          +-06.0-[0000:3e]= --
           = |            = ;        |    &nb= sp;           &nb= sp;   +-07.0-[0000:3f]--
     = ;      |      &nb= sp;           &nb= sp; |           &= nbsp;        +-08.0-[0000:40]----00.0=   Cray Inc Device 0301  <- The device that I'= ve included

However, I'm having a bit of an issue with the MSI-X.=

I'm following the code examples in virtio-pci.c and ivshmem.c th= at Cam pointed out to. I've got bar 0&1 already occupied so I assign= the msix=5Fmmio=5Fmap to bar 2. However, when I do that, Qemu fails to = boot and fails with the following assertion fail:

unused outb: port=3D0x00f1 data=3D= 0x00
qemu: fatal: Trying to execute cod= e=20 outside RAM or ROM at 0x0000000000100000

Couple of thins I= 'm unsure about:
1. Am I registering the 64bit bar addresses correctl= y=3F
pci=5Fregister=5Fbar(&= d->dev, i,=20 BAR=5FRegions[i][0], PCI=5FBASE=5FADDRESS=5FSPACE=5FIO |=20 PCI=5FBASE=5FADDRESS=5FMEM=5FTYPE=5F64, pcie=5Fmsix=5Fio=5Fmap);<= br>
2. In the function int msix=5Finit(PCIDevice *pdev, unsigned shor= t nentries, unsigned bar=5Fnr, unsigned bar=5Fsize);
I'm not sure wha= t bar=5Fnr is, however setting it to 1 (as in code examples above) or to= 2 (bar that I want to register the msix=5Fmmio=5Fmap to) both fail with= the same error.

rc =3D msi= x=5Finit(&d->dev,=20 d->vectors, 2, 0);
:
:
pci=5Fregister=5Fbar(&d->dev, 2,=20 msix=5Fbar=5Fsize(&d->dev), PCI=5FBASE=5FADDRESS=5FSPACE=5FMEMORY= | PCI=5FBASE=5FADDRESS=5FM= EM=5FTYPE=5F64, msix=5Fmmio= =5Fmap);


Here is my init function code in its entirety= :

static const unsigned lon= g long BAR=5FRegions[6][2] =3D
{
    // = len , type
    { 0x2000000000ull, PCI=5FBASE=5FA= DDRESS=5FSPACE=5FMEMORY} ,  //BAR0
    { 0x2= 000000ull,    PCI=5FBASE=5FADDRESS=5FSPACE=5FIO &nbs= p;  } ,  //BAR1
    { 0, 0} ,  //B= AR2
    { 0, 0} ,  //BAR3
 &nbs= p;  { 0, 0} ,  //BAR4
    { 0, 0} =    //BAR5
};

static int pcie=5Fmsix=5Fini= tfn(PCIDevice *pci=5Fdev)
{
    PCIE=5FMS= IX=5FDEVState *d =3D DO=5FUPCAST(PCIE=5FMSIX=5FDEVState, dev, pci=5Fdev)= ;
    PCIBridge *br =3D DO=5FUPCAST(PCIBridge, de= v, pci=5Fdev);
    PCIEPort *p =3D DO=5FUPCAST(PC= IEPort, br, br);
    int rc, i;

    PRINT=5FDEBUG("%s: PCIE = MSIX Device init...\n", =5F=5FFUNCTION=5F=5F);

    pci=5Fconfig=5Fset=5Fvendor=5Fid(d= ->dev.config, PCIE=5FMSIX=5FVID);
    pci=5Fco= nfig=5Fset=5Fdevice=5Fid(d->dev.config, PCIE=5FMSIX=5FDID);&= nbsp;   d->dev.config[PCI=5FREVISION=5FID] =3D PCIE=5FMSIX= =5FVERSION;
    d->dev.config[PCI=5FSUBSYSTEM= =5FVENDOR=5FID]   =3D PCIE=5FMSIX=5FVID & 0xff;
&n= bsp;   d->dev.config[PCI=5FSUBSYSTEM=5FVENDOR=5FID+1] =3D (= PCIE=5FMSIX=5FVID >> 8) & 0xff;
    d-&= gt;dev.config[PCI=5FSUBSYSTEM=5FID]   =3D PCIE=5FMSIX=5FSS=5FD= ID & 0xff;
    d->dev.config[PCI=5FSUBSYST= EM=5FID+1] =3D (PCIE=5FMSIX=5FSS=5FDID >> 8) & 0xff;
    d->mmio=5Findex= =3D cpu=5Fregister=5Fio=5Fmemory(pcie=5Fmsix=5Fmem=5Fread=5Ffn, pcie=5F= msix=5Fmem=5Fwrite=5Ffn, d);
<= span style=3D"font-family: Courier;">   
 &= nbsp;  for(i=3D0; i < PCI=5FNUM=5FREGIONS -1; i++) { //-1 for th= e Exp ROM BAR
        if(BAR= =5FRegions[i][0] !=3D 0)
      &nb= sp; {
         &nb= sp;  if(BAR=5FRegions[i][1] =3D=3D PCI=5FBASE=5FADDRESS=5FSPACE=5FI= O)
          =   {
         &= nbsp;      //io region
  &nbs= p;           &nbs= p; PRINT=5FDEBUG("%s: Registering Bar %i as I/O BAR\n", =5F=5FFUNCTION= =5F=5F, i);
        &nbs= p;       pci=5Fregister=5Fbar(&d->d= ev, i, BAR=5FRegions[i][0], PCI=5FBASE=5FADDRESS=5FSPACE=5FIO | PCI=5FBA= SE=5FADDRESS=5FMEM=5FTYPE=5F64, pcie=5Fmsix=5Fio=5Fmap);
 &= nbsp;          } else {
           =      //mem region
    &n= bsp;           PRINT= =5FDEBUG("%s: Registering Bar %i as MEM BAR\n", =5F=5FFUNCTION=5F=5F, i)= ;
          &n= bsp;     pci=5Fregister=5Fbar(&d->dev, i, BAR= =5FRegions[i][0], PCI=5FBASE=5FADDRESS=5FSPACE=5FMEMORY | PCI=5FBASE=5FA= DDRESS=5FMEM=5FTYPE=5F64, pcie=5Fmsix=5Fmem=5Fmap);
  =           }
 &= nbsp;      }
    }
    d->dev.config[PCI=5FINTERRUPT=5FPIN] =3D 1;

    rc =3D msix= =5Finit(&d->dev, d->vectors, 2, 0);
   =
    if (!rc) {
    =     PRINT=5FDEBUG("%s: Registering Bar %i as I/O BAR\n", = =5F=5FFUNCTION=5F=5F, i);
      &n= bsp; pci=5Fregister=5Fbar(&d->dev, 2, msix=5Fbar=5Fsize(&d-&g= t;dev), PCI=5FBASE=5FADDRESS=5FSPACE=5FMEMORY | PCI=5FBASE=5FADDRESS=5FMEM=5FTYPE=5F64, msix=5Fmmio=5Fmap);
 &nbs= p;      PRINT=5FDEBUG("%s: MSI-X initialized (%= d vectors)\n", =5F=5FFUNCTION=5F=5F, d->vectors);
  = ;  }
    else {
   &n= bsp;    PRINT=5FDEBUG("%s: MSI-X initialization failed!\n= ", =5F=5FFUNCTION=5F=5F);
      &n= bsp; exit(1);
    }
   
    // Activate the vectors
  &n= bsp; for (i =3D 0; i < d->vectors; i++) {
  &nbs= p;     msix=5Fvector=5Fuse(&d->dev, i);
    }

&nbs= p;   rc =3D pci=5Fpcie=5Fcap=5Finit(&d->dev, PCIE=5FMSI= X=5FEXP=5FOFFSET, PCI=5FEXP=5FTYPE=5FENDPOINT, p->port);
&nbs= p;   if (rc < 0) {
     &n= bsp;  return rc;
    }

    pcie=5Fcap=5Fflr=5Finit(&d= ->dev, &pcie=5Fmsix=5Fflr);
    pcie=5Fcap= =5Fdeverr=5Finit(&d->dev);
    pcie=5Fcap= =5Fari=5Finit(&d->dev);
    rc =3D pcie=5F= aer=5Finit(&d->dev, PCIE=5FMSIX=5FAER=5FOFFSET);
 &= nbsp;  if (rc < 0) {
<= span style=3D"font-family: Courier;">      = ;  return rc;
    }

    PRINT=5FDEBUG("%s: Init done\n", = =5F=5FFUNCTION=5F=5F);
    return 0;
}


Thanks

AK



From: Cam Macdonell [mailto:cam@cs.ualbert= a.ca]
To: adnan@khaleel.us
Cc: Isaku Yamahata [mailt= o:yamahata@valinux.co.jp], qemu-devel@nongnu.org
Sent: Fri, 27= Aug 2010 10:48:48 -0500
Subject: Re: [Qemu-devel] Template fo= r developing a Qemu device with PCIe and MSI-X

On Wed, Aug 25, 20= 10 at 4:39 PM, Adnan Khaleel <adn= an@khaleel.us> wrote:
> Hi Isaku,
>
> I've made some progress in coding the device template but its no wh= ere near
> complete.
>
> I've created some files and am attaching it to this note. Based on = what I
> could gather from the pcie source files I've made a stab at creatin= g a
> simple model. I've also attached a file for a simple pci device tha= t works
> under regular Qemu. I would like to duplicate its functionality in = your pcie
> environment for starters.
>
> Could you please take a look at the files I've created and tell me = if I've
> understood your pcie model correctly. Any help will be truly apprec= iated.
>
> Adnan

Hi Adnan,

There is a fairly simple device I've created called "ivshmem" that is in the qemu git tree. It is a regular PCI device that exports a
shared memory object via a BAR and supports a few registers and
optional MSI-X interrupts (I had to pick through the virtio code to
get MSI-X working, so looking at ivshmem might save you some effort). My device is somewhat similar to a graphics card actually which I
recall is your goal. The purpose of ivshmem is to support sharing
memory between multiple guests running on the same host. It follows
the qdev model which you will need to do.

Cam

>
> The five files I've modified from your git repository are as follow= s
>
> hw/pci=5Fids.h         = ;           // Added v= endor id defines
> hw/pc=5Fq35.c         =            // Device i= nstantiation
> hw/pcie=5Fmsix=5Ftemplate.h  // Device header file
> hw/pcie=5Fmsix=5Ftemplate.c  // Device file
> Makefile.objs         =           // Added pcie=5Fm= six=5Ftemplate.o to list of
> objects being built
>
> Everything should compile without any warnings or errors.
>
> The last file:
> sc=5Flink=5Fpci.c
> Is the original PCI device that I'm trying to convert into being PC= Ie and
> MSI-X and is included merely for reference to help you understand w= hat I'd
> like to achieve in your environment.
>
>
> =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F= =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
> From: Isaku Yamahata [mailto:yamahata@valinux.co.jp]
> To: Adnan Khaleel [mailto:adnan= @khaleel.us]
> Cc: qemu-devel@nongnu.org<= /a>
> Sent: Wed, 18 Aug 2010 22:19:04 -0500
> Subject: Re: [Qemu-devel] Template for developing a Qemu device wit= h PCIe
> and MSI-X
>
> On Wed, Aug 18, 2010 at 02:10:10PM -0500, Adnan Khaleel wrote:
>> Hello Qemu developers,
>>
>> I'm interested in developing a device model that plugs into Qem= u that is
>> based
>> on a PCIe interface and uses MSI-X. My goal is to ultimately at= tach a GPU
>> simulator to this PCIe interface and use the entire platfom (Qe= mu + GPU
>> simulator) for studying cpu, gpu interactions.
>>
>> I'm not terribly familiar with the Qemu device model and I'm lo= oking for
>> some
>> assistance, perhaps a starting template for pcie and msi-x that= would
>> offer the
>> basic functionality that I could then build upon.
>>
>> I have looked at the various devices that already modelled that= are
>> included
>> with Qemu (v0.12.5 at least) and I've noticed several a few pci= devices,
>> eg;
>> ne2k and cirrus-pci etc, however only one device truly seems to= utilize
>> both
>> the technologies that I'm interested in and that is the virtio-= pci.c
>>
>> I'm not sure what virtio-pci does so I'm not sure if that is a = suitable
>> starting point for me.
>>
>> Any help, suggestions etc would be extremely helpful and much a= ppreciated.
>
> Qemu doesn't support pcie at the moment.
> Only partial patches have been merged, still more patches have to > be merged for pcie to fully work. The following repo is available.<= br> >
> git clone http://people.valinux.co.jp/~yamahata/qemu/= q35/qemu
> git clone http://people.valinux.co.jp/~yamahata/qe= mu/q35/seabios
> git clone http://people.valinux.co.jp/~yamahata/qe= mu/q35/vgabios
>
> Note: patched seabios and vgabios are needed, you have to pass ACPI= DSDT
> for q35.
> example:
> qemu-system-x86=5F64 -M pc=5Fq35 -acpitable
> load=5Fheader,data=3Droms/seabios/src/q35-acpi-dsdt.aml
>
> This repo is for those who want to try/develop pcie support,
> not for upstream merge. So they include patches unsuitable for upst= ream.
> The repo includes pcie port switch emulator which utilize pcie and<= br> > MSI(not MSI-X).
>
> The difference between PCI device and PCIe device is configuration<= br> > space size.
> By setting PCIDeviceInfo::is=5Fexpress =3D 1, you'll get 4K configu= ration
> space. Helper functions for pcie are found in qemu/hw/pcie.c
> For msi-x, see qemu/hw/msix.c.
>
> Thanks,
> --
> yamahata
>
-------------b0469c9ed3f58912faffa2921ef5b58d--