All of lore.kernel.org
 help / color / mirror / Atom feed
* q35 in xen?  vfio in xen?
@ 2014-02-21 21:41 Zhang, Eniac
  2014-02-21 21:50 ` Konrad Rzeszutek Wilk
  2014-02-21 22:09 ` Fabio Fantoni
  0 siblings, 2 replies; 11+ messages in thread
From: Zhang, Eniac @ 2014-02-21 21:41 UTC (permalink / raw)
  To: xen-devel; +Cc: Zhang, Eniac


[-- Attachment #1.1: Type: text/plain, Size: 880 bytes --]

Hi all,

I am playing with q35 chipset in qemu (1.6.1).  It seems we can't enable q35 machine under xen yet.  I made a few quick hacks which all fail miserably (linux kernel oops and window BSOD).  I was wondering why this hasn't been done (q35 was introduced into qemu in 2009).

Next question, vfio works very well for me in standalone qemu (with Linux host handling iommu), but is that supported under xen?  I haven't tried anything there yet because my gut-feeling is that it won't work.  Because passing vfio device to qemu can only be done on qemu commandline, and xen is not aware of this passing through device, thus not able to make iommu arrangement for this device.  Am I on the right track here?

I am interested in implementing both these two features.  I'd like to connect with anyone who's already on this so we don't duplicate the efforts.

Regards/Eniac

[-- Attachment #1.2: Type: text/html, Size: 3068 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread
* Re: q35 in xen?  vfio in xen?
  2014-02-21 21:41 q35 in xen? vfio in xen? Zhang, Eniac
@ 2014-02-21 21:50 ` Konrad Rzeszutek Wilk
  2014-02-21 21:58   ` Zhang, Eniac
  2014-02-21 22:09 ` Fabio Fantoni
  1 sibling, 1 reply; 11+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-02-21 21:50 UTC (permalink / raw)
  To: Zhang, Eniac; +Cc: xen-devel

On Fri, Feb 21, 2014 at 09:41:39PM +0000, Zhang, Eniac wrote:
> Hi all,
> 
> I am playing with q35 chipset in qemu (1.6.1).  It seems we can't enable q35 machine under xen yet.  I made a few quick hacks which all fail miserably (linux kernel oops and window BSOD).  I was wondering why this hasn't been done (q35 was introduced into qemu in 2009).
> 
> Next question, vfio works very well for me in standalone qemu (with Linux host handling iommu), but is that supported under xen?  I haven't tried anything there yet because my gut-feeling is that it won't work.  Because passing vfio device to qemu can only be done on qemu commandline, and xen is not aware of this passing through device, thus not able to make iommu arrangement for this device.  Am I on the right track here?

Yes and no. VFIO won't work - but QEMU does do PCI passthrough under Xen. It uses
a different mechanism (and you need to bind the device to pciback).

> 
> I am interested in implementing both these two features.  I'd like to connect with anyone who's already on this so we don't duplicate the efforts.

What do you need Q35 for?

> 
> Regards/Eniac

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread
* Re: q35 in xen?  vfio in xen?
  2014-02-21 21:50 ` Konrad Rzeszutek Wilk
@ 2014-02-21 21:58   ` Zhang, Eniac
  0 siblings, 0 replies; 11+ messages in thread
From: Zhang, Eniac @ 2014-02-21 21:58 UTC (permalink / raw)
  To: Konrad Rzeszutek Wilk; +Cc: xen-devel

Hi Konrad,

Thanks for your reply.  

Yes, I am aware of the pciback.  Unfortunately it doesn't seem to support pci-e passthrough. (I could be wrong here)

There are two reason that I am interested in this.  For one, my project calls for pci-e device passthrough, which can't be accomplished with 440fx chipset emulation.  Secondly, I feel we ought to move on with the technology.  440fx is ancient in computer terms.  Qemu is good and all that, but if it refuses to support pci-e natively then it's just a matter of time that it will become obsoleted.  The trend is clear that pci-e is taking over the world.

Regards/Eniac

-----Original Message-----
From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] 
Sent: Friday, February 21, 2014 2:50 PM
To: Zhang, Eniac
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] q35 in xen? vfio in xen?

On Fri, Feb 21, 2014 at 09:41:39PM +0000, Zhang, Eniac wrote:
> Hi all,
> 
> I am playing with q35 chipset in qemu (1.6.1).  It seems we can't enable q35 machine under xen yet.  I made a few quick hacks which all fail miserably (linux kernel oops and window BSOD).  I was wondering why this hasn't been done (q35 was introduced into qemu in 2009).
> 
> Next question, vfio works very well for me in standalone qemu (with Linux host handling iommu), but is that supported under xen?  I haven't tried anything there yet because my gut-feeling is that it won't work.  Because passing vfio device to qemu can only be done on qemu commandline, and xen is not aware of this passing through device, thus not able to make iommu arrangement for this device.  Am I on the right track here?

Yes and no. VFIO won't work - but QEMU does do PCI passthrough under Xen. It uses a different mechanism (and you need to bind the device to pciback).

> 
> I am interested in implementing both these two features.  I'd like to connect with anyone who's already on this so we don't duplicate the efforts.

What do you need Q35 for?

> 
> Regards/Eniac

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread
* Re: q35 in xen? vfio in xen?
  2014-02-21 21:41 q35 in xen? vfio in xen? Zhang, Eniac
  2014-02-21 21:50 ` Konrad Rzeszutek Wilk
@ 2014-02-21 22:09 ` Fabio Fantoni
  1 sibling, 0 replies; 11+ messages in thread
From: Fabio Fantoni @ 2014-02-21 22:09 UTC (permalink / raw)
  To: Zhang, Eniac; +Cc: xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 1522 bytes --]

2014-02-21 22:41 GMT+01:00 Zhang, Eniac <eniac-xw.zhang@hp.com>:

>  Hi all,
>
>
>
> I am playing with q35 chipset in qemu (1.6.1).  It seems we can't enable
> q35 machine under xen yet.  I made a few quick hacks which all fail
> miserably (linux kernel oops and window BSOD).  I was wondering why this
> hasn't been done (q35 was introduced into qemu in 2009).
>
>
>
> Next question, vfio works very well for me in standalone qemu (with Linux
> host handling iommu), but is that supported under xen?  I haven't tried
> anything there yet because my gut-feeling is that it won't work.  Because
> passing vfio device to qemu can only be done on qemu commandline, and xen
> is not aware of this passing through device, thus not able to make iommu
> arrangement for this device.  Am I on the right track here?
>
>
>
> I am interested in implementing both these two features.  I'd like to
> connect with anyone who's already on this so we don't duplicate the efforts.
>

I di some fast test with q35 time ago, one problem found is disks not
working with old qemu parameter, with new parameters (-device) see them on
boot but trying to do a patch with new qemu parameters compatible also with
old cipset with ide I found that automatic bus selection is bugged,
unfortunately, after I had more time to continue nor the knowledge to
make needed
changes in hvmloader.


>
>
> Regards/Eniac
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>
>

[-- Attachment #1.2: Type: text/html, Size: 3074 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread
* Re: q35 in xen?  vfio in xen?
  2014-02-26 11:58         ` Stefano Stabellini
@ 2014-02-26 13:25           ` Fabio Fantoni
  0 siblings, 0 replies; 11+ messages in thread
From: Fabio Fantoni @ 2014-02-26 13:25 UTC (permalink / raw)
  To: Stefano Stabellini, Konrad Rzeszutek Wilk
  Cc: anthony.perard, Zhang, Eniac, xen-devel


[-- Attachment #1.1: Type: text/plain, Size: 8965 bytes --]

Il 26/02/2014 12:58, Stefano Stabellini ha scritto:
> A little while ago Anthony made the Q35 emulation in QEMU work with Xen:
>
> http://marc.info/?l=qemu-devel&m=137813513713296
>
> He might be able to give you some pointers on where to start.

FWIK that was only a patch which renders working back again ram 
inizialisation of qemu 1.6 with xen.

I commented out an assert in hvmloader which prevent domUs start with 
q35 chipset and that added some qemu parameters manually, for example:
device_model_args_hvm=["-machine","q35,accel=xen","-device","i82801b11-bridge","-drive","file=/mnt/vm/disks/W7-q35.disk1.xm,if=none,id=drive-sata-disk0,format=raw,cache=writeback","-device","ide-hd,drive=drive-sata-disk0,id=sata0"]

Actually I don't find my previous mail with details about my q35 tests 
but I believe that the main things to start testing were the one 
mentioned above.
-machine q35,accel=xen to set q35 chipset
-device i82801b11-bridge to made available old pci bus since probably 
that default pci-e need hvmloader changes to be working.
I also found that with actual qemu parameters the disks were not visible 
on boot, instead were visible using new qemu parameters (-device).
I tried to do a patch with new qemu parameters compatible also with old 
chipset with ide controller but I found that automatic bus selection is 
bugged, unfortunately, I did not have more time to continue nor the 
knowledge to make needed changes in hvmloader to make q35 fully working.

>
>
> On Mon, 24 Feb 2014, Konrad Rzeszutek Wilk wrote:
>> On Mon, Feb 24, 2014 at 07:28:32PM +0000, Zhang, Eniac wrote:
>>> Hi Konrad,
>>>
>>> Thanks for the info.  Your guest sees the virtual function as  pci device just like I had suspected.  Unfortunately that won't work for me.  I guess I have to take a hard look at implement pci-e passthrough using pciback then.
>> You won't have to do it with pciback. Keep in mind that pciback just "holds"
>> the device so that other drivers (like ixbgvf) don't use it.
>>
>> 'xl' ends up doing the proper hypercall to assign the device to
>> the guest. And QEMU also does it set of calls to setup the
>> BARS, interrupts, deal with MSI-X, etc.
>>
>> What you are going to have to look at is QEMU - and how to make it
>> work with the newer emulated chipset.
>>
>> Stefano (CC-ed) here is the maintainer of QEMU Xen pieces. Anthony
>> (CC-ed as well) had backported the proper pieces in QEMU to do
>> PCI passthrough.
>>
>> Looking forward to your patches and we will be more than happy
>> to help you upstream them!
>>
>>> Regards/Eniac
>>>
>>> -----Original Message-----
>>> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
>>> Sent: Monday, February 24, 2014 9:40 AM
>>> To: Zhang, Eniac
>>> Cc: xen-devel@lists.xen.org
>>> Subject: Re: [Xen-devel] q35 in xen? vfio in xen?
>>>
>>> On Sat, Feb 22, 2014 at 08:06:50AM +0000, Zhang, Eniac wrote:
>>>> Hi Konrad,
>>>>
>>>> Here's what I see when start a VM under xen using pciback to pass a pci-e device into domU.  The device can be seen by guest, and also functioning fine.  But it's not seen as a pci-e device, rather, it looks just like an ordinary pci device because only the first 0x100 bytes of its configuration space is accessible.  So if a driver needs to use data in the extended configuration space for certain features, it will fail.
>>>>
>>>> When you say you "did PCIe pass through of an VF of an SR-IOV device".  Are you actually using it as a pci-e device or have throttled it back to pci mode without aware of the difference?  If you did see the pci-e device in guest, can you share your xl.cfg file as well as lspci/lspci -t/lspci -xxxx output from guest?
>>> # lspci
>>> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
>>> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
>>> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
>>> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
>>> 00:02.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
>>> 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
>>> 00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) # lspci -t
>>> -[0000:00]-+-00.0
>>>             +-01.0
>>>             +-01.1
>>>             +-01.3
>>>             +-02.0
>>>             +-03.0
>>>             \-04.0
>>> # lspci -s 00:04.0 -xxxxx
>>> 00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
>>> 00: 86 80 ca 10 06 04 10 00 01 00 00 02 00 00 00 00
>>> 10: 04 00 01 f3 00 00 00 00 00 00 00 00 04 40 01 f3
>>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 3c a0
>>> 30: 00 00 00 00 70 00 00 00 00 00 00 00 00 00 00 00
>>> 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> 70: 11 a0 02 80 03 00 00 00 03 20 00 00 00 00 00 00
>>> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> a0: 10 00 02 00 c2 8c 00 00 10 28 00 00 41 6c 03 00
>>> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> c0: 00 00 00 00 1f 00 00 00 00 00 00 00 00 00 00 00
>>> d0: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>
>>> -bash-4.1# more /vm-pci.cfg
>>> builder='hvm'
>>> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
>>> memory = 2048
>>> boot="d"
>>> maxvcpus=32
>>> vcpus=1
>>> serial='pty'
>>> vnclisten="0.0.0.0"
>>> name="latest"
>>> pci = ["0000:02:10.0"]
>>>
>>>> Also to echo your second comment:  I might still be a newbie in qemu field (I started working on this 4 months ago).  I thought the chipset limits what you can see/do in vm.  Ie.  If you have 440fx emulations then you can't have any pci-e devices (fake or passthru) in the same system.  Is that not true?
>>>>
>>>> Regards/Eniac
>>>>
>>>> -----Original Message-----
>>>> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
>>>> Sent: Friday, February 21, 2014 5:32 PM
>>>> To: Zhang, Eniac
>>>> Cc: xen-devel@lists.xen.org
>>>> Subject: RE: [Xen-devel] q35 in xen? vfio in xen?
>>>>
>>>>
>>>> On Feb 21, 2014 4:58 PM, "Zhang, Eniac" <eniac-xw.zhang@hp.com> wrote:
>>>>> Hi Konrad,
>>>>>
>>>>> Thanks for your reply.
>>>>>
>>>>> Yes, I am aware of the pciback.  Unfortunately it doesn't seem to
>>>>> support pci-e passthrough. (I could be wrong here)
>>>> I just did PCIe pass through of an VF of an SR-IOV device. It certainly is PCIe.
>>>>
>>>>> There are two reason that I am interested in this.  For one, my project calls for pci-e device passthrough, which can't be accomplished with 440fx chipset emulation.  Secondly, I feel we ought to move on with the technology.  440fx is ancient in computer terms.  Qemu is good and all that, but if it refuses to support pci-e natively then it's just a matter of time that it will become obsoleted.  The trend is clear that pci-e is taking over the world.
>>>>>
>>>> I am not sure what you are saying but it does not matter whether QEMU emulates 440fx or q35 for PCI pass through .
>>>>
>>>>> Regards/Eniac
>>>>>
>>>>> -----Original Message-----
>>>>> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
>>>>> Sent: Friday, February 21, 2014 2:50 PM
>>>>> To: Zhang, Eniac
>>>>> Cc: xen-devel@lists.xen.org
>>>>> Subject: Re: [Xen-devel] q35 in xen? vfio in xen?
>>>>>
>>>>> On Fri, Feb 21, 2014 at 09:41:39PM +0000, Zhang, Eniac wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I am playing with q35 chipset in qemu (1.6.1).  It seems we can't enable q35 machine under xen yet.  I made a few quick hacks which all fail miserably (linux kernel oops and window BSOD).  I was wondering why this hasn't been done (q35 was introduced into qemu in 2009).
>>>>>>
>>>>>> Next question, vfio works very well for me in standalone qemu (with Linux host handling iommu), but is that supported under xen?  I haven't tried anything there yet because my gut-feeling is that it won't work.  Because passing vfio device to qemu can only be done on qemu commandline, and xen is not aware of this passing through device, thus not able to make iommu arrangement for this device.  Am I on the right track here?
>>>>> Yes and no. VFIO won't work - but QEMU does do PCI passthrough under Xen. It uses a different mechanism (and you need to bind the device to pciback).
>>>>>
>>>>>> I am interested in implementing both these two features.  I'd like to connect with anyone who's already on this so we don't duplicate the efforts.
>>>>> What do you need Q35 for?
>>>>>
>>>>>> Regards/Eniac
>>>>>> _______________________________________________
>>>>>> Xen-devel mailing list
>>>>>> Xen-devel@lists.xen.org
>>>>>> http://lists.xen.org/xen-devel
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xen.org
>> http://lists.xen.org/xen-devel


[-- Attachment #1.2: Type: text/html, Size: 12002 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread
* Re: q35 in xen?  vfio in xen?
  2014-02-24 19:43       ` Konrad Rzeszutek Wilk
@ 2014-02-26 11:58         ` Stefano Stabellini
  2014-02-26 13:25           ` Fabio Fantoni
  0 siblings, 1 reply; 11+ messages in thread
From: Stefano Stabellini @ 2014-02-26 11:58 UTC (permalink / raw)
  To: Konrad Rzeszutek Wilk
  Cc: anthony.perard, xen-devel, Zhang, Eniac, Stefano Stabellini

[-- Attachment #1: Type: text/plain, Size: 7900 bytes --]

A little while ago Anthony made the Q35 emulation in QEMU work with Xen:

http://marc.info/?l=qemu-devel&m=137813513713296

He might be able to give you some pointers on where to start.


On Mon, 24 Feb 2014, Konrad Rzeszutek Wilk wrote:
> On Mon, Feb 24, 2014 at 07:28:32PM +0000, Zhang, Eniac wrote:
> > Hi Konrad,
> > 
> > Thanks for the info.  Your guest sees the virtual function as  pci device just like I had suspected.  Unfortunately that won't work for me.  I guess I have to take a hard look at implement pci-e passthrough using pciback then.
> 
> You won't have to do it with pciback. Keep in mind that pciback just "holds"
> the device so that other drivers (like ixbgvf) don't use it.
> 
> 'xl' ends up doing the proper hypercall to assign the device to
> the guest. And QEMU also does it set of calls to setup the
> BARS, interrupts, deal with MSI-X, etc.
> 
> What you are going to have to look at is QEMU - and how to make it
> work with the newer emulated chipset.
> 
> Stefano (CC-ed) here is the maintainer of QEMU Xen pieces. Anthony
> (CC-ed as well) had backported the proper pieces in QEMU to do
> PCI passthrough.
> 
> Looking forward to your patches and we will be more than happy
> to help you upstream them!
> 
> > 
> > Regards/Eniac
> > 
> > -----Original Message-----
> > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] 
> > Sent: Monday, February 24, 2014 9:40 AM
> > To: Zhang, Eniac
> > Cc: xen-devel@lists.xen.org
> > Subject: Re: [Xen-devel] q35 in xen? vfio in xen?
> > 
> > On Sat, Feb 22, 2014 at 08:06:50AM +0000, Zhang, Eniac wrote:
> > > Hi Konrad,
> > > 
> > > Here's what I see when start a VM under xen using pciback to pass a pci-e device into domU.  The device can be seen by guest, and also functioning fine.  But it's not seen as a pci-e device, rather, it looks just like an ordinary pci device because only the first 0x100 bytes of its configuration space is accessible.  So if a driver needs to use data in the extended configuration space for certain features, it will fail.
> > > 
> > > When you say you "did PCIe pass through of an VF of an SR-IOV device".  Are you actually using it as a pci-e device or have throttled it back to pci mode without aware of the difference?  If you did see the pci-e device in guest, can you share your xl.cfg file as well as lspci/lspci -t/lspci -xxxx output from guest?
> > 
> > # lspci
> > 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> > 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> > 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> > 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> > 00:02.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
> > 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> > 00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) # lspci -t
> > -[0000:00]-+-00.0
> >            +-01.0
> >            +-01.1
> >            +-01.3
> >            +-02.0
> >            +-03.0
> >            \-04.0
> > # lspci -s 00:04.0 -xxxxx
> > 00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
> > 00: 86 80 ca 10 06 04 10 00 01 00 00 02 00 00 00 00
> > 10: 04 00 01 f3 00 00 00 00 00 00 00 00 04 40 01 f3
> > 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 3c a0
> > 30: 00 00 00 00 70 00 00 00 00 00 00 00 00 00 00 00
> > 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 70: 11 a0 02 80 03 00 00 00 03 20 00 00 00 00 00 00
> > 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > a0: 10 00 02 00 c2 8c 00 00 10 28 00 00 41 6c 03 00
> > b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > c0: 00 00 00 00 1f 00 00 00 00 00 00 00 00 00 00 00
> > d0: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > 
> > -bash-4.1# more /vm-pci.cfg
> > builder='hvm'
> > disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
> > memory = 2048
> > boot="d"
> > maxvcpus=32
> > vcpus=1
> > serial='pty'
> > vnclisten="0.0.0.0"
> > name="latest"
> > pci = ["0000:02:10.0"]
> > 
> > > 
> > > Also to echo your second comment:  I might still be a newbie in qemu field (I started working on this 4 months ago).  I thought the chipset limits what you can see/do in vm.  Ie.  If you have 440fx emulations then you can't have any pci-e devices (fake or passthru) in the same system.  Is that not true?
> > > 
> > > Regards/Eniac
> > > 
> > > -----Original Message-----
> > > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> > > Sent: Friday, February 21, 2014 5:32 PM
> > > To: Zhang, Eniac
> > > Cc: xen-devel@lists.xen.org
> > > Subject: RE: [Xen-devel] q35 in xen? vfio in xen?
> > > 
> > > 
> > > On Feb 21, 2014 4:58 PM, "Zhang, Eniac" <eniac-xw.zhang@hp.com> wrote:
> > > >
> > > > Hi Konrad,
> > > >
> > > > Thanks for your reply.
> > > >
> > > > Yes, I am aware of the pciback.  Unfortunately it doesn't seem to 
> > > > support pci-e passthrough. (I could be wrong here)
> > > 
> > > I just did PCIe pass through of an VF of an SR-IOV device. It certainly is PCIe.
> > > 
> > > >
> > > > There are two reason that I am interested in this.  For one, my project calls for pci-e device passthrough, which can't be accomplished with 440fx chipset emulation.  Secondly, I feel we ought to move on with the technology.  440fx is ancient in computer terms.  Qemu is good and all that, but if it refuses to support pci-e natively then it's just a matter of time that it will become obsoleted.  The trend is clear that pci-e is taking over the world. 
> > > >
> > > 
> > > I am not sure what you are saying but it does not matter whether QEMU emulates 440fx or q35 for PCI pass through .
> > > 
> > > > Regards/Eniac
> > > >
> > > > -----Original Message-----
> > > > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> > > > Sent: Friday, February 21, 2014 2:50 PM
> > > > To: Zhang, Eniac
> > > > Cc: xen-devel@lists.xen.org
> > > > Subject: Re: [Xen-devel] q35 in xen? vfio in xen? 
> > > >
> > > > On Fri, Feb 21, 2014 at 09:41:39PM +0000, Zhang, Eniac wrote: 
> > > > > Hi all,
> > > > > 
> > > > > I am playing with q35 chipset in qemu (1.6.1).  It seems we can't enable q35 machine under xen yet.  I made a few quick hacks which all fail miserably (linux kernel oops and window BSOD).  I was wondering why this hasn't been done (q35 was introduced into qemu in 2009). 
> > > > > 
> > > > > Next question, vfio works very well for me in standalone qemu (with Linux host handling iommu), but is that supported under xen?  I haven't tried anything there yet because my gut-feeling is that it won't work.  Because passing vfio device to qemu can only be done on qemu commandline, and xen is not aware of this passing through device, thus not able to make iommu arrangement for this device.  Am I on the right track here? 
> > > >
> > > > Yes and no. VFIO won't work - but QEMU does do PCI passthrough under Xen. It uses a different mechanism (and you need to bind the device to pciback). 
> > > >
> > > > > 
> > > > > I am interested in implementing both these two features.  I'd like to connect with anyone who's already on this so we don't duplicate the efforts. 
> > > >
> > > > What do you need Q35 for? 
> > > >
> > > > > 
> > > > > Regards/Eniac
> > > >
> > > > > _______________________________________________
> > > > > Xen-devel mailing list
> > > > > Xen-devel@lists.xen.org
> > > > > http://lists.xen.org/xen-devel
> > > >
> 

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread
* Re: q35 in xen?  vfio in xen?
  2014-02-24 19:28     ` Zhang, Eniac
@ 2014-02-24 19:43       ` Konrad Rzeszutek Wilk
  2014-02-26 11:58         ` Stefano Stabellini
  0 siblings, 1 reply; 11+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-02-24 19:43 UTC (permalink / raw)
  To: Zhang, Eniac, anthony.perard, Stefano Stabellini; +Cc: xen-devel

On Mon, Feb 24, 2014 at 07:28:32PM +0000, Zhang, Eniac wrote:
> Hi Konrad,
> 
> Thanks for the info.  Your guest sees the virtual function as  pci device just like I had suspected.  Unfortunately that won't work for me.  I guess I have to take a hard look at implement pci-e passthrough using pciback then.

You won't have to do it with pciback. Keep in mind that pciback just "holds"
the device so that other drivers (like ixbgvf) don't use it.

'xl' ends up doing the proper hypercall to assign the device to
the guest. And QEMU also does it set of calls to setup the
BARS, interrupts, deal with MSI-X, etc.

What you are going to have to look at is QEMU - and how to make it
work with the newer emulated chipset.

Stefano (CC-ed) here is the maintainer of QEMU Xen pieces. Anthony
(CC-ed as well) had backported the proper pieces in QEMU to do
PCI passthrough.

Looking forward to your patches and we will be more than happy
to help you upstream them!

> 
> Regards/Eniac
> 
> -----Original Message-----
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] 
> Sent: Monday, February 24, 2014 9:40 AM
> To: Zhang, Eniac
> Cc: xen-devel@lists.xen.org
> Subject: Re: [Xen-devel] q35 in xen? vfio in xen?
> 
> On Sat, Feb 22, 2014 at 08:06:50AM +0000, Zhang, Eniac wrote:
> > Hi Konrad,
> > 
> > Here's what I see when start a VM under xen using pciback to pass a pci-e device into domU.  The device can be seen by guest, and also functioning fine.  But it's not seen as a pci-e device, rather, it looks just like an ordinary pci device because only the first 0x100 bytes of its configuration space is accessible.  So if a driver needs to use data in the extended configuration space for certain features, it will fail.
> > 
> > When you say you "did PCIe pass through of an VF of an SR-IOV device".  Are you actually using it as a pci-e device or have throttled it back to pci mode without aware of the difference?  If you did see the pci-e device in guest, can you share your xl.cfg file as well as lspci/lspci -t/lspci -xxxx output from guest?
> 
> # lspci
> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> 00:02.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
> 00:03.0 VGA compatible controller: Cirrus Logic GD 5446
> 00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) # lspci -t
> -[0000:00]-+-00.0
>            +-01.0
>            +-01.1
>            +-01.3
>            +-02.0
>            +-03.0
>            \-04.0
> # lspci -s 00:04.0 -xxxxx
> 00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
> 00: 86 80 ca 10 06 04 10 00 01 00 00 02 00 00 00 00
> 10: 04 00 01 f3 00 00 00 00 00 00 00 00 04 40 01 f3
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 3c a0
> 30: 00 00 00 00 70 00 00 00 00 00 00 00 00 00 00 00
> 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 70: 11 a0 02 80 03 00 00 00 03 20 00 00 00 00 00 00
> 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> a0: 10 00 02 00 c2 8c 00 00 10 28 00 00 41 6c 03 00
> b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> c0: 00 00 00 00 1f 00 00 00 00 00 00 00 00 00 00 00
> d0: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 
> -bash-4.1# more /vm-pci.cfg
> builder='hvm'
> disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
> memory = 2048
> boot="d"
> maxvcpus=32
> vcpus=1
> serial='pty'
> vnclisten="0.0.0.0"
> name="latest"
> pci = ["0000:02:10.0"]
> 
> > 
> > Also to echo your second comment:  I might still be a newbie in qemu field (I started working on this 4 months ago).  I thought the chipset limits what you can see/do in vm.  Ie.  If you have 440fx emulations then you can't have any pci-e devices (fake or passthru) in the same system.  Is that not true?
> > 
> > Regards/Eniac
> > 
> > -----Original Message-----
> > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> > Sent: Friday, February 21, 2014 5:32 PM
> > To: Zhang, Eniac
> > Cc: xen-devel@lists.xen.org
> > Subject: RE: [Xen-devel] q35 in xen? vfio in xen?
> > 
> > 
> > On Feb 21, 2014 4:58 PM, "Zhang, Eniac" <eniac-xw.zhang@hp.com> wrote:
> > >
> > > Hi Konrad,
> > >
> > > Thanks for your reply.
> > >
> > > Yes, I am aware of the pciback.  Unfortunately it doesn't seem to 
> > > support pci-e passthrough. (I could be wrong here)
> > 
> > I just did PCIe pass through of an VF of an SR-IOV device. It certainly is PCIe.
> > 
> > >
> > > There are two reason that I am interested in this.  For one, my project calls for pci-e device passthrough, which can't be accomplished with 440fx chipset emulation.  Secondly, I feel we ought to move on with the technology.  440fx is ancient in computer terms.  Qemu is good and all that, but if it refuses to support pci-e natively then it's just a matter of time that it will become obsoleted.  The trend is clear that pci-e is taking over the world. 
> > >
> > 
> > I am not sure what you are saying but it does not matter whether QEMU emulates 440fx or q35 for PCI pass through .
> > 
> > > Regards/Eniac
> > >
> > > -----Original Message-----
> > > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> > > Sent: Friday, February 21, 2014 2:50 PM
> > > To: Zhang, Eniac
> > > Cc: xen-devel@lists.xen.org
> > > Subject: Re: [Xen-devel] q35 in xen? vfio in xen? 
> > >
> > > On Fri, Feb 21, 2014 at 09:41:39PM +0000, Zhang, Eniac wrote: 
> > > > Hi all,
> > > > 
> > > > I am playing with q35 chipset in qemu (1.6.1).  It seems we can't enable q35 machine under xen yet.  I made a few quick hacks which all fail miserably (linux kernel oops and window BSOD).  I was wondering why this hasn't been done (q35 was introduced into qemu in 2009). 
> > > > 
> > > > Next question, vfio works very well for me in standalone qemu (with Linux host handling iommu), but is that supported under xen?  I haven't tried anything there yet because my gut-feeling is that it won't work.  Because passing vfio device to qemu can only be done on qemu commandline, and xen is not aware of this passing through device, thus not able to make iommu arrangement for this device.  Am I on the right track here? 
> > >
> > > Yes and no. VFIO won't work - but QEMU does do PCI passthrough under Xen. It uses a different mechanism (and you need to bind the device to pciback). 
> > >
> > > > 
> > > > I am interested in implementing both these two features.  I'd like to connect with anyone who's already on this so we don't duplicate the efforts. 
> > >
> > > What do you need Q35 for? 
> > >
> > > > 
> > > > Regards/Eniac
> > >
> > > > _______________________________________________
> > > > Xen-devel mailing list
> > > > Xen-devel@lists.xen.org
> > > > http://lists.xen.org/xen-devel
> > >

^ permalink raw reply	[flat|nested] 11+ messages in thread
* Re: q35 in xen?  vfio in xen?
  2014-02-24 16:40   ` Konrad Rzeszutek Wilk
@ 2014-02-24 19:28     ` Zhang, Eniac
  2014-02-24 19:43       ` Konrad Rzeszutek Wilk
  0 siblings, 1 reply; 11+ messages in thread
From: Zhang, Eniac @ 2014-02-24 19:28 UTC (permalink / raw)
  To: Konrad Rzeszutek Wilk; +Cc: xen-devel

Hi Konrad,

Thanks for the info.  Your guest sees the virtual function as  pci device just like I had suspected.  Unfortunately that won't work for me.  I guess I have to take a hard look at implement pci-e passthrough using pciback then.

Regards/Eniac

-----Original Message-----
From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] 
Sent: Monday, February 24, 2014 9:40 AM
To: Zhang, Eniac
Cc: xen-devel@lists.xen.org
Subject: Re: [Xen-devel] q35 in xen? vfio in xen?

On Sat, Feb 22, 2014 at 08:06:50AM +0000, Zhang, Eniac wrote:
> Hi Konrad,
> 
> Here's what I see when start a VM under xen using pciback to pass a pci-e device into domU.  The device can be seen by guest, and also functioning fine.  But it's not seen as a pci-e device, rather, it looks just like an ordinary pci device because only the first 0x100 bytes of its configuration space is accessible.  So if a driver needs to use data in the extended configuration space for certain features, it will fail.
> 
> When you say you "did PCIe pass through of an VF of an SR-IOV device".  Are you actually using it as a pci-e device or have throttled it back to pci mode without aware of the difference?  If you did see the pci-e device in guest, can you share your xl.cfg file as well as lspci/lspci -t/lspci -xxxx output from guest?

# lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
00:03.0 VGA compatible controller: Cirrus Logic GD 5446
00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01) # lspci -t
-[0000:00]-+-00.0
           +-01.0
           +-01.1
           +-01.3
           +-02.0
           +-03.0
           \-04.0
# lspci -s 00:04.0 -xxxxx
00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
00: 86 80 ca 10 06 04 10 00 01 00 00 02 00 00 00 00
10: 04 00 01 f3 00 00 00 00 00 00 00 00 04 40 01 f3
20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 3c a0
30: 00 00 00 00 70 00 00 00 00 00 00 00 00 00 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 11 a0 02 80 03 00 00 00 03 20 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 10 00 02 00 c2 8c 00 00 10 28 00 00 41 6c 03 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 1f 00 00 00 00 00 00 00 00 00 00 00
d0: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

-bash-4.1# more /vm-pci.cfg
builder='hvm'
disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
memory = 2048
boot="d"
maxvcpus=32
vcpus=1
serial='pty'
vnclisten="0.0.0.0"
name="latest"
pci = ["0000:02:10.0"]

> 
> Also to echo your second comment:  I might still be a newbie in qemu field (I started working on this 4 months ago).  I thought the chipset limits what you can see/do in vm.  Ie.  If you have 440fx emulations then you can't have any pci-e devices (fake or passthru) in the same system.  Is that not true?
> 
> Regards/Eniac
> 
> -----Original Message-----
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> Sent: Friday, February 21, 2014 5:32 PM
> To: Zhang, Eniac
> Cc: xen-devel@lists.xen.org
> Subject: RE: [Xen-devel] q35 in xen? vfio in xen?
> 
> 
> On Feb 21, 2014 4:58 PM, "Zhang, Eniac" <eniac-xw.zhang@hp.com> wrote:
> >
> > Hi Konrad,
> >
> > Thanks for your reply.
> >
> > Yes, I am aware of the pciback.  Unfortunately it doesn't seem to 
> > support pci-e passthrough. (I could be wrong here)
> 
> I just did PCIe pass through of an VF of an SR-IOV device. It certainly is PCIe.
> 
> >
> > There are two reason that I am interested in this.  For one, my project calls for pci-e device passthrough, which can't be accomplished with 440fx chipset emulation.  Secondly, I feel we ought to move on with the technology.  440fx is ancient in computer terms.  Qemu is good and all that, but if it refuses to support pci-e natively then it's just a matter of time that it will become obsoleted.  The trend is clear that pci-e is taking over the world. 
> >
> 
> I am not sure what you are saying but it does not matter whether QEMU emulates 440fx or q35 for PCI pass through .
> 
> > Regards/Eniac
> >
> > -----Original Message-----
> > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> > Sent: Friday, February 21, 2014 2:50 PM
> > To: Zhang, Eniac
> > Cc: xen-devel@lists.xen.org
> > Subject: Re: [Xen-devel] q35 in xen? vfio in xen? 
> >
> > On Fri, Feb 21, 2014 at 09:41:39PM +0000, Zhang, Eniac wrote: 
> > > Hi all,
> > > 
> > > I am playing with q35 chipset in qemu (1.6.1).  It seems we can't enable q35 machine under xen yet.  I made a few quick hacks which all fail miserably (linux kernel oops and window BSOD).  I was wondering why this hasn't been done (q35 was introduced into qemu in 2009). 
> > > 
> > > Next question, vfio works very well for me in standalone qemu (with Linux host handling iommu), but is that supported under xen?  I haven't tried anything there yet because my gut-feeling is that it won't work.  Because passing vfio device to qemu can only be done on qemu commandline, and xen is not aware of this passing through device, thus not able to make iommu arrangement for this device.  Am I on the right track here? 
> >
> > Yes and no. VFIO won't work - but QEMU does do PCI passthrough under Xen. It uses a different mechanism (and you need to bind the device to pciback). 
> >
> > > 
> > > I am interested in implementing both these two features.  I'd like to connect with anyone who's already on this so we don't duplicate the efforts. 
> >
> > What do you need Q35 for? 
> >
> > > 
> > > Regards/Eniac
> >
> > > _______________________________________________
> > > Xen-devel mailing list
> > > Xen-devel@lists.xen.org
> > > http://lists.xen.org/xen-devel
> >

^ permalink raw reply	[flat|nested] 11+ messages in thread
* Re: q35 in xen?  vfio in xen?
  2014-02-22  8:06 ` Zhang, Eniac
@ 2014-02-24 16:40   ` Konrad Rzeszutek Wilk
  2014-02-24 19:28     ` Zhang, Eniac
  0 siblings, 1 reply; 11+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-02-24 16:40 UTC (permalink / raw)
  To: Zhang, Eniac; +Cc: xen-devel

On Sat, Feb 22, 2014 at 08:06:50AM +0000, Zhang, Eniac wrote:
> Hi Konrad,
> 
> Here's what I see when start a VM under xen using pciback to pass a pci-e device into domU.  The device can be seen by guest, and also functioning fine.  But it's not seen as a pci-e device, rather, it looks just like an ordinary pci device because only the first 0x100 bytes of its configuration space is accessible.  So if a driver needs to use data in the extended configuration space for certain features, it will fail.
> 
> When you say you "did PCIe pass through of an VF of an SR-IOV device".  Are you actually using it as a pci-e device or have throttled it back to pci mode without aware of the difference?  If you did see the pci-e device in guest, can you share your xl.cfg file as well as lspci/lspci -t/lspci -xxxx output from guest?

# lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
00:03.0 VGA compatible controller: Cirrus Logic GD 5446
00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
# lspci -t
-[0000:00]-+-00.0
           +-01.0
           +-01.1
           +-01.3
           +-02.0
           +-03.0
           \-04.0
# lspci -s 00:04.0 -xxxxx
00:04.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
00: 86 80 ca 10 06 04 10 00 01 00 00 02 00 00 00 00
10: 04 00 01 f3 00 00 00 00 00 00 00 00 04 40 01 f3
20: 00 00 00 00 00 00 00 00 00 00 00 00 86 80 3c a0
30: 00 00 00 00 70 00 00 00 00 00 00 00 00 00 00 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 11 a0 02 80 03 00 00 00 03 20 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 10 00 02 00 c2 8c 00 00 10 28 00 00 41 6c 03 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 1f 00 00 00 00 00 00 00 00 00 00 00
d0: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

-bash-4.1# more /vm-pci.cfg 
builder='hvm'
disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
memory = 2048
boot="d"
maxvcpus=32
vcpus=1
serial='pty'
vnclisten="0.0.0.0"
name="latest"
pci = ["0000:02:10.0"]

> 
> Also to echo your second comment:  I might still be a newbie in qemu field (I started working on this 4 months ago).  I thought the chipset limits what you can see/do in vm.  Ie.  If you have 440fx emulations then you can't have any pci-e devices (fake or passthru) in the same system.  Is that not true?
> 
> Regards/Eniac
> 
> -----Original Message-----
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] 
> Sent: Friday, February 21, 2014 5:32 PM
> To: Zhang, Eniac
> Cc: xen-devel@lists.xen.org
> Subject: RE: [Xen-devel] q35 in xen? vfio in xen?
> 
> 
> On Feb 21, 2014 4:58 PM, "Zhang, Eniac" <eniac-xw.zhang@hp.com> wrote:
> >
> > Hi Konrad, 
> >
> > Thanks for your reply.  
> >
> > Yes, I am aware of the pciback.  Unfortunately it doesn't seem to support pci-e passthrough. (I could be wrong here)
> 
> I just did PCIe pass through of an VF of an SR-IOV device. It certainly is PCIe.
> 
> >
> > There are two reason that I am interested in this.  For one, my project calls for pci-e device passthrough, which can't be accomplished with 440fx chipset emulation.  Secondly, I feel we ought to move on with the technology.  440fx is ancient in computer terms.  Qemu is good and all that, but if it refuses to support pci-e natively then it's just a matter of time that it will become obsoleted.  The trend is clear that pci-e is taking over the world. 
> >
> 
> I am not sure what you are saying but it does not matter whether QEMU emulates 440fx or q35 for PCI pass through .
> 
> > Regards/Eniac 
> >
> > -----Original Message----- 
> > From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] 
> > Sent: Friday, February 21, 2014 2:50 PM 
> > To: Zhang, Eniac 
> > Cc: xen-devel@lists.xen.org 
> > Subject: Re: [Xen-devel] q35 in xen? vfio in xen? 
> >
> > On Fri, Feb 21, 2014 at 09:41:39PM +0000, Zhang, Eniac wrote: 
> > > Hi all, 
> > > 
> > > I am playing with q35 chipset in qemu (1.6.1).  It seems we can't enable q35 machine under xen yet.  I made a few quick hacks which all fail miserably (linux kernel oops and window BSOD).  I was wondering why this hasn't been done (q35 was introduced into qemu in 2009). 
> > > 
> > > Next question, vfio works very well for me in standalone qemu (with Linux host handling iommu), but is that supported under xen?  I haven't tried anything there yet because my gut-feeling is that it won't work.  Because passing vfio device to qemu can only be done on qemu commandline, and xen is not aware of this passing through device, thus not able to make iommu arrangement for this device.  Am I on the right track here? 
> >
> > Yes and no. VFIO won't work - but QEMU does do PCI passthrough under Xen. It uses a different mechanism (and you need to bind the device to pciback). 
> >
> > > 
> > > I am interested in implementing both these two features.  I'd like to connect with anyone who's already on this so we don't duplicate the efforts. 
> >
> > What do you need Q35 for? 
> >
> > > 
> > > Regards/Eniac 
> >
> > > _______________________________________________ 
> > > Xen-devel mailing list 
> > > Xen-devel@lists.xen.org 
> > > http://lists.xen.org/xen-devel 
> >

^ permalink raw reply	[flat|nested] 11+ messages in thread
* Re: q35 in xen?  vfio in xen?
  2014-02-22  0:31 Konrad Rzeszutek Wilk
@ 2014-02-22  8:06 ` Zhang, Eniac
  2014-02-24 16:40   ` Konrad Rzeszutek Wilk
  0 siblings, 1 reply; 11+ messages in thread
From: Zhang, Eniac @ 2014-02-22  8:06 UTC (permalink / raw)
  To: Konrad Rzeszutek Wilk; +Cc: xen-devel

Hi Konrad,

Here's what I see when start a VM under xen using pciback to pass a pci-e device into domU.  The device can be seen by guest, and also functioning fine.  But it's not seen as a pci-e device, rather, it looks just like an ordinary pci device because only the first 0x100 bytes of its configuration space is accessible.  So if a driver needs to use data in the extended configuration space for certain features, it will fail.

When you say you "did PCIe pass through of an VF of an SR-IOV device".  Are you actually using it as a pci-e device or have throttled it back to pci mode without aware of the difference?  If you did see the pci-e device in guest, can you share your xl.cfg file as well as lspci/lspci -t/lspci -xxxx output from guest?

Also to echo your second comment:  I might still be a newbie in qemu field (I started working on this 4 months ago).  I thought the chipset limits what you can see/do in vm.  Ie.  If you have 440fx emulations then you can't have any pci-e devices (fake or passthru) in the same system.  Is that not true?

Regards/Eniac

-----Original Message-----
From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] 
Sent: Friday, February 21, 2014 5:32 PM
To: Zhang, Eniac
Cc: xen-devel@lists.xen.org
Subject: RE: [Xen-devel] q35 in xen? vfio in xen?


On Feb 21, 2014 4:58 PM, "Zhang, Eniac" <eniac-xw.zhang@hp.com> wrote:
>
> Hi Konrad, 
>
> Thanks for your reply.  
>
> Yes, I am aware of the pciback.  Unfortunately it doesn't seem to support pci-e passthrough. (I could be wrong here)

I just did PCIe pass through of an VF of an SR-IOV device. It certainly is PCIe.

>
> There are two reason that I am interested in this.  For one, my project calls for pci-e device passthrough, which can't be accomplished with 440fx chipset emulation.  Secondly, I feel we ought to move on with the technology.  440fx is ancient in computer terms.  Qemu is good and all that, but if it refuses to support pci-e natively then it's just a matter of time that it will become obsoleted.  The trend is clear that pci-e is taking over the world. 
>

I am not sure what you are saying but it does not matter whether QEMU emulates 440fx or q35 for PCI pass through .

> Regards/Eniac 
>
> -----Original Message----- 
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] 
> Sent: Friday, February 21, 2014 2:50 PM 
> To: Zhang, Eniac 
> Cc: xen-devel@lists.xen.org 
> Subject: Re: [Xen-devel] q35 in xen? vfio in xen? 
>
> On Fri, Feb 21, 2014 at 09:41:39PM +0000, Zhang, Eniac wrote: 
> > Hi all, 
> > 
> > I am playing with q35 chipset in qemu (1.6.1).  It seems we can't enable q35 machine under xen yet.  I made a few quick hacks which all fail miserably (linux kernel oops and window BSOD).  I was wondering why this hasn't been done (q35 was introduced into qemu in 2009). 
> > 
> > Next question, vfio works very well for me in standalone qemu (with Linux host handling iommu), but is that supported under xen?  I haven't tried anything there yet because my gut-feeling is that it won't work.  Because passing vfio device to qemu can only be done on qemu commandline, and xen is not aware of this passing through device, thus not able to make iommu arrangement for this device.  Am I on the right track here? 
>
> Yes and no. VFIO won't work - but QEMU does do PCI passthrough under Xen. It uses a different mechanism (and you need to bind the device to pciback). 
>
> > 
> > I am interested in implementing both these two features.  I'd like to connect with anyone who's already on this so we don't duplicate the efforts. 
>
> What do you need Q35 for? 
>
> > 
> > Regards/Eniac 
>
> > _______________________________________________ 
> > Xen-devel mailing list 
> > Xen-devel@lists.xen.org 
> > http://lists.xen.org/xen-devel 
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread
* Re: q35 in xen?  vfio in xen?
@ 2014-02-22  0:31 Konrad Rzeszutek Wilk
  2014-02-22  8:06 ` Zhang, Eniac
  0 siblings, 1 reply; 11+ messages in thread
From: Konrad Rzeszutek Wilk @ 2014-02-22  0:31 UTC (permalink / raw)
  To: Zhang, Eniac; +Cc: xen-devel


On Feb 21, 2014 4:58 PM, "Zhang, Eniac" <eniac-xw.zhang@hp.com> wrote:
>
> Hi Konrad, 
>
> Thanks for your reply.  
>
> Yes, I am aware of the pciback.  Unfortunately it doesn't seem to support pci-e passthrough. (I could be wrong here)

I just did PCIe pass through of an VF of an SR-IOV device. It certainly is PCIe.

>
> There are two reason that I am interested in this.  For one, my project calls for pci-e device passthrough, which can't be accomplished with 440fx chipset emulation.  Secondly, I feel we ought to move on with the technology.  440fx is ancient in computer terms.  Qemu is good and all that, but if it refuses to support pci-e natively then it's just a matter of time that it will become obsoleted.  The trend is clear that pci-e is taking over the world. 
>

I am not sure what you are saying but it does not matter whether QEMU emulates 440fx or q35 for PCI pass through .

> Regards/Eniac 
>
> -----Original Message----- 
> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com] 
> Sent: Friday, February 21, 2014 2:50 PM 
> To: Zhang, Eniac 
> Cc: xen-devel@lists.xen.org 
> Subject: Re: [Xen-devel] q35 in xen? vfio in xen? 
>
> On Fri, Feb 21, 2014 at 09:41:39PM +0000, Zhang, Eniac wrote: 
> > Hi all, 
> > 
> > I am playing with q35 chipset in qemu (1.6.1).  It seems we can't enable q35 machine under xen yet.  I made a few quick hacks which all fail miserably (linux kernel oops and window BSOD).  I was wondering why this hasn't been done (q35 was introduced into qemu in 2009). 
> > 
> > Next question, vfio works very well for me in standalone qemu (with Linux host handling iommu), but is that supported under xen?  I haven't tried anything there yet because my gut-feeling is that it won't work.  Because passing vfio device to qemu can only be done on qemu commandline, and xen is not aware of this passing through device, thus not able to make iommu arrangement for this device.  Am I on the right track here? 
>
> Yes and no. VFIO won't work - but QEMU does do PCI passthrough under Xen. It uses a different mechanism (and you need to bind the device to pciback). 
>
> > 
> > I am interested in implementing both these two features.  I'd like to connect with anyone who's already on this so we don't duplicate the efforts. 
>
> What do you need Q35 for? 
>
> > 
> > Regards/Eniac 
>
> > _______________________________________________ 
> > Xen-devel mailing list 
> > Xen-devel@lists.xen.org 
> > http://lists.xen.org/xen-devel 
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread
end of thread, other threads:[~2014-02-26 13:25 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-02-21 21:41 q35 in xen? vfio in xen? Zhang, Eniac
2014-02-21 21:50 ` Konrad Rzeszutek Wilk
2014-02-21 21:58   ` Zhang, Eniac
2014-02-21 22:09 ` Fabio Fantoni
2014-02-22  0:31 Konrad Rzeszutek Wilk
2014-02-22  8:06 ` Zhang, Eniac
2014-02-24 16:40   ` Konrad Rzeszutek Wilk
2014-02-24 19:28     ` Zhang, Eniac
2014-02-24 19:43       ` Konrad Rzeszutek Wilk
2014-02-26 11:58         ` Stefano Stabellini
2014-02-26 13:25           ` Fabio Fantoni

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.