SR-IOV dom0

SR-IOV dom0

[Simon Horman]

Hi Weidong,

I'm wondering what tree you are using for SR-IOV dom0 development.

Also, I am guessing its not linux-xen-2.6.18.8, but are you aware
of the status of SR-IOV in that tree? In particular, the igb driver seems
to not have SR-IOV support. I am considering back-porting it, but I'd like
to avoid duplicating the efforts of others.

Thanks

RE: SR-IOV dom0

[Han, Weidong]

Simon Horman wrote:
> Hi Weidong,
> 
> I'm wondering what tree you are using for SR-IOV dom0 development.
> 
> Also, I am guessing its not linux-xen-2.6.18.8, but are you aware
> of the status of SR-IOV in that tree? In particular, the igb driver
> seems to not have SR-IOV support. I am considering back-porting it,
> but I'd like to avoid duplicating the efforts of others.
> 
> Thanks

Hi Simon,

Yes, I'm working on jeremy's pv-ops dom0. We are re-enabling VT-d on pv-ops dom0 first, my last patch sent out yesterday is to support hotplug device for assignment. That is also needed by SR-IOV. igb driver in pv-ops dom0 supports SR-IOV. The last thing missed for SR-IOV is MSI/MSI-x supported in pv-ops dom0. Now we are working on it. Thanks.

Regards,
Weidong

Re: RE: SR-IOV for pv_ops dom0

[Pasi Kärkkäinen]

On Tue, Jul 28, 2009 at 01:36:55PM +0800, Han, Weidong wrote:
> Simon Horman wrote:
> > Hi Weidong,
> > 
> > I'm wondering what tree you are using for SR-IOV dom0 development.
> > 
> > Also, I am guessing its not linux-xen-2.6.18.8, but are you aware
> > of the status of SR-IOV in that tree? In particular, the igb driver
> > seems to not have SR-IOV support. I am considering back-porting it,
> > but I'd like to avoid duplicating the efforts of others.
> > 
> > Thanks
> 
> Hi Simon,
> 
> Yes, I'm working on jeremy's pv-ops dom0. We are re-enabling VT-d on pv-ops dom0 first, my last patch sent out yesterday is to support hotplug device for assignment. That is also needed by SR-IOV. igb driver in pv-ops dom0 supports SR-IOV. The last thing missed for SR-IOV is MSI/MSI-x supported in pv-ops dom0. Now we are working on it. Thanks.
> 

Hello,

Is pv_ops dom0 (xen/master) currently ready for SR-IOV, or are
additional patches still needed? 

Can upstream/vanilla 2.6.31 be used as PV guest and run the VF driver?
Or does it need additional patches?

Thanks!

-- Pasi

Re: [Xen-devel] RE: SR-IOV for pv_ops dom0

[Qing He]

On Sat, 2009-09-26 at 19:06 +0800, Pasi Kärkkäinen wrote:
> Hello,
> 
> Is pv_ops dom0 (xen/master) currently ready for SR-IOV, or are
> additional patches still needed? 
> 

VFs can be created and assigned to HVM domains, we've tested that. But
assigning VFs to PV domains doesn't work since the support isn't
complete (at least MSI, I'm not sure about the status of INTx device).

> Can upstream/vanilla 2.6.31 be used as PV guest and run the VF driver?
> Or does it need additional patches?

Vanilla kernel doesn't have pirq and msi-x related code, so VF doesn't
work there either.

Thanks,
Qing

Re: [Xen-devel] RE: SR-IOV for pv_ops dom0

[Pasi Kärkkäinen]

On Sun, Sep 27, 2009 at 11:02:38AM +0800, Qing He wrote:
> On Sat, 2009-09-26 at 19:06 +0800, Pasi Kärkkäinen wrote:
> > Hello,
> > 
> > Is pv_ops dom0 (xen/master) currently ready for SR-IOV, or are
> > additional patches still needed? 
> > 
> 
> VFs can be created and assigned to HVM domains, we've tested that. But
> assigning VFs to PV domains doesn't work since the support isn't
> complete (at least MSI, I'm not sure about the status of INTx device).
> 

Ok. Do you know if anyone is working on adding those missing bits? 
Is it missing only from domU kernel, or also from Xen hypervisor PV code?

Also, what kernel you used in the HVM guest for SR-IOV? Just so I can
try this myself some day.. :)

> > Can upstream/vanilla 2.6.31 be used as PV guest and run the VF driver?
> > Or does it need additional patches?
> 
> Vanilla kernel doesn't have pirq and msi-x related code, so VF doesn't
> work there either.
> 

Ok. Do you know if this is being worked on?

Thanks for the replies!

-- Pasi

Re: RE: SR-IOV for pv_ops dom0

[Qing He]

On Sun, 2009-09-27 at 17:47 +0800, Pasi Kärkkäinen wrote:
> On Sun, Sep 27, 2009 at 11:02:38AM +0800, Qing He wrote:
> > On Sat, 2009-09-26 at 19:06 +0800, Pasi Kärkkäinen wrote:
> > > Hello,
> > > 
> > > Is pv_ops dom0 (xen/master) currently ready for SR-IOV, or are
> > > additional patches still needed? 
> > > 
> > 
> > VFs can be created and assigned to HVM domains, we've tested that. But
> > assigning VFs to PV domains doesn't work since the support isn't
> > complete (at least MSI, I'm not sure about the status of INTx device).
> > 
> 
> Ok. Do you know if anyone is working on adding those missing bits? 
> Is it missing only from domU kernel, or also from Xen hypervisor PV code?

Hypervisor has the proper code, and old 2.6.18 based domU works well.
It's just pv_ops based domU lacking this feature. Traditionally, domU
doesn't have sufficient privilege but relies on dom0 to do certain
things, this kind of delegation requires some special handling and
is still subjected to more thinking.

> 
> Also, what kernel you used in the HVM guest for SR-IOV? Just so I can
> try this myself some day.. :)

The kernel version is not important, just need the driver of VFs, like
igbvf.

> 
> > > Can upstream/vanilla 2.6.31 be used as PV guest and run the VF driver?
> > > Or does it need additional patches?
> > 
> > Vanilla kernel doesn't have pirq and msi-x related code, so VF doesn't
> > work there either.
> > 
> 
> Ok. Do you know if this is being worked on?

Yes, but that's likely to take quite a long time to eventually get
everything in.

Thanks,
Qing

> 
> Thanks for the replies!
> 
> -- Pasi
> 

Re: [Xen-devel] RE: SR-IOV for pv_ops dom0

[Pasi Kärkkäinen]

On Mon, Sep 28, 2009 at 04:51:07PM +0800, Qing He wrote:
> On Sun, 2009-09-27 at 17:47 +0800, Pasi Kärkkäinen wrote:
> > On Sun, Sep 27, 2009 at 11:02:38AM +0800, Qing He wrote:
> > > On Sat, 2009-09-26 at 19:06 +0800, Pasi Kärkkäinen wrote:
> > > > Hello,
> > > > 
> > > > Is pv_ops dom0 (xen/master) currently ready for SR-IOV, or are
> > > > additional patches still needed? 
> > > > 
> > > 
> > > VFs can be created and assigned to HVM domains, we've tested that. But
> > > assigning VFs to PV domains doesn't work since the support isn't
> > > complete (at least MSI, I'm not sure about the status of INTx device).
> > > 
> > 
> > Ok. Do you know if anyone is working on adding those missing bits? 
> > Is it missing only from domU kernel, or also from Xen hypervisor PV code?
> 
> Hypervisor has the proper code, and old 2.6.18 based domU works well.
> It's just pv_ops based domU lacking this feature. Traditionally, domU
> doesn't have sufficient privilege but relies on dom0 to do certain
> things, this kind of delegation requires some special handling and
> is still subjected to more thinking.
>

Ok. So if I use linux-2.6.18-xen.hg everything should work also for PV domU.
Does Xen 3.4.1 hypervisor have the needed bits, or is it just xen-unstable? 


> > 
> > Also, what kernel you used in the HVM guest for SR-IOV? Just so I can
> > try this myself some day.. :)
> 
> The kernel version is not important, just need the driver of VFs, like
> igbvf.
> 

OK.

> > 
> > > > Can upstream/vanilla 2.6.31 be used as PV guest and run the VF driver?
> > > > Or does it need additional patches?
> > > 
> > > Vanilla kernel doesn't have pirq and msi-x related code, so VF doesn't
> > > work there either.
> > > 
> > 
> > Ok. Do you know if this is being worked on?
> 
> Yes, but that's likely to take quite a long time to eventually get
> everything in.
> 

Ok. 

Thanks!

-- Pasi