Are there source codes for Xen-IB?

Are there source codes for Xen-IB?

[maus]

[-- Attachment #1.1: Type: text/plain, Size: 200 bytes --]

Hi,

I found a link: http://xenbits.xensource.com/ext/xen-smartio.hg for Xen-IB
source codes, but it did't exist anymore.
Are there other ways to get the Xen-IB source codes?
Thanks.

Regards,
Yi-Man

[-- Attachment #1.2: Type: text/html, Size: 364 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Are there source codes for Xen-IB?

[Fajar A. Nugraha]

On Thu, May 19, 2011 at 1:27 PM, maus <maustudent@gmail.com> wrote:
> Hi,
>
> I found a link: http://xenbits.xensource.com/ext/xen-smartio.hg for Xen-IB
> source codes, but it did't exist anymore.
> Are there other ways to get the Xen-IB source codes?
> Thanks.

Probably http://xenbits.xensource.com/ext/xen-smartio/

-- 
Fajar

Re: Are there source codes for Xen-IB?

[Ian Campbell]

On Mon, 2011-05-23 at 11:03 +0100, Fajar A. Nugraha wrote:
> On Thu, May 19, 2011 at 1:27 PM, maus <maustudent@gmail.com> wrote:
> > Hi,
> >
> > I found a link: http://xenbits.xensource.com/ext/xen-smartio.hg for Xen-IB
> > source codes, but it did't exist anymore.
> > Are there other ways to get the Xen-IB source codes?
> > Thanks.
> 
> Probably http://xenbits.xensource.com/ext/xen-smartio/

Note that this tree hasn't been touched since 2006 and appears to be
based on a Xen 3.0.0 era hypervisor and the changes mostly consist of
patches to the 2.6.16 kernel used in that era.

IOW I doubt you will find much useful code in that repository for
further development and you most certainly should not be deploying
anything based on it.

Ian.

Re: Are there source codes for Xen-IB?

[maus]

[-- Attachment #1.1: Type: text/plain, Size: 1350 bytes --]

Hi,

There many file in  http://xenbits.xensource.com/ext/xen-smartio/ , which
should I use for Xen-IB?
Use the following three?
Updated defconfig to include infiniband driver as modules for
dom0/domU<http://xenbits.xensource.com/ext/xen-smartio/rev/b796b92c0cbe>
Updating drivers/infiniband to latest. Should shrink size of
xen_ib.patch in<http://xenbits.xensource.com/ext/xen-smartio/rev/13ee554d2b7c>
Add support for Infiniband into
Xen<http://xenbits.xensource.com/ext/xen-smartio/rev/610fe641e6e8>


Yi-Man


2011/5/23 Ian Campbell <Ian.Campbell@citrix.com>

> On Mon, 2011-05-23 at 11:03 +0100, Fajar A. Nugraha wrote:
> > On Thu, May 19, 2011 at 1:27 PM, maus <maustudent@gmail.com> wrote:
> > > Hi,
> > >
> > > I found a link: http://xenbits.xensource.com/ext/xen-smartio.hg for
> Xen-IB
> > > source codes, but it did't exist anymore.
> > > Are there other ways to get the Xen-IB source codes?
> > > Thanks.
> >
> > Probably http://xenbits.xensource.com/ext/xen-smartio/
>
> Note that this tree hasn't been touched since 2006 and appears to be
> based on a Xen 3.0.0 era hypervisor and the changes mostly consist of
> patches to the 2.6.16 kernel used in that era.
>
> IOW I doubt you will find much useful code in that repository for
> further development and you most certainly should not be deploying
> anything based on it.
>
> Ian.
>
>

[-- Attachment #1.2: Type: text/html, Size: 2135 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Are there source codes for Xen-IB?

[Ian Campbell]

Please do not top post.

On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
> Hi,
> 
> There many file in  http://xenbits.xensource.com/ext/xen-smartio/ ,
> which should I use for Xen-IB?
> Use the following three?
> Updated defconfig to include infiniband driver as modules for
> dom0/domU 
> Updating drivers/infiniband to latest. Should shrink size of
> xen_ib.patch in
> Add support for Infiniband into Xen

Again, these changesets are >5 years old and have not been looked at or
maintained since -- I am very sceptical that you will find anything of
use on a modern Xen system in there.

I expect that IB support in the kernels we currently use for dom0 is
vastly improved over what was available in the 2.6.16 tree referenced in
the above repository. Perhaps you don't even need anything special
outside of what is available in the upstream dom0 kernel.

Perhaps if you explain what you are actually trying to achieve then
people will be able to better advise you?

Ian.

Re: Are there source codes for Xen-IB?

[maus]

[-- Attachment #1.1: Type: text/plain, Size: 1574 bytes --]

Hi,

I have a project which is make KVM use InfiniBand.
However, since Xen can use InfiniBand through Xen-IB, I want to try to make
Xen use InfiniBand first.
Through that way, I think it may give me some idea about how to make KVM use
InfiniBand.
So, as you said, I just use "Updated defconfig to include infiniband driver
as modules for dom0/domU" to do?
And since I don't know how to I do first for Xen-IB, is there any tutorial
for Xen-IB?
Or does Xen build-in support IB currently?

Yi-Man

2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>

> Please do not top post.
>
> On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
> > Hi,
> >
> > There many file in  http://xenbits.xensource.com/ext/xen-smartio/ ,
> > which should I use for Xen-IB?
> > Use the following three?
> > Updated defconfig to include infiniband driver as modules for
> > dom0/domU
> > Updating drivers/infiniband to latest. Should shrink size of
> > xen_ib.patch in
> > Add support for Infiniband into Xen
>
> Again, these changesets are >5 years old and have not been looked at or
> maintained since -- I am very sceptical that you will find anything of
> use on a modern Xen system in there.
>
> I expect that IB support in the kernels we currently use for dom0 is
> vastly improved over what was available in the 2.6.16 tree referenced in
> the above repository. Perhaps you don't even need anything special
> outside of what is available in the upstream dom0 kernel.
>
> Perhaps if you explain what you are actually trying to achieve then
> people will be able to better advise you?
>
> Ian.
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 2091 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Are there source codes for Xen-IB?

[Ian Campbell]

On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
> Hi,
> 
> I have a project which is make KVM use InfiniBand.
> However, since Xen can use InfiniBand through Xen-IB, I want to try to
> make Xen use InfiniBand first.
> Through that way, I think it may give me some idea about how to make
> KVM use InfiniBand.
> So, as you said, I just use "Updated defconfig to include infiniband
> driver as modules for dom0/domU" to do?
> And since I don't know how to I do first for Xen-IB, is there any
> tutorial for Xen-IB?
> Or does Xen build-in support IB currently?

I'm not sure what IB support you expect Xen to have over and above the
driver and utils support, which are functions of the dom0 kernel and
distro and not the hypervisor.

Have you got IB working on a native system? I expect that if you do that
then Xen or KVM will be able to make use of it basically for free, i.e.
configure a VIF on an IP-over-IB bridge or use a local IP-over-IB
address in dom0 for migration etc etc.

Ian.

> 
> Yi-Man
> 
> 
> 
> 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
>         Please do not top post.
>         
>         On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
>         > Hi,
>         >
>         > There many file in
>          http://xenbits.xensource.com/ext/xen-smartio/ ,
>         > which should I use for Xen-IB?
>         > Use the following three?
>         > Updated defconfig to include infiniband driver as modules
>         for
>         > dom0/domU
>         > Updating drivers/infiniband to latest. Should shrink size of
>         > xen_ib.patch in
>         > Add support for Infiniband into Xen
>         
>         
>         Again, these changesets are >5 years old and have not been
>         looked at or
>         maintained since -- I am very sceptical that you will find
>         anything of
>         use on a modern Xen system in there.
>         
>         I expect that IB support in the kernels we currently use for
>         dom0 is
>         vastly improved over what was available in the 2.6.16 tree
>         referenced in
>         the above repository. Perhaps you don't even need anything
>         special
>         outside of what is available in the upstream dom0 kernel.
>         
>         Perhaps if you explain what you are actually trying to achieve
>         then
>         people will be able to better advise you?
>         
>         Ian.
>         
>         
> 

Re: Are there source codes for Xen-IB?

[maus]

[-- Attachment #1.1: Type: text/plain, Size: 2700 bytes --]

Hi,

I know use IP-over-IB can let KVM or Xen use IB, but what I have to do is
like Xen-IB. Not just use IB.
So, there is no toturial for Xen-IB,right?


Yi-Man



2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>

> On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
> > Hi,
> >
> > I have a project which is make KVM use InfiniBand.
> > However, since Xen can use InfiniBand through Xen-IB, I want to try to
> > make Xen use InfiniBand first.
> > Through that way, I think it may give me some idea about how to make
> > KVM use InfiniBand.
> > So, as you said, I just use "Updated defconfig to include infiniband
> > driver as modules for dom0/domU" to do?
> > And since I don't know how to I do first for Xen-IB, is there any
> > tutorial for Xen-IB?
> > Or does Xen build-in support IB currently?
>
> I'm not sure what IB support you expect Xen to have over and above the
> driver and utils support, which are functions of the dom0 kernel and
> distro and not the hypervisor.
>
> Have you got IB working on a native system? I expect that if you do that
> then Xen or KVM will be able to make use of it basically for free, i.e.
> configure a VIF on an IP-over-IB bridge or use a local IP-over-IB
> address in dom0 for migration etc etc.
>
> Ian.
>
> >
> > Yi-Man
> >
> >
> >
> > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
> >         Please do not top post.
> >
> >         On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
> >         > Hi,
> >         >
> >         > There many file in
> >          http://xenbits.xensource.com/ext/xen-smartio/ ,
> >         > which should I use for Xen-IB?
> >         > Use the following three?
> >         > Updated defconfig to include infiniband driver as modules
> >         for
> >         > dom0/domU
> >         > Updating drivers/infiniband to latest. Should shrink size of
> >         > xen_ib.patch in
> >         > Add support for Infiniband into Xen
> >
> >
> >         Again, these changesets are >5 years old and have not been
> >         looked at or
> >         maintained since -- I am very sceptical that you will find
> >         anything of
> >         use on a modern Xen system in there.
> >
> >         I expect that IB support in the kernels we currently use for
> >         dom0 is
> >         vastly improved over what was available in the 2.6.16 tree
> >         referenced in
> >         the above repository. Perhaps you don't even need anything
> >         special
> >         outside of what is available in the upstream dom0 kernel.
> >
> >         Perhaps if you explain what you are actually trying to achieve
> >         then
> >         people will be able to better advise you?
> >
> >         Ian.
> >
> >
> >
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 3608 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Are there source codes for Xen-IB?

[Ian Campbell]

On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
> Hi,
>  
> I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
> is like Xen-IB. Not just use IB.

What is Xen-IB exactly?

> So, there is no toturial for Xen-IB,right?

That's right, there is no documentation and anyone who would have been
involved with this stuff is long gone.

Ian.

>  
>  
> Yi-Man
> 
> 
>  
> 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
>         On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
>         > Hi,
>         >
>         > I have a project which is make KVM use InfiniBand.
>         > However, since Xen can use InfiniBand through Xen-IB, I want
>         to try to
>         > make Xen use InfiniBand first.
>         > Through that way, I think it may give me some idea about how
>         to make
>         > KVM use InfiniBand.
>         > So, as you said, I just use "Updated defconfig to include
>         infiniband
>         > driver as modules for dom0/domU" to do?
>         > And since I don't know how to I do first for Xen-IB, is
>         there any
>         > tutorial for Xen-IB?
>         > Or does Xen build-in support IB currently?
>         
>         
>         I'm not sure what IB support you expect Xen to have over and
>         above the
>         driver and utils support, which are functions of the dom0
>         kernel and
>         distro and not the hypervisor.
>         
>         Have you got IB working on a native system? I expect that if
>         you do that
>         then Xen or KVM will be able to make use of it basically for
>         free, i.e.
>         configure a VIF on an IP-over-IB bridge or use a local
>         IP-over-IB
>         address in dom0 for migration etc etc.
>         
>         Ian.
>         
>         
>         >
>         > Yi-Man
>         >
>         >
>         >
>         > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
>         >         Please do not top post.
>         >
>         >         On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
>         >         > Hi,
>         >         >
>         >         > There many file in
>         >          http://xenbits.xensource.com/ext/xen-smartio/ ,
>         >         > which should I use for Xen-IB?
>         >         > Use the following three?
>         >         > Updated defconfig to include infiniband driver as
>         modules
>         >         for
>         >         > dom0/domU
>         >         > Updating drivers/infiniband to latest. Should
>         shrink size of
>         >         > xen_ib.patch in
>         >         > Add support for Infiniband into Xen
>         >
>         >
>         >         Again, these changesets are >5 years old and have
>         not been
>         >         looked at or
>         >         maintained since -- I am very sceptical that you
>         will find
>         >         anything of
>         >         use on a modern Xen system in there.
>         >
>         >         I expect that IB support in the kernels we currently
>         use for
>         >         dom0 is
>         >         vastly improved over what was available in the
>         2.6.16 tree
>         >         referenced in
>         >         the above repository. Perhaps you don't even need
>         anything
>         >         special
>         >         outside of what is available in the upstream dom0
>         kernel.
>         >
>         >         Perhaps if you explain what you are actually trying
>         to achieve
>         >         then
>         >         people will be able to better advise you?
>         >
>         >         Ian.
>         >
>         >
>         >
>         
>         
>         
> 

Re: Are there source codes for Xen-IB?

[Pasi Kärkkäinen]

On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
> > Hi,
> >  
> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
> > is like Xen-IB. Not just use IB.
> 
> What is Xen-IB exactly?
> 

Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ?
I can't really think of other uses..

-- Pasi


> > So, there is no toturial for Xen-IB,right?
> 
> That's right, there is no documentation and anyone who would have been
> involved with this stuff is long gone.
> 
> Ian.
> 
> >  
> >  
> > Yi-Man
> > 
> > 
> >  
> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
> >         On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
> >         > Hi,
> >         >
> >         > I have a project which is make KVM use InfiniBand.
> >         > However, since Xen can use InfiniBand through Xen-IB, I want
> >         to try to
> >         > make Xen use InfiniBand first.
> >         > Through that way, I think it may give me some idea about how
> >         to make
> >         > KVM use InfiniBand.
> >         > So, as you said, I just use "Updated defconfig to include
> >         infiniband
> >         > driver as modules for dom0/domU" to do?
> >         > And since I don't know how to I do first for Xen-IB, is
> >         there any
> >         > tutorial for Xen-IB?
> >         > Or does Xen build-in support IB currently?
> >         
> >         
> >         I'm not sure what IB support you expect Xen to have over and
> >         above the
> >         driver and utils support, which are functions of the dom0
> >         kernel and
> >         distro and not the hypervisor.
> >         
> >         Have you got IB working on a native system? I expect that if
> >         you do that
> >         then Xen or KVM will be able to make use of it basically for
> >         free, i.e.
> >         configure a VIF on an IP-over-IB bridge or use a local
> >         IP-over-IB
> >         address in dom0 for migration etc etc.
> >         
> >         Ian.
> >         
> >         
> >         >
> >         > Yi-Man
> >         >
> >         >
> >         >
> >         > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
> >         >         Please do not top post.
> >         >
> >         >         On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
> >         >         > Hi,
> >         >         >
> >         >         > There many file in
> >         >          http://xenbits.xensource.com/ext/xen-smartio/ ,
> >         >         > which should I use for Xen-IB?
> >         >         > Use the following three?
> >         >         > Updated defconfig to include infiniband driver as
> >         modules
> >         >         for
> >         >         > dom0/domU
> >         >         > Updating drivers/infiniband to latest. Should
> >         shrink size of
> >         >         > xen_ib.patch in
> >         >         > Add support for Infiniband into Xen
> >         >
> >         >
> >         >         Again, these changesets are >5 years old and have
> >         not been
> >         >         looked at or
> >         >         maintained since -- I am very sceptical that you
> >         will find
> >         >         anything of
> >         >         use on a modern Xen system in there.
> >         >
> >         >         I expect that IB support in the kernels we currently
> >         use for
> >         >         dom0 is
> >         >         vastly improved over what was available in the
> >         2.6.16 tree
> >         >         referenced in
> >         >         the above repository. Perhaps you don't even need
> >         anything
> >         >         special
> >         >         outside of what is available in the upstream dom0
> >         kernel.
> >         >
> >         >         Perhaps if you explain what you are actually trying
> >         to achieve
> >         >         then
> >         >         people will be able to better advise you?
> >         >
> >         >         Ian.
> >         >
> >         >
> >         >
> >         
> >         
> >         
> > 
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

Re: Are there source codes for Xen-IB?

[Joseph Glanville]

Hi,

I happen to have done a huge amount of research into this as our
infrastructure is all based on converged Infiniband.

Xen-IB and SmartIO were two seperate attempts at building a
paravirtualised Infinband driver for Xen that allowed guests to use
native Infinband protocols at close to native speeds. Being
paravirtualised they were live-migration cappable and many domU could
share a single Infiniband adapter.

As far as I am aware both of the projects are dead, however SR-IOV
isn't long off for Infiniband, Mellanox is on target to push theirs
upstream into OFED 1.6 due in November, Qlogic and Oracle/Sun will be
quick to follow I would imagine.

SR-IOV will allow pretty similar functionality but less portable in
terms of live-migrate etc.

It is worth noting here that you can't bridge IPoIB, IPoIB is not a
layer 2 device and doesn't understand Ethernet. You can use EoIB
(Ethernet over IB) or routing to provide network access to domUs.

Feel free to email me if you need any help with Infiniband and Xen.

Joseph.

On 27 May 2011 04:50, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
>> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
>> > Hi,
>> >
>> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
>> > is like Xen-IB. Not just use IB.
>>
>> What is Xen-IB exactly?
>>
>
> Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ?
> I can't really think of other uses..
>
> -- Pasi
>
>
>> > So, there is no toturial for Xen-IB,right?
>>
>> That's right, there is no documentation and anyone who would have been
>> involved with this stuff is long gone.
>>
>> Ian.
>>
>> >
>> >
>> > Yi-Man
>> >
>> >
>> >
>> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
>> >         On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
>> >         > Hi,
>> >         >
>> >         > I have a project which is make KVM use InfiniBand.
>> >         > However, since Xen can use InfiniBand through Xen-IB, I want
>> >         to try to
>> >         > make Xen use InfiniBand first.
>> >         > Through that way, I think it may give me some idea about how
>> >         to make
>> >         > KVM use InfiniBand.
>> >         > So, as you said, I just use "Updated defconfig to include
>> >         infiniband
>> >         > driver as modules for dom0/domU" to do?
>> >         > And since I don't know how to I do first for Xen-IB, is
>> >         there any
>> >         > tutorial for Xen-IB?
>> >         > Or does Xen build-in support IB currently?
>> >
>> >
>> >         I'm not sure what IB support you expect Xen to have over and
>> >         above the
>> >         driver and utils support, which are functions of the dom0
>> >         kernel and
>> >         distro and not the hypervisor.
>> >
>> >         Have you got IB working on a native system? I expect that if
>> >         you do that
>> >         then Xen or KVM will be able to make use of it basically for
>> >         free, i.e.
>> >         configure a VIF on an IP-over-IB bridge or use a local
>> >         IP-over-IB
>> >         address in dom0 for migration etc etc.
>> >
>> >         Ian.
>> >
>> >
>> >         >
>> >         > Yi-Man
>> >         >
>> >         >
>> >         >
>> >         > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
>> >         >         Please do not top post.
>> >         >
>> >         >         On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
>> >         >         > Hi,
>> >         >         >
>> >         >         > There many file in
>> >         >          http://xenbits.xensource.com/ext/xen-smartio/ ,
>> >         >         > which should I use for Xen-IB?
>> >         >         > Use the following three?
>> >         >         > Updated defconfig to include infiniband driver as
>> >         modules
>> >         >         for
>> >         >         > dom0/domU
>> >         >         > Updating drivers/infiniband to latest. Should
>> >         shrink size of
>> >         >         > xen_ib.patch in
>> >         >         > Add support for Infiniband into Xen
>> >         >
>> >         >
>> >         >         Again, these changesets are >5 years old and have
>> >         not been
>> >         >         looked at or
>> >         >         maintained since -- I am very sceptical that you
>> >         will find
>> >         >         anything of
>> >         >         use on a modern Xen system in there.
>> >         >
>> >         >         I expect that IB support in the kernels we currently
>> >         use for
>> >         >         dom0 is
>> >         >         vastly improved over what was available in the
>> >         2.6.16 tree
>> >         >         referenced in
>> >         >         the above repository. Perhaps you don't even need
>> >         anything
>> >         >         special
>> >         >         outside of what is available in the upstream dom0
>> >         kernel.
>> >         >
>> >         >         Perhaps if you explain what you are actually trying
>> >         to achieve
>> >         >         then
>> >         >         people will be able to better advise you?
>> >         >
>> >         >         Ian.
>> >         >
>> >         >
>> >         >
>> >
>> >
>> >
>> >
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>



-- 
Kind regards,
Joseph.
Founder | Director
Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56
99 52 | Mobile: 0428 754 846

Re: Are there source codes for Xen-IB?

[Muli Ben-Yehuda]

On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
> > Hi,
> >  
> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
> > is like Xen-IB. Not just use IB.
> 
> What is Xen-IB exactly?

OS and hypervisor bypass, i.e., direct access from a user application
running in a domU to the Infiniband adapter (HCA), bypassing both the
domU kernel and Xen for frequent data-path operations. Configuration
and privileged operations still go through the kernel and the
hypervisor.

> > So, there is no toturial for Xen-IB,right?
> 
> That's right, there is no documentation and anyone who would have
> been involved with this stuff is long gone.

Actually, there is a very nice paper describing the approach:

  "High Performance VMM-Bypass I/O in Virtual Machines", Jiuxing
  Liu, Wei Huang, Bulent Abali, Dhabaleswar K. Panda, USENIX ATC '06

Cheers,
Muli

Are there source codes for Xen-IB?

[Joseph Glanville]

Hi,

I happen to have done a huge amount of research into this as our
infrastructure is all based on converged Infiniband.

Xen-IB and SmartIO were two seperate attempts at building a
paravirtualised Infinband driver for Xen that allowed guests to use
native Infinband protocols at close to native speeds. Being
paravirtualised they were live-migration cappable and many domU could
share a single Infiniband adapter.

As far as I am aware both of the projects are dead, however SR-IOV
isn't long off for Infiniband, Mellanox is on target to push theirs
upstream into OFED 1.6 due in November, Qlogic and Oracle/Sun will be
quick to follow I would imagine.

SR-IOV will allow pretty similar functionality but less portable in
terms of live-migrate etc.

It is worth noting here that you can't bridge IPoIB, IPoIB is not a
layer 2 device and doesn't understand Ethernet. You can use EoIB
(Ethernet over IB) or routing to provide network access to domUs.

Feel free to email me if you need any help with Infiniband and Xen.

Joseph.

On 27 May 2011 04:50, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
>> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
>> > Hi,
>> >
>> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
>> > is like Xen-IB. Not just use IB.
>>
>> What is Xen-IB exactly?
>>
>
> Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ?
> I can't really think of other uses..
>
> -- Pasi
>
>
>> > So, there is no toturial for Xen-IB,right?
>>
>> That's right, there is no documentation and anyone who would have been
>> involved with this stuff is long gone.
>>
>> Ian.
>>
>> >
>> >
>> > Yi-Man
>> >
>> >
>> >
>> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
>> >         On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
>> >         > Hi,
>> >         >
>> >         > I have a project which is make KVM use InfiniBand.
>> >         > However, since Xen can use InfiniBand through Xen-IB, I want
>> >         to try to
>> >         > make Xen use InfiniBand first.
>> >         > Through that way, I think it may give me some idea about how
>> >         to make
>> >         > KVM use InfiniBand.
>> >         > So, as you said, I just use "Updated defconfig to include
>> >         infiniband
>> >         > driver as modules for dom0/domU" to do?
>> >         > And since I don't know how to I do first for Xen-IB, is
>> >         there any
>> >         > tutorial for Xen-IB?
>> >         > Or does Xen build-in support IB currently?
>> >
>> >
>> >         I'm not sure what IB support you expect Xen to have over and
>> >         above the
>> >         driver and utils support, which are functions of the dom0
>> >         kernel and
>> >         distro and not the hypervisor.
>> >
>> >         Have you got IB working on a native system? I expect that if
>> >         you do that
>> >         then Xen or KVM will be able to make use of it basically for
>> >         free, i.e.
>> >         configure a VIF on an IP-over-IB bridge or use a local
>> >         IP-over-IB
>> >         address in dom0 for migration etc etc.
>> >
>> >         Ian.
>> >
>> >
>> >         >
>> >         > Yi-Man
>> >         >
>> >         >
>> >         >
>> >         > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
>> >         >         Please do not top post.
>> >         >
>> >         >         On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
>> >         >         > Hi,
>> >         >         >
>> >         >         > There many file in
>> >         >          http://xenbits.xensource.com/ext/xen-smartio/ ,
>> >         >         > which should I use for Xen-IB?
>> >         >         > Use the following three?
>> >         >         > Updated defconfig to include infiniband driver as
>> >         modules
>> >         >         for
>> >         >         > dom0/domU
>> >         >         > Updating drivers/infiniband to latest. Should
>> >         shrink size of
>> >         >         > xen_ib.patch in
>> >         >         > Add support for Infiniband into Xen
>> >         >
>> >         >
>> >         >         Again, these changesets are >5 years old and have
>> >         not been
>> >         >         looked at or
>> >         >         maintained since -- I am very sceptical that you
>> >         will find
>> >         >         anything of
>> >         >         use on a modern Xen system in there.
>> >         >
>> >         >         I expect that IB support in the kernels we currently
>> >         use for
>> >         >         dom0 is
>> >         >         vastly improved over what was available in the
>> >         2.6.16 tree
>> >         >         referenced in
>> >         >         the above repository. Perhaps you don't even need
>> >         anything
>> >         >         special
>> >         >         outside of what is available in the upstream dom0
>> >         kernel.
>> >         >
>> >         >         Perhaps if you explain what you are actually trying
>> >         to achieve
>> >         >         then
>> >         >         people will be able to better advise you?
>> >         >
>> >         >         Ian.
>> >         >
>> >         >
>> >         >
>> >
>> >
>> >
>> >
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>



--
Kind regards,
Joseph.
Founder | Director
Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56
99 52 | Mobile: 0428 754 846



-- 
Kind regards,
Joseph.
Founder | Director
Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56
99 52 | Mobile: 0428 754 846

Re: Are there source codes for Xen-IB?

[Ian Campbell]

On Sun, 2011-05-29 at 10:08 +0100, Muli Ben-Yehuda wrote:
> On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
> > On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
> > > Hi,
> > >  
> > > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
> > > is like Xen-IB. Not just use IB.
> > 
> > What is Xen-IB exactly?
> 
> OS and hypervisor bypass, i.e., direct access from a user application
> running in a domU to the Infiniband adapter (HCA), bypassing both the
> domU kernel and Xen for frequent data-path operations. Configuration
> and privileged operations still go through the kernel and the
> hypervisor.
> 
> > > So, there is no toturial for Xen-IB,right?
> > 
> > That's right, there is no documentation and anyone who would have
> > been involved with this stuff is long gone.
> 
> Actually, there is a very nice paper describing the approach:
> 
>   "High Performance VMM-Bypass I/O in Virtual Machines", Jiuxing
>   Liu, Wei Huang, Bulent Abali, Dhabaleswar K. Panda, USENIX ATC '06

Thanks for the reference.

Are any of these folks still active? It seems like they could be far
more helpful to the OP than anyone here (certainly more useful than
me ;-))...

Ian.

Re: Are there source codes for Xen-IB?

[Ian Campbell]

On Sat, 2011-05-28 at 19:15 +0100, Joseph Glanville wrote:
> Hi,
> 
> I happen to have done a huge amount of research into this as our
> infrastructure is all based on converged Infiniband.
> 
> Xen-IB and SmartIO were two seperate attempts at building a
> paravirtualised Infinband driver for Xen that allowed guests to use
> native Infinband protocols at close to native speeds. Being
> paravirtualised they were live-migration cappable and many domU could
> share a single Infiniband adapter.
>
> As far as I am aware both of the projects are dead,

Ah, thanks for the info, sounds like there was some interesting ideas
here once upon a time. I'm not 100% sure that Xen-IB and SmartIO were
actually different projects,
http://nowlab.cse.ohio-state.edu/projects/xen/index.html seems to
suggest they are actually the same.

So I guess I stand corrected: the xen-smartio repo does contain some PV
IB drivers but since they are for 2.6.16 and for the Xen PV architecture
(not KVM) I don't know how much use they will be to the OP.

FWIW I also found http://wiki.xensource.com/xenwiki/XenSmartIO and
http://www.openfabrics.org/archives/nov2006sc/xen-ib-presentation.pdf
while further researching.

Ian.

>  however SR-IOV
> isn't long off for Infiniband, Mellanox is on target to push theirs
> upstream into OFED 1.6 due in November, Qlogic and Oracle/Sun will be
> quick to follow I would imagine.
> 
> SR-IOV will allow pretty similar functionality but less portable in
> terms of live-migrate etc.
> 
> It is worth noting here that you can't bridge IPoIB, IPoIB is not a
> layer 2 device and doesn't understand Ethernet. You can use EoIB
> (Ethernet over IB) or routing to provide network access to domUs.
> 
> Feel free to email me if you need any help with Infiniband and Xen.
> 
> Joseph.
> 
> On 27 May 2011 04:50, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> > On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
> >> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
> >> > Hi,
> >> >
> >> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
> >> > is like Xen-IB. Not just use IB.
> >>
> >> What is Xen-IB exactly?
> >>
> >
> > Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ?
> > I can't really think of other uses..
> >
> > -- Pasi
> >
> >
> >> > So, there is no toturial for Xen-IB,right?
> >>
> >> That's right, there is no documentation and anyone who would have been
> >> involved with this stuff is long gone.
> >>
> >> Ian.
> >>
> >> >
> >> >
> >> > Yi-Man
> >> >
> >> >
> >> >
> >> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
> >> >         On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
> >> >         > Hi,
> >> >         >
> >> >         > I have a project which is make KVM use InfiniBand.
> >> >         > However, since Xen can use InfiniBand through Xen-IB, I want
> >> >         to try to
> >> >         > make Xen use InfiniBand first.
> >> >         > Through that way, I think it may give me some idea about how
> >> >         to make
> >> >         > KVM use InfiniBand.
> >> >         > So, as you said, I just use "Updated defconfig to include
> >> >         infiniband
> >> >         > driver as modules for dom0/domU" to do?
> >> >         > And since I don't know how to I do first for Xen-IB, is
> >> >         there any
> >> >         > tutorial for Xen-IB?
> >> >         > Or does Xen build-in support IB currently?
> >> >
> >> >
> >> >         I'm not sure what IB support you expect Xen to have over and
> >> >         above the
> >> >         driver and utils support, which are functions of the dom0
> >> >         kernel and
> >> >         distro and not the hypervisor.
> >> >
> >> >         Have you got IB working on a native system? I expect that if
> >> >         you do that
> >> >         then Xen or KVM will be able to make use of it basically for
> >> >         free, i.e.
> >> >         configure a VIF on an IP-over-IB bridge or use a local
> >> >         IP-over-IB
> >> >         address in dom0 for migration etc etc.
> >> >
> >> >         Ian.
> >> >
> >> >
> >> >         >
> >> >         > Yi-Man
> >> >         >
> >> >         >
> >> >         >
> >> >         > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
> >> >         >         Please do not top post.
> >> >         >
> >> >         >         On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
> >> >         >         > Hi,
> >> >         >         >
> >> >         >         > There many file in
> >> >         >          http://xenbits.xensource.com/ext/xen-smartio/ ,
> >> >         >         > which should I use for Xen-IB?
> >> >         >         > Use the following three?
> >> >         >         > Updated defconfig to include infiniband driver as
> >> >         modules
> >> >         >         for
> >> >         >         > dom0/domU
> >> >         >         > Updating drivers/infiniband to latest. Should
> >> >         shrink size of
> >> >         >         > xen_ib.patch in
> >> >         >         > Add support for Infiniband into Xen
> >> >         >
> >> >         >
> >> >         >         Again, these changesets are >5 years old and have
> >> >         not been
> >> >         >         looked at or
> >> >         >         maintained since -- I am very sceptical that you
> >> >         will find
> >> >         >         anything of
> >> >         >         use on a modern Xen system in there.
> >> >         >
> >> >         >         I expect that IB support in the kernels we currently
> >> >         use for
> >> >         >         dom0 is
> >> >         >         vastly improved over what was available in the
> >> >         2.6.16 tree
> >> >         >         referenced in
> >> >         >         the above repository. Perhaps you don't even need
> >> >         anything
> >> >         >         special
> >> >         >         outside of what is available in the upstream dom0
> >> >         kernel.
> >> >         >
> >> >         >         Perhaps if you explain what you are actually trying
> >> >         to achieve
> >> >         >         then
> >> >         >         people will be able to better advise you?
> >> >         >
> >> >         >         Ian.
> >> >         >
> >> >         >
> >> >         >
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@lists.xensource.com
> >> http://lists.xensource.com/xen-devel
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >
> 
> 
> 

Re: Are there source codes for Xen-IB?

[maus]

[-- Attachment #1.1: Type: text/plain, Size: 4208 bytes --]

The Xen-IB I said means VMM-bypass I/O virtualization of InfiniBand on Xen.
At the beginning, the link:
http://xenbits.xensource.com/ext/xen-smartio.hgI found is for that.
However, it looks like Xen-IB(as I said in first line) doesn't exist and no
one would know how to build up it, right?


Yi-Man

2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>

> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
> > Hi,
> >
> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
> > is like Xen-IB. Not just use IB.
>
> What is Xen-IB exactly?
>
> > So, there is no toturial for Xen-IB,right?
>
> That's right, there is no documentation and anyone who would have been
> involved with this stuff is long gone.
>
> Ian.
>
> >
> >
> > Yi-Man
> >
> >
> >
> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
> >         On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
> >         > Hi,
> >         >
> >         > I have a project which is make KVM use InfiniBand.
> >         > However, since Xen can use InfiniBand through Xen-IB, I want
> >         to try to
> >         > make Xen use InfiniBand first.
> >         > Through that way, I think it may give me some idea about how
> >         to make
> >         > KVM use InfiniBand.
> >         > So, as you said, I just use "Updated defconfig to include
> >         infiniband
> >         > driver as modules for dom0/domU" to do?
> >         > And since I don't know how to I do first for Xen-IB, is
> >         there any
> >         > tutorial for Xen-IB?
> >         > Or does Xen build-in support IB currently?
> >
> >
> >         I'm not sure what IB support you expect Xen to have over and
> >         above the
> >         driver and utils support, which are functions of the dom0
> >         kernel and
> >         distro and not the hypervisor.
> >
> >         Have you got IB working on a native system? I expect that if
> >         you do that
> >         then Xen or KVM will be able to make use of it basically for
> >         free, i.e.
> >         configure a VIF on an IP-over-IB bridge or use a local
> >         IP-over-IB
> >         address in dom0 for migration etc etc.
> >
> >         Ian.
> >
> >
> >         >
> >         > Yi-Man
> >         >
> >         >
> >         >
> >         > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
> >         >         Please do not top post.
> >         >
> >         >         On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
> >         >         > Hi,
> >         >         >
> >         >         > There many file in
> >         >          http://xenbits.xensource.com/ext/xen-smartio/ ,
> >         >         > which should I use for Xen-IB?
> >         >         > Use the following three?
> >         >         > Updated defconfig to include infiniband driver as
> >         modules
> >         >         for
> >         >         > dom0/domU
> >         >         > Updating drivers/infiniband to latest. Should
> >         shrink size of
> >         >         > xen_ib.patch in
> >         >         > Add support for Infiniband into Xen
> >         >
> >         >
> >         >         Again, these changesets are >5 years old and have
> >         not been
> >         >         looked at or
> >         >         maintained since -- I am very sceptical that you
> >         will find
> >         >         anything of
> >         >         use on a modern Xen system in there.
> >         >
> >         >         I expect that IB support in the kernels we currently
> >         use for
> >         >         dom0 is
> >         >         vastly improved over what was available in the
> >         2.6.16 tree
> >         >         referenced in
> >         >         the above repository. Perhaps you don't even need
> >         anything
> >         >         special
> >         >         outside of what is available in the upstream dom0
> >         kernel.
> >         >
> >         >         Perhaps if you explain what you are actually trying
> >         to achieve
> >         >         then
> >         >         people will be able to better advise you?
> >         >
> >         >         Ian.
> >         >
> >         >
> >         >
> >
> >
> >
> >
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 5778 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel