* Re: Are there source codes for Xen-IB?
2011-05-26 14:10 ` Ian Campbell
@ 2011-05-26 18:50 ` Pasi Kärkkäinen
2011-05-28 18:15 ` Joseph Glanville
2011-05-29 9:08 ` Muli Ben-Yehuda
2011-06-01 6:25 ` maus
2 siblings, 1 reply; 16+ messages in thread
From: Pasi Kärkkäinen @ 2011-05-26 18:50 UTC (permalink / raw)
To: Ian Campbell; +Cc: maus, xen-devel, Fajar A. Nugraha
On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
> > Hi,
> >
> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
> > is like Xen-IB. Not just use IB.
>
> What is Xen-IB exactly?
>
Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ?
I can't really think of other uses..
-- Pasi
> > So, there is no toturial for Xen-IB,right?
>
> That's right, there is no documentation and anyone who would have been
> involved with this stuff is long gone.
>
> Ian.
>
> >
> >
> > Yi-Man
> >
> >
> >
> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
> > On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
> > > Hi,
> > >
> > > I have a project which is make KVM use InfiniBand.
> > > However, since Xen can use InfiniBand through Xen-IB, I want
> > to try to
> > > make Xen use InfiniBand first.
> > > Through that way, I think it may give me some idea about how
> > to make
> > > KVM use InfiniBand.
> > > So, as you said, I just use "Updated defconfig to include
> > infiniband
> > > driver as modules for dom0/domU" to do?
> > > And since I don't know how to I do first for Xen-IB, is
> > there any
> > > tutorial for Xen-IB?
> > > Or does Xen build-in support IB currently?
> >
> >
> > I'm not sure what IB support you expect Xen to have over and
> > above the
> > driver and utils support, which are functions of the dom0
> > kernel and
> > distro and not the hypervisor.
> >
> > Have you got IB working on a native system? I expect that if
> > you do that
> > then Xen or KVM will be able to make use of it basically for
> > free, i.e.
> > configure a VIF on an IP-over-IB bridge or use a local
> > IP-over-IB
> > address in dom0 for migration etc etc.
> >
> > Ian.
> >
> >
> > >
> > > Yi-Man
> > >
> > >
> > >
> > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
> > > Please do not top post.
> > >
> > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
> > > > Hi,
> > > >
> > > > There many file in
> > > http://xenbits.xensource.com/ext/xen-smartio/ ,
> > > > which should I use for Xen-IB?
> > > > Use the following three?
> > > > Updated defconfig to include infiniband driver as
> > modules
> > > for
> > > > dom0/domU
> > > > Updating drivers/infiniband to latest. Should
> > shrink size of
> > > > xen_ib.patch in
> > > > Add support for Infiniband into Xen
> > >
> > >
> > > Again, these changesets are >5 years old and have
> > not been
> > > looked at or
> > > maintained since -- I am very sceptical that you
> > will find
> > > anything of
> > > use on a modern Xen system in there.
> > >
> > > I expect that IB support in the kernels we currently
> > use for
> > > dom0 is
> > > vastly improved over what was available in the
> > 2.6.16 tree
> > > referenced in
> > > the above repository. Perhaps you don't even need
> > anything
> > > special
> > > outside of what is available in the upstream dom0
> > kernel.
> > >
> > > Perhaps if you explain what you are actually trying
> > to achieve
> > > then
> > > people will be able to better advise you?
> > >
> > > Ian.
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Are there source codes for Xen-IB?
2011-05-26 18:50 ` Pasi Kärkkäinen
@ 2011-05-28 18:15 ` Joseph Glanville
2011-05-30 2:48 ` Joseph Glanville
2011-05-31 9:04 ` Ian Campbell
0 siblings, 2 replies; 16+ messages in thread
From: Joseph Glanville @ 2011-05-28 18:15 UTC (permalink / raw)
To: Pasi Kärkkäinen; +Cc: Ian Campbell, maus, xen-devel, Fajar A. Nugraha
Hi,
I happen to have done a huge amount of research into this as our
infrastructure is all based on converged Infiniband.
Xen-IB and SmartIO were two seperate attempts at building a
paravirtualised Infinband driver for Xen that allowed guests to use
native Infinband protocols at close to native speeds. Being
paravirtualised they were live-migration cappable and many domU could
share a single Infiniband adapter.
As far as I am aware both of the projects are dead, however SR-IOV
isn't long off for Infiniband, Mellanox is on target to push theirs
upstream into OFED 1.6 due in November, Qlogic and Oracle/Sun will be
quick to follow I would imagine.
SR-IOV will allow pretty similar functionality but less portable in
terms of live-migrate etc.
It is worth noting here that you can't bridge IPoIB, IPoIB is not a
layer 2 device and doesn't understand Ethernet. You can use EoIB
(Ethernet over IB) or routing to provide network access to domUs.
Feel free to email me if you need any help with Infiniband and Xen.
Joseph.
On 27 May 2011 04:50, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
>> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
>> > Hi,
>> >
>> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
>> > is like Xen-IB. Not just use IB.
>>
>> What is Xen-IB exactly?
>>
>
> Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ?
> I can't really think of other uses..
>
> -- Pasi
>
>
>> > So, there is no toturial for Xen-IB,right?
>>
>> That's right, there is no documentation and anyone who would have been
>> involved with this stuff is long gone.
>>
>> Ian.
>>
>> >
>> >
>> > Yi-Man
>> >
>> >
>> >
>> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
>> > On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
>> > > Hi,
>> > >
>> > > I have a project which is make KVM use InfiniBand.
>> > > However, since Xen can use InfiniBand through Xen-IB, I want
>> > to try to
>> > > make Xen use InfiniBand first.
>> > > Through that way, I think it may give me some idea about how
>> > to make
>> > > KVM use InfiniBand.
>> > > So, as you said, I just use "Updated defconfig to include
>> > infiniband
>> > > driver as modules for dom0/domU" to do?
>> > > And since I don't know how to I do first for Xen-IB, is
>> > there any
>> > > tutorial for Xen-IB?
>> > > Or does Xen build-in support IB currently?
>> >
>> >
>> > I'm not sure what IB support you expect Xen to have over and
>> > above the
>> > driver and utils support, which are functions of the dom0
>> > kernel and
>> > distro and not the hypervisor.
>> >
>> > Have you got IB working on a native system? I expect that if
>> > you do that
>> > then Xen or KVM will be able to make use of it basically for
>> > free, i.e.
>> > configure a VIF on an IP-over-IB bridge or use a local
>> > IP-over-IB
>> > address in dom0 for migration etc etc.
>> >
>> > Ian.
>> >
>> >
>> > >
>> > > Yi-Man
>> > >
>> > >
>> > >
>> > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
>> > > Please do not top post.
>> > >
>> > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
>> > > > Hi,
>> > > >
>> > > > There many file in
>> > > http://xenbits.xensource.com/ext/xen-smartio/ ,
>> > > > which should I use for Xen-IB?
>> > > > Use the following three?
>> > > > Updated defconfig to include infiniband driver as
>> > modules
>> > > for
>> > > > dom0/domU
>> > > > Updating drivers/infiniband to latest. Should
>> > shrink size of
>> > > > xen_ib.patch in
>> > > > Add support for Infiniband into Xen
>> > >
>> > >
>> > > Again, these changesets are >5 years old and have
>> > not been
>> > > looked at or
>> > > maintained since -- I am very sceptical that you
>> > will find
>> > > anything of
>> > > use on a modern Xen system in there.
>> > >
>> > > I expect that IB support in the kernels we currently
>> > use for
>> > > dom0 is
>> > > vastly improved over what was available in the
>> > 2.6.16 tree
>> > > referenced in
>> > > the above repository. Perhaps you don't even need
>> > anything
>> > > special
>> > > outside of what is available in the upstream dom0
>> > kernel.
>> > >
>> > > Perhaps if you explain what you are actually trying
>> > to achieve
>> > > then
>> > > people will be able to better advise you?
>> > >
>> > > Ian.
>> > >
>> > >
>> > >
>> >
>> >
>> >
>> >
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
--
Kind regards,
Joseph.
Founder | Director
Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56
99 52 | Mobile: 0428 754 846
^ permalink raw reply [flat|nested] 16+ messages in thread
* Are there source codes for Xen-IB?
2011-05-28 18:15 ` Joseph Glanville
@ 2011-05-30 2:48 ` Joseph Glanville
2011-05-31 9:04 ` Ian Campbell
1 sibling, 0 replies; 16+ messages in thread
From: Joseph Glanville @ 2011-05-30 2:48 UTC (permalink / raw)
To: xen-devel
Hi,
I happen to have done a huge amount of research into this as our
infrastructure is all based on converged Infiniband.
Xen-IB and SmartIO were two seperate attempts at building a
paravirtualised Infinband driver for Xen that allowed guests to use
native Infinband protocols at close to native speeds. Being
paravirtualised they were live-migration cappable and many domU could
share a single Infiniband adapter.
As far as I am aware both of the projects are dead, however SR-IOV
isn't long off for Infiniband, Mellanox is on target to push theirs
upstream into OFED 1.6 due in November, Qlogic and Oracle/Sun will be
quick to follow I would imagine.
SR-IOV will allow pretty similar functionality but less portable in
terms of live-migrate etc.
It is worth noting here that you can't bridge IPoIB, IPoIB is not a
layer 2 device and doesn't understand Ethernet. You can use EoIB
(Ethernet over IB) or routing to provide network access to domUs.
Feel free to email me if you need any help with Infiniband and Xen.
Joseph.
On 27 May 2011 04:50, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
>> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
>> > Hi,
>> >
>> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
>> > is like Xen-IB. Not just use IB.
>>
>> What is Xen-IB exactly?
>>
>
> Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ?
> I can't really think of other uses..
>
> -- Pasi
>
>
>> > So, there is no toturial for Xen-IB,right?
>>
>> That's right, there is no documentation and anyone who would have been
>> involved with this stuff is long gone.
>>
>> Ian.
>>
>> >
>> >
>> > Yi-Man
>> >
>> >
>> >
>> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
>> > On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
>> > > Hi,
>> > >
>> > > I have a project which is make KVM use InfiniBand.
>> > > However, since Xen can use InfiniBand through Xen-IB, I want
>> > to try to
>> > > make Xen use InfiniBand first.
>> > > Through that way, I think it may give me some idea about how
>> > to make
>> > > KVM use InfiniBand.
>> > > So, as you said, I just use "Updated defconfig to include
>> > infiniband
>> > > driver as modules for dom0/domU" to do?
>> > > And since I don't know how to I do first for Xen-IB, is
>> > there any
>> > > tutorial for Xen-IB?
>> > > Or does Xen build-in support IB currently?
>> >
>> >
>> > I'm not sure what IB support you expect Xen to have over and
>> > above the
>> > driver and utils support, which are functions of the dom0
>> > kernel and
>> > distro and not the hypervisor.
>> >
>> > Have you got IB working on a native system? I expect that if
>> > you do that
>> > then Xen or KVM will be able to make use of it basically for
>> > free, i.e.
>> > configure a VIF on an IP-over-IB bridge or use a local
>> > IP-over-IB
>> > address in dom0 for migration etc etc.
>> >
>> > Ian.
>> >
>> >
>> > >
>> > > Yi-Man
>> > >
>> > >
>> > >
>> > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
>> > > Please do not top post.
>> > >
>> > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
>> > > > Hi,
>> > > >
>> > > > There many file in
>> > > http://xenbits.xensource.com/ext/xen-smartio/ ,
>> > > > which should I use for Xen-IB?
>> > > > Use the following three?
>> > > > Updated defconfig to include infiniband driver as
>> > modules
>> > > for
>> > > > dom0/domU
>> > > > Updating drivers/infiniband to latest. Should
>> > shrink size of
>> > > > xen_ib.patch in
>> > > > Add support for Infiniband into Xen
>> > >
>> > >
>> > > Again, these changesets are >5 years old and have
>> > not been
>> > > looked at or
>> > > maintained since -- I am very sceptical that you
>> > will find
>> > > anything of
>> > > use on a modern Xen system in there.
>> > >
>> > > I expect that IB support in the kernels we currently
>> > use for
>> > > dom0 is
>> > > vastly improved over what was available in the
>> > 2.6.16 tree
>> > > referenced in
>> > > the above repository. Perhaps you don't even need
>> > anything
>> > > special
>> > > outside of what is available in the upstream dom0
>> > kernel.
>> > >
>> > > Perhaps if you explain what you are actually trying
>> > to achieve
>> > > then
>> > > people will be able to better advise you?
>> > >
>> > > Ian.
>> > >
>> > >
>> > >
>> >
>> >
>> >
>> >
>>
>>
>>
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@lists.xensource.com
>> http://lists.xensource.com/xen-devel
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
--
Kind regards,
Joseph.
Founder | Director
Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56
99 52 | Mobile: 0428 754 846
--
Kind regards,
Joseph.
Founder | Director
Orion Virtualisation Solutions | www.orionvm.com.au | Phone: 1300 56
99 52 | Mobile: 0428 754 846
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Are there source codes for Xen-IB?
2011-05-28 18:15 ` Joseph Glanville
2011-05-30 2:48 ` Joseph Glanville
@ 2011-05-31 9:04 ` Ian Campbell
1 sibling, 0 replies; 16+ messages in thread
From: Ian Campbell @ 2011-05-31 9:04 UTC (permalink / raw)
To: Joseph Glanville; +Cc: maus, xen-devel, Fajar A. Nugraha
On Sat, 2011-05-28 at 19:15 +0100, Joseph Glanville wrote:
> Hi,
>
> I happen to have done a huge amount of research into this as our
> infrastructure is all based on converged Infiniband.
>
> Xen-IB and SmartIO were two seperate attempts at building a
> paravirtualised Infinband driver for Xen that allowed guests to use
> native Infinband protocols at close to native speeds. Being
> paravirtualised they were live-migration cappable and many domU could
> share a single Infiniband adapter.
>
> As far as I am aware both of the projects are dead,
Ah, thanks for the info, sounds like there was some interesting ideas
here once upon a time. I'm not 100% sure that Xen-IB and SmartIO were
actually different projects,
http://nowlab.cse.ohio-state.edu/projects/xen/index.html seems to
suggest they are actually the same.
So I guess I stand corrected: the xen-smartio repo does contain some PV
IB drivers but since they are for 2.6.16 and for the Xen PV architecture
(not KVM) I don't know how much use they will be to the OP.
FWIW I also found http://wiki.xensource.com/xenwiki/XenSmartIO and
http://www.openfabrics.org/archives/nov2006sc/xen-ib-presentation.pdf
while further researching.
Ian.
> however SR-IOV
> isn't long off for Infiniband, Mellanox is on target to push theirs
> upstream into OFED 1.6 due in November, Qlogic and Oracle/Sun will be
> quick to follow I would imagine.
>
> SR-IOV will allow pretty similar functionality but less portable in
> terms of live-migrate etc.
>
> It is worth noting here that you can't bridge IPoIB, IPoIB is not a
> layer 2 device and doesn't understand Ethernet. You can use EoIB
> (Ethernet over IB) or routing to provide network access to domUs.
>
> Feel free to email me if you need any help with Infiniband and Xen.
>
> Joseph.
>
> On 27 May 2011 04:50, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> > On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
> >> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
> >> > Hi,
> >> >
> >> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
> >> > is like Xen-IB. Not just use IB.
> >>
> >> What is Xen-IB exactly?
> >>
> >
> > Could "Xen-IB" mean live migration over Infiniband RDMA (Remote DMA) ?
> > I can't really think of other uses..
> >
> > -- Pasi
> >
> >
> >> > So, there is no toturial for Xen-IB,right?
> >>
> >> That's right, there is no documentation and anyone who would have been
> >> involved with this stuff is long gone.
> >>
> >> Ian.
> >>
> >> >
> >> >
> >> > Yi-Man
> >> >
> >> >
> >> >
> >> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
> >> > On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
> >> > > Hi,
> >> > >
> >> > > I have a project which is make KVM use InfiniBand.
> >> > > However, since Xen can use InfiniBand through Xen-IB, I want
> >> > to try to
> >> > > make Xen use InfiniBand first.
> >> > > Through that way, I think it may give me some idea about how
> >> > to make
> >> > > KVM use InfiniBand.
> >> > > So, as you said, I just use "Updated defconfig to include
> >> > infiniband
> >> > > driver as modules for dom0/domU" to do?
> >> > > And since I don't know how to I do first for Xen-IB, is
> >> > there any
> >> > > tutorial for Xen-IB?
> >> > > Or does Xen build-in support IB currently?
> >> >
> >> >
> >> > I'm not sure what IB support you expect Xen to have over and
> >> > above the
> >> > driver and utils support, which are functions of the dom0
> >> > kernel and
> >> > distro and not the hypervisor.
> >> >
> >> > Have you got IB working on a native system? I expect that if
> >> > you do that
> >> > then Xen or KVM will be able to make use of it basically for
> >> > free, i.e.
> >> > configure a VIF on an IP-over-IB bridge or use a local
> >> > IP-over-IB
> >> > address in dom0 for migration etc etc.
> >> >
> >> > Ian.
> >> >
> >> >
> >> > >
> >> > > Yi-Man
> >> > >
> >> > >
> >> > >
> >> > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
> >> > > Please do not top post.
> >> > >
> >> > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
> >> > > > Hi,
> >> > > >
> >> > > > There many file in
> >> > > http://xenbits.xensource.com/ext/xen-smartio/ ,
> >> > > > which should I use for Xen-IB?
> >> > > > Use the following three?
> >> > > > Updated defconfig to include infiniband driver as
> >> > modules
> >> > > for
> >> > > > dom0/domU
> >> > > > Updating drivers/infiniband to latest. Should
> >> > shrink size of
> >> > > > xen_ib.patch in
> >> > > > Add support for Infiniband into Xen
> >> > >
> >> > >
> >> > > Again, these changesets are >5 years old and have
> >> > not been
> >> > > looked at or
> >> > > maintained since -- I am very sceptical that you
> >> > will find
> >> > > anything of
> >> > > use on a modern Xen system in there.
> >> > >
> >> > > I expect that IB support in the kernels we currently
> >> > use for
> >> > > dom0 is
> >> > > vastly improved over what was available in the
> >> > 2.6.16 tree
> >> > > referenced in
> >> > > the above repository. Perhaps you don't even need
> >> > anything
> >> > > special
> >> > > outside of what is available in the upstream dom0
> >> > kernel.
> >> > >
> >> > > Perhaps if you explain what you are actually trying
> >> > to achieve
> >> > > then
> >> > > people will be able to better advise you?
> >> > >
> >> > > Ian.
> >> > >
> >> > >
> >> > >
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >>
> >> _______________________________________________
> >> Xen-devel mailing list
> >> Xen-devel@lists.xensource.com
> >> http://lists.xensource.com/xen-devel
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> >
>
>
>
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Are there source codes for Xen-IB?
2011-05-26 14:10 ` Ian Campbell
2011-05-26 18:50 ` Pasi Kärkkäinen
@ 2011-05-29 9:08 ` Muli Ben-Yehuda
2011-05-31 9:02 ` Ian Campbell
2011-06-01 6:25 ` maus
2 siblings, 1 reply; 16+ messages in thread
From: Muli Ben-Yehuda @ 2011-05-29 9:08 UTC (permalink / raw)
To: Ian Campbell; +Cc: maus, xen-devel, Fajar A. Nugraha
On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
> > Hi,
> >
> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
> > is like Xen-IB. Not just use IB.
>
> What is Xen-IB exactly?
OS and hypervisor bypass, i.e., direct access from a user application
running in a domU to the Infiniband adapter (HCA), bypassing both the
domU kernel and Xen for frequent data-path operations. Configuration
and privileged operations still go through the kernel and the
hypervisor.
> > So, there is no toturial for Xen-IB,right?
>
> That's right, there is no documentation and anyone who would have
> been involved with this stuff is long gone.
Actually, there is a very nice paper describing the approach:
"High Performance VMM-Bypass I/O in Virtual Machines", Jiuxing
Liu, Wei Huang, Bulent Abali, Dhabaleswar K. Panda, USENIX ATC '06
Cheers,
Muli
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Are there source codes for Xen-IB?
2011-05-29 9:08 ` Muli Ben-Yehuda
@ 2011-05-31 9:02 ` Ian Campbell
0 siblings, 0 replies; 16+ messages in thread
From: Ian Campbell @ 2011-05-31 9:02 UTC (permalink / raw)
To: Muli Ben-Yehuda; +Cc: maus, xen-devel, Fajar A. Nugraha
On Sun, 2011-05-29 at 10:08 +0100, Muli Ben-Yehuda wrote:
> On Thu, May 26, 2011 at 03:10:40PM +0100, Ian Campbell wrote:
> > On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
> > > Hi,
> > >
> > > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
> > > is like Xen-IB. Not just use IB.
> >
> > What is Xen-IB exactly?
>
> OS and hypervisor bypass, i.e., direct access from a user application
> running in a domU to the Infiniband adapter (HCA), bypassing both the
> domU kernel and Xen for frequent data-path operations. Configuration
> and privileged operations still go through the kernel and the
> hypervisor.
>
> > > So, there is no toturial for Xen-IB,right?
> >
> > That's right, there is no documentation and anyone who would have
> > been involved with this stuff is long gone.
>
> Actually, there is a very nice paper describing the approach:
>
> "High Performance VMM-Bypass I/O in Virtual Machines", Jiuxing
> Liu, Wei Huang, Bulent Abali, Dhabaleswar K. Panda, USENIX ATC '06
Thanks for the reference.
Are any of these folks still active? It seems like they could be far
more helpful to the OP than anyone here (certainly more useful than
me ;-))...
Ian.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Are there source codes for Xen-IB?
2011-05-26 14:10 ` Ian Campbell
2011-05-26 18:50 ` Pasi Kärkkäinen
2011-05-29 9:08 ` Muli Ben-Yehuda
@ 2011-06-01 6:25 ` maus
2 siblings, 0 replies; 16+ messages in thread
From: maus @ 2011-06-01 6:25 UTC (permalink / raw)
To: Ian Campbell; +Cc: xen-devel, Fajar A. Nugraha
[-- Attachment #1.1: Type: text/plain, Size: 4208 bytes --]
The Xen-IB I said means VMM-bypass I/O virtualization of InfiniBand on Xen.
At the beginning, the link:
http://xenbits.xensource.com/ext/xen-smartio.hgI found is for that.
However, it looks like Xen-IB(as I said in first line) doesn't exist and no
one would know how to build up it, right?
Yi-Man
2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
> On Thu, 2011-05-26 at 15:04 +0100, maus wrote:
> > Hi,
> >
> > I know use IP-over-IB can let KVM or Xen use IB, but what I have to do
> > is like Xen-IB. Not just use IB.
>
> What is Xen-IB exactly?
>
> > So, there is no toturial for Xen-IB,right?
>
> That's right, there is no documentation and anyone who would have been
> involved with this stuff is long gone.
>
> Ian.
>
> >
> >
> > Yi-Man
> >
> >
> >
> > 2011/5/26 Ian Campbell <Ian.Campbell@eu.citrix.com>
> > On Wed, 2011-05-25 at 14:40 +0100, maus wrote:
> > > Hi,
> > >
> > > I have a project which is make KVM use InfiniBand.
> > > However, since Xen can use InfiniBand through Xen-IB, I want
> > to try to
> > > make Xen use InfiniBand first.
> > > Through that way, I think it may give me some idea about how
> > to make
> > > KVM use InfiniBand.
> > > So, as you said, I just use "Updated defconfig to include
> > infiniband
> > > driver as modules for dom0/domU" to do?
> > > And since I don't know how to I do first for Xen-IB, is
> > there any
> > > tutorial for Xen-IB?
> > > Or does Xen build-in support IB currently?
> >
> >
> > I'm not sure what IB support you expect Xen to have over and
> > above the
> > driver and utils support, which are functions of the dom0
> > kernel and
> > distro and not the hypervisor.
> >
> > Have you got IB working on a native system? I expect that if
> > you do that
> > then Xen or KVM will be able to make use of it basically for
> > free, i.e.
> > configure a VIF on an IP-over-IB bridge or use a local
> > IP-over-IB
> > address in dom0 for migration etc etc.
> >
> > Ian.
> >
> >
> > >
> > > Yi-Man
> > >
> > >
> > >
> > > 2011/5/24 Ian Campbell <Ian.Campbell@eu.citrix.com>
> > > Please do not top post.
> > >
> > > On Tue, 2011-05-24 at 11:07 +0100, maus wrote:
> > > > Hi,
> > > >
> > > > There many file in
> > > http://xenbits.xensource.com/ext/xen-smartio/ ,
> > > > which should I use for Xen-IB?
> > > > Use the following three?
> > > > Updated defconfig to include infiniband driver as
> > modules
> > > for
> > > > dom0/domU
> > > > Updating drivers/infiniband to latest. Should
> > shrink size of
> > > > xen_ib.patch in
> > > > Add support for Infiniband into Xen
> > >
> > >
> > > Again, these changesets are >5 years old and have
> > not been
> > > looked at or
> > > maintained since -- I am very sceptical that you
> > will find
> > > anything of
> > > use on a modern Xen system in there.
> > >
> > > I expect that IB support in the kernels we currently
> > use for
> > > dom0 is
> > > vastly improved over what was available in the
> > 2.6.16 tree
> > > referenced in
> > > the above repository. Perhaps you don't even need
> > anything
> > > special
> > > outside of what is available in the upstream dom0
> > kernel.
> > >
> > > Perhaps if you explain what you are actually trying
> > to achieve
> > > then
> > > people will be able to better advise you?
> > >
> > > Ian.
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
[-- Attachment #1.2: Type: text/html, Size: 5778 bytes --]
[-- Attachment #2: Type: text/plain, Size: 138 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread