size of queue pair number

size of queue pair number

[Ursula Braun]

Hi,

we want to make use of Infiniband communication and I am working on the
Linux part. The peer can be a non-Linux system. Now we see a mismatch in
the size of the queue pair number in Linux (32 bits) and the Infiniband
Architecture Specification (24 bits). Will the additional 8 highest
order bits ever be used in Linux? And if yes, for which purpose? Is the
maximum queue pair number 0xFFFFFF? 

Kind regards, Ursula Braun, IBM Germany


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: size of queue pair number

[Roland Dreier]

On Thu, May 24, 2012 at 1:07 AM, Ursula Braun <ubraun-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> wrote:
> we want to make use of Infiniband communication and I am working on the
> Linux part. The peer can be a non-Linux system. Now we see a mismatch in
> the size of the queue pair number in Linux (32 bits) and the Infiniband
> Architecture Specification (24 bits). Will the additional 8 highest
> order bits ever be used in Linux? And if yes, for which purpose? Is the
> maximum queue pair number 0xFFFFFF?

The reason the queue pair number type is 32 bits in the Linux stack is
that there is no u24 or uint24_t and very few processors have support
for 24-bit integer types :)

The queue pair numbers are allocated by the hardware drivers, and
modulo bugs they should always return a QPN that complies with the IB
architecture.  So in other words, yes, you can safely assume that QPNs
are in the range 0 ... 0xffffff.

 - R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: size of queue pair number

[Ursula Braun]

On Thu, 2012-05-24 at 01:41 -0700, Roland Dreier wrote:
> The reason the queue pair number type is 32 bits in the Linux stack is
> that there is no u24 or uint24_t and very few processors have support
> for 24-bit integer types :)
> 
> The queue pair numbers are allocated by the hardware drivers, and
> modulo bugs they should always return a QPN that complies with the IB
> architecture.  So in other words, yes, you can safely assume that QPNs
> are in the range 0 ... 0xffffff.
> 
Thanks for your good and quick help. May I ask another question about
zero based virtual addressing: The Infiniband Architecture Specification
talks about specifying the "type of VA" when registering a memory
region. Checking the ib_verbs code in the Linux kernel I cannot find an
equivalent for this. Is zero based virtual addressing available in the
infiniband code of the Linux kernel?

Kind regards, Ursula Braun, IBM Germany

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: size of queue pair number

[Roland Dreier]

> Thanks for your good and quick help. May I ask another question about
> zero based virtual addressing: The Infiniband Architecture Specification
> talks about specifying the "type of VA" when registering a memory
> region. Checking the ib_verbs code in the Linux kernel I cannot find an
> equivalent for this. Is zero based virtual addressing available in the
> infiniband code of the Linux kernel?

No, I don't believe we ever implemented it.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: size of queue pair number

[Ursula Braun]

On Fri, 2012-05-25 at 09:01 -0700, Roland Dreier wrote:
> > Thanks for your good and quick help. May I ask another question about
> > zero based virtual addressing: The Infiniband Architecture Specification
> > talks about specifying the "type of VA" when registering a memory
> > region. Checking the ib_verbs code in the Linux kernel I cannot find an
> > equivalent for this. Is zero based virtual addressing available in the
> > infiniband code of the Linux kernel?
> 
> No, I don't believe we ever implemented it.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
If zero based virtual addressing has never been implemented within
Linux, does this mean an ib_post_send() with opcode IB_WR_RDMA_WRITE
requires specification of both - rdma.rkey AND rdma.remote_addr?
And thus means there is a need to exchange both somehow in advance?

Kind regards, Ursula Braun, IBM Germany

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: size of queue pair number

[Roland Dreier]

On Mon, Jun 11, 2012 at 8:10 AM, Ursula Braun <ubraun-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> wrote:
> If zero based virtual addressing has never been implemented within
> Linux, does this mean an ib_post_send() with opcode IB_WR_RDMA_WRITE
> requires specification of both - rdma.rkey AND rdma.remote_addr?
> And thus means there is a need to exchange both somehow in advance?

Yes, correct.

 - R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html