* [PATCH 1/2] IB/core: Add support for enhanced atomic operations
@ 2010-02-01 15:59 Vladimir Sokolovsky
2010-02-11 0:51 ` Sean Hefty
0 siblings, 1 reply; 5+ messages in thread
From: Vladimir Sokolovsky @ 2010-02-01 15:59 UTC (permalink / raw)
To: Roland Dreier; +Cc: linux-rdma
This patchset adds support for the following enhanced atomic
operations:
- Masked atomic compare and swap
- Masked atomic fetch and add
These operations enable using a smaller amount of memory when using
multiple locks by using portions of a 64 bit value in an atomic operation.
- Add new IB_WR_ATOMIC_MASKED_CMP_AND_SWP and
IB_WR_ATOMIC_MASKED_FETCH_AND_ADD send opcodes that can be used to mark a
"masked atomic compare and swap" and "masked atomic fetch and add" work
request correspondingly.
- Add IB_DEVICE_MASKED_ATOMIC capability bit.
- Add mask fields to atomic struct of ib_send_wr
- Add new opcodes to ib_wc_opcode
Signed-off-by: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
---
include/rdma/ib_verbs.h | 9 ++++++++-
1 files changed, 8 insertions(+), 1 deletions(-)
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 09509ed..e2adeef 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -105,6 +105,7 @@ enum ib_device_cap_flags {
IB_DEVICE_UD_TSO = (1<<19),
IB_DEVICE_MEM_MGT_EXTENSIONS = (1<<21),
IB_DEVICE_BLOCK_MULTICAST_LOOPBACK = (1<<22),
+ IB_DEVICE_MASKED_ATOMIC = (1<<23),
};
enum ib_atomic_cap {
@@ -472,7 +473,9 @@ enum ib_wc_opcode {
* receive by testing (opcode & IB_WC_RECV).
*/
IB_WC_RECV = 1 << 7,
- IB_WC_RECV_RDMA_WITH_IMM
+ IB_WC_RECV_RDMA_WITH_IMM,
+ IB_WC_MASKED_COMP_SWAP,
+ IB_WC_MASKED_FETCH_ADD
};
enum ib_wc_flags {
@@ -689,6 +692,8 @@ enum ib_wr_opcode {
IB_WR_RDMA_READ_WITH_INV,
IB_WR_LOCAL_INV,
IB_WR_FAST_REG_MR,
+ IB_WR_ATOMIC_MASKED_CMP_AND_SWP,
+ IB_WR_ATOMIC_MASKED_FETCH_AND_ADD,
};
enum ib_send_flags {
@@ -732,6 +737,8 @@ struct ib_send_wr {
u64 compare_add;
u64 swap;
u32 rkey;
+ u64 compare_add_mask;
+ u64 swap_mask;
} atomic;
struct {
struct ib_ah *ah;
--
1.5.4.3
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 5+ messages in thread
* RE: [PATCH 1/2] IB/core: Add support for enhanced atomic operations
2010-02-01 15:59 [PATCH 1/2] IB/core: Add support for enhanced atomic operations Vladimir Sokolovsky
@ 2010-02-11 0:51 ` Sean Hefty
[not found] ` <96930957548D41E0B47FA465CAAD05E6-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
0 siblings, 1 reply; 5+ messages in thread
From: Sean Hefty @ 2010-02-11 0:51 UTC (permalink / raw)
To: 'Vladimir Sokolovsky', Roland Dreier; +Cc: linux-rdma
> enum ib_atomic_cap {
>@@ -472,7 +473,9 @@ enum ib_wc_opcode {
> * receive by testing (opcode & IB_WC_RECV).
> */
> IB_WC_RECV = 1 << 7,
>- IB_WC_RECV_RDMA_WITH_IMM
>+ IB_WC_RECV_RDMA_WITH_IMM,
>+ IB_WC_MASKED_COMP_SWAP,
>+ IB_WC_MASKED_FETCH_ADD
> };
This breaks the ability to check for a receive by testing opcode & IB_WC_RECV.
Is it even necessary to define new work completion opcodes?
> enum ib_wc_flags {
>@@ -689,6 +692,8 @@ enum ib_wr_opcode {
> IB_WR_RDMA_READ_WITH_INV,
> IB_WR_LOCAL_INV,
> IB_WR_FAST_REG_MR,
>+ IB_WR_ATOMIC_MASKED_CMP_AND_SWP,
>+ IB_WR_ATOMIC_MASKED_FETCH_AND_ADD,
> };
Just tossing this out as a possibility -- the mask itself could be used to
indicate the type of operation. Kernel clients would always set the mask for
all atomic operations.
Also, is there a kernel user for this functionality?
- Sean
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 1/2] IB/core: Add support for enhanced atomic operations
[not found] ` <96930957548D41E0B47FA465CAAD05E6-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
@ 2010-02-11 13:48 ` Vladimir Sokolovsky
2010-02-17 18:49 ` Andy Grover
1 sibling, 0 replies; 5+ messages in thread
From: Vladimir Sokolovsky @ 2010-02-11 13:48 UTC (permalink / raw)
To: Sean Hefty; +Cc: Roland Dreier, linux-rdma
Sean Hefty wrote:
>> enum ib_atomic_cap {
>> @@ -472,7 +473,9 @@ enum ib_wc_opcode {
>> * receive by testing (opcode & IB_WC_RECV).
>> */
>> IB_WC_RECV = 1 << 7,
>> - IB_WC_RECV_RDMA_WITH_IMM
>> + IB_WC_RECV_RDMA_WITH_IMM,
>> + IB_WC_MASKED_COMP_SWAP,
>> + IB_WC_MASKED_FETCH_ADD
>> };
>
> This breaks the ability to check for a receive by testing opcode & IB_WC_RECV.
Thanks Sean,
I'll send an updated patch with the fix.
> Is it even necessary to define new work completion opcodes?
>
>> enum ib_wc_flags {
>> @@ -689,6 +692,8 @@ enum ib_wr_opcode {
>> IB_WR_RDMA_READ_WITH_INV,
>> IB_WR_LOCAL_INV,
>> IB_WR_FAST_REG_MR,
>> + IB_WR_ATOMIC_MASKED_CMP_AND_SWP,
>> + IB_WR_ATOMIC_MASKED_FETCH_AND_ADD,
>> };
>
> Just tossing this out as a possibility -- the mask itself could be used to
> indicate the type of operation. Kernel clients would always set the mask for
> all atomic operations.
I don't have a strong opinion on this, but it looks to me more natural to use
new opcodes and this way it is backward compatible.
Regards,
Vladimir
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 1/2] IB/core: Add support for enhanced atomic operations
[not found] ` <96930957548D41E0B47FA465CAAD05E6-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
2010-02-11 13:48 ` Vladimir Sokolovsky
@ 2010-02-17 18:49 ` Andy Grover
[not found] ` <4B7C3A2C.6070202-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
1 sibling, 1 reply; 5+ messages in thread
From: Andy Grover @ 2010-02-17 18:49 UTC (permalink / raw)
To: Sean Hefty; +Cc: 'Vladimir Sokolovsky', Roland Dreier, linux-rdma
Sean Hefty wrote:
> Just tossing this out as a possibility -- the mask itself could be used to
> indicate the type of operation. Kernel clients would always set the mask for
> all atomic operations.
>
> Also, is there a kernel user for this functionality?
RDS would make use of this.
Regards -- Andy
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 1/2] IB/core: Add support for enhanced atomic operations
[not found] ` <4B7C3A2C.6070202-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
@ 2010-02-17 23:56 ` Richard Frank
0 siblings, 0 replies; 5+ messages in thread
From: Richard Frank @ 2010-02-17 23:56 UTC (permalink / raw)
To: Andy Grover
Cc: Sean Hefty, 'Vladimir Sokolovsky', Roland Dreier, linux-rdma
Oracle is planning to use these masked atomic ops..
Andy Grover wrote:
> Sean Hefty wrote:
>
>
>> Just tossing this out as a possibility -- the mask itself could be used to
>> indicate the type of operation. Kernel clients would always set the mask for
>> all atomic operations.
>>
>> Also, is there a kernel user for this functionality?
>>
>
> RDS would make use of this.
>
> Regards -- Andy
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2010-02-17 23:56 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-02-01 15:59 [PATCH 1/2] IB/core: Add support for enhanced atomic operations Vladimir Sokolovsky
2010-02-11 0:51 ` Sean Hefty
[not found] ` <96930957548D41E0B47FA465CAAD05E6-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
2010-02-11 13:48 ` Vladimir Sokolovsky
2010-02-17 18:49 ` Andy Grover
[not found] ` <4B7C3A2C.6070202-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2010-02-17 23:56 ` Richard Frank
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.