All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 1/2] IB/core: Add support for enhanced atomic operations
@ 2010-02-01 15:59 Vladimir Sokolovsky
  2010-02-11  0:51 ` Sean Hefty
  0 siblings, 1 reply; 5+ messages in thread
From: Vladimir Sokolovsky @ 2010-02-01 15:59 UTC (permalink / raw)
  To: Roland Dreier; +Cc: linux-rdma

This patchset adds support for the following enhanced atomic
operations:
- Masked atomic compare and swap
- Masked atomic fetch and add

These operations enable using a smaller amount of memory when using
multiple locks by using portions of a 64 bit value in an atomic operation.

- Add new IB_WR_ATOMIC_MASKED_CMP_AND_SWP and
  IB_WR_ATOMIC_MASKED_FETCH_AND_ADD send opcodes that can be used to mark a
  "masked atomic compare and swap" and "masked atomic fetch and add" work
  request correspondingly.
- Add IB_DEVICE_MASKED_ATOMIC capability bit.
- Add mask fields to atomic struct of ib_send_wr
- Add new opcodes to ib_wc_opcode

Signed-off-by: Vladimir Sokolovsky <vlad-VPRAkNaXOzVS1MOuV/RT9w@public.gmane.org>
---
 include/rdma/ib_verbs.h |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 09509ed..e2adeef 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -105,6 +105,7 @@ enum ib_device_cap_flags {
 	IB_DEVICE_UD_TSO		= (1<<19),
 	IB_DEVICE_MEM_MGT_EXTENSIONS	= (1<<21),
 	IB_DEVICE_BLOCK_MULTICAST_LOOPBACK = (1<<22),
+	IB_DEVICE_MASKED_ATOMIC		= (1<<23),
 };
 
 enum ib_atomic_cap {
@@ -472,7 +473,9 @@ enum ib_wc_opcode {
  * receive by testing (opcode & IB_WC_RECV).
  */
 	IB_WC_RECV			= 1 << 7,
-	IB_WC_RECV_RDMA_WITH_IMM
+	IB_WC_RECV_RDMA_WITH_IMM,
+	IB_WC_MASKED_COMP_SWAP,
+	IB_WC_MASKED_FETCH_ADD
 };
 
 enum ib_wc_flags {
@@ -689,6 +692,8 @@ enum ib_wr_opcode {
 	IB_WR_RDMA_READ_WITH_INV,
 	IB_WR_LOCAL_INV,
 	IB_WR_FAST_REG_MR,
+	IB_WR_ATOMIC_MASKED_CMP_AND_SWP,
+	IB_WR_ATOMIC_MASKED_FETCH_AND_ADD,
 };
 
 enum ib_send_flags {
@@ -732,6 +737,8 @@ struct ib_send_wr {
 			u64	compare_add;
 			u64	swap;
 			u32	rkey;
+			u64	compare_add_mask;
+			u64	swap_mask;
 		} atomic;
 		struct {
 			struct ib_ah *ah;
-- 
1.5.4.3

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply related	[flat|nested] 5+ messages in thread

* RE: [PATCH 1/2] IB/core: Add support for enhanced atomic operations
  2010-02-01 15:59 [PATCH 1/2] IB/core: Add support for enhanced atomic operations Vladimir Sokolovsky
@ 2010-02-11  0:51 ` Sean Hefty
       [not found]   ` <96930957548D41E0B47FA465CAAD05E6-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
  0 siblings, 1 reply; 5+ messages in thread
From: Sean Hefty @ 2010-02-11  0:51 UTC (permalink / raw)
  To: 'Vladimir Sokolovsky', Roland Dreier; +Cc: linux-rdma

> enum ib_atomic_cap {
>@@ -472,7 +473,9 @@ enum ib_wc_opcode {
>  * receive by testing (opcode & IB_WC_RECV).
>  */
> 	IB_WC_RECV			= 1 << 7,
>-	IB_WC_RECV_RDMA_WITH_IMM
>+	IB_WC_RECV_RDMA_WITH_IMM,
>+	IB_WC_MASKED_COMP_SWAP,
>+	IB_WC_MASKED_FETCH_ADD
> };

This breaks the ability to check for a receive by testing opcode & IB_WC_RECV.
Is it even necessary to define new work completion opcodes?

> enum ib_wc_flags {
>@@ -689,6 +692,8 @@ enum ib_wr_opcode {
> 	IB_WR_RDMA_READ_WITH_INV,
> 	IB_WR_LOCAL_INV,
> 	IB_WR_FAST_REG_MR,
>+	IB_WR_ATOMIC_MASKED_CMP_AND_SWP,
>+	IB_WR_ATOMIC_MASKED_FETCH_AND_ADD,
> };

Just tossing this out as a possibility -- the mask itself could be used to
indicate the type of operation.  Kernel clients would always set the mask for
all atomic operations.

Also, is there a kernel user for this functionality?

- Sean

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH 1/2] IB/core: Add support for enhanced atomic operations
       [not found]   ` <96930957548D41E0B47FA465CAAD05E6-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
@ 2010-02-11 13:48     ` Vladimir Sokolovsky
  2010-02-17 18:49     ` Andy Grover
  1 sibling, 0 replies; 5+ messages in thread
From: Vladimir Sokolovsky @ 2010-02-11 13:48 UTC (permalink / raw)
  To: Sean Hefty; +Cc: Roland Dreier, linux-rdma

Sean Hefty wrote:
>> enum ib_atomic_cap {
>> @@ -472,7 +473,9 @@ enum ib_wc_opcode {
>>  * receive by testing (opcode & IB_WC_RECV).
>>  */
>> 	IB_WC_RECV			= 1 << 7,
>> -	IB_WC_RECV_RDMA_WITH_IMM
>> +	IB_WC_RECV_RDMA_WITH_IMM,
>> +	IB_WC_MASKED_COMP_SWAP,
>> +	IB_WC_MASKED_FETCH_ADD
>> };
> 
> This breaks the ability to check for a receive by testing opcode & IB_WC_RECV.

Thanks Sean,
I'll send an updated patch with the fix.

> Is it even necessary to define new work completion opcodes?
> 
>> enum ib_wc_flags {
>> @@ -689,6 +692,8 @@ enum ib_wr_opcode {
>> 	IB_WR_RDMA_READ_WITH_INV,
>> 	IB_WR_LOCAL_INV,
>> 	IB_WR_FAST_REG_MR,
>> +	IB_WR_ATOMIC_MASKED_CMP_AND_SWP,
>> +	IB_WR_ATOMIC_MASKED_FETCH_AND_ADD,
>> };
> 
> Just tossing this out as a possibility -- the mask itself could be used to
> indicate the type of operation.  Kernel clients would always set the mask for
> all atomic operations.

I don't have a strong opinion on this, but it looks to me more natural to use
new opcodes and this way it is backward compatible.

Regards,
Vladimir
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH 1/2] IB/core: Add support for enhanced atomic operations
       [not found]   ` <96930957548D41E0B47FA465CAAD05E6-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
  2010-02-11 13:48     ` Vladimir Sokolovsky
@ 2010-02-17 18:49     ` Andy Grover
       [not found]       ` <4B7C3A2C.6070202-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
  1 sibling, 1 reply; 5+ messages in thread
From: Andy Grover @ 2010-02-17 18:49 UTC (permalink / raw)
  To: Sean Hefty; +Cc: 'Vladimir Sokolovsky', Roland Dreier, linux-rdma

Sean Hefty wrote:

> Just tossing this out as a possibility -- the mask itself could be used to
> indicate the type of operation.  Kernel clients would always set the mask for
> all atomic operations.
> 
> Also, is there a kernel user for this functionality?

RDS would make use of this.

Regards -- Andy

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: [PATCH 1/2] IB/core: Add support for enhanced atomic operations
       [not found]       ` <4B7C3A2C.6070202-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
@ 2010-02-17 23:56         ` Richard Frank
  0 siblings, 0 replies; 5+ messages in thread
From: Richard Frank @ 2010-02-17 23:56 UTC (permalink / raw)
  To: Andy Grover
  Cc: Sean Hefty, 'Vladimir Sokolovsky', Roland Dreier, linux-rdma

Oracle is planning to use these masked atomic ops..

Andy Grover wrote:
> Sean Hefty wrote:
>
>   
>> Just tossing this out as a possibility -- the mask itself could be used to
>> indicate the type of operation.  Kernel clients would always set the mask for
>> all atomic operations.
>>
>> Also, is there a kernel user for this functionality?
>>     
>
> RDS would make use of this.
>
> Regards -- Andy
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2010-02-17 23:56 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-02-01 15:59 [PATCH 1/2] IB/core: Add support for enhanced atomic operations Vladimir Sokolovsky
2010-02-11  0:51 ` Sean Hefty
     [not found]   ` <96930957548D41E0B47FA465CAAD05E6-Zpru7NauK7drdx17CPfAsdBPR1lH4CV8@public.gmane.org>
2010-02-11 13:48     ` Vladimir Sokolovsky
2010-02-17 18:49     ` Andy Grover
     [not found]       ` <4B7C3A2C.6070202-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2010-02-17 23:56         ` Richard Frank

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.