Linux-RDMA Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH] Provider/rxe: Fix regression to UD traffic
@ 2020-10-15 20:17 Bob Pearson
  2020-10-15 23:47 ` Jason Gunthorpe
  0 siblings, 1 reply; 3+ messages in thread
From: Bob Pearson @ 2020-10-15 20:17 UTC (permalink / raw)
  To: jgg, zyjzyj2000, linux-rdma; +Cc: Bob Pearson

Update enum rdma_network_type copy to match kernel version.
Without this change provider/rxe will send incorrect
network types to the kernel in send WQEs.

This fix keeps rxe functional but should be replaced by a better
implementation.

Signed-off-by: Bob Pearson <rpearson@hpe.com>
---
 providers/rxe/rxe.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/providers/rxe/rxe.h b/providers/rxe/rxe.h
index 96f4ee9c..46e6ce72 100644
--- a/providers/rxe/rxe.h
+++ b/providers/rxe/rxe.h
@@ -44,6 +44,7 @@
 
 enum rdma_network_type {
 	RDMA_NETWORK_IB,
+	RDMA_NETWORK_ROCE_V1,
 	RDMA_NETWORK_IPV4,
 	RDMA_NETWORK_IPV6
 };
-- 
2.25.1


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH] Provider/rxe: Fix regression to UD traffic
  2020-10-15 20:17 [PATCH] Provider/rxe: Fix regression to UD traffic Bob Pearson
@ 2020-10-15 23:47 ` Jason Gunthorpe
  2020-10-16  1:23   ` Bob Pearson
  0 siblings, 1 reply; 3+ messages in thread
From: Jason Gunthorpe @ 2020-10-15 23:47 UTC (permalink / raw)
  To: Bob Pearson; +Cc: zyjzyj2000, linux-rdma, Bob Pearson

On Thu, Oct 15, 2020 at 03:17:51PM -0500, Bob Pearson wrote:
> Update enum rdma_network_type copy to match kernel version.
> Without this change provider/rxe will send incorrect
> network types to the kernel in send WQEs.
> 
> This fix keeps rxe functional but should be replaced by a better
> implementation.
> 
> Signed-off-by: Bob Pearson <rpearson@hpe.com>
> ---
>  providers/rxe/rxe.h | 1 +
>  1 file changed, 1 insertion(+)

Well, we can't just break user space so the kernel has to change in
some way to accommodate this.

Obviously rxe should not have uAPI stuff that is not in
include/uapi/rdma, so lets just fix that directly.

Please confirm I did this right. The PR for this merge window must be
sent Friday.

From 8b20e1ceea413c98cb98930cb549be390226320f Mon Sep 17 00:00:00 2001
From: Jason Gunthorpe <jgg@nvidia.com>
Date: Thu, 15 Oct 2020 20:42:18 -0300
Subject: [PATCH] RDMA/rxe: Move the definitions for rxe_av.network_type to
 uAPI

RXE was wrongly using an internal kernel enum as part of its uAPI, split
this out into a dedicated uAPI enum just for RXE. It only uses the IPv4
and IPv6 values.

This was exposed by changing the internal kernel enum definition which
broke RXE.

Fixes: 1c15b4f2a42f ("RDMA/core: Modify enum ib_gid_type and enum rdma_network_type")
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/infiniband/sw/rxe/rxe_net.c | 8 ++++----
 include/uapi/rdma/rdma_user_rxe.h   | 6 ++++++
 2 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c
index 31b93e7e1e2f41..575e1a4ec82121 100644
--- a/drivers/infiniband/sw/rxe/rxe_net.c
+++ b/drivers/infiniband/sw/rxe/rxe_net.c
@@ -133,14 +133,14 @@ static struct dst_entry *rxe_find_route(struct net_device *ndev,
 		if (dst)
 			dst_release(dst);
 
-		if (av->network_type == RDMA_NETWORK_IPV4) {
+		if (av->network_type == RXE_NETWORK_TYPE_IPV4) {
 			struct in_addr *saddr;
 			struct in_addr *daddr;
 
 			saddr = &av->sgid_addr._sockaddr_in.sin_addr;
 			daddr = &av->dgid_addr._sockaddr_in.sin_addr;
 			dst = rxe_find_route4(ndev, saddr, daddr);
-		} else if (av->network_type == RDMA_NETWORK_IPV6) {
+		} else if (av->network_type == RXE_NETWORK_TYPE_IPV6) {
 			struct in6_addr *saddr6;
 			struct in6_addr *daddr6;
 
@@ -442,7 +442,7 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av,
 	if (IS_ERR(attr))
 		return NULL;
 
-	if (av->network_type == RDMA_NETWORK_IPV4)
+	if (av->network_type == RXE_NETWORK_TYPE_IPV6)
 		hdr_len = ETH_HLEN + sizeof(struct udphdr) +
 			sizeof(struct iphdr);
 	else
@@ -469,7 +469,7 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av,
 	skb->dev	= ndev;
 	rcu_read_unlock();
 
-	if (av->network_type == RDMA_NETWORK_IPV4)
+	if (av->network_type == RXE_NETWORK_TYPE_IPV4)
 		skb->protocol = htons(ETH_P_IP);
 	else
 		skb->protocol = htons(ETH_P_IPV6);
diff --git a/include/uapi/rdma/rdma_user_rxe.h b/include/uapi/rdma/rdma_user_rxe.h
index d8f2e0e46daba7..e591d8c1f3cf10 100644
--- a/include/uapi/rdma/rdma_user_rxe.h
+++ b/include/uapi/rdma/rdma_user_rxe.h
@@ -39,6 +39,11 @@
 #include <linux/in.h>
 #include <linux/in6.h>
 
+enum {
+	RXE_NETWORK_TYPE_IPV4 = 1,
+	RXE_NETWORK_TYPE_IPV6 = 2,
+};
+
 union rxe_gid {
 	__u8	raw[16];
 	struct {
@@ -57,6 +62,7 @@ struct rxe_global_route {
 
 struct rxe_av {
 	__u8			port_num;
+	/* From RXE_NETWORK_TYPE_* */
 	__u8			network_type;
 	__u8			dmac[6];
 	struct rxe_global_route	grh;
-- 
2.28.0


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [PATCH] Provider/rxe: Fix regression to UD traffic
  2020-10-15 23:47 ` Jason Gunthorpe
@ 2020-10-16  1:23   ` Bob Pearson
  0 siblings, 0 replies; 3+ messages in thread
From: Bob Pearson @ 2020-10-16  1:23 UTC (permalink / raw)
  To: Jason Gunthorpe; +Cc: zyjzyj2000, linux-rdma, Bob Pearson

On 10/15/20 6:47 PM, Jason Gunthorpe wrote:
> On Thu, Oct 15, 2020 at 03:17:51PM -0500, Bob Pearson wrote:
>> Update enum rdma_network_type copy to match kernel version.
>> Without this change provider/rxe will send incorrect
>> network types to the kernel in send WQEs.
>>
>> This fix keeps rxe functional but should be replaced by a better
>> implementation.
>>
>> Signed-off-by: Bob Pearson <rpearson@hpe.com>
>> ---
>>  providers/rxe/rxe.h | 1 +
>>  1 file changed, 1 insertion(+)
> 
> Well, we can't just break user space so the kernel has to change in
> some way to accommodate this.
> 
> Obviously rxe should not have uAPI stuff that is not in
> include/uapi/rdma, so lets just fix that directly.
> 
> Please confirm I did this right. The PR for this merge window must be
> sent Friday.

That works too. I am figuring out how to convert ah_handle to ib_ah in rdma_rxe. I think I've mostly got it.
uobj_get_obj_read(type, id, attr) is close to what I need. The problem is that all the code in this area assumes
it is getting called in a user verb API call so there is an attr to pass around. What we need is an API like
rdma_get_obj(ucontext, type, id) since we need a way to get to the user api data structures. Better would be
a pointer from qp to ucontext or ufile that would be set for user QPs. If we did this rxe just use handles from
the core instead of creating its own databases of objects.

Your patch will get us going for now. The above is a longer project that won't fit in your short term schedule.

Bob

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, back to index

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-15 20:17 [PATCH] Provider/rxe: Fix regression to UD traffic Bob Pearson
2020-10-15 23:47 ` Jason Gunthorpe
2020-10-16  1:23   ` Bob Pearson

Linux-RDMA Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-rdma/0 linux-rdma/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-rdma linux-rdma/ https://lore.kernel.org/linux-rdma \
		linux-rdma@vger.kernel.org
	public-inbox-index linux-rdma

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-rdma


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git