* [PATCH for-next v2] rdma_rxe: fix bug rejecting multicast packets
@ 2020-10-08 21:27 Bob Pearson
2020-10-08 23:33 ` Jason Gunthorpe
2020-10-09 15:23 ` Zhu Yanjun
0 siblings, 2 replies; 6+ messages in thread
From: Bob Pearson @ 2020-10-08 21:27 UTC (permalink / raw)
To: jgg, zyjzyj2000, linux-rdma; +Cc: Bob Pearson
- Fix a bug in rxe_rcv that causes all multicast packets to be
dropped. Currently rxe_match_dgid is called for each packet
to verify that the destination IP address matches one of the
entries in the port source GID table. This is incorrect for
IP multicast addresses since they do not appear in the GID table.
- Add code to detect multicast addresses.
- Change function name to rxe_chk_dgid which is clearer.
Signed-off-by: Bob Pearson <rpearson@hpe.com>
---
drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
1 file changed, 16 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c
index a3eed4da1540..b6fee61b2aee 100644
--- a/drivers/infiniband/sw/rxe/rxe_recv.c
+++ b/drivers/infiniband/sw/rxe/rxe_recv.c
@@ -280,7 +280,17 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb)
kfree_skb(skb);
}
-static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
+/**
+ * rxe_chk_dgid - validate destination IP address
+ * @rxe: rxe device that received packet
+ * @skb: the received packet buffer
+ *
+ * Accept any loopback packets
+ * Extract IP address from packet and
+ * Accept if multicast packet
+ * Accept if matches an SGID table entry
+ */
+static int rxe_chk_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
{
struct rxe_pkt_info *pkt = SKB_TO_PKT(skb);
const struct ib_gid_attr *gid_attr;
@@ -298,6 +308,9 @@ static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
pdgid = (union ib_gid *)&ipv6_hdr(skb)->daddr;
}
+ if (rdma_is_multicast_addr((struct in6_addr *)pdgid))
+ return 0;
+
gid_attr = rdma_find_gid_by_port(&rxe->ib_dev, pdgid,
IB_GID_TYPE_ROCE_UDP_ENCAP,
1, skb->dev);
@@ -322,8 +335,8 @@ void rxe_rcv(struct sk_buff *skb)
if (unlikely(skb->len < pkt->offset + RXE_BTH_BYTES))
goto drop;
- if (rxe_match_dgid(rxe, skb) < 0) {
- pr_warn_ratelimited("failed matching dgid\n");
+ if (rxe_chk_dgid(rxe, skb) < 0) {
+ pr_warn_ratelimited("failed checking dgid\n");
goto drop;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH for-next v2] rdma_rxe: fix bug rejecting multicast packets
2020-10-08 21:27 [PATCH for-next v2] rdma_rxe: fix bug rejecting multicast packets Bob Pearson
@ 2020-10-08 23:33 ` Jason Gunthorpe
2020-10-09 15:23 ` Zhu Yanjun
1 sibling, 0 replies; 6+ messages in thread
From: Jason Gunthorpe @ 2020-10-08 23:33 UTC (permalink / raw)
To: Bob Pearson; +Cc: zyjzyj2000, linux-rdma, Bob Pearson
On Thu, Oct 08, 2020 at 04:27:53PM -0500, Bob Pearson wrote:
> - Fix a bug in rxe_rcv that causes all multicast packets to be
> dropped. Currently rxe_match_dgid is called for each packet
> to verify that the destination IP address matches one of the
> entries in the port source GID table. This is incorrect for
> IP multicast addresses since they do not appear in the GID table.
> - Add code to detect multicast addresses.
> - Change function name to rxe_chk_dgid which is clearer.
>
> Signed-off-by: Bob Pearson <rpearson@hpe.com>
> ---
> drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
> 1 file changed, 16 insertions(+), 3 deletions(-)
Looks right, was there 4 patches? I only see three
Applied to for-next
Thanks,
Jason
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH for-next v2] rdma_rxe: fix bug rejecting multicast packets
2020-10-08 21:27 [PATCH for-next v2] rdma_rxe: fix bug rejecting multicast packets Bob Pearson
2020-10-08 23:33 ` Jason Gunthorpe
@ 2020-10-09 15:23 ` Zhu Yanjun
2020-10-09 15:28 ` Jason Gunthorpe
1 sibling, 1 reply; 6+ messages in thread
From: Zhu Yanjun @ 2020-10-09 15:23 UTC (permalink / raw)
To: Bob Pearson, jgg, linux-rdma; +Cc: Bob Pearson
On 10/9/2020 5:27 AM, Bob Pearson wrote:
> - Fix a bug in rxe_rcv that causes all multicast packets to be
> dropped. Currently rxe_match_dgid is called for each packet
> to verify that the destination IP address matches one of the
> entries in the port source GID table. This is incorrect for
> IP multicast addresses since they do not appear in the GID table.
> - Add code to detect multicast addresses.
> - Change function name to rxe_chk_dgid which is clearer.
>
> Signed-off-by: Bob Pearson <rpearson@hpe.com>
> ---
> drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
> 1 file changed, 16 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c
> index a3eed4da1540..b6fee61b2aee 100644
> --- a/drivers/infiniband/sw/rxe/rxe_recv.c
> +++ b/drivers/infiniband/sw/rxe/rxe_recv.c
> @@ -280,7 +280,17 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb)
> kfree_skb(skb);
> }
>
> -static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
> +/**
> + * rxe_chk_dgid - validate destination IP address
> + * @rxe: rxe device that received packet
> + * @skb: the received packet buffer
> + *
> + * Accept any loopback packets
About loopback packets, will rdma_find_gid_by_port return correct value?
In my tests, to loopback packets, sometimes rdma_find_gid_by_port return
incorrect value.
Then the packets will be freed.
Zhu Yanjun
> + * Extract IP address from packet and
> + * Accept if multicast packet
> + * Accept if matches an SGID table entry
> + */
> +static int rxe_chk_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
> {
> struct rxe_pkt_info *pkt = SKB_TO_PKT(skb);
> const struct ib_gid_attr *gid_attr;
> @@ -298,6 +308,9 @@ static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
> pdgid = (union ib_gid *)&ipv6_hdr(skb)->daddr;
> }
>
> + if (rdma_is_multicast_addr((struct in6_addr *)pdgid))
> + return 0;
> +
> gid_attr = rdma_find_gid_by_port(&rxe->ib_dev, pdgid,
> IB_GID_TYPE_ROCE_UDP_ENCAP,
> 1, skb->dev);
> @@ -322,8 +335,8 @@ void rxe_rcv(struct sk_buff *skb)
> if (unlikely(skb->len < pkt->offset + RXE_BTH_BYTES))
> goto drop;
>
> - if (rxe_match_dgid(rxe, skb) < 0) {
> - pr_warn_ratelimited("failed matching dgid\n");
> + if (rxe_chk_dgid(rxe, skb) < 0) {
> + pr_warn_ratelimited("failed checking dgid\n");
> goto drop;
> }
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH for-next v2] rdma_rxe: fix bug rejecting multicast packets
2020-10-09 15:23 ` Zhu Yanjun
@ 2020-10-09 15:28 ` Jason Gunthorpe
2020-10-09 17:18 ` Bob Pearson
0 siblings, 1 reply; 6+ messages in thread
From: Jason Gunthorpe @ 2020-10-09 15:28 UTC (permalink / raw)
To: Zhu Yanjun; +Cc: Bob Pearson, linux-rdma, Bob Pearson
On Fri, Oct 09, 2020 at 11:23:31PM +0800, Zhu Yanjun wrote:
> On 10/9/2020 5:27 AM, Bob Pearson wrote:
> > - Fix a bug in rxe_rcv that causes all multicast packets to be
> > dropped. Currently rxe_match_dgid is called for each packet
> > to verify that the destination IP address matches one of the
> > entries in the port source GID table. This is incorrect for
> > IP multicast addresses since they do not appear in the GID table.
> > - Add code to detect multicast addresses.
> > - Change function name to rxe_chk_dgid which is clearer.
> >
> > Signed-off-by: Bob Pearson <rpearson@hpe.com>
> > drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
> > 1 file changed, 16 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c
> > index a3eed4da1540..b6fee61b2aee 100644
> > +++ b/drivers/infiniband/sw/rxe/rxe_recv.c
> > @@ -280,7 +280,17 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb)
> > kfree_skb(skb);
> > }
> > -static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
> > +/**
> > + * rxe_chk_dgid - validate destination IP address
> > + * @rxe: rxe device that received packet
> > + * @skb: the received packet buffer
> > + *
> > + * Accept any loopback packets
>
> About loopback packets, will rdma_find_gid_by_port return correct value?
I don't think you can use 127.0.0.0 with the RDMA devices, at least
not on the wire. The CM has special code to swap it out with a real
device address
Jason
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH for-next v2] rdma_rxe: fix bug rejecting multicast packets
2020-10-09 15:28 ` Jason Gunthorpe
@ 2020-10-09 17:18 ` Bob Pearson
2020-10-10 3:54 ` Zhu Yanjun
0 siblings, 1 reply; 6+ messages in thread
From: Bob Pearson @ 2020-10-09 17:18 UTC (permalink / raw)
To: Jason Gunthorpe, Zhu Yanjun; +Cc: linux-rdma, Bob Pearson
On 10/9/20 10:28 AM, Jason Gunthorpe wrote:
> On Fri, Oct 09, 2020 at 11:23:31PM +0800, Zhu Yanjun wrote:
>> On 10/9/2020 5:27 AM, Bob Pearson wrote:
>>> - Fix a bug in rxe_rcv that causes all multicast packets to be
>>> dropped. Currently rxe_match_dgid is called for each packet
>>> to verify that the destination IP address matches one of the
>>> entries in the port source GID table. This is incorrect for
>>> IP multicast addresses since they do not appear in the GID table.
>>> - Add code to detect multicast addresses.
>>> - Change function name to rxe_chk_dgid which is clearer.
>>>
>>> Signed-off-by: Bob Pearson <rpearson@hpe.com>
>>> drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
>>> 1 file changed, 16 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c
>>> index a3eed4da1540..b6fee61b2aee 100644
>>> +++ b/drivers/infiniband/sw/rxe/rxe_recv.c
>>> @@ -280,7 +280,17 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb)
>>> kfree_skb(skb);
>>> }
>>> -static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
>>> +/**
>>> + * rxe_chk_dgid - validate destination IP address
>>> + * @rxe: rxe device that received packet
>>> + * @skb: the received packet buffer
>>> + *
>>> + * Accept any loopback packets
>>
>> About loopback packets, will rdma_find_gid_by_port return correct value?
I didn't touch that but the RXE_LOOPBACK test comes before the call to rdma_find_gid_by_port
so it should never get called for loopback packets.
>
> I don't think you can use 127.0.0.0 with the RDMA devices, at least
> not on the wire. The CM has special code to swap it out with a real
> device address
The following does work:
$ ib_send_bw -d rxe_0 (in window A) $ ib_send_bw -d rxe_0 127.0.0.1 (in window B)
This uses the LOOPBACK path and just hands the skb from sender to receiver. It never touches the IP stack.
I have never been able to get this to work:
$ ib_send_bw -d rxe_1 (at 10.0.0.1 in window A) $ ib_send_bw -d rxe_2 10.0.0.1 (at 10.0.0.2 in window B)
If it did work I could test the full path.
>
> Jason
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH for-next v2] rdma_rxe: fix bug rejecting multicast packets
2020-10-09 17:18 ` Bob Pearson
@ 2020-10-10 3:54 ` Zhu Yanjun
0 siblings, 0 replies; 6+ messages in thread
From: Zhu Yanjun @ 2020-10-10 3:54 UTC (permalink / raw)
To: Bob Pearson, Jason Gunthorpe; +Cc: linux-rdma, Bob Pearson
On 10/10/2020 1:18 AM, Bob Pearson wrote:
> On 10/9/20 10:28 AM, Jason Gunthorpe wrote:
>> On Fri, Oct 09, 2020 at 11:23:31PM +0800, Zhu Yanjun wrote:
>>> On 10/9/2020 5:27 AM, Bob Pearson wrote:
>>>> - Fix a bug in rxe_rcv that causes all multicast packets to be
>>>> dropped. Currently rxe_match_dgid is called for each packet
>>>> to verify that the destination IP address matches one of the
>>>> entries in the port source GID table. This is incorrect for
>>>> IP multicast addresses since they do not appear in the GID table.
>>>> - Add code to detect multicast addresses.
>>>> - Change function name to rxe_chk_dgid which is clearer.
>>>>
>>>> Signed-off-by: Bob Pearson <rpearson@hpe.com>
>>>> drivers/infiniband/sw/rxe/rxe_recv.c | 19 ++++++++++++++++---
>>>> 1 file changed, 16 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/drivers/infiniband/sw/rxe/rxe_recv.c b/drivers/infiniband/sw/rxe/rxe_recv.c
>>>> index a3eed4da1540..b6fee61b2aee 100644
>>>> +++ b/drivers/infiniband/sw/rxe/rxe_recv.c
>>>> @@ -280,7 +280,17 @@ static void rxe_rcv_mcast_pkt(struct rxe_dev *rxe, struct sk_buff *skb)
>>>> kfree_skb(skb);
>>>> }
>>>> -static int rxe_match_dgid(struct rxe_dev *rxe, struct sk_buff *skb)
>>>> +/**
>>>> + * rxe_chk_dgid - validate destination IP address
>>>> + * @rxe: rxe device that received packet
>>>> + * @skb: the received packet buffer
>>>> + *
>>>> + * Accept any loopback packets
>>> About loopback packets, will rdma_find_gid_by_port return correct value?
> I didn't touch that but the RXE_LOOPBACK test comes before the call to rdma_find_gid_by_port
> so it should never get called for loopback packets.
I confronted the loopback problem with rdma-core tests.
And I made a patch to fix it. If the following commit exists, this
problem will not occur.
commit 5c99274be8864519328aa74bc550ba410095bc1c
Author: Zhu Yanjun <yanjunz@mellanox.com>
Date: Tue Jun 30 15:36:05 2020 +0300
RDMA/rxe: Skip dgid check in loopback mode
In the loopback tests, the following call trace occurs.
Call Trace:
__rxe_do_task+0x1a/0x30 [rdma_rxe]
rxe_qp_destroy+0x61/0xa0 [rdma_rxe]
rxe_destroy_qp+0x20/0x60 [rdma_rxe]
ib_destroy_qp_user+0xcc/0x220 [ib_core]
uverbs_free_qp+0x3c/0xc0 [ib_uverbs]
destroy_hw_idr_uobject+0x24/0x70 [ib_uverbs]
uverbs_destroy_uobject+0x43/0x1b0 [ib_uverbs]
uobj_destroy+0x41/0x70 [ib_uverbs]
__uobj_get_destroy+0x39/0x70 [ib_uverbs]
ib_uverbs_destroy_qp+0x88/0xc0 [ib_uverbs]
ib_uverbs_handler_UVERBS_METHOD_INVOKE_WRITE+0xb9/0xf0 [ib_uverbs]
ib_uverbs_cmd_verbs+0xb16/0xc30 [ib_uverbs]
The root cause is that the actual RDMA connection is not created in the
loopback tests and the rxe_match_dgid will fail randomly.
To fix this call trace which appear in the loopback tests, skip
check of
the dgid.
Fixes: 8700e3e7c485 ("Soft RoCE driver")
Link: https://lore.kernel.org/r/20200630123605.446959-1-leon@kernel.org
Signed-off-by: Zhu Yanjun <yanjunz@mellanox.com>
Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
>> I don't think you can use 127.0.0.0 with the RDMA devices, at least
>> not on the wire. The CM has special code to swap it out with a real
>> device address
> The following does work:
>
> $ ib_send_bw -d rxe_0 (in window A) $ ib_send_bw -d rxe_0 127.0.0.1 (in window B)
>
> This uses the LOOPBACK path and just hands the skb from sender to receiver. It never touches the IP stack.
>
> I have never been able to get this to work:
>
> $ ib_send_bw -d rxe_1 (at 10.0.0.1 in window A) $ ib_send_bw -d rxe_2 10.0.0.1 (at 10.0.0.2 in window B)
>
> If it did work I could test the full path.
>> Jason
>>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2020-10-10 4:05 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-08 21:27 [PATCH for-next v2] rdma_rxe: fix bug rejecting multicast packets Bob Pearson
2020-10-08 23:33 ` Jason Gunthorpe
2020-10-09 15:23 ` Zhu Yanjun
2020-10-09 15:28 ` Jason Gunthorpe
2020-10-09 17:18 ` Bob Pearson
2020-10-10 3:54 ` Zhu Yanjun
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).