* [PATCH] RDMA/rxe: For invalidate compare according to set keys in mr
@ 2022-07-07 7:30 Md Haris Iqbal
2022-07-15 9:21 ` haris iqbal
2022-07-22 20:38 ` Jason Gunthorpe
0 siblings, 2 replies; 7+ messages in thread
From: Md Haris Iqbal @ 2022-07-07 7:30 UTC (permalink / raw)
To: linux-rdma
Cc: zyjzyj2000, leon, jgg, haris.iqbal, jinpu.wang, aleksei.marov,
Md Haris Iqbal, rpearsonhpe
The 'rkey' input can be an lkey or rkey, and in rxe the lkey or rkey have
the same value, including the variant bits.
So, if mr->rkey is set, compare the invalidate key with it, otherwise
compare with the mr->lkey.
Since we already did a lookup on the non-varient bits to get this far,
the check's only purpose is to confirm that the wqe has the correct
variant bits.
Fixes: 001345339f4c ("RDMA/rxe: Separate HW and SW l/rkeys")
Cc: rpearsonhpe@gmail.com
Signed-off-by: Md Haris Iqbal <haris.phnx@gmail.com>
---
drivers/infiniband/sw/rxe/rxe_loc.h | 2 +-
drivers/infiniband/sw/rxe/rxe_mr.c | 12 ++++++------
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h
index 0e022ae1b8a5..37484a559d20 100644
--- a/drivers/infiniband/sw/rxe/rxe_loc.h
+++ b/drivers/infiniband/sw/rxe/rxe_loc.h
@@ -77,7 +77,7 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
enum rxe_mr_lookup_type type);
int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length);
int advance_dma_data(struct rxe_dma_info *dma, unsigned int length);
-int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey);
+int rxe_invalidate_mr(struct rxe_qp *qp, u32 key);
int rxe_reg_fast_mr(struct rxe_qp *qp, struct rxe_send_wqe *wqe);
int rxe_mr_set_page(struct ib_mr *ibmr, u64 addr);
int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
index fc3942e04a1f..3add52129006 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -576,22 +576,22 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
return mr;
}
-int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey)
+int rxe_invalidate_mr(struct rxe_qp *qp, u32 key)
{
struct rxe_dev *rxe = to_rdev(qp->ibqp.device);
struct rxe_mr *mr;
int ret;
- mr = rxe_pool_get_index(&rxe->mr_pool, rkey >> 8);
+ mr = rxe_pool_get_index(&rxe->mr_pool, key >> 8);
if (!mr) {
- pr_err("%s: No MR for rkey %#x\n", __func__, rkey);
+ pr_err("%s: No MR for key %#x\n", __func__, key);
ret = -EINVAL;
goto err;
}
- if (rkey != mr->rkey) {
- pr_err("%s: rkey (%#x) doesn't match mr->rkey (%#x)\n",
- __func__, rkey, mr->rkey);
+ if (mr->rkey ? (key != mr->rkey) : (key != mr->lkey)) {
+ pr_err("%s: wr key (%#x) doesn't match mr key (%#x)\n",
+ __func__, key, (mr->rkey ? mr->rkey : mr->lkey));
ret = -EINVAL;
goto err_drop_ref;
}
--
2.25.1
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH] RDMA/rxe: For invalidate compare according to set keys in mr
2022-07-07 7:30 [PATCH] RDMA/rxe: For invalidate compare according to set keys in mr Md Haris Iqbal
@ 2022-07-15 9:21 ` haris iqbal
2022-07-22 15:10 ` haris iqbal
2022-07-22 20:38 ` Jason Gunthorpe
1 sibling, 1 reply; 7+ messages in thread
From: haris iqbal @ 2022-07-15 9:21 UTC (permalink / raw)
To: linux-rdma
Cc: zyjzyj2000, Leon Romanovsky, Jason Gunthorpe, haris.iqbal,
Jinpu Wang, aleksei.marov, Bob Pearson
On Thu, Jul 7, 2022 at 9:30 AM Md Haris Iqbal <haris.phnx@gmail.com> wrote:
>
> The 'rkey' input can be an lkey or rkey, and in rxe the lkey or rkey have
> the same value, including the variant bits.
>
> So, if mr->rkey is set, compare the invalidate key with it, otherwise
> compare with the mr->lkey.
>
> Since we already did a lookup on the non-varient bits to get this far,
> the check's only purpose is to confirm that the wqe has the correct
> variant bits.
>
> Fixes: 001345339f4c ("RDMA/rxe: Separate HW and SW l/rkeys")
> Cc: rpearsonhpe@gmail.com
> Signed-off-by: Md Haris Iqbal <haris.phnx@gmail.com>
> ---
> drivers/infiniband/sw/rxe/rxe_loc.h | 2 +-
> drivers/infiniband/sw/rxe/rxe_mr.c | 12 ++++++------
> 2 files changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h
> index 0e022ae1b8a5..37484a559d20 100644
> --- a/drivers/infiniband/sw/rxe/rxe_loc.h
> +++ b/drivers/infiniband/sw/rxe/rxe_loc.h
> @@ -77,7 +77,7 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
> enum rxe_mr_lookup_type type);
> int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length);
> int advance_dma_data(struct rxe_dma_info *dma, unsigned int length);
> -int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey);
> +int rxe_invalidate_mr(struct rxe_qp *qp, u32 key);
> int rxe_reg_fast_mr(struct rxe_qp *qp, struct rxe_send_wqe *wqe);
> int rxe_mr_set_page(struct ib_mr *ibmr, u64 addr);
> int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
> diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
> index fc3942e04a1f..3add52129006 100644
> --- a/drivers/infiniband/sw/rxe/rxe_mr.c
> +++ b/drivers/infiniband/sw/rxe/rxe_mr.c
> @@ -576,22 +576,22 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
> return mr;
> }
>
> -int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey)
> +int rxe_invalidate_mr(struct rxe_qp *qp, u32 key)
> {
> struct rxe_dev *rxe = to_rdev(qp->ibqp.device);
> struct rxe_mr *mr;
> int ret;
>
> - mr = rxe_pool_get_index(&rxe->mr_pool, rkey >> 8);
> + mr = rxe_pool_get_index(&rxe->mr_pool, key >> 8);
> if (!mr) {
> - pr_err("%s: No MR for rkey %#x\n", __func__, rkey);
> + pr_err("%s: No MR for key %#x\n", __func__, key);
> ret = -EINVAL;
> goto err;
> }
>
> - if (rkey != mr->rkey) {
> - pr_err("%s: rkey (%#x) doesn't match mr->rkey (%#x)\n",
> - __func__, rkey, mr->rkey);
> + if (mr->rkey ? (key != mr->rkey) : (key != mr->lkey)) {
> + pr_err("%s: wr key (%#x) doesn't match mr key (%#x)\n",
> + __func__, key, (mr->rkey ? mr->rkey : mr->lkey));
> ret = -EINVAL;
> goto err_drop_ref;
> }
> --
> 2.25.1
Ping.
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] RDMA/rxe: For invalidate compare according to set keys in mr
2022-07-15 9:21 ` haris iqbal
@ 2022-07-22 15:10 ` haris iqbal
2022-07-22 15:14 ` Jason Gunthorpe
0 siblings, 1 reply; 7+ messages in thread
From: haris iqbal @ 2022-07-22 15:10 UTC (permalink / raw)
To: linux-rdma
Cc: zyjzyj2000, Leon Romanovsky, Jason Gunthorpe, haris.iqbal,
Jinpu Wang, aleksei.marov, Bob Pearson
On Fri, Jul 15, 2022 at 11:21 AM haris iqbal <haris.phnx@gmail.com> wrote:
>
> On Thu, Jul 7, 2022 at 9:30 AM Md Haris Iqbal <haris.phnx@gmail.com> wrote:
> >
> > The 'rkey' input can be an lkey or rkey, and in rxe the lkey or rkey have
> > the same value, including the variant bits.
> >
> > So, if mr->rkey is set, compare the invalidate key with it, otherwise
> > compare with the mr->lkey.
> >
> > Since we already did a lookup on the non-varient bits to get this far,
> > the check's only purpose is to confirm that the wqe has the correct
> > variant bits.
> >
> > Fixes: 001345339f4c ("RDMA/rxe: Separate HW and SW l/rkeys")
> > Cc: rpearsonhpe@gmail.com
> > Signed-off-by: Md Haris Iqbal <haris.phnx@gmail.com>
> > ---
> > drivers/infiniband/sw/rxe/rxe_loc.h | 2 +-
> > drivers/infiniband/sw/rxe/rxe_mr.c | 12 ++++++------
> > 2 files changed, 7 insertions(+), 7 deletions(-)
> >
> > diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h
> > index 0e022ae1b8a5..37484a559d20 100644
> > --- a/drivers/infiniband/sw/rxe/rxe_loc.h
> > +++ b/drivers/infiniband/sw/rxe/rxe_loc.h
> > @@ -77,7 +77,7 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
> > enum rxe_mr_lookup_type type);
> > int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length);
> > int advance_dma_data(struct rxe_dma_info *dma, unsigned int length);
> > -int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey);
> > +int rxe_invalidate_mr(struct rxe_qp *qp, u32 key);
> > int rxe_reg_fast_mr(struct rxe_qp *qp, struct rxe_send_wqe *wqe);
> > int rxe_mr_set_page(struct ib_mr *ibmr, u64 addr);
> > int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
> > diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
> > index fc3942e04a1f..3add52129006 100644
> > --- a/drivers/infiniband/sw/rxe/rxe_mr.c
> > +++ b/drivers/infiniband/sw/rxe/rxe_mr.c
> > @@ -576,22 +576,22 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
> > return mr;
> > }
> >
> > -int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey)
> > +int rxe_invalidate_mr(struct rxe_qp *qp, u32 key)
> > {
> > struct rxe_dev *rxe = to_rdev(qp->ibqp.device);
> > struct rxe_mr *mr;
> > int ret;
> >
> > - mr = rxe_pool_get_index(&rxe->mr_pool, rkey >> 8);
> > + mr = rxe_pool_get_index(&rxe->mr_pool, key >> 8);
> > if (!mr) {
> > - pr_err("%s: No MR for rkey %#x\n", __func__, rkey);
> > + pr_err("%s: No MR for key %#x\n", __func__, key);
> > ret = -EINVAL;
> > goto err;
> > }
> >
> > - if (rkey != mr->rkey) {
> > - pr_err("%s: rkey (%#x) doesn't match mr->rkey (%#x)\n",
> > - __func__, rkey, mr->rkey);
> > + if (mr->rkey ? (key != mr->rkey) : (key != mr->lkey)) {
> > + pr_err("%s: wr key (%#x) doesn't match mr key (%#x)\n",
> > + __func__, key, (mr->rkey ? mr->rkey : mr->lkey));
> > ret = -EINVAL;
> > goto err_drop_ref;
> > }
> > --
> > 2.25.1
>
> Ping.
Ping.
Does this need more discussions or changes?
>
> >
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] RDMA/rxe: For invalidate compare according to set keys in mr
2022-07-22 15:10 ` haris iqbal
@ 2022-07-22 15:14 ` Jason Gunthorpe
2022-07-22 16:50 ` haris iqbal
0 siblings, 1 reply; 7+ messages in thread
From: Jason Gunthorpe @ 2022-07-22 15:14 UTC (permalink / raw)
To: haris iqbal
Cc: linux-rdma, zyjzyj2000, Leon Romanovsky, haris.iqbal, Jinpu Wang,
aleksei.marov, Bob Pearson
On Fri, Jul 22, 2022 at 05:10:46PM +0200, haris iqbal wrote:
> On Fri, Jul 15, 2022 at 11:21 AM haris iqbal <haris.phnx@gmail.com> wrote:
> >
> > On Thu, Jul 7, 2022 at 9:30 AM Md Haris Iqbal <haris.phnx@gmail.com> wrote:
> > >
> > > The 'rkey' input can be an lkey or rkey, and in rxe the lkey or rkey have
> > > the same value, including the variant bits.
> > >
> > > So, if mr->rkey is set, compare the invalidate key with it, otherwise
> > > compare with the mr->lkey.
> > >
> > > Since we already did a lookup on the non-varient bits to get this far,
> > > the check's only purpose is to confirm that the wqe has the correct
> > > variant bits.
> > >
> > > Fixes: 001345339f4c ("RDMA/rxe: Separate HW and SW l/rkeys")
> > > Cc: rpearsonhpe@gmail.com
> > > Signed-off-by: Md Haris Iqbal <haris.phnx@gmail.com>
> > > ---
> > > drivers/infiniband/sw/rxe/rxe_loc.h | 2 +-
> > > drivers/infiniband/sw/rxe/rxe_mr.c | 12 ++++++------
> > > 2 files changed, 7 insertions(+), 7 deletions(-)
> > >
> > > diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h
> > > index 0e022ae1b8a5..37484a559d20 100644
> > > --- a/drivers/infiniband/sw/rxe/rxe_loc.h
> > > +++ b/drivers/infiniband/sw/rxe/rxe_loc.h
> > > @@ -77,7 +77,7 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
> > > enum rxe_mr_lookup_type type);
> > > int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length);
> > > int advance_dma_data(struct rxe_dma_info *dma, unsigned int length);
> > > -int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey);
> > > +int rxe_invalidate_mr(struct rxe_qp *qp, u32 key);
> > > int rxe_reg_fast_mr(struct rxe_qp *qp, struct rxe_send_wqe *wqe);
> > > int rxe_mr_set_page(struct ib_mr *ibmr, u64 addr);
> > > int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
> > > diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
> > > index fc3942e04a1f..3add52129006 100644
> > > --- a/drivers/infiniband/sw/rxe/rxe_mr.c
> > > +++ b/drivers/infiniband/sw/rxe/rxe_mr.c
> > > @@ -576,22 +576,22 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
> > > return mr;
> > > }
> > >
> > > -int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey)
> > > +int rxe_invalidate_mr(struct rxe_qp *qp, u32 key)
> > > {
> > > struct rxe_dev *rxe = to_rdev(qp->ibqp.device);
> > > struct rxe_mr *mr;
> > > int ret;
> > >
> > > - mr = rxe_pool_get_index(&rxe->mr_pool, rkey >> 8);
> > > + mr = rxe_pool_get_index(&rxe->mr_pool, key >> 8);
> > > if (!mr) {
> > > - pr_err("%s: No MR for rkey %#x\n", __func__, rkey);
> > > + pr_err("%s: No MR for key %#x\n", __func__, key);
> > > ret = -EINVAL;
> > > goto err;
> > > }
> > >
> > > - if (rkey != mr->rkey) {
> > > - pr_err("%s: rkey (%#x) doesn't match mr->rkey (%#x)\n",
> > > - __func__, rkey, mr->rkey);
> > > + if (mr->rkey ? (key != mr->rkey) : (key != mr->lkey)) {
> > > + pr_err("%s: wr key (%#x) doesn't match mr key (%#x)\n",
> > > + __func__, key, (mr->rkey ? mr->rkey : mr->lkey));
> > > ret = -EINVAL;
> > > goto err_drop_ref;
> > > }
> >
> > Ping.
>
> Ping.
>
> Does this need more discussions or changes?
I was hoping Bob would say something but I am happy with it..
Jason
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] RDMA/rxe: For invalidate compare according to set keys in mr
2022-07-22 15:14 ` Jason Gunthorpe
@ 2022-07-22 16:50 ` haris iqbal
2022-07-22 19:04 ` Bob Pearson
0 siblings, 1 reply; 7+ messages in thread
From: haris iqbal @ 2022-07-22 16:50 UTC (permalink / raw)
To: Jason Gunthorpe, Bob Pearson
Cc: linux-rdma, zyjzyj2000, Leon Romanovsky, haris.iqbal, Jinpu Wang,
aleksei.marov
On Fri, Jul 22, 2022 at 5:14 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Fri, Jul 22, 2022 at 05:10:46PM +0200, haris iqbal wrote:
> > On Fri, Jul 15, 2022 at 11:21 AM haris iqbal <haris.phnx@gmail.com> wrote:
> > >
> > > On Thu, Jul 7, 2022 at 9:30 AM Md Haris Iqbal <haris.phnx@gmail.com> wrote:
> > > >
> > > > The 'rkey' input can be an lkey or rkey, and in rxe the lkey or rkey have
> > > > the same value, including the variant bits.
> > > >
> > > > So, if mr->rkey is set, compare the invalidate key with it, otherwise
> > > > compare with the mr->lkey.
> > > >
> > > > Since we already did a lookup on the non-varient bits to get this far,
> > > > the check's only purpose is to confirm that the wqe has the correct
> > > > variant bits.
> > > >
> > > > Fixes: 001345339f4c ("RDMA/rxe: Separate HW and SW l/rkeys")
> > > > Cc: rpearsonhpe@gmail.com
> > > > Signed-off-by: Md Haris Iqbal <haris.phnx@gmail.com>
> > > > ---
> > > > drivers/infiniband/sw/rxe/rxe_loc.h | 2 +-
> > > > drivers/infiniband/sw/rxe/rxe_mr.c | 12 ++++++------
> > > > 2 files changed, 7 insertions(+), 7 deletions(-)
> > > >
> > > > diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h
> > > > index 0e022ae1b8a5..37484a559d20 100644
> > > > --- a/drivers/infiniband/sw/rxe/rxe_loc.h
> > > > +++ b/drivers/infiniband/sw/rxe/rxe_loc.h
> > > > @@ -77,7 +77,7 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
> > > > enum rxe_mr_lookup_type type);
> > > > int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length);
> > > > int advance_dma_data(struct rxe_dma_info *dma, unsigned int length);
> > > > -int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey);
> > > > +int rxe_invalidate_mr(struct rxe_qp *qp, u32 key);
> > > > int rxe_reg_fast_mr(struct rxe_qp *qp, struct rxe_send_wqe *wqe);
> > > > int rxe_mr_set_page(struct ib_mr *ibmr, u64 addr);
> > > > int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
> > > > diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
> > > > index fc3942e04a1f..3add52129006 100644
> > > > --- a/drivers/infiniband/sw/rxe/rxe_mr.c
> > > > +++ b/drivers/infiniband/sw/rxe/rxe_mr.c
> > > > @@ -576,22 +576,22 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
> > > > return mr;
> > > > }
> > > >
> > > > -int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey)
> > > > +int rxe_invalidate_mr(struct rxe_qp *qp, u32 key)
> > > > {
> > > > struct rxe_dev *rxe = to_rdev(qp->ibqp.device);
> > > > struct rxe_mr *mr;
> > > > int ret;
> > > >
> > > > - mr = rxe_pool_get_index(&rxe->mr_pool, rkey >> 8);
> > > > + mr = rxe_pool_get_index(&rxe->mr_pool, key >> 8);
> > > > if (!mr) {
> > > > - pr_err("%s: No MR for rkey %#x\n", __func__, rkey);
> > > > + pr_err("%s: No MR for key %#x\n", __func__, key);
> > > > ret = -EINVAL;
> > > > goto err;
> > > > }
> > > >
> > > > - if (rkey != mr->rkey) {
> > > > - pr_err("%s: rkey (%#x) doesn't match mr->rkey (%#x)\n",
> > > > - __func__, rkey, mr->rkey);
> > > > + if (mr->rkey ? (key != mr->rkey) : (key != mr->lkey)) {
> > > > + pr_err("%s: wr key (%#x) doesn't match mr key (%#x)\n",
> > > > + __func__, key, (mr->rkey ? mr->rkey : mr->lkey));
> > > > ret = -EINVAL;
> > > > goto err_drop_ref;
> > > > }
> > >
> > > Ping.
> >
> > Ping.
> >
> > Does this need more discussions or changes?
>
> I was hoping Bob would say something but I am happy with it..
Thanks Jason for the response.
We can wait for Bob to share his thoughts.
>
> Jason
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] RDMA/rxe: For invalidate compare according to set keys in mr
2022-07-22 16:50 ` haris iqbal
@ 2022-07-22 19:04 ` Bob Pearson
0 siblings, 0 replies; 7+ messages in thread
From: Bob Pearson @ 2022-07-22 19:04 UTC (permalink / raw)
To: haris iqbal, Jason Gunthorpe
Cc: linux-rdma, zyjzyj2000, Leon Romanovsky, haris.iqbal, Jinpu Wang,
aleksei.marov
On 7/22/22 11:50, haris iqbal wrote:
> On Fri, Jul 22, 2022 at 5:14 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>>
>> On Fri, Jul 22, 2022 at 05:10:46PM +0200, haris iqbal wrote:
>>> On Fri, Jul 15, 2022 at 11:21 AM haris iqbal <haris.phnx@gmail.com> wrote:
>>>>
>>>> On Thu, Jul 7, 2022 at 9:30 AM Md Haris Iqbal <haris.phnx@gmail.com> wrote:
>>>>>
>>>>> The 'rkey' input can be an lkey or rkey, and in rxe the lkey or rkey have
>>>>> the same value, including the variant bits.
>>>>>
>>>>> So, if mr->rkey is set, compare the invalidate key with it, otherwise
>>>>> compare with the mr->lkey.
>>>>>
>>>>> Since we already did a lookup on the non-varient bits to get this far,
>>>>> the check's only purpose is to confirm that the wqe has the correct
>>>>> variant bits.
>>>>>
>>>>> Fixes: 001345339f4c ("RDMA/rxe: Separate HW and SW l/rkeys")
>>>>> Cc: rpearsonhpe@gmail.com
>>>>> Signed-off-by: Md Haris Iqbal <haris.phnx@gmail.com>
>>>>> ---
>>>>> drivers/infiniband/sw/rxe/rxe_loc.h | 2 +-
>>>>> drivers/infiniband/sw/rxe/rxe_mr.c | 12 ++++++------
>>>>> 2 files changed, 7 insertions(+), 7 deletions(-)
>>>>>
>>>>> diff --git a/drivers/infiniband/sw/rxe/rxe_loc.h b/drivers/infiniband/sw/rxe/rxe_loc.h
>>>>> index 0e022ae1b8a5..37484a559d20 100644
>>>>> --- a/drivers/infiniband/sw/rxe/rxe_loc.h
>>>>> +++ b/drivers/infiniband/sw/rxe/rxe_loc.h
>>>>> @@ -77,7 +77,7 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
>>>>> enum rxe_mr_lookup_type type);
>>>>> int mr_check_range(struct rxe_mr *mr, u64 iova, size_t length);
>>>>> int advance_dma_data(struct rxe_dma_info *dma, unsigned int length);
>>>>> -int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey);
>>>>> +int rxe_invalidate_mr(struct rxe_qp *qp, u32 key);
>>>>> int rxe_reg_fast_mr(struct rxe_qp *qp, struct rxe_send_wqe *wqe);
>>>>> int rxe_mr_set_page(struct ib_mr *ibmr, u64 addr);
>>>>> int rxe_dereg_mr(struct ib_mr *ibmr, struct ib_udata *udata);
>>>>> diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
>>>>> index fc3942e04a1f..3add52129006 100644
>>>>> --- a/drivers/infiniband/sw/rxe/rxe_mr.c
>>>>> +++ b/drivers/infiniband/sw/rxe/rxe_mr.c
>>>>> @@ -576,22 +576,22 @@ struct rxe_mr *lookup_mr(struct rxe_pd *pd, int access, u32 key,
>>>>> return mr;
>>>>> }
>>>>>
>>>>> -int rxe_invalidate_mr(struct rxe_qp *qp, u32 rkey)
>>>>> +int rxe_invalidate_mr(struct rxe_qp *qp, u32 key)
>>>>> {
>>>>> struct rxe_dev *rxe = to_rdev(qp->ibqp.device);
>>>>> struct rxe_mr *mr;
>>>>> int ret;
>>>>>
>>>>> - mr = rxe_pool_get_index(&rxe->mr_pool, rkey >> 8);
>>>>> + mr = rxe_pool_get_index(&rxe->mr_pool, key >> 8);
>>>>> if (!mr) {
>>>>> - pr_err("%s: No MR for rkey %#x\n", __func__, rkey);
>>>>> + pr_err("%s: No MR for key %#x\n", __func__, key);
>>>>> ret = -EINVAL;
>>>>> goto err;
>>>>> }
>>>>>
>>>>> - if (rkey != mr->rkey) {
>>>>> - pr_err("%s: rkey (%#x) doesn't match mr->rkey (%#x)\n",
>>>>> - __func__, rkey, mr->rkey);
>>>>> + if (mr->rkey ? (key != mr->rkey) : (key != mr->lkey)) {
>>>>> + pr_err("%s: wr key (%#x) doesn't match mr key (%#x)\n",
>>>>> + __func__, key, (mr->rkey ? mr->rkey : mr->lkey));
>>>>> ret = -EINVAL;
>>>>> goto err_drop_ref;
>>>>> }
>>>>
>>>> Ping.
>>>
>>> Ping.
>>>
>>> Does this need more discussions or changes?
>>
>> I was hoping Bob would say something but I am happy with it..
>
> Thanks Jason for the response.
> We can wait for Bob to share his thoughts.
>
>>
>> JasonFine by me
Reviewed-by: Bob Pearson <rpearsonhpe@gmail.com>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH] RDMA/rxe: For invalidate compare according to set keys in mr
2022-07-07 7:30 [PATCH] RDMA/rxe: For invalidate compare according to set keys in mr Md Haris Iqbal
2022-07-15 9:21 ` haris iqbal
@ 2022-07-22 20:38 ` Jason Gunthorpe
1 sibling, 0 replies; 7+ messages in thread
From: Jason Gunthorpe @ 2022-07-22 20:38 UTC (permalink / raw)
To: Md Haris Iqbal
Cc: linux-rdma, zyjzyj2000, leon, haris.iqbal, jinpu.wang,
aleksei.marov, rpearsonhpe
On Thu, Jul 07, 2022 at 09:30:06AM +0200, Md Haris Iqbal wrote:
> The 'rkey' input can be an lkey or rkey, and in rxe the lkey or rkey have
> the same value, including the variant bits.
>
> So, if mr->rkey is set, compare the invalidate key with it, otherwise
> compare with the mr->lkey.
>
> Since we already did a lookup on the non-varient bits to get this far,
> the check's only purpose is to confirm that the wqe has the correct
> variant bits.
>
> Fixes: 001345339f4c ("RDMA/rxe: Separate HW and SW l/rkeys")
> Cc: rpearsonhpe@gmail.com
> Signed-off-by: Md Haris Iqbal <haris.phnx@gmail.com>
> Reviewed-by: Bob Pearson <rpearsonhpe@gmail.com>
> ---
> drivers/infiniband/sw/rxe/rxe_loc.h | 2 +-
> drivers/infiniband/sw/rxe/rxe_mr.c | 12 ++++++------
> 2 files changed, 7 insertions(+), 7 deletions(-)
Applied to for-next, thanks
Jason
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-07-22 20:38 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-07-07 7:30 [PATCH] RDMA/rxe: For invalidate compare according to set keys in mr Md Haris Iqbal
2022-07-15 9:21 ` haris iqbal
2022-07-22 15:10 ` haris iqbal
2022-07-22 15:14 ` Jason Gunthorpe
2022-07-22 16:50 ` haris iqbal
2022-07-22 19:04 ` Bob Pearson
2022-07-22 20:38 ` Jason Gunthorpe
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.