* Fwd: Bug in pyverbs for test_qp_ex_rc_bind_mw
[not found] <9c9bbda0-4134-207c-96cb-eb594aab40d4@gmail.com>
@ 2022-05-18 3:41 ` Bob Pearson
2022-05-19 3:04 ` lizhijian
0 siblings, 1 reply; 4+ messages in thread
From: Bob Pearson @ 2022-05-18 3:41 UTC (permalink / raw)
To: linux-rdma
-------- Forwarded Message --------
Subject: Re: Bug in pyverbs for test_qp_ex_rc_bind_mw
Date: Tue, 17 May 2022 22:41:08 -0500
From: Bob Pearson <rpearsonhpe@gmail.com>
To: Jason Gunthorpe <jgg@nvidia.com>, Zhu Yanjun <zyjzyj2000@gmail.com>, Edward Srouji <edwards@nvidia.com>, Leon Romanovsky <leon@kernel.org>
On 5/17/22 21:57, Bob Pearson wrote:
> test_qp_ex_rc_bind_mw has an error in that the new_rkey is computed from the old mr rkey and not the old mw rkey.
> The following lines
>
> mw = MW(server.pd, mw_type=e.IBV_MW_TYPE_2)
> ...
> new_rkey = inc_rkey(server.mr.rkey)
> server.qp.wr_bind_mw(mw, new_rkey, bind_info)
>
> will compute a new rkey with the same index as mr and a key portion that is one larger than mr modulo 256.
> This is passed to wr_bind_mw which expects a parameter with a new key portion of the mw (not the mr).
> The memory windows implementation in rxe generates a random initial rkey for mw and for bind_mw it
> checks that the new 8 bit key is different than the old key. Since the mr and mw are random wrt each other
> we expect that the new key will match the old key approx 1 out of 256 test runs which will cause an error
> which is just what I see.
>
> The correct code should be
>
> new_key = inc_rkey(<old mw.rkey>)
>
> which will guarantee that it is always different than the previous key. The problem is I can't figure out
> how to compute the rkey from the mw or I would submit a patch.
>
> Bob
>
If in test_qpex.py I type
print("mw = ", mw)
print("mr = ", self.server.mr)
I get
mw = MW:
Rkey : 12345678
Handle : 4
MW Type : IBV_MW_TYPE_2
mr = MR
lkey : 432134
rkey : 432134
length : 1024
buf : 9403912345678
handle : 2
The difference is the colon ':' after MW and caps.
I can refer to mr.rkey as self.server.mr.rkey no problem
but mw.Rkey doesn't work. Neither does mw.rkey or anything else I have thought of.
I hate python. Just hate it.
Bob
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Fwd: Bug in pyverbs for test_qp_ex_rc_bind_mw
2022-05-18 3:41 ` Fwd: Bug in pyverbs for test_qp_ex_rc_bind_mw Bob Pearson
@ 2022-05-19 3:04 ` lizhijian
2022-05-19 12:22 ` Edward Srouji
0 siblings, 1 reply; 4+ messages in thread
From: lizhijian @ 2022-05-19 3:04 UTC (permalink / raw)
To: Bob Pearson, linux-rdma
On 18/05/2022 11:41, Bob Pearson wrote:
>
>
> -------- Forwarded Message --------
> Subject: Re: Bug in pyverbs for test_qp_ex_rc_bind_mw
> Date: Tue, 17 May 2022 22:41:08 -0500
> From: Bob Pearson <rpearsonhpe@gmail.com>
> To: Jason Gunthorpe <jgg@nvidia.com>, Zhu Yanjun <zyjzyj2000@gmail.com>, Edward Srouji <edwards@nvidia.com>, Leon Romanovsky <leon@kernel.org>
>
> On 5/17/22 21:57, Bob Pearson wrote:
>> test_qp_ex_rc_bind_mw has an error in that the new_rkey is computed from the old mr rkey and not the old mw rkey.
>> The following lines
>>
>> mw = MW(server.pd, mw_type=e.IBV_MW_TYPE_2)
>> ...
>> new_rkey = inc_rkey(server.mr.rkey)
>> server.qp.wr_bind_mw(mw, new_rkey, bind_info)
>>
>> will compute a new rkey with the same index as mr and a key portion that is one larger than mr modulo 256.
>> This is passed to wr_bind_mw which expects a parameter with a new key portion of the mw (not the mr).
>> The memory windows implementation in rxe generates a random initial rkey for mw and for bind_mw it
>> checks that the new 8 bit key is different than the old key. Since the mr and mw are random wrt each other
>> we expect that the new key will match the old key approx 1 out of 256 test runs which will cause an error
>> which is just what I see.
>>
>> The correct code should be
>>
>> new_key = inc_rkey(<old mw.rkey>)
>>
>> which will guarantee that it is always different than the previous key. The problem is I can't figure out
>> how to compute the rkey from the mw or I would submit a patch.
>>
>> Bob
>>
> If in test_qpex.py I type
>
> print("mw = ", mw)
> print("mr = ", self.server.mr)
>
> I get
>
> mw = MW:
> Rkey : 12345678
> Handle : 4
> MW Type : IBV_MW_TYPE_2
>
> mr = MR
> lkey : 432134
> rkey : 432134
> length : 1024
> buf : 9403912345678
> handle : 2
>
> The difference is the colon ':' after MW and caps.
For the difference, just post a RP to make them identical: https://github.com/linux-rdma/rdma-core/pull/1175
Thanks
Zhijian
>
> I can refer to mr.rkey as self.server.mr.rkey no problem
>
> but mw.Rkey doesn't work. Neither does mw.rkey or anything else I have thought of.
>
> I hate python. Just hate it.
>
> Bob
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Fwd: Bug in pyverbs for test_qp_ex_rc_bind_mw
2022-05-19 3:04 ` lizhijian
@ 2022-05-19 12:22 ` Edward Srouji
2022-05-19 16:04 ` Bob Pearson
0 siblings, 1 reply; 4+ messages in thread
From: Edward Srouji @ 2022-05-19 12:22 UTC (permalink / raw)
To: lizhijian, Bob Pearson, linux-rdma
On 5/19/2022 6:04 AM, lizhijian@fujitsu.com wrote:
>
> On 18/05/2022 11:41, Bob Pearson wrote:
>>
>> -------- Forwarded Message --------
>> Subject: Re: Bug in pyverbs for test_qp_ex_rc_bind_mw
>> Date: Tue, 17 May 2022 22:41:08 -0500
>> From: Bob Pearson <rpearsonhpe@gmail.com>
>> To: Jason Gunthorpe <jgg@nvidia.com>, Zhu Yanjun <zyjzyj2000@gmail.com>, Edward Srouji <edwards@nvidia.com>, Leon Romanovsky <leon@kernel.org>
>>
>> On 5/17/22 21:57, Bob Pearson wrote:
>>> test_qp_ex_rc_bind_mw has an error in that the new_rkey is computed from the old mr rkey and not the old mw rkey.
>>> The following lines
>>>
>>> mw = MW(server.pd, mw_type=e.IBV_MW_TYPE_2)
>>> ...
>>> new_rkey = inc_rkey(server.mr.rkey)
>>> server.qp.wr_bind_mw(mw, new_rkey, bind_info)
>>>
>>> will compute a new rkey with the same index as mr and a key portion that is one larger than mr modulo 256.
>>> This is passed to wr_bind_mw which expects a parameter with a new key portion of the mw (not the mr).
>>> The memory windows implementation in rxe generates a random initial rkey for mw and for bind_mw it
>>> checks that the new 8 bit key is different than the old key. Since the mr and mw are random wrt each other
>>> we expect that the new key will match the old key approx 1 out of 256 test runs which will cause an error
>>> which is just what I see.
>>>
>>> The correct code should be
>>>
>>> new_key = inc_rkey(<old mw.rkey>)
>>>
>>> which will guarantee that it is always different than the previous key. The problem is I can't figure out
>>> how to compute the rkey from the mw or I would submit a patch.
>>>
>>> Bob
>>>
>> If in test_qpex.py I type
>>
>> print("mw = ", mw)
>> print("mr = ", self.server.mr)
>>
>> I get
>>
>> mw = MW:
>> Rkey : 12345678
>> Handle : 4
>> MW Type : IBV_MW_TYPE_2
>>
>> mr = MR
>> lkey : 432134
>> rkey : 432134
>> length : 1024
>> buf : 9403912345678
>> handle : 2
>>
>> The difference is the colon ':' after MW and caps.
> For the difference, just post a RP to make them identical: https://github.com/linux-rdma/rdma-core/pull/1175
Thanks for the adjustment.
> Thanks
> Zhijian
>
>> I can refer to mr.rkey as self.server.mr.rkey no problem
>>
>> but mw.Rkey doesn't work. Neither does mw.rkey or anything else I have thought of.
mw.rkey should work... I noticed that you've already figured that out
and sent a patch.
We expose the MW rkey in pyverbs (within the cython code).
If you still have an issue with it please let me know.
>>
>> I hate python. Just hate it.
Don't hate the game, hate the player :) (in this case).
>> Bob
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Fwd: Bug in pyverbs for test_qp_ex_rc_bind_mw
2022-05-19 12:22 ` Edward Srouji
@ 2022-05-19 16:04 ` Bob Pearson
0 siblings, 0 replies; 4+ messages in thread
From: Bob Pearson @ 2022-05-19 16:04 UTC (permalink / raw)
To: Edward Srouji, lizhijian, linux-rdma
On 5/19/22 07:22, Edward Srouji wrote:
>
> On 5/19/2022 6:04 AM, lizhijian@fujitsu.com wrote:
>>
>> On 18/05/2022 11:41, Bob Pearson wrote:
>>>
>>> -------- Forwarded Message --------
>>> Subject: Re: Bug in pyverbs for test_qp_ex_rc_bind_mw
>>> Date: Tue, 17 May 2022 22:41:08 -0500
>>> From: Bob Pearson <rpearsonhpe@gmail.com>
>>> To: Jason Gunthorpe <jgg@nvidia.com>, Zhu Yanjun <zyjzyj2000@gmail.com>, Edward Srouji <edwards@nvidia.com>, Leon Romanovsky <leon@kernel.org>
>>>
>>> On 5/17/22 21:57, Bob Pearson wrote:
>>>> test_qp_ex_rc_bind_mw has an error in that the new_rkey is computed from the old mr rkey and not the old mw rkey.
>>>> The following lines
>>>>
>>>> mw = MW(server.pd, mw_type=e.IBV_MW_TYPE_2)
>>>> ...
>>>> new_rkey = inc_rkey(server.mr.rkey)
>>>> server.qp.wr_bind_mw(mw, new_rkey, bind_info)
>>>>
>>>> will compute a new rkey with the same index as mr and a key portion that is one larger than mr modulo 256.
>>>> This is passed to wr_bind_mw which expects a parameter with a new key portion of the mw (not the mr).
>>>> The memory windows implementation in rxe generates a random initial rkey for mw and for bind_mw it
>>>> checks that the new 8 bit key is different than the old key. Since the mr and mw are random wrt each other
>>>> we expect that the new key will match the old key approx 1 out of 256 test runs which will cause an error
>>>> which is just what I see.
>>>>
>>>> The correct code should be
>>>>
>>>> new_key = inc_rkey(<old mw.rkey>)
>>>>
>>>> which will guarantee that it is always different than the previous key. The problem is I can't figure out
>>>> how to compute the rkey from the mw or I would submit a patch.
>>>>
>>>> Bob
>>>>
>>> If in test_qpex.py I type
>>>
>>> print("mw = ", mw)
>>> print("mr = ", self.server.mr)
>>>
>>> I get
>>>
>>> mw = MW:
>>> Rkey : 12345678
>>> Handle : 4
>>> MW Type : IBV_MW_TYPE_2
>>>
>>> mr = MR
>>> lkey : 432134
>>> rkey : 432134
>>> length : 1024
>>> buf : 9403912345678
>>> handle : 2
>>>
>>> The difference is the colon ':' after MW and caps.
>> For the difference, just post a RP to make them identical: https://github.com/linux-rdma/rdma-core/pull/1175
> Thanks for the adjustment.
>> Thanks
>> Zhijian
>>
>>> I can refer to mr.rkey as self.server.mr.rkey no problem
>>>
>>> but mw.Rkey doesn't work. Neither does mw.rkey or anything else I have thought of.
>
> mw.rkey should work... I noticed that you've already figured that out and sent a patch.
I can't believe I didn't try that. It was 1 AM and I was tired. A nights sleep resolved the
issue and reading the pyx code.
Bob
>
> We expose the MW rkey in pyverbs (within the cython code).
> If you still have an issue with it please let me know.
>
>>>
>>> I hate python. Just hate it.
> Don't hate the game, hate the player :) (in this case).
>>> Bob
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-05-19 16:04 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <9c9bbda0-4134-207c-96cb-eb594aab40d4@gmail.com>
2022-05-18 3:41 ` Fwd: Bug in pyverbs for test_qp_ex_rc_bind_mw Bob Pearson
2022-05-19 3:04 ` lizhijian
2022-05-19 12:22 ` Edward Srouji
2022-05-19 16:04 ` Bob Pearson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.