Re: [PATCH v1 3/3] RDMA/rxe: Increase value of RXE_MAX_AH

[Bob Pearson <rpearsonhpe@gmail.com>]

On 6/17/21 5:57 PM, Shoaib Rao wrote:
> 
> On 6/17/21 1:45 PM, Bob Pearson wrote:
>> On 6/17/21 1:25 PM, Rao Shoaib wrote:
>>> From: Rao Shoaib <rao.shoaib@oracle.com>
>>>
>>> In our internal testing we have found that the
>>> current limit is too small, this patch bumps it
>>> up to a higher value required for our tests, which
>>> are indicative of our customer usage.
>>>
>>> Signed-off-by: Rao Shoaib <rao.shoaib@oracle.com>
>>> ---
>>>   drivers/infiniband/sw/rxe/rxe_param.h | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/infiniband/sw/rxe/rxe_param.h b/drivers/infiniband/sw/rxe/rxe_param.h
>>> index 3b9b1ff4234f..d375f2cff484 100644
>>> --- a/drivers/infiniband/sw/rxe/rxe_param.h
>>> +++ b/drivers/infiniband/sw/rxe/rxe_param.h
>>> @@ -66,7 +66,7 @@ enum rxe_device_param {
>>>       RXE_MAX_MCAST_GRP        = 8192,
>>>       RXE_MAX_MCAST_QP_ATTACH        = 56,
>>>       RXE_MAX_TOT_MCAST_QP_ATTACH    = 0x70000,
>>> -    RXE_MAX_AH            = 100,
>>> +    RXE_MAX_AH            = 64000,
>>>       RXE_MAX_SRQ            = 17408,
>>>       RXE_MAX_SRQ_WR            = 0x4000,
>>>       RXE_MIN_SRQ_WR            = 1,
>>>
>> Interesting. There is no real reason to pick most of these values since it's just memory and does not reflect underlying hardware resources. It is advantageous to also be able to set them smaller to verify whether test cases correctly limit resources. It seems that there should be a way (module parameter or other) to adjust these values without having to recompile the driver. Thoughts?
> 
> I agree with you 100% but it seems like the original design did not intent to make them configurable at run time. I see that recently some other values were bumped up by others so I went with hard coded values. In the inhouse kernel version we used to decide on the values had them configurable because we did not know what values we wanted, and recompiling was time consuming.
> 
> While some tests may require small values we want to test with lots of users/connections.
> 
> If there is consensus I can submit a different patch to make these values configurable. In the meantime I hope we can bump up these values.
> 
> Kindly let me know.
> 
> Shoaib
> 
>>
>> Regards,
>>
>> Bob Pearson

It isn't my call. But I am in favor of tunable parameters. -- Bob Pearson