Re: [PATCH for-next 3/6] RDMA/rxe: Save object pointer in pool element

[Bob Pearson <rpearsonhpe@gmail.com>]

On 10/20/21 6:20 PM, Jason Gunthorpe wrote:
> On Sun, Oct 10, 2021 at 06:59:28PM -0500, Bob Pearson wrote:
>> In rxe_pool.c currently there are many cases where it is necessary to
>> compute the offset from a pool element struct to the object containing
>> the pool element in a type independent way. By saving a pointer to the
>> object when they are created extra work can be saved.
>>
>> Signed-off-by: Bob Pearson <rpearsonhpe@gmail.com>
>> ---
>>  drivers/infiniband/sw/rxe/rxe_pool.c | 16 +++++++++-------
>>  drivers/infiniband/sw/rxe/rxe_pool.h |  1 +
>>  2 files changed, 10 insertions(+), 7 deletions(-)
> 
> This would be better to just add a static_assert or build_bug_on that
> the offsetof() the rxe_pool_entry == 0. There is no reason for these
> to be sprinkled at every offset
> 
> Then you don't need to do anything at all
> 
> Jason
> 
I think you missed something. Once upon a time all the rxe objects had the local
pool entry struct followed by the ib_xx struct and then anything else needed locally.
As Leon has been moving the allocations to rdma-core he had to make the ib_xx struct 
first followed by the pool element. So there is a mish mash of offsets. Some
are/were 0, and the rest are all different depending on the size of the struct from
rdma-core. E.g. sizeof(struct ib_mr) != sizeof(struct ib_qp), etc. In order to make
the API simple and consistent I use a macro to convert from object to pool elem. e.g.

	#define some_function(void *obj) __some_function(&obj->elem)

but to convert back from elem to obj we need to subtract some random offset e.g.

	obj = elem->obj;

without knowing the type of obj which is simpler than what we had before, roughly

	struct rxe_pool_info *info = &pool_info[elem->pool->type];

	obj = (u8 *)(elem - info->elem_offset);