Perftest fails when declaring support for MW_TYPE_2A

Perftest fails when declaring support for MW_TYPE_2A

[Kalderon, Michal]

Hi Mellanox/ Accelio, 

There is a very old commit in perftest 96050b8b that assumes bit 23 in device cap flags 
means there is driver support for contiguous buffer allocation of user data

There is a clash with the following device capability defined in libibverbs/verbs.h: 
#define IBV_DEVICE_MEM_WINDOW_TYPE_2A = 1 << 23

Once driver advertises memory window type 2a capability, the perftest application fails. 

I found that IB_DEVICE_MR_ALLOCATE is defined as bit 23 for freebsd git hub and accelio github
And then I saw there are a lot of clashes between ib_verbs + upstream kernel

Clash between the two needs to be resolved. 
Perftest wise: Perhaps an additional parameter to perftest ? 

Please advise, 
Thanks,
Michal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Perftest fails when declaring support for MW_TYPE_2A

[Zohar Ben Aharon]

Thanks Michal ,

Could you provide the package you were using ? 
Case you working on our git , Do you know in which it was broken ? 

Could you also add more information regarding the driver and device you were using, and the command that you try to execute and caused that clash ? 



Thanks, 

Zohar 


-----Original Message-----
From: Kalderon, Michal [mailto:Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org] 
Sent: Wednesday, November 1, 2017 5:36 PM
To: Zohar Ben Aharon <zoharb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>; Gil Rockah <gilr-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>; lunnys-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; Ido Shamay <idos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Elior, Ariel <Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>; leon@kernel.org; Amrani, Ram <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Subject: Perftest fails when declaring support for MW_TYPE_2A

Hi Mellanox/ Accelio, 

There is a very old commit in perftest 96050b8b that assumes bit 23 in device cap flags means there is driver support for contiguous buffer allocation of user data

There is a clash with the following device capability defined in libibverbs/verbs.h: 
#define IBV_DEVICE_MEM_WINDOW_TYPE_2A = 1 << 23

Once driver advertises memory window type 2a capability, the perftest application fails. 

I found that IB_DEVICE_MR_ALLOCATE is defined as bit 23 for freebsd git hub and accelio github And then I saw there are a lot of clashes between ib_verbs + upstream kernel

Clash between the two needs to be resolved. 
Perftest wise: Perhaps an additional parameter to perftest ? 

Please advise,
Thanks,
Michal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Perftest fails when declaring support for MW_TYPE_2A

[Kalderon, Michal]

Hi Zohar, 

Thanks for the quick response. I'm working on your git.
I don't think it was broken, more that perhaps it never worked over devices
that advertise MEM_WINDOW_TYPE_2A

>From a quick grep I see that the only ones that advertise TYPE_2A are mlx4 and only under certain
Conditions. 

I'm currently adding support for MW_TYPE_2A for our qedr driver and that's how I 
Got the perftest failure. 

Ran ib_write_bw with all default values
Got the error: 
Couldn't allocate MR
Failed to create mr
... 

Thanks,
Michal

> -----Original Message-----
> From: Zohar Ben Aharon [mailto:zoharb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org]
> Sent: Wednesday, November 01, 2017 5:47 PM
> To: Kalderon, Michal <Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>; Gil Rockah
> <gilr-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>; lunnys-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; Ido Shamay
> <idos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Elior, Ariel <Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>;
> leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org; Amrani, Ram <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> Subject: RE: Perftest fails when declaring support for MW_TYPE_2A
> 
> Thanks Michal ,
> 
> Could you provide the package you were using ?
> Case you working on our git , Do you know in which it was broken ?
> 
> Could you also add more information regarding the driver and device you
> were using, and the command that you try to execute and caused that clash
> ?
> 
> 
> 
> Thanks,
> 
> Zohar
> 
> 
> -----Original Message-----
> From: Kalderon, Michal [mailto:Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org]
> Sent: Wednesday, November 1, 2017 5:36 PM
> To: Zohar Ben Aharon <zoharb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>; Gil Rockah
> <gilr-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>; lunnys-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; Ido Shamay
> <idos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Elior, Ariel <Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>;
> leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org; Amrani, Ram <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> Subject: Perftest fails when declaring support for MW_TYPE_2A
> 
> Hi Mellanox/ Accelio,
> 
> There is a very old commit in perftest 96050b8b that assumes bit 23 in device
> cap flags means there is driver support for contiguous buffer allocation of
> user data
> 
> There is a clash with the following device capability defined in
> libibverbs/verbs.h:
> #define IBV_DEVICE_MEM_WINDOW_TYPE_2A = 1 << 23
> 
> Once driver advertises memory window type 2a capability, the perftest
> application fails.
> 
> I found that IB_DEVICE_MR_ALLOCATE is defined as bit 23 for freebsd git hub
> and accelio github And then I saw there are a lot of clashes between ib_verbs
> + upstream kernel
> 
> Clash between the two needs to be resolved.
> Perftest wise: Perhaps an additional parameter to perftest ?
> 
> Please advise,
> Thanks,
> Michal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Perftest fails when declaring support for MW_TYPE_2A

[Jason Gunthorpe]

On Wed, Nov 01, 2017 at 03:35:54PM +0000, Kalderon, Michal wrote:
> Hi Mellanox/ Accelio, 
> 
> There is a very old commit in perftest 96050b8b that assumes bit 23
> in device cap flags means there is driver support for contiguous
> buffer allocation of user data

Wow.

This is

https://github.com/linux-rdma/perftest/blob/master/src/perftest_resources.c#L272

And it is totally unacceptable for upstream'ed open source code to
have crap like this.

Delete it.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Perftest fails when declaring support for MW_TYPE_2A

[Kalderon, Michal]

> From: Jason Gunthorpe [mailto:jgg-uk2M96/98Pc@public.gmane.org]
> Sent: Wednesday, November 01, 2017 6:45 PM
> To: Kalderon, Michal <Michal.Kalderon-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> Cc: Zohar Ben Aharon <zoharb-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>; Gil Rockah
> <gilr-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>; lunnys-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org; idos-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org; linux-
> rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Elior, Ariel <Ariel.Elior-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>;
> leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org; Amrani, Ram <Ram.Amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
> Subject: Re: Perftest fails when declaring support for MW_TYPE_2A
> 
> On Wed, Nov 01, 2017 at 03:35:54PM +0000, Kalderon, Michal wrote:
> > Hi Mellanox/ Accelio,
> >
> > There is a very old commit in perftest 96050b8b that assumes bit 23 in
> > device cap flags means there is driver support for contiguous buffer
> > allocation of user data
> 
> Wow.
> 
> This is
> 
> https://github.com/linux-
> rdma/perftest/blob/master/src/perftest_resources.c#L272
> 
> And it is totally unacceptable for upstream'ed open source code to have crap
> like this.
> 
> Delete it.
> 
> Jason
Zohar, 

Can you comment? Would you like me to provide a patch ? or would you rather? 
Thanks,
Michal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html