[ANNOUNCE] OFED 4.8-rc2 release is available

[ANNOUNCE] OFED 4.8-rc2 release is available

[Vladimir Sokolovsky]

Hi,
OFED-4.8-rc2 is available at:
http://openfabrics.org/downloads/OFED/ofed-4.8/OFED-4.8-rc2.tgz

To get BUILD_ID run ofed_info

Please report any issues in bugzilla
http://bugs.openfabrics.org/bugzilla/ for OFED 4.8

Release notes:
https://openfabrics.org/downloads/OFED/release_notes/OFED_4.8-rc2_release_notes

OFED-4.8-rc2 Main Changes from OFED 4.8-rc1
-------------------------------------------------------------------------------
1. compat-rdma:
- iw_cxgb4: Guard against null cm_id in dump_ep/qp
- Xeon Phi updates
- Fixes for i40iw which have been included in kernels > 4.8
- Updated 0003-add-the-ibp-client-and-server-drivers.patch

2. Updated packages:
- rdma-core v13
- libfabric-1.4.1
- fabtests-1.4.1
- libibscif-1.1.3

Regards,
Vladimir
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Christoph Hellwig]

> - Xeon Phi updates

This driver seems to never have been posted on linux-rdma, neither
for the kernel, nor for rdma-core.  How did it end up in your backports?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Christoph Hellwig]

On Thu, Apr 27, 2017 at 12:06:57AM -0700, Christoph Hellwig wrote:
> > - Xeon Phi updates
> 
> This driver seems to never have been posted on linux-rdma, neither
> for the kernel, nor for rdma-core.  How did it end up in your backports?

ping?  How does OFED manage to include hardware support (for a long time
apparently!) that's never seen any upstream exposure?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Woodruff, Robert J]

Christoph wrote,
>ping?  How does OFED manage to include hardware support (for a long time
>apparently!) that's never seen any upstream exposure?

OFED allows code that is not upstream to be included as technology preview (not built by default) to allow people to use it. 
In the case of the Xeon-Phi code that uses peer to peer PCI transfers, it is not clear the Linux community would every accept it the way it is now.
I saw on the list a year or so back some patches that Mellanox submitted to allow support for PCI peer to peer transfers like what the Xeon Phi code
needs, and they got puked all over. So until that gets all worked out in Linux, I suspect this code will have to stay out of tree. 
_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/mailman/listinfo/ewg

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Christoph Hellwig]

So in plain words: Intel abuses their influence in OFED to ship crap
that has absolutely no chance to get upstream in the current form
instead of working with the community to improve infrastructure.

That's exactly what I guessed, thanks for confirming.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Woodruff, Robert J]

Just because the code is not upstream does not mean it is crap, it just might be coded in a way that the kernel people would like done differently. 
Nvidia drivers are not upstream either, but lots of people use them.

-----Original Message-----
From: Christoph Hellwig [mailto:hch@infradead.org] 
Sent: Thursday, May 04, 2017 7:50 AM
To: Woodruff, Robert J <robert.j.woodruff@intel.com>
Cc: Christoph Hellwig <hch@infradead.org>; Vladimir Sokolovsky <vlad@dev.mellanox.co.il>; ewg@lists.openfabrics.org; linux-rdma@vger.kernel.org
Subject: Re: [ANNOUNCE] OFED 4.8-rc2 release is available

So in plain words: Intel abuses their influence in OFED to ship crap that has absolutely no chance to get upstream in the current form instead of working with the community to improve infrastructure.

That's exactly what I guessed, thanks for confirming.
_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/mailman/listinfo/ewg

RE: [ANNOUNCE] OFED 4.8-rc2 release is available

[Hefty, Sean]

> So in plain words: Intel abuses their influence in OFED to ship crap
> that has absolutely no chance to get upstream in the current form
> instead of working with the community to improve infrastructure.
> 
> That's exactly what I guessed, thanks for confirming.

OFED is a software product of OFA, not Linux.  OFA can put anything that they want in it.  Why do you even care?  It's no different than Intel or Mellanox or any other company shipping out of tree software.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

On Thu, May 04, 2017 at 05:19:47PM +0000, Hefty, Sean wrote:
> > So in plain words: Intel abuses their influence in OFED to ship crap
> > that has absolutely no chance to get upstream in the current form
> > instead of working with the community to improve infrastructure.
> >
> > That's exactly what I guessed, thanks for confirming.
>
> OFED is a software product of OFA, not Linux.  OFA can put anything that they want in it.  Why do you even care?  It's no different than Intel or Mellanox or any other company shipping out of tree software.

It is different. Mellanox doesn't claim that MLNX_OFED is official
source package for OpenFabrics Alliance and their members.
https://www.openfabrics.org/index.php/openfabrics-software.html


> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Doug Ledford]

[-- Attachment #1.1.1: Type: text/plain, Size: 2755 bytes --]

[linux-rdma@ was accidentally dropped on my email, so readding it on
this response]

On 5/4/2017 2:44 PM, Hefty, Sean wrote:
>>> OFED is a software product of OFA, not Linux.  OFA can put anything
>>> that they want in it.  Why do you even care?  It's no different than
>>> Intel or Mellanox or any other company shipping out of tree
>>> software.
>>
>> The primary answer to your question depends on whether or not the
>> software will ever be upstreamed.  If it will, then it really should
>> go
>> there first and not later, and the reason is well exemplified by what
>> happened with XRC where the version that landed in OFED and the
>> version
>> that landed in upstream were two totally different things, and users
>> had
>> to go back and fix up all their code because of the difference once it
>> finally did land upstream.  It's not nice to put users in that
>> position
>> again, and this does sound like it might end up going down that exact
>> road since upstream is pursuing ways of doing peer to peer PCI
>> operations and such without any input from the Xeon Phi folks.
> 
> I'm not defending whatever business decisions any organization (including a multi-company non-profit like OFA) wants to make wrt their software distributions.  I'm claiming that that's their decision.
> 

I'm not sure I agree with that position.  A company has the right to
decide for themselves what to do.  An organization like the OFA is
different, in that it is based upon a collective agreement entered into
by multiple parties with certain specific stated intents and goals
written out in bylaws (although we all know the OFA is already in
violation of those at the moment, let's just assume they aren't for the
purposes of this conversation).  If the organization then takes to
violating those bylaws, it essentially becomes in breach of contract to
itself and all members that originally agreed to those bylaws to my lay
persons legal mind.  I would say that does give people (at a minimum,
any member of the organization who entered into this agreement under
different pretenses, but possibly also to non-member entities affected
by the actions of the organization) grounds to complain.

But all that aside, the OFA at least has a pretense of wanting to get
along with the upstream linux community.  As long as they want to
preserve that relationship, then they should listen when the community
has something to say.  It might well be their decision, but the
ramifications of that decision might sabotage their other interests.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
    GPG Key ID: B826A3330E572FDD
    Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

[-- Attachment #2: Type: text/plain, Size: 140 bytes --]

_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/mailman/listinfo/ewg

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Leon Romanovsky]

[-- Attachment #1.1: Type: text/plain, Size: 642 bytes --]

On Thu, May 04, 2017 at 07:50:26AM -0700, Christoph Hellwig wrote:
> So in plain words: Intel abuses their influence in OFED to ship crap
> that has absolutely no chance to get upstream in the current form
> instead of working with the community to improve infrastructure.
>
> That's exactly what I guessed, thanks for confirming.

Yeah, it is great example why OFA, OFED and all their OF* groups should disappear.

> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 140 bytes --]

_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/mailman/listinfo/ewg

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Jason Gunthorpe]

On Thu, May 04, 2017 at 02:28:26PM +0000, Woodruff, Robert J wrote:
> Christoph wrote,
> >ping?  How does OFED manage to include hardware support (for a long time
> >apparently!) that's never seen any upstream exposure?
> 
> OFED allows code that is not upstream to be included as technology
> preview (not built by default) to allow people to use it.  In the
> case of the Xeon-Phi code that uses peer to peer PCI transfers, it
> is not clear the Linux community would every accept it the way it is
> now.

.. and yet I haven't really seen that group in Intel participating in
the p2p discussions to design something that is mergable..

This is exactly why we so strongly discourage this out of tree stuff -
getting something unmergable in OFED is *NOT* Job Done, Time to Go
Home. Down this path just creates another Lustre mess.

Dan Williams <dan.j.williams@intel.com> has been actively
participating in the p2p discussions, perhaps you can arrange some
internal collaboration..

Jason
_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/mailman/listinfo/ewg

RE: [ewg] [ANNOUNCE] OFED 4.8-rc2 release is available

[Hefty, Sean]

> This is exactly why we so strongly discourage this out of tree stuff -
> getting something unmergable in OFED is *NOT* Job Done, Time to Go
> Home. Down this path just creates another Lustre mess.

Actually, this may be 'job done'.  No individual or company is obligated to provide upstream software for any of their hardware.  OFA decides what to ship in their software products, not the greater linux kernel community.  Individual companies can decide if out of tree maintenance is more cost effective than trying to merge code upstream.  Because that's what this ultimately comes down to.

IMO, the only people who have legitimate complaints here are those people running Xeon Phi with Mellanox HCAs who are being forced to use OFED, rather than upstream code.

Re: [ewg] [ANNOUNCE] OFED 4.8-rc2 release is available

[Jason Gunthorpe]

On Thu, May 04, 2017 at 05:34:35PM +0000, Hefty, Sean wrote:
> > This is exactly why we so strongly discourage this out of tree stuff -
> > getting something unmergable in OFED is *NOT* Job Done, Time to Go
> > Home. Down this path just creates another Lustre mess.
> 
> Actually, this may be 'job done'.  No individual or company is
> obligated to provide upstream software for any of their hardware.
> OFA decides what to ship in their software products, not the greater
> linux kernel community.

The founding imputus for OFA was to get this RDMA code upstream, so
OFA should not be spending their resources supporting non-upstream
works.

> Individual companies can decide if out of tree maintenance is more
> cost effective than trying to merge code upstream.  Because that's
> what this ultimately comes down to.

Sure. Outside the OFA and its works please.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Paul Grun]

>
>The founding imputus for OFA was to get this RDMA code upstream, so OFA
>should not be spending their resources supporting non-upstream works.
>
>
>Jason
>--

That's actually not true.  The founding impetus was to prevent the nascent InfiniBand industry from fracturing under the weight of multiple software implementations of the verbs semantics described in the IB specs.  The solution to that was to provide a single software stack provided by the Alliance.

-Paul
_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/mailman/listinfo/ewg

Re: [ewg] [ANNOUNCE] OFED 4.8-rc2 release is available

[Jason Gunthorpe]

On Thu, May 04, 2017 at 07:07:02PM +0000, Paul Grun wrote:
> >The founding imputus for OFA was to get this RDMA code upstream, so OFA
> >should not be spending their resources supporting non-upstream works.
> 
> That's actually not true.  The founding impetus was to prevent the
> nascent InfiniBand industry from fracturing under the weight of
> multiple software implementations of the verbs semantics described
> in the IB specs.  The solution to that was to provide a single
> software stack provided by the Alliance.

I think you are splitting hairs, from the start alliance members
focused their efforts upstream (eg Roland getting Linus to accept
drivers/infiniband).

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Coulter, Susan K]

The unpleasant fact is that the OFA has not been deeply involved, as an organization, in the development and distribution of OFED.
That may seem counter-intuitive, and it is certainly not appropriate, but it is the raw truth.
The EWG manages, tests, and releases OFED versions.
Until recently, that was all done without official review or approval from the larger OFA constituency.

There are many issues the OFA needs to get straight if it is to continue to contribute to the community in a positive way.
( and I fully acccept some folks would posit that the OFA has never contributed in a positive way )
This is one of them.


________________________________________
From: linux-rdma-owner@vger.kernel.org <linux-rdma-owner@vger.kernel.org> on behalf of Jason Gunthorpe <jgunthorpe@obsidianresearch.com>
Sent: Thursday, May 4, 2017 2:17 PM
To: Paul Grun
Cc: Hefty, Sean; Woodruff, Robert J; Christoph Hellwig; linux-rdma@vger.kernel.org; ewg@lists.openfabrics.org
Subject: Re: [ewg] [ANNOUNCE] OFED 4.8-rc2 release is available

On Thu, May 04, 2017 at 07:07:02PM +0000, Paul Grun wrote:
> >The founding imputus for OFA was to get this RDMA code upstream, so OFA
> >should not be spending their resources supporting non-upstream works.
>
> That's actually not true.  The founding impetus was to prevent the
> nascent InfiniBand industry from fracturing under the weight of
> multiple software implementations of the verbs semantics described
> in the IB specs.  The solution to that was to provide a single
> software stack provided by the Alliance.

I think you are splitting hairs, from the start alliance members
focused their efforts upstream (eg Roland getting Linus to accept
drivers/infiniband).

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/mailman/listinfo/ewg

Re: [ewg] [ANNOUNCE] OFED 4.8-rc2 release is available

[Coulter, Susan K]

________________________________________
From: linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org <linux-rdma-owner-u79uwXL29TY76Z2rM5mHXA@public.gmane.org> on behalf of Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
Sent: Thursday, May 4, 2017 2:17 PM
To: Paul Grun
Cc: Hefty, Sean; Woodruff, Robert J; Christoph Hellwig; linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; ewg-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org
Subject: Re: [ewg] [ANNOUNCE] OFED 4.8-rc2 release is available

On Thu, May 04, 2017 at 07:07:02PM +0000, Paul Grun wrote:
> >The founding imputus for OFA was to get this RDMA code upstream, so OFA
> >should not be spending their resources supporting non-upstream works.
>
> That's actually not true.  The founding impetus was to prevent the
> nascent InfiniBand industry from fracturing under the weight of
> multiple software implementations of the verbs semantics described
> in the IB specs.  The solution to that was to provide a single
> software stack provided by the Alliance.

I think you are splitting hairs, from the start alliance members
focused their efforts upstream (eg Roland getting Linus to accept
drivers/infiniband).

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [ewg] [ANNOUNCE] OFED 4.8-rc2 release is available

[Doug Ledford]

[-- Attachment #1.1: Type: text/plain, Size: 2270 bytes --]

On 5/4/2017 1:34 PM, Hefty, Sean wrote:
>> This is exactly why we so strongly discourage this out of tree
>> stuff - getting something unmergable in OFED is *NOT* Job Done,
>> Time to Go Home. Down this path just creates another Lustre mess.
> 
> Actually, this may be 'job done'.  No individual or company is
> obligated to provide upstream software for any of their hardware.

Very true, but in that case you would expect Intel to be shipping the
software, not OFA.  Much like they already do with their compiler, their
MPI, etc.

> OFA decides what to ship in their software products, not the greater
> linux kernel community.  Individual companies can decide if out of
> tree maintenance is more cost effective than trying to merge code
> upstream.  Because that's what this ultimately comes down to.

The OFA has already learned their lesson once with XRC.  I wonder if
they are getting ready to get hit with another lesson over this.  The
issue is if the OFA ships an API and it isn't upstream, then that API
needs to legitimately be a private, should never conflict with upstream
API.  If upstream then implements the same thing in a different way and
users are exposed to the rather unpleasant choice of code to OFA's API
or to upstream's API for the same thing, it creates a schism in the
user's code base around what API to use/support.  This is entirely
contrary to the OFA's stated goals about the end user experience of
people using their RDMA software.  So Intel can certainly make any cost
analysis they want about their hardware and the software to support it,
but the OFA is not Intel's personal software distributor and the OFA
must look at other, bigger picture issues than Intel.

> IMO, the only people who have legitimate complaints here are those
> people running Xeon Phi with Mellanox HCAs who are being forced to
> use OFED, rather than upstream code.

I suspect there are legitimate grounds to complain about the fact that
it is shipped in the OFA OFED and not limited to an Intel OFED
derivative similar to Mellanox OFED.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
    GPG Key ID: B826A3330E572FDD
    Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

RE: [ewg] [ANNOUNCE] OFED 4.8-rc2 release is available

[Woodruff, Robert J]

Doug Ledford wrote,
>The OFA has already learned their lesson once with XRC.  I wonder if they are getting ready to get hit with another lesson over this.  The issue is if the OFA ships an API and it isn't upstream, then that API needs to legitimately be a >private, should never conflict with upstream API.  If upstream then implements the same thing in a different way and users are exposed to the rather unpleasant choice of code to OFA's API or to upstream's API for the same thing, it >creates a schism in the user's code base around what API to use/support.  This is entirely contrary to the OFA's stated goals about the end user experience of people using their RDMA software.  So Intel can certainly make any cost >analysis they want about their hardware and the software to support it, but the OFA is not Intel's personal software distributor and the OFA must look at other, bigger picture issues than Intel.

The Xeon-Phi code is somewhat different than what happened in the past with XRC.  In the case of XRC, it was integrated into the base OFED package and built-in by default. For the Xeon-Phi code,
it is clearly marked as a technology preview and is not even compiled in at all unless specifically enabled. This is not too terribly different than the experimental branches that the kernel has.
Also, the Xeon-Phi code does not add any new APIs. It simply implements a driver set and library, thus there is no risk in someone coding to an API in OFED that gets totally changed once it gets upstream. The EWG charter allows code that is not upstream to be added as a technology preview. This allows people that want it, to try it out and use it. I personally do not see a reason to change that. You may not like what we are doing, and if you are an OFA promoter member, you could ask for a vote of the board to change the bylaws to prevent it. In the absence of that, you really do not have a say in the matter. 

Re: [ewg] [ANNOUNCE] OFED 4.8-rc2 release is available

[Doug Ledford]

[-- Attachment #1.1: Type: text/plain, Size: 3449 bytes --]

On 5/4/2017 3:14 PM, Woodruff, Robert J wrote:
> Doug Ledford wrote,
>> The OFA has already learned their lesson once with XRC.  I wonder
>> if they are getting ready to get hit with another lesson over this.
>> The issue is if the OFA ships an API and it isn't upstream, then
>> that API needs to legitimately be a >private, should never conflict
>> with upstream API.  If upstream then implements the same thing in a
>> different way and users are exposed to the rather unpleasant choice
>> of code to OFA's API or to upstream's API for the same thing, it
>> >creates a schism in the user's code base around what API to
>> use/support.  This is entirely contrary to the OFA's stated goals
>> about the end user experience of people using their RDMA software.
>> So Intel can certainly make any cost >analysis they want about
>> their hardware and the software to support it, but the OFA is not
>> Intel's personal software distributor and the OFA must look at
>> other, bigger picture issues than Intel.
> 
> The Xeon-Phi code is somewhat different than what happened in the
> past with XRC.  In the case of XRC, it was integrated into the base
> OFED package and built-in by default. For the Xeon-Phi code, it is
> clearly marked as a technology preview and is not even compiled in at
> all unless specifically enabled. This is not too terribly different
> than the experimental branches that the kernel has. Also, the
> Xeon-Phi code does not add any new APIs.

It implements a new kernel internal API, yes?

> It simply implements a
> driver set and library, thus there is no risk in someone coding to an
> API in OFED that gets totally changed once it gets upstream.

I'm not sure I understand your statement here Woody.  If there's no API,
then how do people even use the hardware?  Or are you saying that the
API is in the library, and that API can be preserved even if the
underlying driver implementation is changed to match whatever upstream
might implement instead of what you already have implemented?

> you really do not have a say in the matter.

Nope.  I don't.  But, as I pointed out in my other email, if
relationships matter, then whether or not someone has a say does not
negate the need to listen.

Personally, I haven't really investigated this code so I'm not going to
argue against the fact that the OFA ships it, other than what I have
already which is that it has been a stated goal of the OFA to foster a
unified code base, so collaborating with upstream is generally
necessary.  If you are saying that the Xeon Phi support is implemented
in a library (like nVidia's CUDA support) that insulates the end user
from a possible fracture if upstream implements things differently, then
that mostly settles my concerns.  I still think it would be best if the
Xeon Phi people collaborated with upstream on the kernel internal Peer
to Peer PCI API as that's evidently a requirement of the Xeon Phi
library?  It is conceivably possible failure to collaborate could in
fact break the Xeon Phi library if they simply don't implement something
the library has a hard requirement on.  But that's outside of my
particular wheel house so I'm just suggesting that it might be a wise
thing to do.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
    GPG Key ID: B826A3330E572FDD
    Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

RE: [ewg] [ANNOUNCE] OFED 4.8-rc2 release is available

[Woodruff, Robert J]

Doug wrote,
>I'm not sure I understand your statement here Woody.  If there's no API, then how do people even use the hardware?  Or are you saying that the API is in the library, and that API can be preserved even if the underlying driver >implementation is changed to match whatever upstream might implement instead of what you already have implemented?

The code that goes into OFED does not implement any new APIs or changes to existing verbs APIs. It implements drivers and user-space libraries that plug into the user and kernel space core
and emulate the standard RDMA verbs. Users use it the same way they use any RDMA device in OFED, through libibverbs. Although under the covers, there is much more to it than that
but from the user's point of view it looks just like an HCA that is used through the standard verbs. 

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[Jim Foraker]

On 05/04/2017 10:34 AM, Hefty, Sean wrote:
>> This is exactly why we so strongly discourage this out of tree
>> stuff - getting something unmergable in OFED is *NOT* Job Done,
>> Time to Go Home. Down this path just creates another Lustre mess.
>
> Actually, this may be 'job done'.  No individual or company is
> obligated to provide upstream software for any of their hardware.
> OFA decides what to ship in their software products, not the greater
> linux kernel community.  Individual companies can decide if out of
> tree maintenance is more cost effective than trying to merge code
> upstream.  Because that's what this ultimately comes down to.
>
> IMO, the only people who have legitimate complaints here are those
> people running Xeon Phi with Mellanox HCAs who are being forced to
> use OFED, rather than upstream code.

      Apparently we've experimented with doing just this, so I'll bite. 
And I will say the same thing we've told every vendor repeatedly: "get 
your code upstream!"
      A vendor may weigh the cost of maintaining out-of-tree code versus 
upstreaming it, but as a customer, I'm more concerned about the 
annoyance and time wasted having to run N different software stacks on N 
different hardware types versus being able to use the RDMA stack from my 
preferred linux distribution everywhere.  And the best way to get your 
code into the distributions is to get it upstream.
      I have no particular opinion on whether or not the xeon-phi driver 
belongs in OFED.  But cloaking this conversation in talk of obligations 
and framers' intent misses a bigger point, which is that any code not 
fully upstreamed, and hence likely to be found in your user's distro, 
inconveniences them, increases their support costs, and makes it less 
likely that they will adopt your product.

      Jim
_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/mailman/listinfo/ewg

Re: [ANNOUNCE] OFED 4.8-rc2 release is available

[richard Croucher]

[-- Attachment #1.1: Type: text/plain, Size: 3433 bytes --]

I could not echo this point more.  I principally work with Enterprise
clients.  None use custom kernels and all are dependent on the support
provided from their Linux distro vendors.  
We have repeatedly ended up in finger pointing exercises where RDMA
 vendors insist on us using their drivers and the Linux distro vendor
want us to use there's.   The customer always loses in these
situations.
Only as an absolute last resort we will use proprietary drivers,
 modules  and system libraries because we know it will compromise
support and may hit us later.
Certain vendors try to do this under the radar  now replacing all sorts
of distro system libraries when you innocently install their Ethernet
driver.  Doing it surreptitiously rather than blatantly is not the
answer.   Get your code into mainline and work with the major distro
vendors on the backports and updates.
I continually face resistance from Enterprise accounts in adopting RDMA
in a more widespread manner.   It's the perception of the risk raised
by the Linux SA's in supporting these systems.    I needs to just
'work' running only the standard distro installers to add it if it is
not there by default. 
On Thu, 2017-05-04 at 20:42 -0700, Jim Foraker wrote:
> On 05/04/2017 10:34 AM, Hefty, Sean wrote:
> > 
> > > 
> > > This is exactly why we so strongly discourage this out of tree
> > > stuff - getting something unmergable in OFED is *NOT* Job Done,
> > > Time to Go Home. Down this path just creates another Lustre mess.
> > Actually, this may be 'job done'.  No individual or company is
> > obligated to provide upstream software for any of their hardware.
> > OFA decides what to ship in their software products, not the
> > greater
> > linux kernel community.  Individual companies can decide if out of
> > tree maintenance is more cost effective than trying to merge code
> > upstream.  Because that's what this ultimately comes down to.
> > 
> > IMO, the only people who have legitimate complaints here are those
> > people running Xeon Phi with Mellanox HCAs who are being forced to
> > use OFED, rather than upstream code.
>       Apparently we've experimented with doing just this, so I'll
> bite. 
> And I will say the same thing we've told every vendor repeatedly:
> "get 
> your code upstream!"
>       A vendor may weigh the cost of maintaining out-of-tree code
> versus 
> upstreaming it, but as a customer, I'm more concerned about the 
> annoyance and time wasted having to run N different software stacks
> on N 
> different hardware types versus being able to use the RDMA stack from
> my 
> preferred linux distribution everywhere.  And the best way to get
> your 
> code into the distributions is to get it upstream.
>       I have no particular opinion on whether or not the xeon-phi
> driver 
> belongs in OFED.  But cloaking this conversation in talk of
> obligations 
> and framers' intent misses a bigger point, which is that any code
> not 
> fully upstreamed, and hence likely to be found in your user's
> distro, 
> inconveniences them, increases their support costs, and makes it
> less 
> likely that they will adopt your product.
> 
>       Jim
> _______________________________________________
> ewg mailing list
> ewg-ZwoEplunGu1OwGhvXhtEPSCwEArCW2h5@public.gmane.org
> http://lists.openfabrics.org/mailman/listinfo/ewg
-- 

Richard Croucher
www.informatix-sol.com


[-- Attachment #1.2: Type: text/html, Size: 4106 bytes --]

[-- Attachment #2: Type: text/plain, Size: 140 bytes --]

_______________________________________________
ewg mailing list
ewg@lists.openfabrics.org
http://lists.openfabrics.org/mailman/listinfo/ewg