Re: [MAINTAINER SUMMIT] User-space requirements for accelerator drivers

[Dave Airlie <airlied@gmail.com>]

Outside of all this, I disagree on distros being the target at all.
However distros are probably a good thing to have involved.

It's much easier for a distro to package one project than one
per-vendor project, esp if that one project has a release cycle.

If every company forks an LLVM backend for example distros will never
be able to ship things, getting the LLVM backends upstream into LLVM
means the distro get it for "free" on the next LLVM release. Creating
kernel-like communities for userspace should be the goal, why do we
want to forget the benefits the kernel ecosystem has given us as soon
as we exit a syscall handler?

> >
> > 1. Niche devices - continue to do as they do it now, by supplying
> > out-of-tree solutions for their customers. Such devices and companies
> > rarely need upstream linux kernel support, because the burden to
> > upstream it is very high. We don't want them in the tree either, because
> > once they upstream it, the maintenance burden will be on us.
>
> {sigh}
>
> No, that is NOT our rule at all.
>
> These devices and companies need to be upstream more than anything else
> as that way they become part of our community and are responsible for
> maintaining their code in the tree.  To force them to remain outside is
> to go against everything that many of us have been saying for _decades_
> now.

Name one group that has actively become part of the community via this
advice, (I'll wait).

From my view most of the communities have been created with more
push-back by kernel maintainers, gpus, rdma, media, alsa vs misc (X
accel drivers with no home or common cause).



> > And AI/ML is no different here, someone just need to start build such
> > stack. Otherwise, we will continue to see more free riders like HabanaLabs
> > which don't have any real benefit to the community.
>
> Everyone contributes to Linux in a selfish manner, that's just how the
> community works.  The work that companies like habanalabs is NOT being a
> "free rider" at all, they have worked with us and done the hard work of
> actually getting their code merged into the tree and their userspace
> code released under an open source license (unlike _ALL_ other AI/ML
> companies, including Intel).  It would have been much cheaper and
> quicker of them to just ignore upstream entirely, but that would have
> meant that the community would not have any idea of what exactly these
> use-case models were nor what the problems were that they were trying to
> get Linux to do.

These companies don't get to ignore upstream entirely. They aren't
here because they want to be, at least initially, they are here
because RHEL, Amazon, Facebook, Google whoever told them they would
buy their hw if it had upstream drivers in a contract and they have to
do the minimal amount of work to get past Greg to merge stuff and
satisfy that agreement. The community is very well aware of the needs
of these groups, it's not like we don't have lots of GPUs being using
for AI/ML. The habanalabs hardware is just a VLIW multithreaded
processor almost like taking an AMD evergreen and shaving off the
texture engines and other GPU specific bits. There is nothing new or
exciting here that hasn't been solved.

>
> Linux benefits overall by having everyone participate, do NOT make
> arbitrary rules to somehow prevent one company/group from being allowed
> to upstream their code vs. another.  That is NOT how we have worked in
> the past, and would only cause us to slowly die and become irrelevant.

The Linux Foundation might benefit, Linux doesn't. Linux benefits and
stays maintainable by having responsible maintainers guide the
direction of the kernel design, and creating upstream communities to
sustain that.

Dave.