linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Oded Gabbay <oded.gabbay@gmail.com>
To: Dave Airlie <airlied@gmail.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Yuji Ishikawa <yuji2.ishikawa@toshiba.co.jp>,
	Jiho Chu <jiho.chu@samsung.com>
Cc: dri-devel <dri-devel@lists.freedesktop.org>,
	Arnd Bergmann <arnd@arndb.de>,
	"Linux-Kernel@Vger. Kernel. Org" <linux-kernel@vger.kernel.org>,
	Jason Gunthorpe <jgg@nvidia.com>
Subject: Re: New subsystem for acceleration devices
Date: Wed, 10 Aug 2022 00:42:24 +0300	[thread overview]
Message-ID: <CAFCwf13WU3ZEjurEaEnVC56zorwKr-uuQn-ec10r301Fh+XEtA@mail.gmail.com> (raw)
In-Reply-To: <CAFCwf11TPKTF_Ndi60FneWp5g3SoawJvfJoKVWJ-QjxjpawMmg@mail.gmail.com>

On Sun, Aug 7, 2022 at 9:50 AM Oded Gabbay <oded.gabbay@gmail.com> wrote:
>
> On Fri, Aug 5, 2022 at 6:03 AM Dave Airlie <airlied@gmail.com> wrote:
> >
> > On Thu, 4 Aug 2022 at 17:44, Oded Gabbay <oded.gabbay@gmail.com> wrote:
> > >
> > > On Thu, Aug 4, 2022 at 2:54 AM Dave Airlie <airlied@gmail.com> wrote:
> > > >
> > > > On Thu, 4 Aug 2022 at 06:21, Oded Gabbay <oded.gabbay@gmail.com> wrote:
> > > > >
> > > > > On Wed, Aug 3, 2022 at 10:04 PM Dave Airlie <airlied@gmail.com> wrote:
> > > > > >
> > > > > > On Sun, 31 Jul 2022 at 22:04, Oded Gabbay <oded.gabbay@gmail.com> wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > > Greg and I talked a couple of months ago about preparing a new accel
> > > > > > > subsystem for compute/acceleration devices that are not GPUs and I
> > > > > > > think your drivers that you are now trying to upstream fit it as well.
> > > > > >
> > > > > > We've had some submissions for not-GPUs to the drm subsystem recently.
> > > > > >
> > > > > > Intel GNA, Intel VPU, NVDLA, rpmsg AI processor unit.
> > > > > >
> > > > > > why is creating a new subsystem at this time necessary?
> > > > > >
> > > > > > Are we just creating a subsystem to avoid the open source userspace
> > > > > > consumer rules? Or do we have some concrete reasoning behind it?
> > > > > >
> > > > > > Dave.
> > > > >
> > > > > Hi Dave.
> > > > > The reason it happened now is because I saw two drivers, which are
> > > > > doing h/w acceleration for AI, trying to be accepted to the misc
> > > > > subsystem.
> > > > > Add to that the fact I talked with Greg a couple of months ago about
> > > > > doing a subsystem for any compute accelerators, which he was positive
> > > > > about, I thought it is a good opportunity to finally do it.
> > > > >
> > > > > I also honestly think that I can contribute much to these drivers from
> > > > > my experience with the habana driver (which is now deployed in mass at
> > > > > AWS) and contribute code from the habana driver to a common framework
> > > > > for AI drivers.
> > > >
> > > > Why not port the habana driver to drm now instead? I don't get why it
> > > > wouldn't make sense?
> > >
> > > imho, no, I don't see the upside. This is not a trivial change, and
> > > will require a large effort. What will it give me that I need and I
> > > don't have now ?
> >
> > The opportunity for review, code sharing, experience of locking
> > hierarchy, mm integration?
> >
> > IMHO The biggest thing that drm has is the community of people who
> > understand accelerators, memory management, userspace command
> > submissions, fencing, dma-buf etc.
> >
> > It's hard to have input to those discussions from the outside, and
> > they are always ongoing.
> >
> > I think one of the Intel teams reported dropping a lot of code on
> > their drm port due to stuff already being there, I'd expect the same
> > for you.
> >
> > The opposite question is also valid, what does moving to a new
> > subsystem help you or others, when there is one with all the
> > infrastructure and more importantly reviewers.
> >
> > I'd be happy to have accelerator submaintainers, I'd be happy to even
> > create an ACCELERATOR property for non-gpu drivers, so they can opt
> > out of some of the GPUier stuff, like multiple device node users etc,
> > or even create a new class of device nodes under /dev/dri.
> >
> I'm taking all what you wrote seriously, these are all good points.
> As I wrote to Jason, I don't want to jump the gun here. I think we
> should discuss this and explore the possibilities that you suggested
> because I would like to reach consensus if possible.
> Maybe this is something we can discuss in LPC or in the kernel summit ?
>
> Oded

Hi Jiho, Yuji.

I want to update that I'm currently in discussions with Dave to figure
out what's the best way to move forward. We are writing it down to do
a proper comparison between the two paths (new accel subsystem or
using drm). I guess it will take a week or so.

In the meantime, I'm putting the accel code on hold. I have only
managed to do the very basic infra and add a demo driver that shows
how to register and unregister from it.
You can check the code at:
https://git.kernel.org/pub/scm/linux/kernel/git/ogabbay/linux.git/log/?h=accel

It has two commits. The first adds the subsystem code and the second
adds the demo driver.
The subsystem code is basically drm code copied and renamed and
slightly modified, but I really only worked on it for a couple of
hours so take that into consideration.

The important thing is that the demo driver shows the basic steps are
really simple. You need to add two function calls in your probe and
one function call in your release. Of course you will need to supply
some function callbacks, but I haven't got to fill that in the demo
driver. Once you register, you get /dev/accel/ac0 and
/dev/accel/ac_controlD64 (if you want a control device). If I were to
continue this, the next step is to do the open and close part.

I will update once we know where things are heading. As I said, I
imagine it can take a few weeks.

Thanks,
Oded


Oded

>
> >
> > > I totally agree. We need to set some rules and make sure everyone in
> > > the kernel community is familiar with them, because now you get
> > > different answers based on who you consult with.
> > >
> > > My rules of thumb that I thought of was that if you don't have any
> > > display (you don't need to support X/wayland) and you don't need to
> > > support opengl/vulkan/opencl/directx or any other gpu-related software
> > > stack, then you don't have to go through drm.
> > > In other words, if you don't have gpu-specific h/w and/or you don't
> > > need gpu uAPI, you don't belong in drm.
> >
> > What happens when NVIDIA submit a driver for just compute or intel?
> > for what is actually a GPU?
> > This has been suggested as workaround for our userspace rules a few times.
> >
> > If my GPU can do compute tasks, do I have to add an accelerator
> > subsystem driver alongside my GPU one?
> >
> > > After all, memory management services, or common device chars handling
> > > I can get from other subsystems (e.g. rdma) as well. I'm sure I could
> > > model my uAPI to be rdma uAPI compliant (I can define proprietary uAPI
> > > there as well), but this doesn't mean I belong there, right ?
> >
> > Good point, but I think accelerators do mostly belong in drm or media,
> > because there is enough framework around them to allow them to work,
> > without reinventing everything.
> >
> > > >
> > > > I think the one area I can see a divide where a new subsystem is for
> > > > accelerators that are single-user, one shot type things like media
> > > > drivers (though maybe they could be just media drivers).
> > > >
> > > > I think anything that does command offloading to firmware or queues
> > > > belongs in drm, because that is pretty much what the framework does. I
> > > I think this is a very broad statement which doesn't reflect reality
> > > in the kernel.
> >
> > I think the habanalabs driver is one of the only ones that is outside
> > this really, and in major use. There might be one or two other minor
> > drivers with no real users.
> >
> > Dave.

  reply	other threads:[~2022-08-09 21:43 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <CGME20220731114605epcas1p1afff6b948f542e2062b60d49a8023f6f@epcas1p1.samsung.com>
2022-07-31 11:45 ` New subsystem for acceleration devices Oded Gabbay
2022-07-31 15:37   ` Greg Kroah-Hartman
2022-08-01  2:29     ` yuji2.ishikawa
2022-08-01  8:21       ` Oded Gabbay
2022-08-03  4:39         ` yuji2.ishikawa
2022-08-03  5:34           ` Greg KH
2022-08-03 20:28           ` Oded Gabbay
2022-08-02 17:25   ` Jiho Chu
2022-08-02 19:07     ` Oded Gabbay
2022-08-03 19:04   ` Dave Airlie
2022-08-03 20:20     ` Oded Gabbay
2022-08-03 23:31       ` Daniel Stone
2022-08-04  6:46         ` Oded Gabbay
2022-08-04  9:27           ` Jiho Chu
2022-08-03 23:54       ` Dave Airlie
2022-08-04  7:43         ` Oded Gabbay
2022-08-04 14:50           ` Jason Gunthorpe
2022-08-04 17:48             ` Oded Gabbay
2022-08-05  0:22               ` Jason Gunthorpe
2022-08-07  6:43                 ` Oded Gabbay
2022-08-07 11:25                   ` Oded Gabbay
2022-08-08  6:10                     ` Greg Kroah-Hartman
2022-08-08 17:55                       ` Jason Gunthorpe
2022-08-09  6:23                         ` Greg Kroah-Hartman
2022-08-09  8:04                           ` Christoph Hellwig
2022-08-09  8:32                             ` Arnd Bergmann
2022-08-09 12:18                               ` Jason Gunthorpe
2022-08-09 12:46                                 ` Arnd Bergmann
2022-08-09 14:22                                   ` Jason Gunthorpe
2022-08-09  8:45                             ` Greg Kroah-Hartman
2022-08-08 17:46                   ` Jason Gunthorpe
2022-08-08 20:26                     ` Oded Gabbay
2022-08-09 12:43                       ` Jason Gunthorpe
2022-08-05  3:02           ` Dave Airlie
2022-08-07  6:50             ` Oded Gabbay
2022-08-09 21:42               ` Oded Gabbay [this message]
2022-08-10  9:00                 ` Jiho Chu
2022-08-10 14:05                 ` yuji2.ishikawa
2022-08-10 14:37                   ` Oded Gabbay
2022-08-23 18:23                 ` Kevin Hilman
2022-08-23 20:45                   ` Oded Gabbay
2022-08-29 20:54                     ` Kevin Hilman
2022-09-23 16:21                       ` Oded Gabbay
2022-09-26  8:16                         ` Christoph Hellwig
2022-09-29  6:50                           ` Oded Gabbay
2022-08-04 12:00         ` Tvrtko Ursulin
2022-08-04 15:03           ` Jeffrey Hugo
2022-08-04 17:53             ` Oded Gabbay

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CAFCwf13WU3ZEjurEaEnVC56zorwKr-uuQn-ec10r301Fh+XEtA@mail.gmail.com \
    --to=oded.gabbay@gmail.com \
    --cc=airlied@gmail.com \
    --cc=arnd@arndb.de \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=gregkh@linuxfoundation.org \
    --cc=jgg@nvidia.com \
    --cc=jiho.chu@samsung.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=yuji2.ishikawa@toshiba.co.jp \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).