From: Oded Gabbay <firstname.lastname@example.org>
To: "Linux-Kernel@Vger. Kernel. Org" <email@example.com>,
Yuji Ishikawa <firstname.lastname@example.org>,
Jiho Chu <email@example.com>,
Alexandre Bailon <firstname.lastname@example.org>,
Kevin Hilman <email@example.com>
Cc: Dave Airlie <firstname.lastname@example.org>,
Greg Kroah-Hartman <email@example.com>,
Jason Gunthorpe <firstname.lastname@example.org>, Arnd Bergmann <email@example.com>,
Daniel Vetter <firstname.lastname@example.org>
Subject: Re: New subsystem for acceleration devices
Date: Fri, 23 Sep 2022 19:21:55 +0300 [thread overview]
Message-ID: <CAFCwf12mjshsf+GC-Y9irvPFT=W4Uis10OnZ4PNN1txjXyzLSA@mail.gmail.com> (raw)
On Mon, Aug 29, 2022 at 11:54 PM Kevin Hilman <email@example.com> wrote:
> Hi Oded (and sorry I misspelled your name last time),
> Oded Gabbay <firstname.lastname@example.org> writes:
> > On Tue, Aug 23, 2022 at 9:24 PM Kevin Hilman <email@example.com> wrote:
> >> Hi Obed,
> >> Oded Gabbay <firstname.lastname@example.org> writes:
> >> [...]
> >> > I want to update that I'm currently in discussions with Dave to figure
> >> > out what's the best way to move forward. We are writing it down to do
> >> > a proper comparison between the two paths (new accel subsystem or
> >> > using drm). I guess it will take a week or so.
> >> Any update on the discussions with Dave? and/or are there any plans to
> >> discuss this further at LPC/ksummit yet?
> > Hi Kevin.
> > We are still discussing the details, as at least the habanalabs driver
> > is very complex and there are multiple parts that I need to see if and
> > how they can be mapped to drm.
> > Some of us will attend LPC so we will probably take advantage of that
> > to talk more about this.
> OK, looking forward to some more conversations at LPC.
> >> We (BayLibre) are upstreaming support for APUs on Mediatek SoCs, and are
> >> using the DRM-based approach. I'll also be at LPC and happy to discuss
> >> in person.
> >> For some context on my/our interest: back in Sept 2020 we initially
> >> submitted an rpmesg based driver for kernel communication. After
> >> review comments, we rewrote that based on DRM and are now using it
> >> for some MTK SoCs and supporting our MTK customers with it.
> >> Hopefully we will get the kernel interfaces sorted out soon, but next,
> >> there's the userspace side of things. To that end, we're also working
> >> on libAPU, a common, open userspace stack. Alex Bailon recently
> >> presented a proposal earlier this year at Embedded Recipes in Paris
> >> (video, slides.)
> >> libAPU would include abstractions of the kernel interfaces for DRM
> >> (using libdrm), remoteproc/rpmsg, virtio etc. but also goes farther and
> >> proposes an open firmware for the accelerator side using
> >> libMetal/OpenAMP + rpmsg for communication with (most likely closed
> >> source) vendor firmware. Think of this like sound open firmware (SOF),
> >> but for accelerators.
> > I think your device and the habana device are very different in
> > nature, and it is part of what Dave and I discussed, whether these two
> > classes of devices can live together. I guess they can live together
> > in the kernel, but in the userspace, not so much imo.
> Yeah, for now I think focusing on how to handle both classes of devices
> in the kernel is the most important.
> > The first class is the edge inference devices (usually as part of some
> > SoC). I think your description of the APU on MTK SoC is a classic
> > example of such a device.
> > You usually have some firmware you load, you give it a graph and
> > pointers for input and output and then you just execute the graph
> > again and again to perform inference and just replace the inputs.
> > The second class is the data-center, training accelerators, which
> > habana's gaudi device is classified as such. These devices usually
> > have a number of different compute engines, a fabric for scaling out,
> > on-device memory, internal MMUs and RAS monitoring requirements. Those
> > devices are usually operated via command queues, either through their
> > kernel driver or directly from user-space. They have multiple APIs for
> > memory management, RAS, scaling-out and command-submissions.
> OK, I see.
> >> We've been using this succesfully for Mediatek SoCs (which have a
> >> Cadence VP6 APU) and have submitted/published the code, including the
> >> OpenAMP and libmetal parts in addition to the kernel parts already
> >> mentioned.
> > What's the difference between libmetal and other open-source low-level
> > runtime drivers, such as oneAPI level-zero ?
> TBH, I'd never heard of oneAPI before, so I'm assuming it's mainly
> focused in the data center. libmetal/openAMP are widely used
> in the consumer, industrial embedded space, and heavily used by FPGAs in
> many market segments.
> > Currently we have our own runtime driver which is tightly coupled with
> > our h/w. For example, the method the userspace "talks" to the
> > data-plane firmware is very proprietary as it is hard-wired into the
> > architecture of the entire ASIC and how it performs deep-learning
> > training. Therefore, I don't see how this can be shared with other
> > vendors. Not because of secrecy but because it is simply not relevant
> > to any other ASIC.
> OK, makes sense.
> Thanks for clarifying your use case in more detail.
I wanted to update on this issue for those of you who weren't in LPC.
We had a BOF session about this topic with most, if not all, of the
relevant people - DRM maintainers, Greg and other subsystem and device
Dave Airlie summarized the session in his blog -
Accelerators drivers will use the DRM subsystem code, but they will be
located in a different directory (drivers/accel) and will be exposed
to the userspace using a new major and a new device char name
I'm currently working on preparing some prerequisite patches for the
DRM subsystem to support the new subsystem (e.g. new major number).
After that will be merged, I plan to move the habanalabs driver to the
new location and convert it to use the modified DRM framework code.
next prev parent reply other threads:[~2022-09-23 16:22 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <CGME20220731114605epcas1p1afff6b948f542e2062b60d49a8023f6f@epcas1p1.samsung.com>
2022-07-31 11:45 ` New subsystem for acceleration devices Oded Gabbay
2022-07-31 15:37 ` Greg Kroah-Hartman
2022-08-01 2:29 ` yuji2.ishikawa
2022-08-01 8:21 ` Oded Gabbay
2022-08-03 4:39 ` yuji2.ishikawa
2022-08-03 5:34 ` Greg KH
2022-08-03 20:28 ` Oded Gabbay
2022-08-02 17:25 ` Jiho Chu
2022-08-02 19:07 ` Oded Gabbay
2022-08-03 19:04 ` Dave Airlie
2022-08-03 20:20 ` Oded Gabbay
2022-08-03 23:31 ` Daniel Stone
2022-08-04 6:46 ` Oded Gabbay
2022-08-04 9:27 ` Jiho Chu
2022-08-03 23:54 ` Dave Airlie
2022-08-04 7:43 ` Oded Gabbay
2022-08-04 14:50 ` Jason Gunthorpe
2022-08-04 17:48 ` Oded Gabbay
2022-08-05 0:22 ` Jason Gunthorpe
2022-08-07 6:43 ` Oded Gabbay
2022-08-07 11:25 ` Oded Gabbay
2022-08-08 6:10 ` Greg Kroah-Hartman
2022-08-08 17:55 ` Jason Gunthorpe
2022-08-09 6:23 ` Greg Kroah-Hartman
2022-08-09 8:04 ` Christoph Hellwig
2022-08-09 8:32 ` Arnd Bergmann
2022-08-09 12:18 ` Jason Gunthorpe
2022-08-09 12:46 ` Arnd Bergmann
2022-08-09 14:22 ` Jason Gunthorpe
2022-08-09 8:45 ` Greg Kroah-Hartman
2022-08-08 17:46 ` Jason Gunthorpe
2022-08-08 20:26 ` Oded Gabbay
2022-08-09 12:43 ` Jason Gunthorpe
2022-08-05 3:02 ` Dave Airlie
2022-08-07 6:50 ` Oded Gabbay
2022-08-09 21:42 ` Oded Gabbay
2022-08-10 9:00 ` Jiho Chu
2022-08-10 14:05 ` yuji2.ishikawa
2022-08-10 14:37 ` Oded Gabbay
2022-08-23 18:23 ` Kevin Hilman
2022-08-23 20:45 ` Oded Gabbay
2022-08-29 20:54 ` Kevin Hilman
2022-09-23 16:21 ` Oded Gabbay [this message]
2022-09-26 8:16 ` Christoph Hellwig
2022-09-29 6:50 ` Oded Gabbay
2022-08-04 12:00 ` Tvrtko Ursulin
2022-08-04 15:03 ` Jeffrey Hugo
2022-08-04 17:53 ` Oded Gabbay
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).