Fostering linux community collaboration on hardware accelerators

Fostering linux community collaboration on hardware accelerators

[Jonathan Cameron]

Hi All,

Please forward this email to anyone you think may be interested.

On behalf of Huawei, I am looking into options to foster a wider community
around the various ongoing projects related to Accelerator support within
Linux.  The particular area of interest to Huawei is that of harnessing
accelerators from userspace, but in a collaborative way with the kernel
still able to make efficient use of them, where appropriate.

We are keen to foster a wider community than one just focused on
our own current technology.  This is a field with no clear answers, so the
widest possible range of input is needed!

The address list of this email is drawn from people we have had discussions
with or who have been suggested in response to Kenneth Lee's wrapdrive
presentation at Linaro Connect and earlier presentations on the more general
issue. A few relevant lists added to hopefully catch anyone we missed.
My apologies to anyone who got swept up in this and isn't interested!

Here we are defining accelerators fairly broadly - suggestions for a better
term are also welcome.

The infrastructure may be appropriate for:
* Traditional offload engines - cryptography, compression and similar
* Upcoming AI accelerators
* ODP type requirements for access to elements of networking
* Systems utilizing SVM including CCIX and other cache coherent buses
* Many things we haven't thought of yet...

As I see it, there are several aspects to this:

1) Kernel drivers for accelerators themselves.
   * Traditional drivers such as crypto etc
	- These already have their own communities. The main
          focus of such work will always be through them.
        - What a more general community could add here would be an
          overview of the shared infrastructure of such devices.
	  This is particularly true around VFIO based (or similar)
	  userspace interfaces with a non trivial userspace component.
   * How to support new types of accelerator?

2) The need for lightweight access paths from userspace that 'play well' and
   share resources etc with standard in-kernel drivers.  This is the area
   that Kenneth Lee and Huawei have been focusing on with their wrapdrive
   effort. We know there are other similar efforts going on in other companies.
   * This may involve interacting with existing kernel communities such as
     those around VFIO and mdev.
   * Resource management when we may have many consumers - not all hardware
     has appropriate features to deal with this.

3) Usecases for accelerators. e.g.
   * kTLS
   * Storage encryption
   * ODP - networking dataplane
   * AI toolkits

Discussions we want to get started include:
* A wider range of hardware than we are currently considering. What makes
  sense to target / what hardware do people have they would like to support?
* Upstream paths - potential blockers and how to overcome them. The standard
  kernel drivers should be fairly straightforward, but once we start looking at
  systems with a heavier userspace component, things will get more
  controversial!
* Fostering stronger userspace communities to allow these these accelerators
  to be easily harnessed.

So as ever with a linux community focusing on a particular topic, the
obvious solution is a mailing list. There are a number of options on how
do this.

1) Ask one of the industry bodies to host? Who?

2) Put together a compelling argument for linux-accelerators@vger.kernel.org
as probably the most generic location for such a list.

More open questions are
1) Scope?
 * Would anyone ever use such an overarching list?
 * Are we better off with the usual adhoc list of 'interested parties' + lkml?
 * Do we actually need to define the full scope - are we better with a vague
   definition?

2) Is there an existing community we can use to discuss these issues?
   (beyond the obvious firehose of LKML).

3) Who else to approach for input on these general questions?

In parallel to this there are elements such as git / patchwork etc but
they can all be done as they are needed.

Thanks

--
Jonathan Cameron
Huawei

Re: Fostering linux community collaboration on hardware accelerators

[Jonathan Cameron]

On Tue, 10 Oct 2017 16:55:06 +0200
Francois Ozog <francois.ozog@linaro.org> wrote:

> Hi,
Hi Francois,

Firstly, to anyone seeing this on the mailing lists, there was an attachment so
it presumably bounced.

> 
> I am really happy you started this thread. I think we need to have Grant
> Likely from Arm involved in the topic. I only have his HPE mail and not his
> new Arm. I also added Andrea Gallo who has visibility of all segment groups
> in Linaro.

Good idea. Given the simple nature of your average ARM email address I've one with
guessing Grant's *crosses fingers*.

> 
> 
> As you say, there are more questions than answers at this stage and I'd
> like to share thoughts and related activities I already conducted so that
> you can take those into account (if they are relevant) to build your action
> plan.
> 
> 
> On the scope of accelerators I include media related (transcoding...) and
> FPGAs.

Two more for the list - I knew I've forget some big groups ;)

> I kicked off discussions with various parties on how to build an
> FPGA focused community (see attached document on FPGAs). The FPGAs will
> need proper "factory" of userland drivers and I expect WD to be
> instrumental in this.
> 
> On the "ODP type" requirements: I think we have to include both ODP and
> DPDK.
> 
> Robustness/security of the userland framework: if anything happens to
> userland, the kernel has to recover.

This issue is a key one I'd missed when drawing up my list. Glad you
raised it.

> Exposing pieces of PCI config space or
> portions of MMIOs may be a new area of intellectual property for Arm.
> 
> I was leading acceleration activities in ETSI NFV (IFA stream 2 group that
> was created by Yun-Chao Hu@ Huawei, Bruno Chatras@Orange and myself) and
> talked to Orange, BT, Telecom Italia and AT&T yesterday. The guys I know
> are keen to participate to an advisory board related to acceleration in
> Opensource Communities. They see the value of continuing the work done in
> SDOs (ETSI, IETF, IEEE...) in OCs (Linaro, Linux Foundation...). 

Interesting - of course such a structure might be useful. Of course their
input along with that of others would be of great assistance.

However, any formal arrangement or consortium does bring costs that may put
some individuals or companies off involvement.   To my mind, any Linux
focused effort needs to incorporate those interests, but take in wider views.
That's not to say that any Linux community focused effort would not be a
good place for ideas coming out of industry consortiums / advisory groups
to be presented and evaluated as an integral part of the community.

>I think
> their participation to the acceleration efforts is key to ensure we solve
> the right problem the right way. I am trying to organize an accelerator
> workshop "accelerators: from SDO to Opensource) at the ETSI NFV F2F meeting
> (NFV#20 December 4-8th in Sophia Antipolis, France). An agenda item is also
> how to deal with Intel EPA that introduce bias in platform and accelerator
> "scheduling" in Openstack terminology.
> 
> As of particular interest is accelerator management in the context of Cloud
> and NFV. Huawei is leading efforts in Cyborg project (OpenStack) and I
> think, while it should be kept  as a separate effort, we should be aligning
> our efforts in a way or the other.

Management, in a more general sense isn't something I had really considered.
Efforts such as Cyborg seem to sit at a higher level in the stack but clearly
you are correct in suggesting that aligning efforts makes sense.

> 
> CCIX will have a massive impact on networking. With DMA it is all about
> packet MOVEMENT. With CCIX this is about cacheline games. Dealing with
> headroom and tail room of packets will no longer be an issue. Achieving
> 50Gbps and above line rate bumped into DMA transactions per second problem.
> Wit CCIX this is no longer an issue (at least on paper).

This is true of both CCIX and some competing / alternative technologies.
I am very keen to embrace that competition as well so that any results
of collaboration help everyone!  Not to mention there is plenty of
existing hardware with adhoc solutions in place already.

The one big element I clearly forgot to mention in the below scope is
virtualization (oops).  Any solutions clearly need to take that into
account as well.

Jonathan

> 
> 
> 
> Cordially,
> 
> François-Frédéric
> 
> On 10 October 2017 at 13:28, Jonathan Cameron <Jonathan.Cameron@huawei.com>
> wrote:
> 
> > Hi All,
> >
> > Please forward this email to anyone you think may be interested.
> >
> > On behalf of Huawei, I am looking into options to foster a wider community
> > around the various ongoing projects related to Accelerator support within
> > Linux.  The particular area of interest to Huawei is that of harnessing
> > accelerators from userspace, but in a collaborative way with the kernel
> > still able to make efficient use of them, where appropriate.
> >
> > We are keen to foster a wider community than one just focused on
> > our own current technology.  This is a field with no clear answers, so the
> > widest possible range of input is needed!
> >
> > The address list of this email is drawn from people we have had discussions
> > with or who have been suggested in response to Kenneth Lee's wrapdrive
> > presentation at Linaro Connect and earlier presentations on the more
> > general
> > issue. A few relevant lists added to hopefully catch anyone we missed.
> > My apologies to anyone who got swept up in this and isn't interested!
> >
> > Here we are defining accelerators fairly broadly - suggestions for a better
> > term are also welcome.
> >
> > The infrastructure may be appropriate for:
> > * Traditional offload engines - cryptography, compression and similar
> > * Upcoming AI accelerators
> > * ODP type requirements for access to elements of networking
> > * Systems utilizing SVM including CCIX and other cache coherent buses
> > * Many things we haven't thought of yet...
> >
> > As I see it, there are several aspects to this:
> >
> > 1) Kernel drivers for accelerators themselves.
> >    * Traditional drivers such as crypto etc
> >         - These already have their own communities. The main
> >           focus of such work will always be through them.
> >         - What a more general community could add here would be an
> >           overview of the shared infrastructure of such devices.
> >           This is particularly true around VFIO based (or similar)
> >           userspace interfaces with a non trivial userspace component.
> >    * How to support new types of accelerator?
> >
> > 2) The need for lightweight access paths from userspace that 'play well'
> > and
> >    share resources etc with standard in-kernel drivers.  This is the area
> >    that Kenneth Lee and Huawei have been focusing on with their wrapdrive
> >    effort. We know there are other similar efforts going on in other
> > companies.
> >    * This may involve interacting with existing kernel communities such as
> >      those around VFIO and mdev.
> >    * Resource management when we may have many consumers - not all hardware
> >      has appropriate features to deal with this.
> >
> > 3) Usecases for accelerators. e.g.
> >    * kTLS
> >    * Storage encryption
> >    * ODP - networking dataplane
> >    * AI toolkits
> >
> > Discussions we want to get started include:
> > * A wider range of hardware than we are currently considering. What makes
> >   sense to target / what hardware do people have they would like to
> > support?
> > * Upstream paths - potential blockers and how to overcome them. The
> > standard
> >   kernel drivers should be fairly straightforward, but once we start
> > looking at
> >   systems with a heavier userspace component, things will get more
> >   controversial!
> > * Fostering stronger userspace communities to allow these these
> > accelerators
> >   to be easily harnessed.
> >
> > So as ever with a linux community focusing on a particular topic, the
> > obvious solution is a mailing list. There are a number of options on how
> > do this.
> >
> > 1) Ask one of the industry bodies to host? Who?
> >
> > 2) Put together a compelling argument for linux-accelerators@vger.
> > kernel.org
> > as probably the most generic location for such a list.
> >
> > More open questions are
> > 1) Scope?
> >  * Would anyone ever use such an overarching list?
> >  * Are we better off with the usual adhoc list of 'interested parties' +
> > lkml?
> >  * Do we actually need to define the full scope - are we better with a
> > vague
> >    definition?
> >
> > 2) Is there an existing community we can use to discuss these issues?
> >    (beyond the obvious firehose of LKML).
> >
> > 3) Who else to approach for input on these general questions?
> >
> > In parallel to this there are elements such as git / patchwork etc but
> > they can all be done as they are needed.
> >
> > Thanks
> >
> > --
> > Jonathan Cameron
> > Huawei
> >  
> 
> 
> 

Re: Fostering linux community collaboration on hardware accelerators

[Sandy Harris]

I shortened the cc list radically. If the discussion continues, it may
be a good idea to add people back. I added John Gilmore since I cite
one of his posts below.

Jonathan Cameron <Jonathan.Cameron@huawei.com> wrote:

> On behalf of Huawei, I am looking into options to foster a wider community
> around the various ongoing projects related to Accelerator support within
> Linux.  The particular area of interest to Huawei is that of harnessing
> accelerators from userspace, but in a collaborative way with the kernel
> still able to make efficient use of them, where appropriate.
>
> We are keen to foster a wider community than one just focused on
> our own current technology.

Good stuff, but there are problems. e.g. see the thread starting
with my message here:
https://www.mail-archive.com/linux-crypto@vger.kernel.org/msg27274.html

My perspective is that of a crypto guy working on general-purpose
CPUs, anything from 32-bit ARM up. There are certainly problems for
devices with massive loads like a high-end router or with more limited
CPUs that I will not even pretend to address.

For me, far & away the biggest issue is having a good source of random
numbers; more-or-less all crypto depends on that. The Linux random(4)
RNG gets close, but there are cases where it may not be well
initialized soon enough on some systems. If a system provides a
hardware RNG, I will certainly use it to feed random(4). I do not care
nearly as much about anything else that might be in a hardware crypto
accelerator.

Separate accelerator devices require management, separating accesses
by different kernel threads or by user processes if they are allowed
to play, keeping them from seeing each other's keys, perhaps saving &
restoring state sometimes. Things that can be built into the CPU --
RNG instruction or register, AES instructions, Intel's instruction for
128-bit finite field multiplication which accelerates AES-GCM,
probably some I have not heard of -- require less management and are
usable by  any process, assuming either compiler support or some
assembler code. As a software guy, I'd far rather the hardware
designers gave me those than anything that needs a driver.

Re: Fostering linux community collaboration on hardware accelerators

[Jonathan Cameron]

On Wed, 11 Oct 2017 11:51:49 -0400
Sandy Harris <sandyinchina@gmail.com> wrote:

> I shortened the cc list radically. If the discussion continues, it may
> be a good idea to add people back. I added John Gilmore since I cite
> one of his posts below.

Fair enough - I have cc'd back in our internal list for now.
I cynically take the view that people in this community are very
good at ignoring emails they aren't interested in :)

> 
> Jonathan Cameron <Jonathan.Cameron@huawei.com> wrote:
> 
> > On behalf of Huawei, I am looking into options to foster a wider community
> > around the various ongoing projects related to Accelerator support within
> > Linux.  The particular area of interest to Huawei is that of harnessing
> > accelerators from userspace, but in a collaborative way with the kernel
> > still able to make efficient use of them, where appropriate.
> >
> > We are keen to foster a wider community than one just focused on
> > our own current technology.  
> 
> Good stuff, but there are problems. e.g. see the thread starting
> with my message here:
> https://www.mail-archive.com/linux-crypto@vger.kernel.org/msg27274.html

An interesting email - we have heard (and observed) the same
working with the optimized ARM algorithm implementations. In particular
the point about 'when a given implementation is quickest' is important.
This isn't even obvious for the various ARM options available in mainline.

There is almost always more overhead (for now ;) in using a crypto
accelerators than doing it on the CPU which puts an obvious issue
in place for small data packets.

It's not easy - however not all crypto needs to come out of the kernel
again and with care you can avoid duplicating the overheads. e.g. 
* Kernel TLS moves the crypto down into the kernel so data goes in 
  once (which has to happen anyway to get it to the network card).
* Storage encryption - again data is going across the kernel boundary anyway.

> 
> My perspective is that of a crypto guy working on general-purpose
> CPUs, anything from 32-bit ARM up. There are certainly problems for
> devices with massive loads like a high-end router or with more limited
> CPUs that I will not even pretend to address.

Agreed - the application area is certainly not general. This stuff
consumes a lot of silicon - you really have to need it to make it worth
the expense.
 
> For me, far & away the biggest issue is having a good source of random
> numbers; more-or-less all crypto depends on that. The Linux random(4)
> RNG gets close, but there are cases where it may not be well
> initialized soon enough on some systems. If a system provides a
> hardware RNG, I will certainly use it to feed random(4). I do not care
> nearly as much about anything else that might be in a hardware crypto
> accelerator.

Absolutely - that needs to be part of a complete solution.

> 
> Separate accelerator devices require management, separating accesses
> by different kernel threads or by user processes if they are allowed
> to play, keeping them from seeing each other's keys, perhaps saving &
> restoring state sometimes. 

This can be assisted a lot by hardware context management (this look similar
to what you see on high end network cards with things that look like
queue steering etc).  Note, what we are trying to avoid is everything
going full userspace driver stack as has happened for some of those sorts of
systems (e.g. Solarflare's offering)

This hardware is appearing in crypto accelerators - until it is there
I agree this is a really nasty problem - be it perhaps not an unsolvable
one.  There are intermediate levels where the hardware can handle a small
number of contexts but to be truely useful you need to be able to use lots
and have the throughput to support them all with reasonable latency.

> Things that can be built into the CPU --
> RNG instruction or register, AES instructions, Intel's instruction for
> 128-bit finite field multiplication which accelerates AES-GCM,
> probably some I have not heard of -- require less management and are
> usable by  any process, assuming either compiler support or some
> assembler code. As a software guy, I'd far rather the hardware
> designers gave me those than anything that needs a driver.

I absolutely agree. If you can get away with doing crypto
on a CPU (under constraints of power usage and other needs for the cpu)
then do it that way. 

The targets for this stuff are where crypto has become a bottleneck
not cases where a good software implementation is quick enough.

Our initial focus in my team at Huawei, has been crypto partly because we
have a reasonably capable engine to play with - it is just one of
many options moving forward.

Jonathan

Re: Fostering linux community collaboration on hardware accelerators

[Jonathan Cameron]

<snip>

> > So as ever with a linux community focusing on a particular topic, the
> > obvious solution is a mailing list. There are a number of options on how
> > do this.
> > 
> > 1) Ask one of the industry bodies to host? Who?
> > 
> > 2) Put together a compelling argument for linux-accelerators@vger.kernel.org
> > as probably the most generic location for such a list.  
> 
> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get set 
> up immediately (and if we want patchwork, patchwork.ozlabs.org is 
> available as always, no matter where the list is hosted).
> 

That would be great! Thanks for doing this. Much easier to find out what
such a list is useful for by the practical option of having a list and
see what people do with it.

Thanks,

Jonathan
> > More open questions are
> > 1) Scope?
> >   * Would anyone ever use such an overarching list?
> >   * Are we better off with the usual adhoc list of 'interested parties' + lkml?
> >   * Do we actually need to define the full scope - are we better with a vague
> >     definition?  
> 
> I think a list with a broad and vaguely defined scope is a good idea - 
> it would certainly be helpful to us to be able to follow what other 
> contributors are doing that could be relevant to our CAPI and OpenCAPI work.
> 
> > 
> > 2) Is there an existing community we can use to discuss these issues?
> >     (beyond the obvious firehose of LKML).
> > 
> > 3) Who else to approach for input on these general questions?
> > 
> > In parallel to this there are elements such as git / patchwork etc but
> > they can all be done as they are needed.
> > 
> > Thanks
> > 
> > --
> > Jonathan Cameron
> > Huawei
> >   
> 

Re: Fostering linux community collaboration on hardware accelerators

[Douglas Miller]

On 10/12/2017 10:48 AM, Francois Ozog wrote:
> On 12 October 2017 at 16:57, Jonathan Cameron
> <Jonathan.Cameron@huawei.com> wrote:
>> On Thu, 12 Oct 2017 08:31:36 -0500
>> Douglas Miller <dougmill@linux.vnet.ibm.com> wrote:
>>
>>> Not sure if you're already plugged-in to this, but the OpenMP group is
>>> (has been) working on Accelerator support.
>>>
>>> http://www.openmp.org/updates/openmp-accelerator-support-gpus/
>>>
>>> Maybe you are talking about a different aspect of accelerator support,
>>> but it seems prudent to involve OpenMP as much as makes sense.
>> That's certainly interesting and sits in the area of 'standard'
>> userspace code but it is (I think) really addressing only one aspect
>> of the wider support problem.
>>
>> I do like the emphasis on balancing between CPU and accelerator,
>> that is certainly an open question even a the lowest levels in
>> areas such as cryptography acceleration where you either run
>> out of hardware resources on your accelerator or you actually
>> have a usage pattern that would be quicker on the CPU due
>> to inherent overheads in (current) non cpu crypto engines.
>>
>> Thanks for the pointer.  I can see we are going to need some location
>> for resources like this to be gathered together.
>>
>> Jonathan
>>
>>>
>>> On 10/12/2017 12:22 AM, Andrew Donnellan wrote:
>>>> On 10/10/17 22:28, Jonathan Cameron wrote:
>>>>> Hi All,
>>>>>
>>>>> Please forward this email to anyone you think may be interested.
>>>> Have forwarded this to a number of relevant IBMers.
>>>>
>>>>> On behalf of Huawei, I am looking into options to foster a wider
>>>>> community
>>>>> around the various ongoing projects related to Accelerator support
>>>>> within
>>>>> Linux.  The particular area of interest to Huawei is that of harnessing
>>>>> accelerators from userspace, but in a collaborative way with the kernel
>>>>> still able to make efficient use of them, where appropriate.
>>>>>
>>>>> We are keen to foster a wider community than one just focused on
>>>>> our own current technology.  This is a field with no clear answers,
>>>>> so the
>>>>> widest possible range of input is needed!
>>>>>
>>>>> The address list of this email is drawn from people we have had
>>>>> discussions
>>>>> with or who have been suggested in response to Kenneth Lee's wrapdrive
>>>>> presentation at Linaro Connect and earlier presentations on the more
>>>>> general
>>>>> issue. A few relevant lists added to hopefully catch anyone we missed.
>>>>> My apologies to anyone who got swept up in this and isn't interested!
>>>>>
>>>>> Here we are defining accelerators fairly broadly - suggestions for a
>>>>> better
>>>>> term are also welcome.
>>>>>
>>>>> The infrastructure may be appropriate for:
>>>>> * Traditional offload engines - cryptography, compression and similar
>>>>> * Upcoming AI accelerators
>>>>> * ODP type requirements for access to elements of networking
>>>>> * Systems utilizing SVM including CCIX and other cache coherent buses
>>>>> * Many things we haven't thought of yet...
>>>>>
>>>>> As I see it, there are several aspects to this:
>>>>>
>>>>> 1) Kernel drivers for accelerators themselves.
>>>>>      * Traditional drivers such as crypto etc
>>>>>      - These already have their own communities. The main
>>>>>             focus of such work will always be through them.
>>>>>           - What a more general community could add here would be an
>>>>>             overview of the shared infrastructure of such devices.
>>>>>        This is particularly true around VFIO based (or similar)
>>>>>        userspace interfaces with a non trivial userspace component.
>>>>>      * How to support new types of accelerator?
>>>>>
>>>>> 2) The need for lightweight access paths from userspace that 'play
>>>>> well' and
>>>>>      share resources etc with standard in-kernel drivers.  This is the
>>>>> area
>>>>>      that Kenneth Lee and Huawei have been focusing on with their
>>>>> wrapdrive
>>>>>      effort. We know there are other similar efforts going on in other
>>>>> companies.
>>>>>      * This may involve interacting with existing kernel communities
>>>>> such as
>>>>>        those around VFIO and mdev.
>>>>>      * Resource management when we may have many consumers - not all
>>>>> hardware
>>>>>        has appropriate features to deal with this.
>>>>>
>>>>> 3) Usecases for accelerators. e.g.
>>>>>      * kTLS
>>>>>      * Storage encryption
>>>>>      * ODP - networking dataplane
>>>>>      * AI toolkits
>>>>>
>>>>> Discussions we want to get started include:
>>>>> * A wider range of hardware than we are currently considering. What
>>>>> makes
>>>>>     sense to target / what hardware do people have they would like to
>>>>> support?
>>>>> * Upstream paths - potential blockers and how to overcome them. The
>>>>> standard
>>>>>     kernel drivers should be fairly straightforward, but once we start
>>>>> looking at
>>>>>     systems with a heavier userspace component, things will get more
>>>>>     controversial!
>>>>> * Fostering stronger userspace communities to allow these these
>>>>> accelerators
>>>>>     to be easily harnessed.
>>>>>
>>>>> So as ever with a linux community focusing on a particular topic, the
>>>>> obvious solution is a mailing list. There are a number of options on how
>>>>> do this.
>>>>>
>>>>> 1) Ask one of the industry bodies to host? Who?
>>>>>
>>>>> 2) Put together a compelling argument for
>>>>> linux-accelerators@vger.kernel.org
>>>>> as probably the most generic location for such a list.
>>>> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get
>>>> set up immediately (and if we want patchwork, patchwork.ozlabs.org is
>>>> available as always, no matter where the list is hosted).
>>>>
>>>>> More open questions are
>>>>> 1) Scope?
>>>>>    * Would anyone ever use such an overarching list?
>>>>>    * Are we better off with the usual adhoc list of 'interested
>>>>> parties' + lkml?
>>>>>    * Do we actually need to define the full scope - are we better with
>>>>> a vague
>>>>>      definition?
>>>> I think a list with a broad and vaguely defined scope is a good idea -
>>>> it would certainly be helpful to us to be able to follow what other
>>>> contributors are doing that could be relevant to our CAPI and OpenCAPI
>>>> work.
>>>>
>>>>> 2) Is there an existing community we can use to discuss these issues?
>>>>>      (beyond the obvious firehose of LKML).
>>>>>
>>>>> 3) Who else to approach for input on these general questions?
>>>>>
>>>>> In parallel to this there are elements such as git / patchwork etc but
>>>>> they can all be done as they are needed.
>>>>>
>>>>> Thanks
>>>>>
>>>>> --
>>>>> Jonathan Cameron
>>>>> Huawei
>>>>>
> I'd like to keep sharing thoughts on this.
>
> I understand accelerators can be fixed/parameterized, reconfigurable
> (FPGA), programmable (GPUs, NPUs...).
> With that in mind, there is a preparation phase that can as simple as
> set some parameters, or as complex as loading a "kernel" to a GPU or
> send a bitstream to an FPGA.
> In some cases, there may even be a slicing phase where the accelerator
> is actually sliced to accommodate different "customers" on the host it
> serves.
> Then there is the data supply to the accelerator.
>
> Is it fair to say that one of the main concerns of your proposal is to
> focus on having the userland data supply to the accelerator be as
> native/direct as possible ?
> And if so, then OpenMP would be a user of the userland IO framework
> when it comes to data supply?
>
> It also reminds me some work done by the media community and GStreamer
> arround DMA buf which specializes in a domain where large video
> "chunks" passes from one functional block to the other with specific
> caching policies (write combining is a friend here). While for 100Gbps
> networking were we need to handle 142Mpps the nature of the datapath
> is very different.
>
> Would you like to address both classes of problems? (I mean class 1:
> large chunks of data to be shared between few consummers; class 2:
> very large number of small chunks of data shared with a few to a large
> number of consumers?)
>
>
I've been out of touch with OpenMP for a number of years now, but that 
standard is a programming paradigm, and not (necessarily) limited to 
userland (or kernel). My reason for bringing it up is to make sure the 
right people get involved to help keep OpenMP relevant for things like 
CAPI and intended uses in the kernel. I believe the intent of OpenMP is 
to create a paradigm that will work is (most) all cases.

Re: Fostering linux community collaboration on hardware accelerators

[Francois Ozog]

On 12 October 2017 at 16:57, Jonathan Cameron
<Jonathan.Cameron@huawei.com> wrote:
> On Thu, 12 Oct 2017 08:31:36 -0500
> Douglas Miller <dougmill@linux.vnet.ibm.com> wrote:
>
>> Not sure if you're already plugged-in to this, but the OpenMP group is
>> (has been) working on Accelerator support.
>>
>> http://www.openmp.org/updates/openmp-accelerator-support-gpus/
>>
>> Maybe you are talking about a different aspect of accelerator support,
>> but it seems prudent to involve OpenMP as much as makes sense.
>
> That's certainly interesting and sits in the area of 'standard'
> userspace code but it is (I think) really addressing only one aspect
> of the wider support problem.
>
> I do like the emphasis on balancing between CPU and accelerator,
> that is certainly an open question even a the lowest levels in
> areas such as cryptography acceleration where you either run
> out of hardware resources on your accelerator or you actually
> have a usage pattern that would be quicker on the CPU due
> to inherent overheads in (current) non cpu crypto engines.
>
> Thanks for the pointer.  I can see we are going to need some location
> for resources like this to be gathered together.
>
> Jonathan
>
>>
>>
>> On 10/12/2017 12:22 AM, Andrew Donnellan wrote:
>> > On 10/10/17 22:28, Jonathan Cameron wrote:
>> >> Hi All,
>> >>
>> >> Please forward this email to anyone you think may be interested.
>> >
>> > Have forwarded this to a number of relevant IBMers.
>> >
>> >> On behalf of Huawei, I am looking into options to foster a wider
>> >> community
>> >> around the various ongoing projects related to Accelerator support
>> >> within
>> >> Linux.  The particular area of interest to Huawei is that of harnessing
>> >> accelerators from userspace, but in a collaborative way with the kernel
>> >> still able to make efficient use of them, where appropriate.
>> >>
>> >> We are keen to foster a wider community than one just focused on
>> >> our own current technology.  This is a field with no clear answers,
>> >> so the
>> >> widest possible range of input is needed!
>> >>
>> >> The address list of this email is drawn from people we have had
>> >> discussions
>> >> with or who have been suggested in response to Kenneth Lee's wrapdrive
>> >> presentation at Linaro Connect and earlier presentations on the more
>> >> general
>> >> issue. A few relevant lists added to hopefully catch anyone we missed.
>> >> My apologies to anyone who got swept up in this and isn't interested!
>> >>
>> >> Here we are defining accelerators fairly broadly - suggestions for a
>> >> better
>> >> term are also welcome.
>> >>
>> >> The infrastructure may be appropriate for:
>> >> * Traditional offload engines - cryptography, compression and similar
>> >> * Upcoming AI accelerators
>> >> * ODP type requirements for access to elements of networking
>> >> * Systems utilizing SVM including CCIX and other cache coherent buses
>> >> * Many things we haven't thought of yet...
>> >>
>> >> As I see it, there are several aspects to this:
>> >>
>> >> 1) Kernel drivers for accelerators themselves.
>> >>     * Traditional drivers such as crypto etc
>> >>     - These already have their own communities. The main
>> >>            focus of such work will always be through them.
>> >>          - What a more general community could add here would be an
>> >>            overview of the shared infrastructure of such devices.
>> >>       This is particularly true around VFIO based (or similar)
>> >>       userspace interfaces with a non trivial userspace component.
>> >>     * How to support new types of accelerator?
>> >>
>> >> 2) The need for lightweight access paths from userspace that 'play
>> >> well' and
>> >>     share resources etc with standard in-kernel drivers.  This is the
>> >> area
>> >>     that Kenneth Lee and Huawei have been focusing on with their
>> >> wrapdrive
>> >>     effort. We know there are other similar efforts going on in other
>> >> companies.
>> >>     * This may involve interacting with existing kernel communities
>> >> such as
>> >>       those around VFIO and mdev.
>> >>     * Resource management when we may have many consumers - not all
>> >> hardware
>> >>       has appropriate features to deal with this.
>> >>
>> >> 3) Usecases for accelerators. e.g.
>> >>     * kTLS
>> >>     * Storage encryption
>> >>     * ODP - networking dataplane
>> >>     * AI toolkits
>> >>
>> >> Discussions we want to get started include:
>> >> * A wider range of hardware than we are currently considering. What
>> >> makes
>> >>    sense to target / what hardware do people have they would like to
>> >> support?
>> >> * Upstream paths - potential blockers and how to overcome them. The
>> >> standard
>> >>    kernel drivers should be fairly straightforward, but once we start
>> >> looking at
>> >>    systems with a heavier userspace component, things will get more
>> >>    controversial!
>> >> * Fostering stronger userspace communities to allow these these
>> >> accelerators
>> >>    to be easily harnessed.
>> >>
>> >> So as ever with a linux community focusing on a particular topic, the
>> >> obvious solution is a mailing list. There are a number of options on how
>> >> do this.
>> >>
>> >> 1) Ask one of the industry bodies to host? Who?
>> >>
>> >> 2) Put together a compelling argument for
>> >> linux-accelerators@vger.kernel.org
>> >> as probably the most generic location for such a list.
>> >
>> > Happy to offer linux-accelerators@lists.ozlabs.org, which I can get
>> > set up immediately (and if we want patchwork, patchwork.ozlabs.org is
>> > available as always, no matter where the list is hosted).
>> >
>> >> More open questions are
>> >> 1) Scope?
>> >>   * Would anyone ever use such an overarching list?
>> >>   * Are we better off with the usual adhoc list of 'interested
>> >> parties' + lkml?
>> >>   * Do we actually need to define the full scope - are we better with
>> >> a vague
>> >>     definition?
>> >
>> > I think a list with a broad and vaguely defined scope is a good idea -
>> > it would certainly be helpful to us to be able to follow what other
>> > contributors are doing that could be relevant to our CAPI and OpenCAPI
>> > work.
>> >
>> >>
>> >> 2) Is there an existing community we can use to discuss these issues?
>> >>     (beyond the obvious firehose of LKML).
>> >>
>> >> 3) Who else to approach for input on these general questions?
>> >>
>> >> In parallel to this there are elements such as git / patchwork etc but
>> >> they can all be done as they are needed.
>> >>
>> >> Thanks
>> >>
>> >> --
>> >> Jonathan Cameron
>> >> Huawei
>> >>
>> >
>>
>

I'd like to keep sharing thoughts on this.

I understand accelerators can be fixed/parameterized, reconfigurable
(FPGA), programmable (GPUs, NPUs...).
With that in mind, there is a preparation phase that can as simple as
set some parameters, or as complex as loading a "kernel" to a GPU or
send a bitstream to an FPGA.
In some cases, there may even be a slicing phase where the accelerator
is actually sliced to accommodate different "customers" on the host it
serves.
Then there is the data supply to the accelerator.

Is it fair to say that one of the main concerns of your proposal is to
focus on having the userland data supply to the accelerator be as
native/direct as possible ?
And if so, then OpenMP would be a user of the userland IO framework
when it comes to data supply?

It also reminds me some work done by the media community and GStreamer
arround DMA buf which specializes in a domain where large video
"chunks" passes from one functional block to the other with specific
caching policies (write combining is a friend here). While for 100Gbps
networking were we need to handle 142Mpps the nature of the datapath
is very different.

Would you like to address both classes of problems? (I mean class 1:
large chunks of data to be shared between few consummers; class 2:
very large number of small chunks of data shared with a few to a large
number of consumers?)


-- 
François-Frédéric Ozog | Director Linaro Networking Group
T: +33.67221.6485
francois.ozog@linaro.org | Skype: ffozog

Re: Fostering linux community collaboration on hardware accelerators

[Jonathan Cameron]

On Thu, 12 Oct 2017 08:31:36 -0500
Douglas Miller <dougmill@linux.vnet.ibm.com> wrote:

> Not sure if you're already plugged-in to this, but the OpenMP group is 
> (has been) working on Accelerator support.
> 
> http://www.openmp.org/updates/openmp-accelerator-support-gpus/
> 
> Maybe you are talking about a different aspect of accelerator support, 
> but it seems prudent to involve OpenMP as much as makes sense.

That's certainly interesting and sits in the area of 'standard'
userspace code but it is (I think) really addressing only one aspect
of the wider support problem.

I do like the emphasis on balancing between CPU and accelerator,
that is certainly an open question even a the lowest levels in
areas such as cryptography acceleration where you either run
out of hardware resources on your accelerator or you actually
have a usage pattern that would be quicker on the CPU due
to inherent overheads in (current) non cpu crypto engines.

Thanks for the pointer.  I can see we are going to need some location
for resources like this to be gathered together.

Jonathan

> 
> 
> On 10/12/2017 12:22 AM, Andrew Donnellan wrote:
> > On 10/10/17 22:28, Jonathan Cameron wrote:  
> >> Hi All,
> >>
> >> Please forward this email to anyone you think may be interested.  
> >
> > Have forwarded this to a number of relevant IBMers.
> >  
> >> On behalf of Huawei, I am looking into options to foster a wider 
> >> community
> >> around the various ongoing projects related to Accelerator support 
> >> within
> >> Linux.  The particular area of interest to Huawei is that of harnessing
> >> accelerators from userspace, but in a collaborative way with the kernel
> >> still able to make efficient use of them, where appropriate.
> >>
> >> We are keen to foster a wider community than one just focused on
> >> our own current technology.  This is a field with no clear answers, 
> >> so the
> >> widest possible range of input is needed!
> >>
> >> The address list of this email is drawn from people we have had 
> >> discussions
> >> with or who have been suggested in response to Kenneth Lee's wrapdrive
> >> presentation at Linaro Connect and earlier presentations on the more 
> >> general
> >> issue. A few relevant lists added to hopefully catch anyone we missed.
> >> My apologies to anyone who got swept up in this and isn't interested!
> >>
> >> Here we are defining accelerators fairly broadly - suggestions for a 
> >> better
> >> term are also welcome.
> >>
> >> The infrastructure may be appropriate for:
> >> * Traditional offload engines - cryptography, compression and similar
> >> * Upcoming AI accelerators
> >> * ODP type requirements for access to elements of networking
> >> * Systems utilizing SVM including CCIX and other cache coherent buses
> >> * Many things we haven't thought of yet...
> >>
> >> As I see it, there are several aspects to this:
> >>
> >> 1) Kernel drivers for accelerators themselves.
> >>     * Traditional drivers such as crypto etc
> >>     - These already have their own communities. The main
> >>            focus of such work will always be through them.
> >>          - What a more general community could add here would be an
> >>            overview of the shared infrastructure of such devices.
> >>       This is particularly true around VFIO based (or similar)
> >>       userspace interfaces with a non trivial userspace component.
> >>     * How to support new types of accelerator?
> >>
> >> 2) The need for lightweight access paths from userspace that 'play 
> >> well' and
> >>     share resources etc with standard in-kernel drivers.  This is the 
> >> area
> >>     that Kenneth Lee and Huawei have been focusing on with their 
> >> wrapdrive
> >>     effort. We know there are other similar efforts going on in other 
> >> companies.
> >>     * This may involve interacting with existing kernel communities 
> >> such as
> >>       those around VFIO and mdev.
> >>     * Resource management when we may have many consumers - not all 
> >> hardware
> >>       has appropriate features to deal with this.
> >>
> >> 3) Usecases for accelerators. e.g.
> >>     * kTLS
> >>     * Storage encryption
> >>     * ODP - networking dataplane
> >>     * AI toolkits
> >>
> >> Discussions we want to get started include:
> >> * A wider range of hardware than we are currently considering. What 
> >> makes
> >>    sense to target / what hardware do people have they would like to 
> >> support?
> >> * Upstream paths - potential blockers and how to overcome them. The 
> >> standard
> >>    kernel drivers should be fairly straightforward, but once we start 
> >> looking at
> >>    systems with a heavier userspace component, things will get more
> >>    controversial!
> >> * Fostering stronger userspace communities to allow these these 
> >> accelerators
> >>    to be easily harnessed.
> >>
> >> So as ever with a linux community focusing on a particular topic, the
> >> obvious solution is a mailing list. There are a number of options on how
> >> do this.
> >>
> >> 1) Ask one of the industry bodies to host? Who?
> >>
> >> 2) Put together a compelling argument for 
> >> linux-accelerators@vger.kernel.org
> >> as probably the most generic location for such a list.  
> >
> > Happy to offer linux-accelerators@lists.ozlabs.org, which I can get 
> > set up immediately (and if we want patchwork, patchwork.ozlabs.org is 
> > available as always, no matter where the list is hosted).
> >  
> >> More open questions are
> >> 1) Scope?
> >>   * Would anyone ever use such an overarching list?
> >>   * Are we better off with the usual adhoc list of 'interested 
> >> parties' + lkml?
> >>   * Do we actually need to define the full scope - are we better with 
> >> a vague
> >>     definition?  
> >
> > I think a list with a broad and vaguely defined scope is a good idea - 
> > it would certainly be helpful to us to be able to follow what other 
> > contributors are doing that could be relevant to our CAPI and OpenCAPI 
> > work.
> >  
> >>
> >> 2) Is there an existing community we can use to discuss these issues?
> >>     (beyond the obvious firehose of LKML).
> >>
> >> 3) Who else to approach for input on these general questions?
> >>
> >> In parallel to this there are elements such as git / patchwork etc but
> >> they can all be done as they are needed.
> >>
> >> Thanks
> >>
> >> -- 
> >> Jonathan Cameron
> >> Huawei
> >>  
> >  
> 

Re: Fostering linux community collaboration on hardware accelerators

[Douglas Miller]

Not sure if you're already plugged-in to this, but the OpenMP group is 
(has been) working on Accelerator support.

http://www.openmp.org/updates/openmp-accelerator-support-gpus/

Maybe you are talking about a different aspect of accelerator support, 
but it seems prudent to involve OpenMP as much as makes sense.


On 10/12/2017 12:22 AM, Andrew Donnellan wrote:
> On 10/10/17 22:28, Jonathan Cameron wrote:
>> Hi All,
>>
>> Please forward this email to anyone you think may be interested.
>
> Have forwarded this to a number of relevant IBMers.
>
>> On behalf of Huawei, I am looking into options to foster a wider 
>> community
>> around the various ongoing projects related to Accelerator support 
>> within
>> Linux.  The particular area of interest to Huawei is that of harnessing
>> accelerators from userspace, but in a collaborative way with the kernel
>> still able to make efficient use of them, where appropriate.
>>
>> We are keen to foster a wider community than one just focused on
>> our own current technology.  This is a field with no clear answers, 
>> so the
>> widest possible range of input is needed!
>>
>> The address list of this email is drawn from people we have had 
>> discussions
>> with or who have been suggested in response to Kenneth Lee's wrapdrive
>> presentation at Linaro Connect and earlier presentations on the more 
>> general
>> issue. A few relevant lists added to hopefully catch anyone we missed.
>> My apologies to anyone who got swept up in this and isn't interested!
>>
>> Here we are defining accelerators fairly broadly - suggestions for a 
>> better
>> term are also welcome.
>>
>> The infrastructure may be appropriate for:
>> * Traditional offload engines - cryptography, compression and similar
>> * Upcoming AI accelerators
>> * ODP type requirements for access to elements of networking
>> * Systems utilizing SVM including CCIX and other cache coherent buses
>> * Many things we haven't thought of yet...
>>
>> As I see it, there are several aspects to this:
>>
>> 1) Kernel drivers for accelerators themselves.
>>     * Traditional drivers such as crypto etc
>>     - These already have their own communities. The main
>>            focus of such work will always be through them.
>>          - What a more general community could add here would be an
>>            overview of the shared infrastructure of such devices.
>>       This is particularly true around VFIO based (or similar)
>>       userspace interfaces with a non trivial userspace component.
>>     * How to support new types of accelerator?
>>
>> 2) The need for lightweight access paths from userspace that 'play 
>> well' and
>>     share resources etc with standard in-kernel drivers.  This is the 
>> area
>>     that Kenneth Lee and Huawei have been focusing on with their 
>> wrapdrive
>>     effort. We know there are other similar efforts going on in other 
>> companies.
>>     * This may involve interacting with existing kernel communities 
>> such as
>>       those around VFIO and mdev.
>>     * Resource management when we may have many consumers - not all 
>> hardware
>>       has appropriate features to deal with this.
>>
>> 3) Usecases for accelerators. e.g.
>>     * kTLS
>>     * Storage encryption
>>     * ODP - networking dataplane
>>     * AI toolkits
>>
>> Discussions we want to get started include:
>> * A wider range of hardware than we are currently considering. What 
>> makes
>>    sense to target / what hardware do people have they would like to 
>> support?
>> * Upstream paths - potential blockers and how to overcome them. The 
>> standard
>>    kernel drivers should be fairly straightforward, but once we start 
>> looking at
>>    systems with a heavier userspace component, things will get more
>>    controversial!
>> * Fostering stronger userspace communities to allow these these 
>> accelerators
>>    to be easily harnessed.
>>
>> So as ever with a linux community focusing on a particular topic, the
>> obvious solution is a mailing list. There are a number of options on how
>> do this.
>>
>> 1) Ask one of the industry bodies to host? Who?
>>
>> 2) Put together a compelling argument for 
>> linux-accelerators@vger.kernel.org
>> as probably the most generic location for such a list.
>
> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get 
> set up immediately (and if we want patchwork, patchwork.ozlabs.org is 
> available as always, no matter where the list is hosted).
>
>> More open questions are
>> 1) Scope?
>>   * Would anyone ever use such an overarching list?
>>   * Are we better off with the usual adhoc list of 'interested 
>> parties' + lkml?
>>   * Do we actually need to define the full scope - are we better with 
>> a vague
>>     definition?
>
> I think a list with a broad and vaguely defined scope is a good idea - 
> it would certainly be helpful to us to be able to follow what other 
> contributors are doing that could be relevant to our CAPI and OpenCAPI 
> work.
>
>>
>> 2) Is there an existing community we can use to discuss these issues?
>>     (beyond the obvious firehose of LKML).
>>
>> 3) Who else to approach for input on these general questions?
>>
>> In parallel to this there are elements such as git / patchwork etc but
>> they can all be done as they are needed.
>>
>> Thanks
>>
>> -- 
>> Jonathan Cameron
>> Huawei
>>
>

Re: Fostering linux community collaboration on hardware accelerators

[Andrew Donnellan]

On 10/10/17 22:28, Jonathan Cameron wrote:
> Hi All,
> 
> Please forward this email to anyone you think may be interested.

Have forwarded this to a number of relevant IBMers.

> On behalf of Huawei, I am looking into options to foster a wider community
> around the various ongoing projects related to Accelerator support within
> Linux.  The particular area of interest to Huawei is that of harnessing
> accelerators from userspace, but in a collaborative way with the kernel
> still able to make efficient use of them, where appropriate.
> 
> We are keen to foster a wider community than one just focused on
> our own current technology.  This is a field with no clear answers, so the
> widest possible range of input is needed!
> 
> The address list of this email is drawn from people we have had discussions
> with or who have been suggested in response to Kenneth Lee's wrapdrive
> presentation at Linaro Connect and earlier presentations on the more general
> issue. A few relevant lists added to hopefully catch anyone we missed.
> My apologies to anyone who got swept up in this and isn't interested!
> 
> Here we are defining accelerators fairly broadly - suggestions for a better
> term are also welcome.
> 
> The infrastructure may be appropriate for:
> * Traditional offload engines - cryptography, compression and similar
> * Upcoming AI accelerators
> * ODP type requirements for access to elements of networking
> * Systems utilizing SVM including CCIX and other cache coherent buses
> * Many things we haven't thought of yet...
> 
> As I see it, there are several aspects to this:
> 
> 1) Kernel drivers for accelerators themselves.
>     * Traditional drivers such as crypto etc
> 	- These already have their own communities. The main
>            focus of such work will always be through them.
>          - What a more general community could add here would be an
>            overview of the shared infrastructure of such devices.
> 	  This is particularly true around VFIO based (or similar)
> 	  userspace interfaces with a non trivial userspace component.
>     * How to support new types of accelerator?
> 
> 2) The need for lightweight access paths from userspace that 'play well' and
>     share resources etc with standard in-kernel drivers.  This is the area
>     that Kenneth Lee and Huawei have been focusing on with their wrapdrive
>     effort. We know there are other similar efforts going on in other companies.
>     * This may involve interacting with existing kernel communities such as
>       those around VFIO and mdev.
>     * Resource management when we may have many consumers - not all hardware
>       has appropriate features to deal with this.
> 
> 3) Usecases for accelerators. e.g.
>     * kTLS
>     * Storage encryption
>     * ODP - networking dataplane
>     * AI toolkits
> 
> Discussions we want to get started include:
> * A wider range of hardware than we are currently considering. What makes
>    sense to target / what hardware do people have they would like to support?
> * Upstream paths - potential blockers and how to overcome them. The standard
>    kernel drivers should be fairly straightforward, but once we start looking at
>    systems with a heavier userspace component, things will get more
>    controversial!
> * Fostering stronger userspace communities to allow these these accelerators
>    to be easily harnessed.
> 
> So as ever with a linux community focusing on a particular topic, the
> obvious solution is a mailing list. There are a number of options on how
> do this.
> 
> 1) Ask one of the industry bodies to host? Who?
> 
> 2) Put together a compelling argument for linux-accelerators@vger.kernel.org
> as probably the most generic location for such a list.

Happy to offer linux-accelerators@lists.ozlabs.org, which I can get set 
up immediately (and if we want patchwork, patchwork.ozlabs.org is 
available as always, no matter where the list is hosted).

> More open questions are
> 1) Scope?
>   * Would anyone ever use such an overarching list?
>   * Are we better off with the usual adhoc list of 'interested parties' + lkml?
>   * Do we actually need to define the full scope - are we better with a vague
>     definition?

I think a list with a broad and vaguely defined scope is a good idea - 
it would certainly be helpful to us to be able to follow what other 
contributors are doing that could be relevant to our CAPI and OpenCAPI work.

> 
> 2) Is there an existing community we can use to discuss these issues?
>     (beyond the obvious firehose of LKML).
> 
> 3) Who else to approach for input on these general questions?
> 
> In parallel to this there are elements such as git / patchwork etc but
> they can all be done as they are needed.
> 
> Thanks
> 
> --
> Jonathan Cameron
> Huawei
> 

-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com  IBM Australia Limited