Re: Fostering linux community collaboration on hardware accelerators

Re: Fostering linux community collaboration on hardware accelerators

[Andrew Donnellan]

On 10/10/17 22:28, Jonathan Cameron wrote:
> Hi All,
> 
> Please forward this email to anyone you think may be interested.

Have forwarded this to a number of relevant IBMers.

> On behalf of Huawei, I am looking into options to foster a wider community
> around the various ongoing projects related to Accelerator support within
> Linux.  The particular area of interest to Huawei is that of harnessing
> accelerators from userspace, but in a collaborative way with the kernel
> still able to make efficient use of them, where appropriate.
> 
> We are keen to foster a wider community than one just focused on
> our own current technology.  This is a field with no clear answers, so the
> widest possible range of input is needed!
> 
> The address list of this email is drawn from people we have had discussions
> with or who have been suggested in response to Kenneth Lee's wrapdrive
> presentation at Linaro Connect and earlier presentations on the more general
> issue. A few relevant lists added to hopefully catch anyone we missed.
> My apologies to anyone who got swept up in this and isn't interested!
> 
> Here we are defining accelerators fairly broadly - suggestions for a better
> term are also welcome.
> 
> The infrastructure may be appropriate for:
> * Traditional offload engines - cryptography, compression and similar
> * Upcoming AI accelerators
> * ODP type requirements for access to elements of networking
> * Systems utilizing SVM including CCIX and other cache coherent buses
> * Many things we haven't thought of yet...
> 
> As I see it, there are several aspects to this:
> 
> 1) Kernel drivers for accelerators themselves.
>     * Traditional drivers such as crypto etc
> 	- These already have their own communities. The main
>            focus of such work will always be through them.
>          - What a more general community could add here would be an
>            overview of the shared infrastructure of such devices.
> 	  This is particularly true around VFIO based (or similar)
> 	  userspace interfaces with a non trivial userspace component.
>     * How to support new types of accelerator?
> 
> 2) The need for lightweight access paths from userspace that 'play well' and
>     share resources etc with standard in-kernel drivers.  This is the area
>     that Kenneth Lee and Huawei have been focusing on with their wrapdrive
>     effort. We know there are other similar efforts going on in other companies.
>     * This may involve interacting with existing kernel communities such as
>       those around VFIO and mdev.
>     * Resource management when we may have many consumers - not all hardware
>       has appropriate features to deal with this.
> 
> 3) Usecases for accelerators. e.g.
>     * kTLS
>     * Storage encryption
>     * ODP - networking dataplane
>     * AI toolkits
> 
> Discussions we want to get started include:
> * A wider range of hardware than we are currently considering. What makes
>    sense to target / what hardware do people have they would like to support?
> * Upstream paths - potential blockers and how to overcome them. The standard
>    kernel drivers should be fairly straightforward, but once we start looking at
>    systems with a heavier userspace component, things will get more
>    controversial!
> * Fostering stronger userspace communities to allow these these accelerators
>    to be easily harnessed.
> 
> So as ever with a linux community focusing on a particular topic, the
> obvious solution is a mailing list. There are a number of options on how
> do this.
> 
> 1) Ask one of the industry bodies to host? Who?
> 
> 2) Put together a compelling argument for linux-accelerators@vger.kernel.org
> as probably the most generic location for such a list.

Happy to offer linux-accelerators@lists.ozlabs.org, which I can get set 
up immediately (and if we want patchwork, patchwork.ozlabs.org is 
available as always, no matter where the list is hosted).

> More open questions are
> 1) Scope?
>   * Would anyone ever use such an overarching list?
>   * Are we better off with the usual adhoc list of 'interested parties' + lkml?
>   * Do we actually need to define the full scope - are we better with a vague
>     definition?

I think a list with a broad and vaguely defined scope is a good idea - 
it would certainly be helpful to us to be able to follow what other 
contributors are doing that could be relevant to our CAPI and OpenCAPI work.

> 
> 2) Is there an existing community we can use to discuss these issues?
>     (beyond the obvious firehose of LKML).
> 
> 3) Who else to approach for input on these general questions?
> 
> In parallel to this there are elements such as git / patchwork etc but
> they can all be done as they are needed.
> 
> Thanks
> 
> --
> Jonathan Cameron
> Huawei
> 

-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com  IBM Australia Limited

Re: Fostering linux community collaboration on hardware accelerators

[Douglas Miller]

Not sure if you're already plugged-in to this, but the OpenMP group is 
(has been) working on Accelerator support.

http://www.openmp.org/updates/openmp-accelerator-support-gpus/

Maybe you are talking about a different aspect of accelerator support, 
but it seems prudent to involve OpenMP as much as makes sense.


On 10/12/2017 12:22 AM, Andrew Donnellan wrote:
> On 10/10/17 22:28, Jonathan Cameron wrote:
>> Hi All,
>>
>> Please forward this email to anyone you think may be interested.
>
> Have forwarded this to a number of relevant IBMers.
>
>> On behalf of Huawei, I am looking into options to foster a wider 
>> community
>> around the various ongoing projects related to Accelerator support 
>> within
>> Linux.  The particular area of interest to Huawei is that of harnessing
>> accelerators from userspace, but in a collaborative way with the kernel
>> still able to make efficient use of them, where appropriate.
>>
>> We are keen to foster a wider community than one just focused on
>> our own current technology.  This is a field with no clear answers, 
>> so the
>> widest possible range of input is needed!
>>
>> The address list of this email is drawn from people we have had 
>> discussions
>> with or who have been suggested in response to Kenneth Lee's wrapdrive
>> presentation at Linaro Connect and earlier presentations on the more 
>> general
>> issue. A few relevant lists added to hopefully catch anyone we missed.
>> My apologies to anyone who got swept up in this and isn't interested!
>>
>> Here we are defining accelerators fairly broadly - suggestions for a 
>> better
>> term are also welcome.
>>
>> The infrastructure may be appropriate for:
>> * Traditional offload engines - cryptography, compression and similar
>> * Upcoming AI accelerators
>> * ODP type requirements for access to elements of networking
>> * Systems utilizing SVM including CCIX and other cache coherent buses
>> * Many things we haven't thought of yet...
>>
>> As I see it, there are several aspects to this:
>>
>> 1) Kernel drivers for accelerators themselves.
>>     * Traditional drivers such as crypto etc
>>     - These already have their own communities. The main
>>            focus of such work will always be through them.
>>          - What a more general community could add here would be an
>>            overview of the shared infrastructure of such devices.
>>       This is particularly true around VFIO based (or similar)
>>       userspace interfaces with a non trivial userspace component.
>>     * How to support new types of accelerator?
>>
>> 2) The need for lightweight access paths from userspace that 'play 
>> well' and
>>     share resources etc with standard in-kernel drivers.  This is the 
>> area
>>     that Kenneth Lee and Huawei have been focusing on with their 
>> wrapdrive
>>     effort. We know there are other similar efforts going on in other 
>> companies.
>>     * This may involve interacting with existing kernel communities 
>> such as
>>       those around VFIO and mdev.
>>     * Resource management when we may have many consumers - not all 
>> hardware
>>       has appropriate features to deal with this.
>>
>> 3) Usecases for accelerators. e.g.
>>     * kTLS
>>     * Storage encryption
>>     * ODP - networking dataplane
>>     * AI toolkits
>>
>> Discussions we want to get started include:
>> * A wider range of hardware than we are currently considering. What 
>> makes
>>    sense to target / what hardware do people have they would like to 
>> support?
>> * Upstream paths - potential blockers and how to overcome them. The 
>> standard
>>    kernel drivers should be fairly straightforward, but once we start 
>> looking at
>>    systems with a heavier userspace component, things will get more
>>    controversial!
>> * Fostering stronger userspace communities to allow these these 
>> accelerators
>>    to be easily harnessed.
>>
>> So as ever with a linux community focusing on a particular topic, the
>> obvious solution is a mailing list. There are a number of options on how
>> do this.
>>
>> 1) Ask one of the industry bodies to host? Who?
>>
>> 2) Put together a compelling argument for 
>> linux-accelerators@vger.kernel.org
>> as probably the most generic location for such a list.
>
> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get 
> set up immediately (and if we want patchwork, patchwork.ozlabs.org is 
> available as always, no matter where the list is hosted).
>
>> More open questions are
>> 1) Scope?
>>   * Would anyone ever use such an overarching list?
>>   * Are we better off with the usual adhoc list of 'interested 
>> parties' + lkml?
>>   * Do we actually need to define the full scope - are we better with 
>> a vague
>>     definition?
>
> I think a list with a broad and vaguely defined scope is a good idea - 
> it would certainly be helpful to us to be able to follow what other 
> contributors are doing that could be relevant to our CAPI and OpenCAPI 
> work.
>
>>
>> 2) Is there an existing community we can use to discuss these issues?
>>     (beyond the obvious firehose of LKML).
>>
>> 3) Who else to approach for input on these general questions?
>>
>> In parallel to this there are elements such as git / patchwork etc but
>> they can all be done as they are needed.
>>
>> Thanks
>>
>> -- 
>> Jonathan Cameron
>> Huawei
>>
>

Re: Fostering linux community collaboration on hardware accelerators

[Jonathan Cameron]

On Thu, 12 Oct 2017 08:31:36 -0500
Douglas Miller <dougmill@linux.vnet.ibm.com> wrote:

> Not sure if you're already plugged-in to this, but the OpenMP group is 
> (has been) working on Accelerator support.
> 
> http://www.openmp.org/updates/openmp-accelerator-support-gpus/
> 
> Maybe you are talking about a different aspect of accelerator support, 
> but it seems prudent to involve OpenMP as much as makes sense.

That's certainly interesting and sits in the area of 'standard'
userspace code but it is (I think) really addressing only one aspect
of the wider support problem.

I do like the emphasis on balancing between CPU and accelerator,
that is certainly an open question even a the lowest levels in
areas such as cryptography acceleration where you either run
out of hardware resources on your accelerator or you actually
have a usage pattern that would be quicker on the CPU due
to inherent overheads in (current) non cpu crypto engines.

Thanks for the pointer.  I can see we are going to need some location
for resources like this to be gathered together.

Jonathan

> 
> 
> On 10/12/2017 12:22 AM, Andrew Donnellan wrote:
> > On 10/10/17 22:28, Jonathan Cameron wrote:  
> >> Hi All,
> >>
> >> Please forward this email to anyone you think may be interested.  
> >
> > Have forwarded this to a number of relevant IBMers.
> >  
> >> On behalf of Huawei, I am looking into options to foster a wider 
> >> community
> >> around the various ongoing projects related to Accelerator support 
> >> within
> >> Linux.  The particular area of interest to Huawei is that of harnessing
> >> accelerators from userspace, but in a collaborative way with the kernel
> >> still able to make efficient use of them, where appropriate.
> >>
> >> We are keen to foster a wider community than one just focused on
> >> our own current technology.  This is a field with no clear answers, 
> >> so the
> >> widest possible range of input is needed!
> >>
> >> The address list of this email is drawn from people we have had 
> >> discussions
> >> with or who have been suggested in response to Kenneth Lee's wrapdrive
> >> presentation at Linaro Connect and earlier presentations on the more 
> >> general
> >> issue. A few relevant lists added to hopefully catch anyone we missed.
> >> My apologies to anyone who got swept up in this and isn't interested!
> >>
> >> Here we are defining accelerators fairly broadly - suggestions for a 
> >> better
> >> term are also welcome.
> >>
> >> The infrastructure may be appropriate for:
> >> * Traditional offload engines - cryptography, compression and similar
> >> * Upcoming AI accelerators
> >> * ODP type requirements for access to elements of networking
> >> * Systems utilizing SVM including CCIX and other cache coherent buses
> >> * Many things we haven't thought of yet...
> >>
> >> As I see it, there are several aspects to this:
> >>
> >> 1) Kernel drivers for accelerators themselves.
> >>     * Traditional drivers such as crypto etc
> >>     - These already have their own communities. The main
> >>            focus of such work will always be through them.
> >>          - What a more general community could add here would be an
> >>            overview of the shared infrastructure of such devices.
> >>       This is particularly true around VFIO based (or similar)
> >>       userspace interfaces with a non trivial userspace component.
> >>     * How to support new types of accelerator?
> >>
> >> 2) The need for lightweight access paths from userspace that 'play 
> >> well' and
> >>     share resources etc with standard in-kernel drivers.  This is the 
> >> area
> >>     that Kenneth Lee and Huawei have been focusing on with their 
> >> wrapdrive
> >>     effort. We know there are other similar efforts going on in other 
> >> companies.
> >>     * This may involve interacting with existing kernel communities 
> >> such as
> >>       those around VFIO and mdev.
> >>     * Resource management when we may have many consumers - not all 
> >> hardware
> >>       has appropriate features to deal with this.
> >>
> >> 3) Usecases for accelerators. e.g.
> >>     * kTLS
> >>     * Storage encryption
> >>     * ODP - networking dataplane
> >>     * AI toolkits
> >>
> >> Discussions we want to get started include:
> >> * A wider range of hardware than we are currently considering. What 
> >> makes
> >>    sense to target / what hardware do people have they would like to 
> >> support?
> >> * Upstream paths - potential blockers and how to overcome them. The 
> >> standard
> >>    kernel drivers should be fairly straightforward, but once we start 
> >> looking at
> >>    systems with a heavier userspace component, things will get more
> >>    controversial!
> >> * Fostering stronger userspace communities to allow these these 
> >> accelerators
> >>    to be easily harnessed.
> >>
> >> So as ever with a linux community focusing on a particular topic, the
> >> obvious solution is a mailing list. There are a number of options on how
> >> do this.
> >>
> >> 1) Ask one of the industry bodies to host? Who?
> >>
> >> 2) Put together a compelling argument for 
> >> linux-accelerators@vger.kernel.org
> >> as probably the most generic location for such a list.  
> >
> > Happy to offer linux-accelerators@lists.ozlabs.org, which I can get 
> > set up immediately (and if we want patchwork, patchwork.ozlabs.org is 
> > available as always, no matter where the list is hosted).
> >  
> >> More open questions are
> >> 1) Scope?
> >>   * Would anyone ever use such an overarching list?
> >>   * Are we better off with the usual adhoc list of 'interested 
> >> parties' + lkml?
> >>   * Do we actually need to define the full scope - are we better with 
> >> a vague
> >>     definition?  
> >
> > I think a list with a broad and vaguely defined scope is a good idea - 
> > it would certainly be helpful to us to be able to follow what other 
> > contributors are doing that could be relevant to our CAPI and OpenCAPI 
> > work.
> >  
> >>
> >> 2) Is there an existing community we can use to discuss these issues?
> >>     (beyond the obvious firehose of LKML).
> >>
> >> 3) Who else to approach for input on these general questions?
> >>
> >> In parallel to this there are elements such as git / patchwork etc but
> >> they can all be done as they are needed.
> >>
> >> Thanks
> >>
> >> -- 
> >> Jonathan Cameron
> >> Huawei
> >>  
> >  
> 

Re: Fostering linux community collaboration on hardware accelerators

[Francois Ozog]

On 12 October 2017 at 16:57, Jonathan Cameron
<Jonathan.Cameron@huawei.com> wrote:
> On Thu, 12 Oct 2017 08:31:36 -0500
> Douglas Miller <dougmill@linux.vnet.ibm.com> wrote:
>
>> Not sure if you're already plugged-in to this, but the OpenMP group is
>> (has been) working on Accelerator support.
>>
>> http://www.openmp.org/updates/openmp-accelerator-support-gpus/
>>
>> Maybe you are talking about a different aspect of accelerator support,
>> but it seems prudent to involve OpenMP as much as makes sense.
>
> That's certainly interesting and sits in the area of 'standard'
> userspace code but it is (I think) really addressing only one aspect
> of the wider support problem.
>
> I do like the emphasis on balancing between CPU and accelerator,
> that is certainly an open question even a the lowest levels in
> areas such as cryptography acceleration where you either run
> out of hardware resources on your accelerator or you actually
> have a usage pattern that would be quicker on the CPU due
> to inherent overheads in (current) non cpu crypto engines.
>
> Thanks for the pointer.  I can see we are going to need some location
> for resources like this to be gathered together.
>
> Jonathan
>
>>
>>
>> On 10/12/2017 12:22 AM, Andrew Donnellan wrote:
>> > On 10/10/17 22:28, Jonathan Cameron wrote:
>> >> Hi All,
>> >>
>> >> Please forward this email to anyone you think may be interested.
>> >
>> > Have forwarded this to a number of relevant IBMers.
>> >
>> >> On behalf of Huawei, I am looking into options to foster a wider
>> >> community
>> >> around the various ongoing projects related to Accelerator support
>> >> within
>> >> Linux.  The particular area of interest to Huawei is that of harnessing
>> >> accelerators from userspace, but in a collaborative way with the kernel
>> >> still able to make efficient use of them, where appropriate.
>> >>
>> >> We are keen to foster a wider community than one just focused on
>> >> our own current technology.  This is a field with no clear answers,
>> >> so the
>> >> widest possible range of input is needed!
>> >>
>> >> The address list of this email is drawn from people we have had
>> >> discussions
>> >> with or who have been suggested in response to Kenneth Lee's wrapdrive
>> >> presentation at Linaro Connect and earlier presentations on the more
>> >> general
>> >> issue. A few relevant lists added to hopefully catch anyone we missed.
>> >> My apologies to anyone who got swept up in this and isn't interested!
>> >>
>> >> Here we are defining accelerators fairly broadly - suggestions for a
>> >> better
>> >> term are also welcome.
>> >>
>> >> The infrastructure may be appropriate for:
>> >> * Traditional offload engines - cryptography, compression and similar
>> >> * Upcoming AI accelerators
>> >> * ODP type requirements for access to elements of networking
>> >> * Systems utilizing SVM including CCIX and other cache coherent buses
>> >> * Many things we haven't thought of yet...
>> >>
>> >> As I see it, there are several aspects to this:
>> >>
>> >> 1) Kernel drivers for accelerators themselves.
>> >>     * Traditional drivers such as crypto etc
>> >>     - These already have their own communities. The main
>> >>            focus of such work will always be through them.
>> >>          - What a more general community could add here would be an
>> >>            overview of the shared infrastructure of such devices.
>> >>       This is particularly true around VFIO based (or similar)
>> >>       userspace interfaces with a non trivial userspace component.
>> >>     * How to support new types of accelerator?
>> >>
>> >> 2) The need for lightweight access paths from userspace that 'play
>> >> well' and
>> >>     share resources etc with standard in-kernel drivers.  This is the
>> >> area
>> >>     that Kenneth Lee and Huawei have been focusing on with their
>> >> wrapdrive
>> >>     effort. We know there are other similar efforts going on in other
>> >> companies.
>> >>     * This may involve interacting with existing kernel communities
>> >> such as
>> >>       those around VFIO and mdev.
>> >>     * Resource management when we may have many consumers - not all
>> >> hardware
>> >>       has appropriate features to deal with this.
>> >>
>> >> 3) Usecases for accelerators. e.g.
>> >>     * kTLS
>> >>     * Storage encryption
>> >>     * ODP - networking dataplane
>> >>     * AI toolkits
>> >>
>> >> Discussions we want to get started include:
>> >> * A wider range of hardware than we are currently considering. What
>> >> makes
>> >>    sense to target / what hardware do people have they would like to
>> >> support?
>> >> * Upstream paths - potential blockers and how to overcome them. The
>> >> standard
>> >>    kernel drivers should be fairly straightforward, but once we start
>> >> looking at
>> >>    systems with a heavier userspace component, things will get more
>> >>    controversial!
>> >> * Fostering stronger userspace communities to allow these these
>> >> accelerators
>> >>    to be easily harnessed.
>> >>
>> >> So as ever with a linux community focusing on a particular topic, the
>> >> obvious solution is a mailing list. There are a number of options on how
>> >> do this.
>> >>
>> >> 1) Ask one of the industry bodies to host? Who?
>> >>
>> >> 2) Put together a compelling argument for
>> >> linux-accelerators@vger.kernel.org
>> >> as probably the most generic location for such a list.
>> >
>> > Happy to offer linux-accelerators@lists.ozlabs.org, which I can get
>> > set up immediately (and if we want patchwork, patchwork.ozlabs.org is
>> > available as always, no matter where the list is hosted).
>> >
>> >> More open questions are
>> >> 1) Scope?
>> >>   * Would anyone ever use such an overarching list?
>> >>   * Are we better off with the usual adhoc list of 'interested
>> >> parties' + lkml?
>> >>   * Do we actually need to define the full scope - are we better with
>> >> a vague
>> >>     definition?
>> >
>> > I think a list with a broad and vaguely defined scope is a good idea -
>> > it would certainly be helpful to us to be able to follow what other
>> > contributors are doing that could be relevant to our CAPI and OpenCAPI
>> > work.
>> >
>> >>
>> >> 2) Is there an existing community we can use to discuss these issues?
>> >>     (beyond the obvious firehose of LKML).
>> >>
>> >> 3) Who else to approach for input on these general questions?
>> >>
>> >> In parallel to this there are elements such as git / patchwork etc but
>> >> they can all be done as they are needed.
>> >>
>> >> Thanks
>> >>
>> >> --
>> >> Jonathan Cameron
>> >> Huawei
>> >>
>> >
>>
>

I'd like to keep sharing thoughts on this.

I understand accelerators can be fixed/parameterized, reconfigurable
(FPGA), programmable (GPUs, NPUs...).
With that in mind, there is a preparation phase that can as simple as
set some parameters, or as complex as loading a "kernel" to a GPU or
send a bitstream to an FPGA.
In some cases, there may even be a slicing phase where the accelerator
is actually sliced to accommodate different "customers" on the host it
serves.
Then there is the data supply to the accelerator.

Is it fair to say that one of the main concerns of your proposal is to
focus on having the userland data supply to the accelerator be as
native/direct as possible ?
And if so, then OpenMP would be a user of the userland IO framework
when it comes to data supply?

It also reminds me some work done by the media community and GStreamer
arround DMA buf which specializes in a domain where large video
"chunks" passes from one functional block to the other with specific
caching policies (write combining is a friend here). While for 100Gbps
networking were we need to handle 142Mpps the nature of the datapath
is very different.

Would you like to address both classes of problems? (I mean class 1:
large chunks of data to be shared between few consummers; class 2:
very large number of small chunks of data shared with a few to a large
number of consumers?)


-- 
François-Frédéric Ozog | Director Linaro Networking Group
T: +33.67221.6485
francois.ozog@linaro.org | Skype: ffozog

Re: Fostering linux community collaboration on hardware accelerators

[Douglas Miller]

On 10/12/2017 10:48 AM, Francois Ozog wrote:
> On 12 October 2017 at 16:57, Jonathan Cameron
> <Jonathan.Cameron@huawei.com> wrote:
>> On Thu, 12 Oct 2017 08:31:36 -0500
>> Douglas Miller <dougmill@linux.vnet.ibm.com> wrote:
>>
>>> Not sure if you're already plugged-in to this, but the OpenMP group is
>>> (has been) working on Accelerator support.
>>>
>>> http://www.openmp.org/updates/openmp-accelerator-support-gpus/
>>>
>>> Maybe you are talking about a different aspect of accelerator support,
>>> but it seems prudent to involve OpenMP as much as makes sense.
>> That's certainly interesting and sits in the area of 'standard'
>> userspace code but it is (I think) really addressing only one aspect
>> of the wider support problem.
>>
>> I do like the emphasis on balancing between CPU and accelerator,
>> that is certainly an open question even a the lowest levels in
>> areas such as cryptography acceleration where you either run
>> out of hardware resources on your accelerator or you actually
>> have a usage pattern that would be quicker on the CPU due
>> to inherent overheads in (current) non cpu crypto engines.
>>
>> Thanks for the pointer.  I can see we are going to need some location
>> for resources like this to be gathered together.
>>
>> Jonathan
>>
>>>
>>> On 10/12/2017 12:22 AM, Andrew Donnellan wrote:
>>>> On 10/10/17 22:28, Jonathan Cameron wrote:
>>>>> Hi All,
>>>>>
>>>>> Please forward this email to anyone you think may be interested.
>>>> Have forwarded this to a number of relevant IBMers.
>>>>
>>>>> On behalf of Huawei, I am looking into options to foster a wider
>>>>> community
>>>>> around the various ongoing projects related to Accelerator support
>>>>> within
>>>>> Linux.  The particular area of interest to Huawei is that of harnessing
>>>>> accelerators from userspace, but in a collaborative way with the kernel
>>>>> still able to make efficient use of them, where appropriate.
>>>>>
>>>>> We are keen to foster a wider community than one just focused on
>>>>> our own current technology.  This is a field with no clear answers,
>>>>> so the
>>>>> widest possible range of input is needed!
>>>>>
>>>>> The address list of this email is drawn from people we have had
>>>>> discussions
>>>>> with or who have been suggested in response to Kenneth Lee's wrapdrive
>>>>> presentation at Linaro Connect and earlier presentations on the more
>>>>> general
>>>>> issue. A few relevant lists added to hopefully catch anyone we missed.
>>>>> My apologies to anyone who got swept up in this and isn't interested!
>>>>>
>>>>> Here we are defining accelerators fairly broadly - suggestions for a
>>>>> better
>>>>> term are also welcome.
>>>>>
>>>>> The infrastructure may be appropriate for:
>>>>> * Traditional offload engines - cryptography, compression and similar
>>>>> * Upcoming AI accelerators
>>>>> * ODP type requirements for access to elements of networking
>>>>> * Systems utilizing SVM including CCIX and other cache coherent buses
>>>>> * Many things we haven't thought of yet...
>>>>>
>>>>> As I see it, there are several aspects to this:
>>>>>
>>>>> 1) Kernel drivers for accelerators themselves.
>>>>>      * Traditional drivers such as crypto etc
>>>>>      - These already have their own communities. The main
>>>>>             focus of such work will always be through them.
>>>>>           - What a more general community could add here would be an
>>>>>             overview of the shared infrastructure of such devices.
>>>>>        This is particularly true around VFIO based (or similar)
>>>>>        userspace interfaces with a non trivial userspace component.
>>>>>      * How to support new types of accelerator?
>>>>>
>>>>> 2) The need for lightweight access paths from userspace that 'play
>>>>> well' and
>>>>>      share resources etc with standard in-kernel drivers.  This is the
>>>>> area
>>>>>      that Kenneth Lee and Huawei have been focusing on with their
>>>>> wrapdrive
>>>>>      effort. We know there are other similar efforts going on in other
>>>>> companies.
>>>>>      * This may involve interacting with existing kernel communities
>>>>> such as
>>>>>        those around VFIO and mdev.
>>>>>      * Resource management when we may have many consumers - not all
>>>>> hardware
>>>>>        has appropriate features to deal with this.
>>>>>
>>>>> 3) Usecases for accelerators. e.g.
>>>>>      * kTLS
>>>>>      * Storage encryption
>>>>>      * ODP - networking dataplane
>>>>>      * AI toolkits
>>>>>
>>>>> Discussions we want to get started include:
>>>>> * A wider range of hardware than we are currently considering. What
>>>>> makes
>>>>>     sense to target / what hardware do people have they would like to
>>>>> support?
>>>>> * Upstream paths - potential blockers and how to overcome them. The
>>>>> standard
>>>>>     kernel drivers should be fairly straightforward, but once we start
>>>>> looking at
>>>>>     systems with a heavier userspace component, things will get more
>>>>>     controversial!
>>>>> * Fostering stronger userspace communities to allow these these
>>>>> accelerators
>>>>>     to be easily harnessed.
>>>>>
>>>>> So as ever with a linux community focusing on a particular topic, the
>>>>> obvious solution is a mailing list. There are a number of options on how
>>>>> do this.
>>>>>
>>>>> 1) Ask one of the industry bodies to host? Who?
>>>>>
>>>>> 2) Put together a compelling argument for
>>>>> linux-accelerators@vger.kernel.org
>>>>> as probably the most generic location for such a list.
>>>> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get
>>>> set up immediately (and if we want patchwork, patchwork.ozlabs.org is
>>>> available as always, no matter where the list is hosted).
>>>>
>>>>> More open questions are
>>>>> 1) Scope?
>>>>>    * Would anyone ever use such an overarching list?
>>>>>    * Are we better off with the usual adhoc list of 'interested
>>>>> parties' + lkml?
>>>>>    * Do we actually need to define the full scope - are we better with
>>>>> a vague
>>>>>      definition?
>>>> I think a list with a broad and vaguely defined scope is a good idea -
>>>> it would certainly be helpful to us to be able to follow what other
>>>> contributors are doing that could be relevant to our CAPI and OpenCAPI
>>>> work.
>>>>
>>>>> 2) Is there an existing community we can use to discuss these issues?
>>>>>      (beyond the obvious firehose of LKML).
>>>>>
>>>>> 3) Who else to approach for input on these general questions?
>>>>>
>>>>> In parallel to this there are elements such as git / patchwork etc but
>>>>> they can all be done as they are needed.
>>>>>
>>>>> Thanks
>>>>>
>>>>> --
>>>>> Jonathan Cameron
>>>>> Huawei
>>>>>
> I'd like to keep sharing thoughts on this.
>
> I understand accelerators can be fixed/parameterized, reconfigurable
> (FPGA), programmable (GPUs, NPUs...).
> With that in mind, there is a preparation phase that can as simple as
> set some parameters, or as complex as loading a "kernel" to a GPU or
> send a bitstream to an FPGA.
> In some cases, there may even be a slicing phase where the accelerator
> is actually sliced to accommodate different "customers" on the host it
> serves.
> Then there is the data supply to the accelerator.
>
> Is it fair to say that one of the main concerns of your proposal is to
> focus on having the userland data supply to the accelerator be as
> native/direct as possible ?
> And if so, then OpenMP would be a user of the userland IO framework
> when it comes to data supply?
>
> It also reminds me some work done by the media community and GStreamer
> arround DMA buf which specializes in a domain where large video
> "chunks" passes from one functional block to the other with specific
> caching policies (write combining is a friend here). While for 100Gbps
> networking were we need to handle 142Mpps the nature of the datapath
> is very different.
>
> Would you like to address both classes of problems? (I mean class 1:
> large chunks of data to be shared between few consummers; class 2:
> very large number of small chunks of data shared with a few to a large
> number of consumers?)
>
>
I've been out of touch with OpenMP for a number of years now, but that 
standard is a programming paradigm, and not (necessarily) limited to 
userland (or kernel). My reason for bringing it up is to make sure the 
right people get involved to help keep OpenMP relevant for things like 
CAPI and intended uses in the kernel. I believe the intent of OpenMP is 
to create a paradigm that will work is (most) all cases.

Re: Fostering linux community collaboration on hardware accelerators

[Jonathan Cameron]

<snip>

> > So as ever with a linux community focusing on a particular topic, the
> > obvious solution is a mailing list. There are a number of options on how
> > do this.
> > 
> > 1) Ask one of the industry bodies to host? Who?
> > 
> > 2) Put together a compelling argument for linux-accelerators@vger.kernel.org
> > as probably the most generic location for such a list.  
> 
> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get set 
> up immediately (and if we want patchwork, patchwork.ozlabs.org is 
> available as always, no matter where the list is hosted).
> 

That would be great! Thanks for doing this. Much easier to find out what
such a list is useful for by the practical option of having a list and
see what people do with it.

Thanks,

Jonathan
> > More open questions are
> > 1) Scope?
> >   * Would anyone ever use such an overarching list?
> >   * Are we better off with the usual adhoc list of 'interested parties' + lkml?
> >   * Do we actually need to define the full scope - are we better with a vague
> >     definition?  
> 
> I think a list with a broad and vaguely defined scope is a good idea - 
> it would certainly be helpful to us to be able to follow what other 
> contributors are doing that could be relevant to our CAPI and OpenCAPI work.
> 
> > 
> > 2) Is there an existing community we can use to discuss these issues?
> >     (beyond the obvious firehose of LKML).
> > 
> > 3) Who else to approach for input on these general questions?
> > 
> > In parallel to this there are elements such as git / patchwork etc but
> > they can all be done as they are needed.
> > 
> > Thanks
> > 
> > --
> > Jonathan Cameron
> > Huawei
> >   
> 

New Linux accelerators discussion list [was: Re: Fostering linux community collaboration on hardware accelerators]

[Andrew Donnellan]

On 17/10/17 01:07, Jonathan Cameron wrote:
> <snip>
> 
>>> So as ever with a linux community focusing on a particular topic, the
>>> obvious solution is a mailing list. There are a number of options on how
>>> do this.
>>>
>>> 1) Ask one of the industry bodies to host? Who?
>>>
>>> 2) Put together a compelling argument for linux-accelerators@vger.kernel.org
>>> as probably the most generic location for such a list.
>>
>> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get set
>> up immediately (and if we want patchwork, patchwork.ozlabs.org is
>> available as always, no matter where the list is hosted).
>>
> 
> That would be great! Thanks for doing this. Much easier to find out what
> such a list is useful for by the practical option of having a list and
> see what people do with it.

[+ LKML]

We now have a mailing list:

   List:      linux-accelerators@lists.ozlabs.org
   Info:      https://lists.ozlabs.org/listinfo/linux-accelerators
   Archives:  https://lists.ozlabs.org/pipermail/linux-accelerators

I haven't set up Patchwork as yet, but if people think that's a good 
idea I can get that done too.


Andrew

-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com  IBM Australia Limited

Re: New Linux accelerators discussion list [was: Re: Fostering linux community collaboration on hardware accelerators]

[Jonathan Cameron]

On Tue, 17 Oct 2017 11:00:40 +1100
Andrew Donnellan <andrew.donnellan@au1.ibm.com> wrote:

> On 17/10/17 01:07, Jonathan Cameron wrote:
> > <snip>
> >   
> >>> So as ever with a linux community focusing on a particular topic, the
> >>> obvious solution is a mailing list. There are a number of options on how
> >>> do this.
> >>>
> >>> 1) Ask one of the industry bodies to host? Who?
> >>>
> >>> 2) Put together a compelling argument for linux-accelerators@vger.kernel.org
> >>> as probably the most generic location for such a list.  
> >>
> >> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get set
> >> up immediately (and if we want patchwork, patchwork.ozlabs.org is
> >> available as always, no matter where the list is hosted).
> >>  
> > 
> > That would be great! Thanks for doing this. Much easier to find out what
> > such a list is useful for by the practical option of having a list and
> > see what people do with it.  
> 
> [+ LKML]
> 
> We now have a mailing list:
> 
>    List:      linux-accelerators@lists.ozlabs.org
>    Info:      https://lists.ozlabs.org/listinfo/linux-accelerators
>    Archives:  https://lists.ozlabs.org/pipermail/linux-accelerators
> 
> I haven't set up Patchwork as yet, but if people think that's a good 
> idea I can get that done too.
> 
> 
> Andrew
> 

Thanks Andrew.

A quick summary of initial thoughts on scope of this list for anyone
entering the discussion at this point.
Note it will hopefully evolve in whatever direction people find helpful.
This contains some suggestions not a barrier to wider scope!

There are a number of projects / applications involving what are
termed hardware accelerators.  These include:
* Traditional offload engines
	- Crypto, compression, media transcoding and similar accelerators
	- usecases including kTLS, Storage Encryption etc.
* Dynamic FPGA type accelerators
* ODP, DPDK and similar networking data plane - particularly dual stack
  solutions where the kernel 'plays nicely' with userspace drivers.
* AI Accelerators
* Shared Virtual Memory (SVM) bus systems including Open-CAPI, CCIX etc
* Fast data flow to/from userspace applications.

A number of existing project focus on these:
* Mainline kernel drivers
	- Numerous crypto drivers etc
	- Open-CAPI
* Various networking data plane activities
* VFIO based and similar userspace drivers
Hopefully this list can provide a broader forum where more general
questions are being considered.

The discussion that lead to this list was that a number of people would
like a general open forum on which to discuss ideas with scope beyond
simply one kernel subsystem or one particular userspace framework.

Topics might include
* RFCs and early reviews of new approaches.
* Similar hardware - who is trying to solve the same problems?
* What would we ideally want from new hardware iterations?
* Hardware description - the question of how to chose a particular
  crypto engine is very dependent on the particular data flows.
  Sometimes hardware accelerators don't actually help due to overheads.
  Understanding those barriers would be very useful.
* Upstream paths - blockers and how to work with the communities to
  overcome them.
* Fostering stronger userspace communities to allow these accelerators to 
  be easily harnessed.
	- A number of projects have been highlighted in this thread
	  OpenStack (cyborg project), openMP accelerator support 
* Robustness / security of userspace frameworks.
* Virtualisation of accelerators

Anyhow, this email was just meant to draw together some thoughts.
It will be interesting to see what the list actually gets used for :)

Jonathan

Re: New Linux accelerators discussion list [was: Re: Fostering linux community collaboration on hardware accelerators]

[Jonathan Cameron]

+ linux-accelerators@lists.ozlabs.org

Seems sensible to have this email actually go to the new list so
at least it appears in the archive.

Sorry all, I should have thought of this before pressing send,

Jonathan

On Tue, 17 Oct 2017 13:48:10 +0100
Jonathan Cameron <Jonathan.Cameron@huawei.com> wrote:

> On Tue, 17 Oct 2017 11:00:40 +1100
> Andrew Donnellan <andrew.donnellan@au1.ibm.com> wrote:
> 
> > On 17/10/17 01:07, Jonathan Cameron wrote:  
> > > <snip>
> > >     
> > >>> So as ever with a linux community focusing on a particular topic, the
> > >>> obvious solution is a mailing list. There are a number of options on how
> > >>> do this.
> > >>>
> > >>> 1) Ask one of the industry bodies to host? Who?
> > >>>
> > >>> 2) Put together a compelling argument for linux-accelerators@vger.kernel.org
> > >>> as probably the most generic location for such a list.    
> > >>
> > >> Happy to offer linux-accelerators@lists.ozlabs.org, which I can get set
> > >> up immediately (and if we want patchwork, patchwork.ozlabs.org is
> > >> available as always, no matter where the list is hosted).
> > >>    
> > > 
> > > That would be great! Thanks for doing this. Much easier to find out what
> > > such a list is useful for by the practical option of having a list and
> > > see what people do with it.    
> > 
> > [+ LKML]
> > 
> > We now have a mailing list:
> > 
> >    List:      linux-accelerators@lists.ozlabs.org
> >    Info:      https://lists.ozlabs.org/listinfo/linux-accelerators
> >    Archives:  https://lists.ozlabs.org/pipermail/linux-accelerators
> > 
> > I haven't set up Patchwork as yet, but if people think that's a good 
> > idea I can get that done too.
> > 
> > 
> > Andrew
> >   
> 
> Thanks Andrew.
> 
> A quick summary of initial thoughts on scope of this list for anyone
> entering the discussion at this point.
> Note it will hopefully evolve in whatever direction people find helpful.
> This contains some suggestions not a barrier to wider scope!
> 
> There are a number of projects / applications involving what are
> termed hardware accelerators.  These include:
> * Traditional offload engines
> 	- Crypto, compression, media transcoding and similar accelerators
> 	- usecases including kTLS, Storage Encryption etc.
> * Dynamic FPGA type accelerators
> * ODP, DPDK and similar networking data plane - particularly dual stack
>   solutions where the kernel 'plays nicely' with userspace drivers.
> * AI Accelerators
> * Shared Virtual Memory (SVM) bus systems including Open-CAPI, CCIX etc
> * Fast data flow to/from userspace applications.
> 
> A number of existing project focus on these:
> * Mainline kernel drivers
> 	- Numerous crypto drivers etc
> 	- Open-CAPI
> * Various networking data plane activities
> * VFIO based and similar userspace drivers
> Hopefully this list can provide a broader forum where more general
> questions are being considered.
> 
> The discussion that lead to this list was that a number of people would
> like a general open forum on which to discuss ideas with scope beyond
> simply one kernel subsystem or one particular userspace framework.
> 
> Topics might include
> * RFCs and early reviews of new approaches.
> * Similar hardware - who is trying to solve the same problems?
> * What would we ideally want from new hardware iterations?
> * Hardware description - the question of how to chose a particular
>   crypto engine is very dependent on the particular data flows.
>   Sometimes hardware accelerators don't actually help due to overheads.
>   Understanding those barriers would be very useful.
> * Upstream paths - blockers and how to work with the communities to
>   overcome them.
> * Fostering stronger userspace communities to allow these accelerators to 
>   be easily harnessed.
> 	- A number of projects have been highlighted in this thread
> 	  OpenStack (cyborg project), openMP accelerator support 
> * Robustness / security of userspace frameworks.
> * Virtualisation of accelerators
> 
> Anyhow, this email was just meant to draw together some thoughts.
> It will be interesting to see what the list actually gets used for :)
> 
> Jonathan