Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Daniel Vetter]

On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
> On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> > On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > >
> > > > Introduction:
> > > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > > workloads in a data center environment.
> > > >
> > > > The offical press release can be found at -
> > > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > >
> > > > The offical product website is -
> > > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > >
> > > > At the time of the offical press release, numerious technology news sites
> > > > also covered the product.  Doing a search of your favorite site is likely
> > > > to find their coverage of it.
> > > >
> > > > It is our goal to have the kernel driver for the product fully upstream.
> > > > The purpose of this RFC is to start that process.  We are still doing
> > > > development (see below), and thus not quite looking to gain acceptance quite
> > > > yet, but now that we have a working driver we beleive we are at the stage
> > > > where meaningful conversation with the community can occur.
> > >
> > >
> > > Hi Jeffery,
> > >
> > > Just wondering what the userspace/testing plans for this driver.
> > >
> > > This introduces a new user facing API for a device without pointers to
> > > users or tests for that API.
> >
> > We have daily internal testing, although I don't expect you to take my word
> > for that.
> >
> > I would like to get one of these devices into the hands of Linaro, so that
> > it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> > to convince the powers that be to make this happen.
> >
> > Regarding what the community could do on its own, everything but the Linux
> > driver is considered proprietary - that includes the on device firmware and
> > the entire userspace stack.  This is a decision above my pay grade.
>
> Ok, that's a decision you are going to have to push upward on, as we
> really can't take this without a working, open, userspace.

Uh wut.

So the merge criteria for drivers/accel (atm still drivers/misc but I
thought that was interim until more drivers showed up) isn't actually
"totally-not-a-gpu accel driver without open source userspace".

Instead it's "totally-not-a-gpu accel driver without open source
userspace" _and_ you have to be best buddies with Greg. Or at least
not be on the naughty company list. Since for habanalabs all you
wanted is a few test cases to exercise the ioctls. Not the entire
userspace.

The most bonkers part here is that drivers/gpu actually does have a bunch
of active contributors from codeaurora ...

> Especially given the copyright owner of this code, that would be just
> crazy and foolish to not have open userspace code as well.  Firmware
> would also be wonderful as well, go poke your lawyers about derivative
> work issues and the like for fun conversations :)
>
> So without that changed, I'm not going to take this, and push to object
> that anyone else take this.
>
> I'm not going to be able to review any of this code anymore until that
> changes, sorry.

So you couldn't review the habanalabs driver either?

Get some consistency into your decision making as maintainer. And don't
tell me or anyone else that this is complicated, gpu and rdma driver folks
very much told you and Olof last year that this is what you're getting
yourself into.

Cheers, Daniel

PS: I guess congrats for figuring out you can't write a totally-not-a-gpu
accel driver without making kernel and userspace parts derivatives works
of each another. We told you that last year.
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Daniel Vetter]

On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
> On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> > On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > >
> > > > Introduction:
> > > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > > workloads in a data center environment.
> > > >
> > > > The offical press release can be found at -
> > > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > >
> > > > The offical product website is -
> > > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > >
> > > > At the time of the offical press release, numerious technology news sites
> > > > also covered the product.  Doing a search of your favorite site is likely
> > > > to find their coverage of it.
> > > >
> > > > It is our goal to have the kernel driver for the product fully upstream.
> > > > The purpose of this RFC is to start that process.  We are still doing
> > > > development (see below), and thus not quite looking to gain acceptance quite
> > > > yet, but now that we have a working driver we beleive we are at the stage
> > > > where meaningful conversation with the community can occur.
> > >
> > >
> > > Hi Jeffery,
> > >
> > > Just wondering what the userspace/testing plans for this driver.
> > >
> > > This introduces a new user facing API for a device without pointers to
> > > users or tests for that API.
> >
> > We have daily internal testing, although I don't expect you to take my word
> > for that.
> >
> > I would like to get one of these devices into the hands of Linaro, so that
> > it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> > to convince the powers that be to make this happen.
> >
> > Regarding what the community could do on its own, everything but the Linux
> > driver is considered proprietary - that includes the on device firmware and
> > the entire userspace stack.  This is a decision above my pay grade.
>
> Ok, that's a decision you are going to have to push upward on, as we
> really can't take this without a working, open, userspace.

Uh wut.

So the merge criteria for drivers/accel (atm still drivers/misc but I
thought that was interim until more drivers showed up) isn't actually
"totally-not-a-gpu accel driver without open source userspace".

Instead it's "totally-not-a-gpu accel driver without open source
userspace" _and_ you have to be best buddies with Greg. Or at least
not be on the naughty company list. Since for habanalabs all you
wanted is a few test cases to exercise the ioctls. Not the entire
userspace.

The most bonkers part here is that drivers/gpu actually does have a bunch
of active contributors from codeaurora ...

> Especially given the copyright owner of this code, that would be just
> crazy and foolish to not have open userspace code as well.  Firmware
> would also be wonderful as well, go poke your lawyers about derivative
> work issues and the like for fun conversations :)
>
> So without that changed, I'm not going to take this, and push to object
> that anyone else take this.
>
> I'm not going to be able to review any of this code anymore until that
> changes, sorry.

So you couldn't review the habanalabs driver either?

Get some consistency into your decision making as maintainer. And don't
tell me or anyone else that this is complicated, gpu and rdma driver folks
very much told you and Olof last year that this is what you're getting
yourself into.

Cheers, Daniel

PS: I guess congrats for figuring out you can't write a totally-not-a-gpu
accel driver without making kernel and userspace parts derivatives works
of each another. We told you that last year.
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Jason Gunthorpe]

On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:

> Get some consistency into your decision making as maintainer. And don't
> tell me or anyone else that this is complicated, gpu and rdma driver folks
> very much told you and Olof last year that this is what you're getting
> yourself into.

It is complicated!

One of the big mistakes we learned from in RDMA is that we must have a
cannonical open userspace, that is at least the user side of the uABI
from the kernel. It doesn't have to do a lot but it does have to be
there and everyone must use it.

Some time ago it was all a fragmented mess where every HW had its own
library project with no community and that spilled into the kernel
where it became impossible to be sure everyone was playing nicely and
keeping their parts up to date. We are still digging out where I find
stuff in the kernel that just never seemed to make it into any
userspace..

I feel this is an essential ingredient, and I think I gave this advice
at LPC as well - it is important to start as a proper subsystem with a
proper standard user space. IMHO a random collection of opaque misc
drivers for incredibly complex HW is not going to magically gel into a
subsystem.

Given the state of the industry the userspace doesn't have to do
alot, and maybe that library exposes unique APIs for each HW, but it
is at least a rallying point to handle all these questions like: 'is
the proposed userspace enough?', give some consistency, and be ready
to add in those things that are common (like, say IOMMU PASID setup)

The uacce stuff is sort of interesting here as it does seem to take
some of that approach, it is really simplistic, but the basic idea of
creating a generic DMA work ring is in there, and probably applies
just as well to several of these 'totally-not-a-GPU' drivers.

The other key is that the uABI from the kernel does need to be very
flexible as really any new HW can appear with any new strange need all
the time, and there will not be detailed commonality between HWs. RDMA
has made this mistake a lot in the past too.

The newer RDMA netlink like API is actually turning out not bad for
this purpose.. (again something a subsystem could provide)

Also the approach in this driver to directly connect the device to
userspace for control commands has worked for RDMA in the past few
years.

Jason

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Jason Gunthorpe]

On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:

> Get some consistency into your decision making as maintainer. And don't
> tell me or anyone else that this is complicated, gpu and rdma driver folks
> very much told you and Olof last year that this is what you're getting
> yourself into.

It is complicated!

One of the big mistakes we learned from in RDMA is that we must have a
cannonical open userspace, that is at least the user side of the uABI
from the kernel. It doesn't have to do a lot but it does have to be
there and everyone must use it.

Some time ago it was all a fragmented mess where every HW had its own
library project with no community and that spilled into the kernel
where it became impossible to be sure everyone was playing nicely and
keeping their parts up to date. We are still digging out where I find
stuff in the kernel that just never seemed to make it into any
userspace..

I feel this is an essential ingredient, and I think I gave this advice
at LPC as well - it is important to start as a proper subsystem with a
proper standard user space. IMHO a random collection of opaque misc
drivers for incredibly complex HW is not going to magically gel into a
subsystem.

Given the state of the industry the userspace doesn't have to do
alot, and maybe that library exposes unique APIs for each HW, but it
is at least a rallying point to handle all these questions like: 'is
the proposed userspace enough?', give some consistency, and be ready
to add in those things that are common (like, say IOMMU PASID setup)

The uacce stuff is sort of interesting here as it does seem to take
some of that approach, it is really simplistic, but the basic idea of
creating a generic DMA work ring is in there, and probably applies
just as well to several of these 'totally-not-a-GPU' drivers.

The other key is that the uABI from the kernel does need to be very
flexible as really any new HW can appear with any new strange need all
the time, and there will not be detailed commonality between HWs. RDMA
has made this mistake a lot in the past too.

The newer RDMA netlink like API is actually turning out not bad for
this purpose.. (again something a subsystem could provide)

Also the approach in this driver to directly connect the device to
userspace for control commands has worked for RDMA in the past few
years.

Jason
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > Ok, that's a decision you are going to have to push upward on, as we
> > really can't take this without a working, open, userspace.
> 
> Uh wut.
> 
> So the merge criteria for drivers/accel (atm still drivers/misc but I
> thought that was interim until more drivers showed up) isn't actually
> "totally-not-a-gpu accel driver without open source userspace".
> 
> Instead it's "totally-not-a-gpu accel driver without open source
> userspace" _and_ you have to be best buddies with Greg. Or at least
> not be on the naughty company list. Since for habanalabs all you
> wanted is a few test cases to exercise the ioctls. Not the entire
> userspace.

Habanalabs now has their full library opensourced that their tools use
directly, so that's not an argument anymore.

My primary point here is the copyright owner of this code, because of
that, I'm not going to objet to allowing this to be merged without open
userspace code.

thanks,

greg k-h

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > Ok, that's a decision you are going to have to push upward on, as we
> > really can't take this without a working, open, userspace.
> 
> Uh wut.
> 
> So the merge criteria for drivers/accel (atm still drivers/misc but I
> thought that was interim until more drivers showed up) isn't actually
> "totally-not-a-gpu accel driver without open source userspace".
> 
> Instead it's "totally-not-a-gpu accel driver without open source
> userspace" _and_ you have to be best buddies with Greg. Or at least
> not be on the naughty company list. Since for habanalabs all you
> wanted is a few test cases to exercise the ioctls. Not the entire
> userspace.

Habanalabs now has their full library opensourced that their tools use
directly, so that's not an argument anymore.

My primary point here is the copyright owner of this code, because of
that, I'm not going to objet to allowing this to be merged without open
userspace code.

thanks,

greg k-h
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Bjorn Andersson]

On Tue 19 May 21:59 PDT 2020, Greg Kroah-Hartman wrote:

> On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > > Ok, that's a decision you are going to have to push upward on, as we
> > > really can't take this without a working, open, userspace.
> > 
> > Uh wut.
> > 
> > So the merge criteria for drivers/accel (atm still drivers/misc but I
> > thought that was interim until more drivers showed up) isn't actually
> > "totally-not-a-gpu accel driver without open source userspace".
> > 
> > Instead it's "totally-not-a-gpu accel driver without open source
> > userspace" _and_ you have to be best buddies with Greg. Or at least
> > not be on the naughty company list. Since for habanalabs all you
> > wanted is a few test cases to exercise the ioctls. Not the entire
> > userspace.
> 
> Habanalabs now has their full library opensourced that their tools use
> directly, so that's not an argument anymore.
> 
> My primary point here is the copyright owner of this code, because of
> that, I'm not going to objet to allowing this to be merged without open
> userspace code.
> 

So because it's copyright Linux Foundation you are going to accept it
without user space, after all?

Regards,
Bjorn

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Bjorn Andersson]

On Tue 19 May 21:59 PDT 2020, Greg Kroah-Hartman wrote:

> On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > > Ok, that's a decision you are going to have to push upward on, as we
> > > really can't take this without a working, open, userspace.
> > 
> > Uh wut.
> > 
> > So the merge criteria for drivers/accel (atm still drivers/misc but I
> > thought that was interim until more drivers showed up) isn't actually
> > "totally-not-a-gpu accel driver without open source userspace".
> > 
> > Instead it's "totally-not-a-gpu accel driver without open source
> > userspace" _and_ you have to be best buddies with Greg. Or at least
> > not be on the naughty company list. Since for habanalabs all you
> > wanted is a few test cases to exercise the ioctls. Not the entire
> > userspace.
> 
> Habanalabs now has their full library opensourced that their tools use
> directly, so that's not an argument anymore.
> 
> My primary point here is the copyright owner of this code, because of
> that, I'm not going to objet to allowing this to be merged without open
> userspace code.
> 

So because it's copyright Linux Foundation you are going to accept it
without user space, after all?

Regards,
Bjorn
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
> > On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> > > On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > > > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > > >
> > > > > Introduction:
> > > > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > > > workloads in a data center environment.
> > > > >
> > > > > The offical press release can be found at -
> > > > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > > >
> > > > > The offical product website is -
> > > > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > > >
> > > > > At the time of the offical press release, numerious technology news sites
> > > > > also covered the product.  Doing a search of your favorite site is likely
> > > > > to find their coverage of it.
> > > > >
> > > > > It is our goal to have the kernel driver for the product fully upstream.
> > > > > The purpose of this RFC is to start that process.  We are still doing
> > > > > development (see below), and thus not quite looking to gain acceptance quite
> > > > > yet, but now that we have a working driver we beleive we are at the stage
> > > > > where meaningful conversation with the community can occur.
> > > >
> > > >
> > > > Hi Jeffery,
> > > >
> > > > Just wondering what the userspace/testing plans for this driver.
> > > >
> > > > This introduces a new user facing API for a device without pointers to
> > > > users or tests for that API.
> > >
> > > We have daily internal testing, although I don't expect you to take my word
> > > for that.
> > >
> > > I would like to get one of these devices into the hands of Linaro, so that
> > > it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> > > to convince the powers that be to make this happen.
> > >
> > > Regarding what the community could do on its own, everything but the Linux
> > > driver is considered proprietary - that includes the on device firmware and
> > > the entire userspace stack.  This is a decision above my pay grade.
> >
> > Ok, that's a decision you are going to have to push upward on, as we
> > really can't take this without a working, open, userspace.
> 
> Uh wut.
> 
> So the merge criteria for drivers/accel (atm still drivers/misc but I
> thought that was interim until more drivers showed up) isn't actually
> "totally-not-a-gpu accel driver without open source userspace".
> 
> Instead it's "totally-not-a-gpu accel driver without open source
> userspace" _and_ you have to be best buddies with Greg. Or at least
> not be on the naughty company list. Since for habanalabs all you
> wanted is a few test cases to exercise the ioctls. Not the entire
> userspace.

Also, to be fair, I have changed my mind after seeing the mess of
complexity that these "ioctls for everyone!" type of pass-through
these kinds of drivers are creating.  You were right, we need open
userspace code in order to be able to properly evaluate and figure out
what they are doing is right or not and be able to maintain things over
time correctly.

So I was wrong, and you were right, my apologies for my previous
stubbornness.

thanks,

greg k-h

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
> > On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> > > On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > > > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > > >
> > > > > Introduction:
> > > > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > > > workloads in a data center environment.
> > > > >
> > > > > The offical press release can be found at -
> > > > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > > >
> > > > > The offical product website is -
> > > > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > > >
> > > > > At the time of the offical press release, numerious technology news sites
> > > > > also covered the product.  Doing a search of your favorite site is likely
> > > > > to find their coverage of it.
> > > > >
> > > > > It is our goal to have the kernel driver for the product fully upstream.
> > > > > The purpose of this RFC is to start that process.  We are still doing
> > > > > development (see below), and thus not quite looking to gain acceptance quite
> > > > > yet, but now that we have a working driver we beleive we are at the stage
> > > > > where meaningful conversation with the community can occur.
> > > >
> > > >
> > > > Hi Jeffery,
> > > >
> > > > Just wondering what the userspace/testing plans for this driver.
> > > >
> > > > This introduces a new user facing API for a device without pointers to
> > > > users or tests for that API.
> > >
> > > We have daily internal testing, although I don't expect you to take my word
> > > for that.
> > >
> > > I would like to get one of these devices into the hands of Linaro, so that
> > > it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> > > to convince the powers that be to make this happen.
> > >
> > > Regarding what the community could do on its own, everything but the Linux
> > > driver is considered proprietary - that includes the on device firmware and
> > > the entire userspace stack.  This is a decision above my pay grade.
> >
> > Ok, that's a decision you are going to have to push upward on, as we
> > really can't take this without a working, open, userspace.
> 
> Uh wut.
> 
> So the merge criteria for drivers/accel (atm still drivers/misc but I
> thought that was interim until more drivers showed up) isn't actually
> "totally-not-a-gpu accel driver without open source userspace".
> 
> Instead it's "totally-not-a-gpu accel driver without open source
> userspace" _and_ you have to be best buddies with Greg. Or at least
> not be on the naughty company list. Since for habanalabs all you
> wanted is a few test cases to exercise the ioctls. Not the entire
> userspace.

Also, to be fair, I have changed my mind after seeing the mess of
complexity that these "ioctls for everyone!" type of pass-through
these kinds of drivers are creating.  You were right, we need open
userspace code in order to be able to properly evaluate and figure out
what they are doing is right or not and be able to maintain things over
time correctly.

So I was wrong, and you were right, my apologies for my previous
stubbornness.

thanks,

greg k-h
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Tue, May 19, 2020 at 10:11:35PM -0700, Bjorn Andersson wrote:
> On Tue 19 May 21:59 PDT 2020, Greg Kroah-Hartman wrote:
> 
> > On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > > > Ok, that's a decision you are going to have to push upward on, as we
> > > > really can't take this without a working, open, userspace.
> > > 
> > > Uh wut.
> > > 
> > > So the merge criteria for drivers/accel (atm still drivers/misc but I
> > > thought that was interim until more drivers showed up) isn't actually
> > > "totally-not-a-gpu accel driver without open source userspace".
> > > 
> > > Instead it's "totally-not-a-gpu accel driver without open source
> > > userspace" _and_ you have to be best buddies with Greg. Or at least
> > > not be on the naughty company list. Since for habanalabs all you
> > > wanted is a few test cases to exercise the ioctls. Not the entire
> > > userspace.
> > 
> > Habanalabs now has their full library opensourced that their tools use
> > directly, so that's not an argument anymore.
> > 
> > My primary point here is the copyright owner of this code, because of
> > that, I'm not going to objet to allowing this to be merged without open
> > userspace code.
> > 
> 
> So because it's copyright Linux Foundation you are going to accept it
> without user space, after all?

Huh, no, the exact opposite, sorry, drop the "not" in that above
sentence.  My bad.

greg k-h

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Tue, May 19, 2020 at 10:11:35PM -0700, Bjorn Andersson wrote:
> On Tue 19 May 21:59 PDT 2020, Greg Kroah-Hartman wrote:
> 
> > On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > > > Ok, that's a decision you are going to have to push upward on, as we
> > > > really can't take this without a working, open, userspace.
> > > 
> > > Uh wut.
> > > 
> > > So the merge criteria for drivers/accel (atm still drivers/misc but I
> > > thought that was interim until more drivers showed up) isn't actually
> > > "totally-not-a-gpu accel driver without open source userspace".
> > > 
> > > Instead it's "totally-not-a-gpu accel driver without open source
> > > userspace" _and_ you have to be best buddies with Greg. Or at least
> > > not be on the naughty company list. Since for habanalabs all you
> > > wanted is a few test cases to exercise the ioctls. Not the entire
> > > userspace.
> > 
> > Habanalabs now has their full library opensourced that their tools use
> > directly, so that's not an argument anymore.
> > 
> > My primary point here is the copyright owner of this code, because of
> > that, I'm not going to objet to allowing this to be merged without open
> > userspace code.
> > 
> 
> So because it's copyright Linux Foundation you are going to accept it
> without user space, after all?

Huh, no, the exact opposite, sorry, drop the "not" in that above
sentence.  My bad.

greg k-h
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Daniel Vetter]

On Wed, May 20, 2020 at 7:15 AM Greg Kroah-Hartman
<gregkh@linuxfoundation.org> wrote:
>
> On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
> > > On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> > > > On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > > > > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > > > >
> > > > > > Introduction:
> > > > > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > > > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > > > > workloads in a data center environment.
> > > > > >
> > > > > > The offical press release can be found at -
> > > > > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > > > >
> > > > > > The offical product website is -
> > > > > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > > > >
> > > > > > At the time of the offical press release, numerious technology news sites
> > > > > > also covered the product.  Doing a search of your favorite site is likely
> > > > > > to find their coverage of it.
> > > > > >
> > > > > > It is our goal to have the kernel driver for the product fully upstream.
> > > > > > The purpose of this RFC is to start that process.  We are still doing
> > > > > > development (see below), and thus not quite looking to gain acceptance quite
> > > > > > yet, but now that we have a working driver we beleive we are at the stage
> > > > > > where meaningful conversation with the community can occur.
> > > > >
> > > > >
> > > > > Hi Jeffery,
> > > > >
> > > > > Just wondering what the userspace/testing plans for this driver.
> > > > >
> > > > > This introduces a new user facing API for a device without pointers to
> > > > > users or tests for that API.
> > > >
> > > > We have daily internal testing, although I don't expect you to take my word
> > > > for that.
> > > >
> > > > I would like to get one of these devices into the hands of Linaro, so that
> > > > it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> > > > to convince the powers that be to make this happen.
> > > >
> > > > Regarding what the community could do on its own, everything but the Linux
> > > > driver is considered proprietary - that includes the on device firmware and
> > > > the entire userspace stack.  This is a decision above my pay grade.
> > >
> > > Ok, that's a decision you are going to have to push upward on, as we
> > > really can't take this without a working, open, userspace.
> >
> > Uh wut.
> >
> > So the merge criteria for drivers/accel (atm still drivers/misc but I
> > thought that was interim until more drivers showed up) isn't actually
> > "totally-not-a-gpu accel driver without open source userspace".
> >
> > Instead it's "totally-not-a-gpu accel driver without open source
> > userspace" _and_ you have to be best buddies with Greg. Or at least
> > not be on the naughty company list. Since for habanalabs all you
> > wanted is a few test cases to exercise the ioctls. Not the entire
> > userspace.
>
> Also, to be fair, I have changed my mind after seeing the mess of
> complexity that these "ioctls for everyone!" type of pass-through
> these kinds of drivers are creating.  You were right, we need open
> userspace code in order to be able to properly evaluate and figure out
> what they are doing is right or not and be able to maintain things over
> time correctly.
>
> So I was wrong, and you were right, my apologies for my previous
> stubbornness.

Awesome and don't worry, I'm pretty sure we've all been stubborn
occasionally :-)

From a drivers/gpu pov I think still not quite there since we also
want to see the compiler for these programmable accelerator thingies.
But just having a fairly good consensus that "userspace library with
all the runtime stuff excluding compiler must be open" is a huge step
forward. Next step may be that we (kernel overall, drivers/gpu will
still ask for the full thing) have ISA docs for these programmable
things, so that we can also evaluate that aspect and gauge how many
security issues there might be. Plus have a fighting chance to fix up
the security leaks when (post smeltdown I don't really want to
consider this an if) someone finds a hole in the hw security wall. At
least in drivers/gpu we historically have a ton of drivers with
command checkers to validate what userspace wants to run on the
accelerator thingie. Both in cases where the hw was accidentally too
strict, and not strict enough.

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Daniel Vetter]

On Wed, May 20, 2020 at 7:15 AM Greg Kroah-Hartman
<gregkh@linuxfoundation.org> wrote:
>
> On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
> > > On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> > > > On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > > > > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > > > >
> > > > > > Introduction:
> > > > > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > > > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > > > > workloads in a data center environment.
> > > > > >
> > > > > > The offical press release can be found at -
> > > > > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > > > >
> > > > > > The offical product website is -
> > > > > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > > > >
> > > > > > At the time of the offical press release, numerious technology news sites
> > > > > > also covered the product.  Doing a search of your favorite site is likely
> > > > > > to find their coverage of it.
> > > > > >
> > > > > > It is our goal to have the kernel driver for the product fully upstream.
> > > > > > The purpose of this RFC is to start that process.  We are still doing
> > > > > > development (see below), and thus not quite looking to gain acceptance quite
> > > > > > yet, but now that we have a working driver we beleive we are at the stage
> > > > > > where meaningful conversation with the community can occur.
> > > > >
> > > > >
> > > > > Hi Jeffery,
> > > > >
> > > > > Just wondering what the userspace/testing plans for this driver.
> > > > >
> > > > > This introduces a new user facing API for a device without pointers to
> > > > > users or tests for that API.
> > > >
> > > > We have daily internal testing, although I don't expect you to take my word
> > > > for that.
> > > >
> > > > I would like to get one of these devices into the hands of Linaro, so that
> > > > it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> > > > to convince the powers that be to make this happen.
> > > >
> > > > Regarding what the community could do on its own, everything but the Linux
> > > > driver is considered proprietary - that includes the on device firmware and
> > > > the entire userspace stack.  This is a decision above my pay grade.
> > >
> > > Ok, that's a decision you are going to have to push upward on, as we
> > > really can't take this without a working, open, userspace.
> >
> > Uh wut.
> >
> > So the merge criteria for drivers/accel (atm still drivers/misc but I
> > thought that was interim until more drivers showed up) isn't actually
> > "totally-not-a-gpu accel driver without open source userspace".
> >
> > Instead it's "totally-not-a-gpu accel driver without open source
> > userspace" _and_ you have to be best buddies with Greg. Or at least
> > not be on the naughty company list. Since for habanalabs all you
> > wanted is a few test cases to exercise the ioctls. Not the entire
> > userspace.
>
> Also, to be fair, I have changed my mind after seeing the mess of
> complexity that these "ioctls for everyone!" type of pass-through
> these kinds of drivers are creating.  You were right, we need open
> userspace code in order to be able to properly evaluate and figure out
> what they are doing is right or not and be able to maintain things over
> time correctly.
>
> So I was wrong, and you were right, my apologies for my previous
> stubbornness.

Awesome and don't worry, I'm pretty sure we've all been stubborn
occasionally :-)

From a drivers/gpu pov I think still not quite there since we also
want to see the compiler for these programmable accelerator thingies.
But just having a fairly good consensus that "userspace library with
all the runtime stuff excluding compiler must be open" is a huge step
forward. Next step may be that we (kernel overall, drivers/gpu will
still ask for the full thing) have ISA docs for these programmable
things, so that we can also evaluate that aspect and gauge how many
security issues there might be. Plus have a fighting chance to fix up
the security leaks when (post smeltdown I don't really want to
consider this an if) someone finds a hole in the hw security wall. At
least in drivers/gpu we historically have a ton of drivers with
command checkers to validate what userspace wants to run on the
accelerator thingie. Both in cases where the hw was accidentally too
strict, and not strict enough.

Cheers, Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Jeffrey Hugo]

On 5/20/2020 2:34 AM, Daniel Vetter wrote:
> On Wed, May 20, 2020 at 7:15 AM Greg Kroah-Hartman
> <gregkh@linuxfoundation.org> wrote:
>>
>> On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
>>> On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
>>>> On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
>>>>> On 5/18/2020 11:08 PM, Dave Airlie wrote:
>>>>>> On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>>>>>>
>>>>>>> Introduction:
>>>>>>> Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
>>>>>>> SoC ASIC for the purpose of efficently running Deep Learning inference
>>>>>>> workloads in a data center environment.
>>>>>>>
>>>>>>> The offical press release can be found at -
>>>>>>> https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
>>>>>>>
>>>>>>> The offical product website is -
>>>>>>> https://www.qualcomm.com/products/datacenter-artificial-intelligence
>>>>>>>
>>>>>>> At the time of the offical press release, numerious technology news sites
>>>>>>> also covered the product.  Doing a search of your favorite site is likely
>>>>>>> to find their coverage of it.
>>>>>>>
>>>>>>> It is our goal to have the kernel driver for the product fully upstream.
>>>>>>> The purpose of this RFC is to start that process.  We are still doing
>>>>>>> development (see below), and thus not quite looking to gain acceptance quite
>>>>>>> yet, but now that we have a working driver we beleive we are at the stage
>>>>>>> where meaningful conversation with the community can occur.
>>>>>>
>>>>>>
>>>>>> Hi Jeffery,
>>>>>>
>>>>>> Just wondering what the userspace/testing plans for this driver.
>>>>>>
>>>>>> This introduces a new user facing API for a device without pointers to
>>>>>> users or tests for that API.
>>>>>
>>>>> We have daily internal testing, although I don't expect you to take my word
>>>>> for that.
>>>>>
>>>>> I would like to get one of these devices into the hands of Linaro, so that
>>>>> it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
>>>>> to convince the powers that be to make this happen.
>>>>>
>>>>> Regarding what the community could do on its own, everything but the Linux
>>>>> driver is considered proprietary - that includes the on device firmware and
>>>>> the entire userspace stack.  This is a decision above my pay grade.
>>>>
>>>> Ok, that's a decision you are going to have to push upward on, as we
>>>> really can't take this without a working, open, userspace.
>>>
>>> Uh wut.
>>>
>>> So the merge criteria for drivers/accel (atm still drivers/misc but I
>>> thought that was interim until more drivers showed up) isn't actually
>>> "totally-not-a-gpu accel driver without open source userspace".
>>>
>>> Instead it's "totally-not-a-gpu accel driver without open source
>>> userspace" _and_ you have to be best buddies with Greg. Or at least
>>> not be on the naughty company list. Since for habanalabs all you
>>> wanted is a few test cases to exercise the ioctls. Not the entire
>>> userspace.
>>
>> Also, to be fair, I have changed my mind after seeing the mess of
>> complexity that these "ioctls for everyone!" type of pass-through
>> these kinds of drivers are creating.  You were right, we need open
>> userspace code in order to be able to properly evaluate and figure out
>> what they are doing is right or not and be able to maintain things over
>> time correctly.
>>
>> So I was wrong, and you were right, my apologies for my previous
>> stubbornness.
> 
> Awesome and don't worry, I'm pretty sure we've all been stubborn
> occasionally :-)
> 
>  From a drivers/gpu pov I think still not quite there since we also
> want to see the compiler for these programmable accelerator thingies.
> But just having a fairly good consensus that "userspace library with
> all the runtime stuff excluding compiler must be open" is a huge step
> forward. Next step may be that we (kernel overall, drivers/gpu will
> still ask for the full thing) have ISA docs for these programmable
> things, so that we can also evaluate that aspect and gauge how many
> security issues there might be. Plus have a fighting chance to fix up
> the security leaks when (post smeltdown I don't really want to
> consider this an if) someone finds a hole in the hw security wall. At
> least in drivers/gpu we historically have a ton of drivers with
> command checkers to validate what userspace wants to run on the
> accelerator thingie. Both in cases where the hw was accidentally too
> strict, and not strict enough.

I think this provides a pretty clear guidance on what you/the community 
are looking for, both now and possibly in the future.

Thank you.

 From my perspective, it would be really nice if there was something 
like Mesa that was a/the standard for these sorts of accelerators.  Its 
somewhat the wild west, and we've struggled with it.

I don't work on the compiler end of things, but based on what I've seen 
in my project, I think the vendors are going to be highly resistant to 
opening that up.  There is more than just the raw instruction set that 
goes on in the device, and its viewed as "secret sauce" even though I 
agree with your previous statements on that viewpoint.
-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Jeffrey Hugo]

On 5/20/2020 2:34 AM, Daniel Vetter wrote:
> On Wed, May 20, 2020 at 7:15 AM Greg Kroah-Hartman
> <gregkh@linuxfoundation.org> wrote:
>>
>> On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
>>> On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
>>>> On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
>>>>> On 5/18/2020 11:08 PM, Dave Airlie wrote:
>>>>>> On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>>>>>>
>>>>>>> Introduction:
>>>>>>> Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
>>>>>>> SoC ASIC for the purpose of efficently running Deep Learning inference
>>>>>>> workloads in a data center environment.
>>>>>>>
>>>>>>> The offical press release can be found at -
>>>>>>> https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
>>>>>>>
>>>>>>> The offical product website is -
>>>>>>> https://www.qualcomm.com/products/datacenter-artificial-intelligence
>>>>>>>
>>>>>>> At the time of the offical press release, numerious technology news sites
>>>>>>> also covered the product.  Doing a search of your favorite site is likely
>>>>>>> to find their coverage of it.
>>>>>>>
>>>>>>> It is our goal to have the kernel driver for the product fully upstream.
>>>>>>> The purpose of this RFC is to start that process.  We are still doing
>>>>>>> development (see below), and thus not quite looking to gain acceptance quite
>>>>>>> yet, but now that we have a working driver we beleive we are at the stage
>>>>>>> where meaningful conversation with the community can occur.
>>>>>>
>>>>>>
>>>>>> Hi Jeffery,
>>>>>>
>>>>>> Just wondering what the userspace/testing plans for this driver.
>>>>>>
>>>>>> This introduces a new user facing API for a device without pointers to
>>>>>> users or tests for that API.
>>>>>
>>>>> We have daily internal testing, although I don't expect you to take my word
>>>>> for that.
>>>>>
>>>>> I would like to get one of these devices into the hands of Linaro, so that
>>>>> it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
>>>>> to convince the powers that be to make this happen.
>>>>>
>>>>> Regarding what the community could do on its own, everything but the Linux
>>>>> driver is considered proprietary - that includes the on device firmware and
>>>>> the entire userspace stack.  This is a decision above my pay grade.
>>>>
>>>> Ok, that's a decision you are going to have to push upward on, as we
>>>> really can't take this without a working, open, userspace.
>>>
>>> Uh wut.
>>>
>>> So the merge criteria for drivers/accel (atm still drivers/misc but I
>>> thought that was interim until more drivers showed up) isn't actually
>>> "totally-not-a-gpu accel driver without open source userspace".
>>>
>>> Instead it's "totally-not-a-gpu accel driver without open source
>>> userspace" _and_ you have to be best buddies with Greg. Or at least
>>> not be on the naughty company list. Since for habanalabs all you
>>> wanted is a few test cases to exercise the ioctls. Not the entire
>>> userspace.
>>
>> Also, to be fair, I have changed my mind after seeing the mess of
>> complexity that these "ioctls for everyone!" type of pass-through
>> these kinds of drivers are creating.  You were right, we need open
>> userspace code in order to be able to properly evaluate and figure out
>> what they are doing is right or not and be able to maintain things over
>> time correctly.
>>
>> So I was wrong, and you were right, my apologies for my previous
>> stubbornness.
> 
> Awesome and don't worry, I'm pretty sure we've all been stubborn
> occasionally :-)
> 
>  From a drivers/gpu pov I think still not quite there since we also
> want to see the compiler for these programmable accelerator thingies.
> But just having a fairly good consensus that "userspace library with
> all the runtime stuff excluding compiler must be open" is a huge step
> forward. Next step may be that we (kernel overall, drivers/gpu will
> still ask for the full thing) have ISA docs for these programmable
> things, so that we can also evaluate that aspect and gauge how many
> security issues there might be. Plus have a fighting chance to fix up
> the security leaks when (post smeltdown I don't really want to
> consider this an if) someone finds a hole in the hw security wall. At
> least in drivers/gpu we historically have a ton of drivers with
> command checkers to validate what userspace wants to run on the
> accelerator thingie. Both in cases where the hw was accidentally too
> strict, and not strict enough.

I think this provides a pretty clear guidance on what you/the community 
are looking for, both now and possibly in the future.

Thank you.

 From my perspective, it would be really nice if there was something 
like Mesa that was a/the standard for these sorts of accelerators.  Its 
somewhat the wild west, and we've struggled with it.

I don't work on the compiler end of things, but based on what I've seen 
in my project, I think the vendors are going to be highly resistant to 
opening that up.  There is more than just the raw instruction set that 
goes on in the device, and its viewed as "secret sauce" even though I 
agree with your previous statements on that viewpoint.
-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Daniel Vetter]

On Wed, May 20, 2020 at 08:48:13AM -0600, Jeffrey Hugo wrote:
> On 5/20/2020 2:34 AM, Daniel Vetter wrote:
> > On Wed, May 20, 2020 at 7:15 AM Greg Kroah-Hartman
> > <gregkh@linuxfoundation.org> wrote:
> > > 
> > > On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > > > On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
> > > > > On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> > > > > > On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > > > > > > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > > > > > > 
> > > > > > > > Introduction:
> > > > > > > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > > > > > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > > > > > > workloads in a data center environment.
> > > > > > > > 
> > > > > > > > The offical press release can be found at -
> > > > > > > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > > > > > > 
> > > > > > > > The offical product website is -
> > > > > > > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > > > > > > 
> > > > > > > > At the time of the offical press release, numerious technology news sites
> > > > > > > > also covered the product.  Doing a search of your favorite site is likely
> > > > > > > > to find their coverage of it.
> > > > > > > > 
> > > > > > > > It is our goal to have the kernel driver for the product fully upstream.
> > > > > > > > The purpose of this RFC is to start that process.  We are still doing
> > > > > > > > development (see below), and thus not quite looking to gain acceptance quite
> > > > > > > > yet, but now that we have a working driver we beleive we are at the stage
> > > > > > > > where meaningful conversation with the community can occur.
> > > > > > > 
> > > > > > > 
> > > > > > > Hi Jeffery,
> > > > > > > 
> > > > > > > Just wondering what the userspace/testing plans for this driver.
> > > > > > > 
> > > > > > > This introduces a new user facing API for a device without pointers to
> > > > > > > users or tests for that API.
> > > > > > 
> > > > > > We have daily internal testing, although I don't expect you to take my word
> > > > > > for that.
> > > > > > 
> > > > > > I would like to get one of these devices into the hands of Linaro, so that
> > > > > > it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> > > > > > to convince the powers that be to make this happen.
> > > > > > 
> > > > > > Regarding what the community could do on its own, everything but the Linux
> > > > > > driver is considered proprietary - that includes the on device firmware and
> > > > > > the entire userspace stack.  This is a decision above my pay grade.
> > > > > 
> > > > > Ok, that's a decision you are going to have to push upward on, as we
> > > > > really can't take this without a working, open, userspace.
> > > > 
> > > > Uh wut.
> > > > 
> > > > So the merge criteria for drivers/accel (atm still drivers/misc but I
> > > > thought that was interim until more drivers showed up) isn't actually
> > > > "totally-not-a-gpu accel driver without open source userspace".
> > > > 
> > > > Instead it's "totally-not-a-gpu accel driver without open source
> > > > userspace" _and_ you have to be best buddies with Greg. Or at least
> > > > not be on the naughty company list. Since for habanalabs all you
> > > > wanted is a few test cases to exercise the ioctls. Not the entire
> > > > userspace.
> > > 
> > > Also, to be fair, I have changed my mind after seeing the mess of
> > > complexity that these "ioctls for everyone!" type of pass-through
> > > these kinds of drivers are creating.  You were right, we need open
> > > userspace code in order to be able to properly evaluate and figure out
> > > what they are doing is right or not and be able to maintain things over
> > > time correctly.
> > > 
> > > So I was wrong, and you were right, my apologies for my previous
> > > stubbornness.
> > 
> > Awesome and don't worry, I'm pretty sure we've all been stubborn
> > occasionally :-)
> > 
> >  From a drivers/gpu pov I think still not quite there since we also
> > want to see the compiler for these programmable accelerator thingies.
> > But just having a fairly good consensus that "userspace library with
> > all the runtime stuff excluding compiler must be open" is a huge step
> > forward. Next step may be that we (kernel overall, drivers/gpu will
> > still ask for the full thing) have ISA docs for these programmable
> > things, so that we can also evaluate that aspect and gauge how many
> > security issues there might be. Plus have a fighting chance to fix up
> > the security leaks when (post smeltdown I don't really want to
> > consider this an if) someone finds a hole in the hw security wall. At
> > least in drivers/gpu we historically have a ton of drivers with
> > command checkers to validate what userspace wants to run on the
> > accelerator thingie. Both in cases where the hw was accidentally too
> > strict, and not strict enough.
> 
> I think this provides a pretty clear guidance on what you/the community are
> looking for, both now and possibly in the future.
> 
> Thank you.
> 
> From my perspective, it would be really nice if there was something like
> Mesa that was a/the standard for these sorts of accelerators.  Its somewhat
> the wild west, and we've struggled with it.

Yeah there's currently 0 standard api for ML accelerators that aren't
gpus. So not only don't we have mesa, we don't even have an api.

I do personally think that for anything that's somewhat programmable and
not just fixed-function matrix multiply accelerator for neural networks, a
slimmed down mesa vk or compute driver might be the most reasonable
starting point for an open source ecosystem for these things. Still a ton
of work, and if you're unlucky you might have bet on the wrong standard to
adapt for ML needs, atm no one can predict whether it's going to be vk, or
opencl or maybe something else entirely that will serve as the low-level
driver fabric underneath stuff like tenserflow and other ML libraries.

> I don't work on the compiler end of things, but based on what I've seen in
> my project, I think the vendors are going to be highly resistant to opening
> that up.  There is more than just the raw instruction set that goes on in
> the device, and its viewed as "secret sauce" even though I agree with your
> previous statements on that viewpoint.

Well it's the same with gpus. Sometimes you can get companies to open up
their runtimes. Almost never their compiler. A big problem seems to be
licensing troubles that prevent them from opening up the compiler - it's
often much cheaper and easier to just write a new open compiler than
trying to get the legal review for the existing one done. I don't think
there's a single case where a company succeeded in opening a gpu/accel
compiler that didn't start out with at least open source as an eventual
goal.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Daniel Vetter]

On Wed, May 20, 2020 at 08:48:13AM -0600, Jeffrey Hugo wrote:
> On 5/20/2020 2:34 AM, Daniel Vetter wrote:
> > On Wed, May 20, 2020 at 7:15 AM Greg Kroah-Hartman
> > <gregkh@linuxfoundation.org> wrote:
> > > 
> > > On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > > > On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
> > > > > On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> > > > > > On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > > > > > > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > > > > > > 
> > > > > > > > Introduction:
> > > > > > > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > > > > > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > > > > > > workloads in a data center environment.
> > > > > > > > 
> > > > > > > > The offical press release can be found at -
> > > > > > > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > > > > > > 
> > > > > > > > The offical product website is -
> > > > > > > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > > > > > > 
> > > > > > > > At the time of the offical press release, numerious technology news sites
> > > > > > > > also covered the product.  Doing a search of your favorite site is likely
> > > > > > > > to find their coverage of it.
> > > > > > > > 
> > > > > > > > It is our goal to have the kernel driver for the product fully upstream.
> > > > > > > > The purpose of this RFC is to start that process.  We are still doing
> > > > > > > > development (see below), and thus not quite looking to gain acceptance quite
> > > > > > > > yet, but now that we have a working driver we beleive we are at the stage
> > > > > > > > where meaningful conversation with the community can occur.
> > > > > > > 
> > > > > > > 
> > > > > > > Hi Jeffery,
> > > > > > > 
> > > > > > > Just wondering what the userspace/testing plans for this driver.
> > > > > > > 
> > > > > > > This introduces a new user facing API for a device without pointers to
> > > > > > > users or tests for that API.
> > > > > > 
> > > > > > We have daily internal testing, although I don't expect you to take my word
> > > > > > for that.
> > > > > > 
> > > > > > I would like to get one of these devices into the hands of Linaro, so that
> > > > > > it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> > > > > > to convince the powers that be to make this happen.
> > > > > > 
> > > > > > Regarding what the community could do on its own, everything but the Linux
> > > > > > driver is considered proprietary - that includes the on device firmware and
> > > > > > the entire userspace stack.  This is a decision above my pay grade.
> > > > > 
> > > > > Ok, that's a decision you are going to have to push upward on, as we
> > > > > really can't take this without a working, open, userspace.
> > > > 
> > > > Uh wut.
> > > > 
> > > > So the merge criteria for drivers/accel (atm still drivers/misc but I
> > > > thought that was interim until more drivers showed up) isn't actually
> > > > "totally-not-a-gpu accel driver without open source userspace".
> > > > 
> > > > Instead it's "totally-not-a-gpu accel driver without open source
> > > > userspace" _and_ you have to be best buddies with Greg. Or at least
> > > > not be on the naughty company list. Since for habanalabs all you
> > > > wanted is a few test cases to exercise the ioctls. Not the entire
> > > > userspace.
> > > 
> > > Also, to be fair, I have changed my mind after seeing the mess of
> > > complexity that these "ioctls for everyone!" type of pass-through
> > > these kinds of drivers are creating.  You were right, we need open
> > > userspace code in order to be able to properly evaluate and figure out
> > > what they are doing is right or not and be able to maintain things over
> > > time correctly.
> > > 
> > > So I was wrong, and you were right, my apologies for my previous
> > > stubbornness.
> > 
> > Awesome and don't worry, I'm pretty sure we've all been stubborn
> > occasionally :-)
> > 
> >  From a drivers/gpu pov I think still not quite there since we also
> > want to see the compiler for these programmable accelerator thingies.
> > But just having a fairly good consensus that "userspace library with
> > all the runtime stuff excluding compiler must be open" is a huge step
> > forward. Next step may be that we (kernel overall, drivers/gpu will
> > still ask for the full thing) have ISA docs for these programmable
> > things, so that we can also evaluate that aspect and gauge how many
> > security issues there might be. Plus have a fighting chance to fix up
> > the security leaks when (post smeltdown I don't really want to
> > consider this an if) someone finds a hole in the hw security wall. At
> > least in drivers/gpu we historically have a ton of drivers with
> > command checkers to validate what userspace wants to run on the
> > accelerator thingie. Both in cases where the hw was accidentally too
> > strict, and not strict enough.
> 
> I think this provides a pretty clear guidance on what you/the community are
> looking for, both now and possibly in the future.
> 
> Thank you.
> 
> From my perspective, it would be really nice if there was something like
> Mesa that was a/the standard for these sorts of accelerators.  Its somewhat
> the wild west, and we've struggled with it.

Yeah there's currently 0 standard api for ML accelerators that aren't
gpus. So not only don't we have mesa, we don't even have an api.

I do personally think that for anything that's somewhat programmable and
not just fixed-function matrix multiply accelerator for neural networks, a
slimmed down mesa vk or compute driver might be the most reasonable
starting point for an open source ecosystem for these things. Still a ton
of work, and if you're unlucky you might have bet on the wrong standard to
adapt for ML needs, atm no one can predict whether it's going to be vk, or
opencl or maybe something else entirely that will serve as the low-level
driver fabric underneath stuff like tenserflow and other ML libraries.

> I don't work on the compiler end of things, but based on what I've seen in
> my project, I think the vendors are going to be highly resistant to opening
> that up.  There is more than just the raw instruction set that goes on in
> the device, and its viewed as "secret sauce" even though I agree with your
> previous statements on that viewpoint.

Well it's the same with gpus. Sometimes you can get companies to open up
their runtimes. Almost never their compiler. A big problem seems to be
licensing troubles that prevent them from opening up the compiler - it's
often much cheaper and easier to just write a new open compiler than
trying to get the legal review for the existing one done. I don't think
there's a single case where a company succeeded in opening a gpu/accel
compiler that didn't start out with at least open source as an eventual
goal.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Wed, May 20, 2020 at 08:48:13AM -0600, Jeffrey Hugo wrote:
> On 5/20/2020 2:34 AM, Daniel Vetter wrote:
> > On Wed, May 20, 2020 at 7:15 AM Greg Kroah-Hartman
> > <gregkh@linuxfoundation.org> wrote:
> > > 
> > > On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > > > On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
> > > > > On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> > > > > > On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > > > > > > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > > > > > > 
> > > > > > > > Introduction:
> > > > > > > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > > > > > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > > > > > > workloads in a data center environment.
> > > > > > > > 
> > > > > > > > The offical press release can be found at -
> > > > > > > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > > > > > > 
> > > > > > > > The offical product website is -
> > > > > > > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > > > > > > 
> > > > > > > > At the time of the offical press release, numerious technology news sites
> > > > > > > > also covered the product.  Doing a search of your favorite site is likely
> > > > > > > > to find their coverage of it.
> > > > > > > > 
> > > > > > > > It is our goal to have the kernel driver for the product fully upstream.
> > > > > > > > The purpose of this RFC is to start that process.  We are still doing
> > > > > > > > development (see below), and thus not quite looking to gain acceptance quite
> > > > > > > > yet, but now that we have a working driver we beleive we are at the stage
> > > > > > > > where meaningful conversation with the community can occur.
> > > > > > > 
> > > > > > > 
> > > > > > > Hi Jeffery,
> > > > > > > 
> > > > > > > Just wondering what the userspace/testing plans for this driver.
> > > > > > > 
> > > > > > > This introduces a new user facing API for a device without pointers to
> > > > > > > users or tests for that API.
> > > > > > 
> > > > > > We have daily internal testing, although I don't expect you to take my word
> > > > > > for that.
> > > > > > 
> > > > > > I would like to get one of these devices into the hands of Linaro, so that
> > > > > > it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> > > > > > to convince the powers that be to make this happen.
> > > > > > 
> > > > > > Regarding what the community could do on its own, everything but the Linux
> > > > > > driver is considered proprietary - that includes the on device firmware and
> > > > > > the entire userspace stack.  This is a decision above my pay grade.
> > > > > 
> > > > > Ok, that's a decision you are going to have to push upward on, as we
> > > > > really can't take this without a working, open, userspace.
> > > > 
> > > > Uh wut.
> > > > 
> > > > So the merge criteria for drivers/accel (atm still drivers/misc but I
> > > > thought that was interim until more drivers showed up) isn't actually
> > > > "totally-not-a-gpu accel driver without open source userspace".
> > > > 
> > > > Instead it's "totally-not-a-gpu accel driver without open source
> > > > userspace" _and_ you have to be best buddies with Greg. Or at least
> > > > not be on the naughty company list. Since for habanalabs all you
> > > > wanted is a few test cases to exercise the ioctls. Not the entire
> > > > userspace.
> > > 
> > > Also, to be fair, I have changed my mind after seeing the mess of
> > > complexity that these "ioctls for everyone!" type of pass-through
> > > these kinds of drivers are creating.  You were right, we need open
> > > userspace code in order to be able to properly evaluate and figure out
> > > what they are doing is right or not and be able to maintain things over
> > > time correctly.
> > > 
> > > So I was wrong, and you were right, my apologies for my previous
> > > stubbornness.
> > 
> > Awesome and don't worry, I'm pretty sure we've all been stubborn
> > occasionally :-)
> > 
> >  From a drivers/gpu pov I think still not quite there since we also
> > want to see the compiler for these programmable accelerator thingies.
> > But just having a fairly good consensus that "userspace library with
> > all the runtime stuff excluding compiler must be open" is a huge step
> > forward. Next step may be that we (kernel overall, drivers/gpu will
> > still ask for the full thing) have ISA docs for these programmable
> > things, so that we can also evaluate that aspect and gauge how many
> > security issues there might be. Plus have a fighting chance to fix up
> > the security leaks when (post smeltdown I don't really want to
> > consider this an if) someone finds a hole in the hw security wall. At
> > least in drivers/gpu we historically have a ton of drivers with
> > command checkers to validate what userspace wants to run on the
> > accelerator thingie. Both in cases where the hw was accidentally too
> > strict, and not strict enough.
> 
> I think this provides a pretty clear guidance on what you/the community are
> looking for, both now and possibly in the future.
> 
> Thank you.
> 
> From my perspective, it would be really nice if there was something like
> Mesa that was a/the standard for these sorts of accelerators.  Its somewhat
> the wild west, and we've struggled with it.

Put a first cut at such a thing out there and see how it goes!  Nothing
is preventing you from starting such a project, and it would be most
welcome as you have seen.

good luck,

greg k-h

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Wed, May 20, 2020 at 08:48:13AM -0600, Jeffrey Hugo wrote:
> On 5/20/2020 2:34 AM, Daniel Vetter wrote:
> > On Wed, May 20, 2020 at 7:15 AM Greg Kroah-Hartman
> > <gregkh@linuxfoundation.org> wrote:
> > > 
> > > On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
> > > > On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
> > > > > On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> > > > > > On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > > > > > > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > > > > > > 
> > > > > > > > Introduction:
> > > > > > > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > > > > > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > > > > > > workloads in a data center environment.
> > > > > > > > 
> > > > > > > > The offical press release can be found at -
> > > > > > > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > > > > > > 
> > > > > > > > The offical product website is -
> > > > > > > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > > > > > > 
> > > > > > > > At the time of the offical press release, numerious technology news sites
> > > > > > > > also covered the product.  Doing a search of your favorite site is likely
> > > > > > > > to find their coverage of it.
> > > > > > > > 
> > > > > > > > It is our goal to have the kernel driver for the product fully upstream.
> > > > > > > > The purpose of this RFC is to start that process.  We are still doing
> > > > > > > > development (see below), and thus not quite looking to gain acceptance quite
> > > > > > > > yet, but now that we have a working driver we beleive we are at the stage
> > > > > > > > where meaningful conversation with the community can occur.
> > > > > > > 
> > > > > > > 
> > > > > > > Hi Jeffery,
> > > > > > > 
> > > > > > > Just wondering what the userspace/testing plans for this driver.
> > > > > > > 
> > > > > > > This introduces a new user facing API for a device without pointers to
> > > > > > > users or tests for that API.
> > > > > > 
> > > > > > We have daily internal testing, although I don't expect you to take my word
> > > > > > for that.
> > > > > > 
> > > > > > I would like to get one of these devices into the hands of Linaro, so that
> > > > > > it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> > > > > > to convince the powers that be to make this happen.
> > > > > > 
> > > > > > Regarding what the community could do on its own, everything but the Linux
> > > > > > driver is considered proprietary - that includes the on device firmware and
> > > > > > the entire userspace stack.  This is a decision above my pay grade.
> > > > > 
> > > > > Ok, that's a decision you are going to have to push upward on, as we
> > > > > really can't take this without a working, open, userspace.
> > > > 
> > > > Uh wut.
> > > > 
> > > > So the merge criteria for drivers/accel (atm still drivers/misc but I
> > > > thought that was interim until more drivers showed up) isn't actually
> > > > "totally-not-a-gpu accel driver without open source userspace".
> > > > 
> > > > Instead it's "totally-not-a-gpu accel driver without open source
> > > > userspace" _and_ you have to be best buddies with Greg. Or at least
> > > > not be on the naughty company list. Since for habanalabs all you
> > > > wanted is a few test cases to exercise the ioctls. Not the entire
> > > > userspace.
> > > 
> > > Also, to be fair, I have changed my mind after seeing the mess of
> > > complexity that these "ioctls for everyone!" type of pass-through
> > > these kinds of drivers are creating.  You were right, we need open
> > > userspace code in order to be able to properly evaluate and figure out
> > > what they are doing is right or not and be able to maintain things over
> > > time correctly.
> > > 
> > > So I was wrong, and you were right, my apologies for my previous
> > > stubbornness.
> > 
> > Awesome and don't worry, I'm pretty sure we've all been stubborn
> > occasionally :-)
> > 
> >  From a drivers/gpu pov I think still not quite there since we also
> > want to see the compiler for these programmable accelerator thingies.
> > But just having a fairly good consensus that "userspace library with
> > all the runtime stuff excluding compiler must be open" is a huge step
> > forward. Next step may be that we (kernel overall, drivers/gpu will
> > still ask for the full thing) have ISA docs for these programmable
> > things, so that we can also evaluate that aspect and gauge how many
> > security issues there might be. Plus have a fighting chance to fix up
> > the security leaks when (post smeltdown I don't really want to
> > consider this an if) someone finds a hole in the hw security wall. At
> > least in drivers/gpu we historically have a ton of drivers with
> > command checkers to validate what userspace wants to run on the
> > accelerator thingie. Both in cases where the hw was accidentally too
> > strict, and not strict enough.
> 
> I think this provides a pretty clear guidance on what you/the community are
> looking for, both now and possibly in the future.
> 
> Thank you.
> 
> From my perspective, it would be really nice if there was something like
> Mesa that was a/the standard for these sorts of accelerators.  Its somewhat
> the wild west, and we've struggled with it.

Put a first cut at such a thing out there and see how it goes!  Nothing
is preventing you from starting such a project, and it would be most
welcome as you have seen.

good luck,

greg k-h
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Jeffrey Hugo]

On 5/20/2020 9:59 AM, Greg Kroah-Hartman wrote:
> On Wed, May 20, 2020 at 08:48:13AM -0600, Jeffrey Hugo wrote:
>> On 5/20/2020 2:34 AM, Daniel Vetter wrote:
>>> On Wed, May 20, 2020 at 7:15 AM Greg Kroah-Hartman
>>> <gregkh@linuxfoundation.org> wrote:
>>>>
>>>> On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
>>>>> On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
>>>>>> On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
>>>>>>> On 5/18/2020 11:08 PM, Dave Airlie wrote:
>>>>>>>> On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>>>>>>>>
>>>>>>>>> Introduction:
>>>>>>>>> Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
>>>>>>>>> SoC ASIC for the purpose of efficently running Deep Learning inference
>>>>>>>>> workloads in a data center environment.
>>>>>>>>>
>>>>>>>>> The offical press release can be found at -
>>>>>>>>> https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
>>>>>>>>>
>>>>>>>>> The offical product website is -
>>>>>>>>> https://www.qualcomm.com/products/datacenter-artificial-intelligence
>>>>>>>>>
>>>>>>>>> At the time of the offical press release, numerious technology news sites
>>>>>>>>> also covered the product.  Doing a search of your favorite site is likely
>>>>>>>>> to find their coverage of it.
>>>>>>>>>
>>>>>>>>> It is our goal to have the kernel driver for the product fully upstream.
>>>>>>>>> The purpose of this RFC is to start that process.  We are still doing
>>>>>>>>> development (see below), and thus not quite looking to gain acceptance quite
>>>>>>>>> yet, but now that we have a working driver we beleive we are at the stage
>>>>>>>>> where meaningful conversation with the community can occur.
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Jeffery,
>>>>>>>>
>>>>>>>> Just wondering what the userspace/testing plans for this driver.
>>>>>>>>
>>>>>>>> This introduces a new user facing API for a device without pointers to
>>>>>>>> users or tests for that API.
>>>>>>>
>>>>>>> We have daily internal testing, although I don't expect you to take my word
>>>>>>> for that.
>>>>>>>
>>>>>>> I would like to get one of these devices into the hands of Linaro, so that
>>>>>>> it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
>>>>>>> to convince the powers that be to make this happen.
>>>>>>>
>>>>>>> Regarding what the community could do on its own, everything but the Linux
>>>>>>> driver is considered proprietary - that includes the on device firmware and
>>>>>>> the entire userspace stack.  This is a decision above my pay grade.
>>>>>>
>>>>>> Ok, that's a decision you are going to have to push upward on, as we
>>>>>> really can't take this without a working, open, userspace.
>>>>>
>>>>> Uh wut.
>>>>>
>>>>> So the merge criteria for drivers/accel (atm still drivers/misc but I
>>>>> thought that was interim until more drivers showed up) isn't actually
>>>>> "totally-not-a-gpu accel driver without open source userspace".
>>>>>
>>>>> Instead it's "totally-not-a-gpu accel driver without open source
>>>>> userspace" _and_ you have to be best buddies with Greg. Or at least
>>>>> not be on the naughty company list. Since for habanalabs all you
>>>>> wanted is a few test cases to exercise the ioctls. Not the entire
>>>>> userspace.
>>>>
>>>> Also, to be fair, I have changed my mind after seeing the mess of
>>>> complexity that these "ioctls for everyone!" type of pass-through
>>>> these kinds of drivers are creating.  You were right, we need open
>>>> userspace code in order to be able to properly evaluate and figure out
>>>> what they are doing is right or not and be able to maintain things over
>>>> time correctly.
>>>>
>>>> So I was wrong, and you were right, my apologies for my previous
>>>> stubbornness.
>>>
>>> Awesome and don't worry, I'm pretty sure we've all been stubborn
>>> occasionally :-)
>>>
>>>   From a drivers/gpu pov I think still not quite there since we also
>>> want to see the compiler for these programmable accelerator thingies.
>>> But just having a fairly good consensus that "userspace library with
>>> all the runtime stuff excluding compiler must be open" is a huge step
>>> forward. Next step may be that we (kernel overall, drivers/gpu will
>>> still ask for the full thing) have ISA docs for these programmable
>>> things, so that we can also evaluate that aspect and gauge how many
>>> security issues there might be. Plus have a fighting chance to fix up
>>> the security leaks when (post smeltdown I don't really want to
>>> consider this an if) someone finds a hole in the hw security wall. At
>>> least in drivers/gpu we historically have a ton of drivers with
>>> command checkers to validate what userspace wants to run on the
>>> accelerator thingie. Both in cases where the hw was accidentally too
>>> strict, and not strict enough.
>>
>> I think this provides a pretty clear guidance on what you/the community are
>> looking for, both now and possibly in the future.
>>
>> Thank you.
>>
>>  From my perspective, it would be really nice if there was something like
>> Mesa that was a/the standard for these sorts of accelerators.  Its somewhat
>> the wild west, and we've struggled with it.
> 
> Put a first cut at such a thing out there and see how it goes!  Nothing
> is preventing you from starting such a project, and it would be most
> welcome as you have seen.

I wish.  I'll float the idea, but don't hold your breath.

-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Jeffrey Hugo]

On 5/20/2020 9:59 AM, Greg Kroah-Hartman wrote:
> On Wed, May 20, 2020 at 08:48:13AM -0600, Jeffrey Hugo wrote:
>> On 5/20/2020 2:34 AM, Daniel Vetter wrote:
>>> On Wed, May 20, 2020 at 7:15 AM Greg Kroah-Hartman
>>> <gregkh@linuxfoundation.org> wrote:
>>>>
>>>> On Tue, May 19, 2020 at 10:41:15PM +0200, Daniel Vetter wrote:
>>>>> On Tue, May 19, 2020 at 07:41:20PM +0200, Greg Kroah-Hartman wrote:
>>>>>> On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
>>>>>>> On 5/18/2020 11:08 PM, Dave Airlie wrote:
>>>>>>>> On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>>>>>>>>
>>>>>>>>> Introduction:
>>>>>>>>> Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
>>>>>>>>> SoC ASIC for the purpose of efficently running Deep Learning inference
>>>>>>>>> workloads in a data center environment.
>>>>>>>>>
>>>>>>>>> The offical press release can be found at -
>>>>>>>>> https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
>>>>>>>>>
>>>>>>>>> The offical product website is -
>>>>>>>>> https://www.qualcomm.com/products/datacenter-artificial-intelligence
>>>>>>>>>
>>>>>>>>> At the time of the offical press release, numerious technology news sites
>>>>>>>>> also covered the product.  Doing a search of your favorite site is likely
>>>>>>>>> to find their coverage of it.
>>>>>>>>>
>>>>>>>>> It is our goal to have the kernel driver for the product fully upstream.
>>>>>>>>> The purpose of this RFC is to start that process.  We are still doing
>>>>>>>>> development (see below), and thus not quite looking to gain acceptance quite
>>>>>>>>> yet, but now that we have a working driver we beleive we are at the stage
>>>>>>>>> where meaningful conversation with the community can occur.
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Jeffery,
>>>>>>>>
>>>>>>>> Just wondering what the userspace/testing plans for this driver.
>>>>>>>>
>>>>>>>> This introduces a new user facing API for a device without pointers to
>>>>>>>> users or tests for that API.
>>>>>>>
>>>>>>> We have daily internal testing, although I don't expect you to take my word
>>>>>>> for that.
>>>>>>>
>>>>>>> I would like to get one of these devices into the hands of Linaro, so that
>>>>>>> it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
>>>>>>> to convince the powers that be to make this happen.
>>>>>>>
>>>>>>> Regarding what the community could do on its own, everything but the Linux
>>>>>>> driver is considered proprietary - that includes the on device firmware and
>>>>>>> the entire userspace stack.  This is a decision above my pay grade.
>>>>>>
>>>>>> Ok, that's a decision you are going to have to push upward on, as we
>>>>>> really can't take this without a working, open, userspace.
>>>>>
>>>>> Uh wut.
>>>>>
>>>>> So the merge criteria for drivers/accel (atm still drivers/misc but I
>>>>> thought that was interim until more drivers showed up) isn't actually
>>>>> "totally-not-a-gpu accel driver without open source userspace".
>>>>>
>>>>> Instead it's "totally-not-a-gpu accel driver without open source
>>>>> userspace" _and_ you have to be best buddies with Greg. Or at least
>>>>> not be on the naughty company list. Since for habanalabs all you
>>>>> wanted is a few test cases to exercise the ioctls. Not the entire
>>>>> userspace.
>>>>
>>>> Also, to be fair, I have changed my mind after seeing the mess of
>>>> complexity that these "ioctls for everyone!" type of pass-through
>>>> these kinds of drivers are creating.  You were right, we need open
>>>> userspace code in order to be able to properly evaluate and figure out
>>>> what they are doing is right or not and be able to maintain things over
>>>> time correctly.
>>>>
>>>> So I was wrong, and you were right, my apologies for my previous
>>>> stubbornness.
>>>
>>> Awesome and don't worry, I'm pretty sure we've all been stubborn
>>> occasionally :-)
>>>
>>>   From a drivers/gpu pov I think still not quite there since we also
>>> want to see the compiler for these programmable accelerator thingies.
>>> But just having a fairly good consensus that "userspace library with
>>> all the runtime stuff excluding compiler must be open" is a huge step
>>> forward. Next step may be that we (kernel overall, drivers/gpu will
>>> still ask for the full thing) have ISA docs for these programmable
>>> things, so that we can also evaluate that aspect and gauge how many
>>> security issues there might be. Plus have a fighting chance to fix up
>>> the security leaks when (post smeltdown I don't really want to
>>> consider this an if) someone finds a hole in the hw security wall. At
>>> least in drivers/gpu we historically have a ton of drivers with
>>> command checkers to validate what userspace wants to run on the
>>> accelerator thingie. Both in cases where the hw was accidentally too
>>> strict, and not strict enough.
>>
>> I think this provides a pretty clear guidance on what you/the community are
>> looking for, both now and possibly in the future.
>>
>> Thank you.
>>
>>  From my perspective, it would be really nice if there was something like
>> Mesa that was a/the standard for these sorts of accelerators.  Its somewhat
>> the wild west, and we've struggled with it.
> 
> Put a first cut at such a thing out there and see how it goes!  Nothing
> is preventing you from starting such a project, and it would be most
> welcome as you have seen.

I wish.  I'll float the idea, but don't hold your breath.

-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.
_______________________________________________
dri-devel mailing list
dri-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Tue, May 19, 2020 at 12:26:01PM -0600, Jeffrey Hugo wrote:
> On 5/19/2020 12:12 PM, Greg Kroah-Hartman wrote:
> > > > Especially given the copyright owner of this code, that would be just
> > > > crazy and foolish to not have open userspace code as well.  Firmware
> > > > would also be wonderful as well, go poke your lawyers about derivative
> > > > work issues and the like for fun conversations :)
> > > 
> > > Those are the kind of conversations I try to avoid  :)
> > 
> > Sounds like you are going to now have to have them, have fun!
> 
> Honestly, I fail to see where you think there is a derivative work, so, I'm
> not really sure what discussions I need to revisit with our lawyers.

Given that we are not lawyers, why don't we leave those types of
discussions up to the lawyers, and not depend on people like me and you
for that?  :)

If your lawyers think that the code division is fine as-is, that's
great, I'd be glad to review it if they add their signed-off-by: on it
verifying that the api divide is approved by them.

thanks!

greg k-h

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Jeffrey Hugo]

On 5/19/2020 12:12 PM, Greg Kroah-Hartman wrote:
> On Tue, May 19, 2020 at 12:07:03PM -0600, Jeffrey Hugo wrote:
>> On 5/19/2020 11:41 AM, Greg Kroah-Hartman wrote:
>>> On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
>>>> On 5/18/2020 11:08 PM, Dave Airlie wrote:
>>>>> On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>>>>>
>>>>>> Introduction:
>>>>>> Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
>>>>>> SoC ASIC for the purpose of efficently running Deep Learning inference
>>>>>> workloads in a data center environment.
>>>>>>
>>>>>> The offical press release can be found at -
>>>>>> https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
>>>>>>
>>>>>> The offical product website is -
>>>>>> https://www.qualcomm.com/products/datacenter-artificial-intelligence
>>>>>>
>>>>>> At the time of the offical press release, numerious technology news sites
>>>>>> also covered the product.  Doing a search of your favorite site is likely
>>>>>> to find their coverage of it.
>>>>>>
>>>>>> It is our goal to have the kernel driver for the product fully upstream.
>>>>>> The purpose of this RFC is to start that process.  We are still doing
>>>>>> development (see below), and thus not quite looking to gain acceptance quite
>>>>>> yet, but now that we have a working driver we beleive we are at the stage
>>>>>> where meaningful conversation with the community can occur.
>>>>>
>>>>>
>>>>> Hi Jeffery,
>>>>>
>>>>> Just wondering what the userspace/testing plans for this driver.
>>>>>
>>>>> This introduces a new user facing API for a device without pointers to
>>>>> users or tests for that API.
>>>>
>>>> We have daily internal testing, although I don't expect you to take my word
>>>> for that.
>>>>
>>>> I would like to get one of these devices into the hands of Linaro, so that
>>>> it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
>>>> to convince the powers that be to make this happen.
>>>>
>>>> Regarding what the community could do on its own, everything but the Linux
>>>> driver is considered proprietary - that includes the on device firmware and
>>>> the entire userspace stack.  This is a decision above my pay grade.
>>>
>>> Ok, that's a decision you are going to have to push upward on, as we
>>> really can't take this without a working, open, userspace.
>>
>> Fair enough.  I hope that your position may have made things easier for me.
>>
>> I hope this doesn't widen the rift as it were, but what is the "bar" for
>> this userspace?
>>
>> Is a simple test application that adds two numbers on the hardware
>> acceptable?
> 
> Make it the real library that you use for your applications that anyone
> can then also use as well if they have the hardware.  Why would you want
> something "crippled"?

It makes it easier to dance around real or perceived IP issues, and thus 
I can likely more successfully "push upward" as you put it.

> 
>> What is the bar "working"?  I intend to satisfy this request in good faith,
>> but I wonder, if no one has the hardware besides our customers, and possibly
>> KernelCI, can you really say that I've provided a working userspace?
> 
> How do you know who your customers really are, or who they sell the
> chips to?  I could end up with one of these... :)

At this time, I don't think that is going to happen, but I would like to 
see it regardless.

>>> Especially given the copyright owner of this code, that would be just
>>> crazy and foolish to not have open userspace code as well.  Firmware
>>> would also be wonderful as well, go poke your lawyers about derivative
>>> work issues and the like for fun conversations :)
>>
>> Those are the kind of conversations I try to avoid  :)
> 
> Sounds like you are going to now have to have them, have fun!

Honestly, I fail to see where you think there is a derivative work, so, 
I'm not really sure what discussions I need to revisit with our lawyers.

-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Tue, May 19, 2020 at 12:07:03PM -0600, Jeffrey Hugo wrote:
> On 5/19/2020 11:41 AM, Greg Kroah-Hartman wrote:
> > On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> > > On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > > > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > > > 
> > > > > Introduction:
> > > > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > > > workloads in a data center environment.
> > > > > 
> > > > > The offical press release can be found at -
> > > > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > > > 
> > > > > The offical product website is -
> > > > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > > > 
> > > > > At the time of the offical press release, numerious technology news sites
> > > > > also covered the product.  Doing a search of your favorite site is likely
> > > > > to find their coverage of it.
> > > > > 
> > > > > It is our goal to have the kernel driver for the product fully upstream.
> > > > > The purpose of this RFC is to start that process.  We are still doing
> > > > > development (see below), and thus not quite looking to gain acceptance quite
> > > > > yet, but now that we have a working driver we beleive we are at the stage
> > > > > where meaningful conversation with the community can occur.
> > > > 
> > > > 
> > > > Hi Jeffery,
> > > > 
> > > > Just wondering what the userspace/testing plans for this driver.
> > > > 
> > > > This introduces a new user facing API for a device without pointers to
> > > > users or tests for that API.
> > > 
> > > We have daily internal testing, although I don't expect you to take my word
> > > for that.
> > > 
> > > I would like to get one of these devices into the hands of Linaro, so that
> > > it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> > > to convince the powers that be to make this happen.
> > > 
> > > Regarding what the community could do on its own, everything but the Linux
> > > driver is considered proprietary - that includes the on device firmware and
> > > the entire userspace stack.  This is a decision above my pay grade.
> > 
> > Ok, that's a decision you are going to have to push upward on, as we
> > really can't take this without a working, open, userspace.
> 
> Fair enough.  I hope that your position may have made things easier for me.
> 
> I hope this doesn't widen the rift as it were, but what is the "bar" for
> this userspace?
> 
> Is a simple test application that adds two numbers on the hardware
> acceptable?

Make it the real library that you use for your applications that anyone
can then also use as well if they have the hardware.  Why would you want
something "crippled"?

> What is the bar "working"?  I intend to satisfy this request in good faith,
> but I wonder, if no one has the hardware besides our customers, and possibly
> KernelCI, can you really say that I've provided a working userspace?

How do you know who your customers really are, or who they sell the
chips to?  I could end up with one of these... :)

> > Especially given the copyright owner of this code, that would be just
> > crazy and foolish to not have open userspace code as well.  Firmware
> > would also be wonderful as well, go poke your lawyers about derivative
> > work issues and the like for fun conversations :)
> 
> Those are the kind of conversations I try to avoid  :)

Sounds like you are going to now have to have them, have fun!

greg k-h

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Jeffrey Hugo]

On 5/19/2020 11:41 AM, Greg Kroah-Hartman wrote:
> On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
>> On 5/18/2020 11:08 PM, Dave Airlie wrote:
>>> On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>>>
>>>> Introduction:
>>>> Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
>>>> SoC ASIC for the purpose of efficently running Deep Learning inference
>>>> workloads in a data center environment.
>>>>
>>>> The offical press release can be found at -
>>>> https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
>>>>
>>>> The offical product website is -
>>>> https://www.qualcomm.com/products/datacenter-artificial-intelligence
>>>>
>>>> At the time of the offical press release, numerious technology news sites
>>>> also covered the product.  Doing a search of your favorite site is likely
>>>> to find their coverage of it.
>>>>
>>>> It is our goal to have the kernel driver for the product fully upstream.
>>>> The purpose of this RFC is to start that process.  We are still doing
>>>> development (see below), and thus not quite looking to gain acceptance quite
>>>> yet, but now that we have a working driver we beleive we are at the stage
>>>> where meaningful conversation with the community can occur.
>>>
>>>
>>> Hi Jeffery,
>>>
>>> Just wondering what the userspace/testing plans for this driver.
>>>
>>> This introduces a new user facing API for a device without pointers to
>>> users or tests for that API.
>>
>> We have daily internal testing, although I don't expect you to take my word
>> for that.
>>
>> I would like to get one of these devices into the hands of Linaro, so that
>> it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
>> to convince the powers that be to make this happen.
>>
>> Regarding what the community could do on its own, everything but the Linux
>> driver is considered proprietary - that includes the on device firmware and
>> the entire userspace stack.  This is a decision above my pay grade.
> 
> Ok, that's a decision you are going to have to push upward on, as we
> really can't take this without a working, open, userspace.

Fair enough.  I hope that your position may have made things easier for me.

I hope this doesn't widen the rift as it were, but what is the "bar" for 
this userspace?

Is a simple test application that adds two numbers on the hardware 
acceptable?

What is the bar "working"?  I intend to satisfy this request in good 
faith, but I wonder, if no one has the hardware besides our customers, 
and possibly KernelCI, can you really say that I've provided a working 
userspace?

> Especially given the copyright owner of this code, that would be just
> crazy and foolish to not have open userspace code as well.  Firmware
> would also be wonderful as well, go poke your lawyers about derivative
> work issues and the like for fun conversations :)

Those are the kind of conversations I try to avoid  :)

> So without that changed, I'm not going to take this, and push to object
> that anyone else take this.
> 
> I'm not going to be able to review any of this code anymore until that
> changes, sorry.
> 
> thanks,
> 
> greg k-h
> 


-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Tue, May 19, 2020 at 08:57:38AM -0600, Jeffrey Hugo wrote:
> On 5/18/2020 11:08 PM, Dave Airlie wrote:
> > On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> > > 
> > > Introduction:
> > > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > > SoC ASIC for the purpose of efficently running Deep Learning inference
> > > workloads in a data center environment.
> > > 
> > > The offical press release can be found at -
> > > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> > > 
> > > The offical product website is -
> > > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> > > 
> > > At the time of the offical press release, numerious technology news sites
> > > also covered the product.  Doing a search of your favorite site is likely
> > > to find their coverage of it.
> > > 
> > > It is our goal to have the kernel driver for the product fully upstream.
> > > The purpose of this RFC is to start that process.  We are still doing
> > > development (see below), and thus not quite looking to gain acceptance quite
> > > yet, but now that we have a working driver we beleive we are at the stage
> > > where meaningful conversation with the community can occur.
> > 
> > 
> > Hi Jeffery,
> > 
> > Just wondering what the userspace/testing plans for this driver.
> > 
> > This introduces a new user facing API for a device without pointers to
> > users or tests for that API.
> 
> We have daily internal testing, although I don't expect you to take my word
> for that.
> 
> I would like to get one of these devices into the hands of Linaro, so that
> it can be put into KernelCI.  Similar to other Qualcomm products. I'm trying
> to convince the powers that be to make this happen.
> 
> Regarding what the community could do on its own, everything but the Linux
> driver is considered proprietary - that includes the on device firmware and
> the entire userspace stack.  This is a decision above my pay grade.

Ok, that's a decision you are going to have to push upward on, as we
really can't take this without a working, open, userspace.

Especially given the copyright owner of this code, that would be just
crazy and foolish to not have open userspace code as well.  Firmware
would also be wonderful as well, go poke your lawyers about derivative
work issues and the like for fun conversations :)

So without that changed, I'm not going to take this, and push to object
that anyone else take this.

I'm not going to be able to review any of this code anymore until that
changes, sorry.

thanks,

greg k-h

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Greg Kroah-Hartman]

On Tue, May 19, 2020 at 03:08:42PM +1000, Dave Airlie wrote:
> On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
> >
> > Introduction:
> > Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> > SoC ASIC for the purpose of efficently running Deep Learning inference
> > workloads in a data center environment.
> >
> > The offical press release can be found at -
> > https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> >
> > The offical product website is -
> > https://www.qualcomm.com/products/datacenter-artificial-intelligence
> >
> > At the time of the offical press release, numerious technology news sites
> > also covered the product.  Doing a search of your favorite site is likely
> > to find their coverage of it.
> >
> > It is our goal to have the kernel driver for the product fully upstream.
> > The purpose of this RFC is to start that process.  We are still doing
> > development (see below), and thus not quite looking to gain acceptance quite
> > yet, but now that we have a working driver we beleive we are at the stage
> > where meaningful conversation with the community can occur.
> 
> 
> Hi Jeffery,
> 
> Just wondering what the userspace/testing plans for this driver.
> 
> This introduces a new user facing API for a device without pointers to
> users or tests for that API.
> 
> Although this isn't a graphics driver, and Greg will likely merge
> anything to the kernel you throw at him, I do wonder how to validate
> the uapi from a security perspective. It's always interesting when
> someone wraps a DMA engine with user ioctls, and without enough
> information to decide if the DMA engine is secure against userspace
> misprogramming it.

Hey, I'll not merge just anything!

Oh, well, maybe, if it's in staging :)

> Also if we don't understand the programming API on board the device,
> we can't tell if the "core" on the device are able to reprogram the
> device engines either.
> 
> Figuring this out is difficult at the best of times, it helps if there
> is access to the complete device documentation or user space side
> drivers in order to faciliate this.
> 
> The other area I mention is testing the uAPI, how do you envisage
> regression testing and long term sustainability of the uAPI?

I agree with this request, we should have some code that we can run in
order to test that things work properly.

thanks,

greg k-h

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Jeffrey Hugo]

On 5/18/2020 11:08 PM, Dave Airlie wrote:
> On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>>
>> Introduction:
>> Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
>> SoC ASIC for the purpose of efficently running Deep Learning inference
>> workloads in a data center environment.
>>
>> The offical press release can be found at -
>> https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
>>
>> The offical product website is -
>> https://www.qualcomm.com/products/datacenter-artificial-intelligence
>>
>> At the time of the offical press release, numerious technology news sites
>> also covered the product.  Doing a search of your favorite site is likely
>> to find their coverage of it.
>>
>> It is our goal to have the kernel driver for the product fully upstream.
>> The purpose of this RFC is to start that process.  We are still doing
>> development (see below), and thus not quite looking to gain acceptance quite
>> yet, but now that we have a working driver we beleive we are at the stage
>> where meaningful conversation with the community can occur.
> 
> 
> Hi Jeffery,
> 
> Just wondering what the userspace/testing plans for this driver.
> 
> This introduces a new user facing API for a device without pointers to
> users or tests for that API.

We have daily internal testing, although I don't expect you to take my 
word for that.

I would like to get one of these devices into the hands of Linaro, so 
that it can be put into KernelCI.  Similar to other Qualcomm products. 
I'm trying to convince the powers that be to make this happen.

Regarding what the community could do on its own, everything but the 
Linux driver is considered proprietary - that includes the on device 
firmware and the entire userspace stack.  This is a decision above my 
pay grade.

I've asked for authorization to develop and publish a simple userspace 
application that might enable the community to do such testing, but 
obtaining that authorization has been slow.

> Although this isn't a graphics driver, and Greg will likely merge
> anything to the kernel you throw at him, I do wonder how to validate
> the uapi from a security perspective. It's always interesting when
> someone wraps a DMA engine with user ioctls, and without enough
> information to decide if the DMA engine is secure against userspace
> misprogramming it.

I'm curious, what information might you be looking for?  Are you 
concerned about the device attacking the host, or the host attacking the 
device?

> Also if we don't understand the programming API on board the device,
> we can't tell if the "core" on the device are able to reprogram the
> device engines either.

So, you are looking for details about the messaging protocol which are 
considered opaque to the kernel driver?  Or something else?

> Figuring this out is difficult at the best of times, it helps if there
> is access to the complete device documentation or user space side
> drivers in order to faciliate this.

Regarding access to documentation, sadly that isn't going to happen now, 
or in the near future.  Again, above my pay grade.  The only public 
"documentation" is what you can see from my emails.

I understand your position, and if I can "bound" the information you are 
looking for, I can see what I can do about getting you what you want. 
No promises, but I will try.

> The other area I mention is testing the uAPI, how do you envisage
> regression testing and long term sustainability of the uAPI?

Can you clarify what you mean by "uAPI"?  Are you referring to the 
interface between the device and the kernel driver?

-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Jeffrey Hugo]

On 5/19/2020 12:57 AM, Manivannan Sadhasivam wrote:
> Hi Jeff,
> 
> On Thu, May 14, 2020 at 08:07:38AM -0600, Jeffrey Hugo wrote:
>> Introduction:
>> Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
>> SoC ASIC for the purpose of efficently running Deep Learning inference
>> workloads in a data center environment.
>>
>> The offical press release can be found at -
>> https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
>>
>> The offical product website is -
>> https://www.qualcomm.com/products/datacenter-artificial-intelligence
>>
>> At the time of the offical press release, numerious technology news sites
>> also covered the product.  Doing a search of your favorite site is likely
>> to find their coverage of it.
>>
>> It is our goal to have the kernel driver for the product fully upstream.
>> The purpose of this RFC is to start that process.  We are still doing
>> development (see below), and thus not quite looking to gain acceptance quite
>> yet, but now that we have a working driver we beleive we are at the stage
>> where meaningful conversation with the community can occur.
>>
>> Design:
> 
> Can you add documentation in next revision with all this information (or more)?
> In restructured text ofc. Eventhough it is an RFC series, adding documentation
> doesn't hurt and it will help reviewers to understand the hardware better.

Sorry, saw this hit my inbox as I was sending out the next rev.  There 
will be another rev.

Sure.  I'm open to doing that.  Hmm, Documentation/misc-devices seem good?

Do you have specific additional information you think would be good?

-- 
Jeffrey Hugo
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Manivannan Sadhasivam]

Hi Jeff,

On Thu, May 14, 2020 at 08:07:38AM -0600, Jeffrey Hugo wrote:
> Introduction:
> Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> SoC ASIC for the purpose of efficently running Deep Learning inference
> workloads in a data center environment.
> 
> The offical press release can be found at -
> https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
> 
> The offical product website is -
> https://www.qualcomm.com/products/datacenter-artificial-intelligence
> 
> At the time of the offical press release, numerious technology news sites
> also covered the product.  Doing a search of your favorite site is likely
> to find their coverage of it.
> 
> It is our goal to have the kernel driver for the product fully upstream.
> The purpose of this RFC is to start that process.  We are still doing
> development (see below), and thus not quite looking to gain acceptance quite
> yet, but now that we have a working driver we beleive we are at the stage
> where meaningful conversation with the community can occur.
> 
> Design:

Can you add documentation in next revision with all this information (or more)?
In restructured text ofc. Eventhough it is an RFC series, adding documentation
doesn't hurt and it will help reviewers to understand the hardware better.

Thanks,
Mani

> 
> +--------------------------------+
> |       AI application           |
> |       (userspace)              |
> +-------------+------------------+
>               |
>               | Char dev interface
>               |
>               |
> +-------------+------------------+
> |       QAIC driver              |
> |       (kernel space)           |
> |                                |
> +----+------------------+--------+
>      |                  |
>      |                  |
>      |                  |
>      |                  |
>      |Control path      | Data path
>      |(MHI bus)         |
>      |                  |
>      |                  |
>      |                  |
>      |                  |
> +--------------------------------+
> | +--------+      +------------+ |
> | | MHI HW |      |DMA Bridge  | |
> | +--------+      |(DMA engine)| |
> |                 +------------+ |
> |                                |
> |                                |
> |                                |
> |  Qualcomm Cloud AI 100 device  |
> |                                |
> |                                |
> +--------------------------------+
> 
> A Qualcomm Cloud AI 100 device (QAIC device from here on) is a PCIe hardware
> accelerator for AI inference workloads.  Over the PCIe bus fabric, a QAIC
> device exposes two interfaces via PCI BARs - a MHI hardware region and a
> DMA engine hardware region.
> 
> Before workloads can be run, a QAIC device needs to be initialized.  Similar
> to other Qualcomm products with incorperate MHI, device firmware needs to be
> loaded onto the device from the host.  This occurs in two stages.  First,
> a secondary bootloader (SBL) needs to be loaded onto the device.  This occurs
> via the BHI protocol, and is handled by the MHI bus.  Once the SBL loaded
> and running, it activates the Sahara protocol.  The Sahara protocol is used
> with a userspace application to load and initialize the remaining firmware.
> The Sahara protocol and associated userspace application are outside the
> scope of this series as they have no direct interaction with the QAIC driver.
> 
> Once a QAIC device is fully initialized, workloads can be sent to the device
> and run.  This involves a per-device instance char dev that the QAIC driver
> exposes to userspace.  Running a workload involves two phases - configuring the
> device, and interacting with the workload.
> 
> To configure the device, commands are sent via a MHI channel.  This is referred
> to as the control path.  A command is a single message.  A message contains
> one or more transactions.  Transactions are operations that the device
> is requested to perform.  Most commands are opaque to the kernel driver, however
> some are not.  For example, if the user application wishes to DMA data to the
> device, it requires the assistance of the kernel driver to translate the data
> addresses to an address space that the device can understand.  In this instance
> the transaction for DMAing the data is visible to the kernel driver, and the
> driver will do the required transformation when encoding the message.
> 
> To interact with the workload, the workload is assigned a DMA Bridge Channel
> (dbc).  This is dedicated hardware within the DMA engine.  Interacting with the
> workload consists of sending it input data, and receiving output data.  The
> user application requests appropiate buffers from the kernel driver, prepares
> the buffers, and directs the kernel driver to queue them to the hardware.
> 
> The kernel driver is required to support multiple QAIC devices, and also N
> users per device.
> 
> Status:
> This series introduces the driver for QAIC devices, and builds up the minimum
> functionality for running workloads.  Several features which have been omitted
> or are still planned are indicated in the future work section.
> 
> Before exiting the RFC phase, and attempting full acceptance, we wish to
> complete two features which are currently under development as we expect there
> to be userspace interface changes as a result.
> 
> The first feature is a variable length control message between the kernel driver
> and the device.  This allows us to support the total number of DMA transactions
> we require for certain platforms, while minimizing memory usage.  The interface
> impact of this would be to allow us to drop the size of the manage buffer
> between userspace and the kernel driver from the current 16k, much of which is
> wasted.
> 
> The second feature is an optimization and extension of the data path interface.
> We plan to move the bulk of the data in the qaic_execute structure to the
> qaic_mem_req structure, which optimized our critical path processing.  We also
> plan to extend the qaic_execute structure to allow for a batch submit of
> multiple buffers as an optimization and convenience for userspace.  
> 
> Future work:
> For simplicity, we have omitted work related to the following features, and
> intend to submit in future series:
> 
> -debugfs
> -trace points
> -hwmon (device telemetry)
> 
> We are also investigating what it might mean to support dma_bufs.  We expect
> that such support would come as an extension of the interface.
> 
> Jeffrey Hugo (8):
>   qaic: Add skeleton driver
>   qaic: Add and init a basic mhi controller
>   qaic: Create char dev
>   qaic: Implement control path
>   qaic: Implement data path
>   qaic: Implement PCI link status error handlers
>   qaic: Implement MHI error status handler
>   MAINTAINERS: Add entry for QAIC driver
> 
>  MAINTAINERS                        |    7 +
>  drivers/misc/Kconfig               |    1 +
>  drivers/misc/Makefile              |    1 +
>  drivers/misc/qaic/Kconfig          |   20 +
>  drivers/misc/qaic/Makefile         |   12 +
>  drivers/misc/qaic/mhi_controller.c |  538 +++++++++++++++++++
>  drivers/misc/qaic/mhi_controller.h |   20 +
>  drivers/misc/qaic/qaic.h           |  111 ++++
>  drivers/misc/qaic/qaic_control.c   | 1015 ++++++++++++++++++++++++++++++++++++
>  drivers/misc/qaic/qaic_data.c      |  952 +++++++++++++++++++++++++++++++++
>  drivers/misc/qaic/qaic_drv.c       |  699 +++++++++++++++++++++++++
>  include/uapi/misc/qaic.h           |  246 +++++++++
>  12 files changed, 3622 insertions(+)
>  create mode 100644 drivers/misc/qaic/Kconfig
>  create mode 100644 drivers/misc/qaic/Makefile
>  create mode 100644 drivers/misc/qaic/mhi_controller.c
>  create mode 100644 drivers/misc/qaic/mhi_controller.h
>  create mode 100644 drivers/misc/qaic/qaic.h
>  create mode 100644 drivers/misc/qaic/qaic_control.c
>  create mode 100644 drivers/misc/qaic/qaic_data.c
>  create mode 100644 drivers/misc/qaic/qaic_drv.c
>  create mode 100644 include/uapi/misc/qaic.h
> 
> -- 
> Qualcomm Technologies, Inc. is a member of the
> Code Aurora Forum, a Linux Foundation Collaborative Project.

Re: [RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Dave Airlie]

On Fri, 15 May 2020 at 00:12, Jeffrey Hugo <jhugo@codeaurora.org> wrote:
>
> Introduction:
> Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
> SoC ASIC for the purpose of efficently running Deep Learning inference
> workloads in a data center environment.
>
> The offical press release can be found at -
> https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference
>
> The offical product website is -
> https://www.qualcomm.com/products/datacenter-artificial-intelligence
>
> At the time of the offical press release, numerious technology news sites
> also covered the product.  Doing a search of your favorite site is likely
> to find their coverage of it.
>
> It is our goal to have the kernel driver for the product fully upstream.
> The purpose of this RFC is to start that process.  We are still doing
> development (see below), and thus not quite looking to gain acceptance quite
> yet, but now that we have a working driver we beleive we are at the stage
> where meaningful conversation with the community can occur.


Hi Jeffery,

Just wondering what the userspace/testing plans for this driver.

This introduces a new user facing API for a device without pointers to
users or tests for that API.

Although this isn't a graphics driver, and Greg will likely merge
anything to the kernel you throw at him, I do wonder how to validate
the uapi from a security perspective. It's always interesting when
someone wraps a DMA engine with user ioctls, and without enough
information to decide if the DMA engine is secure against userspace
misprogramming it.

Also if we don't understand the programming API on board the device,
we can't tell if the "core" on the device are able to reprogram the
device engines either.

Figuring this out is difficult at the best of times, it helps if there
is access to the complete device documentation or user space side
drivers in order to faciliate this.

The other area I mention is testing the uAPI, how do you envisage
regression testing and long term sustainability of the uAPI?

Thanks,
Dave.

[RFC PATCH 0/8] Qualcomm Cloud AI 100 driver

[Jeffrey Hugo]

Introduction:
Qualcomm Cloud AI 100 is a PCIe adapter card which contains a dedicated
SoC ASIC for the purpose of efficently running Deep Learning inference
workloads in a data center environment.

The offical press release can be found at -
https://www.qualcomm.com/news/releases/2019/04/09/qualcomm-brings-power-efficient-artificial-intelligence-inference

The offical product website is -
https://www.qualcomm.com/products/datacenter-artificial-intelligence

At the time of the offical press release, numerious technology news sites
also covered the product.  Doing a search of your favorite site is likely
to find their coverage of it.

It is our goal to have the kernel driver for the product fully upstream.
The purpose of this RFC is to start that process.  We are still doing
development (see below), and thus not quite looking to gain acceptance quite
yet, but now that we have a working driver we beleive we are at the stage
where meaningful conversation with the community can occur.

Design:

+--------------------------------+
|       AI application           |
|       (userspace)              |
+-------------+------------------+
              |
              | Char dev interface
              |
              |
+-------------+------------------+
|       QAIC driver              |
|       (kernel space)           |
|                                |
+----+------------------+--------+
     |                  |
     |                  |
     |                  |
     |                  |
     |Control path      | Data path
     |(MHI bus)         |
     |                  |
     |                  |
     |                  |
     |                  |
+--------------------------------+
| +--------+      +------------+ |
| | MHI HW |      |DMA Bridge  | |
| +--------+      |(DMA engine)| |
|                 +------------+ |
|                                |
|                                |
|                                |
|  Qualcomm Cloud AI 100 device  |
|                                |
|                                |
+--------------------------------+

A Qualcomm Cloud AI 100 device (QAIC device from here on) is a PCIe hardware
accelerator for AI inference workloads.  Over the PCIe bus fabric, a QAIC
device exposes two interfaces via PCI BARs - a MHI hardware region and a
DMA engine hardware region.

Before workloads can be run, a QAIC device needs to be initialized.  Similar
to other Qualcomm products with incorperate MHI, device firmware needs to be
loaded onto the device from the host.  This occurs in two stages.  First,
a secondary bootloader (SBL) needs to be loaded onto the device.  This occurs
via the BHI protocol, and is handled by the MHI bus.  Once the SBL loaded
and running, it activates the Sahara protocol.  The Sahara protocol is used
with a userspace application to load and initialize the remaining firmware.
The Sahara protocol and associated userspace application are outside the
scope of this series as they have no direct interaction with the QAIC driver.

Once a QAIC device is fully initialized, workloads can be sent to the device
and run.  This involves a per-device instance char dev that the QAIC driver
exposes to userspace.  Running a workload involves two phases - configuring the
device, and interacting with the workload.

To configure the device, commands are sent via a MHI channel.  This is referred
to as the control path.  A command is a single message.  A message contains
one or more transactions.  Transactions are operations that the device
is requested to perform.  Most commands are opaque to the kernel driver, however
some are not.  For example, if the user application wishes to DMA data to the
device, it requires the assistance of the kernel driver to translate the data
addresses to an address space that the device can understand.  In this instance
the transaction for DMAing the data is visible to the kernel driver, and the
driver will do the required transformation when encoding the message.

To interact with the workload, the workload is assigned a DMA Bridge Channel
(dbc).  This is dedicated hardware within the DMA engine.  Interacting with the
workload consists of sending it input data, and receiving output data.  The
user application requests appropiate buffers from the kernel driver, prepares
the buffers, and directs the kernel driver to queue them to the hardware.

The kernel driver is required to support multiple QAIC devices, and also N
users per device.

Status:
This series introduces the driver for QAIC devices, and builds up the minimum
functionality for running workloads.  Several features which have been omitted
or are still planned are indicated in the future work section.

Before exiting the RFC phase, and attempting full acceptance, we wish to
complete two features which are currently under development as we expect there
to be userspace interface changes as a result.

The first feature is a variable length control message between the kernel driver
and the device.  This allows us to support the total number of DMA transactions
we require for certain platforms, while minimizing memory usage.  The interface
impact of this would be to allow us to drop the size of the manage buffer
between userspace and the kernel driver from the current 16k, much of which is
wasted.

The second feature is an optimization and extension of the data path interface.
We plan to move the bulk of the data in the qaic_execute structure to the
qaic_mem_req structure, which optimized our critical path processing.  We also
plan to extend the qaic_execute structure to allow for a batch submit of
multiple buffers as an optimization and convenience for userspace.  

Future work:
For simplicity, we have omitted work related to the following features, and
intend to submit in future series:

-debugfs
-trace points
-hwmon (device telemetry)

We are also investigating what it might mean to support dma_bufs.  We expect
that such support would come as an extension of the interface.

Jeffrey Hugo (8):
  qaic: Add skeleton driver
  qaic: Add and init a basic mhi controller
  qaic: Create char dev
  qaic: Implement control path
  qaic: Implement data path
  qaic: Implement PCI link status error handlers
  qaic: Implement MHI error status handler
  MAINTAINERS: Add entry for QAIC driver

 MAINTAINERS                        |    7 +
 drivers/misc/Kconfig               |    1 +
 drivers/misc/Makefile              |    1 +
 drivers/misc/qaic/Kconfig          |   20 +
 drivers/misc/qaic/Makefile         |   12 +
 drivers/misc/qaic/mhi_controller.c |  538 +++++++++++++++++++
 drivers/misc/qaic/mhi_controller.h |   20 +
 drivers/misc/qaic/qaic.h           |  111 ++++
 drivers/misc/qaic/qaic_control.c   | 1015 ++++++++++++++++++++++++++++++++++++
 drivers/misc/qaic/qaic_data.c      |  952 +++++++++++++++++++++++++++++++++
 drivers/misc/qaic/qaic_drv.c       |  699 +++++++++++++++++++++++++
 include/uapi/misc/qaic.h           |  246 +++++++++
 12 files changed, 3622 insertions(+)
 create mode 100644 drivers/misc/qaic/Kconfig
 create mode 100644 drivers/misc/qaic/Makefile
 create mode 100644 drivers/misc/qaic/mhi_controller.c
 create mode 100644 drivers/misc/qaic/mhi_controller.h
 create mode 100644 drivers/misc/qaic/qaic.h
 create mode 100644 drivers/misc/qaic/qaic_control.c
 create mode 100644 drivers/misc/qaic/qaic_data.c
 create mode 100644 drivers/misc/qaic/qaic_drv.c
 create mode 100644 include/uapi/misc/qaic.h

-- 
Qualcomm Technologies, Inc. is a member of the
Code Aurora Forum, a Linux Foundation Collaborative Project.