[RFD] Proposal for merging Android sync driver in staging

[RFD] Proposal for merging Android sync driver in staging

[John Stultz]

I'd like to get a discussion going about submitting the Android sync 
driver to staging.

I know there is currently some very similar work going on with the 
dmabuf-fences, and rather then both approaches being worked out 
individually on their own, I suspect there could be better collaboration 
around this effort.

So my proposal is that we merge the Android sync driver into staging.

In my mind, this has the following benefits:
1) It allows other drivers that depend on the sync interface to also be 
submitted to staging, rather then forcing those drivers to be hidden 
away in various out of tree git repos, location unknown.

2) It would provide a baseline view to the upstream community of the 
interface Android is using, providing  a real-world, active use case of 
the functionality.

Once the sync driver is in staging, if the dmabuf-fences work is fully 
sufficient to replace the Android sync driver, we should be able to 
whittle down the sync driver until its just a interface shim (and at 
which point efforts can be made to convert Android userland over to 
dmabuf-fences).

However, if the dmabuf-fences work is not fully sufficient to replace 
the android sync driver, we should be able to at least to whittle down 
the driver to those specific differences, which would provide a concrete 
example of where the dmabuf-fences, or other work may need to be 
expanded, or if maybe the sync driver is the better approach.

I've gone through the Android tree and reworked the sync driver to live 
in staging, while still preserving the full patch history/authorship. 
You can checkout the reworked patch queue here:
  http://git.linaro.org/gitweb?p=people/jstultz/android-dev.git;a=shortlog;h=refs/heads/dev/sync-staging

If folks would take a look and let me know what they think of the 
changes as well as what they think about pushing it to staging, or other 
ideas for how to improve collaboration so we can have common interfaces 
here, I'd appreciate it.

Also note: I've done this so far without any feedback from the Android 
devs (despite my reaching out to Erik a few times recently), so if they 
object to pushing it to staging, in deference to it being their code 
I'll back off, even though I do think it would be good to have the code 
get more visibility upstream in staging. I don't mean to step on 
anyone's toes. :)

thanks
-john

Re: [RFD] Proposal for merging Android sync driver in staging

[Greg KH]

On Wed, Feb 27, 2013 at 06:14:24PM -0800, John Stultz wrote:
> I'd like to get a discussion going about submitting the Android sync
> driver to staging.
> 
> I know there is currently some very similar work going on with the
> dmabuf-fences, and rather then both approaches being worked out
> individually on their own, I suspect there could be better
> collaboration around this effort.
> 
> So my proposal is that we merge the Android sync driver into staging.
> 
> In my mind, this has the following benefits:
> 1) It allows other drivers that depend on the sync interface to also
> be submitted to staging, rather then forcing those drivers to be
> hidden away in various out of tree git repos, location unknown.
> 
> 2) It would provide a baseline view to the upstream community of the
> interface Android is using, providing  a real-world, active use case
> of the functionality.
> 
> Once the sync driver is in staging, if the dmabuf-fences work is
> fully sufficient to replace the Android sync driver, we should be
> able to whittle down the sync driver until its just a interface shim
> (and at which point efforts can be made to convert Android userland
> over to dmabuf-fences).

Sounds like a good plan to me.

> I've gone through the Android tree and reworked the sync driver to
> live in staging, while still preserving the full patch
> history/authorship. You can checkout the reworked patch queue here:
>  http://git.linaro.org/gitweb?p=people/jstultz/android-dev.git;a=shortlog;h=refs/heads/dev/sync-staging

I can't really look at a git tree at the moment, but will always be glad
to review patches.  Feel free to send them on and we can look at them
then :)

thanks,

greg k-h

Re: [RFD] Proposal for merging Android sync driver in staging

[John Stultz]

On 02/27/2013 06:32 PM, Greg KH wrote:
> On Wed, Feb 27, 2013 at 06:14:24PM -0800, John Stultz wrote:
>> I'd like to get a discussion going about submitting the Android sync
>> driver to staging.
>>
>> I know there is currently some very similar work going on with the
>> dmabuf-fences, and rather then both approaches being worked out
>> individually on their own, I suspect there could be better
>> collaboration around this effort.
>>
>> So my proposal is that we merge the Android sync driver into staging.
>>
>> In my mind, this has the following benefits:
>> 1) It allows other drivers that depend on the sync interface to also
>> be submitted to staging, rather then forcing those drivers to be
>> hidden away in various out of tree git repos, location unknown.
>>
>> 2) It would provide a baseline view to the upstream community of the
>> interface Android is using, providing  a real-world, active use case
>> of the functionality.
>>
>> Once the sync driver is in staging, if the dmabuf-fences work is
>> fully sufficient to replace the Android sync driver, we should be
>> able to whittle down the sync driver until its just a interface shim
>> (and at which point efforts can be made to convert Android userland
>> over to dmabuf-fences).
> Sounds like a good plan to me.
>
>> I've gone through the Android tree and reworked the sync driver to
>> live in staging, while still preserving the full patch
>> history/authorship. You can checkout the reworked patch queue here:
>>   http://git.linaro.org/gitweb?p=people/jstultz/android-dev.git;a=shortlog;h=refs/heads/dev/sync-staging
> I can't really look at a git tree at the moment, but will always be glad
> to review patches.  Feel free to send them on and we can look at them
> then :)

Ok, since I preserved the patch history, its currently 30 patches, and I 
didn't want to flood everyone's inboxes with patches (Greg: I know you'd 
never do such a thing! :) before making sure there weren't any 
objections to the idea in concept.

I'll send out the stack later today.

thanks
-john

Re: [RFD] Proposal for merging Android sync driver in staging

[Erik Gilling]

On Wed, Feb 27, 2013 at 6:14 PM, John Stultz <john.stultz@linaro.org> wrote:
> Also note: I've done this so far without any feedback from the Android devs
> (despite my reaching out to Erik a few times recently), so if they object to
> pushing it to staging, in deference to it being their code I'll back off,
> even though I do think it would be good to have the code get more visibility
> upstream in staging. I don't mean to step on anyone's toes. :)

Yeah, sorry about that.  I kept meaning to get back to you but kept
getting distracted.  A little background on the patches:

In Honeycomb where we introduced the Hardware Composer HAL.  This is a
userspace layer that allows composition acceleration on a per platform
basis.  Different SoC vendors have implemented this using overlays, 2d
blitters, a combinations of both, or other clever/disgusting means.
Along with the HWC we consolidated a lot of our camera and media
pipeline to allow their input to be fed into the GPU or
display(overlay.)  In order to exploit parallelism the the graphics
pipeline, this introduced lots of implicit synchronization
dependancies.  After a couple years of working with many different SoC
vendors, we found that it was really difficult to communicate our
system's expectations of the implicit contract and it was difficult
for the SoC vendors to properly implement the implicit contract in
each of their IP blocks (display, gpu, camera, video codecs).  It was
also incredibly difficult to debug when problems/deadlocks arose.

In an effort to clean up the situation we decided to create set of
simple synchronization primitives and have our compositor
(SurfaceFlinger) manage the synchronization contract explicitly.  We
designed these primitives so that they can be passed across processes
(much like ion/dma_buf handles), can be backed by hardware
synchronization primitives, and can be combined with other sync
dependancies in a heterogeneous manner.  We also added enough
debugging information to make pinpointing a synchronization deadlock
bug easier.  There are also OpenGL extensions added (which I believe
have been ratified by Khronos) to convert a "native" sync object to a
gl fence object and vise versa.

So far shipped this system on two products (the Nexus 10 and 4) with
two different SoCs (Samsung Exynos5250 and Qualcomm MSM8064.)  These
two projects were much easier to work out the kinks in the
graphics/compositing pipelines.  In addition we were able to use the
telemetry and tracing features to track down the causes of dropped
frames aka "jank."

As for the implementation, I started with having the main driver op
primitive be a wait() op.  I quickly noticed that most of the tricky
race condition prone code was ending up in the drivers wait() op.  It
also made handling asynchronous waits of more than one type of sync_pt
difficult to manage.  In the end I opted for something roughly like
poll() where all the heavy lifting is done at the high level and the
drivers only need to implement a simple check function.

Happy to hear feedback and (especially) bug reports/fixes.

Cheers,
    Erik