* Chromium acceleration @ 2014-03-19 20:49 Eric Nelson 2014-03-19 21:00 ` Carlos Rafael Giani 2014-03-20 12:47 ` Lauren Post 0 siblings, 2 replies; 37+ messages in thread From: Eric Nelson @ 2014-03-19 20:49 UTC (permalink / raw) To: Post Lauren-RAA013, meta-freescale Hi Lauren, A while back, you mentioned that 3.10.17-ga would help provide support for "Chromium acceleration" but didn't provide details. Did you mean WebGL support (i.e. GPU acceleration), or video acceleration (VPU support)? My understanding is that Chromium is using ffmpeg-mt as the basis for accelerated video: http://www.chromium.org/developers/design-documents/video Is anyone currently working on support for ffmpeg-mt and i.MX video? Is anyone interested? Please advise, Eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-19 20:49 Chromium acceleration Eric Nelson @ 2014-03-19 21:00 ` Carlos Rafael Giani 2014-03-19 22:40 ` Eric Nelson 2014-03-20 12:47 ` Lauren Post 1 sibling, 1 reply; 37+ messages in thread From: Carlos Rafael Giani @ 2014-03-19 21:00 UTC (permalink / raw) To: meta-freescale Hello, I wrote a patch for VPU acceleration in Chromium months ago for a company project. I got the OK from the customer to cleanup and eventually opensource the patches, but didn't have the time yet. Together with other colleagues we also got HW-accelerated WebGL to work, though some other areas weren't running so well (multitouch for example). Also, Canvas remained unaccelerated. As the Chromium developers explained, there are conceptual problems with the Canvas API that make acceleration difficult. What is also missing is a zerocopy method for displaying video frames. In theory, the direct textures from the Vivante GPU could be used; however, this requires passing the physical buffers through to a modified renderer somehow. Chromium can use OpenGL ES for rendering everything, but does not contain anything to introduce special video textures yet. Carlos On 03/19/2014 09:49 PM, Eric Nelson wrote: > Hi Lauren, > > A while back, you mentioned that 3.10.17-ga would help provide > support for "Chromium acceleration" but didn't provide details. > > Did you mean WebGL support (i.e. GPU acceleration), or video > acceleration (VPU support)? > > My understanding is that Chromium is using ffmpeg-mt as the > basis for accelerated video: > http://www.chromium.org/developers/design-documents/video > > Is anyone currently working on support for ffmpeg-mt and i.MX > video? > > Is anyone interested? > > Please advise, > > > Eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-19 21:00 ` Carlos Rafael Giani @ 2014-03-19 22:40 ` Eric Nelson 2014-03-20 1:29 ` Christian Betz 0 siblings, 1 reply; 37+ messages in thread From: Eric Nelson @ 2014-03-19 22:40 UTC (permalink / raw) To: Carlos Rafael Giani, meta-freescale Hi Carlos, On 03/19/2014 02:00 PM, Carlos Rafael Giani wrote: > Hello, > > I wrote a patch for VPU acceleration in Chromium months ago for a > company project. I got the OK from the customer to cleanup and > eventually opensource the patches, but didn't have the time yet. You're the man! What can we do to help find you the time? > Together with other colleagues we also got HW-accelerated WebGL to work, > though some other areas weren't running so well (multitouch for > example). Also, Canvas remained unaccelerated. As the Chromium > developers explained, there are conceptual problems with the Canvas API > that make acceleration difficult. > Nice! I have to admit being a bit less interested in WebGL than in video acceleration, but I'm sure others really want this. > > What is also missing is a zerocopy method for displaying video frames. > Do we really want zero copy? It seems that at some level, having a single copy into a GPU accelerated rendering stack is more convenient. > In theory, the direct textures from the Vivante GPU could be used; > however, this requires passing the physical buffers through to a > modified renderer somehow. > > Chromium can use OpenGL ES for rendering everything, but does not > contain anything to introduce special video textures yet. > I'm not quite sure I'm grokking this. The Vivante libraries can map the physical buffers produced by the VPU directly, so they could do format conversion on their way to the graphics stack if the Chromium bindings have access to that (the "single copy" I referred to above). Or, (he says instead of finding the code), is this handled by ffmpeg-mt? Using an IPU overlay (a.la. mfw_v4lsink) is nice, but makes doing overlays and such more difficult, and you probably know how web-developers can be ;). Regards, Eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-19 22:40 ` Eric Nelson @ 2014-03-20 1:29 ` Christian Betz 2014-03-20 2:50 ` Eric Nelson 0 siblings, 1 reply; 37+ messages in thread From: Christian Betz @ 2014-03-20 1:29 UTC (permalink / raw) To: Eric Nelson; +Cc: meta-freescale [-- Attachment #1: Type: text/plain, Size: 2199 bytes --] > > The Vivante libraries can map the physical buffers produced by > the VPU directly, so they could do format conversion on their > way to the graphics stack if the Chromium bindings have access > to that (the "single copy" I referred to above). > in my experience "zero-copy" really means zero memcpy() (or similar system call, or really anything that involves the cpu to move around image data. *metadata *such as buffer pointers is generally allowed). in other words having the GPU do format conversion isn't considered a copy. here's a great article on this general technique implemented in webkit/gtk+: http://blogs.igalia.com/vjaquez/2013/07/26/composited-video-support-in-webkitgtk/ moving beyond semantics... let me tell you what i know about this particular issue: there is source code in chromium to accomplish a similar goal. specifically there is a "generic" android driver, which works out of the box with i.mx6 because freescale implements the standard android video decode interface that allows VPU -> GPU texture uploading (don't quote me on this, i'm just reading the source code: https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/android_video_decode_accelerator.cc ) sadly there is no such standard interface on GNU/Linux. outside of android (i.e. chromeos), there are drivers for particular chipsets, like samsung exynos. here is the source for that: https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/exynos_video_decode_accelerator.cc the group should also be aware of the current situation regarding these modules *only* working when certain flags to enable "chromeos" are used (explained here: https://code.google.com/p/chromium/wiki/LinuxHWVideoDecode). in other words: this stuff doesn't work on your x86 with the chrome you download from google. @carlos: have you implemented a module along the lines of this exynos accelerator or did you take another route? a team member is actually about to attack this problem next week so it'd be great to see what you have before we reinvent your wheel. [-- Attachment #2: Type: text/html, Size: 3243 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 1:29 ` Christian Betz @ 2014-03-20 2:50 ` Eric Nelson 2014-03-20 8:30 ` Carlos Rafael Giani 0 siblings, 1 reply; 37+ messages in thread From: Eric Nelson @ 2014-03-20 2:50 UTC (permalink / raw) To: Christian Betz; +Cc: meta-freescale Thanks Christian, On 03/19/2014 06:29 PM, Christian Betz wrote: > The Vivante libraries can map the physical buffers produced by > the VPU directly, so they could do format conversion on their > way to the graphics stack if the Chromium bindings have access > to that (the "single copy" I referred to above). > > > in my experience "zero-copy" really means zero memcpy() (or similar > system call, or really anything that involves the cpu to move around > image data. /metadata /such as buffer pointers is generally allowed). in > other words having the GPU do format conversion isn't considered a copy. > > here's a great article on this general technique implemented in > webkit/gtk+: > http://blogs.igalia.com/vjaquez/2013/07/26/composited-video-support-in-webkitgtk/ > That's a nice article, and if Webkit (Blink) retains the same mapping, video buffers from the VPU would end up hitting a RenderObject and the only "copy" would be in rendering that to the Framebuffer. Since we're on the fat end of the pipe here, that copy can be expensive and might limit things to 1x1080P decode, but I'd be happy with that. In the mfw_v4lsink case at least, this can be bypassed by having the V4L2 buffer filled by the VPU and also displayed by the IPU (such that copy), but aiming there would keep the video out of the rest of the RenderLayer tree (I think). This is what I was so awkwardly trying to suggest. > moving beyond semantics... let me tell you what i know about this > particular issue: > > there is source code in chromium to accomplish a similar goal. > specifically there is a "generic" android driver, which works out of the > box with i.mx6 because freescale implements the standard android video > decode interface that allows VPU -> GPU texture uploading (don't quote > me on this, i'm just reading the source code: > https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/android_video_decode_accelerator.cc) > > sadly there is no such standard interface on GNU/Linux. outside of > android (i.e. chromeos), there are drivers for particular chipsets, like > samsung exynos. here is the source for that: > https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/exynos_video_decode_accelerator.cc > > the group should also be aware of the current situation regarding these > modules /only/ working when certain flags to enable "chromeos" are used > (explained here: > https://code.google.com/p/chromium/wiki/LinuxHWVideoDecode). in other > words: this stuff doesn't work on your x86 with the chrome you download > from google. > Again, thanks for the info. Regards, Eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 2:50 ` Eric Nelson @ 2014-03-20 8:30 ` Carlos Rafael Giani 2014-03-20 12:22 ` Otavio Salvador 2014-03-20 13:46 ` Christian Betz 0 siblings, 2 replies; 37+ messages in thread From: Carlos Rafael Giani @ 2014-03-20 8:30 UTC (permalink / raw) To: Eric Nelson, Christian Betz; +Cc: meta-freescale Hi Eric and Christian, I wanted to avoid _any_ copy involving the CPU, including any texture upload. The Vivante GLES direct texture extension can color-convert, yes, but it can also read pixels directly from a DMA buffer, without any copy. This is what I use for the GLES-based sink in my GStreamer plugins, and allows me to play 1080p videos with less than 6% CPU usage. v4lsink would be another idea, but I am a bit skeptical, because it wouldn't play well with any X11 or Wayland compositor, since it would essentially act like an overlay. This is even worse with the isink (if the regular, non-overlay layer is used), because then, the sink simply overwrites pixels drawn by X11. What the isink does is to make sure the pixels are blit in the area of its output window, so it does not overwrite anything outside. But again, with desktop compositing this would break down. (Also, the isink is quite slow.) I would consider this one texture upload only if absolutely necessary. My idea was to actually produce the texture in the decoder myself, and just send the texture GL name around to the renderer. This would require that everything is drawn with GLES of course - what would a software renderer do with a GL name? As for the decoder itself: I implemented it in the Chromium media framework, in media/ . I simply took the vpx decoder code, copied it, and modified it to use the VPU. I had VP8, MPEG2, MPEG4, and h264 decoding working. It wasnt much code, but unfortunately, the interfaces tend to change between versions, so the code would have to be updated for the newest Chromium version. It is not much code, I will try to clean it up a bit and post it. In addition, I will ask the colleagues for the other patches for accelerating the 2D and WebGL rendering. Carlos On 2014-03-20 03:50, Eric Nelson wrote: > Thanks Christian, > > On 03/19/2014 06:29 PM, Christian Betz wrote: >> The Vivante libraries can map the physical buffers produced by >> the VPU directly, so they could do format conversion on their >> way to the graphics stack if the Chromium bindings have access >> to that (the "single copy" I referred to above). >> >> >> in my experience "zero-copy" really means zero memcpy() (or similar >> system call, or really anything that involves the cpu to move around >> image data. /metadata /such as buffer pointers is generally allowed). in >> other words having the GPU do format conversion isn't considered a copy. >> >> here's a great article on this general technique implemented in >> webkit/gtk+: >> http://blogs.igalia.com/vjaquez/2013/07/26/composited-video-support-in-webkitgtk/ >> >> > > That's a nice article, and if Webkit (Blink) retains the same mapping, > video buffers from the VPU would end up hitting a RenderObject and > the only "copy" would be in rendering that to the Framebuffer. > > Since we're on the fat end of the pipe here, that copy can be expensive > and might limit things to 1x1080P decode, but I'd be happy with that. > > In the mfw_v4lsink case at least, this can be bypassed by having > the V4L2 buffer filled by the VPU and also displayed by the IPU > (such that copy), but aiming there would keep the video out of the > rest of the RenderLayer tree (I think). > > This is what I was so awkwardly trying to suggest. > >> moving beyond semantics... let me tell you what i know about this >> particular issue: >> >> there is source code in chromium to accomplish a similar goal. >> specifically there is a "generic" android driver, which works out of the >> box with i.mx6 because freescale implements the standard android video >> decode interface that allows VPU -> GPU texture uploading (don't quote >> me on this, i'm just reading the source code: >> https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/android_video_decode_accelerator.cc) >> >> >> sadly there is no such standard interface on GNU/Linux. outside of >> android (i.e. chromeos), there are drivers for particular chipsets, like >> samsung exynos. here is the source for that: >> https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/exynos_video_decode_accelerator.cc >> >> >> the group should also be aware of the current situation regarding these >> modules /only/ working when certain flags to enable "chromeos" are used >> (explained here: >> https://code.google.com/p/chromium/wiki/LinuxHWVideoDecode). in other >> words: this stuff doesn't work on your x86 with the chrome you download >> from google. >> > > Again, thanks for the info. > > Regards, > > > Eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 8:30 ` Carlos Rafael Giani @ 2014-03-20 12:22 ` Otavio Salvador 2014-03-20 14:07 ` Christian Betz 2014-03-20 14:29 ` Eric Nelson 2014-03-20 13:46 ` Christian Betz 1 sibling, 2 replies; 37+ messages in thread From: Otavio Salvador @ 2014-03-20 12:22 UTC (permalink / raw) To: Carlos Rafael Giani; +Cc: meta-freescale On Thu, Mar 20, 2014 at 5:30 AM, Carlos Rafael Giani <dv@pseudoterminal.org> wrote: > Hi Eric and Christian, > > I wanted to avoid _any_ copy involving the CPU, including any texture > upload. The Vivante GLES direct texture extension can color-convert, yes, > but it can also read pixels directly from a DMA buffer, without any copy. > This is what I use for the GLES-based sink in my GStreamer plugins, and > allows me to play 1080p videos with less than 6% CPU usage. > v4lsink would be another idea, but I am a bit skeptical, because it wouldn't > play well with any X11 or Wayland compositor, since it would essentially act > like an overlay. This is even worse with the isink (if the regular, > non-overlay layer is used), because then, the sink simply overwrites pixels > drawn by X11. What the isink does is to make sure the pixels are blit in the > area of its output window, so it does not overwrite anything outside. But > again, with desktop compositing this would break down. (Also, the isink is > quite slow.) > > I would consider this one texture upload only if absolutely necessary. > > My idea was to actually produce the texture in the decoder myself, and just > send the texture GL name around to the renderer. This would require that > everything is drawn with GLES of course - what would a software renderer do > with a GL name? > > As for the decoder itself: I implemented it in the Chromium media framework, > in media/ . I simply took the vpx decoder code, copied it, and modified it > to use the VPU. I had VP8, MPEG2, MPEG4, and h264 decoding working. It wasnt > much code, but unfortunately, the interfaces tend to change between > versions, so the code would have to be updated for the newest Chromium > version. It is not much code, I will try to clean it up a bit and post it. > In addition, I will ask the colleagues for the other patches for > accelerating the 2D and WebGL rendering. \m/ bear on mind I pushed Chromium 35 in meta-browser ;-) -- Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750 ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 12:22 ` Otavio Salvador @ 2014-03-20 14:07 ` Christian Betz 2014-03-20 15:02 ` Otavio Salvador 2014-03-20 14:29 ` Eric Nelson 1 sibling, 1 reply; 37+ messages in thread From: Christian Betz @ 2014-03-20 14:07 UTC (permalink / raw) To: Otavio Salvador; +Cc: meta-freescale [-- Attachment #1.1: Type: text/plain, Size: 955 bytes --] > > \m/ bear on mind I pushed Chromium 35 in meta-browser ;-) > this is great news. with any luck we'll let you know how it builds and works for us sometime next week. thank you! i assume you got it built successfully and have it running in some fashion. In case we have trouble, can you share what your exact setup was? I'm just assuming you used a imx6qsabresd machine with code from fsl master-next or master-branches (i.e. latest GPU drivers). did you look at chrome://gpu and confirm various acceleration modes are enabled? i attached a screenshot from my **intel/nvidia** system for reference. I think lauren is curious if webgl is working for you with latest vivante drivers. And *I* am curious about 'accelerated compositing'. btw everyone: m35 is a very ambitious release where google has ripped out GTK and replaced it with 'Aura' (a combined widget and compositing toolkit apparently). i just wanted to mention that. [-- Attachment #1.2: Type: text/html, Size: 1428 bytes --] [-- Attachment #2: about_gpu_nvidia_x86_64.png --] [-- Type: image/png, Size: 70457 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 14:07 ` Christian Betz @ 2014-03-20 15:02 ` Otavio Salvador 0 siblings, 0 replies; 37+ messages in thread From: Otavio Salvador @ 2014-03-20 15:02 UTC (permalink / raw) To: Christian Betz; +Cc: meta-freescale On Thu, Mar 20, 2014 at 11:07 AM, Christian Betz <christian.betz@gmail.com> wrote: >> \m/ bear on mind I pushed Chromium 35 in meta-browser ;-) > > > this is great news. with any luck we'll let you know how it builds and works > for us sometime next week. thank you! > > i assume you got it built successfully and have it running in some fashion. > In case we have trouble, can you share what your exact setup was? I'm just > assuming you used a imx6qsabresd machine with code from fsl master-next or > master-branches (i.e. latest GPU drivers). Gary did. > did you look at chrome://gpu and confirm various acceleration modes are > enabled? i attached a screenshot from my **intel/nvidia** system for > reference. > > I think lauren is curious if webgl is working for you with latest vivante > drivers. And *I* am curious about 'accelerated compositing'. I didn't build the new chromium for a while (working for a customer with a way older version for a specific version). > btw everyone: m35 is a very ambitious release where google has ripped out > GTK and replaced it with 'Aura' (a combined widget and compositing toolkit > apparently). i just wanted to mention that. Yes; this also opens the door for Wayland support in future ;-) -- Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750 ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 12:22 ` Otavio Salvador 2014-03-20 14:07 ` Christian Betz @ 2014-03-20 14:29 ` Eric Nelson 2014-03-20 14:58 ` Otavio Salvador 1 sibling, 1 reply; 37+ messages in thread From: Eric Nelson @ 2014-03-20 14:29 UTC (permalink / raw) To: Otavio Salvador, Carlos Rafael Giani; +Cc: meta-freescale Thanks Otavio, On 03/20/2014 05:22 AM, Otavio Salvador wrote: > On Thu, Mar 20, 2014 at 5:30 AM, Carlos Rafael Giani > <dv@pseudoterminal.org> wrote: >> Hi Eric and Christian, >> >> <snip> >> >> As for the decoder itself: I implemented it in the Chromium media framework, >> in media/ . I simply took the vpx decoder code, copied it, and modified it >> to use the VPU. I had VP8, MPEG2, MPEG4, and h264 decoding working. It wasnt >> much code, but unfortunately, the interfaces tend to change between >> versions, so the code would have to be updated for the newest Chromium >> version. It is not much code, I will try to clean it up a bit and post it. >> In addition, I will ask the colleagues for the other patches for >> accelerating the 2D and WebGL rendering. > > \m/ bear on mind I pushed Chromium 35 in meta-browser ;-) > I sent an RFC patch yesterday regarding inclusion of meta-qt5 into fsl-community-bsp-platform. Is there any interest or conflict in the same for meta-browser? A simplistic test shows that it's really straightforward to add both Chromium and Firefox into fsl-image-gui by pulling in meta-browser. Regards, Eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 14:29 ` Eric Nelson @ 2014-03-20 14:58 ` Otavio Salvador 2014-03-21 11:49 ` Diego 0 siblings, 1 reply; 37+ messages in thread From: Otavio Salvador @ 2014-03-20 14:58 UTC (permalink / raw) To: Eric Nelson; +Cc: meta-freescale On Thu, Mar 20, 2014 at 11:29 AM, Eric Nelson <eric.nelson@boundarydevices.com> wrote: > Thanks Otavio, > > > On 03/20/2014 05:22 AM, Otavio Salvador wrote: >> >> On Thu, Mar 20, 2014 at 5:30 AM, Carlos Rafael Giani >> <dv@pseudoterminal.org> wrote: >>> >>> Hi Eric and Christian, >>> >>> <snip> > >>> >>> As for the decoder itself: I implemented it in the Chromium media >>> framework, >>> in media/ . I simply took the vpx decoder code, copied it, and modified >>> it >>> to use the VPU. I had VP8, MPEG2, MPEG4, and h264 decoding working. It >>> wasnt >>> much code, but unfortunately, the interfaces tend to change between >>> versions, so the code would have to be updated for the newest Chromium >>> version. It is not much code, I will try to clean it up a bit and post >>> it. >>> In addition, I will ask the colleagues for the other patches for >>> accelerating the 2D and WebGL rendering. >> >> >> \m/ bear on mind I pushed Chromium 35 in meta-browser ;-) >> > > I sent an RFC patch yesterday regarding inclusion of meta-qt5 into > fsl-community-bsp-platform. > > Is there any interest or conflict in the same for meta-browser? > > A simplistic test shows that it's really straightforward to > add both Chromium and Firefox into fsl-image-gui by pulling > in meta-browser. The biggest problem adding more stuff is maintenance. This increases build time and tests needed for a good coverage. What people think? -- Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750 ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 14:58 ` Otavio Salvador @ 2014-03-21 11:49 ` Diego 2014-03-21 14:17 ` Eric Nelson 0 siblings, 1 reply; 37+ messages in thread From: Diego @ 2014-03-21 11:49 UTC (permalink / raw) To: meta-freescale Otavio Salvador wrote: > Eric Nelson wrote: > > A simplistic test shows that it's really straightforward to > > add both Chromium and Firefox into fsl-image-gui by pulling > > in meta-browser. > > The biggest problem adding more stuff is maintenance. This increases > build time and tests needed for a good coverage. > > What people think? Hi Otavio and Eric, I think we can sum up your discussion in two points: - web applications and in general browser usage is really important nowadays in general, and it is taking its share of interest also in the embedded world; - firefox and chromium require quite a lot of resources both in terms of maintenance and in terms of build time and computation. I can confirm both points, as we have a couple of projects using Chromium. So on one hand it is something important to support and test regularly (and that would help finding build breakages), on the other hand build takes a lot time, and an overwhelming amount of RAM. I used to build images with Firefox and Chromium 29 with 6 bitbake build threads and it worked on a 4GB of RAM and 2GB swap, but now with Chromium 35 I had to pump up the VM to 12GB of RAM (8GB still killed the final ld linking process, which is the real "RAM eater"). So while testing browsers would be nice, I think the best option would be to test it somehow separately from the fsl-image-gui. Do you think defining an image in meta-browser would make sense? Bests, Diego ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-21 11:49 ` Diego @ 2014-03-21 14:17 ` Eric Nelson 0 siblings, 0 replies; 37+ messages in thread From: Eric Nelson @ 2014-03-21 14:17 UTC (permalink / raw) To: Diego, meta-freescale Hi Diego, On 03/21/2014 04:49 AM, Diego wrote: > Otavio Salvador wrote: >> Eric Nelson wrote: >>> A simplistic test shows that it's really straightforward to >>> add both Chromium and Firefox into fsl-image-gui by pulling >>> in meta-browser. >> >> The biggest problem adding more stuff is maintenance. This increases >> build time and tests needed for a good coverage. >> >> What people think? > > Hi Otavio and Eric, > > I think we can sum up your discussion in two points: > - web applications and in general browser usage is really important nowadays > in general, and it is taking its share of interest also in the embedded world; > - firefox and chromium require quite a lot of resources both in terms of > maintenance and in terms of build time and computation. > > I can confirm both points, as we have a couple of projects using Chromium. So > on one hand it is something important to support and test regularly (and that > would help finding build breakages), on the other hand build takes a lot time, > and an overwhelming amount of RAM. I used to build images with Firefox and > Chromium 29 with 6 bitbake build threads and it worked on a 4GB of RAM and 2GB > swap, but now with Chromium 35 I had to pump up the VM to 12GB of RAM (8GB > still killed the final ld linking process, which is the real "RAM eater"). > > So while testing browsers would be nice, I think the best option would be to > test it somehow separately from the fsl-image-gui. Do you think defining an > image in meta-browser would make sense? > I wasn't suggesting that Chromium or Firefox be included in fsl-image-gui. Only that meta-browser (and I think meta-gnome as a dependency) be included in the default manifest. This will prevent the need to "git clone" the repositories separately, and allow inclusion through local.conf. Regards, Eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 8:30 ` Carlos Rafael Giani 2014-03-20 12:22 ` Otavio Salvador @ 2014-03-20 13:46 ` Christian Betz 2014-03-20 23:19 ` Carlos Rafael Giani 1 sibling, 1 reply; 37+ messages in thread From: Christian Betz @ 2014-03-20 13:46 UTC (permalink / raw) To: Carlos Rafael Giani; +Cc: meta-freescale [-- Attachment #1: Type: text/plain, Size: 1507 bytes --] > > As for the decoder itself: I implemented it in the Chromium media > framework, in media/ . I simply took the vpx decoder code, copied it, and > modified it to use the VPU. I had VP8, MPEG2, MPEG4, and h264 decoding > working. It wasnt much code, but unfortunately, the interfaces tend to > change between versions, so the code would have to be updated for the > newest Chromium version. It is not much code, I will try to clean it up a > bit and post it. In addition, I will ask the colleagues for the other > patches for accelerating the 2D and WebGL rendering. thanks carlos! my analysis suggested that my team should reproduce these chunks of code which are for samsung exynos: https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/exynos_video_decode_accelerator.cc to oversimplify greatly: we want to port 'exynos_video_decode_accelerator' to a new 'imx_video_decode_accelerator'. in this module we would make use of the same direct texture techniques as eglvivsink from gstreamer-imx. this is definitely not the route you have taken, from what I can tell. is this something you considered and steered away from? have you examined these code paths at all? there are a few other special modules besides exynos too. i might be wrong, but it could be the best chance of getting something upstreamed would be do it "like everyone else" (i.e. samsung). but i'm probably thinking *way* too far ahead here. [-- Attachment #2: Type: text/html, Size: 2088 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 13:46 ` Christian Betz @ 2014-03-20 23:19 ` Carlos Rafael Giani 2014-03-21 12:21 ` Dmitriy B. 2014-04-01 19:22 ` Eric Nelson 0 siblings, 2 replies; 37+ messages in thread From: Carlos Rafael Giani @ 2014-03-20 23:19 UTC (permalink / raw) To: Christian Betz; +Cc: meta-freescale [-- Attachment #1.1: Type: text/plain, Size: 3898 bytes --] On 2014-03-20 14:46, Christian Betz wrote: > > As for the decoder itself: I implemented it in the Chromium media > framework, in media/ . I simply took the vpx decoder code, copied > it, and modified it to use the VPU. I had VP8, MPEG2, MPEG4, and > h264 decoding working. It wasnt much code, but unfortunately, the > interfaces tend to change between versions, so the code would have > to be updated for the newest Chromium version. It is not much > code, I will try to clean it up a bit and post it. In addition, I > will ask the colleagues for the other patches for accelerating the > 2D and WebGL rendering. > > > thanks carlos! > > my analysis suggested that my team should reproduce these chunks of > code which are for samsung exynos: > > https://chromium.googlesource.com/chromium/src/+/a361fce28da709ea872062f30fb4b65fcc37b695/content/common/gpu/media/exynos_video_decode_accelerator.cc > > to oversimplify greatly: we want to port > 'exynos_video_decode_accelerator' to a new > 'imx_video_decode_accelerator'. in this module we would make use of > the same direct texture techniques as eglvivsink from gstreamer-imx. > > this is definitely not the route you have taken, from what I can tell. > is this something you considered and steered away from? have you > examined these code paths at all? there are a few other special > modules besides exynos too. > > i might be wrong, but it could be the best chance of getting something > upstreamed would be do it "like everyone else" (i.e. samsung). but i'm > probably thinking *way* too far ahead here. Back then we needed some hw decoding quickly, so we did not look further into this, since we had spent a lot of time getting the 2D acceleration stable already. I could only briefly look at the exynos accelerator code, but it does look like the right direction, although the VPU uses physical addresses directly instead of dmabuf FDs. I am currently writing a frontend for the libfslvpuwrap, which takes care of certain complexities (like being able to pass userdata pointers to frames, to make it possible to associate input and output frames, which is essential for correct timestamping, especially when things like h264 frame reordering are active). This frontend will hide these things in a future gstreamer-imx release to make the decoder code clearer and easier to maintain, and is intended to be reusable, for example for Chromium integration, or XBMC. Beyond that, my patches are probably not that helpful anymore, since they were based on the vpx decoder way of decoding, and I agree that the exynos code is a better starting point. Nevertheless, I attached the patch. Keep in mind that it is very basic, old, and wasn't further developed on because it was "good enough" for the project. But here are some additional notes from our efforts to accelerate Chromium on imx6 (keep in mind these apply to version 32.0.1664.3 ) : 1. We started the google-chrome binary with these switches: --ignore-gpu-blacklist --enable-gpu --use-gl=egl --enable-accelerated-2d-canvas 2. To make building Chromium easier, we switched to component build (by default, it is all linked into one enormous binary). I attached a patch that was necessary to fix some minor issues. Perhaps it is not necessary anymore. 3. Chromium tried to load libEGL with dlopen() , which caused problems, because with the Vivante libraries, libEGL also needs libGAL. Usually, build scripts just add -lEGL -lGAL to the linker line. With dlopen() this wasn't possible of course. We patched the gyp scripts to link against this libraries during building instead. I attached this patch. It is really a hack, because it circumvents the sandboxing process (this is why Chromium loads EGL and GLESv2 with dlopen() ). Carlos [-- Attachment #1.2: Type: text/html, Size: 5719 bytes --] [-- Attachment #2: 0002-Fix-issue-with-component-linking.patch --] [-- Type: text/x-patch, Size: 823 bytes --] --- a/build/common.gypi 2013-09-05 12:33:58.000000000 +0200 +++ b/build/common.gypi 2013-09-13 09:29:24.185329621 +0200 @@ -2840,11 +2840,6 @@ '-Wl,--as-needed', ], }], - ['OS=="linux" and target_arch=="ia32"', { - 'ldflags': [ - '-Wl,--no-as-needed', - ], - }], ], }, 'Release_Base': { @@ -3460,6 +3455,10 @@ # gets the right path. '-B<(PRODUCT_DIR)/../../third_party/gold', ], + }, { + 'ldflags': [ + '-Wl,--no-as-needed', + ], }], ['native_discardable_memory', { 'defines': ['DISCARDABLE_MEMORY_ALWAYS_SUPPORTED_NATIVELY'], [-- Attachment #3: 0004-Modified-gl.gyp-to-link-libEGL-and-libGAL-at-build-t.patch --] [-- Type: text/x-patch, Size: 636 bytes --] diff --git a/ui/gl/gl.gyp b/ui/gl/gl.gyp index 274065b..e857f9c 100644 --- a/ui/gl/gl.gyp +++ b/ui/gl/gl.gyp @@ -123,6 +123,12 @@ '<(gl_binding_output_dir)/gl_bindings_autogen_osmesa.h', '<(gl_binding_output_dir)/gl_interface_autogen_gl.h', ], + 'link_settings': { + 'libraries': [ + '-lEGL', + '-lGAL', + ], + }, # hard_dependency is necessary for this target because it has actions # that generate header files included by dependent targets. The header # files must be generated before the dependents are compiled. The usual [-- Attachment #4: 0005-Added-hardware-accelerated-decoding-with-the-i.MX-VP.patch --] [-- Type: text/x-patch, Size: 28854 bytes --] diff --git a/content/renderer/media/webmediaplayer_impl.cc b/content/renderer/media/webmediaplayer_impl.cc index a99c3ae..1ec8bfa 100644 --- a/content/renderer/media/webmediaplayer_impl.cc +++ b/content/renderer/media/webmediaplayer_impl.cc @@ -47,6 +47,7 @@ #include "media/filters/gpu_video_decoder.h" #include "media/filters/opus_audio_decoder.h" #include "media/filters/video_renderer_base.h" +#include "media/filters/vpu_video_decoder.h" #include "media/filters/vpx_video_decoder.h" #include "third_party/WebKit/public/platform/WebMediaSource.h" #include "third_party/WebKit/public/platform/WebRect.h" @@ -1103,6 +1104,10 @@ void WebMediaPlayerImpl::StartPipeline() { new media::GpuVideoDecoder(gpu_factories_, media_log_)); } +#if !defined(MEDIA_DISABLE_IMXVPU) + video_decoders.push_back(new media::VpuVideoDecoder(media_loop_)); +#endif // !defined(MEDIA_DISABLE_IMXVPU) + // TODO(phajdan.jr): Remove ifdefs when libvpx with vp9 support is released // (http://crbug.com/174287) . #if !defined(MEDIA_DISABLE_LIBVPX) diff --git a/media/filters/vpu_framebuffers.cc b/media/filters/vpu_framebuffers.cc new file mode 100644 index 0000000..fce43fc --- /dev/null +++ b/media/filters/vpu_framebuffers.cc @@ -0,0 +1,106 @@ +#include <string.h> +#include <stdint.h> +#include "vpu_framebuffers.h" +#include "base/logging.h" + + +#define ALIGN_VAL_TO(LENGTH, ALIGN_SIZE) ( ((uintptr_t)((LENGTH) + (ALIGN_SIZE) - 1) / (ALIGN_SIZE)) * (ALIGN_SIZE) ) +#define FRAME_ALIGN 16 + + +namespace media { + + +VPUFramebuffers::VPUFramebuffers() { +} + + +void VPUFramebuffers::init(VpuDecHandle handle, VpuDecInitInfo const &init_info) { + framebuffers.resize(init_info.nMinFrameBufferCount); + + y_stride = ALIGN_VAL_TO(init_info.nPicWidth, FRAME_ALIGN); + if (init_info.nInterlace) + y_size = y_stride * ALIGN_VAL_TO(init_info.nPicHeight, (2 * FRAME_ALIGN)); + else + y_size = y_stride * ALIGN_VAL_TO(init_info.nPicHeight, FRAME_ALIGN); + + uv_stride = y_stride / 2; + u_size = v_size = mv_size = y_size / 4; + + int alignment = init_info.nAddressAlignment; + if (alignment > 1) + { + y_size = ALIGN_VAL_TO(y_size, alignment); + u_size = ALIGN_VAL_TO(u_size, alignment); + v_size = ALIGN_VAL_TO(v_size, alignment); + mv_size = ALIGN_VAL_TO(mv_size, alignment); + } + + pic_width = init_info.nPicWidth; + pic_height = init_info.nPicHeight; + + total_size = y_size + u_size + v_size + mv_size + alignment; + + LOG(INFO) << "Num VPU framebuffers: " << framebuffers.size(); + LOG(INFO) << "VPU framebuffer memory block size:" + << " total: " << total_size + << " Y " << y_size + << " U: " << u_size + << " V: " << v_size + << " Mv: " << mv_size + << " alignment: " << alignment + ; + + for (Framebuffers::iterator iter = framebuffers.begin(); iter != framebuffers.end(); ++iter) { + unsigned char *virt_ptr, *phys_ptr; + + VpuFrameBuffer &framebuffer = *iter; + + VpuMemDesc mem_desc; + memset(&mem_desc, 0, sizeof(VpuMemDesc)); + mem_desc.nSize = total_size; + VPU_DecGetMem(&mem_desc); + phys_mem_blocks.push_back(mem_desc); + + virt_ptr = (unsigned char*)(mem_desc.nVirtAddr); + phys_ptr = (unsigned char*)(mem_desc.nPhyAddr); + + if (alignment > 1) { + phys_ptr = (unsigned char*)ALIGN_VAL_TO(phys_ptr, alignment); + virt_ptr = (unsigned char*)ALIGN_VAL_TO(virt_ptr, alignment); + } + + framebuffer.nStrideY = y_stride; + framebuffer.nStrideC = uv_stride; + + /* fill phy addr*/ + framebuffer.pbufY = phys_ptr; + framebuffer.pbufCb = phys_ptr + y_size; + framebuffer.pbufCr = phys_ptr + y_size + u_size; + framebuffer.pbufMvCol = phys_ptr + y_size + u_size + v_size; + + /* fill virt addr */ + framebuffer.pbufVirtY = virt_ptr; + framebuffer.pbufVirtCb = virt_ptr + y_size; + framebuffer.pbufVirtCr = virt_ptr + y_size + u_size; + framebuffer.pbufVirtMvCol = virt_ptr + y_size + u_size + v_size; + + framebuffer.pbufY_tilebot = 0; + framebuffer.pbufCb_tilebot = 0; + framebuffer.pbufVirtY_tilebot = 0; + framebuffer.pbufVirtCb_tilebot = 0; + } + + VPU_DecRegisterFrameBuffer(handle, &(framebuffers[0]), framebuffers.size()); +} + + +VPUFramebuffers::~VPUFramebuffers() { + LOG(INFO) << "Freeing VPU framebuffer memory"; + for (PhysMemBlocks::iterator iter = phys_mem_blocks.begin(); iter != phys_mem_blocks.end(); ++iter) + VPU_DecFreeMem(&(*iter)); +} + + +} + diff --git a/media/filters/vpu_framebuffers.h b/media/filters/vpu_framebuffers.h new file mode 100644 index 0000000..bb9b021 --- /dev/null +++ b/media/filters/vpu_framebuffers.h @@ -0,0 +1,36 @@ +#ifndef MEDIA_FILTERS_VPU_FRAMEBUFFERS_H_ +#define MEDIA_FILTERS_VPU_FRAMEBUFFERS_H_ + +#include <list> +#include <vector> +#include <vpu_wrapper.h> + + +namespace media { + + +class VPUFramebuffers +{ +public: + explicit VPUFramebuffers(); + ~VPUFramebuffers(); + + void init(VpuDecHandle handle, VpuDecInitInfo const &init_info); + +private: + typedef std::vector < VpuFrameBuffer > Framebuffers; + typedef std::list < VpuMemDesc > PhysMemBlocks; + + PhysMemBlocks phys_mem_blocks; + Framebuffers framebuffers; + int y_stride, uv_stride; + int y_size, u_size, v_size, mv_size; + int total_size; + unsigned int pic_width, pic_height; +}; + + +} + + +#endif diff --git a/media/filters/vpu_video_decoder.cc b/media/filters/vpu_video_decoder.cc new file mode 100644 index 0000000..396d4f1 --- /dev/null +++ b/media/filters/vpu_video_decoder.cc @@ -0,0 +1,614 @@ +#include <list> +#include <vector> +#include <stddef.h> +#include <stdint.h> +#include <algorithm> +#include <string> + +#include <string.h> + +#include "media/filters/vpu_video_decoder.h" + +#include "base/bind.h" +#include "base/callback_helpers.h" +#include "base/command_line.h" +#include "base/location.h" +#include "base/logging.h" +#include "base/synchronization/lock.h" +#include "base/message_loop/message_loop_proxy.h" +#include "base/strings/string_number_conversions.h" +#include "base/sys_byteorder.h" +#include "media/base/bind_to_loop.h" +#include "media/base/decoder_buffer.h" +#include "media/base/demuxer_stream.h" +#include "media/base/media_switches.h" +#include "media/base/pipeline.h" +#include "media/base/video_decoder_config.h" +#include "media/base/video_frame.h" +#include "media/base/video_util.h" + +#include "media/filters/vpu_framebuffers.h" + +#include <vpu_wrapper.h> + + +#define ALIGN_VAL_TO(LENGTH, ALIGN_SIZE) ( ((uintptr_t)((LENGTH) + (ALIGN_SIZE) - 1) / (ALIGN_SIZE)) * (ALIGN_SIZE) ) + + + +namespace media { + + +namespace { + + +char const *VpuStrerror(VpuDecRetCode const code) { + switch (code) { + case VPU_DEC_RET_SUCCESS: return "success"; + case VPU_DEC_RET_FAILURE: return "failure"; + case VPU_DEC_RET_INVALID_PARAM: return "invalid param"; + case VPU_DEC_RET_INVALID_HANDLE: return "invalid handle"; + case VPU_DEC_RET_INVALID_FRAME_BUFFER: return "invalid frame buffer"; + case VPU_DEC_RET_INSUFFICIENT_FRAME_BUFFERS: return "insufficient frame buffers"; + case VPU_DEC_RET_INVALID_STRIDE: return "invalid stride"; + case VPU_DEC_RET_WRONG_CALL_SEQUENCE: return "wrong call sequence"; + case VPU_DEC_RET_FAILURE_TIMEOUT: return "failure timeout"; + default: + return NULL; + } +} + + +long inst_count = 0; +base::Lock vpu_load_lock; + + +void LoadDecoder() { + VpuDecRetCode ret; + + base::AutoLock alock(vpu_load_lock); + + if (inst_count == 0) { + ret = VPU_DecLoad(); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "VPU loading failed: " << VpuStrerror(ret); + return; + } + + } + + ++inst_count; + + if (inst_count == 1) { + VpuVersionInfo version; + VpuWrapperVersionInfo wrapper_version; + + ret = VPU_DecGetVersionInfo(&version); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Getting version info failed: " << VpuStrerror(ret); + return; + } + + ret = VPU_DecGetWrapperVersionInfo(&wrapper_version); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Getting wrapper version info failed: " << VpuStrerror(ret); + return; + } + + LOG(INFO) << "VPU Loaded"; + LOG(INFO) << "VPU firmware version " << version.nFwMajor << "." << version.nFwMinor << "." << version.nFwRelease << "_r" << version.nFwCode; + LOG(INFO) << "VPU library version " << version.nLibMajor << "." << version.nLibMinor << "." << version.nLibRelease; + LOG(INFO) << "VPU wrapper version " << wrapper_version.nMajor << "." << wrapper_version.nMinor << "." << wrapper_version.nRelease << " " << wrapper_version.pBinary; + } +} + + +void UnloadDecoder() { + base::AutoLock alock(vpu_load_lock); + + if (inst_count == 0) + return; + + --inst_count; + + if (inst_count == 0) + VPU_DecUnLoad(); +} + + +} + + +struct VpuVideoDecoder::Internal +{ + typedef std::vector < uint8_t > VirtMemBlock; + typedef std::list < VirtMemBlock > VirtMemBlocks; + + typedef std::list < VpuMemDesc > PhysMemBlocks; + + VpuDecHandle handle; + + VpuDecInitInfo init_info; + VpuMemInfo mem_info; + + bool vpu_opened; + + VirtMemBlocks virt_mem_blocks; + PhysMemBlocks phys_mem_blocks; + + VPUFramebuffers framebuffers; + + Internal() + : vpu_opened(false) + { + } + + ~Internal() + { + FreeMemBlocks(); + } + + bool AllocMemBlocks() + { + VpuDecRetCode ret; + + memset(&mem_info, 0, sizeof(VpuMemInfo)); + ret = VPU_DecQueryMem(&mem_info); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get VPU memory information: " << VpuStrerror(ret); + return false; + } + + for (int i = 0; i < mem_info.nSubBlockNum; ++i) + { + int size = mem_info.MemSubBlock[i].nAlignment + mem_info.MemSubBlock[i].nSize; + + LOG(INFO) << "Sub block " << i << " type: " << ((mem_info.MemSubBlock[i].MemType == VPU_MEM_VIRT) ? "virtual" : "phys") << " size: " << size; + + if (mem_info.MemSubBlock[i].MemType == VPU_MEM_VIRT) + { + unsigned char *virt_ptr; + virt_mem_blocks.push_back(VirtMemBlock()); + VirtMemBlock &blk = virt_mem_blocks.back(); + blk.resize(size); + virt_ptr = reinterpret_cast < unsigned char* > (&blk[0]); + + mem_info.MemSubBlock[i].pVirtAddr = (unsigned char *)ALIGN_VAL_TO(virt_ptr, mem_info.MemSubBlock[i].nAlignment); + } + else if (mem_info.MemSubBlock[i].MemType == VPU_MEM_PHY) + { + VpuMemDesc mem_desc; + memset(&mem_desc, 0, sizeof(VpuMemDesc)); + mem_desc.nSize = size; + if (VPU_DecGetMem(&mem_desc) != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Unable to allocate " << size << " bytes of physical memory"; + return false; + } + + phys_mem_blocks.push_back(mem_desc); + + mem_info.MemSubBlock[i].pVirtAddr = (unsigned char *)ALIGN_VAL_TO((unsigned char*)(mem_desc.nVirtAddr), mem_info.MemSubBlock[i].nAlignment); + mem_info.MemSubBlock[i].pPhyAddr = (unsigned char *)ALIGN_VAL_TO((unsigned char*)(mem_desc.nPhyAddr), mem_info.MemSubBlock[i].nAlignment); + } + else + LOG(WARNING) << "Sub block " << i << " type is unknown - skipping"; + } + + LOG(INFO) << "VPU memory blocks allocated"; + + return true; + } + + void FreeMemBlocks() + { + LOG(INFO) << "Freeing VPU memory blocks"; + + for (PhysMemBlocks::iterator iter = phys_mem_blocks.begin(); iter != phys_mem_blocks.end(); ++iter) + VPU_DecFreeMem(&(*iter)); + } +}; + + + +void CopyVpuFramebufferTo(VpuFrameBuffer *framebuffer, scoped_refptr<VideoFrame>* video_frame, VpuDecInitInfo &init_info, const VideoDecoderConfig& config); + + + + +VpuVideoDecoder::VpuVideoDecoder( + const scoped_refptr<base::MessageLoopProxy>& message_loop) + : message_loop_(message_loop), + weak_factory_(this), + state_(kUninitialized) { + LoadDecoder(); + internal = new Internal; +} + +VpuVideoDecoder::~VpuVideoDecoder() { + DCHECK_EQ(kUninitialized, state_); + CloseDecoder(); + delete internal; + UnloadDecoder(); +} + +void VpuVideoDecoder::Initialize(const VideoDecoderConfig& config, + const PipelineStatusCB& status_cb) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK(config.IsValidConfig()); + DCHECK(!config.is_encrypted()); + DCHECK(decode_cb_.is_null()); + DCHECK(reset_cb_.is_null()); + + weak_this_ = weak_factory_.GetWeakPtr(); + + if (!internal->AllocMemBlocks()) { + status_cb.Run(PIPELINE_ERROR_INITIALIZATION_FAILED); + return; + } + + if (!ConfigureDecoder(config)) { + status_cb.Run(DECODER_ERROR_NOT_SUPPORTED); + return; + } + + // Success! + LOG(INFO) << "VPU decoder initialization succeeded"; + config_ = config; + state_ = kNormal; + status_cb.Run(PIPELINE_OK); +} + +bool VpuVideoDecoder::ConfigureDecoder(const VideoDecoderConfig& config) { + bool can_handle = false; + + if ((config.format() == VideoFrame::YV12) || (config.format() == VideoFrame::YV12A) || (config.format() == VideoFrame::I420)) { + switch (config.codec()) { + case kCodecH264: + case kCodecVC1: + case kCodecMPEG2: + case kCodecMPEG4: + case kCodecVP8: + can_handle = true; + default: + break; + } + } + + if (!can_handle) { + LOG(INFO) << "Cannot handle video config: " << config.AsHumanReadableString(); + return false; + } + + CloseDecoder(); + return OpenDecoder(config); +} + +bool VpuVideoDecoder::OpenDecoder(const VideoDecoderConfig& config) { + if (internal->vpu_opened) + return true; + + VpuDecOpenParam open_param; + VpuDecRetCode ret; + + memset(&open_param, 0, sizeof(open_param)); + + switch (config.codec()) + { + case kCodecH264: + open_param.CodecFormat = VPU_V_AVC; + open_param.nReorderEnable = 1; + LOG(INFO) << "Setting h.264 as stream format"; + break; + case kCodecVC1: + open_param.CodecFormat = VPU_V_VC1_AP; + LOG(INFO) << "Setting VC1 as stream format"; + break; + case kCodecMPEG2: + open_param.CodecFormat = VPU_V_MPEG2; + LOG(INFO) << "Setting MPEG-2 as stream format"; + break; + case kCodecMPEG4: + open_param.CodecFormat = VPU_V_MPEG4; + LOG(INFO) << "Setting MPEG-4 as stream format"; + break; + case kCodecVP8: + open_param.CodecFormat = VPU_V_VP8; + LOG(INFO) << "Setting VP8 as stream format"; + break; + default: + LOG(ERROR) << "Could not set unknown stream format"; + break; + } + + open_param.nChromaInterleave = 0; + open_param.nMapType = 0; + open_param.nTiled2LinearEnable = 0; + open_param.nEnableFileMode = 0; + open_param.nPicWidth = config.coded_size().width(); + open_param.nPicHeight = config.coded_size().height(); + + ret = VPU_DecOpen(&(internal->handle), &open_param, &(internal->mem_info)); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Opening new VPU handle failed: " << VpuStrerror(ret); + return false; + } + + int config_param; + + config_param = VPU_DEC_SKIPNONE; + ret = VPU_DecConfig(internal->handle, VPU_DEC_CONF_SKIPMODE, &config_param); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not configure skip mode: " << VpuStrerror(ret); + return false; + } + + config_param = 0; + ret = VPU_DecConfig(internal->handle, VPU_DEC_CONF_BUFDELAY, &config_param); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not configure buffer delay: " << VpuStrerror(ret); + return false; + } + + config_param = VPU_DEC_IN_NORMAL; + ret = VPU_DecConfig(internal->handle, VPU_DEC_CONF_INPUTTYPE, &config_param); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not configure input type: " << VpuStrerror(ret); + return false; + } + + internal->vpu_opened = true; + + return true; +} + +void VpuVideoDecoder::CloseDecoder() { + VpuDecRetCode ret; + + if (!(internal->vpu_opened)) + return; + + ret = VPU_DecFlushAll(internal->handle); + if (ret != VPU_DEC_RET_SUCCESS) + LOG(ERROR) << "Flushing decoder failed: " << VpuStrerror(ret); + VPU_DecClose(internal->handle); + if (ret != VPU_DEC_RET_SUCCESS) + LOG(ERROR) << "Closing decoder failed: " << VpuStrerror(ret); + + internal->vpu_opened = false; +} + +void VpuVideoDecoder::Decode(const scoped_refptr<DecoderBuffer>& buffer, + const DecodeCB& decode_cb) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK(!decode_cb.is_null()); + CHECK_NE(state_, kUninitialized); + CHECK(decode_cb_.is_null()) << "Overlapping decodes are not supported."; + + decode_cb_ = BindToCurrentLoop(decode_cb); + + if (state_ == kError) { + base::ResetAndReturn(&decode_cb_).Run(kDecodeError, NULL); + return; + } + + // Return empty frames if decoding has finished. + if (state_ == kDecodeFinished) { + base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEmptyFrame()); + return; + } + + DecodeBuffer(buffer); +} + +void VpuVideoDecoder::Reset(const base::Closure& closure) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK(reset_cb_.is_null()); + reset_cb_ = BindToCurrentLoop(closure); + + // Defer the reset if a decode is pending. + if (!decode_cb_.is_null()) + return; + + DoReset(); +} + +void VpuVideoDecoder::Stop(const base::Closure& closure) { + DCHECK(message_loop_->BelongsToCurrentThread()); + base::ScopedClosureRunner runner(BindToCurrentLoop(closure)); + + if (state_ == kUninitialized) + return; + + if (!decode_cb_.is_null()) { + base::ResetAndReturn(&decode_cb_).Run(kOk, NULL); + // Reset is pending only when decode is pending. + if (!reset_cb_.is_null()) + base::ResetAndReturn(&reset_cb_).Run(); + } + + state_ = kUninitialized; +} + +bool VpuVideoDecoder::HasAlpha() const { + return false; +} + +void VpuVideoDecoder::DecodeBuffer(const scoped_refptr<DecoderBuffer>& buffer) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK_NE(state_, kUninitialized); + DCHECK_NE(state_, kDecodeFinished); + DCHECK_NE(state_, kError); + DCHECK(reset_cb_.is_null()); + DCHECK(!decode_cb_.is_null()); + DCHECK(buffer); + + // Transition to kDecodeFinished on the first end of stream buffer. + if (state_ == kNormal && buffer->end_of_stream()) { + state_ = kDecodeFinished; + base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEmptyFrame()); + return; + } + + scoped_refptr<VideoFrame> video_frame; + if (!VpuDecode(buffer, &video_frame)) { + state_ = kError; + base::ResetAndReturn(&decode_cb_).Run(kDecodeError, NULL); + return; + } + + // If we didn't get a frame we need more data. + if (!video_frame.get()) { + base::ResetAndReturn(&decode_cb_).Run(kNotEnoughData, NULL); + return; + } + + base::ResetAndReturn(&decode_cb_).Run(kOk, video_frame); +} + +bool VpuVideoDecoder::VpuDecode(const scoped_refptr<DecoderBuffer>& buffer, + scoped_refptr<VideoFrame>* video_frame) { + DCHECK(video_frame); + DCHECK(!buffer->end_of_stream()); + + int buffer_ret_code; + VpuBufferNode in_data; + VpuDecRetCode ret; + + memset(&in_data, 0, sizeof(in_data)); + in_data.pVirAddr = (unsigned char *)(buffer->data()); + in_data.nSize = buffer->data_size(); + + ret = VPU_DecDecodeBuf(internal->handle, &in_data, &buffer_ret_code); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Failed to decode frame: " << VpuStrerror(ret); + return false; + } + + VLOG(1) << "VPU_DecDecodeBuf buffer ret code is 0x" << std::hex << buffer_ret_code << std::dec; + + if (buffer_ret_code & VPU_DEC_INIT_OK) { + ret = VPU_DecGetInitialInfo(internal->handle, &(internal->init_info)); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get init info: " << VpuStrerror(ret); + return false; + } + internal->framebuffers.init(internal->handle, internal->init_info); + } + + if (buffer_ret_code & VPU_DEC_FLUSH) { + ret = VPU_DecFlushAll(internal->handle); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Flushing VPU failed: " << VpuStrerror(ret); + return false; + } + + *video_frame = NULL; + + return true; + } + + if (buffer_ret_code & VPU_DEC_NO_ENOUGH_INBUF) { + LOG(INFO) << "Need more input"; + return true; + } + + if (buffer_ret_code & VPU_DEC_ONE_FRM_CONSUMED) { + VpuDecFrameLengthInfo dec_framelen_info; + + ret = VPU_DecGetConsumedFrameInfo(internal->handle, &dec_framelen_info); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get information about consumed frame: " << VpuStrerror(ret); + } else { + LOG(INFO) << "One frame got consumed:" + << " framebuffer addr: 0x" << std::hex << uintptr_t(dec_framelen_info.pFrame) << std::dec + << " stuff length: " << dec_framelen_info.nStuffLength + << " frame length: " << dec_framelen_info.nFrameLength + ; + } + } + + if (buffer_ret_code & VPU_DEC_OUTPUT_DIS) { + VpuDecOutFrameInfo out_frame_info; + int64 timestamp = buffer->timestamp().InMicroseconds(); + + ret = VPU_DecGetOutputFrame(internal->handle, &out_frame_info); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get decoded output frame: " << VpuStrerror(ret); + return false; + } + + LOG(INFO) << "Output frame:" + << " pic width: " << internal->init_info.nPicWidth + << " pic height: " << internal->init_info.nPicHeight + << " pic type: " << out_frame_info.ePicType + << " Y stride: " << out_frame_info.pDisplayFrameBuf->nStrideY + << " CbCr stride: " << out_frame_info.pDisplayFrameBuf->nStrideC + ; + + CopyVpuFramebufferTo(out_frame_info.pDisplayFrameBuf, video_frame, internal->init_info, config_); + (*video_frame)->SetTimestamp(base::TimeDelta::FromMicroseconds(timestamp)); + + ret = VPU_DecOutFrameDisplayed(internal->handle, out_frame_info.pDisplayFrameBuf); + if (ret != VPU_DEC_RET_SUCCESS) + LOG(ERROR) << "Clearing display framebuffer failed: " << VpuStrerror(ret); + } + else + *video_frame = NULL; + + return true; +} + +void VpuVideoDecoder::DoReset() { + DCHECK(decode_cb_.is_null()); + + state_ = kNormal; + reset_cb_.Run(); + reset_cb_.Reset(); +} + +void CopyVpuFramebufferTo(VpuFrameBuffer *framebuffer, scoped_refptr<VideoFrame>* video_frame, VpuDecInitInfo &init_info, const VideoDecoderConfig& config) { + CHECK(framebuffer); + + gfx::Size size(init_info.nPicWidth, init_info.nPicHeight); + + *video_frame = VideoFrame::CreateFrame( + config.format(), + size, + gfx::Rect(size), + config.natural_size(), + kNoTimestamp()); + + CopyYPlane(framebuffer->pbufVirtY, + framebuffer->nStrideY, + init_info.nPicHeight, + video_frame->get()); + if (config.format() == VideoFrame::I420) { + CopyUPlane(framebuffer->pbufVirtCb, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + CopyVPlane(framebuffer->pbufVirtCr, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + } else { /* YV12 and YV12A */ + /* U and V swapped */ + CopyUPlane(framebuffer->pbufVirtCb, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + CopyVPlane(framebuffer->pbufVirtCr, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + + if (config.format() == VideoFrame::YV12A) { + MakeOpaqueAPlane(framebuffer->nStrideY, + init_info.nPicHeight, + video_frame->get()); + } + } +} + + +} // namespace media diff --git a/media/filters/vpu_video_decoder.h b/media/filters/vpu_video_decoder.h new file mode 100644 index 0000000..d7016aa --- /dev/null +++ b/media/filters/vpu_video_decoder.h @@ -0,0 +1,74 @@ +#ifndef MEDIA_FILTERS_VPU_VIDEO_DECODER_H_ +#define MEDIA_FILTERS_VPU_VIDEO_DECODER_H_ + +#include "base/callback.h" +#include "base/memory/weak_ptr.h" +#include "media/base/demuxer_stream.h" +#include "media/base/video_decoder.h" +#include "media/base/video_decoder_config.h" +#include "media/base/video_frame.h" + +namespace base { +class MessageLoopProxy; +} + +namespace media { + +class MEDIA_EXPORT VpuVideoDecoder : public VideoDecoder { + public: + explicit VpuVideoDecoder( + const scoped_refptr<base::MessageLoopProxy>& message_loop); + virtual ~VpuVideoDecoder(); + + // VideoDecoder implementation. + virtual void Initialize(const VideoDecoderConfig& config, + const PipelineStatusCB& status_cb) OVERRIDE; + virtual void Decode(const scoped_refptr<DecoderBuffer>& buffer, + const DecodeCB& decode_cb) OVERRIDE; + virtual void Reset(const base::Closure& closure) OVERRIDE; + virtual void Stop(const base::Closure& closure) OVERRIDE; + virtual bool HasAlpha() const OVERRIDE; + + private: + enum DecoderState { + kUninitialized, + kNormal, + kFlushCodec, + kDecodeFinished, + kError + }; + + // Handles (re-)initializing the decoder with a (new) config. + // Returns true when initialization was successful. + bool ConfigureDecoder(const VideoDecoderConfig& config); + + bool OpenDecoder(const VideoDecoderConfig& config); + void CloseDecoder(); + + void DecodeBuffer(const scoped_refptr<DecoderBuffer>& buffer); + bool VpuDecode(const scoped_refptr<DecoderBuffer>& buffer, + scoped_refptr<VideoFrame>* video_frame); + + // Reset decoder and call |reset_cb_|. + void DoReset(); + + scoped_refptr<base::MessageLoopProxy> message_loop_; + base::WeakPtrFactory<VpuVideoDecoder> weak_factory_; + base::WeakPtr<VpuVideoDecoder> weak_this_; + + DecoderState state_; + + DecodeCB decode_cb_; + base::Closure reset_cb_; + + VideoDecoderConfig config_; + + struct Internal; + Internal *internal; + + DISALLOW_COPY_AND_ASSIGN(VpuVideoDecoder); +}; + +} // namespace media + +#endif // MEDIA_FILTERS_VPU_VIDEO_DECODER_H_ diff --git a/media/media.gyp b/media/media.gyp index a25571d..b12d386 100644 --- a/media/media.gyp +++ b/media/media.gyp @@ -11,6 +11,7 @@ # (DT_NEEDED) instead of using dlopen. This helps with automated # detection of ABI mismatches and prevents silent errors. 'linux_link_pulseaudio%': 0, + 'media_use_imxvpu%': 1, 'conditions': [ ['OS=="android"', { # Android doesn't use ffmpeg. @@ -377,6 +378,10 @@ 'filters/video_frame_stream.h', 'filters/video_renderer_base.cc', 'filters/video_renderer_base.h', + 'filters/vpu_framebuffers.cc', + 'filters/vpu_framebuffers.h', + 'filters/vpu_video_decoder.cc', + 'filters/vpu_video_decoder.h', 'filters/vpx_video_decoder.cc', 'filters/vpx_video_decoder.h', 'filters/wsola_internals.cc', @@ -592,6 +597,29 @@ 'DISABLE_USER_INPUT_MONITOR', ], }], + ['media_use_imxvpu==1', { + 'cflags': [ + '<!@(<(pkg-config) --cflags libfslvpuwrap)', + ], + 'link_settings': { + 'libraries': [ + '<!@(<(pkg-config) --libs libfslvpuwrap)', + ], + }, + }, { # media_use_imxvpu==0 + 'direct_dependent_settings': { + 'defines': [ + 'MEDIA_DISABLE_IMXVPU', + ], + }, + # Exclude the VPU sources. + 'sources!': [ + 'filters/vpu_framebuffers.cc', + 'filters/vpu_framebuffers.h', + 'filters/vpu_video_decoder.cc', + 'filters/vpu_video_decoder.h', + ], + }], ['use_cras==1', { 'cflags': [ '<!@(<(pkg-config) --cflags libcras)', ^ permalink raw reply related [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 23:19 ` Carlos Rafael Giani @ 2014-03-21 12:21 ` Dmitriy B. 2014-04-01 19:22 ` Eric Nelson 1 sibling, 0 replies; 37+ messages in thread From: Dmitriy B. @ 2014-03-21 12:21 UTC (permalink / raw) To: meta-freescale [-- Attachment #1: Type: text/plain, Size: 513 bytes --] > > 3. Chromium tried to load libEGL with dlopen() , which caused problems, > because with the Vivante libraries, libEGL also needs libGAL. Usually, > build scripts just add -lEGL -lGAL to the linker line. > Vivante finally fixed that in the latest EGL binaries release so now you dont need to manually add libGAL everytime. libEGL/GLESv1-2/GL now depend on libGAL properly (you can check that with ldd). Looking forward to test Chromium with your imx6 patches soon! Best Regards, Dmitriy Beykun [-- Attachment #2: Type: text/html, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 23:19 ` Carlos Rafael Giani 2014-03-21 12:21 ` Dmitriy B. @ 2014-04-01 19:22 ` Eric Nelson 2014-04-02 10:21 ` Carlos Rafael Giani ` (2 more replies) 1 sibling, 3 replies; 37+ messages in thread From: Eric Nelson @ 2014-04-01 19:22 UTC (permalink / raw) To: Carlos Rafael Giani, Christian Betz, meta-freescale [-- Attachment #1: Type: text/plain, Size: 2727 bytes --] Thanks again Carlos, On 03/20/2014 04:19 PM, Carlos Rafael Giani wrote: > > <snip> > > Back then we needed some hw decoding quickly, so we did not look further > into this, since we had spent a lot of time getting the 2D acceleration > stable already. I could only briefly look at the exynos accelerator > code, but it does look like the right direction, although the VPU uses > physical addresses directly instead of dmabuf FDs. I am currently > writing a frontend for the libfslvpuwrap, which takes care of certain > complexities (like being able to pass userdata pointers to frames, to > make it possible to associate input and output frames, which is > essential for correct timestamping, especially when things like h264 > frame reordering are active). This frontend will hide these things in a > future gstreamer-imx release to make the decoder code clearer and easier > to maintain, and is intended to be reusable, for example for Chromium > integration, or XBMC. Beyond that, my patches are probably not that > helpful anymore, since they were based on the vpx decoder way of > decoding, and I agree that the exynos code is a better starting point. > Nevertheless, I attached the patch. Keep in mind that it is very basic, > old, and wasn't further developed on because it was "good enough" for > the project. > > But here are some additional notes from our efforts to accelerate > Chromium on imx6 (keep in mind these apply to version 32.0.1664.3 ) : > > 1. We started the google-chrome binary with these switches: > --ignore-gpu-blacklist --enable-gpu --use-gl=egl > --enable-accelerated-2d-canvas > > 2. To make building Chromium easier, we switched to component build (by > default, it is all linked into one enormous binary). I attached a patch > that was necessary to fix some minor issues. Perhaps it is not necessary > anymore. > > 3. Chromium tried to load libEGL with dlopen() , which caused problems, > because with the Vivante libraries, libEGL also needs libGAL. Usually, > build scripts just add -lEGL -lGAL to the linker line. With dlopen() > this wasn't possible of course. We patched the gyp scripts to link > against this libraries during building instead. I attached this patch. > It is really a hack, because it circumvents the sandboxing process (this > is why Chromium loads EGL and GLESv2 with dlopen() ). > Mahyar updated these patches to apply against the chromium-35.0.1883.0 build currently in meta-browser. Additional notes to follow, but this appears to achieve HTML5 video against Webm/Ogg videos when chromium is started with these command line arguments: --ignore-gpu-blacklist --enable-gpu --usegl-egl Regards, Eric [-- Attachment #2: 0001-Modified-gl.gyp-to-link-libEGL-and-libGAL-at-build-t.patch --] [-- Type: text/x-diff, Size: 1025 bytes --] From 03924737a9378db934da9355bb38c916b5e3fa45 Mon Sep 17 00:00:00 2001 From: Eric Nelson <eric.nelson@boundarydevices.com> Date: Sun, 23 Mar 2014 07:10:36 -0700 Subject: [PATCH 1/3] Modified gl.gyp to link libEGL and libGAL at build time Signed-off-by: Eric Nelson <eric.nelson@boundarydevices.com> --- ui/gl/gl.gyp | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/ui/gl/gl.gyp b/ui/gl/gl.gyp index 2ec6d76..f7ee1a4 100644 --- a/ui/gl/gl.gyp +++ b/ui/gl/gl.gyp @@ -123,6 +123,12 @@ '<(gl_binding_output_dir)/gl_bindings_autogen_osmesa.cc', '<(gl_binding_output_dir)/gl_bindings_autogen_osmesa.h', ], + 'link_settings': { + 'libraries': [ + '-lEGL', + '-lGAL', + ], + }, # hard_dependency is necessary for this target because it has actions # that generate header files included by dependent targets. The header # files must be generated before the dependents are compiled. The usual -- 1.8.3.2 [-- Attachment #3: 0002-Added-hardware-accelerated-decoding-with-the-i.MX-VP.patch --] [-- Type: text/x-diff, Size: 29768 bytes --] From 44ccb7f8a920389897e60e30e3f0ee3319f7bf7d Mon Sep 17 00:00:00 2001 From: Eric Nelson <eric.nelson@boundarydevices.com> Date: Sun, 23 Mar 2014 07:17:08 -0700 Subject: [PATCH 2/3] Added hardware accelerated decoding with the i.MX VPU --- content/renderer/media/webmediaplayer_impl.cc | 5 + media/filters/vpu_framebuffers.cc | 106 +++++ media/filters/vpu_framebuffers.h | 36 ++ media/filters/vpu_video_decoder.cc | 614 ++++++++++++++++++++++++++ media/filters/vpu_video_decoder.h | 74 ++++ media/media.gyp | 28 ++ 6 files changed, 863 insertions(+) create mode 100644 media/filters/vpu_framebuffers.cc create mode 100644 media/filters/vpu_framebuffers.h create mode 100644 media/filters/vpu_video_decoder.cc create mode 100644 media/filters/vpu_video_decoder.h diff --git a/content/renderer/media/webmediaplayer_impl.cc b/content/renderer/media/webmediaplayer_impl.cc index b7094f6..7ac7318 100644 --- a/content/renderer/media/webmediaplayer_impl.cc +++ b/content/renderer/media/webmediaplayer_impl.cc @@ -57,6 +57,7 @@ #include "media/filters/opus_audio_decoder.h" #include "media/filters/video_renderer_impl.h" #include "media/filters/vpx_video_decoder.h" +#include "media/filters/vpu_video_decoder.h" #include "third_party/WebKit/public/platform/WebContentDecryptionModule.h" #include "third_party/WebKit/public/platform/WebMediaSource.h" #include "third_party/WebKit/public/platform/WebRect.h" @@ -1187,6 +1188,10 @@ void WebMediaPlayerImpl::StartPipeline() { new media::GpuVideoDecoder(gpu_factories_, media_log_)); } +#if !defined(MEDIA_DISABLE_IMXVPU) + video_decoders.push_back(new media::VpuVideoDecoder(media_loop_)); +#endif // !defined(MEDIA_DISABLE_IMXVPU) + #if !defined(MEDIA_DISABLE_LIBVPX) video_decoders.push_back(new media::VpxVideoDecoder(media_loop_)); #endif // !defined(MEDIA_DISABLE_LIBVPX) diff --git a/media/filters/vpu_framebuffers.cc b/media/filters/vpu_framebuffers.cc new file mode 100644 index 0000000..fce43fc --- /dev/null +++ b/media/filters/vpu_framebuffers.cc @@ -0,0 +1,106 @@ +#include <string.h> +#include <stdint.h> +#include "vpu_framebuffers.h" +#include "base/logging.h" + + +#define ALIGN_VAL_TO(LENGTH, ALIGN_SIZE) ( ((uintptr_t)((LENGTH) + (ALIGN_SIZE) - 1) / (ALIGN_SIZE)) * (ALIGN_SIZE) ) +#define FRAME_ALIGN 16 + + +namespace media { + + +VPUFramebuffers::VPUFramebuffers() { +} + + +void VPUFramebuffers::init(VpuDecHandle handle, VpuDecInitInfo const &init_info) { + framebuffers.resize(init_info.nMinFrameBufferCount); + + y_stride = ALIGN_VAL_TO(init_info.nPicWidth, FRAME_ALIGN); + if (init_info.nInterlace) + y_size = y_stride * ALIGN_VAL_TO(init_info.nPicHeight, (2 * FRAME_ALIGN)); + else + y_size = y_stride * ALIGN_VAL_TO(init_info.nPicHeight, FRAME_ALIGN); + + uv_stride = y_stride / 2; + u_size = v_size = mv_size = y_size / 4; + + int alignment = init_info.nAddressAlignment; + if (alignment > 1) + { + y_size = ALIGN_VAL_TO(y_size, alignment); + u_size = ALIGN_VAL_TO(u_size, alignment); + v_size = ALIGN_VAL_TO(v_size, alignment); + mv_size = ALIGN_VAL_TO(mv_size, alignment); + } + + pic_width = init_info.nPicWidth; + pic_height = init_info.nPicHeight; + + total_size = y_size + u_size + v_size + mv_size + alignment; + + LOG(INFO) << "Num VPU framebuffers: " << framebuffers.size(); + LOG(INFO) << "VPU framebuffer memory block size:" + << " total: " << total_size + << " Y " << y_size + << " U: " << u_size + << " V: " << v_size + << " Mv: " << mv_size + << " alignment: " << alignment + ; + + for (Framebuffers::iterator iter = framebuffers.begin(); iter != framebuffers.end(); ++iter) { + unsigned char *virt_ptr, *phys_ptr; + + VpuFrameBuffer &framebuffer = *iter; + + VpuMemDesc mem_desc; + memset(&mem_desc, 0, sizeof(VpuMemDesc)); + mem_desc.nSize = total_size; + VPU_DecGetMem(&mem_desc); + phys_mem_blocks.push_back(mem_desc); + + virt_ptr = (unsigned char*)(mem_desc.nVirtAddr); + phys_ptr = (unsigned char*)(mem_desc.nPhyAddr); + + if (alignment > 1) { + phys_ptr = (unsigned char*)ALIGN_VAL_TO(phys_ptr, alignment); + virt_ptr = (unsigned char*)ALIGN_VAL_TO(virt_ptr, alignment); + } + + framebuffer.nStrideY = y_stride; + framebuffer.nStrideC = uv_stride; + + /* fill phy addr*/ + framebuffer.pbufY = phys_ptr; + framebuffer.pbufCb = phys_ptr + y_size; + framebuffer.pbufCr = phys_ptr + y_size + u_size; + framebuffer.pbufMvCol = phys_ptr + y_size + u_size + v_size; + + /* fill virt addr */ + framebuffer.pbufVirtY = virt_ptr; + framebuffer.pbufVirtCb = virt_ptr + y_size; + framebuffer.pbufVirtCr = virt_ptr + y_size + u_size; + framebuffer.pbufVirtMvCol = virt_ptr + y_size + u_size + v_size; + + framebuffer.pbufY_tilebot = 0; + framebuffer.pbufCb_tilebot = 0; + framebuffer.pbufVirtY_tilebot = 0; + framebuffer.pbufVirtCb_tilebot = 0; + } + + VPU_DecRegisterFrameBuffer(handle, &(framebuffers[0]), framebuffers.size()); +} + + +VPUFramebuffers::~VPUFramebuffers() { + LOG(INFO) << "Freeing VPU framebuffer memory"; + for (PhysMemBlocks::iterator iter = phys_mem_blocks.begin(); iter != phys_mem_blocks.end(); ++iter) + VPU_DecFreeMem(&(*iter)); +} + + +} + diff --git a/media/filters/vpu_framebuffers.h b/media/filters/vpu_framebuffers.h new file mode 100644 index 0000000..bb9b021 --- /dev/null +++ b/media/filters/vpu_framebuffers.h @@ -0,0 +1,36 @@ +#ifndef MEDIA_FILTERS_VPU_FRAMEBUFFERS_H_ +#define MEDIA_FILTERS_VPU_FRAMEBUFFERS_H_ + +#include <list> +#include <vector> +#include <vpu_wrapper.h> + + +namespace media { + + +class VPUFramebuffers +{ +public: + explicit VPUFramebuffers(); + ~VPUFramebuffers(); + + void init(VpuDecHandle handle, VpuDecInitInfo const &init_info); + +private: + typedef std::vector < VpuFrameBuffer > Framebuffers; + typedef std::list < VpuMemDesc > PhysMemBlocks; + + PhysMemBlocks phys_mem_blocks; + Framebuffers framebuffers; + int y_stride, uv_stride; + int y_size, u_size, v_size, mv_size; + int total_size; + unsigned int pic_width, pic_height; +}; + + +} + + +#endif diff --git a/media/filters/vpu_video_decoder.cc b/media/filters/vpu_video_decoder.cc new file mode 100644 index 0000000..396d4f1 --- /dev/null +++ b/media/filters/vpu_video_decoder.cc @@ -0,0 +1,614 @@ +#include <list> +#include <vector> +#include <stddef.h> +#include <stdint.h> +#include <algorithm> +#include <string> + +#include <string.h> + +#include "media/filters/vpu_video_decoder.h" + +#include "base/bind.h" +#include "base/callback_helpers.h" +#include "base/command_line.h" +#include "base/location.h" +#include "base/logging.h" +#include "base/synchronization/lock.h" +#include "base/message_loop/message_loop_proxy.h" +#include "base/strings/string_number_conversions.h" +#include "base/sys_byteorder.h" +#include "media/base/bind_to_loop.h" +#include "media/base/decoder_buffer.h" +#include "media/base/demuxer_stream.h" +#include "media/base/media_switches.h" +#include "media/base/pipeline.h" +#include "media/base/video_decoder_config.h" +#include "media/base/video_frame.h" +#include "media/base/video_util.h" + +#include "media/filters/vpu_framebuffers.h" + +#include <vpu_wrapper.h> + + +#define ALIGN_VAL_TO(LENGTH, ALIGN_SIZE) ( ((uintptr_t)((LENGTH) + (ALIGN_SIZE) - 1) / (ALIGN_SIZE)) * (ALIGN_SIZE) ) + + + +namespace media { + + +namespace { + + +char const *VpuStrerror(VpuDecRetCode const code) { + switch (code) { + case VPU_DEC_RET_SUCCESS: return "success"; + case VPU_DEC_RET_FAILURE: return "failure"; + case VPU_DEC_RET_INVALID_PARAM: return "invalid param"; + case VPU_DEC_RET_INVALID_HANDLE: return "invalid handle"; + case VPU_DEC_RET_INVALID_FRAME_BUFFER: return "invalid frame buffer"; + case VPU_DEC_RET_INSUFFICIENT_FRAME_BUFFERS: return "insufficient frame buffers"; + case VPU_DEC_RET_INVALID_STRIDE: return "invalid stride"; + case VPU_DEC_RET_WRONG_CALL_SEQUENCE: return "wrong call sequence"; + case VPU_DEC_RET_FAILURE_TIMEOUT: return "failure timeout"; + default: + return NULL; + } +} + + +long inst_count = 0; +base::Lock vpu_load_lock; + + +void LoadDecoder() { + VpuDecRetCode ret; + + base::AutoLock alock(vpu_load_lock); + + if (inst_count == 0) { + ret = VPU_DecLoad(); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "VPU loading failed: " << VpuStrerror(ret); + return; + } + + } + + ++inst_count; + + if (inst_count == 1) { + VpuVersionInfo version; + VpuWrapperVersionInfo wrapper_version; + + ret = VPU_DecGetVersionInfo(&version); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Getting version info failed: " << VpuStrerror(ret); + return; + } + + ret = VPU_DecGetWrapperVersionInfo(&wrapper_version); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Getting wrapper version info failed: " << VpuStrerror(ret); + return; + } + + LOG(INFO) << "VPU Loaded"; + LOG(INFO) << "VPU firmware version " << version.nFwMajor << "." << version.nFwMinor << "." << version.nFwRelease << "_r" << version.nFwCode; + LOG(INFO) << "VPU library version " << version.nLibMajor << "." << version.nLibMinor << "." << version.nLibRelease; + LOG(INFO) << "VPU wrapper version " << wrapper_version.nMajor << "." << wrapper_version.nMinor << "." << wrapper_version.nRelease << " " << wrapper_version.pBinary; + } +} + + +void UnloadDecoder() { + base::AutoLock alock(vpu_load_lock); + + if (inst_count == 0) + return; + + --inst_count; + + if (inst_count == 0) + VPU_DecUnLoad(); +} + + +} + + +struct VpuVideoDecoder::Internal +{ + typedef std::vector < uint8_t > VirtMemBlock; + typedef std::list < VirtMemBlock > VirtMemBlocks; + + typedef std::list < VpuMemDesc > PhysMemBlocks; + + VpuDecHandle handle; + + VpuDecInitInfo init_info; + VpuMemInfo mem_info; + + bool vpu_opened; + + VirtMemBlocks virt_mem_blocks; + PhysMemBlocks phys_mem_blocks; + + VPUFramebuffers framebuffers; + + Internal() + : vpu_opened(false) + { + } + + ~Internal() + { + FreeMemBlocks(); + } + + bool AllocMemBlocks() + { + VpuDecRetCode ret; + + memset(&mem_info, 0, sizeof(VpuMemInfo)); + ret = VPU_DecQueryMem(&mem_info); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get VPU memory information: " << VpuStrerror(ret); + return false; + } + + for (int i = 0; i < mem_info.nSubBlockNum; ++i) + { + int size = mem_info.MemSubBlock[i].nAlignment + mem_info.MemSubBlock[i].nSize; + + LOG(INFO) << "Sub block " << i << " type: " << ((mem_info.MemSubBlock[i].MemType == VPU_MEM_VIRT) ? "virtual" : "phys") << " size: " << size; + + if (mem_info.MemSubBlock[i].MemType == VPU_MEM_VIRT) + { + unsigned char *virt_ptr; + virt_mem_blocks.push_back(VirtMemBlock()); + VirtMemBlock &blk = virt_mem_blocks.back(); + blk.resize(size); + virt_ptr = reinterpret_cast < unsigned char* > (&blk[0]); + + mem_info.MemSubBlock[i].pVirtAddr = (unsigned char *)ALIGN_VAL_TO(virt_ptr, mem_info.MemSubBlock[i].nAlignment); + } + else if (mem_info.MemSubBlock[i].MemType == VPU_MEM_PHY) + { + VpuMemDesc mem_desc; + memset(&mem_desc, 0, sizeof(VpuMemDesc)); + mem_desc.nSize = size; + if (VPU_DecGetMem(&mem_desc) != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Unable to allocate " << size << " bytes of physical memory"; + return false; + } + + phys_mem_blocks.push_back(mem_desc); + + mem_info.MemSubBlock[i].pVirtAddr = (unsigned char *)ALIGN_VAL_TO((unsigned char*)(mem_desc.nVirtAddr), mem_info.MemSubBlock[i].nAlignment); + mem_info.MemSubBlock[i].pPhyAddr = (unsigned char *)ALIGN_VAL_TO((unsigned char*)(mem_desc.nPhyAddr), mem_info.MemSubBlock[i].nAlignment); + } + else + LOG(WARNING) << "Sub block " << i << " type is unknown - skipping"; + } + + LOG(INFO) << "VPU memory blocks allocated"; + + return true; + } + + void FreeMemBlocks() + { + LOG(INFO) << "Freeing VPU memory blocks"; + + for (PhysMemBlocks::iterator iter = phys_mem_blocks.begin(); iter != phys_mem_blocks.end(); ++iter) + VPU_DecFreeMem(&(*iter)); + } +}; + + + +void CopyVpuFramebufferTo(VpuFrameBuffer *framebuffer, scoped_refptr<VideoFrame>* video_frame, VpuDecInitInfo &init_info, const VideoDecoderConfig& config); + + + + +VpuVideoDecoder::VpuVideoDecoder( + const scoped_refptr<base::MessageLoopProxy>& message_loop) + : message_loop_(message_loop), + weak_factory_(this), + state_(kUninitialized) { + LoadDecoder(); + internal = new Internal; +} + +VpuVideoDecoder::~VpuVideoDecoder() { + DCHECK_EQ(kUninitialized, state_); + CloseDecoder(); + delete internal; + UnloadDecoder(); +} + +void VpuVideoDecoder::Initialize(const VideoDecoderConfig& config, + const PipelineStatusCB& status_cb) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK(config.IsValidConfig()); + DCHECK(!config.is_encrypted()); + DCHECK(decode_cb_.is_null()); + DCHECK(reset_cb_.is_null()); + + weak_this_ = weak_factory_.GetWeakPtr(); + + if (!internal->AllocMemBlocks()) { + status_cb.Run(PIPELINE_ERROR_INITIALIZATION_FAILED); + return; + } + + if (!ConfigureDecoder(config)) { + status_cb.Run(DECODER_ERROR_NOT_SUPPORTED); + return; + } + + // Success! + LOG(INFO) << "VPU decoder initialization succeeded"; + config_ = config; + state_ = kNormal; + status_cb.Run(PIPELINE_OK); +} + +bool VpuVideoDecoder::ConfigureDecoder(const VideoDecoderConfig& config) { + bool can_handle = false; + + if ((config.format() == VideoFrame::YV12) || (config.format() == VideoFrame::YV12A) || (config.format() == VideoFrame::I420)) { + switch (config.codec()) { + case kCodecH264: + case kCodecVC1: + case kCodecMPEG2: + case kCodecMPEG4: + case kCodecVP8: + can_handle = true; + default: + break; + } + } + + if (!can_handle) { + LOG(INFO) << "Cannot handle video config: " << config.AsHumanReadableString(); + return false; + } + + CloseDecoder(); + return OpenDecoder(config); +} + +bool VpuVideoDecoder::OpenDecoder(const VideoDecoderConfig& config) { + if (internal->vpu_opened) + return true; + + VpuDecOpenParam open_param; + VpuDecRetCode ret; + + memset(&open_param, 0, sizeof(open_param)); + + switch (config.codec()) + { + case kCodecH264: + open_param.CodecFormat = VPU_V_AVC; + open_param.nReorderEnable = 1; + LOG(INFO) << "Setting h.264 as stream format"; + break; + case kCodecVC1: + open_param.CodecFormat = VPU_V_VC1_AP; + LOG(INFO) << "Setting VC1 as stream format"; + break; + case kCodecMPEG2: + open_param.CodecFormat = VPU_V_MPEG2; + LOG(INFO) << "Setting MPEG-2 as stream format"; + break; + case kCodecMPEG4: + open_param.CodecFormat = VPU_V_MPEG4; + LOG(INFO) << "Setting MPEG-4 as stream format"; + break; + case kCodecVP8: + open_param.CodecFormat = VPU_V_VP8; + LOG(INFO) << "Setting VP8 as stream format"; + break; + default: + LOG(ERROR) << "Could not set unknown stream format"; + break; + } + + open_param.nChromaInterleave = 0; + open_param.nMapType = 0; + open_param.nTiled2LinearEnable = 0; + open_param.nEnableFileMode = 0; + open_param.nPicWidth = config.coded_size().width(); + open_param.nPicHeight = config.coded_size().height(); + + ret = VPU_DecOpen(&(internal->handle), &open_param, &(internal->mem_info)); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Opening new VPU handle failed: " << VpuStrerror(ret); + return false; + } + + int config_param; + + config_param = VPU_DEC_SKIPNONE; + ret = VPU_DecConfig(internal->handle, VPU_DEC_CONF_SKIPMODE, &config_param); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not configure skip mode: " << VpuStrerror(ret); + return false; + } + + config_param = 0; + ret = VPU_DecConfig(internal->handle, VPU_DEC_CONF_BUFDELAY, &config_param); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not configure buffer delay: " << VpuStrerror(ret); + return false; + } + + config_param = VPU_DEC_IN_NORMAL; + ret = VPU_DecConfig(internal->handle, VPU_DEC_CONF_INPUTTYPE, &config_param); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not configure input type: " << VpuStrerror(ret); + return false; + } + + internal->vpu_opened = true; + + return true; +} + +void VpuVideoDecoder::CloseDecoder() { + VpuDecRetCode ret; + + if (!(internal->vpu_opened)) + return; + + ret = VPU_DecFlushAll(internal->handle); + if (ret != VPU_DEC_RET_SUCCESS) + LOG(ERROR) << "Flushing decoder failed: " << VpuStrerror(ret); + VPU_DecClose(internal->handle); + if (ret != VPU_DEC_RET_SUCCESS) + LOG(ERROR) << "Closing decoder failed: " << VpuStrerror(ret); + + internal->vpu_opened = false; +} + +void VpuVideoDecoder::Decode(const scoped_refptr<DecoderBuffer>& buffer, + const DecodeCB& decode_cb) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK(!decode_cb.is_null()); + CHECK_NE(state_, kUninitialized); + CHECK(decode_cb_.is_null()) << "Overlapping decodes are not supported."; + + decode_cb_ = BindToCurrentLoop(decode_cb); + + if (state_ == kError) { + base::ResetAndReturn(&decode_cb_).Run(kDecodeError, NULL); + return; + } + + // Return empty frames if decoding has finished. + if (state_ == kDecodeFinished) { + base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEmptyFrame()); + return; + } + + DecodeBuffer(buffer); +} + +void VpuVideoDecoder::Reset(const base::Closure& closure) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK(reset_cb_.is_null()); + reset_cb_ = BindToCurrentLoop(closure); + + // Defer the reset if a decode is pending. + if (!decode_cb_.is_null()) + return; + + DoReset(); +} + +void VpuVideoDecoder::Stop(const base::Closure& closure) { + DCHECK(message_loop_->BelongsToCurrentThread()); + base::ScopedClosureRunner runner(BindToCurrentLoop(closure)); + + if (state_ == kUninitialized) + return; + + if (!decode_cb_.is_null()) { + base::ResetAndReturn(&decode_cb_).Run(kOk, NULL); + // Reset is pending only when decode is pending. + if (!reset_cb_.is_null()) + base::ResetAndReturn(&reset_cb_).Run(); + } + + state_ = kUninitialized; +} + +bool VpuVideoDecoder::HasAlpha() const { + return false; +} + +void VpuVideoDecoder::DecodeBuffer(const scoped_refptr<DecoderBuffer>& buffer) { + DCHECK(message_loop_->BelongsToCurrentThread()); + DCHECK_NE(state_, kUninitialized); + DCHECK_NE(state_, kDecodeFinished); + DCHECK_NE(state_, kError); + DCHECK(reset_cb_.is_null()); + DCHECK(!decode_cb_.is_null()); + DCHECK(buffer); + + // Transition to kDecodeFinished on the first end of stream buffer. + if (state_ == kNormal && buffer->end_of_stream()) { + state_ = kDecodeFinished; + base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEmptyFrame()); + return; + } + + scoped_refptr<VideoFrame> video_frame; + if (!VpuDecode(buffer, &video_frame)) { + state_ = kError; + base::ResetAndReturn(&decode_cb_).Run(kDecodeError, NULL); + return; + } + + // If we didn't get a frame we need more data. + if (!video_frame.get()) { + base::ResetAndReturn(&decode_cb_).Run(kNotEnoughData, NULL); + return; + } + + base::ResetAndReturn(&decode_cb_).Run(kOk, video_frame); +} + +bool VpuVideoDecoder::VpuDecode(const scoped_refptr<DecoderBuffer>& buffer, + scoped_refptr<VideoFrame>* video_frame) { + DCHECK(video_frame); + DCHECK(!buffer->end_of_stream()); + + int buffer_ret_code; + VpuBufferNode in_data; + VpuDecRetCode ret; + + memset(&in_data, 0, sizeof(in_data)); + in_data.pVirAddr = (unsigned char *)(buffer->data()); + in_data.nSize = buffer->data_size(); + + ret = VPU_DecDecodeBuf(internal->handle, &in_data, &buffer_ret_code); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Failed to decode frame: " << VpuStrerror(ret); + return false; + } + + VLOG(1) << "VPU_DecDecodeBuf buffer ret code is 0x" << std::hex << buffer_ret_code << std::dec; + + if (buffer_ret_code & VPU_DEC_INIT_OK) { + ret = VPU_DecGetInitialInfo(internal->handle, &(internal->init_info)); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get init info: " << VpuStrerror(ret); + return false; + } + internal->framebuffers.init(internal->handle, internal->init_info); + } + + if (buffer_ret_code & VPU_DEC_FLUSH) { + ret = VPU_DecFlushAll(internal->handle); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Flushing VPU failed: " << VpuStrerror(ret); + return false; + } + + *video_frame = NULL; + + return true; + } + + if (buffer_ret_code & VPU_DEC_NO_ENOUGH_INBUF) { + LOG(INFO) << "Need more input"; + return true; + } + + if (buffer_ret_code & VPU_DEC_ONE_FRM_CONSUMED) { + VpuDecFrameLengthInfo dec_framelen_info; + + ret = VPU_DecGetConsumedFrameInfo(internal->handle, &dec_framelen_info); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get information about consumed frame: " << VpuStrerror(ret); + } else { + LOG(INFO) << "One frame got consumed:" + << " framebuffer addr: 0x" << std::hex << uintptr_t(dec_framelen_info.pFrame) << std::dec + << " stuff length: " << dec_framelen_info.nStuffLength + << " frame length: " << dec_framelen_info.nFrameLength + ; + } + } + + if (buffer_ret_code & VPU_DEC_OUTPUT_DIS) { + VpuDecOutFrameInfo out_frame_info; + int64 timestamp = buffer->timestamp().InMicroseconds(); + + ret = VPU_DecGetOutputFrame(internal->handle, &out_frame_info); + if (ret != VPU_DEC_RET_SUCCESS) { + LOG(ERROR) << "Could not get decoded output frame: " << VpuStrerror(ret); + return false; + } + + LOG(INFO) << "Output frame:" + << " pic width: " << internal->init_info.nPicWidth + << " pic height: " << internal->init_info.nPicHeight + << " pic type: " << out_frame_info.ePicType + << " Y stride: " << out_frame_info.pDisplayFrameBuf->nStrideY + << " CbCr stride: " << out_frame_info.pDisplayFrameBuf->nStrideC + ; + + CopyVpuFramebufferTo(out_frame_info.pDisplayFrameBuf, video_frame, internal->init_info, config_); + (*video_frame)->SetTimestamp(base::TimeDelta::FromMicroseconds(timestamp)); + + ret = VPU_DecOutFrameDisplayed(internal->handle, out_frame_info.pDisplayFrameBuf); + if (ret != VPU_DEC_RET_SUCCESS) + LOG(ERROR) << "Clearing display framebuffer failed: " << VpuStrerror(ret); + } + else + *video_frame = NULL; + + return true; +} + +void VpuVideoDecoder::DoReset() { + DCHECK(decode_cb_.is_null()); + + state_ = kNormal; + reset_cb_.Run(); + reset_cb_.Reset(); +} + +void CopyVpuFramebufferTo(VpuFrameBuffer *framebuffer, scoped_refptr<VideoFrame>* video_frame, VpuDecInitInfo &init_info, const VideoDecoderConfig& config) { + CHECK(framebuffer); + + gfx::Size size(init_info.nPicWidth, init_info.nPicHeight); + + *video_frame = VideoFrame::CreateFrame( + config.format(), + size, + gfx::Rect(size), + config.natural_size(), + kNoTimestamp()); + + CopyYPlane(framebuffer->pbufVirtY, + framebuffer->nStrideY, + init_info.nPicHeight, + video_frame->get()); + if (config.format() == VideoFrame::I420) { + CopyUPlane(framebuffer->pbufVirtCb, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + CopyVPlane(framebuffer->pbufVirtCr, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + } else { /* YV12 and YV12A */ + /* U and V swapped */ + CopyUPlane(framebuffer->pbufVirtCb, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + CopyVPlane(framebuffer->pbufVirtCr, + framebuffer->nStrideC, + init_info.nPicHeight / 2, + video_frame->get()); + + if (config.format() == VideoFrame::YV12A) { + MakeOpaqueAPlane(framebuffer->nStrideY, + init_info.nPicHeight, + video_frame->get()); + } + } +} + + +} // namespace media diff --git a/media/filters/vpu_video_decoder.h b/media/filters/vpu_video_decoder.h new file mode 100644 index 0000000..d7016aa --- /dev/null +++ b/media/filters/vpu_video_decoder.h @@ -0,0 +1,74 @@ +#ifndef MEDIA_FILTERS_VPU_VIDEO_DECODER_H_ +#define MEDIA_FILTERS_VPU_VIDEO_DECODER_H_ + +#include "base/callback.h" +#include "base/memory/weak_ptr.h" +#include "media/base/demuxer_stream.h" +#include "media/base/video_decoder.h" +#include "media/base/video_decoder_config.h" +#include "media/base/video_frame.h" + +namespace base { +class MessageLoopProxy; +} + +namespace media { + +class MEDIA_EXPORT VpuVideoDecoder : public VideoDecoder { + public: + explicit VpuVideoDecoder( + const scoped_refptr<base::MessageLoopProxy>& message_loop); + virtual ~VpuVideoDecoder(); + + // VideoDecoder implementation. + virtual void Initialize(const VideoDecoderConfig& config, + const PipelineStatusCB& status_cb) OVERRIDE; + virtual void Decode(const scoped_refptr<DecoderBuffer>& buffer, + const DecodeCB& decode_cb) OVERRIDE; + virtual void Reset(const base::Closure& closure) OVERRIDE; + virtual void Stop(const base::Closure& closure) OVERRIDE; + virtual bool HasAlpha() const OVERRIDE; + + private: + enum DecoderState { + kUninitialized, + kNormal, + kFlushCodec, + kDecodeFinished, + kError + }; + + // Handles (re-)initializing the decoder with a (new) config. + // Returns true when initialization was successful. + bool ConfigureDecoder(const VideoDecoderConfig& config); + + bool OpenDecoder(const VideoDecoderConfig& config); + void CloseDecoder(); + + void DecodeBuffer(const scoped_refptr<DecoderBuffer>& buffer); + bool VpuDecode(const scoped_refptr<DecoderBuffer>& buffer, + scoped_refptr<VideoFrame>* video_frame); + + // Reset decoder and call |reset_cb_|. + void DoReset(); + + scoped_refptr<base::MessageLoopProxy> message_loop_; + base::WeakPtrFactory<VpuVideoDecoder> weak_factory_; + base::WeakPtr<VpuVideoDecoder> weak_this_; + + DecoderState state_; + + DecodeCB decode_cb_; + base::Closure reset_cb_; + + VideoDecoderConfig config_; + + struct Internal; + Internal *internal; + + DISALLOW_COPY_AND_ASSIGN(VpuVideoDecoder); +}; + +} // namespace media + +#endif // MEDIA_FILTERS_VPU_VIDEO_DECODER_H_ diff --git a/media/media.gyp b/media/media.gyp index 4782741..ec53626 100644 --- a/media/media.gyp +++ b/media/media.gyp @@ -11,6 +11,7 @@ # (DT_NEEDED) instead of using dlopen. This helps with automated # detection of ABI mismatches and prevents silent errors. 'linux_link_pulseaudio%': 0, + 'media_use_imxvpu%': 1, 'conditions': [ ['OS=="android"', { # Android doesn't use ffmpeg. @@ -404,6 +405,10 @@ 'filters/video_frame_painter.h', 'filters/video_renderer_impl.cc', 'filters/video_renderer_impl.h', + 'filters/vpu_framebuffers.cc', + 'filters/vpu_framebuffers.h', + 'filters/vpu_video_decoder.cc', + 'filters/vpu_video_decoder.h', 'filters/vpx_video_decoder.cc', 'filters/vpx_video_decoder.h', 'filters/webvtt_util.h', @@ -658,6 +663,29 @@ 'DISABLE_USER_INPUT_MONITOR', ], }], + ['media_use_imxvpu==1', { + 'cflags': [ + '<!@(<(pkg-config) --cflags libfslvpuwrap)', + ], + 'link_settings': { + 'libraries': [ + '<!@(<(pkg-config) --libs libfslvpuwrap)', + ], + }, + }, { # media_use_imxvpu==0 + 'direct_dependent_settings': { + 'defines': [ + 'MEDIA_DISABLE_IMXVPU', + ], + }, + # Exclude the VPU sources. + 'sources!': [ + 'filters/vpu_framebuffers.cc', + 'filters/vpu_framebuffers.h', + 'filters/vpu_video_decoder.cc', + 'filters/vpu_video_decoder.h', + ], + }], ['use_cras==1', { 'cflags': [ '<!@(<(pkg-config) --cflags libcras)', -- 1.8.3.2 [-- Attachment #4: 0003-Update-vpu_video_decoder.cc-to-match-chromium-35.0.1883.patch --] [-- Type: text/x-diff, Size: 1813 bytes --] From f059d6501b17ab8aed618f53f7e4255cf8d8326b Mon Sep 17 00:00:00 2001 From: Mahyar Yaghmaee <mahyar@boundarydevices.com> Date: Fri, 28 Mar 2014 14:06:40 -0700 Subject: [PATCH 3/3] Update vpu_video_decoder.cc to match chromium-35.0.1883 Signed-off-by: Mahyar Yaghmaee <mahyar@boundarydevices.com> --- media/filters/vpu_video_decoder.cc | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/media/filters/vpu_video_decoder.cc b/media/filters/vpu_video_decoder.cc index 396d4f1..9024484 100644 --- a/media/filters/vpu_video_decoder.cc +++ b/media/filters/vpu_video_decoder.cc @@ -18,7 +18,7 @@ #include "base/message_loop/message_loop_proxy.h" #include "base/strings/string_number_conversions.h" #include "base/sys_byteorder.h" -#include "media/base/bind_to_loop.h" +#include "media/base/bind_to_current_loop.h" #include "media/base/decoder_buffer.h" #include "media/base/demuxer_stream.h" #include "media/base/media_switches.h" @@ -393,7 +393,7 @@ void VpuVideoDecoder::Decode(const scoped_refptr<DecoderBuffer>& buffer, // Return empty frames if decoding has finished. if (state_ == kDecodeFinished) { - base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEmptyFrame()); + base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEOSFrame()); return; } @@ -445,7 +445,7 @@ void VpuVideoDecoder::DecodeBuffer(const scoped_refptr<DecoderBuffer>& buffer) { // Transition to kDecodeFinished on the first end of stream buffer. if (state_ == kNormal && buffer->end_of_stream()) { state_ = kDecodeFinished; - base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEmptyFrame()); + base::ResetAndReturn(&decode_cb_).Run(kOk, VideoFrame::CreateEOSFrame()); return; } -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-04-01 19:22 ` Eric Nelson @ 2014-04-02 10:21 ` Carlos Rafael Giani 2014-04-02 10:28 ` Gary Thomas [not found] ` <533BE4A3.7040301@pr.hu> 2014-04-02 10:29 ` Boszormenyi Zoltan 2 siblings, 1 reply; 37+ messages in thread From: Carlos Rafael Giani @ 2014-04-02 10:21 UTC (permalink / raw) To: Eric Nelson, Christian Betz, meta-freescale Keep in mind what I wrote. This version of VPU acceleration is very basic (it will copy frames with the CPU), and fulfilled the customer's immediate needs back then, but can be done much better. I am currently looking into a better approach. On 2014-04-01 21:22, Eric Nelson wrote: > Thanks again Carlos, > > On 03/20/2014 04:19 PM, Carlos Rafael Giani wrote: > > > > <snip> >> >> Back then we needed some hw decoding quickly, so we did not look further >> into this, since we had spent a lot of time getting the 2D acceleration >> stable already. I could only briefly look at the exynos accelerator >> code, but it does look like the right direction, although the VPU uses >> physical addresses directly instead of dmabuf FDs. I am currently >> writing a frontend for the libfslvpuwrap, which takes care of certain >> complexities (like being able to pass userdata pointers to frames, to >> make it possible to associate input and output frames, which is >> essential for correct timestamping, especially when things like h264 >> frame reordering are active). This frontend will hide these things in a >> future gstreamer-imx release to make the decoder code clearer and easier >> to maintain, and is intended to be reusable, for example for Chromium >> integration, or XBMC. Beyond that, my patches are probably not that >> helpful anymore, since they were based on the vpx decoder way of >> decoding, and I agree that the exynos code is a better starting point. >> Nevertheless, I attached the patch. Keep in mind that it is very basic, >> old, and wasn't further developed on because it was "good enough" for >> the project. >> >> But here are some additional notes from our efforts to accelerate >> Chromium on imx6 (keep in mind these apply to version 32.0.1664.3 ) : >> >> 1. We started the google-chrome binary with these switches: >> --ignore-gpu-blacklist --enable-gpu --use-gl=egl >> --enable-accelerated-2d-canvas >> >> 2. To make building Chromium easier, we switched to component build (by >> default, it is all linked into one enormous binary). I attached a patch >> that was necessary to fix some minor issues. Perhaps it is not necessary >> anymore. >> >> 3. Chromium tried to load libEGL with dlopen() , which caused problems, >> because with the Vivante libraries, libEGL also needs libGAL. Usually, >> build scripts just add -lEGL -lGAL to the linker line. With dlopen() >> this wasn't possible of course. We patched the gyp scripts to link >> against this libraries during building instead. I attached this patch. >> It is really a hack, because it circumvents the sandboxing process (this >> is why Chromium loads EGL and GLESv2 with dlopen() ). >> > > Mahyar updated these patches to apply against the chromium-35.0.1883.0 > build currently in meta-browser. > > Additional notes to follow, but this appears to achieve HTML5 video > against Webm/Ogg videos when chromium is started with these command > line arguments: > --ignore-gpu-blacklist --enable-gpu --usegl-egl > > Regards, > > > Eric > ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-04-02 10:21 ` Carlos Rafael Giani @ 2014-04-02 10:28 ` Gary Thomas 2014-04-02 10:33 ` Carlos Rafael Giani 0 siblings, 1 reply; 37+ messages in thread From: Gary Thomas @ 2014-04-02 10:28 UTC (permalink / raw) To: meta-freescale On 2014-04-02 04:21, Carlos Rafael Giani wrote: > Keep in mind what I wrote. This version of VPU acceleration is very basic (it will copy frames with the CPU), and fulfilled the customer's immediate needs back then, but can be > done much better. I am currently looking into a better approach. > > On 2014-04-01 21:22, Eric Nelson wrote: >> Thanks again Carlos, >> >> On 03/20/2014 04:19 PM, Carlos Rafael Giani wrote: >> > >> > <snip> >>> >>> Back then we needed some hw decoding quickly, so we did not look further >>> into this, since we had spent a lot of time getting the 2D acceleration >>> stable already. I could only briefly look at the exynos accelerator >>> code, but it does look like the right direction, although the VPU uses >>> physical addresses directly instead of dmabuf FDs. I am currently >>> writing a frontend for the libfslvpuwrap, which takes care of certain >>> complexities (like being able to pass userdata pointers to frames, to >>> make it possible to associate input and output frames, which is >>> essential for correct timestamping, especially when things like h264 >>> frame reordering are active). This frontend will hide these things in a >>> future gstreamer-imx release to make the decoder code clearer and easier >>> to maintain, and is intended to be reusable, for example for Chromium >>> integration, or XBMC. Beyond that, my patches are probably not that >>> helpful anymore, since they were based on the vpx decoder way of >>> decoding, and I agree that the exynos code is a better starting point. >>> Nevertheless, I attached the patch. Keep in mind that it is very basic, >>> old, and wasn't further developed on because it was "good enough" for >>> the project. >>> >>> But here are some additional notes from our efforts to accelerate >>> Chromium on imx6 (keep in mind these apply to version 32.0.1664.3 ) : >>> >>> 1. We started the google-chrome binary with these switches: >>> --ignore-gpu-blacklist --enable-gpu --use-gl=egl >>> --enable-accelerated-2d-canvas >>> >>> 2. To make building Chromium easier, we switched to component build (by >>> default, it is all linked into one enormous binary). I attached a patch >>> that was necessary to fix some minor issues. Perhaps it is not necessary >>> anymore. >>> >>> 3. Chromium tried to load libEGL with dlopen() , which caused problems, >>> because with the Vivante libraries, libEGL also needs libGAL. Usually, >>> build scripts just add -lEGL -lGAL to the linker line. With dlopen() >>> this wasn't possible of course. We patched the gyp scripts to link >>> against this libraries during building instead. I attached this patch. >>> It is really a hack, because it circumvents the sandboxing process (this >>> is why Chromium loads EGL and GLESv2 with dlopen() ). >>> >> >> Mahyar updated these patches to apply against the chromium-35.0.1883.0 >> build currently in meta-browser. Are these patches available somewhere? >> Additional notes to follow, but this appears to achieve HTML5 video >> against Webm/Ogg videos when chromium is started with these command >> line arguments: >> --ignore-gpu-blacklist --enable-gpu --usegl-egl -- ------------------------------------------------------------ Gary Thomas | Consulting for the MLB Associates | Embedded world ------------------------------------------------------------ ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-04-02 10:28 ` Gary Thomas @ 2014-04-02 10:33 ` Carlos Rafael Giani 0 siblings, 0 replies; 37+ messages in thread From: Carlos Rafael Giani @ 2014-04-02 10:33 UTC (permalink / raw) To: meta-freescale On 2014-04-02 12:28, Gary Thomas wrote: > On 2014-04-02 04:21, Carlos Rafael Giani wrote: >> Keep in mind what I wrote. This version of VPU acceleration is very basic (it will copy frames with the CPU), and fulfilled the customer's immediate needs back then, but can be >> done much better. I am currently looking into a better approach. > Are these patches available somewhere? No, not yet, there is no finished code at the moment. If I am right, the necessary changes are not big though. The actual bottleneck here is time :) ^ permalink raw reply [flat|nested] 37+ messages in thread
[parent not found: <533BE4A3.7040301@pr.hu>]
* Re: Chromium acceleration [not found] ` <533BE4A3.7040301@pr.hu> @ 2014-04-02 10:23 ` Carlos Rafael Giani 2014-04-02 11:12 ` Boszormenyi Zoltan 2014-04-02 12:02 ` Christian Betz 0 siblings, 2 replies; 37+ messages in thread From: Carlos Rafael Giani @ 2014-04-02 10:23 UTC (permalink / raw) To: Boszormenyi Zoltan, Eric Nelson, Christian Betz, meta-freescale On 2014-04-02 12:21, Boszormenyi Zoltan wrote: > Hi, > > 2014-04-01 21:22 keltezéssel, Eric Nelson írta: >> >> Mahyar updated these patches to apply against the chromium-35.0.1883.0 >> build currently in meta-browser. >> >> Additional notes to follow, but this appears to achieve HTML5 video >> against Webm/Ogg videos when chromium is started with these command >> line arguments: >> --ignore-gpu-blacklist --enable-gpu --usegl-egl > > you meant "--use-gl=egl". > > I have rebuilt Chromium with these patches. The result cannot play a > short MP4 video from a file:// URL. > The console output is very suspicious, libGAL complains about a > missing ID file that actually exists. > The image about the console is attached. > > Best regards, > Zoltán Böszörményi > The VPU part could be because of missing firmware. Check if the vpu files are present in /lib/firmware. As for MP4, this is a known problem. You are building Chromium, not Chrome. MP4 support is part of the restricted feature set, which is included in Chrome but not Chromium. Try a WebM file for example. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-04-02 10:23 ` Carlos Rafael Giani @ 2014-04-02 11:12 ` Boszormenyi Zoltan 2014-04-02 12:02 ` Christian Betz 1 sibling, 0 replies; 37+ messages in thread From: Boszormenyi Zoltan @ 2014-04-02 11:12 UTC (permalink / raw) To: Carlos Rafael Giani, Eric Nelson, Christian Betz, meta-freescale 2014-04-02 12:23 keltezéssel, Carlos Rafael Giani írta: > On 2014-04-02 12:21, Boszormenyi Zoltan wrote: >> Hi, >> >> 2014-04-01 21:22 keltezéssel, Eric Nelson írta: >>> >>> Mahyar updated these patches to apply against the chromium-35.0.1883.0 >>> build currently in meta-browser. >>> >>> Additional notes to follow, but this appears to achieve HTML5 video >>> against Webm/Ogg videos when chromium is started with these command >>> line arguments: >>> --ignore-gpu-blacklist --enable-gpu --usegl-egl >> >> you meant "--use-gl=egl". >> >> I have rebuilt Chromium with these patches. The result cannot play a short MP4 video >> from a file:// URL. >> The console output is very suspicious, libGAL complains about a missing ID file that >> actually exists. >> The image about the console is attached. >> >> Best regards, >> Zoltán Böszörményi >> > > The VPU part could be because of missing firmware. Check if the vpu files are present in > /lib/firmware. I have /lib/firmware/vpu/vpu_fw_imxq6.bin, size 253968, the last 6 digits of md5sum is e8debf. > As for MP4, this is a known problem. You are building Chromium, not Chrome. MP4 support > is part of the restricted feature set, which is included in Chrome but not Chromium. Try > a WebM file for example. I tried these: 1. http://www.webmfiles.org/demo-files/ The webm examples play nicely, even scaled to full screen. 2. http://devfiles.myopera.com/articles/1891/custom-controls-webm-720p.html It's the same "Elephants Dream" as one of the examples in (1) but at 720p. It pays nicely in the browser control but it's a slideshow as full screen. On the other hand, gstreamer 0.10 with gst-fsl-plugin and its gst-fsl-plugin-gplay subpackage, gplay plays a 720p and a 1080p H.264 test file very well without tearing. Best regards, Zoltán Böszörményi ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-04-02 10:23 ` Carlos Rafael Giani 2014-04-02 11:12 ` Boszormenyi Zoltan @ 2014-04-02 12:02 ` Christian Betz 2014-04-02 12:28 ` Carlos Rafael Giani 1 sibling, 1 reply; 37+ messages in thread From: Christian Betz @ 2014-04-02 12:02 UTC (permalink / raw) To: Carlos Rafael Giani; +Cc: meta-freescale [-- Attachment #1: Type: text/plain, Size: 758 bytes --] > > The VPU part could be because of missing firmware. Check if the vpu files >> are present in /lib/firmware. >> > As for MP4, this is a known problem. You are building Chromium, not > Chrome. MP4 support is part of the restricted feature set, which is > included in Chrome but not Chromium. Try a WebM file for example. > this apparently can be worked around with gyp options: "proprietary_codecs=1 ffmpeg_branding=Chrome branding=Chrome to allow Chrome to play h.264 content, which is the only codec VAVDA knows about today." this is described on a wiki page setting up hw video decode on **intel** processors: https://code.google.com/p/chromium/wiki/LinuxHWVideoDecode note: i haven't actually tried this! (but i would like to) [-- Attachment #2: Type: text/html, Size: 1408 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-04-02 12:02 ` Christian Betz @ 2014-04-02 12:28 ` Carlos Rafael Giani 2014-04-04 9:28 ` Boszormenyi Zoltan 0 siblings, 1 reply; 37+ messages in thread From: Carlos Rafael Giani @ 2014-04-02 12:28 UTC (permalink / raw) To: Christian Betz; +Cc: meta-freescale [-- Attachment #1: Type: text/plain, Size: 1060 bytes --] On 2014-04-02 14:02, Christian Betz wrote: > > The VPU part could be because of missing firmware. Check if > the vpu files are present in /lib/firmware. > > As for MP4, this is a known problem. You are building Chromium, > not Chrome. MP4 support is part of the restricted feature set, > which is included in Chrome but not Chromium. Try a WebM file for > example. > > > this apparently can be worked around with gyp options: > > "proprietary_codecs=1 ffmpeg_branding=Chrome branding=Chrome to allow > Chrome to play h.264 content, which is the only codec VAVDA knows > about today." > > this is described on a wiki page setting up hw video decode on > **intel** processors: > > https://code.google.com/p/chromium/wiki/LinuxHWVideoDecode > > note: i haven't actually tried this! (but i would like to) We tried that back then. The Chrome branding enabled a million other things , which caused all sorts of difficulties. Also note that enabling the Chrome branding might have legal repercussions. [-- Attachment #2: Type: text/html, Size: 2597 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-04-02 12:28 ` Carlos Rafael Giani @ 2014-04-04 9:28 ` Boszormenyi Zoltan 0 siblings, 0 replies; 37+ messages in thread From: Boszormenyi Zoltan @ 2014-04-04 9:28 UTC (permalink / raw) To: Carlos Rafael Giani, Christian Betz; +Cc: meta-freescale [-- Attachment #1: Type: text/plain, Size: 1796 bytes --] 2014-04-02 14:28 keltezéssel, Carlos Rafael Giani írta: > On 2014-04-02 14:02, Christian Betz wrote: >> >> The VPU part could be because of missing firmware. Check if the vpu files are >> present in /lib/firmware. >> >> As for MP4, this is a known problem. You are building Chromium, not Chrome. MP4 >> support is part of the restricted feature set, which is included in Chrome but not >> Chromium. Try a WebM file for example. >> >> >> this apparently can be worked around with gyp options: >> >> "proprietary_codecs=1 ffmpeg_branding=Chrome branding=Chrome to allow Chrome to play >> h.264 content, which is the only codec VAVDA knows about today." >> >> this is described on a wiki page setting up hw video decode on **intel** processors: >> >> https://code.google.com/p/chromium/wiki/LinuxHWVideoDecode >> >> note: i haven't actually tried this! (but i would like to) > > We tried that back then. The Chrome branding enabled a million other things , which > caused all sorts of difficulties. Also note that enabling the Chrome branding might have > legal repercussions. A comment on that page says to use these flags (Chrome branding is not needed, only chromeos=1): export GYP_DEFINES="chromeos=1 proprietary_codecs=1 ffmpeg_branding=Chrome" do_configure succeeds but do_compile fails with "brlapi.h" missing. A google search gave me this: https://groups.google.com/a/chromium.org/forum/#!topic/chromium-dev/StpTzFnf8ok Supposedly, running "build/install-build-deps.sh" fixes it but it turned out it only fixes things for building Chromium for the host and the script wants an Ubuntu host OS. Is there a ready to use recipe for brlapi/brltty somewhere or should I make one? Thanks in advance, Zoltán Böszörményi [-- Attachment #2: Type: text/html, Size: 3956 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-04-01 19:22 ` Eric Nelson 2014-04-02 10:21 ` Carlos Rafael Giani [not found] ` <533BE4A3.7040301@pr.hu> @ 2014-04-02 10:29 ` Boszormenyi Zoltan 2014-04-02 14:16 ` Eric Nelson 2 siblings, 1 reply; 37+ messages in thread From: Boszormenyi Zoltan @ 2014-04-02 10:29 UTC (permalink / raw) To: Eric Nelson, Carlos Rafael Giani, Christian Betz, meta-freescale [-- Attachment #1: Type: text/plain, Size: 877 bytes --] Hi, 2014-04-01 21:22 keltezéssel, Eric Nelson írta: > > Mahyar updated these patches to apply against the chromium-35.0.1883.0 > build currently in meta-browser. > > Additional notes to follow, but this appears to achieve HTML5 video > against Webm/Ogg videos when chromium is started with these command > line arguments: > --ignore-gpu-blacklist --enable-gpu --usegl-egl you meant "--use-gl=egl". I have rebuilt Chromium with these patches. The result cannot play a short MP4 video from a file:// URL. The console output is very suspicious, libGAL complains about a missing ID file that actually exists. The image about the console is attached. This is a copy of the original message, I cancelled waiting for moderation. The original image exceeded the list limits so I reduced it to 16-color greyscale. Best regards, Zoltán Böszörményi [-- Attachment #2: 20140402_121516_800x600.png --] [-- Type: image/png, Size: 113651 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-04-02 10:29 ` Boszormenyi Zoltan @ 2014-04-02 14:16 ` Eric Nelson 0 siblings, 0 replies; 37+ messages in thread From: Eric Nelson @ 2014-04-02 14:16 UTC (permalink / raw) To: Boszormenyi Zoltan, Carlos Rafael Giani, Christian Betz, meta-freescale Hi Zoltan, On 04/02/2014 03:29 AM, Boszormenyi Zoltan wrote: > Hi, > > 2014-04-01 21:22 keltezéssel, Eric Nelson írta: >> >> Mahyar updated these patches to apply against the chromium-35.0.1883.0 >> build currently in meta-browser. >> >> Additional notes to follow, but this appears to achieve HTML5 video >> against Webm/Ogg videos when chromium is started with these command >> line arguments: >> --ignore-gpu-blacklist --enable-gpu --usegl-egl > > you meant "--use-gl=egl". > Probably right. > I have rebuilt Chromium with these patches. The result cannot play a > short MP4 video from a file:// URL. > The console output is very suspicious, libGAL complains about a missing > ID file that actually exists. > The image about the console is attached. > > This is a copy of the original message, I cancelled waiting for moderation. > The original image exceeded the list limits so I reduced it to 16-color > greyscale. > There's a lesson here that you should be putting console output either directly into your e-mail (trimmed to important points). Putting the output into an image makes it slow to read and tedious to quote. From what I can tell, you aren't running on an image that otherwise supports GPU or VPU acceleration. That's a pre-requisite for this, and I recommend using fsl-image-gui or some other known good starting point. Also note that we sent these patches not to imply that they're fully functional, but to allow others to have a working example for exploring the code. Regards, Eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-19 20:49 Chromium acceleration Eric Nelson 2014-03-19 21:00 ` Carlos Rafael Giani @ 2014-03-20 12:47 ` Lauren Post 2014-03-20 14:25 ` Eric Nelson 1 sibling, 1 reply; 37+ messages in thread From: Lauren Post @ 2014-03-20 12:47 UTC (permalink / raw) To: Eric Nelson, meta-freescale My original statement about acceleration was for webGL in the browser. ffmpeg-mt is not hardware accelerated with the VPU although it is possible to update it to integrate with VPU. As other emails have mentioned removing data movement and the CSC conversion is the key to improving performance with ffmpeg with vpu acceleration. It is not something we have in our upcoming release. Lauren -----Original Message----- From: Eric Nelson [mailto:eric.nelson@boundarydevices.com] Sent: Wednesday, March 19, 2014 3:50 PM To: Post Lauren-RAA013; meta-freescale@yoctoproject.org Subject: Chromium acceleration Hi Lauren, A while back, you mentioned that 3.10.17-ga would help provide support for "Chromium acceleration" but didn't provide details. Did you mean WebGL support (i.e. GPU acceleration), or video acceleration (VPU support)? My understanding is that Chromium is using ffmpeg-mt as the basis for accelerated video: http://www.chromium.org/developers/design-documents/video Is anyone currently working on support for ffmpeg-mt and i.MX video? Is anyone interested? Please advise, Eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 12:47 ` Lauren Post @ 2014-03-20 14:25 ` Eric Nelson 2014-03-20 14:27 ` Lauren Post 0 siblings, 1 reply; 37+ messages in thread From: Eric Nelson @ 2014-03-20 14:25 UTC (permalink / raw) To: Lauren Post, meta-freescale Thanks Lauren, On 03/20/2014 05:47 AM, Lauren Post wrote: > My original statement about acceleration was for webGL in the browser. > Are you working in the context of OSSystems/meta-browser (i.e. Otavio's push of Chromium 35)? ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-20 14:25 ` Eric Nelson @ 2014-03-20 14:27 ` Lauren Post 0 siblings, 0 replies; 37+ messages in thread From: Lauren Post @ 2014-03-20 14:27 UTC (permalink / raw) To: Eric Nelson, meta-freescale Yes we are using that version for our chromium. -----Original Message----- From: Eric Nelson [mailto:eric.nelson@boundarydevices.com] Sent: Thursday, March 20, 2014 9:25 AM To: Post Lauren-RAA013; meta-freescale@yoctoproject.org Subject: Re: Chromium acceleration Thanks Lauren, On 03/20/2014 05:47 AM, Lauren Post wrote: > My original statement about acceleration was for webGL in the browser. > Are you working in the context of OSSystems/meta-browser (i.e. Otavio's push of Chromium 35)? ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration @ 2014-03-25 9:32 zboszor 2014-03-25 9:35 ` Eric Bénard 2014-03-25 9:46 ` Marco Trillo 0 siblings, 2 replies; 37+ messages in thread From: zboszor @ 2014-03-25 9:32 UTC (permalink / raw) To: meta-freescale Hi, how can I add the meta-browser tree to the set of recipes? I did: $ cd fsl-community-bsp/sources $ git clone https://github.com/OSSystems/meta-browser.git $ cd .. $ . ./setup-environment build-master-next/ $ bitbake chromium Loading cache: 100% |###########################################################################################################################| ETA: 00:00:00 Loaded 1806 entries from dependency cache. ERROR: Nothing PROVIDES 'chromium' Summary: There was 1 WARNING message shown. Summary: There was 1 ERROR message shown, returning a non-zero exit code. [zozo@localhost build-master-next]$ Where should I move the meta-browser directory from the toplevel "sources"? What else should be done? Thanks in advance, Zoltán Böszörményi ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-25 9:32 zboszor @ 2014-03-25 9:35 ` Eric Bénard 2014-03-25 9:46 ` Marco Trillo 1 sibling, 0 replies; 37+ messages in thread From: Eric Bénard @ 2014-03-25 9:35 UTC (permalink / raw) To: zboszor; +Cc: meta-freescale Hi Zoltán, Le Tue, 25 Mar 2014 10:32:22 +0100, zboszor@pr.hu a écrit : > Hi, > > how can I add the meta-browser tree to the set of recipes? I did: > > $ cd fsl-community-bsp/sources > $ git clone https://github.com/OSSystems/meta-browser.git > $ cd .. > $ . ./setup-environment build-master-next/ > $ bitbake chromium > Loading cache: 100% > |###########################################################################################################################| > ETA: 00:00:00 > Loaded 1806 entries from dependency cache. > ERROR: Nothing PROVIDES 'chromium' > > Summary: There was 1 WARNING message shown. > Summary: There was 1 ERROR message shown, returning a non-zero exit code. > [zozo@localhost build-master-next]$ > > Where should I move the meta-browser directory from the toplevel "sources"? > What else should be done? > http://www.yoctoproject.org/docs/1.5.1/dev-manual/dev-manual.html#enabling-your-layer Eric ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-25 9:32 zboszor 2014-03-25 9:35 ` Eric Bénard @ 2014-03-25 9:46 ` Marco Trillo 2014-03-25 12:58 ` Boszormenyi Zoltan 1 sibling, 1 reply; 37+ messages in thread From: Marco Trillo @ 2014-03-25 9:46 UTC (permalink / raw) To: zboszor; +Cc: meta-freescale Hi, On 03/25/2014 10:32 AM, zboszor@pr.hu wrote: > Hi, > > how can I add the meta-browser tree to the set of recipes? I did: > > $ cd fsl-community-bsp/sources > $ git clone https://github.com/OSSystems/meta-browser.git > $ cd .. > $ . ./setup-environment build-master-next/ > $ bitbake chromium > Loading cache: 100% > |###########################################################################################################################| > ETA: 00:00:00 > Loaded 1806 entries from dependency cache. > ERROR: Nothing PROVIDES 'chromium' > > Summary: There was 1 WARNING message shown. > Summary: There was 1 ERROR message shown, returning a non-zero exit code. > [zozo@localhost build-master-next]$ > > Where should I move the meta-browser directory from the toplevel "sources"? > What else should be done? You need to add the "meta-browser" layer to your bblayers.conf (in build/conf/), like this: BBLAYERS += " ${BSPDIR}/sources/meta-browser " You will probably need other to add other layers such as "meta-gnome". Hope it helps, Marco. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-03-25 9:46 ` Marco Trillo @ 2014-03-25 12:58 ` Boszormenyi Zoltan 0 siblings, 0 replies; 37+ messages in thread From: Boszormenyi Zoltan @ 2014-03-25 12:58 UTC (permalink / raw) To: Marco Trillo; +Cc: meta-freescale 2014-03-25 10:46 keltezéssel, Marco Trillo írta: > Hi, > > On 03/25/2014 10:32 AM, zboszor@pr.hu wrote: >> Hi, >> >> how can I add the meta-browser tree to the set of recipes? I did: >> >> $ cd fsl-community-bsp/sources >> $ git clone https://github.com/OSSystems/meta-browser.git >> $ cd .. >> $ . ./setup-environment build-master-next/ >> $ bitbake chromium >> Loading cache: 100% >> |###########################################################################################################################| >> >> ETA: 00:00:00 >> Loaded 1806 entries from dependency cache. >> ERROR: Nothing PROVIDES 'chromium' >> >> Summary: There was 1 WARNING message shown. >> Summary: There was 1 ERROR message shown, returning a non-zero exit code. >> [zozo@localhost build-master-next]$ >> >> Where should I move the meta-browser directory from the toplevel "sources"? >> What else should be done? > > You need to add the "meta-browser" layer to your bblayers.conf (in build/conf/), like this: > > BBLAYERS += " ${BSPDIR}/sources/meta-browser " > > You will probably need other to add other layers such as "meta-gnome". Thanks for the answer. I also added "meta-gnome" to BBLAYERS and I needed these set in local.conf: LICENSE_FLAGS_WHITELIST = "commercial_libav commercial_x264" PREFERRED_PROVIDER_jpeg = "libjpeg" PREFERRED_PROVIDER_jpeg-native = "libjpeg-native" Using libjpeg-turbo fails with: ERROR: Fetcher failure: Fetch command failed with exit code 1, output: svn: E210002: Unable to connect to a repository at URL 'svn://svn.code.sf.net/p/libjpeg-turbo/code/trunk' svn: E210002: Network connection closed unexpectedly ERROR: Function failed: Fetcher failure for URL: 'svn://svn.code.sf.net/p/libjpeg-turbo/code;protocol=svn;module=trunk'. Unable to fetch URL from any source. ERROR: Logfile of failure stored in: /home/zozo/fsl-community-bsp/build-master-next/tmp/work/cortexa9hf-vfp-neon-poky-linux-gnueabi/libjpeg-turbo/8d+1.3.0-r0/temp/log.do_fetch.23323 ERROR: Task 949 (/home/zozo/fsl-community-bsp/sources/meta-openembedded/meta-oe/recipes-core/jpeg/libjpeg-turbo_svn.bb, do_fetch) failed with exit code '1' Best regards, Zoltán Böszörményi ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration @ 2014-11-27 10:14 Becue Paul 2014-12-01 14:34 ` Daiane Angolini 0 siblings, 1 reply; 37+ messages in thread From: Becue Paul @ 2014-11-27 10:14 UTC (permalink / raw) To: meta-freescale [-- Attachment #1: Type: text/plain, Size: 2068 bytes --] Hello I am working on a device with an iMX6 Solo processor for making a kind of web-kiosk with touch interface on Yocto. To learn Yocto I bought your book a month ago. It helped me a lot. Not too long and to the point. Thank you for it. I am building it with Yocto Linux - X11 - chromium.I added the chromium-imx layer from the freescale layer to my build. I builded for daisy and dizzy. But I can't get the hardware acceleration on for chromium. When I look at chrome://gpu I get: Graphics Feature Status Canvas: Software only, hardware acceleration unavailable Flash: Software only, hardware acceleration unavailable Flash Stage3D: Software only, hardware acceleration unavailable Flash Stage3D Baseline profile: Software only, hardware acceleration unavailable Compositing: Software only, hardware acceleration unavailable Rasterization: Software only, hardware acceleration unavailable Threaded Rasterization: Unavailable Video Decode: Software only, hardware acceleration unavailable Video Encode: Software only, hardware acceleration unavailable WebGL: Unavailable and on the command line: ERROR:browser_main_loop.cc(162)] Running without the SUID sandbox! See https://code.google.com/p/chromium/wiki/LinuxSUIDSandboxDevelopment for more information on developing with the sandbox on. ERROR:gl_implementation_x11.cc(86)] glxGetProcAddress not found. ERROR:gpu_child_thread.cc(143)] Exiting GPU process due to errors during initialization ERROR: v4l2 capture: slave not found! ERROR:renderer_main.cc(204)] Running without renderer sandbox ERROR:desktop_window_tree_host_x11.cc(1547)] Not implemented reached in void views::DesktopWindowTreeHostX11::MapWindow(ui::WindowShowState) ERROR:renderer_main.cc(204)] Running without renderer sandbox ERROR:renderer_main.cc(204)] Running without renderer sandbox ERROR:renderer_main.cc(204)] Running without renderer sandbox ERROR:renderer_main.cc(204)] Running without renderer sandbox Did I something wrong or is it simply not possible with the GPU of the iMX6? Thank you in advance. Paul Becue. [-- Attachment #2: Type: text/html, Size: 6114 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-11-27 10:14 Becue Paul @ 2014-12-01 14:34 ` Daiane Angolini 2014-12-01 15:07 ` Otavio Salvador 0 siblings, 1 reply; 37+ messages in thread From: Daiane Angolini @ 2014-12-01 14:34 UTC (permalink / raw) To: Becue Paul; +Cc: meta-freescale On Thu, Nov 27, 2014 at 8:14 AM, Becue Paul <P.Becue@televic.com> wrote: > Hello > > I am working on a device with an iMX6 Solo processor for making a kind of > web-kiosk with touch interface on Yocto. > To learn Yocto I bought your book a month ago. It helped me a lot. Not too > long and to the point. Thank you for it. Thanks a lot. I'm glad to hear that. > I am building it with Yocto Linux - X11 - chromium.I added the chromium-imx > layer from the freescale layer to my build. I builded for daisy and dizzy. > But I can't get the hardware acceleration on for chromium. When I look at > chrome://gpu I get: > Graphics Feature Status Canvas: Software only, hardware acceleration > unavailable Flash: Software only, hardware acceleration unavailable Flash > Stage3D: Software only, hardware acceleration unavailable Flash Stage3D > Baseline profile: Software only, hardware acceleration unavailable > Compositing: Software only, hardware acceleration unavailable Rasterization: > Software only, hardware acceleration unavailable Threaded Rasterization: > Unavailable Video Decode: Software only, hardware acceleration unavailable > Video Encode: Software only, hardware acceleration unavailable WebGL: > Unavailable > and on the command line: > ERROR:browser_main_loop.cc(162)] Running without the SUID sandbox! See > https://code.google.com/p/chromium/wiki/LinuxSUIDSandboxDevelopment for more > information on developing with the sandbox on. > ERROR:gl_implementation_x11.cc(86)] glxGetProcAddress not found. > ERROR:gpu_child_thread.cc(143)] Exiting GPU process due to errors during > initialization > ERROR: v4l2 capture: slave not found! > ERROR:renderer_main.cc(204)] Running without renderer sandbox > ERROR:desktop_window_tree_host_x11.cc(1547)] Not implemented reached in void > views::DesktopWindowTreeHostX11::MapWindow(ui::WindowShowState) > ERROR:renderer_main.cc(204)] Running without renderer sandbox > ERROR:renderer_main.cc(204)] Running without renderer sandbox > ERROR:renderer_main.cc(204)] Running without renderer sandbox > ERROR:renderer_main.cc(204)] Running without renderer sandbox I tested cromium sometime ago with dizzy and it worked fine, I only used 4.8 for toolchain. > Did I something wrong or is it simply not possible with the GPU of the iMX6? > Thank you in advance. What are you using? Freescale Official Release or FSL Community BSP? Daiane > > > > Paul Becue. > > > > > -- > _______________________________________________ > meta-freescale mailing list > meta-freescale@yoctoproject.org > https://lists.yoctoproject.org/listinfo/meta-freescale > ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: Chromium acceleration 2014-12-01 14:34 ` Daiane Angolini @ 2014-12-01 15:07 ` Otavio Salvador 0 siblings, 0 replies; 37+ messages in thread From: Otavio Salvador @ 2014-12-01 15:07 UTC (permalink / raw) To: Daiane Angolini; +Cc: meta-freescale, Becue Paul On Mon, Dec 1, 2014 at 12:34 PM, Daiane Angolini <daiane.list@gmail.com> wrote: > On Thu, Nov 27, 2014 at 8:14 AM, Becue Paul <P.Becue@televic.com> wrote: >> I am working on a device with an iMX6 Solo processor for making a kind of >> web-kiosk with touch interface on Yocto. >> To learn Yocto I bought your book a month ago. It helped me a lot. Not too >> long and to the point. Thank you for it. > > Thanks a lot. I'm glad to hear that. Me too :-) >> I am building it with Yocto Linux - X11 - chromium.I added the chromium-imx >> layer from the freescale layer to my build. I builded for daisy and dizzy. >> But I can't get the hardware acceleration on for chromium. When I look at >> chrome://gpu I get: >> Graphics Feature Status Canvas: Software only, hardware acceleration >> unavailable Flash: Software only, hardware acceleration unavailable Flash >> Stage3D: Software only, hardware acceleration unavailable Flash Stage3D >> Baseline profile: Software only, hardware acceleration unavailable >> Compositing: Software only, hardware acceleration unavailable Rasterization: >> Software only, hardware acceleration unavailable Threaded Rasterization: >> Unavailable Video Decode: Software only, hardware acceleration unavailable >> Video Encode: Software only, hardware acceleration unavailable WebGL: >> Unavailable >> and on the command line: >> ERROR:browser_main_loop.cc(162)] Running without the SUID sandbox! See >> https://code.google.com/p/chromium/wiki/LinuxSUIDSandboxDevelopment for more >> information on developing with the sandbox on. >> ERROR:gl_implementation_x11.cc(86)] glxGetProcAddress not found. >> ERROR:gpu_child_thread.cc(143)] Exiting GPU process due to errors during >> initialization >> ERROR: v4l2 capture: slave not found! >> ERROR:renderer_main.cc(204)] Running without renderer sandbox >> ERROR:desktop_window_tree_host_x11.cc(1547)] Not implemented reached in void >> views::DesktopWindowTreeHostX11::MapWindow(ui::WindowShowState) >> ERROR:renderer_main.cc(204)] Running without renderer sandbox >> ERROR:renderer_main.cc(204)] Running without renderer sandbox >> ERROR:renderer_main.cc(204)] Running without renderer sandbox >> ERROR:renderer_main.cc(204)] Running without renderer sandbox > > I tested cromium sometime ago with dizzy and it worked fine, I only > used 4.8 for toolchain. I am using it with several customers (with different levels of customization) with 4.9 toolchain and it is working fine. -- Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750 ^ permalink raw reply [flat|nested] 37+ messages in thread
![[-- Attachment #2: about_gpu_nvidia_x86_64.png --] [-- Type: image/png, Size: 70457 bytes --]](../../CAC9ffDHxP-2p6Nya_-V6kZTEq8smcw8jQifX7kSqE-Apy=tRpw@mail.gmail.com/2-about_gpu_nvidia_x86_64.png){kind=link}
![[-- Attachment #2: 20140402_121516_800x600.png --] [-- Type: image/png, Size: 113651 bytes --]](../../533BE697.3040800@pr.hu/2-20140402_121516_800x600.png){kind=link}
end of thread, other threads:[~2014-12-01 15:07 UTC | newest]
Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-03-19 20:49 Chromium acceleration Eric Nelson
2014-03-19 21:00 ` Carlos Rafael Giani
2014-03-19 22:40 ` Eric Nelson
2014-03-20 1:29 ` Christian Betz
2014-03-20 2:50 ` Eric Nelson
2014-03-20 8:30 ` Carlos Rafael Giani
2014-03-20 12:22 ` Otavio Salvador
2014-03-20 14:07 ` Christian Betz
2014-03-20 15:02 ` Otavio Salvador
2014-03-20 14:29 ` Eric Nelson
2014-03-20 14:58 ` Otavio Salvador
2014-03-21 11:49 ` Diego
2014-03-21 14:17 ` Eric Nelson
2014-03-20 13:46 ` Christian Betz
2014-03-20 23:19 ` Carlos Rafael Giani
2014-03-21 12:21 ` Dmitriy B.
2014-04-01 19:22 ` Eric Nelson
2014-04-02 10:21 ` Carlos Rafael Giani
2014-04-02 10:28 ` Gary Thomas
2014-04-02 10:33 ` Carlos Rafael Giani
[not found] ` <533BE4A3.7040301@pr.hu>
2014-04-02 10:23 ` Carlos Rafael Giani
2014-04-02 11:12 ` Boszormenyi Zoltan
2014-04-02 12:02 ` Christian Betz
2014-04-02 12:28 ` Carlos Rafael Giani
2014-04-04 9:28 ` Boszormenyi Zoltan
2014-04-02 10:29 ` Boszormenyi Zoltan
2014-04-02 14:16 ` Eric Nelson
2014-03-20 12:47 ` Lauren Post
2014-03-20 14:25 ` Eric Nelson
2014-03-20 14:27 ` Lauren Post
2014-03-25 9:32 zboszor
2014-03-25 9:35 ` Eric Bénard
2014-03-25 9:46 ` Marco Trillo
2014-03-25 12:58 ` Boszormenyi Zoltan
2014-11-27 10:14 Becue Paul
2014-12-01 14:34 ` Daiane Angolini
2014-12-01 15:07 ` Otavio Salvador
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.