Video Hardware Decoding: Jittery Rectangles on Nvidia GT218 NVA8 VP4.

Video Hardware Decoding: Jittery Rectangles on Nvidia GT218 NVA8 VP4.

[Ralph Corderoy]

Hi,

I'm having trouble with hardware decoding of video with mpv(1) having
switched from Nvidia's proprietary driver to nouveau due to Arch Linux
removing its nvidia-340xx package.

The machine is an Acer Revo.

    $ lspci -nn | grep VGA
    01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GT218 [ION] [10de:0a64] (rev a2)

I think this is codename NVA8 with the video-accleration engine VP4.0.

I've these packages installed, following
https://wiki.archlinux.org/index.php/Nouveau and
https://wiki.archlinux.org/index.php/Hardware_video_acceleration

    $ pacman -Q | egrep 'nvidia|nouveau|mesa|vdpau|vaapi|xorg-server'
    libva-mesa-driver 19.1.3-1
    libva-vdpau-driver 0.7.4-4
    libvdpau 1.2-1
    libvdpau-va-gl 0.4.2-2
    mesa 19.1.3-1
    mesa-demos 8.4.0-1
    mesa-vdpau 19.1.3-1
    nouveau-fw 325.15-1
    vdpauinfo 1.0-3
    xf86-video-nouveau 1.0.16-1
    xorg-server 1.20.5-2
    xorg-server-common 1.20.5-2
    $

A paste of vdpauinfo(1)'s output is at http://ix.io/1PHi.

I've many sample videos that exhibit the same problem, but I'll stick
with one from BBC's iPlayer obtained with get_iplayer(1).

    $ ffmpeg -i $f |& grep Video:
        Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661),
            yuv420p(tv, bt709), 960x540 [SAR 1:1 DAR 16:9], 1602 kb/s,
            25 fps, 25 tbr, 90k tbn, 50 tbc (default)

Plain old `mpv $f' plays fine, using software decoding.

    VO: [gpu] 960x540 yuv420p

Adding `--vo=vdpau' gives

    VO: [vdpau] 960x540 yuv420p
    [vo/vdpau] Compositing window manager detected. Assuming timing info
        is inaccurate.

Playback is fine, still software decoding.

`--vo=vdpau --hwdec=vdpau' uses hardware decoding:

    Using hardware decoding (vdpau).
    VO: [vdpau] 960x540 vdpau[yuv420p]
    [vo/vdpau] Compositing window manager detected. Assuming timing info
        is inaccurate.

The video plays, CPU load is less (my aim), but there's ‘tearing’ of the
picture as if small rectangles that are updates are appearing in the
wrong location, off by a little.  If I step through the frames with
mpv's ‘.’ and ‘,’ then I've found a pattern: one frame's picture is
good, followed by N bad ones where N is 3 or 7, i.e. every 4th or 8th
frame is okay.  Don't know if that's a clue or helps someone here
recognise a known problem.

I know X well, but Mesa, etc., below it are unknown to me, even to the
extent of knowing quite how all the bits fit together.  I'd appreciate a
pointer or two as to how to tackle the problem, even if it's a ‘You've
come to the wrong place, go over there’.  :-)

-- 
Cheers, Ralph.
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Video Hardware Decoding: Jittery Rectangles on Nvidia GT218 NVA8 VP4.

[Ilia Mirkin]

On Sat, Jul 27, 2019 at 7:37 AM Ralph Corderoy <ralph@inputplus.co.uk> wrote:
> The video plays, CPU load is less (my aim), but there's ‘tearing’ of the
> picture as if small rectangles that are updates are appearing in the
> wrong location, off by a little.  If I step through the frames with
> mpv's ‘.’ and ‘,’ then I've found a pattern: one frame's picture is
> good, followed by N bad ones where N is 3 or 7, i.e. every 4th or 8th
> frame is okay.  Don't know if that's a clue or helps someone here
> recognise a known problem.
>
> I know X well, but Mesa, etc., below it are unknown to me, even to the
> extent of knowing quite how all the bits fit together.  I'd appreciate a
> pointer or two as to how to tackle the problem, even if it's a ‘You've
> come to the wrong place, go over there’.  :-)

Unfortunately I've never tracked down the cause for this.
https://nouveau.freedesktop.org/wiki/VideoAcceleration/ - see note #4.

I have, over time, collected some sample videos where this happens in
the first few frames. The plan was to do mmt traces of the blob
driver, and figure out what it was doing differently. If this is
something you're interested in, I'd be happy to provide some guidance.
However my past efforts to work this out were fruitless. Hopefully
it's just something dumb we're doing in reference frame management, or
a missing bit in the info structures, or something else entirely since
I've scoured quite heavily for those two.

If you'd like to dedicate some of your time to working this out, you
can join us at #nouveau on freenode.

Cheers,

  -ilia
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Video Hardware Decoding: Jittery Rectangles on Nvidia GT218 NVA8 VP4.

[Solerman Kaplon]

Às 14:46 de 27/07/2019, Ilia Mirkin escreveu:
> On Sat, Jul 27, 2019 at 7:37 AM Ralph Corderoy <ralph@inputplus.co.uk> wrote:
>> The video plays, CPU load is less (my aim), but there's ‘tearing’ of the
>> picture as if small rectangles that are updates are appearing in the
>> wrong location, off by a little.  If I step through the frames with
>> mpv's ‘.’ and ‘,’ then I've found a pattern: one frame's picture is
>> good, followed by N bad ones where N is 3 or 7, i.e. every 4th or 8th
>> frame is okay.  Don't know if that's a clue or helps someone here
>> recognise a known problem.
<snip>
>> Unfortunately I've never tracked down the cause for this.
>> https://nouveau.freedesktop.org/wiki/VideoAcceleration/ - see note #4.
>>
>> I have, over time, collected some sample videos where this happens in
>> the first few frames. The plan was to do mmt traces of the blob
>> driver, and figure out what it was doing differently.

<snip>

I don't really know anything about hw coding, but looking from the outside, it 
seems some kind of ring buffer with exact 3 frames maybe intended to work kinda 
like using tripple buffering? I know for once that nvidia works better using 
tripple buffering from what I've read from the kwin threads.


Solerman

_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Video Hardware Decoding: Jittery Rectangles on Nvidia GT218 NVA8 VP4.

[Ilia Mirkin]

On Mon, Jul 29, 2019 at 7:29 AM Solerman Kaplon <solerman@gmail.com> wrote:
>
> Às 14:46 de 27/07/2019, Ilia Mirkin escreveu:
> > On Sat, Jul 27, 2019 at 7:37 AM Ralph Corderoy <ralph@inputplus.co.uk> wrote:
> >> The video plays, CPU load is less (my aim), but there's ‘tearing’ of the
> >> picture as if small rectangles that are updates are appearing in the
> >> wrong location, off by a little.  If I step through the frames with
> >> mpv's ‘.’ and ‘,’ then I've found a pattern: one frame's picture is
> >> good, followed by N bad ones where N is 3 or 7, i.e. every 4th or 8th
> >> frame is okay.  Don't know if that's a clue or helps someone here
> >> recognise a known problem.
> <snip>
> >> Unfortunately I've never tracked down the cause for this.
> >> https://nouveau.freedesktop.org/wiki/VideoAcceleration/ - see note #4.
> >>
> >> I have, over time, collected some sample videos where this happens in
> >> the first few frames. The plan was to do mmt traces of the blob
> >> driver, and figure out what it was doing differently.
>
> <snip>
>
> I don't really know anything about hw coding, but looking from the outside, it
> seems some kind of ring buffer with exact 3 frames maybe intended to work kinda
> like using tripple buffering? I know for once that nvidia works better using
> tripple buffering from what I've read from the kwin threads.

The actual decoded images are wrong here -- triple buffering normally
refers to how many (image) buffers you hold in the display queue. The
deeper the queue, the more you're isolated from render timing. However
in this case, the images themselves are bad out of the decoding logic.

  -ilia
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Video Hardware Decoding: Jittery Rectangles on Nvidia GT218 NVA8 VP4.

[Ralph Corderoy]

Hi Ilia,

A fortnight ago, you wrote:
> > The video plays, CPU load is less (my aim), but there's ‘tearing’ of
> > the picture as if small rectangles that are updates are appearing in
> > the wrong location, off by a little.  If I step through the frames
> > with mpv's ‘.’ and ‘,’ then I've found a pattern: one frame's
> > picture is good, followed by N bad ones where N is 3 or 7, i.e.
> > every 4th or 8th frame is okay.  Don't know if that's a clue or
> > helps someone here recognise a known problem.
>
> Unfortunately I've never tracked down the cause for this.
> https://nouveau.freedesktop.org/wiki/VideoAcceleration/ - see note #4.

That sounds like the problem, but it is all the way through TV programme
from the BBC iPlayer's service.

> I have, over time, collected some sample videos where this happens in
> the first few frames. The plan was to do mmt traces of the blob
> driver, and figure out what it was doing differently. If this is
> something you're interested in, I'd be happy to provide some guidance.

I've two mmio traces, nouveau and nvidia, each using mpv(1) to play the
same ten-second extract of an iPlayer MP4, with and without ‘--vo=vdpau
--hwdec=vdpau’.

    $ grep -n ^MARK nvidia-4.mmiotrace
    20:MARK 65645.302939 buffer_size_kb: 524288
    21:MARK 65645.385776 starting x
    1447554:MARK 65681.979892 x started
    1448560:MARK 65682.446747 mpv none start
    1868197:MARK 65701.153770 mpv none done
    1869462:MARK 65702.748005 mpv vo start
    2633729:MARK 65724.465154 mpv vo done
    2634956:MARK 65726.259829 mpv vo hwdec start
    5347520:MARK 65759.841432 mpv vo hwdec done
    5349019:MARK 65762.431137 killing cat
    $

hwdec on nvidia easily overflowed the default 1,408 KiB trace buffer,
whereas nouveau didn't.

Are these helpful?  Shall I submit them to the Gmail address given in
https://nouveau.freedesktop.org/wiki/MmioTrace/

-- 
Cheers, Ralph.
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau

Re: Video Hardware Decoding: Jittery Rectangles on Nvidia GT218 NVA8 VP4.

[Ilia Mirkin]

On Wed, Aug 14, 2019 at 7:37 AM Ralph Corderoy <ralph@inputplus.co.uk> wrote:
>
> Hi Ilia,
>
> A fortnight ago, you wrote:
> > > The video plays, CPU load is less (my aim), but there's ‘tearing’ of
> > > the picture as if small rectangles that are updates are appearing in
> > > the wrong location, off by a little.  If I step through the frames
> > > with mpv's ‘.’ and ‘,’ then I've found a pattern: one frame's
> > > picture is good, followed by N bad ones where N is 3 or 7, i.e.
> > > every 4th or 8th frame is okay.  Don't know if that's a clue or
> > > helps someone here recognise a known problem.
> >
> > Unfortunately I've never tracked down the cause for this.
> > https://nouveau.freedesktop.org/wiki/VideoAcceleration/ - see note #4.
>
> That sounds like the problem, but it is all the way through TV programme
> from the BBC iPlayer's service.

Yeah ... there's nothing _intrinsically_ linked to those trailers, and
I've definitely seen it on full files as well. At the time I wrote
that note, I had primarily seen the issue in movie trailers I
downloaded from some h264 demo site.

>
> > I have, over time, collected some sample videos where this happens in
> > the first few frames. The plan was to do mmt traces of the blob
> > driver, and figure out what it was doing differently. If this is
> > something you're interested in, I'd be happy to provide some guidance.
>
> I've two mmio traces, nouveau and nvidia, each using mpv(1) to play the
> same ten-second extract of an iPlayer MP4, with and without ‘--vo=vdpau
> --hwdec=vdpau’.
>
>     $ grep -n ^MARK nvidia-4.mmiotrace
>     20:MARK 65645.302939 buffer_size_kb: 524288
>     21:MARK 65645.385776 starting x
>     1447554:MARK 65681.979892 x started
>     1448560:MARK 65682.446747 mpv none start
>     1868197:MARK 65701.153770 mpv none done
>     1869462:MARK 65702.748005 mpv vo start
>     2633729:MARK 65724.465154 mpv vo done
>     2634956:MARK 65726.259829 mpv vo hwdec start
>     5347520:MARK 65759.841432 mpv vo hwdec done
>     5349019:MARK 65762.431137 killing cat
>     $
>
> hwdec on nvidia easily overflowed the default 1,408 KiB trace buffer,
> whereas nouveau didn't.
>
> Are these helpful?  Shall I submit them to the Gmail address given in
> https://nouveau.freedesktop.org/wiki/MmioTrace/

Well, this might be presumptuous (given that I never figured out what
the issue was), but here was my plan of attack (which, I remind you, I
wasn't able to use to figure out the underlying problem):

1. Find an offending file where the problem occurs _early_, preferably
first few frames, since slight differences in submissions eventually
creep in, esp around reference frame management.
2. Using h264_player (available at https://github.com/imirkin/re-vp2
-- note that it has to have a few parameters hardcoded to a specific
file, and requires extraction of the h264 stream first -- see readme),
do a mmt (not mmio) trace of nvidia blob and of nouveau (look for
valgrind-mmt)
3. Compare and contrast

The theory here is that we're setting up the engines correctly, just
feeding them incorrect data. If we're setting up the engines
incorrectly, then the mmio trace would come in handy.

The reason to use h264_player and not a full-featured player is that I
don't do any funny business with frame timing, reordering, or anything
else extraneous. You can't actually watch a movie with this thing
(frame order will give you a headache), but it can be helpful for
tracing.

Of course a few things have happened in the intervening 6 years since
I wrote that tool... (has it really been that long?) The main one
being that mmt traces of nouveau no longer work out of the box. You
have to edit mesa's winsys/nouveau/drm/nouveau_drm_winsys.c to set
drm->nvif = false after the nouveau_drm_new call. That should allow
the mmt tracing to be decoded by the demmt tool.

Also the demmt tool has bitrotted a fair amount. At the time that I
was doing it, I was using "dedma", which was much more primitive, but
also more resilient to changes. There's a bin2dedma helper somewhere
to convert the "new" mmt trace format to what dedma can consume.

FWIW I was given a file where the issue occurs very early on, but by
then, I was already quite tired of doing this. I may be able to track
it down, but right now, I'm not sure where it is.

A final thought is that this happens on VP2 and VP3+ engines, which
are fairly different from each other. This leads me to believe that
we're missing a similar sort of thing on the two. I've always believed
it was reference frame management-related, but could never quite
figure it out. H264 streams may have up to 16 reference frames. Fun
stuff.

Cheers,

  -ilia
_______________________________________________
Nouveau mailing list
Nouveau@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/nouveau