Re: [PATCH 1/2] media: uapi: h264: Add num_ref_idx_active_override_flag

[Philipp Zabel <p.zabel@pengutronix.de>]

On Mon, 2019-09-09 at 16:09 +0900, Tomasz Figa wrote:
> On Thu, Sep 5, 2019 at 11:17 PM Nicolas Dufresne
> <nicolas.dufresne@collabora.com> wrote:
> > 
> > Le jeudi 05 septembre 2019 à 12:39 +0200, Philipp Zabel a écrit :
> > > On Thu, 2019-09-05 at 19:31 +0900, Tomasz Figa wrote:
> > > > On Thu, Sep 5, 2019 at 7:15 PM Philipp Zabel <p.zabel@pengutronix.de> wrote:
> > > > > This flag tells the kernel whether the slice header contained the
> > > > > num_ref_idx_l[01]_active_minus1 syntax elements, or whether the
> > > > > num_ref_idx_l[01]_default_active_minus1 from PPS should be used
> > > > > instead.
> > > > > 
> > > > > Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
> > > > > ---
> > > > >  Documentation/media/uapi/v4l/ext-ctrls-codec.rst | 3 +++
> > > > >  include/media/h264-ctrls.h                       | 1 +
> > > > >  2 files changed, 4 insertions(+)
> > > > > 
> > > > > diff --git a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> > > > > index bc5dd8e76567..451a5b0f2a35 100644
> > > > > --- a/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> > > > > +++ b/Documentation/media/uapi/v4l/ext-ctrls-codec.rst
> > > > > @@ -1860,6 +1860,9 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type -
> > > > >      * - ``V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH``
> > > > >        - 0x00000008
> > > > >        -
> > > > > +    * - ``V4L2_H264_SLICE_FLAG_NUM_REF_IDX_ACTIVE_OVERRIDE
> > > > > +      - 0x00000010
> > > > > +      - Corresponds to the num_ref_idx_active_override_flag syntax element.
> > > > > 
> > > > 
> > > > As far as I remember, the idea was for the userspace to always put the
> > > > right num_ref_idx in the slice_params and the drivers always use that.
> > > > Was there any problem with that?
> > > 
> > > I don't think so, at least for currently known hardware.
> > > 
> > > In that case we should drop the unused
> > > num_ref_idx_l[01]_default_active_minus1 fields from struct
> > > v4l2_ctrl_h264_pps and document that userspace should fill
> > > the defaults into v4l2_ctrl_h264_slice_params themselves if
> > > num_ref_idx_active_override_flag wasn't set.
> > 
> > It might have been added in a previous effort to allow reconstructing
> > the bitstream from the structures.
> 
> Wouldn't one still be able to reconstruct a valid (but not exact)
> stream without that flag, given the assumption above?

If the Hantro G1, as appears to be the case, parses the slice header and
decides itself whether to use the override from the slice or the default
that was written to a register, it needs the
num_ref_idx_l[01]_default_active_minus1 field to fill the register, but
doesn't need either the num_ref_idx_l[01]_active_minus1 override nor the
flag.

A decoder that doesn't parse the slice header can always be told to use
the override (thus no need to have the flag in the uapi), if userspace
fills the default into the override fields as a fallback. Such a decoder
does need the num_ref_idx_l[01]_active_minus1 override, but doesn't need
the num_ref_idx_l[01]_default_active_minus1 field nor the flag.

That is my current understanding of the intention behind this interface,
I hope this is accurate.
I've tried to make the docs reflect this in ("media: uapi: h264: clarify
num_ref_idx_l[01]_(default_)active fields") [1].

[1] https://lore.kernel.org/linux-media/20190905114210.9232-1-p.zabel@pengutronix.de/T/#u

regards
Philipp