Re: PROBLEM: Broken pixel format for Elgato Cam Link 4K

[Adam Goode <agoode@google.com>]

On Sun, Nov 22, 2020 at 2:34 PM Benjamin Drung
<benjamin.drung@cloud.ionos.com> wrote:
>
> Am Freitag, den 20.11.2020, 16:45 -0500 schrieb Adam Goode:
> > On Fri, Nov 20, 2020 at 1:52 PM Benjamin Drung
> > <benjamin.drung@cloud.ionos.com> wrote:
> > >
> > > Hi,
> > >
> > > I own an Elgato Cam Link 4K which is a very popular USB HDMI
> > > capture
> > > device (number one capture card by click rates on Geizhals [1]).
> > > The
> > > problem is that the video feed is distorted when using the
> > > /dev/videoX
> > > device in the browser (tested on Firefox and Chromium) for video
> > > conferencing (tested with Jitsi Meet and Google Meet). The same
> > > distortion is present when opening `v4l2:///dev/video0` with VLC.
> > >
> > > The Elgato Cam Link 4K reports to have three different pixel
> > > formats:
> > >
> > > ```
> > > $ v4l2-ctl -d /dev/video0 --list-formats-ext
> > > ioctl: VIDIOC_ENUM_FMT
> > >         Type: Video Capture
> > >
> > >         [0]: 'NV12' (Y/CbCr 4:2:0)
> > >                 Size: Discrete 3840x2160
> > >                         Interval: Discrete 0.040s (25.000 fps)
> > >         [1]: 'NV12' (Y/CbCr 4:2:0)
> > >                 Size: Discrete 3840x2160
> > >                         Interval: Discrete 0.040s (25.000 fps)
> > >         [2]: 'YU12' (Planar YUV 4:2:0)
> > >                 Size: Discrete 3840x2160
> > >                         Interval: Discrete 0.040s (25.000 fps)
> > > ```
> > >
> > > When specifying the video format 'YU12' to VLC, the video is
> > > distorted
> > > the same way as using the default video format. When specifying
> > > 'NV12'
> > > to VLC, the video feed is displayed correctly:
> > >
> > > ```
> > > vlc v4l2:///dev/video0 --v4l2-chroma=NV12
> > > ```
> > >
> > > In OBS, the video feed is always displayed correctly. All video
> > > formats
> > > 'Y/CbCr 4:2:0', 'Planar YUV 4:2:0', 'BGR3 (Emulated)', and 'YV12
> > > (Emulated)' combined with the color ranges 'Default', 'Partial',
> > > and
> > > 'Full' produce the same correct output.
> > >
> > > With Linux >= 5.9 this behavior in OBS changes: The video format
> > > 'Y/CbCr 4:2:0' displays the video correctly. Switching to 'Planar
> > > YUV
> > > 4:2:0', 'BGR3 (Emulated)', or 'YV12 (Emulated)' shows the video
> > > distorted and OBS shows this error message:
> > >
> > > ```
> > > info: v4l2-input: Pixelformat: NV12
> > > [...]
> > > libv4l2: error set_fmt gave us a different result than try_fmt!
> > > info: v4l2-input: Resolution: 3840x2160
> > > info: v4l2-input: Pixelformat: NV12
> > > ```
> > >
> > > Changing the video format back does not have an effect until I also
> > > change the color range (does seem to be relevant what to select
> > > there).
> > >
> > > Workaround
> > > ----------
> > >
> > > You can create a v4l2loopback device and use ffmpeg to stream from
> > > the
> > > Cam Link 4K to the loopback device:
> > >
> > > ```
> > > ffmpeg -f v4l2 -input_format yuv420p -video_size 3840x2160 \
> > >   -i "$camlink" -codec copy -f v4l2 "$loopdev"
> > > ```
> > >
> > > This workaround works, but is cumbersome and burns CPU cycles.
> > >
> > > Other reports
> > > -------------
> > >
> > > Searching the web for "Cam Link 4K Linux" reveals many similar
> > > reports
> > > like this. Noteworthy is blog post [3] from Mike Walters who
> > > patched
> > > the Cam Link 4K firmware to report the correct video format. I am
> > > willing to debug this issue and do test, but I don't want to flash
> > > the
> > > firmware to not break the warrenty (bisides I lack the hardware for
> > > flashing).
> > >
> > > Environment
> > > -----------
> > >
> > > This problem is present in Ubuntu 20.04 with linux 5.4.0-54.60 and
> > > Ubuntu 20.10 with linux 5.8.0-29.31. I also tested the mainline
> > > kernels
> > > builds 5.9.8-050908.202011101634 and 5.10.0-051000rc4.202011152030
> > > from
> > > Ubuntu [2].
> > >
> > > The Cam Link 4K shows follow entries in dmesg:
> > >
> > > ```
> > > [    1.575753] usb 2-3: new SuperSpeed Gen 1 USB device number 2
> > > using xhci_hcd
> > > [    1.596664] usb 2-3: LPM exit latency is zeroed, disabling LPM.
> > > [    1.598557] usb 2-3: New USB device found, idVendor=0fd9,
> > > idProduct=0066, bcdDevice= 0.00
> > > [    1.598558] usb 2-3: New USB device strings: Mfr=1, Product=2,
> > > SerialNumber=4
> > > [    1.598559] usb 2-3: Product: Cam Link 4K
> > > [    1.598560] usb 2-3: Manufacturer: Elgato
> > > ```
> > >
> > > I have another problems with 5.9.8-050908.202011101634 and 5.10.0-
> > > 051000rc4.202011152030: Chromium fail to access the video device of
> > > Cam
> > > Link 4K and the notebook integrated webcam has a too low
> > > brightness.
> > >
> > > [1] https://geizhals.de/?cat=vidext
> > > [2] https://kernel.ubuntu.com/~kernel-ppa/mainline/
> > > [3]
> > > https://assortedhackery.com/patching-cam-link-to-play-nicer-on-linux/
> > >
> > > --
> > > Benjamin Drung
> > > Debian & Ubuntu Developer
> > >
> >
> >
> > Hi,
> >
> > I am running on Fedora 32 which has the fix I wrote for the buggy
> > elgato firmware. The bug in the firmware makes it impossible to
> > properly select a non-0 pixel format when following the UVC
> > negotiation protocol. This is because the firmware returns the pixel
> > format in the wrong byte of the packet. The driver was following the
> > UVC protocol but did not send the pixel format back to the v4l2
> > subsystem. It does that now.
> >
> > I'm not surprised that other bugs are emerging now. Ultimately the
> > firmware is buggy and announces pixel formats that it then rejects.
> > If
> > I flip through the settings in OBS, I do manage to wedge the
> > interface. But most of the programs I've seen that use v4l2 are buggy
> > in this way. A reliable one is the qv4l2 test program. I've also had
> > no problems with Chromium.
> >
> > That reverse engineering is interesting! But I think it hides the
> > real
> > problem, where the pixel format negotiation on the firmware writes
> > into the wrong byte of the packet.
>
> Hi Adam,
>
> you are talking about commit ec2c23f628802317f73fab5255cc62a776bc7930
> and 8a652a17e3c005dcdae31b6c8fdf14382a29cbbe that are part of
> v5.10-rc1?
>
> What do you suggest to fix the issue? Should I contact Elgato
> requesting a firmware update? Can the kernel ship a quirk for the Cam
> Link 4K to workaround the firmware bug? I would be happy to try out
> patches.
>

I would file a ticket. I did so some time ago, but there have been no
firmware updates for many months.

> I tested the qv4l2 with Linux 5.10.0-051000rc4.202011152030. When using
> the default settings, it displays the video correctly, but with a
> vertical green one-pixel-wide line. The terminal prints "error" four
> times. Sadly there is no debug mode to figure out where that error
> comes from.

I'm not sure where this line is coming from. Can you post a link to a
screenshot or capture? I have never encountered this issue. Can you
reproduce on a Windows or Mac computer?

>
> Selecting "YU12 (Planar YUV 4:2:0)" print the following error message
> and switches back to the previous format:
>
> libv4l2: error set_fmt gave us a different result than try_fmt!
>

Yes, this is the kernel coping with the buggy firmware. These errors
are expected. It is also expected that YU12 does not work (the device
advertises it, then rejects it when selected). I believe Windows and
Mac (you can use OBS to try) do the same thing.

A quirk is possible (I hacked something up with libuvc when I was
debugging this) but it's not something I have time to look at right
now.


Adam

> --
> Benjamin Drung
> Debian & Ubuntu Developer
>