All of lore.kernel.org
 help / color / mirror / Atom feed
From: Hans Verkuil <hverkuil@xs4all.nl>
To: Guillaume Tucker <guillaume.tucker@collabora.com>
Cc: linux-media@vger.kernel.org, kernel-build-reports@lists.linaro.org
Subject: Re: media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b)
Date: Fri, 24 May 2019 11:20:06 +0200	[thread overview]
Message-ID: <93a38a38-fb86-10aa-291a-1cb9fd86a23c@xs4all.nl> (raw)
In-Reply-To: <29de6255-4b55-a21a-36a0-5ef1ca36ce83@collabora.com>

On 5/24/19 11:05 AM, Guillaume Tucker wrote:
> On 24/05/2019 09:44, Hans Verkuil wrote:
>> On 5/24/19 10:38 AM, Guillaume Tucker wrote:
>>> Hi Hans,
>>>
>>> On 16/05/2019 07:41, Hans Verkuil wrote:
>>>> Hi Guillaume,
>>>>
>>>> I have a few questions/suggestions:
>>>
>>> Thanks for the feedback!  It's good to start seeing these reports
>>> on the linux-media mailing list.  And sorry for the slow reply, I
>>> was away.
>>>
>>>> On 5/15/19 9:04 PM, kernelci.org bot wrote:
>>>>> media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b)
>>>>>
>>>>> Test results summary
>>>>> --------------------
>>>>>
>>>>> V4L2 Compliance on the vivid driver.
>>>>>
>>>>> This test ran "v4l2-compliance -s" from v4l-utils:
>>>>>
>>>>>     https://www.linuxtv.org/wiki/index.php/V4l2-utils
>>>>
>>>> I'd just link directly to the git repo instead of the wiki: https://git.linuxtv.org/v4l-utils.git
>>>
>>> Sure, I thought this had been agreed before but it's easy to
>>> change.
>>>
>>>> You should add the v4l-utils commit that's used to compile v4l2-compliance.
>>>> That's important information to have. I assume that this test always uses the
>>>> latest version of v4l-utils?
>>>
>>> This information is in the detailed results, but the detailed
>>> results are only shown when there are some failures.  So we'll
>>> rework that a bit.
>>>
>>> For example, from the v4l2-compliance-uvc report:
>>>
>>>
>>> Test failures
>>> ------------- 
>>> 1  | rk3399-gru-kevin       | arm64 |  52 total:  43 PASS   9 FAIL   0 SKIP
>>>
>>>   Config:      defconfig
>>>   Compiler:    gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0)
>>>   Lab Name:    lab-collabora
>>>   Plain log:   https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.txt
>>>   HTML log:    https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.html
>>>   Rootfs:      http://storage.kernelci.org/images/rootfs/debian/stretch-v4l2/20190510.0/arm64/rootfs.cpio.gz
>>>   Test Git:    git://linuxtv.org/v4l-utils.git
>>>   Test Commit: 0d61ddede7d340ffa1c75a2882e30c455ef3d8b8
>>>
>>>
>>> The git repo and commit hash here show you which version of
>>> vl4-utils was used.
>>>
>>>
>>> At the moment, the v4l2-compliance is part of a rootfs which gets
>>> updated each time kernelci.org production code gets updated,
>>> which is typically once a week.  This can be improved to have the
>>> rootfs updates independent from the rest, then we could trigger
>>> rebuilds every time v4l-utils changes, but there are a few things
>>> to take into consideration before we can do this safely.
>>
>> I don't think it has to be updated every time v4l-utils changes, at least
>> for now, as long as it is clear which v4l-utils version is used.
>>
>>>
>>>>> See each detailed section in the report below to find out the git URL and
>>>>> particular revision that was used to build the test binaries.
>>>>>
>>>>>
>>>>>   Tree:    media
>>>>>   Branch:  master
>>>>>   Kernel:  media_v5.1-2-16-gfc8670d1f72b
>>>>
>>>> I assume this is the version of the host kernel, right? Perhaps calling this
>>>> "Host Kernel:" would be less ambiguous.
>>>
>>> I have to say I fail to see any ambiguity here: KernelCI is about
>>> testing kernels, and this tells you the kernel revision under
>>> test.  Calling it "host" kernel might actually be confusing when
>>> running with QEMU as people may think it's the version on the
>>> host server running the test.
>>>
>>>>>   URL:     https://git.linuxtv.org/media_tree.git
>>>>>   Commit:  fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f
>>
>> What confuses me is that the kernel above says: media_v5.1-2-16-gfc8670d1f72b
>> but the commit says fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f.
>>
>> So the hashes are different, which led me to conclude that one referred to
>> the host kernel and the other to the kernel under test.
>>
>> Where does the string "media_v5.1-2-16-gfc8670d1f72b" come from?
> 
> This is the output of "git describe", except that the slash in
> the tag name was replaced with an underscore (I know that's not
> great, but otherwise it causes some issues with the path on the
> storage server).  The media/v5.2-1 tag has been created since
> then, but if you run this:
> 
> $ git checkout fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f
> $ git tag -d media/v5.2-1
> $ git describe
> media/v5.1-2-16-gfc8670d1f72b
> 
> and fc8670d1f72b is the beginning of the full hash.

Argh! I totally missed that. And even overlooked that 'g' is not a hex
number :-)

I never use git describe, so I wasn't familiar with the format, hence my
confusion.

So never mind my comment :-)

Regards,

	Hans

> 
>>>>> 1  | qemu                   | arm64 | 118 total: 118 PASS   0 FAIL   0 SKIP
>>>>> 2  | qemu                   | arm   | 118 total: 118 PASS   0 FAIL   0 SKIP  
>>>>
>>>> Even if everything was OK, I think it would still be useful to have a link
>>>> to the full test report.
>>>
>>> Yes, that is essentially the same issue as with the v4l-utils
>>> version as I described above.  The detailed results show a link
>>> to the console output, which isn't just a clean v4l2-compliance
>>> log but it's better than nothing.
>>>
>>> We may also add a feature to publish some files alongside the
>>> parsed test results, and in the case of v4l2-compliance it would
>>> typically be the plain output of the test suite that developers
>>> are familiar with.  It's not a supported feature right now as
>>> only the raw console log is sent from the device to the database.
>>>
>>> Best wishes,
>>> Guillaume
>>>
>>
>> Regards,
>>
>> 	Hans
>>


  reply	other threads:[~2019-05-24  9:20 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-15 19:04 kernelci.org bot
2019-05-16  6:41 ` Hans Verkuil
2019-05-24  8:38   ` Guillaume Tucker
2019-05-24  8:44     ` Hans Verkuil
2019-05-24  9:05       ` Guillaume Tucker
2019-05-24  9:20         ` Hans Verkuil [this message]
2019-05-24  9:27           ` Guillaume Tucker

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=93a38a38-fb86-10aa-291a-1cb9fd86a23c@xs4all.nl \
    --to=hverkuil@xs4all.nl \
    --cc=guillaume.tucker@collabora.com \
    --cc=kernel-build-reports@lists.linaro.org \
    --cc=linux-media@vger.kernel.org \
    --subject='Re: media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b)' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.