All of lore.kernel.org
 help / color / mirror / Atom feed
From: Guillaume Tucker <guillaume.tucker@collabora.com>
To: Hans Verkuil <hverkuil@xs4all.nl>
Cc: linux-media@vger.kernel.org, kernel-build-reports@lists.linaro.org
Subject: Re: media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b)
Date: Fri, 24 May 2019 10:27:23 +0100	[thread overview]
Message-ID: <f41d47e9-ee6a-7993-4fc3-65b3819c5a53@collabora.com> (raw)
In-Reply-To: <93a38a38-fb86-10aa-291a-1cb9fd86a23c@xs4all.nl>

On 24/05/2019 10:20, Hans Verkuil wrote:
> On 5/24/19 11:05 AM, Guillaume Tucker wrote:
>> On 24/05/2019 09:44, Hans Verkuil wrote:
>>> On 5/24/19 10:38 AM, Guillaume Tucker wrote:
>>>> Hi Hans,
>>>>
>>>> On 16/05/2019 07:41, Hans Verkuil wrote:
>>>>> Hi Guillaume,
>>>>>
>>>>> I have a few questions/suggestions:
>>>>
>>>> Thanks for the feedback!  It's good to start seeing these reports
>>>> on the linux-media mailing list.  And sorry for the slow reply, I
>>>> was away.
>>>>
>>>>> On 5/15/19 9:04 PM, kernelci.org bot wrote:
>>>>>> media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b)
>>>>>>
>>>>>> Test results summary
>>>>>> --------------------
>>>>>>
>>>>>> V4L2 Compliance on the vivid driver.
>>>>>>
>>>>>> This test ran "v4l2-compliance -s" from v4l-utils:
>>>>>>
>>>>>>     https://www.linuxtv.org/wiki/index.php/V4l2-utils
>>>>>
>>>>> I'd just link directly to the git repo instead of the wiki: https://git.linuxtv.org/v4l-utils.git
>>>>
>>>> Sure, I thought this had been agreed before but it's easy to
>>>> change.
>>>>
>>>>> You should add the v4l-utils commit that's used to compile v4l2-compliance.
>>>>> That's important information to have. I assume that this test always uses the
>>>>> latest version of v4l-utils?
>>>>
>>>> This information is in the detailed results, but the detailed
>>>> results are only shown when there are some failures.  So we'll
>>>> rework that a bit.
>>>>
>>>> For example, from the v4l2-compliance-uvc report:
>>>>
>>>>
>>>> Test failures
>>>> ------------- 
>>>> 1  | rk3399-gru-kevin       | arm64 |  52 total:  43 PASS   9 FAIL   0 SKIP
>>>>
>>>>   Config:      defconfig
>>>>   Compiler:    gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0)
>>>>   Lab Name:    lab-collabora
>>>>   Plain log:   https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.txt
>>>>   HTML log:    https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.html
>>>>   Rootfs:      http://storage.kernelci.org/images/rootfs/debian/stretch-v4l2/20190510.0/arm64/rootfs.cpio.gz
>>>>   Test Git:    git://linuxtv.org/v4l-utils.git
>>>>   Test Commit: 0d61ddede7d340ffa1c75a2882e30c455ef3d8b8
>>>>
>>>>
>>>> The git repo and commit hash here show you which version of
>>>> vl4-utils was used.
>>>>
>>>>
>>>> At the moment, the v4l2-compliance is part of a rootfs which gets
>>>> updated each time kernelci.org production code gets updated,
>>>> which is typically once a week.  This can be improved to have the
>>>> rootfs updates independent from the rest, then we could trigger
>>>> rebuilds every time v4l-utils changes, but there are a few things
>>>> to take into consideration before we can do this safely.
>>>
>>> I don't think it has to be updated every time v4l-utils changes, at least
>>> for now, as long as it is clear which v4l-utils version is used.
>>>
>>>>
>>>>>> See each detailed section in the report below to find out the git URL and
>>>>>> particular revision that was used to build the test binaries.
>>>>>>
>>>>>>
>>>>>>   Tree:    media
>>>>>>   Branch:  master
>>>>>>   Kernel:  media_v5.1-2-16-gfc8670d1f72b
>>>>>
>>>>> I assume this is the version of the host kernel, right? Perhaps calling this
>>>>> "Host Kernel:" would be less ambiguous.
>>>>
>>>> I have to say I fail to see any ambiguity here: KernelCI is about
>>>> testing kernels, and this tells you the kernel revision under
>>>> test.  Calling it "host" kernel might actually be confusing when
>>>> running with QEMU as people may think it's the version on the
>>>> host server running the test.
>>>>
>>>>>>   URL:     https://git.linuxtv.org/media_tree.git
>>>>>>   Commit:  fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f
>>>
>>> What confuses me is that the kernel above says: media_v5.1-2-16-gfc8670d1f72b
>>> but the commit says fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f.
>>>
>>> So the hashes are different, which led me to conclude that one referred to
>>> the host kernel and the other to the kernel under test.
>>>
>>> Where does the string "media_v5.1-2-16-gfc8670d1f72b" come from?
>>
>> This is the output of "git describe", except that the slash in
>> the tag name was replaced with an underscore (I know that's not
>> great, but otherwise it causes some issues with the path on the
>> storage server).  The media/v5.2-1 tag has been created since
>> then, but if you run this:
>>
>> $ git checkout fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f
>> $ git tag -d media/v5.2-1
>> $ git describe
>> media/v5.1-2-16-gfc8670d1f72b
>>
>> and fc8670d1f72b is the beginning of the full hash.
> 
> Argh! I totally missed that. And even overlooked that 'g' is not a hex
> number :-)
> 
> I never use git describe, so I wasn't familiar with the format, hence my
> confusion.
> 
> So never mind my comment :-)

No worries, and yeah the hashes are not in heptadecimal :)

Best wishes,
Guillaume

> Regards,
> 
> 	Hans
> 
>>
>>>>>> 1  | qemu                   | arm64 | 118 total: 118 PASS   0 FAIL   0 SKIP
>>>>>> 2  | qemu                   | arm   | 118 total: 118 PASS   0 FAIL   0 SKIP  
>>>>>
>>>>> Even if everything was OK, I think it would still be useful to have a link
>>>>> to the full test report.
>>>>
>>>> Yes, that is essentially the same issue as with the v4l-utils
>>>> version as I described above.  The detailed results show a link
>>>> to the console output, which isn't just a clean v4l2-compliance
>>>> log but it's better than nothing.
>>>>
>>>> We may also add a feature to publish some files alongside the
>>>> parsed test results, and in the case of v4l2-compliance it would
>>>> typically be the plain output of the test suite that developers
>>>> are familiar with.  It's not a supported feature right now as
>>>> only the raw console log is sent from the device to the database.
>>>>
>>>> Best wishes,
>>>> Guillaume
>>>>
>>>
>>> Regards,
>>>
>>> 	Hans
>>>
> 


      reply	other threads:[~2019-05-24  9:27 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-15 19:04 kernelci.org bot
2019-05-16  6:41 ` Hans Verkuil
2019-05-24  8:38   ` Guillaume Tucker
2019-05-24  8:44     ` Hans Verkuil
2019-05-24  9:05       ` Guillaume Tucker
2019-05-24  9:20         ` Hans Verkuil
2019-05-24  9:27           ` Guillaume Tucker [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f41d47e9-ee6a-7993-4fc3-65b3819c5a53@collabora.com \
    --to=guillaume.tucker@collabora.com \
    --cc=hverkuil@xs4all.nl \
    --cc=kernel-build-reports@lists.linaro.org \
    --cc=linux-media@vger.kernel.org \
    --subject='Re: media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b)' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.