* media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) @ 2019-05-15 19:04 kernelci.org bot 2019-05-16 6:41 ` Hans Verkuil 0 siblings, 1 reply; 7+ messages in thread From: kernelci.org bot @ 2019-05-15 19:04 UTC (permalink / raw) To: linux-media, kernel-build-reports media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) Test results summary -------------------- V4L2 Compliance on the vivid driver. This test ran "v4l2-compliance -s" from v4l-utils: https://www.linuxtv.org/wiki/index.php/V4l2-utils See each detailed section in the report below to find out the git URL and particular revision that was used to build the test binaries. Tree: media Branch: master Kernel: media_v5.1-2-16-gfc8670d1f72b URL: https://git.linuxtv.org/media_tree.git Commit: fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f 1 | qemu | arm64 | 118 total: 118 PASS 0 FAIL 0 SKIP 2 | qemu | arm | 118 total: 118 PASS 0 FAIL 0 SKIP ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) 2019-05-15 19:04 media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) kernelci.org bot @ 2019-05-16 6:41 ` Hans Verkuil 2019-05-24 8:38 ` Guillaume Tucker 0 siblings, 1 reply; 7+ messages in thread From: Hans Verkuil @ 2019-05-16 6:41 UTC (permalink / raw) To: Guillaume Tucker, linux-media, kernel-build-reports Hi Guillaume, I have a few questions/suggestions: On 5/15/19 9:04 PM, kernelci.org bot wrote: > media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) > > Test results summary > -------------------- > > V4L2 Compliance on the vivid driver. > > This test ran "v4l2-compliance -s" from v4l-utils: > > https://www.linuxtv.org/wiki/index.php/V4l2-utils I'd just link directly to the git repo instead of the wiki: https://git.linuxtv.org/v4l-utils.git You should add the v4l-utils commit that's used to compile v4l2-compliance. That's important information to have. I assume that this test always uses the latest version of v4l-utils? > > See each detailed section in the report below to find out the git URL and > particular revision that was used to build the test binaries. > > > Tree: media > Branch: master > Kernel: media_v5.1-2-16-gfc8670d1f72b I assume this is the version of the host kernel, right? Perhaps calling this "Host Kernel:" would be less ambiguous. > URL: https://git.linuxtv.org/media_tree.git > Commit: fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f > > > 1 | qemu | arm64 | 118 total: 118 PASS 0 FAIL 0 SKIP > 2 | qemu | arm | 118 total: 118 PASS 0 FAIL 0 SKIP Even if everything was OK, I think it would still be useful to have a link to the full test report. Regards, Hans ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) 2019-05-16 6:41 ` Hans Verkuil @ 2019-05-24 8:38 ` Guillaume Tucker 2019-05-24 8:44 ` Hans Verkuil 0 siblings, 1 reply; 7+ messages in thread From: Guillaume Tucker @ 2019-05-24 8:38 UTC (permalink / raw) To: Hans Verkuil; +Cc: linux-media, kernel-build-reports Hi Hans, On 16/05/2019 07:41, Hans Verkuil wrote: > Hi Guillaume, > > I have a few questions/suggestions: Thanks for the feedback! It's good to start seeing these reports on the linux-media mailing list. And sorry for the slow reply, I was away. > On 5/15/19 9:04 PM, kernelci.org bot wrote: >> media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) >> >> Test results summary >> -------------------- >> >> V4L2 Compliance on the vivid driver. >> >> This test ran "v4l2-compliance -s" from v4l-utils: >> >> https://www.linuxtv.org/wiki/index.php/V4l2-utils > > I'd just link directly to the git repo instead of the wiki: https://git.linuxtv.org/v4l-utils.git Sure, I thought this had been agreed before but it's easy to change. > You should add the v4l-utils commit that's used to compile v4l2-compliance. > That's important information to have. I assume that this test always uses the > latest version of v4l-utils? This information is in the detailed results, but the detailed results are only shown when there are some failures. So we'll rework that a bit. For example, from the v4l2-compliance-uvc report: Test failures ------------- 1 | rk3399-gru-kevin | arm64 | 52 total: 43 PASS 9 FAIL 0 SKIP Config: defconfig Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0) Lab Name: lab-collabora Plain log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.txt HTML log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.html Rootfs: http://storage.kernelci.org/images/rootfs/debian/stretch-v4l2/20190510.0/arm64/rootfs.cpio.gz Test Git: git://linuxtv.org/v4l-utils.git Test Commit: 0d61ddede7d340ffa1c75a2882e30c455ef3d8b8 The git repo and commit hash here show you which version of vl4-utils was used. At the moment, the v4l2-compliance is part of a rootfs which gets updated each time kernelci.org production code gets updated, which is typically once a week. This can be improved to have the rootfs updates independent from the rest, then we could trigger rebuilds every time v4l-utils changes, but there are a few things to take into consideration before we can do this safely. >> See each detailed section in the report below to find out the git URL and >> particular revision that was used to build the test binaries. >> >> >> Tree: media >> Branch: master >> Kernel: media_v5.1-2-16-gfc8670d1f72b > > I assume this is the version of the host kernel, right? Perhaps calling this > "Host Kernel:" would be less ambiguous. I have to say I fail to see any ambiguity here: KernelCI is about testing kernels, and this tells you the kernel revision under test. Calling it "host" kernel might actually be confusing when running with QEMU as people may think it's the version on the host server running the test. >> URL: https://git.linuxtv.org/media_tree.git >> Commit: fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f >> >> >> 1 | qemu | arm64 | 118 total: 118 PASS 0 FAIL 0 SKIP >> 2 | qemu | arm | 118 total: 118 PASS 0 FAIL 0 SKIP > > Even if everything was OK, I think it would still be useful to have a link > to the full test report. Yes, that is essentially the same issue as with the v4l-utils version as I described above. The detailed results show a link to the console output, which isn't just a clean v4l2-compliance log but it's better than nothing. We may also add a feature to publish some files alongside the parsed test results, and in the case of v4l2-compliance it would typically be the plain output of the test suite that developers are familiar with. It's not a supported feature right now as only the raw console log is sent from the device to the database. Best wishes, Guillaume ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) 2019-05-24 8:38 ` Guillaume Tucker @ 2019-05-24 8:44 ` Hans Verkuil 2019-05-24 9:05 ` Guillaume Tucker 0 siblings, 1 reply; 7+ messages in thread From: Hans Verkuil @ 2019-05-24 8:44 UTC (permalink / raw) To: Guillaume Tucker; +Cc: linux-media, kernel-build-reports On 5/24/19 10:38 AM, Guillaume Tucker wrote: > Hi Hans, > > On 16/05/2019 07:41, Hans Verkuil wrote: >> Hi Guillaume, >> >> I have a few questions/suggestions: > > Thanks for the feedback! It's good to start seeing these reports > on the linux-media mailing list. And sorry for the slow reply, I > was away. > >> On 5/15/19 9:04 PM, kernelci.org bot wrote: >>> media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) >>> >>> Test results summary >>> -------------------- >>> >>> V4L2 Compliance on the vivid driver. >>> >>> This test ran "v4l2-compliance -s" from v4l-utils: >>> >>> https://www.linuxtv.org/wiki/index.php/V4l2-utils >> >> I'd just link directly to the git repo instead of the wiki: https://git.linuxtv.org/v4l-utils.git > > Sure, I thought this had been agreed before but it's easy to > change. > >> You should add the v4l-utils commit that's used to compile v4l2-compliance. >> That's important information to have. I assume that this test always uses the >> latest version of v4l-utils? > > This information is in the detailed results, but the detailed > results are only shown when there are some failures. So we'll > rework that a bit. > > For example, from the v4l2-compliance-uvc report: > > > Test failures > ------------- > 1 | rk3399-gru-kevin | arm64 | 52 total: 43 PASS 9 FAIL 0 SKIP > > Config: defconfig > Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0) > Lab Name: lab-collabora > Plain log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.txt > HTML log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.html > Rootfs: http://storage.kernelci.org/images/rootfs/debian/stretch-v4l2/20190510.0/arm64/rootfs.cpio.gz > Test Git: git://linuxtv.org/v4l-utils.git > Test Commit: 0d61ddede7d340ffa1c75a2882e30c455ef3d8b8 > > > The git repo and commit hash here show you which version of > vl4-utils was used. > > > At the moment, the v4l2-compliance is part of a rootfs which gets > updated each time kernelci.org production code gets updated, > which is typically once a week. This can be improved to have the > rootfs updates independent from the rest, then we could trigger > rebuilds every time v4l-utils changes, but there are a few things > to take into consideration before we can do this safely. I don't think it has to be updated every time v4l-utils changes, at least for now, as long as it is clear which v4l-utils version is used. > >>> See each detailed section in the report below to find out the git URL and >>> particular revision that was used to build the test binaries. >>> >>> >>> Tree: media >>> Branch: master >>> Kernel: media_v5.1-2-16-gfc8670d1f72b >> >> I assume this is the version of the host kernel, right? Perhaps calling this >> "Host Kernel:" would be less ambiguous. > > I have to say I fail to see any ambiguity here: KernelCI is about > testing kernels, and this tells you the kernel revision under > test. Calling it "host" kernel might actually be confusing when > running with QEMU as people may think it's the version on the > host server running the test. > >>> URL: https://git.linuxtv.org/media_tree.git >>> Commit: fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f What confuses me is that the kernel above says: media_v5.1-2-16-gfc8670d1f72b but the commit says fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f. So the hashes are different, which led me to conclude that one referred to the host kernel and the other to the kernel under test. Where does the string "media_v5.1-2-16-gfc8670d1f72b" come from? >>> >>> >>> 1 | qemu | arm64 | 118 total: 118 PASS 0 FAIL 0 SKIP >>> 2 | qemu | arm | 118 total: 118 PASS 0 FAIL 0 SKIP >> >> Even if everything was OK, I think it would still be useful to have a link >> to the full test report. > > Yes, that is essentially the same issue as with the v4l-utils > version as I described above. The detailed results show a link > to the console output, which isn't just a clean v4l2-compliance > log but it's better than nothing. > > We may also add a feature to publish some files alongside the > parsed test results, and in the case of v4l2-compliance it would > typically be the plain output of the test suite that developers > are familiar with. It's not a supported feature right now as > only the raw console log is sent from the device to the database. > > Best wishes, > Guillaume > Regards, Hans ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) 2019-05-24 8:44 ` Hans Verkuil @ 2019-05-24 9:05 ` Guillaume Tucker 2019-05-24 9:20 ` Hans Verkuil 0 siblings, 1 reply; 7+ messages in thread From: Guillaume Tucker @ 2019-05-24 9:05 UTC (permalink / raw) To: Hans Verkuil; +Cc: linux-media, kernel-build-reports On 24/05/2019 09:44, Hans Verkuil wrote: > On 5/24/19 10:38 AM, Guillaume Tucker wrote: >> Hi Hans, >> >> On 16/05/2019 07:41, Hans Verkuil wrote: >>> Hi Guillaume, >>> >>> I have a few questions/suggestions: >> >> Thanks for the feedback! It's good to start seeing these reports >> on the linux-media mailing list. And sorry for the slow reply, I >> was away. >> >>> On 5/15/19 9:04 PM, kernelci.org bot wrote: >>>> media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) >>>> >>>> Test results summary >>>> -------------------- >>>> >>>> V4L2 Compliance on the vivid driver. >>>> >>>> This test ran "v4l2-compliance -s" from v4l-utils: >>>> >>>> https://www.linuxtv.org/wiki/index.php/V4l2-utils >>> >>> I'd just link directly to the git repo instead of the wiki: https://git.linuxtv.org/v4l-utils.git >> >> Sure, I thought this had been agreed before but it's easy to >> change. >> >>> You should add the v4l-utils commit that's used to compile v4l2-compliance. >>> That's important information to have. I assume that this test always uses the >>> latest version of v4l-utils? >> >> This information is in the detailed results, but the detailed >> results are only shown when there are some failures. So we'll >> rework that a bit. >> >> For example, from the v4l2-compliance-uvc report: >> >> >> Test failures >> ------------- >> 1 | rk3399-gru-kevin | arm64 | 52 total: 43 PASS 9 FAIL 0 SKIP >> >> Config: defconfig >> Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0) >> Lab Name: lab-collabora >> Plain log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.txt >> HTML log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.html >> Rootfs: http://storage.kernelci.org/images/rootfs/debian/stretch-v4l2/20190510.0/arm64/rootfs.cpio.gz >> Test Git: git://linuxtv.org/v4l-utils.git >> Test Commit: 0d61ddede7d340ffa1c75a2882e30c455ef3d8b8 >> >> >> The git repo and commit hash here show you which version of >> vl4-utils was used. >> >> >> At the moment, the v4l2-compliance is part of a rootfs which gets >> updated each time kernelci.org production code gets updated, >> which is typically once a week. This can be improved to have the >> rootfs updates independent from the rest, then we could trigger >> rebuilds every time v4l-utils changes, but there are a few things >> to take into consideration before we can do this safely. > > I don't think it has to be updated every time v4l-utils changes, at least > for now, as long as it is clear which v4l-utils version is used. > >> >>>> See each detailed section in the report below to find out the git URL and >>>> particular revision that was used to build the test binaries. >>>> >>>> >>>> Tree: media >>>> Branch: master >>>> Kernel: media_v5.1-2-16-gfc8670d1f72b >>> >>> I assume this is the version of the host kernel, right? Perhaps calling this >>> "Host Kernel:" would be less ambiguous. >> >> I have to say I fail to see any ambiguity here: KernelCI is about >> testing kernels, and this tells you the kernel revision under >> test. Calling it "host" kernel might actually be confusing when >> running with QEMU as people may think it's the version on the >> host server running the test. >> >>>> URL: https://git.linuxtv.org/media_tree.git >>>> Commit: fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f > > What confuses me is that the kernel above says: media_v5.1-2-16-gfc8670d1f72b > but the commit says fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f. > > So the hashes are different, which led me to conclude that one referred to > the host kernel and the other to the kernel under test. > > Where does the string "media_v5.1-2-16-gfc8670d1f72b" come from? This is the output of "git describe", except that the slash in the tag name was replaced with an underscore (I know that's not great, but otherwise it causes some issues with the path on the storage server). The media/v5.2-1 tag has been created since then, but if you run this: $ git checkout fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f $ git tag -d media/v5.2-1 $ git describe media/v5.1-2-16-gfc8670d1f72b and fc8670d1f72b is the beginning of the full hash. >>>> 1 | qemu | arm64 | 118 total: 118 PASS 0 FAIL 0 SKIP >>>> 2 | qemu | arm | 118 total: 118 PASS 0 FAIL 0 SKIP >>> >>> Even if everything was OK, I think it would still be useful to have a link >>> to the full test report. >> >> Yes, that is essentially the same issue as with the v4l-utils >> version as I described above. The detailed results show a link >> to the console output, which isn't just a clean v4l2-compliance >> log but it's better than nothing. >> >> We may also add a feature to publish some files alongside the >> parsed test results, and in the case of v4l2-compliance it would >> typically be the plain output of the test suite that developers >> are familiar with. It's not a supported feature right now as >> only the raw console log is sent from the device to the database. >> >> Best wishes, >> Guillaume >> > > Regards, > > Hans > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) 2019-05-24 9:05 ` Guillaume Tucker @ 2019-05-24 9:20 ` Hans Verkuil 2019-05-24 9:27 ` Guillaume Tucker 0 siblings, 1 reply; 7+ messages in thread From: Hans Verkuil @ 2019-05-24 9:20 UTC (permalink / raw) To: Guillaume Tucker; +Cc: linux-media, kernel-build-reports On 5/24/19 11:05 AM, Guillaume Tucker wrote: > On 24/05/2019 09:44, Hans Verkuil wrote: >> On 5/24/19 10:38 AM, Guillaume Tucker wrote: >>> Hi Hans, >>> >>> On 16/05/2019 07:41, Hans Verkuil wrote: >>>> Hi Guillaume, >>>> >>>> I have a few questions/suggestions: >>> >>> Thanks for the feedback! It's good to start seeing these reports >>> on the linux-media mailing list. And sorry for the slow reply, I >>> was away. >>> >>>> On 5/15/19 9:04 PM, kernelci.org bot wrote: >>>>> media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) >>>>> >>>>> Test results summary >>>>> -------------------- >>>>> >>>>> V4L2 Compliance on the vivid driver. >>>>> >>>>> This test ran "v4l2-compliance -s" from v4l-utils: >>>>> >>>>> https://www.linuxtv.org/wiki/index.php/V4l2-utils >>>> >>>> I'd just link directly to the git repo instead of the wiki: https://git.linuxtv.org/v4l-utils.git >>> >>> Sure, I thought this had been agreed before but it's easy to >>> change. >>> >>>> You should add the v4l-utils commit that's used to compile v4l2-compliance. >>>> That's important information to have. I assume that this test always uses the >>>> latest version of v4l-utils? >>> >>> This information is in the detailed results, but the detailed >>> results are only shown when there are some failures. So we'll >>> rework that a bit. >>> >>> For example, from the v4l2-compliance-uvc report: >>> >>> >>> Test failures >>> ------------- >>> 1 | rk3399-gru-kevin | arm64 | 52 total: 43 PASS 9 FAIL 0 SKIP >>> >>> Config: defconfig >>> Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0) >>> Lab Name: lab-collabora >>> Plain log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.txt >>> HTML log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.html >>> Rootfs: http://storage.kernelci.org/images/rootfs/debian/stretch-v4l2/20190510.0/arm64/rootfs.cpio.gz >>> Test Git: git://linuxtv.org/v4l-utils.git >>> Test Commit: 0d61ddede7d340ffa1c75a2882e30c455ef3d8b8 >>> >>> >>> The git repo and commit hash here show you which version of >>> vl4-utils was used. >>> >>> >>> At the moment, the v4l2-compliance is part of a rootfs which gets >>> updated each time kernelci.org production code gets updated, >>> which is typically once a week. This can be improved to have the >>> rootfs updates independent from the rest, then we could trigger >>> rebuilds every time v4l-utils changes, but there are a few things >>> to take into consideration before we can do this safely. >> >> I don't think it has to be updated every time v4l-utils changes, at least >> for now, as long as it is clear which v4l-utils version is used. >> >>> >>>>> See each detailed section in the report below to find out the git URL and >>>>> particular revision that was used to build the test binaries. >>>>> >>>>> >>>>> Tree: media >>>>> Branch: master >>>>> Kernel: media_v5.1-2-16-gfc8670d1f72b >>>> >>>> I assume this is the version of the host kernel, right? Perhaps calling this >>>> "Host Kernel:" would be less ambiguous. >>> >>> I have to say I fail to see any ambiguity here: KernelCI is about >>> testing kernels, and this tells you the kernel revision under >>> test. Calling it "host" kernel might actually be confusing when >>> running with QEMU as people may think it's the version on the >>> host server running the test. >>> >>>>> URL: https://git.linuxtv.org/media_tree.git >>>>> Commit: fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f >> >> What confuses me is that the kernel above says: media_v5.1-2-16-gfc8670d1f72b >> but the commit says fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f. >> >> So the hashes are different, which led me to conclude that one referred to >> the host kernel and the other to the kernel under test. >> >> Where does the string "media_v5.1-2-16-gfc8670d1f72b" come from? > > This is the output of "git describe", except that the slash in > the tag name was replaced with an underscore (I know that's not > great, but otherwise it causes some issues with the path on the > storage server). The media/v5.2-1 tag has been created since > then, but if you run this: > > $ git checkout fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f > $ git tag -d media/v5.2-1 > $ git describe > media/v5.1-2-16-gfc8670d1f72b > > and fc8670d1f72b is the beginning of the full hash. Argh! I totally missed that. And even overlooked that 'g' is not a hex number :-) I never use git describe, so I wasn't familiar with the format, hence my confusion. So never mind my comment :-) Regards, Hans > >>>>> 1 | qemu | arm64 | 118 total: 118 PASS 0 FAIL 0 SKIP >>>>> 2 | qemu | arm | 118 total: 118 PASS 0 FAIL 0 SKIP >>>> >>>> Even if everything was OK, I think it would still be useful to have a link >>>> to the full test report. >>> >>> Yes, that is essentially the same issue as with the v4l-utils >>> version as I described above. The detailed results show a link >>> to the console output, which isn't just a clean v4l2-compliance >>> log but it's better than nothing. >>> >>> We may also add a feature to publish some files alongside the >>> parsed test results, and in the case of v4l2-compliance it would >>> typically be the plain output of the test suite that developers >>> are familiar with. It's not a supported feature right now as >>> only the raw console log is sent from the device to the database. >>> >>> Best wishes, >>> Guillaume >>> >> >> Regards, >> >> Hans >> ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) 2019-05-24 9:20 ` Hans Verkuil @ 2019-05-24 9:27 ` Guillaume Tucker 0 siblings, 0 replies; 7+ messages in thread From: Guillaume Tucker @ 2019-05-24 9:27 UTC (permalink / raw) To: Hans Verkuil; +Cc: linux-media, kernel-build-reports On 24/05/2019 10:20, Hans Verkuil wrote: > On 5/24/19 11:05 AM, Guillaume Tucker wrote: >> On 24/05/2019 09:44, Hans Verkuil wrote: >>> On 5/24/19 10:38 AM, Guillaume Tucker wrote: >>>> Hi Hans, >>>> >>>> On 16/05/2019 07:41, Hans Verkuil wrote: >>>>> Hi Guillaume, >>>>> >>>>> I have a few questions/suggestions: >>>> >>>> Thanks for the feedback! It's good to start seeing these reports >>>> on the linux-media mailing list. And sorry for the slow reply, I >>>> was away. >>>> >>>>> On 5/15/19 9:04 PM, kernelci.org bot wrote: >>>>>> media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) >>>>>> >>>>>> Test results summary >>>>>> -------------------- >>>>>> >>>>>> V4L2 Compliance on the vivid driver. >>>>>> >>>>>> This test ran "v4l2-compliance -s" from v4l-utils: >>>>>> >>>>>> https://www.linuxtv.org/wiki/index.php/V4l2-utils >>>>> >>>>> I'd just link directly to the git repo instead of the wiki: https://git.linuxtv.org/v4l-utils.git >>>> >>>> Sure, I thought this had been agreed before but it's easy to >>>> change. >>>> >>>>> You should add the v4l-utils commit that's used to compile v4l2-compliance. >>>>> That's important information to have. I assume that this test always uses the >>>>> latest version of v4l-utils? >>>> >>>> This information is in the detailed results, but the detailed >>>> results are only shown when there are some failures. So we'll >>>> rework that a bit. >>>> >>>> For example, from the v4l2-compliance-uvc report: >>>> >>>> >>>> Test failures >>>> ------------- >>>> 1 | rk3399-gru-kevin | arm64 | 52 total: 43 PASS 9 FAIL 0 SKIP >>>> >>>> Config: defconfig >>>> Compiler: gcc-8 (aarch64-linux-gnu-gcc (Debian 8.3.0-2) 8.3.0) >>>> Lab Name: lab-collabora >>>> Plain log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.txt >>>> HTML log: https://storage.kernelci.org//media/master/media_v5.1-2-16-gfc8670d1f72b/arm64/defconfig/gcc-8/lab-collabora/v4l2-compliance-uvc-rk3399-gru-kevin.html >>>> Rootfs: http://storage.kernelci.org/images/rootfs/debian/stretch-v4l2/20190510.0/arm64/rootfs.cpio.gz >>>> Test Git: git://linuxtv.org/v4l-utils.git >>>> Test Commit: 0d61ddede7d340ffa1c75a2882e30c455ef3d8b8 >>>> >>>> >>>> The git repo and commit hash here show you which version of >>>> vl4-utils was used. >>>> >>>> >>>> At the moment, the v4l2-compliance is part of a rootfs which gets >>>> updated each time kernelci.org production code gets updated, >>>> which is typically once a week. This can be improved to have the >>>> rootfs updates independent from the rest, then we could trigger >>>> rebuilds every time v4l-utils changes, but there are a few things >>>> to take into consideration before we can do this safely. >>> >>> I don't think it has to be updated every time v4l-utils changes, at least >>> for now, as long as it is clear which v4l-utils version is used. >>> >>>> >>>>>> See each detailed section in the report below to find out the git URL and >>>>>> particular revision that was used to build the test binaries. >>>>>> >>>>>> >>>>>> Tree: media >>>>>> Branch: master >>>>>> Kernel: media_v5.1-2-16-gfc8670d1f72b >>>>> >>>>> I assume this is the version of the host kernel, right? Perhaps calling this >>>>> "Host Kernel:" would be less ambiguous. >>>> >>>> I have to say I fail to see any ambiguity here: KernelCI is about >>>> testing kernels, and this tells you the kernel revision under >>>> test. Calling it "host" kernel might actually be confusing when >>>> running with QEMU as people may think it's the version on the >>>> host server running the test. >>>> >>>>>> URL: https://git.linuxtv.org/media_tree.git >>>>>> Commit: fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f >>> >>> What confuses me is that the kernel above says: media_v5.1-2-16-gfc8670d1f72b >>> but the commit says fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f. >>> >>> So the hashes are different, which led me to conclude that one referred to >>> the host kernel and the other to the kernel under test. >>> >>> Where does the string "media_v5.1-2-16-gfc8670d1f72b" come from? >> >> This is the output of "git describe", except that the slash in >> the tag name was replaced with an underscore (I know that's not >> great, but otherwise it causes some issues with the path on the >> storage server). The media/v5.2-1 tag has been created since >> then, but if you run this: >> >> $ git checkout fc8670d1f72b746ff3a5fe441f1fca4c4dba0e6f >> $ git tag -d media/v5.2-1 >> $ git describe >> media/v5.1-2-16-gfc8670d1f72b >> >> and fc8670d1f72b is the beginning of the full hash. > > Argh! I totally missed that. And even overlooked that 'g' is not a hex > number :-) > > I never use git describe, so I wasn't familiar with the format, hence my > confusion. > > So never mind my comment :-) No worries, and yeah the hashes are not in heptadecimal :) Best wishes, Guillaume > Regards, > > Hans > >> >>>>>> 1 | qemu | arm64 | 118 total: 118 PASS 0 FAIL 0 SKIP >>>>>> 2 | qemu | arm | 118 total: 118 PASS 0 FAIL 0 SKIP >>>>> >>>>> Even if everything was OK, I think it would still be useful to have a link >>>>> to the full test report. >>>> >>>> Yes, that is essentially the same issue as with the v4l-utils >>>> version as I described above. The detailed results show a link >>>> to the console output, which isn't just a clean v4l2-compliance >>>> log but it's better than nothing. >>>> >>>> We may also add a feature to publish some files alongside the >>>> parsed test results, and in the case of v4l2-compliance it would >>>> typically be the plain output of the test suite that developers >>>> are familiar with. It's not a supported feature right now as >>>> only the raw console log is sent from the device to the database. >>>> >>>> Best wishes, >>>> Guillaume >>>> >>> >>> Regards, >>> >>> Hans >>> > ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2019-05-24 9:27 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-05-15 19:04 media/master v4l2-compliance on vivid: 236 tests, 0 regressions (media_v5.1-2-16-gfc8670d1f72b) kernelci.org bot 2019-05-16 6:41 ` Hans Verkuil 2019-05-24 8:38 ` Guillaume Tucker 2019-05-24 8:44 ` Hans Verkuil 2019-05-24 9:05 ` Guillaume Tucker 2019-05-24 9:20 ` Hans Verkuil 2019-05-24 9:27 ` Guillaume Tucker
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.