* KCIDB engagement report #kcidb
@ 2021-01-21 10:07 Nikolai Kondrashov
2021-01-21 10:17 ` Mathieu Acher
2021-01-21 11:01 ` [Automated-testing] " Richard Purdie
0 siblings, 2 replies; 7+ messages in thread
From: Nikolai Kondrashov @ 2021-01-21 10:07 UTC (permalink / raw)
To: kernelci, automated-testing
Hi everyone,
Below is the monthly report on KCIDB* engagement. It lists various CI systems
and their status of engagement with KCIDB, and once we get to that, will list
developer engagement.
Lines with updates are marked with "!".
The main news is that Gentoo GKernelCI have started sending their preliminary
data to our "playground" setup. Linaro have also started work on sending
tuxmake build results, but no reports have arrived yet.
KernelCI native
Sending staging results.
Transitioning to production results is at 50%
(KCIDB support done, backend work started).
https://staging.kernelci.org:3000/?var-origin=kernelci
Red Hat CKI
Sending production results.
https://staging.kernelci.org:3000/?var-origin=redhat
Google Syzbot
Sending a subset of production results (failures only).
https://staging.kernelci.org:3000/?var-origin=syzbot
ARM
Sending production results to "production" KCIDB.
Full commit hashes are currently not available, are spoofed, and don't
match the ones reported by others. To be fixed soon.
https://staging.kernelci.org:3000/?var-origin=arm
Sony Fuego
Internal design in progress.
! Gentoo GKernelCI
! Preliminary data (revisions and builds) have started flowing into
! "playground" KCIDB. No timestamps yet.
! https://staging.kernelci.org:3000/d/home/home?var-origin=gkernelci&var-dataset=playground_kernelci04
Intel 0day
Initial conversation concluded, general interest expressed,
no contact since.
! Linaro
! Work has started on sending tuxmake build results
! (a part of https://tuxsuite.com/). No reports have arrived yet.
Please respond with corrections or suggestions of other CI systems to contact.
Nick
*KCIDB is an effort to unify Linux Kernel CI reporting, maintained by Linux
Foundation's KernelCI project:
https://foundation.kernelci.org/blog/2020/08/21/introducing-common-reporting/
^ permalink raw reply [flat|nested] 7+ messages in thread* Re: KCIDB engagement report #kcidb 2021-01-21 10:07 KCIDB engagement report #kcidb Nikolai Kondrashov @ 2021-01-21 10:17 ` Mathieu Acher 2021-01-21 10:26 ` Nikolai Kondrashov 2021-01-21 11:01 ` [Automated-testing] " Richard Purdie 1 sibling, 1 reply; 7+ messages in thread From: Mathieu Acher @ 2021-01-21 10:17 UTC (permalink / raw) To: kernelci, Nikolai Kondrashov; +Cc: automated-testing Hi, TuxML https://github.com/TuxML/tuxml has a plan to synchronize with KCIDB in the upcoming months. Our focus is on testing random configurations of the Linux kernel in the large. We have two use cases (at least): * configuration bug prevention/prediction/fixing (when the kernel config. does not build) * binary kernel size prediction (for whatever kernel version and compiler) We have some data here: https://tuxmlweb.istic.univ-rennes1.fr/ We plan to discuss this in february with KernelCI's team for further refining how we can integrate/interoperate. Best, -- Dr. Mathieu ACHER, Associate Professor Univ Rennes, Inria, CNRS, IRISA, France (DiverSE team) http://www.mathieuacher.com/ ----- Mail original ----- > De: "Nikolai Kondrashov" <Nikolai.Kondrashov@redhat.com> > À: "kernelci" <kernelci@groups.io>, automated-testing@yoctoproject.org > Envoyé: Jeudi 21 Janvier 2021 11:07:02 > Objet: KCIDB engagement report > Hi everyone, > > Below is the monthly report on KCIDB* engagement. It lists various CI systems > and their status of engagement with KCIDB, and once we get to that, will list > developer engagement. > > Lines with updates are marked with "!". > > The main news is that Gentoo GKernelCI have started sending their preliminary > data to our "playground" setup. Linaro have also started work on sending > tuxmake build results, but no reports have arrived yet. > > KernelCI native > Sending staging results. > Transitioning to production results is at 50% > (KCIDB support done, backend work started). > https://staging.kernelci.org:3000/?var-origin=kernelci > > Red Hat CKI > Sending production results. > https://staging.kernelci.org:3000/?var-origin=redhat > > Google Syzbot > Sending a subset of production results (failures only). > https://staging.kernelci.org:3000/?var-origin=syzbot > > ARM > Sending production results to "production" KCIDB. > Full commit hashes are currently not available, are spoofed, and don't > match the ones reported by others. To be fixed soon. > https://staging.kernelci.org:3000/?var-origin=arm > > Sony Fuego > Internal design in progress. > > ! Gentoo GKernelCI > ! Preliminary data (revisions and builds) have started flowing into > ! "playground" KCIDB. No timestamps yet. > ! > https://staging.kernelci.org:3000/d/home/home?var-origin=gkernelci&var-dataset=playground_kernelci04 > > Intel 0day > Initial conversation concluded, general interest expressed, > no contact since. > > ! Linaro > ! Work has started on sending tuxmake build results > ! (a part of https://tuxsuite.com/). No reports have arrived yet. > > Please respond with corrections or suggestions of other CI systems to contact. > > Nick > > *KCIDB is an effort to unify Linux Kernel CI reporting, maintained by Linux > Foundation's KernelCI project: > https://foundation.kernelci.org/blog/2020/08/21/introducing-common-reporting/ > > > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: KCIDB engagement report #kcidb 2021-01-21 10:17 ` Mathieu Acher @ 2021-01-21 10:26 ` Nikolai Kondrashov 0 siblings, 0 replies; 7+ messages in thread From: Nikolai Kondrashov @ 2021-01-21 10:26 UTC (permalink / raw) To: kernelci, mathieu.acher; +Cc: automated-testing Hi Mathieu, On 1/21/21 12:17 PM, Mathieu Acher wrote: > TuxML https://github.com/TuxML/tuxml has a plan to synchronize with KCIDB in the upcoming months. Wonderful, would be happy to help you! Don't hesitate to contact us on the maillist and on the freenode's #kernelci channel, anytime! Nick On 1/21/21 12:17 PM, Mathieu Acher wrote: > Hi, > > TuxML https://github.com/TuxML/tuxml has a plan to synchronize with KCIDB in the upcoming months. > Our focus is on testing random configurations of the Linux kernel in the large. > We have two use cases (at least): > * configuration bug prevention/prediction/fixing (when the kernel config. does not build) > * binary kernel size prediction (for whatever kernel version and compiler) > > We have some data here: https://tuxmlweb.istic.univ-rennes1.fr/ > We plan to discuss this in february with KernelCI's team for further refining how we can integrate/interoperate. > > Best, > > -- > Dr. Mathieu ACHER, Associate Professor > Univ Rennes, Inria, CNRS, IRISA, France (DiverSE team) > http://www.mathieuacher.com/ > > ----- Mail original ----- >> De: "Nikolai Kondrashov" <Nikolai.Kondrashov@redhat.com> >> À: "kernelci" <kernelci@groups.io>, automated-testing@yoctoproject.org >> Envoyé: Jeudi 21 Janvier 2021 11:07:02 >> Objet: KCIDB engagement report > >> Hi everyone, >> >> Below is the monthly report on KCIDB* engagement. It lists various CI systems >> and their status of engagement with KCIDB, and once we get to that, will list >> developer engagement. >> >> Lines with updates are marked with "!". >> >> The main news is that Gentoo GKernelCI have started sending their preliminary >> data to our "playground" setup. Linaro have also started work on sending >> tuxmake build results, but no reports have arrived yet. >> >> KernelCI native >> Sending staging results. >> Transitioning to production results is at 50% >> (KCIDB support done, backend work started). >> https://staging.kernelci.org:3000/?var-origin=kernelci >> >> Red Hat CKI >> Sending production results. >> https://staging.kernelci.org:3000/?var-origin=redhat >> >> Google Syzbot >> Sending a subset of production results (failures only). >> https://staging.kernelci.org:3000/?var-origin=syzbot >> >> ARM >> Sending production results to "production" KCIDB. >> Full commit hashes are currently not available, are spoofed, and don't >> match the ones reported by others. To be fixed soon. >> https://staging.kernelci.org:3000/?var-origin=arm >> >> Sony Fuego >> Internal design in progress. >> >> ! Gentoo GKernelCI >> ! Preliminary data (revisions and builds) have started flowing into >> ! "playground" KCIDB. No timestamps yet. >> ! >> https://staging.kernelci.org:3000/d/home/home?var-origin=gkernelci&var-dataset=playground_kernelci04 >> >> Intel 0day >> Initial conversation concluded, general interest expressed, >> no contact since. >> >> ! Linaro >> ! Work has started on sending tuxmake build results >> ! (a part of https://tuxsuite.com/). No reports have arrived yet. >> >> Please respond with corrections or suggestions of other CI systems to contact. >> >> Nick >> >> *KCIDB is an effort to unify Linux Kernel CI reporting, maintained by Linux >> Foundation's KernelCI project: >> https://foundation.kernelci.org/blog/2020/08/21/introducing-common-reporting/ >> >> >> >> > > > > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Automated-testing] KCIDB engagement report #kcidb 2021-01-21 10:07 KCIDB engagement report #kcidb Nikolai Kondrashov 2021-01-21 10:17 ` Mathieu Acher @ 2021-01-21 11:01 ` Richard Purdie 2021-01-21 12:46 ` Nikolai Kondrashov 1 sibling, 1 reply; 7+ messages in thread From: Richard Purdie @ 2021-01-21 11:01 UTC (permalink / raw) To: Nikolai Kondrashov, kernelci, automated-testing; +Cc: Bruce Ashfield On Thu, 2021-01-21 at 12:07 +0200, Nikolai Kondrashov wrote: > Hi everyone, > > Below is the monthly report on KCIDB* engagement. It lists various CI systems > and their status of engagement with KCIDB, and once we get to that, will list > developer engagement. > > Lines with updates are marked with "!". > > The main news is that Gentoo GKernelCI have started sending their preliminary > data to our "playground" setup. Linaro have also started work on sending > tuxmake build results, but no reports have arrived yet. > > KernelCI native > Sending staging results. > Transitioning to production results is at 50% > (KCIDB support done, backend work started). > https://staging.kernelci.org:3000/?var-origin=kernelci > > Red Hat CKI > Sending production results. > https://staging.kernelci.org:3000/?var-origin=redhat > > Google Syzbot > Sending a subset of production results (failures only). > https://staging.kernelci.org:3000/?var-origin=syzbot > > ARM > Sending production results to "production" KCIDB. > Full commit hashes are currently not available, are spoofed, and don't > match the ones reported by others. To be fixed soon. > https://staging.kernelci.org:3000/?var-origin=arm > > Sony Fuego > Internal design in progress. > > ! Gentoo GKernelCI > ! Preliminary data (revisions and builds) have started flowing into > ! "playground" KCIDB. No timestamps yet. > ! https://staging.kernelci.org:3000/d/home/home?var-origin=gkernelci&var-dataset=playground_kernelci04 > > Intel 0day > Initial conversation concluded, general interest expressed, > no contact since. > > ! Linaro > ! Work has started on sending tuxmake build results > ! (a part of https://tuxsuite.com/). No reports have arrived yet. > > Please respond with corrections or suggestions of other CI systems to contact. I noticed Yocto Project isn't mentioned here but we do a lot of builds of complete systems for multiple architectures and we include kernels in those. We probably have two kernel versions under testing in any given project release series. Would it be useful to get data from us? If so, what kind of data? We're generally building linux-yocto from: https://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto/ which is derived from upstream kernel releases as their baseline. Testing wise, we build them, then usually boot them under qemu with a variety of userspaces. We also run ltp, some lsb and the tests from a variety of projects like lttng. An example output report we use is: https://autobuilder.yocto.io/pub/non-release/20210120-9/testresults/testresult-report.txt So far I'd assumed you were interested in bleeding edge kernel CI but if there is interest around general kernel testing of older kernels, we do have a lot of data. There are some details we don't currently log (e.g. which kernel version in in a given test target) but we can change that if we need it. Cheers, Richard ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Automated-testing] KCIDB engagement report #kcidb 2021-01-21 11:01 ` [Automated-testing] " Richard Purdie @ 2021-01-21 12:46 ` Nikolai Kondrashov 2021-01-26 15:10 ` [Automated-testing] KCIDB engagement report Richard Purdie 0 siblings, 1 reply; 7+ messages in thread From: Nikolai Kondrashov @ 2021-01-21 12:46 UTC (permalink / raw) To: kernelci, richard.purdie, automated-testing; +Cc: Bruce Ashfield Hi Richard, On 1/21/21 1:01 PM, Richard Purdie wrote: > On Thu, 2021-01-21 at 12:07 +0200, Nikolai Kondrashov wrote: >> Please respond with corrections or suggestions of other CI systems to contact. > > I noticed Yocto Project isn't mentioned here but we do a lot of builds > of complete systems for multiple architectures and we include kernels > in those. We probably have two kernel versions under testing in any > given project release series. > > Would it be useful to get data from us? If so, what kind of data? Absolutely! Thanks for reaching out :) > We're generally building linux-yocto from: > https://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto/ > which is derived from upstream kernel releases as their baseline. > > Testing wise, we build them, then usually boot them under qemu with a > variety of userspaces. We also run ltp, some lsb and the tests from a > variety of projects like lttng. > > An example output report we use is: > > https://autobuilder.yocto.io/pub/non-release/20210120-9/testresults/testresult-report.txt > > So far I'd assumed you were interested in bleeding edge kernel CI but > if there is interest around general kernel testing of older kernels, we > do have a lot of data. There are some details we don't currently log > (e.g. which kernel version in in a given test target) but we can change > that if we need it. Wow, that's a lot of tests, very cool! We would definitely welcome reports for older kernel versions as well. Any data helping track down issues is welcome. We could use build/test results, and logs. It would help if you could separate the upstream commit your branches are based on and the patches you apply on top, and identify them separately. However, it's OK if you can't, only mention the base version (e.g. using "git describe"), and otherwise refer to your own repo and commits directly. In any case, that would be good enough for the start. We're still in research stage (and will be for a while), trying to bring the data from various sources together and do something with it. Our strategy is get the data first, organize later :) We would be glad to have you on board and sending whatever data you have. Please take a look at our Submission HOWTO for an overview of how things work: https://github.com/kernelci/kcidb/blob/main/SUBMISSION_HOWTO.md If you'd like to try it out, I can send you credentials and parameters for submitting to our "playground" instance. You can submit data there (either manually or automatically) and experiment freely without worrying about breaking anything. The data will appear on this dashboard (GKernelCI is currently submitting there): https://staging.kernelci.org:3000/d/home/home?var-dataset=playground_kernelci04 Once you're confident with how things work, it would only take a permission tweak on my side and one parameter change on your side to switch to production. Don't hesitate to ping us on the maillist, or on the freenode's #kernelci channel! Nick ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Automated-testing] KCIDB engagement report 2021-01-21 12:46 ` Nikolai Kondrashov @ 2021-01-26 15:10 ` Richard Purdie 2021-01-26 15:19 ` Nikolai Kondrashov 0 siblings, 1 reply; 7+ messages in thread From: Richard Purdie @ 2021-01-26 15:10 UTC (permalink / raw) To: Nikolai Kondrashov, kernelci, automated-testing; +Cc: Bruce Ashfield On Thu, 2021-01-21 at 14:46 +0200, Nikolai Kondrashov wrote: > Hi Richard, > > On 1/21/21 1:01 PM, Richard Purdie wrote: > > On Thu, 2021-01-21 at 12:07 +0200, Nikolai Kondrashov wrote: > >> Please respond with corrections or suggestions of other CI systems to contact. > > > > I noticed Yocto Project isn't mentioned here but we do a lot of builds > > of complete systems for multiple architectures and we include kernels > > in those. We probably have two kernel versions under testing in any > > given project release series. > > > > Would it be useful to get data from us? If so, what kind of data? > > Absolutely! Thanks for reaching out :) > > > We're generally building linux-yocto from: > > https://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto/ > > which is derived from upstream kernel releases as their baseline. > > > > Testing wise, we build them, then usually boot them under qemu with a > > variety of userspaces. We also run ltp, some lsb and the tests from a > > variety of projects like lttng. > > > > An example output report we use is: > > > > https://autobuilder.yocto.io/pub/non-release/20210120-9/testresults/testresult-report.txt > > > > So far I'd assumed you were interested in bleeding edge kernel CI but > > if there is interest around general kernel testing of older kernels, we > > do have a lot of data. There are some details we don't currently log > > (e.g. which kernel version in in a given test target) but we can change > > that if we need it. > > Wow, that's a lot of tests, very cool! > > We would definitely welcome reports for older kernel versions as well. > Any data helping track down issues is welcome. > > We could use build/test results, and logs. > > It would help if you could separate the upstream commit your branches are > based on and the patches you apply on top, and identify them separately. > However, it's OK if you can't, only mention the base version (e.g. using "git > describe"), and otherwise refer to your own repo and commits directly. > > In any case, that would be good enough for the start. We're still in research > stage (and will be for a while), trying to bring the data from various sources > together and do something with it. > > Our strategy is get the data first, organize later :) > > We would be glad to have you on board and sending whatever data you have. > > Please take a look at our Submission HOWTO for an overview of how things work: > > https://github.com/kernelci/kcidb/blob/main/SUBMISSION_HOWTO.md > > If you'd like to try it out, I can send you credentials and parameters for > submitting to our "playground" instance. You can submit data there (either > manually or automatically) and experiment freely without worrying about > breaking anything. The data will appear on this dashboard (GKernelCI is > currently submitting there): > > https://staging.kernelci.org:3000/d/home/home?var-dataset=playground_kernelci04 > > Once you're confident with how things work, it would only take a permission > tweak on my side and one parameter change on your side to switch to > production. > > Don't hesitate to ping us on the maillist, > or on the freenode's #kernelci channel! Thanks, that is all good info. The biggest issue right now is we don't record the kernel version information anywhere that our test results processing is going to be able to find it. That isn't too hard to fix but there are a few systems involved. I'm not sure when or who will have the time to make this happen but I do like the idea and don't want it to get lost. I put an entry into our bug/feature tracker: https://bugzilla.yoctoproject.org/show_bug.cgi?id=14196 Cheers, Richard ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [Automated-testing] KCIDB engagement report 2021-01-26 15:10 ` [Automated-testing] KCIDB engagement report Richard Purdie @ 2021-01-26 15:19 ` Nikolai Kondrashov 0 siblings, 0 replies; 7+ messages in thread From: Nikolai Kondrashov @ 2021-01-26 15:19 UTC (permalink / raw) To: Richard Purdie, kernelci, automated-testing; +Cc: Bruce Ashfield Hi Richard, On 1/26/21 5:10 PM, Richard Purdie wrote: > Thanks, that is all good info. The biggest issue right now is we don't > record the kernel version information anywhere that our test results > processing is going to be able to find it. That isn't too hard to fix > but there are a few systems involved. > > I'm not sure when or who will have the time to make this happen but I > do like the idea and don't want it to get lost. I put an entry into our > bug/feature tracker: > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14196 Great! Don't hesitate to write if you'd like to discuss something, and feel free to join either one of the monthly "Automated Testing" conference calls, or a weekly KernelCI call, if you'd like a quick high-bandwidth chat :) Nick On 1/26/21 5:10 PM, Richard Purdie wrote: > On Thu, 2021-01-21 at 14:46 +0200, Nikolai Kondrashov wrote: >> Hi Richard, >> >> On 1/21/21 1:01 PM, Richard Purdie wrote: >> > On Thu, 2021-01-21 at 12:07 +0200, Nikolai Kondrashov wrote: >> >> Please respond with corrections or suggestions of other CI systems to contact. >> > >> > I noticed Yocto Project isn't mentioned here but we do a lot of builds >> > of complete systems for multiple architectures and we include kernels >> > in those. We probably have two kernel versions under testing in any >> > given project release series. >> > >> > Would it be useful to get data from us? If so, what kind of data? >> >> Absolutely! Thanks for reaching out :) >> >> > We're generally building linux-yocto from: >> > https://git.yoctoproject.org/cgit/cgit.cgi/linux-yocto/ >> > which is derived from upstream kernel releases as their baseline. >> > >> > Testing wise, we build them, then usually boot them under qemu with a >> > variety of userspaces. We also run ltp, some lsb and the tests from a >> > variety of projects like lttng. >> > >> > An example output report we use is: >> > >> > https://autobuilder.yocto.io/pub/non-release/20210120-9/testresults/testresult-report.txt >> > >> > So far I'd assumed you were interested in bleeding edge kernel CI but >> > if there is interest around general kernel testing of older kernels, we >> > do have a lot of data. There are some details we don't currently log >> > (e.g. which kernel version in in a given test target) but we can change >> > that if we need it. >> >> Wow, that's a lot of tests, very cool! >> >> We would definitely welcome reports for older kernel versions as well. >> Any data helping track down issues is welcome. >> >> We could use build/test results, and logs. >> >> It would help if you could separate the upstream commit your branches are >> based on and the patches you apply on top, and identify them separately. >> However, it's OK if you can't, only mention the base version (e.g. using "git >> describe"), and otherwise refer to your own repo and commits directly. >> >> In any case, that would be good enough for the start. We're still in research >> stage (and will be for a while), trying to bring the data from various sources >> together and do something with it. >> >> Our strategy is get the data first, organize later :) >> >> We would be glad to have you on board and sending whatever data you have. >> >> Please take a look at our Submission HOWTO for an overview of how things work: >> >> https://github.com/kernelci/kcidb/blob/main/SUBMISSION_HOWTO.md >> >> If you'd like to try it out, I can send you credentials and parameters for >> submitting to our "playground" instance. You can submit data there (either >> manually or automatically) and experiment freely without worrying about >> breaking anything. The data will appear on this dashboard (GKernelCI is >> currently submitting there): >> >> https://staging.kernelci.org:3000/d/home/home?var-dataset=playground_kernelci04 >> >> Once you're confident with how things work, it would only take a permission >> tweak on my side and one parameter change on your side to switch to >> production. >> >> Don't hesitate to ping us on the maillist, >> or on the freenode's #kernelci channel! > > Thanks, that is all good info. The biggest issue right now is we don't > record the kernel version information anywhere that our test results > processing is going to be able to find it. That isn't too hard to fix > but there are a few systems involved. > > I'm not sure when or who will have the time to make this happen but I > do like the idea and don't want it to get lost. I put an entry into our > bug/feature tracker: > > https://bugzilla.yoctoproject.org/show_bug.cgi?id=14196 > > Cheers, > > Richard > ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-01-26 15:19 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-01-21 10:07 KCIDB engagement report #kcidb Nikolai Kondrashov 2021-01-21 10:17 ` Mathieu Acher 2021-01-21 10:26 ` Nikolai Kondrashov 2021-01-21 11:01 ` [Automated-testing] " Richard Purdie 2021-01-21 12:46 ` Nikolai Kondrashov 2021-01-26 15:10 ` [Automated-testing] KCIDB engagement report Richard Purdie 2021-01-26 15:19 ` Nikolai Kondrashov
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.