* Re: #KCIDB v9 release
[not found] <16BCB376A7748BB3.24346@groups.io>
@ 2022-01-03 11:49 ` Nikolai Kondrashov
2022-01-05 10:10 ` Nikolai Kondrashov
0 siblings, 1 reply; 7+ messages in thread
From: Nikolai Kondrashov @ 2022-01-03 11:49 UTC (permalink / raw)
To: Cristian Marussi, Tim Bird, Dmitry Vyukov, Guillaume Tucker,
Alice Ferrazzi, Iñaki Malerba, kernelci, Vishal Bhoj,
automated-testing
Hi everyone,
On 12/1/21 20:01, Nikolai Kondrashov wrote:
> I'm proud to announce the release and deployment of KCIDB v9.
>
> Please see the detailed list of changes at
> https://github.com/kernelci/kcidb/releases/tag/v9
>
> Most important changes are listed below.
...
> * Rename the submission queue topic for production from "kernelci_new" to
> "kcidb_new", and for playground from "playground_kernelci_new" to
> "playground_kcidb_new". This is done to unify KCIDB namespace inside Google
> Cloud and to help automate deployments. The old names will keep working
> until Dec 31.
...
> * Remember to switch to the new submission topics (above) by Dec 31!
I see that there's still some data being sent to the old topics "kernelci_new"
and "playground_kernelci_new". Please switch to using "kcidb_new" and
"playground_kcidb_new" instead, respectively.
I will remove the old ones on Wednesday at 12:00 EET. After that, submissions
to the old topics will be failing and no data sent there will reach KCIDB.
Nick
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: #KCIDB v9 release
2022-01-03 11:49 ` #KCIDB v9 release Nikolai Kondrashov
@ 2022-01-05 10:10 ` Nikolai Kondrashov
0 siblings, 0 replies; 7+ messages in thread
From: Nikolai Kondrashov @ 2022-01-05 10:10 UTC (permalink / raw)
To: Cristian Marussi, Tim Bird, Dmitry Vyukov, Guillaume Tucker,
Alice Ferrazzi, Iñaki Malerba, kernelci, Vishal Bhoj,
automated-testing
On 1/3/22 13:49, Nikolai Kondrashov wrote:
> Hi everyone,
>
> On 12/1/21 20:01, Nikolai Kondrashov wrote:
>> I'm proud to announce the release and deployment of KCIDB v9.
>>
>> Please see the detailed list of changes at
>> https://github.com/kernelci/kcidb/releases/tag/v9
>>
>> Most important changes are listed below.
> ...
>> * Rename the submission queue topic for production from "kernelci_new" to
>> "kcidb_new", and for playground from "playground_kernelci_new" to
>> "playground_kcidb_new". This is done to unify KCIDB namespace inside Google
>> Cloud and to help automate deployments. The old names will keep working
>> until Dec 31.
> ...
>> * Remember to switch to the new submission topics (above) by Dec 31!
>
> I see that there's still some data being sent to the old topics "kernelci_new"
> and "playground_kernelci_new". Please switch to using "kcidb_new" and
> "playground_kcidb_new" instead, respectively.
>
> I will remove the old ones on Wednesday at 12:00 EET. After that, submissions
> to the old topics will be failing and no data sent there will reach KCIDB.
Old topics are removed now. Please use the new ones instead.
Nick
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: #KCIDB v9 release
2022-06-07 14:35 ` Dmitry Vyukov
@ 2022-06-07 14:36 ` Nikolai Kondrashov
0 siblings, 0 replies; 7+ messages in thread
From: Nikolai Kondrashov @ 2022-06-07 14:36 UTC (permalink / raw)
To: Dmitry Vyukov
Cc: kernelci, Nikolai.Kondrashov, Guillaume Tucker,
automated-testing, syzkaller
On 6/7/22 17:35, Dmitry Vyukov wrote:
> On Tue, 7 Jun 2022 at 15:43, Nikolai Kondrashov <spbnick@gmail.com> wrote:
>>
>> Hi Dmitri,
>>
>> On 6/7/22 09:11, Dmitry Vyukov via groups.io wrote:
>>> If we upload "log_excerpt" with crashes, does the dashboard allow
>>> users to search for substrings in the "log_excerpt"?
>>> Namely, if we upload whole kernel oops as "log_excerpt" (that would be
>>> the intended use, right?), will it be possible to find all crashes
>>> that contain a given function name?
>>
>> At the moment the dashboard doesn't allow that, but it shouldn't be too hard
>> to add support for such filtering. I think I could do that in a couple hours work.
>>
>> So, please go ahead and send those excerpts, and I'll find some time to add
>> that filtering.
>
> I was just considering possibilities. I am not ready to send excerpts
> right now. It will require switching to v4 of the API as well, right?
> We are now on 3.0.
Yes, that would require switching to the v4 schema.
Nick
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: #KCIDB v9 release
2022-06-07 13:43 ` Nikolai Kondrashov
@ 2022-06-07 14:35 ` Dmitry Vyukov
2022-06-07 14:36 ` Nikolai Kondrashov
0 siblings, 1 reply; 7+ messages in thread
From: Dmitry Vyukov @ 2022-06-07 14:35 UTC (permalink / raw)
To: Nikolai Kondrashov
Cc: kernelci, Nikolai.Kondrashov, Guillaume Tucker,
automated-testing, syzkaller
On Tue, 7 Jun 2022 at 15:43, Nikolai Kondrashov <spbnick@gmail.com> wrote:
>
> Hi Dmitri,
>
> On 6/7/22 09:11, Dmitry Vyukov via groups.io wrote:
> > If we upload "log_excerpt" with crashes, does the dashboard allow
> > users to search for substrings in the "log_excerpt"?
> > Namely, if we upload whole kernel oops as "log_excerpt" (that would be
> > the intended use, right?), will it be possible to find all crashes
> > that contain a given function name?
>
> At the moment the dashboard doesn't allow that, but it shouldn't be too hard
> to add support for such filtering. I think I could do that in a couple hours work.
>
> So, please go ahead and send those excerpts, and I'll find some time to add
> that filtering.
I was just considering possibilities. I am not ready to send excerpts
right now. It will require switching to v4 of the API as well, right?
We are now on 3.0.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: #KCIDB v9 release
2022-06-07 6:11 ` Dmitry Vyukov
@ 2022-06-07 13:43 ` Nikolai Kondrashov
2022-06-07 14:35 ` Dmitry Vyukov
0 siblings, 1 reply; 7+ messages in thread
From: Nikolai Kondrashov @ 2022-06-07 13:43 UTC (permalink / raw)
To: kernelci, dvyukov, Nikolai.Kondrashov
Cc: Guillaume Tucker, automated-testing, syzkaller
Hi Dmitri,
On 6/7/22 09:11, Dmitry Vyukov via groups.io wrote:
> If we upload "log_excerpt" with crashes, does the dashboard allow
> users to search for substrings in the "log_excerpt"?
> Namely, if we upload whole kernel oops as "log_excerpt" (that would be
> the intended use, right?), will it be possible to find all crashes
> that contain a given function name?
At the moment the dashboard doesn't allow that, but it shouldn't be too hard
to add support for such filtering. I think I could do that in a couple hours work.
So, please go ahead and send those excerpts, and I'll find some time to add
that filtering.
Thanks!
Nick
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: #KCIDB v9 release
2021-12-01 18:01 Nikolai Kondrashov
@ 2022-06-07 6:11 ` Dmitry Vyukov
2022-06-07 13:43 ` Nikolai Kondrashov
0 siblings, 1 reply; 7+ messages in thread
From: Dmitry Vyukov @ 2022-06-07 6:11 UTC (permalink / raw)
To: Nikolai.Kondrashov
Cc: Guillaume Tucker, kernelci, automated-testing, syzkaller
On Wed, 1 Dec 2021 at 19:01, Nikolai Kondrashov
<Nikolai.Kondrashov@redhat.com> wrote:
>
> Hello everyone,
>
> I'm proud to announce the release and deployment of KCIDB v9.
>
> Please see the detailed list of changes at
> https://github.com/kernelci/kcidb/releases/tag/v9
>
> Most important changes are listed below.
>
> * Switch to using the new v4 I/O Schema, with following changes:
> * rename "revisions" to "checkouts",
> use origin-generated IDs for them, similar to other objects;
> * add "patchset_hash" field to checkouts,
> remove "publishing_time" field, and
> rename "patch_mboxes" to "patchset_files";
> * rename "description" fields to "comment" everywhere;
> * add "log_url" field to tests;
> * add "log_excerpt" field to all objects;
> See complete schema at
> https://github.com/kernelci/kcidb-io/releases/download/v3/kcidb.v4.schema.json
Hi Nikolai,
If we upload "log_excerpt" with crashes, does the dashboard allow
users to search for substrings in the "log_excerpt"?
Namely, if we upload whole kernel oops as "log_excerpt" (that would be
the intended use, right?), will it be possible to find all crashes
that contain a given function name?
^ permalink raw reply [flat|nested] 7+ messages in thread
* #KCIDB v9 release
@ 2021-12-01 18:01 Nikolai Kondrashov
2022-06-07 6:11 ` Dmitry Vyukov
0 siblings, 1 reply; 7+ messages in thread
From: Nikolai Kondrashov @ 2021-12-01 18:01 UTC (permalink / raw)
To: Cristian Marussi, Tim Bird, Dmitry Vyukov, Guillaume Tucker,
Alice Ferrazzi, Iñaki Malerba, kernelci, Vishal Bhoj,
automated-testing
Hello everyone,
I'm proud to announce the release and deployment of KCIDB v9.
Please see the detailed list of changes at
https://github.com/kernelci/kcidb/releases/tag/v9
Most important changes are listed below.
* Switch to using the new v4 I/O Schema, with following changes:
* rename "revisions" to "checkouts",
use origin-generated IDs for them, similar to other objects;
* add "patchset_hash" field to checkouts,
remove "publishing_time" field, and
rename "patch_mboxes" to "patchset_files";
* rename "description" fields to "comment" everywhere;
* add "log_url" field to tests;
* add "log_excerpt" field to all objects;
See complete schema at
https://github.com/kernelci/kcidb-io/releases/download/v3/kcidb.v4.schema.json
* Reimplement the ORM, subscription, and notification systems to provide more
compact notifications with concise summaries for revisions, builds and tests
across all submitting CI systems. Add flexibility to quickly implement new
aggregations and custom, subscription-specific notifications.
Examples:
https://groups.io/g/kernelci-results-staging/message/8607
https://groups.io/g/kernelci-results-staging/message/8639
https://groups.io/g/kernelci-results-staging/message/8626
(notifications are sent on first failures, and don't contain later data)
* Add following features to dashboards:
* Differentiate revisions and checkouts
E.g. see the three separate checkouts for this revision:
https://kcidb.kernelci.org/d/revision/revision?var-git_commit_hash=58e1100fdc5990b0cc0d4beaf2562a92e621ac7d
* Show all tests aggregated for revisions and checkouts, and
allow browsing the test tree and filtering by test tree nodes.
E.g. display just the LTP results for a revision:
https://kcidb.kernelci.org/d/revision/revision?var-git_commit_hash=5f53fa508db05246c7b460829a667ffc9a0bb0c4&var-test_path=ltp
or see all executions of LTP's chdir01a test case:
https://kcidb.kernelci.org/d/revision/revision?var-git_commit_hash=5f53fa508db05246c7b460829a667ffc9a0bb0c4&var-test_path=ltp.chdir01a
(you'll have to scroll down for both of these)
* Display log excerpts for builds and tests directly on their dashboards
(pending actual excerpt submissions from CI systems)
* Rename the submission queue topic for production from "kernelci_new" to
"kcidb_new", and for playground from "playground_kernelci_new" to
"playground_kcidb_new". This is done to unify KCIDB namespace inside Google
Cloud and to help automate deployments. The old names will keep working
until Dec 31.
* Name the new production dataset "kcidb_01", and playground dataset
"playground_kcidb_01". New submissions are arriving there since yesterday,
and the data from the old dataset should finish migration in a few hours.
* Publish KCIDB documentation on the kernelci.org website:
https://kernelci.org/docs/kcidb/
Feedback and contributions welcome!
ATTENTION CI SYSTEMS!
* It's strongly recommended to upgrade to the v4 schema, if you're testing and
sending revisions more than once (from the same or different repositories).
It uses origin-generated checkout IDs, which allows each checkout to be
differentiated, and every one's data preserved.
* Remember to set the checkout's "patchset_hash" field to empty string when
using schema v4, if you're not applying any patches on top of the commits.
If you do apply patches, put their hash there. Otherwise your results might
not appear in notifications and dashboards.
* Remember to switch to the new submission topics (above) by Dec 31!
WHAT'S NEXT
This release cycle was way too long, mostly due to rework of ORM,
subscription, and notification systems. After this release we'll focus on
making faster and smaller releases, and we already started work on automating
our deployment, improving CI, and adding CD, which should help with that.
Apart from that, our main target will be adding support for registering and
masking known failures and regressions, so developers and maintainers could
receive reports only about new failures.
However, even without support for that, we're prepared to use our flexible
subscription and notification system to tailor our messages and minimize noise
in what we send to our subscribers.
Don't hesitate to reach out, provide your requirements, and get subscribed to
our email notifications. We need your feedback to improve! We'll be contacting
potentially interested maintainers in the coming weeks ourselves, as well.
Nick
P.S. Thank you for making it through this long announcement :)
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2022-06-07 14:36 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <16BCB376A7748BB3.24346@groups.io>
2022-01-03 11:49 ` #KCIDB v9 release Nikolai Kondrashov
2022-01-05 10:10 ` Nikolai Kondrashov
2021-12-01 18:01 Nikolai Kondrashov
2022-06-07 6:11 ` Dmitry Vyukov
2022-06-07 13:43 ` Nikolai Kondrashov
2022-06-07 14:35 ` Dmitry Vyukov
2022-06-07 14:36 ` Nikolai Kondrashov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.