KCIDB: Renaming "revisions" to "checkouts"

KCIDB: Renaming "revisions" to "checkouts"

[Nikolai Kondrashov]

Hi everyone,

To anyone involved/interested in contributing to KernelCI's KCIDB data and
code, I'd like to announce a small, but noticeable change to the schema to let
us develop result notifications further, and to move closer to reaching
developers with our data.

I'd like to rename "revisions" to "checkouts", and switch from using the Git
commit hash (plus optional patchset hash) as their IDs, to origin-generated
IDs, similar to "builds" and "tests".

As it is currently, the ID is less granular than the data, which leads to
losses when aggregating it. The new ID would be up to the origin to generate,
and should identify the fact of checking out the kernel source code used to
make one or more builds.

The ID should start with the origin's name, followed by a colon (':'), and
then by the origin-generated ID string, which should be unique across all
"checkouts" submitted by the origin.

Correspondingly, the builds would need to use that ID to refer to their
"checkouts". This will be similar to what we had in schemas prior to v3.0.

For example, this v3.0 report:

     {
         "version": {"major": 3, "minor": 0},
         "revisions": [
             {
                 "id": "e28c0d7c92c89016c12a677616668957351e7542",
                 "origin": "syzbot",
                 "git_commit_hash": "e28c0d7c92c89016c12a677616668957351e7542",
             }
         ]
         "builds": [
             {
                 "revision_id": "e28c0d7c92c89016c12a677616668957351e7542",
                 "id": "syzbot:5d467ff7958f50484d774d9bcc068f81e30bbda5",
                 "origin": "syzbot",
                 "architecture": "x86_64",
             }
         ],
     }

Could look like this in the upcoming v4.0 schema:

     {
         "version": {"major": 4, "minor": 0},
         "checkouts": [
             {
                 "id": "syzbot:121098431",
                 "origin": "syzbot",
                 "git_commit_hash": "e28c0d7c92c89016c12a677616668957351e7542",
             }
         ]
         "builds": [
             {
                 "checkout_id": "syzbot:121098431",
                 "id": "syzbot:5d467ff7958f50484d774d9bcc068f81e30bbda5",
                 "origin": "syzbot",
                 "architecture": "x86_64",
             }
         ],
     }

Above, the "syzbot:121098431" string would need to be generated by syzbot CI,
and the "121098431" part could be e.g. the ID of the pipeline which handled
that particular checkout. For CKI it could be the GitLab pipeline ID, for
example.

Submitters who are still sending v2.0 or older data (like the native KernelCI
setup) wouldn't need to change their ID generation.

This change also required some other, smaller changes.

In particular, to allow us keep identifying patched revisions, we moved the
patchset hash out of the revision ID to a separate field called
"patchset_hash". The semantics and the generation of that field haven't
changed. To match that change, we renamed the "patch_mboxes" field to
"patchset_files".

Finally, we renamed the "discovery_time" field, which was suitable for
"revisions", to "start_time", which suits "checkouts" better.

The specific code and schema changes can be seen in
https://github.com/kernelci/kcidb-io/pull/16

As usual, we will handle data conversion automatically, so after we deploy the
new release, there will be no rush to upgrade the schema version of the data
you send. However, we will advise to do that soon after, to avoid the
aforementioned loss of data (such as the full list of git repositories and
branches containing particular revisions).

Please comment/object/ask questions. If there are no objections I'll merge
the above change next week, to be included into the next release.

Thank you!
Nick