Re: [kernelci-members] Working with the KernelCI project

["Nikolai Kondrashov" <Nikolai.Kondrashov@redhat.com>]

On 10/1/20 7:00 PM, Dmitry Vyukov wrote:
 > On Thu, Oct 1, 2020 at 5:52 PM Nikolai Kondrashov
 > <Nikolai.Kondrashov@redhat.com> wrote:
 >>
 >> On 10/1/20 6:49 PM, Nikolai Kondrashov wrote:
 >>> On 10/1/20 5:48 PM, Dmitry Vyukov via groups.io wrote:
 >>>   > On Thu, Oct 1, 2020 at 3:32 PM Nikolai Kondrashov
 >>>   > <Nikolai.Kondrashov@redhat.com> wrote:
 >>>   >>
 >>>   >> On 10/1/20 1:48 PM, Nikolai Kondrashov wrote:
 >>>   >>   > Here are the things which could be improved:
 >>>   >>
 >>>   >> Oh, and another thing: could you avoid re-sending the identical
 >>>   >> revisions/builds you've already sent? I.e. send them only once, unless you
 >>>   >> have fields to add? Each of those costs us a row in the database, which is not
 >>>   >> a big deal, but would be good to avoid.
 >>>   >
 >>>   > Do you see that we send dups? I've added logic to not send dups and
 >>>   > from what I see it's working.
 >>>
 >>> Please find attached a zip archive with seven submissions from about six hours
 >>> ago. Each has revisions with the same id and git_commit_hash. Some of those
 >>> revisions have different discovery_time, and that alone shouldn't really be
 >>> the reason to resend. Each of those submissions have a build object, but some
 >>> of them are repeated.
 >>>
 >>> Revision with that git_commit_hash was submitted at least 72 times.
 >>
 >> And now the file is attached :D
 >
 > This is intentional (in the current implementation) and is a
 > consequence of the fact that we always send all 3 entities for each
 > issue/test failure. It's much simpler on our side this way. If we
 > would send a test failure multiple times it would be unintentional.

I see.

 > Identical builds should have the same id, though. And if they have
 > different id's, then these are different builds.

Sure.

 > Revisions were discovered separately by different instances, that's
 > why they have different discovery time. Consider that as if different
 > origin systems would discover it separately. Since discovery time is
 > not an inherent property of the commit itself, there is no way they
 > can agree on it.

Ah, I see now why. Yeah, it's similar to multiple systems submitting the same
revision. Although we don't have systems submitting the same build multiple
times.

 > Will it be better if we don't send discovery time at all?

No, it's OK to submit the discover time multiple times, but we can't say which
one will be displayed :)

 > Amount of duplication for builds/revisions is capped by the number of
 > bugs we discover, since build/revision is only sent with a bug once.
 > So it's not that it's unlimited.

I think it's OK that you have some duplication, and certainly OK for the
start. Let's get on with sending the data and see how much actual duplication
we get, how it affects performance, and how the de-duplication implementation
goes :)

Thank you!
Nick