Re: ❌ FAIL: Stable queue: queue-5.2

From: Greg KH <greg@kroah.com>
To: Nikolai Kondrashov <Nikolai.Kondrashov@redhat.com>
Cc: CKI Project <cki-project@redhat.com>,
	Linux Stable maillist <stable@vger.kernel.org>
Subject: Re: ❌ FAIL: Stable queue: queue-5.2
Date: Mon, 26 Aug 2019 10:33:09 +0200	[thread overview]
Message-ID: <20190826083309.GA32549@kroah.com> (raw)
In-Reply-To: <d0567d4e-6bbe-4a93-d657-0ee7f6e4625d@redhat.com>

On Mon, Aug 26, 2019 at 11:23:58AM +0300, Nikolai Kondrashov wrote:
> On 8/25/19 5:41 PM, Greg KH wrote:
> > On Sun, Aug 25, 2019 at 10:37:26AM -0400, CKI Project wrote:
> > > Merge testing
> > > -------------
> > > 
> > > We cloned this repository and checked out the following commit:
> > > 
> > >    Repo: https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
> > >    Commit: f7d5b3dc4792 - Linux 5.2.10
> > > 
> > > 
> > > We grabbed the cc88f4442e50 commit of the stable queue repository.
> > > 
> > > We then merged the patchset with `git am`:
> > > 
> > >    keys-trusted-allow-module-init-if-tpm-is-inactive-or-deactivated.patch
> > 
> > That file is not in the repo, I think your system is messed up :(
> 
> Sorry for the trouble, Greg, but I think it's a race between the changes to
> the two repos.
> 
> The job which triggered this message was started right before the moment this
> commit was made:
> 
>     https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git/commit/?id=af2f46e26e770b3aa0bc304a13ecd24763f3b452
> 
> At that moment, the repo was still on this commit, about five hours old:
> 
>     https://git.kernel.org/pub/scm/linux/kernel/git/stable/stable-queue.git/commit/?id=cc88f4442e505e9f1f21c8c119debe89cbf63ab2
> 
> which still had the file. And when the job finished, and the message reached
> you, yes, the repo no longer contained it.
> 
> At the moment the job started, the latest commit to stable/linux.git
> was about 22 minutes old:
> 
>     https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-5.2.y&id=f7d5b3dc4792a5fe0a4d6b8106a8f3eb20c3c24c
> 
> and the repo already contained the patches from the queue, including the one
> the job tried to merge:
> 
>     https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-5.2.y&id=f820ecf609cc38676071ec6c6d3e96b26c73b747

How in the world are you seeing such a messed up tree?

The 5.2.10 commit moved things around, in one single atomic move.

> IIRC, we agreed to not start testing both of the repos until the latest
> commits are at least 5 minutes old. In this situation the latest commit was 22
> minutes old, so the system started testing.
> 
> We could increase the window to, say, 30 minutes (or something else), to avoid
> misfires like this, but then the response time would be increased accordingly.
> 
> It's your pick :)

Why is there any race at all?

Why do you not have a local mirror of the repo?  When it updates, then
run the tests.  Every commit in the tree is "stand alone" and things
should work at that point in time.  Don't use a commit as a "time to go
mirror something at a later point in time", as you are ending up with
trees that are obviously not correct at all.

I think you need to rework your systems as no one else seems to have
this "stale random tree state" issue.

Git does commits in an atomic fashion, how you all are messing that up
shows you are doing _way_ more work than you probably need to :)

thanks,

greg k-h