Re: [dpdk-dev] [dpdk-ci] [RFC] Proposal for allowing rerun of tests

From: Thomas Monjalon <thomas@monjalon.net>
To: David Marchand <david.marchand@redhat.com>,
	Bruce Richardson <bruce.richardson@intel.com>
Cc: Aaron Conole <aconole@redhat.com>, dev <dev@dpdk.org>,
	ci@dpdk.org, Michael Santana <msantana@redhat.com>,
	Lincoln Lavoie <lylavoie@iol.unh.edu>,
	dpdklab <dpdklab@iol.unh.edu>
Subject: Re: [dpdk-dev] [dpdk-ci] [RFC] Proposal for allowing rerun of tests
Date: Tue, 13 Apr 2021 17:17:06 +0200	[thread overview]
Message-ID: <2009041.gKtNCFKaKa@thomas> (raw)
In-Reply-To: <20210413150425.GA1185@bricha3-MOBL.ger.corp.intel.com>

13/04/2021 17:04, Bruce Richardson:
> On Tue, Apr 13, 2021 at 04:59:00PM +0200, David Marchand wrote:
> > On Tue, Apr 13, 2021 at 4:47 PM Thomas Monjalon <thomas@monjalon.net> wrote:
> > >
> > > 13/04/2021 15:50, Aaron Conole:
> > >
> > > > One proposal we (Michael and I) have toyed with for our lab is having
> > > > the infrastructure monitor patchwork comments for a restart flag, and
> > > > kick off based on that information.  Patchwork tracks all of the
> > > > comments for each patch / series so we could look at the series that
> > > > are still in a state for 'merging' (new, assigned, etc) and check the
> > > > patch .comments API for new comments.  Getting the data from PW should
> > > > be pretty simple - but I think that knowing whether to kick off the
> > > > test might be more difficult.  We have concerns about which messages we
> > > > should accept (for example, can anyone ask for a series to be rerun, and
> > > > we'll need to track which rerun messages we've accepted).  The
> > > > convention needs to be something we all can work with (ie: /Re-check:
> > > > [checkname] or something as a single line in the email).
> > > >
> > > > This is just a start to identify and explain the concern.  Maybe there
> > > > are other issues we've not considered, or maybe folks think this is a
> > > > terrible idea not worth spending any time developing.  I think there's
> > > > enough use for it that I am raising it here, and we can discuss it.
> > >
> > > First question: WHO should be allowed to ask for a re-run?
> > >         - everybody
> > >         - patchwork delegate
> > 
> > Patchwork delegate requires to maintain a map between pw logins and an
> > actual mail address (if we go with email for the second point).
> > 
> > >         - a list of maintainers
> > 
> > I'd vote on any maintainer from MAINTAINERS, _but_ it must be from the
> > files in the repo, not in the series being tested.
> > So maybe the easier is to have an explicit list... ?
> > 
> > 
> > - author
> > Just listing this option for discussion, but this is dangerous, as any
> > user could then call reruns.
> > 
> 
> I would tend towards including this, on the basis that any author can
> already get a re-run just be resubmitting a new version of their patchset.
> This just simplifies that for all concerned.

I agree, and it would be very convenient for authors hitting
a strange failure: they can double check without bothering maintainers.