Re: [dpdk-stable] Regression tests for stable releases from companies involved in DPDK

From: Luca Boccassi <bluca@debian.org>
To: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Cc: dev <dev@dpdk.org>,
	stable@dpdk.org, Thomas Monjalon <thomas@monjalon.net>
Subject: Re: [dpdk-stable] Regression tests for stable releases from companies involved in DPDK
Date: Fri, 01 Jun 2018 10:57:40 +0100	[thread overview]
Message-ID: <1527847060.6997.67.camel@debian.org> (raw)
In-Reply-To: <CAATJJ0KJ8g93JGh1F4XUBryxhwKtDmWM=L_sY_9rHtvESZ+Sbw@mail.gmail.com>

On Fri, 2018-06-01 at 06:38 +0200, Christian Ehrhardt wrote:
> On Thu, May 31, 2018 at 12:26 PM, Luca Boccassi <bluca@debian.org>
> wrote:
> 
> > Hello all,
> > 
> > At this morning's release meeting (minutes coming soon from John),
> > we
> > briefly discussed the state of the regression testing for stable
> > releases and agreed we need to formalise the process.
> > 
> > At the moment we have a firm commitment from Intel and Mellanox to
> > test
> > all stable branches (and if I heard correctly from NXP as well?
> > Please
> > confirm!). AT&T committed to run regressions on the 16.11 branch.
> > 
> > Here's what we need in order to improve the quality of the stable
> > releases process:
> > 
> > 1) More commitments to help from other companies involved in the
> > DPDK
> > community. At the cost of re-stating the obvious, improving the
> > quality
> > of stable releases is for everyone's benefit, as a lot of customers
> > and
> > projects rely on the stable or LTS releases for their production
> > environments.
> > 
> > 2) A formalised deadline - the current proposal is 10 days from the
> > "xx.yy patches review and test" email, which was just sent for
> > 16.11.
> > For the involved companies, please let us know if 10 days is
> > enough. In
> > terms of scheduling, this period will always start within a week
> > from
> > the mainline final release. Again, the signal is the "xx.yy patches
> > review and test" appearing in the inbox, which will detail the
> > deadline.
> > 
> > 
> 
> Hi Luca,
> I discussed with Thomas about it.
> I don't know how much extra effort for the stable maintainers it
> would be,
> but I wonder if there could be a XX.YY.z-rc tarball.
> That would be
> a) a more clear sign what people are used to test
> b) easier to integrate as I assume quite a bunch of tests will
> usually
> start rebasing on tarballs instead of directly from git.
> 
> If you think everyone can derive from git easily I'm fine, I just
> wondered
> if a proper -rc tarball might be more comfortable for the testing
> entities.
> 
> cu
> Christian

I think that's a good idea, and something we can consider for the next
release cycle - the tools to push rc to mainline should work just the
same for the stable repo.

-- 
Kind regards,
Luca Boccassi