staging projects in github

staging projects in github

[Kevin Hilman]

Can we consolidate the various "-staging" project in github with the
originals?  I've never fully understood why we created separate github
projects, when we should just be using a staging branch within the main
project.

I've seen this come up a couple times now in causing confusion with new
developers.

Thanks,

Kevin

Re: staging projects in github

[Matt Hart]

On Tue, 5 Mar 2019 at 05:14, Kevin Hilman <khilman@baylibre.com> wrote:
>
> Can we consolidate the various "-staging" project in github with the
> originals?  I've never fully understood why we created separate github
> projects, when we should just be using a staging branch within the main
> project.

I *think* it was because you couldnt make a pull request against a branch,
however I think you can now set the default branch for PR on a project.
As long as that works, I'm all for it.

>
> I've seen this come up a couple times now in causing confusion with new
> developers.
>
> Thanks,
>
> Kevin
>
>
> 
>

Re: staging projects in github

[Dan Rue]

On Tue, Mar 05, 2019 at 04:48:52PM +0000, Matt Hart wrote:
> On Tue, 5 Mar 2019 at 05:14, Kevin Hilman <khilman@baylibre.com> wrote:
> >
> > Can we consolidate the various "-staging" project in github with the
> > originals?  I've never fully understood why we created separate github
> > projects, when we should just be using a staging branch within the main
> > project.
> 
> I *think* it was because you couldnt make a pull request against a branch,
> however I think you can now set the default branch for PR on a project.
> As long as that works, I'm all for it.

I'm not sure if I'm considering all of the use-cases, but in the past
I've much preferred having staging == "master" and then tagging for
production releases. This does make it difficult (but not impossible) to
have things in staging that are not promoted to production.

As a user of kernelci, it does seem quite slow to get changes included
into staging, and then also to wait for them get into production. I'm
really interested in anything we can do to improve this turnaround time.

Dan

> 
> >
> > I've seen this come up a couple times now in causing confusion with new
> > developers.
> >
> > Thanks,
> >
> > Kevin
> >
> >
> > 
> >
> 
> 
> 

Re: staging projects in github

[Guillaume Tucker]

[-- Attachment #1: Type: text/plain, Size: 2574 bytes --]

On Tue, Mar 5, 2019 at 5:04 PM Dan Rue <dan.rue@linaro.org> wrote:

> On Tue, Mar 05, 2019 at 04:48:52PM +0000, Matt Hart wrote:
> > On Tue, 5 Mar 2019 at 05:14, Kevin Hilman <khilman@baylibre.com> wrote:
> > >
> > > Can we consolidate the various "-staging" project in github with the
> > > originals?  I've never fully understood why we created separate github
> > > projects, when we should just be using a staging branch within the main
> > > project.
>

I never understood it either, and when I suggested to do
something else when I joined the project I was told that it would
be too complicated.  I guess maybe the dual repo set-up worked
initially when there weren't many changes going on.

In fact we're not actually using the master branch on
kernelci-core-staging any more for anything else than just
merging PRs. It's a much better workflow to be able to test each
PR branch individually before merging it using separate jobs on
the staging Jenkins server.

And by the way, the whole point is to not merge a PR before it
has been tested.  We're a CI project, remember?

> I *think* it was because you couldnt make a pull request against a branch,
> > however I think you can now set the default branch for PR on a project.
> > As long as that works, I'm all for it.
>
> I'm not sure if I'm considering all of the use-cases, but in the past
> I've much preferred having staging == "master" and then tagging for
> production releases. This does make it difficult (but not impossible) to
> have things in staging that are not promoted to production.
>

What would make most sense imo would be to have the master branch
receiving the PRs (like Dan wrote) and a live "prod" branch used
by Jenkins.  We can use tags as well, for every time we sync the
master branch with the production one, but I don't think it's
practical to use tags with Jenkins as you would need to update
all the jobs every time.


> As a user of kernelci, it does seem quite slow to get changes included
> into staging, and then also to wait for them get into production. I'm
> really interested in anything we can do to improve this turnaround time.
>

The key things in this respect are the time it takes to test
something on staging, which has improved recently with the extra
builders, and how to automate the steps to update production.  I
think we should soon consider automated testing on the PR
branches, but that's a whole topic of its own.

Guillaume

> > I've seen this come up a couple times now in causing confusion with new
> > > developers.
> > >
> > > Thanks,
> > >
> > > Kevin
>

[-- Attachment #2: Type: text/html, Size: 4198 bytes --]

Re: staging projects in github

[Guillaume Tucker]

[-- Attachment #1: Type: text/plain, Size: 1211 bytes --]

On Tue, Mar 5, 2019 at 5:24 PM Guillaume Tucker via Groups.Io
<guillaume.tucker=gmail.com@groups.io> wrote:

>
> On Tue, Mar 5, 2019 at 5:04 PM Dan Rue <dan.rue@linaro.org> wrote:
>
>> On Tue, Mar 05, 2019 at 04:48:52PM +0000, Matt Hart wrote:
>> > On Tue, 5 Mar 2019 at 05:14, Kevin Hilman <khilman@baylibre.com> wrote:
>> > >
>> > > Can we consolidate the various "-staging" project in github with the
>> > > originals?  I've never fully understood why we created separate github
>> > > projects, when we should just be using a staging branch within the
>> main
>> > > project.
>
> [...]

> I'm not sure if I'm considering all of the use-cases, but in the past
>> I've much preferred having staging == "master" and then tagging for
>> production releases. This does make it difficult (but not impossible) to
>> have things in staging that are not promoted to production.
>>
>
> What would make most sense imo would be to have the master branch
> receiving the PRs (like Dan wrote) and a live "prod" branch used
> by Jenkins.
>

So how about having one last staging -> prod sync this week using
the kernelci-core-staging project, and then switch to using
kernelci-core only starting from next week?

Guillaume

[-- Attachment #2: Type: text/html, Size: 2826 bytes --]

Re: staging projects in github

[Guillaume Tucker]

[-- Attachment #1: Type: text/plain, Size: 1690 bytes --]

On Tue, Mar 12, 2019 at 10:22 AM Guillaume Tucker via Groups.Io
<guillaume.tucker=gmail.com@groups.io> wrote:

>
> On Tue, Mar 5, 2019 at 5:24 PM Guillaume Tucker via Groups.Io
> <guillaume.tucker=gmail.com@groups.io> wrote:
>
>>
>> On Tue, Mar 5, 2019 at 5:04 PM Dan Rue <dan.rue@linaro.org> wrote:
>>
>>> On Tue, Mar 05, 2019 at 04:48:52PM +0000, Matt Hart wrote:
>>> > On Tue, 5 Mar 2019 at 05:14, Kevin Hilman <khilman@baylibre.com>
>>> wrote:
>>> > >
>>> > > Can we consolidate the various "-staging" project in github with the
>>> > > originals?  I've never fully understood why we created separate
>>> github
>>> > > projects, when we should just be using a staging branch within the
>>> main
>>> > > project.
>>
>> [...]
>
>> I'm not sure if I'm considering all of the use-cases, but in the past
>>> I've much preferred having staging == "master" and then tagging for
>>> production releases. This does make it difficult (but not impossible) to
>>> have things in staging that are not promoted to production.
>>>
>>
>> What would make most sense imo would be to have the master branch
>> receiving the PRs (like Dan wrote) and a live "prod" branch used
>> by Jenkins.
>>
>
> So how about having one last staging -> prod sync this week using
> the kernelci-core-staging project, and then switch to using
> kernelci-core only starting from next week?
>

FYI I've also added a section on the wiki about the steps
required to update kernelci.org production:


https://github.com/kernelci/kernelci-doc/wiki/Workflow#to-update-production

That page would also need to be updated if we stopped using
kernelci-core-staging, in the first section about creating
production PRs.

Guillaume

>

[-- Attachment #2: Type: text/html, Size: 3926 bytes --]

Re: staging projects in github

[Kevin Hilman]

Guillaume Tucker <guillaume.tucker@gmail.com> writes:

> On Tue, Mar 5, 2019 at 5:24 PM Guillaume Tucker via Groups.Io
> <guillaume.tucker=gmail.com@groups.io> wrote:
>
>>
>> On Tue, Mar 5, 2019 at 5:04 PM Dan Rue <dan.rue@linaro.org> wrote:
>>
>>> On Tue, Mar 05, 2019 at 04:48:52PM +0000, Matt Hart wrote:
>>> > On Tue, 5 Mar 2019 at 05:14, Kevin Hilman <khilman@baylibre.com> wrote:
>>> > >
>>> > > Can we consolidate the various "-staging" project in github with the
>>> > > originals?  I've never fully understood why we created separate github
>>> > > projects, when we should just be using a staging branch within the
>>> main
>>> > > project.
>>
>> [...]
>
>> I'm not sure if I'm considering all of the use-cases, but in the past
>>> I've much preferred having staging == "master" and then tagging for
>>> production releases. This does make it difficult (but not impossible) to
>>> have things in staging that are not promoted to production.
>>>
>>
>> What would make most sense imo would be to have the master branch
>> receiving the PRs (like Dan wrote) and a live "prod" branch used
>> by Jenkins.
>>
>
> So how about having one last staging -> prod sync this week using
> the kernelci-core-staging project, and then switch to using
> kernelci-core only starting from next week?

Sounds good to me.

Kevin

Re: staging projects in github

[Guillaume Tucker]

[-- Attachment #1: Type: text/plain, Size: 3411 bytes --]

On Tue, Mar 12, 2019 at 5:47 PM Kevin Hilman <khilman@baylibre.com> wrote:

> Guillaume Tucker <guillaume.tucker@gmail.com> writes:
>
> > On Tue, Mar 5, 2019 at 5:24 PM Guillaume Tucker via Groups.Io
> > <guillaume.tucker=gmail.com@groups.io> wrote:
> >
> >>
> >> On Tue, Mar 5, 2019 at 5:04 PM Dan Rue <dan.rue@linaro.org> wrote:
> >>
> >>> On Tue, Mar 05, 2019 at 04:48:52PM +0000, Matt Hart wrote:
> >>> > On Tue, 5 Mar 2019 at 05:14, Kevin Hilman <khilman@baylibre.com>
> wrote:
> >>> > >
> >>> > > Can we consolidate the various "-staging" project in github with
> the
> >>> > > originals?  I've never fully understood why we created separate
> github
> >>> > > projects, when we should just be using a staging branch within the
> >>> main
> >>> > > project.
> >>
> >> [...]
> >
> >> I'm not sure if I'm considering all of the use-cases, but in the past
> >>> I've much preferred having staging == "master" and then tagging for
> >>> production releases. This does make it difficult (but not impossible)
> to
> >>> have things in staging that are not promoted to production.
> >>>
> >>
> >> What would make most sense imo would be to have the master branch
> >> receiving the PRs (like Dan wrote) and a live "prod" branch used
> >> by Jenkins.
> >>
> >
> > So how about having one last staging -> prod sync this week using
> > the kernelci-core-staging project, and then switch to using
> > kernelci-core only starting from next week?
>
> Sounds good to me.


So staging-20190315 was merged and deployed on Friday.  I've now
updated the kernelci.org Jenkins jobs in production to use a
kernelci.org branch from kernelci-core.

We can start using kernelci-core/master branch for pull requests
instead of the kernelci-core-staging repository.  There are a few
PRs left in kernelci-core-staging, I suggest we try to merge
those that are almost ready by Thursday and cherry-pick the
changes in kernelci-core, and any PR left open on Friday should
be closed and probably opened again in kernelci-core.

See the kernelci-core project in Github:

  https://github.com/kernelci/kernelci-core

Open PRs in kernelci-core-staging:

  https://github.com/kernelci/kernelci-core-staging/pulls

I've also created a staging.kernelci.org branch in kernelci-core,
the idea being that we can use that to run some jobs on
staging.kernelci.org.  This should be especially useful for
occasional contributors who don't have an account on the staging
Jennkins instance: someone can put the code from their PRs on
that branch to have it tested for them.  It can also be used like
the staging branch in kernelci-backend, to test that on-going PRs
work when integrated together.

Both the kernelci.org and staging.kernelci.org branches are not
for merging pull requests, they are just a way to configure the
Jenkins jobs on the corresponding instances.

The next step is to update the README file in kernelci-core and
the wiki accordingly.  Essentially, the workflow now becomes very
simple from a contributor's point of view with regular PRs
against the master branch.  For production updates, we should
create a tag on the master branch and update the kernelci.org
branch to match that rather than create a "staging" PR.  We
should however sent notes to the mailing list with a summary like
in past staging PRs with things going into production.  Please
let me know if you have any suggestions to do things differently.

Guillaume

[-- Attachment #2: Type: text/html, Size: 5308 bytes --]

Re: staging projects in github

[Kevin Hilman]

Guillaume Tucker <guillaume.tucker@gmail.com> writes:

[...]

> So staging-20190315 was merged and deployed on Friday.  I've now
> updated the kernelci.org Jenkins jobs in production to use a
> kernelci.org branch from kernelci-core.
>
> We can start using kernelci-core/master branch for pull requests
> instead of the kernelci-core-staging repository.  There are a few
> PRs left in kernelci-core-staging, I suggest we try to merge
> those that are almost ready by Thursday and cherry-pick the
> changes in kernelci-core, and any PR left open on Friday should
> be closed and probably opened again in kernelci-core.
>
> See the kernelci-core project in Github:
>
>   https://github.com/kernelci/kernelci-core
>
> Open PRs in kernelci-core-staging:
>
>   https://github.com/kernelci/kernelci-core-staging/pulls
>
> I've also created a staging.kernelci.org branch in kernelci-core,
> the idea being that we can use that to run some jobs on
> staging.kernelci.org.  This should be especially useful for
> occasional contributors who don't have an account on the staging
> Jennkins instance: someone can put the code from their PRs on
> that branch to have it tested for them.  It can also be used like
> the staging branch in kernelci-backend, to test that on-going PRs
> work when integrated together.
>
> Both the kernelci.org and staging.kernelci.org branches are not
> for merging pull requests, they are just a way to configure the
> Jenkins jobs on the corresponding instances.
>
> The next step is to update the README file in kernelci-core and
> the wiki accordingly.  Essentially, the workflow now becomes very
> simple from a contributor's point of view with regular PRs
> against the master branch.  For production updates, we should
> create a tag on the master branch and update the kernelci.org
> branch to match that rather than create a "staging" PR.  We
> should however sent notes to the mailing list with a summary like
> in past staging PRs with things going into production.  Please
> let me know if you have any suggestions to do things differently.

This is great, thank you for taking the time/effort to implement this
and roll it out.

Kevin