* ceph-qa-suite branching (merge it into ceph.git?)
@ 2016-04-13 10:52 John Spray
2016-04-13 12:45 ` Sage Weil
` (2 more replies)
0 siblings, 3 replies; 23+ messages in thread
From: John Spray @ 2016-04-13 10:52 UTC (permalink / raw)
To: Ceph Development, ceph-qa
Lately, we've instances where ceph.git and ceph-qa-suite.git got
slightly out of sync, as we were adding new stuff and interface
changes to ceph (especially in cephfs).
We used to have two repos (ceph and teuthology), now we have three
(ceph, ceph-qa-suite and teuthology). Splitting tests out of
teuthology was a good thing, but maybe they should have gone into the
ceph tree instead of a new repo? The ceph-qa-suite branching seems to
pretty much mirror what we do with ceph, with master vs jewel etc.
Historically we had a comparatively static set of workloads in the qa
suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change
much with ceph changes. But these days we're adding much more
detailed tests, so there's more effort to keep the two in sync.
I would personally love to be able to have a single PR that contained
my code and the tests for it. What if after Jewel we pulled all of
ceph-qa-suite into the ceph repo?
We could still enable folks running test changes without having to
rebuild ceph packages: the suite sha1 selected when running a
teuthology suite could still be different from that used for
installing ceph, it's just that it would fetch that sha1 from the ceph
repo instead of from a separate repo.
John
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 10:52 ceph-qa-suite branching (merge it into ceph.git?) John Spray @ 2016-04-13 12:45 ` Sage Weil [not found] ` <570E4843.8000102@dachary.org> [not found] ` <CAKPXa=Y5rJ4ygM9Sdvys+86RwufnKopTOkMNdPHcpCh5bFbQ0w@mail.gmail.com> 2 siblings, 0 replies; 23+ messages in thread From: Sage Weil @ 2016-04-13 12:45 UTC (permalink / raw) To: John Spray; +Cc: Ceph Development, ceph-qa On Wed, 13 Apr 2016, John Spray wrote: > Lately, we've instances where ceph.git and ceph-qa-suite.git got > slightly out of sync, as we were adding new stuff and interface > changes to ceph (especially in cephfs). > > We used to have two repos (ceph and teuthology), now we have three > (ceph, ceph-qa-suite and teuthology). Splitting tests out of > teuthology was a good thing, but maybe they should have gone into the > ceph tree instead of a new repo? The ceph-qa-suite branching seems to > pretty much mirror what we do with ceph, with master vs jewel etc. > > Historically we had a comparatively static set of workloads in the qa > suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change > much with ceph changes. But these days we're adding much more > detailed tests, so there's more effort to keep the two in sync. > > I would personally love to be able to have a single PR that contained > my code and the tests for it. What if after Jewel we pulled all of > ceph-qa-suite into the ceph repo? > > We could still enable folks running test changes without having to > rebuild ceph packages: the suite sha1 selected when running a > teuthology suite could still be different from that used for > installing ceph, it's just that it would fetch that sha1 from the ceph > repo instead of from a separate repo. This sounds pretty appealing to me... sage ^ permalink raw reply [flat|nested] 23+ messages in thread
[parent not found: <570E4843.8000102@dachary.org>]
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) [not found] ` <570E4843.8000102@dachary.org> @ 2016-04-13 13:58 ` Gregory Farnum 2016-04-13 18:31 ` John Spray 0 siblings, 1 reply; 23+ messages in thread From: Gregory Farnum @ 2016-04-13 13:58 UTC (permalink / raw) To: Loic Dachary; +Cc: John Spray, Ceph Development, ceph-qa On Wed, Apr 13, 2016 at 6:23 AM, Loic Dachary <loic@dachary.org> wrote: > +1 : I agree it would be a good thing. > > The reason why it would help to merge ceph-qa-suite in ceph is because we have no proper methods or tools to work with interdependent repositories. The problem is not unique to Ceph: every Free Software developer end up bug fixing and adding features to dependencies (ceph-qa-suite in the case of Ceph but also jerasure, rocksdb, s3test etc.). It will take a long time to resolve that more general problem and I don't know about an effort in that direction. Do you ? > > Cheers > > On 13/04/2016 12:52, John Spray wrote: >> Lately, we've instances where ceph.git and ceph-qa-suite.git got >> slightly out of sync, as we were adding new stuff and interface >> changes to ceph (especially in cephfs). >> >> We used to have two repos (ceph and teuthology), now we have three >> (ceph, ceph-qa-suite and teuthology). Splitting tests out of >> teuthology was a good thing, but maybe they should have gone into the >> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >> pretty much mirror what we do with ceph, with master vs jewel etc. >> >> Historically we had a comparatively static set of workloads in the qa >> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >> much with ceph changes. But these days we're adding much more >> detailed tests, so there's more effort to keep the two in sync. >> >> I would personally love to be able to have a single PR that contained >> my code and the tests for it. What if after Jewel we pulled all of >> ceph-qa-suite into the ceph repo? >> >> We could still enable folks running test changes without having to >> rebuild ceph packages: the suite sha1 selected when running a >> teuthology suite could still be different from that used for >> installing ceph, it's just that it would fetch that sha1 from the ceph >> repo instead of from a separate repo. We do have qa-suite tests that don't necessarily make a lot of sense to have in ceph.git. Samba, kernel NFS, ganesha some day. That doesn't mean we shouldn't merge them, but it popped into my head. -Greg ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 13:58 ` [Ceph-qa] " Gregory Farnum @ 2016-04-13 18:31 ` John Spray 2016-04-13 18:35 ` Samuel Just 0 siblings, 1 reply; 23+ messages in thread From: John Spray @ 2016-04-13 18:31 UTC (permalink / raw) To: Gregory Farnum; +Cc: Loic Dachary, Ceph Development, ceph-qa On Wed, Apr 13, 2016 at 2:58 PM, Gregory Farnum <gfarnum@redhat.com> wrote: > On Wed, Apr 13, 2016 at 6:23 AM, Loic Dachary <loic@dachary.org> wrote: >> +1 : I agree it would be a good thing. >> >> The reason why it would help to merge ceph-qa-suite in ceph is because we have no proper methods or tools to work with interdependent repositories. The problem is not unique to Ceph: every Free Software developer end up bug fixing and adding features to dependencies (ceph-qa-suite in the case of Ceph but also jerasure, rocksdb, s3test etc.). It will take a long time to resolve that more general problem and I don't know about an effort in that direction. Do you ? >> >> Cheers >> >> On 13/04/2016 12:52, John Spray wrote: >>> Lately, we've instances where ceph.git and ceph-qa-suite.git got >>> slightly out of sync, as we were adding new stuff and interface >>> changes to ceph (especially in cephfs). >>> >>> We used to have two repos (ceph and teuthology), now we have three >>> (ceph, ceph-qa-suite and teuthology). Splitting tests out of >>> teuthology was a good thing, but maybe they should have gone into the >>> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >>> pretty much mirror what we do with ceph, with master vs jewel etc. >>> >>> Historically we had a comparatively static set of workloads in the qa >>> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >>> much with ceph changes. But these days we're adding much more >>> detailed tests, so there's more effort to keep the two in sync. >>> >>> I would personally love to be able to have a single PR that contained >>> my code and the tests for it. What if after Jewel we pulled all of >>> ceph-qa-suite into the ceph repo? >>> >>> We could still enable folks running test changes without having to >>> rebuild ceph packages: the suite sha1 selected when running a >>> teuthology suite could still be different from that used for >>> installing ceph, it's just that it would fetch that sha1 from the ceph >>> repo instead of from a separate repo. > > We do have qa-suite tests that don't necessarily make a lot of sense > to have in ceph.git. Samba, kernel NFS, ganesha some day. That doesn't > mean we shouldn't merge them, but it popped into my head. Fair point. I think that because those other tasks depend in turn on the ceph tasks, we still ultimately benefit from having them in one place. It's also possible that in the long run things like the samba tests become a bit more "smart" in a way that's more tightly coupled to ceph, e.g. checking the resulting state inside ceph after doing things in the samba/ganesha layer, at which point we'd enjoy having them in the same place as the main body of test code. John ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 18:31 ` John Spray @ 2016-04-13 18:35 ` Samuel Just 2016-04-13 18:38 ` Sage Weil 0 siblings, 1 reply; 23+ messages in thread From: Samuel Just @ 2016-04-13 18:35 UTC (permalink / raw) To: John Spray; +Cc: Gregory Farnum, Loic Dachary, Ceph Development, ceph-qa It also doesn't seem like it would actually present a problem in any case. -Sam On Wed, Apr 13, 2016 at 11:31 AM, John Spray <jspray@redhat.com> wrote: > On Wed, Apr 13, 2016 at 2:58 PM, Gregory Farnum <gfarnum@redhat.com> wrote: >> On Wed, Apr 13, 2016 at 6:23 AM, Loic Dachary <loic@dachary.org> wrote: >>> +1 : I agree it would be a good thing. >>> >>> The reason why it would help to merge ceph-qa-suite in ceph is because we have no proper methods or tools to work with interdependent repositories. The problem is not unique to Ceph: every Free Software developer end up bug fixing and adding features to dependencies (ceph-qa-suite in the case of Ceph but also jerasure, rocksdb, s3test etc.). It will take a long time to resolve that more general problem and I don't know about an effort in that direction. Do you ? >>> >>> Cheers >>> >>> On 13/04/2016 12:52, John Spray wrote: >>>> Lately, we've instances where ceph.git and ceph-qa-suite.git got >>>> slightly out of sync, as we were adding new stuff and interface >>>> changes to ceph (especially in cephfs). >>>> >>>> We used to have two repos (ceph and teuthology), now we have three >>>> (ceph, ceph-qa-suite and teuthology). Splitting tests out of >>>> teuthology was a good thing, but maybe they should have gone into the >>>> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >>>> pretty much mirror what we do with ceph, with master vs jewel etc. >>>> >>>> Historically we had a comparatively static set of workloads in the qa >>>> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >>>> much with ceph changes. But these days we're adding much more >>>> detailed tests, so there's more effort to keep the two in sync. >>>> >>>> I would personally love to be able to have a single PR that contained >>>> my code and the tests for it. What if after Jewel we pulled all of >>>> ceph-qa-suite into the ceph repo? >>>> >>>> We could still enable folks running test changes without having to >>>> rebuild ceph packages: the suite sha1 selected when running a >>>> teuthology suite could still be different from that used for >>>> installing ceph, it's just that it would fetch that sha1 from the ceph >>>> repo instead of from a separate repo. >> >> We do have qa-suite tests that don't necessarily make a lot of sense >> to have in ceph.git. Samba, kernel NFS, ganesha some day. That doesn't >> mean we shouldn't merge them, but it popped into my head. > > Fair point. I think that because those other tasks depend in turn on > the ceph tasks, we still ultimately benefit from having them in one > place. > > It's also possible that in the long run things like the samba tests > become a bit more "smart" in a way that's more tightly coupled to > ceph, e.g. checking the resulting state inside ceph after doing things > in the samba/ganesha layer, at which point we'd enjoy having them in > the same place as the main body of test code. > > John > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 18:35 ` Samuel Just @ 2016-04-13 18:38 ` Sage Weil 2016-04-13 18:50 ` John Spray ` (2 more replies) 0 siblings, 3 replies; 23+ messages in thread From: Sage Weil @ 2016-04-13 18:38 UTC (permalink / raw) To: Samuel Just; +Cc: John Spray, Loic Dachary, Ceph Development, ceph-qa On Wed, 13 Apr 2016, Samuel Just wrote: > It also doesn't seem like it would actually present a problem in any case. The reason we didn't do this before was because we wanted to revise tests independently of the thing being tested. But as John points out, specifying a test branch should accomplish that, at least for testing. It might be nice to preserve the ability specify an alternate repo with totally out-of-tree tests. Maybe that can be done with a simple reorg of the repo, like putting everything under qa/, so that tasks/ and suites/ don't appear at the top level (of ceph.git or ceph-qa-suite.git). sage > -Sam > > On Wed, Apr 13, 2016 at 11:31 AM, John Spray <jspray@redhat.com> wrote: > > On Wed, Apr 13, 2016 at 2:58 PM, Gregory Farnum <gfarnum@redhat.com> wrote: > >> On Wed, Apr 13, 2016 at 6:23 AM, Loic Dachary <loic@dachary.org> wrote: > >>> +1 : I agree it would be a good thing. > >>> > >>> The reason why it would help to merge ceph-qa-suite in ceph is because we have no proper methods or tools to work with interdependent repositories. The problem is not unique to Ceph: every Free Software developer end up bug fixing and adding features to dependencies (ceph-qa-suite in the case of Ceph but also jerasure, rocksdb, s3test etc.). It will take a long time to resolve that more general problem and I don't know about an effort in that direction. Do you ? > >>> > >>> Cheers > >>> > >>> On 13/04/2016 12:52, John Spray wrote: > >>>> Lately, we've instances where ceph.git and ceph-qa-suite.git got > >>>> slightly out of sync, as we were adding new stuff and interface > >>>> changes to ceph (especially in cephfs). > >>>> > >>>> We used to have two repos (ceph and teuthology), now we have three > >>>> (ceph, ceph-qa-suite and teuthology). Splitting tests out of > >>>> teuthology was a good thing, but maybe they should have gone into the > >>>> ceph tree instead of a new repo? The ceph-qa-suite branching seems to > >>>> pretty much mirror what we do with ceph, with master vs jewel etc. > >>>> > >>>> Historically we had a comparatively static set of workloads in the qa > >>>> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change > >>>> much with ceph changes. But these days we're adding much more > >>>> detailed tests, so there's more effort to keep the two in sync. > >>>> > >>>> I would personally love to be able to have a single PR that contained > >>>> my code and the tests for it. What if after Jewel we pulled all of > >>>> ceph-qa-suite into the ceph repo? > >>>> > >>>> We could still enable folks running test changes without having to > >>>> rebuild ceph packages: the suite sha1 selected when running a > >>>> teuthology suite could still be different from that used for > >>>> installing ceph, it's just that it would fetch that sha1 from the ceph > >>>> repo instead of from a separate repo. > >> > >> We do have qa-suite tests that don't necessarily make a lot of sense > >> to have in ceph.git. Samba, kernel NFS, ganesha some day. That doesn't > >> mean we shouldn't merge them, but it popped into my head. > > > > Fair point. I think that because those other tasks depend in turn on > > the ceph tasks, we still ultimately benefit from having them in one > > place. > > > > It's also possible that in the long run things like the samba tests > > become a bit more "smart" in a way that's more tightly coupled to > > ceph, e.g. checking the resulting state inside ceph after doing things > > in the samba/ganesha layer, at which point we'd enjoy having them in > > the same place as the main body of test code. > > > > John > > -- > > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > > the body of a message to majordomo@vger.kernel.org > > More majordomo info at http://vger.kernel.org/majordomo-info.html > _______________________________________________ > Ceph-qa mailing list > Ceph-qa@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-qa-ceph.com > > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 18:38 ` Sage Weil @ 2016-04-13 18:50 ` John Spray 2016-04-13 18:54 ` Gregory Farnum 2016-04-13 20:12 ` Josh Durgin 2 siblings, 0 replies; 23+ messages in thread From: John Spray @ 2016-04-13 18:50 UTC (permalink / raw) To: Sage Weil; +Cc: Samuel Just, Loic Dachary, Ceph Development, ceph-qa On Wed, Apr 13, 2016 at 7:38 PM, Sage Weil <sweil@redhat.com> wrote: > On Wed, 13 Apr 2016, Samuel Just wrote: >> It also doesn't seem like it would actually present a problem in any case. > > The reason we didn't do this before was because we wanted to revise tests > independently of the thing being tested. But as John points out, > specifying a test branch should accomplish that, at least for testing. > > It might be nice to preserve the ability specify an alternate repo with > totally out-of-tree tests. Maybe that can be done with a simple reorg of > the repo, like putting everything under qa/, so that tasks/ and suites/ > don't appear at the top level (of ceph.git or ceph-qa-suite.git). Should be doable, teuthology will in any case still need to know how to pull old-style separate-repo setups for when it is asked to schedule a suite against a <= jewel branch. Might be simplest to just enable teuthology to have a suite path prefix in addition to its repo address, so that you could tell it "get the suites from repo X in /foo/bar" and avoid hardcoding it. Then for older branches it would be "github.com/ceph/ceph-qa-suite path /" and for newer stuff it would be "github.com/ceph/ceph path /wherever" John > sage > > > >> -Sam >> >> On Wed, Apr 13, 2016 at 11:31 AM, John Spray <jspray@redhat.com> wrote: >> > On Wed, Apr 13, 2016 at 2:58 PM, Gregory Farnum <gfarnum@redhat.com> wrote: >> >> On Wed, Apr 13, 2016 at 6:23 AM, Loic Dachary <loic@dachary.org> wrote: >> >>> +1 : I agree it would be a good thing. >> >>> >> >>> The reason why it would help to merge ceph-qa-suite in ceph is because we have no proper methods or tools to work with interdependent repositories. The problem is not unique to Ceph: every Free Software developer end up bug fixing and adding features to dependencies (ceph-qa-suite in the case of Ceph but also jerasure, rocksdb, s3test etc.). It will take a long time to resolve that more general problem and I don't know about an effort in that direction. Do you ? >> >>> >> >>> Cheers >> >>> >> >>> On 13/04/2016 12:52, John Spray wrote: >> >>>> Lately, we've instances where ceph.git and ceph-qa-suite.git got >> >>>> slightly out of sync, as we were adding new stuff and interface >> >>>> changes to ceph (especially in cephfs). >> >>>> >> >>>> We used to have two repos (ceph and teuthology), now we have three >> >>>> (ceph, ceph-qa-suite and teuthology). Splitting tests out of >> >>>> teuthology was a good thing, but maybe they should have gone into the >> >>>> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >> >>>> pretty much mirror what we do with ceph, with master vs jewel etc. >> >>>> >> >>>> Historically we had a comparatively static set of workloads in the qa >> >>>> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >> >>>> much with ceph changes. But these days we're adding much more >> >>>> detailed tests, so there's more effort to keep the two in sync. >> >>>> >> >>>> I would personally love to be able to have a single PR that contained >> >>>> my code and the tests for it. What if after Jewel we pulled all of >> >>>> ceph-qa-suite into the ceph repo? >> >>>> >> >>>> We could still enable folks running test changes without having to >> >>>> rebuild ceph packages: the suite sha1 selected when running a >> >>>> teuthology suite could still be different from that used for >> >>>> installing ceph, it's just that it would fetch that sha1 from the ceph >> >>>> repo instead of from a separate repo. >> >> >> >> We do have qa-suite tests that don't necessarily make a lot of sense >> >> to have in ceph.git. Samba, kernel NFS, ganesha some day. That doesn't >> >> mean we shouldn't merge them, but it popped into my head. >> > >> > Fair point. I think that because those other tasks depend in turn on >> > the ceph tasks, we still ultimately benefit from having them in one >> > place. >> > >> > It's also possible that in the long run things like the samba tests >> > become a bit more "smart" in a way that's more tightly coupled to >> > ceph, e.g. checking the resulting state inside ceph after doing things >> > in the samba/ganesha layer, at which point we'd enjoy having them in >> > the same place as the main body of test code. >> > >> > John >> > -- >> > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> > the body of a message to majordomo@vger.kernel.org >> > More majordomo info at http://vger.kernel.org/majordomo-info.html >> _______________________________________________ >> Ceph-qa mailing list >> Ceph-qa@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-qa-ceph.com >> >> ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 18:38 ` Sage Weil 2016-04-13 18:50 ` John Spray @ 2016-04-13 18:54 ` Gregory Farnum 2016-04-13 19:00 ` Sage Weil 2016-04-13 20:12 ` Josh Durgin 2 siblings, 1 reply; 23+ messages in thread From: Gregory Farnum @ 2016-04-13 18:54 UTC (permalink / raw) To: Sage Weil Cc: Samuel Just, John Spray, Loic Dachary, Ceph Development, ceph-qa On Wed, Apr 13, 2016 at 11:38 AM, Sage Weil <sweil@redhat.com> wrote: > On Wed, 13 Apr 2016, Samuel Just wrote: >> It also doesn't seem like it would actually present a problem in any case. > > The reason we didn't do this before was because we wanted to revise tests > independently of the thing being tested. But as John points out, > specifying a test branch should accomplish that, at least for testing. > > It might be nice to preserve the ability specify an alternate repo with > totally out-of-tree tests. Maybe that can be done with a simple reorg of > the repo, like putting everything under qa/, so that tasks/ and suites/ > don't appear at the top level (of ceph.git or ceph-qa-suite.git). I shudder to suggest it, but would we get what we're really after by including a ceph-qa-suite submodule in ceph.git, and having the teuthology scheduling refer to that? Then we can update ceph-qa-suite independently while still having an atomic switch for the tests that goes into the same Ceph branch which makes the changes requiring them. -Greg ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 18:54 ` Gregory Farnum @ 2016-04-13 19:00 ` Sage Weil 2016-04-13 19:06 ` Samuel Just 0 siblings, 1 reply; 23+ messages in thread From: Sage Weil @ 2016-04-13 19:00 UTC (permalink / raw) To: Gregory Farnum; +Cc: Loic Dachary, Ceph Development, ceph-qa, Samuel Just On Wed, 13 Apr 2016, Gregory Farnum wrote: > On Wed, Apr 13, 2016 at 11:38 AM, Sage Weil <sweil@redhat.com> wrote: > > On Wed, 13 Apr 2016, Samuel Just wrote: > >> It also doesn't seem like it would actually present a problem in any case. > > > > The reason we didn't do this before was because we wanted to revise tests > > independently of the thing being tested. But as John points out, > > specifying a test branch should accomplish that, at least for testing. > > > > It might be nice to preserve the ability specify an alternate repo with > > totally out-of-tree tests. Maybe that can be done with a simple reorg of > > the repo, like putting everything under qa/, so that tasks/ and suites/ > > don't appear at the top level (of ceph.git or ceph-qa-suite.git). > > I shudder to suggest it, but would we get what we're really after by > including a ceph-qa-suite submodule in ceph.git, and having the > teuthology scheduling refer to that? Then we can update ceph-qa-suite > independently while still having an atomic switch for the tests that > goes into the same Ceph branch which makes the changes requiring them. Not really. The workflow for updating submodules is really painful. Something like - commit to sub.git - push to branch in public sub.git - commit to main.git updating submodule sha - push to branch in main.git ... - merge main.git branch - merge sub.git branch (if you remember) ...and there are no "merge" semantics if there are racing sub.git updates. Submodules only really work for things that are infrequently updated... sage ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 19:00 ` Sage Weil @ 2016-04-13 19:06 ` Samuel Just 0 siblings, 0 replies; 23+ messages in thread From: Samuel Just @ 2016-04-13 19:06 UTC (permalink / raw) To: Sage Weil; +Cc: Gregory Farnum, Loic Dachary, Ceph Development, ceph-qa Seems like we just do a subtree merge into the ceph.git and call it good. -Sam On Wed, Apr 13, 2016 at 12:00 PM, Sage Weil <sweil@redhat.com> wrote: > On Wed, 13 Apr 2016, Gregory Farnum wrote: >> On Wed, Apr 13, 2016 at 11:38 AM, Sage Weil <sweil@redhat.com> wrote: >> > On Wed, 13 Apr 2016, Samuel Just wrote: >> >> It also doesn't seem like it would actually present a problem in any case. >> > >> > The reason we didn't do this before was because we wanted to revise tests >> > independently of the thing being tested. But as John points out, >> > specifying a test branch should accomplish that, at least for testing. >> > >> > It might be nice to preserve the ability specify an alternate repo with >> > totally out-of-tree tests. Maybe that can be done with a simple reorg of >> > the repo, like putting everything under qa/, so that tasks/ and suites/ >> > don't appear at the top level (of ceph.git or ceph-qa-suite.git). >> >> I shudder to suggest it, but would we get what we're really after by >> including a ceph-qa-suite submodule in ceph.git, and having the >> teuthology scheduling refer to that? Then we can update ceph-qa-suite >> independently while still having an atomic switch for the tests that >> goes into the same Ceph branch which makes the changes requiring them. > > Not really. The workflow for updating submodules is really painful. > Something like > > - commit to sub.git > - push to branch in public sub.git > - commit to main.git updating submodule sha > - push to branch in main.git > ... > - merge main.git branch > - merge sub.git branch (if you remember) > > ...and there are no "merge" semantics if there are racing sub.git updates. > > Submodules only really work for things that are infrequently updated... > > sage ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 18:38 ` Sage Weil 2016-04-13 18:50 ` John Spray 2016-04-13 18:54 ` Gregory Farnum @ 2016-04-13 20:12 ` Josh Durgin 2016-04-13 20:16 ` Samuel Just [not found] ` <570EB571.10907@dachary.org> 2 siblings, 2 replies; 23+ messages in thread From: Josh Durgin @ 2016-04-13 20:12 UTC (permalink / raw) To: Sage Weil, Samuel Just Cc: John Spray, Loic Dachary, Ceph Development, ceph-qa On 04/13/2016 11:38 AM, Sage Weil wrote: > On Wed, 13 Apr 2016, Samuel Just wrote: >> It also doesn't seem like it would actually present a problem in any case. > > The reason we didn't do this before was because we wanted to revise tests > independently of the thing being tested. But as John points out, > specifying a test branch should accomplish that, at least for testing. > > It might be nice to preserve the ability specify an alternate repo with > totally out-of-tree tests. Maybe that can be done with a simple reorg of > the repo, like putting everything under qa/, so that tasks/ and suites/ > don't appear at the top level (of ceph.git or ceph-qa-suite.git). It'll also be quite helpful to specify an alternate repo (not just branch) for testing suite changes without pushing to the main ceph.git and triggering gitbuilder builds. Josh > sage > > > >> -Sam >> >> On Wed, Apr 13, 2016 at 11:31 AM, John Spray <jspray@redhat.com> wrote: >>> On Wed, Apr 13, 2016 at 2:58 PM, Gregory Farnum <gfarnum@redhat.com> wrote: >>>> On Wed, Apr 13, 2016 at 6:23 AM, Loic Dachary <loic@dachary.org> wrote: >>>>> +1 : I agree it would be a good thing. >>>>> >>>>> The reason why it would help to merge ceph-qa-suite in ceph is because we have no proper methods or tools to work with interdependent repositories. The problem is not unique to Ceph: every Free Software developer end up bug fixing and adding features to dependencies (ceph-qa-suite in the case of Ceph but also jerasure, rocksdb, s3test etc.). It will take a long time to resolve that more general problem and I don't know about an effort in that direction. Do you ? >>>>> >>>>> Cheers >>>>> >>>>> On 13/04/2016 12:52, John Spray wrote: >>>>>> Lately, we've instances where ceph.git and ceph-qa-suite.git got >>>>>> slightly out of sync, as we were adding new stuff and interface >>>>>> changes to ceph (especially in cephfs). >>>>>> >>>>>> We used to have two repos (ceph and teuthology), now we have three >>>>>> (ceph, ceph-qa-suite and teuthology). Splitting tests out of >>>>>> teuthology was a good thing, but maybe they should have gone into the >>>>>> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >>>>>> pretty much mirror what we do with ceph, with master vs jewel etc. >>>>>> >>>>>> Historically we had a comparatively static set of workloads in the qa >>>>>> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >>>>>> much with ceph changes. But these days we're adding much more >>>>>> detailed tests, so there's more effort to keep the two in sync. >>>>>> >>>>>> I would personally love to be able to have a single PR that contained >>>>>> my code and the tests for it. What if after Jewel we pulled all of >>>>>> ceph-qa-suite into the ceph repo? >>>>>> >>>>>> We could still enable folks running test changes without having to >>>>>> rebuild ceph packages: the suite sha1 selected when running a >>>>>> teuthology suite could still be different from that used for >>>>>> installing ceph, it's just that it would fetch that sha1 from the ceph >>>>>> repo instead of from a separate repo. >>>> >>>> We do have qa-suite tests that don't necessarily make a lot of sense >>>> to have in ceph.git. Samba, kernel NFS, ganesha some day. That doesn't >>>> mean we shouldn't merge them, but it popped into my head. >>> >>> Fair point. I think that because those other tasks depend in turn on >>> the ceph tasks, we still ultimately benefit from having them in one >>> place. >>> >>> It's also possible that in the long run things like the samba tests >>> become a bit more "smart" in a way that's more tightly coupled to >>> ceph, e.g. checking the resulting state inside ceph after doing things >>> in the samba/ganesha layer, at which point we'd enjoy having them in >>> the same place as the main body of test code. >>> >>> John >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> _______________________________________________ >> Ceph-qa mailing list >> Ceph-qa@lists.ceph.com >> http://lists.ceph.com/listinfo.cgi/ceph-qa-ceph.com >> >> > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 20:12 ` Josh Durgin @ 2016-04-13 20:16 ` Samuel Just [not found] ` <570EB571.10907@dachary.org> 1 sibling, 0 replies; 23+ messages in thread From: Samuel Just @ 2016-04-13 20:16 UTC (permalink / raw) To: Josh Durgin Cc: Sage Weil, John Spray, Loic Dachary, Ceph Development, ceph-qa We need to do that anyway to make it easier to test ceph-qa-suite branches imho. -Sam On Wed, Apr 13, 2016 at 1:12 PM, Josh Durgin <jdurgin@redhat.com> wrote: > On 04/13/2016 11:38 AM, Sage Weil wrote: >> >> On Wed, 13 Apr 2016, Samuel Just wrote: >>> >>> It also doesn't seem like it would actually present a problem in any >>> case. >> >> >> The reason we didn't do this before was because we wanted to revise tests >> independently of the thing being tested. But as John points out, >> specifying a test branch should accomplish that, at least for testing. >> >> It might be nice to preserve the ability specify an alternate repo with >> totally out-of-tree tests. Maybe that can be done with a simple reorg of >> the repo, like putting everything under qa/, so that tasks/ and suites/ >> don't appear at the top level (of ceph.git or ceph-qa-suite.git). > > > It'll also be quite helpful to specify an alternate repo (not just > branch) for testing suite changes without pushing to the main ceph.git > and triggering gitbuilder builds. > > Josh > > >> sage >> >> >> >>> -Sam >>> >>> On Wed, Apr 13, 2016 at 11:31 AM, John Spray <jspray@redhat.com> wrote: >>>> >>>> On Wed, Apr 13, 2016 at 2:58 PM, Gregory Farnum <gfarnum@redhat.com> >>>> wrote: >>>>> >>>>> On Wed, Apr 13, 2016 at 6:23 AM, Loic Dachary <loic@dachary.org> wrote: >>>>>> >>>>>> +1 : I agree it would be a good thing. >>>>>> >>>>>> The reason why it would help to merge ceph-qa-suite in ceph is because >>>>>> we have no proper methods or tools to work with interdependent repositories. >>>>>> The problem is not unique to Ceph: every Free Software developer end up bug >>>>>> fixing and adding features to dependencies (ceph-qa-suite in the case of >>>>>> Ceph but also jerasure, rocksdb, s3test etc.). It will take a long time to >>>>>> resolve that more general problem and I don't know about an effort in that >>>>>> direction. Do you ? >>>>>> >>>>>> Cheers >>>>>> >>>>>> On 13/04/2016 12:52, John Spray wrote: >>>>>>> >>>>>>> Lately, we've instances where ceph.git and ceph-qa-suite.git got >>>>>>> slightly out of sync, as we were adding new stuff and interface >>>>>>> changes to ceph (especially in cephfs). >>>>>>> >>>>>>> We used to have two repos (ceph and teuthology), now we have three >>>>>>> (ceph, ceph-qa-suite and teuthology). Splitting tests out of >>>>>>> teuthology was a good thing, but maybe they should have gone into the >>>>>>> ceph tree instead of a new repo? The ceph-qa-suite branching seems >>>>>>> to >>>>>>> pretty much mirror what we do with ceph, with master vs jewel etc. >>>>>>> >>>>>>> Historically we had a comparatively static set of workloads in the qa >>>>>>> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >>>>>>> much with ceph changes. But these days we're adding much more >>>>>>> detailed tests, so there's more effort to keep the two in sync. >>>>>>> >>>>>>> I would personally love to be able to have a single PR that contained >>>>>>> my code and the tests for it. What if after Jewel we pulled all of >>>>>>> ceph-qa-suite into the ceph repo? >>>>>>> >>>>>>> We could still enable folks running test changes without having to >>>>>>> rebuild ceph packages: the suite sha1 selected when running a >>>>>>> teuthology suite could still be different from that used for >>>>>>> installing ceph, it's just that it would fetch that sha1 from the >>>>>>> ceph >>>>>>> repo instead of from a separate repo. >>>>> >>>>> >>>>> We do have qa-suite tests that don't necessarily make a lot of sense >>>>> to have in ceph.git. Samba, kernel NFS, ganesha some day. That doesn't >>>>> mean we shouldn't merge them, but it popped into my head. >>>> >>>> >>>> Fair point. I think that because those other tasks depend in turn on >>>> the ceph tasks, we still ultimately benefit from having them in one >>>> place. >>>> >>>> It's also possible that in the long run things like the samba tests >>>> become a bit more "smart" in a way that's more tightly coupled to >>>> ceph, e.g. checking the resulting state inside ceph after doing things >>>> in the samba/ganesha layer, at which point we'd enjoy having them in >>>> the same place as the main body of test code. >>>> >>>> John >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>> the body of a message to majordomo@vger.kernel.org >>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >>> _______________________________________________ >>> Ceph-qa mailing list >>> Ceph-qa@lists.ceph.com >>> http://lists.ceph.com/listinfo.cgi/ceph-qa-ceph.com >>> >>> >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > ^ permalink raw reply [flat|nested] 23+ messages in thread
[parent not found: <570EB571.10907@dachary.org>]
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) [not found] ` <570EB571.10907@dachary.org> @ 2016-04-13 21:11 ` Samuel Just 2016-04-13 21:17 ` Josh Durgin 1 sibling, 0 replies; 23+ messages in thread From: Samuel Just @ 2016-04-13 21:11 UTC (permalink / raw) To: Loic Dachary Cc: Josh Durgin, Sage Weil, John Spray, Ceph Development, ceph-qa Ah, I didn't know about that, I'll have to try it. -Sam On Wed, Apr 13, 2016 at 2:09 PM, Loic Dachary <loic@dachary.org> wrote: > > > On 13/04/2016 22:12, Josh Durgin wrote: >> On 04/13/2016 11:38 AM, Sage Weil wrote: >>> On Wed, 13 Apr 2016, Samuel Just wrote: >>>> It also doesn't seem like it would actually present a problem in any case. >>> >>> The reason we didn't do this before was because we wanted to revise tests >>> independently of the thing being tested. But as John points out, >>> specifying a test branch should accomplish that, at least for testing. >>> >>> It might be nice to preserve the ability specify an alternate repo with >>> totally out-of-tree tests. Maybe that can be done with a simple reorg of >>> the repo, like putting everything under qa/, so that tasks/ and suites/ >>> don't appear at the top level (of ceph.git or ceph-qa-suite.git). >> >> It'll also be quite helpful to specify an alternate repo (not just >> branch) for testing suite changes without pushing to the main ceph.git >> and triggering gitbuilder builds. > > We can keep the current --ceph-qa-suite-git-url argument for that. Or do you have something else in mind ? > >> Josh >> >>> sage >>> >>> >>> >>>> -Sam >>>> >>>> On Wed, Apr 13, 2016 at 11:31 AM, John Spray <jspray@redhat.com> wrote: >>>>> On Wed, Apr 13, 2016 at 2:58 PM, Gregory Farnum <gfarnum@redhat.com> wrote: >>>>>> On Wed, Apr 13, 2016 at 6:23 AM, Loic Dachary <loic@dachary.org> wrote: >>>>>>> +1 : I agree it would be a good thing. >>>>>>> >>>>>>> The reason why it would help to merge ceph-qa-suite in ceph is because we have no proper methods or tools to work with interdependent repositories. The problem is not unique to Ceph: every Free Software developer end up bug fixing and adding features to dependencies (ceph-qa-suite in the case of Ceph but also jerasure, rocksdb, s3test etc.). It will take a long time to resolve that more general problem and I don't know about an effort in that direction. Do you ? >>>>>>> >>>>>>> Cheers >>>>>>> >>>>>>> On 13/04/2016 12:52, John Spray wrote: >>>>>>>> Lately, we've instances where ceph.git and ceph-qa-suite.git got >>>>>>>> slightly out of sync, as we were adding new stuff and interface >>>>>>>> changes to ceph (especially in cephfs). >>>>>>>> >>>>>>>> We used to have two repos (ceph and teuthology), now we have three >>>>>>>> (ceph, ceph-qa-suite and teuthology). Splitting tests out of >>>>>>>> teuthology was a good thing, but maybe they should have gone into the >>>>>>>> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >>>>>>>> pretty much mirror what we do with ceph, with master vs jewel etc. >>>>>>>> >>>>>>>> Historically we had a comparatively static set of workloads in the qa >>>>>>>> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >>>>>>>> much with ceph changes. But these days we're adding much more >>>>>>>> detailed tests, so there's more effort to keep the two in sync. >>>>>>>> >>>>>>>> I would personally love to be able to have a single PR that contained >>>>>>>> my code and the tests for it. What if after Jewel we pulled all of >>>>>>>> ceph-qa-suite into the ceph repo? >>>>>>>> >>>>>>>> We could still enable folks running test changes without having to >>>>>>>> rebuild ceph packages: the suite sha1 selected when running a >>>>>>>> teuthology suite could still be different from that used for >>>>>>>> installing ceph, it's just that it would fetch that sha1 from the ceph >>>>>>>> repo instead of from a separate repo. >>>>>> >>>>>> We do have qa-suite tests that don't necessarily make a lot of sense >>>>>> to have in ceph.git. Samba, kernel NFS, ganesha some day. That doesn't >>>>>> mean we shouldn't merge them, but it popped into my head. >>>>> >>>>> Fair point. I think that because those other tasks depend in turn on >>>>> the ceph tasks, we still ultimately benefit from having them in one >>>>> place. >>>>> >>>>> It's also possible that in the long run things like the samba tests >>>>> become a bit more "smart" in a way that's more tightly coupled to >>>>> ceph, e.g. checking the resulting state inside ceph after doing things >>>>> in the samba/ganesha layer, at which point we'd enjoy having them in >>>>> the same place as the main body of test code. >>>>> >>>>> John >>>>> -- >>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>>>> the body of a message to majordomo@vger.kernel.org >>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>>> _______________________________________________ >>>> Ceph-qa mailing list >>>> Ceph-qa@lists.ceph.com >>>> http://lists.ceph.com/listinfo.cgi/ceph-qa-ceph.com >>>> >>>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >> >> > > -- > Loïc Dachary, Artisan Logiciel Libre -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) [not found] ` <570EB571.10907@dachary.org> 2016-04-13 21:11 ` Samuel Just @ 2016-04-13 21:17 ` Josh Durgin 2016-04-13 21:59 ` Loic Dachary 1 sibling, 1 reply; 23+ messages in thread From: Josh Durgin @ 2016-04-13 21:17 UTC (permalink / raw) To: Loic Dachary, Sage Weil, Samuel Just Cc: John Spray, Ceph Development, ceph-qa On 04/13/2016 02:09 PM, Loic Dachary wrote: > > > On 13/04/2016 22:12, Josh Durgin wrote: >> On 04/13/2016 11:38 AM, Sage Weil wrote: >>> On Wed, 13 Apr 2016, Samuel Just wrote: >>>> It also doesn't seem like it would actually present a problem in any case. >>> >>> The reason we didn't do this before was because we wanted to revise tests >>> independently of the thing being tested. But as John points out, >>> specifying a test branch should accomplish that, at least for testing. >>> >>> It might be nice to preserve the ability specify an alternate repo with >>> totally out-of-tree tests. Maybe that can be done with a simple reorg of >>> the repo, like putting everything under qa/, so that tasks/ and suites/ >>> don't appear at the top level (of ceph.git or ceph-qa-suite.git). >> >> It'll also be quite helpful to specify an alternate repo (not just >> branch) for testing suite changes without pushing to the main ceph.git >> and triggering gitbuilder builds. > > We can keep the current --ceph-qa-suite-git-url argument for that. Or do you have something else in mind ? I wasn't aware of that - looks like it's just a teuthology-openstack option. I was thinking we should have the same option for teuthology-suite. ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 21:17 ` Josh Durgin @ 2016-04-13 21:59 ` Loic Dachary 0 siblings, 0 replies; 23+ messages in thread From: Loic Dachary @ 2016-04-13 21:59 UTC (permalink / raw) To: Josh Durgin; +Cc: Ceph Development On 13/04/2016 23:17, Josh Durgin wrote: > On 04/13/2016 02:09 PM, Loic Dachary wrote: >> >> >> On 13/04/2016 22:12, Josh Durgin wrote: >>> On 04/13/2016 11:38 AM, Sage Weil wrote: >>>> On Wed, 13 Apr 2016, Samuel Just wrote: >>>>> It also doesn't seem like it would actually present a problem in any case. >>>> >>>> The reason we didn't do this before was because we wanted to revise tests >>>> independently of the thing being tested. But as John points out, >>>> specifying a test branch should accomplish that, at least for testing. >>>> >>>> It might be nice to preserve the ability specify an alternate repo with >>>> totally out-of-tree tests. Maybe that can be done with a simple reorg of >>>> the repo, like putting everything under qa/, so that tasks/ and suites/ >>>> don't appear at the top level (of ceph.git or ceph-qa-suite.git). >>> >>> It'll also be quite helpful to specify an alternate repo (not just >>> branch) for testing suite changes without pushing to the main ceph.git >>> and triggering gitbuilder builds. >> >> We can keep the current --ceph-qa-suite-git-url argument for that. Or do you have something else in mind ? > > I wasn't aware of that - looks like it's just a teuthology-openstack > option. I was thinking we should have the same option for teuthology-suite. Right. The teuthology-openstack option sets the ceph_qa_suite_git_url configuration option from ~/.teuthology.yaml which is not OpenStack specific. A naive implementation in teuthology-suite would be to replace the value in ~/.teuthology.yaml in the same way teuthology-openstack does it. It would however be problematic for a long lived cluster with multiple users racing with each other. A better implementation would be to have that as part of the config.yaml. -- Loïc Dachary, Artisan Logiciel Libre -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 23+ messages in thread
[parent not found: <CAKPXa=Y5rJ4ygM9Sdvys+86RwufnKopTOkMNdPHcpCh5bFbQ0w@mail.gmail.com>]
* Fwd: ceph-qa-suite branching (merge it into ceph.git?) [not found] ` <CAKPXa=Y5rJ4ygM9Sdvys+86RwufnKopTOkMNdPHcpCh5bFbQ0w@mail.gmail.com> @ 2016-04-13 18:10 ` Vasu Kulkarni 2016-04-13 18:23 ` Samuel Just 2016-04-13 18:46 ` John Spray 1 sibling, 1 reply; 23+ messages in thread From: Vasu Kulkarni @ 2016-04-13 18:10 UTC (permalink / raw) To: John Spray, Ceph Development, ceph-qa Sorry for spam, the first one was rejected due to text/html format. On Wed, Apr 13, 2016 at 3:52 AM, John Spray <jspray@redhat.com> wrote: > > Lately, we've instances where ceph.git and ceph-qa-suite.git got > slightly out of sync, as we were adding new stuff and interface > changes to ceph (especially in cephfs). > > We used to have two repos (ceph and teuthology), now we have three > (ceph, ceph-qa-suite and teuthology). we also have s3 suites in its own repo, we have big files in ceph.com/qa which i believe is the right place. > > Splitting tests out of > teuthology was a good thing, but maybe they should have gone into the > ceph tree instead of a new repo? The ceph-qa-suite branching seems to > pretty much mirror what we do with ceph, with master vs jewel etc. > > Historically we had a comparatively static set of workloads in the qa > suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change > much with ceph changes. But these days we're adding much more > detailed tests, so there's more effort to keep the two in sync. > > I would personally love to be able to have a single PR that contained > my code and the tests for it. What if after Jewel we pulled all of > ceph-qa-suite into the ceph repo? I understand the advantage of having same sha between builds and tests, but how are we going to guarantee ceph-builds in different places(one built on other than ceph.com) will have same sha? why not move ceph tests from ceph.git into ceph-qa-suite, Ideally this is not an issue with any major release, jewel workunits are supposed to work with jewel ceph, only in rare cases it may cause some issue but easily fixable. All the dev runs can point to any qa-sha using the cli option so much less of an issue in rare case. > > We could still enable folks running test changes without having to > rebuild ceph packages: the suite sha1 selected when running a > teuthology suite could still be different from that used for > installing ceph, it's just that it would fetch that sha1 from the ceph > repo instead of from a separate repo. > > John > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 18:10 ` Fwd: " Vasu Kulkarni @ 2016-04-13 18:23 ` Samuel Just 2016-04-13 18:57 ` Vasu Kulkarni 0 siblings, 1 reply; 23+ messages in thread From: Samuel Just @ 2016-04-13 18:23 UTC (permalink / raw) To: Vasu Kulkarni; +Cc: John Spray, Ceph Development, ceph-qa This seems tidy to me. I'd love to be able to merge a PR with it's ceph-qa-suite PR "atomically". -Sam On Wed, Apr 13, 2016 at 11:10 AM, Vasu Kulkarni <vakulkar@redhat.com> wrote: > Sorry for spam, the first one was rejected due to text/html format. > > On Wed, Apr 13, 2016 at 3:52 AM, John Spray <jspray@redhat.com> wrote: >> >> Lately, we've instances where ceph.git and ceph-qa-suite.git got >> slightly out of sync, as we were adding new stuff and interface >> changes to ceph (especially in cephfs). >> >> We used to have two repos (ceph and teuthology), now we have three >> (ceph, ceph-qa-suite and teuthology). > > > we also have s3 suites in its own repo, we have big files in > ceph.com/qa which i believe is the right place. > >> >> Splitting tests out of >> teuthology was a good thing, but maybe they should have gone into the >> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >> pretty much mirror what we do with ceph, with master vs jewel etc. >> >> Historically we had a comparatively static set of workloads in the qa >> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >> much with ceph changes. But these days we're adding much more >> detailed tests, so there's more effort to keep the two in sync. >> >> I would personally love to be able to have a single PR that contained >> my code and the tests for it. What if after Jewel we pulled all of >> ceph-qa-suite into the ceph repo? > > > I understand the advantage of having same sha between builds and > tests, but how are we going to guarantee ceph-builds in different > places(one built on other than ceph.com) will have same sha? > why not move ceph tests from ceph.git into ceph-qa-suite, Ideally > this is not an issue with any major release, jewel workunits are > supposed to work with jewel ceph, > only in rare cases it may cause some issue but easily fixable. All the > dev runs can point to any qa-sha using the cli option so much less of > an issue in rare case. > >> >> We could still enable folks running test changes without having to >> rebuild ceph packages: the suite sha1 selected when running a >> teuthology suite could still be different from that used for >> installing ceph, it's just that it would fetch that sha1 from the ceph >> repo instead of from a separate repo. >> >> John >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 18:23 ` Samuel Just @ 2016-04-13 18:57 ` Vasu Kulkarni 0 siblings, 0 replies; 23+ messages in thread From: Vasu Kulkarni @ 2016-04-13 18:57 UTC (permalink / raw) To: Samuel Just; +Cc: John Spray, Ceph Development, ceph-qa +1 for "atomic" PR, this should solve the problem. On Wed, Apr 13, 2016 at 11:23 AM, Samuel Just <sjust@redhat.com> wrote: > This seems tidy to me. I'd love to be able to merge a PR with it's > ceph-qa-suite PR "atomically". > -Sam > > On Wed, Apr 13, 2016 at 11:10 AM, Vasu Kulkarni <vakulkar@redhat.com> wrote: >> Sorry for spam, the first one was rejected due to text/html format. >> >> On Wed, Apr 13, 2016 at 3:52 AM, John Spray <jspray@redhat.com> wrote: >>> >>> Lately, we've instances where ceph.git and ceph-qa-suite.git got >>> slightly out of sync, as we were adding new stuff and interface >>> changes to ceph (especially in cephfs). >>> >>> We used to have two repos (ceph and teuthology), now we have three >>> (ceph, ceph-qa-suite and teuthology). >> >> >> we also have s3 suites in its own repo, we have big files in >> ceph.com/qa which i believe is the right place. >> >>> >>> Splitting tests out of >>> teuthology was a good thing, but maybe they should have gone into the >>> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >>> pretty much mirror what we do with ceph, with master vs jewel etc. >>> >>> Historically we had a comparatively static set of workloads in the qa >>> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >>> much with ceph changes. But these days we're adding much more >>> detailed tests, so there's more effort to keep the two in sync. >>> >>> I would personally love to be able to have a single PR that contained >>> my code and the tests for it. What if after Jewel we pulled all of >>> ceph-qa-suite into the ceph repo? >> >> >> I understand the advantage of having same sha between builds and >> tests, but how are we going to guarantee ceph-builds in different >> places(one built on other than ceph.com) will have same sha? >> why not move ceph tests from ceph.git into ceph-qa-suite, Ideally >> this is not an issue with any major release, jewel workunits are >> supposed to work with jewel ceph, >> only in rare cases it may cause some issue but easily fixable. All the >> dev runs can point to any qa-sha using the cli option so much less of >> an issue in rare case. >> >>> >>> We could still enable folks running test changes without having to >>> rebuild ceph packages: the suite sha1 selected when running a >>> teuthology suite could still be different from that used for >>> installing ceph, it's just that it would fetch that sha1 from the ceph >>> repo instead of from a separate repo. >>> >>> John >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: ceph-qa-suite branching (merge it into ceph.git?) [not found] ` <CAKPXa=Y5rJ4ygM9Sdvys+86RwufnKopTOkMNdPHcpCh5bFbQ0w@mail.gmail.com> 2016-04-13 18:10 ` Fwd: " Vasu Kulkarni @ 2016-04-13 18:46 ` John Spray 2016-04-13 19:06 ` Vasu Kulkarni ` (2 more replies) 1 sibling, 3 replies; 23+ messages in thread From: John Spray @ 2016-04-13 18:46 UTC (permalink / raw) To: Vasu Kulkarni; +Cc: Ceph Development, ceph-qa On Wed, Apr 13, 2016 at 7:02 PM, Vasu Kulkarni <vakulkar@redhat.com> wrote: > > > On Wed, Apr 13, 2016 at 3:52 AM, John Spray <jspray@redhat.com> wrote: >> >> Lately, we've instances where ceph.git and ceph-qa-suite.git got >> slightly out of sync, as we were adding new stuff and interface >> changes to ceph (especially in cephfs). >> >> We used to have two repos (ceph and teuthology), now we have three >> (ceph, ceph-qa-suite and teuthology). > > > we also have s3 suites in its own repo, we have big files in ceph.com/qa > which i believe is the right place. I'm a bit ignorant of the s3 tests, is there a particular reason they live separately to the main ceph-qa-suite? As for the stuff in ceph.com/qa, I of course am not suggesting putting those bit binaries into the main git repo. It is sort of annoying that they live out there on a web server, but it's a separate issue from the ceph/ceph-qa-suite one. >> Splitting tests out of >> teuthology was a good thing, but maybe they should have gone into the >> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >> pretty much mirror what we do with ceph, with master vs jewel etc. >> >> Historically we had a comparatively static set of workloads in the qa >> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >> much with ceph changes. But these days we're adding much more >> detailed tests, so there's more effort to keep the two in sync. >> >> I would personally love to be able to have a single PR that contained >> my code and the tests for it. What if after Jewel we pulled all of >> ceph-qa-suite into the ceph repo? > > > I understand the advantage of having same sha between builds and tests, but > how are we going to guarantee ceph-builds in different places(one built on > other than ceph.com) will have same sha? The thing that needs to match is the sha1 of the ceph code you're running, and the sha1 you use when pulling the code to test it with. I believe you currently have a situation where you have a built RPM but you're not sure what ceph sha1 it came from, so you're pulling e.g. the jewel branch of ceph-qa-suite rather than a specific sha1. That needs fixing independently of ceph/ceph-qa-suite separation: you have to run the right revision of the tests, irrespective of where they're getting pulled from. Basically, anyone generating packages with versions that don't map to a ceph sha1 will need to have a way of remembering what ceph sha1 their packages correspond to. This is not a new requirement, it's the same issue that was discussed on IRC recently. > why not move ceph tests from ceph.git into ceph-qa-suite, Ideally this is > not an issue with any major release, jewel workunits are supposed to work > with jewel ceph, That would not help, because you still have the same issue of changes in ceph being out of step with changes in ceph-qa-suite. I think your misconception here is to assume that running tests from the same branch as the code is sufficient. You need to run tests from the same *revision*. Otherwise, when I add a new test to reproduce a bug, and fix the bug at the same time, then you will get failures if you run the newer commit against the older code. John > only in rare cases it may cause some issue but easily fixable. All the dev > runs can point to any qa-sha using the cli option so much less of an issue > in rare case. >> >> We could still enable folks running test changes without having to >> rebuild ceph packages: the suite sha1 selected when running a >> teuthology suite could still be different from that used for >> installing ceph, it's just that it would fetch that sha1 from the ceph >> repo instead of from a separate repo. >> >> John >> -- >> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > > ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 18:46 ` John Spray @ 2016-04-13 19:06 ` Vasu Kulkarni 2016-04-13 19:09 ` [Ceph-qa] " Sage Weil 2016-04-13 19:08 ` Josh Durgin 2016-04-13 19:18 ` Abhishek Lekshmanan 2 siblings, 1 reply; 23+ messages in thread From: Vasu Kulkarni @ 2016-04-13 19:06 UTC (permalink / raw) To: John Spray; +Cc: Ceph Development, ceph-qa On Wed, Apr 13, 2016 at 11:46 AM, John Spray <jspray@redhat.com> wrote: > On Wed, Apr 13, 2016 at 7:02 PM, Vasu Kulkarni <vakulkar@redhat.com> wrote: >> >> >> On Wed, Apr 13, 2016 at 3:52 AM, John Spray <jspray@redhat.com> wrote: >>> >>> Lately, we've instances where ceph.git and ceph-qa-suite.git got >>> slightly out of sync, as we were adding new stuff and interface >>> changes to ceph (especially in cephfs). >>> >>> We used to have two repos (ceph and teuthology), now we have three >>> (ceph, ceph-qa-suite and teuthology). >> >> >> we also have s3 suites in its own repo, we have big files in ceph.com/qa >> which i believe is the right place. > > I'm a bit ignorant of the s3 tests, is there a particular reason they > live separately to the main ceph-qa-suite? Not sure, probably yehuda wanted to make it easier to test with Amazon::S3 and Ceph > > As for the stuff in ceph.com/qa, I of course am not suggesting putting > those bit binaries into the main git repo. It is sort of annoying > that they live out there on a web server, but it's a separate issue > from the ceph/ceph-qa-suite one. > >>> Splitting tests out of >>> teuthology was a good thing, but maybe they should have gone into the >>> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >>> pretty much mirror what we do with ceph, with master vs jewel etc. >>> >>> Historically we had a comparatively static set of workloads in the qa >>> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >>> much with ceph changes. But these days we're adding much more >>> detailed tests, so there's more effort to keep the two in sync. >>> >>> I would personally love to be able to have a single PR that contained >>> my code and the tests for it. What if after Jewel we pulled all of >>> ceph-qa-suite into the ceph repo? >> >> >> I understand the advantage of having same sha between builds and tests, but >> how are we going to guarantee ceph-builds in different places(one built on >> other than ceph.com) will have same sha? > > The thing that needs to match is the sha1 of the ceph code you're > running, and the sha1 you use when pulling the code to test it with. > > I believe you currently have a situation where you have a built RPM > but you're not sure what ceph sha1 it came from, so you're pulling > e.g. the jewel branch of ceph-qa-suite rather than a specific sha1. > That needs fixing independently of ceph/ceph-qa-suite separation: you > have to run the right revision of the tests, irrespective of where > they're getting pulled from. > > Basically, anyone generating packages with versions that don't map to > a ceph sha1 will need to have a way of remembering what ceph sha1 > their packages correspond to. This is not a new requirement, it's the > same issue that was discussed on IRC recently. > >> why not move ceph tests from ceph.git into ceph-qa-suite, Ideally this is >> not an issue with any major release, jewel workunits are supposed to work >> with jewel ceph, > > That would not help, because you still have the same issue of changes > in ceph being out of step with changes in ceph-qa-suite. > > I think your misconception here is to assume that running tests from > the same branch as the code is sufficient. You need to run tests from > the same *revision*. Otherwise, when I add a new test to reproduce a > bug, and fix the bug at the same time, then you will get failures if > you run the newer commit against the older code. This one I think figured out, I agree it needs to match for those rare cases, but the option exists in teuthology with overrides, just need to have right override. So hopefully it shoudn't be an issue anymore. > > John > >> only in rare cases it may cause some issue but easily fixable. All the dev >> runs can point to any qa-sha using the cli option so much less of an issue >> in rare case. > > > >>> >>> We could still enable folks running test changes without having to >>> rebuild ceph packages: the suite sha1 selected when running a >>> teuthology suite could still be different from that used for >>> installing ceph, it's just that it would fetch that sha1 from the ceph >>> repo instead of from a separate repo. >>> >>> John >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Ceph-qa] ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 19:06 ` Vasu Kulkarni @ 2016-04-13 19:09 ` Sage Weil 0 siblings, 0 replies; 23+ messages in thread From: Sage Weil @ 2016-04-13 19:09 UTC (permalink / raw) To: Vasu Kulkarni; +Cc: John Spray, Ceph Development, ceph-qa On Wed, 13 Apr 2016, Vasu Kulkarni wrote: > On Wed, Apr 13, 2016 at 11:46 AM, John Spray <jspray@redhat.com> wrote: > > On Wed, Apr 13, 2016 at 7:02 PM, Vasu Kulkarni <vakulkar@redhat.com> wrote: > >> > >> > >> On Wed, Apr 13, 2016 at 3:52 AM, John Spray <jspray@redhat.com> wrote: > >>> > >>> Lately, we've instances where ceph.git and ceph-qa-suite.git got > >>> slightly out of sync, as we were adding new stuff and interface > >>> changes to ceph (especially in cephfs). > >>> > >>> We used to have two repos (ceph and teuthology), now we have three > >>> (ceph, ceph-qa-suite and teuthology). > >> > >> > >> we also have s3 suites in its own repo, we have big files in ceph.com/qa > >> which i believe is the right place. > > > > I'm a bit ignorant of the s3 tests, is there a particular reason they > > live separately to the main ceph-qa-suite? > > Not sure, probably yehuda wanted to make it easier to test with > Amazon::S3 and Ceph The goal was to make them an independently useful set of functional tests for the S3 API, used not just by radosgw but also by other projects, so that we are not solely responsible for maintaining them. I think it makes sense to keep them separate. sage ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 18:46 ` John Spray 2016-04-13 19:06 ` Vasu Kulkarni @ 2016-04-13 19:08 ` Josh Durgin 2016-04-13 19:18 ` Abhishek Lekshmanan 2 siblings, 0 replies; 23+ messages in thread From: Josh Durgin @ 2016-04-13 19:08 UTC (permalink / raw) To: John Spray, Vasu Kulkarni; +Cc: Ceph Development, ceph-qa On 04/13/2016 11:46 AM, John Spray wrote: > On Wed, Apr 13, 2016 at 7:02 PM, Vasu Kulkarni <vakulkar@redhat.com> wrote: >> we also have s3 suites in its own repo, we have big files in ceph.com/qa >> which i believe is the right place. > > I'm a bit ignorant of the s3 tests, is there a particular reason they > live separately to the main ceph-qa-suite? s3 tests are more generic. They run against other s3 implementations, and only a labeled subset work with rgw. Josh ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: ceph-qa-suite branching (merge it into ceph.git?) 2016-04-13 18:46 ` John Spray 2016-04-13 19:06 ` Vasu Kulkarni 2016-04-13 19:08 ` Josh Durgin @ 2016-04-13 19:18 ` Abhishek Lekshmanan 2 siblings, 0 replies; 23+ messages in thread From: Abhishek Lekshmanan @ 2016-04-13 19:18 UTC (permalink / raw) To: John Spray; +Cc: Vasu Kulkarni, Ceph Development, ceph-qa John Spray writes: > On Wed, Apr 13, 2016 at 7:02 PM, Vasu Kulkarni <vakulkar@redhat.com> wrote: >> >> >> On Wed, Apr 13, 2016 at 3:52 AM, John Spray <jspray@redhat.com> wrote: >>> >>> Lately, we've instances where ceph.git and ceph-qa-suite.git got >>> slightly out of sync, as we were adding new stuff and interface >>> changes to ceph (especially in cephfs). >>> >>> We used to have two repos (ceph and teuthology), now we have three >>> (ceph, ceph-qa-suite and teuthology). >> >> >> we also have s3 suites in its own repo, we have big files in ceph.com/qa >> which i believe is the right place. > > I'm a bit ignorant of the s3 tests, is there a particular reason they > live separately to the main ceph-qa-suite? Technically s3-tests can have tests that do not all pass on rgw (which is true mostly even now as we have a fails_on_rgw filter) and pass on Amazon S3. I'm guessing this may be a reason they live outside of the qa-suite, but I could be wrong Abhishek > > As for the stuff in ceph.com/qa, I of course am not suggesting putting > those bit binaries into the main git repo. It is sort of annoying > that they live out there on a web server, but it's a separate issue > from the ceph/ceph-qa-suite one. > >>> Splitting tests out of >>> teuthology was a good thing, but maybe they should have gone into the >>> ceph tree instead of a new repo? The ceph-qa-suite branching seems to >>> pretty much mirror what we do with ceph, with master vs jewel etc. >>> >>> Historically we had a comparatively static set of workloads in the qa >>> suite (e.g. kernel-untar-build, fsstresss, pjd), which didn't change >>> much with ceph changes. But these days we're adding much more >>> detailed tests, so there's more effort to keep the two in sync. >>> >>> I would personally love to be able to have a single PR that contained >>> my code and the tests for it. What if after Jewel we pulled all of >>> ceph-qa-suite into the ceph repo? >> >> >> I understand the advantage of having same sha between builds and tests, but >> how are we going to guarantee ceph-builds in different places(one built on >> other than ceph.com) will have same sha? > > The thing that needs to match is the sha1 of the ceph code you're > running, and the sha1 you use when pulling the code to test it with. > > I believe you currently have a situation where you have a built RPM > but you're not sure what ceph sha1 it came from, so you're pulling > e.g. the jewel branch of ceph-qa-suite rather than a specific sha1. > That needs fixing independently of ceph/ceph-qa-suite separation: you > have to run the right revision of the tests, irrespective of where > they're getting pulled from. > > Basically, anyone generating packages with versions that don't map to > a ceph sha1 will need to have a way of remembering what ceph sha1 > their packages correspond to. This is not a new requirement, it's the > same issue that was discussed on IRC recently. > >> why not move ceph tests from ceph.git into ceph-qa-suite, Ideally this is >> not an issue with any major release, jewel workunits are supposed to work >> with jewel ceph, > > That would not help, because you still have the same issue of changes > in ceph being out of step with changes in ceph-qa-suite. > > I think your misconception here is to assume that running tests from > the same branch as the code is sufficient. You need to run tests from > the same *revision*. Otherwise, when I add a new test to reproduce a > bug, and fix the bug at the same time, then you will get failures if > you run the newer commit against the older code. > > John > >> only in rare cases it may cause some issue but easily fixable. All the dev >> runs can point to any qa-sha using the cli option so much less of an issue >> in rare case. > > > >>> >>> We could still enable folks running test changes without having to >>> rebuild ceph packages: the suite sha1 selected when running a >>> teuthology suite could still be different from that used for >>> installing ceph, it's just that it would fetch that sha1 from the ceph >>> repo instead of from a separate repo. >>> >>> John >>> -- >>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >>> the body of a message to majordomo@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> -- Abhishek ^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2016-04-13 21:59 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-13 10:52 ceph-qa-suite branching (merge it into ceph.git?) John Spray
2016-04-13 12:45 ` Sage Weil
[not found] ` <570E4843.8000102@dachary.org>
2016-04-13 13:58 ` [Ceph-qa] " Gregory Farnum
2016-04-13 18:31 ` John Spray
2016-04-13 18:35 ` Samuel Just
2016-04-13 18:38 ` Sage Weil
2016-04-13 18:50 ` John Spray
2016-04-13 18:54 ` Gregory Farnum
2016-04-13 19:00 ` Sage Weil
2016-04-13 19:06 ` Samuel Just
2016-04-13 20:12 ` Josh Durgin
2016-04-13 20:16 ` Samuel Just
[not found] ` <570EB571.10907@dachary.org>
2016-04-13 21:11 ` Samuel Just
2016-04-13 21:17 ` Josh Durgin
2016-04-13 21:59 ` Loic Dachary
[not found] ` <CAKPXa=Y5rJ4ygM9Sdvys+86RwufnKopTOkMNdPHcpCh5bFbQ0w@mail.gmail.com>
2016-04-13 18:10 ` Fwd: " Vasu Kulkarni
2016-04-13 18:23 ` Samuel Just
2016-04-13 18:57 ` Vasu Kulkarni
2016-04-13 18:46 ` John Spray
2016-04-13 19:06 ` Vasu Kulkarni
2016-04-13 19:09 ` [Ceph-qa] " Sage Weil
2016-04-13 19:08 ` Josh Durgin
2016-04-13 19:18 ` Abhishek Lekshmanan
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.