* Re: hammer 0.94.10 QE status
[not found] <CAMMFjmHCAM6wmygytBDqwh89UOEex24_GGqrLJvXcOUS+QEg9A@mail.gmail.com>
@ 2017-02-01 17:03 ` Ilya Dryomov
2017-02-02 10:01 ` Ilya Dryomov
2017-02-01 19:12 ` John Spray
1 sibling, 1 reply; 21+ messages in thread
From: Ilya Dryomov @ 2017-02-01 17:03 UTC (permalink / raw)
To: Yuri Weinstein
Cc: Development, Ceph, Nathan Cutler, Weil, Sage, Durgin, Josh, Just,
Samuel, Dillaman, Jason, Gregory Farnum, John Spray,
Sadeh-Weinraub, Yehuda
On Wed, Feb 1, 2017 at 5:52 PM, Yuri Weinstein <yweinste@redhat.com> wrote:
> Here is snapshot of hammer 0.94.10 point release testing status.
>
> Suites were not tested on ubuntu 16.06 due to
> http://tracker.ceph.com/issues/18382 and as a result were forced to test on
> ubuntu 14.04 and centos 7.2 (note we don't have smithi with centos 7.3).
>
> There was an additional noise created by fact that since we tested last
> hammer release in August 2016 we moved from gitbuilders to shaman build
> system and it resulted in issues with some packages.
>
> Details of tests can be found here
> http://tracker.ceph.com/issues/17151#note-61
>
> ======================
> rados ubuntu - packaging issues, need approval - Josh
> rados centos - packaging issues, need approval - Josh
> ======================
> rbd - PASSED
> ======================
> rgw ubuntu - PASSED
> rgw centos - valgrind issues http://tracker.ceph.com/issues/18735, need
> approval Yehuda, Sage
> ======================
> fs ubuntu - looks similar to v0.94.8, need Greg, John approval
> fs centos - noise? need Greg, John review/approval
> ======================
> krbd ubuntu - looks like the same in v0.94.8
> http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
> krbd centos - looks like the same in v0.94.8
> http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
I'll need to look at
http://pulpito.ceph.com/teuthology-2017-01-26_20:58:56-krbd-master-testing-basic-smithi/752997
http://pulpito.ceph.com/teuthology-2017-01-26_21:01:15-krbd-master-testing-basic-smithi/753036
These are unexpected.
Thanks,
Ilya
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
[not found] <CAMMFjmHCAM6wmygytBDqwh89UOEex24_GGqrLJvXcOUS+QEg9A@mail.gmail.com>
2017-02-01 17:03 ` hammer 0.94.10 QE status Ilya Dryomov
@ 2017-02-01 19:12 ` John Spray
1 sibling, 0 replies; 21+ messages in thread
From: John Spray @ 2017-02-01 19:12 UTC (permalink / raw)
To: Yuri Weinstein
Cc: Development, Ceph, Nathan Cutler, Weil, Sage, Durgin, Josh, Just,
Samuel, Dillaman, Jason, Ilya Dryomov, Gregory Farnum,
Sadeh-Weinraub, Yehuda
On Wed, Feb 1, 2017 at 4:52 PM, Yuri Weinstein <yweinste@redhat.com> wrote:
> Here is snapshot of hammer 0.94.10 point release testing status.
>
> Suites were not tested on ubuntu 16.06 due to
> http://tracker.ceph.com/issues/18382 and as a result were forced to test on
> ubuntu 14.04 and centos 7.2 (note we don't have smithi with centos 7.3).
>
> There was an additional noise created by fact that since we tested last
> hammer release in August 2016 we moved from gitbuilders to shaman build
> system and it resulted in issues with some packages.
>
> Details of tests can be found here
> http://tracker.ceph.com/issues/17151#note-61
>
> ======================
> rados ubuntu - packaging issues, need approval - Josh
> rados centos - packaging issues, need approval - Josh
> ======================
> rbd - PASSED
> ======================
> rgw ubuntu - PASSED
> rgw centos - valgrind issues http://tracker.ceph.com/issues/18735, need
> approval Yehuda, Sage
> ======================
> fs ubuntu - looks similar to v0.94.8, need Greg, John approval
> fs centos - noise? need Greg, John review/approval
The MDL_WriteError one is the bug from when ENOSPC stuff was
backported wrongly, the FileJournal/FileStore assertions are from
btrfs.
Good to go.
John
> ======================
> krbd ubuntu - looks like the same in v0.94.8
> http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
> krbd centos - looks like the same in v0.94.8
> http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
> ======================
> kcephfs - PASSED
> ======================
> knfs - PASSED
> ======================
> hadoop - PASSED
> ======================
> samba - packaging issues, need approval - Greg, Sage
> ======================
> rest - PASSED
> ======================
> ceph-deploy - Vasu is fixing, need approval Sage
> ======================
> upgrade/client-upgrade - PASSED
> ======================
> upgrade/firefly-x (hammer) - need Sage approval. maybe deprecate?
> ======================
> upgrade/hammer - PASSED
> ======================
> upgrade/hammer-x (jewel) - FAILED, need Sage approval. maybe deprecate?
> ======================
> powercycle - PASSED
> ======================
>
> Thx
> YuriW
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-01 17:03 ` hammer 0.94.10 QE status Ilya Dryomov
@ 2017-02-02 10:01 ` Ilya Dryomov
2017-02-02 11:17 ` Nathan Cutler
0 siblings, 1 reply; 21+ messages in thread
From: Ilya Dryomov @ 2017-02-02 10:01 UTC (permalink / raw)
To: Yuri Weinstein
Cc: Development, Ceph, Nathan Cutler, Weil, Sage, Durgin, Josh, Just,
Samuel, Dillaman, Jason, Gregory Farnum, John Spray,
Sadeh-Weinraub, Yehuda
On Wed, Feb 1, 2017 at 6:03 PM, Ilya Dryomov <idryomov@gmail.com> wrote:
> On Wed, Feb 1, 2017 at 5:52 PM, Yuri Weinstein <yweinste@redhat.com> wrote:
>> Here is snapshot of hammer 0.94.10 point release testing status.
>>
>> Suites were not tested on ubuntu 16.06 due to
>> http://tracker.ceph.com/issues/18382 and as a result were forced to test on
>> ubuntu 14.04 and centos 7.2 (note we don't have smithi with centos 7.3).
>>
>> There was an additional noise created by fact that since we tested last
>> hammer release in August 2016 we moved from gitbuilders to shaman build
>> system and it resulted in issues with some packages.
>>
>> Details of tests can be found here
>> http://tracker.ceph.com/issues/17151#note-61
>>
>> ======================
>> rados ubuntu - packaging issues, need approval - Josh
>> rados centos - packaging issues, need approval - Josh
>> ======================
>> rbd - PASSED
>> ======================
>> rgw ubuntu - PASSED
>> rgw centos - valgrind issues http://tracker.ceph.com/issues/18735, need
>> approval Yehuda, Sage
>> ======================
>> fs ubuntu - looks similar to v0.94.8, need Greg, John approval
>> fs centos - noise? need Greg, John review/approval
>> ======================
>> krbd ubuntu - looks like the same in v0.94.8
>> http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
>> krbd centos - looks like the same in v0.94.8
>> http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
>
> I'll need to look at
>
> http://pulpito.ceph.com/teuthology-2017-01-26_20:58:56-krbd-master-testing-basic-smithi/752997
> http://pulpito.ceph.com/teuthology-2017-01-26_21:01:15-krbd-master-testing-basic-smithi/753036
>
> These are unexpected.
Turned out to be a buggy test -- hammer is missing an infernalis
commit f217865cf7c4 ("test_librbd_fsx: invalidate before discard in
krbd mode").
Approved.
Thanks,
Ilya
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-02 10:01 ` Ilya Dryomov
@ 2017-02-02 11:17 ` Nathan Cutler
2017-02-07 21:47 ` Yuri Weinstein
2017-02-14 17:05 ` Yuri Weinstein
0 siblings, 2 replies; 21+ messages in thread
From: Nathan Cutler @ 2017-02-02 11:17 UTC (permalink / raw)
To: Ilya Dryomov, Yuri Weinstein
Cc: Development, Ceph, Weil, Sage, Durgin, Josh, Just, Samuel,
Dillaman, Jason, Gregory Farnum, John Spray, Sadeh-Weinraub,
Yehuda
> Turned out to be a buggy test -- hammer is missing an infernalis
> commit f217865cf7c4 ("test_librbd_fsx: invalidate before discard in
> krbd mode").
I gave it a try; please take a look:
* https://github.com/ceph/ceph/pull/13230
Nathan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-02 11:17 ` Nathan Cutler
@ 2017-02-07 21:47 ` Yuri Weinstein
2017-02-08 21:14 ` Yuri Weinstein
2017-02-14 17:05 ` Yuri Weinstein
1 sibling, 1 reply; 21+ messages in thread
From: Yuri Weinstein @ 2017-02-07 21:47 UTC (permalink / raw)
To: Nathan Cutler
Cc: Ilya Dryomov, Development, Ceph, Weil, Sage, Durgin, Josh, Just,
Samuel, Dillaman, Jason, Gregory Farnum, John Spray,
Sadeh-Weinraub, Yehuda
See updated status with approvals http://tracker.ceph.com/issues/17151#note-61
Outstanding suites requiring approval:
rgw
samba
upgrade/firefly-x (hammer)
upgrade/hammer-x (jewel)
Thx
YuriW
On Thu, Feb 2, 2017 at 3:17 AM, Nathan Cutler <ncutler@suse.cz> wrote:
>> Turned out to be a buggy test -- hammer is missing an infernalis
>> commit f217865cf7c4 ("test_librbd_fsx: invalidate before discard in
>> krbd mode").
>
>
> I gave it a try; please take a look:
>
> * https://github.com/ceph/ceph/pull/13230
>
> Nathan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-07 21:47 ` Yuri Weinstein
@ 2017-02-08 21:14 ` Yuri Weinstein
[not found] ` <CAObvjwPb_PosEnGOXP2tV4aWjnTyed7-275svyK+vu4uxYdTzQ@mail.gmail.com>
0 siblings, 1 reply; 21+ messages in thread
From: Yuri Weinstein @ 2017-02-08 21:14 UTC (permalink / raw)
To: Nathan Cutler
Cc: Ilya Dryomov, Development, Ceph, Weil, Sage, Durgin, Josh, Just,
Samuel, Dillaman, Jason, Gregory Farnum, John Spray,
Sadeh-Weinraub, Yehuda
upgrade/hammer-x (jewel) PASSED now!
Outstanding suites requiring approval:
rgw
samba
upgrade/firefly-x (hammer)
Sage?
Thx
YuriW
On Tue, Feb 7, 2017 at 1:47 PM, Yuri Weinstein <yweinste@redhat.com> wrote:
> See updated status with approvals http://tracker.ceph.com/issues/17151#note-61
>
> Outstanding suites requiring approval:
>
> rgw
> samba
> upgrade/firefly-x (hammer)
> upgrade/hammer-x (jewel)
>
> Thx
> YuriW
>
> On Thu, Feb 2, 2017 at 3:17 AM, Nathan Cutler <ncutler@suse.cz> wrote:
>>> Turned out to be a buggy test -- hammer is missing an infernalis
>>> commit f217865cf7c4 ("test_librbd_fsx: invalidate before discard in
>>> krbd mode").
>>
>>
>> I gave it a try; please take a look:
>>
>> * https://github.com/ceph/ceph/pull/13230
>>
>> Nathan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
[not found] ` <CAObvjwPb_PosEnGOXP2tV4aWjnTyed7-275svyK+vu4uxYdTzQ@mail.gmail.com>
@ 2017-02-13 16:24 ` Sage Weil
2017-02-14 14:30 ` Orit Wasserman
0 siblings, 1 reply; 21+ messages in thread
From: Sage Weil @ 2017-02-13 16:24 UTC (permalink / raw)
To: Tamilarasi Muthamizhan
Cc: Yuri Weinstein, Nathan Cutler, Ilya Dryomov, Development, Ceph,
Durgin, Josh, Just, Samuel, Dillaman, Jason, Gregory Farnum,
John Spray, Sadeh-Weinraub, Yehuda
[-- Attachment #1: Type: TEXT/PLAIN, Size: 1656 bytes --]
Let's ignore samba here too.
The rgw failures are all valgrind... can Matt or Yehuda take a look and
make sure they're not new/problematic?
Thanks!
sage
On Thu, 9 Feb 2017, Tamilarasi Muthamizhan wrote:
> Yuri, lets skip firefly-x upgrade issue.
>
> On Wed, Feb 8, 2017 at 1:14 PM, Yuri Weinstein <yweinste@redhat.com> wrote:
> upgrade/hammer-x (jewel) PASSED now!
>
> Outstanding suites requiring approval:
>
> rgw
> samba
> upgrade/firefly-x (hammer)
>
> Sage?
>
> Thx
> YuriW
>
> On Tue, Feb 7, 2017 at 1:47 PM, Yuri Weinstein
> <yweinste@redhat.com> wrote:
> > See updated status with approvals
> http://tracker.ceph.com/issues/17151#note-61
> >
> > Outstanding suites requiring approval:
> >
> > rgw
> > samba
> > upgrade/firefly-x (hammer)
> > upgrade/hammer-x (jewel)
> >
> > Thx
> > YuriW
> >
> > On Thu, Feb 2, 2017 at 3:17 AM, Nathan Cutler
> <ncutler@suse.cz> wrote:
> >>> Turned out to be a buggy test -- hammer is missing an
> infernalis
> >>> commit f217865cf7c4 ("test_librbd_fsx: invalidate before
> discard in
> >>> krbd mode").
> >>
> >>
> >> I gave it a try; please take a look:
> >>
> >> * https://github.com/ceph/ceph/pull/13230
> >>
> >> Nathan
> --
> To unsubscribe from this list: send the line "unsubscribe
> ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at
> http://vger.kernel.org/majordomo-info.html
>
>
>
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-13 16:24 ` Sage Weil
@ 2017-02-14 14:30 ` Orit Wasserman
2017-02-14 15:04 ` Nathan Cutler
0 siblings, 1 reply; 21+ messages in thread
From: Orit Wasserman @ 2017-02-14 14:30 UTC (permalink / raw)
To: Sage Weil
Cc: Tamilarasi Muthamizhan, Yuri Weinstein, Nathan Cutler,
Ilya Dryomov, Development, Ceph, Durgin, Josh, Just, Samuel,
Dillaman, Jason, Gregory Farnum, John Spray, Sadeh-Weinraub,
Yehuda
On Mon, Feb 13, 2017 at 6:24 PM, Sage Weil <sweil@redhat.com> wrote:
> Let's ignore samba here too.
>
> The rgw failures are all valgrind... can Matt or Yehuda take a look and
> make sure they're not new/problematic?
>
They are all resulting of linking with tcmalloc in notcmalloc builds
that valgrind doesn't like.
There is an open tracker issue: http://tracker.ceph.com/issues/18744
Orit
> Thanks!
> sage
>
> On Thu, 9 Feb 2017, Tamilarasi Muthamizhan wrote:
>
>> Yuri, lets skip firefly-x upgrade issue.
>>
>> On Wed, Feb 8, 2017 at 1:14 PM, Yuri Weinstein <yweinste@redhat.com> wrote:
>> upgrade/hammer-x (jewel) PASSED now!
>>
>> Outstanding suites requiring approval:
>>
>> rgw
>> samba
>> upgrade/firefly-x (hammer)
>>
>> Sage?
>>
>> Thx
>> YuriW
>>
>> On Tue, Feb 7, 2017 at 1:47 PM, Yuri Weinstein
>> <yweinste@redhat.com> wrote:
>> > See updated status with approvals
>> http://tracker.ceph.com/issues/17151#note-61
>> >
>> > Outstanding suites requiring approval:
>> >
>> > rgw
>> > samba
>> > upgrade/firefly-x (hammer)
>> > upgrade/hammer-x (jewel)
>> >
>> > Thx
>> > YuriW
>> >
>> > On Thu, Feb 2, 2017 at 3:17 AM, Nathan Cutler
>> <ncutler@suse.cz> wrote:
>> >>> Turned out to be a buggy test -- hammer is missing an
>> infernalis
>> >>> commit f217865cf7c4 ("test_librbd_fsx: invalidate before
>> discard in
>> >>> krbd mode").
>> >>
>> >>
>> >> I gave it a try; please take a look:
>> >>
>> >> * https://github.com/ceph/ceph/pull/13230
>> >>
>> >> Nathan
>> --
>> To unsubscribe from this list: send the line "unsubscribe
>> ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at
>> http://vger.kernel.org/majordomo-info.html
>>
>>
>>
>>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-14 14:30 ` Orit Wasserman
@ 2017-02-14 15:04 ` Nathan Cutler
2017-02-14 17:46 ` Yuri Weinstein
0 siblings, 1 reply; 21+ messages in thread
From: Nathan Cutler @ 2017-02-14 15:04 UTC (permalink / raw)
To: Orit Wasserman, Sage Weil
Cc: Tamilarasi Muthamizhan, Yuri Weinstein, Ilya Dryomov,
Development, Ceph, Durgin, Josh, Just, Samuel, Dillaman, Jason,
Gregory Farnum, John Spray, Sadeh-Weinraub, Yehuda
> They are all resulting of linking with tcmalloc in notcmalloc builds
> that valgrind doesn't like.
> There is an open tracker issue: http://tracker.ceph.com/issues/18744
Note that these failures would probably disappear on re-run, because Dan
fixed the underlying build issue.
Nathan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-02 11:17 ` Nathan Cutler
2017-02-07 21:47 ` Yuri Weinstein
@ 2017-02-14 17:05 ` Yuri Weinstein
1 sibling, 0 replies; 21+ messages in thread
From: Yuri Weinstein @ 2017-02-14 17:05 UTC (permalink / raw)
To: Nathan Cutler
Cc: Ilya Dryomov, Development, Ceph, Weil, Sage, Durgin, Josh, Just,
Samuel, Dillaman, Jason, Gregory Farnum, John Spray,
Sadeh-Weinraub, Yehuda
Let's retest !
On Thu, Feb 2, 2017 at 3:17 AM, Nathan Cutler <ncutler@suse.cz> wrote:
>> Turned out to be a buggy test -- hammer is missing an infernalis
>> commit f217865cf7c4 ("test_librbd_fsx: invalidate before discard in
>> krbd mode").
>
>
> I gave it a try; please take a look:
>
> * https://github.com/ceph/ceph/pull/13230
>
> Nathan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-14 15:04 ` Nathan Cutler
@ 2017-02-14 17:46 ` Yuri Weinstein
2017-02-17 5:06 ` Nathan Cutler
0 siblings, 1 reply; 21+ messages in thread
From: Yuri Weinstein @ 2017-02-14 17:46 UTC (permalink / raw)
To: Nathan Cutler
Cc: Orit Wasserman, Sage Weil, Tamilarasi Muthamizhan, Ilya Dryomov,
Development, Ceph, Durgin, Josh, Just, Samuel, Dillaman, Jason,
Gregory Farnum, John Spray, Sadeh-Weinraub, Yehuda
I am running tests and so far they are still showing the same rgw issues
http://pulpito.ceph.com/teuthology-2017-02-14_16:21:46-rgw-hammer-distro-basic-smithi/
On Tue, Feb 14, 2017 at 7:04 AM, Nathan Cutler <ncutler@suse.cz> wrote:
>> They are all resulting of linking with tcmalloc in notcmalloc builds
>> that valgrind doesn't like.
>> There is an open tracker issue: http://tracker.ceph.com/issues/18744
>
>
> Note that these failures would probably disappear on re-run, because Dan
> fixed the underlying build issue.
>
> Nathan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-14 17:46 ` Yuri Weinstein
@ 2017-02-17 5:06 ` Nathan Cutler
2017-02-17 14:14 ` Sage Weil
2017-02-17 18:55 ` Gregory Farnum
0 siblings, 2 replies; 21+ messages in thread
From: Nathan Cutler @ 2017-02-17 5:06 UTC (permalink / raw)
To: Yuri Weinstein
Cc: Orit Wasserman, Sage Weil, Tamilarasi Muthamizhan, Ilya Dryomov,
Development, Ceph, Durgin, Josh, Just, Samuel, Dillaman, Jason,
Gregory Farnum, John Spray, Sadeh-Weinraub, Yehuda
From re-reading this thread I determined that the only two outstanding
issues are upgrade/firefly-x and rgw+centos (can anyone confirm?)
I don't think upgrade/firefly-x should be a blocker. Rationale: as of
the jewel release, firefly clusters were supposed to be upgraded to at
least hammer. As of the kraken release, support of hammer has been
focused on facilitating users to upgrade from hammer to jewel. In other
words, firefly clusters should already have been upgraded and upgrading
from firefly to hammer is no longer supported.
As for the rgw valgrind/libtcmalloc failures on centos, I made a
wip-hammer-baseline branch based on "hammer" but with a different SHA1
("git commit --amend --reset-author") and pushed it to ceph/ceph-ci.git
so Shaman will re-build it. This should ensure that "flavor=notcmalloc"
really means what it says. Re-running rgw+centos on this
wip-hammer-baseline branch will tell us more.
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-17 5:06 ` Nathan Cutler
@ 2017-02-17 14:14 ` Sage Weil
2017-02-17 18:55 ` Gregory Farnum
1 sibling, 0 replies; 21+ messages in thread
From: Sage Weil @ 2017-02-17 14:14 UTC (permalink / raw)
To: Nathan Cutler
Cc: Yuri Weinstein, Orit Wasserman, Tamilarasi Muthamizhan,
Ilya Dryomov, Development, Ceph, Durgin, Josh, Just, Samuel,
Dillaman, Jason, Gregory Farnum, John Spray, Sadeh-Weinraub,
Yehuda
On Fri, 17 Feb 2017, Nathan Cutler wrote:
> From re-reading this thread I determined that the only two outstanding issues
> are upgrade/firefly-x and rgw+centos (can anyone confirm?)
>
> I don't think upgrade/firefly-x should be a blocker. Rationale: as of the
> jewel release, firefly clusters were supposed to be upgraded to at least
> hammer. As of the kraken release, support of hammer has been focused on
> facilitating users to upgrade from hammer to jewel. In other words, firefly
> clusters should already have been upgraded and upgrading from firefly to
> hammer is no longer supported.
>
> As for the rgw valgrind/libtcmalloc failures on centos, I made a
> wip-hammer-baseline branch based on "hammer" but with a different SHA1 ("git
> commit --amend --reset-author") and pushed it to ceph/ceph-ci.git so Shaman
> will re-build it. This should ensure that "flavor=notcmalloc" really means
> what it says. Re-running rgw+centos on this wip-hammer-baseline branch will
> tell us more.
I do'nt think these are a blocker either.. it's a problem with the
hammer 'notcmalloc' builds with are only for teuthology valgrind testing.
sage
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-17 5:06 ` Nathan Cutler
2017-02-17 14:14 ` Sage Weil
@ 2017-02-17 18:55 ` Gregory Farnum
2017-02-17 19:52 ` Nathan Cutler
1 sibling, 1 reply; 21+ messages in thread
From: Gregory Farnum @ 2017-02-17 18:55 UTC (permalink / raw)
To: Nathan Cutler
Cc: Yuri Weinstein, Orit Wasserman, Sage Weil,
Tamilarasi Muthamizhan, Ilya Dryomov, Durgin, Josh, Just, Samuel,
Dillaman, Jason, John Spray, Sadeh-Weinraub, Yehuda, ceph-devel
On Thu, Feb 16, 2017 at 9:06 PM, Nathan Cutler <ncutler@suse.cz> wrote:
> From re-reading this thread I determined that the only two outstanding
> issues are upgrade/firefly-x and rgw+centos (can anyone confirm?)
>
> I don't think upgrade/firefly-x should be a blocker. Rationale: as of the
> jewel release, firefly clusters were supposed to be upgraded to at least
> hammer. As of the kraken release, support of hammer has been focused on
> facilitating users to upgrade from hammer to jewel. In other words, firefly
> clusters should already have been upgraded and upgrading from firefly to
> hammer is no longer supported.
Woah, that can't be the case. Just because we're not supporting
firefly by putting out new releases doesn't mean it's acceptable to
prevent users from upgrading it to newer LTSes!
I've no idea what's broken there, so eg more valgrind failures are not
a big deal, but if the upgrade itself is failing in some way that's
going to be a problem. o_O
-Greg
>
> As for the rgw valgrind/libtcmalloc failures on centos, I made a
> wip-hammer-baseline branch based on "hammer" but with a different SHA1 ("git
> commit --amend --reset-author") and pushed it to ceph/ceph-ci.git so Shaman
> will re-build it. This should ensure that "flavor=notcmalloc" really means
> what it says. Re-running rgw+centos on this wip-hammer-baseline branch will
> tell us more.
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-17 18:55 ` Gregory Farnum
@ 2017-02-17 19:52 ` Nathan Cutler
2017-02-17 23:21 ` Gregory Farnum
0 siblings, 1 reply; 21+ messages in thread
From: Nathan Cutler @ 2017-02-17 19:52 UTC (permalink / raw)
To: Gregory Farnum
Cc: Yuri Weinstein, Orit Wasserman, Sage Weil,
Tamilarasi Muthamizhan, Ilya Dryomov, Durgin, Josh, Just, Samuel,
Dillaman, Jason, John Spray, Sadeh-Weinraub, Yehuda, ceph-devel
> I've no idea what's broken there, so eg more valgrind failures are not
> a big deal, but if the upgrade itself is failing in some way that's
> going to be a problem. o_O
Hi Greg,
Let me try to clarify :-)
If someone reports problems upgrading their firefly cluster to hammer,
and the problem is in firefly, will we publish a new firefly release?
If someone reports problems upgrading their dumpling cluster to firefly,
and the problem is in dumpling, will we publish a new dumpling release?
I don't know the answer to those questions, but such a release -- if
published -- would be "out-of-band" with regard to what is written in
https://github.com/ceph/ceph/blob/master/doc/releases.rst#understanding-the-release-cycle
- in particular this:
"Every other stable releases is a LTS (Long Term Stable) and will
receive updates until two LTS are published. For instance Dumpling is
retired when Hammer is published, Firefly is retired when Jewel is
published etc."
Nathan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-17 19:52 ` Nathan Cutler
@ 2017-02-17 23:21 ` Gregory Farnum
2017-02-18 8:19 ` Nathan Cutler
0 siblings, 1 reply; 21+ messages in thread
From: Gregory Farnum @ 2017-02-17 23:21 UTC (permalink / raw)
To: Nathan Cutler
Cc: Yuri Weinstein, Orit Wasserman, Sage Weil,
Tamilarasi Muthamizhan, Ilya Dryomov, Durgin, Josh, Just, Samuel,
Dillaman, Jason, John Spray, Sadeh-Weinraub, Yehuda, ceph-devel
On Fri, Feb 17, 2017 at 11:52 AM, Nathan Cutler <ncutler@suse.cz> wrote:
>> I've no idea what's broken there, so eg more valgrind failures are not
>> a big deal, but if the upgrade itself is failing in some way that's
>> going to be a problem. o_O
>
>
> Hi Greg,
>
> Let me try to clarify :-)
>
> If someone reports problems upgrading their firefly cluster to hammer, and
> the problem is in firefly, will we publish a new firefly release?
>
> If someone reports problems upgrading their dumpling cluster to firefly, and
> the problem is in dumpling, will we publish a new dumpling release?
Probably not. But, we do know that upgrades work now and so any new
failures must be a failure of the new version of Hammer. That, we
would hopefully avoid releasing, and would definitely issue a fix
release for. :) We can't strand users!
>
> I don't know the answer to those questions, but such a release -- if
> published -- would be "out-of-band" with regard to what is written in
> https://github.com/ceph/ceph/blob/master/doc/releases.rst#understanding-the-release-cycle
> - in particular this:
>
> "Every other stable releases is a LTS (Long Term Stable) and will receive
> updates until two LTS are published. For instance Dumpling is retired when
> Hammer is published, Firefly is retired when Jewel is published etc."
>
> Nathan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-17 23:21 ` Gregory Farnum
@ 2017-02-18 8:19 ` Nathan Cutler
2017-02-18 8:39 ` Nathan Cutler
0 siblings, 1 reply; 21+ messages in thread
From: Nathan Cutler @ 2017-02-18 8:19 UTC (permalink / raw)
To: Gregory Farnum
Cc: Yuri Weinstein, Orit Wasserman, Sage Weil,
Tamilarasi Muthamizhan, Ilya Dryomov, Durgin, Josh, Just, Samuel,
Dillaman, Jason, John Spray, Sadeh-Weinraub, Yehuda, ceph-devel
>> If someone reports problems upgrading their firefly cluster to hammer, and
>> the problem is in firefly, will we publish a new firefly release?
>>
>> If someone reports problems upgrading their dumpling cluster to firefly, and
>> the problem is in dumpling, will we publish a new dumpling release?
>
> Probably not. But, we do know that upgrades work now and so any new
> failures must be a failure of the new version of Hammer. That, we
> would hopefully avoid releasing, and would definitely issue a fix
> release for. :) We can't strand users!
Several people have looked at the upgrade/firefly-x (hammer) failures
and nobody has found anything that wasn't an issue with the tests
themselves. Maybe you could take a look, too?
http://pulpito.ceph.com/teuthology-2017-01-30_20:52:09-upgrade:firefly-x-hammer-distro-basic-vps/
Note that this QE run was done against the old gitbuilder build system.
Yuri has some special technique for achieving that ;-)
This is because the old builds that are needed to run the tests were/are
missing in the new build system. Injecting the old gitbuilder builds
into the new system is apparently possible, but a manual hit-or-miss
process. See http://tracker.ceph.com/issues/18089
(David Galloway has been _very_ helpful there, but the issues persist.)
This issue is affecting the jewel rados suite, even, since that suite
includes a couple tests that install various dumpling and firefly
releases and then upgrade them.
Thanks,
Nathan
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-18 8:19 ` Nathan Cutler
@ 2017-02-18 8:39 ` Nathan Cutler
0 siblings, 0 replies; 21+ messages in thread
From: Nathan Cutler @ 2017-02-18 8:39 UTC (permalink / raw)
To: Gregory Farnum
Cc: Yuri Weinstein, Orit Wasserman, Sage Weil,
Tamilarasi Muthamizhan, Ilya Dryomov, Durgin, Josh, Just, Samuel,
Dillaman, Jason, John Spray, Sadeh-Weinraub, Yehuda, ceph-devel
> Several people have looked at the upgrade/firefly-x (hammer) failures
> and nobody has found anything that wasn't an issue with the tests
> themselves. Maybe you could take a look, too?
>
> http://pulpito.ceph.com/teuthology-2017-01-30_20:52:09-upgrade:firefly-x-hammer-distro-basic-vps/
There are two failed and three dead jobs. The two failures are
ceph-objectstore-tool apply-layout-settings failed with 1
My totally untested theory: the client is not getting upgraded from
firefly to hammer, and as of
https://github.com/ceph/ceph/commit/e4038a0c ceph_manager.py has the
client running the "apply-layout-settings" subcommand of
ceph-objectstore-tool which is missing in firefly.
PR with fix for review: https://github.com/ceph/ceph/pull/13513 (against
hammer for now, but the same code is jewel, kraken, and master as well)
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-01 16:55 Yuri Weinstein
2017-02-01 20:36 ` Sage Weil
@ 2017-02-01 23:47 ` Vasu Kulkarni
1 sibling, 0 replies; 21+ messages in thread
From: Vasu Kulkarni @ 2017-02-01 23:47 UTC (permalink / raw)
To: Yuri Weinstein
Cc: Development, Ceph, Nathan Cutler, Weil, Sage, Durgin, Josh, Just,
Samuel, Dillaman, Jason, Ilya Dryomov, Gregory Farnum,
John Spray, Sadeh-Weinraub, Yehuda
Sorry resending as text
On Wed, Feb 1, 2017 at 8:55 AM, Yuri Weinstein <yweinste@redhat.com> wrote:
>
> Here is snapshot of hammer 0.94.10 point release testing status.
>
> Suites were not tested on ubuntu 16.06 due to
> http://tracker.ceph.com/issues/18382 and as a result were forced to
> test on ubuntu 14.04 and centos 7.2 (note we don't have smithi with
> centos 7.3).
>
> There was an additional noise created by fact that since we tested
> last hammer release in August 2016 we moved from gitbuilders to shaman
> build system and it resulted in issues with some packages.
>
> Details of tests can be found here http://tracker.ceph.com/issues/17151#note-61
>
> ======================
> rados ubuntu - packaging issues, need approval - Josh
> rados centos - packaging issues, need approval - Josh
> ======================
> rbd - PASSED
> ======================
> rgw ubuntu - PASSED
> rgw centos - valgrind issues http://tracker.ceph.com/issues/18735,
> need approval Yehuda, Sage
> ======================
> fs ubuntu - looks similar to v0.94.8, need Greg, John approval
> fs centos - noise? need Greg, John review/approval
> ======================
> krbd ubuntu - looks like the same in v0.94.8
> http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
> krbd centos - looks like the same in v0.94.8
> http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
> ======================
> kcephfs - PASSED
> ======================
> knfs - PASSED
> ======================
> hadoop - PASSED
> ======================
> samba - packaging issues, need approval - Greg, Sage
> ======================
> rest - PASSED
> ======================
> ceph-deploy - Vasu is fixing, need approval Sage
This one is difficult to test due to package split that happened after
hammer+, It is problem only for *new* installs of hammer.
Old version of ceph-deploy(1.5.27) works but it uses gitbuilder
instead of shaman, so we can test that only with some hacks. you
will have to pass on this for now, once its released we could check
the install with --stable flag instead of --dev flag.
>
> ======================
> upgrade/client-upgrade - PASSED
> ======================
> upgrade/firefly-x (hammer) - need Sage approval. maybe deprecate?
> ======================
> upgrade/hammer - PASSED
> ======================
> upgrade/hammer-x (jewel) - FAILED, need Sage approval. maybe deprecate?
> ======================
> powercycle - PASSED
> ======================
>
> Thx
> YuriW
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: hammer 0.94.10 QE status
2017-02-01 16:55 Yuri Weinstein
@ 2017-02-01 20:36 ` Sage Weil
2017-02-01 23:47 ` Vasu Kulkarni
1 sibling, 0 replies; 21+ messages in thread
From: Sage Weil @ 2017-02-01 20:36 UTC (permalink / raw)
To: Yuri Weinstein
Cc: Development, Ceph, Nathan Cutler, Durgin, Josh, Just, Samuel,
Dillaman, Jason, Ilya Dryomov, Gregory Farnum, John Spray,
Sadeh-Weinraub, Yehuda
On Wed, 1 Feb 2017, Yuri Weinstein wrote:
> Here is snapshot of hammer 0.94.10 point release testing status.
>
> Suites were not tested on ubuntu 16.06 due to
> http://tracker.ceph.com/issues/18382 and as a result were forced to
> test on ubuntu 14.04 and centos 7.2 (note we don't have smithi with
> centos 7.3).
>
> There was an additional noise created by fact that since we tested
> last hammer release in August 2016 we moved from gitbuilders to shaman
> build system and it resulted in issues with some packages.
>
> Details of tests can be found here http://tracker.ceph.com/issues/17151#note-61
>
> ======================
> rados ubuntu - packaging issues, need approval - Josh
> rados centos - packaging issues, need approval - Josh
> ======================
> rbd - PASSED
> ======================
> rgw ubuntu - PASSED
> rgw centos - valgrind issues http://tracker.ceph.com/issues/18735,
> need approval Yehuda, Sage
> ======================
> fs ubuntu - looks similar to v0.94.8, need Greg, John approval
> fs centos - noise? need Greg, John review/approval
> ======================
> krbd ubuntu - looks like the same in v0.94.8
> http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
> krbd centos - looks like the same in v0.94.8
> http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
> ======================
> kcephfs - PASSED
> ======================
> knfs - PASSED
> ======================
> hadoop - PASSED
> ======================
> samba - packaging issues, need approval - Greg, Sage
ok to skip
> ======================
> rest - PASSED
> ======================
> ceph-deploy - Vasu is fixing, need approval Sage
what's the story here?
> ======================
> upgrade/client-upgrade - PASSED
> ======================
> upgrade/firefly-x (hammer) - need Sage approval. maybe deprecate?
somethign is wrong with teh teuthology test, it's looping dumping ops in
flight. i don't have time to look at it, anybody else want to take a
look?
> ======================
> upgrade/hammer - PASSED
> ======================
> upgrade/hammer-x (jewel) - FAILED, need Sage approval. maybe deprecate?
Looks like this test suite is broken, fixing here:
https://github.com/ceph/ceph/pull/13222
> ======================
> powercycle - PASSED
> ======================
>
> Thx
> YuriW
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
^ permalink raw reply [flat|nested] 21+ messages in thread
* hammer 0.94.10 QE status
@ 2017-02-01 16:55 Yuri Weinstein
2017-02-01 20:36 ` Sage Weil
2017-02-01 23:47 ` Vasu Kulkarni
0 siblings, 2 replies; 21+ messages in thread
From: Yuri Weinstein @ 2017-02-01 16:55 UTC (permalink / raw)
To: Development, Ceph, Nathan Cutler, Weil, Sage, Durgin, Josh, Just,
Samuel, Dillaman, Jason, Ilya Dryomov, Gregory Farnum,
John Spray, Sadeh-Weinraub, Yehuda
Here is snapshot of hammer 0.94.10 point release testing status.
Suites were not tested on ubuntu 16.06 due to
http://tracker.ceph.com/issues/18382 and as a result were forced to
test on ubuntu 14.04 and centos 7.2 (note we don't have smithi with
centos 7.3).
There was an additional noise created by fact that since we tested
last hammer release in August 2016 we moved from gitbuilders to shaman
build system and it resulted in issues with some packages.
Details of tests can be found here http://tracker.ceph.com/issues/17151#note-61
======================
rados ubuntu - packaging issues, need approval - Josh
rados centos - packaging issues, need approval - Josh
======================
rbd - PASSED
======================
rgw ubuntu - PASSED
rgw centos - valgrind issues http://tracker.ceph.com/issues/18735,
need approval Yehuda, Sage
======================
fs ubuntu - looks similar to v0.94.8, need Greg, John approval
fs centos - noise? need Greg, John review/approval
======================
krbd ubuntu - looks like the same in v0.94.8
http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
krbd centos - looks like the same in v0.94.8
http://tracker.ceph.com/issues/15895#note-33 need approval Jason, Ilya
======================
kcephfs - PASSED
======================
knfs - PASSED
======================
hadoop - PASSED
======================
samba - packaging issues, need approval - Greg, Sage
======================
rest - PASSED
======================
ceph-deploy - Vasu is fixing, need approval Sage
======================
upgrade/client-upgrade - PASSED
======================
upgrade/firefly-x (hammer) - need Sage approval. maybe deprecate?
======================
upgrade/hammer - PASSED
======================
upgrade/hammer-x (jewel) - FAILED, need Sage approval. maybe deprecate?
======================
powercycle - PASSED
======================
Thx
YuriW
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2017-02-18 8:39 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CAMMFjmHCAM6wmygytBDqwh89UOEex24_GGqrLJvXcOUS+QEg9A@mail.gmail.com>
2017-02-01 17:03 ` hammer 0.94.10 QE status Ilya Dryomov
2017-02-02 10:01 ` Ilya Dryomov
2017-02-02 11:17 ` Nathan Cutler
2017-02-07 21:47 ` Yuri Weinstein
2017-02-08 21:14 ` Yuri Weinstein
[not found] ` <CAObvjwPb_PosEnGOXP2tV4aWjnTyed7-275svyK+vu4uxYdTzQ@mail.gmail.com>
2017-02-13 16:24 ` Sage Weil
2017-02-14 14:30 ` Orit Wasserman
2017-02-14 15:04 ` Nathan Cutler
2017-02-14 17:46 ` Yuri Weinstein
2017-02-17 5:06 ` Nathan Cutler
2017-02-17 14:14 ` Sage Weil
2017-02-17 18:55 ` Gregory Farnum
2017-02-17 19:52 ` Nathan Cutler
2017-02-17 23:21 ` Gregory Farnum
2017-02-18 8:19 ` Nathan Cutler
2017-02-18 8:39 ` Nathan Cutler
2017-02-14 17:05 ` Yuri Weinstein
2017-02-01 19:12 ` John Spray
2017-02-01 16:55 Yuri Weinstein
2017-02-01 20:36 ` Sage Weil
2017-02-01 23:47 ` Vasu Kulkarni
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.