workloads/rbd_fsx_cache_writethrough.yaml hangs on dumpling

workloads/rbd_fsx_cache_writethrough.yaml hangs on dumpling

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1109 bytes --]

Hi,

http://pulpito.ceph.com/loic-2015-01-29_15:39:38-rbd-dumpling-backports---basic-multi/730029/ hangs on dumpling and got killed after two days.

Although workloads/rbd_fsx_cache_writethrough.yaml  was running with the thrasher, it does not seem to be related to http://tracker.ceph.com/issues/10513 : it was running on a mira070 (16GB RAM) and plana78 (8GB RAM). 

I don't see anything in https://github.com/ceph/ceph-qa-suite/blob/master/tasks/rbd_fsx.py that would suggest it needs backporting a fix to the ceph-qa-suite branch of dumpling.

I browsed the commits related to RBD that are in the dumpling-backports branch ( see here for the list I used http://tracker.ceph.com/issues/10560 ). I don't see anything in the issues that need backporting to dumpling either. 

Another run of this specific test has been scheduled to check if it always happens at http://pulpito.ceph.com/loic-2015-02-02_13:58:32-rbd:thrash:workloads:rbd_fsx_cache_writethrough.yaml-dumpling-backports---basic-multi/

Does that ring a bell by any chance ?

Cheers

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: workloads/rbd_fsx_cache_writethrough.yaml hangs on dumpling

[Yan, Zheng]

On Mon, Feb 2, 2015 at 9:18 PM, Loic Dachary <loic@dachary.org> wrote:
> Hi,
>
> http://pulpito.ceph.com/loic-2015-01-29_15:39:38-rbd-dumpling-backports---basic-multi/730029/ hangs on dumpling and got killed after two days.
>
> Although workloads/rbd_fsx_cache_writethrough.yaml  was running with the thrasher, it does not seem to be related to http://tracker.ceph.com/issues/10513 : it was running on a mira070 (16GB RAM) and plana78 (8GB RAM).
>
> I don't see anything in https://github.com/ceph/ceph-qa-suite/blob/master/tasks/rbd_fsx.py that would suggest it needs backporting a fix to the ceph-qa-suite branch of dumpling.
>
> I browsed the commits related to RBD that are in the dumpling-backports branch ( see here for the list I used http://tracker.ceph.com/issues/10560 ). I don't see anything in the issues that need backporting to dumpling either.
>
> Another run of this specific test has been scheduled to check if it always happens at http://pulpito.ceph.com/loic-2015-02-02_13:58:32-rbd:thrash:workloads:rbd_fsx_cache_writethrough.yaml-dumpling-backports---basic-multi/
>
> Does that ring a bell by any chance ?
>

maybe it's the same as http://tracker.ceph.com/issues/10498

>
> --
> Loïc Dachary, Artisan Logiciel Libre
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: workloads/rbd_fsx_cache_writethrough.yaml hangs on dumpling

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1755 bytes --]



On 02/02/2015 14:48, Yan, Zheng wrote:
> On Mon, Feb 2, 2015 at 9:18 PM, Loic Dachary <loic@dachary.org> wrote:
>> Hi,
>>
>> http://pulpito.ceph.com/loic-2015-01-29_15:39:38-rbd-dumpling-backports---basic-multi/730029/ hangs on dumpling and got killed after two days.
>>
>> Although workloads/rbd_fsx_cache_writethrough.yaml  was running with the thrasher, it does not seem to be related to http://tracker.ceph.com/issues/10513 : it was running on a mira070 (16GB RAM) and plana78 (8GB RAM).
>>
>> I don't see anything in https://github.com/ceph/ceph-qa-suite/blob/master/tasks/rbd_fsx.py that would suggest it needs backporting a fix to the ceph-qa-suite branch of dumpling.
>>
>> I browsed the commits related to RBD that are in the dumpling-backports branch ( see here for the list I used http://tracker.ceph.com/issues/10560 ). I don't see anything in the issues that need backporting to dumpling either.
>>
>> Another run of this specific test has been scheduled to check if it always happens at http://pulpito.ceph.com/loic-2015-02-02_13:58:32-rbd:thrash:workloads:rbd_fsx_cache_writethrough.yaml-dumpling-backports---basic-multi/
>>
>> Does that ring a bell by any chance ?
>>
> 
> maybe it's the same as http://tracker.ceph.com/issues/10498
> 

I hoped it would be but from what I read the bug crashes and creates a core dump, right ? What I'm seeing is the test hanging indefinitely and no core.

Cheers

>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
>>
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: workloads/rbd_fsx_cache_writethrough.yaml hangs on dumpling

[Gregory Farnum]

On Mon, Feb 2, 2015 at 7:00 AM, Loic Dachary <loic@dachary.org> wrote:
>
>
> On 02/02/2015 14:48, Yan, Zheng wrote:
>> On Mon, Feb 2, 2015 at 9:18 PM, Loic Dachary <loic@dachary.org> wrote:
>>> Hi,
>>>
>>> http://pulpito.ceph.com/loic-2015-01-29_15:39:38-rbd-dumpling-backports---basic-multi/730029/ hangs on dumpling and got killed after two days.
>>>
>>> Although workloads/rbd_fsx_cache_writethrough.yaml  was running with the thrasher, it does not seem to be related to http://tracker.ceph.com/issues/10513 : it was running on a mira070 (16GB RAM) and plana78 (8GB RAM).
>>>
>>> I don't see anything in https://github.com/ceph/ceph-qa-suite/blob/master/tasks/rbd_fsx.py that would suggest it needs backporting a fix to the ceph-qa-suite branch of dumpling.
>>>
>>> I browsed the commits related to RBD that are in the dumpling-backports branch ( see here for the list I used http://tracker.ceph.com/issues/10560 ). I don't see anything in the issues that need backporting to dumpling either.
>>>
>>> Another run of this specific test has been scheduled to check if it always happens at http://pulpito.ceph.com/loic-2015-02-02_13:58:32-rbd:thrash:workloads:rbd_fsx_cache_writethrough.yaml-dumpling-backports---basic-multi/
>>>
>>> Does that ring a bell by any chance ?
>>>
>>
>> maybe it's the same as http://tracker.ceph.com/issues/10498
>>
>
> I hoped it would be but from what I read the bug crashes and creates a core dump, right ? What I'm seeing is the test hanging indefinitely and no core.

Whether it's crashing or not depends on the nature of the test. The
root cause is just a hang; some things might crash on timeouts but
others won't. (I don't think I recall seeing a crash on that bug
report anyway, though.)
-Greg

Re: workloads/rbd_fsx_cache_writethrough.yaml hangs on dumpling

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 2307 bytes --]



On 02/02/2015 17:46, Gregory Farnum wrote:
> On Mon, Feb 2, 2015 at 7:00 AM, Loic Dachary <loic@dachary.org> wrote:
>>
>>
>> On 02/02/2015 14:48, Yan, Zheng wrote:
>>> On Mon, Feb 2, 2015 at 9:18 PM, Loic Dachary <loic@dachary.org> wrote:
>>>> Hi,
>>>>
>>>> http://pulpito.ceph.com/loic-2015-01-29_15:39:38-rbd-dumpling-backports---basic-multi/730029/ hangs on dumpling and got killed after two days.
>>>>
>>>> Although workloads/rbd_fsx_cache_writethrough.yaml  was running with the thrasher, it does not seem to be related to http://tracker.ceph.com/issues/10513 : it was running on a mira070 (16GB RAM) and plana78 (8GB RAM).
>>>>
>>>> I don't see anything in https://github.com/ceph/ceph-qa-suite/blob/master/tasks/rbd_fsx.py that would suggest it needs backporting a fix to the ceph-qa-suite branch of dumpling.
>>>>
>>>> I browsed the commits related to RBD that are in the dumpling-backports branch ( see here for the list I used http://tracker.ceph.com/issues/10560 ). I don't see anything in the issues that need backporting to dumpling either.
>>>>
>>>> Another run of this specific test has been scheduled to check if it always happens at http://pulpito.ceph.com/loic-2015-02-02_13:58:32-rbd:thrash:workloads:rbd_fsx_cache_writethrough.yaml-dumpling-backports---basic-multi/
>>>>
>>>> Does that ring a bell by any chance ?
>>>>
>>>
>>> maybe it's the same as http://tracker.ceph.com/issues/10498
>>>
>>
>> I hoped it would be but from what I read the bug crashes and creates a core dump, right ? What I'm seeing is the test hanging indefinitely and no core.
> 
> Whether it's crashing or not depends on the nature of the test. The
> root cause is just a hang; some things might crash on timeouts but
> others won't. (I don't think I recall seeing a crash on that bug
> report anyway, though.)

Ah, you are correct, thanks for correcting me. I got confused by another livelock bug http://tracker.ceph.com/issues/9854 and wrongly assumed just by the keyword that they were the same.

Cheers

> -Greg
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]