* teuthology timeout error
@ 2015-05-20 2:20 Miyamae, Takeshi
2015-05-20 7:48 ` Loic Dachary
0 siblings, 1 reply; 7+ messages in thread
From: Miyamae, Takeshi @ 2015-05-20 2:20 UTC (permalink / raw)
To: Loic Dachary, Ceph Development
Cc: Kawaguchi, Shotaro, Imai, Hiroki, Nakao, Takanori, Shiozawa, Kensuke
Hi Loic,
When we fixed our own issue and restarted teuthology, we encountered another issue
(timeout error) which occurs in case of LRC as well.
Do you have any information about that ?
[error messages (in case of LRC pool)]
2015-04-28 12:38:54,128.128 INFO:teuthology.orchestra.run.RX35-1:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph status --format=json-pretty'
2015-04-28 12:38:54,516.516 INFO:tasks.ceph.ceph_manager:no progress seen, keeping timeout for now
2015-04-28 12:38:54,516.516 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
return func(self)
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
timeout=self.config.get('timeout')
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
'failed to recover before timeout expired'
AssertionError: failed to recover before timeout expired
Traceback (most recent call last):
File "/root/work/teuthology/virtualenv/lib/python2.7/site-packages/gevent/greenlet.py", line 390, in run
result = self._run(*self.args, **self.kwargs)
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
return func(self)
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
timeout=self.config.get('timeout')
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
'failed to recover before timeout expired'
AssertionError: failed to recover before timeout expired <Greenlet at 0x2a7d550: <bound method Thrasher.do_thrash of <tasks.ceph_manager.Thrasher instance at 0x2bd12d8>>> failed with AssertionError
[ceph version]
0.93-952-gfe28daa
[teuthology, ceph-qa-suite]
newest version at 3/25/2015
[configurations]
check-locks: false
overrides:
ceph:
conf:
global:
ms inject socket failures: 5000
osd:
osd heartbeat use min delay socket: true
osd sloppy crc: true
fs: xfs
roles:
- - mon.a
- osd.0
- osd.4
- osd.8
- osd.12
- - mon.b
- osd.1
- osd.5
- osd.9
- osd.13
- - mon.c
- osd.2
- osd.6
- osd.10
- osd.14
- - osd.3
- osd.7
- osd.11
- osd.15
- client.0
targets:
ubuntu@RX35-1.primary.ceph-poc.fsc.net:
ubuntu@RX35-2.primary.ceph-poc.fsc.net:
ubuntu@RX35-3.primary.ceph-poc.fsc.net:
ubuntu@RX35-4.primary.ceph-poc.fsc.net:
tasks:
- ceph:
conf:
osd:
osd debug reject backfill probability: 0.3
osd max backfills: 1
osd scrub max interval: 120
osd scrub min interval: 60
log-whitelist:
- wrongly marked me down
- objects unfound and apparently lost
- thrashosds:
chance_pgnum_grow: 1
chance_pgpnum_fix: 1
min_in: 4
timeout: 1200
- rados:
clients:
- client.0
ec_pool: true
erasure_code_profile:
k: 4
l: 3
m: 2
name: lrcprofile
plugin: lrc
ruleset-failure-domain: osd
objects: 50
op_weights:
append: 100
copy_from: 50
delete: 50
read: 100
rmattr: 25
rollback: 50
setattr: 25
snap_create: 50
snap_remove: 50
write: 0
ops: 190000
Best regards,
Takeshi Miyamae
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: teuthology timeout error
2015-05-20 2:20 teuthology timeout error Miyamae, Takeshi
@ 2015-05-20 7:48 ` Loic Dachary
2015-05-21 8:32 ` Miyamae, Takeshi
0 siblings, 1 reply; 7+ messages in thread
From: Loic Dachary @ 2015-05-20 7:48 UTC (permalink / raw)
To: Miyamae, Takeshi, Ceph Development
Cc: Kawaguchi, Shotaro, Imai, Hiroki, Nakao, Takanori, Shiozawa, Kensuke
[-- Attachment #1: Type: text/plain, Size: 4103 bytes --]
Hi,
On 20/05/2015 04:20, Miyamae, Takeshi wrote:
> Hi Loic,
>
> When we fixed our own issue and restarted teuthology,
Great !
> we encountered another issue (timeout error) which occurs in case of LRC as well.
> Do you have any information about that ?
Could you please share the teuthology/ceph-qa-suite repository you are using to run these tests so I can try to reproduce / diagnose the problem ?
Thanks
>
> [error messages (in case of LRC pool)]
>
> 2015-04-28 12:38:54,128.128 INFO:teuthology.orchestra.run.RX35-1:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph status --format=json-pretty'
> 2015-04-28 12:38:54,516.516 INFO:tasks.ceph.ceph_manager:no progress seen, keeping timeout for now
> 2015-04-28 12:38:54,516.516 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
> return func(self)
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
> timeout=self.config.get('timeout')
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
> 'failed to recover before timeout expired'
> AssertionError: failed to recover before timeout expired
>
> Traceback (most recent call last):
> File "/root/work/teuthology/virtualenv/lib/python2.7/site-packages/gevent/greenlet.py", line 390, in run
> result = self._run(*self.args, **self.kwargs)
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
> return func(self)
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
> timeout=self.config.get('timeout')
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
> 'failed to recover before timeout expired'
> AssertionError: failed to recover before timeout expired <Greenlet at 0x2a7d550: <bound method Thrasher.do_thrash of <tasks.ceph_manager.Thrasher instance at 0x2bd12d8>>> failed with AssertionError
>
> [ceph version]
> 0.93-952-gfe28daa
>
> [teuthology, ceph-qa-suite]
> newest version at 3/25/2015
>
> [configurations]
> check-locks: false
> overrides:
> ceph:
> conf:
> global:
> ms inject socket failures: 5000
> osd:
> osd heartbeat use min delay socket: true
> osd sloppy crc: true
> fs: xfs
> roles:
> - - mon.a
> - osd.0
> - osd.4
> - osd.8
> - osd.12
> - - mon.b
> - osd.1
> - osd.5
> - osd.9
> - osd.13
> - - mon.c
> - osd.2
> - osd.6
> - osd.10
> - osd.14
> - - osd.3
> - osd.7
> - osd.11
> - osd.15
> - client.0
> targets:
> ubuntu@RX35-1.primary.ceph-poc.fsc.net:
> ubuntu@RX35-2.primary.ceph-poc.fsc.net:
> ubuntu@RX35-3.primary.ceph-poc.fsc.net:
> ubuntu@RX35-4.primary.ceph-poc.fsc.net:
> tasks:
> - ceph:
> conf:
> osd:
> osd debug reject backfill probability: 0.3
> osd max backfills: 1
> osd scrub max interval: 120
> osd scrub min interval: 60
> log-whitelist:
> - wrongly marked me down
> - objects unfound and apparently lost
> - thrashosds:
> chance_pgnum_grow: 1
> chance_pgpnum_fix: 1
> min_in: 4
> timeout: 1200
> - rados:
> clients:
> - client.0
> ec_pool: true
> erasure_code_profile:
> k: 4
> l: 3
> m: 2
> name: lrcprofile
> plugin: lrc
> ruleset-failure-domain: osd
> objects: 50
> op_weights:
> append: 100
> copy_from: 50
> delete: 50
> read: 100
> rmattr: 25
> rollback: 50
> setattr: 25
> snap_create: 50
> snap_remove: 50
> write: 0
> ops: 190000
>
> Best regards,
> Takeshi Miyamae
>
--
Loïc Dachary, Artisan Logiciel Libre
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* RE: teuthology timeout error
2015-05-20 7:48 ` Loic Dachary
@ 2015-05-21 8:32 ` Miyamae, Takeshi
2015-05-21 9:30 ` Loic Dachary
2015-05-21 9:37 ` Loic Dachary
0 siblings, 2 replies; 7+ messages in thread
From: Miyamae, Takeshi @ 2015-05-21 8:32 UTC (permalink / raw)
To: Loic Dachary, Ceph Development
Cc: Kawaguchi, Shotaro, Imai, Hiroki, Nakao, Takanori, Shiozawa, Kensuke
Hi Loic,
> Could you please share the teuthology/ceph-qa-suite repository you are using to run these tests
> so I can try to reproduce / diagnose the problem ?
https://github.com/kawaguchi-s/teuthology/tree/wip-10886
https://github.com/kawaguchi-s/ceph-qa-suite/tree/wip-10886
Here are our teuthology/ceph-qa-suite repositories. Thanks in advance.
Best regards,
Takeshi Miyamae
-----Original Message-----
From: Loic Dachary [mailto:loic@dachary.org]
Sent: Wednesday, May 20, 2015 4:49 PM
To: Miyamae, Takeshi/宮前 剛; Ceph Development
Cc: Kawaguchi, Shotaro/川口 翔太朗; Imai, Hiroki/今井 宏樹; Nakao, Takanori/中尾 鷹詔; Shiozawa, Kensuke/塩沢 賢輔
Subject: Re: teuthology timeout error
Hi,
On 20/05/2015 04:20, Miyamae, Takeshi wrote:
> Hi Loic,
>
> When we fixed our own issue and restarted teuthology,
Great !
> we encountered another issue (timeout error) which occurs in case of LRC as well.
> Do you have any information about that ?
Could you please share the teuthology/ceph-qa-suite repository you are using to run these tests so I can try to reproduce / diagnose the problem ?
Thanks
>
> [error messages (in case of LRC pool)]
>
> 2015-04-28 12:38:54,128.128 INFO:teuthology.orchestra.run.RX35-1:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph status --format=json-pretty'
> 2015-04-28 12:38:54,516.516 INFO:tasks.ceph.ceph_manager:no progress seen, keeping timeout for now
> 2015-04-28 12:38:54,516.516 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
> return func(self)
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
> timeout=self.config.get('timeout')
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
> 'failed to recover before timeout expired'
> AssertionError: failed to recover before timeout expired
>
> Traceback (most recent call last):
> File "/root/work/teuthology/virtualenv/lib/python2.7/site-packages/gevent/greenlet.py", line 390, in run
> result = self._run(*self.args, **self.kwargs)
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
> return func(self)
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
> timeout=self.config.get('timeout')
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
> 'failed to recover before timeout expired'
> AssertionError: failed to recover before timeout expired <Greenlet at 0x2a7d550: <bound method Thrasher.do_thrash of <tasks.ceph_manager.Thrasher instance at 0x2bd12d8>>> failed with AssertionError
>
> [ceph version]
> 0.93-952-gfe28daa
>
> [teuthology, ceph-qa-suite]
> newest version at 3/25/2015
>
> [configurations]
> check-locks: false
> overrides:
> ceph:
> conf:
> global:
> ms inject socket failures: 5000
> osd:
> osd heartbeat use min delay socket: true
> osd sloppy crc: true
> fs: xfs
> roles:
> - - mon.a
> - osd.0
> - osd.4
> - osd.8
> - osd.12
> - - mon.b
> - osd.1
> - osd.5
> - osd.9
> - osd.13
> - - mon.c
> - osd.2
> - osd.6
> - osd.10
> - osd.14
> - - osd.3
> - osd.7
> - osd.11
> - osd.15
> - client.0
> targets:
> ubuntu@RX35-1.primary.ceph-poc.fsc.net:
> ubuntu@RX35-2.primary.ceph-poc.fsc.net:
> ubuntu@RX35-3.primary.ceph-poc.fsc.net:
> ubuntu@RX35-4.primary.ceph-poc.fsc.net:
> tasks:
> - ceph:
> conf:
> osd:
> osd debug reject backfill probability: 0.3
> osd max backfills: 1
> osd scrub max interval: 120
> osd scrub min interval: 60
> log-whitelist:
> - wrongly marked me down
> - objects unfound and apparently lost
> - thrashosds:
> chance_pgnum_grow: 1
> chance_pgpnum_fix: 1
> min_in: 4
> timeout: 1200
> - rados:
> clients:
> - client.0
> ec_pool: true
> erasure_code_profile:
> k: 4
> l: 3
> m: 2
> name: lrcprofile
> plugin: lrc
> ruleset-failure-domain: osd
> objects: 50
> op_weights:
> append: 100
> copy_from: 50
> delete: 50
> read: 100
> rmattr: 25
> rollback: 50
> setattr: 25
> snap_create: 50
> snap_remove: 50
> write: 0
> ops: 190000
>
> Best regards,
> Takeshi Miyamae
>
--
Loïc Dachary, Artisan Logiciel Libre
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: teuthology timeout error
2015-05-21 8:32 ` Miyamae, Takeshi
@ 2015-05-21 9:30 ` Loic Dachary
2015-05-21 9:37 ` Loic Dachary
1 sibling, 0 replies; 7+ messages in thread
From: Loic Dachary @ 2015-05-21 9:30 UTC (permalink / raw)
To: Miyamae, Takeshi, Ceph Development
Cc: Kawaguchi, Shotaro, Imai, Hiroki, Nakao, Takanori, Shiozawa, Kensuke
[-- Attachment #1: Type: text/plain, Size: 5381 bytes --]
On 21/05/2015 10:32, Miyamae, Takeshi wrote:
> Hi Loic,
>
>> Could you please share the teuthology/ceph-qa-suite repository you are using to run these tests
>> so I can try to reproduce / diagnose the problem ?
>
> https://github.com/kawaguchi-s/teuthology/tree/wip-10886
> https://github.com/kawaguchi-s/ceph-qa-suite/tree/wip-10886
>
When compared against master they show differences that indicate it would be good to rebase:
https://github.com/ceph/teuthology/compare/master...kawaguchi-s:wip-10886
https://github.com/ceph/ceph-qa-suite/compare/master...kawaguchi-s:wip-10886
I think the teuthology commit on top of wip-10886 is a mistake
> Here are our teuthology/ceph-qa-suite repositories. Thanks in advance.
>
> Best regards,
> Takeshi Miyamae
>
> -----Original Message-----
> From: Loic Dachary [mailto:loic@dachary.org]
> Sent: Wednesday, May 20, 2015 4:49 PM
> To: Miyamae, Takeshi/宮前 剛; Ceph Development
> Cc: Kawaguchi, Shotaro/川口 翔太朗; Imai, Hiroki/今井 宏樹; Nakao, Takanori/中尾 鷹詔; Shiozawa, Kensuke/塩沢 賢輔
> Subject: Re: teuthology timeout error
>
> Hi,
>
> On 20/05/2015 04:20, Miyamae, Takeshi wrote:
>> Hi Loic,
>>
>> When we fixed our own issue and restarted teuthology,
>
> Great !
>
>> we encountered another issue (timeout error) which occurs in case of LRC as well.
>> Do you have any information about that ?
>
> Could you please share the teuthology/ceph-qa-suite repository you are using to run these tests so I can try to reproduce / diagnose the problem ?
>
> Thanks
>
>>
>> [error messages (in case of LRC pool)]
>>
>> 2015-04-28 12:38:54,128.128 INFO:teuthology.orchestra.run.RX35-1:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph status --format=json-pretty'
>> 2015-04-28 12:38:54,516.516 INFO:tasks.ceph.ceph_manager:no progress seen, keeping timeout for now
>> 2015-04-28 12:38:54,516.516 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
>> return func(self)
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
>> timeout=self.config.get('timeout')
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
>> 'failed to recover before timeout expired'
>> AssertionError: failed to recover before timeout expired
>>
>> Traceback (most recent call last):
>> File "/root/work/teuthology/virtualenv/lib/python2.7/site-packages/gevent/greenlet.py", line 390, in run
>> result = self._run(*self.args, **self.kwargs)
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
>> return func(self)
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
>> timeout=self.config.get('timeout')
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
>> 'failed to recover before timeout expired'
>> AssertionError: failed to recover before timeout expired <Greenlet at 0x2a7d550: <bound method Thrasher.do_thrash of <tasks.ceph_manager.Thrasher instance at 0x2bd12d8>>> failed with AssertionError
>>
>> [ceph version]
>> 0.93-952-gfe28daa
>>
>> [teuthology, ceph-qa-suite]
>> newest version at 3/25/2015
>>
>> [configurations]
>> check-locks: false
>> overrides:
>> ceph:
>> conf:
>> global:
>> ms inject socket failures: 5000
>> osd:
>> osd heartbeat use min delay socket: true
>> osd sloppy crc: true
>> fs: xfs
>> roles:
>> - - mon.a
>> - osd.0
>> - osd.4
>> - osd.8
>> - osd.12
>> - - mon.b
>> - osd.1
>> - osd.5
>> - osd.9
>> - osd.13
>> - - mon.c
>> - osd.2
>> - osd.6
>> - osd.10
>> - osd.14
>> - - osd.3
>> - osd.7
>> - osd.11
>> - osd.15
>> - client.0
>> targets:
>> ubuntu@RX35-1.primary.ceph-poc.fsc.net:
>> ubuntu@RX35-2.primary.ceph-poc.fsc.net:
>> ubuntu@RX35-3.primary.ceph-poc.fsc.net:
>> ubuntu@RX35-4.primary.ceph-poc.fsc.net:
>> tasks:
>> - ceph:
>> conf:
>> osd:
>> osd debug reject backfill probability: 0.3
>> osd max backfills: 1
>> osd scrub max interval: 120
>> osd scrub min interval: 60
>> log-whitelist:
>> - wrongly marked me down
>> - objects unfound and apparently lost
>> - thrashosds:
>> chance_pgnum_grow: 1
>> chance_pgpnum_fix: 1
>> min_in: 4
>> timeout: 1200
>> - rados:
>> clients:
>> - client.0
>> ec_pool: true
>> erasure_code_profile:
>> k: 4
>> l: 3
>> m: 2
>> name: lrcprofile
>> plugin: lrc
>> ruleset-failure-domain: osd
>> objects: 50
>> op_weights:
>> append: 100
>> copy_from: 50
>> delete: 50
>> read: 100
>> rmattr: 25
>> rollback: 50
>> setattr: 25
>> snap_create: 50
>> snap_remove: 50
>> write: 0
>> ops: 190000
>>
>> Best regards,
>> Takeshi Miyamae
>>
>
--
Loïc Dachary, Artisan Logiciel Libre
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: teuthology timeout error
2015-05-21 8:32 ` Miyamae, Takeshi
2015-05-21 9:30 ` Loic Dachary
@ 2015-05-21 9:37 ` Loic Dachary
2015-05-26 2:39 ` Miyamae, Takeshi
1 sibling, 1 reply; 7+ messages in thread
From: Loic Dachary @ 2015-05-21 9:37 UTC (permalink / raw)
To: Miyamae, Takeshi, Ceph Development
Cc: Kawaguchi, Shotaro, Imai, Hiroki, Nakao, Takanori, Shiozawa, Kensuke
[-- Attachment #1: Type: text/plain, Size: 6396 bytes --]
Hi,
[sorry the previous mail was sent by accident, here is the full mail]
On 21/05/2015 10:32, Miyamae, Takeshi wrote:
> Hi Loic,
>
>> Could you please share the teuthology/ceph-qa-suite repository you are using to run these tests
>> so I can try to reproduce / diagnose the problem ?
>
> https://github.com/kawaguchi-s/teuthology/tree/wip-10886
> https://github.com/kawaguchi-s/ceph-qa-suite/tree/wip-10886
>
When compared against master they show differences that indicate it would be good to rebase:
https://github.com/ceph/teuthology/compare/master...kawaguchi-s:wip-10886
https://github.com/ceph/ceph-qa-suite/compare/master...kawaguchi-s:wip-10886
I think the teuthology commit on top of wip-10886 is a mistake
https://github.com/kawaguchi-s/teuthology/commit/348e54931f89c9b0ae7a84eb931576f8414017b5
do you really need to modify teuthology ? It should just be necessary to use the latest master branch.
It looks like the
https://github.com/kawaguchi-s/ceph-qa-suite/commit/f2e3ca5d12ceef742eae2a9cf4057c436e9040c3
commit in your ceph-qa-suite is not what you intended. However
https://github.com/kawaguchi-s/ceph-qa-suite/commit/4b39d6d4862f9091a849d224e880795be406815d
https://github.com/kawaguchi-s/ceph-qa-suite/commit/d16b4b058ae118931928541a2c8acd68f9703a44
look ok :-) Instead of naming the test 4nodes16osds3mons1client.yaml it would be better to use the same kind of naming you see at https://github.com/ceph/ceph-qa-suite/tree/master/suites/rados/thrash-erasure-code/workloads. That is a file name made of the distinctive parameters for the shec plugin (the parameters that are the default can be omited).
Cheers
> Here are our teuthology/ceph-qa-suite repositories. Thanks in advance.
>
> Best regards,
> Takeshi Miyamae
>
> -----Original Message-----
> From: Loic Dachary [mailto:loic@dachary.org]
> Sent: Wednesday, May 20, 2015 4:49 PM
> To: Miyamae, Takeshi/宮前 剛; Ceph Development
> Cc: Kawaguchi, Shotaro/川口 翔太朗; Imai, Hiroki/今井 宏樹; Nakao, Takanori/中尾 鷹詔; Shiozawa, Kensuke/塩沢 賢輔
> Subject: Re: teuthology timeout error
>
> Hi,
>
> On 20/05/2015 04:20, Miyamae, Takeshi wrote:
>> Hi Loic,
>>
>> When we fixed our own issue and restarted teuthology,
>
> Great !
>
>> we encountered another issue (timeout error) which occurs in case of LRC as well.
>> Do you have any information about that ?
>
> Could you please share the teuthology/ceph-qa-suite repository you are using to run these tests so I can try to reproduce / diagnose the problem ?
>
> Thanks
>
>>
>> [error messages (in case of LRC pool)]
>>
>> 2015-04-28 12:38:54,128.128 INFO:teuthology.orchestra.run.RX35-1:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph status --format=json-pretty'
>> 2015-04-28 12:38:54,516.516 INFO:tasks.ceph.ceph_manager:no progress seen, keeping timeout for now
>> 2015-04-28 12:38:54,516.516 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
>> return func(self)
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
>> timeout=self.config.get('timeout')
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
>> 'failed to recover before timeout expired'
>> AssertionError: failed to recover before timeout expired
>>
>> Traceback (most recent call last):
>> File "/root/work/teuthology/virtualenv/lib/python2.7/site-packages/gevent/greenlet.py", line 390, in run
>> result = self._run(*self.args, **self.kwargs)
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
>> return func(self)
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
>> timeout=self.config.get('timeout')
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
>> 'failed to recover before timeout expired'
>> AssertionError: failed to recover before timeout expired <Greenlet at 0x2a7d550: <bound method Thrasher.do_thrash of <tasks.ceph_manager.Thrasher instance at 0x2bd12d8>>> failed with AssertionError
>>
>> [ceph version]
>> 0.93-952-gfe28daa
>>
>> [teuthology, ceph-qa-suite]
>> newest version at 3/25/2015
>>
>> [configurations]
>> check-locks: false
>> overrides:
>> ceph:
>> conf:
>> global:
>> ms inject socket failures: 5000
>> osd:
>> osd heartbeat use min delay socket: true
>> osd sloppy crc: true
>> fs: xfs
>> roles:
>> - - mon.a
>> - osd.0
>> - osd.4
>> - osd.8
>> - osd.12
>> - - mon.b
>> - osd.1
>> - osd.5
>> - osd.9
>> - osd.13
>> - - mon.c
>> - osd.2
>> - osd.6
>> - osd.10
>> - osd.14
>> - - osd.3
>> - osd.7
>> - osd.11
>> - osd.15
>> - client.0
>> targets:
>> ubuntu@RX35-1.primary.ceph-poc.fsc.net:
>> ubuntu@RX35-2.primary.ceph-poc.fsc.net:
>> ubuntu@RX35-3.primary.ceph-poc.fsc.net:
>> ubuntu@RX35-4.primary.ceph-poc.fsc.net:
>> tasks:
>> - ceph:
>> conf:
>> osd:
>> osd debug reject backfill probability: 0.3
>> osd max backfills: 1
>> osd scrub max interval: 120
>> osd scrub min interval: 60
>> log-whitelist:
>> - wrongly marked me down
>> - objects unfound and apparently lost
>> - thrashosds:
>> chance_pgnum_grow: 1
>> chance_pgpnum_fix: 1
>> min_in: 4
>> timeout: 1200
>> - rados:
>> clients:
>> - client.0
>> ec_pool: true
>> erasure_code_profile:
>> k: 4
>> l: 3
>> m: 2
>> name: lrcprofile
>> plugin: lrc
>> ruleset-failure-domain: osd
>> objects: 50
>> op_weights:
>> append: 100
>> copy_from: 50
>> delete: 50
>> read: 100
>> rmattr: 25
>> rollback: 50
>> setattr: 25
>> snap_create: 50
>> snap_remove: 50
>> write: 0
>> ops: 190000
>>
>> Best regards,
>> Takeshi Miyamae
>>
>
--
Loïc Dachary, Artisan Logiciel Libre
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* RE: teuthology timeout error
2015-05-21 9:37 ` Loic Dachary
@ 2015-05-26 2:39 ` Miyamae, Takeshi
2015-05-26 8:59 ` Loic Dachary
0 siblings, 1 reply; 7+ messages in thread
From: Miyamae, Takeshi @ 2015-05-26 2:39 UTC (permalink / raw)
To: Loic Dachary, Ceph Development
Cc: Kawaguchi, Shotaro, Imai, Hiroki, Nakao, Takanori, Shiozawa, Kensuke
Hi Loic,
We rebased our teuthology/ceph-qa-suite and retried the test toward LRC on current master.
However, we unfortunately got the same result as before (timeout error).
[test conditions]
Target : Ceph-9.0.0-971-gd49d816
https://github.com/kawaguchi-s/teuthology
https://github.com/kawaguchi-s/ceph-qa-suite/tree/wip-10886-lrc
[teuthology log]
2015-05-25 10:18:23 # start
2015-05-25 11:59:52,106.106 INFO:teuthology.orchestra.run.RX35-1:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph status -- format=json-pretty'
2015-05-25 11:59:52,564.564 INFO:tasks.ceph.ceph_manager:no progress seen, keeping timeout for now
2015-05-25 11:59:52,565.565 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 635, in wrapper
return func(self)
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 668, in do_thrash
timeout=self.config.get('timeout')
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1569, in wait_for_recovery
'failed to recover before timeout expired'
AssertionError: failed to recover before timeout expired
Traceback (most recent call last):
File "/root/work/teuthology/virtualenv/lib/python2.7/site-packages/gevent/greenlet.py", line 390, in run
result = self._run(*self.args, **self.kwargs)
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 635, in wrapper
return func(self)
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 668, in do_thrash
timeout=self.config.get('timeout')
File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1569, in wait_for_recovery
'failed to recover before timeout expired'
AssertionError: failed to recover before timeout expired <Greenlet at 0x36cacd0: <bound method Thrasher.do_thrash of <tasks.ceph_manager.Thrasher instance at 0x36df3f8>>> failed with AssertionError
Best regards,
Takeshi Miyamae
-----Original Message-----
From: Loic Dachary [mailto:loic@dachary.org]
Sent: Thursday, May 21, 2015 6:38 PM
To: Miyamae, Takeshi/宮前 剛; Ceph Development
Cc: Kawaguchi, Shotaro/川口 翔太朗; Imai, Hiroki/今井 宏樹; Nakao, Takanori/中尾 鷹詔; Shiozawa, Kensuke/塩沢 賢輔
Subject: Re: teuthology timeout error
Hi,
[sorry the previous mail was sent by accident, here is the full mail]
On 21/05/2015 10:32, Miyamae, Takeshi wrote:
> Hi Loic,
>
>> Could you please share the teuthology/ceph-qa-suite repository you
>> are using to run these tests so I can try to reproduce / diagnose the problem ?
>
> https://github.com/kawaguchi-s/teuthology/tree/wip-10886
> https://github.com/kawaguchi-s/ceph-qa-suite/tree/wip-10886
>
When compared against master they show differences that indicate it would be good to rebase:
https://github.com/ceph/teuthology/compare/master...kawaguchi-s:wip-10886
https://github.com/ceph/ceph-qa-suite/compare/master...kawaguchi-s:wip-10886
I think the teuthology commit on top of wip-10886 is a mistake
https://github.com/kawaguchi-s/teuthology/commit/348e54931f89c9b0ae7a84eb931576f8414017b5
do you really need to modify teuthology ? It should just be necessary to use the latest master branch.
It looks like the
https://github.com/kawaguchi-s/ceph-qa-suite/commit/f2e3ca5d12ceef742eae2a9cf4057c436e9040c3
commit in your ceph-qa-suite is not what you intended. However
https://github.com/kawaguchi-s/ceph-qa-suite/commit/4b39d6d4862f9091a849d224e880795be406815d
https://github.com/kawaguchi-s/ceph-qa-suite/commit/d16b4b058ae118931928541a2c8acd68f9703a44
look ok :-) Instead of naming the test 4nodes16osds3mons1client.yaml it would be better to use the same kind of naming you see at https://github.com/ceph/ceph-qa-suite/tree/master/suites/rados/thrash-erasure-code/workloads. That is a file name made of the distinctive parameters for the shec plugin (the parameters that are the default can be omited).
Cheers
> Here are our teuthology/ceph-qa-suite repositories. Thanks in advance.
>
> Best regards,
> Takeshi Miyamae
>
> -----Original Message-----
> From: Loic Dachary [mailto:loic@dachary.org]
> Sent: Wednesday, May 20, 2015 4:49 PM
> To: Miyamae, Takeshi/宮前 剛; Ceph Development
> Cc: Kawaguchi, Shotaro/川口 翔太朗; Imai, Hiroki/今井 宏樹; Nakao, Takanori/中尾
> 鷹詔; Shiozawa, Kensuke/塩沢 賢輔
> Subject: Re: teuthology timeout error
>
> Hi,
>
> On 20/05/2015 04:20, Miyamae, Takeshi wrote:
>> Hi Loic,
>>
>> When we fixed our own issue and restarted teuthology,
>
> Great !
>
>> we encountered another issue (timeout error) which occurs in case of LRC as well.
>> Do you have any information about that ?
>
> Could you please share the teuthology/ceph-qa-suite repository you are using to run these tests so I can try to reproduce / diagnose the problem ?
>
> Thanks
>
>>
>> [error messages (in case of LRC pool)]
>>
>> 2015-04-28 12:38:54,128.128 INFO:teuthology.orchestra.run.RX35-1:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph status --format=json-pretty'
>> 2015-04-28 12:38:54,516.516 INFO:tasks.ceph.ceph_manager:no progress
>> seen, keeping timeout for now
>> 2015-04-28 12:38:54,516.516 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
>> return func(self)
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
>> timeout=self.config.get('timeout')
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
>> 'failed to recover before timeout expired'
>> AssertionError: failed to recover before timeout expired
>>
>> Traceback (most recent call last):
>> File "/root/work/teuthology/virtualenv/lib/python2.7/site-packages/gevent/greenlet.py", line 390, in run
>> result = self._run(*self.args, **self.kwargs)
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
>> return func(self)
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
>> timeout=self.config.get('timeout')
>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
>> 'failed to recover before timeout expired'
>> AssertionError: failed to recover before timeout expired <Greenlet at
>> 0x2a7d550: <bound method Thrasher.do_thrash of
>> <tasks.ceph_manager.Thrasher instance at 0x2bd12d8>>> failed with
>> AssertionError
>>
>> [ceph version]
>> 0.93-952-gfe28daa
>>
>> [teuthology, ceph-qa-suite]
>> newest version at 3/25/2015
>>
>> [configurations]
>> check-locks: false
>> overrides:
>> ceph:
>> conf:
>> global:
>> ms inject socket failures: 5000
>> osd:
>> osd heartbeat use min delay socket: true
>> osd sloppy crc: true
>> fs: xfs
>> roles:
>> - - mon.a
>> - osd.0
>> - osd.4
>> - osd.8
>> - osd.12
>> - - mon.b
>> - osd.1
>> - osd.5
>> - osd.9
>> - osd.13
>> - - mon.c
>> - osd.2
>> - osd.6
>> - osd.10
>> - osd.14
>> - - osd.3
>> - osd.7
>> - osd.11
>> - osd.15
>> - client.0
>> targets:
>> ubuntu@RX35-1.primary.ceph-poc.fsc.net:
>> ubuntu@RX35-2.primary.ceph-poc.fsc.net:
>> ubuntu@RX35-3.primary.ceph-poc.fsc.net:
>> ubuntu@RX35-4.primary.ceph-poc.fsc.net:
>> tasks:
>> - ceph:
>> conf:
>> osd:
>> osd debug reject backfill probability: 0.3
>> osd max backfills: 1
>> osd scrub max interval: 120
>> osd scrub min interval: 60
>> log-whitelist:
>> - wrongly marked me down
>> - objects unfound and apparently lost
>> - thrashosds:
>> chance_pgnum_grow: 1
>> chance_pgpnum_fix: 1
>> min_in: 4
>> timeout: 1200
>> - rados:
>> clients:
>> - client.0
>> ec_pool: true
>> erasure_code_profile:
>> k: 4
>> l: 3
>> m: 2
>> name: lrcprofile
>> plugin: lrc
>> ruleset-failure-domain: osd
>> objects: 50
>> op_weights:
>> append: 100
>> copy_from: 50
>> delete: 50
>> read: 100
>> rmattr: 25
>> rollback: 50
>> setattr: 25
>> snap_create: 50
>> snap_remove: 50
>> write: 0
>> ops: 190000
>>
>> Best regards,
>> Takeshi Miyamae
>>
>
--
Loïc Dachary, Artisan Logiciel Libre
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: teuthology timeout error
2015-05-26 2:39 ` Miyamae, Takeshi
@ 2015-05-26 8:59 ` Loic Dachary
0 siblings, 0 replies; 7+ messages in thread
From: Loic Dachary @ 2015-05-26 8:59 UTC (permalink / raw)
To: Miyamae, Takeshi, Ceph Development
Cc: Kawaguchi, Shotaro, Imai, Hiroki, Nakao, Takanori, Shiozawa, Kensuke
[-- Attachment #1: Type: text/plain, Size: 9304 bytes --]
Hi Takeshi,
I'm trying to repeat your problem at https://github.com/ceph/ceph-qa-suite/pull/445. To be continued :-)
Cheers
On 26/05/2015 04:39, Miyamae, Takeshi wrote:
> Hi Loic,
>
> We rebased our teuthology/ceph-qa-suite and retried the test toward LRC on current master.
> However, we unfortunately got the same result as before (timeout error).
>
> [test conditions]
> Target : Ceph-9.0.0-971-gd49d816
> https://github.com/kawaguchi-s/teuthology
> https://github.com/kawaguchi-s/ceph-qa-suite/tree/wip-10886-lrc
>
> [teuthology log]
>
> 2015-05-25 10:18:23 # start
>
> 2015-05-25 11:59:52,106.106 INFO:teuthology.orchestra.run.RX35-1:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph status -- format=json-pretty'
> 2015-05-25 11:59:52,564.564 INFO:tasks.ceph.ceph_manager:no progress seen, keeping timeout for now
> 2015-05-25 11:59:52,565.565 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 635, in wrapper
> return func(self)
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 668, in do_thrash
> timeout=self.config.get('timeout')
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1569, in wait_for_recovery
> 'failed to recover before timeout expired'
> AssertionError: failed to recover before timeout expired
>
> Traceback (most recent call last):
> File "/root/work/teuthology/virtualenv/lib/python2.7/site-packages/gevent/greenlet.py", line 390, in run
> result = self._run(*self.args, **self.kwargs)
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 635, in wrapper
> return func(self)
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 668, in do_thrash
> timeout=self.config.get('timeout')
> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1569, in wait_for_recovery
> 'failed to recover before timeout expired'
> AssertionError: failed to recover before timeout expired <Greenlet at 0x36cacd0: <bound method Thrasher.do_thrash of <tasks.ceph_manager.Thrasher instance at 0x36df3f8>>> failed with AssertionError
>
> Best regards,
> Takeshi Miyamae
>
> -----Original Message-----
> From: Loic Dachary [mailto:loic@dachary.org]
> Sent: Thursday, May 21, 2015 6:38 PM
> To: Miyamae, Takeshi/宮前 剛; Ceph Development
> Cc: Kawaguchi, Shotaro/川口 翔太朗; Imai, Hiroki/今井 宏樹; Nakao, Takanori/中尾 鷹詔; Shiozawa, Kensuke/塩沢 賢輔
> Subject: Re: teuthology timeout error
>
> Hi,
>
> [sorry the previous mail was sent by accident, here is the full mail]
>
> On 21/05/2015 10:32, Miyamae, Takeshi wrote:
>> Hi Loic,
>>
>>> Could you please share the teuthology/ceph-qa-suite repository you
>>> are using to run these tests so I can try to reproduce / diagnose the problem ?
>>
>> https://github.com/kawaguchi-s/teuthology/tree/wip-10886
>> https://github.com/kawaguchi-s/ceph-qa-suite/tree/wip-10886
>>
>
> When compared against master they show differences that indicate it would be good to rebase:
>
> https://github.com/ceph/teuthology/compare/master...kawaguchi-s:wip-10886
> https://github.com/ceph/ceph-qa-suite/compare/master...kawaguchi-s:wip-10886
>
> I think the teuthology commit on top of wip-10886 is a mistake
>
> https://github.com/kawaguchi-s/teuthology/commit/348e54931f89c9b0ae7a84eb931576f8414017b5
>
> do you really need to modify teuthology ? It should just be necessary to use the latest master branch.
>
> It looks like the
>
> https://github.com/kawaguchi-s/ceph-qa-suite/commit/f2e3ca5d12ceef742eae2a9cf4057c436e9040c3
>
> commit in your ceph-qa-suite is not what you intended. However
>
> https://github.com/kawaguchi-s/ceph-qa-suite/commit/4b39d6d4862f9091a849d224e880795be406815d
> https://github.com/kawaguchi-s/ceph-qa-suite/commit/d16b4b058ae118931928541a2c8acd68f9703a44
>
> look ok :-) Instead of naming the test 4nodes16osds3mons1client.yaml it would be better to use the same kind of naming you see at https://github.com/ceph/ceph-qa-suite/tree/master/suites/rados/thrash-erasure-code/workloads. That is a file name made of the distinctive parameters for the shec plugin (the parameters that are the default can be omited).
>
> Cheers
>
>> Here are our teuthology/ceph-qa-suite repositories. Thanks in advance.
>>
>> Best regards,
>> Takeshi Miyamae
>>
>> -----Original Message-----
>> From: Loic Dachary [mailto:loic@dachary.org]
>> Sent: Wednesday, May 20, 2015 4:49 PM
>> To: Miyamae, Takeshi/宮前 剛; Ceph Development
>> Cc: Kawaguchi, Shotaro/川口 翔太朗; Imai, Hiroki/今井 宏樹; Nakao, Takanori/中尾
>> 鷹詔; Shiozawa, Kensuke/塩沢 賢輔
>> Subject: Re: teuthology timeout error
>>
>> Hi,
>>
>> On 20/05/2015 04:20, Miyamae, Takeshi wrote:
>>> Hi Loic,
>>>
>>> When we fixed our own issue and restarted teuthology,
>>
>> Great !
>>
>>> we encountered another issue (timeout error) which occurs in case of LRC as well.
>>> Do you have any information about that ?
>>
>> Could you please share the teuthology/ceph-qa-suite repository you are using to run these tests so I can try to reproduce / diagnose the problem ?
>>
>> Thanks
>>
>>>
>>> [error messages (in case of LRC pool)]
>>>
>>> 2015-04-28 12:38:54,128.128 INFO:teuthology.orchestra.run.RX35-1:Running: 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph status --format=json-pretty'
>>> 2015-04-28 12:38:54,516.516 INFO:tasks.ceph.ceph_manager:no progress
>>> seen, keeping timeout for now
>>> 2015-04-28 12:38:54,516.516 INFO:tasks.thrashosds.thrasher:Traceback (most recent call last):
>>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
>>> return func(self)
>>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
>>> timeout=self.config.get('timeout')
>>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
>>> 'failed to recover before timeout expired'
>>> AssertionError: failed to recover before timeout expired
>>>
>>> Traceback (most recent call last):
>>> File "/root/work/teuthology/virtualenv/lib/python2.7/site-packages/gevent/greenlet.py", line 390, in run
>>> result = self._run(*self.args, **self.kwargs)
>>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 632, in wrapper
>>> return func(self)
>>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 665, in do_thrash
>>> timeout=self.config.get('timeout')
>>> File "/root/src/ceph-qa-suite_master/tasks/ceph_manager.py", line 1566, in wait_for_recovery
>>> 'failed to recover before timeout expired'
>>> AssertionError: failed to recover before timeout expired <Greenlet at
>>> 0x2a7d550: <bound method Thrasher.do_thrash of
>>> <tasks.ceph_manager.Thrasher instance at 0x2bd12d8>>> failed with
>>> AssertionError
>>>
>>> [ceph version]
>>> 0.93-952-gfe28daa
>>>
>>> [teuthology, ceph-qa-suite]
>>> newest version at 3/25/2015
>>>
>>> [configurations]
>>> check-locks: false
>>> overrides:
>>> ceph:
>>> conf:
>>> global:
>>> ms inject socket failures: 5000
>>> osd:
>>> osd heartbeat use min delay socket: true
>>> osd sloppy crc: true
>>> fs: xfs
>>> roles:
>>> - - mon.a
>>> - osd.0
>>> - osd.4
>>> - osd.8
>>> - osd.12
>>> - - mon.b
>>> - osd.1
>>> - osd.5
>>> - osd.9
>>> - osd.13
>>> - - mon.c
>>> - osd.2
>>> - osd.6
>>> - osd.10
>>> - osd.14
>>> - - osd.3
>>> - osd.7
>>> - osd.11
>>> - osd.15
>>> - client.0
>>> targets:
>>> ubuntu@RX35-1.primary.ceph-poc.fsc.net:
>>> ubuntu@RX35-2.primary.ceph-poc.fsc.net:
>>> ubuntu@RX35-3.primary.ceph-poc.fsc.net:
>>> ubuntu@RX35-4.primary.ceph-poc.fsc.net:
>>> tasks:
>>> - ceph:
>>> conf:
>>> osd:
>>> osd debug reject backfill probability: 0.3
>>> osd max backfills: 1
>>> osd scrub max interval: 120
>>> osd scrub min interval: 60
>>> log-whitelist:
>>> - wrongly marked me down
>>> - objects unfound and apparently lost
>>> - thrashosds:
>>> chance_pgnum_grow: 1
>>> chance_pgpnum_fix: 1
>>> min_in: 4
>>> timeout: 1200
>>> - rados:
>>> clients:
>>> - client.0
>>> ec_pool: true
>>> erasure_code_profile:
>>> k: 4
>>> l: 3
>>> m: 2
>>> name: lrcprofile
>>> plugin: lrc
>>> ruleset-failure-domain: osd
>>> objects: 50
>>> op_weights:
>>> append: 100
>>> copy_from: 50
>>> delete: 50
>>> read: 100
>>> rmattr: 25
>>> rollback: 50
>>> setattr: 25
>>> snap_create: 50
>>> snap_remove: 50
>>> write: 0
>>> ops: 190000
>>>
>>> Best regards,
>>> Takeshi Miyamae
>>>
>>
>
> --
> Loïc Dachary, Artisan Logiciel Libre
>
>
>
--
Loïc Dachary, Artisan Logiciel Libre
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2015-05-26 8:59 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-05-20 2:20 teuthology timeout error Miyamae, Takeshi
2015-05-20 7:48 ` Loic Dachary
2015-05-21 8:32 ` Miyamae, Takeshi
2015-05-21 9:30 ` Loic Dachary
2015-05-21 9:37 ` Loic Dachary
2015-05-26 2:39 ` Miyamae, Takeshi
2015-05-26 8:59 ` Loic Dachary
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.