* Speed up 'rbd rm' @ 2013-05-29 5:59 Chris Dunlop 2013-05-29 19:21 ` Josh Durgin 0 siblings, 1 reply; 10+ messages in thread From: Chris Dunlop @ 2013-05-29 5:59 UTC (permalink / raw) To: ceph-devel Hi, I know 'rbd rm' is notoriously slow, even on never-written devices: http://thread.gmane.org/gmane.comp.file-systems.ceph.devel/6740 http://tracker.ceph.com/issues/2256 I fat fingered an 'rbd create' and accidentally created a 1.5 PB device, and it's going to take some considerable time to remove this! I see there's a new commit to speed up an 'rbd rm': http://tracker.ceph.com/projects/ceph/repository/revisions/40956410169709c32a282d9b872cb5f618a48926 Is it safe to cherry-pick this commit on top of 0.56.6 (or, if not, v0.61.2) to speed up the remove? Cheers, Chris ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Speed up 'rbd rm' 2013-05-29 5:59 Speed up 'rbd rm' Chris Dunlop @ 2013-05-29 19:21 ` Josh Durgin 2013-05-30 2:23 ` Chris Dunlop 0 siblings, 1 reply; 10+ messages in thread From: Josh Durgin @ 2013-05-29 19:21 UTC (permalink / raw) To: Chris Dunlop; +Cc: ceph-devel On 05/28/2013 10:59 PM, Chris Dunlop wrote: > I see there's a new commit to speed up an 'rbd rm': > > http://tracker.ceph.com/projects/ceph/repository/revisions/40956410169709c32a282d9b872cb5f618a48926 > > Is it safe to cherry-pick this commit on top of 0.56.6 (or, if not, v0.61.2) to speed up the remove? You'll need 537386d906b8c0e395433461dcb03a82eb33f34f as well. It should apply cleanly to 0.61.2, and probably 0.56.6 too. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Speed up 'rbd rm' 2013-05-29 19:21 ` Josh Durgin @ 2013-05-30 2:23 ` Chris Dunlop 2013-05-30 20:50 ` Josh Durgin 0 siblings, 1 reply; 10+ messages in thread From: Chris Dunlop @ 2013-05-30 2:23 UTC (permalink / raw) To: Josh Durgin; +Cc: ceph-devel On Wed, May 29, 2013 at 12:21:07PM -0700, Josh Durgin wrote: > On 05/28/2013 10:59 PM, Chris Dunlop wrote: >> I see there's a new commit to speed up an 'rbd rm': >> >> http://tracker.ceph.com/projects/ceph/repository/revisions/40956410169709c32a282d9b872cb5f618a48926 >> >> Is it safe to cherry-pick this commit on top of 0.56.6 (or, if not, v0.61.2) to speed up the remove? > > You'll need 537386d906b8c0e395433461dcb03a82eb33f34f as well. It should > apply cleanly to 0.61.2, and probably 0.56.6 too. Thanks. I'll see how I go, I may just leave the 'rm' running all weekend rather than futzing around recompiling ceph and getting off the mainline track. Perhaps as a low-churn high-impact (for those that need it) change, this is a candidate to be back-patched into 0.61.3 and perhaps 0.56.7 ? Cheers, Chris. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Speed up 'rbd rm' 2013-05-30 2:23 ` Chris Dunlop @ 2013-05-30 20:50 ` Josh Durgin 2013-05-30 21:09 ` Smart Weblications GmbH - Florian Wiessner 2013-05-31 1:40 ` Chris Dunlop 0 siblings, 2 replies; 10+ messages in thread From: Josh Durgin @ 2013-05-30 20:50 UTC (permalink / raw) To: Chris Dunlop; +Cc: ceph-devel On 05/29/2013 07:23 PM, Chris Dunlop wrote: > On Wed, May 29, 2013 at 12:21:07PM -0700, Josh Durgin wrote: >> On 05/28/2013 10:59 PM, Chris Dunlop wrote: >>> I see there's a new commit to speed up an 'rbd rm': >>> >>> http://tracker.ceph.com/projects/ceph/repository/revisions/40956410169709c32a282d9b872cb5f618a48926 >>> >>> Is it safe to cherry-pick this commit on top of 0.56.6 (or, if not, v0.61.2) to speed up the remove? >> >> You'll need 537386d906b8c0e395433461dcb03a82eb33f34f as well. It should >> apply cleanly to 0.61.2, and probably 0.56.6 too. > > Thanks. I'll see how I go, I may just leave the 'rm' running all > weekend rather than futzing around recompiling ceph and getting > off the mainline track. If you're mainly interested in getting rid of the accidentally 1.5PB image, you can just delete the header (and id object if it's format 2) and then 'rbd rm' will just remove it from the rbd_directory index, and not try to delete all the non-existent data objects. > Perhaps as a low-churn high-impact (for those that need it) change, > this is a candidate to be back-patched into 0.61.3 and perhaps 0.56.7 ? > > Cheers, > > Chris. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Speed up 'rbd rm' 2013-05-30 20:50 ` Josh Durgin @ 2013-05-30 21:09 ` Smart Weblications GmbH - Florian Wiessner 2013-05-31 1:40 ` Chris Dunlop 1 sibling, 0 replies; 10+ messages in thread From: Smart Weblications GmbH - Florian Wiessner @ 2013-05-30 21:09 UTC (permalink / raw) To: Josh Durgin; +Cc: Chris Dunlop, ceph-devel Am 30.05.2013 22:50, schrieb Josh Durgin: > >> Perhaps as a low-churn high-impact (for those that need it) change, >> this is a candidate to be back-patched into 0.61.3 and perhaps 0.56.7 ? >> +1 -- Mit freundlichen Grüßen, Florian Wiessner Smart Weblications GmbH Martinsberger Str. 1 D-95119 Naila fon.: +49 9282 9638 200 fax.: +49 9282 9638 205 24/7: +49 900 144 000 00 - 0,99 EUR/Min* http://www.smart-weblications.de -- Sitz der Gesellschaft: Naila Geschäftsführer: Florian Wiessner HRB-Nr.: HRB 3840 Amtsgericht Hof *aus dem dt. Festnetz, ggf. abweichende Preise aus dem Mobilfunknetz -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Speed up 'rbd rm' 2013-05-30 20:50 ` Josh Durgin 2013-05-30 21:09 ` Smart Weblications GmbH - Florian Wiessner @ 2013-05-31 1:40 ` Chris Dunlop 2013-05-31 2:04 ` Josh Durgin 2013-05-31 6:40 ` Stefan Priebe - Profihost AG 1 sibling, 2 replies; 10+ messages in thread From: Chris Dunlop @ 2013-05-31 1:40 UTC (permalink / raw) To: Josh Durgin; +Cc: ceph-devel On Thu, May 30, 2013 at 01:50:14PM -0700, Josh Durgin wrote: > On 05/29/2013 07:23 PM, Chris Dunlop wrote: >> On Wed, May 29, 2013 at 12:21:07PM -0700, Josh Durgin wrote: >>> On 05/28/2013 10:59 PM, Chris Dunlop wrote: >>>> I see there's a new commit to speed up an 'rbd rm': >>>> >>>> http://tracker.ceph.com/projects/ceph/repository/revisions/40956410169709c32a282d9b872cb5f618a48926 >>>> >>>> Is it safe to cherry-pick this commit on top of 0.56.6 (or, if not, v0.61.2) to speed up the remove? >>> >>> You'll need 537386d906b8c0e395433461dcb03a82eb33f34f as well. It should >>> apply cleanly to 0.61.2, and probably 0.56.6 too. >> >> Thanks. I'll see how I go, I may just leave the 'rm' running all >> weekend rather than futzing around recompiling ceph and getting >> off the mainline track. > > If you're mainly interested in getting rid of the accidentally 1.5PB > image, you can just delete the header (and id object if it's format 2) > and then 'rbd rm' will just remove it from the rbd_directory index, and > not try to delete all the non-existent data objects. Yes, that's my main interest. Sorry, I haven't yet delved far into the details of how the rbd stuff hangs together: can you give me a hint or point me towards any docs regarding what "delete the header (and id object" would look like? Cheers, Chris. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Speed up 'rbd rm' 2013-05-31 1:40 ` Chris Dunlop @ 2013-05-31 2:04 ` Josh Durgin 2013-06-03 9:15 ` Chris Dunlop 2013-05-31 6:40 ` Stefan Priebe - Profihost AG 1 sibling, 1 reply; 10+ messages in thread From: Josh Durgin @ 2013-05-31 2:04 UTC (permalink / raw) To: Chris Dunlop; +Cc: ceph-devel On 05/30/2013 06:40 PM, Chris Dunlop wrote: > On Thu, May 30, 2013 at 01:50:14PM -0700, Josh Durgin wrote: >> On 05/29/2013 07:23 PM, Chris Dunlop wrote: >>> On Wed, May 29, 2013 at 12:21:07PM -0700, Josh Durgin wrote: >>>> On 05/28/2013 10:59 PM, Chris Dunlop wrote: >>>>> I see there's a new commit to speed up an 'rbd rm': >>>>> >>>>> http://tracker.ceph.com/projects/ceph/repository/revisions/40956410169709c32a282d9b872cb5f618a48926 >>>>> >>>>> Is it safe to cherry-pick this commit on top of 0.56.6 (or, if not, v0.61.2) to speed up the remove? >>>> >>>> You'll need 537386d906b8c0e395433461dcb03a82eb33f34f as well. It should >>>> apply cleanly to 0.61.2, and probably 0.56.6 too. >>> >>> Thanks. I'll see how I go, I may just leave the 'rm' running all >>> weekend rather than futzing around recompiling ceph and getting >>> off the mainline track. >> >> If you're mainly interested in getting rid of the accidentally 1.5PB >> image, you can just delete the header (and id object if it's format 2) >> and then 'rbd rm' will just remove it from the rbd_directory index, and >> not try to delete all the non-existent data objects. > > Yes, that's my main interest. Sorry, I haven't yet delved far > into the details of how the rbd stuff hangs together: can you > give me a hint or point me towards any docs regarding what > "delete the header (and id object" would look like? For a format 2 image, 'rbd info imagename' will show a block_prefix like 'rbd_data.101574b0dc51'. The random suffix after the '.' is the id of the image. For format 2, the header is named after this id, so you'd do: rados -p poolname rm rbd_header.101574b0dc51 For format 1 images, the header object is named after the image name, like 'imagename.rbd'. After removing the header object manually, rbd rm will clean up the rest. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Speed up 'rbd rm' 2013-05-31 2:04 ` Josh Durgin @ 2013-06-03 9:15 ` Chris Dunlop 0 siblings, 0 replies; 10+ messages in thread From: Chris Dunlop @ 2013-06-03 9:15 UTC (permalink / raw) To: Josh Durgin; +Cc: ceph-devel On Thu, May 30, 2013 at 07:04:28PM -0700, Josh Durgin wrote: > On 05/30/2013 06:40 PM, Chris Dunlop wrote: >> On Thu, May 30, 2013 at 01:50:14PM -0700, Josh Durgin wrote: >>> On 05/29/2013 07:23 PM, Chris Dunlop wrote: >>>> On Wed, May 29, 2013 at 12:21:07PM -0700, Josh Durgin wrote: >>>>> On 05/28/2013 10:59 PM, Chris Dunlop wrote: >>>>>> I see there's a new commit to speed up an 'rbd rm': >>>>>> >>>>>> http://tracker.ceph.com/projects/ceph/repository/revisions/40956410169709c32a282d9b872cb5f618a48926 >>>>>> >>>>>> Is it safe to cherry-pick this commit on top of 0.56.6 (or, if not, v0.61.2) to speed up the remove? >>>>> >>>>> You'll need 537386d906b8c0e395433461dcb03a82eb33f34f as well. It should >>>>> apply cleanly to 0.61.2, and probably 0.56.6 too. >>>> >>>> Thanks. I'll see how I go, I may just leave the 'rm' running all >>>> weekend rather than futzing around recompiling ceph and getting >>>> off the mainline track. # time rbd rm rbd/large-image Removing image: 36% complete...Terminated real 2819m37.117s I.e. 47 hours and only 36% complete before I gave up (I wanted to restart that server). At that rate it would take 5.5 days to remove! >>> If you're mainly interested in getting rid of the accidentally 1.5PB >>> image, you can just delete the header (and id object if it's format 2) >>> and then 'rbd rm' will just remove it from the rbd_directory index, and >>> not try to delete all the non-existent data objects. >> >> Yes, that's my main interest. Sorry, I haven't yet delved far >> into the details of how the rbd stuff hangs together: can you >> give me a hint or point me towards any docs regarding what >> "delete the header (and id object" would look like? > > For a format 2 image, 'rbd info imagename' will show a block_prefix > like 'rbd_data.101574b0dc51'. > > The random suffix after the '.' is the id of the image. > For format 2, the header is named after this id, so you'd do: > > rados -p poolname rm rbd_header.101574b0dc51 > > For format 1 images, the header object is named after the image name, > like 'imagename.rbd'. > > After removing the header object manually, rbd rm will clean up the > rest. The problematical image is format 2. If it's tricky to manually remove, it's not doing any harm just sitting there (it it??) so I guess I can just wait until the parallelized delete is available in a stable release, i.e. dumpling, or backported to bobtail or cuttlefish. Cheers, Chris. ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Speed up 'rbd rm' 2013-05-31 1:40 ` Chris Dunlop 2013-05-31 2:04 ` Josh Durgin @ 2013-05-31 6:40 ` Stefan Priebe - Profihost AG 2013-05-31 23:53 ` Josh Durgin 1 sibling, 1 reply; 10+ messages in thread From: Stefan Priebe - Profihost AG @ 2013-05-31 6:40 UTC (permalink / raw) To: Chris Dunlop; +Cc: Josh Durgin, ceph-devel Am 31.05.2013 03:40, schrieb Chris Dunlop: > On Thu, May 30, 2013 at 01:50:14PM -0700, Josh Durgin wrote: >> On 05/29/2013 07:23 PM, Chris Dunlop wrote: >>> On Wed, May 29, 2013 at 12:21:07PM -0700, Josh Durgin wrote: >>>> On 05/28/2013 10:59 PM, Chris Dunlop wrote: >>>>> I see there's a new commit to speed up an 'rbd rm': >>>>> >>>>> http://tracker.ceph.com/projects/ceph/repository/revisions/40956410169709c32a282d9b872cb5f618a48926 >>>>> >>>>> Is it safe to cherry-pick this commit on top of 0.56.6 (or, if not, v0.61.2) to speed up the remove? >>>> >>>> You'll need 537386d906b8c0e395433461dcb03a82eb33f34f as well. It should >>>> apply cleanly to 0.61.2, and probably 0.56.6 too. I cherry picked both on top of cuttlefish it works but not very good. rbd command finishes but rbd image is still in rbd ls and just gets deleted later. I also saw a variant, where rbd rm outputs errors even though the image was there but it still got deleted... Stefan ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: Speed up 'rbd rm' 2013-05-31 6:40 ` Stefan Priebe - Profihost AG @ 2013-05-31 23:53 ` Josh Durgin 0 siblings, 0 replies; 10+ messages in thread From: Josh Durgin @ 2013-05-31 23:53 UTC (permalink / raw) To: Stefan Priebe - Profihost AG; +Cc: Chris Dunlop, ceph-devel On 05/30/2013 11:40 PM, Stefan Priebe - Profihost AG wrote: > Am 31.05.2013 03:40, schrieb Chris Dunlop: >> On Thu, May 30, 2013 at 01:50:14PM -0700, Josh Durgin wrote: >>> On 05/29/2013 07:23 PM, Chris Dunlop wrote: >>>> On Wed, May 29, 2013 at 12:21:07PM -0700, Josh Durgin wrote: >>>>> On 05/28/2013 10:59 PM, Chris Dunlop wrote: >>>>>> I see there's a new commit to speed up an 'rbd rm': >>>>>> >>>>>> http://tracker.ceph.com/projects/ceph/repository/revisions/40956410169709c32a282d9b872cb5f618a48926 >>>>>> >>>>>> Is it safe to cherry-pick this commit on top of 0.56.6 (or, if not, v0.61.2) to speed up the remove? >>>>> >>>>> You'll need 537386d906b8c0e395433461dcb03a82eb33f34f as well. It should >>>>> apply cleanly to 0.61.2, and probably 0.56.6 too. > > I cherry picked both on top of cuttlefish it works but not very good. > rbd command finishes but rbd image is still in rbd ls and just gets > deleted later. I also saw a variant, where rbd rm outputs errors even > though the image was there but it still got deleted... I forgot that there was a bug in the original version. You'll need 395a775a8c87b0ee1ce314257e25240214f4081c too. ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2013-06-03 9:15 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2013-05-29 5:59 Speed up 'rbd rm' Chris Dunlop 2013-05-29 19:21 ` Josh Durgin 2013-05-30 2:23 ` Chris Dunlop 2013-05-30 20:50 ` Josh Durgin 2013-05-30 21:09 ` Smart Weblications GmbH - Florian Wiessner 2013-05-31 1:40 ` Chris Dunlop 2013-05-31 2:04 ` Josh Durgin 2013-06-03 9:15 ` Chris Dunlop 2013-05-31 6:40 ` Stefan Priebe - Profihost AG 2013-05-31 23:53 ` Josh Durgin
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.