All of lore.kernel.org
 help / color / mirror / Atom feed
* Translating a RadosGW object name into a filename on disk
@ 2014-08-14 18:34 Craig Lewis
  2014-08-19 17:27 ` Gregory Farnum
  0 siblings, 1 reply; 5+ messages in thread
From: Craig Lewis @ 2014-08-14 18:34 UTC (permalink / raw)
  To: Ceph Devel, Ceph Users

In my effort to learn more of the details of Ceph, I'm trying to
figure out how to get from an object name in RadosGW, through the
layers, down to the files on disk.

clewis@clewis-mac ~ $ s3cmd ls s3://cpltest/
2014-08-13 23:02        14M  28dde9db15fdcb5a342493bc81f91151
s3://cpltest/vmware-freebsd-tools.tar.gz

Looking at the .rgw pool's contents tells me that the cpltest bucket
is default.73886.55:
root@dev-ceph0:/var/lib/ceph/osd/ceph-0/current# rados -p .rgw ls | grep cpltest
cpltest
.bucket.meta.cpltest:default.73886.55

The rados objects that belong to that bucket are:
root@dev-ceph0:~# rados -p .rgw.buckets ls | grep default.73886.55
default.73886.55__shadow__RpwwfOt2X-mhwU65Qa1OHDi--4OMGvQ_1
default.73886.55__shadow__RpwwfOt2X-mhwU65Qa1OHDi--4OMGvQ_3
default.73886.55_vmware-freebsd-tools.tar.gz
default.73886.55__shadow__RpwwfOt2X-mhwU65Qa1OHDi--4OMGvQ_2
default.73886.55__shadow__RpwwfOt2X-mhwU65Qa1OHDi--4OMGvQ_4


I know those shadow__RpwwfOt2X-mhwU65Qa1OHDi--4OMGvQ_ files are the
rest of vmware-freebsd-tools.tar.gz.  I can infer that because this
bucket only has a single file (and the sum of the sizes matches).
With many files, I can't infer the link anymore.

How do I look up that link?

I tried reading the src/rgw/rgw_rados.cc, but I'm getting lost.



My real goal is the reverse.  I recently repaired an inconsistent PG.
The primary replica had the bad data, so I want to verify that the
repaired object is correct.  I have a database that stores the SHA256
of every object.  If I can get from the filename on disk back to an S3
object, I can verify the file.  If it's bad, I can restore from the
replicated zone.


Aside from today's task, I think it's really handy to understand these
low level details.  I know it's been handy in the past, when I had
disk corruption under my PostgreSQL database.  Knowing (and
practicing) ahead of time really saved me a lot of downtime then.


Thanks for any pointers.

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2014-08-20 17:38 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-08-14 18:34 Translating a RadosGW object name into a filename on disk Craig Lewis
2014-08-19 17:27 ` Gregory Farnum
     [not found]   ` <CAPYLRzjaOZc+ZG4aibR18N6O3AW1JXU2440J+NSDHmxn4ircag-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-08-20  2:39     ` Craig Lewis
2014-08-20 17:25   ` Craig Lewis
     [not found]     ` <CADHZLBaAwXy2XUXnTP-dLwn8gKH1Oh+J9YoFkff5bTRs-xLhmw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-08-20 17:38       ` Sage Weil

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.