About deleting container/bucket from radosgw

About deleting container/bucket from radosgw

[蔡權昱]

Hello everyone,

I try to use radosgw to supply s3/swift storage,

everything is fine, but I found that something strange

after deleting a container/bucket from radosgw,

the following are commands I ran:

1. check the pool empty or not
$ rados --pool=.rgw ls | grep -v "^\." ; echo ============; rados
--pool=.rgw.buckets ls
============

2. create a container "buck1"
$ swift -A http://volume/auth -U account -K key post buck1

3. show created objs in ceph
$ rados --pool=.rgw ls | grep -v "^\." ; echo ============; rados
--pool=.rgw.buckets ls
9352.10
buck1
============
.dir.9352.10

4. delete "buck1"
swift -A http://volume/auth -U account -K key delete buck1

5. show objs in ceph
$ rados --pool=.rgw ls | grep -v "^\." ; echo ============; rados
--pool=.rgw.buckets ls
9352.10

Then, the object '9352.10' seems like be leaved in pool forever ?
I have tried create & delete bucket by s3lib, too,
and the result is the same.

However, the function works fine,
I just want to know is this a normal case?

Thanks.

--
Chuanyu Tsai

Re: About deleting container/bucket from radosgw

[Yehuda Sadeh]

On Sat, Jul 14, 2012 at 6:45 AM, 蔡權昱 <chuanyu@cs.nctu.edu.tw> wrote:
> Hello everyone,
>
> I try to use radosgw to supply s3/swift storage,
>
> everything is fine, but I found that something strange
>
> after deleting a container/bucket from radosgw,
>
> the following are commands I ran:
>
> 1. check the pool empty or not
> $ rados --pool=.rgw ls | grep -v "^\." ; echo ============; rados
> --pool=.rgw.buckets ls
> ============
>
> 2. create a container "buck1"
> $ swift -A http://volume/auth -U account -K key post buck1
>
> 3. show created objs in ceph
> $ rados --pool=.rgw ls | grep -v "^\." ; echo ============; rados
> --pool=.rgw.buckets ls
> 9352.10
> buck1
> ============
> .dir.9352.10
>
> 4. delete "buck1"
> swift -A http://volume/auth -U account -K key delete buck1
>
> 5. show objs in ceph
> $ rados --pool=.rgw ls | grep -v "^\." ; echo ============; rados
> --pool=.rgw.buckets ls
> 9352.10
>
> Then, the object '9352.10' seems like be leaved in pool forever ?
> I have tried create & delete bucket by s3lib, too,
> and the result is the same.
>
> However, the function works fine,
> I just want to know is this a normal case?
>

This is normal, though we modified this behavior recently, so you
shouldn't see that in the next versions. We used to keep the bucket
index by bucket instance id for archival and also for usage
processing.

Yehuda
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: About deleting container/bucket from radosgw

[蔡權昱]

2012/7/14 Yehuda Sadeh <yehuda@inktank.com>:
> On Sat, Jul 14, 2012 at 6:45 AM, 蔡權昱 <chuanyu@cs.nctu.edu.tw> wrote:
>> Hello everyone,
>>
>> I try to use radosgw to supply s3/swift storage,
>>
>> everything is fine, but I found that something strange
>>
>> after deleting a container/bucket from radosgw,
>>
>> the following are commands I ran:
>>
>> 1. check the pool empty or not
>> $ rados --pool=.rgw ls | grep -v "^\." ; echo ============; rados
>> --pool=.rgw.buckets ls
>> ============
>>
>> 2. create a container "buck1"
>> $ swift -A http://volume/auth -U account -K key post buck1
>>
>> 3. show created objs in ceph
>> $ rados --pool=.rgw ls | grep -v "^\." ; echo ============; rados
>> --pool=.rgw.buckets ls
>> 9352.10
>> buck1
>> ============
>> .dir.9352.10
>>
>> 4. delete "buck1"
>> swift -A http://volume/auth -U account -K key delete buck1
>>
>> 5. show objs in ceph
>> $ rados --pool=.rgw ls | grep -v "^\." ; echo ============; rados
>> --pool=.rgw.buckets ls
>> 9352.10
>>
>> Then, the object '9352.10' seems like be leaved in pool forever ?
>> I have tried create & delete bucket by s3lib, too,
>> and the result is the same.
>>
>> However, the function works fine,
>> I just want to know is this a normal case?
>>
>
> This is normal, though we modified this behavior recently, so you
> shouldn't see that in the next versions. We used to keep the bucket
> index by bucket instance id for archival and also for usage
> processing.
OK, I see.

On the other hand, why radosgw-admin doesn't supply "bucket rm" call?
When I want to "user rm" a user, it seems like I need to "bucket unlink"
all buckets belongs to the user, but the buckets are still in there!
Then, the other users can not create the same name of bucket
anymore(by s3/swift).

So, Is this the only way I can do by using s3/swift client to delete buckets
which belongs to that user before "radosgw-admin user rm" ?

Can I "rm" all buckets, and "rm" the user only by radosgw-admin?

Thanks for your help!

Chuanyu
>
> Yehuda
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: About deleting container/bucket from radosgw

[Yehuda Sadeh]

On Sat, Jul 14, 2012 at 8:04 AM, 蔡權昱 <chuanyu@cs.nctu.edu.tw> wrote:
...
>
> On the other hand, why radosgw-admin doesn't supply "bucket rm" call?
> When I want to "user rm" a user, it seems like I need to "bucket unlink"
> all buckets belongs to the user, but the buckets are still in there!
> Then, the other users can not create the same name of bucket
> anymore(by s3/swift).

Yeah, currently radosgw-admin doesn't provide a way to remove the
entire content of the bucket. I opened an issue for that (#2786), and
it ties into another issue also (#2499: the ability to remove user
without removing its data first). I think we can provide tools for
manual data removal, but we may also want to explore doing that as
part of a bigger garbage collection scheme that we'll soon be working
on.
>
> So, Is this the only way I can do by using s3/swift client to delete buckets
> which belongs to that user before "radosgw-admin user rm" ?
>
> Can I "rm" all buckets, and "rm" the user only by radosgw-admin?

You can unlink the buckets and link them to a different user (so that
you don't just leak the data), and then remove the user.

Yehuda
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: About deleting container/bucket from radosgw

[蔡權昱]

2012/7/15 Yehuda Sadeh <yehuda@inktank.com>:
> On Sat, Jul 14, 2012 at 8:04 AM, 蔡權昱 <chuanyu@cs.nctu.edu.tw> wrote:
> ...
>>
>> On the other hand, why radosgw-admin doesn't supply "bucket rm" call?
>> When I want to "user rm" a user, it seems like I need to "bucket unlink"
>> all buckets belongs to the user, but the buckets are still in there!
>> Then, the other users can not create the same name of bucket
>> anymore(by s3/swift).
>
> Yeah, currently radosgw-admin doesn't provide a way to remove the
> entire content of the bucket. I opened an issue for that (#2786), and
> it ties into another issue also (#2499: the ability to remove user
> without removing its data first). I think we can provide tools for
> manual data removal, but we may also want to explore doing that as
> part of a bigger garbage collection scheme that we'll soon be working
> on.
It sounds good that rgw will have garbage collection!
>>
>> So, Is this the only way I can do by using s3/swift client to delete buckets
>> which belongs to that user before "radosgw-admin user rm" ?
>>
>> Can I "rm" all buckets, and "rm" the user only by radosgw-admin?
>
> You can unlink the buckets and link them to a different user (so that
> you don't just leak the data), and then remove the user.
OK, it's a good approach now, thanks for your advice :)

Chuanyu
>
> Yehuda



-- 
蔡權昱 Tsai Chuanyu
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html