is it possible to support backend pluggable for RGW bucket index

is it possible to support backend pluggable for RGW bucket index

[liuchang0812]

Hi, Yehuda and Cephers

Is it possible to make RGW bucket index pluggable? If it's possible,
We could save those index into Hbase cluster and improve the
scalability and available.



thanks

Re: is it possible to support backend pluggable for RGW bucket index

[Matt Benjamin]

Hi,

I know you've suggested this concept in an earlier email.

Others should chime in, but my own impression is that there isn't a trivial way to substitute an alternative indexing mechanism, as the mapping is constrained by the mapping of index segments to OSDs.  Once mapped to an OSD, the index ops are OMAP operations, and there may be a variety of ways to accelerate those at the OSD.

It would be great to have you join an RGW standup (several per week) to discuss your ideas, and current ideas/ongoing work to accelerate bucket index ops (and bypass them).

Regards,

Matt

----- Original Message -----
> From: "liuchang0812" <liuchang0812@gmail.com>
> To: "ceph-devel" <ceph-devel@vger.kernel.org>
> Sent: Monday, February 27, 2017 3:44:27 AM
> Subject: is it possible to support backend pluggable for RGW bucket index
> 
> Hi, Yehuda and Cephers
> 
> Is it possible to make RGW bucket index pluggable? If it's possible,
> We could save those index into Hbase cluster and improve the
> scalability and available.
> 
> 
> 
> thanks
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

-- 
Matt Benjamin
Red Hat, Inc.
315 West Huron Street, Suite 140A
Ann Arbor, Michigan 48103

http://www.redhat.com/en/technologies/storage

tel.  734-821-5101
fax.  734-769-8938
cel.  734-216-5309

Re: is it possible to support backend pluggable for RGW bucket index

[liuchang0812]

Hi, Matt

I'm very honored to join RGW standup and share my ideas. We are trying
to use RGW in our service, but bucket index limits cluster scalability
and availability. I have some ideas about bucket index, and wrote a
document(but in Chinese).  There are:

1. two-level prefix-shard based on current hash-shard
2. pre-defined range shard
3. abstract Bucket Index operations and support more back-end.

I am very pleasure to prepare a slide to discuss those ideas.

Thanks

Chang Liu