Supporting partial writes for EC backend

Supporting partial writes for EC backend

[Bassam Tabbara]

Hello,

I realize that the EC Backend does not support all operations like partial writes, truncate, and others. I’d like to understand whether adding the missing operations is something that is architecturally possible, or if this is a limitation of the architecture / approach. Can someone shed some light on this?

I realize that partial writes are going to be slow/inefficient on an EC backend compared to a replicated backend.

Thanks!
Bassam--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Supporting partial writes for EC backend

[Samuel Just]

Basically, it's an architectural choice:

https://github.com/ceph/ceph/blob/master/doc/dev/osd_internals/erasure_coding/pgbackend.rst#client-writes

If we wanted to support partial writes, we'd probably want to
introduce a second EC pool type with different tradeoffs.  Note, you
can get slow partial writes on an EC pool by placing a replicated
cache in front.
-Sam

On Fri, Jan 30, 2015 at 8:36 AM, Bassam Tabbara
<bassamtabbara@hotmail.com> wrote:
> Hello,
>
> I realize that the EC Backend does not support all operations like partial writes, truncate, and others. I’d like to understand whether adding the missing operations is something that is architecturally possible, or if this is a limitation of the architecture / approach. Can someone shed some light on this?
>
> I realize that partial writes are going to be slow/inefficient on an EC backend compared to a replicated backend.
>
> Thanks!
> Bassam--
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Supporting partial writes for EC backend

[Bassam Tabbara]

I’d like to explore adding a new EC pool type as you suggest. The idea is to have RBD sit on-top of a fully functional EC backend. Just as with RAID6 arrays today some applications are aware of the read-modify-write penalty and they could optimize for it. 

I’m new to ceph, so if you can point me in the right direction I would appreciate it. Do you think this is something that could be implemented as a modification to the existing ECBackend or is this a new backend. I’m happy to jump on a quick call if it makes things easier.

> On Jan 30, 2015, at 8:59 AM, Samuel Just <sam.just@inktank.com> wrote:
> 
> Basically, it's an architectural choice:
> 
> https://github.com/ceph/ceph/blob/master/doc/dev/osd_internals/erasure_coding/pgbackend.rst#client-writes
> 
> If we wanted to support partial writes, we'd probably want to
> introduce a second EC pool type with different tradeoffs.  Note, you
> can get slow partial writes on an EC pool by placing a replicated
> cache in front.
> -Sam
> 
> On Fri, Jan 30, 2015 at 8:36 AM, Bassam Tabbara
> <bassamtabbara@hotmail.com> wrote:
>> Hello,
>> 
>> I realize that the EC Backend does not support all operations like partial writes, truncate, and others. I’d like to understand whether adding the missing operations is something that is architecturally possible, or if this is a limitation of the architecture / approach. Can someone shed some light on this?
>> 
>> I realize that partial writes are going to be slow/inefficient on an EC backend compared to a replicated backend.
>> 
>> Thanks!
>> Bassam--
>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Supporting partial writes for EC backend

[Samuel Just]

This would need to be a new backend, and it's going to be a massive
amount of work.  You want to start by reading and understanding all of
the docs in osd_internals.  I also suspect you'll get a lot of what
you need from using a replicated cache in front of an EC pool.
-Sam

On Fri, Jan 30, 2015 at 10:00 AM, Bassam Tabbara
<bassamtabbara@hotmail.com> wrote:
> I’d like to explore adding a new EC pool type as you suggest. The idea is to have RBD sit on-top of a fully functional EC backend. Just as with RAID6 arrays today some applications are aware of the read-modify-write penalty and they could optimize for it.
>
> I’m new to ceph, so if you can point me in the right direction I would appreciate it. Do you think this is something that could be implemented as a modification to the existing ECBackend or is this a new backend. I’m happy to jump on a quick call if it makes things easier.
>
>> On Jan 30, 2015, at 8:59 AM, Samuel Just <sam.just@inktank.com> wrote:
>>
>> Basically, it's an architectural choice:
>>
>> https://github.com/ceph/ceph/blob/master/doc/dev/osd_internals/erasure_coding/pgbackend.rst#client-writes
>>
>> If we wanted to support partial writes, we'd probably want to
>> introduce a second EC pool type with different tradeoffs.  Note, you
>> can get slow partial writes on an EC pool by placing a replicated
>> cache in front.
>> -Sam
>>
>> On Fri, Jan 30, 2015 at 8:36 AM, Bassam Tabbara
>> <bassamtabbara@hotmail.com> wrote:
>>> Hello,
>>>
>>> I realize that the EC Backend does not support all operations like partial writes, truncate, and others. I’d like to understand whether adding the missing operations is something that is architecturally possible, or if this is a limitation of the architecture / approach. Can someone shed some light on this?
>>>
>>> I realize that partial writes are going to be slow/inefficient on an EC backend compared to a replicated backend.
>>>
>>> Thanks!
>>> Bassam--
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Supporting partial writes for EC backend

[Bassam Tabbara]

Thanks, I’ll take a look at osd_internals. 

Are you saying that a cache tier —> EC Pool is good enough? What if any is the downside to using a cache tier ontop of an EC pool? I assume that the size of cache is going to be important, and so is the migration algorithm between the cache tier and the backing tier.	

> On Jan 30, 2015, at 10:49 AM, Samuel Just <sam.just@inktank.com> wrote:
> 
> This would need to be a new backend, and it's going to be a massive
> amount of work.  You want to start by reading and understanding all of
> the docs in osd_internals.  I also suspect you'll get a lot of what
> you need from using a replicated cache in front of an EC pool.
> -Sam
> 
> On Fri, Jan 30, 2015 at 10:00 AM, Bassam Tabbara
> <bassamtabbara@hotmail.com> wrote:
>> I’d like to explore adding a new EC pool type as you suggest. The idea is to have RBD sit on-top of a fully functional EC backend. Just as with RAID6 arrays today some applications are aware of the read-modify-write penalty and they could optimize for it.
>> 
>> I’m new to ceph, so if you can point me in the right direction I would appreciate it. Do you think this is something that could be implemented as a modification to the existing ECBackend or is this a new backend. I’m happy to jump on a quick call if it makes things easier.
>> 
>>> On Jan 30, 2015, at 8:59 AM, Samuel Just <sam.just@inktank.com> wrote:
>>> 
>>> Basically, it's an architectural choice:
>>> 
>>> https://github.com/ceph/ceph/blob/master/doc/dev/osd_internals/erasure_coding/pgbackend.rst#client-writes
>>> 
>>> If we wanted to support partial writes, we'd probably want to
>>> introduce a second EC pool type with different tradeoffs.  Note, you
>>> can get slow partial writes on an EC pool by placing a replicated
>>> cache in front.
>>> -Sam
>>> 
>>> On Fri, Jan 30, 2015 at 8:36 AM, Bassam Tabbara
>>> <bassamtabbara@hotmail.com> wrote:
>>>> Hello,
>>>> 
>>>> I realize that the EC Backend does not support all operations like partial writes, truncate, and others. I’d like to understand whether adding the missing operations is something that is architecturally possible, or if this is a limitation of the architecture / approach. Can someone shed some light on this?
>>>> 
>>>> I realize that partial writes are going to be slow/inefficient on an EC backend compared to a replicated backend.
>>>> 
>>>> Thanks!
>>>> Bassam--
>>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> 

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html