osd pool erasure code stripe width

osd pool erasure code stripe width

[Deneau, Tom]

I am trying to understand the use of "osd pool erasure code stripe width"
For example, I have a single-node system with a k=2,m=1 ec pool
and I write a single 40M object to this pool using rados bench.
But when I look on the disk, I still see only the 3 20M pieces for this object.
Where does the striping get used?

-- Tom Deneau, AMD


Description:
Sets the desired size, in bytes, of an object stripe on every erasure coded pools. Every object if size S will be stored as N stripes and each stripe will be encoded/decoded individually.
Type:
Unsigned 32-bit Integer
Default:
4096


--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in

Re: osd pool erasure code stripe width

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 959 bytes --]

Hi Tom,

A stripe width of 4KB (the default) means the object is encoded 4KB at a time. It does not show in the file written to disk.

Cheers

On 19/06/2015 22:11, Deneau, Tom wrote:
> I am trying to understand the use of "osd pool erasure code stripe width"
> For example, I have a single-node system with a k=2,m=1 ec pool
> and I write a single 40M object to this pool using rados bench.
> But when I look on the disk, I still see only the 3 20M pieces for this object.
> Where does the striping get used?
> 
> -- Tom Deneau, AMD
> 
> 
> Description:
> Sets the desired size, in bytes, of an object stripe on every erasure coded pools. Every object if size S will be stored as N stripes and each stripe will be encoded/decoded individually.
> Type:
> Unsigned 32-bit Integer
> Default:
> 4096
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

RE: osd pool erasure code stripe width

[Deneau, Tom]

So if I am looking at the performance of a system with
ceph_erasure_code_benchmark, does this mean I should concentrate
on the performance with --size 4096 (or whatever stripe width my
ec pools are going to use)?

-- Tom

> -----Original Message-----
> From: Loic Dachary [mailto:loic@dachary.org]
> Sent: Friday, June 19, 2015 5:47 PM
> To: Deneau, Tom; ceph-devel
> Subject: Re: osd pool erasure code stripe width
> 
> Hi Tom,
> 
> A stripe width of 4KB (the default) means the object is encoded 4KB at a
> time. It does not show in the file written to disk.
> 
> Cheers
> 
> On 19/06/2015 22:11, Deneau, Tom wrote:
> > I am trying to understand the use of "osd pool erasure code stripe width"
> > For example, I have a single-node system with a k=2,m=1 ec pool and I
> > write a single 40M object to this pool using rados bench.
> > But when I look on the disk, I still see only the 3 20M pieces for this
> object.
> > Where does the striping get used?
> >
> > -- Tom Deneau, AMD
> >
> >
> > Description:
> > Sets the desired size, in bytes, of an object stripe on every erasure coded
> pools. Every object if size S will be stored as N stripes and each stripe
> will be encoded/decoded individually.
> > Type:
> > Unsigned 32-bit Integer
> > Default:
> > 4096
> >
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe ceph-devel"
> > in
> >
> 
> --
> Loïc Dachary, Artisan Logiciel Libre

--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in

Re: osd pool erasure code stripe width

[Loic Dachary]

[-- Attachment #1: Type: text/plain, Size: 1756 bytes --]

Hi Tom,

On 20/06/2015 03:18, Deneau, Tom wrote:
> So if I am looking at the performance of a system with
> ceph_erasure_code_benchmark, does this mean I should concentrate
> on the performance with --size 4096 (or whatever stripe width my
> ec pools are going to use)?

Yes. But the erasure code plugin encoding speed is not the only factor. You probably want to try a rados bench instead to get more realistic numbers.

Cheers

> 
> -- Tom
> 
>> -----Original Message-----
>> From: Loic Dachary [mailto:loic@dachary.org]
>> Sent: Friday, June 19, 2015 5:47 PM
>> To: Deneau, Tom; ceph-devel
>> Subject: Re: osd pool erasure code stripe width
>>
>> Hi Tom,
>>
>> A stripe width of 4KB (the default) means the object is encoded 4KB at a
>> time. It does not show in the file written to disk.
>>
>> Cheers
>>
>> On 19/06/2015 22:11, Deneau, Tom wrote:
>>> I am trying to understand the use of "osd pool erasure code stripe width"
>>> For example, I have a single-node system with a k=2,m=1 ec pool and I
>>> write a single 40M object to this pool using rados bench.
>>> But when I look on the disk, I still see only the 3 20M pieces for this
>> object.
>>> Where does the striping get used?
>>>
>>> -- Tom Deneau, AMD
>>>
>>>
>>> Description:
>>> Sets the desired size, in bytes, of an object stripe on every erasure coded
>> pools. Every object if size S will be stored as N stripes and each stripe
>> will be encoded/decoded individually.
>>> Type:
>>> Unsigned 32-bit Integer
>>> Default:
>>> 4096
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel"
>>> in
>>>
>>
>> --
>> Loïc Dachary, Artisan Logiciel Libre
> 

-- 
Loïc Dachary, Artisan Logiciel Libre


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 198 bytes --]