How big should raid metadata device be?

How big should raid metadata device be?

[Andy Grover]

Hi all,

In looking at md-raid one question I had was exactly how large does the 
metadata device need to be in comparison to the data device? It contains 
a padded-to-4Kbyte sized raid metadata struct, and then also a write 
intent bitmap that is sized based upon the number of stripes, with 16 
bits per stripe.

Is it correct to say the minimum size of the metadata device should be:

4096 + (2 * stripe_count)

bytes?

Thanks -- Andy

p.s. LVM2 allocates 1 extent (default 4MiB) for the raid meta device. 
But it's not clear if given a large enough raid dev, and configured with 
a much smaller extent size than the default, if one extent might not be 
enough.

Re: How big should raid metadata device be?

[Heinz Mauelshagen]

On 11/02/2015 08:55 PM, Andy Grover wrote:
> Hi all,
>
> In looking at md-raid one question I had was exactly how large does 
> the metadata device need to be in comparison to the data device? It 
> contains a padded-to-4Kbyte sized raid metadata struct, and then also 
> a write intent bitmap that is sized based upon the number of stripes, 
> with 16 bits per stripe.

There is no direct relationship between stripes and bitmap size, because the
bitmap holds a bit per region.

I'm restricting this to metadata kept by dm-raid (neither md/md raid1 
clustered or md with external metadata),
because you seem to utilize that:

the metadata contains a 4KiB padded struct to describe the array (the 
raid superblock) at offset 0
followed by 4KiB for the bitmap header. After those 2, a bit per region 
(i.e. blocks / region_size) is allocated.

Say, for 64KiB region size and 1TiB raid1 set size (1TiB*Bit / 64 KiB = 
16777216 Bit; 16777216 Bit / (8 Bit/Byte) / (1024 Byte/KiB) = 2048 KiB:

4KiB superblock
4KiB bitmap header
2048 KiB bitmap
-------------------------
2056KiB total

>
> Is it correct to say the minimum size of the metadata device should be:
>
> 4096 + (2 * stripe_count)
>
> bytes?
>
> Thanks -- Andy
>
> p.s. LVM2 allocates 1 extent (default 4MiB) for the raid meta device. 
> But it's not clear if given a large enough raid dev, and configured 
> with a much smaller extent size than the default, if one extent might 
> not be enough.

It is actually very clear that in case of small lvm2 extent and raid 
region size,
one extent is easily not big enough.
That's why I am developing dynamic raid metadata allocation for lvm2.

Heinz

>
> -- 
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

Re: How big should raid metadata device be?

[Andy Grover]

On 11/02/2015 05:01 PM, Heinz Mauelshagen wrote:
> On 11/02/2015 08:55 PM, Andy Grover wrote:
>> Hi all,
>>
>> In looking at md-raid one question I had was exactly how large does
>> the metadata device need to be in comparison to the data device? It
>> contains a padded-to-4Kbyte sized raid metadata struct, and then also
>> a write intent bitmap that is sized based upon the number of stripes,
>> with 16 bits per stripe.
>
> There is no direct relationship between stripes and bitmap size, because
> the
> bitmap holds a bit per region.

Ah ok. I was confused because what dm-raid calls a chunk, mdraid calls a 
stripe, and what dm-raid calls a region, bitmap.c calls a chunk :-)

<snip example md size calculation>

I added some explanation to your size calculation and formatted it to be 
suitable to add to Documentation/device-mapper/dm-raid.txt:

The Metadata Device
-------------------
While users of dm-raid need not parse or access the contents of the
metadata device, they must still ensure it is of a certain minimum
size. Fixed-size metadata consumes 8KiB, and then one extra bit must
be allocated for each region in the raid device.

For example:

With a 64KiB region size and 1TiB raid1 set size (1TiB*Bit / 64 KiB =
16777216 Bit; 16777216 Bit / (8 Bit/Byte) / (1024 Byte/KiB) = 2048 KiB

      8 KiB fixed-size metadata
+ 2048 KiB region bitmap
-------------------------
   2056 KiB total

A device must contain fewer than 2^21 regions. With the default region
size of 4MiB this limit is reached at 8TiB. Therefore, raid devices
larger than that will need to use a larger region size, set with the
"region_size" parameter.

(end)

What do you think?

Anything else to add or change in dm-raid.txt while we're at it?

Thanks -- Regards -- Andy