Usage of dm-era

Usage of dm-era

[Markus Hentsch]

[-- Attachment #1.1: Type: text/plain, Size: 2513 bytes --]

Dear dm-devel community,

I'm trying to get familiar with dm-era but I'm stuck.

What I'm trying to achieve is to have a dm-era device which keeps track
of changed blocks on a seperate hard drive or logical volume/mapping in
order to create a live backup solution.

The documentation on dm-era is very sparse. So far I found a Gentoo wiki
entry, a blog entry by Rackcorp and the official kernel doc.
I tried to come up with a basic setup using a virtual machine running
Ubuntu 16.04.

So far I was able to create a setup that looks like this:
http://i.imgur.com/LNCPima.png
To set this up, I came up with the following script:
http://pastebin.com/Y3B2m7wL
It is inspired by the setup done by Rackcorp [1]. The resulting 'lsblk'
looks like this:

sdb                       8:16   0  256M  0 disk 

`-metadev-era           252:2    0  276K  0 dm   

  |-era                 252:4    0  276K  0 dm   

  `-era-access          252:5    0  276K  0 dm   

sdc                       8:32   0  256M  0 disk 

`-metadev-data          252:3    0  256M  0 dm   

  |-era                 252:4    0  276K  0 dm   

  `-metadev-data-access 252:6    0  256M  0 dm   /media/sdc-data



In comparison to Rackcorp's setup, I did the following adjustment:

  * i) mounting /dev/mapper/metadev-data (or the underlying sdc) or ii)
    using it as an origin device for era, seemed to be mutually exclusive
  * that's why I created an additional linear mapping called
    'metadev-data-access' inspired by the 'era-access' device to be able
    to mount and write to the sdc device again

Basically this setup is supposed to track changed blocks on /dev/sdc and
uses a mounted /dev/mapper/metadev-data-access at /media/sdc-data to
access /dev/sdc on top of the dm setup.

However, when reading the era metadata via 'era_dump
/dev/mapper/era-access', I only get this result:

<superblock uuid="" block_size="4096" nr_blocks="1" current_era="1">

  <era_array>

    <era block="0" era="0"/>

  </era_array>

</superblock>


Why is there only one block listed? Shouldn't that list exactly as much
blocks as my 'metadev-data' device has, since it is the origin for the
era target?
Or is there some basic misunderstanding in my interpretation of how
dm-era is supposed to work and be used?

Please be so kind and shed some light on this!

[1] http://blog.rackcorp.com/2016/03/dm-era-device-for-backups/


Best Regards,

Markus Hentsch
Cloud&Heat Technologies


[-- Attachment #1.2: Type: text/html, Size: 3508 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Usage of dm-era

[Markus Hentsch]

[-- Attachment #1.1: Type: text/plain, Size: 3869 bytes --]

Okay, I got it now. There was a misinterpretation of how to use the era
device on my side as well as an error in the Rackcorp blog post. The era
device has to be the same size as the origin not the metadata device.
Also the era device is supposed to be used in place of the underlying
block device (for read/write).
Now it's listing the correct amount of blocks in the metadata.


Finally I'm trying to find the correct calculation of the required disk
space for the era metadata.
The dm-era docs [2] state the following formula:

(4 * nr_blocks) bytes + buffers

 1. So I assume "nr_blocks" is refering to the amount of blocks tracked
    by the era device, i.e. the same amount of blocks as listed by the
    "era_dump" output?
 2. How big is "buffers" supposed to be? Is this what "it uses a few 4k
    blocks for updating metadata" [2] refers to? What are "a few 4k
    blocks" in this context? Several blocks of 4096 bytes or blocks
    consisting of 4k sectors, i.e. 4096 * 512 bytes each?


[2] https://www.kernel.org/doc/Documentation/device-mapper/era.txt


Best Regards,

Markus Hentsch
Cloud&Heat Technologies

> Dear dm-devel community,
>
> I'm trying to get familiar with dm-era but I'm stuck.
>
> What I'm trying to achieve is to have a dm-era device which keeps
> track of changed blocks on a seperate hard drive or logical
> volume/mapping in order to create a live backup solution.
>
> The documentation on dm-era is very sparse. So far I found a Gentoo
> wiki entry, a blog entry by Rackcorp and the official kernel doc.
> I tried to come up with a basic setup using a virtual machine running
> Ubuntu 16.04.
>
> So far I was able to create a setup that looks like this:
> http://i.imgur.com/LNCPima.png
> To set this up, I came up with the following script:
> http://pastebin.com/Y3B2m7wL
> It is inspired by the setup done by Rackcorp [1]. The resulting
> 'lsblk' looks like this:
> sdb                       8:16   0  256M  0 disk 
> `-metadev-era           252:2    0  276K  0 dm   
>   |-era                 252:4    0  276K  0 dm   
>   `-era-access          252:5    0  276K  0 dm   
> sdc                       8:32   0  256M  0 disk 
> `-metadev-data          252:3    0  256M  0 dm   
>   |-era                 252:4    0  276K  0 dm   
>   `-metadev-data-access 252:6    0  256M  0 dm   /media/sdc-data
>
>
> In comparison to Rackcorp's setup, I did the following adjustment:
>
>   * i) mounting /dev/mapper/metadev-data (or the underlying sdc) or
>     ii) using it as an origin device for era, seemed to be mutually
>     exclusive
>   * that's why I created an additional linear mapping called
>     'metadev-data-access' inspired by the 'era-access' device to be
>     able to mount and write to the sdc device again
>
> Basically this setup is supposed to track changed blocks on /dev/sdc
> and uses a mounted /dev/mapper/metadev-data-access at /media/sdc-data
> to access /dev/sdc on top of the dm setup.
>
> However, when reading the era metadata via 'era_dump
> /dev/mapper/era-access', I only get this result:
> <superblock uuid="" block_size="4096" nr_blocks="1" current_era="1">
>   <era_array>
>     <era block="0" era="0"/>
>   </era_array>
> </superblock>
>
> Why is there only one block listed? Shouldn't that list exactly as
> much blocks as my 'metadev-data' device has, since it is the origin
> for the era target?
> Or is there some basic misunderstanding in my interpretation of how
> dm-era is supposed to work and be used?
>
> Please be so kind and shed some light on this!
>
> [1] http://blog.rackcorp.com/2016/03/dm-era-device-for-backups/
>
>
> Best Regards,
>
> Markus Hentsch
> Cloud&Heat Technologies
>
>
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

[-- Attachment #1.2: Type: text/html, Size: 5594 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]