device mapper mapping across reboot

device mapper mapping across reboot

[jitendra kumar khasdev]

[-- Attachment #1.1: Type: text/plain, Size: 298 bytes --]

Hi All,

I wrote the basic target using device mapper framework. Using dmsetup
utility I am able to create device node under /dev/mapper/my_custom_mapper.

But when I do reboot the system, it lost the mapping. Is there any way by
which I can create mapping during boot like LVM does.

---
Jitendra

[-- Attachment #1.2: Type: text/html, Size: 404 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: device mapper mapping across reboot

[Zdenek Kabelac]

Dne 9.3.2018 v 10:58 jitendra kumar khasdev napsal(a):
> Hi All,
> 
> I wrote the basic target using device mapper framework. Using dmsetup utility 
> I am able to create device node under /dev/mapper/my_custom_mapper.
> 
> But when I do reboot the system, it lost the mapping. Is there any way by 
> which I can create mapping during boot like LVM does.
> 

Hi

lvm2 is exactly solving this problem as it maintains consistent 'metadata' on 
every device - so upon reboot devices are discovered and from their metadata 
dm tables are actived/restored.

So  are you looking for recreation of all the lvm2 infrastructure for this 
relatively quite complex task ?

Or you just want to 'create' DM after kernel is booted ?

Or you even want to pass 'DM' table line on kernel boot option line - so even 
your boot device is a 'DM' device ?


It's unclear which problem are you actually solving so please be more specific.


Regards

Zdenek

Re: device mapper mapping across reboot

[Jitendra]

>lvm2 is exactly solving this problem as it maintains consistent
>'metadata' on every device - so upon reboot devices are discovered and
>from their metadata dm tables are actived/restored.

Got it.

>So  are you looking for recreation of all the lvm2 infrastructure for
>this relatively quite complex task ?
Not exactly, but I want to create mapper device created before the root and
other fs get mounted so that I can track I/Os in target.

>Or you just want to 'create' DM after kernel is booted ?
dmsetup create with my target.

>Or you even want to pass 'DM' table line on kernel boot option line -
>so even your boot device is a 'DM' device ?

It is something exactly, I wanna to do.

eg. I wrote a basic target as explained here
http://techgmm.blogspot.in/p/writing-your-own-device-mapper-target.html

Now, to use this target (kernel module), I need to create mapper device as
echo 0 <size_of_device> basic_target /Path/to/your/device 0 | dmsetup create my_basic_dm_device

After creation of device as /dev/mapper/my_basic_dm_device for /dev/sda, if I do I/O from
/dev/mapper/my_basic_dm_device, then all I/O goes through basic_target before it
hits to /dev/sda.

Now, if system is booted and disk is offline then it is very easy to create
mapper device. But now let suppose I want to boot on
/dev/mapper/my_basic_dm_device instead of /dev/sda1 etc. then I have to create
mapper device even before it get switch root.

This is same case for other disks as well. So there should be a way so that I
can use device mapper framework after reboot.

---
Jitendra

Re: device mapper mapping across reboot

[Zdenek Kabelac]

Dne 10.3.2018 v 11:47 Jitendra napsal(a):
> 
> 
>> lvm2 is exactly solving this problem as it maintains consistent
>> 'metadata' on every device - so upon reboot devices are discovered and
>> from their metadata dm tables are actived/restored.
> 
> Got it.
> 
>> So  are you looking for recreation of all the lvm2 infrastructure for
>> this relatively quite complex task ?
> Not exactly, but I want to create mapper device created before the root and
> other fs get mounted so that I can track I/Os in target.
> 
>> Or you just want to 'create' DM after kernel is booted ?
> dmsetup create with my target.
> 
>> Or you even want to pass 'DM' table line on kernel boot option line -
>> so even your boot device is a 'DM' device ?
> 
> It is something exactly, I wanna to do.
> 
> eg. I wrote a basic target as explained here
> http://techgmm.blogspot.in/p/writing-your-own-device-mapper-target.html
> 
> Now, to use this target (kernel module), I need to create mapper device as
> echo 0 <size_of_device> basic_target /Path/to/your/device 0 | dmsetup create 
> my_basic_dm_device
> 
> After creation of device as /dev/mapper/my_basic_dm_device for /dev/sda, if I 
> do I/O from
> /dev/mapper/my_basic_dm_device, then all I/O goes through basic_target before it
> hits to /dev/sda.
> 
> Now, if system is booted and disk is offline then it is very easy to create
> mapper device. But now let suppose I want to boot on
> /dev/mapper/my_basic_dm_device instead of /dev/sda1 etc. then I have to create
> mapper device even before it get switch root.
> 
> This is same case for other disks as well. So there should be a way so that I
> can use device mapper framework after reboot.

Hi

Well in general - that's why every distribution is using init ramdisk.

This ramdisk is loaded from a small /boot partition together with kernel.
As you can see - you simply always need something to boot from - you cannot 
load your kernel from "DM" device.

Once kernel is booted and 'ramdisk' is processed - you have plenty of time and 
lots of binaries there (typically with lvm2 built-in)  - so here you can 
safely activate your root volume being on dm device (even without lvm2 just by 
issuing couple 'dmsetup' commands)

I'm aware there is some  Android person who proposed some booting into DM 
without initramfs - i.e.

https://www.redhat.com/archives/linux-lvm/2017-May/msg00055.html

I'm not sure in which stage this patch is - likely still not upstream thought, 
but there was some progress around this.

Still it's worth to say -  not using ramdisk is pretty much bogus idea - you 
should always use ramdisk - otherwise you are missing lots of important tools 
i.e. for validation of filesystem  before it gets mounted - so whoever wants 
to use 'DM'  to mount  rootfs  without initramfs is doing something wrong.

Anyway - since it's still unclear what is your actually work - it's really 
hard to give you proper advice here.

Regards

Zdenek

Re: device mapper mapping across reboot

[Jitendra]

On Mon, Mar 12, 2018 at 10:49:02AM +0100, Zdenek Kabelac wrote:
>Dne 10.3.2018 v 11:47 Jitendra napsal(a):
>>
>>
>>>lvm2 is exactly solving this problem as it maintains consistent
>>>'metadata' on every device - so upon reboot devices are discovered and
>>>from their metadata dm tables are actived/restored.
>>
>>Got it.
>>
>>>So  are you looking for recreation of all the lvm2 infrastructure for
>>>this relatively quite complex task ?
>>Not exactly, but I want to create mapper device created before the root and
>>other fs get mounted so that I can track I/Os in target.
>>
>>>Or you just want to 'create' DM after kernel is booted ?
>>dmsetup create with my target.
>>
>>>Or you even want to pass 'DM' table line on kernel boot option line -
>>>so even your boot device is a 'DM' device ?
>>
>>It is something exactly, I wanna to do.
>>
>>eg. I wrote a basic target as explained here
>>http://techgmm.blogspot.in/p/writing-your-own-device-mapper-target.html
>>
>>Now, to use this target (kernel module), I need to create mapper device as
>>echo 0 <size_of_device> basic_target /Path/to/your/device 0 |
>>dmsetup create my_basic_dm_device
>>
>>After creation of device as /dev/mapper/my_basic_dm_device for
>>/dev/sda, if I do I/O from
>>/dev/mapper/my_basic_dm_device, then all I/O goes through basic_target before it
>>hits to /dev/sda.
>>
>>Now, if system is booted and disk is offline then it is very easy to create
>>mapper device. But now let suppose I want to boot on
>>/dev/mapper/my_basic_dm_device instead of /dev/sda1 etc. then I have to create
>>mapper device even before it get switch root.
>>
>>This is same case for other disks as well. So there should be a way so that I
>>can use device mapper framework after reboot.
>
>Hi
>
>Well in general - that's why every distribution is using init ramdisk.
>
>This ramdisk is loaded from a small /boot partition together with kernel.
>As you can see - you simply always need something to boot from - you
>cannot load your kernel from "DM" device.

Got it.

>Once kernel is booted and 'ramdisk' is processed - you have plenty of
>time and lots of binaries there (typically with lvm2 built-in)  - so
>here you can safely activate your root volume being on dm device (even
>without lvm2 just by issuing couple 'dmsetup' commands)

This is something which I have tried in following steps.

1. First load my target driver(eg. my_custom_target) in initramfs (RHEL 7) using dracuf.
2. Now  my_custom_target will be available when initramfs get loaded.
3. Before the switch_root, I tried to run following command as,

	[root@localhost ~]# ls -l /usr/lib/systemd/system/initrd-switch-root.service
	-rw-r--r--. 1 root root 737 Mar 12 07:00 /usr/lib/systemd/system/initrd-switch-root.service

	In above file, I have added  dmsetup command to create
	/dev/mapper/my_device as,

	# we have to use "--force" here, otherwise systemd would umount /run
	ExecStart=/usr/bin/echo 0 24035328 my_custom_target /dev/sda 0|/usr/sbin/dmsetup create my_device
	ExecStart=/usr/bin/systemctl --no-block --force switch-root /sysroot
	KillMode=none

But it does not have any effect, after booting I could not see
/dev/mapper/my_device.


>I'm aware there is some  Android person who proposed some booting into
>DM without initramfs - i.e.
>
>https://www.redhat.com/archives/linux-lvm/2017-May/msg00055.html
>
>I'm not sure in which stage this patch is - likely still not upstream
>thought, but there was some progress around this.
I have looked into it. Since I am workig with older kernel version (3.10) so
these patches wont help although it has some really good pointers.

>Still it's worth to say -  not using ramdisk is pretty much bogus idea
>- you should always use ramdisk - otherwise you are missing lots of
>important tools i.e. for validation of filesystem  before it gets
>mounted - so whoever wants to use 'DM'  to mount  rootfs  without
>initramfs is doing something wrong.

This is exactly same understanding of mine as well, but I tried to create mapper
device using dmsetup but no luck.
>
>Anyway - since it's still unclear what is your actually work - it's
>really hard to give you proper advice here.

In my target driver, I am tracking each and every I/O and looking for offset and
length in each bio and doing some logging stuff.

---
Jitendra

Re: device mapper mapping across reboot

[Zdenek Kabelac]

Dne 12.3.2018 v 12:09 Jitendra napsal(a):
> On Mon, Mar 12, 2018 at 10:49:02AM +0100, Zdenek Kabelac wrote:
>> Dne 10.3.2018 v 11:47 Jitendra napsal(a):
>>>
> 
>> Once kernel is booted and 'ramdisk' is processed - you have plenty of
>> time and lots of binaries there (typically with lvm2 built-in)  - so
>> here you can safely activate your root volume being on dm device (even
>> without lvm2 just by issuing couple 'dmsetup' commands)
> 
> This is something which I have tried in following steps.
> 
> 1. First load my target driver(eg. my_custom_target) in initramfs (RHEL 7) 
> using dracuf.
> 2. Now  my_custom_target will be available when initramfs get loaded.
> 3. Before the switch_root, I tried to run following command as,
> 
>      [root@localhost ~]# ls -l /usr/lib/systemd/system/initrd-switch-root.service
>      -rw-r--r--. 1 root root 737 Mar 12 07:00 
> /usr/lib/systemd/system/initrd-switch-root.service
> 

I'm not quite sure why do you ever need to play with any systemd service s

Is there lost support for some kind of /etc/rc.local ?

Of course you have to use UUID on your devices - and shift filesystems on your 
real /dev/sdaX  so  systemd  WAIT for devices that will appear in your system 
only via  'DM' mapping (so your rootfs on sdX needs to be prefixed with some 
header).

You cannot have in your system 2 'same' devices' - that will confuse your 
systemd.

Solving complexity of systemd world is where lvm2 is there for. So if you plan 
to 'recreate' simpler lvm2 just because you don't want to use lvm2 - it's 
going to be hard task...

> In my target driver, I am tracking each and every I/O and looking for offset and
> length in each bio and doing some logging stuff.

Btw - isn't  dmstats  good enough for that already? You can split every device 
in your system in regions and get very detailed start about that.

For using dmstats - you can directly use LVs and attach dmstats to them - so 
you don't have to recreate this machinery yourself...

Regards

Zdenek