RAID 1 | Test Booting from /dev/sdb

RAID 1 | Test Booting from /dev/sdb

[Stefanie Leisestreichler]

Hi.
I want to test if grub is installed on both of the HDs which are part of 
my raid1 array. I wonder which would be the best solution to do so.

I think I will archive that when I shutdown the computer, make /dev/sda 
powerless and see if it is able to boot from /dev/sdb.

If it is not booting /dev/sdb will have no changes, I would shutdown, 
connect /dev/sda with power again, turn it on and do a "grub-install 
/dev/sdb". Depending on the state of the array (I guess it will need 
recovery) I would do a "mdadm /dev/md0 --add /dev/sdb1". After recovery 
I would try it again.

If it is booting from /dev/sdb this HD will have "more" data because of 
the one boot process than /dev/sda. I am not sure if it is a good idea 
to shutdown and just connect /dev/sda with power again, boot (assuming 
/dev/sda is the standard boot medium) because I do not know in which 
state the array will be. What to do in case I do not want to loose data 
from the last boot process with /dev/sdb? Change boot medium to /dev/sdb 
and do a "mdadm /dev/md0 --add /dev/sda1" to get it recovered again 
without loosing the "added" data (i.e. in /var/log) from booting? Also 
device identifiers could change I guess. Even if I am fine with loosing 
the "added" data from booting with /dev/sdb, will - when booting again 
from /dev/sda - /dev/sda be the master in the array again?

It is not clear to me if I understood correctly in which case which 
array member will be the master which will be the base for recovery. Is 
it always the HD one booted from?

Could you please help me with that?

Thanks,
Steffi

Re: RAID 1 | Test Booting from /dev/sdb

[Reindl Harald]

Am 05.05.20 um 10:11 schrieb Stefanie Leisestreichler:
> I want to test if grub is installed on both of the HDs which are part of
> my raid1 array. I wonder which would be the best solution to do so.
> 
> I think I will archive that when I shutdown the computer, make /dev/sda
> powerless and see if it is able to boot from /dev/sdb.
> 
> If it is not booting /dev/sdb will have no changes, I would shutdown,
> connect /dev/sda with power again, turn it on and do a "grub-install
> /dev/sdb". Depending on the state of the array (I guess it will need
> recovery) I would do a "mdadm /dev/md0 --add /dev/sdb1". After recovery
> I would try it again.

grub is *before* the RAID partitions and you need to run "grub2-install
/dev/sda" as well as "grub2-install /dev/sdb"

and you have to repeat that every time a disk fails and is replaced

the boot loader is in front of the first partition and that was the
reason a few years ago that you had to make sure there are 2 MB instead
of the old 512 KB by switch to grub2 on machines dating back to 2008 or
so where the default was smaller

> If it is booting from /dev/sdb this HD will have "more" data because of
> the one boot process than /dev/sda. I am not sure if it is a good idea
> to shutdown and just connect /dev/sda with power again, boot (assuming
> /dev/sda is the standard boot medium) because I do not know in which
> state the array will be. What to do in case I do not want to loose data
> from the last boot process with /dev/sdb? Change boot medium to /dev/sdb
> and do a "mdadm /dev/md0 --add /dev/sda1" to get it recovered again
> without loosing the "added" data (i.e. in /var/log) from booting? Also
> device identifiers could change I guess. Even if I am fine with loosing
> the "added" data from booting with /dev/sdb, will - when booting again
> from /dev/sda - /dev/sda be the master in the array again?
> 
> It is not clear to me if I understood correctly in which case which
> array member will be the master which will be the base for recovery. Is
> it always the HD one booted from?
> 
> Could you please help me with that?

you shouldn't remove disks and put them back without a good reason and
verify booting is not a good one

just change the boot order, switch the cables or explicit select both
disks in BIOS

Re: RAID 1 | Test Booting from /dev/sdb

[Wols Lists]

On 05/05/20 09:11, Stefanie Leisestreichler wrote:
> Hi.
> I want to test if grub is installed on both of the HDs which are part of
> my raid1 array. I wonder which would be the best solution to do so.
> 
> I think I will archive that when I shutdown the computer, make /dev/sda
> powerless and see if it is able to boot from /dev/sdb.
> 
> If it is not booting /dev/sdb will have no changes, I would shutdown,
> connect /dev/sda with power again, turn it on and do a "grub-install
> /dev/sdb". Depending on the state of the array (I guess it will need
> recovery) I would do a "mdadm /dev/md0 --add /dev/sdb1". After recovery
> I would try it again.
> 
> If it is booting from /dev/sdb this HD will have "more" data because of
> the one boot process than /dev/sda.

Why? If it boots to the grub menu, just hit an arrow key to stop the
timeout, and then hold the power button for 5 secs to shut it down again.

> I am not sure if it is a good idea
> to shutdown and just connect /dev/sda with power again, boot (assuming
> /dev/sda is the standard boot medium) because I do not know in which
> state the array will be. What to do in case I do not want to loose data
> from the last boot process with /dev/sdb?

Which would matter why? If you're worried about downloading mail etc,
just kill the network temporarily.

> Change boot medium to /dev/sdb
> and do a "mdadm /dev/md0 --add /dev/sda1" to get it recovered again
> without loosing the "added" data (i.e. in /var/log) from booting? Also
> device identifiers could change I guess. Even if I am fine with loosing
> the "added" data from booting with /dev/sdb, will - when booting again
> from /dev/sda - /dev/sda be the master in the array again?
> 
> It is not clear to me if I understood correctly in which case which
> array member will be the master which will be the base for recovery. Is
> it always the HD one booted from?
> 
The "master" if recovery is required will be the "older" one - in this
case sda because it was disconnected. HOWEVER. Just check whether you
have a bitmap or journal enabled. You can't have both at once, but the
result should be that sda rejoins the array cleanly, raid has a record
of which writes occurred while it was offline, and it will be updated.

> Could you please help me with that?
> 
> Thanks,
> Steffi

Cheers,
Wol

Re: RAID 1 | Test Booting from /dev/sdb

[Stefanie Leisestreichler]

On 05.05.20 10:39, Reindl Harald wrote:
> you shouldn't remove disks and put them back without a good reason and
> verify booting is not a good one
> 
> just change the boot order, switch the cables or explicit select both
> disks in BIOS

Very helpfull again, will check if booting from both devices is working 
exactly like you suggested.

Thanks again, Harald.

Re: RAID 1 | Test Booting from /dev/sdb

[Stefanie Leisestreichler]

On 05.05.20 10:51, Wols Lists wrote:
>> Change boot medium to /dev/sdb
>> and do a "mdadm /dev/md0 --add /dev/sda1" to get it recovered again
>> without loosing the "added" data (i.e. in /var/log) from booting? Also
>> device identifiers could change I guess. Even if I am fine with loosing
>> the "added" data from booting with /dev/sdb, will - when booting again
>> from /dev/sda - /dev/sda be the master in the array again?
>>
>> It is not clear to me if I understood correctly in which case which
>> array member will be the master which will be the base for recovery. Is
>> it always the HD one booted from?
>>
> The "master" if recovery is required will be the "older" one - in this
> case sda because it was disconnected. HOWEVER. Just check whether you
> have a bitmap or journal enabled. You can't have both at once, but the
> result should be that sda rejoins the array cleanly, raid has a record
> of which writes occurred while it was offline, and it will be updated.
> 

Does this mean in my scenario /dev/sda1 will come up again and will held 
data in /var/log/ where I can see i.e. log entries which are written 
when the system booted up using /dev/sdb as boot device?

I try to be prepared, just in case :-)

Thanks again for sharing your knowledge, Wols.
It is highly appreciated.

Re: RAID 1 | Test Booting from /dev/sdb

[Wols Lists]

On 05/05/20 10:25, Stefanie Leisestreichler wrote:
> 
> On 05.05.20 10:51, Wols Lists wrote:
>>> Change boot medium to /dev/sdb
>>> and do a "mdadm /dev/md0 --add /dev/sda1" to get it recovered again
>>> without loosing the "added" data (i.e. in /var/log) from booting? Also
>>> device identifiers could change I guess. Even if I am fine with loosing
>>> the "added" data from booting with /dev/sdb, will - when booting again
>>> from /dev/sda - /dev/sda be the master in the array again?
>>>
>>> It is not clear to me if I understood correctly in which case which
>>> array member will be the master which will be the base for recovery. Is
>>> it always the HD one booted from?
>>>
>> The "master" if recovery is required will be the "older" one - in this
>> case sda because it was disconnected. HOWEVER. Just check whether you
>> have a bitmap or journal enabled. You can't have both at once, but the
>> result should be that sda rejoins the array cleanly, raid has a record
>> of which writes occurred while it was offline, and it will be updated.
>>
> 
> Does this mean in my scenario /dev/sda1 will come up again and will held
> data in /var/log/ where I can see i.e. log entries which are written
> when the system booted up using /dev/sdb as boot device?

No. The journal or bitmap are stored as part of the array, not the
filesystem. sdb will have a load of "flags" set to say "these blocks
were written to while the array was degraded". So when sda is added back
in, these delayed writes will be copied from sdb to sda.
> 
> I try to be prepared, just in case :-)

Don't we all :-)
> 
> Thanks again for sharing your knowledge, Wols.
> It is highly appreciated.

It's not that deep, but I do try to be clear what I do and don't know
:-) I'm lucky I have a very good memory.

Cheers,
Wol

Re: RAID 1 | Test Booting from /dev/sdb

[Stefanie Leisestreichler]

On 05.05.20 11:34, Wols Lists wrote:
> No. The journal or bitmap are stored as part of the array, not the
> filesystem. sdb will have a load of "flags" set to say "these blocks
> were written to while the array was degraded". So when sda is added back
> in, these delayed writes will be copied from sdb to sda.

When sdb copies the changes it has in the blocks /var/log sits on, will 
this result in the array showing the changes made to /var/log when it 
was booted using sdb only or did I get this wrong?

Re: RAID 1 | Test Booting from /dev/sdb

[Reindl Harald]

Am 05.05.20 um 11:42 schrieb Stefanie Leisestreichler:
> On 05.05.20 11:34, Wols Lists wrote:
>> No. The journal or bitmap are stored as part of the array, not the
>> filesystem. sdb will have a load of "flags" set to say "these blocks
>> were written to while the array was degraded". So when sda is added back
>> in, these delayed writes will be copied from sdb to sda.
> 
> When sdb copies the changes it has in the blocks /var/log sits on, will
> this result in the array showing the changes made to /var/log when it
> was booted using sdb only or did I get this wrong?

the array is just fine and the filesystem on-top never takes any notice
because it's a different layer not knowing anything about the RAID

the array is just a block device like a single disk for upper layers