* pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks
@ 2020-10-06 16:07 linux-raid
2020-10-06 17:00 ` Reindl Harald
2020-10-07 11:18 ` linux-raid
0 siblings, 2 replies; 5+ messages in thread
From: linux-raid @ 2020-10-06 16:07 UTC (permalink / raw)
To: Linux-RAID
I setup a crash&burn fedora 32 workstation using two 500GB SSD /dev/sd{a,b}; and two 4TB disks
/dev/sd{c,d}.
I'm effectively a raid and lvm newbie, since I only do this type of thing once every 5 years or so.
Through the blivot GUI, using trial and error I got a bootable system installed with
raid1 (and lvm) as hinted:
* GPT disk label wrote to both SSD disks /dev/sd{a,b}.
* on /dev/sd{a,b}1 created 2MiB "bios boot" partition. I assume only one is used? I could try clobbering /dev/sda1 ...
* on /dev/sd{a,b}2 created 1G "localhost-live:raid1_boot" ext4 /boot filesystem under raid1.
* on /dev/sd{a,b}3 created a 464.8G "localhost-live:raid1_os-main" using the remaining disk space, as LVM ( 464.63g VG named lv_os-main )
* made 8 GiB swap partition over raid1 as /dev/mapper/lv_os--main-swap ( on the SSD )
* made 370 GiB / ext4 partition over raid1 as /dev/mapper/lv_os--main-root
* created a 3.65 TiB raid1 /dev/md125 array on /dev/sd{d,c} with the name ???? using the remaining disk space, as LVM ( 3.64t VG named lv_big )
* /dev/mapper/lv_big-var on /var type ext4
* /dev/mapper/lv_big-00 on /home type ext4
I was able to install and boot fedora 32 OK.
----
Is it possible to get the 2MiB "bios boot" partition into raid1?
This seems to be the most vulnerable part of my setup.
Let me know additional commands you need to document my setup.
I do not know the depth of my ignorance -- any general advice is welcome.
I can redo all of this setup per your suggestions.
--
thanks,
Tom
--------
[root@loki ~]# mdadm --version
mdadm - v4.1 - 2018-10-01
[root@loki ~]# uname -r
5.8.9-200.fc32.x86_64
[root@loki ~]# echo /dev/sd[a-d][0-9]
/dev/sda1 /dev/sda2 /dev/sda3 /dev/sdb1 /dev/sdb2 /dev/sdb3 /dev/sdc1 /dev/sdd1
[root@loki ~]# mount -t ext4
/dev/mapper/lv_os--main-root on / type ext4 (rw,relatime,seclabel)
/dev/md126 on /boot type ext4 (rw,relatime,seclabel)
/dev/mapper/lv_big-var on /var type ext4 (rw,relatime,seclabel)
/dev/mapper/lv_big-00 on /home type ext4 (rw,relatime,seclabel)
[root@loki ~]# ls /dev/md
raid1_boot raid1_os-main raid_big
[root@loki ~]# fdisk -l # shows: mbr or gpt label, 'bios boot' partition, raid devices, and lvm devices
Disk /dev/sda: 465.78 GiB, 500107862016 bytes, 976773168 sectors
Disk model: Samsung SSD 850
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 7CD40013-ACFB-433A-B6A1-B5EF63934004
Device Start End Sectors Size Type
/dev/sda1 2048 6143 4096 2M BIOS boot
/dev/sda2 6144 2103295 2097152 1G Linux RAID
/dev/sda3 2103296 976773119 974669824 464.8G Linux RAID
Disk /dev/sdb: 465.78 GiB, 500107862016 bytes, 976773168 sectors
Disk model: Samsung SSD 850
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 68F51B2D-C077-46D2-96A3-116BAF835998
Device Start End Sectors Size Type
/dev/sdb1 2048 6143 4096 2M BIOS boot
/dev/sdb2 6144 2103295 2097152 1G Linux RAID
/dev/sdb3 2103296 976773119 974669824 464.8G Linux RAID
Disk /dev/sdc: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: ST4000VN008-2DR1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: D263D0C0-2C37-4205-B505-BC3B6DD1DFDA
Device Start End Sectors Size Type
/dev/sdc1 2048 7814035491 7814033444 3.7T Linux RAID
Disk /dev/sdd: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: ST4000VN008-2DR1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 84D9D98B-0C27-4090-B639-AB48A0FA9B1E
Device Start End Sectors Size Type
/dev/sdd1 2048 7814035491 7814033444 3.7T Linux RAID
Disk /dev/md127: 464.65 GiB, 498895683584 bytes, 974405632 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/md126: 1022 MiB, 1071644672 bytes, 2093056 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/lv_os--main-root: 370 GiB, 397284474880 bytes, 775946240 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/lv_os--main-swap: 8 GiB, 8589934592 bytes, 16777216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/md125: 3.65 TiB, 4000649838592 bytes, 7813769216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/mapper/lv_big-00: 2.31 TiB, 2528880099328 bytes, 4939218944 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/mapper/lv_big-var: 400 GiB, 429496729600 bytes, 838860800 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
[root@loki ~]# mdadm --examine /dev/sd[a-d][0-9] ###
mdadm: No md superblock detected on /dev/sda1.
/dev/sda2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : a90395cd:6297a553:d29f32cd:04f3054c
Name : localhost-live:raid1_boot
Creation Time : Sat Sep 19 07:54:55 2020
Raid Level : raid1
Raid Devices : 2
Avail Dev Size : 2093056 (1022.00 MiB 1071.64 MB)
Array Size : 1046528 (1022.00 MiB 1071.64 MB)
Data Offset : 4096 sectors
Super Offset : 8 sectors
Unused Space : before=4016 sectors, after=0 sectors
State : clean
Device UUID : 3f3d4518:967ad3f5:99bdd519:5c7d079b
Internal Bitmap : 8 sectors from superblock
Update Time : Fri Sep 25 07:57:18 2020
Bad Block Log : 512 entries available at offset 16 sectors
Checksum : b6fe8069 - correct
Events : 83
Device Role : Active device 0
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sda3:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1ce0d10e:d281ceb8:a53a4dd4:b00d77b1
Name : localhost-live:raid1_os-main
Creation Time : Sat Sep 19 07:54:43 2020
Raid Level : raid1
Raid Devices : 2
Avail Dev Size : 974405632 (464.63 GiB 498.90 GB)
Array Size : 487202816 (464.63 GiB 498.90 GB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=264112 sectors, after=0 sectors
State : clean
Device UUID : 8317645e:e13b28c5:215484a7:a5505b5d
Internal Bitmap : 8 sectors from superblock
Update Time : Sat Sep 26 05:40:38 2020
Bad Block Log : 512 entries available at offset 16 sectors
Checksum : cfc13cbc - correct
Events : 894
Device Role : Active device 0
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
mdadm: No md superblock detected on /dev/sdb1.
/dev/sdb2:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : a90395cd:6297a553:d29f32cd:04f3054c
Name : localhost-live:raid1_boot
Creation Time : Sat Sep 19 07:54:55 2020
Raid Level : raid1
Raid Devices : 2
Avail Dev Size : 2093056 (1022.00 MiB 1071.64 MB)
Array Size : 1046528 (1022.00 MiB 1071.64 MB)
Data Offset : 4096 sectors
Super Offset : 8 sectors
Unused Space : before=4016 sectors, after=0 sectors
State : clean
Device UUID : c2d52945:bf91d15c:36a8dca5:ac76b9ec
Internal Bitmap : 8 sectors from superblock
Update Time : Fri Sep 25 07:57:18 2020
Bad Block Log : 512 entries available at offset 16 sectors
Checksum : 289a1404 - correct
Events : 83
Device Role : Active device 1
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdb3:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 1ce0d10e:d281ceb8:a53a4dd4:b00d77b1
Name : localhost-live:raid1_os-main
Creation Time : Sat Sep 19 07:54:43 2020
Raid Level : raid1
Raid Devices : 2
Avail Dev Size : 974405632 (464.63 GiB 498.90 GB)
Array Size : 487202816 (464.63 GiB 498.90 GB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=264112 sectors, after=0 sectors
State : clean
Device UUID : dab22a3f:5528d6e6:9da9ce74:a32cf0a6
Internal Bitmap : 8 sectors from superblock
Update Time : Sat Sep 26 05:40:38 2020
Bad Block Log : 512 entries available at offset 16 sectors
Checksum : e914f602 - correct
Events : 894
Device Role : Active device 1
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 434db08c:6fdf4494:19a21b16:6e662faf
Name : localhost-live:raid_big
Creation Time : Sat Sep 19 07:54:00 2020
Raid Level : raid1
Raid Devices : 2
Avail Dev Size : 7813769252 (3725.90 GiB 4000.65 GB)
Array Size : 3906884608 (3725.90 GiB 4000.65 GB)
Used Dev Size : 7813769216 (3725.90 GiB 4000.65 GB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=264112 sectors, after=36 sectors
State : clean
Device UUID : 050a5fc5:b39e6e60:98f3ffad:71e38d71
Internal Bitmap : 8 sectors from superblock
Update Time : Sat Sep 26 06:09:58 2020
Bad Block Log : 512 entries available at offset 24 sectors
Checksum : 476865a4 - correct
Events : 43615
Device Role : Active device 0
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x1
Array UUID : 434db08c:6fdf4494:19a21b16:6e662faf
Name : localhost-live:raid_big
Creation Time : Sat Sep 19 07:54:00 2020
Raid Level : raid1
Raid Devices : 2
Avail Dev Size : 7813769252 (3725.90 GiB 4000.65 GB)
Array Size : 3906884608 (3725.90 GiB 4000.65 GB)
Used Dev Size : 7813769216 (3725.90 GiB 4000.65 GB)
Data Offset : 264192 sectors
Super Offset : 8 sectors
Unused Space : before=264112 sectors, after=36 sectors
State : clean
Device UUID : ebb43ce0:bd684ff7:c8562ac6:8774a702
Internal Bitmap : 8 sectors from superblock
Update Time : Sat Sep 26 06:09:58 2020
Bad Block Log : 512 entries available at offset 24 sectors
Checksum : a26acedb - correct
Events : 43615
Device Role : Active device 1
Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
[root@loki ~]# pvs
PV VG Fmt Attr PSize PFree
/dev/md125 lv_big lvm2 a-- <3.64t 970.69g
/dev/md127 lv_os-main lvm2 a-- <464.63g <86.63g
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks
2020-10-06 16:07 pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks linux-raid
@ 2020-10-06 17:00 ` Reindl Harald
2020-10-06 22:24 ` antlists
2020-10-06 22:56 ` Roger Heflin
2020-10-07 11:18 ` linux-raid
1 sibling, 2 replies; 5+ messages in thread
From: Reindl Harald @ 2020-10-06 17:00 UTC (permalink / raw)
To: linux-raid, Linux-RAID
Am 06.10.20 um 18:07 schrieb linux-raid@zq3q.org:
> Is it possible to get the 2MiB "bios boot" partition into raid1?
> This seems to be the most vulnerable part of my setup.
should be no problem when you have the same partitioning on all disks,
"BIOS boot" normally have no writes at boot time
RAID1 is handeled like a single disk at early boot
the only thing you need to do manually is install the bootloaer on all
disks and don#t forget repeat it after replace one
the only thing i am not sure is the partition type, until now my setups
are MBR and sda1 is a RAID1 over all 4 disks
[root@srv-rhsoft:~]$ fdisk -l /dev/sda
Disk /dev/sda: 1.84 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: Samsung SSD 860
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x000d9ef2
Device Boot Start End Sectors Size Id Type
/dev/sda1 * 2048 1026047 1024000 500M fd Linux raid
autodetect
/dev/sda2 1026048 31746047 30720000 14.7G fd Linux raid
autodetect
/dev/sda3 31746048 3906971647 3875225600 1.8T fd Linux raid
autodetect
[root@srv-rhsoft:~]$ df
Filesystem Type Size Used Avail Use% Mounted on
/dev/md1 ext4 29G 7.4G 22G 26% /
/dev/md2 ext4 3.6T 1.2T 2.5T 33% /mnt/data
/dev/md0 ext4 485M 49M 432M 11% /boot
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks
2020-10-06 17:00 ` Reindl Harald
@ 2020-10-06 22:24 ` antlists
2020-10-06 22:56 ` Roger Heflin
1 sibling, 0 replies; 5+ messages in thread
From: antlists @ 2020-10-06 22:24 UTC (permalink / raw)
To: Reindl Harald, linux-raid, Linux-RAID
On 06/10/2020 18:00, Reindl Harald wrote:
> Am 06.10.20 um 18:07 schrieb linux-raid@zq3q.org:
>> Is it possible to get the 2MiB "bios boot" partition into raid1?
>> This seems to be the most vulnerable part of my setup.
> should be no problem when you have the same partitioning on all disks,
> "BIOS boot" normally have no writes at boot time
>
> RAID1 is handeled like a single disk at early boot
>
> the only thing you need to do manually is install the bootloaer on all
> disks and don#t forget repeat it after replace one
>
Very much so ...
> the only thing i am not sure is the partition type, until now my setups
> are MBR and sda1 is a RAID1 over all 4 disks
And the raid type. Is the bios boot accessed as part of the boot
sequence - sounds like it must be :-) ?
Anything that might be accessed by something *before* linux is loaded,
needs to be a 1.0 superblock. (I won't recommend 0.9 as it's obsolete.)
That way, provided you ONLY access it to read, you can access the
filesystem directly and bypass the raid by referencing the partition eg
/dev/sda1. Then once linux has booted and mounted /dev/md1, any changes
will be made to both partitions.
Cheers,
Wol
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks
2020-10-06 17:00 ` Reindl Harald
2020-10-06 22:24 ` antlists
@ 2020-10-06 22:56 ` Roger Heflin
1 sibling, 0 replies; 5+ messages in thread
From: Roger Heflin @ 2020-10-06 22:56 UTC (permalink / raw)
To: Reindl Harald; +Cc: linux-raid, Linux-RAID
Nothing seems to care about the partition type(LVM or md). My working
md partitions have the following types:
Microsoft basic data
Linux Raid
Linux Filesystem
All successfully reform into a working raid6 after reboot.
On Tue, Oct 6, 2020 at 12:05 PM Reindl Harald <h.reindl@thelounge.net> wrote:
>
>
>
> Am 06.10.20 um 18:07 schrieb linux-raid@zq3q.org:
> > Is it possible to get the 2MiB "bios boot" partition into raid1?
> > This seems to be the most vulnerable part of my setup.
> should be no problem when you have the same partitioning on all disks,
> "BIOS boot" normally have no writes at boot time
>
> RAID1 is handeled like a single disk at early boot
>
> the only thing you need to do manually is install the bootloaer on all
> disks and don#t forget repeat it after replace one
>
> the only thing i am not sure is the partition type, until now my setups
> are MBR and sda1 is a RAID1 over all 4 disks
>
> [root@srv-rhsoft:~]$ fdisk -l /dev/sda
> Disk /dev/sda: 1.84 TiB, 2000398934016 bytes, 3907029168 sectors
> Disk model: Samsung SSD 860
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disklabel type: dos
> Disk identifier: 0x000d9ef2
>
> Device Boot Start End Sectors Size Id Type
> /dev/sda1 * 2048 1026047 1024000 500M fd Linux raid
> autodetect
> /dev/sda2 1026048 31746047 30720000 14.7G fd Linux raid
> autodetect
> /dev/sda3 31746048 3906971647 3875225600 1.8T fd Linux raid
> autodetect
>
> [root@srv-rhsoft:~]$ df
> Filesystem Type Size Used Avail Use% Mounted on
> /dev/md1 ext4 29G 7.4G 22G 26% /
> /dev/md2 ext4 3.6T 1.2T 2.5T 33% /mnt/data
> /dev/md0 ext4 485M 49M 432M 11% /boot
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks
2020-10-06 16:07 pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks linux-raid
2020-10-06 17:00 ` Reindl Harald
@ 2020-10-07 11:18 ` linux-raid
1 sibling, 0 replies; 5+ messages in thread
From: linux-raid @ 2020-10-07 11:18 UTC (permalink / raw)
To: Linux-RAID
Thanks so far for all the help. I'm still digesting it -- will try to respond one by one later.
One of the other posts prompted me to supply this additional info:
[root@loki ~]# blkid
/dev/sda1: PARTUUID="fd645cd2-d5c2-4bff-be9b-19cba21b31c1"
/dev/sda2: UUID="a90395cd-6297-a553-d29f-32cd04f3054c" UUID_SUB="3f3d4518-967a-d3f5-99bd-d5195c7d079b" LABEL="localhost-live:raid1_boot" TYPE="linux_raid_member" PARTUUID="713603aa-7e4f-4d08-b9a2-6f9c6a692357"
/dev/sda3: UUID="1ce0d10e-d281-ceb8-a53a-4dd4b00d77b1" UUID_SUB="8317645e-e13b-28c5-2154-84a7a5505b5d" LABEL="localhost-live:raid1_os-main" TYPE="linux_raid_member" PARTUUID="4a843100-e448-427c-b575-bfc2a1bc6205"
/dev/sdb1: PARTUUID="9f9a91c0-aaf4-4083-a00c-676187a990ee"
/dev/sdb2: UUID="a90395cd-6297-a553-d29f-32cd04f3054c" UUID_SUB="c2d52945-bf91-d15c-36a8-dca5ac76b9ec" LABEL="localhost-live:raid1_boot" TYPE="linux_raid_member" PARTUUID="f004e563-bab2-475d-a2ae-6e1b8d2d53d7"
/dev/sdb3: UUID="1ce0d10e-d281-ceb8-a53a-4dd4b00d77b1" UUID_SUB="dab22a3f-5528-d6e6-9da9-ce74a32cf0a6" LABEL="localhost-live:raid1_os-main" TYPE="linux_raid_member" PARTUUID="0c85fe79-4d0c-441a-87ff-75852f164a86"
/dev/sdc1: UUID="434db08c-6fdf-4494-19a2-1b166e662faf" UUID_SUB="050a5fc5-b39e-6e60-98f3-ffad71e38d71" LABEL="localhost-live:raid_big" TYPE="linux_raid_member" PARTUUID="3a97d627-78f6-4c5f-8a7d-6d005165e3c9"
/dev/sdd1: UUID="434db08c-6fdf-4494-19a2-1b166e662faf" UUID_SUB="ebb43ce0-bd68-4ff7-c856-2ac68774a702" LABEL="localhost-live:raid_big" TYPE="linux_raid_member" PARTUUID="e13a3833-4fdc-48d8-8dac-6993d3a749c7"
/dev/md127: UUID="2k3Khh-ET0b-e1p6-Q36V-Epj0-qwui-CeV1YH" TYPE="LVM2_member"
/dev/md126: UUID="84ed704c-b983-4ce6-8ddd-42b40fa4a407" BLOCK_SIZE="4096" TYPE="ext4"
/dev/mapper/lv_os--main-root: UUID="c08c6b29-7999-4677-b046-594a6685af75" BLOCK_SIZE="4096" TYPE="ext4"
/dev/mapper/lv_os--main-swap: UUID="96d78117-f933-4931-a3ac-033251041bf2" TYPE="swap"
/dev/md125: UUID="6p0Cks-EHQ9-i95n-fJIk-upXg-Qket-ZcOobc" TYPE="LVM2_member"
/dev/mapper/lv_big-00: LABEL="home" UUID="e64f9b00-a4ec-4466-a001-09c96f5aff0a" BLOCK_SIZE="4096" TYPE="ext4"
/dev/mapper/lv_big-var: UUID="f42563bc-bb66-4837-b357-72358d404e32" BLOCK_SIZE="4096" TYPE="ext4"
[root@loki ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 465.8G 0 disk
├─sda1 8:1 0 2M 0 part
├─sda2 8:2 0 1G 0 part
│ └─md126 9:126 0 1022M 0 raid1 /boot
└─sda3 8:3 0 464.8G 0 part
└─md127 9:127 0 464.6G 0 raid1
├─lv_os--main-root 253:0 0 370G 0 lvm /
└─lv_os--main-swap 253:1 0 8G 0 lvm [SWAP]
sdb 8:16 0 465.8G 0 disk
├─sdb1 8:17 0 2M 0 part
├─sdb2 8:18 0 1G 0 part
│ └─md126 9:126 0 1022M 0 raid1 /boot
└─sdb3 8:19 0 464.8G 0 part
└─md127 9:127 0 464.6G 0 raid1
├─lv_os--main-root 253:0 0 370G 0 lvm /
└─lv_os--main-swap 253:1 0 8G 0 lvm [SWAP]
sdc 8:32 0 3.7T 0 disk
└─sdc1 8:33 0 3.7T 0 part
└─md125 9:125 0 3.7T 0 raid1
├─lv_big-00 253:2 0 2.3T 0 lvm /home
└─lv_big-var 253:3 0 400G 0 lvm /var
sdd 8:48 0 3.7T 0 disk
└─sdd1 8:49 0 3.7T 0 part
└─md125 9:125 0 3.7T 0 raid1
├─lv_big-00 253:2 0 2.3T 0 lvm /home
└─lv_big-var 253:3 0 400G 0 lvm /var
sr0 11:0 1 1024M 0 rom
--
Tom
On Tue 10/6/20 11:07 -0500 linux-raid@zq3q.org wrote:
>I setup a crash&burn fedora 32 workstation using two 500GB SSD /dev/sd{a,b}; and two 4TB disks
>/dev/sd{c,d}.
>
>I'm effectively a raid and lvm newbie, since I only do this type of thing once every 5 years or so.
>
>Through the blivot GUI, using trial and error I got a bootable system installed with
>raid1 (and lvm) as hinted:
>
>* GPT disk label wrote to both SSD disks /dev/sd{a,b}.
>
>* on /dev/sd{a,b}1 created 2MiB "bios boot" partition. I assume only one is used? I could try clobbering /dev/sda1 ...
>
>* on /dev/sd{a,b}2 created 1G "localhost-live:raid1_boot" ext4 /boot filesystem under raid1.
>
>* on /dev/sd{a,b}3 created a 464.8G "localhost-live:raid1_os-main" using the remaining disk space, as LVM ( 464.63g VG named lv_os-main )
>
> * made 8 GiB swap partition over raid1 as /dev/mapper/lv_os--main-swap ( on the SSD )
>
> * made 370 GiB / ext4 partition over raid1 as /dev/mapper/lv_os--main-root
>
>* created a 3.65 TiB raid1 /dev/md125 array on /dev/sd{d,c} with the name ???? using the remaining disk space, as LVM ( 3.64t VG named lv_big )
>
> * /dev/mapper/lv_big-var on /var type ext4
> * /dev/mapper/lv_big-00 on /home type ext4
>
>I was able to install and boot fedora 32 OK.
>
>----
>
>Is it possible to get the 2MiB "bios boot" partition into raid1?
>This seems to be the most vulnerable part of my setup.
>
>Let me know additional commands you need to document my setup.
>I do not know the depth of my ignorance -- any general advice is welcome.
>I can redo all of this setup per your suggestions.
>
>--
>thanks,
>Tom
>
>--------
>[root@loki ~]# mdadm --version
>mdadm - v4.1 - 2018-10-01
>[root@loki ~]# uname -r
>5.8.9-200.fc32.x86_64
>[root@loki ~]# echo /dev/sd[a-d][0-9]
>/dev/sda1 /dev/sda2 /dev/sda3 /dev/sdb1 /dev/sdb2 /dev/sdb3 /dev/sdc1 /dev/sdd1
>[root@loki ~]# mount -t ext4
>/dev/mapper/lv_os--main-root on / type ext4 (rw,relatime,seclabel)
>/dev/md126 on /boot type ext4 (rw,relatime,seclabel)
>/dev/mapper/lv_big-var on /var type ext4 (rw,relatime,seclabel)
>/dev/mapper/lv_big-00 on /home type ext4 (rw,relatime,seclabel)
>[root@loki ~]# ls /dev/md
>raid1_boot raid1_os-main raid_big
>[root@loki ~]# fdisk -l # shows: mbr or gpt label, 'bios boot' partition, raid devices, and lvm devices
>Disk /dev/sda: 465.78 GiB, 500107862016 bytes, 976773168 sectors
>Disk model: Samsung SSD 850
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>Disklabel type: gpt
>Disk identifier: 7CD40013-ACFB-433A-B6A1-B5EF63934004
>
>Device Start End Sectors Size Type
>/dev/sda1 2048 6143 4096 2M BIOS boot
>/dev/sda2 6144 2103295 2097152 1G Linux RAID
>/dev/sda3 2103296 976773119 974669824 464.8G Linux RAID
>
>
>Disk /dev/sdb: 465.78 GiB, 500107862016 bytes, 976773168 sectors
>Disk model: Samsung SSD 850
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>Disklabel type: gpt
>Disk identifier: 68F51B2D-C077-46D2-96A3-116BAF835998
>
>Device Start End Sectors Size Type
>/dev/sdb1 2048 6143 4096 2M BIOS boot
>/dev/sdb2 6144 2103295 2097152 1G Linux RAID
>/dev/sdb3 2103296 976773119 974669824 464.8G Linux RAID
>
>
>Disk /dev/sdc: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
>Disk model: ST4000VN008-2DR1
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>Disklabel type: gpt
>Disk identifier: D263D0C0-2C37-4205-B505-BC3B6DD1DFDA
>
>Device Start End Sectors Size Type
>/dev/sdc1 2048 7814035491 7814033444 3.7T Linux RAID
>
>
>Disk /dev/sdd: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
>Disk model: ST4000VN008-2DR1
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>Disklabel type: gpt
>Disk identifier: 84D9D98B-0C27-4090-B639-AB48A0FA9B1E
>
>Device Start End Sectors Size Type
>/dev/sdd1 2048 7814035491 7814033444 3.7T Linux RAID
>
>
>Disk /dev/md127: 464.65 GiB, 498895683584 bytes, 974405632 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>Disk /dev/md126: 1022 MiB, 1071644672 bytes, 2093056 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>Disk /dev/mapper/lv_os--main-root: 370 GiB, 397284474880 bytes, 775946240 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>Disk /dev/mapper/lv_os--main-swap: 8 GiB, 8589934592 bytes, 16777216 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>
>
>Disk /dev/md125: 3.65 TiB, 4000649838592 bytes, 7813769216 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>
>
>Disk /dev/mapper/lv_big-00: 2.31 TiB, 2528880099328 bytes, 4939218944 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>
>
>Disk /dev/mapper/lv_big-var: 400 GiB, 429496729600 bytes, 838860800 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>[root@loki ~]# mdadm --examine /dev/sd[a-d][0-9] ###
>mdadm: No md superblock detected on /dev/sda1.
>/dev/sda2:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : a90395cd:6297a553:d29f32cd:04f3054c
> Name : localhost-live:raid1_boot
> Creation Time : Sat Sep 19 07:54:55 2020
> Raid Level : raid1
> Raid Devices : 2
>
> Avail Dev Size : 2093056 (1022.00 MiB 1071.64 MB)
> Array Size : 1046528 (1022.00 MiB 1071.64 MB)
> Data Offset : 4096 sectors
> Super Offset : 8 sectors
> Unused Space : before=4016 sectors, after=0 sectors
> State : clean
> Device UUID : 3f3d4518:967ad3f5:99bdd519:5c7d079b
>
>Internal Bitmap : 8 sectors from superblock
> Update Time : Fri Sep 25 07:57:18 2020
> Bad Block Log : 512 entries available at offset 16 sectors
> Checksum : b6fe8069 - correct
> Events : 83
>
>
> Device Role : Active device 0
> Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sda3:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : 1ce0d10e:d281ceb8:a53a4dd4:b00d77b1
> Name : localhost-live:raid1_os-main
> Creation Time : Sat Sep 19 07:54:43 2020
> Raid Level : raid1
> Raid Devices : 2
>
> Avail Dev Size : 974405632 (464.63 GiB 498.90 GB)
> Array Size : 487202816 (464.63 GiB 498.90 GB)
> Data Offset : 264192 sectors
> Super Offset : 8 sectors
> Unused Space : before=264112 sectors, after=0 sectors
> State : clean
> Device UUID : 8317645e:e13b28c5:215484a7:a5505b5d
>
>Internal Bitmap : 8 sectors from superblock
> Update Time : Sat Sep 26 05:40:38 2020
> Bad Block Log : 512 entries available at offset 16 sectors
> Checksum : cfc13cbc - correct
> Events : 894
>
>
> Device Role : Active device 0
> Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>mdadm: No md superblock detected on /dev/sdb1.
>/dev/sdb2:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : a90395cd:6297a553:d29f32cd:04f3054c
> Name : localhost-live:raid1_boot
> Creation Time : Sat Sep 19 07:54:55 2020
> Raid Level : raid1
> Raid Devices : 2
>
> Avail Dev Size : 2093056 (1022.00 MiB 1071.64 MB)
> Array Size : 1046528 (1022.00 MiB 1071.64 MB)
> Data Offset : 4096 sectors
> Super Offset : 8 sectors
> Unused Space : before=4016 sectors, after=0 sectors
> State : clean
> Device UUID : c2d52945:bf91d15c:36a8dca5:ac76b9ec
>
>Internal Bitmap : 8 sectors from superblock
> Update Time : Fri Sep 25 07:57:18 2020
> Bad Block Log : 512 entries available at offset 16 sectors
> Checksum : 289a1404 - correct
> Events : 83
>
>
> Device Role : Active device 1
> Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sdb3:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : 1ce0d10e:d281ceb8:a53a4dd4:b00d77b1
> Name : localhost-live:raid1_os-main
> Creation Time : Sat Sep 19 07:54:43 2020
> Raid Level : raid1
> Raid Devices : 2
>
> Avail Dev Size : 974405632 (464.63 GiB 498.90 GB)
> Array Size : 487202816 (464.63 GiB 498.90 GB)
> Data Offset : 264192 sectors
> Super Offset : 8 sectors
> Unused Space : before=264112 sectors, after=0 sectors
> State : clean
> Device UUID : dab22a3f:5528d6e6:9da9ce74:a32cf0a6
>
>Internal Bitmap : 8 sectors from superblock
> Update Time : Sat Sep 26 05:40:38 2020
> Bad Block Log : 512 entries available at offset 16 sectors
> Checksum : e914f602 - correct
> Events : 894
>
>
> Device Role : Active device 1
> Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sdc1:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : 434db08c:6fdf4494:19a21b16:6e662faf
> Name : localhost-live:raid_big
> Creation Time : Sat Sep 19 07:54:00 2020
> Raid Level : raid1
> Raid Devices : 2
>
> Avail Dev Size : 7813769252 (3725.90 GiB 4000.65 GB)
> Array Size : 3906884608 (3725.90 GiB 4000.65 GB)
> Used Dev Size : 7813769216 (3725.90 GiB 4000.65 GB)
> Data Offset : 264192 sectors
> Super Offset : 8 sectors
> Unused Space : before=264112 sectors, after=36 sectors
> State : clean
> Device UUID : 050a5fc5:b39e6e60:98f3ffad:71e38d71
>
>Internal Bitmap : 8 sectors from superblock
> Update Time : Sat Sep 26 06:09:58 2020
> Bad Block Log : 512 entries available at offset 24 sectors
> Checksum : 476865a4 - correct
> Events : 43615
>
>
> Device Role : Active device 0
> Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sdd1:
> Magic : a92b4efc
> Version : 1.2
> Feature Map : 0x1
> Array UUID : 434db08c:6fdf4494:19a21b16:6e662faf
> Name : localhost-live:raid_big
> Creation Time : Sat Sep 19 07:54:00 2020
> Raid Level : raid1
> Raid Devices : 2
>
> Avail Dev Size : 7813769252 (3725.90 GiB 4000.65 GB)
> Array Size : 3906884608 (3725.90 GiB 4000.65 GB)
> Used Dev Size : 7813769216 (3725.90 GiB 4000.65 GB)
> Data Offset : 264192 sectors
> Super Offset : 8 sectors
> Unused Space : before=264112 sectors, after=36 sectors
> State : clean
> Device UUID : ebb43ce0:bd684ff7:c8562ac6:8774a702
>
>Internal Bitmap : 8 sectors from superblock
> Update Time : Sat Sep 26 06:09:58 2020
> Bad Block Log : 512 entries available at offset 24 sectors
> Checksum : a26acedb - correct
> Events : 43615
>
>
> Device Role : Active device 1
> Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>[root@loki ~]# pvs
> PV VG Fmt Attr PSize PFree
> /dev/md125 lv_big lvm2 a-- <3.64t 970.69g
> /dev/md127 lv_os-main lvm2 a-- <464.63g <86.63g
>
>
>
>
>
>
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-10-07 11:18 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-06 16:07 pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks linux-raid
2020-10-06 17:00 ` Reindl Harald
2020-10-06 22:24 ` antlists
2020-10-06 22:56 ` Roger Heflin
2020-10-07 11:18 ` linux-raid
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).