Linux-Raid Archives on lore.kernel.org
 help / color / Atom feed
* pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks
@ 2020-10-06 16:07 linux-raid
  2020-10-06 17:00 ` Reindl Harald
  2020-10-07 11:18 ` linux-raid
  0 siblings, 2 replies; 5+ messages in thread
From: linux-raid @ 2020-10-06 16:07 UTC (permalink / raw)
  To: Linux-RAID

I setup a crash&burn fedora 32 workstation using two 500GB SSD /dev/sd{a,b}; and two 4TB disks
/dev/sd{c,d}.

I'm effectively a raid and lvm newbie, since I only do this type of thing once every 5 years or so.

Through the blivot GUI, using trial and error I got a bootable system installed with
raid1 (and lvm) as hinted:

* GPT disk label wrote to both SSD disks /dev/sd{a,b}.

* on /dev/sd{a,b}1 created 2MiB "bios boot" partition. I assume only one is used? I could try clobbering /dev/sda1 ...

* on /dev/sd{a,b}2 created 1G "localhost-live:raid1_boot"  ext4 /boot filesystem under raid1.  

* on /dev/sd{a,b}3 created a 464.8G "localhost-live:raid1_os-main" using the remaining disk space, as LVM ( 464.63g VG named lv_os-main )

    * made 8 GiB swap partition over raid1 as /dev/mapper/lv_os--main-swap ( on the SSD )

    * made 370 GiB /  ext4 partition over raid1 as /dev/mapper/lv_os--main-root

* created a 3.65 TiB raid1 /dev/md125 array on /dev/sd{d,c} with the name ???? using the remaining disk space, as LVM ( 3.64t VG named lv_big )

    * /dev/mapper/lv_big-var on /var  type ext4
    * /dev/mapper/lv_big-00  on /home type ext4

I was able to install and boot fedora 32 OK.

----

Is it possible to get the 2MiB "bios boot" partition into raid1?
This seems to be the most vulnerable part of my setup.

Let me know additional commands you need to document my setup.
I do not know the depth of my ignorance -- any general advice is welcome.
I can redo all of this setup per your suggestions.  

--
thanks,
Tom

--------
[root@loki ~]# mdadm --version
mdadm - v4.1 - 2018-10-01
[root@loki ~]# uname -r
5.8.9-200.fc32.x86_64
[root@loki ~]# echo /dev/sd[a-d][0-9]
/dev/sda1 /dev/sda2 /dev/sda3 /dev/sdb1 /dev/sdb2 /dev/sdb3 /dev/sdc1 /dev/sdd1
[root@loki ~]# mount -t ext4
/dev/mapper/lv_os--main-root on / type ext4 (rw,relatime,seclabel)
/dev/md126 on /boot type ext4 (rw,relatime,seclabel)
/dev/mapper/lv_big-var on /var type ext4 (rw,relatime,seclabel)
/dev/mapper/lv_big-00 on /home type ext4 (rw,relatime,seclabel)
[root@loki ~]# ls /dev/md
raid1_boot  raid1_os-main  raid_big
[root@loki ~]# fdisk -l    # shows: mbr or gpt label,  'bios boot' partition, raid devices, and lvm devices
Disk /dev/sda: 465.78 GiB, 500107862016 bytes, 976773168 sectors
Disk model: Samsung SSD 850
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 7CD40013-ACFB-433A-B6A1-B5EF63934004

Device       Start       End   Sectors   Size Type
/dev/sda1     2048      6143      4096     2M BIOS boot
/dev/sda2     6144   2103295   2097152     1G Linux RAID
/dev/sda3  2103296 976773119 974669824 464.8G Linux RAID


Disk /dev/sdb: 465.78 GiB, 500107862016 bytes, 976773168 sectors
Disk model: Samsung SSD 850
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 68F51B2D-C077-46D2-96A3-116BAF835998

Device       Start       End   Sectors   Size Type
/dev/sdb1     2048      6143      4096     2M BIOS boot
/dev/sdb2     6144   2103295   2097152     1G Linux RAID
/dev/sdb3  2103296 976773119 974669824 464.8G Linux RAID


Disk /dev/sdc: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: ST4000VN008-2DR1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: D263D0C0-2C37-4205-B505-BC3B6DD1DFDA

Device     Start        End    Sectors  Size Type
/dev/sdc1   2048 7814035491 7814033444  3.7T Linux RAID


Disk /dev/sdd: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
Disk model: ST4000VN008-2DR1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 84D9D98B-0C27-4090-B639-AB48A0FA9B1E

Device     Start        End    Sectors  Size Type
/dev/sdd1   2048 7814035491 7814033444  3.7T Linux RAID


Disk /dev/md127: 464.65 GiB, 498895683584 bytes, 974405632 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/md126: 1022 MiB, 1071644672 bytes, 2093056 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/lv_os--main-root: 370 GiB, 397284474880 bytes, 775946240 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/lv_os--main-swap: 8 GiB, 8589934592 bytes, 16777216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes




Disk /dev/md125: 3.65 TiB, 4000649838592 bytes, 7813769216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/lv_big-00: 2.31 TiB, 2528880099328 bytes, 4939218944 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/lv_big-var: 400 GiB, 429496729600 bytes, 838860800 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
[root@loki ~]# mdadm --examine /dev/sd[a-d][0-9]  ###
mdadm: No md superblock detected on /dev/sda1.
/dev/sda2:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : a90395cd:6297a553:d29f32cd:04f3054c
           Name : localhost-live:raid1_boot
  Creation Time : Sat Sep 19 07:54:55 2020
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 2093056 (1022.00 MiB 1071.64 MB)
     Array Size : 1046528 (1022.00 MiB 1071.64 MB)
    Data Offset : 4096 sectors
   Super Offset : 8 sectors
   Unused Space : before=4016 sectors, after=0 sectors
          State : clean
    Device UUID : 3f3d4518:967ad3f5:99bdd519:5c7d079b

Internal Bitmap : 8 sectors from superblock
    Update Time : Fri Sep 25 07:57:18 2020
  Bad Block Log : 512 entries available at offset 16 sectors
       Checksum : b6fe8069 - correct
         Events : 83


   Device Role : Active device 0
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sda3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 1ce0d10e:d281ceb8:a53a4dd4:b00d77b1
           Name : localhost-live:raid1_os-main
  Creation Time : Sat Sep 19 07:54:43 2020
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 974405632 (464.63 GiB 498.90 GB)
     Array Size : 487202816 (464.63 GiB 498.90 GB)
    Data Offset : 264192 sectors
   Super Offset : 8 sectors
   Unused Space : before=264112 sectors, after=0 sectors
          State : clean
    Device UUID : 8317645e:e13b28c5:215484a7:a5505b5d

Internal Bitmap : 8 sectors from superblock
    Update Time : Sat Sep 26 05:40:38 2020
  Bad Block Log : 512 entries available at offset 16 sectors
       Checksum : cfc13cbc - correct
         Events : 894


   Device Role : Active device 0
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
mdadm: No md superblock detected on /dev/sdb1.
/dev/sdb2:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : a90395cd:6297a553:d29f32cd:04f3054c
           Name : localhost-live:raid1_boot
  Creation Time : Sat Sep 19 07:54:55 2020
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 2093056 (1022.00 MiB 1071.64 MB)
     Array Size : 1046528 (1022.00 MiB 1071.64 MB)
    Data Offset : 4096 sectors
   Super Offset : 8 sectors
   Unused Space : before=4016 sectors, after=0 sectors
          State : clean
    Device UUID : c2d52945:bf91d15c:36a8dca5:ac76b9ec

Internal Bitmap : 8 sectors from superblock
    Update Time : Fri Sep 25 07:57:18 2020
  Bad Block Log : 512 entries available at offset 16 sectors
       Checksum : 289a1404 - correct
         Events : 83


   Device Role : Active device 1
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdb3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 1ce0d10e:d281ceb8:a53a4dd4:b00d77b1
           Name : localhost-live:raid1_os-main
  Creation Time : Sat Sep 19 07:54:43 2020
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 974405632 (464.63 GiB 498.90 GB)
     Array Size : 487202816 (464.63 GiB 498.90 GB)
    Data Offset : 264192 sectors
   Super Offset : 8 sectors
   Unused Space : before=264112 sectors, after=0 sectors
          State : clean
    Device UUID : dab22a3f:5528d6e6:9da9ce74:a32cf0a6

Internal Bitmap : 8 sectors from superblock
    Update Time : Sat Sep 26 05:40:38 2020
  Bad Block Log : 512 entries available at offset 16 sectors
       Checksum : e914f602 - correct
         Events : 894


   Device Role : Active device 1
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 434db08c:6fdf4494:19a21b16:6e662faf
           Name : localhost-live:raid_big
  Creation Time : Sat Sep 19 07:54:00 2020
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 7813769252 (3725.90 GiB 4000.65 GB)
     Array Size : 3906884608 (3725.90 GiB 4000.65 GB)
  Used Dev Size : 7813769216 (3725.90 GiB 4000.65 GB)
    Data Offset : 264192 sectors
   Super Offset : 8 sectors
   Unused Space : before=264112 sectors, after=36 sectors
          State : clean
    Device UUID : 050a5fc5:b39e6e60:98f3ffad:71e38d71

Internal Bitmap : 8 sectors from superblock
    Update Time : Sat Sep 26 06:09:58 2020
  Bad Block Log : 512 entries available at offset 24 sectors
       Checksum : 476865a4 - correct
         Events : 43615


   Device Role : Active device 0
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 434db08c:6fdf4494:19a21b16:6e662faf
           Name : localhost-live:raid_big
  Creation Time : Sat Sep 19 07:54:00 2020
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 7813769252 (3725.90 GiB 4000.65 GB)
     Array Size : 3906884608 (3725.90 GiB 4000.65 GB)
  Used Dev Size : 7813769216 (3725.90 GiB 4000.65 GB)
    Data Offset : 264192 sectors
   Super Offset : 8 sectors
   Unused Space : before=264112 sectors, after=36 sectors
          State : clean
    Device UUID : ebb43ce0:bd684ff7:c8562ac6:8774a702

Internal Bitmap : 8 sectors from superblock
    Update Time : Sat Sep 26 06:09:58 2020
  Bad Block Log : 512 entries available at offset 24 sectors
       Checksum : a26acedb - correct
         Events : 43615


   Device Role : Active device 1
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
[root@loki ~]# pvs
  PV         VG         Fmt  Attr PSize    PFree
  /dev/md125 lv_big     lvm2 a--    <3.64t 970.69g
  /dev/md127 lv_os-main lvm2 a--  <464.63g <86.63g








^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks
  2020-10-06 16:07 pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks linux-raid
@ 2020-10-06 17:00 ` Reindl Harald
  2020-10-06 22:24   ` antlists
  2020-10-06 22:56   ` Roger Heflin
  2020-10-07 11:18 ` linux-raid
  1 sibling, 2 replies; 5+ messages in thread
From: Reindl Harald @ 2020-10-06 17:00 UTC (permalink / raw)
  To: linux-raid, Linux-RAID



Am 06.10.20 um 18:07 schrieb linux-raid@zq3q.org:
> Is it possible to get the 2MiB "bios boot" partition into raid1?
> This seems to be the most vulnerable part of my setup.
should be no problem when you have the same partitioning on all disks, 
"BIOS boot" normally have no writes at boot time

RAID1 is handeled like a single disk at early boot

the only thing you need to do manually is install the bootloaer on all 
disks and don#t forget repeat it after replace one

the only thing i am not sure is the partition type, until now my setups 
are MBR and sda1 is a RAID1 over all 4 disks

[root@srv-rhsoft:~]$ fdisk -l /dev/sda
Disk /dev/sda: 1.84 TiB, 2000398934016 bytes, 3907029168 sectors
Disk model: Samsung SSD 860
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x000d9ef2

Device     Boot    Start        End    Sectors  Size Id Type
/dev/sda1  *        2048    1026047    1024000  500M fd Linux raid 
autodetect
/dev/sda2        1026048   31746047   30720000 14.7G fd Linux raid 
autodetect
/dev/sda3       31746048 3906971647 3875225600  1.8T fd Linux raid 
autodetect

[root@srv-rhsoft:~]$ df
Filesystem     Type  Size  Used Avail Use% Mounted on
/dev/md1       ext4   29G  7.4G   22G  26% /
/dev/md2       ext4  3.6T  1.2T  2.5T  33% /mnt/data
/dev/md0       ext4  485M   49M  432M  11% /boot

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks
  2020-10-06 17:00 ` Reindl Harald
@ 2020-10-06 22:24   ` antlists
  2020-10-06 22:56   ` Roger Heflin
  1 sibling, 0 replies; 5+ messages in thread
From: antlists @ 2020-10-06 22:24 UTC (permalink / raw)
  To: Reindl Harald, linux-raid, Linux-RAID

On 06/10/2020 18:00, Reindl Harald wrote:
> Am 06.10.20 um 18:07 schrieb linux-raid@zq3q.org:
>> Is it possible to get the 2MiB "bios boot" partition into raid1?
>> This seems to be the most vulnerable part of my setup.
> should be no problem when you have the same partitioning on all disks, 
> "BIOS boot" normally have no writes at boot time
> 
> RAID1 is handeled like a single disk at early boot
> 
> the only thing you need to do manually is install the bootloaer on all 
> disks and don#t forget repeat it after replace one
> 
Very much so ...

> the only thing i am not sure is the partition type, until now my setups 
> are MBR and sda1 is a RAID1 over all 4 disks

And the raid type. Is the bios boot accessed as part of the boot 
sequence - sounds like it must be :-) ?

Anything that might be accessed by something *before* linux is loaded, 
needs to be a 1.0 superblock. (I won't recommend 0.9 as it's obsolete.) 
That way, provided you ONLY access it to read, you can access the 
filesystem directly and bypass the raid by referencing the partition eg 
/dev/sda1. Then once linux has booted and mounted /dev/md1, any changes 
will be made to both partitions.

Cheers,
Wol

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks
  2020-10-06 17:00 ` Reindl Harald
  2020-10-06 22:24   ` antlists
@ 2020-10-06 22:56   ` Roger Heflin
  1 sibling, 0 replies; 5+ messages in thread
From: Roger Heflin @ 2020-10-06 22:56 UTC (permalink / raw)
  To: Reindl Harald; +Cc: linux-raid, Linux-RAID

Nothing seems to care about the partition type(LVM or md).  My working
md partitions have the following types:
Microsoft basic data
Linux Raid
Linux Filesystem
All successfully reform into a working raid6 after reboot.


On Tue, Oct 6, 2020 at 12:05 PM Reindl Harald <h.reindl@thelounge.net> wrote:
>
>
>
> Am 06.10.20 um 18:07 schrieb linux-raid@zq3q.org:
> > Is it possible to get the 2MiB "bios boot" partition into raid1?
> > This seems to be the most vulnerable part of my setup.
> should be no problem when you have the same partitioning on all disks,
> "BIOS boot" normally have no writes at boot time
>
> RAID1 is handeled like a single disk at early boot
>
> the only thing you need to do manually is install the bootloaer on all
> disks and don#t forget repeat it after replace one
>
> the only thing i am not sure is the partition type, until now my setups
> are MBR and sda1 is a RAID1 over all 4 disks
>
> [root@srv-rhsoft:~]$ fdisk -l /dev/sda
> Disk /dev/sda: 1.84 TiB, 2000398934016 bytes, 3907029168 sectors
> Disk model: Samsung SSD 860
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disklabel type: dos
> Disk identifier: 0x000d9ef2
>
> Device     Boot    Start        End    Sectors  Size Id Type
> /dev/sda1  *        2048    1026047    1024000  500M fd Linux raid
> autodetect
> /dev/sda2        1026048   31746047   30720000 14.7G fd Linux raid
> autodetect
> /dev/sda3       31746048 3906971647 3875225600  1.8T fd Linux raid
> autodetect
>
> [root@srv-rhsoft:~]$ df
> Filesystem     Type  Size  Used Avail Use% Mounted on
> /dev/md1       ext4   29G  7.4G   22G  26% /
> /dev/md2       ext4  3.6T  1.2T  2.5T  33% /mnt/data
> /dev/md0       ext4  485M   49M  432M  11% /boot

^ permalink raw reply	[flat|nested] 5+ messages in thread

* Re: pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks
  2020-10-06 16:07 pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks linux-raid
  2020-10-06 17:00 ` Reindl Harald
@ 2020-10-07 11:18 ` linux-raid
  1 sibling, 0 replies; 5+ messages in thread
From: linux-raid @ 2020-10-07 11:18 UTC (permalink / raw)
  To: Linux-RAID

Thanks so far for all the help. I'm still digesting it -- will try to respond one by one later.

One of the other posts prompted me to supply this additional info:

[root@loki ~]# blkid
/dev/sda1: PARTUUID="fd645cd2-d5c2-4bff-be9b-19cba21b31c1"
/dev/sda2: UUID="a90395cd-6297-a553-d29f-32cd04f3054c" UUID_SUB="3f3d4518-967a-d3f5-99bd-d5195c7d079b" LABEL="localhost-live:raid1_boot" TYPE="linux_raid_member" PARTUUID="713603aa-7e4f-4d08-b9a2-6f9c6a692357"
/dev/sda3: UUID="1ce0d10e-d281-ceb8-a53a-4dd4b00d77b1" UUID_SUB="8317645e-e13b-28c5-2154-84a7a5505b5d" LABEL="localhost-live:raid1_os-main" TYPE="linux_raid_member" PARTUUID="4a843100-e448-427c-b575-bfc2a1bc6205"
/dev/sdb1: PARTUUID="9f9a91c0-aaf4-4083-a00c-676187a990ee"
/dev/sdb2: UUID="a90395cd-6297-a553-d29f-32cd04f3054c" UUID_SUB="c2d52945-bf91-d15c-36a8-dca5ac76b9ec" LABEL="localhost-live:raid1_boot" TYPE="linux_raid_member" PARTUUID="f004e563-bab2-475d-a2ae-6e1b8d2d53d7"
/dev/sdb3: UUID="1ce0d10e-d281-ceb8-a53a-4dd4b00d77b1" UUID_SUB="dab22a3f-5528-d6e6-9da9-ce74a32cf0a6" LABEL="localhost-live:raid1_os-main" TYPE="linux_raid_member" PARTUUID="0c85fe79-4d0c-441a-87ff-75852f164a86"
/dev/sdc1: UUID="434db08c-6fdf-4494-19a2-1b166e662faf" UUID_SUB="050a5fc5-b39e-6e60-98f3-ffad71e38d71" LABEL="localhost-live:raid_big" TYPE="linux_raid_member" PARTUUID="3a97d627-78f6-4c5f-8a7d-6d005165e3c9"
/dev/sdd1: UUID="434db08c-6fdf-4494-19a2-1b166e662faf" UUID_SUB="ebb43ce0-bd68-4ff7-c856-2ac68774a702" LABEL="localhost-live:raid_big" TYPE="linux_raid_member" PARTUUID="e13a3833-4fdc-48d8-8dac-6993d3a749c7"
/dev/md127: UUID="2k3Khh-ET0b-e1p6-Q36V-Epj0-qwui-CeV1YH" TYPE="LVM2_member"
/dev/md126: UUID="84ed704c-b983-4ce6-8ddd-42b40fa4a407" BLOCK_SIZE="4096" TYPE="ext4"
/dev/mapper/lv_os--main-root: UUID="c08c6b29-7999-4677-b046-594a6685af75" BLOCK_SIZE="4096" TYPE="ext4"
/dev/mapper/lv_os--main-swap: UUID="96d78117-f933-4931-a3ac-033251041bf2" TYPE="swap"
/dev/md125: UUID="6p0Cks-EHQ9-i95n-fJIk-upXg-Qket-ZcOobc" TYPE="LVM2_member"
/dev/mapper/lv_big-00: LABEL="home" UUID="e64f9b00-a4ec-4466-a001-09c96f5aff0a" BLOCK_SIZE="4096" TYPE="ext4"
/dev/mapper/lv_big-var: UUID="f42563bc-bb66-4837-b357-72358d404e32" BLOCK_SIZE="4096" TYPE="ext4"

[root@loki ~]# lsblk
NAME                   MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                      8:0    0 465.8G  0 disk  
├─sda1                   8:1    0     2M  0 part  
├─sda2                   8:2    0     1G  0 part  
│ └─md126                9:126  0  1022M  0 raid1 /boot
└─sda3                   8:3    0 464.8G  0 part  
  └─md127                9:127  0 464.6G  0 raid1 
    ├─lv_os--main-root 253:0    0   370G  0 lvm   /
    └─lv_os--main-swap 253:1    0     8G  0 lvm   [SWAP]
sdb                      8:16   0 465.8G  0 disk  
├─sdb1                   8:17   0     2M  0 part  
├─sdb2                   8:18   0     1G  0 part  
│ └─md126                9:126  0  1022M  0 raid1 /boot
└─sdb3                   8:19   0 464.8G  0 part  
  └─md127                9:127  0 464.6G  0 raid1 
    ├─lv_os--main-root 253:0    0   370G  0 lvm   /
    └─lv_os--main-swap 253:1    0     8G  0 lvm   [SWAP]
sdc                      8:32   0   3.7T  0 disk  
└─sdc1                   8:33   0   3.7T  0 part  
  └─md125                9:125  0   3.7T  0 raid1 
    ├─lv_big-00        253:2    0   2.3T  0 lvm   /home
    └─lv_big-var       253:3    0   400G  0 lvm   /var
sdd                      8:48   0   3.7T  0 disk  
└─sdd1                   8:49   0   3.7T  0 part  
  └─md125                9:125  0   3.7T  0 raid1 
    ├─lv_big-00        253:2    0   2.3T  0 lvm   /home
    └─lv_big-var       253:3    0   400G  0 lvm   /var
sr0                     11:0    1  1024M  0 rom

--
Tom

On Tue 10/6/20 11:07 -0500 linux-raid@zq3q.org wrote:
>I setup a crash&burn fedora 32 workstation using two 500GB SSD /dev/sd{a,b}; and two 4TB disks
>/dev/sd{c,d}.
>
>I'm effectively a raid and lvm newbie, since I only do this type of thing once every 5 years or so.
>
>Through the blivot GUI, using trial and error I got a bootable system installed with
>raid1 (and lvm) as hinted:
>
>* GPT disk label wrote to both SSD disks /dev/sd{a,b}.
>
>* on /dev/sd{a,b}1 created 2MiB "bios boot" partition. I assume only one is used? I could try clobbering /dev/sda1 ...
>
>* on /dev/sd{a,b}2 created 1G "localhost-live:raid1_boot"  ext4 /boot filesystem under raid1.  
>
>* on /dev/sd{a,b}3 created a 464.8G "localhost-live:raid1_os-main" using the remaining disk space, as LVM ( 464.63g VG named lv_os-main )
>
>    * made 8 GiB swap partition over raid1 as /dev/mapper/lv_os--main-swap ( on the SSD )
>
>    * made 370 GiB /  ext4 partition over raid1 as /dev/mapper/lv_os--main-root
>
>* created a 3.65 TiB raid1 /dev/md125 array on /dev/sd{d,c} with the name ???? using the remaining disk space, as LVM ( 3.64t VG named lv_big )
>
>    * /dev/mapper/lv_big-var on /var  type ext4
>    * /dev/mapper/lv_big-00  on /home type ext4
>
>I was able to install and boot fedora 32 OK.
>
>----
>
>Is it possible to get the 2MiB "bios boot" partition into raid1?
>This seems to be the most vulnerable part of my setup.
>
>Let me know additional commands you need to document my setup.
>I do not know the depth of my ignorance -- any general advice is welcome.
>I can redo all of this setup per your suggestions.  
>
>--
>thanks,
>Tom
>
>--------
>[root@loki ~]# mdadm --version
>mdadm - v4.1 - 2018-10-01
>[root@loki ~]# uname -r
>5.8.9-200.fc32.x86_64
>[root@loki ~]# echo /dev/sd[a-d][0-9]
>/dev/sda1 /dev/sda2 /dev/sda3 /dev/sdb1 /dev/sdb2 /dev/sdb3 /dev/sdc1 /dev/sdd1
>[root@loki ~]# mount -t ext4
>/dev/mapper/lv_os--main-root on / type ext4 (rw,relatime,seclabel)
>/dev/md126 on /boot type ext4 (rw,relatime,seclabel)
>/dev/mapper/lv_big-var on /var type ext4 (rw,relatime,seclabel)
>/dev/mapper/lv_big-00 on /home type ext4 (rw,relatime,seclabel)
>[root@loki ~]# ls /dev/md
>raid1_boot  raid1_os-main  raid_big
>[root@loki ~]# fdisk -l    # shows: mbr or gpt label,  'bios boot' partition, raid devices, and lvm devices
>Disk /dev/sda: 465.78 GiB, 500107862016 bytes, 976773168 sectors
>Disk model: Samsung SSD 850
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>Disklabel type: gpt
>Disk identifier: 7CD40013-ACFB-433A-B6A1-B5EF63934004
>
>Device       Start       End   Sectors   Size Type
>/dev/sda1     2048      6143      4096     2M BIOS boot
>/dev/sda2     6144   2103295   2097152     1G Linux RAID
>/dev/sda3  2103296 976773119 974669824 464.8G Linux RAID
>
>
>Disk /dev/sdb: 465.78 GiB, 500107862016 bytes, 976773168 sectors
>Disk model: Samsung SSD 850
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>Disklabel type: gpt
>Disk identifier: 68F51B2D-C077-46D2-96A3-116BAF835998
>
>Device       Start       End   Sectors   Size Type
>/dev/sdb1     2048      6143      4096     2M BIOS boot
>/dev/sdb2     6144   2103295   2097152     1G Linux RAID
>/dev/sdb3  2103296 976773119 974669824 464.8G Linux RAID
>
>
>Disk /dev/sdc: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
>Disk model: ST4000VN008-2DR1
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>Disklabel type: gpt
>Disk identifier: D263D0C0-2C37-4205-B505-BC3B6DD1DFDA
>
>Device     Start        End    Sectors  Size Type
>/dev/sdc1   2048 7814035491 7814033444  3.7T Linux RAID
>
>
>Disk /dev/sdd: 3.65 TiB, 4000787030016 bytes, 7814037168 sectors
>Disk model: ST4000VN008-2DR1
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>Disklabel type: gpt
>Disk identifier: 84D9D98B-0C27-4090-B639-AB48A0FA9B1E
>
>Device     Start        End    Sectors  Size Type
>/dev/sdd1   2048 7814035491 7814033444  3.7T Linux RAID
>
>
>Disk /dev/md127: 464.65 GiB, 498895683584 bytes, 974405632 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>Disk /dev/md126: 1022 MiB, 1071644672 bytes, 2093056 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>Disk /dev/mapper/lv_os--main-root: 370 GiB, 397284474880 bytes, 775946240 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>Disk /dev/mapper/lv_os--main-swap: 8 GiB, 8589934592 bytes, 16777216 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 512 bytes
>I/O size (minimum/optimal): 512 bytes / 512 bytes
>
>
>
>
>Disk /dev/md125: 3.65 TiB, 4000649838592 bytes, 7813769216 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>
>
>Disk /dev/mapper/lv_big-00: 2.31 TiB, 2528880099328 bytes, 4939218944 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>
>
>Disk /dev/mapper/lv_big-var: 400 GiB, 429496729600 bytes, 838860800 sectors
>Units: sectors of 1 * 512 = 512 bytes
>Sector size (logical/physical): 512 bytes / 4096 bytes
>I/O size (minimum/optimal): 4096 bytes / 4096 bytes
>[root@loki ~]# mdadm --examine /dev/sd[a-d][0-9]  ###
>mdadm: No md superblock detected on /dev/sda1.
>/dev/sda2:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : a90395cd:6297a553:d29f32cd:04f3054c
>           Name : localhost-live:raid1_boot
>  Creation Time : Sat Sep 19 07:54:55 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 2093056 (1022.00 MiB 1071.64 MB)
>     Array Size : 1046528 (1022.00 MiB 1071.64 MB)
>    Data Offset : 4096 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=4016 sectors, after=0 sectors
>          State : clean
>    Device UUID : 3f3d4518:967ad3f5:99bdd519:5c7d079b
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Fri Sep 25 07:57:18 2020
>  Bad Block Log : 512 entries available at offset 16 sectors
>       Checksum : b6fe8069 - correct
>         Events : 83
>
>
>   Device Role : Active device 0
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sda3:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : 1ce0d10e:d281ceb8:a53a4dd4:b00d77b1
>           Name : localhost-live:raid1_os-main
>  Creation Time : Sat Sep 19 07:54:43 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 974405632 (464.63 GiB 498.90 GB)
>     Array Size : 487202816 (464.63 GiB 498.90 GB)
>    Data Offset : 264192 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=264112 sectors, after=0 sectors
>          State : clean
>    Device UUID : 8317645e:e13b28c5:215484a7:a5505b5d
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Sat Sep 26 05:40:38 2020
>  Bad Block Log : 512 entries available at offset 16 sectors
>       Checksum : cfc13cbc - correct
>         Events : 894
>
>
>   Device Role : Active device 0
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>mdadm: No md superblock detected on /dev/sdb1.
>/dev/sdb2:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : a90395cd:6297a553:d29f32cd:04f3054c
>           Name : localhost-live:raid1_boot
>  Creation Time : Sat Sep 19 07:54:55 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 2093056 (1022.00 MiB 1071.64 MB)
>     Array Size : 1046528 (1022.00 MiB 1071.64 MB)
>    Data Offset : 4096 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=4016 sectors, after=0 sectors
>          State : clean
>    Device UUID : c2d52945:bf91d15c:36a8dca5:ac76b9ec
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Fri Sep 25 07:57:18 2020
>  Bad Block Log : 512 entries available at offset 16 sectors
>       Checksum : 289a1404 - correct
>         Events : 83
>
>
>   Device Role : Active device 1
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sdb3:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : 1ce0d10e:d281ceb8:a53a4dd4:b00d77b1
>           Name : localhost-live:raid1_os-main
>  Creation Time : Sat Sep 19 07:54:43 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 974405632 (464.63 GiB 498.90 GB)
>     Array Size : 487202816 (464.63 GiB 498.90 GB)
>    Data Offset : 264192 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=264112 sectors, after=0 sectors
>          State : clean
>    Device UUID : dab22a3f:5528d6e6:9da9ce74:a32cf0a6
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Sat Sep 26 05:40:38 2020
>  Bad Block Log : 512 entries available at offset 16 sectors
>       Checksum : e914f602 - correct
>         Events : 894
>
>
>   Device Role : Active device 1
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sdc1:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : 434db08c:6fdf4494:19a21b16:6e662faf
>           Name : localhost-live:raid_big
>  Creation Time : Sat Sep 19 07:54:00 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 7813769252 (3725.90 GiB 4000.65 GB)
>     Array Size : 3906884608 (3725.90 GiB 4000.65 GB)
>  Used Dev Size : 7813769216 (3725.90 GiB 4000.65 GB)
>    Data Offset : 264192 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=264112 sectors, after=36 sectors
>          State : clean
>    Device UUID : 050a5fc5:b39e6e60:98f3ffad:71e38d71
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Sat Sep 26 06:09:58 2020
>  Bad Block Log : 512 entries available at offset 24 sectors
>       Checksum : 476865a4 - correct
>         Events : 43615
>
>
>   Device Role : Active device 0
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>/dev/sdd1:
>          Magic : a92b4efc
>        Version : 1.2
>    Feature Map : 0x1
>     Array UUID : 434db08c:6fdf4494:19a21b16:6e662faf
>           Name : localhost-live:raid_big
>  Creation Time : Sat Sep 19 07:54:00 2020
>     Raid Level : raid1
>   Raid Devices : 2
>
> Avail Dev Size : 7813769252 (3725.90 GiB 4000.65 GB)
>     Array Size : 3906884608 (3725.90 GiB 4000.65 GB)
>  Used Dev Size : 7813769216 (3725.90 GiB 4000.65 GB)
>    Data Offset : 264192 sectors
>   Super Offset : 8 sectors
>   Unused Space : before=264112 sectors, after=36 sectors
>          State : clean
>    Device UUID : ebb43ce0:bd684ff7:c8562ac6:8774a702
>
>Internal Bitmap : 8 sectors from superblock
>    Update Time : Sat Sep 26 06:09:58 2020
>  Bad Block Log : 512 entries available at offset 24 sectors
>       Checksum : a26acedb - correct
>         Events : 43615
>
>
>   Device Role : Active device 1
>   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
>[root@loki ~]# pvs
>  PV         VG         Fmt  Attr PSize    PFree
>  /dev/md125 lv_big     lvm2 a--    <3.64t 970.69g
>  /dev/md127 lv_os-main lvm2 a--  <464.63g <86.63g
>
>
>
>
>
>
>

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, back to index

Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-06 16:07 pls help/review: fed 32 | LVM over raid1, on SSDs & spinning disks linux-raid
2020-10-06 17:00 ` Reindl Harald
2020-10-06 22:24   ` antlists
2020-10-06 22:56   ` Roger Heflin
2020-10-07 11:18 ` linux-raid

Linux-Raid Archives on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-raid/0 linux-raid/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-raid linux-raid/ https://lore.kernel.org/linux-raid \
		linux-raid@vger.kernel.org
	public-inbox-index linux-raid

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-raid


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git