It takes me up to 20 reboots for the system to start sda

It takes me up to 20 reboots for the system to start sda

[Bearcat Şándor]

Folks,

I have a btrfs raid 10 spread out among 4 ssds.  The main drive is
sda. When i reboot my system after the kernel loads i see "a start job
is running for dev-sda1.device.  I've let this sit for 3 hours with no
progress into the boot, and it seems to be a race condition of some
sort.

If i restart around 20 times one of them will let the system continue
booting, though i've been lucky enough to have it happen in 4. I have
the following systemd unit:
/etc/systemd/system/local-fs-pre.target.wants/btrfs-dev-scan.service

[Unit]
Description=Btrfs scan devices
Before=local-fs-pre.target
DefaultDependencies=false

[Service]
Type=oneshot
ExecStart=/sbin/btrfs device scan

[Install]
WantedBy=local-fs-pre.target

These 4 disks were joined to /dev/sda
   btrfs device add -f /dev/sdc /dev/sdd /dev/sde /

followed by
    btrfs balance start -dconvert=raid10 -mconvert=raid10 /

my fstab contains:
/dev/sdf1 /boot vfat defaults,noatime,discard 0 2
/dev/sda / btrfs defaults,noatime,discard,compress=zlib,autodefrag 0 1
/dev/sdb  /home/hometheater btrfs
defaults,noatime,discard,compress=zlib,autodefrag 0 2


Any ideas would be appreciated. For what it's worth i have smartd running.

Thank you,

-- 
Bearcat M. Şándor
Feline Soul Systems LLC
Voice: 872.CAT.SOUL (872.228.7685)
Fax: 406.235.7070

Re: It takes me up to 20 reboots for the system to start sda

[Kai Krakow]

Am Thu, 27 Oct 2016 19:17:52 -0600
schrieb Bearcat Şándor <bearcatsandor@gmail.com>:

> Folks,
> 
> I have a btrfs raid 10 spread out among 4 ssds.  The main drive is
> sda. When i reboot my system after the kernel loads i see "a start job
> is running for dev-sda1.device.  I've let this sit for 3 hours with no
> progress into the boot, and it seems to be a race condition of some
> sort.
> 
> If i restart around 20 times one of them will let the system continue
> booting, though i've been lucky enough to have it happen in 4. I have
> the following systemd unit:
> /etc/systemd/system/local-fs-pre.target.wants/btrfs-dev-scan.service
> 
> [Unit]
> Description=Btrfs scan devices
> Before=local-fs-pre.target
> DefaultDependencies=false
> 
> [Service]
> Type=oneshot
> ExecStart=/sbin/btrfs device scan
> 
> [Install]
> WantedBy=local-fs-pre.target
> 
> These 4 disks were joined to /dev/sda
>    btrfs device add -f /dev/sdc /dev/sdd /dev/sde /
> 
> followed by
>     btrfs balance start -dconvert=raid10 -mconvert=raid10 /
> 
> my fstab contains:
> /dev/sdf1 /boot vfat defaults,noatime,discard 0 2
> /dev/sda / btrfs defaults,noatime,discard,compress=zlib,autodefrag 0 1
> /dev/sdb  /home/hometheater btrfs
> defaults,noatime,discard,compress=zlib,autodefrag 0 2
> 
> 
> Any ideas would be appreciated. For what it's worth i have smartd
> running.

Try kernel parameter rootdelay=2 (or maybe higher value). I had this,
too, back a while. It seems to be fixed in later versions of dracut.
Which initramfs are you using?

I guess sda is not available when btrfs-scan runs, and thus the mount
will be incomplete and systemd just waits.

-- 
Regards,
Kai

Replies to list-only preferred.

Re: It takes me up to 20 reboots for the system to start sda

[Bearcat Şándor]

Thanks for the suggestions Kai.  I'm using dracut 044.

I tried using the rootdelay=2 kernel parameter to little effect other
than slowing down my booting.

I found this, which may be related:
https://github.com/dracutdevs/dracut/issues/149 (btrfs raid on rootdev
is unreliably mounted).

On Thu, Oct 27, 2016 at 7:17 PM, Bearcat Şándor <bearcatsandor@gmail.com> wrote:
> Folks,
>
> I have a btrfs raid 10 spread out among 4 ssds.  The main drive is
> sda. When i reboot my system after the kernel loads i see "a start job
> is running for dev-sda1.device.  I've let this sit for 3 hours with no
> progress into the boot, and it seems to be a race condition of some
> sort.
>
> If i restart around 20 times one of them will let the system continue
> booting, though i've been lucky enough to have it happen in 4. I have
> the following systemd unit:
> /etc/systemd/system/local-fs-pre.target.wants/btrfs-dev-scan.service
>
> [Unit]
> Description=Btrfs scan devices
> Before=local-fs-pre.target
> DefaultDependencies=false
>
> [Service]
> Type=oneshot
> ExecStart=/sbin/btrfs device scan
>
> [Install]
> WantedBy=local-fs-pre.target
>
> These 4 disks were joined to /dev/sda
>    btrfs device add -f /dev/sdc /dev/sdd /dev/sde /
>
> followed by
>     btrfs balance start -dconvert=raid10 -mconvert=raid10 /
>
> my fstab contains:
> /dev/sdf1 /boot vfat defaults,noatime,discard 0 2
> /dev/sda / btrfs defaults,noatime,discard,compress=zlib,autodefrag 0 1
> /dev/sdb  /home/hometheater btrfs
> defaults,noatime,discard,compress=zlib,autodefrag 0 2
>
>
> Any ideas would be appreciated. For what it's worth i have smartd running.
>
> Thank you,
>
> --
> Bearcat M. Şándor
> Feline Soul Systems LLC
> Voice: 872.CAT.SOUL (872.228.7685)
> Fax: 406.235.7070



-- 
Bearcat M. Şándor
Feline Soul Systems LLC
Voice: 872.CAT.SOUL (872.228.7685)
Fax: 406.235.7070

Re: It takes me up to 20 reboots for the system to start sda

[Kai Krakow]

Am Fri, 28 Oct 2016 02:21:27 -0600
schrieb Bearcat Şándor <bearcatsandor@gmail.com>:

> Thanks for the suggestions Kai.  I'm using dracut 044.
> 
> I tried using the rootdelay=2 kernel parameter to little effect other
> than slowing down my booting.
> 
> I found this, which may be related:
> https://github.com/dracutdevs/dracut/issues/149 (btrfs raid on rootdev
> is unreliably mounted).

Ah okay, then I maybe fixed it by setting grub not to use UUIDs for
mounting:

$ cat /etc/default/grub
...
GRUB_DISABLE_LINUX_UUID="true"
GRUB_DEVICE="LABEL=system"

Additionally I manually forced mounting by label (of course you need to
set a label for your filesystem). And I've set rootfstype so it won't
try all the other different filesystems first (which leads to a lot of
error messages in dmesg).

But I later switched to systemd-boot. But it also used labels for
mounting root. It looks like this:

title      Gentoo/Linux 13.0
version    4.8.4-gentoo
machine-id 121b87ca633e8ac0016656680000001b
linux      /vmlinuz-4.8.4-gentoo
initrd     /initramfs-4.8.4-gentoo.img
options    root=LABEL=system rootfstype=btrfs rootflags=compress=lzo,nossd,autodefrag,noatime zswap.enabled=1 splash loglevel=3

As you can see, I no longer need rootdelay or rootwait as parameters.


> On Thu, Oct 27, 2016 at 7:17 PM, Bearcat Şándor
> <bearcatsandor@gmail.com> wrote:
> > Folks,
> >
> > I have a btrfs raid 10 spread out among 4 ssds.  The main drive is
> > sda. When i reboot my system after the kernel loads i see "a start
> > job is running for dev-sda1.device.  I've let this sit for 3 hours
> > with no progress into the boot, and it seems to be a race condition
> > of some sort.
> >
> > If i restart around 20 times one of them will let the system
> > continue booting, though i've been lucky enough to have it happen
> > in 4. I have the following systemd unit:
> > /etc/systemd/system/local-fs-pre.target.wants/btrfs-dev-scan.service
> >
> > [Unit]
> > Description=Btrfs scan devices
> > Before=local-fs-pre.target
> > DefaultDependencies=false
> >
> > [Service]
> > Type=oneshot
> > ExecStart=/sbin/btrfs device scan
> >
> > [Install]
> > WantedBy=local-fs-pre.target
> >
> > These 4 disks were joined to /dev/sda
> >    btrfs device add -f /dev/sdc /dev/sdd /dev/sde /
> >
> > followed by
> >     btrfs balance start -dconvert=raid10 -mconvert=raid10 /
> >
> > my fstab contains:
> > /dev/sdf1 /boot vfat defaults,noatime,discard 0 2
> > /dev/sda / btrfs defaults,noatime,discard,compress=zlib,autodefrag
> > 0 1 /dev/sdb  /home/hometheater btrfs
> > defaults,noatime,discard,compress=zlib,autodefrag 0 2
> >
> >
> > Any ideas would be appreciated. For what it's worth i have smartd
> > running.
> >
> > Thank you,
> >
> > --
> > Bearcat M. Şándor
> > Feline Soul Systems LLC
> > Voice: 872.CAT.SOUL (872.228.7685)
> > Fax: 406.235.7070  
> 
> 
> 



-- 
Regards,
Kai

Replies to list-only preferred.