[patch v2 0/3] add --device option

[patch v2 0/3] add --device option

[dyoung-H+wXaHxf7aLQT0dZR+AlfA]

This series contains 3 patches.

1/3: wait host devs in base module
2/3: Add for_each_host_dev_and_slaves for device only checking
3/3: Add a dracut option --device to bring up a device in initramfs

this patch series fixes below problems:
move fstab wait_for_dev to 99base, convert to persistent name for initramfs
kdump iscsi setup fail due to check_block_and_slaves break if helper
    function success with one of the slaves. 
add --device option.

[patch v2 1/3] wait host devs in base module

[dyoung-H+wXaHxf7aLQT0dZR+AlfA]

[-- Attachment #1: 0816-1-wait-host-devs.patch --]
[-- Type: text/plain, Size: 3451 bytes --]

each dev in host_devs[] should be waited in initqueue to make sure they
are oneline before initqueue finish.

Add a new wait_host_devs.sh in base module to make this a generic thing.
Because all the devs in fstab lines are also added to host_devs, so no need
do same wait in fstab-sys module anymore.

[v2->v3]: do not add slave devices to host_devs
          wait for persistent dev name in initramfs

Signed-off-by: Dave Young <dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
 dracut.sh                               |   16 ++++++++++++++++
 modules.d/95fstab-sys/module-setup.sh   |    1 -
 modules.d/95fstab-sys/wait-mount-dev.sh |   22 ----------------------
 modules.d/99base/module-setup.sh        |    1 +
 modules.d/99base/wait-host-devs.sh      |   20 ++++++++++++++++++++
 5 files changed, 37 insertions(+), 23 deletions(-)

--- dracut.orig/dracut.sh
+++ dracut/dracut.sh
@@ -903,6 +903,22 @@ done
 
 dinfo "*** Including modules done ***"
 
+get_persistent_dev() {
+    local i _tmp
+    local _dev=${1##*/}
+
+    for i in /dev/disk/by-id/*; do
+        _tmp=$(readlink $i)
+        [ "${_tmp##*/}" = "$_dev" ] && echo $i && return
+    done
+}
+
+## save host_devs which we need bring up
+for _dev in ${host_devs[@]}; do
+    _pdev=$(get_persistent_dev $_dev)
+    [ -n "$_pdev" ] && echo $_pdev >> $initdir/etc/host_devs
+done
+
 ## final stuff that has to happen
 if [[ $no_kernel != yes ]]; then
 
--- dracut.orig/modules.d/95fstab-sys/module-setup.sh
+++ dracut/modules.d/95fstab-sys/module-setup.sh
@@ -13,5 +13,4 @@ depends() {
 install() {
     [ -f /etc/fstab.sys ] && inst_simple /etc/fstab.sys
     inst_hook pre-pivot 00 "$moddir/mount-sys.sh"
-    inst_hook cmdline 00 "$moddir/wait-mount-dev.sh"
 }
--- dracut.orig/modules.d/95fstab-sys/wait-mount-dev.sh
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/bin/sh
-# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
-# ex: ts=8 sw=4 sts=4 et filetype=sh
-
-type getarg >/dev/null 2>&1 || . /lib/dracut-lib.sh
-type det_fs >/dev/null 2>&1 || . /lib/fs-lib.sh
-
-fstab_wait_dev() {
-    local _dev _mp _fs _opts _dump _pass _rest
-    test -e "$1" || return 1
-    while read _dev _mp _fs _opts _dump _pass _rest; do
-        [ -z "${_dev%%#*}" ] && continue # Skip comment lines
-        case "$_dev" in
-        /dev/?*)
-            wait_for_dev $_dev;;
-        *) ;;
-        esac
-    done < $1
-    return 0
-}
-
-[ -f /etc/fstab ] && fstab_wait_dev /etc/fstab
--- dracut.orig/modules.d/99base/module-setup.sh
+++ dracut/modules.d/99base/module-setup.sh
@@ -41,6 +41,7 @@ install() {
     dracut_install switch_root || dfatal "Failed to install switch_root"
 
     inst_simple "$moddir/dracut-lib.sh" "/lib/dracut-lib.sh"
+    inst_hook cmdline 00 "$moddir/wait-host-devs.sh"
     inst_hook cmdline 10 "$moddir/parse-root-opts.sh"
     mkdir -p "${initdir}/var"
     [ -x /lib/systemd/systemd-timestamp ] && inst /lib/systemd/systemd-timestamp
--- /dev/null
+++ dracut/modules.d/99base/wait-host-devs.sh
@@ -0,0 +1,20 @@
+#!/bin/sh
+# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
+# ex: ts=8 sw=4 sts=4 et filetype=sh
+
+type getarg >/dev/null 2>&1 || . /lib/dracut-lib.sh
+
+wait_host_devs() {
+    local _dev
+
+    while read _dev; do
+        case "$_dev" in
+        /dev/?*)
+            wait_for_dev $_dev
+            ;;
+        *) ;;
+        esac
+    done < $1
+}
+
+[ -f /etc/host_devs ] && wait_host_devs /etc/host_devs

[patch v2 2/3] Add for_each_host_dev_and_slaves for device only checking

[dyoung-H+wXaHxf7aLQT0dZR+AlfA]

[-- Attachment #1: 0816-2-add-for-each-dev-and-slave.patch --]
[-- Type: text/plain, Size: 4997 bytes --]

For lvm, multipath, iscsi modules they do not care about the filesystem,
Also there could be devcie in host_devs but it does not get formated.

For these kind of modules, use for_each_host_dev_and_slaves will be better than use
for_each_host_dev_fs, here add a new function to iterate the host_devs and
their slave devices.

In original for_each_host_dev_fs, it will call check_block_and_slaves which
will return once helper function return 0, but this is not enough for kdump
iscsi setup. For kdump iscsi case, it need setup each slave devices so that
the iscsi target can be properly setuped in initramfs.

Thus, this patch also add new functions check_block_and_slaves_all and
for_each_host_dev_and_slaves_all.

Signed-off-by: Dave Young <dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Tested-by: WANG Chao <chaowang-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
 dracut-functions.sh                   |   48 +++++++++++++++++++++++++++++++++-
 modules.d/90lvm/module-setup.sh       |    4 +-
 modules.d/90multipath/module-setup.sh |    5 +--
 modules.d/95iscsi/module-setup.sh     |    5 +--
 4 files changed, 53 insertions(+), 9 deletions(-)

--- dracut.orig/dracut-functions.sh
+++ dracut/dracut-functions.sh
@@ -383,7 +383,6 @@ find_mp_fstype() {
     return 1
 }
 
-
 # finds the major:minor of the block device backing the root filesystem.
 find_root_block_device() { find_block_device /; }
 
@@ -429,6 +428,53 @@ check_block_and_slaves() {
     return 1
 }
 
+check_block_and_slaves_all() {
+    local _x _ret=1
+    [[ -b /dev/block/$2 ]] || return 1 # Not a block device? So sorry.
+    if "$1" $2; then
+          _ret=0
+    fi
+    check_vol_slaves "$@" && return 0
+    if [[ -f /sys/dev/block/$2/../dev ]]; then
+        check_block_and_slaves_all $1 $(cat "/sys/dev/block/$2/../dev") && _ret=0
+    fi
+    [[ -d /sys/dev/block/$2/slaves ]] || return 1
+    for _x in /sys/dev/block/$2/slaves/*/dev; do
+        [[ -f $_x ]] || continue
+        check_block_and_slaves_all $1 $(cat "$_x") && _ret=0
+    done
+    return $_ret
+}
+# for_each_host_dev_and_slaves <func>
+# Execute "<func> <dev>" for every "<dev>" found
+# in ${host_devs[@]} and their slaves
+for_each_host_dev_and_slaves_all()
+{
+    local _func="$1"
+    local _dev
+    local _ret=1
+    for _dev in ${host_devs[@]}; do
+        [[ -b "$_dev" ]] || continue
+        echo host_devs: $_dev
+        if check_block_and_slaves_all $_func $(get_maj_min $_dev); then
+               _ret=0
+        fi
+    done
+    return $_ret
+}
+
+for_each_host_dev_and_slaves()
+{
+    local _func="$1"
+    local _dev
+    for _dev in ${host_devs[@]}; do
+        [[ -b "$_dev" ]] || continue
+        echo host_devs: $_dev
+        check_block_and_slaves_all $_func $(get_maj_min $_dev) && return 0
+    done
+    return 1
+}
+
 # ugly workaround for the lvm design
 # There is no volume group device,
 # so, there are no slave devices for volume groups.
--- dracut.orig/modules.d/90lvm/module-setup.sh
+++ dracut/modules.d/90lvm/module-setup.sh
@@ -9,7 +9,7 @@ check() {
 
     check_lvm() {
         local DM_VG_NAME DM_LV_NAME DM_UDEV_DISABLE_DISK_RULES_FLAG
-        eval $(udevadm info --query=property --name=$1|egrep '(DM_VG_NAME|DM_LV_NAME|DM_UDEV_DISABLE_DISK_RULES_FLAG)=')
+        eval $(udevadm info --query=property --name=/dev/block/$1|egrep '(DM_VG_NAME|DM_LV_NAME|DM_UDEV_DISABLE_DISK_RULES_FLAG)=')
         [[ "$DM_UDEV_DISABLE_DISK_RULES_FLAG" = "1" ]] && return 1
         [[ ${DM_VG_NAME} ]] && [[ ${DM_LV_NAME} ]] || return 1
         if ! strstr " ${_activated[*]} " " ${DM_VG_NAME}/${DM_LV_NAME} "; then
@@ -22,7 +22,7 @@ check() {
     }
 
     [[ $hostonly ]] || [[ $mount_needs ]] && {
-        for_each_host_dev_fs check_lvm || return 1
+        for_each_host_dev_and_slaves check_lvm || return 1
     }
 
     return 0
--- dracut.orig/modules.d/90multipath/module-setup.sh
+++ dracut/modules.d/90multipath/module-setup.sh
@@ -8,15 +8,14 @@ check() {
     type -P multipath >/dev/null || return 1
 
     is_mpath() {
-        local _dev
-        _dev=$(get_maj_min $1)
+        local _dev=$1
         [ -e /sys/dev/block/$_dev/dm/uuid ] || return 1
         [[ $(cat /sys/dev/block/$_dev/dm/uuid) =~ mpath- ]] && return 0
         return 1
     }
 
     [[ $hostonly ]] || [[ $mount_needs ]] && {
-        for_each_host_dev_fs is_mpath || return 1
+        for_each_host_dev_and_slaves is_mpath || return 1
     }
 
     return 0
--- dracut.orig/modules.d/95iscsi/module-setup.sh
+++ dracut/modules.d/95iscsi/module-setup.sh
@@ -11,8 +11,7 @@ check() {
     # booting from root.
 
     is_iscsi() (
-        local _dev
-        _dev=$(get_maj_min $1)
+        local _dev=$1
 
         [[ -L /sys/dev/block/$_dev ]] || return
         cd "$(readlink -f /sys/dev/block/$_dev)"
@@ -23,7 +22,7 @@ check() {
     )
 
     [[ $hostonly ]] || [[ $mount_needs ]] && {
-        for_each_host_dev_fs is_iscsi || return 1
+        for_each_host_dev_and_slaves is_iscsi || return 1
     }
     return 0
 }

[patch v2 3/3] Add a dracut option --device to bring up a device in initramfs

[dyoung-H+wXaHxf7aLQT0dZR+AlfA]

[-- Attachment #1: 0816-3-add-raw-device.patch --]
[-- Type: TEXT/PLAIN, Size: 2311 bytes --]

Kdump support dump to raw device which could be on top of complex storage
such as multipath and iscsi which are standalone dracut modules.

Add a --device option to dracut which will add the device to host_devs so
dracut can add the dependent modules automaticlly and enable them in initramfs.

--device will accept device node name as the param.

Signed-off-by: Dave Young <dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Tested-by: Chao Wang <chaowang-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
---
 dracut.8.asc |    3 +++
 dracut.sh    |    3 +++
 2 files changed, 6 insertions(+)

--- dracut.orig/dracut.sh
+++ dracut/dracut.sh
@@ -133,6 +133,7 @@ Creates initial ramdisk images for prelo
   --mount "[DEV] [MP] [FSTYPE] [FSOPTS]"
                         Mount device [DEV] on mountpoint [MP] with filesystem
                         [FSTYPE] and options [FSOPTS] in the initramfs
+  --device "[DEV]"      Bring up [DEV] in initramfs
   -i, --include [SOURCE] [TARGET]
                         Include the files in the SOURCE directory into the
                          Target directory in the final initramfs.
@@ -261,6 +262,7 @@ TEMP=$(unset POSIXLY_CORRECT; getopt \
     --long fscks: \
     --long add-fstab: \
     --long mount: \
+    --long device: \
     --long nofscks: \
     --long ro-mnt \
     --long kmoddir: \
@@ -327,6 +329,7 @@ while :; do
         --fscks)       push fscks_l              "$2"; shift;;
         --add-fstab)   push add_fstab_l          "$2"; shift;;
         --mount)       push fstab_lines          "$2"; shift;;
+        --device)      push host_devs            "$2"; shift;;
         --nofscks)     nofscks_l="yes";;
         --ro-mnt)      ro_mnt_l="yes";;
         -k|--kmoddir)  drivers_dir_l="$2"; shift;;
--- dracut.orig/dracut.8.asc
+++ dracut/dracut.8.asc
@@ -321,6 +321,9 @@ provide a valid _/etc/fstab_.
     Mount _<device>_ on _<mountpoint>_ with _<filesystem type>_ and _<filesystem
     options>_ in the initramfs
 
+**--device** _<device>_ ::
+    Bring up _<device>_ in initramfs, _<device>_ should be the device name
+
 **-i, --include** _<SOURCE>_ _<TARGET>_::
     include the files in the SOURCE directory into the
     TARGET directory in the final initramfs. If SOURCE is a file, it will be

Re: [patch v2 2/3] Add for_each_host_dev_and_slaves for device only checking

[Vivek Goyal]

On Thu, Aug 23, 2012 at 11:02:23AM +0800, dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org wrote:
> For lvm, multipath, iscsi modules they do not care about the filesystem,
> Also there could be devcie in host_devs but it does not get formated.
> 
> For these kind of modules, use for_each_host_dev_and_slaves will be better than use
> for_each_host_dev_fs, here add a new function to iterate the host_devs and
> their slave devices.
> 
> In original for_each_host_dev_fs, it will call check_block_and_slaves which
> will return once helper function return 0, but this is not enough for kdump
> iscsi setup. For kdump iscsi case, it need setup each slave devices so that
> the iscsi target can be properly setuped in initramfs.
> 
> Thus, this patch also add new functions check_block_and_slaves_all and
> for_each_host_dev_and_slaves_all.

I think this patch should be broken in two parts for more clarity.

- Fix the kdump iscsi issue where we don't expect to break out of the
  loop the moment first iscsi device is found.

- Start using host_dev instead of host_dev_fs for modules which don't
  care about fs. Also why to have host_devs[] and host_fs_types[] both.
  Looks like host_fs_types contains both device and fs information. I
  think that includes strings like LVM etc.

  So can we merge both into one and those modules who don't require fs
  info will ignore it. Or, keep one data structure host_dev_fs_types, 
  and provide two helper functions. One which provdes on $dev in $1
  and other which provides both dev and fs as argument to the function.

  This is more of a clean up thing and it is not necessary for this
  patchset. So if you find it is too much of work, just split this patch
  in two for clarity. Also put some comments about what's the difference
  between  for_each_host_dev_and_slaves() and for_each_host_dev_and_slaves_all()

Thanks
Vivek

> 
> Signed-off-by: Dave Young <dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> Tested-by: WANG Chao <chaowang-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> ---
>  dracut-functions.sh                   |   48 +++++++++++++++++++++++++++++++++-
>  modules.d/90lvm/module-setup.sh       |    4 +-
>  modules.d/90multipath/module-setup.sh |    5 +--
>  modules.d/95iscsi/module-setup.sh     |    5 +--
>  4 files changed, 53 insertions(+), 9 deletions(-)
> 
> --- dracut.orig/dracut-functions.sh
> +++ dracut/dracut-functions.sh
> @@ -383,7 +383,6 @@ find_mp_fstype() {
>      return 1
>  }
>  
> -
>  # finds the major:minor of the block device backing the root filesystem.
>  find_root_block_device() { find_block_device /; }
>  
> @@ -429,6 +428,53 @@ check_block_and_slaves() {
>      return 1
>  }
>  
> +check_block_and_slaves_all() {
> +    local _x _ret=1
> +    [[ -b /dev/block/$2 ]] || return 1 # Not a block device? So sorry.
> +    if "$1" $2; then
> +          _ret=0
> +    fi
> +    check_vol_slaves "$@" && return 0
> +    if [[ -f /sys/dev/block/$2/../dev ]]; then
> +        check_block_and_slaves_all $1 $(cat "/sys/dev/block/$2/../dev") && _ret=0
> +    fi
> +    [[ -d /sys/dev/block/$2/slaves ]] || return 1
> +    for _x in /sys/dev/block/$2/slaves/*/dev; do
> +        [[ -f $_x ]] || continue
> +        check_block_and_slaves_all $1 $(cat "$_x") && _ret=0
> +    done
> +    return $_ret
> +}
> +# for_each_host_dev_and_slaves <func>
> +# Execute "<func> <dev>" for every "<dev>" found
> +# in ${host_devs[@]} and their slaves
> +for_each_host_dev_and_slaves_all()
> +{
> +    local _func="$1"
> +    local _dev
> +    local _ret=1
> +    for _dev in ${host_devs[@]}; do
> +        [[ -b "$_dev" ]] || continue
> +        echo host_devs: $_dev
> +        if check_block_and_slaves_all $_func $(get_maj_min $_dev); then
> +               _ret=0
> +        fi
> +    done
> +    return $_ret
> +}
> +
> +for_each_host_dev_and_slaves()
> +{
> +    local _func="$1"
> +    local _dev
> +    for _dev in ${host_devs[@]}; do
> +        [[ -b "$_dev" ]] || continue
> +        echo host_devs: $_dev
> +        check_block_and_slaves_all $_func $(get_maj_min $_dev) && return 0
> +    done
> +    return 1
> +}
> +
>  # ugly workaround for the lvm design
>  # There is no volume group device,
>  # so, there are no slave devices for volume groups.
> --- dracut.orig/modules.d/90lvm/module-setup.sh
> +++ dracut/modules.d/90lvm/module-setup.sh
> @@ -9,7 +9,7 @@ check() {
>  
>      check_lvm() {
>          local DM_VG_NAME DM_LV_NAME DM_UDEV_DISABLE_DISK_RULES_FLAG
> -        eval $(udevadm info --query=property --name=$1|egrep '(DM_VG_NAME|DM_LV_NAME|DM_UDEV_DISABLE_DISK_RULES_FLAG)=')
> +        eval $(udevadm info --query=property --name=/dev/block/$1|egrep '(DM_VG_NAME|DM_LV_NAME|DM_UDEV_DISABLE_DISK_RULES_FLAG)=')
>          [[ "$DM_UDEV_DISABLE_DISK_RULES_FLAG" = "1" ]] && return 1
>          [[ ${DM_VG_NAME} ]] && [[ ${DM_LV_NAME} ]] || return 1
>          if ! strstr " ${_activated[*]} " " ${DM_VG_NAME}/${DM_LV_NAME} "; then
> @@ -22,7 +22,7 @@ check() {
>      }
>  
>      [[ $hostonly ]] || [[ $mount_needs ]] && {
> -        for_each_host_dev_fs check_lvm || return 1
> +        for_each_host_dev_and_slaves check_lvm || return 1
>      }
>  
>      return 0
> --- dracut.orig/modules.d/90multipath/module-setup.sh
> +++ dracut/modules.d/90multipath/module-setup.sh
> @@ -8,15 +8,14 @@ check() {
>      type -P multipath >/dev/null || return 1
>  
>      is_mpath() {
> -        local _dev
> -        _dev=$(get_maj_min $1)
> +        local _dev=$1
>          [ -e /sys/dev/block/$_dev/dm/uuid ] || return 1
>          [[ $(cat /sys/dev/block/$_dev/dm/uuid) =~ mpath- ]] && return 0
>          return 1
>      }
>  
>      [[ $hostonly ]] || [[ $mount_needs ]] && {
> -        for_each_host_dev_fs is_mpath || return 1
> +        for_each_host_dev_and_slaves is_mpath || return 1
>      }
>  
>      return 0
> --- dracut.orig/modules.d/95iscsi/module-setup.sh
> +++ dracut/modules.d/95iscsi/module-setup.sh
> @@ -11,8 +11,7 @@ check() {
>      # booting from root.
>  
>      is_iscsi() (
> -        local _dev
> -        _dev=$(get_maj_min $1)
> +        local _dev=$1
>  
>          [[ -L /sys/dev/block/$_dev ]] || return
>          cd "$(readlink -f /sys/dev/block/$_dev)"
> @@ -23,7 +22,7 @@ check() {
>      )
>  
>      [[ $hostonly ]] || [[ $mount_needs ]] && {
> -        for_each_host_dev_fs is_iscsi || return 1
> +        for_each_host_dev_and_slaves is_iscsi || return 1
>      }
>      return 0
>  }

Re: [patch v2 1/3] wait host devs in base module

[Vivek Goyal]

On Thu, Aug 23, 2012 at 11:02:22AM +0800, dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org wrote:
> each dev in host_devs[] should be waited in initqueue to make sure they
> are oneline before initqueue finish.
> 
> Add a new wait_host_devs.sh in base module to make this a generic thing.
> Because all the devs in fstab lines are also added to host_devs, so no need
> do same wait in fstab-sys module anymore.
> 
> [v2->v3]: do not add slave devices to host_devs
>           wait for persistent dev name in initramfs
> 
> Signed-off-by: Dave Young <dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> ---
>  dracut.sh                               |   16 ++++++++++++++++
>  modules.d/95fstab-sys/module-setup.sh   |    1 -
>  modules.d/95fstab-sys/wait-mount-dev.sh |   22 ----------------------
>  modules.d/99base/module-setup.sh        |    1 +
>  modules.d/99base/wait-host-devs.sh      |   20 ++++++++++++++++++++
>  5 files changed, 37 insertions(+), 23 deletions(-)
> 
> --- dracut.orig/dracut.sh
> +++ dracut/dracut.sh
> @@ -903,6 +903,22 @@ done
>  
>  dinfo "*** Including modules done ***"
>  
> +get_persistent_dev() {
> +    local i _tmp
> +    local _dev=${1##*/}
> +
> +    for i in /dev/disk/by-id/*; do
> +        _tmp=$(readlink $i)
> +        [ "${_tmp##*/}" = "$_dev" ] && echo $i && return
> +    done
> +}
> +

I am wondering that will it be better to rely on user to pass the
persistent device name instead of we trying to convert it into a name
generated by udev.

I kind of prefer to let user provide the persistent name instead of 
dong tricks internally. (Either in fstab or using --device option).

Thanks
Vivek

> +## save host_devs which we need bring up
> +for _dev in ${host_devs[@]}; do
> +    _pdev=$(get_persistent_dev $_dev)
> +    [ -n "$_pdev" ] && echo $_pdev >> $initdir/etc/host_devs
> +done
> +
>  ## final stuff that has to happen
>  if [[ $no_kernel != yes ]]; then
>  
> --- dracut.orig/modules.d/95fstab-sys/module-setup.sh
> +++ dracut/modules.d/95fstab-sys/module-setup.sh
> @@ -13,5 +13,4 @@ depends() {
>  install() {
>      [ -f /etc/fstab.sys ] && inst_simple /etc/fstab.sys
>      inst_hook pre-pivot 00 "$moddir/mount-sys.sh"
> -    inst_hook cmdline 00 "$moddir/wait-mount-dev.sh"
>  }
> --- dracut.orig/modules.d/95fstab-sys/wait-mount-dev.sh
> +++ /dev/null
> @@ -1,22 +0,0 @@
> -#!/bin/sh
> -# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
> -# ex: ts=8 sw=4 sts=4 et filetype=sh
> -
> -type getarg >/dev/null 2>&1 || . /lib/dracut-lib.sh
> -type det_fs >/dev/null 2>&1 || . /lib/fs-lib.sh
> -
> -fstab_wait_dev() {
> -    local _dev _mp _fs _opts _dump _pass _rest
> -    test -e "$1" || return 1
> -    while read _dev _mp _fs _opts _dump _pass _rest; do
> -        [ -z "${_dev%%#*}" ] && continue # Skip comment lines
> -        case "$_dev" in
> -        /dev/?*)
> -            wait_for_dev $_dev;;
> -        *) ;;
> -        esac
> -    done < $1
> -    return 0
> -}
> -
> -[ -f /etc/fstab ] && fstab_wait_dev /etc/fstab
> --- dracut.orig/modules.d/99base/module-setup.sh
> +++ dracut/modules.d/99base/module-setup.sh
> @@ -41,6 +41,7 @@ install() {
>      dracut_install switch_root || dfatal "Failed to install switch_root"
>  
>      inst_simple "$moddir/dracut-lib.sh" "/lib/dracut-lib.sh"
> +    inst_hook cmdline 00 "$moddir/wait-host-devs.sh"
>      inst_hook cmdline 10 "$moddir/parse-root-opts.sh"
>      mkdir -p "${initdir}/var"
>      [ -x /lib/systemd/systemd-timestamp ] && inst /lib/systemd/systemd-timestamp
> --- /dev/null
> +++ dracut/modules.d/99base/wait-host-devs.sh
> @@ -0,0 +1,20 @@
> +#!/bin/sh
> +# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
> +# ex: ts=8 sw=4 sts=4 et filetype=sh
> +
> +type getarg >/dev/null 2>&1 || . /lib/dracut-lib.sh
> +
> +wait_host_devs() {
> +    local _dev
> +
> +    while read _dev; do
> +        case "$_dev" in
> +        /dev/?*)
> +            wait_for_dev $_dev
> +            ;;
> +        *) ;;
> +        esac
> +    done < $1
> +}
> +
> +[ -f /etc/host_devs ] && wait_host_devs /etc/host_devs

Re: [patch v2 1/3] wait host devs in base module

[Dave Young]

On 08/23/2012 11:31 PM, Vivek Goyal wrote:

> On Thu, Aug 23, 2012 at 11:02:22AM +0800, dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org wrote:
>> each dev in host_devs[] should be waited in initqueue to make sure they
>> are oneline before initqueue finish.
>>
>> Add a new wait_host_devs.sh in base module to make this a generic thing.
>> Because all the devs in fstab lines are also added to host_devs, so no need
>> do same wait in fstab-sys module anymore.
>>
>> [v2->v3]: do not add slave devices to host_devs
>>           wait for persistent dev name in initramfs
>>
>> Signed-off-by: Dave Young <dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>> ---
>>  dracut.sh                               |   16 ++++++++++++++++
>>  modules.d/95fstab-sys/module-setup.sh   |    1 -
>>  modules.d/95fstab-sys/wait-mount-dev.sh |   22 ----------------------
>>  modules.d/99base/module-setup.sh        |    1 +
>>  modules.d/99base/wait-host-devs.sh      |   20 ++++++++++++++++++++
>>  5 files changed, 37 insertions(+), 23 deletions(-)
>>
>> --- dracut.orig/dracut.sh
>> +++ dracut/dracut.sh
>> @@ -903,6 +903,22 @@ done
>>  
>>  dinfo "*** Including modules done ***"
>>  
>> +get_persistent_dev() {
>> +    local i _tmp
>> +    local _dev=${1##*/}
>> +
>> +    for i in /dev/disk/by-id/*; do
>> +        _tmp=$(readlink $i)
>> +        [ "${_tmp##*/}" = "$_dev" ] && echo $i && return
>> +    done
>> +}
>> +
> 
> I am wondering that will it be better to rely on user to pass the
> persistent device name instead of we trying to convert it into a name
> generated by udev.
> 
> I kind of prefer to let user provide the persistent name instead of 
> dong tricks internally. (Either in fstab or using --device option).


This patchset make the wait_for_dev globally, it will wait for all
devices in host_devs.  This not only limited to fstab and --device
passed items, such as below code, for these system mountpoint it's still
necessary to find the udev name if we want to wait them in initramfs:

    push host_mp \
        "/" \
        "/etc" \
        "/usr" \
        "/usr/bin" \
        "/usr/sbin" \
        "/usr/lib" \
        "/usr/lib64" \
        "/boot"

    for mp in "${host_mp[@]}"; do
        mountpoint "$mp" >/dev/null 2>&1 || continue
        push host_devs $(readlink -f "/dev/block/$(find_block_device
"$mp")")
    done


-- 
Thanks
Dave

Re: [patch v2 2/3] Add for_each_host_dev_and_slaves for device only checking

[Dave Young]

On 08/23/2012 11:26 PM, Vivek Goyal wrote:

> On Thu, Aug 23, 2012 at 11:02:23AM +0800, dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org wrote:
>> For lvm, multipath, iscsi modules they do not care about the filesystem,
>> Also there could be devcie in host_devs but it does not get formated.
>>
>> For these kind of modules, use for_each_host_dev_and_slaves will be better than use
>> for_each_host_dev_fs, here add a new function to iterate the host_devs and
>> their slave devices.
>>
>> In original for_each_host_dev_fs, it will call check_block_and_slaves which
>> will return once helper function return 0, but this is not enough for kdump
>> iscsi setup. For kdump iscsi case, it need setup each slave devices so that
>> the iscsi target can be properly setuped in initramfs.


Firstly, fix myself, check_block_and_slaves is not called in
for_each_host_dev_fs. It's called when dracut.sh collect and push
host_fs_types, so because there's no enough slave devices in
host_fs_types is_iscsi will only check the target in host_fs_types.

>>
>> Thus, this patch also add new functions check_block_and_slaves_all and
>> for_each_host_dev_and_slaves_all.
> 
> I think this patch should be broken in two parts for more clarity.


Will do

> 
> - Fix the kdump iscsi issue where we don't expect to break out of the
>   loop the moment first iscsi device is found.
> 
> - Start using host_dev instead of host_dev_fs for modules which don't
>   care about fs. Also why to have host_devs[] and host_fs_types[] both.
>   Looks like host_fs_types contains both device and fs information. I
>   think that includes strings like LVM etc.
> 
>   So can we merge both into one and those modules who don't require fs
>   info will ignore it. Or, keep one data structure host_dev_fs_types, 
>   and provide two helper functions. One which provdes on $dev in $1
>   and other which provides both dev and fs as argument to the function.


Will try the host_fs_types with optional fs type like "dev|[fs]"

-- 
Thanks
Dave

Re: [patch v2 2/3] Add for_each_host_dev_and_slaves for device only checking

[Dave Young]

On 08/27/2012 02:57 PM, Dave Young wrote:

> On 08/23/2012 11:26 PM, Vivek Goyal wrote:
> 
>> On Thu, Aug 23, 2012 at 11:02:23AM +0800, dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org wrote:
>>> For lvm, multipath, iscsi modules they do not care about the filesystem,
>>> Also there could be devcie in host_devs but it does not get formated.
>>>
>>> For these kind of modules, use for_each_host_dev_and_slaves will be better than use
>>> for_each_host_dev_fs, here add a new function to iterate the host_devs and
>>> their slave devices.
>>>
>>> In original for_each_host_dev_fs, it will call check_block_and_slaves which
>>> will return once helper function return 0, but this is not enough for kdump
>>> iscsi setup. For kdump iscsi case, it need setup each slave devices so that
>>> the iscsi target can be properly setuped in initramfs.
> 
> 
> Firstly, fix myself, check_block_and_slaves is not called in
> for_each_host_dev_fs. It's called when dracut.sh collect and push
> host_fs_types, so because there's no enough slave devices in
> host_fs_types is_iscsi will only check the target in host_fs_types.
> 
>>>
>>> Thus, this patch also add new functions check_block_and_slaves_all and
>>> for_each_host_dev_and_slaves_all.
>>
>> I think this patch should be broken in two parts for more clarity.
> 
> 
> Will do
> 
>>
>> - Fix the kdump iscsi issue where we don't expect to break out of the
>>   loop the moment first iscsi device is found.
>>
>> - Start using host_dev instead of host_dev_fs for modules which don't
>>   care about fs. Also why to have host_devs[] and host_fs_types[] both.
>>   Looks like host_fs_types contains both device and fs information. I
>>   think that includes strings like LVM etc.
>>
>>   So can we merge both into one and those modules who don't require fs
>>   info will ignore it. Or, keep one data structure host_dev_fs_types, 
>>   and provide two helper functions. One which provdes on $dev in $1
>>   and other which provides both dev and fs as argument to the function.
> 
> 
> Will try the host_fs_types with optional fs type like "dev|[fs]"
> 


Rethinking about this, it's not possible to fix this with host_fs_types.
Because dracut.sh check_block_and_slaves then add them to host_devs, if
we are going to use host_fs_types then we need push slaves into
host_devs as well, thus the patch 1/3 will wait for all host devices
including slaves...

This patchset works just because I created the new for_each_host_dev_and
slaves so it does not need slaves in host_devs.


So I would only split this patch into 2 patches without change the logic
any more...

-- 
Thanks
Dave

Re: [patch v2 1/3] wait host devs in base module

[Vivek Goyal]

On Mon, Aug 27, 2012 at 02:35:28PM +0800, Dave Young wrote:
> On 08/23/2012 11:31 PM, Vivek Goyal wrote:
> 
> > On Thu, Aug 23, 2012 at 11:02:22AM +0800, dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org wrote:
> >> each dev in host_devs[] should be waited in initqueue to make sure they
> >> are oneline before initqueue finish.
> >>
> >> Add a new wait_host_devs.sh in base module to make this a generic thing.
> >> Because all the devs in fstab lines are also added to host_devs, so no need
> >> do same wait in fstab-sys module anymore.
> >>
> >> [v2->v3]: do not add slave devices to host_devs
> >>           wait for persistent dev name in initramfs
> >>
> >> Signed-off-by: Dave Young <dyoung-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> >> ---
> >>  dracut.sh                               |   16 ++++++++++++++++
> >>  modules.d/95fstab-sys/module-setup.sh   |    1 -
> >>  modules.d/95fstab-sys/wait-mount-dev.sh |   22 ----------------------
> >>  modules.d/99base/module-setup.sh        |    1 +
> >>  modules.d/99base/wait-host-devs.sh      |   20 ++++++++++++++++++++
> >>  5 files changed, 37 insertions(+), 23 deletions(-)
> >>
> >> --- dracut.orig/dracut.sh
> >> +++ dracut/dracut.sh
> >> @@ -903,6 +903,22 @@ done
> >>  
> >>  dinfo "*** Including modules done ***"
> >>  
> >> +get_persistent_dev() {
> >> +    local i _tmp
> >> +    local _dev=${1##*/}
> >> +
> >> +    for i in /dev/disk/by-id/*; do
> >> +        _tmp=$(readlink $i)
> >> +        [ "${_tmp##*/}" = "$_dev" ] && echo $i && return
> >> +    done
> >> +}
> >> +
> > 
> > I am wondering that will it be better to rely on user to pass the
> > persistent device name instead of we trying to convert it into a name
> > generated by udev.
> > 
> > I kind of prefer to let user provide the persistent name instead of 
> > dong tricks internally. (Either in fstab or using --device option).
> 
> 
> This patchset make the wait_for_dev globally, it will wait for all
> devices in host_devs.  This not only limited to fstab and --device
> passed items, such as below code, for these system mountpoint it's still
> necessary to find the udev name if we want to wait them in initramfs:
> 
>     push host_mp \
>         "/" \
>         "/etc" \
>         "/usr" \
>         "/usr/bin" \
>         "/usr/sbin" \
>         "/usr/lib" \
>         "/usr/lib64" \
>         "/boot"
> 
>     for mp in "${host_mp[@]}"; do
>         mountpoint "$mp" >/dev/null 2>&1 || continue
>         push host_devs $(readlink -f "/dev/block/$(find_block_device
> "$mp")")
>     done

- If these filesystems are mounted on different devices, they should be
  part of /etc/fstab and shouldn't fstab module take care of making sure
  underlying devices are up.

Anyway, I am not too particular about this. So go ahead and post next
version of patches and let harald decide whether he likes it or not.

Vivek

Re: [patch v2 2/3] Add for_each_host_dev_and_slaves for device only checking

[Vivek Goyal]

On Mon, Aug 27, 2012 at 04:00:42PM +0800, Dave Young wrote:

[..]
> Rethinking about this, it's not possible to fix this with host_fs_types.
> Because dracut.sh check_block_and_slaves then add them to host_devs, if
> we are going to use host_fs_types then we need push slaves into
> host_devs as well, thus the patch 1/3 will wait for all host devices
> including slaves...
> 
> This patchset works just because I created the new for_each_host_dev_and
> slaves so it does not need slaves in host_devs.
> 
> 
> So I would only split this patch into 2 patches without change the logic
> any more...

Looks like Harald has already pulled in this patch series. So if you
want to do any cleanup, it will have to be on top of this patch series.

Thanks
Vivek