Out of tree drivers missing from initramfs

Out of tree drivers missing from initramfs

[Joe Lawrence]

Hello Harold / list,    

I'm trying to build an initramfs that includes an out-of-tree kernel 
module for the mpt2sas driver without much success (by default).

The modified driver in question resides at:

  /lib/modules/3.8.0-rc4.sra+/updates/lsb-ft/extra/mpt2sas/mpt2sas.ko
 
and recent versions of dracut exclude this driver from the generated
initramfs.  This is on a F18 install, 3.8.0-rc4 kernel and a newly
recompiled out-of-tree mpt2sas ko file.

Using git bisect, I found dracut *would* include the updated .ko until...

d8aeb3a72afe03e426f4ea9f07c857c9a499144f is the first bad commit
commit d8aeb3a72afe03e426f4ea9f07c857c9a499144f
Author: Harald Hoyer <harald-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Date:   Wed Jun 27 09:35:39 2012 +0200

    dracut.sh, dracut-functions.sh: do lazy kernel module dep resolving
    
    First, we just install the kernel module and keep track, what we
    installed. At the very end, call modprobe and modinfo for firmwares to
    resolve all kernel module dependencies. This speeds up image creation,
    because we can call modprobe and modinfo with many modules.

I notice that find_kernel_modules_by_path() filters out any string that 
does not start with 'kernel' as well as the given path, ie 'drivers/scsi' 
(by design I would assume).

If the in-tree mpt2sas driver is loaded, then 
find_kernel_modules_by_path() would echo the driver since:

  $1 = drivers/scsi
  $a = kernel/drivers/scsi/mpt2sas/mpt2sas.ko

but when the out-of-tree driver is used, then

  $1 = drivers/scsi
  $a = updates/lsb-ft/extra/mpt2sas/mpt2sas.ko

fails this check.

1 - Should Dracut be checking the /lib/modules/ver/{updates,extras}
    directories as well as the in-tree kernel directory for drivers?

2 - Do modules underneath /lib/modules/ver/{updates,extras} need to 
    include their respective directory structure (ie, 'drivers/scsi'). 

I can avoid this by forcing dracut to include out-of-tree drivers using 
the add_drivers directive, but I defer to the dracut coders here on how I  
can best include them by default.  

Regards,

-- Joe

Re: Out of tree drivers missing from initramfs

[Joe Lawrence]

On Tue, 22 Jan 2013, Joe Lawrence wrote:

> Hello Harold / list,

s/Harold/Harald/  (apologies for the typo)

> 
> I'm trying to build an initramfs that includes an out-of-tree kernel 
> module for the mpt2sas driver without much success (by default).

Follow up info, modules.dep entry might be relevant:

# grep mpt2sas /lib/modules/3.8.0-rc4.sra+/modules.dep
updates/lsb-ft/extra/mpt2sas/mpt2sas.ko: kernel/drivers/scsi/raid_class.ko updates/lsb-ft/extra/sd_mod.ko updates/lsb-ft/extra/scsi_hbas.ko kernel/drivers/scsi/scsi_transport_sas.ko

Regards,

-- Joe

Re: Out of tree drivers missing from initramfs

[Harald Hoyer]

Am 22.01.2013 23:04, schrieb Joe Lawrence:
> Hello Harold / list,    
> 
> I'm trying to build an initramfs that includes an out-of-tree kernel 
> module for the mpt2sas driver without much success (by default).
> 
> The modified driver in question resides at:
> 
>   /lib/modules/3.8.0-rc4.sra+/updates/lsb-ft/extra/mpt2sas/mpt2sas.ko
>  
> and recent versions of dracut exclude this driver from the generated
> initramfs.  This is on a F18 install, 3.8.0-rc4 kernel and a newly
> recompiled out-of-tree mpt2sas ko file.
> 
> Using git bisect, I found dracut *would* include the updated .ko until...
> 
> d8aeb3a72afe03e426f4ea9f07c857c9a499144f is the first bad commit
> commit d8aeb3a72afe03e426f4ea9f07c857c9a499144f
> Author: Harald Hoyer <harald-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
> Date:   Wed Jun 27 09:35:39 2012 +0200
> 
>     dracut.sh, dracut-functions.sh: do lazy kernel module dep resolving
>     
>     First, we just install the kernel module and keep track, what we
>     installed. At the very end, call modprobe and modinfo for firmwares to
>     resolve all kernel module dependencies. This speeds up image creation,
>     because we can call modprobe and modinfo with many modules.
> 
> I notice that find_kernel_modules_by_path() filters out any string that 
> does not start with 'kernel' as well as the given path, ie 'drivers/scsi' 
> (by design I would assume).
> 
> If the in-tree mpt2sas driver is loaded, then 
> find_kernel_modules_by_path() would echo the driver since:
> 
>   $1 = drivers/scsi
>   $a = kernel/drivers/scsi/mpt2sas/mpt2sas.ko
> 
> but when the out-of-tree driver is used, then
> 
>   $1 = drivers/scsi
>   $a = updates/lsb-ft/extra/mpt2sas/mpt2sas.ko
> 
> fails this check.
> 
> 1 - Should Dracut be checking the /lib/modules/ver/{updates,extras}
>     directories as well as the in-tree kernel directory for drivers?

yes

> 
> 2 - Do modules underneath /lib/modules/ver/{updates,extras} need to 
>     include their respective directory structure (ie, 'drivers/scsi'). 

yes

> 
> I can avoid this by forcing dracut to include out-of-tree drivers using 
> the add_drivers directive, but I defer to the dracut coders here on how I  
> can best include them by default.  
> 
> Regards,
> 
> -- Joe

Thanks for reporting this, will fix this bug!

Re: Out of tree drivers missing from initramfs

[Joe Lawrence]

On Wed, 23 Jan 2013, Harald Hoyer wrote:

> Am 22.01.2013 23:04, schrieb Joe Lawrence:
>
> > 2 - Do modules underneath /lib/modules/ver/{updates,extras} need to 
> >     include their respective directory structure (ie, 'drivers/scsi'). 
> 
> yes

Is this a dracut requirement, general convention, etc?  I tried to search 
google for a definitive document on updates/extra modules and couldn't 
find much aside from Documentation/kbuild/modules.txt in the kernel tree.

This change is easy enough to make, but I'd like to get it right. :)
 
> Thanks for reporting this, will fix this bug!

See below for one possible solution.  It may be more correct to filter on 
kernel* and updates* and extra*, but as proof of concept, it 
successfully built a initramfs with a driver located at:

  /updates/lsb-ft/drivers/scsi/mpt2sas/mpt2sas.ko

Regards,

-- Joe

From b45c6d7eb70dea5bc44b683b2f46876b33ca062a Mon Sep 17 00:00:00 2001
From: Joe Lawrence <joe.lawrence-7+ureL1bLXNBDgjK7y7TUQ@public.gmane.org>
Date: Thu, 24 Jan 2013 14:30:56 -0500
Subject: [PATCH] Support out-of-tree and updated drivers in initramfs creation

The find_kernel_modules_by_path function shouldn't filter out modules not
located underneath the /lib/modules/ver/kernel directory as out-of-tree
drivers may reside in /lib/modules/ver/extra and updated drivers in
/lib/modules/ver/updates.

Signed-off-by: Joe Lawrence <joe.lawrence-7+ureL1bLXNBDgjK7y7TUQ@public.gmane.org>
---
 dracut-functions.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/dracut-functions.sh b/dracut-functions.sh
index e23db76..f7e3ec8 100755
--- a/dracut-functions.sh
+++ b/dracut-functions.sh
@@ -1502,7 +1502,7 @@ find_kernel_modules_by_path () (
         _OLDIFS=$IFS
         IFS=:
         while read a rest; do
-            [[ $a = kernel*/$1/* ]] || continue
+            [[ $a = */$1/* ]] || continue
             echo $srcmods/$a
         done < $srcmods/modules.dep
         IFS=$_OLDIFS
-- 
1.8.0.2

Re: Out of tree drivers missing from initramfs

[Harald Hoyer]

Am 24.01.2013 23:26, schrieb Joe Lawrence:
> On Wed, 23 Jan 2013, Harald Hoyer wrote:
> 
>> Am 22.01.2013 23:04, schrieb Joe Lawrence:
>>
>>> 2 - Do modules underneath /lib/modules/ver/{updates,extras} need to 
>>>     include their respective directory structure (ie, 'drivers/scsi'). 
>>
>> yes
> 
> Is this a dracut requirement, general convention, etc?  I tried to search 
> google for a definitive document on updates/extra modules and couldn't 
> find much aside from Documentation/kbuild/modules.txt in the kernel tree.
> 
> This change is easy enough to make, but I'd like to get it right. :)
>  
>> Thanks for reporting this, will fix this bug!
> 
> See below for one possible solution.  It may be more correct to filter on 
> kernel* and updates* and extra*, but as proof of concept, it 
> successfully built a initramfs with a driver located at:
> 
>   /updates/lsb-ft/drivers/scsi/mpt2sas/mpt2sas.ko
> 
> Regards,
> 
> -- Joe

perfekt. patch merged