RFC: modules and 2.5

RFC: modules and 2.5

[Jeff Garzik]

A couple things that would be nice for 2.5 is
- let MOD_INC_USE_COUNT work even when module is built into kernel, and
- let THIS_MODULE exist and be valid even when module is built into
kernel

This introduces bloat into the static kernel for modules which do not
take advantage of this, so perhaps we can make this new behavior
conditional on CONFIG_xxx option.  Individual drivers which make use of
the behavior can do something like

	dep_tristate 'my driver' CONFIG_MYDRIVER $CONFIG_PCI
	if [ "$CONFIG_MYDRIVER" != "n" -a \
	     "$CONFIG_STATIC_MODULES" != "y" ]; then
	   define_bool CONFIG_STATIC_MODULES y
	fi



The reasoning behind this is that module use counts are useful sometimes
even when the driver is built into the kernel.  Other facilities like
inter_xxx are [obviously] useful when built into the kernel, so it makes
sense to at least optionally support homogenous module treatment across
static or modular builds.

-- 
Jeff Garzik      | "I respect faith, but doubt is
Building 1024    |  what gives you an education."
MandrakeSoft     |           -- Wilson Mizner

Re: RFC: modules and 2.5

[Sean Hunter]

Does this defeat my favourite module-related gothcha, that the machine panics
if I have (say) a scsi driver builtin to the kernel and the same driver tries
to load itself as a module?

This normally happens when switching to a custom kernel after a fresh distro
install.  RedHat (and others, I think) use an initial ramdisk to make sure that
all the modules needed to mount the root fs get loaded at boot time.

If you build the drivers in, but forget to comment out the initrd line in
/etc/lilo.conf, the machine panics because it tries to load the module for
something that is already a builtin.

Make sense?

Sean

On Tue, Jul 03, 2001 at 01:13:45AM -0400, Jeff Garzik wrote:
> A couple things that would be nice for 2.5 is
> - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> - let THIS_MODULE exist and be valid even when module is built into
> kernel
> 
> This introduces bloat into the static kernel for modules which do not
> take advantage of this, so perhaps we can make this new behavior
> conditional on CONFIG_xxx option.  Individual drivers which make use of
> the behavior can do something like
> 
> 	dep_tristate 'my driver' CONFIG_MYDRIVER $CONFIG_PCI
> 	if [ "$CONFIG_MYDRIVER" != "n" -a \
> 	     "$CONFIG_STATIC_MODULES" != "y" ]; then
> 	   define_bool CONFIG_STATIC_MODULES y
> 	fi
> 
> 
> 
> The reasoning behind this is that module use counts are useful sometimes
> even when the driver is built into the kernel.  Other facilities like
> inter_xxx are [obviously] useful when built into the kernel, so it makes
> sense to at least optionally support homogenous module treatment across
> static or modular builds.
> 
> -- 
> Jeff Garzik      | "I respect faith, but doubt is
> Building 1024    |  what gives you an education."
> MandrakeSoft     |           -- Wilson Mizner
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: RFC: modules and 2.5

[Jeff Garzik]

Sean Hunter wrote:
> 
> Does this defeat my favourite module-related gothcha, that the machine panics
> if I have (say) a scsi driver builtin to the kernel and the same driver tries
> to load itself as a module?

Other, existing mechanisms should prevent conflicts here.  If a builtin
SCSI driver loads successfully, then its calls to request_region,
request_mem_region, or pci_request_regions should have succeeded.

If so, any attempt to grab those I/O regions by another driver,
including another instance of the same driver, should fail.

-- 
Jeff Garzik      | "I respect faith, but doubt is
Building 1024    |  what gives you an education."
MandrakeSoft     |           -- Wilson Mizner

Re: RFC: modules and 2.5

[Keith Owens]

On Tue, 3 Jul 2001 07:50:50 +0100, 
Sean Hunter <sean@dev.sportingbet.com> wrote:
>Does this defeat my favourite module-related gothcha, that the machine panics
>if I have (say) a scsi driver builtin to the kernel and the same driver tries
>to load itself as a module?

No, but other wip for 2.5 will.  My 2.5 makefile rewrite already
defines for each object, the module it would be linked into if it were
a module.  That gives me a list of all the "modules" already in the
kernel which is what is required to prevent duplicate loads.

Re: RFC: modules and 2.5

[Keith Owens]

On Tue, 03 Jul 2001 01:13:45 -0400, 
Jeff Garzik <jgarzik@mandrakesoft.com> wrote:
>A couple things that would be nice for 2.5 is
>- let MOD_INC_USE_COUNT work even when module is built into kernel, and
>- let THIS_MODULE exist and be valid even when module is built into
>kernel

There is already a list of module related changes that would be nice
for 2.5.  I have added this one to the list, but don't expect it until
September at least, after I get 2.5 module versions working.

Re: RFC: modules and 2.5

[Jeff Garzik]

Keith Owens wrote:
> 
> On Tue, 3 Jul 2001 07:50:50 +0100,
> Sean Hunter <sean@dev.sportingbet.com> wrote:
> >Does this defeat my favourite module-related gothcha, that the machine panics
> >if I have (say) a scsi driver builtin to the kernel and the same driver tries
> >to load itself as a module?
> 
> No, but other wip for 2.5 will.

If this occurs in 2.4 it is a bug and should be fixed in 2.4.

-- 
Jeff Garzik      | "I respect faith, but doubt is
Building 1024    |  what gives you an education."
MandrakeSoft     |           -- Wilson Mizner

Re: RFC: modules and 2.5

[Keith Owens]

On Tue, 03 Jul 2001 03:24:26 -0400, 
Jeff Garzik <jgarzik@mandrakesoft.com> wrote:
>> On Tue, 3 Jul 2001 07:50:50 +0100,
>> Sean Hunter <sean@dev.sportingbet.com> wrote:
>> >Does this defeat my favourite module-related gothcha, that the machine panics
>> >if I have (say) a scsi driver builtin to the kernel and the same driver tries
>> >to load itself as a module?
>
>If this occurs in 2.4 it is a bug and should be fixed in 2.4.

Human error.  Create a new kernel with something built in which used to
be a module and forget to make modules_install, so the code is in the
kernel and in /lib/modules.  Then do an explicit insmod, if probing
does not fail the module load then oops is all she wrote.  One of the
reasons I changed modules_install to erase the old directory first was
to minimize this problem, but humans can still stuff it up.  AFAICT it
cannot be fixed in 2.4 because there is no identifier for which
"modules" are included in the kernel.

Re: RFC: modules and 2.5

[Jeff Garzik]

Keith Owens wrote:
> Human error.  Create a new kernel with something built in which used to
> be a module and forget to make modules_install, so the code is in the
> kernel and in /lib/modules.  Then do an explicit insmod, if probing
> does not fail the module load then oops is all she wrote.

It still sounds like a driver bug.  It is the driver's responsibility to
ensure its resources are locked down for its own use.

Probing should not succeed if there is an existing driver in the
kernel.  request_??? and register_??? functions return failure to
guarantee this sort of thing.

Someone please provide a list of drivers which behave in this manner, so
that they can be fixed up.

Regards,

	Jeff


-- 
Jeff Garzik      | "I respect faith, but doubt is
Building 1024    |  what gives you an education."
MandrakeSoft     |           -- Wilson Mizner

Re: RFC: modules and 2.5

[Fang Han]

> If you build the drivers in, but forget to comment out the initrd line in
> /etc/lilo.conf, the machine panics because it tries to load the module for
> something that is already a builtin.
> 
The only way to solve it smothly need to modify the bootloader, When the 
bootloader like lilo or grub ( it is more powerful ) can read the module from the root partition directly. Your problem will be sloved.

BTW: Is there any system or tools can patch kernel in binary level, It means
     that user doesn't need download the whole kernel RPM or TGZ, It just need
     an patch to patch the current kernel's binary. I think it is useful for
     novice & end user.

Regards

dfbb

Re: RFC: modules and 2.5

[jlnance]

On Tue, Jul 03, 2001 at 01:13:45AM -0400, Jeff Garzik wrote:
> A couple things that would be nice for 2.5 is
> - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> - let THIS_MODULE exist and be valid even when module is built into
> kernel

I have something similar that I have wanted for a long time, and it would
accomplish what you want too.  I would like for the .o files for modules and
compiled in drivers to be identical.  It seems like this would be better for
testing because it should eleminate module vs non-module bugs.  We might
even want them to show up in /proc/modules, perhaps with some mechinism to
keep the reference count from going to 0.

I dont think I would want to unleash it on an end user, but if you omit the
part about letting the reference count go to zero, it should even be possible
to unload a compiled in driver and replace it with a new module.  If you
did not load each module into its own section, you would have to leak its
text and data memory, but this still might be useful for development.

Anyway, just some ideas I have been wanting to share for about 5 years.
Thanks for giving me an excuse.

Jim

Re: RFC: modules and 2.5

[Rusty Russell]

In message <3B415489.77425364@mandrakesoft.com> you write:
> A couple things that would be nice for 2.5 is
> - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> - let THIS_MODULE exist and be valid even when module is built into
> kernel

Hi Jeff,

What use are module use counts, if not used to prevent unloading?

> The reasoning behind this is that module use counts are useful sometimes
> even when the driver is built into the kernel.  Other facilities like
> inter_xxx are [obviously] useful when built into the kernel, so it makes

Let's be clear: inter_module_xxx is Broken as Designed.  It's a
terrible interface that has the added merit of being badly
implemented.

If you have a module B which has a soft dependency ("must use if
there") on module A, inter_xxx doesn't help without opening a can of
worms (if module B inserted after module A, oops).

The best ways out of this are:
1) Create two versions of module B: an A+B one, and a B-alone one.
2) Place infrastructure in the core kernel.
   (This is what I did for ipt_REJECT needing to know about NAT).

Also, I far prefer the simplicity of get_symbol and put_symbol.

Cheers,
Rusty.
--
Premature optmztion is rt of all evl. --DK

Re: RFC: modules and 2.5

[Jeff Garzik]

Rusty Russell wrote:
> 
> In message <3B415489.77425364@mandrakesoft.com> you write:
> > A couple things that would be nice for 2.5 is
> > - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> > - let THIS_MODULE exist and be valid even when module is built into
> > kernel
> 
> Hi Jeff,
> 
> What use are module use counts, if not used to prevent unloading?

IMHO you should be free to bump the module reference count up and down
as you wish, and be able to read the module reference count.

If you make that assumption, then it becomes possible to use the module
ref count as an internal reference counter, for device opens or
something like that.

In i810_rng.c it eliminated the need for an additional reference
count... until I attempted to compile it into the kernel instead of as a
module, for the first time :)

	Jeff


-- 
Jeff Garzik      | A recent study has shown that too much soup
Building 1024    | can cause malaise in laboratory mice.
MandrakeSoft     |

Re: RFC: modules and 2.5

[Rusty Russell]

In message <3B4718CC.483CE54E@mandrakesoft.com> you write:
> IMHO you should be free to bump the module reference count up and down
> as you wish, and be able to read the module reference count.
> 
> If you make that assumption, then it becomes possible to use the module
> ref count as an internal reference counter, for device opens or
> something like that.

Surely the exception rather than the rule?

Sorry, complicating the code and making everyone pay the penalty so
you can take a confusing short cut in your code is not something we're
going to agree on.

Modules are slower than built in; let's not "fix" this by making
builtin code slower. 8)

Rusty.
--
Premature optmztion is rt of all evl. --DK

Re: RFC: modules and 2.5

[Pavel Machek]

Hi!

> > IMHO you should be free to bump the module reference count up and down
> > as you wish, and be able to read the module reference count.
> > 
> > If you make that assumption, then it becomes possible to use the module
> > ref count as an internal reference counter, for device opens or
> > something like that.
> 
> Surely the exception rather than the rule?
> 
> Sorry, complicating the code and making everyone pay the penalty so
> you can take a confusing short cut in your code is not something we're
> going to agree on.

Actually, having uniform interface between kernel and modules is very
nice... And one int per module does not surely hurt, does it? 

Perhaps #define NEED_USE_COUNT to do it on per-module basis?
								Pavel
-- 
I'm pavel@ucw.cz. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at discuss@linmodems.org

Re: RFC: modules and 2.5

[Jeff Garzik]

Andrzej Krzysztofowicz wrote:
> 
> >
> > A couple things that would be nice for 2.5 is
> > - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> > - let THIS_MODULE exist and be valid even when module is built into
> > kernel
> >
> > This introduces bloat into the static kernel for modules which do not
> > take advantage of this, so perhaps we can make this new behavior
> > conditional on CONFIG_xxx option.  Individual drivers which make use of
> > the behavior can do something like
> >
> >       dep_tristate 'my driver' CONFIG_MYDRIVER $CONFIG_PCI
> >       if [ "$CONFIG_MYDRIVER" != "n" -a \
>               ^^^^^^^^^^^^^^^^^^^^^^^
> >            "$CONFIG_STATIC_MODULES" != "y" ]; then
> >          define_bool CONFIG_STATIC_MODULES y
> >       fi
> 
> Hmmm, shouldn't it be written in CML2 if it is for 2.5 ?

no comment


> For 2.4 the marked condition ( != n on a variable defined by dep_*)
> probably would break xconfig. Don't suggest such solutions...

why is != n on a variable defined by dep_xx bad?
That doesn't make sense.

	Jeff


-- 
Jeff Garzik      | "I respect faith, but doubt is
Building 1024    |  what gives you an education."
MandrakeSoft     |           -- Wilson Mizner

Re: RFC: modules and 2.5

[Andrzej Krzysztofowicz]

> 
> A couple things that would be nice for 2.5 is
> - let MOD_INC_USE_COUNT work even when module is built into kernel, and
> - let THIS_MODULE exist and be valid even when module is built into
> kernel
> 
> This introduces bloat into the static kernel for modules which do not
> take advantage of this, so perhaps we can make this new behavior
> conditional on CONFIG_xxx option.  Individual drivers which make use of
> the behavior can do something like
> 
> 	dep_tristate 'my driver' CONFIG_MYDRIVER $CONFIG_PCI
> 	if [ "$CONFIG_MYDRIVER" != "n" -a \
              ^^^^^^^^^^^^^^^^^^^^^^^
> 	     "$CONFIG_STATIC_MODULES" != "y" ]; then
> 	   define_bool CONFIG_STATIC_MODULES y
> 	fi

Hmmm, shouldn't it be written in CML2 if it is for 2.5 ?

For 2.4 the marked condition ( != n on a variable defined by dep_*)
probably would break xconfig. Don't suggest such solutions...

Andrzej
-- 
=======================================================================
  Andrzej M. Krzysztofowicz               ankry@mif.pg.gda.pl
  tel.  (0-58) 347 14 61
Wydz.Fizyki Technicznej i Matematyki Stosowanej Politechniki Gdanskiej