* [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. @ 2015-05-31 15:29 Andreas Mohr 2015-06-01 6:26 ` Rusty Russell 0 siblings, 1 reply; 14+ messages in thread From: Andreas Mohr @ 2015-05-31 15:29 UTC (permalink / raw) To: Andrew Morton Cc: Rusty Russell, Bertrand Jacquin, Marco d'Itri, linux-modules, linux-kernel Hi, I just had a not so nice experience when finally upgrading to a new 4.1-rc5 with CONFIG_MODULE_COMPRESS newly enabled - userspace binary parts (kmod 18 or 20 in my case) did not have compression enabled (at least on Debian 8pre, vs. encountering it enabled on FC21) since it does not seem to be the default build configuration of kmod (yet?). Doing a manual for gz in $(find . -name "*.gz"); do echo gunzip $gz; done depmod -a (somewhere temporarily in bootup scripts) manages to fix the grave problem introduced by erroneously having enabled CONFIG_MODULE_COMPRESS. Thus it seems that the Kconfig text was much more optimistic than a hapless user would want to encounter, thus I decided to have it updated. BTW: kmod and/or module-init-tools packages might want to provide a generic way to openly advertise certain build-configured "capabilities" in the binary - neither kmod --help nor kmod -V indicated whether or not they provided capabilities such as e.g. providing or not providing support of compression types and which ones. That would have been a very helpful way to reliably determine that support in fact is missing in the binary, rather than having to resort to clumsy hacks such as ldd or even strings. While this is a minor but useful addition rather than a severe fix, having a CC to stable@ added subsequently might be useful. Thanks! Signed-off-by: Andreas Mohr <andim2@users.sf.net> --- init/Kconfig | 18 +++++++++++++----- 1 file changed, 13 insertions(+), 5 deletions(-) diff --git a/init/Kconfig b/init/Kconfig index dc24dec..8e451f30 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1951,11 +1951,19 @@ config MODULE_COMPRESS This option compresses the kernel modules when 'make modules_install' is run. - The modules will be compressed either using gzip or xz depend on the - choice made in "Compression algorithm". - - module-init-tools has support for gzip format while kmod handle gzip - and xz compressed modules. + The module files will be compressed either using gzip or xz + depending on the choice made in "Compression algorithm". + + Obviously one will then need appropriate userspace parts + which are actually able to deal with compressed files, too: + module-init-tools has support for gzip format + while kmod handles gzip and xz compressed modules. + However, we observed that in several environments + module loader binaries do not have that enabled (yet?) + and thus bootup will fail fatally - + manually doing ldd on these binaries + to detect compression libraries + is a tell-tale sign of having support enabled. When a kernel module is installed from outside of the main kernel source and uses the Kbuild system for installing modules then that -- 2.1.4 ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-05-31 15:29 [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing Andreas Mohr @ 2015-06-01 6:26 ` Rusty Russell 2015-06-03 12:04 ` Michal Marek 2015-06-03 17:30 ` Lucas De Marchi 0 siblings, 2 replies; 14+ messages in thread From: Rusty Russell @ 2015-06-01 6:26 UTC (permalink / raw) To: Andreas Mohr, Andrew Morton Cc: Bertrand Jacquin, Marco d'Itri, linux-modules, linux-kernel, Jon Masters Andreas Mohr <andi@lisas.de> writes: > Hi, > > I just had a not so nice experience > when finally upgrading to a new 4.1-rc5 > with CONFIG_MODULE_COMPRESS newly enabled - > userspace binary parts (kmod 18 or 20 in my case) > did not have compression enabled > (at least on Debian 8pre, vs. encountering it enabled on FC21) > since it does not seem to be > the default build configuration of kmod (yet?). Sure. Let's get the maintainers to insert the actual version required in the help text though. Thanks, Rusty. > Doing a manual > for gz in $(find . -name "*.gz"); do echo gunzip $gz; done > depmod -a (somewhere temporarily in bootup scripts) > manages to fix the grave problem > introduced by erroneously having enabled CONFIG_MODULE_COMPRESS. > > Thus it seems that the Kconfig text was much more optimistic > than a hapless user would want to encounter, > thus I decided to have it updated. > > > BTW: kmod and/or module-init-tools packages > might want to provide a generic way to openly advertise > certain build-configured "capabilities" in the binary - > neither > kmod --help > nor > kmod -V > indicated whether or not they provided capabilities > such as e.g. providing or not providing support of compression types > and which ones. > That would have been a very helpful way > to reliably determine > that support in fact is missing in the binary, > rather than having to resort to clumsy hacks > such as ldd or even strings. > > > While this is a minor but useful addition > rather than a severe fix, > having a CC to stable@ added subsequently might be useful. > > Thanks! > > Signed-off-by: Andreas Mohr <andim2@users.sf.net> > > --- > init/Kconfig | 18 +++++++++++++----- > 1 file changed, 13 insertions(+), 5 deletions(-) > > diff --git a/init/Kconfig b/init/Kconfig > index dc24dec..8e451f30 100644 > --- a/init/Kconfig > +++ b/init/Kconfig > @@ -1951,11 +1951,19 @@ config MODULE_COMPRESS > This option compresses the kernel modules when 'make > modules_install' is run. > > - The modules will be compressed either using gzip or xz depend on the > - choice made in "Compression algorithm". > - > - module-init-tools has support for gzip format while kmod handle gzip > - and xz compressed modules. > + The module files will be compressed either using gzip or xz > + depending on the choice made in "Compression algorithm". > + > + Obviously one will then need appropriate userspace parts > + which are actually able to deal with compressed files, too: > + module-init-tools has support for gzip format > + while kmod handles gzip and xz compressed modules. > + However, we observed that in several environments > + module loader binaries do not have that enabled (yet?) > + and thus bootup will fail fatally - > + manually doing ldd on these binaries > + to detect compression libraries > + is a tell-tale sign of having support enabled. > > When a kernel module is installed from outside of the main kernel > source and uses the Kbuild system for installing modules then that > -- > 2.1.4 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-01 6:26 ` Rusty Russell @ 2015-06-03 12:04 ` Michal Marek 2015-06-03 17:30 ` Lucas De Marchi 1 sibling, 0 replies; 14+ messages in thread From: Michal Marek @ 2015-06-03 12:04 UTC (permalink / raw) To: Rusty Russell, Andreas Mohr Cc: Andrew Morton, Bertrand Jacquin, Marco d'Itri, linux-modules, linux-kernel, Jon Masters On 2015-06-01 08:26, Rusty Russell wrote: > Andreas Mohr <andi@lisas.de> writes: >> Hi, >> >> I just had a not so nice experience >> when finally upgrading to a new 4.1-rc5 >> with CONFIG_MODULE_COMPRESS newly enabled - >> userspace binary parts (kmod 18 or 20 in my case) >> did not have compression enabled >> (at least on Debian 8pre, vs. encountering it enabled on FC21) >> since it does not seem to be >> the default build configuration of kmod (yet?). > > Sure. Let's get the maintainers to insert the actual version required > in the help text though. I'm not a maintainer of kmod, but the text should probably end with "If unsure, say N." Michal ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-01 6:26 ` Rusty Russell 2015-06-03 12:04 ` Michal Marek @ 2015-06-03 17:30 ` Lucas De Marchi 2015-06-03 17:36 ` Kay Sievers 2015-06-04 1:30 ` Rusty Russell 1 sibling, 2 replies; 14+ messages in thread From: Lucas De Marchi @ 2015-06-03 17:30 UTC (permalink / raw) To: Rusty Russell Cc: Andreas Mohr, Andrew Morton, Bertrand Jacquin, Marco d'Itri, linux-modules, lkml, Jon Masters On Mon, Jun 1, 2015 at 3:26 AM, Rusty Russell <rusty@rustcorp.com.au> wrote: > Andreas Mohr <andi@lisas.de> writes: >> Hi, >> >> I just had a not so nice experience >> when finally upgrading to a new 4.1-rc5 >> with CONFIG_MODULE_COMPRESS newly enabled - >> userspace binary parts (kmod 18 or 20 in my case) >> did not have compression enabled >> (at least on Debian 8pre, vs. encountering it enabled on FC21) >> since it does not seem to be >> the default build configuration of kmod (yet?). > > Sure. Let's get the maintainers to insert the actual version required > in the help text though. kmod supports gz since the first version and xz since version 3. So both of them can be safely fall into "it's supported since the beginning of kmod IMO". Regarding the "default configuration", there's no such thing. Each distribution uses a different one. -- Lucas De Marchi ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-03 17:30 ` Lucas De Marchi @ 2015-06-03 17:36 ` Kay Sievers 2015-06-03 17:51 ` Lucas De Marchi 2015-06-04 1:30 ` Rusty Russell 1 sibling, 1 reply; 14+ messages in thread From: Kay Sievers @ 2015-06-03 17:36 UTC (permalink / raw) To: Lucas De Marchi Cc: Rusty Russell, Andreas Mohr, Andrew Morton, Bertrand Jacquin, Marco d'Itri, linux-modules, lkml, Jon Masters On Wed, Jun 3, 2015 at 7:30 PM, Lucas De Marchi <lucas.de.marchi@gmail.com> wrote: > On Mon, Jun 1, 2015 at 3:26 AM, Rusty Russell <rusty@rustcorp.com.au> wrote: >> Andreas Mohr <andi@lisas.de> writes: >>> Hi, >>> >>> I just had a not so nice experience >>> when finally upgrading to a new 4.1-rc5 >>> with CONFIG_MODULE_COMPRESS newly enabled - >>> userspace binary parts (kmod 18 or 20 in my case) >>> did not have compression enabled >>> (at least on Debian 8pre, vs. encountering it enabled on FC21) >>> since it does not seem to be >>> the default build configuration of kmod (yet?). >> >> Sure. Let's get the maintainers to insert the actual version required >> in the help text though. > > kmod supports gz since the first version and xz since version 3. So both > of them can be safely fall into "it's supported since the beginning of > kmod IMO". > > Regarding the "default configuration", there's no such thing. Each distribution > uses a different one. You could add something similar to this: $ /usr/lib/systemd/systemd --version systemd 220 +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN Kay ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-03 17:36 ` Kay Sievers @ 2015-06-03 17:51 ` Lucas De Marchi 0 siblings, 0 replies; 14+ messages in thread From: Lucas De Marchi @ 2015-06-03 17:51 UTC (permalink / raw) To: Kay Sievers Cc: Rusty Russell, Andreas Mohr, Andrew Morton, Bertrand Jacquin, Marco d'Itri, linux-modules, lkml, Jon Masters On Wed, Jun 3, 2015 at 2:36 PM, Kay Sievers <kay@vrfy.org> wrote: > On Wed, Jun 3, 2015 at 7:30 PM, Lucas De Marchi > <lucas.de.marchi@gmail.com> wrote: >> On Mon, Jun 1, 2015 at 3:26 AM, Rusty Russell <rusty@rustcorp.com.au> wrote: >>> Andreas Mohr <andi@lisas.de> writes: >>>> Hi, >>>> >>>> I just had a not so nice experience >>>> when finally upgrading to a new 4.1-rc5 >>>> with CONFIG_MODULE_COMPRESS newly enabled - >>>> userspace binary parts (kmod 18 or 20 in my case) >>>> did not have compression enabled >>>> (at least on Debian 8pre, vs. encountering it enabled on FC21) >>>> since it does not seem to be >>>> the default build configuration of kmod (yet?). >>> >>> Sure. Let's get the maintainers to insert the actual version required >>> in the help text though. >> >> kmod supports gz since the first version and xz since version 3. So both >> of them can be safely fall into "it's supported since the beginning of >> kmod IMO". >> >> Regarding the "default configuration", there's no such thing. Each distribution >> uses a different one. > > You could add something similar to this: > > $ /usr/lib/systemd/systemd --version > systemd 220 > +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP > +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS > +KMOD +IDN Indeed. I'm planning to do a release very soon and add something like this to it. If anyone is interested in contributing such patch it's very welcome, too :-) -- Lucas De Marchi ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-03 17:30 ` Lucas De Marchi 2015-06-03 17:36 ` Kay Sievers @ 2015-06-04 1:30 ` Rusty Russell 2015-06-04 2:31 ` Lucas De Marchi ` (2 more replies) 1 sibling, 3 replies; 14+ messages in thread From: Rusty Russell @ 2015-06-04 1:30 UTC (permalink / raw) To: Lucas De Marchi Cc: Andreas Mohr, Andrew Morton, Bertrand Jacquin, Marco d'Itri, linux-modules, lkml, Jon Masters Lucas De Marchi <lucas.de.marchi@gmail.com> writes: > On Mon, Jun 1, 2015 at 3:26 AM, Rusty Russell <rusty@rustcorp.com.au> wrote: >> Andreas Mohr <andi@lisas.de> writes: >>> Hi, >>> >>> I just had a not so nice experience >>> when finally upgrading to a new 4.1-rc5 >>> with CONFIG_MODULE_COMPRESS newly enabled - >>> userspace binary parts (kmod 18 or 20 in my case) >>> did not have compression enabled >>> (at least on Debian 8pre, vs. encountering it enabled on FC21) >>> since it does not seem to be >>> the default build configuration of kmod (yet?). >> >> Sure. Let's get the maintainers to insert the actual version required >> in the help text though. > > kmod supports gz since the first version and xz since version 3. So both > of them can be safely fall into "it's supported since the beginning of > kmod IMO". Thanks, that's what I needed. But disappointing that Debian doesn't configure with it, and there's no easy way to check it. Looks like Ubuntu vivid is the same. Might be time to change the default in kmod? Anyway, here's my patch, comments welcome. Subject: modules: clarify CONFIG_MODULE_COMPRESS help, suggest 'N'. Andreas turned this option on, only to find out Debian (and Ubuntu!) don't enable support in their kmod builds. Shorten the text, and suggest N at the bottom (at least for now). Reported-by: Andreas Mohr <andim2@users.sf.net> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> diff --git a/init/Kconfig b/init/Kconfig index 968a001790af..5422c44be5f0 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1948,26 +1948,22 @@ config MODULE_COMPRESS bool "Compress modules on installation" depends on MODULES help - This option compresses the kernel modules when 'make - modules_install' is run. - The modules will be compressed either using gzip or xz depend on the - choice made in "Compression algorithm". + Compresses kernel modules when 'make modules_install' is run; gzip or + xz depending on "Compression algorithm" below. - module-init-tools has support for gzip format while kmod handle gzip - and xz compressed modules. + module-init-tools supports gzip, and kmod can be configured to handle + gzip and xz (but doesn't by default, at least as of version 18!). - When a kernel module is installed from outside of the main kernel - source and uses the Kbuild system for installing modules then that - kernel module will also be compressed when it is installed. + Out-of-tree kernel modules installed using Kbuild will also be + compressed upon installation. - This option provides little benefit when the modules are to be used inside - an initrd or initramfs, it generally is more efficient to compress the whole - initrd or initramfs instead. + Note: for modules inside an initrd or initramfs, it's more efficient + to compress the whole initrd or initramfs instead. - This is fully compatible with signed modules while the signed module is - compressed. module-init-tools or kmod handles decompression and provide to - other layer the uncompressed but signed payload. + Note: This is fully compatible with signed modules. + + If in doubt, say N. choice prompt "Compression algorithm" ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-04 1:30 ` Rusty Russell @ 2015-06-04 2:31 ` Lucas De Marchi 2015-06-04 19:53 ` Andreas Mohr 2015-06-04 20:22 ` Rusty Russell 2015-06-04 2:51 ` Marco d'Itri 2015-06-04 3:19 ` Stephen Rothwell 2 siblings, 2 replies; 14+ messages in thread From: Lucas De Marchi @ 2015-06-04 2:31 UTC (permalink / raw) To: Rusty Russell Cc: Andreas Mohr, Andrew Morton, Bertrand Jacquin, Marco d'Itri, linux-modules, lkml, Jon Masters On Wed, Jun 3, 2015 at 10:30 PM, Rusty Russell <rusty@rustcorp.com.au> wrote: > Lucas De Marchi <lucas.de.marchi@gmail.com> writes: >> On Mon, Jun 1, 2015 at 3:26 AM, Rusty Russell <rusty@rustcorp.com.au> wrote: >>> Andreas Mohr <andi@lisas.de> writes: >>>> Hi, >>>> >>>> I just had a not so nice experience >>>> when finally upgrading to a new 4.1-rc5 >>>> with CONFIG_MODULE_COMPRESS newly enabled - >>>> userspace binary parts (kmod 18 or 20 in my case) >>>> did not have compression enabled >>>> (at least on Debian 8pre, vs. encountering it enabled on FC21) >>>> since it does not seem to be >>>> the default build configuration of kmod (yet?). >>> >>> Sure. Let's get the maintainers to insert the actual version required >>> in the help text though. >> >> kmod supports gz since the first version and xz since version 3. So both >> of them can be safely fall into "it's supported since the beginning of >> kmod IMO". > > Thanks, that's what I needed. > > But disappointing that Debian doesn't configure with it, and there's no > easy way to check it. Looks like Ubuntu vivid is the same. > > Might be time to change the default in kmod? > > Anyway, here's my patch, comments welcome. > > Subject: modules: clarify CONFIG_MODULE_COMPRESS help, suggest 'N'. > > Andreas turned this option on, only to find out Debian (and Ubuntu!) > don't enable support in their kmod builds. > > Shorten the text, and suggest N at the bottom (at least for now). > > Reported-by: Andreas Mohr <andim2@users.sf.net> > Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> > > diff --git a/init/Kconfig b/init/Kconfig > index 968a001790af..5422c44be5f0 100644 > --- a/init/Kconfig > +++ b/init/Kconfig > @@ -1948,26 +1948,22 @@ config MODULE_COMPRESS > bool "Compress modules on installation" > depends on MODULES > help > - This option compresses the kernel modules when 'make > - modules_install' is run. > > - The modules will be compressed either using gzip or xz depend on the > - choice made in "Compression algorithm". > + Compresses kernel modules when 'make modules_install' is run; gzip or > + xz depending on "Compression algorithm" below. > > - module-init-tools has support for gzip format while kmod handle gzip > - and xz compressed modules. > + module-init-tools supports gzip, and kmod can be configured to handle > + gzip and xz (but doesn't by default, at least as of version 18!). This is pretty much misleading. There's no such a default option. It's like saying kernel doesn't support feature X, Y or Z by default because make defconfig doesn't turn them on. The option is there since the beginning for the distros to adapt to their needs. -- Lucas De Marchi ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-04 2:31 ` Lucas De Marchi @ 2015-06-04 19:53 ` Andreas Mohr 2015-06-04 20:22 ` Rusty Russell 1 sibling, 0 replies; 14+ messages in thread From: Andreas Mohr @ 2015-06-04 19:53 UTC (permalink / raw) To: Lucas De Marchi Cc: Rusty Russell, Andreas Mohr, Andrew Morton, Bertrand Jacquin, Marco d'Itri, linux-modules, lkml, Jon Masters On Wed, Jun 03, 2015 at 11:31:20PM -0300, Lucas De Marchi wrote: > On Wed, Jun 3, 2015 at 10:30 PM, Rusty Russell <rusty@rustcorp.com.au> wrote: > > diff --git a/init/Kconfig b/init/Kconfig > > index 968a001790af..5422c44be5f0 100644 > > --- a/init/Kconfig > > +++ b/init/Kconfig > > @@ -1948,26 +1948,22 @@ config MODULE_COMPRESS > > bool "Compress modules on installation" > > depends on MODULES > > help > > - This option compresses the kernel modules when 'make > > - modules_install' is run. > > > > - The modules will be compressed either using gzip or xz depend on the > > - choice made in "Compression algorithm". > > + Compresses kernel modules when 'make modules_install' is run; gzip or > > + xz depending on "Compression algorithm" below. > > > > - module-init-tools has support for gzip format while kmod handle gzip > > - and xz compressed modules. > > + module-init-tools supports gzip, and kmod can be configured to handle > > + gzip and xz (but doesn't by default, at least as of version 18!). > > This is pretty much misleading. There's no such a default option. It's > like saying kernel doesn't support feature X, Y or Z by default > because make defconfig doesn't turn them on. > > The option is there since the beginning for the distros to adapt to their needs. Hmm? Perhaps I'm misunderstanding something in this discussion, but Debian kmod-20 (kmod_20.orig.tar.xz - BTW side note: noted the .xz file rather than .gz? ;)) configure.ac contains: AC_ARG_WITH([xz], AS_HELP_STRING([--with-xz], [handle Xz-compressed modules @<:@default=disabled@:>@]), [], [with_xz=no]) AS_IF([test "x$with_xz" != "xno"], [ PKG_CHECK_MODULES([liblzma], [liblzma >= 4.99]) AC_DEFINE([ENABLE_XZ], [1], [Enable Xz for modules.]) ], [ AC_MSG_NOTICE([Xz support not requested]) ]) AC_ARG_WITH([zlib], AS_HELP_STRING([--with-zlib], [handle gzipped modules @<:@default=disabled@:>@]), [], [with_zlib=no]) So it clearly seems there *is* a specific (hard-coded, as opposed to system introspection) default (i.e., the usual way of getting defaults, namely to simply not specify --with-something configure options), and that is (drumroll...) "disabled". Andreas ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-04 2:31 ` Lucas De Marchi 2015-06-04 19:53 ` Andreas Mohr @ 2015-06-04 20:22 ` Rusty Russell 2015-06-07 6:18 ` Lucas De Marchi 1 sibling, 1 reply; 14+ messages in thread From: Rusty Russell @ 2015-06-04 20:22 UTC (permalink / raw) To: Lucas De Marchi Cc: Andreas Mohr, Andrew Morton, Bertrand Jacquin, Marco d'Itri, linux-modules, lkml, Jon Masters Lucas De Marchi <lucas.de.marchi@gmail.com> writes: >> + gzip and xz (but doesn't by default, at least as of version 18!). > > This is pretty much misleading. There's no such a default option. It's > like saying kernel doesn't support feature X, Y or Z by default > because make defconfig doesn't turn them on. Nobody except Linus expects a defconfig to boot. I expect ./configure to give me a working tool; these *are* defaults, and setting them *is* your responsibility as author. When it became an in-kernel option rather than some weird distro thing, the old default was wrong. ... Though I hadn't realized that Marco had deliberately decided to leave support off. Surprising, since liblzma and libz seem required on Debian anyway. So I'm sorry Andreas: if your maintainer doesn't care about your case, there's not much I can do :( Subject: modules: clarify CONFIG_MODULE_COMPRESS help, suggest 'N'. Andreas turned this option on, only to find out Debian (and Ubuntu!) don't enable support in their kmod builds. Shorten the text, and suggest N at the bottom (at least for now). Reported-by: Andreas Mohr <andim2@users.sf.net> Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> diff --git a/init/Kconfig b/init/Kconfig index 968a001790af..bcb7e1e13f7d 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -1948,26 +1948,21 @@ config MODULE_COMPRESS bool "Compress modules on installation" depends on MODULES help - This option compresses the kernel modules when 'make - modules_install' is run. - The modules will be compressed either using gzip or xz depend on the - choice made in "Compression algorithm". + Compresses kernel modules when 'make modules_install' is run; gzip or + xz depending on "Compression algorithm" below. - module-init-tools has support for gzip format while kmod handle gzip - and xz compressed modules. + module-init-tools MAY support gzip, and kmod MAY support gzip and xz. - When a kernel module is installed from outside of the main kernel - source and uses the Kbuild system for installing modules then that - kernel module will also be compressed when it is installed. + Out-of-tree kernel modules installed using Kbuild will also be + compressed upon installation. - This option provides little benefit when the modules are to be used inside - an initrd or initramfs, it generally is more efficient to compress the whole - initrd or initramfs instead. + Note: for modules inside an initrd or initramfs, it's more efficient + to compress the whole initrd or initramfs instead. - This is fully compatible with signed modules while the signed module is - compressed. module-init-tools or kmod handles decompression and provide to - other layer the uncompressed but signed payload. + Note: This is fully compatible with signed modules. + + If in doubt, say N. choice prompt "Compression algorithm" ^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-04 20:22 ` Rusty Russell @ 2015-06-07 6:18 ` Lucas De Marchi 0 siblings, 0 replies; 14+ messages in thread From: Lucas De Marchi @ 2015-06-07 6:18 UTC (permalink / raw) To: Rusty Russell Cc: Andreas Mohr, Andrew Morton, Bertrand Jacquin, Marco d'Itri, linux-modules, lkml, Jon Masters On Thu, Jun 4, 2015 at 5:22 PM, Rusty Russell <rusty@rustcorp.com.au> wrote: > Lucas De Marchi <lucas.de.marchi@gmail.com> writes: >>> + gzip and xz (but doesn't by default, at least as of version 18!). >> >> This is pretty much misleading. There's no such a default option. It's >> like saying kernel doesn't support feature X, Y or Z by default >> because make defconfig doesn't turn them on. > > Nobody except Linus expects a defconfig to boot. > > I expect ./configure to give me a working tool; these *are* defaults, > and setting them *is* your responsibility as author. When it became an > in-kernel option rather than some weird distro thing, the old default > was wrong. > > ... Though I hadn't realized that Marco had deliberately decided to > leave support off. Surprising, since liblzma and libz seem required on > Debian anyway. As you can see, no default to something else will make distros adopt that. I long ago decided not to provide a default for compression because it's clearly not agreed upon. Instead it's everything disabled and the distro chooses what it wants to support. So if you think the configuration for your distro should be something else, convince the maintainer to change it showing why it's important. FYI I'm adding in kmod the ability to query the flags used during build. So now we have: $ ./tools/kmod --version kmod version 20 +XZ +ZLIB -EXPERIMENTAL This will be present in kmod 21 -- Lucas De Marchi ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-04 1:30 ` Rusty Russell 2015-06-04 2:31 ` Lucas De Marchi @ 2015-06-04 2:51 ` Marco d'Itri 2015-06-04 3:19 ` Stephen Rothwell 2 siblings, 0 replies; 14+ messages in thread From: Marco d'Itri @ 2015-06-04 2:51 UTC (permalink / raw) To: Rusty Russell Cc: Lucas De Marchi, Andreas Mohr, Andrew Morton, Bertrand Jacquin, linux-modules, lkml, Jon Masters [-- Attachment #1: Type: text/plain, Size: 459 bytes --] On Jun 04, Rusty Russell <rusty@rustcorp.com.au> wrote: > But disappointing that Debian doesn't configure with it, and there's no > easy way to check it. Looks like Ubuntu vivid is the same. > > Might be time to change the default in kmod? Changing the default will not make me enable compression support, but providing interesting use cases maybe will. Also, I see no point in enabling support for both gz and xz in kmod. -- ciao, Marco [-- Attachment #2: Type: application/pgp-signature, Size: 648 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-04 1:30 ` Rusty Russell 2015-06-04 2:31 ` Lucas De Marchi 2015-06-04 2:51 ` Marco d'Itri @ 2015-06-04 3:19 ` Stephen Rothwell 2015-06-04 20:09 ` Andreas Mohr 2 siblings, 1 reply; 14+ messages in thread From: Stephen Rothwell @ 2015-06-04 3:19 UTC (permalink / raw) To: Rusty Russell Cc: Lucas De Marchi, Andreas Mohr, Andrew Morton, Bertrand Jacquin, Marco d'Itri, linux-modules, lkml, Jon Masters [-- Attachment #1: Type: text/plain, Size: 1286 bytes --] Hi Rusty, On Thu, 04 Jun 2015 11:00:41 +0930 Rusty Russell <rusty@rustcorp.com.au> wrote: > > Lucas De Marchi <lucas.de.marchi@gmail.com> writes: > > On Mon, Jun 1, 2015 at 3:26 AM, Rusty Russell <rusty@rustcorp.com.au> wrote: > >> Andreas Mohr <andi@lisas.de> writes: > >>> > >>> I just had a not so nice experience > >>> when finally upgrading to a new 4.1-rc5 > >>> with CONFIG_MODULE_COMPRESS newly enabled - > >>> userspace binary parts (kmod 18 or 20 in my case) > >>> did not have compression enabled > >>> (at least on Debian 8pre, vs. encountering it enabled on FC21) > >>> since it does not seem to be > >>> the default build configuration of kmod (yet?). > >> > >> Sure. Let's get the maintainers to insert the actual version required > >> in the help text though. > > > > kmod supports gz since the first version and xz since version 3. So both > > of them can be safely fall into "it's supported since the beginning of > > kmod IMO". > > Thanks, that's what I needed. > > But disappointing that Debian doesn't configure with it, and there's no > easy way to check it. Looks like Ubuntu vivid is the same. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=772628 -- Cheers, Stephen Rothwell sfr@canb.auug.org.au [-- Attachment #2: OpenPGP digital signature --] [-- Type: application/pgp-signature, Size: 819 bytes --] ^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing. 2015-06-04 3:19 ` Stephen Rothwell @ 2015-06-04 20:09 ` Andreas Mohr 0 siblings, 0 replies; 14+ messages in thread From: Andreas Mohr @ 2015-06-04 20:09 UTC (permalink / raw) To: Stephen Rothwell Cc: Rusty Russell, Lucas De Marchi, Andreas Mohr, Andrew Morton, Bertrand Jacquin, Marco d'Itri, linux-modules, lkml, Jon Masters Hi, On Thu, Jun 04, 2015 at 01:19:46PM +1000, Stephen Rothwell wrote: > Hi Rusty, > > But disappointing that Debian doesn't configure with it, and there's no > > easy way to check it. Looks like Ubuntu vivid is the same. >=20 > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=3D772628 Crap, that URL now is an implicit strong suggestion that I didn't do my homework ;) (it already described large parts prior to me regurgitating this issue due to being unaware of this history here). Thanks to R.Russell for a very nice shortening and extension of the help text! Since the issue of .gz vs. .xz redundancy came up (with people "threatening" to support only one alternative), I want to mention that when having to choose one I'd tend to activating the .xz library dependency: - while it has higher compression demands, hardware is getting beefier all the time, thus it should not matter (especially vs. the dominantly many decompression runs) - it's simply the "more modern" and future-proof option, thus it should be favoured slightly since the system as a whole would want to make reasonably quick development/evolution "forward progress" rather than sticking to less favourable mechanisms - .xz has been available for some time already i.e. the time window of "distro support maturity" is a given [a counter-point might be that module-init-tools supports .gz only, but then modern binary setups which chose .xz would already have been shipped with kmod only] OK, having said that, I'm unsure what to think of the Debian package's decision to not support compression so far (and that even in times where kmod does not provide a runtime option yet to query the actual set of support flags). =46rom a library dependency POV it may be attractive to skip compression, and since Debian usually has a fixed setup where non-compressed files are a given, this seems like a valid choice. It's just that for e.g. these situations: - people with many custom kernels installed - space-constrained systems this is quite a nuisance that is a wee bit too unexpected (/show-stopper) - took me roughly > 30 minutes to get it researched / resolved. Thanks, Andreas ^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2015-06-07 6:18 UTC | newest] Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-05-31 15:29 [PATCH] modules: CONFIG_MODULE_COMPRESS: add hint that userspace support may easily be missing Andreas Mohr 2015-06-01 6:26 ` Rusty Russell 2015-06-03 12:04 ` Michal Marek 2015-06-03 17:30 ` Lucas De Marchi 2015-06-03 17:36 ` Kay Sievers 2015-06-03 17:51 ` Lucas De Marchi 2015-06-04 1:30 ` Rusty Russell 2015-06-04 2:31 ` Lucas De Marchi 2015-06-04 19:53 ` Andreas Mohr 2015-06-04 20:22 ` Rusty Russell 2015-06-07 6:18 ` Lucas De Marchi 2015-06-04 2:51 ` Marco d'Itri 2015-06-04 3:19 ` Stephen Rothwell 2015-06-04 20:09 ` Andreas Mohr
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).