[RFC] Simplifying kernel configuration for distro issues

[RFC] Simplifying kernel configuration for distro issues

[Linus Torvalds]

So this has long been one of my pet configuration peeves: as a user I
am perfectly happy answering the questions about what kinds of
hardware I want the kernel to support (I kind of know that), but many
of the "support infrastructure" questions are very opaque, and I have
no idea which of the them any particular distribution actually depends
on.

And it tends to change over time. For example, F14 (iirc) started
using TMPFS and TMPFS_POSIX_ACL/XATTR for /dev. And starting in F16,
the initrd setup requires DEVTMPFS and DEVTMPFS_MOUNT. There's been
several times when I started with my old minimal config, and the
resulting kernel would boot, but something wouldn't quite work right,
and it can be very subtle indeed.

Similarly, the distro ends up having very particular requirements for
exactly *which* security models it uses and needs, and they tend to
change over time. And now with systemd, CGROUPS suddenly aren't just
esoteric things that no normal person would want to use, but are used
for basic infrastructure. And I remember being surprised by OpenSUSE
suddenly needing the RAW table support for netfilter, because it had a
NOTRACK rule or something.

The point I'm slowly getting to is that I would actually love to have
*distro* Kconfig-files, where the distribution would be able to say
"These are the minimums I *require* to work". So we'd have a "Distro"
submenu, where you could pick the distro(s) you use, and then pick
which release, and we'd have something like

 - distro/Kconfig:

    config DISTRO_REQUIREMENTS
        bool "Pick minimal distribution requirements"

    choice DISTRO
        prompt "Distribution"
        depends on DISTRO_REQUIREMENTS

    config FEDORA
    config OPENSUSE
    config UBUNTU
    ...

    endchoice

and then depending on the DISTRO config, we'd include one of the
distro-specific ones with lists of supported distro versions and then
the random config settings for that version:

 - distro/Kconfig.suse:

    config OPENSUSE_121
        select OPENSUSE_11
        select IP_NF_RAW  # ..

 - distro/Kconfig.Fedora:

    config FEDORA_16
        select FEDORA_15
        select DEVTMPFS               # F16 initrd needs this
        select DEVTMPFS_MOUNT  # .. and expects the kernel to mount
DEVTMPFS automatically
        ...

    config FEDORA_17
        select FEDORA_16
        select CGROUP_xyzzy
        ...

and the point would be that it would make it much easier for a normal
user (and quite frankly, I want to put myself in that group too) to
make a kernel config that "just works".

Sure, you can copy the config file that came with the distro, but it
has tons of stuff that really isn't required. Not just in hardware,
but all the debug choices etc that are really a user choice. And it's
really hard to figure out - even for somebody like me - what a minimal
usable kernel is.

And yes, I know about "make localmodconfig". That's missing the point
for the same reason the distro config is missing the point.

Comments? It doesn't have to start out perfect, but I think it would
*really* help make the kernel configuration much easier for people.

In addition to the "minimal distro settings", we might also have a few
"common platform" settings, so that you could basically do a "hey, I
have a modern PC laptop, make it pick the obvious stuff that a normal
person needs, like USB storage, FAT/VFAT support, the core power
management etc". The silly stuff that you need, and that
"localyesconfig" actually misses because if you haven't inserted a USB
thumb drive, you won't necessarily have the FAT module loaded, but we
all know you do want it in real life. But that's really independent
issue, so let's keep it to just distro core things at first, ok?

Would something like this make sense to people? I really think that
"How do I generate a kernel config file" is one of those things that
keeps normal people from compiling their own kernel. And we *want*
people to compile their own kernel so that they can help with things
like bisecting etc. The more, the merrier.

                    Linus

Re: [RFC] Simplifying kernel configuration for distro issues

[Myklebust, Trond]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 4851 bytes --]

On Fri, 2012-07-13 at 13:37 -0700, Linus Torvalds wrote:
> So this has long been one of my pet configuration peeves: as a user I
> am perfectly happy answering the questions about what kinds of
> hardware I want the kernel to support (I kind of know that), but many
> of the "support infrastructure" questions are very opaque, and I have
> no idea which of the them any particular distribution actually depends
> on.
> 
> And it tends to change over time. For example, F14 (iirc) started
> using TMPFS and TMPFS_POSIX_ACL/XATTR for /dev. And starting in F16,
> the initrd setup requires DEVTMPFS and DEVTMPFS_MOUNT. There's been
> several times when I started with my old minimal config, and the
> resulting kernel would boot, but something wouldn't quite work right,
> and it can be very subtle indeed.
> 
> Similarly, the distro ends up having very particular requirements for
> exactly *which* security models it uses and needs, and they tend to
> change over time. And now with systemd, CGROUPS suddenly aren't just
> esoteric things that no normal person would want to use, but are used
> for basic infrastructure. And I remember being surprised by OpenSUSE
> suddenly needing the RAW table support for netfilter, because it had a
> NOTRACK rule or something.
> 
> The point I'm slowly getting to is that I would actually love to have
> *distro* Kconfig-files, where the distribution would be able to say
> "These are the minimums I *require* to work". So we'd have a "Distro"
> submenu, where you could pick the distro(s) you use, and then pick
> which release, and we'd have something like
> 
>  - distro/Kconfig:
> 
>     config DISTRO_REQUIREMENTS
>         bool "Pick minimal distribution requirements"
> 
>     choice DISTRO
>         prompt "Distribution"
>         depends on DISTRO_REQUIREMENTS
> 
>     config FEDORA
>     config OPENSUSE
>     config UBUNTU
>     ...
> 
>     endchoice
> 
> and then depending on the DISTRO config, we'd include one of the
> distro-specific ones with lists of supported distro versions and then
> the random config settings for that version:
> 
>  - distro/Kconfig.suse:
> 
>     config OPENSUSE_121
>         select OPENSUSE_11
>         select IP_NF_RAW  # ..
> 
>  - distro/Kconfig.Fedora:
> 
>     config FEDORA_16
>         select FEDORA_15
>         select DEVTMPFS               # F16 initrd needs this
>         select DEVTMPFS_MOUNT  # .. and expects the kernel to mount
> DEVTMPFS automatically
>         ...
> 
>     config FEDORA_17
>         select FEDORA_16
>         select CGROUP_xyzzy
>         ...
> 
> and the point would be that it would make it much easier for a normal
> user (and quite frankly, I want to put myself in that group too) to
> make a kernel config that "just works".
> 
> Sure, you can copy the config file that came with the distro, but it
> has tons of stuff that really isn't required. Not just in hardware,
> but all the debug choices etc that are really a user choice. And it's
> really hard to figure out - even for somebody like me - what a minimal
> usable kernel is.
> 
> And yes, I know about "make localmodconfig". That's missing the point
> for the same reason the distro config is missing the point.
> 
> Comments? It doesn't have to start out perfect, but I think it would
> *really* help make the kernel configuration much easier for people.
> 
> In addition to the "minimal distro settings", we might also have a few
> "common platform" settings, so that you could basically do a "hey, I
> have a modern PC laptop, make it pick the obvious stuff that a normal
> person needs, like USB storage, FAT/VFAT support, the core power
> management etc". The silly stuff that you need, and that
> "localyesconfig" actually misses because if you haven't inserted a USB
> thumb drive, you won't necessarily have the FAT module loaded, but we
> all know you do want it in real life. But that's really independent
> issue, so let's keep it to just distro core things at first, ok?
> 
> Would something like this make sense to people? I really think that
> "How do I generate a kernel config file" is one of those things that
> keeps normal people from compiling their own kernel. And we *want*
> people to compile their own kernel so that they can help with things
> like bisecting etc. The more, the merrier.
> 
>                     Linus

We could at least make selection of a minimal set of drivers for the
more common virtualised platforms a lot easier.
Right now, you need to hunt through 30+ different menus in order to find
what you need to run in a basic KVM virtual machine...

Cheers
  Trond
-- 
Trond Myklebust
Linux NFS client maintainer

NetApp
Trond.Myklebust@netapp.com
www.netapp.com

ÿôèº{.nÇ+‰·Ÿ®‰­†+%ŠËÿ±éݶ\x17¥Šwÿº{.nÇ+‰·¥Š{±þG«éÿŠ{ayº\x1dʇڙë,j\a­¢f£¢·hšïêÿ‘êçz_è®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿÿ¾\a«þG«éÿ¢¸?™¨è­Ú&£ø§~á¶iO•æ¬z·švØ^\x14\x04\x1a¶^[m§ÿÿÃ\fÿ¶ìÿ¢¸?–I¥

Re: [RFC] Simplifying kernel configuration for distro issues

[Dave Jones]

On Fri, Jul 13, 2012 at 01:37:41PM -0700, Linus Torvalds wrote:

 > The point I'm slowly getting to is that I would actually love to have
 > *distro* Kconfig-files, where the distribution would be able to say
 > "These are the minimums I *require* to work".

As long as you don't mind these being added after the fact, I suppose
it would be workable.  The reason I say that is sometimes, it even catches *us*
by surprise.  We recently found out our virtualisation guys started
using sch_htb for example, and we inadvertantly broke it when we moved
its module to a 'not always installed' kernel subpackage. (and before that, 9PFS..)

People don't tell us anything, but somehow expect things to keep working.

 > In addition to the "minimal distro settings", we might also have a few
 > "common platform" settings, so that you could basically do a "hey, I
 > have a modern PC laptop, make it pick the obvious stuff that a normal
 > person needs, like USB storage, FAT/VFAT support, the core power
 > management etc".

I wish defconfig was actually something useful like this, instead of..
what the hell is it exactly ? No-one even seems to agree, other than
"random selection of options, many of which were removed n years ago"

	Dave

Re: [RFC] Simplifying kernel configuration for distro issues

[Khalid Aziz]

On 07/13/2012 02:37 PM, Linus Torvalds wrote:

>
> Would something like this make sense to people? I really think that
> "How do I generate a kernel config file" is one of those things that
> keeps normal people from compiling their own kernel. And we *want*
> people to compile their own kernel so that they can help with things
> like bisecting etc. The more, the merrier.
>
This is a great idea. 7-8 years ago I used to be able to create a
minimally configured kernel from upstream and run my Debian/Ubuntu/...
install with it. It got much harder in a hurry and now it takes too much
work to figure out how to configure upstream kernel to make it work with
distro. It is a 3-5 hour compile to start with distro config file and that
is just too painful. I will help with testing configs or helping sort
through the config options.

-- 
Khalid Aziz
khalid.aziz@hp.com

Re: [RFC] Simplifying kernel configuration for distro issues

[Casey Schaufler]

On 7/13/2012 1:37 PM, Linus Torvalds wrote:
> So this has long been one of my pet configuration peeves: as a user I
> am perfectly happy answering the questions about what kinds of
> hardware I want the kernel to support (I kind of know that), but many
> of the "support infrastructure" questions are very opaque, and I have
> no idea which of the them any particular distribution actually depends
> on.
>
> And it tends to change over time. For example, F14 (iirc) started
> using TMPFS and TMPFS_POSIX_ACL/XATTR for /dev. And starting in F16,
> the initrd setup requires DEVTMPFS and DEVTMPFS_MOUNT. There's been
> several times when I started with my old minimal config, and the
> resulting kernel would boot, but something wouldn't quite work right,
> and it can be very subtle indeed.
>
> Similarly, the distro ends up having very particular requirements for
> exactly *which* security models it uses and needs, and they tend to
> change over time. And now with systemd, CGROUPS suddenly aren't just
> esoteric things that no normal person would want to use, but are used
> for basic infrastructure. And I remember being surprised by OpenSUSE
> suddenly needing the RAW table support for netfilter, because it had a
> NOTRACK rule or something.
>
> The point I'm slowly getting to is that I would actually love to have
> *distro* Kconfig-files, where the distribution would be able to say
> "These are the minimums I *require* to work".

Oh dear. I would expect Fedora to say that they require SELinux,
thereby making it unusable by anyone doing LSM development. It
would also make it more difficult for the people who don't want
any LSM (e.g. everyone sane) to configure the kernel they want.

This is the example that I see because of my particular biases.
I expect that there are similar things that distros do in other
areas that will have the same effect. The distro developers may
have decided that a feature is too cool to live without and
include it in their configuration even when it's not really
necessary. Plus, do you really think that they're going to
clean things out of their configuration when they decide that
they no longer need them?


>  So we'd have a "Distro"
> submenu, where you could pick the distro(s) you use, and then pick
> which release, and we'd have something like
>
>  - distro/Kconfig:
>
>     config DISTRO_REQUIREMENTS
>         bool "Pick minimal distribution requirements"
>
>     choice DISTRO
>         prompt "Distribution"
>         depends on DISTRO_REQUIREMENTS
>
>     config FEDORA
>     config OPENSUSE
>     config UBUNTU
>     ...
>
>     endchoice
>
> and then depending on the DISTRO config, we'd include one of the
> distro-specific ones with lists of supported distro versions and then
> the random config settings for that version:
>
>  - distro/Kconfig.suse:
>
>     config OPENSUSE_121
>         select OPENSUSE_11
>         select IP_NF_RAW  # ..
>
>  - distro/Kconfig.Fedora:
>
>     config FEDORA_16
>         select FEDORA_15
>         select DEVTMPFS               # F16 initrd needs this
>         select DEVTMPFS_MOUNT  # .. and expects the kernel to mount
> DEVTMPFS automatically
>         ...
>
>     config FEDORA_17
>         select FEDORA_16
>         select CGROUP_xyzzy
>         ...
>
> and the point would be that it would make it much easier for a normal
> user (and quite frankly, I want to put myself in that group too) to
> make a kernel config that "just works".
>
> Sure, you can copy the config file that came with the distro, but it
> has tons of stuff that really isn't required. Not just in hardware,
> but all the debug choices etc that are really a user choice. And it's
> really hard to figure out - even for somebody like me - what a minimal
> usable kernel is.
>
> And yes, I know about "make localmodconfig". That's missing the point
> for the same reason the distro config is missing the point.
>
> Comments? It doesn't have to start out perfect, but I think it would
> *really* help make the kernel configuration much easier for people.
>
> In addition to the "minimal distro settings", we might also have a few
> "common platform" settings, so that you could basically do a "hey, I
> have a modern PC laptop, make it pick the obvious stuff that a normal
> person needs, like USB storage, FAT/VFAT support, the core power
> management etc". The silly stuff that you need, and that
> "localyesconfig" actually misses because if you haven't inserted a USB
> thumb drive, you won't necessarily have the FAT module loaded, but we
> all know you do want it in real life. But that's really independent
> issue, so let's keep it to just distro core things at first, ok?
>
> Would something like this make sense to people? I really think that
> "How do I generate a kernel config file" is one of those things that
> keeps normal people from compiling their own kernel. And we *want*
> people to compile their own kernel so that they can help with things
> like bisecting etc. The more, the merrier.
>
>                     Linus
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: [RFC] Simplifying kernel configuration for distro issues

[Linus Torvalds]

On Fri, Jul 13, 2012 at 2:02 PM, Dave Jones <davej@redhat.com> wrote:
>
> As long as you don't mind these being added after the fact, I suppose
> it would be workable.  The reason I say that is sometimes, it even catches *us*
> by surprise.  We recently found out our virtualisation guys started
> using sch_htb for example, and we inadvertantly broke it when we moved
> its module to a 'not always installed' kernel subpackage. (and before that, 9PFS..)
>
> People don't tell us anything, but somehow expect things to keep working.

I think even a "educated guess" config file is better than what we have now.

The *two* requirements (and they're really the same theme) I
personally think we should have for this are

 -  I think every single "select" for these things should come with a
comment about what it is about and why the distro needs it (to show
there was some thought involved and not just a blind "took it from the
distro config")

 - It should be about *minimal* settings. I'd rather have too few
things and the occasional complaint about "oh, it didn't work because
it missed XYZ" than have it grow to contain all the options just
because somebody decided to just add random things until things
worked.

Other than that, even if it only gets you *closer* to a kernel that
works with that distro, I think it doesn't have to be all that
perfect. Because the alternative is what we have now.

           Linus

Re: [RFC] Simplifying kernel configuration for distro issues

[Linus Torvalds]

On Fri, Jul 13, 2012 at 2:17 PM, Casey Schaufler <casey@schaufler-ca.com> wrote:
>
> Oh dear. I would expect Fedora to say that they require SELinux,
> thereby making it unusable by anyone doing LSM development.

Oh, *absolutely*.

These options would *not* be meant for people doing odd things and
experienting with configs.

If you do that, then you might *start* by saying "I want this distro"
to get the initial guesstimate of the config file you want, but then
edit the .config by hand later (and remove the "I want all the Fedora
requirements" option, of course).

This is explicitly and exclusively for normal users. The whole point
of "expert configurator for special cases" should not be given any
thought at all - those kinds of people should simply answer "No" to
the "Do you want the distro basic kconfig requirements" question.

                Linus

Re: [RFC] Simplifying kernel configuration for distro issues

[Geert Uytterhoeven]

On Fri, Jul 13, 2012 at 11:02 PM, Dave Jones <davej@redhat.com> wrote:
> I wish defconfig was actually something useful like this, instead of..
> what the hell is it exactly ? No-one even seems to agree, other than
> "random selection of options, many of which were removed n years ago"

It's just to difficult to update them in a sane way.

I mean, I have my own set of defconfigs for all supported m68k-platforms,
but getting them in sync and ready for submitting an update seems to be a
multi-year project, and there are always more important (and more fun) things
to do. So that's why I haven't gotten to updating them since the defconfig
reduction.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[richard -rw- weinberger]

On Fri, Jul 13, 2012 at 10:54 PM, Myklebust, Trond
<Trond.Myklebust@netapp.com> wrote:
> We could at least make selection of a minimal set of drivers for the
> more common virtualised platforms a lot easier.
> Right now, you need to hunt through 30+ different menus in order to find
> what you need to run in a basic KVM virtual machine...

Yes, every time I build a kernel to be used on KVM I forget something. :-\

We could introduce a section in Kconfig which contains selections for
common use cases.
E.g. as Linus requested for minimal distro requirements but also
selections for various common
guest configurations.

-- 
Thanks,
//richard

Re: [RFC] Simplifying kernel configuration for distro issues

[Paul Bolle]

On Fri, 2012-07-13 at 17:02 -0400, Dave Jones wrote:
> I wish defconfig was actually something useful like this, instead of..
> what the hell is it exactly ? No-one even seems to agree, other than
> "random selection of options, many of which were removed n years ago"

As for the "many of which were removed n years ago" part: in
https://lkml.org/lkml/2011/11/18/211 I suggested to delete
almost 2000 lines from over 400 defconfig files. That would have deleted
all certainly unused macros from all defconfig files. Nothing happened
after that. My fault, I'm afraid.

But just removing all the certainly unused macros probably wouldn't have
made a noticeable difference to anyone using those defconfig files
anyway.


Paul Bolle

Re: [RFC] Simplifying kernel configuration for distro issues

[Dave Jones]

On Fri, Jul 13, 2012 at 11:50:25PM +0200, Paul Bolle wrote:

 > But just removing all the certainly unused macros probably wouldn't have
 > made a noticeable difference to anyone using those defconfig files
 > anyway.

My point is that I don't think there's many people actually using them.
(maybe more on the niche platforms, but x86[64] ? I'm sceptical they're used at all)

	Dave
 

Re: [RFC] Simplifying kernel configuration for distro issues

[Hans de Bruin]

On 07/13/2012 10:37 PM, Linus Torvalds wrote:
 > So this has long been one of my pet configuration peeves: as a user I
 > am perfectly happy answering the questions about what kinds of
 > hardware I want the kernel to support (I kind of know that), but many
 > of the "support infrastructure" questions are very opaque, and I have
 > no idea which of the them any particular distribution actually depends
 > on.
 >
 > And it tends to change over time. For example, F14 (iirc) started
 > using TMPFS and TMPFS_POSIX_ACL/XATTR for /dev. And starting in F16,
 > the initrd setup requires DEVTMPFS and DEVTMPFS_MOUNT. There's been
 > several times when I started with my old minimal config, and the
 > resulting kernel would boot, but something wouldn't quite work right,
 > and it can be very subtle indeed.
 >
 > Similarly, the distro ends up having very particular requirements for
 > exactly *which* security models it uses and needs, and they tend to
 > change over time. And now with systemd, CGROUPS suddenly aren't just
 > esoteric things that no normal person would want to use, but are used
 > for basic infrastructure. And I remember being surprised by OpenSUSE
 > suddenly needing the RAW table support for netfilter, because it had a
 > NOTRACK rule or something.
 >
 > The point I'm slowly getting to is that I would actually love to have
 > *distro* Kconfig-files, where the distribution would be able to say
 > "These are the minimums I *require* to work". So we'd have a "Distro"
 > submenu, where you could pick the distro(s) you use, and then pick
 > which release, and we'd have something like
 >
 >   - distro/Kconfig:
 >
 >      config DISTRO_REQUIREMENTS
 >          bool "Pick minimal distribution requirements"
 >
 >      choice DISTRO
 >          prompt "Distribution"
 >          depends on DISTRO_REQUIREMENTS
 >
 >      config FEDORA
 >      config OPENSUSE
 >      config UBUNTU
 >      ...
 >
 >      endchoice
 >
 > and then depending on the DISTRO config, we'd include one of the
 > distro-specific ones with lists of supported distro versions and then
 > the random config settings for that version:
 >
 >   - distro/Kconfig.suse:
 >
 >      config OPENSUSE_121
 >          select OPENSUSE_11
 >          select IP_NF_RAW  # ..
 >
 >   - distro/Kconfig.Fedora:
 >
 >      config FEDORA_16
 >          select FEDORA_15
 >          select DEVTMPFS               # F16 initrd needs this
 >          select DEVTMPFS_MOUNT  # .. and expects the kernel to mount
 > DEVTMPFS automatically
 >          ...
 >
 >      config FEDORA_17
 >          select FEDORA_16
 >          select CGROUP_xyzzy
 >          ...

Could this be made more dynamic? I would like to download a minimal 
config file from my distro's website and perhaps add my own minimal 
config for the hardware I own and put both downloads somewhere in my 
local tree, or have makemenuconfig ask me for a location of my minimal 
config files?

-- 
Hans

Re: [RFC] Simplifying kernel configuration for distro issues

[Tony Luck]

I always thought that the x86 defconfig file was the one that Linus
used for his primary machine.

-Tony

Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Fri, 13 Jul 2012, Linus Torvalds wrote:

> On Fri, Jul 13, 2012 at 2:17 PM, Casey Schaufler <casey@schaufler-ca.com> wrote:
>>
>> Oh dear. I would expect Fedora to say that they require SELinux,
>> thereby making it unusable by anyone doing LSM development.
>
> Oh, *absolutely*.
>
> These options would *not* be meant for people doing odd things and
> experienting with configs.
>
> If you do that, then you might *start* by saying "I want this distro"
> to get the initial guesstimate of the config file you want, but then
> edit the .config by hand later (and remove the "I want all the Fedora
> requirements" option, of course).
>
> This is explicitly and exclusively for normal users. The whole point
> of "expert configurator for special cases" should not be given any
> thought at all - those kinds of people should simply answer "No" to
> the "Do you want the distro basic kconfig requirements" question.

hopefully this can be made a little easier.

more of a 'enable anything set in this file, then give me control again so 
I can turn things off' rather than having to manually edit the .config 
file.

If this is done as a hard set of dependancy settings, it will be very 
annoying for people who for any reason want to disable something that the 
distro considers 'essential'.

I also _really_ like the idea of being able to have a vmware option that 
enables the minimum devices that are needed to run.

Having these be hard dependancies also seems like it would make 
interactions between these sorts of things much more likely to cause 
problems.

If however they are one-shot "go through this file and enable anything 
that it says to turn on" things that then let you turn anything off, it 
seems much less likely to cause problems.

and if we can then get some of the big hardware vendors to create such 
files to enable all the drivers needed for their hardware.... (the big 
things are easy, it's when you get into the internal monitoring busses and 
so on that things get messy)

David Lang

Re: [RFC] Simplifying kernel configuration for distro issues

[Paul Bolle]

On Fri, 2012-07-13 at 17:55 -0400, Dave Jones wrote:
> My point is that I don't think there's many people actually using them.
> (maybe more on the niche platforms, but x86[64] ? I'm sceptical they're used at all)

I guess you're right. Personally, I tend to start my journeys in self
compiled kernel land by using something I know that works as a starting
point. Ie, I use some distribution's kernel, notice that things seem to
mostly work, and use the .config of that kernel to start exploring
whatever it is I'm interested in. I can't remember ever feeling the urge
to use some defconfig to start these journeys. 


Paul Bolle

Re: [RFC] Simplifying kernel configuration for distro issues

[Josh Boyer]

On Fri, Jul 13, 2012 at 02:17:30PM -0700, Linus Torvalds wrote:
> On Fri, Jul 13, 2012 at 2:02 PM, Dave Jones <davej@redhat.com> wrote:
> >
> > As long as you don't mind these being added after the fact, I suppose
> > it would be workable.  The reason I say that is sometimes, it even catches *us*
> > by surprise.  We recently found out our virtualisation guys started
> > using sch_htb for example, and we inadvertantly broke it when we moved
> > its module to a 'not always installed' kernel subpackage. (and before that, 9PFS..)
> >
> > People don't tell us anything, but somehow expect things to keep working.
> 
> I think even a "educated guess" config file is better than what we have now.
> 
> The *two* requirements (and they're really the same theme) I
> personally think we should have for this are
> 
>  -  I think every single "select" for these things should come with a
> comment about what it is about and why the distro needs it (to show
> there was some thought involved and not just a blind "took it from the
> distro config")
> 
>  - It should be about *minimal* settings. I'd rather have too few
> things and the occasional complaint about "oh, it didn't work because
> it missed XYZ" than have it grow to contain all the options just
> because somebody decided to just add random things until things
> worked.

I'd agree that should be the goal.  It seems like something worth at
least trying to get to.  Even if we don't wind up merging them into the
kernel, it will at least lead to a better documented distro config for
every one that tries it.

josh

Re: [RFC] Simplifying kernel configuration for distro issues

[Jesper Juhl]

On Fri, 13 Jul 2012, Linus Torvalds wrote:

> So this has long been one of my pet configuration peeves: as a user I
> am perfectly happy answering the questions about what kinds of
> hardware I want the kernel to support (I kind of know that), but many
> of the "support infrastructure" questions are very opaque, and I have
> no idea which of the them any particular distribution actually depends
> on.
> 
> And it tends to change over time. For example, F14 (iirc) started
> using TMPFS and TMPFS_POSIX_ACL/XATTR for /dev. And starting in F16,
> the initrd setup requires DEVTMPFS and DEVTMPFS_MOUNT. There's been
> several times when I started with my old minimal config, and the
> resulting kernel would boot, but something wouldn't quite work right,
> and it can be very subtle indeed.
> 
> Similarly, the distro ends up having very particular requirements for
> exactly *which* security models it uses and needs, and they tend to
> change over time. And now with systemd, CGROUPS suddenly aren't just
> esoteric things that no normal person would want to use, but are used
> for basic infrastructure. And I remember being surprised by OpenSUSE
> suddenly needing the RAW table support for netfilter, because it had a
> NOTRACK rule or something.
> 
> The point I'm slowly getting to is that I would actually love to have
> *distro* Kconfig-files, where the distribution would be able to say
> "These are the minimums I *require* to work". So we'd have a "Distro"
> submenu, where you could pick the distro(s) you use, and then pick
> which release, and we'd have something like
> 
>  - distro/Kconfig:
> 
>     config DISTRO_REQUIREMENTS
>         bool "Pick minimal distribution requirements"
> 
>     choice DISTRO
>         prompt "Distribution"
>         depends on DISTRO_REQUIREMENTS
> 
>     config FEDORA
>     config OPENSUSE
>     config UBUNTU
>     ...
> 
>     endchoice
> 
[...]

We are going to end up with a million+ (or something like that) "config 
<RANDOM_FOO_DISTRO>" options that are going to have to be kept up-to-date 
regularly...
Do we really want that?
Maybe we do, maybe we don't - I'm not saying anything either way - just 
pointing it out.

I like the general idea - let a user pick the "make my distro work" option 
and then tweak from there. But, with hundreds (thousands?) of distroes out 
there, is it realy doable? Will we be able to keep things updated 
properly?

Perhaps a better aproach (and this is going to be controversial, so I'll 
put on my flame-repelling underwear now) would be to severely limit the 
number of available options.
KConfig is a mess (IMHO) - there's no telling what a given Linux kernel 
will support on any given distro on any given arch - there's no known 
mimimum.
How about we start cutting down on the options and start saying "a Linux 
system will provide feature x and y - always ...".
Stuff like (and I'm just pulling random stuff out here) - ASLR, seccomp, 
250HZ minimum etc etc.. We could cut the KConfig options down to 10% of 
what they are now if we just made a few (hard) choices about some things 
that would always be there that everyone could count on.  If people want 
to deviate from the default minimum, sure, let them, but put it under 
*custom*, *embedded*, *specialized distro*, *you know what you are doing* 
menu options.
Configurabillity is good, but only to a certain degree - I think we could 
bennefit from removing a *lot* of options and instead just decreeing that 
"a linux system has this"..


-- 
Jesper Juhl <jj@chaosbits.net>       http://www.chaosbits.net/
Don't top-post http://www.catb.org/jargon/html/T/top-post.html
Plain text mails only, please.

Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Sat, 14 Jul 2012, Jesper Juhl wrote:

> We are going to end up with a million+ (or something like that) "config
> <RANDOM_FOO_DISTRO>" options that are going to have to be kept up-to-date
> regularly...
> Do we really want that?
> Maybe we do, maybe we don't - I'm not saying anything either way - just
> pointing it out.
>
> I like the general idea - let a user pick the "make my distro work" option
> and then tweak from there. But, with hundreds (thousands?) of distroes out
> there, is it realy doable? Will we be able to keep things updated
> properly?

this needs to be more like 'make install' where the build system doesn't 
have specifics for every distro, but instead refrences a separate file 
that's provided in the same place by every distro, ideally separate from 
the kernel itself.

David Lang

Re: [RFC] Simplifying kernel configuration for distro issues

[Frank Rowand]

On 07/13/12 14:55, Dave Jones wrote:
> On Fri, Jul 13, 2012 at 11:50:25PM +0200, Paul Bolle wrote:
> 
>  > But just removing all the certainly unused macros probably wouldn't have
>  > made a noticeable difference to anyone using those defconfig files
>  > anyway.
> 
> My point is that I don't think there's many people actually using them.
> (maybe more on the niche platforms, but x86[64] ? I'm sceptical they're used at all)

I'm one of those people who use default configs.  I build a given kernel version for
many different embedded boards and expect the default config to work for them.  It
makes life much easier.

I also share Linus' pain when building for my host x86 system and try to
remove the cruft from my distro config.

-Frank

Re: [RFC] Simplifying kernel configuration for distro issues

[Ben Hutchings]

[-- Attachment #1: Type: text/plain, Size: 2510 bytes --]

On Fri, 2012-07-13 at 13:37 -0700, Linus Torvalds wrote:
> So this has long been one of my pet configuration peeves: as a user I
> am perfectly happy answering the questions about what kinds of
> hardware I want the kernel to support (I kind of know that), but many
> of the "support infrastructure" questions are very opaque, and I have
> no idea which of the them any particular distribution actually depends
> on.
[...]
> The point I'm slowly getting to is that I would actually love to have
> *distro* Kconfig-files, where the distribution would be able to say
> "These are the minimums I *require* to work". So we'd have a "Distro"
> submenu, where you could pick the distro(s) you use, and then pick
> which release, and we'd have something like

I like this idea in principle.

[...]
>  - distro/Kconfig:
> 
>     config DISTRO_REQUIREMENTS
>         bool "Pick minimal distribution requirements"
> 
>     choice DISTRO
>         prompt "Distribution"
>         depends on DISTRO_REQUIREMENTS
> 
>     config FEDORA
>     config OPENSUSE
>     config UBUNTU
>     ...
> 
>     endchoice
> 
> and then depending on the DISTRO config, we'd include one of the
> distro-specific ones with lists of supported distro versions and then
> the random config settings for that version:

You might also want to *un*select some options like
CONFIG_SYSFS_DEPRECATED and CONFIG_SYSFS_DEPRECATED_V2 that need to be
set one way or the other depending on the version of udev.  (I think
it's possible to kluge this with the addition of a hidden negative
config option.)

How about stuff like NET and INET, that every distro will need and yet
is configurable even without EXPERT?

[...]
> Sure, you can copy the config file that came with the distro, but it
> has tons of stuff that really isn't required. Not just in hardware,
> but all the debug choices etc that are really a user choice. And it's
> really hard to figure out - even for somebody like me - what a minimal
> usable kernel is.
[...]

And it's still hard for me as kernel packager: just because an option
was requested and enabled to support some bit of userland, doesn't mean
I know what's using or depending on it now.  (I think Dave Jones made
this point already.)  I'm not usually concerned with *minimal* config.

Ben.

-- 
Ben Hutchings
The generation of random numbers is too important to be left to chance.
                                                            - Robert Coveyou

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: [RFC] Simplifying kernel configuration for distro issues

[Olivier Galibert]

On Sat, Jul 14, 2012 at 12:33:51AM +0200, Jesper Juhl wrote:
> How about we start cutting down on the options and start saying "a Linux 
> system will provide feature x and y - always ...".
> Stuff like (and I'm just pulling random stuff out here) - ASLR, seccomp, 
> 250HZ minimum etc etc.. We could cut the KConfig options down to 10% of 
> what they are now if we just made a few (hard) choices about some things 
> that would always be there that everyone could count on.  If people want 
> to deviate from the default minimum, sure, let them, but put it under 
> *custom*, *embedded*, *specialized distro*, *you know what you are doing* 
> menu options.

In number of options the "infrastructure" options are at most 20% of
the total.  The other 80% are individual drivers, hardware (like
network cards, serial devices, usb devices, video...) or software
(crypto algorithms, partition formats, codepages, filesystems...).
You're going to have a hard time slashing 90% of that.

  OG.

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Borislav Petkov]

On Fri, Jul 13, 2012 at 11:41:21PM +0200, richard -rw- weinberger wrote:
> On Fri, Jul 13, 2012 at 10:54 PM, Myklebust, Trond
> <Trond.Myklebust@netapp.com> wrote:
> > We could at least make selection of a minimal set of drivers for the
> > more common virtualised platforms a lot easier.
> > Right now, you need to hunt through 30+ different menus in order to find
> > what you need to run in a basic KVM virtual machine...
> 
> Yes, every time I build a kernel to be used on KVM I forget something. :-\
> 
> We could introduce a section in Kconfig which contains selections for
> common use cases.
> E.g. as Linus requested for minimal distro requirements but also
> selections for various common
> guest configurations.

+1.

Same experience with building a kvm kernel so I couldn't agree more.

Thanks.

-- 
Regards/Gruss,
    Boris.

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Pekka Enberg]

On Sat, Jul 14, 2012 at 1:37 PM, Borislav Petkov <bp@alien8.de> wrote:
> On Fri, Jul 13, 2012 at 11:41:21PM +0200, richard -rw- weinberger wrote:
>> On Fri, Jul 13, 2012 at 10:54 PM, Myklebust, Trond
>> <Trond.Myklebust@netapp.com> wrote:
>> > We could at least make selection of a minimal set of drivers for the
>> > more common virtualised platforms a lot easier.
>> > Right now, you need to hunt through 30+ different menus in order to find
>> > what you need to run in a basic KVM virtual machine...
>>
>> Yes, every time I build a kernel to be used on KVM I forget something. :-\
>>
>> We could introduce a section in Kconfig which contains selections for
>> common use cases.
>> E.g. as Linus requested for minimal distro requirements but also
>> selections for various common
>> guest configurations.
>
> +1.
>
> Same experience with building a kvm kernel so I couldn't agree more.

We have "make kvmconfig" in the KVM tool tree that pretty much does
that automatically. There's nothing tools/kvm specific about it so I
guess you could merge that separately.

                        Pekka

Re: [RFC] Simplifying kernel configuration for distro issues

[Josh Boyer]

On Sat, Jul 14, 2012 at 12:18 AM, Ben Hutchings <ben@decadent.org.uk> wrote:
>>  - distro/Kconfig:
>>
>>     config DISTRO_REQUIREMENTS
>>         bool "Pick minimal distribution requirements"
>>
>>     choice DISTRO
>>         prompt "Distribution"
>>         depends on DISTRO_REQUIREMENTS
>>
>>     config FEDORA
>>     config OPENSUSE
>>     config UBUNTU
>>     ...
>>
>>     endchoice
>>
>> and then depending on the DISTRO config, we'd include one of the
>> distro-specific ones with lists of supported distro versions and then
>> the random config settings for that version:
>
> You might also want to *un*select some options like
> CONFIG_SYSFS_DEPRECATED and CONFIG_SYSFS_DEPRECATED_V2 that need to be
> set one way or the other depending on the version of udev.  (I think
> it's possible to kluge this with the addition of a hidden negative
> config option.)
>
> How about stuff like NET and INET, that every distro will need and yet
> is configurable even without EXPERT?

Yeah, I'm sure once distro configs are created we'll find quite a bit
similarity in them.  We could create Kconfig.distro that has all the
common options selected and then have the per distro Kconfig files
select that.  Sure we'd need to watch Kconfig.distro (or
Kconfig.min-distro) for changes, but I'd hope the distro maintainers
would sign up to maintain these things upstream and be keeping an eye
on it anyway.

josh

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Cyrill Gorcunov]

On Sat, Jul 14, 2012 at 03:12:05PM +0300, Pekka Enberg wrote:
> On Sat, Jul 14, 2012 at 1:37 PM, Borislav Petkov <bp@alien8.de> wrote:
> > On Fri, Jul 13, 2012 at 11:41:21PM +0200, richard -rw- weinberger wrote:
> >> On Fri, Jul 13, 2012 at 10:54 PM, Myklebust, Trond
> >> <Trond.Myklebust@netapp.com> wrote:
> >> > We could at least make selection of a minimal set of drivers for the
> >> > more common virtualised platforms a lot easier.
> >> > Right now, you need to hunt through 30+ different menus in order to find
> >> > what you need to run in a basic KVM virtual machine...
> >>
> >> Yes, every time I build a kernel to be used on KVM I forget something. :-\
> >>
> >> We could introduce a section in Kconfig which contains selections for
> >> common use cases.
> >> E.g. as Linus requested for minimal distro requirements but also
> >> selections for various common
> >> guest configurations.
> >
> > +1.
> >
> > Same experience with building a kvm kernel so I couldn't agree more.
> 
> We have "make kvmconfig" in the KVM tool tree that pretty much does
> that automatically. There's nothing tools/kvm specific about it so I
> guess you could merge that separately.

Yup, "make kvmconfig" does things right but I guess a general mechanism
might be preferred here :)

Look, if (at the moment) I'm going to nconfig and virt. section, I see

Virtualization
  Kernel-based Virtual Machine (KVM) support
    KVM for Intel processors support
    KVM for AMD processors support
    Audit KVM MMU
  Host kernel accelerator for virtio net (EXPERIMENTAL)

and everytime I'm here I need to remember where to search for
virtio devices I would like to include into config. Though I
expect to see in this secrion everything related to virtualization
and have a way to turn on virtio-ring and etc.

For example to enable "PCI driver for virtio devices" I need to
go to Device Drivers -> Virtio drivers, while I think it would
be great to have everything virt. related in Virtualization
section.

(note, I know how to enable all this and where to lookup for
 this entries of course, but I guess for regular users it
 would be easier to enable various virt. features if they
 are gathered in one place)

	Cyrill

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Borislav Petkov]

On Sat, Jul 14, 2012 at 04:43:32PM +0400, Cyrill Gorcunov wrote:
> For example to enable "PCI driver for virtio devices" I need to go to
> Device Drivers -> Virtio drivers, while I think it would be great to
> have everything virt. related in Virtualization section.

Actually, we need something more generic than that: everything X-related
should be automatically selected when setting CONFIG_X. And X can be any
subset of configuration options which belong to one feature, be it KVM,
distro-specific stuff, or CPU-vendor specific stuff, or whatever.

I can imagine, for example, that when a user wants to have an
AMD-specific config, it automatically selects CPU_SUP_AMD, X86_MCE_AMD
(for MCE), MICROCODE_AMD (microcode support), AGP_AMD64, EDAC_AMD64 and
a bunch of other AMD-specific features.

This would simplify not only the configuration process but also
Kconfig-related build failures since for your configuration you'll make
sure that required stuff is selected.

What I'm saying is that not only distro-specific configs but also some
sort of hierarchical config options could be defined to automatically
preselect stuff for a specific aspect and save a lot of time when
configuring a new system. Our config options have grown humongous right
about now and we can use the simplification. Of course, you can always
do fine-grained tuning afterwards but it'll save a lot of time, in
general.

Let's have an example: when I have to build upstream on a distro here,
I take the distro config and use it despite that it takes a long time
to build since everything is module - it is still better for me to
wait that one time instead of doing a dozen of trial and errors after
forgetting a config option each time.

Thanks.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
GM: Alberto Bozzo
Reg: Dornach, Landkreis Muenchen
HRB Nr. 43632 WEEE Registernr: 129 19551

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Cyrill Gorcunov]

On Sat, Jul 14, 2012 at 07:48:27PM +0200, Borislav Petkov wrote:
> On Sat, Jul 14, 2012 at 04:43:32PM +0400, Cyrill Gorcunov wrote:
> > For example to enable "PCI driver for virtio devices" I need to go to
> > Device Drivers -> Virtio drivers, while I think it would be great to
> > have everything virt. related in Virtualization section.
> 
> Actually, we need something more generic than that: everything X-related
> should be automatically selected when setting CONFIG_X. And X can be any
> subset of configuration options which belong to one feature, be it KVM,
> distro-specific stuff, or CPU-vendor specific stuff, or whatever.
> 
> I can imagine, for example, that when a user wants to have an
> AMD-specific config, it automatically selects CPU_SUP_AMD, X86_MCE_AMD
> (for MCE), MICROCODE_AMD (microcode support), AGP_AMD64, EDAC_AMD64 and
> a bunch of other AMD-specific features.

sure, no doubts, it would be great! (I must admit I never looked up into
kconfig parser code so i don't know if this could be achieved easily).

	Cyrill

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Sat, 14 Jul 2012, Cyrill Gorcunov wrote:

> On Sat, Jul 14, 2012 at 07:48:27PM +0200, Borislav Petkov wrote:
>> On Sat, Jul 14, 2012 at 04:43:32PM +0400, Cyrill Gorcunov wrote:
>>> For example to enable "PCI driver for virtio devices" I need to go to
>>> Device Drivers -> Virtio drivers, while I think it would be great to
>>> have everything virt. related in Virtualization section.
>>
>> Actually, we need something more generic than that: everything X-related
>> should be automatically selected when setting CONFIG_X. And X can be any
>> subset of configuration options which belong to one feature, be it KVM,
>> distro-specific stuff, or CPU-vendor specific stuff, or whatever.
>>
>> I can imagine, for example, that when a user wants to have an
>> AMD-specific config, it automatically selects CPU_SUP_AMD, X86_MCE_AMD
>> (for MCE), MICROCODE_AMD (microcode support), AGP_AMD64, EDAC_AMD64 and
>> a bunch of other AMD-specific features.
>
> sure, no doubts, it would be great! (I must admit I never looked up into
> kconfig parser code so i don't know if this could be achieved easily).

think of this as a variation of the miniconfig problem.

Instead of the miniconfig being one file that specifies everything you 
want, that you then use to generate a real .config file, you instead have 
one _or_ _more_ config files that get combined and then used to generate 
the real .config file.

Doing this as a set of miniconfig snippets instead of as menu options in 
the main kconfig has the advantage that once the real .config is created, 
it can then be edited freely, without worrying about things like SELINUX 
being enabled when you want to do development on a different LSM.

The distro config would be the first of these, but then you could have 
multiple hardware specific config files (a vmware config, a KVM config, 
one or more config files for the systems that you own, a USB config to 
enable a bunch of the various USB deices that you want to support, etc)

The distros may choose to produce several miniconfig files, one the 
minimum infrastructure features they need, and then additional files for 
the rest of their config. For example, I could see them having a set of 
files

1. Core
2. architecture
3. architecture specific drivers
4. architecture neutral drivers (USB devices and similar)


starting off with the distro config, the people creating this are going to 
be very familiar with kconfig and could create these by hand, but going 
forward we are going to want to have some tools to help sysadmins to 
create these files. Not knowing kconfig, it may be as simple as taking the 
miniconfig snippets that you start with, creating a .config and then doing 
a diff of that .config with the one the admin created, using the result as 
the new miniconfig snippet.

David Lang

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Borislav Petkov]

On Sat, Jul 14, 2012 at 03:12:05PM +0300, Pekka Enberg wrote:
> We have "make kvmconfig" in the KVM tool tree that pretty much does
> that automatically. There's nothing tools/kvm specific about it so I
> guess you could merge that separately.

Yes, something like KVMTOOL_TEST_ENABLE except not a make target but a
Kconfig option.

I can imagine a user selecting this and other coarse-grained options
in Kconfig and then "adjusting" single options like, for example, I
wouldn't need the *9P* options on x86, etc, etc.

Thanks.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
GM: Alberto Bozzo
Reg: Dornach, Landkreis Muenchen
HRB Nr. 43632 WEEE Registernr: 129 19551

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Pekka Enberg]

On Sun, Jul 15, 2012 at 1:14 PM, Borislav Petkov <bp@amd64.org> wrote:
> On Sat, Jul 14, 2012 at 03:12:05PM +0300, Pekka Enberg wrote:
>> We have "make kvmconfig" in the KVM tool tree that pretty much does
>> that automatically. There's nothing tools/kvm specific about it so I
>> guess you could merge that separately.
>
> Yes, something like KVMTOOL_TEST_ENABLE except not a make target but a
> Kconfig option.

It is a config option too - "make kvmconfig" simply enables it.

                        Pekka

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Borislav Petkov]

On Sun, Jul 15, 2012 at 01:17:16PM +0300, Pekka Enberg wrote:
> > Yes, something like KVMTOOL_TEST_ENABLE except not a make target but a
> > Kconfig option.
> 
> It is a config option too - "make kvmconfig" simply enables it.

Right, so how about something more concrete in this whole discussion:

Maybe add the KVMTOOL_TEST_ENABLE (maybe rename it to a more fitting
name like KVM_OPTS or similar) thing to a new toplevel Kconfig menu
called "Distro and other profiles" or so:

  │ │                                       General setup  --->
  │ │                                       Distros and other profiles  --->			<---
  │ │                                   [*] Enable loadable module support  --->
  │ │                                   [*] Enable the block layer  --->
  │ │                                       Processor type and features  --->
 ...

Btw, which Kconfig option is needed for "-usbdevice tablet" support so
that the host mouse can be able to send scroll events to the guest qemu
screen without the need for the qemu screen to capture it so that I need
to "escape" it everytime I need to return to the host? And can we add it
to the list of options KVMTOOL_TEST_ENABLE selects?

Thanks.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
GM: Alberto Bozzo
Reg: Dornach, Landkreis Muenchen
HRB Nr. 43632 WEEE Registernr: 129 19551

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Cyrill Gorcunov]

On Sun, Jul 15, 2012 at 11:18:48PM +0200, Borislav Petkov wrote:
> On Sun, Jul 15, 2012 at 01:17:16PM +0300, Pekka Enberg wrote:
> > > Yes, something like KVMTOOL_TEST_ENABLE except not a make target but a
> > > Kconfig option.
> > 
> > It is a config option too - "make kvmconfig" simply enables it.
> 
> Right, so how about something more concrete in this whole discussion:
> 
> Maybe add the KVMTOOL_TEST_ENABLE (maybe rename it to a more fitting
> name like KVM_OPTS or similar) thing to a new toplevel Kconfig menu
> called "Distro and other profiles" or so:
> 
>   │ │     General setup  --->
>   │ │     Distros and other profiles  --->			<---
>   │ │ [*] Enable loadable module support  --->
>   │ │ [*] Enable the block layer  --->
>   │ │     Processor type and features  --->
>  ...
> 
> Btw, which Kconfig option is needed for "-usbdevice tablet" support so
> that the host mouse can be able to send scroll events to the guest qemu
> screen without the need for the qemu screen to capture it so that I need
> to "escape" it everytime I need to return to the host? And can we add it
> to the list of options KVMTOOL_TEST_ENABLE selects?

All this is pretty promising idea for usability I think (plain "Profiles"
is enough if not only distros will be there methink).

Replying to David's message (sorry for delay) I fear having a bunch of
miniconfig files will end up in a mess. Maybe (maybe (!) I don't know since
I've no time at moment to read kconfig code and I'm not sure if this
is right direction at all) it would worth to add some new keyword to
kconfig language, say "profile", which would tag symbol to a category
if needed, and these categories included into profiles automatically.
On the other hands this might end up in a mess as well.

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Mon, 16 Jul 2012, Cyrill Gorcunov wrote:

> Replying to David's message (sorry for delay) I fear having a bunch of
> miniconfig files will end up in a mess. Maybe (maybe (!) I don't know since
> I've no time at moment to read kconfig code and I'm not sure if this
> is right direction at all) it would worth to add some new keyword to
> kconfig language, say "profile", which would tag symbol to a category
> if needed, and these categories included into profiles automatically.
> On the other hands this might end up in a mess as well.

I have a couple problems with the approach of modifying the existing 
kconfig files

1. how does it handle the case when a profile wants something one way and 
the admin wants it another way

   the example is the fedora default wanting SELINUX and I want some other 
LSM

2. since it requires making changes in the upstream kernel source, the 
number of people who can make these changes is small.

3. since all these changes go into the upstream kernel source, changes to 
these profiles are going to be visible churn (think of the issues with the 
defconfigs for ARM a couple of years ago)

4. the complexity of tagging all possible profiles is very high.

   Even if you limit the profiles to "Linux Distros", how many different 
distros are there? Do you really want to have to start arguing over which 
distros are large enough to get their profile added to the upstream kernel 
source?


If instead we go with something along the lines of the miniconf approach, 
the picture looks very different

1. this approach only sets things one time, after that the person doing 
the compile is free to change anything.

2. since the changes are a local file, separate from the upstream source, 
issues like who can provide the config, what distros will accept it, 
complexity in having many different options, etc all vanish

3. by simply combining miniconfig files, you can combine sets of 
pre-defined options

David Lang

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Cyrill Gorcunov]

On Sun, Jul 15, 2012 at 03:09:12PM -0700, david@lang.hm wrote:
> 
> 3. by simply combining miniconfig files, you can combine sets of
> pre-defined options

Wait, David, I'm lost. These miniconfigs should live somewhere on
my home directory (if they are out of mainline tree)?

	Cyrill

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Mon, 16 Jul 2012, Cyrill Gorcunov wrote:

> On Sun, Jul 15, 2012 at 03:09:12PM -0700, david@lang.hm wrote:
>>
>> 3. by simply combining miniconfig files, you can combine sets of
>> pre-defined options
>
> Wait, David, I'm lost. These miniconfigs should live somewhere on
> my home directory (if they are out of mainline tree)?

the distro provided miniconfig files should live in a "known location" 
(like the distro specific scripts for "make install", but I'm saying that 
we should be able to use this same mechanism to include locally generated 
files.

combining external files doesn't need to be the job of the kernel kconfig 
infrastructure either. There could be external scripts that take one or 
more distro provided files and one or more local files and combine them 
for the kconfig infrastructure.

For a distro, I could see their build farm using this as well, with a file 
organization along the lines of

base config
arch specific options (one per arch)
generic drivers (USB devices, etc)

This sort of thing would help reduce the chances of there being unexpected 
differences between the different architecture builds.

this can be done without having to modify the existing kconfig stuff at 
all (by using make oldconfig), but it's likely that a little bit of 
support from kconfig could make creating these miniconifg files much 
easier.

Linus started off wanting the core requirements, other people want to 
duplicate the full distro kernel, others want everything but the drivers, 
others want to substatute a list of driver options that their local 
systems need, etc. by using separate files that are external to the 
kconfig files in the kernel.org source tree, I think that all these people 
can be satisfied without an explosion of complexity in the source tree.

David Lang

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Borislav Petkov]

On Sun, Jul 15, 2012 at 03:09:12PM -0700, david@lang.hm wrote:
> On Mon, 16 Jul 2012, Cyrill Gorcunov wrote:
> 
> >Replying to David's message (sorry for delay) I fear having a bunch of
> >miniconfig files will end up in a mess. Maybe (maybe (!) I don't know since
> >I've no time at moment to read kconfig code and I'm not sure if this
> >is right direction at all) it would worth to add some new keyword to
> >kconfig language, say "profile", which would tag symbol to a category
> >if needed, and these categories included into profiles automatically.
> >On the other hands this might end up in a mess as well.
> 
> I have a couple problems with the approach of modifying the existing
> kconfig files
> 
> 1. how does it handle the case when a profile wants something one
> way and the admin wants it another way

Select the profile and then fixup the config the normal way.

If what the admin wants is incompatible with the profile, admin doesn't
select the profile.

>   the example is the fedora default wanting SELINUX and I want some
> other LSM

Currently, if you run fedora and want something that's not enabled, you
recompile your kernel too, right?

> 2. since it requires making changes in the upstream kernel source, the
> number of people who can make these changes is small.

I think that's moot since you either select the profile or you don't.

> 3. since all these changes go into the upstream kernel source, changes
> to these profiles are going to be visible churn (think of the issues
> with the defconfigs for ARM a couple of years ago)

AFAICT, those changes will be needed only for a new distro release and
that happens twice a year, tops.

> 4. the complexity of tagging all possible profiles is very high.

That's why we start simple.

> Even if you limit the profiles to "Linux Distros", how many different
> distros are there? Do you really want to have to start arguing over
> which distros are large enough to get their profile added to the
> upstream kernel source?

That's a valid question; its answer could be defined arbitrarily.

<joshing>
Let's say all distros which make money - more than a certain large
amount - are allowed. :-)
</joshing>

> If instead we go with something along the lines of the miniconf
> approach, the picture looks very different
> 
> 1. this approach only sets things one time, after that the person
> doing the compile is free to change anything.

...

Sorry, I don't see the simplification: you need to rebuild your kernel
anyway and before you rebuild it, you can do all the changes you want.
So either you select a profile or you load a miniconfig, it doesn't
really matter how you do it?

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
GM: Alberto Bozzo
Reg: Dornach, Landkreis Muenchen
HRB Nr. 43632 WEEE Registernr: 129 19551

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Mon, 16 Jul 2012, Borislav Petkov wrote:

> On Sun, Jul 15, 2012 at 03:09:12PM -0700, david@lang.hm wrote:
>> On Mon, 16 Jul 2012, Cyrill Gorcunov wrote:
>>
>>> Replying to David's message (sorry for delay) I fear having a bunch of
>>> miniconfig files will end up in a mess. Maybe (maybe (!) I don't know since
>>> I've no time at moment to read kconfig code and I'm not sure if this
>>> is right direction at all) it would worth to add some new keyword to
>>> kconfig language, say "profile", which would tag symbol to a category
>>> if needed, and these categories included into profiles automatically.
>>> On the other hands this might end up in a mess as well.
>>
>> I have a couple problems with the approach of modifying the existing
>> kconfig files
>>
>> 1. how does it handle the case when a profile wants something one
>> way and the admin wants it another way
>
> Select the profile and then fixup the config the normal way.
>
> If what the admin wants is incompatible with the profile, admin doesn't
> select the profile.
>
>>   the example is the fedora default wanting SELINUX and I want some
>> other LSM
>
> Currently, if you run fedora and want something that's not enabled, you
> recompile your kernel too, right?

The problem is that you can't select the Fedora profile and then unselect 
SELINUX, so the profile will do you no good.

It's not a matter of needing to recompile or not, it's a mattter that when 
you want to recompile, you want to be able to easily select the 
infrastructure pieces that the distro needs to operate, without also 
getting their huge selection of 'other' things.

>> 2. since it requires making changes in the upstream kernel source, the
>> number of people who can make these changes is small.
>
> I think that's moot since you either select the profile or you don't.
>
>> 3. since all these changes go into the upstream kernel source, changes
>> to these profiles are going to be visible churn (think of the issues
>> with the defconfigs for ARM a couple of years ago)
>
> AFAICT, those changes will be needed only for a new distro release and
> that happens twice a year, tops.
>
>> 4. the complexity of tagging all possible profiles is very high.
>
> That's why we start simple.
>
>> Even if you limit the profiles to "Linux Distros", how many different
>> distros are there? Do you really want to have to start arguing over
>> which distros are large enough to get their profile added to the
>> upstream kernel source?
>
> That's a valid question; its answer could be defined arbitrarily.
>
> <joshing>
> Let's say all distros which make money - more than a certain large
> amount - are allowed. :-)
> </joshing>

So that would eliminate all linux distros except for Red Hat Enterprise 
and Suse (including eliminating opensuse and fedora), somehow I don't 
think that would produce the benefit that Linus was looking for.

>> If instead we go with something along the lines of the miniconf
>> approach, the picture looks very different
>>
>> 1. this approach only sets things one time, after that the person
>> doing the compile is free to change anything.
>
> ...
>
> Sorry, I don't see the simplification: you need to rebuild your kernel
> anyway and before you rebuild it, you can do all the changes you want.
> So either you select a profile or you load a miniconfig, it doesn't
> really matter how you do it?

Yes, you have to recompile in any case.

The difference is that with a profile, the profile must be in the kernel 
source, and you will have problems if you want to start with the profile 
and then tweak something.

with miniconfig, you can have locally defined configs, you can combine 
multiple configs, and the configs define a starting point, but you can 
then override them if you want to.

David Lang

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Linus Torvalds]

On Mon, Jul 16, 2012 at 9:43 AM,  <david@lang.hm> wrote:
>
> The problem is that you can't select the Fedora profile and then unselect
> SELINUX, so the profile will do you no good.

Guys, stop it now.

Your "problem" isn't what any sane person cares about, and isn't what
I started the RFC for.

Seriously. NOBODY CARES.

You can do what you want to do *today*. Just edit the config file, or
use any of the millions of config tools. Stop whining.

The thing I'm asking for is for normal people. Make it easy for people
who DO NOT CARE about the config file to just build a kernel for their
machine.

Don't complicate the issue by bringing up some totally unrelated
question. Don't derail a useful feature for the 99% because you're not
in it.

                    Linus

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Alan Cox]

> Select the profile and then fixup the config the normal way.
> 
> If what the admin wants is incompatible with the profile, admin doesn't
> select the profile.

Thats ugly - "distro except..." is a standard thing you ask users to do
for debugging.

However providing you separate the initial profile from the later tools
it simply becomes 

make distroconfig
	[cp /etc/system-kconfig(.$ARCH?) .config
	 make oldconfig]

make menuconfig (if you want to customise)

In addition the make oldconfig means you can ship a deliberately
incomplete distroconfig and get the user asked some bits.

make 

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Mon, 16 Jul 2012, Alan Cox wrote:

>> Select the profile and then fixup the config the normal way.
>>
>> If what the admin wants is incompatible with the profile, admin doesn't
>> select the profile.
>
> Thats ugly - "distro except..." is a standard thing you ask users to do
> for debugging.
>
> However providing you separate the initial profile from the later tools
> it simply becomes
>
> make distroconfig
> 	[cp /etc/system-kconfig(.$ARCH?) .config
> 	 make oldconfig]
>
> make menuconfig (if you want to customise)
>
> In addition the make oldconfig means you can ship a deliberately
> incomplete distroconfig and get the user asked some bits.

exactly what I was fumbling to describe, thanks.

David Lang

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Mon, 16 Jul 2012, Linus Torvalds wrote:

> On Mon, Jul 16, 2012 at 9:43 AM,  <david@lang.hm> wrote:
>>
>> The problem is that you can't select the Fedora profile and then unselect
>> SELINUX, so the profile will do you no good.
>
> Guys, stop it now.
>
> Your "problem" isn't what any sane person cares about, and isn't what
> I started the RFC for.
>
> Seriously. NOBODY CARES.
>
> You can do what you want to do *today*. Just edit the config file, or
> use any of the millions of config tools. Stop whining.
>
> The thing I'm asking for is for normal people. Make it easy for people
> who DO NOT CARE about the config file to just build a kernel for their
> machine.
>
> Don't complicate the issue by bringing up some totally unrelated
> question. Don't derail a useful feature for the 99% because you're not
> in it.

Some of the proposed ways to implement the minimum distro kernel would not 
allow you to override the distro defaults because they would be 
implemented by setting dependancies, not by selecting options that you as 
the user could then unselect.

If you have to edit the .config file, and then have your next make *config 
override your edits (due to dependancy resolution) and so have to edit the 
.config file again this is a really ugly way of working.

David Lang

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Linus Torvalds]

On Mon, Jul 16, 2012 at 12:26 PM,  <david@lang.hm> wrote:
>
> Some of the proposed ways to implement the minimum distro kernel would not
> allow you to override the distro defaults because they would be implemented
> by setting dependancies, not by selecting options that you as the user could
> then unselect.

The sanest thing to do is just a list of "select" statements. And in
any case it would have to depend on the "distro config" entry, so EVEN
THEN you could just create the Kconfig file, then edit out the distro
config thing, and then do whatever you want.

So I don't see why you're arguing. Even if it is very much a "if you
select the FEDORA_X kconfig option, that will automatically force all
those other options on", I don't see why you are so upset. Just set
it, generate your kconfig, and then edit to file to NOT set the distro
config. Then you can do whatever the hell you want.

What I'm upset about is simply the fact that EVEN IF your arguments
were valid (and I don't think they are: see above for trivial fix) I
think your arguments are bogus, for the simple reason that what you
ask for isn't even what the whole "distro minconfig" is at all about.
It's about normal users who absolutely DO NOT want to edit the config.
People who want to say "I know I have intel/amd/nvidia graphics,
standard USB, and no CD-ROM drive". They couldn't care *less* about
SELINUX, and in fact that very issue is why we should have this.

Anybody who says "I want to run Fedora without SELINUX because I do my
own security development" is by *definition* not relevant to the whole
feature.

            Linus

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Mon, 16 Jul 2012, Linus Torvalds wrote:

> On Mon, Jul 16, 2012 at 12:26 PM,  <david@lang.hm> wrote:
>>
>> Some of the proposed ways to implement the minimum distro kernel would not
>> allow you to override the distro defaults because they would be implemented
>> by setting dependancies, not by selecting options that you as the user could
>> then unselect.
>
> The sanest thing to do is just a list of "select" statements. And in
> any case it would have to depend on the "distro config" entry, so EVEN
> THEN you could just create the Kconfig file, then edit out the distro
> config thing, and then do whatever you want.
>
> So I don't see why you're arguing. Even if it is very much a "if you
> select the FEDORA_X kconfig option, that will automatically force all
> those other options on", I don't see why you are so upset. Just set
> it, generate your kconfig, and then edit to file to NOT set the distro
> config. Then you can do whatever the hell you want.
>
> What I'm upset about is simply the fact that EVEN IF your arguments
> were valid (and I don't think they are: see above for trivial fix) I
> think your arguments are bogus, for the simple reason that what you
> ask for isn't even what the whole "distro minconfig" is at all about.
> It's about normal users who absolutely DO NOT want to edit the config.
> People who want to say "I know I have intel/amd/nvidia graphics,
> standard USB, and no CD-ROM drive". They couldn't care *less* about
> SELINUX, and in fact that very issue is why we should have this.
>
> Anybody who says "I want to run Fedora without SELINUX because I do my
> own security development" is by *definition* not relevant to the whole
> feature.

Don't mistake the example for the feature. the SELINUX thing is just an 
example. As Alan Cox commented, taking a distro config and disabling one 
thing is a common troubleshooting request from kernel developers.

I am not that familiar with the internals of kconfig, but my understanding 
was that if you select feature X that then turns on A, B, and C, if you 
then edit the .config file to remove feature X features A, B, and C will 
be turned off the next time you do a make *config. If this is not the 
case, then it's not as horrible as I was thinking.

However, there's still the question of how many different variations of 
how many different distros are you going to accept into the upstream 
kernel. How many different versions of Fedora are you going to be willing 
to have in the menu? This seems to me to be the defconfig problem 
multiplied by many distros and many versions of each distro. If we can 
eliminate this combinational explosion by referencing something outside of 
the kernel tree, it seems to me that it's a big win.

As Alan Cox put it in code

> make distroconfig
>       [cp /etc/system-kconfig(.$ARCH?) .config
>        make oldconfig]
> 
> make menuconfig (if you want to customise)

with the distros providing /etc/system-kconfig files seems far more 
scalable as well as far more flexible than either setting up a lot of 
dependancies for new tags or adding a field to every option to make it 
part of some profile.

This approach works both for the users who just want to say "I know I have 
intel/amd/nvidia graphics, standard USB, and no CD-ROM drive", for the 
users who know a bit more, but want to start with a known point, and for 
the production sysadmins who want to use the same mechanism for other 
purposes (standardizing builds for systems as kernel versions move 
forward)

David Lang

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Geert Uytterhoeven]

On Mon, Jul 16, 2012 at 10:56 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Mon, Jul 16, 2012 at 12:26 PM,  <david@lang.hm> wrote:
>> Some of the proposed ways to implement the minimum distro kernel would not
>> allow you to override the distro defaults because they would be implemented
>> by setting dependancies, not by selecting options that you as the user could
>> then unselect.
>
> The sanest thing to do is just a list of "select" statements. And in
> any case it would have to depend on the "distro config" entry, so EVEN
> THEN you could just create the Kconfig file, then edit out the distro
> config thing, and then do whatever you want.

Except that "select" is one of the ugliest things in Kconfig, as it
blindly sets a symbol
without checking if its dependencies are fulfilled.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Ingo Molnar]

* david@lang.hm <david@lang.hm> wrote:

> > Anybody who says "I want to run Fedora without SELINUX 
> > because I do my own security development" is by *definition* 
> > not relevant to the whole feature.
> 
> Don't mistake the example for the feature. the SELINUX thing 
> is just an example. As Alan Cox commented, taking a distro 
> config and disabling one thing is a common troubleshooting 
> request from kernel developers.

It's still irrelevant:

 - if a user chooses a distro config it means that he is using 
   that distro. Disabling an essential component of the distro 
   config, even if a kernel developer asks for it, will likely 
   break that distro and is thus a dumb thing to do. (the
   typical user will also be unlikely to be *able* to edit a 
   .config and make sure it works.)

 - Furthermore, there's *already* over ten thousand select's in 
   our Kconfig's, and it's already hard at times to disable 
   dependent options.

 - I've been using what Linus suggested for many years via 
   private patches to do bootable randconfig testing and the 
   concept works just fine - enabling a distro specific 
   minconfig is absolutely useful, I'm glad it's being pursued 
   upstream as well...

So what you are arguing about is IMO irrelevant, it is 
immaterial to the problem at hand and the concept works just 
fine in practice.

Thanks,

	Ingo

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Wed, 18 Jul 2012, Ingo Molnar wrote:

> * david@lang.hm <david@lang.hm> wrote:
>
>>> Anybody who says "I want to run Fedora without SELINUX
>>> because I do my own security development" is by *definition*
>>> not relevant to the whole feature.
>>
>> Don't mistake the example for the feature. the SELINUX thing
>> is just an example. As Alan Cox commented, taking a distro
>> config and disabling one thing is a common troubleshooting
>> request from kernel developers.
>
> It's still irrelevant:
>
> - if a user chooses a distro config it means that he is using
>   that distro. Disabling an essential component of the distro
>   config, even if a kernel developer asks for it, will likely
>   break that distro and is thus a dumb thing to do. (the
>   typical user will also be unlikely to be *able* to edit a
>   .config and make sure it works.)

that's assuming that everything listed really is essential.

The history of features defaulting to 'Y' in the existing kernel config 
doesn't give me great confidence that reality will be anywhere close to 
this ideal.

> - Furthermore, there's *already* over ten thousand select's in
>   our Kconfig's, and it's already hard at times to disable
>   dependent options.
>
> - I've been using what Linus suggested for many years via
>   private patches to do bootable randconfig testing and the
>   concept works just fine - enabling a distro specific
>   minconfig is absolutely useful, I'm glad it's being pursued
>   upstream as well...
>
> So what you are arguing about is IMO irrelevant, it is
> immaterial to the problem at hand and the concept works just
> fine in practice.

Shrug, you guys have to maintain the result, I'm just a user.

But I don't see why the same logic that kept the kernel installation 
outside of the makefiles and created /sbin/installkernel wouldn't also 
apply here.

using a separate miniconfig in a known place would seem to be less code, 
distribute the work better (as every distro can use it without having to 
patch the same files in the kernel source), and be more flexible.

Flexibility has a way of being leveraged in ways never imagined initially, 
so if it can be gained without complicating the code (and it's 
maintinance), and the initial use case, I always tend to push for the more 
flexible option.

David Lang

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Ingo Molnar]

* david@lang.hm <david@lang.hm> wrote:

> On Wed, 18 Jul 2012, Ingo Molnar wrote:
> 
> >* david@lang.hm <david@lang.hm> wrote:
> >
> >>>Anybody who says "I want to run Fedora without SELINUX
> >>>because I do my own security development" is by *definition*
> >>>not relevant to the whole feature.
> >>
> >>Don't mistake the example for the feature. the SELINUX thing
> >>is just an example. As Alan Cox commented, taking a distro
> >>config and disabling one thing is a common troubleshooting
> >>request from kernel developers.
> >
> >It's still irrelevant:
> >
> >- if a user chooses a distro config it means that he is using
> >  that distro. Disabling an essential component of the distro
> >  config, even if a kernel developer asks for it, will likely
> >  break that distro and is thus a dumb thing to do. (the
> >  typical user will also be unlikely to be *able* to edit a
> >  .config and make sure it works.)
> 
> that's assuming that everything listed really is essential.

See the requirements in Linus's earlier mail:

------------->

The *two* requirements (and they're really the same theme) I
personally think we should have for this are

 - I think every single "select" for these things should come 
with a comment about what it is about and why the distro needs 
it (to show there was some thought involved and not just a blind 
"took it from the distro config")

 - It should be about *minimal* settings. I'd rather have too 
few things and the occasional complaint about "oh, it didn't 
work because it missed XYZ" than have it grow to contain all the 
options just because somebody decided to just add random things 
until things worked.

<-------------

Thanks,

	Ingo

Re: [RFC] Simplifying kernel configuration for distro issues

[Steven Yong]

On Sat, Jul 14, 2012 at 4:37 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> In addition to the "minimal distro settings", we might also have a few
> "common platform" settings, so that you could basically do a "hey, I
> have a modern PC laptop, make it pick the obvious stuff that a normal
> person needs, like USB storage, FAT/VFAT support, the core power
> management etc".

I am a normal user who is trying to compile newer kernel, to me,
minimal settings for "common platform" is more important than minimal
settings for "distro", minimal settings for distro is for sure
important though, thanks for bringing this RFC up for the benefits of
normal users.

But instead of selecting distro myself, can the minimal settings be
auto-selected by checking my current distro?

Steven

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Steven Rostedt]

On Sat, Jul 14, 2012 at 07:48:27PM +0200, Borislav Petkov wrote:
> 
> Let's have an example: when I have to build upstream on a distro here,
> I take the distro config and use it despite that it takes a long time
> to build since everything is module - it is still better for me to
> wait that one time instead of doing a dozen of trial and errors after
> forgetting a config option each time.

This is where 'make localmodconfig' does help. It can remove a lot of
modules for you. And I just recently fixed a bug in the tool that it now
removes even more modules (The fix is in linux-next).

Also, if you are building on another box than what the kernel is for,
you can go to that box and run 'lsmod > /tmp/lsmod'. Copy that file to
the build machine (into /tmp/lsmod), and then run
'make LSMOD=/tmp/lsmod localmodconfig', and this will remove the modules
not used by the target box.

-- Steve

Re: [RFC] Simplifying kernel configuration for distro issues

[Steven Rostedt]

On Fri, Jul 13, 2012 at 02:17:30PM -0700, Linus Torvalds wrote:
> 
> The *two* requirements (and they're really the same theme) I
> personally think we should have for this are
> 
>  -  I think every single "select" for these things should come with a
> comment about what it is about and why the distro needs it (to show
> there was some thought involved and not just a blind "took it from the
> distro config")

What about expanding on Alan's idea. I'm guessing that 99% of the users
build the kernel for the box that they are running. If this is the case,
perhaps we can get the distros to add a:

  /usr/share/Linux/Kconfig

And this Kconfig would have something like:

bool "Distro X config"
 select A
 select B
 select C
 [...]

Perhaps with a comment for each select. Or have the comments in the help
section.

Then have the kernel kbuild system check if this file exists and include
it.

Of course the kbuild system would need to verify that the selects exist,
and perhaps warn if they do not. But the nice thing about this is that
you would get the minconfig for the system you are running. When the
system is updated to a new version, the minconfig would be updated too.
The list of selects would not have to live in the kernel, nor would the
kernel need to maintain the list for N+1 different distributions.


> 
>  - It should be about *minimal* settings. I'd rather have too few
> things and the occasional complaint about "oh, it didn't work because
> it missed XYZ" than have it grow to contain all the options just
> because somebody decided to just add random things until things
> worked.

Side note, and this is for the 1%. If you want a true minconfig for your
system, ktest can do that for you. You can set it up to run a test to
create a minimum config that will boot (and optionally run some test you
specify). It turns off configs in order of importance (chooses those
that select a lot, or are depended on most, first), and sees if it can
boot without the config. The end result can be rather a very small set
of configs.

See tools/testing/ktest/examples/include/min-config.conf for more
details.

-- Steve

> 
> Other than that, even if it only gets you *closer* to a kernel that
> works with that distro, I think it doesn't have to be all that
> perfect. Because the alternative is what we have now.
> 
>            Linus
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [RFC] Simplifying kernel configuration for distro issues

[Linus Torvalds]

On Thu, Jul 19, 2012 at 8:26 AM, Steven Rostedt <rostedt@goodmis.org> wrote:
>
> Side note, and this is for the 1%. If you want a true minconfig for your
> system, ktest can do that for you.

Try it, it's actually much harder than it seems. Like allmodconfig, it
handles the minimum hardware well, but it tends to handle the subtle
issues really badly.

Many config options cause *very* subtle failures that are almost
impossible to see. Like firewalls not loading correctly (and leaving
the machine completely open), or just stuff that you didn't happen to
test (USB sticks, printers, certain programs) not working. Not having
the right audit options will make things still "work", but you'll get
warnings at bootup, and who knows what that causes etc etc.

These kinds of things are exactly why I'd like to have a distro config.

               Linus

Re: [RFC] Simplifying kernel configuration for distro issues

[Josh Boyer]

On Thu, Jul 19, 2012 at 11:26:18AM -0400, Steven Rostedt wrote:
> On Fri, Jul 13, 2012 at 02:17:30PM -0700, Linus Torvalds wrote:
> > 
> > The *two* requirements (and they're really the same theme) I
> > personally think we should have for this are
> > 
> >  -  I think every single "select" for these things should come with a
> > comment about what it is about and why the distro needs it (to show
> > there was some thought involved and not just a blind "took it from the
> > distro config")
> 
> What about expanding on Alan's idea. I'm guessing that 99% of the users
> build the kernel for the box that they are running. If this is the case,
> perhaps we can get the distros to add a:
> 
>   /usr/share/Linux/Kconfig
> 
> And this Kconfig would have something like:
> 
> bool "Distro X config"
>  select A
>  select B
>  select C
>  [...]
> 
> Perhaps with a comment for each select. Or have the comments in the help
> section.
> 
> Then have the kernel kbuild system check if this file exists and include
> it.
> 
> Of course the kbuild system would need to verify that the selects exist,
> and perhaps warn if they do not. But the nice thing about this is that
> you would get the minconfig for the system you are running. When the
> system is updated to a new version, the minconfig would be updated too.
> The list of selects would not have to live in the kernel, nor would the
> kernel need to maintain the list for N+1 different distributions.

Is there a reason you don't want distro maintainers to maintain these
files in the upstream git tree?  (You said "the kernel need to
maintain", but I would expect the distro maintainers to be doing that
work.)

I think it would actually be beneficial to maintain them upstream
instead of in distro kernel packaging.  You'd be able to track the
history of changes with git.  You would see for a given kernel
version what options are set for each distro (e.g. F17 can support
NEW_FOO_THING but F16 userspace can't so it doesn't select that).
Perhaps most importantly, it provides a consolidated view of what
options various distros are setting and allows the distro maintainers to
easily do comparisons.

josh

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Michal Marek]

On 17.7.2012 10:03, Geert Uytterhoeven wrote:
> On Mon, Jul 16, 2012 at 10:56 PM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>> On Mon, Jul 16, 2012 at 12:26 PM,  <david@lang.hm> wrote:
>>> Some of the proposed ways to implement the minimum distro kernel would not
>>> allow you to override the distro defaults because they would be implemented
>>> by setting dependancies, not by selecting options that you as the user could
>>> then unselect.
>>
>> The sanest thing to do is just a list of "select" statements. And in
>> any case it would have to depend on the "distro config" entry, so EVEN
>> THEN you could just create the Kconfig file, then edit out the distro
>> config thing, and then do whatever you want.
> 
> Except that "select" is one of the ugliest things in Kconfig, as it
> blindly sets a symbol
> without checking if its dependencies are fulfilled.

But for the few options Linus proposed (TMPFS, TMPFS_POSIX_*,
DEVTMPFS(_MOUNT)), the amount of additional dependencies is reasonable.
For something more advanced like 'build me a kernel for a laptop with
$VENDOR hardware', we would need a better dependency solver, indeed.

Michal

Re: [RFC] Simplifying kernel configuration for distro issues

[Steven Rostedt]

On Thu, 2012-07-19 at 11:45 -0400, Josh Boyer wrote:
> Of course the kbuild system would need to verify that the selects exist,
> > and perhaps warn if they do not. But the nice thing about this is that
> > you would get the minconfig for the system you are running. When the
> > system is updated to a new version, the minconfig would be updated too.
> > The list of selects would not have to live in the kernel, nor would the
> > kernel need to maintain the list for N+1 different distributions.
> 
> Is there a reason you don't want distro maintainers to maintain these
> files in the upstream git tree?  (You said "the kernel need to
> maintain", but I would expect the distro maintainers to be doing that
> work.)
> 
> I think it would actually be beneficial to maintain them upstream
> instead of in distro kernel packaging.  You'd be able to track the
> history of changes with git.  You would see for a given kernel
> version what options are set for each distro (e.g. F17 can support
> NEW_FOO_THING but F16 userspace can't so it doesn't select that).
> Perhaps most importantly, it provides a consolidated view of what
> options various distros are setting and allows the distro maintainers to
> easily do comparisons.

Then we'll have a list of options in each kernel:

 Fedora 16
 Fedora 17
 Fedora 18
 [...]
 Debian x
 Debian x+1
 Debian x+2
 [...]
 Ubuntu y
 Ubuntu y+1
 [...]

What about older kernels? Say you installed Fedora 18 with an older
kernel that doesn't know what to select? Having the distro tell the
kernel what it needs seems to me the easiest for the 99% case.

Also, if something isn't supported by the older kernel, it would warn
the user about it. That way the user can be told that their older kernel
won't work with this version of the distro. And there wont be as many
surprises. If the user is told "your init wont work with this kernel"
before they compile it, then they shouldn't complain if they decide to
install this older kernel and their box doesn't boot.

-- Steve

Re: [RFC] Simplifying kernel configuration for distro issues

[Steven Rostedt]

On Thu, 2012-07-19 at 08:43 -0700, Linus Torvalds wrote:
> On Thu, Jul 19, 2012 at 8:26 AM, Steven Rostedt <rostedt@goodmis.org> wrote:
> >
> > Side note, and this is for the 1%. If you want a true minconfig for your
> > system, ktest can do that for you.
> 
> Try it, it's actually much harder than it seems. Like allmodconfig, it
> handles the minimum hardware well, but it tends to handle the subtle
> issues really badly.
> 
> Many config options cause *very* subtle failures that are almost
> impossible to see. Like firewalls not loading correctly (and leaving
> the machine completely open), or just stuff that you didn't happen to
> test (USB sticks, printers, certain programs) not working. Not having
> the right audit options will make things still "work", but you'll get
> warnings at bootup, and who knows what that causes etc etc.
> 
> These kinds of things are exactly why I'd like to have a distro config.

This is why it was more of a side note, and for the 1%. If there's
things you have tests for, to confirm that they work, you could add
those to the TEST option, and the config generated will guarantee to fix
them.

But as you stated, there's lots of subtle things that can go wrong. I
was just posting this as a plug for ktest ;-)

For what you want, I think having the distro supply
a /usr/share/Linux/Kconfig that the linux build system can use would be
very helpful.

-- Steve

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Borislav Petkov]

On Thu, Jul 19, 2012 at 10:42:17AM -0400, Steven Rostedt wrote:
> On Sat, Jul 14, 2012 at 07:48:27PM +0200, Borislav Petkov wrote:
> > 
> > Let's have an example: when I have to build upstream on a distro here,
> > I take the distro config and use it despite that it takes a long time
> > to build since everything is module - it is still better for me to
> > wait that one time instead of doing a dozen of trial and errors after
> > forgetting a config option each time.
> 
> This is where 'make localmodconfig' does help. It can remove a lot of
> modules for you. And I just recently fixed a bug in the tool that it now
> removes even more modules (The fix is in linux-next).

Even more modules? When is enough, enough? :-)

> Also, if you are building on another box than what the kernel is for,
> you can go to that box and run 'lsmod > /tmp/lsmod'. Copy that file to
> the build machine (into /tmp/lsmod), and then run
> 'make LSMOD=/tmp/lsmod localmodconfig', and this will remove the modules
> not used by the target box.

Seriously, this helps only in the cases where the stuff the distro
actually needs is in modules. So, there probably are obscure situations
where you need to enable stuff which is bool and not M. Hopefully those
cases are seldom enough so thanks for this, I'll try that the next time.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
GM: Alberto Bozzo
Reg: Dornach, Landkreis Muenchen
HRB Nr. 43632 WEEE Registernr: 129 19551

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Steven Rostedt]

On Thu, 2012-07-19 at 18:48 +0200, Borislav Petkov wrote:

> > Also, if you are building on another box than what the kernel is for,
> > you can go to that box and run 'lsmod > /tmp/lsmod'. Copy that file to
> > the build machine (into /tmp/lsmod), and then run
> > 'make LSMOD=/tmp/lsmod localmodconfig', and this will remove the modules
> > not used by the target box.
> 
> Seriously, this helps only in the cases where the stuff the distro
> actually needs is in modules. So, there probably are obscure situations
> where you need to enable stuff which is bool and not M. Hopefully those
> cases are seldom enough so thanks for this, I'll try that the next time.
> 

This is why I created the make-min-config in ktest. It keeps on
disabling configs to see what the machine needs to boot (and optionally
run some test), and what configs it can disable. It does not touch the
multi options though.

It creates two configs. One that has the configs that it can't turn off
(still enabled with a make allnoconfig, or selected by something that it
must have), and a config that just has the configs that 'if I disable
this, the box doesn't boot'.

Here's an example:

For my min-config files with the configs that couldn't be turned off:

$ wc -l config-min*
  117 config-min
  139 config-min-net

The config-min will get the box to boot (no network). The -net, adds
enough to ssh to the box.

$ wc -l config-skip*
 11 config-skip
 14 config-skip-net

The above are the configs that ktest found if it disabled, would not
boot (or ssh).

$ cat config-skip-net
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SATA_AHCI=y
CONFIG_E1000=y
CONFIG_QUOTA=y
CONFIG_ATA=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_DEVTMPFS=y
CONFIG_EXT4_FS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_SERIAL_8250=y
CONFIG_BLK_DEV_SD=y
CONFIG_NET=y
CONFIG_NETDEVICES=y

I can pass the above to a allnoconfig, and the box will boot and allow
ssh. Note, the reason for the serial config, is that this ktest run uses
a serial port to see if the box booted. If the serial isn't there, then
it thinks it failed.

-- Steve

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Linus Torvalds]

On Thu, Jul 19, 2012 at 9:48 AM, Borislav Petkov <bp@amd64.org> wrote:
>
> Seriously, this helps only in the cases where the stuff the distro
> actually needs is in modules. So, there probably are obscure situations
> where you need to enable stuff which is bool and not M.

Sadly, not obscure at all.

Most of the *drivers* are modules, but most of the "distro config"
options are indeed booleans (or, if tristate, =y).

Even driver-wise, there are some things that are often =y, even though
you generally don't want them. PCMCIA? Not even *laptops* have that
shit any more, but having built-in cardbus support almost certainly
helps in a distro kernel for booting of certain odder cases.

Xen support? Odd partition tables? All the different AGP versions?
Many of us couldn't care less, but again, it makes sense in the actual
distro kernel, even if it does *not* necessarily make sense in a
personalized one.

So doing "make allmodconfig" is certainly a workable thing (modulo the
modules that you need for stuff you hadn't happened to use), but it's
not wonderful.

I also hate having to enable support for modules. A non-modular build
is quicker to build and avoids some security issues. Some drivers
don't work well built-in (they load firmware etc too early), but imho
it's worth doing if you can, and it's something we should make easy
for people to do because of the security side (of course, per-build
randomly generated keys and signed modules with the keys deleted after
the build would be reasonably equivalent from a security standpoint,
but we're not there yet).

              Linus

Re: [RFC] Simplifying kernel configuration for distro issues

[Josh Boyer]

On Thu, Jul 19, 2012 at 12:08:08PM -0400, Steven Rostedt wrote:
> On Thu, 2012-07-19 at 11:45 -0400, Josh Boyer wrote:
> > Of course the kbuild system would need to verify that the selects exist,
> > > and perhaps warn if they do not. But the nice thing about this is that
> > > you would get the minconfig for the system you are running. When the
> > > system is updated to a new version, the minconfig would be updated too.
> > > The list of selects would not have to live in the kernel, nor would the
> > > kernel need to maintain the list for N+1 different distributions.
> > 
> > Is there a reason you don't want distro maintainers to maintain these
> > files in the upstream git tree?  (You said "the kernel need to
> > maintain", but I would expect the distro maintainers to be doing that
> > work.)
> > 
> > I think it would actually be beneficial to maintain them upstream
> > instead of in distro kernel packaging.  You'd be able to track the
> > history of changes with git.  You would see for a given kernel
> > version what options are set for each distro (e.g. F17 can support
> > NEW_FOO_THING but F16 userspace can't so it doesn't select that).
> > Perhaps most importantly, it provides a consolidated view of what
> > options various distros are setting and allows the distro maintainers to
> > easily do comparisons.
> 
> Then we'll have a list of options in each kernel:
> 
>  Fedora 16
>  Fedora 17
>  Fedora 18
>  [...]
>  Debian x
>  Debian x+1
>  Debian x+2
>  [...]
>  Ubuntu y
>  Ubuntu y+1
>  [...]

Well, yes.  I was thinking it would be more like:

distro/Kconfig.fedora
	menuconfig FEDORA
	if FEDORA
	config FEDORA_16
	   select WHATEVER
	config FEDORA_17
	...

distro/Kconfig.debian
	menuconfig DEBIAN
	if DEBIAN
	config DEBIAN_X
	...

etc.

Not one giant distro file with a bunch of varying distros doing a bunch
of selects.  But in general, yes there would be options for each
supported distro release.

> What about older kernels? Say you installed Fedora 18 with an older
> kernel that doesn't know what to select? Having the distro tell the
> kernel what it needs seems to me the easiest for the 99% case.

How is the above not telling the kernel what it needs?  I'm confused how
the location of such a file makes it's functionality and usefulness
differ...  Quite possible I missed what you meant originally, but it
sounds like we're talking about the same thing?

Also, I'm not very convinced the 99% are going to be wanting to install
shiny new versions of a distro with a kernel older than what the distro
ships with.  I could be very wrong, but it seems like in-general the
whole premise of this RFC was geared towards using new kernels on
distros.

> Also, if something isn't supported by the older kernel, it would warn
> the user about it. That way the user can be told that their older kernel
> won't work with this version of the distro. And there wont be as many
> surprises. If the user is told "your init wont work with this kernel"
> before they compile it, then they shouldn't complain if they decide to
> install this older kernel and their box doesn't boot.

kconfig already spits out warnings for symbols being selected that
don't exist.

josh

Re: [RFC] Simplifying kernel configuration for distro issues

[Alan Cox]

> Well, yes.  I was thinking it would be more like:
> 
> distro/Kconfig.fedora
> 	menuconfig FEDORA
> 	if FEDORA
> 	config FEDORA_16
> 	   select WHATEVER
> 	config FEDORA_17

Nope you need

distro/everyarchtheyship/everykernelvarianttkeyship(smp,largemem,arm
boards)/Kconfig

which for some distros is over 20 per release and the end user wouldn't
have a cat in hells chance of knowing which to pick.

For the end user case you need the distro to plonk the right file in the
right place and be done with it, once they do that the rest is
bikeshedding a ten line Makefile rule.

Alan

Re: [RFC] Simplifying kernel configuration for distro issues

[Steven Rostedt]

On Thu, 2012-07-19 at 13:19 -0400, Josh Boyer wrote:

> > What about older kernels? Say you installed Fedora 18 with an older
> > kernel that doesn't know what to select? Having the distro tell the
> > kernel what it needs seems to me the easiest for the 99% case.
> 
> How is the above not telling the kernel what it needs?  I'm confused how
> the location of such a file makes it's functionality and usefulness
> differ...  Quite possible I missed what you meant originally, but it
> sounds like we're talking about the same thing?

The point is, the user wont have to think "What distro am I running? and
what version am I running?". I don't even know what version of the disto
I'm currently running (Debian testing).

The point is, the current running distro supplies what is needed from
the kernel in order to work properly. The user does not need to 'select'
it. They would only have to select a 'add my distro min configs'.

A developer working with a user could just say, "select disto config"
without needing to know what distro the user has.

What happens if someone does a yum update, and the kernel requirement
changes slightly. The yum update should update
this /usr/share/Linux/Kconfig. But it's still set at Fedora X. The
kernel can not be updated for these slight changes.

> 
> Also, I'm not very convinced the 99% are going to be wanting to install
> shiny new versions of a distro with a kernel older than what the distro
> ships with.  I could be very wrong, but it seems like in-general the
> whole premise of this RFC was geared towards using new kernels on
> distros.

There are times when the update breaks something. A user may backport to
an older kernel where their Gizmo worked. I've done this to get webcams
working. I know I'm not the 99%, but the rational for my operation was a
99% thing to do: Crap, I upgraded my kernel and now my webcam doesn't
work. Oh well, download an older version and boot that one.

> 
> > Also, if something isn't supported by the older kernel, it would warn
> > the user about it. That way the user can be told that their older kernel
> > won't work with this version of the distro. And there wont be as many
> > surprises. If the user is told "your init wont work with this kernel"
> > before they compile it, then they shouldn't complain if they decide to
> > install this older kernel and their box doesn't boot.
> 
> kconfig already spits out warnings for symbols being selected that
> don't exist.

We can make these even bigger :-)   Add lots of stars (*) around them!

-- Steve

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Borislav Petkov]

On Thu, Jul 19, 2012 at 01:02:46PM -0400, Steven Rostedt wrote:
> This is why I created the make-min-config in ktest. It keeps on
> disabling configs to see what the machine needs to boot (and optionally
> run some test), and what configs it can disable. It does not touch the
> multi options though.
> 
> It creates two configs. One that has the configs that it can't turn off
> (still enabled with a make allnoconfig, or selected by something that it
> must have), and a config that just has the configs that 'if I disable
> this, the box doesn't boot'.
> 
> Here's an example:
> 
> For my min-config files with the configs that couldn't be turned off:
> 
> $ wc -l config-min*
>   117 config-min
>   139 config-min-net
> 
> The config-min will get the box to boot (no network). The -net, adds
> enough to ssh to the box.
> 
> $ wc -l config-skip*
>  11 config-skip
>  14 config-skip-net
> 
> The above are the configs that ktest found if it disabled, would not
> boot (or ssh).
> 
> $ cat config-skip-net
> CONFIG_SERIAL_8250_CONSOLE=y
> CONFIG_SATA_AHCI=y
> CONFIG_E1000=y
> CONFIG_QUOTA=y
> CONFIG_ATA=y
> CONFIG_UNIX=y
> CONFIG_INET=y
> CONFIG_DEVTMPFS=y
> CONFIG_EXT4_FS=y
> CONFIG_DEVTMPFS_MOUNT=y
> CONFIG_SERIAL_8250=y
> CONFIG_BLK_DEV_SD=y
> CONFIG_NET=y
> CONFIG_NETDEVICES=y
> 
> I can pass the above to a allnoconfig, and the box will boot and allow
> ssh. Note, the reason for the serial config, is that this ktest run uses
> a serial port to see if the box booted. If the serial isn't there, then
> it thinks it failed.

I agree with all this and you've explained this to me live already so
you're preaching to the choir.

But it would be a lot faster/easier if users can select, let's call'em
"profiles" which are not mutually exclusive and can speed up the
configuration process. They can either be distro-specific or generic,
selecting certain features you need.

So configuring your kernel would be like shopping without paying too
much attention to details. Let's look into the head of a person doing a
config like that and read some of her thoughts :):

"Hm, ok, this new configurator is cool, a lot faster I gotta say... So,
what do I need, ah, yes, it is an AMD laptop so from vendors I select
AMD, then I probably need ext4, then I'd like to do packet filtering
so I should enable iptables.. Oh, I'd like to do tracing too so let's
enable tracing and trust Steven with the options he's added by default,
then I need ahci, I'd also like to do encrypted partitions so I'll
enable device mapper with crypto... "

So all those things could be selectable from that profiles menu without
having to go through the gazillion of little suboptions and having to
read help (which is sometimes completely helpless) and figure out do I
need it or not.

And this would simplify configuration a lot. IMHO, anyway.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
GM: Alberto Bozzo
Reg: Dornach, Landkreis Muenchen
HRB Nr. 43632 WEEE Registernr: 129 19551

Re: [RFC] Simplifying kernel configuration for distro issues

[Josh Boyer]

On Thu, Jul 19, 2012 at 06:30:47PM +0100, Alan Cox wrote:
> 
> > Well, yes.  I was thinking it would be more like:
> > 
> > distro/Kconfig.fedora
> > 	menuconfig FEDORA
> > 	if FEDORA
> > 	config FEDORA_16
> > 	   select WHATEVER
> > 	config FEDORA_17
> 
> Nope you need
> 
> distro/everyarchtheyship/everykernelvarianttkeyship(smp,largemem,arm
> boards)/Kconfig
> 
> which for some distros is over 20 per release and the end user wouldn't
> have a cat in hells chance of knowing which to pick.

I wasn't include arch-specific options in the "minimal distro config"
stuff.  That doesn't seem minimal to me.  I was thinking more along the
lines of "distro X needs CGROUPS, SELINUX, HOTPLUG, DEVTMPFS, namespace
stuff".  Stuff that they need that is basically architecture
independent that the distro userspace needs.

Having the distro provide files that select architecture specific
options and variations of that really doesn't seem any better than what
most of them do already, which is just ship the whole damn config file
in /boot (or some other location).

> For the end user case you need the distro to plonk the right file in the
> right place and be done with it, once they do that the rest is
> bikeshedding a ten line Makefile rule.

If people want the distros to plonk some architecture+distro specific
minimal config file down as part of the packaging, I guess that's a
thing that could be done.  I'd honestly wonder if maintaining X number
of those in the packaging is something the distro maintainers would
really like to do, but one can always hope.

josh

Re: [RFC] Simplifying kernel configuration for distro issues

[Alan Cox]

> > kconfig already spits out warnings for symbols being selected that
> > don't exist.
> 
> We can make these even bigger :-)   Add lots of stars (*) around them!

Make oldconfig already handles this just fine

Alan

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Borislav Petkov]

On Thu, Jul 19, 2012 at 10:06:44AM -0700, Linus Torvalds wrote:
> On Thu, Jul 19, 2012 at 9:48 AM, Borislav Petkov <bp@amd64.org> wrote:
> >
> > Seriously, this helps only in the cases where the stuff the distro
> > actually needs is in modules. So, there probably are obscure situations
> > where you need to enable stuff which is bool and not M.
> 
> Sadly, not obscure at all.
> 
> Most of the *drivers* are modules, but most of the "distro config"
> options are indeed booleans (or, if tristate, =y).
> 
> Even driver-wise, there are some things that are often =y, even though
> you generally don't want them.

Tell me about it. I'm always pissed off when someone thinks his stuff is
very important and sets his sacred option to be =y/=m by default so the
wider audience can at least compile-test it while the majority of the
machines don't actually need it.

A more coarse-grained config where most of the stuff is off by default
could take care of that probably.

> PCMCIA? Not even *laptops* have that shit any more, but having
> built-in cardbus support almost certainly helps in a distro kernel for
> booting of certain odder cases.

Yeah, distros need the one-size-fits-all thing so they have to enable
*everything*.

> Xen support? Odd partition tables? All the different AGP versions?
> Many of us couldn't care less, but again, it makes sense in the actual
> distro kernel, even if it does *not* necessarily make sense in a
> personalized one.

Yep.

> So doing "make allmodconfig" is certainly a workable thing (modulo the
> modules that you need for stuff you hadn't happened to use), but it's
> not wonderful.

Oh and I always aim to build distro kernels on a big machine -
allmodconfig build is no fun on a tiny laptop. So would it be better
to have better profiled kernels, obviating the need for an almost full
build? Hell yeah!

> I also hate having to enable support for modules. A non-modular build
> is quicker to build and avoids some security issues. Some drivers
> don't work well built-in (they load firmware etc too early), but imho
> it's worth doing if you can, and it's something we should make easy
> for people to do because of the security side (of course, per-build
> randomly generated keys and signed modules with the keys deleted after
> the build would be reasonably equivalent from a security standpoint,
> but we're not there yet).

Agreed.

So there are some not-so-obscure situations, judging by your examples
above. Ho-humm.

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
GM: Alberto Bozzo
Reg: Dornach, Landkreis Muenchen
HRB Nr. 43632 WEEE Registernr: 129 19551

Re: [RFC] Simplifying kernel configuration for distro issues

[Josh Boyer]

On Thu, Jul 19, 2012 at 01:33:42PM -0400, Steven Rostedt wrote:
> On Thu, 2012-07-19 at 13:19 -0400, Josh Boyer wrote:
> 
> > > What about older kernels? Say you installed Fedora 18 with an older
> > > kernel that doesn't know what to select? Having the distro tell the
> > > kernel what it needs seems to me the easiest for the 99% case.
> > 
> > How is the above not telling the kernel what it needs?  I'm confused how
> > the location of such a file makes it's functionality and usefulness
> > differ...  Quite possible I missed what you meant originally, but it
> > sounds like we're talking about the same thing?
> 
> The point is, the user wont have to think "What distro am I running? and
> what version am I running?". I don't even know what version of the disto
> I'm currently running (Debian testing).
>
> The point is, the current running distro supplies what is needed from
> the kernel in order to work properly. The user does not need to 'select'
> it. They would only have to select a 'add my distro min configs'.

Distros aren't stationary things.  I mean, some of them certainly aim
for that goal, but userspace and kernels get upgraded all the time.  So
if this distro-Kconfig file is provided by some package _other_ than the
kernel the distros are going to have a bit of a hassle keeping track of
it.

> A developer working with a user could just say, "select disto config"
> without needing to know what distro the user has.
> 
> What happens if someone does a yum update, and the kernel requirement
> changes slightly. The yum update should update
> this /usr/share/Linux/Kconfig. But it's still set at Fedora X. The
> kernel can not be updated for these slight changes.

I'm not quite following what you mean in the yum update case, sorry.

> > Also, I'm not very convinced the 99% are going to be wanting to install
> > shiny new versions of a distro with a kernel older than what the distro
> > ships with.  I could be very wrong, but it seems like in-general the
> > whole premise of this RFC was geared towards using new kernels on
> > distros.
> 
> There are times when the update breaks something. A user may backport to
> an older kernel where their Gizmo worked. I've done this to get webcams
> working. I know I'm not the 99%, but the rational for my operation was a
> 99% thing to do: Crap, I upgraded my kernel and now my webcam doesn't
> work. Oh well, download an older version and boot that one.

Upgraded the kernel within the confines of that distro, right?  So you
go back to what was already installed and working.  You don't go back
arbitrarily far just to see what happens.  I would think a reasonably
crafted distro config would work in those scenarios.

josh

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Steven Rostedt]

On Thu, 2012-07-19 at 19:34 +0200, Borislav Petkov wrote:

> > I can pass the above to a allnoconfig, and the box will boot and allow
> > ssh. Note, the reason for the serial config, is that this ktest run uses
> > a serial port to see if the box booted. If the serial isn't there, then
> > it thinks it failed.
> 
> I agree with all this and you've explained this to me live already so
> you're preaching to the choir.

Yes, I know you know this already, as we discussed it in a pub over a
beer (choir practice). But this is a public forum on LKML (the church),
where I now have an audience of heathens. Convert! Convert! You are all
sinners!

> 
> But it would be a lot faster/easier if users can select, let's call'em
> "profiles" which are not mutually exclusive and can speed up the
> configuration process. They can either be distro-specific or generic,
> selecting certain features you need.
> 
> So configuring your kernel would be like shopping without paying too
> much attention to details. Let's look into the head of a person doing a
> config like that and read some of her thoughts :):
> 
> "Hm, ok, this new configurator is cool, a lot faster I gotta say... So,
> what do I need, ah, yes, it is an AMD laptop so from vendors I select
> AMD, then I probably need ext4, then I'd like to do packet filtering
> so I should enable iptables.. Oh, I'd like to do tracing too so let's
> enable tracing and trust Steven with the options he's added by default,
> then I need ahci, I'd also like to do encrypted partitions so I'll
> enable device mapper with crypto... "
> 
> So all those things could be selectable from that profiles menu without
> having to go through the gazillion of little suboptions and having to
> read help (which is sometimes completely helpless) and figure out do I
> need it or not.
> 
> And this would simplify configuration a lot. IMHO, anyway.
> 

I totally agree with this. It would be nice to have a profile list where
you can pick and chose what you have installed:

	network
	nfs
	ext3
	serial
	xen
	kvm
etc etc

Where you can pick and choose what general features you want and it
selects all the core infrastructure to get those features usable. It
wouldn't select the device modules needed, you will still need to select
what hardware you have. But it gets most of the work done for you.

But this still doesn't solve Linus's initial request. That would be a
single option that makes your distro boot, and work well. Again, that
option wont have the drivers needed, but it will enable all the core
infrastructure that you need.

Going with my /usr/share/Linux/Kconfig, this could use the profile
options as well. And just select those that are required. But then
again, Linus did want a minimum selection of stuff.

Side note (again), IIRC, "select" has a bug. If you have Config X
selecting config Y but Y depends on Z, if you enable X, it will enable Y
without enabling Z. I think there were some patches to address this, but
I don't remember.

-- Steve

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Borislav Petkov]

On Thu, Jul 19, 2012 at 01:57:26PM -0400, Steven Rostedt wrote:
> Yes, I know you know this already, as we discussed it in a pub over a
> beer (choir practice). But this is a public forum on LKML (the church),
> where I now have an audience of heathens. Convert! Convert! You are all
> sinners!

Ah, gotcha.

[ … ]

> But this still doesn't solve Linus's initial request. That would be a
> single option that makes your distro boot, and work well. Again, that
> option wont have the drivers needed, but it will enable all the core
> infrastructure that you need.

Oh I'm being additive here. You'll have feature profiles for the stuff
we talk above and distro profiles which solve Linus' issue. Basically
one coarse-grained config option will either select a feature which has
a lot of small subfeatures of which some are sane and want to be enabled
by default when selecting the topfeature.

Or a distro-specific feature which could itself select other
topfeatures.

I haven't tried this in reality to actually be able to say that a
tree-like configure approach would actually make sense and work. It
sounds like a nice idea though, especially having the hierarchical
structure. :)

-- 
Regards/Gruss,
Boris.

Advanced Micro Devices GmbH
Einsteinring 24, 85609 Dornach
GM: Alberto Bozzo
Reg: Dornach, Landkreis Muenchen
HRB Nr. 43632 WEEE Registernr: 129 19551

Re: [RFC] Simplifying kernel configuration for distro issues

[Steven Rostedt]

On Thu, 2012-07-19 at 13:56 -0400, Josh Boyer wrote:

> Distros aren't stationary things.

Exactly my point.

>   I mean, some of them certainly aim
> for that goal, but userspace and kernels get upgraded all the time.  So
> if this distro-Kconfig file is provided by some package _other_ than the
> kernel the distros are going to have a bit of a hassle keeping track of
> it.

How about a directory called /usr/share/Linux/Kconfig.d/

Then have anything installed that needs to work correctly put in its
minimum (must have) requirement configs of the kernel.

Say you are running Debian, and decide to try out systemd. If you set up
your system to run that it would add a file called:

/usr/share/Linux/Kconfig.d/systemd.conf

or something, and this would select things like CGROUPS and the like. We
could make the kernel build select all, or individual files in this
directory. All for the 'make my distro work' or individual for a 'I want
part of my distro to work' option.



> Upgraded the kernel within the confines of that distro, right?  So you
> go back to what was already installed and working.  You don't go back
> arbitrarily far just to see what happens.  I would think a reasonably
> crafted distro config would work in those scenarios.

A reasonable one, but still not the minimum.

One issue with Linus's proposal is that he's asking us to focus on the
99%. But the 99% of who? Because 99% of Linux users do not compile their
own kernels, so he must be asking about the 99% of Linux users that
compile their own kernels. This 99% does not just simply compile their
kernels, but only want to compile the absolutely necessary stuff. That
is, they want their kernels not to include anything they are not using.

A reasonable config would probably need to include a lot that's not
used.

-- Steve

Re: [RFC] Simplifying kernel configuration for distro issues

[Paul Bolle]

On Thu, 2012-07-19 at 13:19 -0400, Josh Boyer wrote:
> kconfig already spits out warnings for symbols being selected that
> don't exist.

Does it? Since when does it do that? Or do you mean select in a more
general way (not just meaning Kconfig's "select" statement)?


Paul Bolle

Re: [RFC] Simplifying kernel configuration for distro issues

[Josh Boyer]

On Thu, Jul 19, 2012 at 08:20:36PM +0200, Paul Bolle wrote:
> On Thu, 2012-07-19 at 13:19 -0400, Josh Boyer wrote:
> > kconfig already spits out warnings for symbols being selected that
> > don't exist.
> 
> Does it? Since when does it do that? Or do you mean select in a more
> general way (not just meaning Kconfig's "select" statement)?

I believe Alan was more correct than me when he said it was 'make
oldconfig' that produced the warnings.

josh

Re: [RFC] Simplifying kernel configuration for distro issues

[Josh Boyer]

On Thu, Jul 19, 2012 at 02:13:40PM -0400, Steven Rostedt wrote:
> On Thu, 2012-07-19 at 13:56 -0400, Josh Boyer wrote:
> 
> > Distros aren't stationary things.
> 
> Exactly my point.
> 
> >   I mean, some of them certainly aim
> > for that goal, but userspace and kernels get upgraded all the time.  So
> > if this distro-Kconfig file is provided by some package _other_ than the
> > kernel the distros are going to have a bit of a hassle keeping track of
> > it.
> 
> How about a directory called /usr/share/Linux/Kconfig.d/
> 
> Then have anything installed that needs to work correctly put in its
> minimum (must have) requirement configs of the kernel.
> 
> Say you are running Debian, and decide to try out systemd. If you set up
> your system to run that it would add a file called:
> 
> /usr/share/Linux/Kconfig.d/systemd.conf
> 
> or something, and this would select things like CGROUPS and the like. We
> could make the kernel build select all, or individual files in this
> directory. All for the 'make my distro work' or individual for a 'I want
> part of my distro to work' option.

That sounds like a pretty good idea, aside from the fact that now your
config is determined by 1) what is currently installed on your system
and 2) people that don't maintain the kernel.

1 is obviously a great thing once you have a stable working set of
packages you use daily, but wouldn't it kind of suck to have to rebuild
the kernel just to install some new package?  I mean... say you wanted
to now use an NFS mount, but you didn't have nfs-utils previously
installed.  So you install it, and it plops the kconfig file in
/usr/share but oops, you have to rebuild the kernel and reboot because
that module isn't built.  Of course I'm extrapolating possibly the worst
usage case here, but it will still happen.

2... yeah.  I don't really know if that is going to pan out, but I am
ever hopeful.  I'd be mostly concerned with people that are coding
userspace applications using every whiz-bang kernel feature.  Or not
paying attention at all to the kernel after the initial file creation
and the options going stale (don't follow renames, etc).


> > Upgraded the kernel within the confines of that distro, right?  So you
> > go back to what was already installed and working.  You don't go back
> > arbitrarily far just to see what happens.  I would think a reasonably
> > crafted distro config would work in those scenarios.
> 
> A reasonable one, but still not the minimum.

The definition of minimum seems to be what we're disagreeing on.  I'm
approaching it from "minimum for a default install of the distro
release".  OK, that and maybe a few common case usages (like NFS, CIFS,
etc).  You seem to be approaching it from literally bare minimum.

> One issue with Linus's proposal is that he's asking us to focus on the
> 99%. But the 99% of who? Because 99% of Linux users do not compile their
> own kernels, so he must be asking about the 99% of Linux users that
> compile their own kernels. This 99% does not just simply compile their
> kernels, but only want to compile the absolutely necessary stuff. That
> is, they want their kernels not to include anything they are not using.
> 
> A reasonable config would probably need to include a lot that's not
> used.

Perhaps.  I thought getting it reasonable would benefit more people,
even at the cost of some smaller bloat than bare minimum.  I don't think
either of us are really wrong, it's more a matter of who is really going
to use this and why I guess.

josh

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Konrad Rzeszutek Wilk]

On Thu, Jul 19, 2012 at 07:53:10PM +0200, Borislav Petkov wrote:
> On Thu, Jul 19, 2012 at 10:06:44AM -0700, Linus Torvalds wrote:
> > On Thu, Jul 19, 2012 at 9:48 AM, Borislav Petkov <bp@amd64.org> wrote:
> > >
> > > Seriously, this helps only in the cases where the stuff the distro
> > > actually needs is in modules. So, there probably are obscure situations
> > > where you need to enable stuff which is bool and not M.
> > 
> > Sadly, not obscure at all.
> > 
> > Most of the *drivers* are modules, but most of the "distro config"
> > options are indeed booleans (or, if tristate, =y).
> > 
> > Even driver-wise, there are some things that are often =y, even though
> > you generally don't want them.
> 
> Tell me about it. I'm always pissed off when someone thinks his stuff is
> very important and sets his sacred option to be =y/=m by default so the
> wider audience can at least compile-test it while the majority of the
> machines don't actually need it.
> 
> A more coarse-grained config where most of the stuff is off by default
> could take care of that probably.
> 
> > PCMCIA? Not even *laptops* have that shit any more, but having
> > built-in cardbus support almost certainly helps in a distro kernel for
> > booting of certain odder cases.
> 
> Yeah, distros need the one-size-fits-all thing so they have to enable
> *everything*.
> 
> > Xen support? Odd partition tables? All the different AGP versions?
> > Many of us couldn't care less, but again, it makes sense in the actual
> > distro kernel, even if it does *not* necessarily make sense in a
> > personalized one.
> 
> Yep.

I proposed something that would solve some of this - but not during
compile time but rather during boot-time
[http://lists.linux-foundation.org/pipermail/ksummit-2012-discuss/2012-June/000099.html]
(interestingly enough hpa was first to propose it 10 years ago :-)

The goal is turn built-in components in well, unloadable components.
That way you won't have at least that much stuff laying around not being
used. Not the full silver bullet, but at least it gets some of this
stuff out of the way and you don't have to worry about the extra
stuff that was built-in.

Re: [RFC] Simplifying kernel configuration for distro issues

[Geert Uytterhoeven]

On Thu, Jul 19, 2012 at 8:22 PM, Josh Boyer <jwboyer@redhat.com> wrote:
> On Thu, Jul 19, 2012 at 08:20:36PM +0200, Paul Bolle wrote:
>> On Thu, 2012-07-19 at 13:19 -0400, Josh Boyer wrote:
>> > kconfig already spits out warnings for symbols being selected that
>> > don't exist.
>
>> Does it? Since when does it do that? Or do you mean select in a more
>> general way (not just meaning Kconfig's "select" statement)?
>
> I believe Alan was more correct than me when he said it was 'make
> oldconfig' that produced the warnings.

Kconfig does spit out warnings for selecting things with unmet dependencies.
But does anyone care?

[...checking logs...]

Oh, only 12 warnings in the v3.5-rc7 builds. Not that bad as my gut feeling
said...

warning: (ETRAX_USB_HOST && MOUSE_APPLETOUCH && MOUSE_BCM5974 &&
MOUSE_SYNAPTICS_USB && JOYSTICK_XPAD && TABLET_USB_ACECAD &&
TABLET_USB_AIPTEK && TABLET_USB_HANWANG && TABLET_USB_KBTAB &&
TABLET_USB_WACOM && TOUCHSCREEN_USB_COMPOSITE && INPUT_ATI_REMOTE2 &&
INPUT_KEYSPAN_REMOTE && INPUT_POWERMATE && INPUT_YEALINK &&
INPUT_CM109) selects USB which has unmet direct dependencies
(USB_SUPPORT && USB_ARCH_HAS_HCD): 10 warnings in 2 logs
warning: (IWLWIFI && IWLEGACY && ATH5K && ATH9K && ATH9K_HTC &&
CARL9170_LEDS) selects MAC80211_LEDS which has unmet direct
dependencies (NET && WIRELESS && MAC80211 && LEDS_CLASS): 10 warnings
in 2 logs
warning: (LOCKDEP && FAULT_INJECTION_STACKTRACE_FILTER && LATENCYTOP
&& KMEMCHECK) selects FRAME_POINTER which has unmet direct
dependencies (DEBUG_KERNEL && (CRIS || M68K || FRV || UML || AVR32 ||
SUPERH || BLACKFIN || MN10300) || ARCH_WANT_FRAME_POINTERS): 10
warnings in 2 logs
warning: (MPC836x_RDK && MTD_NAND_FSL_ELBC && MTD_NAND_FSL_UPM)
selects FSL_LBC which has unmet direct dependencies (FSL_SOC): 9
warnings in 2 logs
warning: (SINGLE_MEMORY_CHUNK) selects NEED_MULTIPLE_NODES which has
unmet direct dependencies (DISCONTIGMEM || NUMA): 9 warnings in 2 logs
warning: (COMPAT) selects COMPAT_BINFMT_ELF which has unmet direct
dependencies (COMPAT && BINFMT_ELF): 3 warnings in 1 logs
warning: (DRM) selects DMA_SHARED_BUFFER which has unmet direct
dependencies (EXPERIMENTAL): 3 warnings in 1 logs
warning: (DRM_RADEON_KMS && DRM_I915 && STUB_POULSBO && FB_BACKLIGHT
&& USB_APPLEDISPLAY && FB_OLPC_DCON && ASUS_LAPTOP && SONY_LAPTOP &&
THINKPAD_ACPI && EEEPC_LAPTOP && ACPI_CMPC && SAMSUNG_Q10 &&
APPLE_GMUX) selects BACKLIGHT_CLASS_DEVICE which has unmet direct
dependencies (HAS_IOMEM && BACKLIGHT_LCD_SUPPORT): 3 warnings in 1
logs
warning: (IA64) selects PM which has unmet direct dependencies
(PM_SLEEP || PM_RUNTIME): 4 warnings in 1 logs
warning: (MOUSE_APPLETOUCH && MOUSE_BCM5974 && MOUSE_SYNAPTICS_USB &&
JOYSTICK_XPAD && TABLET_USB_ACECAD && TABLET_USB_AIPTEK &&
TABLET_USB_HANWANG && TABLET_USB_KBTAB && TABLET_USB_WACOM &&
TOUCHSCREEN_USB_COMPOSITE && INPUT_ATI_REMOTE2 && INPUT_KEYSPAN_REMOTE
&& INPUT_POWERMATE && INPUT_YEALINK && INPUT_CM109 && RC_ATI_REMOTE &&
IR_IMON && IR_MCEUSB && IR_REDRAT3 && IR_STREAMZAP && DRM_USB) selects
USB which has unmet direct dependencies (USB_SUPPORT &&
USB_ARCH_HAS_HCD): 5 warnings in 1 logs
warning: (PLATFORM_AT32AP) selects HAVE_NET_MACB which has unmet
direct dependencies (NETDEVICES && ETHERNET): 3 warnings in 1 logs
warning: (PREEMPT && DEBUG_ATOMIC_SLEEP) selects PREEMPT_COUNT which
has unmet direct dependencies (COLDFIRE): 5 warnings in 1 logs

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [RFC] Simplifying kernel configuration for distro issues

[Paul Bolle]

On Thu, 2012-07-19 at 20:49 +0200, Geert Uytterhoeven wrote:
> > I believe Alan was more correct than me when he said it was 'make
> > oldconfig' that produced the warnings.
> 
> Kconfig does spit out warnings for selecting things with unmet dependencies.
> But does anyone care?
> 
> [...checking logs...]
> 
> Oh, only 12 warnings in the v3.5-rc7 builds. Not that bad as my gut feeling
> said...

Well, that's yet another issue but anyhow. That number of warnings
should presumably drop to (almost) zero if those weren't warnings but
errors. Has that ever been tried?


Paul Bolle

Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Thu, 19 Jul 2012, Josh Boyer wrote:

> On Thu, Jul 19, 2012 at 02:13:40PM -0400, Steven Rostedt wrote:
>> On Thu, 2012-07-19 at 13:56 -0400, Josh Boyer wrote:
>>
>>> Distros aren't stationary things.
>>
>> Exactly my point.
>>
>>>   I mean, some of them certainly aim
>>> for that goal, but userspace and kernels get upgraded all the time.  So
>>> if this distro-Kconfig file is provided by some package _other_ than the
>>> kernel the distros are going to have a bit of a hassle keeping track of
>>> it.
>>
>> How about a directory called /usr/share/Linux/Kconfig.d/
>>
>> Then have anything installed that needs to work correctly put in its
>> minimum (must have) requirement configs of the kernel.
>>
>> Say you are running Debian, and decide to try out systemd. If you set up
>> your system to run that it would add a file called:
>>
>> /usr/share/Linux/Kconfig.d/systemd.conf
>>
>> or something, and this would select things like CGROUPS and the like. We
>> could make the kernel build select all, or individual files in this
>> directory. All for the 'make my distro work' or individual for a 'I want
>> part of my distro to work' option.
>
> That sounds like a pretty good idea, aside from the fact that now your
> config is determined by 1) what is currently installed on your system
> and 2) people that don't maintain the kernel.
>
> 1 is obviously a great thing once you have a stable working set of
> packages you use daily, but wouldn't it kind of suck to have to rebuild
> the kernel just to install some new package?  I mean... say you wanted
> to now use an NFS mount, but you didn't have nfs-utils previously
> installed.  So you install it, and it plops the kconfig file in
> /usr/share but oops, you have to rebuild the kernel and reboot because
> that module isn't built.  Of course I'm extrapolating possibly the worst
> usage case here, but it will still happen.

the alturnative to this is what? compile everything just in case you need 
it some time in the future?

we already have some tools (vmware) that check for the proper kernel 
config when they startup, and if the appropriate stuff isn't there they 
ask for the root password and compile the modules.

> 2... yeah.  I don't really know if that is going to pan out, but I am
> ever hopeful.  I'd be mostly concerned with people that are coding
> userspace applications using every whiz-bang kernel feature.  Or not
> paying attention at all to the kernel after the initial file creation
> and the options going stale (don't follow renames, etc).

it would be determined by the distro maintainers who maintain the kernel 
config for that distro.

David Lang

Re: [RFC] Simplifying kernel configuration for distro issues

[Ben Hutchings]

On Thu, Jul 19, 2012 at 06:30:47PM +0100, Alan Cox wrote:
> 
> > Well, yes.  I was thinking it would be more like:
> > 
> > distro/Kconfig.fedora
> > 	menuconfig FEDORA
> > 	if FEDORA
> > 	config FEDORA_16
> > 	   select WHATEVER
> > 	config FEDORA_17
> 
> Nope you need
> 
> distro/everyarchtheyship/everykernelvarianttkeyship(smp,largemem,arm
> boards)/Kconfig
> 
> which for some distros is over 20 per release and the end user wouldn't
> have a cat in hells chance of knowing which to pick.

20?  Debian's kernel package currently lists 17 architectures (11
source architectures) and 44 variants (excluding PREEMPT_RT and s390
install tape).  (But 'only' 31 will be in the next release, as even
Debian is capable of letting go of an architecture.)

But, assuming a native build (a whole weekend's worth of fun on m68k!)
we already know the target architecture and most of the other
variations can be chosen automatically similarly to localmodconfig.
We already do something like that (choosing between the pre-built
variants) at distribution install time, after all.

> For the end user case you need the distro to plonk the right file in the
> right place and be done with it, once they do that the rest is
> bikeshedding a ten line Makefile rule.
 
This might work well for future releases; is there not a need to
make this work for past releases too?

Ben.

-- 
Ben Hutchings
We get into the habit of living before acquiring the habit of thinking.
                                                              - Albert Camus

Re: [RFC] Simplifying kernel configuration for distro issues

[Geert Uytterhoeven]

On Thu, Jul 19, 2012 at 8:22 PM, Josh Boyer <jwboyer@redhat.com> wrote:
> On Thu, Jul 19, 2012 at 08:20:36PM +0200, Paul Bolle wrote:
>> On Thu, 2012-07-19 at 13:19 -0400, Josh Boyer wrote:
>> > kconfig already spits out warnings for symbols being selected that
>> > don't exist.
>>
>> Does it? Since when does it do that? Or do you mean select in a more
>> general way (not just meaning Kconfig's "select" statement)?
>
> I believe Alan was more correct than me when he said it was 'make
> oldconfig' that produced the warnings.

Indeed, no warnings for all these remaining "select MISC_DEVICES"
(patch sent to remove these).

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [RFC] Simplifying kernel configuration for distro issues

[Josh Boyer]

On Thu, Jul 19, 2012 at 02:04:11PM -0700, david@lang.hm wrote:
> On Thu, 19 Jul 2012, Josh Boyer wrote:
> 
> >On Thu, Jul 19, 2012 at 02:13:40PM -0400, Steven Rostedt wrote:
> >>On Thu, 2012-07-19 at 13:56 -0400, Josh Boyer wrote:
> >>
> >>>Distros aren't stationary things.
> >>
> >>Exactly my point.
> >>
> >>>  I mean, some of them certainly aim
> >>>for that goal, but userspace and kernels get upgraded all the time.  So
> >>>if this distro-Kconfig file is provided by some package _other_ than the
> >>>kernel the distros are going to have a bit of a hassle keeping track of
> >>>it.
> >>
> >>How about a directory called /usr/share/Linux/Kconfig.d/
> >>
> >>Then have anything installed that needs to work correctly put in its
> >>minimum (must have) requirement configs of the kernel.
> >>
> >>Say you are running Debian, and decide to try out systemd. If you set up
> >>your system to run that it would add a file called:
> >>
> >>/usr/share/Linux/Kconfig.d/systemd.conf
> >>
> >>or something, and this would select things like CGROUPS and the like. We
> >>could make the kernel build select all, or individual files in this
> >>directory. All for the 'make my distro work' or individual for a 'I want
> >>part of my distro to work' option.
> >
> >That sounds like a pretty good idea, aside from the fact that now your
> >config is determined by 1) what is currently installed on your system
> >and 2) people that don't maintain the kernel.
> >
> >1 is obviously a great thing once you have a stable working set of
> >packages you use daily, but wouldn't it kind of suck to have to rebuild
> >the kernel just to install some new package?  I mean... say you wanted
> >to now use an NFS mount, but you didn't have nfs-utils previously
> >installed.  So you install it, and it plops the kconfig file in
> >/usr/share but oops, you have to rebuild the kernel and reboot because
> >that module isn't built.  Of course I'm extrapolating possibly the worst
> >usage case here, but it will still happen.
> 
> the alturnative to this is what? compile everything just in case you
> need it some time in the future?

Why do people swing from one extreme to another so quickly?  Surely
there is some middle ground.

> we already have some tools (vmware) that check for the proper kernel
> config when they startup, and if the appropriate stuff isn't there
> they ask for the root password and compile the modules.
> 
> >2... yeah.  I don't really know if that is going to pan out, but I am
> >ever hopeful.  I'd be mostly concerned with people that are coding
> >userspace applications using every whiz-bang kernel feature.  Or not
> >paying attention at all to the kernel after the initial file creation
> >and the options going stale (don't follow renames, etc).
> 
> it would be determined by the distro maintainers who maintain the
> kernel config for that distro.

Erm... not in Steven's scheme.  At least I don't think distro kernel
maintainers are going to willingly crawl through every application
package that might depend on a kernel feature being enabled and maintain
those files across X number of packages.

josh

Re: [RFC] Simplifying kernel configuration for distro issues

[Steven Rostedt]

On Thu, 2012-07-19 at 18:35 -0400, Josh Boyer wrote:

> > >2... yeah.  I don't really know if that is going to pan out, but I am
> > >ever hopeful.  I'd be mostly concerned with people that are coding
> > >userspace applications using every whiz-bang kernel feature.  Or not
> > >paying attention at all to the kernel after the initial file creation
> > >and the options going stale (don't follow renames, etc).
> > 
> > it would be determined by the distro maintainers who maintain the
> > kernel config for that distro.
> 
> Erm... not in Steven's scheme.  At least I don't think distro kernel
> maintainers are going to willingly crawl through every application
> package that might depend on a kernel feature being enabled and maintain
> those files across X number of packages.

Correct. If we keep the selects in the kernel proper, then it would be
the kernel maintainer to make sure it works for the necessary
applications.

If we have a directory called /usr/share/Linux/Kconfig.d/ then the
individual packages could add their needed selects. Now this wouldn't be
for the average package. We don't need emacs developers adding any
configs here. It's just for those packages that are already tightly
coupled with the kernel (systemd, iptables, SELinux tools, etc).

-- Steve

Re: [RFC] Simplifying kernel configuration for distro issues

[david]

On Thu, 19 Jul 2012, Ben Hutchings wrote:

> On Thu, Jul 19, 2012 at 06:30:47PM +0100, Alan Cox wrote:
>>
>> For the end user case you need the distro to plonk the right file in the
>> right place and be done with it, once they do that the rest is
>> bikeshedding a ten line Makefile rule.
>
> This might work well for future releases; is there not a need to
> make this work for past releases too?

This approach can work for any 3.x kernel version with any distro. The 
distro provides the file, with a new kernel version you do "make 
distconfig', with something prior to when this is added you do 'cp 
/etc/kconfig/filename .config ; make oldconfig' instead.

the make oldconfig papers over a LOT of differences between the kernel 
that the distro built with and the kernel the user is trying to compile.

David Lang

Re: [RFC] Simplifying kernel configuration for distro issues

[Jiri Kosina]

On Fri, 13 Jul 2012, Linus Torvalds wrote:

> So this has long been one of my pet configuration peeves: as a user I
> am perfectly happy answering the questions about what kinds of
> hardware I want the kernel to support (I kind of know that), but many
> of the "support infrastructure" questions are very opaque, and I have
> no idea which of the them any particular distribution actually depends
> on.
> 
> And it tends to change over time. For example, F14 (iirc) started
> using TMPFS and TMPFS_POSIX_ACL/XATTR for /dev. And starting in F16,
> the initrd setup requires DEVTMPFS and DEVTMPFS_MOUNT. There's been
> several times when I started with my old minimal config, and the
> resulting kernel would boot, but something wouldn't quite work right,
> and it can be very subtle indeed.
> 
> Similarly, the distro ends up having very particular requirements for
> exactly *which* security models it uses and needs, and they tend to
> change over time. And now with systemd, CGROUPS suddenly aren't just
> esoteric things that no normal person would want to use, but are used
> for basic infrastructure. And I remember being surprised by OpenSUSE
> suddenly needing the RAW table support for netfilter, because it had a
> NOTRACK rule or something.
> 
> The point I'm slowly getting to is that I would actually love to have
> *distro* Kconfig-files, where the distribution would be able to say
> "These are the minimums I *require* to work". So we'd have a "Distro"
> submenu, where you could pick the distro(s) you use, and then pick
> which release, and we'd have something like

I agree that this would be very nice to have exactly for the reasons you 
have pointed out.

[ ... snip ... ]
> and then depending on the DISTRO config, we'd include one of the
> distro-specific ones with lists of supported distro versions and then
> the random config settings for that version:
> 
>  - distro/Kconfig.suse:
> 
>     config OPENSUSE_121
>         select OPENSUSE_11
>         select IP_NF_RAW  # ..
> 
>  - distro/Kconfig.Fedora:
> 
>     config FEDORA_16
>         select FEDORA_15
>         select DEVTMPFS               # F16 initrd needs this
>         select DEVTMPFS_MOUNT  # .. and expects the kernel to mount
> DEVTMPFS automatically
>         ...
> 
>     config FEDORA_17
>         select FEDORA_16
>         select CGROUP_xyzzy
>         ...
> 
> and the point would be that it would make it much easier for a normal
> user (and quite frankly, I want to put myself in that group too) to
> make a kernel config that "just works".

But we'll first have to make 'select' to actually work, right? It 
currently doesn't resolve the dependencies of the selected configs, so it 
will just produce some very broken config.

-- 
Jiri Kosina
SUSE Labs

Re: [RFC] Simplifying kernel configuration for distro issues

[Sam Ravnborg]

> 
> But we'll first have to make 'select' to actually work, right? It 
> currently doesn't resolve the dependencies of the selected configs, so it 
> will just produce some very broken config.

We could restrict "select" to only select symbols with no dependencies,
or *exactly* the same dependencies as the symbol containing the select.

So we could have something like:

config FEDORA
	select WANT_TMPFS


config WANT_TMPFS
	bool

config TMPFS
	defbool y if WANT_TMPFS


This is a pattern used today in many places.
But fixing up all the current select XXX would not be trivial...
I have no idea how many of the select we have today that would fail
the above semantic restrictions - but I guess it is a lot.

We could also come up with something new like:

config FEDORA
	require TMPFS
	require EXT4 = m
	require EXT3 = y

This would set TMPS to y if TMPFS dependencies are met.
And EXT3 to y and EXT4 to m if their dependencies are met.

This should be more or less the same as setting the value to y/m
in the user interface, which is only possible if the value is visible.

	Sam

Re: [RFC] Simplifying kernel configuration for distro issues

[valdis.kletnieks]

[-- Attachment #1: Type: text/plain, Size: 601 bytes --]

On Thu, 19 Jul 2012 13:56:50 -0400, Josh Boyer said:
> On Thu, Jul 19, 2012 at 01:33:42PM -0400, Steven Rostedt wrote:
> > What happens if someone does a yum update, and the kernel requirement
> > changes slightly. The yum update should update
> > this /usr/share/Linux/Kconfig. But it's still set at Fedora X. The
> > kernel can not be updated for these slight changes.
>
> I'm not quite following what you mean in the yum update case, sorry.

I do a 'yum update', and my system won't boot, and the first thing I think of
is "What the fsck did Lennart add as a systemd requirement *this* time?"

:)


[-- Attachment #2: Type: application/pgp-signature, Size: 865 bytes --]

Re: [opensuse-kernel] Re: [RFC] Simplifying kernel configuration for distro issues

[Benjamin Beurdouche]

Dear all,
I usually don't take part of discussions in LKML but I recognized myself
in what Linus is actually describing as a "normal user" (one of the few
insane enough to read the mailing list...) so I wrote a few ideas below...
Please give me your thoughts Š Benjamin

On 7/16/12 6:50 PM, "Linus Torvalds" <torvalds@linux-foundation.org> wrote:
>Your "problem" isn't what any sane person cares about, and isn't what
>I started the RFC for.
>Seriously. NOBODY CARES.
>You can do what you want to do *today*. Just edit the config file, or
>use any of the millions of config tools. Stop whining.

I guess the complexity of configuration in order to build a distro is why
I didn't even try to build a distro-specific kernelŠ Moreover the doc is
terrible for unfamiliar users...

>The thing I'm asking for is for normal people. Make it easy for people
>who DO NOT CARE about the config file to just build a kernel for their
>machine.

If you think about users like me who just want to build a basic featured
kernel we always do a make menuconfig with near default options...


What I would expect to build a distro/kernel is to have access in a
menuconfig-like to 
1) basic features like networking, usb storage, power management ŠetcŠ
_enabled by default_  so I can quickly build a "common platform" kernel
which gives me everything I need on my laptopŠ and
2) have a menu like "*distro* basic requirements" (never changes and only
for a few big distros) and finally
3) have a "*distro* advanced" menu having all the other non-mandatory
options available but with a conflict managerŠ (it could check against a
full Kconfig file kept up-to-date by each distro groups)

Without necessary talking of menuconfig, I liked very much the structure
Linus suggested in the first post regarding Kconfig filesŠ

That maybe some idiotic suggestions but I'd like to know what you think
about it anywayŠ

Cheers! B.

>Don't complicate the issue by bringing up some totally unrelated
>question. Don't derail a useful feature for the 99% because you're not
>in it.
>
>                    Linus