[ANNOUNCE] Framework for automatic Configuration of a Kernel

[ANNOUNCE] Framework for automatic Configuration of a Kernel

[Ahmad Reza Cheraghi]

Hi folks,

For my EndThesis, in the Niederrhein University of
Applied Sciences, I've almost finished a framework
that generates a .config file based on the target
system.This program should help people to generate a
linux kernel Config without spending a lot of time at
the configuration.

The basic idea of the framework is, that you can
specify in the Kconfig files a script which
auto-detect if the hardware involved in this option is
present or not (the script reply 'y' or 'n'). It's up
to the interface to choose what to do with the answer.

This framework is now in its test stage. It works on
my Acer Laptop(TM291LMI with Pentium M and Radeon
9700). That means, that the framework is functional
but additional scripts have to be written for other
type
of hardware/functionality (far away from completed).

The code of this framework has been in a Kernel-Patch.
Since the Patch has a huge size(230kb), you can get
the Patch from the given URL:

http://www.energyparty.de/ahmad/autoconfig_0_4.patch

To try it, just patch a 2.6.13 kernel with the patches
following this
mail and set the scripts as follow: chmod u+rwx
scripts/kconfig/rules/*
Then, just type: 'make autoconfig' or 'make
autochoiceconfig'.

The 'autoconfig' will detect your hardware and
automatically include in the kernel(as a biuld-in)
whatever hardware is detected. The 'autochoiceconfig'
will let you choose to include the feature or not when
a hardware is detected.

Any comments and suggestions are welcome !


Regards
Ahmad Reza Cheraghi



		
__________________________________ 
Yahoo! Mail - PC Magazine Editors' Choice 2005 
http://mail.yahoo.com

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Emmanuel Fleury]

Hi,

First of all, I guess that the point of a script here is to decide
whether or not the hardware device is here or not. So, the output should
be something like "true" or "false", better than echoing some stupid
characters, maybe a direct "exit 0" and "exit 1" would be much less
troublesome.


Then, I might be wrong but I think we can get a more stable detection of
the PCI devices by grabbing directly the PCI vendor and device codes as
numbers instead of looking them up in the PCI ID database.

lspci -n

Or even, ask for a specific device like this:
lspci -d [<vendor>]:[<device>]

Which would give something like this:

[fleury@rade7 ~]$ lspci -d 8086:1a30
0000:00:01.0 PCI bridge: Intel Corporation 82845 845 (Brookdale) Chipset
AGP Bridge (rev 04)

Or if the device is not present:
[fleury@rade7 ~]$ lspci -d 8087:1a30
<no output>

This way just avoid to depend from the way the PCI database is written,
because whenever there is a change in the spelling of one keyword, it
might need quite a lot of updates in the Kconfigs. On the contrary, the
vendor and device PCI code will never change (hopefully).

So, the 'autorule' would look like this:

autorule pciscript.sh "8086:1a30"

And the script would be:
#/bin/sh

if [ -z "`lspci -d $1`" ]
then
    exit 1
else
    exit 0
fi

What do you think ?

Regards
-- 
Emmanuel Fleury

Unix is user-friendly. It's just more specific about his friends.
  -- Unknown

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Ahmad Reza Cheraghi]

Hi Emmanuel

--- Emmanuel Fleury <fleury@cs.aau.dk> wrote:

> Hi,
> 
> First of all, I guess that the point of a script
> here is to decide
> whether or not the hardware device is here or not.
> So, the output should
> be something like "true" or "false", better than
> echoing some stupid
> characters, maybe a direct "exit 0" and "exit 1"
> would be much less
> troublesome.

Good idea! 

> 
> Then, I might be wrong but I think we can get a more
> stable detection of
> the PCI devices by grabbing directly the PCI vendor
> and device codes as
> numbers instead of looking them up in the PCI ID
> database.
> 
> lspci -n
> 
> Or even, ask for a specific device like this:
> lspci -d [<vendor>]:[<device>]
> 
> Which would give something like this:
> 
> [fleury@rade7 ~]$ lspci -d 8086:1a30
> 0000:00:01.0 PCI bridge: Intel Corporation 82845 845
> (Brookdale) Chipset
> AGP Bridge (rev 04)
> 
> Or if the device is not present:
> [fleury@rade7 ~]$ lspci -d 8087:1a30
> <no output>
> 
> This way just avoid to depend from the way the PCI
> database is written,
> because whenever there is a change in the spelling
> of one keyword, it
> might need quite a lot of updates in the Kconfigs.
> On the contrary, the
> vendor and device PCI code will never change
> (hopefully).
> So, the 'autorule' would look like this:
> 
> autorule pciscript.sh "8086:1a30"
> 
> And the script would be:
> #/bin/sh
> 
> if [ -z "`lspci -d $1`" ]
> then
>     exit 1
> else
>     exit 0
> fi
> 
> What do you think ?

Again another good Idea. Your right;-) Its better. 
But it better getting another way of detecting the
Hardware/Software etc. from the System without using
lspci or the proc-files...? Something that gets all
the Hardware Information directly from the I/O and not
from the Kernel. The good thing about lspci is that it
does both . But it doesnt say if there is  a CDROM or
floppy-disc... I tryed alot to search for something
like that but without any success. I heard about this
Otopia Project. I google after it but I didnt find
anything usefule. I think its dead. 

Regards

Ahmad Reza Cheraghi


		
__________________________________ 
Yahoo! Mail - PC Magazine Editors' Choice 2005 
http://mail.yahoo.com

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Emmanuel Fleury]

Hi Ahmad,

Ahmad Reza Cheraghi wrote:
> 
> Again another good Idea. Your right;-) Its better. 
> But it better getting another way of detecting the
> Hardware/Software etc. from the System without using
> lspci or the proc-files...?

Well, the proc files are always here (it removes one requisite which is
to have lspci installed). So, I would go for the proc files.

> Something that gets all
> the Hardware Information directly from the I/O and not
> from the Kernel. The good thing about lspci is that it
> does both . But it doesnt say if there is  a CDROM or
> floppy-disc... 

Well, lspci is for PCI bus devices, it's already a lot, but not
everything (that's why you need several scripts/methods to detect
hardware, I guess).

> I tryed alot to search for something
> like that but without any success. I heard about this
> Otopia Project. I google after it but I didnt find
> anything usefule. I think its dead. 

I don't know this project.

Regards
-- 
Emmanuel Fleury

echo '16i[q]sa[ln0=aln100%Pln100/snlbx]sb20293A2058554E494Csnlbxq'|dc
  -- Unknown

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Ahmad Reza Cheraghi]

> Well, the proc files are always here (it removes one
> requisite which is
> to have lspci installed). So, I would go for the
> proc files.

Proc files are not always on the system. As far as I
know you have to chose in the Kernl-Configuration.
And second of all I had some problem with a  debian
distrubotion. I dont know for what reason there was no
USB given in the /proc/... Files. But lspci did found 
it. And the Kernel was correctly configurated. I didnt
solved that problem. i just installed Kubuntu and
everybody was happy. For that reason I dont trust
/proc files anymore.

> Well, lspci is for PCI bus devices, it's already a
> lot, but not
> everything (that's why you need several
> scripts/methods to detect
> hardware, I guess).

Lspci do find other Hardware as well. Like USB,
Ethernet, ISA, Firewire...

Regards
Ahmad Reza Cheraghi

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Emmanuel Fleury]

Ahmad Reza Cheraghi wrote:
> 
> Again another good Idea. Your right;-) Its better. 
> But it better getting another way of detecting the
> Hardware/Software etc. from the System without using
> lspci or the proc-files...? Something that gets all
> the Hardware Information directly from the I/O and not
> from the Kernel. The good thing about lspci is that it
> does both . But it doesnt say if there is  a CDROM or
> floppy-disc... I tryed alot to search for something
> like that but without any success. I heard about this
> Otopia Project. I google after it but I didnt find
> anything usefule. I think its dead. 

I might be wrong, but I don't think that there is any other way to get
hardware information but through the /proc or /sys interface.

Can somebody comment on this ?

Regards
-- 
Emmanuel Fleury

It really is a nice theory. The only defect I think it has is probably
common to all philosophical theories. It's wrong.
  -- Saul Kripke (Naming and Necessity)

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Andreas Jellinghaus]

Emmanuel Fleury wrote:
> I might be wrong, but I don't think that there is any other way to get
> hardware information but through the /proc or /sys interface.
> 
> Can somebody comment on this ?

dmidecode will give you some hardware information on some systems.

for example my dell latitude c600 laptop the bios claims to support:
                        PCI is supported
                        PC Card (PCMCIA) is supported
                        PNP is supported
                        APM is supported
                        BIOS is upgradeable
                        BIOS shadowing is allowed
                        Boot from CD is supported
                        Selectable boot is supported
                        Boot from PC Card (PCMCIA) is supported
                        3.5"/720 KB floppy services are supported (int 13h)
                        Print screen service is supported (int 5h)
                        8042 keyboard services are supported (int 9h)
                        Serial services are supported (int 14h)
                        Printer services are supported (int 17h)
                        CGA/mono video services are supported (int 10h)
                        ACPI is supported
                        USB legacy is supported
                        AGP is supported
                        LS-120 boot is supported
                        ATAPI Zip drive boot is supported
                        Smart battery is supported
                        BIOS boot specification is supported

and the chipsets might include support for hardware that
is left dead. so it is nice to see which connectors the
mainboard has. dmidecode for example tells me:
Handle 0x0800
        DMI type 8, 9 bytes.
        Port Connector Information
                Internal Reference Designator: PARALLEL
                Internal Connector Type: None
                External Reference Designator: Not Specified
                External Connector Type: DB-25 female
                Port Type: Parallel Port PS/2
Handle 0x0801
        DMI type 8, 9 bytes.
        Port Connector Information
                Internal Reference Designator: SERIAL1
                Internal Connector Type: None
                External Reference Designator: Not Specified
                External Connector Type: DB-9 male
                Port Type: Serial Port 16550A Compatible

i.e. I have a serial and parallel connector. even more interesting
is:
Handle 0x0809
        DMI type 8, 9 bytes.
        Port Connector Information
                Internal Reference Designator: IrDA
                Internal Connector Type: None
                External Reference Designator: Not Specified
                External Connector Type: Infrared
                Port Type: Other

because irda hides behind a fake serial port, if I understand
things right, so now you know the first serial port is real,
the second not, but an irda port is missing.

also dmidecode has details on the cpu.

I hope that information helps? good luck!

Regards, Andreas

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Emmanuel Fleury]

Hi,

This is great, I didn't know this DMI table.

Is there some overlap with the lspci detection ?
And if yes, which one has to be taken into account ?

Regards
-- 
Emmanuel Fleury

Hardware, n.: The parts of a computer system that can be kicked.
  -- Ray Simard

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Emmanuel Fleury]

Emmanuel Fleury wrote:
> Hi,
> 
> This is great, I didn't know this DMI table.
> 
> Is there some overlap with the lspci detection ?
> And if yes, which one has to be taken into account ?

Ok, I read the documentation, this question was stupid... sorry. :)

Regards
-- 
Emmanuel Fleury

Providence protects children and idiots.
I know because I have tested it.
  -- Mark Twain

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Ahmad Reza Cheraghi]

I install it(http://www.nongnu.org/dmidecode/).
It just tell you all the hardware on the Systems but
unfortunatly it doesn't mention the Manufactor. But
still it is a great help for the Hardware-detection.
Thank
you Andreas. 

> 
> This is great, I didn't know this DMI table.
> 
> Is there some overlap with the lspci detection ?
> And if yes, which one has to be taken into account ?
 
I didn't recognise any overlaps.


	
		
______________________________________________________ 
Yahoo! for Good 
Donate to the Hurricane Katrina relief effort. 
http://store.yahoo.com/redcross-donate3/ 

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Andreas Jellinghaus]

as far as I know it is only informative - and most
likely not used by kernel source probing.
the values in it might be as well wrong.

but I'm only guessing, I'm a user, not a kernel
developer.

Andreas

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Randy.Dunlap]

On Tue, 27 Sep 2005 17:51:06 +0200 Emmanuel Fleury wrote:

> Ahmad Reza Cheraghi wrote:
> > 
> > Again another good Idea. Your right;-) Its better. 
> > But it better getting another way of detecting the
> > Hardware/Software etc. from the System without using
> > lspci or the proc-files...? Something that gets all
> > the Hardware Information directly from the I/O and not
> > from the Kernel. The good thing about lspci is that it
> > does both . But it doesnt say if there is  a CDROM or
> > floppy-disc... I tryed alot to search for something
> > like that but without any success. I heard about this
> > Otopia Project. I google after it but I didnt find
> > anything usefule. I think its dead. 
> 
> I might be wrong, but I don't think that there is any other way to get
> hardware information but through the /proc or /sys interface.
> 
> Can somebody comment on this ?

You can use libpciutils (whatever lspci uses) to enumerate
PCI devices  (but I wouldn't, I'd just use lspci :).

You can clone lsusb to enumerate USB devices... but I would
just use lsusb.

You can probably clone 'pnpdump' to enumerate ISA PNP devices.
But I would just use pnpdump.

There's probably a ieee1394/firewire enumeration program.

You won't be able (safely) to discover ISA devices/ports.
(other than "standard" cnipset/motherboard devices)

Anyway, I would just use the existing tools unless there are
things that you must have that they don't provide.

---
~Randy
You can't do anything without having to do something else first.
-- Belefant's Law

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Jody McIntyre]

On Thu, Sep 29, 2005 at 08:09:38PM -0700, Randy.Dunlap wrote:

> There's probably a ieee1394/firewire enumeration program.

Not that I know of, other than gscanbus (unsuitable because it's a GUI
app.)  I'm slowly working on one but it's nowhere close to releasable
and really not a high priority...

Jody

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Emmanuel Fleury]

Hi,

Actually this autoconfig thing raise quite a lot of questions to me.

1) What should be detected, what should not ?

It seems obvious that all the choices beyond hardware are choices that
the user should do (what network protocols, what kind of filesystems,
etc). But, as Boddo Eggert pointed out (if I understood well), all the
hardware which is lying outside of the box (printer, scanner, usb key,
...) could be detected if plugged when the autoconfig is ran.

Should we trust these informations (as theses devices can be removed at
any time) ? Shouldn't we let the user make these choices by himself ?
What is the limit of the devices we can assume to be part of the machine
when building this autoconfig ?

2) What is the surest and the most stable way to detect the hardware ?

lspci, lsusb, dmidecode (or /proc and /sys) have been mentioned. This
brings two sub-questions:
- What kind of software environment / kernel release can we assume ?
- How to minimize the dependencies of the detection from other tools ?
 (changing all the Kconfigs just because the syntax of the output of one
of these command has changed would be quite painful).


3) Can this hardware detection be included in other interfaces ?

It would be nice to have this detection to be ran when no pre existing
.config is detected in all other interfaces (menuconfig, config, ...).


Regards
-- 
Emmanuel Fleury

Discontent is the first step in the progress of a man or a nation.
  -- Oscar Wilde

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Ahmad Reza Cheraghi]

> 1) What should be detected, what should not ?

I think its good to detect everything, that let a
Kernel work after the installation. I mean it the
autoconfiguration should't be only for Kernel-Hackers.
Maybe its good idea to make two type of detection

1. which detects only the HW(For Kernel-Hackers)
2. which detects all the HW and configure everything
that let the Kernel work.(for beginners)
 
> But, as Boddo Eggert pointed out (if I
> understood well), all the
> hardware which is lying outside of the box (printer,
> scanner, usb key,
> ...) could be detected if plugged when the
> autoconfig is ran. 
> Should we trust these informations (as theses
> devices can be removed at
> any time) ? Shouldn't we let the user make these
> choices by himself ?
I think it should be included, since it is up to the
user has to plug the HW in to those Ports so the
autoconfig can be detected. 

> What is the limit of the devices we can assume to be
> part of the machine
> when building this autoconfig ?

All the needed HW that the user wants or need must at
least be detected and configured.

> 2) What is the surest and the most stable way to
> detect the hardware ?
> 
> lspci, lsusb, dmidecode (or /proc and /sys) have
> been mentioned. This
> brings two sub-questions:
> - What kind of software environment / kernel release
> can we assume ?

The best was is to use HW-Detection-Tools that are in
naked Kernel. But I dont know if is lspci is in the
Kernel. I mean dmidecode is good for detecting all the
HW but it has to be installed first. And its no good
to let the user install first of all some tools so the
autoconfigure can work.
How do we get the HW detected from a naked Kernel
without any Distrubotion or whatever.   

> - How to minimize the dependencies of the detection
> from other tools ?
>  (changing all the Kconfigs just because the syntax
> of the output of one
> of these command has changed would be quite
> painful).

If something like that hapend we can put a filter in
the script, that changes the Parameter in to a new
one. 



> 3) Can this hardware detection be included in other
> interfaces ?
> 
> It would be nice to have this detection to be ran
> when no pre existing
> .config is detected in all other interfaces
> (menuconfig, config, ...).

I dont understand your question. What you mean with
interfaces??
 

Regards

Ahmad Reza Cheraghi


		
__________________________________ 
Yahoo! Mail - PC Magazine Editors' Choice 2005 
http://mail.yahoo.com

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Emmanuel Fleury]

Ahmad Reza Cheraghi wrote:
> 
> I think its good to detect everything, that let a
> Kernel work after the installation. I mean it the
> autoconfiguration should't be only for Kernel-Hackers.
> Maybe its good idea to make two type of detection
> 
> 1. which detects only the HW(For Kernel-Hackers)
> 2. which detects all the HW and configure everything
> that let the Kernel work.(for beginners)

Why would a beginner compile a kernel ?

I would even say this would be bad if you can avoid them to go through
all the documentations of each option. :)

> The best was is to use HW-Detection-Tools that are in
> naked Kernel. But I dont know if is lspci is in the
> Kernel. I mean dmidecode is good for detecting all the
> HW but it has to be installed first. And its no good
> to let the user install first of all some tools so the
> autoconfigure can work.
> How do we get the HW detected from a naked Kernel
> without any Distrubotion or whatever.   

I doubt we can, at least not in the current status.

And requiring more external tools as lspci, dmidecode, etc, is probably
not right.

More I am thinking of it, more it seems that the hardware detection
softwares (lspci, dmidecode, lsusb) should be stripped down and probably
included somewhere in the kernel tree... But wouldn't be too much to pay
for having an autoconfig ? (I mean maintenance, update, debug, ... will
be way out of the scope of the kernel).

Did anybody had some though about this previously (I tried to look in
the archive of the LKM but didn't find anything relevant) ?

> I dont understand your question. What you mean with
> interfaces??

ncurses, Qt, Gtk, ... (aka menuconfig, xconfig, gconfig).

Regards
-- 
Emmanuel Fleury

My behaviour is addictive functioning in a disease process of
toxic co-dependency. I need holistic healing and wellness
before I'll accept any responsibility for my actions.
  -- Calvin & Hobbes (Bill Waterson)

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Ahmad Reza Cheraghi]

--- Emmanuel Fleury <fleury@cs.aau.dk> wrote:

> > I think its good to detect everything, that let a
> > Kernel work after the installation. I mean it the
> > autoconfiguration should't be only for
> Kernel-Hackers.
> > Maybe its good idea to make two type of detection
> > 
> > 1. which detects only the HW(For Kernel-Hackers)
> > 2. which detects all the HW and configure
> everything
> > that let the Kernel work.(for beginners)
> 
> Why would a beginner compile a kernel ?

Because he wants to be upto date
 
> I would even say this would be bad if you can avoid
> them to go through
> all the documentations of each option. :)

Why? 

> > The best was is to use HW-Detection-Tools that are
> in
> > naked Kernel. But I dont know if is lspci is in
> the
> > Kernel. I mean dmidecode is good for detecting all
> the
> > HW but it has to be installed first. And its no
> good
> > to let the user install first of all some tools so
> the
> > autoconfigure can work.
> > How do we get the HW detected from a naked Kernel
> > without any Distrubotion or whatever.   
> 
> I doubt we can, at least not in the current status.
> 
> And requiring more external tools as lspci,
> dmidecode, etc, is probably
> not right.
> 
> More I am thinking of it, more it seems that the
> hardware detection
> softwares (lspci, dmidecode, lsusb) should be
> stripped down and probably
> included somewhere in the kernel tree... But
> wouldn't be too much to pay
> for having an autoconfig ? (I mean maintenance,
> update, debug, ... will
> be way out of the scope of the kernel).

How about making a new project for calling all the
Hardware on the system directly from the I/O and let
then be a part of the Kernel? Its just a suggestion!!

> Did anybody had some though about this previously (I
> tried to look in
> the archive of the LKM but didn't find anything
> relevant) ?
> 
> > I dont understand your question. What you mean
> with
> > interfaces??
> 
> ncurses, Qt, Gtk, ... (aka menuconfig, xconfig,
> gconfig).

Why should it be in other interfaces?? The plan of the
autoconfig is just to configure the Kernel and nothing
more... What is the advantage of having it in other
interfaces?? 

Regards 
Ahmad Reza Cheraghi


		
__________________________________ 
Yahoo! Mail - PC Magazine Editors' Choice 2005 
http://mail.yahoo.com

Re: [ANNOUNCE] Framework for automatic Configuration of a Kernel

[Emmanuel Fleury]

Bodo Eggert wrote:
> 
> You can actually try to probe the legacy ports like LPT and COM if you've
> got the correct permissions. However, you SHOULD avoid any port except the
> default port ranges and ask the user for non-standard hardware and you
> SHOULD NOT touch devices being in use.
> 
> Each piece of hardware will be different, and you'll fell like having opened
> a large freight container of worms.

But you are speaking about printer/scanner/... or other external
devices, isn't it ?

Do you think they should be included in the automatic detection ? Or
would it be safer to let the user make the configuration by himself ?

As you pointed it out, it requires special rights and might actually be
dangerous. I'm not sure that this is worthing to try.

My view of this "autoconfig" was more to detect what is present in the
box, not what is lying outside...

Regards
-- 
Emmanuel Fleury

Security is a process, not a product.
  -- Bruce Schneier (Crypto-gram, February 15, 2002)