Kernel thoughts of a Linux user

Kernel thoughts of a Linux user

[Gerold J. Wucherpfennig]

- setting up kernel boot parameters with graphical tools is unreliable, 
because the system doesn't know which bootloader entry was chosen.
One solution to this issue is to create a new kernel parameter "loaderhint"
where the bootloader will be able to set the number of the chosen boot entry.
In the configuration file of the bootloader this will have to be explicitly 
remarked e.g. ... loaderhint=%selection% ... . Unfortunately this could be
circumvented, because it isn't mandatory and could be manipulated in the boot 
loader configuration. (Other/better suggestions are welcome)

-speed up the booting process and make the system more reliable and
secure by skipping to autoprobe all devices. Instead initialize only the
devices which were set up at the last system startup. The data can be given
by kernel parameters or there could be established a more convenient
mechanism. This data will be stored automatically when the system startup
is completed (by some shell script etc.) There will be a kernel mechanism to
tell userspace about available, but unconfigured, unprobed devices (sysfs?)
and a mechanism to set up each device separately. There has to be a fallback
mechanism which lets the kernel boot with full probing like it is done today,
of course. (Other/better suggestions are welcome).

- The above mentioned ideas will pave the way to some sort of device manager
which can enable/disable devices, initialize new devices and store all those
settings across system reboots as it has never been possible in the UNIX/Linux
world before

- Make sysfs optional and enable to publish kernel <-> userspace data
especially the kernel's KObject data across the kernel's netlink interface as
it has been summarized on www.kerneltrap.org. This will avoid the
deadlocks sysfs does introduce when some userspace app holds an open file
handle of an sysfs object (KObject) which is to be removed. An importrant side 
effect for embedded systems will be that the RAM overhead introduced by sysfs
will vaporize.

- Replace DRI with sth. slimmer and intoduce real kernel drivers
and introduce real kernel drivers which handel all the initialization and 
interrupt handling (only minimal hardware abstraction). One goal is to
remove X.org's PCI magic. Ultimately this shall give framebuffer and X
the same basis. This was summarized on kerneltrap.org.


I hope this thoughts will be helpful to someone and may lead to a discussion
which will lead to code which resolves some deficiencies of Linux systems
(compared to features of some other prominent operating system).

Re: Kernel thoughts of a Linux user

[Greg KH]

On Thu, Nov 18, 2004 at 06:59:27PM +0100, Gerold J. Wucherpfennig wrote:
> 
> - Make sysfs optional and enable to publish kernel <-> userspace data
> especially the kernel's KObject data across the kernel's netlink interface as
> it has been summarized on www.kerneltrap.org. This will avoid the
> deadlocks sysfs does introduce when some userspace app holds an open file
> handle of an sysfs object (KObject) which is to be removed. An importrant side 
> effect for embedded systems will be that the RAM overhead introduced by sysfs
> will vaporize.

What RAM overhead?  With 2.6.10-rc2 the memory footprint of sysfs has
been drasticly shrunk.

What deadlocks are you referring to?

And the netlink interface for hotplug events is already present in the
latest kernel.

thanks,

greg k-h

Re: Kernel thoughts of a Linux user

[Tomas Carnecky]

Gerold J. Wucherpfennig wrote:
 > - Replace DRI with sth. slimmer and intoduce real kernel drivers
> and introduce real kernel drivers which handel all the initialization and 
> interrupt handling (only minimal hardware abstraction). One goal is to
> remove X.org's PCI magic. Ultimately this shall give framebuffer and X
> the same basis. This was summarized on kerneltrap.org.

Is it possible to have two or more 'workstations' on one computer?
A 'workstation' is a monitor, keyboard, mouse etc. tied together and
represents a place where someone can work.
I know it's possible to do this using a Xserver (running two Xservers on
different virtual consoles, each with its own
configuration/keyboard/mouse/monitor), but I'd like to realise it more
low-level, on the level of virtual terminals, so that each 'workstation'
would have it's own 'Ctrl+F1', 'Ctrl+F2' etc.

Background:
Today, you can buy video cards with two connectors for monitors, or even
put two of those cards into one mainboard, making it possible to connect
four monitors to one computer. A P4 HT enabled CPU would be enough for
four office workers who edit text documents, unless they aren't playing
games :) So you could cut costs by buying one set of Mainboard/CPU/RAM
and then for every worker just a monitor/keyboard/mouse.
Places like internet-cafes could profit, they usually have many same
computers side by side, each with the same configuration, but on many no
one is working, they just run and consume energy.

tom

Re: Kernel thoughts of a Linux user

[Jan Engelhardt]

>Places like internet-cafes could profit, they usually have many same
>computers side by side, each with the same configuration, but on many no
>one is working, they just run and consume energy.

So they could make themselves a favor and run something like seti@home.



Jan Engelhardt
-- 
Gesellschaft für Wissenschaftliche Datenverarbeitung
Am Fassberg, 37077 Göttingen, www.gwdg.de

Re: Kernel thoughts of a Linux user

[Tomas Carnecky]

Jan Engelhardt wrote:
>>Places like internet-cafes could profit, they usually have many same
>>computers side by side, each with the same configuration, but on many no
>>one is working, they just run and consume energy.
> 
> 
> So they could make themselves a favor and run something like seti@home.
> 

I didn't want to point to the unused CPU cycles, but to the wasted money 
spent to buy the hardware.

tom

Re: Kernel thoughts of a Linux user

[Jan Engelhardt]

>> So they could make themselves a favor and run something like seti@home.
>
>I didn't want to point to the unused CPU cycles, but to the wasted money
>spent to buy the hardware.

Well, they certainly won't trash the hardware they have ATM...
might run <your favorite distributed client> on 4 machines rather than 1 ;-)


Jan Engelhardt
-- 
Gesellschaft für Wissenschaftliche Datenverarbeitung
Am Fassberg, 37077 Göttingen, www.gwdg.de

Re: Kernel thoughts of a Linux user

[Chris Friesen]

Jan Engelhardt wrote:

> So they could make themselves a favor and run something like seti@home.

That does consume more energy than just sitting at idle.  I've seen some 
estimates of how much it costs to run seti 24/7 rather than just sit idle, and 
the price was something like $80/year.

Chris

Re: Kernel thoughts of a Linux user

[Jan Engelhardt]

>> So they could make themselves a favor and run something like seti@home.
>
>That does consume more energy than just sitting at idle.  I've seen some
>estimates of how much it costs to run seti 24/7 rather than just sit idle, and
>the price was something like $80/year.

For CPUs which don't have some sort of speedstep, it does not matter.
(Please correct me if I am wrong. It might be that HLT cycles are still more
power-conservative even without speedstep than 24/7 on the FPU.)


Jan Engelhardt
-- 
Gesellschaft für Wissenschaftliche Datenverarbeitung
Am Fassberg, 37077 Göttingen, www.gwdg.de

Re: Kernel thoughts of a Linux user

[Chris Friesen]

Jan Engelhardt wrote:

> For CPUs which don't have some sort of speedstep, it does not matter.
> (Please correct me if I am wrong. It might be that HLT cycles are still more
> power-conservative even without speedstep than 24/7 on the FPU.)

Actually, it does matter.  Transistors that aren't being used don't consume 
power.  So if you just sit spinning on hlt, you don't use a lot of the chip, and 
you use less power.  This is why overclockers will test their cooling solutions 
while running the cpu at full throttle with various stress tests.

Aside from the cpu itself, when idling you also don't consume memory bus cycles, 
you never hit the network, you don't use the disk, etc., etc.  It all adds up.

Chris

Re: Kernel thoughts of a Linux user

[Paulo Marques]

Jan Engelhardt wrote:
>>>So they could make themselves a favor and run something like seti@home.
>>
>>That does consume more energy than just sitting at idle.  I've seen some
>>estimates of how much it costs to run seti 24/7 rather than just sit idle, and
>>the price was something like $80/year.
> 
> 
> For CPUs which don't have some sort of speedstep, it does not matter.
> (Please correct me if I am wrong. It might be that HLT cycles are still more
> power-conservative even without speedstep than 24/7 on the FPU.)

You're wrong :)

Nowadays the power consumption of a CPU is more than the rest of the 
machine altogether (including hard disks, etc.).

On my P4 2.8GHz HT CPU, I've measured the power consumed by *the entire 
computer* more than doubling as the processor went from idle into 100% 
load.

Of course, this doesn't include a monster 3D card, is it could very well 
consume something close to the processor when doing a lot of 3D operations.

-- 
Paulo Marques - www.grupopie.com

All that is necessary for the triumph of evil is that good men do nothing.
Edmund Burke (1729 - 1797)

Re: Kernel thoughts of a Linux user

[Jan Engelhardt]

>You're wrong :)
>
>Nowadays the power consumption of a CPU is more than the rest of the
>machine altogether (including hard disks, etc.).

I'd counter that: Sun E250 Enterprise servers sound like the CPUs take the
fewest power of all components together :)



Jan Engelhardt
-- 
Gesellschaft für Wissenschaftliche Datenverarbeitung
Am Fassberg, 37077 Göttingen, www.gwdg.de

Re: Kernel thoughts of a Linux user

[Alan Cox]

On Gwe, 2004-11-19 at 12:02, Paulo Marques wrote:
> Nowadays the power consumption of a CPU is more than the rest of the 
> machine altogether (including hard disks, etc.).

This depends hugely on the hardware. A 1Ghz Geode NX (Athlon 1Ghz ish)
is 
under 10W. The Pentium-M is also stunningly more efficient than the last
Intel attempt (Pentium IV) and performs remarkly for its power level.

Re: Kernel thoughts of a Linux user

[Helge Hafting]

Tomas Carnecky wrote:

> Gerold J. Wucherpfennig wrote:
> > - Replace DRI with sth. slimmer and intoduce real kernel drivers
>
>> and introduce real kernel drivers which handel all the initialization 
>> and interrupt handling (only minimal hardware abstraction). One goal 
>> is to
>> remove X.org's PCI magic. Ultimately this shall give framebuffer and X
>> the same basis. This was summarized on kerneltrap.org.
>
>
> Is it possible to have two or more 'workstations' on one computer?

Yes - thats what the "ruby" kernel patch is all about.  I have a computer
with two "workstations" at home.  Compared to two computers, it
saves space, power, parts,  and above all - administrative work.  Only one
machine to upgrade, secure, configure.

> A 'workstation' is a monitor, keyboard, mouse etc. tied together and
> represents a place where someone can work.
> I know it's possible to do this using a Xserver (running two Xservers on
> different virtual consoles, each with its own
> configuration/keyboard/mouse/monitor), but I'd like to realise it more
> low-level, on the level of virtual terminals, so that each 'workstation'
> would have it's own 'Ctrl+F1', 'Ctrl+F2' etc.

Sure - ruby gives you that.  X may need a patch in order to support
ctrl+F2... on the scond keyboard, as the second console uses vt numbers
from 17 to 32.

>
> Background:
> Today, you can buy video cards with two connectors for monitors, or even
> put two of those cards into one mainboard, making it possible to connect
> four monitors to one computer. A P4 HT enabled CPU would be enough for
> four office workers who edit text documents, unless they aren't playing
> games :) So you could cut costs by buying one set of Mainboard/CPU/RAM
> and then for every worker just a monitor/keyboard/mouse.
> Places like internet-cafes could profit, they usually have many same
> computers side by side, each with the same configuration, but on many no
> one is working, they just run and consume energy. 

Yes, you can do that.   The limit seems to be how many monitors you can
connect - there seems to be no practical limit to how many USB keyboards
& mice you can use.  The lengt of wires might also be a problem
with more than four.

Helge Hafting

Re: Kernel thoughts of a Linux user

[Jan Engelhardt]

>Yes, you can do that.   The limit seems to be how many monitors you can
>connect - there seems to be no practical limit to how many USB keyboards
>& mice you can use.  The lengt of wires might also be a problem
>with more than four.

Well wait until we get USB monitors.



Jan Engelhardt
-- 
Gesellschaft für Wissenschaftliche Datenverarbeitung
Am Fassberg, 37077 Göttingen, www.gwdg.de

Re: Kernel thoughts of a Linux user

[Helge Hafting]

Jan Engelhardt wrote:

>>Yes, you can do that.   The limit seems to be how many monitors you can
>>connect - there seems to be no practical limit to how many USB keyboards
>>& mice you can use.  The lengt of wires might also be a problem
>>with more than four.
>>    
>>
>
>Well wait until we get USB monitors.
>  
>
You can make a USB terminal if you wish, but I don't
think USB has enough bandwith for a serious monitor.
I think I heard of a VGA over USB product, but running
X or even 3D on that cannot be fun.  And VGA is too
low resolution these days.

Helge Hafting

Re: Kernel thoughts of a Linux user

[Tomas Carnecky]

Helge Hafting wrote:
>> Is it possible to have two or more 'workstations' on one computer?
> 
> Yes - thats what the "ruby" kernel patch is all about.  I have a computer
> with two "workstations" at home.  Compared to two computers, it
> saves space, power, parts,  and above all - administrative work.  Only one
> machine to upgrade, secure, configure.
> 

Thanks, that's what I've been looking for...

 From the documentation:
The main problem up to this date (November 2004) is that linux kernel 
has only one behaviour regarding multiple keyboards : any key pressed 
catched on any keyboard is redirected to the one and only active Virtual 
Terminal (VT).

Will this be changed/improved when the console code is moved into 
userspace, like some have proposed?

tom

Re: Kernel thoughts of a Linux user

[Helge Hafting]

Tomas Carnecky wrote:

> Helge Hafting wrote:
>
>>> Is it possible to have two or more 'workstations' on one computer?
>>
>>
>> Yes - thats what the "ruby" kernel patch is all about.  I have a 
>> computer
>> with two "workstations" at home.  Compared to two computers, it
>> saves space, power, parts,  and above all - administrative work.  
>> Only one
>> machine to upgrade, secure, configure.
>>
>
> Thanks, that's what I've been looking for...
>
> From the documentation:
> The main problem up to this date (November 2004) is that linux kernel 
> has only one behaviour regarding multiple keyboards : any key pressed 
> catched on any keyboard is redirected to the one and only active 
> Virtual Terminal (VT).
>
> Will this be changed/improved when the console code is moved into 
> userspace, like some have proposed?

I don't know about any userspace console, but the ruby patch lets
you have several independent active VTs at the same time.  So
the above mentioned problem is solved - they keyboards does
not interfere with each other.

Helge Hafting

Re: Kernel thoughts of a Linux user

[Tomas Carnecky]

Helge Hafting wrote:
>> From the [ruby patch] documentation:
>> The main problem up to this date (November 2004) is that linux kernel 
>> has only one behaviour regarding multiple keyboards : any key pressed 
>> catched on any keyboard is redirected to the one and only active 
>> Virtual Terminal (VT).
>>
>> Will this be changed/improved when the console code is moved into 
>> userspace, like some have proposed?
> 
> 
> I don't know about any userspace console, but the ruby patch lets
> you have several independent active VTs at the same time.  So
> the above mentioned problem is solved - they keyboards does
> not interfere with each other.
> 

I think the it would be much nicer to habe the console code in 
userspace, ruby is only a patch, not in the mainline kernel, and AFAIK 
not even in any experimental (-mm/-ac/-etc) tree.
There are many aproaches how to solve the problem of having more than 
one ative VT, and the userspace console seems to be the nicest one.

I know that Jon Smirl wrote an email some time ago, here it is: 
http://lkml.org/lkml/2004/8/2/111, look at point 15. I like the idea and 
I've written him several times, but he didn't answer :(
Anyone knows what's happened with him?
I know he's involved in the DRM development, so I CC to the dri-devel list.

I'd really like to help with this, as I like and share his ideas.

Is anyone already working on this? I mean pulling the console code out 
of the kernel into the userspace.

tom

Re: Kernel thoughts of a Linux user

[Helge Hafting]

On Wed, Nov 24, 2004 at 12:31:05PM +0100, Tomas Carnecky wrote:
> Helge Hafting wrote:
> >>From the [ruby patch] documentation:
> >>The main problem up to this date (November 2004) is that linux kernel 
> >>has only one behaviour regarding multiple keyboards : any key pressed 
> >>catched on any keyboard is redirected to the one and only active 
> >>Virtual Terminal (VT).
> >>
> >>Will this be changed/improved when the console code is moved into 
> >>userspace, like some have proposed?
> >
> >
> >I don't know about any userspace console, but the ruby patch lets
> >you have several independent active VTs at the same time.  So
> >the above mentioned problem is solved - they keyboards does
> >not interfere with each other.
> >
> 
> I think the it would be much nicer to habe the console code in 
> userspace, ruby is only a patch, not in the mainline kernel, and AFAIK 
> not even in any experimental (-mm/-ac/-etc) tree.

So what? 
It may not be ready for inclusion yet, how does that matter?
It is being worked on.  I see problems with a userspace implememtation,
the console have to be available - but a userspace console
can be killed.  By the never perfect OOM-killer, for example.

> There are many aproaches how to solve the problem of having more than 
> one ative VT, and the userspace console seems to be the nicest one.
> 
Why nicest?
Of course ruby isn't there right now - but is there a working
userspace console anywhere?

> I know that Jon Smirl wrote an email some time ago, here it is: 
> http://lkml.org/lkml/2004/8/2/111, look at point 15. I like the idea and 
> I've written him several times, but he didn't answer :(

Interesting ideas and many good points there.  The console is 
only a small part of it though, it seems to be mostly 
2D/3D/framebuffer/drm problems.

A console is a small thing and separating it from the rest
is a good idea. I am not so sure putting it in userspace is
though.

Helge Hafting

Re: Kernel thoughts of a Linux user

[H. Peter Anvin]

Followup to:  <200411181859.27722.gjwucherpfennig@gmx.net>
By author:    "Gerold J. Wucherpfennig" <gjwucherpfennig@gmx.net>
In newsgroup: linux.dev.kernel
>
> - setting up kernel boot parameters with graphical tools is unreliable, 
> because the system doesn't know which bootloader entry was chosen.
> One solution to this issue is to create a new kernel parameter "loaderhint"
> where the bootloader will be able to set the number of the chosen boot entry.
> In the configuration file of the bootloader this will have to be explicitly 
> remarked e.g. ... loaderhint=%selection% ... . Unfortunately this could be
> circumvented, because it isn't mandatory and could be manipulated in the boot 
> loader configuration. (Other/better suggestions are welcome)
> 

Most boot loaders have an BOOT_IMAGE option visible in /proc/cmdline.

	-hpa

Re: Kernel thoughts of a Linux user

[Gerold J. Wucherpfennig]

> On Thu, Nov 18, 2004 at 06:59:27PM +0100, Gerold J. Wucherpfennig wrote:
> > 
> > - Make sysfs optional and enable to publish kernel <-> userspace data
> > especially the kernel's KObject data across the kernel's netlink interface 
as
> > it has been summarized on www.kerneltrap.org. This will avoid the
> > deadlocks sysfs does introduce when some userspace app holds an open file
> > handle of an sysfs object (KObject) which is to be removed. An importrant 
side 
> > effect for embedded systems will be that the RAM overhead introduced by 
sysfs
> > will vaporize.
> 
> What RAM overhead?  With 2.6.10-rc2 the memory footprint of sysfs has
> been drasticly shrunk.

Sorry I my kernel knowledge only consists of kerneltrap.org news :-(
I didn't knew that.
 
> 
> What deadlocks are you referring to?
> 


I don't know if it are deadlocks, please read last years article from lwn:
http://lwn.net/Articles/36850/


> And the netlink interface for hotplug events is already present in the
> latest kernel.

I don't know much about netlink. But sysfs --> libsysfs --> hal --> dbus
seems to be a lot of an overhead. Maybe create an in-kernel queue
for hardware information requests and publish the hardware information
with netlink would be a little less overhead??? Just a though...

Re: Kernel thoughts of a Linux user

[Greg KH]

On Sat, Nov 20, 2004 at 11:31:12AM +0100, Gerold J. Wucherpfennig wrote:
> > On Thu, Nov 18, 2004 at 06:59:27PM +0100, Gerold J. Wucherpfennig wrote:
> > > 
> > > - Make sysfs optional and enable to publish kernel <-> userspace data
> > > especially the kernel's KObject data across the kernel's netlink interface 
> as
> > > it has been summarized on www.kerneltrap.org. This will avoid the
> > > deadlocks sysfs does introduce when some userspace app holds an open file
> > > handle of an sysfs object (KObject) which is to be removed. An importrant 
> side 
> > > effect for embedded systems will be that the RAM overhead introduced by 
> sysfs
> > > will vaporize.
> > 
> > What RAM overhead?  With 2.6.10-rc2 the memory footprint of sysfs has
> > been drasticly shrunk.
> 
> Sorry I my kernel knowledge only consists of kerneltrap.org news :-(
> I didn't knew that.

Please research things before claiming they are a problem.

> > What deadlocks are you referring to?
> 
> I don't know if it are deadlocks, please read last years article from lwn:
> http://lwn.net/Articles/36850/

My word, that's a year and a half old article.  Do you really think that
we would have not fixed this issue by now?  Again, please do a semblance
of research before claiming there are problems in today's kernels.

> > And the netlink interface for hotplug events is already present in the
> > latest kernel.
> 
> I don't know much about netlink. But sysfs --> libsysfs --> hal --> dbus
> seems to be a lot of an overhead. Maybe create an in-kernel queue
> for hardware information requests and publish the hardware information
> with netlink would be a little less overhead??? Just a though...

Again, please do a bit of research.  This is not how HAL or the hotplug
interfaces work today.

Not to be rude, but again, if you had spent a little ammount of time
looing into the claims you were making, you would have found out that
they were false.

greg k-h

Re: Kernel thoughts of a Linux user

[Gerold J. Wucherpfennig]

On Sunday 21 November 2004 19:29, you wrote:
> On Sat, Nov 20, 2004 at 11:31:12AM +0100, Gerold J. Wucherpfennig wrote:
> > > On Thu, Nov 18, 2004 at 06:59:27PM +0100, Gerold J. Wucherpfennig wrote:
> > > > - Make sysfs optional and enable to publish kernel <-> userspace data
> > > > especially the kernel's KObject data across the kernel's netlink
> > > > interface
> >
> > as
> >
> > > > it has been summarized on www.kerneltrap.org. This will avoid the
> > > > deadlocks sysfs does introduce when some userspace app holds an open
> > > > file handle of an sysfs object (KObject) which is to be removed. An
> > > > importrant
> >
> > side
> >
> > > > effect for embedded systems will be that the RAM overhead introduced
> > > > by
> >
> > sysfs
> >
> > > > will vaporize.
> > >
> > > What RAM overhead?  With 2.6.10-rc2 the memory footprint of sysfs has
> > > been drasticly shrunk.
> >
> > Sorry I my kernel knowledge only consists of kerneltrap.org news :-(
> > I didn't knew that.
>
> Please research things before claiming they are a problem.
>
> > > What deadlocks are you referring to?
> >
> > I don't know if it are deadlocks, please read last years article from
> > lwn: http://lwn.net/Articles/36850/
>
> My word, that's a year and a half old article.  Do you really think that
> we would have not fixed this issue by now?  Again, please do a semblance
> of research before claiming there are problems in today's kernels.

I'm a stupid idiot, but I'm sure that the sysfs and hal thing still has to
mature for a few years. Just imagine such things like listing all
available modem devices. Listing /sys/class/tty/*/dev without
the virtual consoles just isn't enough.

Regards,
Gerold

>
> > > And the netlink interface for hotplug events is already present in the
> > > latest kernel.
> >
> > I don't know much about netlink. But sysfs --> libsysfs --> hal --> dbus
> > seems to be a lot of an overhead. Maybe create an in-kernel queue
> > for hardware information requests and publish the hardware information
> > with netlink would be a little less overhead??? Just a though...
>
> Again, please do a bit of research.  This is not how HAL or the hotplug
> interfaces work today.
>
> Not to be rude, but again, if you had spent a little ammount of time
> looing into the claims you were making, you would have found out that
> they were false.
>
> greg k-h

Re: Kernel thoughts of a Linux user

[Greg KH]

On Mon, Nov 22, 2004 at 10:33:45PM +0100, Gerold J. Wucherpfennig wrote:
> 
> I'm a stupid idiot, but I'm sure that the sysfs and hal thing still has to
> mature for a few years.

"mature for a few years" before what happens?  It becomes a fine
vintage, and everyone enjoys it?  Or it becomes a stinking pile of
decaying matter?  I don't understand what you are getting at here.

> Just imagine such things like listing all available modem devices.
> Listing /sys/class/tty/*/dev without the virtual consoles just isn't
> enough.

You can not determine "modem devices" by just listing tty devices.  tty
devices are tty devices, some of them can be acting like a modem (like a
ACM device) and others can just be a serial port connected to a modem.

If you have issues with how HAL works, talk to the developers of it.
It's a relativly new project, and they can use the help.  But please
realize that sysfs doesn't exist for the sole reason of HAL.  HAL was
created because sysfs enabled it to be created.

thanks,

greg k-h

Re: Kernel thoughts of a Linux user

[Måns Rullgård]

"Gerold J. Wucherpfennig" <gjwucherpfennig@gmx.net> writes:

> I'm a stupid idiot, but I'm sure that the sysfs and hal thing still has to
> mature for a few years. Just imagine such things like listing all
> available modem devices. Listing /sys/class/tty/*/dev without
> the virtual consoles just isn't enough.

How would you know what's connected to a serial port?  There's
absolutely no way to tell whether it's a modem or something else, at
least no way that should be attempted inside the kernel.  Manually
checking if there's a modem can be as simple as sending some harmless
AT commands, and check for reasonable replies.  The problem is that
nobody knows what these commands might do to some other device.

-- 
Måns Rullgård
mru@inprovide.com

Re: Kernel thoughts of a Linux user

[Adam J. Richter]

On 2004-11-18 18:50:11, Greg KH wrote:
>On Thu, Nov 18, 2004 at 06:59:27PM +0100, Gerold J. Wucherpfennig wrote:
>> 
>> - Make sysfs optional and enable to publish kernel <-> userspace data
>> especially the kernel's KObject data across the kernel's netlink interface as
>> it has been summarized on www.kerneltrap.org. This will avoid the
>> deadlocks sysfs does introduce when some userspace app holds an open file
>> handle of an sysfs object (KObject) which is to be removed. An importrant side 
>> effect for embedded systems will be that the RAM overhead introduced by sysfs
>> will vaporize.
>
>What RAM overhead?  With 2.6.10-rc2 the memory footprint of sysfs has
>been drasticly shrunk.

	Looking through 2.6.10-rc2-bk6/fs/sysfs/, it appears to me that
sysfs unswappable memory usage for this desktop system with two
hard disks and a couple of USB devices attached is over 600kB, even
if I do not count the underlying attribute or kobject structures that
are being registered in sysfs.

	Please correct me if I am wrong, but, as far as I can tell,
in 2.6.10-rc2-bk6, a struct dentry is held for each node in the sysfs
tree at all times.  I infer this from noticing that sysfs_drop_dentry
and sysfs_hash_and_remove in fs/sysfs/inode.c only seem to be called
on operations to delete a node.  If I've missed something and the dentry
structures are all or mostly released, I would love to be corrected about
it as that would be really good news to me.

	Here is a partial tally of what I believe are the bytes used for
each sysfs file in the old and new versions (using sizes on my 32-bit
x86 box; your sizes may vary depending on architecture and file system
options that you've enabled).

				Old		New
	dentry			144		144
	inode			344		---
	sysfs_dirent		---		 36
	
	Total			488		180

	There is also an underlying struct attribute (12 bytes) for
sysfs files and struct kobject (52 bytes for sysfs directories), but
let's assume that all of those would still be useful without sysfs.

	The desktop computer on which I am writing this email has
3405 nodes.  So, it's pinned memory consumption has presumbably
dropped from over 1.6 megabytes in the old version to something
over 600kB.

	This is a huge improvement _over the old memory consumption_,
and may also mark the point where most of the rest of the memory
shrink could come from outside of sysfs (for example, by splitting
struct dentry into the part that virtual file systems want to
keep and the part that is only useful when the VFS layer wants
to hold the dentry, or perhaps by making struct dentry and
struct kobject use the same name string), but I would not look at
600kB and say "What RAM overhead?"

	Am I missing something?  Are the dentries being freed
dynamically (for nodes that have not been deleted) in sysfs
in version 2.6.10-rc2-bk6?

                    __     ______________ 
Adam J. Richter        \ /
adam@yggdrasil.com      | g g d r a s i l

Re: Kernel thoughts of a Linux user

[Adam J. Richter]

I wrote:
>        Please correct me if I am wrong, but, as far as I can tell,
>in 2.6.10-rc2-bk6, a struct dentry is held for each node in the sysfs
>tree at all times.  I infer this from noticing that sysfs_drop_dentry
>and sysfs_hash_and_remove in fs/sysfs/inode.c only seem to be called
>on operations to delete a node.  If I've missed something and the dentry
>structures are all or mostly released, I would love to be corrected about
>it as that would be really good news to me.

	I should correct myself, although this correction suggests
that sysfs currently uses slightly _more_ memory than I previously
thought in the case of my computer (1100 directories and 2305
non-directories in sysfs).

	In 2.6.10-rc2-bk6, it looks like sysfs releases the dname
structures as well in the case of a file (attribute) or symlink,
but keeps these structures *and* a struct inode for every directory
(kobject).  So, it looks like the non-swappable memory usage of my
/sys is actually about 900kB.

			directories		non-directories
	dentry		144			0
	inode		344			0
	sysfs_dirent	36			36

	Bytes per:	524			36
	#of nodes:	1100			2305
	Subtotal:	576,400			82,980

	Total:		659,380 bytes


	Perhaps the code that allows non-directories in sysfs to free
their inode and dname structures will in the future be extended to allow
directories do so also, which would reduce that total to 122kB.
However, even then, one might still consider how much of the 57kB of
kobject's and 36kB of attribute's and pointers to them in kset's
are due soley to sysfs, and therefore still consider it a RAM
overhead worth avoiding in some cases.

                    __     ______________ 
Adam J. Richter        \ /
adam@yggdrasil.com      | g g d r a s i l

Re: Kernel thoughts of a Linux user

[Maneesh Soni]

On Mon, Nov 22, 2004 at 10:54:01PM +0800, Adam J. Richter wrote:
> I wrote:
> >        Please correct me if I am wrong, but, as far as I can tell,
> >in 2.6.10-rc2-bk6, a struct dentry is held for each node in the sysfs
> >tree at all times.  I infer this from noticing that sysfs_drop_dentry
> >and sysfs_hash_and_remove in fs/sysfs/inode.c only seem to be called
> >on operations to delete a node.  If I've missed something and the dentry
> >structures are all or mostly released, I would love to be corrected about
> >it as that would be really good news to me.
> 

The new code doesnot pin the leaf nodes but still pins (by having an extra 
ref count will creation) the non-leaf nodes or directory dentry and inode. 
The non-directory (leaf nodes) dentry and inodes are freed as per the normal 
VFS/dcache cache eviction policies. Dentries not in use, will be pruned under 
memory pressure and eventually evict inodes also.

> 	I should correct myself, although this correction suggests
> that sysfs currently uses slightly _more_ memory than I previously
> thought in the case of my computer (1100 directories and 2305
> non-directories in sysfs).
> 
> 	In 2.6.10-rc2-bk6, it looks like sysfs releases the dname
> structures as well in the case of a file (attribute) or symlink,
> but keeps these structures *and* a struct inode for every directory
> (kobject).  So, it looks like the non-swappable memory usage of my
> /sys is actually about 900kB.
> 
> 			directories		non-directories
> 	dentry		144			0
> 	inode		344			0
> 	sysfs_dirent	36			36
> 
> 	Bytes per:	524			36
> 	#of nodes:	1100			2305
> 	Subtotal:	576,400			82,980
> 
> 	Total:		659,380 bytes
> 
> 
> 	Perhaps the code that allows non-directories in sysfs to free
> their inode and dname structures will in the future be extended to allow
> directories do so also, which would reduce that total to 122kB.

Extending this to directories introduces lots of race conditions in 
maintaining the coherancy between VFS dentry tree and sysfs_dirent
based tree. It could be doable but not without complications.

Thanks
Maneesh

-- 
Maneesh Soni
Linux Technology Center, 
IBM Austin
email: maneesh@in.ibm.com
Phone: 1-512-838-1896 Fax: 
T/L : 6781896

Re: Kernel thoughts of a Linux user

[Adam J. Richter]

I wrote:
>[...] So, it looks like the non-swappable memory usage of my
>/sys is actually about 900kB.

That number should be 659kB, as is show in the calculation that
followed it.  I somehow forgot to put in the correct figure before
I posted.  Sorry for any confusion that might have caused.

                    __     ______________ 
Adam J. Richter        \ /
adam@yggdrasil.com      | g g d r a s i l

Re: Kernel thoughts of a Linux user

[Jan Engelhardt]

>>>>So they could make themselves a favor and run something like seti@home.
>>>
>>>That does consume more energy than just sitting at idle.  I've seen some
>>>estimates of how much it costs to run seti 24/7 rather than just sit idle, 
>>>and the price was something like $80/year.
>>
>> For CPUs which don't have some sort of speedstep, it does not matter.
>> (Please correct me if I am wrong. It might be that HLT cycles are still 
>> more power-conservative even without speedstep than 24/7 on the FPU.)
>
>You're wrong :)
>Nowadays the power consumption of a CPU is more than the rest of the
>machine altogether (including hard disks, etc.).
>
>On my P4 2.8GHz HT CPU, I've measured the power consumed by *the entire
>computer* more than doubling as the processor went from idle into 100%
>load.
>
>Of course, this doesn't include a monster 3D card, is it could very well
>consume something close to the processor when doing a lot of 3D operations.

I have got a power measure device from university and experimented myself.
I keep it short: running SETI (in constrast to nothing, i.e. HLT insns),
only costs me 17 more Watts. With a price of 6 cent per kWh, this makes 
roughly 5.54 EUR per year when the machine is on 16h/340days.

(The theoretical case of 24/365 would make up 8.91 EUR.)

Wait, did not Intel pull back some processors because of their enormous heat 
of some P4 (which melted some)? Well, I guess *there* is all your $$ going.


Jan Engelhardt
-- 
ENOSPC