[ANNOUNCE] udev 0.1 release

[ANNOUNCE] udev 0.1 release

[Greg KH]

[-- Attachment #1: Type: text/plain, Size: 1906 bytes --]


Hi all,

I'd like to finally announce the previously vapor-ware udev program that
I've talked a lot about with a lot of people over the past months.  The
first, very rough cut is at:
	kernel.org/pub/linux/utils/kernel/hotplug/udev-0.1.tar.gz

But what is it?  I've included an initial design document below that was
originally written by Dan Stekloff, and hacked up a bit by me.  But in
short, udev is a userspace replacement for devfs.  It will create and
destroy /dev entries based on the current system configuration.  It does
this by watching the /sbin/hotplug events on the system, and reading
information about these events from sysfs.

Right now the program is only in 1 piece, not the 3 pieces that the
design document talks about, but it does work with the default Linux
/dev naming scheme that almost everyone uses.  It can only work for
devices that create a dev file in sysfs, exposing their major/minor
number, so this is limited (currently only block and usb-serial devices
do this.)

If you want to test this with block devices, you will need the kobject
hotplug patches previously posted here for 2.5.67, which are also
available at:
	kernel.org/pub/linux/kernel/people/gregkh/misc/kobject-hotplug-?-2.5.67.patch

Anyway, this works for me, on my machines, and I am very interested in
feedback from everyone about both this concept, and the implementation
of this.  I've cced a lot of different lists, as they have all expressed
interest in this project.

Yes, I know there's still a lot of work to do (serialization, symlinks,
hooking hotplug so that others can also use it, etc.) but it's a first
step :)

I'd like to thank Dan Stekloff for constantly badgering me about this
project and for writing lots of good design documentation, it is greatly
appreciated.  Also, thanks to Pat Mochel for coming up with sysfs which
allows this project to be able to work at all.

thanks,

greg k-h

[-- Attachment #2: overview --]
[-- Type: text/plain, Size: 4078 bytes --]


We've got a couple goals for udev:

1) dynamic replacement for /dev
2) device naming
3) API to access info about current system devices

Splitting these goals into separate subsystems:

1) udev - dynamic replacement for /dev
2) namedev - device naming
3) libsysfs - a standard library for accessing device information on the
              system.

Udev
------

Udev will be responsible for responding to /sbin/hotplug on device
events.  It will receive the device class information along with
device's sysfs directory.  Udev will call the name_device function from
the naming device subsystem with that information and receive a unique
device name in return.  Udev will then query sysfs through the libsysfs
for specific device information required for creating the /dev node like
major and minor number.  Once it has the important information, udev
will create a /dev entry for the device, add the device to the in memory
table of current devices, and send notification of the successful event
through a D-BUS message.  On a remove call, udev will remove the /dev
entry, remove the device from the in memory table, and send
notification.

Udev will consist of a command udev - to be called from /sbin/hotplug.
It will require the in memory dynamic database/table for keeping track
of current system devices, and a library of routines for accessing that
database/table.  Udev will not care about "how" devices are named, that
will be separated into the device naming subsystem.  It's presented a
common device naming API by the device naming subsystem to use for
naming devices.



namedev
----------

>From comments people have made, the device naming part of udev has been
pushed into its own "subsystem".  The reason is to make this as flexible
and pluggable as possible.  The device naming subsystem, or namedev, will
present a standard interface for udev to call for naming a particular
device.  Under that interface, system administrators can plug in their
own methods for device naming.

We would provide a default naming scheme. The first prototype
implementation could simply take the sysfs directory passed in with the
device name function, query sysfs for the major and minor numbers, and
then look up in a static device name mapping file the name of the
device. The static device naming file could look just like devices.txt
in the Linux kernel's Documentation directory.  Obviously, this isn't a
great implementation because eventually we'd like major an minor numbers
to be dynamic.

The default naming scheme in the future would have a set of policies to
go through in order to determine the name of the device.  The device
naming subsystem would get the sysfs directory of the to be named device
and would use the following information in order to map the device's
name:

1) Label info - like SCSI's UUID
2) Bus Device Number
3) Topology on Bus
4) Kernel Name - DEFAULT

System administrators could use the default naming system or enterprise
computing environments could plug in their Universal Unique Identifier
(UUID) policies.  The idea is to make the device naming as flexible and
pluggable as possible.

The device naming subsystem would require accessing sysfs for device
information.  It will receive the device's sysfs directory in the call
from udev and use it to get more information to determine naming.  The
namedev subsystem will include a standard naming API for udev to use.
The default naming scheme will include a set of functions and a static
device naming file, which will reside in /etc or /var.



libsysfs
--------

There is a need for a common API to access device information in sysfs.
The device naming subsystem and the udev subsystem need to take the
sysfs directory path and query device information.  Instead of copying
code so each one will have to readdir, etc., splitting this logic of
sysfs calls into a separate library that will sit atop sysfs makes more
sense.  Sysfs callbacks aren't standard across devices, so this is
another reason for creating a common and standard library interface for
querying device information. 

Re: [ANNOUNCE] udev 0.1 release

[Oliver Neukum]

> Anyway, this works for me, on my machines, and I am very interested in
> feedback from everyone about both this concept, and the implementation
> of this.  I've cced a lot of different lists, as they have all expressed
> interest in this project.

Cool, an utterly new piece of code to play with.
Well, it has some nice issues.

- There's a race with replugging, which you can do little about
- Error handling. What do you do if the invocation ends in EIO ?
- Performance. What happens if you plug in 4000 disks at once?

	Regards
		Oliver

Re: [ANNOUNCE] udev 0.1 release

[Jeremy Jackson]

What about read-only root fs?  What about the root= kernel command line
ever working?  What about initrd issues?

On Thu, 2003-04-10 at 23:24, Greg KH wrote:
> Hi all,
> 
> I'd like to finally announce the previously vapor-ware udev program that
> I've talked a lot about with a lot of people over the past months.  The
> first, very rough cut is at:
> 	kernel.org/pub/linux/utils/kernel/hotplug/udev-0.1.tar.gz
> 
> But what is it?  I've included an initial design document below that was
> originally written by Dan Stekloff, and hacked up a bit by me.  But in
> short, udev is a userspace replacement for devfs.  It will create and
> destroy /dev entries based on the current system configuration.  It does
> this by watching the /sbin/hotplug events on the system, and reading
> information about these events from sysfs.
> 

Re: [ANNOUNCE] udev 0.1 release

[Justin Cormack]

On Fri, 2003-04-11 at 18:10, Jeremy Jackson wrote:
> What about read-only root fs? 

run it on a tmpfs partition mounted at /dev


Justin

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 08:37:37AM +0200, Oliver Neukum wrote:
> 
> > Anyway, this works for me, on my machines, and I am very interested in
> > feedback from everyone about both this concept, and the implementation
> > of this.  I've cced a lot of different lists, as they have all expressed
> > interest in this project.
> 
> Cool, an utterly new piece of code to play with.
> Well, it has some nice issues.
> 
> - There's a race with replugging, which you can do little about

True, but this can get smaller.

> - Error handling. What do you do if the invocation ends in EIO ?

Which invocation?  From /sbin/hotplug?

> - Performance. What happens if you plug in 4000 disks at once?

You crash your power supply :)

Seriously, the kernel spawns 4000 instances of /sbin/hotplug just like
it always does.  I'm working on keeping udev from spawning anything else
to keep the process cound down (right now it fork/execs for mknod, but
that was just me being lazy.)

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 01:10:47PM -0400, Jeremy Jackson wrote:
> What about read-only root fs?

Make /dev a ramfs partition.

> What about the root= kernel command line ever working?

See next answer.

> What about initrd issues?

You mean initramfs, right?  :)
I wrote this in C for it to go into initramfs so that it would get
called at device discovery early in the boot process so that the root=
issue would work transparently.

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[John Bradford]

> > - Performance. What happens if you plug in 4000 disks at once?
> 
> You crash your power supply :)

[Puzzle]

Say the power supply had five 5.25" drive power connecters, how many 1
into 3 power cable splitters would you need to connect all 4000 disks?

:-)

John.

Re: [ANNOUNCE] udev 0.1 release

[Roman Zippel]

Hi,

On Thu, 10 Apr 2003, Greg KH wrote:

> I'd like to finally announce the previously vapor-ware udev program that
> I've talked a lot about with a lot of people over the past months.  The
> first, very rough cut is at:
> 	kernel.org/pub/linux/utils/kernel/hotplug/udev-0.1.tar.gz

Is there a special reason why you call mknod?
Otherwise you could simply do:

	syscall(SYS_mknod, name, S_IFBLK | mode, dev); 

This way you don't have to care about major/minor/glibc issues.

> Yes, I know there's still a lot of work to do (serialization, symlinks,
> hooking hotplug so that others can also use it, etc.) but it's a first
> step :)

To help serialization and perfomance issues, it might help to add a daemon 
mode to hotplug. The kernel calls hotplug with a pipe from which it reads 
the event data, after a certain timeout it can close the pipe and exit.

bye, Roman

Re: [ANNOUNCE] udev 0.1 release

[Michael Buesch]

On Friday 11 April 2003 19:46, John Bradford wrote:
> [Puzzle]
> Say the power supply had five 5.25" drive power connecters, how many 1
> into 3 power cable splitters would you need to connect all 4000 disks?

400 (hit me if this is wrong :)

Regards
Michael Buesch.


-- 
My homepage: http://www.8ung.at/tuxsoft
fighting for peace is like fu**ing for virginity

Re: [ANNOUNCE] udev 0.1 release

[Oliver Neukum]

> > - There's a race with replugging, which you can do little about
>
> True, but this can get smaller.

There isn't such a thing as a small race. Either there is a race or there
is no race. 'Should usually work' is not enough, especially when security
is concerned.

> > - Error handling. What do you do if the invocation ends in EIO ?
>
> Which invocation?  From /sbin/hotplug?

Yes.
This is a serious problem. Your scheme has very nasty failure modes.
By implementing this in user space you are introducing additional
failure modes.
- You need disk access -> EIO
- You have no control over memory allocation -> ENOMEM, EIO in swap space
Usually I'd not care about EIO, but here security is threatened. EIO crashing
the system under some circumstances is inevitable, EIO opening a security
hole is not acceptable however.

> > - Performance. What happens if you plug in 4000 disks at once?
>
> You crash your power supply :)
>
> Seriously, the kernel spawns 4000 instances of /sbin/hotplug just like
> it always does.  I'm working on keeping udev from spawning anything else
> to keep the process cound down (right now it fork/execs for mknod, but
> that was just me being lazy.)

4000 spawnings is 32MB for kernel stacks alone.
You cannot assume that resources will be sufficient for that.

That again is a serious problem, because you cannot resync.
If you lose a 'remove' event you're screwed.

And of course, what do you do if the driver is not yet loaded?

	Regards
		Oliver

Re: [ANNOUNCE] udev 0.1 release

[Nicholas Berry]

Hit hit hit.

Could be wrong, but I make it 1939.

Nik

>>> Michael Buesch <freesoftwaredeveloper@web.de> 04/11/03 02:03PM >>>
On Friday 11 April 2003 19:46, John Bradford wrote:
> [Puzzle]
> Say the power supply had five 5.25" drive power connecters, how many 1
> into 3 power cable splitters would you need to connect all 4000 disks?

400 (hit me if this is wrong :)

Regards
Michael Buesch.


-- 
My homepage: http://www.8ung.at/tuxsoft 
fighting for peace is like fu**ing for virginity

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org 
More majordomo info at  http://vger.kernel.org/majordomo-info.html 
Please read the FAQ at  http://www.tux.org/lkml/

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 08:02:41PM +0200, Roman Zippel wrote:
> On Thu, 10 Apr 2003, Greg KH wrote:
> 
> > I'd like to finally announce the previously vapor-ware udev program that
> > I've talked a lot about with a lot of people over the past months.  The
> > first, very rough cut is at:
> > 	kernel.org/pub/linux/utils/kernel/hotplug/udev-0.1.tar.gz
> 
> Is there a special reason why you call mknod?

I was lazy and trying to get this to work :)

> Otherwise you could simply do:
> 
> 	syscall(SYS_mknod, name, S_IFBLK | mode, dev); 

Yes, I'll switch to this way soon, just have to make sure my version of
dev stays the same size as the kernel's version...

> > Yes, I know there's still a lot of work to do (serialization, symlinks,
> > hooking hotplug so that others can also use it, etc.) but it's a first
> > step :)
> 
> To help serialization and perfomance issues, it might help to add a daemon 
> mode to hotplug. The kernel calls hotplug with a pipe from which it reads 
> the event data, after a certain timeout it can close the pipe and exit.

Yes, this is probably what is going to happen soon.

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Antonio Vargas]

On Fri, Apr 11, 2003 at 06:46:09PM +0100, John Bradford wrote:
> > > - Performance. What happens if you plug in 4000 disks at once?
> > 
> > You crash your power supply :)
> 
> [Puzzle]
> 
> Say the power supply had five 5.25" drive power connecters, how many 1
> into 3 power cable splitters would you need to connect all 4000 disks?
> 
> :-)
> 
> John.

Also, assuming 1 second spinup, you would need
4000 seconds == 66.66... minutes == 1.11... hours
for a sequential spinup (which _may_ not crash
the power supply)

Antonio.

Re: [ANNOUNCE] udev 0.1 release

[Oliver Neukum]

Am Freitag, 11. April 2003 19:46 schrieb John Bradford:
> > > - Performance. What happens if you plug in 4000 disks at once?
> >
> > You crash your power supply :)
>
> [Puzzle]
>
> Say the power supply had five 5.25" drive power connecters, how many 1
> into 3 power cable splitters would you need to connect all 4000 disks?

About 2000.
5 + 15 + 45 + 135 + 405 +1215 and a few more.

	Regards
		Oliver

Re: [ANNOUNCE] udev 0.1 release

[Kevin P. Fleming]

Antonio Vargas wrote:

> On Fri, Apr 11, 2003 at 06:46:09PM +0100, John Bradford wrote:
> 
>>>>- Performance. What happens if you plug in 4000 disks at once?
>>>
>>>You crash your power supply :)
>>
>>[Puzzle]
>>
>>Say the power supply had five 5.25" drive power connecters, how many 1
>>into 3 power cable splitters would you need to connect all 4000 disks?
>>

OK, this is all fun and games, but this is a valid point. All it takes for the 
driver for a Fibre Channel host adapter to load, and enumerate the devices it 
can see. In a matter of seconds many hundreds or thousands of disk devices could 
be registered with the kernel.

This is definitely an issue that will need to be addressed, and I think Oliver's 
suggestion of using a pipe (i'm going to say it: like devfs did :-) to forward 
the events to /sbin/hotplug in a FIFO fashion makes some sense. I have also been 
considering this issue from another angle; I am working on userspace partition 
discovery, which will be driven by /sbin/hotplug (and udev, probably). I have 
concerns that the following scenario will cause problems, if not extreme problems:

- kernel driver finds an IDE drive, registers it and the hotplug event happens
- udev gets called and gives it device node /dev/discs/disc0 (or whatever)
- /sbin/hotplug calls userspace partition discovery, which opens the device and 
scans for partitions
- if any partitions are found, they are registered with the kernel using 
device-mapper ioctls
- because these new "mapped sections" of the drive are _also_ usable block 
devices in their own right, they generate hotplug events
- because these hotplug events are for new block devices, userspace partition 
discovery will get called _again_ to handle them (it may not find anything (the 
normal case), but this model will support nearly infinite levels of partitioning 
on any block device supported by the kernel)

What happens if these secondary hotplug events occur while /sbin/hotplug has not 
yet finished processing the first one? Ignoring locking/race issues for the 
moment, I'm concerned about memory consumption as many layers of 
hotplug/udev/kpartx/etc. are running processing these events.

Of course, another possibility I'll look into this weekend is to actually have 
kpartx run as a daemon and receive messages over D-BUS, instead of being invoked 
directly by /sbin/hotplug. This would mean it could serialize the events itself 
and reduce some of the load (if D-BUS supports message queueing, which I believe 
it does).

Actually, here's another thought: have the kernel continue to call /sbin/hotplug 
for every event, just as it does now. However, /sbin/hotplug would do _nothing_ 
but translate that into D-BUS messages and post them. udev, kpartx, etc. would 
all just be D-BUS clients that would respond to their messages as they are received.

Re: [ANNOUNCE] udev 0.1 release

[Eric Weigle]

> > [Puzzle]
> > Say the power supply had five 5.25" drive power connecters, how many 1
> > into 3 power cable splitters would you need to connect all 4000 disks?
>> 400 (hit me if this is wrong :)
> Hit hit hit.
> 
> Could be wrong, but I make it 1939.
> Nik
Uhm.. how? You start with 5, each 1-3 splitter takes one and provides 3,
net gain of 2, regardless of how it's wired. 5+2*x >= 4000 --> x>=1998.

-Eric

-- 
------------------------------------------------------------
        Eric H. Weigle -- http://public.lanl.gov/ehw/
"They that can give up essential liberty to obtain a little
temporary safety deserve neither" -- Benjamin Franklin
------------------------------------------------------------

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 08:12:05PM +0200, Oliver Neukum wrote:
> 
> > > - There's a race with replugging, which you can do little about
> >
> > True, but this can get smaller.
> 
> There isn't such a thing as a small race. Either there is a race or there
> is no race. 'Should usually work' is not enough, especially when security
> is concerned.

You are talking about the "issue" of /dev/foo going away because that
device was removed, and then another device added which creates /dev/foo
just as the user starts to open /dev/foo?  Or something else?

> > > - Error handling. What do you do if the invocation ends in EIO ?
> >
> > Which invocation?  From /sbin/hotplug?
> 
> Yes.
> This is a serious problem. Your scheme has very nasty failure modes.
> By implementing this in user space you are introducing additional
> failure modes.
> - You need disk access -> EIO

If udev becomes a deamon, disk access isn't needed.  Actually the
current version of udev doesn't require any disk access, other than
loading it into memory.

> - You have no control over memory allocation -> ENOMEM, EIO in swap space
> Usually I'd not care about EIO, but here security is threatened. EIO crashing
> the system under some circumstances is inevitable, EIO opening a security
> hole is not acceptable however.

So yes, doing this in userspace causes a number of these kinds of
"problems".  The same kinds of "problems" that all other user programs
have to deal with, right?

So, if udev can't be read from the disk, the machine isn't in a very
workable state, creating that new device node is going to be the least
of your worries.

If udev can't get access to memory (actually it does no malloc calls, so
it would have to run out of stack space), or there is no memory to load
udev into memory, then again, you have a unstable machine, and there's
not much else we can do about it.

> 4000 spawnings is 32MB for kernel stacks alone.
> You cannot assume that resources will be sufficient for that.

If you have 4000 disks, you have to have a _lot_ of memory just to deal
with it.  See the other 4000 disk threads for more discussions about
this issue.

If we fix up the kernel to handle that many different disks, then
userspace can surely handle 4000 tasks (it can handle that today,
right?)

Anyway, it will be quite difficult to plug 4000 disks in "all at once".
There is a time delay inbetween discovering each of those disks from
within the kernel, not to mention the physical issues of spinning them
all up.

> That again is a serious problem, because you cannot resync.
> If you lose a 'remove' event you're screwed.

Yes, if you lose a remove, things can get out of whack.  My goal is to
not lose any.

> And of course, what do you do if the driver is not yet loaded?

Nothing.  udev requires that the kernel assign a major/minor to a
device, which means that a driver has to be bound to the device.
Binding drivers to devices is the current hotplug task, and has nothing
to do with udev (with the exception that we have to be able to call both
programs for each hotplug event, but I'm working on that.)

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Oliver Neukum]

Am Freitag, 11. April 2003 20:30 schrieb Oliver Neukum:
> Am Freitag, 11. April 2003 19:46 schrieb John Bradford:
> > > > - Performance. What happens if you plug in 4000 disks at once?
> > >
> > > You crash your power supply :)
> >
> > [Puzzle]
> >
> > Say the power supply had five 5.25" drive power connecters, how many 1
> > into 3 power cable splitters would you need to connect all 4000 disks?
>
> About 2000.
> 5 + 15 + 45 + 135 + 405 +1215 and a few more.

Or to be more exact. To 1215 connectors you can connect 3645
drives. 4000-3645 = 355 drives need 178 further splitters.

	Regards
		Oliver

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 11:31:28AM -0700, Kevin P. Fleming wrote:
> 
> OK, this is all fun and games, but this is a valid point. All it takes for 
> the driver for a Fibre Channel host adapter to load, and enumerate the 
> devices it can see. In a matter of seconds many hundreds or thousands of 
> disk devices could be registered with the kernel.

Sure, the kernel can handle spawning hundreds or thousands of tasks all
at once, it's not a problem.

> This is definitely an issue that will need to be addressed, and I think 
> Oliver's suggestion of using a pipe (i'm going to say it: like devfs did 
> :-) to forward the events to /sbin/hotplug in a FIFO fashion makes some 
> sense.

I agree too.  Having /sbin/hotplug send events to a pipe where a daemon
can get them from makes a lot of sense.  It will handle most of the
synchronization and spawning a zillion tasks problems.

> I have also been considering this issue from another angle; I am 
> working on userspace partition discovery, which will be driven by 
> /sbin/hotplug (and udev, probably). I have concerns that the following 
> scenario will cause problems, if not extreme problems:
> 
> - kernel driver finds an IDE drive, registers it and the hotplug event 
> happens
> - udev gets called and gives it device node /dev/discs/disc0 (or whatever)
> - /sbin/hotplug calls userspace partition discovery, which opens the device 
> and scans for partitions
> - if any partitions are found, they are registered with the kernel using 
> device-mapper ioctls
> - because these new "mapped sections" of the drive are _also_ usable block 
> devices in their own right, they generate hotplug events
> - because these hotplug events are for new block devices, userspace 
> partition discovery will get called _again_ to handle them (it may not find 
> anything (the normal case), but this model will support nearly infinite 
> levels of partitioning on any block device supported by the kernel)
> 
> What happens if these secondary hotplug events occur while /sbin/hotplug 
> has not yet finished processing the first one? Ignoring locking/race issues 
> for the moment, I'm concerned about memory consumption as many layers of 
> hotplug/udev/kpartx/etc. are running processing these events.

Yes, this can quickly get recursive up to a point.  There will never be
an infinite number of partitions, so we will eventually quiet down.

> Of course, another possibility I'll look into this weekend is to actually 
> have kpartx run as a daemon and receive messages over D-BUS, instead of 
> being invoked directly by /sbin/hotplug. This would mean it could serialize 
> the events itself and reduce some of the load (if D-BUS supports message 
> queueing, which I believe it does).

Problem is I don't think we can use D-BUS messages during early boot,
before init is called, so we still have to be able to handle startup
issues.  But hopefully the D-BUS code can be small enough to possibly be
used in this manner, I haven't checked that out yet.

> Actually, here's another thought: have the kernel continue to call 
> /sbin/hotplug for every event, just as it does now. However, /sbin/hotplug 
> would do _nothing_ but translate that into D-BUS messages and post them. 
> udev, kpartx, etc. would all just be D-BUS clients that would respond to 
> their messages as they are received.

That's another possibility too.  This is getting interesting :)

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Mike Dresser]

On Fri, 11 Apr 2003, John Bradford wrote:

> [Puzzle]
>
> Say the power supply had five 5.25" drive power connecters, how many 1
> into 3 power cable splitters would you need to connect all 4000 disks?

never seen a 3 into 1, but we'll play along.

every 3 in 1 connector you use, triples your connections, go two
levels deep, it's 3 * 3 = 9, pretty obvious.

I'm just going to use one connector off the power supply for now.

3 ^ 7 = 2187

So we'll use another two off the PS and not go as deep

2 * (3 ^ 6) = 1458

2187 + 1458 = 3645

Need another 4000 - 3645 = 355

3 ^ 5 = 243.

Need another 355 - 243 = 112

Use another connector

3 ^ 4 = 81

Need another 112 - 81 = 31.  So close but out of first level connectors.

So I put a 3 way splitter on one of those.

3 ^ 3 = 27.  argh!

Need another 31+1(had to use up a connector) - 27 = 5.
So I put another 3 way splitter. 5 + 1 = 6.

3 ^ 1 = 3.  Need another three.  Take one off, pass it around, and we're
almost done.  Add another 3 way splitter, we've got two connectors left
over, and everything is plugged in.  Turn on the ps.  Boom, done.

Now, to count up the total.

level 1 = 1	    = 1
level 2 = 1 + 3	    =  4
Level 3 = 4 + 9     = 13
Level 4 = 13 + 27   = 40
Level 5 = 40 + 81   = 121
Level 6 = 121 + 243 = 324
Level 7 = 324 + 729 = 1053

So anyways, I used one level 7, two level 6's, one level 5, one level 4,
one level 1, one level 3, one level 1, another level 1 and finally another
level 1

total of 1053 + 324 + 324 + 121 + 40 + 1 + 13 + 1 + 1 + 1

I come up with 1879.

If i'm too high, that's to account for the fact you broke some connectors
at some point, so you used the spares.

If i'm too low, that's to account for the fact that out of 4000 drives, a
few are going to be DOA and you couldn't hook them up anyways.

Someone PLEASE tell me the simpler way to do this.

Mike

Re: [ANNOUNCE] udev 0.1 release

[Joel Becker]

On Fri, Apr 11, 2003 at 11:31:28AM -0700, Kevin P. Fleming wrote:
> - if any partitions are found, they are registered with the kernel using 
> device-mapper ioctls
> - because these new "mapped sections" of the drive are _also_ usable block 
> devices in their own right, they generate hotplug events

	In reality, we need /dev/disk0 for disks, and /dev/part0 for
partitions, and /dev/lv0 for logical volumes from the LVM.  There's
going to be a war over this naming, and that's why this is hard.

Joel

-- 

"And yet I fight,
 And yet I fight this battle all alone.
 No one to cry to;
 No place to call home."

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: [ANNOUNCE] udev 0.1 release

[Havoc Pennington]

On Fri, Apr 11, 2003 at 12:07:17PM -0700, Greg KH wrote: 
> Problem is I don't think we can use D-BUS messages during early boot,
> before init is called, so we still have to be able to handle startup
> issues.  But hopefully the D-BUS code can be small enough to possibly be
> used in this manner, I haven't checked that out yet.

I'm not sure what the threshold for small enough is, but I'll give you
an analysis of D-BUS size.

The current size is:

# size /usr/lib/libdbus-1.so
   text    data     bss     dec     hex filename
 138905    1548     148  140601   22539 /usr/lib/libdbus-1.so
# size /usr/bin/dbus-daemon-1
   text    data     bss     dec     hex filename
 170756    1616     160  172532   2a1f4 /usr/bin/dbus-daemon-1

The daemon statically links in the whole library, so the 138K is in
fact copied twice; the daemon itself is not that much code.  I'm not
sure if libtool is already doing GC on unused statically-linked-in
code or if -fgc-sections would help.

The library contains some dead/unused code that could be cleaned up,
but there are also a few small things not yet implemented.  So I
expect to stay in approximately the 150K range over time.

We could probably get down to 100K by not handling out of memory (just
aborting on out of memory), but it seems like we need to handle that.
The OOM error codepaths are a pretty substantial percentage of overall
code size, surprisingly so.

The other significant fraction of the library is that D-BUS contains
its own little utility lib that reinvents bits of GLib/Qt - hash
table, linked list, memory pools, etc.

This utility lib API isn't exported from libdbus-1.so, but is used by
the daemon, which is why the daemon is statically linked.

Library code specific to D-BUS functionality, minus generic utility
code, minus OOM handling, is probably on the order of 50K.

There is one external dependency, which is an XML parser. 
libexpat is the smallest one I know of:
  # size /usr/lib/libexpat.so
    text    data     bss     dec     hex filename
  122440    6048       4  128492   1f5ec /usr/lib/libexpat.so

As this dependency is only for the daemon, -fgc-sections and static
linkage could help. We could also cut-and-paste the "gmarkup" 
pseudo-fake-xml-subset parser from GLib, that is about this big:

# size /cvs/gnome-cvs/glib/glib/gmarkup.o
   text    data     bss     dec     hex filename
  13816       4       0   13820    35fc  /cvs/gnome-cvs/glib/glib/gmarkup.o

Finally, as D-BUS is meant to be a well-defined protocol, it would
also be possible to create a "lite" client library perhaps, by
choosing to make certain sacrifices. (e.g. by allowing only a single
connection at a time, not handling OOM, and always blocking instead of
having the ability to integrate into a main loop.) Obviously that
would be a maintenance burden, though.

Havoc

Re: [ANNOUNCE] udev 0.1 release

[Oliver Neukum]

> You are talking about the "issue" of /dev/foo going away because that
> device was removed, and then another device added which creates /dev/foo
> just as the user starts to open /dev/foo?  Or something else?

The name is not important. The numbers matter.
A device goes away. It has major:minor x:y. Say it has the name /dev/foo. 
A new device is added and reuses x:y, name /dev/foo2. During the window
while /dev/foo isn't unlinked, you have access to /dev/foo2 with possibly
wrong permissions. It's worse, if you miss a 'remove' event. In that case
you are potentially permanently screwed.

You can avoid that if you never reuse device numbers. But in that case
you'd better think about a 16:48 major:minor split.

> > > > - Error handling. What do you do if the invocation ends in EIO ?
> > >
> > > Which invocation?  From /sbin/hotplug?
> >
> > Yes.
> > This is a serious problem. Your scheme has very nasty failure modes.
> > By implementing this in user space you are introducing additional
> > failure modes.
> > - You need disk access -> EIO
>
> If udev becomes a deamon, disk access isn't needed.  Actually the
> current version of udev doesn't require any disk access, other than
> loading it into memory.

Not true. You might need to swap out memory it uses.

> > - You have no control over memory allocation -> ENOMEM, EIO in swap space
> > Usually I'd not care about EIO, but here security is threatened. EIO
> > crashing the system under some circumstances is inevitable, EIO opening a
> > security hole is not acceptable however.
>
> So yes, doing this in userspace causes a number of these kinds of
> "problems".  The same kinds of "problems" that all other user programs
> have to deal with, right?

They don't and don't have to. Everything else fails securely. If a PAM
module fails, you get no access.

And yes, any scheme that handles device removal in user space has this
problem.

> So, if udev can't be read from the disk, the machine isn't in a very
> workable state, creating that new device node is going to be the least
> of your worries.

Why? You need just a single sector to fail. Murphy will always strike.

> If udev can't get access to memory (actually it does no malloc calls, so
> it would have to run out of stack space), or there is no memory to load
> udev into memory, then again, you have a unstable machine, and there's
> not much else we can do about it.

By this logic you should remove the checks for kmalloc failing.
Running a box into OOM happens. Worse, it can be triggered intentionally.

> > 4000 spawnings is 32MB for kernel stacks alone.
> > You cannot assume that resources will be sufficient for that.
>
> If you have 4000 disks, you have to have a _lot_ of memory just to deal
> with it.  See the other 4000 disk threads for more discussions about
> this issue.
>
> If we fix up the kernel to handle that many different disks, then
> userspace can surely handle 4000 tasks (it can handle that today,
> right?)

Nevertheless, kmalloc can fail and you need to allocate a kernel stack and
page tables to spawn a task. Security must not suffer from that.

> Anyway, it will be quite difficult to plug 4000 disks in "all at once".
> There is a time delay inbetween discovering each of those disks from
> within the kernel, not to mention the physical issues of spinning them
> all up.

Spinning them up? Plug in a cable to a FibreChannel fabric and you get
exactly that situation.

> > That again is a serious problem, because you cannot resync.
> > If you lose a 'remove' event you're screwed.
>
> Yes, if you lose a remove, things can get out of whack.  My goal is to
> not lose any.

How? Or precisely, how can you guarantee it?

	Regards
		Oliver

Re: [ANNOUNCE] udev 0.1 release

[Kevin P. Fleming]

Joel Becker wrote:

> On Fri, Apr 11, 2003 at 11:31:28AM -0700, Kevin P. Fleming wrote:
> 
>>- if any partitions are found, they are registered with the kernel using 
>>device-mapper ioctls
>>- because these new "mapped sections" of the drive are _also_ usable block 
>>devices in their own right, they generate hotplug events
> 
> 
> 	In reality, we need /dev/disk0 for disks, and /dev/part0 for
> partitions, and /dev/lv0 for logical volumes from the LVM.  There's
> going to be a war over this naming, and that's why this is hard.
> 
> Joel
> 

No doubt. And then you get into the situation where the devices themselves have 
names and/or UUIDs, and you want that to be incorporated into the device name. 
As it stands today, the only way to achieve that is pass that information to 
device-mapper so it can create devices with those names.

Personally, I wouldn't be upset if _all_ "physical volumes" (meaning an 
accessible block devices or portion thereof) appeared under /dev/volume/... Even 
logical volumes could be done that way, since they have names as well. I don't 
really see the need to have "whole disks", "partitions" and other types of 
volumes in separate directories under /dev, but then I may be way off base with 
the rest of the world wants to do :-)

Re: [ANNOUNCE] udev 0.1 release

[Richard B. Johnson]

On Fri, 11 Apr 2003, Mike Dresser wrote:

> On Fri, 11 Apr 2003, John Bradford wrote:
>
> > [Puzzle]
> >
> > Say the power supply had five 5.25" drive power connecters, how many 1
> > into 3 power cable splitters would you need to connect all 4000 disks?
>
> never seen a 3 into 1, but we'll play along.
>
> every 3 in 1 connector you use, triples your connections, go two
> levels deep, it's 3 * 3 = 9, pretty obvious.
>
> I'm just going to use one connector off the power supply for now.
>
> 3 ^ 7 = 2187
>
> So we'll use another two off the PS and not go as deep
>
> 2 * (3 ^ 6) = 1458
>
> 2187 + 1458 = 3645
>
> Need another 4000 - 3645 = 355
>
> 3 ^ 5 = 243.
>
> Need another 355 - 243 = 112
>
> Use another connector
>
> 3 ^ 4 = 81
>
> Need another 112 - 81 = 31.  So close but out of first level connectors.
>
> So I put a 3 way splitter on one of those.
>
> 3 ^ 3 = 27.  argh!
>
> Need another 31+1(had to use up a connector) - 27 = 5.
> So I put another 3 way splitter. 5 + 1 = 6.
>
> 3 ^ 1 = 3.  Need another three.  Take one off, pass it around, and we're
> almost done.  Add another 3 way splitter, we've got two connectors left
> over, and everything is plugged in.  Turn on the ps.  Boom, done.
>
> Now, to count up the total.
>
> level 1 = 1	    = 1
> level 2 = 1 + 3	    =  4
> Level 3 = 4 + 9     = 13
> Level 4 = 13 + 27   = 40
> Level 5 = 40 + 81   = 121
> Level 6 = 121 + 243 = 324
> Level 7 = 324 + 729 = 1053
>
> So anyways, I used one level 7, two level 6's, one level 5, one level 4,
> one level 1, one level 3, one level 1, another level 1 and finally another
> level 1
>
> total of 1053 + 324 + 324 + 121 + 40 + 1 + 13 + 1 + 1 + 1
>
> I come up with 1879.
>
> If i'm too high, that's to account for the fact you broke some connectors
> at some point, so you used the spares.
>
> If i'm too low, that's to account for the fact that out of 4000 drives, a
> few are going to be DOA and you couldn't hook them up anyways.
>
> Someone PLEASE tell me the simpler way to do this.
>
> Mike
>



Every three-connection connector supplies power to two drives.

     |--------D1
-----|--------D2    ________D3
     |______________|_______D4
                    |_______Continue

If you have 4000, drives, you need 2,000 connectors if you only
had one power cable to start. Or 'N' power cables! The number
of power cables doesn't count!  Note that you can only
add drives in "2s" so odd-numbers of drives give you an extra
connection. Also, for the limit-check, you need 0 connectors
if you have 4000 lines to start, and 1 connector on each line
if you have 2000 lines, (look above, 2,000 connectors). Now,
if you have 1000 lines, you need 2 connectors for each line.
That's still 2000 connectors!
If you have 500 lines, you need 4 connectors for each line.
That's still 2000 connectors!
If you have 250 lines, you need 8 connectors for each line.
That's still 2000 connectors!
.etc!


Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 12:28:27PM -0700, Joel Becker wrote:
> On Fri, Apr 11, 2003 at 11:31:28AM -0700, Kevin P. Fleming wrote:
> > - if any partitions are found, they are registered with the kernel using 
> > device-mapper ioctls
> > - because these new "mapped sections" of the drive are _also_ usable block 
> > devices in their own right, they generate hotplug events
> 
> 	In reality, we need /dev/disk0 for disks, and /dev/part0 for
> partitions, and /dev/lv0 for logical volumes from the LVM.

Well, that's maybe what _you_ want to call your disks and partitions,
but not what I want to call them :)

> There's going to be a war over this naming, and that's why this is
> hard.

No, there isn't.  That's what I am trying to completely avoid with udev.
It will allow you to plug in whatever device naming scheme that you
want.  This will keep all of the disc vs. disk flamewars from every
happening in the kernel community (well, I can hope...)

I'll be providing a small default name scheme that happens to match the
current Documentation/devices.txt names, and possibly a devfs naming
scheme for those people who like that scheme.  After that, it will be
quite easy for you to create a Oracle naming scheme where you give
different prefixes to your partitions vs. disks.

This scheme allows you to use databases, flat files, or even send a
message across the network to a admin console to get the name of a new
device.  Much more flexible than setting a kernel naming scheme, and
moves this whole argument and policy out of the kernel.

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Mike Dresser]

On Fri, 11 Apr 2003, Richard B. Johnson wrote:

> Every three-connection connector supplies power to two drives.
>
>      |--------D1
> -----|--------D2    ________D3
>      |______________|_______D4
>                     |_______Continue
Here's the way I thought of it.

                     |--x -- 3
       |--------X----|--x -- 3
       |             |--x -- 3
       |
       |             |--x -- 3
-------|--------X----|--x -- 3
       |             |--x -- 3
       |
       |             |--x -- 3
       |--------X----|--x -- 3
                     |--x -- 3

I now have 1 + 3 + 9 = 13 splitters, giving me 27 connections, out of 1.
etc, etc. Same numbers I'd have doing it your way, yours would be 13
levels deep instead.

I think I just went for the massively parallel method of hooking
these up and from there got massively lost.

Mike

Re: [ANNOUNCE] udev 0.1 release

[Nick Craig-Wood]

On Fri, Apr 11, 2003 at 11:12:45AM -0700, Greg KH wrote:
> On Fri, Apr 11, 2003 at 08:02:41PM +0200, Roman Zippel wrote:
> > To help serialization and perfomance issues, it might help to add a daemon 
> > mode to hotplug. The kernel calls hotplug with a pipe from which it reads 
> > the event data, after a certain timeout it can close the pipe and exit.
> 
> Yes, this is probably what is going to happen soon.

Excellent!  This should fix our continuing problems with many
identical USB devices on one bus!  In particular 6 keyspans quad
serial ports on the same USB bus never come up in the same order with
hotplug which is a bit embarrassing.  Without hotplug they come up in
the right order every time!

-- 
Nick Craig-Wood
ncw1@axis.demon.co.uk

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 09:31:56PM +0200, Oliver Neukum wrote:
> 
> > You are talking about the "issue" of /dev/foo going away because that
> > device was removed, and then another device added which creates /dev/foo
> > just as the user starts to open /dev/foo?  Or something else?
> 
> The name is not important. The numbers matter.

Agreed, I was just using names here as a simplification.

> A device goes away. It has major:minor x:y. Say it has the name /dev/foo. 
> A new device is added and reuses x:y, name /dev/foo2.  During the window
> while /dev/foo isn't unlinked, you have access to /dev/foo2 with possibly
> wrong permissions.

If you are worried about this, don't reuse x:y.  Make them purely
dynamic, and incrementing :)  Yes, this is a 2.7 thing, but will happen
eventually.  I need this framework in order to be able to do that, so
one can't happen without the other...

> It's worse, if you miss a 'remove' event. In that case you are
> potentially permanently screwed.

I don't want to ever miss events.

> You can avoid that if you never reuse device numbers. But in that case
> you'd better think about a 16:48 major:minor split.

I thought we were thinking about a 32:32 split.  But whatever, it
doesn't matter to me.

> > > > > - Error handling. What do you do if the invocation ends in EIO ?
> > > >
> > > > Which invocation?  From /sbin/hotplug?
> > >
> > > Yes.
> > > This is a serious problem. Your scheme has very nasty failure modes.
> > > By implementing this in user space you are introducing additional
> > > failure modes.
> > > - You need disk access -> EIO
> >
> > If udev becomes a deamon, disk access isn't needed.  Actually the
> > current version of udev doesn't require any disk access, other than
> > loading it into memory.
> 
> Not true. You might need to swap out memory it uses.

Then it gets swapped back in.  There isn't anything I can do from
userspace about this.  Hm, well I could pin the memory for the daemon,
but that wouldn't be nice :)

Ok, if you are worried about these kinds of things, then use the
in-kernel devfs.  I'm not going to dispute that userspace faults can
happen.

> > > - You have no control over memory allocation -> ENOMEM, EIO in swap space
> > > Usually I'd not care about EIO, but here security is threatened. EIO
> > > crashing the system under some circumstances is inevitable, EIO opening a
> > > security hole is not acceptable however.
> >
> > So yes, doing this in userspace causes a number of these kinds of
> > "problems".  The same kinds of "problems" that all other user programs
> > have to deal with, right?
> 
> They don't and don't have to. Everything else fails securely. If a PAM
> module fails, you get no access.

See above solution about not re-using numbers :)

> And yes, any scheme that handles device removal in user space has this
> problem.

True.  This is hard, let's go shopping...

> > Anyway, it will be quite difficult to plug 4000 disks in "all at once".
> > There is a time delay inbetween discovering each of those disks from
> > within the kernel, not to mention the physical issues of spinning them
> > all up.
> 
> Spinning them up? Plug in a cable to a FibreChannel fabric and you get
> exactly that situation.

Ok, and the scsi code takes a while to discover them all.  Try it with
scsi-debug, you can watch them get all created, one after another.  Yes,
we spawn a lot of userspace tasks.

> > > That again is a serious problem, because you cannot resync.
> > > If you lose a 'remove' event you're screwed.
> >
> > Yes, if you lose a remove, things can get out of whack.  My goal is to
> > not lose any.
> 
> How? Or precisely, how can you guarantee it?

I can guarantee nothing :)
But I can do a lot to prevent losses.  A lot of people around here point
to the old way PTX used to regenerate the device naming database on the
fly.  We could do that by periodically scanning sysfs to make sure we
are keeping /dev in sync with what the system has physically present.
That's one way, I'm sure there are others.

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[John Bradford]

> > Every three-connection connector supplies power to two drives.
> >
> >      |--------D1
> > -----|--------D2    ________D3
> >      |______________|_______D4
> >                     |_______Continue
> Here's the way I thought of it.
> 
>                      |--x -- 3
>        |--------X----|--x -- 3
>        |             |--x -- 3
>        |
>        |             |--x -- 3
> -------|--------X----|--x -- 3
>        |             |--x -- 3
>        |
>        |             |--x -- 3
>        |--------X----|--x -- 3
>                      |--x -- 3
> 
> I now have 1 + 3 + 9 = 13 splitters, giving me 27 connections, out of 1.
> etc, etc. Same numbers I'd have doing it your way, yours would be 13
> levels deep instead.
> 
> I think I just went for the massively parallel method of hooking
> these up and from there got massively lost.

Now, assuming a voltage drop of 0.05V across each cable...

:-)

John.

Re: [ANNOUNCE] udev 0.1 release

[Mike Dresser]

On Fri, 11 Apr 2003, John Bradford wrote:

> Now, assuming a voltage drop of 0.05V across each cable...
>
> :-)
>
> John.
>
Ah yes, I was going to mention that, but didn't know which way would be
better.  My instinct tells me the massively parallel, but I could and
probably am wrong again. :)

I did lose a few bits of my grey matter somewhere, that's for sure.

Mike

[ANNOUNCE] udev 0.1 release

[Chris Hanson]

   Date: Fri, 11 Apr 2003 16:16:52 -0400 (EDT)
   From: Mike Dresser <mdresser_l@windsormachine.com>

   On Fri, 11 Apr 2003, John Bradford wrote:

   > Now, assuming a voltage drop of 0.05V across each cable...
   >
   > :-)
   >
   > John.
   >
   Ah yes, I was going to mention that, but didn't know which way would be
   better.  My instinct tells me the massively parallel, but I could and
   probably am wrong again. :)

Using a tree (what you are calling massively parallel) for
distribution produces a uniform voltage drop for all of the devices,
and has a better worst-case voltage drop than a serial chaining
distribution.  The serial chain has different voltage drops for each
pair of disks, depending on how far down the chain they are, but the
worst case is very bad.

The reason is that the tree has O(log N) depth, and the serial chain
has O(N) depth.

Chris

Re: [ANNOUNCE] udev 0.1 release

[Steven Dake]

Kevin P. Fleming wrote:

>
> What happens if these secondary hotplug events occur while 
> /sbin/hotplug has not yet finished processing the first one? Ignoring 
> locking/race issues for the moment, I'm concerned about memory 
> consumption as many layers of hotplug/udev/kpartx/etc. are running 
> processing these events.

It gets even worse, because performance of hotswap events on disk adds 
is critical and spawning processes is incredibly slow compared to the 
performance required by some telecom applications...

A much better solution could be had by select()ing on a filehandle 
indicating when a new hotswap event is ready to be processed.  No races, 
no security issues, no performance issues.

>

Re: [ANNOUNCE] udev 0.1 release

[Mike Dresser]

On Fri, 11 Apr 2003, Chris Hanson wrote:

> Using a tree (what you are calling massively parallel) for
> distribution produces a uniform voltage drop for all of the devices,
> and has a better worst-case voltage drop than a serial chaining
> distribution.  The serial chain has different voltage drops for each
> pair of disks, depending on how far down the chain they are, but the
> worst case is very bad.
>

I do wonder how we're going to run the 46KW of power (Assuming these
drives pull similar to the drive i just checked) down the line.

Should we use solid copper bars in a bus setup?  You'll be pulling 2000
amps off the 5v, and 3000 off the 12V.  We may wish to rethink the method
of hooking up our 46000 watt power supply.  I suspect a bus may be a
better way, and probably easier to setup and maintain.

And much more fun when you drop a screwdriver across it.

Mike

Re: [ANNOUNCE] udev 0.1 release

[Richard B. Johnson]

On Fri, 11 Apr 2003, John Bradford wrote:

> > > Every three-connection connector supplies power to two drives.
> > >
> > >      |--------D1
> > > -----|--------D2    ________D3
> > >      |______________|_______D4
> > >                     |_______Continue
> > Here's the way I thought of it.
> >
> >                      |--x -- 3
> >        |--------X----|--x -- 3
> >        |             |--x -- 3
> >        |
> >        |             |--x -- 3
> > -------|--------X----|--x -- 3
> >        |             |--x -- 3
> >        |
> >        |             |--x -- 3
> >        |--------X----|--x -- 3
> >                      |--x -- 3
> >
> > I now have 1 + 3 + 9 = 13 splitters, giving me 27 connections, out of 1.
> > etc, etc. Same numbers I'd have doing it your way, yours would be 13
> > levels deep instead.
> >
> > I think I just went for the massively parallel method of hooking
> > these up and from there got massively lost.
>
> Now, assuming a voltage drop of 0.05V across each cable...
>
> :-)
>
> John.
>

Also the 1 ampere that each of drives take on the 12-volt line
that's 4000 amperes for 4000 drives. You need 8 parallel wires
of AWG #00 to handle that current.... and you need to get rid
of 4,000 * 12 = 48,000 watts of heat ;^


Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 01:29:57PM -0700, Steven Dake wrote:
> 
> It gets even worse, because performance of hotswap events on disk adds 
> is critical and spawning processes is incredibly slow compared to the 
> performance required by some telecom applications...

It's critical that we quick name this disk within X milliseconds after
it has been added to the system?  What spec declares this?

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Richard B. Johnson]

On Fri, 11 Apr 2003, Mike Dresser wrote:

> On Fri, 11 Apr 2003, Chris Hanson wrote:
>
> > Using a tree (what you are calling massively parallel) for
> > distribution produces a uniform voltage drop for all of the devices,
> > and has a better worst-case voltage drop than a serial chaining
> > distribution.  The serial chain has different voltage drops for each
> > pair of disks, depending on how far down the chain they are, but the
> > worst case is very bad.
> >
>
> I do wonder how we're going to run the 46KW of power (Assuming these
> drives pull similar to the drive i just checked) down the line.
>
> Should we use solid copper bars in a bus setup?  You'll be pulling 2000
> amps off the 5v, and 3000 off the 12V.  We may wish to rethink the method
> of hooking up our 46000 watt power supply.  I suspect a bus may be a
> better way, and probably easier to setup and maintain.
>
> And much more fun when you drop a screwdriver across it.
>
> Mike

Well the "real-world" parallels at a much higher voltage. You
probably want a single power supply per drive, which eliminates
any single-point failure problems and allows you to use standard
stuff. To keep the heat-load down, you need to distribute the
drives over several large racks anyway. That allows you to get
perhaps 5 kW per rack if you use 9 racks so things are more
reasonable.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.

Re: [ANNOUNCE] udev 0.1 release

[Oliver Neukum]

> If you are worried about this, don't reuse x:y.  Make them purely
> dynamic, and incrementing :)  Yes, this is a 2.7 thing, but will happen

Not enough numbers. Not even in a 32:32 major:minor split.

> eventually.  I need this framework in order to be able to do that, so
> one can't happen without the other...
>
> > It's worse, if you miss a 'remove' event. In that case you are
> > potentially permanently screwed.
>
> I don't want to ever miss events.

Surely, you don't want to miss them. Yet there are cases you are screwed.
Look at the hotplug spawning code and you'll see that it takes only one
kmalloc failling.

[..]
> Then it gets swapped back in.  There isn't anything I can do from
> userspace about this.  Hm, well I could pin the memory for the daemon,
> but that wouldn't be nice :)

That's exactly what you need to do. You can do this, if you change
the hotplugging notification to a pure pipe thing and lock the demon
into memory.  But then you have no advantage freom doing it in user
space, in fact you'll have the overhead of page tables for no benefit.

> Ok, if you are worried about these kinds of things, then use the
> in-kernel devfs.  I'm not going to dispute that userspace faults can
> happen.

Yes, in my oppinion putting such things into user space is stupid.
Your considerable talents would be better used to help Adam getting
his simplified devfs ready.

[..]
> > And yes, any scheme that handles device removal in user space has this
> > problem.
>
> True.  This is hard, let's go shopping...

Your attitude is admirably relaxed :-)

[..]
> > > Yes, if you lose a remove, things can get out of whack.  My goal is to
> > > not lose any.
> >
> > How? Or precisely, how can you guarantee it?
>
> I can guarantee nothing :)

Then you fail. Security without guarantee is no security.

> But I can do a lot to prevent losses.  A lot of people around here point
> to the old way PTX used to regenerate the device naming database on the
> fly.  We could do that by periodically scanning sysfs to make sure we
> are keeping /dev in sync with what the system has physically present.
> That's one way, I'm sure there are others.

Walking sysfs is a race condition by itself.
Don't get me started on that.

	Regards
		Oliver

Re: [ANNOUNCE] udev 0.1 release

[Oliver Neukum]

> I can guarantee nothing :)

I cannot resist. Believe me I tried, but I cannot. I waited years to do
that line.

> But I can do a lot to prevent losses.  A lot of people around here point

Do or don't do. There is no trying.

		-- Yoda

	May the source be with you
		Oliver

Re: [ANNOUNCE] udev 0.1 release

[Arnd Bergmann]

Greg KH wrote:

> On Fri, Apr 11, 2003 at 01:29:57PM -0700, Steven Dake wrote:
>> 
>> It gets even worse, because performance of hotswap events on disk adds 
>> is critical and spawning processes is incredibly slow compared to the 
>> performance required by some telecom applications...
> 
> It's critical that we quick name this disk within X milliseconds after
> it has been added to the system?  What spec declares this?

While the problem might not really be the per-device latency, what should 
be handled correctly is the following scenario:

- Someone accidentally removes the cable that connects a few hundred 
  (mounted) disks
- The cable is replaced, but - oops - to the wrong socket
- The person notices the error and now places the cable into the right
  socket.

At this time we have four concurrent hotplug events for every single
disks that we want to be finished in order and we want every disk
to end up with its original minor number in the end. If this is not
possible, the system still needs to be in a sensible state after this.

        Arnd <><

Re: [ANNOUNCE] udev 0.1 release

[Martin Mares]

Hello, world!\n

> Someone PLEASE tell me the simpler way to do this.

If you have an arbitrary system of splitters and you hook another 3-way
splitter to any of its outputs, you lose one output and gain 3 new
outputs, so the total number of outputs increases by 2. Hence if you
take K inputs and N 3-way splitters, the network has K+2N outputs,
no matter how the splitters are connected (of course unless you create
a cycle :-) ).

So in our case, we are searching for the smallest possible N such that
5+2N >= 4000, which equals ceil((4000-5)/2) = 1998.

				Have a nice fortnight
-- 
Martin `MJ' Mares   <mj@ucw.cz>   http://atrey.karlin.mff.cuni.cz/~mj/
Faculty of Math and Physics, Charles University, Prague, Czech Rep., Earth
God is real, unless declared integer.

Re: [ANNOUNCE] udev 0.1 release

[Alex Bligh - linux-kernel]

Each poewr connector adds 2 more (turning 1 into 3), you have 5,
you thus need to add 3995 more, and thus it seems to me you need
1998 such connectors.

A

--On 11 April 2003 14:12 -0400 Nicholas Berry <nikberry@med.umich.edu> 
wrote:

>
> Hit hit hit.
>
> Could be wrong, but I make it 1939.
>
> Nik
>
>>>> Michael Buesch <freesoftwaredeveloper@web.de> 04/11/03 02:03PM >>>
> On Friday 11 April 2003 19:46, John Bradford wrote:
>> [Puzzle]
>> Say the power supply had five 5.25" drive power connecters, how many 1
>> into 3 power cable splitters would you need to connect all 4000 disks?
>
> 400 (hit me if this is wrong :)
>
> Regards
> Michael Buesch.
>
>
> --
> My homepage: http://www.8ung.at/tuxsoft
> fighting for peace is like fu**ing for virginity
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>
>



--
Alex Bligh

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 11:09:03PM +0200, Arnd Bergmann wrote:
> 
> - Someone accidentally removes the cable that connects a few hundred 
>   (mounted) disks
> - The cable is replaced, but - oops - to the wrong socket
> - The person notices the error and now places the cable into the right
>   socket.
> 
> At this time we have four concurrent hotplug events for every single
> disks that we want to be finished in order and we want every disk
> to end up with its original minor number in the end. If this is not
> possible, the system still needs to be in a sensible state after this.

No, you want to make sure you have the same "name" after that.  As any
userspace apps that had a open file on the original disks are basically
screwed anyway, we want a way to enable them to recover nicely.

And no, I don't want to go into the whole, remove a device and plug it
back in, and userspace never noticed the difference while it held an
open file handle.  That's a problem I don't want to even go near right
now, and is totally different from what udev is trying to do :)

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Alex Bligh - linux-kernel]

--On 11 April 2003 15:59 -0400 Mike Dresser <mdresser_l@windsormachine.com> 
wrote:

>  now have 1 + 3 + 9 = 13 splitters, giving me 27 connections, out of 1.
> etc, etc. Same numbers I'd have doing it your way, yours would be 13
> levels deep instead.
>
> I think I just went for the massively parallel method of hooking
> these up and from there got massively lost.

Your 13 splitters have each added 2 connections to the 1, giving
1 + 13 x 2 = 27. There is no multiplicative effect.

Similarly to get to 4000 drives from 5, you need to add 3995
connections, i.e. 1998 splitters (giving one spare for the
floppy/cdrom).

--
Alex Bligh

Re: [ANNOUNCE] udev 0.1 release

[Alex Bligh - linux-kernel]

--On 11 April 2003 16:39 -0400 "Richard B. Johnson" 
<root@chaos.analogic.com> wrote:

> Also the 1 ampere that each of drives take on the 12-volt line
> that's 4000 amperes for 4000 drives. You need 8 parallel wires
> of AWG #00 to handle that current.... and you need to get rid
> of 4,000 * 12 = 48,000 watts of heat ;^

& I thought 64-bit dev_t flames were generating all the heat...

--
Alex Bligh

Re: [ANNOUNCE] udev 0.1 release

[Andrew Morton]

Steven Dake <sdake@mvista.com> wrote:
>
> A much better solution could be had by select()ing on a filehandle 
> indicating when a new hotswap event is ready to be processed.  No races, 
> no security issues, no performance issues.

I must say that I've always felt this to be a better approach than the
/sbin/hotplug callout.

Apart from the performance issue, it means that the kernel can buffer the
"insertion" events which happen at boot-time discovery until the userspace
handler attaches itself.

Re: [ANNOUNCE] udev 0.1 release

[Arnd Bergmann]

On Friday 11 April 2003 23:57, Greg KH wrote:

> No, you want to make sure you have the same "name" after that.  As any
> userspace apps that had a open file on the original disks are basically
> screwed anyway, we want a way to enable them to recover nicely.
>
> And no, I don't want to go into the whole, remove a device and plug it
> back in, and userspace never noticed the difference while it held an
> open file handle.  That's a problem I don't want to even go near right
> now, and is totally different from what udev is trying to do :)

Ok, but that still leaves the race between creation and removal of devices
files. In my scenario three different bad things can happen:

- The file is removed after the device is added
- The file is added when the device is already unplugged again
- The file gets added with the wrong device number -> worst case

This can be avoided by completely serializing all events, both inside
the kernel and the execution of /sbin/hotplug,  but that might cause
significant downtime.

	Arnd <><

Re: [ANNOUNCE] udev 0.1 release

[Tim Hockin]

> > A much better solution could be had by select()ing on a filehandle 
> > indicating when a new hotswap event is ready to be processed.  No races, 
> > no security issues, no performance issues.
> 
> I must say that I've always felt this to be a better approach than the
> /sbin/hotplug callout.

I've always liked this approach, too - if you look at acpid, it is designed
to be gereically useful for this model of kernel->userland notification.

With minor mods, it could become 'eventd' and handle ACPI, hotplug, netlink,
and any other style kernel->user notice.

RE: [ANNOUNCE] udev 0.1 release

[Perez-Gonzalez, Inaky]

> From: Greg KH [mailto:greg@kroah.com]
>
> But I can do a lot to prevent losses.  A lot of people around here point
> to the old way PTX used to regenerate the device naming database on the
> fly.  We could do that by periodically scanning sysfs to make sure we
> are keeping /dev in sync with what the system has physically present.
> That's one way, I'm sure there are others.

This might be a tad over-simplification, but sysfs knows by heart when
anything
is modified, because it goes through it's interface. If we only care about,
for example, devices, we could hook up into device_create() [was this the
name?];
line up in a queue all the devices for which an plug/unplug event hasn't
been
delivered to user space and create symlinks in /sysfs/hotplug-events/. 

Each entry in there is a symlink to the new device directory, named with an 
increasing integer for easy serialization. When the event is fully
processed, 
remove the entry from user space.

To avoid having to scan huge directories, it could be moved to have a single
file, event. When present, it means there are events available. It
represents
the head of the in-kernel event queue. Read it [or read the symlink] for
getting
the event information. If you remove it, that means the event is delivered.
If there is an event in the queue, the file is created again for that event.

Enhance it by creating another file for out-of-band events if needed ...

Iñaky Pérez-González -- Not speaking for Intel -- all opinions are my own
(and my fault)

Re: [ANNOUNCE] udev 0.1 release

[Steven Dake]

There is no "spec" that states this is a requirement, however, telecom 
customers require the elapsed time from the time they request the disk 
to be used, to the disk being usable by the operating system to be 20 msec.

Its even more helpful for their applications if the call that hotswap 
inserts blocks until the device is actually ready to use and available 
in the filesystem.  Another requirement of any system that attempts to 
replace devfs would be this capability (vs constantly checking for the 
device in the filesystem).

Greg KH wrote:

>On Fri, Apr 11, 2003 at 01:29:57PM -0700, Steven Dake wrote:
>  
>
>>It gets even worse, because performance of hotswap events on disk adds 
>>is critical and spawning processes is incredibly slow compared to the 
>>performance required by some telecom applications...
>>    
>>
>
>It's critical that we quick name this disk within X milliseconds after
>it has been added to the system?  What spec declares this?
>
>thanks,
>
>greg k-h
>
>
>
>  
>

Re: [ANNOUNCE] udev 0.1 release

[Steven Dake]

Arnd Bergmann wrote:

>Greg KH wrote:
>
>  
>
>>On Fri, Apr 11, 2003 at 01:29:57PM -0700, Steven Dake wrote:
>>    
>>
>>>It gets even worse, because performance of hotswap events on disk adds 
>>>is critical and spawning processes is incredibly slow compared to the 
>>>performance required by some telecom applications...
>>>      
>>>
>>It's critical that we quick name this disk within X milliseconds after
>>it has been added to the system?  What spec declares this?
>>    
>>
>
>While the problem might not really be the per-device latency, what should 
>be handled correctly is the following scenario:
>
>- Someone accidentally removes the cable that connects a few hundred 
>  (mounted) disks
>- The cable is replaced, but - oops - to the wrong socket
>- The person notices the error and now places the cable into the right
>  socket.
>
>At this time we have four concurrent hotplug events for every single
>disks that we want to be finished in order and we want every disk
>to end up with its original minor number in the end. If this is not
>possible, the system still needs to be in a sensible state after this.
>  
>
Yes, and in the rest of my email, I stated this would be solved by 
replacing /sbin/hotplug with a mechanism to communicate events to a user 
space daemon which would process requests in order as submitted by the 
kernel. (All except the same minor/major, which may or may not happen).  
If the kernel is evil and doesn't submit events in order, thats another 
problem to solve, but it should for disk events, atleast.

>        Arnd <><
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>
>
>  
>

Re: [ANNOUNCE] udev 0.1 release

[Lars Marowsky-Bree]

On 2003-04-11T15:30:21,
   Steven Dake <sdake@mvista.com> said:

> There is no "spec" that states this is a requirement, however, telecom 
> customers require the elapsed time from the time they request the disk 
> to be used, to the disk being usable by the operating system to be 20 msec.

Heh. Yes, I've read that spec, and some of it involves some good crack smoking
;-) The current Linux scheduler will make that rather hard for you, you'll
need hard realtime for such guarantees.

> Its even more helpful for their applications if the call that hotswap 
> inserts blocks until the device is actually ready to use and available 
> in the filesystem.  Another requirement of any system that attempts to 
> replace devfs would be this capability (vs constantly checking for the 
> device in the filesystem).

Uh. Can you please clarify?

You want open(/dev/not_there_yet) to block until /dev/not_there_yet is
inserted? But if it is not inserted, the device file does not exist yet, so
the open() will simply return a ENOENT.

The application (or a library, providing this capability you want) could
interact with the hotplug subsystem to be notified when this device is
inserted.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
SuSE Labs - Research & Development, SuSE Linux AG
  
"If anything can go wrong, it will." "Chance favors the prepared (mind)."
  -- Capt. Edward A. Murphy            -- Louis Pasteur

Re: [ANNOUNCE] udev 0.1 release

[Steven Dake]

Lars Marowsky-Bree wrote:

>On 2003-04-11T15:30:21,
>   Steven Dake <sdake@mvista.com> said:
>
>  
>
>>There is no "spec" that states this is a requirement, however, telecom 
>>customers require the elapsed time from the time they request the disk 
>>to be used, to the disk being usable by the operating system to be 20 msec.
>>    
>>
>
>Heh. Yes, I've read that spec, and some of it involves some good crack smoking
>;-) The current Linux scheduler will make that rather hard for you, you'll
>need hard realtime for such guarantees.
>
Its quite easy to do if you are not dependent upon spawning an entire 
process to execute the insertion and creation even of the device node.

>
>  
>
>>Its even more helpful for their applications if the call that hotswap 
>>inserts blocks until the device is actually ready to use and available 
>>in the filesystem.  Another requirement of any system that attempts to 
>>replace devfs would be this capability (vs constantly checking for the 
>>device in the filesystem).
>>    
>>
>
>Uh. Can you please clarify?
>
>You want open(/dev/not_there_yet) to block until /dev/not_there_yet is
>inserted? But if it is not inserted, the device file does not exist yet, so
>the open() will simply return a ENOENT.
>
>The application (or a library, providing this capability you want) could
>interact with the hotplug subsystem to be notified when this device is
>inserted.
>
>
>Sincerely,
>    Lars Marowsky-Brée <lmb@suse.de>
>
>  
>

Re: [ANNOUNCE] udev 0.1 release

[Andrew Morton]

Tim Hockin <thockin@hockin.org> wrote:
>
> > > A much better solution could be had by select()ing on a filehandle 
> > > indicating when a new hotswap event is ready to be processed.  No races, 
> > > no security issues, no performance issues.
> > 
> > I must say that I've always felt this to be a better approach than the
> > /sbin/hotplug callout.
> 
> I've always liked this approach, too - if you look at acpid, it is designed
> to be gereically useful for this model of kernel->userland notification.
> 
> With minor mods, it could become 'eventd' and handle ACPI, hotplug, netlink,
> and any other style kernel->user notice.

It also has the advantage that events are handled in reliable and repeatable
order.

Right now, if you plug and then quickly unplug a device, the unplug event can
be handled first.

It may not happen much in practice, but we have had problem with cardbus
contact bounce causing an event storm in the past.  The daemon could just
swallow the first 5 insert/remove pairs and process the final insert only.

The kernel would have to drop messages on the floor at some point though.

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 03:27:19PM -0700, Perez-Gonzalez, Inaky wrote:
> 
> 
> > From: Greg KH [mailto:greg@kroah.com]
> >
> > But I can do a lot to prevent losses.  A lot of people around here point
> > to the old way PTX used to regenerate the device naming database on the
> > fly.  We could do that by periodically scanning sysfs to make sure we
> > are keeping /dev in sync with what the system has physically present.
> > That's one way, I'm sure there are others.
> 
> This might be a tad over-simplification, but sysfs knows by heart when
> anything is modified, because it goes through it's interface. If we
> only care about, for example, devices, we could hook up into
> device_create() [was this the name?]; line up in a queue all the
> devices for which an plug/unplug event hasn't been delivered to user
> space and create symlinks in /sysfs/hotplug-events/. 
> 
> Each entry in there is a symlink to the new device directory, named with an 
> increasing integer for easy serialization. When the event is fully
> processed, remove the entry from user space.

Um, how do you show a symlink to a device that is no long there when the
device is removed?  :)

It would also require that users can delete files from sysfs, which
isn't currently possible.  Special casing one directory for this would
be a pain.

In the end, it's a nice idea, but the current one is much simpler, and
works today :)

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 03:30:21PM -0700, Steven Dake wrote:
> There is no "spec" that states this is a requirement, however, telecom 
> customers require the elapsed time from the time they request the disk 
> to be used, to the disk being usable by the operating system to be 20 msec.

What defines the term, "request the disk to be used"?  Slamming it into
the SCSI tray?  Mounting the device on the command line?  I don't think
you can spin up a scsi disk in 20msec today :)

> Its even more helpful for their applications if the call that hotswap 
> inserts blocks until the device is actually ready to use and available 
> in the filesystem.

What would it block from happening?  The kernel?  Userspace?  I'm
confused.

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 03:43:12PM -0700, Steven Dake wrote:
> 
> Lars Marowsky-Bree wrote:
> 
> >On 2003-04-11T15:30:21,
> >  Steven Dake <sdake@mvista.com> said:
> >
> > 
> >
> >>There is no "spec" that states this is a requirement, however, telecom 
> >>customers require the elapsed time from the time they request the disk 
> >>to be used, to the disk being usable by the operating system to be 20 
> >>msec.
> >>   
> >>
> >
> >Heh. Yes, I've read that spec, and some of it involves some good crack 
> >smoking
> >;-) The current Linux scheduler will make that rather hard for you, you'll
> >need hard realtime for such guarantees.
> >
> Its quite easy to do if you are not dependent upon spawning an entire 
> process to execute the insertion and creation even of the device node.

Then have the telcos live with the static /dev that they have today :)

There's always a price to pay for new features...

greg k-h

Happily using his "pleasure boating" version of Linux...

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 03:09:33PM -0700, Andrew Morton wrote:
> Steven Dake <sdake@mvista.com> wrote:
> >
> > A much better solution could be had by select()ing on a filehandle 
> > indicating when a new hotswap event is ready to be processed.  No races, 
> > no security issues, no performance issues.
> 
> I must say that I've always felt this to be a better approach than the
> /sbin/hotplug callout.
> 
> Apart from the performance issue, it means that the kernel can buffer the
> "insertion" events which happen at boot-time discovery until the userspace
> handler attaches itself.

But how many events to we buffer?  When do we start to throw them away?
Fun policy decisions that we don't have to worry about in the current
scheme.

Also, what's the format of the kernel->user interface.  Today with
/sbin/hotplug it's a very simple, and easily changed interaction.  Using
a event reading mechanism lends itself to binary interfaces, which have
to be kept in sync with user code very tightly.

And yes, we could use ascii in the event list, but then again, a
userspace version of /sbin/hotplug that writes events to a pipe that is
read from a daemon enables the same thing to happen :)

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 03:47:09PM -0700, Andrew Morton wrote:
> Tim Hockin <thockin@hockin.org> wrote:
> >
> > > > A much better solution could be had by select()ing on a filehandle 
> > > > indicating when a new hotswap event is ready to be processed.  No races, 
> > > > no security issues, no performance issues.
> > > 
> > > I must say that I've always felt this to be a better approach than the
> > > /sbin/hotplug callout.
> > 
> > I've always liked this approach, too - if you look at acpid, it is designed
> > to be gereically useful for this model of kernel->userland notification.
> > 
> > With minor mods, it could become 'eventd' and handle ACPI, hotplug, netlink,
> > and any other style kernel->user notice.
> 
> It also has the advantage that events are handled in reliable and repeatable
> order.

I'm looking at making device probing a zillion different threads from
within the kernel, allowing us to probe devices much faster than we do
today (and fixes a lot of nasty problems with broken hardware that we
currently have today).  That would ensure that those events would never
be in a reliable and repeatable order :)

> It may not happen much in practice, but we have had problem with cardbus
> contact bounce causing an event storm in the past.  The daemon could just
> swallow the first 5 insert/remove pairs and process the final insert only.
> 
> The kernel would have to drop messages on the floor at some point though.

Exactly, let's not do that.  The current scheme does not do that.  Out
of order is solvable, while missing events is not simple.

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 03:35:52PM -0700, Steven Dake wrote:
> Yes, and in the rest of my email, I stated this would be solved by 
> replacing /sbin/hotplug with a mechanism to communicate events to a user 
> space daemon which would process requests in order as submitted by the 
> kernel. (All except the same minor/major, which may or may not happen).  
> If the kernel is evil and doesn't submit events in order, thats another 
> problem to solve, but it should for disk events, atleast.

Multi-threaded device discovery is a good thing, it solves a lot of
real-life problems that we have today.  But it's a 2.7 thing, not on the
near horizon.

thanks,

greg k-h

Making the kernel evil since 2001

Re: [ANNOUNCE] udev 0.1 release

[Andrew Morton]

Greg KH <greg@kroah.com> wrote:
>
> On Fri, Apr 11, 2003 at 03:09:33PM -0700, Andrew Morton wrote:
> > Steven Dake <sdake@mvista.com> wrote:
> > >
> > > A much better solution could be had by select()ing on a filehandle 
> > > indicating when a new hotswap event is ready to be processed.  No races, 
> > > no security issues, no performance issues.
> > 
> > I must say that I've always felt this to be a better approach than the
> > /sbin/hotplug callout.
> > 
> > Apart from the performance issue, it means that the kernel can buffer the
> > "insertion" events which happen at boot-time discovery until the userspace
> > handler attaches itself.
> 
> But how many events to we buffer?

On a large machine: 856,432.

> When do we start to throw them away?
> Fun policy decisions that we don't have to worry about in the current
> scheme.

The current scheme will run out of processes, kernel stacks, etc before a
message scheme would.

> Also, what's the format of the kernel->user interface.

Exactly the same as at present, with /sbin/hotplug chopped off.  So you can
run the daemon:

	while read x
	do
		/sbin/hotplug $x
	done < /dev/hotplug_event_pipe

for compatibility with existing scripts.

But I'm not really very opinionated about it.  I don't expect any of it to be
super-robust, really.

Re: [ANNOUNCE] udev 0.1 release

[Joel Becker]

On Fri, Apr 11, 2003 at 12:58:43PM -0700, Greg KH wrote:
> > There's going to be a war over this naming, and that's why this is
> > hard.
> 
> No, there isn't.  That's what I am trying to completely avoid with udev.
> It will allow you to plug in whatever device naming scheme that you
> want.

	And this is exactly what we can't have happen.  If I am an
administrator, I don't want to have to write all my scripts to do:

if [ -f /etc/redhat-release ]
then
    DISKPREFIX="/dev/disk"
elif [ -f /etc/unitedlinux-release ]
then
    DISKPREFIX="/dev/disc"
elif [ -f /dev/volume0 ]
then
    DISKPREFIX="/dev/volume"
elif 
    ...


	Please, let's not repeat GNOME/KDE, OpenLook/Motif,
mkinitrd/mkinitrd, etc for all our devices.

Joel

-- 

"Friends may come and go, but enemies accumulate." 
        - Thomas Jones

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: [ANNOUNCE] udev 0.1 release

[Joel Becker]

On Fri, Apr 11, 2003 at 12:38:49PM -0700, Kevin P. Fleming wrote:
> No doubt. And then you get into the situation where the devices themselves 
> have names and/or UUIDs, and you want that to be incorporated into the 
> device name. As it stands today, the only way to achieve that is pass that 
> information to device-mapper so it can create devices with those names.

	Oh, I don't want that ever in the name.  I don't want to have to
know much about my disks, except that once the association is there,
it's there until I delete it.

> Personally, I wouldn't be upset if _all_ "physical volumes" (meaning an 
> accessible block devices or portion thereof) appeared under /dev/volume/... 
> Even logical volumes could be done that way, since they have names as well. 
> I don't really see the need to have "whole disks", "partitions" and other 
> types of volumes in separate directories under /dev, but then I may be way 
> off base with the rest of the world wants to do :-)

	There are multiple reasons.  I need to know what's a disk so I
can configure it.  I need to know what's a partition or a logical volume
so I can put filesystems on it.
	It's always fun these days when folks try to use 'whole disks'
and 'partitions' as equivalent things.  They are indeed from a block
device standpoint, but they aren't from an administration standpoint.

Joel

-- 

Life's Little Instruction Book #99

	"Think big thoughts, but relish small pleasures."

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: [ANNOUNCE] udev 0.1 release

[Arnd Bergmann]

On Saturday 12 April 2003 01:05, Greg KH wrote:

> Multi-threaded device discovery is a good thing, it solves a lot of
> real-life problems that we have today.  But it's a 2.7 thing, not on the
> near horizon.

At least the device detection in drivers/s390/cio/ is already multi-threaded
in 2.5 and it significantly speeds up the boot process, so I don't want
to revert this.

	Arnd <><

Re: [ANNOUNCE] udev 0.1 release

[Steven Dake]

Greg KH wrote:

>On Fri, Apr 11, 2003 at 03:30:21PM -0700, Steven Dake wrote:
>  
>
>>There is no "spec" that states this is a requirement, however, telecom 
>>customers require the elapsed time from the time they request the disk 
>>to be used, to the disk being usable by the operating system to be 20 msec.
>>    
>>
>
>What defines the term, "request the disk to be used"?  Slamming it into
>the SCSI tray?  Mounting the device on the command line?  I don't think
>you can spin up a scsi disk in 20msec today :)
>
I should have been more clear.

What I actually mean is:
disk is in the bus/loop/etc, powered on, ready to be enumerated.
The user then tells the OS "please insert the disk"  This is the request 
which starts the clock.
The point where a device entry is in /dev ready to be used stops the clock.

>
>  
>
>>Its even more helpful for their applications if the call that hotswap 
>>inserts blocks until the device is actually ready to use and available 
>>in the filesystem.
>>    
>>
>
>What would it block from happening?  The kernel?  Userspace?  I'm
>confused.
>  
>
An insert operation would block until the following sequence occurs:
insert a disk, disk added to subsystem (scsi, etc), hotplug event, 
hotplug handler creates device node.

Without blocking until the device is created and ready to be used, it 
becomes difficult to actually "hotswap insert" and then immediatly use 
the device, requiring polling.  Most users would like to wait for the 
event to complete, or have a select()able fd to wait on to know when the 
event has been completed.

This might be possible to emulate through dnotify, but would still 
require a rescan of the dev directory, causing poor performance.

>thanks,
>
>greg k-h
>
>
>
>  
>

Re: [ANNOUNCE] udev 0.1 release

[Steven Dake]

Greg KH wrote:

>On Fri, Apr 11, 2003 at 03:43:12PM -0700, Steven Dake wrote:
>  
>
>>Lars Marowsky-Bree wrote:
>>
>>    
>>
>>>On 2003-04-11T15:30:21,
>>> Steven Dake <sdake@mvista.com> said:
>>>
>>>
>>>
>>>      
>>>
>>>>There is no "spec" that states this is a requirement, however, telecom 
>>>>customers require the elapsed time from the time they request the disk 
>>>>to be used, to the disk being usable by the operating system to be 20 
>>>>msec.
>>>>  
>>>>
>>>>        
>>>>
>>>Heh. Yes, I've read that spec, and some of it involves some good crack 
>>>smoking
>>>;-) The current Linux scheduler will make that rather hard for you, you'll
>>>need hard realtime for such guarantees.
>>>
>>>      
>>>
>>Its quite easy to do if you are not dependent upon spawning an entire 
>>process to execute the insertion and creation even of the device node.
>>    
>>
>
>Then have the telcos live with the static /dev that they have today :)
>  
>
Unfortunately they are willing to live with devfs, but not a static 
/dev....  There are problems with devfs which I'm sure your well aware 
of which a dynamic /dev would solve...  But performance is an important 
goal.

>There's always a price to pay for new features...
>
>greg k-h
>
>Happily using his "pleasure boating" version of Linux...
>
>
>
>  
>

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 04:23:38PM -0700, Andrew Morton wrote:
> > 
> > But how many events to we buffer?
> 
> On a large machine: 856,432.

Heh.

> > When do we start to throw them away?
> > Fun policy decisions that we don't have to worry about in the current
> > scheme.
> 
> The current scheme will run out of processes, kernel stacks, etc before a
> message scheme would.
> 
> > Also, what's the format of the kernel->user interface.
> 
> Exactly the same as at present, with /sbin/hotplug chopped off.  So you can
> run the daemon:
> 
> 	while read x
> 	do
> 		/sbin/hotplug $x
> 	done < /dev/hotplug_event_pipe
> 
> for compatibility with existing scripts.

But the current interface is a single argument, and whole lot of
variable length environment variables that may or may not be set,
depending on the argument.  That would be a mess to parse from a single
stream (not undoable, but painful).

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Steven Dake]

Greg KH wrote:

>On Fri, Apr 11, 2003 at 03:09:33PM -0700, Andrew Morton wrote:
>  
>
>>Steven Dake <sdake@mvista.com> wrote:
>>    
>>
>>>A much better solution could be had by select()ing on a filehandle 
>>>indicating when a new hotswap event is ready to be processed.  No races, 
>>>no security issues, no performance issues.
>>>      
>>>
>>I must say that I've always felt this to be a better approach than the
>>/sbin/hotplug callout.
>>
>>Apart from the performance issue, it means that the kernel can buffer the
>>"insertion" events which happen at boot-time discovery until the userspace
>>handler attaches itself.
>>    
>>
>
>But how many events to we buffer?  When do we start to throw them away?
>Fun policy decisions that we don't have to worry about in the current
>scheme.
>  
>
There are all kinds of policy decisions about how many of something is 
in the kernel...  For example, file descriptors, selectable file 
descriptors, etc.

It would be simple to determine how many events should be saved by even 
the most complex application or event counts could be configurable.  The 
issue of dropping events only matters for startup, since anything later 
is likely not to have as many new devices as a startup sequence might...

>Also, what's the format of the kernel->user interface.  Today with
>/sbin/hotplug it's a very simple, and easily changed interaction.  Using
>a event reading mechanism lends itself to binary interfaces, which have
>to be kept in sync with user code very tightly.
>  
>
Binary is fine by me.  Ascii *also* has to be kept in sync, even if its 
more readable, its just as much an interface as binary.

>And yes, we could use ascii in the event list, but then again, a
>userspace version of /sbin/hotplug that writes events to a pipe that is
>read from a daemon enables the same thing to happen :)
>  
>
still a new process is spawned per event which is what we should want to 
avoid.

>thanks,
>
>greg k-h
>
>
>
>  
>

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 04:25:07PM -0700, Joel Becker wrote:
> On Fri, Apr 11, 2003 at 12:58:43PM -0700, Greg KH wrote:
> > > There's going to be a war over this naming, and that's why this is
> > > hard.
> > 
> > No, there isn't.  That's what I am trying to completely avoid with udev.
> > It will allow you to plug in whatever device naming scheme that you
> > want.
> 
> 	And this is exactly what we can't have happen.  If I am an
> administrator, I don't want to have to write all my scripts to do:
> 
> if [ -f /etc/redhat-release ]
> then
>     DISKPREFIX="/dev/disk"
> elif [ -f /etc/unitedlinux-release ]
> then
>     DISKPREFIX="/dev/disc"
> elif [ -f /dev/volume0 ]
> then
>     DISKPREFIX="/dev/volume"
> elif 
>     ...

But all the distros will do that for you :)

> 	Please, let's not repeat GNOME/KDE, OpenLook/Motif,
> mkinitrd/mkinitrd, etc for all our devices.

Then try to convince LSB to add a device naming document to their spec.
That's the only way this is going to happen...

Good luck,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Miquel van Smoorenburg]

In article <20030411190717.GH1821@kroah.com>,
Greg KH  <greg@kroah.com> wrote:
>I agree too.  Having /sbin/hotplug send events to a pipe where a daemon
>can get them from makes a lot of sense.  It will handle most of the
>synchronization and spawning a zillion tasks problems.

Why not serialize /sbin/hotplug at the kernel level. Queue hotplug
events and only allow one /sbin/hotplug to run at the same time.

Mike.

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 04:32:54PM -0700, Steven Dake wrote:
> >Then have the telcos live with the static /dev that they have today :)
> >
> Unfortunately they are willing to live with devfs, but not a static 
> /dev....  There are problems with devfs which I'm sure your well aware 
> of which a dynamic /dev would solve...  But performance is an important 
> goal.

Adam's rewrite of devfs has removed many of the older problems, it looks
quite good now, and I'm not advocating for it's removal anymore.

Different requirements result in different solutions, so if the telcos
find their requirements met in devfs, by all means, they should use it.

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 11:39:07PM +0000, Miquel van Smoorenburg wrote:
> In article <20030411190717.GH1821@kroah.com>,
> Greg KH  <greg@kroah.com> wrote:
> >I agree too.  Having /sbin/hotplug send events to a pipe where a daemon
> >can get them from makes a lot of sense.  It will handle most of the
> >synchronization and spawning a zillion tasks problems.
> 
> Why not serialize /sbin/hotplug at the kernel level. Queue hotplug
> events and only allow one /sbin/hotplug to run at the same time.

We don't want the kernel to stop based on a user program.

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Joel Becker]

On Fri, Apr 11, 2003 at 04:37:19PM -0700, Greg KH wrote:
> > if [ -f /etc/redhat-release ]
> > then
> >     DISKPREFIX="/dev/disk"
> 
> But all the distros will do that for you :)

	Oh no they won't.  They're just going to fix their own scripts
to accept their own scheme.  Never mind my own.  I want my own scheme,
the distro scripts break.  I want a script running out of shared NFS?
I lose.

> Then try to convince LSB to add a device naming document to their spec.
> That's the only way this is going to happen...

	LSB isn't even followed now.  What we need is a naming czar.  As
you point out, Good Luck.
	Linux devices are going to stop sucking from one perspective and
start sucking from another.  Yay.

Joel

-- 

"Can any of you seriously say the Bill of Rights could get through
 Congress today?  It wouldn't even get out of committee."
	- F. Lee Bailey

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: [ANNOUNCE] udev 0.1 release

[Miquel van Smoorenburg]

In article <20030412000829.GL4539@kroah.com>,
Greg KH  <greg@kroah.com> wrote:
>On Fri, Apr 11, 2003 at 11:39:07PM +0000, Miquel van Smoorenburg wrote:
>> Why not serialize /sbin/hotplug at the kernel level. Queue hotplug
>> events and only allow one /sbin/hotplug to run at the same time.
>
>We don't want the kernel to stop based on a user program.

It would not stop if you queued the events.

What is the difference between queueing events to be read from
a pipe or socket or queueing them for a kernel thread that empties
the queue by executing /sbin/hotplug for each entry in the queue.

The pipe/socket solution is probably better anyway, I was just
wondering why /sbin/hotplug wasn't serialized from the start.
 
Mike.

Re: [ANNOUNCE] udev 0.1 release

[H. Peter Anvin]

Followup to:  <20030412001913.GG4917@ca-server1.us.oracle.com>
By author:    Joel Becker <Joel.Becker@oracle.com>
In newsgroup: linux.dev.kernel
> 
> > Then try to convince LSB to add a device naming document to their spec.
> > That's the only way this is going to happen...
> 
> 	LSB isn't even followed now.  What we need is a naming czar.  As
> you point out, Good Luck.
> 	Linux devices are going to stop sucking from one perspective and
> start sucking from another.  Yay.
> 

LSB *has* a naming document.  The LSB spec clearly states that
devices.txt produced by LANANA *is* the naming document.

John Cagle is your naming czar.

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 05:19:13PM -0700, Joel Becker wrote:
> On Fri, Apr 11, 2003 at 04:37:19PM -0700, Greg KH wrote:
> > > if [ -f /etc/redhat-release ]
> > > then
> > >     DISKPREFIX="/dev/disk"
> > 
> > But all the distros will do that for you :)
> 
> 	Oh no they won't.  They're just going to fix their own scripts
> to accept their own scheme.  Never mind my own.  I want my own scheme,
> the distro scripts break.  I want a script running out of shared NFS?
> I lose.

If you want your own naming scheme, you will have to handle all of the
script changes needed, yes.  devfs ran into this same problem :(

> > Then try to convince LSB to add a device naming document to their spec.
> > That's the only way this is going to happen...
> 
> 	LSB isn't even followed now.  What we need is a naming czar.  As
> you point out, Good Luck.

Hm, naming czar, that has a nice ring to it... :)

> 	Linux devices are going to stop sucking from one perspective and
> start sucking from another.  Yay.

Well, let's work on fixing the the first suckage.  If that creats more
suckage, we'll tackle that issue next.

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 06:06:51PM -0700, H. Peter Anvin wrote:
> Followup to:  <20030412001913.GG4917@ca-server1.us.oracle.com>
> By author:    Joel Becker <Joel.Becker@oracle.com>
> In newsgroup: linux.dev.kernel
> > 
> > > Then try to convince LSB to add a device naming document to their spec.
> > > That's the only way this is going to happen...
> > 
> > 	LSB isn't even followed now.  What we need is a naming czar.  As
> > you point out, Good Luck.
> > 	Linux devices are going to stop sucking from one perspective and
> > start sucking from another.  Yay.
> > 
> 
> LSB *has* a naming document.  The LSB spec clearly states that
> devices.txt produced by LANANA *is* the naming document.

Cool, I missed that the last time I looked.

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Lars Marowsky-Bree]

On 2003-04-11T16:31:47,
   Steven Dake <sdake@mvista.com> said:

> What I actually mean is:
> disk is in the bus/loop/etc, powered on, ready to be enumerated.
> The user then tells the OS "please insert the disk"  This is the request 
> which starts the clock.
> The point where a device entry is in /dev ready to be used stops the clock.

A bus rescan on SCSI / FC will take longer than your 20ms already.

A single hickup in the block IO involved updating the information in /dev will
break this requirement. Having to fork /sbin/hotplug will.

I've set Linux on an unloaded, rather well powered machine to heartbeat every
10ms and checked in what intervals it actually did - mostly it was 10-30ms,
but there have been exceptions for upto 200-1000ms. And that's for a
locked-into-memory "soft realtime", no swapping very small piece of code.

Okay, this was our latest 2.4 kernel, and it would be interesting to see
whether 2.5 does better, but still. 20ms are highly ambitious.

I'm not saying "performance" is not a high goal. Having a permanently running
daemon to talk with instead of forking all the time is certainly a very
sensible idea, and the kernel _must_ be able to cope with 4k disks being
plugged in at once.

But evidence suggests that _guaranteeing_ 20ms seems a bit over the top.


Sincerely,
    Lars Marowsky-Brée <lmb@suse.de>

-- 
SuSE Labs - Research & Development, SuSE Linux AG
  
"If anything can go wrong, it will." "Chance favors the prepared (mind)."
  -- Capt. Edward A. Murphy            -- Louis Pasteur

Re: [ANNOUNCE] udev 0.1 release

[John Bradford]

> > - Someone accidentally removes the cable that connects a few hundred 
> >   (mounted) disks
> > - The cable is replaced, but - oops - to the wrong socket
> > - The person notices the error and now places the cable into the right
> >   socket.
> > 
> > At this time we have four concurrent hotplug events for every single
> > disks that we want to be finished in order and we want every disk
> > to end up with its original minor number in the end. If this is not
> > possible, the system still needs to be in a sensible state after this.
> 
> No, you want to make sure you have the same "name" after that.  As any
> userspace apps that had a open file on the original disks are basically
> screwed anyway, we want a way to enable them to recover nicely.

Another scenario would be if a hotpluggable disk interface failed.
Someone could purposely move the cable to a second adaptor, before
replacing the failed one.  Relevant userspace apps could be suspended
during this switch over, but when they are resumed, it would be
desirable to have the disks with the same minor numbers as before,
thereby simplifying the userspace application's requirements.

> And no, I don't want to go into the whole, remove a device and plug it
> back in, and userspace never noticed the difference while it held an
> open file handle.  That's a problem I don't want to even go near right
> now, and is totally different from what udev is trying to do :)

It's also arguably better done in userspace.  The kernel would see the
device disappear and re-appear, but a userspace abstraction layer
could identify it as being the same device, (and unmodified since it
last saw it), and act accordingly.

Effectively, reads from an unconnected device would succeed if the
data was cached, and writes would succeed at least until a certain
amount of write cache had been filled.  The result of disconnecting a
device, modifying it's contents, then re-connecting it would be
undefined.

John.

Re: [ANNOUNCE] udev 0.1 release

[Oliver Neukum]

> And yes, we could use ascii in the event list, but then again, a
> userspace version of /sbin/hotplug that writes events to a pipe that is
> read from a daemon enables the same thing to happen :)

No, it doesn't. You throw away the ordering the kernel naturally has.
If you write out events from within the kernel, you know that you'll
never see a replug before an unplug.

Writing a pipe from the kernel, which is generally called a character
device, has other advantages as well.
You can report events dropped as errors. The hotplug spawning
scheme cannot do that. No spawning cannot differentiate between
no event and an event lost.

	Regards
		Oliver

Re: [ANNOUNCE] udev 0.1 release

[Oliver Neukum]

> Right now, if you plug and then quickly unplug a device, the unplug event
> can be handled first.
>
> It may not happen much in practice, but we have had problem with cardbus
> contact bounce causing an event storm in the past.  The daemon could just
> swallow the first 5 insert/remove pairs and process the final insert only.
>
> The kernel would have to drop messages on the floor at some point though.

That is unavoidable in _any_ scheme. The hotplug spawn scheme wishes
to weasle through this by letting kmalloc determine, when to throw away
an event. No system can process an infinite amount of events.

What's important is an ability to report loss of events and to export
a consistent view of devices connected, that can be read at will.

	Regards
		Oliver

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 03:29:20PM -0400, Havoc Pennington wrote:
> On Fri, Apr 11, 2003 at 12:07:17PM -0700, Greg KH wrote: 
> > Problem is I don't think we can use D-BUS messages during early boot,
> > before init is called, so we still have to be able to handle startup
> > issues.  But hopefully the D-BUS code can be small enough to possibly be
> > used in this manner, I haven't checked that out yet.
> 
> I'm not sure what the threshold for small enough is, but I'll give you
> an analysis of D-BUS size.

<nice summary snipped>

Thanks for the information.  Hm, that seems a bit big for what I want to do,
but might be workable.

Oh, and to compare sizes, with udev linked against klibc (static link)
it comes out to a whopping big 6004 bytes:
$ size udev
   text    data     bss     dec     hex filename
   5572       4     392    5968    1750 udev

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Oliver Neukum]

> The pipe/socket solution is probably better anyway, I was just
> wondering why /sbin/hotplug wasn't serialized from the start.

It was. Deadlocks happened and the semaphore was removed.
I don't remember details. They might be in the archives.

	HTH
		Oliver

Re: [ANNOUNCE] udev 0.1 release

[Andrew Morton]

Oliver Neukum <oliver@neukum.org> wrote:
>
> 
> > The pipe/socket solution is probably better anyway, I was just
> > wondering why /sbin/hotplug wasn't serialized from the start.
> 
> It was. Deadlocks happened and the semaphore was removed.
> I don't remember details. They might be in the archives.
> 

register_netdevice() is called under rtnl_lock.  It calls
net_run_sbin_hotplug() which ends up waiting on ifconfig.  But
ifconfig needs rtnl_lock.

Re: [ANNOUNCE] udev 0.1 release

[Roman Zippel]

Hi,

On 11 Apr 2003, H. Peter Anvin wrote:

> LSB *has* a naming document.  The LSB spec clearly states that
> devices.txt produced by LANANA *is* the naming document.
> 
> John Cagle is your naming czar.

Um, I thought he is our device numbering czar and names are still picked 
by the driver authors?
Further this mail http://marc.theaimsgroup.com/?l=linux-kernel&m=101540151714636&w=2
only mentions Alan and Marcelo, will these device numbers be valid for 
2.6? Has Linus changed his mind in the meantime?

bye, Roman

Re: [ANNOUNCE] udev 0.1 release

[Alan Cox]

On Gwe, 2003-04-11 at 23:30, Steven Dake wrote:
> There is no "spec" that states this is a requirement, however, telecom 
> customers require the elapsed time from the time they request the disk 
> to be used, to the disk being usable by the operating system to be 20 msec.

That will be amusing. It takes 30 seconds for an IDE disk to go from
poweroff to online.  If its decided to have a quick powersaving nap
it'll take you longer than that too

Re: [ANNOUNCE] udev 0.1 release

[Havoc Pennington]

On Sat, Apr 12, 2003 at 01:07:21AM -0700, Greg KH wrote: 
> Oh, and to compare sizes, with udev linked against klibc (static link)
> it comes out to a whopping big 6004 bytes:
> $ size udev
>    text    data     bss     dec     hex filename
>    5572       4     392    5968    1750 udev
> 

If it isn't obvious (I guess it is), that's an apples and oranges
comparison - though udev being smaller than /bin/true is either bad
for /bin/true or pretty good for udev. ;-)

I would want to compare D-BUS to a CORBA implementation, DCOP, M-BUS,
SOAP, or something like that (though it's reasonably different from
all of those). It's about the same size as DCOP, and about half the
size of ORBit2 which is a small CORBA. Then there's a large CORBA like
MICO that uses several megabytes. No clue how big an M-BUS
implementation is.

The important thing though in my mind isn't the raw size comparison
but what the size is "spent" on - for D-BUS the size is partially
spent on avoiding dependencies (DCOP and ORBit rely on GLib/Qt for
many things), and robustness (thread locks, OOM handling, robust API,
unit testability, and careful input validation). There's also size
cost to the abstractions made: system vs. per-session bus, bus daemon
vs. one-to-one, network transport (tcp vs. unix domain vs. whatever),
authentication mechanism (cookies, socket credentials, kerberos), and
so forth.

Other IPC sofware that's about the same size spends its code size on
different things, it's all about the tradeoffs.

Havoc

Re: [ANNOUNCE] udev 0.1 release

[David Brownell]

Kevin P. Fleming wrote:
> 
> 	 All it takes 
> for the driver for a Fibre Channel host adapter to load, and enumerate 
> the devices it can see. In a matter of seconds many hundreds or 
> thousands of disk devices could be registered with the kernel.

Not the design center for the original "exec /sbin/hotplug" style
event delivery, no way!

A near-term option would be just to have some kind of throttle on
how many hotplug invocations are going on concurrently -- avoiding
process creation storms.  That should easily take Linux all the way
into the 2.7 series, for those who want to tread carefully in terms
of changing sysadmin technologies.

- Dave

Re: [ANNOUNCE] udev 0.1 release

[David Brownell]

Andrew Morton wrote:
> Steven Dake <sdake@mvista.com> wrote:
> 
>>A much better solution could be had by select()ing on a filehandle 
>>indicating when a new hotswap event is ready to be processed.  No races, 
>>no security issues, no performance issues.
> 
> 
> I must say that I've always felt this to be a better approach than the
> /sbin/hotplug callout.

Better in some environments, not all.  It's a tradeoff.

Me, I'd far rather allocate those hotplug resources on demand
than pre-allocate them into Yet Another Daemon.  For bounded
response-time systems, or for systems that are generating huge
event streams, I'd likely want more control though.


> Apart from the performance issue, it means that the kernel can buffer the
> "insertion" events which happen at boot-time discovery until the userspace
> handler attaches itself.

It could do that regardless of whether the event is eventually
delivered through some socket or by spawning a new process.

The way hotplug events are handled after some subsystem decides
to send them isn't written in stone.  They've been asynchronous
all through the 2.4 series, so queueing is semantically OK.

- Dave

Re: [ANNOUNCE] udev 0.1 release

[Arnd Bergmann]

Steven Dake wrote:

> I have not tried it, but dnotify should be generic and not fs specific.  
> If it doesn't work, it could be fixed to match other filesystem types...

It currently does not work for files that are modified by sysfs itself,
because the generic dnotify mechanism only works for file system operations
initiated by the user.

I have verified that adding calls to dnotify_parent() to sysfs_create and
sysfs_hash_and_remove makes it work. I'm not sure about the locking rules
there, so I'm not sending a patch but it should be trivial to do for
someone more familiar with the sysfs code.

	Arnd <><

Re: [ANNOUNCE] udev 0.1 release

[Oliver Neukum]

Am Samstag, 12. April 2003 00:36 schrieb Perez-Gonzalez, Inaky:
> > From: Oliver Neukum [mailto:oliver@neukum.org]
> >
> > > Ok, if you are worried about these kinds of things, then use the
> > > in-kernel devfs.  I'm not going to dispute that userspace faults can
> > > happen.
> >
> > Yes, in my oppinion putting such things into user space is stupid.
> > Your considerable talents would be better used to help Adam getting
> > his simplified devfs ready.
>
> Fixating naming policy in the kernel goes along the lines too;
> unless kdevfs gets the ability to be policy-configurable, it is no use.

Why? Who cares about names? IMHO that's useless frill.
If he absolutely cannot stand the kernel default, use a symlink.
You do not protest against numbering policy in the kernel,
why are names worse?

	Regards
		Oliver

RE: [ANNOUNCE] udev 0.1 release

[Perez-Gonzalez, Inaky]

> -----Original Message-----
> From: Steven Dake [mailto:sdake@mvista.com]
>
> Without blocking until the device is created and ready to be used, it
> becomes difficult to actually "hotswap insert" and then immediatly use
> the device, requiring polling.  Most users would like to wait for the
> event to complete, or have a select()able fd to wait on to know when the
> event has been completed.
> 
> This might be possible to emulate through dnotify, but would still
> require a rescan of the dev directory, causing poor performance. 

Would you take a look at the patch I just sent and tell me if something
along the lines of that would help? [the subject is Simple Kernel-User
Event Interface]

Iñaky Pérez-González -- Not speaking for Intel -- all opinions are my own
(and my fault)

Re: [ANNOUNCE] udev 0.1 release

[Joel Becker]

On Fri, Apr 11, 2003 at 01:59:48PM -0700, Greg KH wrote:
> You can store the default permissions in the database you use to store
> the naming data.  This solves the reboot problem, as long as you can
> convince people to not modify the permissions on their own (well even if
> they do, at shutdown, you can always validate that they are the same
> before you clean up the node.)

	There is no reason this can't live on ext3.  The permissions are
always right.  At device insertion, udev sees that the new device is
'disk0' and modifies /dev/disk0 to point to the new device (whatever
number).  Permissions are preserved.

Joel

-- 

"The whole principle is wrong; it's like demanding that grown men 
 live on skim milk because the baby can't eat steak."
        - author Robert A. Heinlein on censorship

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: joel.becker@oracle.com
Phone: (650) 506-8127

Re: [ANNOUNCE] udev 0.1 release

[Steven Dake]

Greg KH wrote:

>On Fri, Apr 11, 2003 at 04:27:36PM -0700, Steven Dake wrote:
>  
>
>>Hmm, I thought you were creating a tmpfs in /dev.  I think that 
>>particular case would allow dnotify to tell you when permissions and 
>>owners changed?
>>    
>>
>
>My bad, you are correct.  Sorry, got confused there.
>
>Hm, does dnotify show events on a ramfs or tmpfs partition?  Haven't
>tried that out before.
>  
>
I have not tried it, but dnotify should be generic and not fs specific.  
If it doesn't work, it could be fixed to match other filesystem types...

>thanks,
>
>greg k-h
>
>
>
>  
>

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 04:27:36PM -0700, Steven Dake wrote:
> Hmm, I thought you were creating a tmpfs in /dev.  I think that 
> particular case would allow dnotify to tell you when permissions and 
> owners changed?

My bad, you are correct.  Sorry, got confused there.

Hm, does dnotify show events on a ramfs or tmpfs partition?  Haven't
tried that out before.

thanks,

greg k-h

Re: [ANNOUNCE] udev 0.1 release

[Steven Dake]

Greg KH wrote:

>On Fri, Apr 11, 2003 at 03:32:57PM -0700, Steven Dake wrote:
>  
>
>>I've been thinking of how to solve this particular problem, and believe 
>>you could use dnotify in a daemon to track permission and ownership 
>>changes and store them in a backing database.  In fact, we do something 
>>similiar to this today.  This allows the user to use any type of 
>>application for changing permissions/owners, even syscalls directly 
>>without having to go "through" any sort of tracking database.
>>    
>>
>
>That would be cool.  But I think we need to add dnotify support to sysfs
>first :)
>
Hmm, I thought you were creating a tmpfs in /dev.  I think that 
particular case would allow dnotify to tell you when permissions and 
owners changed?

>
>thanks,
>
>greg k-h
>
>
>
>  
>

RE: [ANNOUNCE] udev 0.1 release

[Perez-Gonzalez, Inaky]

> From: Greg KH [mailto:greg@kroah.com]


> > > From: Greg KH [mailto:greg@kroah.com]
> > >
> > > But I can do a lot to prevent losses.  A lot of people around here
point
> > > to the old way PTX used to regenerate the device naming database on
the
> > > fly.  We could do that by periodically scanning sysfs to make sure we
> > > are keeping /dev in sync with what the system has physically present.
> > > That's one way, I'm sure there are others.
> >
> > This might be a tad over-simplification, but sysfs knows by heart when
> > anything is modified, because it goes through it's interface. If we
> > only care about, for example, devices, we could hook up into
> > device_create() [was this the name?]; line up in a queue all the
> > devices for which an plug/unplug event hasn't been delivered to user
> > space and create symlinks in /sysfs/hotplug-events/.
> >
> > Each entry in there is a symlink to the new device directory, named with
an
> > increasing integer for easy serialization. When the event is fully
> > processed, remove the entry from user space.
> 
> Um, how do you show a symlink to a device that is no long there when the
> device is removed?  :)

Broken link - removal event [my fault, should have explained];
still, this is kind of broken because of what you mention in
the next paragraph and because it limits you to just two types
of events - addition and removal.

I like better the having of a file where you can get events
from (that at the end goes with the select() thingie).

> In the end, it's a nice idea, but the current one is much simpler, and
> works today :)

Sure - let's just not forget it and build on top of it little
by little.

Iñaky Pérez-González -- Not speaking for Intel -- all opinions are my own
(and my fault)

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 03:32:57PM -0700, Steven Dake wrote:
> 
> I've been thinking of how to solve this particular problem, and believe 
> you could use dnotify in a daemon to track permission and ownership 
> changes and store them in a backing database.  In fact, we do something 
> similiar to this today.  This allows the user to use any type of 
> application for changing permissions/owners, even syscalls directly 
> without having to go "through" any sort of tracking database.

That would be cool.  But I think we need to add dnotify support to sysfs
first :)

thanks,

greg k-h

RE: [ANNOUNCE] udev 0.1 release

[Perez-Gonzalez, Inaky]

> -----Original Message-----
> From: Steven Dake [mailto:sdake@mvista.com]
> 
> There is no "spec" that states this is a requirement, however, telecom
> customers require the elapsed time from the time they request the disk
> to be used, to the disk being usable by the operating system to be 20
msec.

How do you qualify "request"?

Is it plug the cable? Insert the disk into the bay? 
Flip a switch on? Manually? Computer controlled?

I cannot think of a physical action to plug a disk to a system that
is going to take an amount of time small enough so that 20 msec is
not noise.

A computer controlled switch might make sense, assuming the disk is
already powered, spinning and ready to rock - still, I guess SCSI
would like to enumerate it ... dunno how that works.

Iñaky Pérez-González -- Not speaking for Intel -- all opinions are my own
(and my fault)

Re: [ANNOUNCE] udev 0.1 release

[David Lang]

you can only do this if you use a virtual directory for dev, otherwise you
have to do something like scanning /dev at shutdown and storing the
result (this was the famous devfs tar archive of permission issue)

you really ahve two situations.

1. a virtual filesystem for /dev.

  this lets you notice all changes to anything in /dev and store it
apropriatly

2. a real filesystem for /dev with a userspace daemon to update it.

  this involves the kernel less, but unless you can convince people to
stop managing their devices the old way you have the problem of missing
changes

David Lang

 On Fri, 11 Apr 2003, Steven Dake wrote:

> Date: Fri, 11 Apr 2003 15:32:57 -0700
> From: Steven Dake <sdake@mvista.com>
> To: Greg KH <greg@kroah.com>
> Cc: David Lang <david.lang@digitalinsight.com>,
>      "Perez-Gonzalez, Inaky" <inaky.perez-gonzalez@intel.com>,
>      'Jeremy Jackson' <jerj@coplanar.net>,
>      "'linux-kernel@vger.kernel.org'" <linux-kernel@vger.kernel.org>,
>      "'linux-hotplug-devel@lists.sourceforge.net'"
>     <linux-hotplug-devel@lists.sourceforge.net>
> Subject: Re: [ANNOUNCE] udev 0.1 release
>
>
>
> Greg KH wrote:
>
> >On Fri, Apr 11, 2003 at 01:48:17PM -0700, David Lang wrote:
> >
> >
> >>ant then you also have all the same problems as devfs about default
> >>permissions, making permissions persistant across reboots, etc.
> >>
> >>
> >
> >You can store the default permissions in the database you use to store
> >the naming data.  This solves the reboot problem, as long as you can
> >convince people to not modify the permissions on their own (well even if
> >they do, at shutdown, you can always validate that they are the same
> >before you clean up the node.)
> >
> >And provide an easy way for users to change the permissions so they show
> >up in the database.
> >
> >devchmod and devchown anyone?  :)
> >
> >
> Greg,
>
> I've been thinking of how to solve this particular problem, and believe
> you could use dnotify in a daemon to track permission and ownership
> changes and store them in a backing database.  In fact, we do something
> similiar to this today.  This allows the user to use any type of
> application for changing permissions/owners, even syscalls directly
> without having to go "through" any sort of tracking database.
>
> >thanks,
> >
> >greg k-h
> >-
> >To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> >the body of a message to majordomo@vger.kernel.org
> >More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >Please read the FAQ at  http://www.tux.org/lkml/
> >
> >
> >
> >
> >
>

RE: [ANNOUNCE] udev 0.1 release

[Perez-Gonzalez, Inaky]

> From: Oliver Neukum [mailto:oliver@neukum.org]
>
> > Ok, if you are worried about these kinds of things, then use the
> > in-kernel devfs.  I'm not going to dispute that userspace faults can
> > happen.
> 
> Yes, in my oppinion putting such things into user space is stupid.
> Your considerable talents would be better used to help Adam getting
> his simplified devfs ready.

Fixating naming policy in the kernel goes along the lines too; 
unless kdevfs gets the ability to be policy-configurable, it is no use.

Iñaky Pérez-González -- Not speaking for Intel -- all opinions are my own
(and my fault)

Re: [ANNOUNCE] udev 0.1 release

[Steven Dake]

Greg KH wrote:

>On Fri, Apr 11, 2003 at 01:48:17PM -0700, David Lang wrote:
>  
>
>>ant then you also have all the same problems as devfs about default
>>permissions, making permissions persistant across reboots, etc.
>>    
>>
>
>You can store the default permissions in the database you use to store
>the naming data.  This solves the reboot problem, as long as you can
>convince people to not modify the permissions on their own (well even if
>they do, at shutdown, you can always validate that they are the same
>before you clean up the node.)
>
>And provide an easy way for users to change the permissions so they show
>up in the database.
>
>devchmod and devchown anyone?  :)
>  
>
Greg,

I've been thinking of how to solve this particular problem, and believe 
you could use dnotify in a daemon to track permission and ownership 
changes and store them in a backing database.  In fact, we do something 
similiar to this today.  This allows the user to use any type of 
application for changing permissions/owners, even syscalls directly 
without having to go "through" any sort of tracking database.

>thanks,
>
>greg k-h
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>
>
>  
>

Re: [ANNOUNCE] udev 0.1 release

[Greg KH]

On Fri, Apr 11, 2003 at 01:48:17PM -0700, David Lang wrote:
> ant then you also have all the same problems as devfs about default
> permissions, making permissions persistant across reboots, etc.

You can store the default permissions in the database you use to store
the naming data.  This solves the reboot problem, as long as you can
convince people to not modify the permissions on their own (well even if
they do, at shutdown, you can always validate that they are the same
before you clean up the node.)

And provide an easy way for users to change the permissions so they show
up in the database.

devchmod and devchown anyone?  :)

thanks,

greg k-h

RE: [ANNOUNCE] udev 0.1 release

[David Lang]

ant then you also have all the same problems as devfs about default
permissions, making permissions persistant across reboots, etc.

David Lang

On Fri, 11 Apr 2003, Perez-Gonzalez, Inaky wrote:

> Date: Fri, 11 Apr 2003 13:42:58 -0700
> From: "Perez-Gonzalez, Inaky" <inaky.perez-gonzalez@intel.com>
> To: 'Jeremy Jackson' <jerj@coplanar.net>, 'Greg KH' <greg@kroah.com>
> Cc: "'linux-kernel@vger.kernel.org'" <linux-kernel@vger.kernel.org>,
>      "'linux-hotplug-devel@lists.sourceforge.net'"
>     <linux-hotplug-devel@lists.sourceforge.net>
> Subject: RE: [ANNOUNCE] udev 0.1 release
>
>
> > From: Jeremy Jackson [mailto:jerj@coplanar.net]
> >
> > What about read-only root fs?
>
> /dev on a tmpfs?
>
> Iñaky Pérez-González -- Not speaking for Intel -- all opinions are my own
> (and my fault)
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

RE: [ANNOUNCE] udev 0.1 release

[Perez-Gonzalez, Inaky]

> From: Jeremy Jackson [mailto:jerj@coplanar.net]
>
> What about read-only root fs?  

/dev on a tmpfs?

Iñaky Pérez-González -- Not speaking for Intel -- all opinions are my own
(and my fault)

Re: [ANNOUNCE] udev 0.1 release

[Michael Buesch]

On Friday 11 April 2003 20:50, Steve Lee wrote:
> Yes, 1998 3 into 1 splitters would be needed.

Yes, I think this is correct. :)
My previously posted 400 ment "400 to each
powersupply cable" (but I wasn't correct with it,
because as you said to some are only 399 connected.

PS: We should do something more productive than this. :)

Regards
Michael Buesch.

-- 
My homepage: http://www.8ung.at/tuxsoft
fighting for peace is like fu**ing for virginity

RE: [ANNOUNCE] udev 0.1 release

[Steve Lee]

Yes, 1998 3 into 1 splitters would be needed.  The break down is as
follows:

Three of the original power connectors would need 400 splitters, while
the remaining two would just need 399 splitters.  This would result in
4001 power connectors, or 4001 disks.  Because it's a Friday, here is a
simple program that verified my findings.

int main(int argc,char **argv)
{
    int con_a,con_b,con_c,con_d,con_e;
    int pow_a,pow_b,pow_c,pow_d,pow_e;

    con_a=con_b=con_c=con_d=con_e=0;
    pow_a=pow_b=pow_c=pow_d=pow_e=1;
    while(1) {
        con_a++;
        pow_a=con_a*3-(con_a==1?0:con_a-1);
        if(pow_a+pow_b+pow_c+pow_d+pow_e>=4000)
            break;
        con_b++;
        pow_b=con_b*3-(con_b==1?0:con_b-1);
        if(pow_a+pow_b+pow_c+pow_d+pow_e>=4000)
            break;
        con_c++;
        pow_c=con_c*3-(con_c==1?0:con_c-1);
        if(pow_a+pow_b+pow_c+pow_d+pow_e>=4000)
            break;
        con_d++;
        pow_d=con_d*3-(con_d==1?0:con_d-1);
        if(pow_a+pow_b+pow_c+pow_d+pow_e>=4000)
            break;
        con_e++;
        pow_e=con_e*3-(con_e==1?0:con_e-1);
        if(pow_a+pow_b+pow_c+pow_d+pow_e>=4000)
            break;
    }
    fprintf(stderr,"con_a = %d, pow_a = %d\n",con_a,pow_a);
    fprintf(stderr,"con_b = %d, pow_b = %d\n",con_b,pow_b);
    fprintf(stderr,"con_c = %d, pow_c = %d\n",con_c,pow_c);
    fprintf(stderr,"con_d = %d, pow_d = %d\n",con_d,pow_d);
    fprintf(stderr,"con_e = %d, pow_e = %d\n",con_e,pow_e);
    fprintf(stderr,"pow_a+pow_b+pow_c+pow_d+pow_e =
%d\n",pow_a+pow_b+pow_c+pow_d+pow_e);
    return 0;
}

-----Original Message-----
From: freesoftwaredeveloper@web.de [mailto:freesoftwaredeveloper@web.de]

Sent: Friday, April 11, 2003 1:18 PM
To: oliver@neukum.name
Cc: Steve Lee; Desmet_Jochen@emc.com
Subject: Re: [ANNOUNCE] udev 0.1 release

On Friday 11 April 2003 20:15, Oliver Neukum wrote:
> Am Freitag, 11. April 2003 20:03 schrieb Michael Buesch:
> > On Friday 11 April 2003 19:46, John Bradford wrote:
> > > [Puzzle]
> > > Say the power supply had five 5.25" drive power connecters, how
many 1
> > > into 3 power cable splitters would you need to connect all 4000
disks?

> You are hereby hit. All drivers need to be connected to a splitter.
> Thus any answer under 4000/3 must be wrong.

Yea, but: "Say the power supply had five 5.25" drive power connecters"
:)

-- 
My homepage: http://www.8ung.at/tuxsoft
fighting for peace is like fu**ing for virginity