RE: hotplug (was devfs)

RE: hotplug (was devfs)

[Ed Vance]

On Wed, November 13, 2002 at 10:45 AM, Greg KH wrote:
> On Wed, Nov 13, 2002 at 10:45:43AM -0800, Ed Vance wrote:
> > On Wed, November 13, 2002 at 10:05 AM, Greg KH wrote: 
> > > On Wed, Nov 13, 2002 at 06:06:06PM +0000, Nick Craig-Wood wrote:
> > > > 
> > > > So I'll be able to say usb bus1/1/4/1 port 3 should be 
> /dev/ttyUSB15
> > > > and it will always be that port?  That would be perfect.
> > > 
> > > Yes, that is the goal.
> > > 
> > 
> > Do you expect that goal to eventually be applied to 
> CompactPCI Hot-Swap
> > bus/slot port 3?
> 
> Yes, in a round-about way.  If the device that is in that 
> specific slot
> with that specific port, registers with, for example, the network
> subsystem, and we have decided that anything in that slot should be
> called "eth42", then we can do that based on the topology of 
> the device.
> 
> It really depends on the device that is existing in a 
> specific location
> (network, scsi, etc.) and not so much as the specific location will
> always be a network card called "ethX", as you have to look 
> at the type
> of device too.
> 
> Does that make sense?

Yes.
> 
> I can tell it's getting to be the time to start writing all 
> of this down
> for people to hash out... :)
> 

Very cool. 

Thanks,
Ed

---------------------------------------------------------------- 
Ed Vance              edv (at) macrolink (dot) com
Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
----------------------------------------------------------------

Re: hotplug (was devfs)

[Oliver Neukum]

> > So - perhaps hotplug ought to be serialised?
>
> No, it's not needed for this problem.  There has been talk of
> serializing stuff in userspace, which is the proper way to handle some
> of the remove before add was seen problems.

We may need some kind of load control.
The thought of firing up hundreds of hotplug scripts
simultaneously is not pretty.

	Regards
		Oliver

RE: hotplug (was devfs)

[Ed Vance]

On Wed, November 13, 2002 at 10:05 AM, Greg KH wrote: 
> 
> On Wed, Nov 13, 2002 at 06:06:06PM +0000, Nick Craig-Wood wrote:
> > 
> > So I'll be able to say usb bus1/1/4/1 port 3 should be /dev/ttyUSB15
> > and it will always be that port?  That would be perfect.
> 
> Yes, that is the goal.
> 

Do you expect that goal to eventually be applied to CompactPCI Hot-Swap
bus/slot port 3?

Thanks,
-Ed

---------------------------------------------------------------- 
Ed Vance              edv (at) macrolink (dot) com
Macrolink, Inc.       1500 N. Kellogg Dr  Anaheim, CA  92807
----------------------------------------------------------------

Re: hotplug (was devfs)

[Greg KH]

On Wed, Nov 13, 2002 at 10:45:43AM -0800, Ed Vance wrote:
> On Wed, November 13, 2002 at 10:05 AM, Greg KH wrote: 
> > 
> > On Wed, Nov 13, 2002 at 06:06:06PM +0000, Nick Craig-Wood wrote:
> > > 
> > > So I'll be able to say usb bus1/1/4/1 port 3 should be /dev/ttyUSB15
> > > and it will always be that port?  That would be perfect.
> > 
> > Yes, that is the goal.
> > 
> 
> Do you expect that goal to eventually be applied to CompactPCI Hot-Swap
> bus/slot port 3?

Yes, in a round-about way.  If the device that is in that specific slot
with that specific port, registers with, for example, the network
subsystem, and we have decided that anything in that slot should be
called "eth42", then we can do that based on the topology of the device.

It really depends on the device that is existing in a specific location
(network, scsi, etc.) and not so much as the specific location will
always be a network card called "ethX", as you have to look at the type
of device too.

Does that make sense?

I can tell it's getting to be the time to start writing all of this down
for people to hash out... :)

thanks,

greg k-h

Re: hotplug (was devfs)

[Nick Craig-Wood]

On Wed, Nov 13, 2002 at 09:02:04AM -0800, Greg KH wrote:
> On Wed, Nov 13, 2002 at 10:48:09AM +0000, Nick Craig-Wood wrote:
> > 
> > We fixed these problems by removing hotplug and loading the relevant
> > kernel modules in the correct order and voila a perfectly
> > deterministic order for the /dev/ttyUSBs with all devices initialised.
> 
> deterministic for you :)

Indeed!  It was deterministic in the sense of

1) I booted the machine 10 times and the devices came up in the same
order ;-)

2) the order of the devices was related to the usb topology, like this
(these are usb bus positions as noted by hub.c)

  1/1     <- hub1
  1/1/1   <- keyspan 1 /dev/ttyUSB0..3
  1/1/2   <- keyspan 2 /dev/ttyUSB4..7
  1/1/3   <- keyspan 3 /dev/ttyUSB8..11
  1/1/4   <- hub2
  1/1/4/1 <- keyspan 4 /dev/ttyUSB12..15
  1/1/4/2 <- keyspan 5 /dev/ttyUSB16..19
  1/1/4/3 <- keyspan 6 /dev/ttyUSB20..23

That seemed like a sensible order to me!

> What hotplug will do is allow you to assign a /dev entry to a specific
> device, so that you can go off of the topology, and not just the order
> in which the devices are found.  That is how this problem will be
> solved properly.

So I'll be able to say usb bus1/1/4/1 port 3 should be /dev/ttyUSB15
and it will always be that port?  That would be perfect.

> > Plugging in our USB bus with 24 devices on it does indeed produce a
> > mini-forkbomb effect ;-) (Especially since these Keyspan devices are
> > initialised twice - once without firmware and once with firmware.)
> > 
> > So - perhaps hotplug ought to be serialised?
> 
> No, it's not needed for this problem.  There has been talk of
> serializing stuff in userspace, which is the proper way to handle some
> of the remove before add was seen problems.

Userspace serialisation would have solved this problem for us too I
think without the extra mapping mechanism.

-- 
Nick Craig-Wood
ncw1@axis.demon.co.uk

Re: hotplug (was devfs)

[Greg KH]

On Wed, Nov 13, 2002 at 06:06:06PM +0000, Nick Craig-Wood wrote:
> 
> So I'll be able to say usb bus1/1/4/1 port 3 should be /dev/ttyUSB15
> and it will always be that port?  That would be perfect.

Yes, that is the goal.

thanks,

greg k-h

Re: hotplug (was devfs)

[Greg KH]

On Wed, Nov 13, 2002 at 10:48:09AM +0000, Nick Craig-Wood wrote:
> 
> We fixed these problems by removing hotplug and loading the relevant
> kernel modules in the correct order and voila a perfectly
> deterministic order for the /dev/ttyUSBs with all devices initialised.

deterministic for you :)

What hotplug will do is allow you to assign a /dev entry to a specific
device, so that you can go off of the topology, and not just the order
in which the devices are found.  That is how this problem will be
solved properly.

> Plugging in our USB bus with 24 devices on it does indeed produce a
> mini-forkbomb effect ;-) (Especially since these Keyspan devices are
> initialised twice - once without firmware and once with firmware.)
> 
> So - perhaps hotplug ought to be serialised?

No, it's not needed for this problem.  There has been talk of
serializing stuff in userspace, which is the proper way to handle some
of the remove before add was seen problems.

thanks,

greg k-h

Re: hotplug (was devfs)

[Nick Craig-Wood]

On Tue, Nov 12, 2002 at 01:51:08PM +0100, Oliver Neukum wrote:
> > Actually, here's a question: are /sbin/hotplug upcalls serialized in
> > some fashion?  I'd hate to online a thousand devices in my disk array
> > and have the machine forkbomb itself.
> 
> Nope, no serialisation. You don't have any guarantee even that
> addition will be delivered before removal.

And that is why (we finally discovered) we were getting
non-deterministic device numbering of USB nodes.

We have machines with 6 x 4 port USB <-> serial converters attached.
These would get randomly assigned usb device ids and hence random
/dev/ttyUSB nodes.  Not very useful when there is a load of different
things attached to the 24 serial ports!

Sometimes we also found that one of the devices wouldn't get
initialised properly.

We fixed these problems by removing hotplug and loading the relevant
kernel modules in the correct order and voila a perfectly
deterministic order for the /dev/ttyUSBs with all devices initialised.

Plugging in our USB bus with 24 devices on it does indeed produce a
mini-forkbomb effect ;-) (Especially since these Keyspan devices are
initialised twice - once without firmware and once with firmware.)

So - perhaps hotplug ought to be serialised?

-- 
Nick Craig-Wood
ncw1@axis.demon.co.uk