linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
@ 2002-10-14 18:42 Steven Dake
  2002-10-15  0:59 ` Greg KH
  2002-10-15  5:29 ` Greg KH
  0 siblings, 2 replies; 17+ messages in thread
From: Steven Dake @ 2002-10-14 18:42 UTC (permalink / raw)
  To: linux-kernel

lkml,

http://www.sourceforge.net/project/showfiles.php?group_id=64580

I am announcing a sourceforge project for developing support in Linux 
kernel for Advanced TCA (PICMG 3.0) architecture.  Advanced TCA is a 
technology where boards exist in a chassis and can either be processor 
nodes or storage nodes.  All boards in the chassis are connected by 
FibreChannel and Ethernet.  The blades can be hot added or hot removed 
while the Linux processor nodes are active, meaning, that the SCSI 
subsystem must add devices on insertion request and remove devices on 
ejection requests.  Further the typical /dev/sda naming of devices is 
not appropriate since device nodes can change depending on the insertion 
order of disks.

These patches are for Linux 2.4.19 and work with the Qlogic 2300 
FibreChannel driver and at this point mostly support hotswap of the disk 
subsystem.

The patches consist of a SCSI hotswap patch for 2.4 kernel and QLA 2300 
support.

The patches also consist of a GA Mapper library which maps fibrechannel 
WWNs to devices in devfs filesystems such as /dev/scsi/chassis/slot/disc.

The sourceforge project also contains a userland library for each patch 
and userland applications such that these operations can be scripted.

Advanced TCA uses IPMI, so this project will use the IPMI driver being 
developed by Corey Minyard.

To be posted are userland or kernel hotswap managers.  I've not decided 
how to do this yet, so I'll post the bits when they are done.

I welcome comments questions or code to be added to the sourceforge project.

Thanks,
-steve


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-14 18:42 [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel Steven Dake
@ 2002-10-15  0:59 ` Greg KH
  2002-10-15 18:39   ` Steven Dake
  2002-10-15  5:29 ` Greg KH
  1 sibling, 1 reply; 17+ messages in thread
From: Greg KH @ 2002-10-15  0:59 UTC (permalink / raw)
  To: Steven Dake; +Cc: linux-kernel

On Mon, Oct 14, 2002 at 11:42:15AM -0700, Steven Dake wrote:
> lkml,
> 
> http://www.sourceforge.net/project/showfiles.php?group_id=64580
> 
> I am announcing a sourceforge project for developing support in Linux 
> kernel for Advanced TCA (PICMG 3.0) architecture.  Advanced TCA is a 
> technology where boards exist in a chassis and can either be processor 
> nodes or storage nodes.  All boards in the chassis are connected by 
> FibreChannel and Ethernet.  The blades can be hot added or hot removed 
> while the Linux processor nodes are active, meaning, that the SCSI 
> subsystem must add devices on insertion request and remove devices on 
> ejection requests.  Further the typical /dev/sda naming of devices is 
> not appropriate since device nodes can change depending on the insertion 
> order of disks.
> 
> These patches are for Linux 2.4.19 and work with the Qlogic 2300 
> FibreChannel driver and at this point mostly support hotswap of the disk 
> subsystem.

Some questions:
	- is there a public spec for this architecture?
	- are you going to be generating a 2.5 version of this so that
	  this feature can be added to the main kernel tree?
	- Why don't you use the existing kernel way of notifying
	  userspace of hotplug events, through /sbin/hotplug?
	- You create a lot of new ioctls, which is not nice.  You should
	  probably do what was done for the pci hotplug subsystem, and
	  create a ram based filesystem for this subsystem.  That way
	  you don't need to have a /dev node, and the userspace tools
	  become dirt simple.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-14 18:42 [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel Steven Dake
  2002-10-15  0:59 ` Greg KH
@ 2002-10-15  5:29 ` Greg KH
  2002-10-15 17:38   ` Steven Dake
  1 sibling, 1 reply; 17+ messages in thread
From: Greg KH @ 2002-10-15  5:29 UTC (permalink / raw)
  To: Steven Dake; +Cc: linux-kernel

On Mon, Oct 14, 2002 at 11:42:15AM -0700, Steven Dake wrote:
> 
> I welcome comments questions or code to be added to the sourceforge project.

Hi,

Here's some specific comments on your different patches.

--- linux-2.4.19/Documentation/Configure.help	Fri Aug  2 17:39:42 2002
+++ linux-2.4.19-gamap/Documentation/Configure.help	Mon Oct 14 11:16:22 2002
@@ -6795,6 +6795,19 @@
   module if your root file system (the one containing the directory /)
   is located on a SCSI device.
 
+Geographical Address Mapping support
+CONFIG_GAMAP
+  This driver will provide device filesystem dynamic mapping of WWNs
+  to their logical geographical address.  The result is that block
+  storage devices can be accessed using:
+
+      /dev/scsi/chassis/slot/host
+
+  This feature also works after an fdisk updating the appropriate files
+  to match the state of the system.
+
+  This feature is only supported by the Qlogic V6 driver.
+

In looking at this patch (and the others) it looks like you are relying
on devfs being in the system.  Is this true?  What about the other 99%
of machines out there with devfs disabled?

diff -uNr linux-2.4.19/Documentation/devices.txt linux-2.4.19-gamap/Documentation/devices.txt
--- linux-2.4.19/Documentation/devices.txt	Wed Nov  7 15:46:01 2001
+++ linux-2.4.19-gamap/Documentation/devices.txt	Mon Oct 14 11:06:00 2002
@@ -417,8 +417,8 @@
 		218 = /dev/kchuid	Inter-process chuid control
 		219 = /dev/modems/mwave	MWave modem firmware upload
 		220 = /dev/mptctl	Message passing technology (MPT) control
-		221 = /dev/mvista/hssdsi	Montavista PICMG hot swap system driver
-		222 = /dev/mvista/hasi		Montavista PICMG high availability
+		221 = /dev/scsi/hotswap	MontaVista SCSI/FC hotswap driver
+		222 = /dev/scsi/gamap	MontaVista SCSI/FC geographical address mapping driver
 		240-255			Reserved for local use
 
  11 char	Raw keyboard device

You are re-using minors that have previously been reserved.  Does this
mean Montavista is dropping their PICMG patches?

diff -uNr linux-2.4.19/drivers/scsi/gamap.c linux-2.4.19-gamap/drivers/scsi/gamap.c
--- linux-2.4.19/drivers/scsi/gamap.c	Wed Dec 31 17:00:00 1969
+++ linux-2.4.19-gamap/drivers/scsi/gamap.c	Mon Oct 14 11:06:00 2002
<snip>
+#define __KERNEL_SYSCALLS__
+
+#include <linux/unistd.h>

Any reason for this?  I didn't see any internal syscalls being made, but
I might have missed them.

+int ioctl_gamap_getga_from_fc_wwn (unsigned long parameters);
+int ioctl_gamap_getwwn_from_ga (unsigned long parameters);
+int ioctl_gamap_insert_by_scsi_id (unsigned long parameters);
+int ioctl_gamap_remove_by_scsi_id (unsigned long parameters);
+int ioctl_gamap_insert_by_fc_wwn (unsigned long parameters);
+int ioctl_gamap_remove_by_fc_wwn (unsigned long parameters);

These should all be static (along with being a little shorter).

+int gamap_insert_by_fc_wwn (int chassis, int slot, unsigned long long wwn) {
+int i = 0;
+struct Scsi_Host *hba_p;
+Scsi_Device *scsi_device;
+int host, channel, lun, id;
+char devname[32];
+struct gendisk *gendisk;
+int part;
+
+	/*
+	 * Ensure entry not already present in map
+	 * chassis and slot are a match, wwn is seperate match
+	 */

This style is all through the patches.  It is a SCSI way of defining
local variables?  I'd really recommend indenting them to follow the rest
of the kernel style.

+static int gamap_ioctl (struct inode *inode,
+	struct file *file,
+	unsigned int cmd,
+	unsigned long parameters) {
+
+int result = -EINVAL;
+
+	switch (cmd) {

So you let any user do an ioctl command on the device (this goes for the
other patch too.)  You should restrict this to CAP_SYS_ADMIN users only.

All of these ioctl commands look like they can easily be done through a
ramfs like interface.  

+EXPORT_SYMBOL (gamap_getga_from_fc_wwn);
+EXPORT_SYMBOL (gamap_getwwn_from_ga);
+EXPORT_SYMBOL (gamap_insert_by_scsi_id);
+EXPORT_SYMBOL (gamap_remove_by_scsi_id);
+EXPORT_SYMBOL (gamap_insert_by_fc_wwn);
+EXPORT_SYMBOL (gamap_remove_by_fc_wwn);

I don't see any other code using these functions.  Is there a need to
export them?

diff -uNr linux-2.4.19/fs/partitions/check.c linux-2.4.19-gamap/fs/partitions/check.c
--- linux-2.4.19/fs/partitions/check.c	Fri Aug  2 17:39:45 2002
+++ linux-2.4.19-gamap/fs/partitions/check.c	Mon Oct 14 11:06:21 2002
@@ -334,6 +334,35 @@
 }
 #endif  /*  CONFIG_DEVFS_FS  */
 
+#ifdef CONFIG_GAMAP
+static void devfs_gamap_register_partition (struct gendisk *gendisk, int minor, int part) {
+char devname[16];

You shouldn't use a #ifdef within this .c file.  I think you could move
it to your specific file, and then use #ifdef within a .h file.  This
also goes for your other change to this file.

diff -uNr linux-2.4.19/include/linux/genhd.h linux-2.4.19-gamap/include/linux/genhd.h
--- linux-2.4.19/include/linux/genhd.h	Fri Aug  2 17:39:45 2002
+++ linux-2.4.19-gamap/include/linux/genhd.h	Mon Oct 14 11:14:20 2002
@@ -63,6 +63,8 @@
 	unsigned long nr_sects;
 	devfs_handle_t de;              /* primary (master) devfs entry  */
 	int number;                     /* stupid old code wastes space  */
+	devfs_handle_t de_gamap;	/* ga map device entry */
+
 
 	/* Performance stats: */
 	unsigned int ios_in_flight;
@@ -98,6 +100,8 @@
 	struct gendisk *next;
 	struct block_device_operations *fops;
 
+	devfs_handle_t de_gadir;	/* GA device entry directory chassis/slot/host */
+	devfs_handle_t de_gamap[17];	/* for GA Mapping, need 17 entries  (disc + 16 parts ) */
 	devfs_handle_t *de_arr;         /* one per physical disc */
 	char *flags;                    /* one per physical disc */
 };


Ouch, do you really need all of these handles?

As the gendisk interface has been cleaned up a _lot_ in 2.5, I'm not so
sure how well this implementation will now work.

Pretty much the same comments apply for your scsi-hotswap-main.patch
(devfs reliance, coding style, loads of ioctls, long ioctl function
names, global functions that don't need to be, etc.)

I also noticed that this code is included in the CGL CVS tree.  What is
MontaVista's (and yours) future plans for this feature?  Do you want it
in the main kernel tree, and are you willing to port it to 2.5?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15  5:29 ` Greg KH
@ 2002-10-15 17:38   ` Steven Dake
  2002-10-15 19:11     ` Michael Clark
  2002-10-15 20:52     ` Greg KH
  0 siblings, 2 replies; 17+ messages in thread
From: Steven Dake @ 2002-10-15 17:38 UTC (permalink / raw)
  To: Greg KH; +Cc: linux-kernel

Greg,

Thanks for the comments.

Responses below.

Greg KH wrote:

>On Mon, Oct 14, 2002 at 11:42:15AM -0700, Steven Dake wrote:
>  
>
>>I welcome comments questions or code to be added to the sourceforge project.
>>    
>>
>
>Hi,
>
>Here's some specific comments on your different patches.
>
>--- linux-2.4.19/Documentation/Configure.help	Fri Aug  2 17:39:42 2002
>+++ linux-2.4.19-gamap/Documentation/Configure.help	Mon Oct 14 11:16:22 2002
>@@ -6795,6 +6795,19 @@
>   module if your root file system (the one containing the directory /)
>   is located on a SCSI device.
> 
>+Geographical Address Mapping support
>+CONFIG_GAMAP
>+  This driver will provide device filesystem dynamic mapping of WWNs
>+  to their logical geographical address.  The result is that block
>+  storage devices can be accessed using:
>+
>+      /dev/scsi/chassis/slot/host
>+
>+  This feature also works after an fdisk updating the appropriate files
>+  to match the state of the system.
>+
>+  This feature is only supported by the Qlogic V6 driver.
>+
>
>In looking at this patch (and the others) it looks like you are relying
>on devfs being in the system.  Is this true?  What about the other 99%
>of machines out there with devfs disabled?
>  
>
Hopefully these machines that have devfs disabled aren't ATCA machines 
:)  It is true that devfs is REQUIRED for the GA Mapping.  It isn't 
required for the scsi hotswap feature.  The reason it is required for 
gamapping is that devices must be defineable at runtime during hot swap 
events and /dev/sda is inappropriate for several reasons as named access 
to a device.  I can't see any other way to do it, although I could 
provide some library in userspace that translates /dev/sda into 
chassis/slots.  The downside of this technique is now the user must use 
some utility to find out which device to access when using fdisk, etc 
instead of just using the device directly.

>diff -uNr linux-2.4.19/Documentation/devices.txt linux-2.4.19-gamap/Documentation/devices.txt
>--- linux-2.4.19/Documentation/devices.txt	Wed Nov  7 15:46:01 2001
>+++ linux-2.4.19-gamap/Documentation/devices.txt	Mon Oct 14 11:06:00 2002
>@@ -417,8 +417,8 @@
> 		218 = /dev/kchuid	Inter-process chuid control
> 		219 = /dev/modems/mwave	MWave modem firmware upload
> 		220 = /dev/mptctl	Message passing technology (MPT) control
>-		221 = /dev/mvista/hssdsi	Montavista PICMG hot swap system driver
>-		222 = /dev/mvista/hasi		Montavista PICMG high availability
>+		221 = /dev/scsi/hotswap	MontaVista SCSI/FC hotswap driver
>+		222 = /dev/scsi/gamap	MontaVista SCSI/FC geographical address mapping driver
> 		240-255			Reserved for local use
> 
>  11 char	Raw keyboard device
>
>You are re-using minors that have previously been reserved.  Does this
>mean Montavista is dropping their PICMG patches?
>  
>
I had requested these minors in our rev1 of the PICMG 2.12 hotswap 
patches.  During a rework of the picmg patches another developer dropped 
the minors and nobody is using the old code, so I thought I could reuse 
them.  This is probably bad mojo, but I think its ok :)

>diff -uNr linux-2.4.19/drivers/scsi/gamap.c linux-2.4.19-gamap/drivers/scsi/gamap.c
>--- linux-2.4.19/drivers/scsi/gamap.c	Wed Dec 31 17:00:00 1969
>+++ linux-2.4.19-gamap/drivers/scsi/gamap.c	Mon Oct 14 11:06:00 2002
><snip>
>+#define __KERNEL_SYSCALLS__
>+
>+#include <linux/unistd.h>
>
>Any reason for this?  I didn't see any internal syscalls being made, but
>I might have missed them.
>  
>
thanks i'll remove it.  sloppy on my part.

>+int ioctl_gamap_getga_from_fc_wwn (unsigned long parameters);
>+int ioctl_gamap_getwwn_from_ga (unsigned long parameters);
>+int ioctl_gamap_insert_by_scsi_id (unsigned long parameters);
>+int ioctl_gamap_remove_by_scsi_id (unsigned long parameters);
>+int ioctl_gamap_insert_by_fc_wwn (unsigned long parameters);
>+int ioctl_gamap_remove_by_fc_wwn (unsigned long parameters);
>
>These should all be static (along with being a little shorter).
>  
>
thanks i'll fix.

>+int gamap_insert_by_fc_wwn (int chassis, int slot, unsigned long long wwn) {
>+int i = 0;
>+struct Scsi_Host *hba_p;
>+Scsi_Device *scsi_device;
>+int host, channel, lun, id;
>+char devname[32];
>+struct gendisk *gendisk;
>+int part;
>+
>+	/*
>+	 * Ensure entry not already present in map
>+	 * chassis and slot are a match, wwn is seperate match
>+	 */
>
>This style is all through the patches.  It is a SCSI way of defining
>local variables?  I'd really recommend indenting them to follow the rest
>of the kernel style.
>  
>
Ok thanks will do.

>+static int gamap_ioctl (struct inode *inode,
>+	struct file *file,
>+	unsigned int cmd,
>+	unsigned long parameters) {
>+
>+int result = -EINVAL;
>+
>+	switch (cmd) {
>
>So you let any user do an ioctl command on the device (this goes for the
>other patch too.)  You should restrict this to CAP_SYS_ADMIN users only.
>  
>
I thought about this, and determined that access control can happen 
through permissions on the device.  I'll probably add something like 
this though.  We definately don't want typical users hotswap removing a 
device!

>All of these ioctl commands look like they can easily be done through a
>ramfs like interface.  
>
>+EXPORT_SYMBOL (gamap_getga_from_fc_wwn);
>+EXPORT_SYMBOL (gamap_getwwn_from_ga);
>+EXPORT_SYMBOL (gamap_insert_by_scsi_id);
>+EXPORT_SYMBOL (gamap_remove_by_scsi_id);
>+EXPORT_SYMBOL (gamap_insert_by_fc_wwn);
>+EXPORT_SYMBOL (gamap_remove_by_fc_wwn);
>
>I don't see any other code using these functions.  Is there a need to
>export them?
>  
>
At this point, there isn't anything using them.  I am working on a 
hotswap manager, that may be in kernel space (for performance reasons) 
that may use these interfaces.  I'm also working on a SAFTE Hotswap 
processor module (ie; drivers/scsi/sp.c) for the SCSI subsystem that 
uses these interfaces.  (Safte is a hotswap standard for SCSI chassis).

>diff -uNr linux-2.4.19/fs/partitions/check.c linux-2.4.19-gamap/fs/partitions/check.c
>--- linux-2.4.19/fs/partitions/check.c	Fri Aug  2 17:39:45 2002
>+++ linux-2.4.19-gamap/fs/partitions/check.c	Mon Oct 14 11:06:21 2002
>@@ -334,6 +334,35 @@
> }
> #endif  /*  CONFIG_DEVFS_FS  */
> 
>+#ifdef CONFIG_GAMAP
>+static void devfs_gamap_register_partition (struct gendisk *gendisk, int minor, int part) {
>+char devname[16];
>
>You shouldn't use a #ifdef within this .c file.  I think you could move
>it to your specific file, and then use #ifdef within a .h file.  This
>also goes for your other change to this file.
>  
>
I'm not so sure about this.  The reason this file is patched is to 
support things like fdisk changes (rexporting partitions after an fdisk) 
and initial setup.  I could provide an "alternate" check.c but would 
rather not have to maintain the bug fixes from check.c to check_alt.c. 
 devfs_gamap_register_partition could be in a seperate file, but 
devfs_register_partition is a generic function used by other parts of 
the kernel that needs ifdef patches (or two seperate implementations).

Would two seperate implementations one in check.c (unpatched) and one in 
gamap.c (GAMAP version) with the implementation in check.c ifndef'ed out 
if GAMAP is defined be more palatable?

>diff -uNr linux-2.4.19/include/linux/genhd.h linux-2.4.19-gamap/include/linux/genhd.h
>--- linux-2.4.19/include/linux/genhd.h	Fri Aug  2 17:39:45 2002
>+++ linux-2.4.19-gamap/include/linux/genhd.h	Mon Oct 14 11:14:20 2002
>@@ -63,6 +63,8 @@
> 	unsigned long nr_sects;
> 	devfs_handle_t de;              /* primary (master) devfs entry  */
> 	int number;                     /* stupid old code wastes space  */
>+	devfs_handle_t de_gamap;	/* ga map device entry */
>+
> 
> 	/* Performance stats: */
> 	unsigned int ios_in_flight;
>@@ -98,6 +100,8 @@
> 	struct gendisk *next;
> 	struct block_device_operations *fops;
> 
>+	devfs_handle_t de_gadir;	/* GA device entry directory chassis/slot/host */
>+	devfs_handle_t de_gamap[17];	/* for GA Mapping, need 17 entries  (disc + 16 parts ) */
> 	devfs_handle_t *de_arr;         /* one per physical disc */
> 	char *flags;                    /* one per physical disc */
> };
>
>
>Ouch, do you really need all of these handles?
>  
>
maybe not.  I think there is still a little bit of slop in the patches. 
 As you can tell, I've not code reviewed the driver yet (thats what you 
guys are for :) and there are some things that need improvement.

>As the gendisk interface has been cleaned up a _lot_ in 2.5, I'm not so
>sure how well this implementation will now work.
>
>Pretty much the same comments apply for your scsi-hotswap-main.patch
>(devfs reliance, coding style, loads of ioctls, long ioctl function
>names, global functions that don't need to be, etc.)
>  
>
the scsi-hotswap-main patch can use devfs, but shouldn't rely on it. 
 Ioctls are only to provide user space access to the kernel features. 
 global functions, as in global in the symbol table (via EXPORT_SYMBOL?)

Thanks for the comments i"ll try to apply to the scsi-hotswap patches as 
well.

>I also noticed that this code is included in the CGL CVS tree.  What is
>MontaVista's (and yours) future plans for this feature?  Do you want it
>in the main kernel tree, and are you willing to port it to 2.5?
>  
>
I'd love to see ATCA support in the main kernel (if it would be 
accepted). Maintaining patches against 2.4 is ok (since it has slowed 
down to bug fixes mostly).   I really don't want to maintain these 
patches against every minor release of 2.5 for now until 2.6 is 
stabilized if I can help it.  I think these sorts of things belong in 
the kernel, atleast the SCSI hotswap patches.

These patches also depend on the Qlogic QLA 2300 driver which isn't in 
the kernel.  Would this also be included?  We can talk to the QLogic 
guys and see if they want to submit their driver to the 2.5 trees...

If you can suggest a route for getting these patches accepted into the 
2.5 trees (beyond a port to 2.5 and the above suggested changes) i'd 
love to hear them.

>thanks,
>
>greg k-h
>  
>
thanks
-steve

>
>
>  
>


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15  0:59 ` Greg KH
@ 2002-10-15 18:39   ` Steven Dake
  2002-10-15 20:42     ` Greg KH
  0 siblings, 1 reply; 17+ messages in thread
From: Steven Dake @ 2002-10-15 18:39 UTC (permalink / raw)
  To: Greg KH; +Cc: linux-kernel

Greg,

Responses below.

Greg KH wrote:

>On Mon, Oct 14, 2002 at 11:42:15AM -0700, Steven Dake wrote:
>  
>
>>lkml,
>>
>>http://www.sourceforge.net/project/showfiles.php?group_id=64580
>>
>>I am announcing a sourceforge project for developing support in Linux 
>>kernel for Advanced TCA (PICMG 3.0) architecture.  Advanced TCA is a 
>>technology where boards exist in a chassis and can either be processor 
>>nodes or storage nodes.  All boards in the chassis are connected by 
>>FibreChannel and Ethernet.  The blades can be hot added or hot removed 
>>while the Linux processor nodes are active, meaning, that the SCSI 
>>subsystem must add devices on insertion request and remove devices on 
>>ejection requests.  Further the typical /dev/sda naming of devices is 
>>not appropriate since device nodes can change depending on the insertion 
>>order of disks.
>>
>>These patches are for Linux 2.4.19 and work with the Qlogic 2300 
>>FibreChannel driver and at this point mostly support hotswap of the disk 
>>subsystem.
>>    
>>
>
>Some questions:
>	- is there a public spec for this architecture?
>  
>
The spec hasn't ratified yet and I don't have a copy (I only have 
pre-spec hardware).  I think distribution is limited to PICMG members 
once a spec is available, but I'm not sure.  Who needs specs anyway :)

>	- are you going to be generating a 2.5 version of this so that
>	  this feature can be added to the main kernel tree?
>
If you think it would be accepted, I'd spend the time making 2.5 kernel 
patches.  Beyond your other comments, any suggestions to get it accepted?

>	- Why don't you use the existing kernel way of notifying
>	  userspace of hotplug events, through /sbin/hotplug?
>
The hotplug events occur through IPMI (a system management interface 
specification) messages.  I'm not sure if the hotswap manager will go in 
the kernel or not, but if it were in the kernel, it could use 
/sbin/hotplug to notify management software of hotswap events (which 
would allow the scsi hotswap commands to be used to add or remove 
devices).  Initially I am going to probably do a user space manager 
since its simpler and I think that sort of thing probably belongs in 
user space.  It will access the IPMI driver, read hotswap events from 
the IPMI driver, and swap in and out devices and map/unmap devices via 
the ga mapper.

Perhaps what is really needed is a kernel driver that uses the IPMI 
driver kernel interface to pump disk device hotswap messages through 
/sbin/hotplug.  After I get a userspace implementation working (which is 
easier to debug and test) I can start work on something like this.  What 
would you think of that?  The nice thing about using /sbin/hotplug is 
more things can be scripted like automatically removing a MD disk if the 
hotswapped device is part of an MD device.

I've not started on this component yet and am just figuring out the IPMI 
messaging at this point.  Any comments you have on how to best integrate 
this into the current hotplug system would be highly welcomed.

>	- You create a lot of new ioctls, which is not nice.  You should
>	  probably do what was done for the pci hotplug subsystem, and
>	  create a ram based filesystem for this subsystem.  That way
>	  you don't need to have a /dev node, and the userspace tools
>	  become dirt simple.
>  
>
I'll have to look at that.  I'm not familiar with the ram based 
filesystem.  Could you point me to a source file that uses some of the 
interfaces?

Thanks
-steve

>thanks,
>
>greg k-h
>
>
>
>  
>


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15 17:38   ` Steven Dake
@ 2002-10-15 19:11     ` Michael Clark
  2002-10-15 19:28       ` Steven Dake
  2002-10-15 20:52     ` Greg KH
  1 sibling, 1 reply; 17+ messages in thread
From: Michael Clark @ 2002-10-15 19:11 UTC (permalink / raw)
  To: Steven Dake; +Cc: Greg KH, linux-kernel

On 10/16/02 01:38, Steven Dake wrote:

> At this point, there isn't anything using them.  I am working on a 
> hotswap manager, that may be in kernel space (for performance reasons) 
> that may use these interfaces.  I'm also working on a SAFTE Hotswap 
> processor module (ie; drivers/scsi/sp.c) for the SCSI subsystem that 
> uses these interfaces.  (Safte is a hotswap standard for SCSI chassis).

Do you really want to have SAF-TE polling in the kernel?

This can easily be accomplished in userspace using sg.

safte-monitor <http://gort.metaparadigm.com/safte-monitor/> can already
provde disk insertion, removal notifications in userspace and already
supports calling out to a script with the physical slot location information
(and with tweaks to the code, scsi device of the disk inserted)

There is even code in safte-monitor to identify the wwn of the devices
in each slot, although it needs updating to the latest qlogic ioctl
interface (or hbaapi).

~mc


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15 19:11     ` Michael Clark
@ 2002-10-15 19:28       ` Steven Dake
  2002-10-15 20:34         ` Greg KH
  0 siblings, 1 reply; 17+ messages in thread
From: Steven Dake @ 2002-10-15 19:28 UTC (permalink / raw)
  To: Michael Clark; +Cc: Greg KH, linux-kernel


Michael Clark wrote:

> On 10/16/02 01:38, Steven Dake wrote:
>
>> At this point, there isn't anything using them.  I am working on a 
>> hotswap manager, that may be in kernel space (for performance 
>> reasons) that may use these interfaces.  I'm also working on a SAFTE 
>> Hotswap processor module (ie; drivers/scsi/sp.c) for the SCSI 
>> subsystem that uses these interfaces.  (Safte is a hotswap standard 
>> for SCSI chassis).
>
>
> Do you really want to have SAF-TE polling in the kernel?

Good code thanks for the pointer.

When I searched months ago, there wasn't anything out there.

Safte polling in the kernel isn't inherently bad and could be tied into 
the hotplug mechanism.

Making SAFTE hotswap available via SG would also work but system 
performance would be bad at small poll intervals (like 100 msec).

Thanks
-steve

>
> This can easily be accomplished in userspace using sg.
>
> safte-monitor <http://gort.metaparadigm.com/safte-monitor/> can already
> provde disk insertion, removal notifications in userspace and already
> supports calling out to a script with the physical slot location 
> information
> (and with tweaks to the code, scsi device of the disk inserted)
>
> There is even code in safte-monitor to identify the wwn of the devices
> in each slot, although it needs updating to the latest qlogic ioctl
> interface (or hbaapi).
>
> ~mc
>
>
>


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15 19:28       ` Steven Dake
@ 2002-10-15 20:34         ` Greg KH
  2002-10-15 20:46           ` Steven Dake
  0 siblings, 1 reply; 17+ messages in thread
From: Greg KH @ 2002-10-15 20:34 UTC (permalink / raw)
  To: Steven Dake; +Cc: Michael Clark, linux-kernel

On Tue, Oct 15, 2002 at 12:28:59PM -0700, Steven Dake wrote:
> 
> 
> Safte polling in the kernel isn't inherently bad and could be tied into 
> the hotplug mechanism.
> 
> Making SAFTE hotswap available via SG would also work but system 
> performance would be bad at small poll intervals (like 100 msec).

Is there a real nead to get hotplug notification any faster than that?

And yes, it should all be done in userspace, whenever possible :)

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15 18:39   ` Steven Dake
@ 2002-10-15 20:42     ` Greg KH
  0 siblings, 0 replies; 17+ messages in thread
From: Greg KH @ 2002-10-15 20:42 UTC (permalink / raw)
  To: Steven Dake; +Cc: linux-kernel

On Tue, Oct 15, 2002 at 11:39:02AM -0700, Steven Dake wrote:
> The spec hasn't ratified yet and I don't have a copy (I only have 
> pre-spec hardware).  I think distribution is limited to PICMG members 
> once a spec is available, but I'm not sure.  Who needs specs anyway :)

Heh, so are there any other devices besides the qlogic device that
support this?

> >	- are you going to be generating a 2.5 version of this so that
> >	  this feature can be added to the main kernel tree?
> >
> If you think it would be accepted, I'd spend the time making 2.5 kernel 
> patches.  Beyond your other comments, any suggestions to get it accepted?

I think those comments are a great start, fix all of them, and I'd be
glad to look at the code again.

> >	- Why don't you use the existing kernel way of notifying
> >	  userspace of hotplug events, through /sbin/hotplug?
> >
> The hotplug events occur through IPMI (a system management interface 
> specification) messages.  I'm not sure if the hotswap manager will go in 
> the kernel or not, but if it were in the kernel, it could use 
> /sbin/hotplug to notify management software of hotswap events (which 
> would allow the scsi hotswap commands to be used to add or remove 
> devices).  Initially I am going to probably do a user space manager 
> since its simpler and I think that sort of thing probably belongs in 
> user space.  It will access the IPMI driver, read hotswap events from 
> the IPMI driver, and swap in and out devices and map/unmap devices via 
> the ga mapper.

Hm, sounds like the IPMI driver needs to be generating /sbin/hotplug
events itself.  That way everything could be done in userspace, right?

> Perhaps what is really needed is a kernel driver that uses the IPMI 
> driver kernel interface to pump disk device hotswap messages through 
> /sbin/hotplug.

Could the IPMI core do that itself?

> After I get a userspace implementation working (which is 
> easier to debug and test) I can start work on something like this.  What 
> would you think of that?  The nice thing about using /sbin/hotplug is 
> more things can be scripted like automatically removing a MD disk if the 
> hotswapped device is part of an MD device.
> 
> I've not started on this component yet and am just figuring out the IPMI 
> messaging at this point.  Any comments you have on how to best integrate 
> this into the current hotplug system would be highly welcomed.

I don't know a thing about IPMI.  Feel free to ask questions here, or on
the linux-hotplug-devel list if you want to.

> >	- You create a lot of new ioctls, which is not nice.  You should
> >	  probably do what was done for the pci hotplug subsystem, and
> >	  create a ram based filesystem for this subsystem.  That way
> >	  you don't need to have a /dev node, and the userspace tools
> >	  become dirt simple.
> > 
> >
> I'll have to look at that.  I'm not familiar with the ram based 
> filesystem.  Could you point me to a source file that uses some of the 
> interfaces?

In the 2.4 kernel tree take a look at:
	drivers/hotplug/pci_hotplug_core.c
and there's an article at:
	http://www.linuxjournal.com/article.php?sid=5633
on how some of that stuff works.

In the 2.5 kernel, things are much easier, with the libfs code.  Take a
look at:
	drivers/hotplug/pci_hotplug_core.c
	drivers/usb/core/inode.c
	fs/driverfs/inode.c
for 3 different implementations of ramfs based file systems.

Hope this helps,

greg k-h

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15 20:34         ` Greg KH
@ 2002-10-15 20:46           ` Steven Dake
  2002-10-15 20:54             ` Greg KH
  2002-10-16  1:05             ` Michael Clark
  0 siblings, 2 replies; 17+ messages in thread
From: Steven Dake @ 2002-10-15 20:46 UTC (permalink / raw)
  To: Greg KH; +Cc: Michael Clark, linux-kernel

The data/telecoms I've talked to require disk hotswap times of less then 
20 msec from notification of hotwap to blue led (a light used to 
indicate the device can be removed).  They would like 10 msec if it 
could be done.  This is because of how long it takes on a surprise 
extraction for the hardware to send the signal vs the user to disconnect 
the hardware.

For legacy systems such as SAFTE hotswap, polling through sg at 10 msec 
intervals would be extremely painful because of all the context 
switches.  A timer scheduled every 10 msec to send out a SCSI message 
and handle a response if there is a hotswap event is a much better course.

Thanks
-steve

Greg KH wrote:

>On Tue, Oct 15, 2002 at 12:28:59PM -0700, Steven Dake wrote:
>  
>
>>Safte polling in the kernel isn't inherently bad and could be tied into 
>>the hotplug mechanism.
>>
>>Making SAFTE hotswap available via SG would also work but system 
>>performance would be bad at small poll intervals (like 100 msec).
>>    
>>
>
>Is there a real nead to get hotplug notification any faster than that?
>
>And yes, it should all be done in userspace, whenever possible :)
>
>thanks,
>
>greg k-h
>
>
>
>  
>


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15 17:38   ` Steven Dake
  2002-10-15 19:11     ` Michael Clark
@ 2002-10-15 20:52     ` Greg KH
       [not found]       ` <3DAC89FA.9000505@mvista.com>
  1 sibling, 1 reply; 17+ messages in thread
From: Greg KH @ 2002-10-15 20:52 UTC (permalink / raw)
  To: Steven Dake; +Cc: linux-kernel

On Tue, Oct 15, 2002 at 10:38:47AM -0700, Steven Dake wrote:
> >In looking at this patch (and the others) it looks like you are relying
> >on devfs being in the system.  Is this true?  What about the other 99%
> >of machines out there with devfs disabled?
> > 
> >
> Hopefully these machines that have devfs disabled aren't ATCA machines 
> :)  It is true that devfs is REQUIRED for the GA Mapping.

Given that there are a number of kernel developers working their
respective asses off trying to get devfs out of the kernel (by obsoleting
it), I would not really recommend tying your driver to it if you want it
to be around for very long :)

> It isn't 
> required for the scsi hotswap feature.  The reason it is required for 
> gamapping is that devices must be defineable at runtime during hot swap 
> events and /dev/sda is inappropriate for several reasons as named access 
> to a device.  I can't see any other way to do it, although I could 
> provide some library in userspace that translates /dev/sda into 
> chassis/slots.  The downside of this technique is now the user must use 
> some utility to find out which device to access when using fdisk, etc 
> instead of just using the device directly.

Take a look at the driverfs tree in 2.5, and see how you could tie into
that.  I think everything you need is there, if not, please let us know.

> >You are re-using minors that have previously been reserved.  Does this
> >mean Montavista is dropping their PICMG patches?
> > 
> I had requested these minors in our rev1 of the PICMG 2.12 hotswap 
> patches.  During a rework of the picmg patches another developer dropped 
> the minors and nobody is using the old code, so I thought I could reuse 
> them.  This is probably bad mojo, but I think its ok :)

Hm, if the picmg code is dropped, why is it in the CGL release?  And a
lot of code in the picmg patch was good, I'd hate to see it go away :(

But I agree, the picmg minors are not needed (it should tie into the
existing pci hotplug interface, and not use a minor), so as long as you
talked the pcimg people into it, that's fine.

> At this point, there isn't anything using them.  I am working on a 
> hotswap manager, that may be in kernel space (for performance reasons) 
> that may use these interfaces.  I'm also working on a SAFTE Hotswap 
> processor module (ie; drivers/scsi/sp.c) for the SCSI subsystem that 
> uses these interfaces.  (Safte is a hotswap standard for SCSI chassis).

As I mentioned previously, this can probably be done in userspace (unless
you have some unreasonable performance reasons, what are the
requirements?)

> >You shouldn't use a #ifdef within this .c file.  I think you could move
> >it to your specific file, and then use #ifdef within a .h file.  This
> >also goes for your other change to this file.
> > 
> >
> I'm not so sure about this.  The reason this file is patched is to 
> support things like fdisk changes (rexporting partitions after an fdisk) 
> and initial setup.  I could provide an "alternate" check.c but would 
> rather not have to maintain the bug fixes from check.c to check_alt.c. 
> devfs_gamap_register_partition could be in a seperate file, but 
> devfs_register_partition is a generic function used by other parts of 
> the kernel that needs ifdef patches (or two seperate implementations).

Hm, no I don't think you need to do that.  Just put your logic in a
function, and provide a null function if your CONFIG variable is not
present.

> maybe not.  I think there is still a little bit of slop in the patches. 
> As you can tell, I've not code reviewed the driver yet (thats what you 
> guys are for :) and there are some things that need improvement.

Ah, didn't realize we were the first line of quality control :)

> >As the gendisk interface has been cleaned up a _lot_ in 2.5, I'm not so
> >sure how well this implementation will now work.
> >
> >Pretty much the same comments apply for your scsi-hotswap-main.patch
> >(devfs reliance, coding style, loads of ioctls, long ioctl function
> >names, global functions that don't need to be, etc.)
> > 
> >
> the scsi-hotswap-main patch can use devfs, but shouldn't rely on it. 
> Ioctls are only to provide user space access to the kernel features. 
> global functions, as in global in the symbol table (via EXPORT_SYMBOL?)

ioctls should be not used, instead a ram based fs like I mentioned in
the previous reply.

> >I also noticed that this code is included in the CGL CVS tree.  What is
> >MontaVista's (and yours) future plans for this feature?  Do you want it
> >in the main kernel tree, and are you willing to port it to 2.5?
> > 
> >
> I'd love to see ATCA support in the main kernel (if it would be 
> accepted). Maintaining patches against 2.4 is ok (since it has slowed 
> down to bug fixes mostly).   I really don't want to maintain these 
> patches against every minor release of 2.5 for now until 2.6 is 
> stabilized if I can help it.  I think these sorts of things belong in 
> the kernel, atleast the SCSI hotswap patches.

I agree, come up with a 2.5 version, and let us see what it looks like.

> These patches also depend on the Qlogic QLA 2300 driver which isn't in 
> the kernel.  Would this also be included?  We can talk to the QLogic 
> guys and see if they want to submit their driver to the 2.5 trees...

That's up to the QLogic people, I'm just too confused with the vast
number of different drivers from them :)

> If you can suggest a route for getting these patches accepted into the 
> 2.5 trees (beyond a port to 2.5 and the above suggested changes) i'd 
> love to hear them.

That's a good start.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15 20:46           ` Steven Dake
@ 2002-10-15 20:54             ` Greg KH
  2002-10-15 21:07               ` Steven Dake
  2002-10-16  1:05             ` Michael Clark
  1 sibling, 1 reply; 17+ messages in thread
From: Greg KH @ 2002-10-15 20:54 UTC (permalink / raw)
  To: Steven Dake; +Cc: Michael Clark, linux-kernel

On Tue, Oct 15, 2002 at 01:46:34PM -0700, Steven Dake wrote:
> The data/telecoms I've talked to require disk hotswap times of less then 
> 20 msec from notification of hotwap to blue led (a light used to 
> indicate the device can be removed).  They would like 10 msec if it 
> could be done.  This is because of how long it takes on a surprise 
> extraction for the hardware to send the signal vs the user to disconnect 
> the hardware.

But what starts the "notification of hotswap"?  Is this driven by the
user somehow, or is it a hardware event that happens out of the blue?

> For legacy systems such as SAFTE hotswap, polling through sg at 10 msec 
> intervals would be extremely painful because of all the context 
> switches.  A timer scheduled every 10 msec to send out a SCSI message 
> and handle a response if there is a hotswap event is a much better course.

What generates the hotswap event?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15 20:54             ` Greg KH
@ 2002-10-15 21:07               ` Steven Dake
  2002-10-15 21:16                 ` Greg KH
  0 siblings, 1 reply; 17+ messages in thread
From: Steven Dake @ 2002-10-15 21:07 UTC (permalink / raw)
  To: Greg KH; +Cc: Michael Clark, linux-kernel



Greg KH wrote:

>On Tue, Oct 15, 2002 at 01:46:34PM -0700, Steven Dake wrote:
>  
>
>>The data/telecoms I've talked to require disk hotswap times of less then 
>>20 msec from notification of hotwap to blue led (a light used to 
>>indicate the device can be removed).  They would like 10 msec if it 
>>could be done.  This is because of how long it takes on a surprise 
>>extraction for the hardware to send the signal vs the user to disconnect 
>>the hardware.
>>    
>>
>
>But what starts the "notification of hotswap"?  Is this driven by the
>user somehow, or is it a hardware event that happens out of the blue?
>  
>
In the case of Advanced TCA, an IPMI message is sent to the CPU blade 
indicating the hotswap button is pressed on the front panel of a disk 
blade.  The hotswap manager software unmaps the GA address, removes the 
device from the linux kernel via the scsi-hotswap-main stuff, and sends 
another IPMI message to the disk node telling it to light its "blue 
led".  The user removes the disk.  Insertion is easier.

In this case, the hotswap button on the front panel is used to indicate 
a hotswap event.  There is talk of making the removal of the board 
indicate a hotswap event (surprise extraction) because the technicians 
don't wait for the blue led to remove the boards occasionally and the 
system should be able to handle this use case.

>>For legacy systems such as SAFTE hotswap, polling through sg at 10 msec 
>>intervals would be extremely painful because of all the context 
>>switches.  A timer scheduled every 10 msec to send out a SCSI message 
>>and handle a response if there is a hotswap event is a much better course.
>>    
>>
>
>What generates the hotswap event?
>  
>
In the case of SAFTE, a SCSI processor (ASIC) is polled by some polling 
interval about the state of the SAFTE (SCSI) backplane.  When the state 
changes, software generates a hotswap event and removes the device.

Thanks
-steve

>  
>


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15 21:07               ` Steven Dake
@ 2002-10-15 21:16                 ` Greg KH
  2002-10-15 21:48                   ` Steven Dake
  0 siblings, 1 reply; 17+ messages in thread
From: Greg KH @ 2002-10-15 21:16 UTC (permalink / raw)
  To: Steven Dake; +Cc: Michael Clark, linux-kernel

On Tue, Oct 15, 2002 at 02:07:43PM -0700, Steven Dake wrote:
> 
> 
> Greg KH wrote:
> 
> >On Tue, Oct 15, 2002 at 01:46:34PM -0700, Steven Dake wrote:
> > 
> >
> >>The data/telecoms I've talked to require disk hotswap times of less then 
> >>20 msec from notification of hotwap to blue led (a light used to 
> >>indicate the device can be removed).  They would like 10 msec if it 
> >>could be done.  This is because of how long it takes on a surprise 
> >>extraction for the hardware to send the signal vs the user to disconnect 
> >>the hardware.
> >>   
> >>
> >
> >But what starts the "notification of hotswap"?  Is this driven by the
> >user somehow, or is it a hardware event that happens out of the blue?
> > 
> >
> In the case of Advanced TCA, an IPMI message is sent to the CPU blade 
> indicating the hotswap button is pressed on the front panel of a disk 
> blade.  The hotswap manager software unmaps the GA address, removes the 
> device from the linux kernel via the scsi-hotswap-main stuff, and sends 
> another IPMI message to the disk node telling it to light its "blue 
> led".  The user removes the disk.  Insertion is easier.

So if userspace gets the event that a button was pressed, it can decide
to light up the led after is spins the disk down, right?

> In this case, the hotswap button on the front panel is used to indicate 
> a hotswap event.  There is talk of making the removal of the board 
> indicate a hotswap event (surprise extraction) because the technicians 
> don't wait for the blue led to remove the boards occasionally and the 
> system should be able to handle this use case.

Hotplug PCI works much the same way.  A user could just walk up, pop the
slot, and pull out the card without waiting for the LED to go out.  We
don't care about that, as the user was obviously stupid in doing such a
thing.  The spec even states something like this :)

> >>For legacy systems such as SAFTE hotswap, polling through sg at 10 msec 
> >>intervals would be extremely painful because of all the context 
> >>switches.  A timer scheduled every 10 msec to send out a SCSI message 
> >>and handle a response if there is a hotswap event is a much better course.
> >>   
> >>
> >
> >What generates the hotswap event?
> > 
> >
> In the case of SAFTE, a SCSI processor (ASIC) is polled by some polling 
> interval about the state of the SAFTE (SCSI) backplane.  When the state 
> changes, software generates a hotswap event and removes the device.

So polling can be done within the kernel, right?  Then notify userspace
of the event, which can decide what to do.  Sound ok?

Or do you think this should be like the pci hotplug code, in that when a
slot is opened (or button pressed, depending on the hardware), the
kernel should scramble as fast as possible to unload the driver, and
shut down the power to the card?  Then when it is finished, notify
userspace of what just happened.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15 21:16                 ` Greg KH
@ 2002-10-15 21:48                   ` Steven Dake
  0 siblings, 0 replies; 17+ messages in thread
From: Steven Dake @ 2002-10-15 21:48 UTC (permalink / raw)
  To: Greg KH; +Cc: Michael Clark, linux-kernel



Greg KH wrote:

>On Tue, Oct 15, 2002 at 02:07:43PM -0700, Steven Dake wrote:
>  
>
>>Greg KH wrote:
>>
>>    
>>
>>>On Tue, Oct 15, 2002 at 01:46:34PM -0700, Steven Dake wrote:
>>>
>>>
>>>      
>>>
>>>>The data/telecoms I've talked to require disk hotswap times of less then 
>>>>20 msec from notification of hotwap to blue led (a light used to 
>>>>indicate the device can be removed).  They would like 10 msec if it 
>>>>could be done.  This is because of how long it takes on a surprise 
>>>>extraction for the hardware to send the signal vs the user to disconnect 
>>>>the hardware.
>>>>  
>>>>
>>>>        
>>>>
>>>But what starts the "notification of hotswap"?  Is this driven by the
>>>user somehow, or is it a hardware event that happens out of the blue?
>>>
>>>
>>>      
>>>
>>In the case of Advanced TCA, an IPMI message is sent to the CPU blade 
>>indicating the hotswap button is pressed on the front panel of a disk 
>>blade.  The hotswap manager software unmaps the GA address, removes the 
>>device from the linux kernel via the scsi-hotswap-main stuff, and sends 
>>another IPMI message to the disk node telling it to light its "blue 
>>led".  The user removes the disk.  Insertion is easier.
>>    
>>
>
>So if userspace gets the event that a button was pressed, it can decide
>to light up the led after is spins the disk down, right?
>  
>
The userspace app of some sort should light up the led after its call to 
scsi hotswap remove.  I believe the correct thing to do is that the scsi 
hotswap kernel code should spin down the disk (which it doesn't do 
today) since its a function of the bus protocol to spin up/down disks.

>  
>
>>In this case, the hotswap button on the front panel is used to indicate 
>>a hotswap event.  There is talk of making the removal of the board 
>>indicate a hotswap event (surprise extraction) because the technicians 
>>don't wait for the blue led to remove the boards occasionally and the 
>>system should be able to handle this use case.
>>    
>>
>
>Hotplug PCI works much the same way.  A user could just walk up, pop the
>slot, and pull out the card without waiting for the LED to go out.  We
>don't care about that, as the user was obviously stupid in doing such a
>thing.  The spec even states something like this :)
>  
>
unforutnately those picky telecoms want exactly this type of thing to 
not cause system crashes.  Evil I say.  The bad user should just be 
smacked with a clue-by-4.

>  
>
>>>>For legacy systems such as SAFTE hotswap, polling through sg at 10 msec 
>>>>intervals would be extremely painful because of all the context 
>>>>switches.  A timer scheduled every 10 msec to send out a SCSI message 
>>>>and handle a response if there is a hotswap event is a much better course.
>>>>  
>>>>
>>>>        
>>>>
>>>What generates the hotswap event?
>>>
>>>
>>>      
>>>
>>In the case of SAFTE, a SCSI processor (ASIC) is polled by some polling 
>>interval about the state of the SAFTE (SCSI) backplane.  When the state 
>>changes, software generates a hotswap event and removes the device.
>>    
>>
>
>So polling can be done within the kernel, right?  Then notify userspace
>of the event, which can decide what to do.  Sound ok?
>  
>
This is what I planned to do with the drivers/scsi/sp.c code or perhaps 
do the hotswap directly.  This is in my work queue for sometime in 
December so I've not thought about it much yet :)

>Or do you think this should be like the pci hotplug code, in that when a
>slot is opened (or button pressed, depending on the hardware), the
>kernel should scramble as fast as possible to unload the driver, and
>shut down the power to the card?  Then when it is finished, notify
>userspace of what just happened.
>
I think if there isn't polling from userland, response times of 20msec 
can be obtained for disk devices.  In our case, the backplane only has 
serial connections so there isn't a typical driver controlling the 
interface.  They are talking about adding support for PCI Express (3gio) 
to the PICMG3 spec, so this may become an issue in which case unloading 
the driver could probably happen from userland or kernel although kernel 
would probably give better response times.  I've not thought about it 
much since PCI Express seems a long way out.

I know our PICMG 2.12 driver unloads PCI drivers controlling cPCI 
backplane devices via a userspace daemon that retrieves a hotswap event 
via a kernel event manager, turns off the interface (in the case of 
networking) and unloads the driver if there are no more devices 
controlled by it.

In the case of SAFTE there is no event-driven hotswap (hotswaps are 
driven by a poll routine) so a userland poll for hotswap removal 
wouldn't work and this is probably better serviced by a kernel 
thread/timer of some sort.  PCI is a whole nother beast which I'll leave 
to the PICMG2.12 experts and PCI hotplug experts (you) :)

Safte is a pretty old technology and I'm only working on it because some 
platforms I support have a build in SAFTE ASIC.  

>
>thanks,
>
>greg k-h
>
>
>
>  
>


^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
       [not found]       ` <3DAC89FA.9000505@mvista.com>
@ 2002-10-15 22:04         ` Greg KH
  0 siblings, 0 replies; 17+ messages in thread
From: Greg KH @ 2002-10-15 22:04 UTC (permalink / raw)
  To: Steven Dake; +Cc: linux-kernel

On Tue, Oct 15, 2002 at 02:34:50PM -0700, Steven Dake wrote:
> >Given that there are a number of kernel developers working their
> >respective asses off trying to get devfs out of the kernel (by obsoleting
> >it), I would not really recommend tying your driver to it if you want it
> >to be around for very long :)
> > 
> >
> What is replacing devfs?  Atleast for kernel 2.4, devfs is a good 
> solution.  Is something new in 2.5 used to provide device nodes at 
> kernel runtime?

A combination of /sbin/hotplug notification, and information in the
driverfs tree will enable userspace to create and remove device nodes
whereever and with whatever name they want to.

> We didn't drop picmg2.12 support at all.  What was dropped was the 
> initial prototype implementation that used those minors (and has been 
> replaced by a better implementation that doesn't use them).  PICMG 2.12 
> support (as well as redundant system slot) is still supported in the CGL 
> tress and in MontaVista Linux.  I don't know much about picmg 2.12 
> implementation, another developer @ Mvista is doing the work.

My main complaint (as the pci hotplug maintainer) is that Mvista isn't
trying to work with the community for their code.  I had a number of
good emails with the main developer (at the time), but unfortunatly I
haven't heard anything in quite some time.  Any proding you might
provide in this area would be greatly appreciated.

Otherwise Mvista is going to be supporting that patch outside of the
main tree for forever, and that can get old and expensive over time.

> >As I mentioned previously, this can probably be done in userspace (unless
> >you have some unreasonable performance reasons, what are the
> >requirements?)
> > 
> >
> 20 msec from hotswap request to disk extraction.

Yes, that sounds pretty unreasonable :)  What drives such a quick
request turnaround time?

> The functionality is intertwined in devfs_register_partition.  I don't 
> think I understand what you mean here, could you expand in mail?

Hm, ok, let me go dig up the patch again, ugh.  Ok, I don't really know
how, but it could proably be done cleaner :)

> Ok and I'll take a look at using ramfs and driverfs instead of devfs and 
> ioctls in both patches for the 2.5 tree.  We may probably keep the patch 
> for the 2.4 tree since I dont think there is driverfs or ramfs in 2.4.

Yes, ramfs is in 2.4, and the pci hotplug core uses a filesystem just
like this in the 2.4 kernel.  But if you want to keep your patch out of
the kernel, it's fine with me :)

> >That's up to the QLogic people, I'm just too confused with the vast
> >number of different drivers from them :)
> > 
> >
> Mark Bellon @ MontaVista is working with them to support their driver in 
> MontaVista Linux.  We have made a number of quality improvements and 
> submitted them back to Qlogic.  We have talked to their development team 
> and they would be willing to submit their Qlogic 6 driver to the Linux 
> kernel.  We can do the integration work here of including it in the 
> kernel.  Should I submit a patch to the list ?

Here, and/or to the linux-scsi mailing list.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 17+ messages in thread

* Re: [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel
  2002-10-15 20:46           ` Steven Dake
  2002-10-15 20:54             ` Greg KH
@ 2002-10-16  1:05             ` Michael Clark
  1 sibling, 0 replies; 17+ messages in thread
From: Michael Clark @ 2002-10-16  1:05 UTC (permalink / raw)
  To: Steven Dake; +Cc: Greg KH, linux-kernel

On 10/16/02 04:46, Steven Dake wrote:
> The data/telecoms I've talked to require disk hotswap times of less then 
> 20 msec from notification of hotwap to blue led (a light used to 
> indicate the device can be removed).  They would like 10 msec if it 
> could be done.  This is because of how long it takes on a surprise 
> extraction for the hardware to send the signal vs the user to disconnect 
> the hardware.

I'm just surprised that the SAF-TE processers will respond this quickly.

> For legacy systems such as SAFTE hotswap, polling through sg at 10 msec 
> intervals would be extremely painful because of all the context 
> switches.  A timer scheduled every 10 msec to send out a SCSI message 
> and handle a response if there is a hotswap event is a much better course.

100 context switchs a second isn't that much is it?

I'll adjust safte-monitor from its default 2 second polling down
to 10msec and see what the result is.

~mc


^ permalink raw reply	[flat|nested] 17+ messages in thread

end of thread, other threads:[~2002-10-16  0:59 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2002-10-14 18:42 [ANNOUNCE] [PATCHES] Advanced TCA Hotswap Support in Linux Kernel Steven Dake
2002-10-15  0:59 ` Greg KH
2002-10-15 18:39   ` Steven Dake
2002-10-15 20:42     ` Greg KH
2002-10-15  5:29 ` Greg KH
2002-10-15 17:38   ` Steven Dake
2002-10-15 19:11     ` Michael Clark
2002-10-15 19:28       ` Steven Dake
2002-10-15 20:34         ` Greg KH
2002-10-15 20:46           ` Steven Dake
2002-10-15 20:54             ` Greg KH
2002-10-15 21:07               ` Steven Dake
2002-10-15 21:16                 ` Greg KH
2002-10-15 21:48                   ` Steven Dake
2002-10-16  1:05             ` Michael Clark
2002-10-15 20:52     ` Greg KH
     [not found]       ` <3DAC89FA.9000505@mvista.com>
2002-10-15 22:04         ` Greg KH

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).