problem with a cobalt RaQ550 system and DMA (Serverworks OSB4 in impossible state)

problem with a cobalt RaQ550 system and DMA (Serverworks OSB4 in impossible state)

[Olivier Bornet]

[-- Attachment #1: Type: text/plain, Size: 2875 bytes --]

Hello,

I'm trying to install Debian on 4 RaQ550 with each 2 80GB disks. All
seems OK with 3 of RaQ, but with one, it crash when I put the two disks
in a RAID1 meta device. In fact, it as crash at about 6% before the 70GB
partition is fully synchronized.

The error message is:

===================================================================
Serverworks OSB4 in impossible state.
Disable UDMA or if you are using Seagate then try switching disk types
on this controller. Please report this event to osb4-bug@ide.cabal.tm
OSB4: continuing might cause disk corruption. 
===================================================================

I have also send the same email as this one to osb4-bug@ide.cabal.tm.

After some search, it seems I must enable MWDMA for my disks. (ref:
http://www.cs.helsinki.fi/linux/linux-kernel/2002-33/0836.html )

Here is the configuration : 2 IDE 80GB disks, one on hda, one on hdc.
The kernel is the stock 2.4.20, with cobalt patches. The cobalt patches
don't modify the drivers/ide/serverworks.c.

IDE part of lspci -v:

00:0f.1 IDE interface: ServerWorks CSB5 IDE Controller (rev 92) (prog-if 8f [Master SecP SecO PriP PriO])
        Subsystem: Sun Microsystems Computer Corp.: Unknown device c000
        Flags: bus master, medium devsel, latency 64
        I/O ports at fcd8 [size=8]
        I/O ports at fcd4 [size=4]
        I/O ports at fcc8 [size=8]
        I/O ports at fcc4 [size=4]
        I/O ports at fcb0 [size=16]
        I/O ports at fcac [size=4]

/proc/ide/hda and /proc/ide/hdc say it's a ST380021A. (I have no
physical access to the system now...)

I have try to set the DMA of the disks with:
    hdparm -X66 -d 1 /dev/hda
    hdparm -X66 -d 1 /dev/hdc
but this has cause a direct hangup (with the same message "Serverworks
OSB4 in impossible state...."). Maybe I don't use correctly hdparm, as
this is my first attempt with it.

At this time, I have disabled the dma with:
    hdparm -d 0 /dev/hda
    hdparm -d 0 /dev/hdc
and the sync in in the way (but say we need about 6 hours to finish,
comparing to the unmodifed test giving about 1 hour).

So my questions are :

- what can I do ?
- may this problem come also with the others RaQ we have (as far as I
  know, they are the same, and they are ordered at the same time)
- is the system safe without dma ?

Thanks in advance for any help, or any pointer to a solution. (Of
course, I can apply a patch to the kernel if this may correct the
problem).

		Olivier
-- 
Olivier Bornet                 |      français : http://puck.ch/f
Swiss Ice Hockey Results       |      english  : http://puck.ch/e
http://puck.ch/                |      deutsch  : http://puck.ch/g
Olivier.Bornet@puck.ch         |      italiano : http://puck.ch/i
Get my PGP-key at http://puck.ch/pgp or at http://wwwkeys.pgp.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: problem with a cobalt RaQ550 system and DMA (Serverworks OSB4 in impossible state)

[Alan Cox]

On Mer, 2003-04-23 at 22:27, Olivier Bornet wrote:
> I'm trying to install Debian on 4 RaQ550 with each 2 80GB disks. All
> seems OK with 3 of RaQ, but with one, it crash when I put the two disks
> in a RAID1 meta device. In fact, it as crash at about 6% before the 70GB
> partition is fully synchronized.


Bad block I think. Its a bug fixed in 2.4.21pre.  It trips a sanity
check for an OSB4 bug inadvertantly. 2.4.21pre handles CSB5 with full
UDMA and OSB4 in MWDMA2 without tripping wrongly.

If your chipset is CSB5 you can also just comment out the check and
rebuild


Alan

Re: problem with a cobalt RaQ550 system and DMA (Serverworks OSB4 in impossible state)

[Olivier Bornet]

[-- Attachment #1: Type: text/plain, Size: 1463 bytes --]

Hi Alan,

On Wed, Apr 23, 2003 at 11:21:10PM +0100, Alan Cox wrote:
> On Mer, 2003-04-23 at 22:27, Olivier Bornet wrote:
> > I'm trying to install Debian on 4 RaQ550 with each 2 80GB disks. All
> > seems OK with 3 of RaQ, but with one, it crash when I put the two disks
> > in a RAID1 meta device. In fact, it as crash at about 6% before the 70GB
> > partition is fully synchronized.
> 
> 
> Bad block I think. Its a bug fixed in 2.4.21pre.  It trips a sanity
> check for an OSB4 bug inadvertantly. 2.4.21pre handles CSB5 with full
> UDMA and OSB4 in MWDMA2 without tripping wrongly.
> 
> If your chipset is CSB5 you can also just comment out the check and
> rebuild

At this time, I have compiled and installed a 2.4.20-ac2 + some cobalt
patches. Is the bug also fixed in 2.4.20-ac2, or must I rebuild the
2.4.20 with the check commented out ?

(currently, the sync of the partition is in the way with the ac2
kernel... just started... need about 100 minutes to finish...).

I will comment out the check tomorrow if this is needed.

Thanks for your help, and keep up the good work.

		Olivier
-- 
Olivier Bornet                 |      français : http://puck.ch/f
Swiss Ice Hockey Results       |      english  : http://puck.ch/e
http://puck.ch/                |      deutsch  : http://puck.ch/g
Olivier.Bornet@puck.ch         |      italiano : http://puck.ch/i
Get my PGP-key at http://puck.ch/pgp or at http://wwwkeys.pgp.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: problem with a cobalt RaQ550 system and DMA (Serverworks OSB4 in impossible state)

[Duncan Laurie]

[-- Attachment #1: Type: text/plain, Size: 1084 bytes --]

Olivier Bornet wrote:
> Hello,
> 
> I'm trying to install Debian on 4 RaQ550 with each 2 80GB disks. All
> seems OK with 3 of RaQ, but with one, it crash when I put the two disks
> in a RAID1 meta device. In fact, it as crash at about 6% before the 70GB
> partition is fully synchronized.
> 

Hi Olivier,

There is an interaction problem between the Serverworks IDE controller
and some Seagate BlockPoint drives.  They aren't handling the bus
throttling correctly on ATA bus writes and are sometimes overflowing
their FIFOs.

The CSB5 in UDMA mode 5 sends 3 words after the drive de-asserts
DDMARDY- (per ATA spec section 9.14.3.2) but these drives sometimes
have problems when the three extra words are sent and overflows its
FIFO.  Other IDE controllers only send 2 extra words and don't cause
this to happen.

The only work around we have found is to limit these drives to ATA
mode 4.  Supposedly newer Seagate firmware for these drives will report
only UDMA mode 4 capable, but for earlier drives (like those probably
found in your raq550) try this patch against 2.4.20.

-duncan

[-- Attachment #2: serverworks.c.diff --]
[-- Type: text/plain, Size: 1205 bytes --]

--- serverworks.c	2002-08-02 17:39:44.000000000 -0700
+++ /home/duncan/cvs/linux-2.4/drivers/ide/serverworks.c	2003-04-23 16:37:45.000000000 -0700
@@ -100,6 +100,12 @@
 #include <linux/stat.h>
 #include <linux/proc_fs.h>
 
+const char *bad_ata100[] = {
+	"ST340016A",
+	"ST380021A",
+	NULL
+};
+
 static struct pci_dev *bmide_dev;
 static byte svwks_revision = 0;
 
@@ -107,6 +113,14 @@
 extern int (*svwks_display_info)(char *, char **, off_t, int); /* ide-proc.c */
 extern char *ide_media_verbose(ide_drive_t *);
 
+static int check_in_drive_lists (ide_drive_t *drive, const char **list)
+{
+	while (*list)
+		if (!strcmp(*list++, drive->id->model))
+			return 1;
+	return 0;
+}
+
 static int svwks_get_info (char *buffer, char **addr, off_t offset, int count)
 {
 	char *p = buffer;
@@ -420,7 +434,8 @@
 	int ultra100 	= (ultra66 && svwks_revision >= SVWKS_CSB5_REVISION_NEW) ? 1 : 0;
 	byte speed;
 
-	if ((id->dma_ultra & 0x0020) && (udma_66) && (ultra100)) {
+	if ((id->dma_ultra & 0x0020) && (udma_66) && (ultra100) &&
+	    !check_in_drive_lists(drive, bad_ata100)) {
 		speed = XFER_UDMA_5;
 	} else if (id->dma_ultra & 0x0010) {
 		speed = ((udma_66) && (ultra66)) ? XFER_UDMA_4 : XFER_UDMA_2;

Re: problem with a cobalt RaQ550 system and DMA (Serverworks OSB4 in impossible state)

[Olivier Bornet]

[-- Attachment #1: Type: text/plain, Size: 893 bytes --]

Hi Duncan,

> The only work around we have found is to limit these drives to ATA
> mode 4.  Supposedly newer Seagate firmware for these drives will report
> only UDMA mode 4 capable, but for earlier drives (like those probably
> found in your raq550) try this patch against 2.4.20.

Thanks ! This is the solution. I have applied the patch, and the sync
was now OK after about 1 hours for the 70 GB.

Is this patch already applyed on the 2.4.21-pre kernels ? If not can
someone apply it ?

One more time, thanks for your help.

		Olivier
-- 
Olivier Bornet                 |      français : http://puck.ch/f
Swiss Ice Hockey Results       |      english  : http://puck.ch/e
http://puck.ch/                |      deutsch  : http://puck.ch/g
Olivier.Bornet@puck.ch         |      italiano : http://puck.ch/i
Get my PGP-key at http://puck.ch/pgp or at http://wwwkeys.pgp.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: problem with a cobalt RaQ550 system and DMA (Serverworks OSB4 in impossible state)

[Olivier Bornet]

[-- Attachment #1: Type: text/plain, Size: 1106 bytes --]

Hi all,

I reply to myself, after having test this solution.

> At this time, I have compiled and installed a 2.4.20-ac2 + some cobalt
> patches. Is the bug also fixed in 2.4.20-ac2, or must I rebuild the
> 2.4.20 with the check commented out ?

The 2.4.20-ac2 patched kernel help a little : the system don't crash
anymore. But the disk is marked as defective, and is removed from the
raid1 metadevice.

One other problem with the -ac2 is the speed for the rebuild : it seems
to be 2 times slower than with the Ducan patch. (about 2 hours instead
of 1 hour).

So, my solution is to use the patch from Ducan. I hope it (or a
derivative form of it) will be included in the next kernel releases.

Good day, and thanks all for the help.

		Olivier
-- 
Olivier Bornet                 |      français : http://puck.ch/f
Swiss Ice Hockey Results       |      english  : http://puck.ch/e
http://puck.ch/                |      deutsch  : http://puck.ch/g
Olivier.Bornet@puck.ch         |      italiano : http://puck.ch/i
Get my PGP-key at http://puck.ch/pgp or at http://wwwkeys.pgp.net

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: problem with Serverworks CSB5 IDE

[Duncan Laurie]

[-- Attachment #1: Type: text/plain, Size: 1401 bytes --]

Olivier Bornet wrote:
> Hi all,
> 
> I reply to myself, after having test this solution.
> 
> 
>>At this time, I have compiled and installed a 2.4.20-ac2 + some cobalt
>>patches. Is the bug also fixed in 2.4.20-ac2, or must I rebuild the
>>2.4.20 with the check commented out ?
> 
> 
> The 2.4.20-ac2 patched kernel help a little : the system don't crash
> anymore. But the disk is marked as defective, and is removed from the
> raid1 metadevice.
> 
> One other problem with the -ac2 is the speed for the rebuild : it seems
> to be 2 times slower than with the Ducan patch. (about 2 hours instead
> of 1 hour).
> 
> So, my solution is to use the patch from Ducan. I hope it (or a
> derivative form of it) will be included in the next kernel releases.
> 
> Good day, and thanks all for the help.
>

Here is a 2.4.21-rc1 version of the patch, with a few modificaions
due to the changes in IDE..

Actually UDMA mode detection is not working at all for CSB5 in
2.4.21-rc1 because svwks_revision variable is set in __init function
so was reading as 0 in svwks_ratemask().  This made it think UDMA
mode 2 was the max supported, when in reality new revisions do UDMA
mode 5 and old revisions are mode 4 max.

The bad_ata100 list has been extended to include the whole family
of Barracuda ATA IV drives; we only shipped the 80gb and 40gb models
from that series so the others were not listed before.

-duncan

[-- Attachment #2: serverworks-2.4.21-rc1.diff --]
[-- Type: text/plain, Size: 2213 bytes --]

diff -ruN linux-2.4.21-rc1.orig/drivers/ide/pci/serverworks.c linux-2.4.21-rc1/drivers/ide/pci/serverworks.c
--- linux-2.4.21-rc1.orig/drivers/ide/pci/serverworks.c	Thu Apr 24 14:16:28 2003
+++ linux-2.4.21-rc1/drivers/ide/pci/serverworks.c	Thu Apr 24 14:41:26 2003
@@ -203,11 +203,22 @@
 }
 #endif  /* defined(DISPLAY_SVWKS_TIMINGS) && defined(CONFIG_PROC_FS) */
 
+static int check_in_drive_lists (ide_drive_t *drive, const char **list)
+{
+	while (*list)
+		if (!strcmp(*list++, drive->id->model))
+			return 1;
+	return 0;
+}
+
 static u8 svwks_ratemask (ide_drive_t *drive)
 {
 	struct pci_dev *dev     = HWIF(drive)->pci_dev;
 	u8 mode;
 
+	if (!svwks_revision)
+		pci_read_config_byte(dev, PCI_REVISION_ID, &svwks_revision);
+
 	if (dev->device == PCI_DEVICE_ID_SERVERWORKS_OSB4IDE) {
 		u32 reg = 0;
 		if (isa_dev)
@@ -221,13 +232,15 @@
 		/* Check the OSB4 DMA33 enable bit */
 		return ((reg & 0x00004000) == 0x00004000) ? 1 : 0;
 	} else if (svwks_revision < SVWKS_CSB5_REVISION_NEW) {
-		return 1;
+		return 2;
 	} else if (svwks_revision >= SVWKS_CSB5_REVISION_NEW) {
 		u8 btr = 0;
 		pci_read_config_byte(dev, 0x5A, &btr);
-		mode = btr;
+		mode = btr & 0x3;
 		if (!eighty_ninty_three(drive))
 			mode = min(mode, (u8)1);
+		if (mode == 3 && check_in_drive_lists(drive, svwks_bad_ata100))
+			mode = 2;
 	}
 	if (((dev->device == PCI_DEVICE_ID_SERVERWORKS_CSB6IDE) ||
 	     (dev->device == PCI_DEVICE_ID_SERVERWORKS_CSB6IDE2)) &&
diff -ruN linux-2.4.21-rc1.orig/drivers/ide/pci/serverworks.h linux-2.4.21-rc1/drivers/ide/pci/serverworks.h
--- linux-2.4.21-rc1.orig/drivers/ide/pci/serverworks.h	Thu Apr 24 14:16:28 2003
+++ linux-2.4.21-rc1/drivers/ide/pci/serverworks.h	Thu Apr 24 14:41:41 2003
@@ -11,6 +11,16 @@
 #define SVWKS_CSB5_REVISION_NEW	0x92 /* min PCI_REVISION_ID for UDMA5 (A2.0) */
 #define SVWKS_CSB6_REVISION	0xa0 /* min PCI_REVISION_ID for UDMA4 (A1.0) */
 
+/* Seagate Barracuda ATA IV Family drives in UDMA mode 5
+ * can overrun their FIFOs when used with the CSB5 */
+const char *svwks_bad_ata100[] = {
+	"ST320011A",
+	"ST340016A",
+	"ST360021A",
+	"ST380021A",
+	NULL
+};
+
 #define DISPLAY_SVWKS_TIMINGS	1
 
 #if defined(DISPLAY_SVWKS_TIMINGS) && defined(CONFIG_PROC_FS)

Re: problem with Serverworks CSB5 IDE

[Duncan Laurie]

Duncan Laurie wrote:
> Olivier Bornet wrote:
> 
>> Hi all,
>>
>> I reply to myself, after having test this solution.
>>
>>
>>> At this time, I have compiled and installed a 2.4.20-ac2 + some cobalt
>>> patches. Is the bug also fixed in 2.4.20-ac2, or must I rebuild the
>>> 2.4.20 with the check commented out ?
>>
>>
>>
>> The 2.4.20-ac2 patched kernel help a little : the system don't crash
>> anymore. But the disk is marked as defective, and is removed from the
>> raid1 metadevice.
>>
>> One other problem with the -ac2 is the speed for the rebuild : it seems
>> to be 2 times slower than with the Ducan patch. (about 2 hours instead
>> of 1 hour).
>>
>> So, my solution is to use the patch from Ducan. I hope it (or a
>> derivative form of it) will be included in the next kernel releases.
>>
>> Good day, and thanks all for the help.
>>
> 
> Here is a 2.4.21-rc1 version of the patch, with a few modificaions
> due to the changes in IDE..
> 
> Actually UDMA mode detection is not working at all for CSB5 in
> 2.4.21-rc1 because svwks_revision variable is set in __init function
> so was reading as 0 in svwks_ratemask().  This made it think UDMA

Oops, this analysis is wrong..  The svwks_revision variable is set
in the init_chipset_svwks() function, which doesn't appear to ever
get called because dev->irq==0.  The init_chipset function is also
responsible for the call to ide_pci_register_host_proc(), so since
it isn't being called there is no /proc/ide/svwks.

I think for Serverworks IDE you will never get a valid dev->irq out
of the pci config register, so maybe should there still be a call to
d->init_chipset in this block from drivers/ide/setup-pci.c:

   } else if (!pciirq) {
       if (noisy)
           printk(KERN_WARNING "%s: bad irq (%d): will probe later\n",
                  d->name, pciirq);
       pciirq = 0;
   }

-duncan

Re: problem with Serverworks CSB5 IDE

[Alan Cox]

> Actually UDMA mode detection is not working at all for CSB5 in
> 2.4.21-rc1 because svwks_revision variable is set in __init function
> so was reading as 0 in svwks_ratemask().  This made it think UDMA
> mode 2 was the max supported, when in reality new revisions do UDMA
> mode 5 and old revisions are mode 4 max.

The revision id is read when we init_chipset_svwks, which comes from the
PCI setup. If the chip is in legacy mode we call init chipset early on
regardless. If it is in native mode it gets called too and we ignore
its view of the IRQ (since thats now PCI defined).

>  		/* Check the OSB4 DMA33 enable bit */
>  		return ((reg & 0x00004000) == 0x00004000) ? 1 : 0;
>  	} else if (svwks_revision < SVWKS_CSB5_REVISION_NEW) {
> -		return 1;
> +		return 2;

Why this change ?

Re: problem with Serverworks CSB5 IDE

[Alan Cox]

On Gwe, 2003-04-25 at 17:39, Duncan Laurie wrote:
> mode because the PCI interrupt pin register is hardwired to zero (don't
> ask me why...) so it follows a codepath in do_ide_setup_pci_device()
> where init_chipset isn't called.

That would imply a problem in the PCI layer, since the IRQ should have 
been assigned, and if the IRQ is not assigned we can't use the device.

I'll take a look. 

Re: problem with Serverworks CSB5 IDE

[Duncan Laurie]

Alan Cox wrote:
> 
> The revision id is read when we init_chipset_svwks, which comes from the
> PCI setup. If the chip is in legacy mode we call init chipset early on
> regardless. If it is in native mode it gets called too and we ignore
> its view of the IRQ (since thats now PCI defined).
> 

Yeah I saw that after I hit send, but for serverworks the init_chipset
is not always called because it can fall into a corner case when in native
mode because the PCI interrupt pin register is hardwired to zero (don't
ask me why...) so it follows a codepath in do_ide_setup_pci_device()
where init_chipset isn't called.

This patch adds the function call, which fixes the svwks_revision
variable and the missing /proc/ide/svwks:

--- setup-pci.c~        2003-04-25 09:20:31.000000000 -0700
+++ setup-pci.c 2003-04-25 09:24:27.000000000 -0700
@@ -609,7 +609,7 @@
                 if (noisy)
                         printk(KERN_WARNING "%s: bad irq (%d): will probe later\n",
                                 d->name, pciirq);
-               pciirq = 0;
+               pciirq = (d->init_chipset) ? d->init_chipset(dev, d->name) : 0;
         } else {
                 if (d->init_chipset)
                         d->init_chipset(dev, d->name);

> 
>> 		/* Check the OSB4 DMA33 enable bit */
>> 		return ((reg & 0x00004000) == 0x00004000) ? 1 : 0;
>> 	} else if (svwks_revision < SVWKS_CSB5_REVISION_NEW) {
>>-		return 1;
>>+		return 2;
> 
> 
> Why this change ?
> 
> 

Because the max supported mode for CSB5 < rev 0x92 is udma 4 (=2),
not udma 2 (=1).

-duncan

Re: problem with Serverworks CSB5 IDE

[Alan Cox]

On Gwe, 2003-04-25 at 19:52, Duncan Laurie wrote:
> It might just be another unfortunate serverworks chipset bug...
> 
> The CSB5 doesn't appear to fully support native mode--sure you can
> put it in native mode (!) and you're free to assign the BARs any
> way you want, but it still assumes IRQ14 for ide0 and IRQ15 for
> ide1 when they should really be collapsed and shared on a single
> PCI (non-compatibility) interrupt.

Could be. I've never seen one in native mode. 

Re: problem with Serverworks CSB5 IDE

[Duncan Laurie]

Alan Cox wrote:
> On Gwe, 2003-04-25 at 17:39, Duncan Laurie wrote:
> 
>>mode because the PCI interrupt pin register is hardwired to zero (don't
>>ask me why...) so it follows a codepath in do_ide_setup_pci_device()
>>where init_chipset isn't called.
> 
> 
> That would imply a problem in the PCI layer, since the IRQ should have 
> been assigned, and if the IRQ is not assigned we can't use the device.
> 
> I'll take a look. 
> 

It might just be another unfortunate serverworks chipset bug...

The CSB5 doesn't appear to fully support native mode--sure you can
put it in native mode (!) and you're free to assign the BARs any
way you want, but it still assumes IRQ14 for ide0 and IRQ15 for
ide1 when they should really be collapsed and shared on a single
PCI (non-compatibility) interrupt.

While it might be possible to re-route the interrupts using the
pirq table and a few different bits in the southbridge, that still
wouldn't solve the problem of PCI config reg 0x3c being read-only
and set to zero in the IDE function..

The best answer might be to not use native mode.  We can force it
into legacy mode with a pci quirk and it works, but its ugly:


--- quirks.c~   2003-04-25 11:37:46.000000000 -0700
+++ quirks.c    2003-04-25 11:46:45.000000000 -0700
@@ -631,6 +631,22 @@
                 interrupt_line_quirk = 1;
  }

+/*
+ *     Serverworks CSB5 IDE does not fully support native mode
+ */
+static void __init quirk_svwks_csb5ide(struct pci_dev *pdev)
+{
+       u8 prog;
+       pci_read_config_byte(pdev, PCI_CLASS_PROG, &prog);
+       if (prog & 5) {
+               prog &= ~5;
+               pdev->class &= ~5;
+               pci_write_config_byte(pdev, PCI_CLASS_PROG, prog);
+               /* need to re-assign BARs for compat mode */
+               quirk_ide_bases(pdev);
+       }
+}
+
  /*
   *  The main table of quirks.
   */
@@ -702,6 +718,8 @@

         { PCI_FIXUP_FINAL,      PCI_VENDOR_ID_CYRIX,    PCI_DEVICE_ID_CYRIX_PCI_MASTER, 
quirk_mediagx_master },

+       { PCI_FIXUP_HEADER,     PCI_VENDOR_ID_SERVERWORKS, 
PCI_DEVICE_ID_SERVERWORKS_CSB5IDE, quirk_svwks_csb5ide },
+
  #ifdef CONFIG_X86_IO_APIC
         { PCI_FIXUP_FINAL,      PCI_VENDOR_ID_AMD,      PCI_DEVICE_ID_AMD_8131_APIC,
           quirk_amd_8131_ioapic },


-duncan

Re: problem with Serverworks CSB5 IDE

[Alan Cox]

Looks sane but your mailer ate the patch (line wrap monster got it)

Can you resend as an attachment ?