2.4.21 ServerWorks DMA Bugs

2.4.21 ServerWorks DMA Bugs

[Ryan Mack]

I've real the other threads but nothing touches on my specific issue.  I
have a dual P4 Xeon Dell PowerEdge 1600SC with a Fusion MPT SCSI
controller and a ServerWorks CSB5 IDE chipset.  All the HDs are on the
SCSI bus, and only my CD reader and my DVD writer are on the IDE bus (one
on each channel).  Hyperthreading is enabled (4 logical processors).  I am 
using GCC 3.2.2.

The CD readers is the blacklisted 'SAMSUNG CD-ROM SC-148C' and I never use
it so I can remove it if needed.  The DVD writer is a 'SONY DVD RW
DRU-500A'.  Both are going through the ide-scsi driver.  Whenever I
read/write CDs in the DVD writer, I get very high system load (50% on one
CPU), even though DMA seems to be enabled.

In addition, I get horrible clock skew.  After burning a 50 meg audio
track, my clock is 10 seconds off.  Enabled unmasked interrupts and 32 bit
bus IO reduces the clock skew to about .1 seconds, but this adds up.

I've tried everything, even downgrading to 2.4.20, although under 2.4.20 I
can't seem to find an appropriate devfs entry to run hdparm on, so I'm
unable to unmask interrupts and switch bus IO width.  Not compiling the
CSB5 driver doesn't make a difference either.

Anyhow, anybody know what might be going on?  Excerpts from the relevant
files included below.

Thanks, Ryan


dmesg:
Uniform Multi-Platform E-IDE driver Revision: 7.00beta4-2.4
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
SvrWks CSB5: IDE controller at PCI slot 00:0f.1
SvrWks CSB5: chipset revision 147
SvrWks CSB5: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0x08b0-0x08b7, BIOS settings: hda:DMA, hdb:pio
    ide1: BM-DMA at 0x08b8-0x08bf, BIOS settings: hdc:DMA, hdd:pio
hda: SONY DVD RW DRU-500A, ATAPI CD/DVD-ROM drive
hdc: SAMSUNG CD-ROM SC-148C, ATAPI CD/DVD-ROM drive
hdc: Disabling (U)DMA for SAMSUNG CD-ROM SC-148C
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
hda: attached ide-scsi driver.
hdc: attached ide-scsi driver.
scsi1 : SCSI host adapter emulation for IDE ATAPI devices
  Vendor: SONY      Model: DVD RW DRU-500A   Rev: 2.0c
  Type:   CD-ROM                             ANSI SCSI revision: 02
  Vendor: SAMSUNG   Model: CD-ROM SC-148C    Rev: B105
  Type:   CD-ROM                             ANSI SCSI revision: 02
Attached scsi CD-ROM sr0 at scsi1, channel 0, id 0, lun 0
Attached scsi CD-ROM sr1 at scsi1, channel 0, id 1, lun 0
sr0: scsi3-mmc drive: 32x/32x writer cd/rw xa/form2 cdda tray
Uniform CD-ROM driver Revision: 3.12
sr1: scsi3-mmc drive: 1x/48x cd/rw xa/form2 cdda tray


.config:
CONFIG_IDE=y
CONFIG_BLK_DEV_IDE=y
CONFIG_BLK_DEV_IDESCSI=m
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_BLK_DEV_GENERIC=y
CONFIG_BLK_DEV_IDEDMA_PCI=y
CONFIG_IDEDMA_PCI_AUTO=y
CONFIG_BLK_DEV_IDEDMA=y
CONFIG_BLK_DEV_SVWKS=y
CONFIG_IDEDMA_AUTO=y
CONFIG_BLK_DEV_IDE_MODES=y

CONFIG_SCSI=y
CONFIG_BLK_DEV_SD=y
CONFIG_SD_EXTRA_DEVS=40
CONFIG_BLK_DEV_SR=m
CONFIG_SR_EXTRA_DEVS=2
CONFIG_CHR_DEV_SG=m
CONFIG_SCSI_CONSTANTS=y

CONFIG_FUSION=y
CONFIG_FUSION_BOOT=y
CONFIG_FUSION_MAX_SGE=40
CONFIG_FUSION_ISENSE=m


/proc/svwks (note that like others, this is being misidentified as only
UDMA2 capable, when it should be UDMA5):

                             ServerWorks OSB4/CSB5/CSB6

                            ServerWorks CSB5 Chipset (rev 93)
------------------------------- General Status ---------------------------------
--------------- Primary Channel ---------------- Secondary Channel -------------
                 enabled                          enabled
--------------- drive0 --------- drive1 -------- drive0 ---------- drive1 ------
DMA enabled:    yes              no              no                no 
UDMA enabled:   yes              no              no                no 
UDMA enabled:   2                0               0                 0
DMA enabled:    2                2               ?                 2
PIO  enabled:   4                0               4                 0

Re: 2.4.21 ServerWorks DMA Bugs

[Andre Hedrick]

cat /proc/ide/hd{a,c}/settings

You should see unmask listed.

echo unmask:1 > /proc/ide/hd{a,c}/settings

Cheers,

Andre Hedrick
LAD Storage Consulting Group

On Fri, 4 Jul 2003, Ryan Mack wrote:

> I've real the other threads but nothing touches on my specific issue.  I
> have a dual P4 Xeon Dell PowerEdge 1600SC with a Fusion MPT SCSI
> controller and a ServerWorks CSB5 IDE chipset.  All the HDs are on the
> SCSI bus, and only my CD reader and my DVD writer are on the IDE bus (one
> on each channel).  Hyperthreading is enabled (4 logical processors).  I am 
> using GCC 3.2.2.
> 
> The CD readers is the blacklisted 'SAMSUNG CD-ROM SC-148C' and I never use
> it so I can remove it if needed.  The DVD writer is a 'SONY DVD RW
> DRU-500A'.  Both are going through the ide-scsi driver.  Whenever I
> read/write CDs in the DVD writer, I get very high system load (50% on one
> CPU), even though DMA seems to be enabled.
> 
> In addition, I get horrible clock skew.  After burning a 50 meg audio
> track, my clock is 10 seconds off.  Enabled unmasked interrupts and 32 bit
> bus IO reduces the clock skew to about .1 seconds, but this adds up.
> 
> I've tried everything, even downgrading to 2.4.20, although under 2.4.20 I
> can't seem to find an appropriate devfs entry to run hdparm on, so I'm
> unable to unmask interrupts and switch bus IO width.  Not compiling the
> CSB5 driver doesn't make a difference either.
> 
> Anyhow, anybody know what might be going on?  Excerpts from the relevant
> files included below.
> 
> Thanks, Ryan
> 
> 
> dmesg:
> Uniform Multi-Platform E-IDE driver Revision: 7.00beta4-2.4
> ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
> SvrWks CSB5: IDE controller at PCI slot 00:0f.1
> SvrWks CSB5: chipset revision 147
> SvrWks CSB5: not 100% native mode: will probe irqs later
>     ide0: BM-DMA at 0x08b0-0x08b7, BIOS settings: hda:DMA, hdb:pio
>     ide1: BM-DMA at 0x08b8-0x08bf, BIOS settings: hdc:DMA, hdd:pio
> hda: SONY DVD RW DRU-500A, ATAPI CD/DVD-ROM drive
> hdc: SAMSUNG CD-ROM SC-148C, ATAPI CD/DVD-ROM drive
> hdc: Disabling (U)DMA for SAMSUNG CD-ROM SC-148C
> ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
> ide1 at 0x170-0x177,0x376 on irq 15
> hda: attached ide-scsi driver.
> hdc: attached ide-scsi driver.
> scsi1 : SCSI host adapter emulation for IDE ATAPI devices
>   Vendor: SONY      Model: DVD RW DRU-500A   Rev: 2.0c
>   Type:   CD-ROM                             ANSI SCSI revision: 02
>   Vendor: SAMSUNG   Model: CD-ROM SC-148C    Rev: B105
>   Type:   CD-ROM                             ANSI SCSI revision: 02
> Attached scsi CD-ROM sr0 at scsi1, channel 0, id 0, lun 0
> Attached scsi CD-ROM sr1 at scsi1, channel 0, id 1, lun 0
> sr0: scsi3-mmc drive: 32x/32x writer cd/rw xa/form2 cdda tray
> Uniform CD-ROM driver Revision: 3.12
> sr1: scsi3-mmc drive: 1x/48x cd/rw xa/form2 cdda tray
> 
> 
> .config:
> CONFIG_IDE=y
> CONFIG_BLK_DEV_IDE=y
> CONFIG_BLK_DEV_IDESCSI=m
> CONFIG_BLK_DEV_IDEPCI=y
> CONFIG_BLK_DEV_GENERIC=y
> CONFIG_BLK_DEV_IDEDMA_PCI=y
> CONFIG_IDEDMA_PCI_AUTO=y
> CONFIG_BLK_DEV_IDEDMA=y
> CONFIG_BLK_DEV_SVWKS=y
> CONFIG_IDEDMA_AUTO=y
> CONFIG_BLK_DEV_IDE_MODES=y
> 
> CONFIG_SCSI=y
> CONFIG_BLK_DEV_SD=y
> CONFIG_SD_EXTRA_DEVS=40
> CONFIG_BLK_DEV_SR=m
> CONFIG_SR_EXTRA_DEVS=2
> CONFIG_CHR_DEV_SG=m
> CONFIG_SCSI_CONSTANTS=y
> 
> CONFIG_FUSION=y
> CONFIG_FUSION_BOOT=y
> CONFIG_FUSION_MAX_SGE=40
> CONFIG_FUSION_ISENSE=m
> 
> 
> /proc/svwks (note that like others, this is being misidentified as only
> UDMA2 capable, when it should be UDMA5):
> 
>                              ServerWorks OSB4/CSB5/CSB6
> 
>                             ServerWorks CSB5 Chipset (rev 93)
> ------------------------------- General Status ---------------------------------
> --------------- Primary Channel ---------------- Secondary Channel -------------
>                  enabled                          enabled
> --------------- drive0 --------- drive1 -------- drive0 ---------- drive1 ------
> DMA enabled:    yes              no              no                no 
> UDMA enabled:   yes              no              no                no 
> UDMA enabled:   2                0               0                 0
> DMA enabled:    2                2               ?                 2
> PIO  enabled:   4                0               4                 0
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: 2.4.21 ServerWorks DMA Bugs

[Markus Plail]

On Fri, 4 Jul 2003, Ryan Mack wrote:

> I've real the other threads but nothing touches on my specific issue.
> I have a dual P4 Xeon Dell PowerEdge 1600SC with a Fusion MPT SCSI
> controller and a ServerWorks CSB5 IDE chipset.  All the HDs are on the
> SCSI bus, and only my CD reader and my DVD writer are on the IDE bus
> (one on each channel).  Hyperthreading is enabled (4 logical
> processors).  I am using GCC 3.2.2.
> 
> The CD readers is the blacklisted 'SAMSUNG CD-ROM SC-148C' and I never
> use it so I can remove it if needed.  The DVD writer is a 'SONY DVD RW
> DRU-500A'.  Both are going through the ide-scsi driver.  Whenever I
> read/write CDs in the DVD writer, I get very high system load (50% on
> one CPU), even though DMA seems to be enabled.

If you are writing CDs with unusual block sizes (audio CDs, (S)VCDs,
RAW mode -> blocksize != 2048) you won't get DMA with ide-scsi, no
matter what you do. It's simply not supported.

regards
Markus

Re: 2.4.21 ServerWorks DMA Bugs

[Ryan Mack]

That at least explains the lack of DMA, but why does non-DMA IO result in 
such significant clock skew?

Also, does anybody know what the status of the failure to recognize higher 
UDMA modes on the CSB5 chipset?  Is there a working patch out there?

Thanks again, Ryan

On Sat, 5 Jul 2003, Markus Plail wrote:

> On Fri, 4 Jul 2003, Ryan Mack wrote:
> 
> > I've real the other threads but nothing touches on my specific issue.
> > I have a dual P4 Xeon Dell PowerEdge 1600SC with a Fusion MPT SCSI
> > controller and a ServerWorks CSB5 IDE chipset.  All the HDs are on the
> > SCSI bus, and only my CD reader and my DVD writer are on the IDE bus
> > (one on each channel).  Hyperthreading is enabled (4 logical
> > processors).  I am using GCC 3.2.2.
> > 
> > The CD readers is the blacklisted 'SAMSUNG CD-ROM SC-148C' and I never
> > use it so I can remove it if needed.  The DVD writer is a 'SONY DVD RW
> > DRU-500A'.  Both are going through the ide-scsi driver.  Whenever I
> > read/write CDs in the DVD writer, I get very high system load (50% on
> > one CPU), even though DMA seems to be enabled.
> 
> If you are writing CDs with unusual block sizes (audio CDs, (S)VCDs,
> RAW mode -> blocksize != 2048) you won't get DMA with ide-scsi, no
> matter what you do. It's simply not supported.

Re: 2.4.21 ServerWorks DMA Bugs

[Jeff Garzik]

Ryan Mack wrote:
> That at least explains the lack of DMA, but why does non-DMA IO result in 
> such significant clock skew?

Sounds like the kernel is disabling interrupts for a long time, when 
doing PIO data transfer (non-DMA IO).

	Jeff

Re: 2.4.21 ServerWorks DMA Bugs

[Alan Cox]

On Sad, 2003-07-05 at 17:58, Ryan Mack wrote:
> That at least explains the lack of DMA, but why does non-DMA IO result in 
> such significant clock skew?

Because for PIO we default to I/O with IRQ blocked (which is needed for
some older 486 type hardware). I really should make setup_pci set the
unmask flag

> Also, does anybody know what the status of the failure to recognize higher 
> UDMA modes on the CSB5 chipset?  Is there a working patch out there?

CSB5 UDMA works properly to all my knowledge

Re: 2.4.21 ServerWorks DMA Bugs

[Tomas Szepe]

> [alan@lxorguk.ukuu.org.uk]
> 
> > Also, does anybody know what the status of the failure to recognize higher 
> > UDMA modes on the CSB5 chipset?  Is there a working patch out there?
> 
> CSB5 UDMA works properly to all my knowledge

It doesn't all right. :)

On a G3 Compaq Proliant, all drives come up in PIO by default;
DMA needs to be enabled by "/usr/sbin/hdparm -d1 -X69 /dev/hdX".

# /sbin/lspci| grep IDE
00:0f.1 IDE interface: ServerWorks CSB5 IDE Controller (rev 93)

# hdparm -Iv /dev/hdc| grep -ai model
        Model Number:       WDC WD1200JB-00CRA1                     

-- 
Tomas Szepe <szepe@pinerecords.com>

Re: 2.4.21 ServerWorks DMA Bugs

[Alan Cox]

On Sul, 2003-07-06 at 10:06, Tomas Szepe wrote:
> It doesn't all right. :)
> 
> On a G3 Compaq Proliant, all drives come up in PIO by default;
> DMA needs to be enabled by "/usr/sbin/hdparm -d1 -X69 /dev/hdX".

This is because your compaq BIOS decided to set it up this way.

Re: 2.4.21 ServerWorks DMA Bugs

[Tomas Szepe]

> [alan@lxorguk.ukuu.org.uk]
> 
> On Sul, 2003-07-06 at 10:06, Tomas Szepe wrote:
> > It doesn't all right. :)
> > 
> > On a G3 Compaq Proliant, all drives come up in PIO by default;
> > DMA needs to be enabled by "/usr/sbin/hdparm -d1 -X69 /dev/hdX".
> 
> This is because your compaq BIOS decided to set it up this way.

Also note that when the '-X' switch is omitted (i.e. one only issues
"/usr/sbin/hdparm -d1 /dev/hdX"), the driver sets up a mode that doesn't
work and then quickly falls back to PIO.

Re: 2.4.21 ServerWorks DMA Bugs

[Alan Cox]

On Sul, 2003-07-06 at 12:10, Tomas Szepe wrote:
> Also note that when the '-X' switch is omitted (i.e. one only issues
> "/usr/sbin/hdparm -d1 /dev/hdX"), the driver sets up a mode that doesn't
> work and then quickly falls back to PIO.

Your BIOS has not tuned the drive for DMA either.

Re: 2.4.21 ServerWorks DMA Bugs

[Vojtech Pavlik]

On Sun, Jul 06, 2003 at 12:38:11PM +0100, Alan Cox wrote:
> On Sul, 2003-07-06 at 12:10, Tomas Szepe wrote:
> > Also note that when the '-X' switch is omitted (i.e. one only issues
> > "/usr/sbin/hdparm -d1 /dev/hdX"), the driver sets up a mode that doesn't
> > work and then quickly falls back to PIO.
> 
> Your BIOS has not tuned the drive for DMA either.

IMO the driver should do that in that case. There are way too many
broken BIOSes to make following what they decided to set up worthwhile.

-- 
Vojtech Pavlik
SuSE Labs, SuSE CR

Re: 2.4.21 ServerWorks DMA Bugs

[Alan Cox]

On Sul, 2003-07-06 at 17:42, Vojtech Pavlik wrote:
> IMO the driver should do that in that case. There are way too many
> broken BIOSes to make following what they decided to set up worthwhile.

It is required in the serverworks case. In the Compaq case there is also
a pending fix to do the basic same stuff for OSB4.

Re: 2.4.21 ServerWorks DMA Bugs

[Andre Hedrick]

Caution!

Serverworks has OEM's who have BIOS's which are setup for a reason.
If the BIOS is configured for any reason the the driver stomps on the BIOS
settings there will be damage to the content on the media.

Just the view from an insider in the know.

Andre Hedrick
LAD Storage Consulting Group

On Sun, 6 Jul 2003, Vojtech Pavlik wrote:

> On Sun, Jul 06, 2003 at 12:38:11PM +0100, Alan Cox wrote:
> > On Sul, 2003-07-06 at 12:10, Tomas Szepe wrote:
> > > Also note that when the '-X' switch is omitted (i.e. one only issues
> > > "/usr/sbin/hdparm -d1 /dev/hdX"), the driver sets up a mode that doesn't
> > > work and then quickly falls back to PIO.
> > 
> > Your BIOS has not tuned the drive for DMA either.
> 
> IMO the driver should do that in that case. There are way too many
> broken BIOSes to make following what they decided to set up worthwhile.
> 
> -- 
> Vojtech Pavlik
> SuSE Labs, SuSE CR
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: 2.4.21 ServerWorks DMA Bugs

[Tomas Szepe]

> [andre@linux-ide.org]
> 
> Caution!
> 
> Serverworks has OEM's who have BIOS's which are setup for a reason.
> If the BIOS is configured for any reason the the driver stomps on the BIOS
> settings there will be damage to the content on the media.
> 
> Just the view from an insider in the know.

Hmm, I certainly wouldn't expect a $3500 server to come with busted
IDE, but thanks for the suggestion, we'll take extra care.

-- 
Tomas Szepe <szepe@pinerecords.com>

Re: 2.4.21 ServerWorks DMA Bugs

[Alan Cox]

On Llu, 2003-07-07 at 04:42, Tomas Szepe wrote:
> > Just the view from an insider in the know.
> 
> Hmm, I certainly wouldn't expect a $3500 server to come with busted
> IDE, but thanks for the suggestion, we'll take extra care.

You might also want to check with HPaq about BIOS updates. I have
seen a couple of boxes which didnt set DMA but meant to or did
after BIOS update.

Re: 2.4.21 ServerWorks DMA Bugs

[Andre Hedrick]

You missed the point!  Sheesh.
They know the issues and take into account the problem.
History in the past showed us the driver is superior to the BIOS setups.
This time it bites us in the arse, then kicks us.

As for OSB4 it is a dead product.

Since BroadCom killed two of my principle (trusted) contacts in
Serverworks (CEO and VP of Business) there is only one person high in the
food chain who may be trustable but I worry.

Cheers,

Andre Hedrick
LAD Storage Consulting Group

On Mon, 7 Jul 2003, Tomas Szepe wrote:

> > [andre@linux-ide.org]
> > 
> > Caution!
> > 
> > Serverworks has OEM's who have BIOS's which are setup for a reason.
> > If the BIOS is configured for any reason the the driver stomps on the BIOS
> > settings there will be damage to the content on the media.
> > 
> > Just the view from an insider in the know.
> 
> Hmm, I certainly wouldn't expect a $3500 server to come with busted
> IDE, but thanks for the suggestion, we'll take extra care.
> 
> -- 
> Tomas Szepe <szepe@pinerecords.com>
> 

Re: 2.4.21 ServerWorks DMA Bugs

[Andre Hedrick]

Here is the skinny!

SGlist with a 2 byte boundary error on the FIFO's 510-511 or is 509-510

It all is determined on if 1 or 0 is starting as the counting number.
The FIFO flush to generate the interrupt is a DMA quirk.  Similar to the
the 8K FIFO Phy issues in SATA.

Buy me a Coke (Red Can, no BLUE Crap) and we can discuss the issues.

Cheers,

Andre Hedrick
LAD Storage Consulting Group

On Mon, 7 Jul 2003, Tomas Szepe wrote:

> > [andre@linux-ide.org]
> > 
> > Caution!
> > 
> > Serverworks has OEM's who have BIOS's which are setup for a reason.
> > If the BIOS is configured for any reason the the driver stomps on the BIOS
> > settings there will be damage to the content on the media.
> > 
> > Just the view from an insider in the know.
> 
> Hmm, I certainly wouldn't expect a $3500 server to come with busted
> IDE, but thanks for the suggestion, we'll take extra care.
> 
> -- 
> Tomas Szepe <szepe@pinerecords.com>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: 2.4.21 ServerWorks DMA Bugs

[Tomas Szepe]

> Here is the skinny!
> SGlist with a 2 byte boundary error on the FIFO's 510-511 or is 509-510
> 
> It all is determined on if 1 or 0 is starting as the counting number.
> The FIFO flush to generate the interrupt is a DMA quirk.  Similar to the
> the 8K FIFO Phy issues in SATA.

So would you know of a way to implement a workaround in the driver
or is this fatal?

> Buy me a Coke (Red Can, no BLUE Crap) and we can discuss the issues.

It'd be fun to, but I guess it's not really doable on account of
geographic issues. :)

-- 
Tomas Szepe <szepe@pinerecords.com>

Re: 2.4.21 ServerWorks DMA Bugs

[Tomas Szepe]

> [alan@lxorguk.ukuu.org.uk]
> 
> On Llu, 2003-07-07 at 04:42, Tomas Szepe wrote:
> > > Just the view from an insider in the know.
> > 
> > Hmm, I certainly wouldn't expect a $3500 server to come with busted
> > IDE, but thanks for the suggestion, we'll take extra care.
> 
> You might also want to check with HPaq about BIOS updates. I have
> seen a couple of boxes which didnt set DMA but meant to or did
> after BIOS update.

Too late now, the machine is collecting IP accounting statistics
for 20 C's, 24/7.  Flashing the BIOS seems too risky given we're
only using the IDE drives for nightly backups of a SCSI raid1 root
fs.  I guess I'll just write a simple script to see if the backup
data tends to get hosed.

Thanks!
-- 
Tomas Szepe <szepe@pinerecords.com>