ISA DMA controller hangs

ISA DMA controller hangs

[Pierre Ossman]

I've been having some problems with ISA DMA transfers failing. They work
fine until the machine does a suspend-to-ram. After that all DMA
transfers stall. Does perhaps the DMA controller need a kick in the ***
after a suspend? I'm no expert on the ISA DMA controller so I could use
some help here.

Rgds
Pierre

Re: ISA DMA controller hangs

[Alan Cox]

On Sad, 2005-05-28 at 14:38, Pierre Ossman wrote:
> I've been having some problems with ISA DMA transfers failing. They work
> fine until the machine does a suspend-to-ram. After that all DMA
> transfers stall. Does perhaps the DMA controller need a kick in the ***
> after a suspend? I'm no expert on the ISA DMA controller so I could use
> some help here.

The DMA controller has some bits of state which are potentially in need
of restoration as well as a need to ensure you don't suspend while it is
running I would imagine. Even for bus masters I believe you would need
to restore the DMA enable bits.

See include/asm-i386/dma.h but note that some registers have side
effects and the dma_outb etc are used to get required delays on some PCs

Alan

Re: ISA DMA controller hangs

[Pierre Ossman]

Alan Cox wrote:

>
>The DMA controller has some bits of state which are potentially in need
>of restoration as well as a need to ensure you don't suspend while it is
>running I would imagine. Even for bus masters I believe you would need
>to restore the DMA enable bits.
>  
>

I avoid using DMA during suspend to make sure it doesn't get into some
wierd state, but that doesn't solve the problem.

I added some debug output to the driver, dumping the DMA controller's
registers before and after suspend and it seems it goes completely
apeshit. The registers are filled with, what it seems, random data. The
reason it stops working seems to be that channel 4 gets disabled killing
the cascaded channels. I'm going to try and confirm this today.

Now for the solution. Reseting the DMA controller from a device driver
isn't really a good solution. I had a look in dma.c but it didn't seem
to have any place to attach resume/suspend functions. Also, I couldn't
find where DMA channel 4 is enabled during startup. Or does the kernel
count on BIOS doing this?

Rgds
Pierre

Re: ISA DMA controller hangs

[Pierre Ossman]

Pierre Ossman wrote:
> 
> I added some debug output to the driver, dumping the DMA controller's
> registers before and after suspend and it seems it goes completely
> apeshit. The registers are filled with, what it seems, random data. The
> reason it stops working seems to be that channel 4 gets disabled killing
> the cascaded channels. I'm going to try and confirm this today.
> 

Confirmed. By explicitly reenabling DMA channel 4 in the driver on
resumt the DMA transfers work fine.

Re: ISA DMA controller hangs

[Pierre Ossman]

So how do we solve this problem? We should do a master clear and then
enable channel 4 after a suspend. The question is where. I see three
possible places:

* In the suspend code in kernel/power.
* In the driver actually handling the suspend (ACPI/APM/etc.).
* Via the device layer by adding a device for the DMA controller.

Which would be the preferred solution?


Rgds
Pierre

Re: ISA DMA controller hangs

[Russell King]

On Thu, Jun 16, 2005 at 05:53:47PM +0200, Pierre Ossman wrote:
> So how do we solve this problem? We should do a master clear and then
> enable channel 4 after a suspend. The question is where. I see three
> possible places:
> 
> * In the suspend code in kernel/power.
> * In the driver actually handling the suspend (ACPI/APM/etc.).
> * Via the device layer by adding a device for the DMA controller.
> 
> Which would be the preferred solution?

Shouldn't there be a system device for the DMA controller?  I think
that should have appropriate hooks into the power management system
to do the necessary magic to restore whatever's needed - just like
we do for the PIC.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: ISA DMA controller hangs

[Pierre Ossman]

Russell King wrote:

>
>Shouldn't there be a system device for the DMA controller?  I think
>that should have appropriate hooks into the power management system
>to do the necessary magic to restore whatever's needed - just like
>we do for the PIC.
>
>  
>

I'll have a look at how the PIC is handled then. Any corner cases I
should be aware of?

Rgds
Pierre

Re: ISA DMA controller hangs

[Russell King]

On Sat, Jun 18, 2005 at 10:15:44AM +0200, Pierre Ossman wrote:
> Russell King wrote:
> >Shouldn't there be a system device for the DMA controller?  I think
> >that should have appropriate hooks into the power management system
> >to do the necessary magic to restore whatever's needed - just like
> >we do for the PIC.
> 
> I'll have a look at how the PIC is handled then. Any corner cases I
> should be aware of?

No idea on that score I'm afraid.  Alan may know better.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:  2.6 Serial core

Re: ISA DMA controller hangs

[Pierre Ossman]

[-- Attachment #1: Type: text/plain, Size: 374 bytes --]

Reset the ISA DMA controller into a known state after a suspend. Primary
concern was reenabling the cascading DMA channel (4).

Signed-off-by: Pierre Ossman <drzeus@drzeus.cx>

If this patch seems ok then I'll make another one for x86_64 (the only
other platform with ISA DMA AFAIK).

I couldn't find who was responsible for the i386 arch so I've cc:ed Linus.

Rgds
Pierre


[-- Attachment #2: i8237-i386.patch --]
[-- Type: text/x-patch, Size: 2134 bytes --]

Index: linux-wbsd/arch/i386/kernel/i8237.c
===================================================================
--- linux-wbsd/arch/i386/kernel/i8237.c	(revision 0)
+++ linux-wbsd/arch/i386/kernel/i8237.c	(revision 0)
@@ -0,0 +1,67 @@
+/*
+ * i8237.c: 8237A DMA controller suspend functions.
+ *
+ * Written by Pierre Ossman, 2005.
+ */
+
+#include <linux/init.h>
+#include <linux/sysdev.h>
+
+#include <asm/dma.h>
+
+/*
+ * This module just handles suspend/resume issues with the
+ * 8237A DMA controller (used for ISA and LPC).
+ * Allocation is handled in kernel/dma.c and normal usage is
+ * in asm/dma.h.
+ */
+
+static int i8237A_resume(struct sys_device *dev)
+{
+	unsigned long flags;
+	int i;
+	
+	flags = claim_dma_lock();
+	
+	dma_outb(DMA1_RESET_REG, 0);
+	dma_outb(DMA2_RESET_REG, 0);
+
+	for (i = 0;i < 8;i++) {
+	  set_dma_addr(i, 0x000000);
+	  /* DMA count is a bit wierd so this is not 0 */
+	  set_dma_count(i, 1);
+	}
+	
+	/* Enable cascade DMA or channel 0-3 won't work */
+	enable_dma(4);
+	
+	release_dma_lock(flags);
+	
+	return 0;
+}
+
+static int i8237A_suspend(struct sys_device *dev, pm_message_t state)
+{
+	return 0;
+}
+
+static struct sysdev_class i8237_sysdev_class = {
+	set_kset_name("i8237"),
+	.suspend = i8237A_suspend,
+	.resume = i8237A_resume,
+};
+
+static struct sys_device device_i8237A = {
+	.id	= 0,
+	.cls	= &i8237_sysdev_class,
+};
+
+static int __init i8237A_init_sysfs(void)
+{
+	int error = sysdev_class_register(&i8237_sysdev_class);
+	if (!error)
+		error = sysdev_register(&device_i8237A);
+	return error;
+}
+
+device_initcall(i8237A_init_sysfs);
Index: linux-wbsd/arch/i386/kernel/Makefile
===================================================================
--- linux-wbsd/arch/i386/kernel/Makefile	(revision 153)
+++ linux-wbsd/arch/i386/kernel/Makefile	(working copy)
@@ -7,7 +7,7 @@
 obj-y	:= process.o semaphore.o signal.o entry.o traps.o irq.o vm86.o \
 		ptrace.o time.o ioport.o ldt.o setup.o i8259.o sys_i386.o \
 		pci-dma.o i386_ksyms.o i387.o dmi_scan.o bootflag.o \
-		doublefault.o quirks.o
+		doublefault.o quirks.o i8237.o
 
 obj-y				+= cpu/
 obj-y				+= timers/

Re: ISA DMA controller hangs

[Alan Cox]

On Iau, 2005-06-30 at 09:00, Pierre Ossman wrote:
> +	for (i = 0;i < 8;i++) {
> +	  set_dma_addr(i, 0x000000);
> +	  /* DMA count is a bit wierd so this is not 0 */
> +	  set_dma_count(i, 1);

It is spelt "weird"

Looks basically OK although it would be good to document the situation
for a bus mastering DMA controller. Does the device have to reconfigure
the DMA on a resume or is that something the restore code for the device
should handle ?

My own feeling is tha we should dump that on the device (safer) and also
expect the device to prevent suspends during an active DMA transfer (eg
floppy)

Alan

Re: ISA DMA controller hangs

[Pierre Ossman]

Alan Cox wrote:

>
>It is spelt "weird"
>  
>

Ooops... :)

>Looks basically OK although it would be good to document the situation
>for a bus mastering DMA controller. Does the device have to reconfigure
>the DMA on a resume or is that something the restore code for the device
>should handle ?
>
>My own feeling is tha we should dump that on the device (safer) and also
>expect the device to prevent suspends during an active DMA transfer (eg
>floppy)
>
>  
>

If you mean that the device drivers should restore any state then I
fully agree. A central restore of the complete state of the DMA
controller would require keeping a copy of all data passed to it (since
some registers are write only). The reason for my piece of code here is
that nobody "owns" channel 4 so it must be restored centrally. Also,
resetting the other registers makes the DMA controller behave the same
way on all systems (from the drivers' point of view at least).

Preventing suspend also needs to be done from the drivers. Simply
because only they can determine when the devices are doing the transfer
(examining the DMA controller will only tell us if it's _prepared_ for a
transfer).

I'll fix the typo and whip up a patch for x86_64 then.

Rgds
Pierre

[PATCH 1/2] ISA DMA suspend for i386

[Pierre Ossman]

[-- Attachment #1: Type: text/plain, Size: 177 bytes --]

Reset the ISA DMA controller into a known state after a suspend. Primary
concern was reenabling the cascading DMA channel (4).

Signed-off-by: Pierre Ossman <drzeus@drzeus.cx>


[-- Attachment #2: i8237-i386.patch --]
[-- Type: text/x-patch, Size: 2126 bytes --]

Index: linux-wbsd/arch/i386/kernel/i8237.c
===================================================================
--- linux-wbsd/arch/i386/kernel/i8237.c	(revision 0)
+++ linux-wbsd/arch/i386/kernel/i8237.c	(revision 0)
@@ -0,0 +1,67 @@
+/*
+ * i8237.c: 8237A DMA controller suspend functions.
+ *
+ * Written by Pierre Ossman, 2005.
+ */
+
+#include <linux/init.h>
+#include <linux/sysdev.h>
+
+#include <asm/dma.h>
+
+/*
+ * This module just handles suspend/resume issues with the
+ * 8237A DMA controller (used for ISA and LPC).
+ * Allocation is handled in kernel/dma.c and normal usage is
+ * in asm/dma.h.
+ */
+
+static int i8237A_resume(struct sys_device *dev)
+{
+	unsigned long flags;
+	int i;
+
+	flags = claim_dma_lock();
+
+	dma_outb(DMA1_RESET_REG, 0);
+	dma_outb(DMA2_RESET_REG, 0);
+
+	for (i = 0;i < 8;i++) {
+		set_dma_addr(i, 0x000000);
+		/* DMA count is a bit weird so this is not 0 */
+		set_dma_count(i, 1);
+	}
+
+	/* Enable cascade DMA or channel 0-3 won't work */
+	enable_dma(4);
+
+	release_dma_lock(flags);
+
+	return 0;
+}
+
+static int i8237A_suspend(struct sys_device *dev, pm_message_t state)
+{
+	return 0;
+}
+
+static struct sysdev_class i8237_sysdev_class = {
+	set_kset_name("i8237"),
+	.suspend = i8237A_suspend,
+	.resume = i8237A_resume,
+};
+
+static struct sys_device device_i8237A = {
+	.id	= 0,
+	.cls	= &i8237_sysdev_class,
+};
+
+static int __init i8237A_init_sysfs(void)
+{
+	int error = sysdev_class_register(&i8237_sysdev_class);
+	if (!error)
+		error = sysdev_register(&device_i8237A);
+	return error;
+}
+
+device_initcall(i8237A_init_sysfs);
Index: linux-wbsd/arch/i386/kernel/Makefile
===================================================================
--- linux-wbsd/arch/i386/kernel/Makefile	(revision 153)
+++ linux-wbsd/arch/i386/kernel/Makefile	(working copy)
@@ -7,7 +7,7 @@
 obj-y	:= process.o semaphore.o signal.o entry.o traps.o irq.o vm86.o \
 		ptrace.o time.o ioport.o ldt.o setup.o i8259.o sys_i386.o \
 		pci-dma.o i386_ksyms.o i387.o dmi_scan.o bootflag.o \
-		doublefault.o quirks.o
+		doublefault.o quirks.o i8237.o
 
 obj-y				+= cpu/
 obj-y				+= timers/

[PATCH 2/2] ISA DMA suspend for x86_64

[Pierre Ossman]

[-- Attachment #1: Type: text/plain, Size: 301 bytes --]

Reset the ISA DMA controller into a known state after a suspend. Primary
concern was reenabling the cascading DMA channel (4).

Signed-off-by: Pierre Ossman <drzeus@drzeus.cx>

Word of warning, I haven't tested this platform since I don't have any
x86_64 hardware. But it shouldn't differ from i386.


[-- Attachment #2: i8237-x86_64.patch --]
[-- Type: text/x-patch, Size: 2213 bytes --]

Index: linux-wbsd/arch/x86_64/kernel/i8237.c
===================================================================
--- linux-wbsd/arch/x86_64/kernel/i8237.c	(revision 0)
+++ linux-wbsd/arch/x86_64/kernel/i8237.c	(revision 0)
@@ -0,0 +1,67 @@
+/*
+ * i8237.c: 8237A DMA controller suspend functions.
+ *
+ * Written by Pierre Ossman, 2005.
+ */
+
+#include <linux/init.h>
+#include <linux/sysdev.h>
+
+#include <asm/dma.h>
+
+/*
+ * This module just handles suspend/resume issues with the
+ * 8237A DMA controller (used for ISA and LPC).
+ * Allocation is handled in kernel/dma.c and normal usage is
+ * in asm/dma.h.
+ */
+
+static int i8237A_resume(struct sys_device *dev)
+{
+	unsigned long flags;
+	int i;
+
+	flags = claim_dma_lock();
+
+	dma_outb(DMA1_RESET_REG, 0);
+	dma_outb(DMA2_RESET_REG, 0);
+
+	for (i = 0;i < 8;i++) {
+		set_dma_addr(i, 0x000000);
+		/* DMA count is a bit weird so this is not 0 */
+		set_dma_count(i, 1);
+	}
+
+	/* Enable cascade DMA or channel 0-3 won't work */
+	enable_dma(4);
+
+	release_dma_lock(flags);
+
+	return 0;
+}
+
+static int i8237A_suspend(struct sys_device *dev, pm_message_t state)
+{
+	return 0;
+}
+
+static struct sysdev_class i8237_sysdev_class = {
+	set_kset_name("i8237"),
+	.suspend = i8237A_suspend,
+	.resume = i8237A_resume,
+};
+
+static struct sys_device device_i8237A = {
+	.id	= 0,
+	.cls	= &i8237_sysdev_class,
+};
+
+static int __init i8237A_init_sysfs(void)
+{
+	int error = sysdev_class_register(&i8237_sysdev_class);
+	if (!error)
+		error = sysdev_register(&device_i8237A);
+	return error;
+}
+
+device_initcall(i8237A_init_sysfs);
Index: linux-wbsd/arch/x86_64/kernel/Makefile
===================================================================
--- linux-wbsd/arch/x86_64/kernel/Makefile	(revision 153)
+++ linux-wbsd/arch/x86_64/kernel/Makefile	(working copy)
@@ -7,7 +7,7 @@
 obj-y	:= process.o semaphore.o signal.o entry.o traps.o irq.o \
 		ptrace.o time.o ioport.o ldt.o setup.o i8259.o sys_x86_64.o \
 		x8664_ksyms.o i387.o syscall.o vsyscall.o \
-		setup64.o bootflag.o e820.o reboot.o quirks.o
+		setup64.o bootflag.o e820.o reboot.o quirks.o i8237.o

 obj-$(CONFIG_X86_MCE)         += mce.o
 obj-$(CONFIG_X86_MCE_INTEL)	+= mce_intel.o