RE: [PATCH] [broken?] Add MSI support to e1000

RE: [PATCH] [broken?] Add MSI support to e1000

[Nguyen, Tom L]

On Monday, August 23, 2004  Andi Kleen wrote:
>I guess it's an x86-64 specific issue? Did you test that recently?

I just tested an x86-64 MSI support in 2.6.8.1 kernel. It works fine.

Thanks,
Long

Re: [PATCH] [broken?] Add MSI support to e1000

[Roland Dreier]

    Chris> Unfortunately, there are issues with current PRO/1000
    Chris> devices that make MSI unusable.  Other testing has show
    Chris> similar results to what you are reporting, under low load a
    Chris> few interrupts can be observed to work but the part stops
    Chris> working when stressed.  This is why MSI has not already
    Chris> been enabled in the e1000 driver.

OK, thanks for the info.  Oh well...

 - Roland

Re: [PATCH] [broken?] Add MSI support to e1000

[Chris Leech]

On Mon, 23 Aug 2004 10:25:39 -0700, Roland Dreier 
> Based on the e1000 documentation I have, the only thing required for
> the e1000 to use MSI is to set the MSI enable bit in the PCI header.
> Of course there may be some e1000 erratum involving MSI but I have not
> been able to find any indication that this is the case.

Unfortunately, there are issues with current PRO/1000 devices that
make MSI unusable.  Other testing has show similar results to what you
are reporting, under low load a few interrupts can be observed to work
but the part stops working when stressed.  This is why MSI has not
already been enabled in the e1000 driver.

-Chris Leech

RE: [PATCH] [broken?] Add MSI support to e1000

[Nguyen, Tom L]

On Tuesday, August 24 Roland Dreier wrote: 
>    Andi> Yes, the flag word is 0x8b after the call. And
>    Andi> pci_enable_msi returns 0.
>
>Actually I bet the problem is that the driver is doing request_irq()
>on the wrong IRQ.  In s2io.c, s2io_init_nic() does
>
>	sp->irq = pdev->irq;
>
>and then sometime later s2io_open() does
>
>	err =
>	    request_irq((int) sp->irq, s2io_isr, SA_SHIRQ, sp->name,
dev);
>
>If you put the call to pci_enable_msi() after sp->irq is assigned, the
>driver will request the original irq (which will still be in INTx
>mode, of course), rather than the new vector assigned by the MSI core.

My guess is the same as Roland's bet. I've not tested MSI support in 
2.6.8.1 kernel. I'll provide an update later if MSI support
in x86-64 has any issue.

Thanks,
Long

Re: [PATCH] [broken?] Add MSI support to e1000

[Roland Dreier]

    Andi> Yes, the flag word is 0x8b after the call. And
    Andi> pci_enable_msi returns 0.

Actually I bet the problem is that the driver is doing request_irq()
on the wrong IRQ.  In s2io.c, s2io_init_nic() does

	sp->irq = pdev->irq;

and then sometime later s2io_open() does

	err =
	    request_irq((int) sp->irq, s2io_isr, SA_SHIRQ, sp->name, dev);

If you put the call to pci_enable_msi() after sp->irq is assigned, the
driver will request the original irq (which will still be in INTx
mode, of course), rather than the new vector assigned by the MSI core.

 - Roland

Re: [PATCH] [broken?] Add MSI support to e1000

[Roland Dreier]

    Tom> Agree. The question is that how many egde-triggered MSI with
    Tom> the same vector are generated by e1000 hardware when its
    Tom> service handler is still running.

One piece of information that I just noticed is that all of my tests
were done with CONFIG_E1000_NAPI turned on, so I may be hitting some
NAPI race condition.  Unfortunately I can't try anything on my Dell P4
system (I don't have remote serial access and I'm not in front of the
machine, so I can't mess with the network driver).

I did do a little more testing on the dual Nocona/Lindenhurst system
that I do have remote serial access to (lspci at the end of this
email).  In my earlier testing I only looked at "ifconfig" and
"/proc/interrupts" information -- what I didn't notice was that
networking didn't work at all (and tcpdump shows no packets being
received).  This happens even with NAPI turned off.

I added a little more debugging to the driver and it seems that it
reads a bogus RX descriptor (status == 0x63, errors == 0x6f).  This
happens exactly when MSI is turned on, although I have no idea why.

Unfortunately I'm afraid that I may be running into some Nocona or
Lindenhurst errata on this system.  I'll be able to try my P4 system
(with 865 chipset) tomorrow.

One other thing I don't understand independent of any e1000 issues I'm
running into is what could cause this type of message:

    CPU 0: Machine Check Exception: 0000000000000000
    CPU 0: EIP: 00000000 EFLAGS: 00000000
            eax: 00000000 ebx: 00000000 ecx: 00000000 edx: 00000000
            esi: 00000000 edi: 00000000 ebp: 00000000 esp: 00000000

The "0000000000000000" above is coming from the IA32_MCG_STATUS MSR,
and it seems to show a machine check occurring with MCIP ("machine
check in progress") == 0 -- any idea how this could happen?

Thanks,
  Roland

0000:00:00.0 Host bridge: Intel Corp. Server Memory Controller Hub (rev 09)
0000:00:00.1 ff00: Intel Corp. Memory Controller Hub Error Reporting Register (rev 09)
0000:00:01.0 System peripheral: Intel Corp. Memory Controller Hub DMA Controller (rev 09)
0000:00:02.0 PCI bridge: Intel Corp. Memory Controller Hub PCI Express Port A0 (rev 09)
0000:00:04.0 PCI bridge: Intel Corp. Memory Controller Hub PCI Express Port B0 (rev 09)
0000:00:05.0 PCI bridge: Intel Corp. Memory Controller Hub PCI Express Port B1 (rev 09)
0000:00:06.0 PCI bridge: Intel Corp. Memory Controller Hub PCI Express Port C0 (rev 09)
0000:00:07.0 PCI bridge: Intel Corp. Memory Controller Hub PCI Express Port C1 (rev 09)
0000:00:08.0 System peripheral: Intel Corp. Memory Controller Hub Extended Configuration Registers (rev 09)
0000:00:1d.0 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI #1 (rev 02)
0000:00:1d.1 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI #2 (rev 02)
0000:00:1d.2 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI #3 (rev 02)
0000:00:1d.7 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB2 EHCI Controller (rev 02)
0000:00:1e.0 PCI bridge: Intel Corp. 82801 PCI Bridge (rev c2)
0000:00:1f.0 ISA bridge: Intel Corp. 82801EB/ER (ICH5/ICH5R) LPC Bridge (rev 02)
0000:00:1f.2 IDE interface: Intel Corp. 82801EB (ICH5) Serial ATA 150 Storage Controller (rev 02)
0000:00:1f.3 SMBus: Intel Corp. 82801EB/ER (ICH5/ICH5R) SMBus Controller (rev 02)
0000:01:02.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
0000:02:00.0 PCI bridge: Intel Corp. PCI Bridge Hub A (rev 09)
0000:02:00.1 PIC: Intel Corp. PCI Bridge Hub I/OxAPIC Interrupt Controller A (rev 09)
0000:02:00.2 PCI bridge: Intel Corp. PCI Bridge Hub B (rev 09)
0000:02:00.3 PIC: Intel Corp. PCI Bridge Hub I/OxAPIC Interrupt Controller B (rev 09)
0000:04:01.0 Ethernet controller: Intel Corp. 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
0000:04:03.0 SCSI storage controller: Adaptec AIC-7902 U320 (rev 03)
0000:04:03.1 SCSI storage controller: Adaptec AIC-7902 U320 (rev 03)
0000:07:00.0 InfiniBand: Mellanox Technology: Unknown device 6278 (rev a0)

detailed lspci of e1000:

0000:04:01.0 Ethernet controller: Intel Corp. 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
	Subsystem: Intel Corp. PRO/1000 MT Server Adapter
	Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 209
	Memory at fe2c0000 (64-bit, non-prefetchable) [size=128K]
	I/O ports at d480 [size=64]
	Capabilities: [dc] Power Management version 2
	Capabilities: [e4] PCI-X non-bridge device.
	Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
00: 86 80 0f 10 17 01 30 02 01 00 00 02 10 40 00 00
10: 04 00 2c fe 00 00 00 00 00 00 00 00 00 00 00 00
20: 81 d4 00 00 00 00 00 00 00 00 00 00 86 80 01 10
30: 00 00 00 00 dc 00 00 00 00 00 00 00 0b 01 ff 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 01 e4 22 00
e0: 00 00 00 00 07 f0 02 00 08 04 43 04 00 00 00 00
f0: 05 00 80 00 04 10 e0 fe 00 00 00 00 32 40 00 00

Re: [PATCH] [broken?] Add MSI support to e1000

[Andi Kleen]

On Mon, Aug 23, 2004 at 12:41:36PM -0700, Nguyen, Tom L wrote:
> On Monday, August 23, 2004 Andi Kleen wrote:
> >There seems to be something wrong with the MSI code in the kernel.
> >I tried to add MSI support to the s2io driver on x86-64, but it just
> didn't
> >work (/proc/interrupts still displayed IO-APIC mode). I haven't 
> >investigated in detail yet though.
> 
> Would you please tell me whether the MSI enable bit of the MSI
> capability 
> structure of the s2io device is set or not after successfully calling 
> pci_enable_msi()?

Yes, the flag word is 0x8b after the call. And pci_enable_msi returns 0.

I guess it's an x86-64 specific issue? Did you test that recently?

-Andi

RE: [PATCH] [broken?] Add MSI support to e1000

[Nguyen, Tom L]

On Monday, August 23, 2004 Roland Dreier wrote:
>It seems e1000 does not support the per-vector masking feature of MSI
>(see full PCI header dump below).

You're right that e1000 hardware does not support the per-vector masking
feature.

>However if I understand the x86 APIC properly, even without masking,
>edge-triggered MSI interrupts should work OK.  As I understand it,
>when the interrupt is dispatched, its bit is moved from the IRR to the
>ISR.  If the same interrupt is received while the interrupt handler is
>running, its bit will be set again in the IRR and it will be
>dispatched again as soon as the handler exits.

Agree. The question is that how many egde-triggered MSI with the same 
vector are generated by e1000 hardware when its service handler is 
still running.

>It seems this should work OK for e1000 -- the chip should not generate
>another MSI until the driver reads the ICR in the interrupt handler,
>although it might generate an interrupt immediately afterward (while
>the interrupt handler is still running).

I do not know much about e1000 hardware. I leave it to Ganesh for an
answer.

Thanks,
Long 

RE: [PATCH] [broken?] Add MSI support to e1000

[Nguyen, Tom L]

On Monday, August 23, 2004 Andi Kleen wrote:
>There seems to be something wrong with the MSI code in the kernel.
>I tried to add MSI support to the s2io driver on x86-64, but it just
didn't
>work (/proc/interrupts still displayed IO-APIC mode). I haven't 
>investigated in detail yet though.

Would you please tell me whether the MSI enable bit of the MSI
capability 
structure of the s2io device is set or not after successfully calling 
pci_enable_msi()?

Thanks,
Long

Re: [PATCH] [broken?] Add MSI support to e1000

[Roland Dreier]

    Tom> MSI is an edge trigger, which requires the synchronization
    Tom> handshake between the hardware device and its software device
    Tom> driver. For the MSI-X capability structure, the kernel
    Tom> handles the synchronization by masking and unmasking the MSI
    Tom> maskbits. For the MSI capability structure, the MSI maskbits
    Tom> is optional. If the e1000 hardware does not support the MSI
    Tom> maskbits in its MSI capability structure, I guess it could be
    Tom> a race condition in e1000 hardware, which results an
    Tom> unpredictable behavior.

It seems e1000 does not support the per-vector masking feature of MSI
(see full PCI header dump below).

However if I understand the x86 APIC properly, even without masking,
edge-triggered MSI interrupts should work OK.  As I understand it,
when the interrupt is dispatched, its bit is moved from the IRR to the
ISR.  If the same interrupt is received while the interrupt handler is
running, its bit will be set again in the IRR and it will be
dispatched again as soon as the handler exits.

It seems this should work OK for e1000 -- the chip should not generate
another MSI until the driver reads the ICR in the interrupt handler,
although it might generate an interrupt immediately afterward (while
the interrupt handler is still running).

Am I misunderstanding something?  Or is there something in the
chipset's interrupt handling, outside of the CPU, that will break in
this case?

Thanks,
  Roland

0000:02:0c.0 Ethernet controller: Intel Corp. 82540EM Gigabit Ethernet Controller (rev 02)        Subsystem: Dell: Unknown device 0151
        Flags: bus master, 66MHz, medium devsel, latency 64, IRQ 185
        Memory at fcfe0000 (32-bit, non-prefetchable) [size=128K]
        I/O ports at df40 [size=64]
        Capabilities: [dc] Power Management version 2
        Capabilities: [e4] PCI-X non-bridge device.
        Capabilities: [f0] Message Signalled Interrupts: 64bit+ Queue=0/0 Enable-
00: 86 80 0e 10 17 01 30 02 02 00 00 02 10 40 00 00
10: 00 00 fe fc 00 00 00 00 41 df 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 28 10 51 01
30: 00 00 00 00 dc 00 00 00 00 00 00 00 09 01 ff 00
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 01 e4 22 c8
e0: 00 20 00 1b 07 f0 02 00 00 00 40 04 00 00 00 00
f0: 05 00 80 00 00 00 00 00 00 00 00 00 00 00 00 00

RE: [PATCH] [broken?] Add MSI support to e1000

[Nguyen, Tom L]

On Monday, August 23, 2004 Roland Dreier wrote: 
>    Tom> I do not see anything wrong with the patch and the kernel MSI
>    Tom> support because it works for a short time. Ganesh may provide
>    Tom> an answer on the MSI support in e1000 hardware.
>
>Based on the e1000 documentation I have, the only thing required for
>the e1000 to use MSI is to set the MSI enable bit in the PCI header.
>Of course there may be some e1000 erratum involving MSI but I have not
>been able to find any indication that this is the case.
>
>It seems possible that there could be some problem in the core Linux
>interrupt code even though some interrupts work -- for example there
>could be a race condition triggered when a second interrupt is
>delivered while handling the first interrupt.  However I couldn't find
>any such bug, although I am not at all an expert about low-level
>interrupt handling/APIC programming.

MSI is an edge trigger, which requires the synchronization handshake 
between the hardware device and its software device driver. For the 
MSI-X capability structure, the kernel handles the synchronization 
by masking and unmasking the MSI maskbits. For the MSI capability 
structure, the MSI maskbits is optional. If the e1000 hardware does not
support the MSI maskbits in its MSI capability structure, I guess it 
could be a race condition in e1000 hardware, which results an 
unpredictable behavior.

Thanks,
Long

Re: [PATCH] [broken?] Add MSI support to e1000

[Roland Dreier]

    Andi> There seems to be something wrong with the MSI code in the
    Andi> kernel.  I tried to add MSI support to the s2io driver on
    Andi> x86-64, but it just didn't work (/proc/interrupts still
    Andi> displayed IO-APIC mode). I haven't investigated in detail
    Andi> yet though.

Hmm... I've not tried MSI on x86-64 (although I have tried it on
Nocona running a 32-bit kernel).  I have successfully used both MSI
and MSI-X on i386 with my mthca (InfiniBand HCA) driver, although with
some occasional stability glitches, which may be connected to the
e1000 problem I'm seeing.  In any case, /proc/interrupts definitely
shows "PCI-MSI" mode with a non-zero number of interrupts delivered
with my patched e1000 driver.

 - Roland

Re: [PATCH] [broken?] Add MSI support to e1000

[Andi Kleen]

Roland Dreier <roland@topspin.com> writes:

>     Tom> I do not see anything wrong with the patch and the kernel MSI
>     Tom> support because it works for a short time. Ganesh may provide
>     Tom> an answer on the MSI support in e1000 hardware.
>
> Based on the e1000 documentation I have, the only thing required for
> the e1000 to use MSI is to set the MSI enable bit in the PCI header.
> Of course there may be some e1000 erratum involving MSI but I have not
> been able to find any indication that this is the case.

There seems to be something wrong with the MSI code in the kernel.
I tried to add MSI support to the s2io driver on x86-64, but it just didn't
work (/proc/interrupts still displayed IO-APIC mode). I haven't 
investigated in detail yet though.

-Andi

Re: [PATCH] [broken?] Add MSI support to e1000

[Roland Dreier]

    Tom> I do not see anything wrong with the patch and the kernel MSI
    Tom> support because it works for a short time. Ganesh may provide
    Tom> an answer on the MSI support in e1000 hardware.

Based on the e1000 documentation I have, the only thing required for
the e1000 to use MSI is to set the MSI enable bit in the PCI header.
Of course there may be some e1000 erratum involving MSI but I have not
been able to find any indication that this is the case.

It seems possible that there could be some problem in the core Linux
interrupt code even though some interrupts work -- for example there
could be a race condition triggered when a second interrupt is
delivered while handling the first interrupt.  However I couldn't find
any such bug, although I am not at all an expert about low-level
interrupt handling/APIC programming.

 - Roland

RE: [PATCH] [broken?] Add MSI support to e1000

[Nguyen, Tom L]

On Friday, August 20 Roland Dreier wrote: 
>However, on my Dell box with 865 chipset (lspci below), loading e1000
>(from kernel 2.6.8.1 with this patch applied) with MSI=1 only works
>for a short time (maybe ~1000 e1000 interrupts) before network traffic
>stops.  
>
>Is there something wrong with the patch?  Something wrong with the
>kernel MSI support?  Something wrong with the hardware?

I do not see anything wrong with the patch and the kernel MSI support
because it works for a short time. Ganesh may provide an answer on the 
MSI support in e1000 hardware.

Thanks,
Long

[PATCH] [broken?] Add MSI support to e1000

[Roland Dreier]

I recently tried to add MSI support to the e1000 driver, since on my
box, several interrupts seem to be wired together with the e1000:

     18:    1028608   IO-APIC-level  uhci_hcd, libata, eth0

I came up with the patch at the end of this email.

However, on my Dell box with 865 chipset (lspci below), loading e1000
(from kernel 2.6.8.1 with this patch applied) with MSI=1 only works
for a short time (maybe ~1000 e1000 interrupts) before network traffic
stops.  I often get the following as well:

    CPU 0: Machine Check Exception: 0000000000000000
    CPU 0: EIP: 00000000 EFLAGS: 00000000
            eax: 00000000 ebx: 00000000 ecx: 00000000 edx: 00000000
            esi: 00000000 edi: 00000000 ebp: 00000000 esp: 00000000

I also tried the same patch on a dual 3.4 GHz Xeon system with
Lindenhurst chipset with the same results.

Is there something wrong with the patch?  Something wrong with the
kernel MSI support?  Something wrong with the hardware?

I'm glad to provide any further information required to debug this.

Thanks,
  Roland

Here's the lspci output:

    0000:00:00.0 Host bridge: Intel Corp. 82865G/PE/P DRAM Controller/Host-Hub Interface (rev 02)
    0000:00:01.0 PCI bridge: Intel Corp. 82865G/PE/P PCI to AGP Controller (rev 02)
    0000:00:1d.0 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI #1 (rev 02)
    0000:00:1d.1 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI #2 (rev 02)
    0000:00:1d.2 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI #3 (rev 02)
    0000:00:1d.3 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB UHCI #4 (rev 02)
    0000:00:1d.7 USB Controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) USB2 EHCI Controller (rev 02)
    0000:00:1e.0 PCI bridge: Intel Corp. 82801 PCI Bridge (rev c2)
    0000:00:1f.0 ISA bridge: Intel Corp. 82801EB/ER (ICH5/ICH5R) LPC Bridge (rev 02)
    0000:00:1f.1 IDE interface: Intel Corp. 82801EB/ER (ICH5/ICH5R) Ultra ATA 100 Storage Controller (rev 02)
    0000:00:1f.2 IDE interface: Intel Corp. 82801EB (ICH5) Serial ATA 150 Storage Controller (rev 02)
    0000:00:1f.3 SMBus: Intel Corp. 82801EB/ER (ICH5/ICH5R) SMBus Controller (rev 02)
    0000:00:1f.5 Multimedia audio controller: Intel Corp. 82801EB/ER (ICH5/ICH5R) AC'97 Audio Controller (rev 02)
    0000:01:00.0 VGA compatible controller: nVidia Corporation NV34 [GeForce FX 5200] (rev a1)
    0000:02:0c.0 Ethernet controller: Intel Corp. 82540EM Gigabit Ethernet Controller (rev 02)

Here's the patch I tried:

Index: linux-2.6.8.1/drivers/net/e1000/e1000.h
===================================================================
--- linux-2.6.8.1.orig/drivers/net/e1000/e1000.h	2004-08-14 03:54:47.000000000 -0700
+++ linux-2.6.8.1/drivers/net/e1000/e1000.h	2004-08-20 13:10:55.000000000 -0700
@@ -251,8 +251,10 @@
 	struct e1000_desc_ring test_tx_ring;
 	struct e1000_desc_ring test_rx_ring;
 
-
 	uint32_t pci_state[16];
 	int msg_enable;
+
+	int use_msi;
+	int msi_enabled;
 };
 #endif /* _E1000_H_ */
Index: linux-2.6.8.1/drivers/net/e1000/e1000_main.c
===================================================================
--- linux-2.6.8.1.orig/drivers/net/e1000/e1000_main.c	2004-08-14 03:55:10.000000000 -0700
+++ linux-2.6.8.1/drivers/net/e1000/e1000_main.c	2004-08-20 13:22:28.000000000 -0700
@@ -541,6 +541,10 @@
 	DPRINTK(PROBE, INFO, "Intel(R) PRO/1000 Network Connection\n");
 	e1000_check_options(adapter);
 
+	/* Message Signaled Interrupts */
+	if (adapter->use_msi && !pci_enable_msi(pdev))
+		adapter->msi_enabled = 1;
+
 	/* Initial Wake on LAN setting
 	 * If APM wake is enabled in the EEPROM,
 	 * enable the ACPI Magic Packet filter
@@ -624,6 +628,8 @@
 	e1000_phy_hw_reset(&adapter->hw);
 
 	iounmap(adapter->hw.hw_addr);
+	if (adapter->msi_enabled)
+		pci_disable_msi(pdev);
 	pci_release_regions(pdev);
 
 	free_netdev(netdev);
Index: linux-2.6.8.1/drivers/net/e1000/e1000_param.c
===================================================================
--- linux-2.6.8.1.orig/drivers/net/e1000/e1000_param.c	2004-08-20 13:06:34.000000000 -0700
+++ linux-2.6.8.1/drivers/net/e1000/e1000_param.c	2004-08-20 13:32:22.000000000 -0700
@@ -194,6 +194,15 @@
 
 E1000_PARAM(InterruptThrottleRate, "Interrupt Throttling Rate");
 
+/* Message Signaled Interrupts
+ *
+ * Valid Range: 0-1 (0=off, 1=on)
+ *
+ * Default Value: 0
+ */
+
+E1000_PARAM(MSI, "Message Signaled Interrupts");
+
 #define AUTONEG_ADV_DEFAULT  0x2F
 #define AUTONEG_ADV_MASK     0x2F
 #define FLOW_CONTROL_DEFAULT FLOW_CONTROL_FULL
@@ -459,6 +468,18 @@
 			break;
 		}
 	}
+	{ /* Message Signaled Interrupts */
+		struct e1000_option opt = {
+			.type = enable_option,
+			.name = "Message Signaled Interrupts",
+			.err  = "defaulting to Disabled",
+			.def  = OPTION_DISABLED
+		};
+
+		int use_msi = MSI[bd];
+		e1000_validate_option(&use_msi, &opt, adapter);
+		adapter->use_msi = use_msi;
+	}
 
 	switch(adapter->hw.media_type) {
 	case e1000_media_type_fiber: