RE: Extremely slow network with e1000 & ip_conntrack

RE: Extremely slow network with e1000 & ip_conntrack

[Feldman, Scott]

> > > I compiled lots of kernels :-( and narrowed it down to between 
> > > 2.5.26 and 2.5.46.
> > > 
> > > Kernel version	Chip		Problem?
> > > 2.4.22		82540EM		N
> > > 2.5.26		82540EM		N
> > > 2.5.46		82540EM		Y
> > > 2.6.0-test10		82540EM		Y
> > > 2.6.0-test11		82540EM		Y
> > > 2.6.0-test11		82547EI		N
> > > 2.4.22nptlsmp		82547EI		N

In e1000, check this out:

  #ifdef NETIF_F_TSO
          if((adapter->hw.mac_type >= e1000_82544) &&
             (adapter->hw.mac_type != e1000_82547))
                  netdev->features |= NETIF_F_TSO;
  #endif

Try turning off TSO by disabling this code or by using "ethtool -K tso
off" (need version 1.8).

-scott

Re: Extremely slow network with e1000 & ip_conntrack

[Stephen Lee]

"Feldman, Scott" <scott.feldman@intel.com> wrote:
> 
> Try turning off TSO by disabling this code or by using "ethtool -K tso
> off" (need version 1.8).

Yes, turning off TSO with ethtool fixed it (tested on 2.6.0-test11).  At
least we have a workaround now.

Thanks Scott, Harald and Dave.

Is it not supported by the hardware?  Seems TSO could improve
performance a bit since the 1000/MT Desktop is starved for PCI bandwidth
at 32-bit/33MHz.

Thanks,
Stephen

Re: Extremely slow network with e1000 & ip_conntrack

[David S. Miller]

On Thu, 04 Dec 2003 21:36:09 +0900
Stephen Lee <mukansai@emailplus.org> wrote:

> "Feldman, Scott" <scott.feldman@intel.com> wrote:
> > 
> > Try turning off TSO by disabling this code or by using "ethtool -K tso
> > off" (need version 1.8).
> 
> Yes, turning off TSO with ethtool fixed it (tested on 2.6.0-test11).  At
> least we have a workaround now.
> 
> Thanks Scott, Harald and Dave.
> 
> Is it not supported by the hardware?  Seems TSO could improve
> performance a bit since the 1000/MT Desktop is starved for PCI bandwidth
> at 32-bit/33MHz.

This workaround explains everything.  The TSO packets have to be
"un-TSO'd" in order for netfilter to look at the packet and parse
the contents.  This means copying all the data around, allocating
several networking buffers, etc.

Re: Extremely slow network with e1000 & ip_conntrack

[David S. Miller]

On Thu, 04 Dec 2003 21:36:09 +0900
Stephen Lee <mukansai@emailplus.org> wrote:

> "Feldman, Scott" <scott.feldman@intel.com> wrote:
> > 
> > Try turning off TSO by disabling this code or by using "ethtool -K tso
> > off" (need version 1.8).
> 
> Yes, turning off TSO with ethtool fixed it (tested on 2.6.0-test11).  At
> least we have a workaround now.

OK, I've found out what IP conntack does that creates the problems.

In fact, it's a bug in conntrack and this ends up corrupting the TSO
packet.  This forces TSO-disabling on that connection, and
retransmission of all the data.  Then the data flows correctly so TSO
is re-enabled, and on and on and on like this.  Performance goes into
the toilet.

The culprit is net/ipv4/netfilter/ip_conntrack_standalone.c,
in ip_refrag(), it does this:

        if ((*pskb)->len > dst_pmtu(&rt->u.dst)) {
                /* No hook can be after us, so this should be OK. */
                ip_fragment(*pskb, okfn);
                return NF_STOLEN;
        }

Which fragments TSO packets, oops :)

People can confirm this analysis by applying the patch below, enabling
TSO with conntrack loaded, and see if the problem goes away.

Some auditing is definitely necessary wrt. TSO and netfilter.  In particular
I am incredibly confident that we have issues in cases like when the FTP
netfilter modules mangle the data.  Another area for inspection are the
cases where TCP header bits are changed and thus the checksum needs to
be adjusted.

===== net/ipv4/netfilter/ip_conntrack_standalone.c 1.22 vs edited =====
--- 1.22/net/ipv4/netfilter/ip_conntrack_standalone.c	Thu Oct  2 23:21:19 2003
+++ edited/net/ipv4/netfilter/ip_conntrack_standalone.c	Fri Dec  5 12:25:22 2003
@@ -201,7 +201,8 @@
 	/* Local packets are never produced too large for their
 	   interface.  We degfragment them at LOCAL_OUT, however,
 	   so we have to refragment them here. */
-	if ((*pskb)->len > dst_pmtu(&rt->u.dst)) {
+	if ((*pskb)->len > dst_pmtu(&rt->u.dst) &&
+	    !skb_shinfo(*pskb)->tso_size) {
 		/* No hook can be after us, so this should be OK. */
 		ip_fragment(*pskb, okfn);
 		return NF_STOLEN;

Re: Extremely slow network with e1000 & ip_conntrack

[Stephen Lee]

"David S. Miller" <davem@redhat.com> wrote:
> Stephen Lee <mukansai@emailplus.org> wrote:
> > Yes, turning off TSO with ethtool fixed it (tested on 2.6.0-test11).  At
> > least we have a workaround now.
> 
> This workaround explains everything.  The TSO packets have to be
> "un-TSO'd" in order for netfilter to look at the packet and parse
> the contents.  This means copying all the data around, allocating
> several networking buffers, etc.

Sorry if I am talking out of my ass, but can this be solved in one
of the following ways?  (But it would seem 2, 3 or 4 are not 2.6
material).

(1) Turn off TSO altogether (Duh).

(2) Do what you suggest, but pass the original TSO packet to the
ethernet chip.  Still have to copy around, but save us some interrupts? 
(Could introduce subtle bugs if we un-TSO it differently from the
hardware).

(3) Just pass the original packet to netfilter with a special flag and
have netfilter "deduce" what the rest of the headers are.

(4) Similar idea, but totally separate the headers and the payload.  (I
think this was suggested somewhere for some other problem...  I read it
in one of the archives but can't find it now)  Then we can just generate
a set of "un-TSO'd" headers (with appropriate pointers into the big
payload) for netfilter to look at, but leave the original for passing to
hardware eventually.

Stephen

Re: Extremely slow network with e1000 & ip_conntrack

[Stephen Lee]

"David S. Miller" <davem@redhat.com> wrote:
> 
> OK, I've found out what IP conntack does that creates the problems.
> [...]
> People can confirm this analysis by applying the patch below, enabling
> TSO with conntrack loaded, and see if the problem goes away.

I tested it with both e1000 & tg3, TSO on, and it is working fine for me
using ftp and scp.

Thanks,
Stephen

Re: Extremely slow network with e1000 & ip_conntrack

[David S. Miller]

On Sat, 06 Dec 2003 07:20:09 +0900
Stephen Lee <mukansai@emailplus.org> wrote:

> "David S. Miller" <davem@redhat.com> wrote:
> > 
> > OK, I've found out what IP conntack does that creates the problems.
> > [...]
> > People can confirm this analysis by applying the patch below, enabling
> > TSO with conntrack loaded, and see if the problem goes away.
> 
> I tested it with both e1000 & tg3, TSO on, and it is working fine for me
> using ftp and scp.

Great, I'll push the fix to Linus.

Thanks.

Re: Extremely slow network with e1000 & ip_conntrack

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 1284 bytes --]

On Fri, Dec 05, 2003 at 12:28:19PM -0800, David S. Miller wrote:

> The culprit is net/ipv4/netfilter/ip_conntrack_standalone.c,
> in ip_refrag(), it does this:
> 

Sorry for getting back to you so late, but as indicated before, I was
offline while travelling during the last week.

Thanks for spotting and fixing the bug.

> Some auditing is definitely necessary wrt. TSO and netfilter.  In particular
> I am incredibly confident that we have issues in cases like when the FTP
> netfilter modules mangle the data.  Another area for inspection are the
> cases where TCP header bits are changed and thus the checksum needs to
> be adjusted.

yes, this is certainly a problem - but not with conntrack, only with
nat.  So maybe we should add a safeguard, preventing
iptables_nat/ipchains/ipfwadm from being loaded when TSO on any
interface is enabled?  Or at least print a warining in syslog?

-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Extremely slow network with e1000 & ip_conntrack

[Henrik Nordstrom]

On Thu, 11 Dec 2003, Harald Welte wrote:

> yes, this is certainly a problem - but not with conntrack, only with
> nat.  So maybe we should add a safeguard, preventing
> iptables_nat/ipchains/ipfwadm from being loaded when TSO on any
> interface is enabled?  Or at least print a warining in syslog?

TSO can be enabled while NAT is running so you better do this in the 
packet flow or if there is a suitable notifier hook that can be used.

Most firewalls etc load the ruleset before activating the 
interfaces, i..e before even loading the nic drivers, so there is no 
interfaces to look at when iptables_nat is loaded.

Regards
Henrik

TSO and netfilter (Re: Extremely slow network with e1000 & ip_conntrack)

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 2121 bytes --]

On Fri, Dec 05, 2003 at 12:28:19PM -0800, David S. Miller wrote:

> Some auditing is definitely necessary wrt. TSO and netfilter.  In particular
> I am incredibly confident that we have issues in cases like when the FTP
> netfilter modules mangle the data.

I didn't have a look into how TSO is implemented until today.  From my
naive point of view, I cannot think of any issues.  From a netfilter
point of view, a TSO-enabled skb just looks like a single large packet,
right?

I mean, the TSO-enabled skb still contains a fully valid IP and TCP
packet.  If we do any changes to the IP header or tcp header bits, or
even to the payload of the packet, this happens before the TSO-enabled
driver and the network board start creating multiple tcp/ip datagrams
from this skb (by using the information present in the
netfilter-modified ip/tcp headers).

The only interesting case is in ip_output.c:ip_queue_xmit(), where
tso_size and tso_segs are calculated, before NF_IP_LOCAL_OUT is run.

But changing the content or the size of the tcp payload should not
affect those calculations. 

A real problem would be resizing the TCP header (where th.doff is
affected).  But I cannot think of any case where any of the current
netfilter/iptables/conntrack/nat code does that.  Even in the past, when
we used to remove SACKPERM from the tcp header, we just NOP'ed it out
instead of resizing the header.

> Another area for inspection are the cases where TCP header bits are
> changed and thus the checksum needs to be adjusted.

Why is this a problem?  The netfilter code has to adjust the checksum
anyway... or is the checksum calculation for TSO-enabled skb's
different?

Please enlighten me if I have missed something.

-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSO and netfilter (Re: Extremely slow network with e1000 & ip_conntrack)

[David S. Miller]

On Thu, 11 Dec 2003 12:03:15 +0100
Harald Welte <laforge@netfilter.org> wrote:

> The only interesting case is in ip_output.c:ip_queue_xmit(), where
> tso_size and tso_segs are calculated, before NF_IP_LOCAL_OUT is run.
> 
> But changing the content or the size of the tcp payload should not
> affect those calculations. 

It changes at least tso_segs, since if you decrease of increase the
size of the payload the number of real TCP/IP packets the TSO engine
will end up spitting out could be different.

The one netfilter module I'm most concerned about is the one that
handles non-passive FTP, I remember that one did strange things with
the data stream, removed TCP options, and stuff like that.

> A real problem would be resizing the TCP header (where th.doff is
> affected).  But I cannot think of any case where any of the current
> netfilter/iptables/conntrack/nat code does that.

As mentioned above, I thought the netfilter module handling non-passive
FTP stripped TCP options.

> Even in the past, when
> we used to remove SACKPERM from the tcp header, we just NOP'ed it out
> instead of resizing the header.

This may be what I was thinking about.

> > Another area for inspection are the cases where TCP header bits are
> > changed and thus the checksum needs to be adjusted.
> 
> Why is this a problem?  The netfilter code has to adjust the checksum
> anyway... or is the checksum calculation for TSO-enabled skb's
> different?

Currently all the TSO supporting drivers set the ip and tcp header
checksum values themselves as appropriate, so there are no worries in
this area.

Re: TSO and netfilter (Re: Extremely slow network with e1000 & ip_conntrack)

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 2989 bytes --]

On Thu, Dec 11, 2003 at 05:41:36PM -0800, David S. Miller wrote:
> On Thu, 11 Dec 2003 12:03:15 +0100
> Harald Welte <laforge@netfilter.org> wrote:
> 
> > The only interesting case is in ip_output.c:ip_queue_xmit(), where
> > tso_size and tso_segs are calculated, before NF_IP_LOCAL_OUT is run.
> > 
> > But changing the content or the size of the tcp payload should not
> > affect those calculations. 
> 
> It changes at least tso_segs, since if you decrease of increase the
> size of the payload the number of real TCP/IP packets the TSO engine
> will end up spitting out could be different.

I see.  So what about the networking core exporting an [inline] function
that recalculates tso_segs and tso_size (like the 'Hack zone' code
fragment in ip_queue_xmit() right now), called skb_tso_recalc() or
whatever name you prefer.

Or even better (since I assume TSO can only happen with
locally-originated datagrams), why don't we move the tso_size/tso_segs
calculation to happen after the LOCAL_OUT netfilter hook?  This way we
also get the ip_select_ident_more() right, which we couldn't easily
update from the proposed skb_tso_recalc() function.

yes, in that case we would need to have some fake code like
	if (skb->len > mtu && (sk->sk_route_caps&NETIF_F_TSO))
		skb_shinfo(skb)->tso_segs = 1;
in order to make the newly-created check for refragmentation in
conntrack still work.  Alternatively, create some inline function that 
gives a yes/no return if the skb would later become TSO or not.

> The one netfilter module I'm most concerned about is the one that
> handles non-passive FTP, I remember that one did strange things with
> the data stream, removed TCP options, and stuff like that.

There are no NAT helper modules as of now that touch the size of the TCP
header.  We have some experimental stuff in patch-o-matic (like
IPV3OPTSSTRIP target), but nothing in the stock kernel.  I will put a
review of those on our TODO list - but for the vanilla kernel there
shouldn't be a problem.

> > Even in the past, when we used to remove SACKPERM from the tcp
> > header, we just NOP'ed it out instead of resizing the header.
> 
> This may be what I was thinking about.

We now don't do that anymore and mangle the SACK options accordingly.
In any way, there is nothing that changes the size of the tcp or udp
header.  

> Currently all the TSO supporting drivers set the ip and tcp header
> checksum values themselves as appropriate, so there are no worries in
> this area.

good news.

Please get back to me with any comments you might have, thanks.

-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSO and netfilter (Re: Extremely slow network with e1000 & ip_conntrack)

[David S. Miller]

On Fri, 12 Dec 2003 08:01:31 +0100
Harald Welte <laforge@netfilter.org> wrote:

> So what about the networking core exporting an [inline] function
> that recalculates tso_segs and tso_size (like the 'Hack zone' code
> fragment in ip_queue_xmit() right now), called skb_tso_recalc() or
> whatever name you prefer.

This might work.

> Or even better (since I assume TSO can only happen with
> locally-originated datagrams), why don't we move the tso_size/tso_segs
> calculation to happen after the LOCAL_OUT netfilter hook?  This way we
> also get the ip_select_ident_more() right, which we couldn't easily
> update from the proposed skb_tso_recalc() function.
>
> yes, in that case we would need to have some fake code like
> 	if (skb->len > mtu && (sk->sk_route_caps&NETIF_F_TSO))
> 		skb_shinfo(skb)->tso_segs = 1;
> in order to make the newly-created check for refragmentation in
> conntrack still work.  Alternatively, create some inline function that 
> gives a yes/no return if the skb would later become TSO or not.

I don't know about this.  The local-out hook always had a fully
functional finalized packet to work with, and I doubt we should change
that.

Also, dst_output() might invoke IPSEC encapsulators which absolutely
must have the final packet in hand when they run (f.e. you can't
choose the IP ID after encryption of the IP header).

Anyways, that leaves us with the helper function idea, does this
(untested) look like what you want?

--- include/linux/skbuff.h.~1~	Thu Dec 11 23:55:43 2003
+++ include/linux/skbuff.h	Thu Dec 11 23:57:45 2003
@@ -1155,6 +1155,17 @@
 #endif
 }
 
+static __inline__ void skb_tso_recalc(struct sk_buff *skb, struct dst_entry *dst)
+{
+	unsigned int hlen = ((skb->h.raw-skb->data)+(skb->h.th->doff<<2));
+	u32 mtu = dst_pmtu(dst);;
+
+	skb_shinfo(skb)->tso_size = mtu - hlen;
+	skb_shinfo(skb)->tso_segs =
+		(skb->len - hlen + skb_shinfo(skb)->tso_size - 1) /
+		skb_shinfo(skb)->tso_size - 1;
+}
+
 #define skb_queue_walk(queue, skb) \
 		for (skb = (queue)->next, prefetch(skb->next);	\
 		     (skb != (struct sk_buff *)(queue));	\

Re: Extremely slow network with e1000 & ip_conntrack

[Stephen Lee]

Hi,

Okay, I reproduced it with tg3 on 2.6.0-test11 as well, by turning on
TSO with ethtool.  So it does seem to be an incompatibility between TSO
& conntrack.

"David S. Miller" <davem@redhat.com> wrote:
> This workaround explains everything.  The TSO packets have to be
> "un-TSO'd" in order for netfilter to look at the packet and parse
> the contents.  This means copying all the data around, allocating
> several networking buffers, etc.

But, if netfilter needs to see all the contents in order to function,
shouldn't TSO break other parts of netfilter as well?  Why is only
conntrack affected?

Jeff Garzik <jgarzik@pobox.com> wrote:
> Regardless, the problem is identified -- when TSO+conntrack is present,
> we "TSO" net traffic then "un-TSO" it.

This "un-TSO"ing more or less defeats the purpose of TSO, right?  

> It's simply better to not use TSO at all, for this case.

If that is the case, I suppose so.

Thanks,
Stephen

Re: Extremely slow network with e1000 & ip_conntrack

[David S. Miller]

On Fri, 05 Dec 2003 04:53:33 +0900
Stephen Lee <mukansai@emailplus.org> wrote:

> I have a 32-bit PCI card with bcm5705 on it, does that support TSO? 
> I'll give it a try today.

Yep it does, but it is not enabled by default.  You have
to enable it explicitly using ethtool.

Re: Extremely slow network with e1000 & ip_conntrack

[Jeff Garzik]

Stephen Lee wrote:
> I have a 32-bit PCI card with bcm5705 on it, does that support TSO? 
> I'll give it a try today.


I'm pretty sure tg3 does not enable TSO support by default.

Regardless, the problem is identified -- when TSO+conntrack is present,
we "TSO" net traffic then "un-TSO" it.

It's simply better to not use TSO at all, for this case.

	Jeff

Re: Extremely slow network with e1000 & ip_conntrack

[Stephen Lee]

"Feldman, Scott" <scott.feldman@intel.com> wrote:
> 
> TSO is support on 82540.  Turning off TSO is a workaround, but what's
> behind the dependency of TSO and ip_conntrack?  You indicated in an
> earlier note that having the ip_conntrack module loaded was a factor.
> Do you have a nic to try with tg3?  I believe tg3 has TSO enabled as
> well (in 2.6.0-test11).

I have a 32-bit PCI card with bcm5705 on it, does that support TSO? 
I'll give it a try today.

Stephen

Re: Extremely slow network with e1000 & ip_conntrack

[David S. Miller]

On Thu, 4 Dec 2003 09:37:19 -0800
"Feldman, Scott" <scott.feldman@intel.com> wrote:

> TSO is support on 82540.  Turning off TSO is a workaround, but what's
> behind the dependency of TSO and ip_conntrack?

Netfilter wants to see the _real_ packets that will be sent onto the
wire.  TSO is a template by which to create such packets, not the real
thing.

So when and if we go into netfilter, we must un-TSO the packet so
that netfilter can look at what it really wants to.

RE: Extremely slow network with e1000 & ip_conntrack

[Feldman, Scott]

> Is it not supported by the hardware?  Seems TSO could improve 
> performance a bit since the 1000/MT Desktop is starved for 
> PCI bandwidth at 32-bit/33MHz.

TSO is support on 82540.  Turning off TSO is a workaround, but what's
behind the dependency of TSO and ip_conntrack?  You indicated in an
earlier note that having the ip_conntrack module loaded was a factor.
Do you have a nic to try with tg3?  I believe tg3 has TSO enabled as
well (in 2.6.0-test11).

-scott

RE: Extremely slow network with e1000 & ip_conntrack

[Feldman, Scott]

> > > > 2.4.22		82540EM		N
> > > > 2.5.26		82540EM		N
> > > > 2.5.46		82540EM		Y
> > > > 2.6.0-test10		82540EM		Y
> > > > 2.6.0-test11		82540EM		Y
> > > > 2.6.0-test11		82547EI		N
> > > > 2.4.22nptlsmp		82547EI		N
> 
> In e1000, check this out:
> 
>   #ifdef NETIF_F_TSO
>           if((adapter->hw.mac_type >= e1000_82544) &&
>              (adapter->hw.mac_type != e1000_82547))
>                   netdev->features |= NETIF_F_TSO;
>   #endif

Oh, I forgot one thing: 82540 is actually greater than 82544.  The
comparison is chronological order, not numerical.  This is the order the
controllers where released:

82542
82543
82544
82540
82545
82546
82541
82547

Don't ask.

-scott

Re: Extremely slow network with e1000 & ip_conntrack

[David S. Miller]

On Tue, 02 Dec 2003 20:44:06 +0900
Stephen Lee <mukansai@emailplus.org> wrote:

> After lots of trial-and-error, I can make the problem disappear by
> backing these 4 files out from 2.5.44 to their 2.5.43 versions.
> 
> net/ipv4/icmp.c
> net/ipv4/ip_output.c
> net/ipv4/raw.c
> net/ipv4/udp.c

Interesting, but not very.  This essentially backs out the whole
ipv4 packet sending engine rewrite we did to support IPSEC and
UDP sendfile support.

Ie. it's a lot of large interrelated changes.  We know now what
introduced the problem, but this hasn't really narrowed it down
much.

Re: Extremely slow network with e1000 & ip_conntrack

[Stephen Lee]

[-- Attachment #1: Type: text/plain, Size: 2034 bytes --]

Harald Welte <laforge@netfilter.org> wrote:
> 
> Well, the problem is certainly triggered by connection tracking.  

Thanks for the reply, and for forwarding to lkml.

To recap for lkml (I do not subscribe to lkml, so please CC either me or
netfilter-devel):

e1000 becomes extremely slow for traffic generated from the box.  I
tested by issuing an ftp from another box and doing a "get", but other
kind of traffic (scp, samba) are similarly affected.  The speed drops to
~40kbps.  This happens when ip_conntrack is loaded, even with no rule in
the tables.  Unloading the module makes the speed recover, but I'd like
to make use of state tracking.

There is no change in the "put" direction.  I didn't notice any change
in speed of traffic routed through the box, though I did not test that
case sufficiently.

This bug does not appear on all chips supported by the driver.  Please
see previous email for the 2 cases I tested.

> If I understood correctly:
> 
> 1) stock 2.5.43: OK
> 2) stock 2.5.44: PROBLEM
> 3) stock 2.5.44 with the driver from 2.5.43: PROBLEM
> 4) stock 2.5.44 with ip_conntrack.o from 2.5.43: PROBLEM

Correct.  The problem exists still on 2.6.0-test11.

> So it has to be a change outside of the e1000 driver and outside of the
> connection tracking code.
> 
> In the meantime, I think taking the discussion back to lkml seems to be
> a good idea - since neither e1000 nor ip_conntrack code seem to be the
> direct cause of the problem.

After lots of trial-and-error, I can make the problem disappear by
backing these 4 files out from 2.5.44 to their 2.5.43 versions.

net/ipv4/icmp.c
net/ipv4/ip_output.c
net/ipv4/raw.c
net/ipv4/udp.c

Note that in both cases I needed to #include <linux/netfilter_ipv4.h>
rather than <linux/netfilter.h> in net/ipv4/raw.c to get it to compile.

I have attached my .config, though note that I have reproduced the
problem on another motherboard also (intel P4 on 865G running
2.6.0-test11).

gcc 3.3.2 was used on the P3 box and 3.2.1 was used on the P4 box.

Regards,
Stephen

[-- Attachment #2: config --]
[-- Type: application/octet-stream, Size: 21870 bytes --]

#
# Automatically generated make config: don't edit
#
CONFIG_X86=y
# CONFIG_SBUS is not set
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y

#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y

#
# General setup
#
CONFIG_NET=y
CONFIG_SYSVIPC=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_SYSCTL=y

#
# Loadable module support
#
CONFIG_MODULES=y
# CONFIG_MODVERSIONS is not set
CONFIG_KMOD=y

#
# Processor type and features
#
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
CONFIG_MPENTIUMIII=y
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MELAN is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MCYRIXIII is not set
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INVLPG=y
CONFIG_X86_CMPXCHG=y
CONFIG_X86_XADD=y
CONFIG_X86_BSWAP=y
CONFIG_X86_POPAD_OK=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_X86_L1_CACHE_SHIFT=5
CONFIG_X86_TSC=y
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_HUGETLB_PAGE=y
# CONFIG_SMP is not set
CONFIG_PREEMPT=y
CONFIG_X86_UP_APIC=y
CONFIG_X86_UP_IOAPIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_MCE=y
# CONFIG_X86_MCE_NONFATAL is not set
# CONFIG_X86_MCE_P4THERMAL is not set
# CONFIG_CPU_FREQ is not set
# CONFIG_TOSHIBA is not set
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
CONFIG_X86_MSR=m
CONFIG_X86_CPUID=m
# CONFIG_EDD is not set
CONFIG_NOHIGHMEM=y
# CONFIG_HIGHMEM4G is not set
# CONFIG_HIGHMEM64G is not set
# CONFIG_MATH_EMULATION is not set
CONFIG_MTRR=y
CONFIG_HAVE_DEC_LOCK=y

#
# Power management options (ACPI, APM)
#

#
# ACPI Support
#
CONFIG_ACPI=y
# CONFIG_ACPI_HT_ONLY is not set
CONFIG_ACPI_BOOT=y
# CONFIG_ACPI_SLEEP is not set
CONFIG_ACPI_AC=y
CONFIG_ACPI_BATTERY=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_FAN=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_TOSHIBA is not set
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_BOOT=y
CONFIG_ACPI_BUS=y
CONFIG_ACPI_INTERPRETER=y
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_PCI=y
# CONFIG_ACPI_SLEEP is not set
CONFIG_ACPI_SYSTEM=y
CONFIG_PM=y
# CONFIG_APM is not set

#
# Bus options (PCI, PCMCIA, EISA, MCA, ISA)
#
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
# CONFIG_SCx200 is not set
# CONFIG_PCI_NAMES is not set
# CONFIG_ISA is not set
# CONFIG_EISA is not set
# CONFIG_MCA is not set
CONFIG_HOTPLUG=y

#
# PCMCIA/CardBus support
#
# CONFIG_PCMCIA is not set

#
# PCI Hotplug Support
#
# CONFIG_HOTPLUG_PCI is not set
# CONFIG_HOTPLUG_PCI_COMPAQ is not set
# CONFIG_HOTPLUG_PCI_COMPAQ_NVRAM is not set
# CONFIG_HOTPLUG_PCI_IBM is not set

#
# Executable file formats
#
CONFIG_KCORE_ELF=y
# CONFIG_KCORE_AOUT is not set
# CONFIG_BINFMT_AOUT is not set
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_MISC=m

#
# Memory Technology Devices (MTD)
#
# CONFIG_MTD is not set

#
# Parallel port support
#
# CONFIG_PARPORT is not set

#
# Plug and Play configuration
#
# CONFIG_PNP is not set
# CONFIG_PNP_NAMES is not set
# CONFIG_PNP_DEBUG is not set

#
# Protocols
#
# CONFIG_ISAPNP is not set
# CONFIG_PNPBIOS is not set

#
# Block devices
#
CONFIG_BLK_DEV_FD=y
# CONFIG_BLK_DEV_XD is not set
# CONFIG_PARIDE is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_CISS_SCSI_TAPE is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_RAM is not set
# CONFIG_BLK_DEV_INITRD is not set
CONFIG_LBD=y

#
# ATA/ATAPI/MFM/RLL device support
#
CONFIG_IDE=y

#
# IDE, ATA and ATAPI Block devices
#
CONFIG_BLK_DEV_IDE=y

#
# Please see Documentation/ide.txt for help/info on IDE drives
#
# CONFIG_BLK_DEV_HD_IDE is not set
# CONFIG_BLK_DEV_HD is not set
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_IDEDISK_MULTI_MODE=y
# CONFIG_IDEDISK_STROKE is not set
# CONFIG_BLK_DEV_IDECS is not set
CONFIG_BLK_DEV_IDECD=y
# CONFIG_BLK_DEV_IDEFLOPPY is not set
# CONFIG_BLK_DEV_IDESCSI is not set
# CONFIG_IDE_TASK_IOCTL is not set

#
# IDE chipset support/bugfixes
#
# CONFIG_BLK_DEV_CMD640 is not set
# CONFIG_BLK_DEV_CMD640_ENHANCED is not set
# CONFIG_BLK_DEV_ISAPNP is not set
CONFIG_BLK_DEV_IDEPCI=y
CONFIG_BLK_DEV_GENERIC=y
CONFIG_IDEPCI_SHARE_IRQ=y
CONFIG_BLK_DEV_IDEDMA_PCI=y
CONFIG_BLK_DEV_IDE_TCQ=y
CONFIG_BLK_DEV_IDE_TCQ_DEFAULT=y
CONFIG_BLK_DEV_IDE_TCQ_DEPTH=32
# CONFIG_BLK_DEV_OFFBOARD is not set
# CONFIG_BLK_DEV_IDEDMA_FORCED is not set
CONFIG_IDEDMA_PCI_AUTO=y
# CONFIG_IDEDMA_ONLYDISK is not set
CONFIG_BLK_DEV_IDEDMA=y
# CONFIG_IDEDMA_PCI_WIP is not set
# CONFIG_IDEDMA_NEW_DRIVE_LISTINGS is not set
CONFIG_BLK_DEV_ADMA=y
# CONFIG_BLK_DEV_AEC62XX is not set
# CONFIG_BLK_DEV_ALI15X3 is not set
# CONFIG_WDC_ALI15X3 is not set
# CONFIG_BLK_DEV_AMD74XX is not set
# CONFIG_AMD74XX_OVERRIDE is not set
# CONFIG_BLK_DEV_CMD64X is not set
# CONFIG_BLK_DEV_CY82C693 is not set
# CONFIG_BLK_DEV_CS5530 is not set
# CONFIG_BLK_DEV_HPT34X is not set
# CONFIG_HPT34X_AUTODMA is not set
# CONFIG_BLK_DEV_HPT366 is not set
# CONFIG_BLK_DEV_PIIX is not set
# CONFIG_BLK_DEV_NFORCE is not set
# CONFIG_BLK_DEV_NS87415 is not set
# CONFIG_BLK_DEV_OPTI621 is not set
# CONFIG_BLK_DEV_PDC202XX_OLD is not set
# CONFIG_PDC202XX_BURST is not set
# CONFIG_BLK_DEV_PDC202XX_NEW is not set
# CONFIG_PDC202XX_FORCE is not set
# CONFIG_BLK_DEV_RZ1000 is not set
# CONFIG_BLK_DEV_SVWKS is not set
# CONFIG_BLK_DEV_SIIMAGE is not set
# CONFIG_BLK_DEV_SIS5513 is not set
# CONFIG_BLK_DEV_SLC90E66 is not set
# CONFIG_BLK_DEV_TRM290 is not set
CONFIG_BLK_DEV_VIA82CXXX=y
# CONFIG_IDE_CHIPSETS is not set
CONFIG_IDEDMA_AUTO=y
# CONFIG_IDEDMA_IVB is not set
# CONFIG_DMA_NONPCI is not set
CONFIG_BLK_DEV_IDE_MODES=y

#
# SCSI device support
#
# CONFIG_SCSI is not set

#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
# CONFIG_MD_LINEAR is not set
# CONFIG_MD_RAID0 is not set
CONFIG_MD_RAID1=y
# CONFIG_MD_RAID5 is not set
# CONFIG_MD_MULTIPATH is not set
# CONFIG_BLK_DEV_LVM is not set

#
# Fusion MPT device support
#
# CONFIG_FUSION is not set
# CONFIG_FUSION_BOOT is not set
# CONFIG_FUSION_ISENSE is not set
# CONFIG_FUSION_CTL is not set
# CONFIG_FUSION_LAN is not set

#
# IEEE 1394 (FireWire) support (EXPERIMENTAL)
#
# CONFIG_IEEE1394 is not set

#
# I2O device support
#
# CONFIG_I2O is not set
# CONFIG_I2O_PCI is not set
# CONFIG_I2O_BLOCK is not set
# CONFIG_I2O_LAN is not set
# CONFIG_I2O_SCSI is not set
# CONFIG_I2O_PROC is not set

#
# Networking options
#
CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
CONFIG_NETLINK_DEV=y
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
CONFIG_FILTER=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_FWMARK=y
CONFIG_IP_ROUTE_NAT=y
CONFIG_IP_ROUTE_MULTIPATH=y
CONFIG_IP_ROUTE_TOS=y
CONFIG_IP_ROUTE_VERBOSE=y
# CONFIG_IP_ROUTE_LARGE_TABLES is not set
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=y
CONFIG_NET_IPGRE=y
CONFIG_NET_IPGRE_BROADCAST=y
# CONFIG_IP_MROUTE is not set
# CONFIG_ARPD is not set
CONFIG_INET_ECN=y
CONFIG_SYN_COOKIES=y

#
#   IP: Netfilter Configuration
#
CONFIG_IP_NF_CONNTRACK=m
CONFIG_IP_NF_FTP=m
CONFIG_IP_NF_IRC=m
# CONFIG_IP_NF_QUEUE is not set
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_LIMIT=m
CONFIG_IP_NF_MATCH_MAC=m
CONFIG_IP_NF_MATCH_PKTTYPE=m
CONFIG_IP_NF_MATCH_MARK=m
CONFIG_IP_NF_MATCH_MULTIPORT=m
CONFIG_IP_NF_MATCH_TOS=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_DSCP=m
CONFIG_IP_NF_MATCH_AH_ESP=m
CONFIG_IP_NF_MATCH_LENGTH=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_MATCH_TCPMSS=m
CONFIG_IP_NF_MATCH_HELPER=m
CONFIG_IP_NF_MATCH_STATE=m
CONFIG_IP_NF_MATCH_CONNTRACK=m
CONFIG_IP_NF_MATCH_UNCLEAN=m
CONFIG_IP_NF_MATCH_OWNER=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_MIRROR=m
CONFIG_IP_NF_NAT=m
CONFIG_IP_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_REDIRECT=m
# CONFIG_IP_NF_NAT_LOCAL is not set
CONFIG_IP_NF_NAT_SNMP_BASIC=m
CONFIG_IP_NF_NAT_IRC=m
CONFIG_IP_NF_NAT_FTP=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_TOS=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_DSCP=m
CONFIG_IP_NF_TARGET_MARK=m
CONFIG_IP_NF_TARGET_LOG=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_IP_NF_TARGET_TCPMSS=m
# CONFIG_IP_NF_ARPTABLES is not set
# CONFIG_IP_NF_COMPAT_IPCHAINS is not set
# CONFIG_IP_NF_COMPAT_IPFWADM is not set
CONFIG_IPV6=y

#
#   IPv6: Netfilter Configuration
#
# CONFIG_IP6_NF_QUEUE is not set
CONFIG_IP6_NF_IPTABLES=m
CONFIG_IP6_NF_MATCH_LIMIT=m
CONFIG_IP6_NF_MATCH_MAC=m
CONFIG_IP6_NF_MATCH_MULTIPORT=m
CONFIG_IP6_NF_MATCH_OWNER=m
CONFIG_IP6_NF_MATCH_MARK=m
CONFIG_IP6_NF_MATCH_LENGTH=m
CONFIG_IP6_NF_MATCH_EUI64=m
CONFIG_IP6_NF_FILTER=m
CONFIG_IP6_NF_TARGET_LOG=m
CONFIG_IP6_NF_MANGLE=m
CONFIG_IP6_NF_TARGET_MARK=m

#
#    SCTP Configuration (EXPERIMENTAL)
#
CONFIG_IPV6_SCTP__=y
# CONFIG_IP_SCTP is not set
# CONFIG_ATM is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_LLC is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_DEV_APPLETALK is not set
# CONFIG_DECNET is not set
# CONFIG_BRIDGE is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_NET_DIVERT is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_FASTROUTE is not set
# CONFIG_NET_HW_FLOWCONTROL is not set

#
# QoS and/or fair queueing
#
CONFIG_NET_SCHED=y
CONFIG_NET_SCH_CBQ=m
CONFIG_NET_SCH_HTB=m
CONFIG_NET_SCH_CSZ=m
CONFIG_NET_SCH_PRIO=m
CONFIG_NET_SCH_RED=m
CONFIG_NET_SCH_SFQ=m
CONFIG_NET_SCH_TEQL=m
CONFIG_NET_SCH_TBF=m
CONFIG_NET_SCH_GRED=m
# CONFIG_NET_SCH_DSMARK is not set
CONFIG_NET_SCH_INGRESS=m
CONFIG_NET_QOS=y
CONFIG_NET_ESTIMATOR=y
CONFIG_NET_CLS=y
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_ROUTE=y
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
CONFIG_NET_CLS_POLICE=y

#
# Network device support
#
CONFIG_NETDEVICES=y

#
# ARCnet devices
#
# CONFIG_ARCNET is not set
CONFIG_DUMMY=m
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
CONFIG_TUN=m
# CONFIG_ETHERTAP is not set

#
# Ethernet (10 or 100Mbit)
#
CONFIG_NET_ETHERNET=y
# CONFIG_SUNLANCE is not set
# CONFIG_HAPPYMEAL is not set
# CONFIG_SUNBMAC is not set
# CONFIG_SUNQE is not set
# CONFIG_SUNGEM is not set
# CONFIG_NET_VENDOR_3COM is not set
# CONFIG_LANCE is not set
# CONFIG_NET_VENDOR_SMC is not set
# CONFIG_NET_VENDOR_RACAL is not set

#
# Tulip family network device support
#
CONFIG_NET_TULIP=y
# CONFIG_DE2104X is not set
CONFIG_TULIP=m
# CONFIG_TULIP_MWI is not set
# CONFIG_TULIP_MMIO is not set
# CONFIG_DE4X5 is not set
# CONFIG_WINBOND_840 is not set
# CONFIG_DM9102 is not set
# CONFIG_HP100 is not set
# CONFIG_NET_ISA is not set
CONFIG_NET_PCI=y
# CONFIG_PCNET32 is not set
# CONFIG_ADAPTEC_STARFIRE is not set
# CONFIG_APRICOT is not set
# CONFIG_CS89x0 is not set
# CONFIG_DGRS is not set
# CONFIG_EEPRO100 is not set
CONFIG_E100=m
# CONFIG_LNE390 is not set
# CONFIG_FEALNX is not set
CONFIG_NATSEMI=m
# CONFIG_NE2K_PCI is not set
# CONFIG_NE3210 is not set
# CONFIG_ES3210 is not set
# CONFIG_8139CP is not set
CONFIG_8139TOO=m
# CONFIG_8139TOO_PIO is not set
# CONFIG_8139TOO_TUNE_TWISTER is not set
# CONFIG_8139TOO_8129 is not set
# CONFIG_8139_OLD_RX_RESET is not set
# CONFIG_SIS900 is not set
# CONFIG_EPIC100 is not set
# CONFIG_SUNDANCE is not set
# CONFIG_SUNDANCE_MMIO is not set
# CONFIG_TLAN is not set
# CONFIG_VIA_RHINE is not set
# CONFIG_VIA_RHINE_MMIO is not set
# CONFIG_NET_POCKET is not set

#
# Ethernet (1000 Mbit)
#
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
CONFIG_E1000=m
CONFIG_E1000_NAPI=y
# CONFIG_MYRI_SBUS is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_SK98LIN is not set
# CONFIG_TIGON3 is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PLIP is not set
CONFIG_PPP=m
CONFIG_PPP_MULTILINK=y
# CONFIG_PPP_FILTER is not set
CONFIG_PPP_ASYNC=m
CONFIG_PPP_SYNC_TTY=m
CONFIG_PPP_DEFLATE=m
CONFIG_PPP_BSDCOMP=m
CONFIG_PPPOE=m
CONFIG_SLIP=m
CONFIG_SLIP_COMPRESSED=y
CONFIG_SLIP_SMART=y
# CONFIG_SLIP_MODE_SLIP6 is not set

#
# Wireless LAN (non-hamradio)
#
# CONFIG_NET_RADIO is not set

#
# Token Ring devices
#
# CONFIG_TR is not set
# CONFIG_NET_FC is not set
# CONFIG_RCPCI is not set
CONFIG_SHAPER=m

#
# Wan interfaces
#
# CONFIG_WAN is not set

#
# Amateur Radio support
#
# CONFIG_HAMRADIO is not set

#
# IrDA (infrared) support
#
# CONFIG_IRDA is not set

#
# ISDN subsystem
#
# CONFIG_ISDN_BOOL is not set

#
# Telephony Support
#
# CONFIG_PHONE is not set
# CONFIG_PHONE_IXJ is not set
# CONFIG_PHONE_IXJ_PCMCIA is not set

#
# Input device support
#
CONFIG_INPUT=y

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_PSAUX=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
CONFIG_INPUT_JOYDEV=m
# CONFIG_INPUT_TSDEV is not set
CONFIG_INPUT_EVDEV=m
# CONFIG_INPUT_EVBUG is not set

#
# Input I/O drivers
#
# CONFIG_GAMEPORT is not set
CONFIG_SOUND_GAMEPORT=y
# CONFIG_GAMEPORT_NS558 is not set
# CONFIG_GAMEPORT_L4 is not set
# CONFIG_GAMEPORT_EMU10K1 is not set
# CONFIG_GAMEPORT_VORTEX is not set
# CONFIG_GAMEPORT_FM801 is not set
# CONFIG_GAMEPORT_CS461x is not set
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PARKBD is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_INPORT is not set
# CONFIG_MOUSE_LOGIBM is not set
# CONFIG_MOUSE_PC110PAD is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_JOYSTICK_ANALOG is not set
# CONFIG_JOYSTICK_A3D is not set
# CONFIG_JOYSTICK_ADI is not set
# CONFIG_JOYSTICK_COBRA is not set
# CONFIG_JOYSTICK_GF2K is not set
# CONFIG_JOYSTICK_GRIP is not set
# CONFIG_JOYSTICK_GRIP_MP is not set
# CONFIG_JOYSTICK_GUILLEMOT is not set
# CONFIG_JOYSTICK_INTERACT is not set
# CONFIG_JOYSTICK_SIDEWINDER is not set
# CONFIG_JOYSTICK_TMDC is not set
# CONFIG_JOYSTICK_IFORCE is not set
# CONFIG_JOYSTICK_WARRIOR is not set
# CONFIG_JOYSTICK_MAGELLAN is not set
# CONFIG_JOYSTICK_SPACEORB is not set
# CONFIG_JOYSTICK_SPACEBALL is not set
# CONFIG_JOYSTICK_STINGER is not set
# CONFIG_JOYSTICK_TWIDDLER is not set
# CONFIG_JOYSTICK_DB9 is not set
# CONFIG_JOYSTICK_GAMECON is not set
# CONFIG_JOYSTICK_TURBOGRAFX is not set
# CONFIG_INPUT_JOYDUMP is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_TOUCHSCREEN_GUNZE is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PCSPKR=y
# CONFIG_INPUT_UINPUT is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_SERIAL_NONSTANDARD is not set

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
# CONFIG_SERIAL_8250_CS is not set
# CONFIG_SERIAL_8250_EXTENDED is not set
# CONFIG_SERIAL_8250_MANY_PORTS is not set
# CONFIG_SERIAL_8250_SHARE_IRQ is not set
# CONFIG_SERIAL_8250_DETECT_IRQ is not set
# CONFIG_SERIAL_8250_MULTIPORT is not set
# CONFIG_SERIAL_8250_RSA is not set

#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
CONFIG_UNIX98_PTYS=y
CONFIG_UNIX98_PTY_COUNT=256

#
# I2C support
#
# CONFIG_I2C is not set

#
# Mice
#
# CONFIG_BUSMOUSE is not set
# CONFIG_QIC02_TAPE is not set

#
# Watchdog Cards
#
# CONFIG_WATCHDOG is not set
# CONFIG_INTEL_RNG is not set
# CONFIG_AMD_RNG is not set
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_DTLK is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_SONYPI is not set

#
# Ftape, the floppy tape device driver
#
# CONFIG_FTAPE is not set
CONFIG_AGP=y
# CONFIG_AGP_INTEL is not set
# CONFIG_AGP_I810 is not set
CONFIG_AGP_VIA=y
# CONFIG_AGP_AMD is not set
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_ALI is not set
# CONFIG_AGP_SWORKS is not set
# CONFIG_AGP_AMD_8151 is not set
# CONFIG_DRM is not set
# CONFIG_MWAVE is not set
# CONFIG_SCx200_GPIO is not set
# CONFIG_RAW_DRIVER is not set

#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set

#
# File systems
#
# CONFIG_QUOTA is not set
# CONFIG_QFMT_V1 is not set
# CONFIG_QFMT_V2 is not set
# CONFIG_AUTOFS_FS is not set
CONFIG_AUTOFS4_FS=y
CONFIG_REISERFS_FS=y
# CONFIG_REISERFS_CHECK is not set
# CONFIG_REISERFS_PROC_INFO is not set
# CONFIG_ADFS_FS is not set
# CONFIG_ADFS_FS_RW is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_BFS_FS is not set
CONFIG_EXT3_FS=y
CONFIG_JBD=y
# CONFIG_JBD_DEBUG is not set
CONFIG_FAT_FS=y
# CONFIG_MSDOS_FS is not set
# CONFIG_UMSDOS_FS is not set
CONFIG_VFAT_FS=y
# CONFIG_EFS_FS is not set
# CONFIG_JFFS_FS is not set
# CONFIG_JFFS2_FS is not set
# CONFIG_CRAMFS is not set
CONFIG_TMPFS=y
CONFIG_RAMFS=y
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
# CONFIG_ZISOFS is not set
# CONFIG_JFS_FS is not set
# CONFIG_JFS_DEBUG is not set
# CONFIG_JFS_STATISTICS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_NTFS_FS is not set
# CONFIG_NTFS_DEBUG is not set
# CONFIG_NTFS_RW is not set
# CONFIG_HPFS_FS is not set
CONFIG_PROC_FS=y
# CONFIG_DEVFS_FS is not set
# CONFIG_DEVFS_MOUNT is not set
# CONFIG_DEVFS_DEBUG is not set
CONFIG_DEVPTS_FS=y
# CONFIG_QNX4FS_FS is not set
# CONFIG_QNX4FS_RW is not set
# CONFIG_ROMFS_FS is not set
CONFIG_EXT2_FS=y
# CONFIG_SYSV_FS is not set
# CONFIG_UDF_FS is not set
# CONFIG_UDF_RW is not set
# CONFIG_UFS_FS is not set
# CONFIG_UFS_FS_WRITE is not set
# CONFIG_XFS_FS is not set
# CONFIG_XFS_RT is not set
# CONFIG_XFS_QUOTA is not set

#
# Network File Systems
#
# CONFIG_CODA_FS is not set
# CONFIG_INTERMEZZO_FS is not set
# CONFIG_NFS_FS is not set
# CONFIG_NFS_V3 is not set
# CONFIG_NFS_V4 is not set
# CONFIG_ROOT_NFS is not set
# CONFIG_NFSD is not set
# CONFIG_NFSD_V3 is not set
# CONFIG_NFSD_V4 is not set
# CONFIG_NFSD_TCP is not set
# CONFIG_SUNRPC is not set
# CONFIG_LOCKD is not set
# CONFIG_EXPORTFS is not set
# CONFIG_CIFS is not set
CONFIG_SMB_FS=m
# CONFIG_SMB_NLS_DEFAULT is not set
# CONFIG_NCP_FS is not set
# CONFIG_NCPFS_PACKET_SIGNING is not set
# CONFIG_NCPFS_IOCTL_LOCKING is not set
# CONFIG_NCPFS_STRONG is not set
# CONFIG_NCPFS_NFS_NS is not set
# CONFIG_NCPFS_OS2_NS is not set
# CONFIG_NCPFS_SMALLDOS is not set
# CONFIG_NCPFS_NLS is not set
# CONFIG_NCPFS_EXTRAS is not set
# CONFIG_AFS_FS is not set
# CONFIG_ZISOFS_FS is not set

#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
# CONFIG_MAC_PARTITION is not set
CONFIG_MSDOS_PARTITION=y
CONFIG_BSD_DISKLABEL=y
# CONFIG_MINIX_SUBPARTITION is not set
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_UNIXWARE_DISKLABEL is not set
CONFIG_LDM_PARTITION=y
# CONFIG_LDM_DEBUG is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_EFI_PARTITION is not set
CONFIG_SMB_NLS=y
CONFIG_NLS=y

#
# Native Language Support
#
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=m
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
CONFIG_NLS_CODEPAGE_850=m
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
CONFIG_NLS_CODEPAGE_936=m
CONFIG_NLS_CODEPAGE_950=m
CONFIG_NLS_CODEPAGE_932=m
CONFIG_NLS_CODEPAGE_949=m
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ISO8859_1=m
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
CONFIG_NLS_ISO8859_15=m
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
CONFIG_NLS_UTF8=m

#
# Console drivers
#
CONFIG_VGA_CONSOLE=y
CONFIG_VIDEO_SELECT=y
# CONFIG_MDA_CONSOLE is not set

#
# Frame-buffer support
#
CONFIG_FB=y
CONFIG_DUMMY_CONSOLE=y
# CONFIG_FB_CLGEN is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
CONFIG_FB_VESA=y
# CONFIG_FB_VGA16 is not set
# CONFIG_FB_HGA is not set
CONFIG_VIDEO_SELECT=y
# CONFIG_FB_RIVA is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_VIRTUAL is not set
# CONFIG_FBCON_ADVANCED is not set
CONFIG_FBCON_CFB24=y
CONFIG_FBCON_ACCEL=y
# CONFIG_FBCON_FONTWIDTH8_ONLY is not set
# CONFIG_FBCON_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y

#
# Sound
#
# CONFIG_SOUND is not set

#
# USB support
#
# CONFIG_USB is not set

#
# Bluetooth support
#
# CONFIG_BT is not set

#
# Profiling support
#
# CONFIG_PROFILING is not set

#
# Kernel hacking
#
# CONFIG_SOFTWARE_SUSPEND is not set
# CONFIG_DEBUG_KERNEL is not set
CONFIG_X86_EXTRA_IRQS=y
CONFIG_X86_FIND_SMP_CONFIG=y
CONFIG_X86_MPPARSE=y

#
# Security options
#
CONFIG_SECURITY_CAPABILITIES=y

#
# Library routines
#
# CONFIG_CRC32 is not set
CONFIG_ZLIB_INFLATE=m
CONFIG_ZLIB_DEFLATE=m
CONFIG_X86_BIOS_REBOOT=y

Re: Extremely slow network with e1000 & ip_conntrack

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 2117 bytes --]

On Sun, Nov 30, 2003 at 07:54:15AM +0900, Stephen Lee wrote:
> Stephen Lee <mukansai@emailplus.org> wrote:
> > 
> > I compiled lots of kernels :-( and narrowed it down to between 2.5.26
> > and 2.5.46.
> > 
> > Kernel version	Chip		Problem?
> > 2.4.22		82540EM		N
> > 2.5.26		82540EM		N
> > 2.5.46		82540EM		Y
> > 2.6.0-test10		82540EM		Y
> > 2.6.0-test11		82540EM		Y
> > 2.6.0-test11		82547EI		N
> > 2.4.22nptlsmp		82547EI		N
> 
> I narrowed it down to patch-2.5.44.  e1000 is updated, but I backed out
> the driver to the 2.5.43 version and it didn't fix the problem. 
> Besides, 2.4.23 has a newer version of the driver and it doesn't have
> this problem.
> 
> As far as I see netfilter was not updated in that patch, was it? I tried
> insmod -f ip_conntrack.o from 2.5.43 into the 2.5.44 kernel and it
> didn't stop the problem from happening, either.
> 
> Please advice.  If you don't think this is a netfilter problem I'll go
> check with linux-kernel.

Well, the problem is certainly triggered by connection tracking.  

If I understood correctly:

1) stock 2.5.43: OK
2) stock 2.5.44: PROBLEM
3) stock 2.5.44 with the driver from 2.5.43: PROBLEM
4) stock 2.5.44 with ip_conntrack.o from 2.5.43: PROBLEM

So it has to be a change outside of the e1000 driver and outside of the
connection tracking code.

Unfortunately I don't have a 2.5.44.patch right here on my notebook atm
(travelling to India). I'll download it at the next opportunity and try
to review which change could be the culprit.

In the meantime, I think taking the discussion back to lkml seems to be
a good idea - since neither e1000 nor ip_conntrack code seem to be the
direct cause of the problem.

> Regards,
> Stephen

-- 
- Harald Welte <laforge@netfilter.org>             http://www.netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]