Linux 2.4.22-pre10

Linux 2.4.22-pre10

[Marcelo Tosatti]

Hello,

Here goes -pre10, hopefully the last -pre of 2.4.22. 

It contains a bunch of important fixes, detailed below.

Please help testing.


Summary of changes from v2.4.22-pre9 to v2.4.22-pre10
============================================

<achirica:telefonica.net>:
  o [wireless airo] sync with 2.6
  o [wireless airo] Simplify dynamic buffer code in Cisco extensions
  o [wireless airo] Update  structs with the new fields in latest firmwares
  o [wireless airo] Make locking "per thread" so it's fully preemptive
  o [wireless airo] Don't sleep when the stats are requested
  o [wireless airo] Don't call MIC functions if the card doesn't support them
  o [wireless airo] Fix small endianness bug
  o [wireless airo] Returns proper status in case of transmission error
  o [wireless airo] Checks for small packets before transmitting them
  o [wireless airo] Return channel in infrastructure mode
  o [wireless airo] Update to wireless extensions 15 (add monitor mode)
  o [wireless airo] Update to wireless extensions 16 (new spy API)
  o [wireless airo] fix Tx race
  o [wireless airo] safer shutdown sequence
  o [wireless airo] eliminate infinite loop
  o [wireless airo] makes the card passive when entering monitor mode
  o [wireless airo] adds support for noise level reporting (if available)

<bjorn.helgaas:hp.com>:
  o trivial 2.4 HCDP documentation/config patch

<herbert:13thfloor.at>:
  o ROOT NFS fixes

<marcelo:logos.cnet>:
  o NMI watchdog documentation for x86-64

<mike.miller:hp.com>:
  o cciss update: author change
  o cciss update: Fix problem with shared IRQs

Adam Radford:
  o 3ware driver update

Adrian Bunk:
  o fix IPMI build error #if CONFIG_ACPI_HT_ONLY

Benjamin Herrenschmidt:
  o ppc32: export hash_table_lock on SMP for MacOnLinux

Dave Kleikamp:
  o JFS: write_super_lockfs should mark superblock clean

Jan Harkes:
  o Coda fixes

Jay Vosburgh:
  o [netdrvr bonding] fix ifenslave ia64 build

Jeff Garzik:
  o [netdrvr] add new broadcom 440x net driver, "b44"

Marc-Christian Petersen:
  o Fix AGPGART problem with 4GB RAM
  o Fix irq handling of IO-APIC edge IRQs on UP
  o MXCSR Handler Unspecified Vulnerability
  o Fix /proc/self security issue
  o Add missing -EFAULT for sysctl

Marcelo Tosatti:
  o Changed EXTRAVERSION to -pre10

Oleg Drokin:
  o reiserfs: fix savelinks for bigendian arches

Petr Vandrovec:
  o ncpfs: Support for clustered NetWare volumes
  o matroxfb: extended support for mplayer

Shmulik Hen:
  o [bonding] fix ifenslave ABI bug
  o [netdrvr bonding] fix ARP monitoring bug

Trond Myklebust:
  o If an RPC request has to be resent due to a timeout, it turns out that call_encode() may cause rq_rcv_buf to be reset despite the fact that a reply might be delivered at any moment by a softirq.
  o If xdr_kmap() fails, we need to ensure that it unmaps all the pages, and returns 0. We don't want to be sending partial RPC requests to the server.

Willy Tarreau:
  o ACPI poweroff fix
  o [netdrvr bonding] fix a typo in the MODULE_PARM_DESC
  o [netdrvr bonding] fix kernel panic when optional feature used

Re: Linux 2.4.22-pre10

[Willy Tarreau]

On Fri, Aug 01, 2003 at 01:19:11PM -0300, Marcelo Tosatti wrote:
> 
> Hello,
> 
> Here goes -pre10, hopefully the last -pre of 2.4.22. 
> 
> It contains a bunch of important fixes, detailed below.
> 
> Please help testing.

Hi Marcelo,

First, one word : Congratulations !

This is the _first_ vanilla 2.4 kernel which I can run _unpatched_ on my
customer's firewalls. This one was stressed all the day at 4000 hits/s.
Subsystems and drivers include aic7xxx, cpqarray, bonding, tulip, eepro100,
sunhme, PIII / PPro SMP, netfilter. Everything looks fine and smooth even at a
sustained write rate of 900 kB/s (logs). I only loose and corrupt significant
number of firewall logs above 3000 lines/s if I don't extend the log buffer
size. I've been using the fairly simple attached patch for a few months now
with success (no loss up to 5600 lines/s). I believe Randy Dunlap has already
got nearly the same one included in 2.5/2.6, so may want to include it too
since it's not really intrusive, although my customer can survive with one
patch :-)

Second, I'm writing this mail from my alpha :

bash-2.03$ uname -a
Linux alpha 2.4.22-pre10 #1 Fri Aug 1 23:20:31 CEST 2003 alpha unknown

It compiled without a glitch and I've got no error in the logs yet. The
previous stable version on this machine was 2.4.21-rc3 + aic7xxx from Justin.
For the record, this one is an NFS server on reiserfs on soft raid5 on aic7xxx.

Third, my VAIO likes it a lot since I can now power it off without holding the
button during 4 seconds !

So for me, it looks like the cleanest 2.4 to date. I will only tell you in 450
days if it's as much reliable as have been my old ones for the last 450 days of
interrupted service :-)

I hope we'll get other positive records so that we can quickly get 2.4.22.

Thanks to you and all others in $ChangeLog for this good version !
Willy
 
============
patch : make LOG_BUF_LEN configurable at config time
============

diff -urN wt10-pre3/Documentation/Configure.help wt10-pre3-log-buf-len/Documentation/Configure.help
--- wt10-pre3/Documentation/Configure.help	Wed Mar 19 09:58:25 2003
+++ wt10-pre3-log-buf-len/Documentation/Configure.help	Tue Mar 25 08:20:35 2003
@@ -25231,6 +25231,19 @@
   output to the second serial port on these devices.  Saying N will
   cause the debug messages to appear on the first serial port.
 
+Kernel log buffer length shift
+CONFIG_LOG_BUF_SHIFT
+  The kernel log buffer has a fixed size of :
+      64 kB (2^16) on MULTIQUAD and IA64,
+     128 kB (2^17) on S390
+      32 kB (2^15) on SMP systems
+      16 kB (2^14) on UP systems
+
+  You have the ability to change this size with this parameter which
+  fixes the bit shift used to get the buffer length (which must be a
+  power of 2). Eg: a value of 16 sets the buffer to 64 kB (2^16).
+  The default value of 0 uses standard values above.
+
 Disable pgtable cache
 CONFIG_NO_PGT_CACHE
   Normally the kernel maintains a `quicklist' of preallocated
diff -urN wt10-pre3/arch/i386/config.in wt10-pre3-log-buf-len/arch/i386/config.in
--- wt10-pre3/arch/i386/config.in	Wed Mar 19 09:58:25 2003
+++ wt10-pre3-log-buf-len/arch/i386/config.in	Tue Mar 25 08:25:12 2003
@@ -508,6 +508,8 @@
     string '   Initial kernel command line' CONFIG_CMDLINE "root=301 ro"
 fi
 
+int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0
+
 endmenu
 
 source lib/Config.in
diff -urN wt10-pre3/kernel/printk.c wt10-pre3-log-buf-len/kernel/printk.c
--- wt10-pre3/kernel/printk.c	Wed Mar 19 09:58:20 2003
+++ wt10-pre3-log-buf-len/kernel/printk.c	Tue Mar 25 08:14:55 2003
@@ -29,6 +29,7 @@
 
 #include <asm/uaccess.h>
 
+#if !defined(CONFIG_LOG_BUF_SHIFT) || (CONFIG_LOG_BUF_SHIFT - 0 == 0)
 #if defined(CONFIG_MULTIQUAD) || defined(CONFIG_IA64)
 #define LOG_BUF_LEN	(65536)
 #elif defined(CONFIG_ARCH_S390)
@@ -37,6 +38,9 @@
 #define LOG_BUF_LEN	(32768)
 #else	
 #define LOG_BUF_LEN	(16384)			/* This must be a power of two */
+#endif
+#else /* CONFIG_LOG_BUF_SHIFT */
+#define LOG_BUF_LEN (1 << CONFIG_LOG_BUF_SHIFT)
 #endif
 
 #define LOG_BUF_MASK	(LOG_BUF_LEN-1)

Re: Linux 2.4.22-pre10

[Martin Josefsson]

On Sat, 2003-08-02 at 00:47, Willy Tarreau wrote:

> This is the _first_ vanilla 2.4 kernel which I can run _unpatched_ on my
> customer's firewalls. This one was stressed all the day at 4000 hits/s.
> Subsystems and drivers include aic7xxx, cpqarray, bonding, tulip, eepro100,
> sunhme, PIII / PPro SMP, netfilter. Everything looks fine and smooth even at a
> sustained write rate of 900 kB/s (logs). I only loose and corrupt significant
> number of firewall logs above 3000 lines/s if I don't extend the log buffer
> size. I've been using the fairly simple attached patch for a few months now
> with success (no loss up to 5600 lines/s). I believe Randy Dunlap has already
> got nearly the same one included in 2.5/2.6, so may want to include it too
> since it's not really intrusive, although my customer can survive with one
> patch :-)

Have you tried using the ULOG target and the ulogd userspace daemon?
It uses netlink and can batch several entries together before it sends
them to userspace. Works a lot better than syslog.

Are you using ip_conntrack on that machine? if you are, be aware that
ip_conntrack doesn't scale well at all on SMP. It's beeing worked on.

-- 
/Martin

Re: Linux 2.4.22-pre10

[Willy Tarreau]

Hi Martin,

On Sat, Aug 02, 2003 at 11:42:50AM +0200, Martin Josefsson wrote:
 
> Have you tried using the ULOG target and the ulogd userspace daemon?
> It uses netlink and can batch several entries together before it sends
> them to userspace. Works a lot better than syslog.

yes, I tried it about february. I thought it to be the ideal solution, and it
performed better than the standard syslog, but not as well as syslog-ng. I
could catch about 1500 lines/s at most, and the daemon was very hungry, it ate
between 55 and 70% of the CPU, while syslog-ng eats about 25-30. So I thought
it was still a bit experimental and switched to syslog-ng.

> Are you using ip_conntrack on that machine? if you are, be aware that
> ip_conntrack doesn't scale well at all on SMP. It's beeing worked on.

Yes I do. I noticed the scalability problems a long time ago on my dual athlon
at home, but wasn't really concerned. At my customer's, the only SMP one is
used as a development gateway. All the others are UP (PIII/1G, P4/2.4G).

BTW, I get the same numbers with 2.4.22-pre10 / standard ip_conntrack as with
2.4.21-rc2 with tcp_window_tracking. I would have thought that
tcp_window_tracking costs a bit more, but it doesn't seem to be noticeable here.

Cheers,
Willy

Re: Linux 2.4.22-pre10

[Martin Josefsson]

On Sat, 2003-08-02 at 20:10, Willy Tarreau wrote:
 
> > Have you tried using the ULOG target and the ulogd userspace daemon?
> > It uses netlink and can batch several entries together before it sends
> > them to userspace. Works a lot better than syslog.
> 
> yes, I tried it about february. I thought it to be the ideal solution, and it
> performed better than the standard syslog, but not as well as syslog-ng. I
> could catch about 1500 lines/s at most, and the daemon was very hungry, it ate
> between 55 and 70% of the CPU, while syslog-ng eats about 25-30. So I thought
> it was still a bit experimental and switched to syslog-ng.

Uhm, my tests have shown it to be very fast and efficient. But I didn't
look to see if all packets got through to the logfile. But getting it to
write logs at ~35MB/s wasn't a problem.

Did you specify --ulog-qthreshold 50 ?
and did you specify --ulog-cprange at all? if you don't it will copy the
entire packet to userspace. I copy 64 bytes to userspace and that's more
than enough to log everything needed.

> > Are you using ip_conntrack on that machine? if you are, be aware that
> > ip_conntrack doesn't scale well at all on SMP. It's beeing worked on.
> 
> Yes I do. I noticed the scalability problems a long time ago on my dual athlon
> at home, but wasn't really concerned. At my customer's, the only SMP one is
> used as a development gateway. All the others are UP (PIII/1G, P4/2.4G).

Ok, we are working on memory-usage and scalability stuff.

> BTW, I get the same numbers with 2.4.22-pre10 / standard ip_conntrack as with
> 2.4.21-rc2 with tcp_window_tracking. I would have thought that
> tcp_window_tracking costs a bit more, but it doesn't seem to be noticeable here.

It should cost a little bit more but not very much.

-- 
/Martin

Re: Linux 2.4.22-pre10

[Willy Tarreau]

On Sat, Aug 02, 2003 at 08:28:46PM +0200, Martin Josefsson wrote:
 
> Uhm, my tests have shown it to be very fast and efficient. But I didn't
> look to see if all packets got through to the logfile. But getting it to
> write logs at ~35MB/s wasn't a problem.

I clearly didn't reach these numbers, I used LOGEMU, and while you're at it,
I must say that when I speak about 1500/s, it's about logs _written_ down.
The firewall can still can process 5k sessions/s, but looses many logs (3.5k
every second).

When I read the LOGEMU code, I had the impression that it was given more as
a proof of concept than anything else. Because there are many many many
"fprintf(of, something_trivial_enough_to_support_memcpy)".

> Did you specify --ulog-qthreshold 50 ?
> and did you specify --ulog-cprange at all? if you don't it will copy the
> entire packet to userspace. I copy 64 bytes to userspace and that's more
> than enough to log everything needed.

honnestly, i't 6 months old in my head, and I don't remember with which
parameters I played. But I'd happily restart a bench if you have some tuning
advices (provided that they are compatible with basic production constraints,
such as log rotation, and a few CPU left for monitoring processes :-))
I cannot promise to to it within a few days though.

Cheers,
Willy

Re: Linux 2.4.22-pre10

[Marcelo Tosatti]

On Sat, 2 Aug 2003, Willy Tarreau wrote:

> On Fri, Aug 01, 2003 at 01:19:11PM -0300, Marcelo Tosatti wrote:
> > 
> > Hello,
> > 
> > Here goes -pre10, hopefully the last -pre of 2.4.22. 
> > 
> > It contains a bunch of important fixes, detailed below.
> > 
> > Please help testing.
> 
> Hi Marcelo,
> 
> First, one word : Congratulations !
> 
> This is the _first_ vanilla 2.4 kernel which I can run _unpatched_ on my
> customer's firewalls. This one was stressed all the day at 4000 hits/s.
> Subsystems and drivers include aic7xxx, cpqarray, bonding, tulip, eepro100,
> sunhme, PIII / PPro SMP, netfilter. Everything looks fine and smooth even at a
> sustained write rate of 900 kB/s (logs). I only loose and corrupt significant
> number of firewall logs above 3000 lines/s if I don't extend the log buffer
> size. I've been using the fairly simple attached patch for a few months now
> with success (no loss up to 5600 lines/s). I believe Randy Dunlap has already
> got nearly the same one included in 2.5/2.6, so may want to include it too
> since it's not really intrusive, although my customer can survive with one
> patch :-)
> 
> Second, I'm writing this mail from my alpha :
> 
> bash-2.03$ uname -a
> Linux alpha 2.4.22-pre10 #1 Fri Aug 1 23:20:31 CEST 2003 alpha unknown
> 
> It compiled without a glitch and I've got no error in the logs yet. The
> previous stable version on this machine was 2.4.21-rc3 + aic7xxx from Justin.
> For the record, this one is an NFS server on reiserfs on soft raid5 on aic7xxx.
> 
> Third, my VAIO likes it a lot since I can now power it off without holding the
> button during 4 seconds !
> 
> So for me, it looks like the cleanest 2.4 to date. I will only tell you in 450
> days if it's as much reliable as have been my old ones for the last 450 days of
> interrupted service :-)
> 
> I hope we'll get other positive records so that we can quickly get 2.4.22.

Great! 

Its good to see everyone who contributed to latter 2.4 having their work
recognized.



> Thanks to you and all others in $ChangeLog for this good version ! Willy
>  ============ patch : make LOG_BUF_LEN configurable at config time
> ============
> 
> diff -urN wt10-pre3/Documentation/Configure.help wt10-pre3-log-buf-len/Documentation/Configure.help
> --- wt10-pre3/Documentation/Configure.help	Wed Mar 19 09:58:25 2003
> +++ wt10-pre3-log-buf-len/Documentation/Configure.help	Tue Mar 25 08:20:35 2003
> @@ -25231,6 +25231,19 @@
>    output to the second serial port on these devices.  Saying N will
>    cause the debug messages to appear on the first serial port.
>  
> +Kernel log buffer length shift
> +CONFIG_LOG_BUF_SHIFT
> +  The kernel log buffer has a fixed size of :
> +      64 kB (2^16) on MULTIQUAD and IA64,
> +     128 kB (2^17) on S390
> +      32 kB (2^15) on SMP systems
> +      16 kB (2^14) on UP systems
> +
> +  You have the ability to change this size with this parameter which
> +  fixes the bit shift used to get the buffer length (which must be a
> +  power of 2). Eg: a value of 16 sets the buffer to 64 kB (2^16).
> +  The default value of 0 uses standard values above.

I dont see a problem with this patch and it is useful. 

Please resubmit it at 2.4.23-pre time, okey? 

Thanks for your patches and testing. They are very welcome.

Re: Linux 2.4.22-pre10

[Stephan von Krawczynski]

On Fri, 1 Aug 2003 13:19:11 -0300 (BRT)
Marcelo Tosatti <marcelo@conectiva.com.br> wrote:

> 
> Hello,
> 
> Here goes -pre10, hopefully the last -pre of 2.4.22. 

Have I missed something before, is this expected during build:

make[3]: Circular /usr/src/linux-2.4.22-pre10/include/asm/smplock.h <-
/usr/src/linux-2.4.22-pre10/include/linux/interrupt.h dependency dropped.

?

Regards,
Stephan

[2.4 patch] fix a compile warning in acpi/system.c

[Adrian Bunk]

On Fri, Aug 01, 2003 at 01:19:11PM -0300, Marcelo Tosatti wrote:
>...
> Summary of changes from v2.4.22-pre9 to v2.4.22-pre10
> ============================================
>...
> Willy Tarreau:
>   o ACPI poweroff fix
>...

This patch causes the following compile warnings:

<--  snip  -->

...
gcc -D__KERNEL__ 
-I/home/bunk/linux/kernel-2.4/linux-2.4.22-pre10-full/include -
Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fno-strict-aliasing 
-fno-common -pipe -mpreferred-stack-boundary=2 -march=k6  -Os  -DACPI_DEBUG_OUTPUT 
-nostdinc -iwithprefix include -DKBUILD_BASENAME=system  -c -o system.o system.c
system.c: In function `acpi_power_off':
system.c:93: warning: implicit declaration of function `acpi_suspend'
system.c: At top level:
system.c:303: warning: type mismatch with previous implicit declaration
system.c:93: warning: previous implicit declaration of `acpi_suspend'
system.c:303: warning: `acpi_suspend' was previously implicitly declared to return `int'
...

<--  snip  -->


The following patch removes these warnings:

--- linux-2.4.22-pre10-full/drivers/acpi/system.c.old	2003-08-02 22:23:08.000000000 +0200
+++ linux-2.4.22-pre10-full/drivers/acpi/system.c	2003-08-02 22:26:28.000000000 +0200
@@ -59,6 +59,8 @@
 static int acpi_system_add (struct acpi_device *device);
 static int acpi_system_remove (struct acpi_device *device, int type);
 
+acpi_status acpi_suspend (u32 state);
+
 static struct acpi_driver acpi_system_driver = {
 	.name =		ACPI_SYSTEM_DRIVER_NAME,
 	.class =	ACPI_SYSTEM_CLASS,



I've tested the compilation with 2.4.22-pre10.

Please apply
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

[patch] 2.4.22-pre10: fix circular dependency

[Adrian Bunk]

On Fri, Aug 01, 2003 at 01:19:11PM -0300, Marcelo Tosatti wrote:
>...
> Summary of changes from v2.4.22-pre9 to v2.4.22-pre10
> ============================================
>...
> Marc-Christian Petersen:
>...
>   o Fix irq handling of IO-APIC edge IRQs on UP
>...

This patch adds for no good reason two #include's to 
include/asm-i386/hw_irq.h resulting in a circular dependency between 
headers.

The patch below removes these #include's again.

I've tested the compilation with 2.4.22-pre10.

cu
Adrian

--- linux-2.4.22-pre10-full/include/asm-i386/hw_irq.h.old	2003-08-03 01:34:22.000000000 +0200
+++ linux-2.4.22-pre10-full/include/asm-i386/hw_irq.h	2003-08-03 01:34:57.000000000 +0200
@@ -13,10 +13,8 @@
  */
 
 #include <linux/config.h>
-#include <linux/smp_lock.h>
 #include <asm/atomic.h>
 #include <asm/irq.h>
-#include <asm/current.h>
 
 /*
  * IDT vectors usable for external interrupt sources start

Re: [patch] 2.4.22-pre10: fix circular dependency

[Marc-Christian Petersen]

On Sunday 03 August 2003 15:16, Adrian Bunk wrote:

Hi Adrian,

> >   o Fix irq handling of IO-APIC edge IRQs on UP
> This patch adds for no good reason two #include's to
> include/asm-i386/hw_irq.h resulting in a circular dependency between
> headers.
grmpf, yeah. right. Sorry :(

ciao, Marc