Re: Linux 2.4.22-pre10

Re: Linux 2.4.22-pre10

[Margit Schubert-While]

Well ignoring the almost 1000 occurrences of __FUNCTION__ deprecated and
unused variables, we are left with this:


system.c: In function `acpi_power_off':
system.c:93: warning: implicit declaration of function `acpi_suspend'
system.c: At top level:
system.c:303: warning: type mismatch with previous implicit declaration
system.c:93: warning: previous implicit declaration of `acpi_suspend'
system.c:303: warning: `acpi_suspend' was previously implicitly declared to 
retu
rn `int'
siimage.c: In function `pdev_is_sata':
siimage.c:65: warning: control reaches end of non-void function
generic.h:151: warning: `unknown_chipset' defined but not used
tipar.c:76:1: warning: "minor" redefined
In file included from /var/tmp/linux-2.4.21/include/linux/fs.h:16,
                  from /var/tmp/linux-2.4.21/include/linux/capability.h:17,
                  from /var/tmp/linux-2.4.21/include/linux/binfmts.h:5,
                  from /var/tmp/linux-2.4.21/include/linux/sched.h:9,
                  from tipar.c:49:
/var/tmp/linux-2.4.21/include/linux/kdev_t.h:81:1: warning: this is the 
location
  of the previous definition
applicom.c:268:2: warning: #warning "LEAK"
applicom.c:532:2: warning: #warning "Je suis stupide. DW. - copy*user in cli"
i2o_pci.c:393:1: warning: no newline at end of file
cfi_cmdset_0020.c: In function `do_write_buffer':
cfi_cmdset_0020.c:491: warning: unsigned int format, different type arg (arg 3)
cfi_cmdset_0020.c: In function `do_erase_oneblock':
cfi_cmdset_0020.c:851: warning: unsigned int format, different type arg (arg 3)
cfi_cmdset_0020.c: In function `do_lock_oneblock':
cfi_cmdset_0020.c:1137: warning: unsigned int format, different type arg 
(arg 3)
cfi_cmdset_0020.c: In function `do_unlock_oneblock':
cfi_cmdset_0020.c:1286: warning: unsigned int format, different type arg 
(arg 3)
cfi_cmdset_0001.c: In function `do_write_oneword':
cfi_cmdset_0001.c:826: warning: unsigned int format, different type arg (arg 2)
cfi_cmdset_0001.c: In function `do_write_buffer':
cfi_cmdset_0001.c:1135: warning: unsigned int format, different type arg 
(arg 2)
cfi_cmdset_0001.c:1165: warning: unsigned int format, different type arg 
(arg 2)
ma600.c:51:22: warning: extra tokens at end of #undef directive
crc32.c:91: warning: static declaration for `fn_calc_memory_chunk_crc32' 
follows
  non-static
cardbus.c: In function `cb_scan_slot':
cardbus.c:240: warning: implicit declaration of function `pci_scan_device'
cardbus.c:240: warning: assignment makes pointer from integer without a cast
cardbus.c:226: warning: unused variable `bus'
cardbus.c: In function `program_bridge':
cardbus.c:405: warning: control reaches end of non-void function
In file included from yenta.c:837:
ti113x.h: In function `ti_intctl':
ti113x.h:182: warning: suggest parentheses around comparison in operand of &
qlogicfas.c: In function `qlogicfas_detect':
qlogicfas.c:650: warning: passing arg 1 of `scsi_unregister' from 
incompatible p
ointer type
cpqfcTSi2c.c:62: warning: `i2c_delay' declared `static' but never defined
../qlogicfas.c: In function `qlogicfas_detect':
../qlogicfas.c:650: warning: passing arg 1 of `scsi_unregister' from 
incompatibl
e pointer type
main.c:236:2: warning: #warning "Initialisation order race. Must register after
usable"
hid-core.c: In function `hid_input_report':
hid-core.c:879: warning: implicit declaration of function `hiddev_report_event'
kaweth.c: In function `kaweth_start_xmit':
kaweth.c:738: warning: assignment from incompatible pointer type
pm3fb.c: In function `cleanup_module':
pm3fb.c:3835: warning: passing arg 2 of `__release_region' makes integer 
from po
inter without a cast
pm3fb.c:3838: warning: passing arg 2 of `__release_region' makes integer 
from po
inter without a cast
matroxfb_g450.c: In function `g450_compute_bwlevel':
matroxfb_g450.c:134: warning: duplicate `const'
matroxfb_g450.c:135: warning: duplicate `const'
matroxfb_maven.c: In function `maven_compute_bwlevel':
matroxfb_maven.c:359: warning: duplicate `const'
matroxfb_maven.c:360: warning: duplicate `const'
fs.c: In function `_ntfs_clear_inode':
fs.c:852: warning: deprecated use of label at end of compound statement
sock.c:64: warning: static declaration for `sockfd_lookup' follows non-static
sock.c:56: warning: static declaration for `sockfd_lookup' follows non-static
if [ -r System.map ]; then /sbin/depmod -ae -F System.map  2.4.22-pre10; fi
depmod: *** Unresolved symbols in 
/lib/modules/2.4.22-pre10/kernel/drivers/net/w
an/comx.o
depmod:         proc_get_inode

Margit 

Re: Linux 2.4.22-pre10

[Marcelo Tosatti]

On Sat, 2 Aug 2003, Willy Tarreau wrote:

> On Fri, Aug 01, 2003 at 01:19:11PM -0300, Marcelo Tosatti wrote:
> > 
> > Hello,
> > 
> > Here goes -pre10, hopefully the last -pre of 2.4.22. 
> > 
> > It contains a bunch of important fixes, detailed below.
> > 
> > Please help testing.
> 
> Hi Marcelo,
> 
> First, one word : Congratulations !
> 
> This is the _first_ vanilla 2.4 kernel which I can run _unpatched_ on my
> customer's firewalls. This one was stressed all the day at 4000 hits/s.
> Subsystems and drivers include aic7xxx, cpqarray, bonding, tulip, eepro100,
> sunhme, PIII / PPro SMP, netfilter. Everything looks fine and smooth even at a
> sustained write rate of 900 kB/s (logs). I only loose and corrupt significant
> number of firewall logs above 3000 lines/s if I don't extend the log buffer
> size. I've been using the fairly simple attached patch for a few months now
> with success (no loss up to 5600 lines/s). I believe Randy Dunlap has already
> got nearly the same one included in 2.5/2.6, so may want to include it too
> since it's not really intrusive, although my customer can survive with one
> patch :-)
> 
> Second, I'm writing this mail from my alpha :
> 
> bash-2.03$ uname -a
> Linux alpha 2.4.22-pre10 #1 Fri Aug 1 23:20:31 CEST 2003 alpha unknown
> 
> It compiled without a glitch and I've got no error in the logs yet. The
> previous stable version on this machine was 2.4.21-rc3 + aic7xxx from Justin.
> For the record, this one is an NFS server on reiserfs on soft raid5 on aic7xxx.
> 
> Third, my VAIO likes it a lot since I can now power it off without holding the
> button during 4 seconds !
> 
> So for me, it looks like the cleanest 2.4 to date. I will only tell you in 450
> days if it's as much reliable as have been my old ones for the last 450 days of
> interrupted service :-)
> 
> I hope we'll get other positive records so that we can quickly get 2.4.22.

Great! 

Its good to see everyone who contributed to latter 2.4 having their work
recognized.



> Thanks to you and all others in $ChangeLog for this good version ! Willy
>  ============ patch : make LOG_BUF_LEN configurable at config time
> ============
> 
> diff -urN wt10-pre3/Documentation/Configure.help wt10-pre3-log-buf-len/Documentation/Configure.help
> --- wt10-pre3/Documentation/Configure.help	Wed Mar 19 09:58:25 2003
> +++ wt10-pre3-log-buf-len/Documentation/Configure.help	Tue Mar 25 08:20:35 2003
> @@ -25231,6 +25231,19 @@
>    output to the second serial port on these devices.  Saying N will
>    cause the debug messages to appear on the first serial port.
>  
> +Kernel log buffer length shift
> +CONFIG_LOG_BUF_SHIFT
> +  The kernel log buffer has a fixed size of :
> +      64 kB (2^16) on MULTIQUAD and IA64,
> +     128 kB (2^17) on S390
> +      32 kB (2^15) on SMP systems
> +      16 kB (2^14) on UP systems
> +
> +  You have the ability to change this size with this parameter which
> +  fixes the bit shift used to get the buffer length (which must be a
> +  power of 2). Eg: a value of 16 sets the buffer to 64 kB (2^16).
> +  The default value of 0 uses standard values above.

I dont see a problem with this patch and it is useful. 

Please resubmit it at 2.4.23-pre time, okey? 

Thanks for your patches and testing. They are very welcome.

Re: Linux 2.4.22-pre10

[Willy Tarreau]

On Sat, Aug 02, 2003 at 08:28:46PM +0200, Martin Josefsson wrote:
 
> Uhm, my tests have shown it to be very fast and efficient. But I didn't
> look to see if all packets got through to the logfile. But getting it to
> write logs at ~35MB/s wasn't a problem.

I clearly didn't reach these numbers, I used LOGEMU, and while you're at it,
I must say that when I speak about 1500/s, it's about logs _written_ down.
The firewall can still can process 5k sessions/s, but looses many logs (3.5k
every second).

When I read the LOGEMU code, I had the impression that it was given more as
a proof of concept than anything else. Because there are many many many
"fprintf(of, something_trivial_enough_to_support_memcpy)".

> Did you specify --ulog-qthreshold 50 ?
> and did you specify --ulog-cprange at all? if you don't it will copy the
> entire packet to userspace. I copy 64 bytes to userspace and that's more
> than enough to log everything needed.

honnestly, i't 6 months old in my head, and I don't remember with which
parameters I played. But I'd happily restart a bench if you have some tuning
advices (provided that they are compatible with basic production constraints,
such as log rotation, and a few CPU left for monitoring processes :-))
I cannot promise to to it within a few days though.

Cheers,
Willy

Re: Linux 2.4.22-pre10

[Martin Josefsson]

On Sat, 2003-08-02 at 20:10, Willy Tarreau wrote:
 
> > Have you tried using the ULOG target and the ulogd userspace daemon?
> > It uses netlink and can batch several entries together before it sends
> > them to userspace. Works a lot better than syslog.
> 
> yes, I tried it about february. I thought it to be the ideal solution, and it
> performed better than the standard syslog, but not as well as syslog-ng. I
> could catch about 1500 lines/s at most, and the daemon was very hungry, it ate
> between 55 and 70% of the CPU, while syslog-ng eats about 25-30. So I thought
> it was still a bit experimental and switched to syslog-ng.

Uhm, my tests have shown it to be very fast and efficient. But I didn't
look to see if all packets got through to the logfile. But getting it to
write logs at ~35MB/s wasn't a problem.

Did you specify --ulog-qthreshold 50 ?
and did you specify --ulog-cprange at all? if you don't it will copy the
entire packet to userspace. I copy 64 bytes to userspace and that's more
than enough to log everything needed.

> > Are you using ip_conntrack on that machine? if you are, be aware that
> > ip_conntrack doesn't scale well at all on SMP. It's beeing worked on.
> 
> Yes I do. I noticed the scalability problems a long time ago on my dual athlon
> at home, but wasn't really concerned. At my customer's, the only SMP one is
> used as a development gateway. All the others are UP (PIII/1G, P4/2.4G).

Ok, we are working on memory-usage and scalability stuff.

> BTW, I get the same numbers with 2.4.22-pre10 / standard ip_conntrack as with
> 2.4.21-rc2 with tcp_window_tracking. I would have thought that
> tcp_window_tracking costs a bit more, but it doesn't seem to be noticeable here.

It should cost a little bit more but not very much.

-- 
/Martin

Re: Linux 2.4.22-pre10

[Willy Tarreau]

Hi Martin,

On Sat, Aug 02, 2003 at 11:42:50AM +0200, Martin Josefsson wrote:
 
> Have you tried using the ULOG target and the ulogd userspace daemon?
> It uses netlink and can batch several entries together before it sends
> them to userspace. Works a lot better than syslog.

yes, I tried it about february. I thought it to be the ideal solution, and it
performed better than the standard syslog, but not as well as syslog-ng. I
could catch about 1500 lines/s at most, and the daemon was very hungry, it ate
between 55 and 70% of the CPU, while syslog-ng eats about 25-30. So I thought
it was still a bit experimental and switched to syslog-ng.

> Are you using ip_conntrack on that machine? if you are, be aware that
> ip_conntrack doesn't scale well at all on SMP. It's beeing worked on.

Yes I do. I noticed the scalability problems a long time ago on my dual athlon
at home, but wasn't really concerned. At my customer's, the only SMP one is
used as a development gateway. All the others are UP (PIII/1G, P4/2.4G).

BTW, I get the same numbers with 2.4.22-pre10 / standard ip_conntrack as with
2.4.21-rc2 with tcp_window_tracking. I would have thought that
tcp_window_tracking costs a bit more, but it doesn't seem to be noticeable here.

Cheers,
Willy

Re: Linux 2.4.22-pre10

[Stephan von Krawczynski]

On Fri, 1 Aug 2003 13:19:11 -0300 (BRT)
Marcelo Tosatti <marcelo@conectiva.com.br> wrote:

> 
> Hello,
> 
> Here goes -pre10, hopefully the last -pre of 2.4.22. 

Have I missed something before, is this expected during build:

make[3]: Circular /usr/src/linux-2.4.22-pre10/include/asm/smplock.h <-
/usr/src/linux-2.4.22-pre10/include/linux/interrupt.h dependency dropped.

?

Regards,
Stephan

Re: Linux 2.4.22-pre10

[Fridtjof Busse]

> Hello,
> 
> Here goes -pre10, hopefully the last -pre of 2.4.22. 
> 
> It contains a bunch of important fixes, detailed below.
> 
> Please help testing.

Dumping to an USB-harddisk still doesn't work:

kernel: hub.c: new USB device 00:02.2-2, assigned address 4
kernel: scsi1 : SCSI emulation for USB Mass Storage devices
kernel:   Vendor: Maxtor 6  Model: Y120L0            Rev: 0811
kernel:   Type:   Direct-Access                   ANSI SCSI revision: 02
kernel: Attached scsi disk sda at scsi1, channel 0, id 0, lun 0
kernel: SCSI device sda: 240121728 512-byte hdwr sectors (122942 MB)
kernel:  /dev/scsi/host1/bus0/target0/lun0: p1
kernel: WARNING: USB Mass Storage data integrity not assured
kernel: USB Mass Storage device found at 4
kernel: usb_control/bulk_msg: timeout

No start dump:

last message repeated 2 times
kernel: usb.c: USB disconnect on device 00:02.2-2 address 4
kernel: usb-storage: host_reset() requested but not implemented
kernel: scsi: device set offline - command error recover failed: host 1 
channel 0 id 0 lun 0
kernel: 192
kernel:  I/O error: dev 08:01, sector 81655440
lots of I/O errors following

I also reported this to usb-devel, but never got a reply.
Works fine with 2.4.21.
Please CC me, thanks

-- 
Fridtjof Busse
printk("Illegal format on cdrom.  Pester manufacturer.\n"); 
	2.2.16 /usr/src/linux/fs/isofs/inode.c

Re: Linux 2.4.22-pre10

[Martin Josefsson]

On Sat, 2003-08-02 at 00:47, Willy Tarreau wrote:

> This is the _first_ vanilla 2.4 kernel which I can run _unpatched_ on my
> customer's firewalls. This one was stressed all the day at 4000 hits/s.
> Subsystems and drivers include aic7xxx, cpqarray, bonding, tulip, eepro100,
> sunhme, PIII / PPro SMP, netfilter. Everything looks fine and smooth even at a
> sustained write rate of 900 kB/s (logs). I only loose and corrupt significant
> number of firewall logs above 3000 lines/s if I don't extend the log buffer
> size. I've been using the fairly simple attached patch for a few months now
> with success (no loss up to 5600 lines/s). I believe Randy Dunlap has already
> got nearly the same one included in 2.5/2.6, so may want to include it too
> since it's not really intrusive, although my customer can survive with one
> patch :-)

Have you tried using the ULOG target and the ulogd userspace daemon?
It uses netlink and can batch several entries together before it sends
them to userspace. Works a lot better than syslog.

Are you using ip_conntrack on that machine? if you are, be aware that
ip_conntrack doesn't scale well at all on SMP. It's beeing worked on.

-- 
/Martin

Re: Linux 2.4.22-pre10

[Willy Tarreau]

On Fri, Aug 01, 2003 at 01:19:11PM -0300, Marcelo Tosatti wrote:
> 
> Hello,
> 
> Here goes -pre10, hopefully the last -pre of 2.4.22. 
> 
> It contains a bunch of important fixes, detailed below.
> 
> Please help testing.

Hi Marcelo,

First, one word : Congratulations !

This is the _first_ vanilla 2.4 kernel which I can run _unpatched_ on my
customer's firewalls. This one was stressed all the day at 4000 hits/s.
Subsystems and drivers include aic7xxx, cpqarray, bonding, tulip, eepro100,
sunhme, PIII / PPro SMP, netfilter. Everything looks fine and smooth even at a
sustained write rate of 900 kB/s (logs). I only loose and corrupt significant
number of firewall logs above 3000 lines/s if I don't extend the log buffer
size. I've been using the fairly simple attached patch for a few months now
with success (no loss up to 5600 lines/s). I believe Randy Dunlap has already
got nearly the same one included in 2.5/2.6, so may want to include it too
since it's not really intrusive, although my customer can survive with one
patch :-)

Second, I'm writing this mail from my alpha :

bash-2.03$ uname -a
Linux alpha 2.4.22-pre10 #1 Fri Aug 1 23:20:31 CEST 2003 alpha unknown

It compiled without a glitch and I've got no error in the logs yet. The
previous stable version on this machine was 2.4.21-rc3 + aic7xxx from Justin.
For the record, this one is an NFS server on reiserfs on soft raid5 on aic7xxx.

Third, my VAIO likes it a lot since I can now power it off without holding the
button during 4 seconds !

So for me, it looks like the cleanest 2.4 to date. I will only tell you in 450
days if it's as much reliable as have been my old ones for the last 450 days of
interrupted service :-)

I hope we'll get other positive records so that we can quickly get 2.4.22.

Thanks to you and all others in $ChangeLog for this good version !
Willy
 
============
patch : make LOG_BUF_LEN configurable at config time
============

diff -urN wt10-pre3/Documentation/Configure.help wt10-pre3-log-buf-len/Documentation/Configure.help
--- wt10-pre3/Documentation/Configure.help	Wed Mar 19 09:58:25 2003
+++ wt10-pre3-log-buf-len/Documentation/Configure.help	Tue Mar 25 08:20:35 2003
@@ -25231,6 +25231,19 @@
   output to the second serial port on these devices.  Saying N will
   cause the debug messages to appear on the first serial port.
 
+Kernel log buffer length shift
+CONFIG_LOG_BUF_SHIFT
+  The kernel log buffer has a fixed size of :
+      64 kB (2^16) on MULTIQUAD and IA64,
+     128 kB (2^17) on S390
+      32 kB (2^15) on SMP systems
+      16 kB (2^14) on UP systems
+
+  You have the ability to change this size with this parameter which
+  fixes the bit shift used to get the buffer length (which must be a
+  power of 2). Eg: a value of 16 sets the buffer to 64 kB (2^16).
+  The default value of 0 uses standard values above.
+
 Disable pgtable cache
 CONFIG_NO_PGT_CACHE
   Normally the kernel maintains a `quicklist' of preallocated
diff -urN wt10-pre3/arch/i386/config.in wt10-pre3-log-buf-len/arch/i386/config.in
--- wt10-pre3/arch/i386/config.in	Wed Mar 19 09:58:25 2003
+++ wt10-pre3-log-buf-len/arch/i386/config.in	Tue Mar 25 08:25:12 2003
@@ -508,6 +508,8 @@
     string '   Initial kernel command line' CONFIG_CMDLINE "root=301 ro"
 fi
 
+int 'Kernel messages buffer length shift (0 = default)' CONFIG_LOG_BUF_SHIFT 0
+
 endmenu
 
 source lib/Config.in
diff -urN wt10-pre3/kernel/printk.c wt10-pre3-log-buf-len/kernel/printk.c
--- wt10-pre3/kernel/printk.c	Wed Mar 19 09:58:20 2003
+++ wt10-pre3-log-buf-len/kernel/printk.c	Tue Mar 25 08:14:55 2003
@@ -29,6 +29,7 @@
 
 #include <asm/uaccess.h>
 
+#if !defined(CONFIG_LOG_BUF_SHIFT) || (CONFIG_LOG_BUF_SHIFT - 0 == 0)
 #if defined(CONFIG_MULTIQUAD) || defined(CONFIG_IA64)
 #define LOG_BUF_LEN	(65536)
 #elif defined(CONFIG_ARCH_S390)
@@ -37,6 +38,9 @@
 #define LOG_BUF_LEN	(32768)
 #else	
 #define LOG_BUF_LEN	(16384)			/* This must be a power of two */
+#endif
+#else /* CONFIG_LOG_BUF_SHIFT */
+#define LOG_BUF_LEN (1 << CONFIG_LOG_BUF_SHIFT)
 #endif
 
 #define LOG_BUF_MASK	(LOG_BUF_LEN-1)

Linux 2.4.22-pre10

[Marcelo Tosatti]

Hello,

Here goes -pre10, hopefully the last -pre of 2.4.22. 

It contains a bunch of important fixes, detailed below.

Please help testing.


Summary of changes from v2.4.22-pre9 to v2.4.22-pre10
============================================

<achirica:telefonica.net>:
  o [wireless airo] sync with 2.6
  o [wireless airo] Simplify dynamic buffer code in Cisco extensions
  o [wireless airo] Update  structs with the new fields in latest firmwares
  o [wireless airo] Make locking "per thread" so it's fully preemptive
  o [wireless airo] Don't sleep when the stats are requested
  o [wireless airo] Don't call MIC functions if the card doesn't support them
  o [wireless airo] Fix small endianness bug
  o [wireless airo] Returns proper status in case of transmission error
  o [wireless airo] Checks for small packets before transmitting them
  o [wireless airo] Return channel in infrastructure mode
  o [wireless airo] Update to wireless extensions 15 (add monitor mode)
  o [wireless airo] Update to wireless extensions 16 (new spy API)
  o [wireless airo] fix Tx race
  o [wireless airo] safer shutdown sequence
  o [wireless airo] eliminate infinite loop
  o [wireless airo] makes the card passive when entering monitor mode
  o [wireless airo] adds support for noise level reporting (if available)

<bjorn.helgaas:hp.com>:
  o trivial 2.4 HCDP documentation/config patch

<herbert:13thfloor.at>:
  o ROOT NFS fixes

<marcelo:logos.cnet>:
  o NMI watchdog documentation for x86-64

<mike.miller:hp.com>:
  o cciss update: author change
  o cciss update: Fix problem with shared IRQs

Adam Radford:
  o 3ware driver update

Adrian Bunk:
  o fix IPMI build error #if CONFIG_ACPI_HT_ONLY

Benjamin Herrenschmidt:
  o ppc32: export hash_table_lock on SMP for MacOnLinux

Dave Kleikamp:
  o JFS: write_super_lockfs should mark superblock clean

Jan Harkes:
  o Coda fixes

Jay Vosburgh:
  o [netdrvr bonding] fix ifenslave ia64 build

Jeff Garzik:
  o [netdrvr] add new broadcom 440x net driver, "b44"

Marc-Christian Petersen:
  o Fix AGPGART problem with 4GB RAM
  o Fix irq handling of IO-APIC edge IRQs on UP
  o MXCSR Handler Unspecified Vulnerability
  o Fix /proc/self security issue
  o Add missing -EFAULT for sysctl

Marcelo Tosatti:
  o Changed EXTRAVERSION to -pre10

Oleg Drokin:
  o reiserfs: fix savelinks for bigendian arches

Petr Vandrovec:
  o ncpfs: Support for clustered NetWare volumes
  o matroxfb: extended support for mplayer

Shmulik Hen:
  o [bonding] fix ifenslave ABI bug
  o [netdrvr bonding] fix ARP monitoring bug

Trond Myklebust:
  o If an RPC request has to be resent due to a timeout, it turns out that call_encode() may cause rq_rcv_buf to be reset despite the fact that a reply might be delivered at any moment by a softirq.
  o If xdr_kmap() fails, we need to ensure that it unmaps all the pages, and returns 0. We don't want to be sending partial RPC requests to the server.

Willy Tarreau:
  o ACPI poweroff fix
  o [netdrvr bonding] fix a typo in the MODULE_PARM_DESC
  o [netdrvr bonding] fix kernel panic when optional feature used