* 2.4.23 hard lock, 100% reproducible.
@ 2003-12-07 6:50 Alex Davis
2003-12-07 6:55 ` William Lee Irwin III
2003-12-07 7:33 ` Mark Symonds
0 siblings, 2 replies; 30+ messages in thread
From: Alex Davis @ 2003-12-07 6:50 UTC (permalink / raw)
To: linux-kernel
I had the exact same thing happen to me about
a year ago, with the same error message. It
started after I had upraded my kernel. It
turned out one of my RAM sticks had gone
bad. Do you have another machine you can
test 2.4.23 with?
[...]
>
> Not even sysrq?
>
I did get msgs twice here in the past few hours,
but only sometimes does it give anything. Here's
what it said:
Unable to handle kernel NULL pointer
dereference at virtual address: 00000000
printing eip:
c02363dd
*pde=00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c02363d>] Not tainted
EFLAGS: 00010217
eax: 00000006 ebx: 00000000 ecx: 7a01a8c0 ecx: c700b2a0
esi: c0299ce0 edi: 000001b7 ebp: c0299d94 esp: c0299c54
ds: 0018 es: 0018 ss: 0018
process: swapper (pid: 0, stackpage = c0299000)
Other than that, nothing. Is there a patch out there
that will simply make 2.4.22 secure? Things run great
on that kernel.
--
Mark
-
=====
I code, therefore I am
__________________________________
Do you Yahoo!?
New Yahoo! Photos - easier uploading and sharing.
http://photos.yahoo.com/
^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 6:50 2.4.23 hard lock, 100% reproducible Alex Davis @ 2003-12-07 6:55 ` William Lee Irwin III [not found] ` <046a01c3bca1$267ba5e0$7a01a8c0@gandalf> 2003-12-07 7:33 ` Mark Symonds 1 sibling, 1 reply; 30+ messages in thread From: William Lee Irwin III @ 2003-12-07 6:55 UTC (permalink / raw) To: Alex Davis; +Cc: linux-kernel On Sat, Dec 06, 2003 at 10:50:17PM -0800, Alex Davis wrote: > Unable to handle kernel NULL pointer > dereference at virtual address: 00000000 > printing eip: > c02363dd > *pde=00000000 > Oops: 0000 > CPU: 0 > EIP: 0010:[<c02363d>] Not tainted > EFLAGS: 00010217 > eax: 00000006 ebx: 00000000 ecx: 7a01a8c0 ecx: c700b2a0 > esi: c0299ce0 edi: 000001b7 ebp: c0299d94 esp: c0299c54 > ds: 0018 es: 0018 ss: 0018 > process: swapper (pid: 0, stackpage = c0299000) > Other than that, nothing. Is there a patch out there > that will simply make 2.4.22 secure? Things run great > on that kernel. Compile your kernel with debug symbols and use addr2line on that EIP. -- wli ^ permalink raw reply [flat|nested] 30+ messages in thread
[parent not found: <046a01c3bca1$267ba5e0$7a01a8c0@gandalf>]
* Re: 2.4.23 hard lock, 100% reproducible. [not found] ` <046a01c3bca1$267ba5e0$7a01a8c0@gandalf> @ 2003-12-07 9:07 ` William Lee Irwin III 2003-12-07 9:16 ` Keith Owens 0 siblings, 1 reply; 30+ messages in thread From: William Lee Irwin III @ 2003-12-07 9:07 UTC (permalink / raw) To: Mark Symonds; +Cc: linux-kernel At some point in the past, I wrote: >> Compile your kernel with debug symbols and use addr2line on that EIP. On Sun, Dec 07, 2003 at 01:04:41AM -0800, Mark Symonds wrote: > I'm not a kernel developer, but here goes: > puggy:/usr/src/linux/2.4.23# addr2line c02363dd -e vmlinux > ??:0 > puggy:/usr/src/linux/2.4.23# > That doesn't look right. I compiled with kernel debugging > enabled only; none of the other options. Should I enable > others/all of them? For 2.4 you have to add -g to the compile options by hand in the kernel makefile. -- wli ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 9:07 ` William Lee Irwin III @ 2003-12-07 9:16 ` Keith Owens 2003-12-07 9:36 ` William Lee Irwin III 2003-12-07 10:01 ` Mark Symonds 0 siblings, 2 replies; 30+ messages in thread From: Keith Owens @ 2003-12-07 9:16 UTC (permalink / raw) To: Mark Symonds, linux-kernel On Sun, Dec 07, 2003 at 01:04:41AM -0800, Mark Symonds wrote: > I'm not a kernel developer, but here goes: > puggy:/usr/src/linux/2.4.23# addr2line c02363dd -e vmlinux > ??:0 addr2line requires compiling with -g. You can also do ksymoops -m /path/to/your/System.map -A c02363dd which does not require a recompile. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 9:16 ` Keith Owens @ 2003-12-07 9:36 ` William Lee Irwin III 2003-12-08 16:34 ` Oliver Teuber 2003-12-07 10:01 ` Mark Symonds 1 sibling, 1 reply; 30+ messages in thread From: William Lee Irwin III @ 2003-12-07 9:36 UTC (permalink / raw) To: Keith Owens; +Cc: Mark Symonds, linux-kernel On Sun, Dec 07, 2003 at 01:04:41AM -0800, Mark Symonds wrote: >> I'm not a kernel developer, but here goes: >> puggy:/usr/src/linux/2.4.23# addr2line c02363dd -e vmlinux >> ??:0 On Sun, Dec 07, 2003 at 08:16:54PM +1100, Keith Owens wrote: > addr2line requires compiling with -g. You can also do > ksymoops -m /path/to/your/System.map -A c02363dd > which does not require a recompile. That certainly would have been faster; I'll suggest that first next time (though addr2line has the small advantage of handing back a line number). -- wli ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 9:36 ` William Lee Irwin III @ 2003-12-08 16:34 ` Oliver Teuber 0 siblings, 0 replies; 30+ messages in thread From: Oliver Teuber @ 2003-12-08 16:34 UTC (permalink / raw) To: William Lee Irwin III; +Cc: Keith Owens, linux-kernel [-- Attachment #1: Type: text/plain, Size: 152 bytes --] hi i have got some oops too ... reported some days earlier. yours, oliver teuber http://marc.theaimsgroup.com/?l=linux-kernel&m=107036079102352&w=2 [-- Attachment #2: oops-2423-20031202.txt --] [-- Type: text/plain, Size: 3303 bytes --] ksymoops 2.4.9 on i686 2.4.23. Options used -v /usr/src/linux/vmlinux (specified) -k /proc/ksyms (default) -l /proc/modules (default) -o /lib/modules/2.4.23/ (default) -m /usr/src/linux/System.map (default) Reading Oops report from the terminal Oops: 0000 CPU: 0 EIP: 0010:[<c0119780>] Not tainted Using defaults from ksymoops -t elf32-i386 -a i386 EFLAGS: 00010086 eax: c83a643c ebx: 00000000 ecx: 00000001 edx: 00000001 esi: ce6d2980 edi: c83a643c ebp: cdb61a6c esp: cdb61a54 ds: 0018 es: 0018 ss: 0018 Process lpd (pid: 4136, stackpage=cdb61000) Stack: 00000001 00000286 00000001 c41c1680 ce6d2980 00000000 00000046 c02282d4 cfca1400 00000000 00000202 c41c1680 c022789b c41c1680 c8c9b180 c02288d1 ce6d2980 cfca1560 fffffffd c022c7cb ce6d2980 cdb61af0 00000001 c033aa88 Call Trace: [<c02282d4>] [<c022789b>] [<c02288d1>] [<c022c7cb>] [<c0120bb1>] [<c010aa19>] [<c010cf18>] [<d094c782>] [<d094cbe4>] [<d094c3c0>] [<d094d048>] [<d094ead2>] [<d094f0f2>] [<d095e82f>] [<d093d719>] [<d095e83d>] [<d0955057>] [<d093ebd0>] [<d095e83d>] [<c0150356>] [<c013e224>] [<c013cd7d>] [<c013ce0b>] [<c0108f27>] Code: 8b 13 0f 18 02 39 c3 74 76 8d b4 26 00 00 00 00 8b 4b fc 8b >>EIP; c0119780 <__wake_up+20/b0> <===== >>eax; c83a643c <_end+80372e8/1057bf0c> >>esi; ce6d2980 <_end+e36382c/1057bf0c> >>edi; c83a643c <_end+80372e8/1057bf0c> >>ebp; cdb61a6c <_end+d7f2918/1057bf0c> >>esp; cdb61a54 <_end+d7f2900/1057bf0c> Trace; c02282d4 <sock_def_write_space+64/90> Trace; c022789b <sock_wfree+3b/40> Trace; c02288d1 <__kfree_skb+41/100> Trace; c022c7cb <net_tx_action+2b/b0> Trace; c0120bb1 <do_softirq+51/a0> Trace; c010aa19 <do_IRQ+99/b0> Trace; c010cf18 <call_do_IRQ+5/d> Trace; d094c782 <[reiserfs]comp_keys+362/3f0> Trace; d094cbe4 <[reiserfs]is_tree_node+64/70> Trace; d094c3c0 <[reiserfs]__constant_memcpy+c0/120> Trace; d094d048 <[reiserfs]search_for_position_by_key+f8/4c0> Trace; d094ead2 <[reiserfs]reiserfs_cut_from_item+222/4b0> Trace; d094f0f2 <[reiserfs]reiserfs_do_truncate+322/580> Trace; d095e82f <[reiserfs].rodata.end+5ab0/5ca1> Trace; d093d719 <[reiserfs]reiserfs_truncate_file+e9/230> Trace; d095e83d <[reiserfs].rodata.end+5abe/5ca1> Trace; d0955057 <[reiserfs]journal_end+27/30> Trace; d093ebd0 <[reiserfs]reiserfs_file_release+3a0/450> Trace; d095e83d <[reiserfs].rodata.end+5abe/5ca1> Trace; c0150356 <locks_remove_flock+76/80> Trace; c013e224 <fput+114/120> Trace; c013cd7d <filp_close+4d/90> Trace; c013ce0b <sys_close+4b/60> Trace; c0108f27 <system_call+33/38> Code; c0119780 <__wake_up+20/b0> 00000000 <_EIP>: Code; c0119780 <__wake_up+20/b0> <===== 0: 8b 13 mov (%ebx),%edx <===== Code; c0119782 <__wake_up+22/b0> 2: 0f 18 02 prefetchnta (%edx) Code; c0119785 <__wake_up+25/b0> 5: 39 c3 cmp %eax,%ebx Code; c0119787 <__wake_up+27/b0> 7: 74 76 je 7f <_EIP+0x7f> Code; c0119789 <__wake_up+29/b0> 9: 8d b4 26 00 00 00 00 lea 0x0(%esi,1),%esi Code; c0119790 <__wake_up+30/b0> 10: 8b 4b fc mov 0xfffffffc(%ebx),%ecx Code; c0119793 <__wake_up+33/b0> 13: 8b 00 mov (%eax),%eax <0>Kernel panic: Aiee, killing interrupt handler! [-- Attachment #3: oops-2423-20031203.txt --] [-- Type: text/plain, Size: 2257 bytes --] ksymoops 2.4.9 on i686 2.4.23. Options used -v /usr/src/linux/vmlinux (specified) -k /proc/ksyms (default) -l /proc/modules (default) -o /lib/modules/2.4.23/ (default) -m /usr/src/linux/System.map (default) Reading Oops report from the terminal Oops: 0000 CPU: 0 EIP: 0010:[<c0119780>] Not tainted Using defaults from ksymoops -t elf32-i386 -a i386 EFLAGS: 00010086 eax: c86ef23c ebx: 00000000 ecx: 00000001 edx: 00000001 esi: ce074e80 edi: c86ef23c ebp: cee19f40 esp: cee19f28 ds: 0018 es: 0018 ss: 0018 Process if (pid: 4571, stackpage=cee19000) Stack: 00000001 00000286 00000001 cd876c80 ce074e80 00000000 00000046 c02282d4 00001000 00000000 00000202 cd876c80 c022789b cd876c80 cb01ca80 c02288d1 ce074e80 cfcbcd60 fffffffd c022c7cb ce074e80 cee19fc4 00000001 c033aa88 Call Trace: [<c02282d4>] [<c022789b>] [<c02288d1>] [<c022c7cb>] [<c0120bb1>] [<c010aa19>] [<c010cf18>] Code: 8b 13 0f 18 02 39 c3 74 76 8d b4 26 00 00 00 00 8b 4b fc 8b >>EIP; c0119780 <__wake_up+20/b0> <===== >>eax; c86ef23c <_end+83800e8/1057bf0c> >>esi; ce074e80 <_end+dd05d2c/1057bf0c> >>edi; c86ef23c <_end+83800e8/1057bf0c> >>ebp; cee19f40 <_end+eaaadec/1057bf0c> >>esp; cee19f28 <_end+eaaadd4/1057bf0c> Trace; c02282d4 <sock_def_write_space+64/90> Trace; c022789b <sock_wfree+3b/40> Trace; c02288d1 <__kfree_skb+41/100> Trace; c022c7cb <net_tx_action+2b/b0> Trace; c0120bb1 <do_softirq+51/a0> Trace; c010aa19 <do_IRQ+99/b0> Trace; c010cf18 <call_do_IRQ+5/d> Code; c0119780 <__wake_up+20/b0> 00000000 <_EIP>: Code; c0119780 <__wake_up+20/b0> <===== 0: 8b 13 mov (%ebx),%edx <===== Code; c0119782 <__wake_up+22/b0> 2: 0f 18 02 prefetchnta (%edx) Code; c0119785 <__wake_up+25/b0> 5: 39 c3 cmp %eax,%ebx Code; c0119787 <__wake_up+27/b0> 7: 74 76 je 7f <_EIP+0x7f> Code; c0119789 <__wake_up+29/b0> 9: 8d b4 26 00 00 00 00 lea 0x0(%esi,1),%esi Code; c0119790 <__wake_up+30/b0> 10: 8b 4b fc mov 0xfffffffc(%ebx),%ecx Code; c0119793 <__wake_up+33/b0> 13: 8b 00 mov (%eax),%eax <0>Kernel panic: Aiee, killing interrupt handler! ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 9:16 ` Keith Owens 2003-12-07 9:36 ` William Lee Irwin III @ 2003-12-07 10:01 ` Mark Symonds 2003-12-07 11:34 ` Martin Josefsson 2003-12-07 14:40 ` Marcelo Tosatti 1 sibling, 2 replies; 30+ messages in thread From: Mark Symonds @ 2003-12-07 10:01 UTC (permalink / raw) To: Keith Owens, linux-kernel [...] > > addr2line requires compiling with -g. You can also do > ksymoops -m /path/to/your/System.map -A c02363dd > which does not require a recompile. > Excellent, this is alot easier. Should note that this kernel is compiled without support for loadable modules. Here goes: ------------- puggy:/usr/src/linux/2.4.23# ksymoops -m ./System.map -A c02363dd ksymoops 2.4.9 on i686 2.4.23. Options used -V (default) -k /proc/ksyms (default) -l /proc/modules (default) -o /lib/modules/2.4.23/ (default) -m ./System.map (specified) Error (regular_file): read_ksyms stat /proc/ksyms failed ksymoops: No such file or directory No modules in ksyms, skipping objects No ksyms, skipping lsmod Adhoc c02363dd <tcp_print_conntrack+2d/60> 1 error issued. Results may not be reliable. puggy:/usr/src/linux/2.4.23# ------------- -- Mark ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 10:01 ` Mark Symonds @ 2003-12-07 11:34 ` Martin Josefsson 2003-12-07 14:40 ` Marcelo Tosatti 1 sibling, 0 replies; 30+ messages in thread From: Martin Josefsson @ 2003-12-07 11:34 UTC (permalink / raw) To: Mark Symonds; +Cc: Keith Owens, linux-kernel [-- Attachment #1: Type: text/plain, Size: 941 bytes --] On Sun, 2003-12-07 at 11:01, Mark Symonds wrote: > puggy:/usr/src/linux/2.4.23# ksymoops -m ./System.map -A c02363dd > ksymoops 2.4.9 on i686 2.4.23. Options used > -V (default) > -k /proc/ksyms (default) > -l /proc/modules (default) > -o /lib/modules/2.4.23/ (default) > -m ./System.map (specified) > > Error (regular_file): read_ksyms stat /proc/ksyms failed > ksymoops: No such file or directory > No modules in ksyms, skipping objects > No ksyms, skipping lsmod > > > Adhoc c02363dd <tcp_print_conntrack+2d/60> Hmm, it looks like it's this line in tcp_print_conntrack() return sprintf(buffer, "%s ", tcp_conntrack_names[state]); But it would be good if you could recompile and use addr2line (or gdb) to get the exact line (offsets and memory-addresses depends on compiler used and other stuff). I took a quick look at the code but didn't see anything obvious. -- /Martin [-- Attachment #2: This is a digitally signed message part --] [-- Type: application/pgp-signature, Size: 189 bytes --] ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 10:01 ` Mark Symonds 2003-12-07 11:34 ` Martin Josefsson @ 2003-12-07 14:40 ` Marcelo Tosatti 2003-12-07 23:18 ` Mark Symonds 1 sibling, 1 reply; 30+ messages in thread From: Marcelo Tosatti @ 2003-12-07 14:40 UTC (permalink / raw) To: Mark Symonds Cc: Keith Owens, linux-kernel, David S. Miller, William Lee Irwin III, Harald Welte The first oops looks like: Unable to handle kernel NULL pointer dereference at virtual address: 00000000 printing eip: c02363dd *pde=00000000 Oops: 0000 CPU: 0 EIP: 0010:[<c02363d>] Not tainted EFLAGS: 00010217 eax: 00000006 ebx: 00000000 ecx: 7a01a8c0 ecx: c700b2a0 esi: c0299ce0 edi: 000001b7 ebp: c0299d94 esp: c0299c54 ds: 0018 es: 0018 ss: 0018 process: swapper (pid: 0, stackpage = c0299000) Isnt it a bit weird that the full backtrace is not reported ? wli suggests that might stack corruption. I dont see any suspicious change around tcp_print_conntrack(). Any clues? On Sun, 7 Dec 2003, Mark Symonds wrote: > > [...] > > > > addr2line requires compiling with -g. You can also do > > ksymoops -m /path/to/your/System.map -A c02363dd > > which does not require a recompile. > > > > Excellent, this is alot easier. Should note that this > kernel is compiled without support for loadable modules. > Here goes: > > ------------- > > puggy:/usr/src/linux/2.4.23# ksymoops -m ./System.map -A c02363dd > ksymoops 2.4.9 on i686 2.4.23. Options used > -V (default) > -k /proc/ksyms (default) > -l /proc/modules (default) > -o /lib/modules/2.4.23/ (default) > -m ./System.map (specified) > > Error (regular_file): read_ksyms stat /proc/ksyms failed > ksymoops: No such file or directory > No modules in ksyms, skipping objects > No ksyms, skipping lsmod > > > Adhoc c02363dd <tcp_print_conntrack+2d/60> > > 1 error issued. Results may not be reliable. > puggy:/usr/src/linux/2.4.23# ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 14:40 ` Marcelo Tosatti @ 2003-12-07 23:18 ` Mark Symonds 2003-12-08 4:53 ` William Lee Irwin III ` (2 more replies) 0 siblings, 3 replies; 30+ messages in thread From: Mark Symonds @ 2003-12-07 23:18 UTC (permalink / raw) To: Marcelo Tosatti Cc: Keith Owens, linux-kernel, David S. Miller, William Lee Irwin III, Harald Welte Hi, > > The first oops looks like: > > Unable to handle kernel NULL pointer > dereference at virtual address: 00000000 > [...] > > > Isnt it a bit weird that the full backtrace is not reported ? > > wli suggests that might stack corruption. > My bad - wrote it down by hand originally since it was locked hard. > > I dont see any suspicious change around tcp_print_conntrack(). > > Any clues? > I'm using ipchains compatability in there, looks like this is a possible cause - getting a patch right now, will test and let y'all know (and then switch to iptables, finally). -- Mark ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 23:18 ` Mark Symonds @ 2003-12-08 4:53 ` William Lee Irwin III 2003-12-08 7:12 ` Tim Timmerman 2003-12-08 10:17 ` Marcelo Tosatti 2 siblings, 0 replies; 30+ messages in thread From: William Lee Irwin III @ 2003-12-08 4:53 UTC (permalink / raw) To: Mark Symonds Cc: Marcelo Tosatti, Keith Owens, linux-kernel, David S. Miller, Harald Welte On Sun, Dec 07, 2003 at 03:18:24PM -0800, Mark Symonds wrote: > I'm using ipchains compatability in there, looks like > this is a possible cause - getting a patch right now, > will test and let y'all know (and then switch to > iptables, finally). For the purposes of fixing the bogon, it might be helpful to stick to whatever triggered the problem just long enough to extract more information. -- wli ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 23:18 ` Mark Symonds 2003-12-08 4:53 ` William Lee Irwin III @ 2003-12-08 7:12 ` Tim Timmerman 2003-12-08 11:06 ` Marcelo Tosatti 2003-12-08 10:17 ` Marcelo Tosatti 2 siblings, 1 reply; 30+ messages in thread From: Tim Timmerman @ 2003-12-08 7:12 UTC (permalink / raw) To: Mark Symonds, linux-kernel; +Cc: Marcelo Tosatti >>>>> "Mark" == Mark Symonds <mark@symonds.net> writes: Mark> Hi, >> >> The first oops looks like: >> >> Unable to handle kernel NULL pointer >> dereference at virtual address: 00000000 >> Mark> [...] Mark> I'm using ipchains compatability in there, looks like Mark> this is a possible cause - getting a patch right now, Mark> will test and let y'all know (and then switch to Mark> iptables, finally). Let me just add a me-too here. Haven't got the oops on my desk, here, but from what I could see, the error occurred in find_appropriate_src, somewhere in ipchains. Further, possibly irrelevant datapoint: ABIT BP6, ne2k-pci and 3Com590 network cards. When the oops occurs, everything locks, capslock and scrolllock are lit. I can reproduce the error by letting a second system ping the first, on the internal network. Sometimes it doesn't even complete a full boot. I'll try and capture more detail tonight. TimT. -- tim.timmerman@asml.nl 040-2683613 timt@timt.org Voodoo Programmer/Keeper of the Rubber Chicken One time I went to a museum where all the work in the museum had been done by children. They had all the paintings up on refrigerators. -- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. ASML is neither liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-08 7:12 ` Tim Timmerman @ 2003-12-08 11:06 ` Marcelo Tosatti 2003-12-09 6:29 ` Tim Timmerman 0 siblings, 1 reply; 30+ messages in thread From: Marcelo Tosatti @ 2003-12-08 11:06 UTC (permalink / raw) To: Tim Timmerman; +Cc: Mark Symonds, linux-kernel, Marcelo Tosatti On Mon, 8 Dec 2003, Tim Timmerman wrote: > >>>>> "Mark" == Mark Symonds <mark@symonds.net> writes: > > Mark> Hi, > > >> > >> The first oops looks like: > >> > >> Unable to handle kernel NULL pointer > >> dereference at virtual address: 00000000 > >> > Mark> [...] > > Mark> I'm using ipchains compatability in there, looks like > Mark> this is a possible cause - getting a patch right now, > Mark> will test and let y'all know (and then switch to > Mark> iptables, finally). > Let me just add a me-too here. > > Haven't got the oops on my desk, here, but from what I could > see, the error occurred in find_appropriate_src, somewhere in > ipchains. > > Further, possibly irrelevant datapoint: ABIT BP6, ne2k-pci and > 3Com590 network cards. When the oops occurs, everything locks, > capslock and scrolllock are lit. > > I can reproduce the error by letting a second system ping the > first, on the internal network. Sometimes it doesn't even > complete a full boot. > > I'll try and capture more detail tonight. Tim, Please try the updated 2.4 BK tree (you can use -bk5, http://www.kernel.org/pub/linux/kernel/v2.4/snapshots/patch-2.4.23-bk5.bz2). It contains a fix for a known bug in the netfilter which might what you're hitting. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-08 11:06 ` Marcelo Tosatti @ 2003-12-09 6:29 ` Tim Timmerman 0 siblings, 0 replies; 30+ messages in thread From: Tim Timmerman @ 2003-12-09 6:29 UTC (permalink / raw) To: Marcelo Tosatti; +Cc: Tim Timmerman, Mark Symonds, linux-kernel >>>>> "Marcelo" == Marcelo Tosatti <marcelo.tosatti@cyclades.com> writes: Marcelo> On Mon, 8 Dec 2003, Tim Timmerman wrote: >> Let me just add a me-too here. >> >> Haven't got the oops on my desk, here, but from what I could >> see, the error occurred in find_appropriate_src, somewhere in >> ipchains. >> >> Further, possibly irrelevant datapoint: ABIT BP6, ne2k-pci and >> 3Com590 network cards. When the oops occurs, everything locks, >> capslock and scrolllock are lit. >> >> I can reproduce the error by letting a second system ping the >> first, on the internal network. Sometimes it doesn't even >> complete a full boot. >> >> I'll try and capture more detail tonight. Marcelo> Tim, Marcelo> Please try the updated 2.4 BK tree (you can use -bk5, Marcelo> http://www.kernel.org/pub/linux/kernel/v2.4/snapshots/patch-2.4.23-bk5.bz2). Marcelo> It contains a fix for a known bug in the netfilter which Marcelo> might what you're hitting. Marcello, Thanks ! I can confirm that this seems to fix the bug: system has been running the patched kernel for the past 12 hours, and is stable, even under load. TimT -- tim.timmerman@asml.nl 040-2683613 timt@timt.org Voodoo Programmer/Keeper of the Rubber Chicken Do Lipton employees take coffee breaks? -- The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. ASML is neither liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 23:18 ` Mark Symonds 2003-12-08 4:53 ` William Lee Irwin III 2003-12-08 7:12 ` Tim Timmerman @ 2003-12-08 10:17 ` Marcelo Tosatti 2003-12-08 15:54 ` Stephan von Krawczynski 2003-12-08 19:56 ` Mark Symonds 2 siblings, 2 replies; 30+ messages in thread From: Marcelo Tosatti @ 2003-12-08 10:17 UTC (permalink / raw) To: Mark Symonds Cc: linux-kernel, David S. Miller, William Lee Irwin III, Harald Welte On Sun, 7 Dec 2003, Mark Symonds wrote: > > Hi, > > > > > The first oops looks like: > > > > Unable to handle kernel NULL pointer > > dereference at virtual address: 00000000 > > > [...] > > > > > > Isnt it a bit weird that the full backtrace is not reported ? > > > > wli suggests that might stack corruption. > > > > My bad - wrote it down by hand originally since > it was locked hard. > > > > > I dont see any suspicious change around tcp_print_conntrack(). > > > > Any clues? > > > > I'm using ipchains compatability in there, looks like > this is a possible cause - getting a patch right now, > will test and let y'all know (and then switch to > iptables, finally). Mark, Please try the latest BK tree. There was a known bug in the netfilter code which could cause the lockups. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-08 10:17 ` Marcelo Tosatti @ 2003-12-08 15:54 ` Stephan von Krawczynski 2003-12-08 16:15 ` Marcelo Tosatti 2003-12-08 19:56 ` Mark Symonds 1 sibling, 1 reply; 30+ messages in thread From: Stephan von Krawczynski @ 2003-12-08 15:54 UTC (permalink / raw) To: Marcelo Tosatti; +Cc: mark, linux-kernel, davem, wli, laforge On Mon, 8 Dec 2003 08:17:30 -0200 (BRST) Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote: > Mark, > > Please try the latest BK tree. There was a known bug in the netfilter code > which could cause the lockups. ...which leads me to the most-simple question: where can I find a changelog for 2.4.23-bkX from www.kernel.org ? Inside ? Regards, Stephan ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-08 15:54 ` Stephan von Krawczynski @ 2003-12-08 16:15 ` Marcelo Tosatti 2003-12-08 16:31 ` Jeff Garzik 2003-12-08 16:55 ` Stephan von Krawczynski 0 siblings, 2 replies; 30+ messages in thread From: Marcelo Tosatti @ 2003-12-08 16:15 UTC (permalink / raw) To: Stephan von Krawczynski Cc: Marcelo Tosatti, mark, linux-kernel, William Lee Irwin III On Mon, 8 Dec 2003, Stephan von Krawczynski wrote: > On Mon, 8 Dec 2003 08:17:30 -0200 (BRST) > Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote: > > > Mark, > > > > Please try the latest BK tree. There was a known bug in the netfilter code > > which could cause the lockups. > > ...which leads me to the most-simple question: where can I find a changelog for > 2.4.23-bkX from www.kernel.org ? Inside ? I believe there is nothing which generates 2.4.2x-bk changelogs right now. It can be easily done. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-08 16:15 ` Marcelo Tosatti @ 2003-12-08 16:31 ` Jeff Garzik 2003-12-08 16:55 ` Stephan von Krawczynski 1 sibling, 0 replies; 30+ messages in thread From: Jeff Garzik @ 2003-12-08 16:31 UTC (permalink / raw) To: Marcelo Tosatti Cc: Stephan von Krawczynski, mark, linux-kernel, William Lee Irwin III On Mon, Dec 08, 2003 at 02:15:27PM -0200, Marcelo Tosatti wrote: > > > On Mon, 8 Dec 2003, Stephan von Krawczynski wrote: > > > On Mon, 8 Dec 2003 08:17:30 -0200 (BRST) > > Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote: > > > > > Mark, > > > > > > Please try the latest BK tree. There was a known bug in the netfilter code > > > which could cause the lockups. > > > > ...which leads me to the most-simple question: where can I find a changelog for > > 2.4.23-bkX from www.kernel.org ? Inside ? > > I believe there is nothing which generates 2.4.2x-bk changelogs right now. It has been automatically generated for a while now ;-) ... http://www.kernel.org/pub/linux/kernel/v2.4/snapshots/patch-2.4.23-bk5.log http://www.kernel.org/pub/linux/kernel/v2.4/snapshots/patch-2.4.23-bk6.log ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-08 16:15 ` Marcelo Tosatti 2003-12-08 16:31 ` Jeff Garzik @ 2003-12-08 16:55 ` Stephan von Krawczynski 1 sibling, 0 replies; 30+ messages in thread From: Stephan von Krawczynski @ 2003-12-08 16:55 UTC (permalink / raw) To: Marcelo Tosatti; +Cc: mark, linux-kernel, wli On Mon, 8 Dec 2003 14:15:27 -0200 (BRST) Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote: > > > On Mon, 8 Dec 2003, Stephan von Krawczynski wrote: > > > On Mon, 8 Dec 2003 08:17:30 -0200 (BRST) > > Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote: > > > > > Mark, > > > > > > Please try the latest BK tree. There was a known bug in the netfilter > > > code which could cause the lockups. > > > > ...which leads me to the most-simple question: where can I find a changelog > > for 2.4.23-bkX from www.kernel.org ? Inside ? > > I believe there is nothing which generates 2.4.2x-bk changelogs right now. > > It can be easily done. Are you continuing to create pre-patches with readable changelogs on kernel.org? ONTOPIC: indeed it took me only few days to crash a 2.4.23 with heavy net(filter) usage... Mine broke with "killing interrupt handler ..." Regards, Stephan ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-08 10:17 ` Marcelo Tosatti 2003-12-08 15:54 ` Stephan von Krawczynski @ 2003-12-08 19:56 ` Mark Symonds 1 sibling, 0 replies; 30+ messages in thread From: Mark Symonds @ 2003-12-08 19:56 UTC (permalink / raw) To: Marcelo Tosatti Cc: linux-kernel, David S. Miller, William Lee Irwin III, Harald Welte > > > > I'm using ipchains compatability in there, looks like > > this is a possible cause - getting a patch right now, > > will test and let y'all know (and then switch to > > iptables, finally). > > Mark, > > Please try the latest BK tree. There was a known bug in the netfilter code > which could cause the lockups. > What I did a couple of days ago was remove ipchains and switch to iptables (instead of applying any patches or anything). Ever since the ipchains code was removed from my kernel, the box has been running fine. Thanks much! -- Mark ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 6:50 2.4.23 hard lock, 100% reproducible Alex Davis 2003-12-07 6:55 ` William Lee Irwin III @ 2003-12-07 7:33 ` Mark Symonds 2003-12-07 16:24 ` Alex Davis 1 sibling, 1 reply; 30+ messages in thread From: Mark Symonds @ 2003-12-07 7:33 UTC (permalink / raw) To: Alex Davis, linux-kernel > I had the exact same thing happen to me about > a year ago, with the same error message. It > started after I had upraded my kernel. It > turned out one of my RAM sticks had gone > bad. Do you have another machine you can > test 2.4.23 with? > I do, but not with identical hardware. Thing is it ran just fine for months on previous kernels, and even now will run just fine with them. The crashing only happens when using 2.4.23. -- Mark ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 7:33 ` Mark Symonds @ 2003-12-07 16:24 ` Alex Davis 0 siblings, 0 replies; 30+ messages in thread From: Alex Davis @ 2003-12-07 16:24 UTC (permalink / raw) To: Mark Symonds, linux-kernel Would it be possible to swap the RAM from another machine? --- Mark Symonds <mark@symonds.net> wrote: > > > I had the exact same thing happen to me about > > a year ago, with the same error message. It > > started after I had upraded my kernel. It > > turned out one of my RAM sticks had gone > > bad. Do you have another machine you can > > test 2.4.23 with? > > > > I do, but not with identical hardware. Thing is > it ran just fine for months on previous kernels, > and even now will run just fine with them. The > crashing only happens when using 2.4.23. > > -- > Mark > ===== I code, therefore I am __________________________________ Do you Yahoo!? New Yahoo! Photos - easier uploading and sharing. http://photos.yahoo.com/ ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible.
@ 2003-12-08 17:06 Xose Vazquez Perez
0 siblings, 0 replies; 30+ messages in thread
From: Xose Vazquez Perez @ 2003-12-08 17:06 UTC (permalink / raw)
To: linux-kernel
Stephan von Krawczynski wrote:
> ...which leads me to the most-simple question: where can I find a changelog for
> 2.4.23-bkX from www.kernel.org ? Inside ?
BK used to mirror it from http://linux.bkbits.net:8080/linux-2.4/ to
http://www.kernel.org/pub/linux/kernel/v2.4/testing/cset/
But since last week it doesn't work.
^ permalink raw reply [flat|nested] 30+ messages in thread
* 2.4.23 hard lock, 100% reproducible. @ 2003-12-07 2:36 Mark Symonds 2003-12-07 3:30 ` Philippe Troin 0 siblings, 1 reply; 30+ messages in thread From: Mark Symonds @ 2003-12-07 2:36 UTC (permalink / raw) To: linux-kernel Hi, I've got a machine here that is locking hard under 2.4.23. Normally would suspect it's a hardware problem but it runs fine on 2.4.22 and also 2.2 series kernels. In a bit of a quandry here since that box has shell users... I'm getting no oopses on the monitor nor in the logs - this is a hard, instantaneous crash. No kbd, no nothing, good night. I've got a kernel compiling right now with hacking support, but none of the additional hacking options are enabled. Wondering if anyone else has seen this? lspci output is below, will wait until requested before dumping a bunch of crap about my hardware onto the list. Anything I can do from userland, let me know. -- Mark ----------------------- symonds:~# lspci 00:00.0 Host bridge: Intel Corp. 82810 GMCH [Graphics Memory Controller Hub] (rev 03) 00:01.0 VGA compatible controller: Intel Corp. 82810 CGC [Chipset Graphics Controller] (rev 03) 00:1e.0 PCI bridge: Intel Corp. 82801AA PCI Bridge (rev 02) 00:1f.0 ISA bridge: Intel Corp. 82801AA ISA Bridge (LPC) (rev 02) 00:1f.1 IDE interface: Intel Corp. 82801AA IDE (rev 02) 00:1f.2 USB Controller: Intel Corp. 82801AA USB (rev 02) 00:1f.3 SMBus: Intel Corp. 82801AA SMBus (rev 02) 01:09.0 Multimedia audio controller: Cirrus Logic Crystal CS4281 PCI Audio (rev 01) 01:0b.0 Ethernet controller: D-Link System Inc RTL8139 Ethernet (rev 10) 01:0d.0 Ethernet controller: VIA Technologies, Inc. VT86C100A [Rhine 10/100] (rev 06) 01:0e.0 Ethernet controller: Lite-On Communications Inc LNE100TX (rev 20) symonds:~# ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 2:36 Mark Symonds @ 2003-12-07 3:30 ` Philippe Troin 2003-12-07 4:34 ` Mark Symonds 0 siblings, 1 reply; 30+ messages in thread From: Philippe Troin @ 2003-12-07 3:30 UTC (permalink / raw) To: Mark Symonds; +Cc: linux-kernel Mark Symonds <mark@symonds.net> writes: > Hi, > > I've got a machine here that is locking hard under > 2.4.23. Normally would suspect it's a hardware problem > but it runs fine on 2.4.22 and also 2.2 series kernels. > In a bit of a quandry here since that box has shell > users... > > I'm getting no oopses on the monitor nor in the logs - > this is a hard, instantaneous crash. No kbd, no nothing, > good night. Not even sysrq? > I've got a kernel compiling right now with hacking > support, but none of the additional hacking options > are enabled. > > Wondering if anyone else has seen this? lspci output is > below, will wait until requested before dumping a bunch > of crap about my hardware onto the list. Same for me. 2.4.23 locks up hard. 2.4.22 worked perfectly (save the random usb oops). I was not able to capture a register trace or a process dump with sysrq because this box has a usb keyboard and usb seems to get shot. I've plugged a ps/2 keyboard to get more details the next lock-up. If this fail, I might try the NMI oopser. lspci: 00:00.0 Host bridge: VIA Technologies, Inc. VT82C693A/694x [Apollo PRO133x] (rev c4) 00:01.0 PCI bridge: VIA Technologies, Inc. VT82C598/694x [Apollo MVP3/Pro133x AGP] 00:07.0 ISA bridge: VIA Technologies, Inc. VT82C596 ISA [Mobile South] (rev 23) 00:07.1 IDE interface: VIA Technologies, Inc. Bus Master IDE (rev 10) 00:07.2 USB Controller: VIA Technologies, Inc. UHCI USB (rev 11) 00:07.3 Host bridge: VIA Technologies, Inc. VT82C596 Power Management (rev 30) 00:0f.0 Multimedia audio controller: Creative Labs SB Live! EMU10k1 (rev 08) 00:0f.1 Input device controller: Creative Labs SB Live! (rev 08) 00:10.0 Ethernet controller: 3Com Corporation 3c905 100BaseTX [Boomerang] 00:12.0 SCSI storage controller: Adaptec 7892A (rev 02) 01:00.0 VGA compatible controller: Matrox Graphics, Inc. MGA G400 AGP (rev 04) Phil. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 3:30 ` Philippe Troin @ 2003-12-07 4:34 ` Mark Symonds 2003-12-07 18:32 ` Chris Frey 0 siblings, 1 reply; 30+ messages in thread From: Mark Symonds @ 2003-12-07 4:34 UTC (permalink / raw) To: Philippe Troin; +Cc: linux-kernel [...] > > Not even sysrq? > I did get msgs twice here in the past few hours, but only sometimes does it give anything. Here's what it said: Unable to handle kernel NULL pointer dereference at virtual address: 00000000 printing eip: c02363dd *pde=00000000 Oops: 0000 CPU: 0 EIP: 0010:[<c02363d>] Not tainted EFLAGS: 00010217 eax: 00000006 ebx: 00000000 ecx: 7a01a8c0 ecx: c700b2a0 esi: c0299ce0 edi: 000001b7 ebp: c0299d94 esp: c0299c54 ds: 0018 es: 0018 ss: 0018 process: swapper (pid: 0, stackpage = c0299000) Other than that, nothing. Is there a patch out there that will simply make 2.4.22 secure? Things run great on that kernel. -- Mark ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 4:34 ` Mark Symonds @ 2003-12-07 18:32 ` Chris Frey 2003-12-07 18:49 ` James Bourne 0 siblings, 1 reply; 30+ messages in thread From: Chris Frey @ 2003-12-07 18:32 UTC (permalink / raw) To: Mark Symonds; +Cc: linux-kernel On Sat, Dec 06, 2003 at 08:34:32PM -0800, Mark Symonds wrote: > Other than that, nothing. Is there a patch out there > that will simply make 2.4.22 secure? Things run great > on that kernel. Here's the relevant section from patch-2.4.23 - Chris diff -urN linux-2.4.22/mm/mmap.c linux-2.4.23/mm/mmap.c --- linux-2.4.22/mm/mmap.c 2003-06-13 07:51:39.000000000 -0700 +++ linux-2.4.23/mm/mmap.c 2003-11-28 10:26:21.000000000 -0800 @@ -1041,6 +1041,9 @@ if (!len) return addr; + if ((addr + len) > TASK_SIZE || (addr + len) < addr) + return -EINVAL; + /* * mlock MCL_FUTURE? */ ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 18:32 ` Chris Frey @ 2003-12-07 18:49 ` James Bourne 2003-12-07 19:21 ` David Rees 0 siblings, 1 reply; 30+ messages in thread From: James Bourne @ 2003-12-07 18:49 UTC (permalink / raw) To: Chris Frey; +Cc: Mark Symonds, linux-kernel On Sun, 7 Dec 2003, Chris Frey wrote: > On Sat, Dec 06, 2003 at 08:34:32PM -0800, Mark Symonds wrote: > > Other than that, nothing. Is there a patch out there > > that will simply make 2.4.22 secure? Things run great > > on that kernel. > > Here's the relevant section from patch-2.4.23 Hi, This is included with the patch set I just posted, 2.4.22-uv3 available at http://www.hardrock.org/current-updates/linux-2.4.22-uv3-updates.patch. This also includes patches for some other issues in 2.4.22. I haven't yet posted one for 2.4.23 due to some outstanding issues with what appear to be (or at least could be) hard locks (such as this thread). Once these are resolved I will be releasing a -uv for 2.4.23. Regards James > > - Chris -- James Bourne | Email: jbourne@hardrock.org Unix Systems Administrator | WWW: http://www.hardrock.org Custom Unix Programming | Linux: The choice of a GNU generation ---------------------------------------------------------------------- "All you need's an occasional kick in the philosophy." Frank Herbert ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 18:49 ` James Bourne @ 2003-12-07 19:21 ` David Rees 2003-12-07 20:19 ` James Bourne 0 siblings, 1 reply; 30+ messages in thread From: David Rees @ 2003-12-07 19:21 UTC (permalink / raw) To: linux-kernel James Bourne wrote: > This is included with the patch set I just posted, 2.4.22-uv3 available at > http://www.hardrock.org/current-updates/linux-2.4.22-uv3-updates.patch. The correct link appears to be http://www.hardrock.org/kernel/current-updates/linux-2.4.22-uv3-updates.patch -Dave ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: 2.4.23 hard lock, 100% reproducible. 2003-12-07 19:21 ` David Rees @ 2003-12-07 20:19 ` James Bourne 0 siblings, 0 replies; 30+ messages in thread From: James Bourne @ 2003-12-07 20:19 UTC (permalink / raw) To: David Rees; +Cc: linux-kernel On Sun, 7 Dec 2003, David Rees wrote: > James Bourne wrote: > > This is included with the patch set I just posted, 2.4.22-uv3 available at > > http://www.hardrock.org/current-updates/linux-2.4.22-uv3-updates.patch. > > The correct link appears to be > http://www.hardrock.org/kernel/current-updates/linux-2.4.22-uv3-updates.patch Sorry, yes this is correct. Regards James > -Dave -- James Bourne | Email: jbourne@hardrock.org Unix Systems Administrator | WWW: http://www.hardrock.org Custom Unix Programming | Linux: The choice of a GNU generation ---------------------------------------------------------------------- "All you need's an occasional kick in the philosophy." Frank Herbert ^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2003-12-09 6:30 UTC | newest] Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2003-12-07 6:50 2.4.23 hard lock, 100% reproducible Alex Davis 2003-12-07 6:55 ` William Lee Irwin III [not found] ` <046a01c3bca1$267ba5e0$7a01a8c0@gandalf> 2003-12-07 9:07 ` William Lee Irwin III 2003-12-07 9:16 ` Keith Owens 2003-12-07 9:36 ` William Lee Irwin III 2003-12-08 16:34 ` Oliver Teuber 2003-12-07 10:01 ` Mark Symonds 2003-12-07 11:34 ` Martin Josefsson 2003-12-07 14:40 ` Marcelo Tosatti 2003-12-07 23:18 ` Mark Symonds 2003-12-08 4:53 ` William Lee Irwin III 2003-12-08 7:12 ` Tim Timmerman 2003-12-08 11:06 ` Marcelo Tosatti 2003-12-09 6:29 ` Tim Timmerman 2003-12-08 10:17 ` Marcelo Tosatti 2003-12-08 15:54 ` Stephan von Krawczynski 2003-12-08 16:15 ` Marcelo Tosatti 2003-12-08 16:31 ` Jeff Garzik 2003-12-08 16:55 ` Stephan von Krawczynski 2003-12-08 19:56 ` Mark Symonds 2003-12-07 7:33 ` Mark Symonds 2003-12-07 16:24 ` Alex Davis -- strict thread matches above, loose matches on Subject: below -- 2003-12-08 17:06 Xose Vazquez Perez 2003-12-07 2:36 Mark Symonds 2003-12-07 3:30 ` Philippe Troin 2003-12-07 4:34 ` Mark Symonds 2003-12-07 18:32 ` Chris Frey 2003-12-07 18:49 ` James Bourne 2003-12-07 19:21 ` David Rees 2003-12-07 20:19 ` James Bourne
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).