2.4.23 hard lock, 100% reproducible.

2.4.23 hard lock, 100% reproducible.

[Alex Davis]

I had the exact same thing happen to me about 
a year ago, with the same error message. It
started after I had upraded my kernel. It 
turned out one of my RAM sticks had gone
bad. Do you have another machine you can
test 2.4.23 with?


[...]
> 
> Not even sysrq?
>  

I did get msgs twice here in the past few hours, 
but only sometimes does it give anything.  Here's 
what it said: 

Unable to handle kernel NULL pointer
dereference at virtual address: 00000000

 printing eip:
c02363dd
*pde=00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c02363d>]  Not tainted
EFLAGS: 00010217

eax: 00000006   ebx: 00000000  ecx: 7a01a8c0   ecx: c700b2a0
esi: c0299ce0   edi: 000001b7  ebp: c0299d94   esp: c0299c54
ds: 0018  es: 0018  ss: 0018

process: swapper (pid: 0, stackpage = c0299000)

Other than that, nothing.  Is there a patch out there 
that will simply make 2.4.22 secure?  Things run great
on that kernel. 

-- 
Mark

-

=====
I code, therefore I am

__________________________________
Do you Yahoo!?
New Yahoo! Photos - easier uploading and sharing.
http://photos.yahoo.com/

Re: 2.4.23 hard lock, 100% reproducible.

[William Lee Irwin III]

On Sat, Dec 06, 2003 at 10:50:17PM -0800, Alex Davis wrote:
> Unable to handle kernel NULL pointer
> dereference at virtual address: 00000000
>  printing eip:
> c02363dd
> *pde=00000000
> Oops: 0000
> CPU: 0
> EIP: 0010:[<c02363d>]  Not tainted
> EFLAGS: 00010217
> eax: 00000006   ebx: 00000000  ecx: 7a01a8c0   ecx: c700b2a0
> esi: c0299ce0   edi: 000001b7  ebp: c0299d94   esp: c0299c54
> ds: 0018  es: 0018  ss: 0018
> process: swapper (pid: 0, stackpage = c0299000)
> Other than that, nothing.  Is there a patch out there 
> that will simply make 2.4.22 secure?  Things run great
> on that kernel. 

Compile your kernel with debug symbols and use addr2line on that EIP.


-- wli

Re: 2.4.23 hard lock, 100% reproducible.

[Mark Symonds]

> I had the exact same thing happen to me about 
> a year ago, with the same error message. It
> started after I had upraded my kernel. It 
> turned out one of my RAM sticks had gone
> bad. Do you have another machine you can
> test 2.4.23 with?
> 

I do, but not with identical hardware.  Thing is 
it ran just fine for months on previous kernels,
and even now will run just fine with them.  The
crashing only happens when using 2.4.23. 

-- 
Mark

Re: 2.4.23 hard lock, 100% reproducible.

[William Lee Irwin III]

At some point in the past, I wrote:
>> Compile your kernel with debug symbols and use addr2line on that EIP.

On Sun, Dec 07, 2003 at 01:04:41AM -0800, Mark Symonds wrote:
> I'm not a kernel developer, but here goes: 
> puggy:/usr/src/linux/2.4.23# addr2line c02363dd -e vmlinux
> ??:0
> puggy:/usr/src/linux/2.4.23#
> That doesn't look right.  I compiled with kernel debugging 
> enabled only; none of the other options.  Should I enable 
> others/all of them?  

For 2.4 you have to add -g to the compile options by hand in the kernel
makefile.


-- wli

Re: 2.4.23 hard lock, 100% reproducible.

[Keith Owens]

On Sun, Dec 07, 2003 at 01:04:41AM -0800, Mark Symonds wrote:
> I'm not a kernel developer, but here goes: 
> puggy:/usr/src/linux/2.4.23# addr2line c02363dd -e vmlinux
> ??:0

addr2line requires compiling with -g.  You can also do
  ksymoops -m /path/to/your/System.map -A c02363dd
which does not require a recompile.

Re: 2.4.23 hard lock, 100% reproducible.

[William Lee Irwin III]

On Sun, Dec 07, 2003 at 01:04:41AM -0800, Mark Symonds wrote:
>> I'm not a kernel developer, but here goes: 
>> puggy:/usr/src/linux/2.4.23# addr2line c02363dd -e vmlinux
>> ??:0

On Sun, Dec 07, 2003 at 08:16:54PM +1100, Keith Owens wrote:
> addr2line requires compiling with -g.  You can also do
>   ksymoops -m /path/to/your/System.map -A c02363dd
> which does not require a recompile.

That certainly would have been faster; I'll suggest that first next
time (though addr2line has the small advantage of handing back a line
number).


-- wli

Re: 2.4.23 hard lock, 100% reproducible.

[Mark Symonds]

[...]
> 
> addr2line requires compiling with -g.  You can also do
>   ksymoops -m /path/to/your/System.map -A c02363dd
> which does not require a recompile.
> 

Excellent, this is alot easier.  Should note that this 
kernel is compiled without support for loadable modules.
Here goes: 

------------- 

puggy:/usr/src/linux/2.4.23# ksymoops -m ./System.map -A c02363dd
ksymoops 2.4.9 on i686 2.4.23.  Options used
     -V (default)
     -k /proc/ksyms (default)
     -l /proc/modules (default)
     -o /lib/modules/2.4.23/ (default)
     -m ./System.map (specified)

Error (regular_file): read_ksyms stat /proc/ksyms failed
ksymoops: No such file or directory
No modules in ksyms, skipping objects
No ksyms, skipping lsmod


Adhoc c02363dd <tcp_print_conntrack+2d/60>


1 error issued.  Results may not be reliable.
puggy:/usr/src/linux/2.4.23#

-------------

-- 
Mark

Re: 2.4.23 hard lock, 100% reproducible.

[Martin Josefsson]

[-- Attachment #1: Type: text/plain, Size: 941 bytes --]

On Sun, 2003-12-07 at 11:01, Mark Symonds wrote:

> puggy:/usr/src/linux/2.4.23# ksymoops -m ./System.map -A c02363dd
> ksymoops 2.4.9 on i686 2.4.23.  Options used
>      -V (default)
>      -k /proc/ksyms (default)
>      -l /proc/modules (default)
>      -o /lib/modules/2.4.23/ (default)
>      -m ./System.map (specified)
> 
> Error (regular_file): read_ksyms stat /proc/ksyms failed
> ksymoops: No such file or directory
> No modules in ksyms, skipping objects
> No ksyms, skipping lsmod
> 
> 
> Adhoc c02363dd <tcp_print_conntrack+2d/60>

Hmm, it looks like it's this line in tcp_print_conntrack()

return sprintf(buffer, "%s ", tcp_conntrack_names[state]);

But it would be good if you could recompile and use addr2line (or gdb)
to get the exact line (offsets and memory-addresses depends on compiler
used and other stuff).

I took a quick look at the code but didn't see anything obvious.

-- 
/Martin

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: 2.4.23 hard lock, 100% reproducible.

[Marcelo Tosatti]

The first oops looks like:

Unable to handle kernel NULL pointer
dereference at virtual address: 00000000

printing eip:
c02363dd
*pde=00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c02363d>]  Not tainted
EFLAGS: 00010217

eax: 00000006   ebx: 00000000  ecx: 7a01a8c0   ecx: c700b2a0
esi: c0299ce0   edi: 000001b7  ebp: c0299d94   esp: c0299c54
ds: 0018  es: 0018  ss: 0018

process: swapper (pid: 0, stackpage = c0299000)


Isnt it a bit weird that the full backtrace is not reported ? 

wli suggests that might stack corruption.


I dont see any suspicious change around tcp_print_conntrack().

Any clues? 



On Sun, 7 Dec 2003, Mark Symonds wrote:

> 
> [...]
> > 
> > addr2line requires compiling with -g.  You can also do
> >   ksymoops -m /path/to/your/System.map -A c02363dd
> > which does not require a recompile.
> > 
> 
> Excellent, this is alot easier.  Should note that this 
> kernel is compiled without support for loadable modules.
> Here goes: 
> 
> ------------- 
> 
> puggy:/usr/src/linux/2.4.23# ksymoops -m ./System.map -A c02363dd
> ksymoops 2.4.9 on i686 2.4.23.  Options used
>      -V (default)
>      -k /proc/ksyms (default)
>      -l /proc/modules (default)
>      -o /lib/modules/2.4.23/ (default)
>      -m ./System.map (specified)
> 
> Error (regular_file): read_ksyms stat /proc/ksyms failed
> ksymoops: No such file or directory
> No modules in ksyms, skipping objects
> No ksyms, skipping lsmod
> 
> 
> Adhoc c02363dd <tcp_print_conntrack+2d/60>
> 
> 1 error issued.  Results may not be reliable.
> puggy:/usr/src/linux/2.4.23#

Re: 2.4.23 hard lock, 100% reproducible.

[Alex Davis]

Would it be possible to swap the RAM from another
machine? 


--- Mark Symonds <mark@symonds.net> wrote:

> 

> > I had the exact same thing happen to me about 

> > a year ago, with the same error message. It

> > started after I had upraded my kernel. It 

> > turned out one of my RAM sticks had gone

> > bad. Do you have another machine you can

> > test 2.4.23 with?

> > 

> 

> I do, but not with identical hardware.  Thing is 

> it ran just fine for months on previous kernels,

> and even now will run just fine with them.  The

> crashing only happens when using 2.4.23. 

> 

> -- 

> Mark

> 


=====
I code, therefore I am

__________________________________
Do you Yahoo!?
New Yahoo! Photos - easier uploading and sharing.
http://photos.yahoo.com/

Re: 2.4.23 hard lock, 100% reproducible.

[Mark Symonds]

Hi,

> 
> The first oops looks like:
> 
> Unable to handle kernel NULL pointer
> dereference at virtual address: 00000000
> 
[...]
> 
> 
> Isnt it a bit weird that the full backtrace is not reported ? 
> 
> wli suggests that might stack corruption.
> 

My bad - wrote it down by hand originally since 
it was locked hard.  

> 
> I dont see any suspicious change around tcp_print_conntrack().
> 
> Any clues? 
> 

I'm using ipchains compatability in there, looks like 
this is a possible cause - getting a patch right now,
will test and let y'all know (and then switch to 
iptables, finally). 

-- 
Mark

Re: 2.4.23 hard lock, 100% reproducible.

[William Lee Irwin III]

On Sun, Dec 07, 2003 at 03:18:24PM -0800, Mark Symonds wrote:
> I'm using ipchains compatability in there, looks like 
> this is a possible cause - getting a patch right now,
> will test and let y'all know (and then switch to 
> iptables, finally). 

For the purposes of fixing the bogon, it might be helpful to stick to
whatever triggered the problem just long enough to extract more
information.


-- wli

Re: 2.4.23 hard lock, 100% reproducible.

[Tim Timmerman]

>>>>> "Mark" == Mark Symonds <mark@symonds.net> writes:

Mark> Hi,

>> 
>> The first oops looks like:
>> 
>> Unable to handle kernel NULL pointer
>> dereference at virtual address: 00000000
>> 
Mark> [...]

Mark> I'm using ipchains compatability in there, looks like 
Mark> this is a possible cause - getting a patch right now,
Mark> will test and let y'all know (and then switch to 
Mark> iptables, finally). 
      Let me just add a me-too here. 

      Haven't got the oops on my desk, here, but from what I could
      see, the error occurred in find_appropriate_src, somewhere in
      ipchains.  

      Further, possibly irrelevant datapoint: ABIT BP6, ne2k-pci and
      3Com590 network cards. When the oops occurs, everything locks,
      capslock and scrolllock are lit. 

      I can reproduce the error by letting a second system ping the
      first, on the internal network. Sometimes it doesn't even
      complete a full boot. 
      
      I'll try and capture more detail tonight. 

      TimT.

-- 
tim.timmerman@asml.nl                              040-2683613
timt@timt.org   Voodoo Programmer/Keeper of the Rubber Chicken
One time I went to a museum where all the work in the museum had been
done by children. They had all the paintings up on refrigerators. 



-- 
The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. ASML is neither liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt.

Re: 2.4.23 hard lock, 100% reproducible.

[Marcelo Tosatti]

On Sun, 7 Dec 2003, Mark Symonds wrote:

> 
> Hi,
> 
> > 
> > The first oops looks like:
> > 
> > Unable to handle kernel NULL pointer
> > dereference at virtual address: 00000000
> > 
> [...]
> > 
> > 
> > Isnt it a bit weird that the full backtrace is not reported ? 
> > 
> > wli suggests that might stack corruption.
> > 
> 
> My bad - wrote it down by hand originally since 
> it was locked hard.  
> 
> > 
> > I dont see any suspicious change around tcp_print_conntrack().
> > 
> > Any clues? 
> > 
> 
> I'm using ipchains compatability in there, looks like 
> this is a possible cause - getting a patch right now,
> will test and let y'all know (and then switch to 
> iptables, finally). 

Mark,

Please try the latest BK tree. There was a known bug in the netfilter code 
which could cause the lockups. 

 

Re: 2.4.23 hard lock, 100% reproducible.

[Marcelo Tosatti]

On Mon, 8 Dec 2003, Tim Timmerman wrote:

> >>>>> "Mark" == Mark Symonds <mark@symonds.net> writes:
> 
> Mark> Hi,
> 
> >> 
> >> The first oops looks like:
> >> 
> >> Unable to handle kernel NULL pointer
> >> dereference at virtual address: 00000000
> >> 
> Mark> [...]
> 
> Mark> I'm using ipchains compatability in there, looks like 
> Mark> this is a possible cause - getting a patch right now,
> Mark> will test and let y'all know (and then switch to 
> Mark> iptables, finally). 
>       Let me just add a me-too here. 
> 
>       Haven't got the oops on my desk, here, but from what I could
>       see, the error occurred in find_appropriate_src, somewhere in
>       ipchains.  
> 
>       Further, possibly irrelevant datapoint: ABIT BP6, ne2k-pci and
>       3Com590 network cards. When the oops occurs, everything locks,
>       capslock and scrolllock are lit. 
> 
>       I can reproduce the error by letting a second system ping the
>       first, on the internal network. Sometimes it doesn't even
>       complete a full boot. 
>       
>       I'll try and capture more detail tonight. 

Tim,

Please try the updated 2.4 BK tree (you can use -bk5, 
http://www.kernel.org/pub/linux/kernel/v2.4/snapshots/patch-2.4.23-bk5.bz2).

It contains a fix for a known bug in the netfilter which might what you're 
hitting.

Re: 2.4.23 hard lock, 100% reproducible.

[Stephan von Krawczynski]

On Mon, 8 Dec 2003 08:17:30 -0200 (BRST)
Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:

> Mark,
> 
> Please try the latest BK tree. There was a known bug in the netfilter code 
> which could cause the lockups. 

...which leads me to the most-simple question: where can I find a changelog for
2.4.23-bkX from www.kernel.org ? Inside ?

Regards,
Stephan

Re: 2.4.23 hard lock, 100% reproducible.

[Marcelo Tosatti]

On Mon, 8 Dec 2003, Stephan von Krawczynski wrote:

> On Mon, 8 Dec 2003 08:17:30 -0200 (BRST)
> Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:
> 
> > Mark,
> > 
> > Please try the latest BK tree. There was a known bug in the netfilter code 
> > which could cause the lockups. 
> 
> ...which leads me to the most-simple question: where can I find a changelog for
> 2.4.23-bkX from www.kernel.org ? Inside ?

I believe there is nothing which generates 2.4.2x-bk changelogs right now.

It can be easily done.

Re: 2.4.23 hard lock, 100% reproducible.

[Jeff Garzik]

On Mon, Dec 08, 2003 at 02:15:27PM -0200, Marcelo Tosatti wrote:
> 
> 
> On Mon, 8 Dec 2003, Stephan von Krawczynski wrote:
> 
> > On Mon, 8 Dec 2003 08:17:30 -0200 (BRST)
> > Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:
> > 
> > > Mark,
> > > 
> > > Please try the latest BK tree. There was a known bug in the netfilter code 
> > > which could cause the lockups. 
> > 
> > ...which leads me to the most-simple question: where can I find a changelog for
> > 2.4.23-bkX from www.kernel.org ? Inside ?
> 
> I believe there is nothing which generates 2.4.2x-bk changelogs right now.

It has been automatically generated for a while now ;-)

...
http://www.kernel.org/pub/linux/kernel/v2.4/snapshots/patch-2.4.23-bk5.log
http://www.kernel.org/pub/linux/kernel/v2.4/snapshots/patch-2.4.23-bk6.log

Re: 2.4.23 hard lock, 100% reproducible.

[Oliver Teuber]

[-- Attachment #1: Type: text/plain, Size: 152 bytes --]

hi

i have got some oops too ... reported some days earlier.

yours, oliver teuber

http://marc.theaimsgroup.com/?l=linux-kernel&m=107036079102352&w=2


[-- Attachment #2: oops-2423-20031202.txt --]
[-- Type: text/plain, Size: 3303 bytes --]

ksymoops 2.4.9 on i686 2.4.23.  Options used
     -v /usr/src/linux/vmlinux (specified)
     -k /proc/ksyms (default)
     -l /proc/modules (default)
     -o /lib/modules/2.4.23/ (default)
     -m /usr/src/linux/System.map (default)

Reading Oops report from the terminal
Oops: 0000
CPU:    0
EIP:    0010:[<c0119780>]    Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010086
eax: c83a643c   ebx: 00000000   ecx: 00000001   edx: 00000001
esi: ce6d2980   edi: c83a643c   ebp: cdb61a6c   esp: cdb61a54
ds: 0018   es: 0018   ss: 0018
Process lpd (pid: 4136, stackpage=cdb61000)
Stack: 00000001 00000286 00000001 c41c1680 ce6d2980 00000000 00000046 c02282d4
       cfca1400 00000000 00000202 c41c1680 c022789b c41c1680 c8c9b180 c02288d1
       ce6d2980 cfca1560 fffffffd c022c7cb ce6d2980 cdb61af0 00000001 c033aa88
Call Trace:    [<c02282d4>] [<c022789b>] [<c02288d1>] [<c022c7cb>] [<c0120bb1>]
  [<c010aa19>] [<c010cf18>] [<d094c782>] [<d094cbe4>] [<d094c3c0>] [<d094d048>]
  [<d094ead2>] [<d094f0f2>] [<d095e82f>] [<d093d719>] [<d095e83d>] [<d0955057>]
  [<d093ebd0>] [<d095e83d>] [<c0150356>] [<c013e224>] [<c013cd7d>] [<c013ce0b>]
  [<c0108f27>]
Code: 8b 13 0f 18 02 39 c3 74 76 8d b4 26 00 00 00 00 8b 4b fc 8b


>>EIP; c0119780 <__wake_up+20/b0>   <=====

>>eax; c83a643c <_end+80372e8/1057bf0c>
>>esi; ce6d2980 <_end+e36382c/1057bf0c>
>>edi; c83a643c <_end+80372e8/1057bf0c>
>>ebp; cdb61a6c <_end+d7f2918/1057bf0c>
>>esp; cdb61a54 <_end+d7f2900/1057bf0c>

Trace; c02282d4 <sock_def_write_space+64/90>
Trace; c022789b <sock_wfree+3b/40>
Trace; c02288d1 <__kfree_skb+41/100>
Trace; c022c7cb <net_tx_action+2b/b0>
Trace; c0120bb1 <do_softirq+51/a0>
Trace; c010aa19 <do_IRQ+99/b0>
Trace; c010cf18 <call_do_IRQ+5/d>
Trace; d094c782 <[reiserfs]comp_keys+362/3f0>
Trace; d094cbe4 <[reiserfs]is_tree_node+64/70>
Trace; d094c3c0 <[reiserfs]__constant_memcpy+c0/120>
Trace; d094d048 <[reiserfs]search_for_position_by_key+f8/4c0>
Trace; d094ead2 <[reiserfs]reiserfs_cut_from_item+222/4b0>
Trace; d094f0f2 <[reiserfs]reiserfs_do_truncate+322/580>
Trace; d095e82f <[reiserfs].rodata.end+5ab0/5ca1>
Trace; d093d719 <[reiserfs]reiserfs_truncate_file+e9/230>
Trace; d095e83d <[reiserfs].rodata.end+5abe/5ca1>
Trace; d0955057 <[reiserfs]journal_end+27/30>
Trace; d093ebd0 <[reiserfs]reiserfs_file_release+3a0/450>
Trace; d095e83d <[reiserfs].rodata.end+5abe/5ca1>
Trace; c0150356 <locks_remove_flock+76/80>
Trace; c013e224 <fput+114/120>
Trace; c013cd7d <filp_close+4d/90>
Trace; c013ce0b <sys_close+4b/60>
Trace; c0108f27 <system_call+33/38>

Code;  c0119780 <__wake_up+20/b0>
00000000 <_EIP>:
Code;  c0119780 <__wake_up+20/b0>   <=====
   0:   8b 13                     mov    (%ebx),%edx   <=====
Code;  c0119782 <__wake_up+22/b0>
   2:   0f 18 02                  prefetchnta (%edx)
Code;  c0119785 <__wake_up+25/b0>
   5:   39 c3                     cmp    %eax,%ebx
Code;  c0119787 <__wake_up+27/b0>
   7:   74 76                     je     7f <_EIP+0x7f>
Code;  c0119789 <__wake_up+29/b0>
   9:   8d b4 26 00 00 00 00      lea    0x0(%esi,1),%esi
Code;  c0119790 <__wake_up+30/b0>
  10:   8b 4b fc                  mov    0xfffffffc(%ebx),%ecx
Code;  c0119793 <__wake_up+33/b0>
  13:   8b 00                     mov    (%eax),%eax

 <0>Kernel panic: Aiee, killing interrupt handler!

[-- Attachment #3: oops-2423-20031203.txt --]
[-- Type: text/plain, Size: 2257 bytes --]

ksymoops 2.4.9 on i686 2.4.23.  Options used
     -v /usr/src/linux/vmlinux (specified)
     -k /proc/ksyms (default)
     -l /proc/modules (default)
     -o /lib/modules/2.4.23/ (default)
     -m /usr/src/linux/System.map (default)

Reading Oops report from the terminal
Oops: 0000
CPU:    0
EIP:    0010:[<c0119780>]     Not tainted
Using defaults from ksymoops -t elf32-i386 -a i386
EFLAGS: 00010086
eax: c86ef23c   ebx: 00000000  ecx: 00000001  edx: 00000001
esi: ce074e80   edi: c86ef23c  ebp: cee19f40  esp: cee19f28
ds: 0018   es: 0018   ss: 0018
Process if (pid: 4571, stackpage=cee19000)
Stack: 00000001 00000286 00000001 cd876c80 ce074e80 00000000 00000046 c02282d4
       00001000 00000000 00000202 cd876c80 c022789b cd876c80 cb01ca80 c02288d1
       ce074e80 cfcbcd60 fffffffd c022c7cb ce074e80 cee19fc4 00000001 c033aa88
Call Trace:    [<c02282d4>] [<c022789b>] [<c02288d1>] [<c022c7cb>] [<c0120bb1>]
  [<c010aa19>] [<c010cf18>]
Code: 8b 13 0f 18 02 39 c3 74 76 8d b4 26 00 00 00 00 8b 4b fc 8b


>>EIP; c0119780 <__wake_up+20/b0>   <=====

>>eax; c86ef23c <_end+83800e8/1057bf0c>
>>esi; ce074e80 <_end+dd05d2c/1057bf0c>
>>edi; c86ef23c <_end+83800e8/1057bf0c>
>>ebp; cee19f40 <_end+eaaadec/1057bf0c>
>>esp; cee19f28 <_end+eaaadd4/1057bf0c>

Trace; c02282d4 <sock_def_write_space+64/90>
Trace; c022789b <sock_wfree+3b/40>
Trace; c02288d1 <__kfree_skb+41/100>
Trace; c022c7cb <net_tx_action+2b/b0>
Trace; c0120bb1 <do_softirq+51/a0>
Trace; c010aa19 <do_IRQ+99/b0>
Trace; c010cf18 <call_do_IRQ+5/d>

Code;  c0119780 <__wake_up+20/b0>
00000000 <_EIP>:
Code;  c0119780 <__wake_up+20/b0>   <=====
   0:   8b 13                     mov    (%ebx),%edx   <=====
Code;  c0119782 <__wake_up+22/b0>
   2:   0f 18 02                  prefetchnta (%edx)
Code;  c0119785 <__wake_up+25/b0>
   5:   39 c3                     cmp    %eax,%ebx
Code;  c0119787 <__wake_up+27/b0>
   7:   74 76                     je     7f <_EIP+0x7f>
Code;  c0119789 <__wake_up+29/b0>
   9:   8d b4 26 00 00 00 00      lea    0x0(%esi,1),%esi
Code;  c0119790 <__wake_up+30/b0>
  10:   8b 4b fc                  mov    0xfffffffc(%ebx),%ecx
Code;  c0119793 <__wake_up+33/b0>
  13:   8b 00                     mov    (%eax),%eax

 <0>Kernel panic: Aiee, killing interrupt handler!

Re: 2.4.23 hard lock, 100% reproducible.

[Stephan von Krawczynski]

On Mon, 8 Dec 2003 14:15:27 -0200 (BRST)
Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:

> 
> 
> On Mon, 8 Dec 2003, Stephan von Krawczynski wrote:
> 
> > On Mon, 8 Dec 2003 08:17:30 -0200 (BRST)
> > Marcelo Tosatti <marcelo.tosatti@cyclades.com> wrote:
> > 
> > > Mark,
> > > 
> > > Please try the latest BK tree. There was a known bug in the netfilter
> > > code which could cause the lockups. 
> > 
> > ...which leads me to the most-simple question: where can I find a changelog
> > for 2.4.23-bkX from www.kernel.org ? Inside ?
> 
> I believe there is nothing which generates 2.4.2x-bk changelogs right now.
> 
> It can be easily done.

Are you continuing to create pre-patches with readable changelogs on
kernel.org?

ONTOPIC: indeed it took me only few days to crash a 2.4.23 with heavy
net(filter) usage... Mine broke with "killing interrupt handler ..."


Regards,
Stephan

Re: 2.4.23 hard lock, 100% reproducible.

[Mark Symonds]

> > 
> > I'm using ipchains compatability in there, looks like 
> > this is a possible cause - getting a patch right now,
> > will test and let y'all know (and then switch to 
> > iptables, finally). 
> 
> Mark,
> 
> Please try the latest BK tree. There was a known bug in the netfilter code 
> which could cause the lockups. 
> 

What I did a couple of days ago was remove ipchains and switch 
to iptables (instead of applying any patches or anything).  Ever 
since the ipchains code was removed from my kernel, the box has 
been running fine.

Thanks much! 

-- 
Mark
 

Re: 2.4.23 hard lock, 100% reproducible.

[Tim Timmerman]

>>>>> "Marcelo" == Marcelo Tosatti <marcelo.tosatti@cyclades.com> writes:

Marcelo> On Mon, 8 Dec 2003, Tim Timmerman wrote:
>> Let me just add a me-too here. 
>> 
>> Haven't got the oops on my desk, here, but from what I could
>> see, the error occurred in find_appropriate_src, somewhere in
>> ipchains.  
>> 
>> Further, possibly irrelevant datapoint: ABIT BP6, ne2k-pci and
>> 3Com590 network cards. When the oops occurs, everything locks,
>> capslock and scrolllock are lit. 
>> 
>> I can reproduce the error by letting a second system ping the
>> first, on the internal network. Sometimes it doesn't even
>> complete a full boot. 
>> 
>> I'll try and capture more detail tonight. 

Marcelo> Tim,

Marcelo> Please try the updated 2.4 BK tree (you can use -bk5, 
Marcelo> http://www.kernel.org/pub/linux/kernel/v2.4/snapshots/patch-2.4.23-bk5.bz2).

Marcelo> It contains a fix for a known bug in the netfilter which
Marcelo> might what you're hitting.

   Marcello,

	Thanks ! I can confirm that this seems to fix the bug: system
	has been running the patched kernel for the past 12 hours, and
	is stable, even under load.

	TimT

-- 
tim.timmerman@asml.nl                              040-2683613
timt@timt.org   Voodoo Programmer/Keeper of the Rubber Chicken
Do Lipton employees take coffee breaks?



-- 
The information contained in this communication and any attachments is confidential and may be privileged, and is for the sole use of the intended recipient(s). Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please notify the sender immediately by replying to this message and destroy all copies of this message and any attachments. ASML is neither liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt.

Re: 2.4.23 hard lock, 100% reproducible.

[Xose Vazquez Perez]

Stephan von Krawczynski wrote:

> ...which leads me to the most-simple question: where can I find a changelog for
> 2.4.23-bkX from www.kernel.org ? Inside ?

BK used to mirror it from http://linux.bkbits.net:8080/linux-2.4/ to
http://www.kernel.org/pub/linux/kernel/v2.4/testing/cset/
But since last week it doesn't work.

Re: 2.4.23 hard lock, 100% reproducible.

[James Bourne]

On Sun, 7 Dec 2003, David Rees wrote:

> James Bourne wrote:
> > This is included with the patch set I just posted, 2.4.22-uv3 available at 
> > http://www.hardrock.org/current-updates/linux-2.4.22-uv3-updates.patch.
> 
> The correct link appears to be
> http://www.hardrock.org/kernel/current-updates/linux-2.4.22-uv3-updates.patch

Sorry, yes this is correct. 

Regards
James

> -Dave

-- 
James Bourne                  | Email:            jbourne@hardrock.org          
Unix Systems Administrator    | WWW:           http://www.hardrock.org
Custom Unix Programming       | Linux:  The choice of a GNU generation
----------------------------------------------------------------------
 "All you need's an occasional kick in the philosophy." Frank Herbert  

Re: 2.4.23 hard lock, 100% reproducible.

[David Rees]

James Bourne wrote:
> This is included with the patch set I just posted, 2.4.22-uv3 available at 
> http://www.hardrock.org/current-updates/linux-2.4.22-uv3-updates.patch.

The correct link appears to be
http://www.hardrock.org/kernel/current-updates/linux-2.4.22-uv3-updates.patch

-Dave

Re: 2.4.23 hard lock, 100% reproducible.

[James Bourne]

On Sun, 7 Dec 2003, Chris Frey wrote:

> On Sat, Dec 06, 2003 at 08:34:32PM -0800, Mark Symonds wrote:
> > Other than that, nothing.  Is there a patch out there 
> > that will simply make 2.4.22 secure?  Things run great
> > on that kernel. 
> 
> Here's the relevant section from patch-2.4.23

Hi,
This is included with the patch set I just posted, 2.4.22-uv3 available at
http://www.hardrock.org/current-updates/linux-2.4.22-uv3-updates.patch.

This also includes patches for some other issues in 2.4.22.  I haven't yet
posted one for 2.4.23 due to some outstanding issues with what appear to be
(or at least could be) hard locks (such as this thread).  Once these are
resolved I will be releasing a -uv for 2.4.23.

Regards
James

> 
> - Chris

-- 
James Bourne                  | Email:            jbourne@hardrock.org          
Unix Systems Administrator    | WWW:           http://www.hardrock.org
Custom Unix Programming       | Linux:  The choice of a GNU generation
----------------------------------------------------------------------
 "All you need's an occasional kick in the philosophy." Frank Herbert  

Re: 2.4.23 hard lock, 100% reproducible.

[Chris Frey]

On Sat, Dec 06, 2003 at 08:34:32PM -0800, Mark Symonds wrote:
> Other than that, nothing.  Is there a patch out there 
> that will simply make 2.4.22 secure?  Things run great
> on that kernel. 

Here's the relevant section from patch-2.4.23

- Chris


diff -urN linux-2.4.22/mm/mmap.c linux-2.4.23/mm/mmap.c
--- linux-2.4.22/mm/mmap.c      2003-06-13 07:51:39.000000000 -0700
+++ linux-2.4.23/mm/mmap.c      2003-11-28 10:26:21.000000000 -0800
@@ -1041,6 +1041,9 @@
        if (!len)
                return addr;

+       if ((addr + len) > TASK_SIZE || (addr + len) < addr)
+               return -EINVAL;
+
        /*
         * mlock MCL_FUTURE?
         */

Re: 2.4.23 hard lock, 100% reproducible.

[Mark Symonds]

[...]
> 
> Not even sysrq?
>  

I did get msgs twice here in the past few hours, 
but only sometimes does it give anything.  Here's 
what it said: 

Unable to handle kernel NULL pointer
dereference at virtual address: 00000000

 printing eip:
c02363dd
*pde=00000000
Oops: 0000
CPU: 0
EIP: 0010:[<c02363d>]  Not tainted
EFLAGS: 00010217

eax: 00000006   ebx: 00000000  ecx: 7a01a8c0   ecx: c700b2a0
esi: c0299ce0   edi: 000001b7  ebp: c0299d94   esp: c0299c54
ds: 0018  es: 0018  ss: 0018

process: swapper (pid: 0, stackpage = c0299000)

Other than that, nothing.  Is there a patch out there 
that will simply make 2.4.22 secure?  Things run great
on that kernel. 

-- 
Mark

Re: 2.4.23 hard lock, 100% reproducible.

[Philippe Troin]

Mark Symonds <mark@symonds.net> writes:

> Hi, 
> 
> I've got a machine here that is locking hard under
> 2.4.23.  Normally would suspect it's a hardware problem
> but it runs fine on 2.4.22 and also 2.2 series kernels.  
> In a bit of a quandry here since that box has shell
> users... 
> 
> I'm getting no oopses on the monitor nor in the logs -
> this is a hard, instantaneous crash.  No kbd, no nothing,
> good night. 

Not even sysrq?
 
> I've got a kernel compiling right now with hacking
> support, but none of the additional hacking  options
> are enabled. 
> 
> Wondering if anyone else has seen this?  lspci output is
> below, will wait until requested before dumping a bunch
> of crap about my hardware onto the list. 

Same for me.

2.4.23 locks up hard. 2.4.22 worked perfectly (save the random usb
oops).

I was not able to capture a register trace or a process dump with
sysrq because this box has a usb keyboard and usb seems to get
shot. I've plugged a ps/2 keyboard to get more details the next
lock-up. If this fail, I might try the NMI oopser.

lspci:

00:00.0 Host bridge: VIA Technologies, Inc. VT82C693A/694x [Apollo PRO133x] (rev c4)
00:01.0 PCI bridge: VIA Technologies, Inc. VT82C598/694x [Apollo MVP3/Pro133x AGP]
00:07.0 ISA bridge: VIA Technologies, Inc. VT82C596 ISA [Mobile South] (rev 23)
00:07.1 IDE interface: VIA Technologies, Inc. Bus Master IDE (rev 10)
00:07.2 USB Controller: VIA Technologies, Inc. UHCI USB (rev 11)
00:07.3 Host bridge: VIA Technologies, Inc. VT82C596 Power Management (rev 30)
00:0f.0 Multimedia audio controller: Creative Labs SB Live! EMU10k1 (rev 08)
00:0f.1 Input device controller: Creative Labs SB Live! (rev 08)
00:10.0 Ethernet controller: 3Com Corporation 3c905 100BaseTX [Boomerang]
00:12.0 SCSI storage controller: Adaptec 7892A (rev 02)
01:00.0 VGA compatible controller: Matrox Graphics, Inc. MGA G400 AGP (rev 04)

Phil.

2.4.23 hard lock, 100% reproducible.

[Mark Symonds]

Hi, 

I've got a machine here that is locking hard under
2.4.23.  Normally would suspect it's a hardware problem
but it runs fine on 2.4.22 and also 2.2 series kernels.  
In a bit of a quandry here since that box has shell
users... 

I'm getting no oopses on the monitor nor in the logs -
this is a hard, instantaneous crash.  No kbd, no nothing,
good night. 

I've got a kernel compiling right now with hacking
support, but none of the additional hacking  options
are enabled. 

Wondering if anyone else has seen this?  lspci output is
below, will wait until requested before dumping a bunch
of crap about my hardware onto the list. 

Anything I can do from userland, let me know. 

-- 
Mark 



----------------------- 
symonds:~# lspci
00:00.0 Host bridge: Intel Corp. 82810 GMCH [Graphics
Memory Controller Hub] (rev 03)
00:01.0 VGA compatible controller: Intel Corp. 82810 CGC
[Chipset Graphics Controller] (rev 03)
00:1e.0 PCI bridge: Intel Corp. 82801AA PCI Bridge (rev
02)
00:1f.0 ISA bridge: Intel Corp. 82801AA ISA Bridge (LPC)
(rev 02)
00:1f.1 IDE interface: Intel Corp. 82801AA IDE (rev 02)
00:1f.2 USB Controller: Intel Corp. 82801AA USB (rev 02)
00:1f.3 SMBus: Intel Corp. 82801AA SMBus (rev 02)
01:09.0 Multimedia audio controller: Cirrus Logic
Crystal CS4281 PCI Audio (rev 01)
01:0b.0 Ethernet controller: D-Link System Inc RTL8139
Ethernet (rev 10)
01:0d.0 Ethernet controller: VIA Technologies, Inc.
VT86C100A [Rhine 10/100] (rev 06)
01:0e.0 Ethernet controller: Lite-On Communications Inc
LNE100TX (rev 20)
symonds:~#