* Re: [Oops] 2.6.10: PREEMPT SMP
@ 2005-02-18 11:29 Klaus Steinberger
0 siblings, 0 replies; 6+ messages in thread
From: Klaus Steinberger @ 2005-02-18 11:29 UTC (permalink / raw)
To: linux-kernel
David Howells wrote:
> Hmmm... I see it involves the key stuff I wrote.
Did you find out something about this bug? I did get the same crash on a
heavily loaded NFS and Samba Server running Fedora Core 2 with
kernel-2.6.10-1.12_FC2.
Here the OOPS:
Feb 18 09:58:08 mllrd02 kernel: Unable to handle kernel NULL pointer
dereference at virtual address 0000000c
Feb 18 09:58:08 mllrd02 kernel: printing eip:
Feb 18 09:58:08 mllrd02 kernel: c01b48ec
Feb 18 09:58:08 mllrd02 kernel: *pde = 111b3001
Feb 18 09:58:08 mllrd02 kernel: Oops: 0000 [#1]
Feb 18 09:58:08 mllrd02 kernel: SMP
Feb 18 09:58:08 mllrd02 kernel: Modules linked in: nfsd exportfs nfs lockd
parport_pc lp parport autofs4 sunrpc iptable_filter ip_tables tg3 floppy sg
microcode ohci_hcd video button battery ac md5 ipv6 ext3 jbd dm_mod qla2300
qla2xxx scsi_transport_fc aacraid(U) aic79xx sd_mod scsi_mod
Feb 18 09:58:08 mllrd02 kernel: CPU: 3
Feb 18 09:58:08 mllrd02 kernel: EIP: 0060:[<c01b48ec>] Not tainted VLI
Feb 18 09:58:08 mllrd02 kernel: EFLAGS: 00010203 (2.6.10-1.12_FC2smp)
Feb 18 09:58:08 mllrd02 kernel: EIP is at __rb_rotate_left+0x8/0x36
Feb 18 09:58:08 mllrd02 kernel: eax: de05f940 ebx: c04265e4 ecx: de05f940
edx: 00000000
Feb 18 09:58:08 mllrd02 kernel: esi: de05f940 edi: f652db80 ebp: c04265e4
esp: eef86ed0
Feb 18 09:58:08 mllrd02 kernel: ds: 007b es: 007b ss: 0068
Feb 18 09:58:08 mllrd02 kernel: Process smbd (pid: 3204, threadinfo=eef86000
task=f72e7020)
Feb 18 09:58:08 mllrd02 kernel: Stack: c5887700 c01b49fd f652db80 f652db80
c5887708 000000b1 c0197650 c5887700
Feb 18 09:58:08 mllrd02 kernel: 0000000d eef86f54 eef86f61 ffffffea
c0197704 00000015 00000000 000000b1
Feb 18 09:58:08 mllrd02 kernel: c031f1e0 eef86f54 00000000 f5675a40
000000b1 c019887d ffffffff 001f0000
Feb 18 09:58:08 mllrd02 kernel: Call Trace:
Feb 18 09:58:08 mllrd02 kernel: [<c01b49fd>] rb_insert_color+0xad/0xcc
Feb 18 09:58:08 mllrd02 kernel: [<c0197650>] key_user_lookup+0xd4/0x101
Feb 18 09:58:08 mllrd02 kernel: [<c0197704>] key_alloc+0x53/0x2bf
Feb 18 09:58:08 mllrd02 kernel: [<c019887d>] keyring_alloc+0x1a/0x48
Feb 18 09:58:08 mllrd02 kernel: [<c0199dfb>] alloc_uid_keyring+0x2b/0x7b
Feb 18 09:58:08 mllrd02 kernel: [<c0125fa2>] alloc_uid+0xb6/0x143
Feb 18 09:58:08 mllrd02 kernel: [<c0129489>] set_user+0xb/0x8c
Feb 18 09:58:08 mllrd02 kernel: [<c012988b>] sys_setresuid+0x105/0x1a4
Feb 18 09:58:08 mllrd02 kernel: [<c0103ccb>] syscall_call+0x7/0xb
Feb 18 09:58:08 mllrd02 kernel: Code: 59 83 bc 82 04 01 00 00 00 75 ea 41 83
f9 01 76 ed 31 c0 5b c3 57 b9 45 00 00 00 89 c7 31 c0 f3 ab 5f c3 53 89 c1 89
d3 8b 50 08 <8b> 42 0c 85 c0 89 41 08 74 02 89 08 89 4a 0c 8b 01 85 c0 89 02
Sincerly,
Klaus
--
Klaus Steinberger Maier-Leibnitz Labor
Phone: (+49 89)289 14287 Am Coulombwall 6, D-85748 Garching, Germany
FAX: (+49 89)289 14280 EMail: Klaus.Steinberger@Physik.Uni-Muenchen.DE
URL: http://www.physik.uni-muenchen.de/~k2/
In a world without Walls and Fences, who needs Windows and Gates
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Oops] 2.6.10: PREEMPT SMP
2005-03-21 23:35 ` Andrew Morton
@ 2005-03-24 0:42 ` Andrew Taylor
0 siblings, 0 replies; 6+ messages in thread
From: Andrew Taylor @ 2005-03-24 0:42 UTC (permalink / raw)
To: Andrew Morton; +Cc: Andrew Taylor
Hello,
On Mon, Mar 21, 2005 at 03:35:37PM -0800, Andrew Morton wrote:
> Andrew Taylor <taylor@array.ca> wrote:
> >
> > Hello,
> >
> > We have been experiencing a very similar oops on a Dell poweredge 2600, running Fedora Core 3 (FC3). This host is mainly used as a samba fileserver. The problem has been seen on 2 different machines (both Dell-pe-2600s).
>
> Could you pleae confirm that 2.6.12-rc1 fixes this?
Would be glad to however, between 2.6.11-5 (which appears to runs fine) and 2.6.12-rc1 the mptfusion driver got broken. Attempts to isolate the mptfusion changes have not been successful. Will look at this issue again on tuesday.
Cheers,
Andrew
>
> Thanks.
>
> > The problem is only ever seen when running the SMP kernel in conjunction with samba.
> >
> > We have been experiencing problems since early January when the samba/nfs/io load was increased considerably. Since then a number of 2.4 kernels have been used:
> > 2.4.22.2115nptlsmp (FC1)
> > 2.4.22-1.2199smp (FC1)
> > 2.4.29 (vanilla kernel)
> > 2.6.10-1.741_FC3smp (FC3)
> > 2.6.10-1.766_FC3smp (FC3)
> >
> > All 2.4 kernels produced the same result: full system lock-up, with no messages on the console or the log files.
> >
> > The 2.6.10 kernel is giving an oops which has been seen 3 time in the last 3 days. A very similar one was seen pointing to eip: c01b2d98 on the 2.6.10-1.741 kernel.
> >
> > I have included the kallsyms and System.map relevant to the instruction pointed to by the EIP. I can include the full files if required.
> >
> > printing eip:
> > c01b387c
> > *pde = 2fcdd001
> > Oops: 0000 [#1]
> > SMP
> > Modules linked in: nfsd exportfs md5 ipv6 parport_pc lp parport autofs4 nfs lockd sunrpc video button battery ac uhci_hcd hw_random e1000 floppy sg dm_snapshot dm_zero dm_mirror ext3 jbd dm_mod mptscsih mptbase sd_mod scsi_mod
> > CPU: 1
> > EIP: 0060:[<c01b387c>] Not tainted VLI
> > EFLAGS: 00010203 (2.6.10-1.766_FC3smp)
> > EIP is at __rb_rotate_left+0x8/0x36
> > eax: f57be640 ebx: c041e5e4 ecx: f57be640 edx: 00000000
> > esi: f57be640 edi: f5f39c80 ebp: c041e5e4 esp: efa2eed0
> > ds: 007b es: 007b ss: 0068
> > Process smbd (pid: 8536, threadinfo=efa2e000 task=f7612a60)
> > Stack: f5ac1980 c01b3988 f5f39c80 f5f39c80 f5ac1988 00000267 c01965d7 f5ac1980
> > 0000000d efa2ef54 efa2ef61 ffffffea c019668b 00000015 00000000 00000267
> > c03191e0 efa2ef54 00000000 f599ed80 00000267 c01977df ffffffff 001f0000
> > Call Trace:
> > [<c01b3988>] rb_insert_color+0xa8/0xc1
> > [<c01965d7>] key_user_lookup+0xcf/0xfc
> > [<c019668b>] key_alloc+0x53/0x2b6
> > [<c01977df>] keyring_alloc+0x1a/0x48
> > [<c0198d43>] alloc_uid_keyring+0x2b/0x7c
> > [<c0125d2e>] alloc_uid+0xae/0x133
> > [<c01291d4>] set_user+0xb/0x8b
> > [<c01295f2>] sys_setresuid+0x11a/0x1b9
> > [<c0103c97>] syscall_call+0x7/0xb
> > Code: 82 04 01 00 00 00 75 ea 41 83 f9 01 76 ed 31 c0 5b c3 57 b9 45 00 00 00 89 c7 31 c0 f3 ab 5f c3 90 90 90 53 89 d3 8b 50 08 89 c1 <8b> 42 0c 85 c0 89 41 08 74 02 89 08 89 4a 0c 8b 01 85 c0 89 02
> >
> >
> > /proc/kallsyms grepped on c01b3.
> >
> > c01b3024 t newary
> > c01b313c T sys_semget
> > c01b325a t try_atomic_semop
> > c01b336b t update_queue
> > c01b33de t count_semncnt
> > c01b342c t count_semzcnt
> > c01b347a t freeary
> > c01b34f6 t copy_semid_to_user
> > c01b354b t semctl_nolock
> > c01b3731 t semctl_main
> > c01b3ad6 t semctl_down
> > c01b3c39 T sys_semctl
> > c01b3ce9 t lookup_undo
> > c01b3d19 t find_undo
> >
> >
> > root@msh2-c boot]# grep -i c01b3 /boot/System.map-2.6.10-1.766_FC3smp
> > c01b304b T match_int
> > c01b3052 T match_octal
> > c01b305c T match_hex
> > c01b3066 T match_strcpy
> > c01b3098 T match_strdup
> > c01b30c0 t radix_tree_node_alloc
> > c01b3109 T radix_tree_preload
> > c01b317d t radix_tree_extend
> > c01b321c T radix_tree_insert
> > c01b3303 T radix_tree_lookup
> > c01b3349 T radix_tree_tag_set
> > c01b33b8 T radix_tree_tag_clear
> > c01b347a t __lookup
> > c01b3547 T radix_tree_gang_lookup
> > c01b3595 t __lookup_tag
> > c01b369d T radix_tree_gang_lookup_tag
> > c01b36f0 T radix_tree_delete
> > c01b3835 T radix_tree_tagged
> > c01b3863 t radix_tree_node_ctor
> > c01b3874 t __rb_rotate_left
> > c01b38aa t __rb_rotate_right
> > c01b38e0 T rb_insert_color
> > c01b39a1 t __rb_erase_color
> > c01b3b0c T rb_erase
> > c01b3bcd T rb_first
> > c01b3be5 T rb_last
> > c01b3bfd T rb_next
> > c01b3c26 T rb_prev
> > c01b3c4f T rb_replace_node
> > c01b3c94 T rwsem_wake
> > c01b3d89 T rwsem_downgrade_wake
> > c01b3e14 T strnicmp
> > c01b3e75 T strcpy
> > c01b3e89 T strncpy
> > c01b3ea8 T strlcpy
> > c01b3eeb T strcat
> > c01b3f0a T strncat
> > c01b3f35 T strlcat
> > c01b3f97 T strcmp
> > c01b3fae T strncmp
> > c01b3fe0 T strchr
> > c01b3ff0 T strrchr
> >
> > Cheers,
> >
> > Andrew Taylor
> >
> > Systems Engineer
> > Array Systems Computing Inc.
> > email: taylor@array.ca
> > http: www.array.ca
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at http://www.tux.org/lkml/
> Andrew Taylor <taylor@array.ca> wrote:
> >
> > Hello,
> >
> > We have been experiencing a very similar oops on a Dell poweredge 2600, running Fedora Core 3 (FC3). This host is mainly used as a samba fileserver. The problem has been seen on 2 different machines (both Dell-pe-2600s).
>
> Could you pleae confirm that 2.6.12-rc1 fixes this?
>
> Thanks.
>
> > The problem is only ever seen when running the SMP kernel in conjunction with samba.
> >
> > We have been experiencing problems since early January when the samba/nfs/io load was increased considerably. Since then a number of 2.4 kernels have been used:
> > 2.4.22.2115nptlsmp (FC1)
> > 2.4.22-1.2199smp (FC1)
> > 2.4.29 (vanilla kernel)
> > 2.6.10-1.741_FC3smp (FC3)
> > 2.6.10-1.766_FC3smp (FC3)
> >
> > All 2.4 kernels produced the same result: full system lock-up, with no messages on the console or the log files.
> >
> > The 2.6.10 kernel is giving an oops which has been seen 3 time in the last 3 days. A very similar one was seen pointing to eip: c01b2d98 on the 2.6.10-1.741 kernel.
> >
> > I have included the kallsyms and System.map relevant to the instruction pointed to by the EIP. I can include the full files if required.
> >
> > printing eip:
> > c01b387c
> > *pde = 2fcdd001
> > Oops: 0000 [#1]
> > SMP
> > Modules linked in: nfsd exportfs md5 ipv6 parport_pc lp parport autofs4 nfs lockd sunrpc video button battery ac uhci_hcd hw_random e1000 floppy sg dm_snapshot dm_zero dm_mirror ext3 jbd dm_mod mptscsih mptbase sd_mod scsi_mod
> > CPU: 1
> > EIP: 0060:[<c01b387c>] Not tainted VLI
> > EFLAGS: 00010203 (2.6.10-1.766_FC3smp)
> > EIP is at __rb_rotate_left+0x8/0x36
> > eax: f57be640 ebx: c041e5e4 ecx: f57be640 edx: 00000000
> > esi: f57be640 edi: f5f39c80 ebp: c041e5e4 esp: efa2eed0
> > ds: 007b es: 007b ss: 0068
> > Process smbd (pid: 8536, threadinfo=efa2e000 task=f7612a60)
> > Stack: f5ac1980 c01b3988 f5f39c80 f5f39c80 f5ac1988 00000267 c01965d7 f5ac1980
> > 0000000d efa2ef54 efa2ef61 ffffffea c019668b 00000015 00000000 00000267
> > c03191e0 efa2ef54 00000000 f599ed80 00000267 c01977df ffffffff 001f0000
> > Call Trace:
> > [<c01b3988>] rb_insert_color+0xa8/0xc1
> > [<c01965d7>] key_user_lookup+0xcf/0xfc
> > [<c019668b>] key_alloc+0x53/0x2b6
> > [<c01977df>] keyring_alloc+0x1a/0x48
> > [<c0198d43>] alloc_uid_keyring+0x2b/0x7c
> > [<c0125d2e>] alloc_uid+0xae/0x133
> > [<c01291d4>] set_user+0xb/0x8b
> > [<c01295f2>] sys_setresuid+0x11a/0x1b9
> > [<c0103c97>] syscall_call+0x7/0xb
> > Code: 82 04 01 00 00 00 75 ea 41 83 f9 01 76 ed 31 c0 5b c3 57 b9 45 00 00 00 89 c7 31 c0 f3 ab 5f c3 90 90 90 53 89 d3 8b 50 08 89 c1 <8b> 42 0c 85 c0 89 41 08 74 02 89 08 89 4a 0c 8b 01 85 c0 89 02
> >
> >
> > /proc/kallsyms grepped on c01b3.
> >
> > c01b3024 t newary
> > c01b313c T sys_semget
> > c01b325a t try_atomic_semop
> > c01b336b t update_queue
> > c01b33de t count_semncnt
> > c01b342c t count_semzcnt
> > c01b347a t freeary
> > c01b34f6 t copy_semid_to_user
> > c01b354b t semctl_nolock
> > c01b3731 t semctl_main
> > c01b3ad6 t semctl_down
> > c01b3c39 T sys_semctl
> > c01b3ce9 t lookup_undo
> > c01b3d19 t find_undo
> >
> >
> > root@msh2-c boot]# grep -i c01b3 /boot/System.map-2.6.10-1.766_FC3smp
> > c01b304b T match_int
> > c01b3052 T match_octal
> > c01b305c T match_hex
> > c01b3066 T match_strcpy
> > c01b3098 T match_strdup
> > c01b30c0 t radix_tree_node_alloc
> > c01b3109 T radix_tree_preload
> > c01b317d t radix_tree_extend
> > c01b321c T radix_tree_insert
> > c01b3303 T radix_tree_lookup
> > c01b3349 T radix_tree_tag_set
> > c01b33b8 T radix_tree_tag_clear
> > c01b347a t __lookup
> > c01b3547 T radix_tree_gang_lookup
> > c01b3595 t __lookup_tag
> > c01b369d T radix_tree_gang_lookup_tag
> > c01b36f0 T radix_tree_delete
> > c01b3835 T radix_tree_tagged
> > c01b3863 t radix_tree_node_ctor
> > c01b3874 t __rb_rotate_left
> > c01b38aa t __rb_rotate_right
> > c01b38e0 T rb_insert_color
> > c01b39a1 t __rb_erase_color
> > c01b3b0c T rb_erase
> > c01b3bcd T rb_first
> > c01b3be5 T rb_last
> > c01b3bfd T rb_next
> > c01b3c26 T rb_prev
> > c01b3c4f T rb_replace_node
> > c01b3c94 T rwsem_wake
> > c01b3d89 T rwsem_downgrade_wake
> > c01b3e14 T strnicmp
> > c01b3e75 T strcpy
> > c01b3e89 T strncpy
> > c01b3ea8 T strlcpy
> > c01b3eeb T strcat
> > c01b3f0a T strncat
> > c01b3f35 T strlcat
> > c01b3f97 T strcmp
> > c01b3fae T strncmp
> > c01b3fe0 T strchr
> > c01b3ff0 T strrchr
> >
> > Cheers,
> >
> > Andrew Taylor
> >
> > Systems Engineer
> > Array Systems Computing Inc.
> > email: taylor@array.ca
> > http: www.array.ca
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Oops] 2.6.10: PREEMPT SMP
2005-03-08 20:50 Andrew Taylor
@ 2005-03-21 23:35 ` Andrew Morton
2005-03-24 0:42 ` Andrew Taylor
0 siblings, 1 reply; 6+ messages in thread
From: Andrew Morton @ 2005-03-21 23:35 UTC (permalink / raw)
To: Andrew Taylor; +Cc: linux-kernel, taylor, biswa
Andrew Taylor <taylor@array.ca> wrote:
>
> Hello,
>
> We have been experiencing a very similar oops on a Dell poweredge 2600, running Fedora Core 3 (FC3). This host is mainly used as a samba fileserver. The problem has been seen on 2 different machines (both Dell-pe-2600s).
Could you pleae confirm that 2.6.12-rc1 fixes this?
Thanks.
> The problem is only ever seen when running the SMP kernel in conjunction with samba.
>
> We have been experiencing problems since early January when the samba/nfs/io load was increased considerably. Since then a number of 2.4 kernels have been used:
> 2.4.22.2115nptlsmp (FC1)
> 2.4.22-1.2199smp (FC1)
> 2.4.29 (vanilla kernel)
> 2.6.10-1.741_FC3smp (FC3)
> 2.6.10-1.766_FC3smp (FC3)
>
> All 2.4 kernels produced the same result: full system lock-up, with no messages on the console or the log files.
>
> The 2.6.10 kernel is giving an oops which has been seen 3 time in the last 3 days. A very similar one was seen pointing to eip: c01b2d98 on the 2.6.10-1.741 kernel.
>
> I have included the kallsyms and System.map relevant to the instruction pointed to by the EIP. I can include the full files if required.
>
> printing eip:
> c01b387c
> *pde = 2fcdd001
> Oops: 0000 [#1]
> SMP
> Modules linked in: nfsd exportfs md5 ipv6 parport_pc lp parport autofs4 nfs lockd sunrpc video button battery ac uhci_hcd hw_random e1000 floppy sg dm_snapshot dm_zero dm_mirror ext3 jbd dm_mod mptscsih mptbase sd_mod scsi_mod
> CPU: 1
> EIP: 0060:[<c01b387c>] Not tainted VLI
> EFLAGS: 00010203 (2.6.10-1.766_FC3smp)
> EIP is at __rb_rotate_left+0x8/0x36
> eax: f57be640 ebx: c041e5e4 ecx: f57be640 edx: 00000000
> esi: f57be640 edi: f5f39c80 ebp: c041e5e4 esp: efa2eed0
> ds: 007b es: 007b ss: 0068
> Process smbd (pid: 8536, threadinfo=efa2e000 task=f7612a60)
> Stack: f5ac1980 c01b3988 f5f39c80 f5f39c80 f5ac1988 00000267 c01965d7 f5ac1980
> 0000000d efa2ef54 efa2ef61 ffffffea c019668b 00000015 00000000 00000267
> c03191e0 efa2ef54 00000000 f599ed80 00000267 c01977df ffffffff 001f0000
> Call Trace:
> [<c01b3988>] rb_insert_color+0xa8/0xc1
> [<c01965d7>] key_user_lookup+0xcf/0xfc
> [<c019668b>] key_alloc+0x53/0x2b6
> [<c01977df>] keyring_alloc+0x1a/0x48
> [<c0198d43>] alloc_uid_keyring+0x2b/0x7c
> [<c0125d2e>] alloc_uid+0xae/0x133
> [<c01291d4>] set_user+0xb/0x8b
> [<c01295f2>] sys_setresuid+0x11a/0x1b9
> [<c0103c97>] syscall_call+0x7/0xb
> Code: 82 04 01 00 00 00 75 ea 41 83 f9 01 76 ed 31 c0 5b c3 57 b9 45 00 00 00 89 c7 31 c0 f3 ab 5f c3 90 90 90 53 89 d3 8b 50 08 89 c1 <8b> 42 0c 85 c0 89 41 08 74 02 89 08 89 4a 0c 8b 01 85 c0 89 02
>
>
> /proc/kallsyms grepped on c01b3.
>
> c01b3024 t newary
> c01b313c T sys_semget
> c01b325a t try_atomic_semop
> c01b336b t update_queue
> c01b33de t count_semncnt
> c01b342c t count_semzcnt
> c01b347a t freeary
> c01b34f6 t copy_semid_to_user
> c01b354b t semctl_nolock
> c01b3731 t semctl_main
> c01b3ad6 t semctl_down
> c01b3c39 T sys_semctl
> c01b3ce9 t lookup_undo
> c01b3d19 t find_undo
>
>
> root@msh2-c boot]# grep -i c01b3 /boot/System.map-2.6.10-1.766_FC3smp
> c01b304b T match_int
> c01b3052 T match_octal
> c01b305c T match_hex
> c01b3066 T match_strcpy
> c01b3098 T match_strdup
> c01b30c0 t radix_tree_node_alloc
> c01b3109 T radix_tree_preload
> c01b317d t radix_tree_extend
> c01b321c T radix_tree_insert
> c01b3303 T radix_tree_lookup
> c01b3349 T radix_tree_tag_set
> c01b33b8 T radix_tree_tag_clear
> c01b347a t __lookup
> c01b3547 T radix_tree_gang_lookup
> c01b3595 t __lookup_tag
> c01b369d T radix_tree_gang_lookup_tag
> c01b36f0 T radix_tree_delete
> c01b3835 T radix_tree_tagged
> c01b3863 t radix_tree_node_ctor
> c01b3874 t __rb_rotate_left
> c01b38aa t __rb_rotate_right
> c01b38e0 T rb_insert_color
> c01b39a1 t __rb_erase_color
> c01b3b0c T rb_erase
> c01b3bcd T rb_first
> c01b3be5 T rb_last
> c01b3bfd T rb_next
> c01b3c26 T rb_prev
> c01b3c4f T rb_replace_node
> c01b3c94 T rwsem_wake
> c01b3d89 T rwsem_downgrade_wake
> c01b3e14 T strnicmp
> c01b3e75 T strcpy
> c01b3e89 T strncpy
> c01b3ea8 T strlcpy
> c01b3eeb T strcat
> c01b3f0a T strncat
> c01b3f35 T strlcat
> c01b3f97 T strcmp
> c01b3fae T strncmp
> c01b3fe0 T strchr
> c01b3ff0 T strrchr
>
> Cheers,
>
> Andrew Taylor
>
> Systems Engineer
> Array Systems Computing Inc.
> email: taylor@array.ca
> http: www.array.ca
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Oops] 2.6.10: PREEMPT SMP
@ 2005-03-08 20:50 Andrew Taylor
2005-03-21 23:35 ` Andrew Morton
0 siblings, 1 reply; 6+ messages in thread
From: Andrew Taylor @ 2005-03-08 20:50 UTC (permalink / raw)
To: linux-kernel; +Cc: Andrew Taylor, Biswa Patnaik
Hello,
We have been experiencing a very similar oops on a Dell poweredge 2600, running Fedora Core 3 (FC3). This host is mainly used as a samba fileserver. The problem has been seen on 2 different machines (both Dell-pe-2600s).
The problem is only ever seen when running the SMP kernel in conjunction with samba.
We have been experiencing problems since early January when the samba/nfs/io load was increased considerably. Since then a number of 2.4 kernels have been used:
2.4.22.2115nptlsmp (FC1)
2.4.22-1.2199smp (FC1)
2.4.29 (vanilla kernel)
2.6.10-1.741_FC3smp (FC3)
2.6.10-1.766_FC3smp (FC3)
All 2.4 kernels produced the same result: full system lock-up, with no messages on the console or the log files.
The 2.6.10 kernel is giving an oops which has been seen 3 time in the last 3 days. A very similar one was seen pointing to eip: c01b2d98 on the 2.6.10-1.741 kernel.
I have included the kallsyms and System.map relevant to the instruction pointed to by the EIP. I can include the full files if required.
printing eip:
c01b387c
*pde = 2fcdd001
Oops: 0000 [#1]
SMP
Modules linked in: nfsd exportfs md5 ipv6 parport_pc lp parport autofs4 nfs lockd sunrpc video button battery ac uhci_hcd hw_random e1000 floppy sg dm_snapshot dm_zero dm_mirror ext3 jbd dm_mod mptscsih mptbase sd_mod scsi_mod
CPU: 1
EIP: 0060:[<c01b387c>] Not tainted VLI
EFLAGS: 00010203 (2.6.10-1.766_FC3smp)
EIP is at __rb_rotate_left+0x8/0x36
eax: f57be640 ebx: c041e5e4 ecx: f57be640 edx: 00000000
esi: f57be640 edi: f5f39c80 ebp: c041e5e4 esp: efa2eed0
ds: 007b es: 007b ss: 0068
Process smbd (pid: 8536, threadinfo=efa2e000 task=f7612a60)
Stack: f5ac1980 c01b3988 f5f39c80 f5f39c80 f5ac1988 00000267 c01965d7 f5ac1980
0000000d efa2ef54 efa2ef61 ffffffea c019668b 00000015 00000000 00000267
c03191e0 efa2ef54 00000000 f599ed80 00000267 c01977df ffffffff 001f0000
Call Trace:
[<c01b3988>] rb_insert_color+0xa8/0xc1
[<c01965d7>] key_user_lookup+0xcf/0xfc
[<c019668b>] key_alloc+0x53/0x2b6
[<c01977df>] keyring_alloc+0x1a/0x48
[<c0198d43>] alloc_uid_keyring+0x2b/0x7c
[<c0125d2e>] alloc_uid+0xae/0x133
[<c01291d4>] set_user+0xb/0x8b
[<c01295f2>] sys_setresuid+0x11a/0x1b9
[<c0103c97>] syscall_call+0x7/0xb
Code: 82 04 01 00 00 00 75 ea 41 83 f9 01 76 ed 31 c0 5b c3 57 b9 45 00 00 00 89 c7 31 c0 f3 ab 5f c3 90 90 90 53 89 d3 8b 50 08 89 c1 <8b> 42 0c 85 c0 89 41 08 74 02 89 08 89 4a 0c 8b 01 85 c0 89 02
/proc/kallsyms grepped on c01b3.
c01b3024 t newary
c01b313c T sys_semget
c01b325a t try_atomic_semop
c01b336b t update_queue
c01b33de t count_semncnt
c01b342c t count_semzcnt
c01b347a t freeary
c01b34f6 t copy_semid_to_user
c01b354b t semctl_nolock
c01b3731 t semctl_main
c01b3ad6 t semctl_down
c01b3c39 T sys_semctl
c01b3ce9 t lookup_undo
c01b3d19 t find_undo
root@msh2-c boot]# grep -i c01b3 /boot/System.map-2.6.10-1.766_FC3smp
c01b304b T match_int
c01b3052 T match_octal
c01b305c T match_hex
c01b3066 T match_strcpy
c01b3098 T match_strdup
c01b30c0 t radix_tree_node_alloc
c01b3109 T radix_tree_preload
c01b317d t radix_tree_extend
c01b321c T radix_tree_insert
c01b3303 T radix_tree_lookup
c01b3349 T radix_tree_tag_set
c01b33b8 T radix_tree_tag_clear
c01b347a t __lookup
c01b3547 T radix_tree_gang_lookup
c01b3595 t __lookup_tag
c01b369d T radix_tree_gang_lookup_tag
c01b36f0 T radix_tree_delete
c01b3835 T radix_tree_tagged
c01b3863 t radix_tree_node_ctor
c01b3874 t __rb_rotate_left
c01b38aa t __rb_rotate_right
c01b38e0 T rb_insert_color
c01b39a1 t __rb_erase_color
c01b3b0c T rb_erase
c01b3bcd T rb_first
c01b3be5 T rb_last
c01b3bfd T rb_next
c01b3c26 T rb_prev
c01b3c4f T rb_replace_node
c01b3c94 T rwsem_wake
c01b3d89 T rwsem_downgrade_wake
c01b3e14 T strnicmp
c01b3e75 T strcpy
c01b3e89 T strncpy
c01b3ea8 T strlcpy
c01b3eeb T strcat
c01b3f0a T strncat
c01b3f35 T strlcat
c01b3f97 T strcmp
c01b3fae T strncmp
c01b3fe0 T strchr
c01b3ff0 T strrchr
Cheers,
Andrew Taylor
Systems Engineer
Array Systems Computing Inc.
email: taylor@array.ca
http: www.array.ca
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Oops] 2.6.10: PREEMPT SMP
2005-01-31 16:27 Xavier Bestel
@ 2005-02-04 11:43 ` David Howells
0 siblings, 0 replies; 6+ messages in thread
From: David Howells @ 2005-02-04 11:43 UTC (permalink / raw)
To: Xavier Bestel; +Cc: Linux Kernel List, akpm
Xavier Bestel <xavier.bestel@free.fr> wrote:
> I just got this Oops with 2.6.10 (debian/sid stock kernel).
>
> Kernel is tainted by VMWare, but it wasn't used (machine powered on
> remotely and used just to run gaim though ssh). I can perhaps try to
> reproduce it without it though if you need.
Hmmm... I see it involves the key stuff I wrote.
I don't think it can be a problem with preemption interfering with the key
management code accessing the key tree; every access to the tree outside of
the bootup initialisation is made with the appropriate spinlock held - and
that disables preemption.
It seems unlikely to be a double free... keys aren't freed the moment their
usage count reaches zero; a separate daemon is enlisted to go through the tree
when there's something to dispose of and extract and free all unused keys.
However, it's not impossible that there's a race there that I can't see
(though it doesn't look likely). Are you willing to try patching your kernel
with something? If so, if you can look through security/keys/key.c, and every
time you see a line saying:
kmem_cache_free(key_jar, key);
insert this line before it:
memset(key, 0xbb, sizeof(*key);
This will corrupt the memory that held the dead key before freeing it. Then if
something is touching a dead key, the pattern 0xbbbbbbbb or similar will crop
up in a register or on the stack, and the kernel will very likely crash.
David
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Oops] 2.6.10: PREEMPT SMP
@ 2005-01-31 16:27 Xavier Bestel
2005-02-04 11:43 ` David Howells
0 siblings, 1 reply; 6+ messages in thread
From: Xavier Bestel @ 2005-01-31 16:27 UTC (permalink / raw)
To: Linux Kernel List
Hi,
I just got this Oops with 2.6.10 (debian/sid stock kernel).
Kernel is tainted by VMWare, but it wasn't used (machine powered on
remotely and used just to run gaim though ssh). I can perhaps try to
reproduce it without it though if you need.
Xav
Jan 31 14:08:01 bip kernel: c01c1447
Jan 31 14:08:01 bip kernel: PREEMPT SMP
Jan 31 14:08:01 bip kernel: Modules linked in: vmnet vmmon ipv6 lp thermal fan button processor ac battery nfs lockd sunrpc eth1394 af_packet eepro100 e100 ohci1394 ieee1394 snd_ens1371 snd_rawmidi snd_seq_device snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc gameport uhci_hcd usbcore pci_hotplug
via_agp agpgart parport_pc parport floppy pcspkr rtc ext2 reiserfs tsdev mousedev evdev capability commoncap ide_cd cdrom psmouse via686a eeprom i2c_sensor i2c_isa i2c_viapro i2c_core 8139too mii ext3 jbd mbcache ide_generic via82cxxx trm290 triflex slc90e66 sis5513 siimage serverworks sc1200 rz1000 piix pdc202xx_old opti621 ns87415 hpt366 ide_disk hpt34x generic cy82c693 cs5530 cs5520 cmd64x atiixp amd74xx alim15x3 aec62xx pdc202xx_new ide_core unix fbcon font bitblit vesafb cfbcopyarea cfbimgblt cfbfillrect
Jan 31 14:08:01 bip kernel: CPU: 1
Jan 31 14:08:01 bip kernel: EIP: 0060:[__rb_rotate_left+7/64] Tainted: P VLI
Jan 31 14:08:01 bip kernel: EFLAGS: 00010286 (2.6.10-1-686-smp)
Jan 31 14:08:01 bip kernel: EIP is at __rb_rotate_left+0x7/0x40
Jan 31 14:08:01 bip kernel: eax: f1b8da60 ebx: da7f38e0 ecx: f58864a0 edx: 00000000
Jan 31 14:08:01 bip kernel: esi: f58864a0 edi: f1b8da60 ebp: c03b17a4 esp: eaa93ea0
Jan 31 14:08:01 bip kernel: ds: 007b es: 007b ss: 0068
Jan 31 14:08:01 bip kernel: Process cron (pid: 8568, threadinfo=eaa92000 task=f5b65a20)
Jan 31 14:08:01 bip kernel: Stack: c01c154f f58864a0 c03b17a4 da7f38e0 f1b8da6c f1b8da60 da7f38e0 c01a6d07
Jan 31 14:08:01 bip kernel: da7f38e0 c03b17a4 eaa93f40 00000008 eaa93f4b ffffffea c01a6dfc 00000008
Jan 31 14:08:01 bip kernel: ffffffff 00000000 0000000b 00000013 00000000 eaa93f40 ffffffff f60d20e0
Jan 31 14:08:01 bip kernel: Call Trace:
Jan 31 14:08:01 bip kernel: [rb_insert_color+143/240] rb_insert_color+0x8f/0xf0
Jan 31 14:08:01 bip kernel: [key_user_lookup+215/272] key_user_lookup+0xd7/0x110
Jan 31 14:08:01 bip kernel: [key_alloc+92/800] key_alloc+0x5c/0x320
Jan 31 14:08:01 bip kernel: [keyring_alloc+64/144] keyring_alloc+0x40/0x90
Jan 31 14:08:01 bip kernel: [alloc_uid_keyring+74/192] alloc_uid_keyring+0x4a/0xc0
Jan 31 14:08:01 bip kernel: [alloc_uid+199/384] alloc_uid+0xc7/0x180
Jan 31 14:08:01 bip kernel: [set_user+19/144] set_user+0x13/0x90
Jan 31 14:08:01 bip kernel: [sys_setuid+179/336] sys_setuid+0xb3/0x150
Jan 31 14:08:01 bip kernel: [sysenter_past_esp+82/117] sysenter_past_esp+0x52/0x75
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2005-03-24 0:50 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2005-02-18 11:29 [Oops] 2.6.10: PREEMPT SMP Klaus Steinberger
-- strict thread matches above, loose matches on Subject: below --
2005-03-08 20:50 Andrew Taylor
2005-03-21 23:35 ` Andrew Morton
2005-03-24 0:42 ` Andrew Taylor
2005-01-31 16:27 Xavier Bestel
2005-02-04 11:43 ` David Howells
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).