From: Grzegorz Nosek <grzegorz.nosek@gmail.com>
To: Andrew Morton <akpm@osdl.org>
Cc: linux-kernel@vger.kernel.org
Subject: Re: [PATCH] race condition in procfs
Date: Tue, 29 Nov 2005 09:38:29 +0100 [thread overview]
Message-ID: <121a28810511290038h37067fecx@mail.gmail.com> (raw)
In-Reply-To: <20051129000916.6306da8b.akpm@osdl.org>
[-- Attachment #1: Type: text/plain, Size: 1601 bytes --]
2005/11/29, Andrew Morton <akpm@osdl.org>:
> > I found a race condition in procfs on SMP systems. The result is an
> > oops in processes like pidof. Apparently ->proc_read() gets passed a
> > potentially NULL pointer.
>
> Do you know what the race is?
Apparently it's a race between deleting a process and accessing its
/proc/pid entries. It came out in pidof while it was accessing
/proc/pid/stat (fs/proc/array.c:do_task_stat crashed on first
instruction - it was an inline function accessing task->state,
get_task_state IIRC). oops (with vserver history data - I'm using a
patch mentioned below) is attached.
>
> How does one reproduce it?
I managed to reproduce it (although not reliably) during high CPU load
and I/O (parallel kernel compiles) on SMP systems with the vserver
patch (http://linux-vserver.org, the exact patch is
http://vserver.13thfloor.at/Experimental/patch-2.6.14.2-vs2.1.0-rc8.diff),
but the vserver maintainer pointed out that it probably is a mainline
issue. We're not using 2.6 systems too much except for the vserver
test beds so I cannot tell if it happens on vanilla kernels.
>
> > The following micro-patch seems to fix it.
>
> It might be right, or it might be a workaround..
>
I'm not a kernel guru so it's just my proposal. Can it break anything?
An alternative _might_ be somewhat coarser task_struct locking
(do_task_stat grabs a spinlock but then it's already too late).
However, if no "right" solution appears, I'll keep using my two-liner
because it seems to help, at least in my setup.
Best regards,
Grzegorz Nosek
[-- Attachment #2: oops.s35 --]
[-- Type: application/octet-stream, Size: 8426 bytes --]
Nov 27 00:15:26 s35 [43281574.240000] Unable to handle kernel NULL pointer dereference
Nov 27 00:15:26 s35 at virtual address 00000000
Nov 27 00:15:26 s35 [43281574.240000] printing eip:
Nov 27 00:15:26 s35 [43281574.240000] a01b50eb
Nov 27 00:15:26 s35 [43281574.240000] *pde = 00000000
Nov 27 00:15:26 s35 [43281574.240000] Oops: 0000 [#1]
Nov 27 00:15:26 s35 [43281574.240000] SMP
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] Modules linked in:
Nov 27 00:15:26 s35 ipt_owner
Nov 27 00:15:26 s35 ipt_state
Nov 27 00:15:26 s35 iptable_filter
Nov 27 00:15:26 s35 netconsole
Nov 27 00:15:26 s35 uhci_hcd
Nov 27 00:15:26 s35 ohci_hcd
Nov 27 00:15:26 s35 ehci_hcd
Nov 27 00:15:26 s35 usbcore
Nov 27 00:15:26 s35 ip_conntrack_ftp
Nov 27 00:15:26 s35 ip_conntrack
Nov 27 00:15:26 s35 forcedeth
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] CPU: 1
Nov 27 00:15:26 s35 [43281574.240000] EIP: 0060:[<a01b50eb>] Not tainted VLI
Nov 27 00:15:26 s35 [43281574.240000] EFLAGS: 00010257 (2.6.14.2amd64smp.17)
Nov 27 00:15:26 s35 [43281574.240000] EIP is at do_task_stat+0x8b/0x890
Nov 27 00:15:26 s35 [43281574.240000] eax: 00000000 ebx: 00000000 ecx: a0601700 edx: c804ad48
Nov 27 00:15:26 s35 [43281574.240000] esi: b3fbe000 edi: f666aa70 ebp: d7e65f20 esp: d7e65da0
Nov 27 00:15:26 s35 [43281574.240000] ds: 007b es: 007b ss: 0068
Nov 27 00:15:26 s35 [43281574.240000] Process pidof (pid: 4723, threadinfo=d7e64000 task=e24e7550)
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] Stack:
Nov 27 00:15:26 s35 a01b1e2e
Nov 27 00:15:26 s35 f666aa70
Nov 27 00:15:26 s35 d7e65f28
Nov 27 00:15:26 s35 a8cab11c
Nov 27 00:15:26 s35 d7e65e24
Nov 27 00:15:26 s35 d7e65de8
Nov 27 00:15:26 s35 a0184934
Nov 27 00:15:26 s35 d7e65e24
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000]
Nov 27 00:15:26 s35 a8cab544
Nov 27 00:15:26 s35 d7e65de8
Nov 27 00:15:26 s35 a019090d
Nov 27 00:15:26 s35 a8cab544
Nov 27 00:15:26 s35 a0720a00
Nov 27 00:15:26 s35 d7e65df8
Nov 27 00:15:26 s35 a2227140
Nov 27 00:15:26 s35 00000000
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000]
Nov 27 00:15:26 s35 00000000
Nov 27 00:15:26 s35 d7e65e2c
Nov 27 00:15:26 s35 d7e65e48
Nov 27 00:15:26 s35 a0185664
Nov 27 00:15:26 s35 a8cab544
Nov 27 00:15:26 s35 d7e65e2c
Nov 27 00:15:26 s35 d7e65e24
Nov 27 00:15:26 s35 c94ff00b
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] Call Trace:
Nov 27 00:15:26 s35 [43281574.240000] [<a0103e9f>]
Nov 27 00:15:26 s35 show_stack+0x7f/0xa0
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a010403d>]
Nov 27 00:15:26 s35 show_registers+0x15d/0x1d0
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a0104252>]
Nov 27 00:15:26 s35 die+0x112/0x1c0
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a055c2b9>]
Nov 27 00:15:26 s35 do_page_fault+0x3d9/0x650
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a0103b53>]
Nov 27 00:15:26 s35 error_code+0x4f/0x54
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a01b5940>]
Nov 27 00:15:26 s35 proc_tgid_stat+0x20/0x30
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a01b0f75>]
Nov 27 00:15:26 s35 proc_info_read+0x55/0xa0
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a0174d68>]
Nov 27 00:15:26 s35 vfs_read+0x198/0x1a0
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a017506b>]
Nov 27 00:15:26 s35 sys_read+0x4b/0x80
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] [<a010302d>]
Nov 27 00:15:26 s35 syscall_call+0x7/0xb
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] Code:
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 c7
Nov 27 00:15:26 s35 85
Nov 27 00:15:26 s35 6c
Nov 27 00:15:26 s35 ff
Nov 27 00:15:26 s35 ff
Nov 27 00:15:26 s35 ff
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 8b
Nov 27 00:15:26 s35 07
Nov 27 00:15:26 s35 8b
Nov 27 00:15:26 s35 9f
Nov 27 00:15:26 s35 84
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 25
Nov 27 00:15:26 s35 8f
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 83
Nov 27 00:15:26 s35 e3
Nov 27 00:15:26 s35 30
Nov 27 00:15:26 s35 09
Nov 27 00:15:26 s35 d8
Nov 27 00:15:26 s35 eb
Nov 27 00:15:26 s35 05
Nov 27 00:15:26 s35 83
Nov 27 00:15:26 s35 c1
Nov 27 00:15:26 s35 04
Nov 27 00:15:26 s35 d1
Nov 27 00:15:26 s35 e8
Nov 27 00:15:26 s35 75
Nov 27 00:15:26 s35 f9
Nov 27 00:15:26 s35 8b
Nov 27 00:15:26 s35 01
Nov 27 00:15:26 s35 unparseable log message: "<0f> "
Nov 27 00:15:26 s35 b6
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 c7
Nov 27 00:15:26 s35 45
Nov 27 00:15:26 s35 c8
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 c7
Nov 27 00:15:26 s35 45
Nov 27 00:15:26 s35 cc
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35 c7
Nov 27 00:15:26 s35 45
Nov 27 00:15:26 s35 d0
Nov 27 00:15:26 s35 00
Nov 27 00:15:26 s35
Nov 27 00:15:26 s35 [43281574.240000] History: SEQ: 3ddca14 NR_CPUS: 8
Nov 27 00:15:26 s35 [43281574.240000] (#c9ae,*0):a04d546e set_vx_info f6e48000[#830,190.71] @f4fcf4e8
Nov 27 00:15:26 s35 [43281574.240000] (#c964,*1):a013ac82 release_vx_info f6e48000[#830,190.74] @c5cdb030
Nov 27 00:15:26 s35 [43281574.240000] (#ca13,*0):a04d40b2 clr_vx_info f6e48000[#830,188.71] @db738068
Nov 27 00:15:26 s35 [43281574.240000] (#ca14,*1):a0104140 oops
Nov 27 00:15:26 s35 [43281574.240000] (#ca12,*0):a04d40b2 clr_vx_info f6e48000[#830,189.71] @db739b68
Nov 27 00:15:26 s35 [43281574.240000] (#ca0f,*1):a011c57c clr_vx_info f6e1e000[#831,151.39] @f6f1bad0
Nov 27 00:15:26 s35 [43281574.240000] (#ca11,*0):a04d40b2 clr_vx_info f6e48000[#830,190.71] @ad3a16e8
Nov 27 00:15:26 s35 [43281574.240000] (#ca0e,*1):a011c45c set_vx_info f6e1e000[#831,150.39] @f6f1a210
Nov 27 00:15:26 s35 [43281574.240000] (#ca10,*0):a04d40b2 clr_vx_info f6e48000[#830,191.71] @ed1a7b68
Nov 27 00:15:26 s35 [43281574.240000] (#ca0d,*1):a04d40b2 clr_vx_info f6e48000[#830,192.71] @b6b98ae8
Nov 27 00:15:26 s35 [43281574.240000] (#ca03,*0):a04d4556 set_vx_info f6e48000[#830,190.71] @ba389268
Nov 27 00:15:26 s35 [43281574.240000] (#ca0c,*1):a011ddac claim_vx_info f6e1e000[#831,150.38] @e24e7550
Nov 27 00:15:26 s35 [43281574.240000] (#ca02,*0):a04d40b2 clr_vx_info f6e48000[#830,191.71] @b6b98d68
Nov 27 00:15:26 s35 [43281574.240000] (#ca0b,*1):a011c45c set_vx_info f6e1e000[#831,149.38] @f6f1bad0
Nov 27 00:15:26 s35 [43281574.240000] (#ca01,*0):a04d546e set_vx_info f6e48000[#830,190.71] @b6b98d68
Nov 27 00:15:26 s35 [43281574.240000] (#ca0a,*1):a011d38c init_vx_info f6e1e000[#831,148.38] @e24e79f8
Nov 27 00:15:26 s35 [43281574.240000] (#ca00,*0):a04d40b2 clr_vx_info f6e48000[#830,191.71] @f4fcf4e8
Nov 27 00:15:26 s35 [43281574.240000] (#ca09,*1):a011bf10 clr_vx_info f6e1e000[#831,149.38] @e24e79f8
Nov 27 00:15:26 s35 [43281574.240000] (#c9ff,*0):a04d40b2 clr_vx_info f6e48000[#830,192.71] @f56bbde8
Nov 27 00:15:26 s35 [43281574.240000] (#ca08,*1):a04d546e set_vx_info f6e48000[#830,191.71] @b6b98ae8
Nov 27 00:15:26 s35 [43281574.240000] (#c9fe,*0):a04d546e set_vx_info f6e48000[#830,191.71] @f56bbde8
Nov 27 00:15:26 s35 [43281574.240000] (#ca07,*1):a04d40b2 clr_vx_info f6e48000[#830,192.71] @cbec5068
Nov 27 00:15:26 s35 [43281574.240000] (#c9fd,*0):a04d546e set_vx_info f6e48000[#830,190.71] @f4fcf4e8
Nov 27 00:15:26 s35 [43281574.240000] (#ca06,*1):a04d40b2 clr_vx_info f6e48000[#830,193.71] @cbec5ba8
Nov 27 00:15:26 s35 [43281574.240000] (#c9fc,*0):a04d4556 set_vx_info f6e48000[#830,189.71] @ad3a16e8
Nov 27 00:15:26 s35 [43281574.240000] (#ca05,*1):a04d546e set_vx_info f6e48000[#830,192.71] @cbec5ba8
Nov 27 00:15:26 s35 [43281574.240000] (#c9fb,*0):a04d40b2 clr_vx_info f6e48000[#830,190.71] @ad3a16e8
Nov 27 00:15:26 s35 [43281574.240000] (#ca04,*1):a04d546e set_vx_info f6e48000[#830,191.71] @cbec5068
Nov 27 00:15:26 s35 [43281574.240000] (#c9f7,*0):a011c57c clr_vx_info f6e1e000[#831,148.37] @f6f1a790
Nov 27 00:15:26 s35 [43281574.240000] (#c9fa,*1):a011ddac claim_vx_info f6e1e000[#831,149.37] @e4856550
next prev parent reply other threads:[~2005-11-29 8:38 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2005-11-29 7:17 [PATCH] race condition in procfs Grzegorz Nosek
2005-11-29 8:09 ` Andrew Morton
2005-11-29 8:38 ` Grzegorz Nosek [this message]
2005-11-29 13:25 ` Grzegorz Nosek
2005-11-29 14:04 ` Grzegorz Nosek
2005-11-29 14:28 ` Steven Rostedt
2005-11-29 14:39 ` Grzegorz Nosek
2005-11-29 14:49 ` Steven Rostedt
2005-11-30 14:41 ` Grzegorz Nosek
2005-11-30 15:14 ` Steven Rostedt
2005-11-30 15:29 ` Grzegorz Nosek
2005-11-30 16:25 ` Steven Rostedt
2005-11-30 17:23 ` Grzegorz Nosek
2005-12-01 20:38 ` Grzegorz Nosek
2005-11-29 15:22 ` [PATCH] shrinks dentry struct Eric Dumazet
2005-11-30 2:06 ` Paul Jackson
2005-11-30 2:14 ` Andrew Morton
2005-11-30 2:43 ` Paul Jackson
2005-11-30 6:56 ` Hugh Dickins
2005-12-03 1:15 ` [PATCH] remove unused blkp field in percpu_data Eric Dumazet
2005-12-13 18:03 ` [PATCH] shrinks dentry struct Paul E. McKenney
2005-12-13 18:24 ` Eric Dumazet
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=121a28810511290038h37067fecx@mail.gmail.com \
--to=grzegorz.nosek@gmail.com \
--cc=akpm@osdl.org \
--cc=linux-kernel@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.