From: Vegard Nossum <vegard.nossum@oracle.com>
To: Michal Hocko <mhocko@kernel.org>
Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org,
Rik van Riel <riel@redhat.com>,
Matthew Wilcox <mawilcox@microsoft.com>,
Peter Zijlstra <peterz@infradead.org>,
Andrew Morton <akpm@linux-foundation.org>,
Al Viro <viro@zeniv.linux.org.uk>, Ingo Molnar <mingo@kernel.org>,
Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: crash during oom reaper
Date: Fri, 16 Dec 2016 15:53:11 +0100 [thread overview]
Message-ID: <53b709f5-c560-a8ae-616f-cc7a64ff536c@oracle.com> (raw)
In-Reply-To: <20161216143235.GO13940@dhcp22.suse.cz>
On 12/16/2016 03:32 PM, Michal Hocko wrote:
> On Fri 16-12-16 15:25:27, Vegard Nossum wrote:
>> On 12/16/2016 03:00 PM, Michal Hocko wrote:
>>> On Fri 16-12-16 14:14:17, Vegard Nossum wrote:
>>> [...]
>>>> Out of memory: Kill process 1650 (trinity-main) score 90 or sacrifice child
>>>> Killed process 1724 (trinity-c14) total-vm:37280kB, anon-rss:236kB,
>>>> file-rss:112kB, shmem-rss:112kB
>>>> BUG: unable to handle kernel NULL pointer dereference at 00000000000001e8
>>>> IP: [<ffffffff8126b1c0>] copy_process.part.41+0x2150/0x5580
>>>> PGD c001067 PUD c000067
>>>> PMD 0
>>>> Oops: 0002 [#1] PREEMPT SMP KASAN
>>>> Dumping ftrace buffer:
>>>> (ftrace buffer empty)
>>>> CPU: 28 PID: 1650 Comm: trinity-main Not tainted 4.9.0-rc6+ #317
>>>
>>> Hmm, so this was the oom victim initially but we have decided to kill
>>> its child 1724 instead.
>>>
>>>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
>>>> Ubuntu-1.8.2-1ubuntu1 04/01/2014
>>>> task: ffff88000f9bc440 task.stack: ffff88000c778000
>>>> RIP: 0010:[<ffffffff8126b1c0>] [<ffffffff8126b1c0>]
>>>> copy_process.part.41+0x2150/0x5580
>>>
>>> Could you match this to the kernel source please?
>>
>> kernel/fork.c:629 dup_mmap()
>
> Ok, so this is before the child is made visible so the oom reaper
> couldn't have seen it.
>
>> it's atomic_dec(&inode->i_writecount), it matches up with
>> file_inode(file) == NULL:
>>
>> (gdb) p &((struct inode *)0)->i_writecount
>> $1 = (atomic_t *) 0x1e8 <irq_stack_union+488>
>
> is this a p9 inode?
When I looked at this before it always crashed in this spot for the very
first VMA in the mm (which happens to be the exe, which is on a 9p root fs).
I added a trace_printk() to dup_mmap() to print inode->i_sb->s_type and
the last thing I see for a new crash in the same place is:
trinity--9280 28.... 136345090us : copy_process.part.41: ffffffff8485ec40
---------------------------------
CPU: 0 PID: 9302 Comm: trinity-c0 Not tainted 4.9.0-rc8+ #332
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
Ubuntu-1.8.2-1ubuntu1 04/01/2014
task: ffff880000070000 task.stack: ffff8800099e0000
RIP: 0010:[<ffffffff8126c7c9>] [<ffffffff8126c7c9>]
copy_process.part.41+0x22c9/0x55b0
As you can see, the addresses match:
(gdb) p &v9fs_fs_type
$1 = (struct file_system_type *) 0xffffffff8485ec40 <v9fs_fs_type>
So I think we can safely say that yes, it's a p9 inode.
Vegard
next prev parent reply other threads:[~2016-12-16 14:53 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-12-16 8:21 [PATCH 1/4] mm: add new mmgrab() helper Vegard Nossum
2016-12-16 8:22 ` [PATCH 2/4] mm: add new mmget() helper Vegard Nossum
2016-12-16 9:26 ` Michal Hocko
2016-12-16 8:22 ` [PATCH 3/4] mm: use mmget_not_zero() helper Vegard Nossum
2016-12-16 9:27 ` Michal Hocko
2016-12-16 8:22 ` [PATCH 4/4] [RFC!] mm: 'struct mm_struct' reference counting debugging Vegard Nossum
2016-12-16 9:01 ` Michal Hocko
2016-12-16 9:43 ` Vegard Nossum
2016-12-16 10:11 ` crash during oom reaper (was: Re: [PATCH 4/4] [RFC!] mm: 'struct mm_struct' reference counting debugging) Michal Hocko
2016-12-16 10:44 ` Kirill A. Shutemov
2016-12-16 11:42 ` crash during oom reaper Michal Hocko
2016-12-16 12:12 ` Michal Hocko
2016-12-16 12:35 ` Kirill A. Shutemov
2016-12-16 12:56 ` Michal Hocko
2016-12-16 13:07 ` Kirill A. Shutemov
2016-12-16 13:14 ` Michal Hocko
2016-12-18 13:47 ` Tetsuo Handa
2016-12-18 16:06 ` Michal Hocko
2016-12-16 13:14 ` Vegard Nossum
2016-12-16 14:00 ` Michal Hocko
2016-12-16 14:25 ` Vegard Nossum
2016-12-16 14:32 ` Michal Hocko
2016-12-16 14:53 ` Vegard Nossum [this message]
2016-12-16 14:04 ` Vegard Nossum
2016-12-16 9:14 ` [PATCH 1/4] mm: add new mmgrab() helper Michal Hocko
2016-12-16 9:56 ` Peter Zijlstra
2016-12-16 10:19 ` Kirill A. Shutemov
2016-12-16 10:20 ` Vegard Nossum
2016-12-16 10:36 ` Michal Hocko
2016-12-16 11:14 ` Vegard Nossum
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=53b709f5-c560-a8ae-616f-cc7a64ff536c@oracle.com \
--to=vegard.nossum@oracle.com \
--cc=akpm@linux-foundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mawilcox@microsoft.com \
--cc=mhocko@kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=riel@redhat.com \
--cc=torvalds@linux-foundation.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).