From: Minchan Kim <minchan.kim@gmail.com>
To: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: Mel Gorman <mel@csn.ul.ie>,
linux-kernel@vger.kernel.org, akpm@linux-foundation.org,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"kosaki.motohiro@jp.fujitsu.com" <kosaki.motohiro@jp.fujitsu.com>
Subject: Re: error at compaction (Re: mmotm 2010-04-15-14-42 uploaded
Date: Tue, 20 Apr 2010 12:58:43 +0900 [thread overview]
Message-ID: <g2x28c262361004192058y64f4d316qcb1547909168e31f@mail.gmail.com> (raw)
In-Reply-To: <20100420120753.b161dea9.kamezawa.hiroyu@jp.fujitsu.com>
On Tue, Apr 20, 2010 at 12:07 PM, KAMEZAWA Hiroyuki
<kamezawa.hiroyu@jp.fujitsu.com> wrote:
> On Tue, 20 Apr 2010 11:39:46 +0900
> Minchan Kim <minchan.kim@gmail.com> wrote:
>
>> On Tue, Apr 20, 2010 at 4:39 AM, Mel Gorman <mel@csn.ul.ie> wrote:
>> > On Mon, Apr 19, 2010 at 07:14:42PM +0100, Mel Gorman wrote:
>> >> On Mon, Apr 19, 2010 at 07:01:33PM +0900, KAMEZAWA Hiroyuki wrote:
>> >> >
>> >> > mmotm 2010-04-15-14-42
>> >> >
>> >> > When I tried
>> >> > # echo 0 > /proc/sys/vm/compaction
>> >> >
>> >> > I see following.
>> >> >
>> >> > My enviroment was
>> >> > 2.6.34-rc4-mm1+ (2010-04-15-14-42) (x86-64) CPUx8
>> >> > allocating tons of hugepages and reduce free memory.
>> >> >
>> >> > What I did was:
>> >> > # echo 0 > /proc/sys/vm/compact_memory
>> >> >
>> >> > Hmm, I see this kind of error at migation for the 1st time..
>> >> > my.config is attached. Hmm... ?
>> >> >
>> >> > (I'm sorry I'll be offline soon.)
>> >>
>> >> That's ok, thanks you for the report. I'm afraid I made little progress
>> >> as I spent most of the day on other bugs but I do have something for
>> >> you.
>> >>
>> >> First, I reproduced the problem using your .config. However, the problem does
>> >> not manifest with the .config I normally use which is derived from the distro
>> >> kernel configuration (Debian Lenny). So, there is something in your .config
>> >> that triggers the problem. I very strongly suspect this is an interaction
>> >> between migration, compaction and page allocation debug.
>> >
>> > I unexpecedly had the time to dig into this. Does the following patch fix
>> > your problem? It Worked For Me.
>>
>> Nice catch during shot time. Below is comment.
>>
>> >
>> > ==== CUT HERE ====
>> > mm,compaction: Map free pages in the address space after they get split for compaction
>> >
>> > split_free_page() is a helper function which takes a free page from the
>> > buddy lists and splits it into order-0 pages. It is used by memory
>> > compaction to build a list of destination pages. If
>> > CONFIG_DEBUG_PAGEALLOC is set, a kernel paging request bug is triggered
>> > because split_free_page() did not call the arch-allocation hooks or map
>> > the page into the kernel address space.
>> >
>> > This patch does not update split_free_page() as it is called with
>> > interrupts held. Instead it documents that callers of split_free_page()
>> > are responsible for calling the arch hooks and to map the page and fixes
>> > compaction.
>>
>> Dumb question. Why can't we call arch_alloc_page and kernel_map_pages
>> as interrupt disabled? It's deadlock issue or latency issue?
>> I don't found any comment about it.
>> It should have added the comment around that functions. :)
>>
>
> I guess it's from the same reason as vfree(), which can't be called under
> irq-disabled.
>
> Both of them has to flush TLB of all cpus. At flushing TLB (of other cpus), cpus has
> to send IPI via smp_call_function. What I know from old stories is below.
>
> At sendinf IPI, usual sequence is following. (This may be old.)
>
> spin_lock(&ipi_lock);
> set up cpu mask for getting notification from other cpu for declearing
> "I received IPI and finished my own work".
> spin_unlock(&ipi_lock);
>
> Then,
> CPU0 CPU1
>
> irq_disable (somewhere) spin_lock
> send IPI and wait for notification.
> spin_lock()
>
> deadlock. Seeing decription of kernel/smp.c::smp_call_function_many(), it says
> this function should not be called under irq-disabled.
> (Maybe the same kind of spin-wait deadlock can happen.)
>
Thanks for kind explanation.
Actually I guessed TLB issue but I can't find any glue point which
connect tlb flush to smp_call_function_xxx. :(
Now look at the __native_flush_tlb_global.
It just read and write cr4 with just mask off X86_CR4_PGE.
So i don't know how connect this and smp_schedule_xxxx.
Hmm,, maybe APIC?
Sorry for dumb question.
--
Kind regards,
Minchan Kim
next prev parent reply other threads:[~2010-04-20 3:58 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-04-15 21:42 mmotm 2010-04-15-14-42 uploaded akpm
2010-04-15 23:38 ` [PATCH -mmotm] vmstat: fix build errors when PROC_FS is disabled Randy Dunlap
2010-04-16 16:03 ` mmotm 2010-04-15-14-42 uploaded (shmem, CGROUP_MEM_RES_CTLR) Randy Dunlap
2010-04-19 1:49 ` Daisuke Nishimura
2010-04-19 2:17 ` Randy Dunlap
2010-04-19 10:01 ` error at compaction (Re: mmotm 2010-04-15-14-42 uploaded KAMEZAWA Hiroyuki
2010-04-19 18:14 ` Mel Gorman
2010-04-19 19:39 ` Mel Gorman
2010-04-20 2:30 ` KAMEZAWA Hiroyuki
2010-04-20 2:39 ` Minchan Kim
2010-04-20 3:07 ` KAMEZAWA Hiroyuki
2010-04-20 3:58 ` Minchan Kim [this message]
2010-04-20 4:24 ` KAMEZAWA Hiroyuki
2010-04-20 8:20 ` Mel Gorman
2010-04-20 8:32 ` Minchan Kim
2010-04-20 8:44 ` Mel Gorman
2010-04-20 9:50 ` Minchan Kim
2010-04-20 9:58 ` Mel Gorman
2010-04-20 10:14 ` Minchan Kim
2010-04-21 8:28 ` KAMEZAWA Hiroyuki
2010-04-21 9:48 ` KAMEZAWA Hiroyuki
2010-04-21 10:20 ` Mel Gorman
2010-04-21 16:52 ` Minchan Kim
2010-04-21 23:01 ` Minchan Kim
2010-04-20 2:35 ` KAMEZAWA Hiroyuki
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=g2x28c262361004192058y64f4d316qcb1547909168e31f@mail.gmail.com \
--to=minchan.kim@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=kamezawa.hiroyu@jp.fujitsu.com \
--cc=kosaki.motohiro@jp.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mel@csn.ul.ie \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).