From: Mark Rutland <mark.rutland@arm.com>
To: Laura Abbott <labbott@redhat.com>
Cc: Laura Abbott <labbott@fedoraproject.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will.deacon@arm.com>,
Ard Biesheuvel <ard.biesheuvel@linaro.org>,
linux-arm-kernel@lists.infradead.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCHv2 2/3] arm64: Add support for ARCH_SUPPORTS_DEBUG_PAGEALLOC
Date: Tue, 2 Feb 2016 12:23:18 +0000 [thread overview]
Message-ID: <20160202122317.GA32305@leverpostej> (raw)
In-Reply-To: <56AFCD09.8000807@redhat.com>
On Mon, Feb 01, 2016 at 01:24:25PM -0800, Laura Abbott wrote:
> On 02/01/2016 04:29 AM, Mark Rutland wrote:
> >Hi,
> >
> >On Fri, Jan 29, 2016 at 03:46:57PM -0800, Laura Abbott wrote:
> >>
> >>ARCH_SUPPORTS_DEBUG_PAGEALLOC provides a hook to map and unmap
> >>pages for debugging purposes. This requires memory be mapped
> >>with PAGE_SIZE mappings since breaking down larger mappings
> >>at runtime will lead to TLB conflicts. Check if debug_pagealloc
> >>is enabled at runtime and if so, map everyting with PAGE_SIZE
> >>pages. Implement the functions to actually map/unmap the
> >>pages at runtime.
> >>
> >>
> >>Signed-off-by: Laura Abbott <labbott@fedoraproject.org>
> >
> >I tried to apply atop of the arm64 for-next/pgtable branch, but git
> >wasn't very happy about that -- which branch/patches is this based on?
> >
> >I'm not sure if I'm missing something, have something I shouldn't, or if
> >my MTA is corrupting patches again...
> >
>
> Hmmm, I based it off of your arm64-pagetable-rework-20160125 tag and
> Ard's patch for vmalloc and set_memory_* . The patches seem to apply
> on the for-next/pgtable branch as well so I'm guessing you are missing
> Ard's patch.
Yup, that was it. I evidently was paying far too little attention as I'd
also missed the mm/ patch for the !CONFIG_DEBUG_PAGEALLOC case.
Is there anything else in mm/ that I've potentially missed? I'm seeing a
hang on Juno just after reaching userspace (splat below) with
debug_pagealloc=on.
It looks like something's gone wrong around find_vmap_area -- at least
one CPU is forever awaiting vmap_area_lock, and presumably some other
CPU has held it and gone into the weeds, leading to the RCU stalls and
NMI lockup warnings.
[ 31.037054] INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 31.042684] 0-...: (1 ticks this GP) idle=999/140000000000001/0 softirq=738/738 fqs=418
[ 31.050795] (detected by 1, t=5255 jiffies, g=340, c=339, q=50)
[ 31.056935] rcu_preempt kthread starved for 4838 jiffies! g340 c339 f0x0 RCU_GP_WAIT_FQS(3) ->state=0x0
[ 36.509055] NMI watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [kworker/2:2H:995]
[ 36.521059] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 23s! [systemd-udevd:1048]
[ 36.533056] NMI watchdog: BUG: soft lockup - CPU#4 stuck for 23s! [systemd-udevd:1037]
[ 36.545055] NMI watchdog: BUG: soft lockup - CPU#5 stuck for 23s! [systemd-udevd:1036]
[ 56.497055] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [upstart-file-br:1012]
[ 94.057052] INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 94.062671] 0-...: (1 ticks this GP) idle=999/140000000000001/0 softirq=738/738 fqs=418
[ 94.070780] (detected by 1, t=21010 jiffies, g=340, c=339, q=50)
[ 94.076981] rcu_preempt kthread starved for 20593 jiffies! g340 c339 f0x2 RCU_GP_WAIT_FQS(3) ->state=0x0
[ 157.077052] INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 157.082673] 0-...: (1 ticks this GP) idle=999/140000000000001/0 softirq=738/738 fqs=418
[ 157.090782] (detected by 2, t=36765 jiffies, g=340, c=339, q=50)
[ 157.096986] rcu_preempt kthread starved for 36348 jiffies! g340 c339 f0x2 RCU_GP_WAIT_FQS(3) ->state=0x0
[ 220.097052] INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 220.102670] 0-...: (1 ticks this GP) idle=999/140000000000001/0 softirq=738/738 fqs=418
[ 220.110779] (detected by 2, t=52520 jiffies, g=340, c=339, q=50)
[ 220.116971] rcu_preempt kthread starved for 52103 jiffies! g340 c339 f0x2 RCU_GP_WAIT_FQS(3) ->state=0x0
[ 283.117052] INFO: rcu_preempt detected stalls on CPUs/tasks:
[ 283.122670] 0-...: (1 ticks this GP) idle=999/140000000000001/0 softirq=738/738 fqs=418
[ 283.130779] (detected by 1, t=68275 jiffies, g=340, c=339, q=50)
[ 283.136973] rcu_preempt kthread starved for 67858 jiffies! g340 c339 f0x2 RCU_GP_WAIT_FQS(3) ->state=0x0
Typically show-backtrace-all-active-cpus(l) gives me something like:
[ 183.282835] CPU: 0 PID: 998 Comm: systemd-udevd Tainted: G L 4.5.0-rc1+ #7
[ 183.290783] Hardware name: ARM Juno development board (r0) (DT)
[ 183.296659] task: ffffffc97437a400 ti: ffffffc973ec8000 task.ti: ffffffc973ec8000
[ 183.304095] PC is at _raw_spin_lock+0x34/0x48
[ 183.308421] LR is at find_vmap_area+0x24/0xa0
[ 183.312746] pc : [<ffffffc00065faf4>] lr : [<ffffffc000185bc4>] pstate: 60000145
[ 183.320092] sp : ffffffc973ecb6c0
[ 183.323382] x29: ffffffc973ecb6c0 x28: ffffffbde7d50300
[ 183.328662] x27: ffffffffffffffff x26: ffffffbde7d50300
[ 183.333941] x25: 000000097e513000 x24: 0000000000000001
[ 183.339219] x23: 0000000000000000 x22: 0000000000000001
[ 183.344498] x21: ffffffc000a6dd90 x20: ffffffc000a6d000
[ 183.349778] x19: ffffffc97540c000 x18: 0000007fc4e8b960
[ 183.355057] x17: 0000007fac3088d4 x16: ffffffc0001be448
[ 183.360336] x15: 003b9aca00000000 x14: 0032aa26d4000000
[ 183.365614] x13: ffffffffa94f64df x12: 0000000000000018
[ 183.370894] x11: ffffffc97eecd730 x10: 0000000000000030
[ 183.376173] x9 : ffffffbde7d50340 x8 : ffffffc0008556a0
[ 183.381451] x7 : ffffffc0008556b8 x6 : ffffffc0008556d0
[ 183.386729] x5 : ffffffc0009d2000 x4 : 0000000000000001
[ 183.392008] x3 : 000000000000d033 x2 : 000000000000000b
[ 183.397286] x1 : 00000000d038d033 x0 : ffffffc000a6dd90
[ 183.402563]
I'll have a go with lock debugging. Otherwise do you have any ideas?
Thanks,
Mark.
next prev parent reply other threads:[~2016-02-02 12:23 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-29 23:46 [PATCHv2 0/3] ARCH_SUPPORTS_DEBUG_PAGEALLOC for arm64 Laura Abbott
2016-01-29 23:46 ` [PATCHv2 1/3] arm64: Drop alloc function from create_mapping Laura Abbott
2016-01-30 10:34 ` Ard Biesheuvel
2016-02-01 12:11 ` Mark Rutland
2016-01-29 23:46 ` [PATCHv2 2/3] arm64: Add support for ARCH_SUPPORTS_DEBUG_PAGEALLOC Laura Abbott
2016-02-01 12:29 ` Mark Rutland
2016-02-01 21:24 ` Laura Abbott
2016-02-02 8:57 ` Ard Biesheuvel
2016-02-02 12:23 ` Mark Rutland [this message]
2016-02-02 12:31 ` Mark Rutland
2016-02-02 16:08 ` Laura Abbott
2016-01-29 23:46 ` [PATCHv2 3/3] arm64: ptdump: Indicate whether memory should be faulting Laura Abbott
[not found] <1454615017-24672-1-git-send-email-labbott@fedoraproject.org>
2016-02-04 19:43 ` [PATCHv2 2/3] arm64: Add support for ARCH_SUPPORTS_DEBUG_PAGEALLOC Laura Abbott
2016-02-05 12:05 ` Ard Biesheuvel
2016-02-05 14:20 ` Mark Rutland
2016-02-06 0:08 ` Laura Abbott
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160202122317.GA32305@leverpostej \
--to=mark.rutland@arm.com \
--cc=ard.biesheuvel@linaro.org \
--cc=catalin.marinas@arm.com \
--cc=labbott@fedoraproject.org \
--cc=labbott@redhat.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).