From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.1 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D67BC433DF for ; Thu, 2 Jul 2020 19:22:30 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id D4914221E8 for ; Thu, 2 Jul 2020 19:22:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CFHRBkvV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D4914221E8 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0F0D98D0037; Thu, 2 Jul 2020 15:22:29 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0A0FC8D000C; Thu, 2 Jul 2020 15:22:29 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED17F8D0037; Thu, 2 Jul 2020 15:22:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0061.hostedemail.com [216.40.44.61]) by kanga.kvack.org (Postfix) with ESMTP id D7ADB8D000C for ; Thu, 2 Jul 2020 15:22:28 -0400 (EDT) Received: from smtpin29.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 84B0D45A8 for ; Thu, 2 Jul 2020 19:22:28 +0000 (UTC) X-FDA: 76994107176.29.slope23_041605a26e8b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin29.hostedemail.com (Postfix) with ESMTP id 5BA8818086CDA for ; Thu, 2 Jul 2020 19:22:28 +0000 (UTC) X-HE-Tag: slope23_041605a26e8b X-Filterd-Recvd-Size: 8589 Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by imf31.hostedemail.com (Postfix) with ESMTP for ; Thu, 2 Jul 2020 19:22:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1593717747; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Wow0Ygu65peLja/h/OTF4fc7jwjBqgs+uK8UVoodCFw=; b=CFHRBkvVWPm4Z15JWbOvOmsFErwENDVi5yzNrQbhqpIK97fttnbgd6x56ATtmMkDAq+qQZ oDtBy1vxwviQtJiYV1iBElajW8qRApZUuVtc9zVkrkNpxcNZ3NN51d8V/OFDwW24Ko8N8j uFtoV1pWJQm4s44kIDQf8jDbeW1NB/0= Received: from mail-qv1-f72.google.com (mail-qv1-f72.google.com [209.85.219.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-233-Un_tEUZiMKubS30bex4dAw-1; Thu, 02 Jul 2020 15:22:21 -0400 X-MC-Unique: Un_tEUZiMKubS30bex4dAw-1 Received: by mail-qv1-f72.google.com with SMTP id r12so19073415qvk.3 for ; Thu, 02 Jul 2020 12:22:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Wow0Ygu65peLja/h/OTF4fc7jwjBqgs+uK8UVoodCFw=; b=EjsNyyMnaCwE60MviELmBI06SSUy9orInS2eEGCoXJr4DjtlP1I2ZrBjWUQcg714Oe CtrxsoQvW/3QCuU9yM6itufAghpx0yQHaTiurdibdDcmwvozJSTtWwrgr1wngcPLNTeW e5MN8pD/LddqXfG5SOPRA/wsD6788cu/Ph7sianWcfCxi5d1xYPwaR8sE6dhPx4cncJD NP49+HDhRn5+1FABricEuUNNnNSVUtH0YKSjL8sQXUUUTlh3engVM7yNVi/okeXzT8nQ UxtFTT7aG16enP+3awaFnm9GfqQ2VhXRpNjNR833VT4mTJw+NnqVpYtvDMvVoXdN/Fp0 Sw/Q== X-Gm-Message-State: AOAM532NWr2yJ64gXDJBel0K7UDnfwHIxqy/tFzwa25X5/f8DeQ3RQMj 1AhPz+vcB6GFjZ6fyQ65gGrADGXzoy6QKSmWKQWG+fsCRMY9Ya3Qm0ry67UM5NoWXIZy+PgjFrH gUndZaRiD679QDSY38iFPRA4m9cM= X-Received: by 2002:ac8:429b:: with SMTP id o27mr21653198qtl.124.1593717740705; Thu, 02 Jul 2020 12:22:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxT/Yz66Mm2DZnNpTLY1kn5bds91N12gXuAl8xcrffYobR0MkCqmKoDHO8iv5NQ7rsNdb5NF+5YXp8MRlwfnAI= X-Received: by 2002:ac8:429b:: with SMTP id o27mr21653164qtl.124.1593717740325; Thu, 02 Jul 2020 12:22:20 -0700 (PDT) MIME-Version: 1.0 References: <1593641660-13254-1-git-send-email-bhsharma@redhat.com> <1593641660-13254-3-git-send-email-bhsharma@redhat.com> <20200702075001.GA16113@willie-the-truck> In-Reply-To: <20200702075001.GA16113@willie-the-truck> From: Bhupesh Sharma Date: Fri, 3 Jul 2020 00:52:08 +0530 Message-ID: Subject: Re: [PATCH 2/2] arm64: Allocate crashkernel always in ZONE_DMA To: Will Deacon Cc: cgroups@vger.kernel.org, linux-mm@kvack.org, linux-arm-kernel , Bhupesh SHARMA , Johannes Weiner , Michal Hocko , Vladimir Davydov , James Morse , Mark Rutland , Catalin Marinas , Linux Kernel Mailing List , kexec mailing list Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=bhsharma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 5BA8818086CDA X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam05 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi Will, On Thu, Jul 2, 2020 at 1:20 PM Will Deacon wrote: > > On Thu, Jul 02, 2020 at 03:44:20AM +0530, Bhupesh Sharma wrote: > > commit bff3b04460a8 ("arm64: mm: reserve CMA and crashkernel in > > ZONE_DMA32") allocates crashkernel for arm64 in the ZONE_DMA32. > > > > However as reported by Prabhakar, this breaks kdump kernel booting in > > ThunderX2 like arm64 systems. I have noticed this on another ampere > > arm64 machine. The OOM log in the kdump kernel looks like this: > > > > [ 0.240552] DMA: preallocated 128 KiB GFP_KERNEL pool for atomic allocations > > [ 0.247713] swapper/0: page allocation failure: order:1, mode:0xcc1(GFP_KERNEL|GFP_DMA), nodemask=(null),cpuset=/,mems_allowed=0 > > <..snip..> > > [ 0.274706] Call trace: > > [ 0.277170] dump_backtrace+0x0/0x208 > > [ 0.280863] show_stack+0x1c/0x28 > > [ 0.284207] dump_stack+0xc4/0x10c > > [ 0.287638] warn_alloc+0x104/0x170 > > [ 0.291156] __alloc_pages_slowpath.constprop.106+0xb08/0xb48 > > [ 0.296958] __alloc_pages_nodemask+0x2ac/0x2f8 > > [ 0.301530] alloc_page_interleave+0x20/0x90 > > [ 0.305839] alloc_pages_current+0xdc/0xf8 > > [ 0.309972] atomic_pool_expand+0x60/0x210 > > [ 0.314108] __dma_atomic_pool_init+0x50/0xa4 > > [ 0.318504] dma_atomic_pool_init+0xac/0x158 > > [ 0.322813] do_one_initcall+0x50/0x218 > > [ 0.326684] kernel_init_freeable+0x22c/0x2d0 > > [ 0.331083] kernel_init+0x18/0x110 > > [ 0.334600] ret_from_fork+0x10/0x18 > > > > This patch limits the crashkernel allocation to the first 1GB of > > the RAM accessible (ZONE_DMA), as otherwise we might run into OOM > > issues when crashkernel is executed, as it might have been originally > > allocated from either a ZONE_DMA32 memory or mixture of memory chunks > > belonging to both ZONE_DMA and ZONE_DMA32. > > How does this interact with this ongoing series: > > https://lore.kernel.org/r/20200628083458.40066-1-chenzhou10@huawei.com > > (patch 4, in particular) Many thanks for having a look at this patchset. I was not aware that Chen had sent out a new version. I had noted in the v9 review of the high/low range allocation that I was working on a generic solution (irrespective of the crashkernel, low and high range allocation) which resulted in this patchset. The issue is two-fold: OOPs in memcfg layer (PATCH 1/2, which has been Acked-by memcfg maintainer) and OOM in the kdump kernel due to crashkernel allocation in ZONE_DMA32 regions(s) which is addressed by this PATCH. I will have a closer look at the v10 patchset Chen shared, but seems it needs some rework as per Dave's review comments which he shared today. IMO, in the meanwhile this patchset can be used to fix the existing kdump issue with upstream kernel. > > Fixes: bff3b04460a8 ("arm64: mm: reserve CMA and crashkernel in ZONE_DMA32") > > Cc: Johannes Weiner > > Cc: Michal Hocko > > Cc: Vladimir Davydov > > Cc: James Morse > > Cc: Mark Rutland > > Cc: Will Deacon > > Cc: Catalin Marinas > > Cc: cgroups@vger.kernel.org > > Cc: linux-mm@kvack.org > > Cc: linux-arm-kernel@lists.infradead.org > > Cc: linux-kernel@vger.kernel.org > > Cc: kexec@lists.infradead.org > > Reported-by: Prabhakar Kushwaha > > Signed-off-by: Bhupesh Sharma > > --- > > arch/arm64/mm/init.c | 16 ++++++++++++++-- > > 1 file changed, 14 insertions(+), 2 deletions(-) > > > > diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c > > index 1e93cfc7c47a..02ae4d623802 100644 > > --- a/arch/arm64/mm/init.c > > +++ b/arch/arm64/mm/init.c > > @@ -91,8 +91,15 @@ static void __init reserve_crashkernel(void) > > crash_size = PAGE_ALIGN(crash_size); > > > > if (crash_base == 0) { > > - /* Current arm64 boot protocol requires 2MB alignment */ > > - crash_base = memblock_find_in_range(0, arm64_dma32_phys_limit, > > + /* Current arm64 boot protocol requires 2MB alignment. > > + * Also limit the crashkernel allocation to the first > > + * 1GB of the RAM accessible (ZONE_DMA), as otherwise we > > + * might run into OOM issues when crashkernel is executed, > > + * as it might have been originally allocated from > > + * either a ZONE_DMA32 memory or mixture of memory > > + * chunks belonging to both ZONE_DMA and ZONE_DMA32. > > + */ > > This comment needs help. Why does putting the crashkernel in ZONE_DMA > prevent "OOM issues"? Sure, I can work on adding more details in the comment so that it explains the potential OOM issue(s) better. Thanks, Bhupesh