From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B65FFC433E2 for ; Mon, 14 Sep 2020 04:07:42 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0567820771 for ; Mon, 14 Sep 2020 04:07:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="iv5jIcsF" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0567820771 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ud0Weqxhlg+wjfGUqQCpwflqKIHk8jV4fxvX7IjafWA=; b=iv5jIcsFt4L8rGCHAr7B6/4lh jhUQZz3hy6yOA+7l1bIIcIt8fjcmgzmtXG//5L4kwtYZ15SSlRjRTov7o6RvAHEnbJHOQyWIZPNVY kbig45UPzWGO0gEZwFn/QKr+pwLEUY9y4N8iI715RB1b5BZ/+c4kA75w4iUW6Y9b7cxhiI4q59GE9 OLvxEi+Ji12UuuORNEHzLwBYTJJdHEoblN8NZvIG0Gpco7Y5rgXOHgRtk5wzkM7+pec5KMADtwsbj GtR+AHIejW5pS4gMcYPC5Yck9+beSAujQUz0j45uIFKcGbOMxZ/ti+hAtLoFwUp9CCthiELZTOKLK DbXkiKQ6A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kHfke-0003Ao-Vr; Mon, 14 Sep 2020 04:06:01 +0000 Received: from foss.arm.com ([217.140.110.172]) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kHfkb-0003AR-8a for linux-arm-kernel@lists.infradead.org; Mon, 14 Sep 2020 04:05:58 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 799661045; Sun, 13 Sep 2020 21:05:52 -0700 (PDT) Received: from [192.168.0.130] (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B12343F718; Sun, 13 Sep 2020 21:05:49 -0700 (PDT) Subject: Re: [PATCH V2] arm64/hotplug: Improve memory offline event notifier To: Catalin Marinas References: <1598241869-28416-1-git-send-email-anshuman.khandual@arm.com> <20200911140603.GB12835@gaia> From: Anshuman Khandual Message-ID: Date: Mon, 14 Sep 2020 09:35:05 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20200911140603.GB12835@gaia> Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200914_000557_404654_861ED245 X-CRM114-Status: GOOD ( 24.90 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Steve Capper , Marc Zyngier , linux-kernel@vger.kernel.org, Mark Brown , Will Deacon , linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 09/11/2020 07:36 PM, Catalin Marinas wrote: > Hi Anshuman, > > On Mon, Aug 24, 2020 at 09:34:29AM +0530, Anshuman Khandual wrote: >> This brings about three different changes to the sole memory event notifier >> for arm64 platform and improves it's robustness while also enhancing debug >> capabilities during potential memory offlining error conditions. >> >> This moves the memory notifier registration bit earlier in the boot process >> from device_initcall() to setup_arch() which will help in guarding against >> potential early boot memory offline requests. >> >> This enables MEM_OFFLINE memory event handling. It will help intercept any >> possible error condition such as if boot memory some how still got offlined >> even after an expilicit notifier failure, potentially by a future change in >> generic hotplug framework. This would help detect such scenarious and help >> debug further. >> >> It also adds a validation function which scans entire boot memory and makes >> sure that early memory sections are online. This check is essential for the >> memory notifier to work properly as it cannot prevent boot memory offlining >> if they are not online to begin with. But this additional sanity check is >> enabled only with DEBUG_VM. > > Could you please split this in separate patches rather than having a > single one doing three somewhat related things? Sure, will do. > >> --- a/arch/arm64/kernel/setup.c >> +++ b/arch/arm64/kernel/setup.c >> @@ -376,6 +376,14 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p) >> "This indicates a broken bootloader or old kernel\n", >> boot_args[1], boot_args[2], boot_args[3]); >> } >> + >> + /* >> + * Register the memory notifier which will prevent boot >> + * memory offlining requests - early enough. But there >> + * should not be any actual offlinig request till memory >> + * block devices are initialized with memory_dev_init(). >> + */ >> + memory_hotremove_notifier(); > > Why can this not be an early_initcall()? As you said, memory_dev_init() > is called much later, after the SMP was initialised. This proposal moves memory_hotremove_notifier() to setup_arch() because it could and there is no harm in calling this too early than required for now. But in case generic MM sequence of events during memory init changes later, this notifier will still work. IIUC, the notifier chain registration can be called very early in the boot process without much problem. There are some precedence on other platforms. 1. arch/s390/mm/init.c - In device_initcall() via s390_cma_mem_init() 2. arch/s390/mm/setup.c - In setup_arch() via reserve_crashkernel() 3. arch/powerpc/platforms/pseries/cmm.c - In module_init() via cmm_init() 4. arch/powerpc/platforms/pseries/iommu.c - via iommu_init_early_pSeries() via pSeries_init() via pSeries_probe() aka ppc_md.porbe() via probe_machine() via setup_arch() > > You could even combine this with validate_bootmem_online_state() in a > single early_initcall() which, after checking, registers the notifier. > Yes, that will be definitely simpler but there might be still some value in having this registration in setup_arch() which guard against future generic MM changes while keeping it separate from the sanity check i.e validate_bootmem_online_state() which is enabled only with DEBUG_VM. But will combine both in early_initcall() with some name changes if that is preferred. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel