From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA154C433FE for ; Fri, 4 Dec 2020 01:18:06 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 11B62224BE for ; Fri, 4 Dec 2020 01:18:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 11B62224BE Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D6BAF6B0036; Thu, 3 Dec 2020 20:18:04 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CF4C26B005C; Thu, 3 Dec 2020 20:18:04 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B95CB6B0068; Thu, 3 Dec 2020 20:18:04 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0222.hostedemail.com [216.40.44.222]) by kanga.kvack.org (Postfix) with ESMTP id 9ED1E6B0036 for ; Thu, 3 Dec 2020 20:18:04 -0500 (EST) Received: from smtpin14.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 5D84F3632 for ; Fri, 4 Dec 2020 01:18:04 +0000 (UTC) X-FDA: 77553838488.14.queen55_6114eb2273c0 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin14.hostedemail.com (Postfix) with ESMTP id 426601822987A for ; Fri, 4 Dec 2020 01:18:04 +0000 (UTC) X-HE-Tag: queen55_6114eb2273c0 X-Filterd-Recvd-Size: 3515 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) by imf34.hostedemail.com (Postfix) with ESMTP for ; Fri, 4 Dec 2020 01:18:02 +0000 (UTC) IronPort-SDR: uyY5DCF4w/vbLj5d0lTc79Io42Elh3udbU199dEvnIRpU6jvL7sA2fywhYkwOn+0TPC4+tOVc9 MgL8JXDMrX2Q== X-IronPort-AV: E=McAfee;i="6000,8403,9824"; a="152558622" X-IronPort-AV: E=Sophos;i="5.78,390,1599548400"; d="scan'208";a="152558622" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Dec 2020 17:18:00 -0800 IronPort-SDR: 0QWTRh/KWtPJqTfzrdYK6wNaN4V21nzh+yWikdr6Y94Pi7DaIPKrGFGXAb1dP0g17DzLmJ8ULl yOrtRfqHMwCQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.78,390,1599548400"; d="scan'208";a="550744704" Received: from shbuild999.sh.intel.com (HELO localhost) ([10.239.147.98]) by orsmga005.jf.intel.com with ESMTP; 03 Dec 2020 17:17:58 -0800 Date: Fri, 4 Dec 2020 09:17:58 +0800 From: Feng Tang To: David Hildenbrand Cc: Paul Menzel , linux-mm@kvack.org, Arjan van de Ven Subject: Re: Deferred Memory Init: How to bring rest of memory online after limiting it with `mem=XG`? Message-ID: <20201204011758.GA29328@shbuild999.sh.intel.com> References: <808e493f-18a2-fe82-46b3-e44fe87e989c@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <808e493f-18a2-fe82-46b3-e44fe87e989c@redhat.com> User-Agent: Mutt/1.5.24 (2015-08-30) Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi David, On Thu, Dec 03, 2020 at 01:25:04PM +0100, David Hildenbrand wrote: > On 03.12.20 11:51, Paul Menzel wrote: > > Dear Feng, > >=20 > >=20 > > I am trying to reduce the startup time of Debian=E2=80=99s Linux 5.9.= 9 on a=20 > > Intel Kaby Lake system with 32 GB of memory (TUXEDO Book BU1406 (Clev= o=20 > > N240BU)). On your Linux Plumbers Conference 2019 slides of your talk=20 > > *Linux Kernel Fastboot On the Way* [1], you mention *Deferred Memory = Init*: > >=20 > >> Deferred Memory Init > >> > >> =E2=80=A2 8GB RAM=E2=80=99s initialization costs 100+ ms > >> =E2=80=A2 In early boot phase, we don=E2=80=99t need that much mem= ory > >> =E2=80=A2 Utilize the memory hotplug feature > >> =E2=80=A2 =E2=80=9Cmem=3D4096m=E2=80=9D in cmdline to only ini= t 2 GB > >> =E2=80=A2 Use systemd service to add rest memory in parallel >=20 > Uh, that sounds very wrong and flawed. >=20 > Even if you would be adding+onlining memory in parallel, memory > hotplug/onlining code runs strictly sequential. This does not work. >=20 > And I question this approach in general. >=20 > We do have deferred meminit in the kernel during boot that can > initialize memory in parallel. Yes, this is what can now use decently. The foil was written in 2019 and the work is done in 2018 with kernel 4.9~4.19 kernel, where this feature was not availabe. Interestingly, I called for the in-kernel deferred init in LPC, and in the LPC room, Pavel Tatashin told me that they were working on it, and it's great to see it is completed, which is really useful for optimizing kernel boot time. btw, the user space task to online the other memory was actually some script, like for i in x..y do echo online > /sys/devices/system/memory$i/state done Thanks, Feng >=20 > --=20 > Thanks, >=20 > David / dhildenb