From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0E70CC4361A for ; Fri, 4 Dec 2020 07:31:12 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 57A992255F for ; Fri, 4 Dec 2020 07:31:10 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 57A992255F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=molgen.mpg.de Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6F5A76B0036; Fri, 4 Dec 2020 02:31:09 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 680738D0001; Fri, 4 Dec 2020 02:31:09 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 546276B0068; Fri, 4 Dec 2020 02:31:09 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0100.hostedemail.com [216.40.44.100]) by kanga.kvack.org (Postfix) with ESMTP id 3626E6B0036 for ; Fri, 4 Dec 2020 02:31:09 -0500 (EST) Received: from smtpin13.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id ED7ED249D for ; Fri, 4 Dec 2020 07:31:08 +0000 (UTC) X-FDA: 77554778616.13.shock70_5500246273c2 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin13.hostedemail.com (Postfix) with ESMTP id CA6BE18140B60 for ; Fri, 4 Dec 2020 07:31:08 +0000 (UTC) X-HE-Tag: shock70_5500246273c2 X-Filterd-Recvd-Size: 5708 Received: from mx1.molgen.mpg.de (mx3.molgen.mpg.de [141.14.17.11]) by imf09.hostedemail.com (Postfix) with ESMTP for ; Fri, 4 Dec 2020 07:31:08 +0000 (UTC) Received: from [192.168.0.2] (ip5f5af1d8.dynamic.kabel-deutschland.de [95.90.241.216]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: pmenzel) by mx.molgen.mpg.de (Postfix) with ESMTPSA id B7D872064786F; Fri, 4 Dec 2020 08:31:05 +0100 (CET) Subject: Re: Deferred Memory Init: How to bring rest of memory online after limiting it with `mem=XG`? To: Daniel Jordan , David Hildenbrand Cc: linux-mm@kvack.org, Arjan van de Ven , Feng Tang References: <808e493f-18a2-fe82-46b3-e44fe87e989c@redhat.com> <8a395c1a-764b-9d82-0a12-b1a9254f8d53@molgen.mpg.de> <7692e410-886e-a1bb-4158-e02305fcd013@redhat.com> <87v9disj2s.fsf@oracle.com> From: Paul Menzel Message-ID: <75c699e1-0f2f-dd83-5f1f-ebc0753ad762@molgen.mpg.de> Date: Fri, 4 Dec 2020 08:31:05 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.5.1 MIME-Version: 1.0 In-Reply-To: <87v9disj2s.fsf@oracle.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Dear David, dear Daniel, Am 03.12.20 um 21:58 schrieb Daniel Jordan: > David Hildenbrand writes: >=20 >> On 03.12.20 13:52, Paul Menzel wrote: >>> Am 03.12.20 um 13:25 schrieb David Hildenbrand: >>>> On 03.12.20 11:51, Paul Menzel wrote: >>> >>>>> I am trying to reduce the startup time of Debian=E2=80=99s Linux 5.= 9.9 on a >>>>> Intel Kaby Lake system with 32 GB of memory (TUXEDO Book BU1406 (Cl= evo >>>>> N240BU)). >>> >>> [=E2=80=A6] >>> >>>> We do have deferred meminit in the kernel during boot that can >>>> initialize memory in parallel. >>> >>> Is that used automatically, or do I need to activate it somehow? >> >> If your kernel is compiled with >> >> CONFIG_DEFERRED_STRUCT_PAGE_INIT >> >> it should be enabled automatically. [=E2=80=A6] Yes, in Debian=E2=80=99s Linux kernel configuration, that option is selec= ted. > If it is enabled, what does >=20 > dmesg | grep 'deferred pages' >=20 > give you? $ grep 'deferred pages' dmesg-full.txt [ 0.140199] node 0 deferred pages initialised in 40ms $ grep 'deferred pages' dmesg-2g.txt [ 0.077892] node 0 deferred pages initialised in 4ms > And assuming you're running systemd, what does systemd-analyze show you= ? Please find it below. In my experience, unpacking the initrd is also=20 good measurement point, and the time to write protect the kernel=20 read-only data. Without `mem=3D` (32 GB): $ dmesg [=E2=80=A6] [ 0.000000] Command line: BOOT_IMAGE=3D/boot/vmlinuz-5.9.0-3-amd6= 4=20 root=3DUUID=3Dd23ce27e-5c5c-45fb-bfa8-79a87caff13f ro quiet=20 cryptomgr.notests ipv6.disable=3D1 log_buf_len=3D2M initcall_debug=20 init=3D/lib/systemd/systemd-bootchart [=E2=80=A6] [ 0.266513] Trying to unpack rootfs image as initramfs... [ 0.275829] Freeing initrd memory: 4468K [=E2=80=A6] [ 0.295890] Freeing unused kernel image (initmem) memory: 1640K [ 0.335585] Write protecting the kernel read-only data: 18432k [ 0.336261] Freeing unused kernel image (text/rodata gap)=20 memory: 2044K [ 0.336406] Freeing unused kernel image (rodata/data gap)=20 memory: 292K [ 0.392213] x86/mm: Checked W+X mappings: passed, no W+X pages=20 found. [ 0.392213] x86/mm: Checking user space page tables [ 0.432697] x86/mm: Checked W+X mappings: passed, no W+X pages=20 found. [ 0.432701] Run /init as init process [=E2=80=A6] $ systemd-analyze time Startup finished in 3.792s (firmware) + 4.116s (loader) + 767ms=20 (kernel) + 1.414s (userspace) =3D 10.091s graphical.target reached after 1.401s in userspace With mem=3D2G $ dmesg [=E2=80=A6] [ 0.000000] Command line: BOOT_IMAGE=3D/boot/vmlinuz-5.9.0-3-amd6= 4=20 root=3DUUID=3Dd23ce27e-5c5c-45fb-bfa8-79a87caff13f ro quiet=20 cryptomgr.notests ipv6.disable=3D1 log_buf_len=3D2M initcall_debug=20 init=3D/lib/systemd/systemd-bootchart mem=3D2G [=E2=80=A6] [ 0.199720] Trying to unpack rootfs image as initramfs... [ 0.209058] Freeing initrd memory: 4468K [=E2=80=A6] [ 0.227433] Freeing unused kernel image (initmem) memory: 1640K [ 0.253494] Write protecting the kernel read-only data: 18432k [ 0.253898] Freeing unused kernel image (text/rodata gap)=20 memory: 2044K [ 0.253987] Freeing unused kernel image (rodata/data gap)=20 memory: 292K [ 0.297996] x86/mm: Checked W+X mappings: passed, no W+X pages=20 found. [ 0.297997] x86/mm: Checking user space page tables [ 0.338337] x86/mm: Checked W+X mappings: passed, no W+X pages=20 found. [ 0.338341] Run /init as init process $ sytsemd-analyze time Startup finished in 650ms (kernel) + 3.009s (userspace) =3D 3.659s graphical.target reached after 2.979s in userspace (No idea why the firmware and loader timestamps are sometimes not=20 available.) So the different memory sizes result in an almost 100 ms difference=20 during start-up, and initializing 32 GB results in a 30 percent longer=20 boottime up to starting the init process. (Unfortunately userspace and=20 several drivers later on take also quite some time.) If you have ideas how to reduce the boot time of the full 32 GB, that=E2=80= =99d=20 be great. Kind regards, Paul