From: Ryan Roberts <ryan.roberts@arm.com> To: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, Ard Biesheuvel <ardb@kernel.org>, David Hildenbrand <david@redhat.com>, Donald Dutile <ddutile@redhat.com>, Eric Chanudet <echanude@redhat.com> Cc: Ryan Roberts <ryan.roberts@arm.com>, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 0/3] Speed up boot with faster linear map creation Date: Tue, 26 Mar 2024 10:14:45 +0000 [thread overview] Message-ID: <20240326101448.3453626-1-ryan.roberts@arm.com> (raw) Hi All, It turns out that creating the linear map can take a significant proportion of the total boot time, especially when rodata=full. And a large portion of the time it takes to create the linear map is issuing TLBIs. This series reworks the kernel pgtable generation code to significantly reduce the number of TLBIs. See each patch for details. The below shows the execution time of map_mem() across a couple of different systems with different RAM configurations. We measure after applying each patch and show the improvement relative to base (v6.9-rc1): | Apple M2 VM | Ampere Altra| Ampere Altra| Ampere Altra | VM, 16G | VM, 64G | VM, 256G | Metal, 512G ---------------|-------------|-------------|-------------|------------- | ms (%) | ms (%) | ms (%) | ms (%) ---------------|-------------|-------------|-------------|------------- base | 151 (0%) | 2191 (0%) | 8990 (0%) | 17443 (0%) no-cont-remap | 77 (-49%) | 429 (-80%) | 1753 (-80%) | 3796 (-78%) no-alloc-remap | 77 (-49%) | 375 (-83%) | 1532 (-83%) | 3366 (-81%) lazy-unmap | 63 (-58%) | 330 (-85%) | 1312 (-85%) | 2929 (-83%) This series applies on top of v6.9-rc1. All mm selftests pass. I haven't yet tested all VA size configs (although I don't anticipate any issues); I'll do this as part of followup. Thanks, Ryan Ryan Roberts (3): arm64: mm: Don't remap pgtables per- cont(pte|pmd) block arm64: mm: Don't remap pgtables for allocate vs populate arm64: mm: Lazily clear pte table mappings from fixmap arch/arm64/include/asm/fixmap.h | 5 +- arch/arm64/include/asm/mmu.h | 8 + arch/arm64/include/asm/pgtable.h | 4 - arch/arm64/kernel/cpufeature.c | 10 +- arch/arm64/mm/fixmap.c | 11 + arch/arm64/mm/mmu.c | 364 +++++++++++++++++++++++-------- include/linux/pgtable.h | 8 + 7 files changed, 307 insertions(+), 103 deletions(-) -- 2.25.1
WARNING: multiple messages have this Message-ID (diff)
From: Ryan Roberts <ryan.roberts@arm.com> To: Catalin Marinas <catalin.marinas@arm.com>, Will Deacon <will@kernel.org>, Mark Rutland <mark.rutland@arm.com>, Ard Biesheuvel <ardb@kernel.org>, David Hildenbrand <david@redhat.com>, Donald Dutile <ddutile@redhat.com>, Eric Chanudet <echanude@redhat.com> Cc: Ryan Roberts <ryan.roberts@arm.com>, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH v1 0/3] Speed up boot with faster linear map creation Date: Tue, 26 Mar 2024 10:14:45 +0000 [thread overview] Message-ID: <20240326101448.3453626-1-ryan.roberts@arm.com> (raw) Hi All, It turns out that creating the linear map can take a significant proportion of the total boot time, especially when rodata=full. And a large portion of the time it takes to create the linear map is issuing TLBIs. This series reworks the kernel pgtable generation code to significantly reduce the number of TLBIs. See each patch for details. The below shows the execution time of map_mem() across a couple of different systems with different RAM configurations. We measure after applying each patch and show the improvement relative to base (v6.9-rc1): | Apple M2 VM | Ampere Altra| Ampere Altra| Ampere Altra | VM, 16G | VM, 64G | VM, 256G | Metal, 512G ---------------|-------------|-------------|-------------|------------- | ms (%) | ms (%) | ms (%) | ms (%) ---------------|-------------|-------------|-------------|------------- base | 151 (0%) | 2191 (0%) | 8990 (0%) | 17443 (0%) no-cont-remap | 77 (-49%) | 429 (-80%) | 1753 (-80%) | 3796 (-78%) no-alloc-remap | 77 (-49%) | 375 (-83%) | 1532 (-83%) | 3366 (-81%) lazy-unmap | 63 (-58%) | 330 (-85%) | 1312 (-85%) | 2929 (-83%) This series applies on top of v6.9-rc1. All mm selftests pass. I haven't yet tested all VA size configs (although I don't anticipate any issues); I'll do this as part of followup. Thanks, Ryan Ryan Roberts (3): arm64: mm: Don't remap pgtables per- cont(pte|pmd) block arm64: mm: Don't remap pgtables for allocate vs populate arm64: mm: Lazily clear pte table mappings from fixmap arch/arm64/include/asm/fixmap.h | 5 +- arch/arm64/include/asm/mmu.h | 8 + arch/arm64/include/asm/pgtable.h | 4 - arch/arm64/kernel/cpufeature.c | 10 +- arch/arm64/mm/fixmap.c | 11 + arch/arm64/mm/mmu.c | 364 +++++++++++++++++++++++-------- include/linux/pgtable.h | 8 + 7 files changed, 307 insertions(+), 103 deletions(-) -- 2.25.1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next reply other threads:[~2024-03-26 10:15 UTC|newest] Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top 2024-03-26 10:14 Ryan Roberts [this message] 2024-03-26 10:14 ` [PATCH v1 0/3] Speed up boot with faster linear map creation Ryan Roberts 2024-03-26 10:14 ` [PATCH v1 1/3] arm64: mm: Don't remap pgtables per- cont(pte|pmd) block Ryan Roberts 2024-03-26 10:14 ` Ryan Roberts 2024-03-26 10:14 ` [PATCH v1 2/3] arm64: mm: Don't remap pgtables for allocate vs populate Ryan Roberts 2024-03-26 10:14 ` Ryan Roberts 2024-03-27 2:05 ` kernel test robot 2024-03-27 2:05 ` kernel test robot 2024-03-26 10:14 ` [PATCH v1 3/3] arm64: mm: Lazily clear pte table mappings from fixmap Ryan Roberts 2024-03-26 10:14 ` Ryan Roberts 2024-03-27 10:09 ` [PATCH v1 0/3] Speed up boot with faster linear map creation Ard Biesheuvel 2024-03-27 10:09 ` Ard Biesheuvel 2024-03-27 10:43 ` Ryan Roberts 2024-03-27 10:43 ` Ryan Roberts 2024-03-27 13:36 ` Ard Biesheuvel 2024-03-27 13:36 ` Ard Biesheuvel 2024-03-27 15:01 ` Ryan Roberts 2024-03-27 15:01 ` Ryan Roberts 2024-03-27 15:57 ` Ard Biesheuvel 2024-03-27 15:57 ` Ard Biesheuvel 2024-03-27 16:11 ` Ryan Roberts 2024-03-27 16:11 ` Ryan Roberts 2024-03-27 11:06 ` Itaru Kitayama 2024-03-27 11:06 ` Itaru Kitayama 2024-03-27 11:10 ` Ryan Roberts 2024-03-27 11:10 ` Ryan Roberts 2024-03-27 19:07 ` [PATCH v1] arm64: mm: Batch dsb and isb when populating pgtables Ryan Roberts 2024-03-27 19:07 ` Ryan Roberts 2024-03-28 7:23 ` Ard Biesheuvel 2024-03-28 7:23 ` Ard Biesheuvel 2024-03-28 8:45 ` Ryan Roberts 2024-03-28 8:45 ` Ryan Roberts 2024-03-28 8:56 ` Ard Biesheuvel 2024-03-28 8:56 ` Ard Biesheuvel 2024-03-27 19:12 ` [PATCH v1 0/3] Speed up boot with faster linear map creation Ryan Roberts 2024-03-27 19:12 ` Ryan Roberts 2024-03-28 23:08 ` Eric Chanudet 2024-03-28 23:08 ` Eric Chanudet
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20240326101448.3453626-1-ryan.roberts@arm.com \ --to=ryan.roberts@arm.com \ --cc=ardb@kernel.org \ --cc=catalin.marinas@arm.com \ --cc=david@redhat.com \ --cc=ddutile@redhat.com \ --cc=echanude@redhat.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mark.rutland@arm.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.