From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AE247C433ED for ; Thu, 6 May 2021 15:26:52 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 4D66961406 for ; Thu, 6 May 2021 15:26:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4D66961406 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=sent.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id BBEAC6B006C; Thu, 6 May 2021 11:26:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B95F26B006E; Thu, 6 May 2021 11:26:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9E9146B0070; Thu, 6 May 2021 11:26:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0116.hostedemail.com [216.40.44.116]) by kanga.kvack.org (Postfix) with ESMTP id 825766B006C for ; Thu, 6 May 2021 11:26:51 -0400 (EDT) Received: from smtpin32.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 2EE46181AEF07 for ; Thu, 6 May 2021 15:26:51 +0000 (UTC) X-FDA: 78111183822.32.E52882F Received: from new4-smtp.messagingengine.com (new4-smtp.messagingengine.com [66.111.4.230]) by imf12.hostedemail.com (Postfix) with ESMTP id 980A7135 for ; Thu, 6 May 2021 15:26:36 +0000 (UTC) Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailnew.nyi.internal (Postfix) with ESMTP id 42165580B72; Thu, 6 May 2021 11:26:50 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Thu, 06 May 2021 11:26:50 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=from :to:cc:subject:date:message-id:reply-to:mime-version :content-transfer-encoding; s=fm3; bh=idfxFi2niIlXphs1rSRpisMaNe RwPzwfRsm8hAUYBvQ=; b=hdYJWH8eNG6Nd+MRc3r283kN9IufmtoATaqfCU96YH ZKyLzDOs9mGhZSlD83jeL7H3DOqnFDCXn/VXD229UiHg68Vz+1cHYe64OVsP7fXV 7i456IoTE2D8mIiUzaHreIvLE4WMfptERGMoBchm/EYEioelxClLvi0cEjk2Z8se Arvfi8vRUcvLA33pxjnXxQmlCVsyXNryd5LfhOBp+3pZvqI36t6IK+pdROusiIec gXu5SgWTBHSw0qxydioj5F1Ec3AtLzDDc5vIue2gJTGi+JN2UwBs88PaeFZ6CQwn 7QhhXtghvdNckXVzjjQVUrQkB4SzeptEzI7DHQV/sbdw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:date:from :message-id:mime-version:reply-to:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm2; bh=idfxFi 2niIlXphs1rSRpisMaNeRwPzwfRsm8hAUYBvQ=; b=qf4gh/OodsbTyJHDo4/xMy XWnV10CwU190E3R4EbQH9v7/GANI/SHKBnKoNo+YtkMbM4Vnc0BA5STeVvOx5JAk QbOEJpYSN6+uvVxGfYHFJ7L3SvqPtmevmMEDxN1/w8HICPWdYg8gr/6WF1OWeC4H x2aX6/hzkjEnSGvUlD4WKg8F33YuV3SGpHhpJSiE/1iJqiNVsfPZAdb84Jetu6qY XNunEtvZhRhT1qIed+Qn+KrN2qp8++eFknjSTrCea322qDnqF8r0r698iXZ52kMQ De9S9QiodAWedpRqKM1DYX+rI3FeGwLbUo0bV5DgW4WFjPJIr4TakXPY/JZ5BjHg == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduledrvdegtddgkeelucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvufffkfforhgggfestdhqredtredttdenucfhrhhomhepkghiucgjrghn uceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepleetvddtvd dttdetffdvkeehffdtjefhkedtheffjeehvdejueektedukedufeffnecukfhppeefhedr udeigedrvdegiedrfeegnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepiihirdihrghnsehsvghnthdrtghomh X-ME-Proxy: Received: from Threadripper.local (ec2-35-164-246-34.us-west-2.compute.amazonaws.com [35.164.246.34]) by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 6 May 2021 11:26:44 -0400 (EDT) From: Zi Yan To: David Hildenbrand , Oscar Salvador Cc: Michael Ellerman , Benjamin Herrenschmidt , Thomas Gleixner , x86@kernel.org, Andy Lutomirski , "Rafael J . Wysocki" , Andrew Morton , Mike Rapoport , Anshuman Khandual , Michal Hocko , Dan Williams , Wei Yang , linux-ia64@vger.kernel.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-mm@kvack.org, Zi Yan Subject: [RFC PATCH 0/7] Memory hotplug/hotremove at subsection size Date: Thu, 6 May 2021 11:26:16 -0400 Message-Id: <20210506152623.178731-1-zi.yan@sent.com> X-Mailer: git-send-email 2.30.2 Reply-To: Zi Yan MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=sent.com header.s=fm3 header.b=hdYJWH8e; dkim=pass header.d=messagingengine.com header.s=fm2 header.b="qf4gh/Oo"; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf12.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.230 as permitted sender) smtp.mailfrom=zi.yan@sent.com X-Rspamd-Server: rspam03 X-Stat-Signature: 6hdq3saenaqaf47qx8zn56bbmcasix5r X-Rspamd-Queue-Id: 980A7135 Received-SPF: none (sent.com>: No applicable sender policy available) receiver=imf12; identity=mailfrom; envelope-from=""; helo=new4-smtp.messagingengine.com; client-ip=66.111.4.230 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1620314796-158553 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zi Yan Hi all, This patchset tries to remove the restriction on memory hotplug/hotremove granularity, which is always greater or equal to memory section size[1]. With the patchset, kernel is able to online/offline memory at a size indepe= ndent of memory section size, as small as 2MB (the subsection size). The motivation is to increase MAX_ORDER of the buddy allocator and pageblock size without increasing memory hotplug/hotremove granularity at the same ti= me, so that the kernel can allocator 1GB pages using buddy allocator and utiliz= es existing pageblock based anti-fragmentation, paving the road for 1GB THP support[2]. The patchset utilizes the existing subsection support[3] and changes the section size alignment checks to subsection size alignment checks. There are also changes to pageblock code to support partial pageblocks, when pageblock size is increased along with MAX_ORDER. Increasing pageblock size can enable kernel to utilize existing anti-fragmentation mechanism for gigantic page allocations. The last patch increases SECTION_SIZE_BITS to demonstrate the use of memory hotplug/hotremove subsection, but is not intended to be merged as is. It is there in case one wants to try this out and will be removed during the final submission. Feel free to give suggestions and comments. I am looking forward to your feedback. Thanks. Zi Yan (7): mm: sparse: set/clear subsection bitmap when pages are onlined/offlined. mm: set pageblock_order to the max of HUGETLB_PAGE_ORDER and MAX_ORDER-1 mm: memory_hotplug: decouple memory_block size with section size. mm: pageblock: allow set/unset migratetype for partial pageblock mm: memory_hotplug, sparse: enable memory hotplug/hotremove subsections arch: x86: no MAX_ORDER exceeds SECTION_SIZE check for 32bit vdso. [not for merge] mm: increase SECTION_SIZE_BITS to 31 arch/ia64/Kconfig | 1 - arch/powerpc/Kconfig | 1 - arch/x86/Kconfig | 15 +++ arch/x86/entry/vdso/Makefile | 1 + arch/x86/include/asm/sparsemem.h | 2 +- drivers/base/memory.c | 176 +++++++++++++++---------------- drivers/base/node.c | 2 +- include/linux/memory.h | 8 +- include/linux/mmzone.h | 2 + include/linux/page-isolation.h | 8 +- include/linux/pageblock-flags.h | 9 -- mm/Kconfig | 7 -- mm/memory_hotplug.c | 22 ++-- mm/page_alloc.c | 40 ++++--- mm/page_isolation.c | 30 +++--- mm/sparse.c | 55 ++++++++-- 16 files changed, 219 insertions(+), 160 deletions(-) --=20 2.30.2