From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD941C47082 for ; Tue, 8 Jun 2021 14:16:58 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 67EDA610C7 for ; Tue, 8 Jun 2021 14:16:58 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 67EDA610C7 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9BD646B006C; Tue, 8 Jun 2021 10:16:57 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 993536B006E; Tue, 8 Jun 2021 10:16:57 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85B946B0070; Tue, 8 Jun 2021 10:16:57 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0216.hostedemail.com [216.40.44.216]) by kanga.kvack.org (Postfix) with ESMTP id 526E56B006C for ; Tue, 8 Jun 2021 10:16:57 -0400 (EDT) Received: from smtpin27.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id E641A181AEF21 for ; Tue, 8 Jun 2021 14:16:56 +0000 (UTC) X-FDA: 78230758032.27.3A6E2E3 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by imf03.hostedemail.com (Postfix) with ESMTP id 05FE9C00CBE0 for ; Tue, 8 Jun 2021 14:16:52 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id B3CE260FE9; Tue, 8 Jun 2021 14:16:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1623161815; bh=gF49svYEd54M0WQUzTD4wVKyLxtm83atdq+n9wjbLUI=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=JTjhm3VxDHcISK0AaRrckS2Y7FKCPy6k+BgNSVfwvbDBNhsWQ8OYkZh19wPKWJiYL 4JZDIyk/QYRlwL+kHSYmiP/+btGGzCz1rJ0xVRwO4bxI2iQ+6VfTA0Spj6QJY6a9qY eDApnO0knwUYGN9CLzMGGYZWZfo6bcLfv+LUlHBnIyyfUl2GRQgMFygQ7PX/JkxNDt nPZiYKQtT91oiRoujWDgK2HgsTmFdwnxBTjo7hiKsxJOna5ODUbfh5ezX8iiMkZ5t4 e1wiHAd3cyQmoI2FO6nddlk5ubIdXGxXkaiQcC5XmURqxFjJ2+RN2HYFl/pCt/It13 H9bWeYUBm0JwA== Date: Tue, 8 Jun 2021 17:16:47 +0300 From: Mike Rapoport To: David Hildenbrand Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, Andrew Morton , Oscar Salvador , Michal Hocko , Mike Kravetz , Dave Hansen , Matthew Wilcox , Anshuman Khandual , Muchun Song , Pavel Tatashin , Jonathan Corbet , Stephen Rothwell , linux-doc@vger.kernel.org Subject: Re: [PATCH v1] memory-hotplug.rst: complete admin-guide overhaul Message-ID: References: <20210525102604.8770-1-david@redhat.com> <385d2bd0-8857-9d40-c8f9-c302f0b56e12@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <385d2bd0-8857-9d40-c8f9-c302f0b56e12@redhat.com> Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=JTjhm3Vx; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf03.hostedemail.com: domain of rppt@kernel.org designates 198.145.29.99 as permitted sender) smtp.mailfrom=rppt@kernel.org X-Rspamd-Server: rspam02 X-Stat-Signature: kmuxr6he76xa69q6jnecz4e81kxfaa76 X-Rspamd-Queue-Id: 05FE9C00CBE0 X-HE-Tag: 1623161812-55970 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi David, On Tue, Jun 08, 2021 at 02:05:50PM +0200, David Hildenbrand wrote: > Hi Mike, > > thansk for the review! ... > > > -Phases of memory hotplug > > > +Further, the basic memory hot(un)plug infrastructure in Linux is nowadays > > > +also used to expose PMEM, other performance-differentiated > > > > ^ persistent memory (PMEM) > > Just in case you've missed this one ^ ;-) > > > +memory and reserved memory regions as ordinary system RAM to Linux. > > > + > > > +Phases of Memory Hotplug > > > ------------------------ > > > -There are 2 phases in Memory Hotplug: > > > +Memory hotplug consists of two phases: > > > + > > > +(1) Adding the memory to Linux > > > +(2) Onlining memory blocks > > > - 1) Physical Memory Hotplug phase > > > - 2) Logical Memory Hotplug phase. > > > +In the first phase, metadata (such as the memmap) is allocated, page tables > > > +for the direct mapping are allocated and initialized, and memory blocks > > > > User/administrator should not care about memmap or direct map and these > > details are better suited for Documentation/vm but since we don't have it > > how about: > > > > ... metadata, such as the memory map and page tables for the direct map, > > are allocated and initialized, ... > > Admins will have to know/care about the "memmap" terminology, because we now > have features that use that name (for example, "memmap_on_memory") I should have split my comment in two, first would be that admins do not care about what exact metadata is there and this would have been more relevant to Documentationvm/vm/memory-hotplug.rst if we had one. And the second comment would be that I prefer to have things like memmap, pmem etc fully spelled out even if the short versions are common and understandable. > So I'll tweak it to > > ".. metadata, such as the memory map ("memmap") and page tables for the > direct map, > are allocated and initialized, ..." Works for me. ... > > > +increases memory offlining reliability; still, memory offlining can fail in > > > +some corner cases. > > > - % echo start_address_of_new_memory > /sys/devices/system/memory/probe > > > +Further, memory offlining might retry for a long time (or even forever), > > > +until aborted by the user. > > > -Then, [start_address_of_new_memory, start_address_of_new_memory + > > > -memory_block_size] memory range is hot-added. In this case, hotplug script is > > > -not called (in current implementation). You'll have to online memory by > > > -yourself. Please see :ref:`memory_hotplug_how_to_online_memory`. > > > +Offlining of a memory block can be triggered via:: > > > -Logical Memory hot-add phase > > > -============================ > > > + % echo offline > /sys/devices/system/memory/memoryXXX/state > > > -State of memory > > > ---------------- > > > +Or alternatively:: > > > -To see (online/offline) state of a memory block, read 'state' file:: > > > + % echo 0 > /sys/devices/system/memory/memoryXXX/online > > > - % cat /sys/device/system/memory/memoryXXX/state > > > +If offline succeeds, the state of the memory block is changed to be "offline". > > > +If it fails, an error will be returned by the kernel. > > > > I think elaborating here how the error is returned would be nice. > > I *think* it's returned via the system call that tries modifying the file. > > "If it fails, an error will be returned by the kernel via the systemcall > that triggered modifying of the respective file." I also think that write(2) to /sys/devices/system/memory/memoryXXX/online will fail. But the inner workings of system call, its return value and the ERRNO are probably not very interesting to a person that did echo 0 > /sys/devices/system/memory/memoryXXX/online Maybe something like If it fails, the state of the memory block will remain unchanged and the above command will fail. And maybe an example of how echo reports some unrelated error message :) > > > +Observing the State of Memory Blocks ... > > > -Now, a boot option for making a memory block which consists of migratable pages > > > -is supported. By specifying "kernelcore=" or "movablecore=" boot option, you can > > > -create ZONE_MOVABLE...a zone which is just used for movable pages. > > > -(See also Documentation/admin-guide/kernel-parameters.rst) > > > + For online memory blocks, ``DMA``, ``DMA32``, ``Normal``, > > > + ``Movable`` and ``none`` may be returned. ``none`` indicates > > > > Highmem? Or we don't support hotplug on 32 bits? > > We only support 64 bit: > > config MEMORY_HOTPLUG > ... > depends on 64BIT || BROKEN > > Worth a comment in the document "Introduction": > > "Linux only supports memory hot(un)plug on selected 64 bit architectures, > such as x86_64, aarch64, ppc64, s390x and ia64." ^ arm64 ? > I can spot that sh also enables it -- but I never even tested it or saw any > BUG reports related to it, so I'll not mention it for now explicitly in the > document. Agree. -- Sincerely yours, Mike.