From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 92D19C83009 for ; Wed, 29 Apr 2020 16:08:38 +0000 (UTC) Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6920E2072A for ; Wed, 29 Apr 2020 16:08:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QZPrrd8z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6920E2072A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvdimm-bounces@lists.01.org Received: from ml01.vlan13.01.org (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 637DA1006F366; Wed, 29 Apr 2020 09:07:33 -0700 (PDT) Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=205.139.110.120; helo=us-smtp-1.mimecast.com; envelope-from=david@redhat.com; receiver= Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 5AA3810083E5B for ; Wed, 29 Apr 2020 09:07:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1588176513; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=RqcnavPFirGVOTXHhjY5q5DM9FLFgis6bxz7RvWdVJE=; b=QZPrrd8zNNebgZXnzhIAYA2aP84wbXzXTC4SfYYO6rv5ju2/aeWwsXYRoUwkzVKNDRurdJ YxExOEc0hr+rJQkkFgPUr+ph31cSB3GXU2c+/TgswYJtJaAdE4hLZ9KBhaCdgY7s0ei/vk ZW/gaMqEpwiuaxXcYVMdRLbEU5DvYnE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-480-F36vCcGRMVu93iH7TXheIg-1; Wed, 29 Apr 2020 12:08:26 -0400 X-MC-Unique: F36vCcGRMVu93iH7TXheIg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E4E0C1895A28; Wed, 29 Apr 2020 16:08:21 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-55.ams2.redhat.com [10.36.114.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id BAF7C605FB; Wed, 29 Apr 2020 16:08:07 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Subject: [PATCH v1 0/3] mm/memory_hotplug: Make virtio-mem play nicely with kexec-tools Date: Wed, 29 Apr 2020 18:08:00 +0200 Message-Id: <20200429160803.109056-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Message-ID-Hash: UXVK73GGRCZD7JMMO2NG23PEZMNF76DL X-Message-ID-Hash: UXVK73GGRCZD7JMMO2NG23PEZMNF76DL X-MailFrom: david@redhat.com X-Mailman-Rule-Hits: nonmember-moderation X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation CC: linux-mm@kvack.org, virtio-dev@lists.oasis-open.org, virtualization@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, linux-acpi@vger.kernel.org, linux-nvdimm@lists.01.org, linux-hyperv@vger.kernel.org, linux-s390@vger.kernel.org, xen-devel@lists.xenproject.org, Michal Hocko , Andrew Morton , "Michael S . Tsirkin" , David Hildenbrand , Baoquan He , Benjamin Herrenschmidt , Boris Ostrovsky , Christian Borntraeger , Eric Biederman , Greg Kroah-Hartman , Haiyang Zhang , Heiko Carstens , Jason Wang , Juergen Gross , "K. Y. Srinivasan" , Len Brown , Leonardo Bras , Michael Ellerman , Michal H ocko , Nathan Lynch , Oscar Salvador , Pankaj Gupta , Paul Mackerras , Pingfan Liu , "Rafael J. Wysocki" , Stefano Stabellini , Stephen Hemminger , Thomas Gleixner , Vasily Gorbik , Wei Liu , Wei Yang X-Mailman-Version: 3.1.1 Precedence: list List-Id: "Linux-nvdimm developer list." Archived-At: List-Archive: List-Help: List-Post: List-Subscribe: List-Unsubscribe: Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit This series is based on [1]: [PATCH v2 00/10] virtio-mem: paravirtualized memory That will hopefull get picked up soon, rebased to -next. The following patches were reverted from -next [2]: [PATCH 0/3] kexec/memory_hotplug: Prevent removal and accidental use As discussed in that thread, they should be reverted from -next already. In theory, if people agree, we could take the first two patches via the -mm tree now and the last (virtio-mem) patch via MST's tree once picking up virtio-mem. No strong feelings. Memory added by virtio-mem is special and might contain logical holes, especially after memory unplug, but also when adding memory in sub-section size. While memory in these holes can usually be read, that memory should not be touched. virtio-mem managed device memory is never exposed via any firmware memmap (esp., e820). The device driver will request to plug memory from the hypervisor and add it to Linux. On a cold start, all memory is unplugged, and the guest driver will first request to plug memory from the hypervisor, to then add it to Linux. After a reboot, all memory will get unplugged (except in rare, special cases). In case the device driver comes up and detects that some memory is still plugged after a reboot, it will manually request to unplug all memory from the hypervisor first - to then request to plug memory from the hypervisor and add to Linux. This is essentially a defragmentation step, where all logical holes are removed. As the device driver is responsible for detecting, adding and managing that memory, also kexec should treat it like that. It is special. We need a way to teach kexec-tools to not add that memory to the fixed-up firmware memmap, to not place kexec images onto this memory, but still allow kdump to dump it. Add a flag to tell memory hotplug code to not create /sys/firmware/memmap entries and to indicate it via "System RAM (driver managed)" in /proc/iomem. Before this series, kexec_file_load() already did the right thing (for virtio-mem) by not adding that memory to the fixed-up firmware memmap and letting the device driver handle it. With this series, also kexec_load() - which relies on user space to provide a fixed up firmware memmap - does the right thing with virtio-mem memory. When the virtio-mem device driver(s) come up, they will request to unplug all memory from the hypervisor first (esp. defragment), to then request to plug consecutive memory ranges from the hypervisor, and add them to Linux - just like on a reboot where we still have memory plugged. [1] https://lore.kernel.org/r/20200311171422.10484-1-david@redhat.com/ [2] https://lore.kernel.org/r/20200326180730.4754-1-james.morse@arm.com David Hildenbrand (3): mm/memory_hotplug: Prepare passing flags to add_memory() and friends mm/memory_hotplug: Introduce MHP_DRIVER_MANAGED virtio-mem: Add memory with MHP_DRIVER_MANAGED arch/powerpc/platforms/powernv/memtrace.c | 2 +- .../platforms/pseries/hotplug-memory.c | 2 +- drivers/acpi/acpi_memhotplug.c | 2 +- drivers/base/memory.c | 2 +- drivers/dax/kmem.c | 2 +- drivers/hv/hv_balloon.c | 2 +- drivers/s390/char/sclp_cmd.c | 2 +- drivers/virtio/virtio_mem.c | 3 +- drivers/xen/balloon.c | 2 +- include/linux/memory_hotplug.h | 15 +++++++-- mm/memory_hotplug.c | 31 +++++++++++++------ 11 files changed, 44 insertions(+), 21 deletions(-) -- 2.25.3 _______________________________________________ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-leave@lists.01.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.9 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1920DC83000 for ; Wed, 29 Apr 2020 16:08:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E5D2E21D79 for ; Wed, 29 Apr 2020 16:08:36 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QZPrrd8z" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726519AbgD2QIg (ORCPT ); Wed, 29 Apr 2020 12:08:36 -0400 Received: from us-smtp-1.mimecast.com ([205.139.110.61]:29774 "EHLO us-smtp-delivery-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726476AbgD2QIf (ORCPT ); Wed, 29 Apr 2020 12:08:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1588176513; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=RqcnavPFirGVOTXHhjY5q5DM9FLFgis6bxz7RvWdVJE=; b=QZPrrd8zNNebgZXnzhIAYA2aP84wbXzXTC4SfYYO6rv5ju2/aeWwsXYRoUwkzVKNDRurdJ YxExOEc0hr+rJQkkFgPUr+ph31cSB3GXU2c+/TgswYJtJaAdE4hLZ9KBhaCdgY7s0ei/vk ZW/gaMqEpwiuaxXcYVMdRLbEU5DvYnE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-480-F36vCcGRMVu93iH7TXheIg-1; Wed, 29 Apr 2020 12:08:26 -0400 X-MC-Unique: F36vCcGRMVu93iH7TXheIg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E4E0C1895A28; Wed, 29 Apr 2020 16:08:21 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-55.ams2.redhat.com [10.36.114.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id BAF7C605FB; Wed, 29 Apr 2020 16:08:07 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, virtio-dev@lists.oasis-open.org, virtualization@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, linux-acpi@vger.kernel.org, linux-nvdimm@lists.01.org, linux-hyperv@vger.kernel.org, linux-s390@vger.kernel.org, xen-devel@lists.xenproject.org, Michal Hocko , Andrew Morton , "Michael S . Tsirkin" , David Hildenbrand , Baoquan He , Benjamin Herrenschmidt , Boris Ostrovsky , Christian Borntraeger , Dan Williams , Dave Jiang , Eric Biederman , Greg Kroah-Hartman , Haiyang Zhang , Heiko Carstens , Jason Wang , Juergen Gross , "K. Y. Srinivasan" , Len Brown , Leonardo Bras , Michael Ellerman , Michal Hocko , Nathan Lynch , Oscar Salvador , Pankaj Gupta , Paul Mackerras , Pingfan Liu , "Rafael J. Wysocki" , Stefano Stabellini , Stephen Hemminger , Thomas Gleixner , Vasily Gorbik , Vishal Verma , Wei Liu , Wei Yang Subject: [PATCH v1 0/3] mm/memory_hotplug: Make virtio-mem play nicely with kexec-tools Date: Wed, 29 Apr 2020 18:08:00 +0200 Message-Id: <20200429160803.109056-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Content-Transfer-Encoding: quoted-printable Sender: linux-acpi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-acpi@vger.kernel.org This series is based on [1]: [PATCH v2 00/10] virtio-mem: paravirtualized memory That will hopefull get picked up soon, rebased to -next. The following patches were reverted from -next [2]: [PATCH 0/3] kexec/memory_hotplug: Prevent removal and accidental use As discussed in that thread, they should be reverted from -next already. In theory, if people agree, we could take the first two patches via the -mm tree now and the last (virtio-mem) patch via MST's tree once picking = up virtio-mem. No strong feelings. Memory added by virtio-mem is special and might contain logical holes, especially after memory unplug, but also when adding memory in sub-section size. While memory in these holes can usually be read, that memory should not be touched. virtio-mem managed device memory is never exposed via any firmware memmap (esp., e820). The device driver will request to plug memory from the hypervisor and add it to Linux. On a cold start, all memory is unplugged, and the guest driver will first request to plug memory from the hypervisor, to then add it to Linux. Afte= r a reboot, all memory will get unplugged (except in rare, special cases). = In case the device driver comes up and detects that some memory is still plugged after a reboot, it will manually request to unplug all memory fro= m the hypervisor first - to then request to plug memory from the hypervisor and add to Linux. This is essentially a defragmentation step, where all logical holes are removed. As the device driver is responsible for detecting, adding and managing th= at memory, also kexec should treat it like that. It is special. We need a wa= y to teach kexec-tools to not add that memory to the fixed-up firmware memmap, to not place kexec images onto this memory, but still allow kdump to dump it. Add a flag to tell memory hotplug code to not create /sys/firmware/memmap entries and to indicate it via "System RAM (driver managed)" in /proc/iomem. Before this series, kexec_file_load() already did the right thing (for virtio-mem) by not adding that memory to the fixed-up firmware memmap and letting the device driver handle it. With this series, also kexec_load() = - which relies on user space to provide a fixed up firmware memmap - does the right thing with virtio-mem memory. When the virtio-mem device driver(s) come up, they will request to unplug all memory from the hypervisor first (esp. defragment), to then request t= o plug consecutive memory ranges from the hypervisor, and add them to Linux - just like on a reboot where we still have memory plugged. [1] https://lore.kernel.org/r/20200311171422.10484-1-david@redhat.com/ [2] https://lore.kernel.org/r/20200326180730.4754-1-james.morse@arm.com David Hildenbrand (3): mm/memory_hotplug: Prepare passing flags to add_memory() and friends mm/memory_hotplug: Introduce MHP_DRIVER_MANAGED virtio-mem: Add memory with MHP_DRIVER_MANAGED arch/powerpc/platforms/powernv/memtrace.c | 2 +- .../platforms/pseries/hotplug-memory.c | 2 +- drivers/acpi/acpi_memhotplug.c | 2 +- drivers/base/memory.c | 2 +- drivers/dax/kmem.c | 2 +- drivers/hv/hv_balloon.c | 2 +- drivers/s390/char/sclp_cmd.c | 2 +- drivers/virtio/virtio_mem.c | 3 +- drivers/xen/balloon.c | 2 +- include/linux/memory_hotplug.h | 15 +++++++-- mm/memory_hotplug.c | 31 +++++++++++++------ 11 files changed, 44 insertions(+), 21 deletions(-) --=20 2.25.3 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA2DCC83000 for ; Wed, 29 Apr 2020 16:13:37 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1ADE52072A for ; Wed, 29 Apr 2020 16:13:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QZPrrd8z"; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QZPrrd8z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1ADE52072A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 49C3Tp00GJzDrCC for ; Thu, 30 Apr 2020 02:13:34 +1000 (AEST) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=redhat.com (client-ip=205.139.110.61; helo=us-smtp-delivery-1.mimecast.com; envelope-from=david@redhat.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: lists.ozlabs.org; dkim=pass (1024-bit key; unprotected) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=QZPrrd8z; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.a=rsa-sha256 header.s=mimecast20190719 header.b=QZPrrd8z; dkim-atps=neutral Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 49C3N523bXzDrB2 for ; Thu, 30 Apr 2020 02:08:36 +1000 (AEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1588176513; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=RqcnavPFirGVOTXHhjY5q5DM9FLFgis6bxz7RvWdVJE=; b=QZPrrd8zNNebgZXnzhIAYA2aP84wbXzXTC4SfYYO6rv5ju2/aeWwsXYRoUwkzVKNDRurdJ YxExOEc0hr+rJQkkFgPUr+ph31cSB3GXU2c+/TgswYJtJaAdE4hLZ9KBhaCdgY7s0ei/vk ZW/gaMqEpwiuaxXcYVMdRLbEU5DvYnE= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1588176513; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=RqcnavPFirGVOTXHhjY5q5DM9FLFgis6bxz7RvWdVJE=; b=QZPrrd8zNNebgZXnzhIAYA2aP84wbXzXTC4SfYYO6rv5ju2/aeWwsXYRoUwkzVKNDRurdJ YxExOEc0hr+rJQkkFgPUr+ph31cSB3GXU2c+/TgswYJtJaAdE4hLZ9KBhaCdgY7s0ei/vk ZW/gaMqEpwiuaxXcYVMdRLbEU5DvYnE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-480-F36vCcGRMVu93iH7TXheIg-1; Wed, 29 Apr 2020 12:08:26 -0400 X-MC-Unique: F36vCcGRMVu93iH7TXheIg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E4E0C1895A28; Wed, 29 Apr 2020 16:08:21 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-55.ams2.redhat.com [10.36.114.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id BAF7C605FB; Wed, 29 Apr 2020 16:08:07 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Subject: [PATCH v1 0/3] mm/memory_hotplug: Make virtio-mem play nicely with kexec-tools Date: Wed, 29 Apr 2020 18:08:00 +0200 Message-Id: <20200429160803.109056-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Content-Transfer-Encoding: quoted-printable X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-hyperv@vger.kernel.org, Michal Hocko , "Michael S . Tsirkin" , Jason Wang , Heiko Carstens , Pingfan Liu , virtualization@lists.linux-foundation.org, linux-mm@kvack.org, Paul Mackerras , "K. Y. Srinivasan" , Dan Williams , virtio-dev@lists.oasis-open.org, Wei Liu , Stefano Stabellini , Dave Jiang , Baoquan He , linux-nvdimm@lists.01.org, David Hildenbrand , linux-acpi@vger.kernel.org, Wei Yang , xen-devel@lists.xenproject.org, Len Brown , Nathan Lynch , Stephen Hemminger , Vasily Gorbik , linux-s390@vger.kernel.org, Haiyang Zhang , Leonardo Bras , Boris Ostrovsky , Michal Hocko , Christian Borntraeger , Oscar Salvador , Juergen Gross , Pankaj Gupta , Greg Kroah-Hartman , "Rafael J. Wysocki" , Thomas Gleixner , Eric Biederman , Vishal Verma , Andrew Morton , linuxppc-dev@lists.ozlabs.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" This series is based on [1]: [PATCH v2 00/10] virtio-mem: paravirtualized memory That will hopefull get picked up soon, rebased to -next. The following patches were reverted from -next [2]: [PATCH 0/3] kexec/memory_hotplug: Prevent removal and accidental use As discussed in that thread, they should be reverted from -next already. In theory, if people agree, we could take the first two patches via the -mm tree now and the last (virtio-mem) patch via MST's tree once picking = up virtio-mem. No strong feelings. Memory added by virtio-mem is special and might contain logical holes, especially after memory unplug, but also when adding memory in sub-section size. While memory in these holes can usually be read, that memory should not be touched. virtio-mem managed device memory is never exposed via any firmware memmap (esp., e820). The device driver will request to plug memory from the hypervisor and add it to Linux. On a cold start, all memory is unplugged, and the guest driver will first request to plug memory from the hypervisor, to then add it to Linux. Afte= r a reboot, all memory will get unplugged (except in rare, special cases). = In case the device driver comes up and detects that some memory is still plugged after a reboot, it will manually request to unplug all memory fro= m the hypervisor first - to then request to plug memory from the hypervisor and add to Linux. This is essentially a defragmentation step, where all logical holes are removed. As the device driver is responsible for detecting, adding and managing th= at memory, also kexec should treat it like that. It is special. We need a wa= y to teach kexec-tools to not add that memory to the fixed-up firmware memmap, to not place kexec images onto this memory, but still allow kdump to dump it. Add a flag to tell memory hotplug code to not create /sys/firmware/memmap entries and to indicate it via "System RAM (driver managed)" in /proc/iomem. Before this series, kexec_file_load() already did the right thing (for virtio-mem) by not adding that memory to the fixed-up firmware memmap and letting the device driver handle it. With this series, also kexec_load() = - which relies on user space to provide a fixed up firmware memmap - does the right thing with virtio-mem memory. When the virtio-mem device driver(s) come up, they will request to unplug all memory from the hypervisor first (esp. defragment), to then request t= o plug consecutive memory ranges from the hypervisor, and add them to Linux - just like on a reboot where we still have memory plugged. [1] https://lore.kernel.org/r/20200311171422.10484-1-david@redhat.com/ [2] https://lore.kernel.org/r/20200326180730.4754-1-james.morse@arm.com David Hildenbrand (3): mm/memory_hotplug: Prepare passing flags to add_memory() and friends mm/memory_hotplug: Introduce MHP_DRIVER_MANAGED virtio-mem: Add memory with MHP_DRIVER_MANAGED arch/powerpc/platforms/powernv/memtrace.c | 2 +- .../platforms/pseries/hotplug-memory.c | 2 +- drivers/acpi/acpi_memhotplug.c | 2 +- drivers/base/memory.c | 2 +- drivers/dax/kmem.c | 2 +- drivers/hv/hv_balloon.c | 2 +- drivers/s390/char/sclp_cmd.c | 2 +- drivers/virtio/virtio_mem.c | 3 +- drivers/xen/balloon.c | 2 +- include/linux/memory_hotplug.h | 15 +++++++-- mm/memory_hotplug.c | 31 +++++++++++++------ 11 files changed, 44 insertions(+), 21 deletions(-) --=20 2.25.3 From mboxrd@z Thu Jan 1 00:00:00 1970 From: David Hildenbrand Subject: [PATCH v1 0/3] mm/memory_hotplug: Make virtio-mem play nicely with kexec-tools Date: Wed, 29 Apr 2020 18:08:00 +0200 Message-ID: <20200429160803.109056-1-david@redhat.com> Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable Return-path: Sender: List-Post: List-Help: List-Unsubscribe: List-Subscribe: To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, virtio-dev@lists.oasis-open.org, virtualization@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, linux-acpi@vger.kernel.org, linux-nvdimm@lists.01.org, linux-hyperv@vger.kernel.org, linux-s390@vger.kernel.org, xen-devel@lists.xenproject.org, Michal Hocko , Andrew Morton , "Michael S . Tsirkin" , David Hildenbrand , Baoquan He , Benjamin Herrenschmidt , Boris Ostrovsky , Christian Borntraeger , Dan Williams , Dave Jiang , Eric Biederman , Greg Kroah-Hartman , Haiyang Zhang , Heiko Carstens , Jason Wang List-Id: virtualization@lists.linuxfoundation.org This series is based on [1]: [PATCH v2 00/10] virtio-mem: paravirtualized memory That will hopefull get picked up soon, rebased to -next. The following patches were reverted from -next [2]: [PATCH 0/3] kexec/memory_hotplug: Prevent removal and accidental use As discussed in that thread, they should be reverted from -next already. In theory, if people agree, we could take the first two patches via the -mm tree now and the last (virtio-mem) patch via MST's tree once picking = up virtio-mem. No strong feelings. Memory added by virtio-mem is special and might contain logical holes, especially after memory unplug, but also when adding memory in sub-section size. While memory in these holes can usually be read, that memory should not be touched. virtio-mem managed device memory is never exposed via any firmware memmap (esp., e820). The device driver will request to plug memory from the hypervisor and add it to Linux. On a cold start, all memory is unplugged, and the guest driver will first request to plug memory from the hypervisor, to then add it to Linux. Afte= r a reboot, all memory will get unplugged (except in rare, special cases). = In case the device driver comes up and detects that some memory is still plugged after a reboot, it will manually request to unplug all memory fro= m the hypervisor first - to then request to plug memory from the hypervisor and add to Linux. This is essentially a defragmentation step, where all logical holes are removed. As the device driver is responsible for detecting, adding and managing th= at memory, also kexec should treat it like that. It is special. We need a wa= y to teach kexec-tools to not add that memory to the fixed-up firmware memmap, to not place kexec images onto this memory, but still allow kdump to dump it. Add a flag to tell memory hotplug code to not create /sys/firmware/memmap entries and to indicate it via "System RAM (driver managed)" in /proc/iomem. Before this series, kexec_file_load() already did the right thing (for virtio-mem) by not adding that memory to the fixed-up firmware memmap and letting the device driver handle it. With this series, also kexec_load() = - which relies on user space to provide a fixed up firmware memmap - does the right thing with virtio-mem memory. When the virtio-mem device driver(s) come up, they will request to unplug all memory from the hypervisor first (esp. defragment), to then request t= o plug consecutive memory ranges from the hypervisor, and add them to Linux - just like on a reboot where we still have memory plugged. [1] https://lore.kernel.org/r/20200311171422.10484-1-david@redhat.com/ [2] https://lore.kernel.org/r/20200326180730.4754-1-james.morse@arm.com David Hildenbrand (3): mm/memory_hotplug: Prepare passing flags to add_memory() and friends mm/memory_hotplug: Introduce MHP_DRIVER_MANAGED virtio-mem: Add memory with MHP_DRIVER_MANAGED arch/powerpc/platforms/powernv/memtrace.c | 2 +- .../platforms/pseries/hotplug-memory.c | 2 +- drivers/acpi/acpi_memhotplug.c | 2 +- drivers/base/memory.c | 2 +- drivers/dax/kmem.c | 2 +- drivers/hv/hv_balloon.c | 2 +- drivers/s390/char/sclp_cmd.c | 2 +- drivers/virtio/virtio_mem.c | 3 +- drivers/xen/balloon.c | 2 +- include/linux/memory_hotplug.h | 15 +++++++-- mm/memory_hotplug.c | 31 +++++++++++++------ 11 files changed, 44 insertions(+), 21 deletions(-) --=20 2.25.3 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79319C83000 for ; Wed, 29 Apr 2020 16:08:55 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3E4AD20757 for ; Wed, 29 Apr 2020 16:08:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QZPrrd8z" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3E4AD20757 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jTpGF-0001DQ-Ut; Wed, 29 Apr 2020 16:08:35 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1jTpGE-0001DL-PX for xen-devel@lists.xenproject.org; Wed, 29 Apr 2020 16:08:34 +0000 X-Inumbo-ID: acda2e01-8a33-11ea-9965-12813bfff9fa Received: from us-smtp-delivery-1.mimecast.com (unknown [205.139.110.61]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTP id acda2e01-8a33-11ea-9965-12813bfff9fa; Wed, 29 Apr 2020 16:08:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1588176513; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding; bh=RqcnavPFirGVOTXHhjY5q5DM9FLFgis6bxz7RvWdVJE=; b=QZPrrd8zNNebgZXnzhIAYA2aP84wbXzXTC4SfYYO6rv5ju2/aeWwsXYRoUwkzVKNDRurdJ YxExOEc0hr+rJQkkFgPUr+ph31cSB3GXU2c+/TgswYJtJaAdE4hLZ9KBhaCdgY7s0ei/vk ZW/gaMqEpwiuaxXcYVMdRLbEU5DvYnE= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-480-F36vCcGRMVu93iH7TXheIg-1; Wed, 29 Apr 2020 12:08:26 -0400 X-MC-Unique: F36vCcGRMVu93iH7TXheIg-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E4E0C1895A28; Wed, 29 Apr 2020 16:08:21 +0000 (UTC) Received: from t480s.redhat.com (ovpn-114-55.ams2.redhat.com [10.36.114.55]) by smtp.corp.redhat.com (Postfix) with ESMTP id BAF7C605FB; Wed, 29 Apr 2020 16:08:07 +0000 (UTC) From: David Hildenbrand To: linux-kernel@vger.kernel.org Subject: [PATCH v1 0/3] mm/memory_hotplug: Make virtio-mem play nicely with kexec-tools Date: Wed, 29 Apr 2020 18:08:00 +0200 Message-Id: <20200429160803.109056-1-david@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Content-Transfer-Encoding: quoted-printable X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: linux-hyperv@vger.kernel.org, Michal Hocko , "Michael S . Tsirkin" , Benjamin Herrenschmidt , Jason Wang , Heiko Carstens , Pingfan Liu , virtualization@lists.linux-foundation.org, linux-mm@kvack.org, Paul Mackerras , "K. Y. Srinivasan" , Dan Williams , virtio-dev@lists.oasis-open.org, Wei Liu , Stefano Stabellini , Dave Jiang , Baoquan He , linux-nvdimm@lists.01.org, Michael Ellerman , David Hildenbrand , linux-acpi@vger.kernel.org, Wei Yang , xen-devel@lists.xenproject.org, Len Brown , Nathan Lynch , Stephen Hemminger , Vasily Gorbik , linux-s390@vger.kernel.org, Haiyang Zhang , Leonardo Bras , Boris Ostrovsky , Michal Hocko , Christian Borntraeger , Oscar Salvador , Juergen Gross , Pankaj Gupta , Greg Kroah-Hartman , "Rafael J. Wysocki" , Thomas Gleixner , Eric Biederman , Vishal Verma , Andrew Morton , linuxppc-dev@lists.ozlabs.org Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" This series is based on [1]: [PATCH v2 00/10] virtio-mem: paravirtualized memory That will hopefull get picked up soon, rebased to -next. The following patches were reverted from -next [2]: [PATCH 0/3] kexec/memory_hotplug: Prevent removal and accidental use As discussed in that thread, they should be reverted from -next already. In theory, if people agree, we could take the first two patches via the -mm tree now and the last (virtio-mem) patch via MST's tree once picking = up virtio-mem. No strong feelings. Memory added by virtio-mem is special and might contain logical holes, especially after memory unplug, but also when adding memory in sub-section size. While memory in these holes can usually be read, that memory should not be touched. virtio-mem managed device memory is never exposed via any firmware memmap (esp., e820). The device driver will request to plug memory from the hypervisor and add it to Linux. On a cold start, all memory is unplugged, and the guest driver will first request to plug memory from the hypervisor, to then add it to Linux. Afte= r a reboot, all memory will get unplugged (except in rare, special cases). = In case the device driver comes up and detects that some memory is still plugged after a reboot, it will manually request to unplug all memory fro= m the hypervisor first - to then request to plug memory from the hypervisor and add to Linux. This is essentially a defragmentation step, where all logical holes are removed. As the device driver is responsible for detecting, adding and managing th= at memory, also kexec should treat it like that. It is special. We need a wa= y to teach kexec-tools to not add that memory to the fixed-up firmware memmap, to not place kexec images onto this memory, but still allow kdump to dump it. Add a flag to tell memory hotplug code to not create /sys/firmware/memmap entries and to indicate it via "System RAM (driver managed)" in /proc/iomem. Before this series, kexec_file_load() already did the right thing (for virtio-mem) by not adding that memory to the fixed-up firmware memmap and letting the device driver handle it. With this series, also kexec_load() = - which relies on user space to provide a fixed up firmware memmap - does the right thing with virtio-mem memory. When the virtio-mem device driver(s) come up, they will request to unplug all memory from the hypervisor first (esp. defragment), to then request t= o plug consecutive memory ranges from the hypervisor, and add them to Linux - just like on a reboot where we still have memory plugged. [1] https://lore.kernel.org/r/20200311171422.10484-1-david@redhat.com/ [2] https://lore.kernel.org/r/20200326180730.4754-1-james.morse@arm.com David Hildenbrand (3): mm/memory_hotplug: Prepare passing flags to add_memory() and friends mm/memory_hotplug: Introduce MHP_DRIVER_MANAGED virtio-mem: Add memory with MHP_DRIVER_MANAGED arch/powerpc/platforms/powernv/memtrace.c | 2 +- .../platforms/pseries/hotplug-memory.c | 2 +- drivers/acpi/acpi_memhotplug.c | 2 +- drivers/base/memory.c | 2 +- drivers/dax/kmem.c | 2 +- drivers/hv/hv_balloon.c | 2 +- drivers/s390/char/sclp_cmd.c | 2 +- drivers/virtio/virtio_mem.c | 3 +- drivers/xen/balloon.c | 2 +- include/linux/memory_hotplug.h | 15 +++++++-- mm/memory_hotplug.c | 31 +++++++++++++------ 11 files changed, 44 insertions(+), 21 deletions(-) --=20 2.25.3 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: virtio-dev-return-7207-cohuck=redhat.com@lists.oasis-open.org Sender: List-Post: List-Help: List-Unsubscribe: List-Subscribe: Received: from lists.oasis-open.org (oasis-open.org [10.110.1.242]) by lists.oasis-open.org (Postfix) with ESMTP id CC3C8985E81 for ; Wed, 29 Apr 2020 16:08:34 +0000 (UTC) From: David Hildenbrand Date: Wed, 29 Apr 2020 18:08:00 +0200 Message-Id: <20200429160803.109056-1-david@redhat.com> MIME-Version: 1.0 Subject: [virtio-dev] [PATCH v1 0/3] mm/memory_hotplug: Make virtio-mem play nicely with kexec-tools Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable To: linux-kernel@vger.kernel.org Cc: linux-mm@kvack.org, virtio-dev@lists.oasis-open.org, virtualization@lists.linux-foundation.org, linuxppc-dev@lists.ozlabs.org, linux-acpi@vger.kernel.org, linux-nvdimm@lists.01.org, linux-hyperv@vger.kernel.org, linux-s390@vger.kernel.org, xen-devel@lists.xenproject.org, Michal Hocko , Andrew Morton , "Michael S . Tsirkin" , David Hildenbrand , Baoquan He , Benjamin Herrenschmidt , Boris Ostrovsky , Christian Borntraeger , Dan Williams , Dave Jiang , Eric Biederman , Greg Kroah-Hartman , Haiyang Zhang , Heiko Carstens , Jason Wang , Juergen Gross , "K. Y. Srinivasan" , Len Brown , Leonardo Bras , Michael Ellerman , Michal Hocko , Nathan Lynch , Oscar Salvador , Pankaj Gupta , Paul Mackerras , Pingfan Liu , "Rafael J. Wysocki" , Stefano Stabellini , Stephen Hemminger , Thomas Gleixner , Vasily Gorbik , Vishal Verma , Wei Liu , Wei Yang List-ID: This series is based on [1]: =09[PATCH v2 00/10] virtio-mem: paravirtualized memory That will hopefull get picked up soon, rebased to -next. The following patches were reverted from -next [2]: =09[PATCH 0/3] kexec/memory_hotplug: Prevent removal and accidental use As discussed in that thread, they should be reverted from -next already. In theory, if people agree, we could take the first two patches via the -mm tree now and the last (virtio-mem) patch via MST's tree once picking up virtio-mem. No strong feelings. Memory added by virtio-mem is special and might contain logical holes, especially after memory unplug, but also when adding memory in sub-section size. While memory in these holes can usually be read, that memory should not be touched. virtio-mem managed device memory is never exposed via any firmware memmap (esp., e820). The device driver will request to plug memory from the hypervisor and add it to Linux. On a cold start, all memory is unplugged, and the guest driver will first request to plug memory from the hypervisor, to then add it to Linux. After a reboot, all memory will get unplugged (except in rare, special cases). In case the device driver comes up and detects that some memory is still plugged after a reboot, it will manually request to unplug all memory from the hypervisor first - to then request to plug memory from the hypervisor and add to Linux. This is essentially a defragmentation step, where all logical holes are removed. As the device driver is responsible for detecting, adding and managing that memory, also kexec should treat it like that. It is special. We need a way to teach kexec-tools to not add that memory to the fixed-up firmware memmap, to not place kexec images onto this memory, but still allow kdump to dump it. Add a flag to tell memory hotplug code to not create /sys/firmware/memmap entries and to indicate it via "System RAM (driver managed)" in /proc/iomem. Before this series, kexec_file_load() already did the right thing (for virtio-mem) by not adding that memory to the fixed-up firmware memmap and letting the device driver handle it. With this series, also kexec_load() - which relies on user space to provide a fixed up firmware memmap - does the right thing with virtio-mem memory. When the virtio-mem device driver(s) come up, they will request to unplug all memory from the hypervisor first (esp. defragment), to then request to plug consecutive memory ranges from the hypervisor, and add them to Linux - just like on a reboot where we still have memory plugged. [1] https://lore.kernel.org/r/20200311171422.10484-1-david@redhat.com/ [2] https://lore.kernel.org/r/20200326180730.4754-1-james.morse@arm.com David Hildenbrand (3): mm/memory_hotplug: Prepare passing flags to add_memory() and friends mm/memory_hotplug: Introduce MHP_DRIVER_MANAGED virtio-mem: Add memory with MHP_DRIVER_MANAGED arch/powerpc/platforms/powernv/memtrace.c | 2 +- .../platforms/pseries/hotplug-memory.c | 2 +- drivers/acpi/acpi_memhotplug.c | 2 +- drivers/base/memory.c | 2 +- drivers/dax/kmem.c | 2 +- drivers/hv/hv_balloon.c | 2 +- drivers/s390/char/sclp_cmd.c | 2 +- drivers/virtio/virtio_mem.c | 3 +- drivers/xen/balloon.c | 2 +- include/linux/memory_hotplug.h | 15 +++++++-- mm/memory_hotplug.c | 31 +++++++++++++------ 11 files changed, 44 insertions(+), 21 deletions(-) --=20 2.25.3 --------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org