From mboxrd@z Thu Jan 1 00:00:00 1970 From: Vasilis Liaskovitis Subject: Re: [Qemu-devel] [RFC PATCH v2 00/21] ACPI memory hotplug Date: Fri, 13 Jul 2012 19:49:48 +0200 Message-ID: <20120713174948.GE13216@dhcp-192-168-178-175.profitbricks.localdomain> References: <1342002726-18258-1-git-send-email-vasilis.liaskovitis@profitbricks.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, seabios@seabios.org, gleb@redhat.com, kevin@koconnor.net, avi@redhat.com, anthony@codemonkey.ws, imammedo@redhat.com To: Blue Swirl Return-path: Received: from mail-bk0-f46.google.com ([209.85.214.46]:55367 "EHLO mail-bk0-f46.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753451Ab2GMRtw (ORCPT ); Fri, 13 Jul 2012 13:49:52 -0400 Received: by bkwj10 with SMTP id j10so3395146bkw.19 for ; Fri, 13 Jul 2012 10:49:51 -0700 (PDT) Content-Disposition: inline In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: On Thu, Jul 12, 2012 at 08:04:56PM +0000, Blue Swirl wrote: > On Wed, Jul 11, 2012 at 10:31 AM, Vasilis Liaskovitis > wrote: > > This is v2 of the ACPI memory hotplug prototype for x86_64 target. > > I think the concept of DIMMs (what about SIMMs? SODIMMs? I liked > memslot) would be useful for most targets, but hotplugging may be > limited to x86 only. It would be nice to keep these two separate or as > loosely coupled as possible. agreed. what specific usecases besides hotplugging are you thinking about? Also are there non-acpi hotplug platforms? I am trying to keep generic dimm manipulation functions (e.g. population / depopulation and searching) in hw/dimm[.ch]. Currently the x86-acpi_piix4 "backend" registers a callback for hot-add / hot-remove. In theory other hotplug backends can hook in. btw I don't mind using "-memslot" (I think someone during v1 mentioned -dimm), we just need some consensus on the naming. > > > > > Changes v1->v2 > > > > - memory map is automatically calculated for hotplug dimms. Dimms are added from > > top-of-memory skipping the pci hole at [PCI_HOLE_START, 4G). > > - Renamed from "-memslot" to "-dimm". Commands changed to "dimm_add", "dimm_del". > > - Seabios ejection array reduced to a byte. Use extraction macros for dimm ssdt. > > - additional SRAT paravirt info does not break previous SRAT fw_cfg layout. > > - Documentation of new acpi_piix4 registers and paravirt data. > > - add ACPI _OST support for _OST enabled guests. This allows qemu to receive > > notification for success / failure of memory hot-add and hot-remove operations. > > Guest needs to support _OST (https://lkml.org/lkml/2012/6/25/321) > > - add monitor info command to report total guest memory (initial + hot-added) > > - add command line options and monitor commands for batch dimm creation/population > > > > Overview: > > > > Dimm devices are modeled with a new qemu command line > > > > "-dimm id=name,size=sz,node=pxm,populated=on|off" > > > > As already mentioned, the starting physical address for all dimms is calculated > > automatically from top of memory, skipping the pci hole at [PCI_HOLE_START, 4G). > > Node is defining numa proximity for this dimm. When not defined it defaults > > to zero. > > "-dimm id=dimm0,size=512M,node=0,populated=off" > > will define a 512M memory slot belonging to numa node 0. > > > > Dimms are added or removed with a new hmp command "dimm_add/dimm_del": > > Hot-add syntax: "dimm_add id" > > Hot-remove syntax: "dimm_del id" > > > > Issues: > > > > - Live migration works as long as populated field is changed to "on" for > > hotplugged dimms at the destination qemu command line (patch 12/21 lifts > > this requirement). The DimmState structure does not yet define a > > VMStateDescription, but i assume this is the preferred way to pass state > > for migration. > > > > - Dimms are abstracted as qdevices attached to the main system bus. However, > > memory hotplugging has its own side channel ignoring main_system_bus's hotplug > > incapability. A cleaner integration is still needed, probably attaching memory > > devices as children-links of an acpi-capable device (in the pc case acpi_piix4) > > instead of the system bus (TBD). Then device_add/device_del instead of new > > commands can hopefully be used. > > > > Comments/review welcome. > > > > series is based on uq/master for qemu-kvm, and master for seabios. Can be found > > also at: > > http://github.com/vliaskov/qemu-kvm/commits/memhp-v2 > > http://github.com/vliaskov/seabios/commits/memhp-v2 > > > > Vasilis Liaskovitis (14): > > dimm: Implement memory device abstraction > > acpi_piix4: Implement memory device hotplug registers > > pc: calculate dimm physical addresses and adjust memory map > > pc: Add dimm paravirt SRAT info > > Implement "-dimm" command line option > > Implement dimm_add and dimm_del commands for hmp and qmp > > fix live-migration when "populated=on" is missing > > Implement memory hotplug notification lists > > acpi_piix4: _OST dimm support > > acpi_piix4: Update dimm state on VM reboot > > acpi_piix4: Update dimm bitmap state on hot-remove fail > > Implement "info memtotal" and "query-memtotal" > > Implement -dimms, -dimmspop command line options > > Implement mem_increase, mem_decrease hmp/qmp commands > > > > arch_init.c | 23 ++- > > docs/specs/acpi_hotplug.txt | 46 +++++ > > docs/specs/fwcfg.txt | 28 +++ > > hmp-commands.hx | 67 +++++++ > > hmp.c | 24 +++ > > hmp.h | 2 + > > hw/Makefile.objs | 2 +- > > hw/acpi_piix4.c | 131 ++++++++++++- > > hw/dimm.c | 449 +++++++++++++++++++++++++++++++++++++++++++ > > hw/dimm.h | 72 +++++++ > > hw/pc.c | 94 +++++++++- > > hw/pc.h | 6 + > > hw/pc_piix.c | 18 ++- > > monitor.c | 35 ++++ > > monitor.h | 5 + > > qapi-schema.json | 38 ++++ > > qemu-config.c | 70 +++++++ > > qemu-options.hx | 15 ++ > > qmp-commands.hx | 137 +++++++++++++ > > sysemu.h | 1 + > > vl.c | 122 ++++++++++++- > > 21 files changed, 1368 insertions(+), 17 deletions(-) > > create mode 100644 docs/specs/acpi_hotplug.txt > > create mode 100644 docs/specs/fwcfg.txt > > create mode 100644 hw/dimm.c > > create mode 100644 hw/dimm.h > > > > Vasilis Liaskovitis (7): > > Add ACPI_EXTRACT_DEVICE* macros > > Add SSDT memory device support > > acpi-dsdt: Implement functions for memory hotplug. > > acpi: generate hotplug memory devices. > > pciinit: Fix pcimem_start value > > acpi_dsdt: Support _OST dimm method > > acpi_dsdt: Revert internal dimm state on _OST failure > > > > Makefile | 2 +- > > src/acpi-dsdt.dsl | 120 ++++++++++++++++++++++++++++++++++++- > > src/acpi.c | 158 +++++++++++++++++++++++++++++++++++++++++++++++-- > > src/pciinit.c | 2 +- > > src/ssdt-mem.dsl | 69 +++++++++++++++++++++ > > tools/acpi_extract.py | 28 +++++++++ > > 6 files changed, 369 insertions(+), 10 deletions(-) > > create mode 100644 src/ssdt-mem.dsl > > From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([208.118.235.92]:58315) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Spk03-0001mz-6b for qemu-devel@nongnu.org; Fri, 13 Jul 2012 13:49:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Spk00-0000Tu-SI for qemu-devel@nongnu.org; Fri, 13 Jul 2012 13:49:55 -0400 Received: from mail-bk0-f45.google.com ([209.85.214.45]:54329) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Spk00-0000Te-Hw for qemu-devel@nongnu.org; Fri, 13 Jul 2012 13:49:52 -0400 Received: by bkcji1 with SMTP id ji1so2844766bkc.4 for ; Fri, 13 Jul 2012 10:49:51 -0700 (PDT) Date: Fri, 13 Jul 2012 19:49:48 +0200 From: Vasilis Liaskovitis Message-ID: <20120713174948.GE13216@dhcp-192-168-178-175.profitbricks.localdomain> References: <1342002726-18258-1-git-send-email-vasilis.liaskovitis@profitbricks.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Subject: Re: [Qemu-devel] [RFC PATCH v2 00/21] ACPI memory hotplug List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Blue Swirl Cc: kvm@vger.kernel.org, gleb@redhat.com, seabios@seabios.org, qemu-devel@nongnu.org, kevin@koconnor.net, avi@redhat.com, anthony@codemonkey.ws, imammedo@redhat.com On Thu, Jul 12, 2012 at 08:04:56PM +0000, Blue Swirl wrote: > On Wed, Jul 11, 2012 at 10:31 AM, Vasilis Liaskovitis > wrote: > > This is v2 of the ACPI memory hotplug prototype for x86_64 target. > > I think the concept of DIMMs (what about SIMMs? SODIMMs? I liked > memslot) would be useful for most targets, but hotplugging may be > limited to x86 only. It would be nice to keep these two separate or as > loosely coupled as possible. agreed. what specific usecases besides hotplugging are you thinking about? Also are there non-acpi hotplug platforms? I am trying to keep generic dimm manipulation functions (e.g. population / depopulation and searching) in hw/dimm[.ch]. Currently the x86-acpi_piix4 "backend" registers a callback for hot-add / hot-remove. In theory other hotplug backends can hook in. btw I don't mind using "-memslot" (I think someone during v1 mentioned -dimm), we just need some consensus on the naming. > > > > > Changes v1->v2 > > > > - memory map is automatically calculated for hotplug dimms. Dimms are added from > > top-of-memory skipping the pci hole at [PCI_HOLE_START, 4G). > > - Renamed from "-memslot" to "-dimm". Commands changed to "dimm_add", "dimm_del". > > - Seabios ejection array reduced to a byte. Use extraction macros for dimm ssdt. > > - additional SRAT paravirt info does not break previous SRAT fw_cfg layout. > > - Documentation of new acpi_piix4 registers and paravirt data. > > - add ACPI _OST support for _OST enabled guests. This allows qemu to receive > > notification for success / failure of memory hot-add and hot-remove operations. > > Guest needs to support _OST (https://lkml.org/lkml/2012/6/25/321) > > - add monitor info command to report total guest memory (initial + hot-added) > > - add command line options and monitor commands for batch dimm creation/population > > > > Overview: > > > > Dimm devices are modeled with a new qemu command line > > > > "-dimm id=name,size=sz,node=pxm,populated=on|off" > > > > As already mentioned, the starting physical address for all dimms is calculated > > automatically from top of memory, skipping the pci hole at [PCI_HOLE_START, 4G). > > Node is defining numa proximity for this dimm. When not defined it defaults > > to zero. > > "-dimm id=dimm0,size=512M,node=0,populated=off" > > will define a 512M memory slot belonging to numa node 0. > > > > Dimms are added or removed with a new hmp command "dimm_add/dimm_del": > > Hot-add syntax: "dimm_add id" > > Hot-remove syntax: "dimm_del id" > > > > Issues: > > > > - Live migration works as long as populated field is changed to "on" for > > hotplugged dimms at the destination qemu command line (patch 12/21 lifts > > this requirement). The DimmState structure does not yet define a > > VMStateDescription, but i assume this is the preferred way to pass state > > for migration. > > > > - Dimms are abstracted as qdevices attached to the main system bus. However, > > memory hotplugging has its own side channel ignoring main_system_bus's hotplug > > incapability. A cleaner integration is still needed, probably attaching memory > > devices as children-links of an acpi-capable device (in the pc case acpi_piix4) > > instead of the system bus (TBD). Then device_add/device_del instead of new > > commands can hopefully be used. > > > > Comments/review welcome. > > > > series is based on uq/master for qemu-kvm, and master for seabios. Can be found > > also at: > > http://github.com/vliaskov/qemu-kvm/commits/memhp-v2 > > http://github.com/vliaskov/seabios/commits/memhp-v2 > > > > Vasilis Liaskovitis (14): > > dimm: Implement memory device abstraction > > acpi_piix4: Implement memory device hotplug registers > > pc: calculate dimm physical addresses and adjust memory map > > pc: Add dimm paravirt SRAT info > > Implement "-dimm" command line option > > Implement dimm_add and dimm_del commands for hmp and qmp > > fix live-migration when "populated=on" is missing > > Implement memory hotplug notification lists > > acpi_piix4: _OST dimm support > > acpi_piix4: Update dimm state on VM reboot > > acpi_piix4: Update dimm bitmap state on hot-remove fail > > Implement "info memtotal" and "query-memtotal" > > Implement -dimms, -dimmspop command line options > > Implement mem_increase, mem_decrease hmp/qmp commands > > > > arch_init.c | 23 ++- > > docs/specs/acpi_hotplug.txt | 46 +++++ > > docs/specs/fwcfg.txt | 28 +++ > > hmp-commands.hx | 67 +++++++ > > hmp.c | 24 +++ > > hmp.h | 2 + > > hw/Makefile.objs | 2 +- > > hw/acpi_piix4.c | 131 ++++++++++++- > > hw/dimm.c | 449 +++++++++++++++++++++++++++++++++++++++++++ > > hw/dimm.h | 72 +++++++ > > hw/pc.c | 94 +++++++++- > > hw/pc.h | 6 + > > hw/pc_piix.c | 18 ++- > > monitor.c | 35 ++++ > > monitor.h | 5 + > > qapi-schema.json | 38 ++++ > > qemu-config.c | 70 +++++++ > > qemu-options.hx | 15 ++ > > qmp-commands.hx | 137 +++++++++++++ > > sysemu.h | 1 + > > vl.c | 122 ++++++++++++- > > 21 files changed, 1368 insertions(+), 17 deletions(-) > > create mode 100644 docs/specs/acpi_hotplug.txt > > create mode 100644 docs/specs/fwcfg.txt > > create mode 100644 hw/dimm.c > > create mode 100644 hw/dimm.h > > > > Vasilis Liaskovitis (7): > > Add ACPI_EXTRACT_DEVICE* macros > > Add SSDT memory device support > > acpi-dsdt: Implement functions for memory hotplug. > > acpi: generate hotplug memory devices. > > pciinit: Fix pcimem_start value > > acpi_dsdt: Support _OST dimm method > > acpi_dsdt: Revert internal dimm state on _OST failure > > > > Makefile | 2 +- > > src/acpi-dsdt.dsl | 120 ++++++++++++++++++++++++++++++++++++- > > src/acpi.c | 158 +++++++++++++++++++++++++++++++++++++++++++++++-- > > src/pciinit.c | 2 +- > > src/ssdt-mem.dsl | 69 +++++++++++++++++++++ > > tools/acpi_extract.py | 28 +++++++++ > > 6 files changed, 369 insertions(+), 10 deletions(-) > > create mode 100644 src/ssdt-mem.dsl > >