From: Igor Mammedov <imammedo@redhat.com>
To: qemu-devel@nongnu.org
Subject: [PATCH v4 00/80] refactor main RAM allocation to use hostmem backend
Date: Fri, 31 Jan 2020 16:08:30 +0100 [thread overview]
Message-ID: <1580483390-131164-1-git-send-email-imammedo@redhat.com> (raw)
v4:
- pick up new Reviewed-by-s
- Fix access to uninitialized pagesize/hpsize in
"[PATCH REPOST v3 74/80] exec: cleanup qemu_minrampagesize()/qemu_maxrampagesize()"
- Make explicitly provided memory-backend work by using string property instead
of link so it would be possible to delay access to the backend to the time
when backends are initialized.
- added new patches to make explicit backend work nice with -m and do sanity
check on ram_size
'vl.c: move -m parsing after memory backends has been processed'
'vl.c: ensure that ram_size matches size of machine.memory-backend"
all this ram_size business needs cleanup too, but that's out of the scope
of this series.
- include m68k/q800 board into conversion
- drop patches that were merged through other trees
v3:
- due to libvirt not being ready, postpone till 5.1
* [PATCH v2 82/86] numa: forbid '-numa node, mem' for 5.0 and newer machine types
and depended
[PATCH v2 86/86] numa: remove deprecated implicit RAM distribution between nodes
- drop as not related "[PATCH v2 85/86] numa: make exit() usage consistent"
- drop "[PATCH v2 76/86] post conversion default_ram_id cleanup"
so that default memory-backedend won't be created for boards that do not care
about -m. Which makes -m optin feature. We should decide what do in case
board doesn't use -m (but that's out of scope of this series)
- use object_register_sugar_prop() instead of hacking compat props directly
- simplified/reworked aspeed patches
- s/RAM_ADDR_FMT/size_to_str()/
- rename 'ram-memdev' property to 'memory-backend'
- minor fixes to numa-test
- fixes for issues noticed during review of
[PATCH v2 66/86] ppc/{ppc440_bamboo,sam460x}: drop RAM size fixup
v2:
- fix compile errors on mingw32 host by introducing RAM_ADDR_UFMT [11/86]
- replace "[PATCH 43/86] hppa: drop RAM size fixup" with alternative
patches made by Philippe (which effectively do the same thing but other
way around)
- ppc440: fix crash and add suggested valid RAM size in error output.
s/ppc4xx_sdram_adjust/ppc4xx_sdram_prep/ and simplify it by removing
not necessary nested loop
- rebase on current master due to new conflicts
Series removes ad hoc RAM allocation API (memory_region_allocate_system_memory)
and consolidates it around hostmem backend. It allows to
* resolve conflicts between global -mem-prealloc and hostmem's "policy" option
fixing premature allocation before binding policy is applied
* simplify complicated memory allocation routines which had to deal with 2 ways
to allocate RAM.
* it allows to reuse hostmem backends of a choice for main RAM without adding
extra CLI options to duplicate hostmem features.
Recent case was -mem-shared, to enable vhost-user on targets that don't
support hostmem backends [1] (ex: s390)
* move RAM allocation from individual boards into generic machine code and
provide them with prepared MemoryRegion.
* clean up deprecated NUMA features which were tied to the old API (see patches)
- "numa: remove deprecated -mem-path fallback to anonymous RAM"
- (POSTPONED, waiting on libvirt side) "forbid '-numa node,mem' for 5.0 and newer machine types"
- (POSTPONED) "numa: remove deprecated implicit RAM distribution between nodes"
Conversion introduces a new machine.memory-backend property and wrapper code that
aliases global -mem-path and -mem-alloc into automatically created hostmem
backend properties (provided memory-backend was not set explicitly given by user).
And then follows bulk of trivial patches that incrementally convert individual
boards to using machine.memory-backend provided MemoryRegion.
Board conversion typically involves:
* providing MachineClass::default_ram_size and MachineClass::default_ram_id
so generic code could create default backend if user didn't explicitly provide
memory-backend or -m options
* dropping memory_region_allocate_system_memory() call
* using convenience MachineState::ram MemoryRegion, which points to MemoryRegion
allocated by ram-memdev
On top of that for some boards:
* added missing ram_size checks (typically it were boards with fixed ram size)
* ram_size fixups were replaced by checks and hard errors, forcing user to
provide correct "-m" values instead of ignoring it and continuing running.
After all boards are converted the old API is removed and memory allocation
routines are cleaned up.
git tree for testing:
https://github.com/imammedo/qemu convert_main_ram_to_memdev_v4
previous rev:
https://github.com/imammedo/qemu convert_main_ram_to_memdev_v3
https://lists.nongnu.org/archive/html/qemu-devel/2020-01/msg05299.html
Igor Mammedov (80):
numa: remove deprecated -mem-path fallback to anonymous RAM
machine: introduce memory-backend property
machine: alias -mem-path and -mem-prealloc into memory-foo backend
machine: introduce convenience MachineState::ram
initialize MachineState::ram in NUMA case
vl.c: move -m parsing after memory backends has been processed
vl.c: ensure that ram_size matches size of machine.memory-backend
alpha/dp264: use memdev for RAM
arm/aspeed: actually check RAM size
arm/aspeed: use memdev for RAM
arm/collie: use memdev for RAM
arm/cubieboard: use memdev for RAM
arm/digic_boards: use memdev for RAM
arm/highbank: use memdev for RAM
arm/imx25_pdk: drop RAM size fixup
arm/imx25_pdk: use memdev for RAM
arm/integratorcp: use memdev for RAM
arm/kzm: drop RAM size fixup
arm/kzm: use memdev for RAM
arm/mcimx6ul-evk: use memdev for RAM
arm/mcimx7d-sabre: use memdev for RAM
arm/mps2-tz: use memdev for RAM
arm/mps2: use memdev for RAM
arm/musicpal: use memdev for RAM
arm/nseries: use memdev for RAM
arm/omap_sx1: use memdev for RAM
arm/palm: use memdev for RAM
arm/raspi: use memdev for RAM
arm/sabrelite: use memdev for RAM
arm/sbsa-ref: use memdev for RAM
arm/versatilepb: use memdev for RAM
arm/vexpress: use memdev for RAM
arm/virt: use memdev for RAM
arm/xilinx_zynq: drop RAM size fixup
arm/xilinx_zynq: use memdev for RAM
arm/xlnx-versal-virt: use memdev for RAM
arm/xlnx-zcu102: use memdev for RAM
s390x/s390-virtio-ccw: use memdev for RAM
null-machine: use memdev for RAM
cris/axis_dev88: use memdev for RAM
hppa: use memdev for RAM
x86/microvm: use memdev for RAM
x86/pc: use memdev for RAM
lm32/lm32_boards: use memdev for RAM
lm32/milkymist: use memdev for RAM
m68k/an5206: use memdev for RAM
m68k/q800: use memdev for RAM
m68k/mcf5208: use memdev for RAM
m68k/next-cube: use memdev for RAM
mips/boston: use memdev for RAM
mips/mips_fulong2e: drop RAM size fixup
mips/mips_fulong2e: use memdev for RAM
mips/mips_jazz: use memdev for RAM
mips/mips_malta: use memdev for RAM
mips/mips_mipssim: use memdev for RAM
mips/mips_r4k: use memdev for RAM
ppc/e500: drop RAM size fixup
ppc/e500: use memdev for RAM
ppc/mac_newworld: use memdev for RAM
ppc/mac_oldworld: use memdev for RAM
ppc/pnv: use memdev for RAM
ppc/ppc405_boards: add RAM size checks
ppc/ppc405_boards: use memdev for RAM
ppc/{ppc440_bamboo, sam460ex}: drop RAM size fixup
ppc/{ppc440_bamboo, sam460ex}: use memdev for RAM
ppc/prep: use memdev for RAM
ppc/spapr: use memdev for RAM
ppc/virtex_ml507: remove unused arguments
ppc/virtex_ml507: use memdev for RAM
sparc/leon3: use memdev for RAM
sparc/sun4m: use memdev for RAM
sparc/niagara: use memdev for RAM
remove no longer used memory_region_allocate_system_memory()
exec: cleanup qemu_minrampagesize()/qemu_maxrampagesize()
exec: drop bogus mem_path from qemu_ram_alloc_from_fd()
make mem_path local variable
hostmem: introduce "prealloc-threads" property
hostmem: fix strict bind policy
tests/numa-test: make top level args dynamic and g_autofree(cli)
cleanups
tests:numa-test: use explicit memdev to specify node RAM
hw/alpha/alpha_sys.h | 2 +-
include/hw/boards.h | 51 ++++++----------
include/hw/misc/aspeed_sdmc.h | 1 +
include/hw/ppc/ppc4xx.h | 9 ++-
include/sysemu/hostmem.h | 20 +++++-
include/sysemu/numa.h | 1 +
include/sysemu/sysemu.h | 2 -
backends/hostmem-file.c | 8 ---
backends/hostmem-memfd.c | 1 -
backends/hostmem-ram.c | 2 -
backends/hostmem.c | 53 +++++++++++-----
exec.c | 64 ++------------------
hw/alpha/dp264.c | 3 +-
hw/alpha/typhoon.c | 8 +--
hw/arm/aspeed.c | 18 +++---
hw/arm/collie.c | 17 ++++--
hw/arm/cubieboard.c | 25 +++-----
hw/arm/digic_boards.c | 40 ++++++------
hw/arm/highbank.c | 10 ++-
hw/arm/imx25_pdk.c | 13 ++--
hw/arm/integratorcp.c | 9 ++-
hw/arm/kzm.c | 18 +++---
hw/arm/mcimx6ul-evk.c | 25 +++-----
hw/arm/mcimx7d-sabre.c | 25 +++-----
hw/arm/mps2-tz.c | 15 +++--
hw/arm/mps2.c | 15 +++--
hw/arm/musicpal.c | 18 ++++--
hw/arm/nseries.c | 32 ++++++----
hw/arm/omap_sx1.c | 20 ++++--
hw/arm/palm.c | 20 ++++--
hw/arm/raspi.c | 32 ++++------
hw/arm/sabrelite.c | 23 +++----
hw/arm/sbsa-ref.c | 7 +--
hw/arm/versatilepb.c | 7 +--
hw/arm/vexpress.c | 14 ++---
hw/arm/virt.c | 7 +--
hw/arm/xilinx_zynq.c | 20 +++---
hw/arm/xlnx-versal-virt.c | 7 +--
hw/arm/xlnx-zcu102.c | 7 +--
hw/core/machine.c | 48 +++++++++++++++
hw/core/null-machine.c | 8 +--
hw/core/numa.c | 101 +++++++++----------------------
hw/cris/axis_dev88.c | 8 +--
hw/hppa/machine.c | 10 +--
hw/i386/microvm.c | 12 ++--
hw/i386/pc.c | 19 +++---
hw/lm32/lm32_boards.c | 39 ++++++++----
hw/lm32/milkymist.c | 21 ++++---
hw/m68k/an5206.c | 5 +-
hw/m68k/mcf5208.c | 5 +-
hw/m68k/next-cube.c | 5 +-
hw/m68k/q800.c | 6 +-
hw/mips/boston.c | 11 ++--
hw/mips/mips_fulong2e.c | 15 ++---
hw/mips/mips_jazz.c | 7 +--
hw/mips/mips_malta.c | 10 ++-
hw/mips/mips_mipssim.c | 9 +--
hw/mips/mips_r4k.c | 12 ++--
hw/misc/aspeed_sdmc.c | 83 +++++++++++++++++++------
hw/ppc/e500.c | 17 +++---
hw/ppc/e500plat.c | 1 +
hw/ppc/mac_newworld.c | 6 +-
hw/ppc/mac_oldworld.c | 6 +-
hw/ppc/mpc8544ds.c | 1 +
hw/ppc/pnv.c | 8 +--
hw/ppc/ppc405_boards.c | 48 +++++++++------
hw/ppc/ppc440_bamboo.c | 12 ++--
hw/ppc/ppc4xx_devs.c | 63 +++++++++----------
hw/ppc/prep.c | 15 +++--
hw/ppc/sam460ex.c | 6 +-
hw/ppc/spapr.c | 8 +--
hw/ppc/virtex_ml507.c | 19 +++---
hw/s390x/s390-virtio-ccw.c | 7 +--
hw/sparc/leon3.c | 6 +-
hw/sparc/sun4m.c | 74 +++++++++++-----------
hw/sparc64/niagara.c | 7 +--
qemu-deprecated.texi | 9 ---
tests/qtest/numa-test.c | 138 ++++++++++++++++++++++--------------------
vl.c | 78 +++++++++++++++++++-----
79 files changed, 831 insertions(+), 801 deletions(-)
--
2.7.4
next reply other threads:[~2020-01-31 15:27 UTC|newest]
Thread overview: 102+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-31 15:08 Igor Mammedov [this message]
2020-01-31 15:08 ` [PATCH v4 01/80] numa: remove deprecated -mem-path fallback to anonymous RAM Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 02/80] machine: introduce memory-backend property Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 03/80] machine: alias -mem-path and -mem-prealloc into memory-foo backend Igor Mammedov
2020-02-03 9:04 ` Michael S. Tsirkin
2020-02-03 9:27 ` Igor Mammedov
2020-02-03 9:42 ` Michael S. Tsirkin
2020-02-03 10:40 ` Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 04/80] machine: introduce convenience MachineState::ram Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 05/80] initialize MachineState::ram in NUMA case Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 06/80] vl.c: move -m parsing after memory backends has been processed Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 07/80] vl.c: ensure that ram_size matches size of machine.memory-backend Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 08/80] alpha/dp264: use memdev for RAM Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 09/80] arm/aspeed: actually check RAM size Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 10/80] arm/aspeed: use memdev for RAM Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 11/80] arm/collie: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 12/80] arm/cubieboard: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 13/80] arm/digic_boards: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 14/80] arm/highbank: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 15/80] arm/imx25_pdk: drop RAM size fixup Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 16/80] arm/imx25_pdk: use memdev for RAM Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 17/80] arm/integratorcp: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 18/80] arm/kzm: drop RAM size fixup Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 19/80] arm/kzm: use memdev for RAM Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 20/80] arm/mcimx6ul-evk: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 21/80] arm/mcimx7d-sabre: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 22/80] arm/mps2-tz: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 23/80] arm/mps2: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 24/80] arm/musicpal: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 25/80] arm/nseries: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 26/80] arm/omap_sx1: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 27/80] arm/palm: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 28/80] arm/raspi: " Igor Mammedov
2020-01-31 15:08 ` [PATCH v4 29/80] arm/sabrelite: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 30/80] arm/sbsa-ref: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 31/80] arm/versatilepb: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 32/80] arm/vexpress: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 33/80] arm/virt: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 34/80] arm/xilinx_zynq: drop RAM size fixup Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 35/80] arm/xilinx_zynq: use memdev for RAM Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 36/80] arm/xlnx-versal-virt: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 37/80] arm/xlnx-zcu102: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 38/80] s390x/s390-virtio-ccw: " Igor Mammedov
2020-02-05 20:11 ` Halil Pasic
2020-02-06 13:15 ` Igor Mammedov
2020-02-10 19:33 ` Halil Pasic
2020-01-31 15:09 ` [PATCH v4 39/80] null-machine: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 40/80] cris/axis_dev88: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 41/80] hppa: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 42/80] x86/microvm: " Igor Mammedov
2020-02-03 8:58 ` Michael S. Tsirkin
2020-01-31 15:09 ` [PATCH v4 43/80] x86/pc: " Igor Mammedov
2020-02-03 9:07 ` Michael S. Tsirkin
2020-02-03 9:45 ` Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 44/80] lm32/lm32_boards: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 45/80] lm32/milkymist: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 46/80] m68k/an5206: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 47/80] m68k/q800: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 48/80] m68k/mcf5208: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 49/80] m68k/next-cube: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 50/80] mips/boston: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 51/80] mips/mips_fulong2e: drop RAM size fixup Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 52/80] mips/mips_fulong2e: use memdev for RAM Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 53/80] mips/mips_jazz: " Igor Mammedov
2020-02-09 16:41 ` Philippe Mathieu-Daudé
2020-02-10 15:06 ` [PATCH v5 81/80] mips/mips_jazz: add max ram size check Igor Mammedov
2020-02-10 17:04 ` Philippe Mathieu-Daudé
2020-01-31 15:09 ` [PATCH v4 54/80] mips/mips_malta: use memdev for RAM Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 55/80] mips/mips_mipssim: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 56/80] mips/mips_r4k: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 57/80] ppc/e500: drop RAM size fixup Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 58/80] ppc/e500: use memdev for RAM Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 59/80] ppc/mac_newworld: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 60/80] ppc/mac_oldworld: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 61/80] ppc/pnv: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 62/80] ppc/ppc405_boards: add RAM size checks Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 63/80] ppc/ppc405_boards: use memdev for RAM Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 64/80] ppc/{ppc440_bamboo, sam460ex}: drop RAM size fixup Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 65/80] ppc/{ppc440_bamboo, sam460ex}: use memdev for RAM Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 66/80] ppc/prep: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 67/80] ppc/spapr: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 68/80] ppc/virtex_ml507: remove unused arguments Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 69/80] ppc/virtex_ml507: use memdev for RAM Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 70/80] sparc/leon3: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 71/80] sparc/sun4m: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 72/80] sparc/niagara: " Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 73/80] remove no longer used memory_region_allocate_system_memory() Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 74/80] exec: cleanup qemu_minrampagesize()/qemu_maxrampagesize() Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 75/80] exec: drop bogus mem_path from qemu_ram_alloc_from_fd() Igor Mammedov
2020-02-07 14:42 ` Marc-André Lureau
2020-01-31 15:09 ` [PATCH v4 76/80] make mem_path local variable Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 77/80] hostmem: introduce "prealloc-threads" property Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 78/80] hostmem: fix strict bind policy Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 79/80] tests/numa-test: make top level args dynamic and g_autofree(cli) cleanups Igor Mammedov
2020-01-31 15:09 ` [PATCH v4 80/80] tests:numa-test: use explicit memdev to specify node RAM Igor Mammedov
2020-01-31 16:16 ` [PATCH v4 00/80] refactor main RAM allocation to use hostmem backend no-reply
2020-01-31 16:28 ` no-reply
2020-02-03 9:49 ` Igor Mammedov
2020-02-04 15:08 ` Igor Mammedov
2020-02-04 15:39 ` Igor Mammedov
2020-02-04 22:05 ` Paolo Bonzini
2020-02-05 13:20 ` Philippe Mathieu-Daudé
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1580483390-131164-1-git-send-email-imammedo@redhat.com \
--to=imammedo@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.