From: Igor Mammedov <imammedo@redhat.com>
To: qemu-devel@nongnu.org
Cc: Paolo Bonzini <pbonzini@redhat.com>
Subject: [PATCH v6 00/79] refactor main RAM allocation to use hostmem backend
Date: Wed, 19 Feb 2020 11:08:34 -0500 [thread overview]
Message-ID: <20200219160953.13771-1-imammedo@redhat.com> (raw)
v6:
- put back arm_boot_info::ram_size initialization removed by mistake in
'arm/collie: use memdev for RAM' and 'arm/palm: use memdev for RAM' patches
- 'arm/xilinx_zynq: drop RAM size fixup', use GiB instead of hex numbers
to make code/error message more readable
- do not mention 'follow up patches' in commit message of
'tests:numa-test: use explicit memdev to specify node RAM'
to avoid confusion.
- pick up new Reviewed-by-s
v5:
- pick up new Reviewed-by-s
- drop "ppc/prep: use memdev for RAM" since machine was removed
- add patch "mips/mips_jazz: add max ram size check"
- extend comment for "default_ram_id" to clarify that's setting it serves
as optin flag for -m support. [03/79]
- rebase "arm/raspi: use memdev for RAM" once more
v4:
- pick up new Reviewed-by-s
- Fix access to uninitialized pagesize/hpsize in
"[PATCH REPOST v3 74/80] exec: cleanup qemu_minrampagesize()/qemu_maxrampagesize()"
- Make explicitly provided memory-backend work by using string property instead
of link so it would be possible to delay access to the backend to the time
when backends are initialized.
- added new patches to make explicit backend work nice with -m and do sanity
check on ram_size
'vl.c: move -m parsing after memory backends has been processed'
'vl.c: ensure that ram_size matches size of machine.memory-backend"
all this ram_size business needs cleanup too, but that's out of the scope
of this series.
- include m68k/q800 board into conversion
- drop patches that were merged through other trees
v3:
- due to libvirt not being ready, postpone till 5.1
* [PATCH v2 82/86] numa: forbid '-numa node, mem' for 5.0 and newer machine types
and depended
[PATCH v2 86/86] numa: remove deprecated implicit RAM distribution between nodes
- drop as not related "[PATCH v2 85/86] numa: make exit() usage consistent"
- drop "[PATCH v2 76/86] post conversion default_ram_id cleanup"
so that default memory-backedend won't be created for boards that do not care
about -m. Which makes -m optin feature. We should decide what do in case
board doesn't use -m (but that's out of scope of this series)
- use object_register_sugar_prop() instead of hacking compat props directly
- simplified/reworked aspeed patches
- s/RAM_ADDR_FMT/size_to_str()/
- rename 'ram-memdev' property to 'memory-backend'
- minor fixes to numa-test
- fixes for issues noticed during review of
[PATCH v2 66/86] ppc/{ppc440_bamboo,sam460x}: drop RAM size fixup
v2:
- fix compile errors on mingw32 host by introducing RAM_ADDR_UFMT [11/86]
- replace "[PATCH 43/86] hppa: drop RAM size fixup" with alternative
patches made by Philippe (which effectively do the same thing but other
way around)
- ppc440: fix crash and add suggested valid RAM size in error output.
s/ppc4xx_sdram_adjust/ppc4xx_sdram_prep/ and simplify it by removing
not necessary nested loop
- rebase on current master due to new conflicts
Series removes ad hoc RAM allocation API (memory_region_allocate_system_memory)
and consolidates it around hostmem backend. It allows to
* resolve conflicts between global -mem-prealloc and hostmem's "policy" option
fixing premature allocation before binding policy is applied
* simplify complicated memory allocation routines which had to deal with 2 ways
to allocate RAM.
* it allows to reuse hostmem backends of a choice for main RAM without adding
extra CLI options to duplicate hostmem features.
Recent case was -mem-shared, to enable vhost-user on targets that don't
support hostmem backends [1] (ex: s390)
* move RAM allocation from individual boards into generic machine code and
provide them with prepared MemoryRegion.
* clean up deprecated NUMA features which were tied to the old API (see patches)
- "numa: remove deprecated -mem-path fallback to anonymous RAM"
- (POSTPONED, waiting on libvirt side) "forbid '-numa node,mem' for 5.0 and newer machine types"
- (POSTPONED) "numa: remove deprecated implicit RAM distribution between nodes"
Conversion introduces a new machine.memory-backend property and wrapper code that
aliases global -mem-path and -mem-alloc into automatically created hostmem
backend properties (provided memory-backend was not set explicitly given by user).
And then follows bulk of trivial patches that incrementally convert individual
boards to using machine.memory-backend provided MemoryRegion.
Board conversion typically involves:
* providing MachineClass::default_ram_size and MachineClass::default_ram_id
so generic code could create default backend if user didn't explicitly provide
memory-backend or -m options
* dropping memory_region_allocate_system_memory() call
* using convenience MachineState::ram MemoryRegion, which points to MemoryRegion
allocated by ram-memdev
On top of that for some boards:
* added missing ram_size checks (typically it were boards with fixed ram size)
* ram_size fixups were replaced by checks and hard errors, forcing user to
provide correct "-m" values instead of ignoring it and continuing running.
After all boards are converted the old API is removed and memory allocation
routines are cleaned up.
git tree for testing:
https://github.com/imammedo/qemu convert_main_ram_to_memdev_v6
previous rev:
https://github.com/imammedo/qemu convert_main_ram_to_memdev_v5
CC: Paolo Bonzini <pbonzini@redhat.com>
Igor Mammedov (79):
numa: remove deprecated -mem-path fallback to anonymous RAM
machine: introduce memory-backend property
machine: alias -mem-path and -mem-prealloc into memory-foo backend
machine: introduce convenience MachineState::ram
initialize MachineState::ram in NUMA case
vl.c: move -m parsing after memory backends has been processed
vl.c: ensure that ram_size matches size of machine.memory-backend
alpha/dp264: use memdev for RAM
arm/aspeed: actually check RAM size
arm/aspeed: use memdev for RAM
arm/collie: use memdev for RAM
arm/cubieboard: use memdev for RAM
arm/digic_boards: use memdev for RAM
arm/highbank: use memdev for RAM
arm/imx25_pdk: drop RAM size fixup
arm/imx25_pdk: use memdev for RAM
arm/integratorcp: use memdev for RAM
arm/kzm: drop RAM size fixup
arm/kzm: use memdev for RAM
arm/mcimx6ul-evk: use memdev for RAM
arm/mcimx7d-sabre: use memdev for RAM
arm/mps2-tz: use memdev for RAM
arm/mps2: use memdev for RAM
arm/musicpal: use memdev for RAM
arm/nseries: use memdev for RAM
arm/omap_sx1: use memdev for RAM
arm/palm: use memdev for RAM
arm/sabrelite: use memdev for RAM
arm/raspi: use memdev for RAM
arm/sbsa-ref: use memdev for RAM
arm/versatilepb: use memdev for RAM
arm/vexpress: use memdev for RAM
arm/virt: use memdev for RAM
arm/xilinx_zynq: drop RAM size fixup
arm/xilinx_zynq: use memdev for RAM
arm/xlnx-versal-virt: use memdev for RAM
arm/xlnx-zcu102: use memdev for RAM
s390x/s390-virtio-ccw: use memdev for RAM
null-machine: use memdev for RAM
cris/axis_dev88: use memdev for RAM
hppa: use memdev for RAM
x86/microvm: use memdev for RAM
x86/pc: use memdev for RAM
lm32/lm32_boards: use memdev for RAM
lm32/milkymist: use memdev for RAM
m68k/an5206: use memdev for RAM
m68k/q800: use memdev for RAM
m68k/mcf5208: use memdev for RAM
m68k/next-cube: use memdev for RAM
mips/boston: use memdev for RAM
mips/mips_fulong2e: drop RAM size fixup
mips/mips_fulong2e: use memdev for RAM
mips/mips_jazz: use memdev for RAM
mips/mips_jazz: add max ram size check
mips/mips_malta: use memdev for RAM
mips/mips_mipssim: use memdev for RAM
mips/mips_r4k: use memdev for RAM
ppc/e500: drop RAM size fixup
ppc/e500: use memdev for RAM
ppc/mac_newworld: use memdev for RAM
ppc/mac_oldworld: use memdev for RAM
ppc/pnv: use memdev for RAM
ppc/ppc405_boards: add RAM size checks
ppc/ppc405_boards: use memdev for RAM
ppc/{ppc440_bamboo, sam460ex}: drop RAM size fixup
ppc/{ppc440_bamboo, sam460ex}: use memdev for RAM
ppc/spapr: use memdev for RAM
ppc/virtex_ml507: use memdev for RAM
sparc/leon3: use memdev for RAM
sparc/sun4m: use memdev for RAM
sparc/niagara: use memdev for RAM
remove no longer used memory_region_allocate_system_memory()
exec: cleanup qemu_minrampagesize()/qemu_maxrampagesize()
exec: drop bogus mem_path from qemu_ram_alloc_from_fd()
make mem_path local variable
hostmem: introduce "prealloc-threads" property
hostmem: fix strict bind policy
tests/numa-test: make top level args dynamic and g_autofree(cli)
cleanups
tests:numa-test: use explicit memdev to specify node RAM
hw/alpha/alpha_sys.h | 2 +-
include/hw/boards.h | 54 ++++++-------
include/hw/misc/aspeed_sdmc.h | 1 +
include/hw/ppc/ppc4xx.h | 9 +--
include/sysemu/hostmem.h | 20 ++++-
include/sysemu/numa.h | 1 +
include/sysemu/sysemu.h | 2 -
backends/hostmem-file.c | 8 --
backends/hostmem-memfd.c | 1 -
backends/hostmem-ram.c | 2 -
backends/hostmem.c | 53 +++++++++----
exec.c | 64 ++--------------
hw/alpha/dp264.c | 3 +-
hw/alpha/typhoon.c | 8 +-
hw/arm/aspeed.c | 18 ++---
hw/arm/collie.c | 16 +++-
hw/arm/cubieboard.c | 25 ++----
hw/arm/digic_boards.c | 40 +++++-----
hw/arm/highbank.c | 10 +--
hw/arm/imx25_pdk.c | 19 ++---
hw/arm/integratorcp.c | 9 +--
hw/arm/kzm.c | 18 ++---
hw/arm/mcimx6ul-evk.c | 25 +++---
hw/arm/mcimx7d-sabre.c | 25 +++---
hw/arm/mps2-tz.c | 15 +++-
hw/arm/mps2.c | 15 +++-
hw/arm/musicpal.c | 18 +++--
hw/arm/nseries.c | 32 ++++----
hw/arm/omap_sx1.c | 20 +++--
hw/arm/palm.c | 19 +++--
hw/arm/raspi.c | 10 +--
hw/arm/sabrelite.c | 23 ++----
hw/arm/sbsa-ref.c | 7 +-
hw/arm/versatilepb.c | 7 +-
hw/arm/vexpress.c | 14 ++--
hw/arm/virt.c | 7 +-
hw/arm/xilinx_zynq.c | 20 +++--
hw/arm/xlnx-versal-virt.c | 7 +-
hw/arm/xlnx-zcu102.c | 7 +-
hw/core/machine.c | 48 ++++++++++++
hw/core/null-machine.c | 8 +-
hw/core/numa.c | 101 +++++++------------------
hw/cris/axis_dev88.c | 8 +-
hw/hppa/machine.c | 10 +--
hw/i386/microvm.c | 12 ++-
hw/i386/pc.c | 19 +++--
hw/lm32/lm32_boards.c | 39 ++++++----
hw/lm32/milkymist.c | 21 ++++--
hw/m68k/an5206.c | 5 +-
hw/m68k/mcf5208.c | 5 +-
hw/m68k/next-cube.c | 5 +-
hw/m68k/q800.c | 6 +-
hw/mips/boston.c | 11 ++-
hw/mips/mips_fulong2e.c | 15 ++--
hw/mips/mips_jazz.c | 12 ++-
hw/mips/mips_malta.c | 10 +--
hw/mips/mips_mipssim.c | 9 +--
hw/mips/mips_r4k.c | 12 +--
hw/misc/aspeed_sdmc.c | 83 +++++++++++++++-----
hw/ppc/e500.c | 17 ++---
hw/ppc/e500plat.c | 1 +
hw/ppc/mac_newworld.c | 6 +-
hw/ppc/mac_oldworld.c | 6 +-
hw/ppc/mpc8544ds.c | 1 +
hw/ppc/pnv.c | 8 +-
hw/ppc/ppc405_boards.c | 48 +++++++-----
hw/ppc/ppc440_bamboo.c | 12 ++-
hw/ppc/ppc4xx_devs.c | 67 ++++++++---------
hw/ppc/sam460ex.c | 6 +-
hw/ppc/spapr.c | 8 +-
hw/ppc/virtex_ml507.c | 12 ++-
hw/s390x/s390-virtio-ccw.c | 7 +-
hw/sparc/leon3.c | 6 +-
hw/sparc/sun4m.c | 74 +++++++++---------
hw/sparc64/niagara.c | 7 +-
qemu-deprecated.texi | 9 ---
tests/qtest/numa-test.c | 138 ++++++++++++++++++----------------
vl.c | 78 +++++++++++++++----
78 files changed, 829 insertions(+), 775 deletions(-)
--
2.18.1
next reply other threads:[~2020-02-19 16:13 UTC|newest]
Thread overview: 84+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-02-19 16:08 Igor Mammedov [this message]
2020-02-19 16:08 ` [PATCH v6 01/79] numa: remove deprecated -mem-path fallback to anonymous RAM Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 02/79] machine: introduce memory-backend property Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 03/79] machine: alias -mem-path and -mem-prealloc into memory-foo backend Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 04/79] machine: introduce convenience MachineState::ram Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 05/79] initialize MachineState::ram in NUMA case Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 06/79] vl.c: move -m parsing after memory backends has been processed Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 07/79] vl.c: ensure that ram_size matches size of machine.memory-backend Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 08/79] alpha/dp264: use memdev for RAM Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 09/79] arm/aspeed: actually check RAM size Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 10/79] arm/aspeed: use memdev for RAM Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 11/79] arm/collie: " Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 12/79] arm/cubieboard: " Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 13/79] arm/digic_boards: " Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 14/79] arm/highbank: " Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 15/79] arm/imx25_pdk: drop RAM size fixup Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 16/79] arm/imx25_pdk: use memdev for RAM Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 17/79] arm/integratorcp: " Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 18/79] arm/kzm: drop RAM size fixup Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 19/79] arm/kzm: use memdev for RAM Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 20/79] arm/mcimx6ul-evk: " Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 21/79] arm/mcimx7d-sabre: " Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 22/79] arm/mps2-tz: " Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 23/79] arm/mps2: " Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 24/79] arm/musicpal: " Igor Mammedov
2020-02-19 16:08 ` [PATCH v6 25/79] arm/nseries: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 26/79] arm/omap_sx1: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 27/79] arm/palm: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 28/79] arm/sabrelite: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 29/79] arm/raspi: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 30/79] arm/sbsa-ref: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 31/79] arm/versatilepb: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 32/79] arm/vexpress: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 33/79] arm/virt: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 34/79] arm/xilinx_zynq: drop RAM size fixup Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 35/79] arm/xilinx_zynq: use memdev for RAM Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 36/79] arm/xlnx-versal-virt: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 37/79] arm/xlnx-zcu102: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 38/79] s390x/s390-virtio-ccw: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 39/79] null-machine: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 40/79] cris/axis_dev88: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 41/79] hppa: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 42/79] x86/microvm: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 43/79] x86/pc: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 44/79] lm32/lm32_boards: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 45/79] lm32/milkymist: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 46/79] m68k/an5206: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 47/79] m68k/q800: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 48/79] m68k/mcf5208: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 49/79] m68k/next-cube: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 50/79] mips/boston: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 51/79] mips/mips_fulong2e: drop RAM size fixup Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 52/79] mips/mips_fulong2e: use memdev for RAM Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 53/79] mips/mips_jazz: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 54/79] mips/mips_jazz: add max ram size check Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 55/79] mips/mips_malta: use memdev for RAM Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 56/79] mips/mips_mipssim: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 57/79] mips/mips_r4k: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 58/79] ppc/e500: drop RAM size fixup Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 59/79] ppc/e500: use memdev for RAM Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 60/79] ppc/mac_newworld: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 61/79] ppc/mac_oldworld: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 62/79] ppc/pnv: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 63/79] ppc/ppc405_boards: add RAM size checks Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 64/79] ppc/ppc405_boards: use memdev for RAM Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 65/79] ppc/{ppc440_bamboo, sam460ex}: drop RAM size fixup Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 66/79] ppc/{ppc440_bamboo, sam460ex}: use memdev for RAM Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 67/79] ppc/spapr: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 68/79] ppc/virtex_ml507: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 69/79] sparc/leon3: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 70/79] sparc/sun4m: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 71/79] sparc/niagara: " Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 72/79] remove no longer used memory_region_allocate_system_memory() Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 73/79] exec: cleanup qemu_minrampagesize()/qemu_maxrampagesize() Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 74/79] exec: drop bogus mem_path from qemu_ram_alloc_from_fd() Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 75/79] make mem_path local variable Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 76/79] hostmem: introduce "prealloc-threads" property Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 77/79] hostmem: fix strict bind policy Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 78/79] tests/numa-test: make top level args dynamic and g_autofree(cli) cleanups Igor Mammedov
2020-02-19 16:09 ` [PATCH v6 79/79] tests:numa-test: use explicit memdev to specify node RAM Igor Mammedov
2020-02-19 16:58 ` [PATCH v6 00/79] refactor main RAM allocation to use hostmem backend no-reply
2020-02-24 8:45 ` Philippe Mathieu-Daudé
2020-02-24 11:33 ` Igor Mammedov
2020-02-24 11:38 ` Philippe Mathieu-Daudé
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200219160953.13771-1-imammedo@redhat.com \
--to=imammedo@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.