* [PULL 01/20] configure: Only clone softfloat-3 repositories if TCG is enabled
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 02/20] configure: check for submodules if --with-git-submodules=ignore Paolo Bonzini
` (20 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210512045821.3257963-1-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
configure | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/configure b/configure
index 0e4233fd8a..1bb29fcda1 100755
--- a/configure
+++ b/configure
@@ -257,8 +257,6 @@ if test -e "$source_path/.git"
then
git_submodules_action="update"
git_submodules="ui/keycodemapdb"
- git_submodules="$git_submodules tests/fp/berkeley-testfloat-3"
- git_submodules="$git_submodules tests/fp/berkeley-softfloat-3"
else
git_submodules_action="ignore"
git_submodules=""
@@ -2271,6 +2269,11 @@ if test "$solaris" = "yes" ; then
fi
fi
+if test "$tcg" = "enabled"; then
+ git_submodules="$git_submodules tests/fp/berkeley-testfloat-3"
+ git_submodules="$git_submodules tests/fp/berkeley-softfloat-3"
+fi
+
if test -z "${target_list+xxx}" ; then
default_targets=yes
for target in $default_target_list; do
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 02/20] configure: check for submodules if --with-git-submodules=ignore
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
2021-05-17 11:19 ` [PULL 01/20] configure: Only clone softfloat-3 repositories if TCG is enabled Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 03/20] configure: simplify assignment to GIT_SUBMODULES Paolo Bonzini
` (19 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel
Right now --with-git-submodules=ignore has a subtle difference from
just running without a .git directory, in that it does not check
that submodule sources actually exist. Move the check for
ui/keycodemapdb/README so that it happens even if the user
specified --with-git-submodules=ignore, with a customized
error message that is more suitable for this situation.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
configure | 40 ++++++++++++++++++++++------------------
1 file changed, 22 insertions(+), 18 deletions(-)
diff --git a/configure b/configure
index 1bb29fcda1..4681cbe2d7 100755
--- a/configure
+++ b/configure
@@ -260,24 +260,6 @@ then
else
git_submodules_action="ignore"
git_submodules=""
-
- if ! test -f "$source_path/ui/keycodemapdb/README"
- then
- echo
- echo "ERROR: missing file $source_path/ui/keycodemapdb/README"
- echo
- echo "This is not a GIT checkout but module content appears to"
- echo "be missing. Do not use 'git archive' or GitHub download links"
- echo "to acquire QEMU source archives. Non-GIT builds are only"
- echo "supported with source archives linked from:"
- echo
- echo " https://www.qemu.org/download/#source"
- echo
- echo "Developers working with GIT can use scripts/archive-source.sh"
- echo "if they need to create valid source archives."
- echo
- exit 1
- fi
fi
git="git"
@@ -1582,6 +1564,28 @@ case $git_submodules_action in
fi
;;
ignore)
+ if ! test -f "$source_path/ui/keycodemapdb/README"
+ then
+ echo
+ echo "ERROR: missing GIT submodules"
+ echo
+ if test -e "$source_path/.git"; then
+ echo "--with-git-submodules=ignore specified but submodules were not"
+ echo "checked out. Please initialize and update submodules."
+ else
+ echo "This is not a GIT checkout but module content appears to"
+ echo "be missing. Do not use 'git archive' or GitHub download links"
+ echo "to acquire QEMU source archives. Non-GIT builds are only"
+ echo "supported with source archives linked from:"
+ echo
+ echo " https://www.qemu.org/download/#source"
+ echo
+ echo "Developers working with GIT can use scripts/archive-source.sh"
+ echo "if they need to create valid source archives."
+ fi
+ echo
+ exit 1
+ fi
;;
*)
echo "ERROR: invalid --with-git-submodules= value '$git_submodules_action'"
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 03/20] configure: simplify assignment to GIT_SUBMODULES
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
2021-05-17 11:19 ` [PULL 01/20] configure: Only clone softfloat-3 repositories if TCG is enabled Paolo Bonzini
2021-05-17 11:19 ` [PULL 02/20] configure: check for submodules if --with-git-submodules=ignore Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 04/20] hw/mem/nvdimm: Use Kconfig 'imply' instead of 'depends on' Paolo Bonzini
` (18 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel
Do not guard each assignment with a check for --with-git-submodules=ignore.
To avoid a confusing "GIT" line from the Makefile, guard the git-submodule-update
recipe so that it is empty when --with-git-submodules=ignore.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
Makefile | 2 ++
configure | 20 ++++++--------------
2 files changed, 8 insertions(+), 14 deletions(-)
diff --git a/Makefile b/Makefile
index 4cab10a2a4..30f19d33bb 100644
--- a/Makefile
+++ b/Makefile
@@ -48,9 +48,11 @@ Makefile: .git-submodule-status
.PHONY: git-submodule-update
git-submodule-update:
+ifneq ($(GIT_SUBMODULES_ACTION),ignore)
$(call quiet-command, \
(GIT="$(GIT)" "$(SRC_PATH)/scripts/git-submodule.sh" $(GIT_SUBMODULES_ACTION) $(GIT_SUBMODULES)), \
"GIT","$(GIT_SUBMODULES)")
+endif
# 0. ensure the build tree is okay
diff --git a/configure b/configure
index 4681cbe2d7..55049fe930 100755
--- a/configure
+++ b/configure
@@ -256,11 +256,11 @@ gdb_bin=$(command -v "gdb-multiarch" || command -v "gdb")
if test -e "$source_path/.git"
then
git_submodules_action="update"
- git_submodules="ui/keycodemapdb"
else
git_submodules_action="ignore"
- git_submodules=""
fi
+
+git_submodules="ui/keycodemapdb"
git="git"
# Don't accept a target_list environment variable.
@@ -3617,9 +3617,7 @@ fi
case "$fdt" in
auto | enabled | internal)
# Simpler to always update submodule, even if not needed.
- if test "$git_submodules_action" != "ignore"; then
- git_submodules="${git_submodules} dtc"
- fi
+ git_submodules="${git_submodules} dtc"
;;
esac
@@ -4328,9 +4326,7 @@ fi
case "$capstone" in
auto | enabled | internal)
# Simpler to always update submodule, even if not needed.
- if test "$git_submodules_action" != "ignore"; then
- git_submodules="${git_submodules} capstone"
- fi
+ git_submodules="${git_submodules} capstone"
;;
esac
@@ -5260,9 +5256,7 @@ fi
case "$slirp" in
auto | enabled | internal)
# Simpler to always update submodule, even if not needed.
- if test "$git_submodules_action" != "ignore"; then
- git_submodules="${git_submodules} slirp"
- fi
+ git_submodules="${git_submodules} slirp"
;;
esac
@@ -5454,9 +5448,7 @@ if test "$cpu" = "s390x" ; then
roms="$roms s390-ccw"
# SLOF is required for building the s390-ccw firmware on s390x,
# since it is using the libnet code from SLOF for network booting.
- if test "$git_submodules_action" != "ignore"; then
- git_submodules="${git_submodules} roms/SLOF"
- fi
+ git_submodules="${git_submodules} roms/SLOF"
fi
fi
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 04/20] hw/mem/nvdimm: Use Kconfig 'imply' instead of 'depends on'
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (2 preceding siblings ...)
2021-05-17 11:19 ` [PULL 03/20] configure: simplify assignment to GIT_SUBMODULES Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 05/20] backends/tpm: Replace qemu_mutex_lock calls with QEMU_LOCK_GUARD Paolo Bonzini
` (17 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Philippe Mathieu-Daudé
From: Philippe Mathieu-Daudé <philmd@redhat.com>
Per the kconfig.rst:
A device should be listed [...] ``imply`` if (depending on
the QEMU command line) the board may or may not be started
without it.
This is the case with the NVDIMM device, so use the 'imply'
weak reverse dependency to select the symbol.
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210511155354.3069141-2-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
default-configs/devices/ppc64-softmmu.mak | 1 -
hw/arm/Kconfig | 1 +
hw/i386/Kconfig | 1 +
hw/mem/Kconfig | 2 --
hw/ppc/Kconfig | 1 +
5 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/default-configs/devices/ppc64-softmmu.mak b/default-configs/devices/ppc64-softmmu.mak
index ae0841fa3a..cca52665d9 100644
--- a/default-configs/devices/ppc64-softmmu.mak
+++ b/default-configs/devices/ppc64-softmmu.mak
@@ -8,4 +8,3 @@ CONFIG_POWERNV=y
# For pSeries
CONFIG_PSERIES=y
-CONFIG_NVDIMM=y
diff --git a/hw/arm/Kconfig b/hw/arm/Kconfig
index b887f6a5b1..67723d9ea6 100644
--- a/hw/arm/Kconfig
+++ b/hw/arm/Kconfig
@@ -6,6 +6,7 @@ config ARM_VIRT
imply VFIO_PLATFORM
imply VFIO_XGMAC
imply TPM_TIS_SYSBUS
+ imply NVDIMM
select ARM_GIC
select ACPI
select ARM_SMMUV3
diff --git a/hw/i386/Kconfig b/hw/i386/Kconfig
index 7f91f30877..66838fa397 100644
--- a/hw/i386/Kconfig
+++ b/hw/i386/Kconfig
@@ -23,6 +23,7 @@ config PC
imply TPM_TIS_ISA
imply VGA_PCI
imply VIRTIO_VGA
+ imply NVDIMM
select FDC
select I8259
select I8254
diff --git a/hw/mem/Kconfig b/hw/mem/Kconfig
index a0ef2cf648..8b19fdc49f 100644
--- a/hw/mem/Kconfig
+++ b/hw/mem/Kconfig
@@ -7,6 +7,4 @@ config MEM_DEVICE
config NVDIMM
bool
- default y
- depends on (PC || PSERIES || ARM_VIRT)
select MEM_DEVICE
diff --git a/hw/ppc/Kconfig b/hw/ppc/Kconfig
index e51e0e5e5a..66e0b15d9e 100644
--- a/hw/ppc/Kconfig
+++ b/hw/ppc/Kconfig
@@ -3,6 +3,7 @@ config PSERIES
imply PCI_DEVICES
imply TEST_DEVICES
imply VIRTIO_VGA
+ imply NVDIMM
select DIMM
select PCI
select SPAPR_VSCSI
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 05/20] backends/tpm: Replace qemu_mutex_lock calls with QEMU_LOCK_GUARD
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (3 preceding siblings ...)
2021-05-17 11:19 ` [PULL 04/20] hw/mem/nvdimm: Use Kconfig 'imply' instead of 'depends on' Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 06/20] i386/cpu: Expose AVX_VNNI instruction to guest Paolo Bonzini
` (16 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel
Cc: Christophe de Dinechin, Philippe Mathieu-Daudé, Stefan Berger
From: Philippe Mathieu-Daudé <philmd@redhat.com>
Simplify the tpm_emulator_ctrlcmd() handler by replacing a pair of
qemu_mutex_lock/qemu_mutex_unlock calls by the WITH_QEMU_LOCK_GUARD
macro.
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
Reviewed-by: Christophe de Dinechin <dinechin@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210512070713.3286188-1-philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
backends/tpm/tpm_emulator.c | 34 +++++++++++++++-------------------
1 file changed, 15 insertions(+), 19 deletions(-)
diff --git a/backends/tpm/tpm_emulator.c b/backends/tpm/tpm_emulator.c
index a012adc193..e5f1063ab6 100644
--- a/backends/tpm/tpm_emulator.c
+++ b/backends/tpm/tpm_emulator.c
@@ -30,6 +30,7 @@
#include "qemu/error-report.h"
#include "qemu/module.h"
#include "qemu/sockets.h"
+#include "qemu/lockable.h"
#include "io/channel-socket.h"
#include "sysemu/tpm_backend.h"
#include "sysemu/tpm_util.h"
@@ -124,31 +125,26 @@ static int tpm_emulator_ctrlcmd(TPMEmulator *tpm, unsigned long cmd, void *msg,
uint32_t cmd_no = cpu_to_be32(cmd);
ssize_t n = sizeof(uint32_t) + msg_len_in;
uint8_t *buf = NULL;
- int ret = -1;
- qemu_mutex_lock(&tpm->mutex);
+ WITH_QEMU_LOCK_GUARD(&tpm->mutex) {
+ buf = g_alloca(n);
+ memcpy(buf, &cmd_no, sizeof(cmd_no));
+ memcpy(buf + sizeof(cmd_no), msg, msg_len_in);
- buf = g_alloca(n);
- memcpy(buf, &cmd_no, sizeof(cmd_no));
- memcpy(buf + sizeof(cmd_no), msg, msg_len_in);
-
- n = qemu_chr_fe_write_all(dev, buf, n);
- if (n <= 0) {
- goto end;
- }
-
- if (msg_len_out != 0) {
- n = qemu_chr_fe_read_all(dev, msg, msg_len_out);
+ n = qemu_chr_fe_write_all(dev, buf, n);
if (n <= 0) {
- goto end;
+ return -1;
}
- }
- ret = 0;
+ if (msg_len_out != 0) {
+ n = qemu_chr_fe_read_all(dev, msg, msg_len_out);
+ if (n <= 0) {
+ return -1;
+ }
+ }
+ }
-end:
- qemu_mutex_unlock(&tpm->mutex);
- return ret;
+ return 0;
}
static int tpm_emulator_unix_tx_bufs(TPMEmulator *tpm_emu,
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 06/20] i386/cpu: Expose AVX_VNNI instruction to guest
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (4 preceding siblings ...)
2021-05-17 11:19 ` [PULL 05/20] backends/tpm: Replace qemu_mutex_lock calls with QEMU_LOCK_GUARD Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 07/20] meson: bump submodule to 0.57.2 Paolo Bonzini
` (15 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Yang Zhong
From: Yang Zhong <yang.zhong@intel.com>
Expose AVX (VEX-encoded) versions of the Vector Neural Network
Instructions to guest.
The bit definition:
CPUID.(EAX=7,ECX=1):EAX[bit 4] AVX_VNNI
The following instructions are available when this feature is
present in the guest.
1. VPDPBUS: Multiply and Add Unsigned and Signed Bytes
2. VPDPBUSDS: Multiply and Add Unsigned and Signed Bytes with Saturation
3. VPDPWSSD: Multiply and Add Signed Word Integers
4. VPDPWSSDS: Multiply and Add Signed Integers with Saturation
As for the kvm related code, please reference Linux commit id 1085a6b585d7.
The release document ref below link:
https://software.intel.com/content/www/us/en/develop/download/\
intel-architecture-instruction-set-extensions-programming-reference.html
Signed-off-by: Yang Zhong <yang.zhong@intel.com>
Message-Id: <20210407015609.22936-1-yang.zhong@intel.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
target/i386/cpu.c | 4 ++--
target/i386/cpu.h | 2 ++
2 files changed, 4 insertions(+), 2 deletions(-)
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index c496bfa1c2..9e211ac2ce 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -940,7 +940,7 @@ FeatureWordInfo feature_word_info[FEATURE_WORDS] = {
.type = CPUID_FEATURE_WORD,
.feat_names = {
NULL, NULL, NULL, NULL,
- NULL, "avx512-bf16", NULL, NULL,
+ "avx-vnni", "avx512-bf16", NULL, NULL,
NULL, NULL, NULL, NULL,
NULL, NULL, NULL, NULL,
NULL, NULL, NULL, NULL,
@@ -3194,7 +3194,7 @@ static X86CPUDefinition builtin_x86_defs[] = {
MSR_ARCH_CAP_SKIP_L1DFL_VMENTRY | MSR_ARCH_CAP_MDS_NO |
MSR_ARCH_CAP_PSCHANGE_MC_NO | MSR_ARCH_CAP_TAA_NO,
.features[FEAT_7_1_EAX] =
- CPUID_7_1_EAX_AVX512_BF16,
+ CPUID_7_1_EAX_AVX_VNNI | CPUID_7_1_EAX_AVX512_BF16,
/*
* Missing: XSAVES (not supported by some Linux versions,
* including v4.1 to v4.12).
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
index 324ef92beb..d4e80e1344 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
@@ -817,6 +817,8 @@ typedef uint64_t FeatureWordArray[FEATURE_WORDS];
/* Speculative Store Bypass Disable */
#define CPUID_7_0_EDX_SPEC_CTRL_SSBD (1U << 31)
+/* AVX VNNI Instruction */
+#define CPUID_7_1_EAX_AVX_VNNI (1U << 4)
/* AVX512 BFloat16 Instruction */
#define CPUID_7_1_EAX_AVX512_BF16 (1U << 5)
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 07/20] meson: bump submodule to 0.57.2
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (5 preceding siblings ...)
2021-05-17 11:19 ` [PULL 06/20] i386/cpu: Expose AVX_VNNI instruction to guest Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 08/20] object: add more commands to preconfig mode Paolo Bonzini
` (14 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel
The main advantage of 0.57 is that it fixes
https://github.com/mesonbuild/meson/pull/7900, thus avoiding unnecessary
rebuilds after running meson.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
meson | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meson b/meson
index 776acd2a80..6f745e980b 160000
--- a/meson
+++ b/meson
@@ -1 +1 @@
-Subproject commit 776acd2a805c9b42b4f0375150977df42130317f
+Subproject commit 6f745e980bcab036104c6b7f9064191e489012b9
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 08/20] object: add more commands to preconfig mode
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (6 preceding siblings ...)
2021-05-17 11:19 ` [PULL 07/20] meson: bump submodule to 0.57.2 Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 09/20] qtest: add a QOM object for qtest Paolo Bonzini
` (13 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel
Creating and destroying QOM objects does not require a fully constructed
machine. Allow running object-add and object-del before machine
initialization has concluded.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
hmp-commands.hx | 2 ++
qapi/qom.json | 6 ++++--
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/hmp-commands.hx b/hmp-commands.hx
index 435c591a1c..146a13c896 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1297,6 +1297,7 @@ ERST
.help = "create QOM object",
.cmd = hmp_object_add,
.command_completion = object_add_completion,
+ .flags = "p",
},
SRST
@@ -1311,6 +1312,7 @@ ERST
.help = "destroy QOM object",
.cmd = hmp_object_del,
.command_completion = object_del_completion,
+ .flags = "p",
},
SRST
diff --git a/qapi/qom.json b/qapi/qom.json
index cd0e76d564..f1e243dbe5 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -857,7 +857,8 @@
# <- { "return": {} }
#
##
-{ 'command': 'object-add', 'data': 'ObjectOptions', 'boxed': true }
+{ 'command': 'object-add', 'data': 'ObjectOptions', 'boxed': true,
+ 'allow-preconfig': true }
##
# @object-del:
@@ -877,4 +878,5 @@
# <- { "return": {} }
#
##
-{ 'command': 'object-del', 'data': {'id': 'str'} }
+{ 'command': 'object-del', 'data': {'id': 'str'},
+ 'allow-preconfig': true }
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 09/20] qtest: add a QOM object for qtest
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (7 preceding siblings ...)
2021-05-17 11:19 ` [PULL 08/20] object: add more commands to preconfig mode Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 10/20] KVM: do not allow setting properties at runtime Paolo Bonzini
` (12 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel
The qtest server right now can only be created using the -qtest
and -qtest-log options. Allow an alternative way to create it
using "-object qtest,chardev=...,log=...".
This is part of the long term plan to make more (or all) of
QEMU configurable through QMP and preconfig mode.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
qapi/qom.json | 17 +++++
softmmu/qtest.c | 185 +++++++++++++++++++++++++++++++++++++++++++++---
softmmu/vl.c | 5 +-
3 files changed, 196 insertions(+), 11 deletions(-)
diff --git a/qapi/qom.json b/qapi/qom.json
index f1e243dbe5..bcb7e6ba51 100644
--- a/qapi/qom.json
+++ b/qapi/qom.json
@@ -644,6 +644,21 @@
{ 'struct': 'PrManagerHelperProperties',
'data': { 'path': 'str' } }
+##
+# @QtestProperties:
+#
+# Properties for qtest objects.
+#
+# @chardev: the chardev to be used to receive qtest commands on.
+#
+# @log: the path to a log file
+#
+# Since: 6.0
+##
+{ 'struct': 'QtestProperties',
+ 'data': { 'chardev': 'str',
+ '*log': 'str' } }
+
##
# @RemoteObjectProperties:
#
@@ -769,6 +784,7 @@
'memory-backend-ram',
'pef-guest',
'pr-manager-helper',
+ 'qtest',
'rng-builtin',
'rng-egd',
'rng-random',
@@ -825,6 +841,7 @@
'if': 'defined(CONFIG_LINUX)' },
'memory-backend-ram': 'MemoryBackendProperties',
'pr-manager-helper': 'PrManagerHelperProperties',
+ 'qtest': 'QtestProperties',
'rng-builtin': 'RngProperties',
'rng-egd': 'RngEgdProperties',
'rng-random': 'RngRandomProperties',
diff --git a/softmmu/qtest.c b/softmmu/qtest.c
index 130c366615..72751e1fd8 100644
--- a/softmmu/qtest.c
+++ b/softmmu/qtest.c
@@ -27,6 +27,8 @@
#include "qemu/error-report.h"
#include "qemu/module.h"
#include "qemu/cutils.h"
+#include "qapi/qmp/qerror.h"
+#include "qom/object_interfaces.h"
#include CONFIG_DEVICES
#ifdef CONFIG_PSERIES
#include "hw/ppc/spapr_rtas.h"
@@ -34,11 +36,25 @@
#define MAX_IRQ 256
+#define TYPE_QTEST "qtest"
+
+OBJECT_DECLARE_SIMPLE_TYPE(QTest, QTEST)
+
+struct QTest {
+ Object parent;
+
+ bool has_machine_link;
+ char *chr_name;
+ Chardev *chr;
+ CharBackend qtest_chr;
+ char *log;
+};
+
bool qtest_allowed;
static DeviceState *irq_intercept_dev;
static FILE *qtest_log_fp;
-static CharBackend qtest_chr;
+static QTest *qtest;
static GString *inbuf;
static int irq_levels[MAX_IRQ];
static qemu_timeval start_time;
@@ -320,7 +336,7 @@ static void qtest_irq_handler(void *opaque, int n, int level)
qemu_set_irq(old_irq, level);
if (irq_levels[n] != level) {
- CharBackend *chr = &qtest_chr;
+ CharBackend *chr = &qtest->qtest_chr;
irq_levels[n] = level;
qtest_send_prefix(chr);
qtest_sendf(chr, "IRQ %s %d\n",
@@ -849,18 +865,39 @@ static void qtest_event(void *opaque, QEMUChrEvent event)
break;
}
}
+
void qtest_server_init(const char *qtest_chrdev, const char *qtest_log, Error **errp)
{
+ ERRP_GUARD();
Chardev *chr;
+ Object *qtest;
chr = qemu_chr_new("qtest", qtest_chrdev, NULL);
-
if (chr == NULL) {
error_setg(errp, "Failed to initialize device for qtest: \"%s\"",
qtest_chrdev);
return;
}
+ qtest = object_new(TYPE_QTEST);
+ object_property_set_str(qtest, "chardev", "qtest", &error_abort);
+ if (qtest_log) {
+ object_property_set_str(qtest, "log", qtest_log, &error_abort);
+ }
+ object_property_add_child(qdev_get_machine(), "qtest", qtest);
+ user_creatable_complete(USER_CREATABLE(qtest), errp);
+ if (*errp) {
+ object_unparent(qtest);
+ }
+ object_unref(OBJECT(chr));
+ object_unref(qtest);
+}
+
+static bool qtest_server_start(QTest *q, Error **errp)
+{
+ Chardev *chr = q->chr;
+ const char *qtest_log = q->log;
+
if (qtest_log) {
if (strcmp(qtest_log, "none") != 0) {
qtest_log_fp = fopen(qtest_log, "w+");
@@ -869,16 +906,20 @@ void qtest_server_init(const char *qtest_chrdev, const char *qtest_log, Error **
qtest_log_fp = stderr;
}
- qemu_chr_fe_init(&qtest_chr, chr, errp);
- qemu_chr_fe_set_handlers(&qtest_chr, qtest_can_read, qtest_read,
- qtest_event, NULL, &qtest_chr, NULL, true);
- qemu_chr_fe_set_echo(&qtest_chr, true);
+ if (!qemu_chr_fe_init(&q->qtest_chr, chr, errp)) {
+ return false;
+ }
+ qemu_chr_fe_set_handlers(&q->qtest_chr, qtest_can_read, qtest_read,
+ qtest_event, NULL, &q->qtest_chr, NULL, true);
+ qemu_chr_fe_set_echo(&q->qtest_chr, true);
inbuf = g_string_new("");
if (!qtest_server_send) {
- qtest_server_set_send_handler(qtest_server_char_be_send, &qtest_chr);
+ qtest_server_set_send_handler(qtest_server_char_be_send, &q->qtest_chr);
}
+ qtest = q;
+ return true;
}
void qtest_server_set_send_handler(void (*send)(void*, const char*),
@@ -890,7 +931,7 @@ void qtest_server_set_send_handler(void (*send)(void*, const char*),
bool qtest_driver(void)
{
- return qtest_chr.chr != NULL;
+ return qtest && qtest->qtest_chr.chr != NULL;
}
void qtest_server_inproc_recv(void *dummy, const char *buf)
@@ -905,3 +946,129 @@ void qtest_server_inproc_recv(void *dummy, const char *buf)
g_string_truncate(gstr, 0);
}
}
+
+static void qtest_complete(UserCreatable *uc, Error **errp)
+{
+ QTest *q = QTEST(uc);
+ if (qtest) {
+ error_setg(errp, "Only one instance of qtest can be created");
+ return;
+ }
+ if (!q->chr_name) {
+ error_setg(errp, "No backend specified");
+ return;
+ }
+
+ if (OBJECT(uc)->parent != qdev_get_machine()) {
+ q->has_machine_link = true;
+ object_property_add_const_link(qdev_get_machine(), "qtest", OBJECT(uc));
+ } else {
+ /* -qtest was used. */
+ }
+
+ qtest_server_start(q, errp);
+}
+
+static void qtest_unparent(Object *obj)
+{
+ QTest *q = QTEST(obj);
+
+ if (qtest == q) {
+ qemu_chr_fe_disconnect(&q->qtest_chr);
+ assert(!qtest_opened);
+ qemu_chr_fe_deinit(&q->qtest_chr, false);
+ if (qtest_log_fp) {
+ fclose(qtest_log_fp);
+ qtest_log_fp = NULL;
+ }
+ qtest = NULL;
+ }
+
+ if (q->has_machine_link) {
+ object_property_del(qdev_get_machine(), "qtest");
+ q->has_machine_link = false;
+ }
+}
+
+static void qtest_set_log(Object *obj, const char *value, Error **errp)
+{
+ QTest *q = QTEST(obj);
+
+ if (qtest == q) {
+ error_setg(errp, QERR_PERMISSION_DENIED);
+ } else {
+ g_free(q->log);
+ q->log = g_strdup(value);
+ }
+}
+
+static char *qtest_get_log(Object *obj, Error **errp)
+{
+ QTest *q = QTEST(obj);
+
+ return g_strdup(q->log);
+}
+
+static void qtest_set_chardev(Object *obj, const char *value, Error **errp)
+{
+ QTest *q = QTEST(obj);
+ Chardev *chr;
+
+ if (qtest == q) {
+ error_setg(errp, QERR_PERMISSION_DENIED);
+ return;
+ }
+
+ chr = qemu_chr_find(value);
+ if (!chr) {
+ error_setg(errp, "Cannot find character device '%s'", value);
+ return;
+ }
+
+ g_free(q->chr_name);
+ q->chr_name = g_strdup(value);
+
+ if (q->chr) {
+ object_unref(q->chr);
+ }
+ q->chr = chr;
+ object_ref(chr);
+}
+
+static char *qtest_get_chardev(Object *obj, Error **errp)
+{
+ QTest *q = QTEST(obj);
+
+ return g_strdup(q->chr_name);
+}
+
+static void qtest_class_init(ObjectClass *oc, void *data)
+{
+ UserCreatableClass *ucc = USER_CREATABLE_CLASS(oc);
+
+ oc->unparent = qtest_unparent;
+ ucc->complete = qtest_complete;
+
+ object_class_property_add_str(oc, "chardev",
+ qtest_get_chardev, qtest_set_chardev);
+ object_class_property_add_str(oc, "log",
+ qtest_get_log, qtest_set_log);
+}
+
+static const TypeInfo qtest_info = {
+ .name = TYPE_QTEST,
+ .parent = TYPE_OBJECT,
+ .class_init = qtest_class_init,
+ .instance_size = sizeof(QTest),
+ .interfaces = (InterfaceInfo[]) {
+ { TYPE_USER_CREATABLE },
+ { }
+ }
+};
+
+static void register_types(void)
+{
+ type_register_static(&qtest_info);
+}
+
+type_init(register_types);
diff --git a/softmmu/vl.c b/softmmu/vl.c
index 93e78469bc..11ac3750d8 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
@@ -1758,8 +1758,9 @@ static bool object_create_early(const char *type)
* add one, state the reason in a comment!
*/
- /* Reason: rng-egd property "chardev" */
- if (g_str_equal(type, "rng-egd")) {
+ /* Reason: property "chardev" */
+ if (g_str_equal(type, "rng-egd") ||
+ g_str_equal(type, "qtest")) {
return false;
}
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 10/20] KVM: do not allow setting properties at runtime
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (8 preceding siblings ...)
2021-05-17 11:19 ` [PULL 09/20] qtest: add a QOM object for qtest Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 11/20] memory: Introduce log_sync_global() to memory listener Paolo Bonzini
` (11 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel
Only allow accelerator properties to be set when the
accelerator is being created.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 4e0168e88b..54c0860979 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2017,7 +2017,6 @@ static int kvm_init(MachineState *ms)
QTAILQ_INIT(&s->kvm_sw_breakpoints);
#endif
QLIST_INIT(&s->kvm_parked_vcpus);
- s->vmfd = -1;
s->fd = qemu_open_old("/dev/kvm", O_RDWR);
if (s->fd == -1) {
fprintf(stderr, "Could not access KVM kernel module: %m\n");
@@ -3112,6 +3111,11 @@ static void kvm_set_kvm_shadow_mem(Object *obj, Visitor *v,
KVMState *s = KVM_STATE(obj);
int64_t value;
+ if (s->fd != -1) {
+ error_setg(errp, "Cannot set properties after the accelerator has been initialized");
+ return;
+ }
+
if (!visit_type_int(v, name, &value, errp)) {
return;
}
@@ -3126,6 +3130,11 @@ static void kvm_set_kernel_irqchip(Object *obj, Visitor *v,
KVMState *s = KVM_STATE(obj);
OnOffSplit mode;
+ if (s->fd != -1) {
+ error_setg(errp, "Cannot set properties after the accelerator has been initialized");
+ return;
+ }
+
if (!visit_type_OnOffSplit(v, name, &mode, errp)) {
return;
}
@@ -3172,6 +3181,8 @@ static void kvm_accel_instance_init(Object *obj)
{
KVMState *s = KVM_STATE(obj);
+ s->fd = -1;
+ s->vmfd = -1;
s->kvm_shadow_mem = -1;
s->kernel_irqchip_allowed = true;
s->kernel_irqchip_split = ON_OFF_AUTO_AUTO;
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 11/20] memory: Introduce log_sync_global() to memory listener
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (9 preceding siblings ...)
2021-05-17 11:19 ` [PULL 10/20] KVM: do not allow setting properties at runtime Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 12/20] KVM: Use a big lock to replace per-kml slots_lock Paolo Bonzini
` (10 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Dr . David Alan Gilbert, Peter Xu
From: Peter Xu <peterx@redhat.com>
Some of the memory listener may want to do log synchronization without
being able to specify a range of memory to sync but always globally.
Such a memory listener should provide this new method instead of the
log_sync() method.
Obviously we can also achieve similar thing when we put the global
sync logic into a log_sync() handler. However that's not efficient
enough because otherwise memory_global_dirty_log_sync() may do the
global sync N times, where N is the number of flat ranges in the
address space.
Make this new method be exclusive to log_sync().
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210506160549.130416-2-peterx@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
include/exec/memory.h | 12 ++++++++++++
softmmu/memory.c | 33 +++++++++++++++++++++++----------
2 files changed, 35 insertions(+), 10 deletions(-)
diff --git a/include/exec/memory.h b/include/exec/memory.h
index c8b9088924..e38b7e3dce 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -616,6 +616,18 @@ struct MemoryListener {
*/
void (*log_sync)(MemoryListener *listener, MemoryRegionSection *section);
+ /**
+ * @log_sync_global:
+ *
+ * This is the global version of @log_sync when the listener does
+ * not have a way to synchronize the log with finer granularity.
+ * When the listener registers with @log_sync_global defined, then
+ * its @log_sync must be NULL. Vice versa.
+ *
+ * @listener: The #MemoryListener.
+ */
+ void (*log_sync_global)(MemoryListener *listener);
+
/**
* @log_clear:
*
diff --git a/softmmu/memory.c b/softmmu/memory.c
index 3bb533c0bc..c19b0be6b1 100644
--- a/softmmu/memory.c
+++ b/softmmu/memory.c
@@ -2055,6 +2055,10 @@ void memory_region_set_dirty(MemoryRegion *mr, hwaddr addr,
memory_region_get_dirty_log_mask(mr));
}
+/*
+ * If memory region `mr' is NULL, do global sync. Otherwise, sync
+ * dirty bitmap for the specified memory region.
+ */
static void memory_region_sync_dirty_bitmap(MemoryRegion *mr)
{
MemoryListener *listener;
@@ -2068,18 +2072,24 @@ static void memory_region_sync_dirty_bitmap(MemoryRegion *mr)
* address space once.
*/
QTAILQ_FOREACH(listener, &memory_listeners, link) {
- if (!listener->log_sync) {
- continue;
- }
- as = listener->address_space;
- view = address_space_get_flatview(as);
- FOR_EACH_FLAT_RANGE(fr, view) {
- if (fr->dirty_log_mask && (!mr || fr->mr == mr)) {
- MemoryRegionSection mrs = section_from_flat_range(fr, view);
- listener->log_sync(listener, &mrs);
+ if (listener->log_sync) {
+ as = listener->address_space;
+ view = address_space_get_flatview(as);
+ FOR_EACH_FLAT_RANGE(fr, view) {
+ if (fr->dirty_log_mask && (!mr || fr->mr == mr)) {
+ MemoryRegionSection mrs = section_from_flat_range(fr, view);
+ listener->log_sync(listener, &mrs);
+ }
}
+ flatview_unref(view);
+ } else if (listener->log_sync_global) {
+ /*
+ * No matter whether MR is specified, what we can do here
+ * is to do a global sync, because we are not capable to
+ * sync in a finer granularity.
+ */
+ listener->log_sync_global(listener);
}
- flatview_unref(view);
}
}
@@ -2767,6 +2777,9 @@ void memory_listener_register(MemoryListener *listener, AddressSpace *as)
{
MemoryListener *other = NULL;
+ /* Only one of them can be defined for a listener */
+ assert(!(listener->log_sync && listener->log_sync_global));
+
listener->address_space = as;
if (QTAILQ_EMPTY(&memory_listeners)
|| listener->priority >= QTAILQ_LAST(&memory_listeners)->priority) {
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 12/20] KVM: Use a big lock to replace per-kml slots_lock
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (10 preceding siblings ...)
2021-05-17 11:19 ` [PULL 11/20] memory: Introduce log_sync_global() to memory listener Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 13/20] KVM: Create the KVMSlot dirty bitmap on flag changes Paolo Bonzini
` (9 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Xu
From: Peter Xu <peterx@redhat.com>
Per-kml slots_lock will bring some trouble if we want to take all slots_lock of
all the KMLs, especially when we're in a context that we could have taken some
of the KML slots_lock, then we even need to figure out what we've taken and
what we need to take.
Make this simple by merging all KML slots_lock into a single slots lock.
Per-kml slots_lock isn't anything that helpful anyway - so far only x86 has two
address spaces (so, two slots_locks). All the rest archs will be having one
address space always, which means there's actually one slots_lock so it will be
the same as before.
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210506160549.130416-3-peterx@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 33 ++++++++++++++++++---------------
include/sysemu/kvm_int.h | 2 --
2 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 54c0860979..f8980e155b 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -172,8 +172,10 @@ typedef struct KVMResampleFd KVMResampleFd;
static QLIST_HEAD(, KVMResampleFd) kvm_resample_fd_list =
QLIST_HEAD_INITIALIZER(kvm_resample_fd_list);
-#define kvm_slots_lock(kml) qemu_mutex_lock(&(kml)->slots_lock)
-#define kvm_slots_unlock(kml) qemu_mutex_unlock(&(kml)->slots_lock)
+static QemuMutex kml_slots_lock;
+
+#define kvm_slots_lock() qemu_mutex_lock(&kml_slots_lock)
+#define kvm_slots_unlock() qemu_mutex_unlock(&kml_slots_lock)
static inline void kvm_resample_fd_remove(int gsi)
{
@@ -239,9 +241,9 @@ bool kvm_has_free_slot(MachineState *ms)
bool result;
KVMMemoryListener *kml = &s->memory_listener;
- kvm_slots_lock(kml);
+ kvm_slots_lock();
result = !!kvm_get_free_slot(kml);
- kvm_slots_unlock(kml);
+ kvm_slots_unlock();
return result;
}
@@ -307,7 +309,7 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,
KVMMemoryListener *kml = &s->memory_listener;
int i, ret = 0;
- kvm_slots_lock(kml);
+ kvm_slots_lock();
for (i = 0; i < s->nr_slots; i++) {
KVMSlot *mem = &kml->slots[i];
@@ -317,7 +319,7 @@ int kvm_physical_memory_addr_from_host(KVMState *s, void *ram,
break;
}
}
- kvm_slots_unlock(kml);
+ kvm_slots_unlock();
return ret;
}
@@ -513,7 +515,7 @@ static int kvm_section_update_flags(KVMMemoryListener *kml,
return 0;
}
- kvm_slots_lock(kml);
+ kvm_slots_lock();
while (size && !ret) {
slot_size = MIN(kvm_max_slot_size, size);
@@ -529,7 +531,7 @@ static int kvm_section_update_flags(KVMMemoryListener *kml,
}
out:
- kvm_slots_unlock(kml);
+ kvm_slots_unlock();
return ret;
}
@@ -810,7 +812,7 @@ static int kvm_physical_log_clear(KVMMemoryListener *kml,
return ret;
}
- kvm_slots_lock(kml);
+ kvm_slots_lock();
for (i = 0; i < s->nr_slots; i++) {
mem = &kml->slots[i];
@@ -836,7 +838,7 @@ static int kvm_physical_log_clear(KVMMemoryListener *kml,
}
}
- kvm_slots_unlock(kml);
+ kvm_slots_unlock();
return ret;
}
@@ -1141,7 +1143,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
ram = memory_region_get_ram_ptr(mr) + section->offset_within_region +
(start_addr - section->offset_within_address_space);
- kvm_slots_lock(kml);
+ kvm_slots_lock();
if (!add) {
do {
@@ -1199,7 +1201,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
} while (size);
out:
- kvm_slots_unlock(kml);
+ kvm_slots_unlock();
}
static void kvm_region_add(MemoryListener *listener,
@@ -1226,9 +1228,9 @@ static void kvm_log_sync(MemoryListener *listener,
KVMMemoryListener *kml = container_of(listener, KVMMemoryListener, listener);
int r;
- kvm_slots_lock(kml);
+ kvm_slots_lock();
r = kvm_physical_sync_dirty_bitmap(kml, section);
- kvm_slots_unlock(kml);
+ kvm_slots_unlock();
if (r < 0) {
abort();
}
@@ -1328,7 +1330,6 @@ void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml,
{
int i;
- qemu_mutex_init(&kml->slots_lock);
kml->slots = g_malloc0(s->nr_slots * sizeof(KVMSlot));
kml->as_id = as_id;
@@ -2001,6 +2002,8 @@ static int kvm_init(MachineState *ms)
int type = 0;
uint64_t dirty_log_manual_caps;
+ qemu_mutex_init(&kml_slots_lock);
+
s = KVM_STATE(ms->accelerator);
/*
diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
index ccb8869f01..1da30e1884 100644
--- a/include/sysemu/kvm_int.h
+++ b/include/sysemu/kvm_int.h
@@ -27,8 +27,6 @@ typedef struct KVMSlot
typedef struct KVMMemoryListener {
MemoryListener listener;
- /* Protects the slots and all inside them */
- QemuMutex slots_lock;
KVMSlot *slots;
int as_id;
} KVMMemoryListener;
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 13/20] KVM: Create the KVMSlot dirty bitmap on flag changes
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (11 preceding siblings ...)
2021-05-17 11:19 ` [PULL 12/20] KVM: Use a big lock to replace per-kml slots_lock Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 14/20] KVM: Provide helper to get kvm dirty log Paolo Bonzini
` (8 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Dr . David Alan Gilbert, Peter Xu
From: Peter Xu <peterx@redhat.com>
Previously we have two places that will create the per KVMSlot dirty
bitmap:
1. When a newly created KVMSlot has dirty logging enabled,
2. When the first log_sync() happens for a memory slot.
The 2nd case is lazy-init, while the 1st case is not (which is a fix
of what the 2nd case missed).
To do explicit initialization of dirty bitmaps, what we're missing is
to create the dirty bitmap when the slot changed from not-dirty-track
to dirty-track. Do that in kvm_slot_update_flags().
With that, we can safely remove the 2nd lazy-init.
This change will be needed for kvm dirty ring because kvm dirty ring
does not use the log_sync() interface at all.
Also move all the pre-checks into kvm_slot_init_dirty_bitmap().
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210506160549.130416-4-peterx@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 23 +++++++++--------------
1 file changed, 9 insertions(+), 14 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index f8980e155b..5bc40fd71b 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -177,6 +177,8 @@ static QemuMutex kml_slots_lock;
#define kvm_slots_lock() qemu_mutex_lock(&kml_slots_lock)
#define kvm_slots_unlock() qemu_mutex_unlock(&kml_slots_lock)
+static void kvm_slot_init_dirty_bitmap(KVMSlot *mem);
+
static inline void kvm_resample_fd_remove(int gsi)
{
KVMResampleFd *rfd;
@@ -500,6 +502,7 @@ static int kvm_slot_update_flags(KVMMemoryListener *kml, KVMSlot *mem,
return 0;
}
+ kvm_slot_init_dirty_bitmap(mem);
return kvm_set_user_memory_region(kml, mem, false);
}
@@ -584,8 +587,12 @@ static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
#define ALIGN(x, y) (((x)+(y)-1) & ~((y)-1))
/* Allocate the dirty bitmap for a slot */
-static void kvm_memslot_init_dirty_bitmap(KVMSlot *mem)
+static void kvm_slot_init_dirty_bitmap(KVMSlot *mem)
{
+ if (!(mem->flags & KVM_MEM_LOG_DIRTY_PAGES) || mem->dirty_bmap) {
+ return;
+ }
+
/*
* XXX bad kernel interface alert
* For dirty bitmap, kernel allocates array of size aligned to
@@ -640,11 +647,6 @@ static int kvm_physical_sync_dirty_bitmap(KVMMemoryListener *kml,
goto out;
}
- if (!mem->dirty_bmap) {
- /* Allocate on the first log_sync, once and for all */
- kvm_memslot_init_dirty_bitmap(mem);
- }
-
d.dirty_bitmap = mem->dirty_bmap;
d.slot = mem->slot | (kml->as_id << 16);
ret = kvm_vm_ioctl(s, KVM_GET_DIRTY_LOG, &d);
@@ -1181,14 +1183,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
mem->start_addr = start_addr;
mem->ram = ram;
mem->flags = kvm_mem_flags(mr);
-
- if (mem->flags & KVM_MEM_LOG_DIRTY_PAGES) {
- /*
- * Reallocate the bmap; it means it doesn't disappear in
- * middle of a migrate.
- */
- kvm_memslot_init_dirty_bitmap(mem);
- }
+ kvm_slot_init_dirty_bitmap(mem);
err = kvm_set_user_memory_region(kml, mem, true);
if (err) {
fprintf(stderr, "%s: error registering slot: %s\n", __func__,
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 14/20] KVM: Provide helper to get kvm dirty log
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (12 preceding siblings ...)
2021-05-17 11:19 ` [PULL 13/20] KVM: Create the KVMSlot dirty bitmap on flag changes Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 15/20] KVM: Provide helper to sync dirty bitmap from slot to ramblock Paolo Bonzini
` (7 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Xu
From: Peter Xu <peterx@redhat.com>
Provide a helper kvm_slot_get_dirty_log() to make the function
kvm_physical_sync_dirty_bitmap() clearer. We can even cache the as_id
into KVMSlot when it is created, so that we don't even need to pass it
down every time.
Since at it, remove return value of kvm_physical_sync_dirty_bitmap()
because it should never fail.
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210506160549.130416-5-peterx@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 52 +++++++++++++++++++++++-----------------
include/sysemu/kvm_int.h | 2 ++
2 files changed, 32 insertions(+), 22 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 5bc40fd71b..caaa2a5c98 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -615,6 +615,30 @@ static void kvm_slot_init_dirty_bitmap(KVMSlot *mem)
mem->dirty_bmap = g_malloc0(bitmap_size);
}
+/*
+ * Sync dirty bitmap from kernel to KVMSlot.dirty_bmap, return true if
+ * succeeded, false otherwise
+ */
+static bool kvm_slot_get_dirty_log(KVMState *s, KVMSlot *slot)
+{
+ struct kvm_dirty_log d = {};
+ int ret;
+
+ d.dirty_bitmap = slot->dirty_bmap;
+ d.slot = slot->slot | (slot->as_id << 16);
+ ret = kvm_vm_ioctl(s, KVM_GET_DIRTY_LOG, &d);
+
+ if (ret == -ENOENT) {
+ /* kernel does not have dirty bitmap in this slot */
+ ret = 0;
+ }
+ if (ret) {
+ error_report_once("%s: KVM_GET_DIRTY_LOG failed with %d",
+ __func__, ret);
+ }
+ return ret == 0;
+}
+
/**
* kvm_physical_sync_dirty_bitmap - Sync dirty bitmap from kernel space
*
@@ -626,15 +650,13 @@ static void kvm_slot_init_dirty_bitmap(KVMSlot *mem)
* @kml: the KVM memory listener object
* @section: the memory section to sync the dirty bitmap with
*/
-static int kvm_physical_sync_dirty_bitmap(KVMMemoryListener *kml,
- MemoryRegionSection *section)
+static void kvm_physical_sync_dirty_bitmap(KVMMemoryListener *kml,
+ MemoryRegionSection *section)
{
KVMState *s = kvm_state;
- struct kvm_dirty_log d = {};
KVMSlot *mem;
hwaddr start_addr, size;
hwaddr slot_size, slot_offset = 0;
- int ret = 0;
size = kvm_align_section(section, &start_addr);
while (size) {
@@ -644,19 +666,10 @@ static int kvm_physical_sync_dirty_bitmap(KVMMemoryListener *kml,
mem = kvm_lookup_matching_slot(kml, start_addr, slot_size);
if (!mem) {
/* We don't have a slot if we want to trap every access. */
- goto out;
+ return;
}
- d.dirty_bitmap = mem->dirty_bmap;
- d.slot = mem->slot | (kml->as_id << 16);
- ret = kvm_vm_ioctl(s, KVM_GET_DIRTY_LOG, &d);
- if (ret == -ENOENT) {
- /* kernel does not have dirty bitmap in this slot */
- ret = 0;
- } else if (ret < 0) {
- error_report("ioctl KVM_GET_DIRTY_LOG failed: %d", errno);
- goto out;
- } else {
+ if (kvm_slot_get_dirty_log(s, mem)) {
subsection.offset_within_region += slot_offset;
subsection.size = int128_make64(slot_size);
kvm_get_dirty_pages_log_range(&subsection, d.dirty_bitmap);
@@ -666,8 +679,6 @@ static int kvm_physical_sync_dirty_bitmap(KVMMemoryListener *kml,
start_addr += slot_size;
size -= slot_size;
}
-out:
- return ret;
}
/* Alignment requirement for KVM_CLEAR_DIRTY_LOG - 64 pages */
@@ -1179,6 +1190,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
do {
slot_size = MIN(kvm_max_slot_size, size);
mem = kvm_alloc_slot(kml);
+ mem->as_id = kml->as_id;
mem->memory_size = slot_size;
mem->start_addr = start_addr;
mem->ram = ram;
@@ -1221,14 +1233,10 @@ static void kvm_log_sync(MemoryListener *listener,
MemoryRegionSection *section)
{
KVMMemoryListener *kml = container_of(listener, KVMMemoryListener, listener);
- int r;
kvm_slots_lock();
- r = kvm_physical_sync_dirty_bitmap(kml, section);
+ kvm_physical_sync_dirty_bitmap(kml, section);
kvm_slots_unlock();
- if (r < 0) {
- abort();
- }
}
static void kvm_log_clear(MemoryListener *listener,
diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
index 1da30e1884..e13075f738 100644
--- a/include/sysemu/kvm_int.h
+++ b/include/sysemu/kvm_int.h
@@ -23,6 +23,8 @@ typedef struct KVMSlot
int old_flags;
/* Dirty bitmap cache for the slot */
unsigned long *dirty_bmap;
+ /* Cache of the address space ID */
+ int as_id;
} KVMSlot;
typedef struct KVMMemoryListener {
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 15/20] KVM: Provide helper to sync dirty bitmap from slot to ramblock
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (13 preceding siblings ...)
2021-05-17 11:19 ` [PULL 14/20] KVM: Provide helper to get kvm dirty log Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 16/20] KVM: Simplify dirty log sync in kvm_set_phys_mem Paolo Bonzini
` (6 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Dr . David Alan Gilbert, Peter Xu
From: Peter Xu <peterx@redhat.com>
kvm_physical_sync_dirty_bitmap() calculates the ramblock offset in an
awkward way from the MemoryRegionSection that passed in from the
caller. The truth is for each KVMSlot the ramblock offset never
change for the lifecycle. Cache the ramblock offset for each KVMSlot
into the structure when the KVMSlot is created.
With that, we can further simplify kvm_physical_sync_dirty_bitmap()
with a helper to sync KVMSlot dirty bitmap to the ramblock dirty
bitmap of a specific KVMSlot.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210506160549.130416-6-peterx@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 37 +++++++++++++++++--------------------
include/sysemu/kvm_int.h | 2 ++
2 files changed, 19 insertions(+), 20 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index caaa2a5c98..7031dd4250 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -573,15 +573,12 @@ static void kvm_log_stop(MemoryListener *listener,
}
/* get kvm's dirty pages bitmap and update qemu's */
-static int kvm_get_dirty_pages_log_range(MemoryRegionSection *section,
- unsigned long *bitmap)
+static void kvm_slot_sync_dirty_pages(KVMSlot *slot)
{
- ram_addr_t start = section->offset_within_region +
- memory_region_get_ram_addr(section->mr);
- ram_addr_t pages = int128_get64(section->size) / qemu_real_host_page_size;
+ ram_addr_t start = slot->ram_start_offset;
+ ram_addr_t pages = slot->memory_size / qemu_real_host_page_size;
- cpu_physical_memory_set_dirty_lebitmap(bitmap, start, pages);
- return 0;
+ cpu_physical_memory_set_dirty_lebitmap(slot->dirty_bmap, start, pages);
}
#define ALIGN(x, y) (((x)+(y)-1) & ~((y)-1))
@@ -656,26 +653,19 @@ static void kvm_physical_sync_dirty_bitmap(KVMMemoryListener *kml,
KVMState *s = kvm_state;
KVMSlot *mem;
hwaddr start_addr, size;
- hwaddr slot_size, slot_offset = 0;
+ hwaddr slot_size;
size = kvm_align_section(section, &start_addr);
while (size) {
- MemoryRegionSection subsection = *section;
-
slot_size = MIN(kvm_max_slot_size, size);
mem = kvm_lookup_matching_slot(kml, start_addr, slot_size);
if (!mem) {
/* We don't have a slot if we want to trap every access. */
return;
}
-
if (kvm_slot_get_dirty_log(s, mem)) {
- subsection.offset_within_region += slot_offset;
- subsection.size = int128_make64(slot_size);
- kvm_get_dirty_pages_log_range(&subsection, d.dirty_bitmap);
+ kvm_slot_sync_dirty_pages(mem);
}
-
- slot_offset += slot_size;
start_addr += slot_size;
size -= slot_size;
}
@@ -1134,7 +1124,8 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
int err;
MemoryRegion *mr = section->mr;
bool writeable = !mr->readonly && !mr->rom_device;
- hwaddr start_addr, size, slot_size;
+ hwaddr start_addr, size, slot_size, mr_offset;
+ ram_addr_t ram_start_offset;
void *ram;
if (!memory_region_is_ram(mr)) {
@@ -1152,9 +1143,13 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
return;
}
- /* use aligned delta to align the ram address */
- ram = memory_region_get_ram_ptr(mr) + section->offset_within_region +
- (start_addr - section->offset_within_address_space);
+ /* The offset of the kvmslot within the memory region */
+ mr_offset = section->offset_within_region + start_addr -
+ section->offset_within_address_space;
+
+ /* use aligned delta to align the ram address and offset */
+ ram = memory_region_get_ram_ptr(mr) + mr_offset;
+ ram_start_offset = memory_region_get_ram_addr(mr) + mr_offset;
kvm_slots_lock();
@@ -1193,6 +1188,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
mem->as_id = kml->as_id;
mem->memory_size = slot_size;
mem->start_addr = start_addr;
+ mem->ram_start_offset = ram_start_offset;
mem->ram = ram;
mem->flags = kvm_mem_flags(mr);
kvm_slot_init_dirty_bitmap(mem);
@@ -1203,6 +1199,7 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
abort();
}
start_addr += slot_size;
+ ram_start_offset += slot_size;
ram += slot_size;
size -= slot_size;
} while (size);
diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
index e13075f738..ab09a150e1 100644
--- a/include/sysemu/kvm_int.h
+++ b/include/sysemu/kvm_int.h
@@ -25,6 +25,8 @@ typedef struct KVMSlot
unsigned long *dirty_bmap;
/* Cache of the address space ID */
int as_id;
+ /* Cache of the offset in ram address space */
+ ram_addr_t ram_start_offset;
} KVMSlot;
typedef struct KVMMemoryListener {
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 16/20] KVM: Simplify dirty log sync in kvm_set_phys_mem
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (14 preceding siblings ...)
2021-05-17 11:19 ` [PULL 15/20] KVM: Provide helper to sync dirty bitmap from slot to ramblock Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 17/20] KVM: Cache kvm slot dirty bitmap size Paolo Bonzini
` (5 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Xu
From: Peter Xu <peterx@redhat.com>
kvm_physical_sync_dirty_bitmap() on the whole section is inaccurate, because
the section can be a superset of the memslot that we're working on. The result
is that if the section covers multiple kvm memslots, we could be doing the
synchronization for multiple times for each kvmslot in the section.
With the two helpers that we just introduced, it's very easy to do it right now
by calling the helpers.
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210506160549.130416-7-peterx@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 7031dd4250..0ba6d48120 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -1161,7 +1161,8 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
goto out;
}
if (mem->flags & KVM_MEM_LOG_DIRTY_PAGES) {
- kvm_physical_sync_dirty_bitmap(kml, section);
+ kvm_slot_get_dirty_log(kvm_state, mem);
+ kvm_slot_sync_dirty_pages(mem);
}
/* unregister the slot */
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 17/20] KVM: Cache kvm slot dirty bitmap size
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (15 preceding siblings ...)
2021-05-17 11:19 ` [PULL 16/20] KVM: Simplify dirty log sync in kvm_set_phys_mem Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:19 ` [PULL 18/20] KVM: Add dirty-ring-size property Paolo Bonzini
` (4 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Dr . David Alan Gilbert, Peter Xu
From: Peter Xu <peterx@redhat.com>
Cache it too because we'll reference it more frequently in the future.
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210506160549.130416-8-peterx@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 1 +
include/sysemu/kvm_int.h | 1 +
2 files changed, 2 insertions(+)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 0ba6d48120..df9fbf59a6 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -610,6 +610,7 @@ static void kvm_slot_init_dirty_bitmap(KVMSlot *mem)
hwaddr bitmap_size = ALIGN(mem->memory_size / qemu_real_host_page_size,
/*HOST_LONG_BITS*/ 64) / 8;
mem->dirty_bmap = g_malloc0(bitmap_size);
+ mem->dirty_bmap_size = bitmap_size;
}
/*
diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
index ab09a150e1..c788452cd9 100644
--- a/include/sysemu/kvm_int.h
+++ b/include/sysemu/kvm_int.h
@@ -23,6 +23,7 @@ typedef struct KVMSlot
int old_flags;
/* Dirty bitmap cache for the slot */
unsigned long *dirty_bmap;
+ unsigned long dirty_bmap_size;
/* Cache of the address space ID */
int as_id;
/* Cache of the offset in ram address space */
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 18/20] KVM: Add dirty-ring-size property
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (16 preceding siblings ...)
2021-05-17 11:19 ` [PULL 17/20] KVM: Cache kvm slot dirty bitmap size Paolo Bonzini
@ 2021-05-17 11:19 ` Paolo Bonzini
2021-05-17 11:20 ` [PULL 19/20] KVM: Disable manual dirty log when dirty ring enabled Paolo Bonzini
` (3 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:19 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Xu
From: Peter Xu <peterx@redhat.com>
Add a parameter for dirty gfn count for dirty rings. If zero, dirty ring is
disabled. Otherwise dirty ring will be enabled with the per-vcpu gfn count as
specified. If dirty ring cannot be enabled due to unsupported kernel or
illegal parameter, it'll fallback to dirty logging.
By default, dirty ring is not enabled (dirty-gfn-count default to 0).
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210506160549.130416-9-peterx@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 46 +++++++++++++++++++++++++++++++++++++++++++++
qemu-options.hx | 12 ++++++++++++
2 files changed, 58 insertions(+)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index df9fbf59a6..5afe15ae66 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -126,6 +126,8 @@ struct KVMState
KVMMemoryListener *ml;
AddressSpace *as;
} *as;
+ uint64_t kvm_dirty_ring_bytes; /* Size of the per-vcpu dirty ring */
+ uint32_t kvm_dirty_ring_size; /* Number of dirty GFNs per ring */
};
KVMState *kvm_state;
@@ -3182,6 +3184,42 @@ bool kvm_kernel_irqchip_split(void)
return kvm_state->kernel_irqchip_split == ON_OFF_AUTO_ON;
}
+static void kvm_get_dirty_ring_size(Object *obj, Visitor *v,
+ const char *name, void *opaque,
+ Error **errp)
+{
+ KVMState *s = KVM_STATE(obj);
+ uint32_t value = s->kvm_dirty_ring_size;
+
+ visit_type_uint32(v, name, &value, errp);
+}
+
+static void kvm_set_dirty_ring_size(Object *obj, Visitor *v,
+ const char *name, void *opaque,
+ Error **errp)
+{
+ KVMState *s = KVM_STATE(obj);
+ Error *error = NULL;
+ uint32_t value;
+
+ if (s->fd != -1) {
+ error_setg(errp, "Cannot set properties after the accelerator has been initialized");
+ return;
+ }
+
+ visit_type_uint32(v, name, &value, &error);
+ if (error) {
+ error_propagate(errp, error);
+ return;
+ }
+ if (value & (value - 1)) {
+ error_setg(errp, "dirty-ring-size must be a power of two.");
+ return;
+ }
+
+ s->kvm_dirty_ring_size = value;
+}
+
static void kvm_accel_instance_init(Object *obj)
{
KVMState *s = KVM_STATE(obj);
@@ -3191,6 +3229,8 @@ static void kvm_accel_instance_init(Object *obj)
s->kvm_shadow_mem = -1;
s->kernel_irqchip_allowed = true;
s->kernel_irqchip_split = ON_OFF_AUTO_AUTO;
+ /* KVM dirty ring is by default off */
+ s->kvm_dirty_ring_size = 0;
}
static void kvm_accel_class_init(ObjectClass *oc, void *data)
@@ -3212,6 +3252,12 @@ static void kvm_accel_class_init(ObjectClass *oc, void *data)
NULL, NULL);
object_class_property_set_description(oc, "kvm-shadow-mem",
"KVM shadow MMU size");
+
+ object_class_property_add(oc, "dirty-ring-size", "uint32",
+ kvm_get_dirty_ring_size, kvm_set_dirty_ring_size,
+ NULL, NULL);
+ object_class_property_set_description(oc, "dirty-ring-size",
+ "Size of KVM dirty page ring buffer (default: 0, i.e. use bitmap)");
}
static const TypeInfo kvm_accel_type = {
diff --git a/qemu-options.hx b/qemu-options.hx
index e22fb94d99..ecdb064409 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -141,6 +141,7 @@ DEF("accel", HAS_ARG, QEMU_OPTION_accel,
" kvm-shadow-mem=size of KVM shadow MMU in bytes\n"
" split-wx=on|off (enable TCG split w^x mapping)\n"
" tb-size=n (TCG translation block cache size)\n"
+ " dirty-ring-size=n (KVM dirty ring GFN count, default 0)\n"
" thread=single|multi (enable multi-threaded TCG)\n", QEMU_ARCH_ALL)
SRST
``-accel name[,prop=value[,...]]``
@@ -181,6 +182,17 @@ SRST
where both the back-end and front-ends support it and no
incompatible TCG features have been enabled (e.g.
icount/replay).
+
+ ``dirty-ring-size=n``
+ When the KVM accelerator is used, it controls the size of the per-vCPU
+ dirty page ring buffer (number of entries for each vCPU). It should
+ be a value that is power of two, and it should be 1024 or bigger (but
+ still less than the maximum value that the kernel supports). 4096
+ could be a good initial value if you have no idea which is the best.
+ Set this value to 0 to disable the feature. By default, this feature
+ is disabled (dirty-ring-size=0). When enabled, KVM will instead
+ record dirty pages in a bitmap.
+
ERST
DEF("smp", HAS_ARG, QEMU_OPTION_smp,
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 19/20] KVM: Disable manual dirty log when dirty ring enabled
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (17 preceding siblings ...)
2021-05-17 11:19 ` [PULL 18/20] KVM: Add dirty-ring-size property Paolo Bonzini
@ 2021-05-17 11:20 ` Paolo Bonzini
2021-05-17 11:20 ` [PULL 20/20] KVM: Dirty ring support Paolo Bonzini
` (2 subsequent siblings)
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:20 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Xu
From: Peter Xu <peterx@redhat.com>
KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 is for KVM_CLEAR_DIRTY_LOG, which is only
useful for KVM_GET_DIRTY_LOG. Skip enabling it for kvm dirty ring.
More importantly, KVM_DIRTY_LOG_INITIALLY_SET will not wr-protect all the pages
initially, which is against how kvm dirty ring is used - there's no way for kvm
dirty ring to re-protect a page before it's notified as being written first
with a GFN entry in the ring! So when KVM_DIRTY_LOG_INITIALLY_SET is enabled
with dirty ring, we'll see silent data loss after migration.
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210506160549.130416-10-peterx@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 37 +++++++++++++++++++++++--------------
1 file changed, 23 insertions(+), 14 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 5afe15ae66..e1a6c7c89b 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2131,20 +2131,29 @@ static int kvm_init(MachineState *ms)
s->coalesced_pio = s->coalesced_mmio &&
kvm_check_extension(s, KVM_CAP_COALESCED_PIO);
- dirty_log_manual_caps =
- kvm_check_extension(s, KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2);
- dirty_log_manual_caps &= (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE |
- KVM_DIRTY_LOG_INITIALLY_SET);
- s->manual_dirty_log_protect = dirty_log_manual_caps;
- if (dirty_log_manual_caps) {
- ret = kvm_vm_enable_cap(s, KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2, 0,
- dirty_log_manual_caps);
- if (ret) {
- warn_report("Trying to enable capability %"PRIu64" of "
- "KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 but failed. "
- "Falling back to the legacy mode. ",
- dirty_log_manual_caps);
- s->manual_dirty_log_protect = 0;
+ /*
+ * KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 is not needed when dirty ring is
+ * enabled. More importantly, KVM_DIRTY_LOG_INITIALLY_SET will assume no
+ * page is wr-protected initially, which is against how kvm dirty ring is
+ * usage - kvm dirty ring requires all pages are wr-protected at the very
+ * beginning. Enabling this feature for dirty ring causes data corruption.
+ */
+ if (!s->kvm_dirty_ring_size) {
+ dirty_log_manual_caps =
+ kvm_check_extension(s, KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2);
+ dirty_log_manual_caps &= (KVM_DIRTY_LOG_MANUAL_PROTECT_ENABLE |
+ KVM_DIRTY_LOG_INITIALLY_SET);
+ s->manual_dirty_log_protect = dirty_log_manual_caps;
+ if (dirty_log_manual_caps) {
+ ret = kvm_vm_enable_cap(s, KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2, 0,
+ dirty_log_manual_caps);
+ if (ret) {
+ warn_report("Trying to enable capability %"PRIu64" of "
+ "KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 but failed. "
+ "Falling back to the legacy mode. ",
+ dirty_log_manual_caps);
+ s->manual_dirty_log_protect = 0;
+ }
}
}
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* [PULL 20/20] KVM: Dirty ring support
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (18 preceding siblings ...)
2021-05-17 11:20 ` [PULL 19/20] KVM: Disable manual dirty log when dirty ring enabled Paolo Bonzini
@ 2021-05-17 11:20 ` Paolo Bonzini
2021-05-17 12:15 ` [PULL 00/20] Misc patches for 2020-05-17 no-reply
2021-05-18 15:16 ` Peter Maydell
21 siblings, 0 replies; 23+ messages in thread
From: Paolo Bonzini @ 2021-05-17 11:20 UTC (permalink / raw)
To: qemu-devel; +Cc: Peter Xu
From: Peter Xu <peterx@redhat.com>
KVM dirty ring is a new interface to pass over dirty bits from kernel to the
userspace. Instead of using a bitmap for each memory region, the dirty ring
contains an array of dirtied GPAs to fetch (in the form of offset in slots).
For each vcpu there will be one dirty ring that binds to it.
kvm_dirty_ring_reap() is the major function to collect dirty rings. It can be
called either by a standalone reaper thread that runs in the background,
collecting dirty pages for the whole VM. It can also be called directly by any
thread that has BQL taken.
Signed-off-by: Peter Xu <peterx@redhat.com>
Message-Id: <20210506160549.130416-11-peterx@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
accel/kvm/kvm-all.c | 374 ++++++++++++++++++++++++++++++++++++++++-
accel/kvm/trace-events | 7 +
include/hw/core/cpu.h | 7 +
3 files changed, 385 insertions(+), 3 deletions(-)
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index e1a6c7c89b..3f16901950 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -15,6 +15,7 @@
#include "qemu/osdep.h"
#include <sys/ioctl.h>
+#include <poll.h>
#include <linux/kvm.h>
@@ -78,6 +79,25 @@ struct KVMParkedVcpu {
QLIST_ENTRY(KVMParkedVcpu) node;
};
+enum KVMDirtyRingReaperState {
+ KVM_DIRTY_RING_REAPER_NONE = 0,
+ /* The reaper is sleeping */
+ KVM_DIRTY_RING_REAPER_WAIT,
+ /* The reaper is reaping for dirty pages */
+ KVM_DIRTY_RING_REAPER_REAPING,
+};
+
+/*
+ * KVM reaper instance, responsible for collecting the KVM dirty bits
+ * via the dirty ring.
+ */
+struct KVMDirtyRingReaper {
+ /* The reaper thread */
+ QemuThread reaper_thr;
+ volatile uint64_t reaper_iteration; /* iteration number of reaper thr */
+ volatile enum KVMDirtyRingReaperState reaper_state; /* reap thr state */
+};
+
struct KVMState
{
AccelState parent_obj;
@@ -128,6 +148,7 @@ struct KVMState
} *as;
uint64_t kvm_dirty_ring_bytes; /* Size of the per-vcpu dirty ring */
uint32_t kvm_dirty_ring_size; /* Number of dirty GFNs per ring */
+ struct KVMDirtyRingReaper reaper;
};
KVMState *kvm_state;
@@ -389,6 +410,13 @@ static int do_kvm_destroy_vcpu(CPUState *cpu)
goto err;
}
+ if (cpu->kvm_dirty_gfns) {
+ ret = munmap(cpu->kvm_dirty_gfns, s->kvm_dirty_ring_size);
+ if (ret < 0) {
+ goto err;
+ }
+ }
+
vcpu = g_malloc0(sizeof(*vcpu));
vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
vcpu->kvm_fd = cpu->kvm_fd;
@@ -465,6 +493,19 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
(void *)cpu->kvm_run + s->coalesced_mmio * PAGE_SIZE;
}
+ if (s->kvm_dirty_ring_size) {
+ /* Use MAP_SHARED to share pages with the kernel */
+ cpu->kvm_dirty_gfns = mmap(NULL, s->kvm_dirty_ring_size,
+ PROT_READ | PROT_WRITE, MAP_SHARED,
+ cpu->kvm_fd,
+ PAGE_SIZE * KVM_DIRTY_LOG_PAGE_OFFSET);
+ if (cpu->kvm_dirty_gfns == MAP_FAILED) {
+ ret = -errno;
+ DPRINTF("mmap'ing vcpu dirty gfns failed: %d\n", ret);
+ goto err;
+ }
+ }
+
ret = kvm_arch_init_vcpu(cpu);
if (ret < 0) {
error_setg_errno(errp, -ret,
@@ -583,6 +624,11 @@ static void kvm_slot_sync_dirty_pages(KVMSlot *slot)
cpu_physical_memory_set_dirty_lebitmap(slot->dirty_bmap, start, pages);
}
+static void kvm_slot_reset_dirty_pages(KVMSlot *slot)
+{
+ memset(slot->dirty_bmap, 0, slot->dirty_bmap_size);
+}
+
#define ALIGN(x, y) (((x)+(y)-1) & ~((y)-1))
/* Allocate the dirty bitmap for a slot */
@@ -639,6 +685,171 @@ static bool kvm_slot_get_dirty_log(KVMState *s, KVMSlot *slot)
return ret == 0;
}
+/* Should be with all slots_lock held for the address spaces. */
+static void kvm_dirty_ring_mark_page(KVMState *s, uint32_t as_id,
+ uint32_t slot_id, uint64_t offset)
+{
+ KVMMemoryListener *kml;
+ KVMSlot *mem;
+
+ if (as_id >= s->nr_as) {
+ return;
+ }
+
+ kml = s->as[as_id].ml;
+ mem = &kml->slots[slot_id];
+
+ if (!mem->memory_size || offset >=
+ (mem->memory_size / qemu_real_host_page_size)) {
+ return;
+ }
+
+ set_bit(offset, mem->dirty_bmap);
+}
+
+static bool dirty_gfn_is_dirtied(struct kvm_dirty_gfn *gfn)
+{
+ return gfn->flags == KVM_DIRTY_GFN_F_DIRTY;
+}
+
+static void dirty_gfn_set_collected(struct kvm_dirty_gfn *gfn)
+{
+ gfn->flags = KVM_DIRTY_GFN_F_RESET;
+}
+
+/*
+ * Should be with all slots_lock held for the address spaces. It returns the
+ * dirty page we've collected on this dirty ring.
+ */
+static uint32_t kvm_dirty_ring_reap_one(KVMState *s, CPUState *cpu)
+{
+ struct kvm_dirty_gfn *dirty_gfns = cpu->kvm_dirty_gfns, *cur;
+ uint32_t ring_size = s->kvm_dirty_ring_size;
+ uint32_t count = 0, fetch = cpu->kvm_fetch_index;
+
+ assert(dirty_gfns && ring_size);
+ trace_kvm_dirty_ring_reap_vcpu(cpu->cpu_index);
+
+ while (true) {
+ cur = &dirty_gfns[fetch % ring_size];
+ if (!dirty_gfn_is_dirtied(cur)) {
+ break;
+ }
+ kvm_dirty_ring_mark_page(s, cur->slot >> 16, cur->slot & 0xffff,
+ cur->offset);
+ dirty_gfn_set_collected(cur);
+ trace_kvm_dirty_ring_page(cpu->cpu_index, fetch, cur->offset);
+ fetch++;
+ count++;
+ }
+ cpu->kvm_fetch_index = fetch;
+
+ return count;
+}
+
+/* Must be with slots_lock held */
+static uint64_t kvm_dirty_ring_reap_locked(KVMState *s)
+{
+ int ret;
+ CPUState *cpu;
+ uint64_t total = 0;
+ int64_t stamp;
+
+ stamp = get_clock();
+
+ CPU_FOREACH(cpu) {
+ total += kvm_dirty_ring_reap_one(s, cpu);
+ }
+
+ if (total) {
+ ret = kvm_vm_ioctl(s, KVM_RESET_DIRTY_RINGS);
+ assert(ret == total);
+ }
+
+ stamp = get_clock() - stamp;
+
+ if (total) {
+ trace_kvm_dirty_ring_reap(total, stamp / 1000);
+ }
+
+ return total;
+}
+
+/*
+ * Currently for simplicity, we must hold BQL before calling this. We can
+ * consider to drop the BQL if we're clear with all the race conditions.
+ */
+static uint64_t kvm_dirty_ring_reap(KVMState *s)
+{
+ uint64_t total;
+
+ /*
+ * We need to lock all kvm slots for all address spaces here,
+ * because:
+ *
+ * (1) We need to mark dirty for dirty bitmaps in multiple slots
+ * and for tons of pages, so it's better to take the lock here
+ * once rather than once per page. And more importantly,
+ *
+ * (2) We must _NOT_ publish dirty bits to the other threads
+ * (e.g., the migration thread) via the kvm memory slot dirty
+ * bitmaps before correctly re-protect those dirtied pages.
+ * Otherwise we can have potential risk of data corruption if
+ * the page data is read in the other thread before we do
+ * reset below.
+ */
+ kvm_slots_lock();
+ total = kvm_dirty_ring_reap_locked(s);
+ kvm_slots_unlock();
+
+ return total;
+}
+
+static void do_kvm_cpu_synchronize_kick(CPUState *cpu, run_on_cpu_data arg)
+{
+ /* No need to do anything */
+}
+
+/*
+ * Kick all vcpus out in a synchronized way. When returned, we
+ * guarantee that every vcpu has been kicked and at least returned to
+ * userspace once.
+ */
+static void kvm_cpu_synchronize_kick_all(void)
+{
+ CPUState *cpu;
+
+ CPU_FOREACH(cpu) {
+ run_on_cpu(cpu, do_kvm_cpu_synchronize_kick, RUN_ON_CPU_NULL);
+ }
+}
+
+/*
+ * Flush all the existing dirty pages to the KVM slot buffers. When
+ * this call returns, we guarantee that all the touched dirty pages
+ * before calling this function have been put into the per-kvmslot
+ * dirty bitmap.
+ *
+ * This function must be called with BQL held.
+ */
+static void kvm_dirty_ring_flush(void)
+{
+ trace_kvm_dirty_ring_flush(0);
+ /*
+ * The function needs to be serialized. Since this function
+ * should always be with BQL held, serialization is guaranteed.
+ * However, let's be sure of it.
+ */
+ assert(qemu_mutex_iothread_locked());
+ /*
+ * First make sure to flush the hardware buffers by kicking all
+ * vcpus out in a synchronous way.
+ */
+ kvm_cpu_synchronize_kick_all();
+ kvm_dirty_ring_reap(kvm_state);
+ trace_kvm_dirty_ring_flush(1);
+}
+
/**
* kvm_physical_sync_dirty_bitmap - Sync dirty bitmap from kernel space
*
@@ -1164,7 +1375,24 @@ static void kvm_set_phys_mem(KVMMemoryListener *kml,
goto out;
}
if (mem->flags & KVM_MEM_LOG_DIRTY_PAGES) {
- kvm_slot_get_dirty_log(kvm_state, mem);
+ /*
+ * NOTE: We should be aware of the fact that here we're only
+ * doing a best effort to sync dirty bits. No matter whether
+ * we're using dirty log or dirty ring, we ignored two facts:
+ *
+ * (1) dirty bits can reside in hardware buffers (PML)
+ *
+ * (2) after we collected dirty bits here, pages can be dirtied
+ * again before we do the final KVM_SET_USER_MEMORY_REGION to
+ * remove the slot.
+ *
+ * Not easy. Let's cross the fingers until it's fixed.
+ */
+ if (kvm_state->kvm_dirty_ring_size) {
+ kvm_dirty_ring_reap_locked(kvm_state);
+ } else {
+ kvm_slot_get_dirty_log(kvm_state, mem);
+ }
kvm_slot_sync_dirty_pages(mem);
}
@@ -1212,6 +1440,51 @@ out:
kvm_slots_unlock();
}
+static void *kvm_dirty_ring_reaper_thread(void *data)
+{
+ KVMState *s = data;
+ struct KVMDirtyRingReaper *r = &s->reaper;
+
+ rcu_register_thread();
+
+ trace_kvm_dirty_ring_reaper("init");
+
+ while (true) {
+ r->reaper_state = KVM_DIRTY_RING_REAPER_WAIT;
+ trace_kvm_dirty_ring_reaper("wait");
+ /*
+ * TODO: provide a smarter timeout rather than a constant?
+ */
+ sleep(1);
+
+ trace_kvm_dirty_ring_reaper("wakeup");
+ r->reaper_state = KVM_DIRTY_RING_REAPER_REAPING;
+
+ qemu_mutex_lock_iothread();
+ kvm_dirty_ring_reap(s);
+ qemu_mutex_unlock_iothread();
+
+ r->reaper_iteration++;
+ }
+
+ trace_kvm_dirty_ring_reaper("exit");
+
+ rcu_unregister_thread();
+
+ return NULL;
+}
+
+static int kvm_dirty_ring_reaper_init(KVMState *s)
+{
+ struct KVMDirtyRingReaper *r = &s->reaper;
+
+ qemu_thread_create(&r->reaper_thr, "kvm-reaper",
+ kvm_dirty_ring_reaper_thread,
+ s, QEMU_THREAD_JOINABLE);
+
+ return 0;
+}
+
static void kvm_region_add(MemoryListener *listener,
MemoryRegionSection *section)
{
@@ -1240,6 +1513,36 @@ static void kvm_log_sync(MemoryListener *listener,
kvm_slots_unlock();
}
+static void kvm_log_sync_global(MemoryListener *l)
+{
+ KVMMemoryListener *kml = container_of(l, KVMMemoryListener, listener);
+ KVMState *s = kvm_state;
+ KVMSlot *mem;
+ int i;
+
+ /* Flush all kernel dirty addresses into KVMSlot dirty bitmap */
+ kvm_dirty_ring_flush();
+
+ /*
+ * TODO: make this faster when nr_slots is big while there are
+ * only a few used slots (small VMs).
+ */
+ kvm_slots_lock();
+ for (i = 0; i < s->nr_slots; i++) {
+ mem = &kml->slots[i];
+ if (mem->memory_size && mem->flags & KVM_MEM_LOG_DIRTY_PAGES) {
+ kvm_slot_sync_dirty_pages(mem);
+ /*
+ * This is not needed by KVM_GET_DIRTY_LOG because the
+ * ioctl will unconditionally overwrite the whole region.
+ * However kvm dirty ring has no such side effect.
+ */
+ kvm_slot_reset_dirty_pages(mem);
+ }
+ }
+ kvm_slots_unlock();
+}
+
static void kvm_log_clear(MemoryListener *listener,
MemoryRegionSection *section)
{
@@ -1345,10 +1648,15 @@ void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml,
kml->listener.region_del = kvm_region_del;
kml->listener.log_start = kvm_log_start;
kml->listener.log_stop = kvm_log_stop;
- kml->listener.log_sync = kvm_log_sync;
- kml->listener.log_clear = kvm_log_clear;
kml->listener.priority = 10;
+ if (s->kvm_dirty_ring_size) {
+ kml->listener.log_sync_global = kvm_log_sync_global;
+ } else {
+ kml->listener.log_sync = kvm_log_sync;
+ kml->listener.log_clear = kvm_log_clear;
+ }
+
memory_listener_register(&kml->listener, as);
for (i = 0; i < s->nr_as; ++i) {
@@ -2131,12 +2439,53 @@ static int kvm_init(MachineState *ms)
s->coalesced_pio = s->coalesced_mmio &&
kvm_check_extension(s, KVM_CAP_COALESCED_PIO);
+ /*
+ * Enable KVM dirty ring if supported, otherwise fall back to
+ * dirty logging mode
+ */
+ if (s->kvm_dirty_ring_size > 0) {
+ uint64_t ring_bytes;
+
+ ring_bytes = s->kvm_dirty_ring_size * sizeof(struct kvm_dirty_gfn);
+
+ /* Read the max supported pages */
+ ret = kvm_vm_check_extension(s, KVM_CAP_DIRTY_LOG_RING);
+ if (ret > 0) {
+ if (ring_bytes > ret) {
+ error_report("KVM dirty ring size %" PRIu32 " too big "
+ "(maximum is %ld). Please use a smaller value.",
+ s->kvm_dirty_ring_size,
+ (long)ret / sizeof(struct kvm_dirty_gfn));
+ ret = -EINVAL;
+ goto err;
+ }
+
+ ret = kvm_vm_enable_cap(s, KVM_CAP_DIRTY_LOG_RING, 0, ring_bytes);
+ if (ret) {
+ error_report("Enabling of KVM dirty ring failed: %s. "
+ "Suggested mininum value is 1024.", strerror(-ret));
+ goto err;
+ }
+
+ s->kvm_dirty_ring_bytes = ring_bytes;
+ }
+ } else {
+ warn_report("KVM dirty ring not available, using bitmap method");
+ s->kvm_dirty_ring_size = 0;
+ }
+
/*
* KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 is not needed when dirty ring is
* enabled. More importantly, KVM_DIRTY_LOG_INITIALLY_SET will assume no
* page is wr-protected initially, which is against how kvm dirty ring is
* usage - kvm dirty ring requires all pages are wr-protected at the very
* beginning. Enabling this feature for dirty ring causes data corruption.
+ *
+ * TODO: Without KVM_CAP_MANUAL_DIRTY_LOG_PROTECT2 and kvm clear dirty log,
+ * we may expect a higher stall time when starting the migration. In the
+ * future we can enable KVM_CLEAR_DIRTY_LOG to work with dirty ring too:
+ * instead of clearing dirty bit, it can be a way to explicitly wr-protect
+ * guest pages.
*/
if (!s->kvm_dirty_ring_size) {
dirty_log_manual_caps =
@@ -2239,6 +2588,14 @@ static int kvm_init(MachineState *ms)
ret = ram_block_discard_disable(true);
assert(!ret);
}
+
+ if (s->kvm_dirty_ring_size) {
+ ret = kvm_dirty_ring_reaper_init(s);
+ if (ret) {
+ goto err;
+ }
+ }
+
return 0;
err:
@@ -2551,6 +2908,17 @@ int kvm_cpu_exec(CPUState *cpu)
case KVM_EXIT_INTERNAL_ERROR:
ret = kvm_handle_internal_error(cpu, run);
break;
+ case KVM_EXIT_DIRTY_RING_FULL:
+ /*
+ * We shouldn't continue if the dirty ring of this vcpu is
+ * still full. Got kicked by KVM_RESET_DIRTY_RINGS.
+ */
+ trace_kvm_dirty_ring_full(cpu->cpu_index);
+ qemu_mutex_lock_iothread();
+ kvm_dirty_ring_reap(kvm_state);
+ qemu_mutex_unlock_iothread();
+ ret = 0;
+ break;
case KVM_EXIT_SYSTEM_EVENT:
switch (run->system_event.type) {
case KVM_SYSTEM_EVENT_SHUTDOWN:
diff --git a/accel/kvm/trace-events b/accel/kvm/trace-events
index e15ae8980d..72a01320a1 100644
--- a/accel/kvm/trace-events
+++ b/accel/kvm/trace-events
@@ -18,4 +18,11 @@ kvm_set_ioeventfd_pio(int fd, uint16_t addr, uint32_t val, bool assign, uint32_t
kvm_set_user_memory(uint32_t slot, uint32_t flags, uint64_t guest_phys_addr, uint64_t memory_size, uint64_t userspace_addr, int ret) "Slot#%d flags=0x%x gpa=0x%"PRIx64 " size=0x%"PRIx64 " ua=0x%"PRIx64 " ret=%d"
kvm_clear_dirty_log(uint32_t slot, uint64_t start, uint32_t size) "slot#%"PRId32" start 0x%"PRIx64" size 0x%"PRIx32
kvm_resample_fd_notify(int gsi) "gsi %d"
+kvm_dirty_ring_full(int id) "vcpu %d"
+kvm_dirty_ring_reap_vcpu(int id) "vcpu %d"
+kvm_dirty_ring_page(int vcpu, uint32_t slot, uint64_t offset) "vcpu %d fetch %"PRIu32" offset 0x%"PRIx64
+kvm_dirty_ring_reaper(const char *s) "%s"
+kvm_dirty_ring_reap(uint64_t count, int64_t t) "reaped %"PRIu64" pages (took %"PRIi64" us)"
+kvm_dirty_ring_reaper_kick(const char *reason) "%s"
+kvm_dirty_ring_flush(int finished) "%d"
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index d45f78290e..753ca90668 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -329,6 +329,10 @@ struct qemu_work_item;
* @ignore_memory_transaction_failures: Cached copy of the MachineState
* flag of the same name: allows the board to suppress calling of the
* CPU do_transaction_failed hook function.
+ * @kvm_dirty_gfns: Points to the KVM dirty ring for this CPU when KVM dirty
+ * ring is enabled.
+ * @kvm_fetch_index: Keeps the index that we last fetched from the per-vCPU
+ * dirty ring structure.
*
* State of one CPU core or thread.
*/
@@ -400,9 +404,12 @@ struct CPUState {
*/
uintptr_t mem_io_pc;
+ /* Only used in KVM */
int kvm_fd;
struct KVMState *kvm_state;
struct kvm_run *kvm_run;
+ struct kvm_dirty_gfn *kvm_dirty_gfns;
+ uint32_t kvm_fetch_index;
/* Used for events with 'vcpu' and *without* the 'disabled' properties */
DECLARE_BITMAP(trace_dstate_delayed, CPU_TRACE_DSTATE_MAX_EVENTS);
--
2.27.0
^ permalink raw reply related [flat|nested] 23+ messages in thread* Re: [PULL 00/20] Misc patches for 2020-05-17
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (19 preceding siblings ...)
2021-05-17 11:20 ` [PULL 20/20] KVM: Dirty ring support Paolo Bonzini
@ 2021-05-17 12:15 ` no-reply
2021-05-18 15:16 ` Peter Maydell
21 siblings, 0 replies; 23+ messages in thread
From: no-reply @ 2021-05-17 12:15 UTC (permalink / raw)
To: pbonzini; +Cc: qemu-devel
Patchew URL: https://patchew.org/QEMU/20210517112001.2564006-1-pbonzini@redhat.com/
Hi,
This series seems to have some coding style problems. See output below for
more information:
Type: series
Message-id: 20210517112001.2564006-1-pbonzini@redhat.com
Subject: [PULL 00/20] Misc patches for 2020-05-17
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
* [new tag] patchew/20210517112001.2564006-1-pbonzini@redhat.com -> patchew/20210517112001.2564006-1-pbonzini@redhat.com
Switched to a new branch 'test'
77eb22a KVM: Dirty ring support
520cdc9 KVM: Disable manual dirty log when dirty ring enabled
d672dde KVM: Add dirty-ring-size property
f325098 KVM: Cache kvm slot dirty bitmap size
568e8ff KVM: Simplify dirty log sync in kvm_set_phys_mem
7d5d872 KVM: Provide helper to sync dirty bitmap from slot to ramblock
7b2937f KVM: Provide helper to get kvm dirty log
8160151 KVM: Create the KVMSlot dirty bitmap on flag changes
a013508 KVM: Use a big lock to replace per-kml slots_lock
3549fdf memory: Introduce log_sync_global() to memory listener
415d0d0 KVM: do not allow setting properties at runtime
d44087e qtest: add a QOM object for qtest
b21d58b object: add more commands to preconfig mode
80d034a2 meson: bump submodule to 0.57.2
91a9f7f i386/cpu: Expose AVX_VNNI instruction to guest
4ed41f8 backends/tpm: Replace qemu_mutex_lock calls with QEMU_LOCK_GUARD
525090e hw/mem/nvdimm: Use Kconfig 'imply' instead of 'depends on'
5451f01 configure: simplify assignment to GIT_SUBMODULES
fd5bbbe configure: check for submodules if --with-git-submodules=ignore
4a71809 configure: Only clone softfloat-3 repositories if TCG is enabled
=== OUTPUT BEGIN ===
1/20 Checking commit 4a7180969269 (configure: Only clone softfloat-3 repositories if TCG is enabled)
ERROR: Doubly-encoded UTF-8
#7:
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
total: 1 errors, 0 warnings, 19 lines checked
Patch 1/20 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
2/20 Checking commit fd5bbbe5111e (configure: check for submodules if --with-git-submodules=ignore)
3/20 Checking commit 5451f0196365 (configure: simplify assignment to GIT_SUBMODULES)
4/20 Checking commit 525090e9e978 (hw/mem/nvdimm: Use Kconfig 'imply' instead of 'depends on')
ERROR: Doubly-encoded UTF-8
#16:
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
total: 1 errors, 0 warnings, 30 lines checked
Patch 4/20 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
5/20 Checking commit 4ed41f88b595 (backends/tpm: Replace qemu_mutex_lock calls with QEMU_LOCK_GUARD)
6/20 Checking commit 91a9f7fa5700 (i386/cpu: Expose AVX_VNNI instruction to guest)
7/20 Checking commit 80d034a230fb (meson: bump submodule to 0.57.2)
8/20 Checking commit b21d58bf2249 (object: add more commands to preconfig mode)
9/20 Checking commit d44087e84505 (qtest: add a QOM object for qtest)
10/20 Checking commit 415d0d0f4894 (KVM: do not allow setting properties at runtime)
ERROR: line over 90 characters
#33: FILE: accel/kvm/kvm-all.c:3115:
+ error_setg(errp, "Cannot set properties after the accelerator has been initialized");
ERROR: line over 90 characters
#45: FILE: accel/kvm/kvm-all.c:3134:
+ error_setg(errp, "Cannot set properties after the accelerator has been initialized");
total: 2 errors, 0 warnings, 37 lines checked
Patch 10/20 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
11/20 Checking commit 3549fdff41f9 (memory: Introduce log_sync_global() to memory listener)
12/20 Checking commit a01350856d61 (KVM: Use a big lock to replace per-kml slots_lock)
13/20 Checking commit 8160151d18d7 (KVM: Create the KVMSlot dirty bitmap on flag changes)
14/20 Checking commit 7b2937fa36c2 (KVM: Provide helper to get kvm dirty log)
15/20 Checking commit 7d5d87209ba9 (KVM: Provide helper to sync dirty bitmap from slot to ramblock)
16/20 Checking commit 568e8ffbabee (KVM: Simplify dirty log sync in kvm_set_phys_mem)
17/20 Checking commit f32509862618 (KVM: Cache kvm slot dirty bitmap size)
18/20 Checking commit d672dde943f7 (KVM: Add dirty-ring-size property)
ERROR: line over 90 characters
#59: FILE: accel/kvm/kvm-all.c:3206:
+ error_setg(errp, "Cannot set properties after the accelerator has been initialized");
total: 1 errors, 0 warnings, 94 lines checked
Patch 18/20 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
19/20 Checking commit 520cdc91f820 (KVM: Disable manual dirty log when dirty ring enabled)
20/20 Checking commit 77eb22ac14f1 (KVM: Dirty ring support)
WARNING: line over 80 characters
#451: FILE: accel/kvm/kvm-all.c:2466:
+ "Suggested mininum value is 1024.", strerror(-ret));
total: 0 errors, 1 warnings, 498 lines checked
Patch 20/20 has style problems, please review. If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/20210517112001.2564006-1-pbonzini@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
^ permalink raw reply [flat|nested] 23+ messages in thread* Re: [PULL 00/20] Misc patches for 2020-05-17
2021-05-17 11:19 [PULL 00/20] Misc patches for 2020-05-17 Paolo Bonzini
` (20 preceding siblings ...)
2021-05-17 12:15 ` [PULL 00/20] Misc patches for 2020-05-17 no-reply
@ 2021-05-18 15:16 ` Peter Maydell
21 siblings, 0 replies; 23+ messages in thread
From: Peter Maydell @ 2021-05-18 15:16 UTC (permalink / raw)
To: Paolo Bonzini; +Cc: QEMU Developers
On Mon, 17 May 2021 at 12:24, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> The following changes since commit 6005ee07c380cbde44292f5f6c96e7daa70f4f7d:
>
> Merge remote-tracking branch 'remotes/mst/tags/for_upstream' into staging (2021-05-16 17:22:46 +0100)
>
> are available in the Git repository at:
>
> https://gitlab.com/bonzini/qemu.git tags/for-upstream
>
> for you to fetch changes up to 9b2de449e9593facd562fa7478b5ab15c9a8b588:
>
> KVM: Dirty ring support (2021-05-17 06:16:22 -0400)
>
> ----------------------------------------------------------------
> * submodule cleanups (Philippe, myself)
> * tiny step towards a usable preconfig mode (myself)
> * bump Meson submodule (myself)
> * Kconfig and LOCK_GUARD cleanups (philippe)
> * new x86 CPUID feature (Yang Zhong)
> * "-object qtest" support (myself)
> * Dirty ring support for KVM (Peter)
>
> ----------------------------------------------------------------
> Paolo Bonzini (6):
> configure: check for submodules if --with-git-submodules=ignore
> configure: simplify assignment to GIT_SUBMODULES
> meson: bump submodule to 0.57.2
> object: add more commands to preconfig mode
> qtest: add a QOM object for qtest
> KVM: do not allow setting properties at runtime
>
> Peter Xu (10):
> memory: Introduce log_sync_global() to memory listener
> KVM: Use a big lock to replace per-kml slots_lock
> KVM: Create the KVMSlot dirty bitmap on flag changes
> KVM: Provide helper to get kvm dirty log
> KVM: Provide helper to sync dirty bitmap from slot to ramblock
> KVM: Simplify dirty log sync in kvm_set_phys_mem
> KVM: Cache kvm slot dirty bitmap size
> KVM: Add dirty-ring-size property
> KVM: Disable manual dirty log when dirty ring enabled
> KVM: Dirty ring support
>
> Philippe Mathieu-Daudé (3):
> configure: Only clone softfloat-3 repositories if TCG is enabled
> hw/mem/nvdimm: Use Kconfig 'imply' instead of 'depends on'
> backends/tpm: Replace qemu_mutex_lock calls with QEMU_LOCK_GUARD
>
> Yang Zhong (1):
> i386/cpu: Expose AVX_VNNI instruction to guest
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
ERROR: pull request includes tag with UTF-8 error in person name
-- PMM
^ permalink raw reply [flat|nested] 23+ messages in thread