* [PULL 0/4] Python patches
@ 2022-02-03 1:59 John Snow
2022-02-03 1:59 ` [PULL 1/4] python/aqmp: Fix negotiation with pre-"oob" QEMU John Snow
` (5 more replies)
0 siblings, 6 replies; 16+ messages in thread
From: John Snow @ 2022-02-03 1:59 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Kevin Wolf, qemu-block, Peter Maydell,
Markus Armbruster, Hanna Reitz, Cleber Rosa, John Snow
The following changes since commit 47cc1a3655135b89fa75c2824fbddd29df874612:
Merge remote-tracking branch 'remotes/kwolf-gitlab/tags/for-upstream' into staging (2022-02-01 19:48:15 +0000)
are available in the Git repository at:
https://gitlab.com/jsnow/qemu.git tags/python-pull-request
for you to fetch changes up to b0b662bb2b340d63529672b5bdae596a6243c4d0:
python/aqmp: add socket bind step to legacy.py (2022-02-02 14:12:22 -0500)
----------------------------------------------------------------
Python patches
Peter: I expect this to address the iotest 040,041 failures you observed
on NetBSD. If it doesn't, let me know.
----------------------------------------------------------------
John Snow (4):
python/aqmp: Fix negotiation with pre-"oob" QEMU
python/machine: raise VMLaunchFailure exception from launch()
python: upgrade mypy to 0.780
python/aqmp: add socket bind step to legacy.py
python/Pipfile.lock | 66 +++++++++++++----------
python/qemu/aqmp/legacy.py | 3 ++
python/qemu/aqmp/protocol.py | 41 ++++++++++++--
python/qemu/aqmp/qmp_client.py | 4 +-
python/qemu/machine/machine.py | 45 +++++++++++++---
python/setup.cfg | 2 +-
tests/qemu-iotests/tests/mirror-top-perms | 3 +-
7 files changed, 123 insertions(+), 41 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PULL 1/4] python/aqmp: Fix negotiation with pre-"oob" QEMU
2022-02-03 1:59 [PULL 0/4] Python patches John Snow
@ 2022-02-03 1:59 ` John Snow
2022-02-03 1:59 ` [PULL 2/4] python/machine: raise VMLaunchFailure exception from launch() John Snow
` (4 subsequent siblings)
5 siblings, 0 replies; 16+ messages in thread
From: John Snow @ 2022-02-03 1:59 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Kevin Wolf, qemu-block, Peter Maydell,
Markus Armbruster, Hanna Reitz, Cleber Rosa, John Snow
QEMU versions prior to the "oob" capability *also* can't accept the
"enable" keyword argument at all. Fix the handshake process with older
QEMU versions.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/qemu/aqmp/qmp_client.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/python/qemu/aqmp/qmp_client.py b/python/qemu/aqmp/qmp_client.py
index f1a845cc82..90a8737f03 100644
--- a/python/qemu/aqmp/qmp_client.py
+++ b/python/qemu/aqmp/qmp_client.py
@@ -292,9 +292,9 @@ async def _negotiate(self) -> None:
"""
self.logger.debug("Negotiating capabilities ...")
- arguments: Dict[str, List[str]] = {'enable': []}
+ arguments: Dict[str, List[str]] = {}
if self._greeting and 'oob' in self._greeting.QMP.capabilities:
- arguments['enable'].append('oob')
+ arguments.setdefault('enable', []).append('oob')
msg = self.make_execute_msg('qmp_capabilities', arguments=arguments)
# It's not safe to use execute() here, because the reader/writers
--
2.31.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PULL 2/4] python/machine: raise VMLaunchFailure exception from launch()
2022-02-03 1:59 [PULL 0/4] Python patches John Snow
2022-02-03 1:59 ` [PULL 1/4] python/aqmp: Fix negotiation with pre-"oob" QEMU John Snow
@ 2022-02-03 1:59 ` John Snow
2022-02-03 1:59 ` [PULL 3/4] python: upgrade mypy to 0.780 John Snow
` (3 subsequent siblings)
5 siblings, 0 replies; 16+ messages in thread
From: John Snow @ 2022-02-03 1:59 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Kevin Wolf, qemu-block, Peter Maydell,
Markus Armbruster, Hanna Reitz, Cleber Rosa, John Snow
This allows us to pack in some extra information about the failure,
which guarantees that if the caller did not *intentionally* cause a
failure (by capturing this Exception), some pretty good clues will be
printed at the bottom of the traceback information.
This will help make failures in the event of a non-negative return code
more obvious when they go unhandled; the current behavior in
_post_shutdown() is to print a warning message only in the event of
signal-based terminations (for negative return codes).
(Note: In Python, catching BaseException instead of Exception catches a
broader array of Exception events, including SystemExit and
KeyboardInterrupt. We do not want to "wrap" such exceptions as a
VMLaunchFailure, because that will 'downgrade' the exception from a
BaseException to a regular Exception. We do, however, want to perform
cleanup in either case, so catch on the broadest scope and
wrap-and-re-raise only in the more targeted scope.)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Hanna Reitz <hreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/qemu/machine/machine.py | 45 ++++++++++++++++++++---
tests/qemu-iotests/tests/mirror-top-perms | 3 +-
2 files changed, 40 insertions(+), 8 deletions(-)
diff --git a/python/qemu/machine/machine.py b/python/qemu/machine/machine.py
index 67ab06ca2b..a5972fab4d 100644
--- a/python/qemu/machine/machine.py
+++ b/python/qemu/machine/machine.py
@@ -74,6 +74,35 @@ class QEMUMachineAddDeviceError(QEMUMachineError):
"""
+class VMLaunchFailure(QEMUMachineError):
+ """
+ Exception raised when a VM launch was attempted, but failed.
+ """
+ def __init__(self, exitcode: Optional[int],
+ command: str, output: Optional[str]):
+ super().__init__(exitcode, command, output)
+ self.exitcode = exitcode
+ self.command = command
+ self.output = output
+
+ def __str__(self) -> str:
+ ret = ''
+ if self.__cause__ is not None:
+ name = type(self.__cause__).__name__
+ reason = str(self.__cause__)
+ if reason:
+ ret += f"{name}: {reason}"
+ else:
+ ret += f"{name}"
+ ret += '\n'
+
+ if self.exitcode is not None:
+ ret += f"\tExit code: {self.exitcode}\n"
+ ret += f"\tCommand: {self.command}\n"
+ ret += f"\tOutput: {self.output}\n"
+ return ret
+
+
class AbnormalShutdown(QEMUMachineError):
"""
Exception raised when a graceful shutdown was requested, but not performed.
@@ -397,7 +426,7 @@ def launch(self) -> None:
try:
self._launch()
- except:
+ except BaseException as exc:
# We may have launched the process but it may
# have exited before we could connect via QMP.
# Assume the VM didn't launch or is exiting.
@@ -408,11 +437,15 @@ def launch(self) -> None:
else:
self._post_shutdown()
- LOG.debug('Error launching VM')
- if self._qemu_full_args:
- LOG.debug('Command: %r', ' '.join(self._qemu_full_args))
- if self._iolog:
- LOG.debug('Output: %r', self._iolog)
+ if isinstance(exc, Exception):
+ raise VMLaunchFailure(
+ exitcode=self.exitcode(),
+ command=' '.join(self._qemu_full_args),
+ output=self._iolog
+ ) from exc
+
+ # Don't wrap 'BaseException'; doing so would downgrade
+ # that exception. However, we still want to clean up.
raise
def _launch(self) -> None:
diff --git a/tests/qemu-iotests/tests/mirror-top-perms b/tests/qemu-iotests/tests/mirror-top-perms
index 0a51a613f3..b5849978c4 100755
--- a/tests/qemu-iotests/tests/mirror-top-perms
+++ b/tests/qemu-iotests/tests/mirror-top-perms
@@ -21,7 +21,6 @@
import os
-from qemu.aqmp import ConnectError
from qemu.machine import machine
from qemu.qmp import QMPConnectError
@@ -107,7 +106,7 @@ class TestMirrorTopPerms(iotests.QMPTestCase):
self.vm_b.launch()
print('ERROR: VM B launched successfully, '
'this should not have happened')
- except (QMPConnectError, ConnectError):
+ except (QMPConnectError, machine.VMLaunchFailure):
assert 'Is another process using the image' in self.vm_b.get_log()
result = self.vm.qmp('block-job-cancel',
--
2.31.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PULL 3/4] python: upgrade mypy to 0.780
2022-02-03 1:59 [PULL 0/4] Python patches John Snow
2022-02-03 1:59 ` [PULL 1/4] python/aqmp: Fix negotiation with pre-"oob" QEMU John Snow
2022-02-03 1:59 ` [PULL 2/4] python/machine: raise VMLaunchFailure exception from launch() John Snow
@ 2022-02-03 1:59 ` John Snow
2022-02-03 1:59 ` [PULL 4/4] python/aqmp: add socket bind step to legacy.py John Snow
` (2 subsequent siblings)
5 siblings, 0 replies; 16+ messages in thread
From: John Snow @ 2022-02-03 1:59 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Kevin Wolf, qemu-block, Peter Maydell,
Markus Armbruster, Hanna Reitz, Cleber Rosa, John Snow
We need a slightly newer version of mypy in order to use some features
of the asyncio server functions in the next commit.
(Note: pipenv is not really suited to upgrading individual packages; I
need to replace this tool with something better for the task. For now,
the miscellaneous updates not related to the mypy upgrade are simply
beyond my control. It's on my list to take care of soon.)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/Pipfile.lock | 66 ++++++++++++++++++++++++++-------------------
python/setup.cfg | 2 +-
2 files changed, 40 insertions(+), 28 deletions(-)
diff --git a/python/Pipfile.lock b/python/Pipfile.lock
index d2a7dbd88b..ce46404ce0 100644
--- a/python/Pipfile.lock
+++ b/python/Pipfile.lock
@@ -1,7 +1,7 @@
{
"_meta": {
"hash": {
- "sha256": "784b327272db32403d5a488507853b5afba850ba26a5948e5b6a90c1baef2d9c"
+ "sha256": "f1a25654d884a5b450e38d78b1f2e3ebb9073e421cc4358d4bbb83ac251a5670"
},
"pipfile-spec": 6,
"requires": {
@@ -34,7 +34,7 @@
"sha256:09bdb456e02564731f8b5957cdd0c98a7f01d2db5e90eb1d794c353c28bfd705",
"sha256:6a8a51f64dae307f6e0c9db752b66a7951e282389d8362cc1d39a56f3feeb31d"
],
- "markers": "python_version ~= '3.6'",
+ "index": "pypi",
"version": "==2.6.0"
},
"avocado-framework": {
@@ -50,6 +50,7 @@
"sha256:106fef6dc37dd8c0e2c0a60d3fca3e77460a48907f335fa28420463a6f799736",
"sha256:23e223426b28491b1ced97dc3bbe183027419dfc7982b4fa2f05d5f3ff10711c"
],
+ "index": "pypi",
"version": "==0.3.2"
},
"filelock": {
@@ -57,6 +58,7 @@
"sha256:18d82244ee114f543149c66a6e0c14e9c4f8a1044b5cdaadd0f82159d6a6ff59",
"sha256:929b7d63ec5b7d6b71b0fa5ac14e030b3f70b75747cef1b10da9b879fef15836"
],
+ "index": "pypi",
"version": "==3.0.12"
},
"flake8": {
@@ -88,7 +90,7 @@
"sha256:54161657e8ffc76596c4ede7080ca68cb02962a2e074a2586b695a93a925d36e",
"sha256:e962bff7440364183203d179d7ae9ad90cb1f2b74dcb84300e88ecc42dca3351"
],
- "markers": "python_version < '3.7'",
+ "index": "pypi",
"version": "==5.1.4"
},
"isort": {
@@ -124,7 +126,7 @@
"sha256:ed361bb83436f117f9917d282a456f9e5009ea12fd6de8742d1a4752c3017e93",
"sha256:f5144c75445ae3ca2057faac03fda5a902eff196702b0a24daf1d6ce0650514b"
],
- "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'",
+ "index": "pypi",
"version": "==1.6.0"
},
"mccabe": {
@@ -136,23 +138,23 @@
},
"mypy": {
"hashes": [
- "sha256:15b948e1302682e3682f11f50208b726a246ab4e6c1b39f9264a8796bb416aa2",
- "sha256:219a3116ecd015f8dca7b5d2c366c973509dfb9a8fc97ef044a36e3da66144a1",
- "sha256:3b1fc683fb204c6b4403a1ef23f0b1fac8e4477091585e0c8c54cbdf7d7bb164",
- "sha256:3beff56b453b6ef94ecb2996bea101a08f1f8a9771d3cbf4988a61e4d9973761",
- "sha256:7687f6455ec3ed7649d1ae574136835a4272b65b3ddcf01ab8704ac65616c5ce",
- "sha256:7ec45a70d40ede1ec7ad7f95b3c94c9cf4c186a32f6bacb1795b60abd2f9ef27",
- "sha256:86c857510a9b7c3104cf4cde1568f4921762c8f9842e987bc03ed4f160925754",
- "sha256:8a627507ef9b307b46a1fea9513d5c98680ba09591253082b4c48697ba05a4ae",
- "sha256:8dfb69fbf9f3aeed18afffb15e319ca7f8da9642336348ddd6cab2713ddcf8f9",
- "sha256:a34b577cdf6313bf24755f7a0e3f3c326d5c1f4fe7422d1d06498eb25ad0c600",
- "sha256:a8ffcd53cb5dfc131850851cc09f1c44689c2812d0beb954d8138d4f5fc17f65",
- "sha256:b90928f2d9eb2f33162405f32dde9f6dcead63a0971ca8a1b50eb4ca3e35ceb8",
- "sha256:c56ffe22faa2e51054c5f7a3bc70a370939c2ed4de308c690e7949230c995913",
- "sha256:f91c7ae919bbc3f96cd5e5b2e786b2b108343d1d7972ea130f7de27fdd547cf3"
+ "sha256:00cb1964a7476e871d6108341ac9c1a857d6bd20bf5877f4773ac5e9d92cd3cd",
+ "sha256:127de5a9b817a03a98c5ae8a0c46a20dc44442af6dcfa2ae7f96cb519b312efa",
+ "sha256:1f3976a945ad7f0a0727aafdc5651c2d3278e3c88dee94e2bf75cd3386b7b2f4",
+ "sha256:2f8c098f12b402c19b735aec724cc9105cc1a9eea405d08814eb4b14a6fb1a41",
+ "sha256:4ef13b619a289aa025f2273e05e755f8049bb4eaba6d703a425de37d495d178d",
+ "sha256:5d142f219bf8c7894dfa79ebfb7d352c4c63a325e75f10dfb4c3db9417dcd135",
+ "sha256:62eb5dd4ea86bda8ce386f26684f7f26e4bfe6283c9f2b6ca6d17faf704dcfad",
+ "sha256:64c36eb0936d0bfb7d8da49f92c18e312ad2e3ed46e5548ae4ca997b0d33bd59",
+ "sha256:75eed74d2faf2759f79c5f56f17388defd2fc994222312ec54ee921e37b31ad4",
+ "sha256:974bebe3699b9b46278a7f076635d219183da26e1a675c1f8243a69221758273",
+ "sha256:a5e5bb12b7982b179af513dddb06fca12285f0316d74f3964078acbfcf4c68f2",
+ "sha256:d31291df31bafb997952dc0a17ebb2737f802c754aed31dd155a8bfe75112c57",
+ "sha256:d3b4941de44341227ece1caaf5b08b23e42ad4eeb8b603219afb11e9d4cfb437",
+ "sha256:eadb865126da4e3c4c95bdb47fe1bb087a3e3ea14d39a3b13224b8a4d9f9a102"
],
"index": "pypi",
- "version": "==0.770"
+ "version": "==0.780"
},
"mypy-extensions": {
"hashes": [
@@ -166,7 +168,7 @@
"sha256:5b327ac1320dc863dca72f4514ecc086f31186744b84a230374cc1fd776feae5",
"sha256:67714da7f7bc052e064859c05c595155bd1ee9f69f76557e21f051443c20947a"
],
- "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "index": "pypi",
"version": "==20.9"
},
"pluggy": {
@@ -174,7 +176,7 @@
"sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0",
"sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"
],
- "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "index": "pypi",
"version": "==0.13.1"
},
"py": {
@@ -182,7 +184,7 @@
"sha256:21b81bda15b66ef5e1a777a21c4dcd9c20ad3efd0b3f817e7a809035269e1bd3",
"sha256:3b80836aa6d1feeaa108e046da6423ab8f6ceda6468545ae8d02d9d58d18818a"
],
- "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "index": "pypi",
"version": "==1.10.0"
},
"pycodestyle": {
@@ -205,7 +207,7 @@
"sha256:a18f47b506a429f6f4b9df81bb02beab9ca21d0a5fee38ed15aef65f0545519f",
"sha256:d66e804411278594d764fc69ec36ec13d9ae9147193a1740cd34d272ca383b8e"
],
- "markers": "python_version >= '3.5'",
+ "index": "pypi",
"version": "==2.9.0"
},
"pylint": {
@@ -221,13 +223,21 @@
"sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1",
"sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
],
- "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "index": "pypi",
"version": "==2.4.7"
},
"qemu": {
"editable": true,
"path": "."
},
+ "setuptools": {
+ "hashes": [
+ "sha256:22c7348c6d2976a52632c67f7ab0cdf40147db7789f9aed18734643fe9cf3373",
+ "sha256:4ce92f1e1f8f01233ee9952c04f6b81d1e02939d6e1b488428154974a4d0783e"
+ ],
+ "markers": "python_version >= '3.6'",
+ "version": "==59.6.0"
+ },
"six": {
"hashes": [
"sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926",
@@ -294,19 +304,21 @@
"sha256:50b6f157849174217d0656f99dc82fe932884fb250826c18350e159ec6cdf342",
"sha256:779383f6086d90c99ae41cf0ff39aac8a7937a9283ce0a414e5dd782f4c94a84"
],
- "markers": "python_version < '3.8'",
+ "index": "pypi",
"version": "==3.10.0.0"
},
"urwid": {
"hashes": [
"sha256:588bee9c1cb208d0906a9f73c613d2bd32c3ed3702012f51efe318a3f2127eae"
],
+ "index": "pypi",
"version": "==2.1.2"
},
"urwid-readline": {
"hashes": [
"sha256:018020cbc864bb5ed87be17dc26b069eae2755cb29f3a9c569aac3bded1efaf4"
],
+ "index": "pypi",
"version": "==0.13"
},
"virtualenv": {
@@ -314,7 +326,7 @@
"sha256:14fdf849f80dbb29a4eb6caa9875d476ee2a5cf76a5f5415fa2f1606010ab467",
"sha256:2b0126166ea7c9c3661f5b8e06773d28f83322de7a3ff7d06f0aed18c9de6a76"
],
- "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "index": "pypi",
"version": "==20.4.7"
},
"wrapt": {
@@ -328,7 +340,7 @@
"sha256:3607921face881ba3e026887d8150cca609d517579abe052ac81fc5aeffdbd76",
"sha256:51cb66cc54621609dd593d1787f286ee42a5c0adbb4b29abea5a63edc3e03098"
],
- "markers": "python_version < '3.10'",
+ "index": "pypi",
"version": "==3.4.1"
}
}
diff --git a/python/setup.cfg b/python/setup.cfg
index 3fb18f845d..18aea2bab3 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -41,7 +41,7 @@ devel =
flake8 >= 3.6.0
fusepy >= 2.0.4
isort >= 5.1.2
- mypy >= 0.770
+ mypy >= 0.780
pylint >= 2.8.0
tox >= 3.18.0
urwid >= 2.1.2
--
2.31.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PULL 4/4] python/aqmp: add socket bind step to legacy.py
2022-02-03 1:59 [PULL 0/4] Python patches John Snow
` (2 preceding siblings ...)
2022-02-03 1:59 ` [PULL 3/4] python: upgrade mypy to 0.780 John Snow
@ 2022-02-03 1:59 ` John Snow
2022-02-03 16:20 ` [PULL 0/4] Python patches Peter Maydell
2022-02-04 17:03 ` Peter Maydell
5 siblings, 0 replies; 16+ messages in thread
From: John Snow @ 2022-02-03 1:59 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, Kevin Wolf, qemu-block, Peter Maydell,
Markus Armbruster, Hanna Reitz, Cleber Rosa, John Snow
The synchronous QMP library would bind to the server address during
__init__(). The new library delays this to the accept() call, because
binding occurs inside of the call to start_[unix_]server(), which is an
async method -- so it cannot happen during __init__ anymore.
Python 3.7+ adds the ability to create the server (and thus the bind()
call) and begin the active listening in separate steps, but we don't
have that functionality in 3.6, our current minimum.
Therefore ... Add a temporary workaround that allows the synchronous
version of the client to bind the socket in advance, guaranteeing that
there will be a UNIX socket in the filesystem ready for the QEMU client
to connect to without a race condition.
(Yes, it's a bit ugly. Fixing it more nicely will have to wait until our
minimum Python version is 3.7+.)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Message-id: 20220201041134.1237016-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/qemu/aqmp/legacy.py | 3 +++
python/qemu/aqmp/protocol.py | 41 +++++++++++++++++++++++++++++++++---
2 files changed, 41 insertions(+), 3 deletions(-)
diff --git a/python/qemu/aqmp/legacy.py b/python/qemu/aqmp/legacy.py
index 0890f95b16..6baa5f3409 100644
--- a/python/qemu/aqmp/legacy.py
+++ b/python/qemu/aqmp/legacy.py
@@ -56,6 +56,9 @@ def __init__(self, address: SocketAddrT,
self._address = address
self._timeout: Optional[float] = None
+ if server:
+ self._aqmp._bind_hack(address) # pylint: disable=protected-access
+
_T = TypeVar('_T')
def _sync(
diff --git a/python/qemu/aqmp/protocol.py b/python/qemu/aqmp/protocol.py
index 50e973c2f2..33358f5cd7 100644
--- a/python/qemu/aqmp/protocol.py
+++ b/python/qemu/aqmp/protocol.py
@@ -15,6 +15,7 @@
from enum import Enum
from functools import wraps
import logging
+import socket
from ssl import SSLContext
from typing import (
Any,
@@ -238,6 +239,9 @@ def __init__(self, name: Optional[str] = None) -> None:
self._runstate = Runstate.IDLE
self._runstate_changed: Optional[asyncio.Event] = None
+ # Workaround for bind()
+ self._sock: Optional[socket.socket] = None
+
def __repr__(self) -> str:
cls_name = type(self).__name__
tokens = []
@@ -427,6 +431,34 @@ async def _establish_connection(
else:
await self._do_connect(address, ssl)
+ def _bind_hack(self, address: Union[str, Tuple[str, int]]) -> None:
+ """
+ Used to create a socket in advance of accept().
+
+ This is a workaround to ensure that we can guarantee timing of
+ precisely when a socket exists to avoid a connection attempt
+ bouncing off of nothing.
+
+ Python 3.7+ adds a feature to separate the server creation and
+ listening phases instead, and should be used instead of this
+ hack.
+ """
+ if isinstance(address, tuple):
+ family = socket.AF_INET
+ else:
+ family = socket.AF_UNIX
+
+ sock = socket.socket(family, socket.SOCK_STREAM)
+ sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
+
+ try:
+ sock.bind(address)
+ except:
+ sock.close()
+ raise
+
+ self._sock = sock
+
@upper_half
async def _do_accept(self, address: SocketAddrT,
ssl: Optional[SSLContext] = None) -> None:
@@ -464,24 +496,27 @@ async def _client_connected_cb(reader: asyncio.StreamReader,
if isinstance(address, tuple):
coro = asyncio.start_server(
_client_connected_cb,
- host=address[0],
- port=address[1],
+ host=None if self._sock else address[0],
+ port=None if self._sock else address[1],
ssl=ssl,
backlog=1,
limit=self._limit,
+ sock=self._sock,
)
else:
coro = asyncio.start_unix_server(
_client_connected_cb,
- path=address,
+ path=None if self._sock else address,
ssl=ssl,
backlog=1,
limit=self._limit,
+ sock=self._sock,
)
server = await coro # Starts listening
await connected.wait() # Waits for the callback to fire (and finish)
assert server is None
+ self._sock = None
self.logger.debug("Connection accepted.")
--
2.31.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [PULL 0/4] Python patches
2022-02-03 1:59 [PULL 0/4] Python patches John Snow
` (3 preceding siblings ...)
2022-02-03 1:59 ` [PULL 4/4] python/aqmp: add socket bind step to legacy.py John Snow
@ 2022-02-03 16:20 ` Peter Maydell
2022-02-03 16:38 ` John Snow
2022-02-04 17:03 ` Peter Maydell
5 siblings, 1 reply; 16+ messages in thread
From: Peter Maydell @ 2022-02-03 16:20 UTC (permalink / raw)
To: John Snow
Cc: Eduardo Habkost, Kevin Wolf, qemu-block, qemu-devel,
Markus Armbruster, Hanna Reitz, Cleber Rosa
On Thu, 3 Feb 2022 at 01:59, John Snow <jsnow@redhat.com> wrote:
>
> The following changes since commit 47cc1a3655135b89fa75c2824fbddd29df874612:
>
> Merge remote-tracking branch 'remotes/kwolf-gitlab/tags/for-upstream' into staging (2022-02-01 19:48:15 +0000)
>
> are available in the Git repository at:
>
> https://gitlab.com/jsnow/qemu.git tags/python-pull-request
>
> for you to fetch changes up to b0b662bb2b340d63529672b5bdae596a6243c4d0:
>
> python/aqmp: add socket bind step to legacy.py (2022-02-02 14:12:22 -0500)
>
> ----------------------------------------------------------------
> Python patches
>
> Peter: I expect this to address the iotest 040,041 failures you observed
> on NetBSD. If it doesn't, let me know.
I still see this one, which is different from the 040,041 stuff,
and where 'make check' is for some reason giving a lot less useful
detail. (This is a prexisting intermittent from before this patchset).
[etc]
▶ 175/704 /io/channel/pipe/sync
OK
▶ 175/704 /io/channel/pipe/async
OK
175/704 qemu:unit / test-io-channel-file
OK 0.11s 5 subtests passed
177/704 qemu:unit / test-io-channel-tls
RUNNING
>>> G_TEST_BUILDDIR=/home/qemu/qemu-test.yiYr4m/build/tests/unit MALLOC_PERTURB_=5 G_TEST_SRCDIR=/home/qemu/qemu-test.yiYr4m/src/tests/unit /home/
qemu/qemu-test.yiYr4m/build/tests/unit/test-io-channel-tls --tap -k
▶ 176/704 /io/channel/socket/ipv4-sync
OK
▶ 176/704 /io/channel/socket/ipv4-async
OK
▶ 176/704 /io/channel/socket/ipv4-fd
OK
▶ 176/704 /io/channel/socket/ipv6-sync
OK
▶ 176/704 /io/channel/socket/ipv6-async
OK
▶ 176/704 /io/channel/socket/unix-sync
OK
▶ 176/704 /io/channel/socket/unix-async
OK
▶ 176/704 /io/channel/socket/unix-fd-pass
OK
▶ 176/704 /io/channel/socket/unix-listen-cleanup
OK
176/704 qemu:unit / test-io-channel-socket
OK 0.13s 9 subtests passed
▶ 1/1 qcow2 qsd-jobs OK
1/1 qemu:block / qemu-iotests qcow2 ERROR 243.14s exit status 1
178/704 qemu:unit / test-io-task
RUNNING
>>> G_TEST_BUILDDIR=/home/qemu/qemu-test.yiYr4m/build/tests/unit MALLOC_PERTURB_=194 G_TEST_SRCDIR=/home/qemu/qemu-test.yiYr4m/src/tests/unit /hom
e/qemu/qemu-test.yiYr4m/build/tests/unit/test-io-task --tap -k
▶ 147/704 /bdrv-drain/blockjob/iothread/error/drain_subtree
OK
Summary of Failures:
1/1 qemu:block / qemu-iotests qcow2 ERROR 243.14s exit status 1
Ok: 0
Expected Fail: 0
Fail: 1
Unexpected Pass: 0
Skipped: 0
Timeout: 0
Full log written to /home/qemu/qemu-test.yiYr4m/build/meson-logs/iotestslog.txt
▶ 147/704 /bdrv-drain/deletion/drain
OK
▶ 178/704 /crypto/task/complete
OK
▶ 178/704 /crypto/task/datafree
OK
[etc]
thanks
-- PMM
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PULL 0/4] Python patches
2022-02-03 16:20 ` [PULL 0/4] Python patches Peter Maydell
@ 2022-02-03 16:38 ` John Snow
2022-02-03 16:51 ` Peter Maydell
0 siblings, 1 reply; 16+ messages in thread
From: John Snow @ 2022-02-03 16:38 UTC (permalink / raw)
To: Peter Maydell
Cc: Eduardo Habkost, Kevin Wolf, Qemu-block, qemu-devel,
Markus Armbruster, Hanna Reitz, Cleber Rosa
[-- Attachment #1: Type: text/plain, Size: 3504 bytes --]
On Thu, Feb 3, 2022, 11:20 AM Peter Maydell <peter.maydell@linaro.org>
wrote:
> On Thu, 3 Feb 2022 at 01:59, John Snow <jsnow@redhat.com> wrote:
> >
> > The following changes since commit
> 47cc1a3655135b89fa75c2824fbddd29df874612:
> >
> > Merge remote-tracking branch 'remotes/kwolf-gitlab/tags/for-upstream'
> into staging (2022-02-01 19:48:15 +0000)
> >
> > are available in the Git repository at:
> >
> > https://gitlab.com/jsnow/qemu.git tags/python-pull-request
> >
> > for you to fetch changes up to b0b662bb2b340d63529672b5bdae596a6243c4d0:
> >
> > python/aqmp: add socket bind step to legacy.py (2022-02-02 14:12:22
> -0500)
> >
> > ----------------------------------------------------------------
> > Python patches
> >
> > Peter: I expect this to address the iotest 040,041 failures you observed
> > on NetBSD. If it doesn't, let me know.
>
> I still see this one, which is different from the 040,041 stuff,
> and where 'make check' is for some reason giving a lot less useful
> detail. (This is a prexisting intermittent from before this patchset).
>
I'm assuming there's less detail because of the meson test-runner doing
some io redirection into the logfile.
[etc]
>
▶ 175/704 /io/channel/pipe/sync
> OK
> ▶ 175/704 /io/channel/pipe/async
> OK
> 175/704 qemu:unit / test-io-channel-file
> OK 0.11s 5 subtests passed
>
> 177/704 qemu:unit / test-io-channel-tls
> RUNNING
> >>> G_TEST_BUILDDIR=/home/qemu/qemu-test.yiYr4m/build/tests/unit
> MALLOC_PERTURB_=5 G_TEST_SRCDIR=/home/qemu/qemu-test.yiYr4m/src/tests/unit
> /home/
> qemu/qemu-test.yiYr4m/build/tests/unit/test-io-channel-tls --tap -k
> ▶ 176/704 /io/channel/socket/ipv4-sync
> OK
> ▶ 176/704 /io/channel/socket/ipv4-async
> OK
> ▶ 176/704 /io/channel/socket/ipv4-fd
> OK
> ▶ 176/704 /io/channel/socket/ipv6-sync
> OK
> ▶ 176/704 /io/channel/socket/ipv6-async
> OK
> ▶ 176/704 /io/channel/socket/unix-sync
> OK
> ▶ 176/704 /io/channel/socket/unix-async
> OK
> ▶ 176/704 /io/channel/socket/unix-fd-pass
> OK
> ▶ 176/704 /io/channel/socket/unix-listen-cleanup
> OK
> 176/704 qemu:unit / test-io-channel-socket
> OK 0.13s 9 subtests passed
>
> ▶ 1/1 qcow2 qsd-jobs OK
> 1/1 qemu:block / qemu-iotests qcow2 ERROR 243.14s exit status 1
>
> 178/704 qemu:unit / test-io-task
> RUNNING
> >>> G_TEST_BUILDDIR=/home/qemu/qemu-test.yiYr4m/build/tests/unit
> MALLOC_PERTURB_=194
> G_TEST_SRCDIR=/home/qemu/qemu-test.yiYr4m/src/tests/unit /hom
> e/qemu/qemu-test.yiYr4m/build/tests/unit/test-io-task --tap -k
> ▶ 147/704 /bdrv-drain/blockjob/iothread/error/drain_subtree
> OK
>
> Summary of Failures:
>
> 1/1 qemu:block / qemu-iotests qcow2 ERROR 243.14s exit status 1
>
>
> Ok: 0
> Expected Fail: 0
> Fail: 1
> Unexpected Pass: 0
> Skipped: 0
> Timeout: 0
>
> Full log written to
> /home/qemu/qemu-test.yiYr4m/build/meson-logs/iotestslog.txt
> ▶ 147/704 /bdrv-drain/deletion/drain
> OK
> ▶ 178/704 /crypto/task/complete
> OK
> ▶ 178/704 /crypto/task/datafree
> OK
> [etc]
>
> thanks
> -- PMM
>
Any chance of seeing that meson-logs/iotestslog.txt file?
>
[-- Attachment #2: Type: text/html, Size: 4959 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PULL 0/4] Python patches
2022-02-03 16:38 ` John Snow
@ 2022-02-03 16:51 ` Peter Maydell
2022-02-03 23:22 ` John Snow
0 siblings, 1 reply; 16+ messages in thread
From: Peter Maydell @ 2022-02-03 16:51 UTC (permalink / raw)
To: John Snow
Cc: Eduardo Habkost, Kevin Wolf, Qemu-block, qemu-devel,
Markus Armbruster, Hanna Reitz, Cleber Rosa
On Thu, 3 Feb 2022 at 16:38, John Snow <jsnow@redhat.com> wrote:
> On Thu, Feb 3, 2022, 11:20 AM Peter Maydell <peter.maydell@linaro.org> wrote:
>> Summary of Failures:
>>
>> 1/1 qemu:block / qemu-iotests qcow2 ERROR 243.14s exit status 1
>>
>>
>> Ok: 0
>> Expected Fail: 0
>> Fail: 1
>> Unexpected Pass: 0
>> Skipped: 0
>> Timeout: 0
>>
>> Full log written to /home/qemu/qemu-test.yiYr4m/build/meson-logs/iotestslog.txt
>> ▶ 147/704 /bdrv-drain/deletion/drain
>> OK
>> ▶ 178/704 /crypto/task/complete
>> OK
>> ▶ 178/704 /crypto/task/datafree
>> OK
>> [etc]
> Any chance of seeing that meson-logs/iotestslog.txt file?
Sorry, no. The VM runs, and it produces output to stdout, and
then it goes away again. The test cases and test harnesses
*must* output to standard output any information that might
be useful for diagnosing problems. The same scenario applies
for the gitlab CI jobs -- all we get is the job's output.
-- PMM
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PULL 0/4] Python patches
2022-02-03 16:51 ` Peter Maydell
@ 2022-02-03 23:22 ` John Snow
2022-02-08 14:40 ` Peter Maydell
0 siblings, 1 reply; 16+ messages in thread
From: John Snow @ 2022-02-03 23:22 UTC (permalink / raw)
To: Peter Maydell
Cc: Eduardo Habkost, Kevin Wolf, Qemu-block, qemu-devel,
Markus Armbruster, Hanna Reitz, Cleber Rosa
On Thu, Feb 3, 2022 at 11:52 AM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Thu, 3 Feb 2022 at 16:38, John Snow <jsnow@redhat.com> wrote:
>
> > On Thu, Feb 3, 2022, 11:20 AM Peter Maydell <peter.maydell@linaro.org> wrote:
> >> Summary of Failures:
> >>
> >> 1/1 qemu:block / qemu-iotests qcow2 ERROR 243.14s exit status 1
I'm not too familiar with this new test runner, yet. (Is this error
even anything to do with the python lib? I guess I can't rule it
out...)
I just got a clean run of 'make vm-build-netbsd', so I'm using that
output as reference and making some guesses.
If I search the output for 'qcow2', I see the following output (with
possibly many lines between each hit):
1/1 qemu:block / qemu-iotests qcow2 RUNNING
>>> MALLOC_PERTURB_=205 PYTHON=/usr/pkg/bin/python3.7 /bin/sh /home/qemu/qemu-test.lj6FNa/build/../src/tests/qemu-iotests/../check-block.sh qcow2
▶ 1/1 qcow2 001 OK
▶ 1/1 qcow2 002 OK
▶ 1/1 qcow2 004 OK
... and so on and so forth ...
▶ 1/1 qcow2 299 OK
▶ 1/1 qcow2 313 SKIP
▶ 1/1 qcow2 nbd-qemu-allocation SKIP
▶ 1/1 qcow2 qsd-jobs OK
1/1 qemu:block / qemu-iotests qcow2 OK 176.35s 74
subtests passed
I tried modifying 040 to fail on purpose, and I see:
▶ 1/1 qcow2 039 OK
▶ 1/1 qcow2 040 FAIL
▶ 1/1 qcow2 041 OK
[...]
▶ 1/1 qcow2 nbd-qemu-allocation OK
▶ 1/1 qcow2 qsd-jobs OK
1/1 qemu:block / qemu-iotests qcow2 ERROR 106.06s
exit status 1
Summary of Failures:
1/1 qemu:block / qemu-iotests qcow2 ERROR 106.06s exit status 1
I don't think I see it on the output you mailed, but can you point out
which test is failing, at least? Grepping for 'FAIL' should be
helpful.
--js
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PULL 0/4] Python patches
2022-02-03 1:59 [PULL 0/4] Python patches John Snow
` (4 preceding siblings ...)
2022-02-03 16:20 ` [PULL 0/4] Python patches Peter Maydell
@ 2022-02-04 17:03 ` Peter Maydell
5 siblings, 0 replies; 16+ messages in thread
From: Peter Maydell @ 2022-02-04 17:03 UTC (permalink / raw)
To: John Snow
Cc: Eduardo Habkost, Kevin Wolf, qemu-block, qemu-devel,
Markus Armbruster, Hanna Reitz, Cleber Rosa
On Thu, 3 Feb 2022 at 01:59, John Snow <jsnow@redhat.com> wrote:
>
> The following changes since commit 47cc1a3655135b89fa75c2824fbddd29df874612:
>
> Merge remote-tracking branch 'remotes/kwolf-gitlab/tags/for-upstream' into staging (2022-02-01 19:48:15 +0000)
>
> are available in the Git repository at:
>
> https://gitlab.com/jsnow/qemu.git tags/python-pull-request
>
> for you to fetch changes up to b0b662bb2b340d63529672b5bdae596a6243c4d0:
>
> python/aqmp: add socket bind step to legacy.py (2022-02-02 14:12:22 -0500)
>
> ----------------------------------------------------------------
> Python patches
>
> Peter: I expect this to address the iotest 040,041 failures you observed
> on NetBSD. If it doesn't, let me know.
>
> ----------------------------------------------------------------
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/7.0
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PULL 0/4] Python patches
2022-02-03 23:22 ` John Snow
@ 2022-02-08 14:40 ` Peter Maydell
2022-02-15 17:45 ` John Snow
0 siblings, 1 reply; 16+ messages in thread
From: Peter Maydell @ 2022-02-08 14:40 UTC (permalink / raw)
To: John Snow
Cc: Eduardo Habkost, Kevin Wolf, Qemu-block, qemu-devel,
Markus Armbruster, Hanna Reitz, Cleber Rosa
On Thu, 3 Feb 2022 at 23:22, John Snow <jsnow@redhat.com> wrote:
>
> On Thu, Feb 3, 2022 at 11:52 AM Peter Maydell <peter.maydell@linaro.org> wrote:
> >
> > On Thu, 3 Feb 2022 at 16:38, John Snow <jsnow@redhat.com> wrote:
> >
> > > On Thu, Feb 3, 2022, 11:20 AM Peter Maydell <peter.maydell@linaro.org> wrote:
> > >> Summary of Failures:
> > >>
> > >> 1/1 qemu:block / qemu-iotests qcow2 ERROR 243.14s exit status 1
>
> I'm not too familiar with this new test runner, yet. (Is this error
> even anything to do with the python lib? I guess I can't rule it
> out...)
> I just got a clean run of 'make vm-build-netbsd', so I'm using that
> output as reference and making some guesses.
Rerunning on the netbsd VM with Paolo's "revert the iotests
conversion" patch, here's the output from a failing run, where
iotest 041 failed:
TEST iotest-qcow2: 041 [fail]
QEMU --
"/home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/../../qemu-system-aarch64"
-nodefaults -display none -accel qtest -machine virt
QEMU_IMG --
"/home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/../../qemu-img"
QEMU_IO --
"/home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/../../qemu-io"
--cache writeback --aio threads -f qcow2
QEMU_NBD --
"/home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT -- qcow2
IMGPROTO -- file
PLATFORM -- NetBSD/amd64 localhost 9.2
TEST_DIR -- /home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/scratch
SOCK_DIR -- /tmp/tmp6fiu68sr
GDB_OPTIONS --
VALGRIND_QEMU --
PRINT_QEMU_OUTPUT --
--- /home/qemu/qemu-test.Kywnb7/src/tests/qemu-iotests/041.out
+++ 041.out.bad
@@ -1,5 +1,44 @@
-...........................................................................................................
+........................................ERROR:qemu.aqmp.qmp_client.qemu-14411:Failed
to establish connection: concurrent.futures._base.CancelledError
+E..................................................................
+======================================================================
+ERROR: test_mirror_to_self (__main__.TestSingleBlockdev)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+ File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/machine/machine.py",
line 428, in launch
+ self._launch()
+ File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/machine/machine.py",
line 467, in _launch
+ self._post_launch()
+ File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/machine/qtest.py",
line 147, in _post_launch
+ super()._post_launch()
+ File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/machine/machine.py",
line 369, in _post_launch
+ self._qmp.accept(self._qmp_timer)
+ File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/aqmp/legacy.py",
line 95, in accept
+ timeout
+ File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/aqmp/legacy.py",
line 68, in _sync
+ asyncio.wait_for(future, timeout=timeout)
+ File "/usr/pkg/lib/python3.7/asyncio/base_events.py", line 587, in
run_until_complete
+ return future.result()
+ File "/usr/pkg/lib/python3.7/asyncio/tasks.py", line 449, in wait_for
+ raise futures.TimeoutError()
+concurrent.futures._base.TimeoutError
+
+The above exception was the direct cause of the following exception:
+
+Traceback (most recent call last):
+ File "/home/qemu/qemu-test.Kywnb7/src/tests/qemu-iotests/041", line
233, in setUp
+ TestSingleDrive.setUp(self)
+ File "/home/qemu/qemu-test.Kywnb7/src/tests/qemu-iotests/041", line
54, in setUp
+ self.vm.launch()
+ File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/machine/machine.py",
line 445, in launch
+ ) from exc
+qemu.machine.machine.VMLaunchFailure: TimeoutError
+ Exit code: 1
+ Command:
/home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/../../qemu-system-aarch64
-display none -vga none -chardev
socket,id=mon,path=/tmp/tmp6fiu68sr/qemu-14411-monitor.sock -mon
chardev=mon,mode=control -qtest
unix:path=/tmp/tmp6fiu68sr/qemu-14411-qtest.sock -accel qtest
-nodefaults -display none -accel qtest -machine virt -drive
if=virtio,id=drive0,file=/home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/scratch/test.img,format=qcow2,cache=writeback,aio=threads,node-name=top,backing.node-name=base
+ Output: qemu-system-aarch64: -chardev
socket,id=mon,path=/tmp/tmp6fiu68sr/qemu-14411-monitor.sock: Failed to
connect to '/tmp/tmp6fiu68sr/qemu-14411-monitor.sock': Connection
refused
+
+
+
----------------------------------------------------------------------
Ran 107 tests
-OK
+FAILED (errors=1)
thanks
-- PMM
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PULL 0/4] Python patches
2022-02-08 14:40 ` Peter Maydell
@ 2022-02-15 17:45 ` John Snow
2022-02-15 18:00 ` Peter Maydell
0 siblings, 1 reply; 16+ messages in thread
From: John Snow @ 2022-02-15 17:45 UTC (permalink / raw)
To: Peter Maydell
Cc: Eduardo Habkost, Kevin Wolf, Qemu-block, qemu-devel,
Markus Armbruster, Hanna Reitz, Cleber Rosa
On Tue, Feb 8, 2022 at 9:40 AM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Thu, 3 Feb 2022 at 23:22, John Snow <jsnow@redhat.com> wrote:
> >
> > On Thu, Feb 3, 2022 at 11:52 AM Peter Maydell <peter.maydell@linaro.org> wrote:
> > >
> > > On Thu, 3 Feb 2022 at 16:38, John Snow <jsnow@redhat.com> wrote:
> > >
> > > > On Thu, Feb 3, 2022, 11:20 AM Peter Maydell <peter.maydell@linaro.org> wrote:
> > > >> Summary of Failures:
> > > >>
> > > >> 1/1 qemu:block / qemu-iotests qcow2 ERROR 243.14s exit status 1
> >
> > I'm not too familiar with this new test runner, yet. (Is this error
> > even anything to do with the python lib? I guess I can't rule it
> > out...)
> > I just got a clean run of 'make vm-build-netbsd', so I'm using that
> > output as reference and making some guesses.
>
> Rerunning on the netbsd VM with Paolo's "revert the iotests
> conversion" patch, here's the output from a failing run, where
> iotest 041 failed:
>
> TEST iotest-qcow2: 041 [fail]
> QEMU --
> "/home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/../../qemu-system-aarch64"
> -nodefaults -display none -accel qtest -machine virt
> QEMU_IMG --
> "/home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/../../qemu-img"
> QEMU_IO --
> "/home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/../../qemu-io"
> --cache writeback --aio threads -f qcow2
> QEMU_NBD --
> "/home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/../../qemu-nbd"
> IMGFMT -- qcow2
> IMGPROTO -- file
> PLATFORM -- NetBSD/amd64 localhost 9.2
> TEST_DIR -- /home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/scratch
> SOCK_DIR -- /tmp/tmp6fiu68sr
> GDB_OPTIONS --
> VALGRIND_QEMU --
> PRINT_QEMU_OUTPUT --
>
> --- /home/qemu/qemu-test.Kywnb7/src/tests/qemu-iotests/041.out
> +++ 041.out.bad
> @@ -1,5 +1,44 @@
> -...........................................................................................................
> +........................................ERROR:qemu.aqmp.qmp_client.qemu-14411:Failed
> to establish connection: concurrent.futures._base.CancelledError
> +E..................................................................
> +======================================================================
> +ERROR: test_mirror_to_self (__main__.TestSingleBlockdev)
> +----------------------------------------------------------------------
> +Traceback (most recent call last):
> + File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/machine/machine.py",
> line 428, in launch
> + self._launch()
> + File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/machine/machine.py",
> line 467, in _launch
> + self._post_launch()
> + File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/machine/qtest.py",
> line 147, in _post_launch
> + super()._post_launch()
> + File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/machine/machine.py",
> line 369, in _post_launch
> + self._qmp.accept(self._qmp_timer)
> + File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/aqmp/legacy.py",
> line 95, in accept
> + timeout
> + File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/aqmp/legacy.py",
> line 68, in _sync
> + asyncio.wait_for(future, timeout=timeout)
> + File "/usr/pkg/lib/python3.7/asyncio/base_events.py", line 587, in
> run_until_complete
> + return future.result()
> + File "/usr/pkg/lib/python3.7/asyncio/tasks.py", line 449, in wait_for
> + raise futures.TimeoutError()
> +concurrent.futures._base.TimeoutError
> +
> +The above exception was the direct cause of the following exception:
> +
> +Traceback (most recent call last):
> + File "/home/qemu/qemu-test.Kywnb7/src/tests/qemu-iotests/041", line
> 233, in setUp
> + TestSingleDrive.setUp(self)
> + File "/home/qemu/qemu-test.Kywnb7/src/tests/qemu-iotests/041", line
> 54, in setUp
> + self.vm.launch()
> + File "/home/qemu/qemu-test.Kywnb7/src/python/qemu/machine/machine.py",
> line 445, in launch
> + ) from exc
> +qemu.machine.machine.VMLaunchFailure: TimeoutError
> + Exit code: 1
> + Command:
> /home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/../../qemu-system-aarch64
> -display none -vga none -chardev
> socket,id=mon,path=/tmp/tmp6fiu68sr/qemu-14411-monitor.sock -mon
> chardev=mon,mode=control -qtest
> unix:path=/tmp/tmp6fiu68sr/qemu-14411-qtest.sock -accel qtest
> -nodefaults -display none -accel qtest -machine virt -drive
> if=virtio,id=drive0,file=/home/qemu/qemu-test.Kywnb7/build/tests/qemu-iotests/scratch/test.img,format=qcow2,cache=writeback,aio=threads,node-name=top,backing.node-name=base
> + Output: qemu-system-aarch64: -chardev
> socket,id=mon,path=/tmp/tmp6fiu68sr/qemu-14411-monitor.sock: Failed to
> connect to '/tmp/tmp6fiu68sr/qemu-14411-monitor.sock': Connection
> refused
> +
> +
> +
> ----------------------------------------------------------------------
> Ran 107 tests
>
> -OK
> +FAILED (errors=1)
>
>
> thanks
> -- PMM
>
Just so I don't leave this thread hanging, I filed a GitLab issue and
I'm working on it, but this one isn't as quick to solve as the other.
https://gitlab.com/qemu-project/qemu/-/issues/874
--js
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PULL 0/4] Python patches
2022-02-15 17:45 ` John Snow
@ 2022-02-15 18:00 ` Peter Maydell
2022-02-15 19:02 ` John Snow
0 siblings, 1 reply; 16+ messages in thread
From: Peter Maydell @ 2022-02-15 18:00 UTC (permalink / raw)
To: John Snow
Cc: Eduardo Habkost, Kevin Wolf, Qemu-block, qemu-devel,
Markus Armbruster, Hanna Reitz, Cleber Rosa
On Tue, 15 Feb 2022 at 17:46, John Snow <jsnow@redhat.com> wrote:
> Just so I don't leave this thread hanging, I filed a GitLab issue and
> I'm working on it, but this one isn't as quick to solve as the other.
>
> https://gitlab.com/qemu-project/qemu/-/issues/874
Is there anything particular to NetBSD that means it happens
more often there, or is it just random luck that we hit
the race there and haven't seen it elsewhere ?
-- PMM
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PULL 0/4] Python patches
2022-02-15 18:00 ` Peter Maydell
@ 2022-02-15 19:02 ` John Snow
0 siblings, 0 replies; 16+ messages in thread
From: John Snow @ 2022-02-15 19:02 UTC (permalink / raw)
To: Peter Maydell
Cc: Eduardo Habkost, Kevin Wolf, Qemu-block, qemu-devel,
Markus Armbruster, Hanna Reitz, Cleber Rosa
On Tue, Feb 15, 2022 at 1:01 PM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Tue, 15 Feb 2022 at 17:46, John Snow <jsnow@redhat.com> wrote:
> > Just so I don't leave this thread hanging, I filed a GitLab issue and
> > I'm working on it, but this one isn't as quick to solve as the other.
> >
> > https://gitlab.com/qemu-project/qemu/-/issues/874
>
> Is there anything particular to NetBSD that means it happens
> more often there, or is it just random luck that we hit
> the race there and haven't seen it elsewhere ?
>
> -- PMM
Complete random luck, something jostled loose by the scheduler.
I need to change the interface in the async library entirely to make
the process more granular -- We don't need the granularity in a truly
async mode, but the sync wrapper that allows the existing iotests
corpus to use the library in a synchronous manner *requires* a more
granular connection API, so I have to write one. It's in progress, it
just might be a few more days; verifying and testing the error
pathways has been slow work.
(In detail: python's asyncio.create_unix_server() call combines bind()
+ listen() + accept() into a single discrete step. A synchronous
client, though, needs to have a reprieve from all of those blocking
steps to launch the QEMU process after listen() but before accept() so
it can launch the QEMU process. I was able to pull the bind() step
out, but the async listen() + accept() steps the way I initially wrote
it are inseparable. Live and learn.)
In the meantime, there *IS* a way to use the old library, but I don't
think the environment variable in question is routed down into the VM
tests. I can look at (as a very quick fix) amending the VM launcher to
pass along that environment variable if it sees it set in the host
environment -- that should get you on the old, tried-and-true library
when you want it, and the test should pass.
--js
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PULL 0/4] Python patches
2022-01-10 23:25 John Snow
@ 2022-01-12 9:20 ` Peter Maydell
0 siblings, 0 replies; 16+ messages in thread
From: Peter Maydell @ 2022-01-12 9:20 UTC (permalink / raw)
To: John Snow
Cc: Kevin Wolf, Eduardo Habkost, Thomas Huth,
Vladimir Sementsov-Ogievskiy, Daniel Berrange, Eduardo Habkost,
qemu-block, Markus Armbruster, Wainer dos Santos Moschetta,
qemu-devel, Willian Rampazzo, Hanna Reitz, Cleber Rosa,
Alex Bennée, Philippe Mathieu-Daudé
On Mon, 10 Jan 2022 at 23:25, John Snow <jsnow@redhat.com> wrote:
>
> The following changes since commit de3f5223fa4cf8bfc5e3fe1fd495ddf468edcdf7:
>
> Merge remote-tracking branch 'remotes/vivier/tags/m68k-for-7.0-pull-request' into staging (2022-01-10 14:43:03 +0000)
>
> are available in the Git repository at:
>
> https://gitlab.com/jsnow/qemu.git tags/python-pull-request
>
> for you to fetch changes up to 9ebfc5a583d8aa94bf1bc37c1f71559187fd809c:
>
> simplebench: Fix Python syntax error (reported by LGTM) (2022-01-10 18:23:10 -0500)
>
> ----------------------------------------------------------------
> Python pull request
>
> Fixes for the tests that broke during vacation, plus a simple syntax fix
> for a python script.
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/7.0
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 16+ messages in thread
* [PULL 0/4] Python patches
@ 2022-01-10 23:25 John Snow
2022-01-12 9:20 ` Peter Maydell
0 siblings, 1 reply; 16+ messages in thread
From: John Snow @ 2022-01-10 23:25 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, Peter Maydell, Thomas Huth,
Vladimir Sementsov-Ogievskiy, Daniel Berrange, Eduardo Habkost,
qemu-block, John Snow, Markus Armbruster,
Wainer dos Santos Moschetta, Philippe Mathieu-Daudé,
Willian Rampazzo, Eduardo Habkost, Hanna Reitz, Cleber Rosa,
Alex Bennée
The following changes since commit de3f5223fa4cf8bfc5e3fe1fd495ddf468edcdf7:
Merge remote-tracking branch 'remotes/vivier/tags/m68k-for-7.0-pull-request' into staging (2022-01-10 14:43:03 +0000)
are available in the Git repository at:
https://gitlab.com/jsnow/qemu.git tags/python-pull-request
for you to fetch changes up to 9ebfc5a583d8aa94bf1bc37c1f71559187fd809c:
simplebench: Fix Python syntax error (reported by LGTM) (2022-01-10 18:23:10 -0500)
----------------------------------------------------------------
Python pull request
Fixes for the tests that broke during vacation, plus a simple syntax fix
for a python script.
----------------------------------------------------------------
John Snow (3):
python/aqmp: use absolute import statement
Python/aqmp: fix type definitions for mypy 0.920
python: update type hints for mypy 0.930
Stefan Weil (1):
simplebench: Fix Python syntax error (reported by LGTM)
python/qemu/aqmp/aqmp_tui.py | 3 ++-
python/qemu/aqmp/protocol.py | 5 +++--
python/qemu/qmp/qom_common.py | 6 +-----
scripts/simplebench/bench-example.py | 2 +-
4 files changed, 7 insertions(+), 9 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2022-02-15 19:04 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-03 1:59 [PULL 0/4] Python patches John Snow
2022-02-03 1:59 ` [PULL 1/4] python/aqmp: Fix negotiation with pre-"oob" QEMU John Snow
2022-02-03 1:59 ` [PULL 2/4] python/machine: raise VMLaunchFailure exception from launch() John Snow
2022-02-03 1:59 ` [PULL 3/4] python: upgrade mypy to 0.780 John Snow
2022-02-03 1:59 ` [PULL 4/4] python/aqmp: add socket bind step to legacy.py John Snow
2022-02-03 16:20 ` [PULL 0/4] Python patches Peter Maydell
2022-02-03 16:38 ` John Snow
2022-02-03 16:51 ` Peter Maydell
2022-02-03 23:22 ` John Snow
2022-02-08 14:40 ` Peter Maydell
2022-02-15 17:45 ` John Snow
2022-02-15 18:00 ` Peter Maydell
2022-02-15 19:02 ` John Snow
2022-02-04 17:03 ` Peter Maydell
-- strict thread matches above, loose matches on Subject: below --
2022-01-10 23:25 John Snow
2022-01-12 9:20 ` Peter Maydell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.