* [PATCH v3 01/19] python/pipenv: Update Pipfile.lock
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 02/19] python/qmp: Fix type of SocketAddrT John Snow
` (18 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
In a previous commit, I added tox to the development requirements of the
Python library. I never bothered to add them to the Pipfile, because
they aren't needed there. Here, I sync it anyway in its own commit so
that when we add new packages later that the diffstats will not
confusingly appear to pull in lots of extra packages.
Ideally I could tell Pipenv simply not to install these, but it doesn't
seem to support that, exactly. The alternative is removing Tox from the
development requires, which I'd rather not do.
The other alternative is re-specifying all of the dependencies of
setup.cfg in the Pipfile, which I'd also rather not do.
Picking what feels least-worst here.
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/Pipfile.lock | 91 +++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 88 insertions(+), 3 deletions(-)
diff --git a/python/Pipfile.lock b/python/Pipfile.lock
index 6e344f5fadf..f2a3f91d0fa 100644
--- a/python/Pipfile.lock
+++ b/python/Pipfile.lock
@@ -22,6 +22,13 @@
}
},
"develop": {
+ "appdirs": {
+ "hashes": [
+ "sha256:7d5d0167b2b1ba821647616af46a749d1c653740dd0d2415100fe26e27afdf41",
+ "sha256:a841dacd6b99318a741b166adb07e19ee71a274450e68237b4650ca1055ab128"
+ ],
+ "version": "==1.4.4"
+ },
"astroid": {
"hashes": [
"sha256:4db03ab5fc3340cf619dbc25e42c2cc3755154ce6009469766d7143d1fc2ee4e",
@@ -38,6 +45,20 @@
"markers": "python_version >= '3.6'",
"version": "==88.1"
},
+ "distlib": {
+ "hashes": [
+ "sha256:106fef6dc37dd8c0e2c0a60d3fca3e77460a48907f335fa28420463a6f799736",
+ "sha256:23e223426b28491b1ced97dc3bbe183027419dfc7982b4fa2f05d5f3ff10711c"
+ ],
+ "version": "==0.3.2"
+ },
+ "filelock": {
+ "hashes": [
+ "sha256:18d82244ee114f543149c66a6e0c14e9c4f8a1044b5cdaadd0f82159d6a6ff59",
+ "sha256:929b7d63ec5b7d6b71b0fa5ac14e030b3f70b75747cef1b10da9b879fef15836"
+ ],
+ "version": "==3.0.12"
+ },
"flake8": {
"hashes": [
"sha256:07528381786f2a6237b061f6e96610a4167b226cb926e2aa2b6b1d78057c576b",
@@ -54,6 +75,14 @@
"markers": "python_version < '3.8'",
"version": "==4.0.1"
},
+ "importlib-resources": {
+ "hashes": [
+ "sha256:54161657e8ffc76596c4ede7080ca68cb02962a2e074a2586b695a93a925d36e",
+ "sha256:e962bff7440364183203d179d7ae9ad90cb1f2b74dcb84300e88ecc42dca3351"
+ ],
+ "markers": "python_version < '3.7'",
+ "version": "==5.1.4"
+ },
"isort": {
"hashes": [
"sha256:0a943902919f65c5684ac4e0154b1ad4fac6dcaa5d9f3426b732f1c8b5419be6",
@@ -132,6 +161,30 @@
],
"version": "==0.4.3"
},
+ "packaging": {
+ "hashes": [
+ "sha256:5b327ac1320dc863dca72f4514ecc086f31186744b84a230374cc1fd776feae5",
+ "sha256:67714da7f7bc052e064859c05c595155bd1ee9f69f76557e21f051443c20947a"
+ ],
+ "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "version": "==20.9"
+ },
+ "pluggy": {
+ "hashes": [
+ "sha256:15b2acde666561e1298d71b523007ed7364de07029219b604cf808bfa1c765b0",
+ "sha256:966c145cd83c96502c3c3868f50408687b38434af77734af1e9ca461a4081d2d"
+ ],
+ "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "version": "==0.13.1"
+ },
+ "py": {
+ "hashes": [
+ "sha256:21b81bda15b66ef5e1a777a21c4dcd9c20ad3efd0b3f817e7a809035269e1bd3",
+ "sha256:3b80836aa6d1feeaa108e046da6423ab8f6ceda6468545ae8d02d9d58d18818a"
+ ],
+ "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "version": "==1.10.0"
+ },
"pycodestyle": {
"hashes": [
"sha256:514f76d918fcc0b55c6680472f0a37970994e07bbb80725808c17089be302068",
@@ -156,18 +209,42 @@
"markers": "python_version ~= '3.6'",
"version": "==2.8.2"
},
+ "pyparsing": {
+ "hashes": [
+ "sha256:c203ec8783bf771a155b207279b9bccb8dea02d8f0c9e5f8ead507bc3246ecc1",
+ "sha256:ef9d7589ef3c200abe66653d3f1ab1033c3c419ae9b9bdb1240a85b024efc88b"
+ ],
+ "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "version": "==2.4.7"
+ },
"qemu": {
"editable": true,
"path": "."
},
+ "six": {
+ "hashes": [
+ "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926",
+ "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254"
+ ],
+ "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "version": "==1.16.0"
+ },
"toml": {
"hashes": [
"sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b",
"sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f"
],
- "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2'",
+ "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'",
"version": "==0.10.2"
},
+ "tox": {
+ "hashes": [
+ "sha256:307a81ddb82bd463971a273f33e9533a24ed22185f27db8ce3386bff27d324e3",
+ "sha256:b0b5818049a1c1997599d42012a637a33f24c62ab8187223fdd318fa8522637b"
+ ],
+ "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
+ "version": "==3.23.1"
+ },
"typed-ast": {
"hashes": [
"sha256:01ae5f73431d21eead5015997ab41afa53aa1fbe252f9da060be5dad2c730ace",
@@ -201,7 +278,7 @@
"sha256:f8afcf15cc511ada719a88e013cec87c11aff7b91f019295eb4530f96fe5ef2f",
"sha256:fb1bbeac803adea29cedd70781399c99138358c26d05fcbd23c13016b7f5ec65"
],
- "markers": "implementation_name == 'cpython' and python_version < '3.8'",
+ "markers": "python_version < '3.8' and implementation_name == 'cpython'",
"version": "==1.4.3"
},
"typing-extensions": {
@@ -213,6 +290,14 @@
"markers": "python_version < '3.8'",
"version": "==3.10.0.0"
},
+ "virtualenv": {
+ "hashes": [
+ "sha256:14fdf849f80dbb29a4eb6caa9875d476ee2a5cf76a5f5415fa2f1606010ab467",
+ "sha256:2b0126166ea7c9c3661f5b8e06773d28f83322de7a3ff7d06f0aed18c9de6a76"
+ ],
+ "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'",
+ "version": "==20.4.7"
+ },
"wrapt": {
"hashes": [
"sha256:b62ffa81fb85f4332a4f609cab4ac40709470da05643a082ec1eb88e6d9b97d7"
@@ -224,7 +309,7 @@
"sha256:3607921face881ba3e026887d8150cca609d517579abe052ac81fc5aeffdbd76",
"sha256:51cb66cc54621609dd593d1787f286ee42a5c0adbb4b29abea5a63edc3e03098"
],
- "markers": "python_version >= '3.6'",
+ "markers": "python_version < '3.10'",
"version": "==3.4.1"
}
}
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 02/19] python/qmp: Fix type of SocketAddrT
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
2021-06-03 0:37 ` [PATCH v3 01/19] python/pipenv: Update Pipfile.lock John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 7:07 ` Philippe Mathieu-Daudé
2021-06-03 0:37 ` [PATCH v3 03/19] python/qmp: add parse_address classmethod John Snow
` (17 subsequent siblings)
19 siblings, 1 reply; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
In porting the qom tools, qmp-shell, etc; it becomes evident that this
type is wrong.
This is an integer, not a string. We didn't catch this before because
none of QEMUMonitorProtocol's *users* happen to be checked, and the
internal logic of this class is otherwise self-consistent. Additionally,
mypy was not introspecting into the socket() interface to realize we
were passing a bad type for AF_INET. Fixed now.
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/qemu/qmp/__init__.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/python/qemu/qmp/__init__.py b/python/qemu/qmp/__init__.py
index 9606248a3d2..5fb970f8a80 100644
--- a/python/qemu/qmp/__init__.py
+++ b/python/qemu/qmp/__init__.py
@@ -44,7 +44,7 @@
QMPMessage = Dict[str, Any]
QMPReturnValue = Dict[str, Any]
-InternetAddrT = Tuple[str, str]
+InternetAddrT = Tuple[str, int]
UnixAddrT = str
SocketAddrT = Union[InternetAddrT, UnixAddrT]
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v3 02/19] python/qmp: Fix type of SocketAddrT
2021-06-03 0:37 ` [PATCH v3 02/19] python/qmp: Fix type of SocketAddrT John Snow
@ 2021-06-03 7:07 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 25+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-06-03 7:07 UTC (permalink / raw)
To: John Snow, qemu-devel
Cc: Cleber Rosa, Markus Armbruster, Eduardo Habkost, Niteesh G . S .
On 6/3/21 2:37 AM, John Snow wrote:
> In porting the qom tools, qmp-shell, etc; it becomes evident that this
> type is wrong.
>
> This is an integer, not a string. We didn't catch this before because
> none of QEMUMonitorProtocol's *users* happen to be checked, and the
> internal logic of this class is otherwise self-consistent. Additionally,
> mypy was not introspecting into the socket() interface to realize we
> were passing a bad type for AF_INET. Fixed now.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> python/qemu/qmp/__init__.py | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v3 03/19] python/qmp: add parse_address classmethod
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
2021-06-03 0:37 ` [PATCH v3 01/19] python/pipenv: Update Pipfile.lock John Snow
2021-06-03 0:37 ` [PATCH v3 02/19] python/qmp: Fix type of SocketAddrT John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 04/19] python/qmp: Add qom script rewrites John Snow
` (16 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
This takes the place of qmp-shell's __get_address function. It also
allows other utilities to share the same parser and syntax for
specifying QMP locations.
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/qemu/qmp/__init__.py | 26 ++++++++++++++++++++++++++
scripts/qmp/qmp-shell | 21 ++-------------------
2 files changed, 28 insertions(+), 19 deletions(-)
diff --git a/python/qemu/qmp/__init__.py b/python/qemu/qmp/__init__.py
index 5fb970f8a80..822c793c320 100644
--- a/python/qemu/qmp/__init__.py
+++ b/python/qemu/qmp/__init__.py
@@ -92,6 +92,12 @@ def __init__(self, reply: QMPMessage):
self.reply = reply
+class QMPBadPortError(QMPError):
+ """
+ Unable to parse socket address: Port was non-numerical.
+ """
+
+
class QEMUMonitorProtocol:
"""
Provide an API to connect to QEMU via QEMU Monitor Protocol (QMP) and then
@@ -219,6 +225,26 @@ def __exit__(self,
# Implement context manager exit function.
self.close()
+ @classmethod
+ def parse_address(cls, address: str) -> SocketAddrT:
+ """
+ Parse a string into a QMP address.
+
+ Figure out if the argument is in the port:host form.
+ If it's not, it's probably a file path.
+ """
+ components = address.split(':')
+ if len(components) == 2:
+ try:
+ port = int(components[1])
+ except ValueError:
+ msg = f"Bad port: '{components[1]}' in '{address}'."
+ raise QMPBadPortError(msg) from None
+ return (components[0], port)
+
+ # Treat as filepath.
+ return address
+
def connect(self, negotiate: bool = True) -> Optional[QMPMessage]:
"""
Connect to the QMP Monitor and perform capabilities negotiation.
diff --git a/scripts/qmp/qmp-shell b/scripts/qmp/qmp-shell
index b4d06096abd..d5ae8a9b212 100755
--- a/scripts/qmp/qmp-shell
+++ b/scripts/qmp/qmp-shell
@@ -89,8 +89,6 @@ class QMPCompleter(list):
class QMPShellError(Exception):
pass
-class QMPShellBadPort(QMPShellError):
- pass
class FuzzyJSON(ast.NodeTransformer):
'''This extension of ast.NodeTransformer filters literal "true/false/null"
@@ -109,7 +107,7 @@ class FuzzyJSON(ast.NodeTransformer):
# _execute_cmd()). Let's design a better one.
class QMPShell(qmp.QEMUMonitorProtocol):
def __init__(self, address, pretty=False):
- super(QMPShell, self).__init__(self.__get_address(address))
+ super(QMPShell, self).__init__(self.parse_address(address))
self._greeting = None
self._completer = None
self._pretty = pretty
@@ -118,21 +116,6 @@ class QMPShell(qmp.QEMUMonitorProtocol):
self._histfile = os.path.join(os.path.expanduser('~'),
'.qmp-shell_history')
- def __get_address(self, arg):
- """
- Figure out if the argument is in the port:host form, if it's not it's
- probably a file path.
- """
- addr = arg.split(':')
- if len(addr) == 2:
- try:
- port = int(addr[1])
- except ValueError:
- raise QMPShellBadPort
- return ( addr[0], port )
- # socket path
- return arg
-
def _fill_completion(self):
cmds = self.cmd('query-commands')
if 'error' in cmds:
@@ -437,7 +420,7 @@ def main():
if qemu is None:
fail_cmdline()
- except QMPShellBadPort:
+ except qmp.QMPBadPortError:
die('bad port number in command-line')
try:
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 04/19] python/qmp: Add qom script rewrites
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (2 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 03/19] python/qmp: add parse_address classmethod John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 05/19] python/qmp: add qom script entry points John Snow
` (15 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
Inspired by qom-set, qom-get, qom-tree and qom-list; combine all four of
those scripts into a single script.
A later addition of qom-fuse as an 'extension' necessitates that some
common features are split out and shared between them.
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/qemu/qmp/qom.py | 262 ++++++++++++++++++++++++++++++++++
python/qemu/qmp/qom_common.py | 178 +++++++++++++++++++++++
2 files changed, 440 insertions(+)
create mode 100644 python/qemu/qmp/qom.py
create mode 100644 python/qemu/qmp/qom_common.py
diff --git a/python/qemu/qmp/qom.py b/python/qemu/qmp/qom.py
new file mode 100644
index 00000000000..7fe1448b5d9
--- /dev/null
+++ b/python/qemu/qmp/qom.py
@@ -0,0 +1,262 @@
+"""
+QEMU Object Model testing tools.
+
+usage: qom [-h] {set,get,list,tree} ...
+
+Query and manipulate QOM data
+
+optional arguments:
+ -h, --help show this help message and exit
+
+QOM commands:
+ {set,get,list,tree}
+ set Set a QOM property value
+ get Get a QOM property value
+ list List QOM properties at a given path
+ tree Show QOM tree from a given path
+"""
+##
+# Copyright John Snow 2020, for Red Hat, Inc.
+# Copyright IBM, Corp. 2011
+#
+# Authors:
+# John Snow <jsnow@redhat.com>
+# Anthony Liguori <aliguori@amazon.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# Based on ./scripts/qmp/qom-[set|get|tree|list]
+##
+
+import argparse
+
+from . import QMPResponseError
+from .qom_common import QOMCommand
+
+
+class QOMSet(QOMCommand):
+ """
+ QOM Command - Set a property to a given value.
+
+ usage: qom-set [-h] [--socket SOCKET] <path>.<property> <value>
+
+ Set a QOM property value
+
+ positional arguments:
+ <path>.<property> QOM path and property, separated by a period '.'
+ <value> new QOM property value
+
+ optional arguments:
+ -h, --help show this help message and exit
+ --socket SOCKET, -s SOCKET
+ QMP socket path or address (addr:port). May also be
+ set via QMP_SOCKET environment variable.
+ """
+ name = 'set'
+ help = 'Set a QOM property value'
+
+ @classmethod
+ def configure_parser(cls, parser: argparse.ArgumentParser) -> None:
+ super().configure_parser(parser)
+ cls.add_path_prop_arg(parser)
+ parser.add_argument(
+ 'value',
+ metavar='<value>',
+ action='store',
+ help='new QOM property value'
+ )
+
+ def __init__(self, args: argparse.Namespace):
+ super().__init__(args)
+ self.path, self.prop = args.path_prop.rsplit('.', 1)
+ self.value = args.value
+
+ def run(self) -> int:
+ rsp = self.qmp.command(
+ 'qom-set',
+ path=self.path,
+ property=self.prop,
+ value=self.value
+ )
+ print(rsp)
+ return 0
+
+
+class QOMGet(QOMCommand):
+ """
+ QOM Command - Get a property's current value.
+
+ usage: qom-get [-h] [--socket SOCKET] <path>.<property>
+
+ Get a QOM property value
+
+ positional arguments:
+ <path>.<property> QOM path and property, separated by a period '.'
+
+ optional arguments:
+ -h, --help show this help message and exit
+ --socket SOCKET, -s SOCKET
+ QMP socket path or address (addr:port). May also be
+ set via QMP_SOCKET environment variable.
+ """
+ name = 'get'
+ help = 'Get a QOM property value'
+
+ @classmethod
+ def configure_parser(cls, parser: argparse.ArgumentParser) -> None:
+ super().configure_parser(parser)
+ cls.add_path_prop_arg(parser)
+
+ def __init__(self, args: argparse.Namespace):
+ super().__init__(args)
+ try:
+ tmp = args.path_prop.rsplit('.', 1)
+ except ValueError as err:
+ raise ValueError('Invalid format for <path>.<property>') from err
+ self.path = tmp[0]
+ self.prop = tmp[1]
+
+ def run(self) -> int:
+ rsp = self.qmp.command(
+ 'qom-get',
+ path=self.path,
+ property=self.prop
+ )
+ if isinstance(rsp, dict):
+ for key, value in rsp.items():
+ print(f"{key}: {value}")
+ else:
+ print(rsp)
+ return 0
+
+
+class QOMList(QOMCommand):
+ """
+ QOM Command - List the properties at a given path.
+
+ usage: qom-list [-h] [--socket SOCKET] <path>
+
+ List QOM properties at a given path
+
+ positional arguments:
+ <path> QOM path
+
+ optional arguments:
+ -h, --help show this help message and exit
+ --socket SOCKET, -s SOCKET
+ QMP socket path or address (addr:port). May also be
+ set via QMP_SOCKET environment variable.
+ """
+ name = 'list'
+ help = 'List QOM properties at a given path'
+
+ @classmethod
+ def configure_parser(cls, parser: argparse.ArgumentParser) -> None:
+ super().configure_parser(parser)
+ parser.add_argument(
+ 'path',
+ metavar='<path>',
+ action='store',
+ help='QOM path',
+ )
+
+ def __init__(self, args: argparse.Namespace):
+ super().__init__(args)
+ self.path = args.path
+
+ def run(self) -> int:
+ rsp = self.qom_list(self.path)
+ for item in rsp:
+ if item.child:
+ print(f"{item.name}/")
+ elif item.link:
+ print(f"@{item.name}/")
+ else:
+ print(item.name)
+ return 0
+
+
+class QOMTree(QOMCommand):
+ """
+ QOM Command - Show the full tree below a given path.
+
+ usage: qom-tree [-h] [--socket SOCKET] [<path>]
+
+ Show QOM tree from a given path
+
+ positional arguments:
+ <path> QOM path
+
+ optional arguments:
+ -h, --help show this help message and exit
+ --socket SOCKET, -s SOCKET
+ QMP socket path or address (addr:port). May also be
+ set via QMP_SOCKET environment variable.
+ """
+ name = 'tree'
+ help = 'Show QOM tree from a given path'
+
+ @classmethod
+ def configure_parser(cls, parser: argparse.ArgumentParser) -> None:
+ super().configure_parser(parser)
+ parser.add_argument(
+ 'path',
+ metavar='<path>',
+ action='store',
+ help='QOM path',
+ nargs='?',
+ default='/'
+ )
+
+ def __init__(self, args: argparse.Namespace):
+ super().__init__(args)
+ self.path = args.path
+
+ def _list_node(self, path: str) -> None:
+ print(path)
+ items = self.qom_list(path)
+ for item in items:
+ if item.child:
+ continue
+ try:
+ rsp = self.qmp.command('qom-get', path=path,
+ property=item.name)
+ print(f" {item.name}: {rsp} ({item.type})")
+ except QMPResponseError as err:
+ print(f" {item.name}: <EXCEPTION: {err!s}> ({item.type})")
+ print('')
+ for item in items:
+ if not item.child:
+ continue
+ if path == '/':
+ path = ''
+ self._list_node(f"{path}/{item.name}")
+
+ def run(self) -> int:
+ self._list_node(self.path)
+ return 0
+
+
+def main() -> int:
+ """QOM script main entry point."""
+ parser = argparse.ArgumentParser(
+ description='Query and manipulate QOM data'
+ )
+ subparsers = parser.add_subparsers(
+ title='QOM commands',
+ dest='command'
+ )
+
+ for command in QOMCommand.__subclasses__():
+ command.register(subparsers)
+
+ args = parser.parse_args()
+
+ if args.command is None:
+ parser.error('Command not specified.')
+ return 1
+
+ cmd_class = args.cmd_class
+ assert isinstance(cmd_class, type(QOMCommand))
+ return cmd_class.command_runner(args)
diff --git a/python/qemu/qmp/qom_common.py b/python/qemu/qmp/qom_common.py
new file mode 100644
index 00000000000..f82b16772df
--- /dev/null
+++ b/python/qemu/qmp/qom_common.py
@@ -0,0 +1,178 @@
+"""
+QOM Command abstractions.
+"""
+##
+# Copyright John Snow 2020, for Red Hat, Inc.
+# Copyright IBM, Corp. 2011
+#
+# Authors:
+# John Snow <jsnow@redhat.com>
+# Anthony Liguori <aliguori@amazon.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+#
+# Based on ./scripts/qmp/qom-[set|get|tree|list]
+##
+
+import argparse
+import os
+import sys
+from typing import (
+ Any,
+ Dict,
+ List,
+ Optional,
+ Type,
+ TypeVar,
+)
+
+from . import QEMUMonitorProtocol, QMPError
+
+
+# The following is needed only for a type alias.
+Subparsers = argparse._SubParsersAction # pylint: disable=protected-access
+
+
+class ObjectPropertyInfo:
+ """
+ Represents the return type from e.g. qom-list.
+ """
+ def __init__(self, name: str, type_: str,
+ description: Optional[str] = None,
+ default_value: Optional[object] = None):
+ self.name = name
+ self.type = type_
+ self.description = description
+ self.default_value = default_value
+
+ @classmethod
+ def make(cls, value: Dict[str, Any]) -> 'ObjectPropertyInfo':
+ """
+ Build an ObjectPropertyInfo from a Dict with an unknown shape.
+ """
+ assert value.keys() >= {'name', 'type'}
+ assert value.keys() <= {'name', 'type', 'description', 'default-value'}
+ return cls(value['name'], value['type'],
+ value.get('description'),
+ value.get('default-value'))
+
+ @property
+ def child(self) -> bool:
+ """Is this property a child property?"""
+ return self.type.startswith('child<')
+
+ @property
+ def link(self) -> bool:
+ """Is this property a link property?"""
+ return self.type.startswith('link<')
+
+
+CommandT = TypeVar('CommandT', bound='QOMCommand')
+
+
+class QOMCommand:
+ """
+ Represents a QOM sub-command.
+
+ :param args: Parsed arguments, as returned from parser.parse_args.
+ """
+ name: str
+ help: str
+
+ def __init__(self, args: argparse.Namespace):
+ if args.socket is None:
+ raise QMPError("No QMP socket path or address given")
+ self.qmp = QEMUMonitorProtocol(
+ QEMUMonitorProtocol.parse_address(args.socket)
+ )
+ self.qmp.connect()
+
+ @classmethod
+ def register(cls, subparsers: Subparsers) -> None:
+ """
+ Register this command with the argument parser.
+
+ :param subparsers: argparse subparsers object, from "add_subparsers".
+ """
+ subparser = subparsers.add_parser(cls.name, help=cls.help,
+ description=cls.help)
+ cls.configure_parser(subparser)
+
+ @classmethod
+ def configure_parser(cls, parser: argparse.ArgumentParser) -> None:
+ """
+ Configure a parser with this command's arguments.
+
+ :param parser: argparse parser or subparser object.
+ """
+ default_path = os.environ.get('QMP_SOCKET')
+ parser.add_argument(
+ '--socket', '-s',
+ dest='socket',
+ action='store',
+ help='QMP socket path or address (addr:port).'
+ ' May also be set via QMP_SOCKET environment variable.',
+ default=default_path
+ )
+ parser.set_defaults(cmd_class=cls)
+
+ @classmethod
+ def add_path_prop_arg(cls, parser: argparse.ArgumentParser) -> None:
+ """
+ Add the <path>.<proptery> positional argument to this command.
+
+ :param parser: The parser to add the argument to.
+ """
+ parser.add_argument(
+ 'path_prop',
+ metavar='<path>.<property>',
+ action='store',
+ help="QOM path and property, separated by a period '.'"
+ )
+
+ def run(self) -> int:
+ """
+ Run this command.
+
+ :return: 0 on success, 1 otherwise.
+ """
+ raise NotImplementedError
+
+ def qom_list(self, path: str) -> List[ObjectPropertyInfo]:
+ """
+ :return: a strongly typed list from the 'qom-list' command.
+ """
+ rsp = self.qmp.command('qom-list', path=path)
+ # qom-list returns List[ObjectPropertyInfo]
+ assert isinstance(rsp, list)
+ return [ObjectPropertyInfo.make(x) for x in rsp]
+
+ @classmethod
+ def command_runner(
+ cls: Type[CommandT],
+ args: argparse.Namespace
+ ) -> int:
+ """
+ Run a fully-parsed subcommand, with error-handling for the CLI.
+
+ :return: The return code from `.run()`.
+ """
+ try:
+ cmd = cls(args)
+ return cmd.run()
+ except QMPError as err:
+ print(f"{type(err).__name__}: {err!s}", file=sys.stderr)
+ return -1
+
+ @classmethod
+ def entry_point(cls) -> int:
+ """
+ Build this command's parser, parse arguments, and run the command.
+
+ :return: `run`'s return code.
+ """
+ parser = argparse.ArgumentParser(description=cls.help)
+ cls.configure_parser(parser)
+ args = parser.parse_args()
+ return cls.command_runner(args)
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 05/19] python/qmp: add qom script entry points
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (3 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 04/19] python/qmp: Add qom script rewrites John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 06/19] scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/ John Snow
` (14 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, John Snow, Markus Armbruster, Niteesh G . S .,
Cleber Rosa, Philippe Mathieu-Daudé
Add the 'qom', 'qom-set', 'qom-get', 'qom-list', and 'qom-tree' scripts
to the qemu.qmp package. When you install this package, these scripts
will become available on your command line.
(e.g. when inside of a venv, `cd python && pip install .` will add
'qom', 'qom-set', etc to your $PATH.)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
python/setup.cfg | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/python/setup.cfg b/python/setup.cfg
index 0fcdec6f322..a19029d5388 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -37,6 +37,14 @@ devel =
pylint >= 2.8.0
tox >= 3.18.0
+[options.entry_points]
+console_scripts =
+ qom = qemu.qmp.qom:main
+ qom-set = qemu.qmp.qom:QOMSet.entry_point
+ qom-get = qemu.qmp.qom:QOMGet.entry_point
+ qom-list = qemu.qmp.qom:QOMList.entry_point
+ qom-tree = qemu.qmp.qom:QOMTree.entry_point
+
[flake8]
extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's
exclude = __pycache__,
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 06/19] scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (4 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 05/19] python/qmp: add qom script entry points John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 07/19] scripts/qom-fuse: apply isort rules John Snow
` (13 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, John Snow, Markus Armbruster, Niteesh G . S .,
Cleber Rosa, Philippe Mathieu-Daudé
Redirect to the new qom scripts. These forwarders can be deleted
eventually when there has been more time for the dust on the Python
packaging to settle and people understand how to find these commands.
Note: You can run these by setting $PYTHONPATH in your shell and then
running "python3 -m qemu.qmp.qom", or you can install the qemu namespace
package and use the "qom" or "qom-set" scripts.
I've written how to install the package elsewhere, but for the sake of
git-blame, cd to ./python, and then do:
- pip3 install [--user] [-e] .
--user will install to your local user install (will not work inside of
a venv), omitting this flag installs to your system-wide packages
(outside of a venv) or to your current virtual environment (inside the
venv).
When installing to a venv or to your system-wide packages, "qom"
should be in your $PATH already. If you do a user install, you may
need to add ~/.local/bin to your $PATH if you haven't already.
-e installs in editable mode: the installed package is effectively just
a symlink to this folder; so changes to your git working tree are
reflected in the installed package.
Note: installing these packages to an environment outside a venv can be
dangerous: Many QEMU scripts will begin to prefer the installed version
instead of the version directly in the tree. Use with caution. editable
mode is recommended when working outside of a venv.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
scripts/qmp/qom-get | 66 +++------------------------------------
scripts/qmp/qom-list | 63 +++----------------------------------
scripts/qmp/qom-set | 63 +++----------------------------------
scripts/qmp/qom-tree | 74 +++-----------------------------------------
4 files changed, 16 insertions(+), 250 deletions(-)
diff --git a/scripts/qmp/qom-get b/scripts/qmp/qom-get
index 666df718320..e4f3e0c0138 100755
--- a/scripts/qmp/qom-get
+++ b/scripts/qmp/qom-get
@@ -1,69 +1,11 @@
#!/usr/bin/env python3
-##
-# QEMU Object Model test tools
-#
-# Copyright IBM, Corp. 2011
-#
-# Authors:
-# Anthony Liguori <aliguori@us.ibm.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or later. See
-# the COPYING file in the top-level directory.
-##
-import sys
import os
+import sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.qmp import QEMUMonitorProtocol
+from qemu.qmp.qom import QOMGet
-cmd, args = sys.argv[0], sys.argv[1:]
-socket_path = None
-path = None
-prop = None
-def usage():
- return '''environment variables:
- QMP_SOCKET=<path | addr:port>
-usage:
- %s [-h] [-s <QMP socket path | addr:port>] <path>.<property>
-''' % cmd
-
-def usage_error(error_msg = "unspecified error"):
- sys.stderr.write('%s\nERROR: %s\n' % (usage(), error_msg))
- exit(1)
-
-if len(args) > 0:
- if args[0] == "-h":
- print(usage())
- exit(0);
- elif args[0] == "-s":
- try:
- socket_path = args[1]
- except:
- usage_error("missing argument: QMP socket path or address");
- args = args[2:]
-
-if not socket_path:
- if 'QMP_SOCKET' in os.environ:
- socket_path = os.environ['QMP_SOCKET']
- else:
- usage_error("no QMP socket path or address given");
-
-if len(args) > 0:
- try:
- path, prop = args[0].rsplit('.', 1)
- except:
- usage_error("invalid format for path/property/value")
-else:
- usage_error("not enough arguments")
-
-srv = QEMUMonitorProtocol(socket_path)
-srv.connect()
-
-rsp = srv.command('qom-get', path=path, property=prop)
-if type(rsp) == dict:
- for i in rsp.keys():
- print('%s: %s' % (i, rsp[i]))
-else:
- print(rsp)
+if __name__ == '__main__':
+ sys.exit(QOMGet.entry_point())
diff --git a/scripts/qmp/qom-list b/scripts/qmp/qom-list
index 5074fd939f4..7a071a54e1e 100755
--- a/scripts/qmp/qom-list
+++ b/scripts/qmp/qom-list
@@ -1,66 +1,11 @@
#!/usr/bin/env python3
-##
-# QEMU Object Model test tools
-#
-# Copyright IBM, Corp. 2011
-#
-# Authors:
-# Anthony Liguori <aliguori@us.ibm.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or later. See
-# the COPYING file in the top-level directory.
-##
-import sys
import os
+import sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.qmp import QEMUMonitorProtocol
+from qemu.qmp.qom import QOMList
-cmd, args = sys.argv[0], sys.argv[1:]
-socket_path = None
-path = None
-prop = None
-def usage():
- return '''environment variables:
- QMP_SOCKET=<path | addr:port>
-usage:
- %s [-h] [-s <QMP socket path | addr:port>] [<path>]
-''' % cmd
-
-def usage_error(error_msg = "unspecified error"):
- sys.stderr.write('%s\nERROR: %s\n' % (usage(), error_msg))
- exit(1)
-
-if len(args) > 0:
- if args[0] == "-h":
- print(usage())
- exit(0);
- elif args[0] == "-s":
- try:
- socket_path = args[1]
- except:
- usage_error("missing argument: QMP socket path or address");
- args = args[2:]
-
-if not socket_path:
- if 'QMP_SOCKET' in os.environ:
- socket_path = os.environ['QMP_SOCKET']
- else:
- usage_error("no QMP socket path or address given");
-
-srv = QEMUMonitorProtocol(socket_path)
-srv.connect()
-
-if len(args) == 0:
- print('/')
- sys.exit(0)
-
-for item in srv.command('qom-list', path=args[0]):
- if item['type'].startswith('child<'):
- print('%s/' % item['name'])
- elif item['type'].startswith('link<'):
- print('@%s/' % item['name'])
- else:
- print('%s' % item['name'])
+if __name__ == '__main__':
+ sys.exit(QOMList.entry_point())
diff --git a/scripts/qmp/qom-set b/scripts/qmp/qom-set
index 240a78187f9..9ca9e2ba106 100755
--- a/scripts/qmp/qom-set
+++ b/scripts/qmp/qom-set
@@ -1,66 +1,11 @@
#!/usr/bin/env python3
-##
-# QEMU Object Model test tools
-#
-# Copyright IBM, Corp. 2011
-#
-# Authors:
-# Anthony Liguori <aliguori@us.ibm.com>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or later. See
-# the COPYING file in the top-level directory.
-##
-import sys
import os
+import sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.qmp import QEMUMonitorProtocol
+from qemu.qmp.qom import QOMSet
-cmd, args = sys.argv[0], sys.argv[1:]
-socket_path = None
-path = None
-prop = None
-value = None
-def usage():
- return '''environment variables:
- QMP_SOCKET=<path | addr:port>
-usage:
- %s [-h] [-s <QMP socket path | addr:port>] <path>.<property> <value>
-''' % cmd
-
-def usage_error(error_msg = "unspecified error"):
- sys.stderr.write('%s\nERROR: %s\n' % (usage(), error_msg))
- exit(1)
-
-if len(args) > 0:
- if args[0] == "-h":
- print(usage())
- exit(0);
- elif args[0] == "-s":
- try:
- socket_path = args[1]
- except:
- usage_error("missing argument: QMP socket path or address");
- args = args[2:]
-
-if not socket_path:
- if 'QMP_SOCKET' in os.environ:
- socket_path = os.environ['QMP_SOCKET']
- else:
- usage_error("no QMP socket path or address given");
-
-if len(args) > 1:
- try:
- path, prop = args[0].rsplit('.', 1)
- except:
- usage_error("invalid format for path/property/value")
- value = args[1]
-else:
- usage_error("not enough arguments")
-
-srv = QEMUMonitorProtocol(socket_path)
-srv.connect()
-
-print(srv.command('qom-set', path=path, property=prop, value=value))
+if __name__ == '__main__':
+ sys.exit(QOMSet.entry_point())
diff --git a/scripts/qmp/qom-tree b/scripts/qmp/qom-tree
index 25b0781323c..7d0ccca3a4d 100755
--- a/scripts/qmp/qom-tree
+++ b/scripts/qmp/qom-tree
@@ -1,77 +1,11 @@
#!/usr/bin/env python3
-##
-# QEMU Object Model test tools
-#
-# Copyright IBM, Corp. 2011
-# Copyright (c) 2013 SUSE LINUX Products GmbH
-#
-# Authors:
-# Anthony Liguori <aliguori@amazon.com>
-# Andreas Faerber <afaerber@suse.de>
-#
-# This work is licensed under the terms of the GNU GPL, version 2 or later. See
-# the COPYING file in the top-level directory.
-##
-import sys
import os
+import sys
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.qmp import QEMUMonitorProtocol
+from qemu.qmp.qom import QOMTree
-cmd, args = sys.argv[0], sys.argv[1:]
-socket_path = None
-path = None
-prop = None
-def usage():
- return '''environment variables:
- QMP_SOCKET=<path | addr:port>
-usage:
- %s [-h] [-s <QMP socket path | addr:port>] [<path>]
-''' % cmd
-
-def usage_error(error_msg = "unspecified error"):
- sys.stderr.write('%s\nERROR: %s\n' % (usage(), error_msg))
- exit(1)
-
-if len(args) > 0:
- if args[0] == "-h":
- print(usage())
- exit(0);
- elif args[0] == "-s":
- try:
- socket_path = args[1]
- except:
- usage_error("missing argument: QMP socket path or address");
- args = args[2:]
-
-if not socket_path:
- if 'QMP_SOCKET' in os.environ:
- socket_path = os.environ['QMP_SOCKET']
- else:
- usage_error("no QMP socket path or address given");
-
-srv = QEMUMonitorProtocol(socket_path)
-srv.connect()
-
-def list_node(path):
- print('%s' % path)
- items = srv.command('qom-list', path=path)
- for item in items:
- if not item['type'].startswith('child<'):
- try:
- print(' %s: %s (%s)' % (item['name'], srv.command('qom-get', path=path, property=item['name']), item['type']))
- except:
- print(' %s: <EXCEPTION> (%s)' % (item['name'], item['type']))
- print('')
- for item in items:
- if item['type'].startswith('child<'):
- list_node((path if (path != '/') else '') + '/' + item['name'])
-
-if len(args) == 0:
- path = '/'
-else:
- path = args[0]
-
-list_node(path)
+if __name__ == '__main__':
+ sys.exit(QOMTree.entry_point())
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 07/19] scripts/qom-fuse: apply isort rules
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (5 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 06/19] scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/ John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 08/19] scripts/qom-fuse: apply flake8 rules John Snow
` (12 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, John Snow, Markus Armbruster, Niteesh G . S .,
Cleber Rosa, Philippe Mathieu-Daudé
Hint: you can use symlinks to create qom_fuse.py in python/qemu/qmp/ and
point to scripts/qom-fuse to apply the standard linting rules to this
script.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
scripts/qmp/qom-fuse | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 7c7cff8edfb..62deb9adb12 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -13,14 +13,20 @@
# the COPYING file in the top-level directory.
##
-import fuse, stat
-from fuse import FUSE, FuseOSError, Operations
-import os, posix, sys
from errno import *
+import os
+import posix
+import stat
+import sys
+
+import fuse
+from fuse import FUSE, FuseOSError, Operations
+
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
from qemu.qmp import QEMUMonitorProtocol
+
fuse.fuse_python_api = (0, 2)
class QOMFS(Operations):
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 08/19] scripts/qom-fuse: apply flake8 rules
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (6 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 07/19] scripts/qom-fuse: apply isort rules John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 7:08 ` Philippe Mathieu-Daudé
2021-06-03 0:37 ` [PATCH v3 09/19] python: Add 'fh' to known-good variable names John Snow
` (11 subsequent siblings)
19 siblings, 1 reply; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
flake8 still has one warning because of the sys.path hack, but that will
be going away by the end of this patch series.
Signed-off-by: John Snow <jsnow@redhat.com>
---
scripts/qmp/qom-fuse | 81 +++++++++++++++++++++++---------------------
1 file changed, 43 insertions(+), 38 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 62deb9adb12..ca30e928679 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -9,13 +9,12 @@
# Anthony Liguori <aliguori@us.ibm.com>
# Markus Armbruster <armbru@redhat.com>
#
-# This work is licensed under the terms of the GNU GPL, version 2 or later. See
-# the COPYING file in the top-level directory.
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
##
-from errno import *
+from errno import ENOENT, EPERM
import os
-import posix
import stat
import sys
@@ -29,6 +28,7 @@ from qemu.qmp import QEMUMonitorProtocol
fuse.fuse_python_api = (0, 2)
+
class QOMFS(Operations):
def __init__(self, qmp):
self.qmp = qmp
@@ -45,7 +45,7 @@ class QOMFS(Operations):
def is_object(self, path):
try:
- items = self.qmp.command('qom-list', path=path)
+ self.qmp.command('qom-list', path=path)
return True
except:
return False
@@ -85,7 +85,7 @@ class QOMFS(Operations):
path = '/'
try:
data = self.qmp.command('qom-get', path=path, property=prop)
- data += '\n' # make values shell friendly
+ data += '\n' # make values shell friendly
except:
raise FuseOSError(EPERM)
@@ -104,38 +104,44 @@ class QOMFS(Operations):
def getattr(self, path, fh=None):
if self.is_link(path):
- value = { 'st_mode': 0o755 | stat.S_IFLNK,
- 'st_ino': self.get_ino(path),
- 'st_dev': 0,
- 'st_nlink': 2,
- 'st_uid': 1000,
- 'st_gid': 1000,
- 'st_size': 4096,
- 'st_atime': 0,
- 'st_mtime': 0,
- 'st_ctime': 0 }
+ value = {
+ 'st_mode': 0o755 | stat.S_IFLNK,
+ 'st_ino': self.get_ino(path),
+ 'st_dev': 0,
+ 'st_nlink': 2,
+ 'st_uid': 1000,
+ 'st_gid': 1000,
+ 'st_size': 4096,
+ 'st_atime': 0,
+ 'st_mtime': 0,
+ 'st_ctime': 0
+ }
elif self.is_object(path):
- value = { 'st_mode': 0o755 | stat.S_IFDIR,
- 'st_ino': self.get_ino(path),
- 'st_dev': 0,
- 'st_nlink': 2,
- 'st_uid': 1000,
- 'st_gid': 1000,
- 'st_size': 4096,
- 'st_atime': 0,
- 'st_mtime': 0,
- 'st_ctime': 0 }
+ value = {
+ 'st_mode': 0o755 | stat.S_IFDIR,
+ 'st_ino': self.get_ino(path),
+ 'st_dev': 0,
+ 'st_nlink': 2,
+ 'st_uid': 1000,
+ 'st_gid': 1000,
+ 'st_size': 4096,
+ 'st_atime': 0,
+ 'st_mtime': 0,
+ 'st_ctime': 0
+ }
elif self.is_property(path):
- value = { 'st_mode': 0o644 | stat.S_IFREG,
- 'st_ino': self.get_ino(path),
- 'st_dev': 0,
- 'st_nlink': 1,
- 'st_uid': 1000,
- 'st_gid': 1000,
- 'st_size': 4096,
- 'st_atime': 0,
- 'st_mtime': 0,
- 'st_ctime': 0 }
+ value = {
+ 'st_mode': 0o644 | stat.S_IFREG,
+ 'st_ino': self.get_ino(path),
+ 'st_dev': 0,
+ 'st_nlink': 1,
+ 'st_uid': 1000,
+ 'st_gid': 1000,
+ 'st_size': 4096,
+ 'st_atime': 0,
+ 'st_mtime': 0,
+ 'st_ctime': 0
+ }
else:
raise FuseOSError(ENOENT)
return value
@@ -146,8 +152,7 @@ class QOMFS(Operations):
for item in self.qmp.command('qom-list', path=path):
yield str(item['name'])
+
if __name__ == '__main__':
- import os
-
fuse = FUSE(QOMFS(QEMUMonitorProtocol(os.environ['QMP_SOCKET'])),
sys.argv[1], foreground=True)
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v3 08/19] scripts/qom-fuse: apply flake8 rules
2021-06-03 0:37 ` [PATCH v3 08/19] scripts/qom-fuse: apply flake8 rules John Snow
@ 2021-06-03 7:08 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 25+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-06-03 7:08 UTC (permalink / raw)
To: John Snow, qemu-devel
Cc: Cleber Rosa, Markus Armbruster, Eduardo Habkost, Niteesh G . S .
On 6/3/21 2:37 AM, John Snow wrote:
> flake8 still has one warning because of the sys.path hack, but that will
> be going away by the end of this patch series.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> scripts/qmp/qom-fuse | 81 +++++++++++++++++++++++---------------------
> 1 file changed, 43 insertions(+), 38 deletions(-)
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
^ permalink raw reply [flat|nested] 25+ messages in thread
* [PATCH v3 09/19] python: Add 'fh' to known-good variable names
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (7 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 08/19] scripts/qom-fuse: apply flake8 rules John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 10/19] scripts/qom-fuse: Apply pylint rules John Snow
` (10 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
fd and fh are fine: we often use these for "file descriptor" or "file
handle" accordingly. It is rarely the case that you need to enforce a
more semantically meaningful name beyond "This is the file we are using
right now."
While we're here: add comments for all of the non-standard pylint
names. (And the underscore.)
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/setup.cfg | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/python/setup.cfg b/python/setup.cfg
index a19029d5388..c9b9445af98 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -78,9 +78,10 @@ good-names=i,
k,
ex,
Run,
- _,
- fd,
- c,
+ _, # By convention: Unused variable
+ fh, # fh = open(...)
+ fd, # fd = os.open(...)
+ c, # for c in string: ...
[pylint.similarities]
# Ignore imports when computing similarities.
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 10/19] scripts/qom-fuse: Apply pylint rules
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (8 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 09/19] python: Add 'fh' to known-good variable names John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 11/19] scripts/qom-fuse: Add docstrings John Snow
` (9 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, John Snow, Markus Armbruster, Niteesh G . S .,
Cleber Rosa, Philippe Mathieu-Daudé
- Catch specific exceptions from QMP
- Reraise errors with explicit context
- method parameters should match parent's names
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
scripts/qmp/qom-fuse | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index ca30e928679..805e99c8ecd 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -23,7 +23,7 @@ from fuse import FUSE, FuseOSError, Operations
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.qmp import QEMUMonitorProtocol
+from qemu.qmp import QEMUMonitorProtocol, QMPResponseError
fuse.fuse_python_api = (0, 2)
@@ -47,7 +47,7 @@ class QOMFS(Operations):
try:
self.qmp.command('qom-list', path=path)
return True
- except:
+ except QMPResponseError:
return False
def is_property(self, path):
@@ -59,7 +59,7 @@ class QOMFS(Operations):
if item['name'] == prop:
return True
return False
- except:
+ except QMPResponseError:
return False
def is_link(self, path):
@@ -73,10 +73,10 @@ class QOMFS(Operations):
return True
return False
return False
- except:
+ except QMPResponseError:
return False
- def read(self, path, length, offset, fh):
+ def read(self, path, size, offset, fh):
if not self.is_property(path):
return -ENOENT
@@ -86,13 +86,13 @@ class QOMFS(Operations):
try:
data = self.qmp.command('qom-get', path=path, property=prop)
data += '\n' # make values shell friendly
- except:
- raise FuseOSError(EPERM)
+ except QMPResponseError as err:
+ raise FuseOSError(EPERM) from err
if offset > len(data):
return ''
- return bytes(data[offset:][:length], encoding='utf-8')
+ return bytes(data[offset:][:size], encoding='utf-8')
def readlink(self, path):
if not self.is_link(path):
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 11/19] scripts/qom-fuse: Add docstrings
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (9 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 10/19] scripts/qom-fuse: Apply pylint rules John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 12/19] scripts/qom-fuse: Convert to QOMCommand John Snow
` (8 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
The methods inherited from fuse don't need docstrings; that's up to
fusepy to handle.
Signed-off-by: John Snow <jsnow@redhat.com>
---
scripts/qmp/qom-fuse | 21 +++++++++++++++++++--
1 file changed, 19 insertions(+), 2 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 805e99c8ecd..1fb3008a167 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -1,7 +1,19 @@
#!/usr/bin/env python3
+"""
+QEMU Object Model FUSE filesystem tool
+
+This script offers a simple FUSE filesystem within which the QOM tree
+may be browsed, queried and edited using traditional shell tooling.
+
+This script requires the 'fusepy' python package.
+
+ENV:
+ QMP_SOCKET: Path to the QMP server socket
+
+Usage:
+ qom-fuse /mount/to/here
+"""
##
-# QEMU Object Model test tools
-#
# Copyright IBM, Corp. 2012
# Copyright (C) 2020 Red Hat, Inc.
#
@@ -30,6 +42,7 @@ fuse.fuse_python_api = (0, 2)
class QOMFS(Operations):
+ """QOMFS implements fuse.Operations to provide a QOM filesystem."""
def __init__(self, qmp):
self.qmp = qmp
self.qmp.connect()
@@ -37,6 +50,7 @@ class QOMFS(Operations):
self.ino_count = 1
def get_ino(self, path):
+ """Get an inode number for a given QOM path."""
if path in self.ino_map:
return self.ino_map[path]
self.ino_map[path] = self.ino_count
@@ -44,6 +58,7 @@ class QOMFS(Operations):
return self.ino_map[path]
def is_object(self, path):
+ """Is the given QOM path an object?"""
try:
self.qmp.command('qom-list', path=path)
return True
@@ -51,6 +66,7 @@ class QOMFS(Operations):
return False
def is_property(self, path):
+ """Is the given QOM path a property?"""
path, prop = path.rsplit('/', 1)
if path == '':
path = '/'
@@ -63,6 +79,7 @@ class QOMFS(Operations):
return False
def is_link(self, path):
+ """Is the given QOM path a link?"""
path, prop = path.rsplit('/', 1)
if path == '':
path = '/'
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 12/19] scripts/qom-fuse: Convert to QOMCommand
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (10 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 11/19] scripts/qom-fuse: Add docstrings John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 13/19] scripts/qom-fuse: use QOMCommand.qom_list() John Snow
` (7 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
Move qom-fuse onto the QOMCommand base established in
python/qemu/qmp/qom_common.py. The interface doesn't change
incompatibly, "qom-fuse mountpoint" still works as an invocation, and
QMP_SOCKET is still used as the environment variable.
Signed-off-by: John Snow <jsnow@redhat.com>
---
scripts/qmp/qom-fuse | 59 ++++++++++++++++++++++++++++++++++----------
1 file changed, 46 insertions(+), 13 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 1fb3008a167..1676fb78d99 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -7,11 +7,19 @@ may be browsed, queried and edited using traditional shell tooling.
This script requires the 'fusepy' python package.
-ENV:
- QMP_SOCKET: Path to the QMP server socket
-Usage:
- qom-fuse /mount/to/here
+usage: qom-fuse [-h] [--socket SOCKET] <mount>
+
+Mount a QOM tree as a FUSE filesystem
+
+positional arguments:
+ <mount> Mount point
+
+optional arguments:
+ -h, --help show this help message and exit
+ --socket SOCKET, -s SOCKET
+ QMP socket path or address (addr:port). May also be
+ set via QMP_SOCKET environment variable.
"""
##
# Copyright IBM, Corp. 2012
@@ -25,30 +33,56 @@ Usage:
# See the COPYING file in the top-level directory.
##
+import argparse
from errno import ENOENT, EPERM
import os
import stat
import sys
+from typing import Dict
import fuse
from fuse import FUSE, FuseOSError, Operations
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.qmp import QEMUMonitorProtocol, QMPResponseError
+from qemu.qmp import QMPResponseError
+from qemu.qmp.qom_common import QOMCommand
fuse.fuse_python_api = (0, 2)
-class QOMFS(Operations):
- """QOMFS implements fuse.Operations to provide a QOM filesystem."""
- def __init__(self, qmp):
- self.qmp = qmp
- self.qmp.connect()
- self.ino_map = {}
+class QOMFuse(QOMCommand, Operations):
+ """
+ QOMFuse implements both fuse.Operations and QOMCommand.
+
+ Operations implements the FS, and QOMCommand implements the CLI command.
+ """
+ name = 'fuse'
+ help = 'Mount a QOM tree as a FUSE filesystem'
+ fuse: FUSE
+
+ @classmethod
+ def configure_parser(cls, parser: argparse.ArgumentParser) -> None:
+ super().configure_parser(parser)
+ parser.add_argument(
+ 'mount',
+ metavar='<mount>',
+ action='store',
+ help="Mount point",
+ )
+
+ def __init__(self, args: argparse.Namespace):
+ super().__init__(args)
+ self.mount = args.mount
+ self.ino_map: Dict[str, int] = {}
self.ino_count = 1
+ def run(self) -> int:
+ print(f"Mounting QOMFS to '{self.mount}'", file=sys.stderr)
+ self.fuse = FUSE(self, self.mount, foreground=True)
+ return 0
+
def get_ino(self, path):
"""Get an inode number for a given QOM path."""
if path in self.ino_map:
@@ -171,5 +205,4 @@ class QOMFS(Operations):
if __name__ == '__main__':
- fuse = FUSE(QOMFS(QEMUMonitorProtocol(os.environ['QMP_SOCKET'])),
- sys.argv[1], foreground=True)
+ sys.exit(QOMFuse.entry_point())
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 13/19] scripts/qom-fuse: use QOMCommand.qom_list()
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (11 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 12/19] scripts/qom-fuse: Convert to QOMCommand John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 7:09 ` Philippe Mathieu-Daudé
2021-06-03 0:37 ` [PATCH v3 14/19] scripts/qom-fuse: ensure QOMFuse.read always returns bytes John Snow
` (6 subsequent siblings)
19 siblings, 1 reply; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
the qom_list method provides a type-safe object that's easier to type
check, so switch to using it.
Signed-off-by: John Snow <jsnow@redhat.com>
---
scripts/qmp/qom-fuse | 18 ++++++++----------
1 file changed, 8 insertions(+), 10 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 1676fb78d99..703a97e75ff 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -94,7 +94,7 @@ class QOMFuse(QOMCommand, Operations):
def is_object(self, path):
"""Is the given QOM path an object?"""
try:
- self.qmp.command('qom-list', path=path)
+ self.qom_list(path)
return True
except QMPResponseError:
return False
@@ -105,8 +105,8 @@ class QOMFuse(QOMCommand, Operations):
if path == '':
path = '/'
try:
- for item in self.qmp.command('qom-list', path=path):
- if item['name'] == prop:
+ for item in self.qom_list(path):
+ if item.name == prop:
return True
return False
except QMPResponseError:
@@ -118,11 +118,9 @@ class QOMFuse(QOMCommand, Operations):
if path == '':
path = '/'
try:
- for item in self.qmp.command('qom-list', path=path):
- if item['name'] == prop:
- if item['type'].startswith('link<'):
- return True
- return False
+ for item in self.qom_list(path):
+ if item.name == prop and item.link:
+ return True
return False
except QMPResponseError:
return False
@@ -200,8 +198,8 @@ class QOMFuse(QOMCommand, Operations):
def readdir(self, path, fh):
yield '.'
yield '..'
- for item in self.qmp.command('qom-list', path=path):
- yield str(item['name'])
+ for item in self.qom_list(path):
+ yield item.name
if __name__ == '__main__':
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 14/19] scripts/qom-fuse: ensure QOMFuse.read always returns bytes
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (12 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 13/19] scripts/qom-fuse: use QOMCommand.qom_list() John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 15/19] scripts/qom-fuse: add static type hints John Snow
` (5 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, John Snow, Markus Armbruster, Niteesh G . S .,
Cleber Rosa, Philippe Mathieu-Daudé
- Use FuseOSError to signal ENOENT instead of returning it
- Wrap qom-get in str(), as we don't always know its type
- The empty return should be b'', not ''.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
scripts/qmp/qom-fuse | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 703a97e75ff..0d11f731526 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -127,19 +127,19 @@ class QOMFuse(QOMCommand, Operations):
def read(self, path, size, offset, fh):
if not self.is_property(path):
- return -ENOENT
+ raise FuseOSError(ENOENT)
path, prop = path.rsplit('/', 1)
if path == '':
path = '/'
try:
- data = self.qmp.command('qom-get', path=path, property=prop)
+ data = str(self.qmp.command('qom-get', path=path, property=prop))
data += '\n' # make values shell friendly
except QMPResponseError as err:
raise FuseOSError(EPERM) from err
if offset > len(data):
- return ''
+ return b''
return bytes(data[offset:][:size], encoding='utf-8')
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 15/19] scripts/qom-fuse: add static type hints
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (13 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 14/19] scripts/qom-fuse: ensure QOMFuse.read always returns bytes John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 16/19] python: add optional FUSE dependencies John Snow
` (4 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
Because fusepy does not have type hints, add some targeted warning
suppressions.
Namely, we need to allow subclassing something of an unknown type (in
qom_fuse.py), and we need to allow missing imports (recorded against
fuse itself) because mypy will be unable to import fusepy (even when
installed) as it has no types nor type stubs available.
Note: Until now, it was possible to run invocations like 'mypy qemu/'
from ./python and have that work. However, these targeted suppressions
require that you run 'mypy -p qemu/' instead. The correct, canonical
invocation is recorded in ./python/tests/mypy.sh and all of the various
CI invocations always use this correct form.
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/setup.cfg | 8 ++++++++
scripts/qmp/qom-fuse | 26 +++++++++++++++++---------
2 files changed, 25 insertions(+), 9 deletions(-)
diff --git a/python/setup.cfg b/python/setup.cfg
index c9b9445af98..ba8d29fd62d 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -57,6 +57,14 @@ python_version = 3.6
warn_unused_configs = True
namespace_packages = True
+[mypy-qemu.qmp.qom_fuse]
+# fusepy has no type stubs:
+allow_subclassing_any = True
+
+[mypy-fuse]
+# fusepy has no type stubs:
+ignore_missing_imports = True
+
[pylint.messages control]
# Disable the message, report, category or checker with the given id(s). You
# can either give multiple identifiers separated by comma (,) or put this
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
index 0d11f731526..a5a7a304a3e 100755
--- a/scripts/qmp/qom-fuse
+++ b/scripts/qmp/qom-fuse
@@ -38,7 +38,14 @@ from errno import ENOENT, EPERM
import os
import stat
import sys
-from typing import Dict
+from typing import (
+ IO,
+ Dict,
+ Iterator,
+ Mapping,
+ Optional,
+ Union,
+)
import fuse
from fuse import FUSE, FuseOSError, Operations
@@ -83,7 +90,7 @@ class QOMFuse(QOMCommand, Operations):
self.fuse = FUSE(self, self.mount, foreground=True)
return 0
- def get_ino(self, path):
+ def get_ino(self, path: str) -> int:
"""Get an inode number for a given QOM path."""
if path in self.ino_map:
return self.ino_map[path]
@@ -91,7 +98,7 @@ class QOMFuse(QOMCommand, Operations):
self.ino_count += 1
return self.ino_map[path]
- def is_object(self, path):
+ def is_object(self, path: str) -> bool:
"""Is the given QOM path an object?"""
try:
self.qom_list(path)
@@ -99,7 +106,7 @@ class QOMFuse(QOMCommand, Operations):
except QMPResponseError:
return False
- def is_property(self, path):
+ def is_property(self, path: str) -> bool:
"""Is the given QOM path a property?"""
path, prop = path.rsplit('/', 1)
if path == '':
@@ -112,7 +119,7 @@ class QOMFuse(QOMCommand, Operations):
except QMPResponseError:
return False
- def is_link(self, path):
+ def is_link(self, path: str) -> bool:
"""Is the given QOM path a link?"""
path, prop = path.rsplit('/', 1)
if path == '':
@@ -125,7 +132,7 @@ class QOMFuse(QOMCommand, Operations):
except QMPResponseError:
return False
- def read(self, path, size, offset, fh):
+ def read(self, path: str, size: int, offset: int, fh: IO[bytes]) -> bytes:
if not self.is_property(path):
raise FuseOSError(ENOENT)
@@ -143,7 +150,7 @@ class QOMFuse(QOMCommand, Operations):
return bytes(data[offset:][:size], encoding='utf-8')
- def readlink(self, path):
+ def readlink(self, path: str) -> Union[bool, str]:
if not self.is_link(path):
return False
path, prop = path.rsplit('/', 1)
@@ -151,7 +158,8 @@ class QOMFuse(QOMCommand, Operations):
return prefix + str(self.qmp.command('qom-get', path=path,
property=prop))
- def getattr(self, path, fh=None):
+ def getattr(self, path: str,
+ fh: Optional[IO[bytes]] = None) -> Mapping[str, object]:
if self.is_link(path):
value = {
'st_mode': 0o755 | stat.S_IFLNK,
@@ -195,7 +203,7 @@ class QOMFuse(QOMCommand, Operations):
raise FuseOSError(ENOENT)
return value
- def readdir(self, path, fh):
+ def readdir(self, path: str, fh: IO[bytes]) -> Iterator[str]:
yield '.'
yield '..'
for item in self.qom_list(path):
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 16/19] python: add optional FUSE dependencies
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (14 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 15/19] scripts/qom-fuse: add static type hints John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 17/19] scripts/qom-fuse: move to python/qemu/qmp/qom_fuse.py John Snow
` (3 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
In preparation for moving qom-fuse over to the python package, we need
some new dependencies to support it.
Add an optional 'fusepy' dependency that users of the package can opt
into with e.g. "pip install qemu[fuse]" which installs the requirements
necessary to obtain the additional functionality.
Add the same fusepy dependency to the 'devel' extras group --
unfortunately I do not see a way for optional groups to imply other
optional groups at present, so the dependency is repeated. The
development group needs to include the full set of dependencies for the
purpose of static analysis of all features offered by this library.
Lastly, add the [fuse] extras group to tox's configuration as a
workaround so that if a stale tox environment is found when running
`make check-tox`, tox will know to rebuild its environments.
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/Pipfile.lock | 6 ++++++
python/setup.cfg | 9 ++++++++-
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/python/Pipfile.lock b/python/Pipfile.lock
index f2a3f91d0fa..5bb3f1b6351 100644
--- a/python/Pipfile.lock
+++ b/python/Pipfile.lock
@@ -67,6 +67,12 @@
"markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'",
"version": "==3.9.2"
},
+ "fusepy": {
+ "hashes": [
+ "sha256:72ff783ec2f43de3ab394e3f7457605bf04c8cf288a2f4068b4cde141d4ee6bd"
+ ],
+ "version": "==3.0.1"
+ },
"importlib-metadata": {
"hashes": [
"sha256:8c501196e49fb9df5df43833bdb1e4328f64847763ec8a50703148b73784d581",
diff --git a/python/setup.cfg b/python/setup.cfg
index ba8d29fd62d..aca6f311853 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -32,11 +32,16 @@ packages =
devel =
avocado-framework >= 87.0
flake8 >= 3.6.0
+ fusepy >= 2.0.4
isort >= 5.1.2
mypy >= 0.770
pylint >= 2.8.0
tox >= 3.18.0
+# Provides qom-fuse functionality
+fuse =
+ fusepy >= 2.0.4
+
[options.entry_points]
console_scripts =
qom = qemu.qmp.qom:main
@@ -114,6 +119,8 @@ envlist = py36, py37, py38, py39, py310
[testenv]
allowlist_externals = make
-deps = .[devel]
+deps =
+ .[devel]
+ .[fuse] # Workaround to trigger tox venv rebuild
commands =
make check
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 17/19] scripts/qom-fuse: move to python/qemu/qmp/qom_fuse.py
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (15 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 16/19] python: add optional FUSE dependencies John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 18/19] scripts/qom-fuse: add redirection shim to python/qemu/qmp/qom-fuse.py John Snow
` (2 subsequent siblings)
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
Move qom-fuse over to the python package now that it passes the
linter. Update the import paradigms so that it continues to pass in the
context of the Python package.
Signed-off-by: John Snow <jsnow@redhat.com>
---
scripts/qmp/qom-fuse => python/qemu/qmp/qom_fuse.py | 12 ++----------
1 file changed, 2 insertions(+), 10 deletions(-)
rename scripts/qmp/qom-fuse => python/qemu/qmp/qom_fuse.py (95%)
mode change 100755 => 100644
diff --git a/scripts/qmp/qom-fuse b/python/qemu/qmp/qom_fuse.py
old mode 100755
new mode 100644
similarity index 95%
rename from scripts/qmp/qom-fuse
rename to python/qemu/qmp/qom_fuse.py
index a5a7a304a3e..43f4671fdb1
--- a/scripts/qmp/qom-fuse
+++ b/python/qemu/qmp/qom_fuse.py
@@ -1,4 +1,3 @@
-#!/usr/bin/env python3
"""
QEMU Object Model FUSE filesystem tool
@@ -35,7 +34,6 @@
import argparse
from errno import ENOENT, EPERM
-import os
import stat
import sys
from typing import (
@@ -50,10 +48,8 @@
import fuse
from fuse import FUSE, FuseOSError, Operations
-
-sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
-from qemu.qmp import QMPResponseError
-from qemu.qmp.qom_common import QOMCommand
+from . import QMPResponseError
+from .qom_common import QOMCommand
fuse.fuse_python_api = (0, 2)
@@ -208,7 +204,3 @@ def readdir(self, path: str, fh: IO[bytes]) -> Iterator[str]:
yield '..'
for item in self.qom_list(path):
yield item.name
-
-
-if __name__ == '__main__':
- sys.exit(QOMFuse.entry_point())
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 18/19] scripts/qom-fuse: add redirection shim to python/qemu/qmp/qom-fuse.py
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (16 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 17/19] scripts/qom-fuse: move to python/qemu/qmp/qom_fuse.py John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 0:37 ` [PATCH v3 19/19] python/qmp: add fuse command to 'qom' tools John Snow
2021-06-09 20:32 ` [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Eduardo Habkost, John Snow, Markus Armbruster, Niteesh G . S .,
Cleber Rosa, Philippe Mathieu-Daudé
By leaving the script absent for a commit, git-blame travels to the new
file instead of staying on the shim.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
scripts/qmp/qom-fuse | 11 +++++++++++
1 file changed, 11 insertions(+)
create mode 100755 scripts/qmp/qom-fuse
diff --git a/scripts/qmp/qom-fuse b/scripts/qmp/qom-fuse
new file mode 100755
index 00000000000..a58c8ef9793
--- /dev/null
+++ b/scripts/qmp/qom-fuse
@@ -0,0 +1,11 @@
+#!/usr/bin/env python3
+
+import os
+import sys
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp.qom_fuse import QOMFuse
+
+
+if __name__ == '__main__':
+ sys.exit(QOMFuse.entry_point())
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* [PATCH v3 19/19] python/qmp: add fuse command to 'qom' tools
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (17 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 18/19] scripts/qom-fuse: add redirection shim to python/qemu/qmp/qom-fuse.py John Snow
@ 2021-06-03 0:37 ` John Snow
2021-06-03 7:11 ` Philippe Mathieu-Daudé
2021-06-09 20:32 ` [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
19 siblings, 1 reply; 25+ messages in thread
From: John Snow @ 2021-06-03 0:37 UTC (permalink / raw)
To: qemu-devel
Cc: Markus Armbruster, John Snow, Niteesh G . S .,
Eduardo Habkost, Cleber Rosa
The 'fuse' command will be unavailable if 'fusepy' is not installed. It
will simply not load and subsequently be unavailable as a subcommand.
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/qemu/qmp/qom.py | 14 ++++++++++++--
python/setup.cfg | 1 +
2 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/python/qemu/qmp/qom.py b/python/qemu/qmp/qom.py
index 7fe1448b5d9..7ec7843d57a 100644
--- a/python/qemu/qmp/qom.py
+++ b/python/qemu/qmp/qom.py
@@ -1,7 +1,7 @@
"""
QEMU Object Model testing tools.
-usage: qom [-h] {set,get,list,tree} ...
+usage: qom [-h] {set,get,list,tree,fuse} ...
Query and manipulate QOM data
@@ -9,11 +9,12 @@
-h, --help show this help message and exit
QOM commands:
- {set,get,list,tree}
+ {set,get,list,tree,fuse}
set Set a QOM property value
get Get a QOM property value
list List QOM properties at a given path
tree Show QOM tree from a given path
+ fuse Mount a QOM tree as a FUSE filesystem
"""
##
# Copyright John Snow 2020, for Red Hat, Inc.
@@ -35,6 +36,15 @@
from .qom_common import QOMCommand
+try:
+ from .qom_fuse import QOMFuse
+except ModuleNotFoundError as err:
+ if err.name != 'fuse':
+ raise
+else:
+ assert issubclass(QOMFuse, QOMCommand)
+
+
class QOMSet(QOMCommand):
"""
QOM Command - Set a property to a given value.
diff --git a/python/setup.cfg b/python/setup.cfg
index aca6f311853..6b6be8b03c6 100644
--- a/python/setup.cfg
+++ b/python/setup.cfg
@@ -49,6 +49,7 @@ console_scripts =
qom-get = qemu.qmp.qom:QOMGet.entry_point
qom-list = qemu.qmp.qom:QOMList.entry_point
qom-tree = qemu.qmp.qom:QOMTree.entry_point
+ qom-fuse = qemu.qmp.qom_fuse:QOMFuse.entry_point [fuse]
[flake8]
extend-ignore = E722 # Prefer pylint's bare-except checks to flake8's
--
2.31.1
^ permalink raw reply related [flat|nested] 25+ messages in thread* Re: [PATCH v3 19/19] python/qmp: add fuse command to 'qom' tools
2021-06-03 0:37 ` [PATCH v3 19/19] python/qmp: add fuse command to 'qom' tools John Snow
@ 2021-06-03 7:11 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 25+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-06-03 7:11 UTC (permalink / raw)
To: John Snow, qemu-devel
Cc: Cleber Rosa, Markus Armbruster, Eduardo Habkost, Niteesh G . S .
On 6/3/21 2:37 AM, John Snow wrote:
> The 'fuse' command will be unavailable if 'fusepy' is not installed. It
> will simply not load and subsequently be unavailable as a subcommand.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> python/qemu/qmp/qom.py | 14 ++++++++++++--
> python/setup.cfg | 1 +
> 2 files changed, 13 insertions(+), 2 deletions(-)
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
^ permalink raw reply [flat|nested] 25+ messages in thread
* Re: [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom*
2021-06-03 0:37 [PATCH v3 00/19] Python: move /scripts/qmp/qom* to /python/qemu/qmp/qom* John Snow
` (18 preceding siblings ...)
2021-06-03 0:37 ` [PATCH v3 19/19] python/qmp: add fuse command to 'qom' tools John Snow
@ 2021-06-09 20:32 ` John Snow
19 siblings, 0 replies; 25+ messages in thread
From: John Snow @ 2021-06-09 20:32 UTC (permalink / raw)
To: qemu-devel
Cc: Cleber Rosa, Philippe Mathieu-Daudé,
Markus Armbruster, Eduardo Habkost, Niteesh G . S .
On 6/2/21 8:37 PM, John Snow wrote:
> Closes: https://gitlab.com/qemu-project/qemu/-/issues/202
> GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-package-qom
> CI: https://gitlab.com/jsnow/qemu/-/pipelines/313932818
>
> Hello!
> ------
>
> This series applies the usual linting cleanups to modernize the qom
> tools and then integrates them into the python packaging hierarchy.
>
> This will help prevent further bitrot of these tools.
>
> I don't expect or need a detailed review of the QOM tools themselves --
> these tools are not used during build OR testing, and some are fairly
> bitrotted in places.
>
> However, some details of how the python packaging system is being
> utilized here may attract your attention and could be worth a look.
> (Patches 5-6 and 16-19 are the interesting ones.)
>
> Since these scripts aren't critical, I'm OK with sending a fairly hasty
> PR to merge these sooner rather than later.
>
> Overview:
> ---------
>
> Patch 1: Update Pipfile.lock (See the commit as for why ...)
>
> Patches 2-3: Correct some existing typing issues in qemu.qmp
>
> Patch 4: Combine qom-set, qom-get, (etc) into one, newly written script
> that makes all of the command invocations, help text, etc. consistent.
> (I ask that review for this patch should be limited to critical
> mistakes: I have no interest in developing the QOM tools further.)
>
> Patches 5-6: Integrate the qom tools into the python package.
>
> Patches 7-15: Delinting of the qom_fuse script. Similarly, I am not
> terribly interested in further improvements here, personally.
>
> Patches 16-19: Integrating qom-fuse into the Python packaging directory;
> additional care is taken to ensure that "optional" dependencies like
> fusepy are handled well.
>
> Changelog
> ---------
>
> V3:
> - Technically, I sent two versions of this before, a long time ago.
> This has been cleaned up and based on the latest origin/master.
>
> John Snow (19):
> python/pipenv: Update Pipfile.lock
> python/qmp: Fix type of SocketAddrT
> python/qmp: add parse_address classmethod
> python/qmp: Add qom script rewrites
> python/qmp: add qom script entry points
> scripts/qmp: redirect qom-xxx scripts to python/qemu/qmp/
> scripts/qom-fuse: apply isort rules
> scripts/qom-fuse: apply flake8 rules
> python: Add 'fh' to known-good variable names
> scripts/qom-fuse: Apply pylint rules
> scripts/qom-fuse: Add docstrings
> scripts/qom-fuse: Convert to QOMCommand
> scripts/qom-fuse: use QOMCommand.qom_list()
> scripts/qom-fuse: ensure QOMFuse.read always returns bytes
> scripts/qom-fuse: add static type hints
> python: add optional FUSE dependencies
> scripts/qom-fuse: move to python/qemu/qmp/qom_fuse.py
> scripts/qom-fuse: add redirection shim to python/qemu/qmp/qom-fuse.py
> python/qmp: add fuse command to 'qom' tools
>
> python/Pipfile.lock | 97 +++++++++++-
> python/qemu/qmp/__init__.py | 28 +++-
> python/qemu/qmp/qom.py | 272 ++++++++++++++++++++++++++++++++++
> python/qemu/qmp/qom_common.py | 178 ++++++++++++++++++++++
> python/qemu/qmp/qom_fuse.py | 206 +++++++++++++++++++++++++
> python/setup.cfg | 33 ++++-
> scripts/qmp/qmp-shell | 21 +--
> scripts/qmp/qom-fuse | 144 +-----------------
> scripts/qmp/qom-get | 66 +--------
> scripts/qmp/qom-list | 63 +-------
> scripts/qmp/qom-set | 63 +-------
> scripts/qmp/qom-tree | 74 +--------
> 12 files changed, 828 insertions(+), 417 deletions(-)
> create mode 100644 python/qemu/qmp/qom.py
> create mode 100644 python/qemu/qmp/qom_common.py
> create mode 100644 python/qemu/qmp/qom_fuse.py
>
Provisionally staged to my Python branch:
https://gitlab.com/jsnow/qemu/-/commits/python/
(Barring objections, I intend to send a PR for all the ./scripts/qmp/
cleanups at once, next Friday.)
--js
^ permalink raw reply [flat|nested] 25+ messages in thread