* [Qemu-devel] [PATCH RFC 0/3] qemu-img: remove command documentation duplication
@ 2019-04-10 1:24 ` John Snow
0 siblings, 0 replies; 17+ messages in thread
From: John Snow @ 2019-04-10 1:24 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: armbru, eblake, Eduardo Habkost, Max Reitz, Kevin Wolf,
Cleber Rosa, John Snow
This might hopefully cut down on the doc duplication/mismatching
until I can devise something more comprehensive.
Ideally, I'd like to redo all of the documentation for qemu-img
nearly from scratch; with a parser generator that helps generate
the documentation as well so they'll never get out of date.
That's probably quite a ways off, and maybe there are other
structural changes we want to make with respect to sphinx and
other build tools, so I am sending something very minimal instead.
This ought to be functionally identical down to the last char.
(I've re-included patch one which I have sent to the list separately,
purely as a dependency if you want to apply these patches.)
John Snow (3):
qemu-img: fix .hx and .texi disparity
pxtool: Add new qemu-img command info generation tool
qemu-img.texi: use macros for command summaries
qemu-img.texi | 28 ++++----
Makefile | 8 +--
qemu-img-cmds.json | 165 +++++++++++++++++++++++++++++++++++++++++++++
qemu-img-cmds.hx | 102 ----------------------------
scripts/pxtool.py | 53 +++++++++++++++
5 files changed, 236 insertions(+), 120 deletions(-)
create mode 100644 qemu-img-cmds.json
delete mode 100644 qemu-img-cmds.hx
create mode 100755 scripts/pxtool.py
--
2.17.2
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH RFC 0/3] qemu-img: remove command documentation duplication
@ 2019-04-10 1:24 ` John Snow
0 siblings, 0 replies; 17+ messages in thread
From: John Snow @ 2019-04-10 1:24 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: Kevin Wolf, Eduardo Habkost, John Snow, armbru, Max Reitz, Cleber Rosa
This might hopefully cut down on the doc duplication/mismatching
until I can devise something more comprehensive.
Ideally, I'd like to redo all of the documentation for qemu-img
nearly from scratch; with a parser generator that helps generate
the documentation as well so they'll never get out of date.
That's probably quite a ways off, and maybe there are other
structural changes we want to make with respect to sphinx and
other build tools, so I am sending something very minimal instead.
This ought to be functionally identical down to the last char.
(I've re-included patch one which I have sent to the list separately,
purely as a dependency if you want to apply these patches.)
John Snow (3):
qemu-img: fix .hx and .texi disparity
pxtool: Add new qemu-img command info generation tool
qemu-img.texi: use macros for command summaries
qemu-img.texi | 28 ++++----
Makefile | 8 +--
qemu-img-cmds.json | 165 +++++++++++++++++++++++++++++++++++++++++++++
qemu-img-cmds.hx | 102 ----------------------------
scripts/pxtool.py | 53 +++++++++++++++
5 files changed, 236 insertions(+), 120 deletions(-)
create mode 100644 qemu-img-cmds.json
delete mode 100644 qemu-img-cmds.hx
create mode 100755 scripts/pxtool.py
--
2.17.2
^ permalink raw reply [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH RFC 1/3] qemu-img: fix .hx and .texi disparity
@ 2019-04-10 1:24 ` John Snow
0 siblings, 0 replies; 17+ messages in thread
From: John Snow @ 2019-04-10 1:24 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: armbru, eblake, Eduardo Habkost, Max Reitz, Kevin Wolf,
Cleber Rosa, John Snow
It turns out that having options listed in three places continues to be
a bad idea. I'm still toying with the idea of an improved infrastructure
here, but in the meantime, another bandaid.
There are three locations:
(1) .hx file, formatted as texi
(2) .hx file, formatted as human readable.
(3) .texi file, as section headers, formatted as texi.
You can compare the two summaries within the .hx file like so:
Human-readable command summaries:
`./qemu-img --help | grep 'Command syntax' -A14`
Detokenized texi command summaries:
`grep "@item" qemu-img-cmds.hx | sed -E 's|@var\{([^\}]*?)\}|\1|g'`
You can compare the two separate texi summaries like so:
Texi command summaries:
`grep "@item" qemu-img-cmds.hx"`
Texi command headers:
grep -E "@item.*@var" qemu-img.texi | tail -14
Signed-off-by: John Snow <jsnow@redhat.com>
---
qemu-img.texi | 4 ++--
qemu-img-cmds.hx | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/qemu-img.texi b/qemu-img.texi
index 3b6710a580..724f244ba1 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -197,7 +197,7 @@ Command description:
@table @option
-@item amend [--object @var{objectdef}] [--image-opts] [-p] [-p] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
+@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
Amends the image format specific @var{options} for the image file
@var{filename}. Not all file formats support this operation.
@@ -424,7 +424,7 @@ To enumerate information about each disk image in the above chain, starting from
qemu-img info --backing-chain snap2.qcow2
@end example
-@item map [-f @var{fmt}] [--output=@var{ofmt}] @var{filename}
+@item map [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--output=@var{ofmt}] [-U] @var{filename}
Dump the metadata of image @var{filename} and its backing file chain.
In particular, this commands dumps the allocation state of every sector
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 1526f327a5..4b47f7495d 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -46,7 +46,7 @@ ETEXI
DEF("convert", img_convert,
"convert [--object objectdef] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] filename [filename2 [...]] output_filename")
STEXI
-@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] [-U] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m @var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] @var{output_filename}
+@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m @var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] @var{output_filename}
ETEXI
DEF("create", img_create,
--
2.17.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH RFC 1/3] qemu-img: fix .hx and .texi disparity
@ 2019-04-10 1:24 ` John Snow
0 siblings, 0 replies; 17+ messages in thread
From: John Snow @ 2019-04-10 1:24 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: Kevin Wolf, Eduardo Habkost, John Snow, armbru, Max Reitz, Cleber Rosa
It turns out that having options listed in three places continues to be
a bad idea. I'm still toying with the idea of an improved infrastructure
here, but in the meantime, another bandaid.
There are three locations:
(1) .hx file, formatted as texi
(2) .hx file, formatted as human readable.
(3) .texi file, as section headers, formatted as texi.
You can compare the two summaries within the .hx file like so:
Human-readable command summaries:
`./qemu-img --help | grep 'Command syntax' -A14`
Detokenized texi command summaries:
`grep "@item" qemu-img-cmds.hx | sed -E 's|@var\{([^\}]*?)\}|\1|g'`
You can compare the two separate texi summaries like so:
Texi command summaries:
`grep "@item" qemu-img-cmds.hx"`
Texi command headers:
grep -E "@item.*@var" qemu-img.texi | tail -14
Signed-off-by: John Snow <jsnow@redhat.com>
---
qemu-img.texi | 4 ++--
qemu-img-cmds.hx | 2 +-
2 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/qemu-img.texi b/qemu-img.texi
index 3b6710a580..724f244ba1 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -197,7 +197,7 @@ Command description:
@table @option
-@item amend [--object @var{objectdef}] [--image-opts] [-p] [-p] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
+@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
Amends the image format specific @var{options} for the image file
@var{filename}. Not all file formats support this operation.
@@ -424,7 +424,7 @@ To enumerate information about each disk image in the above chain, starting from
qemu-img info --backing-chain snap2.qcow2
@end example
-@item map [-f @var{fmt}] [--output=@var{ofmt}] @var{filename}
+@item map [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--output=@var{ofmt}] [-U] @var{filename}
Dump the metadata of image @var{filename} and its backing file chain.
In particular, this commands dumps the allocation state of every sector
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 1526f327a5..4b47f7495d 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -46,7 +46,7 @@ ETEXI
DEF("convert", img_convert,
"convert [--object objectdef] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] filename [filename2 [...]] output_filename")
STEXI
-@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] [-U] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m @var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] @var{output_filename}
+@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m @var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] @var{output_filename}
ETEXI
DEF("create", img_create,
--
2.17.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool
@ 2019-04-10 1:24 ` John Snow
0 siblings, 0 replies; 17+ messages in thread
From: John Snow @ 2019-04-10 1:24 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: armbru, eblake, Eduardo Habkost, Max Reitz, Kevin Wolf,
Cleber Rosa, John Snow
Presently we use hxtool and a .hx format to generate a few things like
the qemu_img subcommand dispatch table, the qemu_img help() display output,
and a help output in qemu-img.texi.
Unfortunately, this means that this information is duplicated in at least
three places:
(1) in qemu-img-cmds.hx as a human readable string
(2) in qemu-img-cmds.hx as a texi string
(3) in qemu-img.texi again, as a texi string
We can do a little better, though: all of these sources can be generated
from a single json file. Add a new small tool ("pxtool") that can do this.
This tool can at least handle generating (1) and (2) from the same source
without needing to reduplicate that information. Deduplicating (3) happens
in the next patch.
Notes:
- The json format was chosen to be very "stupid", and the command line
documentation is being kept one-per-line to make future diffs easier
to read.
- It's easy enough to generate the human-readable output from the texi
output by removing '@var{foo}' with 'foo'.
- The qemu-img command dispatch always calls img_cmdname, so we don't
need to store this information separately, either.
- The need for DEF() macros could be removed as well, but I left it in
to create a minimally disruptive patch.
Signed-off-by: John Snow <jsnow@redhat.com>
---
Makefile | 8 +--
qemu-img-cmds.json | 165 +++++++++++++++++++++++++++++++++++++++++++++
qemu-img-cmds.hx | 102 ----------------------------
scripts/pxtool.py | 49 ++++++++++++++
4 files changed, 218 insertions(+), 106 deletions(-)
create mode 100644 qemu-img-cmds.json
delete mode 100644 qemu-img-cmds.hx
create mode 100755 scripts/pxtool.py
diff --git a/Makefile b/Makefile
index 04a0d45050..d203bb90dc 100644
--- a/Makefile
+++ b/Makefile
@@ -524,8 +524,8 @@ ifdef CONFIG_MPATH
scsi/qemu-pr-helper$(EXESUF): LIBS += -ludev -lmultipath -lmpathpersist
endif
-qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool
- $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > $@,"GEN","$@")
+qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.json $(SRC_PATH)/scripts/pxtool.py
+ $(call quiet-command,$(PYTOHN) $(SRC_PATH)/scripts/pxtool.py --macros $< > $@,"GEN","$@")
qemu-ga$(EXESUF): LIBS = $(LIBS_QGA)
qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated
@@ -918,8 +918,8 @@ qemu-monitor.texi: $(SRC_PATH)/hmp-commands.hx $(SRC_PATH)/scripts/hxtool
qemu-monitor-info.texi: $(SRC_PATH)/hmp-commands-info.hx $(SRC_PATH)/scripts/hxtool
$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"GEN","$@")
-qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool
- $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"GEN","$@")
+qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.json $(SRC_PATH)/scripts/pxtool.py
+ $(call quiet-command,$(PYTON) $(SRC_PATH)/scripts/pxtool.py --texi $< > $@,"GEN","$@")
docs/interop/qemu-qmp-qapi.texi: qapi/qapi-doc.texi
@cp -p $< $@
diff --git a/qemu-img-cmds.json b/qemu-img-cmds.json
new file mode 100644
index 0000000000..35d96f92a3
--- /dev/null
+++ b/qemu-img-cmds.json
@@ -0,0 +1,165 @@
+{
+ "amend": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-p]",
+ "[-q]",
+ "[-f @var{fmt}]",
+ "[-t @var{cache}]",
+ "-o @var{options}",
+ "@var{filename}"
+ ],
+ "bench": [
+ "[-c @var{count}]",
+ "[-d @var{depth}]",
+ "[-f @var{fmt}]",
+ "[--flush-interval=@var{flush_interval}]",
+ "[-n]",
+ "[--no-drain]",
+ "[-o @var{offset}]",
+ "[--pattern=@var{pattern}]",
+ "[-q]",
+ "[-s @var{buffer_size}]",
+ "[-S @var{step_size}]",
+ "[-t @var{cache}]",
+ "[-w]",
+ "[-U]",
+ "@var{filename}"
+ ],
+ "check": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-q]",
+ "[-f @var{fmt}]",
+ "[--output=@var{ofmt}]",
+ "[-r [leaks | all]]",
+ "[-T @var{src_cache}]",
+ "[-U]",
+ "@var{filename}"
+ ],
+ "commit": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-q]",
+ "[-f @var{fmt}]",
+ "[-t @var{cache}]",
+ "[-b @var{base}]",
+ "[-d]",
+ "[-p]",
+ "@var{filename}"
+ ],
+ "compare": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-f @var{fmt}]",
+ "[-F @var{fmt}]",
+ "[-T @var{src_cache}]",
+ "[-p]",
+ "[-q]",
+ "[-s]",
+ "[-U]",
+ "@var{filename1}",
+ "@var{filename2}"
+ ],
+ "convert": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[--target-image-opts]",
+ "[-U]",
+ "[-C]",
+ "[-c]",
+ "[-p]",
+ "[-q]",
+ "[-n]",
+ "[-f @var{fmt}]",
+ "[-t @var{cache}]",
+ "[-T @var{src_cache}]",
+ "[-O @var{output_fmt}]",
+ "[-B @var{backing_file}]",
+ "[-o @var{options}]",
+ "[-l @var{snapshot_param}]",
+ "[-S @var{sparse_size}]",
+ "[-m @var{num_coroutines}]",
+ "[-W]",
+ "@var{filename}",
+ "[@var{filename2} [...]]",
+ "@var{output_filename}"
+ ],
+ "create": [
+ "[--object @var{objectdef}]",
+ "[-q]",
+ "[-f @var{fmt}]",
+ "[-b @var{backing_file}]",
+ "[-F @var{backing_fmt}]",
+ "[-u]",
+ "[-o @var{options}]",
+ "@var{filename}",
+ "[@var{size}]"
+ ],
+ "dd": [
+ "[--image-opts]",
+ "[-U]",
+ "[-f @var{fmt}]",
+ "[-O @var{output_fmt}]",
+ "[bs=@var{block_size}]",
+ "[count=@var{blocks}]",
+ "[skip=@var{blocks}]",
+ "if=@var{input}",
+ "of=@var{output}"
+ ],
+ "info": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-f @var{fmt}]",
+ "[--output=@var{ofmt}]",
+ "[--backing-chain]",
+ "[-U]",
+ "@var{filename}"
+ ],
+ "map": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-f @var{fmt}]",
+ "[--output=@var{ofmt}]",
+ "[-U]",
+ "@var{filename}"
+ ],
+ "measure": [
+ "[--output=@var{ofmt}]",
+ "[-O @var{output_fmt}]",
+ "[-o @var{options}]",
+ "[--size @var{N} | [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [-l @var{snapshot_param}] @var{filename}]"
+ ],
+ "snapshot": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-U]",
+ "[-q]",
+ "[-l | -a @var{snapshot} | -c @var{snapshot} | -d @var{snapshot}]",
+ "@var{filename}"
+ ],
+ "rebase": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-U]",
+ "[-q]",
+ "[-f @var{fmt}]",
+ "[-t @var{cache}]",
+ "[-T @var{src_cache}]",
+ "[-p]",
+ "[-u]",
+ "-b @var{backing_file}",
+ "[-F @var{backing_fmt}]",
+ "@var{filename}"
+ ],
+ "resize": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-f @var{fmt}]",
+ "[--preallocation=@var{prealloc}]",
+ "[-q]",
+ "[--shrink]",
+ "@var{filename}",
+ "[+ | -]@var{size}"
+ ]
+}
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
deleted file mode 100644
index 4b47f7495d..0000000000
--- a/qemu-img-cmds.hx
+++ /dev/null
@@ -1,102 +0,0 @@
-HXCOMM Keep the list of subcommands sorted by name.
-HXCOMM Use DEFHEADING() to define headings in both help text and texi
-HXCOMM Text between STEXI and ETEXI are copied to texi version and
-HXCOMM discarded from C version
-HXCOMM DEF(command, callback, arg_string) is used to construct
-HXCOMM command structures and help message.
-HXCOMM HXCOMM can be used for comments, discarded from both texi and C
-
-HXCOMM When amending the TEXI sections, please remember to copy the usage
-HXCOMM over to the per-command sections in qemu-img.texi.
-
-STEXI
-@table @option
-ETEXI
-
-DEF("amend", img_amend,
- "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] -o options filename")
-STEXI
-@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
-ETEXI
-
-DEF("bench", img_bench,
- "bench [-c count] [-d depth] [-f fmt] [--flush-interval=flush_interval] [-n] [--no-drain] [-o offset] [--pattern=pattern] [-q] [-s buffer_size] [-S step_size] [-t cache] [-w] [-U] filename")
-STEXI
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] [-U] @var{filename}
-ETEXI
-
-DEF("check", img_check,
- "check [--object objectdef] [--image-opts] [-q] [-f fmt] [--output=ofmt] [-r [leaks | all]] [-T src_cache] [-U] filename")
-STEXI
-@item check [--object @var{objectdef}] [--image-opts] [-q] [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] [-T @var{src_cache}] [-U] @var{filename}
-ETEXI
-
-DEF("commit", img_commit,
- "commit [--object objectdef] [--image-opts] [-q] [-f fmt] [-t cache] [-b base] [-d] [-p] filename")
-STEXI
-@item commit [--object @var{objectdef}] [--image-opts] [-q] [-f @var{fmt}] [-t @var{cache}] [-b @var{base}] [-d] [-p] @var{filename}
-ETEXI
-
-DEF("compare", img_compare,
- "compare [--object objectdef] [--image-opts] [-f fmt] [-F fmt] [-T src_cache] [-p] [-q] [-s] [-U] filename1 filename2")
-STEXI
-@item compare [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [-F @var{fmt}] [-T @var{src_cache}] [-p] [-q] [-s] [-U] @var{filename1} @var{filename2}
-ETEXI
-
-DEF("convert", img_convert,
- "convert [--object objectdef] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] filename [filename2 [...]] output_filename")
-STEXI
-@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m @var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] @var{output_filename}
-ETEXI
-
-DEF("create", img_create,
- "create [--object objectdef] [-q] [-f fmt] [-b backing_file] [-F backing_fmt] [-u] [-o options] filename [size]")
-STEXI
-@item create [--object @var{objectdef}] [-q] [-f @var{fmt}] [-b @var{backing_file}] [-F @var{backing_fmt}] [-u] [-o @var{options}] @var{filename} [@var{size}]
-ETEXI
-
-DEF("dd", img_dd,
- "dd [--image-opts] [-U] [-f fmt] [-O output_fmt] [bs=block_size] [count=blocks] [skip=blocks] if=input of=output")
-STEXI
-@item dd [--image-opts] [-U] [-f @var{fmt}] [-O @var{output_fmt}] [bs=@var{block_size}] [count=@var{blocks}] [skip=@var{blocks}] if=@var{input} of=@var{output}
-ETEXI
-
-DEF("info", img_info,
- "info [--object objectdef] [--image-opts] [-f fmt] [--output=ofmt] [--backing-chain] [-U] filename")
-STEXI
-@item info [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--output=@var{ofmt}] [--backing-chain] [-U] @var{filename}
-ETEXI
-
-DEF("map", img_map,
- "map [--object objectdef] [--image-opts] [-f fmt] [--output=ofmt] [-U] filename")
-STEXI
-@item map [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--output=@var{ofmt}] [-U] @var{filename}
-ETEXI
-
-DEF("measure", img_measure,
-"measure [--output=ofmt] [-O output_fmt] [-o options] [--size N | [--object objectdef] [--image-opts] [-f fmt] [-l snapshot_param] filename]")
-STEXI
-@item measure [--output=@var{ofmt}] [-O @var{output_fmt}] [-o @var{options}] [--size @var{N} | [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [-l @var{snapshot_param}] @var{filename}]
-ETEXI
-
-DEF("snapshot", img_snapshot,
- "snapshot [--object objectdef] [--image-opts] [-U] [-q] [-l | -a snapshot | -c snapshot | -d snapshot] filename")
-STEXI
-@item snapshot [--object @var{objectdef}] [--image-opts] [-U] [-q] [-l | -a @var{snapshot} | -c @var{snapshot} | -d @var{snapshot}] @var{filename}
-ETEXI
-
-DEF("rebase", img_rebase,
- "rebase [--object objectdef] [--image-opts] [-U] [-q] [-f fmt] [-t cache] [-T src_cache] [-p] [-u] -b backing_file [-F backing_fmt] filename")
-STEXI
-@item rebase [--object @var{objectdef}] [--image-opts] [-U] [-q] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-p] [-u] -b @var{backing_file} [-F @var{backing_fmt}] @var{filename}
-ETEXI
-
-DEF("resize", img_resize,
- "resize [--object objectdef] [--image-opts] [-f fmt] [--preallocation=prealloc] [-q] [--shrink] filename [+ | -]size")
-STEXI
-@item resize [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--preallocation=@var{prealloc}] [-q] [--shrink] @var{filename} [+ | -]@var{size}
-ETEXI
-
-STEXI
-@end table
-ETEXI
diff --git a/scripts/pxtool.py b/scripts/pxtool.py
new file mode 100755
index 0000000000..008fea839e
--- /dev/null
+++ b/scripts/pxtool.py
@@ -0,0 +1,49 @@
+#!/usr/bin/env python
+
+import argparse
+from collections import OrderedDict
+import json
+import re
+
+# Helpers:
+def scrub_texi_string(tstring):
+ return re.sub(r"@var{([^{}]*?)}", r"\1", tstring)
+
+def human_readable_usage(usage_list):
+ return scrub_texi_string(" ".join(usage_list))
+
+def callback_name(command):
+ return "img_{:s}".format(command)
+
+# Output modes:
+def generate_def_header(conf):
+ """Print DEF() macros to be used by qemu-img.c"""
+ for command, usage_strs in conf.items():
+ print("DEF(\"{}\", {}, \"{} {}\")".format(
+ command,
+ callback_name(command),
+ command,
+ human_readable_usage(usage_strs)))
+
+def generate_texi(conf):
+ """Generate texi command summary table"""
+ print("@table @option")
+ for command, usage_strs in conf.items():
+ usage = " ".join(usage_strs)
+ print("@item {} {}".format(command, usage))
+ print("@end table")
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser(description="Generate qemu-img command information")
+ parser.add_argument("--macros", action="store_true", dest="macros")
+ parser.add_argument("--texi", action="store_true")
+ parser.add_argument("commands_json")
+ args = parser.parse_args()
+
+ with open(args.commands_json, "r") as f:
+ conf = json.load(f, object_pairs_hook=OrderedDict)
+
+ if args.macros:
+ generate_def_header(conf)
+ if args.texi:
+ generate_texi(conf)
--
2.17.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool
@ 2019-04-10 1:24 ` John Snow
0 siblings, 0 replies; 17+ messages in thread
From: John Snow @ 2019-04-10 1:24 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: Kevin Wolf, Eduardo Habkost, John Snow, armbru, Max Reitz, Cleber Rosa
Presently we use hxtool and a .hx format to generate a few things like
the qemu_img subcommand dispatch table, the qemu_img help() display output,
and a help output in qemu-img.texi.
Unfortunately, this means that this information is duplicated in at least
three places:
(1) in qemu-img-cmds.hx as a human readable string
(2) in qemu-img-cmds.hx as a texi string
(3) in qemu-img.texi again, as a texi string
We can do a little better, though: all of these sources can be generated
from a single json file. Add a new small tool ("pxtool") that can do this.
This tool can at least handle generating (1) and (2) from the same source
without needing to reduplicate that information. Deduplicating (3) happens
in the next patch.
Notes:
- The json format was chosen to be very "stupid", and the command line
documentation is being kept one-per-line to make future diffs easier
to read.
- It's easy enough to generate the human-readable output from the texi
output by removing '@var{foo}' with 'foo'.
- The qemu-img command dispatch always calls img_cmdname, so we don't
need to store this information separately, either.
- The need for DEF() macros could be removed as well, but I left it in
to create a minimally disruptive patch.
Signed-off-by: John Snow <jsnow@redhat.com>
---
Makefile | 8 +--
qemu-img-cmds.json | 165 +++++++++++++++++++++++++++++++++++++++++++++
qemu-img-cmds.hx | 102 ----------------------------
scripts/pxtool.py | 49 ++++++++++++++
4 files changed, 218 insertions(+), 106 deletions(-)
create mode 100644 qemu-img-cmds.json
delete mode 100644 qemu-img-cmds.hx
create mode 100755 scripts/pxtool.py
diff --git a/Makefile b/Makefile
index 04a0d45050..d203bb90dc 100644
--- a/Makefile
+++ b/Makefile
@@ -524,8 +524,8 @@ ifdef CONFIG_MPATH
scsi/qemu-pr-helper$(EXESUF): LIBS += -ludev -lmultipath -lmpathpersist
endif
-qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool
- $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -h < $< > $@,"GEN","$@")
+qemu-img-cmds.h: $(SRC_PATH)/qemu-img-cmds.json $(SRC_PATH)/scripts/pxtool.py
+ $(call quiet-command,$(PYTOHN) $(SRC_PATH)/scripts/pxtool.py --macros $< > $@,"GEN","$@")
qemu-ga$(EXESUF): LIBS = $(LIBS_QGA)
qemu-ga$(EXESUF): QEMU_CFLAGS += -I qga/qapi-generated
@@ -918,8 +918,8 @@ qemu-monitor.texi: $(SRC_PATH)/hmp-commands.hx $(SRC_PATH)/scripts/hxtool
qemu-monitor-info.texi: $(SRC_PATH)/hmp-commands-info.hx $(SRC_PATH)/scripts/hxtool
$(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"GEN","$@")
-qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.hx $(SRC_PATH)/scripts/hxtool
- $(call quiet-command,sh $(SRC_PATH)/scripts/hxtool -t < $< > $@,"GEN","$@")
+qemu-img-cmds.texi: $(SRC_PATH)/qemu-img-cmds.json $(SRC_PATH)/scripts/pxtool.py
+ $(call quiet-command,$(PYTON) $(SRC_PATH)/scripts/pxtool.py --texi $< > $@,"GEN","$@")
docs/interop/qemu-qmp-qapi.texi: qapi/qapi-doc.texi
@cp -p $< $@
diff --git a/qemu-img-cmds.json b/qemu-img-cmds.json
new file mode 100644
index 0000000000..35d96f92a3
--- /dev/null
+++ b/qemu-img-cmds.json
@@ -0,0 +1,165 @@
+{
+ "amend": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-p]",
+ "[-q]",
+ "[-f @var{fmt}]",
+ "[-t @var{cache}]",
+ "-o @var{options}",
+ "@var{filename}"
+ ],
+ "bench": [
+ "[-c @var{count}]",
+ "[-d @var{depth}]",
+ "[-f @var{fmt}]",
+ "[--flush-interval=@var{flush_interval}]",
+ "[-n]",
+ "[--no-drain]",
+ "[-o @var{offset}]",
+ "[--pattern=@var{pattern}]",
+ "[-q]",
+ "[-s @var{buffer_size}]",
+ "[-S @var{step_size}]",
+ "[-t @var{cache}]",
+ "[-w]",
+ "[-U]",
+ "@var{filename}"
+ ],
+ "check": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-q]",
+ "[-f @var{fmt}]",
+ "[--output=@var{ofmt}]",
+ "[-r [leaks | all]]",
+ "[-T @var{src_cache}]",
+ "[-U]",
+ "@var{filename}"
+ ],
+ "commit": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-q]",
+ "[-f @var{fmt}]",
+ "[-t @var{cache}]",
+ "[-b @var{base}]",
+ "[-d]",
+ "[-p]",
+ "@var{filename}"
+ ],
+ "compare": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-f @var{fmt}]",
+ "[-F @var{fmt}]",
+ "[-T @var{src_cache}]",
+ "[-p]",
+ "[-q]",
+ "[-s]",
+ "[-U]",
+ "@var{filename1}",
+ "@var{filename2}"
+ ],
+ "convert": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[--target-image-opts]",
+ "[-U]",
+ "[-C]",
+ "[-c]",
+ "[-p]",
+ "[-q]",
+ "[-n]",
+ "[-f @var{fmt}]",
+ "[-t @var{cache}]",
+ "[-T @var{src_cache}]",
+ "[-O @var{output_fmt}]",
+ "[-B @var{backing_file}]",
+ "[-o @var{options}]",
+ "[-l @var{snapshot_param}]",
+ "[-S @var{sparse_size}]",
+ "[-m @var{num_coroutines}]",
+ "[-W]",
+ "@var{filename}",
+ "[@var{filename2} [...]]",
+ "@var{output_filename}"
+ ],
+ "create": [
+ "[--object @var{objectdef}]",
+ "[-q]",
+ "[-f @var{fmt}]",
+ "[-b @var{backing_file}]",
+ "[-F @var{backing_fmt}]",
+ "[-u]",
+ "[-o @var{options}]",
+ "@var{filename}",
+ "[@var{size}]"
+ ],
+ "dd": [
+ "[--image-opts]",
+ "[-U]",
+ "[-f @var{fmt}]",
+ "[-O @var{output_fmt}]",
+ "[bs=@var{block_size}]",
+ "[count=@var{blocks}]",
+ "[skip=@var{blocks}]",
+ "if=@var{input}",
+ "of=@var{output}"
+ ],
+ "info": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-f @var{fmt}]",
+ "[--output=@var{ofmt}]",
+ "[--backing-chain]",
+ "[-U]",
+ "@var{filename}"
+ ],
+ "map": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-f @var{fmt}]",
+ "[--output=@var{ofmt}]",
+ "[-U]",
+ "@var{filename}"
+ ],
+ "measure": [
+ "[--output=@var{ofmt}]",
+ "[-O @var{output_fmt}]",
+ "[-o @var{options}]",
+ "[--size @var{N} | [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [-l @var{snapshot_param}] @var{filename}]"
+ ],
+ "snapshot": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-U]",
+ "[-q]",
+ "[-l | -a @var{snapshot} | -c @var{snapshot} | -d @var{snapshot}]",
+ "@var{filename}"
+ ],
+ "rebase": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-U]",
+ "[-q]",
+ "[-f @var{fmt}]",
+ "[-t @var{cache}]",
+ "[-T @var{src_cache}]",
+ "[-p]",
+ "[-u]",
+ "-b @var{backing_file}",
+ "[-F @var{backing_fmt}]",
+ "@var{filename}"
+ ],
+ "resize": [
+ "[--object @var{objectdef}]",
+ "[--image-opts]",
+ "[-f @var{fmt}]",
+ "[--preallocation=@var{prealloc}]",
+ "[-q]",
+ "[--shrink]",
+ "@var{filename}",
+ "[+ | -]@var{size}"
+ ]
+}
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
deleted file mode 100644
index 4b47f7495d..0000000000
--- a/qemu-img-cmds.hx
+++ /dev/null
@@ -1,102 +0,0 @@
-HXCOMM Keep the list of subcommands sorted by name.
-HXCOMM Use DEFHEADING() to define headings in both help text and texi
-HXCOMM Text between STEXI and ETEXI are copied to texi version and
-HXCOMM discarded from C version
-HXCOMM DEF(command, callback, arg_string) is used to construct
-HXCOMM command structures and help message.
-HXCOMM HXCOMM can be used for comments, discarded from both texi and C
-
-HXCOMM When amending the TEXI sections, please remember to copy the usage
-HXCOMM over to the per-command sections in qemu-img.texi.
-
-STEXI
-@table @option
-ETEXI
-
-DEF("amend", img_amend,
- "amend [--object objectdef] [--image-opts] [-p] [-q] [-f fmt] [-t cache] -o options filename")
-STEXI
-@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
-ETEXI
-
-DEF("bench", img_bench,
- "bench [-c count] [-d depth] [-f fmt] [--flush-interval=flush_interval] [-n] [--no-drain] [-o offset] [--pattern=pattern] [-q] [-s buffer_size] [-S step_size] [-t cache] [-w] [-U] filename")
-STEXI
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] [-U] @var{filename}
-ETEXI
-
-DEF("check", img_check,
- "check [--object objectdef] [--image-opts] [-q] [-f fmt] [--output=ofmt] [-r [leaks | all]] [-T src_cache] [-U] filename")
-STEXI
-@item check [--object @var{objectdef}] [--image-opts] [-q] [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] [-T @var{src_cache}] [-U] @var{filename}
-ETEXI
-
-DEF("commit", img_commit,
- "commit [--object objectdef] [--image-opts] [-q] [-f fmt] [-t cache] [-b base] [-d] [-p] filename")
-STEXI
-@item commit [--object @var{objectdef}] [--image-opts] [-q] [-f @var{fmt}] [-t @var{cache}] [-b @var{base}] [-d] [-p] @var{filename}
-ETEXI
-
-DEF("compare", img_compare,
- "compare [--object objectdef] [--image-opts] [-f fmt] [-F fmt] [-T src_cache] [-p] [-q] [-s] [-U] filename1 filename2")
-STEXI
-@item compare [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [-F @var{fmt}] [-T @var{src_cache}] [-p] [-q] [-s] [-U] @var{filename1} @var{filename2}
-ETEXI
-
-DEF("convert", img_convert,
- "convert [--object objectdef] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f fmt] [-t cache] [-T src_cache] [-O output_fmt] [-B backing_file] [-o options] [-l snapshot_param] [-S sparse_size] [-m num_coroutines] [-W] filename [filename2 [...]] output_filename")
-STEXI
-@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m @var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] @var{output_filename}
-ETEXI
-
-DEF("create", img_create,
- "create [--object objectdef] [-q] [-f fmt] [-b backing_file] [-F backing_fmt] [-u] [-o options] filename [size]")
-STEXI
-@item create [--object @var{objectdef}] [-q] [-f @var{fmt}] [-b @var{backing_file}] [-F @var{backing_fmt}] [-u] [-o @var{options}] @var{filename} [@var{size}]
-ETEXI
-
-DEF("dd", img_dd,
- "dd [--image-opts] [-U] [-f fmt] [-O output_fmt] [bs=block_size] [count=blocks] [skip=blocks] if=input of=output")
-STEXI
-@item dd [--image-opts] [-U] [-f @var{fmt}] [-O @var{output_fmt}] [bs=@var{block_size}] [count=@var{blocks}] [skip=@var{blocks}] if=@var{input} of=@var{output}
-ETEXI
-
-DEF("info", img_info,
- "info [--object objectdef] [--image-opts] [-f fmt] [--output=ofmt] [--backing-chain] [-U] filename")
-STEXI
-@item info [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--output=@var{ofmt}] [--backing-chain] [-U] @var{filename}
-ETEXI
-
-DEF("map", img_map,
- "map [--object objectdef] [--image-opts] [-f fmt] [--output=ofmt] [-U] filename")
-STEXI
-@item map [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--output=@var{ofmt}] [-U] @var{filename}
-ETEXI
-
-DEF("measure", img_measure,
-"measure [--output=ofmt] [-O output_fmt] [-o options] [--size N | [--object objectdef] [--image-opts] [-f fmt] [-l snapshot_param] filename]")
-STEXI
-@item measure [--output=@var{ofmt}] [-O @var{output_fmt}] [-o @var{options}] [--size @var{N} | [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [-l @var{snapshot_param}] @var{filename}]
-ETEXI
-
-DEF("snapshot", img_snapshot,
- "snapshot [--object objectdef] [--image-opts] [-U] [-q] [-l | -a snapshot | -c snapshot | -d snapshot] filename")
-STEXI
-@item snapshot [--object @var{objectdef}] [--image-opts] [-U] [-q] [-l | -a @var{snapshot} | -c @var{snapshot} | -d @var{snapshot}] @var{filename}
-ETEXI
-
-DEF("rebase", img_rebase,
- "rebase [--object objectdef] [--image-opts] [-U] [-q] [-f fmt] [-t cache] [-T src_cache] [-p] [-u] -b backing_file [-F backing_fmt] filename")
-STEXI
-@item rebase [--object @var{objectdef}] [--image-opts] [-U] [-q] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-p] [-u] -b @var{backing_file} [-F @var{backing_fmt}] @var{filename}
-ETEXI
-
-DEF("resize", img_resize,
- "resize [--object objectdef] [--image-opts] [-f fmt] [--preallocation=prealloc] [-q] [--shrink] filename [+ | -]size")
-STEXI
-@item resize [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--preallocation=@var{prealloc}] [-q] [--shrink] @var{filename} [+ | -]@var{size}
-ETEXI
-
-STEXI
-@end table
-ETEXI
diff --git a/scripts/pxtool.py b/scripts/pxtool.py
new file mode 100755
index 0000000000..008fea839e
--- /dev/null
+++ b/scripts/pxtool.py
@@ -0,0 +1,49 @@
+#!/usr/bin/env python
+
+import argparse
+from collections import OrderedDict
+import json
+import re
+
+# Helpers:
+def scrub_texi_string(tstring):
+ return re.sub(r"@var{([^{}]*?)}", r"\1", tstring)
+
+def human_readable_usage(usage_list):
+ return scrub_texi_string(" ".join(usage_list))
+
+def callback_name(command):
+ return "img_{:s}".format(command)
+
+# Output modes:
+def generate_def_header(conf):
+ """Print DEF() macros to be used by qemu-img.c"""
+ for command, usage_strs in conf.items():
+ print("DEF(\"{}\", {}, \"{} {}\")".format(
+ command,
+ callback_name(command),
+ command,
+ human_readable_usage(usage_strs)))
+
+def generate_texi(conf):
+ """Generate texi command summary table"""
+ print("@table @option")
+ for command, usage_strs in conf.items():
+ usage = " ".join(usage_strs)
+ print("@item {} {}".format(command, usage))
+ print("@end table")
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser(description="Generate qemu-img command information")
+ parser.add_argument("--macros", action="store_true", dest="macros")
+ parser.add_argument("--texi", action="store_true")
+ parser.add_argument("commands_json")
+ args = parser.parse_args()
+
+ with open(args.commands_json, "r") as f:
+ conf = json.load(f, object_pairs_hook=OrderedDict)
+
+ if args.macros:
+ generate_def_header(conf)
+ if args.texi:
+ generate_texi(conf)
--
2.17.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH RFC 3/3] qemu-img.texi: use macros for command summaries
@ 2019-04-10 1:24 ` John Snow
0 siblings, 0 replies; 17+ messages in thread
From: John Snow @ 2019-04-10 1:24 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: armbru, eblake, Eduardo Habkost, Max Reitz, Kevin Wolf,
Cleber Rosa, John Snow
Use a macro to recall this text instead of re-entering it manually.
Modify the pxtool script to create macros for each subcommand, then
alter the qemu-img.texi file to recall these macros.
Signed-off-by: John Snow <jsnow@redhat.com>
---
qemu-img.texi | 28 ++++++++++++++--------------
scripts/pxtool.py | 4 ++++
2 files changed, 18 insertions(+), 14 deletions(-)
diff --git a/qemu-img.texi b/qemu-img.texi
index 724f244ba1..c21a97e934 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -197,12 +197,12 @@ Command description:
@table @option
-@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
+@qemuimgcmdamend
Amends the image format specific @var{options} for the image file
@var{filename}. Not all file formats support this operation.
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] [-U] @var{filename}
+@qemuimgcmdbench
Run a simple sequential I/O benchmark on the specified image. If @code{-w} is
specified, a write test is performed, otherwise a read test is performed.
@@ -226,7 +226,7 @@ specified as well.
For write tests, by default a buffer filled with zeros is written. This can be
overridden with a pattern byte specified by @var{pattern}.
-@item check [--object @var{objectdef}] [--image-opts] [-q] [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] [-T @var{src_cache}] [-U] @var{filename}
+@qemuimgcmdcheck
Perform a consistency check on the disk image @var{filename}. The command can
output in the format @var{ofmt} which is either @code{human} or @code{json}.
@@ -262,7 +262,7 @@ If @code{-r} is specified, exit codes representing the image state refer to the
state after (the attempt at) repairing it. That is, a successful @code{-r all}
will yield the exit code 0, independently of the image state before.
-@item commit [--object @var{objectdef}] [--image-opts] [-q] [-f @var{fmt}] [-t @var{cache}] [-b @var{base}] [-d] [-p] @var{filename}
+@qemuimgcmdcommit
Commit the changes recorded in @var{filename} in its base image or backing file.
If the backing file is smaller than the snapshot, then the backing file will be
@@ -284,7 +284,7 @@ all images between @var{base} and the top image will be invalid and may return
garbage data when read. For this reason, @code{-b} implies @code{-d} (so that
the top image stays valid).
-@item compare [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [-F @var{fmt}] [-T @var{src_cache}] [-p] [-q] [-s] [-U] @var{filename1} @var{filename2}
+@qemuimgcmdcompare
Check if two images have the same content. You can compare images with
different format or settings.
@@ -325,7 +325,7 @@ Error on reading data
@end table
-@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m @var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] @var{output_filename}
+@qemuimgcmdconvert
Convert the disk image @var{filename} or a snapshot @var{snapshot_param}
to disk image @var{output_filename} using format @var{output_fmt}. It can be optionally compressed (@code{-c}
@@ -366,7 +366,7 @@ creating compressed images.
@var{num_coroutines} specifies how many coroutines work in parallel during
the convert process (defaults to 8).
-@item create [--object @var{objectdef}] [-q] [-f @var{fmt}] [-b @var{backing_file}] [-F @var{backing_fmt}] [-u] [-o @var{options}] @var{filename} [@var{size}]
+@qemuimgcmdcreate
Create the new disk image @var{filename} of size @var{size} and format
@var{fmt}. Depending on the file format, you can add one or more @var{options}
@@ -390,7 +390,7 @@ way.
The size can also be specified using the @var{size} option with @code{-o},
it doesn't need to be specified separately in this case.
-@item dd [--image-opts] [-U] [-f @var{fmt}] [-O @var{output_fmt}] [bs=@var{block_size}] [count=@var{blocks}] [skip=@var{blocks}] if=@var{input} of=@var{output}
+@qemuimgcmddd
Dd copies from @var{input} file to @var{output} file converting it from
@var{fmt} format to @var{output_fmt} format.
@@ -401,7 +401,7 @@ dd will stop reading input after reading @var{blocks} input blocks.
The size syntax is similar to dd(1)'s size syntax.
-@item info [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--output=@var{ofmt}] [--backing-chain] [-U] @var{filename}
+@qemuimgcmdinfo
Give information about the disk image @var{filename}. Use it in
particular to know the size reserved on disk which can be different
@@ -424,7 +424,7 @@ To enumerate information about each disk image in the above chain, starting from
qemu-img info --backing-chain snap2.qcow2
@end example
-@item map [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--output=@var{ofmt}] [-U] @var{filename}
+@qemuimgcmdmap
Dump the metadata of image @var{filename} and its backing file chain.
In particular, this commands dumps the allocation state of every sector
@@ -479,7 +479,7 @@ preallocated.
For more information, consult @file{include/block/block.h} in QEMU's
source code.
-@item measure [--output=@var{ofmt}] [-O @var{output_fmt}] [-o @var{options}] [--size @var{N} | [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [-l @var{snapshot_param}] @var{filename}]
+@qemuimgcmdmeasure
Calculate the file size required for a new image. This information can be used
to size logical volumes or SAN LUNs appropriately for the image that will be
@@ -509,11 +509,11 @@ been written to all sectors. This is the maximum size that the image file can
occupy with the exception of internal snapshots, dirty bitmaps, vmstate data,
and other advanced image format features.
-@item snapshot [--object @var{objectdef}] [--image-opts] [-U] [-q] [-l | -a @var{snapshot} | -c @var{snapshot} | -d @var{snapshot}] @var{filename}
+@qemuimgcmdsnapshot
List, apply, create or delete snapshots in image @var{filename}.
-@item rebase [--object @var{objectdef}] [--image-opts] [-U] [-q] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-p] [-u] -b @var{backing_file} [-F @var{backing_fmt}] @var{filename}
+@qemuimgcmdrebase
Changes the backing file of an image. Only the formats @code{qcow2} and
@code{qed} support changing the backing file.
@@ -573,7 +573,7 @@ qemu-img rebase -b base.img diff.qcow2
At this point, @code{modified.img} can be discarded, since
@code{base.img + diff.qcow2} contains the same information.
-@item resize [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--preallocation=@var{prealloc}] [-q] [--shrink] @var{filename} [+ | -]@var{size}
+@qemuimgcmdresize
Change the disk image as if it had been created with @var{size}.
diff --git a/scripts/pxtool.py b/scripts/pxtool.py
index 008fea839e..7ed5550b29 100755
--- a/scripts/pxtool.py
+++ b/scripts/pxtool.py
@@ -30,7 +30,11 @@ def generate_texi(conf):
print("@table @option")
for command, usage_strs in conf.items():
usage = " ".join(usage_strs)
+ macro = "qemuimgcmd{}".format(command)
+ print("@macro {}".format(macro))
print("@item {} {}".format(command, usage))
+ print("@end macro")
+ print("@{}".format(macro))
print("@end table")
if __name__ == '__main__':
--
2.17.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* [Qemu-devel] [PATCH RFC 3/3] qemu-img.texi: use macros for command summaries
@ 2019-04-10 1:24 ` John Snow
0 siblings, 0 replies; 17+ messages in thread
From: John Snow @ 2019-04-10 1:24 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: Kevin Wolf, Eduardo Habkost, John Snow, armbru, Max Reitz, Cleber Rosa
Use a macro to recall this text instead of re-entering it manually.
Modify the pxtool script to create macros for each subcommand, then
alter the qemu-img.texi file to recall these macros.
Signed-off-by: John Snow <jsnow@redhat.com>
---
qemu-img.texi | 28 ++++++++++++++--------------
scripts/pxtool.py | 4 ++++
2 files changed, 18 insertions(+), 14 deletions(-)
diff --git a/qemu-img.texi b/qemu-img.texi
index 724f244ba1..c21a97e934 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -197,12 +197,12 @@ Command description:
@table @option
-@item amend [--object @var{objectdef}] [--image-opts] [-p] [-q] [-f @var{fmt}] [-t @var{cache}] -o @var{options} @var{filename}
+@qemuimgcmdamend
Amends the image format specific @var{options} for the image file
@var{filename}. Not all file formats support this operation.
-@item bench [-c @var{count}] [-d @var{depth}] [-f @var{fmt}] [--flush-interval=@var{flush_interval}] [-n] [--no-drain] [-o @var{offset}] [--pattern=@var{pattern}] [-q] [-s @var{buffer_size}] [-S @var{step_size}] [-t @var{cache}] [-w] [-U] @var{filename}
+@qemuimgcmdbench
Run a simple sequential I/O benchmark on the specified image. If @code{-w} is
specified, a write test is performed, otherwise a read test is performed.
@@ -226,7 +226,7 @@ specified as well.
For write tests, by default a buffer filled with zeros is written. This can be
overridden with a pattern byte specified by @var{pattern}.
-@item check [--object @var{objectdef}] [--image-opts] [-q] [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] [-T @var{src_cache}] [-U] @var{filename}
+@qemuimgcmdcheck
Perform a consistency check on the disk image @var{filename}. The command can
output in the format @var{ofmt} which is either @code{human} or @code{json}.
@@ -262,7 +262,7 @@ If @code{-r} is specified, exit codes representing the image state refer to the
state after (the attempt at) repairing it. That is, a successful @code{-r all}
will yield the exit code 0, independently of the image state before.
-@item commit [--object @var{objectdef}] [--image-opts] [-q] [-f @var{fmt}] [-t @var{cache}] [-b @var{base}] [-d] [-p] @var{filename}
+@qemuimgcmdcommit
Commit the changes recorded in @var{filename} in its base image or backing file.
If the backing file is smaller than the snapshot, then the backing file will be
@@ -284,7 +284,7 @@ all images between @var{base} and the top image will be invalid and may return
garbage data when read. For this reason, @code{-b} implies @code{-d} (so that
the top image stays valid).
-@item compare [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [-F @var{fmt}] [-T @var{src_cache}] [-p] [-q] [-s] [-U] @var{filename1} @var{filename2}
+@qemuimgcmdcompare
Check if two images have the same content. You can compare images with
different format or settings.
@@ -325,7 +325,7 @@ Error on reading data
@end table
-@item convert [--object @var{objectdef}] [--image-opts] [--target-image-opts] [-U] [-C] [-c] [-p] [-q] [-n] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-O @var{output_fmt}] [-B @var{backing_file}] [-o @var{options}] [-l @var{snapshot_param}] [-S @var{sparse_size}] [-m @var{num_coroutines}] [-W] @var{filename} [@var{filename2} [...]] @var{output_filename}
+@qemuimgcmdconvert
Convert the disk image @var{filename} or a snapshot @var{snapshot_param}
to disk image @var{output_filename} using format @var{output_fmt}. It can be optionally compressed (@code{-c}
@@ -366,7 +366,7 @@ creating compressed images.
@var{num_coroutines} specifies how many coroutines work in parallel during
the convert process (defaults to 8).
-@item create [--object @var{objectdef}] [-q] [-f @var{fmt}] [-b @var{backing_file}] [-F @var{backing_fmt}] [-u] [-o @var{options}] @var{filename} [@var{size}]
+@qemuimgcmdcreate
Create the new disk image @var{filename} of size @var{size} and format
@var{fmt}. Depending on the file format, you can add one or more @var{options}
@@ -390,7 +390,7 @@ way.
The size can also be specified using the @var{size} option with @code{-o},
it doesn't need to be specified separately in this case.
-@item dd [--image-opts] [-U] [-f @var{fmt}] [-O @var{output_fmt}] [bs=@var{block_size}] [count=@var{blocks}] [skip=@var{blocks}] if=@var{input} of=@var{output}
+@qemuimgcmddd
Dd copies from @var{input} file to @var{output} file converting it from
@var{fmt} format to @var{output_fmt} format.
@@ -401,7 +401,7 @@ dd will stop reading input after reading @var{blocks} input blocks.
The size syntax is similar to dd(1)'s size syntax.
-@item info [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--output=@var{ofmt}] [--backing-chain] [-U] @var{filename}
+@qemuimgcmdinfo
Give information about the disk image @var{filename}. Use it in
particular to know the size reserved on disk which can be different
@@ -424,7 +424,7 @@ To enumerate information about each disk image in the above chain, starting from
qemu-img info --backing-chain snap2.qcow2
@end example
-@item map [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--output=@var{ofmt}] [-U] @var{filename}
+@qemuimgcmdmap
Dump the metadata of image @var{filename} and its backing file chain.
In particular, this commands dumps the allocation state of every sector
@@ -479,7 +479,7 @@ preallocated.
For more information, consult @file{include/block/block.h} in QEMU's
source code.
-@item measure [--output=@var{ofmt}] [-O @var{output_fmt}] [-o @var{options}] [--size @var{N} | [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [-l @var{snapshot_param}] @var{filename}]
+@qemuimgcmdmeasure
Calculate the file size required for a new image. This information can be used
to size logical volumes or SAN LUNs appropriately for the image that will be
@@ -509,11 +509,11 @@ been written to all sectors. This is the maximum size that the image file can
occupy with the exception of internal snapshots, dirty bitmaps, vmstate data,
and other advanced image format features.
-@item snapshot [--object @var{objectdef}] [--image-opts] [-U] [-q] [-l | -a @var{snapshot} | -c @var{snapshot} | -d @var{snapshot}] @var{filename}
+@qemuimgcmdsnapshot
List, apply, create or delete snapshots in image @var{filename}.
-@item rebase [--object @var{objectdef}] [--image-opts] [-U] [-q] [-f @var{fmt}] [-t @var{cache}] [-T @var{src_cache}] [-p] [-u] -b @var{backing_file} [-F @var{backing_fmt}] @var{filename}
+@qemuimgcmdrebase
Changes the backing file of an image. Only the formats @code{qcow2} and
@code{qed} support changing the backing file.
@@ -573,7 +573,7 @@ qemu-img rebase -b base.img diff.qcow2
At this point, @code{modified.img} can be discarded, since
@code{base.img + diff.qcow2} contains the same information.
-@item resize [--object @var{objectdef}] [--image-opts] [-f @var{fmt}] [--preallocation=@var{prealloc}] [-q] [--shrink] @var{filename} [+ | -]@var{size}
+@qemuimgcmdresize
Change the disk image as if it had been created with @var{size}.
diff --git a/scripts/pxtool.py b/scripts/pxtool.py
index 008fea839e..7ed5550b29 100755
--- a/scripts/pxtool.py
+++ b/scripts/pxtool.py
@@ -30,7 +30,11 @@ def generate_texi(conf):
print("@table @option")
for command, usage_strs in conf.items():
usage = " ".join(usage_strs)
+ macro = "qemuimgcmd{}".format(command)
+ print("@macro {}".format(macro))
print("@item {} {}".format(command, usage))
+ print("@end macro")
+ print("@{}".format(macro))
print("@end table")
if __name__ == '__main__':
--
2.17.2
^ permalink raw reply related [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool
@ 2019-04-10 5:54 ` Markus Armbruster
0 siblings, 0 replies; 17+ messages in thread
From: Markus Armbruster @ 2019-04-10 5:54 UTC (permalink / raw)
To: John Snow
Cc: qemu-devel, qemu-block, Kevin Wolf, Eduardo Habkost, Max Reitz,
Cleber Rosa
John Snow <jsnow@redhat.com> writes:
> Presently we use hxtool and a .hx format to generate a few things like
> the qemu_img subcommand dispatch table, the qemu_img help() display output,
> and a help output in qemu-img.texi.
>
> Unfortunately, this means that this information is duplicated in at least
> three places:
>
> (1) in qemu-img-cmds.hx as a human readable string
> (2) in qemu-img-cmds.hx as a texi string
> (3) in qemu-img.texi again, as a texi string
>
> We can do a little better, though: all of these sources can be generated
> from a single json file. Add a new small tool ("pxtool") that can do this.
>
> This tool can at least handle generating (1) and (2) from the same source
> without needing to reduplicate that information. Deduplicating (3) happens
> in the next patch.
>
> Notes:
> - The json format was chosen to be very "stupid", and the command line
> documentation is being kept one-per-line to make future diffs easier
> to read.
> - It's easy enough to generate the human-readable output from the texi
> output by removing '@var{foo}' with 'foo'.
> - The qemu-img command dispatch always calls img_cmdname, so we don't
> need to store this information separately, either.
> - The need for DEF() macros could be removed as well, but I left it in
> to create a minimally disruptive patch.
> Signed-off-by: John Snow <jsnow@redhat.com>
We got just five .hx:
qemu-img.cmds.hx killed off by this patch
qemu-options.hx CLI QAPIfication should kill this off
hw/audio/pl041.hx misnamed, not actually food for hxtool
hmp-commands.hx no exit strategy
hmp-commands-info.hx for these two, yet
CLI QAPIfication got stuck in the back-burner, and as long as that's the
case, it's not an alternative to your patches.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool
@ 2019-04-10 5:54 ` Markus Armbruster
0 siblings, 0 replies; 17+ messages in thread
From: Markus Armbruster @ 2019-04-10 5:54 UTC (permalink / raw)
To: John Snow
Cc: Kevin Wolf, Eduardo Habkost, qemu-block, qemu-devel, Max Reitz,
Cleber Rosa
John Snow <jsnow@redhat.com> writes:
> Presently we use hxtool and a .hx format to generate a few things like
> the qemu_img subcommand dispatch table, the qemu_img help() display output,
> and a help output in qemu-img.texi.
>
> Unfortunately, this means that this information is duplicated in at least
> three places:
>
> (1) in qemu-img-cmds.hx as a human readable string
> (2) in qemu-img-cmds.hx as a texi string
> (3) in qemu-img.texi again, as a texi string
>
> We can do a little better, though: all of these sources can be generated
> from a single json file. Add a new small tool ("pxtool") that can do this.
>
> This tool can at least handle generating (1) and (2) from the same source
> without needing to reduplicate that information. Deduplicating (3) happens
> in the next patch.
>
> Notes:
> - The json format was chosen to be very "stupid", and the command line
> documentation is being kept one-per-line to make future diffs easier
> to read.
> - It's easy enough to generate the human-readable output from the texi
> output by removing '@var{foo}' with 'foo'.
> - The qemu-img command dispatch always calls img_cmdname, so we don't
> need to store this information separately, either.
> - The need for DEF() macros could be removed as well, but I left it in
> to create a minimally disruptive patch.
> Signed-off-by: John Snow <jsnow@redhat.com>
We got just five .hx:
qemu-img.cmds.hx killed off by this patch
qemu-options.hx CLI QAPIfication should kill this off
hw/audio/pl041.hx misnamed, not actually food for hxtool
hmp-commands.hx no exit strategy
hmp-commands-info.hx for these two, yet
CLI QAPIfication got stuck in the back-burner, and as long as that's the
case, it's not an alternative to your patches.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool
@ 2019-04-10 17:55 ` John Snow
0 siblings, 0 replies; 17+ messages in thread
From: John Snow @ 2019-04-10 17:55 UTC (permalink / raw)
To: Markus Armbruster
Cc: qemu-devel, qemu-block, Kevin Wolf, Eduardo Habkost, Max Reitz,
Cleber Rosa
On 4/10/19 1:54 AM, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
>
>> Presently we use hxtool and a .hx format to generate a few things like
>> the qemu_img subcommand dispatch table, the qemu_img help() display output,
>> and a help output in qemu-img.texi.
>>
>> Unfortunately, this means that this information is duplicated in at least
>> three places:
>>
>> (1) in qemu-img-cmds.hx as a human readable string
>> (2) in qemu-img-cmds.hx as a texi string
>> (3) in qemu-img.texi again, as a texi string
>>
>> We can do a little better, though: all of these sources can be generated
>> from a single json file. Add a new small tool ("pxtool") that can do this.
>>
>> This tool can at least handle generating (1) and (2) from the same source
>> without needing to reduplicate that information. Deduplicating (3) happens
>> in the next patch.
>>
>> Notes:
>> - The json format was chosen to be very "stupid", and the command line
>> documentation is being kept one-per-line to make future diffs easier
>> to read.
>> - It's easy enough to generate the human-readable output from the texi
>> output by removing '@var{foo}' with 'foo'.
>> - The qemu-img command dispatch always calls img_cmdname, so we don't
>> need to store this information separately, either.
>> - The need for DEF() macros could be removed as well, but I left it in
>> to create a minimally disruptive patch.
>> Signed-off-by: John Snow <jsnow@redhat.com>
>
> We got just five .hx:
>
> qemu-img.cmds.hx killed off by this patch
> qemu-options.hx CLI QAPIfication should kill this off
> hw/audio/pl041.hx misnamed, not actually food for hxtool
> hmp-commands.hx no exit strategy
> hmp-commands-info.hx for these two, yet
>
> CLI QAPIfication got stuck in the back-burner, and as long as that's the
> case, it's not an alternative to your patches.
>
Good to know. I figure that at least while we wait on something more
comprehensive there's no real short term harm in tidying up.
Something I'd really like to do is define a python/json-esque
configuration file that allows you to specify some "common options" that
are shared between all of the sub-commands, and then define each command
in terms of both which common options it possesses, and then any local
command-specific options it has.
(Why the common options design? So that argument names and command
options can be encouraged to be identical and identically documented
between all subcommands that use them.)
Then, from the configuration file, generate all of the necessary C
parsers (I have a protoype for this, it's not difficult to do in at
least the basic case) that can return boxed command arguments, and then
also generate the help strings from that metadata.
I suspect that work *IS* something that might brush up against / should
use (or extend) QAPI code.
Then, I'd like to find a way to split qemu-img.texi into sub-command
files and find a way to source the same "informative text" for both:
(1) The texi output, as per usual, and
(2) qemu-img subcommand --help
such that --help, when passed as an argument to the subcommand, prints
out help only relevant to the subcommand instead, e.g.
`qemu-img check --help`
might print the "common options" section only as it relates to commands
actually used by the check command, then prints the check section of the
texi as formatted for terminals.
This way, we can have manpages, html pages, and interactive help text
all derived from the same semi-automated sources, always up to date and
much easier to read/discover/parse by human eyeballs.
That's what I'd like to accomplish, ultimately.
For now, I think this RFC set is not harmful and won't bother anybody.
It's definitely not worse than what we have now, fragility of removing
@var{} tokens and all.
--js
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool
@ 2019-04-10 17:55 ` John Snow
0 siblings, 0 replies; 17+ messages in thread
From: John Snow @ 2019-04-10 17:55 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Eduardo Habkost, qemu-block, qemu-devel, Max Reitz,
Cleber Rosa
On 4/10/19 1:54 AM, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
>
>> Presently we use hxtool and a .hx format to generate a few things like
>> the qemu_img subcommand dispatch table, the qemu_img help() display output,
>> and a help output in qemu-img.texi.
>>
>> Unfortunately, this means that this information is duplicated in at least
>> three places:
>>
>> (1) in qemu-img-cmds.hx as a human readable string
>> (2) in qemu-img-cmds.hx as a texi string
>> (3) in qemu-img.texi again, as a texi string
>>
>> We can do a little better, though: all of these sources can be generated
>> from a single json file. Add a new small tool ("pxtool") that can do this.
>>
>> This tool can at least handle generating (1) and (2) from the same source
>> without needing to reduplicate that information. Deduplicating (3) happens
>> in the next patch.
>>
>> Notes:
>> - The json format was chosen to be very "stupid", and the command line
>> documentation is being kept one-per-line to make future diffs easier
>> to read.
>> - It's easy enough to generate the human-readable output from the texi
>> output by removing '@var{foo}' with 'foo'.
>> - The qemu-img command dispatch always calls img_cmdname, so we don't
>> need to store this information separately, either.
>> - The need for DEF() macros could be removed as well, but I left it in
>> to create a minimally disruptive patch.
>> Signed-off-by: John Snow <jsnow@redhat.com>
>
> We got just five .hx:
>
> qemu-img.cmds.hx killed off by this patch
> qemu-options.hx CLI QAPIfication should kill this off
> hw/audio/pl041.hx misnamed, not actually food for hxtool
> hmp-commands.hx no exit strategy
> hmp-commands-info.hx for these two, yet
>
> CLI QAPIfication got stuck in the back-burner, and as long as that's the
> case, it's not an alternative to your patches.
>
Good to know. I figure that at least while we wait on something more
comprehensive there's no real short term harm in tidying up.
Something I'd really like to do is define a python/json-esque
configuration file that allows you to specify some "common options" that
are shared between all of the sub-commands, and then define each command
in terms of both which common options it possesses, and then any local
command-specific options it has.
(Why the common options design? So that argument names and command
options can be encouraged to be identical and identically documented
between all subcommands that use them.)
Then, from the configuration file, generate all of the necessary C
parsers (I have a protoype for this, it's not difficult to do in at
least the basic case) that can return boxed command arguments, and then
also generate the help strings from that metadata.
I suspect that work *IS* something that might brush up against / should
use (or extend) QAPI code.
Then, I'd like to find a way to split qemu-img.texi into sub-command
files and find a way to source the same "informative text" for both:
(1) The texi output, as per usual, and
(2) qemu-img subcommand --help
such that --help, when passed as an argument to the subcommand, prints
out help only relevant to the subcommand instead, e.g.
`qemu-img check --help`
might print the "common options" section only as it relates to commands
actually used by the check command, then prints the check section of the
texi as formatted for terminals.
This way, we can have manpages, html pages, and interactive help text
all derived from the same semi-automated sources, always up to date and
much easier to read/discover/parse by human eyeballs.
That's what I'd like to accomplish, ultimately.
For now, I think this RFC set is not harmful and won't bother anybody.
It's definitely not worse than what we have now, fragility of removing
@var{} tokens and all.
--js
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool
2019-04-10 17:55 ` John Snow
(?)
@ 2019-04-11 6:22 ` Markus Armbruster
2019-04-11 20:02 ` John Snow
-1 siblings, 1 reply; 17+ messages in thread
From: Markus Armbruster @ 2019-04-11 6:22 UTC (permalink / raw)
To: John Snow
Cc: Kevin Wolf, Eduardo Habkost, qemu-block, qemu-devel, Max Reitz,
Cleber Rosa
John Snow <jsnow@redhat.com> writes:
> On 4/10/19 1:54 AM, Markus Armbruster wrote:
>> John Snow <jsnow@redhat.com> writes:
>>
>>> Presently we use hxtool and a .hx format to generate a few things like
>>> the qemu_img subcommand dispatch table, the qemu_img help() display output,
>>> and a help output in qemu-img.texi.
>>>
>>> Unfortunately, this means that this information is duplicated in at least
>>> three places:
>>>
>>> (1) in qemu-img-cmds.hx as a human readable string
>>> (2) in qemu-img-cmds.hx as a texi string
>>> (3) in qemu-img.texi again, as a texi string
>>>
>>> We can do a little better, though: all of these sources can be generated
>>> from a single json file. Add a new small tool ("pxtool") that can do this.
>>>
>>> This tool can at least handle generating (1) and (2) from the same source
>>> without needing to reduplicate that information. Deduplicating (3) happens
>>> in the next patch.
>>>
>>> Notes:
>>> - The json format was chosen to be very "stupid", and the command line
>>> documentation is being kept one-per-line to make future diffs easier
>>> to read.
>>> - It's easy enough to generate the human-readable output from the texi
>>> output by removing '@var{foo}' with 'foo'.
>>> - The qemu-img command dispatch always calls img_cmdname, so we don't
>>> need to store this information separately, either.
>>> - The need for DEF() macros could be removed as well, but I left it in
>>> to create a minimally disruptive patch.
>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>
>> We got just five .hx:
>>
>> qemu-img.cmds.hx killed off by this patch
>> qemu-options.hx CLI QAPIfication should kill this off
>> hw/audio/pl041.hx misnamed, not actually food for hxtool
>> hmp-commands.hx no exit strategy
>> hmp-commands-info.hx for these two, yet
>>
>> CLI QAPIfication got stuck in the back-burner, and as long as that's the
>> case, it's not an alternative to your patches.
>>
>
> Good to know. I figure that at least while we wait on something more
> comprehensive there's no real short term harm in tidying up.
>
> Something I'd really like to do is define a python/json-esque
> configuration file that allows you to specify some "common options" that
> are shared between all of the sub-commands, and then define each command
> in terms of both which common options it possesses, and then any local
> command-specific options it has.
>
> (Why the common options design? So that argument names and command
> options can be encouraged to be identical and identically documented
> between all subcommands that use them.)
In QAPI, you'd define a common base type.
> Then, from the configuration file, generate all of the necessary C
> parsers (I have a protoype for this, it's not difficult to do in at
> least the basic case) that can return boxed command arguments, and then
> also generate the help strings from that metadata.
With QAPI, we generate reference documentation from the doc comments.
My RFC patches for command line QAPIfication generate a CLI parser using
getopt_long_only().
They keep help texts in the QAPI schema. Generating their formal part
from the type definition should be feasible. Generating it without
regressing readability might turn out to be non-trivial.
My patches generate *one* CLI parser. We have many, and some of them
want to share types. There's more than one way to skin this cat, but
it'll have to be skinned.
> I suspect that work *IS* something that might brush up against / should
> use (or extend) QAPI code.
Seems likely.
> Then, I'd like to find a way to split qemu-img.texi into sub-command
> files and find a way to source the same "informative text" for both:
>
> (1) The texi output, as per usual, and
> (2) qemu-img subcommand --help
>
> such that --help, when passed as an argument to the subcommand, prints
> out help only relevant to the subcommand instead, e.g.
>
> `qemu-img check --help`
>
> might print the "common options" section only as it relates to commands
> actually used by the check command, then prints the check section of the
> texi as formatted for terminals.
>
> This way, we can have manpages, html pages, and interactive help text
> all derived from the same semi-automated sources, always up to date and
> much easier to read/discover/parse by human eyeballs.
>
> That's what I'd like to accomplish, ultimately.
>
> For now, I think this RFC set is not harmful and won't bother anybody.
> It's definitely not worse than what we have now, fragility of removing
> @var{} tokens and all.
Makes sense. I just hope we can avoid duplicating work.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool
2019-04-11 6:22 ` Markus Armbruster
@ 2019-04-11 20:02 ` John Snow
2019-04-12 8:04 ` Markus Armbruster
0 siblings, 1 reply; 17+ messages in thread
From: John Snow @ 2019-04-11 20:02 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Eduardo Habkost, qemu-block, qemu-devel, Max Reitz,
Cleber Rosa
On 4/11/19 2:22 AM, Markus Armbruster wrote:
> John Snow <jsnow@redhat.com> writes:
>
>> On 4/10/19 1:54 AM, Markus Armbruster wrote:
>>> John Snow <jsnow@redhat.com> writes:
>>>
>>>> Presently we use hxtool and a .hx format to generate a few things like
>>>> the qemu_img subcommand dispatch table, the qemu_img help() display output,
>>>> and a help output in qemu-img.texi.
>>>>
>>>> Unfortunately, this means that this information is duplicated in at least
>>>> three places:
>>>>
>>>> (1) in qemu-img-cmds.hx as a human readable string
>>>> (2) in qemu-img-cmds.hx as a texi string
>>>> (3) in qemu-img.texi again, as a texi string
>>>>
>>>> We can do a little better, though: all of these sources can be generated
>>>> from a single json file. Add a new small tool ("pxtool") that can do this.
>>>>
>>>> This tool can at least handle generating (1) and (2) from the same source
>>>> without needing to reduplicate that information. Deduplicating (3) happens
>>>> in the next patch.
>>>>
>>>> Notes:
>>>> - The json format was chosen to be very "stupid", and the command line
>>>> documentation is being kept one-per-line to make future diffs easier
>>>> to read.
>>>> - It's easy enough to generate the human-readable output from the texi
>>>> output by removing '@var{foo}' with 'foo'.
>>>> - The qemu-img command dispatch always calls img_cmdname, so we don't
>>>> need to store this information separately, either.
>>>> - The need for DEF() macros could be removed as well, but I left it in
>>>> to create a minimally disruptive patch.
>>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>>
>>> We got just five .hx:
>>>
>>> qemu-img.cmds.hx killed off by this patch
>>> qemu-options.hx CLI QAPIfication should kill this off
>>> hw/audio/pl041.hx misnamed, not actually food for hxtool
>>> hmp-commands.hx no exit strategy
>>> hmp-commands-info.hx for these two, yet
>>>
>>> CLI QAPIfication got stuck in the back-burner, and as long as that's the
>>> case, it's not an alternative to your patches.
>>>
>>
>> Good to know. I figure that at least while we wait on something more
>> comprehensive there's no real short term harm in tidying up.
>>
>> Something I'd really like to do is define a python/json-esque
>> configuration file that allows you to specify some "common options" that
>> are shared between all of the sub-commands, and then define each command
>> in terms of both which common options it possesses, and then any local
>> command-specific options it has.
>>
>> (Why the common options design? So that argument names and command
>> options can be encouraged to be identical and identically documented
>> between all subcommands that use them.)
>
> In QAPI, you'd define a common base type.
>
>> Then, from the configuration file, generate all of the necessary C
>> parsers (I have a protoype for this, it's not difficult to do in at
>> least the basic case) that can return boxed command arguments, and then
>> also generate the help strings from that metadata.
>
> With QAPI, we generate reference documentation from the doc comments.
>
> My RFC patches for command line QAPIfication generate a CLI parser using
> getopt_long_only().
>
> They keep help texts in the QAPI schema. Generating their formal part
> from the type definition should be feasible. Generating it without
> regressing readability might turn out to be non-trivial.
>
> My patches generate *one* CLI parser. We have many, and some of them
> want to share types. There's more than one way to skin this cat, but
> it'll have to be skinned.
>
>> I suspect that work *IS* something that might brush up against / should
>> use (or extend) QAPI code.
>
> Seems likely.
>
>> Then, I'd like to find a way to split qemu-img.texi into sub-command
>> files and find a way to source the same "informative text" for both:
>>
>> (1) The texi output, as per usual, and
>> (2) qemu-img subcommand --help
>>
>> such that --help, when passed as an argument to the subcommand, prints
>> out help only relevant to the subcommand instead, e.g.
>>
>> `qemu-img check --help`
>>
>> might print the "common options" section only as it relates to commands
>> actually used by the check command, then prints the check section of the
>> texi as formatted for terminals.
>>
>> This way, we can have manpages, html pages, and interactive help text
>> all derived from the same semi-automated sources, always up to date and
>> much easier to read/discover/parse by human eyeballs.
>>
>> That's what I'd like to accomplish, ultimately.
>>
>> For now, I think this RFC set is not harmful and won't bother anybody.
>> It's definitely not worse than what we have now, fragility of removing
>> @var{} tokens and all.
>
> Makes sense. I just hope we can avoid duplicating work.
>
I'm not going to proceed any further than this RFC without us having a
meeting to discuss the work. I'm willing to learn QAPI and do it a bit,
it's a thread I'd rather like to pull, but I don't want to duplicate any
work, no.
If that involves me reviewing patches when the tree opens, please CC me,
and let's have a chat!
In the meantime, what are your thoughts on patches 2-3 here?
--js
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool
2019-04-11 20:02 ` John Snow
@ 2019-04-12 8:04 ` Markus Armbruster
0 siblings, 0 replies; 17+ messages in thread
From: Markus Armbruster @ 2019-04-12 8:04 UTC (permalink / raw)
To: John Snow
Cc: Kevin Wolf, Eduardo Habkost, qemu-block, qemu-devel, Max Reitz,
Cleber Rosa
John Snow <jsnow@redhat.com> writes:
> On 4/11/19 2:22 AM, Markus Armbruster wrote:
>> John Snow <jsnow@redhat.com> writes:
[...]
>>> I suspect that work *IS* something that might brush up against / should
>>> use (or extend) QAPI code.
>>
>> Seems likely.
>>
>>> Then, I'd like to find a way to split qemu-img.texi into sub-command
>>> files and find a way to source the same "informative text" for both:
>>>
>>> (1) The texi output, as per usual, and
>>> (2) qemu-img subcommand --help
>>>
>>> such that --help, when passed as an argument to the subcommand, prints
>>> out help only relevant to the subcommand instead, e.g.
>>>
>>> `qemu-img check --help`
>>>
>>> might print the "common options" section only as it relates to commands
>>> actually used by the check command, then prints the check section of the
>>> texi as formatted for terminals.
>>>
>>> This way, we can have manpages, html pages, and interactive help text
>>> all derived from the same semi-automated sources, always up to date and
>>> much easier to read/discover/parse by human eyeballs.
>>>
>>> That's what I'd like to accomplish, ultimately.
>>>
>>> For now, I think this RFC set is not harmful and won't bother anybody.
>>> It's definitely not worse than what we have now, fragility of removing
>>> @var{} tokens and all.
>>
>> Makes sense. I just hope we can avoid duplicating work.
>>
>
> I'm not going to proceed any further than this RFC without us having a
> meeting to discuss the work. I'm willing to learn QAPI and do it a bit,
> it's a thread I'd rather like to pull, but I don't want to duplicate any
> work, no.
I think the path forward depends on just how itchy the thing is for you.
I do want to resume the work on CLI QAPIfication. Whenever I think my
queue is about to reach a state where I can ignore y'all for the
uninterrupted time the CLI project needs, something else lands in my
queue. Right now, Kevin wants the next QAPI project to be "features
flags", and he has v1 patches to back this up[*]. I have additional
uses for feature flags in mind, which I expect to require more patches.
> If that involves me reviewing patches when the tree opens, please CC me,
> and let's have a chat!
Yes, let's talk.
> In the meantime, what are your thoughts on patches 2-3 here?
I'll try to have a closer look.
[*] Which I still have to review, but when my queue goes out of control,
I enter batch mode for better throughput, sacrificing latency.
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH RFC 0/3] qemu-img: remove command documentation duplication
2019-04-10 1:24 ` John Snow
` (3 preceding siblings ...)
(?)
@ 2019-06-26 18:03 ` Max Reitz
2019-06-26 18:07 ` John Snow
-1 siblings, 1 reply; 17+ messages in thread
From: Max Reitz @ 2019-06-26 18:03 UTC (permalink / raw)
To: John Snow, qemu-devel, qemu-block
Cc: Kevin Wolf, Cleber Rosa, armbru, Eduardo Habkost
[-- Attachment #1.1: Type: text/plain, Size: 979 bytes --]
On 10.04.19 03:24, John Snow wrote:
> This might hopefully cut down on the doc duplication/mismatching
> until I can devise something more comprehensive.
>
> Ideally, I'd like to redo all of the documentation for qemu-img
> nearly from scratch; with a parser generator that helps generate
> the documentation as well so they'll never get out of date.
>
> That's probably quite a ways off, and maybe there are other
> structural changes we want to make with respect to sphinx and
> other build tools, so I am sending something very minimal instead.
>
> This ought to be functionally identical down to the last char.
>
> (I've re-included patch one which I have sent to the list separately,
> purely as a dependency if you want to apply these patches.)
>
> John Snow (3):
> qemu-img: fix .hx and .texi disparity
> pxtool: Add new qemu-img command info generation tool
> qemu-img.texi: use macros for command summaries
Hm. Non-RFC ping?
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* Re: [Qemu-devel] [PATCH RFC 0/3] qemu-img: remove command documentation duplication
2019-06-26 18:03 ` [Qemu-devel] [PATCH RFC 0/3] qemu-img: remove command documentation duplication Max Reitz
@ 2019-06-26 18:07 ` John Snow
0 siblings, 0 replies; 17+ messages in thread
From: John Snow @ 2019-06-26 18:07 UTC (permalink / raw)
To: Max Reitz, qemu-devel, qemu-block
Cc: Kevin Wolf, Cleber Rosa, armbru, Eduardo Habkost
On 6/26/19 2:03 PM, Max Reitz wrote:
> On 10.04.19 03:24, John Snow wrote:
>> This might hopefully cut down on the doc duplication/mismatching
>> until I can devise something more comprehensive.
>>
>> Ideally, I'd like to redo all of the documentation for qemu-img
>> nearly from scratch; with a parser generator that helps generate
>> the documentation as well so they'll never get out of date.
>>
>> That's probably quite a ways off, and maybe there are other
>> structural changes we want to make with respect to sphinx and
>> other build tools, so I am sending something very minimal instead.
>>
>> This ought to be functionally identical down to the last char.
>>
>> (I've re-included patch one which I have sent to the list separately,
>> purely as a dependency if you want to apply these patches.)
>>
>> John Snow (3):
>> qemu-img: fix .hx and .texi disparity
>> pxtool: Add new qemu-img command info generation tool
>> qemu-img.texi: use macros for command summaries
>
> Hm. Non-RFC ping?
>
> Max
>
This was mostly a quick hack; it didn't seek to eliminate ALL of the .hx
files we have, and it's not a comprehensive solution.
It is a quick stopgap that should work well enough for now, but it's
replacing one hack with another hack.
...Well, this hack is SLIGHTLY nicer, but I think I did some silly
things like using a zero-argument macro for texi instead of just
defining a variable. I don't know what I'm doing with texi stuff.
And then since I wrote it, there's been more buzz about reworking the
Sphinx documentation, so maybe this is too lateral of a move. I don't know.
I don't think it got review from the QAPI powers, though, and I think
I'd like to hear what they have to say before I invest any more time in
trying to fix this up.
--js
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2019-06-26 18:09 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-10 1:24 [Qemu-devel] [PATCH RFC 0/3] qemu-img: remove command documentation duplication John Snow
2019-04-10 1:24 ` John Snow
2019-04-10 1:24 ` [Qemu-devel] [PATCH RFC 1/3] qemu-img: fix .hx and .texi disparity John Snow
2019-04-10 1:24 ` John Snow
2019-04-10 1:24 ` [Qemu-devel] [PATCH RFC 2/3] pxtool: Add new qemu-img command info generation tool John Snow
2019-04-10 1:24 ` John Snow
2019-04-10 5:54 ` Markus Armbruster
2019-04-10 5:54 ` Markus Armbruster
2019-04-10 17:55 ` John Snow
2019-04-10 17:55 ` John Snow
2019-04-11 6:22 ` Markus Armbruster
2019-04-11 20:02 ` John Snow
2019-04-12 8:04 ` Markus Armbruster
2019-04-10 1:24 ` [Qemu-devel] [PATCH RFC 3/3] qemu-img.texi: use macros for command summaries John Snow
2019-04-10 1:24 ` John Snow
2019-06-26 18:03 ` [Qemu-devel] [PATCH RFC 0/3] qemu-img: remove command documentation duplication Max Reitz
2019-06-26 18:07 ` John Snow
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.