[PATCH 0/2] trace: documentation improvements

[PATCH 0/2] trace: documentation improvements

[Stefan Hajnoczi]

A recent discussion raised missing information in the tracing documentation.
Add it!

Based-on: 20201216160923.722894-1-stefanha@redhat.com

Stefan Hajnoczi (2):
  trace: document how to specify multiple --trace patterns
  trace: update docs with meson build information

 docs/devel/tracing.rst | 66 ++++++++++++++++++++++++++----------------
 1 file changed, 41 insertions(+), 25 deletions(-)

-- 
2.29.2

[PATCH 1/2] trace: document how to specify multiple --trace patterns

[Stefan Hajnoczi]

It is possible to repeat the --trace option to specify multiple
patterns. This may be preferrable to users who do not want to create a
file with a list of patterns.

Suggested-by: BALATON Zoltan <balaton@eik.bme.hu>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 docs/devel/tracing.rst | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/docs/devel/tracing.rst b/docs/devel/tracing.rst
index 4ebf8e38ea..8777c19d14 100644
--- a/docs/devel/tracing.rst
+++ b/docs/devel/tracing.rst
@@ -22,10 +22,15 @@ events::
 This output comes from the "log" trace backend that is enabled by default when
 ``./configure --enable-trace-backends=BACKENDS`` was not explicitly specified.
 
-More than one trace event pattern can be specified by providing a file
-instead::
+Multiple patterns can be specified by repeating the ``--trace`` option::
+
+    $ qemu --trace "kvm_*" --trace "virtio_*" ...
+
+When patterns are used frequently it is more convenient to store them in a
+file to avoid long command-line options::
 
     $ echo "memory_region_ops_*" >/tmp/events
+    $ echo "kvm_*" >>/tmp/events
     $ qemu --trace events=/tmp/events ...
 
 Trace events
-- 
2.29.2

[PATCH 2/2] trace: update docs with meson build information

[Stefan Hajnoczi]

The documentation still refers to the makefile and the old sub-directory
layout. Meson works differently: tracetool output is placed into the
builddir with mangled filenames like <builddir>/trace/trace-accel_kvm.h
for the accel/kvm/ trace.h definition.

This meson setup also requires a manually-created accel/kvm/trace.h file
that #includes the <builddir>/trace/trace-accel_kvm.h file. Document
this!

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 docs/devel/tracing.rst | 57 +++++++++++++++++++++++++-----------------
 1 file changed, 34 insertions(+), 23 deletions(-)

diff --git a/docs/devel/tracing.rst b/docs/devel/tracing.rst
index 8777c19d14..b58b37175d 100644
--- a/docs/devel/tracing.rst
+++ b/docs/devel/tracing.rst
@@ -39,40 +39,51 @@ Trace events
 Sub-directory setup
 -------------------
 
-Each directory in the source tree can declare a set of static trace events
-in a local "trace-events" file. All directories which contain "trace-events"
-files must be listed in the "trace-events-subdirs" make variable in the top
-level Makefile.objs. During build, the "trace-events" file in each listed
-subdirectory will be processed by the "tracetool" script to generate code for
-the trace events.
+Each directory in the source tree can declare a set of trace events in a local
+"trace-events" file. All directories which contain "trace-events" files must be
+listed in the "trace_events_subdirs" variable in the top level meson.build
+file. During build, the "trace-events" file in each listed subdirectory will be
+processed by the "tracetool" script to generate code for the trace events.
 
 The individual "trace-events" files are merged into a "trace-events-all" file,
 which is also installed into "/usr/share/qemu" with the name "trace-events".
 This merged file is to be used by the "simpletrace.py" script to later analyse
 traces in the simpletrace data format.
 
-In the sub-directory the following files will be automatically generated
+The following files are automatically generated in <builddir>/trace/ during the
+build:
 
- - trace.c - the trace event state declarations
- - trace.h - the trace event enums and probe functions
- - trace-dtrace.h - DTrace event probe specification
- - trace-dtrace.dtrace - DTrace event probe helper declaration
- - trace-dtrace.o - binary DTrace provider (generated by dtrace)
- - trace-ust.h - UST event probe helper declarations
+ - trace-<subdir>.c - the trace event state declarations
+ - trace-<subdir>.h - the trace event enums and probe functions
+ - trace-dtrace-<subdir>.h - DTrace event probe specification
+ - trace-dtrace-<subdir>.dtrace - DTrace event probe helper declaration
+ - trace-dtrace-<subdir>.o - binary DTrace provider (generated by dtrace)
+ - trace-ust-<subdir>.h - UST event probe helper declarations
 
-Source files in the sub-directory should #include the local 'trace.h' file,
-without any sub-directory path prefix. eg io/channel-buffer.c would do::
+Here <subdir> is the sub-directory path with '/' replaced by '_'. For example,
+"accel/kvm" becomes "accel_kvm" and the final filename for "trace-<subdir>.c"
+becomes "trace-accel_kvm.c".
+
+Source files in the source tree do not directly include generated files in
+"<builddir>/trace/". Instead they #include the local "trace.h" file, without
+any sub-directory path prefix. eg io/channel-buffer.c would do::
 
   #include "trace.h"
 
-To access the 'io/trace.h' file. While it is possible to include a trace.h
-file from outside a source file's own sub-directory, this is discouraged in
-general. It is strongly preferred that all events be declared directly in
-the sub-directory that uses them. The only exception is where there are some
-shared trace events defined in the top level directory trace-events file.
-The top level directory generates trace files with a filename prefix of
-"trace/trace-root" instead of just "trace". This is to avoid ambiguity between
-a trace.h in the current directory, vs the top level directory.
+The "io/trace.h" file must be created manually with an #include of the
+corresponding "trace/trace-<subdir>.h" file that will be generated in the
+builddir::
+
+  $ echo '#include "trace/trace-io.h"' >io/trace.h
+
+While it is possible to include a trace.h file from outside a source file's own
+sub-directory, this is discouraged in general. It is strongly preferred that
+all events be declared directly in the sub-directory that uses them. The only
+exception is where there are some shared trace events defined in the top level
+directory trace-events file.  The top level directory generates trace files
+with a filename prefix of "trace/trace-root" instead of just "trace". This is
+to avoid ambiguity between a trace.h in the current directory, vs the top level
+directory.
 
 Using trace events
 ------------------
-- 
2.29.2

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[Philippe Mathieu-Daudé]

On 1/12/21 5:58 PM, Stefan Hajnoczi wrote:
> It is possible to repeat the --trace option to specify multiple
> patterns. This may be preferrable to users who do not want to create a
> file with a list of patterns.
> 
> Suggested-by: BALATON Zoltan <balaton@eik.bme.hu>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>  docs/devel/tracing.rst | 9 +++++++--
>  1 file changed, 7 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/devel/tracing.rst b/docs/devel/tracing.rst
> index 4ebf8e38ea..8777c19d14 100644
> --- a/docs/devel/tracing.rst
> +++ b/docs/devel/tracing.rst
> @@ -22,10 +22,15 @@ events::
>  This output comes from the "log" trace backend that is enabled by default when
>  ``./configure --enable-trace-backends=BACKENDS`` was not explicitly specified.
>  
> -More than one trace event pattern can be specified by providing a file
> -instead::
> +Multiple patterns can be specified by repeating the ``--trace`` option::
> +
> +    $ qemu --trace "kvm_*" --trace "virtio_*" ...

Yet another possible improvement is to describe the deselect pattern:

       --trace "virtio_*" --trace "-virtio_mem*"

> +
> +When patterns are used frequently it is more convenient to store them in a
> +file to avoid long command-line options::
>  
>      $ echo "memory_region_ops_*" >/tmp/events
> +    $ echo "kvm_*" >>/tmp/events

Ditto:

       $ echo "-kvm_irqchip_*" >> /tmp/events

>      $ qemu --trace events=/tmp/events ...
>  
>  Trace events
> 

Probably better in a different patch, so for this one:
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[BALATON Zoltan]

On Tue, 12 Jan 2021, Stefan Hajnoczi wrote:
> It is possible to repeat the --trace option to specify multiple
> patterns. This may be preferrable to users who do not want to create a
> file with a list of patterns.
>
> Suggested-by: BALATON Zoltan <balaton@eik.bme.hu>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
> docs/devel/tracing.rst | 9 +++++++--
> 1 file changed, 7 insertions(+), 2 deletions(-)
>
> diff --git a/docs/devel/tracing.rst b/docs/devel/tracing.rst
> index 4ebf8e38ea..8777c19d14 100644
> --- a/docs/devel/tracing.rst
> +++ b/docs/devel/tracing.rst
> @@ -22,10 +22,15 @@ events::
> This output comes from the "log" trace backend that is enabled by default when
> ``./configure --enable-trace-backends=BACKENDS`` was not explicitly specified.
> 
> -More than one trace event pattern can be specified by providing a file
> -instead::
> +Multiple patterns can be specified by repeating the ``--trace`` option::
> +
> +    $ qemu --trace "kvm_*" --trace "virtio_*" ...

QEMU options are single dash with double dash accepted for compatibility 
but help and other docs have single dash so these (and below) should be 
-trace. (Also a bit less typing for otherwise already way too long command 
lines.)

Regards,
BALATON Zoltan

> +
> +When patterns are used frequently it is more convenient to store them in a
> +file to avoid long command-line options::
>
>     $ echo "memory_region_ops_*" >/tmp/events
> +    $ echo "kvm_*" >>/tmp/events
>     $ qemu --trace events=/tmp/events ...
> 
> Trace events
> -- 
> 2.29.2

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1647 bytes --]

On Tue, Jan 12, 2021 at 09:44:03PM +0100, BALATON Zoltan wrote:
> On Tue, 12 Jan 2021, Stefan Hajnoczi wrote:
> > It is possible to repeat the --trace option to specify multiple
> > patterns. This may be preferrable to users who do not want to create a
> > file with a list of patterns.
> > 
> > Suggested-by: BALATON Zoltan <balaton@eik.bme.hu>
> > Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> > ---
> > docs/devel/tracing.rst | 9 +++++++--
> > 1 file changed, 7 insertions(+), 2 deletions(-)
> > 
> > diff --git a/docs/devel/tracing.rst b/docs/devel/tracing.rst
> > index 4ebf8e38ea..8777c19d14 100644
> > --- a/docs/devel/tracing.rst
> > +++ b/docs/devel/tracing.rst
> > @@ -22,10 +22,15 @@ events::
> > This output comes from the "log" trace backend that is enabled by default when
> > ``./configure --enable-trace-backends=BACKENDS`` was not explicitly specified.
> > 
> > -More than one trace event pattern can be specified by providing a file
> > -instead::
> > +Multiple patterns can be specified by repeating the ``--trace`` option::
> > +
> > +    $ qemu --trace "kvm_*" --trace "virtio_*" ...
> 
> QEMU options are single dash with double dash accepted for compatibility but
> help and other docs have single dash so these (and below) should be -trace.
> (Also a bit less typing for otherwise already way too long command lines.)

Is this documented somewhere?

I was under the impression that '-' is legacy syntax and '--' is the
preferred syntax. There are examples of '--' on the QEMU man page.

Let's reach agreement, document it, and then make the documentation
consistent.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2956 bytes --]

On Wed, Jan 13, 2021 at 01:51:17PM +0100, BALATON Zoltan wrote:
> On Wed, 13 Jan 2021, Stefan Hajnoczi wrote:
> > On Tue, Jan 12, 2021 at 09:44:03PM +0100, BALATON Zoltan wrote:
> > > On Tue, 12 Jan 2021, Stefan Hajnoczi wrote:
> > > > It is possible to repeat the --trace option to specify multiple
> > > > patterns. This may be preferrable to users who do not want to create a
> > > > file with a list of patterns.
> > > > 
> > > > Suggested-by: BALATON Zoltan <balaton@eik.bme.hu>
> > > > Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> > > > ---
> > > > docs/devel/tracing.rst | 9 +++++++--
> > > > 1 file changed, 7 insertions(+), 2 deletions(-)
> > > > 
> > > > diff --git a/docs/devel/tracing.rst b/docs/devel/tracing.rst
> > > > index 4ebf8e38ea..8777c19d14 100644
> > > > --- a/docs/devel/tracing.rst
> > > > +++ b/docs/devel/tracing.rst
> > > > @@ -22,10 +22,15 @@ events::
> > > > This output comes from the "log" trace backend that is enabled by default when
> > > > ``./configure --enable-trace-backends=BACKENDS`` was not explicitly specified.
> > > > 
> > > > -More than one trace event pattern can be specified by providing a file
> > > > -instead::
> > > > +Multiple patterns can be specified by repeating the ``--trace`` option::
> > > > +
> > > > +    $ qemu --trace "kvm_*" --trace "virtio_*" ...
> > > 
> > > QEMU options are single dash with double dash accepted for compatibility but
> > > help and other docs have single dash so these (and below) should be -trace.
> > > (Also a bit less typing for otherwise already way too long command lines.)
> > 
> > Is this documented somewhere?
> 
> Maybe qemu-system-* -help ?

I mean developer documentation like CODING_STYLE.rst so we can point to
that when someone submits a patch that does not use the preferred
syntax.

> > I was under the impression that '-' is legacy syntax and '--' is the
> > preferred syntax. There are examples of '--' on the QEMU man page.
> 
> -- is also accepted but they are the same as single dash options. Some tools
> may have -- syntax preferred but not QEMU itself. If so that may be an
> inconsistency.
> 
> > Let's reach agreement, document it, and then make the documentation
> > consistent.
> 
> Since we don't have long and short arguments for the same options (like
> -m,--memory) I think -- does not make much sense. Also single dash is less
> typing and there are other programs using the same convention (e.g. whole X
> Window System) so I think the current one dash options are fine and should
> be kept consistent. As long as we can agree on this I can agree with that.
> :-)

I'm fine with either (or even using both interchangeably) but want to
make sure it's agreed for all of QEMU so we can really follow it and
don't need to spend time on it in the future.

Kevin: You used '--' in qemu-storage-daemon --help. Does this mean you
want QEMU to stop using '-'?

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[John Snow]

On 1/13/21 4:48 AM, Stefan Hajnoczi wrote:
>> QEMU options are single dash with double dash accepted for compatibility but
>> help and other docs have single dash so these (and below) should be -trace.
>> (Also a bit less typing for otherwise already way too long command lines.)
> Is this documented somewhere?
> 
> I was under the impression that '-' is legacy syntax and '--' is the
> preferred syntax. There are examples of '--' on the QEMU man page.
> 
> Let's reach agreement, document it, and then make the documentation
> consistent.
> 
> Stefan

My naive impression was that double-dash is the preferred idiom in 
linuxdom in general for any multi-character option.

We might hang on to single-dash for backwards compatibility, but I doubt 
we want to enshrine that as our preferred way.

Is there a reasoning I am unaware of?

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[Eric Blake]

On 1/13/21 3:42 PM, John Snow wrote:
> On 1/13/21 4:48 AM, Stefan Hajnoczi wrote:
>>> QEMU options are single dash with double dash accepted for
>>> compatibility but
>>> help and other docs have single dash so these (and below) should be
>>> -trace.
>>> (Also a bit less typing for otherwise already way too long command
>>> lines.)
>> Is this documented somewhere?
>>
>> I was under the impression that '-' is legacy syntax and '--' is the
>> preferred syntax. There are examples of '--' on the QEMU man page.

Historically, uses of 'getopt_long()' support only double dash, while
'getopt_long_only()' support both single and double.  While qemu does
not use getopt_long_only() (because it rolls its own parser instead), it
certainly tries to behave as if it does.  Meanwhile, our other tools
like qemu-img or qemu-storage-daemon use getopt_long().  I'm in favor of
preferring the double-dash in documentation, even when single-dash
works, especially for any option (like --trace) that is used for more
tools than just qemu proper, because it makes for easier copy-paste
between options that work for all tools in the qemu suite rather than
just qemu.

>>
>> Let's reach agreement, document it, and then make the documentation
>> consistent.
>>
>> Stefan
> 
> My naive impression was that double-dash is the preferred idiom in
> linuxdom in general for any multi-character option.
> 
> We might hang on to single-dash for backwards compatibility, but I doubt
> we want to enshrine that as our preferred way.
> 
> Is there a reasoning I am unaware of?

Continuing to document '-machine' instead of '--machine' for qemu is
debatable because we don't support 'qemu-img --machine' or
'qemu-storage-daemon --machine'; but since 'qemu-img -trace' is an error
while 'qemu-img --trace' works, I'm definitely in favor of preferring
'--trace' everywhere in our docs.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[John Snow]

On 1/13/21 5:15 PM, Eric Blake wrote:
> On 1/13/21 3:42 PM, John Snow wrote:
>> On 1/13/21 4:48 AM, Stefan Hajnoczi wrote:
>>>> QEMU options are single dash with double dash accepted for
>>>> compatibility but
>>>> help and other docs have single dash so these (and below) should be
>>>> -trace.
>>>> (Also a bit less typing for otherwise already way too long command
>>>> lines.)
>>> Is this documented somewhere?
>>>
>>> I was under the impression that '-' is legacy syntax and '--' is the
>>> preferred syntax. There are examples of '--' on the QEMU man page.
> 
> Historically, uses of 'getopt_long()' support only double dash, while
> 'getopt_long_only()' support both single and double.  While qemu does
> not use getopt_long_only() (because it rolls its own parser instead), it
> certainly tries to behave as if it does.  Meanwhile, our other tools
> like qemu-img or qemu-storage-daemon use getopt_long().  I'm in favor of
> preferring the double-dash in documentation, even when single-dash
> works, especially for any option (like --trace) that is used for more
> tools than just qemu proper, because it makes for easier copy-paste
> between options that work for all tools in the qemu suite rather than
> just qemu.
> 

Yes, this makes sense.

>>>
>>> Let's reach agreement, document it, and then make the documentation
>>> consistent.
>>>
>>> Stefan
>>
>> My naive impression was that double-dash is the preferred idiom in
>> linuxdom in general for any multi-character option.
>>
>> We might hang on to single-dash for backwards compatibility, but I doubt
>> we want to enshrine that as our preferred way.
>>
>> Is there a reasoning I am unaware of?
> 
> Continuing to document '-machine' instead of '--machine' for qemu is
> debatable because we don't support 'qemu-img --machine' or
> 'qemu-storage-daemon --machine'; but since 'qemu-img -trace' is an error
> while 'qemu-img --trace' works, I'm definitely in favor of preferring
> '--trace' everywhere in our docs.
> 

At that point, it's a guessing game as to which binaries support which 
flags and using which spellings -- maybe some will pick up new flags 
later and so on.

Skip the fuss and just insist on the double dash, I think.

--js

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[BALATON Zoltan]

On Wed, 13 Jan 2021, John Snow wrote:
> On 1/13/21 5:15 PM, Eric Blake wrote:
>> On 1/13/21 3:42 PM, John Snow wrote:
>>> On 1/13/21 4:48 AM, Stefan Hajnoczi wrote:
>>>>> QEMU options are single dash with double dash accepted for
>>>>> compatibility but
>>>>> help and other docs have single dash so these (and below) should be
>>>>> -trace.
>>>>> (Also a bit less typing for otherwise already way too long command
>>>>> lines.)
>>>> Is this documented somewhere?
>>>> 
>>>> I was under the impression that '-' is legacy syntax and '--' is the
>>>> preferred syntax. There are examples of '--' on the QEMU man page.
>> 
>> Historically, uses of 'getopt_long()' support only double dash, while
>> 'getopt_long_only()' support both single and double.  While qemu does
>> not use getopt_long_only() (because it rolls its own parser instead), it
>> certainly tries to behave as if it does.  Meanwhile, our other tools
>> like qemu-img or qemu-storage-daemon use getopt_long().  I'm in favor of
>> preferring the double-dash in documentation, even when single-dash
>> works, especially for any option (like --trace) that is used for more
>> tools than just qemu proper, because it makes for easier copy-paste
>> between options that work for all tools in the qemu suite rather than
>> just qemu.
>> 
>
> Yes, this makes sense.
>
>>>> 
>>>> Let's reach agreement, document it, and then make the documentation
>>>> consistent.
>>>> 
>>>> Stefan
>>> 
>>> My naive impression was that double-dash is the preferred idiom in
>>> linuxdom in general for any multi-character option.
>>> 
>>> We might hang on to single-dash for backwards compatibility, but I doubt
>>> we want to enshrine that as our preferred way.
>>> 
>>> Is there a reasoning I am unaware of?
>> 
>> Continuing to document '-machine' instead of '--machine' for qemu is
>> debatable because we don't support 'qemu-img --machine' or
>> 'qemu-storage-daemon --machine'; but since 'qemu-img -trace' is an error
>> while 'qemu-img --trace' works, I'm definitely in favor of preferring
>> '--trace' everywhere in our docs.
>> 
>
> At that point, it's a guessing game as to which binaries support which flags 
> and using which spellings -- maybe some will pick up new flags later and so 
> on.
>
> Skip the fuss and just insist on the double dash, I think.

Please don't. That would break all scripts that already call qemu using 
single dash options. Forcing everyone to edit their scripts is not nice. 
How about also accepting single dash in qemu-img for consistency then you 
can use whatever you want in docs as long as single dash still works for 
consistency and backward compatibility.

Regards,
BALATON Zoltan

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[John Snow]

On 1/13/21 6:45 PM, BALATON Zoltan wrote:
> 
> Please don't. That would break all scripts that already call qemu using 
> single dash options. Forcing everyone to edit their scripts is not nice. 
> How about also accepting single dash in qemu-img for consistency then 
> you can use whatever you want in docs as long as single dash still works 
> for consistency and backward compatibility.

That's not what I meant -- I meant for documentation purposes. QEMU can 
keep accepting single dash, but we should standardize on the spelling 
for purposes of interactive messages, documentation, etc.

--js

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1299 bytes --]

On Wed, Jan 13, 2021 at 07:20:42PM -0500, John Snow wrote:
> On 1/13/21 6:45 PM, BALATON Zoltan wrote:
> > 
> > Please don't. That would break all scripts that already call qemu using
> > single dash options. Forcing everyone to edit their scripts is not nice.
> > How about also accepting single dash in qemu-img for consistency then
> > you can use whatever you want in docs as long as single dash still works
> > for consistency and backward compatibility.
> 
> That's not what I meant -- I meant for documentation purposes. QEMU can keep
> accepting single dash, but we should standardize on the spelling for
> purposes of interactive messages, documentation, etc.

I wasn't aware of the fact that some of the utilities are sensitive to
'--' vs '-'! I'm in favor of consistently using '--' in documentation
but allowing both for backwards compatibility where '-' is currently
supported.

If we are in agreement, then let's:

1. Add a section to CODING_STYLE.rst or other developer documentation
   documenting this.

2. Convert existing documentation to use '--'. This will make it more
   consistent and also avoid confusion about '-' vs '--'.

John: Do you have time to do these things as part of your QEMU
command-line and API improvement work?

Thanks,
Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[BALATON Zoltan]

On Thu, 14 Jan 2021, Stefan Hajnoczi wrote:
> On Wed, Jan 13, 2021 at 07:20:42PM -0500, John Snow wrote:
>> On 1/13/21 6:45 PM, BALATON Zoltan wrote:
>>>
>>> Please don't. That would break all scripts that already call qemu using
>>> single dash options. Forcing everyone to edit their scripts is not nice.
>>> How about also accepting single dash in qemu-img for consistency then
>>> you can use whatever you want in docs as long as single dash still works
>>> for consistency and backward compatibility.
>>
>> That's not what I meant -- I meant for documentation purposes. QEMU can keep
>> accepting single dash, but we should standardize on the spelling for
>> purposes of interactive messages, documentation, etc.
>
> I wasn't aware of the fact that some of the utilities are sensitive to
> '--' vs '-'! I'm in favor of consistently using '--' in documentation
> but allowing both for backwards compatibility where '-' is currently
> supported.
>
> If we are in agreement, then let's:
>
> 1. Add a section to CODING_STYLE.rst or other developer documentation
>   documenting this.

I'd be more in favour of documenting that QEMU accepts - options but also 
-- as alternative and fixing the tools that currently use getopt_long to 
use getopt_long_only to keep it consistent with main QEMU executable. 
Otherwise this will get more and more inconsistent with new options added 
with -- and old ones only exist in - form so to keep consistency we should 
standardise on - not --.

> 2. Convert existing documentation to use '--'. This will make it more
>   consistent and also avoid confusion about '-' vs '--'.

You could still use -- in documentation but what's the problem with - if 
-- is also accepted if one wants to type that?

Regards,
BALATON Zoltan

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2086 bytes --]

On Thu, Jan 14, 2021 at 03:18:48PM +0100, BALATON Zoltan wrote:
> On Thu, 14 Jan 2021, Stefan Hajnoczi wrote:
> > On Wed, Jan 13, 2021 at 07:20:42PM -0500, John Snow wrote:
> > > On 1/13/21 6:45 PM, BALATON Zoltan wrote:
> > > > 
> > > > Please don't. That would break all scripts that already call qemu using
> > > > single dash options. Forcing everyone to edit their scripts is not nice.
> > > > How about also accepting single dash in qemu-img for consistency then
> > > > you can use whatever you want in docs as long as single dash still works
> > > > for consistency and backward compatibility.
> > > 
> > > That's not what I meant -- I meant for documentation purposes. QEMU can keep
> > > accepting single dash, but we should standardize on the spelling for
> > > purposes of interactive messages, documentation, etc.
> > 
> > I wasn't aware of the fact that some of the utilities are sensitive to
> > '--' vs '-'! I'm in favor of consistently using '--' in documentation
> > but allowing both for backwards compatibility where '-' is currently
> > supported.
> > 
> > If we are in agreement, then let's:
> > 
> > 1. Add a section to CODING_STYLE.rst or other developer documentation
> >   documenting this.
> 
> I'd be more in favour of documenting that QEMU accepts - options but also --
> as alternative and fixing the tools that currently use getopt_long to use
> getopt_long_only to keep it consistent with main QEMU executable. Otherwise
> this will get more and more inconsistent with new options added with -- and
> old ones only exist in - form so to keep consistency we should standardise
> on - not --.
> 
> > 2. Convert existing documentation to use '--'. This will make it more
> >   consistent and also avoid confusion about '-' vs '--'.
> 
> You could still use -- in documentation but what's the problem with - if --
> is also accepted if one wants to type that?

What is the policy for QEMU documentation?

1. Use '-' everywhere

or

2. Use '--' everywhere

or

3. Use whichever you prefer

or

?

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[John Snow]

On 1/14/21 9:02 AM, Stefan Hajnoczi wrote:
> 
> 1. Add a section to CODING_STYLE.rst or other developer documentation
>     documenting this.
> 
> 2. Convert existing documentation to use '--'. This will make it more
>     consistent and also avoid confusion about '-' vs '--'.
> 
> John: Do you have time to do these things as part of your QEMU
> command-line and API improvement work?

I will add it to my todoist task list, but I am still neck deep in the 
QAPI forest, so it won't be a priority in the short term.

--js

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[Eric Blake]

On 1/14/21 8:18 AM, BALATON Zoltan wrote:

>> I wasn't aware of the fact that some of the utilities are sensitive to
>> '--' vs '-'! I'm in favor of consistently using '--' in documentation
>> but allowing both for backwards compatibility where '-' is currently
>> supported.
>>
>> If we are in agreement, then let's:
>>
>> 1. Add a section to CODING_STYLE.rst or other developer documentation
>>   documenting this.

Seems reasonable to me.

> 
> I'd be more in favour of documenting that QEMU accepts - options but
> also -- as alternative and fixing the tools that currently use
> getopt_long to use getopt_long_only to keep it consistent with main QEMU
> executable. Otherwise this will get more and more inconsistent with new
> options added with -- and old ones only exist in - form so to keep
> consistency we should standardise on - not --.

I've got less practical experience with getopt_long_only(); I know there
are some utilities like gcc that have to use it, but GNU coding
standards prefer getopt_long() over getopt_long_only().  I think one of
the reasons is the potential for ambiguity: if you have a program that
accepts a series of short options without arguments, you can combine
them together (think 'ls -lF'), but what happens when your combination
of letters then resembles a long option?  A bit contrived, but 'ls --no'
is short for 'ls --no-group' (aka 'ls -G'), while 'ls -no' is the same
as 'ls -n -o', which has different behavior.  ls uses getopt_long(),
hence the use of -- matters; but if it were to use getopt_long_only(),
you would have changed the behavior of 'ls -no' (it would now favor
--no-group over -n -o).

That's not to say we can't switch qemu-img, qemu-storage-daemon,
qemu-io, and friends to use getopt_long_only(), but merely that we have
to be careful of what it will do to their command line parsing, and
whether it will introduce any unintended regressions.

So the conservative answer from me is to prefer documenting '--' options
everywhere, rather than trying to figure out when '-' is acceptable with
long option names.

> 
>> 2. Convert existing documentation to use '--'. This will make it more
>>   consistent and also avoid confusion about '-' vs '--'.
> 
> You could still use -- in documentation but what's the problem with - if
> -- is also accepted if one wants to type that?

Supporting lazy typists is one thing, but our documentation should stick
to the preferred form, even when shorter forms are possible.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 2838 bytes --]

On Tue, Jan 19, 2021 at 10:31:49AM -0600, Eric Blake wrote:
> On 1/14/21 8:18 AM, BALATON Zoltan wrote:
> 
> >> I wasn't aware of the fact that some of the utilities are sensitive to
> >> '--' vs '-'! I'm in favor of consistently using '--' in documentation
> >> but allowing both for backwards compatibility where '-' is currently
> >> supported.
> >>
> >> If we are in agreement, then let's:
> >>
> >> 1. Add a section to CODING_STYLE.rst or other developer documentation
> >>   documenting this.
> 
> Seems reasonable to me.
> 
> > 
> > I'd be more in favour of documenting that QEMU accepts - options but
> > also -- as alternative and fixing the tools that currently use
> > getopt_long to use getopt_long_only to keep it consistent with main QEMU
> > executable. Otherwise this will get more and more inconsistent with new
> > options added with -- and old ones only exist in - form so to keep
> > consistency we should standardise on - not --.
> 
> I've got less practical experience with getopt_long_only(); I know there
> are some utilities like gcc that have to use it, but GNU coding
> standards prefer getopt_long() over getopt_long_only().  I think one of
> the reasons is the potential for ambiguity: if you have a program that
> accepts a series of short options without arguments, you can combine
> them together (think 'ls -lF'), but what happens when your combination
> of letters then resembles a long option?  A bit contrived, but 'ls --no'
> is short for 'ls --no-group' (aka 'ls -G'), while 'ls -no' is the same
> as 'ls -n -o', which has different behavior.  ls uses getopt_long(),
> hence the use of -- matters; but if it were to use getopt_long_only(),
> you would have changed the behavior of 'ls -no' (it would now favor
> --no-group over -n -o).
> 
> That's not to say we can't switch qemu-img, qemu-storage-daemon,
> qemu-io, and friends to use getopt_long_only(), but merely that we have
> to be careful of what it will do to their command line parsing, and
> whether it will introduce any unintended regressions.
> 
> So the conservative answer from me is to prefer documenting '--' options
> everywhere, rather than trying to figure out when '-' is acceptable with
> long option names.
> 
> > 
> >> 2. Convert existing documentation to use '--'. This will make it more
> >>   consistent and also avoid confusion about '-' vs '--'.
> > 
> > You could still use -- in documentation but what's the problem with - if
> > -- is also accepted if one wants to type that?
> 
> Supporting lazy typists is one thing, but our documentation should stick
> to the preferred form, even when shorter forms are possible.

I lost track of this email thread.

Do we have agreement on that QEMU documentation should consistently use
'--' for long options?

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 0/2] trace: documentation improvements

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 618 bytes --]

On Tue, Jan 12, 2021 at 04:58:57PM +0000, Stefan Hajnoczi wrote:
> A recent discussion raised missing information in the tracing documentation.
> Add it!
> 
> Based-on: 20201216160923.722894-1-stefanha@redhat.com
> 
> Stefan Hajnoczi (2):
>   trace: document how to specify multiple --trace patterns
>   trace: update docs with meson build information
> 
>  docs/devel/tracing.rst | 66 ++++++++++++++++++++++++++----------------
>  1 file changed, 41 insertions(+), 25 deletions(-)
> 
> -- 
> 2.29.2
> 

Thanks, applied to my tracing tree:
https://gitlab.com/stefanha/qemu/commits/tracing

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 1/2] trace: document how to specify multiple --trace patterns

[Kevin Wolf]

[-- Attachment #1: Type: text/plain, Size: 3478 bytes --]

Am 13.01.2021 um 15:15 hat Stefan Hajnoczi geschrieben:
> On Wed, Jan 13, 2021 at 01:51:17PM +0100, BALATON Zoltan wrote:
> > On Wed, 13 Jan 2021, Stefan Hajnoczi wrote:
> > > On Tue, Jan 12, 2021 at 09:44:03PM +0100, BALATON Zoltan wrote:
> > > > On Tue, 12 Jan 2021, Stefan Hajnoczi wrote:
> > > > > It is possible to repeat the --trace option to specify multiple
> > > > > patterns. This may be preferrable to users who do not want to create a
> > > > > file with a list of patterns.
> > > > > 
> > > > > Suggested-by: BALATON Zoltan <balaton@eik.bme.hu>
> > > > > Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> > > > > ---
> > > > > docs/devel/tracing.rst | 9 +++++++--
> > > > > 1 file changed, 7 insertions(+), 2 deletions(-)
> > > > > 
> > > > > diff --git a/docs/devel/tracing.rst b/docs/devel/tracing.rst
> > > > > index 4ebf8e38ea..8777c19d14 100644
> > > > > --- a/docs/devel/tracing.rst
> > > > > +++ b/docs/devel/tracing.rst
> > > > > @@ -22,10 +22,15 @@ events::
> > > > > This output comes from the "log" trace backend that is enabled by default when
> > > > > ``./configure --enable-trace-backends=BACKENDS`` was not explicitly specified.
> > > > > 
> > > > > -More than one trace event pattern can be specified by providing a file
> > > > > -instead::
> > > > > +Multiple patterns can be specified by repeating the ``--trace`` option::
> > > > > +
> > > > > +    $ qemu --trace "kvm_*" --trace "virtio_*" ...
> > > > 
> > > > QEMU options are single dash with double dash accepted for compatibility but
> > > > help and other docs have single dash so these (and below) should be -trace.
> > > > (Also a bit less typing for otherwise already way too long command lines.)
> > > 
> > > Is this documented somewhere?
> > 
> > Maybe qemu-system-* -help ?
> 
> I mean developer documentation like CODING_STYLE.rst so we can point to
> that when someone submits a patch that does not use the preferred
> syntax.
> 
> > > I was under the impression that '-' is legacy syntax and '--' is the
> > > preferred syntax. There are examples of '--' on the QEMU man page.
> > 
> > -- is also accepted but they are the same as single dash options. Some tools
> > may have -- syntax preferred but not QEMU itself. If so that may be an
> > inconsistency.
> > 
> > > Let's reach agreement, document it, and then make the documentation
> > > consistent.
> > 
> > Since we don't have long and short arguments for the same options (like
> > -m,--memory) I think -- does not make much sense. Also single dash is less
> > typing and there are other programs using the same convention (e.g. whole X
> > Window System) so I think the current one dash options are fine and should
> > be kept consistent. As long as we can agree on this I can agree with that.
> > :-)
> 
> I'm fine with either (or even using both interchangeably) but want to
> make sure it's agreed for all of QEMU so we can really follow it and
> don't need to spend time on it in the future.
> 
> Kevin: You used '--' in qemu-storage-daemon --help. Does this mean you
> want QEMU to stop using '-'?

qemu-storage-daemon just follows the example of the other tools which
use getopt() instead of a hand-written parser, which means that in all
of the tools, long options require '--'.

I don't have a strong opinion about '-' in the system emulator, though
if I tried out dropping it, I guess I'd find my muscle memory does have
one.

Kevin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]