* [Xen-devel] [PATCH v8 00/10] error: auto propagated local_err part I
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Laszlo Ersek, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Greg Kurz, armbru, Stefano Stabellini,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Eric Blake, Michael Roth, Stefan Berger
File with errp-cleaning APIs dropped for two reasons:
1. I'm tired after a 3-days war with coccinelle, and don't want to add more
patches here.
2. Markus noted, that we forget two more functions which needs such wrappers
and corresponding conversion, so seems better to handle all these things
in same manner for now.
changes in v8:
01: - update comments
- fix bug in macro [Markus]
- use do {} while(0)
02: a lot of changes
- about error propagation: try to update only patterns where we propagate
local_error to errp. So, patches 09 and 10 changed (wow!)
- Now clearing functions are not defined. Still, do the conversion, so
that when script applied where these functions needed, compilation will
be broken.
- improve ordering of hunks, and comment everything
09,10: drop hunks, which are converted for nothing.
v8 is available at
https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-auto-local-err-partI-v8
v7 is available at
https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-auto-local-err-partI-v7
In these series, there is no commit-per-subsystem script, each generated
commit is generated in separate.
Still, generating commands are very similar, and looks like
sed -n '/^<Subsystem name>$/,/^$/{s/^F: //p}' MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80
Note, that in each generated commit, generation command is the only
text, indented by 8 spaces in 'git log -1' output, so, to regenerate all
commits (for example, after rebase, or change in coccinelle script), you
may use the following command:
git rebase -x "sh -c \"git show --pretty= --name-only | xargs git checkout HEAD^ -- ; git reset; git log -1 | grep '^ ' | sh\"" HEAD~7
Which will start automated interactive rebase for generated patches,
which will stop if generated patch changed
(you may do git commit --amend to apply updated generated changes).
Note:
git show --pretty= --name-only - lists files, changed in HEAD
git log -1 | grep '^ ' | sh - rerun generation command of HEAD
Check for compilation of changed .c files
git rebase -x "sh -c \"git show --pretty= --name-only | sed -n 's/\.c$/.o/p' | xargs make -j9\"" HEAD~7
Vladimir Sementsov-Ogievskiy (10):
error: auto propagated local_err
scripts: add coccinelle script to use auto propagated errp
hw/sd/ssi-sd: fix error handling in ssi_sd_realize
SD (Secure Card): introduce ERRP_AUTO_PROPAGATE
pflash: introduce ERRP_AUTO_PROPAGATE
fw_cfg: introduce ERRP_AUTO_PROPAGATE
virtio-9p: introduce ERRP_AUTO_PROPAGATE
TPM: introduce ERRP_AUTO_PROPAGATE
nbd: introduce ERRP_AUTO_PROPAGATE
xen: introduce ERRP_AUTO_PROPAGATE
include/block/nbd.h | 1 +
include/qapi/error.h | 206 +++++++++++++---
block/nbd.c | 21 +-
hw/9pfs/9p-local.c | 12 +-
hw/9pfs/9p.c | 1 +
hw/block/dataplane/xen-block.c | 17 +-
hw/block/pflash_cfi01.c | 7 +-
hw/block/pflash_cfi02.c | 7 +-
hw/block/xen-block.c | 125 ++++------
hw/nvram/fw_cfg.c | 14 +-
hw/pci-host/xen_igd_pt.c | 7 +-
hw/sd/sdhci-pci.c | 7 +-
hw/sd/sdhci.c | 21 +-
hw/sd/ssi-sd.c | 26 +-
hw/tpm/tpm_util.c | 7 +-
hw/xen/xen-backend.c | 7 +-
hw/xen/xen-bus.c | 92 +++----
hw/xen/xen-host-pci-device.c | 27 +-
hw/xen/xen_pt.c | 25 +-
hw/xen/xen_pt_config_init.c | 20 +-
nbd/client.c | 5 +
nbd/server.c | 5 +
tpm.c | 7 +-
scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
24 files changed, 617 insertions(+), 281 deletions(-)
create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
Cc: Eric Blake <eblake@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Max Reitz <mreitz@redhat.com>
Cc: Greg Kurz <groug@kaod.org>
Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Paul Durrant <paul@xen.org>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
Cc: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org
Cc: xen-devel@lists.xenproject.org
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* [PATCH v8 01/10] error: auto propagated local_err
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Laszlo Ersek, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Greg Kurz, armbru, Stefano Stabellini,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Michael Roth, Stefan Berger
Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of
functions with an errp OUT parameter.
It has three goals:
1. Fix issue with error_fatal and error_prepend/error_append_hint: user
can't see this additional information, because exit() happens in
error_setg earlier than information is added. [Reported by Greg Kurz]
2. Fix issue with error_abort and error_propagate: when we wrap
error_abort by local_err+error_propagate, the resulting coredump will
refer to error_propagate and not to the place where error happened.
(the macro itself doesn't fix the issue, but it allows us to [3.] drop
the local_err+error_propagate pattern, which will definitely fix the
issue) [Reported by Kevin Wolf]
3. Drop local_err+error_propagate pattern, which is used to workaround
void functions with errp parameter, when caller wants to know resulting
status. (Note: actually these functions could be merely updated to
return int error code).
To achieve these goals, later patches will add invocations
of this macro at the start of functions with either use
error_prepend/error_append_hint (solving 1) or which use
local_err+error_propagate to check errors, switching those
functions to use *errp instead (solving 2 and 3).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
Cc: Eric Blake <eblake@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Max Reitz <mreitz@redhat.com>
Cc: Greg Kurz <groug@kaod.org>
Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Paul Durrant <paul@xen.org>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
Cc: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org
Cc: xen-devel@lists.xenproject.org
include/qapi/error.h | 203 ++++++++++++++++++++++++++++++++++++-------
1 file changed, 170 insertions(+), 33 deletions(-)
diff --git a/include/qapi/error.h b/include/qapi/error.h
index ad5b6e896d..bb9bcf02fb 100644
--- a/include/qapi/error.h
+++ b/include/qapi/error.h
@@ -15,6 +15,8 @@
/*
* Error reporting system loosely patterned after Glib's GError.
*
+ * = Deal with Error object =
+ *
* Create an error:
* error_setg(&err, "situation normal, all fouled up");
*
@@ -47,28 +49,88 @@
* reporting it (primarily useful in testsuites):
* error_free_or_abort(&err);
*
- * Pass an existing error to the caller:
- * error_propagate(errp, err);
- * where Error **errp is a parameter, by convention the last one.
+ * = Deal with Error ** function parameter =
*
- * Pass an existing error to the caller with the message modified:
- * error_propagate_prepend(errp, err);
+ * Function may use error system to return errors. In this case function
+ * defines Error **errp parameter, which should be the last one (except for
+ * functions which varidic argument list), which has the following API:
*
- * Avoid
- * error_propagate(errp, err);
- * error_prepend(errp, "Could not frobnicate '%s': ", name);
- * because this fails to prepend when @errp is &error_fatal.
+ * Caller may pass as errp:
+ * 1. &error_abort
+ * This means abort on any error
+ * 2. &error_fatal
+ * Exit with non-zero return code on error
+ * 3. NULL
+ * Ignore errors
+ * 4. Another value
+ * On error allocate error object and set errp
*
- * Create a new error and pass it to the caller:
- * error_setg(errp, "situation normal, all fouled up");
+ * Error API functions with Error ** (like error_setg) argument supports these
+ * rules, so user functions just need to use them appropriately (read below).
*
- * Call a function and receive an error from it:
+ * Simple pass error to the caller:
+ * error_setg(errp, "Some error");
+ *
+ * Subcall of another errp-based function, passing the error to the caller
+ * f(..., errp);
+ *
+ * == Checking success of subcall ==
+ *
+ * If function returns error code in addition to errp (which is recommended),
+ * you don't need any additional code, just do:
+ * int ret = f(..., errp);
+ * if (ret < 0) {
+ * ... handle error ...
+ * return ret;
+ * }
+ *
+ * If function returns nothing (which is not recommended API) and the only way
+ * to check success is checking errp, we must care about cases [1-3] above. We
+ * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:
+ *
+ * int our_func(..., Error **errp) {
+ * ERRP_AUTO_PROPAGATE();
+ * ...
+ * subcall(..., errp);
+ * if (*errp) {
+ * ...
+ * return -ERRNO;
+ * }
+ * ...
+ * }
+ *
+ * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
+ * so that it can be safely used (including dereferencing), and auto-propagates
+ * error to original errp on function end.
+ *
+ * In some cases, we need to check result of subcall, but do not want to
+ * propagate the Error object to our caller. In such cases we don't need
+ * ERRP_AUTO_PROPAGATE, but just a local Error object:
+ *
+ * Receive an error and not pass it:
* Error *err = NULL;
- * foo(arg, &err);
+ * subcall(arg, &err);
* if (err) {
* handle the error...
+ * error_free(err);
* }
*
+ * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
+ * error_propagate() instead of error_free()) was used to check and pass error
+ * to the caller. Now this is DEPRECATED* (see below).
+ *
+ * Note also, that if you want to use error_append_hint/error_prepend or their
+ * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
+ * error_fatal, you'll miss the chance to insert your additional information
+ * into Error object.
+ *
+ * In rare cases, we need to pass existing Error object to the caller by hand:
+ * error_propagate(errp, err);
+ *
+ * Pass an existing error to the caller with the message modified:
+ * error_propagate_prepend(errp, err);
+ *
+ *
* Call a function ignoring errors:
* foo(arg, NULL);
*
@@ -78,26 +140,6 @@
* Call a function treating errors as fatal:
* foo(arg, &error_fatal);
*
- * Receive an error and pass it on to the caller:
- * Error *err = NULL;
- * foo(arg, &err);
- * if (err) {
- * handle the error...
- * error_propagate(errp, err);
- * }
- * where Error **errp is a parameter, by convention the last one.
- *
- * Do *not* "optimize" this to
- * foo(arg, errp);
- * if (*errp) { // WRONG!
- * handle the error...
- * }
- * because errp may be NULL!
- *
- * But when all you do with the error is pass it on, please use
- * foo(arg, errp);
- * for readability.
- *
* Receive and accumulate multiple errors (first one wins):
* Error *err = NULL, *local_err = NULL;
* foo(arg, &err);
@@ -114,6 +156,61 @@
* handle the error...
* }
* because this may pass a non-null err to bar().
+ *
+ * DEPRECATED*
+ *
+ * The following pattern of receiving checking and passing the caller of the
+ * error by hand is deprecated now:
+ *
+ * Error *err = NULL;
+ * foo(arg, &err);
+ * if (err) {
+ * handle the error...
+ * error_propagate(errp, err);
+ * }
+ *
+ * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).
+ *
+ * The old pattern is deprecated because of two things:
+ *
+ * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
+ * local_err+error_propagate, the resulting coredump will refer to
+ * error_propagate and not to the place where error happened.
+ *
+ * 2. A lot of extra code of the same pattern
+ *
+ * How to update old code to use ERRP_AUTO_PROPAGATE?
+ *
+ * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
+ * than you may safely dereference errp to check errors and do not need any
+ * additional local Error variables or calls to error_propagate().
+ *
+ * Example:
+ *
+ * old code
+ *
+ * void fn(..., Error **errp) {
+ * Error *err = NULL;
+ * foo(arg, &err);
+ * if (err) {
+ * handle the error...
+ * error_propagate(errp, err);
+ * return;
+ * }
+ * ...
+ * }
+ *
+ * updated code
+ *
+ * void fn(..., Error **errp) {
+ * ERRP_AUTO_PROPAGATE();
+ * foo(arg, errp);
+ * if (*errp) {
+ * handle the error...
+ * return;
+ * }
+ * ...
+ * }
*/
#ifndef ERROR_H
@@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
ErrorClass err_class, const char *fmt, ...)
GCC_FMT_ATTR(6, 7);
+typedef struct ErrorPropagator {
+ Error *local_err;
+ Error **errp;
+} ErrorPropagator;
+
+static inline void error_propagator_cleanup(ErrorPropagator *prop)
+{
+ error_propagate(prop->errp, prop->local_err);
+}
+
+G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
+
+/*
+ * ERRP_AUTO_PROPAGATE
+ *
+ * This macro is created to be the first line of a function which use
+ * Error **errp parameter to report error. It's needed only in cases where we
+ * want to use error_prepend, error_append_hint or dereference *errp. It's
+ * still safe (but useless) in other cases.
+ *
+ * If errp is NULL or points to error_fatal, it is rewritten to point to a
+ * local Error object, which will be automatically propagated to the original
+ * errp on function exit (see error_propagator_cleanup).
+ *
+ * After invocation of this macro it is always safe to dereference errp
+ * (as it's not NULL anymore) and to add information by error_prepend or
+ * error_append_hint (as, if it was error_fatal, we swapped it with a
+ * local_error to be propagated on cleanup).
+ *
+ * Note: we don't wrap the error_abort case, as we want resulting coredump
+ * to point to the place where the error happened, not to error_propagate.
+ */
+#define ERRP_AUTO_PROPAGATE() \
+ g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \
+ do { \
+ if (!errp || errp == &error_fatal) { \
+ errp = &_auto_errp_prop.local_err; \
+ } \
+ } while (0)
+
/*
* Special error destination to abort on error.
* See error_setg() and error_propagate() for details.
--
2.21.0
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Xen-devel] [PATCH v8 01/10] error: auto propagated local_err
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Laszlo Ersek, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Greg Kurz, armbru, Stefano Stabellini,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Eric Blake, Michael Roth, Stefan Berger
Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of
functions with an errp OUT parameter.
It has three goals:
1. Fix issue with error_fatal and error_prepend/error_append_hint: user
can't see this additional information, because exit() happens in
error_setg earlier than information is added. [Reported by Greg Kurz]
2. Fix issue with error_abort and error_propagate: when we wrap
error_abort by local_err+error_propagate, the resulting coredump will
refer to error_propagate and not to the place where error happened.
(the macro itself doesn't fix the issue, but it allows us to [3.] drop
the local_err+error_propagate pattern, which will definitely fix the
issue) [Reported by Kevin Wolf]
3. Drop local_err+error_propagate pattern, which is used to workaround
void functions with errp parameter, when caller wants to know resulting
status. (Note: actually these functions could be merely updated to
return int error code).
To achieve these goals, later patches will add invocations
of this macro at the start of functions with either use
error_prepend/error_append_hint (solving 1) or which use
local_err+error_propagate to check errors, switching those
functions to use *errp instead (solving 2 and 3).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
Cc: Eric Blake <eblake@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Max Reitz <mreitz@redhat.com>
Cc: Greg Kurz <groug@kaod.org>
Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Paul Durrant <paul@xen.org>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
Cc: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org
Cc: xen-devel@lists.xenproject.org
include/qapi/error.h | 203 ++++++++++++++++++++++++++++++++++++-------
1 file changed, 170 insertions(+), 33 deletions(-)
diff --git a/include/qapi/error.h b/include/qapi/error.h
index ad5b6e896d..bb9bcf02fb 100644
--- a/include/qapi/error.h
+++ b/include/qapi/error.h
@@ -15,6 +15,8 @@
/*
* Error reporting system loosely patterned after Glib's GError.
*
+ * = Deal with Error object =
+ *
* Create an error:
* error_setg(&err, "situation normal, all fouled up");
*
@@ -47,28 +49,88 @@
* reporting it (primarily useful in testsuites):
* error_free_or_abort(&err);
*
- * Pass an existing error to the caller:
- * error_propagate(errp, err);
- * where Error **errp is a parameter, by convention the last one.
+ * = Deal with Error ** function parameter =
*
- * Pass an existing error to the caller with the message modified:
- * error_propagate_prepend(errp, err);
+ * Function may use error system to return errors. In this case function
+ * defines Error **errp parameter, which should be the last one (except for
+ * functions which varidic argument list), which has the following API:
*
- * Avoid
- * error_propagate(errp, err);
- * error_prepend(errp, "Could not frobnicate '%s': ", name);
- * because this fails to prepend when @errp is &error_fatal.
+ * Caller may pass as errp:
+ * 1. &error_abort
+ * This means abort on any error
+ * 2. &error_fatal
+ * Exit with non-zero return code on error
+ * 3. NULL
+ * Ignore errors
+ * 4. Another value
+ * On error allocate error object and set errp
*
- * Create a new error and pass it to the caller:
- * error_setg(errp, "situation normal, all fouled up");
+ * Error API functions with Error ** (like error_setg) argument supports these
+ * rules, so user functions just need to use them appropriately (read below).
*
- * Call a function and receive an error from it:
+ * Simple pass error to the caller:
+ * error_setg(errp, "Some error");
+ *
+ * Subcall of another errp-based function, passing the error to the caller
+ * f(..., errp);
+ *
+ * == Checking success of subcall ==
+ *
+ * If function returns error code in addition to errp (which is recommended),
+ * you don't need any additional code, just do:
+ * int ret = f(..., errp);
+ * if (ret < 0) {
+ * ... handle error ...
+ * return ret;
+ * }
+ *
+ * If function returns nothing (which is not recommended API) and the only way
+ * to check success is checking errp, we must care about cases [1-3] above. We
+ * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:
+ *
+ * int our_func(..., Error **errp) {
+ * ERRP_AUTO_PROPAGATE();
+ * ...
+ * subcall(..., errp);
+ * if (*errp) {
+ * ...
+ * return -ERRNO;
+ * }
+ * ...
+ * }
+ *
+ * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
+ * so that it can be safely used (including dereferencing), and auto-propagates
+ * error to original errp on function end.
+ *
+ * In some cases, we need to check result of subcall, but do not want to
+ * propagate the Error object to our caller. In such cases we don't need
+ * ERRP_AUTO_PROPAGATE, but just a local Error object:
+ *
+ * Receive an error and not pass it:
* Error *err = NULL;
- * foo(arg, &err);
+ * subcall(arg, &err);
* if (err) {
* handle the error...
+ * error_free(err);
* }
*
+ * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
+ * error_propagate() instead of error_free()) was used to check and pass error
+ * to the caller. Now this is DEPRECATED* (see below).
+ *
+ * Note also, that if you want to use error_append_hint/error_prepend or their
+ * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
+ * error_fatal, you'll miss the chance to insert your additional information
+ * into Error object.
+ *
+ * In rare cases, we need to pass existing Error object to the caller by hand:
+ * error_propagate(errp, err);
+ *
+ * Pass an existing error to the caller with the message modified:
+ * error_propagate_prepend(errp, err);
+ *
+ *
* Call a function ignoring errors:
* foo(arg, NULL);
*
@@ -78,26 +140,6 @@
* Call a function treating errors as fatal:
* foo(arg, &error_fatal);
*
- * Receive an error and pass it on to the caller:
- * Error *err = NULL;
- * foo(arg, &err);
- * if (err) {
- * handle the error...
- * error_propagate(errp, err);
- * }
- * where Error **errp is a parameter, by convention the last one.
- *
- * Do *not* "optimize" this to
- * foo(arg, errp);
- * if (*errp) { // WRONG!
- * handle the error...
- * }
- * because errp may be NULL!
- *
- * But when all you do with the error is pass it on, please use
- * foo(arg, errp);
- * for readability.
- *
* Receive and accumulate multiple errors (first one wins):
* Error *err = NULL, *local_err = NULL;
* foo(arg, &err);
@@ -114,6 +156,61 @@
* handle the error...
* }
* because this may pass a non-null err to bar().
+ *
+ * DEPRECATED*
+ *
+ * The following pattern of receiving checking and passing the caller of the
+ * error by hand is deprecated now:
+ *
+ * Error *err = NULL;
+ * foo(arg, &err);
+ * if (err) {
+ * handle the error...
+ * error_propagate(errp, err);
+ * }
+ *
+ * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).
+ *
+ * The old pattern is deprecated because of two things:
+ *
+ * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
+ * local_err+error_propagate, the resulting coredump will refer to
+ * error_propagate and not to the place where error happened.
+ *
+ * 2. A lot of extra code of the same pattern
+ *
+ * How to update old code to use ERRP_AUTO_PROPAGATE?
+ *
+ * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
+ * than you may safely dereference errp to check errors and do not need any
+ * additional local Error variables or calls to error_propagate().
+ *
+ * Example:
+ *
+ * old code
+ *
+ * void fn(..., Error **errp) {
+ * Error *err = NULL;
+ * foo(arg, &err);
+ * if (err) {
+ * handle the error...
+ * error_propagate(errp, err);
+ * return;
+ * }
+ * ...
+ * }
+ *
+ * updated code
+ *
+ * void fn(..., Error **errp) {
+ * ERRP_AUTO_PROPAGATE();
+ * foo(arg, errp);
+ * if (*errp) {
+ * handle the error...
+ * return;
+ * }
+ * ...
+ * }
*/
#ifndef ERROR_H
@@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
ErrorClass err_class, const char *fmt, ...)
GCC_FMT_ATTR(6, 7);
+typedef struct ErrorPropagator {
+ Error *local_err;
+ Error **errp;
+} ErrorPropagator;
+
+static inline void error_propagator_cleanup(ErrorPropagator *prop)
+{
+ error_propagate(prop->errp, prop->local_err);
+}
+
+G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
+
+/*
+ * ERRP_AUTO_PROPAGATE
+ *
+ * This macro is created to be the first line of a function which use
+ * Error **errp parameter to report error. It's needed only in cases where we
+ * want to use error_prepend, error_append_hint or dereference *errp. It's
+ * still safe (but useless) in other cases.
+ *
+ * If errp is NULL or points to error_fatal, it is rewritten to point to a
+ * local Error object, which will be automatically propagated to the original
+ * errp on function exit (see error_propagator_cleanup).
+ *
+ * After invocation of this macro it is always safe to dereference errp
+ * (as it's not NULL anymore) and to add information by error_prepend or
+ * error_append_hint (as, if it was error_fatal, we swapped it with a
+ * local_error to be propagated on cleanup).
+ *
+ * Note: we don't wrap the error_abort case, as we want resulting coredump
+ * to point to the place where the error happened, not to error_propagate.
+ */
+#define ERRP_AUTO_PROPAGATE() \
+ g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \
+ do { \
+ if (!errp || errp == &error_fatal) { \
+ errp = &_auto_errp_prop.local_err; \
+ } \
+ } while (0)
+
/*
* Special error destination to abort on error.
* See error_setg() and error_propagate() for details.
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [PATCH v8 01/10] error: auto propagated local_err
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-06 8:55 ` Paul Durrant
-1 siblings, 0 replies; 77+ messages in thread
From: Paul Durrant @ 2020-03-06 8:55 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, Laszlo Ersek, qemu-block,
open list:All patches CC here, Markus Armbruster,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Greg Kurz, Max Reitz, Gerd Hoffmann,
Stefan Hajnoczi, Anthony Perard, xen-devel, Michael Roth,
Stefan Berger
On Fri, 6 Mar 2020 at 05:16, Vladimir Sementsov-Ogievskiy
<vsementsov@virtuozzo.com> wrote:
>
> Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of
> functions with an errp OUT parameter.
>
> It has three goals:
>
> 1. Fix issue with error_fatal and error_prepend/error_append_hint: user
> can't see this additional information, because exit() happens in
> error_setg earlier than information is added. [Reported by Greg Kurz]
>
> 2. Fix issue with error_abort and error_propagate: when we wrap
> error_abort by local_err+error_propagate, the resulting coredump will
> refer to error_propagate and not to the place where error happened.
> (the macro itself doesn't fix the issue, but it allows us to [3.] drop
> the local_err+error_propagate pattern, which will definitely fix the
> issue) [Reported by Kevin Wolf]
>
> 3. Drop local_err+error_propagate pattern, which is used to workaround
> void functions with errp parameter, when caller wants to know resulting
> status. (Note: actually these functions could be merely updated to
> return int error code).
>
> To achieve these goals, later patches will add invocations
> of this macro at the start of functions with either use
> error_prepend/error_append_hint (solving 1) or which use
> local_err+error_propagate to check errors, switching those
> functions to use *errp instead (solving 2 and 3).
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Paul Durrant <paul@xen.org>
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [Xen-devel] [PATCH v8 01/10] error: auto propagated local_err
@ 2020-03-06 8:55 ` Paul Durrant
0 siblings, 0 replies; 77+ messages in thread
From: Paul Durrant @ 2020-03-06 8:55 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, Laszlo Ersek, qemu-block,
open list:All patches CC here, Markus Armbruster,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Greg Kurz, Max Reitz, Gerd Hoffmann,
Stefan Hajnoczi, Anthony Perard, xen-devel, Eric Blake,
Michael Roth, Stefan Berger
On Fri, 6 Mar 2020 at 05:16, Vladimir Sementsov-Ogievskiy
<vsementsov@virtuozzo.com> wrote:
>
> Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of
> functions with an errp OUT parameter.
>
> It has three goals:
>
> 1. Fix issue with error_fatal and error_prepend/error_append_hint: user
> can't see this additional information, because exit() happens in
> error_setg earlier than information is added. [Reported by Greg Kurz]
>
> 2. Fix issue with error_abort and error_propagate: when we wrap
> error_abort by local_err+error_propagate, the resulting coredump will
> refer to error_propagate and not to the place where error happened.
> (the macro itself doesn't fix the issue, but it allows us to [3.] drop
> the local_err+error_propagate pattern, which will definitely fix the
> issue) [Reported by Kevin Wolf]
>
> 3. Drop local_err+error_propagate pattern, which is used to workaround
> void functions with errp parameter, when caller wants to know resulting
> status. (Note: actually these functions could be merely updated to
> return int error code).
>
> To achieve these goals, later patches will add invocations
> of this macro at the start of functions with either use
> error_prepend/error_append_hint (solving 1) or which use
> local_err+error_propagate to check errors, switching those
> functions to use *errp instead (solving 2 and 3).
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Paul Durrant <paul@xen.org>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 01/10] error: auto propagated local_err
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-06 10:28 ` Greg Kurz
-1 siblings, 0 replies; 77+ messages in thread
From: Greg Kurz @ 2020-03-06 10:28 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, Laszlo Ersek, qemu-block,
Paul Durrant, armbru, Philippe Mathieu-Daudé,
Christian Schoenebeck, qemu-devel, Max Reitz, Gerd Hoffmann,
Stefan Hajnoczi, Anthony Perard, xen-devel, Michael Roth,
Stefan Berger
On Fri, 6 Mar 2020 08:15:27 +0300
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> wrote:
> Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of
> functions with an errp OUT parameter.
>
> It has three goals:
>
> 1. Fix issue with error_fatal and error_prepend/error_append_hint: user
> can't see this additional information, because exit() happens in
> error_setg earlier than information is added. [Reported by Greg Kurz]
>
> 2. Fix issue with error_abort and error_propagate: when we wrap
> error_abort by local_err+error_propagate, the resulting coredump will
> refer to error_propagate and not to the place where error happened.
> (the macro itself doesn't fix the issue, but it allows us to [3.] drop
> the local_err+error_propagate pattern, which will definitely fix the
> issue) [Reported by Kevin Wolf]
>
> 3. Drop local_err+error_propagate pattern, which is used to workaround
> void functions with errp parameter, when caller wants to know resulting
> status. (Note: actually these functions could be merely updated to
> return int error code).
>
> To achieve these goals, later patches will add invocations
> of this macro at the start of functions with either use
> error_prepend/error_append_hint (solving 1) or which use
> local_err+error_propagate to check errors, switching those
> functions to use *errp instead (solving 2 and 3).
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>
Thanks for this impressive work Vladimir !
Reviewed-by: Greg Kurz <groug@kaod.org>
> Cc: Eric Blake <eblake@redhat.com>
> Cc: Kevin Wolf <kwolf@redhat.com>
> Cc: Max Reitz <mreitz@redhat.com>
> Cc: Greg Kurz <groug@kaod.org>
> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
> Cc: Stefano Stabellini <sstabellini@kernel.org>
> Cc: Anthony Perard <anthony.perard@citrix.com>
> Cc: Paul Durrant <paul@xen.org>
> Cc: Stefan Hajnoczi <stefanha@redhat.com>
> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Stefan Berger <stefanb@linux.ibm.com>
> Cc: Markus Armbruster <armbru@redhat.com>
> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
> Cc: qemu-block@nongnu.org
> Cc: qemu-devel@nongnu.org
> Cc: xen-devel@lists.xenproject.org
>
> include/qapi/error.h | 203 ++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 170 insertions(+), 33 deletions(-)
>
> diff --git a/include/qapi/error.h b/include/qapi/error.h
> index ad5b6e896d..bb9bcf02fb 100644
> --- a/include/qapi/error.h
> +++ b/include/qapi/error.h
> @@ -15,6 +15,8 @@
> /*
> * Error reporting system loosely patterned after Glib's GError.
> *
> + * = Deal with Error object =
> + *
> * Create an error:
> * error_setg(&err, "situation normal, all fouled up");
> *
> @@ -47,28 +49,88 @@
> * reporting it (primarily useful in testsuites):
> * error_free_or_abort(&err);
> *
> - * Pass an existing error to the caller:
> - * error_propagate(errp, err);
> - * where Error **errp is a parameter, by convention the last one.
> + * = Deal with Error ** function parameter =
> *
> - * Pass an existing error to the caller with the message modified:
> - * error_propagate_prepend(errp, err);
> + * Function may use error system to return errors. In this case function
> + * defines Error **errp parameter, which should be the last one (except for
> + * functions which varidic argument list), which has the following API:
> *
> - * Avoid
> - * error_propagate(errp, err);
> - * error_prepend(errp, "Could not frobnicate '%s': ", name);
> - * because this fails to prepend when @errp is &error_fatal.
> + * Caller may pass as errp:
> + * 1. &error_abort
> + * This means abort on any error
> + * 2. &error_fatal
> + * Exit with non-zero return code on error
> + * 3. NULL
> + * Ignore errors
> + * 4. Another value
> + * On error allocate error object and set errp
> *
> - * Create a new error and pass it to the caller:
> - * error_setg(errp, "situation normal, all fouled up");
> + * Error API functions with Error ** (like error_setg) argument supports these
> + * rules, so user functions just need to use them appropriately (read below).
> *
> - * Call a function and receive an error from it:
> + * Simple pass error to the caller:
> + * error_setg(errp, "Some error");
> + *
> + * Subcall of another errp-based function, passing the error to the caller
> + * f(..., errp);
> + *
> + * == Checking success of subcall ==
> + *
> + * If function returns error code in addition to errp (which is recommended),
> + * you don't need any additional code, just do:
> + * int ret = f(..., errp);
> + * if (ret < 0) {
> + * ... handle error ...
> + * return ret;
> + * }
> + *
> + * If function returns nothing (which is not recommended API) and the only way
> + * to check success is checking errp, we must care about cases [1-3] above. We
> + * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:
> + *
> + * int our_func(..., Error **errp) {
> + * ERRP_AUTO_PROPAGATE();
> + * ...
> + * subcall(..., errp);
> + * if (*errp) {
> + * ...
> + * return -ERRNO;
> + * }
> + * ...
> + * }
> + *
> + * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
> + * so that it can be safely used (including dereferencing), and auto-propagates
> + * error to original errp on function end.
> + *
> + * In some cases, we need to check result of subcall, but do not want to
> + * propagate the Error object to our caller. In such cases we don't need
> + * ERRP_AUTO_PROPAGATE, but just a local Error object:
> + *
> + * Receive an error and not pass it:
> * Error *err = NULL;
> - * foo(arg, &err);
> + * subcall(arg, &err);
> * if (err) {
> * handle the error...
> + * error_free(err);
> * }
> *
> + * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
> + * error_propagate() instead of error_free()) was used to check and pass error
> + * to the caller. Now this is DEPRECATED* (see below).
> + *
> + * Note also, that if you want to use error_append_hint/error_prepend or their
> + * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
> + * error_fatal, you'll miss the chance to insert your additional information
> + * into Error object.
> + *
> + * In rare cases, we need to pass existing Error object to the caller by hand:
> + * error_propagate(errp, err);
> + *
> + * Pass an existing error to the caller with the message modified:
> + * error_propagate_prepend(errp, err);
> + *
> + *
> * Call a function ignoring errors:
> * foo(arg, NULL);
> *
> @@ -78,26 +140,6 @@
> * Call a function treating errors as fatal:
> * foo(arg, &error_fatal);
> *
> - * Receive an error and pass it on to the caller:
> - * Error *err = NULL;
> - * foo(arg, &err);
> - * if (err) {
> - * handle the error...
> - * error_propagate(errp, err);
> - * }
> - * where Error **errp is a parameter, by convention the last one.
> - *
> - * Do *not* "optimize" this to
> - * foo(arg, errp);
> - * if (*errp) { // WRONG!
> - * handle the error...
> - * }
> - * because errp may be NULL!
> - *
> - * But when all you do with the error is pass it on, please use
> - * foo(arg, errp);
> - * for readability.
> - *
> * Receive and accumulate multiple errors (first one wins):
> * Error *err = NULL, *local_err = NULL;
> * foo(arg, &err);
> @@ -114,6 +156,61 @@
> * handle the error...
> * }
> * because this may pass a non-null err to bar().
> + *
> + * DEPRECATED*
> + *
> + * The following pattern of receiving checking and passing the caller of the
> + * error by hand is deprecated now:
> + *
> + * Error *err = NULL;
> + * foo(arg, &err);
> + * if (err) {
> + * handle the error...
> + * error_propagate(errp, err);
> + * }
> + *
> + * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).
> + *
> + * The old pattern is deprecated because of two things:
> + *
> + * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
> + * local_err+error_propagate, the resulting coredump will refer to
> + * error_propagate and not to the place where error happened.
> + *
> + * 2. A lot of extra code of the same pattern
> + *
> + * How to update old code to use ERRP_AUTO_PROPAGATE?
> + *
> + * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
> + * than you may safely dereference errp to check errors and do not need any
> + * additional local Error variables or calls to error_propagate().
> + *
> + * Example:
> + *
> + * old code
> + *
> + * void fn(..., Error **errp) {
> + * Error *err = NULL;
> + * foo(arg, &err);
> + * if (err) {
> + * handle the error...
> + * error_propagate(errp, err);
> + * return;
> + * }
> + * ...
> + * }
> + *
> + * updated code
> + *
> + * void fn(..., Error **errp) {
> + * ERRP_AUTO_PROPAGATE();
> + * foo(arg, errp);
> + * if (*errp) {
> + * handle the error...
> + * return;
> + * }
> + * ...
> + * }
> */
>
> #ifndef ERROR_H
> @@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
> ErrorClass err_class, const char *fmt, ...)
> GCC_FMT_ATTR(6, 7);
>
> +typedef struct ErrorPropagator {
> + Error *local_err;
> + Error **errp;
> +} ErrorPropagator;
> +
> +static inline void error_propagator_cleanup(ErrorPropagator *prop)
> +{
> + error_propagate(prop->errp, prop->local_err);
> +}
> +
> +G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
> +
> +/*
> + * ERRP_AUTO_PROPAGATE
> + *
> + * This macro is created to be the first line of a function which use
> + * Error **errp parameter to report error. It's needed only in cases where we
> + * want to use error_prepend, error_append_hint or dereference *errp. It's
> + * still safe (but useless) in other cases.
> + *
> + * If errp is NULL or points to error_fatal, it is rewritten to point to a
> + * local Error object, which will be automatically propagated to the original
> + * errp on function exit (see error_propagator_cleanup).
> + *
> + * After invocation of this macro it is always safe to dereference errp
> + * (as it's not NULL anymore) and to add information by error_prepend or
> + * error_append_hint (as, if it was error_fatal, we swapped it with a
> + * local_error to be propagated on cleanup).
> + *
> + * Note: we don't wrap the error_abort case, as we want resulting coredump
> + * to point to the place where the error happened, not to error_propagate.
> + */
> +#define ERRP_AUTO_PROPAGATE() \
> + g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \
> + do { \
> + if (!errp || errp == &error_fatal) { \
> + errp = &_auto_errp_prop.local_err; \
> + } \
> + } while (0)
> +
> /*
> * Special error destination to abort on error.
> * See error_setg() and error_propagate() for details.
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 01/10] error: auto propagated local_err
@ 2020-03-06 10:28 ` Greg Kurz
0 siblings, 0 replies; 77+ messages in thread
From: Greg Kurz @ 2020-03-06 10:28 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, Laszlo Ersek, qemu-block,
Paul Durrant, armbru, Philippe Mathieu-Daudé,
Christian Schoenebeck, qemu-devel, Max Reitz, Gerd Hoffmann,
Stefan Hajnoczi, Anthony Perard, xen-devel, Eric Blake,
Michael Roth, Stefan Berger
On Fri, 6 Mar 2020 08:15:27 +0300
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> wrote:
> Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of
> functions with an errp OUT parameter.
>
> It has three goals:
>
> 1. Fix issue with error_fatal and error_prepend/error_append_hint: user
> can't see this additional information, because exit() happens in
> error_setg earlier than information is added. [Reported by Greg Kurz]
>
> 2. Fix issue with error_abort and error_propagate: when we wrap
> error_abort by local_err+error_propagate, the resulting coredump will
> refer to error_propagate and not to the place where error happened.
> (the macro itself doesn't fix the issue, but it allows us to [3.] drop
> the local_err+error_propagate pattern, which will definitely fix the
> issue) [Reported by Kevin Wolf]
>
> 3. Drop local_err+error_propagate pattern, which is used to workaround
> void functions with errp parameter, when caller wants to know resulting
> status. (Note: actually these functions could be merely updated to
> return int error code).
>
> To achieve these goals, later patches will add invocations
> of this macro at the start of functions with either use
> error_prepend/error_append_hint (solving 1) or which use
> local_err+error_propagate to check errors, switching those
> functions to use *errp instead (solving 2 and 3).
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>
Thanks for this impressive work Vladimir !
Reviewed-by: Greg Kurz <groug@kaod.org>
> Cc: Eric Blake <eblake@redhat.com>
> Cc: Kevin Wolf <kwolf@redhat.com>
> Cc: Max Reitz <mreitz@redhat.com>
> Cc: Greg Kurz <groug@kaod.org>
> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
> Cc: Stefano Stabellini <sstabellini@kernel.org>
> Cc: Anthony Perard <anthony.perard@citrix.com>
> Cc: Paul Durrant <paul@xen.org>
> Cc: Stefan Hajnoczi <stefanha@redhat.com>
> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Stefan Berger <stefanb@linux.ibm.com>
> Cc: Markus Armbruster <armbru@redhat.com>
> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
> Cc: qemu-block@nongnu.org
> Cc: qemu-devel@nongnu.org
> Cc: xen-devel@lists.xenproject.org
>
> include/qapi/error.h | 203 ++++++++++++++++++++++++++++++++++++-------
> 1 file changed, 170 insertions(+), 33 deletions(-)
>
> diff --git a/include/qapi/error.h b/include/qapi/error.h
> index ad5b6e896d..bb9bcf02fb 100644
> --- a/include/qapi/error.h
> +++ b/include/qapi/error.h
> @@ -15,6 +15,8 @@
> /*
> * Error reporting system loosely patterned after Glib's GError.
> *
> + * = Deal with Error object =
> + *
> * Create an error:
> * error_setg(&err, "situation normal, all fouled up");
> *
> @@ -47,28 +49,88 @@
> * reporting it (primarily useful in testsuites):
> * error_free_or_abort(&err);
> *
> - * Pass an existing error to the caller:
> - * error_propagate(errp, err);
> - * where Error **errp is a parameter, by convention the last one.
> + * = Deal with Error ** function parameter =
> *
> - * Pass an existing error to the caller with the message modified:
> - * error_propagate_prepend(errp, err);
> + * Function may use error system to return errors. In this case function
> + * defines Error **errp parameter, which should be the last one (except for
> + * functions which varidic argument list), which has the following API:
> *
> - * Avoid
> - * error_propagate(errp, err);
> - * error_prepend(errp, "Could not frobnicate '%s': ", name);
> - * because this fails to prepend when @errp is &error_fatal.
> + * Caller may pass as errp:
> + * 1. &error_abort
> + * This means abort on any error
> + * 2. &error_fatal
> + * Exit with non-zero return code on error
> + * 3. NULL
> + * Ignore errors
> + * 4. Another value
> + * On error allocate error object and set errp
> *
> - * Create a new error and pass it to the caller:
> - * error_setg(errp, "situation normal, all fouled up");
> + * Error API functions with Error ** (like error_setg) argument supports these
> + * rules, so user functions just need to use them appropriately (read below).
> *
> - * Call a function and receive an error from it:
> + * Simple pass error to the caller:
> + * error_setg(errp, "Some error");
> + *
> + * Subcall of another errp-based function, passing the error to the caller
> + * f(..., errp);
> + *
> + * == Checking success of subcall ==
> + *
> + * If function returns error code in addition to errp (which is recommended),
> + * you don't need any additional code, just do:
> + * int ret = f(..., errp);
> + * if (ret < 0) {
> + * ... handle error ...
> + * return ret;
> + * }
> + *
> + * If function returns nothing (which is not recommended API) and the only way
> + * to check success is checking errp, we must care about cases [1-3] above. We
> + * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:
> + *
> + * int our_func(..., Error **errp) {
> + * ERRP_AUTO_PROPAGATE();
> + * ...
> + * subcall(..., errp);
> + * if (*errp) {
> + * ...
> + * return -ERRNO;
> + * }
> + * ...
> + * }
> + *
> + * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
> + * so that it can be safely used (including dereferencing), and auto-propagates
> + * error to original errp on function end.
> + *
> + * In some cases, we need to check result of subcall, but do not want to
> + * propagate the Error object to our caller. In such cases we don't need
> + * ERRP_AUTO_PROPAGATE, but just a local Error object:
> + *
> + * Receive an error and not pass it:
> * Error *err = NULL;
> - * foo(arg, &err);
> + * subcall(arg, &err);
> * if (err) {
> * handle the error...
> + * error_free(err);
> * }
> *
> + * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
> + * error_propagate() instead of error_free()) was used to check and pass error
> + * to the caller. Now this is DEPRECATED* (see below).
> + *
> + * Note also, that if you want to use error_append_hint/error_prepend or their
> + * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
> + * error_fatal, you'll miss the chance to insert your additional information
> + * into Error object.
> + *
> + * In rare cases, we need to pass existing Error object to the caller by hand:
> + * error_propagate(errp, err);
> + *
> + * Pass an existing error to the caller with the message modified:
> + * error_propagate_prepend(errp, err);
> + *
> + *
> * Call a function ignoring errors:
> * foo(arg, NULL);
> *
> @@ -78,26 +140,6 @@
> * Call a function treating errors as fatal:
> * foo(arg, &error_fatal);
> *
> - * Receive an error and pass it on to the caller:
> - * Error *err = NULL;
> - * foo(arg, &err);
> - * if (err) {
> - * handle the error...
> - * error_propagate(errp, err);
> - * }
> - * where Error **errp is a parameter, by convention the last one.
> - *
> - * Do *not* "optimize" this to
> - * foo(arg, errp);
> - * if (*errp) { // WRONG!
> - * handle the error...
> - * }
> - * because errp may be NULL!
> - *
> - * But when all you do with the error is pass it on, please use
> - * foo(arg, errp);
> - * for readability.
> - *
> * Receive and accumulate multiple errors (first one wins):
> * Error *err = NULL, *local_err = NULL;
> * foo(arg, &err);
> @@ -114,6 +156,61 @@
> * handle the error...
> * }
> * because this may pass a non-null err to bar().
> + *
> + * DEPRECATED*
> + *
> + * The following pattern of receiving checking and passing the caller of the
> + * error by hand is deprecated now:
> + *
> + * Error *err = NULL;
> + * foo(arg, &err);
> + * if (err) {
> + * handle the error...
> + * error_propagate(errp, err);
> + * }
> + *
> + * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).
> + *
> + * The old pattern is deprecated because of two things:
> + *
> + * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
> + * local_err+error_propagate, the resulting coredump will refer to
> + * error_propagate and not to the place where error happened.
> + *
> + * 2. A lot of extra code of the same pattern
> + *
> + * How to update old code to use ERRP_AUTO_PROPAGATE?
> + *
> + * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
> + * than you may safely dereference errp to check errors and do not need any
> + * additional local Error variables or calls to error_propagate().
> + *
> + * Example:
> + *
> + * old code
> + *
> + * void fn(..., Error **errp) {
> + * Error *err = NULL;
> + * foo(arg, &err);
> + * if (err) {
> + * handle the error...
> + * error_propagate(errp, err);
> + * return;
> + * }
> + * ...
> + * }
> + *
> + * updated code
> + *
> + * void fn(..., Error **errp) {
> + * ERRP_AUTO_PROPAGATE();
> + * foo(arg, errp);
> + * if (*errp) {
> + * handle the error...
> + * return;
> + * }
> + * ...
> + * }
> */
>
> #ifndef ERROR_H
> @@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
> ErrorClass err_class, const char *fmt, ...)
> GCC_FMT_ATTR(6, 7);
>
> +typedef struct ErrorPropagator {
> + Error *local_err;
> + Error **errp;
> +} ErrorPropagator;
> +
> +static inline void error_propagator_cleanup(ErrorPropagator *prop)
> +{
> + error_propagate(prop->errp, prop->local_err);
> +}
> +
> +G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
> +
> +/*
> + * ERRP_AUTO_PROPAGATE
> + *
> + * This macro is created to be the first line of a function which use
> + * Error **errp parameter to report error. It's needed only in cases where we
> + * want to use error_prepend, error_append_hint or dereference *errp. It's
> + * still safe (but useless) in other cases.
> + *
> + * If errp is NULL or points to error_fatal, it is rewritten to point to a
> + * local Error object, which will be automatically propagated to the original
> + * errp on function exit (see error_propagator_cleanup).
> + *
> + * After invocation of this macro it is always safe to dereference errp
> + * (as it's not NULL anymore) and to add information by error_prepend or
> + * error_append_hint (as, if it was error_fatal, we swapped it with a
> + * local_error to be propagated on cleanup).
> + *
> + * Note: we don't wrap the error_abort case, as we want resulting coredump
> + * to point to the place where the error happened, not to error_propagate.
> + */
> +#define ERRP_AUTO_PROPAGATE() \
> + g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \
> + do { \
> + if (!errp || errp == &error_fatal) { \
> + errp = &_auto_errp_prop.local_err; \
> + } \
> + } while (0)
> +
> /*
> * Special error destination to abort on error.
> * See error_setg() and error_propagate() for details.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 01/10] error: auto propagated local_err
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-06 11:02 ` Alberto Garcia
-1 siblings, 0 replies; 77+ messages in thread
From: Alberto Garcia @ 2020-03-06 11:02 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, vsementsov, Michael Roth, qemu-block, Paul Durrant,
Laszlo Ersek, Christian Schoenebeck, Greg Kurz, armbru,
Stefano Stabellini, Gerd Hoffmann, Stefan Hajnoczi,
Anthony Perard, xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
On Fri 06 Mar 2020 06:15:27 AM CET, Vladimir Sementsov-Ogievskiy wrote:
Sorry I just gave a quick look at these patches and noticed this:
> + * Function may use error system to return errors. In this case function
> + * defines Error **errp parameter, which should be the last one (except for
> + * functions which varidic argument list), which has the following API:
I don't think that "functions which varidic argument list" is correct
English. Perhaps something like "except for variadic functions" or
"except for functions with a variable number of arguments".
Berto
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [Xen-devel] [PATCH v8 01/10] error: auto propagated local_err
@ 2020-03-06 11:02 ` Alberto Garcia
0 siblings, 0 replies; 77+ messages in thread
From: Alberto Garcia @ 2020-03-06 11:02 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, vsementsov, Michael Roth, qemu-block, Paul Durrant,
Laszlo Ersek, Christian Schoenebeck, Greg Kurz, armbru,
Stefano Stabellini, Gerd Hoffmann, Stefan Hajnoczi,
Anthony Perard, xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
On Fri 06 Mar 2020 06:15:27 AM CET, Vladimir Sementsov-Ogievskiy wrote:
Sorry I just gave a quick look at these patches and noticed this:
> + * Function may use error system to return errors. In this case function
> + * defines Error **errp parameter, which should be the last one (except for
> + * functions which varidic argument list), which has the following API:
I don't think that "functions which varidic argument list" is correct
English. Perhaps something like "except for variadic functions" or
"except for functions with a variable number of arguments".
Berto
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 01/10] error: auto propagated local_err
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-06 12:37 ` Eric Blake
-1 siblings, 0 replies; 77+ messages in thread
From: Eric Blake @ 2020-03-06 12:37 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, Greg Kurz,
armbru, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
On 3/5/20 11:15 PM, Vladimir Sementsov-Ogievskiy wrote:
> Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of
> functions with an errp OUT parameter.
As an aid to writing imperative-style commit messages, I like to prepend
an implicit "Apply this patch to..." before the user's text, to see if
things still make sense. By that construct, this paragraph might read
better as:
Introduce a new ERRP_AUTO_PROPAGATE macro, ...
>
> It has three goals:
>
> 1. Fix issue with error_fatal and error_prepend/error_append_hint: user
> can't see this additional information, because exit() happens in
> error_setg earlier than information is added. [Reported by Greg Kurz]
>
> 2. Fix issue with error_abort and error_propagate: when we wrap
> error_abort by local_err+error_propagate, the resulting coredump will
> refer to error_propagate and not to the place where error happened.
> (the macro itself doesn't fix the issue, but it allows us to [3.] drop
> the local_err+error_propagate pattern, which will definitely fix the
> issue) [Reported by Kevin Wolf]
>
> 3. Drop local_err+error_propagate pattern, which is used to workaround
> void functions with errp parameter, when caller wants to know resulting
> status. (Note: actually these functions could be merely updated to
> return int error code).
>
> To achieve these goals, later patches will add invocations
> of this macro at the start of functions with either use
> error_prepend/error_append_hint (solving 1) or which use
> local_err+error_propagate to check errors, switching those
> functions to use *errp instead (solving 2 and 3).
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
I have lots of grammar suggestions for the comments (and I know Markus
is generally okay doing wording tweaks, so it may still end up different
than my suggestions):
> +++ b/include/qapi/error.h
> @@ -15,6 +15,8 @@
> /*
> * Error reporting system loosely patterned after Glib's GError.
> *
> + * = Deal with Error object =
> + *
> * Create an error:
> * error_setg(&err, "situation normal, all fouled up");
> *
> @@ -47,28 +49,88 @@
> * reporting it (primarily useful in testsuites):
> * error_free_or_abort(&err);
> *
> - * Pass an existing error to the caller:
> - * error_propagate(errp, err);
> - * where Error **errp is a parameter, by convention the last one.
> + * = Deal with Error ** function parameter =
> *
> - * Pass an existing error to the caller with the message modified:
> - * error_propagate_prepend(errp, err);
> + * Function may use error system to return errors. In this case function
> + * defines Error **errp parameter, which should be the last one (except for
> + * functions which varidic argument list), which has the following API:
A function may use the error system to return errors. In this case, the
function defines an Error **errp parameter, by convention the last one
(with exceptions for functions using ... or va_list).
> *
> - * Avoid
> - * error_propagate(errp, err);
> - * error_prepend(errp, "Could not frobnicate '%s': ", name);
> - * because this fails to prepend when @errp is &error_fatal.
> + * Caller may pass as errp:
The caller may then pass in the following errp values:
> + * 1. &error_abort
> + * This means abort on any error
Any error will result in abort()
> + * 2. &error_fatal
> + * Exit with non-zero return code on error
Any error will result in exit() with a non-zero status
> + * 3. NULL
> + * Ignore errors
Any error will be ignored
> + * 4. Another value
4. The address of a NULL-initialized Error *err
> + * On error allocate error object and set errp
Any error will populate errp with an error object
> *
> - * Create a new error and pass it to the caller:
> - * error_setg(errp, "situation normal, all fouled up");
> + * Error API functions with Error ** (like error_setg) argument supports these
> + * rules, so user functions just need to use them appropriately (read below).
The following rules then implement the correct semantics desired by the
caller.
> *
> - * Call a function and receive an error from it:
> + * Simple pass error to the caller:
Create a new error to pass to the caller:
> + * error_setg(errp, "Some error");
You lost the fun wording in Markus' earlier example ("situation normal,
all fouled up").
> + *
> + * Subcall of another errp-based function, passing the error to the caller
> + * f(..., errp);
Calling another errp-based function:
> + *
> + * == Checking success of subcall ==
> + *
> + * If function returns error code in addition to errp (which is recommended),
If a function returns a value indicating an error in addition to setting
errp (which is recommended), then
> + * you don't need any additional code, just do:
> + * int ret = f(..., errp);
> + * if (ret < 0) {
> + * ... handle error ...
> + * return ret;
> + * }
> + *
> + * If function returns nothing (which is not recommended API) and the only way
> + * to check success is checking errp, we must care about cases [1-3] above. We
> + * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:
If a function returns nothing (not recommended for new code), the only
way to check success is by consulting errp; doing this safely requires
the use of the ERRP_AUTO_PROPAGATE macro, like this:
> + *
> + * int our_func(..., Error **errp) {
> + * ERRP_AUTO_PROPAGATE();
> + * ...
> + * subcall(..., errp);
> + * if (*errp) {
> + * ...
> + * return -ERRNO;
do we want ERRNO capitalized here?
> + * }
> + * ...
> + * }
> + *
> + * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
> + * so that it can be safely used (including dereferencing), and auto-propagates
> + * error to original errp on function end.
ERRP_AUTO_PROPAGATE takes care of wrapping the original errp as needed,
so that the rest of the function can directly use errp (including
dereferencing), where any errors will then be propagated on to the
original errp when leaving the function.
> + *
> + * In some cases, we need to check result of subcall, but do not want to
> + * propagate the Error object to our caller. In such cases we don't need
> + * ERRP_AUTO_PROPAGATE, but just a local Error object:
> + *
> + * Receive an error and not pass it:
> * Error *err = NULL;
> - * foo(arg, &err);
> + * subcall(arg, &err);
> * if (err) {
> * handle the error...
> + * error_free(err);
> * }
> *
> + * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
> + * error_propagate() instead of error_free()) was used to check and pass error
> + * to the caller. Now this is DEPRECATED* (see below).
Hmm - if we bulk-convert the entire tree, then there won't be any
deprecated uses to be worth documenting. But if we do keep this paragraph:
Note that older code that did not use ERRP_AUTO_PROPAGATE would instead
need a local Err variable and the use of error_propagate() to properly
handle all possible caller values of errp.
> + *
> + * Note also, that if you want to use error_append_hint/error_prepend or their
> + * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
> + * error_fatal, you'll miss the chance to insert your additional information
> + * into Error object.
Note that any function that wants to modify an error object, such as by
calling error_append_hint or error_prepend, must use
ERRP_AUTO_PROPAGATE, in order for a caller's use of &error_fatal to see
the additional information.
> + *
> + * In rare cases, we need to pass existing Error object to the caller by hand:
> + * error_propagate(errp, err);
> + *
> + * Pass an existing error to the caller with the message modified:
> + * error_propagate_prepend(errp, err);
> + *
> + *
> * Call a function ignoring errors:
> * foo(arg, NULL);
> *
> @@ -78,26 +140,6 @@
> * Call a function treating errors as fatal:
> * foo(arg, &error_fatal);
> *
> - * Receive an error and pass it on to the caller:
> - * Error *err = NULL;
> - * foo(arg, &err);
> - * if (err) {
> - * handle the error...
> - * error_propagate(errp, err);
> - * }
> - * where Error **errp is a parameter, by convention the last one.
> - *
> - * Do *not* "optimize" this to
> - * foo(arg, errp);
> - * if (*errp) { // WRONG!
> - * handle the error...
> - * }
> - * because errp may be NULL!
> - *
> - * But when all you do with the error is pass it on, please use
> - * foo(arg, errp);
> - * for readability.
> - *
> * Receive and accumulate multiple errors (first one wins):
> * Error *err = NULL, *local_err = NULL;
> * foo(arg, &err);
> @@ -114,6 +156,61 @@
> * handle the error...
> * }
> * because this may pass a non-null err to bar().
> + *
> + * DEPRECATED*
> + *
Again, I'm not sure we need this section in the codebase if we do a bulk
conversion. But moving it to the commit message is still useful.
> + * The following pattern of receiving checking and passing the caller of the
> + * error by hand is deprecated now:
The following pattern of receiving, checking, and then forwarding an
error to the caller by hand is now deprecated:
> + *
> + * Error *err = NULL;
> + * foo(arg, &err);
> + * if (err) {
> + * handle the error...
> + * error_propagate(errp, err);
> + * }
> + *
> + * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).
Drop "(defined below)".
> + *
> + * The old pattern is deprecated because of two things:
> + *
> + * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
> + * local_err+error_propagate, the resulting coredump will refer to
> + * error_propagate and not to the place where error happened.
> + *
> + * 2. A lot of extra code of the same pattern
> + *
> + * How to update old code to use ERRP_AUTO_PROPAGATE?
> + *
> + * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
> + * than you may safely dereference errp to check errors and do not need any
> + * additional local Error variables or calls to error_propagate().
> + *
> + * Example:
> + *
> + * old code
> + *
> + * void fn(..., Error **errp) {
> + * Error *err = NULL;
> + * foo(arg, &err);
> + * if (err) {
> + * handle the error...
> + * error_propagate(errp, err);
> + * return;
> + * }
> + * ...
> + * }
> + *
> + * updated code
> + *
> + * void fn(..., Error **errp) {
> + * ERRP_AUTO_PROPAGATE();
> + * foo(arg, errp);
> + * if (*errp) {
> + * handle the error...
> + * return;
> + * }
> + * ...
> + * }
Again, I'm thinking the above example is more useful in the commit
message instead of permanently in the .h file.
> */
>
> #ifndef ERROR_H
> @@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
> ErrorClass err_class, const char *fmt, ...)
> GCC_FMT_ATTR(6, 7);
>
> +typedef struct ErrorPropagator {
> + Error *local_err;
> + Error **errp;
> +} ErrorPropagator;
> +
> +static inline void error_propagator_cleanup(ErrorPropagator *prop)
> +{
> + error_propagate(prop->errp, prop->local_err);
> +}
> +
> +G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
> +
> +/*
> + * ERRP_AUTO_PROPAGATE
> + *
> + * This macro is created to be the first line of a function which use
> + * Error **errp parameter to report error. It's needed only in cases where we
> + * want to use error_prepend, error_append_hint or dereference *errp. It's
> + * still safe (but useless) in other cases.
This macro exists to assist with proper error handling in a function
which uses an Error **errp parameter. It must be used as the first line
of a function which modifies an error (with error_prepend,
error_append_hint, or similar) or which wants to dereference *errp. It
is still safe (but useless) to use in other functions.
> + *
> + * If errp is NULL or points to error_fatal, it is rewritten to point to a
> + * local Error object, which will be automatically propagated to the original
> + * errp on function exit (see error_propagator_cleanup).
> + *
> + * After invocation of this macro it is always safe to dereference errp
> + * (as it's not NULL anymore) and to add information by error_prepend or
> + * error_append_hint (as, if it was error_fatal, we swapped it with a
> + * local_error to be propagated on cleanup).
> + *
> + * Note: we don't wrap the error_abort case, as we want resulting coredump
> + * to point to the place where the error happened, not to error_propagate.
> + */
> +#define ERRP_AUTO_PROPAGATE() \
> + g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \
> + do { \
> + if (!errp || errp == &error_fatal) { \
> + errp = &_auto_errp_prop.local_err; \
> + } \
> + } while (0)
> +
> /*
> * Special error destination to abort on error.
> * See error_setg() and error_propagate() for details.
>
The macro itself looks correct. I'll leave it up to Markus how to handle
the comment text, but you can add:
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 01/10] error: auto propagated local_err
@ 2020-03-06 12:37 ` Eric Blake
0 siblings, 0 replies; 77+ messages in thread
From: Eric Blake @ 2020-03-06 12:37 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, Greg Kurz,
armbru, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
On 3/5/20 11:15 PM, Vladimir Sementsov-Ogievskiy wrote:
> Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of
> functions with an errp OUT parameter.
As an aid to writing imperative-style commit messages, I like to prepend
an implicit "Apply this patch to..." before the user's text, to see if
things still make sense. By that construct, this paragraph might read
better as:
Introduce a new ERRP_AUTO_PROPAGATE macro, ...
>
> It has three goals:
>
> 1. Fix issue with error_fatal and error_prepend/error_append_hint: user
> can't see this additional information, because exit() happens in
> error_setg earlier than information is added. [Reported by Greg Kurz]
>
> 2. Fix issue with error_abort and error_propagate: when we wrap
> error_abort by local_err+error_propagate, the resulting coredump will
> refer to error_propagate and not to the place where error happened.
> (the macro itself doesn't fix the issue, but it allows us to [3.] drop
> the local_err+error_propagate pattern, which will definitely fix the
> issue) [Reported by Kevin Wolf]
>
> 3. Drop local_err+error_propagate pattern, which is used to workaround
> void functions with errp parameter, when caller wants to know resulting
> status. (Note: actually these functions could be merely updated to
> return int error code).
>
> To achieve these goals, later patches will add invocations
> of this macro at the start of functions with either use
> error_prepend/error_append_hint (solving 1) or which use
> local_err+error_propagate to check errors, switching those
> functions to use *errp instead (solving 2 and 3).
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
I have lots of grammar suggestions for the comments (and I know Markus
is generally okay doing wording tweaks, so it may still end up different
than my suggestions):
> +++ b/include/qapi/error.h
> @@ -15,6 +15,8 @@
> /*
> * Error reporting system loosely patterned after Glib's GError.
> *
> + * = Deal with Error object =
> + *
> * Create an error:
> * error_setg(&err, "situation normal, all fouled up");
> *
> @@ -47,28 +49,88 @@
> * reporting it (primarily useful in testsuites):
> * error_free_or_abort(&err);
> *
> - * Pass an existing error to the caller:
> - * error_propagate(errp, err);
> - * where Error **errp is a parameter, by convention the last one.
> + * = Deal with Error ** function parameter =
> *
> - * Pass an existing error to the caller with the message modified:
> - * error_propagate_prepend(errp, err);
> + * Function may use error system to return errors. In this case function
> + * defines Error **errp parameter, which should be the last one (except for
> + * functions which varidic argument list), which has the following API:
A function may use the error system to return errors. In this case, the
function defines an Error **errp parameter, by convention the last one
(with exceptions for functions using ... or va_list).
> *
> - * Avoid
> - * error_propagate(errp, err);
> - * error_prepend(errp, "Could not frobnicate '%s': ", name);
> - * because this fails to prepend when @errp is &error_fatal.
> + * Caller may pass as errp:
The caller may then pass in the following errp values:
> + * 1. &error_abort
> + * This means abort on any error
Any error will result in abort()
> + * 2. &error_fatal
> + * Exit with non-zero return code on error
Any error will result in exit() with a non-zero status
> + * 3. NULL
> + * Ignore errors
Any error will be ignored
> + * 4. Another value
4. The address of a NULL-initialized Error *err
> + * On error allocate error object and set errp
Any error will populate errp with an error object
> *
> - * Create a new error and pass it to the caller:
> - * error_setg(errp, "situation normal, all fouled up");
> + * Error API functions with Error ** (like error_setg) argument supports these
> + * rules, so user functions just need to use them appropriately (read below).
The following rules then implement the correct semantics desired by the
caller.
> *
> - * Call a function and receive an error from it:
> + * Simple pass error to the caller:
Create a new error to pass to the caller:
> + * error_setg(errp, "Some error");
You lost the fun wording in Markus' earlier example ("situation normal,
all fouled up").
> + *
> + * Subcall of another errp-based function, passing the error to the caller
> + * f(..., errp);
Calling another errp-based function:
> + *
> + * == Checking success of subcall ==
> + *
> + * If function returns error code in addition to errp (which is recommended),
If a function returns a value indicating an error in addition to setting
errp (which is recommended), then
> + * you don't need any additional code, just do:
> + * int ret = f(..., errp);
> + * if (ret < 0) {
> + * ... handle error ...
> + * return ret;
> + * }
> + *
> + * If function returns nothing (which is not recommended API) and the only way
> + * to check success is checking errp, we must care about cases [1-3] above. We
> + * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:
If a function returns nothing (not recommended for new code), the only
way to check success is by consulting errp; doing this safely requires
the use of the ERRP_AUTO_PROPAGATE macro, like this:
> + *
> + * int our_func(..., Error **errp) {
> + * ERRP_AUTO_PROPAGATE();
> + * ...
> + * subcall(..., errp);
> + * if (*errp) {
> + * ...
> + * return -ERRNO;
do we want ERRNO capitalized here?
> + * }
> + * ...
> + * }
> + *
> + * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
> + * so that it can be safely used (including dereferencing), and auto-propagates
> + * error to original errp on function end.
ERRP_AUTO_PROPAGATE takes care of wrapping the original errp as needed,
so that the rest of the function can directly use errp (including
dereferencing), where any errors will then be propagated on to the
original errp when leaving the function.
> + *
> + * In some cases, we need to check result of subcall, but do not want to
> + * propagate the Error object to our caller. In such cases we don't need
> + * ERRP_AUTO_PROPAGATE, but just a local Error object:
> + *
> + * Receive an error and not pass it:
> * Error *err = NULL;
> - * foo(arg, &err);
> + * subcall(arg, &err);
> * if (err) {
> * handle the error...
> + * error_free(err);
> * }
> *
> + * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
> + * error_propagate() instead of error_free()) was used to check and pass error
> + * to the caller. Now this is DEPRECATED* (see below).
Hmm - if we bulk-convert the entire tree, then there won't be any
deprecated uses to be worth documenting. But if we do keep this paragraph:
Note that older code that did not use ERRP_AUTO_PROPAGATE would instead
need a local Err variable and the use of error_propagate() to properly
handle all possible caller values of errp.
> + *
> + * Note also, that if you want to use error_append_hint/error_prepend or their
> + * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
> + * error_fatal, you'll miss the chance to insert your additional information
> + * into Error object.
Note that any function that wants to modify an error object, such as by
calling error_append_hint or error_prepend, must use
ERRP_AUTO_PROPAGATE, in order for a caller's use of &error_fatal to see
the additional information.
> + *
> + * In rare cases, we need to pass existing Error object to the caller by hand:
> + * error_propagate(errp, err);
> + *
> + * Pass an existing error to the caller with the message modified:
> + * error_propagate_prepend(errp, err);
> + *
> + *
> * Call a function ignoring errors:
> * foo(arg, NULL);
> *
> @@ -78,26 +140,6 @@
> * Call a function treating errors as fatal:
> * foo(arg, &error_fatal);
> *
> - * Receive an error and pass it on to the caller:
> - * Error *err = NULL;
> - * foo(arg, &err);
> - * if (err) {
> - * handle the error...
> - * error_propagate(errp, err);
> - * }
> - * where Error **errp is a parameter, by convention the last one.
> - *
> - * Do *not* "optimize" this to
> - * foo(arg, errp);
> - * if (*errp) { // WRONG!
> - * handle the error...
> - * }
> - * because errp may be NULL!
> - *
> - * But when all you do with the error is pass it on, please use
> - * foo(arg, errp);
> - * for readability.
> - *
> * Receive and accumulate multiple errors (first one wins):
> * Error *err = NULL, *local_err = NULL;
> * foo(arg, &err);
> @@ -114,6 +156,61 @@
> * handle the error...
> * }
> * because this may pass a non-null err to bar().
> + *
> + * DEPRECATED*
> + *
Again, I'm not sure we need this section in the codebase if we do a bulk
conversion. But moving it to the commit message is still useful.
> + * The following pattern of receiving checking and passing the caller of the
> + * error by hand is deprecated now:
The following pattern of receiving, checking, and then forwarding an
error to the caller by hand is now deprecated:
> + *
> + * Error *err = NULL;
> + * foo(arg, &err);
> + * if (err) {
> + * handle the error...
> + * error_propagate(errp, err);
> + * }
> + *
> + * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).
Drop "(defined below)".
> + *
> + * The old pattern is deprecated because of two things:
> + *
> + * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
> + * local_err+error_propagate, the resulting coredump will refer to
> + * error_propagate and not to the place where error happened.
> + *
> + * 2. A lot of extra code of the same pattern
> + *
> + * How to update old code to use ERRP_AUTO_PROPAGATE?
> + *
> + * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
> + * than you may safely dereference errp to check errors and do not need any
> + * additional local Error variables or calls to error_propagate().
> + *
> + * Example:
> + *
> + * old code
> + *
> + * void fn(..., Error **errp) {
> + * Error *err = NULL;
> + * foo(arg, &err);
> + * if (err) {
> + * handle the error...
> + * error_propagate(errp, err);
> + * return;
> + * }
> + * ...
> + * }
> + *
> + * updated code
> + *
> + * void fn(..., Error **errp) {
> + * ERRP_AUTO_PROPAGATE();
> + * foo(arg, errp);
> + * if (*errp) {
> + * handle the error...
> + * return;
> + * }
> + * ...
> + * }
Again, I'm thinking the above example is more useful in the commit
message instead of permanently in the .h file.
> */
>
> #ifndef ERROR_H
> @@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
> ErrorClass err_class, const char *fmt, ...)
> GCC_FMT_ATTR(6, 7);
>
> +typedef struct ErrorPropagator {
> + Error *local_err;
> + Error **errp;
> +} ErrorPropagator;
> +
> +static inline void error_propagator_cleanup(ErrorPropagator *prop)
> +{
> + error_propagate(prop->errp, prop->local_err);
> +}
> +
> +G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
> +
> +/*
> + * ERRP_AUTO_PROPAGATE
> + *
> + * This macro is created to be the first line of a function which use
> + * Error **errp parameter to report error. It's needed only in cases where we
> + * want to use error_prepend, error_append_hint or dereference *errp. It's
> + * still safe (but useless) in other cases.
This macro exists to assist with proper error handling in a function
which uses an Error **errp parameter. It must be used as the first line
of a function which modifies an error (with error_prepend,
error_append_hint, or similar) or which wants to dereference *errp. It
is still safe (but useless) to use in other functions.
> + *
> + * If errp is NULL or points to error_fatal, it is rewritten to point to a
> + * local Error object, which will be automatically propagated to the original
> + * errp on function exit (see error_propagator_cleanup).
> + *
> + * After invocation of this macro it is always safe to dereference errp
> + * (as it's not NULL anymore) and to add information by error_prepend or
> + * error_append_hint (as, if it was error_fatal, we swapped it with a
> + * local_error to be propagated on cleanup).
> + *
> + * Note: we don't wrap the error_abort case, as we want resulting coredump
> + * to point to the place where the error happened, not to error_propagate.
> + */
> +#define ERRP_AUTO_PROPAGATE() \
> + g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \
> + do { \
> + if (!errp || errp == &error_fatal) { \
> + errp = &_auto_errp_prop.local_err; \
> + } \
> + } while (0)
> +
> /*
> * Special error destination to abort on error.
> * See error_setg() and error_propagate() for details.
>
The macro itself looks correct. I'll leave it up to Markus how to handle
the comment text, but you can add:
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 01/10] error: auto propagated local_err
2020-03-06 12:37 ` [Xen-devel] " Eric Blake
@ 2020-03-06 13:00 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 13:00 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, Greg Kurz,
armbru, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
06.03.2020 15:37, Eric Blake wrote:
> On 3/5/20 11:15 PM, Vladimir Sementsov-Ogievskiy wrote:
>> Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of
>> functions with an errp OUT parameter.
>
> As an aid to writing imperative-style commit messages, I like to prepend an implicit "Apply this patch to..." before the user's text, to see if things still make sense. By that construct, this paragraph might read better as:
>
> Introduce a new ERRP_AUTO_PROPAGATE macro, ...
>
>>
>> It has three goals:
>>
>> 1. Fix issue with error_fatal and error_prepend/error_append_hint: user
>> can't see this additional information, because exit() happens in
>> error_setg earlier than information is added. [Reported by Greg Kurz]
>>
>> 2. Fix issue with error_abort and error_propagate: when we wrap
>> error_abort by local_err+error_propagate, the resulting coredump will
>> refer to error_propagate and not to the place where error happened.
>> (the macro itself doesn't fix the issue, but it allows us to [3.] drop
>> the local_err+error_propagate pattern, which will definitely fix the
>> issue) [Reported by Kevin Wolf]
>>
>> 3. Drop local_err+error_propagate pattern, which is used to workaround
>> void functions with errp parameter, when caller wants to know resulting
>> status. (Note: actually these functions could be merely updated to
>> return int error code).
>>
>> To achieve these goals, later patches will add invocations
>> of this macro at the start of functions with either use
>> error_prepend/error_append_hint (solving 1) or which use
>> local_err+error_propagate to check errors, switching those
>> functions to use *errp instead (solving 2 and 3).
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>
> I have lots of grammar suggestions for the comments (and I know Markus is generally okay doing wording tweaks, so it may still end up different than my suggestions):
>
>> +++ b/include/qapi/error.h
>> @@ -15,6 +15,8 @@
>> /*
>> * Error reporting system loosely patterned after Glib's GError.
>> *
>> + * = Deal with Error object =
>> + *
>> * Create an error:
>> * error_setg(&err, "situation normal, all fouled up");
>> *
>> @@ -47,28 +49,88 @@
>> * reporting it (primarily useful in testsuites):
>> * error_free_or_abort(&err);
>> *
>> - * Pass an existing error to the caller:
>> - * error_propagate(errp, err);
>> - * where Error **errp is a parameter, by convention the last one.
>> + * = Deal with Error ** function parameter =
>> *
>> - * Pass an existing error to the caller with the message modified:
>> - * error_propagate_prepend(errp, err);
>> + * Function may use error system to return errors. In this case function
>> + * defines Error **errp parameter, which should be the last one (except for
>> + * functions which varidic argument list), which has the following API:
>
> A function may use the error system to return errors. In this case, the function defines an Error **errp parameter, by convention the last one (with exceptions for functions using ... or va_list).
>
>> *
>> - * Avoid
>> - * error_propagate(errp, err);
>> - * error_prepend(errp, "Could not frobnicate '%s': ", name);
>> - * because this fails to prepend when @errp is &error_fatal.
>> + * Caller may pass as errp:
>
> The caller may then pass in the following errp values:
>
>> + * 1. &error_abort
>> + * This means abort on any error
>
> Any error will result in abort()
>
>> + * 2. &error_fatal
>> + * Exit with non-zero return code on error
>
> Any error will result in exit() with a non-zero status
>
>> + * 3. NULL
>> + * Ignore errors
>
> Any error will be ignored
>
>> + * 4. Another value
>
> 4. The address of a NULL-initialized Error *err
>
>> + * On error allocate error object and set errp
>
> Any error will populate errp with an error object
>
>> *
>> - * Create a new error and pass it to the caller:
>> - * error_setg(errp, "situation normal, all fouled up");
>> + * Error API functions with Error ** (like error_setg) argument supports these
>> + * rules, so user functions just need to use them appropriately (read below).
>
> The following rules then implement the correct semantics desired by the caller.
>
>> *
>> - * Call a function and receive an error from it:
>> + * Simple pass error to the caller:
>
> Create a new error to pass to the caller:
>
>> + * error_setg(errp, "Some error");
>
> You lost the fun wording in Markus' earlier example ("situation normal, all fouled up").
>
>> + *
>> + * Subcall of another errp-based function, passing the error to the caller
>> + * f(..., errp);
>
> Calling another errp-based function:
>
>> + *
>> + * == Checking success of subcall ==
>> + *
>> + * If function returns error code in addition to errp (which is recommended),
>
> If a function returns a value indicating an error in addition to setting errp (which is recommended), then
>
>> + * you don't need any additional code, just do:
>> + * int ret = f(..., errp);
>> + * if (ret < 0) {
>> + * ... handle error ...
>> + * return ret;
>> + * }
>> + *
>> + * If function returns nothing (which is not recommended API) and the only way
>> + * to check success is checking errp, we must care about cases [1-3] above. We
>> + * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:
>
> If a function returns nothing (not recommended for new code), the only way to check success is by consulting errp; doing this safely requires the use of the ERRP_AUTO_PROPAGATE macro, like this:
>
>
>> + *
>> + * int our_func(..., Error **errp) {
>> + * ERRP_AUTO_PROPAGATE();
>> + * ...
>> + * subcall(..., errp);
>> + * if (*errp) {
>> + * ...
>> + * return -ERRNO;
>
> do we want ERRNO capitalized here?
I wrote it capitalized to not conflict with errno variable, to make it obvious that this is just a pseudo-code. But yes, it looks weird anyway.
We can write -errno here, or just -EINVAL (I hope, nobady will thing, that EINVAL should always be used :)
>
>> + * }
>> + * ...
>> + * }
>> + *
>> + * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
>> + * so that it can be safely used (including dereferencing), and auto-propagates
>> + * error to original errp on function end.
>
> ERRP_AUTO_PROPAGATE takes care of wrapping the original errp as needed, so that the rest of the function can directly use errp (including dereferencing), where any errors will then be propagated on to the original errp when leaving the function.
>
>> + *
>> + * In some cases, we need to check result of subcall, but do not want to
>> + * propagate the Error object to our caller. In such cases we don't need
>> + * ERRP_AUTO_PROPAGATE, but just a local Error object:
>> + *
>> + * Receive an error and not pass it:
>> * Error *err = NULL;
>> - * foo(arg, &err);
>> + * subcall(arg, &err);
>> * if (err) {
>> * handle the error...
>> + * error_free(err);
>> * }
>> *
>> + * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
>> + * error_propagate() instead of error_free()) was used to check and pass error
>> + * to the caller. Now this is DEPRECATED* (see below).
>
> Hmm - if we bulk-convert the entire tree, then there won't be any deprecated uses to be worth documenting.
Yes, I think we'll drop it after everything is converted.
> But if we do keep this paragraph:
>
> Note that older code that did not use ERRP_AUTO_PROPAGATE would instead need a local Err variable and the use of error_propagate() to properly handle all possible caller values of errp.
>
>> + *
>> + * Note also, that if you want to use error_append_hint/error_prepend or their
>> + * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
>> + * error_fatal, you'll miss the chance to insert your additional information
>> + * into Error object.
>
> Note that any function that wants to modify an error object, such as by calling error_append_hint or error_prepend, must use ERRP_AUTO_PROPAGATE, in order for a caller's use of &error_fatal to see the additional information.
>
>> + *
>> + * In rare cases, we need to pass existing Error object to the caller by hand:
>> + * error_propagate(errp, err);
>> + *
>> + * Pass an existing error to the caller with the message modified:
>> + * error_propagate_prepend(errp, err);
>> + *
>> + *
>> * Call a function ignoring errors:
>> * foo(arg, NULL);
>> *
>> @@ -78,26 +140,6 @@
>> * Call a function treating errors as fatal:
>> * foo(arg, &error_fatal);
>> *
>> - * Receive an error and pass it on to the caller:
>> - * Error *err = NULL;
>> - * foo(arg, &err);
>> - * if (err) {
>> - * handle the error...
>> - * error_propagate(errp, err);
>> - * }
>> - * where Error **errp is a parameter, by convention the last one.
>> - *
>> - * Do *not* "optimize" this to
>> - * foo(arg, errp);
>> - * if (*errp) { // WRONG!
>> - * handle the error...
>> - * }
>> - * because errp may be NULL!
>> - *
>> - * But when all you do with the error is pass it on, please use
>> - * foo(arg, errp);
>> - * for readability.
>> - *
>> * Receive and accumulate multiple errors (first one wins):
>> * Error *err = NULL, *local_err = NULL;
>> * foo(arg, &err);
>> @@ -114,6 +156,61 @@
>> * handle the error...
>> * }
>> * because this may pass a non-null err to bar().
>> + *
>> + * DEPRECATED*
>> + *
>
> Again, I'm not sure we need this section in the codebase if we do a bulk conversion. But moving it to the commit message is still useful.
Conversion can't be atomic anyway, there would be several series. But I'm not against dropping the paragraph, so, it's up to Markus.
>
>> + * The following pattern of receiving checking and passing the caller of the
>> + * error by hand is deprecated now:
>
> The following pattern of receiving, checking, and then forwarding an error to the caller by hand is now deprecated:
>
>> + *
>> + * Error *err = NULL;
>> + * foo(arg, &err);
>> + * if (err) {
>> + * handle the error...
>> + * error_propagate(errp, err);
>> + * }
>> + *
>> + * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).
>
> Drop "(defined below)".
>
>> + *
>> + * The old pattern is deprecated because of two things:
>> + *
>> + * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
>> + * local_err+error_propagate, the resulting coredump will refer to
>> + * error_propagate and not to the place where error happened.
>> + *
>> + * 2. A lot of extra code of the same pattern
>> + *
>> + * How to update old code to use ERRP_AUTO_PROPAGATE?
>> + *
>> + * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
>> + * than you may safely dereference errp to check errors and do not need any
>> + * additional local Error variables or calls to error_propagate().
>> + *
>> + * Example:
>> + *
>> + * old code
>> + *
>> + * void fn(..., Error **errp) {
>> + * Error *err = NULL;
>> + * foo(arg, &err);
>> + * if (err) {
>> + * handle the error...
>> + * error_propagate(errp, err);
>> + * return;
>> + * }
>> + * ...
>> + * }
>> + *
>> + * updated code
>> + *
>> + * void fn(..., Error **errp) {
>> + * ERRP_AUTO_PROPAGATE();
>> + * foo(arg, errp);
>> + * if (*errp) {
>> + * handle the error...
>> + * return;
>> + * }
>> + * ...
>> + * }
>
> Again, I'm thinking the above example is more useful in the commit message instead of permanently in the .h file.
>
>> */
>> #ifndef ERROR_H
>> @@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
>> ErrorClass err_class, const char *fmt, ...)
>> GCC_FMT_ATTR(6, 7);
>> +typedef struct ErrorPropagator {
>> + Error *local_err;
>> + Error **errp;
>> +} ErrorPropagator;
>> +
>> +static inline void error_propagator_cleanup(ErrorPropagator *prop)
>> +{
>> + error_propagate(prop->errp, prop->local_err);
>> +}
>> +
>> +G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
>> +
>> +/*
>> + * ERRP_AUTO_PROPAGATE
>> + *
>> + * This macro is created to be the first line of a function which use
>> + * Error **errp parameter to report error. It's needed only in cases where we
>> + * want to use error_prepend, error_append_hint or dereference *errp. It's
>> + * still safe (but useless) in other cases.
>
> This macro exists to assist with proper error handling in a function which uses an Error **errp parameter. It must be used as the first line of a function which modifies an error (with error_prepend, error_append_hint, or similar) or which wants to dereference *errp. It is still safe (but useless) to use in other functions.
>
>> + *
>> + * If errp is NULL or points to error_fatal, it is rewritten to point to a
>> + * local Error object, which will be automatically propagated to the original
>> + * errp on function exit (see error_propagator_cleanup).
>> + *
>> + * After invocation of this macro it is always safe to dereference errp
>> + * (as it's not NULL anymore) and to add information by error_prepend or
>> + * error_append_hint (as, if it was error_fatal, we swapped it with a
>> + * local_error to be propagated on cleanup).
>> + *
>> + * Note: we don't wrap the error_abort case, as we want resulting coredump
>> + * to point to the place where the error happened, not to error_propagate.
>> + */
>> +#define ERRP_AUTO_PROPAGATE() \
>> + g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \
>> + do { \
>> + if (!errp || errp == &error_fatal) { \
>> + errp = &_auto_errp_prop.local_err; \
>> + } \
>> + } while (0)
>> +
>> /*
>> * Special error destination to abort on error.
>> * See error_setg() and error_propagate() for details.
>>
>
> The macro itself looks correct. I'll leave it up to Markus how to handle the comment text, but you can add:
>
> Reviewed-by: Eric Blake <eblake@redhat.com>
>
Thanks for reviewing and wording suggestions, all looks good, as always.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 01/10] error: auto propagated local_err
@ 2020-03-06 13:00 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 13:00 UTC (permalink / raw)
To: Eric Blake, qemu-devel
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, Greg Kurz,
armbru, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
06.03.2020 15:37, Eric Blake wrote:
> On 3/5/20 11:15 PM, Vladimir Sementsov-Ogievskiy wrote:
>> Here is introduced ERRP_AUTO_PROPAGATE macro, to be used at start of
>> functions with an errp OUT parameter.
>
> As an aid to writing imperative-style commit messages, I like to prepend an implicit "Apply this patch to..." before the user's text, to see if things still make sense. By that construct, this paragraph might read better as:
>
> Introduce a new ERRP_AUTO_PROPAGATE macro, ...
>
>>
>> It has three goals:
>>
>> 1. Fix issue with error_fatal and error_prepend/error_append_hint: user
>> can't see this additional information, because exit() happens in
>> error_setg earlier than information is added. [Reported by Greg Kurz]
>>
>> 2. Fix issue with error_abort and error_propagate: when we wrap
>> error_abort by local_err+error_propagate, the resulting coredump will
>> refer to error_propagate and not to the place where error happened.
>> (the macro itself doesn't fix the issue, but it allows us to [3.] drop
>> the local_err+error_propagate pattern, which will definitely fix the
>> issue) [Reported by Kevin Wolf]
>>
>> 3. Drop local_err+error_propagate pattern, which is used to workaround
>> void functions with errp parameter, when caller wants to know resulting
>> status. (Note: actually these functions could be merely updated to
>> return int error code).
>>
>> To achieve these goals, later patches will add invocations
>> of this macro at the start of functions with either use
>> error_prepend/error_append_hint (solving 1) or which use
>> local_err+error_propagate to check errors, switching those
>> functions to use *errp instead (solving 2 and 3).
>>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>
> I have lots of grammar suggestions for the comments (and I know Markus is generally okay doing wording tweaks, so it may still end up different than my suggestions):
>
>> +++ b/include/qapi/error.h
>> @@ -15,6 +15,8 @@
>> /*
>> * Error reporting system loosely patterned after Glib's GError.
>> *
>> + * = Deal with Error object =
>> + *
>> * Create an error:
>> * error_setg(&err, "situation normal, all fouled up");
>> *
>> @@ -47,28 +49,88 @@
>> * reporting it (primarily useful in testsuites):
>> * error_free_or_abort(&err);
>> *
>> - * Pass an existing error to the caller:
>> - * error_propagate(errp, err);
>> - * where Error **errp is a parameter, by convention the last one.
>> + * = Deal with Error ** function parameter =
>> *
>> - * Pass an existing error to the caller with the message modified:
>> - * error_propagate_prepend(errp, err);
>> + * Function may use error system to return errors. In this case function
>> + * defines Error **errp parameter, which should be the last one (except for
>> + * functions which varidic argument list), which has the following API:
>
> A function may use the error system to return errors. In this case, the function defines an Error **errp parameter, by convention the last one (with exceptions for functions using ... or va_list).
>
>> *
>> - * Avoid
>> - * error_propagate(errp, err);
>> - * error_prepend(errp, "Could not frobnicate '%s': ", name);
>> - * because this fails to prepend when @errp is &error_fatal.
>> + * Caller may pass as errp:
>
> The caller may then pass in the following errp values:
>
>> + * 1. &error_abort
>> + * This means abort on any error
>
> Any error will result in abort()
>
>> + * 2. &error_fatal
>> + * Exit with non-zero return code on error
>
> Any error will result in exit() with a non-zero status
>
>> + * 3. NULL
>> + * Ignore errors
>
> Any error will be ignored
>
>> + * 4. Another value
>
> 4. The address of a NULL-initialized Error *err
>
>> + * On error allocate error object and set errp
>
> Any error will populate errp with an error object
>
>> *
>> - * Create a new error and pass it to the caller:
>> - * error_setg(errp, "situation normal, all fouled up");
>> + * Error API functions with Error ** (like error_setg) argument supports these
>> + * rules, so user functions just need to use them appropriately (read below).
>
> The following rules then implement the correct semantics desired by the caller.
>
>> *
>> - * Call a function and receive an error from it:
>> + * Simple pass error to the caller:
>
> Create a new error to pass to the caller:
>
>> + * error_setg(errp, "Some error");
>
> You lost the fun wording in Markus' earlier example ("situation normal, all fouled up").
>
>> + *
>> + * Subcall of another errp-based function, passing the error to the caller
>> + * f(..., errp);
>
> Calling another errp-based function:
>
>> + *
>> + * == Checking success of subcall ==
>> + *
>> + * If function returns error code in addition to errp (which is recommended),
>
> If a function returns a value indicating an error in addition to setting errp (which is recommended), then
>
>> + * you don't need any additional code, just do:
>> + * int ret = f(..., errp);
>> + * if (ret < 0) {
>> + * ... handle error ...
>> + * return ret;
>> + * }
>> + *
>> + * If function returns nothing (which is not recommended API) and the only way
>> + * to check success is checking errp, we must care about cases [1-3] above. We
>> + * need to use macro ERRP_AUTO_PROPAGATE (see below for details) like this:
>
> If a function returns nothing (not recommended for new code), the only way to check success is by consulting errp; doing this safely requires the use of the ERRP_AUTO_PROPAGATE macro, like this:
>
>
>> + *
>> + * int our_func(..., Error **errp) {
>> + * ERRP_AUTO_PROPAGATE();
>> + * ...
>> + * subcall(..., errp);
>> + * if (*errp) {
>> + * ...
>> + * return -ERRNO;
>
> do we want ERRNO capitalized here?
I wrote it capitalized to not conflict with errno variable, to make it obvious that this is just a pseudo-code. But yes, it looks weird anyway.
We can write -errno here, or just -EINVAL (I hope, nobady will thing, that EINVAL should always be used :)
>
>> + * }
>> + * ...
>> + * }
>> + *
>> + * ERRP_AUTO_PROPAGATE cares about Error ** API, wraps original errp if needed,
>> + * so that it can be safely used (including dereferencing), and auto-propagates
>> + * error to original errp on function end.
>
> ERRP_AUTO_PROPAGATE takes care of wrapping the original errp as needed, so that the rest of the function can directly use errp (including dereferencing), where any errors will then be propagated on to the original errp when leaving the function.
>
>> + *
>> + * In some cases, we need to check result of subcall, but do not want to
>> + * propagate the Error object to our caller. In such cases we don't need
>> + * ERRP_AUTO_PROPAGATE, but just a local Error object:
>> + *
>> + * Receive an error and not pass it:
>> * Error *err = NULL;
>> - * foo(arg, &err);
>> + * subcall(arg, &err);
>> * if (err) {
>> * handle the error...
>> + * error_free(err);
>> * }
>> *
>> + * Note, that before ERRP_AUTO_PROPAGATE introduction the pattern above (with
>> + * error_propagate() instead of error_free()) was used to check and pass error
>> + * to the caller. Now this is DEPRECATED* (see below).
>
> Hmm - if we bulk-convert the entire tree, then there won't be any deprecated uses to be worth documenting.
Yes, I think we'll drop it after everything is converted.
> But if we do keep this paragraph:
>
> Note that older code that did not use ERRP_AUTO_PROPAGATE would instead need a local Err variable and the use of error_propagate() to properly handle all possible caller values of errp.
>
>> + *
>> + * Note also, that if you want to use error_append_hint/error_prepend or their
>> + * variants, you must use ERRP_AUTO_PROPAGATE too. Otherwise, in case of
>> + * error_fatal, you'll miss the chance to insert your additional information
>> + * into Error object.
>
> Note that any function that wants to modify an error object, such as by calling error_append_hint or error_prepend, must use ERRP_AUTO_PROPAGATE, in order for a caller's use of &error_fatal to see the additional information.
>
>> + *
>> + * In rare cases, we need to pass existing Error object to the caller by hand:
>> + * error_propagate(errp, err);
>> + *
>> + * Pass an existing error to the caller with the message modified:
>> + * error_propagate_prepend(errp, err);
>> + *
>> + *
>> * Call a function ignoring errors:
>> * foo(arg, NULL);
>> *
>> @@ -78,26 +140,6 @@
>> * Call a function treating errors as fatal:
>> * foo(arg, &error_fatal);
>> *
>> - * Receive an error and pass it on to the caller:
>> - * Error *err = NULL;
>> - * foo(arg, &err);
>> - * if (err) {
>> - * handle the error...
>> - * error_propagate(errp, err);
>> - * }
>> - * where Error **errp is a parameter, by convention the last one.
>> - *
>> - * Do *not* "optimize" this to
>> - * foo(arg, errp);
>> - * if (*errp) { // WRONG!
>> - * handle the error...
>> - * }
>> - * because errp may be NULL!
>> - *
>> - * But when all you do with the error is pass it on, please use
>> - * foo(arg, errp);
>> - * for readability.
>> - *
>> * Receive and accumulate multiple errors (first one wins):
>> * Error *err = NULL, *local_err = NULL;
>> * foo(arg, &err);
>> @@ -114,6 +156,61 @@
>> * handle the error...
>> * }
>> * because this may pass a non-null err to bar().
>> + *
>> + * DEPRECATED*
>> + *
>
> Again, I'm not sure we need this section in the codebase if we do a bulk conversion. But moving it to the commit message is still useful.
Conversion can't be atomic anyway, there would be several series. But I'm not against dropping the paragraph, so, it's up to Markus.
>
>> + * The following pattern of receiving checking and passing the caller of the
>> + * error by hand is deprecated now:
>
> The following pattern of receiving, checking, and then forwarding an error to the caller by hand is now deprecated:
>
>> + *
>> + * Error *err = NULL;
>> + * foo(arg, &err);
>> + * if (err) {
>> + * handle the error...
>> + * error_propagate(errp, err);
>> + * }
>> + *
>> + * Instead, use ERRP_AUTO_PROPAGATE macro (defined below).
>
> Drop "(defined below)".
>
>> + *
>> + * The old pattern is deprecated because of two things:
>> + *
>> + * 1. Issue with error_abort and error_propagate: when we wrap error_abort by
>> + * local_err+error_propagate, the resulting coredump will refer to
>> + * error_propagate and not to the place where error happened.
>> + *
>> + * 2. A lot of extra code of the same pattern
>> + *
>> + * How to update old code to use ERRP_AUTO_PROPAGATE?
>> + *
>> + * All you need is to add ERRP_AUTO_PROPAGATE() invocation at function start,
>> + * than you may safely dereference errp to check errors and do not need any
>> + * additional local Error variables or calls to error_propagate().
>> + *
>> + * Example:
>> + *
>> + * old code
>> + *
>> + * void fn(..., Error **errp) {
>> + * Error *err = NULL;
>> + * foo(arg, &err);
>> + * if (err) {
>> + * handle the error...
>> + * error_propagate(errp, err);
>> + * return;
>> + * }
>> + * ...
>> + * }
>> + *
>> + * updated code
>> + *
>> + * void fn(..., Error **errp) {
>> + * ERRP_AUTO_PROPAGATE();
>> + * foo(arg, errp);
>> + * if (*errp) {
>> + * handle the error...
>> + * return;
>> + * }
>> + * ...
>> + * }
>
> Again, I'm thinking the above example is more useful in the commit message instead of permanently in the .h file.
>
>> */
>> #ifndef ERROR_H
>> @@ -322,6 +419,46 @@ void error_set_internal(Error **errp,
>> ErrorClass err_class, const char *fmt, ...)
>> GCC_FMT_ATTR(6, 7);
>> +typedef struct ErrorPropagator {
>> + Error *local_err;
>> + Error **errp;
>> +} ErrorPropagator;
>> +
>> +static inline void error_propagator_cleanup(ErrorPropagator *prop)
>> +{
>> + error_propagate(prop->errp, prop->local_err);
>> +}
>> +
>> +G_DEFINE_AUTO_CLEANUP_CLEAR_FUNC(ErrorPropagator, error_propagator_cleanup);
>> +
>> +/*
>> + * ERRP_AUTO_PROPAGATE
>> + *
>> + * This macro is created to be the first line of a function which use
>> + * Error **errp parameter to report error. It's needed only in cases where we
>> + * want to use error_prepend, error_append_hint or dereference *errp. It's
>> + * still safe (but useless) in other cases.
>
> This macro exists to assist with proper error handling in a function which uses an Error **errp parameter. It must be used as the first line of a function which modifies an error (with error_prepend, error_append_hint, or similar) or which wants to dereference *errp. It is still safe (but useless) to use in other functions.
>
>> + *
>> + * If errp is NULL or points to error_fatal, it is rewritten to point to a
>> + * local Error object, which will be automatically propagated to the original
>> + * errp on function exit (see error_propagator_cleanup).
>> + *
>> + * After invocation of this macro it is always safe to dereference errp
>> + * (as it's not NULL anymore) and to add information by error_prepend or
>> + * error_append_hint (as, if it was error_fatal, we swapped it with a
>> + * local_error to be propagated on cleanup).
>> + *
>> + * Note: we don't wrap the error_abort case, as we want resulting coredump
>> + * to point to the place where the error happened, not to error_propagate.
>> + */
>> +#define ERRP_AUTO_PROPAGATE() \
>> + g_auto(ErrorPropagator) _auto_errp_prop = {.errp = errp}; \
>> + do { \
>> + if (!errp || errp == &error_fatal) { \
>> + errp = &_auto_errp_prop.local_err; \
>> + } \
>> + } while (0)
>> +
>> /*
>> * Special error destination to abort on error.
>> * See error_setg() and error_propagate() for details.
>>
>
> The macro itself looks correct. I'll leave it up to Markus how to handle the comment text, but you can add:
>
> Reviewed-by: Eric Blake <eblake@redhat.com>
>
Thanks for reviewing and wording suggestions, all looks good, as always.
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Laszlo Ersek, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Greg Kurz, armbru, Stefano Stabellini,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Michael Roth, Stefan Berger
Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
does corresponding changes in code (look for details in
include/qapi/error.h)
Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
Cc: Eric Blake <eblake@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Max Reitz <mreitz@redhat.com>
Cc: Greg Kurz <groug@kaod.org>
Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Paul Durrant <paul@xen.org>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
Cc: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org
Cc: xen-devel@lists.xenproject.org
include/qapi/error.h | 3 +
scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
2 files changed, 234 insertions(+)
create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
diff --git a/include/qapi/error.h b/include/qapi/error.h
index bb9bcf02fb..fbfc6f1c0b 100644
--- a/include/qapi/error.h
+++ b/include/qapi/error.h
@@ -211,6 +211,9 @@
* }
* ...
* }
+ *
+ * For mass conversion use script
+ * scripts/coccinelle/auto-propagated-errp.cocci
*/
#ifndef ERROR_H
diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
new file mode 100644
index 0000000000..bff274bd6d
--- /dev/null
+++ b/scripts/coccinelle/auto-propagated-errp.cocci
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program. If not, see <http://www.gnu.org/licenses/>.
+//
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+// --max-width 80 blockdev-nbd.c qemu-nbd.c \
+// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
+
+// Switch unusual (Error **) parameter names to errp
+// (this is necessary to use ERRP_AUTO_PROPAGATE).
+//
+// Disable optional_qualifier to skip functions with "Error *const *errp"
+// parameter.
+//
+// Skip functions with "assert(_errp && *_errp)" statement, as they have
+// non generic semantics and may have unusual Error ** argument name for purpose
+// (like nbd_iter_channel_error()).
+//
+// Skip util/error.c to not touch, for example, error_propagate and
+// error_propagate_prepend().
+@ depends on !(file in "util/error.c") disable optional_qualifier@
+identifier fn;
+identifier _errp != errp;
+@@
+
+ fn(...,
+- Error **_errp
++ Error **errp
+ ,...)
+ {
+(
+ ... when != assert(_errp && *_errp)
+&
+ <...
+- _errp
++ errp
+ ...>
+)
+ }
+
+// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
+//
+// Note, that without "when any" final "..." may not want to mach something
+// matched by previous pattern, i.e. the rule will not match double
+// error_prepend in control flow like in vfio_set_irq_signaling().
+//
+// Note, "exists" says that we want apply rule even if it matches not on
+// all possible control flows (otherwise, it will not match standard pattern
+// when error_propagate() call is in if branch).
+@ disable optional_qualifier exists@
+identifier fn, local_err, errp;
+@@
+
+ fn(..., Error **errp, ...)
+ {
++ ERRP_AUTO_PROPAGATE();
+ ... when != ERRP_AUTO_PROPAGATE();
+(
+ error_append_hint(errp, ...);
+|
+ error_prepend(errp, ...);
+|
+ error_vprepend(errp, ...);
+|
+ Error *local_err = NULL;
+ ...
+(
+ error_propagate_prepend(errp, local_err, ...);
+|
+ error_propagate(errp, local_err);
+)
+)
+ ... when any
+ }
+
+
+// Match scenarios with propagation of local error to errp.
+@rule1 disable optional_qualifier exists@
+identifier fn, local_err;
+symbol errp;
+@@
+
+ fn(..., Error **errp, ...)
+ {
+ ...
+ Error *local_err = NULL;
+ ...
+(
+ error_propagate_prepend(errp, local_err, ...);
+|
+ error_propagate(errp, local_err);
+)
+ ...
+ }
+
+// Convert special case with goto in separate.
+// We can probably merge this into the following hunk with help of ( | )
+// operator, but it significantly reduce performance on block.c parsing (or it
+// hangs, I don't know)
+//
+// Note interesting thing: if we don't do it here, and try to fixup "out: }"
+// things later after all transformations (the rule will be the same, just
+// without error_propagate() call), coccinelle fails to match this "out: }".
+@@
+identifier rule1.fn, rule1.local_err, out;
+symbol errp;
+@@
+
+ fn(...)
+ {
+ <...
+- goto out;
++ return;
+ ...>
+- out:
+- error_propagate(errp, local_err);
+ }
+
+// Convert most of local_err related staff.
+//
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).
+//
+// Note also that errp-cleaning functions
+// error_free_errp
+// error_report_errp
+// error_reportf_errp
+// warn_report_errp
+// warn_reportf_errp
+// are not yet implemented. They must call corresponding Error* - freeing
+// function and then set *errp to NULL, to avoid further propagation to
+// original errp (consider ERRP_AUTO_PROPAGATE in use).
+// For example, error_free_errp may look like this:
+//
+// void error_free_errp(Error **errp)
+// {
+// error_free(*errp);
+// *errp = NULL;
+// }
+@ exists@
+identifier rule1.fn, rule1.local_err;
+expression list args;
+symbol errp;
+@@
+
+ fn(...)
+ {
+ <...
+(
+- Error *local_err = NULL;
+|
+
+// Convert error clearing functions
+(
+- error_free(local_err);
++ error_free_errp(errp);
+|
+- error_report_err(local_err);
++ error_report_errp(errp);
+|
+- error_reportf_err(local_err, args);
++ error_reportf_errp(errp, args);
+|
+- warn_report_err(local_err);
++ warn_report_errp(errp);
+|
+- warn_reportf_err(local_err, args);
++ warn_reportf_errp(errp, args);
+)
+?- local_err = NULL;
+
+|
+- error_propagate_prepend(errp, local_err, args);
++ error_prepend(errp, args);
+|
+- error_propagate(errp, local_err);
+|
+- &local_err
++ errp
+)
+ ...>
+ }
+
+// Convert remaining local_err usage. It should be different kinds of error
+// checking in if operators. We can't merge this into previous hunk, as this
+// conflicts with other substitutions in it (at least with "- local_err = NULL").
+@@
+identifier rule1.fn, rule1.local_err;
+symbol errp;
+@@
+
+ fn(...)
+ {
+ <...
+- local_err
++ *errp
+ ...>
+ }
+
+// Always use the same patter for checking error
+@@
+identifier rule1.fn;
+symbol errp;
+@@
+
+ fn(...)
+ {
+ <...
+- *errp != NULL
++ *errp
+ ...>
+ }
--
2.21.0
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Laszlo Ersek, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Greg Kurz, armbru, Stefano Stabellini,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Eric Blake, Michael Roth, Stefan Berger
Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
does corresponding changes in code (look for details in
include/qapi/error.h)
Usage example:
spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
Cc: Eric Blake <eblake@redhat.com>
Cc: Kevin Wolf <kwolf@redhat.com>
Cc: Max Reitz <mreitz@redhat.com>
Cc: Greg Kurz <groug@kaod.org>
Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Anthony Perard <anthony.perard@citrix.com>
Cc: Paul Durrant <paul@xen.org>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
Cc: Laszlo Ersek <lersek@redhat.com>
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Stefan Berger <stefanb@linux.ibm.com>
Cc: Markus Armbruster <armbru@redhat.com>
Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
Cc: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org
Cc: xen-devel@lists.xenproject.org
include/qapi/error.h | 3 +
scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
2 files changed, 234 insertions(+)
create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
diff --git a/include/qapi/error.h b/include/qapi/error.h
index bb9bcf02fb..fbfc6f1c0b 100644
--- a/include/qapi/error.h
+++ b/include/qapi/error.h
@@ -211,6 +211,9 @@
* }
* ...
* }
+ *
+ * For mass conversion use script
+ * scripts/coccinelle/auto-propagated-errp.cocci
*/
#ifndef ERROR_H
diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
new file mode 100644
index 0000000000..bff274bd6d
--- /dev/null
+++ b/scripts/coccinelle/auto-propagated-errp.cocci
@@ -0,0 +1,231 @@
+// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
+//
+// Copyright (c) 2020 Virtuozzo International GmbH.
+//
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+//
+// This program is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+//
+// You should have received a copy of the GNU General Public License
+// along with this program. If not, see <http://www.gnu.org/licenses/>.
+//
+// Usage example:
+// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
+// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
+// --max-width 80 blockdev-nbd.c qemu-nbd.c \
+// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
+
+// Switch unusual (Error **) parameter names to errp
+// (this is necessary to use ERRP_AUTO_PROPAGATE).
+//
+// Disable optional_qualifier to skip functions with "Error *const *errp"
+// parameter.
+//
+// Skip functions with "assert(_errp && *_errp)" statement, as they have
+// non generic semantics and may have unusual Error ** argument name for purpose
+// (like nbd_iter_channel_error()).
+//
+// Skip util/error.c to not touch, for example, error_propagate and
+// error_propagate_prepend().
+@ depends on !(file in "util/error.c") disable optional_qualifier@
+identifier fn;
+identifier _errp != errp;
+@@
+
+ fn(...,
+- Error **_errp
++ Error **errp
+ ,...)
+ {
+(
+ ... when != assert(_errp && *_errp)
+&
+ <...
+- _errp
++ errp
+ ...>
+)
+ }
+
+// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
+//
+// Note, that without "when any" final "..." may not want to mach something
+// matched by previous pattern, i.e. the rule will not match double
+// error_prepend in control flow like in vfio_set_irq_signaling().
+//
+// Note, "exists" says that we want apply rule even if it matches not on
+// all possible control flows (otherwise, it will not match standard pattern
+// when error_propagate() call is in if branch).
+@ disable optional_qualifier exists@
+identifier fn, local_err, errp;
+@@
+
+ fn(..., Error **errp, ...)
+ {
++ ERRP_AUTO_PROPAGATE();
+ ... when != ERRP_AUTO_PROPAGATE();
+(
+ error_append_hint(errp, ...);
+|
+ error_prepend(errp, ...);
+|
+ error_vprepend(errp, ...);
+|
+ Error *local_err = NULL;
+ ...
+(
+ error_propagate_prepend(errp, local_err, ...);
+|
+ error_propagate(errp, local_err);
+)
+)
+ ... when any
+ }
+
+
+// Match scenarios with propagation of local error to errp.
+@rule1 disable optional_qualifier exists@
+identifier fn, local_err;
+symbol errp;
+@@
+
+ fn(..., Error **errp, ...)
+ {
+ ...
+ Error *local_err = NULL;
+ ...
+(
+ error_propagate_prepend(errp, local_err, ...);
+|
+ error_propagate(errp, local_err);
+)
+ ...
+ }
+
+// Convert special case with goto in separate.
+// We can probably merge this into the following hunk with help of ( | )
+// operator, but it significantly reduce performance on block.c parsing (or it
+// hangs, I don't know)
+//
+// Note interesting thing: if we don't do it here, and try to fixup "out: }"
+// things later after all transformations (the rule will be the same, just
+// without error_propagate() call), coccinelle fails to match this "out: }".
+@@
+identifier rule1.fn, rule1.local_err, out;
+symbol errp;
+@@
+
+ fn(...)
+ {
+ <...
+- goto out;
++ return;
+ ...>
+- out:
+- error_propagate(errp, local_err);
+ }
+
+// Convert most of local_err related staff.
+//
+// Note, that we update everything related to matched by rule1 function name
+// and local_err name. We may match something not related to the pattern
+// matched by rule1. For example, local_err may be defined with the same name
+// in different blocks inside one function, and in one block follow the
+// propagation pattern and in other block doesn't. Or we may have several
+// functions with the same name (for different configurations).
+//
+// Note also that errp-cleaning functions
+// error_free_errp
+// error_report_errp
+// error_reportf_errp
+// warn_report_errp
+// warn_reportf_errp
+// are not yet implemented. They must call corresponding Error* - freeing
+// function and then set *errp to NULL, to avoid further propagation to
+// original errp (consider ERRP_AUTO_PROPAGATE in use).
+// For example, error_free_errp may look like this:
+//
+// void error_free_errp(Error **errp)
+// {
+// error_free(*errp);
+// *errp = NULL;
+// }
+@ exists@
+identifier rule1.fn, rule1.local_err;
+expression list args;
+symbol errp;
+@@
+
+ fn(...)
+ {
+ <...
+(
+- Error *local_err = NULL;
+|
+
+// Convert error clearing functions
+(
+- error_free(local_err);
++ error_free_errp(errp);
+|
+- error_report_err(local_err);
++ error_report_errp(errp);
+|
+- error_reportf_err(local_err, args);
++ error_reportf_errp(errp, args);
+|
+- warn_report_err(local_err);
++ warn_report_errp(errp);
+|
+- warn_reportf_err(local_err, args);
++ warn_reportf_errp(errp, args);
+)
+?- local_err = NULL;
+
+|
+- error_propagate_prepend(errp, local_err, args);
++ error_prepend(errp, args);
+|
+- error_propagate(errp, local_err);
+|
+- &local_err
++ errp
+)
+ ...>
+ }
+
+// Convert remaining local_err usage. It should be different kinds of error
+// checking in if operators. We can't merge this into previous hunk, as this
+// conflicts with other substitutions in it (at least with "- local_err = NULL").
+@@
+identifier rule1.fn, rule1.local_err;
+symbol errp;
+@@
+
+ fn(...)
+ {
+ <...
+- local_err
++ *errp
+ ...>
+ }
+
+// Always use the same patter for checking error
+@@
+identifier rule1.fn;
+symbol errp;
+@@
+
+ fn(...)
+ {
+ <...
+- *errp != NULL
++ *errp
+ ...>
+ }
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-06 12:43 ` Eric Blake
-1 siblings, 0 replies; 77+ messages in thread
From: Eric Blake @ 2020-03-06 12:43 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, Greg Kurz,
armbru, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
On 3/5/20 11:15 PM, Vladimir Sementsov-Ogievskiy wrote:
> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
> does corresponding changes in code (look for details in
> include/qapi/error.h)
>
> Usage example:
> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>
I'll let Markus do the final review of this, but my personal take is
that if the subsequent patches created by using this script are
reasonable, then this script was good enough.
> +// Always use the same patter for checking error
pattern
> +@@
> +identifier rule1.fn;
> +symbol errp;
> +@@
> +
> + fn(...)
> + {
> + <...
> +- *errp != NULL
> ++ *errp
> + ...>
> + }
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-06 12:43 ` Eric Blake
0 siblings, 0 replies; 77+ messages in thread
From: Eric Blake @ 2020-03-06 12:43 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, Greg Kurz,
armbru, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
On 3/5/20 11:15 PM, Vladimir Sementsov-Ogievskiy wrote:
> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
> does corresponding changes in code (look for details in
> include/qapi/error.h)
>
> Usage example:
> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>
I'll let Markus do the final review of this, but my personal take is
that if the subsequent patches created by using this script are
reasonable, then this script was good enough.
> +// Always use the same patter for checking error
pattern
> +@@
> +identifier rule1.fn;
> +symbol errp;
> +@@
> +
> + fn(...)
> + {
> + <...
> +- *errp != NULL
> ++ *errp
> + ...>
> + }
>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-08 19:09 ` Christian Schoenebeck
-1 siblings, 0 replies; 77+ messages in thread
From: Christian Schoenebeck @ 2020-03-08 19:09 UTC (permalink / raw)
To: qemu-devel
Cc: Vladimir Sementsov-Ogievskiy, Kevin Wolf, Laszlo Ersek,
qemu-block, Paul Durrant, Philippe Mathieu-Daudé,
Greg Kurz, armbru, Stefano Stabellini, Gerd Hoffmann,
Stefan Hajnoczi, Anthony Perard, xen-devel, Max Reitz,
Michael Roth, Stefan Berger
On Freitag, 6. März 2020 06:15:28 CET Vladimir Sementsov-Ogievskiy wrote:
> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci
> b/scripts/coccinelle/auto-propagated-errp.cocci new file mode 100644
> index 0000000000..bff274bd6d
> --- /dev/null
> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
> @@ -0,0 +1,231 @@
> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
> +//
> +// Copyright (c) 2020 Virtuozzo International GmbH.
Just in case:
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
Best regards,
Christian Schoenebeck
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-08 19:09 ` Christian Schoenebeck
0 siblings, 0 replies; 77+ messages in thread
From: Christian Schoenebeck @ 2020-03-08 19:09 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, Vladimir Sementsov-Ogievskiy, Michael Roth,
qemu-block, Paul Durrant, Laszlo Ersek, Greg Kurz, armbru,
Stefano Stabellini, Gerd Hoffmann, Stefan Hajnoczi,
Anthony Perard, xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
On Freitag, 6. März 2020 06:15:28 CET Vladimir Sementsov-Ogievskiy wrote:
> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci
> b/scripts/coccinelle/auto-propagated-errp.cocci new file mode 100644
> index 0000000000..bff274bd6d
> --- /dev/null
> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
> @@ -0,0 +1,231 @@
> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
> +//
> +// Copyright (c) 2020 Virtuozzo International GmbH.
Just in case:
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
Best regards,
Christian Schoenebeck
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-08 19:09 ` [Xen-devel] " Christian Schoenebeck
@ 2020-03-10 6:47 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-10 6:47 UTC (permalink / raw)
To: Christian Schoenebeck, qemu-devel
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Greg Kurz, armbru, Gerd Hoffmann,
Stefan Hajnoczi, Anthony Perard, xen-devel, Max Reitz,
Philippe Mathieu-Daudé,
Stefan Berger
08.03.2020 22:09, Christian Schoenebeck wrote:
> On Freitag, 6. März 2020 06:15:28 CET Vladimir Sementsov-Ogievskiy wrote:
>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci
>> b/scripts/coccinelle/auto-propagated-errp.cocci new file mode 100644
>> index 0000000000..bff274bd6d
>> --- /dev/null
>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>> @@ -0,0 +1,231 @@
>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>> +//
>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>
> Just in case:
>
> WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
Hmm, seems this, and some other coccinelle scripts should be added to "Error reporting"
section.
>
> Best regards,
> Christian Schoenebeck
>
>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-10 6:47 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-10 6:47 UTC (permalink / raw)
To: Christian Schoenebeck, qemu-devel
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Greg Kurz, armbru, Gerd Hoffmann,
Stefan Hajnoczi, Anthony Perard, xen-devel, Max Reitz,
Philippe Mathieu-Daudé,
Stefan Berger
08.03.2020 22:09, Christian Schoenebeck wrote:
> On Freitag, 6. März 2020 06:15:28 CET Vladimir Sementsov-Ogievskiy wrote:
>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci
>> b/scripts/coccinelle/auto-propagated-errp.cocci new file mode 100644
>> index 0000000000..bff274bd6d
>> --- /dev/null
>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>> @@ -0,0 +1,231 @@
>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>> +//
>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>
> Just in case:
>
> WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
Hmm, seems this, and some other coccinelle scripts should be added to "Error reporting"
section.
>
> Best regards,
> Christian Schoenebeck
>
>
>
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-09 9:56 ` Markus Armbruster
-1 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-09 9:56 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
Suggest
scripts: Coccinelle script to use auto-propagated errp
or
scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
> does corresponding changes in code (look for details in
> include/qapi/error.h)
>
> Usage example:
> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
Suggest FILES... instead of a specific set of files.
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>
> Cc: Eric Blake <eblake@redhat.com>
> Cc: Kevin Wolf <kwolf@redhat.com>
> Cc: Max Reitz <mreitz@redhat.com>
> Cc: Greg Kurz <groug@kaod.org>
> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
> Cc: Stefano Stabellini <sstabellini@kernel.org>
> Cc: Anthony Perard <anthony.perard@citrix.com>
> Cc: Paul Durrant <paul@xen.org>
> Cc: Stefan Hajnoczi <stefanha@redhat.com>
> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Stefan Berger <stefanb@linux.ibm.com>
> Cc: Markus Armbruster <armbru@redhat.com>
> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
> Cc: qemu-block@nongnu.org
> Cc: qemu-devel@nongnu.org
> Cc: xen-devel@lists.xenproject.org
>
> include/qapi/error.h | 3 +
> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
> 2 files changed, 234 insertions(+)
> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>
> diff --git a/include/qapi/error.h b/include/qapi/error.h
> index bb9bcf02fb..fbfc6f1c0b 100644
> --- a/include/qapi/error.h
> +++ b/include/qapi/error.h
> @@ -211,6 +211,9 @@
> * }
> * ...
> * }
> + *
> + * For mass conversion use script
mass-conversion (we're not converting mass, we're converting en masse)
> + * scripts/coccinelle/auto-propagated-errp.cocci
> */
>
> #ifndef ERROR_H
> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
> new file mode 100644
> index 0000000000..bff274bd6d
> --- /dev/null
> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
Preface to my review of this script: may aim isn't to make it
bullet-proof. I want to (1) make it good enough (explained in a
jiffie), and (2) automatically identify the spots where it still isn't
obviously safe for manual review.
The latter may involve additional scripting. That's okay.
The script is good enough when the number of possibly unsafe spots is
low enough for careful manual review.
When I ask for improvements that, in your opinion, go beyond "good
enough", please push back. I'm sure we can work it out together.
> @@ -0,0 +1,231 @@
> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
> +//
> +// Copyright (c) 2020 Virtuozzo International GmbH.
> +//
> +// This program is free software; you can redistribute it and/or modify
> +// it under the terms of the GNU General Public License as published by
> +// the Free Software Foundation; either version 2 of the License, or
> +// (at your option) any later version.
> +//
> +// This program is distributed in the hope that it will be useful,
> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License
> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
> +//
> +// Usage example:
> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
You have --max-width 80 here, but not in the commit message. Default
seems to be 78. Any particular reason to change it to 80?
> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
> +
> +// Switch unusual (Error **) parameter names to errp
Let's drop the parenthesis around Error **
> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
make the fact we're messing with @errp more obvious. Too late; I
shouldn't rock the boat that much now.
> +//
> +// Disable optional_qualifier to skip functions with "Error *const *errp"
> +// parameter.
> +//
> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
> +// non generic semantics and may have unusual Error ** argument name for purpose
non-generic
for a purpose
Wrap comment lines around column 70, please. It's easier to read.
Maybe
// Skip functions with "assert(_errp && *_errp)" statement, because that
// signals unusual semantics, and the parameter name may well serve a
// purpose.
> +// (like nbd_iter_channel_error()).
> +//
> +// Skip util/error.c to not touch, for example, error_propagate and
> +// error_propagate_prepend().
error_propagate()
I much appreciate your meticulous explanation of what you skip and why.
> +@ depends on !(file in "util/error.c") disable optional_qualifier@
> +identifier fn;
> +identifier _errp != errp;
> +@@
> +
> + fn(...,
> +- Error **_errp
> ++ Error **errp
> + ,...)
> + {
> +(
> + ... when != assert(_errp && *_errp)
> +&
> + <...
> +- _errp
> ++ errp
> + ...>
> +)
> + }
This rule is required to make the actual transformations (below) work
even for parameters with names other than @errp. I believe it's not
used in this series. In fact, I can't see a use for it in the entire
tree right now. Okay anyway.
> +
> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
> +//
> +// Note, that without "when any" final "..." may not want to mach something
s/final "..." may not mach/the final "..." does not match/
> +// matched by previous pattern, i.e. the rule will not match double
> +// error_prepend in control flow like in vfio_set_irq_signaling().
Can't say I fully understand Coccinelle there. I figure you came to
this knowledge the hard way.
> +//
> +// Note, "exists" says that we want apply rule even if it matches not on
> +// all possible control flows (otherwise, it will not match standard pattern
> +// when error_propagate() call is in if branch).
Learned something new. Example: kvm_set_kvm_shadow_mem().
Spelling it "exists disable optional_qualifier" would avoid giving
readers the idea we're disabling "exists", but Coccinelle doesn't let
us. Oh well.
> +@ disable optional_qualifier exists@
> +identifier fn, local_err, errp;
I believe this causes
warning: line 98: errp, previously declared as a metavariable, is used as an identifier
warning: line 104: errp, previously declared as a metavariable, is used as an identifier
warning: line 106: errp, previously declared as a metavariable, is used as an identifier
warning: line 131: errp, previously declared as a metavariable, is used as an identifier
warning: line 192: errp, previously declared as a metavariable, is used as an identifier
warning: line 195: errp, previously declared as a metavariable, is used as an identifier
warning: line 228: errp, previously declared as a metavariable, is used as an identifier
Making @errp symbol instead of identifier should fix this.
> +@@
> +
> + fn(..., Error **errp, ...)
> + {
> ++ ERRP_AUTO_PROPAGATE();
> + ... when != ERRP_AUTO_PROPAGATE();
> +(
> + error_append_hint(errp, ...);
> +|
> + error_prepend(errp, ...);
> +|
> + error_vprepend(errp, ...);
> +|
> + Error *local_err = NULL;
> + ...
> +(
> + error_propagate_prepend(errp, local_err, ...);
> +|
> + error_propagate(errp, local_err);
> +)
> +)
> + ... when any
> + }
> +
> +
> +// Match scenarios with propagation of local error to errp.
> +@rule1 disable optional_qualifier exists@
> +identifier fn, local_err;
> +symbol errp;
> +@@
> +
> + fn(..., Error **errp, ...)
> + {
> + ...
> + Error *local_err = NULL;
> + ...
> +(
> + error_propagate_prepend(errp, local_err, ...);
> +|
> + error_propagate(errp, local_err);
> +)
Indentation off by one.
> + ...
> + }
> +
> +// Convert special case with goto in separate.
s/in separate/separately/
> +// We can probably merge this into the following hunk with help of ( | )
> +// operator, but it significantly reduce performance on block.c parsing (or it
s/reduce/reduces/
> +// hangs, I don't know)
Sounds like you tried to merge this into the following hunk, but then
spatch took so long on block.c that you killed it. Correct?
> +//
> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
> +// things later after all transformations (the rule will be the same, just
> +// without error_propagate() call), coccinelle fails to match this "out: }".
Weird, but not worth further investigation.
> +@@
> +identifier rule1.fn, rule1.local_err, out;
> +symbol errp;
> +@@
> +
> + fn(...)
> + {
> + <...
> +- goto out;
> ++ return;
> + ...>
> +- out:
> +- error_propagate(errp, local_err);
You neglect to match error_propagate_prepend(). Okay, because (1) that
pattern doesn't occur in the tree right now, and (2) if it gets added,
gcc will complain.
> + }
> +
> +// Convert most of local_err related staff.
s/staff/stuff/
> +//
> +// Note, that we update everything related to matched by rule1 function name
> +// and local_err name. We may match something not related to the pattern
> +// matched by rule1. For example, local_err may be defined with the same name
> +// in different blocks inside one function, and in one block follow the
> +// propagation pattern and in other block doesn't. Or we may have several
> +// functions with the same name (for different configurations).
Context: rule1 matches functions that have all three of
* an Error **errp parameter
* an Error *local_err = NULL variable declaration
* an error_propagate(errp, local_err) or error_propagate_prepend(errp,
local_err, ...) expression, where @errp is the parameter and
@local_err is the variable.
If I understand you correctly, you're pointing out two potential issues:
1. This rule can match functions rule1 does not match if there is
another function with the same name that rule1 does match.
2. This rule matches in the entire function matched by rule1, even when
parts of that function use a different @errp or @local_err.
I figure these apply to all rules with identifier rule1.fn, not just
this one. Correct?
Regarding 1. There must be a better way to chain rules together, but I
don't know it. Can we make Coccinelle at least warn us when it converts
multiple functions with the same name? What about this:
@initialize:python@
@@
fnprev = {}
def pr(fn, p):
print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
@r@
identifier rule1.fn;
position p;
@@
fn(...)@p
{
...
}
@script:python@
fn << rule1.fn;
p << r.p;
@@
if fn not in fnprev:
fnprev[fn] = p
else:
if fnprev[fn]:
pr(fn, fnprev[fn])
fnprev[fn] = None
pr(fn, p)
For each function @fn matched by rule1, fncnt[fn] is an upper limit of
the number of functions with the same name we touch. If it's more than
one, we print.
Reports about a dozen function names for the whole tree in my testing.
Inspecting the changes to them manually is feasible. None of them are
in files touched by this series.
The line printed for the first match is pretty useless for me: it points
to a Coccinelle temporary file *shrug*.
Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
sure hope we don't do that. Multiple @local_err variables... hmm.
Perhaps we could again concoct some script rules to lead us to spots to
check manually. See below for my attempt.
What's the worst that could happen if we blindly converted such code?
The answer to that question tells us how hard to work on finding and
checking these guys.
> +//
> +// Note also that errp-cleaning functions
> +// error_free_errp
> +// error_report_errp
> +// error_reportf_errp
> +// warn_report_errp
> +// warn_reportf_errp
> +// are not yet implemented. They must call corresponding Error* - freeing
> +// function and then set *errp to NULL, to avoid further propagation to
> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
> +// For example, error_free_errp may look like this:
> +//
> +// void error_free_errp(Error **errp)
> +// {
> +// error_free(*errp);
> +// *errp = NULL;
> +// }
> +@ exists@
> +identifier rule1.fn, rule1.local_err;
> +expression list args;
> +symbol errp;
> +@@
> +
> + fn(...)
> + {
> + <...
> +(
Each of the following patterns applies anywhere in the function.
First pattern: delete @local_err
> +- Error *local_err = NULL;
Common case: occurs just once, not nested. Anything else is suspicious.
Both can be detected in the resulting patches with a bit of AWK
wizardry:
$ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
nested
static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
nested
static void xen_block_device_destroy(XenBackendInstance *backend,
nested
static void xen_block_device_destroy(XenBackendInstance *backend,
more than one
static void xen_block_device_destroy(XenBackendInstance *backend,
Oh.
xen_block_drive_destroy() nests its Error *local_err in a conditional.
xen_block_device_destroy() has multiple Error *local_err.
In both cases, manual review is required to ensure the conversion is
okay. I believe it is.
Note that the AWK script relies on diff showing the function name in @@
lines, which doesn't always work due to our coding style.
For the whole tree, I get some 30 spots. Feasible.
> +|
Second pattern: clear @errp after freeing it
> +
> +// Convert error clearing functions
Suggest: Ensure @local_err is cleared on free
> +(
> +- error_free(local_err);
> ++ error_free_errp(errp);
> +|
> +- error_report_err(local_err);
> ++ error_report_errp(errp);
> +|
> +- error_reportf_err(local_err, args);
> ++ error_reportf_errp(errp, args);
> +|
> +- warn_report_err(local_err);
> ++ warn_report_errp(errp);
> +|
> +- warn_reportf_err(local_err, args);
> ++ warn_reportf_errp(errp, args);
> +)
As you mention above, these guys don't exist, yet. Builds anyway,
because this part of the rule is not used in this patch series. You
don't want to omit it, because then the script becomes unsafe to use.
We could also open-code:
// Convert error clearing functions
(
- error_free(local_err);
+ error_free(*errp);
+ *errp = NULL;
|
... and so forth ...
)
Matter of taste. Whatever is easier to explain in the comments. Since
you already wrote one...
We talked about extending this series slightly so these guys are used.
I may still look into that.
> +?- local_err = NULL;
> +
The new helpers clear @local_err. Assignment now redundant, delete.
Okay.
> +|
Third and fourth pattern: delete error_propagate()
> +- error_propagate_prepend(errp, local_err, args);
> ++ error_prepend(errp, args);
> +|
> +- error_propagate(errp, local_err);
> +|
Fifth pattern: use @errp directly
> +- &local_err
> ++ errp
> +)
> + ...>
> + }
> +
> +// Convert remaining local_err usage. It should be different kinds of error
> +// checking in if operators. We can't merge this into previous hunk, as this
In if conditionals, I suppose. It's the case for this patch. If I
apply the script to the whole tree, the rule gets also applied in other
contexts. The sentence might mislead as much as it helps. Keep it or
delete it?
> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
> +@@
> +identifier rule1.fn, rule1.local_err;
> +symbol errp;
> +@@
> +
> + fn(...)
> + {
> + <...
> +- local_err
> ++ *errp
> + ...>
> + }
> +
> +// Always use the same patter for checking error
s/patter/pattern/
> +@@
> +identifier rule1.fn;
> +symbol errp;
> +@@
> +
> + fn(...)
> + {
> + <...
> +- *errp != NULL
> ++ *errp
> + ...>
> + }
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-09 9:56 ` Markus Armbruster
0 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-09 9:56 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
Suggest
scripts: Coccinelle script to use auto-propagated errp
or
scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
> does corresponding changes in code (look for details in
> include/qapi/error.h)
>
> Usage example:
> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
Suggest FILES... instead of a specific set of files.
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>
> Cc: Eric Blake <eblake@redhat.com>
> Cc: Kevin Wolf <kwolf@redhat.com>
> Cc: Max Reitz <mreitz@redhat.com>
> Cc: Greg Kurz <groug@kaod.org>
> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
> Cc: Stefano Stabellini <sstabellini@kernel.org>
> Cc: Anthony Perard <anthony.perard@citrix.com>
> Cc: Paul Durrant <paul@xen.org>
> Cc: Stefan Hajnoczi <stefanha@redhat.com>
> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
> Cc: Laszlo Ersek <lersek@redhat.com>
> Cc: Gerd Hoffmann <kraxel@redhat.com>
> Cc: Stefan Berger <stefanb@linux.ibm.com>
> Cc: Markus Armbruster <armbru@redhat.com>
> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
> Cc: qemu-block@nongnu.org
> Cc: qemu-devel@nongnu.org
> Cc: xen-devel@lists.xenproject.org
>
> include/qapi/error.h | 3 +
> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
> 2 files changed, 234 insertions(+)
> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>
> diff --git a/include/qapi/error.h b/include/qapi/error.h
> index bb9bcf02fb..fbfc6f1c0b 100644
> --- a/include/qapi/error.h
> +++ b/include/qapi/error.h
> @@ -211,6 +211,9 @@
> * }
> * ...
> * }
> + *
> + * For mass conversion use script
mass-conversion (we're not converting mass, we're converting en masse)
> + * scripts/coccinelle/auto-propagated-errp.cocci
> */
>
> #ifndef ERROR_H
> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
> new file mode 100644
> index 0000000000..bff274bd6d
> --- /dev/null
> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
Preface to my review of this script: may aim isn't to make it
bullet-proof. I want to (1) make it good enough (explained in a
jiffie), and (2) automatically identify the spots where it still isn't
obviously safe for manual review.
The latter may involve additional scripting. That's okay.
The script is good enough when the number of possibly unsafe spots is
low enough for careful manual review.
When I ask for improvements that, in your opinion, go beyond "good
enough", please push back. I'm sure we can work it out together.
> @@ -0,0 +1,231 @@
> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
> +//
> +// Copyright (c) 2020 Virtuozzo International GmbH.
> +//
> +// This program is free software; you can redistribute it and/or modify
> +// it under the terms of the GNU General Public License as published by
> +// the Free Software Foundation; either version 2 of the License, or
> +// (at your option) any later version.
> +//
> +// This program is distributed in the hope that it will be useful,
> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +// GNU General Public License for more details.
> +//
> +// You should have received a copy of the GNU General Public License
> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
> +//
> +// Usage example:
> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
You have --max-width 80 here, but not in the commit message. Default
seems to be 78. Any particular reason to change it to 80?
> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
> +
> +// Switch unusual (Error **) parameter names to errp
Let's drop the parenthesis around Error **
> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
make the fact we're messing with @errp more obvious. Too late; I
shouldn't rock the boat that much now.
> +//
> +// Disable optional_qualifier to skip functions with "Error *const *errp"
> +// parameter.
> +//
> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
> +// non generic semantics and may have unusual Error ** argument name for purpose
non-generic
for a purpose
Wrap comment lines around column 70, please. It's easier to read.
Maybe
// Skip functions with "assert(_errp && *_errp)" statement, because that
// signals unusual semantics, and the parameter name may well serve a
// purpose.
> +// (like nbd_iter_channel_error()).
> +//
> +// Skip util/error.c to not touch, for example, error_propagate and
> +// error_propagate_prepend().
error_propagate()
I much appreciate your meticulous explanation of what you skip and why.
> +@ depends on !(file in "util/error.c") disable optional_qualifier@
> +identifier fn;
> +identifier _errp != errp;
> +@@
> +
> + fn(...,
> +- Error **_errp
> ++ Error **errp
> + ,...)
> + {
> +(
> + ... when != assert(_errp && *_errp)
> +&
> + <...
> +- _errp
> ++ errp
> + ...>
> +)
> + }
This rule is required to make the actual transformations (below) work
even for parameters with names other than @errp. I believe it's not
used in this series. In fact, I can't see a use for it in the entire
tree right now. Okay anyway.
> +
> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
> +//
> +// Note, that without "when any" final "..." may not want to mach something
s/final "..." may not mach/the final "..." does not match/
> +// matched by previous pattern, i.e. the rule will not match double
> +// error_prepend in control flow like in vfio_set_irq_signaling().
Can't say I fully understand Coccinelle there. I figure you came to
this knowledge the hard way.
> +//
> +// Note, "exists" says that we want apply rule even if it matches not on
> +// all possible control flows (otherwise, it will not match standard pattern
> +// when error_propagate() call is in if branch).
Learned something new. Example: kvm_set_kvm_shadow_mem().
Spelling it "exists disable optional_qualifier" would avoid giving
readers the idea we're disabling "exists", but Coccinelle doesn't let
us. Oh well.
> +@ disable optional_qualifier exists@
> +identifier fn, local_err, errp;
I believe this causes
warning: line 98: errp, previously declared as a metavariable, is used as an identifier
warning: line 104: errp, previously declared as a metavariable, is used as an identifier
warning: line 106: errp, previously declared as a metavariable, is used as an identifier
warning: line 131: errp, previously declared as a metavariable, is used as an identifier
warning: line 192: errp, previously declared as a metavariable, is used as an identifier
warning: line 195: errp, previously declared as a metavariable, is used as an identifier
warning: line 228: errp, previously declared as a metavariable, is used as an identifier
Making @errp symbol instead of identifier should fix this.
> +@@
> +
> + fn(..., Error **errp, ...)
> + {
> ++ ERRP_AUTO_PROPAGATE();
> + ... when != ERRP_AUTO_PROPAGATE();
> +(
> + error_append_hint(errp, ...);
> +|
> + error_prepend(errp, ...);
> +|
> + error_vprepend(errp, ...);
> +|
> + Error *local_err = NULL;
> + ...
> +(
> + error_propagate_prepend(errp, local_err, ...);
> +|
> + error_propagate(errp, local_err);
> +)
> +)
> + ... when any
> + }
> +
> +
> +// Match scenarios with propagation of local error to errp.
> +@rule1 disable optional_qualifier exists@
> +identifier fn, local_err;
> +symbol errp;
> +@@
> +
> + fn(..., Error **errp, ...)
> + {
> + ...
> + Error *local_err = NULL;
> + ...
> +(
> + error_propagate_prepend(errp, local_err, ...);
> +|
> + error_propagate(errp, local_err);
> +)
Indentation off by one.
> + ...
> + }
> +
> +// Convert special case with goto in separate.
s/in separate/separately/
> +// We can probably merge this into the following hunk with help of ( | )
> +// operator, but it significantly reduce performance on block.c parsing (or it
s/reduce/reduces/
> +// hangs, I don't know)
Sounds like you tried to merge this into the following hunk, but then
spatch took so long on block.c that you killed it. Correct?
> +//
> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
> +// things later after all transformations (the rule will be the same, just
> +// without error_propagate() call), coccinelle fails to match this "out: }".
Weird, but not worth further investigation.
> +@@
> +identifier rule1.fn, rule1.local_err, out;
> +symbol errp;
> +@@
> +
> + fn(...)
> + {
> + <...
> +- goto out;
> ++ return;
> + ...>
> +- out:
> +- error_propagate(errp, local_err);
You neglect to match error_propagate_prepend(). Okay, because (1) that
pattern doesn't occur in the tree right now, and (2) if it gets added,
gcc will complain.
> + }
> +
> +// Convert most of local_err related staff.
s/staff/stuff/
> +//
> +// Note, that we update everything related to matched by rule1 function name
> +// and local_err name. We may match something not related to the pattern
> +// matched by rule1. For example, local_err may be defined with the same name
> +// in different blocks inside one function, and in one block follow the
> +// propagation pattern and in other block doesn't. Or we may have several
> +// functions with the same name (for different configurations).
Context: rule1 matches functions that have all three of
* an Error **errp parameter
* an Error *local_err = NULL variable declaration
* an error_propagate(errp, local_err) or error_propagate_prepend(errp,
local_err, ...) expression, where @errp is the parameter and
@local_err is the variable.
If I understand you correctly, you're pointing out two potential issues:
1. This rule can match functions rule1 does not match if there is
another function with the same name that rule1 does match.
2. This rule matches in the entire function matched by rule1, even when
parts of that function use a different @errp or @local_err.
I figure these apply to all rules with identifier rule1.fn, not just
this one. Correct?
Regarding 1. There must be a better way to chain rules together, but I
don't know it. Can we make Coccinelle at least warn us when it converts
multiple functions with the same name? What about this:
@initialize:python@
@@
fnprev = {}
def pr(fn, p):
print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
@r@
identifier rule1.fn;
position p;
@@
fn(...)@p
{
...
}
@script:python@
fn << rule1.fn;
p << r.p;
@@
if fn not in fnprev:
fnprev[fn] = p
else:
if fnprev[fn]:
pr(fn, fnprev[fn])
fnprev[fn] = None
pr(fn, p)
For each function @fn matched by rule1, fncnt[fn] is an upper limit of
the number of functions with the same name we touch. If it's more than
one, we print.
Reports about a dozen function names for the whole tree in my testing.
Inspecting the changes to them manually is feasible. None of them are
in files touched by this series.
The line printed for the first match is pretty useless for me: it points
to a Coccinelle temporary file *shrug*.
Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
sure hope we don't do that. Multiple @local_err variables... hmm.
Perhaps we could again concoct some script rules to lead us to spots to
check manually. See below for my attempt.
What's the worst that could happen if we blindly converted such code?
The answer to that question tells us how hard to work on finding and
checking these guys.
> +//
> +// Note also that errp-cleaning functions
> +// error_free_errp
> +// error_report_errp
> +// error_reportf_errp
> +// warn_report_errp
> +// warn_reportf_errp
> +// are not yet implemented. They must call corresponding Error* - freeing
> +// function and then set *errp to NULL, to avoid further propagation to
> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
> +// For example, error_free_errp may look like this:
> +//
> +// void error_free_errp(Error **errp)
> +// {
> +// error_free(*errp);
> +// *errp = NULL;
> +// }
> +@ exists@
> +identifier rule1.fn, rule1.local_err;
> +expression list args;
> +symbol errp;
> +@@
> +
> + fn(...)
> + {
> + <...
> +(
Each of the following patterns applies anywhere in the function.
First pattern: delete @local_err
> +- Error *local_err = NULL;
Common case: occurs just once, not nested. Anything else is suspicious.
Both can be detected in the resulting patches with a bit of AWK
wizardry:
$ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
nested
static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
nested
static void xen_block_device_destroy(XenBackendInstance *backend,
nested
static void xen_block_device_destroy(XenBackendInstance *backend,
more than one
static void xen_block_device_destroy(XenBackendInstance *backend,
Oh.
xen_block_drive_destroy() nests its Error *local_err in a conditional.
xen_block_device_destroy() has multiple Error *local_err.
In both cases, manual review is required to ensure the conversion is
okay. I believe it is.
Note that the AWK script relies on diff showing the function name in @@
lines, which doesn't always work due to our coding style.
For the whole tree, I get some 30 spots. Feasible.
> +|
Second pattern: clear @errp after freeing it
> +
> +// Convert error clearing functions
Suggest: Ensure @local_err is cleared on free
> +(
> +- error_free(local_err);
> ++ error_free_errp(errp);
> +|
> +- error_report_err(local_err);
> ++ error_report_errp(errp);
> +|
> +- error_reportf_err(local_err, args);
> ++ error_reportf_errp(errp, args);
> +|
> +- warn_report_err(local_err);
> ++ warn_report_errp(errp);
> +|
> +- warn_reportf_err(local_err, args);
> ++ warn_reportf_errp(errp, args);
> +)
As you mention above, these guys don't exist, yet. Builds anyway,
because this part of the rule is not used in this patch series. You
don't want to omit it, because then the script becomes unsafe to use.
We could also open-code:
// Convert error clearing functions
(
- error_free(local_err);
+ error_free(*errp);
+ *errp = NULL;
|
... and so forth ...
)
Matter of taste. Whatever is easier to explain in the comments. Since
you already wrote one...
We talked about extending this series slightly so these guys are used.
I may still look into that.
> +?- local_err = NULL;
> +
The new helpers clear @local_err. Assignment now redundant, delete.
Okay.
> +|
Third and fourth pattern: delete error_propagate()
> +- error_propagate_prepend(errp, local_err, args);
> ++ error_prepend(errp, args);
> +|
> +- error_propagate(errp, local_err);
> +|
Fifth pattern: use @errp directly
> +- &local_err
> ++ errp
> +)
> + ...>
> + }
> +
> +// Convert remaining local_err usage. It should be different kinds of error
> +// checking in if operators. We can't merge this into previous hunk, as this
In if conditionals, I suppose. It's the case for this patch. If I
apply the script to the whole tree, the rule gets also applied in other
contexts. The sentence might mislead as much as it helps. Keep it or
delete it?
> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
> +@@
> +identifier rule1.fn, rule1.local_err;
> +symbol errp;
> +@@
> +
> + fn(...)
> + {
> + <...
> +- local_err
> ++ *errp
> + ...>
> + }
> +
> +// Always use the same patter for checking error
s/patter/pattern/
> +@@
> +identifier rule1.fn;
> +symbol errp;
> +@@
> +
> + fn(...)
> + {
> + <...
> +- *errp != NULL
> ++ *errp
> + ...>
> + }
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-09 9:56 ` [Xen-devel] " Markus Armbruster
@ 2020-03-10 7:44 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-10 7:44 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
09.03.2020 12:56, Markus Armbruster wrote:
> Suggest
>
> scripts: Coccinelle script to use auto-propagated errp
>
> or
>
> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>> does corresponding changes in code (look for details in
>> include/qapi/error.h)
>>
>> Usage example:
>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>
> Suggest FILES... instead of a specific set of files.
>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>
>> Cc: Eric Blake <eblake@redhat.com>
>> Cc: Kevin Wolf <kwolf@redhat.com>
>> Cc: Max Reitz <mreitz@redhat.com>
>> Cc: Greg Kurz <groug@kaod.org>
>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>> Cc: Anthony Perard <anthony.perard@citrix.com>
>> Cc: Paul Durrant <paul@xen.org>
>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>> Cc: Markus Armbruster <armbru@redhat.com>
>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>> Cc: qemu-block@nongnu.org
>> Cc: qemu-devel@nongnu.org
>> Cc: xen-devel@lists.xenproject.org
>>
>> include/qapi/error.h | 3 +
>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>> 2 files changed, 234 insertions(+)
>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>
>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>> index bb9bcf02fb..fbfc6f1c0b 100644
>> --- a/include/qapi/error.h
>> +++ b/include/qapi/error.h
>> @@ -211,6 +211,9 @@
>> * }
>> * ...
>> * }
>> + *
>> + * For mass conversion use script
>
> mass-conversion (we're not converting mass, we're converting en masse)
>
>> + * scripts/coccinelle/auto-propagated-errp.cocci
>> */
>>
>> #ifndef ERROR_H
>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>> new file mode 100644
>> index 0000000000..bff274bd6d
>> --- /dev/null
>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>
> Preface to my review of this script: may aim isn't to make it
> bullet-proof. I want to (1) make it good enough (explained in a
> jiffie), and (2) automatically identify the spots where it still isn't
> obviously safe for manual review.
>
> The latter may involve additional scripting. That's okay.
>
> The script is good enough when the number of possibly unsafe spots is
> low enough for careful manual review.
>
> When I ask for improvements that, in your opinion, go beyond "good
> enough", please push back. I'm sure we can work it out together.
>
>> @@ -0,0 +1,231 @@
>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>> +//
>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>> +//
>> +// This program is free software; you can redistribute it and/or modify
>> +// it under the terms of the GNU General Public License as published by
>> +// the Free Software Foundation; either version 2 of the License, or
>> +// (at your option) any later version.
>> +//
>> +// This program is distributed in the hope that it will be useful,
>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +// GNU General Public License for more details.
>> +//
>> +// You should have received a copy of the GNU General Public License
>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>> +//
>> +// Usage example:
>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>
> You have --max-width 80 here, but not in the commit message. Default
> seems to be 78. Any particular reason to change it to 80?
Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
Still, may be I'm wrong, we can check it.
>
>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>> +
>> +// Switch unusual (Error **) parameter names to errp
>
> Let's drop the parenthesis around Error **
>
>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>
> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
> make the fact we're messing with @errp more obvious. Too late; I
> shouldn't rock the boat that much now.
>
>> +//
>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>> +// parameter.
>> +//
>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>> +// non generic semantics and may have unusual Error ** argument name for purpose
>
> non-generic
>
> for a purpose
>
> Wrap comment lines around column 70, please. It's easier to read.
>
> Maybe
>
> // Skip functions with "assert(_errp && *_errp)" statement, because that
> // signals unusual semantics, and the parameter name may well serve a
> // purpose.
Sounds good.
>
>> +// (like nbd_iter_channel_error()).
>> +//
>> +// Skip util/error.c to not touch, for example, error_propagate and
>> +// error_propagate_prepend().
>
> error_propagate()
>
> I much appreciate your meticulous explanation of what you skip and why.
>
>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>> +identifier fn;
>> +identifier _errp != errp;
>> +@@
>> +
>> + fn(...,
>> +- Error **_errp
>> ++ Error **errp
>> + ,...)
>> + {
>> +(
>> + ... when != assert(_errp && *_errp)
>> +&
>> + <...
>> +- _errp
>> ++ errp
>> + ...>
>> +)
>> + }
>
> This rule is required to make the actual transformations (below) work
> even for parameters with names other than @errp. I believe it's not
> used in this series. In fact, I can't see a use for it in the entire
> tree right now. Okay anyway.
>
>> +
>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>> +//
>> +// Note, that without "when any" final "..." may not want to mach something
>
> s/final "..." may not mach/the final "..." does not match/
>
>> +// matched by previous pattern, i.e. the rule will not match double
>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>
> Can't say I fully understand Coccinelle there. I figure you came to
> this knowledge the hard way.
It's follows from smpl grammar document:
"Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
...
"_when any_ removes the aforementioned constraint that “...” matches the shortest path"
>
>> +//
>> +// Note, "exists" says that we want apply rule even if it matches not on
>> +// all possible control flows (otherwise, it will not match standard pattern
>> +// when error_propagate() call is in if branch).
>
> Learned something new. Example: kvm_set_kvm_shadow_mem().
>
> Spelling it "exists disable optional_qualifier" would avoid giving
> readers the idea we're disabling "exists", but Coccinelle doesn't let
> us. Oh well.
>
>> +@ disable optional_qualifier exists@
>> +identifier fn, local_err, errp;
>
> I believe this causes
>
> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>
> Making @errp symbol instead of identifier should fix this.
Hmm, I didn't see these warnings.. But yes, it should be symbol.
>
>> +@@
>> +
>> + fn(..., Error **errp, ...)
>> + {
>> ++ ERRP_AUTO_PROPAGATE();
>> + ... when != ERRP_AUTO_PROPAGATE();
>> +(
>> + error_append_hint(errp, ...);
>> +|
>> + error_prepend(errp, ...);
>> +|
>> + error_vprepend(errp, ...);
>> +|
>> + Error *local_err = NULL;
>> + ...
>> +(
>> + error_propagate_prepend(errp, local_err, ...);
>> +|
>> + error_propagate(errp, local_err);
>> +)
>> +)
>> + ... when any
>> + }
>> +
>> +
>> +// Match scenarios with propagation of local error to errp.
>> +@rule1 disable optional_qualifier exists@
>> +identifier fn, local_err;
>> +symbol errp;
>> +@@
>> +
>> + fn(..., Error **errp, ...)
>> + {
>> + ...
>> + Error *local_err = NULL;
>> + ...
>> +(
>> + error_propagate_prepend(errp, local_err, ...);
>> +|
>> + error_propagate(errp, local_err);
>> +)
>
> Indentation off by one.
>
>> + ...
>> + }
>> +
>> +// Convert special case with goto in separate.
>
> s/in separate/separately/
>
>> +// We can probably merge this into the following hunk with help of ( | )
>> +// operator, but it significantly reduce performance on block.c parsing (or it
>
> s/reduce/reduces/
>
>> +// hangs, I don't know)
>
> Sounds like you tried to merge this into the following hunk, but then
> spatch took so long on block.c that you killed it. Correct?
Yes.
>
>> +//
>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>> +// things later after all transformations (the rule will be the same, just
>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>
> Weird, but not worth further investigation.
It partially match to the idea which I saw somewhere in coccinelle documentation,
that coccinelle converts correct C code to correct C code. "out: }" is an example
of incorrect, impossible code flow, and coccinelle can't work with it... But it's
just a thought.
>
>> +@@
>> +identifier rule1.fn, rule1.local_err, out;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- goto out;
>> ++ return;
>> + ...>
>> +- out:
>> +- error_propagate(errp, local_err);
>
> You neglect to match error_propagate_prepend(). Okay, because (1) that
> pattern doesn't occur in the tree right now, and (2) if it gets added,
> gcc will complain.
No, because it should not removed. error_propagate_prepend should be converted
to prepend, not removed. So, corresponding gotos should not be removed as well.
>
>> + }
>> +
>> +// Convert most of local_err related staff.
>
> s/staff/stuff/
>
>> +//
>> +// Note, that we update everything related to matched by rule1 function name
>> +// and local_err name. We may match something not related to the pattern
>> +// matched by rule1. For example, local_err may be defined with the same name
>> +// in different blocks inside one function, and in one block follow the
>> +// propagation pattern and in other block doesn't. Or we may have several
>> +// functions with the same name (for different configurations).
>
> Context: rule1 matches functions that have all three of
>
> * an Error **errp parameter
>
> * an Error *local_err = NULL variable declaration
>
> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
> local_err, ...) expression, where @errp is the parameter and
> @local_err is the variable.
>
> If I understand you correctly, you're pointing out two potential issues:
>
> 1. This rule can match functions rule1 does not match if there is
> another function with the same name that rule1 does match.
>
> 2. This rule matches in the entire function matched by rule1, even when
> parts of that function use a different @errp or @local_err.
>
> I figure these apply to all rules with identifier rule1.fn, not just
> this one. Correct?
Yes.
>
> Regarding 1. There must be a better way to chain rules together, but I
> don't know it.
> Can we make Coccinelle at least warn us when it converts
> multiple functions with the same name? What about this:
>
> @initialize:python@
> @@
> fnprev = {}
>
> def pr(fn, p):
> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>
> @r@
> identifier rule1.fn;
> position p;
> @@
> fn(...)@p
> {
> ...
> }
> @script:python@
> fn << rule1.fn;
> p << r.p;
> @@
> if fn not in fnprev:
> fnprev[fn] = p
> else:
> if fnprev[fn]:
hmm, the condition can't be false
> pr(fn, fnprev[fn])
> fnprev[fn] = None
> pr(fn, p)
and we'll miss next duplication..
But I like the idea.
>
> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
> the number of functions with the same name we touch. If it's more than
> one, we print.
>
> Reports about a dozen function names for the whole tree in my testing.
> Inspecting the changes to them manually is feasible. None of them are
> in files touched by this series.
>
> The line printed for the first match is pretty useless for me: it points
> to a Coccinelle temporary file *shrug*.
>
> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
> sure hope we don't do that. Multiple @local_err variables... hmm.
> Perhaps we could again concoct some script rules to lead us to spots to
> check manually. See below for my attempt.
>
> What's the worst that could happen if we blindly converted such code?
> The answer to that question tells us how hard to work on finding and
> checking these guys.
>
>> +//
>> +// Note also that errp-cleaning functions
>> +// error_free_errp
>> +// error_report_errp
>> +// error_reportf_errp
>> +// warn_report_errp
>> +// warn_reportf_errp
>> +// are not yet implemented. They must call corresponding Error* - freeing
>> +// function and then set *errp to NULL, to avoid further propagation to
>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>> +// For example, error_free_errp may look like this:
>> +//
>> +// void error_free_errp(Error **errp)
>> +// {
>> +// error_free(*errp);
>> +// *errp = NULL;
>> +// }
>> +@ exists@
>> +identifier rule1.fn, rule1.local_err;
>> +expression list args;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +(
>
> Each of the following patterns applies anywhere in the function.
>
> First pattern: delete @local_err
>
>> +- Error *local_err = NULL;
>
> Common case: occurs just once, not nested. Anything else is suspicious.
>
> Both can be detected in the resulting patches with a bit of AWK
> wizardry:
>
> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
> nested
> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
> nested
> static void xen_block_device_destroy(XenBackendInstance *backend,
> nested
> static void xen_block_device_destroy(XenBackendInstance *backend,
> more than one
> static void xen_block_device_destroy(XenBackendInstance *backend,
>
> Oh.
>
> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>
> xen_block_device_destroy() has multiple Error *local_err.
>
> In both cases, manual review is required to ensure the conversion is
> okay. I believe it is.
>
> Note that the AWK script relies on diff showing the function name in @@
> lines, which doesn't always work due to our coding style.
>
> For the whole tree, I get some 30 spots. Feasible.
>
>> +|
>
> Second pattern: clear @errp after freeing it
>
>> +
>> +// Convert error clearing functions
>
> Suggest: Ensure @local_err is cleared on free
>
>> +(
>> +- error_free(local_err);
>> ++ error_free_errp(errp);
>> +|
>> +- error_report_err(local_err);
>> ++ error_report_errp(errp);
>> +|
>> +- error_reportf_err(local_err, args);
>> ++ error_reportf_errp(errp, args);
>> +|
>> +- warn_report_err(local_err);
>> ++ warn_report_errp(errp);
>> +|
>> +- warn_reportf_err(local_err, args);
>> ++ warn_reportf_errp(errp, args);
>> +)
>
> As you mention above, these guys don't exist, yet. Builds anyway,
> because this part of the rule is not used in this patch series. You
> don't want to omit it, because then the script becomes unsafe to use.
>
> We could also open-code:
>
> // Convert error clearing functions
> (
> - error_free(local_err);
> + error_free(*errp);
> + *errp = NULL;
> |
> ... and so forth ...
> )
>
> Matter of taste. Whatever is easier to explain in the comments. Since
> you already wrote one...
I just feel that using helper functions is safer way..
>
> We talked about extending this series slightly so these guys are used.
> I may still look into that.
>
>> +?- local_err = NULL;
>> +
>
> The new helpers clear @local_err. Assignment now redundant, delete.
> Okay.
>
>> +|
>
> Third and fourth pattern: delete error_propagate()
>
>> +- error_propagate_prepend(errp, local_err, args);
>> ++ error_prepend(errp, args);
>> +|
>> +- error_propagate(errp, local_err);
>> +|
>
> Fifth pattern: use @errp directly
>
>> +- &local_err
>> ++ errp
>> +)
>> + ...>
>> + }
>> +
>> +// Convert remaining local_err usage. It should be different kinds of error
>> +// checking in if operators. We can't merge this into previous hunk, as this
>
> In if conditionals, I suppose. It's the case for this patch. If I
> apply the script to the whole tree, the rule gets also applied in other
> contexts. The sentence might mislead as much as it helps. Keep it or
> delete it?
Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
>
>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>> +@@
>> +identifier rule1.fn, rule1.local_err;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- local_err
>> ++ *errp
>> + ...>
>> + }
>> +
>> +// Always use the same patter for checking error
>
> s/patter/pattern/
>
>> +@@
>> +identifier rule1.fn;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- *errp != NULL
>> ++ *errp
>> + ...>
>> + }
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-10 7:44 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-10 7:44 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
09.03.2020 12:56, Markus Armbruster wrote:
> Suggest
>
> scripts: Coccinelle script to use auto-propagated errp
>
> or
>
> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>> does corresponding changes in code (look for details in
>> include/qapi/error.h)
>>
>> Usage example:
>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>
> Suggest FILES... instead of a specific set of files.
>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>
>> Cc: Eric Blake <eblake@redhat.com>
>> Cc: Kevin Wolf <kwolf@redhat.com>
>> Cc: Max Reitz <mreitz@redhat.com>
>> Cc: Greg Kurz <groug@kaod.org>
>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>> Cc: Anthony Perard <anthony.perard@citrix.com>
>> Cc: Paul Durrant <paul@xen.org>
>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>> Cc: Markus Armbruster <armbru@redhat.com>
>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>> Cc: qemu-block@nongnu.org
>> Cc: qemu-devel@nongnu.org
>> Cc: xen-devel@lists.xenproject.org
>>
>> include/qapi/error.h | 3 +
>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>> 2 files changed, 234 insertions(+)
>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>
>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>> index bb9bcf02fb..fbfc6f1c0b 100644
>> --- a/include/qapi/error.h
>> +++ b/include/qapi/error.h
>> @@ -211,6 +211,9 @@
>> * }
>> * ...
>> * }
>> + *
>> + * For mass conversion use script
>
> mass-conversion (we're not converting mass, we're converting en masse)
>
>> + * scripts/coccinelle/auto-propagated-errp.cocci
>> */
>>
>> #ifndef ERROR_H
>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>> new file mode 100644
>> index 0000000000..bff274bd6d
>> --- /dev/null
>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>
> Preface to my review of this script: may aim isn't to make it
> bullet-proof. I want to (1) make it good enough (explained in a
> jiffie), and (2) automatically identify the spots where it still isn't
> obviously safe for manual review.
>
> The latter may involve additional scripting. That's okay.
>
> The script is good enough when the number of possibly unsafe spots is
> low enough for careful manual review.
>
> When I ask for improvements that, in your opinion, go beyond "good
> enough", please push back. I'm sure we can work it out together.
>
>> @@ -0,0 +1,231 @@
>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>> +//
>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>> +//
>> +// This program is free software; you can redistribute it and/or modify
>> +// it under the terms of the GNU General Public License as published by
>> +// the Free Software Foundation; either version 2 of the License, or
>> +// (at your option) any later version.
>> +//
>> +// This program is distributed in the hope that it will be useful,
>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +// GNU General Public License for more details.
>> +//
>> +// You should have received a copy of the GNU General Public License
>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>> +//
>> +// Usage example:
>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>
> You have --max-width 80 here, but not in the commit message. Default
> seems to be 78. Any particular reason to change it to 80?
Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
Still, may be I'm wrong, we can check it.
>
>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>> +
>> +// Switch unusual (Error **) parameter names to errp
>
> Let's drop the parenthesis around Error **
>
>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>
> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
> make the fact we're messing with @errp more obvious. Too late; I
> shouldn't rock the boat that much now.
>
>> +//
>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>> +// parameter.
>> +//
>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>> +// non generic semantics and may have unusual Error ** argument name for purpose
>
> non-generic
>
> for a purpose
>
> Wrap comment lines around column 70, please. It's easier to read.
>
> Maybe
>
> // Skip functions with "assert(_errp && *_errp)" statement, because that
> // signals unusual semantics, and the parameter name may well serve a
> // purpose.
Sounds good.
>
>> +// (like nbd_iter_channel_error()).
>> +//
>> +// Skip util/error.c to not touch, for example, error_propagate and
>> +// error_propagate_prepend().
>
> error_propagate()
>
> I much appreciate your meticulous explanation of what you skip and why.
>
>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>> +identifier fn;
>> +identifier _errp != errp;
>> +@@
>> +
>> + fn(...,
>> +- Error **_errp
>> ++ Error **errp
>> + ,...)
>> + {
>> +(
>> + ... when != assert(_errp && *_errp)
>> +&
>> + <...
>> +- _errp
>> ++ errp
>> + ...>
>> +)
>> + }
>
> This rule is required to make the actual transformations (below) work
> even for parameters with names other than @errp. I believe it's not
> used in this series. In fact, I can't see a use for it in the entire
> tree right now. Okay anyway.
>
>> +
>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>> +//
>> +// Note, that without "when any" final "..." may not want to mach something
>
> s/final "..." may not mach/the final "..." does not match/
>
>> +// matched by previous pattern, i.e. the rule will not match double
>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>
> Can't say I fully understand Coccinelle there. I figure you came to
> this knowledge the hard way.
It's follows from smpl grammar document:
"Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
...
"_when any_ removes the aforementioned constraint that “...” matches the shortest path"
>
>> +//
>> +// Note, "exists" says that we want apply rule even if it matches not on
>> +// all possible control flows (otherwise, it will not match standard pattern
>> +// when error_propagate() call is in if branch).
>
> Learned something new. Example: kvm_set_kvm_shadow_mem().
>
> Spelling it "exists disable optional_qualifier" would avoid giving
> readers the idea we're disabling "exists", but Coccinelle doesn't let
> us. Oh well.
>
>> +@ disable optional_qualifier exists@
>> +identifier fn, local_err, errp;
>
> I believe this causes
>
> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>
> Making @errp symbol instead of identifier should fix this.
Hmm, I didn't see these warnings.. But yes, it should be symbol.
>
>> +@@
>> +
>> + fn(..., Error **errp, ...)
>> + {
>> ++ ERRP_AUTO_PROPAGATE();
>> + ... when != ERRP_AUTO_PROPAGATE();
>> +(
>> + error_append_hint(errp, ...);
>> +|
>> + error_prepend(errp, ...);
>> +|
>> + error_vprepend(errp, ...);
>> +|
>> + Error *local_err = NULL;
>> + ...
>> +(
>> + error_propagate_prepend(errp, local_err, ...);
>> +|
>> + error_propagate(errp, local_err);
>> +)
>> +)
>> + ... when any
>> + }
>> +
>> +
>> +// Match scenarios with propagation of local error to errp.
>> +@rule1 disable optional_qualifier exists@
>> +identifier fn, local_err;
>> +symbol errp;
>> +@@
>> +
>> + fn(..., Error **errp, ...)
>> + {
>> + ...
>> + Error *local_err = NULL;
>> + ...
>> +(
>> + error_propagate_prepend(errp, local_err, ...);
>> +|
>> + error_propagate(errp, local_err);
>> +)
>
> Indentation off by one.
>
>> + ...
>> + }
>> +
>> +// Convert special case with goto in separate.
>
> s/in separate/separately/
>
>> +// We can probably merge this into the following hunk with help of ( | )
>> +// operator, but it significantly reduce performance on block.c parsing (or it
>
> s/reduce/reduces/
>
>> +// hangs, I don't know)
>
> Sounds like you tried to merge this into the following hunk, but then
> spatch took so long on block.c that you killed it. Correct?
Yes.
>
>> +//
>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>> +// things later after all transformations (the rule will be the same, just
>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>
> Weird, but not worth further investigation.
It partially match to the idea which I saw somewhere in coccinelle documentation,
that coccinelle converts correct C code to correct C code. "out: }" is an example
of incorrect, impossible code flow, and coccinelle can't work with it... But it's
just a thought.
>
>> +@@
>> +identifier rule1.fn, rule1.local_err, out;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- goto out;
>> ++ return;
>> + ...>
>> +- out:
>> +- error_propagate(errp, local_err);
>
> You neglect to match error_propagate_prepend(). Okay, because (1) that
> pattern doesn't occur in the tree right now, and (2) if it gets added,
> gcc will complain.
No, because it should not removed. error_propagate_prepend should be converted
to prepend, not removed. So, corresponding gotos should not be removed as well.
>
>> + }
>> +
>> +// Convert most of local_err related staff.
>
> s/staff/stuff/
>
>> +//
>> +// Note, that we update everything related to matched by rule1 function name
>> +// and local_err name. We may match something not related to the pattern
>> +// matched by rule1. For example, local_err may be defined with the same name
>> +// in different blocks inside one function, and in one block follow the
>> +// propagation pattern and in other block doesn't. Or we may have several
>> +// functions with the same name (for different configurations).
>
> Context: rule1 matches functions that have all three of
>
> * an Error **errp parameter
>
> * an Error *local_err = NULL variable declaration
>
> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
> local_err, ...) expression, where @errp is the parameter and
> @local_err is the variable.
>
> If I understand you correctly, you're pointing out two potential issues:
>
> 1. This rule can match functions rule1 does not match if there is
> another function with the same name that rule1 does match.
>
> 2. This rule matches in the entire function matched by rule1, even when
> parts of that function use a different @errp or @local_err.
>
> I figure these apply to all rules with identifier rule1.fn, not just
> this one. Correct?
Yes.
>
> Regarding 1. There must be a better way to chain rules together, but I
> don't know it.
> Can we make Coccinelle at least warn us when it converts
> multiple functions with the same name? What about this:
>
> @initialize:python@
> @@
> fnprev = {}
>
> def pr(fn, p):
> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>
> @r@
> identifier rule1.fn;
> position p;
> @@
> fn(...)@p
> {
> ...
> }
> @script:python@
> fn << rule1.fn;
> p << r.p;
> @@
> if fn not in fnprev:
> fnprev[fn] = p
> else:
> if fnprev[fn]:
hmm, the condition can't be false
> pr(fn, fnprev[fn])
> fnprev[fn] = None
> pr(fn, p)
and we'll miss next duplication..
But I like the idea.
>
> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
> the number of functions with the same name we touch. If it's more than
> one, we print.
>
> Reports about a dozen function names for the whole tree in my testing.
> Inspecting the changes to them manually is feasible. None of them are
> in files touched by this series.
>
> The line printed for the first match is pretty useless for me: it points
> to a Coccinelle temporary file *shrug*.
>
> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
> sure hope we don't do that. Multiple @local_err variables... hmm.
> Perhaps we could again concoct some script rules to lead us to spots to
> check manually. See below for my attempt.
>
> What's the worst that could happen if we blindly converted such code?
> The answer to that question tells us how hard to work on finding and
> checking these guys.
>
>> +//
>> +// Note also that errp-cleaning functions
>> +// error_free_errp
>> +// error_report_errp
>> +// error_reportf_errp
>> +// warn_report_errp
>> +// warn_reportf_errp
>> +// are not yet implemented. They must call corresponding Error* - freeing
>> +// function and then set *errp to NULL, to avoid further propagation to
>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>> +// For example, error_free_errp may look like this:
>> +//
>> +// void error_free_errp(Error **errp)
>> +// {
>> +// error_free(*errp);
>> +// *errp = NULL;
>> +// }
>> +@ exists@
>> +identifier rule1.fn, rule1.local_err;
>> +expression list args;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +(
>
> Each of the following patterns applies anywhere in the function.
>
> First pattern: delete @local_err
>
>> +- Error *local_err = NULL;
>
> Common case: occurs just once, not nested. Anything else is suspicious.
>
> Both can be detected in the resulting patches with a bit of AWK
> wizardry:
>
> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
> nested
> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
> nested
> static void xen_block_device_destroy(XenBackendInstance *backend,
> nested
> static void xen_block_device_destroy(XenBackendInstance *backend,
> more than one
> static void xen_block_device_destroy(XenBackendInstance *backend,
>
> Oh.
>
> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>
> xen_block_device_destroy() has multiple Error *local_err.
>
> In both cases, manual review is required to ensure the conversion is
> okay. I believe it is.
>
> Note that the AWK script relies on diff showing the function name in @@
> lines, which doesn't always work due to our coding style.
>
> For the whole tree, I get some 30 spots. Feasible.
>
>> +|
>
> Second pattern: clear @errp after freeing it
>
>> +
>> +// Convert error clearing functions
>
> Suggest: Ensure @local_err is cleared on free
>
>> +(
>> +- error_free(local_err);
>> ++ error_free_errp(errp);
>> +|
>> +- error_report_err(local_err);
>> ++ error_report_errp(errp);
>> +|
>> +- error_reportf_err(local_err, args);
>> ++ error_reportf_errp(errp, args);
>> +|
>> +- warn_report_err(local_err);
>> ++ warn_report_errp(errp);
>> +|
>> +- warn_reportf_err(local_err, args);
>> ++ warn_reportf_errp(errp, args);
>> +)
>
> As you mention above, these guys don't exist, yet. Builds anyway,
> because this part of the rule is not used in this patch series. You
> don't want to omit it, because then the script becomes unsafe to use.
>
> We could also open-code:
>
> // Convert error clearing functions
> (
> - error_free(local_err);
> + error_free(*errp);
> + *errp = NULL;
> |
> ... and so forth ...
> )
>
> Matter of taste. Whatever is easier to explain in the comments. Since
> you already wrote one...
I just feel that using helper functions is safer way..
>
> We talked about extending this series slightly so these guys are used.
> I may still look into that.
>
>> +?- local_err = NULL;
>> +
>
> The new helpers clear @local_err. Assignment now redundant, delete.
> Okay.
>
>> +|
>
> Third and fourth pattern: delete error_propagate()
>
>> +- error_propagate_prepend(errp, local_err, args);
>> ++ error_prepend(errp, args);
>> +|
>> +- error_propagate(errp, local_err);
>> +|
>
> Fifth pattern: use @errp directly
>
>> +- &local_err
>> ++ errp
>> +)
>> + ...>
>> + }
>> +
>> +// Convert remaining local_err usage. It should be different kinds of error
>> +// checking in if operators. We can't merge this into previous hunk, as this
>
> In if conditionals, I suppose. It's the case for this patch. If I
> apply the script to the whole tree, the rule gets also applied in other
> contexts. The sentence might mislead as much as it helps. Keep it or
> delete it?
Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
>
>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>> +@@
>> +identifier rule1.fn, rule1.local_err;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- local_err
>> ++ *errp
>> + ...>
>> + }
>> +
>> +// Always use the same patter for checking error
>
> s/patter/pattern/
>
>> +@@
>> +identifier rule1.fn;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- *errp != NULL
>> ++ *errp
>> + ...>
>> + }
>
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-10 7:44 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-10 15:47 ` Markus Armbruster
-1 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-10 15:47 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 09.03.2020 12:56, Markus Armbruster wrote:
>> Suggest
>>
>> scripts: Coccinelle script to use auto-propagated errp
>>
>> or
>>
>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>
>>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>>> does corresponding changes in code (look for details in
>>> include/qapi/error.h)
>>>
>>> Usage example:
>>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>
>> Suggest FILES... instead of a specific set of files.
>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>>
>>> Cc: Eric Blake <eblake@redhat.com>
>>> Cc: Kevin Wolf <kwolf@redhat.com>
>>> Cc: Max Reitz <mreitz@redhat.com>
>>> Cc: Greg Kurz <groug@kaod.org>
>>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>>> Cc: Anthony Perard <anthony.perard@citrix.com>
>>> Cc: Paul Durrant <paul@xen.org>
>>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>>> Cc: Markus Armbruster <armbru@redhat.com>
>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>> Cc: qemu-block@nongnu.org
>>> Cc: qemu-devel@nongnu.org
>>> Cc: xen-devel@lists.xenproject.org
>>>
>>> include/qapi/error.h | 3 +
>>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>>> 2 files changed, 234 insertions(+)
>>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>
>>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>>> index bb9bcf02fb..fbfc6f1c0b 100644
>>> --- a/include/qapi/error.h
>>> +++ b/include/qapi/error.h
>>> @@ -211,6 +211,9 @@
>>> * }
>>> * ...
>>> * }
>>> + *
>>> + * For mass conversion use script
>>
>> mass-conversion (we're not converting mass, we're converting en masse)
>>
>>> + * scripts/coccinelle/auto-propagated-errp.cocci
>>> */
>>> #ifndef ERROR_H
>>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>>> new file mode 100644
>>> index 0000000000..bff274bd6d
>>> --- /dev/null
>>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>>
>> Preface to my review of this script: may aim isn't to make it
>> bullet-proof. I want to (1) make it good enough (explained in a
>> jiffie), and (2) automatically identify the spots where it still isn't
>> obviously safe for manual review.
>>
>> The latter may involve additional scripting. That's okay.
>>
>> The script is good enough when the number of possibly unsafe spots is
>> low enough for careful manual review.
>>
>> When I ask for improvements that, in your opinion, go beyond "good
>> enough", please push back. I'm sure we can work it out together.
>>
>>> @@ -0,0 +1,231 @@
>>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>>> +//
>>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>>> +//
>>> +// This program is free software; you can redistribute it and/or modify
>>> +// it under the terms of the GNU General Public License as published by
>>> +// the Free Software Foundation; either version 2 of the License, or
>>> +// (at your option) any later version.
>>> +//
>>> +// This program is distributed in the hope that it will be useful,
>>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>> +// GNU General Public License for more details.
>>> +//
>>> +// You should have received a copy of the GNU General Public License
>>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>>> +//
>>> +// Usage example:
>>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>>
>> You have --max-width 80 here, but not in the commit message. Default
>> seems to be 78. Any particular reason to change it to 80?
>
> Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
> So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
> Still, may be I'm wrong, we can check it.
If you can point to an example where --max-width helps, keep it, and
update the commit message to match. Else, drop it.
>>
>>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>> +
>>> +// Switch unusual (Error **) parameter names to errp
>>
>> Let's drop the parenthesis around Error **
>>
>>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>>
>> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
>> make the fact we're messing with @errp more obvious. Too late; I
>> shouldn't rock the boat that much now.
>>
>>> +//
>>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>>> +// parameter.
>>> +//
>>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>>> +// non generic semantics and may have unusual Error ** argument name for purpose
>>
>> non-generic
>>
>> for a purpose
>>
>> Wrap comment lines around column 70, please. It's easier to read.
>>
>> Maybe
>>
>> // Skip functions with "assert(_errp && *_errp)" statement, because that
>> // signals unusual semantics, and the parameter name may well serve a
>> // purpose.
>
> Sounds good.
>
>>
>>> +// (like nbd_iter_channel_error()).
>>> +//
>>> +// Skip util/error.c to not touch, for example, error_propagate and
>>> +// error_propagate_prepend().
>>
>> error_propagate()
>>
>> I much appreciate your meticulous explanation of what you skip and why.
>>
>>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>>> +identifier fn;
>>> +identifier _errp != errp;
>>> +@@
>>> +
>>> + fn(...,
>>> +- Error **_errp
>>> ++ Error **errp
>>> + ,...)
>>> + {
>>> +(
>>> + ... when != assert(_errp && *_errp)
>>> +&
>>> + <...
>>> +- _errp
>>> ++ errp
>>> + ...>
>>> +)
>>> + }
>>
>> This rule is required to make the actual transformations (below) work
>> even for parameters with names other than @errp. I believe it's not
>> used in this series. In fact, I can't see a use for it in the entire
>> tree right now. Okay anyway.
>>
>>> +
>>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>>> +//
>>> +// Note, that without "when any" final "..." may not want to mach something
>>
>> s/final "..." may not mach/the final "..." does not match/
>>
>>> +// matched by previous pattern, i.e. the rule will not match double
>>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>>
>> Can't say I fully understand Coccinelle there. I figure you came to
>> this knowledge the hard way.
>
> It's follows from smpl grammar document:
>
> "Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
> ...
> "_when any_ removes the aforementioned constraint that “...” matches the shortest path"
Let me think that through.
The pattern with the cases other than error_prepend() omitted:
fn(..., Error **errp, ...)
{
+ ERRP_AUTO_PROPAGATE();
... when != ERRP_AUTO_PROPAGATE();
error_prepend(errp, ...);
... when any
}
Tail of vfio_set_irq_signaling():
name = index_to_str(vbasedev, index);
if (name) {
error_prepend(errp, "%s-%d: ", name, subindex);
} else {
error_prepend(errp, "index %d-%d: ", index, subindex);
}
error_prepend(errp,
"Failed to %s %s eventfd signaling for interrupt ",
fd < 0 ? "tear down" : "set up", action_to_str(action));
return ret;
}
The pattern's first ... matches a "shortest" path to an error_prepend(),
where "shortest" means "does not cross an error_prepend(). Its when
clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().
There are two such "shortest" paths, one to the first error_prepend() in
vfio_set_irq_signaling(), and one to the second. Neither path to the
third one is not "shortest": they both cross one of the other two
error_prepend().
The pattern' s second ... matches a path from a matched error_prepend()
to the end of the function. There are two paths. Both cross the third
error_prepend(). You need "when any" to make the pattern match anyway.
Alright, I think I got it. But now I'm paranoid about ... elsewhere.
For instance, here's rule1 with error_propagate_prepend() omitted:
// Match scenarios with propagation of local error to errp.
@rule1 disable optional_qualifier exists@
identifier fn, local_err;
symbol errp;
@@
fn(..., Error **errp, ...)
{
...
Error *local_err = NULL;
...
error_propagate(errp, local_err);
...
}
The second and third ... won't match anything containing
error_propagate(). What if a function has multiple error_propagate() on
all paths? Like this one:
extern foo(int, Error **);
extern bar(int, Error **);
void frob(Error **errp)
{
Error *local_err = NULL;
int arg;
foo(arg, errp);
bar(arg, &local_err);
error_propagate(errp, local_err);
bar(arg + 1, &local_err);
error_propagate(errp, local_err);
}
This is actually a variation of error.h's "Receive and accumulate
multiple errors (first one wins)" code snippet.
The Coccinelle script transforms it like this:
void frob(Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
Error *local_err = NULL;
int arg;
The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any),
but rule1 does not, and we therefore don't convert any of the
error_propagate().
The result isn't wrong, just useless.
Is this the worst case?
Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use
"... when any" in the error_propagate() case, only in the other cases.
Would that help?
I think this is the only other rule with "..." matching control flow.
>>
>>> +//
>>> +// Note, "exists" says that we want apply rule even if it matches not on
>>> +// all possible control flows (otherwise, it will not match standard pattern
>>> +// when error_propagate() call is in if branch).
>>
>> Learned something new. Example: kvm_set_kvm_shadow_mem().
>>
>> Spelling it "exists disable optional_qualifier" would avoid giving
>> readers the idea we're disabling "exists", but Coccinelle doesn't let
>> us. Oh well.
>>
>>> +@ disable optional_qualifier exists@
>>> +identifier fn, local_err, errp;
>>
>> I believe this causes
>>
>> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>>
>> Making @errp symbol instead of identifier should fix this.
>
> Hmm, I didn't see these warnings.. But yes, it should be symbol.
>
>>
>>> +@@
>>> +
>>> + fn(..., Error **errp, ...)
>>> + {
>>> ++ ERRP_AUTO_PROPAGATE();
>>> + ... when != ERRP_AUTO_PROPAGATE();
>>> +(
>>> + error_append_hint(errp, ...);
>>> +|
>>> + error_prepend(errp, ...);
>>> +|
>>> + error_vprepend(errp, ...);
>>> +|
>>> + Error *local_err = NULL;
>>> + ...
>>> +(
>>> + error_propagate_prepend(errp, local_err, ...);
>>> +|
>>> + error_propagate(errp, local_err);
>>> +)
>>> +)
>>> + ... when any
>>> + }
>>> +
>>> +
>>> +// Match scenarios with propagation of local error to errp.
>>> +@rule1 disable optional_qualifier exists@
>>> +identifier fn, local_err;
>>> +symbol errp;
>>> +@@
>>> +
>>> + fn(..., Error **errp, ...)
>>> + {
>>> + ...
>>> + Error *local_err = NULL;
>>> + ...
>>> +(
>>> + error_propagate_prepend(errp, local_err, ...);
>>> +|
>>> + error_propagate(errp, local_err);
>>> +)
>>
>> Indentation off by one.
>>
>>> + ...
>>> + }
>>> +
>>> +// Convert special case with goto in separate.
>>
>> s/in separate/separately/
>>
>>> +// We can probably merge this into the following hunk with help of ( | )
>>> +// operator, but it significantly reduce performance on block.c parsing (or it
>>
>> s/reduce/reduces/
>>
>>> +// hangs, I don't know)
>>
>> Sounds like you tried to merge this into the following hunk, but then
>> spatch took so long on block.c that you killed it. Correct?
>
> Yes.
I'd say something like "I tried merging this into the following rule the
obvious way, but it made Coccinelle hang on block.c."
>>
>>> +//
>>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>>> +// things later after all transformations (the rule will be the same, just
>>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>>
>> Weird, but not worth further investigation.
>
> It partially match to the idea which I saw somewhere in coccinelle documentation,
> that coccinelle converts correct C code to correct C code. "out: }" is an example
> of incorrect, impossible code flow, and coccinelle can't work with it... But it's
> just a thought.
>
>>
>>> +@@
>>> +identifier rule1.fn, rule1.local_err, out;
>>> +symbol errp;
>>> +@@
>>> +
>>> + fn(...)
>>> + {
>>> + <...
>>> +- goto out;
>>> ++ return;
>>> + ...>
>>> +- out:
>>> +- error_propagate(errp, local_err);
>>
>> You neglect to match error_propagate_prepend(). Okay, because (1) that
>> pattern doesn't occur in the tree right now, and (2) if it gets added,
>> gcc will complain.
>
> No, because it should not removed. error_propagate_prepend should be converted
> to prepend, not removed. So, corresponding gotos should not be removed as well.
You're right.
>>
>>> + }
>>> +
>>> +// Convert most of local_err related staff.
>>
>> s/staff/stuff/
>>
>>> +//
>>> +// Note, that we update everything related to matched by rule1 function name
>>> +// and local_err name. We may match something not related to the pattern
>>> +// matched by rule1. For example, local_err may be defined with the same name
>>> +// in different blocks inside one function, and in one block follow the
>>> +// propagation pattern and in other block doesn't. Or we may have several
>>> +// functions with the same name (for different configurations).
>>
>> Context: rule1 matches functions that have all three of
>>
>> * an Error **errp parameter
>>
>> * an Error *local_err = NULL variable declaration
>>
>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>> local_err, ...) expression, where @errp is the parameter and
>> @local_err is the variable.
>>
>> If I understand you correctly, you're pointing out two potential issues:
>>
>> 1. This rule can match functions rule1 does not match if there is
>> another function with the same name that rule1 does match.
>>
>> 2. This rule matches in the entire function matched by rule1, even when
>> parts of that function use a different @errp or @local_err.
>>
>> I figure these apply to all rules with identifier rule1.fn, not just
>> this one. Correct?
>
> Yes.
Thanks!
>>
>> Regarding 1. There must be a better way to chain rules together, but I
>> don't know it.
>> Can we make Coccinelle at least warn us when it converts
>> multiple functions with the same name? What about this:
>>
>> @initialize:python@
>> @@
>> fnprev = {}
>>
>> def pr(fn, p):
>> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>>
>> @r@
>> identifier rule1.fn;
>> position p;
>> @@
>> fn(...)@p
>> {
>> ...
>> }
>> @script:python@
>> fn << rule1.fn;
>> p << r.p;
>> @@
>> if fn not in fnprev:
>> fnprev[fn] = p
>> else:
>> if fnprev[fn]:
>
> hmm, the condition can't be false
>
>> pr(fn, fnprev[fn])
>> fnprev[fn] = None
>> pr(fn, p)
>
> and we'll miss next duplication..
The idea is
first instance of fn:
fn not in fnprev
fnprev[fn] = position of instance
don't print
second instance:
fnprev[fn] is the position of the first instance
print first two instances
subsequent instances: fnprev[fn] is None
print this instance
I might have screwed up the coding, of course :)
> But I like the idea.
>
>>
>> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
>> the number of functions with the same name we touch. If it's more than
>> one, we print.
>>
>> Reports about a dozen function names for the whole tree in my testing.
>> Inspecting the changes to them manually is feasible. None of them are
>> in files touched by this series.
>>
>> The line printed for the first match is pretty useless for me: it points
>> to a Coccinelle temporary file *shrug*.
>>
>> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
>> sure hope we don't do that. Multiple @local_err variables... hmm.
>> Perhaps we could again concoct some script rules to lead us to spots to
>> check manually. See below for my attempt.
>>
>> What's the worst that could happen if we blindly converted such code?
>> The answer to that question tells us how hard to work on finding and
>> checking these guys.
>>
>>> +//
>>> +// Note also that errp-cleaning functions
>>> +// error_free_errp
>>> +// error_report_errp
>>> +// error_reportf_errp
>>> +// warn_report_errp
>>> +// warn_reportf_errp
>>> +// are not yet implemented. They must call corresponding Error* - freeing
>>> +// function and then set *errp to NULL, to avoid further propagation to
>>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>>> +// For example, error_free_errp may look like this:
>>> +//
>>> +// void error_free_errp(Error **errp)
>>> +// {
>>> +// error_free(*errp);
>>> +// *errp = NULL;
>>> +// }
>>> +@ exists@
>>> +identifier rule1.fn, rule1.local_err;
>>> +expression list args;
>>> +symbol errp;
>>> +@@
>>> +
>>> + fn(...)
>>> + {
>>> + <...
>>> +(
>>
>> Each of the following patterns applies anywhere in the function.
>>
>> First pattern: delete @local_err
>>
>>> +- Error *local_err = NULL;
>>
>> Common case: occurs just once, not nested. Anything else is suspicious.
>>
>> Both can be detected in the resulting patches with a bit of AWK
>> wizardry:
>>
>> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
>> nested
>> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
>> nested
>> static void xen_block_device_destroy(XenBackendInstance *backend,
>> nested
>> static void xen_block_device_destroy(XenBackendInstance *backend,
>> more than one
>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>
>> Oh.
>>
>> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>>
>> xen_block_device_destroy() has multiple Error *local_err.
>>
>> In both cases, manual review is required to ensure the conversion is
>> okay. I believe it is.
>>
>> Note that the AWK script relies on diff showing the function name in @@
>> lines, which doesn't always work due to our coding style.
>>
>> For the whole tree, I get some 30 spots. Feasible.
>>
>>> +|
>>
>> Second pattern: clear @errp after freeing it
>>
>>> +
>>> +// Convert error clearing functions
>>
>> Suggest: Ensure @local_err is cleared on free
>>
>>> +(
>>> +- error_free(local_err);
>>> ++ error_free_errp(errp);
>>> +|
>>> +- error_report_err(local_err);
>>> ++ error_report_errp(errp);
>>> +|
>>> +- error_reportf_err(local_err, args);
>>> ++ error_reportf_errp(errp, args);
>>> +|
>>> +- warn_report_err(local_err);
>>> ++ warn_report_errp(errp);
>>> +|
>>> +- warn_reportf_err(local_err, args);
>>> ++ warn_reportf_errp(errp, args);
>>> +)
>>
>> As you mention above, these guys don't exist, yet. Builds anyway,
>> because this part of the rule is not used in this patch series. You
>> don't want to omit it, because then the script becomes unsafe to use.
>>
>> We could also open-code:
>>
>> // Convert error clearing functions
>> (
>> - error_free(local_err);
>> + error_free(*errp);
>> + *errp = NULL;
>> |
>> ... and so forth ...
>> )
>>
>> Matter of taste. Whatever is easier to explain in the comments. Since
>> you already wrote one...
>
> I just feel that using helper functions is safer way..
>
>>
>> We talked about extending this series slightly so these guys are used.
>> I may still look into that.
>>
>>> +?- local_err = NULL;
>>> +
>>
>> The new helpers clear @local_err. Assignment now redundant, delete.
>> Okay.
>>
>>> +|
>>
>> Third and fourth pattern: delete error_propagate()
>>
>>> +- error_propagate_prepend(errp, local_err, args);
>>> ++ error_prepend(errp, args);
>>> +|
>>> +- error_propagate(errp, local_err);
>>> +|
>>
>> Fifth pattern: use @errp directly
>>
>>> +- &local_err
>>> ++ errp
>>> +)
>>> + ...>
>>> + }
>>> +
>>> +// Convert remaining local_err usage. It should be different kinds of error
>>> +// checking in if operators. We can't merge this into previous hunk, as this
>>
>> In if conditionals, I suppose. It's the case for this patch. If I
>> apply the script to the whole tree, the rule gets also applied in other
>> contexts. The sentence might mislead as much as it helps. Keep it or
>> delete it?
>
> Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
"Need to be careful" means "needs careful manual review", which I
believe is not feasible; see "Preface to my review of this script"
above.
But do we really need to be careful here?
This rule should apply only where we added ERRP_AUTO_PROPAGATE().
Except when rule chaining via function name fails us, but we plan to
detect that and review manually, so let's ignore this issue here.
Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling
replacement of @local_err by @errp is its whole point.
What exactly do we need to be careful about?
>
>>
>>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>>> +@@
>>> +identifier rule1.fn, rule1.local_err;
>>> +symbol errp;
>>> +@@
>>> +
>>> + fn(...)
>>> + {
>>> + <...
>>> +- local_err
>>> ++ *errp
>>> + ...>
>>> + }
>>> +
>>> +// Always use the same patter for checking error
>>
>> s/patter/pattern/
>>
>>> +@@
>>> +identifier rule1.fn;
>>> +symbol errp;
>>> +@@
>>> +
>>> + fn(...)
>>> + {
>>> + <...
>>> +- *errp != NULL
>>> ++ *errp
>>> + ...>
>>> + }
>>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-10 15:47 ` Markus Armbruster
0 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-10 15:47 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 09.03.2020 12:56, Markus Armbruster wrote:
>> Suggest
>>
>> scripts: Coccinelle script to use auto-propagated errp
>>
>> or
>>
>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>
>>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>>> does corresponding changes in code (look for details in
>>> include/qapi/error.h)
>>>
>>> Usage example:
>>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>
>> Suggest FILES... instead of a specific set of files.
>>
>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>> ---
>>>
>>> Cc: Eric Blake <eblake@redhat.com>
>>> Cc: Kevin Wolf <kwolf@redhat.com>
>>> Cc: Max Reitz <mreitz@redhat.com>
>>> Cc: Greg Kurz <groug@kaod.org>
>>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>>> Cc: Anthony Perard <anthony.perard@citrix.com>
>>> Cc: Paul Durrant <paul@xen.org>
>>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>>> Cc: Markus Armbruster <armbru@redhat.com>
>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>> Cc: qemu-block@nongnu.org
>>> Cc: qemu-devel@nongnu.org
>>> Cc: xen-devel@lists.xenproject.org
>>>
>>> include/qapi/error.h | 3 +
>>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>>> 2 files changed, 234 insertions(+)
>>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>
>>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>>> index bb9bcf02fb..fbfc6f1c0b 100644
>>> --- a/include/qapi/error.h
>>> +++ b/include/qapi/error.h
>>> @@ -211,6 +211,9 @@
>>> * }
>>> * ...
>>> * }
>>> + *
>>> + * For mass conversion use script
>>
>> mass-conversion (we're not converting mass, we're converting en masse)
>>
>>> + * scripts/coccinelle/auto-propagated-errp.cocci
>>> */
>>> #ifndef ERROR_H
>>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>>> new file mode 100644
>>> index 0000000000..bff274bd6d
>>> --- /dev/null
>>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>>
>> Preface to my review of this script: may aim isn't to make it
>> bullet-proof. I want to (1) make it good enough (explained in a
>> jiffie), and (2) automatically identify the spots where it still isn't
>> obviously safe for manual review.
>>
>> The latter may involve additional scripting. That's okay.
>>
>> The script is good enough when the number of possibly unsafe spots is
>> low enough for careful manual review.
>>
>> When I ask for improvements that, in your opinion, go beyond "good
>> enough", please push back. I'm sure we can work it out together.
>>
>>> @@ -0,0 +1,231 @@
>>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>>> +//
>>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>>> +//
>>> +// This program is free software; you can redistribute it and/or modify
>>> +// it under the terms of the GNU General Public License as published by
>>> +// the Free Software Foundation; either version 2 of the License, or
>>> +// (at your option) any later version.
>>> +//
>>> +// This program is distributed in the hope that it will be useful,
>>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>> +// GNU General Public License for more details.
>>> +//
>>> +// You should have received a copy of the GNU General Public License
>>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>>> +//
>>> +// Usage example:
>>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>>
>> You have --max-width 80 here, but not in the commit message. Default
>> seems to be 78. Any particular reason to change it to 80?
>
> Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
> So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
> Still, may be I'm wrong, we can check it.
If you can point to an example where --max-width helps, keep it, and
update the commit message to match. Else, drop it.
>>
>>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>> +
>>> +// Switch unusual (Error **) parameter names to errp
>>
>> Let's drop the parenthesis around Error **
>>
>>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>>
>> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
>> make the fact we're messing with @errp more obvious. Too late; I
>> shouldn't rock the boat that much now.
>>
>>> +//
>>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>>> +// parameter.
>>> +//
>>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>>> +// non generic semantics and may have unusual Error ** argument name for purpose
>>
>> non-generic
>>
>> for a purpose
>>
>> Wrap comment lines around column 70, please. It's easier to read.
>>
>> Maybe
>>
>> // Skip functions with "assert(_errp && *_errp)" statement, because that
>> // signals unusual semantics, and the parameter name may well serve a
>> // purpose.
>
> Sounds good.
>
>>
>>> +// (like nbd_iter_channel_error()).
>>> +//
>>> +// Skip util/error.c to not touch, for example, error_propagate and
>>> +// error_propagate_prepend().
>>
>> error_propagate()
>>
>> I much appreciate your meticulous explanation of what you skip and why.
>>
>>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>>> +identifier fn;
>>> +identifier _errp != errp;
>>> +@@
>>> +
>>> + fn(...,
>>> +- Error **_errp
>>> ++ Error **errp
>>> + ,...)
>>> + {
>>> +(
>>> + ... when != assert(_errp && *_errp)
>>> +&
>>> + <...
>>> +- _errp
>>> ++ errp
>>> + ...>
>>> +)
>>> + }
>>
>> This rule is required to make the actual transformations (below) work
>> even for parameters with names other than @errp. I believe it's not
>> used in this series. In fact, I can't see a use for it in the entire
>> tree right now. Okay anyway.
>>
>>> +
>>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>>> +//
>>> +// Note, that without "when any" final "..." may not want to mach something
>>
>> s/final "..." may not mach/the final "..." does not match/
>>
>>> +// matched by previous pattern, i.e. the rule will not match double
>>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>>
>> Can't say I fully understand Coccinelle there. I figure you came to
>> this knowledge the hard way.
>
> It's follows from smpl grammar document:
>
> "Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
> ...
> "_when any_ removes the aforementioned constraint that “...” matches the shortest path"
Let me think that through.
The pattern with the cases other than error_prepend() omitted:
fn(..., Error **errp, ...)
{
+ ERRP_AUTO_PROPAGATE();
... when != ERRP_AUTO_PROPAGATE();
error_prepend(errp, ...);
... when any
}
Tail of vfio_set_irq_signaling():
name = index_to_str(vbasedev, index);
if (name) {
error_prepend(errp, "%s-%d: ", name, subindex);
} else {
error_prepend(errp, "index %d-%d: ", index, subindex);
}
error_prepend(errp,
"Failed to %s %s eventfd signaling for interrupt ",
fd < 0 ? "tear down" : "set up", action_to_str(action));
return ret;
}
The pattern's first ... matches a "shortest" path to an error_prepend(),
where "shortest" means "does not cross an error_prepend(). Its when
clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().
There are two such "shortest" paths, one to the first error_prepend() in
vfio_set_irq_signaling(), and one to the second. Neither path to the
third one is not "shortest": they both cross one of the other two
error_prepend().
The pattern' s second ... matches a path from a matched error_prepend()
to the end of the function. There are two paths. Both cross the third
error_prepend(). You need "when any" to make the pattern match anyway.
Alright, I think I got it. But now I'm paranoid about ... elsewhere.
For instance, here's rule1 with error_propagate_prepend() omitted:
// Match scenarios with propagation of local error to errp.
@rule1 disable optional_qualifier exists@
identifier fn, local_err;
symbol errp;
@@
fn(..., Error **errp, ...)
{
...
Error *local_err = NULL;
...
error_propagate(errp, local_err);
...
}
The second and third ... won't match anything containing
error_propagate(). What if a function has multiple error_propagate() on
all paths? Like this one:
extern foo(int, Error **);
extern bar(int, Error **);
void frob(Error **errp)
{
Error *local_err = NULL;
int arg;
foo(arg, errp);
bar(arg, &local_err);
error_propagate(errp, local_err);
bar(arg + 1, &local_err);
error_propagate(errp, local_err);
}
This is actually a variation of error.h's "Receive and accumulate
multiple errors (first one wins)" code snippet.
The Coccinelle script transforms it like this:
void frob(Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
Error *local_err = NULL;
int arg;
The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any),
but rule1 does not, and we therefore don't convert any of the
error_propagate().
The result isn't wrong, just useless.
Is this the worst case?
Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use
"... when any" in the error_propagate() case, only in the other cases.
Would that help?
I think this is the only other rule with "..." matching control flow.
>>
>>> +//
>>> +// Note, "exists" says that we want apply rule even if it matches not on
>>> +// all possible control flows (otherwise, it will not match standard pattern
>>> +// when error_propagate() call is in if branch).
>>
>> Learned something new. Example: kvm_set_kvm_shadow_mem().
>>
>> Spelling it "exists disable optional_qualifier" would avoid giving
>> readers the idea we're disabling "exists", but Coccinelle doesn't let
>> us. Oh well.
>>
>>> +@ disable optional_qualifier exists@
>>> +identifier fn, local_err, errp;
>>
>> I believe this causes
>>
>> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
>> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>>
>> Making @errp symbol instead of identifier should fix this.
>
> Hmm, I didn't see these warnings.. But yes, it should be symbol.
>
>>
>>> +@@
>>> +
>>> + fn(..., Error **errp, ...)
>>> + {
>>> ++ ERRP_AUTO_PROPAGATE();
>>> + ... when != ERRP_AUTO_PROPAGATE();
>>> +(
>>> + error_append_hint(errp, ...);
>>> +|
>>> + error_prepend(errp, ...);
>>> +|
>>> + error_vprepend(errp, ...);
>>> +|
>>> + Error *local_err = NULL;
>>> + ...
>>> +(
>>> + error_propagate_prepend(errp, local_err, ...);
>>> +|
>>> + error_propagate(errp, local_err);
>>> +)
>>> +)
>>> + ... when any
>>> + }
>>> +
>>> +
>>> +// Match scenarios with propagation of local error to errp.
>>> +@rule1 disable optional_qualifier exists@
>>> +identifier fn, local_err;
>>> +symbol errp;
>>> +@@
>>> +
>>> + fn(..., Error **errp, ...)
>>> + {
>>> + ...
>>> + Error *local_err = NULL;
>>> + ...
>>> +(
>>> + error_propagate_prepend(errp, local_err, ...);
>>> +|
>>> + error_propagate(errp, local_err);
>>> +)
>>
>> Indentation off by one.
>>
>>> + ...
>>> + }
>>> +
>>> +// Convert special case with goto in separate.
>>
>> s/in separate/separately/
>>
>>> +// We can probably merge this into the following hunk with help of ( | )
>>> +// operator, but it significantly reduce performance on block.c parsing (or it
>>
>> s/reduce/reduces/
>>
>>> +// hangs, I don't know)
>>
>> Sounds like you tried to merge this into the following hunk, but then
>> spatch took so long on block.c that you killed it. Correct?
>
> Yes.
I'd say something like "I tried merging this into the following rule the
obvious way, but it made Coccinelle hang on block.c."
>>
>>> +//
>>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>>> +// things later after all transformations (the rule will be the same, just
>>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>>
>> Weird, but not worth further investigation.
>
> It partially match to the idea which I saw somewhere in coccinelle documentation,
> that coccinelle converts correct C code to correct C code. "out: }" is an example
> of incorrect, impossible code flow, and coccinelle can't work with it... But it's
> just a thought.
>
>>
>>> +@@
>>> +identifier rule1.fn, rule1.local_err, out;
>>> +symbol errp;
>>> +@@
>>> +
>>> + fn(...)
>>> + {
>>> + <...
>>> +- goto out;
>>> ++ return;
>>> + ...>
>>> +- out:
>>> +- error_propagate(errp, local_err);
>>
>> You neglect to match error_propagate_prepend(). Okay, because (1) that
>> pattern doesn't occur in the tree right now, and (2) if it gets added,
>> gcc will complain.
>
> No, because it should not removed. error_propagate_prepend should be converted
> to prepend, not removed. So, corresponding gotos should not be removed as well.
You're right.
>>
>>> + }
>>> +
>>> +// Convert most of local_err related staff.
>>
>> s/staff/stuff/
>>
>>> +//
>>> +// Note, that we update everything related to matched by rule1 function name
>>> +// and local_err name. We may match something not related to the pattern
>>> +// matched by rule1. For example, local_err may be defined with the same name
>>> +// in different blocks inside one function, and in one block follow the
>>> +// propagation pattern and in other block doesn't. Or we may have several
>>> +// functions with the same name (for different configurations).
>>
>> Context: rule1 matches functions that have all three of
>>
>> * an Error **errp parameter
>>
>> * an Error *local_err = NULL variable declaration
>>
>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>> local_err, ...) expression, where @errp is the parameter and
>> @local_err is the variable.
>>
>> If I understand you correctly, you're pointing out two potential issues:
>>
>> 1. This rule can match functions rule1 does not match if there is
>> another function with the same name that rule1 does match.
>>
>> 2. This rule matches in the entire function matched by rule1, even when
>> parts of that function use a different @errp or @local_err.
>>
>> I figure these apply to all rules with identifier rule1.fn, not just
>> this one. Correct?
>
> Yes.
Thanks!
>>
>> Regarding 1. There must be a better way to chain rules together, but I
>> don't know it.
>> Can we make Coccinelle at least warn us when it converts
>> multiple functions with the same name? What about this:
>>
>> @initialize:python@
>> @@
>> fnprev = {}
>>
>> def pr(fn, p):
>> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>>
>> @r@
>> identifier rule1.fn;
>> position p;
>> @@
>> fn(...)@p
>> {
>> ...
>> }
>> @script:python@
>> fn << rule1.fn;
>> p << r.p;
>> @@
>> if fn not in fnprev:
>> fnprev[fn] = p
>> else:
>> if fnprev[fn]:
>
> hmm, the condition can't be false
>
>> pr(fn, fnprev[fn])
>> fnprev[fn] = None
>> pr(fn, p)
>
> and we'll miss next duplication..
The idea is
first instance of fn:
fn not in fnprev
fnprev[fn] = position of instance
don't print
second instance:
fnprev[fn] is the position of the first instance
print first two instances
subsequent instances: fnprev[fn] is None
print this instance
I might have screwed up the coding, of course :)
> But I like the idea.
>
>>
>> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
>> the number of functions with the same name we touch. If it's more than
>> one, we print.
>>
>> Reports about a dozen function names for the whole tree in my testing.
>> Inspecting the changes to them manually is feasible. None of them are
>> in files touched by this series.
>>
>> The line printed for the first match is pretty useless for me: it points
>> to a Coccinelle temporary file *shrug*.
>>
>> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
>> sure hope we don't do that. Multiple @local_err variables... hmm.
>> Perhaps we could again concoct some script rules to lead us to spots to
>> check manually. See below for my attempt.
>>
>> What's the worst that could happen if we blindly converted such code?
>> The answer to that question tells us how hard to work on finding and
>> checking these guys.
>>
>>> +//
>>> +// Note also that errp-cleaning functions
>>> +// error_free_errp
>>> +// error_report_errp
>>> +// error_reportf_errp
>>> +// warn_report_errp
>>> +// warn_reportf_errp
>>> +// are not yet implemented. They must call corresponding Error* - freeing
>>> +// function and then set *errp to NULL, to avoid further propagation to
>>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>>> +// For example, error_free_errp may look like this:
>>> +//
>>> +// void error_free_errp(Error **errp)
>>> +// {
>>> +// error_free(*errp);
>>> +// *errp = NULL;
>>> +// }
>>> +@ exists@
>>> +identifier rule1.fn, rule1.local_err;
>>> +expression list args;
>>> +symbol errp;
>>> +@@
>>> +
>>> + fn(...)
>>> + {
>>> + <...
>>> +(
>>
>> Each of the following patterns applies anywhere in the function.
>>
>> First pattern: delete @local_err
>>
>>> +- Error *local_err = NULL;
>>
>> Common case: occurs just once, not nested. Anything else is suspicious.
>>
>> Both can be detected in the resulting patches with a bit of AWK
>> wizardry:
>>
>> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
>> nested
>> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
>> nested
>> static void xen_block_device_destroy(XenBackendInstance *backend,
>> nested
>> static void xen_block_device_destroy(XenBackendInstance *backend,
>> more than one
>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>
>> Oh.
>>
>> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>>
>> xen_block_device_destroy() has multiple Error *local_err.
>>
>> In both cases, manual review is required to ensure the conversion is
>> okay. I believe it is.
>>
>> Note that the AWK script relies on diff showing the function name in @@
>> lines, which doesn't always work due to our coding style.
>>
>> For the whole tree, I get some 30 spots. Feasible.
>>
>>> +|
>>
>> Second pattern: clear @errp after freeing it
>>
>>> +
>>> +// Convert error clearing functions
>>
>> Suggest: Ensure @local_err is cleared on free
>>
>>> +(
>>> +- error_free(local_err);
>>> ++ error_free_errp(errp);
>>> +|
>>> +- error_report_err(local_err);
>>> ++ error_report_errp(errp);
>>> +|
>>> +- error_reportf_err(local_err, args);
>>> ++ error_reportf_errp(errp, args);
>>> +|
>>> +- warn_report_err(local_err);
>>> ++ warn_report_errp(errp);
>>> +|
>>> +- warn_reportf_err(local_err, args);
>>> ++ warn_reportf_errp(errp, args);
>>> +)
>>
>> As you mention above, these guys don't exist, yet. Builds anyway,
>> because this part of the rule is not used in this patch series. You
>> don't want to omit it, because then the script becomes unsafe to use.
>>
>> We could also open-code:
>>
>> // Convert error clearing functions
>> (
>> - error_free(local_err);
>> + error_free(*errp);
>> + *errp = NULL;
>> |
>> ... and so forth ...
>> )
>>
>> Matter of taste. Whatever is easier to explain in the comments. Since
>> you already wrote one...
>
> I just feel that using helper functions is safer way..
>
>>
>> We talked about extending this series slightly so these guys are used.
>> I may still look into that.
>>
>>> +?- local_err = NULL;
>>> +
>>
>> The new helpers clear @local_err. Assignment now redundant, delete.
>> Okay.
>>
>>> +|
>>
>> Third and fourth pattern: delete error_propagate()
>>
>>> +- error_propagate_prepend(errp, local_err, args);
>>> ++ error_prepend(errp, args);
>>> +|
>>> +- error_propagate(errp, local_err);
>>> +|
>>
>> Fifth pattern: use @errp directly
>>
>>> +- &local_err
>>> ++ errp
>>> +)
>>> + ...>
>>> + }
>>> +
>>> +// Convert remaining local_err usage. It should be different kinds of error
>>> +// checking in if operators. We can't merge this into previous hunk, as this
>>
>> In if conditionals, I suppose. It's the case for this patch. If I
>> apply the script to the whole tree, the rule gets also applied in other
>> contexts. The sentence might mislead as much as it helps. Keep it or
>> delete it?
>
> Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
"Need to be careful" means "needs careful manual review", which I
believe is not feasible; see "Preface to my review of this script"
above.
But do we really need to be careful here?
This rule should apply only where we added ERRP_AUTO_PROPAGATE().
Except when rule chaining via function name fails us, but we plan to
detect that and review manually, so let's ignore this issue here.
Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling
replacement of @local_err by @errp is its whole point.
What exactly do we need to be careful about?
>
>>
>>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>>> +@@
>>> +identifier rule1.fn, rule1.local_err;
>>> +symbol errp;
>>> +@@
>>> +
>>> + fn(...)
>>> + {
>>> + <...
>>> +- local_err
>>> ++ *errp
>>> + ...>
>>> + }
>>> +
>>> +// Always use the same patter for checking error
>>
>> s/patter/pattern/
>>
>>> +@@
>>> +identifier rule1.fn;
>>> +symbol errp;
>>> +@@
>>> +
>>> + fn(...)
>>> + {
>>> + <...
>>> +- *errp != NULL
>>> ++ *errp
>>> + ...>
>>> + }
>>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-10 15:47 ` [Xen-devel] " Markus Armbruster
@ 2020-03-11 6:55 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 6:55 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
10.03.2020 18:47, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 09.03.2020 12:56, Markus Armbruster wrote:
>>> Suggest
>>>
>>> scripts: Coccinelle script to use auto-propagated errp
>>>
>>> or
>>>
>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>
>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>
>>>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>>>> does corresponding changes in code (look for details in
>>>> include/qapi/error.h)
>>>>
>>>> Usage example:
>>>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>
>>> Suggest FILES... instead of a specific set of files.
>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>>
>>>> Cc: Eric Blake <eblake@redhat.com>
>>>> Cc: Kevin Wolf <kwolf@redhat.com>
>>>> Cc: Max Reitz <mreitz@redhat.com>
>>>> Cc: Greg Kurz <groug@kaod.org>
>>>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>>>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>>>> Cc: Anthony Perard <anthony.perard@citrix.com>
>>>> Cc: Paul Durrant <paul@xen.org>
>>>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>>>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>>>> Cc: Markus Armbruster <armbru@redhat.com>
>>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>>> Cc: qemu-block@nongnu.org
>>>> Cc: qemu-devel@nongnu.org
>>>> Cc: xen-devel@lists.xenproject.org
>>>>
>>>> include/qapi/error.h | 3 +
>>>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>>>> 2 files changed, 234 insertions(+)
>>>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>>
>>>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>>>> index bb9bcf02fb..fbfc6f1c0b 100644
>>>> --- a/include/qapi/error.h
>>>> +++ b/include/qapi/error.h
>>>> @@ -211,6 +211,9 @@
>>>> * }
>>>> * ...
>>>> * }
>>>> + *
>>>> + * For mass conversion use script
>>>
>>> mass-conversion (we're not converting mass, we're converting en masse)
>>>
>>>> + * scripts/coccinelle/auto-propagated-errp.cocci
>>>> */
>>>> #ifndef ERROR_H
>>>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>>>> new file mode 100644
>>>> index 0000000000..bff274bd6d
>>>> --- /dev/null
>>>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>>>
>>> Preface to my review of this script: may aim isn't to make it
>>> bullet-proof. I want to (1) make it good enough (explained in a
>>> jiffie), and (2) automatically identify the spots where it still isn't
>>> obviously safe for manual review.
>>>
>>> The latter may involve additional scripting. That's okay.
>>>
>>> The script is good enough when the number of possibly unsafe spots is
>>> low enough for careful manual review.
>>>
>>> When I ask for improvements that, in your opinion, go beyond "good
>>> enough", please push back. I'm sure we can work it out together.
>>>
>>>> @@ -0,0 +1,231 @@
>>>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>>>> +//
>>>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>>>> +//
>>>> +// This program is free software; you can redistribute it and/or modify
>>>> +// it under the terms of the GNU General Public License as published by
>>>> +// the Free Software Foundation; either version 2 of the License, or
>>>> +// (at your option) any later version.
>>>> +//
>>>> +// This program is distributed in the hope that it will be useful,
>>>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>> +// GNU General Public License for more details.
>>>> +//
>>>> +// You should have received a copy of the GNU General Public License
>>>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>> +//
>>>> +// Usage example:
>>>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>>>
>>> You have --max-width 80 here, but not in the commit message. Default
>>> seems to be 78. Any particular reason to change it to 80?
>>
>> Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
>> So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
>> Still, may be I'm wrong, we can check it.
>
> If you can point to an example where --max-width helps, keep it, and
> update the commit message to match. Else, drop it.
>
>>>
>>>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>> +
>>>> +// Switch unusual (Error **) parameter names to errp
>>>
>>> Let's drop the parenthesis around Error **
>>>
>>>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>>>
>>> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
>>> make the fact we're messing with @errp more obvious. Too late; I
>>> shouldn't rock the boat that much now.
>>>
>>>> +//
>>>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>>>> +// parameter.
>>>> +//
>>>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>>>> +// non generic semantics and may have unusual Error ** argument name for purpose
>>>
>>> non-generic
>>>
>>> for a purpose
>>>
>>> Wrap comment lines around column 70, please. It's easier to read.
>>>
>>> Maybe
>>>
>>> // Skip functions with "assert(_errp && *_errp)" statement, because that
>>> // signals unusual semantics, and the parameter name may well serve a
>>> // purpose.
>>
>> Sounds good.
>>
>>>
>>>> +// (like nbd_iter_channel_error()).
>>>> +//
>>>> +// Skip util/error.c to not touch, for example, error_propagate and
>>>> +// error_propagate_prepend().
>>>
>>> error_propagate()
>>>
>>> I much appreciate your meticulous explanation of what you skip and why.
>>>
>>>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>>>> +identifier fn;
>>>> +identifier _errp != errp;
>>>> +@@
>>>> +
>>>> + fn(...,
>>>> +- Error **_errp
>>>> ++ Error **errp
>>>> + ,...)
>>>> + {
>>>> +(
>>>> + ... when != assert(_errp && *_errp)
>>>> +&
>>>> + <...
>>>> +- _errp
>>>> ++ errp
>>>> + ...>
>>>> +)
>>>> + }
>>>
>>> This rule is required to make the actual transformations (below) work
>>> even for parameters with names other than @errp. I believe it's not
>>> used in this series. In fact, I can't see a use for it in the entire
>>> tree right now. Okay anyway.
>>>
>>>> +
>>>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>>>> +//
>>>> +// Note, that without "when any" final "..." may not want to mach something
>>>
>>> s/final "..." may not mach/the final "..." does not match/
>>>
>>>> +// matched by previous pattern, i.e. the rule will not match double
>>>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>>>
>>> Can't say I fully understand Coccinelle there. I figure you came to
>>> this knowledge the hard way.
>>
>> It's follows from smpl grammar document:
>>
>> "Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
>> ...
>> "_when any_ removes the aforementioned constraint that “...” matches the shortest path"
>
> Let me think that through.
>
> The pattern with the cases other than error_prepend() omitted:
>
> fn(..., Error **errp, ...)
> {
> + ERRP_AUTO_PROPAGATE();
> ... when != ERRP_AUTO_PROPAGATE();
> error_prepend(errp, ...);
> ... when any
> }
>
> Tail of vfio_set_irq_signaling():
>
> name = index_to_str(vbasedev, index);
> if (name) {
> error_prepend(errp, "%s-%d: ", name, subindex);
> } else {
> error_prepend(errp, "index %d-%d: ", index, subindex);
> }
> error_prepend(errp,
> "Failed to %s %s eventfd signaling for interrupt ",
> fd < 0 ? "tear down" : "set up", action_to_str(action));
> return ret;
> }
>
> The pattern's first ... matches a "shortest" path to an error_prepend(),
> where "shortest" means "does not cross an error_prepend(). Its when
> clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().
>
> There are two such "shortest" paths, one to the first error_prepend() in
> vfio_set_irq_signaling(), and one to the second. Neither path to the
> third one is not "shortest": they both cross one of the other two
> error_prepend().
>
> The pattern' s second ... matches a path from a matched error_prepend()
> to the end of the function. There are two paths. Both cross the third
> error_prepend(). You need "when any" to make the pattern match anyway.
>
> Alright, I think I got it. But now I'm paranoid about ... elsewhere.
> For instance, here's rule1 with error_propagate_prepend() omitted:
>
> // Match scenarios with propagation of local error to errp.
> @rule1 disable optional_qualifier exists@
> identifier fn, local_err;
> symbol errp;
> @@
>
> fn(..., Error **errp, ...)
> {
> ...
> Error *local_err = NULL;
> ...
> error_propagate(errp, local_err);
> ...
> }
>
> The second and third ... won't match anything containing
> error_propagate(). What if a function has multiple error_propagate() on
> all paths?
I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...
> Like this one:
>
> extern foo(int, Error **);
> extern bar(int, Error **);
>
> void frob(Error **errp)
> {
> Error *local_err = NULL;
> int arg;
>
> foo(arg, errp);
> bar(arg, &local_err);
> error_propagate(errp, local_err);
> bar(arg + 1, &local_err);
> error_propagate(errp, local_err);
> }
>
> This is actually a variation of error.h's "Receive and accumulate
> multiple errors (first one wins)" code snippet.
ah yes, we can propagate to already filled errp, which just clean local_err.
>
> The Coccinelle script transforms it like this:
>
> void frob(Error **errp)
> {
> + ERRP_AUTO_PROPAGATE();
> Error *local_err = NULL;
> int arg;
>
> The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any),
> but rule1 does not, and we therefore don't convert any of the
> error_propagate().
>
> The result isn't wrong, just useless.
>
> Is this the worst case?
>
> Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use
> "... when any" in the error_propagate() case, only in the other cases.
> Would that help?
I think not, as it will anyway match functions with error_prepend (and any
number of following error_propagate calls)...
>
> I think this is the only other rule with "..." matching control flow.
>
>>>
>>>> +//
>>>> +// Note, "exists" says that we want apply rule even if it matches not on
>>>> +// all possible control flows (otherwise, it will not match standard pattern
>>>> +// when error_propagate() call is in if branch).
>>>
>>> Learned something new. Example: kvm_set_kvm_shadow_mem().
>>>
>>> Spelling it "exists disable optional_qualifier" would avoid giving
>>> readers the idea we're disabling "exists", but Coccinelle doesn't let
>>> us. Oh well.
>>>
>>>> +@ disable optional_qualifier exists@
>>>> +identifier fn, local_err, errp;
>>>
>>> I believe this causes
>>>
>>> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>>>
>>> Making @errp symbol instead of identifier should fix this.
>>
>> Hmm, I didn't see these warnings.. But yes, it should be symbol.
>>
>>>
>>>> +@@
>>>> +
>>>> + fn(..., Error **errp, ...)
>>>> + {
>>>> ++ ERRP_AUTO_PROPAGATE();
>>>> + ... when != ERRP_AUTO_PROPAGATE();
>>>> +(
>>>> + error_append_hint(errp, ...);
>>>> +|
>>>> + error_prepend(errp, ...);
>>>> +|
>>>> + error_vprepend(errp, ...);
>>>> +|
>>>> + Error *local_err = NULL;
>>>> + ...
>>>> +(
>>>> + error_propagate_prepend(errp, local_err, ...);
>>>> +|
>>>> + error_propagate(errp, local_err);
>>>> +)
>>>> +)
>>>> + ... when any
>>>> + }
>>>> +
>>>> +
>>>> +// Match scenarios with propagation of local error to errp.
>>>> +@rule1 disable optional_qualifier exists@
>>>> +identifier fn, local_err;
>>>> +symbol errp;
>>>> +@@
>>>> +
>>>> + fn(..., Error **errp, ...)
>>>> + {
>>>> + ...
>>>> + Error *local_err = NULL;
>>>> + ...
>>>> +(
>>>> + error_propagate_prepend(errp, local_err, ...);
>>>> +|
>>>> + error_propagate(errp, local_err);
>>>> +)
>>>
>>> Indentation off by one.
>>>
>>>> + ...
>>>> + }
>>>> +
>>>> +// Convert special case with goto in separate.
>>>
>>> s/in separate/separately/
>>>
>>>> +// We can probably merge this into the following hunk with help of ( | )
>>>> +// operator, but it significantly reduce performance on block.c parsing (or it
>>>
>>> s/reduce/reduces/
>>>
>>>> +// hangs, I don't know)
>>>
>>> Sounds like you tried to merge this into the following hunk, but then
>>> spatch took so long on block.c that you killed it. Correct?
>>
>> Yes.
>
> I'd say something like "I tried merging this into the following rule the
> obvious way, but it made Coccinelle hang on block.c."
>
>>>
>>>> +//
>>>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>>>> +// things later after all transformations (the rule will be the same, just
>>>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>>>
>>> Weird, but not worth further investigation.
>>
>> It partially match to the idea which I saw somewhere in coccinelle documentation,
>> that coccinelle converts correct C code to correct C code. "out: }" is an example
>> of incorrect, impossible code flow, and coccinelle can't work with it... But it's
>> just a thought.
>>
>>>
>>>> +@@
>>>> +identifier rule1.fn, rule1.local_err, out;
>>>> +symbol errp;
>>>> +@@
>>>> +
>>>> + fn(...)
>>>> + {
>>>> + <...
>>>> +- goto out;
>>>> ++ return;
>>>> + ...>
>>>> +- out:
>>>> +- error_propagate(errp, local_err);
>>>
>>> You neglect to match error_propagate_prepend(). Okay, because (1) that
>>> pattern doesn't occur in the tree right now, and (2) if it gets added,
>>> gcc will complain.
>>
>> No, because it should not removed. error_propagate_prepend should be converted
>> to prepend, not removed. So, corresponding gotos should not be removed as well.
>
> You're right.
>
>>>
>>>> + }
>>>> +
>>>> +// Convert most of local_err related staff.
>>>
>>> s/staff/stuff/
>>>
>>>> +//
>>>> +// Note, that we update everything related to matched by rule1 function name
>>>> +// and local_err name. We may match something not related to the pattern
>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>> +// in different blocks inside one function, and in one block follow the
>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>> +// functions with the same name (for different configurations).
>>>
>>> Context: rule1 matches functions that have all three of
>>>
>>> * an Error **errp parameter
>>>
>>> * an Error *local_err = NULL variable declaration
>>>
>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>> local_err, ...) expression, where @errp is the parameter and
>>> @local_err is the variable.
>>>
>>> If I understand you correctly, you're pointing out two potential issues:
>>>
>>> 1. This rule can match functions rule1 does not match if there is
>>> another function with the same name that rule1 does match.
>>>
>>> 2. This rule matches in the entire function matched by rule1, even when
>>> parts of that function use a different @errp or @local_err.
>>>
>>> I figure these apply to all rules with identifier rule1.fn, not just
>>> this one. Correct?
>>
>> Yes.
>
> Thanks!
>
>>>
>>> Regarding 1. There must be a better way to chain rules together, but I
>>> don't know it.
>>> Can we make Coccinelle at least warn us when it converts
>>> multiple functions with the same name? What about this:
>>>
>>> @initialize:python@
>>> @@
>>> fnprev = {}
>>>
>>> def pr(fn, p):
>>> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>>>
>>> @r@
>>> identifier rule1.fn;
>>> position p;
>>> @@
>>> fn(...)@p
>>> {
>>> ...
>>> }
>>> @script:python@
>>> fn << rule1.fn;
>>> p << r.p;
>>> @@
>>> if fn not in fnprev:
>>> fnprev[fn] = p
>>> else:
>>> if fnprev[fn]:
>>
>> hmm, the condition can't be false
>>
>>> pr(fn, fnprev[fn])
>>> fnprev[fn] = None
>>> pr(fn, p)
>>
>> and we'll miss next duplication..
>
> The idea is
>
> first instance of fn:
> fn not in fnprev
> fnprev[fn] = position of instance
> don't print
> second instance:
> fnprev[fn] is the position of the first instance
> print first two instances
> subsequent instances: fnprev[fn] is None
> print this instance
>
> I might have screwed up the coding, of course :)
>
>> But I like the idea.
>>
>>>
>>> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
>>> the number of functions with the same name we touch. If it's more than
>>> one, we print.
>>>
>>> Reports about a dozen function names for the whole tree in my testing.
>>> Inspecting the changes to them manually is feasible. None of them are
>>> in files touched by this series.
>>>
>>> The line printed for the first match is pretty useless for me: it points
>>> to a Coccinelle temporary file *shrug*.
>>>
>>> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
>>> sure hope we don't do that. Multiple @local_err variables... hmm.
>>> Perhaps we could again concoct some script rules to lead us to spots to
>>> check manually. See below for my attempt.
>>>
>>> What's the worst that could happen if we blindly converted such code?
>>> The answer to that question tells us how hard to work on finding and
>>> checking these guys.
>>>
>>>> +//
>>>> +// Note also that errp-cleaning functions
>>>> +// error_free_errp
>>>> +// error_report_errp
>>>> +// error_reportf_errp
>>>> +// warn_report_errp
>>>> +// warn_reportf_errp
>>>> +// are not yet implemented. They must call corresponding Error* - freeing
>>>> +// function and then set *errp to NULL, to avoid further propagation to
>>>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>>>> +// For example, error_free_errp may look like this:
>>>> +//
>>>> +// void error_free_errp(Error **errp)
>>>> +// {
>>>> +// error_free(*errp);
>>>> +// *errp = NULL;
>>>> +// }
>>>> +@ exists@
>>>> +identifier rule1.fn, rule1.local_err;
>>>> +expression list args;
>>>> +symbol errp;
>>>> +@@
>>>> +
>>>> + fn(...)
>>>> + {
>>>> + <...
>>>> +(
>>>
>>> Each of the following patterns applies anywhere in the function.
>>>
>>> First pattern: delete @local_err
>>>
>>>> +- Error *local_err = NULL;
>>>
>>> Common case: occurs just once, not nested. Anything else is suspicious.
>>>
>>> Both can be detected in the resulting patches with a bit of AWK
>>> wizardry:
>>>
>>> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
>>> nested
>>> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
>>> nested
>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>> nested
>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>> more than one
>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>
>>> Oh.
>>>
>>> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>>>
>>> xen_block_device_destroy() has multiple Error *local_err.
>>>
>>> In both cases, manual review is required to ensure the conversion is
>>> okay. I believe it is.
>>>
>>> Note that the AWK script relies on diff showing the function name in @@
>>> lines, which doesn't always work due to our coding style.
>>>
>>> For the whole tree, I get some 30 spots. Feasible.
>>>
>>>> +|
>>>
>>> Second pattern: clear @errp after freeing it
>>>
>>>> +
>>>> +// Convert error clearing functions
>>>
>>> Suggest: Ensure @local_err is cleared on free
>>>
>>>> +(
>>>> +- error_free(local_err);
>>>> ++ error_free_errp(errp);
>>>> +|
>>>> +- error_report_err(local_err);
>>>> ++ error_report_errp(errp);
>>>> +|
>>>> +- error_reportf_err(local_err, args);
>>>> ++ error_reportf_errp(errp, args);
>>>> +|
>>>> +- warn_report_err(local_err);
>>>> ++ warn_report_errp(errp);
>>>> +|
>>>> +- warn_reportf_err(local_err, args);
>>>> ++ warn_reportf_errp(errp, args);
>>>> +)
>>>
>>> As you mention above, these guys don't exist, yet. Builds anyway,
>>> because this part of the rule is not used in this patch series. You
>>> don't want to omit it, because then the script becomes unsafe to use.
>>>
>>> We could also open-code:
>>>
>>> // Convert error clearing functions
>>> (
>>> - error_free(local_err);
>>> + error_free(*errp);
>>> + *errp = NULL;
>>> |
>>> ... and so forth ...
>>> )
>>>
>>> Matter of taste. Whatever is easier to explain in the comments. Since
>>> you already wrote one...
>>
>> I just feel that using helper functions is safer way..
>>
>>>
>>> We talked about extending this series slightly so these guys are used.
>>> I may still look into that.
>>>
>>>> +?- local_err = NULL;
>>>> +
>>>
>>> The new helpers clear @local_err. Assignment now redundant, delete.
>>> Okay.
>>>
>>>> +|
>>>
>>> Third and fourth pattern: delete error_propagate()
>>>
>>>> +- error_propagate_prepend(errp, local_err, args);
>>>> ++ error_prepend(errp, args);
>>>> +|
>>>> +- error_propagate(errp, local_err);
>>>> +|
>>>
>>> Fifth pattern: use @errp directly
>>>
>>>> +- &local_err
>>>> ++ errp
>>>> +)
>>>> + ...>
>>>> + }
>>>> +
>>>> +// Convert remaining local_err usage. It should be different kinds of error
>>>> +// checking in if operators. We can't merge this into previous hunk, as this
>>>
>>> In if conditionals, I suppose. It's the case for this patch. If I
>>> apply the script to the whole tree, the rule gets also applied in other
>>> contexts. The sentence might mislead as much as it helps. Keep it or
>>> delete it?
>>
>> Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
>
> "Need to be careful" means "needs careful manual review", which I
> believe is not feasible; see "Preface to my review of this script"
> above.
>
> But do we really need to be careful here?
>
> This rule should apply only where we added ERRP_AUTO_PROPAGATE().
>
> Except when rule chaining via function name fails us, but we plan to
> detect that and review manually, so let's ignore this issue here.
>
> Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling
> replacement of @local_err by @errp is its whole point.
>
> What exactly do we need to be careful about?
Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."
>
>>
>>>
>>>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>>>> +@@
>>>> +identifier rule1.fn, rule1.local_err;
>>>> +symbol errp;
>>>> +@@
>>>> +
>>>> + fn(...)
>>>> + {
>>>> + <...
>>>> +- local_err
>>>> ++ *errp
>>>> + ...>
>>>> + }
>>>> +
>>>> +// Always use the same patter for checking error
>>>
>>> s/patter/pattern/
>>>
>>>> +@@
>>>> +identifier rule1.fn;
>>>> +symbol errp;
>>>> +@@
>>>> +
>>>> + fn(...)
>>>> + {
>>>> + <...
>>>> +- *errp != NULL
>>>> ++ *errp
>>>> + ...>
>>>> + }
>>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 6:55 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 6:55 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
10.03.2020 18:47, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 09.03.2020 12:56, Markus Armbruster wrote:
>>> Suggest
>>>
>>> scripts: Coccinelle script to use auto-propagated errp
>>>
>>> or
>>>
>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>
>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>
>>>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>>>> does corresponding changes in code (look for details in
>>>> include/qapi/error.h)
>>>>
>>>> Usage example:
>>>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>
>>> Suggest FILES... instead of a specific set of files.
>>>
>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>> ---
>>>>
>>>> Cc: Eric Blake <eblake@redhat.com>
>>>> Cc: Kevin Wolf <kwolf@redhat.com>
>>>> Cc: Max Reitz <mreitz@redhat.com>
>>>> Cc: Greg Kurz <groug@kaod.org>
>>>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>>>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>>>> Cc: Anthony Perard <anthony.perard@citrix.com>
>>>> Cc: Paul Durrant <paul@xen.org>
>>>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>>>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>>>> Cc: Markus Armbruster <armbru@redhat.com>
>>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>>> Cc: qemu-block@nongnu.org
>>>> Cc: qemu-devel@nongnu.org
>>>> Cc: xen-devel@lists.xenproject.org
>>>>
>>>> include/qapi/error.h | 3 +
>>>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>>>> 2 files changed, 234 insertions(+)
>>>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>>
>>>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>>>> index bb9bcf02fb..fbfc6f1c0b 100644
>>>> --- a/include/qapi/error.h
>>>> +++ b/include/qapi/error.h
>>>> @@ -211,6 +211,9 @@
>>>> * }
>>>> * ...
>>>> * }
>>>> + *
>>>> + * For mass conversion use script
>>>
>>> mass-conversion (we're not converting mass, we're converting en masse)
>>>
>>>> + * scripts/coccinelle/auto-propagated-errp.cocci
>>>> */
>>>> #ifndef ERROR_H
>>>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>>>> new file mode 100644
>>>> index 0000000000..bff274bd6d
>>>> --- /dev/null
>>>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>>>
>>> Preface to my review of this script: may aim isn't to make it
>>> bullet-proof. I want to (1) make it good enough (explained in a
>>> jiffie), and (2) automatically identify the spots where it still isn't
>>> obviously safe for manual review.
>>>
>>> The latter may involve additional scripting. That's okay.
>>>
>>> The script is good enough when the number of possibly unsafe spots is
>>> low enough for careful manual review.
>>>
>>> When I ask for improvements that, in your opinion, go beyond "good
>>> enough", please push back. I'm sure we can work it out together.
>>>
>>>> @@ -0,0 +1,231 @@
>>>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>>>> +//
>>>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>>>> +//
>>>> +// This program is free software; you can redistribute it and/or modify
>>>> +// it under the terms of the GNU General Public License as published by
>>>> +// the Free Software Foundation; either version 2 of the License, or
>>>> +// (at your option) any later version.
>>>> +//
>>>> +// This program is distributed in the hope that it will be useful,
>>>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>> +// GNU General Public License for more details.
>>>> +//
>>>> +// You should have received a copy of the GNU General Public License
>>>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>> +//
>>>> +// Usage example:
>>>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>>>
>>> You have --max-width 80 here, but not in the commit message. Default
>>> seems to be 78. Any particular reason to change it to 80?
>>
>> Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
>> So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
>> Still, may be I'm wrong, we can check it.
>
> If you can point to an example where --max-width helps, keep it, and
> update the commit message to match. Else, drop it.
>
>>>
>>>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>> +
>>>> +// Switch unusual (Error **) parameter names to errp
>>>
>>> Let's drop the parenthesis around Error **
>>>
>>>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>>>
>>> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
>>> make the fact we're messing with @errp more obvious. Too late; I
>>> shouldn't rock the boat that much now.
>>>
>>>> +//
>>>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>>>> +// parameter.
>>>> +//
>>>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>>>> +// non generic semantics and may have unusual Error ** argument name for purpose
>>>
>>> non-generic
>>>
>>> for a purpose
>>>
>>> Wrap comment lines around column 70, please. It's easier to read.
>>>
>>> Maybe
>>>
>>> // Skip functions with "assert(_errp && *_errp)" statement, because that
>>> // signals unusual semantics, and the parameter name may well serve a
>>> // purpose.
>>
>> Sounds good.
>>
>>>
>>>> +// (like nbd_iter_channel_error()).
>>>> +//
>>>> +// Skip util/error.c to not touch, for example, error_propagate and
>>>> +// error_propagate_prepend().
>>>
>>> error_propagate()
>>>
>>> I much appreciate your meticulous explanation of what you skip and why.
>>>
>>>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>>>> +identifier fn;
>>>> +identifier _errp != errp;
>>>> +@@
>>>> +
>>>> + fn(...,
>>>> +- Error **_errp
>>>> ++ Error **errp
>>>> + ,...)
>>>> + {
>>>> +(
>>>> + ... when != assert(_errp && *_errp)
>>>> +&
>>>> + <...
>>>> +- _errp
>>>> ++ errp
>>>> + ...>
>>>> +)
>>>> + }
>>>
>>> This rule is required to make the actual transformations (below) work
>>> even for parameters with names other than @errp. I believe it's not
>>> used in this series. In fact, I can't see a use for it in the entire
>>> tree right now. Okay anyway.
>>>
>>>> +
>>>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>>>> +//
>>>> +// Note, that without "when any" final "..." may not want to mach something
>>>
>>> s/final "..." may not mach/the final "..." does not match/
>>>
>>>> +// matched by previous pattern, i.e. the rule will not match double
>>>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>>>
>>> Can't say I fully understand Coccinelle there. I figure you came to
>>> this knowledge the hard way.
>>
>> It's follows from smpl grammar document:
>>
>> "Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
>> ...
>> "_when any_ removes the aforementioned constraint that “...” matches the shortest path"
>
> Let me think that through.
>
> The pattern with the cases other than error_prepend() omitted:
>
> fn(..., Error **errp, ...)
> {
> + ERRP_AUTO_PROPAGATE();
> ... when != ERRP_AUTO_PROPAGATE();
> error_prepend(errp, ...);
> ... when any
> }
>
> Tail of vfio_set_irq_signaling():
>
> name = index_to_str(vbasedev, index);
> if (name) {
> error_prepend(errp, "%s-%d: ", name, subindex);
> } else {
> error_prepend(errp, "index %d-%d: ", index, subindex);
> }
> error_prepend(errp,
> "Failed to %s %s eventfd signaling for interrupt ",
> fd < 0 ? "tear down" : "set up", action_to_str(action));
> return ret;
> }
>
> The pattern's first ... matches a "shortest" path to an error_prepend(),
> where "shortest" means "does not cross an error_prepend(). Its when
> clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().
>
> There are two such "shortest" paths, one to the first error_prepend() in
> vfio_set_irq_signaling(), and one to the second. Neither path to the
> third one is not "shortest": they both cross one of the other two
> error_prepend().
>
> The pattern' s second ... matches a path from a matched error_prepend()
> to the end of the function. There are two paths. Both cross the third
> error_prepend(). You need "when any" to make the pattern match anyway.
>
> Alright, I think I got it. But now I'm paranoid about ... elsewhere.
> For instance, here's rule1 with error_propagate_prepend() omitted:
>
> // Match scenarios with propagation of local error to errp.
> @rule1 disable optional_qualifier exists@
> identifier fn, local_err;
> symbol errp;
> @@
>
> fn(..., Error **errp, ...)
> {
> ...
> Error *local_err = NULL;
> ...
> error_propagate(errp, local_err);
> ...
> }
>
> The second and third ... won't match anything containing
> error_propagate(). What if a function has multiple error_propagate() on
> all paths?
I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...
> Like this one:
>
> extern foo(int, Error **);
> extern bar(int, Error **);
>
> void frob(Error **errp)
> {
> Error *local_err = NULL;
> int arg;
>
> foo(arg, errp);
> bar(arg, &local_err);
> error_propagate(errp, local_err);
> bar(arg + 1, &local_err);
> error_propagate(errp, local_err);
> }
>
> This is actually a variation of error.h's "Receive and accumulate
> multiple errors (first one wins)" code snippet.
ah yes, we can propagate to already filled errp, which just clean local_err.
>
> The Coccinelle script transforms it like this:
>
> void frob(Error **errp)
> {
> + ERRP_AUTO_PROPAGATE();
> Error *local_err = NULL;
> int arg;
>
> The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any),
> but rule1 does not, and we therefore don't convert any of the
> error_propagate().
>
> The result isn't wrong, just useless.
>
> Is this the worst case?
>
> Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use
> "... when any" in the error_propagate() case, only in the other cases.
> Would that help?
I think not, as it will anyway match functions with error_prepend (and any
number of following error_propagate calls)...
>
> I think this is the only other rule with "..." matching control flow.
>
>>>
>>>> +//
>>>> +// Note, "exists" says that we want apply rule even if it matches not on
>>>> +// all possible control flows (otherwise, it will not match standard pattern
>>>> +// when error_propagate() call is in if branch).
>>>
>>> Learned something new. Example: kvm_set_kvm_shadow_mem().
>>>
>>> Spelling it "exists disable optional_qualifier" would avoid giving
>>> readers the idea we're disabling "exists", but Coccinelle doesn't let
>>> us. Oh well.
>>>
>>>> +@ disable optional_qualifier exists@
>>>> +identifier fn, local_err, errp;
>>>
>>> I believe this causes
>>>
>>> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
>>> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>>>
>>> Making @errp symbol instead of identifier should fix this.
>>
>> Hmm, I didn't see these warnings.. But yes, it should be symbol.
>>
>>>
>>>> +@@
>>>> +
>>>> + fn(..., Error **errp, ...)
>>>> + {
>>>> ++ ERRP_AUTO_PROPAGATE();
>>>> + ... when != ERRP_AUTO_PROPAGATE();
>>>> +(
>>>> + error_append_hint(errp, ...);
>>>> +|
>>>> + error_prepend(errp, ...);
>>>> +|
>>>> + error_vprepend(errp, ...);
>>>> +|
>>>> + Error *local_err = NULL;
>>>> + ...
>>>> +(
>>>> + error_propagate_prepend(errp, local_err, ...);
>>>> +|
>>>> + error_propagate(errp, local_err);
>>>> +)
>>>> +)
>>>> + ... when any
>>>> + }
>>>> +
>>>> +
>>>> +// Match scenarios with propagation of local error to errp.
>>>> +@rule1 disable optional_qualifier exists@
>>>> +identifier fn, local_err;
>>>> +symbol errp;
>>>> +@@
>>>> +
>>>> + fn(..., Error **errp, ...)
>>>> + {
>>>> + ...
>>>> + Error *local_err = NULL;
>>>> + ...
>>>> +(
>>>> + error_propagate_prepend(errp, local_err, ...);
>>>> +|
>>>> + error_propagate(errp, local_err);
>>>> +)
>>>
>>> Indentation off by one.
>>>
>>>> + ...
>>>> + }
>>>> +
>>>> +// Convert special case with goto in separate.
>>>
>>> s/in separate/separately/
>>>
>>>> +// We can probably merge this into the following hunk with help of ( | )
>>>> +// operator, but it significantly reduce performance on block.c parsing (or it
>>>
>>> s/reduce/reduces/
>>>
>>>> +// hangs, I don't know)
>>>
>>> Sounds like you tried to merge this into the following hunk, but then
>>> spatch took so long on block.c that you killed it. Correct?
>>
>> Yes.
>
> I'd say something like "I tried merging this into the following rule the
> obvious way, but it made Coccinelle hang on block.c."
>
>>>
>>>> +//
>>>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>>>> +// things later after all transformations (the rule will be the same, just
>>>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>>>
>>> Weird, but not worth further investigation.
>>
>> It partially match to the idea which I saw somewhere in coccinelle documentation,
>> that coccinelle converts correct C code to correct C code. "out: }" is an example
>> of incorrect, impossible code flow, and coccinelle can't work with it... But it's
>> just a thought.
>>
>>>
>>>> +@@
>>>> +identifier rule1.fn, rule1.local_err, out;
>>>> +symbol errp;
>>>> +@@
>>>> +
>>>> + fn(...)
>>>> + {
>>>> + <...
>>>> +- goto out;
>>>> ++ return;
>>>> + ...>
>>>> +- out:
>>>> +- error_propagate(errp, local_err);
>>>
>>> You neglect to match error_propagate_prepend(). Okay, because (1) that
>>> pattern doesn't occur in the tree right now, and (2) if it gets added,
>>> gcc will complain.
>>
>> No, because it should not removed. error_propagate_prepend should be converted
>> to prepend, not removed. So, corresponding gotos should not be removed as well.
>
> You're right.
>
>>>
>>>> + }
>>>> +
>>>> +// Convert most of local_err related staff.
>>>
>>> s/staff/stuff/
>>>
>>>> +//
>>>> +// Note, that we update everything related to matched by rule1 function name
>>>> +// and local_err name. We may match something not related to the pattern
>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>> +// in different blocks inside one function, and in one block follow the
>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>> +// functions with the same name (for different configurations).
>>>
>>> Context: rule1 matches functions that have all three of
>>>
>>> * an Error **errp parameter
>>>
>>> * an Error *local_err = NULL variable declaration
>>>
>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>> local_err, ...) expression, where @errp is the parameter and
>>> @local_err is the variable.
>>>
>>> If I understand you correctly, you're pointing out two potential issues:
>>>
>>> 1. This rule can match functions rule1 does not match if there is
>>> another function with the same name that rule1 does match.
>>>
>>> 2. This rule matches in the entire function matched by rule1, even when
>>> parts of that function use a different @errp or @local_err.
>>>
>>> I figure these apply to all rules with identifier rule1.fn, not just
>>> this one. Correct?
>>
>> Yes.
>
> Thanks!
>
>>>
>>> Regarding 1. There must be a better way to chain rules together, but I
>>> don't know it.
>>> Can we make Coccinelle at least warn us when it converts
>>> multiple functions with the same name? What about this:
>>>
>>> @initialize:python@
>>> @@
>>> fnprev = {}
>>>
>>> def pr(fn, p):
>>> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>>>
>>> @r@
>>> identifier rule1.fn;
>>> position p;
>>> @@
>>> fn(...)@p
>>> {
>>> ...
>>> }
>>> @script:python@
>>> fn << rule1.fn;
>>> p << r.p;
>>> @@
>>> if fn not in fnprev:
>>> fnprev[fn] = p
>>> else:
>>> if fnprev[fn]:
>>
>> hmm, the condition can't be false
>>
>>> pr(fn, fnprev[fn])
>>> fnprev[fn] = None
>>> pr(fn, p)
>>
>> and we'll miss next duplication..
>
> The idea is
>
> first instance of fn:
> fn not in fnprev
> fnprev[fn] = position of instance
> don't print
> second instance:
> fnprev[fn] is the position of the first instance
> print first two instances
> subsequent instances: fnprev[fn] is None
> print this instance
>
> I might have screwed up the coding, of course :)
>
>> But I like the idea.
>>
>>>
>>> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
>>> the number of functions with the same name we touch. If it's more than
>>> one, we print.
>>>
>>> Reports about a dozen function names for the whole tree in my testing.
>>> Inspecting the changes to them manually is feasible. None of them are
>>> in files touched by this series.
>>>
>>> The line printed for the first match is pretty useless for me: it points
>>> to a Coccinelle temporary file *shrug*.
>>>
>>> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
>>> sure hope we don't do that. Multiple @local_err variables... hmm.
>>> Perhaps we could again concoct some script rules to lead us to spots to
>>> check manually. See below for my attempt.
>>>
>>> What's the worst that could happen if we blindly converted such code?
>>> The answer to that question tells us how hard to work on finding and
>>> checking these guys.
>>>
>>>> +//
>>>> +// Note also that errp-cleaning functions
>>>> +// error_free_errp
>>>> +// error_report_errp
>>>> +// error_reportf_errp
>>>> +// warn_report_errp
>>>> +// warn_reportf_errp
>>>> +// are not yet implemented. They must call corresponding Error* - freeing
>>>> +// function and then set *errp to NULL, to avoid further propagation to
>>>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>>>> +// For example, error_free_errp may look like this:
>>>> +//
>>>> +// void error_free_errp(Error **errp)
>>>> +// {
>>>> +// error_free(*errp);
>>>> +// *errp = NULL;
>>>> +// }
>>>> +@ exists@
>>>> +identifier rule1.fn, rule1.local_err;
>>>> +expression list args;
>>>> +symbol errp;
>>>> +@@
>>>> +
>>>> + fn(...)
>>>> + {
>>>> + <...
>>>> +(
>>>
>>> Each of the following patterns applies anywhere in the function.
>>>
>>> First pattern: delete @local_err
>>>
>>>> +- Error *local_err = NULL;
>>>
>>> Common case: occurs just once, not nested. Anything else is suspicious.
>>>
>>> Both can be detected in the resulting patches with a bit of AWK
>>> wizardry:
>>>
>>> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
>>> nested
>>> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
>>> nested
>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>> nested
>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>> more than one
>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>
>>> Oh.
>>>
>>> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>>>
>>> xen_block_device_destroy() has multiple Error *local_err.
>>>
>>> In both cases, manual review is required to ensure the conversion is
>>> okay. I believe it is.
>>>
>>> Note that the AWK script relies on diff showing the function name in @@
>>> lines, which doesn't always work due to our coding style.
>>>
>>> For the whole tree, I get some 30 spots. Feasible.
>>>
>>>> +|
>>>
>>> Second pattern: clear @errp after freeing it
>>>
>>>> +
>>>> +// Convert error clearing functions
>>>
>>> Suggest: Ensure @local_err is cleared on free
>>>
>>>> +(
>>>> +- error_free(local_err);
>>>> ++ error_free_errp(errp);
>>>> +|
>>>> +- error_report_err(local_err);
>>>> ++ error_report_errp(errp);
>>>> +|
>>>> +- error_reportf_err(local_err, args);
>>>> ++ error_reportf_errp(errp, args);
>>>> +|
>>>> +- warn_report_err(local_err);
>>>> ++ warn_report_errp(errp);
>>>> +|
>>>> +- warn_reportf_err(local_err, args);
>>>> ++ warn_reportf_errp(errp, args);
>>>> +)
>>>
>>> As you mention above, these guys don't exist, yet. Builds anyway,
>>> because this part of the rule is not used in this patch series. You
>>> don't want to omit it, because then the script becomes unsafe to use.
>>>
>>> We could also open-code:
>>>
>>> // Convert error clearing functions
>>> (
>>> - error_free(local_err);
>>> + error_free(*errp);
>>> + *errp = NULL;
>>> |
>>> ... and so forth ...
>>> )
>>>
>>> Matter of taste. Whatever is easier to explain in the comments. Since
>>> you already wrote one...
>>
>> I just feel that using helper functions is safer way..
>>
>>>
>>> We talked about extending this series slightly so these guys are used.
>>> I may still look into that.
>>>
>>>> +?- local_err = NULL;
>>>> +
>>>
>>> The new helpers clear @local_err. Assignment now redundant, delete.
>>> Okay.
>>>
>>>> +|
>>>
>>> Third and fourth pattern: delete error_propagate()
>>>
>>>> +- error_propagate_prepend(errp, local_err, args);
>>>> ++ error_prepend(errp, args);
>>>> +|
>>>> +- error_propagate(errp, local_err);
>>>> +|
>>>
>>> Fifth pattern: use @errp directly
>>>
>>>> +- &local_err
>>>> ++ errp
>>>> +)
>>>> + ...>
>>>> + }
>>>> +
>>>> +// Convert remaining local_err usage. It should be different kinds of error
>>>> +// checking in if operators. We can't merge this into previous hunk, as this
>>>
>>> In if conditionals, I suppose. It's the case for this patch. If I
>>> apply the script to the whole tree, the rule gets also applied in other
>>> contexts. The sentence might mislead as much as it helps. Keep it or
>>> delete it?
>>
>> Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
>
> "Need to be careful" means "needs careful manual review", which I
> believe is not feasible; see "Preface to my review of this script"
> above.
>
> But do we really need to be careful here?
>
> This rule should apply only where we added ERRP_AUTO_PROPAGATE().
>
> Except when rule chaining via function name fails us, but we plan to
> detect that and review manually, so let's ignore this issue here.
>
> Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling
> replacement of @local_err by @errp is its whole point.
>
> What exactly do we need to be careful about?
Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."
>
>>
>>>
>>>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>>>> +@@
>>>> +identifier rule1.fn, rule1.local_err;
>>>> +symbol errp;
>>>> +@@
>>>> +
>>>> + fn(...)
>>>> + {
>>>> + <...
>>>> +- local_err
>>>> ++ *errp
>>>> + ...>
>>>> + }
>>>> +
>>>> +// Always use the same patter for checking error
>>>
>>> s/patter/pattern/
>>>
>>>> +@@
>>>> +identifier rule1.fn;
>>>> +symbol errp;
>>>> +@@
>>>> +
>>>> + fn(...)
>>>> + {
>>>> + <...
>>>> +- *errp != NULL
>>>> ++ *errp
>>>> + ...>
>>>> + }
>>>
>
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-11 6:55 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-11 8:32 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 8:32 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
11.03.2020 9:55, Vladimir Sementsov-Ogievskiy wrote:
> 10.03.2020 18:47, Markus Armbruster wrote:
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>
>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>> Suggest
>>>>
>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>
>>>> or
>>>>
>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>
>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>
>>>>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>>>>> does corresponding changes in code (look for details in
>>>>> include/qapi/error.h)
>>>>>
>>>>> Usage example:
>>>>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>>
>>>> Suggest FILES... instead of a specific set of files.
>>>>
>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>> ---
>>>>>
>>>>> Cc: Eric Blake <eblake@redhat.com>
>>>>> Cc: Kevin Wolf <kwolf@redhat.com>
>>>>> Cc: Max Reitz <mreitz@redhat.com>
>>>>> Cc: Greg Kurz <groug@kaod.org>
>>>>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>>>>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>>>>> Cc: Anthony Perard <anthony.perard@citrix.com>
>>>>> Cc: Paul Durrant <paul@xen.org>
>>>>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>>>>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>>>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>>>>> Cc: Markus Armbruster <armbru@redhat.com>
>>>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>>>> Cc: qemu-block@nongnu.org
>>>>> Cc: qemu-devel@nongnu.org
>>>>> Cc: xen-devel@lists.xenproject.org
>>>>>
>>>>> include/qapi/error.h | 3 +
>>>>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>>>>> 2 files changed, 234 insertions(+)
>>>>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>>>
>>>>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>>>>> index bb9bcf02fb..fbfc6f1c0b 100644
>>>>> --- a/include/qapi/error.h
>>>>> +++ b/include/qapi/error.h
>>>>> @@ -211,6 +211,9 @@
>>>>> * }
>>>>> * ...
>>>>> * }
>>>>> + *
>>>>> + * For mass conversion use script
>>>>
>>>> mass-conversion (we're not converting mass, we're converting en masse)
>>>>
>>>>> + * scripts/coccinelle/auto-propagated-errp.cocci
>>>>> */
>>>>> #ifndef ERROR_H
>>>>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>> new file mode 100644
>>>>> index 0000000000..bff274bd6d
>>>>> --- /dev/null
>>>>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>
>>>> Preface to my review of this script: may aim isn't to make it
>>>> bullet-proof. I want to (1) make it good enough (explained in a
>>>> jiffie), and (2) automatically identify the spots where it still isn't
>>>> obviously safe for manual review.
>>>>
>>>> The latter may involve additional scripting. That's okay.
>>>>
>>>> The script is good enough when the number of possibly unsafe spots is
>>>> low enough for careful manual review.
>>>>
>>>> When I ask for improvements that, in your opinion, go beyond "good
>>>> enough", please push back. I'm sure we can work it out together.
>>>>
>>>>> @@ -0,0 +1,231 @@
>>>>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>>>>> +//
>>>>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>>>>> +//
>>>>> +// This program is free software; you can redistribute it and/or modify
>>>>> +// it under the terms of the GNU General Public License as published by
>>>>> +// the Free Software Foundation; either version 2 of the License, or
>>>>> +// (at your option) any later version.
>>>>> +//
>>>>> +// This program is distributed in the hope that it will be useful,
>>>>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>>> +// GNU General Public License for more details.
>>>>> +//
>>>>> +// You should have received a copy of the GNU General Public License
>>>>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>>> +//
>>>>> +// Usage example:
>>>>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>>>>
>>>> You have --max-width 80 here, but not in the commit message. Default
>>>> seems to be 78. Any particular reason to change it to 80?
>>>
>>> Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
>>> So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
>>> Still, may be I'm wrong, we can check it.
>>
>> If you can point to an example where --max-width helps, keep it, and
>> update the commit message to match. Else, drop it.
>>
>>>>
>>>>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>>> +
>>>>> +// Switch unusual (Error **) parameter names to errp
>>>>
>>>> Let's drop the parenthesis around Error **
>>>>
>>>>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>>>>
>>>> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
>>>> make the fact we're messing with @errp more obvious. Too late; I
>>>> shouldn't rock the boat that much now.
>>>>
>>>>> +//
>>>>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>>>>> +// parameter.
>>>>> +//
>>>>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>>>>> +// non generic semantics and may have unusual Error ** argument name for purpose
>>>>
>>>> non-generic
>>>>
>>>> for a purpose
>>>>
>>>> Wrap comment lines around column 70, please. It's easier to read.
>>>>
>>>> Maybe
>>>>
>>>> // Skip functions with "assert(_errp && *_errp)" statement, because that
>>>> // signals unusual semantics, and the parameter name may well serve a
>>>> // purpose.
>>>
>>> Sounds good.
>>>
>>>>
>>>>> +// (like nbd_iter_channel_error()).
>>>>> +//
>>>>> +// Skip util/error.c to not touch, for example, error_propagate and
>>>>> +// error_propagate_prepend().
>>>>
>>>> error_propagate()
>>>>
>>>> I much appreciate your meticulous explanation of what you skip and why.
>>>>
>>>>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>>>>> +identifier fn;
>>>>> +identifier _errp != errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...,
>>>>> +- Error **_errp
>>>>> ++ Error **errp
>>>>> + ,...)
>>>>> + {
>>>>> +(
>>>>> + ... when != assert(_errp && *_errp)
>>>>> +&
>>>>> + <...
>>>>> +- _errp
>>>>> ++ errp
>>>>> + ...>
>>>>> +)
>>>>> + }
>>>>
>>>> This rule is required to make the actual transformations (below) work
>>>> even for parameters with names other than @errp. I believe it's not
>>>> used in this series. In fact, I can't see a use for it in the entire
>>>> tree right now. Okay anyway.
>>>>
>>>>> +
>>>>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>>>>> +//
>>>>> +// Note, that without "when any" final "..." may not want to mach something
>>>>
>>>> s/final "..." may not mach/the final "..." does not match/
>>>>
>>>>> +// matched by previous pattern, i.e. the rule will not match double
>>>>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>>>>
>>>> Can't say I fully understand Coccinelle there. I figure you came to
>>>> this knowledge the hard way.
>>>
>>> It's follows from smpl grammar document:
>>>
>>> "Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
>>> ...
>>> "_when any_ removes the aforementioned constraint that “...” matches the shortest path"
>>
>> Let me think that through.
>>
>> The pattern with the cases other than error_prepend() omitted:
>>
>> fn(..., Error **errp, ...)
>> {
>> + ERRP_AUTO_PROPAGATE();
>> ... when != ERRP_AUTO_PROPAGATE();
>> error_prepend(errp, ...);
>> ... when any
>> }
>>
>> Tail of vfio_set_irq_signaling():
>>
>> name = index_to_str(vbasedev, index);
>> if (name) {
>> error_prepend(errp, "%s-%d: ", name, subindex);
>> } else {
>> error_prepend(errp, "index %d-%d: ", index, subindex);
>> }
>> error_prepend(errp,
>> "Failed to %s %s eventfd signaling for interrupt ",
>> fd < 0 ? "tear down" : "set up", action_to_str(action));
>> return ret;
>> }
>>
>> The pattern's first ... matches a "shortest" path to an error_prepend(),
>> where "shortest" means "does not cross an error_prepend(). Its when
>> clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().
>>
>> There are two such "shortest" paths, one to the first error_prepend() in
>> vfio_set_irq_signaling(), and one to the second. Neither path to the
>> third one is not "shortest": they both cross one of the other two
>> error_prepend().
>>
>> The pattern' s second ... matches a path from a matched error_prepend()
>> to the end of the function. There are two paths. Both cross the third
>> error_prepend(). You need "when any" to make the pattern match anyway.
>>
>> Alright, I think I got it. But now I'm paranoid about ... elsewhere.
>> For instance, here's rule1 with error_propagate_prepend() omitted:
>>
>> // Match scenarios with propagation of local error to errp.
>> @rule1 disable optional_qualifier exists@
>> identifier fn, local_err;
>> symbol errp;
>> @@
>>
>> fn(..., Error **errp, ...)
>> {
>> ...
>> Error *local_err = NULL;
>> ...
>> error_propagate(errp, local_err);
>> ...
>> }
>>
>> The second and third ... won't match anything containing
>> error_propagate(). What if a function has multiple error_propagate() on
>> all paths?
>
> I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...
>
>> Like this one:
>>
>> extern foo(int, Error **);
>> extern bar(int, Error **);
>>
>> void frob(Error **errp)
>> {
>> Error *local_err = NULL;
>> int arg;
>>
>> foo(arg, errp);
>> bar(arg, &local_err);
>> error_propagate(errp, local_err);
>> bar(arg + 1, &local_err);
>> error_propagate(errp, local_err);
>> }
>>
>> This is actually a variation of error.h's "Receive and accumulate
>> multiple errors (first one wins)" code snippet.
>
> ah yes, we can propagate to already filled errp, which just clean local_err.
>
>>
>> The Coccinelle script transforms it like this:
>>
>> void frob(Error **errp)
>> {
>> + ERRP_AUTO_PROPAGATE();
>> Error *local_err = NULL;
>> int arg;
>>
>> The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any),
>> but rule1 does not, and we therefore don't convert any of the
>> error_propagate().
>>
>> The result isn't wrong, just useless.
>>
>> Is this the worst case?
>>
>> Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use
>> "... when any" in the error_propagate() case, only in the other cases.
>> Would that help?
>
> I think not, as it will anyway match functions with error_prepend (and any
> number of following error_propagate calls)...
But it's correct to add ERRP_AUTO_PROPAGATE to such functions.. So, may be you
are right. Let's do it, seems better.
>
>>
>> I think this is the only other rule with "..." matching control flow.
>>
>>>>
>>>>> +//
>>>>> +// Note, "exists" says that we want apply rule even if it matches not on
>>>>> +// all possible control flows (otherwise, it will not match standard pattern
>>>>> +// when error_propagate() call is in if branch).
>>>>
>>>> Learned something new. Example: kvm_set_kvm_shadow_mem().
>>>>
>>>> Spelling it "exists disable optional_qualifier" would avoid giving
>>>> readers the idea we're disabling "exists", but Coccinelle doesn't let
>>>> us. Oh well.
>>>>
>>>>> +@ disable optional_qualifier exists@
>>>>> +identifier fn, local_err, errp;
>>>>
>>>> I believe this causes
>>>>
>>>> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>>>>
>>>> Making @errp symbol instead of identifier should fix this.
>>>
>>> Hmm, I didn't see these warnings.. But yes, it should be symbol.
>>>
>>>>
>>>>> +@@
>>>>> +
>>>>> + fn(..., Error **errp, ...)
>>>>> + {
>>>>> ++ ERRP_AUTO_PROPAGATE();
>>>>> + ... when != ERRP_AUTO_PROPAGATE();
>>>>> +(
>>>>> + error_append_hint(errp, ...);
>>>>> +|
>>>>> + error_prepend(errp, ...);
>>>>> +|
>>>>> + error_vprepend(errp, ...);
>>>>> +|
>>>>> + Error *local_err = NULL;
>>>>> + ...
>>>>> +(
>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>> +|
>>>>> + error_propagate(errp, local_err);
>>>>> +)
>>>>> +)
>>>>> + ... when any
>>>>> + }
>>>>> +
>>>>> +
>>>>> +// Match scenarios with propagation of local error to errp.
>>>>> +@rule1 disable optional_qualifier exists@
>>>>> +identifier fn, local_err;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(..., Error **errp, ...)
>>>>> + {
>>>>> + ...
>>>>> + Error *local_err = NULL;
>>>>> + ...
>>>>> +(
>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>> +|
>>>>> + error_propagate(errp, local_err);
>>>>> +)
>>>>
>>>> Indentation off by one.
>>>>
>>>>> + ...
>>>>> + }
>>>>> +
>>>>> +// Convert special case with goto in separate.
>>>>
>>>> s/in separate/separately/
>>>>
>>>>> +// We can probably merge this into the following hunk with help of ( | )
>>>>> +// operator, but it significantly reduce performance on block.c parsing (or it
>>>>
>>>> s/reduce/reduces/
>>>>
>>>>> +// hangs, I don't know)
>>>>
>>>> Sounds like you tried to merge this into the following hunk, but then
>>>> spatch took so long on block.c that you killed it. Correct?
>>>
>>> Yes.
>>
>> I'd say something like "I tried merging this into the following rule the
>> obvious way, but it made Coccinelle hang on block.c."
>>
>>>>
>>>>> +//
>>>>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>>>>> +// things later after all transformations (the rule will be the same, just
>>>>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>>>>
>>>> Weird, but not worth further investigation.
>>>
>>> It partially match to the idea which I saw somewhere in coccinelle documentation,
>>> that coccinelle converts correct C code to correct C code. "out: }" is an example
>>> of incorrect, impossible code flow, and coccinelle can't work with it... But it's
>>> just a thought.
>>>
>>>>
>>>>> +@@
>>>>> +identifier rule1.fn, rule1.local_err, out;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- goto out;
>>>>> ++ return;
>>>>> + ...>
>>>>> +- out:
>>>>> +- error_propagate(errp, local_err);
>>>>
>>>> You neglect to match error_propagate_prepend(). Okay, because (1) that
>>>> pattern doesn't occur in the tree right now, and (2) if it gets added,
>>>> gcc will complain.
>>>
>>> No, because it should not removed. error_propagate_prepend should be converted
>>> to prepend, not removed. So, corresponding gotos should not be removed as well.
>>
>> You're right.
>>
>>>>
>>>>> + }
>>>>> +
>>>>> +// Convert most of local_err related staff.
>>>>
>>>> s/staff/stuff/
>>>>
>>>>> +//
>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>> +// and local_err name. We may match something not related to the pattern
>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>> +// in different blocks inside one function, and in one block follow the
>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>> +// functions with the same name (for different configurations).
>>>>
>>>> Context: rule1 matches functions that have all three of
>>>>
>>>> * an Error **errp parameter
>>>>
>>>> * an Error *local_err = NULL variable declaration
>>>>
>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>> local_err, ...) expression, where @errp is the parameter and
>>>> @local_err is the variable.
>>>>
>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>
>>>> 1. This rule can match functions rule1 does not match if there is
>>>> another function with the same name that rule1 does match.
>>>>
>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>> parts of that function use a different @errp or @local_err.
>>>>
>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>> this one. Correct?
>>>
>>> Yes.
>>
>> Thanks!
>>
>>>>
>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>> don't know it.
>>>> Can we make Coccinelle at least warn us when it converts
>>>> multiple functions with the same name? What about this:
>>>>
>>>> @initialize:python@
>>>> @@
>>>> fnprev = {}
>>>>
>>>> def pr(fn, p):
>>>> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>>>>
>>>> @r@
>>>> identifier rule1.fn;
>>>> position p;
>>>> @@
>>>> fn(...)@p
>>>> {
>>>> ...
>>>> }
>>>> @script:python@
>>>> fn << rule1.fn;
>>>> p << r.p;
>>>> @@
>>>> if fn not in fnprev:
>>>> fnprev[fn] = p
>>>> else:
>>>> if fnprev[fn]:
>>>
>>> hmm, the condition can't be false
>>>
>>>> pr(fn, fnprev[fn])
>>>> fnprev[fn] = None
>>>> pr(fn, p)
>>>
>>> and we'll miss next duplication..
>>
>> The idea is
>>
>> first instance of fn:
>> fn not in fnprev
>> fnprev[fn] = position of instance
>> don't print
>> second instance:
>> fnprev[fn] is the position of the first instance
>> print first two instances
>> subsequent instances: fnprev[fn] is None
>> print this instance
>>
>> I might have screwed up the coding, of course :)
>>
>>> But I like the idea.
>>>
>>>>
>>>> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
>>>> the number of functions with the same name we touch. If it's more than
>>>> one, we print.
>>>>
>>>> Reports about a dozen function names for the whole tree in my testing.
>>>> Inspecting the changes to them manually is feasible. None of them are
>>>> in files touched by this series.
>>>>
>>>> The line printed for the first match is pretty useless for me: it points
>>>> to a Coccinelle temporary file *shrug*.
>>>>
>>>> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
>>>> sure hope we don't do that. Multiple @local_err variables... hmm.
>>>> Perhaps we could again concoct some script rules to lead us to spots to
>>>> check manually. See below for my attempt.
>>>>
>>>> What's the worst that could happen if we blindly converted such code?
>>>> The answer to that question tells us how hard to work on finding and
>>>> checking these guys.
>>>>
>>>>> +//
>>>>> +// Note also that errp-cleaning functions
>>>>> +// error_free_errp
>>>>> +// error_report_errp
>>>>> +// error_reportf_errp
>>>>> +// warn_report_errp
>>>>> +// warn_reportf_errp
>>>>> +// are not yet implemented. They must call corresponding Error* - freeing
>>>>> +// function and then set *errp to NULL, to avoid further propagation to
>>>>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>>>>> +// For example, error_free_errp may look like this:
>>>>> +//
>>>>> +// void error_free_errp(Error **errp)
>>>>> +// {
>>>>> +// error_free(*errp);
>>>>> +// *errp = NULL;
>>>>> +// }
>>>>> +@ exists@
>>>>> +identifier rule1.fn, rule1.local_err;
>>>>> +expression list args;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +(
>>>>
>>>> Each of the following patterns applies anywhere in the function.
>>>>
>>>> First pattern: delete @local_err
>>>>
>>>>> +- Error *local_err = NULL;
>>>>
>>>> Common case: occurs just once, not nested. Anything else is suspicious.
>>>>
>>>> Both can be detected in the resulting patches with a bit of AWK
>>>> wizardry:
>>>>
>>>> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
>>>> nested
>>>> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
>>>> nested
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>> nested
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>> more than one
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>>
>>>> Oh.
>>>>
>>>> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>>>>
>>>> xen_block_device_destroy() has multiple Error *local_err.
>>>>
>>>> In both cases, manual review is required to ensure the conversion is
>>>> okay. I believe it is.
>>>>
>>>> Note that the AWK script relies on diff showing the function name in @@
>>>> lines, which doesn't always work due to our coding style.
>>>>
>>>> For the whole tree, I get some 30 spots. Feasible.
>>>>
>>>>> +|
>>>>
>>>> Second pattern: clear @errp after freeing it
>>>>
>>>>> +
>>>>> +// Convert error clearing functions
>>>>
>>>> Suggest: Ensure @local_err is cleared on free
>>>>
>>>>> +(
>>>>> +- error_free(local_err);
>>>>> ++ error_free_errp(errp);
>>>>> +|
>>>>> +- error_report_err(local_err);
>>>>> ++ error_report_errp(errp);
>>>>> +|
>>>>> +- error_reportf_err(local_err, args);
>>>>> ++ error_reportf_errp(errp, args);
>>>>> +|
>>>>> +- warn_report_err(local_err);
>>>>> ++ warn_report_errp(errp);
>>>>> +|
>>>>> +- warn_reportf_err(local_err, args);
>>>>> ++ warn_reportf_errp(errp, args);
>>>>> +)
>>>>
>>>> As you mention above, these guys don't exist, yet. Builds anyway,
>>>> because this part of the rule is not used in this patch series. You
>>>> don't want to omit it, because then the script becomes unsafe to use.
>>>>
>>>> We could also open-code:
>>>>
>>>> // Convert error clearing functions
>>>> (
>>>> - error_free(local_err);
>>>> + error_free(*errp);
>>>> + *errp = NULL;
>>>> |
>>>> ... and so forth ...
>>>> )
>>>>
>>>> Matter of taste. Whatever is easier to explain in the comments. Since
>>>> you already wrote one...
>>>
>>> I just feel that using helper functions is safer way..
>>>
>>>>
>>>> We talked about extending this series slightly so these guys are used.
>>>> I may still look into that.
>>>>
>>>>> +?- local_err = NULL;
>>>>> +
>>>>
>>>> The new helpers clear @local_err. Assignment now redundant, delete.
>>>> Okay.
>>>>
>>>>> +|
>>>>
>>>> Third and fourth pattern: delete error_propagate()
>>>>
>>>>> +- error_propagate_prepend(errp, local_err, args);
>>>>> ++ error_prepend(errp, args);
>>>>> +|
>>>>> +- error_propagate(errp, local_err);
>>>>> +|
>>>>
>>>> Fifth pattern: use @errp directly
>>>>
>>>>> +- &local_err
>>>>> ++ errp
>>>>> +)
>>>>> + ...>
>>>>> + }
>>>>> +
>>>>> +// Convert remaining local_err usage. It should be different kinds of error
>>>>> +// checking in if operators. We can't merge this into previous hunk, as this
>>>>
>>>> In if conditionals, I suppose. It's the case for this patch. If I
>>>> apply the script to the whole tree, the rule gets also applied in other
>>>> contexts. The sentence might mislead as much as it helps. Keep it or
>>>> delete it?
>>>
>>> Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
>>
>> "Need to be careful" means "needs careful manual review", which I
>> believe is not feasible; see "Preface to my review of this script"
>> above.
>>
>> But do we really need to be careful here?
>>
>> This rule should apply only where we added ERRP_AUTO_PROPAGATE().
>>
>> Except when rule chaining via function name fails us, but we plan to
>> detect that and review manually, so let's ignore this issue here.
>>
>> Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling
>> replacement of @local_err by @errp is its whole point.
>>
>> What exactly do we need to be careful about?
>
> Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."
>
>>
>>>
>>>>
>>>>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>>>>> +@@
>>>>> +identifier rule1.fn, rule1.local_err;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- local_err
>>>>> ++ *errp
>>>>> + ...>
>>>>> + }
>>>>> +
>>>>> +// Always use the same patter for checking error
>>>>
>>>> s/patter/pattern/
>>>>
>>>>> +@@
>>>>> +identifier rule1.fn;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- *errp != NULL
>>>>> ++ *errp
>>>>> + ...>
>>>>> + }
>>>>
>>
>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 8:32 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 8:32 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
11.03.2020 9:55, Vladimir Sementsov-Ogievskiy wrote:
> 10.03.2020 18:47, Markus Armbruster wrote:
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>
>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>> Suggest
>>>>
>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>
>>>> or
>>>>
>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>
>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>
>>>>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>>>>> does corresponding changes in code (look for details in
>>>>> include/qapi/error.h)
>>>>>
>>>>> Usage example:
>>>>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>>
>>>> Suggest FILES... instead of a specific set of files.
>>>>
>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>> ---
>>>>>
>>>>> Cc: Eric Blake <eblake@redhat.com>
>>>>> Cc: Kevin Wolf <kwolf@redhat.com>
>>>>> Cc: Max Reitz <mreitz@redhat.com>
>>>>> Cc: Greg Kurz <groug@kaod.org>
>>>>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>>>>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>>>>> Cc: Anthony Perard <anthony.perard@citrix.com>
>>>>> Cc: Paul Durrant <paul@xen.org>
>>>>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>>>>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>>>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>>>>> Cc: Markus Armbruster <armbru@redhat.com>
>>>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>>>> Cc: qemu-block@nongnu.org
>>>>> Cc: qemu-devel@nongnu.org
>>>>> Cc: xen-devel@lists.xenproject.org
>>>>>
>>>>> include/qapi/error.h | 3 +
>>>>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>>>>> 2 files changed, 234 insertions(+)
>>>>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>>>
>>>>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>>>>> index bb9bcf02fb..fbfc6f1c0b 100644
>>>>> --- a/include/qapi/error.h
>>>>> +++ b/include/qapi/error.h
>>>>> @@ -211,6 +211,9 @@
>>>>> * }
>>>>> * ...
>>>>> * }
>>>>> + *
>>>>> + * For mass conversion use script
>>>>
>>>> mass-conversion (we're not converting mass, we're converting en masse)
>>>>
>>>>> + * scripts/coccinelle/auto-propagated-errp.cocci
>>>>> */
>>>>> #ifndef ERROR_H
>>>>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>> new file mode 100644
>>>>> index 0000000000..bff274bd6d
>>>>> --- /dev/null
>>>>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>
>>>> Preface to my review of this script: may aim isn't to make it
>>>> bullet-proof. I want to (1) make it good enough (explained in a
>>>> jiffie), and (2) automatically identify the spots where it still isn't
>>>> obviously safe for manual review.
>>>>
>>>> The latter may involve additional scripting. That's okay.
>>>>
>>>> The script is good enough when the number of possibly unsafe spots is
>>>> low enough for careful manual review.
>>>>
>>>> When I ask for improvements that, in your opinion, go beyond "good
>>>> enough", please push back. I'm sure we can work it out together.
>>>>
>>>>> @@ -0,0 +1,231 @@
>>>>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>>>>> +//
>>>>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>>>>> +//
>>>>> +// This program is free software; you can redistribute it and/or modify
>>>>> +// it under the terms of the GNU General Public License as published by
>>>>> +// the Free Software Foundation; either version 2 of the License, or
>>>>> +// (at your option) any later version.
>>>>> +//
>>>>> +// This program is distributed in the hope that it will be useful,
>>>>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>>> +// GNU General Public License for more details.
>>>>> +//
>>>>> +// You should have received a copy of the GNU General Public License
>>>>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>>> +//
>>>>> +// Usage example:
>>>>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>>>>
>>>> You have --max-width 80 here, but not in the commit message. Default
>>>> seems to be 78. Any particular reason to change it to 80?
>>>
>>> Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
>>> So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
>>> Still, may be I'm wrong, we can check it.
>>
>> If you can point to an example where --max-width helps, keep it, and
>> update the commit message to match. Else, drop it.
>>
>>>>
>>>>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>>> +
>>>>> +// Switch unusual (Error **) parameter names to errp
>>>>
>>>> Let's drop the parenthesis around Error **
>>>>
>>>>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>>>>
>>>> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
>>>> make the fact we're messing with @errp more obvious. Too late; I
>>>> shouldn't rock the boat that much now.
>>>>
>>>>> +//
>>>>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>>>>> +// parameter.
>>>>> +//
>>>>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>>>>> +// non generic semantics and may have unusual Error ** argument name for purpose
>>>>
>>>> non-generic
>>>>
>>>> for a purpose
>>>>
>>>> Wrap comment lines around column 70, please. It's easier to read.
>>>>
>>>> Maybe
>>>>
>>>> // Skip functions with "assert(_errp && *_errp)" statement, because that
>>>> // signals unusual semantics, and the parameter name may well serve a
>>>> // purpose.
>>>
>>> Sounds good.
>>>
>>>>
>>>>> +// (like nbd_iter_channel_error()).
>>>>> +//
>>>>> +// Skip util/error.c to not touch, for example, error_propagate and
>>>>> +// error_propagate_prepend().
>>>>
>>>> error_propagate()
>>>>
>>>> I much appreciate your meticulous explanation of what you skip and why.
>>>>
>>>>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>>>>> +identifier fn;
>>>>> +identifier _errp != errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...,
>>>>> +- Error **_errp
>>>>> ++ Error **errp
>>>>> + ,...)
>>>>> + {
>>>>> +(
>>>>> + ... when != assert(_errp && *_errp)
>>>>> +&
>>>>> + <...
>>>>> +- _errp
>>>>> ++ errp
>>>>> + ...>
>>>>> +)
>>>>> + }
>>>>
>>>> This rule is required to make the actual transformations (below) work
>>>> even for parameters with names other than @errp. I believe it's not
>>>> used in this series. In fact, I can't see a use for it in the entire
>>>> tree right now. Okay anyway.
>>>>
>>>>> +
>>>>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>>>>> +//
>>>>> +// Note, that without "when any" final "..." may not want to mach something
>>>>
>>>> s/final "..." may not mach/the final "..." does not match/
>>>>
>>>>> +// matched by previous pattern, i.e. the rule will not match double
>>>>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>>>>
>>>> Can't say I fully understand Coccinelle there. I figure you came to
>>>> this knowledge the hard way.
>>>
>>> It's follows from smpl grammar document:
>>>
>>> "Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
>>> ...
>>> "_when any_ removes the aforementioned constraint that “...” matches the shortest path"
>>
>> Let me think that through.
>>
>> The pattern with the cases other than error_prepend() omitted:
>>
>> fn(..., Error **errp, ...)
>> {
>> + ERRP_AUTO_PROPAGATE();
>> ... when != ERRP_AUTO_PROPAGATE();
>> error_prepend(errp, ...);
>> ... when any
>> }
>>
>> Tail of vfio_set_irq_signaling():
>>
>> name = index_to_str(vbasedev, index);
>> if (name) {
>> error_prepend(errp, "%s-%d: ", name, subindex);
>> } else {
>> error_prepend(errp, "index %d-%d: ", index, subindex);
>> }
>> error_prepend(errp,
>> "Failed to %s %s eventfd signaling for interrupt ",
>> fd < 0 ? "tear down" : "set up", action_to_str(action));
>> return ret;
>> }
>>
>> The pattern's first ... matches a "shortest" path to an error_prepend(),
>> where "shortest" means "does not cross an error_prepend(). Its when
>> clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().
>>
>> There are two such "shortest" paths, one to the first error_prepend() in
>> vfio_set_irq_signaling(), and one to the second. Neither path to the
>> third one is not "shortest": they both cross one of the other two
>> error_prepend().
>>
>> The pattern' s second ... matches a path from a matched error_prepend()
>> to the end of the function. There are two paths. Both cross the third
>> error_prepend(). You need "when any" to make the pattern match anyway.
>>
>> Alright, I think I got it. But now I'm paranoid about ... elsewhere.
>> For instance, here's rule1 with error_propagate_prepend() omitted:
>>
>> // Match scenarios with propagation of local error to errp.
>> @rule1 disable optional_qualifier exists@
>> identifier fn, local_err;
>> symbol errp;
>> @@
>>
>> fn(..., Error **errp, ...)
>> {
>> ...
>> Error *local_err = NULL;
>> ...
>> error_propagate(errp, local_err);
>> ...
>> }
>>
>> The second and third ... won't match anything containing
>> error_propagate(). What if a function has multiple error_propagate() on
>> all paths?
>
> I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...
>
>> Like this one:
>>
>> extern foo(int, Error **);
>> extern bar(int, Error **);
>>
>> void frob(Error **errp)
>> {
>> Error *local_err = NULL;
>> int arg;
>>
>> foo(arg, errp);
>> bar(arg, &local_err);
>> error_propagate(errp, local_err);
>> bar(arg + 1, &local_err);
>> error_propagate(errp, local_err);
>> }
>>
>> This is actually a variation of error.h's "Receive and accumulate
>> multiple errors (first one wins)" code snippet.
>
> ah yes, we can propagate to already filled errp, which just clean local_err.
>
>>
>> The Coccinelle script transforms it like this:
>>
>> void frob(Error **errp)
>> {
>> + ERRP_AUTO_PROPAGATE();
>> Error *local_err = NULL;
>> int arg;
>>
>> The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any),
>> but rule1 does not, and we therefore don't convert any of the
>> error_propagate().
>>
>> The result isn't wrong, just useless.
>>
>> Is this the worst case?
>>
>> Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use
>> "... when any" in the error_propagate() case, only in the other cases.
>> Would that help?
>
> I think not, as it will anyway match functions with error_prepend (and any
> number of following error_propagate calls)...
But it's correct to add ERRP_AUTO_PROPAGATE to such functions.. So, may be you
are right. Let's do it, seems better.
>
>>
>> I think this is the only other rule with "..." matching control flow.
>>
>>>>
>>>>> +//
>>>>> +// Note, "exists" says that we want apply rule even if it matches not on
>>>>> +// all possible control flows (otherwise, it will not match standard pattern
>>>>> +// when error_propagate() call is in if branch).
>>>>
>>>> Learned something new. Example: kvm_set_kvm_shadow_mem().
>>>>
>>>> Spelling it "exists disable optional_qualifier" would avoid giving
>>>> readers the idea we're disabling "exists", but Coccinelle doesn't let
>>>> us. Oh well.
>>>>
>>>>> +@ disable optional_qualifier exists@
>>>>> +identifier fn, local_err, errp;
>>>>
>>>> I believe this causes
>>>>
>>>> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>>>>
>>>> Making @errp symbol instead of identifier should fix this.
>>>
>>> Hmm, I didn't see these warnings.. But yes, it should be symbol.
>>>
>>>>
>>>>> +@@
>>>>> +
>>>>> + fn(..., Error **errp, ...)
>>>>> + {
>>>>> ++ ERRP_AUTO_PROPAGATE();
>>>>> + ... when != ERRP_AUTO_PROPAGATE();
>>>>> +(
>>>>> + error_append_hint(errp, ...);
>>>>> +|
>>>>> + error_prepend(errp, ...);
>>>>> +|
>>>>> + error_vprepend(errp, ...);
>>>>> +|
>>>>> + Error *local_err = NULL;
>>>>> + ...
>>>>> +(
>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>> +|
>>>>> + error_propagate(errp, local_err);
>>>>> +)
>>>>> +)
>>>>> + ... when any
>>>>> + }
>>>>> +
>>>>> +
>>>>> +// Match scenarios with propagation of local error to errp.
>>>>> +@rule1 disable optional_qualifier exists@
>>>>> +identifier fn, local_err;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(..., Error **errp, ...)
>>>>> + {
>>>>> + ...
>>>>> + Error *local_err = NULL;
>>>>> + ...
>>>>> +(
>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>> +|
>>>>> + error_propagate(errp, local_err);
>>>>> +)
>>>>
>>>> Indentation off by one.
>>>>
>>>>> + ...
>>>>> + }
>>>>> +
>>>>> +// Convert special case with goto in separate.
>>>>
>>>> s/in separate/separately/
>>>>
>>>>> +// We can probably merge this into the following hunk with help of ( | )
>>>>> +// operator, but it significantly reduce performance on block.c parsing (or it
>>>>
>>>> s/reduce/reduces/
>>>>
>>>>> +// hangs, I don't know)
>>>>
>>>> Sounds like you tried to merge this into the following hunk, but then
>>>> spatch took so long on block.c that you killed it. Correct?
>>>
>>> Yes.
>>
>> I'd say something like "I tried merging this into the following rule the
>> obvious way, but it made Coccinelle hang on block.c."
>>
>>>>
>>>>> +//
>>>>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>>>>> +// things later after all transformations (the rule will be the same, just
>>>>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>>>>
>>>> Weird, but not worth further investigation.
>>>
>>> It partially match to the idea which I saw somewhere in coccinelle documentation,
>>> that coccinelle converts correct C code to correct C code. "out: }" is an example
>>> of incorrect, impossible code flow, and coccinelle can't work with it... But it's
>>> just a thought.
>>>
>>>>
>>>>> +@@
>>>>> +identifier rule1.fn, rule1.local_err, out;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- goto out;
>>>>> ++ return;
>>>>> + ...>
>>>>> +- out:
>>>>> +- error_propagate(errp, local_err);
>>>>
>>>> You neglect to match error_propagate_prepend(). Okay, because (1) that
>>>> pattern doesn't occur in the tree right now, and (2) if it gets added,
>>>> gcc will complain.
>>>
>>> No, because it should not removed. error_propagate_prepend should be converted
>>> to prepend, not removed. So, corresponding gotos should not be removed as well.
>>
>> You're right.
>>
>>>>
>>>>> + }
>>>>> +
>>>>> +// Convert most of local_err related staff.
>>>>
>>>> s/staff/stuff/
>>>>
>>>>> +//
>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>> +// and local_err name. We may match something not related to the pattern
>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>> +// in different blocks inside one function, and in one block follow the
>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>> +// functions with the same name (for different configurations).
>>>>
>>>> Context: rule1 matches functions that have all three of
>>>>
>>>> * an Error **errp parameter
>>>>
>>>> * an Error *local_err = NULL variable declaration
>>>>
>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>> local_err, ...) expression, where @errp is the parameter and
>>>> @local_err is the variable.
>>>>
>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>
>>>> 1. This rule can match functions rule1 does not match if there is
>>>> another function with the same name that rule1 does match.
>>>>
>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>> parts of that function use a different @errp or @local_err.
>>>>
>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>> this one. Correct?
>>>
>>> Yes.
>>
>> Thanks!
>>
>>>>
>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>> don't know it.
>>>> Can we make Coccinelle at least warn us when it converts
>>>> multiple functions with the same name? What about this:
>>>>
>>>> @initialize:python@
>>>> @@
>>>> fnprev = {}
>>>>
>>>> def pr(fn, p):
>>>> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>>>>
>>>> @r@
>>>> identifier rule1.fn;
>>>> position p;
>>>> @@
>>>> fn(...)@p
>>>> {
>>>> ...
>>>> }
>>>> @script:python@
>>>> fn << rule1.fn;
>>>> p << r.p;
>>>> @@
>>>> if fn not in fnprev:
>>>> fnprev[fn] = p
>>>> else:
>>>> if fnprev[fn]:
>>>
>>> hmm, the condition can't be false
>>>
>>>> pr(fn, fnprev[fn])
>>>> fnprev[fn] = None
>>>> pr(fn, p)
>>>
>>> and we'll miss next duplication..
>>
>> The idea is
>>
>> first instance of fn:
>> fn not in fnprev
>> fnprev[fn] = position of instance
>> don't print
>> second instance:
>> fnprev[fn] is the position of the first instance
>> print first two instances
>> subsequent instances: fnprev[fn] is None
>> print this instance
>>
>> I might have screwed up the coding, of course :)
>>
>>> But I like the idea.
>>>
>>>>
>>>> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
>>>> the number of functions with the same name we touch. If it's more than
>>>> one, we print.
>>>>
>>>> Reports about a dozen function names for the whole tree in my testing.
>>>> Inspecting the changes to them manually is feasible. None of them are
>>>> in files touched by this series.
>>>>
>>>> The line printed for the first match is pretty useless for me: it points
>>>> to a Coccinelle temporary file *shrug*.
>>>>
>>>> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
>>>> sure hope we don't do that. Multiple @local_err variables... hmm.
>>>> Perhaps we could again concoct some script rules to lead us to spots to
>>>> check manually. See below for my attempt.
>>>>
>>>> What's the worst that could happen if we blindly converted such code?
>>>> The answer to that question tells us how hard to work on finding and
>>>> checking these guys.
>>>>
>>>>> +//
>>>>> +// Note also that errp-cleaning functions
>>>>> +// error_free_errp
>>>>> +// error_report_errp
>>>>> +// error_reportf_errp
>>>>> +// warn_report_errp
>>>>> +// warn_reportf_errp
>>>>> +// are not yet implemented. They must call corresponding Error* - freeing
>>>>> +// function and then set *errp to NULL, to avoid further propagation to
>>>>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>>>>> +// For example, error_free_errp may look like this:
>>>>> +//
>>>>> +// void error_free_errp(Error **errp)
>>>>> +// {
>>>>> +// error_free(*errp);
>>>>> +// *errp = NULL;
>>>>> +// }
>>>>> +@ exists@
>>>>> +identifier rule1.fn, rule1.local_err;
>>>>> +expression list args;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +(
>>>>
>>>> Each of the following patterns applies anywhere in the function.
>>>>
>>>> First pattern: delete @local_err
>>>>
>>>>> +- Error *local_err = NULL;
>>>>
>>>> Common case: occurs just once, not nested. Anything else is suspicious.
>>>>
>>>> Both can be detected in the resulting patches with a bit of AWK
>>>> wizardry:
>>>>
>>>> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
>>>> nested
>>>> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
>>>> nested
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>> nested
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>> more than one
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>>
>>>> Oh.
>>>>
>>>> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>>>>
>>>> xen_block_device_destroy() has multiple Error *local_err.
>>>>
>>>> In both cases, manual review is required to ensure the conversion is
>>>> okay. I believe it is.
>>>>
>>>> Note that the AWK script relies on diff showing the function name in @@
>>>> lines, which doesn't always work due to our coding style.
>>>>
>>>> For the whole tree, I get some 30 spots. Feasible.
>>>>
>>>>> +|
>>>>
>>>> Second pattern: clear @errp after freeing it
>>>>
>>>>> +
>>>>> +// Convert error clearing functions
>>>>
>>>> Suggest: Ensure @local_err is cleared on free
>>>>
>>>>> +(
>>>>> +- error_free(local_err);
>>>>> ++ error_free_errp(errp);
>>>>> +|
>>>>> +- error_report_err(local_err);
>>>>> ++ error_report_errp(errp);
>>>>> +|
>>>>> +- error_reportf_err(local_err, args);
>>>>> ++ error_reportf_errp(errp, args);
>>>>> +|
>>>>> +- warn_report_err(local_err);
>>>>> ++ warn_report_errp(errp);
>>>>> +|
>>>>> +- warn_reportf_err(local_err, args);
>>>>> ++ warn_reportf_errp(errp, args);
>>>>> +)
>>>>
>>>> As you mention above, these guys don't exist, yet. Builds anyway,
>>>> because this part of the rule is not used in this patch series. You
>>>> don't want to omit it, because then the script becomes unsafe to use.
>>>>
>>>> We could also open-code:
>>>>
>>>> // Convert error clearing functions
>>>> (
>>>> - error_free(local_err);
>>>> + error_free(*errp);
>>>> + *errp = NULL;
>>>> |
>>>> ... and so forth ...
>>>> )
>>>>
>>>> Matter of taste. Whatever is easier to explain in the comments. Since
>>>> you already wrote one...
>>>
>>> I just feel that using helper functions is safer way..
>>>
>>>>
>>>> We talked about extending this series slightly so these guys are used.
>>>> I may still look into that.
>>>>
>>>>> +?- local_err = NULL;
>>>>> +
>>>>
>>>> The new helpers clear @local_err. Assignment now redundant, delete.
>>>> Okay.
>>>>
>>>>> +|
>>>>
>>>> Third and fourth pattern: delete error_propagate()
>>>>
>>>>> +- error_propagate_prepend(errp, local_err, args);
>>>>> ++ error_prepend(errp, args);
>>>>> +|
>>>>> +- error_propagate(errp, local_err);
>>>>> +|
>>>>
>>>> Fifth pattern: use @errp directly
>>>>
>>>>> +- &local_err
>>>>> ++ errp
>>>>> +)
>>>>> + ...>
>>>>> + }
>>>>> +
>>>>> +// Convert remaining local_err usage. It should be different kinds of error
>>>>> +// checking in if operators. We can't merge this into previous hunk, as this
>>>>
>>>> In if conditionals, I suppose. It's the case for this patch. If I
>>>> apply the script to the whole tree, the rule gets also applied in other
>>>> contexts. The sentence might mislead as much as it helps. Keep it or
>>>> delete it?
>>>
>>> Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
>>
>> "Need to be careful" means "needs careful manual review", which I
>> believe is not feasible; see "Preface to my review of this script"
>> above.
>>
>> But do we really need to be careful here?
>>
>> This rule should apply only where we added ERRP_AUTO_PROPAGATE().
>>
>> Except when rule chaining via function name fails us, but we plan to
>> detect that and review manually, so let's ignore this issue here.
>>
>> Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling
>> replacement of @local_err by @errp is its whole point.
>>
>> What exactly do we need to be careful about?
>
> Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."
>
>>
>>>
>>>>
>>>>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>>>>> +@@
>>>>> +identifier rule1.fn, rule1.local_err;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- local_err
>>>>> ++ *errp
>>>>> + ...>
>>>>> + }
>>>>> +
>>>>> +// Always use the same patter for checking error
>>>>
>>>> s/patter/pattern/
>>>>
>>>>> +@@
>>>>> +identifier rule1.fn;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- *errp != NULL
>>>>> ++ *errp
>>>>> + ...>
>>>>> + }
>>>>
>>
>
>
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-11 6:55 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-11 9:04 ` Markus Armbruster
-1 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-11 9:04 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Laszlo Ersek, Christian Schoenebeck, Michael Roth, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 10.03.2020 18:47, Markus Armbruster wrote:
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>
>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>> Suggest
>>>>
>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>
>>>> or
>>>>
>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>
>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>
>>>>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>>>>> does corresponding changes in code (look for details in
>>>>> include/qapi/error.h)
>>>>>
>>>>> Usage example:
>>>>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>>
>>>> Suggest FILES... instead of a specific set of files.
>>>>
>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>> ---
>>>>>
>>>>> Cc: Eric Blake <eblake@redhat.com>
>>>>> Cc: Kevin Wolf <kwolf@redhat.com>
>>>>> Cc: Max Reitz <mreitz@redhat.com>
>>>>> Cc: Greg Kurz <groug@kaod.org>
>>>>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>>>>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>>>>> Cc: Anthony Perard <anthony.perard@citrix.com>
>>>>> Cc: Paul Durrant <paul@xen.org>
>>>>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>>>>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>>>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>>>>> Cc: Markus Armbruster <armbru@redhat.com>
>>>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>>>> Cc: qemu-block@nongnu.org
>>>>> Cc: qemu-devel@nongnu.org
>>>>> Cc: xen-devel@lists.xenproject.org
>>>>>
>>>>> include/qapi/error.h | 3 +
>>>>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>>>>> 2 files changed, 234 insertions(+)
>>>>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>>>
>>>>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>>>>> index bb9bcf02fb..fbfc6f1c0b 100644
>>>>> --- a/include/qapi/error.h
>>>>> +++ b/include/qapi/error.h
>>>>> @@ -211,6 +211,9 @@
>>>>> * }
>>>>> * ...
>>>>> * }
>>>>> + *
>>>>> + * For mass conversion use script
>>>>
>>>> mass-conversion (we're not converting mass, we're converting en masse)
>>>>
>>>>> + * scripts/coccinelle/auto-propagated-errp.cocci
>>>>> */
>>>>> #ifndef ERROR_H
>>>>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>> new file mode 100644
>>>>> index 0000000000..bff274bd6d
>>>>> --- /dev/null
>>>>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>
>>>> Preface to my review of this script: may aim isn't to make it
>>>> bullet-proof. I want to (1) make it good enough (explained in a
>>>> jiffie), and (2) automatically identify the spots where it still isn't
>>>> obviously safe for manual review.
>>>>
>>>> The latter may involve additional scripting. That's okay.
>>>>
>>>> The script is good enough when the number of possibly unsafe spots is
>>>> low enough for careful manual review.
>>>>
>>>> When I ask for improvements that, in your opinion, go beyond "good
>>>> enough", please push back. I'm sure we can work it out together.
>>>>
>>>>> @@ -0,0 +1,231 @@
>>>>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>>>>> +//
>>>>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>>>>> +//
>>>>> +// This program is free software; you can redistribute it and/or modify
>>>>> +// it under the terms of the GNU General Public License as published by
>>>>> +// the Free Software Foundation; either version 2 of the License, or
>>>>> +// (at your option) any later version.
>>>>> +//
>>>>> +// This program is distributed in the hope that it will be useful,
>>>>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>>> +// GNU General Public License for more details.
>>>>> +//
>>>>> +// You should have received a copy of the GNU General Public License
>>>>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>>> +//
>>>>> +// Usage example:
>>>>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>>>>
>>>> You have --max-width 80 here, but not in the commit message. Default
>>>> seems to be 78. Any particular reason to change it to 80?
>>>
>>> Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
>>> So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
>>> Still, may be I'm wrong, we can check it.
>>
>> If you can point to an example where --max-width helps, keep it, and
>> update the commit message to match. Else, drop it.
>>
>>>>
>>>>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
As our discussion shows, this script is somewhat hard to understand.
That's okay, it solves a somewhat thorny problem, and I don't have
better ideas. But let me state the intended transformations once more,
so I don't get lost in the details.
Motivation:
1. Make error propagation less error-prone and improve stack backtraces
The "receive error in &local_error, propagate to @errp" pattern is
tedious, error-prone, and produces unhelpful stack backtraces with
&error_abort.
ERRP_AUTO_PROPAGATE() removes manual propagation. It additionally
gives us the stack backtraces we want.
2. Improve error messages with &error_fatal
Passing @errp to error_append_hint(), error_prepend() or
error_vprepend() is useless when @errp is &error_fatal.
ERRP_AUTO_PROPAGATE() fixes this by delaying &error_fatal handling
until the automatic propagation.
The intended transformation has three parts:
* Replace declaration of @local_err by ERRP_AUTO_PROPAGATE() where
needed for 1 or 2.
It's needed when we also drop some error propagation from the function
(motivation 1), or the function calls error_append_hint(),
error_prepend() or error_vprepend() (motivation 2). A function could
do both. I'll refer back to this below.
* Drop error_propagate(errp, local_err)
Special case: error_propagate_prepend(errp, local_err, ...) becomes
error_prepend(errp, ...).
Only correct if there is no other such error_propagate() /
error_propagate_prepend() on any path from the here to return.
* Replace remaining use of @local_err by *errp
>>>>> +
>>>>> +// Switch unusual (Error **) parameter names to errp
>>>>
>>>> Let's drop the parenthesis around Error **
>>>>
>>>>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>>>>
>>>> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
>>>> make the fact we're messing with @errp more obvious. Too late; I
>>>> shouldn't rock the boat that much now.
>>>>
>>>>> +//
>>>>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>>>>> +// parameter.
>>>>> +//
>>>>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>>>>> +// non generic semantics and may have unusual Error ** argument name for purpose
>>>>
>>>> non-generic
>>>>
>>>> for a purpose
>>>>
>>>> Wrap comment lines around column 70, please. It's easier to read.
>>>>
>>>> Maybe
>>>>
>>>> // Skip functions with "assert(_errp && *_errp)" statement, because that
>>>> // signals unusual semantics, and the parameter name may well serve a
>>>> // purpose.
>>>
>>> Sounds good.
>>>
>>>>
>>>>> +// (like nbd_iter_channel_error()).
>>>>> +//
>>>>> +// Skip util/error.c to not touch, for example, error_propagate and
>>>>> +// error_propagate_prepend().
>>>>
>>>> error_propagate()
>>>>
>>>> I much appreciate your meticulous explanation of what you skip and why.
>>>>
>>>>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>>>>> +identifier fn;
>>>>> +identifier _errp != errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...,
>>>>> +- Error **_errp
>>>>> ++ Error **errp
>>>>> + ,...)
>>>>> + {
>>>>> +(
>>>>> + ... when != assert(_errp && *_errp)
>>>>> +&
>>>>> + <...
>>>>> +- _errp
>>>>> ++ errp
>>>>> + ...>
>>>>> +)
>>>>> + }
>>>>
>>>> This rule is required to make the actual transformations (below) work
>>>> even for parameters with names other than @errp. I believe it's not
>>>> used in this series. In fact, I can't see a use for it in the entire
>>>> tree right now. Okay anyway.
>>>>
>>>>> +
>>>>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>>>>> +//
>>>>> +// Note, that without "when any" final "..." may not want to mach something
>>>>
>>>> s/final "..." may not mach/the final "..." does not match/
>>>>
>>>>> +// matched by previous pattern, i.e. the rule will not match double
>>>>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>>>>
>>>> Can't say I fully understand Coccinelle there. I figure you came to
>>>> this knowledge the hard way.
>>>
>>> It's follows from smpl grammar document:
>>>
>>> "Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
>>> ...
>>> "_when any_ removes the aforementioned constraint that “...” matches the shortest path"
>>
>> Let me think that through.
>>
>> The pattern with the cases other than error_prepend() omitted:
>>
>> fn(..., Error **errp, ...)
>> {
>> + ERRP_AUTO_PROPAGATE();
>> ... when != ERRP_AUTO_PROPAGATE();
>> error_prepend(errp, ...);
>> ... when any
>> }
>>
>> Tail of vfio_set_irq_signaling():
>>
>> name = index_to_str(vbasedev, index);
>> if (name) {
>> error_prepend(errp, "%s-%d: ", name, subindex);
>> } else {
>> error_prepend(errp, "index %d-%d: ", index, subindex);
>> }
>> error_prepend(errp,
>> "Failed to %s %s eventfd signaling for interrupt ",
>> fd < 0 ? "tear down" : "set up", action_to_str(action));
>> return ret;
>> }
>>
>> The pattern's first ... matches a "shortest" path to an error_prepend(),
>> where "shortest" means "does not cross an error_prepend(). Its when
>> clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().
>>
>> There are two such "shortest" paths, one to the first error_prepend() in
>> vfio_set_irq_signaling(), and one to the second. Neither path to the
>> third one is not "shortest": they both cross one of the other two
>> error_prepend().
>>
>> The pattern' s second ... matches a path from a matched error_prepend()
>> to the end of the function. There are two paths. Both cross the third
>> error_prepend(). You need "when any" to make the pattern match anyway.
>>
>> Alright, I think I got it. But now I'm paranoid about ... elsewhere.
>> For instance, here's rule1 with error_propagate_prepend() omitted:
>>
>> // Match scenarios with propagation of local error to errp.
>> @rule1 disable optional_qualifier exists@
>> identifier fn, local_err;
>> symbol errp;
>> @@
>>
>> fn(..., Error **errp, ...)
>> {
>> ...
>> Error *local_err = NULL;
>> ...
>> error_propagate(errp, local_err);
>> ...
>> }
>>
>> The second and third ... won't match anything containing
>> error_propagate(). What if a function has multiple error_propagate() on
>> all paths?
>
> I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...
I'm fine with not touching "strange" patterns. But we do at least in
the example I gave below: we still add ERRP_AUTO_PROPAGATE().
We either avoid that in the Coccinelle script, or we catch it
afterwards. Catching it afterwards should be feasible:
* If we also drop some error propagation from this function: okay.
* If this function call error_append_hint(), error_prepend() or
error_vprepend(): okay.
* Else: unwanted, back out.
Moreover, I'd like us to double-check we really don't want to touch the
things we don't touch. Feels feasible to me, too: after running
Coccinelle, search for unconverted error_append_hint(), error_prepend(),
error_vprepend(), error_propagate_prepend(), error_propagate().
>> Like this one:
>>
>> extern foo(int, Error **);
>> extern bar(int, Error **);
>>
>> void frob(Error **errp)
>> {
>> Error *local_err = NULL;
>> int arg;
>>
>> foo(arg, errp);
>> bar(arg, &local_err);
>> error_propagate(errp, local_err);
>> bar(arg + 1, &local_err);
>> error_propagate(errp, local_err);
>> }
>>
>> This is actually a variation of error.h's "Receive and accumulate
>> multiple errors (first one wins)" code snippet.
>
> ah yes, we can propagate to already filled errp, which just clean local_err.
>
>>
>> The Coccinelle script transforms it like this:
>>
>> void frob(Error **errp)
>> {
>> + ERRP_AUTO_PROPAGATE();
>> Error *local_err = NULL;
>> int arg;
>>
>> The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any),
>> but rule1 does not, and we therefore don't convert any of the
>> error_propagate().
>>
>> The result isn't wrong, just useless.
>>
>> Is this the worst case?
>>
>> Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use
>> "... when any" in the error_propagate() case, only in the other cases.
>> Would that help?
>
> I think not, as it will anyway match functions with error_prepend (and any
> number of following error_propagate calls)...
I'm not sure I understand this sentence.
The aim of my "possible improvement" is to avoid unwanted
ERRP_AUTO_PROPAGATE() in the Coccinelle script. We can instead search
for unwanted ones after the Coccinelle run, and back them out.
>> I think this is the only other rule with "..." matching control flow.
>>
>>>>
>>>>> +//
>>>>> +// Note, "exists" says that we want apply rule even if it matches not on
>>>>> +// all possible control flows (otherwise, it will not match standard pattern
>>>>> +// when error_propagate() call is in if branch).
>>>>
>>>> Learned something new. Example: kvm_set_kvm_shadow_mem().
>>>>
>>>> Spelling it "exists disable optional_qualifier" would avoid giving
>>>> readers the idea we're disabling "exists", but Coccinelle doesn't let
>>>> us. Oh well.
>>>>
>>>>> +@ disable optional_qualifier exists@
>>>>> +identifier fn, local_err, errp;
>>>>
>>>> I believe this causes
>>>>
>>>> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>>>>
>>>> Making @errp symbol instead of identifier should fix this.
>>>
>>> Hmm, I didn't see these warnings.. But yes, it should be symbol.
>>>
>>>>
>>>>> +@@
>>>>> +
>>>>> + fn(..., Error **errp, ...)
>>>>> + {
>>>>> ++ ERRP_AUTO_PROPAGATE();
>>>>> + ... when != ERRP_AUTO_PROPAGATE();
>>>>> +(
>>>>> + error_append_hint(errp, ...);
>>>>> +|
>>>>> + error_prepend(errp, ...);
>>>>> +|
>>>>> + error_vprepend(errp, ...);
>>>>> +|
>>>>> + Error *local_err = NULL;
>>>>> + ...
>>>>> +(
>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>> +|
>>>>> + error_propagate(errp, local_err);
>>>>> +)
>>>>> +)
>>>>> + ... when any
>>>>> + }
>>>>> +
>>>>> +
>>>>> +// Match scenarios with propagation of local error to errp.
>>>>> +@rule1 disable optional_qualifier exists@
>>>>> +identifier fn, local_err;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(..., Error **errp, ...)
>>>>> + {
>>>>> + ...
>>>>> + Error *local_err = NULL;
>>>>> + ...
>>>>> +(
>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>> +|
>>>>> + error_propagate(errp, local_err);
>>>>> +)
>>>>
>>>> Indentation off by one.
>>>>
>>>>> + ...
>>>>> + }
>>>>> +
>>>>> +// Convert special case with goto in separate.
>>>>
>>>> s/in separate/separately/
>>>>
>>>>> +// We can probably merge this into the following hunk with help of ( | )
>>>>> +// operator, but it significantly reduce performance on block.c parsing (or it
>>>>
>>>> s/reduce/reduces/
>>>>
>>>>> +// hangs, I don't know)
>>>>
>>>> Sounds like you tried to merge this into the following hunk, but then
>>>> spatch took so long on block.c that you killed it. Correct?
>>>
>>> Yes.
>>
>> I'd say something like "I tried merging this into the following rule the
>> obvious way, but it made Coccinelle hang on block.c."
>>
>>>>
>>>>> +//
>>>>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>>>>> +// things later after all transformations (the rule will be the same, just
>>>>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>>>>
>>>> Weird, but not worth further investigation.
>>>
>>> It partially match to the idea which I saw somewhere in coccinelle documentation,
>>> that coccinelle converts correct C code to correct C code. "out: }" is an example
>>> of incorrect, impossible code flow, and coccinelle can't work with it... But it's
>>> just a thought.
>>>
>>>>
>>>>> +@@
>>>>> +identifier rule1.fn, rule1.local_err, out;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- goto out;
>>>>> ++ return;
>>>>> + ...>
>>>>> +- out:
>>>>> +- error_propagate(errp, local_err);
>>>>
>>>> You neglect to match error_propagate_prepend(). Okay, because (1) that
>>>> pattern doesn't occur in the tree right now, and (2) if it gets added,
>>>> gcc will complain.
>>>
>>> No, because it should not removed. error_propagate_prepend should be converted
>>> to prepend, not removed. So, corresponding gotos should not be removed as well.
>>
>> You're right.
>>
>>>>
>>>>> + }
>>>>> +
>>>>> +// Convert most of local_err related staff.
>>>>
>>>> s/staff/stuff/
>>>>
>>>>> +//
>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>> +// and local_err name. We may match something not related to the pattern
>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>> +// in different blocks inside one function, and in one block follow the
>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>> +// functions with the same name (for different configurations).
>>>>
>>>> Context: rule1 matches functions that have all three of
>>>>
>>>> * an Error **errp parameter
>>>>
>>>> * an Error *local_err = NULL variable declaration
>>>>
>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>> local_err, ...) expression, where @errp is the parameter and
>>>> @local_err is the variable.
>>>>
>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>
>>>> 1. This rule can match functions rule1 does not match if there is
>>>> another function with the same name that rule1 does match.
>>>>
>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>> parts of that function use a different @errp or @local_err.
>>>>
>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>> this one. Correct?
>>>
>>> Yes.
>>
>> Thanks!
>>
>>>>
>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>> don't know it.
>>>> Can we make Coccinelle at least warn us when it converts
>>>> multiple functions with the same name? What about this:
>>>>
>>>> @initialize:python@
>>>> @@
>>>> fnprev = {}
>>>>
>>>> def pr(fn, p):
>>>> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>>>>
>>>> @r@
>>>> identifier rule1.fn;
>>>> position p;
>>>> @@
>>>> fn(...)@p
>>>> {
>>>> ...
>>>> }
>>>> @script:python@
>>>> fn << rule1.fn;
>>>> p << r.p;
>>>> @@
>>>> if fn not in fnprev:
>>>> fnprev[fn] = p
>>>> else:
>>>> if fnprev[fn]:
>>>
>>> hmm, the condition can't be false
>>>
>>>> pr(fn, fnprev[fn])
>>>> fnprev[fn] = None
>>>> pr(fn, p)
>>>
>>> and we'll miss next duplication..
>>
>> The idea is
>>
>> first instance of fn:
>> fn not in fnprev
>> fnprev[fn] = position of instance
>> don't print
>> second instance:
>> fnprev[fn] is the position of the first instance
>> print first two instances
>> subsequent instances: fnprev[fn] is None
>> print this instance
>>
>> I might have screwed up the coding, of course :)
>>
>>> But I like the idea.
>>>
>>>>
>>>> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
>>>> the number of functions with the same name we touch. If it's more than
>>>> one, we print.
>>>>
>>>> Reports about a dozen function names for the whole tree in my testing.
>>>> Inspecting the changes to them manually is feasible. None of them are
>>>> in files touched by this series.
>>>>
>>>> The line printed for the first match is pretty useless for me: it points
>>>> to a Coccinelle temporary file *shrug*.
>>>>
>>>> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
>>>> sure hope we don't do that. Multiple @local_err variables... hmm.
>>>> Perhaps we could again concoct some script rules to lead us to spots to
>>>> check manually. See below for my attempt.
>>>>
>>>> What's the worst that could happen if we blindly converted such code?
>>>> The answer to that question tells us how hard to work on finding and
>>>> checking these guys.
>>>>
>>>>> +//
>>>>> +// Note also that errp-cleaning functions
>>>>> +// error_free_errp
>>>>> +// error_report_errp
>>>>> +// error_reportf_errp
>>>>> +// warn_report_errp
>>>>> +// warn_reportf_errp
>>>>> +// are not yet implemented. They must call corresponding Error* - freeing
>>>>> +// function and then set *errp to NULL, to avoid further propagation to
>>>>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>>>>> +// For example, error_free_errp may look like this:
>>>>> +//
>>>>> +// void error_free_errp(Error **errp)
>>>>> +// {
>>>>> +// error_free(*errp);
>>>>> +// *errp = NULL;
>>>>> +// }
>>>>> +@ exists@
>>>>> +identifier rule1.fn, rule1.local_err;
>>>>> +expression list args;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +(
>>>>
>>>> Each of the following patterns applies anywhere in the function.
>>>>
>>>> First pattern: delete @local_err
>>>>
>>>>> +- Error *local_err = NULL;
>>>>
>>>> Common case: occurs just once, not nested. Anything else is suspicious.
>>>>
>>>> Both can be detected in the resulting patches with a bit of AWK
>>>> wizardry:
>>>>
>>>> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
>>>> nested
>>>> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
>>>> nested
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>> nested
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>> more than one
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>>
>>>> Oh.
>>>>
>>>> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>>>>
>>>> xen_block_device_destroy() has multiple Error *local_err.
>>>>
>>>> In both cases, manual review is required to ensure the conversion is
>>>> okay. I believe it is.
>>>>
>>>> Note that the AWK script relies on diff showing the function name in @@
>>>> lines, which doesn't always work due to our coding style.
>>>>
>>>> For the whole tree, I get some 30 spots. Feasible.
>>>>
>>>>> +|
>>>>
>>>> Second pattern: clear @errp after freeing it
>>>>
>>>>> +
>>>>> +// Convert error clearing functions
>>>>
>>>> Suggest: Ensure @local_err is cleared on free
>>>>
>>>>> +(
>>>>> +- error_free(local_err);
>>>>> ++ error_free_errp(errp);
>>>>> +|
>>>>> +- error_report_err(local_err);
>>>>> ++ error_report_errp(errp);
>>>>> +|
>>>>> +- error_reportf_err(local_err, args);
>>>>> ++ error_reportf_errp(errp, args);
>>>>> +|
>>>>> +- warn_report_err(local_err);
>>>>> ++ warn_report_errp(errp);
>>>>> +|
>>>>> +- warn_reportf_err(local_err, args);
>>>>> ++ warn_reportf_errp(errp, args);
>>>>> +)
>>>>
>>>> As you mention above, these guys don't exist, yet. Builds anyway,
>>>> because this part of the rule is not used in this patch series. You
>>>> don't want to omit it, because then the script becomes unsafe to use.
>>>>
>>>> We could also open-code:
>>>>
>>>> // Convert error clearing functions
>>>> (
>>>> - error_free(local_err);
>>>> + error_free(*errp);
>>>> + *errp = NULL;
>>>> |
>>>> ... and so forth ...
>>>> )
>>>>
>>>> Matter of taste. Whatever is easier to explain in the comments. Since
>>>> you already wrote one...
>>>
>>> I just feel that using helper functions is safer way..
>>>
>>>>
>>>> We talked about extending this series slightly so these guys are used.
>>>> I may still look into that.
>>>>
>>>>> +?- local_err = NULL;
>>>>> +
>>>>
>>>> The new helpers clear @local_err. Assignment now redundant, delete.
>>>> Okay.
>>>>
>>>>> +|
>>>>
>>>> Third and fourth pattern: delete error_propagate()
>>>>
>>>>> +- error_propagate_prepend(errp, local_err, args);
>>>>> ++ error_prepend(errp, args);
>>>>> +|
>>>>> +- error_propagate(errp, local_err);
>>>>> +|
>>>>
>>>> Fifth pattern: use @errp directly
>>>>
>>>>> +- &local_err
>>>>> ++ errp
>>>>> +)
>>>>> + ...>
>>>>> + }
>>>>> +
>>>>> +// Convert remaining local_err usage. It should be different kinds of error
>>>>> +// checking in if operators. We can't merge this into previous hunk, as this
>>>>
>>>> In if conditionals, I suppose. It's the case for this patch. If I
>>>> apply the script to the whole tree, the rule gets also applied in other
>>>> contexts. The sentence might mislead as much as it helps. Keep it or
>>>> delete it?
>>>
>>> Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
>>
>> "Need to be careful" means "needs careful manual review", which I
>> believe is not feasible; see "Preface to my review of this script"
>> above.
>>
>> But do we really need to be careful here?
>>
>> This rule should apply only where we added ERRP_AUTO_PROPAGATE().
>>
>> Except when rule chaining via function name fails us, but we plan to
>> detect that and review manually, so let's ignore this issue here.
>>
>> Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling
>> replacement of @local_err by @errp is its whole point.
>>
>> What exactly do we need to be careful about?
>
> Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."
Something like this, perhaps?
// Convert remaining local_err usage, typically error checking.
// We can't merge this into previous hunk, as this conflicts with other
// substitutions in it (at least with "- local_err = NULL").
>>>>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>>>>> +@@
>>>>> +identifier rule1.fn, rule1.local_err;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- local_err
>>>>> ++ *errp
>>>>> + ...>
>>>>> + }
>>>>> +
>>>>> +// Always use the same patter for checking error
>>>>
>>>> s/patter/pattern/
>>>>
>>>>> +@@
>>>>> +identifier rule1.fn;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- *errp != NULL
>>>>> ++ *errp
>>>>> + ...>
>>>>> + }
>>>>
>>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 9:04 ` Markus Armbruster
0 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-11 9:04 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Laszlo Ersek, Christian Schoenebeck, Michael Roth, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 10.03.2020 18:47, Markus Armbruster wrote:
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>
>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>> Suggest
>>>>
>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>
>>>> or
>>>>
>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>
>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>
>>>>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>>>>> does corresponding changes in code (look for details in
>>>>> include/qapi/error.h)
>>>>>
>>>>> Usage example:
>>>>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>>
>>>> Suggest FILES... instead of a specific set of files.
>>>>
>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>> ---
>>>>>
>>>>> Cc: Eric Blake <eblake@redhat.com>
>>>>> Cc: Kevin Wolf <kwolf@redhat.com>
>>>>> Cc: Max Reitz <mreitz@redhat.com>
>>>>> Cc: Greg Kurz <groug@kaod.org>
>>>>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>>>>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>>>>> Cc: Anthony Perard <anthony.perard@citrix.com>
>>>>> Cc: Paul Durrant <paul@xen.org>
>>>>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>>>>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>>>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>>>>> Cc: Markus Armbruster <armbru@redhat.com>
>>>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>>>> Cc: qemu-block@nongnu.org
>>>>> Cc: qemu-devel@nongnu.org
>>>>> Cc: xen-devel@lists.xenproject.org
>>>>>
>>>>> include/qapi/error.h | 3 +
>>>>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>>>>> 2 files changed, 234 insertions(+)
>>>>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>>>
>>>>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>>>>> index bb9bcf02fb..fbfc6f1c0b 100644
>>>>> --- a/include/qapi/error.h
>>>>> +++ b/include/qapi/error.h
>>>>> @@ -211,6 +211,9 @@
>>>>> * }
>>>>> * ...
>>>>> * }
>>>>> + *
>>>>> + * For mass conversion use script
>>>>
>>>> mass-conversion (we're not converting mass, we're converting en masse)
>>>>
>>>>> + * scripts/coccinelle/auto-propagated-errp.cocci
>>>>> */
>>>>> #ifndef ERROR_H
>>>>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>> new file mode 100644
>>>>> index 0000000000..bff274bd6d
>>>>> --- /dev/null
>>>>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>
>>>> Preface to my review of this script: may aim isn't to make it
>>>> bullet-proof. I want to (1) make it good enough (explained in a
>>>> jiffie), and (2) automatically identify the spots where it still isn't
>>>> obviously safe for manual review.
>>>>
>>>> The latter may involve additional scripting. That's okay.
>>>>
>>>> The script is good enough when the number of possibly unsafe spots is
>>>> low enough for careful manual review.
>>>>
>>>> When I ask for improvements that, in your opinion, go beyond "good
>>>> enough", please push back. I'm sure we can work it out together.
>>>>
>>>>> @@ -0,0 +1,231 @@
>>>>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>>>>> +//
>>>>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>>>>> +//
>>>>> +// This program is free software; you can redistribute it and/or modify
>>>>> +// it under the terms of the GNU General Public License as published by
>>>>> +// the Free Software Foundation; either version 2 of the License, or
>>>>> +// (at your option) any later version.
>>>>> +//
>>>>> +// This program is distributed in the hope that it will be useful,
>>>>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>>> +// GNU General Public License for more details.
>>>>> +//
>>>>> +// You should have received a copy of the GNU General Public License
>>>>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>>> +//
>>>>> +// Usage example:
>>>>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>>>>
>>>> You have --max-width 80 here, but not in the commit message. Default
>>>> seems to be 78. Any particular reason to change it to 80?
>>>
>>> Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
>>> So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
>>> Still, may be I'm wrong, we can check it.
>>
>> If you can point to an example where --max-width helps, keep it, and
>> update the commit message to match. Else, drop it.
>>
>>>>
>>>>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
As our discussion shows, this script is somewhat hard to understand.
That's okay, it solves a somewhat thorny problem, and I don't have
better ideas. But let me state the intended transformations once more,
so I don't get lost in the details.
Motivation:
1. Make error propagation less error-prone and improve stack backtraces
The "receive error in &local_error, propagate to @errp" pattern is
tedious, error-prone, and produces unhelpful stack backtraces with
&error_abort.
ERRP_AUTO_PROPAGATE() removes manual propagation. It additionally
gives us the stack backtraces we want.
2. Improve error messages with &error_fatal
Passing @errp to error_append_hint(), error_prepend() or
error_vprepend() is useless when @errp is &error_fatal.
ERRP_AUTO_PROPAGATE() fixes this by delaying &error_fatal handling
until the automatic propagation.
The intended transformation has three parts:
* Replace declaration of @local_err by ERRP_AUTO_PROPAGATE() where
needed for 1 or 2.
It's needed when we also drop some error propagation from the function
(motivation 1), or the function calls error_append_hint(),
error_prepend() or error_vprepend() (motivation 2). A function could
do both. I'll refer back to this below.
* Drop error_propagate(errp, local_err)
Special case: error_propagate_prepend(errp, local_err, ...) becomes
error_prepend(errp, ...).
Only correct if there is no other such error_propagate() /
error_propagate_prepend() on any path from the here to return.
* Replace remaining use of @local_err by *errp
>>>>> +
>>>>> +// Switch unusual (Error **) parameter names to errp
>>>>
>>>> Let's drop the parenthesis around Error **
>>>>
>>>>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>>>>
>>>> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
>>>> make the fact we're messing with @errp more obvious. Too late; I
>>>> shouldn't rock the boat that much now.
>>>>
>>>>> +//
>>>>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>>>>> +// parameter.
>>>>> +//
>>>>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>>>>> +// non generic semantics and may have unusual Error ** argument name for purpose
>>>>
>>>> non-generic
>>>>
>>>> for a purpose
>>>>
>>>> Wrap comment lines around column 70, please. It's easier to read.
>>>>
>>>> Maybe
>>>>
>>>> // Skip functions with "assert(_errp && *_errp)" statement, because that
>>>> // signals unusual semantics, and the parameter name may well serve a
>>>> // purpose.
>>>
>>> Sounds good.
>>>
>>>>
>>>>> +// (like nbd_iter_channel_error()).
>>>>> +//
>>>>> +// Skip util/error.c to not touch, for example, error_propagate and
>>>>> +// error_propagate_prepend().
>>>>
>>>> error_propagate()
>>>>
>>>> I much appreciate your meticulous explanation of what you skip and why.
>>>>
>>>>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>>>>> +identifier fn;
>>>>> +identifier _errp != errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...,
>>>>> +- Error **_errp
>>>>> ++ Error **errp
>>>>> + ,...)
>>>>> + {
>>>>> +(
>>>>> + ... when != assert(_errp && *_errp)
>>>>> +&
>>>>> + <...
>>>>> +- _errp
>>>>> ++ errp
>>>>> + ...>
>>>>> +)
>>>>> + }
>>>>
>>>> This rule is required to make the actual transformations (below) work
>>>> even for parameters with names other than @errp. I believe it's not
>>>> used in this series. In fact, I can't see a use for it in the entire
>>>> tree right now. Okay anyway.
>>>>
>>>>> +
>>>>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>>>>> +//
>>>>> +// Note, that without "when any" final "..." may not want to mach something
>>>>
>>>> s/final "..." may not mach/the final "..." does not match/
>>>>
>>>>> +// matched by previous pattern, i.e. the rule will not match double
>>>>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>>>>
>>>> Can't say I fully understand Coccinelle there. I figure you came to
>>>> this knowledge the hard way.
>>>
>>> It's follows from smpl grammar document:
>>>
>>> "Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
>>> ...
>>> "_when any_ removes the aforementioned constraint that “...” matches the shortest path"
>>
>> Let me think that through.
>>
>> The pattern with the cases other than error_prepend() omitted:
>>
>> fn(..., Error **errp, ...)
>> {
>> + ERRP_AUTO_PROPAGATE();
>> ... when != ERRP_AUTO_PROPAGATE();
>> error_prepend(errp, ...);
>> ... when any
>> }
>>
>> Tail of vfio_set_irq_signaling():
>>
>> name = index_to_str(vbasedev, index);
>> if (name) {
>> error_prepend(errp, "%s-%d: ", name, subindex);
>> } else {
>> error_prepend(errp, "index %d-%d: ", index, subindex);
>> }
>> error_prepend(errp,
>> "Failed to %s %s eventfd signaling for interrupt ",
>> fd < 0 ? "tear down" : "set up", action_to_str(action));
>> return ret;
>> }
>>
>> The pattern's first ... matches a "shortest" path to an error_prepend(),
>> where "shortest" means "does not cross an error_prepend(). Its when
>> clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().
>>
>> There are two such "shortest" paths, one to the first error_prepend() in
>> vfio_set_irq_signaling(), and one to the second. Neither path to the
>> third one is not "shortest": they both cross one of the other two
>> error_prepend().
>>
>> The pattern' s second ... matches a path from a matched error_prepend()
>> to the end of the function. There are two paths. Both cross the third
>> error_prepend(). You need "when any" to make the pattern match anyway.
>>
>> Alright, I think I got it. But now I'm paranoid about ... elsewhere.
>> For instance, here's rule1 with error_propagate_prepend() omitted:
>>
>> // Match scenarios with propagation of local error to errp.
>> @rule1 disable optional_qualifier exists@
>> identifier fn, local_err;
>> symbol errp;
>> @@
>>
>> fn(..., Error **errp, ...)
>> {
>> ...
>> Error *local_err = NULL;
>> ...
>> error_propagate(errp, local_err);
>> ...
>> }
>>
>> The second and third ... won't match anything containing
>> error_propagate(). What if a function has multiple error_propagate() on
>> all paths?
>
> I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...
I'm fine with not touching "strange" patterns. But we do at least in
the example I gave below: we still add ERRP_AUTO_PROPAGATE().
We either avoid that in the Coccinelle script, or we catch it
afterwards. Catching it afterwards should be feasible:
* If we also drop some error propagation from this function: okay.
* If this function call error_append_hint(), error_prepend() or
error_vprepend(): okay.
* Else: unwanted, back out.
Moreover, I'd like us to double-check we really don't want to touch the
things we don't touch. Feels feasible to me, too: after running
Coccinelle, search for unconverted error_append_hint(), error_prepend(),
error_vprepend(), error_propagate_prepend(), error_propagate().
>> Like this one:
>>
>> extern foo(int, Error **);
>> extern bar(int, Error **);
>>
>> void frob(Error **errp)
>> {
>> Error *local_err = NULL;
>> int arg;
>>
>> foo(arg, errp);
>> bar(arg, &local_err);
>> error_propagate(errp, local_err);
>> bar(arg + 1, &local_err);
>> error_propagate(errp, local_err);
>> }
>>
>> This is actually a variation of error.h's "Receive and accumulate
>> multiple errors (first one wins)" code snippet.
>
> ah yes, we can propagate to already filled errp, which just clean local_err.
>
>>
>> The Coccinelle script transforms it like this:
>>
>> void frob(Error **errp)
>> {
>> + ERRP_AUTO_PROPAGATE();
>> Error *local_err = NULL;
>> int arg;
>>
>> The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any),
>> but rule1 does not, and we therefore don't convert any of the
>> error_propagate().
>>
>> The result isn't wrong, just useless.
>>
>> Is this the worst case?
>>
>> Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use
>> "... when any" in the error_propagate() case, only in the other cases.
>> Would that help?
>
> I think not, as it will anyway match functions with error_prepend (and any
> number of following error_propagate calls)...
I'm not sure I understand this sentence.
The aim of my "possible improvement" is to avoid unwanted
ERRP_AUTO_PROPAGATE() in the Coccinelle script. We can instead search
for unwanted ones after the Coccinelle run, and back them out.
>> I think this is the only other rule with "..." matching control flow.
>>
>>>>
>>>>> +//
>>>>> +// Note, "exists" says that we want apply rule even if it matches not on
>>>>> +// all possible control flows (otherwise, it will not match standard pattern
>>>>> +// when error_propagate() call is in if branch).
>>>>
>>>> Learned something new. Example: kvm_set_kvm_shadow_mem().
>>>>
>>>> Spelling it "exists disable optional_qualifier" would avoid giving
>>>> readers the idea we're disabling "exists", but Coccinelle doesn't let
>>>> us. Oh well.
>>>>
>>>>> +@ disable optional_qualifier exists@
>>>>> +identifier fn, local_err, errp;
>>>>
>>>> I believe this causes
>>>>
>>>> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
>>>> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>>>>
>>>> Making @errp symbol instead of identifier should fix this.
>>>
>>> Hmm, I didn't see these warnings.. But yes, it should be symbol.
>>>
>>>>
>>>>> +@@
>>>>> +
>>>>> + fn(..., Error **errp, ...)
>>>>> + {
>>>>> ++ ERRP_AUTO_PROPAGATE();
>>>>> + ... when != ERRP_AUTO_PROPAGATE();
>>>>> +(
>>>>> + error_append_hint(errp, ...);
>>>>> +|
>>>>> + error_prepend(errp, ...);
>>>>> +|
>>>>> + error_vprepend(errp, ...);
>>>>> +|
>>>>> + Error *local_err = NULL;
>>>>> + ...
>>>>> +(
>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>> +|
>>>>> + error_propagate(errp, local_err);
>>>>> +)
>>>>> +)
>>>>> + ... when any
>>>>> + }
>>>>> +
>>>>> +
>>>>> +// Match scenarios with propagation of local error to errp.
>>>>> +@rule1 disable optional_qualifier exists@
>>>>> +identifier fn, local_err;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(..., Error **errp, ...)
>>>>> + {
>>>>> + ...
>>>>> + Error *local_err = NULL;
>>>>> + ...
>>>>> +(
>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>> +|
>>>>> + error_propagate(errp, local_err);
>>>>> +)
>>>>
>>>> Indentation off by one.
>>>>
>>>>> + ...
>>>>> + }
>>>>> +
>>>>> +// Convert special case with goto in separate.
>>>>
>>>> s/in separate/separately/
>>>>
>>>>> +// We can probably merge this into the following hunk with help of ( | )
>>>>> +// operator, but it significantly reduce performance on block.c parsing (or it
>>>>
>>>> s/reduce/reduces/
>>>>
>>>>> +// hangs, I don't know)
>>>>
>>>> Sounds like you tried to merge this into the following hunk, but then
>>>> spatch took so long on block.c that you killed it. Correct?
>>>
>>> Yes.
>>
>> I'd say something like "I tried merging this into the following rule the
>> obvious way, but it made Coccinelle hang on block.c."
>>
>>>>
>>>>> +//
>>>>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>>>>> +// things later after all transformations (the rule will be the same, just
>>>>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>>>>
>>>> Weird, but not worth further investigation.
>>>
>>> It partially match to the idea which I saw somewhere in coccinelle documentation,
>>> that coccinelle converts correct C code to correct C code. "out: }" is an example
>>> of incorrect, impossible code flow, and coccinelle can't work with it... But it's
>>> just a thought.
>>>
>>>>
>>>>> +@@
>>>>> +identifier rule1.fn, rule1.local_err, out;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- goto out;
>>>>> ++ return;
>>>>> + ...>
>>>>> +- out:
>>>>> +- error_propagate(errp, local_err);
>>>>
>>>> You neglect to match error_propagate_prepend(). Okay, because (1) that
>>>> pattern doesn't occur in the tree right now, and (2) if it gets added,
>>>> gcc will complain.
>>>
>>> No, because it should not removed. error_propagate_prepend should be converted
>>> to prepend, not removed. So, corresponding gotos should not be removed as well.
>>
>> You're right.
>>
>>>>
>>>>> + }
>>>>> +
>>>>> +// Convert most of local_err related staff.
>>>>
>>>> s/staff/stuff/
>>>>
>>>>> +//
>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>> +// and local_err name. We may match something not related to the pattern
>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>> +// in different blocks inside one function, and in one block follow the
>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>> +// functions with the same name (for different configurations).
>>>>
>>>> Context: rule1 matches functions that have all three of
>>>>
>>>> * an Error **errp parameter
>>>>
>>>> * an Error *local_err = NULL variable declaration
>>>>
>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>> local_err, ...) expression, where @errp is the parameter and
>>>> @local_err is the variable.
>>>>
>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>
>>>> 1. This rule can match functions rule1 does not match if there is
>>>> another function with the same name that rule1 does match.
>>>>
>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>> parts of that function use a different @errp or @local_err.
>>>>
>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>> this one. Correct?
>>>
>>> Yes.
>>
>> Thanks!
>>
>>>>
>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>> don't know it.
>>>> Can we make Coccinelle at least warn us when it converts
>>>> multiple functions with the same name? What about this:
>>>>
>>>> @initialize:python@
>>>> @@
>>>> fnprev = {}
>>>>
>>>> def pr(fn, p):
>>>> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>>>>
>>>> @r@
>>>> identifier rule1.fn;
>>>> position p;
>>>> @@
>>>> fn(...)@p
>>>> {
>>>> ...
>>>> }
>>>> @script:python@
>>>> fn << rule1.fn;
>>>> p << r.p;
>>>> @@
>>>> if fn not in fnprev:
>>>> fnprev[fn] = p
>>>> else:
>>>> if fnprev[fn]:
>>>
>>> hmm, the condition can't be false
>>>
>>>> pr(fn, fnprev[fn])
>>>> fnprev[fn] = None
>>>> pr(fn, p)
>>>
>>> and we'll miss next duplication..
>>
>> The idea is
>>
>> first instance of fn:
>> fn not in fnprev
>> fnprev[fn] = position of instance
>> don't print
>> second instance:
>> fnprev[fn] is the position of the first instance
>> print first two instances
>> subsequent instances: fnprev[fn] is None
>> print this instance
>>
>> I might have screwed up the coding, of course :)
>>
>>> But I like the idea.
>>>
>>>>
>>>> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
>>>> the number of functions with the same name we touch. If it's more than
>>>> one, we print.
>>>>
>>>> Reports about a dozen function names for the whole tree in my testing.
>>>> Inspecting the changes to them manually is feasible. None of them are
>>>> in files touched by this series.
>>>>
>>>> The line printed for the first match is pretty useless for me: it points
>>>> to a Coccinelle temporary file *shrug*.
>>>>
>>>> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
>>>> sure hope we don't do that. Multiple @local_err variables... hmm.
>>>> Perhaps we could again concoct some script rules to lead us to spots to
>>>> check manually. See below for my attempt.
>>>>
>>>> What's the worst that could happen if we blindly converted such code?
>>>> The answer to that question tells us how hard to work on finding and
>>>> checking these guys.
>>>>
>>>>> +//
>>>>> +// Note also that errp-cleaning functions
>>>>> +// error_free_errp
>>>>> +// error_report_errp
>>>>> +// error_reportf_errp
>>>>> +// warn_report_errp
>>>>> +// warn_reportf_errp
>>>>> +// are not yet implemented. They must call corresponding Error* - freeing
>>>>> +// function and then set *errp to NULL, to avoid further propagation to
>>>>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>>>>> +// For example, error_free_errp may look like this:
>>>>> +//
>>>>> +// void error_free_errp(Error **errp)
>>>>> +// {
>>>>> +// error_free(*errp);
>>>>> +// *errp = NULL;
>>>>> +// }
>>>>> +@ exists@
>>>>> +identifier rule1.fn, rule1.local_err;
>>>>> +expression list args;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +(
>>>>
>>>> Each of the following patterns applies anywhere in the function.
>>>>
>>>> First pattern: delete @local_err
>>>>
>>>>> +- Error *local_err = NULL;
>>>>
>>>> Common case: occurs just once, not nested. Anything else is suspicious.
>>>>
>>>> Both can be detected in the resulting patches with a bit of AWK
>>>> wizardry:
>>>>
>>>> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
>>>> nested
>>>> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
>>>> nested
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>> nested
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>> more than one
>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>>
>>>> Oh.
>>>>
>>>> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>>>>
>>>> xen_block_device_destroy() has multiple Error *local_err.
>>>>
>>>> In both cases, manual review is required to ensure the conversion is
>>>> okay. I believe it is.
>>>>
>>>> Note that the AWK script relies on diff showing the function name in @@
>>>> lines, which doesn't always work due to our coding style.
>>>>
>>>> For the whole tree, I get some 30 spots. Feasible.
>>>>
>>>>> +|
>>>>
>>>> Second pattern: clear @errp after freeing it
>>>>
>>>>> +
>>>>> +// Convert error clearing functions
>>>>
>>>> Suggest: Ensure @local_err is cleared on free
>>>>
>>>>> +(
>>>>> +- error_free(local_err);
>>>>> ++ error_free_errp(errp);
>>>>> +|
>>>>> +- error_report_err(local_err);
>>>>> ++ error_report_errp(errp);
>>>>> +|
>>>>> +- error_reportf_err(local_err, args);
>>>>> ++ error_reportf_errp(errp, args);
>>>>> +|
>>>>> +- warn_report_err(local_err);
>>>>> ++ warn_report_errp(errp);
>>>>> +|
>>>>> +- warn_reportf_err(local_err, args);
>>>>> ++ warn_reportf_errp(errp, args);
>>>>> +)
>>>>
>>>> As you mention above, these guys don't exist, yet. Builds anyway,
>>>> because this part of the rule is not used in this patch series. You
>>>> don't want to omit it, because then the script becomes unsafe to use.
>>>>
>>>> We could also open-code:
>>>>
>>>> // Convert error clearing functions
>>>> (
>>>> - error_free(local_err);
>>>> + error_free(*errp);
>>>> + *errp = NULL;
>>>> |
>>>> ... and so forth ...
>>>> )
>>>>
>>>> Matter of taste. Whatever is easier to explain in the comments. Since
>>>> you already wrote one...
>>>
>>> I just feel that using helper functions is safer way..
>>>
>>>>
>>>> We talked about extending this series slightly so these guys are used.
>>>> I may still look into that.
>>>>
>>>>> +?- local_err = NULL;
>>>>> +
>>>>
>>>> The new helpers clear @local_err. Assignment now redundant, delete.
>>>> Okay.
>>>>
>>>>> +|
>>>>
>>>> Third and fourth pattern: delete error_propagate()
>>>>
>>>>> +- error_propagate_prepend(errp, local_err, args);
>>>>> ++ error_prepend(errp, args);
>>>>> +|
>>>>> +- error_propagate(errp, local_err);
>>>>> +|
>>>>
>>>> Fifth pattern: use @errp directly
>>>>
>>>>> +- &local_err
>>>>> ++ errp
>>>>> +)
>>>>> + ...>
>>>>> + }
>>>>> +
>>>>> +// Convert remaining local_err usage. It should be different kinds of error
>>>>> +// checking in if operators. We can't merge this into previous hunk, as this
>>>>
>>>> In if conditionals, I suppose. It's the case for this patch. If I
>>>> apply the script to the whole tree, the rule gets also applied in other
>>>> contexts. The sentence might mislead as much as it helps. Keep it or
>>>> delete it?
>>>
>>> Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
>>
>> "Need to be careful" means "needs careful manual review", which I
>> believe is not feasible; see "Preface to my review of this script"
>> above.
>>
>> But do we really need to be careful here?
>>
>> This rule should apply only where we added ERRP_AUTO_PROPAGATE().
>>
>> Except when rule chaining via function name fails us, but we plan to
>> detect that and review manually, so let's ignore this issue here.
>>
>> Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling
>> replacement of @local_err by @errp is its whole point.
>>
>> What exactly do we need to be careful about?
>
> Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."
Something like this, perhaps?
// Convert remaining local_err usage, typically error checking.
// We can't merge this into previous hunk, as this conflicts with other
// substitutions in it (at least with "- local_err = NULL").
>>>>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>>>>> +@@
>>>>> +identifier rule1.fn, rule1.local_err;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- local_err
>>>>> ++ *errp
>>>>> + ...>
>>>>> + }
>>>>> +
>>>>> +// Always use the same patter for checking error
>>>>
>>>> s/patter/pattern/
>>>>
>>>>> +@@
>>>>> +identifier rule1.fn;
>>>>> +symbol errp;
>>>>> +@@
>>>>> +
>>>>> + fn(...)
>>>>> + {
>>>>> + <...
>>>>> +- *errp != NULL
>>>>> ++ *errp
>>>>> + ...>
>>>>> + }
>>>>
>>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-11 9:04 ` [Xen-devel] " Markus Armbruster
@ 2020-03-11 9:16 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 9:16 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Laszlo Ersek, Christian Schoenebeck, Michael Roth, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
11.03.2020 12:04, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 10.03.2020 18:47, Markus Armbruster wrote:
>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>
>>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>>> Suggest
>>>>>
>>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>>
>>>>> or
>>>>>
>>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>>
>>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>>
>>>>>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>>>>>> does corresponding changes in code (look for details in
>>>>>> include/qapi/error.h)
>>>>>>
>>>>>> Usage example:
>>>>>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>>>
>>>>> Suggest FILES... instead of a specific set of files.
>>>>>
>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>>> ---
>>>>>>
>>>>>> Cc: Eric Blake <eblake@redhat.com>
>>>>>> Cc: Kevin Wolf <kwolf@redhat.com>
>>>>>> Cc: Max Reitz <mreitz@redhat.com>
>>>>>> Cc: Greg Kurz <groug@kaod.org>
>>>>>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>>>>>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>>>>>> Cc: Anthony Perard <anthony.perard@citrix.com>
>>>>>> Cc: Paul Durrant <paul@xen.org>
>>>>>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>>>>>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>>>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>>>>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>>>>>> Cc: Markus Armbruster <armbru@redhat.com>
>>>>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>>>>> Cc: qemu-block@nongnu.org
>>>>>> Cc: qemu-devel@nongnu.org
>>>>>> Cc: xen-devel@lists.xenproject.org
>>>>>>
>>>>>> include/qapi/error.h | 3 +
>>>>>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>>>>>> 2 files changed, 234 insertions(+)
>>>>>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>>>>
>>>>>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>>>>>> index bb9bcf02fb..fbfc6f1c0b 100644
>>>>>> --- a/include/qapi/error.h
>>>>>> +++ b/include/qapi/error.h
>>>>>> @@ -211,6 +211,9 @@
>>>>>> * }
>>>>>> * ...
>>>>>> * }
>>>>>> + *
>>>>>> + * For mass conversion use script
>>>>>
>>>>> mass-conversion (we're not converting mass, we're converting en masse)
>>>>>
>>>>>> + * scripts/coccinelle/auto-propagated-errp.cocci
>>>>>> */
>>>>>> #ifndef ERROR_H
>>>>>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>>> new file mode 100644
>>>>>> index 0000000000..bff274bd6d
>>>>>> --- /dev/null
>>>>>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>>
>>>>> Preface to my review of this script: may aim isn't to make it
>>>>> bullet-proof. I want to (1) make it good enough (explained in a
>>>>> jiffie), and (2) automatically identify the spots where it still isn't
>>>>> obviously safe for manual review.
>>>>>
>>>>> The latter may involve additional scripting. That's okay.
>>>>>
>>>>> The script is good enough when the number of possibly unsafe spots is
>>>>> low enough for careful manual review.
>>>>>
>>>>> When I ask for improvements that, in your opinion, go beyond "good
>>>>> enough", please push back. I'm sure we can work it out together.
>>>>>
>>>>>> @@ -0,0 +1,231 @@
>>>>>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>>>>>> +//
>>>>>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>>>>>> +//
>>>>>> +// This program is free software; you can redistribute it and/or modify
>>>>>> +// it under the terms of the GNU General Public License as published by
>>>>>> +// the Free Software Foundation; either version 2 of the License, or
>>>>>> +// (at your option) any later version.
>>>>>> +//
>>>>>> +// This program is distributed in the hope that it will be useful,
>>>>>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>>>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>>>> +// GNU General Public License for more details.
>>>>>> +//
>>>>>> +// You should have received a copy of the GNU General Public License
>>>>>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>>>> +//
>>>>>> +// Usage example:
>>>>>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>>>>>
>>>>> You have --max-width 80 here, but not in the commit message. Default
>>>>> seems to be 78. Any particular reason to change it to 80?
>>>>
>>>> Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
>>>> So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
>>>> Still, may be I'm wrong, we can check it.
>>>
>>> If you can point to an example where --max-width helps, keep it, and
>>> update the commit message to match. Else, drop it.
>>>
>>>>>
>>>>>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>
> As our discussion shows, this script is somewhat hard to understand.
> That's okay, it solves a somewhat thorny problem, and I don't have
> better ideas. But let me state the intended transformations once more,
> so I don't get lost in the details.
>
> Motivation:
>
> 1. Make error propagation less error-prone and improve stack backtraces
>
> The "receive error in &local_error, propagate to @errp" pattern is
> tedious, error-prone, and produces unhelpful stack backtraces with
> &error_abort.
>
> ERRP_AUTO_PROPAGATE() removes manual propagation. It additionally
> gives us the stack backtraces we want.
>
> 2. Improve error messages with &error_fatal
>
> Passing @errp to error_append_hint(), error_prepend() or
> error_vprepend() is useless when @errp is &error_fatal.
>
> ERRP_AUTO_PROPAGATE() fixes this by delaying &error_fatal handling
> until the automatic propagation.
>
> The intended transformation has three parts:
>
> * Replace declaration of @local_err by ERRP_AUTO_PROPAGATE() where
> needed for 1 or 2.
actually, we just add macro invocation. local_err is removed together with
other conversions.
>
> It's needed when we also drop some error propagation from the function
> (motivation 1), or the function calls error_append_hint(),
> error_prepend() or error_vprepend() (motivation 2). A function could
> do both. I'll refer back to this below.
>
> * Drop error_propagate(errp, local_err)
>
> Special case: error_propagate_prepend(errp, local_err, ...) becomes
> error_prepend(errp, ...).
>
> Only correct if there is no other such error_propagate() /
> error_propagate_prepend() on any path from the here to return.
>
> * Replace remaining use of @local_err by *errp
>
>>>>>> +
>>>>>> +// Switch unusual (Error **) parameter names to errp
>>>>>
>>>>> Let's drop the parenthesis around Error **
>>>>>
>>>>>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>>>>>
>>>>> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
>>>>> make the fact we're messing with @errp more obvious. Too late; I
>>>>> shouldn't rock the boat that much now.
>>>>>
>>>>>> +//
>>>>>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>>>>>> +// parameter.
>>>>>> +//
>>>>>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>>>>>> +// non generic semantics and may have unusual Error ** argument name for purpose
>>>>>
>>>>> non-generic
>>>>>
>>>>> for a purpose
>>>>>
>>>>> Wrap comment lines around column 70, please. It's easier to read.
>>>>>
>>>>> Maybe
>>>>>
>>>>> // Skip functions with "assert(_errp && *_errp)" statement, because that
>>>>> // signals unusual semantics, and the parameter name may well serve a
>>>>> // purpose.
>>>>
>>>> Sounds good.
>>>>
>>>>>
>>>>>> +// (like nbd_iter_channel_error()).
>>>>>> +//
>>>>>> +// Skip util/error.c to not touch, for example, error_propagate and
>>>>>> +// error_propagate_prepend().
>>>>>
>>>>> error_propagate()
>>>>>
>>>>> I much appreciate your meticulous explanation of what you skip and why.
>>>>>
>>>>>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>>>>>> +identifier fn;
>>>>>> +identifier _errp != errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(...,
>>>>>> +- Error **_errp
>>>>>> ++ Error **errp
>>>>>> + ,...)
>>>>>> + {
>>>>>> +(
>>>>>> + ... when != assert(_errp && *_errp)
>>>>>> +&
>>>>>> + <...
>>>>>> +- _errp
>>>>>> ++ errp
>>>>>> + ...>
>>>>>> +)
>>>>>> + }
>>>>>
>>>>> This rule is required to make the actual transformations (below) work
>>>>> even for parameters with names other than @errp. I believe it's not
>>>>> used in this series. In fact, I can't see a use for it in the entire
>>>>> tree right now. Okay anyway.
>>>>>
>>>>>> +
>>>>>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>>>>>> +//
>>>>>> +// Note, that without "when any" final "..." may not want to mach something
>>>>>
>>>>> s/final "..." may not mach/the final "..." does not match/
>>>>>
>>>>>> +// matched by previous pattern, i.e. the rule will not match double
>>>>>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>>>>>
>>>>> Can't say I fully understand Coccinelle there. I figure you came to
>>>>> this knowledge the hard way.
>>>>
>>>> It's follows from smpl grammar document:
>>>>
>>>> "Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
>>>> ...
>>>> "_when any_ removes the aforementioned constraint that “...” matches the shortest path"
>>>
>>> Let me think that through.
>>>
>>> The pattern with the cases other than error_prepend() omitted:
>>>
>>> fn(..., Error **errp, ...)
>>> {
>>> + ERRP_AUTO_PROPAGATE();
>>> ... when != ERRP_AUTO_PROPAGATE();
>>> error_prepend(errp, ...);
>>> ... when any
>>> }
>>>
>>> Tail of vfio_set_irq_signaling():
>>>
>>> name = index_to_str(vbasedev, index);
>>> if (name) {
>>> error_prepend(errp, "%s-%d: ", name, subindex);
>>> } else {
>>> error_prepend(errp, "index %d-%d: ", index, subindex);
>>> }
>>> error_prepend(errp,
>>> "Failed to %s %s eventfd signaling for interrupt ",
>>> fd < 0 ? "tear down" : "set up", action_to_str(action));
>>> return ret;
>>> }
>>>
>>> The pattern's first ... matches a "shortest" path to an error_prepend(),
>>> where "shortest" means "does not cross an error_prepend(). Its when
>>> clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().
>>>
>>> There are two such "shortest" paths, one to the first error_prepend() in
>>> vfio_set_irq_signaling(), and one to the second. Neither path to the
>>> third one is not "shortest": they both cross one of the other two
>>> error_prepend().
>>>
>>> The pattern' s second ... matches a path from a matched error_prepend()
>>> to the end of the function. There are two paths. Both cross the third
>>> error_prepend(). You need "when any" to make the pattern match anyway.
>>>
>>> Alright, I think I got it. But now I'm paranoid about ... elsewhere.
>>> For instance, here's rule1 with error_propagate_prepend() omitted:
>>>
>>> // Match scenarios with propagation of local error to errp.
>>> @rule1 disable optional_qualifier exists@
>>> identifier fn, local_err;
>>> symbol errp;
>>> @@
>>>
>>> fn(..., Error **errp, ...)
>>> {
>>> ...
>>> Error *local_err = NULL;
>>> ...
>>> error_propagate(errp, local_err);
>>> ...
>>> }
>>>
>>> The second and third ... won't match anything containing
>>> error_propagate(). What if a function has multiple error_propagate() on
>>> all paths?
>>
>> I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...
>
> I'm fine with not touching "strange" patterns. But we do at least in
> the example I gave below: we still add ERRP_AUTO_PROPAGATE().
>
> We either avoid that in the Coccinelle script,
I think avoiding is best thing, and it seems possible as you proposed.
> or we catch it
> afterwards. Catching it afterwards should be feasible:
>
> * If we also drop some error propagation from this function: okay.
>
> * If this function call error_append_hint(), error_prepend() or
> error_vprepend(): okay.
>
> * Else: unwanted, back out.
>
> Moreover, I'd like us to double-check we really don't want to touch the
> things we don't touch. Feels feasible to me, too: after running
> Coccinelle, search for unconverted error_append_hint(), error_prepend(),
> error_vprepend(), error_propagate_prepend(), error_propagate().
>
>>> Like this one:
>>>
>>> extern foo(int, Error **);
>>> extern bar(int, Error **);
>>>
>>> void frob(Error **errp)
>>> {
>>> Error *local_err = NULL;
>>> int arg;
>>>
>>> foo(arg, errp);
>>> bar(arg, &local_err);
>>> error_propagate(errp, local_err);
>>> bar(arg + 1, &local_err);
>>> error_propagate(errp, local_err);
>>> }
>>>
>>> This is actually a variation of error.h's "Receive and accumulate
>>> multiple errors (first one wins)" code snippet.
>>
>> ah yes, we can propagate to already filled errp, which just clean local_err.
>>
>>>
>>> The Coccinelle script transforms it like this:
>>>
>>> void frob(Error **errp)
>>> {
>>> + ERRP_AUTO_PROPAGATE();
>>> Error *local_err = NULL;
>>> int arg;
>>>
>>> The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any),
>>> but rule1 does not, and we therefore don't convert any of the
>>> error_propagate().
>>>
>>> The result isn't wrong, just useless.
>>>
>>> Is this the worst case?
>>>
>>> Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use
>>> "... when any" in the error_propagate() case, only in the other cases.
>>> Would that help?
>>
>> I think not, as it will anyway match functions with error_prepend (and any
>> number of following error_propagate calls)...
>
> I'm not sure I understand this sentence.
>
> The aim of my "possible improvement" is to avoid unwanted
> ERRP_AUTO_PROPAGATE() in the Coccinelle script. We can instead search
> for unwanted ones after the Coccinelle run, and back them out.
You are right.
>
>>> I think this is the only other rule with "..." matching control flow.
>>>
>>>>>
>>>>>> +//
>>>>>> +// Note, "exists" says that we want apply rule even if it matches not on
>>>>>> +// all possible control flows (otherwise, it will not match standard pattern
>>>>>> +// when error_propagate() call is in if branch).
>>>>>
>>>>> Learned something new. Example: kvm_set_kvm_shadow_mem().
>>>>>
>>>>> Spelling it "exists disable optional_qualifier" would avoid giving
>>>>> readers the idea we're disabling "exists", but Coccinelle doesn't let
>>>>> us. Oh well.
>>>>>
>>>>>> +@ disable optional_qualifier exists@
>>>>>> +identifier fn, local_err, errp;
>>>>>
>>>>> I believe this causes
>>>>>
>>>>> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>>>>>
>>>>> Making @errp symbol instead of identifier should fix this.
>>>>
>>>> Hmm, I didn't see these warnings.. But yes, it should be symbol.
>>>>
>>>>>
>>>>>> +@@
>>>>>> +
>>>>>> + fn(..., Error **errp, ...)
>>>>>> + {
>>>>>> ++ ERRP_AUTO_PROPAGATE();
>>>>>> + ... when != ERRP_AUTO_PROPAGATE();
>>>>>> +(
>>>>>> + error_append_hint(errp, ...);
>>>>>> +|
>>>>>> + error_prepend(errp, ...);
>>>>>> +|
>>>>>> + error_vprepend(errp, ...);
>>>>>> +|
>>>>>> + Error *local_err = NULL;
>>>>>> + ...
>>>>>> +(
>>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>>> +|
>>>>>> + error_propagate(errp, local_err);
>>>>>> +)
>>>>>> +)
>>>>>> + ... when any
>>>>>> + }
>>>>>> +
>>>>>> +
>>>>>> +// Match scenarios with propagation of local error to errp.
>>>>>> +@rule1 disable optional_qualifier exists@
>>>>>> +identifier fn, local_err;
>>>>>> +symbol errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(..., Error **errp, ...)
>>>>>> + {
>>>>>> + ...
>>>>>> + Error *local_err = NULL;
>>>>>> + ...
>>>>>> +(
>>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>>> +|
>>>>>> + error_propagate(errp, local_err);
>>>>>> +)
>>>>>
>>>>> Indentation off by one.
>>>>>
>>>>>> + ...
>>>>>> + }
>>>>>> +
>>>>>> +// Convert special case with goto in separate.
>>>>>
>>>>> s/in separate/separately/
>>>>>
>>>>>> +// We can probably merge this into the following hunk with help of ( | )
>>>>>> +// operator, but it significantly reduce performance on block.c parsing (or it
>>>>>
>>>>> s/reduce/reduces/
>>>>>
>>>>>> +// hangs, I don't know)
>>>>>
>>>>> Sounds like you tried to merge this into the following hunk, but then
>>>>> spatch took so long on block.c that you killed it. Correct?
>>>>
>>>> Yes.
>>>
>>> I'd say something like "I tried merging this into the following rule the
>>> obvious way, but it made Coccinelle hang on block.c."
>>>
>>>>>
>>>>>> +//
>>>>>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>>>>>> +// things later after all transformations (the rule will be the same, just
>>>>>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>>>>>
>>>>> Weird, but not worth further investigation.
>>>>
>>>> It partially match to the idea which I saw somewhere in coccinelle documentation,
>>>> that coccinelle converts correct C code to correct C code. "out: }" is an example
>>>> of incorrect, impossible code flow, and coccinelle can't work with it... But it's
>>>> just a thought.
>>>>
>>>>>
>>>>>> +@@
>>>>>> +identifier rule1.fn, rule1.local_err, out;
>>>>>> +symbol errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(...)
>>>>>> + {
>>>>>> + <...
>>>>>> +- goto out;
>>>>>> ++ return;
>>>>>> + ...>
>>>>>> +- out:
>>>>>> +- error_propagate(errp, local_err);
>>>>>
>>>>> You neglect to match error_propagate_prepend(). Okay, because (1) that
>>>>> pattern doesn't occur in the tree right now, and (2) if it gets added,
>>>>> gcc will complain.
>>>>
>>>> No, because it should not removed. error_propagate_prepend should be converted
>>>> to prepend, not removed. So, corresponding gotos should not be removed as well.
>>>
>>> You're right.
>>>
>>>>>
>>>>>> + }
>>>>>> +
>>>>>> +// Convert most of local_err related staff.
>>>>>
>>>>> s/staff/stuff/
>>>>>
>>>>>> +//
>>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>>> +// and local_err name. We may match something not related to the pattern
>>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>>> +// in different blocks inside one function, and in one block follow the
>>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>>> +// functions with the same name (for different configurations).
>>>>>
>>>>> Context: rule1 matches functions that have all three of
>>>>>
>>>>> * an Error **errp parameter
>>>>>
>>>>> * an Error *local_err = NULL variable declaration
>>>>>
>>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>>> local_err, ...) expression, where @errp is the parameter and
>>>>> @local_err is the variable.
>>>>>
>>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>>
>>>>> 1. This rule can match functions rule1 does not match if there is
>>>>> another function with the same name that rule1 does match.
>>>>>
>>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>>> parts of that function use a different @errp or @local_err.
>>>>>
>>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>>> this one. Correct?
>>>>
>>>> Yes.
>>>
>>> Thanks!
>>>
>>>>>
>>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>>> don't know it.
>>>>> Can we make Coccinelle at least warn us when it converts
>>>>> multiple functions with the same name? What about this:
>>>>>
>>>>> @initialize:python@
>>>>> @@
>>>>> fnprev = {}
>>>>>
>>>>> def pr(fn, p):
>>>>> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>>>>>
>>>>> @r@
>>>>> identifier rule1.fn;
>>>>> position p;
>>>>> @@
>>>>> fn(...)@p
>>>>> {
>>>>> ...
>>>>> }
>>>>> @script:python@
>>>>> fn << rule1.fn;
>>>>> p << r.p;
>>>>> @@
>>>>> if fn not in fnprev:
>>>>> fnprev[fn] = p
>>>>> else:
>>>>> if fnprev[fn]:
>>>>
>>>> hmm, the condition can't be false
>>>>
>>>>> pr(fn, fnprev[fn])
>>>>> fnprev[fn] = None
>>>>> pr(fn, p)
>>>>
>>>> and we'll miss next duplication..
>>>
>>> The idea is
>>>
>>> first instance of fn:
>>> fn not in fnprev
>>> fnprev[fn] = position of instance
>>> don't print
>>> second instance:
>>> fnprev[fn] is the position of the first instance
>>> print first two instances
>>> subsequent instances: fnprev[fn] is None
>>> print this instance
>>>
>>> I might have screwed up the coding, of course :)
>>>
>>>> But I like the idea.
>>>>
>>>>>
>>>>> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
>>>>> the number of functions with the same name we touch. If it's more than
>>>>> one, we print.
>>>>>
>>>>> Reports about a dozen function names for the whole tree in my testing.
>>>>> Inspecting the changes to them manually is feasible. None of them are
>>>>> in files touched by this series.
>>>>>
>>>>> The line printed for the first match is pretty useless for me: it points
>>>>> to a Coccinelle temporary file *shrug*.
>>>>>
>>>>> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
>>>>> sure hope we don't do that. Multiple @local_err variables... hmm.
>>>>> Perhaps we could again concoct some script rules to lead us to spots to
>>>>> check manually. See below for my attempt.
>>>>>
>>>>> What's the worst that could happen if we blindly converted such code?
>>>>> The answer to that question tells us how hard to work on finding and
>>>>> checking these guys.
>>>>>
>>>>>> +//
>>>>>> +// Note also that errp-cleaning functions
>>>>>> +// error_free_errp
>>>>>> +// error_report_errp
>>>>>> +// error_reportf_errp
>>>>>> +// warn_report_errp
>>>>>> +// warn_reportf_errp
>>>>>> +// are not yet implemented. They must call corresponding Error* - freeing
>>>>>> +// function and then set *errp to NULL, to avoid further propagation to
>>>>>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>>>>>> +// For example, error_free_errp may look like this:
>>>>>> +//
>>>>>> +// void error_free_errp(Error **errp)
>>>>>> +// {
>>>>>> +// error_free(*errp);
>>>>>> +// *errp = NULL;
>>>>>> +// }
>>>>>> +@ exists@
>>>>>> +identifier rule1.fn, rule1.local_err;
>>>>>> +expression list args;
>>>>>> +symbol errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(...)
>>>>>> + {
>>>>>> + <...
>>>>>> +(
>>>>>
>>>>> Each of the following patterns applies anywhere in the function.
>>>>>
>>>>> First pattern: delete @local_err
>>>>>
>>>>>> +- Error *local_err = NULL;
>>>>>
>>>>> Common case: occurs just once, not nested. Anything else is suspicious.
>>>>>
>>>>> Both can be detected in the resulting patches with a bit of AWK
>>>>> wizardry:
>>>>>
>>>>> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
>>>>> nested
>>>>> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
>>>>> nested
>>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>>> nested
>>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>>> more than one
>>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>>>
>>>>> Oh.
>>>>>
>>>>> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>>>>>
>>>>> xen_block_device_destroy() has multiple Error *local_err.
>>>>>
>>>>> In both cases, manual review is required to ensure the conversion is
>>>>> okay. I believe it is.
>>>>>
>>>>> Note that the AWK script relies on diff showing the function name in @@
>>>>> lines, which doesn't always work due to our coding style.
>>>>>
>>>>> For the whole tree, I get some 30 spots. Feasible.
>>>>>
>>>>>> +|
>>>>>
>>>>> Second pattern: clear @errp after freeing it
>>>>>
>>>>>> +
>>>>>> +// Convert error clearing functions
>>>>>
>>>>> Suggest: Ensure @local_err is cleared on free
>>>>>
>>>>>> +(
>>>>>> +- error_free(local_err);
>>>>>> ++ error_free_errp(errp);
>>>>>> +|
>>>>>> +- error_report_err(local_err);
>>>>>> ++ error_report_errp(errp);
>>>>>> +|
>>>>>> +- error_reportf_err(local_err, args);
>>>>>> ++ error_reportf_errp(errp, args);
>>>>>> +|
>>>>>> +- warn_report_err(local_err);
>>>>>> ++ warn_report_errp(errp);
>>>>>> +|
>>>>>> +- warn_reportf_err(local_err, args);
>>>>>> ++ warn_reportf_errp(errp, args);
>>>>>> +)
>>>>>
>>>>> As you mention above, these guys don't exist, yet. Builds anyway,
>>>>> because this part of the rule is not used in this patch series. You
>>>>> don't want to omit it, because then the script becomes unsafe to use.
>>>>>
>>>>> We could also open-code:
>>>>>
>>>>> // Convert error clearing functions
>>>>> (
>>>>> - error_free(local_err);
>>>>> + error_free(*errp);
>>>>> + *errp = NULL;
>>>>> |
>>>>> ... and so forth ...
>>>>> )
>>>>>
>>>>> Matter of taste. Whatever is easier to explain in the comments. Since
>>>>> you already wrote one...
>>>>
>>>> I just feel that using helper functions is safer way..
>>>>
>>>>>
>>>>> We talked about extending this series slightly so these guys are used.
>>>>> I may still look into that.
>>>>>
>>>>>> +?- local_err = NULL;
>>>>>> +
>>>>>
>>>>> The new helpers clear @local_err. Assignment now redundant, delete.
>>>>> Okay.
>>>>>
>>>>>> +|
>>>>>
>>>>> Third and fourth pattern: delete error_propagate()
>>>>>
>>>>>> +- error_propagate_prepend(errp, local_err, args);
>>>>>> ++ error_prepend(errp, args);
>>>>>> +|
>>>>>> +- error_propagate(errp, local_err);
>>>>>> +|
>>>>>
>>>>> Fifth pattern: use @errp directly
>>>>>
>>>>>> +- &local_err
>>>>>> ++ errp
>>>>>> +)
>>>>>> + ...>
>>>>>> + }
>>>>>> +
>>>>>> +// Convert remaining local_err usage. It should be different kinds of error
>>>>>> +// checking in if operators. We can't merge this into previous hunk, as this
>>>>>
>>>>> In if conditionals, I suppose. It's the case for this patch. If I
>>>>> apply the script to the whole tree, the rule gets also applied in other
>>>>> contexts. The sentence might mislead as much as it helps. Keep it or
>>>>> delete it?
>>>>
>>>> Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
>>>
>>> "Need to be careful" means "needs careful manual review", which I
>>> believe is not feasible; see "Preface to my review of this script"
>>> above.
>>>
>>> But do we really need to be careful here?
>>>
>>> This rule should apply only where we added ERRP_AUTO_PROPAGATE().
>>>
>>> Except when rule chaining via function name fails us, but we plan to
>>> detect that and review manually, so let's ignore this issue here.
>>>
>>> Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling
>>> replacement of @local_err by @errp is its whole point.
>>>
>>> What exactly do we need to be careful about?
>>
>> Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."
>
> Something like this, perhaps?
>
> // Convert remaining local_err usage, typically error checking.
> // We can't merge this into previous hunk, as this conflicts with other
> // substitutions in it (at least with "- local_err = NULL").
OK
>
>>>>>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>>>>>> +@@
>>>>>> +identifier rule1.fn, rule1.local_err;
>>>>>> +symbol errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(...)
>>>>>> + {
>>>>>> + <...
>>>>>> +- local_err
>>>>>> ++ *errp
>>>>>> + ...>
>>>>>> + }
>>>>>> +
>>>>>> +// Always use the same patter for checking error
>>>>>
>>>>> s/patter/pattern/
>>>>>
>>>>>> +@@
>>>>>> +identifier rule1.fn;
>>>>>> +symbol errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(...)
>>>>>> + {
>>>>>> + <...
>>>>>> +- *errp != NULL
>>>>>> ++ *errp
>>>>>> + ...>
>>>>>> + }
>>>>>
>>>
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 9:16 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 9:16 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Laszlo Ersek, Christian Schoenebeck, Michael Roth, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
11.03.2020 12:04, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 10.03.2020 18:47, Markus Armbruster wrote:
>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>
>>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>>> Suggest
>>>>>
>>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>>
>>>>> or
>>>>>
>>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>>
>>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>>
>>>>>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>>>>>> does corresponding changes in code (look for details in
>>>>>> include/qapi/error.h)
>>>>>>
>>>>>> Usage example:
>>>>>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>>>>>
>>>>> Suggest FILES... instead of a specific set of files.
>>>>>
>>>>>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>>>>>> ---
>>>>>>
>>>>>> Cc: Eric Blake <eblake@redhat.com>
>>>>>> Cc: Kevin Wolf <kwolf@redhat.com>
>>>>>> Cc: Max Reitz <mreitz@redhat.com>
>>>>>> Cc: Greg Kurz <groug@kaod.org>
>>>>>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>>>>>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>>>>>> Cc: Anthony Perard <anthony.perard@citrix.com>
>>>>>> Cc: Paul Durrant <paul@xen.org>
>>>>>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>>>>>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>>>>>> Cc: Laszlo Ersek <lersek@redhat.com>
>>>>>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>>>>>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>>>>>> Cc: Markus Armbruster <armbru@redhat.com>
>>>>>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>>>>>> Cc: qemu-block@nongnu.org
>>>>>> Cc: qemu-devel@nongnu.org
>>>>>> Cc: xen-devel@lists.xenproject.org
>>>>>>
>>>>>> include/qapi/error.h | 3 +
>>>>>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>>>>>> 2 files changed, 234 insertions(+)
>>>>>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>>>>>
>>>>>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>>>>>> index bb9bcf02fb..fbfc6f1c0b 100644
>>>>>> --- a/include/qapi/error.h
>>>>>> +++ b/include/qapi/error.h
>>>>>> @@ -211,6 +211,9 @@
>>>>>> * }
>>>>>> * ...
>>>>>> * }
>>>>>> + *
>>>>>> + * For mass conversion use script
>>>>>
>>>>> mass-conversion (we're not converting mass, we're converting en masse)
>>>>>
>>>>>> + * scripts/coccinelle/auto-propagated-errp.cocci
>>>>>> */
>>>>>> #ifndef ERROR_H
>>>>>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>>> new file mode 100644
>>>>>> index 0000000000..bff274bd6d
>>>>>> --- /dev/null
>>>>>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>>>>>
>>>>> Preface to my review of this script: may aim isn't to make it
>>>>> bullet-proof. I want to (1) make it good enough (explained in a
>>>>> jiffie), and (2) automatically identify the spots where it still isn't
>>>>> obviously safe for manual review.
>>>>>
>>>>> The latter may involve additional scripting. That's okay.
>>>>>
>>>>> The script is good enough when the number of possibly unsafe spots is
>>>>> low enough for careful manual review.
>>>>>
>>>>> When I ask for improvements that, in your opinion, go beyond "good
>>>>> enough", please push back. I'm sure we can work it out together.
>>>>>
>>>>>> @@ -0,0 +1,231 @@
>>>>>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>>>>>> +//
>>>>>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>>>>>> +//
>>>>>> +// This program is free software; you can redistribute it and/or modify
>>>>>> +// it under the terms of the GNU General Public License as published by
>>>>>> +// the Free Software Foundation; either version 2 of the License, or
>>>>>> +// (at your option) any later version.
>>>>>> +//
>>>>>> +// This program is distributed in the hope that it will be useful,
>>>>>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>>>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>>>> +// GNU General Public License for more details.
>>>>>> +//
>>>>>> +// You should have received a copy of the GNU General Public License
>>>>>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>>>> +//
>>>>>> +// Usage example:
>>>>>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>>>>>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>>>>>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>>>>>
>>>>> You have --max-width 80 here, but not in the commit message. Default
>>>>> seems to be 78. Any particular reason to change it to 80?
>>>>
>>>> Hmm. As I remember, without this parameter, reindenting doesn't work correctly.
>>>> So, I'm OK with "--max-width 78", but I doubt that it will work without a parameter.
>>>> Still, may be I'm wrong, we can check it.
>>>
>>> If you can point to an example where --max-width helps, keep it, and
>>> update the commit message to match. Else, drop it.
>>>
>>>>>
>>>>>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>
> As our discussion shows, this script is somewhat hard to understand.
> That's okay, it solves a somewhat thorny problem, and I don't have
> better ideas. But let me state the intended transformations once more,
> so I don't get lost in the details.
>
> Motivation:
>
> 1. Make error propagation less error-prone and improve stack backtraces
>
> The "receive error in &local_error, propagate to @errp" pattern is
> tedious, error-prone, and produces unhelpful stack backtraces with
> &error_abort.
>
> ERRP_AUTO_PROPAGATE() removes manual propagation. It additionally
> gives us the stack backtraces we want.
>
> 2. Improve error messages with &error_fatal
>
> Passing @errp to error_append_hint(), error_prepend() or
> error_vprepend() is useless when @errp is &error_fatal.
>
> ERRP_AUTO_PROPAGATE() fixes this by delaying &error_fatal handling
> until the automatic propagation.
>
> The intended transformation has three parts:
>
> * Replace declaration of @local_err by ERRP_AUTO_PROPAGATE() where
> needed for 1 or 2.
actually, we just add macro invocation. local_err is removed together with
other conversions.
>
> It's needed when we also drop some error propagation from the function
> (motivation 1), or the function calls error_append_hint(),
> error_prepend() or error_vprepend() (motivation 2). A function could
> do both. I'll refer back to this below.
>
> * Drop error_propagate(errp, local_err)
>
> Special case: error_propagate_prepend(errp, local_err, ...) becomes
> error_prepend(errp, ...).
>
> Only correct if there is no other such error_propagate() /
> error_propagate_prepend() on any path from the here to return.
>
> * Replace remaining use of @local_err by *errp
>
>>>>>> +
>>>>>> +// Switch unusual (Error **) parameter names to errp
>>>>>
>>>>> Let's drop the parenthesis around Error **
>>>>>
>>>>>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>>>>>
>>>>> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
>>>>> make the fact we're messing with @errp more obvious. Too late; I
>>>>> shouldn't rock the boat that much now.
>>>>>
>>>>>> +//
>>>>>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>>>>>> +// parameter.
>>>>>> +//
>>>>>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>>>>>> +// non generic semantics and may have unusual Error ** argument name for purpose
>>>>>
>>>>> non-generic
>>>>>
>>>>> for a purpose
>>>>>
>>>>> Wrap comment lines around column 70, please. It's easier to read.
>>>>>
>>>>> Maybe
>>>>>
>>>>> // Skip functions with "assert(_errp && *_errp)" statement, because that
>>>>> // signals unusual semantics, and the parameter name may well serve a
>>>>> // purpose.
>>>>
>>>> Sounds good.
>>>>
>>>>>
>>>>>> +// (like nbd_iter_channel_error()).
>>>>>> +//
>>>>>> +// Skip util/error.c to not touch, for example, error_propagate and
>>>>>> +// error_propagate_prepend().
>>>>>
>>>>> error_propagate()
>>>>>
>>>>> I much appreciate your meticulous explanation of what you skip and why.
>>>>>
>>>>>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>>>>>> +identifier fn;
>>>>>> +identifier _errp != errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(...,
>>>>>> +- Error **_errp
>>>>>> ++ Error **errp
>>>>>> + ,...)
>>>>>> + {
>>>>>> +(
>>>>>> + ... when != assert(_errp && *_errp)
>>>>>> +&
>>>>>> + <...
>>>>>> +- _errp
>>>>>> ++ errp
>>>>>> + ...>
>>>>>> +)
>>>>>> + }
>>>>>
>>>>> This rule is required to make the actual transformations (below) work
>>>>> even for parameters with names other than @errp. I believe it's not
>>>>> used in this series. In fact, I can't see a use for it in the entire
>>>>> tree right now. Okay anyway.
>>>>>
>>>>>> +
>>>>>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>>>>>> +//
>>>>>> +// Note, that without "when any" final "..." may not want to mach something
>>>>>
>>>>> s/final "..." may not mach/the final "..." does not match/
>>>>>
>>>>>> +// matched by previous pattern, i.e. the rule will not match double
>>>>>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>>>>>
>>>>> Can't say I fully understand Coccinelle there. I figure you came to
>>>>> this knowledge the hard way.
>>>>
>>>> It's follows from smpl grammar document:
>>>>
>>>> "Implicitly, “...” matches the shortest path between something that matches the pattern before the dots (or the beginning of the function, if there is nothing before the dots) and something that matches the pattern after the dots (or the end of the function, if there is nothing after the dots)."
>>>> ...
>>>> "_when any_ removes the aforementioned constraint that “...” matches the shortest path"
>>>
>>> Let me think that through.
>>>
>>> The pattern with the cases other than error_prepend() omitted:
>>>
>>> fn(..., Error **errp, ...)
>>> {
>>> + ERRP_AUTO_PROPAGATE();
>>> ... when != ERRP_AUTO_PROPAGATE();
>>> error_prepend(errp, ...);
>>> ... when any
>>> }
>>>
>>> Tail of vfio_set_irq_signaling():
>>>
>>> name = index_to_str(vbasedev, index);
>>> if (name) {
>>> error_prepend(errp, "%s-%d: ", name, subindex);
>>> } else {
>>> error_prepend(errp, "index %d-%d: ", index, subindex);
>>> }
>>> error_prepend(errp,
>>> "Failed to %s %s eventfd signaling for interrupt ",
>>> fd < 0 ? "tear down" : "set up", action_to_str(action));
>>> return ret;
>>> }
>>>
>>> The pattern's first ... matches a "shortest" path to an error_prepend(),
>>> where "shortest" means "does not cross an error_prepend(). Its when
>>> clause makes us ignore functions that already use ERRP_AUTO_PROPAGATE().
>>>
>>> There are two such "shortest" paths, one to the first error_prepend() in
>>> vfio_set_irq_signaling(), and one to the second. Neither path to the
>>> third one is not "shortest": they both cross one of the other two
>>> error_prepend().
>>>
>>> The pattern' s second ... matches a path from a matched error_prepend()
>>> to the end of the function. There are two paths. Both cross the third
>>> error_prepend(). You need "when any" to make the pattern match anyway.
>>>
>>> Alright, I think I got it. But now I'm paranoid about ... elsewhere.
>>> For instance, here's rule1 with error_propagate_prepend() omitted:
>>>
>>> // Match scenarios with propagation of local error to errp.
>>> @rule1 disable optional_qualifier exists@
>>> identifier fn, local_err;
>>> symbol errp;
>>> @@
>>>
>>> fn(..., Error **errp, ...)
>>> {
>>> ...
>>> Error *local_err = NULL;
>>> ...
>>> error_propagate(errp, local_err);
>>> ...
>>> }
>>>
>>> The second and third ... won't match anything containing
>>> error_propagate(). What if a function has multiple error_propagate() on
>>> all paths?
>>
>> I thought about this, but decided that double error propagation is a strange pattern, and may be better not match it...
>
> I'm fine with not touching "strange" patterns. But we do at least in
> the example I gave below: we still add ERRP_AUTO_PROPAGATE().
>
> We either avoid that in the Coccinelle script,
I think avoiding is best thing, and it seems possible as you proposed.
> or we catch it
> afterwards. Catching it afterwards should be feasible:
>
> * If we also drop some error propagation from this function: okay.
>
> * If this function call error_append_hint(), error_prepend() or
> error_vprepend(): okay.
>
> * Else: unwanted, back out.
>
> Moreover, I'd like us to double-check we really don't want to touch the
> things we don't touch. Feels feasible to me, too: after running
> Coccinelle, search for unconverted error_append_hint(), error_prepend(),
> error_vprepend(), error_propagate_prepend(), error_propagate().
>
>>> Like this one:
>>>
>>> extern foo(int, Error **);
>>> extern bar(int, Error **);
>>>
>>> void frob(Error **errp)
>>> {
>>> Error *local_err = NULL;
>>> int arg;
>>>
>>> foo(arg, errp);
>>> bar(arg, &local_err);
>>> error_propagate(errp, local_err);
>>> bar(arg + 1, &local_err);
>>> error_propagate(errp, local_err);
>>> }
>>>
>>> This is actually a variation of error.h's "Receive and accumulate
>>> multiple errors (first one wins)" code snippet.
>>
>> ah yes, we can propagate to already filled errp, which just clean local_err.
>>
>>>
>>> The Coccinelle script transforms it like this:
>>>
>>> void frob(Error **errp)
>>> {
>>> + ERRP_AUTO_PROPAGATE();
>>> Error *local_err = NULL;
>>> int arg;
>>>
>>> The rule that adds ERRP_AUTO_PROPAGATE() matches (it has ... when any),
>>> but rule1 does not, and we therefore don't convert any of the
>>> error_propagate().
>>>
>>> The result isn't wrong, just useless.
>>>
>>> Is this the worst case?
>>>
>>> Possible improvement to the ERRP_AUTO_PROPAGATE() rule: don't use
>>> "... when any" in the error_propagate() case, only in the other cases.
>>> Would that help?
>>
>> I think not, as it will anyway match functions with error_prepend (and any
>> number of following error_propagate calls)...
>
> I'm not sure I understand this sentence.
>
> The aim of my "possible improvement" is to avoid unwanted
> ERRP_AUTO_PROPAGATE() in the Coccinelle script. We can instead search
> for unwanted ones after the Coccinelle run, and back them out.
You are right.
>
>>> I think this is the only other rule with "..." matching control flow.
>>>
>>>>>
>>>>>> +//
>>>>>> +// Note, "exists" says that we want apply rule even if it matches not on
>>>>>> +// all possible control flows (otherwise, it will not match standard pattern
>>>>>> +// when error_propagate() call is in if branch).
>>>>>
>>>>> Learned something new. Example: kvm_set_kvm_shadow_mem().
>>>>>
>>>>> Spelling it "exists disable optional_qualifier" would avoid giving
>>>>> readers the idea we're disabling "exists", but Coccinelle doesn't let
>>>>> us. Oh well.
>>>>>
>>>>>> +@ disable optional_qualifier exists@
>>>>>> +identifier fn, local_err, errp;
>>>>>
>>>>> I believe this causes
>>>>>
>>>>> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
>>>>> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>>>>>
>>>>> Making @errp symbol instead of identifier should fix this.
>>>>
>>>> Hmm, I didn't see these warnings.. But yes, it should be symbol.
>>>>
>>>>>
>>>>>> +@@
>>>>>> +
>>>>>> + fn(..., Error **errp, ...)
>>>>>> + {
>>>>>> ++ ERRP_AUTO_PROPAGATE();
>>>>>> + ... when != ERRP_AUTO_PROPAGATE();
>>>>>> +(
>>>>>> + error_append_hint(errp, ...);
>>>>>> +|
>>>>>> + error_prepend(errp, ...);
>>>>>> +|
>>>>>> + error_vprepend(errp, ...);
>>>>>> +|
>>>>>> + Error *local_err = NULL;
>>>>>> + ...
>>>>>> +(
>>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>>> +|
>>>>>> + error_propagate(errp, local_err);
>>>>>> +)
>>>>>> +)
>>>>>> + ... when any
>>>>>> + }
>>>>>> +
>>>>>> +
>>>>>> +// Match scenarios with propagation of local error to errp.
>>>>>> +@rule1 disable optional_qualifier exists@
>>>>>> +identifier fn, local_err;
>>>>>> +symbol errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(..., Error **errp, ...)
>>>>>> + {
>>>>>> + ...
>>>>>> + Error *local_err = NULL;
>>>>>> + ...
>>>>>> +(
>>>>>> + error_propagate_prepend(errp, local_err, ...);
>>>>>> +|
>>>>>> + error_propagate(errp, local_err);
>>>>>> +)
>>>>>
>>>>> Indentation off by one.
>>>>>
>>>>>> + ...
>>>>>> + }
>>>>>> +
>>>>>> +// Convert special case with goto in separate.
>>>>>
>>>>> s/in separate/separately/
>>>>>
>>>>>> +// We can probably merge this into the following hunk with help of ( | )
>>>>>> +// operator, but it significantly reduce performance on block.c parsing (or it
>>>>>
>>>>> s/reduce/reduces/
>>>>>
>>>>>> +// hangs, I don't know)
>>>>>
>>>>> Sounds like you tried to merge this into the following hunk, but then
>>>>> spatch took so long on block.c that you killed it. Correct?
>>>>
>>>> Yes.
>>>
>>> I'd say something like "I tried merging this into the following rule the
>>> obvious way, but it made Coccinelle hang on block.c."
>>>
>>>>>
>>>>>> +//
>>>>>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>>>>>> +// things later after all transformations (the rule will be the same, just
>>>>>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>>>>>
>>>>> Weird, but not worth further investigation.
>>>>
>>>> It partially match to the idea which I saw somewhere in coccinelle documentation,
>>>> that coccinelle converts correct C code to correct C code. "out: }" is an example
>>>> of incorrect, impossible code flow, and coccinelle can't work with it... But it's
>>>> just a thought.
>>>>
>>>>>
>>>>>> +@@
>>>>>> +identifier rule1.fn, rule1.local_err, out;
>>>>>> +symbol errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(...)
>>>>>> + {
>>>>>> + <...
>>>>>> +- goto out;
>>>>>> ++ return;
>>>>>> + ...>
>>>>>> +- out:
>>>>>> +- error_propagate(errp, local_err);
>>>>>
>>>>> You neglect to match error_propagate_prepend(). Okay, because (1) that
>>>>> pattern doesn't occur in the tree right now, and (2) if it gets added,
>>>>> gcc will complain.
>>>>
>>>> No, because it should not removed. error_propagate_prepend should be converted
>>>> to prepend, not removed. So, corresponding gotos should not be removed as well.
>>>
>>> You're right.
>>>
>>>>>
>>>>>> + }
>>>>>> +
>>>>>> +// Convert most of local_err related staff.
>>>>>
>>>>> s/staff/stuff/
>>>>>
>>>>>> +//
>>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>>> +// and local_err name. We may match something not related to the pattern
>>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>>> +// in different blocks inside one function, and in one block follow the
>>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>>> +// functions with the same name (for different configurations).
>>>>>
>>>>> Context: rule1 matches functions that have all three of
>>>>>
>>>>> * an Error **errp parameter
>>>>>
>>>>> * an Error *local_err = NULL variable declaration
>>>>>
>>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>>> local_err, ...) expression, where @errp is the parameter and
>>>>> @local_err is the variable.
>>>>>
>>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>>
>>>>> 1. This rule can match functions rule1 does not match if there is
>>>>> another function with the same name that rule1 does match.
>>>>>
>>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>>> parts of that function use a different @errp or @local_err.
>>>>>
>>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>>> this one. Correct?
>>>>
>>>> Yes.
>>>
>>> Thanks!
>>>
>>>>>
>>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>>> don't know it.
>>>>> Can we make Coccinelle at least warn us when it converts
>>>>> multiple functions with the same name? What about this:
>>>>>
>>>>> @initialize:python@
>>>>> @@
>>>>> fnprev = {}
>>>>>
>>>>> def pr(fn, p):
>>>>> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>>>>>
>>>>> @r@
>>>>> identifier rule1.fn;
>>>>> position p;
>>>>> @@
>>>>> fn(...)@p
>>>>> {
>>>>> ...
>>>>> }
>>>>> @script:python@
>>>>> fn << rule1.fn;
>>>>> p << r.p;
>>>>> @@
>>>>> if fn not in fnprev:
>>>>> fnprev[fn] = p
>>>>> else:
>>>>> if fnprev[fn]:
>>>>
>>>> hmm, the condition can't be false
>>>>
>>>>> pr(fn, fnprev[fn])
>>>>> fnprev[fn] = None
>>>>> pr(fn, p)
>>>>
>>>> and we'll miss next duplication..
>>>
>>> The idea is
>>>
>>> first instance of fn:
>>> fn not in fnprev
>>> fnprev[fn] = position of instance
>>> don't print
>>> second instance:
>>> fnprev[fn] is the position of the first instance
>>> print first two instances
>>> subsequent instances: fnprev[fn] is None
>>> print this instance
>>>
>>> I might have screwed up the coding, of course :)
>>>
>>>> But I like the idea.
>>>>
>>>>>
>>>>> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
>>>>> the number of functions with the same name we touch. If it's more than
>>>>> one, we print.
>>>>>
>>>>> Reports about a dozen function names for the whole tree in my testing.
>>>>> Inspecting the changes to them manually is feasible. None of them are
>>>>> in files touched by this series.
>>>>>
>>>>> The line printed for the first match is pretty useless for me: it points
>>>>> to a Coccinelle temporary file *shrug*.
>>>>>
>>>>> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
>>>>> sure hope we don't do that. Multiple @local_err variables... hmm.
>>>>> Perhaps we could again concoct some script rules to lead us to spots to
>>>>> check manually. See below for my attempt.
>>>>>
>>>>> What's the worst that could happen if we blindly converted such code?
>>>>> The answer to that question tells us how hard to work on finding and
>>>>> checking these guys.
>>>>>
>>>>>> +//
>>>>>> +// Note also that errp-cleaning functions
>>>>>> +// error_free_errp
>>>>>> +// error_report_errp
>>>>>> +// error_reportf_errp
>>>>>> +// warn_report_errp
>>>>>> +// warn_reportf_errp
>>>>>> +// are not yet implemented. They must call corresponding Error* - freeing
>>>>>> +// function and then set *errp to NULL, to avoid further propagation to
>>>>>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>>>>>> +// For example, error_free_errp may look like this:
>>>>>> +//
>>>>>> +// void error_free_errp(Error **errp)
>>>>>> +// {
>>>>>> +// error_free(*errp);
>>>>>> +// *errp = NULL;
>>>>>> +// }
>>>>>> +@ exists@
>>>>>> +identifier rule1.fn, rule1.local_err;
>>>>>> +expression list args;
>>>>>> +symbol errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(...)
>>>>>> + {
>>>>>> + <...
>>>>>> +(
>>>>>
>>>>> Each of the following patterns applies anywhere in the function.
>>>>>
>>>>> First pattern: delete @local_err
>>>>>
>>>>>> +- Error *local_err = NULL;
>>>>>
>>>>> Common case: occurs just once, not nested. Anything else is suspicious.
>>>>>
>>>>> Both can be detected in the resulting patches with a bit of AWK
>>>>> wizardry:
>>>>>
>>>>> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
>>>>> nested
>>>>> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
>>>>> nested
>>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>>> nested
>>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>>> more than one
>>>>> static void xen_block_device_destroy(XenBackendInstance *backend,
>>>>>
>>>>> Oh.
>>>>>
>>>>> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>>>>>
>>>>> xen_block_device_destroy() has multiple Error *local_err.
>>>>>
>>>>> In both cases, manual review is required to ensure the conversion is
>>>>> okay. I believe it is.
>>>>>
>>>>> Note that the AWK script relies on diff showing the function name in @@
>>>>> lines, which doesn't always work due to our coding style.
>>>>>
>>>>> For the whole tree, I get some 30 spots. Feasible.
>>>>>
>>>>>> +|
>>>>>
>>>>> Second pattern: clear @errp after freeing it
>>>>>
>>>>>> +
>>>>>> +// Convert error clearing functions
>>>>>
>>>>> Suggest: Ensure @local_err is cleared on free
>>>>>
>>>>>> +(
>>>>>> +- error_free(local_err);
>>>>>> ++ error_free_errp(errp);
>>>>>> +|
>>>>>> +- error_report_err(local_err);
>>>>>> ++ error_report_errp(errp);
>>>>>> +|
>>>>>> +- error_reportf_err(local_err, args);
>>>>>> ++ error_reportf_errp(errp, args);
>>>>>> +|
>>>>>> +- warn_report_err(local_err);
>>>>>> ++ warn_report_errp(errp);
>>>>>> +|
>>>>>> +- warn_reportf_err(local_err, args);
>>>>>> ++ warn_reportf_errp(errp, args);
>>>>>> +)
>>>>>
>>>>> As you mention above, these guys don't exist, yet. Builds anyway,
>>>>> because this part of the rule is not used in this patch series. You
>>>>> don't want to omit it, because then the script becomes unsafe to use.
>>>>>
>>>>> We could also open-code:
>>>>>
>>>>> // Convert error clearing functions
>>>>> (
>>>>> - error_free(local_err);
>>>>> + error_free(*errp);
>>>>> + *errp = NULL;
>>>>> |
>>>>> ... and so forth ...
>>>>> )
>>>>>
>>>>> Matter of taste. Whatever is easier to explain in the comments. Since
>>>>> you already wrote one...
>>>>
>>>> I just feel that using helper functions is safer way..
>>>>
>>>>>
>>>>> We talked about extending this series slightly so these guys are used.
>>>>> I may still look into that.
>>>>>
>>>>>> +?- local_err = NULL;
>>>>>> +
>>>>>
>>>>> The new helpers clear @local_err. Assignment now redundant, delete.
>>>>> Okay.
>>>>>
>>>>>> +|
>>>>>
>>>>> Third and fourth pattern: delete error_propagate()
>>>>>
>>>>>> +- error_propagate_prepend(errp, local_err, args);
>>>>>> ++ error_prepend(errp, args);
>>>>>> +|
>>>>>> +- error_propagate(errp, local_err);
>>>>>> +|
>>>>>
>>>>> Fifth pattern: use @errp directly
>>>>>
>>>>>> +- &local_err
>>>>>> ++ errp
>>>>>> +)
>>>>>> + ...>
>>>>>> + }
>>>>>> +
>>>>>> +// Convert remaining local_err usage. It should be different kinds of error
>>>>>> +// checking in if operators. We can't merge this into previous hunk, as this
>>>>>
>>>>> In if conditionals, I suppose. It's the case for this patch. If I
>>>>> apply the script to the whole tree, the rule gets also applied in other
>>>>> contexts. The sentence might mislead as much as it helps. Keep it or
>>>>> delete it?
>>>>
>>>> Maybe, just be more honest: "It should be ..., but it may be any other pattern, be careful"
>>>
>>> "Need to be careful" means "needs careful manual review", which I
>>> believe is not feasible; see "Preface to my review of this script"
>>> above.
>>>
>>> But do we really need to be careful here?
>>>
>>> This rule should apply only where we added ERRP_AUTO_PROPAGATE().
>>>
>>> Except when rule chaining via function name fails us, but we plan to
>>> detect that and review manually, so let's ignore this issue here.
>>>
>>> Thanks to ERRP_AUTO_PROPAGATE(), @errp is not null. Enabling
>>> replacement of @local_err by @errp is its whole point.
>>>
>>> What exactly do we need to be careful about?
>>
>> Hmm.. About some unpredicted patterns. OK, then "For example, different kinds of .."
>
> Something like this, perhaps?
>
> // Convert remaining local_err usage, typically error checking.
> // We can't merge this into previous hunk, as this conflicts with other
> // substitutions in it (at least with "- local_err = NULL").
OK
>
>>>>>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>>>>>> +@@
>>>>>> +identifier rule1.fn, rule1.local_err;
>>>>>> +symbol errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(...)
>>>>>> + {
>>>>>> + <...
>>>>>> +- local_err
>>>>>> ++ *errp
>>>>>> + ...>
>>>>>> + }
>>>>>> +
>>>>>> +// Always use the same patter for checking error
>>>>>
>>>>> s/patter/pattern/
>>>>>
>>>>>> +@@
>>>>>> +identifier rule1.fn;
>>>>>> +symbol errp;
>>>>>> +@@
>>>>>> +
>>>>>> + fn(...)
>>>>>> + {
>>>>>> + <...
>>>>>> +- *errp != NULL
>>>>>> ++ *errp
>>>>>> + ...>
>>>>>> + }
>>>>>
>>>
>
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-09 9:56 ` [Xen-devel] " Markus Armbruster
@ 2020-03-11 8:29 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 8:29 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
09.03.2020 12:56, Markus Armbruster wrote:
> Suggest
>
> scripts: Coccinelle script to use auto-propagated errp
>
> or
>
> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>> does corresponding changes in code (look for details in
>> include/qapi/error.h)
>>
>> Usage example:
>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>
> Suggest FILES... instead of a specific set of files.
>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>
>> Cc: Eric Blake <eblake@redhat.com>
>> Cc: Kevin Wolf <kwolf@redhat.com>
>> Cc: Max Reitz <mreitz@redhat.com>
>> Cc: Greg Kurz <groug@kaod.org>
>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>> Cc: Anthony Perard <anthony.perard@citrix.com>
>> Cc: Paul Durrant <paul@xen.org>
>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>> Cc: Markus Armbruster <armbru@redhat.com>
>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>> Cc: qemu-block@nongnu.org
>> Cc: qemu-devel@nongnu.org
>> Cc: xen-devel@lists.xenproject.org
>>
>> include/qapi/error.h | 3 +
>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>> 2 files changed, 234 insertions(+)
>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>
>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>> index bb9bcf02fb..fbfc6f1c0b 100644
>> --- a/include/qapi/error.h
>> +++ b/include/qapi/error.h
>> @@ -211,6 +211,9 @@
>> * }
>> * ...
>> * }
>> + *
>> + * For mass conversion use script
>
> mass-conversion (we're not converting mass, we're converting en masse)
>
>> + * scripts/coccinelle/auto-propagated-errp.cocci
>> */
>>
>> #ifndef ERROR_H
>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>> new file mode 100644
>> index 0000000000..bff274bd6d
>> --- /dev/null
>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>
> Preface to my review of this script: may aim isn't to make it
> bullet-proof. I want to (1) make it good enough (explained in a
> jiffie), and (2) automatically identify the spots where it still isn't
> obviously safe for manual review.
>
> The latter may involve additional scripting. That's okay.
>
> The script is good enough when the number of possibly unsafe spots is
> low enough for careful manual review.
>
> When I ask for improvements that, in your opinion, go beyond "good
> enough", please push back. I'm sure we can work it out together.
>
>> @@ -0,0 +1,231 @@
>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>> +//
>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>> +//
>> +// This program is free software; you can redistribute it and/or modify
>> +// it under the terms of the GNU General Public License as published by
>> +// the Free Software Foundation; either version 2 of the License, or
>> +// (at your option) any later version.
>> +//
>> +// This program is distributed in the hope that it will be useful,
>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +// GNU General Public License for more details.
>> +//
>> +// You should have received a copy of the GNU General Public License
>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>> +//
>> +// Usage example:
>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>
> You have --max-width 80 here, but not in the commit message. Default
> seems to be 78. Any particular reason to change it to 80?
>
>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>> +
>> +// Switch unusual (Error **) parameter names to errp
>
> Let's drop the parenthesis around Error **
>
>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>
> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
> make the fact we're messing with @errp more obvious. Too late; I
> shouldn't rock the boat that much now.
>
>> +//
>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>> +// parameter.
>> +//
>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>> +// non generic semantics and may have unusual Error ** argument name for purpose
>
> non-generic
>
> for a purpose
>
> Wrap comment lines around column 70, please. It's easier to read.
>
> Maybe
>
> // Skip functions with "assert(_errp && *_errp)" statement, because that
> // signals unusual semantics, and the parameter name may well serve a
> // purpose.
>
>> +// (like nbd_iter_channel_error()).
>> +//
>> +// Skip util/error.c to not touch, for example, error_propagate and
>> +// error_propagate_prepend().
>
> error_propagate()
>
> I much appreciate your meticulous explanation of what you skip and why.
>
>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>> +identifier fn;
>> +identifier _errp != errp;
>> +@@
>> +
>> + fn(...,
>> +- Error **_errp
>> ++ Error **errp
>> + ,...)
>> + {
>> +(
>> + ... when != assert(_errp && *_errp)
>> +&
>> + <...
>> +- _errp
>> ++ errp
>> + ...>
>> +)
>> + }
>
> This rule is required to make the actual transformations (below) work
> even for parameters with names other than @errp. I believe it's not
> used in this series. In fact, I can't see a use for it in the entire
> tree right now. Okay anyway.
>
>> +
>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>> +//
>> +// Note, that without "when any" final "..." may not want to mach something
>
> s/final "..." may not mach/the final "..." does not match/
>
>> +// matched by previous pattern, i.e. the rule will not match double
>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>
> Can't say I fully understand Coccinelle there. I figure you came to
> this knowledge the hard way.
>
>> +//
>> +// Note, "exists" says that we want apply rule even if it matches not on
>> +// all possible control flows (otherwise, it will not match standard pattern
>> +// when error_propagate() call is in if branch).
>
> Learned something new. Example: kvm_set_kvm_shadow_mem().
>
> Spelling it "exists disable optional_qualifier" would avoid giving
> readers the idea we're disabling "exists", but Coccinelle doesn't let
> us. Oh well.
>
>> +@ disable optional_qualifier exists@
>> +identifier fn, local_err, errp;
>
> I believe this causes
>
> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>
> Making @errp symbol instead of identifier should fix this.
>
>> +@@
>> +
>> + fn(..., Error **errp, ...)
>> + {
>> ++ ERRP_AUTO_PROPAGATE();
>> + ... when != ERRP_AUTO_PROPAGATE();
>> +(
>> + error_append_hint(errp, ...);
>> +|
>> + error_prepend(errp, ...);
>> +|
>> + error_vprepend(errp, ...);
>> +|
>> + Error *local_err = NULL;
>> + ...
>> +(
>> + error_propagate_prepend(errp, local_err, ...);
>> +|
>> + error_propagate(errp, local_err);
>> +)
>> +)
>> + ... when any
>> + }
>> +
>> +
>> +// Match scenarios with propagation of local error to errp.
>> +@rule1 disable optional_qualifier exists@
>> +identifier fn, local_err;
>> +symbol errp;
>> +@@
>> +
>> + fn(..., Error **errp, ...)
>> + {
>> + ...
>> + Error *local_err = NULL;
>> + ...
>> +(
>> + error_propagate_prepend(errp, local_err, ...);
>> +|
>> + error_propagate(errp, local_err);
>> +)
>
> Indentation off by one.
>
>> + ...
>> + }
>> +
>> +// Convert special case with goto in separate.
>
> s/in separate/separately/
>
>> +// We can probably merge this into the following hunk with help of ( | )
>> +// operator, but it significantly reduce performance on block.c parsing (or it
>
> s/reduce/reduces/
>
>> +// hangs, I don't know)
>
> Sounds like you tried to merge this into the following hunk, but then
> spatch took so long on block.c that you killed it. Correct?
>
>> +//
>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>> +// things later after all transformations (the rule will be the same, just
>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>
> Weird, but not worth further investigation.
>
>> +@@
>> +identifier rule1.fn, rule1.local_err, out;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- goto out;
>> ++ return;
>> + ...>
>> +- out:
>> +- error_propagate(errp, local_err);
>
> You neglect to match error_propagate_prepend(). Okay, because (1) that
> pattern doesn't occur in the tree right now, and (2) if it gets added,
> gcc will complain.
>
>> + }
>> +
>> +// Convert most of local_err related staff.
>
> s/staff/stuff/
>
>> +//
>> +// Note, that we update everything related to matched by rule1 function name
>> +// and local_err name. We may match something not related to the pattern
>> +// matched by rule1. For example, local_err may be defined with the same name
>> +// in different blocks inside one function, and in one block follow the
>> +// propagation pattern and in other block doesn't. Or we may have several
>> +// functions with the same name (for different configurations).
>
> Context: rule1 matches functions that have all three of
>
> * an Error **errp parameter
>
> * an Error *local_err = NULL variable declaration
>
> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
> local_err, ...) expression, where @errp is the parameter and
> @local_err is the variable.
>
> If I understand you correctly, you're pointing out two potential issues:
>
> 1. This rule can match functions rule1 does not match if there is
> another function with the same name that rule1 does match.
>
> 2. This rule matches in the entire function matched by rule1, even when
> parts of that function use a different @errp or @local_err.
>
> I figure these apply to all rules with identifier rule1.fn, not just
> this one. Correct?
>
> Regarding 1. There must be a better way to chain rules together, but I
> don't know it.
Hmm, what about something like this:
@rule1 disable optional_qualifier exists@
identifier fn, local_err;
symbol errp;
@@
fn(..., Error **
- errp
+ ___errp_coccinelle_updating___
, ...)
{
...
Error *local_err = NULL;
...
(
error_propagate_prepend(errp, local_err, ...);
|
error_propagate(errp, local_err);
)
...
}
[..]
match symbol ___errp_coccinelle_updating___ in following rules in function header
[..]
@ disable optional_qualifier@
identifier fn, local_err;
symbol errp;
@@
fn(..., Error **
- ___errp_coccinelle_updating___
+ errp
, ...)
{
...
}
- hacky, but seems not more hacky than python detection, and should work better
> Can we make Coccinelle at least warn us when it converts
> multiple functions with the same name? What about this:
>
> @initialize:python@
> @@
> fnprev = {}
>
> def pr(fn, p):
> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>
> @r@
> identifier rule1.fn;
> position p;
> @@
> fn(...)@p
> {
> ...
> }
> @script:python@
> fn << rule1.fn;
> p << r.p;
> @@
> if fn not in fnprev:
> fnprev[fn] = p
> else:
> if fnprev[fn]:
> pr(fn, fnprev[fn])
> fnprev[fn] = None
> pr(fn, p)
>
> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
> the number of functions with the same name we touch. If it's more than
> one, we print.
>
> Reports about a dozen function names for the whole tree in my testing.
> Inspecting the changes to them manually is feasible. None of them are
> in files touched by this series.
>
> The line printed for the first match is pretty useless for me: it points
> to a Coccinelle temporary file *shrug*.
>
> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
> sure hope we don't do that. Multiple @local_err variables... hmm.
> Perhaps we could again concoct some script rules to lead us to spots to
> check manually. See below for my attempt.
>
> What's the worst that could happen if we blindly converted such code?
> The answer to that question tells us how hard to work on finding and
> checking these guys.
>
>> +//
>> +// Note also that errp-cleaning functions
>> +// error_free_errp
>> +// error_report_errp
>> +// error_reportf_errp
>> +// warn_report_errp
>> +// warn_reportf_errp
>> +// are not yet implemented. They must call corresponding Error* - freeing
>> +// function and then set *errp to NULL, to avoid further propagation to
>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>> +// For example, error_free_errp may look like this:
>> +//
>> +// void error_free_errp(Error **errp)
>> +// {
>> +// error_free(*errp);
>> +// *errp = NULL;
>> +// }
>> +@ exists@
>> +identifier rule1.fn, rule1.local_err;
>> +expression list args;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +(
>
> Each of the following patterns applies anywhere in the function.
>
> First pattern: delete @local_err
>
>> +- Error *local_err = NULL;
>
> Common case: occurs just once, not nested. Anything else is suspicious.
>
> Both can be detected in the resulting patches with a bit of AWK
> wizardry:
>
> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
> nested
> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
> nested
> static void xen_block_device_destroy(XenBackendInstance *backend,
> nested
> static void xen_block_device_destroy(XenBackendInstance *backend,
> more than one
> static void xen_block_device_destroy(XenBackendInstance *backend,
>
> Oh.
>
> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>
> xen_block_device_destroy() has multiple Error *local_err.
>
> In both cases, manual review is required to ensure the conversion is
> okay. I believe it is.
>
> Note that the AWK script relies on diff showing the function name in @@
> lines, which doesn't always work due to our coding style.
>
> For the whole tree, I get some 30 spots. Feasible.
>
>> +|
>
> Second pattern: clear @errp after freeing it
>
>> +
>> +// Convert error clearing functions
>
> Suggest: Ensure @local_err is cleared on free
>
>> +(
>> +- error_free(local_err);
>> ++ error_free_errp(errp);
>> +|
>> +- error_report_err(local_err);
>> ++ error_report_errp(errp);
>> +|
>> +- error_reportf_err(local_err, args);
>> ++ error_reportf_errp(errp, args);
>> +|
>> +- warn_report_err(local_err);
>> ++ warn_report_errp(errp);
>> +|
>> +- warn_reportf_err(local_err, args);
>> ++ warn_reportf_errp(errp, args);
>> +)
>
> As you mention above, these guys don't exist, yet. Builds anyway,
> because this part of the rule is not used in this patch series. You
> don't want to omit it, because then the script becomes unsafe to use.
>
> We could also open-code:
>
> // Convert error clearing functions
> (
> - error_free(local_err);
> + error_free(*errp);
> + *errp = NULL;
> |
> ... and so forth ...
> )
>
> Matter of taste. Whatever is easier to explain in the comments. Since
> you already wrote one...
>
> We talked about extending this series slightly so these guys are used.
> I may still look into that.
>
>> +?- local_err = NULL;
>> +
>
> The new helpers clear @local_err. Assignment now redundant, delete.
> Okay.
>
>> +|
>
> Third and fourth pattern: delete error_propagate()
>
>> +- error_propagate_prepend(errp, local_err, args);
>> ++ error_prepend(errp, args);
>> +|
>> +- error_propagate(errp, local_err);
>> +|
>
> Fifth pattern: use @errp directly
>
>> +- &local_err
>> ++ errp
>> +)
>> + ...>
>> + }
>> +
>> +// Convert remaining local_err usage. It should be different kinds of error
>> +// checking in if operators. We can't merge this into previous hunk, as this
>
> In if conditionals, I suppose. It's the case for this patch. If I
> apply the script to the whole tree, the rule gets also applied in other
> contexts. The sentence might mislead as much as it helps. Keep it or
> delete it?
>
>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>> +@@
>> +identifier rule1.fn, rule1.local_err;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- local_err
>> ++ *errp
>> + ...>
>> + }
>> +
>> +// Always use the same patter for checking error
>
> s/patter/pattern/
>
>> +@@
>> +identifier rule1.fn;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- *errp != NULL
>> ++ *errp
>> + ...>
>> + }
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 8:29 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 8:29 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
09.03.2020 12:56, Markus Armbruster wrote:
> Suggest
>
> scripts: Coccinelle script to use auto-propagated errp
>
> or
>
> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> Script adds ERRP_AUTO_PROPAGATE macro invocation where appropriate and
>> does corresponding changes in code (look for details in
>> include/qapi/error.h)
>>
>> Usage example:
>> spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>> --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>> blockdev-nbd.c qemu-nbd.c {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>
> Suggest FILES... instead of a specific set of files.
>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>> ---
>>
>> Cc: Eric Blake <eblake@redhat.com>
>> Cc: Kevin Wolf <kwolf@redhat.com>
>> Cc: Max Reitz <mreitz@redhat.com>
>> Cc: Greg Kurz <groug@kaod.org>
>> Cc: Christian Schoenebeck <qemu_oss@crudebyte.com>
>> Cc: Stefano Stabellini <sstabellini@kernel.org>
>> Cc: Anthony Perard <anthony.perard@citrix.com>
>> Cc: Paul Durrant <paul@xen.org>
>> Cc: Stefan Hajnoczi <stefanha@redhat.com>
>> Cc: "Philippe Mathieu-Daudé" <philmd@redhat.com>
>> Cc: Laszlo Ersek <lersek@redhat.com>
>> Cc: Gerd Hoffmann <kraxel@redhat.com>
>> Cc: Stefan Berger <stefanb@linux.ibm.com>
>> Cc: Markus Armbruster <armbru@redhat.com>
>> Cc: Michael Roth <mdroth@linux.vnet.ibm.com>
>> Cc: qemu-block@nongnu.org
>> Cc: qemu-devel@nongnu.org
>> Cc: xen-devel@lists.xenproject.org
>>
>> include/qapi/error.h | 3 +
>> scripts/coccinelle/auto-propagated-errp.cocci | 231 ++++++++++++++++++
>> 2 files changed, 234 insertions(+)
>> create mode 100644 scripts/coccinelle/auto-propagated-errp.cocci
>>
>> diff --git a/include/qapi/error.h b/include/qapi/error.h
>> index bb9bcf02fb..fbfc6f1c0b 100644
>> --- a/include/qapi/error.h
>> +++ b/include/qapi/error.h
>> @@ -211,6 +211,9 @@
>> * }
>> * ...
>> * }
>> + *
>> + * For mass conversion use script
>
> mass-conversion (we're not converting mass, we're converting en masse)
>
>> + * scripts/coccinelle/auto-propagated-errp.cocci
>> */
>>
>> #ifndef ERROR_H
>> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
>> new file mode 100644
>> index 0000000000..bff274bd6d
>> --- /dev/null
>> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
>
> Preface to my review of this script: may aim isn't to make it
> bullet-proof. I want to (1) make it good enough (explained in a
> jiffie), and (2) automatically identify the spots where it still isn't
> obviously safe for manual review.
>
> The latter may involve additional scripting. That's okay.
>
> The script is good enough when the number of possibly unsafe spots is
> low enough for careful manual review.
>
> When I ask for improvements that, in your opinion, go beyond "good
> enough", please push back. I'm sure we can work it out together.
>
>> @@ -0,0 +1,231 @@
>> +// Use ERRP_AUTO_PROPAGATE (see include/qapi/error.h)
>> +//
>> +// Copyright (c) 2020 Virtuozzo International GmbH.
>> +//
>> +// This program is free software; you can redistribute it and/or modify
>> +// it under the terms of the GNU General Public License as published by
>> +// the Free Software Foundation; either version 2 of the License, or
>> +// (at your option) any later version.
>> +//
>> +// This program is distributed in the hope that it will be useful,
>> +// but WITHOUT ANY WARRANTY; without even the implied warranty of
>> +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> +// GNU General Public License for more details.
>> +//
>> +// You should have received a copy of the GNU General Public License
>> +// along with this program. If not, see <http://www.gnu.org/licenses/>.
>> +//
>> +// Usage example:
>> +// spatch --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>> +// --macro-file scripts/cocci-macro-file.h --in-place --no-show-diff \
>> +// --max-width 80 blockdev-nbd.c qemu-nbd.c \
>
> You have --max-width 80 here, but not in the commit message. Default
> seems to be 78. Any particular reason to change it to 80?
>
>> +// {block/nbd*,nbd/*,include/block/nbd*}.[hc]
>> +
>> +// Switch unusual (Error **) parameter names to errp
>
> Let's drop the parenthesis around Error **
>
>> +// (this is necessary to use ERRP_AUTO_PROPAGATE).
>
> Perhaps ERRP_AUTO_PROPAGATE() should be ERRP_AUTO_PROPAGATE(errp) to
> make the fact we're messing with @errp more obvious. Too late; I
> shouldn't rock the boat that much now.
>
>> +//
>> +// Disable optional_qualifier to skip functions with "Error *const *errp"
>> +// parameter.
>> +//
>> +// Skip functions with "assert(_errp && *_errp)" statement, as they have
>> +// non generic semantics and may have unusual Error ** argument name for purpose
>
> non-generic
>
> for a purpose
>
> Wrap comment lines around column 70, please. It's easier to read.
>
> Maybe
>
> // Skip functions with "assert(_errp && *_errp)" statement, because that
> // signals unusual semantics, and the parameter name may well serve a
> // purpose.
>
>> +// (like nbd_iter_channel_error()).
>> +//
>> +// Skip util/error.c to not touch, for example, error_propagate and
>> +// error_propagate_prepend().
>
> error_propagate()
>
> I much appreciate your meticulous explanation of what you skip and why.
>
>> +@ depends on !(file in "util/error.c") disable optional_qualifier@
>> +identifier fn;
>> +identifier _errp != errp;
>> +@@
>> +
>> + fn(...,
>> +- Error **_errp
>> ++ Error **errp
>> + ,...)
>> + {
>> +(
>> + ... when != assert(_errp && *_errp)
>> +&
>> + <...
>> +- _errp
>> ++ errp
>> + ...>
>> +)
>> + }
>
> This rule is required to make the actual transformations (below) work
> even for parameters with names other than @errp. I believe it's not
> used in this series. In fact, I can't see a use for it in the entire
> tree right now. Okay anyway.
>
>> +
>> +// Add invocation of ERRP_AUTO_PROPAGATE to errp-functions where necessary
>> +//
>> +// Note, that without "when any" final "..." may not want to mach something
>
> s/final "..." may not mach/the final "..." does not match/
>
>> +// matched by previous pattern, i.e. the rule will not match double
>> +// error_prepend in control flow like in vfio_set_irq_signaling().
>
> Can't say I fully understand Coccinelle there. I figure you came to
> this knowledge the hard way.
>
>> +//
>> +// Note, "exists" says that we want apply rule even if it matches not on
>> +// all possible control flows (otherwise, it will not match standard pattern
>> +// when error_propagate() call is in if branch).
>
> Learned something new. Example: kvm_set_kvm_shadow_mem().
>
> Spelling it "exists disable optional_qualifier" would avoid giving
> readers the idea we're disabling "exists", but Coccinelle doesn't let
> us. Oh well.
>
>> +@ disable optional_qualifier exists@
>> +identifier fn, local_err, errp;
>
> I believe this causes
>
> warning: line 98: errp, previously declared as a metavariable, is used as an identifier
> warning: line 104: errp, previously declared as a metavariable, is used as an identifier
> warning: line 106: errp, previously declared as a metavariable, is used as an identifier
> warning: line 131: errp, previously declared as a metavariable, is used as an identifier
> warning: line 192: errp, previously declared as a metavariable, is used as an identifier
> warning: line 195: errp, previously declared as a metavariable, is used as an identifier
> warning: line 228: errp, previously declared as a metavariable, is used as an identifier
>
> Making @errp symbol instead of identifier should fix this.
>
>> +@@
>> +
>> + fn(..., Error **errp, ...)
>> + {
>> ++ ERRP_AUTO_PROPAGATE();
>> + ... when != ERRP_AUTO_PROPAGATE();
>> +(
>> + error_append_hint(errp, ...);
>> +|
>> + error_prepend(errp, ...);
>> +|
>> + error_vprepend(errp, ...);
>> +|
>> + Error *local_err = NULL;
>> + ...
>> +(
>> + error_propagate_prepend(errp, local_err, ...);
>> +|
>> + error_propagate(errp, local_err);
>> +)
>> +)
>> + ... when any
>> + }
>> +
>> +
>> +// Match scenarios with propagation of local error to errp.
>> +@rule1 disable optional_qualifier exists@
>> +identifier fn, local_err;
>> +symbol errp;
>> +@@
>> +
>> + fn(..., Error **errp, ...)
>> + {
>> + ...
>> + Error *local_err = NULL;
>> + ...
>> +(
>> + error_propagate_prepend(errp, local_err, ...);
>> +|
>> + error_propagate(errp, local_err);
>> +)
>
> Indentation off by one.
>
>> + ...
>> + }
>> +
>> +// Convert special case with goto in separate.
>
> s/in separate/separately/
>
>> +// We can probably merge this into the following hunk with help of ( | )
>> +// operator, but it significantly reduce performance on block.c parsing (or it
>
> s/reduce/reduces/
>
>> +// hangs, I don't know)
>
> Sounds like you tried to merge this into the following hunk, but then
> spatch took so long on block.c that you killed it. Correct?
>
>> +//
>> +// Note interesting thing: if we don't do it here, and try to fixup "out: }"
>> +// things later after all transformations (the rule will be the same, just
>> +// without error_propagate() call), coccinelle fails to match this "out: }".
>
> Weird, but not worth further investigation.
>
>> +@@
>> +identifier rule1.fn, rule1.local_err, out;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- goto out;
>> ++ return;
>> + ...>
>> +- out:
>> +- error_propagate(errp, local_err);
>
> You neglect to match error_propagate_prepend(). Okay, because (1) that
> pattern doesn't occur in the tree right now, and (2) if it gets added,
> gcc will complain.
>
>> + }
>> +
>> +// Convert most of local_err related staff.
>
> s/staff/stuff/
>
>> +//
>> +// Note, that we update everything related to matched by rule1 function name
>> +// and local_err name. We may match something not related to the pattern
>> +// matched by rule1. For example, local_err may be defined with the same name
>> +// in different blocks inside one function, and in one block follow the
>> +// propagation pattern and in other block doesn't. Or we may have several
>> +// functions with the same name (for different configurations).
>
> Context: rule1 matches functions that have all three of
>
> * an Error **errp parameter
>
> * an Error *local_err = NULL variable declaration
>
> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
> local_err, ...) expression, where @errp is the parameter and
> @local_err is the variable.
>
> If I understand you correctly, you're pointing out two potential issues:
>
> 1. This rule can match functions rule1 does not match if there is
> another function with the same name that rule1 does match.
>
> 2. This rule matches in the entire function matched by rule1, even when
> parts of that function use a different @errp or @local_err.
>
> I figure these apply to all rules with identifier rule1.fn, not just
> this one. Correct?
>
> Regarding 1. There must be a better way to chain rules together, but I
> don't know it.
Hmm, what about something like this:
@rule1 disable optional_qualifier exists@
identifier fn, local_err;
symbol errp;
@@
fn(..., Error **
- errp
+ ___errp_coccinelle_updating___
, ...)
{
...
Error *local_err = NULL;
...
(
error_propagate_prepend(errp, local_err, ...);
|
error_propagate(errp, local_err);
)
...
}
[..]
match symbol ___errp_coccinelle_updating___ in following rules in function header
[..]
@ disable optional_qualifier@
identifier fn, local_err;
symbol errp;
@@
fn(..., Error **
- ___errp_coccinelle_updating___
+ errp
, ...)
{
...
}
- hacky, but seems not more hacky than python detection, and should work better
> Can we make Coccinelle at least warn us when it converts
> multiple functions with the same name? What about this:
>
> @initialize:python@
> @@
> fnprev = {}
>
> def pr(fn, p):
> print("### %s:%s: %s()" % (p[0].file, p[0].line, fn))
>
> @r@
> identifier rule1.fn;
> position p;
> @@
> fn(...)@p
> {
> ...
> }
> @script:python@
> fn << rule1.fn;
> p << r.p;
> @@
> if fn not in fnprev:
> fnprev[fn] = p
> else:
> if fnprev[fn]:
> pr(fn, fnprev[fn])
> fnprev[fn] = None
> pr(fn, p)
>
> For each function @fn matched by rule1, fncnt[fn] is an upper limit of
> the number of functions with the same name we touch. If it's more than
> one, we print.
>
> Reports about a dozen function names for the whole tree in my testing.
> Inspecting the changes to them manually is feasible. None of them are
> in files touched by this series.
>
> The line printed for the first match is pretty useless for me: it points
> to a Coccinelle temporary file *shrug*.
>
> Regarding 2. Shadowing @errp or @local_err would be in bad taste, and I
> sure hope we don't do that. Multiple @local_err variables... hmm.
> Perhaps we could again concoct some script rules to lead us to spots to
> check manually. See below for my attempt.
>
> What's the worst that could happen if we blindly converted such code?
> The answer to that question tells us how hard to work on finding and
> checking these guys.
>
>> +//
>> +// Note also that errp-cleaning functions
>> +// error_free_errp
>> +// error_report_errp
>> +// error_reportf_errp
>> +// warn_report_errp
>> +// warn_reportf_errp
>> +// are not yet implemented. They must call corresponding Error* - freeing
>> +// function and then set *errp to NULL, to avoid further propagation to
>> +// original errp (consider ERRP_AUTO_PROPAGATE in use).
>> +// For example, error_free_errp may look like this:
>> +//
>> +// void error_free_errp(Error **errp)
>> +// {
>> +// error_free(*errp);
>> +// *errp = NULL;
>> +// }
>> +@ exists@
>> +identifier rule1.fn, rule1.local_err;
>> +expression list args;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +(
>
> Each of the following patterns applies anywhere in the function.
>
> First pattern: delete @local_err
>
>> +- Error *local_err = NULL;
>
> Common case: occurs just once, not nested. Anything else is suspicious.
>
> Both can be detected in the resulting patches with a bit of AWK
> wizardry:
>
> $ git-diff -U0 master..review-error-v8 | awk '/^@@ / { ctx = $5; for (i = 6; i <= NF; i++) ctx = ctx " " $i; if (ctx != octx) { octx = ctx; n = 0 } } /^- *Error *\* *[A-Za-z0-9_]+ *= *NULL;/ { if (index($0, "E") > 6) print "nested\n " ctx; if (n) print "more than one\n " ctx; n++ }'
> nested
> static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
> nested
> static void xen_block_device_destroy(XenBackendInstance *backend,
> nested
> static void xen_block_device_destroy(XenBackendInstance *backend,
> more than one
> static void xen_block_device_destroy(XenBackendInstance *backend,
>
> Oh.
>
> xen_block_drive_destroy() nests its Error *local_err in a conditional.
>
> xen_block_device_destroy() has multiple Error *local_err.
>
> In both cases, manual review is required to ensure the conversion is
> okay. I believe it is.
>
> Note that the AWK script relies on diff showing the function name in @@
> lines, which doesn't always work due to our coding style.
>
> For the whole tree, I get some 30 spots. Feasible.
>
>> +|
>
> Second pattern: clear @errp after freeing it
>
>> +
>> +// Convert error clearing functions
>
> Suggest: Ensure @local_err is cleared on free
>
>> +(
>> +- error_free(local_err);
>> ++ error_free_errp(errp);
>> +|
>> +- error_report_err(local_err);
>> ++ error_report_errp(errp);
>> +|
>> +- error_reportf_err(local_err, args);
>> ++ error_reportf_errp(errp, args);
>> +|
>> +- warn_report_err(local_err);
>> ++ warn_report_errp(errp);
>> +|
>> +- warn_reportf_err(local_err, args);
>> ++ warn_reportf_errp(errp, args);
>> +)
>
> As you mention above, these guys don't exist, yet. Builds anyway,
> because this part of the rule is not used in this patch series. You
> don't want to omit it, because then the script becomes unsafe to use.
>
> We could also open-code:
>
> // Convert error clearing functions
> (
> - error_free(local_err);
> + error_free(*errp);
> + *errp = NULL;
> |
> ... and so forth ...
> )
>
> Matter of taste. Whatever is easier to explain in the comments. Since
> you already wrote one...
>
> We talked about extending this series slightly so these guys are used.
> I may still look into that.
>
>> +?- local_err = NULL;
>> +
>
> The new helpers clear @local_err. Assignment now redundant, delete.
> Okay.
>
>> +|
>
> Third and fourth pattern: delete error_propagate()
>
>> +- error_propagate_prepend(errp, local_err, args);
>> ++ error_prepend(errp, args);
>> +|
>> +- error_propagate(errp, local_err);
>> +|
>
> Fifth pattern: use @errp directly
>
>> +- &local_err
>> ++ errp
>> +)
>> + ...>
>> + }
>> +
>> +// Convert remaining local_err usage. It should be different kinds of error
>> +// checking in if operators. We can't merge this into previous hunk, as this
>
> In if conditionals, I suppose. It's the case for this patch. If I
> apply the script to the whole tree, the rule gets also applied in other
> contexts. The sentence might mislead as much as it helps. Keep it or
> delete it?
>
>> +// conflicts with other substitutions in it (at least with "- local_err = NULL").
>> +@@
>> +identifier rule1.fn, rule1.local_err;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- local_err
>> ++ *errp
>> + ...>
>> + }
>> +
>> +// Always use the same patter for checking error
>
> s/patter/pattern/
>
>> +@@
>> +identifier rule1.fn;
>> +symbol errp;
>> +@@
>> +
>> + fn(...)
>> + {
>> + <...
>> +- *errp != NULL
>> ++ *errp
>> + ...>
>> + }
>
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-11 8:29 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-11 9:38 ` Markus Armbruster
-1 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-11 9:38 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 09.03.2020 12:56, Markus Armbruster wrote:
>> Suggest
>>
>> scripts: Coccinelle script to use auto-propagated errp
>>
>> or
>>
>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
[...]
>>> +// Note, that we update everything related to matched by rule1 function name
>>> +// and local_err name. We may match something not related to the pattern
>>> +// matched by rule1. For example, local_err may be defined with the same name
>>> +// in different blocks inside one function, and in one block follow the
>>> +// propagation pattern and in other block doesn't. Or we may have several
>>> +// functions with the same name (for different configurations).
>>
>> Context: rule1 matches functions that have all three of
>>
>> * an Error **errp parameter
>>
>> * an Error *local_err = NULL variable declaration
>>
>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>> local_err, ...) expression, where @errp is the parameter and
>> @local_err is the variable.
>>
>> If I understand you correctly, you're pointing out two potential issues:
>>
>> 1. This rule can match functions rule1 does not match if there is
>> another function with the same name that rule1 does match.
>>
>> 2. This rule matches in the entire function matched by rule1, even when
>> parts of that function use a different @errp or @local_err.
>>
>> I figure these apply to all rules with identifier rule1.fn, not just
>> this one. Correct?
>>
>> Regarding 1. There must be a better way to chain rules together, but I
>> don't know it.
>
> Hmm, what about something like this:
>
> @rule1 disable optional_qualifier exists@
> identifier fn, local_err;
> symbol errp;
> @@
>
> fn(..., Error **
> - errp
> + ___errp_coccinelle_updating___
> , ...)
> {
> ...
> Error *local_err = NULL;
> ...
> (
> error_propagate_prepend(errp, local_err, ...);
> |
> error_propagate(errp, local_err);
> )
> ...
> }
>
>
> [..]
>
> match symbol ___errp_coccinelle_updating___ in following rules in function header
>
> [..]
>
>
> @ disable optional_qualifier@
> identifier fn, local_err;
> symbol errp;
> @@
>
> fn(..., Error **
> - ___errp_coccinelle_updating___
> + errp
> , ...)
> {
> ...
> }
>
>
> - hacky, but seems not more hacky than python detection, and should work better
As simple, forceful and unsubtle as a sledgehammer. I like it :)
[...]
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 9:38 ` Markus Armbruster
0 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-11 9:38 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 09.03.2020 12:56, Markus Armbruster wrote:
>> Suggest
>>
>> scripts: Coccinelle script to use auto-propagated errp
>>
>> or
>>
>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
[...]
>>> +// Note, that we update everything related to matched by rule1 function name
>>> +// and local_err name. We may match something not related to the pattern
>>> +// matched by rule1. For example, local_err may be defined with the same name
>>> +// in different blocks inside one function, and in one block follow the
>>> +// propagation pattern and in other block doesn't. Or we may have several
>>> +// functions with the same name (for different configurations).
>>
>> Context: rule1 matches functions that have all three of
>>
>> * an Error **errp parameter
>>
>> * an Error *local_err = NULL variable declaration
>>
>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>> local_err, ...) expression, where @errp is the parameter and
>> @local_err is the variable.
>>
>> If I understand you correctly, you're pointing out two potential issues:
>>
>> 1. This rule can match functions rule1 does not match if there is
>> another function with the same name that rule1 does match.
>>
>> 2. This rule matches in the entire function matched by rule1, even when
>> parts of that function use a different @errp or @local_err.
>>
>> I figure these apply to all rules with identifier rule1.fn, not just
>> this one. Correct?
>>
>> Regarding 1. There must be a better way to chain rules together, but I
>> don't know it.
>
> Hmm, what about something like this:
>
> @rule1 disable optional_qualifier exists@
> identifier fn, local_err;
> symbol errp;
> @@
>
> fn(..., Error **
> - errp
> + ___errp_coccinelle_updating___
> , ...)
> {
> ...
> Error *local_err = NULL;
> ...
> (
> error_propagate_prepend(errp, local_err, ...);
> |
> error_propagate(errp, local_err);
> )
> ...
> }
>
>
> [..]
>
> match symbol ___errp_coccinelle_updating___ in following rules in function header
>
> [..]
>
>
> @ disable optional_qualifier@
> identifier fn, local_err;
> symbol errp;
> @@
>
> fn(..., Error **
> - ___errp_coccinelle_updating___
> + errp
> , ...)
> {
> ...
> }
>
>
> - hacky, but seems not more hacky than python detection, and should work better
As simple, forceful and unsubtle as a sledgehammer. I like it :)
[...]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-11 9:38 ` [Xen-devel] " Markus Armbruster
@ 2020-03-11 14:05 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 14:05 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
11.03.2020 12:38, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 09.03.2020 12:56, Markus Armbruster wrote:
>>> Suggest
>>>
>>> scripts: Coccinelle script to use auto-propagated errp
>>>
>>> or
>>>
>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>
>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> [...]
>>>> +// Note, that we update everything related to matched by rule1 function name
>>>> +// and local_err name. We may match something not related to the pattern
>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>> +// in different blocks inside one function, and in one block follow the
>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>> +// functions with the same name (for different configurations).
>>>
>>> Context: rule1 matches functions that have all three of
>>>
>>> * an Error **errp parameter
>>>
>>> * an Error *local_err = NULL variable declaration
>>>
>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>> local_err, ...) expression, where @errp is the parameter and
>>> @local_err is the variable.
>>>
>>> If I understand you correctly, you're pointing out two potential issues:
>>>
>>> 1. This rule can match functions rule1 does not match if there is
>>> another function with the same name that rule1 does match.
>>>
>>> 2. This rule matches in the entire function matched by rule1, even when
>>> parts of that function use a different @errp or @local_err.
>>>
>>> I figure these apply to all rules with identifier rule1.fn, not just
>>> this one. Correct?
>>>
>>> Regarding 1. There must be a better way to chain rules together, but I
>>> don't know it.
>>
>> Hmm, what about something like this:
>>
>> @rule1 disable optional_qualifier exists@
>> identifier fn, local_err;
>> symbol errp;
>> @@
>>
>> fn(..., Error **
>> - errp
>> + ___errp_coccinelle_updating___
>> , ...)
>> {
>> ...
>> Error *local_err = NULL;
>> ...
>> (
>> error_propagate_prepend(errp, local_err, ...);
>> |
>> error_propagate(errp, local_err);
>> )
>> ...
>> }
>>
>>
>> [..]
>>
>> match symbol ___errp_coccinelle_updating___ in following rules in function header
>>
>> [..]
>>
>>
>> @ disable optional_qualifier@
>> identifier fn, local_err;
>> symbol errp;
>> @@
>>
>> fn(..., Error **
>> - ___errp_coccinelle_updating___
>> + errp
>> , ...)
>> {
>> ...
>> }
>>
>>
>> - hacky, but seems not more hacky than python detection, and should work better
>
> As simple, forceful and unsubtle as a sledgehammer. I like it :)
>
Hmm, not so simple.
It leads to reindenting of function header, which is bad.
Possible solution is instead
fn(...)
{
+ ___errp_coccinelle_updating___();
but this slow down coccinelle. For example, on block.c from ~3s to 1m16s.
.
So, I'm returning to just a warning.
I think something simple like
@@
identifier rule1.fn;
position p != rule1.p;
@@
fn@p(...) {...}
@ script:python@
<print warning>
should work.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 14:05 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 14:05 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
11.03.2020 12:38, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 09.03.2020 12:56, Markus Armbruster wrote:
>>> Suggest
>>>
>>> scripts: Coccinelle script to use auto-propagated errp
>>>
>>> or
>>>
>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>
>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> [...]
>>>> +// Note, that we update everything related to matched by rule1 function name
>>>> +// and local_err name. We may match something not related to the pattern
>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>> +// in different blocks inside one function, and in one block follow the
>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>> +// functions with the same name (for different configurations).
>>>
>>> Context: rule1 matches functions that have all three of
>>>
>>> * an Error **errp parameter
>>>
>>> * an Error *local_err = NULL variable declaration
>>>
>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>> local_err, ...) expression, where @errp is the parameter and
>>> @local_err is the variable.
>>>
>>> If I understand you correctly, you're pointing out two potential issues:
>>>
>>> 1. This rule can match functions rule1 does not match if there is
>>> another function with the same name that rule1 does match.
>>>
>>> 2. This rule matches in the entire function matched by rule1, even when
>>> parts of that function use a different @errp or @local_err.
>>>
>>> I figure these apply to all rules with identifier rule1.fn, not just
>>> this one. Correct?
>>>
>>> Regarding 1. There must be a better way to chain rules together, but I
>>> don't know it.
>>
>> Hmm, what about something like this:
>>
>> @rule1 disable optional_qualifier exists@
>> identifier fn, local_err;
>> symbol errp;
>> @@
>>
>> fn(..., Error **
>> - errp
>> + ___errp_coccinelle_updating___
>> , ...)
>> {
>> ...
>> Error *local_err = NULL;
>> ...
>> (
>> error_propagate_prepend(errp, local_err, ...);
>> |
>> error_propagate(errp, local_err);
>> )
>> ...
>> }
>>
>>
>> [..]
>>
>> match symbol ___errp_coccinelle_updating___ in following rules in function header
>>
>> [..]
>>
>>
>> @ disable optional_qualifier@
>> identifier fn, local_err;
>> symbol errp;
>> @@
>>
>> fn(..., Error **
>> - ___errp_coccinelle_updating___
>> + errp
>> , ...)
>> {
>> ...
>> }
>>
>>
>> - hacky, but seems not more hacky than python detection, and should work better
>
> As simple, forceful and unsubtle as a sledgehammer. I like it :)
>
Hmm, not so simple.
It leads to reindenting of function header, which is bad.
Possible solution is instead
fn(...)
{
+ ___errp_coccinelle_updating___();
but this slow down coccinelle. For example, on block.c from ~3s to 1m16s.
.
So, I'm returning to just a warning.
I think something simple like
@@
identifier rule1.fn;
position p != rule1.p;
@@
fn@p(...) {...}
@ script:python@
<print warning>
should work.
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-11 14:05 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-11 14:41 ` Markus Armbruster
-1 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-11 14:41 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
qemu-devel, Laszlo Ersek, Christian Schoenebeck,
Markus Armbruster, Michael Roth, Greg Kurz, Gerd Hoffmann,
Stefan Hajnoczi, Anthony Perard, xen-devel, Max Reitz,
Philippe Mathieu-Daudé,
Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 11.03.2020 12:38, Markus Armbruster wrote:
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>
>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>> Suggest
>>>>
>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>
>>>> or
>>>>
>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>
>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>> [...]
>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>> +// and local_err name. We may match something not related to the pattern
>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>> +// in different blocks inside one function, and in one block follow the
>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>> +// functions with the same name (for different configurations).
>>>>
>>>> Context: rule1 matches functions that have all three of
>>>>
>>>> * an Error **errp parameter
>>>>
>>>> * an Error *local_err = NULL variable declaration
>>>>
>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>> local_err, ...) expression, where @errp is the parameter and
>>>> @local_err is the variable.
>>>>
>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>
>>>> 1. This rule can match functions rule1 does not match if there is
>>>> another function with the same name that rule1 does match.
>>>>
>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>> parts of that function use a different @errp or @local_err.
>>>>
>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>> this one. Correct?
>>>>
>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>> don't know it.
>>>
>>> Hmm, what about something like this:
>>>
>>> @rule1 disable optional_qualifier exists@
>>> identifier fn, local_err;
>>> symbol errp;
>>> @@
>>>
>>> fn(..., Error **
>>> - errp
>>> + ___errp_coccinelle_updating___
>>> , ...)
>>> {
>>> ...
>>> Error *local_err = NULL;
>>> ...
>>> (
>>> error_propagate_prepend(errp, local_err, ...);
>>> |
>>> error_propagate(errp, local_err);
>>> )
>>> ...
>>> }
>>>
>>>
>>> [..]
>>>
>>> match symbol ___errp_coccinelle_updating___ in following rules in function header
>>>
>>> [..]
>>>
>>>
>>> @ disable optional_qualifier@
>>> identifier fn, local_err;
>>> symbol errp;
>>> @@
>>>
>>> fn(..., Error **
>>> - ___errp_coccinelle_updating___
>>> + errp
>>> , ...)
>>> {
>>> ...
>>> }
>>>
>>>
>>> - hacky, but seems not more hacky than python detection, and should work better
>>
>> As simple, forceful and unsubtle as a sledgehammer. I like it :)
>>
>
>
> Hmm, not so simple.
>
> It leads to reindenting of function header, which is bad.
Because ___errp_coccinelle_updating___ is longer than errp, I guess.
Try ____?
> Possible solution is instead
>
> fn(...)
> {
> + ___errp_coccinelle_updating___();
>
>
> but this slow down coccinelle. For example, on block.c from ~3s to 1m16s.
>
> .
>
> So, I'm returning to just a warning.
>
> I think something simple like
>
> @@
> identifier rule1.fn;
> position p != rule1.p;
> @@
>
> fn@p(...) {...}
>
> @ script:python@
>
> <print warning>
>
> should work.
Up to you.
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 14:41 ` Markus Armbruster
0 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-11 14:41 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
qemu-devel, Laszlo Ersek, Christian Schoenebeck,
Markus Armbruster, Michael Roth, Greg Kurz, Gerd Hoffmann,
Stefan Hajnoczi, Anthony Perard, xen-devel, Max Reitz,
Philippe Mathieu-Daudé,
Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 11.03.2020 12:38, Markus Armbruster wrote:
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>
>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>> Suggest
>>>>
>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>
>>>> or
>>>>
>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>
>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>> [...]
>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>> +// and local_err name. We may match something not related to the pattern
>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>> +// in different blocks inside one function, and in one block follow the
>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>> +// functions with the same name (for different configurations).
>>>>
>>>> Context: rule1 matches functions that have all three of
>>>>
>>>> * an Error **errp parameter
>>>>
>>>> * an Error *local_err = NULL variable declaration
>>>>
>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>> local_err, ...) expression, where @errp is the parameter and
>>>> @local_err is the variable.
>>>>
>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>
>>>> 1. This rule can match functions rule1 does not match if there is
>>>> another function with the same name that rule1 does match.
>>>>
>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>> parts of that function use a different @errp or @local_err.
>>>>
>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>> this one. Correct?
>>>>
>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>> don't know it.
>>>
>>> Hmm, what about something like this:
>>>
>>> @rule1 disable optional_qualifier exists@
>>> identifier fn, local_err;
>>> symbol errp;
>>> @@
>>>
>>> fn(..., Error **
>>> - errp
>>> + ___errp_coccinelle_updating___
>>> , ...)
>>> {
>>> ...
>>> Error *local_err = NULL;
>>> ...
>>> (
>>> error_propagate_prepend(errp, local_err, ...);
>>> |
>>> error_propagate(errp, local_err);
>>> )
>>> ...
>>> }
>>>
>>>
>>> [..]
>>>
>>> match symbol ___errp_coccinelle_updating___ in following rules in function header
>>>
>>> [..]
>>>
>>>
>>> @ disable optional_qualifier@
>>> identifier fn, local_err;
>>> symbol errp;
>>> @@
>>>
>>> fn(..., Error **
>>> - ___errp_coccinelle_updating___
>>> + errp
>>> , ...)
>>> {
>>> ...
>>> }
>>>
>>>
>>> - hacky, but seems not more hacky than python detection, and should work better
>>
>> As simple, forceful and unsubtle as a sledgehammer. I like it :)
>>
>
>
> Hmm, not so simple.
>
> It leads to reindenting of function header, which is bad.
Because ___errp_coccinelle_updating___ is longer than errp, I guess.
Try ____?
> Possible solution is instead
>
> fn(...)
> {
> + ___errp_coccinelle_updating___();
>
>
> but this slow down coccinelle. For example, on block.c from ~3s to 1m16s.
>
> .
>
> So, I'm returning to just a warning.
>
> I think something simple like
>
> @@
> identifier rule1.fn;
> position p != rule1.p;
> @@
>
> fn@p(...) {...}
>
> @ script:python@
>
> <print warning>
>
> should work.
Up to you.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-11 14:41 ` [Xen-devel] " Markus Armbruster
@ 2020-03-11 14:46 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 14:46 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Laszlo Ersek, Christian Schoenebeck, Michael Roth, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
11.03.2020 17:41, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 11.03.2020 12:38, Markus Armbruster wrote:
>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>
>>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>>> Suggest
>>>>>
>>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>>
>>>>> or
>>>>>
>>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>>
>>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>> [...]
>>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>>> +// and local_err name. We may match something not related to the pattern
>>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>>> +// in different blocks inside one function, and in one block follow the
>>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>>> +// functions with the same name (for different configurations).
>>>>>
>>>>> Context: rule1 matches functions that have all three of
>>>>>
>>>>> * an Error **errp parameter
>>>>>
>>>>> * an Error *local_err = NULL variable declaration
>>>>>
>>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>>> local_err, ...) expression, where @errp is the parameter and
>>>>> @local_err is the variable.
>>>>>
>>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>>
>>>>> 1. This rule can match functions rule1 does not match if there is
>>>>> another function with the same name that rule1 does match.
>>>>>
>>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>>> parts of that function use a different @errp or @local_err.
>>>>>
>>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>>> this one. Correct?
>>>>>
>>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>>> don't know it.
>>>>
>>>> Hmm, what about something like this:
>>>>
>>>> @rule1 disable optional_qualifier exists@
>>>> identifier fn, local_err;
>>>> symbol errp;
>>>> @@
>>>>
>>>> fn(..., Error **
>>>> - errp
>>>> + ___errp_coccinelle_updating___
>>>> , ...)
>>>> {
>>>> ...
>>>> Error *local_err = NULL;
>>>> ...
>>>> (
>>>> error_propagate_prepend(errp, local_err, ...);
>>>> |
>>>> error_propagate(errp, local_err);
>>>> )
>>>> ...
>>>> }
>>>>
>>>>
>>>> [..]
>>>>
>>>> match symbol ___errp_coccinelle_updating___ in following rules in function header
>>>>
>>>> [..]
>>>>
>>>>
>>>> @ disable optional_qualifier@
>>>> identifier fn, local_err;
>>>> symbol errp;
>>>> @@
>>>>
>>>> fn(..., Error **
>>>> - ___errp_coccinelle_updating___
>>>> + errp
>>>> , ...)
>>>> {
>>>> ...
>>>> }
>>>>
>>>>
>>>> - hacky, but seems not more hacky than python detection, and should work better
>>>
>>> As simple, forceful and unsubtle as a sledgehammer. I like it :)
>>>
>>
>>
>> Hmm, not so simple.
>>
>> It leads to reindenting of function header, which is bad.
>
> Because ___errp_coccinelle_updating___ is longer than errp, I guess.
> Try ____?
I'm afraid not. It's because it just adds \n, when I do
...,
- errp
+ ___errp_coccinelle_updating___
,...
>
>> Possible solution is instead
>>
>> fn(...)
>> {
>> + ___errp_coccinelle_updating___();
>>
>>
>> but this slow down coccinelle. For example, on block.c from ~3s to 1m16s.
>>
>> .
>>
>> So, I'm returning to just a warning.
>>
>> I think something simple like
>>
>> @@
>> identifier rule1.fn;
>> position p != rule1.p;
>> @@
>>
>> fn@p(...) {...}
>>
>> @ script:python@
>>
>> <print warning>
>>
>> should work.
>
> Up to you.
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 14:46 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 14:46 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Laszlo Ersek, Christian Schoenebeck, Michael Roth, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
11.03.2020 17:41, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 11.03.2020 12:38, Markus Armbruster wrote:
>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>
>>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>>> Suggest
>>>>>
>>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>>
>>>>> or
>>>>>
>>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>>
>>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>> [...]
>>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>>> +// and local_err name. We may match something not related to the pattern
>>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>>> +// in different blocks inside one function, and in one block follow the
>>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>>> +// functions with the same name (for different configurations).
>>>>>
>>>>> Context: rule1 matches functions that have all three of
>>>>>
>>>>> * an Error **errp parameter
>>>>>
>>>>> * an Error *local_err = NULL variable declaration
>>>>>
>>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>>> local_err, ...) expression, where @errp is the parameter and
>>>>> @local_err is the variable.
>>>>>
>>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>>
>>>>> 1. This rule can match functions rule1 does not match if there is
>>>>> another function with the same name that rule1 does match.
>>>>>
>>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>>> parts of that function use a different @errp or @local_err.
>>>>>
>>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>>> this one. Correct?
>>>>>
>>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>>> don't know it.
>>>>
>>>> Hmm, what about something like this:
>>>>
>>>> @rule1 disable optional_qualifier exists@
>>>> identifier fn, local_err;
>>>> symbol errp;
>>>> @@
>>>>
>>>> fn(..., Error **
>>>> - errp
>>>> + ___errp_coccinelle_updating___
>>>> , ...)
>>>> {
>>>> ...
>>>> Error *local_err = NULL;
>>>> ...
>>>> (
>>>> error_propagate_prepend(errp, local_err, ...);
>>>> |
>>>> error_propagate(errp, local_err);
>>>> )
>>>> ...
>>>> }
>>>>
>>>>
>>>> [..]
>>>>
>>>> match symbol ___errp_coccinelle_updating___ in following rules in function header
>>>>
>>>> [..]
>>>>
>>>>
>>>> @ disable optional_qualifier@
>>>> identifier fn, local_err;
>>>> symbol errp;
>>>> @@
>>>>
>>>> fn(..., Error **
>>>> - ___errp_coccinelle_updating___
>>>> + errp
>>>> , ...)
>>>> {
>>>> ...
>>>> }
>>>>
>>>>
>>>> - hacky, but seems not more hacky than python detection, and should work better
>>>
>>> As simple, forceful and unsubtle as a sledgehammer. I like it :)
>>>
>>
>>
>> Hmm, not so simple.
>>
>> It leads to reindenting of function header, which is bad.
>
> Because ___errp_coccinelle_updating___ is longer than errp, I guess.
> Try ____?
I'm afraid not. It's because it just adds \n, when I do
...,
- errp
+ ___errp_coccinelle_updating___
,...
>
>> Possible solution is instead
>>
>> fn(...)
>> {
>> + ___errp_coccinelle_updating___();
>>
>>
>> but this slow down coccinelle. For example, on block.c from ~3s to 1m16s.
>>
>> .
>>
>> So, I'm returning to just a warning.
>>
>> I think something simple like
>>
>> @@
>> identifier rule1.fn;
>> position p != rule1.p;
>> @@
>>
>> fn@p(...) {...}
>>
>> @ script:python@
>>
>> <print warning>
>>
>> should work.
>
> Up to you.
>
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-11 14:46 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-12 7:23 ` Markus Armbruster
-1 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-12 7:23 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
qemu-devel, Philippe Mathieu-Daudé,
Christian Schoenebeck, Markus Armbruster, Michael Roth,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Laszlo Ersek, Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 11.03.2020 17:41, Markus Armbruster wrote:
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>
>>> 11.03.2020 12:38, Markus Armbruster wrote:
>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>
>>>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>>>> Suggest
>>>>>>
>>>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>>>
>>>>>> or
>>>>>>
>>>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>>>
>>>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>> [...]
>>>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>>>> +// and local_err name. We may match something not related to the pattern
>>>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>>>> +// in different blocks inside one function, and in one block follow the
>>>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>>>> +// functions with the same name (for different configurations).
>>>>>>
>>>>>> Context: rule1 matches functions that have all three of
>>>>>>
>>>>>> * an Error **errp parameter
>>>>>>
>>>>>> * an Error *local_err = NULL variable declaration
>>>>>>
>>>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>>>> local_err, ...) expression, where @errp is the parameter and
>>>>>> @local_err is the variable.
>>>>>>
>>>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>>>
>>>>>> 1. This rule can match functions rule1 does not match if there is
>>>>>> another function with the same name that rule1 does match.
>>>>>>
>>>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>>>> parts of that function use a different @errp or @local_err.
>>>>>>
>>>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>>>> this one. Correct?
>>>>>>
>>>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>>>> don't know it.
>>>>>
>>>>> Hmm, what about something like this:
>>>>>
>>>>> @rule1 disable optional_qualifier exists@
>>>>> identifier fn, local_err;
>>>>> symbol errp;
>>>>> @@
>>>>>
>>>>> fn(..., Error **
>>>>> - errp
>>>>> + ___errp_coccinelle_updating___
>>>>> , ...)
>>>>> {
>>>>> ...
>>>>> Error *local_err = NULL;
>>>>> ...
>>>>> (
>>>>> error_propagate_prepend(errp, local_err, ...);
>>>>> |
>>>>> error_propagate(errp, local_err);
>>>>> )
>>>>> ...
>>>>> }
>>>>>
>>>>>
>>>>> [..]
>>>>>
>>>>> match symbol ___errp_coccinelle_updating___ in following rules in function header
>>>>>
>>>>> [..]
>>>>>
>>>>>
>>>>> @ disable optional_qualifier@
>>>>> identifier fn, local_err;
>>>>> symbol errp;
>>>>> @@
>>>>>
>>>>> fn(..., Error **
>>>>> - ___errp_coccinelle_updating___
>>>>> + errp
>>>>> , ...)
>>>>> {
>>>>> ...
>>>>> }
>>>>>
>>>>>
>>>>> - hacky, but seems not more hacky than python detection, and should work better
>>>>
>>>> As simple, forceful and unsubtle as a sledgehammer. I like it :)
>>>>
>>>
>>>
>>> Hmm, not so simple.
>>>
>>> It leads to reindenting of function header, which is bad.
>>
>> Because ___errp_coccinelle_updating___ is longer than errp, I guess.
>> Try ____?
>
> I'm afraid not. It's because it just adds \n, when I do
>
> ...,
>
> - errp
> + ___errp_coccinelle_updating___
> ,...
I was thinking of something like the appended patch, which in my
(superficial!) testing leaves alone newlines unless lines are long, but
hangs for block.c. Oh well.
diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
index bff274bd6d..492a4db826 100644
--- a/scripts/coccinelle/auto-propagated-errp.cocci
+++ b/scripts/coccinelle/auto-propagated-errp.cocci
@@ -35,12 +35,12 @@
// error_propagate_prepend().
@ depends on !(file in "util/error.c") disable optional_qualifier@
identifier fn;
-identifier _errp != errp;
+identifier _errp;
@@
fn(...,
- Error **_errp
-+ Error **errp
++ Error **____
,...)
{
(
@@ -48,7 +48,7 @@ identifier _errp != errp;
&
<...
- _errp
-+ errp
++ ____
...>
)
}
@@ -63,26 +63,26 @@ identifier _errp != errp;
// all possible control flows (otherwise, it will not match standard pattern
// when error_propagate() call is in if branch).
@ disable optional_qualifier exists@
-identifier fn, local_err, errp;
+identifier fn, local_err;
@@
- fn(..., Error **errp, ...)
+ fn(..., Error **____, ...)
{
+ ERRP_AUTO_PROPAGATE();
... when != ERRP_AUTO_PROPAGATE();
(
- error_append_hint(errp, ...);
+ error_append_hint(____, ...);
|
- error_prepend(errp, ...);
+ error_prepend(____, ...);
|
- error_vprepend(errp, ...);
+ error_vprepend(____, ...);
|
Error *local_err = NULL;
...
(
- error_propagate_prepend(errp, local_err, ...);
+ error_propagate_prepend(____, local_err, ...);
|
- error_propagate(errp, local_err);
+ error_propagate(____, local_err);
)
)
... when any
@@ -92,18 +92,17 @@ identifier fn, local_err, errp;
// Match scenarios with propagation of local error to errp.
@rule1 disable optional_qualifier exists@
identifier fn, local_err;
-symbol errp;
@@
- fn(..., Error **errp, ...)
+ fn(..., Error **____, ...)
{
...
Error *local_err = NULL;
...
(
- error_propagate_prepend(errp, local_err, ...);
+ error_propagate_prepend(____, local_err, ...);
|
- error_propagate(errp, local_err);
+ error_propagate(____, local_err);
)
...
}
@@ -118,7 +117,6 @@ symbol errp;
// without error_propagate() call), coccinelle fails to match this "out: }".
@@
identifier rule1.fn, rule1.local_err, out;
-symbol errp;
@@
fn(...)
@@ -128,7 +126,7 @@ symbol errp;
+ return;
...>
- out:
-- error_propagate(errp, local_err);
+- error_propagate(____, local_err);
}
// Convert most of local_err related staff.
@@ -159,7 +157,6 @@ symbol errp;
@ exists@
identifier rule1.fn, rule1.local_err;
expression list args;
-symbol errp;
@@
fn(...)
@@ -172,30 +169,30 @@ symbol errp;
// Convert error clearing functions
(
- error_free(local_err);
-+ error_free_errp(errp);
++ error_free_errp(____);
|
- error_report_err(local_err);
-+ error_report_errp(errp);
++ error_report_errp(____);
|
- error_reportf_err(local_err, args);
-+ error_reportf_errp(errp, args);
++ error_reportf_errp(____, args);
|
- warn_report_err(local_err);
-+ warn_report_errp(errp);
++ warn_report_errp(____);
|
- warn_reportf_err(local_err, args);
-+ warn_reportf_errp(errp, args);
++ warn_reportf_errp(____, args);
)
?- local_err = NULL;
|
-- error_propagate_prepend(errp, local_err, args);
-+ error_prepend(errp, args);
+- error_propagate_prepend(____, local_err, args);
++ error_prepend(____, args);
|
-- error_propagate(errp, local_err);
+- error_propagate(____, local_err);
|
- &local_err
-+ errp
++ ____
)
...>
}
@@ -205,27 +202,43 @@ symbol errp;
// conflicts with other substitutions in it (at least with "- local_err = NULL").
@@
identifier rule1.fn, rule1.local_err;
-symbol errp;
@@
fn(...)
{
<...
- local_err
-+ *errp
++ *____
...>
}
// Always use the same patter for checking error
@@
identifier rule1.fn;
-symbol errp;
@@
fn(...)
{
<...
-- *errp != NULL
-+ *errp
+- *____ != NULL
++ *____
...>
}
+
+@@
+identifier fn;
+symbol errp;
+@@
+
+ fn(...,
+- Error **____
++ Error **errp
+ ,...)
+ {
+ ...
+ }
+
+@@
+@@
+-____
++errp
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-12 7:23 ` Markus Armbruster
0 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-12 7:23 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
qemu-devel, Philippe Mathieu-Daudé,
Christian Schoenebeck, Markus Armbruster, Michael Roth,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Laszlo Ersek, Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 11.03.2020 17:41, Markus Armbruster wrote:
>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>
>>> 11.03.2020 12:38, Markus Armbruster wrote:
>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>
>>>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>>>> Suggest
>>>>>>
>>>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>>>
>>>>>> or
>>>>>>
>>>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>>>
>>>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>> [...]
>>>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>>>> +// and local_err name. We may match something not related to the pattern
>>>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>>>> +// in different blocks inside one function, and in one block follow the
>>>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>>>> +// functions with the same name (for different configurations).
>>>>>>
>>>>>> Context: rule1 matches functions that have all three of
>>>>>>
>>>>>> * an Error **errp parameter
>>>>>>
>>>>>> * an Error *local_err = NULL variable declaration
>>>>>>
>>>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>>>> local_err, ...) expression, where @errp is the parameter and
>>>>>> @local_err is the variable.
>>>>>>
>>>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>>>
>>>>>> 1. This rule can match functions rule1 does not match if there is
>>>>>> another function with the same name that rule1 does match.
>>>>>>
>>>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>>>> parts of that function use a different @errp or @local_err.
>>>>>>
>>>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>>>> this one. Correct?
>>>>>>
>>>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>>>> don't know it.
>>>>>
>>>>> Hmm, what about something like this:
>>>>>
>>>>> @rule1 disable optional_qualifier exists@
>>>>> identifier fn, local_err;
>>>>> symbol errp;
>>>>> @@
>>>>>
>>>>> fn(..., Error **
>>>>> - errp
>>>>> + ___errp_coccinelle_updating___
>>>>> , ...)
>>>>> {
>>>>> ...
>>>>> Error *local_err = NULL;
>>>>> ...
>>>>> (
>>>>> error_propagate_prepend(errp, local_err, ...);
>>>>> |
>>>>> error_propagate(errp, local_err);
>>>>> )
>>>>> ...
>>>>> }
>>>>>
>>>>>
>>>>> [..]
>>>>>
>>>>> match symbol ___errp_coccinelle_updating___ in following rules in function header
>>>>>
>>>>> [..]
>>>>>
>>>>>
>>>>> @ disable optional_qualifier@
>>>>> identifier fn, local_err;
>>>>> symbol errp;
>>>>> @@
>>>>>
>>>>> fn(..., Error **
>>>>> - ___errp_coccinelle_updating___
>>>>> + errp
>>>>> , ...)
>>>>> {
>>>>> ...
>>>>> }
>>>>>
>>>>>
>>>>> - hacky, but seems not more hacky than python detection, and should work better
>>>>
>>>> As simple, forceful and unsubtle as a sledgehammer. I like it :)
>>>>
>>>
>>>
>>> Hmm, not so simple.
>>>
>>> It leads to reindenting of function header, which is bad.
>>
>> Because ___errp_coccinelle_updating___ is longer than errp, I guess.
>> Try ____?
>
> I'm afraid not. It's because it just adds \n, when I do
>
> ...,
>
> - errp
> + ___errp_coccinelle_updating___
> ,...
I was thinking of something like the appended patch, which in my
(superficial!) testing leaves alone newlines unless lines are long, but
hangs for block.c. Oh well.
diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
index bff274bd6d..492a4db826 100644
--- a/scripts/coccinelle/auto-propagated-errp.cocci
+++ b/scripts/coccinelle/auto-propagated-errp.cocci
@@ -35,12 +35,12 @@
// error_propagate_prepend().
@ depends on !(file in "util/error.c") disable optional_qualifier@
identifier fn;
-identifier _errp != errp;
+identifier _errp;
@@
fn(...,
- Error **_errp
-+ Error **errp
++ Error **____
,...)
{
(
@@ -48,7 +48,7 @@ identifier _errp != errp;
&
<...
- _errp
-+ errp
++ ____
...>
)
}
@@ -63,26 +63,26 @@ identifier _errp != errp;
// all possible control flows (otherwise, it will not match standard pattern
// when error_propagate() call is in if branch).
@ disable optional_qualifier exists@
-identifier fn, local_err, errp;
+identifier fn, local_err;
@@
- fn(..., Error **errp, ...)
+ fn(..., Error **____, ...)
{
+ ERRP_AUTO_PROPAGATE();
... when != ERRP_AUTO_PROPAGATE();
(
- error_append_hint(errp, ...);
+ error_append_hint(____, ...);
|
- error_prepend(errp, ...);
+ error_prepend(____, ...);
|
- error_vprepend(errp, ...);
+ error_vprepend(____, ...);
|
Error *local_err = NULL;
...
(
- error_propagate_prepend(errp, local_err, ...);
+ error_propagate_prepend(____, local_err, ...);
|
- error_propagate(errp, local_err);
+ error_propagate(____, local_err);
)
)
... when any
@@ -92,18 +92,17 @@ identifier fn, local_err, errp;
// Match scenarios with propagation of local error to errp.
@rule1 disable optional_qualifier exists@
identifier fn, local_err;
-symbol errp;
@@
- fn(..., Error **errp, ...)
+ fn(..., Error **____, ...)
{
...
Error *local_err = NULL;
...
(
- error_propagate_prepend(errp, local_err, ...);
+ error_propagate_prepend(____, local_err, ...);
|
- error_propagate(errp, local_err);
+ error_propagate(____, local_err);
)
...
}
@@ -118,7 +117,6 @@ symbol errp;
// without error_propagate() call), coccinelle fails to match this "out: }".
@@
identifier rule1.fn, rule1.local_err, out;
-symbol errp;
@@
fn(...)
@@ -128,7 +126,7 @@ symbol errp;
+ return;
...>
- out:
-- error_propagate(errp, local_err);
+- error_propagate(____, local_err);
}
// Convert most of local_err related staff.
@@ -159,7 +157,6 @@ symbol errp;
@ exists@
identifier rule1.fn, rule1.local_err;
expression list args;
-symbol errp;
@@
fn(...)
@@ -172,30 +169,30 @@ symbol errp;
// Convert error clearing functions
(
- error_free(local_err);
-+ error_free_errp(errp);
++ error_free_errp(____);
|
- error_report_err(local_err);
-+ error_report_errp(errp);
++ error_report_errp(____);
|
- error_reportf_err(local_err, args);
-+ error_reportf_errp(errp, args);
++ error_reportf_errp(____, args);
|
- warn_report_err(local_err);
-+ warn_report_errp(errp);
++ warn_report_errp(____);
|
- warn_reportf_err(local_err, args);
-+ warn_reportf_errp(errp, args);
++ warn_reportf_errp(____, args);
)
?- local_err = NULL;
|
-- error_propagate_prepend(errp, local_err, args);
-+ error_prepend(errp, args);
+- error_propagate_prepend(____, local_err, args);
++ error_prepend(____, args);
|
-- error_propagate(errp, local_err);
+- error_propagate(____, local_err);
|
- &local_err
-+ errp
++ ____
)
...>
}
@@ -205,27 +202,43 @@ symbol errp;
// conflicts with other substitutions in it (at least with "- local_err = NULL").
@@
identifier rule1.fn, rule1.local_err;
-symbol errp;
@@
fn(...)
{
<...
- local_err
-+ *errp
++ *____
...>
}
// Always use the same patter for checking error
@@
identifier rule1.fn;
-symbol errp;
@@
fn(...)
{
<...
-- *errp != NULL
-+ *errp
+- *____ != NULL
++ *____
...>
}
+
+@@
+identifier fn;
+symbol errp;
+@@
+
+ fn(...,
+- Error **____
++ Error **errp
+ ,...)
+ {
+ ...
+ }
+
+@@
+@@
+-____
++errp
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-12 7:23 ` [Xen-devel] " Markus Armbruster
@ 2020-03-12 7:42 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-12 7:42 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
12.03.2020 10:23, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 11.03.2020 17:41, Markus Armbruster wrote:
>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>
>>>> 11.03.2020 12:38, Markus Armbruster wrote:
>>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>>
>>>>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>>>>> Suggest
>>>>>>>
>>>>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>>>>
>>>>>>> or
>>>>>>>
>>>>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>>>>
>>>>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>> [...]
>>>>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>>>>> +// and local_err name. We may match something not related to the pattern
>>>>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>>>>> +// in different blocks inside one function, and in one block follow the
>>>>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>>>>> +// functions with the same name (for different configurations).
>>>>>>>
>>>>>>> Context: rule1 matches functions that have all three of
>>>>>>>
>>>>>>> * an Error **errp parameter
>>>>>>>
>>>>>>> * an Error *local_err = NULL variable declaration
>>>>>>>
>>>>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>>>>> local_err, ...) expression, where @errp is the parameter and
>>>>>>> @local_err is the variable.
>>>>>>>
>>>>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>>>>
>>>>>>> 1. This rule can match functions rule1 does not match if there is
>>>>>>> another function with the same name that rule1 does match.
>>>>>>>
>>>>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>>>>> parts of that function use a different @errp or @local_err.
>>>>>>>
>>>>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>>>>> this one. Correct?
>>>>>>>
>>>>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>>>>> don't know it.
>>>>>>
>>>>>> Hmm, what about something like this:
>>>>>>
>>>>>> @rule1 disable optional_qualifier exists@
>>>>>> identifier fn, local_err;
>>>>>> symbol errp;
>>>>>> @@
>>>>>>
>>>>>> fn(..., Error **
>>>>>> - errp
>>>>>> + ___errp_coccinelle_updating___
>>>>>> , ...)
>>>>>> {
>>>>>> ...
>>>>>> Error *local_err = NULL;
>>>>>> ...
>>>>>> (
>>>>>> error_propagate_prepend(errp, local_err, ...);
>>>>>> |
>>>>>> error_propagate(errp, local_err);
>>>>>> )
>>>>>> ...
>>>>>> }
>>>>>>
>>>>>>
>>>>>> [..]
>>>>>>
>>>>>> match symbol ___errp_coccinelle_updating___ in following rules in function header
>>>>>>
>>>>>> [..]
>>>>>>
>>>>>>
>>>>>> @ disable optional_qualifier@
>>>>>> identifier fn, local_err;
>>>>>> symbol errp;
>>>>>> @@
>>>>>>
>>>>>> fn(..., Error **
>>>>>> - ___errp_coccinelle_updating___
>>>>>> + errp
>>>>>> , ...)
>>>>>> {
>>>>>> ...
>>>>>> }
>>>>>>
>>>>>>
>>>>>> - hacky, but seems not more hacky than python detection, and should work better
>>>>>
>>>>> As simple, forceful and unsubtle as a sledgehammer. I like it :)
>>>>>
>>>>
>>>>
>>>> Hmm, not so simple.
>>>>
>>>> It leads to reindenting of function header, which is bad.
>>>
>>> Because ___errp_coccinelle_updating___ is longer than errp, I guess.
>>> Try ____?
>>
>> I'm afraid not. It's because it just adds \n, when I do
>>
>> ...,
>>
>> - errp
>> + ___errp_coccinelle_updating___
>> ,...
>
> I was thinking of something like the appended patch, which in my
> (superficial!) testing leaves alone newlines unless lines are long, but
> hangs for block.c. Oh well.
Sorry, I didn't say, but I've checked and I was wrong: ___ works fine. But
we need --max-width 80, otherwise coccinelle wraps some lines which fit into
80 characters and produces extra hunks.
So, now I'm preparing v9 with errp->____->errp concept. It differs from your patch and works on block.c
We don't need to switch all errp into ____, it's an extra complication. Changing name only in function
header is enough. And don't worry about broken compilation: it's broken anyway among the rules, and all
is OK after all rules applied.
>
>
> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
> index bff274bd6d..492a4db826 100644
> --- a/scripts/coccinelle/auto-propagated-errp.cocci
> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
> @@ -35,12 +35,12 @@
> // error_propagate_prepend().
> @ depends on !(file in "util/error.c") disable optional_qualifier@
> identifier fn;
> -identifier _errp != errp;
> +identifier _errp;
> @@
>
> fn(...,
> - Error **_errp
> -+ Error **errp
> ++ Error **____
> ,...)
> {
> (
> @@ -48,7 +48,7 @@ identifier _errp != errp;
> &
> <...
> - _errp
> -+ errp
> ++ ____
> ...>
> )
> }
> @@ -63,26 +63,26 @@ identifier _errp != errp;
> // all possible control flows (otherwise, it will not match standard pattern
> // when error_propagate() call is in if branch).
> @ disable optional_qualifier exists@
> -identifier fn, local_err, errp;
> +identifier fn, local_err;
> @@
>
> - fn(..., Error **errp, ...)
> + fn(..., Error **____, ...)
> {
> + ERRP_AUTO_PROPAGATE();
> ... when != ERRP_AUTO_PROPAGATE();
> (
> - error_append_hint(errp, ...);
> + error_append_hint(____, ...);
> |
> - error_prepend(errp, ...);
> + error_prepend(____, ...);
> |
> - error_vprepend(errp, ...);
> + error_vprepend(____, ...);
> |
> Error *local_err = NULL;
> ...
> (
> - error_propagate_prepend(errp, local_err, ...);
> + error_propagate_prepend(____, local_err, ...);
> |
> - error_propagate(errp, local_err);
> + error_propagate(____, local_err);
> )
> )
> ... when any
> @@ -92,18 +92,17 @@ identifier fn, local_err, errp;
> // Match scenarios with propagation of local error to errp.
> @rule1 disable optional_qualifier exists@
> identifier fn, local_err;
> -symbol errp;
> @@
>
> - fn(..., Error **errp, ...)
> + fn(..., Error **____, ...)
> {
> ...
> Error *local_err = NULL;
> ...
> (
> - error_propagate_prepend(errp, local_err, ...);
> + error_propagate_prepend(____, local_err, ...);
> |
> - error_propagate(errp, local_err);
> + error_propagate(____, local_err);
> )
> ...
> }
> @@ -118,7 +117,6 @@ symbol errp;
> // without error_propagate() call), coccinelle fails to match this "out: }".
> @@
> identifier rule1.fn, rule1.local_err, out;
> -symbol errp;
> @@
>
> fn(...)
> @@ -128,7 +126,7 @@ symbol errp;
> + return;
> ...>
> - out:
> -- error_propagate(errp, local_err);
> +- error_propagate(____, local_err);
> }
>
> // Convert most of local_err related staff.
> @@ -159,7 +157,6 @@ symbol errp;
> @ exists@
> identifier rule1.fn, rule1.local_err;
> expression list args;
> -symbol errp;
> @@
>
> fn(...)
> @@ -172,30 +169,30 @@ symbol errp;
> // Convert error clearing functions
> (
> - error_free(local_err);
> -+ error_free_errp(errp);
> ++ error_free_errp(____);
> |
> - error_report_err(local_err);
> -+ error_report_errp(errp);
> ++ error_report_errp(____);
> |
> - error_reportf_err(local_err, args);
> -+ error_reportf_errp(errp, args);
> ++ error_reportf_errp(____, args);
> |
> - warn_report_err(local_err);
> -+ warn_report_errp(errp);
> ++ warn_report_errp(____);
> |
> - warn_reportf_err(local_err, args);
> -+ warn_reportf_errp(errp, args);
> ++ warn_reportf_errp(____, args);
> )
> ?- local_err = NULL;
>
> |
> -- error_propagate_prepend(errp, local_err, args);
> -+ error_prepend(errp, args);
> +- error_propagate_prepend(____, local_err, args);
> ++ error_prepend(____, args);
> |
> -- error_propagate(errp, local_err);
> +- error_propagate(____, local_err);
> |
> - &local_err
> -+ errp
> ++ ____
> )
> ...>
> }
> @@ -205,27 +202,43 @@ symbol errp;
> // conflicts with other substitutions in it (at least with "- local_err = NULL").
> @@
> identifier rule1.fn, rule1.local_err;
> -symbol errp;
> @@
>
> fn(...)
> {
> <...
> - local_err
> -+ *errp
> ++ *____
> ...>
> }
>
> // Always use the same patter for checking error
> @@
> identifier rule1.fn;
> -symbol errp;
> @@
>
> fn(...)
> {
> <...
> -- *errp != NULL
> -+ *errp
> +- *____ != NULL
> ++ *____
> ...>
> }
> +
> +@@
> +identifier fn;
> +symbol errp;
> +@@
> +
> + fn(...,
> +- Error **____
> ++ Error **errp
> + ,...)
> + {
> + ...
> + }
> +
> +@@
> +@@
> +-____
> ++errp
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-12 7:42 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-12 7:42 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
12.03.2020 10:23, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 11.03.2020 17:41, Markus Armbruster wrote:
>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>
>>>> 11.03.2020 12:38, Markus Armbruster wrote:
>>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>>
>>>>>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>>>>> Suggest
>>>>>>>
>>>>>>> scripts: Coccinelle script to use auto-propagated errp
>>>>>>>
>>>>>>> or
>>>>>>>
>>>>>>> scripts: Coccinelle script to use ERRP_AUTO_PROPAGATE()
>>>>>>>
>>>>>>> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>>>>> [...]
>>>>>>>> +// Note, that we update everything related to matched by rule1 function name
>>>>>>>> +// and local_err name. We may match something not related to the pattern
>>>>>>>> +// matched by rule1. For example, local_err may be defined with the same name
>>>>>>>> +// in different blocks inside one function, and in one block follow the
>>>>>>>> +// propagation pattern and in other block doesn't. Or we may have several
>>>>>>>> +// functions with the same name (for different configurations).
>>>>>>>
>>>>>>> Context: rule1 matches functions that have all three of
>>>>>>>
>>>>>>> * an Error **errp parameter
>>>>>>>
>>>>>>> * an Error *local_err = NULL variable declaration
>>>>>>>
>>>>>>> * an error_propagate(errp, local_err) or error_propagate_prepend(errp,
>>>>>>> local_err, ...) expression, where @errp is the parameter and
>>>>>>> @local_err is the variable.
>>>>>>>
>>>>>>> If I understand you correctly, you're pointing out two potential issues:
>>>>>>>
>>>>>>> 1. This rule can match functions rule1 does not match if there is
>>>>>>> another function with the same name that rule1 does match.
>>>>>>>
>>>>>>> 2. This rule matches in the entire function matched by rule1, even when
>>>>>>> parts of that function use a different @errp or @local_err.
>>>>>>>
>>>>>>> I figure these apply to all rules with identifier rule1.fn, not just
>>>>>>> this one. Correct?
>>>>>>>
>>>>>>> Regarding 1. There must be a better way to chain rules together, but I
>>>>>>> don't know it.
>>>>>>
>>>>>> Hmm, what about something like this:
>>>>>>
>>>>>> @rule1 disable optional_qualifier exists@
>>>>>> identifier fn, local_err;
>>>>>> symbol errp;
>>>>>> @@
>>>>>>
>>>>>> fn(..., Error **
>>>>>> - errp
>>>>>> + ___errp_coccinelle_updating___
>>>>>> , ...)
>>>>>> {
>>>>>> ...
>>>>>> Error *local_err = NULL;
>>>>>> ...
>>>>>> (
>>>>>> error_propagate_prepend(errp, local_err, ...);
>>>>>> |
>>>>>> error_propagate(errp, local_err);
>>>>>> )
>>>>>> ...
>>>>>> }
>>>>>>
>>>>>>
>>>>>> [..]
>>>>>>
>>>>>> match symbol ___errp_coccinelle_updating___ in following rules in function header
>>>>>>
>>>>>> [..]
>>>>>>
>>>>>>
>>>>>> @ disable optional_qualifier@
>>>>>> identifier fn, local_err;
>>>>>> symbol errp;
>>>>>> @@
>>>>>>
>>>>>> fn(..., Error **
>>>>>> - ___errp_coccinelle_updating___
>>>>>> + errp
>>>>>> , ...)
>>>>>> {
>>>>>> ...
>>>>>> }
>>>>>>
>>>>>>
>>>>>> - hacky, but seems not more hacky than python detection, and should work better
>>>>>
>>>>> As simple, forceful and unsubtle as a sledgehammer. I like it :)
>>>>>
>>>>
>>>>
>>>> Hmm, not so simple.
>>>>
>>>> It leads to reindenting of function header, which is bad.
>>>
>>> Because ___errp_coccinelle_updating___ is longer than errp, I guess.
>>> Try ____?
>>
>> I'm afraid not. It's because it just adds \n, when I do
>>
>> ...,
>>
>> - errp
>> + ___errp_coccinelle_updating___
>> ,...
>
> I was thinking of something like the appended patch, which in my
> (superficial!) testing leaves alone newlines unless lines are long, but
> hangs for block.c. Oh well.
Sorry, I didn't say, but I've checked and I was wrong: ___ works fine. But
we need --max-width 80, otherwise coccinelle wraps some lines which fit into
80 characters and produces extra hunks.
So, now I'm preparing v9 with errp->____->errp concept. It differs from your patch and works on block.c
We don't need to switch all errp into ____, it's an extra complication. Changing name only in function
header is enough. And don't worry about broken compilation: it's broken anyway among the rules, and all
is OK after all rules applied.
>
>
> diff --git a/scripts/coccinelle/auto-propagated-errp.cocci b/scripts/coccinelle/auto-propagated-errp.cocci
> index bff274bd6d..492a4db826 100644
> --- a/scripts/coccinelle/auto-propagated-errp.cocci
> +++ b/scripts/coccinelle/auto-propagated-errp.cocci
> @@ -35,12 +35,12 @@
> // error_propagate_prepend().
> @ depends on !(file in "util/error.c") disable optional_qualifier@
> identifier fn;
> -identifier _errp != errp;
> +identifier _errp;
> @@
>
> fn(...,
> - Error **_errp
> -+ Error **errp
> ++ Error **____
> ,...)
> {
> (
> @@ -48,7 +48,7 @@ identifier _errp != errp;
> &
> <...
> - _errp
> -+ errp
> ++ ____
> ...>
> )
> }
> @@ -63,26 +63,26 @@ identifier _errp != errp;
> // all possible control flows (otherwise, it will not match standard pattern
> // when error_propagate() call is in if branch).
> @ disable optional_qualifier exists@
> -identifier fn, local_err, errp;
> +identifier fn, local_err;
> @@
>
> - fn(..., Error **errp, ...)
> + fn(..., Error **____, ...)
> {
> + ERRP_AUTO_PROPAGATE();
> ... when != ERRP_AUTO_PROPAGATE();
> (
> - error_append_hint(errp, ...);
> + error_append_hint(____, ...);
> |
> - error_prepend(errp, ...);
> + error_prepend(____, ...);
> |
> - error_vprepend(errp, ...);
> + error_vprepend(____, ...);
> |
> Error *local_err = NULL;
> ...
> (
> - error_propagate_prepend(errp, local_err, ...);
> + error_propagate_prepend(____, local_err, ...);
> |
> - error_propagate(errp, local_err);
> + error_propagate(____, local_err);
> )
> )
> ... when any
> @@ -92,18 +92,17 @@ identifier fn, local_err, errp;
> // Match scenarios with propagation of local error to errp.
> @rule1 disable optional_qualifier exists@
> identifier fn, local_err;
> -symbol errp;
> @@
>
> - fn(..., Error **errp, ...)
> + fn(..., Error **____, ...)
> {
> ...
> Error *local_err = NULL;
> ...
> (
> - error_propagate_prepend(errp, local_err, ...);
> + error_propagate_prepend(____, local_err, ...);
> |
> - error_propagate(errp, local_err);
> + error_propagate(____, local_err);
> )
> ...
> }
> @@ -118,7 +117,6 @@ symbol errp;
> // without error_propagate() call), coccinelle fails to match this "out: }".
> @@
> identifier rule1.fn, rule1.local_err, out;
> -symbol errp;
> @@
>
> fn(...)
> @@ -128,7 +126,7 @@ symbol errp;
> + return;
> ...>
> - out:
> -- error_propagate(errp, local_err);
> +- error_propagate(____, local_err);
> }
>
> // Convert most of local_err related staff.
> @@ -159,7 +157,6 @@ symbol errp;
> @ exists@
> identifier rule1.fn, rule1.local_err;
> expression list args;
> -symbol errp;
> @@
>
> fn(...)
> @@ -172,30 +169,30 @@ symbol errp;
> // Convert error clearing functions
> (
> - error_free(local_err);
> -+ error_free_errp(errp);
> ++ error_free_errp(____);
> |
> - error_report_err(local_err);
> -+ error_report_errp(errp);
> ++ error_report_errp(____);
> |
> - error_reportf_err(local_err, args);
> -+ error_reportf_errp(errp, args);
> ++ error_reportf_errp(____, args);
> |
> - warn_report_err(local_err);
> -+ warn_report_errp(errp);
> ++ warn_report_errp(____);
> |
> - warn_reportf_err(local_err, args);
> -+ warn_reportf_errp(errp, args);
> ++ warn_reportf_errp(____, args);
> )
> ?- local_err = NULL;
>
> |
> -- error_propagate_prepend(errp, local_err, args);
> -+ error_prepend(errp, args);
> +- error_propagate_prepend(____, local_err, args);
> ++ error_prepend(____, args);
> |
> -- error_propagate(errp, local_err);
> +- error_propagate(____, local_err);
> |
> - &local_err
> -+ errp
> ++ ____
> )
> ...>
> }
> @@ -205,27 +202,43 @@ symbol errp;
> // conflicts with other substitutions in it (at least with "- local_err = NULL").
> @@
> identifier rule1.fn, rule1.local_err;
> -symbol errp;
> @@
>
> fn(...)
> {
> <...
> - local_err
> -+ *errp
> ++ *____
> ...>
> }
>
> // Always use the same patter for checking error
> @@
> identifier rule1.fn;
> -symbol errp;
> @@
>
> fn(...)
> {
> <...
> -- *errp != NULL
> -+ *errp
> +- *____ != NULL
> ++ *____
> ...>
> }
> +
> +@@
> +identifier fn;
> +symbol errp;
> +@@
> +
> + fn(...,
> +- Error **____
> ++ Error **errp
> + ,...)
> + {
> + ...
> + }
> +
> +@@
> +@@
> +-____
> ++errp
>
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-09 9:56 ` [Xen-devel] " Markus Armbruster
@ 2020-03-11 8:35 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 8:35 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
09.03.2020 12:56, Markus Armbruster wrote:
>> +
>> +// Convert error clearing functions
> Suggest: Ensure @local_err is cleared on free
But there is no local_err after conversion
>
>> +(
>> +- error_free(local_err);
>> ++ error_free_errp(errp);
>> +|
>> +- error_report_err(local_err);
>> ++ error_report_errp(errp);
>> +|
>> +- error_reportf_err(local_err, args);
>> ++ error_reportf_errp(errp, args);
>> +|
>> +- warn_report_err(local_err);
>> ++ warn_report_errp(errp);
>> +|
>> +- warn_reportf_err(local_err, args);
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 8:35 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 8:35 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
09.03.2020 12:56, Markus Armbruster wrote:
>> +
>> +// Convert error clearing functions
> Suggest: Ensure @local_err is cleared on free
But there is no local_err after conversion
>
>> +(
>> +- error_free(local_err);
>> ++ error_free_errp(errp);
>> +|
>> +- error_report_err(local_err);
>> ++ error_report_errp(errp);
>> +|
>> +- error_reportf_err(local_err, args);
>> ++ error_reportf_errp(errp, args);
>> +|
>> +- warn_report_err(local_err);
>> ++ warn_report_errp(errp);
>> +|
>> +- warn_reportf_err(local_err, args);
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-11 8:35 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-11 9:33 ` Markus Armbruster
-1 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-11 9:33 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
qemu-devel, Philippe Mathieu-Daudé,
Christian Schoenebeck, Markus Armbruster, Michael Roth,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Laszlo Ersek, Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 09.03.2020 12:56, Markus Armbruster wrote:
>>> +
>>> +// Convert error clearing functions
>> Suggest: Ensure @local_err is cleared on free
>
> But there is no local_err after conversion
True. Hmm. What about this:
// Convert calls to error_free(), possibly indirect
// In addition to replacing @local_err by *errp, we have to clear *errp
// to avoid use-after-free in the automatic error propagation.
>>> +(
>>> +- error_free(local_err);
>>> ++ error_free_errp(errp);
>>> +|
>>> +- error_report_err(local_err);
>>> ++ error_report_errp(errp);
>>> +|
>>> +- error_reportf_err(local_err, args);
>>> ++ error_reportf_errp(errp, args);
>>> +|
>>> +- warn_report_err(local_err);
>>> ++ warn_report_errp(errp);
>>> +|
>>> +- warn_reportf_err(local_err, args);
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 9:33 ` Markus Armbruster
0 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-11 9:33 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
qemu-devel, Philippe Mathieu-Daudé,
Christian Schoenebeck, Markus Armbruster, Michael Roth,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Laszlo Ersek, Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> 09.03.2020 12:56, Markus Armbruster wrote:
>>> +
>>> +// Convert error clearing functions
>> Suggest: Ensure @local_err is cleared on free
>
> But there is no local_err after conversion
True. Hmm. What about this:
// Convert calls to error_free(), possibly indirect
// In addition to replacing @local_err by *errp, we have to clear *errp
// to avoid use-after-free in the automatic error propagation.
>>> +(
>>> +- error_free(local_err);
>>> ++ error_free_errp(errp);
>>> +|
>>> +- error_report_err(local_err);
>>> ++ error_report_errp(errp);
>>> +|
>>> +- error_reportf_err(local_err, args);
>>> ++ error_reportf_errp(errp, args);
>>> +|
>>> +- warn_report_err(local_err);
>>> ++ warn_report_errp(errp);
>>> +|
>>> +- warn_reportf_err(local_err, args);
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-11 9:33 ` [Xen-devel] " Markus Armbruster
@ 2020-03-11 9:49 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 9:49 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
11.03.2020 12:33, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>> +
>>>> +// Convert error clearing functions
>>> Suggest: Ensure @local_err is cleared on free
>>
>> But there is no local_err after conversion
>
> True. Hmm. What about this:
>
> // Convert calls to error_free(), possibly indirect
> // In addition to replacing @local_err by *errp, we have to clear *errp
> // to avoid use-after-free in the automatic error propagation.
>
OK
>>>> +(
>>>> +- error_free(local_err);
>>>> ++ error_free_errp(errp);
>>>> +|
>>>> +- error_report_err(local_err);
>>>> ++ error_report_errp(errp);
>>>> +|
>>>> +- error_reportf_err(local_err, args);
>>>> ++ error_reportf_errp(errp, args);
>>>> +|
>>>> +- warn_report_err(local_err);
>>>> ++ warn_report_errp(errp);
>>>> +|
>>>> +- warn_reportf_err(local_err, args);
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 9:49 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 9:49 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Paul Durrant,
Philippe Mathieu-Daudé,
Christian Schoenebeck, Michael Roth, qemu-devel, Greg Kurz,
Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard, xen-devel,
Max Reitz, Laszlo Ersek, Stefan Berger
11.03.2020 12:33, Markus Armbruster wrote:
> Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
>
>> 09.03.2020 12:56, Markus Armbruster wrote:
>>>> +
>>>> +// Convert error clearing functions
>>> Suggest: Ensure @local_err is cleared on free
>>
>> But there is no local_err after conversion
>
> True. Hmm. What about this:
>
> // Convert calls to error_free(), possibly indirect
> // In addition to replacing @local_err by *errp, we have to clear *errp
> // to avoid use-after-free in the automatic error propagation.
>
OK
>>>> +(
>>>> +- error_free(local_err);
>>>> ++ error_free_errp(errp);
>>>> +|
>>>> +- error_report_err(local_err);
>>>> ++ error_report_errp(errp);
>>>> +|
>>>> +- error_reportf_err(local_err, args);
>>>> ++ error_reportf_errp(errp, args);
>>>> +|
>>>> +- warn_report_err(local_err);
>>>> ++ warn_report_errp(errp);
>>>> +|
>>>> +- warn_reportf_err(local_err, args);
>
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-11 9:53 ` Markus Armbruster
-1 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-11 9:53 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
I think a v9 makes sense now.
If any of the improvement ideas should turn into time sinks for you,
let's talk. We don't need perfection, we only need to get to the point
where we trust the script to do what we believe it does, understand its
limitations, and know how to compensate for them.
Right now I'm optimistic v9 will be ready for merging, perhaps with some
minor tweaking in my tree.
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 9:53 ` Markus Armbruster
0 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-11 9:53 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
I think a v9 makes sense now.
If any of the improvement ideas should turn into time sinks for you,
let's talk. We don't need perfection, we only need to get to the point
where we trust the script to do what we believe it does, understand its
limitations, and know how to compensate for them.
Right now I'm optimistic v9 will be ready for merging, perhaps with some
minor tweaking in my tree.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
2020-03-11 9:53 ` [Xen-devel] " Markus Armbruster
@ 2020-03-11 10:11 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 10:11 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
11.03.2020 12:53, Markus Armbruster wrote:
> I think a v9 makes sense now.
>
> If any of the improvement ideas should turn into time sinks for you,
> let's talk. We don't need perfection, we only need to get to the point
> where we trust the script to do what we believe it does, understand its
> limitations, and know how to compensate for them.
>
> Right now I'm optimistic v9 will be ready for merging, perhaps with some
> minor tweaking in my tree.
>
Good. I hope, I'll resend today.
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [Xen-devel] [PATCH v8 02/10] scripts: add coccinelle script to use auto propagated errp
@ 2020-03-11 10:11 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-11 10:11 UTC (permalink / raw)
To: Markus Armbruster
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
11.03.2020 12:53, Markus Armbruster wrote:
> I think a v9 makes sense now.
>
> If any of the improvement ideas should turn into time sinks for you,
> let's talk. We don't need perfection, we only need to get to the point
> where we trust the script to do what we believe it does, understand its
> limitations, and know how to compensate for them.
>
> Right now I'm optimistic v9 will be ready for merging, perhaps with some
> minor tweaking in my tree.
>
Good. I hope, I'll resend today.
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* [PATCH v8 03/10] hw/sd/ssi-sd: fix error handling in ssi_sd_realize
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
` (2 preceding siblings ...)
(?)
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel; +Cc: Philippe Mathieu-Daudé, vsementsov, armbru
It's wrong to use same err object as errp parameter for several
function calls without intermediate checking for error: we'll crash if
try to set err object twice.
Fix that, using new ERRP_AUTO_PROPAGATE macro.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
hw/sd/ssi-sd.c | 26 +++++++++++++++++++-------
1 file changed, 19 insertions(+), 7 deletions(-)
diff --git a/hw/sd/ssi-sd.c b/hw/sd/ssi-sd.c
index 91db069212..bc44e1a0f5 100644
--- a/hw/sd/ssi-sd.c
+++ b/hw/sd/ssi-sd.c
@@ -241,10 +241,10 @@ static const VMStateDescription vmstate_ssi_sd = {
static void ssi_sd_realize(SSISlave *d, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
ssi_sd_state *s = FROM_SSI_SLAVE(ssi_sd_state, d);
DeviceState *carddev;
DriveInfo *dinfo;
- Error *err = NULL;
qbus_create_inplace(&s->sdbus, sizeof(s->sdbus), TYPE_SD_BUS,
DEVICE(d), "sd-bus");
@@ -254,14 +254,26 @@ static void ssi_sd_realize(SSISlave *d, Error **errp)
dinfo = drive_get_next(IF_SD);
carddev = qdev_create(BUS(&s->sdbus), TYPE_SD_CARD);
if (dinfo) {
- qdev_prop_set_drive(carddev, "drive", blk_by_legacy_dinfo(dinfo), &err);
+ qdev_prop_set_drive(carddev, "drive", blk_by_legacy_dinfo(dinfo), errp);
+ if (*errp) {
+ goto fail;
+ }
+ }
+
+ object_property_set_bool(OBJECT(carddev), true, "spi", errp);
+ if (*errp) {
+ goto fail;
}
- object_property_set_bool(OBJECT(carddev), true, "spi", &err);
- object_property_set_bool(OBJECT(carddev), true, "realized", &err);
- if (err) {
- error_setg(errp, "failed to init SD card: %s", error_get_pretty(err));
- return;
+
+ object_property_set_bool(OBJECT(carddev), true, "realized", errp);
+ if (*errp) {
+ goto fail;
}
+
+ return;
+
+fail:
+ error_prepend(errp, "failed to init SD card: ");
}
static void ssi_sd_reset(DeviceState *dev)
--
2.21.0
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 04/10] SD (Secure Card): introduce ERRP_AUTO_PROPAGATE
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
` (3 preceding siblings ...)
(?)
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Philippe Mathieu-Daudé, armbru, Greg Kurz
If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call). Fix such cases.
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
&error_fatal, this means that we don't break error_abort
(we'll abort on error_set, not on error_propagate)
This commit is generated by command
sed -n '/^SD (Secure Card)$/,/^$/{s/^F: //p}' \
MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80
Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
hw/sd/sdhci-pci.c | 7 +++----
hw/sd/sdhci.c | 21 +++++++++------------
2 files changed, 12 insertions(+), 16 deletions(-)
diff --git a/hw/sd/sdhci-pci.c b/hw/sd/sdhci-pci.c
index 19fa8bd8ed..ceaf3ea35b 100644
--- a/hw/sd/sdhci-pci.c
+++ b/hw/sd/sdhci-pci.c
@@ -29,13 +29,12 @@ static Property sdhci_pci_properties[] = {
static void sdhci_pci_realize(PCIDevice *dev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
SDHCIState *s = PCI_SDHCI(dev);
- Error *local_err = NULL;
sdhci_initfn(s);
- sdhci_common_realize(s, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ sdhci_common_realize(s, errp);
+ if (*errp) {
return;
}
diff --git a/hw/sd/sdhci.c b/hw/sd/sdhci.c
index de63ffb037..c25ac3c099 100644
--- a/hw/sd/sdhci.c
+++ b/hw/sd/sdhci.c
@@ -1288,7 +1288,7 @@ static const MemoryRegionOps sdhci_mmio_ops = {
static void sdhci_init_readonly_registers(SDHCIState *s, Error **errp)
{
- Error *local_err = NULL;
+ ERRP_AUTO_PROPAGATE();
switch (s->sd_spec_version) {
case 2 ... 3:
@@ -1299,9 +1299,8 @@ static void sdhci_init_readonly_registers(SDHCIState *s, Error **errp)
}
s->version = (SDHC_HCVER_VENDOR << 8) | (s->sd_spec_version - 1);
- sdhci_check_capareg(s, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ sdhci_check_capareg(s, errp);
+ if (*errp) {
return;
}
}
@@ -1332,11 +1331,10 @@ void sdhci_uninitfn(SDHCIState *s)
void sdhci_common_realize(SDHCIState *s, Error **errp)
{
- Error *local_err = NULL;
+ ERRP_AUTO_PROPAGATE();
- sdhci_init_readonly_registers(s, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ sdhci_init_readonly_registers(s, errp);
+ if (*errp) {
return;
}
s->buf_maxsz = sdhci_get_fifolen(s);
@@ -1456,13 +1454,12 @@ static void sdhci_sysbus_finalize(Object *obj)
static void sdhci_sysbus_realize(DeviceState *dev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
SDHCIState *s = SYSBUS_SDHCI(dev);
SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
- Error *local_err = NULL;
- sdhci_common_realize(s, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ sdhci_common_realize(s, errp);
+ if (*errp) {
return;
}
--
2.21.0
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 05/10] pflash: introduce ERRP_AUTO_PROPAGATE
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
` (4 preceding siblings ...)
(?)
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, qemu-block, armbru, Max Reitz, Greg Kurz,
Philippe Mathieu-Daudé
If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call). Fix such cases.
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
&error_fatal, this means that we don't break error_abort
(we'll abort on error_set, not on error_propagate)
This commit is generated by command
sed -n '/^Parallel NOR Flash devices$/,/^$/{s/^F: //p}' \
MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80
Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
hw/block/pflash_cfi01.c | 7 +++----
hw/block/pflash_cfi02.c | 7 +++----
2 files changed, 6 insertions(+), 8 deletions(-)
diff --git a/hw/block/pflash_cfi01.c b/hw/block/pflash_cfi01.c
index 24f3bce7ef..a5755e6119 100644
--- a/hw/block/pflash_cfi01.c
+++ b/hw/block/pflash_cfi01.c
@@ -700,12 +700,12 @@ static const MemoryRegionOps pflash_cfi01_ops = {
static void pflash_cfi01_realize(DeviceState *dev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
PFlashCFI01 *pfl = PFLASH_CFI01(dev);
uint64_t total_len;
int ret;
uint64_t blocks_per_device, sector_len_per_device, device_len;
int num_devices;
- Error *local_err = NULL;
if (pfl->sector_len == 0) {
error_setg(errp, "attribute \"sector-length\" not specified or zero.");
@@ -739,9 +739,8 @@ static void pflash_cfi01_realize(DeviceState *dev, Error **errp)
&pfl->mem, OBJECT(dev),
&pflash_cfi01_ops,
pfl,
- pfl->name, total_len, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ pfl->name, total_len, errp);
+ if (*errp) {
return;
}
diff --git a/hw/block/pflash_cfi02.c b/hw/block/pflash_cfi02.c
index 12f18d401a..3da9f56391 100644
--- a/hw/block/pflash_cfi02.c
+++ b/hw/block/pflash_cfi02.c
@@ -718,9 +718,9 @@ static const MemoryRegionOps pflash_cfi02_ops = {
static void pflash_cfi02_realize(DeviceState *dev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
PFlashCFI02 *pfl = PFLASH_CFI02(dev);
int ret;
- Error *local_err = NULL;
if (pfl->uniform_sector_len == 0 && pfl->sector_len[0] == 0) {
error_setg(errp, "attribute \"sector-length\" not specified or zero.");
@@ -786,9 +786,8 @@ static void pflash_cfi02_realize(DeviceState *dev, Error **errp)
memory_region_init_rom_device(&pfl->orig_mem, OBJECT(pfl),
&pflash_cfi02_ops, pfl, pfl->name,
- pfl->chip_len, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ pfl->chip_len, errp);
+ if (*errp) {
return;
}
--
2.21.0
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 06/10] fw_cfg: introduce ERRP_AUTO_PROPAGATE
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
` (5 preceding siblings ...)
(?)
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Philippe Mathieu-Daudé,
armbru, Greg Kurz, Gerd Hoffmann, Laszlo Ersek
If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call). Fix such cases.
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
&error_fatal, this means that we don't break error_abort
(we'll abort on error_set, not on error_propagate)
This commit is generated by command
sed -n '/^Firmware configuration (fw_cfg)$/,/^$/{s/^F: //p}' \
MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80
Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
hw/nvram/fw_cfg.c | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/hw/nvram/fw_cfg.c b/hw/nvram/fw_cfg.c
index 179b302f01..22ff2be0c0 100644
--- a/hw/nvram/fw_cfg.c
+++ b/hw/nvram/fw_cfg.c
@@ -1142,12 +1142,11 @@ static Property fw_cfg_io_properties[] = {
static void fw_cfg_io_realize(DeviceState *dev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
FWCfgIoState *s = FW_CFG_IO(dev);
- Error *local_err = NULL;
- fw_cfg_file_slots_allocate(FW_CFG(s), &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ fw_cfg_file_slots_allocate(FW_CFG(s), errp);
+ if (*errp) {
return;
}
@@ -1193,14 +1192,13 @@ static Property fw_cfg_mem_properties[] = {
static void fw_cfg_mem_realize(DeviceState *dev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
FWCfgMemState *s = FW_CFG_MEM(dev);
SysBusDevice *sbd = SYS_BUS_DEVICE(dev);
const MemoryRegionOps *data_ops = &fw_cfg_data_mem_ops;
- Error *local_err = NULL;
- fw_cfg_file_slots_allocate(FW_CFG(s), &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ fw_cfg_file_slots_allocate(FW_CFG(s), errp);
+ if (*errp) {
return;
}
--
2.21.0
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 07/10] virtio-9p: introduce ERRP_AUTO_PROPAGATE
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
` (6 preceding siblings ...)
(?)
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
2020-03-08 18:55 ` Christian Schoenebeck
-1 siblings, 1 reply; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Christian Schoenebeck, armbru, Greg Kurz
If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call). Fix such cases.
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
&error_fatal, this means that we don't break error_abort
(we'll abort on error_set, not on error_propagate)
This commit is generated by command
sed -n '/^virtio-9p$/,/^$/{s/^F: //p}' MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80
Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Acked-by: Greg Kurz <groug@kaod.org>
---
hw/9pfs/9p-local.c | 12 +++++-------
hw/9pfs/9p.c | 1 +
2 files changed, 6 insertions(+), 7 deletions(-)
diff --git a/hw/9pfs/9p-local.c b/hw/9pfs/9p-local.c
index 54e012e5b4..0361e0c0b4 100644
--- a/hw/9pfs/9p-local.c
+++ b/hw/9pfs/9p-local.c
@@ -1479,10 +1479,10 @@ static void error_append_security_model_hint(Error *const *errp)
static int local_parse_opts(QemuOpts *opts, FsDriverEntry *fse, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
const char *sec_model = qemu_opt_get(opts, "security_model");
const char *path = qemu_opt_get(opts, "path");
const char *multidevs = qemu_opt_get(opts, "multidevs");
- Error *local_err = NULL;
if (!sec_model) {
error_setg(errp, "security_model property not set");
@@ -1516,11 +1516,10 @@ static int local_parse_opts(QemuOpts *opts, FsDriverEntry *fse, Error **errp)
fse->export_flags &= ~V9FS_FORBID_MULTIDEVS;
fse->export_flags &= ~V9FS_REMAP_INODES;
} else {
- error_setg(&local_err, "invalid multidevs property '%s'",
+ error_setg(errp, "invalid multidevs property '%s'",
multidevs);
- error_append_hint(&local_err, "Valid options are: multidevs="
+ error_append_hint(errp, "Valid options are: multidevs="
"[remap|forbid|warn]\n");
- error_propagate(errp, local_err);
return -1;
}
}
@@ -1530,9 +1529,8 @@ static int local_parse_opts(QemuOpts *opts, FsDriverEntry *fse, Error **errp)
return -1;
}
- if (fsdev_throttle_parse_opts(opts, &fse->fst, &local_err)) {
- error_propagate_prepend(errp, local_err,
- "invalid throttle configuration: ");
+ if (fsdev_throttle_parse_opts(opts, &fse->fst, errp)) {
+ error_prepend(errp, "invalid throttle configuration: ");
return -1;
}
diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
index 9e046f7acb..3aa6a57f3a 100644
--- a/hw/9pfs/9p.c
+++ b/hw/9pfs/9p.c
@@ -4023,6 +4023,7 @@ void pdu_submit(V9fsPDU *pdu, P9MsgHeader *hdr)
int v9fs_device_realize_common(V9fsState *s, const V9fsTransport *t,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
int i, len;
struct stat stat;
FsDriverEntry *fse;
--
2.21.0
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [PATCH v8 07/10] virtio-9p: introduce ERRP_AUTO_PROPAGATE
2020-03-06 5:15 ` [PATCH v8 07/10] virtio-9p: " Vladimir Sementsov-Ogievskiy
@ 2020-03-08 18:55 ` Christian Schoenebeck
0 siblings, 0 replies; 77+ messages in thread
From: Christian Schoenebeck @ 2020-03-08 18:55 UTC (permalink / raw)
To: qemu-devel; +Cc: Vladimir Sementsov-Ogievskiy, Kevin Wolf, armbru, Greg Kurz
On Freitag, 6. März 2020 06:15:33 CET Vladimir Sementsov-Ogievskiy wrote:
> If we want to add some info to errp (by error_prepend() or
> error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
> Otherwise, this info will not be added when errp == &error_fatal
> (the program will exit prior to the error_append_hint() or
> error_prepend() call). Fix such cases.
>
> If we want to check error after errp-function call, we need to
> introduce local_err and then propagate it to errp. Instead, use
> ERRP_AUTO_PROPAGATE macro, benefits are:
> 1. No need of explicit error_propagate call
> 2. No need of explicit local_err variable: use errp directly
> 3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
> &error_fatal, this means that we don't break error_abort
> (we'll abort on error_set, not on error_propagate)
>
> This commit is generated by command
>
> sed -n '/^virtio-9p$/,/^$/{s/^F: //p}' MAINTAINERS | \
> xargs git ls-files | grep '\.[hc]$' | \
> xargs spatch \
> --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> --macro-file scripts/cocci-macro-file.h \
> --in-place --no-show-diff --max-width 80
>
> Reported-by: Kevin Wolf <kwolf@redhat.com>
> Reported-by: Greg Kurz <groug@kaod.org>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> Acked-by: Greg Kurz <groug@kaod.org>
Reviewed-by: Christian Schoenebeck <qemu_oss@crudebyte.com>
> ---
> hw/9pfs/9p-local.c | 12 +++++-------
> hw/9pfs/9p.c | 1 +
> 2 files changed, 6 insertions(+), 7 deletions(-)
>
> diff --git a/hw/9pfs/9p-local.c b/hw/9pfs/9p-local.c
> index 54e012e5b4..0361e0c0b4 100644
> --- a/hw/9pfs/9p-local.c
> +++ b/hw/9pfs/9p-local.c
> @@ -1479,10 +1479,10 @@ static void error_append_security_model_hint(Error
> *const *errp)
>
> static int local_parse_opts(QemuOpts *opts, FsDriverEntry *fse, Error
> **errp) {
> + ERRP_AUTO_PROPAGATE();
> const char *sec_model = qemu_opt_get(opts, "security_model");
> const char *path = qemu_opt_get(opts, "path");
> const char *multidevs = qemu_opt_get(opts, "multidevs");
> - Error *local_err = NULL;
>
> if (!sec_model) {
> error_setg(errp, "security_model property not set");
> @@ -1516,11 +1516,10 @@ static int local_parse_opts(QemuOpts *opts,
> FsDriverEntry *fse, Error **errp) fse->export_flags &=
> ~V9FS_FORBID_MULTIDEVS;
> fse->export_flags &= ~V9FS_REMAP_INODES;
> } else {
> - error_setg(&local_err, "invalid multidevs property '%s'",
> + error_setg(errp, "invalid multidevs property '%s'",
> multidevs);
> - error_append_hint(&local_err, "Valid options are: multidevs="
> + error_append_hint(errp, "Valid options are: multidevs="
> "[remap|forbid|warn]\n");
> - error_propagate(errp, local_err);
> return -1;
> }
> }
> @@ -1530,9 +1529,8 @@ static int local_parse_opts(QemuOpts *opts,
> FsDriverEntry *fse, Error **errp) return -1;
> }
>
> - if (fsdev_throttle_parse_opts(opts, &fse->fst, &local_err)) {
> - error_propagate_prepend(errp, local_err,
> - "invalid throttle configuration: ");
> + if (fsdev_throttle_parse_opts(opts, &fse->fst, errp)) {
> + error_prepend(errp, "invalid throttle configuration: ");
> return -1;
> }
>
> diff --git a/hw/9pfs/9p.c b/hw/9pfs/9p.c
> index 9e046f7acb..3aa6a57f3a 100644
> --- a/hw/9pfs/9p.c
> +++ b/hw/9pfs/9p.c
> @@ -4023,6 +4023,7 @@ void pdu_submit(V9fsPDU *pdu, P9MsgHeader *hdr)
> int v9fs_device_realize_common(V9fsState *s, const V9fsTransport *t,
> Error **errp)
> {
> + ERRP_AUTO_PROPAGATE();
> int i, len;
> struct stat stat;
> FsDriverEntry *fse;
^ permalink raw reply [flat|nested] 77+ messages in thread
* [PATCH v8 08/10] TPM: introduce ERRP_AUTO_PROPAGATE
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
` (7 preceding siblings ...)
(?)
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Berger, vsementsov, armbru, Greg Kurz
If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call). Fix such cases.
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
&error_fatal, this means that we don't break error_abort
(we'll abort on error_set, not on error_propagate)
This commit is generated by command
sed -n '/^TPM$/,/^$/{s/^F: //p}' MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80
Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Stefan Berger <stefanb@linux.ibm.com>
---
hw/tpm/tpm_util.c | 7 +++----
tpm.c | 7 +++----
2 files changed, 6 insertions(+), 8 deletions(-)
diff --git a/hw/tpm/tpm_util.c b/hw/tpm/tpm_util.c
index c0a0f3d71f..66e0e0c7f3 100644
--- a/hw/tpm/tpm_util.c
+++ b/hw/tpm/tpm_util.c
@@ -47,8 +47,8 @@ static void get_tpm(Object *obj, Visitor *v, const char *name, void *opaque,
static void set_tpm(Object *obj, Visitor *v, const char *name, void *opaque,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
DeviceState *dev = DEVICE(obj);
- Error *local_err = NULL;
Property *prop = opaque;
TPMBackend *s, **be = qdev_get_prop_ptr(dev, prop);
char *str;
@@ -58,9 +58,8 @@ static void set_tpm(Object *obj, Visitor *v, const char *name, void *opaque,
return;
}
- visit_type_str(v, name, &str, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ visit_type_str(v, name, &str, errp);
+ if (*errp) {
return;
}
diff --git a/tpm.c b/tpm.c
index 9c9e20bbb7..359ebb7f68 100644
--- a/tpm.c
+++ b/tpm.c
@@ -81,11 +81,11 @@ TPMBackend *qemu_find_tpm_be(const char *id)
static int tpm_init_tpmdev(void *dummy, QemuOpts *opts, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
const char *value;
const char *id;
const TPMBackendClass *be;
TPMBackend *drv;
- Error *local_err = NULL;
int i;
if (!QLIST_EMPTY(&tpm_backends)) {
@@ -116,9 +116,8 @@ static int tpm_init_tpmdev(void *dummy, QemuOpts *opts, Error **errp)
}
/* validate backend specific opts */
- qemu_opts_validate(opts, be->opts, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ qemu_opts_validate(opts, be->opts, errp);
+ if (*errp) {
return 1;
}
--
2.21.0
^ permalink raw reply related [flat|nested] 77+ messages in thread* [PATCH v8 09/10] nbd: introduce ERRP_AUTO_PROPAGATE
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
` (8 preceding siblings ...)
(?)
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
2020-03-06 12:45 ` Eric Blake
-1 siblings, 1 reply; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, qemu-block, armbru, Max Reitz, Greg Kurz
If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call). Fix such cases.
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
&error_fatal, this means that we don't break error_abort
(we'll abort on error_set, not on error_propagate)
This commit is generated by command
sed -n '/^Network Block Device (NBD)$/,/^$/{s/^F: //p}' \
MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80
Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
include/block/nbd.h | 1 +
block/nbd.c | 21 +++++++++------------
nbd/client.c | 5 +++++
nbd/server.c | 5 +++++
4 files changed, 20 insertions(+), 12 deletions(-)
diff --git a/include/block/nbd.h b/include/block/nbd.h
index 7f46932d80..4ab8d917ed 100644
--- a/include/block/nbd.h
+++ b/include/block/nbd.h
@@ -360,6 +360,7 @@ void nbd_server_start(SocketAddress *addr, const char *tls_creds,
static inline int nbd_read(QIOChannel *ioc, void *buffer, size_t size,
const char *desc, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
int ret = qio_channel_read_all(ioc, buffer, size, errp) < 0 ? -EIO : 0;
if (ret < 0) {
diff --git a/block/nbd.c b/block/nbd.c
index 976be76647..70274c31de 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -1410,16 +1410,15 @@ static void nbd_client_close(BlockDriverState *bs)
static QIOChannelSocket *nbd_establish_connection(SocketAddress *saddr,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
QIOChannelSocket *sioc;
- Error *local_err = NULL;
sioc = qio_channel_socket_new();
qio_channel_set_name(QIO_CHANNEL(sioc), "nbd-client");
- qio_channel_socket_connect_sync(sioc, saddr, &local_err);
- if (local_err) {
+ qio_channel_socket_connect_sync(sioc, saddr, errp);
+ if (*errp) {
object_unref(OBJECT(sioc));
- error_propagate(errp, local_err);
return NULL;
}
@@ -1725,10 +1724,10 @@ static bool nbd_process_legacy_socket_options(QDict *output_options,
static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
SocketAddress *saddr = NULL;
QDict *addr = NULL;
Visitor *iv = NULL;
- Error *local_err = NULL;
qdict_extract_subqdict(options, &addr, "server.");
if (!qdict_size(addr)) {
@@ -1741,9 +1740,8 @@ static SocketAddress *nbd_config(BDRVNBDState *s, QDict *options,
goto done;
}
- visit_type_SocketAddress(iv, NULL, &saddr, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ visit_type_SocketAddress(iv, NULL, &saddr, errp);
+ if (*errp) {
goto done;
}
@@ -1836,15 +1834,14 @@ static QemuOptsList nbd_runtime_opts = {
static int nbd_process_options(BlockDriverState *bs, QDict *options,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
BDRVNBDState *s = bs->opaque;
QemuOpts *opts;
- Error *local_err = NULL;
int ret = -EINVAL;
opts = qemu_opts_create(&nbd_runtime_opts, NULL, 0, &error_abort);
- qemu_opts_absorb_qdict(opts, options, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ qemu_opts_absorb_qdict(opts, options, errp);
+ if (*errp) {
goto error;
}
diff --git a/nbd/client.c b/nbd/client.c
index ba173108ba..e258ef3f7e 100644
--- a/nbd/client.c
+++ b/nbd/client.c
@@ -68,6 +68,7 @@ static int nbd_send_option_request(QIOChannel *ioc, uint32_t opt,
uint32_t len, const char *data,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
NBDOption req;
QEMU_BUILD_BUG_ON(sizeof(req) != 16);
@@ -153,6 +154,7 @@ static int nbd_receive_option_reply(QIOChannel *ioc, uint32_t opt,
static int nbd_handle_reply_err(QIOChannel *ioc, NBDOptionReply *reply,
bool strict, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
g_autofree char *msg = NULL;
if (!(reply->type & (1 << 31))) {
@@ -337,6 +339,7 @@ static int nbd_receive_list(QIOChannel *ioc, char **name, char **description,
static int nbd_opt_info_or_go(QIOChannel *ioc, uint32_t opt,
NBDExportInfo *info, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
NBDOptionReply reply;
uint32_t len = strlen(info->name);
uint16_t type;
@@ -882,6 +885,7 @@ static int nbd_start_negotiate(AioContext *aio_context, QIOChannel *ioc,
bool structured_reply, bool *zeroes,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
uint64_t magic;
trace_nbd_start_negotiate(tlscreds, hostname ? hostname : "<null>");
@@ -1017,6 +1021,7 @@ int nbd_receive_negotiate(AioContext *aio_context, QIOChannel *ioc,
const char *hostname, QIOChannel **outioc,
NBDExportInfo *info, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
int result;
bool zeroes;
bool base_allocation = info->base_allocation;
diff --git a/nbd/server.c b/nbd/server.c
index 11a31094ff..ffc0cc69a7 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -211,6 +211,7 @@ static int GCC_FMT_ATTR(4, 0)
nbd_negotiate_send_rep_verr(NBDClient *client, uint32_t type,
Error **errp, const char *fmt, va_list va)
{
+ ERRP_AUTO_PROPAGATE();
g_autofree char *msg = NULL;
int ret;
size_t len;
@@ -369,6 +370,7 @@ static int nbd_opt_read_name(NBDClient *client, char **name, uint32_t *length,
static int nbd_negotiate_send_rep_list(NBDClient *client, NBDExport *exp,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
size_t name_len, desc_len;
uint32_t len;
const char *name = exp->name ? exp->name : "";
@@ -432,6 +434,7 @@ static void nbd_check_meta_export(NBDClient *client)
static int nbd_negotiate_handle_export_name(NBDClient *client, bool no_zeroes,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
g_autofree char *name = NULL;
char buf[NBD_REPLY_EXPORT_NAME_SIZE] = "";
size_t len;
@@ -1272,6 +1275,7 @@ static int nbd_negotiate_options(NBDClient *client, Error **errp)
*/
static coroutine_fn int nbd_negotiate(NBDClient *client, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
char buf[NBD_OLDSTYLE_NEGOTIATE_SIZE] = "";
int ret;
@@ -1646,6 +1650,7 @@ void nbd_export_close(NBDExport *exp)
void nbd_export_remove(NBDExport *exp, NbdServerRemoveMode mode, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
if (mode == NBD_SERVER_REMOVE_MODE_HARD || QTAILQ_EMPTY(&exp->clients)) {
nbd_export_close(exp);
return;
--
2.21.0
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [PATCH v8 09/10] nbd: introduce ERRP_AUTO_PROPAGATE
2020-03-06 5:15 ` [PATCH v8 09/10] nbd: " Vladimir Sementsov-Ogievskiy
@ 2020-03-06 12:45 ` Eric Blake
0 siblings, 0 replies; 77+ messages in thread
From: Eric Blake @ 2020-03-06 12:45 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, Max Reitz, armbru, qemu-block, Greg Kurz
On 3/5/20 11:15 PM, Vladimir Sementsov-Ogievskiy wrote:
> If we want to add some info to errp (by error_prepend() or
> error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
> Otherwise, this info will not be added when errp == &error_fatal
> (the program will exit prior to the error_append_hint() or
> error_prepend() call). Fix such cases.
>
> If we want to check error after errp-function call, we need to
> introduce local_err and then propagate it to errp. Instead, use
> ERRP_AUTO_PROPAGATE macro, benefits are:
> 1. No need of explicit error_propagate call
> 2. No need of explicit local_err variable: use errp directly
> 3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
> &error_fatal, this means that we don't break error_abort
> (we'll abort on error_set, not on error_propagate)
>
> This commit is generated by command
>
> sed -n '/^Network Block Device (NBD)$/,/^$/{s/^F: //p}' \
> MAINTAINERS | \
> xargs git ls-files | grep '\.[hc]$' | \
> xargs spatch \
> --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> --macro-file scripts/cocci-macro-file.h \
> --in-place --no-show-diff --max-width 80
>
> Reported-by: Kevin Wolf <kwolf@redhat.com>
> Reported-by: Greg Kurz <groug@kaod.org>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 77+ messages in thread
* [PATCH v8 10/10] xen: introduce ERRP_AUTO_PROPAGATE
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, qemu-block, Paul Durrant, armbru,
Greg Kurz, Stefano Stabellini, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz
If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call). Fix such cases.
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
&error_fatal, this means that we don't break error_abort
(we'll abort on error_set, not on error_propagate)
This commit is generated by command
sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80
Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
hw/block/dataplane/xen-block.c | 17 ++---
hw/block/xen-block.c | 125 ++++++++++++++-------------------
hw/pci-host/xen_igd_pt.c | 7 +-
hw/xen/xen-backend.c | 7 +-
hw/xen/xen-bus.c | 92 ++++++++++--------------
hw/xen/xen-host-pci-device.c | 27 ++++---
hw/xen/xen_pt.c | 25 +++----
hw/xen/xen_pt_config_init.c | 20 +++---
8 files changed, 138 insertions(+), 182 deletions(-)
diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 288a87a814..3a8469fe87 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -739,8 +739,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
unsigned int protocol,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenDevice *xendev = dataplane->xendev;
- Error *local_err = NULL;
unsigned int ring_size;
unsigned int i;
@@ -776,9 +776,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
}
xen_device_set_max_grant_refs(xendev, dataplane->nr_ring_ref,
- &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ errp);
+ if (*errp) {
goto stop;
}
@@ -786,9 +785,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
dataplane->ring_ref,
dataplane->nr_ring_ref,
PROT_READ | PROT_WRITE,
- &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ errp);
+ if (*errp) {
goto stop;
}
@@ -821,9 +819,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
dataplane->event_channel =
xen_device_bind_event_channel(xendev, event_channel,
xen_block_dataplane_event, dataplane,
- &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ errp);
+ if (*errp) {
goto stop;
}
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index 686bbc3f0d..717a80d5b5 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -194,6 +194,7 @@ static const BlockDevOps xen_block_dev_ops = {
static void xen_block_realize(XenDevice *xendev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
XenBlockDeviceClass *blockdev_class =
XEN_BLOCK_DEVICE_GET_CLASS(xendev);
@@ -201,7 +202,6 @@ static void xen_block_realize(XenDevice *xendev, Error **errp)
XenBlockVdev *vdev = &blockdev->props.vdev;
BlockConf *conf = &blockdev->props.conf;
BlockBackend *blk = conf->blk;
- Error *local_err = NULL;
if (vdev->type == XEN_BLOCK_VDEV_TYPE_INVALID) {
error_setg(errp, "vdev property not set");
@@ -211,9 +211,8 @@ static void xen_block_realize(XenDevice *xendev, Error **errp)
trace_xen_block_realize(type, vdev->disk, vdev->partition);
if (blockdev_class->realize) {
- blockdev_class->realize(blockdev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ blockdev_class->realize(blockdev, errp);
+ if (*errp) {
return;
}
}
@@ -283,8 +282,8 @@ static void xen_block_frontend_changed(XenDevice *xendev,
enum xenbus_state frontend_state,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
enum xenbus_state backend_state = xen_device_backend_get_state(xendev);
- Error *local_err = NULL;
switch (frontend_state) {
case XenbusStateInitialised:
@@ -293,15 +292,13 @@ static void xen_block_frontend_changed(XenDevice *xendev,
break;
}
- xen_block_disconnect(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_block_disconnect(xendev, errp);
+ if (*errp) {
break;
}
- xen_block_connect(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_block_connect(xendev, errp);
+ if (*errp) {
break;
}
@@ -314,9 +311,8 @@ static void xen_block_frontend_changed(XenDevice *xendev,
case XenbusStateClosed:
case XenbusStateUnknown:
- xen_block_disconnect(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_block_disconnect(xendev, errp);
+ if (*errp) {
break;
}
@@ -403,10 +399,10 @@ static int vbd_name_to_disk(const char *name, const char **endp,
static void xen_block_set_vdev(Object *obj, Visitor *v, const char *name,
void *opaque, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
DeviceState *dev = DEVICE(obj);
Property *prop = opaque;
XenBlockVdev *vdev = qdev_get_prop_ptr(dev, prop);
- Error *local_err = NULL;
char *str, *p;
const char *end;
@@ -415,9 +411,8 @@ static void xen_block_set_vdev(Object *obj, Visitor *v, const char *name,
return;
}
- visit_type_str(v, name, &str, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ visit_type_str(v, name, &str, errp);
+ if (*errp) {
return;
}
@@ -671,9 +666,9 @@ static void xen_block_blockdev_del(const char *node_name, Error **errp)
static char *xen_block_blockdev_add(const char *id, QDict *qdict,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
const char *driver = qdict_get_try_str(qdict, "driver");
BlockdevOptions *options = NULL;
- Error *local_err = NULL;
char *node_name;
Visitor *v;
@@ -688,18 +683,16 @@ static char *xen_block_blockdev_add(const char *id, QDict *qdict,
trace_xen_block_blockdev_add(node_name);
v = qobject_input_visitor_new(QOBJECT(qdict));
- visit_type_BlockdevOptions(v, NULL, &options, &local_err);
+ visit_type_BlockdevOptions(v, NULL, &options, errp);
visit_free(v);
- if (local_err) {
- error_propagate(errp, local_err);
+ if (*errp) {
goto fail;
}
- qmp_blockdev_add(options, &local_err);
+ qmp_blockdev_add(options, errp);
- if (local_err) {
- error_propagate(errp, local_err);
+ if (*errp) {
goto fail;
}
@@ -718,14 +711,12 @@ fail:
static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
char *node_name = drive->node_name;
if (node_name) {
- Error *local_err = NULL;
-
- xen_block_blockdev_del(node_name, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_block_blockdev_del(node_name, errp);
+ if (*errp) {
return;
}
g_free(node_name);
@@ -739,6 +730,7 @@ static XenBlockDrive *xen_block_drive_create(const char *id,
const char *device_type,
QDict *opts, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
const char *params = qdict_get_try_str(opts, "params");
const char *mode = qdict_get_try_str(opts, "mode");
const char *direct_io_safe = qdict_get_try_str(opts, "direct-io-safe");
@@ -746,7 +738,6 @@ static XenBlockDrive *xen_block_drive_create(const char *id,
char *driver = NULL;
char *filename = NULL;
XenBlockDrive *drive = NULL;
- Error *local_err = NULL;
QDict *file_layer;
QDict *driver_layer;
@@ -825,13 +816,12 @@ static XenBlockDrive *xen_block_drive_create(const char *id,
g_assert(!drive->node_name);
drive->node_name = xen_block_blockdev_add(drive->id, driver_layer,
- &local_err);
+ errp);
qobject_unref(driver_layer);
done:
- if (local_err) {
- error_propagate(errp, local_err);
+ if (*errp) {
xen_block_drive_destroy(drive, NULL);
return NULL;
}
@@ -856,15 +846,13 @@ static void xen_block_iothread_destroy(XenBlockIOThread *iothread,
static XenBlockIOThread *xen_block_iothread_create(const char *id,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBlockIOThread *iothread = g_new(XenBlockIOThread, 1);
- Error *local_err = NULL;
iothread->id = g_strdup(id);
- qmp_object_add(TYPE_IOTHREAD, id, false, NULL, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
-
+ qmp_object_add(TYPE_IOTHREAD, id, false, NULL, errp);
+ if (*errp) {
g_free(iothread->id);
g_free(iothread);
return NULL;
@@ -876,6 +864,7 @@ static XenBlockIOThread *xen_block_iothread_create(const char *id,
static void xen_block_device_create(XenBackendInstance *backend,
QDict *opts, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBus *xenbus = xen_backend_get_bus(backend);
const char *name = xen_backend_get_name(backend);
unsigned long number;
@@ -883,7 +872,6 @@ static void xen_block_device_create(XenBackendInstance *backend,
XenBlockDrive *drive = NULL;
XenBlockIOThread *iothread = NULL;
XenDevice *xendev = NULL;
- Error *local_err = NULL;
const char *type;
XenBlockDevice *blockdev;
@@ -915,52 +903,48 @@ static void xen_block_device_create(XenBackendInstance *backend,
goto fail;
}
- drive = xen_block_drive_create(vdev, device_type, opts, &local_err);
+ drive = xen_block_drive_create(vdev, device_type, opts, errp);
if (!drive) {
- error_propagate_prepend(errp, local_err, "failed to create drive: ");
+ error_prepend(errp, "failed to create drive: ");
goto fail;
}
- iothread = xen_block_iothread_create(vdev, &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to create iothread: ");
+ iothread = xen_block_iothread_create(vdev, errp);
+ if (*errp) {
+ error_prepend(errp, "failed to create iothread: ");
goto fail;
}
xendev = XEN_DEVICE(qdev_create(BUS(xenbus), type));
blockdev = XEN_BLOCK_DEVICE(xendev);
- object_property_set_str(OBJECT(xendev), vdev, "vdev", &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err, "failed to set 'vdev': ");
+ object_property_set_str(OBJECT(xendev), vdev, "vdev", errp);
+ if (*errp) {
+ error_prepend(errp, "failed to set 'vdev': ");
goto fail;
}
object_property_set_str(OBJECT(xendev),
xen_block_drive_get_node_name(drive), "drive",
- &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err, "failed to set 'drive': ");
+ errp);
+ if (*errp) {
+ error_prepend(errp, "failed to set 'drive': ");
goto fail;
}
object_property_set_str(OBJECT(xendev), iothread->id, "iothread",
- &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to set 'iothread': ");
+ errp);
+ if (*errp) {
+ error_prepend(errp, "failed to set 'iothread': ");
goto fail;
}
blockdev->iothread = iothread;
blockdev->drive = drive;
- object_property_set_bool(OBJECT(xendev), true, "realized", &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "realization of device %s failed: ",
- type);
+ object_property_set_bool(OBJECT(xendev), true, "realized", errp);
+ if (*errp) {
+ error_prepend(errp, "realization of device %s failed: ", type);
goto fail;
}
@@ -984,6 +968,7 @@ fail:
static void xen_block_device_destroy(XenBackendInstance *backend,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenDevice *xendev = xen_backend_get_device(backend);
XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
XenBlockVdev *vdev = &blockdev->props.vdev;
@@ -995,23 +980,17 @@ static void xen_block_device_destroy(XenBackendInstance *backend,
object_unparent(OBJECT(xendev));
if (iothread) {
- Error *local_err = NULL;
-
- xen_block_iothread_destroy(iothread, &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to destroy iothread: ");
+ xen_block_iothread_destroy(iothread, errp);
+ if (*errp) {
+ error_prepend(errp, "failed to destroy iothread: ");
return;
}
}
if (drive) {
- Error *local_err = NULL;
-
- xen_block_drive_destroy(drive, &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to destroy drive: ");
+ xen_block_drive_destroy(drive, errp);
+ if (*errp) {
+ error_prepend(errp, "failed to destroy drive: ");
}
}
}
diff --git a/hw/pci-host/xen_igd_pt.c b/hw/pci-host/xen_igd_pt.c
index efcc9347ff..29ade9ca25 100644
--- a/hw/pci-host/xen_igd_pt.c
+++ b/hw/pci-host/xen_igd_pt.c
@@ -79,17 +79,16 @@ static void host_pci_config_read(int pos, int len, uint32_t *val, Error **errp)
static void igd_pt_i440fx_realize(PCIDevice *pci_dev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
uint32_t val = 0;
size_t i;
int pos, len;
- Error *local_err = NULL;
for (i = 0; i < ARRAY_SIZE(igd_host_bridge_infos); i++) {
pos = igd_host_bridge_infos[i].offset;
len = igd_host_bridge_infos[i].len;
- host_pci_config_read(pos, len, &val, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ host_pci_config_read(pos, len, &val, errp);
+ if (*errp) {
return;
}
pci_default_write_config(pci_dev, pos, val, len);
diff --git a/hw/xen/xen-backend.c b/hw/xen/xen-backend.c
index da065f81b7..1cc0694053 100644
--- a/hw/xen/xen-backend.c
+++ b/hw/xen/xen-backend.c
@@ -98,9 +98,9 @@ static void xen_backend_list_remove(XenBackendInstance *backend)
void xen_backend_device_create(XenBus *xenbus, const char *type,
const char *name, QDict *opts, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
const XenBackendImpl *impl = xen_backend_table_lookup(type);
XenBackendInstance *backend;
- Error *local_error = NULL;
if (!impl) {
return;
@@ -110,9 +110,8 @@ void xen_backend_device_create(XenBus *xenbus, const char *type,
backend->xenbus = xenbus;
backend->name = g_strdup(name);
- impl->create(backend, opts, &local_error);
- if (local_error) {
- error_propagate(errp, local_error);
+ impl->create(backend, opts, errp);
+ if (*errp) {
g_free(backend->name);
g_free(backend);
return;
diff --git a/hw/xen/xen-bus.c b/hw/xen/xen-bus.c
index 18237b34ea..97259c964b 100644
--- a/hw/xen/xen-bus.c
+++ b/hw/xen/xen-bus.c
@@ -53,9 +53,9 @@ static char *xen_device_get_frontend_path(XenDevice *xendev)
static void xen_device_unplug(XenDevice *xendev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBus *xenbus = XEN_BUS(qdev_get_parent_bus(DEVICE(xendev)));
const char *type = object_get_typename(OBJECT(xendev));
- Error *local_err = NULL;
xs_transaction_t tid;
trace_xen_device_unplug(type, xendev->name);
@@ -69,14 +69,14 @@ again:
}
xs_node_printf(xenbus->xsh, tid, xendev->backend_path, "online",
- &local_err, "%u", 0);
- if (local_err) {
+ errp, "%u", 0);
+ if (*errp) {
goto abort;
}
xs_node_printf(xenbus->xsh, tid, xendev->backend_path, "state",
- &local_err, "%u", XenbusStateClosing);
- if (local_err) {
+ errp, "%u", XenbusStateClosing);
+ if (*errp) {
goto abort;
}
@@ -96,7 +96,6 @@ abort:
* from ending the transaction.
*/
xs_transaction_end(xenbus->xsh, tid, true);
- error_propagate(errp, local_err);
}
static void xen_bus_print_dev(Monitor *mon, DeviceState *dev, int indent)
@@ -205,15 +204,13 @@ static XenWatch *watch_list_add(XenWatchList *watch_list, const char *node,
const char *key, XenWatchHandler handler,
void *opaque, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenWatch *watch = new_watch(node, key, handler, opaque);
- Error *local_err = NULL;
notifier_list_add(&watch_list->notifiers, &watch->notifier);
- xs_node_watch(watch_list->xsh, node, key, watch->token, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
-
+ xs_node_watch(watch_list->xsh, node, key, watch->token, errp);
+ if (*errp) {
notifier_remove(&watch->notifier);
free_watch(watch);
@@ -255,11 +252,11 @@ static void xen_bus_backend_create(XenBus *xenbus, const char *type,
const char *name, char *path,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
xs_transaction_t tid;
char **key;
QDict *opts;
unsigned int i, n;
- Error *local_err = NULL;
trace_xen_bus_backend_create(type, path);
@@ -314,13 +311,11 @@ again:
return;
}
- xen_backend_device_create(xenbus, type, name, opts, &local_err);
+ xen_backend_device_create(xenbus, type, name, opts, errp);
qobject_unref(opts);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to create '%s' device '%s': ",
- type, name);
+ if (*errp) {
+ error_prepend(errp, "failed to create '%s' device '%s': ", type, name);
}
}
@@ -692,9 +687,9 @@ static void xen_device_remove_watch(XenDevice *xendev, XenWatch *watch,
static void xen_device_backend_create(XenDevice *xendev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBus *xenbus = XEN_BUS(qdev_get_parent_bus(DEVICE(xendev)));
struct xs_permissions perms[2];
- Error *local_err = NULL;
xendev->backend_path = xen_device_get_backend_path(xendev);
@@ -706,30 +701,27 @@ static void xen_device_backend_create(XenDevice *xendev, Error **errp)
g_assert(xenbus->xsh);
xs_node_create(xenbus->xsh, XBT_NULL, xendev->backend_path, perms,
- ARRAY_SIZE(perms), &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to create backend: ");
+ ARRAY_SIZE(perms), errp);
+ if (*errp) {
+ error_prepend(errp, "failed to create backend: ");
return;
}
xendev->backend_state_watch =
xen_device_add_watch(xendev, xendev->backend_path,
"state", xen_device_backend_changed,
- &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to watch backend state: ");
+ errp);
+ if (*errp) {
+ error_prepend(errp, "failed to watch backend state: ");
return;
}
xendev->backend_online_watch =
xen_device_add_watch(xendev, xendev->backend_path,
"online", xen_device_backend_changed,
- &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to watch backend online: ");
+ errp);
+ if (*errp) {
+ error_prepend(errp, "failed to watch backend online: ");
return;
}
}
@@ -866,9 +858,9 @@ static bool xen_device_frontend_exists(XenDevice *xendev)
static void xen_device_frontend_create(XenDevice *xendev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBus *xenbus = XEN_BUS(qdev_get_parent_bus(DEVICE(xendev)));
struct xs_permissions perms[2];
- Error *local_err = NULL;
xendev->frontend_path = xen_device_get_frontend_path(xendev);
@@ -885,20 +877,18 @@ static void xen_device_frontend_create(XenDevice *xendev, Error **errp)
g_assert(xenbus->xsh);
xs_node_create(xenbus->xsh, XBT_NULL, xendev->frontend_path, perms,
- ARRAY_SIZE(perms), &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to create frontend: ");
+ ARRAY_SIZE(perms), errp);
+ if (*errp) {
+ error_prepend(errp, "failed to create frontend: ");
return;
}
}
xendev->frontend_state_watch =
xen_device_add_watch(xendev, xendev->frontend_path, "state",
- xen_device_frontend_changed, &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to watch frontend state: ");
+ xen_device_frontend_changed, errp);
+ if (*errp) {
+ error_prepend(errp, "failed to watch frontend state: ");
}
}
@@ -1247,11 +1237,11 @@ static void xen_device_exit(Notifier *n, void *data)
static void xen_device_realize(DeviceState *dev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenDevice *xendev = XEN_DEVICE(dev);
XenDeviceClass *xendev_class = XEN_DEVICE_GET_CLASS(xendev);
XenBus *xenbus = XEN_BUS(qdev_get_parent_bus(DEVICE(xendev)));
const char *type = object_get_typename(OBJECT(xendev));
- Error *local_err = NULL;
if (xendev->frontend_id == DOMID_INVALID) {
xendev->frontend_id = xen_domid;
@@ -1267,10 +1257,9 @@ static void xen_device_realize(DeviceState *dev, Error **errp)
goto unrealize;
}
- xendev->name = xendev_class->get_name(xendev, &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to get device name: ");
+ xendev->name = xendev_class->get_name(xendev, errp);
+ if (*errp) {
+ error_prepend(errp, "failed to get device name: ");
goto unrealize;
}
@@ -1293,22 +1282,19 @@ static void xen_device_realize(DeviceState *dev, Error **errp)
xendev->feature_grant_copy =
(xengnttab_grant_copy(xendev->xgth, 0, NULL) == 0);
- xen_device_backend_create(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_device_backend_create(xendev, errp);
+ if (*errp) {
goto unrealize;
}
- xen_device_frontend_create(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_device_frontend_create(xendev, errp);
+ if (*errp) {
goto unrealize;
}
if (xendev_class->realize) {
- xendev_class->realize(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xendev_class->realize(xendev, errp);
+ if (*errp) {
goto unrealize;
}
}
diff --git a/hw/xen/xen-host-pci-device.c b/hw/xen/xen-host-pci-device.c
index 1b44dcafaf..02379c341c 100644
--- a/hw/xen/xen-host-pci-device.c
+++ b/hw/xen/xen-host-pci-device.c
@@ -333,8 +333,8 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
uint8_t bus, uint8_t dev, uint8_t func,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
unsigned int v;
- Error *err = NULL;
d->config_fd = -1;
d->domain = domain;
@@ -342,36 +342,36 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
d->dev = dev;
d->func = func;
- xen_host_pci_config_open(d, &err);
- if (err) {
+ xen_host_pci_config_open(d, errp);
+ if (*errp) {
goto error;
}
- xen_host_pci_get_resource(d, &err);
- if (err) {
+ xen_host_pci_get_resource(d, errp);
+ if (*errp) {
goto error;
}
- xen_host_pci_get_hex_value(d, "vendor", &v, &err);
- if (err) {
+ xen_host_pci_get_hex_value(d, "vendor", &v, errp);
+ if (*errp) {
goto error;
}
d->vendor_id = v;
- xen_host_pci_get_hex_value(d, "device", &v, &err);
- if (err) {
+ xen_host_pci_get_hex_value(d, "device", &v, errp);
+ if (*errp) {
goto error;
}
d->device_id = v;
- xen_host_pci_get_dec_value(d, "irq", &v, &err);
- if (err) {
+ xen_host_pci_get_dec_value(d, "irq", &v, errp);
+ if (*errp) {
goto error;
}
d->irq = v;
- xen_host_pci_get_hex_value(d, "class", &v, &err);
- if (err) {
+ xen_host_pci_get_hex_value(d, "class", &v, errp);
+ if (*errp) {
goto error;
}
d->class_code = v;
@@ -381,7 +381,6 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
return;
error:
- error_propagate(errp, err);
if (d->config_fd >= 0) {
close(d->config_fd);
diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
index b91082cb8b..f57b81588e 100644
--- a/hw/xen/xen_pt.c
+++ b/hw/xen/xen_pt.c
@@ -767,12 +767,12 @@ static void xen_pt_destroy(PCIDevice *d) {
static void xen_pt_realize(PCIDevice *d, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenPCIPassthroughState *s = XEN_PT_DEVICE(d);
int i, rc = 0;
uint8_t machine_irq = 0, scratch;
uint16_t cmd = 0;
int pirq = XEN_PT_UNASSIGNED_PIRQ;
- Error *err = NULL;
/* register real device */
XEN_PT_LOG(d, "Assigning real physical device %02x:%02x.%d"
@@ -783,10 +783,9 @@ static void xen_pt_realize(PCIDevice *d, Error **errp)
xen_host_pci_device_get(&s->real_device,
s->hostaddr.domain, s->hostaddr.bus,
s->hostaddr.slot, s->hostaddr.function,
- &err);
- if (err) {
- error_append_hint(&err, "Failed to \"open\" the real pci device");
- error_propagate(errp, err);
+ errp);
+ if (*errp) {
+ error_append_hint(errp, "Failed to \"open\" the real pci device");
return;
}
@@ -813,11 +812,10 @@ static void xen_pt_realize(PCIDevice *d, Error **errp)
return;
}
- xen_pt_setup_vga(s, &s->real_device, &err);
- if (err) {
- error_append_hint(&err, "Setup VGA BIOS of passthrough"
- " GFX failed");
- error_propagate(errp, err);
+ xen_pt_setup_vga(s, &s->real_device, errp);
+ if (*errp) {
+ error_append_hint(errp, "Setup VGA BIOS of passthrough"
+ " GFX failed");
xen_host_pci_device_put(&s->real_device);
return;
}
@@ -830,10 +828,9 @@ static void xen_pt_realize(PCIDevice *d, Error **errp)
xen_pt_register_regions(s, &cmd);
/* reinitialize each config register to be emulated */
- xen_pt_config_init(s, &err);
- if (err) {
- error_append_hint(&err, "PCI Config space initialisation failed");
- error_propagate(errp, err);
+ xen_pt_config_init(s, errp);
+ if (*errp) {
+ error_append_hint(errp, "PCI Config space initialisation failed");
rc = -1;
goto err_out;
}
diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c
index 31ec5add1d..af3fbd1bfb 100644
--- a/hw/xen/xen_pt_config_init.c
+++ b/hw/xen/xen_pt_config_init.c
@@ -2008,8 +2008,8 @@ static void xen_pt_config_reg_init(XenPCIPassthroughState *s,
void xen_pt_config_init(XenPCIPassthroughState *s, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
int i, rc;
- Error *err = NULL;
QLIST_INIT(&s->reg_grps);
@@ -2052,10 +2052,9 @@ void xen_pt_config_init(XenPCIPassthroughState *s, Error **errp)
reg_grp_offset,
®_grp_entry->size);
if (rc < 0) {
- error_setg(&err, "Failed to initialize %d/%zu, type = 0x%x,"
+ error_setg(errp, "Failed to initialize %d/%zu, type = 0x%x,"
" rc: %d", i, ARRAY_SIZE(xen_pt_emu_reg_grps),
xen_pt_emu_reg_grps[i].grp_type, rc);
- error_propagate(errp, err);
xen_pt_config_delete(s);
return;
}
@@ -2068,13 +2067,14 @@ void xen_pt_config_init(XenPCIPassthroughState *s, Error **errp)
/* initialize capability register */
for (j = 0; regs->size != 0; j++, regs++) {
- xen_pt_config_reg_init(s, reg_grp_entry, regs, &err);
- if (err) {
- error_append_hint(&err, "Failed to init register %d"
- " offsets 0x%x in grp_type = 0x%x (%d/%zu)", j,
- regs->offset, xen_pt_emu_reg_grps[i].grp_type,
- i, ARRAY_SIZE(xen_pt_emu_reg_grps));
- error_propagate(errp, err);
+ xen_pt_config_reg_init(s, reg_grp_entry, regs, errp);
+ if (*errp) {
+ error_append_hint(errp, "Failed to init register %d"
+ " offsets 0x%x in grp_type = 0x%x (%d/%zu)",
+ j,
+ regs->offset,
+ xen_pt_emu_reg_grps[i].grp_type,
+ i, ARRAY_SIZE(xen_pt_emu_reg_grps));
xen_pt_config_delete(s);
return;
}
--
2.21.0
^ permalink raw reply related [flat|nested] 77+ messages in thread* [Xen-devel] [PATCH v8 10/10] xen: introduce ERRP_AUTO_PROPAGATE
@ 2020-03-06 5:15 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 5:15 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, qemu-block, Paul Durrant, armbru,
Greg Kurz, Stefano Stabellini, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz
If we want to add some info to errp (by error_prepend() or
error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
Otherwise, this info will not be added when errp == &error_fatal
(the program will exit prior to the error_append_hint() or
error_prepend() call). Fix such cases.
If we want to check error after errp-function call, we need to
introduce local_err and then propagate it to errp. Instead, use
ERRP_AUTO_PROPAGATE macro, benefits are:
1. No need of explicit error_propagate call
2. No need of explicit local_err variable: use errp directly
3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
&error_fatal, this means that we don't break error_abort
(we'll abort on error_set, not on error_propagate)
This commit is generated by command
sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
xargs git ls-files | grep '\.[hc]$' | \
xargs spatch \
--sp-file scripts/coccinelle/auto-propagated-errp.cocci \
--macro-file scripts/cocci-macro-file.h \
--in-place --no-show-diff --max-width 80
Reported-by: Kevin Wolf <kwolf@redhat.com>
Reported-by: Greg Kurz <groug@kaod.org>
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
hw/block/dataplane/xen-block.c | 17 ++---
hw/block/xen-block.c | 125 ++++++++++++++-------------------
hw/pci-host/xen_igd_pt.c | 7 +-
hw/xen/xen-backend.c | 7 +-
hw/xen/xen-bus.c | 92 ++++++++++--------------
hw/xen/xen-host-pci-device.c | 27 ++++---
hw/xen/xen_pt.c | 25 +++----
hw/xen/xen_pt_config_init.c | 20 +++---
8 files changed, 138 insertions(+), 182 deletions(-)
diff --git a/hw/block/dataplane/xen-block.c b/hw/block/dataplane/xen-block.c
index 288a87a814..3a8469fe87 100644
--- a/hw/block/dataplane/xen-block.c
+++ b/hw/block/dataplane/xen-block.c
@@ -739,8 +739,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
unsigned int protocol,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenDevice *xendev = dataplane->xendev;
- Error *local_err = NULL;
unsigned int ring_size;
unsigned int i;
@@ -776,9 +776,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
}
xen_device_set_max_grant_refs(xendev, dataplane->nr_ring_ref,
- &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ errp);
+ if (*errp) {
goto stop;
}
@@ -786,9 +785,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
dataplane->ring_ref,
dataplane->nr_ring_ref,
PROT_READ | PROT_WRITE,
- &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ errp);
+ if (*errp) {
goto stop;
}
@@ -821,9 +819,8 @@ void xen_block_dataplane_start(XenBlockDataPlane *dataplane,
dataplane->event_channel =
xen_device_bind_event_channel(xendev, event_channel,
xen_block_dataplane_event, dataplane,
- &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ errp);
+ if (*errp) {
goto stop;
}
diff --git a/hw/block/xen-block.c b/hw/block/xen-block.c
index 686bbc3f0d..717a80d5b5 100644
--- a/hw/block/xen-block.c
+++ b/hw/block/xen-block.c
@@ -194,6 +194,7 @@ static const BlockDevOps xen_block_dev_ops = {
static void xen_block_realize(XenDevice *xendev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
XenBlockDeviceClass *blockdev_class =
XEN_BLOCK_DEVICE_GET_CLASS(xendev);
@@ -201,7 +202,6 @@ static void xen_block_realize(XenDevice *xendev, Error **errp)
XenBlockVdev *vdev = &blockdev->props.vdev;
BlockConf *conf = &blockdev->props.conf;
BlockBackend *blk = conf->blk;
- Error *local_err = NULL;
if (vdev->type == XEN_BLOCK_VDEV_TYPE_INVALID) {
error_setg(errp, "vdev property not set");
@@ -211,9 +211,8 @@ static void xen_block_realize(XenDevice *xendev, Error **errp)
trace_xen_block_realize(type, vdev->disk, vdev->partition);
if (blockdev_class->realize) {
- blockdev_class->realize(blockdev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ blockdev_class->realize(blockdev, errp);
+ if (*errp) {
return;
}
}
@@ -283,8 +282,8 @@ static void xen_block_frontend_changed(XenDevice *xendev,
enum xenbus_state frontend_state,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
enum xenbus_state backend_state = xen_device_backend_get_state(xendev);
- Error *local_err = NULL;
switch (frontend_state) {
case XenbusStateInitialised:
@@ -293,15 +292,13 @@ static void xen_block_frontend_changed(XenDevice *xendev,
break;
}
- xen_block_disconnect(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_block_disconnect(xendev, errp);
+ if (*errp) {
break;
}
- xen_block_connect(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_block_connect(xendev, errp);
+ if (*errp) {
break;
}
@@ -314,9 +311,8 @@ static void xen_block_frontend_changed(XenDevice *xendev,
case XenbusStateClosed:
case XenbusStateUnknown:
- xen_block_disconnect(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_block_disconnect(xendev, errp);
+ if (*errp) {
break;
}
@@ -403,10 +399,10 @@ static int vbd_name_to_disk(const char *name, const char **endp,
static void xen_block_set_vdev(Object *obj, Visitor *v, const char *name,
void *opaque, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
DeviceState *dev = DEVICE(obj);
Property *prop = opaque;
XenBlockVdev *vdev = qdev_get_prop_ptr(dev, prop);
- Error *local_err = NULL;
char *str, *p;
const char *end;
@@ -415,9 +411,8 @@ static void xen_block_set_vdev(Object *obj, Visitor *v, const char *name,
return;
}
- visit_type_str(v, name, &str, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ visit_type_str(v, name, &str, errp);
+ if (*errp) {
return;
}
@@ -671,9 +666,9 @@ static void xen_block_blockdev_del(const char *node_name, Error **errp)
static char *xen_block_blockdev_add(const char *id, QDict *qdict,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
const char *driver = qdict_get_try_str(qdict, "driver");
BlockdevOptions *options = NULL;
- Error *local_err = NULL;
char *node_name;
Visitor *v;
@@ -688,18 +683,16 @@ static char *xen_block_blockdev_add(const char *id, QDict *qdict,
trace_xen_block_blockdev_add(node_name);
v = qobject_input_visitor_new(QOBJECT(qdict));
- visit_type_BlockdevOptions(v, NULL, &options, &local_err);
+ visit_type_BlockdevOptions(v, NULL, &options, errp);
visit_free(v);
- if (local_err) {
- error_propagate(errp, local_err);
+ if (*errp) {
goto fail;
}
- qmp_blockdev_add(options, &local_err);
+ qmp_blockdev_add(options, errp);
- if (local_err) {
- error_propagate(errp, local_err);
+ if (*errp) {
goto fail;
}
@@ -718,14 +711,12 @@ fail:
static void xen_block_drive_destroy(XenBlockDrive *drive, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
char *node_name = drive->node_name;
if (node_name) {
- Error *local_err = NULL;
-
- xen_block_blockdev_del(node_name, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_block_blockdev_del(node_name, errp);
+ if (*errp) {
return;
}
g_free(node_name);
@@ -739,6 +730,7 @@ static XenBlockDrive *xen_block_drive_create(const char *id,
const char *device_type,
QDict *opts, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
const char *params = qdict_get_try_str(opts, "params");
const char *mode = qdict_get_try_str(opts, "mode");
const char *direct_io_safe = qdict_get_try_str(opts, "direct-io-safe");
@@ -746,7 +738,6 @@ static XenBlockDrive *xen_block_drive_create(const char *id,
char *driver = NULL;
char *filename = NULL;
XenBlockDrive *drive = NULL;
- Error *local_err = NULL;
QDict *file_layer;
QDict *driver_layer;
@@ -825,13 +816,12 @@ static XenBlockDrive *xen_block_drive_create(const char *id,
g_assert(!drive->node_name);
drive->node_name = xen_block_blockdev_add(drive->id, driver_layer,
- &local_err);
+ errp);
qobject_unref(driver_layer);
done:
- if (local_err) {
- error_propagate(errp, local_err);
+ if (*errp) {
xen_block_drive_destroy(drive, NULL);
return NULL;
}
@@ -856,15 +846,13 @@ static void xen_block_iothread_destroy(XenBlockIOThread *iothread,
static XenBlockIOThread *xen_block_iothread_create(const char *id,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBlockIOThread *iothread = g_new(XenBlockIOThread, 1);
- Error *local_err = NULL;
iothread->id = g_strdup(id);
- qmp_object_add(TYPE_IOTHREAD, id, false, NULL, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
-
+ qmp_object_add(TYPE_IOTHREAD, id, false, NULL, errp);
+ if (*errp) {
g_free(iothread->id);
g_free(iothread);
return NULL;
@@ -876,6 +864,7 @@ static XenBlockIOThread *xen_block_iothread_create(const char *id,
static void xen_block_device_create(XenBackendInstance *backend,
QDict *opts, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBus *xenbus = xen_backend_get_bus(backend);
const char *name = xen_backend_get_name(backend);
unsigned long number;
@@ -883,7 +872,6 @@ static void xen_block_device_create(XenBackendInstance *backend,
XenBlockDrive *drive = NULL;
XenBlockIOThread *iothread = NULL;
XenDevice *xendev = NULL;
- Error *local_err = NULL;
const char *type;
XenBlockDevice *blockdev;
@@ -915,52 +903,48 @@ static void xen_block_device_create(XenBackendInstance *backend,
goto fail;
}
- drive = xen_block_drive_create(vdev, device_type, opts, &local_err);
+ drive = xen_block_drive_create(vdev, device_type, opts, errp);
if (!drive) {
- error_propagate_prepend(errp, local_err, "failed to create drive: ");
+ error_prepend(errp, "failed to create drive: ");
goto fail;
}
- iothread = xen_block_iothread_create(vdev, &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to create iothread: ");
+ iothread = xen_block_iothread_create(vdev, errp);
+ if (*errp) {
+ error_prepend(errp, "failed to create iothread: ");
goto fail;
}
xendev = XEN_DEVICE(qdev_create(BUS(xenbus), type));
blockdev = XEN_BLOCK_DEVICE(xendev);
- object_property_set_str(OBJECT(xendev), vdev, "vdev", &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err, "failed to set 'vdev': ");
+ object_property_set_str(OBJECT(xendev), vdev, "vdev", errp);
+ if (*errp) {
+ error_prepend(errp, "failed to set 'vdev': ");
goto fail;
}
object_property_set_str(OBJECT(xendev),
xen_block_drive_get_node_name(drive), "drive",
- &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err, "failed to set 'drive': ");
+ errp);
+ if (*errp) {
+ error_prepend(errp, "failed to set 'drive': ");
goto fail;
}
object_property_set_str(OBJECT(xendev), iothread->id, "iothread",
- &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to set 'iothread': ");
+ errp);
+ if (*errp) {
+ error_prepend(errp, "failed to set 'iothread': ");
goto fail;
}
blockdev->iothread = iothread;
blockdev->drive = drive;
- object_property_set_bool(OBJECT(xendev), true, "realized", &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "realization of device %s failed: ",
- type);
+ object_property_set_bool(OBJECT(xendev), true, "realized", errp);
+ if (*errp) {
+ error_prepend(errp, "realization of device %s failed: ", type);
goto fail;
}
@@ -984,6 +968,7 @@ fail:
static void xen_block_device_destroy(XenBackendInstance *backend,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenDevice *xendev = xen_backend_get_device(backend);
XenBlockDevice *blockdev = XEN_BLOCK_DEVICE(xendev);
XenBlockVdev *vdev = &blockdev->props.vdev;
@@ -995,23 +980,17 @@ static void xen_block_device_destroy(XenBackendInstance *backend,
object_unparent(OBJECT(xendev));
if (iothread) {
- Error *local_err = NULL;
-
- xen_block_iothread_destroy(iothread, &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to destroy iothread: ");
+ xen_block_iothread_destroy(iothread, errp);
+ if (*errp) {
+ error_prepend(errp, "failed to destroy iothread: ");
return;
}
}
if (drive) {
- Error *local_err = NULL;
-
- xen_block_drive_destroy(drive, &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to destroy drive: ");
+ xen_block_drive_destroy(drive, errp);
+ if (*errp) {
+ error_prepend(errp, "failed to destroy drive: ");
}
}
}
diff --git a/hw/pci-host/xen_igd_pt.c b/hw/pci-host/xen_igd_pt.c
index efcc9347ff..29ade9ca25 100644
--- a/hw/pci-host/xen_igd_pt.c
+++ b/hw/pci-host/xen_igd_pt.c
@@ -79,17 +79,16 @@ static void host_pci_config_read(int pos, int len, uint32_t *val, Error **errp)
static void igd_pt_i440fx_realize(PCIDevice *pci_dev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
uint32_t val = 0;
size_t i;
int pos, len;
- Error *local_err = NULL;
for (i = 0; i < ARRAY_SIZE(igd_host_bridge_infos); i++) {
pos = igd_host_bridge_infos[i].offset;
len = igd_host_bridge_infos[i].len;
- host_pci_config_read(pos, len, &val, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ host_pci_config_read(pos, len, &val, errp);
+ if (*errp) {
return;
}
pci_default_write_config(pci_dev, pos, val, len);
diff --git a/hw/xen/xen-backend.c b/hw/xen/xen-backend.c
index da065f81b7..1cc0694053 100644
--- a/hw/xen/xen-backend.c
+++ b/hw/xen/xen-backend.c
@@ -98,9 +98,9 @@ static void xen_backend_list_remove(XenBackendInstance *backend)
void xen_backend_device_create(XenBus *xenbus, const char *type,
const char *name, QDict *opts, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
const XenBackendImpl *impl = xen_backend_table_lookup(type);
XenBackendInstance *backend;
- Error *local_error = NULL;
if (!impl) {
return;
@@ -110,9 +110,8 @@ void xen_backend_device_create(XenBus *xenbus, const char *type,
backend->xenbus = xenbus;
backend->name = g_strdup(name);
- impl->create(backend, opts, &local_error);
- if (local_error) {
- error_propagate(errp, local_error);
+ impl->create(backend, opts, errp);
+ if (*errp) {
g_free(backend->name);
g_free(backend);
return;
diff --git a/hw/xen/xen-bus.c b/hw/xen/xen-bus.c
index 18237b34ea..97259c964b 100644
--- a/hw/xen/xen-bus.c
+++ b/hw/xen/xen-bus.c
@@ -53,9 +53,9 @@ static char *xen_device_get_frontend_path(XenDevice *xendev)
static void xen_device_unplug(XenDevice *xendev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBus *xenbus = XEN_BUS(qdev_get_parent_bus(DEVICE(xendev)));
const char *type = object_get_typename(OBJECT(xendev));
- Error *local_err = NULL;
xs_transaction_t tid;
trace_xen_device_unplug(type, xendev->name);
@@ -69,14 +69,14 @@ again:
}
xs_node_printf(xenbus->xsh, tid, xendev->backend_path, "online",
- &local_err, "%u", 0);
- if (local_err) {
+ errp, "%u", 0);
+ if (*errp) {
goto abort;
}
xs_node_printf(xenbus->xsh, tid, xendev->backend_path, "state",
- &local_err, "%u", XenbusStateClosing);
- if (local_err) {
+ errp, "%u", XenbusStateClosing);
+ if (*errp) {
goto abort;
}
@@ -96,7 +96,6 @@ abort:
* from ending the transaction.
*/
xs_transaction_end(xenbus->xsh, tid, true);
- error_propagate(errp, local_err);
}
static void xen_bus_print_dev(Monitor *mon, DeviceState *dev, int indent)
@@ -205,15 +204,13 @@ static XenWatch *watch_list_add(XenWatchList *watch_list, const char *node,
const char *key, XenWatchHandler handler,
void *opaque, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenWatch *watch = new_watch(node, key, handler, opaque);
- Error *local_err = NULL;
notifier_list_add(&watch_list->notifiers, &watch->notifier);
- xs_node_watch(watch_list->xsh, node, key, watch->token, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
-
+ xs_node_watch(watch_list->xsh, node, key, watch->token, errp);
+ if (*errp) {
notifier_remove(&watch->notifier);
free_watch(watch);
@@ -255,11 +252,11 @@ static void xen_bus_backend_create(XenBus *xenbus, const char *type,
const char *name, char *path,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
xs_transaction_t tid;
char **key;
QDict *opts;
unsigned int i, n;
- Error *local_err = NULL;
trace_xen_bus_backend_create(type, path);
@@ -314,13 +311,11 @@ again:
return;
}
- xen_backend_device_create(xenbus, type, name, opts, &local_err);
+ xen_backend_device_create(xenbus, type, name, opts, errp);
qobject_unref(opts);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to create '%s' device '%s': ",
- type, name);
+ if (*errp) {
+ error_prepend(errp, "failed to create '%s' device '%s': ", type, name);
}
}
@@ -692,9 +687,9 @@ static void xen_device_remove_watch(XenDevice *xendev, XenWatch *watch,
static void xen_device_backend_create(XenDevice *xendev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBus *xenbus = XEN_BUS(qdev_get_parent_bus(DEVICE(xendev)));
struct xs_permissions perms[2];
- Error *local_err = NULL;
xendev->backend_path = xen_device_get_backend_path(xendev);
@@ -706,30 +701,27 @@ static void xen_device_backend_create(XenDevice *xendev, Error **errp)
g_assert(xenbus->xsh);
xs_node_create(xenbus->xsh, XBT_NULL, xendev->backend_path, perms,
- ARRAY_SIZE(perms), &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to create backend: ");
+ ARRAY_SIZE(perms), errp);
+ if (*errp) {
+ error_prepend(errp, "failed to create backend: ");
return;
}
xendev->backend_state_watch =
xen_device_add_watch(xendev, xendev->backend_path,
"state", xen_device_backend_changed,
- &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to watch backend state: ");
+ errp);
+ if (*errp) {
+ error_prepend(errp, "failed to watch backend state: ");
return;
}
xendev->backend_online_watch =
xen_device_add_watch(xendev, xendev->backend_path,
"online", xen_device_backend_changed,
- &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to watch backend online: ");
+ errp);
+ if (*errp) {
+ error_prepend(errp, "failed to watch backend online: ");
return;
}
}
@@ -866,9 +858,9 @@ static bool xen_device_frontend_exists(XenDevice *xendev)
static void xen_device_frontend_create(XenDevice *xendev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenBus *xenbus = XEN_BUS(qdev_get_parent_bus(DEVICE(xendev)));
struct xs_permissions perms[2];
- Error *local_err = NULL;
xendev->frontend_path = xen_device_get_frontend_path(xendev);
@@ -885,20 +877,18 @@ static void xen_device_frontend_create(XenDevice *xendev, Error **errp)
g_assert(xenbus->xsh);
xs_node_create(xenbus->xsh, XBT_NULL, xendev->frontend_path, perms,
- ARRAY_SIZE(perms), &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to create frontend: ");
+ ARRAY_SIZE(perms), errp);
+ if (*errp) {
+ error_prepend(errp, "failed to create frontend: ");
return;
}
}
xendev->frontend_state_watch =
xen_device_add_watch(xendev, xendev->frontend_path, "state",
- xen_device_frontend_changed, &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to watch frontend state: ");
+ xen_device_frontend_changed, errp);
+ if (*errp) {
+ error_prepend(errp, "failed to watch frontend state: ");
}
}
@@ -1247,11 +1237,11 @@ static void xen_device_exit(Notifier *n, void *data)
static void xen_device_realize(DeviceState *dev, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenDevice *xendev = XEN_DEVICE(dev);
XenDeviceClass *xendev_class = XEN_DEVICE_GET_CLASS(xendev);
XenBus *xenbus = XEN_BUS(qdev_get_parent_bus(DEVICE(xendev)));
const char *type = object_get_typename(OBJECT(xendev));
- Error *local_err = NULL;
if (xendev->frontend_id == DOMID_INVALID) {
xendev->frontend_id = xen_domid;
@@ -1267,10 +1257,9 @@ static void xen_device_realize(DeviceState *dev, Error **errp)
goto unrealize;
}
- xendev->name = xendev_class->get_name(xendev, &local_err);
- if (local_err) {
- error_propagate_prepend(errp, local_err,
- "failed to get device name: ");
+ xendev->name = xendev_class->get_name(xendev, errp);
+ if (*errp) {
+ error_prepend(errp, "failed to get device name: ");
goto unrealize;
}
@@ -1293,22 +1282,19 @@ static void xen_device_realize(DeviceState *dev, Error **errp)
xendev->feature_grant_copy =
(xengnttab_grant_copy(xendev->xgth, 0, NULL) == 0);
- xen_device_backend_create(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_device_backend_create(xendev, errp);
+ if (*errp) {
goto unrealize;
}
- xen_device_frontend_create(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xen_device_frontend_create(xendev, errp);
+ if (*errp) {
goto unrealize;
}
if (xendev_class->realize) {
- xendev_class->realize(xendev, &local_err);
- if (local_err) {
- error_propagate(errp, local_err);
+ xendev_class->realize(xendev, errp);
+ if (*errp) {
goto unrealize;
}
}
diff --git a/hw/xen/xen-host-pci-device.c b/hw/xen/xen-host-pci-device.c
index 1b44dcafaf..02379c341c 100644
--- a/hw/xen/xen-host-pci-device.c
+++ b/hw/xen/xen-host-pci-device.c
@@ -333,8 +333,8 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
uint8_t bus, uint8_t dev, uint8_t func,
Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
unsigned int v;
- Error *err = NULL;
d->config_fd = -1;
d->domain = domain;
@@ -342,36 +342,36 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
d->dev = dev;
d->func = func;
- xen_host_pci_config_open(d, &err);
- if (err) {
+ xen_host_pci_config_open(d, errp);
+ if (*errp) {
goto error;
}
- xen_host_pci_get_resource(d, &err);
- if (err) {
+ xen_host_pci_get_resource(d, errp);
+ if (*errp) {
goto error;
}
- xen_host_pci_get_hex_value(d, "vendor", &v, &err);
- if (err) {
+ xen_host_pci_get_hex_value(d, "vendor", &v, errp);
+ if (*errp) {
goto error;
}
d->vendor_id = v;
- xen_host_pci_get_hex_value(d, "device", &v, &err);
- if (err) {
+ xen_host_pci_get_hex_value(d, "device", &v, errp);
+ if (*errp) {
goto error;
}
d->device_id = v;
- xen_host_pci_get_dec_value(d, "irq", &v, &err);
- if (err) {
+ xen_host_pci_get_dec_value(d, "irq", &v, errp);
+ if (*errp) {
goto error;
}
d->irq = v;
- xen_host_pci_get_hex_value(d, "class", &v, &err);
- if (err) {
+ xen_host_pci_get_hex_value(d, "class", &v, errp);
+ if (*errp) {
goto error;
}
d->class_code = v;
@@ -381,7 +381,6 @@ void xen_host_pci_device_get(XenHostPCIDevice *d, uint16_t domain,
return;
error:
- error_propagate(errp, err);
if (d->config_fd >= 0) {
close(d->config_fd);
diff --git a/hw/xen/xen_pt.c b/hw/xen/xen_pt.c
index b91082cb8b..f57b81588e 100644
--- a/hw/xen/xen_pt.c
+++ b/hw/xen/xen_pt.c
@@ -767,12 +767,12 @@ static void xen_pt_destroy(PCIDevice *d) {
static void xen_pt_realize(PCIDevice *d, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
XenPCIPassthroughState *s = XEN_PT_DEVICE(d);
int i, rc = 0;
uint8_t machine_irq = 0, scratch;
uint16_t cmd = 0;
int pirq = XEN_PT_UNASSIGNED_PIRQ;
- Error *err = NULL;
/* register real device */
XEN_PT_LOG(d, "Assigning real physical device %02x:%02x.%d"
@@ -783,10 +783,9 @@ static void xen_pt_realize(PCIDevice *d, Error **errp)
xen_host_pci_device_get(&s->real_device,
s->hostaddr.domain, s->hostaddr.bus,
s->hostaddr.slot, s->hostaddr.function,
- &err);
- if (err) {
- error_append_hint(&err, "Failed to \"open\" the real pci device");
- error_propagate(errp, err);
+ errp);
+ if (*errp) {
+ error_append_hint(errp, "Failed to \"open\" the real pci device");
return;
}
@@ -813,11 +812,10 @@ static void xen_pt_realize(PCIDevice *d, Error **errp)
return;
}
- xen_pt_setup_vga(s, &s->real_device, &err);
- if (err) {
- error_append_hint(&err, "Setup VGA BIOS of passthrough"
- " GFX failed");
- error_propagate(errp, err);
+ xen_pt_setup_vga(s, &s->real_device, errp);
+ if (*errp) {
+ error_append_hint(errp, "Setup VGA BIOS of passthrough"
+ " GFX failed");
xen_host_pci_device_put(&s->real_device);
return;
}
@@ -830,10 +828,9 @@ static void xen_pt_realize(PCIDevice *d, Error **errp)
xen_pt_register_regions(s, &cmd);
/* reinitialize each config register to be emulated */
- xen_pt_config_init(s, &err);
- if (err) {
- error_append_hint(&err, "PCI Config space initialisation failed");
- error_propagate(errp, err);
+ xen_pt_config_init(s, errp);
+ if (*errp) {
+ error_append_hint(errp, "PCI Config space initialisation failed");
rc = -1;
goto err_out;
}
diff --git a/hw/xen/xen_pt_config_init.c b/hw/xen/xen_pt_config_init.c
index 31ec5add1d..af3fbd1bfb 100644
--- a/hw/xen/xen_pt_config_init.c
+++ b/hw/xen/xen_pt_config_init.c
@@ -2008,8 +2008,8 @@ static void xen_pt_config_reg_init(XenPCIPassthroughState *s,
void xen_pt_config_init(XenPCIPassthroughState *s, Error **errp)
{
+ ERRP_AUTO_PROPAGATE();
int i, rc;
- Error *err = NULL;
QLIST_INIT(&s->reg_grps);
@@ -2052,10 +2052,9 @@ void xen_pt_config_init(XenPCIPassthroughState *s, Error **errp)
reg_grp_offset,
®_grp_entry->size);
if (rc < 0) {
- error_setg(&err, "Failed to initialize %d/%zu, type = 0x%x,"
+ error_setg(errp, "Failed to initialize %d/%zu, type = 0x%x,"
" rc: %d", i, ARRAY_SIZE(xen_pt_emu_reg_grps),
xen_pt_emu_reg_grps[i].grp_type, rc);
- error_propagate(errp, err);
xen_pt_config_delete(s);
return;
}
@@ -2068,13 +2067,14 @@ void xen_pt_config_init(XenPCIPassthroughState *s, Error **errp)
/* initialize capability register */
for (j = 0; regs->size != 0; j++, regs++) {
- xen_pt_config_reg_init(s, reg_grp_entry, regs, &err);
- if (err) {
- error_append_hint(&err, "Failed to init register %d"
- " offsets 0x%x in grp_type = 0x%x (%d/%zu)", j,
- regs->offset, xen_pt_emu_reg_grps[i].grp_type,
- i, ARRAY_SIZE(xen_pt_emu_reg_grps));
- error_propagate(errp, err);
+ xen_pt_config_reg_init(s, reg_grp_entry, regs, errp);
+ if (*errp) {
+ error_append_hint(errp, "Failed to init register %d"
+ " offsets 0x%x in grp_type = 0x%x (%d/%zu)",
+ j,
+ regs->offset,
+ xen_pt_emu_reg_grps[i].grp_type,
+ i, ARRAY_SIZE(xen_pt_emu_reg_grps));
xen_pt_config_delete(s);
return;
}
--
2.21.0
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 77+ messages in thread* Re: [PATCH v8 10/10] xen: introduce ERRP_AUTO_PROPAGATE
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-06 9:12 ` Paul Durrant
-1 siblings, 0 replies; 77+ messages in thread
From: Paul Durrant @ 2020-03-06 9:12 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Markus Armbruster,
Greg Kurz, open list:All patches CC here, Stefan Hajnoczi,
Anthony Perard, xen-devel, Max Reitz
On Fri, 6 Mar 2020 at 05:16, Vladimir Sementsov-Ogievskiy
<vsementsov@virtuozzo.com> wrote:
>
> If we want to add some info to errp (by error_prepend() or
> error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
> Otherwise, this info will not be added when errp == &error_fatal
> (the program will exit prior to the error_append_hint() or
> error_prepend() call). Fix such cases.
>
> If we want to check error after errp-function call, we need to
> introduce local_err and then propagate it to errp. Instead, use
> ERRP_AUTO_PROPAGATE macro, benefits are:
> 1. No need of explicit error_propagate call
> 2. No need of explicit local_err variable: use errp directly
> 3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
> &error_fatal, this means that we don't break error_abort
> (we'll abort on error_set, not on error_propagate)
>
> This commit is generated by command
>
> sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
> xargs git ls-files | grep '\.[hc]$' | \
> xargs spatch \
> --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> --macro-file scripts/cocci-macro-file.h \
> --in-place --no-show-diff --max-width 80
>
> Reported-by: Kevin Wolf <kwolf@redhat.com>
> Reported-by: Greg Kurz <groug@kaod.org>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Paul Durrant <paul@xen.org>
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 10/10] xen: introduce ERRP_AUTO_PROPAGATE
@ 2020-03-06 9:12 ` Paul Durrant
0 siblings, 0 replies; 77+ messages in thread
From: Paul Durrant @ 2020-03-06 9:12 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Markus Armbruster,
Greg Kurz, open list:All patches CC here, Stefan Hajnoczi,
Anthony Perard, xen-devel, Max Reitz
On Fri, 6 Mar 2020 at 05:16, Vladimir Sementsov-Ogievskiy
<vsementsov@virtuozzo.com> wrote:
>
> If we want to add some info to errp (by error_prepend() or
> error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
> Otherwise, this info will not be added when errp == &error_fatal
> (the program will exit prior to the error_append_hint() or
> error_prepend() call). Fix such cases.
>
> If we want to check error after errp-function call, we need to
> introduce local_err and then propagate it to errp. Instead, use
> ERRP_AUTO_PROPAGATE macro, benefits are:
> 1. No need of explicit error_propagate call
> 2. No need of explicit local_err variable: use errp directly
> 3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
> &error_fatal, this means that we don't break error_abort
> (we'll abort on error_set, not on error_propagate)
>
> This commit is generated by command
>
> sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
> xargs git ls-files | grep '\.[hc]$' | \
> xargs spatch \
> --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> --macro-file scripts/cocci-macro-file.h \
> --in-place --no-show-diff --max-width 80
>
> Reported-by: Kevin Wolf <kwolf@redhat.com>
> Reported-by: Greg Kurz <groug@kaod.org>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Paul Durrant <paul@xen.org>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [PATCH v8 10/10] xen: introduce ERRP_AUTO_PROPAGATE
2020-03-06 9:12 ` [Xen-devel] " Paul Durrant
@ 2020-03-06 9:18 ` Vladimir Sementsov-Ogievskiy
-1 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 9:18 UTC (permalink / raw)
To: Paul Durrant
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Markus Armbruster,
Greg Kurz, open list:All patches CC here, Stefan Hajnoczi,
Anthony Perard, xen-devel, Max Reitz
06.03.2020 12:12, Paul Durrant wrote:
> On Fri, 6 Mar 2020 at 05:16, Vladimir Sementsov-Ogievskiy
> <vsementsov@virtuozzo.com> wrote:
>>
>> If we want to add some info to errp (by error_prepend() or
>> error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
>> Otherwise, this info will not be added when errp == &error_fatal
>> (the program will exit prior to the error_append_hint() or
>> error_prepend() call). Fix such cases.
>>
>> If we want to check error after errp-function call, we need to
>> introduce local_err and then propagate it to errp. Instead, use
>> ERRP_AUTO_PROPAGATE macro, benefits are:
>> 1. No need of explicit error_propagate call
>> 2. No need of explicit local_err variable: use errp directly
>> 3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
>> &error_fatal, this means that we don't break error_abort
>> (we'll abort on error_set, not on error_propagate)
>>
>> This commit is generated by command
>>
>> sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
>> xargs git ls-files | grep '\.[hc]$' | \
>> xargs spatch \
>> --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>> --macro-file scripts/cocci-macro-file.h \
>> --in-place --no-show-diff --max-width 80
>>
>> Reported-by: Kevin Wolf <kwolf@redhat.com>
>> Reported-by: Greg Kurz <groug@kaod.org>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
> Reviewed-by: Paul Durrant <paul@xen.org>
>
Thanks for reviewing!
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 10/10] xen: introduce ERRP_AUTO_PROPAGATE
@ 2020-03-06 9:18 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 77+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-03-06 9:18 UTC (permalink / raw)
To: Paul Durrant
Cc: Kevin Wolf, Stefano Stabellini, qemu-block, Markus Armbruster,
Greg Kurz, open list:All patches CC here, Stefan Hajnoczi,
Anthony Perard, xen-devel, Max Reitz
06.03.2020 12:12, Paul Durrant wrote:
> On Fri, 6 Mar 2020 at 05:16, Vladimir Sementsov-Ogievskiy
> <vsementsov@virtuozzo.com> wrote:
>>
>> If we want to add some info to errp (by error_prepend() or
>> error_append_hint()), we must use the ERRP_AUTO_PROPAGATE macro.
>> Otherwise, this info will not be added when errp == &error_fatal
>> (the program will exit prior to the error_append_hint() or
>> error_prepend() call). Fix such cases.
>>
>> If we want to check error after errp-function call, we need to
>> introduce local_err and then propagate it to errp. Instead, use
>> ERRP_AUTO_PROPAGATE macro, benefits are:
>> 1. No need of explicit error_propagate call
>> 2. No need of explicit local_err variable: use errp directly
>> 3. ERRP_AUTO_PROPAGATE leaves errp as is if it's not NULL or
>> &error_fatal, this means that we don't break error_abort
>> (we'll abort on error_set, not on error_propagate)
>>
>> This commit is generated by command
>>
>> sed -n '/^X86 Xen CPUs$/,/^$/{s/^F: //p}' MAINTAINERS | \
>> xargs git ls-files | grep '\.[hc]$' | \
>> xargs spatch \
>> --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
>> --macro-file scripts/cocci-macro-file.h \
>> --in-place --no-show-diff --max-width 80
>>
>> Reported-by: Kevin Wolf <kwolf@redhat.com>
>> Reported-by: Greg Kurz <groug@kaod.org>
>> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
> Reviewed-by: Paul Durrant <paul@xen.org>
>
Thanks for reviewing!
--
Best regards,
Vladimir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread
* Re: [PATCH v8 00/10] error: auto propagated local_err part I
2020-03-06 5:15 ` [Xen-devel] " Vladimir Sementsov-Ogievskiy
@ 2020-03-06 15:21 ` Markus Armbruster
-1 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-06 15:21 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> File with errp-cleaning APIs dropped for two reasons:
>
> 1. I'm tired after a 3-days war with coccinelle, and don't want to add more
> patches here.
Oww. In my experience, Coccinelle is both awesome and terrible. I hope
you didn't do all that work just to address minor complaints from me. I
*try* to make it clear where on the spectrum from "I want you to improve
this" to "perhaps you'd like to try this idea" my review comments are,
but I'm certainly not perfect there. Anyway, here we are, so let's look
at the patches.
> 2. Markus noted, that we forget two more functions which needs such wrappers
> and corresponding conversion, so seems better to handle all these things
> in same manner for now.
>
> changes in v8:
>
> 01: - update comments
> - fix bug in macro [Markus]
> - use do {} while(0)
>
> 02: a lot of changes
> - about error propagation: try to update only patterns where we propagate
> local_error to errp. So, patches 09 and 10 changed (wow!)
> - Now clearing functions are not defined. Still, do the conversion, so
> that when script applied where these functions needed, compilation will
> be broken.
> - improve ordering of hunks, and comment everything
>
> 09,10: drop hunks, which are converted for nothing.
Quick eye-over: you no longer wrap nbd_co_receive_cmdread_reply(),
nbd_co_receive_blockstatus_reply(), and xen_bus_realize(). Clearly
better.
> v8 is available at
> https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-auto-local-err-partI-v8
> v7 is available at
> https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-auto-local-err-partI-v7
>
> In these series, there is no commit-per-subsystem script, each generated
> commit is generated in separate.
>
> Still, generating commands are very similar, and looks like
>
> sed -n '/^<Subsystem name>$/,/^$/{s/^F: //p}' MAINTAINERS | \
> xargs git ls-files | grep '\.[hc]$' | \
> xargs spatch \
> --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> --macro-file scripts/cocci-macro-file.h \
> --in-place --no-show-diff --max-width 80
>
> Note, that in each generated commit, generation command is the only
> text, indented by 8 spaces in 'git log -1' output, so, to regenerate all
> commits (for example, after rebase, or change in coccinelle script), you
> may use the following command:
>
> git rebase -x "sh -c \"git show --pretty= --name-only | xargs git checkout HEAD^ -- ; git reset; git log -1 | grep '^ ' | sh\"" HEAD~7
>
> Which will start automated interactive rebase for generated patches,
> which will stop if generated patch changed
> (you may do git commit --amend to apply updated generated changes).
>
> Note:
> git show --pretty= --name-only - lists files, changed in HEAD
> git log -1 | grep '^ ' | sh - rerun generation command of HEAD
>
>
> Check for compilation of changed .c files
> git rebase -x "sh -c \"git show --pretty= --name-only | sed -n 's/\.c$/.o/p' | xargs make -j9\"" HEAD~7
Thanks for the write-up!
^ permalink raw reply [flat|nested] 77+ messages in thread* Re: [Xen-devel] [PATCH v8 00/10] error: auto propagated local_err part I
@ 2020-03-06 15:21 ` Markus Armbruster
0 siblings, 0 replies; 77+ messages in thread
From: Markus Armbruster @ 2020-03-06 15:21 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy
Cc: Kevin Wolf, Stefano Stabellini, Michael Roth, qemu-block,
Paul Durrant, Laszlo Ersek, Christian Schoenebeck, qemu-devel,
Greg Kurz, Gerd Hoffmann, Stefan Hajnoczi, Anthony Perard,
xen-devel, Max Reitz, Philippe Mathieu-Daudé,
Stefan Berger
Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> writes:
> File with errp-cleaning APIs dropped for two reasons:
>
> 1. I'm tired after a 3-days war with coccinelle, and don't want to add more
> patches here.
Oww. In my experience, Coccinelle is both awesome and terrible. I hope
you didn't do all that work just to address minor complaints from me. I
*try* to make it clear where on the spectrum from "I want you to improve
this" to "perhaps you'd like to try this idea" my review comments are,
but I'm certainly not perfect there. Anyway, here we are, so let's look
at the patches.
> 2. Markus noted, that we forget two more functions which needs such wrappers
> and corresponding conversion, so seems better to handle all these things
> in same manner for now.
>
> changes in v8:
>
> 01: - update comments
> - fix bug in macro [Markus]
> - use do {} while(0)
>
> 02: a lot of changes
> - about error propagation: try to update only patterns where we propagate
> local_error to errp. So, patches 09 and 10 changed (wow!)
> - Now clearing functions are not defined. Still, do the conversion, so
> that when script applied where these functions needed, compilation will
> be broken.
> - improve ordering of hunks, and comment everything
>
> 09,10: drop hunks, which are converted for nothing.
Quick eye-over: you no longer wrap nbd_co_receive_cmdread_reply(),
nbd_co_receive_blockstatus_reply(), and xen_bus_realize(). Clearly
better.
> v8 is available at
> https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-auto-local-err-partI-v8
> v7 is available at
> https://src.openvz.org/scm/~vsementsov/qemu.git #tag up-auto-local-err-partI-v7
>
> In these series, there is no commit-per-subsystem script, each generated
> commit is generated in separate.
>
> Still, generating commands are very similar, and looks like
>
> sed -n '/^<Subsystem name>$/,/^$/{s/^F: //p}' MAINTAINERS | \
> xargs git ls-files | grep '\.[hc]$' | \
> xargs spatch \
> --sp-file scripts/coccinelle/auto-propagated-errp.cocci \
> --macro-file scripts/cocci-macro-file.h \
> --in-place --no-show-diff --max-width 80
>
> Note, that in each generated commit, generation command is the only
> text, indented by 8 spaces in 'git log -1' output, so, to regenerate all
> commits (for example, after rebase, or change in coccinelle script), you
> may use the following command:
>
> git rebase -x "sh -c \"git show --pretty= --name-only | xargs git checkout HEAD^ -- ; git reset; git log -1 | grep '^ ' | sh\"" HEAD~7
>
> Which will start automated interactive rebase for generated patches,
> which will stop if generated patch changed
> (you may do git commit --amend to apply updated generated changes).
>
> Note:
> git show --pretty= --name-only - lists files, changed in HEAD
> git log -1 | grep '^ ' | sh - rerun generation command of HEAD
>
>
> Check for compilation of changed .c files
> git rebase -x "sh -c \"git show --pretty= --name-only | sed -n 's/\.c$/.o/p' | xargs make -j9\"" HEAD~7
Thanks for the write-up!
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 77+ messages in thread