[PATCH v2] scripts: Convert qemu-version.sh to qemu-version.py

[PATCH v2] scripts: Convert qemu-version.sh to qemu-version.py

[Yonggang Luo]

The sh script are harder to maintain for compatible different
xsh environment

Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
---
 meson.build             |  2 +-
 scripts/qemu-version.py | 30 ++++++++++++++++++++++++++++++
 scripts/qemu-version.sh | 25 -------------------------
 3 files changed, 31 insertions(+), 26 deletions(-)
 create mode 100644 scripts/qemu-version.py
 delete mode 100755 scripts/qemu-version.sh

diff --git a/meson.build b/meson.build
index 95a532bd29..20f653b6eb 100644
--- a/meson.build
+++ b/meson.build
@@ -1072,7 +1072,7 @@ tracetool = [
    '--backend=' + config_host['TRACE_BACKENDS']
 ]
 
-qemu_version_cmd = [find_program('scripts/qemu-version.sh'),
+qemu_version_cmd = [find_program('scripts/qemu-version.py'),
                     meson.current_source_dir(),
                     config_host['PKGVERSION'], meson.project_version()]
 qemu_version = custom_target('qemu-version.h',
diff --git a/scripts/qemu-version.py b/scripts/qemu-version.py
new file mode 100644
index 0000000000..384c54027d
--- /dev/null
+++ b/scripts/qemu-version.py
@@ -0,0 +1,30 @@
+#!/usr/bin/env python3
+
+# Script for retrieve qemu git version information
+# and output to stdout as QEMU_PKGVERSION and QEMU_FULL_VERSION header
+# Author: Yonggang Luo <luoyonggang@gmail.com>
+
+import sys
+import subprocess
+
+def main(args):
+    if len(args) <= 3:
+        sys.exit(0)
+
+    dir = args[1]
+    pkgversion = args[2]
+    version = args[3]
+    pc = subprocess.run(['git', 'describe', '--match', "'v*'", '--dirty', '--always'],
+        stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, cwd=dir)
+    if pc.returncode == 0:
+        pkgversion = pc.stdout.decode('utf8').strip()
+    fullversion = version
+    if len(pkgversion) > 0:
+        fullversion = "{} ({})".format(version, pkgversion)
+
+    version_header = '''#define QEMU_PKGVERSION "{}"
+#define QEMU_FULL_VERSION "{}"'''.format(pkgversion, fullversion)
+    sys.stdout.buffer.write(version_header.encode('utf8'))
+
+if __name__ == "__main__":
+    main(sys.argv)
diff --git a/scripts/qemu-version.sh b/scripts/qemu-version.sh
deleted file mode 100755
index 03128c56a2..0000000000
--- a/scripts/qemu-version.sh
+++ /dev/null
@@ -1,25 +0,0 @@
-#!/bin/sh
-
-set -eu
-
-dir="$1"
-pkgversion="$2"
-version="$3"
-
-if [ -z "$pkgversion" ]; then
-    cd "$dir"
-    if [ -e .git ]; then
-        pkgversion=$(git describe --match 'v*' --dirty | echo "")
-    fi
-fi
-
-if [ -n "$pkgversion" ]; then
-    fullversion="$version ($pkgversion)"
-else
-    fullversion="$version"
-fi
-
-cat <<EOF
-#define QEMU_PKGVERSION "$pkgversion"
-#define QEMU_FULL_VERSION "$fullversion"
-EOF
-- 
2.28.0.windows.1

Re: [PATCH v2] scripts: Convert qemu-version.sh to qemu-version.py

[Philippe Mathieu-Daudé]

On 10/5/20 7:21 PM, Yonggang Luo wrote:
> The sh script are harder to maintain for compatible different
> xsh environment
> 
> Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
> ---

What are the changes since v1?

>  meson.build             |  2 +-
>  scripts/qemu-version.py | 30 ++++++++++++++++++++++++++++++
>  scripts/qemu-version.sh | 25 -------------------------
>  3 files changed, 31 insertions(+), 26 deletions(-)
>  create mode 100644 scripts/qemu-version.py
>  delete mode 100755 scripts/qemu-version.sh
> 
> diff --git a/meson.build b/meson.build
> index 95a532bd29..20f653b6eb 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1072,7 +1072,7 @@ tracetool = [
>     '--backend=' + config_host['TRACE_BACKENDS']
>  ]
>  
> -qemu_version_cmd = [find_program('scripts/qemu-version.sh'),
> +qemu_version_cmd = [find_program('scripts/qemu-version.py'),
>                      meson.current_source_dir(),
>                      config_host['PKGVERSION'], meson.project_version()]
>  qemu_version = custom_target('qemu-version.h',
> diff --git a/scripts/qemu-version.py b/scripts/qemu-version.py
> new file mode 100644
> index 0000000000..384c54027d
> --- /dev/null
> +++ b/scripts/qemu-version.py
> @@ -0,0 +1,30 @@
> +#!/usr/bin/env python3
> +
> +# Script for retrieve qemu git version information
> +# and output to stdout as QEMU_PKGVERSION and QEMU_FULL_VERSION header
> +# Author: Yonggang Luo <luoyonggang@gmail.com>
> +
> +import sys
> +import subprocess
> +
> +def main(args):
> +    if len(args) <= 3:
> +        sys.exit(0)
> +
> +    dir = args[1]
> +    pkgversion = args[2]
> +    version = args[3]
> +    pc = subprocess.run(['git', 'describe', '--match', "'v*'", '--dirty', '--always'],
> +        stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, cwd=dir)
> +    if pc.returncode == 0:
> +        pkgversion = pc.stdout.decode('utf8').strip()
> +    fullversion = version
> +    if len(pkgversion) > 0:
> +        fullversion = "{} ({})".format(version, pkgversion)
> +
> +    version_header = '''#define QEMU_PKGVERSION "{}"
> +#define QEMU_FULL_VERSION "{}"'''.format(pkgversion, fullversion)
> +    sys.stdout.buffer.write(version_header.encode('utf8'))
> +
> +if __name__ == "__main__":
> +    main(sys.argv)
> diff --git a/scripts/qemu-version.sh b/scripts/qemu-version.sh
> deleted file mode 100755
> index 03128c56a2..0000000000
> --- a/scripts/qemu-version.sh
> +++ /dev/null
> @@ -1,25 +0,0 @@
> -#!/bin/sh
> -
> -set -eu
> -
> -dir="$1"
> -pkgversion="$2"
> -version="$3"
> -
> -if [ -z "$pkgversion" ]; then
> -    cd "$dir"
> -    if [ -e .git ]; then
> -        pkgversion=$(git describe --match 'v*' --dirty | echo "")
> -    fi
> -fi
> -
> -if [ -n "$pkgversion" ]; then
> -    fullversion="$version ($pkgversion)"
> -else
> -    fullversion="$version"
> -fi
> -
> -cat <<EOF
> -#define QEMU_PKGVERSION "$pkgversion"
> -#define QEMU_FULL_VERSION "$fullversion"
> -EOF
> 

Re: [PATCH v2] scripts: Convert qemu-version.sh to qemu-version.py

[罗勇刚(Yonggang Luo)]

[-- Attachment #1: Type: text/plain, Size: 3603 bytes --]

On Tue, Oct 6, 2020 at 2:10 AM Philippe Mathieu-Daudé <f4bug@amsat.org>
wrote:
>
> On 10/5/20 7:21 PM, Yonggang Luo wrote:
> > The sh script are harder to maintain for compatible different
> > xsh environment
> >
> > Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
> > ---
>
> What are the changes since v1?

+    pc = subprocess.run(['git', 'describe', '--match', "'v*'", '--dirty',
'--always'],
+        stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, cwd=dir)

I output the stderr to devnull so that doesn't pollute the console.

>
> >  meson.build             |  2 +-
> >  scripts/qemu-version.py | 30 ++++++++++++++++++++++++++++++
> >  scripts/qemu-version.sh | 25 -------------------------
> >  3 files changed, 31 insertions(+), 26 deletions(-)
> >  create mode 100644 scripts/qemu-version.py
> >  delete mode 100755 scripts/qemu-version.sh
> >
> > diff --git a/meson.build b/meson.build
> > index 95a532bd29..20f653b6eb 100644
> > --- a/meson.build
> > +++ b/meson.build
> > @@ -1072,7 +1072,7 @@ tracetool = [
> >     '--backend=' + config_host['TRACE_BACKENDS']
> >  ]
> >
> > -qemu_version_cmd = [find_program('scripts/qemu-version.sh'),
> > +qemu_version_cmd = [find_program('scripts/qemu-version.py'),
> >                      meson.current_source_dir(),
> >                      config_host['PKGVERSION'], meson.project_version()]
> >  qemu_version = custom_target('qemu-version.h',
> > diff --git a/scripts/qemu-version.py b/scripts/qemu-version.py
> > new file mode 100644
> > index 0000000000..384c54027d
> > --- /dev/null
> > +++ b/scripts/qemu-version.py
> > @@ -0,0 +1,30 @@
> > +#!/usr/bin/env python3
> > +
> > +# Script for retrieve qemu git version information
> > +# and output to stdout as QEMU_PKGVERSION and QEMU_FULL_VERSION header
> > +# Author: Yonggang Luo <luoyonggang@gmail.com>
> > +
> > +import sys
> > +import subprocess
> > +
> > +def main(args):
> > +    if len(args) <= 3:
> > +        sys.exit(0)
> > +
> > +    dir = args[1]
> > +    pkgversion = args[2]
> > +    version = args[3]
> > +    pc = subprocess.run(['git', 'describe', '--match', "'v*'",
'--dirty', '--always'],
> > +        stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, cwd=dir)
> > +    if pc.returncode == 0:
> > +        pkgversion = pc.stdout.decode('utf8').strip()
> > +    fullversion = version
> > +    if len(pkgversion) > 0:
> > +        fullversion = "{} ({})".format(version, pkgversion)
> > +
> > +    version_header = '''#define QEMU_PKGVERSION "{}"
> > +#define QEMU_FULL_VERSION "{}"'''.format(pkgversion, fullversion)
> > +    sys.stdout.buffer.write(version_header.encode('utf8'))
> > +
> > +if __name__ == "__main__":
> > +    main(sys.argv)
> > diff --git a/scripts/qemu-version.sh b/scripts/qemu-version.sh
> > deleted file mode 100755
> > index 03128c56a2..0000000000
> > --- a/scripts/qemu-version.sh
> > +++ /dev/null
> > @@ -1,25 +0,0 @@
> > -#!/bin/sh
> > -
> > -set -eu
> > -
> > -dir="$1"
> > -pkgversion="$2"
> > -version="$3"
> > -
> > -if [ -z "$pkgversion" ]; then
> > -    cd "$dir"
> > -    if [ -e .git ]; then
> > -        pkgversion=$(git describe --match 'v*' --dirty | echo "")
> > -    fi
> > -fi
> > -
> > -if [ -n "$pkgversion" ]; then
> > -    fullversion="$version ($pkgversion)"
> > -else
> > -    fullversion="$version"
> > -fi
> > -
> > -cat <<EOF
> > -#define QEMU_PKGVERSION "$pkgversion"
> > -#define QEMU_FULL_VERSION "$fullversion"
> > -EOF
> >



--
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

[-- Attachment #2: Type: text/html, Size: 5158 bytes --]

Re: [PATCH v2] scripts: Convert qemu-version.sh to qemu-version.py

[Peter Maydell]

On Mon, 5 Oct 2020 at 18:24, Yonggang Luo <luoyonggang@gmail.com> wrote:
>
> The sh script are harder to maintain for compatible different
> xsh environment
>
> Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
> ---
>  meson.build             |  2 +-
>  scripts/qemu-version.py | 30 ++++++++++++++++++++++++++++++
>  scripts/qemu-version.sh | 25 -------------------------
>  3 files changed, 31 insertions(+), 26 deletions(-)
>  create mode 100644 scripts/qemu-version.py
>  delete mode 100755 scripts/qemu-version.sh
>
> diff --git a/meson.build b/meson.build
> index 95a532bd29..20f653b6eb 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1072,7 +1072,7 @@ tracetool = [
>     '--backend=' + config_host['TRACE_BACKENDS']
>  ]
>
> -qemu_version_cmd = [find_program('scripts/qemu-version.sh'),
> +qemu_version_cmd = [find_program('scripts/qemu-version.py'),
>                      meson.current_source_dir(),
>                      config_host['PKGVERSION'], meson.project_version()]
>  qemu_version = custom_target('qemu-version.h',
> diff --git a/scripts/qemu-version.py b/scripts/qemu-version.py
> new file mode 100644
> index 0000000000..384c54027d
> --- /dev/null
> +++ b/scripts/qemu-version.py
> @@ -0,0 +1,30 @@
> +#!/usr/bin/env python3
> +
> +# Script for retrieve qemu git version information
> +# and output to stdout as QEMU_PKGVERSION and QEMU_FULL_VERSION header
> +# Author: Yonggang Luo <luoyonggang@gmail.com>

Can we have a license statement in all new files, please?

> +
> +import sys
> +import subprocess
> +
> +def main(args):
> +    if len(args) <= 3:
> +        sys.exit(0)
> +
> +    dir = args[1]
> +    pkgversion = args[2]
> +    version = args[3]
> +    pc = subprocess.run(['git', 'describe', '--match', "'v*'", '--dirty', '--always'],
> +        stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, cwd=dir)
> +    if pc.returncode == 0:
> +        pkgversion = pc.stdout.decode('utf8').strip()
> +    fullversion = version
> +    if len(pkgversion) > 0:
> +        fullversion = "{} ({})".format(version, pkgversion)
> +
> +    version_header = '''#define QEMU_PKGVERSION "{}"
> +#define QEMU_FULL_VERSION "{}"'''.format(pkgversion, fullversion)
> +    sys.stdout.buffer.write(version_header.encode('utf8'))
> +
> +if __name__ == "__main__":
> +    main(sys.argv)

Wow, python really makes this kind of task clunky compared to shell...

This doesn't seem to be the same logic as the original shell.
I'm not too familiar with python, but:
 * the shell script doesn't run git if pkgversion is not the empty string
 * the shell script doesn't run git unless the .git directory exists
If these are intentional behaviour changes you should mention
them in the commit message.

Also worth mentioning that you're fixing the bug in the shell
script where we ignore the output from git (the intention was
to ignore a failure from git but otherwise to keep its output;
this patch:
https://patchew.org/QEMU/20200929143654.518157-1-marcandre.lureau@redhat.com/
is the fix for that in the existing shell script).

> diff --git a/scripts/qemu-version.sh b/scripts/qemu-version.sh
> deleted file mode 100755
> index 03128c56a2..0000000000
> --- a/scripts/qemu-version.sh
> +++ /dev/null
> @@ -1,25 +0,0 @@
> -#!/bin/sh
> -
> -set -eu
> -
> -dir="$1"
> -pkgversion="$2"
> -version="$3"
> -
> -if [ -z "$pkgversion" ]; then
> -    cd "$dir"
> -    if [ -e .git ]; then
> -        pkgversion=$(git describe --match 'v*' --dirty | echo "")
> -    fi
> -fi
> -
> -if [ -n "$pkgversion" ]; then
> -    fullversion="$version ($pkgversion)"
> -else
> -    fullversion="$version"
> -fi
> -
> -cat <<EOF
> -#define QEMU_PKGVERSION "$pkgversion"
> -#define QEMU_FULL_VERSION "$fullversion"
> -EOF

thanks
-- PMM

Re: [PATCH v2] scripts: Convert qemu-version.sh to qemu-version.py

[罗勇刚(Yonggang Luo)]

[-- Attachment #1: Type: text/plain, Size: 4305 bytes --]

On Tue, Oct 6, 2020 at 3:23 AM Peter Maydell <peter.maydell@linaro.org>
wrote:
>
> On Mon, 5 Oct 2020 at 18:24, Yonggang Luo <luoyonggang@gmail.com> wrote:
> >
> > The sh script are harder to maintain for compatible different
> > xsh environment
> >
> > Signed-off-by: Yonggang Luo <luoyonggang@gmail.com>
> > ---
> >  meson.build             |  2 +-
> >  scripts/qemu-version.py | 30 ++++++++++++++++++++++++++++++
> >  scripts/qemu-version.sh | 25 -------------------------
> >  3 files changed, 31 insertions(+), 26 deletions(-)
> >  create mode 100644 scripts/qemu-version.py
> >  delete mode 100755 scripts/qemu-version.sh
> >
> > diff --git a/meson.build b/meson.build
> > index 95a532bd29..20f653b6eb 100644
> > --- a/meson.build
> > +++ b/meson.build
> > @@ -1072,7 +1072,7 @@ tracetool = [
> >     '--backend=' + config_host['TRACE_BACKENDS']
> >  ]
> >
> > -qemu_version_cmd = [find_program('scripts/qemu-version.sh'),
> > +qemu_version_cmd = [find_program('scripts/qemu-version.py'),
> >                      meson.current_source_dir(),
> >                      config_host['PKGVERSION'], meson.project_version()]
> >  qemu_version = custom_target('qemu-version.h',
> > diff --git a/scripts/qemu-version.py b/scripts/qemu-version.py
> > new file mode 100644
> > index 0000000000..384c54027d
> > --- /dev/null
> > +++ b/scripts/qemu-version.py
> > @@ -0,0 +1,30 @@
> > +#!/usr/bin/env python3
> > +
> > +# Script for retrieve qemu git version information
> > +# and output to stdout as QEMU_PKGVERSION and QEMU_FULL_VERSION header
> > +# Author: Yonggang Luo <luoyonggang@gmail.com>
>
> Can we have a license statement in all new files, please?
>
> > +
> > +import sys
> > +import subprocess
> > +
> > +def main(args):
> > +    if len(args) <= 3:
> > +        sys.exit(0)
> > +
> > +    dir = args[1]
> > +    pkgversion = args[2]
> > +    version = args[3]
> > +    pc = subprocess.run(['git', 'describe', '--match', "'v*'",
'--dirty', '--always'],
> > +        stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, cwd=dir)
> > +    if pc.returncode == 0:
> > +        pkgversion = pc.stdout.decode('utf8').strip()
> > +    fullversion = version
> > +    if len(pkgversion) > 0:
> > +        fullversion = "{} ({})".format(version, pkgversion)
> > +
> > +    version_header = '''#define QEMU_PKGVERSION "{}"
> > +#define QEMU_FULL_VERSION "{}"'''.format(pkgversion, fullversion)
> > +    sys.stdout.buffer.write(version_header.encode('utf8'))
> > +
> > +if __name__ == "__main__":
> > +    main(sys.argv)
>
> Wow, python really makes this kind of task clunky compared to shell...
That's right, the size are bloated, but for me, easier to understand
>
> This doesn't seem to be the same logic as the original shell.
> I'm not too familiar with python, but:
>  * the shell script doesn't run git if pkgversion is not the empty string
>  * the shell script doesn't run git unless the .git directory exists
> If these are intentional behaviour changes you should mention
> them in the commit message.
>
> Also worth mentioning that you're fixing the bug in the shell
> script where we ignore the output from git (the intention was
> to ignore a failure from git but otherwise to keep its output;
> this patch:
>
https://patchew.org/QEMU/20200929143654.518157-1-marcandre.lureau@redhat.com/
> is the fix for that in the existing shell script).
>
> > diff --git a/scripts/qemu-version.sh b/scripts/qemu-version.sh
> > deleted file mode 100755
> > index 03128c56a2..0000000000
> > --- a/scripts/qemu-version.sh
> > +++ /dev/null
> > @@ -1,25 +0,0 @@
> > -#!/bin/sh
> > -
> > -set -eu
> > -
> > -dir="$1"
> > -pkgversion="$2"
> > -version="$3"
> > -
> > -if [ -z "$pkgversion" ]; then
> > -    cd "$dir"
> > -    if [ -e .git ]; then
> > -        pkgversion=$(git describe --match 'v*' --dirty | echo "")
> > -    fi
> > -fi
> > -
> > -if [ -n "$pkgversion" ]; then
> > -    fullversion="$version ($pkgversion)"
> > -else
> > -    fullversion="$version"
> > -fi
> > -
> > -cat <<EOF
> > -#define QEMU_PKGVERSION "$pkgversion"
> > -#define QEMU_FULL_VERSION "$fullversion"
> > -EOF
>
> thanks
> -- PMM



--
         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

[-- Attachment #2: Type: text/html, Size: 5960 bytes --]

Re: [PATCH v2] scripts: Convert qemu-version.sh to qemu-version.py

[Paolo Bonzini]

On 05/10/20 21:22, Peter Maydell wrote:
>> +def main(args):
>> +    if len(args) <= 3:
>> +        sys.exit(0)
>> +
>> +    dir = args[1]
>> +    pkgversion = args[2]
>> +    version = args[3]

In addition to what Peter pointed out, all these lines can be changed to

    def main(dir, pkgversion, version, *unused):

and below

    if __name__ == "__main__":
        main(*sys.argv)

>> +    pc = subprocess.run(['git', 'describe', '--match', "'v*'", '--dirty', '--always'],
>> +        stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, cwd=dir)
>> +    if pc.returncode == 0:
>> +        pkgversion = pc.stdout.decode('utf8').strip()
>> +    fullversion = version
>> +    if len(pkgversion) > 0:

Just "if pkgversion:" please.

>> +        fullversion = "{} ({})".format(version, pkgversion)
>> +
>> +    version_header = '''#define QEMU_PKGVERSION "{}"
>> +#define QEMU_FULL_VERSION "{}"'''.format(pkgversion, fullversion)
>> +    sys.stdout.buffer.write(version_header.encode('utf8'))

No need to use buffer and encode, just

    print('#define QEMU_PKGVERSION "%s"' % pkgversion)
    print('#define QEMU_FULLVERSION "%s"' % fullversion)

It is still a bit more clunky than a shell script, but at least not as much.

Paolo