Re: [PATCH 6/6] waf: python2 -> python3

Re: [PATCH 6/6] waf: python2 -> python3

[Burton, Ross]

Did you just do the minimum required, or take a new copy of waf from upstream?

(https://gitlab.com/ita1024/waf/)

Ross

On Fri, 28 Jun 2019 at 13:01, Robert Yang <liezhi.yang@windriver.com> wrote:
>
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  meta/recipes-graphics/eglinfo/files/waf | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/meta/recipes-graphics/eglinfo/files/waf b/meta/recipes-graphics/eglinfo/files/waf
> index 04ddd9f..2c83ed4 100755
> --- a/meta/recipes-graphics/eglinfo/files/waf
> +++ b/meta/recipes-graphics/eglinfo/files/waf
> @@ -1,4 +1,4 @@
> -#!/usr/bin/env python
> +#!/usr/bin/env python3
>  # encoding: latin-1
>  # Thomas Nagy, 2005-2018
>  #
> @@ -113,7 +113,7 @@ def unpack_wafdir(dir, src):
>         os.remove(tmp)
>         os.chdir(cwd)
>
> -       try: dir = unicode(dir, 'mbcs')
> +       try: dir = str(dir, 'mbcs')
>         except: pass
>         try:
>                 from ctypes import windll
> --
> 2.7.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH 6/6] waf: python2 -> python3

[Robert Yang]

On 6/28/19 8:15 PM, Burton, Ross wrote:
> Did you just do the minimum required, or take a new copy of waf from upstream?
> 
> (https://gitlab.com/ita1024/waf/)

No, I didn't take it from upstream, I just modified it and ran:

$ bitbake eglinfo-fb eglinfo-x11 eglinfo-wayland

// RObert

> 
> Ross
> 
> On Fri, 28 Jun 2019 at 13:01, Robert Yang <liezhi.yang@windriver.com> wrote:
>>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>   meta/recipes-graphics/eglinfo/files/waf | 4 ++--
>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/meta/recipes-graphics/eglinfo/files/waf b/meta/recipes-graphics/eglinfo/files/waf
>> index 04ddd9f..2c83ed4 100755
>> --- a/meta/recipes-graphics/eglinfo/files/waf
>> +++ b/meta/recipes-graphics/eglinfo/files/waf
>> @@ -1,4 +1,4 @@
>> -#!/usr/bin/env python
>> +#!/usr/bin/env python3
>>   # encoding: latin-1
>>   # Thomas Nagy, 2005-2018
>>   #
>> @@ -113,7 +113,7 @@ def unpack_wafdir(dir, src):
>>          os.remove(tmp)
>>          os.chdir(cwd)
>>
>> -       try: dir = unicode(dir, 'mbcs')
>> +       try: dir = str(dir, 'mbcs')
>>          except: pass
>>          try:
>>                  from ctypes import windll
>> --
>> 2.7.4
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 

[PATCH 0/6] meta: Cleanup python2

[Robert Yang]

The following changes since commit 9f674a88c781c7092d5b3460922a1579b9fe4bf9:

  busybox: Fix typo in syslog initscript (2019-06-27 13:27:54 +0100)

are available in the git repository at:

  git://git.openembedded.org/openembedded-core-contrib rbt/py3
  http://cgit.openembedded.org//log/?h=rbt/py3

Robert Yang (6):
  ksum.py: python -> python3
  wic: python2 -> python3
  ext-sdk-prepare.py: python2 -> python3
  oeqa: Cleanup /usr/bin/env python
  package_rpm.bbclass: python2 -> python3
  waf: python2 -> python3

 meta/classes/package_rpm.bbclass        |  2 +-
 meta/files/ext-sdk-prepare.py           |  2 +-
 meta/lib/oeqa/selftest/cases/wic.py     |  1 -
 meta/lib/oeqa/utils/logparser.py        |  1 -
 meta/recipes-graphics/eglinfo/files/waf |  4 +--
 scripts/lib/wic/__init__.py             |  2 +-
 scripts/lib/wic/ksparser.py             |  2 +-
 scripts/lib/wic/pluginbase.py           |  2 +-
 scripts/tiny/ksum.py                    | 44 ++++++++++++++++-----------------
 9 files changed, 29 insertions(+), 31 deletions(-)

-- 
2.7.4

[PATCH 1/6] ksum.py: python -> python3

[Robert Yang]

* Testinfo:
  In kernel's builddir:
  $ /path/to/oe-core/scripts/tiny/ksum.py

Collecting object files [DONE]

Totals:

vmlinux:
    text        data            bss             total
    10933110    3824470         1605632         16363212

modules (2004):
    text        data            bss             total
    46144408    3047516         167580          49359504

vmlinux + modules:
    text        data            bss             total
    57077518    6871986         1773212         65722716

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 scripts/tiny/ksum.py | 44 ++++++++++++++++++++++----------------------
 1 file changed, 22 insertions(+), 22 deletions(-)

diff --git a/scripts/tiny/ksum.py b/scripts/tiny/ksum.py
index f38ae05..8f0e4c0 100755
--- a/scripts/tiny/ksum.py
+++ b/scripts/tiny/ksum.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 #
 # Copyright (c) 2016, Intel Corporation.
 #
@@ -62,7 +62,7 @@ def is_ko_file(filename):
     return False
 
 def collect_object_files():
-    print "Collecting object files recursively from %s..." % os.getcwd()
+    print("Collecting object files recursively from %s..." % os.getcwd())
     for dirpath, dirs, files in os.walk(os.getcwd()):
         for filename in files:
             if is_ko_file(filename):
@@ -70,7 +70,7 @@ def collect_object_files():
             elif is_vmlinux_file(filename):
                 global vmlinux_file
                 vmlinux_file = os.path.join(dirpath, filename)
-    print "Collecting object files [DONE]"
+    print("Collecting object files [DONE]")
 
 def add_ko_file(filename):
         p = Popen("size -t " + filename, shell=True, stdout=PIPE, stderr=PIPE)
@@ -78,9 +78,9 @@ def add_ko_file(filename):
         if len(output) > 2:
             sizes = output[-1].split()[0:4]
             if verbose:
-                print "     %10d %10d %10d %10d\t" % \
-                    (int(sizes[0]), int(sizes[1]), int(sizes[2]), int(sizes[3])),
-                print "%s" % filename[len(os.getcwd()) + 1:]
+                print("     %10d %10d %10d %10d\t" % \
+                    (int(sizes[0]), int(sizes[1]), int(sizes[2]), int(sizes[3])), end=' ')
+                print("%s" % filename[len(os.getcwd()) + 1:])
             global n_ko_files, ko_text, ko_data, ko_bss, ko_total
             ko_text += int(sizes[0])
             ko_data += int(sizes[1])
@@ -94,9 +94,9 @@ def get_vmlinux_totals():
         if len(output) > 2:
             sizes = output[-1].split()[0:4]
             if verbose:
-                print "     %10d %10d %10d %10d\t" % \
-                    (int(sizes[0]), int(sizes[1]), int(sizes[2]), int(sizes[3])),
-                print "%s" % vmlinux_file[len(os.getcwd()) + 1:]
+                print("     %10d %10d %10d %10d\t" % \
+                    (int(sizes[0]), int(sizes[1]), int(sizes[2]), int(sizes[3])), end=' ')
+                print("%s" % vmlinux_file[len(os.getcwd()) + 1:])
             global vmlinux_text, vmlinux_data, vmlinux_bss, vmlinux_total
             vmlinux_text += int(sizes[0])
             vmlinux_data += int(sizes[1])
@@ -129,20 +129,20 @@ def main():
     sum_ko_files()
     get_vmlinux_totals()
 
-    print "\nTotals:"
-    print "\nvmlinux:"
-    print "    text\tdata\t\tbss\t\ttotal"
-    print "    %-10d\t%-10d\t%-10d\t%-10d" % \
-        (vmlinux_text, vmlinux_data, vmlinux_bss, vmlinux_total)
-    print "\nmodules (%d):" % n_ko_files
-    print "    text\tdata\t\tbss\t\ttotal"
-    print "    %-10d\t%-10d\t%-10d\t%-10d" % \
-        (ko_text, ko_data, ko_bss, ko_total)
-    print "\nvmlinux + modules:"
-    print "    text\tdata\t\tbss\t\ttotal"
-    print "    %-10d\t%-10d\t%-10d\t%-10d" % \
+    print("\nTotals:")
+    print("\nvmlinux:")
+    print("    text\tdata\t\tbss\t\ttotal")
+    print("    %-10d\t%-10d\t%-10d\t%-10d" % \
+        (vmlinux_text, vmlinux_data, vmlinux_bss, vmlinux_total))
+    print("\nmodules (%d):" % n_ko_files)
+    print("    text\tdata\t\tbss\t\ttotal")
+    print("    %-10d\t%-10d\t%-10d\t%-10d" % \
+        (ko_text, ko_data, ko_bss, ko_total))
+    print("\nvmlinux + modules:")
+    print("    text\tdata\t\tbss\t\ttotal")
+    print("    %-10d\t%-10d\t%-10d\t%-10d" % \
         (vmlinux_text + ko_text, vmlinux_data + ko_data, \
-         vmlinux_bss + ko_bss, vmlinux_total + ko_total)
+         vmlinux_bss + ko_bss, vmlinux_total + ko_total))
 
 if __name__ == "__main__":
     try:
-- 
2.7.4

[PATCH 2/6] wic: python2 -> python3

[Robert Yang]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 scripts/lib/wic/__init__.py   | 2 +-
 scripts/lib/wic/ksparser.py   | 2 +-
 scripts/lib/wic/pluginbase.py | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/scripts/lib/wic/__init__.py b/scripts/lib/wic/__init__.py
index ba2d614..8556793 100644
--- a/scripts/lib/wic/__init__.py
+++ b/scripts/lib/wic/__init__.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python -tt
+#!/usr/bin/env python3
 #
 # Copyright (c) 2007 Red Hat, Inc.
 # Copyright (c) 2011 Intel, Inc.
diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index 6204821..3e67003 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python -tt
+#!/usr/bin/env python3
 #
 # Copyright (c) 2016 Intel, Inc.
 #
diff --git a/scripts/lib/wic/pluginbase.py b/scripts/lib/wic/pluginbase.py
index bfb73ca..f74d643 100644
--- a/scripts/lib/wic/pluginbase.py
+++ b/scripts/lib/wic/pluginbase.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python -tt
+#!/usr/bin/env python3
 #
 # Copyright (c) 2011 Intel, Inc.
 #
-- 
2.7.4

[PATCH 3/6] ext-sdk-prepare.py: python2 -> python3

[Robert Yang]

This script is python3 indeed.

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/files/ext-sdk-prepare.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/files/ext-sdk-prepare.py b/meta/files/ext-sdk-prepare.py
index 96c5212..163d5e9 100644
--- a/meta/files/ext-sdk-prepare.py
+++ b/meta/files/ext-sdk-prepare.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # Prepare the build system within the extensible SDK
 
-- 
2.7.4

[PATCH 4/6] oeqa: Cleanup /usr/bin/env python

[Robert Yang]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/lib/oeqa/selftest/cases/wic.py | 1 -
 meta/lib/oeqa/utils/logparser.py    | 1 -
 2 files changed, 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py
index d16eae5..f1c6f39 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 #
 # Copyright (c) 2015, Intel Corporation.
 #
diff --git a/meta/lib/oeqa/utils/logparser.py b/meta/lib/oeqa/utils/logparser.py
index b31214b..7313df8 100644
--- a/meta/lib/oeqa/utils/logparser.py
+++ b/meta/lib/oeqa/utils/logparser.py
@@ -1,4 +1,3 @@
-#!/usr/bin/env python
 #
 # SPDX-License-Identifier: MIT
 #
-- 
2.7.4

[PATCH 5/6] package_rpm.bbclass: python2 -> python3

[Robert Yang]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/package_rpm.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 1a64cb2..a605a57 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -36,7 +36,7 @@ def write_rpm_perfiledata(srcname, d):
     pkgd = d.getVar('PKGD')
 
     def dump_filerdeps(varname, outfile, d):
-        outfile.write("#!/usr/bin/env python\n\n")
+        outfile.write("#!/usr/bin/env python3\n\n")
         outfile.write("# Dependency table\n")
         outfile.write('deps = {\n')
         for pkg in packages.split():
-- 
2.7.4

[PATCH 6/6] waf: python2 -> python3

[Robert Yang]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/recipes-graphics/eglinfo/files/waf | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-graphics/eglinfo/files/waf b/meta/recipes-graphics/eglinfo/files/waf
index 04ddd9f..2c83ed4 100755
--- a/meta/recipes-graphics/eglinfo/files/waf
+++ b/meta/recipes-graphics/eglinfo/files/waf
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # encoding: latin-1
 # Thomas Nagy, 2005-2018
 #
@@ -113,7 +113,7 @@ def unpack_wafdir(dir, src):
 	os.remove(tmp)
 	os.chdir(cwd)
 
-	try: dir = unicode(dir, 'mbcs')
+	try: dir = str(dir, 'mbcs')
 	except: pass
 	try:
 		from ctypes import windll
-- 
2.7.4

Re: [PATCH 6/6] waf: python2 -> python3

[Robert Yang]

Hi Ross,

On 6/28/19 8:15 PM, Burton, Ross wrote:
> Did you just do the minimum required, or take a new copy of waf from upstream?
> 
> (https://gitlab.com/ita1024/waf/)

I checked the one from upstream, it is still python2. The waf is the only one
which still requires python2 in oe-core. We can get rid of python2 after this
patch, what's your opinion, please?

// Robert

> 
> Ross
> 
> On Fri, 28 Jun 2019 at 13:01, Robert Yang <liezhi.yang@windriver.com> wrote:
>>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>   meta/recipes-graphics/eglinfo/files/waf | 4 ++--
>>   1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/meta/recipes-graphics/eglinfo/files/waf b/meta/recipes-graphics/eglinfo/files/waf
>> index 04ddd9f..2c83ed4 100755
>> --- a/meta/recipes-graphics/eglinfo/files/waf
>> +++ b/meta/recipes-graphics/eglinfo/files/waf
>> @@ -1,4 +1,4 @@
>> -#!/usr/bin/env python
>> +#!/usr/bin/env python3
>>   # encoding: latin-1
>>   # Thomas Nagy, 2005-2018
>>   #
>> @@ -113,7 +113,7 @@ def unpack_wafdir(dir, src):
>>          os.remove(tmp)
>>          os.chdir(cwd)
>>
>> -       try: dir = unicode(dir, 'mbcs')
>> +       try: dir = str(dir, 'mbcs')
>>          except: pass
>>          try:
>>                  from ctypes import windll
>> --
>> 2.7.4
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
> 

Re: [PATCH 6/6] waf: python2 -> python3

[Burton, Ross]

On Thu, 11 Jul 2019 at 03:59, Robert Yang <liezhi.yang@windriver.com> wrote:
> On 6/28/19 8:15 PM, Burton, Ross wrote:
> > Did you just do the minimum required, or take a new copy of waf from upstream?
> >
> > (https://gitlab.com/ita1024/waf/)
>
> I checked the one from upstream, it is still python2. The waf is the only one
> which still requires python2 in oe-core. We can get rid of python2 after this
> patch, what's your opinion, please?

If that's the case then YES YES YES.  :)

Ross

Re: [PATCH 6/6] waf: python2 -> python3

[Burton, Ross]

On Thu, 11 Jul 2019 at 11:26, Burton, Ross <ross.burton@intel.com> wrote:
>
> On Thu, 11 Jul 2019 at 03:59, Robert Yang <liezhi.yang@windriver.com> wrote:
> > On 6/28/19 8:15 PM, Burton, Ross wrote:
> > > Did you just do the minimum required, or take a new copy of waf from upstream?
> > >
> > > (https://gitlab.com/ita1024/waf/)
> >
> > I checked the one from upstream, it is still python2. The waf is the only one
> > which still requires python2 in oe-core. We can get rid of python2 after this
> > patch, what's your opinion, please?
>
> If that's the case then YES YES YES.  :)

Follow-up: do you mean remove python2 from HOSTTOOLS, or the recipe
from oe-core?

$ bitbake -g world
$ grep -- '-> "python"' recipe-depends.dot
"kernel-devsrc" -> "python"
"ltp" -> "python"
"meta-world-pkgdata" -> "python"
"packagegroup-core-tools-profile" -> "python"
"packagegroup-core-tools-testapps" -> "python"
"packagegroup-self-hosted" -> "python"
"perf" -> "python"
"python-nose" -> "python"
"python-numpy" -> "python"
"python-scons" -> "python"
"python-setuptools" -> "python"

Nothing depends on the python-* in core, we can ignore the
packagegroups, so that leaves kernel-devsrc, ltp, and perf. Also
u-boot uses nativepython.

HOSTTOOLS is more complicated and there's still plenty of pieces that
use host python.  I've an old poky-contrib:ross/nopy2 branch that
fixes a few more users of host python2 that needs rebasing and
retesting.

Ross

Ross

Re: [PATCH 6/6] waf: python2 -> python3

[Robert Yang]

On 7/11/19 6:46 PM, Burton, Ross wrote:
> On Thu, 11 Jul 2019 at 11:26, Burton, Ross <ross.burton@intel.com> wrote:
>>
>> On Thu, 11 Jul 2019 at 03:59, Robert Yang <liezhi.yang@windriver.com> wrote:
>>> On 6/28/19 8:15 PM, Burton, Ross wrote:
>>>> Did you just do the minimum required, or take a new copy of waf from upstream?
>>>>
>>>> (https://gitlab.com/ita1024/waf/)
>>>
>>> I checked the one from upstream, it is still python2. The waf is the only one
>>> which still requires python2 in oe-core. We can get rid of python2 after this
>>> patch, what's your opinion, please?
>>
>> If that's the case then YES YES YES.  :)
> 
> Follow-up: do you mean remove python2 from HOSTTOOLS, or the recipe
> from oe-core?
> 
> $ bitbake -g world
> $ grep -- '-> "python"' recipe-depends.dot
> "kernel-devsrc" -> "python"
> "ltp" -> "python"
> "meta-world-pkgdata" -> "python"
> "packagegroup-core-tools-profile" -> "python"
> "packagegroup-core-tools-testapps" -> "python"
> "packagegroup-self-hosted" -> "python"
> "perf" -> "python"
> "python-nose" -> "python"
> "python-numpy" -> "python"
> "python-scons" -> "python"
> "python-setuptools" -> "python"
> 
> Nothing depends on the python-* in core, we can ignore the
> packagegroups, so that leaves kernel-devsrc, ltp, and perf. Also
> u-boot uses nativepython.
> 
> HOSTTOOLS is more complicated and there's still plenty of pieces that
> use host python.  I've an old poky-contrib:ross/nopy2 branch that
> fixes a few more users of host python2 that needs rebasing and
> retesting.

I meant remove python2 from HOSTTOOLS, but it seems not easy to do atm, so maybe
we still need keep it for many years. I have this idea is because host such as
Redhat Linux 8 doesn't install python2 by default any more.

// Robert

> 
> Ross
> 
> Ross
> 

Re: [PATCH 6/6] waf: python2 -> python3

[Burton, Ross]

On Fri, 12 Jul 2019 at 11:33, Robert Yang <liezhi.yang@windriver.com> wrote:
> > HOSTTOOLS is more complicated and there's still plenty of pieces that
> > use host python.  I've an old poky-contrib:ross/nopy2 branch that
> > fixes a few more users of host python2 that needs rebasing and
> > retesting.
>
> I meant remove python2 from HOSTTOOLS, but it seems not easy to do atm, so maybe
> we still need keep it for many years. I have this idea is because host such as
> Redhat Linux 8 doesn't install python2 by default any more.

I left a world build going last night, some more fixes in nopy2.
There's still about 10 recipes failing.

Ross