[Buildroot] [PATCH] utils/genrandconfig: fix runtime issue with Python 3

[Buildroot] [PATCH] utils/genrandconfig: fix runtime issue with Python 3

[Thomas Petazzoni]

With Python 3.7, genrandconfig fails with:

'str' object has no attribute 'decode'

We are already working on str objects, and there is no need to decode
them, so we drop the call to decode_byte_list() and its definition as
it was only used there.

Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
---
This is of course very strange, as the decode_byte_list() function was
introduced specifically to handle Python 3.x. Does anyone has an
explanation?
---
 utils/genrandconfig | 10 +---------
 1 file changed, 1 insertion(+), 9 deletions(-)

diff --git a/utils/genrandconfig b/utils/genrandconfig
index fb28f4d356..5b60bc21d6 100755
--- a/utils/genrandconfig
+++ b/utils/genrandconfig
@@ -39,14 +39,6 @@ def urlopen_closing(uri):
     return contextlib.closing(_urllib.urlopen(uri))
 
 
-if sys.hexversion >= 0x3000000:
-    def decode_byte_list(bl):
-        return [b.decode() for b in bl]
-else:
-    def decode_byte_list(e):
-        return e
-
-
 class SystemInfo:
     DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"]
     DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar", "diffoscope"]
@@ -128,7 +120,7 @@ def get_toolchain_configs(toolchains_csv, buildrootdir):
     with open(toolchains_csv) as r:
         # filter empty lines and comments
         lines = [t for t in r.readlines() if len(t.strip()) > 0 and t[0] != '#']
-        toolchains = decode_byte_list(lines)
+        toolchains = lines
     configs = []
 
     (_, _, _, _, hostarch) = os.uname()
-- 
2.23.0

[Buildroot] [PATCH] utils/genrandconfig: fix runtime issue with Python 3

[Arnout Vandecappelle]

On 03/12/2019 17:51, Thomas Petazzoni wrote:
> With Python 3.7, genrandconfig fails with:
> 
> 'str' object has no attribute 'decode'
> 
> We are already working on str objects, and there is no need to decode
> them, so we drop the call to decode_byte_list() and its definition as
> it was only used there.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>
> ---
> This is of course very strange, as the decode_byte_list() function was
> introduced specifically to handle Python 3.x. Does anyone has an
> explanation?

 Just guessing here, but: originally this file was downloaded with
urlopen_closing() instead of being read from a local file. Since we only use
genrandconfig in the autobuilders in python2 (cfr. the shebang), nobody noticed
that this broke when I switched to reading from file instead of URL two years
ago (84929a53a4).

 Regards,
 Arnout

> ---
>  utils/genrandconfig | 10 +---------
>  1 file changed, 1 insertion(+), 9 deletions(-)
> 
> diff --git a/utils/genrandconfig b/utils/genrandconfig
> index fb28f4d356..5b60bc21d6 100755
> --- a/utils/genrandconfig
> +++ b/utils/genrandconfig
> @@ -39,14 +39,6 @@ def urlopen_closing(uri):
>      return contextlib.closing(_urllib.urlopen(uri))
>  
>  
> -if sys.hexversion >= 0x3000000:
> -    def decode_byte_list(bl):
> -        return [b.decode() for b in bl]
> -else:
> -    def decode_byte_list(e):
> -        return e
> -
> -
>  class SystemInfo:
>      DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"]
>      DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar", "diffoscope"]
> @@ -128,7 +120,7 @@ def get_toolchain_configs(toolchains_csv, buildrootdir):
>      with open(toolchains_csv) as r:
>          # filter empty lines and comments
>          lines = [t for t in r.readlines() if len(t.strip()) > 0 and t[0] != '#']
> -        toolchains = decode_byte_list(lines)
> +        toolchains = lines
>      configs = []
>  
>      (_, _, _, _, hostarch) = os.uname()
> 

[Buildroot] [PATCH] utils/genrandconfig: fix runtime issue with Python 3

[Thomas Petazzoni]

Hello,

On Tue, 3 Dec 2019 20:36:17 +0100
Arnout Vandecappelle <arnout@mind.be> wrote:

>  Just guessing here, but: originally this file was downloaded with
> urlopen_closing() instead of being read from a local file.

Ah, that must be it!

> Since we only use genrandconfig in the autobuilders in python2 (cfr.
> the shebang)

I'm not sure why you say "in python2" ? The shebang in genrandconfig is:

#!/usr/bin/env python

and I indeed noticed the issue with Python 3.x because I started using
genrandconfig on my updated Fedora system, which now uses Python 3.x as
"python".

>, nobody noticed that this broke when I switched to
> reading from file instead of URL two years ago (84929a53a4).

Indeed. I'll update the commit log and resend.

Thanks for the hint!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH] utils/genrandconfig: fix runtime issue with Python 3

[Arnout Vandecappelle]

On 03/12/2019 17:51, Thomas Petazzoni wrote:
> With Python 3.7, genrandconfig fails with:
> 
> 'str' object has no attribute 'decode'
> 
> We are already working on str objects, and there is no need to decode
> them, so we drop the call to decode_byte_list() and its definition as
> it was only used there.
> 
> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
> Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

 Applied to master, thanks.

 Regards,
 Arnout

> ---
> This is of course very strange, as the decode_byte_list() function was
> introduced specifically to handle Python 3.x. Does anyone has an
> explanation?
> ---
>  utils/genrandconfig | 10 +---------
>  1 file changed, 1 insertion(+), 9 deletions(-)
> 
> diff --git a/utils/genrandconfig b/utils/genrandconfig
> index fb28f4d356..5b60bc21d6 100755
> --- a/utils/genrandconfig
> +++ b/utils/genrandconfig
> @@ -39,14 +39,6 @@ def urlopen_closing(uri):
>      return contextlib.closing(_urllib.urlopen(uri))
>  
>  
> -if sys.hexversion >= 0x3000000:
> -    def decode_byte_list(bl):
> -        return [b.decode() for b in bl]
> -else:
> -    def decode_byte_list(e):
> -        return e
> -
> -
>  class SystemInfo:
>      DEFAULT_NEEDED_PROGS = ["make", "git", "gcc", "timeout"]
>      DEFAULT_OPTIONAL_PROGS = ["bzr", "java", "javac", "jar", "diffoscope"]
> @@ -128,7 +120,7 @@ def get_toolchain_configs(toolchains_csv, buildrootdir):
>      with open(toolchains_csv) as r:
>          # filter empty lines and comments
>          lines = [t for t in r.readlines() if len(t.strip()) > 0 and t[0] != '#']
> -        toolchains = decode_byte_list(lines)
> +        toolchains = lines
>      configs = []
>  
>      (_, _, _, _, hostarch) = os.uname()
> 

[Buildroot] [PATCH] utils/genrandconfig: fix runtime issue with Python 3

[Arnout Vandecappelle]

On 03/12/2019 20:58, Thomas Petazzoni wrote:
> Hello,
> 
> On Tue, 3 Dec 2019 20:36:17 +0100
> Arnout Vandecappelle <arnout@mind.be> wrote:
> 
>>  Just guessing here, but: originally this file was downloaded with
>> urlopen_closing() instead of being read from a local file.
> 
> Ah, that must be it!
> 
>> Since we only use genrandconfig in the autobuilders in python2 (cfr.
>> the shebang)
> 
> I'm not sure why you say "in python2" ? The shebang in genrandconfig is:
> 
> #!/usr/bin/env python

 Almost all distros resolve this to Python2. I think even now only Arch and
Fedora 31 resolve it to Python3.


> and I indeed noticed the issue with Python 3.x because I started using
> genrandconfig on my updated Fedora system, which now uses Python 3.x as
> "python".
> 
>> , nobody noticed that this broke when I switched to
>> reading from file instead of URL two years ago (84929a53a4).
> 
> Indeed. I'll update the commit log and resend.

 Didn't I mention that I already merged it?

 Regards,
 Arnout

[Buildroot] [PATCH] utils/genrandconfig: fix runtime issue with Python 3

[Thomas Petazzoni]

On Wed, 4 Dec 2019 13:52:48 +0100
Arnout Vandecappelle <arnout@mind.be> wrote:

> > and I indeed noticed the issue with Python 3.x because I started using
> > genrandconfig on my updated Fedora system, which now uses Python 3.x as
> > "python".
> >   
> >> , nobody noticed that this broke when I switched to
> >> reading from file instead of URL two years ago (84929a53a4).  
> > 
> > Indeed. I'll update the commit log and resend.  
> 
>  Didn't I mention that I already merged it?

You did, but I saw it after replying to this e-mail :)

Thanks!

Thomas
-- 
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

[Buildroot] [PATCH] utils/genrandconfig: fix runtime issue with Python 3

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > With Python 3.7, genrandconfig fails with:
 > 'str' object has no attribute 'decode'

 > We are already working on str objects, and there is no need to decode
 > them, so we drop the call to decode_byte_list() and its definition as
 > it was only used there.

 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
 > Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

Committed to 2019.02.x and 2019.08.x, thanks.

-- 
Bye, Peter Korsgaard

[Buildroot] [PATCH] utils/genrandconfig: fix runtime issue with Python 3

[Peter Korsgaard]

>>>>> "Thomas" == Thomas Petazzoni <thomas.petazzoni@bootlin.com> writes:

 > With Python 3.7, genrandconfig fails with:
 > 'str' object has no attribute 'decode'

 > We are already working on str objects, and there is no need to decode
 > them, so we drop the call to decode_byte_list() and its definition as
 > it was only used there.

 > Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>
 > Cc: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>

Committed to 2019.11.x, thanks.

-- 
Bye, Peter Korsgaard