All of lore.kernel.org
 help / color / mirror / Atom feed
* devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
@ 2020-11-12 11:00 Peter Bergin
  2020-11-12 11:07 ` ernst.sjostrand
  0 siblings, 1 reply; 7+ messages in thread
From: Peter Bergin @ 2020-11-12 11:00 UTC (permalink / raw)
  To: Patches and discussions about the oe-core layer

Hi,

when running 'devtool modify u-boot-imx' I get a stacktrace ending up in 
"ModuleNotFoundError: No module named '_sysconfigdata'". 'devtool 
modify' works for other recipes. Detailed stacktrace and layer info is 
added below.

The stacktrace seems very similar to 
https://lists.openembedded.org/g/openembedded-core/message/144484 but I 
decided to start a new thread to not highjack another issue. As the 
stacktarces are very similar I have  guess that they are related in some 
way. My host machine is running Ubuntu 20.04.

I'm out of ideas how to debug this further and would like some help to 
sort this out.

Best regards,
/Peter



     $ devtool modify u-boot-imx

     ...

     Build Configuration:
     BB_VERSION           = "1.49.0"
     BUILD_SYS            = "x86_64-linux"
     NATIVELSBSTRING      = "universal"
     TARGET_SYS           = "aarch64-fslc-linux"
     MACHINE              = "imx8mnevk"
     DISTRO               = "fslc-wayland"
     DISTRO_VERSION       = "3.2-snapshot-20201112"
     TUNE_FEATURES        = "aarch64 armv8a crc cortexa53 crypto"
     TARGET_FPU           = ""
     meta
     meta-poky            = 
"master:be17f6bacc6ba8ee5c92cf4b2ec5b43c8e660b33"
     meta-freescale       = 
"master:447032d809427a20cc066b32254cea8821073281"
     meta-freescale-3rdparty = 
"master:b85d08a55cb833bfc4e8b5034ff804286c67620e"
     meta-freescale-distro = 
"master:11be3f01962df8436c5c7b0d61cd3dbd1b872905"
     meta-oe
     meta-networking
     meta-python          = 
"master:b6e14805923a0dec17140c9df45434a9bea951d9"

     ERROR: Error executing a python function in exec_python_func() 
autogenerated:

     The stack trace of python calls that resulted in this 
exception/failure was:
     File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
          0001:
      *** 0002:devtool_post_unpack(d)
          0003:
     File: 
'/work/yocto/elektron-master/sources/poky/meta/classes/devtool-source.bbclass', 
lineno: 68, function:    devtool_post_unpack
          0064:}
          0065:
          0066:
          0067:python devtool_post_unpack() {
      *** 0068:    import oe.recipeutils
          0069:    import shutil
          0070:    sys.path.insert(0, os.path.join(d.getVar('COREBASE'), 
'scripts', 'lib'))
          0071:    import scriptutils
          0072:    from devtool import setup_git_repo
     File: 
'/work/yocto/elektron-master/sources/poky/meta/lib/oe/recipeutils.py', 
lineno: 21, function: <module>
          0017:import shutil
          0018:import re
          0019:import fnmatch
          0020:import glob
      *** 0021:import bb.tinfoil
          0022:
          0023:from collections import OrderedDict, defaultdict
          0024:from bb.utils import vercmp_string
          0025:
     File: 
'/work/yocto/elektron-master/sources/poky/bitbake/lib/bb/tinfoil.py', 
lineno: 19, function: <module>
          0015:from collections import OrderedDict, defaultdict
          0016:from functools import partial
          0017:
          0018:import bb.cache
      *** 0019:import bb.cooker
          0020:import bb.providers
          0021:import bb.taskdata
          0022:import bb.utils
          0023:import bb.command
     File: 
'/work/yocto/elektron-master/sources/poky/bitbake/lib/bb/cooker.py', 
lineno: 25, function: <module>
          0021:import bb, bb.exceptions, bb.command
          0022:from bb import utils, data, parse, event, cache, 
providers, taskdata, runqueue, build
          0023:import queue
          0024:import signal
      *** 0025:import prserv.serv
          0026:import pyinotify
          0027:import json
          0028:import pickle
          0029:import codecs
     File: 
'/work/yocto/elektron-master/sources/poky/bitbake/lib/prserv/serv.py', 
lineno: 7, function: <module>
          0003:#
          0004:
          0005:import os,sys,logging
          0006:import signal, time
      *** 0007:from xmlrpc.server import SimpleXMLRPCServer, 
SimpleXMLRPCRequestHandler
          0008:import threading
          0009:import queue
          0010:import socket
          0011:import io
     File: '/usr/lib/python3.8/xmlrpc/server.py', lineno: 117, function: 
<module>
          0113:import socketserver
          0114:import sys
          0115:import os
          0116:import re
      *** 0117:import pydoc
          0118:import traceback
          0119:try:
          0120:    import fcntl
          0121:except ImportError:
     File: '/usr/lib/python3.8/pydoc.py', lineno: 370, function: <module>
          0366:    return module
          0367:
          0368:# ---------------------------------------------------- 
formatter base class
          0369:
      *** 0370:class Doc:
          0371:
          0372:    PYTHONDOCS = os.environ.get("PYTHONDOCS",
          0373: "https://docs.python.org/%d.%d/library"
          0374:                                % sys.version_info[:2])
     File: '/usr/lib/python3.8/pydoc.py', lineno: 400, function: Doc
          0396:        raise TypeError(message)
          0397:
          0398:    docmodule = docclass = docroutine = docother = 
docproperty = docdata = fail
          0399:
      *** 0400:    def getdocloc(self, object, 
basedir=sysconfig.get_path('stdlib')):
          0401:        """Return the location of module docs or None"""
          0402:
          0403:        try:
          0404:            file = inspect.getabsfile(object)
     File: '/usr/lib/python3.8/sysconfig.py', lineno: 512, function: 
get_path
          0508:    """Return a path corresponding to the scheme.
          0509:
          0510:    ``scheme`` is the install scheme name.
          0511:    """
      *** 0512:    return get_paths(scheme, vars, expand)[name]
          0513:
          0514:
          0515:def get_config_vars(*args):
          0516:    """With no arguments, return a dictionary of all 
configuration
     File: '/usr/lib/python3.8/sysconfig.py', lineno: 502, function: 
get_paths
          0498:    ``scheme`` is the install scheme name. If not 
provided, it will
          0499:    return the default scheme for the current platform.
          0500:    """
          0501:    if expand:
      *** 0502:        return _expand_vars(scheme, vars)
          0503:    else:
          0504:        return _INSTALL_SCHEMES[scheme]
          0505:
          0506:
     File: '/usr/lib/python3.8/sysconfig.py', lineno: 172, function: 
_expand_vars
          0168:def _expand_vars(scheme, vars):
          0169:    res = {}
          0170:    if vars is None:
          0171:        vars = {}
      *** 0172:    _extend_dict(vars, get_config_vars())
          0173:
          0174:    for key, value in _INSTALL_SCHEMES[scheme].items():
          0175:        if os.name in ('posix', 'nt'):
          0176:            value = os.path.expanduser(value)
     File: '/usr/lib/python3.8/sysconfig.py', lineno: 550, function: 
get_config_vars
          0546:
          0547:        if os.name == 'nt':
          0548:            _init_non_posix(_CONFIG_VARS)
          0549:        if os.name == 'posix':
      *** 0550:            _init_posix(_CONFIG_VARS)
          0551:        # For backward compatibility, see issue19555
          0552:        SO = _CONFIG_VARS.get('EXT_SUFFIX')
          0553:        if SO is not None:
          0554:            _CONFIG_VARS['SO'] = SO
     File: '/usr/lib/python3.8/sysconfig.py', lineno: 421, function: 
_init_posix
          0417:def _init_posix(vars):
          0418:    """Initialize the module as appropriate for POSIX 
systems."""
          0419:    # _sysconfigdata is generated at build time, see 
_generate_posix_vars()
          0420:    name = _get_sysconfigdata_name()
      *** 0421:    _temp = __import__(name, globals(), locals(), 
['build_time_vars'], 0)
          0422:    build_time_vars = _temp.build_time_vars
          0423:    vars.update(build_time_vars)
          0424:
          0425:def _init_non_posix(vars):
     Exception: ModuleNotFoundError: No module named '_sysconfigdata'




^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
  2020-11-12 11:00 devtool modify - ModuleNotFoundError: No module named '_sysconfigdata' Peter Bergin
@ 2020-11-12 11:07 ` ernst.sjostrand
  2020-11-12 11:38   ` [OE-core] " Alexander Kanavin
  0 siblings, 1 reply; 7+ messages in thread
From: ernst.sjostrand @ 2020-11-12 11:07 UTC (permalink / raw)
  To: openembedded-core

[-- Attachment #1: Type: text/plain, Size: 266 bytes --]

Here are two other threads on the same topic. I've also run into this.

https://lists.yoctoproject.org/g/yocto/topic/74637733#51189

and

https://lists.yoctoproject.org/g/yocto/message/50856?p=,,,20,0,0,0::Created,,devtool,20,2,0,77178360

Regards
//Ernst

[-- Attachment #2: Type: text/html, Size: 330 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
  2020-11-12 11:07 ` ernst.sjostrand
@ 2020-11-12 11:38   ` Alexander Kanavin
  2020-11-12 12:28     ` Martin Jansa
  2020-11-12 12:59     ` Peter Bergin
  0 siblings, 2 replies; 7+ messages in thread
From: Alexander Kanavin @ 2020-11-12 11:38 UTC (permalink / raw)
  To: ernst.sjostrand, Peter Bergin; +Cc: OE-core

[-- Attachment #1: Type: text/plain, Size: 590 bytes --]

If you can provide exact steps to reproduce (the recipe is not available in
oe-core), I can take a look. It's been reported plenty of times, but always
with something not in core and not easily available.

Alex

On Thu, 12 Nov 2020 at 12:07, <ernst.sjostrand@lists.verisure.com> wrote:

> Here are two other threads on the same topic. I've also run into this.
>
> https://lists.yoctoproject.org/g/yocto/topic/74637733#51189
>
> and
>
>
> https://lists.yoctoproject.org/g/yocto/message/50856?p=,,,20,0,0,0::Created,,devtool,20,2,0,77178360
>
> Regards
> //Ernst
> 
>
>

[-- Attachment #2: Type: text/html, Size: 1163 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
  2020-11-12 11:38   ` [OE-core] " Alexander Kanavin
@ 2020-11-12 12:28     ` Martin Jansa
  2020-11-12 12:59     ` Peter Bergin
  1 sibling, 0 replies; 7+ messages in thread
From: Martin Jansa @ 2020-11-12 12:28 UTC (permalink / raw)
  To: Alexander Kanavin; +Cc: ernst.sjostrand, Peter Bergin, OE-core

[-- Attachment #1: Type: text/plain, Size: 2775 bytes --]

Breaking host python in ubuntu 18.04-20.10 like this is very easy, just
exporting _PYTHON_SYSCONFIGDATA_NAME like OE tasks do is enough.

I can reproduce it e.g. with command_not_found_handle in ubuntu:

$ export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
$ command_not_found_handle foo
Failed to import the site module
Traceback (most recent call last):
  File "/usr/lib/python3.6/site.py", line 570, in <module>
    main()
  File "/usr/lib/python3.6/site.py", line 556, in main
    known_paths = addusersitepackages(known_paths)
  File "/usr/lib/python3.6/site.py", line 288, in addusersitepackages
    user_site = getusersitepackages()
  File "/usr/lib/python3.6/site.py", line 264, in getusersitepackages
    user_base = getuserbase() # this will also set USER_BASE
  File "/usr/lib/python3.6/site.py", line 254, in getuserbase
    USER_BASE = get_config_var('userbase')
  File "/usr/lib/python3.6/sysconfig.py", line 607, in get_config_var
    return get_config_vars().get(name)
  File "/usr/lib/python3.6/sysconfig.py", line 550, in get_config_vars
    _init_posix(_CONFIG_VARS)
  File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix
    _temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
ModuleNotFoundError: No module named '_sysconfigdata'

$ unset _PYTHON_SYSCONFIGDATA_NAME
$ command_not_found_handle foo

Command 'foo' not found, did you mean:

  command 'fox' from deb objcryst-fox
  command 'fio' from deb fio
  command 'goo' from deb goo
  command 'fop' from deb fop
  command 'fog' from deb ruby-fog
  command 'woo' from deb python-woo
  command 'fgo' from deb fgo

Try: sudo apt install <deb name>

So whenever _PYTHON_SYSCONFIGDATA_NAME is exported you need to ensure that
you're exclusively using python from python3native any call from this
environment to host python (other than  python3 --version: Python 3.6.9).
With 20.04 it's usually more common to see, because default python on host
will be most likely python3 not python2 which doesn't have this issue.

On Thu, Nov 12, 2020 at 12:38 PM Alexander Kanavin <alex.kanavin@gmail.com>
wrote:

> If you can provide exact steps to reproduce (the recipe is not available
> in oe-core), I can take a look. It's been reported plenty of times, but
> always with something not in core and not easily available.
>
> Alex
>
> On Thu, 12 Nov 2020 at 12:07, <ernst.sjostrand@lists.verisure.com> wrote:
>
>> Here are two other threads on the same topic. I've also run into this.
>>
>> https://lists.yoctoproject.org/g/yocto/topic/74637733#51189
>>
>> and
>>
>>
>> https://lists.yoctoproject.org/g/yocto/message/50856?p=,,,20,0,0,0::Created,,devtool,20,2,0,77178360
>>
>> Regards
>> //Ernst
>>
>>
>>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 4019 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
  2020-11-12 11:38   ` [OE-core] " Alexander Kanavin
  2020-11-12 12:28     ` Martin Jansa
@ 2020-11-12 12:59     ` Peter Bergin
  2020-11-12 16:03       ` Alexander Kanavin
       [not found]       ` <1646CE2BCADFDD56.31886@lists.openembedded.org>
  1 sibling, 2 replies; 7+ messages in thread
From: Peter Bergin @ 2020-11-12 12:59 UTC (permalink / raw)
  To: Alexander Kanavin; +Cc: ernst.sjostrand, OE-core

Hi Alex,

On 2020-11-12 12:38, Alexander Kanavin wrote:
> If you can provide exact steps to reproduce (the recipe is not 
> available in oe-core), I can take a look. It's been reported plenty of 
> times, but always with something not in core and not easily available.

here are the steps to reproduce with only poky. I hope it can help you 
find the root cause:

     $ git clone https://git.yoctoproject.org/git/poky sources/poky
     $ . ./sources/poky/oe-init-build-env
     $ echo 'MACHINE = "qemuarm64"' >> conf/auto.conf
     $ devtool modify u-boot

And here is the output on my machine:

     Build Configuration:
     BB_VERSION           = "1.49.0"
     BUILD_SYS            = "x86_64-linux"
     NATIVELSBSTRING      = "ubuntu-20.04"
     TARGET_SYS           = "aarch64-poky-linux"
     MACHINE              = "qemuarm64"
     DISTRO               = "poky"
     DISTRO_VERSION       = "3.2+snapshot-20201112"
     TUNE_FEATURES        = "aarch64 armv8a crc cortexa57"
     TARGET_FPU           = ""
     meta
     meta-poky
     meta-yocto-bsp       = 
"master:e949e76d0f6897ec5d4feaa47afd2b049b32707f"
     workspace            = "<unknown>:<unknown>"

     Initialising tasks: 100% 
|###########################################################| Time: 0:00:00
     Sstate summary: Wanted 20 Found 20 Missed 0 Current 0 (100% match, 
0% complete)
     NOTE: Executing Tasks
     ERROR: Error executing a python function in exec_python_func() 
autogenerated:

     The stack trace of python calls that resulted in this 
exception/failure was:
     File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
          0001:
      *** 0002:devtool_post_unpack(d)
          0003:
     File: 
'/work/yocto/devtool-reproduce2/sources/poky/meta/classes/devtool-source.bbclass', 
lineno: 68, function: devtool_post_unpack
          0064:}
          0065:
          0066:
          0067:python devtool_post_unpack() {
      *** 0068:    import oe.recipeutils
          0069:    import shutil
          0070:    sys.path.insert(0, os.path.join(d.getVar('COREBASE'), 
'scripts', 'lib'))
          0071:    import scriptutils
          0072:    from devtool import setup_git_repo
     File: 
'/work/yocto/devtool-reproduce2/sources/poky/meta/lib/oe/recipeutils.py', 
lineno: 21, function: <module>
          0017:import shutil
          0018:import re
          0019:import fnmatch
          0020:import glob
      *** 0021:import bb.tinfoil
          0022:
          0023:from collections import OrderedDict, defaultdict
          0024:from bb.utils import vercmp_string
          0025:
     File: 
'/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/tinfoil.py', 
lineno: 19, function: <module>
          0015:from collections import OrderedDict, defaultdict
          0016:from functools import partial
          0017:
          0018:import bb.cache
      *** 0019:import bb.cooker
          0020:import bb.providers
          0021:import bb.taskdata
          0022:import bb.utils
          0023:import bb.command
     File: 
'/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/cooker.py', 
lineno: 25, function: <module>
          0021:import bb, bb.exceptions, bb.command
          0022:from bb import utils, data, parse, event, cache, 
providers, taskdata, runqueue, build
          0023:import queue
          0024:import signal
      *** 0025:import prserv.serv
          0026:import pyinotify
          0027:import json
          0028:import pickle
          0029:import codecs
     File: 
'/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/prserv/serv.py', 
lineno: 7, function: <module>
          0003:#
          0004:
          0005:import os,sys,logging
          0006:import signal, time
      *** 0007:from xmlrpc.server import SimpleXMLRPCServer, 
SimpleXMLRPCRequestHandler
          0008:import threading
          0009:import queue
          0010:import socket
          0011:import io
     File: '/usr/lib/python3.8/xmlrpc/server.py', lineno: 117, function: 
<module>
          0113:import socketserver
          0114:import sys
          0115:import os
          0116:import re
      *** 0117:import pydoc
          0118:import traceback
          0119:try:
          0120:    import fcntl
          0121:except ImportError:
     File: '/usr/lib/python3.8/pydoc.py', lineno: 370, function: <module>
          0366:    return module
          0367:
          0368:# ---------------------------------------------------- 
formatter base class
          0369:
      *** 0370:class Doc:
          0371:
          0372:    PYTHONDOCS = os.environ.get("PYTHONDOCS",
          0373: "https://docs.python.org/%d.%d/library"
          0374:                                % sys.version_info[:2])
     File: '/usr/lib/python3.8/pydoc.py', lineno: 400, function: Doc
          0396:        raise TypeError(message)
          0397:
          0398:    docmodule = docclass = docroutine = docother = 
docproperty = docdata = fail
          0399:
      *** 0400:    def getdocloc(self, object, 
basedir=sysconfig.get_path('stdlib')):
          0401:        """Return the location of module docs or None"""
          0402:
          0403:        try:
          0404:            file = inspect.getabsfile(object)
     File: '/usr/lib/python3.8/sysconfig.py', lineno: 512, function: 
get_path
          0508:    """Return a path corresponding to the scheme.
          0509:
          0510:    ``scheme`` is the install scheme name.
          0511:    """
      *** 0512:    return get_paths(scheme, vars, expand)[name]
          0513:
          0514:
          0515:def get_config_vars(*args):
          0516:    """With no arguments, return a dictionary of all 
configuration
     File: '/usr/lib/python3.8/sysconfig.py', lineno: 502, function: 
get_paths
          0498:    ``scheme`` is the install scheme name. If not 
provided, it will
          0499:    return the default scheme for the current platform.
          0500:    """
          0501:    if expand:
      *** 0502:        return _expand_vars(scheme, vars)
          0503:    else:
          0504:        return _INSTALL_SCHEMES[scheme]
          0505:
          0506:
     File: '/usr/lib/python3.8/sysconfig.py', lineno: 172, function: 
_expand_vars
          0168:def _expand_vars(scheme, vars):
          0169:    res = {}
          0170:    if vars is None:
          0171:        vars = {}
      *** 0172:    _extend_dict(vars, get_config_vars())
          0173:
          0174:    for key, value in _INSTALL_SCHEMES[scheme].items():
          0175:        if os.name in ('posix', 'nt'):
          0176:            value = os.path.expanduser(value)
     File: '/usr/lib/python3.8/sysconfig.py', lineno: 550, function: 
get_config_vars
          0546:
          0547:        if os.name == 'nt':
          0548:            _init_non_posix(_CONFIG_VARS)
          0549:        if os.name == 'posix':
      *** 0550:            _init_posix(_CONFIG_VARS)
          0551:        # For backward compatibility, see issue19555
          0552:        SO = _CONFIG_VARS.get('EXT_SUFFIX')
          0553:        if SO is not None:
          0554:            _CONFIG_VARS['SO'] = SO
     File: '/usr/lib/python3.8/sysconfig.py', lineno: 421, function: 
_init_posix
          0417:def _init_posix(vars):
          0418:    """Initialize the module as appropriate for POSIX 
systems."""
          0419:    # _sysconfigdata is generated at build time, see 
_generate_posix_vars()
          0420:    name = _get_sysconfigdata_name()
      *** 0421:    _temp = __import__(name, globals(), locals(), 
['build_time_vars'], 0)
          0422:    build_time_vars = _temp.build_time_vars
          0423:    vars.update(build_time_vars)
          0424:
          0425:def _init_non_posix(vars):
     Exception: ModuleNotFoundError: No module named '_sysconfigdata'


Thanks,
/Peter


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
  2020-11-12 12:59     ` Peter Bergin
@ 2020-11-12 16:03       ` Alexander Kanavin
       [not found]       ` <1646CE2BCADFDD56.31886@lists.openembedded.org>
  1 sibling, 0 replies; 7+ messages in thread
From: Alexander Kanavin @ 2020-11-12 16:03 UTC (permalink / raw)
  To: Peter Bergin; +Cc: ernst.sjostrand, OE-core, Martin Jansa

[-- Attachment #1: Type: text/plain, Size: 8799 bytes --]

I have just seen it first-hand for the first time. It happens on Ubuntu
20.04, but not on for example OpenSuse tumbleweed. So something in Ubuntu
causes the native python environment to leak into the host python
environment. Will investigate.

Alex

On Thu, 12 Nov 2020 at 13:59, Peter Bergin <peter@berginkonsult.se> wrote:

> Hi Alex,
>
> On 2020-11-12 12:38, Alexander Kanavin wrote:
> > If you can provide exact steps to reproduce (the recipe is not
> > available in oe-core), I can take a look. It's been reported plenty of
> > times, but always with something not in core and not easily available.
>
> here are the steps to reproduce with only poky. I hope it can help you
> find the root cause:
>
>      $ git clone https://git.yoctoproject.org/git/poky sources/poky
>      $ . ./sources/poky/oe-init-build-env
>      $ echo 'MACHINE = "qemuarm64"' >> conf/auto.conf
>      $ devtool modify u-boot
>
> And here is the output on my machine:
>
>      Build Configuration:
>      BB_VERSION           = "1.49.0"
>      BUILD_SYS            = "x86_64-linux"
>      NATIVELSBSTRING      = "ubuntu-20.04"
>      TARGET_SYS           = "aarch64-poky-linux"
>      MACHINE              = "qemuarm64"
>      DISTRO               = "poky"
>      DISTRO_VERSION       = "3.2+snapshot-20201112"
>      TUNE_FEATURES        = "aarch64 armv8a crc cortexa57"
>      TARGET_FPU           = ""
>      meta
>      meta-poky
>      meta-yocto-bsp       =
> "master:e949e76d0f6897ec5d4feaa47afd2b049b32707f"
>      workspace            = "<unknown>:<unknown>"
>
>      Initialising tasks: 100%
> |###########################################################| Time: 0:00:00
>      Sstate summary: Wanted 20 Found 20 Missed 0 Current 0 (100% match,
> 0% complete)
>      NOTE: Executing Tasks
>      ERROR: Error executing a python function in exec_python_func()
> autogenerated:
>
>      The stack trace of python calls that resulted in this
> exception/failure was:
>      File: 'exec_python_func() autogenerated', lineno: 2, function:
> <module>
>           0001:
>       *** 0002:devtool_post_unpack(d)
>           0003:
>      File:
> '/work/yocto/devtool-reproduce2/sources/poky/meta/classes/devtool-source.bbclass',
>
> lineno: 68, function: devtool_post_unpack
>           0064:}
>           0065:
>           0066:
>           0067:python devtool_post_unpack() {
>       *** 0068:    import oe.recipeutils
>           0069:    import shutil
>           0070:    sys.path.insert(0, os.path.join(d.getVar('COREBASE'),
> 'scripts', 'lib'))
>           0071:    import scriptutils
>           0072:    from devtool import setup_git_repo
>      File:
> '/work/yocto/devtool-reproduce2/sources/poky/meta/lib/oe/recipeutils.py',
> lineno: 21, function: <module>
>           0017:import shutil
>           0018:import re
>           0019:import fnmatch
>           0020:import glob
>       *** 0021:import bb.tinfoil
>           0022:
>           0023:from collections import OrderedDict, defaultdict
>           0024:from bb.utils import vercmp_string
>           0025:
>      File:
> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/tinfoil.py',
> lineno: 19, function: <module>
>           0015:from collections import OrderedDict, defaultdict
>           0016:from functools import partial
>           0017:
>           0018:import bb.cache
>       *** 0019:import bb.cooker
>           0020:import bb.providers
>           0021:import bb.taskdata
>           0022:import bb.utils
>           0023:import bb.command
>      File:
> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/cooker.py',
> lineno: 25, function: <module>
>           0021:import bb, bb.exceptions, bb.command
>           0022:from bb import utils, data, parse, event, cache,
> providers, taskdata, runqueue, build
>           0023:import queue
>           0024:import signal
>       *** 0025:import prserv.serv
>           0026:import pyinotify
>           0027:import json
>           0028:import pickle
>           0029:import codecs
>      File:
> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/prserv/serv.py',
> lineno: 7, function: <module>
>           0003:#
>           0004:
>           0005:import os,sys,logging
>           0006:import signal, time
>       *** 0007:from xmlrpc.server import SimpleXMLRPCServer,
> SimpleXMLRPCRequestHandler
>           0008:import threading
>           0009:import queue
>           0010:import socket
>           0011:import io
>      File: '/usr/lib/python3.8/xmlrpc/server.py', lineno: 117, function:
> <module>
>           0113:import socketserver
>           0114:import sys
>           0115:import os
>           0116:import re
>       *** 0117:import pydoc
>           0118:import traceback
>           0119:try:
>           0120:    import fcntl
>           0121:except ImportError:
>      File: '/usr/lib/python3.8/pydoc.py', lineno: 370, function: <module>
>           0366:    return module
>           0367:
>           0368:# ----------------------------------------------------
> formatter base class
>           0369:
>       *** 0370:class Doc:
>           0371:
>           0372:    PYTHONDOCS = os.environ.get("PYTHONDOCS",
>           0373: "https://docs.python.org/%d.%d/library"
>           0374:                                % sys.version_info[:2])
>      File: '/usr/lib/python3.8/pydoc.py', lineno: 400, function: Doc
>           0396:        raise TypeError(message)
>           0397:
>           0398:    docmodule = docclass = docroutine = docother =
> docproperty = docdata = fail
>           0399:
>       *** 0400:    def getdocloc(self, object,
> basedir=sysconfig.get_path('stdlib')):
>           0401:        """Return the location of module docs or None"""
>           0402:
>           0403:        try:
>           0404:            file = inspect.getabsfile(object)
>      File: '/usr/lib/python3.8/sysconfig.py', lineno: 512, function:
> get_path
>           0508:    """Return a path corresponding to the scheme.
>           0509:
>           0510:    ``scheme`` is the install scheme name.
>           0511:    """
>       *** 0512:    return get_paths(scheme, vars, expand)[name]
>           0513:
>           0514:
>           0515:def get_config_vars(*args):
>           0516:    """With no arguments, return a dictionary of all
> configuration
>      File: '/usr/lib/python3.8/sysconfig.py', lineno: 502, function:
> get_paths
>           0498:    ``scheme`` is the install scheme name. If not
> provided, it will
>           0499:    return the default scheme for the current platform.
>           0500:    """
>           0501:    if expand:
>       *** 0502:        return _expand_vars(scheme, vars)
>           0503:    else:
>           0504:        return _INSTALL_SCHEMES[scheme]
>           0505:
>           0506:
>      File: '/usr/lib/python3.8/sysconfig.py', lineno: 172, function:
> _expand_vars
>           0168:def _expand_vars(scheme, vars):
>           0169:    res = {}
>           0170:    if vars is None:
>           0171:        vars = {}
>       *** 0172:    _extend_dict(vars, get_config_vars())
>           0173:
>           0174:    for key, value in _INSTALL_SCHEMES[scheme].items():
>           0175:        if os.name in ('posix', 'nt'):
>           0176:            value = os.path.expanduser(value)
>      File: '/usr/lib/python3.8/sysconfig.py', lineno: 550, function:
> get_config_vars
>           0546:
>           0547:        if os.name == 'nt':
>           0548:            _init_non_posix(_CONFIG_VARS)
>           0549:        if os.name == 'posix':
>       *** 0550:            _init_posix(_CONFIG_VARS)
>           0551:        # For backward compatibility, see issue19555
>           0552:        SO = _CONFIG_VARS.get('EXT_SUFFIX')
>           0553:        if SO is not None:
>           0554:            _CONFIG_VARS['SO'] = SO
>      File: '/usr/lib/python3.8/sysconfig.py', lineno: 421, function:
> _init_posix
>           0417:def _init_posix(vars):
>           0418:    """Initialize the module as appropriate for POSIX
> systems."""
>           0419:    # _sysconfigdata is generated at build time, see
> _generate_posix_vars()
>           0420:    name = _get_sysconfigdata_name()
>       *** 0421:    _temp = __import__(name, globals(), locals(),
> ['build_time_vars'], 0)
>           0422:    build_time_vars = _temp.build_time_vars
>           0423:    vars.update(build_time_vars)
>           0424:
>           0425:def _init_non_posix(vars):
>      Exception: ModuleNotFoundError: No module named '_sysconfigdata'
>
>
> Thanks,
> /Peter
>
>

[-- Attachment #2: Type: text/html, Size: 11760 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [OE-core] devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
       [not found]       ` <1646CE2BCADFDD56.31886@lists.openembedded.org>
@ 2020-11-12 19:25         ` Alexander Kanavin
  0 siblings, 0 replies; 7+ messages in thread
From: Alexander Kanavin @ 2020-11-12 19:25 UTC (permalink / raw)
  To: OE-core; +Cc: Peter Bergin, ernst.sjostrand, Martin Jansa

[-- Attachment #1: Type: text/plain, Size: 9485 bytes --]

I have a tentative fix, but need to test it thoroughly, and adjust recipes
to use the new class where needed:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=akanavin/package-version-updates&id=d6b6a6f67cc967c0ead6dbab3c95435b7ca85246

It does resolve the devtool issue though.

Alex

On Thu, 12 Nov 2020 at 17:03, Alexander Kanavin via lists.openembedded.org
<alex.kanavin=gmail.com@lists.openembedded.org> wrote:

> I have just seen it first-hand for the first time. It happens on Ubuntu
> 20.04, but not on for example OpenSuse tumbleweed. So something in Ubuntu
> causes the native python environment to leak into the host python
> environment. Will investigate.
>
> Alex
>
> On Thu, 12 Nov 2020 at 13:59, Peter Bergin <peter@berginkonsult.se> wrote:
>
>> Hi Alex,
>>
>> On 2020-11-12 12:38, Alexander Kanavin wrote:
>> > If you can provide exact steps to reproduce (the recipe is not
>> > available in oe-core), I can take a look. It's been reported plenty of
>> > times, but always with something not in core and not easily available.
>>
>> here are the steps to reproduce with only poky. I hope it can help you
>> find the root cause:
>>
>>      $ git clone https://git.yoctoproject.org/git/poky sources/poky
>>      $ . ./sources/poky/oe-init-build-env
>>      $ echo 'MACHINE = "qemuarm64"' >> conf/auto.conf
>>      $ devtool modify u-boot
>>
>> And here is the output on my machine:
>>
>>      Build Configuration:
>>      BB_VERSION           = "1.49.0"
>>      BUILD_SYS            = "x86_64-linux"
>>      NATIVELSBSTRING      = "ubuntu-20.04"
>>      TARGET_SYS           = "aarch64-poky-linux"
>>      MACHINE              = "qemuarm64"
>>      DISTRO               = "poky"
>>      DISTRO_VERSION       = "3.2+snapshot-20201112"
>>      TUNE_FEATURES        = "aarch64 armv8a crc cortexa57"
>>      TARGET_FPU           = ""
>>      meta
>>      meta-poky
>>      meta-yocto-bsp       =
>> "master:e949e76d0f6897ec5d4feaa47afd2b049b32707f"
>>      workspace            = "<unknown>:<unknown>"
>>
>>      Initialising tasks: 100%
>> |###########################################################| Time:
>> 0:00:00
>>      Sstate summary: Wanted 20 Found 20 Missed 0 Current 0 (100% match,
>> 0% complete)
>>      NOTE: Executing Tasks
>>      ERROR: Error executing a python function in exec_python_func()
>> autogenerated:
>>
>>      The stack trace of python calls that resulted in this
>> exception/failure was:
>>      File: 'exec_python_func() autogenerated', lineno: 2, function:
>> <module>
>>           0001:
>>       *** 0002:devtool_post_unpack(d)
>>           0003:
>>      File:
>> '/work/yocto/devtool-reproduce2/sources/poky/meta/classes/devtool-source.bbclass',
>>
>> lineno: 68, function: devtool_post_unpack
>>           0064:}
>>           0065:
>>           0066:
>>           0067:python devtool_post_unpack() {
>>       *** 0068:    import oe.recipeutils
>>           0069:    import shutil
>>           0070:    sys.path.insert(0, os.path.join(d.getVar('COREBASE'),
>> 'scripts', 'lib'))
>>           0071:    import scriptutils
>>           0072:    from devtool import setup_git_repo
>>      File:
>> '/work/yocto/devtool-reproduce2/sources/poky/meta/lib/oe/recipeutils.py',
>> lineno: 21, function: <module>
>>           0017:import shutil
>>           0018:import re
>>           0019:import fnmatch
>>           0020:import glob
>>       *** 0021:import bb.tinfoil
>>           0022:
>>           0023:from collections import OrderedDict, defaultdict
>>           0024:from bb.utils import vercmp_string
>>           0025:
>>      File:
>> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/tinfoil.py',
>> lineno: 19, function: <module>
>>           0015:from collections import OrderedDict, defaultdict
>>           0016:from functools import partial
>>           0017:
>>           0018:import bb.cache
>>       *** 0019:import bb.cooker
>>           0020:import bb.providers
>>           0021:import bb.taskdata
>>           0022:import bb.utils
>>           0023:import bb.command
>>      File:
>> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/cooker.py',
>> lineno: 25, function: <module>
>>           0021:import bb, bb.exceptions, bb.command
>>           0022:from bb import utils, data, parse, event, cache,
>> providers, taskdata, runqueue, build
>>           0023:import queue
>>           0024:import signal
>>       *** 0025:import prserv.serv
>>           0026:import pyinotify
>>           0027:import json
>>           0028:import pickle
>>           0029:import codecs
>>      File:
>> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/prserv/serv.py',
>> lineno: 7, function: <module>
>>           0003:#
>>           0004:
>>           0005:import os,sys,logging
>>           0006:import signal, time
>>       *** 0007:from xmlrpc.server import SimpleXMLRPCServer,
>> SimpleXMLRPCRequestHandler
>>           0008:import threading
>>           0009:import queue
>>           0010:import socket
>>           0011:import io
>>      File: '/usr/lib/python3.8/xmlrpc/server.py', lineno: 117, function:
>> <module>
>>           0113:import socketserver
>>           0114:import sys
>>           0115:import os
>>           0116:import re
>>       *** 0117:import pydoc
>>           0118:import traceback
>>           0119:try:
>>           0120:    import fcntl
>>           0121:except ImportError:
>>      File: '/usr/lib/python3.8/pydoc.py', lineno: 370, function: <module>
>>           0366:    return module
>>           0367:
>>           0368:# ----------------------------------------------------
>> formatter base class
>>           0369:
>>       *** 0370:class Doc:
>>           0371:
>>           0372:    PYTHONDOCS = os.environ.get("PYTHONDOCS",
>>           0373: "https://docs.python.org/%d.%d/library"
>>           0374:                                % sys.version_info[:2])
>>      File: '/usr/lib/python3.8/pydoc.py', lineno: 400, function: Doc
>>           0396:        raise TypeError(message)
>>           0397:
>>           0398:    docmodule = docclass = docroutine = docother =
>> docproperty = docdata = fail
>>           0399:
>>       *** 0400:    def getdocloc(self, object,
>> basedir=sysconfig.get_path('stdlib')):
>>           0401:        """Return the location of module docs or None"""
>>           0402:
>>           0403:        try:
>>           0404:            file = inspect.getabsfile(object)
>>      File: '/usr/lib/python3.8/sysconfig.py', lineno: 512, function:
>> get_path
>>           0508:    """Return a path corresponding to the scheme.
>>           0509:
>>           0510:    ``scheme`` is the install scheme name.
>>           0511:    """
>>       *** 0512:    return get_paths(scheme, vars, expand)[name]
>>           0513:
>>           0514:
>>           0515:def get_config_vars(*args):
>>           0516:    """With no arguments, return a dictionary of all
>> configuration
>>      File: '/usr/lib/python3.8/sysconfig.py', lineno: 502, function:
>> get_paths
>>           0498:    ``scheme`` is the install scheme name. If not
>> provided, it will
>>           0499:    return the default scheme for the current platform.
>>           0500:    """
>>           0501:    if expand:
>>       *** 0502:        return _expand_vars(scheme, vars)
>>           0503:    else:
>>           0504:        return _INSTALL_SCHEMES[scheme]
>>           0505:
>>           0506:
>>      File: '/usr/lib/python3.8/sysconfig.py', lineno: 172, function:
>> _expand_vars
>>           0168:def _expand_vars(scheme, vars):
>>           0169:    res = {}
>>           0170:    if vars is None:
>>           0171:        vars = {}
>>       *** 0172:    _extend_dict(vars, get_config_vars())
>>           0173:
>>           0174:    for key, value in _INSTALL_SCHEMES[scheme].items():
>>           0175:        if os.name in ('posix', 'nt'):
>>           0176:            value = os.path.expanduser(value)
>>      File: '/usr/lib/python3.8/sysconfig.py', lineno: 550, function:
>> get_config_vars
>>           0546:
>>           0547:        if os.name == 'nt':
>>           0548:            _init_non_posix(_CONFIG_VARS)
>>           0549:        if os.name == 'posix':
>>       *** 0550:            _init_posix(_CONFIG_VARS)
>>           0551:        # For backward compatibility, see issue19555
>>           0552:        SO = _CONFIG_VARS.get('EXT_SUFFIX')
>>           0553:        if SO is not None:
>>           0554:            _CONFIG_VARS['SO'] = SO
>>      File: '/usr/lib/python3.8/sysconfig.py', lineno: 421, function:
>> _init_posix
>>           0417:def _init_posix(vars):
>>           0418:    """Initialize the module as appropriate for POSIX
>> systems."""
>>           0419:    # _sysconfigdata is generated at build time, see
>> _generate_posix_vars()
>>           0420:    name = _get_sysconfigdata_name()
>>       *** 0421:    _temp = __import__(name, globals(), locals(),
>> ['build_time_vars'], 0)
>>           0422:    build_time_vars = _temp.build_time_vars
>>           0423:    vars.update(build_time_vars)
>>           0424:
>>           0425:def _init_non_posix(vars):
>>      Exception: ModuleNotFoundError: No module named '_sysconfigdata'
>>
>>
>> Thanks,
>> /Peter
>>
>>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 12800 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2020-11-12 19:25 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-12 11:00 devtool modify - ModuleNotFoundError: No module named '_sysconfigdata' Peter Bergin
2020-11-12 11:07 ` ernst.sjostrand
2020-11-12 11:38   ` [OE-core] " Alexander Kanavin
2020-11-12 12:28     ` Martin Jansa
2020-11-12 12:59     ` Peter Bergin
2020-11-12 16:03       ` Alexander Kanavin
     [not found]       ` <1646CE2BCADFDD56.31886@lists.openembedded.org>
2020-11-12 19:25         ` Alexander Kanavin

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.