* devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
@ 2020-11-12 11:00 Peter Bergin
2020-11-12 11:07 ` ernst.sjostrand
0 siblings, 1 reply; 7+ messages in thread
From: Peter Bergin @ 2020-11-12 11:00 UTC (permalink / raw)
To: Patches and discussions about the oe-core layer
Hi,
when running 'devtool modify u-boot-imx' I get a stacktrace ending up in
"ModuleNotFoundError: No module named '_sysconfigdata'". 'devtool
modify' works for other recipes. Detailed stacktrace and layer info is
added below.
The stacktrace seems very similar to
https://lists.openembedded.org/g/openembedded-core/message/144484 but I
decided to start a new thread to not highjack another issue. As the
stacktarces are very similar I have guess that they are related in some
way. My host machine is running Ubuntu 20.04.
I'm out of ideas how to debug this further and would like some help to
sort this out.
Best regards,
/Peter
$ devtool modify u-boot-imx
...
Build Configuration:
BB_VERSION = "1.49.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "universal"
TARGET_SYS = "aarch64-fslc-linux"
MACHINE = "imx8mnevk"
DISTRO = "fslc-wayland"
DISTRO_VERSION = "3.2-snapshot-20201112"
TUNE_FEATURES = "aarch64 armv8a crc cortexa53 crypto"
TARGET_FPU = ""
meta
meta-poky =
"master:be17f6bacc6ba8ee5c92cf4b2ec5b43c8e660b33"
meta-freescale =
"master:447032d809427a20cc066b32254cea8821073281"
meta-freescale-3rdparty =
"master:b85d08a55cb833bfc4e8b5034ff804286c67620e"
meta-freescale-distro =
"master:11be3f01962df8436c5c7b0d61cd3dbd1b872905"
meta-oe
meta-networking
meta-python =
"master:b6e14805923a0dec17140c9df45434a9bea951d9"
ERROR: Error executing a python function in exec_python_func()
autogenerated:
The stack trace of python calls that resulted in this
exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:devtool_post_unpack(d)
0003:
File:
'/work/yocto/elektron-master/sources/poky/meta/classes/devtool-source.bbclass',
lineno: 68, function: devtool_post_unpack
0064:}
0065:
0066:
0067:python devtool_post_unpack() {
*** 0068: import oe.recipeutils
0069: import shutil
0070: sys.path.insert(0, os.path.join(d.getVar('COREBASE'),
'scripts', 'lib'))
0071: import scriptutils
0072: from devtool import setup_git_repo
File:
'/work/yocto/elektron-master/sources/poky/meta/lib/oe/recipeutils.py',
lineno: 21, function: <module>
0017:import shutil
0018:import re
0019:import fnmatch
0020:import glob
*** 0021:import bb.tinfoil
0022:
0023:from collections import OrderedDict, defaultdict
0024:from bb.utils import vercmp_string
0025:
File:
'/work/yocto/elektron-master/sources/poky/bitbake/lib/bb/tinfoil.py',
lineno: 19, function: <module>
0015:from collections import OrderedDict, defaultdict
0016:from functools import partial
0017:
0018:import bb.cache
*** 0019:import bb.cooker
0020:import bb.providers
0021:import bb.taskdata
0022:import bb.utils
0023:import bb.command
File:
'/work/yocto/elektron-master/sources/poky/bitbake/lib/bb/cooker.py',
lineno: 25, function: <module>
0021:import bb, bb.exceptions, bb.command
0022:from bb import utils, data, parse, event, cache,
providers, taskdata, runqueue, build
0023:import queue
0024:import signal
*** 0025:import prserv.serv
0026:import pyinotify
0027:import json
0028:import pickle
0029:import codecs
File:
'/work/yocto/elektron-master/sources/poky/bitbake/lib/prserv/serv.py',
lineno: 7, function: <module>
0003:#
0004:
0005:import os,sys,logging
0006:import signal, time
*** 0007:from xmlrpc.server import SimpleXMLRPCServer,
SimpleXMLRPCRequestHandler
0008:import threading
0009:import queue
0010:import socket
0011:import io
File: '/usr/lib/python3.8/xmlrpc/server.py', lineno: 117, function:
<module>
0113:import socketserver
0114:import sys
0115:import os
0116:import re
*** 0117:import pydoc
0118:import traceback
0119:try:
0120: import fcntl
0121:except ImportError:
File: '/usr/lib/python3.8/pydoc.py', lineno: 370, function: <module>
0366: return module
0367:
0368:# ----------------------------------------------------
formatter base class
0369:
*** 0370:class Doc:
0371:
0372: PYTHONDOCS = os.environ.get("PYTHONDOCS",
0373: "https://docs.python.org/%d.%d/library"
0374: % sys.version_info[:2])
File: '/usr/lib/python3.8/pydoc.py', lineno: 400, function: Doc
0396: raise TypeError(message)
0397:
0398: docmodule = docclass = docroutine = docother =
docproperty = docdata = fail
0399:
*** 0400: def getdocloc(self, object,
basedir=sysconfig.get_path('stdlib')):
0401: """Return the location of module docs or None"""
0402:
0403: try:
0404: file = inspect.getabsfile(object)
File: '/usr/lib/python3.8/sysconfig.py', lineno: 512, function:
get_path
0508: """Return a path corresponding to the scheme.
0509:
0510: ``scheme`` is the install scheme name.
0511: """
*** 0512: return get_paths(scheme, vars, expand)[name]
0513:
0514:
0515:def get_config_vars(*args):
0516: """With no arguments, return a dictionary of all
configuration
File: '/usr/lib/python3.8/sysconfig.py', lineno: 502, function:
get_paths
0498: ``scheme`` is the install scheme name. If not
provided, it will
0499: return the default scheme for the current platform.
0500: """
0501: if expand:
*** 0502: return _expand_vars(scheme, vars)
0503: else:
0504: return _INSTALL_SCHEMES[scheme]
0505:
0506:
File: '/usr/lib/python3.8/sysconfig.py', lineno: 172, function:
_expand_vars
0168:def _expand_vars(scheme, vars):
0169: res = {}
0170: if vars is None:
0171: vars = {}
*** 0172: _extend_dict(vars, get_config_vars())
0173:
0174: for key, value in _INSTALL_SCHEMES[scheme].items():
0175: if os.name in ('posix', 'nt'):
0176: value = os.path.expanduser(value)
File: '/usr/lib/python3.8/sysconfig.py', lineno: 550, function:
get_config_vars
0546:
0547: if os.name == 'nt':
0548: _init_non_posix(_CONFIG_VARS)
0549: if os.name == 'posix':
*** 0550: _init_posix(_CONFIG_VARS)
0551: # For backward compatibility, see issue19555
0552: SO = _CONFIG_VARS.get('EXT_SUFFIX')
0553: if SO is not None:
0554: _CONFIG_VARS['SO'] = SO
File: '/usr/lib/python3.8/sysconfig.py', lineno: 421, function:
_init_posix
0417:def _init_posix(vars):
0418: """Initialize the module as appropriate for POSIX
systems."""
0419: # _sysconfigdata is generated at build time, see
_generate_posix_vars()
0420: name = _get_sysconfigdata_name()
*** 0421: _temp = __import__(name, globals(), locals(),
['build_time_vars'], 0)
0422: build_time_vars = _temp.build_time_vars
0423: vars.update(build_time_vars)
0424:
0425:def _init_non_posix(vars):
Exception: ModuleNotFoundError: No module named '_sysconfigdata'
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
2020-11-12 11:00 devtool modify - ModuleNotFoundError: No module named '_sysconfigdata' Peter Bergin
@ 2020-11-12 11:07 ` ernst.sjostrand
2020-11-12 11:38 ` [OE-core] " Alexander Kanavin
0 siblings, 1 reply; 7+ messages in thread
From: ernst.sjostrand @ 2020-11-12 11:07 UTC (permalink / raw)
To: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 266 bytes --]
Here are two other threads on the same topic. I've also run into this.
https://lists.yoctoproject.org/g/yocto/topic/74637733#51189
and
https://lists.yoctoproject.org/g/yocto/message/50856?p=,,,20,0,0,0::Created,,devtool,20,2,0,77178360
Regards
//Ernst
[-- Attachment #2: Type: text/html, Size: 330 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
2020-11-12 11:07 ` ernst.sjostrand
@ 2020-11-12 11:38 ` Alexander Kanavin
2020-11-12 12:28 ` Martin Jansa
2020-11-12 12:59 ` Peter Bergin
0 siblings, 2 replies; 7+ messages in thread
From: Alexander Kanavin @ 2020-11-12 11:38 UTC (permalink / raw)
To: ernst.sjostrand, Peter Bergin; +Cc: OE-core
[-- Attachment #1: Type: text/plain, Size: 590 bytes --]
If you can provide exact steps to reproduce (the recipe is not available in
oe-core), I can take a look. It's been reported plenty of times, but always
with something not in core and not easily available.
Alex
On Thu, 12 Nov 2020 at 12:07, <ernst.sjostrand@lists.verisure.com> wrote:
> Here are two other threads on the same topic. I've also run into this.
>
> https://lists.yoctoproject.org/g/yocto/topic/74637733#51189
>
> and
>
>
> https://lists.yoctoproject.org/g/yocto/message/50856?p=,,,20,0,0,0::Created,,devtool,20,2,0,77178360
>
> Regards
> //Ernst
>
>
>
[-- Attachment #2: Type: text/html, Size: 1163 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
2020-11-12 11:38 ` [OE-core] " Alexander Kanavin
@ 2020-11-12 12:28 ` Martin Jansa
2020-11-12 12:59 ` Peter Bergin
1 sibling, 0 replies; 7+ messages in thread
From: Martin Jansa @ 2020-11-12 12:28 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: ernst.sjostrand, Peter Bergin, OE-core
[-- Attachment #1: Type: text/plain, Size: 2775 bytes --]
Breaking host python in ubuntu 18.04-20.10 like this is very easy, just
exporting _PYTHON_SYSCONFIGDATA_NAME like OE tasks do is enough.
I can reproduce it e.g. with command_not_found_handle in ubuntu:
$ export _PYTHON_SYSCONFIGDATA_NAME="_sysconfigdata"
$ command_not_found_handle foo
Failed to import the site module
Traceback (most recent call last):
File "/usr/lib/python3.6/site.py", line 570, in <module>
main()
File "/usr/lib/python3.6/site.py", line 556, in main
known_paths = addusersitepackages(known_paths)
File "/usr/lib/python3.6/site.py", line 288, in addusersitepackages
user_site = getusersitepackages()
File "/usr/lib/python3.6/site.py", line 264, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/usr/lib/python3.6/site.py", line 254, in getuserbase
USER_BASE = get_config_var('userbase')
File "/usr/lib/python3.6/sysconfig.py", line 607, in get_config_var
return get_config_vars().get(name)
File "/usr/lib/python3.6/sysconfig.py", line 550, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/usr/lib/python3.6/sysconfig.py", line 421, in _init_posix
_temp = __import__(name, globals(), locals(), ['build_time_vars'], 0)
ModuleNotFoundError: No module named '_sysconfigdata'
$ unset _PYTHON_SYSCONFIGDATA_NAME
$ command_not_found_handle foo
Command 'foo' not found, did you mean:
command 'fox' from deb objcryst-fox
command 'fio' from deb fio
command 'goo' from deb goo
command 'fop' from deb fop
command 'fog' from deb ruby-fog
command 'woo' from deb python-woo
command 'fgo' from deb fgo
Try: sudo apt install <deb name>
So whenever _PYTHON_SYSCONFIGDATA_NAME is exported you need to ensure that
you're exclusively using python from python3native any call from this
environment to host python (other than python3 --version: Python 3.6.9).
With 20.04 it's usually more common to see, because default python on host
will be most likely python3 not python2 which doesn't have this issue.
On Thu, Nov 12, 2020 at 12:38 PM Alexander Kanavin <alex.kanavin@gmail.com>
wrote:
> If you can provide exact steps to reproduce (the recipe is not available
> in oe-core), I can take a look. It's been reported plenty of times, but
> always with something not in core and not easily available.
>
> Alex
>
> On Thu, 12 Nov 2020 at 12:07, <ernst.sjostrand@lists.verisure.com> wrote:
>
>> Here are two other threads on the same topic. I've also run into this.
>>
>> https://lists.yoctoproject.org/g/yocto/topic/74637733#51189
>>
>> and
>>
>>
>> https://lists.yoctoproject.org/g/yocto/message/50856?p=,,,20,0,0,0::Created,,devtool,20,2,0,77178360
>>
>> Regards
>> //Ernst
>>
>>
>>
>
>
>
[-- Attachment #2: Type: text/html, Size: 4019 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
2020-11-12 11:38 ` [OE-core] " Alexander Kanavin
2020-11-12 12:28 ` Martin Jansa
@ 2020-11-12 12:59 ` Peter Bergin
2020-11-12 16:03 ` Alexander Kanavin
[not found] ` <1646CE2BCADFDD56.31886@lists.openembedded.org>
1 sibling, 2 replies; 7+ messages in thread
From: Peter Bergin @ 2020-11-12 12:59 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: ernst.sjostrand, OE-core
Hi Alex,
On 2020-11-12 12:38, Alexander Kanavin wrote:
> If you can provide exact steps to reproduce (the recipe is not
> available in oe-core), I can take a look. It's been reported plenty of
> times, but always with something not in core and not easily available.
here are the steps to reproduce with only poky. I hope it can help you
find the root cause:
$ git clone https://git.yoctoproject.org/git/poky sources/poky
$ . ./sources/poky/oe-init-build-env
$ echo 'MACHINE = "qemuarm64"' >> conf/auto.conf
$ devtool modify u-boot
And here is the output on my machine:
Build Configuration:
BB_VERSION = "1.49.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "ubuntu-20.04"
TARGET_SYS = "aarch64-poky-linux"
MACHINE = "qemuarm64"
DISTRO = "poky"
DISTRO_VERSION = "3.2+snapshot-20201112"
TUNE_FEATURES = "aarch64 armv8a crc cortexa57"
TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp =
"master:e949e76d0f6897ec5d4feaa47afd2b049b32707f"
workspace = "<unknown>:<unknown>"
Initialising tasks: 100%
|###########################################################| Time: 0:00:00
Sstate summary: Wanted 20 Found 20 Missed 0 Current 0 (100% match,
0% complete)
NOTE: Executing Tasks
ERROR: Error executing a python function in exec_python_func()
autogenerated:
The stack trace of python calls that resulted in this
exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:devtool_post_unpack(d)
0003:
File:
'/work/yocto/devtool-reproduce2/sources/poky/meta/classes/devtool-source.bbclass',
lineno: 68, function: devtool_post_unpack
0064:}
0065:
0066:
0067:python devtool_post_unpack() {
*** 0068: import oe.recipeutils
0069: import shutil
0070: sys.path.insert(0, os.path.join(d.getVar('COREBASE'),
'scripts', 'lib'))
0071: import scriptutils
0072: from devtool import setup_git_repo
File:
'/work/yocto/devtool-reproduce2/sources/poky/meta/lib/oe/recipeutils.py',
lineno: 21, function: <module>
0017:import shutil
0018:import re
0019:import fnmatch
0020:import glob
*** 0021:import bb.tinfoil
0022:
0023:from collections import OrderedDict, defaultdict
0024:from bb.utils import vercmp_string
0025:
File:
'/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/tinfoil.py',
lineno: 19, function: <module>
0015:from collections import OrderedDict, defaultdict
0016:from functools import partial
0017:
0018:import bb.cache
*** 0019:import bb.cooker
0020:import bb.providers
0021:import bb.taskdata
0022:import bb.utils
0023:import bb.command
File:
'/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/cooker.py',
lineno: 25, function: <module>
0021:import bb, bb.exceptions, bb.command
0022:from bb import utils, data, parse, event, cache,
providers, taskdata, runqueue, build
0023:import queue
0024:import signal
*** 0025:import prserv.serv
0026:import pyinotify
0027:import json
0028:import pickle
0029:import codecs
File:
'/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/prserv/serv.py',
lineno: 7, function: <module>
0003:#
0004:
0005:import os,sys,logging
0006:import signal, time
*** 0007:from xmlrpc.server import SimpleXMLRPCServer,
SimpleXMLRPCRequestHandler
0008:import threading
0009:import queue
0010:import socket
0011:import io
File: '/usr/lib/python3.8/xmlrpc/server.py', lineno: 117, function:
<module>
0113:import socketserver
0114:import sys
0115:import os
0116:import re
*** 0117:import pydoc
0118:import traceback
0119:try:
0120: import fcntl
0121:except ImportError:
File: '/usr/lib/python3.8/pydoc.py', lineno: 370, function: <module>
0366: return module
0367:
0368:# ----------------------------------------------------
formatter base class
0369:
*** 0370:class Doc:
0371:
0372: PYTHONDOCS = os.environ.get("PYTHONDOCS",
0373: "https://docs.python.org/%d.%d/library"
0374: % sys.version_info[:2])
File: '/usr/lib/python3.8/pydoc.py', lineno: 400, function: Doc
0396: raise TypeError(message)
0397:
0398: docmodule = docclass = docroutine = docother =
docproperty = docdata = fail
0399:
*** 0400: def getdocloc(self, object,
basedir=sysconfig.get_path('stdlib')):
0401: """Return the location of module docs or None"""
0402:
0403: try:
0404: file = inspect.getabsfile(object)
File: '/usr/lib/python3.8/sysconfig.py', lineno: 512, function:
get_path
0508: """Return a path corresponding to the scheme.
0509:
0510: ``scheme`` is the install scheme name.
0511: """
*** 0512: return get_paths(scheme, vars, expand)[name]
0513:
0514:
0515:def get_config_vars(*args):
0516: """With no arguments, return a dictionary of all
configuration
File: '/usr/lib/python3.8/sysconfig.py', lineno: 502, function:
get_paths
0498: ``scheme`` is the install scheme name. If not
provided, it will
0499: return the default scheme for the current platform.
0500: """
0501: if expand:
*** 0502: return _expand_vars(scheme, vars)
0503: else:
0504: return _INSTALL_SCHEMES[scheme]
0505:
0506:
File: '/usr/lib/python3.8/sysconfig.py', lineno: 172, function:
_expand_vars
0168:def _expand_vars(scheme, vars):
0169: res = {}
0170: if vars is None:
0171: vars = {}
*** 0172: _extend_dict(vars, get_config_vars())
0173:
0174: for key, value in _INSTALL_SCHEMES[scheme].items():
0175: if os.name in ('posix', 'nt'):
0176: value = os.path.expanduser(value)
File: '/usr/lib/python3.8/sysconfig.py', lineno: 550, function:
get_config_vars
0546:
0547: if os.name == 'nt':
0548: _init_non_posix(_CONFIG_VARS)
0549: if os.name == 'posix':
*** 0550: _init_posix(_CONFIG_VARS)
0551: # For backward compatibility, see issue19555
0552: SO = _CONFIG_VARS.get('EXT_SUFFIX')
0553: if SO is not None:
0554: _CONFIG_VARS['SO'] = SO
File: '/usr/lib/python3.8/sysconfig.py', lineno: 421, function:
_init_posix
0417:def _init_posix(vars):
0418: """Initialize the module as appropriate for POSIX
systems."""
0419: # _sysconfigdata is generated at build time, see
_generate_posix_vars()
0420: name = _get_sysconfigdata_name()
*** 0421: _temp = __import__(name, globals(), locals(),
['build_time_vars'], 0)
0422: build_time_vars = _temp.build_time_vars
0423: vars.update(build_time_vars)
0424:
0425:def _init_non_posix(vars):
Exception: ModuleNotFoundError: No module named '_sysconfigdata'
Thanks,
/Peter
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
2020-11-12 12:59 ` Peter Bergin
@ 2020-11-12 16:03 ` Alexander Kanavin
[not found] ` <1646CE2BCADFDD56.31886@lists.openembedded.org>
1 sibling, 0 replies; 7+ messages in thread
From: Alexander Kanavin @ 2020-11-12 16:03 UTC (permalink / raw)
To: Peter Bergin; +Cc: ernst.sjostrand, OE-core, Martin Jansa
[-- Attachment #1: Type: text/plain, Size: 8799 bytes --]
I have just seen it first-hand for the first time. It happens on Ubuntu
20.04, but not on for example OpenSuse tumbleweed. So something in Ubuntu
causes the native python environment to leak into the host python
environment. Will investigate.
Alex
On Thu, 12 Nov 2020 at 13:59, Peter Bergin <peter@berginkonsult.se> wrote:
> Hi Alex,
>
> On 2020-11-12 12:38, Alexander Kanavin wrote:
> > If you can provide exact steps to reproduce (the recipe is not
> > available in oe-core), I can take a look. It's been reported plenty of
> > times, but always with something not in core and not easily available.
>
> here are the steps to reproduce with only poky. I hope it can help you
> find the root cause:
>
> $ git clone https://git.yoctoproject.org/git/poky sources/poky
> $ . ./sources/poky/oe-init-build-env
> $ echo 'MACHINE = "qemuarm64"' >> conf/auto.conf
> $ devtool modify u-boot
>
> And here is the output on my machine:
>
> Build Configuration:
> BB_VERSION = "1.49.0"
> BUILD_SYS = "x86_64-linux"
> NATIVELSBSTRING = "ubuntu-20.04"
> TARGET_SYS = "aarch64-poky-linux"
> MACHINE = "qemuarm64"
> DISTRO = "poky"
> DISTRO_VERSION = "3.2+snapshot-20201112"
> TUNE_FEATURES = "aarch64 armv8a crc cortexa57"
> TARGET_FPU = ""
> meta
> meta-poky
> meta-yocto-bsp =
> "master:e949e76d0f6897ec5d4feaa47afd2b049b32707f"
> workspace = "<unknown>:<unknown>"
>
> Initialising tasks: 100%
> |###########################################################| Time: 0:00:00
> Sstate summary: Wanted 20 Found 20 Missed 0 Current 0 (100% match,
> 0% complete)
> NOTE: Executing Tasks
> ERROR: Error executing a python function in exec_python_func()
> autogenerated:
>
> The stack trace of python calls that resulted in this
> exception/failure was:
> File: 'exec_python_func() autogenerated', lineno: 2, function:
> <module>
> 0001:
> *** 0002:devtool_post_unpack(d)
> 0003:
> File:
> '/work/yocto/devtool-reproduce2/sources/poky/meta/classes/devtool-source.bbclass',
>
> lineno: 68, function: devtool_post_unpack
> 0064:}
> 0065:
> 0066:
> 0067:python devtool_post_unpack() {
> *** 0068: import oe.recipeutils
> 0069: import shutil
> 0070: sys.path.insert(0, os.path.join(d.getVar('COREBASE'),
> 'scripts', 'lib'))
> 0071: import scriptutils
> 0072: from devtool import setup_git_repo
> File:
> '/work/yocto/devtool-reproduce2/sources/poky/meta/lib/oe/recipeutils.py',
> lineno: 21, function: <module>
> 0017:import shutil
> 0018:import re
> 0019:import fnmatch
> 0020:import glob
> *** 0021:import bb.tinfoil
> 0022:
> 0023:from collections import OrderedDict, defaultdict
> 0024:from bb.utils import vercmp_string
> 0025:
> File:
> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/tinfoil.py',
> lineno: 19, function: <module>
> 0015:from collections import OrderedDict, defaultdict
> 0016:from functools import partial
> 0017:
> 0018:import bb.cache
> *** 0019:import bb.cooker
> 0020:import bb.providers
> 0021:import bb.taskdata
> 0022:import bb.utils
> 0023:import bb.command
> File:
> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/cooker.py',
> lineno: 25, function: <module>
> 0021:import bb, bb.exceptions, bb.command
> 0022:from bb import utils, data, parse, event, cache,
> providers, taskdata, runqueue, build
> 0023:import queue
> 0024:import signal
> *** 0025:import prserv.serv
> 0026:import pyinotify
> 0027:import json
> 0028:import pickle
> 0029:import codecs
> File:
> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/prserv/serv.py',
> lineno: 7, function: <module>
> 0003:#
> 0004:
> 0005:import os,sys,logging
> 0006:import signal, time
> *** 0007:from xmlrpc.server import SimpleXMLRPCServer,
> SimpleXMLRPCRequestHandler
> 0008:import threading
> 0009:import queue
> 0010:import socket
> 0011:import io
> File: '/usr/lib/python3.8/xmlrpc/server.py', lineno: 117, function:
> <module>
> 0113:import socketserver
> 0114:import sys
> 0115:import os
> 0116:import re
> *** 0117:import pydoc
> 0118:import traceback
> 0119:try:
> 0120: import fcntl
> 0121:except ImportError:
> File: '/usr/lib/python3.8/pydoc.py', lineno: 370, function: <module>
> 0366: return module
> 0367:
> 0368:# ----------------------------------------------------
> formatter base class
> 0369:
> *** 0370:class Doc:
> 0371:
> 0372: PYTHONDOCS = os.environ.get("PYTHONDOCS",
> 0373: "https://docs.python.org/%d.%d/library"
> 0374: % sys.version_info[:2])
> File: '/usr/lib/python3.8/pydoc.py', lineno: 400, function: Doc
> 0396: raise TypeError(message)
> 0397:
> 0398: docmodule = docclass = docroutine = docother =
> docproperty = docdata = fail
> 0399:
> *** 0400: def getdocloc(self, object,
> basedir=sysconfig.get_path('stdlib')):
> 0401: """Return the location of module docs or None"""
> 0402:
> 0403: try:
> 0404: file = inspect.getabsfile(object)
> File: '/usr/lib/python3.8/sysconfig.py', lineno: 512, function:
> get_path
> 0508: """Return a path corresponding to the scheme.
> 0509:
> 0510: ``scheme`` is the install scheme name.
> 0511: """
> *** 0512: return get_paths(scheme, vars, expand)[name]
> 0513:
> 0514:
> 0515:def get_config_vars(*args):
> 0516: """With no arguments, return a dictionary of all
> configuration
> File: '/usr/lib/python3.8/sysconfig.py', lineno: 502, function:
> get_paths
> 0498: ``scheme`` is the install scheme name. If not
> provided, it will
> 0499: return the default scheme for the current platform.
> 0500: """
> 0501: if expand:
> *** 0502: return _expand_vars(scheme, vars)
> 0503: else:
> 0504: return _INSTALL_SCHEMES[scheme]
> 0505:
> 0506:
> File: '/usr/lib/python3.8/sysconfig.py', lineno: 172, function:
> _expand_vars
> 0168:def _expand_vars(scheme, vars):
> 0169: res = {}
> 0170: if vars is None:
> 0171: vars = {}
> *** 0172: _extend_dict(vars, get_config_vars())
> 0173:
> 0174: for key, value in _INSTALL_SCHEMES[scheme].items():
> 0175: if os.name in ('posix', 'nt'):
> 0176: value = os.path.expanduser(value)
> File: '/usr/lib/python3.8/sysconfig.py', lineno: 550, function:
> get_config_vars
> 0546:
> 0547: if os.name == 'nt':
> 0548: _init_non_posix(_CONFIG_VARS)
> 0549: if os.name == 'posix':
> *** 0550: _init_posix(_CONFIG_VARS)
> 0551: # For backward compatibility, see issue19555
> 0552: SO = _CONFIG_VARS.get('EXT_SUFFIX')
> 0553: if SO is not None:
> 0554: _CONFIG_VARS['SO'] = SO
> File: '/usr/lib/python3.8/sysconfig.py', lineno: 421, function:
> _init_posix
> 0417:def _init_posix(vars):
> 0418: """Initialize the module as appropriate for POSIX
> systems."""
> 0419: # _sysconfigdata is generated at build time, see
> _generate_posix_vars()
> 0420: name = _get_sysconfigdata_name()
> *** 0421: _temp = __import__(name, globals(), locals(),
> ['build_time_vars'], 0)
> 0422: build_time_vars = _temp.build_time_vars
> 0423: vars.update(build_time_vars)
> 0424:
> 0425:def _init_non_posix(vars):
> Exception: ModuleNotFoundError: No module named '_sysconfigdata'
>
>
> Thanks,
> /Peter
>
>
[-- Attachment #2: Type: text/html, Size: 11760 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [OE-core] devtool modify - ModuleNotFoundError: No module named '_sysconfigdata'
[not found] ` <1646CE2BCADFDD56.31886@lists.openembedded.org>
@ 2020-11-12 19:25 ` Alexander Kanavin
0 siblings, 0 replies; 7+ messages in thread
From: Alexander Kanavin @ 2020-11-12 19:25 UTC (permalink / raw)
To: OE-core; +Cc: Peter Bergin, ernst.sjostrand, Martin Jansa
[-- Attachment #1: Type: text/plain, Size: 9485 bytes --]
I have a tentative fix, but need to test it thoroughly, and adjust recipes
to use the new class where needed:
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=akanavin/package-version-updates&id=d6b6a6f67cc967c0ead6dbab3c95435b7ca85246
It does resolve the devtool issue though.
Alex
On Thu, 12 Nov 2020 at 17:03, Alexander Kanavin via lists.openembedded.org
<alex.kanavin=gmail.com@lists.openembedded.org> wrote:
> I have just seen it first-hand for the first time. It happens on Ubuntu
> 20.04, but not on for example OpenSuse tumbleweed. So something in Ubuntu
> causes the native python environment to leak into the host python
> environment. Will investigate.
>
> Alex
>
> On Thu, 12 Nov 2020 at 13:59, Peter Bergin <peter@berginkonsult.se> wrote:
>
>> Hi Alex,
>>
>> On 2020-11-12 12:38, Alexander Kanavin wrote:
>> > If you can provide exact steps to reproduce (the recipe is not
>> > available in oe-core), I can take a look. It's been reported plenty of
>> > times, but always with something not in core and not easily available.
>>
>> here are the steps to reproduce with only poky. I hope it can help you
>> find the root cause:
>>
>> $ git clone https://git.yoctoproject.org/git/poky sources/poky
>> $ . ./sources/poky/oe-init-build-env
>> $ echo 'MACHINE = "qemuarm64"' >> conf/auto.conf
>> $ devtool modify u-boot
>>
>> And here is the output on my machine:
>>
>> Build Configuration:
>> BB_VERSION = "1.49.0"
>> BUILD_SYS = "x86_64-linux"
>> NATIVELSBSTRING = "ubuntu-20.04"
>> TARGET_SYS = "aarch64-poky-linux"
>> MACHINE = "qemuarm64"
>> DISTRO = "poky"
>> DISTRO_VERSION = "3.2+snapshot-20201112"
>> TUNE_FEATURES = "aarch64 armv8a crc cortexa57"
>> TARGET_FPU = ""
>> meta
>> meta-poky
>> meta-yocto-bsp =
>> "master:e949e76d0f6897ec5d4feaa47afd2b049b32707f"
>> workspace = "<unknown>:<unknown>"
>>
>> Initialising tasks: 100%
>> |###########################################################| Time:
>> 0:00:00
>> Sstate summary: Wanted 20 Found 20 Missed 0 Current 0 (100% match,
>> 0% complete)
>> NOTE: Executing Tasks
>> ERROR: Error executing a python function in exec_python_func()
>> autogenerated:
>>
>> The stack trace of python calls that resulted in this
>> exception/failure was:
>> File: 'exec_python_func() autogenerated', lineno: 2, function:
>> <module>
>> 0001:
>> *** 0002:devtool_post_unpack(d)
>> 0003:
>> File:
>> '/work/yocto/devtool-reproduce2/sources/poky/meta/classes/devtool-source.bbclass',
>>
>> lineno: 68, function: devtool_post_unpack
>> 0064:}
>> 0065:
>> 0066:
>> 0067:python devtool_post_unpack() {
>> *** 0068: import oe.recipeutils
>> 0069: import shutil
>> 0070: sys.path.insert(0, os.path.join(d.getVar('COREBASE'),
>> 'scripts', 'lib'))
>> 0071: import scriptutils
>> 0072: from devtool import setup_git_repo
>> File:
>> '/work/yocto/devtool-reproduce2/sources/poky/meta/lib/oe/recipeutils.py',
>> lineno: 21, function: <module>
>> 0017:import shutil
>> 0018:import re
>> 0019:import fnmatch
>> 0020:import glob
>> *** 0021:import bb.tinfoil
>> 0022:
>> 0023:from collections import OrderedDict, defaultdict
>> 0024:from bb.utils import vercmp_string
>> 0025:
>> File:
>> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/tinfoil.py',
>> lineno: 19, function: <module>
>> 0015:from collections import OrderedDict, defaultdict
>> 0016:from functools import partial
>> 0017:
>> 0018:import bb.cache
>> *** 0019:import bb.cooker
>> 0020:import bb.providers
>> 0021:import bb.taskdata
>> 0022:import bb.utils
>> 0023:import bb.command
>> File:
>> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/bb/cooker.py',
>> lineno: 25, function: <module>
>> 0021:import bb, bb.exceptions, bb.command
>> 0022:from bb import utils, data, parse, event, cache,
>> providers, taskdata, runqueue, build
>> 0023:import queue
>> 0024:import signal
>> *** 0025:import prserv.serv
>> 0026:import pyinotify
>> 0027:import json
>> 0028:import pickle
>> 0029:import codecs
>> File:
>> '/work/yocto/devtool-reproduce2/sources/poky/bitbake/lib/prserv/serv.py',
>> lineno: 7, function: <module>
>> 0003:#
>> 0004:
>> 0005:import os,sys,logging
>> 0006:import signal, time
>> *** 0007:from xmlrpc.server import SimpleXMLRPCServer,
>> SimpleXMLRPCRequestHandler
>> 0008:import threading
>> 0009:import queue
>> 0010:import socket
>> 0011:import io
>> File: '/usr/lib/python3.8/xmlrpc/server.py', lineno: 117, function:
>> <module>
>> 0113:import socketserver
>> 0114:import sys
>> 0115:import os
>> 0116:import re
>> *** 0117:import pydoc
>> 0118:import traceback
>> 0119:try:
>> 0120: import fcntl
>> 0121:except ImportError:
>> File: '/usr/lib/python3.8/pydoc.py', lineno: 370, function: <module>
>> 0366: return module
>> 0367:
>> 0368:# ----------------------------------------------------
>> formatter base class
>> 0369:
>> *** 0370:class Doc:
>> 0371:
>> 0372: PYTHONDOCS = os.environ.get("PYTHONDOCS",
>> 0373: "https://docs.python.org/%d.%d/library"
>> 0374: % sys.version_info[:2])
>> File: '/usr/lib/python3.8/pydoc.py', lineno: 400, function: Doc
>> 0396: raise TypeError(message)
>> 0397:
>> 0398: docmodule = docclass = docroutine = docother =
>> docproperty = docdata = fail
>> 0399:
>> *** 0400: def getdocloc(self, object,
>> basedir=sysconfig.get_path('stdlib')):
>> 0401: """Return the location of module docs or None"""
>> 0402:
>> 0403: try:
>> 0404: file = inspect.getabsfile(object)
>> File: '/usr/lib/python3.8/sysconfig.py', lineno: 512, function:
>> get_path
>> 0508: """Return a path corresponding to the scheme.
>> 0509:
>> 0510: ``scheme`` is the install scheme name.
>> 0511: """
>> *** 0512: return get_paths(scheme, vars, expand)[name]
>> 0513:
>> 0514:
>> 0515:def get_config_vars(*args):
>> 0516: """With no arguments, return a dictionary of all
>> configuration
>> File: '/usr/lib/python3.8/sysconfig.py', lineno: 502, function:
>> get_paths
>> 0498: ``scheme`` is the install scheme name. If not
>> provided, it will
>> 0499: return the default scheme for the current platform.
>> 0500: """
>> 0501: if expand:
>> *** 0502: return _expand_vars(scheme, vars)
>> 0503: else:
>> 0504: return _INSTALL_SCHEMES[scheme]
>> 0505:
>> 0506:
>> File: '/usr/lib/python3.8/sysconfig.py', lineno: 172, function:
>> _expand_vars
>> 0168:def _expand_vars(scheme, vars):
>> 0169: res = {}
>> 0170: if vars is None:
>> 0171: vars = {}
>> *** 0172: _extend_dict(vars, get_config_vars())
>> 0173:
>> 0174: for key, value in _INSTALL_SCHEMES[scheme].items():
>> 0175: if os.name in ('posix', 'nt'):
>> 0176: value = os.path.expanduser(value)
>> File: '/usr/lib/python3.8/sysconfig.py', lineno: 550, function:
>> get_config_vars
>> 0546:
>> 0547: if os.name == 'nt':
>> 0548: _init_non_posix(_CONFIG_VARS)
>> 0549: if os.name == 'posix':
>> *** 0550: _init_posix(_CONFIG_VARS)
>> 0551: # For backward compatibility, see issue19555
>> 0552: SO = _CONFIG_VARS.get('EXT_SUFFIX')
>> 0553: if SO is not None:
>> 0554: _CONFIG_VARS['SO'] = SO
>> File: '/usr/lib/python3.8/sysconfig.py', lineno: 421, function:
>> _init_posix
>> 0417:def _init_posix(vars):
>> 0418: """Initialize the module as appropriate for POSIX
>> systems."""
>> 0419: # _sysconfigdata is generated at build time, see
>> _generate_posix_vars()
>> 0420: name = _get_sysconfigdata_name()
>> *** 0421: _temp = __import__(name, globals(), locals(),
>> ['build_time_vars'], 0)
>> 0422: build_time_vars = _temp.build_time_vars
>> 0423: vars.update(build_time_vars)
>> 0424:
>> 0425:def _init_non_posix(vars):
>> Exception: ModuleNotFoundError: No module named '_sysconfigdata'
>>
>>
>> Thanks,
>> /Peter
>>
>>
>
>
>
[-- Attachment #2: Type: text/html, Size: 12800 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2020-11-12 19:25 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-12 11:00 devtool modify - ModuleNotFoundError: No module named '_sysconfigdata' Peter Bergin
2020-11-12 11:07 ` ernst.sjostrand
2020-11-12 11:38 ` [OE-core] " Alexander Kanavin
2020-11-12 12:28 ` Martin Jansa
2020-11-12 12:59 ` Peter Bergin
2020-11-12 16:03 ` Alexander Kanavin
[not found] ` <1646CE2BCADFDD56.31886@lists.openembedded.org>
2020-11-12 19:25 ` Alexander Kanavin
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.