* [PATCH v3 0/8] support profile-optimized build for Python
@ 2017-05-18 8:42 Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 1/8] python3-native: support profile optimized build Markus Lehtonen
` (7 more replies)
0 siblings, 8 replies; 9+ messages in thread
From: Markus Lehtonen @ 2017-05-18 8:42 UTC (permalink / raw)
To: openembedded-core
Third incarnation of enabling profile-guided-optimization (PGO) for Python. The
patchset is significantly simplified from the previous version, mainly because
Python 2.x is not supported, and because of that, there is no need to install
packages under modified (opt/) prefix. Also, RSS made libpython soname mangling
unnecessary.
The last patch of this changeset is not obligatory after the changes since v2.
However, it will remove some hacks that should not be needed.
Changes since v2:
- support for Python 2.x dropped
- profiling recipe name changed from python-profile-opt3 to python3-profile-opt
- python3-profile-opt now uses standard 'usr/' prefix, and thus, it now also
rconflicts with "regular" python3
- libpython name mangling in python3-profile-opt removed
- all patches to python-pgo-image squashed into one
[YOCTO #9338]
The following changes since commit 138f3e3f81d866e8272b01dd88c164b367adc4cd:
byacc: remove the recipe (2017-05-16 14:08:19 +0100)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib marquiz/fixes-9338
http://cgit.openembedded.org/openembedded-core-contrib/log/?h=marquiz/fixes-9338
Markus Lehtonen (8):
python3-native: support profile optimized build
python3: fix depends of python3-tests
python3: add python3-profile-opt recipe
devtools/images: add python-pgo-image
python3: support profile optimized build
python3: fix profile-optimized build of modules
python3: add python3-tools subpackage
python3: remove two setup.py cross-compile hacks
meta/recipes-devtools/images/python-pgo-image.bb | 85 ++++++++++++++++++++++
.../python/python-3.5-manifest.inc | 10 ++-
.../python/python3-native_3.5.2.bb | 9 +++
meta/recipes-devtools/python/python3-profile-opt | 1 +
.../python/python3-profile-opt_3.5.2.bb | 15 ++++
.../python/python3/000-cross-compile.patch | 9 ---
...efile-add-install_generate_profile-target.patch | 25 +++++++
...-CFLAGS-for-extensions-when-cross-compili.patch | 56 ++++++++++++++
.../python3-setup.py-no-host-headers-libs.patch | 33 ---------
.../setup.py-find-libraries-in-staging-dirs.patch | 30 --------
meta/recipes-devtools/python/python3_3.5.2.bb | 58 +++++++++++----
11 files changed, 239 insertions(+), 92 deletions(-)
create mode 100644 meta/recipes-devtools/images/python-pgo-image.bb
create mode 120000 meta/recipes-devtools/python/python3-profile-opt
create mode 100644 meta/recipes-devtools/python/python3-profile-opt_3.5.2.bb
create mode 100644 meta/recipes-devtools/python/python3/Makefile-add-install_generate_profile-target.patch
create mode 100644 meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch
delete mode 100644 meta/recipes-devtools/python/python3/python3-setup.py-no-host-headers-libs.patch
delete mode 100644 meta/recipes-devtools/python/python3/setup.py-find-libraries-in-staging-dirs.patch
--
2.12.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH v3 1/8] python3-native: support profile optimized build
2017-05-18 8:42 [PATCH v3 0/8] support profile-optimized build for Python Markus Lehtonen
@ 2017-05-18 8:42 ` Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 2/8] python3: fix depends of python3-tests Markus Lehtonen
` (6 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Markus Lehtonen @ 2017-05-18 8:42 UTC (permalink / raw)
To: openembedded-core
Make it possible to build python3-native with profile directed optimization
enabled. The feature is enabled by specifying
PYTHON3_NATIVE_PROFILE_OPT = "1" in local.conf.
The profile task to be run may be specified with PYTHON3_NATIVE_PROFILE_TASK
variable in local.conf, e.g.
PYTHON3_NATIVE_PROFILE_TASK = "${S}/Tools/pybench/pybench.py -n 2 --with-gc --with-syscheck"
[YOCTO #9338]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/recipes-devtools/python/python3-native_3.5.2.bb | 9 +++++++++
meta/recipes-devtools/python/python3/000-cross-compile.patch | 9 ---------
2 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/meta/recipes-devtools/python/python3-native_3.5.2.bb b/meta/recipes-devtools/python/python3-native_3.5.2.bb
index edcf2244f5..1d99e56054 100644
--- a/meta/recipes-devtools/python/python3-native_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3-native_3.5.2.bb
@@ -56,10 +56,19 @@ EXTRA_OEMAKE = '\
# No ctypes option for python 3
PYTHONLSBOPTS = ""
+PYTHON3_NATIVE_MAKE_TARGET ?= "${@'profile-opt' if d.getVar('PYTHON3_NATIVE_PROFILE_OPT', True) == '1' else ''}"
+
do_configure_append() {
autoreconf --verbose --install --force --exclude=autopoint ../Python-${PV}/Modules/_ctypes/libffi
}
+do_compile() {
+ if [ -n "${PYTHON3_NATIVE_PROFILE_TASK}" ]; then
+ sed -i -e 's,^PROFILE_TASK=.*,PROFILE_TASK=${PYTHON3_NATIVE_PROFILE_TASK},g' Makefile
+ fi
+ oe_runmake ${PYTHON3_NATIVE_MAKE_TARGET}
+}
+
do_install() {
install -d ${D}${libdir}/pkgconfig
oe_runmake 'DESTDIR=${D}' install
diff --git a/meta/recipes-devtools/python/python3/000-cross-compile.patch b/meta/recipes-devtools/python/python3/000-cross-compile.patch
index 2d822218f4..d6d7820ee8 100644
--- a/meta/recipes-devtools/python/python3/000-cross-compile.patch
+++ b/meta/recipes-devtools/python/python3/000-cross-compile.patch
@@ -29,15 +29,6 @@ Index: Python-3.5.2/Makefile.pre.in
PSRCS= \
Parser/acceler.c \
-@@ -509,7 +511,7 @@ build_all_generate_profile:
-
- run_profile_task:
- : # FIXME: can't run for a cross build
-- $(LLVM_PROF_FILE) $(RUNSHARED) ./$(BUILDPYTHON) $(PROFILE_TASK) || true
-+ $(LLVM_PROF_FILE) $(RUNSHARED) $(HOSTPYTHON) $(PROFILE_TASK) || true
-
- build_all_merge_profile:
- $(LLVM_PROF_MERGER)
@@ -792,7 +794,7 @@ $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGEN)
@$(MKDIR_P) Include
# Avoid copying the file onto itself for an in-tree build
--
2.12.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v3 2/8] python3: fix depends of python3-tests
2017-05-18 8:42 [PATCH v3 0/8] support profile-optimized build for Python Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 1/8] python3-native: support profile optimized build Markus Lehtonen
@ 2017-05-18 8:42 ` Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 3/8] python3: add python3-profile-opt recipe Markus Lehtonen
` (5 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Markus Lehtonen @ 2017-05-18 8:42 UTC (permalink / raw)
To: openembedded-core
Similar to an earlier fix for Python 2.7. Make the tests subpackage
depend on all modules as test.regrtest uses most (if not all) of them.
Some tests also depend on libgcc so add that as a runtime dependency as
well.
[YOCTO #9338]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/recipes-devtools/python/python-3.5-manifest.inc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/meta/recipes-devtools/python/python-3.5-manifest.inc b/meta/recipes-devtools/python/python-3.5-manifest.inc
index 1e20f00c64..3f29b53a73 100644
--- a/meta/recipes-devtools/python/python-3.5-manifest.inc
+++ b/meta/recipes-devtools/python/python-3.5-manifest.inc
@@ -241,7 +241,7 @@ RDEPENDS_${PN}-terminal="${PN}-core ${PN}-io"
FILES_${PN}-terminal="${libdir}/python3.5/pty.* ${libdir}/python3.5/__pycache__/pty.* ${libdir}/python3.5/tty.* ${libdir}/python3.5/__pycache__/tty.* "
SUMMARY_${PN}-tests="Python tests"
-RDEPENDS_${PN}-tests="${PN}-core"
+RDEPENDS_${PN}-tests="${PN}-core ${PN}-modules libgcc"
FILES_${PN}-tests="${libdir}/python3.5/test ${libdir}/python3.5/test/__pycache__ "
SUMMARY_${PN}-textutils="Python option parsing, text wrapping and CSV support"
--
2.12.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v3 3/8] python3: add python3-profile-opt recipe
2017-05-18 8:42 [PATCH v3 0/8] support profile-optimized build for Python Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 1/8] python3-native: support profile optimized build Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 2/8] python3: fix depends of python3-tests Markus Lehtonen
@ 2017-05-18 8:42 ` Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 4/8] devtools/images: add python-pgo-image Markus Lehtonen
` (4 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Markus Lehtonen @ 2017-05-18 8:42 UTC (permalink / raw)
To: openembedded-core
This patch adds a new recipe that builds a special version of python3 that
produces profile data used for optimization. The new recipe directly includes
the base python recipe so that they are build in as similar way as possible and
this hopefully decreases the recipe maintenance burden, too. Also, its files
clash with the "normal" python3 so they cannot be installed in an image at the
same time.
Normally, profile-guided-optimization in Python is done simply by doing "make
profile-opt" which first builds python with profile instrumentation enabled,
then runs a profile task to get the profile data, and last, re-builds python
with profile data guiding the optimization. However, in our cross-build
environment this gets a lot trickier. We need to split out the steps as
building is done on the build host but we need to run the second step (i.e. run
the profile task) on the target hardware. This patch enables the first step,
i.e. building python with profile instrumentation enabled
[YOCTO #9338]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/recipes-devtools/python/python3-profile-opt | 1 +
.../python/python3-profile-opt_3.5.2.bb | 12 +++++++++
...efile-add-install_generate_profile-target.patch | 25 ++++++++++++++++++
meta/recipes-devtools/python/python3_3.5.2.bb | 30 ++++++++++++----------
4 files changed, 54 insertions(+), 14 deletions(-)
create mode 120000 meta/recipes-devtools/python/python3-profile-opt
create mode 100644 meta/recipes-devtools/python/python3-profile-opt_3.5.2.bb
create mode 100644 meta/recipes-devtools/python/python3/Makefile-add-install_generate_profile-target.patch
diff --git a/meta/recipes-devtools/python/python3-profile-opt b/meta/recipes-devtools/python/python3-profile-opt
new file mode 120000
index 0000000000..b8a0adbbb9
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-profile-opt
@@ -0,0 +1 @@
+python3
\ No newline at end of file
diff --git a/meta/recipes-devtools/python/python3-profile-opt_3.5.2.bb b/meta/recipes-devtools/python/python3-profile-opt_3.5.2.bb
new file mode 100644
index 0000000000..bd8889cbed
--- /dev/null
+++ b/meta/recipes-devtools/python/python3-profile-opt_3.5.2.bb
@@ -0,0 +1,12 @@
+require python3_${PV}.bb
+
+SRC_URI += "file://Makefile-add-install_generate_profile-target.patch \
+ "
+
+PYTHON3_MAKE_TARGET = "build_all_generate_profile"
+
+# Prevent a clash with libpython3
+EXCLUDE_FROM_SHLIBS = "1"
+RDEPENDS_${PN}-core += "lib${BPN}"
+DEBIAN_NOAUTONAME_lib${BPN} = "1"
+DEBIAN_NOAUTONAME_lib${BPN}-staticdev = "1"
diff --git a/meta/recipes-devtools/python/python3/Makefile-add-install_generate_profile-target.patch b/meta/recipes-devtools/python/python3/Makefile-add-install_generate_profile-target.patch
new file mode 100644
index 0000000000..7aee7fb799
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/Makefile-add-install_generate_profile-target.patch
@@ -0,0 +1,25 @@
+Makefile: add install_generate_profile target
+
+Making it possible to install the binaries with profile generation
+enabled. We need this as linking is also performed at the install stage.
+
+Upstream-Status: Inappropriate [oe-specific]
+
+Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index 2b87614..826155b 100644
+--- a/Makefile.pre.in
++++ b/Makefile.pre.in
+@@ -1046,6 +1046,8 @@ QUICKTESTOPTS= $(TESTOPTS) -x test_subprocess test_io test_lib2to3 \
+ quicktest: all platform
+ $(TESTRUNNER) $(QUICKTESTOPTS)
+
++install_generate_profile:
++ $(MAKE) install LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)"
+
+ install: @FRAMEWORKINSTALLFIRST@ commoninstall bininstall maninstall @FRAMEWORKINSTALLLAST@
+ if test "x$(ENSUREPIP)" != "xno" ; then \
+--
+2.6.6
+
diff --git a/meta/recipes-devtools/python/python3_3.5.2.bb b/meta/recipes-devtools/python/python3_3.5.2.bb
index 7f2bc7186f..97ecbab5e3 100644
--- a/meta/recipes-devtools/python/python3_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3_3.5.2.bb
@@ -112,14 +112,10 @@ do_compile() {
# then call do_install twice we get Makefile.orig == Makefile.sysroot
install -m 0644 Makefile Makefile.sysroot
- oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
- HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \
- STAGING_LIBDIR=${STAGING_LIBDIR} \
- STAGING_BASELIBDIR=${STAGING_BASELIBDIR} \
- STAGING_INCDIR=${STAGING_INCDIR} \
- LIB=${baselib} \
- ARCH=${TARGET_ARCH} \
- OPT="${CFLAGS}" libpython3.so
+ if [ "${PYTHON3_MAKE_TARGET}" = "build_all_generate_profile" ]; then
+ # This is only used in PGO profiling by python3-profile-opt package
+ export EXTRA_CFLAGS="-fprofile-dir=./python3-pgo-profiles/"
+ fi
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \
@@ -128,7 +124,7 @@ do_compile() {
STAGING_BASELIBDIR=${STAGING_BASELIBDIR} \
LIB=${baselib} \
ARCH=${TARGET_ARCH} \
- OPT="${CFLAGS}"
+ OPT="${CFLAGS}" ${PYTHON3_MAKE_TARGET}
}
do_install() {
@@ -148,8 +144,14 @@ do_install() {
STAGING_BASELIBDIR=${STAGING_BASELIBDIR} \
LIB=${baselib} \
ARCH=${TARGET_ARCH} \
- DESTDIR=${D} LIBDIR=${libdir}
+ DESTDIR=${D} LIBDIR=${libdir} ${PYTHON3_MAKE_TARGET}
+ if [ "${PYTHON3_MAKE_TARGET}" = "build_all_generate_profile" ]; then
+ # Need special make install if pgo generation is enabled
+ _PYTHON3_MAKE_INSTALL_TARGET="install_generate_profile"
+ else
+ _PYTHON3_MAKE_INSTALL_TARGET="install"
+ fi
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
HOSTPYTHON=${STAGING_BINDIR_NATIVE}/python3-native/python3 \
STAGING_LIBDIR=${STAGING_LIBDIR} \
@@ -157,7 +159,7 @@ do_install() {
STAGING_BASELIBDIR=${STAGING_BASELIBDIR} \
LIB=${baselib} \
ARCH=${TARGET_ARCH} \
- DESTDIR=${D} LIBDIR=${libdir} install
+ DESTDIR=${D} LIBDIR=${libdir} ${_PYTHON3_MAKE_INSTALL_TARGET}
# avoid conflict with 2to3 from Python 2
rm -f ${D}/${bindir}/2to3
@@ -206,9 +208,9 @@ PACKAGES =+ "${PN}-pyvenv"
FILES_${PN}-pyvenv += "${bindir}/pyvenv-${PYTHON_MAJMIN} ${bindir}/pyvenv"
# package libpython3
-PACKAGES =+ "libpython3 libpython3-staticdev"
-FILES_libpython3 = "${libdir}/libpython*.so.*"
-FILES_libpython3-staticdev += "${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_BINABI}/libpython${PYTHON_BINABI}.a"
+PACKAGES =+ "lib${BPN} lib${BPN}-staticdev"
+FILES_lib${BPN} = "${libdir}/libpython*.so.*"
+FILES_lib${BPN}-staticdev += "${libdir}/python${PYTHON_MAJMIN}/config-${PYTHON_BINABI}/libpython*.a"
INSANE_SKIP_${PN}-dev += "dev-elf"
# catch all the rest (unsorted)
--
2.12.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v3 4/8] devtools/images: add python-pgo-image
2017-05-18 8:42 [PATCH v3 0/8] support profile-optimized build for Python Markus Lehtonen
` (2 preceding siblings ...)
2017-05-18 8:42 ` [PATCH v3 3/8] python3: add python3-profile-opt recipe Markus Lehtonen
@ 2017-05-18 8:42 ` Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 5/8] python3: support profile optimized build Markus Lehtonen
` (3 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Markus Lehtonen @ 2017-05-18 8:42 UTC (permalink / raw)
To: openembedded-core
This is a special image for profiling Python 3.x in order to utilize
profile-guided-optimization. Profile data can be obtained by by running
bitbake python-pgo-image -c profile. Profile data will be copied into a
directory pointed by PYTHON3_PROFILE_DIR on the host system. The profile
task may be specified with PYTHON3_PROFILE_TASK.
The default profile task is pybench, although an example for using
test.regrtest is included in the recipe.
[YOCTO #9338]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/recipes-devtools/images/python-pgo-image.bb | 85 ++++++++++++++++++++++++
1 file changed, 85 insertions(+)
create mode 100644 meta/recipes-devtools/images/python-pgo-image.bb
diff --git a/meta/recipes-devtools/images/python-pgo-image.bb b/meta/recipes-devtools/images/python-pgo-image.bb
new file mode 100644
index 0000000000..76733a8c61
--- /dev/null
+++ b/meta/recipes-devtools/images/python-pgo-image.bb
@@ -0,0 +1,85 @@
+SUMMARY = "Minimal image for doing Python profiling (for PGO)"
+
+IMAGE_FEATURES += "ssh-server-dropbear"
+IMAGE_INSTALL = "packagegroup-core-boot"
+IMAGE_INSTALL += "python3-profile-opt python3-profile-opt-tests python3-profile-opt-tools"
+
+LICENSE = "MIT"
+
+inherit core-image
+
+PROFILE_DATA_WORKDIR = "${WORKDIR}/profile-data"
+
+PYTHON3_PROFILE_DIR ?= "${TMPDIR}/work-shared/${MACHINE}/python3/pgo-data"
+PYTHON3_PROFILE_TASK_DEFAULT = "-m test.regrtest --pgo -w -x test_asyncore test_gdb test_multiprocessing_fork test_multiprocessing_forkserver test_multiprocessing_main_handling test_multiprocessing_spawn test_subprocess"
+# Exclude tests that are failing on qemux86
+PYTHON3_PROFILE_TASK_DEFAULT += "test_builtin test_cmath test_concurrent_futures test_difflib test_distutils test_float test_format test_math test_optparse test_shutil test_statistics test_types test_unicode"
+# Exclude tests that are taking very long on qemux86
+PYTHON3_PROFILE_TASK_DEFAULT += "test_lib2to3 test_buffer test_pickle test_io test_threading test_asyncio test_urllib2_localnet test_itertools test_tuple test_trace test_tarfile test_unicodedata test_decimal test_long test_zipfile test_deque test_descr test_email test_venv test_bytes test_compileall test_ast test_multibytecodec"change python3 profile target to pybenchchange python3 profile target to pybench
+
+# Change default profile target to pybench. Running test.regrtest takes
+# ridiculously long, i.e. around 4 hours in qemux86 on i7-3770K desktop
+# machine. Pybench "only" takes around 55 minutes.
+PYTHON3_PROFILE_TASK_DEFAULT = "${docdir}/python3-profile-opt/Tools/pybench/pybench.py -n 2 --with-gc --with-syscheck"
+
+PYTHON3_PROFILE_TASK ?= "${PYTHON3_PROFILE_TASK_DEFAULT}"
+
+# We need these because we're utilizing the runtime test helpers from oeqa
+TEST_TARGET ?= "qemu"
+TEST_QEMUBOOT_TIMEOUT ?= "1000"
+TEST_LOG_DIR ?= "${WORKDIR}/qemulogs"
+FIND_ROOTFS = "1"
+
+def run_profile(d, profile_bin, profile_task, tgt_in_dir, host_out_dir):
+ from oeqa.targetcontrol import get_target_controller
+ from oe.utils import getstatusoutput
+
+ target = get_target_controller(d)
+ target.deploy()
+ try:
+ # Boot target
+ bootparams = None
+ if d.getVar('VIRTUAL-RUNTIME_init_manager', True) == 'systemd':
+ bootparams = 'systemd.log_level=debug systemd.log_target=console'
+ target.start(extra_bootparams=bootparams)
+
+ # Run profile task
+ ret, output = target.run(profile_bin + ' ' + profile_task, timeout=7200)
+ if ret:
+ bb.fatal("Failed to run profile task on target: %s" % output)
+ ret, output = target.run('tar czf pgo-data.tgz -C %s .' % tgt_in_dir)
+ if ret:
+ bb.fatal("Failed to archive profile data on target: %s" % output)
+
+ # Retrieve and unpack profile data
+ target.copy_from('/home/root/pgo-data.tgz', host_out_dir)
+
+ profile_tarball = os.path.join(host_out_dir, 'pgo-data.tgz')
+ ret, output = getstatusoutput('tar xf %s -C %s' % (profile_tarball, host_out_dir))
+ os.unlink(profile_tarball)
+ if ret:
+ bb.fatal("Failed to unpack python profile data: %s" % output)
+ finally:
+ target.stop()
+
+
+# Profile task for Python 3.x
+python do_profile() {
+ run_profile(d, 'python3',
+ d.getVar('PYTHON3_PROFILE_TASK', True),
+ '/home/root/python3-pgo-profiles/',
+ os.path.join(d.getVar("PROFILE_DATA_WORKDIR", True), 'python3'))
+}
+
+addtask profile after do_build
+do_profile[depends] += "qemu-native:do_populate_sysroot qemu-helper-native:do_populate_sysroot"
+do_profile[cleandirs] = "${PROFILE_DATA_WORKDIR}/python3"
+
+python do_profile_setscene () {
+ sstate_setscene(d)
+}
+
+SSTATETASKS += "do_profile"
+do_profile[sstate-inputdirs] = "${PROFILE_DATA_WORKDIR}/python3"
+do_profile[sstate-outputdirs] = "${PYTHON3_PROFILE_DIR}"
+addtask do_profile_setscene
--
2.12.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v3 5/8] python3: support profile optimized build
2017-05-18 8:42 [PATCH v3 0/8] support profile-optimized build for Python Markus Lehtonen
` (3 preceding siblings ...)
2017-05-18 8:42 ` [PATCH v3 4/8] devtools/images: add python-pgo-image Markus Lehtonen
@ 2017-05-18 8:42 ` Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 6/8] python3: fix profile-optimized build of modules Markus Lehtonen
` (2 subsequent siblings)
7 siblings, 0 replies; 9+ messages in thread
From: Markus Lehtonen @ 2017-05-18 8:42 UTC (permalink / raw)
To: openembedded-core
This patch makes it possible to build python3 with profile directed
optimization. That is, feed python build process with profile data to
guide optimization. This is the third (and the last) step in profile
directed optimization for Python 3.x.
In order to do a profile-optimized build you need to specify
PYTHON3_PROFILE_OPT = "1" in your local.conf, and, have profile data available
in the location pointed to by PYTHON3_PROFILE_DIR. Profile data can be
obtainen e.g. by running bitbake python-pgo-image -c profile3.
[YOCTO #9338]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/recipes-devtools/python/python3_3.5.2.bb | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/meta/recipes-devtools/python/python3_3.5.2.bb b/meta/recipes-devtools/python/python3_3.5.2.bb
index 97ecbab5e3..d40bc14d93 100644
--- a/meta/recipes-devtools/python/python3_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3_3.5.2.bb
@@ -76,6 +76,17 @@ export CROSSPYTHONPATH = "${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dy
# No ctypes option for python 3
PYTHONLSBOPTS = ""
+# Automatic profile guided optimization
+PYTHON3_MAKE_TARGET ?= "${@'build_all_use_profile' if d.getVar('PYTHON3_PROFILE_OPT', True) == '1' else ''}"
+PYTHON3_PROFILE_DIR ?= "${@'${TMPDIR}/work-shared/${MACHINE}/python3/pgo-data' if d.getVar('PYTHON3_PROFILE_OPT', True) == '1' else ''}"
+python () {
+ if (d.getVar('PYTHON3_PROFILE_OPT', True) == '1' and
+ d.getVar('PYTHON3_MAKE_TARGET', True) == 'build_all_use_profile'):
+ profile_dir = d.getVar('PYTHON3_PROFILE_DIR', True)
+ bb.utils.mkdirhier(profile_dir)
+ d.setVarFlag('do_compile', 'file-checksums', '%s:True' % profile_dir)
+}
+
do_configure_append() {
rm -f ${S}/Makefile.orig
autoreconf -Wcross --verbose --install --force --exclude=autopoint ../Python-${PV}/Modules/_ctypes/libffi
@@ -115,6 +126,10 @@ do_compile() {
if [ "${PYTHON3_MAKE_TARGET}" = "build_all_generate_profile" ]; then
# This is only used in PGO profiling by python3-profile-opt package
export EXTRA_CFLAGS="-fprofile-dir=./python3-pgo-profiles/"
+ elif [ -n "${PYTHON3_PROFILE_DIR}" ]; then
+ export EXTRA_CFLAGS="-fprofile-dir=${PYTHON3_PROFILE_DIR}"
+ # Remove non-optimized build artefacts
+ oe_runmake clean
fi
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
@@ -135,6 +150,11 @@ do_install() {
install -d ${D}${libdir}/pkgconfig
install -d ${D}${libdir}/python${PYTHON_MAJMIN}/config
+ # This only has effect if we build with -fprofile-use, e.g. when make
+ # target is build_all_use_profile
+ if [ -n "${PYTHON3_PROFILE_DIR}" ]; then
+ export EXTRA_CFLAGS="-fprofile-dir=${PYTHON3_PROFILE_DIR}"
+ fi
# rerun the build once again with original makefile this time
# run install in a separate step to avoid compile/install race
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
--
2.12.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v3 6/8] python3: fix profile-optimized build of modules
2017-05-18 8:42 [PATCH v3 0/8] support profile-optimized build for Python Markus Lehtonen
` (4 preceding siblings ...)
2017-05-18 8:42 ` [PATCH v3 5/8] python3: support profile optimized build Markus Lehtonen
@ 2017-05-18 8:42 ` Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 7/8] python3: add python3-tools subpackage Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 8/8] python3: remove two setup.py cross-compile hacks Markus Lehtonen
7 siblings, 0 replies; 9+ messages in thread
From: Markus Lehtonen @ 2017-05-18 8:42 UTC (permalink / raw)
To: openembedded-core
Without this the pgo-related compiler flags are not used in
cross-builds.
[YOCTO #9338]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
...-CFLAGS-for-extensions-when-cross-compili.patch | 56 ++++++++++++++++++++++
meta/recipes-devtools/python/python3_3.5.2.bb | 1 +
2 files changed, 57 insertions(+)
create mode 100644 meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch
diff --git a/meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch b/meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch
new file mode 100644
index 0000000000..bf239c6261
--- /dev/null
+++ b/meta/recipes-devtools/python/python3/Use-correct-CFLAGS-for-extensions-when-cross-compili.patch
@@ -0,0 +1,56 @@
+From 0fd8b986888ddf1995b503edf46ac827e83114d9 Mon Sep 17 00:00:00 2001
+From: Markus Lehtonen <markus.lehtonen@linux.intel.com>
+Date: Wed, 23 Nov 2016 16:08:04 +0200
+Subject: [PATCH] Use correct CFLAGS for extensions when cross-compiling
+
+Take PY_CFLAGS_NODIST into account, like in native build. This is needed
+in order to to profile-optimized build. Also, pass EXTRA_CFLAGS to
+profile-optimized build.
+
+Upstream-Status: Pending
+
+Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
+---
+ Makefile.pre.in | 4 ++--
+ setup.py | 3 ++-
+ 2 files changed, 4 insertions(+), 3 deletions(-)
+
+diff --git a/Makefile.pre.in b/Makefile.pre.in
+index bace00e..9635aa4 100644
+--- a/Makefile.pre.in
++++ b/Makefile.pre.in
+@@ -509,7 +509,7 @@ profile-opt:
+ $(MAKE) profile-removal
+
+ build_all_generate_profile:
+- $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)"
++ $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(EXTRA_CFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) $(PGO_PROF_GEN_FLAG) @LTOFLAGS@" LIBS="$(LIBS)"
+
+ run_profile_task:
+ : # FIXME: can't run for a cross build
+@@ -519,7 +519,7 @@ build_all_merge_profile:
+ $(LLVM_PROF_MERGER)
+
+ build_all_use_profile:
+- $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(PGO_PROF_USE_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) @LTOFLAGS@"
++ $(MAKE) all CFLAGS_NODIST="$(CFLAGS) $(EXTRA_CFLAGS) $(PGO_PROF_USE_FLAG) @LTOFLAGS@" LDFLAGS="$(LDFLAGS) @LTOFLAGS@"
+
+ # Compile and run with gcov
+ .PHONY=coverage coverage-lcov coverage-report
+diff --git a/setup.py b/setup.py
+index c1ce87e..72d37cf 100644
+--- a/setup.py
++++ b/setup.py
+@@ -271,7 +271,8 @@ class PyBuildExt(build_ext):
+ # compilers
+ if compiler is not None:
+ if cross_compiling:
+- (ccshared,cflags) = (os.environ.get('CCSHARED') or '', os.environ.get('CFLAGS') or '')
++ (ccshared,cflags) = (os.environ.get('CCSHARED') or '',
++ (os.environ.get('CFLAGS') or '') + ' ' + sysconfig.get_config_var('PY_CFLAGS_NODIST'))
+ else:
+ (ccshared,cflags) = sysconfig.get_config_vars('CCSHARED','CFLAGS')
+ args['compiler_so'] = compiler + ' ' + ccshared + ' ' + cflags
+--
+2.6.6
+
diff --git a/meta/recipes-devtools/python/python3_3.5.2.bb b/meta/recipes-devtools/python/python3_3.5.2.bb
index d40bc14d93..e3fe674cd7 100644
--- a/meta/recipes-devtools/python/python3_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3_3.5.2.bb
@@ -37,6 +37,7 @@ SRC_URI += "\
file://configure.ac-fix-LIBPL.patch \
file://python3-fix-CVE-2016-1000110.patch \
file://upstream-random-fixes.patch \
+ file://Use-correct-CFLAGS-for-extensions-when-cross-compili.patch \
"
SRC_URI[md5sum] = "8906efbacfcdc7c3c9198aeefafd159e"
SRC_URI[sha256sum] = "0010f56100b9b74259ebcd5d4b295a32324b58b517403a10d1a2aa7cb22bca40"
--
2.12.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v3 7/8] python3: add python3-tools subpackage
2017-05-18 8:42 [PATCH v3 0/8] support profile-optimized build for Python Markus Lehtonen
` (5 preceding siblings ...)
2017-05-18 8:42 ` [PATCH v3 6/8] python3: fix profile-optimized build of modules Markus Lehtonen
@ 2017-05-18 8:42 ` Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 8/8] python3: remove two setup.py cross-compile hacks Markus Lehtonen
7 siblings, 0 replies; 9+ messages in thread
From: Markus Lehtonen @ 2017-05-18 8:42 UTC (permalink / raw)
To: openembedded-core
Useful in developing Python, e.g. in benchmarking.
[YOCTO #9338]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/recipes-devtools/python/python-3.5-manifest.inc | 8 +++++---
meta/recipes-devtools/python/python3_3.5.2.bb | 5 +++++
2 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/meta/recipes-devtools/python/python-3.5-manifest.inc b/meta/recipes-devtools/python/python-3.5-manifest.inc
index 3f29b53a73..8e40aec9c9 100644
--- a/meta/recipes-devtools/python/python-3.5-manifest.inc
+++ b/meta/recipes-devtools/python/python-3.5-manifest.inc
@@ -4,9 +4,9 @@
-PROVIDES+="${PN}-2to3 ${PN}-argparse ${PN}-asyncio ${PN}-audio ${PN}-codecs ${PN}-compile ${PN}-compression ${PN}-core ${PN}-crypt ${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger ${PN}-dev ${PN}-difflib ${PN}-distutils ${PN}-distutils-staticdev ${PN}-doctest ${PN}-email ${PN}-enum ${PN}-fcntl ${PN}-gdbm ${PN}-html ${PN}-idle ${PN}-image ${PN}-importlib ${PN}-io ${PN}-json ${PN}-lang ${PN}-logging ${PN}-mailbox ${PN}-math ${PN}-mime ${PN}-mmap ${PN}-multiprocessing ${PN}-netclient ${PN}-netserver ${PN}-numbers ${PN}-pickle ${PN}-pkgutil ${PN}-pprint ${PN}-profile ${PN}-pydoc ${PN}-re ${PN}-readline ${PN}-reprlib ${PN}-resource ${PN}-selectors ${PN}-shell ${PN}-signal ${PN}-smtpd ${PN}-sqlite3 ${PN}-sqlite3-tests ${PN}-stringold ${PN}-subprocess ${PN}-syslog ${PN}-terminal ${PN}-tests ${PN}-textutils ${PN}-threading ${PN}-tkinter ${PN}-typing ${PN}-unittest ${PN}-unixadmin ${PN}-xml ${PN}-xmlrpc "
+PROVIDES+="${PN}-2to3 ${PN}-argparse ${PN}-asyncio ${PN}-audio ${PN}-codecs ${PN}-compile ${PN}-compression ${PN}-core ${PN}-crypt ${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger ${PN}-dev ${PN}-difflib ${PN}-distutils ${PN}-distutils-staticdev ${PN}-doctest ${PN}-email ${PN}-enum ${PN}-fcntl ${PN}-gdbm ${PN}-html ${PN}-idle ${PN}-image ${PN}-importlib ${PN}-io ${PN}-json ${PN}-lang ${PN}-logging ${PN}-mailbox ${PN}-math ${PN}-mime ${PN}-mmap ${PN}-multiprocessing ${PN}-netclient ${PN}-netserver ${PN}-numbers ${PN}-pickle ${PN}-pkgutil ${PN}-pprint ${PN}-profile ${PN}-pydoc ${PN}-re ${PN}-readline ${PN}-reprlib ${PN}-resource ${PN}-selectors ${PN}-shell ${PN}-signal ${PN}-smtpd ${PN}-sqlite3 ${PN}-sqlite3-tests ${PN}-stringold ${PN}-subprocess ${PN}-syslog ${PN}-terminal ${PN}-tests ${PN}-textutils ${PN}-threading ${PN}-tkinter ${PN}-typing ${PN}-unittest ${PN}-unixadmin ${PN}-xml ${PN}-xmlrpc ${PN}-tools "
-PACKAGES="${PN}-dbg ${PN}-2to3 ${PN}-argparse ${PN}-asyncio ${PN}-audio ${PN}-codecs ${PN}-compile ${PN}-compression ${PN}-core ${PN}-crypt ${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger ${PN}-dev ${PN}-difflib ${PN}-distutils-staticdev ${PN}-distutils ${PN}-doctest ${PN}-email ${PN}-enum ${PN}-fcntl ${PN}-gdbm ${PN}-html ${PN}-idle ${PN}-image ${PN}-importlib ${PN}-io ${PN}-json ${PN}-lang ${PN}-logging ${PN}-mailbox ${PN}-math ${PN}-mime ${PN}-mmap ${PN}-multiprocessing ${PN}-netclient ${PN}-netserver ${PN}-numbers ${PN}-pickle ${PN}-pkgutil ${PN}-pprint ${PN}-profile ${PN}-pydoc ${PN}-re ${PN}-readline ${PN}-reprlib ${PN}-resource ${PN}-selectors ${PN}-shell ${PN}-signal ${PN}-smtpd ${PN}-sqlite3 ${PN}-sqlite3-tests ${PN}-stringold ${PN}-subprocess ${PN}-syslog ${PN}-terminal ${PN}-tests ${PN}-textutils ${PN}-threading ${PN}-tkinter ${PN}-typing ${PN}-unittest ${PN}-unixadmin ${PN}-xml ${PN}-xmlrpc ${PN}-modules"
+PACKAGES="${PN}-dbg ${PN}-2to3 ${PN}-argparse ${PN}-asyncio ${PN}-audio ${PN}-codecs ${PN}-compile ${PN}-compression ${PN}-core ${PN}-crypt ${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger ${PN}-dev ${PN}-difflib ${PN}-distutils-staticdev ${PN}-distutils ${PN}-doctest ${PN}-email ${PN}-enum ${PN}-fcntl ${PN}-gdbm ${PN}-html ${PN}-idle ${PN}-image ${PN}-importlib ${PN}-io ${PN}-json ${PN}-lang ${PN}-logging ${PN}-mailbox ${PN}-math ${PN}-mime ${PN}-mmap ${PN}-multiprocessing ${PN}-netclient ${PN}-netserver ${PN}-numbers ${PN}-pickle ${PN}-pkgutil ${PN}-pprint ${PN}-profile ${PN}-pydoc ${PN}-re ${PN}-readline ${PN}-reprlib ${PN}-resource ${PN}-selectors ${PN}-shell ${PN}-signal ${PN}-smtpd ${PN}-sqlite3 ${PN}-sqlite3-tests ${PN}-stringold ${PN}-subprocess ${PN}-syslog ${PN}-terminal ${PN}-tests ${PN}-textutils ${PN}-threading ${PN}-tkinter ${PN}-typing ${PN}-unittest ${PN}-unixadmin ${PN}-xml ${PN}-xmlrpc ${PN}-modules ${PN}-tools"
SUMMARY_${PN}-2to3="Python automated Python 2 to 3 code translator"
RDEPENDS_${PN}-2to3="${PN}-core"
@@ -280,4 +280,6 @@ SUMMARY_${PN}-modules="All Python modules"
RDEPENDS_${PN}-modules="${PN}-2to3 ${PN}-argparse ${PN}-asyncio ${PN}-audio ${PN}-codecs ${PN}-compile ${PN}-compression ${PN}-core ${PN}-crypt ${PN}-ctypes ${PN}-curses ${PN}-datetime ${PN}-db ${PN}-debugger ${PN}-difflib ${PN}-distutils ${PN}-doctest ${PN}-email ${PN}-enum ${PN}-fcntl ${PN}-gdbm ${PN}-html ${PN}-idle ${PN}-image ${PN}-importlib ${PN}-io ${PN}-json ${PN}-lang ${PN}-logging ${PN}-mailbox ${PN}-math ${PN}-mime ${PN}-mmap ${PN}-multiprocessing ${PN}-netclient ${PN}-netserver ${PN}-numbers ${PN}-pickle ${PN}-pkgutil ${PN}-pprint ${PN}-profile ${PN}-pydoc ${PN}-re ${PN}-readline ${PN}-reprlib ${PN}-resource ${PN}-selectors ${PN}-shell ${PN}-signal ${PN}-smtpd ${PN}-sqlite3 ${PN}-sqlite3-tests ${PN}-stringold ${PN}-subprocess ${PN}-syslog ${PN}-terminal ${PN}-tests ${PN}-textutils ${PN}-threading ${PN}-tkinter ${PN}-typing ${PN}-unittest ${PN}-unixadmin ${PN}-xml ${PN}-xmlrpc "
ALLOW_EMPTY_${PN}-modules = "1"
-
+SUMMARY_${PN}-tools="Useful scripts from Python"
+RDEPENDS_${PN}-tools="${PN}-modules"
+FILES_${PN}-tools="${docdir}/${PN}/Tools"
diff --git a/meta/recipes-devtools/python/python3_3.5.2.bb b/meta/recipes-devtools/python/python3_3.5.2.bb
index e3fe674cd7..757909a893 100644
--- a/meta/recipes-devtools/python/python3_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3_3.5.2.bb
@@ -193,6 +193,11 @@ do_install() {
fi
oe_multilib_header python${PYTHON_BINABI}/pyconfig.h
+
+ # Install tools
+ install -d -m 755 ${D}/${docdir}/${PN}
+ cp -r ${S}/Tools ${D}/${docdir}/${PN}/
+ find ${D}/${docdir}/${PN}/ -name Makefile -delete
}
do_install_append_class-nativesdk () {
--
2.12.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v3 8/8] python3: remove two setup.py cross-compile hacks
2017-05-18 8:42 [PATCH v3 0/8] support profile-optimized build for Python Markus Lehtonen
` (6 preceding siblings ...)
2017-05-18 8:42 ` [PATCH v3 7/8] python3: add python3-tools subpackage Markus Lehtonen
@ 2017-05-18 8:42 ` Markus Lehtonen
7 siblings, 0 replies; 9+ messages in thread
From: Markus Lehtonen @ 2017-05-18 8:42 UTC (permalink / raw)
To: openembedded-core
Remove two unneeded hacks. The first hack ("setup.py: no host headers
libs" patch) is not needed because we use cross-compiler (e.g.
i586-oe-linux-gcc) which has not been configured with any host system
include or library directories, and thus, we don't get any host system
directories when running "gcc -E -v".
The second hack becomes useless after the first hack has been removed
and we get the standard include and lib directories normally from gcc.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
.../python/python3-profile-opt_3.5.2.bb | 3 ++
| 33 ----------------------
.../setup.py-find-libraries-in-staging-dirs.patch | 30 --------------------
meta/recipes-devtools/python/python3_3.5.2.bb | 2 --
4 files changed, 3 insertions(+), 65 deletions(-)
delete mode 100644 meta/recipes-devtools/python/python3/python3-setup.py-no-host-headers-libs.patch
delete mode 100644 meta/recipes-devtools/python/python3/setup.py-find-libraries-in-staging-dirs.patch
diff --git a/meta/recipes-devtools/python/python3-profile-opt_3.5.2.bb b/meta/recipes-devtools/python/python3-profile-opt_3.5.2.bb
index bd8889cbed..d752cd8fd7 100644
--- a/meta/recipes-devtools/python/python3-profile-opt_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3-profile-opt_3.5.2.bb
@@ -5,6 +5,9 @@ SRC_URI += "file://Makefile-add-install_generate_profile-target.patch \
PYTHON3_MAKE_TARGET = "build_all_generate_profile"
+RCONFLICTS_${PN}-core = "python3-core"
+RCONFLICTS_lib${BPN} = "libpython3"
+
# Prevent a clash with libpython3
EXCLUDE_FROM_SHLIBS = "1"
RDEPENDS_${PN}-core += "lib${BPN}"
diff --git a/meta/recipes-devtools/python/python3/python3-setup.py-no-host-headers-libs.patch b/meta/recipes-devtools/python/python3/python3-setup.py-no-host-headers-libs.patch
deleted file mode 100644
index 2bc8b8c460..0000000000
--- a/meta/recipes-devtools/python/python3/python3-setup.py-no-host-headers-libs.patch
+++ /dev/null
@@ -1,33 +0,0 @@
-From 43238e1ac13e32984d015c92a5841f3de1fe1d15 Mon Sep 17 00:00:00 2001
-From: Jackie Huang <jackie.huang@windriver.com>
-Date: Tue, 18 Nov 2014 00:07:07 -0500
-Subject: [PATCH] setup.py: no host headers libs
-
-When we are cross-compiling, setup.py should never look in /usr
-or /usr/local to find headers or libraries.
-
-Upstream-Status: Inappropriate [Cross compile specific]
-
-Signed-off-by: Jackie Huang <jackie.huang@windriver.com>
----
- setup.py | 3 ---
- 1 file changed, 3 deletions(-)
-
-diff --git a/setup.py b/setup.py
-index f020b28..e8339cd 100644
---- a/setup.py
-+++ b/setup.py
-@@ -444,10 +444,7 @@ class PyBuildExt(build_ext):
- if not cross_compiling:
- add_dir_to_list(self.compiler.library_dirs, os.path.join('/usr/local', sys.lib))
- add_dir_to_list(self.compiler.include_dirs, '/usr/local/include')
-- # only change this for cross builds for 3.3, issues on Mageia
-- if cross_compiling:
- self.add_gcc_paths()
-- if not cross_compiling:
- self.add_multiarch_paths()
-
- # Add paths specified in the environment variables LDFLAGS and
---
-2.0.0
-
diff --git a/meta/recipes-devtools/python/python3/setup.py-find-libraries-in-staging-dirs.patch b/meta/recipes-devtools/python/python3/setup.py-find-libraries-in-staging-dirs.patch
deleted file mode 100644
index f26bd38b02..0000000000
--- a/meta/recipes-devtools/python/python3/setup.py-find-libraries-in-staging-dirs.patch
+++ /dev/null
@@ -1,30 +0,0 @@
-Patch setup.py so that the detect_modules() function looks for required
-libraries and headers in STAGING_LIBDIR / STAGING_INCDIR.
-
-Without this patch, several extension modules are not built, even though
-their dependencies are present in the compiler's search paths.
-The result is the following warning, and ultimately incomplete packages:
-
-| The necessary bits to build these optional modules were not found:
-| _bz2 _curses_panel _dbm
-| _gdbm _lzma _sqlite3
-| nis readline zlib
-| To find the necessary bits, look in setup.py in detect_modules() for the module's name.
-
-Upstream-Status: Inappropriate [Cross compile specific]
-
-Signed-off-by: Dominic Sacré <dominic.sacre@gmx.de>
-
-Index: Python-3.4.3/setup.py
-===================================================================
---- Python-3.4.3.orig/setup.py
-+++ Python-3.4.3/setup.py
-@@ -521,6 +521,8 @@ class PyBuildExt(build_ext):
- else:
- lib_dirs = self.compiler.library_dirs[:]
- inc_dirs = self.compiler.include_dirs[:]
-+ lib_dirs.append(os.environ.get('STAGING_LIBDIR'))
-+ inc_dirs.append(os.environ.get('STAGING_INCDIR'))
- exts = []
- missing = []
-
diff --git a/meta/recipes-devtools/python/python3_3.5.2.bb b/meta/recipes-devtools/python/python3_3.5.2.bb
index 757909a893..9512a4d4a9 100644
--- a/meta/recipes-devtools/python/python3_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3_3.5.2.bb
@@ -30,10 +30,8 @@ SRC_URI += "\
file://unixccompiler.patch \
file://avoid-ncursesw-include-path.patch \
file://python3-use-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch \
- file://python3-setup.py-no-host-headers-libs.patch \
file://sysconfig.py-add-_PYTHON_PROJECT_SRC.patch \
file://setup.py-check-cross_compiling-when-get-FLAGS.patch \
- file://setup.py-find-libraries-in-staging-dirs.patch \
file://configure.ac-fix-LIBPL.patch \
file://python3-fix-CVE-2016-1000110.patch \
file://upstream-random-fixes.patch \
--
2.12.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
end of thread, other threads:[~2017-05-18 8:43 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-05-18 8:42 [PATCH v3 0/8] support profile-optimized build for Python Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 1/8] python3-native: support profile optimized build Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 2/8] python3: fix depends of python3-tests Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 3/8] python3: add python3-profile-opt recipe Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 4/8] devtools/images: add python-pgo-image Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 5/8] python3: support profile optimized build Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 6/8] python3: fix profile-optimized build of modules Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 7/8] python3: add python3-tools subpackage Markus Lehtonen
2017-05-18 8:42 ` [PATCH v3 8/8] python3: remove two setup.py cross-compile hacks Markus Lehtonen
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.