From: Markus Lehtonen <markus.lehtonen@linux.intel.com>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH v2 16/20] python3: support profile optimized build
Date: Tue, 16 May 2017 16:18:46 +0300 [thread overview]
Message-ID: <a5fbb6d5fd39abfd52627f05e73c2c58352eb17f.1494939394.git.markus.lehtonen@linux.intel.com> (raw)
In-Reply-To: <cover.1494939394.git.markus.lehtonen@linux.intel.com>
In-Reply-To: <cover.1494939394.git.markus.lehtonen@linux.intel.com>
Make it possible to do profile-optimized build. The implementation is
virtually identical to how we do it in Python 2.7. Basically, you need
to define PYTHON3_PROFILE_OPT = "1" in your local.conf and have profile
data available in the location pointed to by PYTHON3_PROFILE_DIR.
Profile data can be obtainen e.g. by running bitbake python-pgo-image -c
profile3.
[YOCTO #9338]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
meta/recipes-devtools/python/python3_3.5.2.bb | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/meta/recipes-devtools/python/python3_3.5.2.bb b/meta/recipes-devtools/python/python3_3.5.2.bb
index 18d683e4c7..7a3d2b367f 100644
--- a/meta/recipes-devtools/python/python3_3.5.2.bb
+++ b/meta/recipes-devtools/python/python3_3.5.2.bb
@@ -78,6 +78,17 @@ export CROSSPYTHONPATH = "${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dy
# No ctypes option for python 3
PYTHONLSBOPTS = ""
+# Automatic profile guided optimization
+PYTHON3_MAKE_TARGET ?= "${@'build_all_use_profile' if d.getVar('PYTHON3_PROFILE_OPT', True) == '1' else ''}"
+PYTHON3_PROFILE_DIR ?= "${@'${TMPDIR}/work-shared/${MACHINE}/python3/pgo-data' if d.getVar('PYTHON3_PROFILE_OPT', True) == '1' else ''}"
+python () {
+ if (d.getVar('PYTHON3_PROFILE_OPT', True) == '1' and
+ d.getVar('PYTHON3_MAKE_TARGET', True) == 'build_all_use_profile'):
+ profile_dir = d.getVar('PYTHON3_PROFILE_DIR', True)
+ bb.utils.mkdirhier(profile_dir)
+ d.setVarFlag('do_compile', 'file-checksums', '%s:True' % profile_dir)
+}
+
do_configure_append() {
rm -f ${S}/Makefile.orig
autoreconf -Wcross --verbose --install --force --exclude=autopoint ../Python-${PV}/Modules/_ctypes/libffi
@@ -117,6 +128,10 @@ do_compile() {
if [ "${PYTHON3_MAKE_TARGET}" = "build_all_generate_profile" ]; then
# This is only used in PGO profiling by python-profile-opt package
export EXTRA_CFLAGS="-fprofile-dir=./python3-pgo-profiles/"
+ elif [ -n "${PYTHON3_PROFILE_DIR}" ]; then
+ export EXTRA_CFLAGS="-fprofile-dir=${PYTHON3_PROFILE_DIR}"
+ # Remove non-optimized build artefacts
+ oe_runmake clean
fi
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
@@ -137,6 +152,11 @@ do_install() {
install -d ${D}${libdir}/pkgconfig
install -d ${D}${libdir}/python${PYTHON_MAJMIN}/config
+ # This only has effect if we build with -fprofile-use, e.g. when make
+ # target is build_all_use_profile
+ if [ -n "${PYTHON3_PROFILE_DIR}" ]; then
+ export EXTRA_CFLAGS="-fprofile-dir=${PYTHON3_PROFILE_DIR}"
+ fi
# rerun the build once again with original makefile this time
# run install in a separate step to avoid compile/install race
oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \
--
2.12.0
next prev parent reply other threads:[~2017-05-16 13:19 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-16 13:18 [PATCH v2 00/20] support profile-optimized build for Python Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 01/20] python-native: support profile optimized build Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 02/20] python: add python-profile-opt recipe Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 03/20] python: remove path hack from setup.py Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 04/20] python-profile-opt: rename libpython Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 05/20] devtools/images: add python-pgo-image Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 06/20] python: make profile-optimized build possible Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 07/20] python-pgo-image: exclude tests from the default profile target Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 08/20] python: add python-tools subpackage Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 09/20] python-pgo-image: switch python default profile task to pybench Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 10/20] python-pgo-image: enable sstate for do_profile Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 11/20] python3-native: support profile optimized build Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 12/20] python3: fix depends of python-tests Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 13/20] python3: add python-profile-opt3 recipe Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 14/20] python-pgo-image: profiling for python3 Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 15/20] python3: remove two setup.py cross-compile hacks Markus Lehtonen
2017-05-16 13:18 ` Markus Lehtonen [this message]
2017-05-16 13:18 ` [PATCH v2 17/20] python3: fix profile-optimized build of modules Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 18/20] python-pgo-image: exclude tests from the python3 profile target Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 19/20] python3: add python3-tools subpackage Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 20/20] python-pgo-image: change python3 profile target to pybench Markus Lehtonen
2017-05-16 13:23 ` [PATCH v2 00/20] support profile-optimized build for Python Alexander Kanavin
2017-05-16 14:07 ` Markus Lehtonen
2017-05-16 14:08 ` Alexander Kanavin
2017-05-16 13:45 ` Richard Purdie
2017-05-16 14:16 ` Markus Lehtonen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a5fbb6d5fd39abfd52627f05e73c2c58352eb17f.1494939394.git.markus.lehtonen@linux.intel.com \
--to=markus.lehtonen@linux.intel.com \
--cc=openembedded-core@lists.openembedded.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.