From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) by mail.openembedded.org (Postfix) with ESMTP id 845CF77E3E for ; Tue, 16 May 2017 13:19:13 +0000 (UTC) Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga105.jf.intel.com with ESMTP; 16 May 2017 06:19:15 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.38,349,1491289200"; d="scan'208";a="102711917" Received: from bkearns-mobl.ger.corp.intel.com (HELO mqz-osx-suse64.fi.intel.com) ([10.252.10.90]) by fmsmga006.fm.intel.com with ESMTP; 16 May 2017 06:19:14 -0700 From: Markus Lehtonen To: openembedded-core@lists.openembedded.org Date: Tue, 16 May 2017 16:18:46 +0300 Message-Id: X-Mailer: git-send-email 2.12.0 In-Reply-To: References: In-Reply-To: References: Subject: [PATCH v2 16/20] python3: support profile optimized build X-BeenThere: openembedded-core@lists.openembedded.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Patches and discussions about the oe-core layer List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 May 2017 13:19:13 -0000 Make it possible to do profile-optimized build. The implementation is virtually identical to how we do it in Python 2.7. Basically, you need to define PYTHON3_PROFILE_OPT = "1" in your local.conf and have profile data available in the location pointed to by PYTHON3_PROFILE_DIR. Profile data can be obtainen e.g. by running bitbake python-pgo-image -c profile3. [YOCTO #9338] Signed-off-by: Markus Lehtonen --- meta/recipes-devtools/python/python3_3.5.2.bb | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/meta/recipes-devtools/python/python3_3.5.2.bb b/meta/recipes-devtools/python/python3_3.5.2.bb index 18d683e4c7..7a3d2b367f 100644 --- a/meta/recipes-devtools/python/python3_3.5.2.bb +++ b/meta/recipes-devtools/python/python3_3.5.2.bb @@ -78,6 +78,17 @@ export CROSSPYTHONPATH = "${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dy # No ctypes option for python 3 PYTHONLSBOPTS = "" +# Automatic profile guided optimization +PYTHON3_MAKE_TARGET ?= "${@'build_all_use_profile' if d.getVar('PYTHON3_PROFILE_OPT', True) == '1' else ''}" +PYTHON3_PROFILE_DIR ?= "${@'${TMPDIR}/work-shared/${MACHINE}/python3/pgo-data' if d.getVar('PYTHON3_PROFILE_OPT', True) == '1' else ''}" +python () { + if (d.getVar('PYTHON3_PROFILE_OPT', True) == '1' and + d.getVar('PYTHON3_MAKE_TARGET', True) == 'build_all_use_profile'): + profile_dir = d.getVar('PYTHON3_PROFILE_DIR', True) + bb.utils.mkdirhier(profile_dir) + d.setVarFlag('do_compile', 'file-checksums', '%s:True' % profile_dir) +} + do_configure_append() { rm -f ${S}/Makefile.orig autoreconf -Wcross --verbose --install --force --exclude=autopoint ../Python-${PV}/Modules/_ctypes/libffi @@ -117,6 +128,10 @@ do_compile() { if [ "${PYTHON3_MAKE_TARGET}" = "build_all_generate_profile" ]; then # This is only used in PGO profiling by python-profile-opt package export EXTRA_CFLAGS="-fprofile-dir=./python3-pgo-profiles/" + elif [ -n "${PYTHON3_PROFILE_DIR}" ]; then + export EXTRA_CFLAGS="-fprofile-dir=${PYTHON3_PROFILE_DIR}" + # Remove non-optimized build artefacts + oe_runmake clean fi oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \ @@ -137,6 +152,11 @@ do_install() { install -d ${D}${libdir}/pkgconfig install -d ${D}${libdir}/python${PYTHON_MAJMIN}/config + # This only has effect if we build with -fprofile-use, e.g. when make + # target is build_all_use_profile + if [ -n "${PYTHON3_PROFILE_DIR}" ]; then + export EXTRA_CFLAGS="-fprofile-dir=${PYTHON3_PROFILE_DIR}" + fi # rerun the build once again with original makefile this time # run install in a separate step to avoid compile/install race oe_runmake HOSTPGEN=${STAGING_BINDIR_NATIVE}/python3-native/pgen \ -- 2.12.0