All of lore.kernel.org
 help / color / mirror / Atom feed
From: Markus Lehtonen <markus.lehtonen@linux.intel.com>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH v2 10/20] python-pgo-image: enable sstate for do_profile
Date: Tue, 16 May 2017 16:18:40 +0300	[thread overview]
Message-ID: <889d6ff9ec2b0cc9eaa5d9200ec9e7c0809356bc.1494939394.git.markus.lehtonen@linux.intel.com> (raw)
In-Reply-To: <cover.1494939394.git.markus.lehtonen@linux.intel.com>
In-Reply-To: <cover.1494939394.git.markus.lehtonen@linux.intel.com>

There are caveats however, basically because of the circular dependency
between python-pgo-image and python (python is a build dep of
python-pgo-image, and at the same time, depends on the profile data
generated by python-pgo-image). In practice the sstate data is only used
when no profile data is yet present. Existing profile data (i.e.
non-empty PYTHON_PROFILE_DIR) changes signature hash of python which, in
turn, changes signature of python-pgo-image and causes a sstate
signature mismatch and rebuild of python-pgo-image. Thus, in order to
utilize sstate data the user should run "bitbake python-pgo-image -c clean"
(or otherwise wipe pre-existing profile data) before running
"bitbake python-pgo-image -c profile".

[YOCTO #9338]

Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com>
---
 meta/recipes-devtools/images/python-pgo-image.bb | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-devtools/images/python-pgo-image.bb b/meta/recipes-devtools/images/python-pgo-image.bb
index 33aa9a1fec..05252a1a46 100644
--- a/meta/recipes-devtools/images/python-pgo-image.bb
+++ b/meta/recipes-devtools/images/python-pgo-image.bb
@@ -9,6 +9,7 @@ LICENSE = "MIT"
 inherit core-image
 
 PYTHON_PROFILE_DIR ?= "${TMPDIR}/work-shared/${MACHINE}/python/pgo-data"
+PROFILE_DATA_WORKDIR = "${WORKDIR}/profile-data"
 #PYTHON_PROFILE_TASK_DEFAULT = "-m test.regrtest --pgo -w -x test_asyncore test_gdb test_multiprocessing test_subprocess"
 ## Exclude tests that are segfaulting on qemux86 target
 #PYTHON_PROFILE_TASK_DEFAULT += "test_bytes test_str test_string test_tuple test_unicode test_userstring test_xmlrpc"
@@ -48,7 +49,7 @@ python do_profile() {
             bb.fatal("Failed to archive profile data on target: %s" % output)
 
         # Retrieve and unpack profile data
-        profile_dir = d.getVar("PYTHON_PROFILE_DIR", True)
+        profile_dir = d.getVar("PROFILE_DATA_WORKDIR", True)
         target.copy_from('/home/root/pgo-data.tgz', profile_dir)
 
         profile_tarball = os.path.join(profile_dir, 'pgo-data.tgz')
@@ -68,4 +69,14 @@ python do_profile() {
 
 addtask profile after do_build
 do_profile[depends] += "qemu-native:do_populate_sysroot qemu-helper-native:do_populate_sysroot"
-do_profile[cleandirs] = "${PYTHON_PROFILE_DIR}"
+do_profile[cleandirs] = "${PROFILE_DATA_WORKDIR}"
+
+
+python do_profile_setscene () {
+    sstate_setscene(d)
+}
+
+SSTATETASKS += "do_profile"
+do_profile[sstate-inputdirs] = "${PROFILE_DATA_WORKDIR}"
+do_profile[sstate-outputdirs] = "${PYTHON_PROFILE_DIR}"
+addtask do_profile_setscene
-- 
2.12.0



  parent reply	other threads:[~2017-05-16 13:19 UTC|newest]

Thread overview: 26+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-05-16 13:18 [PATCH v2 00/20] support profile-optimized build for Python Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 01/20] python-native: support profile optimized build Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 02/20] python: add python-profile-opt recipe Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 03/20] python: remove path hack from setup.py Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 04/20] python-profile-opt: rename libpython Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 05/20] devtools/images: add python-pgo-image Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 06/20] python: make profile-optimized build possible Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 07/20] python-pgo-image: exclude tests from the default profile target Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 08/20] python: add python-tools subpackage Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 09/20] python-pgo-image: switch python default profile task to pybench Markus Lehtonen
2017-05-16 13:18 ` Markus Lehtonen [this message]
2017-05-16 13:18 ` [PATCH v2 11/20] python3-native: support profile optimized build Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 12/20] python3: fix depends of python-tests Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 13/20] python3: add python-profile-opt3 recipe Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 14/20] python-pgo-image: profiling for python3 Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 15/20] python3: remove two setup.py cross-compile hacks Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 16/20] python3: support profile optimized build Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 17/20] python3: fix profile-optimized build of modules Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 18/20] python-pgo-image: exclude tests from the python3 profile target Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 19/20] python3: add python3-tools subpackage Markus Lehtonen
2017-05-16 13:18 ` [PATCH v2 20/20] python-pgo-image: change python3 profile target to pybench Markus Lehtonen
2017-05-16 13:23 ` [PATCH v2 00/20] support profile-optimized build for Python Alexander Kanavin
2017-05-16 14:07   ` Markus Lehtonen
2017-05-16 14:08     ` Alexander Kanavin
2017-05-16 13:45 ` Richard Purdie
2017-05-16 14:16   ` Markus Lehtonen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=889d6ff9ec2b0cc9eaa5d9200ec9e7c0809356bc.1494939394.git.markus.lehtonen@linux.intel.com \
    --to=markus.lehtonen@linux.intel.com \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.