All of lore.kernel.org
 help / color / mirror / Atom feed
From: Ross Burton <ross.burton@intel.com>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH 5/5] python3: use regrtest instead of PyBench for profile-guided-optimisation
Date: Thu,  6 Sep 2018 12:15:21 +0100	[thread overview]
Message-ID: <20180906111521.20997-5-ross.burton@intel.com> (raw)
In-Reply-To: <20180906111521.20997-1-ross.burton@intel.com>

PyBench takes a long time to run, also upstream have removed it from Python and
instead use test.regrtest —pgo to profile the interpreter.

The results are good: not only does Python compile faster (~300s vs ~600s on my
machine) but Phoronix’s PyBench test runs in 2130ms compared to 2229ms when
using PyBench to train (and 2345ms with PGO disabled).

Signed-off-by: Ross Burton <ross.burton@intel.com>
---
 meta/recipes-devtools/python/python3_3.5.5.bb | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-devtools/python/python3_3.5.5.bb b/meta/recipes-devtools/python/python3_3.5.5.bb
index b89e3b04c86..386b46a763d 100644
--- a/meta/recipes-devtools/python/python3_3.5.5.bb
+++ b/meta/recipes-devtools/python/python3_3.5.5.bb
@@ -76,7 +76,7 @@ SDK_CC_ARCH += "-DNDEBUG -fno-inline"
 EXTRA_OEMAKE += "CROSS_COMPILE=yes"
 EXTRA_OECONF += "CROSSPYTHONPATH=${STAGING_LIBDIR_NATIVE}/python${PYTHON_MAJMIN}/lib-dynload/ --without-ensurepip"
 
-PYTHON3_PROFILE_TASK ?= "${S}/Tools/pybench/pybench.py -n 1"
+PYTHON3_PROFILE_TASK ?= "./python -m test.regrtest --pgo test_grammar test_opcodes test_dict test_builtin test_exceptions test_types test_support || true"
 
 export CROSS_COMPILE = "${TARGET_PREFIX}"
 export _PYTHON_PROJECT_BASE = "${B}"
@@ -144,13 +144,15 @@ do_compile() {
 	if ${@bb.utils.contains('PACKAGECONFIG', 'pgo', 'true', 'false', d)}; then
 		run_make profile-opt
 		qemu_binary="${@qemu_wrapper_cmdline(d, '${STAGING_DIR_TARGET}', ['${B}', '${STAGING_DIR_TARGET}/${base_libdir}'])}"
-		cat > pgo-image-qemuwrapper << EOF
+		cat >pgo-wrapper <<EOF
 #!/bin/sh
-set -x
+cd ${B}
 $qemu_binary "\$@"
 EOF
-		chmod +x pgo-image-qemuwrapper
-		./pgo-image-qemuwrapper ${B}/python ${PYTHON3_PROFILE_TASK} || true
+		chmod +x pgo-wrapper
+		bbnote Gathering profiling data
+		./pgo-wrapper ${PYTHON3_PROFILE_TASK}
+		bbnote Profiling data gathered, rebuilding
 		run_make clean_and_use_profile
 	else
 		run_make libpython3.so
-- 
2.11.0



      parent reply	other threads:[~2018-09-06 11:15 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-09-06 11:15 [PATCH 1/5] python3: generalise make commands Ross Burton
2018-09-06 11:15 ` [PATCH 2/5] python3: add toggle for profile-guided-optimisation Ross Burton
2018-09-06 11:15 ` [PATCH 3/5] python3: trim PGO patch Ross Burton
2018-09-06 11:15 ` [PATCH 4/5] python3: don't use runtime checks to identify float endianism Ross Burton
2018-09-06 11:15 ` Ross Burton [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180906111521.20997-5-ross.burton@intel.com \
    --to=ross.burton@intel.com \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.