[OE-core][dunfell 0/8] Patch review

[OE-core][dunfell 0/8] Patch review

[Steve Sakoman]

Please review this next set of patches for dunfell and have comments back
by end of day Monday.

Passed a-full on autobuilder:

https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/1478

The following changes since commit 656d2070f8448681cb69a3d43dbae84a681c1a75:

  linux-yocto/5.4: update to v5.4.69 (2020-10-07 13:41:09 -1000)

are available in the Git repository at:

  git://git.openembedded.org/openembedded-core-contrib stable/dunfell-nut
  http://cgit.openembedded.org/openembedded-core-contrib/log/?h=stable/dunfell-nut

Armin Kuster (1):
  timezone: update to 2020b

Naoki Hayama (1):
  uninative: Fix typo in error message

Richard Purdie (2):
  scripts/oe-build-perf-report: Allow operation with no buildstats
  oe-build-perf-report: Ensure correct data is shown for multiple branch
    options

Steve Sakoman (1):
  Revert "package: get_package_mapping: avoid dependency mapping if
    renamed package provides original name"

Victor Kamensky (2):
  qemu: add 34Kf-64tlb fictitious cpu type
  qemumips: use 34Kf-64tlb CPU emulation

Yoann Congal (1):
  bitbake-bblayers/create: Make the example recipe print its message

 meta/classes/package.bbclass                  |  16 +--
 meta/classes/uninative.bbclass                |   2 +-
 meta/conf/machine/qemumips.conf               |   2 +-
 meta/lib/bblayers/templates/example.bb        |   4 +-
 meta/recipes-devtools/qemu/qemu.inc           |   1 +
 ...tlb-fictitious-cpu-type-like-34Kf-bu.patch | 118 ++++++++++++++++++
 meta/recipes-extended/timezone/timezone.inc   |   8 +-
 meta/recipes-extended/timezone/tzdata.bb      |   8 +-
 scripts/oe-build-perf-report                  |   6 +-
 9 files changed, 137 insertions(+), 28 deletions(-)
 create mode 100644 meta/recipes-devtools/qemu/qemu/0001-mips-add-34Kf-64tlb-fictitious-cpu-type-like-34Kf-bu.patch

-- 
2.17.1

[OE-core][dunfell 1/8] Revert "package: get_package_mapping: avoid dependency mapping if renamed package provides original name"

[Steve Sakoman]

Breaks wireguard

This reverts commit 210ebed1e9c2285d6e457bf03d1f1a1f3ddc7fda.

Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/classes/package.bbclass | 16 +++-------------
 1 file changed, 3 insertions(+), 13 deletions(-)

diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
index cc64ddffc3..23ff4772fa 100644
--- a/meta/classes/package.bbclass
+++ b/meta/classes/package.bbclass
@@ -535,7 +535,7 @@ def copydebugsources(debugsrcdir, sources, d):
 # Package data handling routines
 #
 
-def get_package_mapping (pkg, basepkg, d, depversions=None):
+def get_package_mapping (pkg, basepkg, d):
     import oe.packagedata
 
     data = oe.packagedata.read_subpkgdata(pkg, d)
@@ -546,14 +546,6 @@ def get_package_mapping (pkg, basepkg, d, depversions=None):
         if bb.data.inherits_class('allarch', d) and not d.getVar('MULTILIB_VARIANTS') \
             and data[key] == basepkg:
             return pkg
-        if depversions == []:
-            # Avoid returning a mapping if the renamed package rprovides its original name
-            rprovkey = "RPROVIDES_%s" % pkg
-            if rprovkey in data:
-                if pkg in bb.utils.explode_dep_versions2(data[rprovkey]):
-                    bb.note("%s rprovides %s, not replacing the latter" % (data[key], pkg))
-                    return pkg
-        # Do map to rewritten package name
         return data[key]
 
     return pkg
@@ -574,10 +566,8 @@ def runtime_mapping_rename (varname, pkg, d):
 
     new_depends = {}
     deps = bb.utils.explode_dep_versions2(d.getVar(varname) or "")
-    for depend, depversions in deps.items():
-        new_depend = get_package_mapping(depend, pkg, d, depversions)
-        if depend != new_depend:
-            bb.note("package name mapping done: %s -> %s" % (depend, new_depend))
+    for depend in deps:
+        new_depend = get_package_mapping(depend, pkg, d)
         new_depends[new_depend] = deps[depend]
 
     d.setVar(varname, bb.utils.join_deps(new_depends, commasep=False))
-- 
2.17.1

[OE-core][dunfell 2/8] timezone: update to 2020b

[Steve Sakoman]

From: Armin Kuster <akuster@mvista.com>

For more info see: https://github.com/eggert/tz/blob/master/NEWS

Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b19bc7ae71a59673bd725e1de3251667c2026ed5)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/recipes-extended/timezone/timezone.inc | 8 +++-----
 meta/recipes-extended/timezone/tzdata.bb    | 8 ++++----
 2 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/meta/recipes-extended/timezone/timezone.inc b/meta/recipes-extended/timezone/timezone.inc
index 32723f3dec..8eb17c5eaf 100644
--- a/meta/recipes-extended/timezone/timezone.inc
+++ b/meta/recipes-extended/timezone/timezone.inc
@@ -6,7 +6,7 @@ SECTION = "base"
 LICENSE = "PD & BSD & BSD-3-Clause"
 LIC_FILES_CHKSUM = "file://LICENSE;md5=c679c9d6b02bc2757b3eaf8f53c43fba"
 
-PV = "2020a"
+PV = "2020b"
 
 SRC_URI =" http://www.iana.org/time-zones/repository/releases/tzcode${PV}.tar.gz;name=tzcode \
            http://www.iana.org/time-zones/repository/releases/tzdata${PV}.tar.gz;name=tzdata \
@@ -14,7 +14,5 @@ SRC_URI =" http://www.iana.org/time-zones/repository/releases/tzcode${PV}.tar.gz
 
 UPSTREAM_CHECK_URI = "http://www.iana.org/time-zones"
 
-SRC_URI[tzcode.md5sum] = "f87c3477e85a5c4b00df0def6c6a0055"
-SRC_URI[tzcode.sha256sum] = "7d2af7120ee03df71fbca24031ccaf42404752e639196fe93c79a41b38a6d669"
-SRC_URI[tzdata.md5sum] = "96a985bb8eeab535fb8aa2132296763a"
-SRC_URI[tzdata.sha256sum] = "547161eca24d344e0b5f96aff6a76b454da295dc14ed4ca50c2355043fb899a2"
+SRC_URI[tzcode.sha256sum] = "47eff8944de4a64f7629b851e4a32338ab12c9b73edd62063795167ff1fe43da"
+SRC_URI[tzdata.sha256sum] = "9b053f951d245ce89d850b96ee4711d82d833559b1fc96ba19f90bc4d745e809"
diff --git a/meta/recipes-extended/timezone/tzdata.bb b/meta/recipes-extended/timezone/tzdata.bb
index 1e2b440fb0..e6a0655afe 100644
--- a/meta/recipes-extended/timezone/tzdata.bb
+++ b/meta/recipes-extended/timezone/tzdata.bb
@@ -15,18 +15,18 @@ DEFAULT_TIMEZONE ?= "Universal"
 INSTALL_TIMEZONE_FILE ?= "1"
 
 TZONES= "africa antarctica asia australasia europe northamerica southamerica  \
-         factory etcetera backward systemv \
+         factory etcetera backward \
         "
 # pacificnew 
 
 do_compile () {
         for zone in ${TZONES}; do \
             ${STAGING_BINDIR_NATIVE}/zic -d ${WORKDIR}${datadir}/zoneinfo -L /dev/null \
-                -y ${S}/yearistype.sh ${S}/${zone} ; \
+                ${S}/${zone} ; \
             ${STAGING_BINDIR_NATIVE}/zic -d ${WORKDIR}${datadir}/zoneinfo/posix -L /dev/null \
-                -y ${S}/yearistype.sh ${S}/${zone} ; \
+                ${S}/${zone} ; \
             ${STAGING_BINDIR_NATIVE}/zic -d ${WORKDIR}${datadir}/zoneinfo/right -L ${S}/leapseconds \
-                -y ${S}/yearistype.sh ${S}/${zone} ; \
+                ${S}/${zone} ; \
         done
 }
 
-- 
2.17.1

[OE-core][dunfell 3/8] scripts/oe-build-perf-report: Allow operation with no buildstats

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

If buildstats is missing, avoid a backtrace.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 23c4b21f947c7aae1303ca6526fc5aaaa9fc7bb8)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 scripts/oe-build-perf-report | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index 7ed86a72f6..b46d61869c 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -353,7 +353,7 @@ def print_html_report(data, id_comp, buildstats):
             bs_key = test + '.' + meas
             rev = str(metadata['commit_num']['value'])
             comp_rev = str(metadata['commit_num']['value_old'])
-            if (rev in buildstats and bs_key in buildstats[rev] and
+            if (buildstats and rev in buildstats and bs_key in buildstats[rev] and
                     comp_rev in buildstats and bs_key in buildstats[comp_rev]):
                 new_meas['buildstats'] = BSSummary(buildstats[comp_rev][bs_key],
                                                    buildstats[rev][bs_key])
-- 
2.17.1

[OE-core][dunfell 4/8] oe-build-perf-report: Ensure correct data is shown for multiple branch options

[Steve Sakoman]

From: Richard Purdie <richard.purdie@linuxfoundation.org>

If --branch and --branch2 are the same, there is extra confusing data plotted
onto the graph. Only do that if the branches are actually different.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c263d810b7eb47ee90f2adeb5ab6decf7332bec2)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 scripts/oe-build-perf-report | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/oe-build-perf-report b/scripts/oe-build-perf-report
index b46d61869c..7812ea4540 100755
--- a/scripts/oe-build-perf-report
+++ b/scripts/oe-build-perf-report
@@ -509,7 +509,7 @@ def main(argv=None):
 
     revs = gitarchive.get_test_revs(log, repo, args.tag_name, hostname=args.hostname,
                                     branch=args.branch, machine=args.machine)
-    if args.branch2:
+    if args.branch2 and args.branch2 != args.branch:
         revs2 = gitarchive.get_test_revs(log, repo, args.tag_name, hostname=args.hostname,
                                          branch=args.branch2, machine=args.machine)
         if not len(revs2):
@@ -533,7 +533,7 @@ def main(argv=None):
     else:
         index1 = len(revs) - 1
 
-    if args.branch2:
+    if args.branch2 and args.branch2 != args.branch:
         revs2.append(revs[index1])
         index1 = len(revs2) - 1
         revs = revs2
-- 
2.17.1

[OE-core][dunfell 5/8] bitbake-bblayers/create: Make the example recipe print its message

[Steve Sakoman]

From: Yoann Congal <yoann.congal@smile.fr>

The example recipe is setup to print a message using bb.plain() in the
"do_build" task but this task is "noexec" so the message never prints.
This might be confusing.

This moves the message printing into another "do_display_banner" task
and add it to the do_build "before" list.

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 252385bef9b226f32691b8513869ea3e41813b40)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/lib/bblayers/templates/example.bb | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/lib/bblayers/templates/example.bb b/meta/lib/bblayers/templates/example.bb
index c4b873d593..facaae35d2 100644
--- a/meta/lib/bblayers/templates/example.bb
+++ b/meta/lib/bblayers/templates/example.bb
@@ -2,10 +2,12 @@ SUMMARY = "bitbake-layers recipe"
 DESCRIPTION = "Recipe created by bitbake-layers"
 LICENSE = "MIT"
 
-python do_build() {
+python do_display_banner() {
     bb.plain("***********************************************");
     bb.plain("*                                             *");
     bb.plain("*  Example recipe created by bitbake-layers   *");
     bb.plain("*                                             *");
     bb.plain("***********************************************");
 }
+
+addtask display_banner before do_build
-- 
2.17.1

[OE-core][dunfell 6/8] uninative: Fix typo in error message

[Steve Sakoman]

From: Naoki Hayama <naoki.hayama@lineo.co.jp>

Fix typo in an error message.
s/verson/version/

Signed-off-by: Naoki Hayama <naoki.hayama@lineo.co.jp>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit bc96db2e0b5b8a9cc2c909ea70df290e03a50b94)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/classes/uninative.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/uninative.bbclass b/meta/classes/uninative.bbclass
index 316c0f0616..1e19917a97 100644
--- a/meta/classes/uninative.bbclass
+++ b/meta/classes/uninative.bbclass
@@ -89,7 +89,7 @@ python uninative_event_fetchloader() {
         # ldd output is "ldd (Ubuntu GLIBC 2.23-0ubuntu10) 2.23", extract last option from first line
         glibcver = subprocess.check_output(["ldd", "--version"]).decode('utf-8').split('\n')[0].split()[-1]
         if bb.utils.vercmp_string(d.getVar("UNINATIVE_MAXGLIBCVERSION"), glibcver) < 0:
-            raise RuntimeError("Your host glibc verson (%s) is newer than that in uninative (%s). Disabling uninative so that sstate is not corrupted." % (glibcver, d.getVar("UNINATIVE_MAXGLIBCVERSION")))
+            raise RuntimeError("Your host glibc version (%s) is newer than that in uninative (%s). Disabling uninative so that sstate is not corrupted." % (glibcver, d.getVar("UNINATIVE_MAXGLIBCVERSION")))
 
         cmd = d.expand("\
 mkdir -p ${UNINATIVE_STAGING_DIR}-uninative; \
-- 
2.17.1

[OE-core][dunfell 7/8] qemu: add 34Kf-64tlb fictitious cpu type

[Steve Sakoman]

From: Victor Kamensky <kamensky@cisco.com>

In Yocto Project PR 13992 it was reported that qemumips
in autobuilder runs almost twice slower then qemumips64 and
some times hit time out.

Upon investigations of qemu-system with perf, gdb, and
SystemTap and comparing qemumips and qemumips64 machines
behavior it was noticed that qemu soft mmu code behaves
quite different and in case if qemumips tlbwr instruction
called 16 times more oftern. It happens that in qemumips64
case qemu runs with cpu type that contains 64 TLB, but in case
of qemumips qemu runs with cpu type that contains only
16 TLBs.

The idea of proposed qemu patch is to introduce fictitious
34Kf-64tlb cpu type that defined exactly as 34Kf but has
64 TLBs, instead of original 16 TLBs.

Testing of core-image-full-cmdline:do_testimage with
34Kf-64tlb shows 40% or so test execution real time
improvement.

Note for future porters of the patch: easiest way to update
the patch and be in sync with 34Kf definition is to copy
34Kf machine definition and apply the following changes to
it (just change 15 to 63 of CP0C1_MMU bits value)

[kamensky@coreos-lnx2 qemu]$ diff ~/34Kf.c ~/34Kf-64tlb.c
2c2
<         .name = "34Kf",
>         .name = "34Kf-64tlb",
6c6
<         .CP0_Config1 = MIPS_CONFIG1 | (1 << CP0C1_FP) | (15 << CP0C1_MMU) |
>         .CP0_Config1 = MIPS_CONFIG1 | (1 << CP0C1_FP) | (63 << CP0C1_MMU) |

Fixes https://bugzilla.yoctoproject.org/show_bug.cgi?id=13992

Upstream Status: Inappropriate

Signed-off-by: Victor Kamensky <kamensky@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4470a04943352224955f17e004962f0f9e1c9b0c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/recipes-devtools/qemu/qemu.inc           |   1 +
 ...tlb-fictitious-cpu-type-like-34Kf-bu.patch | 118 ++++++++++++++++++
 2 files changed, 119 insertions(+)
 create mode 100644 meta/recipes-devtools/qemu/qemu/0001-mips-add-34Kf-64tlb-fictitious-cpu-type-like-34Kf-bu.patch

diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc
index 7ce89c0023..7c21b66a0c 100644
--- a/meta/recipes-devtools/qemu/qemu.inc
+++ b/meta/recipes-devtools/qemu/qemu.inc
@@ -48,6 +48,7 @@ SRC_URI = "https://download.qemu.org/${BPN}-${PV}.tar.xz \
 	   file://CVE-2020-14364.patch \
 	   file://CVE-2020-14415.patch \
 	   file://CVE-2020-16092.patch \
+	   file://0001-mips-add-34Kf-64tlb-fictitious-cpu-type-like-34Kf-bu.patch \
 	   "
 UPSTREAM_CHECK_REGEX = "qemu-(?P<pver>\d+(\.\d+)+)\.tar"
 
diff --git a/meta/recipes-devtools/qemu/qemu/0001-mips-add-34Kf-64tlb-fictitious-cpu-type-like-34Kf-bu.patch b/meta/recipes-devtools/qemu/qemu/0001-mips-add-34Kf-64tlb-fictitious-cpu-type-like-34Kf-bu.patch
new file mode 100644
index 0000000000..b6312e1543
--- /dev/null
+++ b/meta/recipes-devtools/qemu/qemu/0001-mips-add-34Kf-64tlb-fictitious-cpu-type-like-34Kf-bu.patch
@@ -0,0 +1,118 @@
+From b3fcc7d96523ad8e3ea28c09d495ef08529d01ce Mon Sep 17 00:00:00 2001
+From: Victor Kamensky <kamensky@cisco.com>
+Date: Wed, 7 Oct 2020 10:19:42 -0700
+Subject: [PATCH] mips: add 34Kf-64tlb fictitious cpu type like 34Kf but with
+ 64 TLBs
+
+In Yocto Project CI runs it was observed that test run
+of 32 bit mips image takes almost twice longer than 64 bit
+mips image with the same logical load and CI execution
+hits timeout.
+
+See https://bugzilla.yoctoproject.org/show_bug.cgi?id=13992
+
+Yocto project uses 34Kf cpu type to run 32 bit mips image,
+and MIPS64R2-generic cpu type to run 64 bit mips64 image.
+
+Upon qemu behavior differences investigation between mips
+and mips64 two prominent observations came up: under
+logically similar load (same definition and configuration
+of user-land image) in case of mips get_physical_address
+function is called almost twice more often, meaning
+twice more memory accesses involved in this case. Also
+number of tlbwr instruction executed (r4k_helper_tlbwr
+qemu function) almost 16 time bigger in mips case than in
+mips64.
+
+It turns out that 34Kf cpu has 16 TLBs, but in case of
+MIPS64R2-generic it is 64 TLBs. So that explains why
+some many more tlbwr had to be execute by kernel TLB refill
+handler in case of 32 bit misp.
+
+The idea of the fix is to come up with new 34Kf-64tlb fictitious
+cpu type, that would behave exactly as 34Kf but it would
+contain 64 TLBs to reduce TLB trashing. After all, adding
+more TLBs to soft mmu is easy.
+
+Experiment with some significant non-trvial load in Yocto
+environment by running do_testimage load shows that 34Kf-64tlb
+cpu performs 40% or so better than original 34Kf cpu wrt test
+execution real time.
+
+It is not ideal to have cpu type that does not exist in the
+wild but given performance gains it seems to be justified.
+
+Signed-off-by: Victor Kamensky <kamensky@cisco.com>
+---
+ target/mips/translate_init.inc.c | 55 ++++++++++++++++++++++++++++++++++++++++
+ 1 file changed, 55 insertions(+)
+
+diff --git a/target/mips/translate_init.inc.c b/target/mips/translate_init.inc.c
+index 637caccd89..b73ab48231 100644
+--- a/target/mips/translate_init.inc.c
++++ b/target/mips/translate_init.inc.c
+@@ -297,6 +297,61 @@ const mips_def_t mips_defs[] =
+         .insn_flags = CPU_MIPS32R2 | ASE_MIPS16 | ASE_DSP | ASE_MT,
+         .mmu_type = MMU_TYPE_R4000,
+     },
++    /*
++     * Verbatim copy of "34Kf" cpu, only bumped up number of TLB entries
++     * from 16 to 64 (see CP0_Config0 value at CP0C1_MMU bits) to improve
++     * performance by reducing number of TLB refill exceptions and
++     * eliminating need to run all corresponding TLB refill handling
++     * instructions.
++     */
++    {
++        .name = "34Kf-64tlb",
++        .CP0_PRid = 0x00019500,
++        .CP0_Config0 = MIPS_CONFIG0 | (0x1 << CP0C0_AR) |
++                       (MMU_TYPE_R4000 << CP0C0_MT),
++        .CP0_Config1 = MIPS_CONFIG1 | (1 << CP0C1_FP) | (63 << CP0C1_MMU) |
++                       (0 << CP0C1_IS) | (3 << CP0C1_IL) | (1 << CP0C1_IA) |
++                       (0 << CP0C1_DS) | (3 << CP0C1_DL) | (1 << CP0C1_DA) |
++                       (1 << CP0C1_CA),
++        .CP0_Config2 = MIPS_CONFIG2,
++        .CP0_Config3 = MIPS_CONFIG3 | (1 << CP0C3_VInt) | (1 << CP0C3_MT) |
++                       (1 << CP0C3_DSPP),
++        .CP0_LLAddr_rw_bitmask = 0,
++        .CP0_LLAddr_shift = 0,
++        .SYNCI_Step = 32,
++        .CCRes = 2,
++        .CP0_Status_rw_bitmask = 0x3778FF1F,
++        .CP0_TCStatus_rw_bitmask = (0 << CP0TCSt_TCU3) | (0 << CP0TCSt_TCU2) |
++                    (1 << CP0TCSt_TCU1) | (1 << CP0TCSt_TCU0) |
++                    (0 << CP0TCSt_TMX) | (1 << CP0TCSt_DT) |
++                    (1 << CP0TCSt_DA) | (1 << CP0TCSt_A) |
++                    (0x3 << CP0TCSt_TKSU) | (1 << CP0TCSt_IXMT) |
++                    (0xff << CP0TCSt_TASID),
++        .CP1_fcr0 = (1 << FCR0_F64) | (1 << FCR0_L) | (1 << FCR0_W) |
++                    (1 << FCR0_D) | (1 << FCR0_S) | (0x95 << FCR0_PRID),
++        .CP1_fcr31 = 0,
++        .CP1_fcr31_rw_bitmask = 0xFF83FFFF,
++        .CP0_SRSCtl = (0xf << CP0SRSCtl_HSS),
++        .CP0_SRSConf0_rw_bitmask = 0x3fffffff,
++        .CP0_SRSConf0 = (1U << CP0SRSC0_M) | (0x3fe << CP0SRSC0_SRS3) |
++                    (0x3fe << CP0SRSC0_SRS2) | (0x3fe << CP0SRSC0_SRS1),
++        .CP0_SRSConf1_rw_bitmask = 0x3fffffff,
++        .CP0_SRSConf1 = (1U << CP0SRSC1_M) | (0x3fe << CP0SRSC1_SRS6) |
++                    (0x3fe << CP0SRSC1_SRS5) | (0x3fe << CP0SRSC1_SRS4),
++        .CP0_SRSConf2_rw_bitmask = 0x3fffffff,
++        .CP0_SRSConf2 = (1U << CP0SRSC2_M) | (0x3fe << CP0SRSC2_SRS9) |
++                    (0x3fe << CP0SRSC2_SRS8) | (0x3fe << CP0SRSC2_SRS7),
++        .CP0_SRSConf3_rw_bitmask = 0x3fffffff,
++        .CP0_SRSConf3 = (1U << CP0SRSC3_M) | (0x3fe << CP0SRSC3_SRS12) |
++                    (0x3fe << CP0SRSC3_SRS11) | (0x3fe << CP0SRSC3_SRS10),
++        .CP0_SRSConf4_rw_bitmask = 0x3fffffff,
++        .CP0_SRSConf4 = (0x3fe << CP0SRSC4_SRS15) |
++                    (0x3fe << CP0SRSC4_SRS14) | (0x3fe << CP0SRSC4_SRS13),
++        .SEGBITS = 32,
++        .PABITS = 32,
++        .insn_flags = CPU_MIPS32R2 | ASE_MIPS16 | ASE_DSP | ASE_MT,
++        .mmu_type = MMU_TYPE_R4000,
++    },
+     {
+         .name = "74Kf",
+         .CP0_PRid = 0x00019700,
+-- 
+2.14.5
+
-- 
2.17.1

[OE-core][dunfell 8/8] qemumips: use 34Kf-64tlb CPU emulation

[Steve Sakoman]

From: Victor Kamensky <kamensky@cisco.com>

In order to improve performance of qemumips autobuilder
test runs, let's use 34Kf-64tlb cpu type that was introduced
in OE version of qemu. 34Kf-64tlb cpu type is identical to
34Kf but it has 64 TLBs configure vs original 16 TLBs.

The change in number of TLBs in emulated CPU reduces
TLB trash and reduces number of times TLB refill kernel
code runs, and therefore siginificantly improves tests
execution time.

Note 34Kf-64tlb qemu cpu type does not exist in upstream,
so far it is added as patch in OE. See qemu 34Kf-64tlb
cpu addition commit for more details.

Fixes https://bugzilla.yoctoproject.org/show_bug.cgi?id=13992

Signed-off-by: Victor Kamensky <kamensky@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6a9568401cbf8bed89d1f63dae61a07a83a20b50)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
---
 meta/conf/machine/qemumips.conf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/conf/machine/qemumips.conf b/meta/conf/machine/qemumips.conf
index 1373e4cba0..b8c80f02ef 100644
--- a/meta/conf/machine/qemumips.conf
+++ b/meta/conf/machine/qemumips.conf
@@ -15,4 +15,4 @@ SERIAL_CONSOLES ?= "115200;ttyS0 115200;ttyS1"
 
 QB_SYSTEM_NAME = "qemu-system-mips"
 
-QB_CPU = "-cpu 34Kf"
+QB_CPU = "-cpu 34Kf-64tlb"
-- 
2.17.1

Re: [OE-core][dunfell 1/8] Revert "package: get_package_mapping: avoid dependency mapping if renamed package provides original name"

[Steve Sakoman]

I've included this revert patch for comment because I have gotten
several requests to revert and I'd like wider discussion of the issue.

At the moment wireguard is the only known recipe that is failing and
there are no proposed fixes.

If you are aware of recipes in other layers that are affected, please
let me know.

And if someone would like to tackle fixing wireguard in
meta-openembedded that would be much preferred to a revert.

Thanks!

Steve

On Thu, Oct 15, 2020 at 4:15 AM Steve Sakoman via
lists.openembedded.org <steve=sakoman.com@lists.openembedded.org>
wrote:
>
> Breaks wireguard
>
> This reverts commit 210ebed1e9c2285d6e457bf03d1f1a1f3ddc7fda.
>
> Signed-off-by: Steve Sakoman <steve@sakoman.com>
> ---
>  meta/classes/package.bbclass | 16 +++-------------
>  1 file changed, 3 insertions(+), 13 deletions(-)
>
> diff --git a/meta/classes/package.bbclass b/meta/classes/package.bbclass
> index cc64ddffc3..23ff4772fa 100644
> --- a/meta/classes/package.bbclass
> +++ b/meta/classes/package.bbclass
> @@ -535,7 +535,7 @@ def copydebugsources(debugsrcdir, sources, d):
>  # Package data handling routines
>  #
>
> -def get_package_mapping (pkg, basepkg, d, depversions=None):
> +def get_package_mapping (pkg, basepkg, d):
>      import oe.packagedata
>
>      data = oe.packagedata.read_subpkgdata(pkg, d)
> @@ -546,14 +546,6 @@ def get_package_mapping (pkg, basepkg, d, depversions=None):
>          if bb.data.inherits_class('allarch', d) and not d.getVar('MULTILIB_VARIANTS') \
>              and data[key] == basepkg:
>              return pkg
> -        if depversions == []:
> -            # Avoid returning a mapping if the renamed package rprovides its original name
> -            rprovkey = "RPROVIDES_%s" % pkg
> -            if rprovkey in data:
> -                if pkg in bb.utils.explode_dep_versions2(data[rprovkey]):
> -                    bb.note("%s rprovides %s, not replacing the latter" % (data[key], pkg))
> -                    return pkg
> -        # Do map to rewritten package name
>          return data[key]
>
>      return pkg
> @@ -574,10 +566,8 @@ def runtime_mapping_rename (varname, pkg, d):
>
>      new_depends = {}
>      deps = bb.utils.explode_dep_versions2(d.getVar(varname) or "")
> -    for depend, depversions in deps.items():
> -        new_depend = get_package_mapping(depend, pkg, d, depversions)
> -        if depend != new_depend:
> -            bb.note("package name mapping done: %s -> %s" % (depend, new_depend))
> +    for depend in deps:
> +        new_depend = get_package_mapping(depend, pkg, d)
>          new_depends[new_depend] = deps[depend]
>
>      d.setVar(varname, bb.utils.join_deps(new_depends, commasep=False))
> --
> 2.17.1
>
>
> 
>

Re: [OE-core][dunfell 1/8] Revert "package: get_package_mapping: avoid dependency mapping if renamed package provides original name"

[Richard Purdie]

On Fri, 2020-10-16 at 04:34 -1000, Steve Sakoman wrote:
> I've included this revert patch for comment because I have gotten
> several requests to revert and I'd like wider discussion of the
> issue.
> 
> At the moment wireguard is the only known recipe that is failing and
> there are no proposed fixes.
> 
> If you are aware of recipes in other layers that are affected, please
> let me know.
> 
> And if someone would like to tackle fixing wireguard in
> meta-openembedded that would be much preferred to a revert.

It would also be good to know if the issue exists on master or not. I'm
worried we may have the issue there as well which is a reason I'd
favour looking into and understanding this rather than blindly
reverting. I've not seen anyone have a full understanding of what the
issue is as yet.

Cheers,

Richard

Re: [OE-core][dunfell 7/8] qemu: add 34Kf-64tlb fictitious cpu type

[Richard Purdie]

On Thu, 2020-10-15 at 04:15 -1000, Steve Sakoman wrote:
> From: Victor Kamensky <kamensky@cisco.com>
> 
> In Yocto Project PR 13992 it was reported that qemumips
> in autobuilder runs almost twice slower then qemumips64 and
> some times hit time out.
> 
> Upon investigations of qemu-system with perf, gdb, and
> SystemTap and comparing qemumips and qemumips64 machines
> behavior it was noticed that qemu soft mmu code behaves
> quite different and in case if qemumips tlbwr instruction
> called 16 times more oftern. It happens that in qemumips64
> case qemu runs with cpu type that contains 64 TLB, but in case
> of qemumips qemu runs with cpu type that contains only
> 16 TLBs.
> 
> The idea of proposed qemu patch is to introduce fictitious
> 34Kf-64tlb cpu type that defined exactly as 34Kf but has
> 64 TLBs, instead of original 16 TLBs.
> 
> Testing of core-image-full-cmdline:do_testimage with
> 34Kf-64tlb shows 40% or so test execution real time
> improvement.
> 
> Note for future porters of the patch: easiest way to update
> the patch and be in sync with 34Kf definition is to copy
> 34Kf machine definition and apply the following changes to
> it (just change 15 to 63 of CP0C1_MMU bits value)

There is discussion upstream about just changing the tlb to 64 for this
machine. I'd suggest we hold off this for dunfell until its resolved
there.

Cheers,

Richard

Re: [OE-core][dunfell 7/8] qemu: add 34Kf-64tlb fictitious cpu type

[Steve Sakoman]

On Sun, Oct 18, 2020 at 4:27 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2020-10-15 at 04:15 -1000, Steve Sakoman wrote:
> > From: Victor Kamensky <kamensky@cisco.com>
> >
> > In Yocto Project PR 13992 it was reported that qemumips
> > in autobuilder runs almost twice slower then qemumips64 and
> > some times hit time out.
> >
> > Upon investigations of qemu-system with perf, gdb, and
> > SystemTap and comparing qemumips and qemumips64 machines
> > behavior it was noticed that qemu soft mmu code behaves
> > quite different and in case if qemumips tlbwr instruction
> > called 16 times more oftern. It happens that in qemumips64
> > case qemu runs with cpu type that contains 64 TLB, but in case
> > of qemumips qemu runs with cpu type that contains only
> > 16 TLBs.
> >
> > The idea of proposed qemu patch is to introduce fictitious
> > 34Kf-64tlb cpu type that defined exactly as 34Kf but has
> > 64 TLBs, instead of original 16 TLBs.
> >
> > Testing of core-image-full-cmdline:do_testimage with
> > 34Kf-64tlb shows 40% or so test execution real time
> > improvement.
> >
> > Note for future porters of the patch: easiest way to update
> > the patch and be in sync with 34Kf definition is to copy
> > 34Kf machine definition and apply the following changes to
> > it (just change 15 to 63 of CP0C1_MMU bits value)
>
> There is discussion upstream about just changing the tlb to 64 for this
> machine. I'd suggest we hold off this for dunfell until its resolved
> there.

OK, I'll remove these two patches from the pull request.

Steve

Re: [OE-core][dunfell 1/8] Revert "package: get_package_mapping: avoid dependency mapping if renamed package provides original name"

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1413 bytes --]

Looks like it's caused by wireguard setting PKG variable as:
PKG_${PN} = "kernel-module-${MODULE_NAME}-${KERNEL_VERSION}"

as discussed in:
https://github.com/openembedded/meta-openembedded/issues/285#issuecomment-714401682

it was added in
https://git.openembedded.org/meta-openembedded/commit/?id=ad0db129c7c8484520cd20cdfbb4670489125280
but the explanation there doesn't make much sense to me.

On Sat, Oct 17, 2020 at 9:55 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Fri, 2020-10-16 at 04:34 -1000, Steve Sakoman wrote:
> > I've included this revert patch for comment because I have gotten
> > several requests to revert and I'd like wider discussion of the
> > issue.
> >
> > At the moment wireguard is the only known recipe that is failing and
> > there are no proposed fixes.
> >
> > If you are aware of recipes in other layers that are affected, please
> > let me know.
> >
> > And if someone would like to tackle fixing wireguard in
> > meta-openembedded that would be much preferred to a revert.
>
> It would also be good to know if the issue exists on master or not. I'm
> worried we may have the issue there as well which is a reason I'd
> favour looking into and understanding this rather than blindly
> reverting. I've not seen anyone have a full understanding of what the
> issue is as yet.
>
> Cheers,
>
> Richard
>
>
>
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 2712 bytes --]