[PATCH 0/4] linux-yocto: build changes, 4.1 and libc-headers

[PATCH 0/4] linux-yocto: build changes, 4.1 and libc-headers

[Bruce Ashfield]

Hi all,

This series absolutely needs some more testing and runs through the
autobuilder, but I've done pretty much all I can do with it here.

I have more changes to the build process that will follow this, but 
these changes need to show that they are stable, so here's the first
blast.

With this series:

  - I create the 4.1, and 4.1-rt linux-yocto content. This will be
    the new LTSI kernel. Deletion of 3.14 and 3.19 will follow once
    all machines have been moved forward.

  - libc-headers updated to match the 4.1 kernel version.

And the one that could cause some issues/compatibility issues is the
split of the kernel configuration data from the kernel tree itself.

From the commit log:

  The linux-yocto tree has always been a combined set of kernel changes
  and configuration (meta) data carried in a single tree. While this
  format is effective at keeping kernel configuration and source
  modifications synchronized, it isn't always obvious to developers on
  how to manipulate the meta data versus the source.

  With this change, we remove the meta data processing from the
  kernel-yocto class and use the external meta-data repository that
  has always been used to seed the linux-yocto meta branch.

  After this change, linux-yocto can no longer process combined trees,
  and is simplified as a result.

I'm interested to find out if we get any breakages or severe compability
issues with the change. I don't want to support both variants of the
meta data (in and out of tree), since the goal is to reduce complexity
in the code. Once this change lands, I'll further streamline things.

I've built core-image-kernel-dev and boot tested the qemu* machines.
qemuppc continues to show a boot failure with systemd, but we have a
bugzilla to track that.

I'm now doing graphical testing, but wanted to get this out and soaking
for other images types while I wait for builds to churn.

Cheers,

Bruce

The following changes since commit 0f8d89ce66cfffccc9911d6c61470a585173b79d:

  core-image-lsb-sdk: use kernel-devsrc to ensure kernel module builds work (2015-07-20 10:40:42 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib zedd/kernel
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel

Bruce Ashfield (4):
  linux-yocto: split meta data from kernel repository
  kern-tools: standalone tree configuration
  linux-libc-headers: update to 4.1
  linux-yocto: introduce 4.1 versioned recipes

 meta/classes/kernel-yocto.bbclass                  | 76 ++--------------------
 meta/conf/distro/include/tcmode-default.inc        |  2 +-
 .../kern-tools/kern-tools-native_git.bb            |  2 +-
 .../linux-libc-headers/linux-libc-headers.inc      |  2 +
 .../linux-libc-headers/linux-libc-headers_3.19.bb  |  7 --
 .../linux-libc-headers/linux-libc-headers_4.1.bb   |  7 ++
 meta/recipes-kernel/linux/linux-yocto-dev.bb       |  5 +-
 meta/recipes-kernel/linux/linux-yocto-rt_3.14.bb   |  7 +-
 meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb    | 26 ++++++++
 meta/recipes-kernel/linux/linux-yocto-tiny_3.14.bb |  7 +-
 meta/recipes-kernel/linux/linux-yocto-tiny_3.19.bb |  7 +-
 meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb  | 23 +++++++
 meta/recipes-kernel/linux/linux-yocto_3.14.bb      |  7 +-
 meta/recipes-kernel/linux/linux-yocto_3.19.bb      |  7 +-
 meta/recipes-kernel/linux/linux-yocto_4.1.bb       | 41 ++++++++++++
 15 files changed, 129 insertions(+), 97 deletions(-)
 delete mode 100644 meta/recipes-kernel/linux-libc-headers/linux-libc-headers_3.19.bb
 create mode 100644 meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.1.bb
 create mode 100644 meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb
 create mode 100644 meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb
 create mode 100644 meta/recipes-kernel/linux/linux-yocto_4.1.bb

-- 
2.1.0

[PATCH 1/4] linux-yocto: split meta data from kernel repository

[Bruce Ashfield]

The linux-yocto tree has always been a combined set of kernel changes
and configuration (meta) data carried in a single tree. While this
format is effective at keeping kernel configuration and source
modifications synchronized, it isn't always obvious to developers on
how to manipulate the meta data versus the source.

With this change, we remove the meta data processing from the
kernel-yocto class and use the external meta-data repository that
has always been used to seed the linux-yocto meta branch.

After this change, linux-yocto can no longer process combined trees,
and is simplified as a result.

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/classes/kernel-yocto.bbclass                  | 76 ++--------------------
 meta/recipes-kernel/linux/linux-yocto-dev.bb       |  5 +-
 meta/recipes-kernel/linux/linux-yocto-rt_3.14.bb   |  7 +-
 meta/recipes-kernel/linux/linux-yocto-tiny_3.14.bb |  7 +-
 meta/recipes-kernel/linux/linux-yocto-tiny_3.19.bb |  7 +-
 meta/recipes-kernel/linux/linux-yocto_3.14.bb      |  7 +-
 meta/recipes-kernel/linux/linux-yocto_3.19.bb      |  7 +-
 7 files changed, 28 insertions(+), 88 deletions(-)

diff --git a/meta/classes/kernel-yocto.bbclass b/meta/classes/kernel-yocto.bbclass
index 2f9a41f34d0f..231e08deba68 100644
--- a/meta/classes/kernel-yocto.bbclass
+++ b/meta/classes/kernel-yocto.bbclass
@@ -33,6 +33,7 @@ def find_kernel_feature_dirs(d):
     for url in fetch.urls:
         urldata = fetch.ud[url]
         parm = urldata.parm
+        type=""
         if "type" in parm:
             type = parm["type"]
         if "destsuffix" in parm:
@@ -112,17 +113,6 @@ do_kernel_metadata() {
 		fi
 	fi
 
-	# if we have a defined/set meta branch we should not be generating
-	# any meta data. The passed branch has what we need.
-	if [ -n "${KMETA}" ]; then
-		createme_flags="--disable-meta-gen --meta ${KMETA}"
-	fi
-
-	createme -v -v ${createme_flags} ${ARCH} ${machine_branch}
-	if [ $? -ne 0 ]; then
-		bbfatal_log "Could not create ${machine_branch}"
-	fi
-
 	sccs="$sccs ${@" ".join(find_sccs(d))}"
 	patches="${@" ".join(find_patches(d))}"
 	feat_dirs="${@" ".join(find_kernel_feature_dirs(d))}"
@@ -189,34 +179,18 @@ do_patch() {
 do_kernel_checkout() {
 	set +e
 
-	# A linux yocto SRC_URI should use the bareclone option. That
-	# ensures that all the branches are available in the WORKDIR version
-	# of the repository.
 	source_dir=`echo ${S} | sed 's%/$%%'`
 	source_workdir="${WORKDIR}/git"
-	if [ -d "${WORKDIR}/git/" ] && [ -d "${WORKDIR}/git/.git" ]; then
-		# case2: the repository is a non-bare clone
-
+	if [ -d "${WORKDIR}/git/" ]; then
+		# case: git repository (bare or non-bare)
 		# if S is WORKDIR/git, then we shouldn't be moving or deleting the tree.
 		if [ "${source_dir}" != "${source_workdir}" ]; then
 			rm -rf ${S}
 			mv ${WORKDIR}/git ${S}
 		fi
 		cd ${S}
-	elif [ -d "${WORKDIR}/git/" ] && [ ! -d "${WORKDIR}/git/.git" ]; then
-		# case2: the repository is a bare clone
-
-		# if S is WORKDIR/git, then we shouldn't be moving or deleting the tree.
-		if [ "${source_dir}" != "${source_workdir}" ]; then
-			rm -rf ${S}
-			mkdir -p ${S}/.git
-			mv ${WORKDIR}/git/* ${S}/.git
-			rm -rf ${WORKDIR}/git/
-		fi
-		cd ${S}	
-		git config core.bare false
 	else
-		# case 3: we have no git repository at all. 
+		# case: we have no git repository at all. 
 		# To support low bandwidth options for building the kernel, we'll just 
 		# convert the tree to a git repo and let the rest of the process work unchanged
 		
@@ -235,7 +209,6 @@ do_kernel_checkout() {
 		git commit -q -m "baseline commit: creating repo for ${PN}-${PV}"
 		git clean -d -f
 	fi
-	# end debare
 
 	# convert any remote branches to local tracking ones
 	for i in `git branch -a --no-color | grep remotes | grep -v HEAD`; do
@@ -246,24 +219,8 @@ do_kernel_checkout() {
 		fi
 	done
 
-       	# If KMETA is defined, the branch must exist, but a machine branch
-	# can be missing since it may be created later by the tools.
-	if [ -n "${KMETA}" ]; then
-		git show-ref --quiet --verify -- "refs/heads/${KMETA}"
-		if [ $? -eq 1 ]; then
-			bberror "The branch '${KMETA}' is required and was not found"
-			bberror "Ensure that the SRC_URI points to a valid linux-yocto"
-			bbfatal_log "kernel repository"
-		fi
-	fi
-	
-
 	# Create a working tree copy of the kernel by checking out a branch
 	machine_branch="${@ get_machine_branch(d, "${KBRANCH}" )}"
-	git show-ref --quiet --verify -- "refs/heads/${machine_branch}"
-	if [ $? -ne 0 ]; then
-		machine_branch="master"
-	fi
 
 	# checkout and clobber any unimportant files
 	git checkout -f ${machine_branch}
@@ -313,7 +270,7 @@ python do_kernel_configcheck() {
         kmeta = "." + kmeta
 
     pathprefix = "export PATH=%s:%s; " % (d.getVar('PATH', True), "${S}/scripts/util/")
-    cmd = d.expand("cd ${S}; kconf_check -config- %s/meta-series ${S} ${B}" % kmeta)
+    cmd = d.expand("cd ${S}; kconf_check -config %s/meta-series ${S} ${B}" % kmeta)
     ret, result = oe.utils.getstatusoutput("%s%s" % (pathprefix, cmd))
 
     config_check_visibility = int(d.getVar( "KCONF_AUDIT_LEVEL", True ) or 0)
@@ -351,7 +308,6 @@ python do_kernel_configcheck() {
 do_validate_branches() {
 	set +e
 	cd ${S}
-	export KMETA=${KMETA}
 
 	machine_branch="${@ get_machine_branch(d, "${KBRANCH}" )}"
 	machine_srcrev="${SRCREV_machine}"
@@ -377,28 +333,6 @@ do_validate_branches() {
 		force_srcrev=${machine_srcrev}
 	fi
 
-	## KMETA branch validation.
-	target_meta_head="${SRCREV_meta}"
-	if [ "${target_meta_head}" = "AUTOINC" ] || [ "${target_meta_head}" = "" ]; then
-		bbnote "SRCREV validation skipped for AUTOREV or empty meta branch"
-	else
-	 	meta_head=`git show-ref -s --heads ${KMETA}`
-
-		git cat-file -t ${target_meta_head} > /dev/null
-		if [ $? -ne 0 ]; then
-			bberror "${target_meta_head} is not a valid commit ID"
-			bbfatal_log "The kernel source tree may be out of sync"
-		fi
-		if [ "$meta_head" != "$target_meta_head" ]; then
-			bbnote "Setting branch ${KMETA} to ${target_meta_head}"
-			git branch -m ${KMETA} ${KMETA}-orig
-			git checkout -q -b ${KMETA} ${target_meta_head}
-			if [ $? -ne 0 ];then
-				bbfatal_log "Could not checkout ${KMETA} branch from known hash ${target_meta_head}"
-			fi
-		fi
-	fi
-
 	git checkout -q -f ${machine_branch}
 	if [ -n "${force_srcrev}" ]; then
 		# see if the branch we are about to patch has been properly reset to the defined
diff --git a/meta/recipes-kernel/linux/linux-yocto-dev.bb b/meta/recipes-kernel/linux/linux-yocto-dev.bb
index 588a89f6411b..0be30893acec 100644
--- a/meta/recipes-kernel/linux/linux-yocto-dev.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-dev.bb
@@ -23,9 +23,10 @@ python () {
 }
 
 KBRANCH = "standard/base"
-KMETA = "meta"
+KMETA = "kernel-meta"
 
-SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;bareclone=1;branch=${KBRANCH},${KMETA};name=machine,meta"
+SRC_URI = "git://git.yoctoproject.org/linux-yocto-dev.git;branch=${KBRANCH};name=machine \
+           git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=master;destsuffix=${KMETA}"
 
 # Set default SRCREVs. Both the machine and meta SRCREVs are statically set
 # to the korg v3.7 tag, and hence prevent network access during parsing. If
diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_3.14.bb b/meta/recipes-kernel/linux/linux-yocto-rt_3.14.bb
index 9bdbc49ffeb2..0bd94f0c600d 100644
--- a/meta/recipes-kernel/linux/linux-yocto-rt_3.14.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_3.14.bb
@@ -5,15 +5,16 @@ require recipes-kernel/linux/linux-yocto.inc
 
 SRCREV_machine ?= "863ba0912f559ba9d64ab94bf04f0226fdf0cb49"
 SRCREV_machine_qemuppc ?= "9d464d6696a0fc755c65a2cf75ad7a4656ac6e1e"
-SRCREV_meta ?= "94fa1d7e980c97fcd59b37daedcd863bd6daaee4"
+SRCREV_meta ?= "b55cfc0308bf7158843db4b8f69f866487a0919e"
 
-SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.14.git;bareclone=1;branch=${KBRANCH},meta;name=machine,meta"
+SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.14.git;branch=${KBRANCH};name=machine \
+           git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-3.14;destsuffix=${KMETA}"
 
 LINUX_VERSION ?= "3.14.36"
 
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
-KMETA = "meta"
+KMETA = "kernel-meta"
 KCONF_BSP_AUDIT_LEVEL = "2"
 
 LINUX_KERNEL_TYPE = "preempt-rt"
diff --git a/meta/recipes-kernel/linux/linux-yocto-tiny_3.14.bb b/meta/recipes-kernel/linux/linux-yocto-tiny_3.14.bb
index bd7b6da28c2c..f486d41efad9 100644
--- a/meta/recipes-kernel/linux/linux-yocto-tiny_3.14.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-tiny_3.14.bb
@@ -6,15 +6,16 @@ require recipes-kernel/linux/linux-yocto.inc
 
 LINUX_VERSION ?= "3.14.36"
 
-KMETA = "meta"
+KMETA = "kernel-meta"
 KCONF_BSP_AUDIT_LEVEL = "2"
 
 SRCREV_machine ?= "7534aeb01883b48cc42eb4900d0a8c64e8160e14"
-SRCREV_meta ?= "94fa1d7e980c97fcd59b37daedcd863bd6daaee4"
+SRCREV_meta ?= "b55cfc0308bf7158843db4b8f69f866487a0919e"
 
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
-SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.14.git;bareclone=1;branch=${KBRANCH},meta;name=machine,meta"
+SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.14.git;branch=${KBRANCH};name=machine; \
+           git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-3.14;destsuffix=${KMETA}"
 
 COMPATIBLE_MACHINE = "(qemux86)"
 
diff --git a/meta/recipes-kernel/linux/linux-yocto-tiny_3.19.bb b/meta/recipes-kernel/linux/linux-yocto-tiny_3.19.bb
index 172d3b480009..c82da372681d 100644
--- a/meta/recipes-kernel/linux/linux-yocto-tiny_3.19.bb
+++ b/meta/recipes-kernel/linux/linux-yocto-tiny_3.19.bb
@@ -6,15 +6,16 @@ require recipes-kernel/linux/linux-yocto.inc
 
 LINUX_VERSION ?= "3.19.5"
 
-KMETA = "meta"
+KMETA = "kernel-meta"
 KCONF_BSP_AUDIT_LEVEL = "2"
 
 SRCREV_machine ?= "e152349de59b43b2a75f2c332b44171df461d5a0"
-SRCREV_meta ?= "a70b2eb273ef6349d344920474a494697474b98e"
+SRCREV_meta ?= "361c186effc0b0038dfbfd3fe71ecf3df5edb43d"
 
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
-SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.19.git;bareclone=1;branch=${KBRANCH},meta;name=machine,meta"
+SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.19.git;branch=${KBRANCH};name=machine \
+           git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-3.19;destsuffix=${KMETA}"
 
 COMPATIBLE_MACHINE = "(qemux86)"
 
diff --git a/meta/recipes-kernel/linux/linux-yocto_3.14.bb b/meta/recipes-kernel/linux/linux-yocto_3.14.bb
index 7f32c7a1b52c..9bbcecc76f33 100644
--- a/meta/recipes-kernel/linux/linux-yocto_3.14.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_3.14.bb
@@ -19,15 +19,16 @@ SRCREV_machine_qemux86 ?= "48833301518748d82cbc2412fea3617eeee6d01b"
 SRCREV_machine_qemux86-64 ?= "7534aeb01883b48cc42eb4900d0a8c64e8160e14"
 SRCREV_machine_qemumips64 ?= "c910c6d8338ab7291f066edc06de83de5b645d8f"
 SRCREV_machine ?= "7534aeb01883b48cc42eb4900d0a8c64e8160e14"
-SRCREV_meta ?= "94fa1d7e980c97fcd59b37daedcd863bd6daaee4"
+SRCREV_meta ?= "b55cfc0308bf7158843db4b8f69f866487a0919e"
 
-SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.14.git;bareclone=1;branch=${KBRANCH},${KMETA};name=machine,meta"
+SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.14.git;branch=${KBRANCH};name=machine; \
+           git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-3.14;destsuffix=${KMETA}"
 
 LINUX_VERSION ?= "3.14.36"
 
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
-KMETA = "meta"
+KMETA = "kernel-meta"
 KCONF_BSP_AUDIT_LEVEL = "2"
 
 COMPATIBLE_MACHINE = "qemuarm|qemuarm64|qemux86|qemuppc|qemumips|qemumips64|qemux86-64"
diff --git a/meta/recipes-kernel/linux/linux-yocto_3.19.bb b/meta/recipes-kernel/linux/linux-yocto_3.19.bb
index cab06266f931..364f1f60ddcd 100644
--- a/meta/recipes-kernel/linux/linux-yocto_3.19.bb
+++ b/meta/recipes-kernel/linux/linux-yocto_3.19.bb
@@ -19,15 +19,16 @@ SRCREV_machine_qemux86 ?= "e152349de59b43b2a75f2c332b44171df461d5a0"
 SRCREV_machine_qemux86-64 ?= "e152349de59b43b2a75f2c332b44171df461d5a0"
 SRCREV_machine_qemumips64 ?= "3eb70cea3532e22ab1b6da9864446621229e6616"
 SRCREV_machine ?= "e152349de59b43b2a75f2c332b44171df461d5a0"
-SRCREV_meta ?= "a70b2eb273ef6349d344920474a494697474b98e"
+SRCREV_meta ?= "361c186effc0b0038dfbfd3fe71ecf3df5edb43d"
 
-SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.19.git;bareclone=1;branch=${KBRANCH},${KMETA};name=machine,meta"
+SRC_URI = "git://git.yoctoproject.org/linux-yocto-3.19.git;name=machine;branch=${KBRANCH}; \
+           git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-3.19;destsuffix=${KMETA}"
 
 LINUX_VERSION ?= "3.19.5"
 
 PV = "${LINUX_VERSION}+git${SRCPV}"
 
-KMETA = "meta"
+KMETA = "kernel-meta"
 KCONF_BSP_AUDIT_LEVEL = "2"
 
 COMPATIBLE_MACHINE = "qemuarm|qemuarm64|qemux86|qemuppc|qemumips|qemumips64|qemux86-64"
-- 
2.1.0

[PATCH 2/4] kern-tools: standalone tree configuration

[Bruce Ashfield]

Updating the kern-tools SRCREV to import the following changes:

  cbd4b7102668 patchme/updateme: unify meta directory handling
  b65075997152 configme: standalone operation

The change of note is [configme: standalone operation], which makes the
kernel configuration script free from dependencies on other parts of the
kern-tools.

With this change, we set the stage to extend kernel configuration
fragments and auditing to arbitrary trees.

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/recipes-kernel/kern-tools/kern-tools-native_git.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
index e83f195a5393..4753094aa40c 100644
--- a/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
+++ b/meta/recipes-kernel/kern-tools/kern-tools-native_git.bb
@@ -4,7 +4,7 @@ LIC_FILES_CHKSUM = "file://git/tools/kgit;beginline=5;endline=9;md5=d8d1d729a70c
 
 DEPENDS = "git-native"
 
-SRCREV = "79bc82453c26d426d3d8fd05d5da37cf5fc4068c"
+SRCREV = "bd144d43ca5b1eaf9e727bced4ce3b61b642297c"
 PR = "r12"
 PV = "0.2+git${SRCPV}"
 
-- 
2.1.0

[PATCH 3/4] linux-libc-headers: update to 4.1

[Bruce Ashfield]

Updating the libc-headers to match the 4.1 LTSI kernel version.

We also tweak the logic in linux-libc-headers.inc to look in the
4.x subdirectory if a 4.x kernel is specified as the header source.

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/conf/distro/include/tcmode-default.inc                       | 2 +-
 meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc     | 2 ++
 meta/recipes-kernel/linux-libc-headers/linux-libc-headers_3.19.bb | 7 -------
 meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.1.bb  | 7 +++++++
 4 files changed, 10 insertions(+), 8 deletions(-)
 delete mode 100644 meta/recipes-kernel/linux-libc-headers/linux-libc-headers_3.19.bb
 create mode 100644 meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.1.bb

diff --git a/meta/conf/distro/include/tcmode-default.inc b/meta/conf/distro/include/tcmode-default.inc
index 096bec9fd623..84ecd7248021 100644
--- a/meta/conf/distro/include/tcmode-default.inc
+++ b/meta/conf/distro/include/tcmode-default.inc
@@ -28,7 +28,7 @@ BINUVERSION ?= "2.25"
 GDBVERSION ?= "7.9%"
 GLIBCVERSION ?= "2.21"
 UCLIBCVERSION ?= "0.9.33+git%"
-LINUXLIBCVERSION ?= "3.19"
+LINUXLIBCVERSION ?= "4.1"
 
 PREFERRED_VERSION_gcc ?= "${GCCVERSION}"
 PREFERRED_VERSION_gcc-cross-${TARGET_ARCH} ?= "${GCCVERSION}"
diff --git a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
index b18d09fd6c3e..566bc1e6154b 100644
--- a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
+++ b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
@@ -35,6 +35,8 @@ python __anonymous () {
     major = d.getVar("PV",True).split('.')[0]
     if major == "3":
         d.setVar("HEADER_FETCH_VER", "3.0")
+    elif major == "4":
+        d.setVar("HEADER_FETCH_VER", "4.x")
     else:
         d.setVar("HEADER_FETCH_VER", "2.6")
 }
diff --git a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_3.19.bb b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_3.19.bb
deleted file mode 100644
index 078df4512ba1..000000000000
--- a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_3.19.bb
+++ /dev/null
@@ -1,7 +0,0 @@
-KORG_ARCHIVE_COMPRESSION = "xz"
-
-require linux-libc-headers.inc
-
-SRC_URI[md5sum] = "d3fc8316d4d4d04b65cbc2d70799e763"
-SRC_URI[sha256sum] = "be42511fe5321012bb4a2009167ce56a9e5fe362b4af43e8c371b3666859806c"
-
diff --git a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.1.bb b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.1.bb
new file mode 100644
index 000000000000..d816e5e7845e
--- /dev/null
+++ b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.1.bb
@@ -0,0 +1,7 @@
+KORG_ARCHIVE_COMPRESSION = "xz"
+
+require linux-libc-headers.inc
+
+SRC_URI[md5sum] = "fe9dc0f6729f36400ea81aa41d614c37"
+SRC_URI[sha256sum] = "caf51f085aac1e1cea4d00dbbf3093ead07b551fc07b31b2a989c05f8ea72d9f"
+
-- 
2.1.0

[PATCH 4/4] linux-yocto: introduce 4.1 versioned recipes

[Bruce Ashfield]

Updating the linux-yocto kernel recipes to include a named/versioned 4.1
recipe. This will be the LTSI kernel, and the 3.14 and 3.19 kernels will
be removed in subsequent commites (once reference boards have transitioned).

Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
---
 meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb   | 26 ++++++++++++++
 meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb | 23 +++++++++++++
 meta/recipes-kernel/linux/linux-yocto_4.1.bb      | 41 +++++++++++++++++++++++
 3 files changed, 90 insertions(+)
 create mode 100644 meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb
 create mode 100644 meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb
 create mode 100644 meta/recipes-kernel/linux/linux-yocto_4.1.bb

diff --git a/meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb b/meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb
new file mode 100644
index 000000000000..1cefff871440
--- /dev/null
+++ b/meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb
@@ -0,0 +1,26 @@
+KBRANCH ?= "standard/preempt-rt"
+
+require recipes-kernel/linux/linux-yocto.inc
+
+SRCREV_machine ?= "fe1ee443489d4ab5ce04c903b0362ae565547bd1"
+SRCREV_meta ?= "45393dd54f5ad77d43014c407c2b3520da42f427"
+
+SRC_URI = "git://git.yoctoproject.org/linux-yocto-4.1.git;branch=${KBRANCH};name=machine \
+           git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.1;destsuffix=${KMETA}"
+
+LINUX_VERSION ?= "4.1.2"
+
+PV = "${LINUX_VERSION}+git${SRCPV}"
+
+KMETA = "kernel-meta"
+KCONF_BSP_AUDIT_LEVEL = "2"
+
+LINUX_KERNEL_TYPE = "preempt-rt"
+
+COMPATIBLE_MACHINE = "(qemux86|qemux86-64|qemuarm|qemuppc|qemumips)"
+
+# Functionality flags
+KERNEL_EXTRA_FEATURES ?= "features/netfilter/netfilter.scc features/taskstats/taskstats.scc"
+KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
+KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
+KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc"
diff --git a/meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb b/meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb
new file mode 100644
index 000000000000..38167cc2e85e
--- /dev/null
+++ b/meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb
@@ -0,0 +1,23 @@
+KBRANCH ?= "standard/tiny/base"
+LINUX_KERNEL_TYPE = "tiny"
+KCONFIG_MODE = "--allnoconfig"
+
+require recipes-kernel/linux/linux-yocto.inc
+
+LINUX_VERSION ?= "4.1.2"
+
+KMETA = "kernel-meta"
+KCONF_BSP_AUDIT_LEVEL = "2"
+
+SRCREV_machine ?= "4e30e64c44df9e59bd13239951bb8d2b5b276e6f"
+SRCREV_meta ?= "45393dd54f5ad77d43014c407c2b3520da42f427"
+
+PV = "${LINUX_VERSION}+git${SRCPV}"
+
+SRC_URI = "git://git.yoctoproject.org/linux-yocto-4.1.git;branch=${KBRANCH};name=machine \
+           git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.1;destsuffix=${KMETA}"
+
+COMPATIBLE_MACHINE = "(qemux86)"
+
+# Functionality flags
+KERNEL_FEATURES = ""
diff --git a/meta/recipes-kernel/linux/linux-yocto_4.1.bb b/meta/recipes-kernel/linux/linux-yocto_4.1.bb
new file mode 100644
index 000000000000..a7a52cb62762
--- /dev/null
+++ b/meta/recipes-kernel/linux/linux-yocto_4.1.bb
@@ -0,0 +1,41 @@
+KBRANCH ?= "standard/base"
+
+require recipes-kernel/linux/linux-yocto.inc
+
+# board specific branches
+KBRANCH_qemuarm  ?= "standard/arm-versatile-926ejs"
+KBRANCH_qemuarm64 ?= "standard/qemuarm64"
+KBRANCH_qemumips ?= "standard/mti-malta32"
+KBRANCH_qemuppc  ?= "standard/qemuppc"
+KBRANCH_qemux86  ?= "standard/base"
+KBRANCH_qemux86-64 ?= "standard/base"
+KBRANCH_qemumips64 ?= "standard/mti-malta64"
+
+SRCREV_machine_qemuarm ?= "c4df99c91fc947afb7010a11eceeef0e05bf964f"
+SRCREV_machine_qemuarm64 ?= "4e30e64c44df9e59bd13239951bb8d2b5b276e6f"
+SRCREV_machine_qemumips ?= "6a24f7e60b9b2ecc1579f5f767be5845086533e1"
+SRCREV_machine_qemuppc ?= "4e30e64c44df9e59bd13239951bb8d2b5b276e6f"
+SRCREV_machine_qemux86 ?= "4e30e64c44df9e59bd13239951bb8d2b5b276e6f"
+SRCREV_machine_qemux86-64 ?= "4e30e64c44df9e59bd13239951bb8d2b5b276e6f"
+SRCREV_machine_qemumips64 ?= "5c2b3697082a4ec6641aa5a8eca3974ca609cecf"
+SRCREV_machine ?= "4e30e64c44df9e59bd13239951bb8d2b5b276e6f"
+SRCREV_meta ?= "45393dd54f5ad77d43014c407c2b3520da42f427"
+
+SRC_URI = "git://git.yoctoproject.org/linux-yocto-4.1.git;name=machine;branch=${KBRANCH}; \
+           git://git.yoctoproject.org/yocto-kernel-cache;type=kmeta;name=meta;branch=yocto-4.1;destsuffix=${KMETA}"
+
+LINUX_VERSION ?= "4.1.2"
+
+PV = "${LINUX_VERSION}+git${SRCPV}"
+
+KMETA = "kernel-meta"
+KCONF_BSP_AUDIT_LEVEL = "2"
+
+COMPATIBLE_MACHINE = "qemuarm|qemuarm64|qemux86|qemuppc|qemumips|qemumips64|qemux86-64"
+
+# Functionality flags
+KERNEL_EXTRA_FEATURES ?= "features/netfilter/netfilter.scc"
+KERNEL_FEATURES_append = " ${KERNEL_EXTRA_FEATURES}"
+KERNEL_FEATURES_append_qemux86=" cfg/sound.scc cfg/paravirt_kvm.scc"
+KERNEL_FEATURES_append_qemux86-64=" cfg/sound.scc cfg/paravirt_kvm.scc"
+KERNEL_FEATURES_append = " ${@bb.utils.contains("TUNE_FEATURES", "mx32", " cfg/x32.scc", "" ,d)}"
-- 
2.1.0

Re: [PATCH 0/4] linux-yocto: build changes, 4.1 and libc-headers

[Bruce Ashfield]

On Tue, Jul 21, 2015 at 11:21 AM, Bruce Ashfield
<bruce.ashfield@windriver.com> wrote:
> Hi all,
>
> This series absolutely needs some more testing and runs through the
> autobuilder, but I've done pretty much all I can do with it here.
>
> I have more changes to the build process that will follow this, but
> these changes need to show that they are stable, so here's the first
> blast.
>
> With this series:
>
>   - I create the 4.1, and 4.1-rt linux-yocto content. This will be
>     the new LTSI kernel. Deletion of 3.14 and 3.19 will follow once
>     all machines have been moved forward.
>
>   - libc-headers updated to match the 4.1 kernel version.
>
> And the one that could cause some issues/compatibility issues is the
> split of the kernel configuration data from the kernel tree itself.
>
> From the commit log:
>
>   The linux-yocto tree has always been a combined set of kernel changes
>   and configuration (meta) data carried in a single tree. While this
>   format is effective at keeping kernel configuration and source
>   modifications synchronized, it isn't always obvious to developers on
>   how to manipulate the meta data versus the source.
>
>   With this change, we remove the meta data processing from the
>   kernel-yocto class and use the external meta-data repository that
>   has always been used to seed the linux-yocto meta branch.
>
>   After this change, linux-yocto can no longer process combined trees,
>   and is simplified as a result.
>
> I'm interested to find out if we get any breakages or severe compability
> issues with the change. I don't want to support both variants of the
> meta data (in and out of tree), since the goal is to reduce complexity
> in the code. Once this change lands, I'll further streamline things.
>
> I've built core-image-kernel-dev and boot tested the qemu* machines.
> qemuppc continues to show a boot failure with systemd, but we have a
> bugzilla to track that.
>
> I'm now doing graphical testing, but wanted to get this out and soaking
> for other images types while I wait for builds to churn.

As a follow up, my qemuarm boots that worked on Friday, are now hanging.
I'm bisecting to find out what happened.

But I have tested graphics on x86 and mips without issue.

Cheers,

Bruce

>
> Cheers,
>
> Bruce
>
> The following changes since commit 0f8d89ce66cfffccc9911d6c61470a585173b79d:
>
>   core-image-lsb-sdk: use kernel-devsrc to ensure kernel module builds work (2015-07-20 10:40:42 +0100)
>
> are available in the git repository at:
>
>   git://git.pokylinux.org/poky-contrib zedd/kernel
>   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel
>
> Bruce Ashfield (4):
>   linux-yocto: split meta data from kernel repository
>   kern-tools: standalone tree configuration
>   linux-libc-headers: update to 4.1
>   linux-yocto: introduce 4.1 versioned recipes
>
>  meta/classes/kernel-yocto.bbclass                  | 76 ++--------------------
>  meta/conf/distro/include/tcmode-default.inc        |  2 +-
>  .../kern-tools/kern-tools-native_git.bb            |  2 +-
>  .../linux-libc-headers/linux-libc-headers.inc      |  2 +
>  .../linux-libc-headers/linux-libc-headers_3.19.bb  |  7 --
>  .../linux-libc-headers/linux-libc-headers_4.1.bb   |  7 ++
>  meta/recipes-kernel/linux/linux-yocto-dev.bb       |  5 +-
>  meta/recipes-kernel/linux/linux-yocto-rt_3.14.bb   |  7 +-
>  meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb    | 26 ++++++++
>  meta/recipes-kernel/linux/linux-yocto-tiny_3.14.bb |  7 +-
>  meta/recipes-kernel/linux/linux-yocto-tiny_3.19.bb |  7 +-
>  meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb  | 23 +++++++
>  meta/recipes-kernel/linux/linux-yocto_3.14.bb      |  7 +-
>  meta/recipes-kernel/linux/linux-yocto_3.19.bb      |  7 +-
>  meta/recipes-kernel/linux/linux-yocto_4.1.bb       | 41 ++++++++++++
>  15 files changed, 129 insertions(+), 97 deletions(-)
>  delete mode 100644 meta/recipes-kernel/linux-libc-headers/linux-libc-headers_3.19.bb
>  create mode 100644 meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.1.bb
>  create mode 100644 meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb
>  create mode 100644 meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb
>  create mode 100644 meta/recipes-kernel/linux/linux-yocto_4.1.bb
>
> --
> 2.1.0
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: [PATCH 0/4] linux-yocto: build changes, 4.1 and libc-headers

[Bruce Ashfield]

On Wed, Jul 22, 2015 at 10:03 AM, Bruce Ashfield
<bruce.ashfield@gmail.com> wrote:
> On Tue, Jul 21, 2015 at 11:21 AM, Bruce Ashfield
> <bruce.ashfield@windriver.com> wrote:
>> Hi all,
>>
>> This series absolutely needs some more testing and runs through the
>> autobuilder, but I've done pretty much all I can do with it here.
>>
>> I have more changes to the build process that will follow this, but
>> these changes need to show that they are stable, so here's the first
>> blast.
>>
>> With this series:
>>
>>   - I create the 4.1, and 4.1-rt linux-yocto content. This will be
>>     the new LTSI kernel. Deletion of 3.14 and 3.19 will follow once
>>     all machines have been moved forward.
>>
>>   - libc-headers updated to match the 4.1 kernel version.
>>
>> And the one that could cause some issues/compatibility issues is the
>> split of the kernel configuration data from the kernel tree itself.
>>
>> From the commit log:
>>
>>   The linux-yocto tree has always been a combined set of kernel changes
>>   and configuration (meta) data carried in a single tree. While this
>>   format is effective at keeping kernel configuration and source
>>   modifications synchronized, it isn't always obvious to developers on
>>   how to manipulate the meta data versus the source.
>>
>>   With this change, we remove the meta data processing from the
>>   kernel-yocto class and use the external meta-data repository that
>>   has always been used to seed the linux-yocto meta branch.
>>
>>   After this change, linux-yocto can no longer process combined trees,
>>   and is simplified as a result.
>>
>> I'm interested to find out if we get any breakages or severe compability
>> issues with the change. I don't want to support both variants of the
>> meta data (in and out of tree), since the goal is to reduce complexity
>> in the code. Once this change lands, I'll further streamline things.
>>
>> I've built core-image-kernel-dev and boot tested the qemu* machines.
>> qemuppc continues to show a boot failure with systemd, but we have a
>> bugzilla to track that.
>>
>> I'm now doing graphical testing, but wanted to get this out and soaking
>> for other images types while I wait for builds to churn.
>
> As a follow up, my qemuarm boots that worked on Friday, are now hanging.
> I'm bisecting to find out what happened.

And a final follow up.

qemuarm (console and graphics) works fine .. but only if you use gcc
4.9.x, gcc 5.1
is doing something evil.

We've looked into this in the past, so it is a known issue, but I need
to handle these
separately.

Cheers,

Bruce

>
> But I have tested graphics on x86 and mips without issue.
>
> Cheers,
>
> Bruce
>
>>
>> Cheers,
>>
>> Bruce
>>
>> The following changes since commit 0f8d89ce66cfffccc9911d6c61470a585173b79d:
>>
>>   core-image-lsb-sdk: use kernel-devsrc to ensure kernel module builds work (2015-07-20 10:40:42 +0100)
>>
>> are available in the git repository at:
>>
>>   git://git.pokylinux.org/poky-contrib zedd/kernel
>>   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=zedd/kernel
>>
>> Bruce Ashfield (4):
>>   linux-yocto: split meta data from kernel repository
>>   kern-tools: standalone tree configuration
>>   linux-libc-headers: update to 4.1
>>   linux-yocto: introduce 4.1 versioned recipes
>>
>>  meta/classes/kernel-yocto.bbclass                  | 76 ++--------------------
>>  meta/conf/distro/include/tcmode-default.inc        |  2 +-
>>  .../kern-tools/kern-tools-native_git.bb            |  2 +-
>>  .../linux-libc-headers/linux-libc-headers.inc      |  2 +
>>  .../linux-libc-headers/linux-libc-headers_3.19.bb  |  7 --
>>  .../linux-libc-headers/linux-libc-headers_4.1.bb   |  7 ++
>>  meta/recipes-kernel/linux/linux-yocto-dev.bb       |  5 +-
>>  meta/recipes-kernel/linux/linux-yocto-rt_3.14.bb   |  7 +-
>>  meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb    | 26 ++++++++
>>  meta/recipes-kernel/linux/linux-yocto-tiny_3.14.bb |  7 +-
>>  meta/recipes-kernel/linux/linux-yocto-tiny_3.19.bb |  7 +-
>>  meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb  | 23 +++++++
>>  meta/recipes-kernel/linux/linux-yocto_3.14.bb      |  7 +-
>>  meta/recipes-kernel/linux/linux-yocto_3.19.bb      |  7 +-
>>  meta/recipes-kernel/linux/linux-yocto_4.1.bb       | 41 ++++++++++++
>>  15 files changed, 129 insertions(+), 97 deletions(-)
>>  delete mode 100644 meta/recipes-kernel/linux-libc-headers/linux-libc-headers_3.19.bb
>>  create mode 100644 meta/recipes-kernel/linux-libc-headers/linux-libc-headers_4.1.bb
>>  create mode 100644 meta/recipes-kernel/linux/linux-yocto-rt_4.1.bb
>>  create mode 100644 meta/recipes-kernel/linux/linux-yocto-tiny_4.1.bb
>>  create mode 100644 meta/recipes-kernel/linux/linux-yocto_4.1.bb
>>
>> --
>> 2.1.0
>>
>> --
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
>
> --
> "Thou shalt not follow the NULL pointer, for chaos and madness await
> thee at its end"



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: [PATCH 0/4] linux-yocto: build changes, 4.1 and libc-headers

[Richard Purdie]

On Wed, 2015-07-22 at 15:48 -0400, Bruce Ashfield wrote:
> On Wed, Jul 22, 2015 at 10:03 AM, Bruce Ashfield
> <bruce.ashfield@gmail.com> wrote:
> > On Tue, Jul 21, 2015 at 11:21 AM, Bruce Ashfield
> > <bruce.ashfield@windriver.com> wrote:
> >> Hi all,
> >>
> >> This series absolutely needs some more testing and runs through the
> >> autobuilder, but I've done pretty much all I can do with it here.
> >>
> >> I have more changes to the build process that will follow this, but
> >> these changes need to show that they are stable, so here's the first
> >> blast.
> >>
> >> With this series:
> >>
> >>   - I create the 4.1, and 4.1-rt linux-yocto content. This will be
> >>     the new LTSI kernel. Deletion of 3.14 and 3.19 will follow once
> >>     all machines have been moved forward.
> >>
> >>   - libc-headers updated to match the 4.1 kernel version.
> >>
> >> And the one that could cause some issues/compatibility issues is the
> >> split of the kernel configuration data from the kernel tree itself.
> >>
> >> From the commit log:
> >>
> >>   The linux-yocto tree has always been a combined set of kernel changes
> >>   and configuration (meta) data carried in a single tree. While this
> >>   format is effective at keeping kernel configuration and source
> >>   modifications synchronized, it isn't always obvious to developers on
> >>   how to manipulate the meta data versus the source.
> >>
> >>   With this change, we remove the meta data processing from the
> >>   kernel-yocto class and use the external meta-data repository that
> >>   has always been used to seed the linux-yocto meta branch.
> >>
> >>   After this change, linux-yocto can no longer process combined trees,
> >>   and is simplified as a result.
> >>
> >> I'm interested to find out if we get any breakages or severe compability
> >> issues with the change. I don't want to support both variants of the
> >> meta data (in and out of tree), since the goal is to reduce complexity
> >> in the code. Once this change lands, I'll further streamline things.
> >>
> >> I've built core-image-kernel-dev and boot tested the qemu* machines.
> >> qemuppc continues to show a boot failure with systemd, but we have a
> >> bugzilla to track that.
> >>
> >> I'm now doing graphical testing, but wanted to get this out and soaking
> >> for other images types while I wait for builds to churn.
> >
> > As a follow up, my qemuarm boots that worked on Friday, are now hanging.
> > I'm bisecting to find out what happened.
> 
> And a final follow up.
> 
> qemuarm (console and graphics) works fine .. but only if you use gcc
> 4.9.x, gcc 5.1
> is doing something evil.
> 
> We've looked into this in the past, so it is a known issue, but I need
> to handle these
> separately.

We got the results of a run of those changes through the autobuilder
without the noise of other failures. The error reporting system shows:

http://errors.yoctoproject.org/Errors/Search/?items=10&query=d687cdfd04f3d97923c93239ea902bb38e2ea336

and I believe we have the following issues:

a) qemux86/qemux86-64 are throwing warnings about errors in logs
b) perf has some weird make race
c) the qemuarm build looks like qemu either segfaulted or was killed
d) we're occasionally seeing 3.19 and 3.14 fetch failures:
https://autobuilder.yoctoproject.org/main/builders/nightly-intel-gpl/builds/394/steps/BuildImages/logs/stdio
https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/87/steps/Running%20oe-selftest/logs/stdio

To try and promote more sstate cache reuse whilst we sort the remaining
issues, I merged the linux-libc-header part, we need to get the above
fixed before the other pieces can merge.

Cheers,

Richard

Re: [PATCH 0/4] linux-yocto: build changes, 4.1 and libc-headers

[Bruce Ashfield]

On Thu, Jul 23, 2015 at 3:56 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Wed, 2015-07-22 at 15:48 -0400, Bruce Ashfield wrote:
>> On Wed, Jul 22, 2015 at 10:03 AM, Bruce Ashfield
>> <bruce.ashfield@gmail.com> wrote:
>> > On Tue, Jul 21, 2015 at 11:21 AM, Bruce Ashfield
>> > <bruce.ashfield@windriver.com> wrote:
>> >> Hi all,
>> >>
>> >> This series absolutely needs some more testing and runs through the
>> >> autobuilder, but I've done pretty much all I can do with it here.
>> >>
>> >> I have more changes to the build process that will follow this, but
>> >> these changes need to show that they are stable, so here's the first
>> >> blast.
>> >>
>> >> With this series:
>> >>
>> >>   - I create the 4.1, and 4.1-rt linux-yocto content. This will be
>> >>     the new LTSI kernel. Deletion of 3.14 and 3.19 will follow once
>> >>     all machines have been moved forward.
>> >>
>> >>   - libc-headers updated to match the 4.1 kernel version.
>> >>
>> >> And the one that could cause some issues/compatibility issues is the
>> >> split of the kernel configuration data from the kernel tree itself.
>> >>
>> >> From the commit log:
>> >>
>> >>   The linux-yocto tree has always been a combined set of kernel changes
>> >>   and configuration (meta) data carried in a single tree. While this
>> >>   format is effective at keeping kernel configuration and source
>> >>   modifications synchronized, it isn't always obvious to developers on
>> >>   how to manipulate the meta data versus the source.
>> >>
>> >>   With this change, we remove the meta data processing from the
>> >>   kernel-yocto class and use the external meta-data repository that
>> >>   has always been used to seed the linux-yocto meta branch.
>> >>
>> >>   After this change, linux-yocto can no longer process combined trees,
>> >>   and is simplified as a result.
>> >>
>> >> I'm interested to find out if we get any breakages or severe compability
>> >> issues with the change. I don't want to support both variants of the
>> >> meta data (in and out of tree), since the goal is to reduce complexity
>> >> in the code. Once this change lands, I'll further streamline things.
>> >>
>> >> I've built core-image-kernel-dev and boot tested the qemu* machines.
>> >> qemuppc continues to show a boot failure with systemd, but we have a
>> >> bugzilla to track that.
>> >>
>> >> I'm now doing graphical testing, but wanted to get this out and soaking
>> >> for other images types while I wait for builds to churn.
>> >
>> > As a follow up, my qemuarm boots that worked on Friday, are now hanging.
>> > I'm bisecting to find out what happened.
>>
>> And a final follow up.
>>
>> qemuarm (console and graphics) works fine .. but only if you use gcc
>> 4.9.x, gcc 5.1
>> is doing something evil.
>>
>> We've looked into this in the past, so it is a known issue, but I need
>> to handle these
>> separately.
>
> We got the results of a run of those changes through the autobuilder
> without the noise of other failures. The error reporting system shows:
>
> http://errors.yoctoproject.org/Errors/Search/?items=10&query=d687cdfd04f3d97923c93239ea902bb38e2ea336
>
> and I believe we have the following issues:
>
> a) qemux86/qemux86-64 are throwing warnings about errors in logs
> b) perf has some weird make race
> c) the qemuarm build looks like qemu either segfaulted or was killed
> d) we're occasionally seeing 3.19 and 3.14 fetch failures:
> https://autobuilder.yoctoproject.org/main/builders/nightly-intel-gpl/builds/394/steps/BuildImages/logs/stdio
> https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/87/steps/Running%20oe-selftest/logs/stdio

I explained to Ross that dependent layers need to be updated if they set
their own SRCREV_meta, since the repository is completely different. I
sent a patch for meta-intel to deal with it. Obviously, I only built with the
core BSPs during testing.

>

I'll definitely need help on these, since I wasn't seeing any of these issues
during my testing. Otherwise, this won't be done until September (vacation
and other issues to deal with).

But I'll see what I can do.

I unfortunately can't merge anything but critical kernel patches until
these go in as well
since the meta branch is gone in the new scripts, which means SRCREV_meta is
a constant conflict.

Cheers,

Bruce

> To try and promote more sstate cache reuse whilst we sort the remaining
> issues, I merged the linux-libc-header part, we need to get the above
> fixed before the other pieces can merge.
>
> Cheers,
>
> Richard
>
>
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: [PATCH 0/4] linux-yocto: build changes, 4.1 and libc-headers

[Richard Purdie]

On Thu, 2015-07-23 at 09:10 -0400, Bruce Ashfield wrote:
> On Thu, Jul 23, 2015 at 3:56 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Wed, 2015-07-22 at 15:48 -0400, Bruce Ashfield wrote:
> >> On Wed, Jul 22, 2015 at 10:03 AM, Bruce Ashfield
> >> <bruce.ashfield@gmail.com> wrote:
> >> > On Tue, Jul 21, 2015 at 11:21 AM, Bruce Ashfield
> >> > <bruce.ashfield@windriver.com> wrote:
> >> >> Hi all,
> >> >>
> >> >> This series absolutely needs some more testing and runs through the
> >> >> autobuilder, but I've done pretty much all I can do with it here.
> >> >>
> >> >> I have more changes to the build process that will follow this, but
> >> >> these changes need to show that they are stable, so here's the first
> >> >> blast.
> >> >>
> >> >> With this series:
> >> >>
> >> >>   - I create the 4.1, and 4.1-rt linux-yocto content. This will be
> >> >>     the new LTSI kernel. Deletion of 3.14 and 3.19 will follow once
> >> >>     all machines have been moved forward.
> >> >>
> >> >>   - libc-headers updated to match the 4.1 kernel version.
> >> >>
> >> >> And the one that could cause some issues/compatibility issues is the
> >> >> split of the kernel configuration data from the kernel tree itself.
> >> >>
> >> >> From the commit log:
> >> >>
> >> >>   The linux-yocto tree has always been a combined set of kernel changes
> >> >>   and configuration (meta) data carried in a single tree. While this
> >> >>   format is effective at keeping kernel configuration and source
> >> >>   modifications synchronized, it isn't always obvious to developers on
> >> >>   how to manipulate the meta data versus the source.
> >> >>
> >> >>   With this change, we remove the meta data processing from the
> >> >>   kernel-yocto class and use the external meta-data repository that
> >> >>   has always been used to seed the linux-yocto meta branch.
> >> >>
> >> >>   After this change, linux-yocto can no longer process combined trees,
> >> >>   and is simplified as a result.
> >> >>
> >> >> I'm interested to find out if we get any breakages or severe compability
> >> >> issues with the change. I don't want to support both variants of the
> >> >> meta data (in and out of tree), since the goal is to reduce complexity
> >> >> in the code. Once this change lands, I'll further streamline things.
> >> >>
> >> >> I've built core-image-kernel-dev and boot tested the qemu* machines.
> >> >> qemuppc continues to show a boot failure with systemd, but we have a
> >> >> bugzilla to track that.
> >> >>
> >> >> I'm now doing graphical testing, but wanted to get this out and soaking
> >> >> for other images types while I wait for builds to churn.
> >> >
> >> > As a follow up, my qemuarm boots that worked on Friday, are now hanging.
> >> > I'm bisecting to find out what happened.
> >>
> >> And a final follow up.
> >>
> >> qemuarm (console and graphics) works fine .. but only if you use gcc
> >> 4.9.x, gcc 5.1
> >> is doing something evil.
> >>
> >> We've looked into this in the past, so it is a known issue, but I need
> >> to handle these
> >> separately.
> >
> > We got the results of a run of those changes through the autobuilder
> > without the noise of other failures. The error reporting system shows:
> >
> > http://errors.yoctoproject.org/Errors/Search/?items=10&query=d687cdfd04f3d97923c93239ea902bb38e2ea336
> >
> > and I believe we have the following issues:
> >
> > a) qemux86/qemux86-64 are throwing warnings about errors in logs
> > b) perf has some weird make race
> > c) the qemuarm build looks like qemu either segfaulted or was killed
> > d) we're occasionally seeing 3.19 and 3.14 fetch failures:
> > https://autobuilder.yoctoproject.org/main/builders/nightly-intel-gpl/builds/394/steps/BuildImages/logs/stdio
> > https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/87/steps/Running%20oe-selftest/logs/stdio
> 
> I explained to Ross that dependent layers need to be updated if they set
> their own SRCREV_meta, since the repository is completely different. I
> sent a patch for meta-intel to deal with it. Obviously, I only built with the
> core BSPs during testing.

That could explain d). I believe c) is from the ongoing selftest issues
we're having and I we've work in progress for that. That leaves a) and
b) to look into and fix.

> I'll definitely need help on these, since I wasn't seeing any of these issues
> during my testing. Otherwise, this won't be done until September (vacation
> and other issues to deal with).
> 
> But I'll see what I can do.
> 
> I unfortunately can't merge anything but critical kernel patches until
> these go in as well
> since the meta branch is gone in the new scripts, which means SRCREV_meta is
> a constant conflict.

a) should be straightfoward to fix, its just updating the message
whitelists (again).

b) I'm less sure about. Are there any race fixes in mainline for perf?
Hard to believe others don't see that one.

Cheers,

Richard

Re: [PATCH 0/4] linux-yocto: build changes, 4.1 and libc-headers

[Bruce Ashfield]

On Thu, Jul 23, 2015 at 10:14 AM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
> On Thu, 2015-07-23 at 09:10 -0400, Bruce Ashfield wrote:
>> On Thu, Jul 23, 2015 at 3:56 AM, Richard Purdie
>> <richard.purdie@linuxfoundation.org> wrote:
>> > On Wed, 2015-07-22 at 15:48 -0400, Bruce Ashfield wrote:
>> >> On Wed, Jul 22, 2015 at 10:03 AM, Bruce Ashfield
>> >> <bruce.ashfield@gmail.com> wrote:
>> >> > On Tue, Jul 21, 2015 at 11:21 AM, Bruce Ashfield
>> >> > <bruce.ashfield@windriver.com> wrote:
>> >> >> Hi all,
>> >> >>
>> >> >> This series absolutely needs some more testing and runs through the
>> >> >> autobuilder, but I've done pretty much all I can do with it here.
>> >> >>
>> >> >> I have more changes to the build process that will follow this, but
>> >> >> these changes need to show that they are stable, so here's the first
>> >> >> blast.
>> >> >>
>> >> >> With this series:
>> >> >>
>> >> >>   - I create the 4.1, and 4.1-rt linux-yocto content. This will be
>> >> >>     the new LTSI kernel. Deletion of 3.14 and 3.19 will follow once
>> >> >>     all machines have been moved forward.
>> >> >>
>> >> >>   - libc-headers updated to match the 4.1 kernel version.
>> >> >>
>> >> >> And the one that could cause some issues/compatibility issues is the
>> >> >> split of the kernel configuration data from the kernel tree itself.
>> >> >>
>> >> >> From the commit log:
>> >> >>
>> >> >>   The linux-yocto tree has always been a combined set of kernel changes
>> >> >>   and configuration (meta) data carried in a single tree. While this
>> >> >>   format is effective at keeping kernel configuration and source
>> >> >>   modifications synchronized, it isn't always obvious to developers on
>> >> >>   how to manipulate the meta data versus the source.
>> >> >>
>> >> >>   With this change, we remove the meta data processing from the
>> >> >>   kernel-yocto class and use the external meta-data repository that
>> >> >>   has always been used to seed the linux-yocto meta branch.
>> >> >>
>> >> >>   After this change, linux-yocto can no longer process combined trees,
>> >> >>   and is simplified as a result.
>> >> >>
>> >> >> I'm interested to find out if we get any breakages or severe compability
>> >> >> issues with the change. I don't want to support both variants of the
>> >> >> meta data (in and out of tree), since the goal is to reduce complexity
>> >> >> in the code. Once this change lands, I'll further streamline things.
>> >> >>
>> >> >> I've built core-image-kernel-dev and boot tested the qemu* machines.
>> >> >> qemuppc continues to show a boot failure with systemd, but we have a
>> >> >> bugzilla to track that.
>> >> >>
>> >> >> I'm now doing graphical testing, but wanted to get this out and soaking
>> >> >> for other images types while I wait for builds to churn.
>> >> >
>> >> > As a follow up, my qemuarm boots that worked on Friday, are now hanging.
>> >> > I'm bisecting to find out what happened.
>> >>
>> >> And a final follow up.
>> >>
>> >> qemuarm (console and graphics) works fine .. but only if you use gcc
>> >> 4.9.x, gcc 5.1
>> >> is doing something evil.
>> >>
>> >> We've looked into this in the past, so it is a known issue, but I need
>> >> to handle these
>> >> separately.
>> >
>> > We got the results of a run of those changes through the autobuilder
>> > without the noise of other failures. The error reporting system shows:
>> >
>> > http://errors.yoctoproject.org/Errors/Search/?items=10&query=d687cdfd04f3d97923c93239ea902bb38e2ea336
>> >
>> > and I believe we have the following issues:
>> >
>> > a) qemux86/qemux86-64 are throwing warnings about errors in logs
>> > b) perf has some weird make race
>> > c) the qemuarm build looks like qemu either segfaulted or was killed
>> > d) we're occasionally seeing 3.19 and 3.14 fetch failures:
>> > https://autobuilder.yoctoproject.org/main/builders/nightly-intel-gpl/builds/394/steps/BuildImages/logs/stdio
>> > https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/87/steps/Running%20oe-selftest/logs/stdio
>>
>> I explained to Ross that dependent layers need to be updated if they set
>> their own SRCREV_meta, since the repository is completely different. I
>> sent a patch for meta-intel to deal with it. Obviously, I only built with the
>> core BSPs during testing.
>
> That could explain d). I believe c) is from the ongoing selftest issues
> we're having and I we've work in progress for that. That leaves a) and
> b) to look into and fix.
>
>> I'll definitely need help on these, since I wasn't seeing any of these issues
>> during my testing. Otherwise, this won't be done until September (vacation
>> and other issues to deal with).
>>
>> But I'll see what I can do.
>>
>> I unfortunately can't merge anything but critical kernel patches until
>> these go in as well
>> since the meta branch is gone in the new scripts, which means SRCREV_meta is
>> a constant conflict.
>
> a) should be straightfoward to fix, its just updating the message
> whitelists (again).

I'm building a fresh qemux86-64 now, I'm going to see if the warnings
are something
I should fix, versus the whitelist. I should know tomorrow on this one.

>
> b) I'm less sure about. Are there any race fixes in mainline for perf?
> Hard to believe others don't see that one.

I'm searching as well. I'll share any findings ASAP.

Bruce

>
> Cheers,
>
> Richard
>



-- 
"Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end"

Re: [PATCH 0/4] linux-yocto: build changes, 4.1 and libc-headers

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 392 bytes --]

On 23 July 2015 at 16:01, Bruce Ashfield <bruce.ashfield@gmail.com> wrote:

> > a) should be straightfoward to fix, its just updating the message
> > whitelists (again).
>
> I'm building a fresh qemux86-64 now, I'm going to see if the warnings
> are something
> I should fix, versus the whitelist. I should know tomorrow on this one.


Patch sent to update the whitelists.

Ross

[-- Attachment #2: Type: text/html, Size: 799 bytes --]

Re: [PATCH 0/4] linux-yocto: build changes, 4.1 and libc-headers

[Richard Purdie]

On Thu, 2015-07-23 at 11:01 -0400, Bruce Ashfield wrote:
> On Thu, Jul 23, 2015 at 10:14 AM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Thu, 2015-07-23 at 09:10 -0400, Bruce Ashfield wrote:
> >> > a) qemux86/qemux86-64 are throwing warnings about errors in logs
> >> > b) perf has some weird make race
> >> > c) the qemuarm build looks like qemu either segfaulted or was killed
> >> > d) we're occasionally seeing 3.19 and 3.14 fetch failures:
> >> > https://autobuilder.yoctoproject.org/main/builders/nightly-intel-gpl/builds/394/steps/BuildImages/logs/stdio
> >> > https://autobuilder.yoctoproject.org/main/builders/nightly-oe-selftest/builds/87/steps/Running%20oe-selftest/logs/stdio
> >>
> >> I explained to Ross that dependent layers need to be updated if they set
> >> their own SRCREV_meta, since the repository is completely different. I
> >> sent a patch for meta-intel to deal with it. Obviously, I only built with the
> >> core BSPs during testing.
> >
> > That could explain d). I believe c) is from the ongoing selftest issues
> > we're having and I we've work in progress for that. That leaves a) and
> > b) to look into and fix.
> >
> >> I'll definitely need help on these, since I wasn't seeing any of these issues
> >> during my testing. Otherwise, this won't be done until September (vacation
> >> and other issues to deal with).
> >>
> >> But I'll see what I can do.
> >>
> >> I unfortunately can't merge anything but critical kernel patches until
> >> these go in as well
> >> since the meta branch is gone in the new scripts, which means SRCREV_meta is
> >> a constant conflict.
> >
> > a) should be straightfoward to fix, its just updating the message
> > whitelists (again).
> 
> I'm building a fresh qemux86-64 now, I'm going to see if the warnings
> are something
> I should fix, versus the whitelist. I should know tomorrow on this one.
> 
> >
> > b) I'm less sure about. Are there any race fixes in mainline for perf?
> > Hard to believe others don't see that one.
> 
> I'm searching as well. I'll share any findings ASAP.

I believe we have everything except b), the last build was much greener.
For b), what strikes me as odd:

https://autobuilder.yoctoproject.org/main/builders/nightly-world/builds/409/steps/BuildImages/logs/stdio

|   CC       /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-world/build/build/tmp/work/qemux86-poky-linux/perf/1.0-r9/perf-1.0/util/thread-stack.o
|   CC       /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-world/build/build/tmp/work/qemux86-poky-linux/perf/1.0-r9/perf-1.0/util/symbol-elf.o
|   CC       /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-world/build/build/tmp/work/qemux86-poky-linux/perf/1.0-r9/perf-1.0/util/probe-event.o
|   CC       /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-world/build/build/tmp/work/qemux86-pokymllib32-linux/lib32-perf/1.0-r9/perf-1.0/builtin-annotate.o
|   CC       /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-world/build/build/tmp/work/qemux86-poky-linux/perf/1.0-r9/perf-1.0/util/perf_regs.o
|   MKDIR    /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-world/build/build/tmp/work//lib32-perf/1.0-r9/perf-1.0/util/scripting-engines/
|   CC       /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-world/build/build/tmp/work/qemux86-pokymllib32-linux/lib32-perf/1.0-r9/perf-1.0/util/scripting-engines/trace-event-perl.o
|   CC       /home/pokybuild/yocto-autobuilder/yocto-worker/nightly-world/build/build/tmp/work/qemux86-poky-linux/perf/1.0-r9/perf-1.0/util/zlib.o

Why is it building in two different directories (qemux86-poky-linux and
qemux86-pokymllib32-linux) ?

I'd bet that would explain the errors and that it would only show up in
a multilib case (which is the default on the AB for this part of the
build).

Hopefully this gets us closer to figuring out what its doing...

Cheers,

Richard