* [PATCH 0/6] Misc changes relating to toolchain and image gen
@ 2013-11-13 1:23 Mark Hatle
2013-11-13 1:23 ` [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages Mark Hatle
` (11 more replies)
0 siblings, 12 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 1:23 UTC (permalink / raw)
To: openembedded-core
A few misc fixes relating to toolchain and image generation.
The first two in the set simply updated gcc to remove the -symlinks package
in favor of update-alternatives.
The next fixes an issue w/ the previous binutils patch and upgrades.
Next 2 update packagegroups to make revise and use the
packagegroup-core-buildessentials.
The final patch fixes an issue w/ the image generatation. If the
IMAGE_INSTALL contains a dependency instead of a package name, the system
was not able to translate it into a package. This now works properly.
Note, there is a case where an unresolvable set could be determined (rare,
but it is possible), in this case the user would need to add whatever is
required to fix it to the IMAGE_INSTALL. (This could happen when 2 or more
conflicting packages all have the same provide, but none of them are in
the IMAGE_INSTALL line. The system will attempt to determine which to
use, by checking if any of them are already in the IMAGE_INSTALL -- otherwise
it chooses the first entry. If something else implied by a packagegroup
chooses a different version, a conflict would be generated.)
The following changes since commit 616354f13732d13c17434d5b60b166f691c25761:
binutils: Add gnu-config-native to DEPENDS (2013-11-12 16:00:20 +0000)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib mhatle/misc1
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mhatle/misc1
Mark Hatle (6):
gcc: Use alternatives for the *-symlinks packages.
gcc: Drop *-symlinks
binutils: Ensure old -symlinks packages get removed
packagegroups: Remove toolchain *-symlinks packages
packagegroup-self-hosted: Use packagegroup-core-buildessentials
package_rpm: Allow translation of requirement to package name
meta/classes/package_rpm.bbclass | 79 +++++++++++++++-
.../packagegroup-core-buildessential.bb | 4 -
.../packagegroups/packagegroup-self-hosted.bb | 16 +---
meta/recipes-devtools/binutils/binutils.inc | 1 +
meta/recipes-devtools/gcc/gcc-target.inc | 103 +++++++++++++--------
5 files changed, 141 insertions(+), 62 deletions(-)
--
1.8.1.2.545.g2f19ada
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages.
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
@ 2013-11-13 1:23 ` Mark Hatle
2013-11-13 2:24 ` Otavio Salvador
2013-11-13 1:23 ` [PATCH 2/6] gcc: Drop *-symlinks Mark Hatle
` (10 subsequent siblings)
11 siblings, 1 reply; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 1:23 UTC (permalink / raw)
To: openembedded-core
The various gcc related symlinks should be provided as alternatives instead
of hard coded symlinks. This will permit multiple toolchains on a system.
Multiple toolchains could come from multilib configurations or alternative
open source or commerical sources.
Note, gccbug was skipped since it doesn't seem to be generated anymore.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/recipes-devtools/gcc/gcc-target.inc | 91 ++++++++++++++++++++------------
1 file changed, 58 insertions(+), 33 deletions(-)
diff --git a/meta/recipes-devtools/gcc/gcc-target.inc b/meta/recipes-devtools/gcc/gcc-target.inc
index e1179ac..ee41d3f 100644
--- a/meta/recipes-devtools/gcc/gcc-target.inc
+++ b/meta/recipes-devtools/gcc/gcc-target.inc
@@ -50,11 +50,8 @@ FILES_${PN}-plugin-dev = "\
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/plugin/gengtype \
${gcclibdir}/${TARGET_SYS}/${BINV}/plugin/gtype.state \
"
-FILES_${PN}-symlinks = "\
- ${bindir}/cc \
- ${bindir}/gcc \
- ${bindir}/gccbug \
-"
+FILES_${PN}-symlinks = ""
+ALLOW_EMPTY_${PN}-symlinks = "1"
FILES_${PN}-plugins = "\
${gcclibdir}/${TARGET_SYS}/${BINV}/plugin \
@@ -65,36 +62,33 @@ FILES_g77 = "\
${bindir}/${TARGET_PREFIX}g77 \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f771 \
"
-FILES_g77-symlinks = "\
- ${bindir}/g77 \
- ${bindir}/f77 \
-"
+FILES_g77-symlinks = ""
+ALLOW_EMPTY_g77-symlinks = "1"
+
FILES_gfortran = "\
${bindir}/${TARGET_PREFIX}gfortran \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f951 \
"
-FILES_gfortran-symlinks = "\
- ${bindir}/gfortran \
- ${bindir}/f95"
+FILES_gfortran-symlinks = ""
+ALLOW_EMPTY_gfortran-symlinks = "1"
FILES_cpp = "\
${bindir}/${TARGET_PREFIX}cpp \
${base_libdir}/cpp \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1"
-FILES_cpp-symlinks = "${bindir}/cpp"
+FILES_cpp-symlinks = ""
+ALLOW_EMPTY_cpp-symlinks = "1"
FILES_gcov = "${bindir}/${TARGET_PREFIX}gcov"
-FILES_gcov-symlinks = "${bindir}/gcov"
+FILES_gcov-symlinks = ""
+ALLOW_EMPTY_gconv-symlinks = "1"
FILES_g++ = "\
${bindir}/${TARGET_PREFIX}g++ \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1plus \
"
-FILES_g++-symlinks = "\
- ${bindir}/c++ \
- ${bindir}/g++ \
-"
-
+FILES_g++-symlinks = ""
+ALLOW_EMPTY_g++-symlinks = "1"
FILES_${PN}-doc = "\
${infodir} \
@@ -133,22 +127,53 @@ do_install () {
# Not sure why we end up with these but we don't want them...
rm -f ${TARGET_PREFIX}${TARGET_PREFIX}*
- # Symlinks so we can use these trivially on the target
- if [ -e ${TARGET_PREFIX}g77 ]; then
- ln -sf ${TARGET_PREFIX}g77 g77 || true
- ln -sf g77 f77 || true
- fi
- if [ -e ${TARGET_PREFIX}gfortran ]; then
- ln -sf ${TARGET_PREFIX}gfortran gfortran || true
- ln -sf gfortran f95 || true
- fi
- ln -sf ${TARGET_PREFIX}g++ g++
- ln -sf ${TARGET_PREFIX}gcc gcc
- ln -sf ${TARGET_PREFIX}cpp cpp
install -d ${D}${base_libdir}
ln -sf ${bindir}/${TARGET_PREFIX}cpp ${D}${base_libdir}/cpp
- ln -sf g++ c++
- ln -sf gcc cc
chown -R root:root ${D}
}
+
+inherit update-alternatives
+
+ALTERNATIVE_PRIORITY = "100"
+
+ALTERNATIVE_${PN}-symlinks = "cc gcc"
+ALTERNATIVE_g77-symlinks = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_gfortran-symlinks = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_cpp-symlinks = "cpp"
+ALTERNATIVE_gcov-symlinks = "gcov"
+ALTERNATIVE_g++-symlinks = "c++ g++"
+
+ALTERNATIVE_LINK_NAME[cc] = "${bindir}/cc"
+ALTERNATIVE_TARGET[cc] = "${bindir}/${TARGET_PREFIX}gcc"
+
+ALTERNATIVE_LINK_NAME[gcc] = "${bindir}/gcc"
+ALTERNATIVE_TARGET[gcc] = "${bindir}/${TARGET_PREFIX}gcc"
+
+# Not really generated any longer, avoid warnings...
+#ALTERNATIVE_LINK_NAME[gccbug] = "${bindir}/gccbug"
+#ALTERNATIVE_TARGET[gccbug] = "${bindir}/${TARGET_PREFIX}gccbug"
+
+ALTERNATIVE_LINK_NAME[g77] = "${bindir}/g77"
+ALTERNATIVE_TARGET[g77] = "${bindir}/${TARGET_PREFIX}g77"
+
+ALTERNATIVE_LINK_NAME[f77] = "${bindir}/f77"
+ALTERNATIVE_TARGET[f77] = "${bindir}/${TARGET_PREFIX}g77"
+
+ALTERNATIVE_LINK_NAME[gfortran] = "${bindir}/gfortran"
+ALTERNATIVE_TARGET[gfortran] = "${bindir}/${TARGET_PREFIX}gfortran"
+
+ALTERNATIVE_LINK_NAME[f95] = "${bindir}/f95"
+ALTERNATIVE_TARGET[f95] = "${bindir}/${TARGET_PREFIX}gfortran"
+
+ALTERNATIVE_LINK_NAME[cpp] = "${bindir}/cpp"
+ALTERNATIVE_TARGET[cpp] = "${bindir}/${TARGET_PREFIX}cpp"
+
+ALTERNATIVE_LINK_NAME[gcov] = "${bindir}/gcov"
+ALTERNATIVE_TARGET[gcov] = "${bindir}/${TARGET_PREFIX}gcov"
+
+ALTERNATIVE_LINK_NAME[c++] = "${bindir}/c++"
+ALTERNATIVE_TARGET[c++] = "${bindir}/${TARGET_PREFIX}g++"
+
+ALTERNATIVE_LINK_NAME[g++] = "${bindir}/g++"
+ALTERNATIVE_TARGET[g++] = "${bindir}/${TARGET_PREFIX}g++"
--
1.8.1.2.545.g2f19ada
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH 2/6] gcc: Drop *-symlinks
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
2013-11-13 1:23 ` [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages Mark Hatle
@ 2013-11-13 1:23 ` Mark Hatle
2013-11-13 1:23 ` [PATCH 3/6] binutils: Ensure old -symlinks packages get removed Mark Hatle
` (9 subsequent siblings)
11 siblings, 0 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 1:23 UTC (permalink / raw)
To: openembedded-core
The -symlinks packages don't really add any value if we're using
update-alternatives. Drop it, leaving an RPROVIDES in case anyone thought
they needed it still.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/recipes-devtools/gcc/gcc-target.inc | 48 ++++++++++++++++----------------
1 file changed, 24 insertions(+), 24 deletions(-)
diff --git a/meta/recipes-devtools/gcc/gcc-target.inc b/meta/recipes-devtools/gcc/gcc-target.inc
index ee41d3f..426be15 100644
--- a/meta/recipes-devtools/gcc/gcc-target.inc
+++ b/meta/recipes-devtools/gcc/gcc-target.inc
@@ -10,12 +10,12 @@ EXTRA_OECONF_PATHS = " \
ARCH_FLAGS_FOR_TARGET += "-isystem${STAGING_INCDIR} -I${B}/gcc/include/ "
PACKAGES = "\
- ${PN} ${PN}-plugins ${PN}-symlinks \
- g++ g++-symlinks \
- cpp cpp-symlinks \
- g77 g77-symlinks \
- gfortran gfortran-symlinks \
- gcov gcov-symlinks \
+ ${PN} ${PN}-plugins \
+ g++ \
+ cpp \
+ g77 \
+ gfortran \
+ gcov \
${PN}-plugin-dev \
${PN}-doc \
${PN}-dev \
@@ -36,6 +36,8 @@ FILES_${PN} = "\
${gcclibdir}/${TARGET_SYS}/${BINV}/include-fixed \
"
INSANE_SKIP_${PN} += "dev-so"
+RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"
FILES_${PN}-dbg += "\
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/.debug/ \
@@ -50,8 +52,6 @@ FILES_${PN}-plugin-dev = "\
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/plugin/gengtype \
${gcclibdir}/${TARGET_SYS}/${BINV}/plugin/gtype.state \
"
-FILES_${PN}-symlinks = ""
-ALLOW_EMPTY_${PN}-symlinks = "1"
FILES_${PN}-plugins = "\
${gcclibdir}/${TARGET_SYS}/${BINV}/plugin \
@@ -62,33 +62,33 @@ FILES_g77 = "\
${bindir}/${TARGET_PREFIX}g77 \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f771 \
"
-FILES_g77-symlinks = ""
-ALLOW_EMPTY_g77-symlinks = "1"
+RPROVIDES_g77 += "g77-symlinks"
+RREPLACES_g77 += "g77-symlinks"
FILES_gfortran = "\
${bindir}/${TARGET_PREFIX}gfortran \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f951 \
"
-FILES_gfortran-symlinks = ""
-ALLOW_EMPTY_gfortran-symlinks = "1"
+RPROVIDES_gfortran += "gfortran-symlinks"
+RREPLACES_gfortran += "gfortran-symlinks"
FILES_cpp = "\
${bindir}/${TARGET_PREFIX}cpp \
${base_libdir}/cpp \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1"
-FILES_cpp-symlinks = ""
-ALLOW_EMPTY_cpp-symlinks = "1"
+RPROVIDES_cpp += "cpp-symlinks"
+RREPLACES_cpp += "cpp-symlinks"
FILES_gcov = "${bindir}/${TARGET_PREFIX}gcov"
-FILES_gcov-symlinks = ""
-ALLOW_EMPTY_gconv-symlinks = "1"
+RPROVIDES_gcov += "gcov-symlinks"
+RREPLACES_gcov += "gcov-symlinks"
FILES_g++ = "\
${bindir}/${TARGET_PREFIX}g++ \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1plus \
"
-FILES_g++-symlinks = ""
-ALLOW_EMPTY_g++-symlinks = "1"
+RPROVIDES_g++ += "g++-symlinks"
+RREPLACES_g++ += "g++-symlinks"
FILES_${PN}-doc = "\
${infodir} \
@@ -137,12 +137,12 @@ inherit update-alternatives
ALTERNATIVE_PRIORITY = "100"
-ALTERNATIVE_${PN}-symlinks = "cc gcc"
-ALTERNATIVE_g77-symlinks = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}"
-ALTERNATIVE_gfortran-symlinks = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}"
-ALTERNATIVE_cpp-symlinks = "cpp"
-ALTERNATIVE_gcov-symlinks = "gcov"
-ALTERNATIVE_g++-symlinks = "c++ g++"
+ALTERNATIVE_${PN} = "cc gcc"
+ALTERNATIVE_g77 = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_gfortran = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_cpp = "cpp"
+ALTERNATIVE_gcov = "gcov"
+ALTERNATIVE_g++ = "c++ g++"
ALTERNATIVE_LINK_NAME[cc] = "${bindir}/cc"
ALTERNATIVE_TARGET[cc] = "${bindir}/${TARGET_PREFIX}gcc"
--
1.8.1.2.545.g2f19ada
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH 3/6] binutils: Ensure old -symlinks packages get removed
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
2013-11-13 1:23 ` [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages Mark Hatle
2013-11-13 1:23 ` [PATCH 2/6] gcc: Drop *-symlinks Mark Hatle
@ 2013-11-13 1:23 ` Mark Hatle
2013-11-13 8:12 ` Martin Jansa
2013-11-13 1:23 ` [PATCH 4/6] packagegroups: Remove toolchain *-symlinks packages Mark Hatle
` (8 subsequent siblings)
11 siblings, 1 reply; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 1:23 UTC (permalink / raw)
To: openembedded-core
When the -symlinks package was removed and an RPROVIDES was added, nothing
was put in to cause the removal of the older package in an upgrade.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/recipes-devtools/binutils/binutils.inc | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
index 17c66bc..ea2383e 100644
--- a/meta/recipes-devtools/binutils/binutils.inc
+++ b/meta/recipes-devtools/binutils/binutils.inc
@@ -21,6 +21,7 @@ FILES_${PN} = " \
${prefix}/${TARGET_SYS}/bin/*"
RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"
FILES_${PN}-dev = " \
${includedir} \
--
1.8.1.2.545.g2f19ada
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH 4/6] packagegroups: Remove toolchain *-symlinks packages
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
` (2 preceding siblings ...)
2013-11-13 1:23 ` [PATCH 3/6] binutils: Ensure old -symlinks packages get removed Mark Hatle
@ 2013-11-13 1:23 ` Mark Hatle
2013-11-13 1:23 ` [PATCH 5/6] packagegroup-self-hosted: Use packagegroup-core-buildessentials Mark Hatle
` (7 subsequent siblings)
11 siblings, 0 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 1:23 UTC (permalink / raw)
To: openembedded-core
We are no longer producing the *-symlinks packages, so remove the dependencies
to those items.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/recipes-core/packagegroups/packagegroup-core-buildessential.bb | 4 ----
meta/recipes-core/packagegroups/packagegroup-self-hosted.bb | 4 ----
2 files changed, 8 deletions(-)
diff --git a/meta/recipes-core/packagegroups/packagegroup-core-buildessential.bb b/meta/recipes-core/packagegroups/packagegroup-core-buildessential.bb
index 74ed247..e21f8c1 100644
--- a/meta/recipes-core/packagegroups/packagegroup-core-buildessential.bb
+++ b/meta/recipes-core/packagegroups/packagegroup-core-buildessential.bb
@@ -12,13 +12,9 @@ RDEPENDS_packagegroup-core-buildessential = "\
autoconf \
automake \
binutils \
- binutils-symlinks \
cpp \
- cpp-symlinks \
gcc \
- gcc-symlinks \
g++ \
- g++-symlinks \
gettext \
make \
libstdc++ \
diff --git a/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb b/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb
index 108e678..8e43201 100644
--- a/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb
+++ b/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb
@@ -70,20 +70,16 @@ RDEPENDS_packagegroup-self-hosted-sdk = "\
autoconf \
automake \
binutils \
- binutils-symlinks \
ccache \
coreutils \
cpp \
- cpp-symlinks \
distcc \
eglibc-utils \
eglibc-gconv-ibm850 \
file \
findutils \
g++ \
- g++-symlinks \
gcc \
- gcc-symlinks \
intltool \
ldd \
less \
--
1.8.1.2.545.g2f19ada
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH 5/6] packagegroup-self-hosted: Use packagegroup-core-buildessentials
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
` (3 preceding siblings ...)
2013-11-13 1:23 ` [PATCH 4/6] packagegroups: Remove toolchain *-symlinks packages Mark Hatle
@ 2013-11-13 1:23 ` Mark Hatle
2013-11-13 1:23 ` [PATCH 6/6] package_rpm: Allow translation of requirement to package name Mark Hatle
` (6 subsequent siblings)
11 siblings, 0 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 1:23 UTC (permalink / raw)
To: openembedded-core
Instead of manually specifying build dependencies, use the
packagegroup-core-buildessentials. This ensures there is only one place
that needs to be modified when toolchain items and dependencies change.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/recipes-core/packagegroups/packagegroup-self-hosted.bb | 12 +-----------
1 file changed, 1 insertion(+), 11 deletions(-)
diff --git a/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb b/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb
index 8e43201..e91a8d9 100644
--- a/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb
+++ b/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb
@@ -67,33 +67,23 @@ RRECOMMENDS_packagegroup-self-hosted-host-tools = "\
# eglibc-utils: for rpcgen
RDEPENDS_packagegroup-self-hosted-sdk = "\
- autoconf \
- automake \
- binutils \
ccache \
coreutils \
- cpp \
distcc \
eglibc-utils \
eglibc-gconv-ibm850 \
file \
findutils \
- g++ \
- gcc \
intltool \
ldd \
less \
libssp \
libssp-dev \
libssp-staticdev \
- libstdc++ \
- libstdc++-dev \
- libtool \
- make \
mktemp \
+ packagegroup-core-buildessential \
perl-module-re \
perl-module-text-wrap \
- pkgconfig \
quilt \
sed \
"
--
1.8.1.2.545.g2f19ada
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH 6/6] package_rpm: Allow translation of requirement to package name
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
` (4 preceding siblings ...)
2013-11-13 1:23 ` [PATCH 5/6] packagegroup-self-hosted: Use packagegroup-core-buildessentials Mark Hatle
@ 2013-11-13 1:23 ` Mark Hatle
2013-11-13 1:33 ` [PATCH 0/6] Misc changes relating to toolchain and image gen Otavio Salvador
` (5 subsequent siblings)
11 siblings, 0 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 1:23 UTC (permalink / raw)
To: openembedded-core
In the translate oe to smart function, we only translated package names.
However, it's allowed that people can put in a dependency name in the
IMAGE_INSTALL. So on a failure to translate a package name, we fall back
and attempt to resolve based on a package's provide.
Note: it may be possible to generate an unsolvable install solution. If the
dependency is provided by one or more things that conflict with something else
set to be installed. We can't determine this until smart is run.
If this occurs, file a bug and we'll have to identify a way to deal with the
RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with
actual package names.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++--
1 file changed, 75 insertions(+), 4 deletions(-)
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 31265d9..64da1f5 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -166,6 +166,7 @@ translate_oe_to_smart() {
[ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query
pkgs_to_install=""
+ not_found=""
for pkg in "$@" ; do
new_pkg="$pkg"
if [ -z "$sdk_mode" ]; then
@@ -184,7 +185,6 @@ translate_oe_to_smart() {
fi
subst=${pkg#${mlib}-}
if [ "$subst" != "$pkg" ]; then
- feeds=$@
while [ -n "$1" ]; do
arch="$1"
arch=`echo "$arch" | tr - _`
@@ -197,6 +197,76 @@ translate_oe_to_smart() {
done
if [ "$pkg" = "$new_pkg" ]; then
# Failed to translate, package not found!
+ not_found="$not_found $pkg"
+ continue
+ fi
+ fi
+ done
+ fi
+ # Apparently not a multilib package...
+ if [ "$pkg" = "$new_pkg" ]; then
+ default_archs_fixed=`echo "$default_archs" | tr - _`
+ for arch in $default_archs_fixed ; do
+ if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
+ new_pkg="$pkg@$arch"
+ # First found is best match
+ break
+ fi
+ done
+ if [ "$pkg" = "$new_pkg" ]; then
+ # Failed to translate, package not found!
+ not_found="$not_found $pkg"
+ continue
+ fi
+ fi
+ #echo "$pkg -> $new_pkg" >&2
+ pkgs_to_install="${pkgs_to_install} ${new_pkg}"
+ done
+
+ # Parse the not_found items and see if they were dependencies (RPROVIDES)
+ # Follow the parsing example above...
+ for pkg in $not_found ; do
+ new_pkg="$pkg"
+ smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query
+ grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+ prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :`
+ if [ -n "$prov" ]; then
+ # Nothing to do, already in the list
+ #echo "Skipping $pkg -> $prov, already in install set" >&2
+ continue
+ fi
+ if [ -z "$sdk_mode" ]; then
+ for i in ${MULTILIB_PREFIX_LIST} ; do
+ old_IFS="$IFS"
+ IFS=":"
+ set $i
+ IFS="$old_IFS"
+ mlib="$1"
+ shift
+ if [ "$mlib" = "default" ]; then
+ if [ -z "$default_archs" ]; then
+ default_archs=$@
+ fi
+ continue
+ fi
+ subst=${pkg#${mlib}-}
+ if [ "$subst" != "$pkg" ]; then
+ feeds=$@
+ smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query
+ grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+ while [ -n "$1" ]; do
+ arch="$1"
+ arch=`echo "$arch" | tr - _`
+ shift
+ # Select first found, we don't know if one is better then another...
+ prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+ if [ -n "$prov" ]; then
+ new_pkg=$prov
+ break
+ fi
+ done
+ if [ "$pkg" = "$new_pkg" ]; then
+ # Failed to translate, package not found!
echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2
if [ "$attemptonly" = "Error" ]; then
exit 1
@@ -210,9 +280,10 @@ translate_oe_to_smart() {
if [ "$pkg" = "$new_pkg" ]; then
default_archs_fixed=`echo "$default_archs" | tr - _`
for arch in $default_archs_fixed ; do
- if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
- new_pkg="$pkg@$arch"
- # First found is best match
+ # Select first found, we don't know if one is better then another...
+ prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+ if [ -n "$prov" ]; then
+ new_pkg=$prov
break
fi
done
--
1.8.1.2.545.g2f19ada
^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PATCH 0/6] Misc changes relating to toolchain and image gen
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
` (5 preceding siblings ...)
2013-11-13 1:23 ` [PATCH 6/6] package_rpm: Allow translation of requirement to package name Mark Hatle
@ 2013-11-13 1:33 ` Otavio Salvador
2013-11-13 2:01 ` Mark Hatle
2013-11-13 1:59 ` [PATCH 5/6 v2] packagegroup-self-hosted: Use packagegroup-core-buildessential Mark Hatle
` (4 subsequent siblings)
11 siblings, 1 reply; 27+ messages in thread
From: Otavio Salvador @ 2013-11-13 1:33 UTC (permalink / raw)
To: Mark Hatle; +Cc: Patches and discussions about the oe-core layer
On Tue, Nov 12, 2013 at 11:23 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
> A few misc fixes relating to toolchain and image generation.
>
> The first two in the set simply updated gcc to remove the -symlinks package
> in favor of update-alternatives.
>
> The next fixes an issue w/ the previous binutils patch and upgrades.
>
> Next 2 update packagegroups to make revise and use the
> packagegroup-core-buildessentials.
>
> The final patch fixes an issue w/ the image generatation. If the
> IMAGE_INSTALL contains a dependency instead of a package name, the system
> was not able to translate it into a package. This now works properly.
>
> Note, there is a case where an unresolvable set could be determined (rare,
> but it is possible), in this case the user would need to add whatever is
> required to fix it to the IMAGE_INSTALL. (This could happen when 2 or more
> conflicting packages all have the same provide, but none of them are in
> the IMAGE_INSTALL line. The system will attempt to determine which to
> use, by checking if any of them are already in the IMAGE_INSTALL -- otherwise
> it chooses the first entry. If something else implied by a packagegroup
> chooses a different version, a conflict would be generated.)
Which package backends has been tested with those changes?
--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 5/6 v2] packagegroup-self-hosted: Use packagegroup-core-buildessential
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
` (6 preceding siblings ...)
2013-11-13 1:33 ` [PATCH 0/6] Misc changes relating to toolchain and image gen Otavio Salvador
@ 2013-11-13 1:59 ` Mark Hatle
2013-11-13 1:59 ` [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name Mark Hatle
` (3 subsequent siblings)
11 siblings, 0 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 1:59 UTC (permalink / raw)
To: openembedded-core
Instead of manually specifying build dependencies, use the
packagegroup-core-buildessential. This ensures there is only one place
that needs to be modified when toolchain items and dependencies change.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/recipes-core/packagegroups/packagegroup-self-hosted.bb | 12 +-----------
1 file changed, 1 insertion(+), 11 deletions(-)
diff --git a/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb b/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb
index 8e43201..e91a8d9 100644
--- a/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb
+++ b/meta/recipes-core/packagegroups/packagegroup-self-hosted.bb
@@ -67,33 +67,23 @@ RRECOMMENDS_packagegroup-self-hosted-host-tools = "\
# eglibc-utils: for rpcgen
RDEPENDS_packagegroup-self-hosted-sdk = "\
- autoconf \
- automake \
- binutils \
ccache \
coreutils \
- cpp \
distcc \
eglibc-utils \
eglibc-gconv-ibm850 \
file \
findutils \
- g++ \
- gcc \
intltool \
ldd \
less \
libssp \
libssp-dev \
libssp-staticdev \
- libstdc++ \
- libstdc++-dev \
- libtool \
- make \
mktemp \
+ packagegroup-core-buildessential \
perl-module-re \
perl-module-text-wrap \
- pkgconfig \
quilt \
sed \
"
--
1.8.1.2.545.g2f19ada
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
` (7 preceding siblings ...)
2013-11-13 1:59 ` [PATCH 5/6 v2] packagegroup-self-hosted: Use packagegroup-core-buildessential Mark Hatle
@ 2013-11-13 1:59 ` Mark Hatle
2013-11-13 10:33 ` Paul Eggleton
2013-11-15 14:02 ` Richard Purdie
2013-11-13 14:19 ` [PATCH 2/6 v3] gcc: Drop *-symlinks Mark Hatle
` (2 subsequent siblings)
11 siblings, 2 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 1:59 UTC (permalink / raw)
To: openembedded-core
In the translate oe to smart function, we only translated package names.
However, it's allowed that people can put in a dependency name in the
IMAGE_INSTALL. So on a failure to translate a package name, we fall back
and attempt to resolve based on a package's provide.
Note: it may be possible to generate an unsolvable install solution. If the
dependency is provided by one or more things that conflict with something else
set to be installed. We can't determine this until smart is run.
If this occurs, file a bug and we'll have to identify a way to deal with the
RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with
actual package names.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++--
1 file changed, 75 insertions(+), 4 deletions(-)
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index 31265d9..9fe0e6c 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -166,6 +166,7 @@ translate_oe_to_smart() {
[ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query
pkgs_to_install=""
+ not_found=""
for pkg in "$@" ; do
new_pkg="$pkg"
if [ -z "$sdk_mode" ]; then
@@ -184,7 +185,6 @@ translate_oe_to_smart() {
fi
subst=${pkg#${mlib}-}
if [ "$subst" != "$pkg" ]; then
- feeds=$@
while [ -n "$1" ]; do
arch="$1"
arch=`echo "$arch" | tr - _`
@@ -197,6 +197,76 @@ translate_oe_to_smart() {
done
if [ "$pkg" = "$new_pkg" ]; then
# Failed to translate, package not found!
+ not_found="$not_found $pkg"
+ continue
+ fi
+ fi
+ done
+ fi
+ # Apparently not a multilib package...
+ if [ "$pkg" = "$new_pkg" ]; then
+ default_archs_fixed=`echo "$default_archs" | tr - _`
+ for arch in $default_archs_fixed ; do
+ if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
+ new_pkg="$pkg@$arch"
+ # First found is best match
+ break
+ fi
+ done
+ if [ "$pkg" = "$new_pkg" ]; then
+ # Failed to translate, package not found!
+ not_found="$not_found $pkg"
+ continue
+ fi
+ fi
+ #echo "$pkg -> $new_pkg" >&2
+ pkgs_to_install="${pkgs_to_install} ${new_pkg}"
+ done
+
+ # Parse the not_found items and see if they were dependencies (RPROVIDES)
+ # Follow the parsing example above...
+ for pkg in $not_found ; do
+ new_pkg="$pkg"
+ smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query
+ grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+ prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :`
+ if [ -n "$prov" ]; then
+ # Nothing to do, already in the list
+ #echo "Skipping $pkg -> $prov, already in install set" >&2
+ continue
+ fi
+ if [ -z "$sdk_mode" ]; then
+ for i in ${MULTILIB_PREFIX_LIST} ; do
+ old_IFS="$IFS"
+ IFS=":"
+ set $i
+ IFS="$old_IFS"
+ mlib="$1"
+ shift
+ if [ "$mlib" = "default" ]; then
+ if [ -z "$default_archs" ]; then
+ default_archs=$@
+ fi
+ continue
+ fi
+ subst=${pkg#${mlib}-}
+ if [ "$subst" != "$pkg" ]; then
+ feeds=$@
+ smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query
+ grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
+ while [ -n "$1" ]; do
+ arch="$1"
+ arch=`echo "$arch" | tr - _`
+ shift
+ # Select first found, we don't know if one is better then another...
+ prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+ if [ -n "$prov" ]; then
+ new_pkg=$prov
+ break
+ fi
+ done
+ if [ "$pkg" = "$new_pkg" ]; then
+ # Failed to translate, package not found!
echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2
if [ "$attemptonly" = "Error" ]; then
exit 1
@@ -210,9 +280,10 @@ translate_oe_to_smart() {
if [ "$pkg" = "$new_pkg" ]; then
default_archs_fixed=`echo "$default_archs" | tr - _`
for arch in $default_archs_fixed ; do
- if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
- new_pkg="$pkg@$arch"
- # First found is best match
+ # Select first found, we don't know if one is better then another...
+ prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
+ if [ -n "$prov" ]; then
+ new_pkg=$prov
break
fi
done
--
1.8.1.2.545.g2f19ada
^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PATCH 0/6] Misc changes relating to toolchain and image gen
2013-11-13 1:33 ` [PATCH 0/6] Misc changes relating to toolchain and image gen Otavio Salvador
@ 2013-11-13 2:01 ` Mark Hatle
2013-11-13 3:42 ` Mark Hatle
0 siblings, 1 reply; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 2:01 UTC (permalink / raw)
To: Otavio Salvador; +Cc: Patches and discussions about the oe-core layer
On 11/12/13, 7:33 PM, Otavio Salvador wrote:
> On Tue, Nov 12, 2013 at 11:23 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
>> A few misc fixes relating to toolchain and image generation.
>>
>> The first two in the set simply updated gcc to remove the -symlinks package
>> in favor of update-alternatives.
>>
>> The next fixes an issue w/ the previous binutils patch and upgrades.
>>
>> Next 2 update packagegroups to make revise and use the
>> packagegroup-core-buildessentials.
>>
>> The final patch fixes an issue w/ the image generatation. If the
>> IMAGE_INSTALL contains a dependency instead of a package name, the system
>> was not able to translate it into a package. This now works properly.
>>
>> Note, there is a case where an unresolvable set could be determined (rare,
>> but it is possible), in this case the user would need to add whatever is
>> required to fix it to the IMAGE_INSTALL. (This could happen when 2 or more
>> conflicting packages all have the same provide, but none of them are in
>> the IMAGE_INSTALL line. The system will attempt to determine which to
>> use, by checking if any of them are already in the IMAGE_INSTALL -- otherwise
>> it chooses the first entry. If something else implied by a packagegroup
>> chooses a different version, a conflict would be generated.)
>
> Which package backends has been tested with those changes?
>
6/6 only affects RPM. So only RPM back end.
The other changes are generic and follow the existing models. However, I only
tested the RPM backend.
(Note v2 of 5/6 and 6/6 has been sent. I accidentally sent a version prior to
two minor revisions.)
--Mark
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages.
2013-11-13 1:23 ` [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages Mark Hatle
@ 2013-11-13 2:24 ` Otavio Salvador
2013-11-13 2:27 ` Mark Hatle
0 siblings, 1 reply; 27+ messages in thread
From: Otavio Salvador @ 2013-11-13 2:24 UTC (permalink / raw)
To: Mark Hatle; +Cc: Patches and discussions about the oe-core layer
On Tue, Nov 12, 2013 at 11:23 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
> The various gcc related symlinks should be provided as alternatives instead
> of hard coded symlinks. This will permit multiple toolchains on a system.
>
> Multiple toolchains could come from multilib configurations or alternative
> open source or commerical sources.
>
> Note, gccbug was skipped since it doesn't seem to be generated anymore.
>
> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Wouldn't be better to squash patches 1 and 2 so it makes a real 'logic
change'? You add code to remove in patch 2, I think the end patch
would be easier to review, no?
--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages.
2013-11-13 2:24 ` Otavio Salvador
@ 2013-11-13 2:27 ` Mark Hatle
2013-11-13 2:28 ` Otavio Salvador
0 siblings, 1 reply; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 2:27 UTC (permalink / raw)
To: Otavio Salvador; +Cc: Patches and discussions about the oe-core layer
On 11/12/13, 8:24 PM, Otavio Salvador wrote:
> On Tue, Nov 12, 2013 at 11:23 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
>> The various gcc related symlinks should be provided as alternatives instead
>> of hard coded symlinks. This will permit multiple toolchains on a system.
>>
>> Multiple toolchains could come from multilib configurations or alternative
>> open source or commerical sources.
>>
>> Note, gccbug was skipped since it doesn't seem to be generated anymore.
>>
>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>
> Wouldn't be better to squash patches 1 and 2 so it makes a real 'logic
> change'? You add code to remove in patch 2, I think the end patch
> would be easier to review, no?
>
It was done this way to match how binutils was implemented. These are actually
two logically separate patches. The first switches from hard coded symlinks to
using update-alternatives. The second patch says that a separate -symlinks
package is no longer needed.
See binutils:
1395aefcaeac94dd0e6ed3a718b7e58dd43b355e
24093e26f246f222c385dc37a2f8cf8b0f183175
(The second of the patches can be reverted -- if ever needed -- and the update
alternatives functionality will still work properly.)
--Mark
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages.
2013-11-13 2:27 ` Mark Hatle
@ 2013-11-13 2:28 ` Otavio Salvador
0 siblings, 0 replies; 27+ messages in thread
From: Otavio Salvador @ 2013-11-13 2:28 UTC (permalink / raw)
To: Mark Hatle; +Cc: Patches and discussions about the oe-core layer
On Wed, Nov 13, 2013 at 12:27 AM, Mark Hatle <mark.hatle@windriver.com> wrote:
> On 11/12/13, 8:24 PM, Otavio Salvador wrote:
>>
>> On Tue, Nov 12, 2013 at 11:23 PM, Mark Hatle <mark.hatle@windriver.com>
>> wrote:
>>>
>>> The various gcc related symlinks should be provided as alternatives
>>> instead
>>> of hard coded symlinks. This will permit multiple toolchains on a
>>> system.
>>>
>>> Multiple toolchains could come from multilib configurations or
>>> alternative
>>> open source or commerical sources.
>>>
>>> Note, gccbug was skipped since it doesn't seem to be generated anymore.
>>>
>>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>>
>>
>> Wouldn't be better to squash patches 1 and 2 so it makes a real 'logic
>> change'? You add code to remove in patch 2, I think the end patch
>> would be easier to review, no?
>>
>
> It was done this way to match how binutils was implemented. These are
> actually two logically separate patches. The first switches from hard coded
> symlinks to using update-alternatives. The second patch says that a
> separate -symlinks package is no longer needed.
>
> See binutils:
>
> 1395aefcaeac94dd0e6ed3a718b7e58dd43b355e
> 24093e26f246f222c385dc37a2f8cf8b0f183175
>
> (The second of the patches can be reverted -- if ever needed -- and the
> update alternatives functionality will still work properly.)
Ok; you got a point :-)
Agreed.
--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 0/6] Misc changes relating to toolchain and image gen
2013-11-13 2:01 ` Mark Hatle
@ 2013-11-13 3:42 ` Mark Hatle
0 siblings, 0 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 3:42 UTC (permalink / raw)
To: Otavio Salvador; +Cc: Patches and discussions about the oe-core layer
On 11/12/13, 8:01 PM, Mark Hatle wrote:
> On 11/12/13, 7:33 PM, Otavio Salvador wrote:
>> On Tue, Nov 12, 2013 at 11:23 PM, Mark Hatle <mark.hatle@windriver.com> wrote:
>>> A few misc fixes relating to toolchain and image generation.
>>>
>>> The first two in the set simply updated gcc to remove the -symlinks package
>>> in favor of update-alternatives.
>>>
>>> The next fixes an issue w/ the previous binutils patch and upgrades.
>>>
>>> Next 2 update packagegroups to make revise and use the
>>> packagegroup-core-buildessentials.
>>>
>>> The final patch fixes an issue w/ the image generatation. If the
>>> IMAGE_INSTALL contains a dependency instead of a package name, the system
>>> was not able to translate it into a package. This now works properly.
>>>
>>> Note, there is a case where an unresolvable set could be determined (rare,
>>> but it is possible), in this case the user would need to add whatever is
>>> required to fix it to the IMAGE_INSTALL. (This could happen when 2 or more
>>> conflicting packages all have the same provide, but none of them are in
>>> the IMAGE_INSTALL line. The system will attempt to determine which to
>>> use, by checking if any of them are already in the IMAGE_INSTALL -- otherwise
>>> it chooses the first entry. If something else implied by a packagegroup
>>> chooses a different version, a conflict would be generated.)
>>
>> Which package backends has been tested with those changes?
>>
>
> 6/6 only affects RPM. So only RPM back end.
>
> The other changes are generic and follow the existing models. However, I only
> tested the RPM backend.
BTW I just verified that the 1-5/6 work properly with an opkg backend as well.
--Mark
> (Note v2 of 5/6 and 6/6 has been sent. I accidentally sent a version prior to
> two minor revisions.)
>
> --Mark
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed
2013-11-13 1:23 ` [PATCH 3/6] binutils: Ensure old -symlinks packages get removed Mark Hatle
@ 2013-11-13 8:12 ` Martin Jansa
2013-11-13 11:14 ` Phil Blundell
0 siblings, 1 reply; 27+ messages in thread
From: Martin Jansa @ 2013-11-13 8:12 UTC (permalink / raw)
To: Mark Hatle; +Cc: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 1219 bytes --]
On Tue, Nov 12, 2013 at 07:23:22PM -0600, Mark Hatle wrote:
> When the -symlinks package was removed and an RPROVIDES was added, nothing
> was put in to cause the removal of the older package in an upgrade.
>
> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> ---
> meta/recipes-devtools/binutils/binutils.inc | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
> index 17c66bc..ea2383e 100644
> --- a/meta/recipes-devtools/binutils/binutils.inc
> +++ b/meta/recipes-devtools/binutils/binutils.inc
> @@ -21,6 +21,7 @@ FILES_${PN} = " \
> ${prefix}/${TARGET_SYS}/bin/*"
>
> RPROVIDES_${PN} += "${PN}-symlinks"
> +RREPLACES_${PN} += "${PN}-symlinks"
IIRC for opkg backend you need also RCONFLICTS for opkg to really remove
old ${PN}-symlinks.
>
> FILES_${PN}-dev = " \
> ${includedir} \
> --
> 1.8.1.2.545.g2f19ada
>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name
2013-11-13 1:59 ` [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name Mark Hatle
@ 2013-11-13 10:33 ` Paul Eggleton
2013-11-13 12:31 ` Otavio Salvador
2013-11-13 13:12 ` Mark Hatle
2013-11-15 14:02 ` Richard Purdie
1 sibling, 2 replies; 27+ messages in thread
From: Paul Eggleton @ 2013-11-13 10:33 UTC (permalink / raw)
To: Mark Hatle; +Cc: openembedded-core
Hi Mark,
On Tuesday 12 November 2013 19:59:20 Mark Hatle wrote:
> In the translate oe to smart function, we only translated package names.
> However, it's allowed that people can put in a dependency name in the
> IMAGE_INSTALL. So on a failure to translate a package name, we fall back
> and attempt to resolve based on a package's provide.
>
> Note: it may be possible to generate an unsolvable install solution. If the
> dependency is provided by one or more things that conflict with something
> else set to be installed. We can't determine this until smart is run.
>
> If this occurs, file a bug and we'll have to identify a way to deal with the
> RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES
> with actual package names.
>
> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> ---
> meta/classes/package_rpm.bbclass | 79
> ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+),
> 4 deletions(-)
>
> diff --git a/meta/classes/package_rpm.bbclass
> b/meta/classes/package_rpm.bbclass index 31265d9..9fe0e6c 100644
> --- a/meta/classes/package_rpm.bbclass
> +++ b/meta/classes/package_rpm.bbclass
> @@ -166,6 +166,7 @@ translate_oe_to_smart() {
> [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart
> --data-dir=${target_rootfs}/var/lib/smart query --output
> ${target_rootfs}/install/tmp/fullpkglist.query
>
> pkgs_to_install=""
> + not_found=""
> for pkg in "$@" ; do
> new_pkg="$pkg"
> if [ -z "$sdk_mode" ]; then
> @@ -184,7 +185,6 @@ translate_oe_to_smart() {
> fi
> subst=${pkg#${mlib}-}
> if [ "$subst" != "$pkg" ]; then
> - feeds=$@
> while [ -n "$1" ]; do
> arch="$1"
> arch=`echo "$arch" | tr - _`
> @@ -197,6 +197,76 @@ translate_oe_to_smart() {
> done
> if [ "$pkg" = "$new_pkg" ]; then
> # Failed to translate, package not found!
> + not_found="$not_found $pkg"
> + continue
> + fi
> + fi
> + done
> + fi
> + # Apparently not a multilib package...
> + if [ "$pkg" = "$new_pkg" ]; then
> + default_archs_fixed=`echo "$default_archs" | tr - _`
> + for arch in $default_archs_fixed ; do
> + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'
> ${target_rootfs}/install/tmp/fullpkglist.query ; then
> + new_pkg="$pkg@$arch"
> + # First found is best match
> + break
> + fi
> + done
> + if [ "$pkg" = "$new_pkg" ]; then
> + # Failed to translate, package not found!
> + not_found="$not_found $pkg"
> + continue
> + fi
> + fi
> + #echo "$pkg -> $new_pkg" >&2
> + pkgs_to_install="${pkgs_to_install} ${new_pkg}"
> + done
> +
> + # Parse the not_found items and see if they were dependencies (RPROVIDES)
> + # Follow the parsing example above...
> + for pkg in $not_found ; do
> + new_pkg="$pkg"
> + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg
> --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@
> ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e
> 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
> ${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo
> $pkgs_to_install | xargs -n 1 echo | grep -f
> ${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov"
];
> then
> + # Nothing to do, already in the list
> + #echo "Skipping $pkg -> $prov, already in install set" >&2
> + continue
> + fi
> + if [ -z "$sdk_mode" ]; then
> + for i in ${MULTILIB_PREFIX_LIST} ; do
> + old_IFS="$IFS"
> + IFS=":"
> + set $i
> + IFS="$old_IFS"
> + mlib="$1"
> + shift
> + if [ "$mlib" = "default" ]; then
> + if [ -z "$default_archs" ]; then
> + default_archs=$@
> + fi
> + continue
> + fi
> + subst=${pkg#${mlib}-}
> + if [ "$subst" != "$pkg" ]; then
> + feeds=$@
> + smart --data-dir=${target_rootfs}/var/lib/smart query
> --provides=$subst --output ${target_rootfs}/install/tmp/provide.query
> + grep '^[^@ ]*@[^@]*$'
${target_rootfs}/install/tmp/provide.query |
> sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
> ${target_rootfs}/install/tmp/provide.query.list || : + while [
-n "$1"
> ]; do
> + arch="$1"
> + arch=`echo "$arch" | tr - _`
> + shift
> + # Select first found, we don't know if one is better then
another...
> + prov=`grep '^[^@ ]*@'$arch'$'
> ${target_rootfs}/install/tmp/provide.query.list | head -n 1` +
if [ -n
> "$prov" ]; then
> + new_pkg=$prov
> + break
> + fi
> + done
> + if [ "$pkg" = "$new_pkg" ]; then
> + # Failed to translate, package not found!
> echo "$attemptonly: $pkg not found in the $mlib feeds
($feeds)." >&2
> if [ "$attemptonly" = "Error" ]; then
> exit 1
> @@ -210,9 +280,10 @@ translate_oe_to_smart() {
> if [ "$pkg" = "$new_pkg" ]; then
> default_archs_fixed=`echo "$default_archs" | tr - _`
> for arch in $default_archs_fixed ; do
> - if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'
> ${target_rootfs}/install/tmp/fullpkglist.query ; then
> - new_pkg="$pkg@$arch"
> - # First found is best match
> + # Select first found, we don't know if one is better then
another...
> + prov=`grep '^[^@ ]*@'$arch'$'
> ${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if
[ -n
> "$prov" ]; then
> + new_pkg=$prov
> break
> fi
> done
Sigh... this code is getting ridiculously complicated. Shouldn't Smart be
doing more of the heavy lifting here?
Cheers,
Paul
--
Paul Eggleton
Intel Open Source Technology Centre
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed
2013-11-13 8:12 ` Martin Jansa
@ 2013-11-13 11:14 ` Phil Blundell
2013-11-13 13:39 ` Mark Hatle
0 siblings, 1 reply; 27+ messages in thread
From: Phil Blundell @ 2013-11-13 11:14 UTC (permalink / raw)
To: Martin Jansa; +Cc: openembedded-core
On Wed, 2013-11-13 at 09:12 +0100, Martin Jansa wrote:
> On Tue, Nov 12, 2013 at 07:23:22PM -0600, Mark Hatle wrote:
> > When the -symlinks package was removed and an RPROVIDES was added, nothing
> > was put in to cause the removal of the older package in an upgrade.
> >
> > Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> > ---
> > meta/recipes-devtools/binutils/binutils.inc | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
> > index 17c66bc..ea2383e 100644
> > --- a/meta/recipes-devtools/binutils/binutils.inc
> > +++ b/meta/recipes-devtools/binutils/binutils.inc
> > @@ -21,6 +21,7 @@ FILES_${PN} = " \
> > ${prefix}/${TARGET_SYS}/bin/*"
> >
> > RPROVIDES_${PN} += "${PN}-symlinks"
> > +RREPLACES_${PN} += "${PN}-symlinks"
>
> IIRC for opkg backend you need also RCONFLICTS for opkg to really remove
> old ${PN}-symlinks.
Also for dpkg. If you have Replaces: without Conflicts: then this just
means that the replacing package is allowed to overwrite files in the
replaced one, not that the replaced one will actually be uninstalled.
p.
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name
2013-11-13 10:33 ` Paul Eggleton
@ 2013-11-13 12:31 ` Otavio Salvador
2013-11-13 13:12 ` Mark Hatle
1 sibling, 0 replies; 27+ messages in thread
From: Otavio Salvador @ 2013-11-13 12:31 UTC (permalink / raw)
To: Paul Eggleton; +Cc: Patches and discussions about the oe-core layer
On Wed, Nov 13, 2013 at 8:33 AM, Paul Eggleton
<paul.eggleton@linux.intel.com> wrote:
> On Tuesday 12 November 2013 19:59:20 Mark Hatle wrote:
>> In the translate oe to smart function, we only translated package names.
>> However, it's allowed that people can put in a dependency name in the
>> IMAGE_INSTALL. So on a failure to translate a package name, we fall back
>> and attempt to resolve based on a package's provide.
>>
>> Note: it may be possible to generate an unsolvable install solution. If the
>> dependency is provided by one or more things that conflict with something
>> else set to be installed. We can't determine this until smart is run.
>>
>> If this occurs, file a bug and we'll have to identify a way to deal with the
>> RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES
>> with actual package names.
>>
>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>> ---
>> meta/classes/package_rpm.bbclass | 79
>> ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+),
>> 4 deletions(-)
>>
>> diff --git a/meta/classes/package_rpm.bbclass
>> b/meta/classes/package_rpm.bbclass index 31265d9..9fe0e6c 100644
>> --- a/meta/classes/package_rpm.bbclass
>> +++ b/meta/classes/package_rpm.bbclass
>> @@ -166,6 +166,7 @@ translate_oe_to_smart() {
>> [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart
>> --data-dir=${target_rootfs}/var/lib/smart query --output
>> ${target_rootfs}/install/tmp/fullpkglist.query
>>
>> pkgs_to_install=""
>> + not_found=""
>> for pkg in "$@" ; do
>> new_pkg="$pkg"
>> if [ -z "$sdk_mode" ]; then
>> @@ -184,7 +185,6 @@ translate_oe_to_smart() {
>> fi
>> subst=${pkg#${mlib}-}
>> if [ "$subst" != "$pkg" ]; then
>> - feeds=$@
>> while [ -n "$1" ]; do
>> arch="$1"
>> arch=`echo "$arch" | tr - _`
>> @@ -197,6 +197,76 @@ translate_oe_to_smart() {
>> done
>> if [ "$pkg" = "$new_pkg" ]; then
>> # Failed to translate, package not found!
>> + not_found="$not_found $pkg"
>> + continue
>> + fi
>> + fi
>> + done
>> + fi
>> + # Apparently not a multilib package...
>> + if [ "$pkg" = "$new_pkg" ]; then
>> + default_archs_fixed=`echo "$default_archs" | tr - _`
>> + for arch in $default_archs_fixed ; do
>> + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'
>> ${target_rootfs}/install/tmp/fullpkglist.query ; then
>> + new_pkg="$pkg@$arch"
>> + # First found is best match
>> + break
>> + fi
>> + done
>> + if [ "$pkg" = "$new_pkg" ]; then
>> + # Failed to translate, package not found!
>> + not_found="$not_found $pkg"
>> + continue
>> + fi
>> + fi
>> + #echo "$pkg -> $new_pkg" >&2
>> + pkgs_to_install="${pkgs_to_install} ${new_pkg}"
>> + done
>> +
>> + # Parse the not_found items and see if they were dependencies (RPROVIDES)
>> + # Follow the parsing example above...
>> + for pkg in $not_found ; do
>> + new_pkg="$pkg"
>> + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg
>> --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@
>> ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e
>> 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
>> ${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo
>> $pkgs_to_install | xargs -n 1 echo | grep -f
>> ${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov"
> ];
>> then
>> + # Nothing to do, already in the list
>> + #echo "Skipping $pkg -> $prov, already in install set" >&2
>> + continue
>> + fi
>> + if [ -z "$sdk_mode" ]; then
>> + for i in ${MULTILIB_PREFIX_LIST} ; do
>> + old_IFS="$IFS"
>> + IFS=":"
>> + set $i
>> + IFS="$old_IFS"
>> + mlib="$1"
>> + shift
>> + if [ "$mlib" = "default" ]; then
>> + if [ -z "$default_archs" ]; then
>> + default_archs=$@
>> + fi
>> + continue
>> + fi
>> + subst=${pkg#${mlib}-}
>> + if [ "$subst" != "$pkg" ]; then
>> + feeds=$@
>> + smart --data-dir=${target_rootfs}/var/lib/smart query
>> --provides=$subst --output ${target_rootfs}/install/tmp/provide.query
>> + grep '^[^@ ]*@[^@]*$'
> ${target_rootfs}/install/tmp/provide.query |
>> sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
>> ${target_rootfs}/install/tmp/provide.query.list || : + while [
> -n "$1"
>> ]; do
>> + arch="$1"
>> + arch=`echo "$arch" | tr - _`
>> + shift
>> + # Select first found, we don't know if one is better then
> another...
>> + prov=`grep '^[^@ ]*@'$arch'$'
>> ${target_rootfs}/install/tmp/provide.query.list | head -n 1` +
> if [ -n
>> "$prov" ]; then
>> + new_pkg=$prov
>> + break
>> + fi
>> + done
>> + if [ "$pkg" = "$new_pkg" ]; then
>> + # Failed to translate, package not found!
>> echo "$attemptonly: $pkg not found in the $mlib feeds
> ($feeds)." >&2
>> if [ "$attemptonly" = "Error" ]; then
>> exit 1
>> @@ -210,9 +280,10 @@ translate_oe_to_smart() {
>> if [ "$pkg" = "$new_pkg" ]; then
>> default_archs_fixed=`echo "$default_archs" | tr - _`
>> for arch in $default_archs_fixed ; do
>> - if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'
>> ${target_rootfs}/install/tmp/fullpkglist.query ; then
>> - new_pkg="$pkg@$arch"
>> - # First found is best match
>> + # Select first found, we don't know if one is better then
> another...
>> + prov=`grep '^[^@ ]*@'$arch'$'
>> ${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if
> [ -n
>> "$prov" ]; then
>> + new_pkg=$prov
>> break
>> fi
>> done
>
> Sigh... this code is getting ridiculously complicated. Shouldn't Smart be
> doing more of the heavy lifting here?
Agreed; this seems OE is doing too much.
Another think which bothers me is it being RPM specific ... :-(
--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name
2013-11-13 10:33 ` Paul Eggleton
2013-11-13 12:31 ` Otavio Salvador
@ 2013-11-13 13:12 ` Mark Hatle
1 sibling, 0 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 13:12 UTC (permalink / raw)
To: Paul Eggleton; +Cc: openembedded-core
On 11/13/13, 4:33 AM, Paul Eggleton wrote:
> Hi Mark,
>
> On Tuesday 12 November 2013 19:59:20 Mark Hatle wrote:
>> In the translate oe to smart function, we only translated package names.
>> However, it's allowed that people can put in a dependency name in the
>> IMAGE_INSTALL. So on a failure to translate a package name, we fall back
>> and attempt to resolve based on a package's provide.
>>
>> Note: it may be possible to generate an unsolvable install solution. If the
>> dependency is provided by one or more things that conflict with something
>> else set to be installed. We can't determine this until smart is run.
>>
>> If this occurs, file a bug and we'll have to identify a way to deal with the
>> RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES
>> with actual package names.
>>
>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>> ---
>> meta/classes/package_rpm.bbclass | 79
>> ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 75 insertions(+),
>> 4 deletions(-)
>>
>> diff --git a/meta/classes/package_rpm.bbclass
>> b/meta/classes/package_rpm.bbclass index 31265d9..9fe0e6c 100644
>> --- a/meta/classes/package_rpm.bbclass
>> +++ b/meta/classes/package_rpm.bbclass
>> @@ -166,6 +166,7 @@ translate_oe_to_smart() {
>> [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart
>> --data-dir=${target_rootfs}/var/lib/smart query --output
>> ${target_rootfs}/install/tmp/fullpkglist.query
>>
>> pkgs_to_install=""
>> + not_found=""
>> for pkg in "$@" ; do
>> new_pkg="$pkg"
>> if [ -z "$sdk_mode" ]; then
>> @@ -184,7 +185,6 @@ translate_oe_to_smart() {
>> fi
>> subst=${pkg#${mlib}-}
>> if [ "$subst" != "$pkg" ]; then
>> - feeds=$@
>> while [ -n "$1" ]; do
>> arch="$1"
>> arch=`echo "$arch" | tr - _`
>> @@ -197,6 +197,76 @@ translate_oe_to_smart() {
>> done
>> if [ "$pkg" = "$new_pkg" ]; then
>> # Failed to translate, package not found!
Just to be clear the following diff is rather confusing.. there are two one line
changes (the error to the no_found line below) in the existing code, and then
the new chunk of code after both to process the not_found entries, which are
presumed to be dependencies.)
If you apply this, or look at it in context it's easier to see..
>> + not_found="$not_found $pkg"
>> + continue
>> + fi
>> + fi
>> + done
>> + fi
>> + # Apparently not a multilib package...
>> + if [ "$pkg" = "$new_pkg" ]; then
>> + default_archs_fixed=`echo "$default_archs" | tr - _`
>> + for arch in $default_archs_fixed ; do
>> + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'
>> ${target_rootfs}/install/tmp/fullpkglist.query ; then
>> + new_pkg="$pkg@$arch"
>> + # First found is best match
>> + break
>> + fi
>> + done
>> + if [ "$pkg" = "$new_pkg" ]; then
>> + # Failed to translate, package not found!
>> + not_found="$not_found $pkg"
>> + continue
>> + fi
>> + fi
>> + #echo "$pkg -> $new_pkg" >&2
>> + pkgs_to_install="${pkgs_to_install} ${new_pkg}"
>> + done
>> +
>> + # Parse the not_found items and see if they were dependencies (RPROVIDES)
>> + # Follow the parsing example above...
>> + for pkg in $not_found ; do
>> + new_pkg="$pkg"
>> + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg
>> --output ${target_rootfs}/install/tmp/provide.query + grep '^[^@
>> ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e
>> 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
>> ${target_rootfs}/install/tmp/provide.query.list || : + prov=`echo
>> $pkgs_to_install | xargs -n 1 echo | grep -f
>> ${target_rootfs}/install/tmp/provide.query.list || :` + if [ -n "$prov"
> ];
>> then
>> + # Nothing to do, already in the list
>> + #echo "Skipping $pkg -> $prov, already in install set" >&2
>> + continue
>> + fi
>> + if [ -z "$sdk_mode" ]; then
>> + for i in ${MULTILIB_PREFIX_LIST} ; do
>> + old_IFS="$IFS"
>> + IFS=":"
>> + set $i
>> + IFS="$old_IFS"
>> + mlib="$1"
>> + shift
>> + if [ "$mlib" = "default" ]; then
>> + if [ -z "$default_archs" ]; then
>> + default_archs=$@
>> + fi
>> + continue
>> + fi
>> + subst=${pkg#${mlib}-}
>> + if [ "$subst" != "$pkg" ]; then
>> + feeds=$@
>> + smart --data-dir=${target_rootfs}/var/lib/smart query
>> --provides=$subst --output ${target_rootfs}/install/tmp/provide.query
>> + grep '^[^@ ]*@[^@]*$'
> ${target_rootfs}/install/tmp/provide.query |
>> sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' >
>> ${target_rootfs}/install/tmp/provide.query.list || : + while [
> -n "$1"
>> ]; do
>> + arch="$1"
>> + arch=`echo "$arch" | tr - _`
>> + shift
>> + # Select first found, we don't know if one is better then
> another...
>> + prov=`grep '^[^@ ]*@'$arch'$'
>> ${target_rootfs}/install/tmp/provide.query.list | head -n 1` +
> if [ -n
>> "$prov" ]; then
>> + new_pkg=$prov
>> + break
>> + fi
>> + done
>> + if [ "$pkg" = "$new_pkg" ]; then
>> + # Failed to translate, package not found!
>> echo "$attemptonly: $pkg not found in the $mlib feeds
> ($feeds)." >&2
>> if [ "$attemptonly" = "Error" ]; then
>> exit 1
>> @@ -210,9 +280,10 @@ translate_oe_to_smart() {
>> if [ "$pkg" = "$new_pkg" ]; then
>> default_archs_fixed=`echo "$default_archs" | tr - _`
>> for arch in $default_archs_fixed ; do
>> - if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$'
>> ${target_rootfs}/install/tmp/fullpkglist.query ; then
>> - new_pkg="$pkg@$arch"
>> - # First found is best match
>> + # Select first found, we don't know if one is better then
> another...
>> + prov=`grep '^[^@ ]*@'$arch'$'
>> ${target_rootfs}/install/tmp/provide.query.list | head -n 1` + if
> [ -n
>> "$prov" ]; then
>> + new_pkg=$prov
>> break
>> fi
>> done
>
> Sigh... this code is getting ridiculously complicated. Shouldn't Smart be
> doing more of the heavy lifting here?
This function translate_oe_to_smart (and the corresponding
translate_smart_to_oe) are used to switch from one name space to another.
Remember in the RPM world, we don't use prefixes like "lib32-" on our package
names, instead we use specific arch's that refer to specific ABI and processor
tunings.
This is what enables the seamless multilib install on RPM.
What this code does is allows for dependencies to be added to the mix. If we
didn't have multilibs then none of this code would be needed, but since we do --
the name space of both the package and dependencies need to be translate, not
just packages as we'd done before.
Generally what this function does:
for each multilib prefx:
if PN starts w/ a known multilib prefix:
we translate it to: BPN@arch
check if the package exists
if it does add to pkg_list
else add to "not_found"
anything remaining is translated to the default arch
check if the package exists
if it does add to pkg_list
else add to "not_found"
if there are any "not_found", we assume they must be provides:
for each multilib prefix:
if the DEPENDENCY starts w/ a known multilib prefix:
we translate it without the prefix
ask smart what provides that dependency
check if something already in the pkg_list is there, if so skip it
else use priority ordering to select the best provider -- add to pkg_list
for anything w/o a multilib prefix, follow the same steps
The reverse smart_to_oe function doesn't ever need to pay attention to
dependencies so that is not an issue, and the translation is significantly more
straight forward.
If we had unique namespaces in oe, and didn't use the lib* markers on
multilibs.. then none of this would be necessary.. (but then of course dpkg and
opkg wouldn't work properly.) So the translation is a necessary evil.
Could it be done differently, most likely. Perhaps identifying a way to do it
in python would be quicker.. but it's actually pretty fast sine the majority of
translations happen via a static list and a single grep. It's only the
(hopefully) small number of dependencies that happen via individual calls to
smart which impose a time penalty.
--Mark
> Cheers,
> Paul
>
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed
2013-11-13 11:14 ` Phil Blundell
@ 2013-11-13 13:39 ` Mark Hatle
2013-11-13 13:44 ` Phil Blundell
0 siblings, 1 reply; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 13:39 UTC (permalink / raw)
To: Phil Blundell, Martin Jansa; +Cc: openembedded-core
On 11/13/13, 5:14 AM, Phil Blundell wrote:
> On Wed, 2013-11-13 at 09:12 +0100, Martin Jansa wrote:
>> On Tue, Nov 12, 2013 at 07:23:22PM -0600, Mark Hatle wrote:
>>> When the -symlinks package was removed and an RPROVIDES was added, nothing
>>> was put in to cause the removal of the older package in an upgrade.
>>>
>>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>>> ---
>>> meta/recipes-devtools/binutils/binutils.inc | 1 +
>>> 1 file changed, 1 insertion(+)
>>>
>>> diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
>>> index 17c66bc..ea2383e 100644
>>> --- a/meta/recipes-devtools/binutils/binutils.inc
>>> +++ b/meta/recipes-devtools/binutils/binutils.inc
>>> @@ -21,6 +21,7 @@ FILES_${PN} = " \
>>> ${prefix}/${TARGET_SYS}/bin/*"
>>>
>>> RPROVIDES_${PN} += "${PN}-symlinks"
>>> +RREPLACES_${PN} += "${PN}-symlinks"
>>
>> IIRC for opkg backend you need also RCONFLICTS for opkg to really remove
>> old ${PN}-symlinks.
>
> Also for dpkg. If you have Replaces: without Conflicts: then this just
> means that the replacing package is allowed to overwrite files in the
> replaced one, not that the replaced one will actually be uninstalled.
This seems incredibly strange to me. But I can adjust the 2/6 and 3/6 patches
by adding the appropriate RCONFLICTS.
--Mark
> p.
>
>
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 3/6] binutils: Ensure old -symlinks packages get removed
2013-11-13 13:39 ` Mark Hatle
@ 2013-11-13 13:44 ` Phil Blundell
0 siblings, 0 replies; 27+ messages in thread
From: Phil Blundell @ 2013-11-13 13:44 UTC (permalink / raw)
To: Mark Hatle; +Cc: openembedded-core
On Wed, 2013-11-13 at 07:39 -0600, Mark Hatle wrote:
> On 11/13/13, 5:14 AM, Phil Blundell wrote:
> > On Wed, 2013-11-13 at 09:12 +0100, Martin Jansa wrote:
> >> On Tue, Nov 12, 2013 at 07:23:22PM -0600, Mark Hatle wrote:
> >>> When the -symlinks package was removed and an RPROVIDES was added, nothing
> >>> was put in to cause the removal of the older package in an upgrade.
> >>>
> >>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> >>> ---
> >>> meta/recipes-devtools/binutils/binutils.inc | 1 +
> >>> 1 file changed, 1 insertion(+)
> >>>
> >>> diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
> >>> index 17c66bc..ea2383e 100644
> >>> --- a/meta/recipes-devtools/binutils/binutils.inc
> >>> +++ b/meta/recipes-devtools/binutils/binutils.inc
> >>> @@ -21,6 +21,7 @@ FILES_${PN} = " \
> >>> ${prefix}/${TARGET_SYS}/bin/*"
> >>>
> >>> RPROVIDES_${PN} += "${PN}-symlinks"
> >>> +RREPLACES_${PN} += "${PN}-symlinks"
> >>
> >> IIRC for opkg backend you need also RCONFLICTS for opkg to really remove
> >> old ${PN}-symlinks.
> >
> > Also for dpkg. If you have Replaces: without Conflicts: then this just
> > means that the replacing package is allowed to overwrite files in the
> > replaced one, not that the replaced one will actually be uninstalled.
>
> This seems incredibly strange to me.
Well, perhaps, but it's always been that way. See the documentation for
Replaces:
http://www.debian.org/doc/debian-policy/ch-relationships.html#s-replaces
p.
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 2/6 v3] gcc: Drop *-symlinks
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
` (8 preceding siblings ...)
2013-11-13 1:59 ` [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name Mark Hatle
@ 2013-11-13 14:19 ` Mark Hatle
2013-11-13 14:19 ` [PATCH 3/6 v3] binutils: Ensure old -symlinks packages get removed Mark Hatle
2013-11-19 0:40 ` [PATCH 0/6] Misc changes relating to toolchain and image gen Saul Wold
11 siblings, 0 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 14:19 UTC (permalink / raw)
To: openembedded-core
The -symlinks packages don't really add any value if we're using
update-alternatives. Drop it, leaving an RPROVIDES in case anyone thought
they needed it still.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/recipes-devtools/gcc/gcc-target.inc | 54 ++++++++++++++++++--------------
1 file changed, 30 insertions(+), 24 deletions(-)
diff --git a/meta/recipes-devtools/gcc/gcc-target.inc b/meta/recipes-devtools/gcc/gcc-target.inc
index ee41d3f..019dbc0 100644
--- a/meta/recipes-devtools/gcc/gcc-target.inc
+++ b/meta/recipes-devtools/gcc/gcc-target.inc
@@ -10,12 +10,12 @@ EXTRA_OECONF_PATHS = " \
ARCH_FLAGS_FOR_TARGET += "-isystem${STAGING_INCDIR} -I${B}/gcc/include/ "
PACKAGES = "\
- ${PN} ${PN}-plugins ${PN}-symlinks \
- g++ g++-symlinks \
- cpp cpp-symlinks \
- g77 g77-symlinks \
- gfortran gfortran-symlinks \
- gcov gcov-symlinks \
+ ${PN} ${PN}-plugins \
+ g++ \
+ cpp \
+ g77 \
+ gfortran \
+ gcov \
${PN}-plugin-dev \
${PN}-doc \
${PN}-dev \
@@ -36,6 +36,9 @@ FILES_${PN} = "\
${gcclibdir}/${TARGET_SYS}/${BINV}/include-fixed \
"
INSANE_SKIP_${PN} += "dev-so"
+RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"
+RCONFLICTS_${PN} += "${PN}-symlinks"
FILES_${PN}-dbg += "\
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/.debug/ \
@@ -50,8 +53,6 @@ FILES_${PN}-plugin-dev = "\
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/plugin/gengtype \
${gcclibdir}/${TARGET_SYS}/${BINV}/plugin/gtype.state \
"
-FILES_${PN}-symlinks = ""
-ALLOW_EMPTY_${PN}-symlinks = "1"
FILES_${PN}-plugins = "\
${gcclibdir}/${TARGET_SYS}/${BINV}/plugin \
@@ -62,33 +63,38 @@ FILES_g77 = "\
${bindir}/${TARGET_PREFIX}g77 \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f771 \
"
-FILES_g77-symlinks = ""
-ALLOW_EMPTY_g77-symlinks = "1"
+RPROVIDES_g77 += "g77-symlinks"
+RREPLACES_g77 += "g77-symlinks"
+RCONFLICTS_g77 += "${PN}-symlinks"
FILES_gfortran = "\
${bindir}/${TARGET_PREFIX}gfortran \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/f951 \
"
-FILES_gfortran-symlinks = ""
-ALLOW_EMPTY_gfortran-symlinks = "1"
+RPROVIDES_gfortran += "gfortran-symlinks"
+RREPLACES_gfortran += "gfortran-symlinks"
+RCONFLICTS_gfortran += "${PN}-symlinks"
FILES_cpp = "\
${bindir}/${TARGET_PREFIX}cpp \
${base_libdir}/cpp \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1"
-FILES_cpp-symlinks = ""
-ALLOW_EMPTY_cpp-symlinks = "1"
+RPROVIDES_cpp += "cpp-symlinks"
+RREPLACES_cpp += "cpp-symlinks"
+RCONFLICTS_cpp += "${PN}-symlinks"
FILES_gcov = "${bindir}/${TARGET_PREFIX}gcov"
-FILES_gcov-symlinks = ""
-ALLOW_EMPTY_gconv-symlinks = "1"
+RPROVIDES_gcov += "gcov-symlinks"
+RREPLACES_gcov += "gcov-symlinks"
+RCONFLICTS_gcov += "${PN}-symlinks"
FILES_g++ = "\
${bindir}/${TARGET_PREFIX}g++ \
${libexecdir}/gcc/${TARGET_SYS}/${BINV}/cc1plus \
"
-FILES_g++-symlinks = ""
-ALLOW_EMPTY_g++-symlinks = "1"
+RPROVIDES_g++ += "g++-symlinks"
+RREPLACES_g++ += "g++-symlinks"
+RCONFLICTS_g++ += "${PN}-symlinks"
FILES_${PN}-doc = "\
${infodir} \
@@ -137,12 +143,12 @@ inherit update-alternatives
ALTERNATIVE_PRIORITY = "100"
-ALTERNATIVE_${PN}-symlinks = "cc gcc"
-ALTERNATIVE_g77-symlinks = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}"
-ALTERNATIVE_gfortran-symlinks = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}"
-ALTERNATIVE_cpp-symlinks = "cpp"
-ALTERNATIVE_gcov-symlinks = "gcov"
-ALTERNATIVE_g++-symlinks = "c++ g++"
+ALTERNATIVE_${PN} = "cc gcc"
+ALTERNATIVE_g77 = "${@['', 'g77 f77']['f77' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_gfortran = "${@['', 'gfortran f95']['fortran' in d.getVar('FORTRAN', True)]}"
+ALTERNATIVE_cpp = "cpp"
+ALTERNATIVE_gcov = "gcov"
+ALTERNATIVE_g++ = "c++ g++"
ALTERNATIVE_LINK_NAME[cc] = "${bindir}/cc"
ALTERNATIVE_TARGET[cc] = "${bindir}/${TARGET_PREFIX}gcc"
--
1.8.3.4
^ permalink raw reply related [flat|nested] 27+ messages in thread
* [PATCH 3/6 v3] binutils: Ensure old -symlinks packages get removed
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
` (9 preceding siblings ...)
2013-11-13 14:19 ` [PATCH 2/6 v3] gcc: Drop *-symlinks Mark Hatle
@ 2013-11-13 14:19 ` Mark Hatle
2013-11-19 0:40 ` [PATCH 0/6] Misc changes relating to toolchain and image gen Saul Wold
11 siblings, 0 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-13 14:19 UTC (permalink / raw)
To: openembedded-core
When the -symlinks package was removed and an RPROVIDES was added, nothing
was put in to cause the removal of the older package in an upgrade.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
---
meta/recipes-devtools/binutils/binutils.inc | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta/recipes-devtools/binutils/binutils.inc b/meta/recipes-devtools/binutils/binutils.inc
index 17c66bc..77b3718 100644
--- a/meta/recipes-devtools/binutils/binutils.inc
+++ b/meta/recipes-devtools/binutils/binutils.inc
@@ -21,6 +21,8 @@ FILES_${PN} = " \
${prefix}/${TARGET_SYS}/bin/*"
RPROVIDES_${PN} += "${PN}-symlinks"
+RREPLACES_${PN} += "${PN}-symlinks"
+RCONFLICTS_${PN} += "${PN}-symlinks"
FILES_${PN}-dev = " \
${includedir} \
--
1.8.3.4
^ permalink raw reply related [flat|nested] 27+ messages in thread
* Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name
2013-11-13 1:59 ` [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name Mark Hatle
2013-11-13 10:33 ` Paul Eggleton
@ 2013-11-15 14:02 ` Richard Purdie
2013-11-15 17:15 ` Mark Hatle
1 sibling, 1 reply; 27+ messages in thread
From: Richard Purdie @ 2013-11-15 14:02 UTC (permalink / raw)
To: Mark Hatle; +Cc: openembedded-core
On Tue, 2013-11-12 at 19:59 -0600, Mark Hatle wrote:
> In the translate oe to smart function, we only translated package names.
> However, it's allowed that people can put in a dependency name in the
> IMAGE_INSTALL.
For what its worth, I find this description hard to understand, I think
as you leave the keyword "provide" right until the end. A mention of
RPROVIDES earlier here would help a lot.
> So on a failure to translate a package name, we fall back
> and attempt to resolve based on a package's provide.
>
> Note: it may be possible to generate an unsolvable install solution. If the
> dependency is provided by one or more things that conflict with something else
> set to be installed. We can't determine this until smart is run.
>
> If this occurs, file a bug and we'll have to identify a way to deal with the
> RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with
> actual package names.
>
> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
> ---
> meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++--
> 1 file changed, 75 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
> index 31265d9..9fe0e6c 100644
> --- a/meta/classes/package_rpm.bbclass
> +++ b/meta/classes/package_rpm.bbclass
> @@ -166,6 +166,7 @@ translate_oe_to_smart() {
> [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query
>
> pkgs_to_install=""
> + not_found=""
> for pkg in "$@" ; do
> new_pkg="$pkg"
> if [ -z "$sdk_mode" ]; then
> @@ -184,7 +185,6 @@ translate_oe_to_smart() {
> fi
> subst=${pkg#${mlib}-}
> if [ "$subst" != "$pkg" ]; then
> - feeds=$@
> while [ -n "$1" ]; do
> arch="$1"
> arch=`echo "$arch" | tr - _`
> @@ -197,6 +197,76 @@ translate_oe_to_smart() {
> done
> if [ "$pkg" = "$new_pkg" ]; then
> # Failed to translate, package not found!
> + not_found="$not_found $pkg"
> + continue
> + fi
> + fi
> + done
> + fi
> + # Apparently not a multilib package...
> + if [ "$pkg" = "$new_pkg" ]; then
> + default_archs_fixed=`echo "$default_archs" | tr - _`
> + for arch in $default_archs_fixed ; do
> + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
> + new_pkg="$pkg@$arch"
> + # First found is best match
> + break
> + fi
> + done
> + if [ "$pkg" = "$new_pkg" ]; then
> + # Failed to translate, package not found!
> + not_found="$not_found $pkg"
> + continue
> + fi
> + fi
> + #echo "$pkg -> $new_pkg" >&2
> + pkgs_to_install="${pkgs_to_install} ${new_pkg}"
> + done
> +
> + # Parse the not_found items and see if they were dependencies (RPROVIDES)
> + # Follow the parsing example above...
> + for pkg in $not_found ; do
> + new_pkg="$pkg"
> + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query
> + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
> + prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :`
> + if [ -n "$prov" ]; then
> + # Nothing to do, already in the list
> + #echo "Skipping $pkg -> $prov, already in install set" >&2
> + continue
> + fi
> + if [ -z "$sdk_mode" ]; then
> + for i in ${MULTILIB_PREFIX_LIST} ; do
> + old_IFS="$IFS"
> + IFS=":"
> + set $i
> + IFS="$old_IFS"
> + mlib="$1"
> + shift
> + if [ "$mlib" = "default" ]; then
> + if [ -z "$default_archs" ]; then
> + default_archs=$@
> + fi
> + continue
> + fi
> + subst=${pkg#${mlib}-}
> + if [ "$subst" != "$pkg" ]; then
> + feeds=$@
> + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query
> + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
> + while [ -n "$1" ]; do
> + arch="$1"
> + arch=`echo "$arch" | tr - _`
> + shift
> + # Select first found, we don't know if one is better then another...
> + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
> + if [ -n "$prov" ]; then
> + new_pkg=$prov
> + break
> + fi
> + done
> + if [ "$pkg" = "$new_pkg" ]; then
> + # Failed to translate, package not found!
> echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2
> if [ "$attemptonly" = "Error" ]; then
> exit 1
> @@ -210,9 +280,10 @@ translate_oe_to_smart() {
> if [ "$pkg" = "$new_pkg" ]; then
> default_archs_fixed=`echo "$default_archs" | tr - _`
> for arch in $default_archs_fixed ; do
> - if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
> - new_pkg="$pkg@$arch"
> - # First found is best match
> + # Select first found, we don't know if one is better then another...
> + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
> + if [ -n "$prov" ]; then
> + new_pkg=$prov
> break
> fi
> done
This 'shell calling rpm commands' code with all the mangling is near
impossible to read/understand. This patch just makes the mess more
complicated and harder to follow.
What is the plan for turning this into something better?
My instinct says more functions and a language better at data
manipulation (python instead of shell) might start to resolve this.
One of the reasons for using smart was that we could probably enhance it
to better suit our needs. Can we consider that too?
I really don't want to see more patches like this one piling on top of
already crumbling foundations.
Cheers,
Richard
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name
2013-11-15 14:02 ` Richard Purdie
@ 2013-11-15 17:15 ` Mark Hatle
0 siblings, 0 replies; 27+ messages in thread
From: Mark Hatle @ 2013-11-15 17:15 UTC (permalink / raw)
To: Richard Purdie; +Cc: openembedded-core
On 11/15/13, 8:02 AM, Richard Purdie wrote:
> On Tue, 2013-11-12 at 19:59 -0600, Mark Hatle wrote:
>> In the translate oe to smart function, we only translated package names.
>> However, it's allowed that people can put in a dependency name in the
>> IMAGE_INSTALL.
>
> For what its worth, I find this description hard to understand, I think
> as you leave the keyword "provide" right until the end. A mention of
> RPROVIDES earlier here would help a lot.
Ok.
>> So on a failure to translate a package name, we fall back
>> and attempt to resolve based on a package's provide.
>>
>> Note: it may be possible to generate an unsolvable install solution. If the
>> dependency is provided by one or more things that conflict with something else
>> set to be installed. We can't determine this until smart is run.
>>
>> If this occurs, file a bug and we'll have to identify a way to deal with the
>> RCONFLICTS and RREPLACES. As a workaround replace the conflict REQUIRES with
>> actual package names.
>>
>> Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
>> ---
>> meta/classes/package_rpm.bbclass | 79 ++++++++++++++++++++++++++++++++++++++--
>> 1 file changed, 75 insertions(+), 4 deletions(-)
>>
>> diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
>> index 31265d9..9fe0e6c 100644
>> --- a/meta/classes/package_rpm.bbclass
>> +++ b/meta/classes/package_rpm.bbclass
>> @@ -166,6 +166,7 @@ translate_oe_to_smart() {
>> [ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query
>>
>> pkgs_to_install=""
>> + not_found=""
>> for pkg in "$@" ; do
>> new_pkg="$pkg"
>> if [ -z "$sdk_mode" ]; then
>> @@ -184,7 +185,6 @@ translate_oe_to_smart() {
>> fi
>> subst=${pkg#${mlib}-}
>> if [ "$subst" != "$pkg" ]; then
>> - feeds=$@
>> while [ -n "$1" ]; do
>> arch="$1"
>> arch=`echo "$arch" | tr - _`
>> @@ -197,6 +197,76 @@ translate_oe_to_smart() {
>> done
>> if [ "$pkg" = "$new_pkg" ]; then
>> # Failed to translate, package not found!
>> + not_found="$not_found $pkg"
>> + continue
>> + fi
>> + fi
>> + done
>> + fi
>> + # Apparently not a multilib package...
>> + if [ "$pkg" = "$new_pkg" ]; then
>> + default_archs_fixed=`echo "$default_archs" | tr - _`
>> + for arch in $default_archs_fixed ; do
>> + if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
>> + new_pkg="$pkg@$arch"
>> + # First found is best match
>> + break
>> + fi
>> + done
>> + if [ "$pkg" = "$new_pkg" ]; then
>> + # Failed to translate, package not found!
>> + not_found="$not_found $pkg"
>> + continue
>> + fi
>> + fi
>> + #echo "$pkg -> $new_pkg" >&2
>> + pkgs_to_install="${pkgs_to_install} ${new_pkg}"
>> + done
>> +
>> + # Parse the not_found items and see if they were dependencies (RPROVIDES)
>> + # Follow the parsing example above...
>> + for pkg in $not_found ; do
>> + new_pkg="$pkg"
>> + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$pkg --output ${target_rootfs}/install/tmp/provide.query
>> + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
>> + prov=`echo $pkgs_to_install | xargs -n 1 echo | grep -f ${target_rootfs}/install/tmp/provide.query.list || :`
>> + if [ -n "$prov" ]; then
>> + # Nothing to do, already in the list
>> + #echo "Skipping $pkg -> $prov, already in install set" >&2
>> + continue
>> + fi
>> + if [ -z "$sdk_mode" ]; then
>> + for i in ${MULTILIB_PREFIX_LIST} ; do
>> + old_IFS="$IFS"
>> + IFS=":"
>> + set $i
>> + IFS="$old_IFS"
>> + mlib="$1"
>> + shift
>> + if [ "$mlib" = "default" ]; then
>> + if [ -z "$default_archs" ]; then
>> + default_archs=$@
>> + fi
>> + continue
>> + fi
>> + subst=${pkg#${mlib}-}
>> + if [ "$subst" != "$pkg" ]; then
>> + feeds=$@
>> + smart --data-dir=${target_rootfs}/var/lib/smart query --provides=$subst --output ${target_rootfs}/install/tmp/provide.query
>> + grep '^[^@ ]*@[^@]*$' ${target_rootfs}/install/tmp/provide.query | sed -e 's,\(.*\)-[^-]*-[^-]*\(@[^@]*\)$,\1\2,' > ${target_rootfs}/install/tmp/provide.query.list || :
>> + while [ -n "$1" ]; do
>> + arch="$1"
>> + arch=`echo "$arch" | tr - _`
>> + shift
>> + # Select first found, we don't know if one is better then another...
>> + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
>> + if [ -n "$prov" ]; then
>> + new_pkg=$prov
>> + break
>> + fi
>> + done
>> + if [ "$pkg" = "$new_pkg" ]; then
>> + # Failed to translate, package not found!
>> echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2
>> if [ "$attemptonly" = "Error" ]; then
>> exit 1
>> @@ -210,9 +280,10 @@ translate_oe_to_smart() {
>> if [ "$pkg" = "$new_pkg" ]; then
>> default_archs_fixed=`echo "$default_archs" | tr - _`
>> for arch in $default_archs_fixed ; do
>> - if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
>> - new_pkg="$pkg@$arch"
>> - # First found is best match
>> + # Select first found, we don't know if one is better then another...
>> + prov=`grep '^[^@ ]*@'$arch'$' ${target_rootfs}/install/tmp/provide.query.list | head -n 1`
>> + if [ -n "$prov" ]; then
>> + new_pkg=$prov
>> break
>> fi
>> done
>
> This 'shell calling rpm commands' code with all the mangling is near
> impossible to read/understand. This patch just makes the mess more
> complicated and harder to follow.
The root of this is the namespace issue. In RPM, the architecture (and arch
compatibility), as well as file contents all provide the namespace and conflict
resolution. While in opkg/deb the name of the package itself is where the
namespace is defined. (i.e. lib32-...)
The translation is necessary due to this multilib support, otherwise we wouldn't
need to do any of it. Somehow we have to translate one namespace to another --
and since the translation can't occur until the packages are built, it'd be
difficult to do it 'earlier'.
> What is the plan for turning this into something better?
>
> My instinct says more functions and a language better at data
> manipulation (python instead of shell) might start to resolve this.
>
> One of the reasons for using smart was that we could probably enhance it
> to better suit our needs. Can we consider that too?
The only thing we can do is build a python (or similar) app that takes the
inputs of the translation 'namespace' IDs and the corresponding architectures,
and RPM feed. Then translate from one to the other. My biggest concern with
this translation is to cover all of the corner cases we could start getting more
and more complex.
> I really don't want to see more patches like this one piling on top of
> already crumbling foundations.
Easiest way to avoid this is to make dependencies in the IMAGE/PACKAGE_INSTALL
forbidden and do the translation at a higher level generically. But frankly I
don't see that happening -- so the solution becomes RPM specific and we end up
with code like this. I'm not sure if we can move this code into a python
function -- or if we need to simply create an external translation program...
As far as plans, frankly I don't have any plans to focus on this work for the
time being. There are too many other things I need to get working properly
first. If someone else wants to step in and work on a solution, I'm more then
happy to help them out.
--Mark
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 0/6] Misc changes relating to toolchain and image gen
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
` (10 preceding siblings ...)
2013-11-13 14:19 ` [PATCH 3/6 v3] binutils: Ensure old -symlinks packages get removed Mark Hatle
@ 2013-11-19 0:40 ` Saul Wold
11 siblings, 0 replies; 27+ messages in thread
From: Saul Wold @ 2013-11-19 0:40 UTC (permalink / raw)
To: Mark Hatle, openembedded-core
On 11/12/2013 05:23 PM, Mark Hatle wrote:
> A few misc fixes relating to toolchain and image generation.
>
> The first two in the set simply updated gcc to remove the -symlinks package
> in favor of update-alternatives.
>
> The next fixes an issue w/ the previous binutils patch and upgrades.
>
> Next 2 update packagegroups to make revise and use the
> packagegroup-core-buildessentials.
>
> The final patch fixes an issue w/ the image generatation. If the
> IMAGE_INSTALL contains a dependency instead of a package name, the system
> was not able to translate it into a package. This now works properly.
>
> Note, there is a case where an unresolvable set could be determined (rare,
> but it is possible), in this case the user would need to add whatever is
> required to fix it to the IMAGE_INSTALL. (This could happen when 2 or more
> conflicting packages all have the same provide, but none of them are in
> the IMAGE_INSTALL line. The system will attempt to determine which to
> use, by checking if any of them are already in the IMAGE_INSTALL -- otherwise
> it chooses the first entry. If something else implied by a packagegroup
> chooses a different version, a conflict would be generated.)
>
> The following changes since commit 616354f13732d13c17434d5b60b166f691c25761:
>
> binutils: Add gnu-config-native to DEPENDS (2013-11-12 16:00:20 +0000)
>
> are available in the git repository at:
>
> git://git.yoctoproject.org/poky-contrib mhatle/misc1
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mhatle/misc1
>
> Mark Hatle (6):
> gcc: Use alternatives for the *-symlinks packages.
> gcc: Drop *-symlinks
Mark
This change set seems to be giving the Autobuilder some issues, please
take a look at the AB (for example:
http://autobuilder.yoctoproject.org:8011/builders/nightly-mips/builds/1/steps/Building%20Images/logs/stdio
Thanks
Sau!
> binutils: Ensure old -symlinks packages get removed
> packagegroups: Remove toolchain *-symlinks packages
> packagegroup-self-hosted: Use packagegroup-core-buildessentials
> package_rpm: Allow translation of requirement to package name
>
> meta/classes/package_rpm.bbclass | 79 +++++++++++++++-
> .../packagegroup-core-buildessential.bb | 4 -
> .../packagegroups/packagegroup-self-hosted.bb | 16 +---
> meta/recipes-devtools/binutils/binutils.inc | 1 +
> meta/recipes-devtools/gcc/gcc-target.inc | 103 +++++++++++++--------
> 5 files changed, 141 insertions(+), 62 deletions(-)
>
^ permalink raw reply [flat|nested] 27+ messages in thread
end of thread, other threads:[~2013-11-19 0:40 UTC | newest]
Thread overview: 27+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2013-11-13 1:23 [PATCH 0/6] Misc changes relating to toolchain and image gen Mark Hatle
2013-11-13 1:23 ` [PATCH 1/6] gcc: Use alternatives for the *-symlinks packages Mark Hatle
2013-11-13 2:24 ` Otavio Salvador
2013-11-13 2:27 ` Mark Hatle
2013-11-13 2:28 ` Otavio Salvador
2013-11-13 1:23 ` [PATCH 2/6] gcc: Drop *-symlinks Mark Hatle
2013-11-13 1:23 ` [PATCH 3/6] binutils: Ensure old -symlinks packages get removed Mark Hatle
2013-11-13 8:12 ` Martin Jansa
2013-11-13 11:14 ` Phil Blundell
2013-11-13 13:39 ` Mark Hatle
2013-11-13 13:44 ` Phil Blundell
2013-11-13 1:23 ` [PATCH 4/6] packagegroups: Remove toolchain *-symlinks packages Mark Hatle
2013-11-13 1:23 ` [PATCH 5/6] packagegroup-self-hosted: Use packagegroup-core-buildessentials Mark Hatle
2013-11-13 1:23 ` [PATCH 6/6] package_rpm: Allow translation of requirement to package name Mark Hatle
2013-11-13 1:33 ` [PATCH 0/6] Misc changes relating to toolchain and image gen Otavio Salvador
2013-11-13 2:01 ` Mark Hatle
2013-11-13 3:42 ` Mark Hatle
2013-11-13 1:59 ` [PATCH 5/6 v2] packagegroup-self-hosted: Use packagegroup-core-buildessential Mark Hatle
2013-11-13 1:59 ` [PATCH 6/6 v2] package_rpm: Allow translation of requirement to package name Mark Hatle
2013-11-13 10:33 ` Paul Eggleton
2013-11-13 12:31 ` Otavio Salvador
2013-11-13 13:12 ` Mark Hatle
2013-11-15 14:02 ` Richard Purdie
2013-11-15 17:15 ` Mark Hatle
2013-11-13 14:19 ` [PATCH 2/6 v3] gcc: Drop *-symlinks Mark Hatle
2013-11-13 14:19 ` [PATCH 3/6 v3] binutils: Ensure old -symlinks packages get removed Mark Hatle
2013-11-19 0:40 ` [PATCH 0/6] Misc changes relating to toolchain and image gen Saul Wold
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.