* [PATCH 1/4] insane.bbclass: add QA check: package-missing
2014-12-25 1:48 [PATCH V2 0/4] insane.bbclass: add QA check: package-missing Hongxu Jia
@ 2014-12-25 1:48 ` Hongxu Jia
2014-12-25 1:48 ` [PATCH 2/4] perl_5.20.0: fix QA package-missing warnings Hongxu Jia
` (3 subsequent siblings)
4 siblings, 0 replies; 15+ messages in thread
From: Hongxu Jia @ 2014-12-25 1:48 UTC (permalink / raw)
To: openembedded-core, richard.purdie
During packaging, check if there are any dependencies (RDEPENDS)
on packages that have ended up empty and not produced; and if so
produce an warn/error as soon as possible, instead of allowing
the build to proceed up to do_rootfs and then the package manager
reporting the package as missing.
At the moment, we use packageinfo which saved in $PKGDATA_DIR to
collect all available packages rdepends and rprovides. So in the
do_package_qa task, while package not produced, we could compute
rdepends chain, and report warn/error immediately.
[YOCTO #5531]
[YOCTO #6420]
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/classes/insane.bbclass | 100 +++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 99 insertions(+), 1 deletion(-)
diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass
index 0b45374..59129b7 100644
--- a/meta/classes/insane.bbclass
+++ b/meta/classes/insane.bbclass
@@ -29,7 +29,7 @@ QA_SANE = "True"
WARN_QA ?= "ldflags useless-rpaths rpaths staticdev libdir xorg-driver-abi \
textrel already-stripped incompatible-license files-invalid \
installed-vs-shipped compile-host-path install-host-path \
- pn-overrides infodir build-deps file-rdeps \
+ pn-overrides infodir build-deps file-rdeps package-missing \
"
ERROR_QA ?= "dev-so debug-deps dev-deps debug-files arch pkgconfig la \
perms dep-cmp pkgvarcheck perm-config perm-line perm-link \
@@ -868,6 +868,15 @@ def package_qa_check_rdepends(pkg, pkgdest, skip, taskdeps, packages, d):
(pkg, ', '.join(str(e) for e in filerdepends))
sane = package_qa_handle_error("file-rdeps", error_msg, d)
+ if "package-missing" not in skip and \
+ not pkg.endswith("-dev") and not pkg.endswith("-staticdev") and \
+ not pkg.endswith("-locale") and not pkg.endswith("-dbg") and \
+ not pkg.endswith("-doc"):
+ if not oe.packagedata.packaged(pkg, d):
+ error_msg = "package %s not generated, but listed in PACKAGES\n" % pkg
+ error_msg += compute_rdepends_chain(pkg, d)
+ sane = package_qa_handle_error("package-missing", error_msg, d)
+
return sane
def package_qa_check_deps(pkg, pkgdest, skip, d):
@@ -1152,3 +1161,92 @@ python () {
for i in issues:
package_qa_handle_error("pkgvarcheck", "%s: Variable %s is set as not being package specific, please fix this." % (d.getVar("FILE", True), i), d)
}
+
+# It collects all available packages rdepends and rprovides,
+# except the "-dev, -staticdev, -locale, -dbg, -doc" packages.
+# Such as:
+# RDEPENDS_pkg1 = "pkg2 pkg3"
+# RDEPENDS_pkg4 = "pkg3 pkg5"
+# equals:
+# rdeps_dict = {'pkg2': 'pkg1', 'pkg3': 'pkg1 pkg4', 'pkg5':'pkg4'}
+#
+# RPROVIDES_pkg6 = "pkg2 pkg7"
+# RPROVIDES_pkg7 = "pkg5"
+# equals:
+# rprovides_dict = {'pkg6': 'pkg2 pkg7', 'pkg7': 'pkg5'}
+def collect_packageinfo(d):
+ import re
+ rdeps_dict = dict()
+ rprovides_dict = dict()
+ pkgdata_dir = d.getVar("PKGDATA_DIR", True) + '/runtime/'
+ if os.path.exists(pkgdata_dir):
+ for root, dirs, files in os.walk(pkgdata_dir):
+ for pkgname in files:
+ if pkgname.endswith('.packaged') or \
+ pkgname.endswith("-dev") or \
+ pkgname.endswith("-staticdev") or \
+ pkgname.endswith("-locale") or \
+ pkgname.endswith("-dbg") or pkgname.endswith("-doc"):
+ continue
+
+ pkgdatafile = root + pkgname
+ try:
+ sdata = oe.packagedata.read_pkgdatafile(pkgdatafile)
+ rdepends = sdata.get('RDEPENDS_%s' % pkgname)
+ if rdepends:
+ # Filter out sub string "(***)" in RDEPENDS
+ p = re.compile('\([^\)]*\)')
+ for rdep in p.sub('', rdepends).split():
+ # Filter out pkg which rdepends itself
+ if pkgname == rdep:
+ continue
+
+ if rdep in rdeps_dict and \
+ pkgname not in rdeps_dict[rdep].split():
+ rdeps_dict[rdep] += ' %s' % pkgname
+ elif rdep not in rdeps_dict:
+ rdeps_dict[rdep] = '%s' % pkgname
+
+ rprovides = sdata.get('RPROVIDES_%s' % pkgname)
+ if rprovides:
+ rprovides_dict[pkgname] = rprovides
+
+ except Exception as e:
+ bb.warn("%s: Failed to read pkgdata file %s: %s: %s" % (pkgname, pkgdatafile, e.__class__, str(e)))
+
+ return rdeps_dict, rprovides_dict
+
+def compute_rdepends_chain(pkg, d):
+ rdeps_dict, rprovides_dict = collect_packageinfo(d)
+ def get_parents(pkg, messages):
+ parents = []
+
+ if pkg not in rdeps_dict and pkg in rprovides_dict:
+ for rprovide in rprovides_dict[pkg].split():
+ # Use rprovide to instead of pkg
+ if rprovide in rdeps_dict:
+ messages.append('("%s" rprovides "%s")' % (pkg, rprovide))
+ pkg = rprovide
+ break
+
+ if pkg in rdeps_dict:
+ parents = rdeps_dict[pkg].split()
+ for parent in parents:
+ message = '"%s" -> "%s"' % (parent, pkg)
+ if message in messages:
+ return
+ messages.append(message)
+
+ for parent in parents:
+ get_parents(parent, messages)
+
+ return
+
+ messages = []
+ get_parents(pkg, messages)
+ if messages:
+ message = "Compute rdepends chains, '-->' means 'runtime depends'\n" \
+ + '\n'.join(messages)
+ return message
+ return ''
+
--
1.9.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH 2/4] perl_5.20.0: fix QA package-missing warnings
2014-12-25 1:48 [PATCH V2 0/4] insane.bbclass: add QA check: package-missing Hongxu Jia
2014-12-25 1:48 ` [PATCH 1/4] " Hongxu Jia
@ 2014-12-25 1:48 ` Hongxu Jia
2014-12-25 1:48 ` [PATCH 3/4] linux-libc-headers.inc: " Hongxu Jia
` (2 subsequent siblings)
4 siblings, 0 replies; 15+ messages in thread
From: Hongxu Jia @ 2014-12-25 1:48 UTC (permalink / raw)
To: openembedded-core, richard.purdie
While building perl with QA package-missing checking enabled, there was warning:
...
WARNING: QA Issue: package perl-module-cpanplus not generated, but listed in PACKAGES
WARNING: QA Issue: package perl-module-config not generated, but listed in PACKAGES
WARNING: QA Issue: package perl-module-config-heavy not generated, but listed in PACKAGES
WARNING: QA Issue: package perl-module-strict not generated, but listed in PACKAGES
WARNING: QA Issue: package perl-module-vars not generated, but listed in PACKAGES
WARNING: QA Issue: package perl-module-warnings not generated, but listed in PACKAGES
WARNING: QA Issue: package perl-module-warnings-register not generated, but listed in PACKAGES
...
Drop perl-module-cpanplus, from git://perl5.git.perl.org/perl.git:
...
commit fb598ba5e55920eb59105c932df653f4fea6966c
Author: Chris 'BinGOs' Williams <chris@bingosnet.co.uk>
Date: Sat May 18 01:01:42 2013 +0100
Remove cpan/CPANPLUS and associated utilities
...
Drop perl-module-config from PACKAGES_DYNAMIC:
Config.pm was listed in "FILES_${PN}", we should filter it out from
do_split_packages.
Drop perl-module-config-heavy from PACKAGES_DYNAMIC:
Config_heavy.pl was listed in "FILES_${PN}-lib", we should filter it out from
do_split_packages.
Drop perl-module-strict from PACKAGES_DYNAMIC:
strict.pm was listed in "FILES_${PN}", we should filter it out from
do_split_packages.
Drop perl-module-vars from PACKAGES_DYNAMIC:
vars.pm was listed in "FILES_${PN}", we should filter it out from
do_split_packages.
Drop perl-module-warnings from PACKAGES_DYNAMIC:
warnings.pm was listed in "FILES_${PN}", we should filter it out from
do_split_packages.
Drop perl-module-warnings-register from PACKAGES_DYNAMIC:
Dir warnings was listed in "FILES_${PN}", we should filter it out from
do_split_packages.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-devtools/perl/perl_5.20.0.bb | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/meta/recipes-devtools/perl/perl_5.20.0.bb b/meta/recipes-devtools/perl/perl_5.20.0.bb
index 3ca0f53..b63cb46 100644
--- a/meta/recipes-devtools/perl/perl_5.20.0.bb
+++ b/meta/recipes-devtools/perl/perl_5.20.0.bb
@@ -272,7 +272,7 @@ perl_package_preprocess () {
}
PACKAGES = "perl-dbg perl perl-misc perl-dev perl-pod perl-doc perl-lib \
- perl-module-cpan perl-module-cpanplus perl-module-unicore"
+ perl-module-cpan perl-module-unicore"
FILES_${PN} = "${bindir}/perl ${bindir}/perl${PV} \
${libdir}/perl/${PV}/Config.pm \
${libdir}/perl/${PV}/strict.pm \
@@ -326,8 +326,6 @@ FILES_${PN}-doc = "${libdir}/perl/${PV}/*/*.txt \
FILES_perl-module-cpan += "${libdir}/perl/${PV}/CPAN \
${libdir}/perl/${PV}/CPAN.pm"
-FILES_perl-module-cpanplus += "${libdir}/perl/${PV}/CPANPLUS \
- ${libdir}/perl/${PV}/CPANPLUS.pm"
FILES_perl-module-unicore += "${libdir}/perl/${PV}/unicore"
# Create a perl-modules package recommending all the other perl
@@ -340,7 +338,11 @@ python populate_packages_prepend () {
do_split_packages(d, libdir, 'auto/([^.]*)/[^/]*\.(so|ld|ix|al)', 'perl-module-%s', 'perl module %s', recursive=True, match_path=True, prepend=False)
do_split_packages(d, libdir, 'Module/([^\/]*)\.pm', 'perl-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
do_split_packages(d, libdir, 'Module/([^\/]*)/.*', 'perl-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
- do_split_packages(d, libdir, '(^(?!(CPAN\/|CPANPLUS\/|Module\/|unicore\/|auto\/)[^\/]).*)\.(pm|pl|e2x)', 'perl-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
+
+ unmatch_dirs = '(CPAN\/|Module\/|unicore\/|auto\/|warnings\/)[^\/]'
+ unmatch_files = 'vars.pm$|Config.pm$|strict.pm$|Config_heavy.pl$|warnings.pm$'
+ file_regex = '(^(?!(%s|%s)).*)\.(pm|pl|e2x)' % (unmatch_dirs, unmatch_files)
+ do_split_packages(d, libdir, file_regex, 'perl-module-%s', 'perl module %s', recursive=True, allow_dirs=False, match_path=True, prepend=False)
# perl-modules should recommend every perl module, and only the
# modules. Don't attempt to use the result of do_split_packages() as some
--
1.9.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* [PATCH 3/4] linux-libc-headers.inc: fix QA package-missing warnings
2014-12-25 1:48 [PATCH V2 0/4] insane.bbclass: add QA check: package-missing Hongxu Jia
2014-12-25 1:48 ` [PATCH 1/4] " Hongxu Jia
2014-12-25 1:48 ` [PATCH 2/4] perl_5.20.0: fix QA package-missing warnings Hongxu Jia
@ 2014-12-25 1:48 ` Hongxu Jia
2014-12-26 13:58 ` Otavio Salvador
2014-12-29 8:16 ` [PATCH V2 " Hongxu Jia
2014-12-25 1:48 ` [PATCH 4/4] busybox.inc: " Hongxu Jia
2015-01-09 11:50 ` [PATCH V2 0/4] insane.bbclass: add QA check: package-missing Hongxu Jia
4 siblings, 2 replies; 15+ messages in thread
From: Hongxu Jia @ 2014-12-25 1:48 UTC (permalink / raw)
To: openembedded-core, richard.purdie
While building linux-libc-headers with QA package-missing checking enabled,
there was warning:
...
QA Issue: package linux-libc-headers not generated, but listed in PACKAGES
...
Only linux-libc-headers-dev was generated, the others are empty, we should
set PACKAGE manually.
We also set linux-libc-headers-dev as linux-libc-headers's provider, so
package management could install linux-libc-headers.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
| 2 ++
1 file changed, 2 insertions(+)
--git a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
index b18d09f..a69bbe6 100644
--- a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
+++ b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
@@ -71,3 +71,5 @@ RRECOMMENDS_${PN}-dbg = "${PN}-dev (= ${EXTENDPKGV})"
INHIBIT_DEFAULT_DEPS = "1"
DEPENDS += "unifdef-native"
+PACKAGES="${PN}-dev"
+RPROVIDES_${PN}-dev = "${PN}"
--
1.9.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH 3/4] linux-libc-headers.inc: fix QA package-missing warnings
2014-12-25 1:48 ` [PATCH 3/4] linux-libc-headers.inc: " Hongxu Jia
@ 2014-12-26 13:58 ` Otavio Salvador
2014-12-29 8:16 ` [PATCH V2 " Hongxu Jia
1 sibling, 0 replies; 15+ messages in thread
From: Otavio Salvador @ 2014-12-26 13:58 UTC (permalink / raw)
To: Hongxu Jia; +Cc: Patches and discussions about the oe-core layer
On Wed, Dec 24, 2014 at 11:48 PM, Hongxu Jia <hongxu.jia@windriver.com> wrote:
> While building linux-libc-headers with QA package-missing checking enabled,
> there was warning:
> ...
> QA Issue: package linux-libc-headers not generated, but listed in PACKAGES
> ...
>
> Only linux-libc-headers-dev was generated, the others are empty, we should
> set PACKAGE manually.
>
> We also set linux-libc-headers-dev as linux-libc-headers's provider, so
> package management could install linux-libc-headers.
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
> meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
> index b18d09f..a69bbe6 100644
> --- a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
> +++ b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
> @@ -71,3 +71,5 @@ RRECOMMENDS_${PN}-dbg = "${PN}-dev (= ${EXTENDPKGV})"
>
> INHIBIT_DEFAULT_DEPS = "1"
> DEPENDS += "unifdef-native"
> +PACKAGES="${PN}-dev"
Fix spaces around =
> +RPROVIDES_${PN}-dev = "${PN}"
> --
> 1.9.1
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH V2 3/4] linux-libc-headers.inc: fix QA package-missing warnings
2014-12-25 1:48 ` [PATCH 3/4] linux-libc-headers.inc: " Hongxu Jia
2014-12-26 13:58 ` Otavio Salvador
@ 2014-12-29 8:16 ` Hongxu Jia
1 sibling, 0 replies; 15+ messages in thread
From: Hongxu Jia @ 2014-12-29 8:16 UTC (permalink / raw)
To: openembedded-core, richard.purdie, otavio
While building linux-libc-headers with QA package-missing checking enabled,
there was warning:
...
QA Issue: package linux-libc-headers not generated, but listed in PACKAGES
...
Only linux-libc-headers-dev was generated, the others are empty, we should
set PACKAGE manually.
We also set linux-libc-headers-dev as linux-libc-headers's provider, so
package management could install linux-libc-headers.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
| 2 ++
1 file changed, 2 insertions(+)
--git a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
index b18d09f..aaa02b1 100644
--- a/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
+++ b/meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc
@@ -71,3 +71,5 @@ RRECOMMENDS_${PN}-dbg = "${PN}-dev (= ${EXTENDPKGV})"
INHIBIT_DEFAULT_DEPS = "1"
DEPENDS += "unifdef-native"
+PACKAGES = "${PN}-dev"
+RPROVIDES_${PN}-dev = "${PN}"
--
1.9.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH 4/4] busybox.inc: fix QA package-missing warnings
2014-12-25 1:48 [PATCH V2 0/4] insane.bbclass: add QA check: package-missing Hongxu Jia
` (2 preceding siblings ...)
2014-12-25 1:48 ` [PATCH 3/4] linux-libc-headers.inc: " Hongxu Jia
@ 2014-12-25 1:48 ` Hongxu Jia
2014-12-26 13:59 ` Otavio Salvador
2014-12-29 8:16 ` [PATCH V2 " Hongxu Jia
2015-01-09 11:50 ` [PATCH V2 0/4] insane.bbclass: add QA check: package-missing Hongxu Jia
4 siblings, 2 replies; 15+ messages in thread
From: Hongxu Jia @ 2014-12-25 1:48 UTC (permalink / raw)
To: openembedded-core, richard.purdie
While building busybox with QA package-missing checking enabled,
there was warning:
...
WARNING: QA Issue: package busybox-mdev not generated, but listed in PACKAGES
WARNING: QA Issue: package busybox-httpd not generated, but listed in PACKAGES
...
The config in ${B}/.config decides to generate busybox-mdev,
busybox-httpd or not. Files are copied if the packages generated.
So set PACKAGES in do_package according to the existance of files.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-core/busybox/busybox.inc | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes-core/busybox/busybox.inc
index 0769d92..0035f12 100644
--- a/meta/recipes-core/busybox/busybox.inc
+++ b/meta/recipes-core/busybox/busybox.inc
@@ -317,6 +317,16 @@ python () {
}
python do_package_prepend () {
+ if not os.path.exists(d.expand("${D}${sysconfdir}/init.d/busybox-httpd")):
+ packages = (d.getVar('PACKAGES', True) or '').split()
+ packages.remove(d.expand('${PN}-httpd'))
+ d.setVar('PACKAGES', ' '.join(packages))
+
+ if not os.path.exists(d.expand("${D}${sysconfdir}/init.d/mdev")):
+ packages = (d.getVar('PACKAGES', True) or '').split()
+ packages.remove(d.expand('${PN}-mdev'))
+ d.setVar('PACKAGES', ' '.join(packages))
+
# We need to load the full set of busybox provides from the /etc/busybox.links
# Use this to see the update-alternatives with the right information
--
1.9.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH 4/4] busybox.inc: fix QA package-missing warnings
2014-12-25 1:48 ` [PATCH 4/4] busybox.inc: " Hongxu Jia
@ 2014-12-26 13:59 ` Otavio Salvador
2014-12-27 14:00 ` Mike Looijmans
2014-12-29 8:16 ` [PATCH V2 " Hongxu Jia
1 sibling, 1 reply; 15+ messages in thread
From: Otavio Salvador @ 2014-12-26 13:59 UTC (permalink / raw)
To: Hongxu Jia; +Cc: Patches and discussions about the oe-core layer
On Wed, Dec 24, 2014 at 11:48 PM, Hongxu Jia <hongxu.jia@windriver.com> wrote:
> While building busybox with QA package-missing checking enabled,
> there was warning:
> ...
> WARNING: QA Issue: package busybox-mdev not generated, but listed in PACKAGES
> WARNING: QA Issue: package busybox-httpd not generated, but listed in PACKAGES
> ...
>
> The config in ${B}/.config decides to generate busybox-mdev,
> busybox-httpd or not. Files are copied if the packages generated.
> So set PACKAGES in do_package according to the existance of files.
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
Shouldn't we "invert" the logic here? This patch 'removes' httpd and
mdev when those are not enabled. Why we don't /add/ those when they
are enabled?
--
Otavio Salvador O.S. Systems
http://www.ossystems.com.br http://code.ossystems.com.br
Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH 4/4] busybox.inc: fix QA package-missing warnings
2014-12-26 13:59 ` Otavio Salvador
@ 2014-12-27 14:00 ` Mike Looijmans
2014-12-27 14:28 ` Richard Purdie
0 siblings, 1 reply; 15+ messages in thread
From: Mike Looijmans @ 2014-12-27 14:00 UTC (permalink / raw)
To: openembedded-core
On 12/26/2014 02:59 PM, Otavio Salvador wrote:
> On Wed, Dec 24, 2014 at 11:48 PM, Hongxu Jia <hongxu.jia@windriver.com> wrote:
>> While building busybox with QA package-missing checking enabled,
>> there was warning:
>> ...
>> WARNING: QA Issue: package busybox-mdev not generated, but listed in PACKAGES
>> WARNING: QA Issue: package busybox-httpd not generated, but listed in PACKAGES
>> ...
>>
>> The config in ${B}/.config decides to generate busybox-mdev,
>> busybox-httpd or not. Files are copied if the packages generated.
>> So set PACKAGES in do_package according to the existance of files.
>>
>> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
>
> Shouldn't we "invert" the logic here? This patch 'removes' httpd and
> mdev when those are not enabled. Why we don't /add/ those when they
> are enabled?
maybe the answer is in a question:
What would happen then if someone typed "bitbake busybox-mdev"?
--
Mike Looijmans
^ permalink raw reply [flat|nested] 15+ messages in thread* Re: [PATCH 4/4] busybox.inc: fix QA package-missing warnings
2014-12-27 14:00 ` Mike Looijmans
@ 2014-12-27 14:28 ` Richard Purdie
0 siblings, 0 replies; 15+ messages in thread
From: Richard Purdie @ 2014-12-27 14:28 UTC (permalink / raw)
To: Mike Looijmans; +Cc: openembedded-core
On Sat, 2014-12-27 at 15:00 +0100, Mike Looijmans wrote:
> On 12/26/2014 02:59 PM, Otavio Salvador wrote:
> > On Wed, Dec 24, 2014 at 11:48 PM, Hongxu Jia <hongxu.jia@windriver.com> wrote:
> >> While building busybox with QA package-missing checking enabled,
> >> there was warning:
> >> ...
> >> WARNING: QA Issue: package busybox-mdev not generated, but listed in PACKAGES
> >> WARNING: QA Issue: package busybox-httpd not generated, but listed in PACKAGES
> >> ...
> >>
> >> The config in ${B}/.config decides to generate busybox-mdev,
> >> busybox-httpd or not. Files are copied if the packages generated.
> >> So set PACKAGES in do_package according to the existance of files.
> >>
> >> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> >
> > Shouldn't we "invert" the logic here? This patch 'removes' httpd and
> > mdev when those are not enabled. Why we don't /add/ those when they
> > are enabled?
>
> maybe the answer is in a question:
> What would happen then if someone typed "bitbake busybox-mdev"?
You'd get an error regardless since "busybox-mdev" isn't a recipe, its a
package and bitbake doesn't take package names.
Cheers,
Richard
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH V2 4/4] busybox.inc: fix QA package-missing warnings
2014-12-25 1:48 ` [PATCH 4/4] busybox.inc: " Hongxu Jia
2014-12-26 13:59 ` Otavio Salvador
@ 2014-12-29 8:16 ` Hongxu Jia
2014-12-29 8:19 ` Hongxu Jia
2014-12-29 10:45 ` [PATCH V3 " Hongxu Jia
1 sibling, 2 replies; 15+ messages in thread
From: Hongxu Jia @ 2014-12-29 8:16 UTC (permalink / raw)
To: openembedded-core, richard.purdie, otavio
While building busybox with QA package-missing checking enabled,
there was warning:
...
WARNING: QA Issue: package busybox-mdev not generated, but listed in PACKAGES
WARNING: QA Issue: package busybox-httpd not generated, but listed in PACKAGES
...
The config in ${B}/.config decides to generate busybox-mdev, busybox-hwclock,
busybox-udhcpd, busybox-httpd or not. Files are copied if the packages generated.
Set PACKAGES in do_package according to the existance of these files, so we could
dynamic generate these packages according to the configuration.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-core/busybox/busybox.inc | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes-core/busybox/busybox.inc
index 0769d92..95c3b0f 100644
--- a/meta/recipes-core/busybox/busybox.inc
+++ b/meta/recipes-core/busybox/busybox.inc
@@ -19,7 +19,7 @@ export EXTRA_CFLAGS = "${CFLAGS}"
export EXTRA_LDFLAGS = "${LDFLAGS}"
export EXTRA_OEMAKE += "'LD=${CCLD}'"
-PACKAGES =+ "${PN}-httpd ${PN}-udhcpd ${PN}-udhcpc ${PN}-syslog ${PN}-mdev ${PN}-hwclock"
+PACKAGES =+ "${PN}-udhcpc ${PN}-syslog"
FILES_${PN}-httpd = "${sysconfdir}/init.d/busybox-httpd /srv/www"
FILES_${PN}-syslog = "${sysconfdir}/init.d/syslog* ${sysconfdir}/syslog-startup.conf* ${sysconfdir}/syslog.conf* ${systemd_unitdir}/system/syslog.service ${sysconfdir}/default/busybox-syslog"
@@ -317,6 +317,18 @@ python () {
}
python do_package_prepend () {
+ if os.path.exists(d.expand("${D}${sysconfdir}/init.d/busybox-httpd")):
+ d.prependVar('PACKAGES', d.expand('${PN}-httpd '))
+
+ if os.path.exists(d.expand("${D}${sysconfdir}/init.d/hwclock.sh")):
+ d.prependVar('PACKAGES', d.expand('${PN}-hwclock '))
+
+ if os.path.exists(d.expand("${D}${sysconfdir}/init.d/busybox-udhcpd")):
+ d.prependVar('PACKAGES', d.expand('${PN}-udhcpd '))
+
+ if os.path.exists(d.expand("${D}${sysconfdir}/init.d/mdev")):
+ d.prependVar('PACKAGES', d.expand('${PN}-mdev '))
+
# We need to load the full set of busybox provides from the /etc/busybox.links
# Use this to see the update-alternatives with the right information
--
1.9.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH V2 4/4] busybox.inc: fix QA package-missing warnings
2014-12-29 8:16 ` [PATCH V2 " Hongxu Jia
@ 2014-12-29 8:19 ` Hongxu Jia
2014-12-29 10:45 ` [PATCH V3 " Hongxu Jia
1 sibling, 0 replies; 15+ messages in thread
From: Hongxu Jia @ 2014-12-29 8:19 UTC (permalink / raw)
To: openembedded-core, richard.purdie, otavio
BTW, just update the git repo in this thread:
git://git.pokylinux.org/poky-contrib hongxu/package-missing
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/package-missing
//Hongxu
On 12/29/2014 04:16 PM, Hongxu Jia wrote:
> While building busybox with QA package-missing checking enabled,
> there was warning:
> ...
> WARNING: QA Issue: package busybox-mdev not generated, but listed in PACKAGES
> WARNING: QA Issue: package busybox-httpd not generated, but listed in PACKAGES
> ...
>
> The config in ${B}/.config decides to generate busybox-mdev, busybox-hwclock,
> busybox-udhcpd, busybox-httpd or not. Files are copied if the packages generated.
> Set PACKAGES in do_package according to the existance of these files, so we could
> dynamic generate these packages according to the configuration.
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
> meta/recipes-core/busybox/busybox.inc | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes-core/busybox/busybox.inc
> index 0769d92..95c3b0f 100644
> --- a/meta/recipes-core/busybox/busybox.inc
> +++ b/meta/recipes-core/busybox/busybox.inc
> @@ -19,7 +19,7 @@ export EXTRA_CFLAGS = "${CFLAGS}"
> export EXTRA_LDFLAGS = "${LDFLAGS}"
> export EXTRA_OEMAKE += "'LD=${CCLD}'"
>
> -PACKAGES =+ "${PN}-httpd ${PN}-udhcpd ${PN}-udhcpc ${PN}-syslog ${PN}-mdev ${PN}-hwclock"
> +PACKAGES =+ "${PN}-udhcpc ${PN}-syslog"
>
> FILES_${PN}-httpd = "${sysconfdir}/init.d/busybox-httpd /srv/www"
> FILES_${PN}-syslog = "${sysconfdir}/init.d/syslog* ${sysconfdir}/syslog-startup.conf* ${sysconfdir}/syslog.conf* ${systemd_unitdir}/system/syslog.service ${sysconfdir}/default/busybox-syslog"
> @@ -317,6 +317,18 @@ python () {
> }
>
> python do_package_prepend () {
> + if os.path.exists(d.expand("${D}${sysconfdir}/init.d/busybox-httpd")):
> + d.prependVar('PACKAGES', d.expand('${PN}-httpd '))
> +
> + if os.path.exists(d.expand("${D}${sysconfdir}/init.d/hwclock.sh")):
> + d.prependVar('PACKAGES', d.expand('${PN}-hwclock '))
> +
> + if os.path.exists(d.expand("${D}${sysconfdir}/init.d/busybox-udhcpd")):
> + d.prependVar('PACKAGES', d.expand('${PN}-udhcpd '))
> +
> + if os.path.exists(d.expand("${D}${sysconfdir}/init.d/mdev")):
> + d.prependVar('PACKAGES', d.expand('${PN}-mdev '))
> +
> # We need to load the full set of busybox provides from the /etc/busybox.links
> # Use this to see the update-alternatives with the right information
>
^ permalink raw reply [flat|nested] 15+ messages in thread* [PATCH V3 4/4] busybox.inc: fix QA package-missing warnings
2014-12-29 8:16 ` [PATCH V2 " Hongxu Jia
2014-12-29 8:19 ` Hongxu Jia
@ 2014-12-29 10:45 ` Hongxu Jia
2014-12-29 10:50 ` Hongxu Jia
1 sibling, 1 reply; 15+ messages in thread
From: Hongxu Jia @ 2014-12-29 10:45 UTC (permalink / raw)
To: openembedded-core, richard.purdie, otavio
While building busybox with QA package-missing checking enabled,
there was warning:
...
WARNING: QA Issue: package busybox-mdev not generated, but listed in PACKAGES
WARNING: QA Issue: package busybox-httpd not generated, but listed in PACKAGES
...
The config in ${B}/.config decides to generate busybox-mdev, busybox-hwclock,
busybox-udhcpd, busybox-httpd, busybox-udhcpc, busybox-syslog or not. Files
are copied if the packages generated. Set PACKAGES in do_package according
to the existance of these files, so we could dynamicly generate these packages
according to the configuration.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
---
meta/recipes-core/busybox/busybox.inc | 20 +++++++++++++++++++-
1 file changed, 19 insertions(+), 1 deletion(-)
diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes-core/busybox/busybox.inc
index 0769d92..7b23190 100644
--- a/meta/recipes-core/busybox/busybox.inc
+++ b/meta/recipes-core/busybox/busybox.inc
@@ -19,7 +19,7 @@ export EXTRA_CFLAGS = "${CFLAGS}"
export EXTRA_LDFLAGS = "${LDFLAGS}"
export EXTRA_OEMAKE += "'LD=${CCLD}'"
-PACKAGES =+ "${PN}-httpd ${PN}-udhcpd ${PN}-udhcpc ${PN}-syslog ${PN}-mdev ${PN}-hwclock"
+PACKAGES_DYNAMIC += "^${PN}-(udhcpd|hwclock|mdev|httpd|syslog|udhcpc)"
FILES_${PN}-httpd = "${sysconfdir}/init.d/busybox-httpd /srv/www"
FILES_${PN}-syslog = "${sysconfdir}/init.d/syslog* ${sysconfdir}/syslog-startup.conf* ${sysconfdir}/syslog.conf* ${systemd_unitdir}/system/syslog.service ${sysconfdir}/default/busybox-syslog"
@@ -317,6 +317,24 @@ python () {
}
python do_package_prepend () {
+ if os.path.exists(d.expand("${D}${sysconfdir}/udhcpc.d")):
+ d.prependVar('PACKAGES', d.expand('${PN}-udhcpc '))
+
+ if os.path.exists(d.expand("${D}${sysconfdir}/init.d/syslog.${BPN}")):
+ d.prependVar('PACKAGES', d.expand('${PN}-syslog '))
+
+ if os.path.exists(d.expand("${D}${sysconfdir}/init.d/busybox-httpd")):
+ d.prependVar('PACKAGES', d.expand('${PN}-httpd '))
+
+ if os.path.exists(d.expand("${D}${sysconfdir}/init.d/hwclock.sh")):
+ d.prependVar('PACKAGES', d.expand('${PN}-hwclock '))
+
+ if os.path.exists(d.expand("${D}${sysconfdir}/init.d/busybox-udhcpd")):
+ d.prependVar('PACKAGES', d.expand('${PN}-udhcpd '))
+
+ if os.path.exists(d.expand("${D}${sysconfdir}/init.d/mdev")):
+ d.prependVar('PACKAGES', d.expand('${PN}-mdev '))
+
# We need to load the full set of busybox provides from the /etc/busybox.links
# Use this to see the update-alternatives with the right information
--
1.9.1
^ permalink raw reply related [flat|nested] 15+ messages in thread* Re: [PATCH V3 4/4] busybox.inc: fix QA package-missing warnings
2014-12-29 10:45 ` [PATCH V3 " Hongxu Jia
@ 2014-12-29 10:50 ` Hongxu Jia
0 siblings, 0 replies; 15+ messages in thread
From: Hongxu Jia @ 2014-12-29 10:50 UTC (permalink / raw)
To: openembedded-core, richard.purdie, otavio
Changed in V3:
There is a failure to build core-image-minimal while
IMAGE_INSTALL_append = " busybox-udhcpd"
...
NOTE: Runtime target 'busybox-udhcpd' is unbuildable, removing...
Missing or unbuildable dependency chain was: ['busybox-udhcpd']
ERROR: Required build target 'core-image-minimal' has no buildable
providers.
Missing or unbuildable dependency chain was: ['core-image-minimal',
'busybox-udhcpd']
...
Treat these as dynamic packages could fix this issue.
//Hongxu
On 12/29/2014 06:45 PM, Hongxu Jia wrote:
> While building busybox with QA package-missing checking enabled,
> there was warning:
> ...
> WARNING: QA Issue: package busybox-mdev not generated, but listed in PACKAGES
> WARNING: QA Issue: package busybox-httpd not generated, but listed in PACKAGES
> ...
>
> The config in ${B}/.config decides to generate busybox-mdev, busybox-hwclock,
> busybox-udhcpd, busybox-httpd, busybox-udhcpc, busybox-syslog or not. Files
> are copied if the packages generated. Set PACKAGES in do_package according
> to the existance of these files, so we could dynamicly generate these packages
> according to the configuration.
>
> Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com>
> ---
> meta/recipes-core/busybox/busybox.inc | 20 +++++++++++++++++++-
> 1 file changed, 19 insertions(+), 1 deletion(-)
>
> diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes-core/busybox/busybox.inc
> index 0769d92..7b23190 100644
> --- a/meta/recipes-core/busybox/busybox.inc
> +++ b/meta/recipes-core/busybox/busybox.inc
> @@ -19,7 +19,7 @@ export EXTRA_CFLAGS = "${CFLAGS}"
> export EXTRA_LDFLAGS = "${LDFLAGS}"
> export EXTRA_OEMAKE += "'LD=${CCLD}'"
>
> -PACKAGES =+ "${PN}-httpd ${PN}-udhcpd ${PN}-udhcpc ${PN}-syslog ${PN}-mdev ${PN}-hwclock"
> +PACKAGES_DYNAMIC += "^${PN}-(udhcpd|hwclock|mdev|httpd|syslog|udhcpc)"
>
> FILES_${PN}-httpd = "${sysconfdir}/init.d/busybox-httpd /srv/www"
> FILES_${PN}-syslog = "${sysconfdir}/init.d/syslog* ${sysconfdir}/syslog-startup.conf* ${sysconfdir}/syslog.conf* ${systemd_unitdir}/system/syslog.service ${sysconfdir}/default/busybox-syslog"
> @@ -317,6 +317,24 @@ python () {
> }
>
> python do_package_prepend () {
> + if os.path.exists(d.expand("${D}${sysconfdir}/udhcpc.d")):
> + d.prependVar('PACKAGES', d.expand('${PN}-udhcpc '))
> +
> + if os.path.exists(d.expand("${D}${sysconfdir}/init.d/syslog.${BPN}")):
> + d.prependVar('PACKAGES', d.expand('${PN}-syslog '))
> +
> + if os.path.exists(d.expand("${D}${sysconfdir}/init.d/busybox-httpd")):
> + d.prependVar('PACKAGES', d.expand('${PN}-httpd '))
> +
> + if os.path.exists(d.expand("${D}${sysconfdir}/init.d/hwclock.sh")):
> + d.prependVar('PACKAGES', d.expand('${PN}-hwclock '))
> +
> + if os.path.exists(d.expand("${D}${sysconfdir}/init.d/busybox-udhcpd")):
> + d.prependVar('PACKAGES', d.expand('${PN}-udhcpd '))
> +
> + if os.path.exists(d.expand("${D}${sysconfdir}/init.d/mdev")):
> + d.prependVar('PACKAGES', d.expand('${PN}-mdev '))
> +
> # We need to load the full set of busybox provides from the /etc/busybox.links
> # Use this to see the update-alternatives with the right information
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH V2 0/4] insane.bbclass: add QA check: package-missing
2014-12-25 1:48 [PATCH V2 0/4] insane.bbclass: add QA check: package-missing Hongxu Jia
` (3 preceding siblings ...)
2014-12-25 1:48 ` [PATCH 4/4] busybox.inc: " Hongxu Jia
@ 2015-01-09 11:50 ` Hongxu Jia
4 siblings, 0 replies; 15+ messages in thread
From: Hongxu Jia @ 2015-01-09 11:50 UTC (permalink / raw)
To: openembedded-core, richard.purdie
Ping
[PATCH 1/4] insane.bbclass: add QA check: package-missing
[PATCH 2/4] perl_5.20.0: fix QA package-missing warnings
[PATCH V2 3/4] linux-libc-headers.inc: fix QA package-missing warnings
[PATCH V3 4/4] busybox.inc: fix QA package-missing warnings
//Hongxu
On 12/25/2014 09:48 AM, Hongxu Jia wrote:
> Changed in V2:
> Follow Richard Purdie suggestions, use packageinfo which saved in
> $PKGDATA_DIR to collect all available packages RDEPENDS and RPROVIDES
> rather than bb.persist_data.
>
> V1:
> We also tried to fix perl, linux-libc-headers and busybox according to
> the result of QA checking.
>
> Here is a test to explain what the QA check showing, we could figure out
> package missing issue at package generation time rather than do_rootfs
> time.
>
> Test Steps:
>
> 1. Edit xz and gzip recipe, to simulate a package missing situation (the manually added
> rdepends is not real but):
> ..............
> diff --git a/meta/recipes-extended/gzip/gzip.inc b/meta/recipes-extended/gzip/gzip.inc
> index 671b1eb..7e643e5 100644
> --- a/meta/recipes-extended/gzip/gzip.inc
> +++ b/meta/recipes-extended/gzip/gzip.inc
> @@ -32,3 +32,8 @@ ALTERNATIVE_LINK_NAME[gzip] = "${base_bindir}/gzip"
> ALTERNATIVE_LINK_NAME[zcat] = "${base_bindir}/zcat"
>
> export CONFIG_SHELL="/bin/sh"
> +
> +PACKAGES =+ "${PN}-missing1 ${PN}-missing2"
> +FILES_${PN}-missing1 = ""
> +FILES_${PN}-missing2 = ""
> +RPROVIDES_${PN}-missing1 = "${PN}-missing"
> diff --git a/meta/recipes-extended/xz/xz_5.1.3alpha.bb b/meta/recipes-extended/xz/xz_5.1.3alpha.bb
> index 168e383..c5d8b57 100644
> --- a/meta/recipes-extended/xz/xz_5.1.3alpha.bb
> +++ b/meta/recipes-extended/xz/xz_5.1.3alpha.bb
> @@ -37,6 +37,9 @@ FILES_liblzma-dev = "${includedir}/lzma* ${libdir}/liblzma*${SOLIBSDEV} ${libdir
> FILES_liblzma-staticdev = "${libdir}/liblzma.a"
> FILES_liblzma-dbg = "${libdir}/.debug/liblzma*"
>
> +
> +RDEPENDS_${PN}_class-target += "gzip-missing gzip-missing2"
> +
> BBCLASSEXTEND = "native nativesdk"
>
> export CONFIG_SHELL="/bin/sh"
> ..............
>
> 2. Vim local.conf
> ...
> IMAGE_INSTALL_append = " xz"
> WARN_QA_append = " package-missing"
> ...
>
> 3. bitbake core-image-minimal
> ..............
> |WARNING: QA Issue: package gzip-missing1 not generated, but listed in PACKAGES
> |Compute rdepends chains, '-->' means 'runtime depends'
> |("gzip-missing1" rprovides "gzip-missing")
> |"xz" -> "gzip-missing"
> | [package-missing]
> |WARNING: QA Issue: package gzip-missing2 not generated, but listed in PACKAGES
> |Compute rdepends chains, '-->' means 'runtime depends'
> |"xz" -> "gzip-missing2"
> | [package-missing]
> |ERROR: Unable to install packages. Command '/home/jiahongxu/yocto/build-2014120
> 2-mips/tmp/sysroots/x86_64-linux/usr/bin/smart --quiet --data-dir=/home/jiahongx
> u/yocto/build-20141202-mips/tmp/work/qemumips-poky-linux/core-image-minimal/1.0
> -r0/rootfs/var/lib/smart install -y xz@mips32r2 run-postinsts@all packagegroup-
> core-boot@qemumips' returned 1:
> error: Can't install xz-5.1.3alpha-r0@mips32r2: no package provides gzip-missing
>
>
> |ERROR: Function failed: do_rootfs
> |ERROR: Logfile of failure stored in: /home/jiahongxu/yocto/build-20141202-mips/
> tmp/work/qemumips-poky-linux/core-image-minimal/1.0-r0/temp/log.do_rootfs.24338
> |ERROR: Task 7 (/home/jiahongxu/yocto/poky/meta/recipes-core/images/core-image-
> minimal.bb, do_rootfs) failed with exit code '1'
> ..............
>
> //Hongxu
>
> The following changes since commit 15de188692d2572d897799424e2a22eab24c78e2:
>
> kernel-devsrc: Ensure we have a dependency on the actual source (2014-12-21 13:52:17 +0000)
>
> are available in the git repository at:
>
> git://git.pokylinux.org/poky-contrib hongxu/package-missing
> http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=hongxu/package-missing
>
> Hongxu Jia (4):
> insane.bbclass: add QA check: package-missing
> perl_5.20.0: fix QA package-missing warnings
> linux-libc-headers.inc: fix QA package-missing warnings
> busybox.inc: fix QA package-missing warnings
>
> meta/classes/insane.bbclass | 100 ++++++++++++++++++++-
> meta/recipes-core/busybox/busybox.inc | 10 +++
> meta/recipes-devtools/perl/perl_5.20.0.bb | 10 ++-
> .../linux-libc-headers/linux-libc-headers.inc | 2 +
> 4 files changed, 117 insertions(+), 5 deletions(-)
>
^ permalink raw reply [flat|nested] 15+ messages in thread