* Bitbake output format changed Was: [oe-commits] [bitbake] branch master updated (a3f464d -> 0219271) [not found] <20160615073901.27831.65435@opal.openembedded.org> @ 2016-06-22 15:58 ` Martin Jansa 2016-06-22 21:35 ` Richard Purdie 2016-06-23 10:58 ` [PATCH] test-dependencies.sh: Strip also '\.bb; .*' before adding failed recipe to list of failed Martin Jansa 0 siblings, 2 replies; 7+ messages in thread From: Martin Jansa @ 2016-06-22 15:58 UTC (permalink / raw) To: openembedded-core, Richard Purdie, bitbake-devel; +Cc: openembedded-commits [-- Attachment #1: Type: text/plain, Size: 17909 bytes --] On Wed, Jun 15, 2016 at 07:39:01AM +0000, git@git.openembedded.org wrote: > rpurdie pushed a change to branch master > in repository bitbake. > > from a3f464d cooker.py: fix warn() -> warning() > adds 81798ab toaster: testsjs Add call back to append elements on completion of tests > adds 5d4172b toaster: tweaks to recipe file downloads > adds 2cafb16 toaster: tests Add a BuildTest helper class > adds 0748177 toaster: migrate typeahead library > adds afbaf32 toaster: work-around our lack of a synchronous fetch for typeaheads > adds 69527a7 toaster: Migrate project configuration from bootstrap 2 to bootstrap 3 > adds 9f21a17 toaster: toaster tables Enable complex empty states > adds 4c0e07b toaster: ToasterTable widget add an 'a' to Name in Exception class > adds df62f38 toaster: port table for Built packages to ToasterTable > adds 9434d39 toaster: port Built recipes table to toastertables > adds bebcef7 toaster: port Task tables to ToasterTables widget > adds 2418c09 toaster: port Installed packages table to ToasterTable > adds 927bfc2 toaster: table.js Add the ability to highlight a particular row > adds 7cf2367 toaster: tests Add new build tables to tests > adds 566b4fb toaster: widgets ToasterTable Catch template rendering exceptions > adds 31e7c26 toaster: Rework displaying package dependencies across Toaster > adds 3f61bef toaster-tests: maximize browser window when running UI tests > adds 37fb9a3 toaster: fix typo which prevents filters from working > adds 7c377e1 toaster: buildtables Remove links from non name fields > adds f963b73 toaster: port all build analysis pages to bootstrap 3 > adds 3d7f5d6 toaster: build data Fix left navigation > adds 70f600f toaster: BuiltPackagesTable format empty state in packages table > adds 4e21817 toaster: BuildTasksTable filters remove outcome NA option > adds e795154 toaster: bin Use python 3 for our django modules check > adds 8ae2fc2 toaster: replace viewkeys() -> keys() > adds bfb4488 toaster: fix migrations > adds 3566ecc toaster: decode response content > adds eb23a84 toaster: fix incorrect file mode > adds 1626e98 toaster: fix test_toaster_tables > adds f66ee68 toaster: fix broken reference to urllib > adds 571c2b7 toaster-tests: fix tests for latest Selenium version > adds c4d6796 toaster: open image files in binary mode when sending in response > adds d34546e toaster: toastergui tests Use new BeautifulSoup syntax > adds 3638b8e toaster: changed python version in shebang > adds c5a4893 toaster: fix wrong usage of print_exc and format_exc > adds d382b2a toaster: js tests Twitter typeahead library object is now ttTypeahead > adds 830743b toaster: selenium tests Update unique custom image string and fix import > adds fccc886 toaster: selenium tests Fix all_projects page and sample tests > adds 0bdfcaf toaster: tests Rename test settings to python compliant name and fix import > adds 5f80dac toaster: tests browser Fix selenium tests after bootstrap3 breakage > adds 8c1ed57 runqueue: Change buildable/running lists to sets > adds 2c88afb taskdata/runqueue: Rewrite without use of ID indirection Is it expected that this change also changes the format of summary shown at the end of the build and failed tasks. The commit message doesn't mention this (it even says: .. On the most part there shouldn't be user visible changes. .. There should be no functionality changes in this patch, its purely a data structure change and that is visible in the patch. .. ) So before I start fixing scripts (e.g. test-dependencies) which are trying to parse bitbake output to work with new format, please confirm that this was intentional and that it will stay this way. It's also not clear why the task is mentioned twice, e.g. as: NOTE: Running task 512 of 548 (ID: /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch, /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit code '1' are there cases where these 2 are different? and can both messages use the same "format" for task description? Thanks --- /OE/failures.full.before 2016-06-22 17:49:20.332082880 +0200 +++ /OE/failures.full.after 2016-06-22 17:48:57.018084068 +0200 @@ -1,4 +1,4 @@ -NOTE: Started PRServer with DBfile: /OE/build/oe-core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 34615, PID: 9540 +NOTE: Started PRServer with DBfile: /OE/build/oe-core/cache/prserv.sqlite3, IP: 127.0.0.1, PORT: 36143, PID: 9266 Loading cache...done. Loaded 1327 entries from dependency cache. Parsing recipes...done. @@ -20,7 +20,7 @@ NOTE: Preparing RunQueue NOTE: Executing SetScene Tasks NOTE: Executing RunQueue Tasks -NOTE: Running task 512 of 548 (ID: 4, /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb, do_fetch) +NOTE: Running task 512 of 548 (ID: /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch, /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) NOTE: recipe sftp-1.0-r0: task do_fetch: Started WARNING: sftp-1.0-r0 do_fetch: Failed to fetch URL sftp://jama@garnet.openembedded.org~/setup_jenkins.sh;name=setup, attempting MIRRORS if available ERROR: sftp-1.0-r0 do_fetch: Fetcher failure: Fetch command failed with exit code 255, output: @@ -28,15 +28,15 @@ Couldn't read packet: Connection reset by peer ERROR: sftp-1.0-r0 do_fetch: Function failed: Fetcher failure for URL: 'sftp://jama@garnet.openembedded.org~/setup_jenkins.sh;name=setup'. Unable to fetch URL from any source. -ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/i586-oe-linux/sftp/1.0-r0/temp/log.do_fetch.9611 +ERROR: Logfile of failure stored in: /OE/build/oe-core/tmp-glibc/work/i586-oe-linux/sftp/1.0-r0/temp/log.do_fetch.9339 NOTE: recipe sftp-1.0-r0: task do_fetch: Failed -ERROR: Task 4 (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb, do_fetch) failed with exit code '1' +ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit code '1' NOTE: Tasks Summary: Attempted 534 tasks of which 533 didn't need to be rerun and 1 failed. NOTE: Writing buildhistory -NOTE: Uploading the report /OE/build/oe-core/tmp-glibc/log/error-report/error_report_20160622154916.txt to errors.yoctoproject.org:80 -NOTE: Report submitted: 200 Your entry can be found here: http://errors.yoctoproject.org/Errors/Build/17616/ +NOTE: Uploading the report /OE/build/oe-core/tmp-glibc/log/error-report/error_report_20160622154853.txt to errors.yoctoproject.org:80 +NOTE: Report submitted: 200 Your entry can be found here: http://errors.yoctoproject.org/Errors/Build/17615/ Summary: 1 task failed: - /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb, do_fetch + /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch Summary: There was 1 WARNING message shown. Summary: There were 2 ERROR messages shown, returning a non-zero exit code. > adds 8aa84ab toaster: Remove mismerged files > adds 0219271 parse/ast, event: Ensure we reset registered handlers during parsing > > No new revisions were added by this update. > > Summary of changes: > LICENSE | 5 +- > bin/toaster | 2 +- > lib/bb/cooker.py | 72 +- > lib/bb/event.py | 7 + > lib/bb/parse/ast.py | 4 +- > lib/bb/runqueue.py | 1011 ++- > lib/bb/taskdata.py | 412 +- > lib/bb/ui/buildinfohelper.py | 2 +- > .../management/commands/checksettings.py | 2 +- > .../bldcontrol/management/commands/runbuilds.py | 4 +- > .../migrations/0004_auto_20160523_1446.py | 6 +- > lib/toaster/contrib/tts/runner.py | 4 +- > lib/toaster/manage.py | 2 +- > .../orm/migrations/0007_auto_20160523_1446.py | 20 +- > lib/toaster/orm/models.py | 63 +- > lib/toaster/tests/browser/README | 7 +- > lib/toaster/tests/browser/selenium_helpers.py | 1 + > lib/toaster/tests/browser/test_all_builds_page.py | 15 +- > .../tests/browser/test_all_projects_page.py | 13 +- > .../tests/browser/test_builddashboard_page.py | 33 +- > lib/toaster/tests/browser/test_js_unit_tests.py | 8 +- > .../tests/browser/test_new_custom_image_page.py | 3 +- > .../tests/browser/test_project_config_page.py | 5 +- > lib/toaster/tests/browser/test_sample.py | 2 +- > lib/toaster/tests/builds/README | 4 +- > lib/toaster/toastergui/buildtables.py | 568 ++ > .../fixtures/toastergui-unittest-data.xml | 91 +- > lib/toaster/toastergui/static/css/bootstrap.css | 6760 ++++++++++++++++++++ > .../toastergui/static/css/bootstrap.css.map | 1 + > .../toastergui/static/css/bootstrap.min.css | 13 +- > .../toastergui/static/css/bootstrap.min.css.map | 1 + > .../static/css/bootstrap3-transition.css | 238 + > lib/toaster/toastergui/static/css/default.css | 585 +- > .../static/fonts/glyphicons-halflings-regular.eot | Bin 20290 -> 20127 bytes > .../static/fonts/glyphicons-halflings-regular.svg | 487 +- > .../static/fonts/glyphicons-halflings-regular.ttf | Bin 41236 -> 45404 bytes > .../static/fonts/glyphicons-halflings-regular.woff | Bin 23292 -> 23424 bytes > .../fonts/glyphicons-halflings-regular.woff2 | Bin 0 -> 18028 bytes > .../toastergui/static/html/layer_deps_modal.html | 38 +- > .../static/img/glyphicons-halflings-white.png | Bin 8777 -> 0 bytes > .../toastergui/static/img/glyphicons-halflings.png | Bin 12799 -> 0 bytes > lib/toaster/toastergui/static/js/bootstrap.js | 2363 +++++++ > lib/toaster/toastergui/static/js/bootstrap.min.js | 11 +- > lib/toaster/toastergui/static/js/customrecipe.js | 4 +- > lib/toaster/toastergui/static/js/importlayer.js | 11 +- > lib/toaster/toastergui/static/js/layerDepsModal.js | 4 +- > lib/toaster/toastergui/static/js/layerdetails.js | 67 +- > lib/toaster/toastergui/static/js/libtoaster.js | 146 +- > .../toastergui/static/js/newcustomimage_modal.js | 76 +- > lib/toaster/toastergui/static/js/projectpage.js | 8 +- > lib/toaster/toastergui/static/js/table.js | 92 +- > lib/toaster/toastergui/static/js/tests/test.js | 9 +- > .../toastergui/static/js/typeahead.jquery.js | 1551 +++++ > lib/toaster/toastergui/tables.py | 201 +- > lib/toaster/toastergui/templates/base.html | 90 +- > .../toastergui/templates/basebuilddetailpage.html | 47 +- > .../toastergui/templates/basebuildpage.html | 118 +- > .../toastergui/templates/baseprojectpage.html | 38 +- > .../toastergui/templates/basetable_bottom.html | 26 +- > .../toastergui/templates/basetable_top.html | 121 +- > lib/toaster/toastergui/templates/bpackage.html | 108 - > .../toastergui/templates/builddashboard.html | 194 +- > .../templates/buildinfo-toastertable.html | 25 + > .../toastergui/templates/buildrequestdetails.html | 8 +- > .../toastergui/templates/builds-toastertable.html | 8 +- > lib/toaster/toastergui/templates/buildtime.html | 4 - > .../toastergui/templates/configuration.html | 57 +- > lib/toaster/toastergui/templates/configvars.html | 150 +- > .../toastergui/templates/customise_btn.html | 6 +- > lib/toaster/toastergui/templates/customrecipe.html | 205 +- > .../templates/detail_pagination_bottom.html | 22 +- > .../toastergui/templates/detail_search_header.html | 78 +- > .../toastergui/templates/detail_sorted_header.html | 8 +- > lib/toaster/toastergui/templates/dirinfo.html | 38 +- > .../templates/editcustomimage_modal.html | 67 +- > .../toastergui/templates/filtersnippet.html | 93 +- > .../templates/generic-toastertable-page.html | 2 +- > lib/toaster/toastergui/templates/importlayer.html | 260 +- > lib/toaster/toastergui/templates/landing.html | 23 +- > .../toastergui/templates/landing_not_managed.html | 4 +- > lib/toaster/toastergui/templates/layer_btn.html | 12 +- > lib/toaster/toastergui/templates/layerdetails.html | 457 +- > lib/toaster/toastergui/templates/machine_btn.html | 10 +- > lib/toaster/toastergui/templates/mrb_section.html | 121 +- > .../toastergui/templates/newcustomimage.html | 15 +- > .../toastergui/templates/newcustomimage_modal.html | 63 +- > lib/toaster/toastergui/templates/newproject.html | 51 +- > .../templates/package_built_dependencies.html | 16 +- > .../toastergui/templates/package_built_detail.html | 8 +- > .../toastergui/templates/package_detail_base.html | 44 +- > .../templates/package_included_dependencies.html | 20 +- > .../templates/package_included_tabs.html | 8 +- > .../toastergui/templates/pkg_add_rm_btn.html | 12 +- > lib/toaster/toastergui/templates/project.html | 102 +- > .../templates/projectbuilds-toastertable.html | 67 +- > .../toastergui/templates/projectbuilds.html | 4 +- > lib/toaster/toastergui/templates/projectconf.html | 1969 +++--- > .../templates/projects-toastertable.html | 58 +- > .../toastergui/templates/projecttopbar.html | 86 +- > lib/toaster/toastergui/templates/recipe.html | 115 +- > lib/toaster/toastergui/templates/recipe_btn.html | 13 +- > .../toastergui/templates/recipe_packages.html | 38 +- > .../toastergui/templates/recipedetails.html | 154 +- > lib/toaster/toastergui/templates/recipes.html | 113 - > .../templates/snippets/gitrev_popover.html | 8 + > .../snippets/pkg_dependencies_popover.html | 44 +- > .../snippets/pkg_revdependencies_popover.html | 46 +- > lib/toaster/toastergui/templates/target.html | 131 +- > lib/toaster/toastergui/templates/task.html | 171 +- > lib/toaster/toastergui/templates/tasks.html | 143 - > .../toastergui/templates/toastertable-filter.html | 32 +- > .../toastergui/templates/toastertable-simple.html | 94 +- > lib/toaster/toastergui/templates/toastertable.html | 122 +- > .../toastergui/templates/unavailable_artifact.html | 10 +- > lib/toaster/toastergui/templatetags/projecttags.py | 20 +- > lib/toaster/toastergui/tests.py | 90 +- > lib/toaster/toastergui/urls.py | 46 +- > lib/toaster/toastergui/views.py | 661 +- > lib/toaster/toastergui/widgets.py | 36 +- > .../{settings-test.py => settings_test.py} | 2 +- > 120 files changed, 16436 insertions(+), 5608 deletions(-) > rename bitbake/lib/toaster/bldcontrol/migrations/0004_auto_20160509_0946.py => lib/toaster/bldcontrol/migrations/0004_auto_20160523_1446.py (66%) > rename bitbake/lib/toaster/orm/migrations/0007_auto_20160509_0946.py => lib/toaster/orm/migrations/0007_auto_20160523_1446.py (69%) > create mode 100644 lib/toaster/toastergui/buildtables.py > create mode 100644 lib/toaster/toastergui/static/css/bootstrap.css > create mode 100644 lib/toaster/toastergui/static/css/bootstrap.css.map > create mode 100644 lib/toaster/toastergui/static/css/bootstrap.min.css.map > create mode 100644 lib/toaster/toastergui/static/css/bootstrap3-transition.css > create mode 100644 lib/toaster/toastergui/static/fonts/glyphicons-halflings-regular.woff2 > delete mode 100644 lib/toaster/toastergui/static/img/glyphicons-halflings-white.png > delete mode 100644 lib/toaster/toastergui/static/img/glyphicons-halflings.png > create mode 100644 lib/toaster/toastergui/static/js/bootstrap.js > create mode 100644 lib/toaster/toastergui/static/js/typeahead.jquery.js > delete mode 100644 lib/toaster/toastergui/templates/bpackage.html > create mode 100644 lib/toaster/toastergui/templates/buildinfo-toastertable.html > delete mode 100644 lib/toaster/toastergui/templates/buildtime.html > delete mode 100644 lib/toaster/toastergui/templates/recipes.html > create mode 100644 lib/toaster/toastergui/templates/snippets/gitrev_popover.html > delete mode 100644 lib/toaster/toastergui/templates/tasks.html > rename lib/toaster/toastermain/{settings-test.py => settings_test.py} (97%) > > -- > To stop receiving notification emails like this one, please contact > the administrator of this repository. > -- > _______________________________________________ > Openembedded-commits mailing list > Openembedded-commits@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-commits -- Martin 'JaMa' Jansa jabber: Martin.Jansa@gmail.com [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 188 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Bitbake output format changed Was: [oe-commits] [bitbake] branch master updated (a3f464d -> 0219271) 2016-06-22 15:58 ` Bitbake output format changed Was: [oe-commits] [bitbake] branch master updated (a3f464d -> 0219271) Martin Jansa @ 2016-06-22 21:35 ` Richard Purdie 2016-06-22 21:56 ` [bitbake-devel] " Mark Hatle 2016-06-23 10:58 ` [PATCH] test-dependencies.sh: Strip also '\.bb; .*' before adding failed recipe to list of failed Martin Jansa 1 sibling, 1 reply; 7+ messages in thread From: Richard Purdie @ 2016-06-22 21:35 UTC (permalink / raw) To: Martin Jansa, openembedded-core, bitbake-devel; +Cc: openembedded-commits On Wed, 2016-06-22 at 17:58 +0200, Martin Jansa wrote: > On Wed, Jun 15, 2016 at 07:39:01AM +0000, git@git.openembedded.org wr > ote: > > rpurdie pushed a change to branch master > > in repository bitbake. > > > > adds 2c88afb taskdata/runqueue: Rewrite without use of ID > > indirection > > Is it expected that this change also changes the format of summary > shown at the end of the build and failed tasks. > > The commit message doesn't mention this (it even says: > .. > On the most part there shouldn't be user visible changes. > .. > There should be no functionality changes in this patch, its purely a > data structure change and that is visible in the patch. > .. > ) > > So before I start fixing scripts (e.g. test-dependencies) which are > trying to parse bitbake output to work with new format, please > confirm that this was intentional and that it will stay this way. > > It's also not clear why the task is mentioned twice, e.g. as: > NOTE: Running task 512 of 548 (ID: /OE/build/oe-core/openembedded > -core/meta/recipes-devtools/rpm/sftp.bb:do_fetch, /OE/build/oe > -core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) > ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes > -devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded > -core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit > code '1' > > are there cases where these 2 are different? and can both messages > use the same "format" for task description? The taskids are no more, there is simply no notion of them inside bitbake any longer. That means that yes, we probably do need to change the output a bit since the number doesn't mean anything. Equally, it shouldn't be showing the same thing twice, that is a bug. Internally to bitbake, everthing is now in the form (or will be when multiconfig lands) "[multiconfig:<configname>:]<recipe filename>:<taskname>". The question is whether we expose that to the user or massage it at all. I'm tempted just to expose that string to the user. Sorry about the script changes, not sure we can avoid it though :( The code is *way* easier to understand, debug and extend after these changes FWIW so I do believe it very worth doing. Cheers, Richard ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Bitbake output format changed Was: [oe-commits] [bitbake] branch master updated (a3f464d -> 0219271) 2016-06-22 21:35 ` Richard Purdie @ 2016-06-22 21:56 ` Mark Hatle 0 siblings, 0 replies; 7+ messages in thread From: Mark Hatle @ 2016-06-22 21:56 UTC (permalink / raw) To: Richard Purdie, Martin Jansa, openembedded-core, bitbake-devel Cc: openembedded-commits On 6/22/16 4:35 PM, Richard Purdie wrote: > On Wed, 2016-06-22 at 17:58 +0200, Martin Jansa wrote: >> On Wed, Jun 15, 2016 at 07:39:01AM +0000, git@git.openembedded.org wr >> ote: >>> rpurdie pushed a change to branch master >>> in repository bitbake. >>> >>> adds 2c88afb taskdata/runqueue: Rewrite without use of ID >>> indirection >> >> Is it expected that this change also changes the format of summary >> shown at the end of the build and failed tasks. >> >> The commit message doesn't mention this (it even says: >> .. >> On the most part there shouldn't be user visible changes. >> .. >> There should be no functionality changes in this patch, its purely a >> data structure change and that is visible in the patch. >> .. >> ) >> >> So before I start fixing scripts (e.g. test-dependencies) which are >> trying to parse bitbake output to work with new format, please >> confirm that this was intentional and that it will stay this way. >> >> It's also not clear why the task is mentioned twice, e.g. as: >> NOTE: Running task 512 of 548 (ID: /OE/build/oe-core/openembedded >> -core/meta/recipes-devtools/rpm/sftp.bb:do_fetch, /OE/build/oe >> -core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) >> ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes >> -devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded >> -core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit >> code '1' >> >> are there cases where these 2 are different? and can both messages >> use the same "format" for task description? > > The taskids are no more, there is simply no notion of them inside > bitbake any longer. That means that yes, we probably do need to change > the output a bit since the number doesn't mean anything. > > Equally, it shouldn't be showing the same thing twice, that is a bug. > Internally to bitbake, everthing is now in the form (or will be when > multiconfig lands) "[multiconfig:<configname>:]<recipe > filename>:<taskname>". The question is whether we expose that to the > user or massage it at all. I'm tempted just to expose that string to > the user. I'm certainly and advocate of just exposing that string. I think in the end that is easier to explain to someone, then trying to understand a massaged output. --Mark > Sorry about the script changes, not sure we can avoid it though :( The > code is *way* easier to understand, debug and extend after these > changes FWIW so I do believe it very worth doing. > > Cheers, > > Richard > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [bitbake-devel] Bitbake output format changed Was: [oe-commits] [bitbake] branch master updated (a3f464d -> 0219271) @ 2016-06-22 21:56 ` Mark Hatle 0 siblings, 0 replies; 7+ messages in thread From: Mark Hatle @ 2016-06-22 21:56 UTC (permalink / raw) To: Richard Purdie, Martin Jansa, openembedded-core, bitbake-devel Cc: openembedded-commits On 6/22/16 4:35 PM, Richard Purdie wrote: > On Wed, 2016-06-22 at 17:58 +0200, Martin Jansa wrote: >> On Wed, Jun 15, 2016 at 07:39:01AM +0000, git@git.openembedded.org wr >> ote: >>> rpurdie pushed a change to branch master >>> in repository bitbake. >>> >>> adds 2c88afb taskdata/runqueue: Rewrite without use of ID >>> indirection >> >> Is it expected that this change also changes the format of summary >> shown at the end of the build and failed tasks. >> >> The commit message doesn't mention this (it even says: >> .. >> On the most part there shouldn't be user visible changes. >> .. >> There should be no functionality changes in this patch, its purely a >> data structure change and that is visible in the patch. >> .. >> ) >> >> So before I start fixing scripts (e.g. test-dependencies) which are >> trying to parse bitbake output to work with new format, please >> confirm that this was intentional and that it will stay this way. >> >> It's also not clear why the task is mentioned twice, e.g. as: >> NOTE: Running task 512 of 548 (ID: /OE/build/oe-core/openembedded >> -core/meta/recipes-devtools/rpm/sftp.bb:do_fetch, /OE/build/oe >> -core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) >> ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes >> -devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded >> -core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit >> code '1' >> >> are there cases where these 2 are different? and can both messages >> use the same "format" for task description? > > The taskids are no more, there is simply no notion of them inside > bitbake any longer. That means that yes, we probably do need to change > the output a bit since the number doesn't mean anything. > > Equally, it shouldn't be showing the same thing twice, that is a bug. > Internally to bitbake, everthing is now in the form (or will be when > multiconfig lands) "[multiconfig:<configname>:]<recipe > filename>:<taskname>". The question is whether we expose that to the > user or massage it at all. I'm tempted just to expose that string to > the user. I'm certainly and advocate of just exposing that string. I think in the end that is easier to explain to someone, then trying to understand a massaged output. --Mark > Sorry about the script changes, not sure we can avoid it though :( The > code is *way* easier to understand, debug and extend after these > changes FWIW so I do believe it very worth doing. > > Cheers, > > Richard > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Bitbake output format changed Was: [oe-commits] [bitbake] branch master updated (a3f464d -> 0219271) 2016-06-22 21:56 ` [bitbake-devel] " Mark Hatle @ 2016-06-22 22:07 ` Christopher Larson -1 siblings, 0 replies; 7+ messages in thread From: Christopher Larson @ 2016-06-22 22:07 UTC (permalink / raw) To: Mark Hatle Cc: openembedded-commits, bitbake-devel, Patches and discussions about the oe-core layer [-- Attachment #1: Type: text/plain, Size: 2795 bytes --] On Wed, Jun 22, 2016 at 2:56 PM, Mark Hatle <mark.hatle@windriver.com> wrote: > On 6/22/16 4:35 PM, Richard Purdie wrote: > > On Wed, 2016-06-22 at 17:58 +0200, Martin Jansa wrote: > >> On Wed, Jun 15, 2016 at 07:39:01AM +0000, git@git.openembedded.org wr > >> ote: > >>> rpurdie pushed a change to branch master > >>> in repository bitbake. > >>> > >>> adds 2c88afb taskdata/runqueue: Rewrite without use of ID > >>> indirection > >> > >> Is it expected that this change also changes the format of summary > >> shown at the end of the build and failed tasks. > >> > >> The commit message doesn't mention this (it even says: > >> .. > >> On the most part there shouldn't be user visible changes. > >> .. > >> There should be no functionality changes in this patch, its purely a > >> data structure change and that is visible in the patch. > >> .. > >> ) > >> > >> So before I start fixing scripts (e.g. test-dependencies) which are > >> trying to parse bitbake output to work with new format, please > >> confirm that this was intentional and that it will stay this way. > >> > >> It's also not clear why the task is mentioned twice, e.g. as: > >> NOTE: Running task 512 of 548 (ID: /OE/build/oe-core/openembedded > >> -core/meta/recipes-devtools/rpm/sftp.bb:do_fetch, /OE/build/oe > >> -core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) > >> ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes > >> -devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded > >> -core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit > >> code '1' > >> > >> are there cases where these 2 are different? and can both messages > >> use the same "format" for task description? > > > > The taskids are no more, there is simply no notion of them inside > > bitbake any longer. That means that yes, we probably do need to change > > the output a bit since the number doesn't mean anything. > > > > Equally, it shouldn't be showing the same thing twice, that is a bug. > > Internally to bitbake, everthing is now in the form (or will be when > > multiconfig lands) "[multiconfig:<configname>:]<recipe > > filename>:<taskname>". The question is whether we expose that to the > > user or massage it at all. I'm tempted just to expose that string to > > the user. > > I'm certainly and advocate of just exposing that string. I think in the > end > that is easier to explain to someone, then trying to understand a massaged > output. I'd agree with that. Using the filenames is actually a nice convenience when you need to examine a recipe. -- Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics [-- Attachment #2: Type: text/html, Size: 3726 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [bitbake-devel] Bitbake output format changed Was: [oe-commits] [bitbake] branch master updated (a3f464d -> 0219271) @ 2016-06-22 22:07 ` Christopher Larson 0 siblings, 0 replies; 7+ messages in thread From: Christopher Larson @ 2016-06-22 22:07 UTC (permalink / raw) To: Mark Hatle Cc: openembedded-commits, bitbake-devel, Patches and discussions about the oe-core layer [-- Attachment #1: Type: text/plain, Size: 2795 bytes --] On Wed, Jun 22, 2016 at 2:56 PM, Mark Hatle <mark.hatle@windriver.com> wrote: > On 6/22/16 4:35 PM, Richard Purdie wrote: > > On Wed, 2016-06-22 at 17:58 +0200, Martin Jansa wrote: > >> On Wed, Jun 15, 2016 at 07:39:01AM +0000, git@git.openembedded.org wr > >> ote: > >>> rpurdie pushed a change to branch master > >>> in repository bitbake. > >>> > >>> adds 2c88afb taskdata/runqueue: Rewrite without use of ID > >>> indirection > >> > >> Is it expected that this change also changes the format of summary > >> shown at the end of the build and failed tasks. > >> > >> The commit message doesn't mention this (it even says: > >> .. > >> On the most part there shouldn't be user visible changes. > >> .. > >> There should be no functionality changes in this patch, its purely a > >> data structure change and that is visible in the patch. > >> .. > >> ) > >> > >> So before I start fixing scripts (e.g. test-dependencies) which are > >> trying to parse bitbake output to work with new format, please > >> confirm that this was intentional and that it will stay this way. > >> > >> It's also not clear why the task is mentioned twice, e.g. as: > >> NOTE: Running task 512 of 548 (ID: /OE/build/oe-core/openembedded > >> -core/meta/recipes-devtools/rpm/sftp.bb:do_fetch, /OE/build/oe > >> -core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) > >> ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes > >> -devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded > >> -core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit > >> code '1' > >> > >> are there cases where these 2 are different? and can both messages > >> use the same "format" for task description? > > > > The taskids are no more, there is simply no notion of them inside > > bitbake any longer. That means that yes, we probably do need to change > > the output a bit since the number doesn't mean anything. > > > > Equally, it shouldn't be showing the same thing twice, that is a bug. > > Internally to bitbake, everthing is now in the form (or will be when > > multiconfig lands) "[multiconfig:<configname>:]<recipe > > filename>:<taskname>". The question is whether we expose that to the > > user or massage it at all. I'm tempted just to expose that string to > > the user. > > I'm certainly and advocate of just exposing that string. I think in the > end > that is easier to explain to someone, then trying to understand a massaged > output. I'd agree with that. Using the filenames is actually a nice convenience when you need to examine a recipe. -- Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics [-- Attachment #2: Type: text/html, Size: 3726 bytes --] ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH] test-dependencies.sh: Strip also '\.bb; .*' before adding failed recipe to list of failed 2016-06-22 15:58 ` Bitbake output format changed Was: [oe-commits] [bitbake] branch master updated (a3f464d -> 0219271) Martin Jansa 2016-06-22 21:35 ` Richard Purdie @ 2016-06-23 10:58 ` Martin Jansa 1 sibling, 0 replies; 7+ messages in thread From: Martin Jansa @ 2016-06-23 10:58 UTC (permalink / raw) To: openembedded-core * format of bitbake tasks changed in: 2c88afb taskdata/runqueue: Rewrite without use of ID indirection -ERROR: Task 4 (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb, do_fetch) failed with exit code '1' +ERROR: Task /OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch (/OE/build/oe-core/openembedded-core/meta/recipes-devtools/rpm/sftp.bb:do_fetch) failed with exit code '1' so strip not only '\.bb, .*' used before, but also '\.bb;.*' to drop the task name to get recipe name. * for more details see: http://lists.openembedded.org/pipermail/openembedded-core/2016-June/123132.html * without this change you can see test-dependencies.sh trying to rebuild packages like: Building recipe: fbprogress (6/21) Building recipe: fbprogress.bb:do (7/21) where the later of course doesn't exist as a recipe Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> --- scripts/test-dependencies.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/test-dependencies.sh b/scripts/test-dependencies.sh index 0170947..00c50e0 100755 --- a/scripts/test-dependencies.sh +++ b/scripts/test-dependencies.sh @@ -141,7 +141,7 @@ build_all() { bitbake -k $targets 2>&1 | tee -a ${OUTPUT1}/complete.log RESULT+=${PIPESTATUS[0]} grep "ERROR: Task.*failed" ${OUTPUT1}/complete.log > ${OUTPUT1}/failed-tasks.log - cat ${OUTPUT1}/failed-tasks.log | sed 's@.*/@@g; s@_.*@@g; s@\.bb, .*@@g' | sort -u > ${OUTPUT1}/failed-recipes.log + cat ${OUTPUT1}/failed-tasks.log | sed 's@.*/@@g; s@_.*@@g; s@\.bb, .*@@g; s@\.bb;.*@@g' | sort -u > ${OUTPUT1}/failed-recipes.log } build_every_recipe() { @@ -178,7 +178,7 @@ build_every_recipe() { RESULT+=${RECIPE_RESULT} mv ${OUTPUTB}/${recipe}.log ${OUTPUTB}/failed/ grep "ERROR: Task.*failed" ${OUTPUTB}/failed/${recipe}.log | tee -a ${OUTPUTB}/failed-tasks.log - grep "ERROR: Task.*failed" ${OUTPUTB}/failed/${recipe}.log | sed 's@.*/@@g; s@_.*@@g; s@\.bb, .*@@g' >> ${OUTPUTB}/failed-recipes.log + grep "ERROR: Task.*failed" ${OUTPUTB}/failed/${recipe}.log | sed 's@.*/@@g; s@_.*@@g; s@\.bb, .*@@g; s@\.bb;.*@@g' >> ${OUTPUTB}/failed-recipes.log # and append also ${recipe} in case the failed task was from some dependency echo ${recipe} >> ${OUTPUTB}/failed-recipes.log else -- 2.9.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2016-06-23 10:58 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20160615073901.27831.65435@opal.openembedded.org>
2016-06-22 15:58 ` Bitbake output format changed Was: [oe-commits] [bitbake] branch master updated (a3f464d -> 0219271) Martin Jansa
2016-06-22 21:35 ` Richard Purdie
2016-06-22 21:56 ` Mark Hatle
2016-06-22 21:56 ` [bitbake-devel] " Mark Hatle
2016-06-22 22:07 ` Christopher Larson
2016-06-22 22:07 ` [bitbake-devel] " Christopher Larson
2016-06-23 10:58 ` [PATCH] test-dependencies.sh: Strip also '\.bb; .*' before adding failed recipe to list of failed Martin Jansa
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.