From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: by yocto-www.yoctoproject.org (Postfix, from userid 118) id 29BBCE00CCC; Tue, 23 Feb 2016 04:58:54 -0800 (PST) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on yocto-www.yoctoproject.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 X-Spam-HAM-Report: * -5.0 RCVD_IN_DNSWL_HI RBL: Sender listed at http://www.dnswl.org/, high * trust * [134.134.136.65 listed in list.dnswl.org] * -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by yocto-www.yoctoproject.org (Postfix) with ESMTP id 55E93E00C79 for ; Tue, 23 Feb 2016 04:58:53 -0800 (PST) Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga103.jf.intel.com with ESMTP; 23 Feb 2016 04:58:55 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.22,489,1449561600"; d="scan'208";a="53161189" Received: from unknown (HELO lp.ger.corp.intel.com) ([10.252.4.192]) by fmsmga004.fm.intel.com with ESMTP; 23 Feb 2016 04:58:51 -0800 From: Elliot Smith To: toaster@yoctoproject.org Date: Tue, 23 Feb 2016 12:58:45 +0000 Message-Id: <1456232328-4319-1-git-send-email-elliot.smith@intel.com> X-Mailer: git-send-email 1.9.3 Subject: [PATCH 0/3][v4] Fix task buildstats gathering X-BeenThere: toaster@yoctoproject.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Web based interface for BitBake List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 Feb 2016 12:58:54 -0000 v4: I messed up the patch submission, so I'm sending the correct patches this time. v3: Only show a single navigation item in the left-hand nav panel for CPU times, but show both System and User time columns when it is selected. Add a test for CPU time subpage. v2: Store "started" and "ended" times for each task. (This will enable other improvements later, such as showing when tasks overlap each other.) Compute elapsed_time from start and end times, rather than in toaster.bbclass. Store disk IO in two extra fields, one for read and the other for write. Compute total disk_io from these two values. (elapsed_time and disk_io are still stored, as these are used for sorting and we can't easily sort on a computed field. This can be amended when we get round to putting the tasks table into ToasterTable; see https://bugzilla.yoctoproject.org/show_bug.cgi?id=8363.) Modify the UI to show the CPU times split between system and user. v1: Our buildstats gathering for tasks (CPU usage, Disk IO etc.) is broken, partly because of changes to how the data is organised in the buildstats files, and partly because tasks aren't being updated correctly anyway. Clean up the code which collects the buildstats (it had lots of duplication), collect the stats from the new stat file format (some line leaders had changed), and correctly identify the Task objects to update in BuildInfoHelper. To test: 1. Remove the content of the directories which cache tasks to see the best effects (I usually delete buildstats, tmp, sstate-cache, and cache, just to be safe). 2. Run a build against a local Yocto (as one patch is for toaster.bbclass). 3. When the build finishes, view the "Tasks" page for the build and enable the "Disk IO" and "CPU usage" columns. Note that some CPU usage percentages are > 100, as discussed in bug 8842. Changes since 961893fa05f4d51187f4d5daadacd7927d1a0847 (toaster-next) are in git://git.yoctoproject.org/poky-contrib, elliot/toaster/build_data-8842 http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=elliot/toaster/build_data-8842 Related bug: https://bugzilla.yoctoproject.org/show_bug.cgi?id=8842 NOTE: The first commit in this series is for oe-core. Elliot Smith (3): toaster.bbclass: improve how we gather buildstats for Toaster toaster: rework task buildstats storage and display toaster-tests: add test for cputime subpage bitbake/lib/bb/ui/buildinfohelper.py | 72 ++++++++++------------ .../orm/migrations/0005_task_field_separation.py | 48 +++++++++++++++ bitbake/lib/toaster/orm/models.py | 18 +++++- .../toastergui/templates/basebuildpage.html | 4 +- .../lib/toaster/toastergui/templates/tasks.html | 20 +++--- bitbake/lib/toaster/toastergui/tests.py | 45 +++++++++++++- bitbake/lib/toaster/toastergui/urls.py | 2 +- bitbake/lib/toaster/toastergui/views.py | 55 +++++++++++------ meta/classes/toaster.bbclass | 70 +++++++++++++-------- 9 files changed, 237 insertions(+), 97 deletions(-) create mode 100644 bitbake/lib/toaster/orm/migrations/0005_task_field_separation.py -- Elliot Smith Software Engineer Intel OTC --------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47 This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.