All of lore.kernel.org
 help / color / mirror / Atom feed
From: Patrick Venture <venture@google.com>
To: Lakshminarayana R Kammath <lkammath@in.ibm.com>
Cc: OpenBMC Maillist <openbmc@lists.ozlabs.org>
Subject: Re: [OpenBMC]: Unit test coverage analysis on openbmc repositories
Date: Fri, 28 Feb 2020 10:20:56 -0800	[thread overview]
Message-ID: <CAO=notw9Gt2=meykgiff09finUnMtLNqnW1PLpwX4HESm4OXVA@mail.gmail.com> (raw)
In-Reply-To: <OFBAC0E531.E7E68FE5-ON0025851C.0053A4C8-0025851C.0055AA1E@notes.na.collabserv.com>

On Fri, Feb 28, 2020 at 7:36 AM Lakshminarayana R Kammath
<lkammath@in.ibm.com> wrote:
>
> Hi All,
>
> Recently we worked on pulling the UT coverage analysis from all the available OpenBMC repositories
>
> https://github.com/openbmc/openbmc-build-scripts/commit/229b76a95f87af60c976a0c0dfe84716c9ce5318
>
> This script does following
>
> Clone the repo
> Use the CI build environment to build code
> Publish the result in the following format
>
> Latest report:
> *************************UNIT TEST COVERAGE REPORT*************************
> https://github.com/openbmc/bmcweb.git NO
> https://github.com/openbmc/boost-dbus.git ERROR
> https://github.com/openbmc/btbridge.git NO
> https://github.com/openbmc/dbus-sensors.git NO
> https://github.com/openbmc/entity-manager.git NO
> https://github.com/openbmc/fb-ipmi-oem.git NO
> https://github.com/openbmc/google-ipmi-i2c.git ERROR
> https://github.com/openbmc/google-ipmi-sys.git YES, UNIT TEST
> https://github.com/openbmc/gpioplus.git YES, COVERAGE
> https://github.com/openbmc/hiomapd.git YES, UNIT TEST
> https://github.com/openbmc/ibm-dbus-interfaces.git NO
> https://github.com/openbmc/ibm-logging.git YES, UNIT TEST
> https://github.com/openbmc/ibm-pldm-oem.git ERROR
> https://github.com/openbmc/inarp.git ERROR
> https://github.com/openbmc/intel-dbus-interfaces.git NO
> https://github.com/openbmc/intel-ipmi-oem.git NO
> https://github.com/openbmc/ipmbbridge.git NO
> https://github.com/openbmc/ipmi-blob-tool.git YES, UNIT TEST
> https://github.com/openbmc/ipmi-fru-parser.git NO
> https://github.com/openbmc/ipmitool.git ERROR
> https://github.com/openbmc/jsnbd.git ERROR
> https://github.com/openbmc/kcsbridge.git NO
> https://github.com/openbmc/lenovo-ipmi-oem.git NO
> https://github.com/openbmc/libmctp.git YES, UNIT TEST
> https://github.com/openbmc/librrdplus.git NO
> https://github.com/openbmc/obmc-console.git YES, UNIT TEST
> https://github.com/openbmc/obmc-ikvm.git NO
> https://github.com/openbmc/openbmc.git NO
> https://github.com/openbmc/openbmc-tools.git NO
> https://github.com/openbmc/openbmc.github.io.git NO
> https://github.com/openbmc/openpower-dbus-interfaces.git NO
> https://github.com/openbmc/openpower-debug-collector.git NO
> https://github.com/openbmc/openpower-host-ipmi-flash.git NO
> https://github.com/openbmc/openpower-host-ipmi-oem.git NO
> https://github.com/openbmc/openpower-hw-diags.git YES, COVERAGE
> https://github.com/openbmc/openpower-inventory-upload.git NO
> https://github.com/openbmc/openpower-libhei.git YES, COVERAGE
> https://github.com/openbmc/openpower-logging.git NO
> https://github.com/openbmc/openpower-occ-control.git YES, UNIT TEST
> https://github.com/openbmc/openpower-pnor-code-mgmt.git YES, UNIT TEST
> https://github.com/openbmc/openpower-power-control.git NO
> https://github.com/openbmc/openpower-proc-control.git YES, UNIT TEST
> https://github.com/openbmc/openpower-sbe-interface.git NO
> https://github.com/openbmc/openpower-vpd-parser.git YES, UNIT TEST
> https://github.com/openbmc/pam-ipmi.git NO
> https://github.com/openbmc/phosphor-bmc-code-mgmt.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-buttons.git NO
> https://github.com/openbmc/phosphor-certificate-manager.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-dbus-interfaces.git NO
> https://github.com/openbmc/phosphor-dbus-monitor.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-debug-collector.git NO
> https://github.com/openbmc/phosphor-ecc.git NO
> https://github.com/openbmc/phosphor-event.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-fan-presence.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-gpio-monitor.git YES, COVERAGE
> https://github.com/openbmc/phosphor-host-ipmid.git ERROR
> https://github.com/openbmc/phosphor-host-postd.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-hostlogger.git NO
> https://github.com/openbmc/phosphor-hwmon.git NO

^---- This one should be YES, UNIT TEST.

Thanks for working on identifying missing coverage.  I wanted to point
this out so you can take another swing at this, -- Thanks!

> https://github.com/openbmc/phosphor-inventory-manager.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-ipmi-blobs.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-ipmi-blobs-binarystore.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-ipmi-ethstats.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-ipmi-flash.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-led-manager.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-led-sysfs.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-logging.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-mboxd.git ERROR
> https://github.com/openbmc/phosphor-misc.git NO
> https://github.com/openbmc/phosphor-mrw-tools.git NO
> https://github.com/openbmc/phosphor-net-ipmid.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-networkd.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-nvme.git NO
> https://github.com/openbmc/phosphor-objmgr.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-pcie-presence.git NO
> https://github.com/openbmc/phosphor-pid-control.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-post-code-manager.git NO
> https://github.com/openbmc/phosphor-power.git YES, COVERAGE
> https://github.com/openbmc/phosphor-power-control.git NO
> https://github.com/openbmc/phosphor-power-monitor.git NO
> https://github.com/openbmc/phosphor-psu-code-mgmt.git YES, COVERAGE
> https://github.com/openbmc/phosphor-rest-server.git NO
> https://github.com/openbmc/phosphor-sel-logger.git NO
> https://github.com/openbmc/phosphor-settingsd.git NO
> https://github.com/openbmc/phosphor-snmp.git ERROR
> https://github.com/openbmc/phosphor-state-manager.git YES, COVERAGE
> https://github.com/openbmc/phosphor-time-manager.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-u-boot-env-mgr.git NO
> https://github.com/openbmc/phosphor-user-manager.git YES, UNIT TEST
> https://github.com/openbmc/phosphor-watchdog.git YES, COVERAGE
> https://github.com/openbmc/phosphor-webui.git NO
> https://github.com/openbmc/pldm.git YES, COVERAGE
> https://github.com/openbmc/pyphosphor.git NO
> https://github.com/openbmc/qualcomm-ipmi-oem.git NO
> https://github.com/openbmc/qualcomm-misc.git NO
> https://github.com/openbmc/quanta-ipmi-oem.git NO
> https://github.com/openbmc/rest-dbus.git NO
> https://github.com/openbmc/rrd-ipmi-blobs.git NO
> https://github.com/openbmc/s2600wf-misc.git NO
> https://github.com/openbmc/sdbusplus.git YES, UNIT TEST
> https://github.com/openbmc/sdeventplus.git YES, COVERAGE
> https://github.com/openbmc/skeleton.git ERROR
> https://github.com/openbmc/slpd-lite.git NO
> https://github.com/openbmc/smbios-mdr.git ERROR
> https://github.com/openbmc/stdplus.git YES, COVERAGE
> https://github.com/openbmc/telemetry.git NO
> https://github.com/openbmc/u-boot NO
> https://github.com/openbmc/webui-vue.git NO
> https://github.com/openbmc/witherspoon-pfault-analysis.git YES, COVERAGE
> https://github.com/openbmc/x86-power-control.git NO
> *************************UNIT TEST COVERAGE REPORT*************************
> SUMMARY:
> TOTAL REPOSITORIES : 110
> TESTED REPOSITORIES : 43
> ERROR : 11
> COVERAGE REPORT : 12
> UNIT TEST REPORT : 31
> NO REPORT : 56
> ***************************************************************************
>
>
> Following actions, we can draw from above
>
> There are 12 repo's that have GCOV/LCOV plugged in - We need to maintain which has good coverage and try to improve those which has less coverage
> There are 31 repo's that have unit test cases added but no coverage analysis because they aren't plugged in yet to GCOV/LCOV  - Is there a plan for plugging in GCOV
> There are 11 repo's which fail to compile and errors out - Fix compilation issue. Some of these maybe not used currently?
> There are 56 repo's for which there is no UT tests available  - This is a major issue we need to tackle and improve upon
>
> Feedback's , views highly appreciated
>
> Thanks & Regards,
> Lakshminarayana Kamath
> --------------------------------------------------------------------------------------------------
> Senior Engineer,
> IBM India Software Labs
> Bangalore
> github:lkammath
> ---------------------------------------------------------------------------------------------------
>
>

  parent reply	other threads:[~2020-02-28 18:21 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-28 15:35 [OpenBMC]: Unit test coverage analysis on openbmc repositories Lakshminarayana R Kammath
2020-02-28 16:18 ` Brad Bishop
2020-02-28 16:30 ` Lakshminarayana R Kammath
2020-02-28 16:31   ` Brad Bishop
2020-02-28 16:36     ` Brad Bishop
2020-03-04  6:42       ` Lakshminarayana R Kammath
2020-02-28 16:39     ` Lakshminarayana R Kammath
2020-02-28 18:12       ` Nancy Yuen
2020-02-28 18:20 ` Patrick Venture [this message]
2020-03-05  1:25   ` James Feist
2020-03-03  4:48 ` Andrew Jeffery
2020-03-03  6:16   ` Lakshminarayana R Kammath
2020-03-05  7:57   ` Lakshminarayana R Kammath
2020-03-06  9:27     ` Lei YU
2020-03-04 18:43 ` Andrew Geissler
2020-02-28 16:08 lakshminarayan kamath

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to='CAO=notw9Gt2=meykgiff09finUnMtLNqnW1PLpwX4HESm4OXVA@mail.gmail.com' \
    --to=venture@google.com \
    --cc=lkammath@in.ibm.com \
    --cc=openbmc@lists.ozlabs.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.