OBMC Internship Project: Per-Build Code Coverage

OBMC Internship Project: Per-Build Code Coverage

[Jose Iciano]

[-- Attachment #1: Type: text/plain, Size: 3396 bytes --]

Hello,

My name is Jose Iciano and I currently working on an implementation of per-build code coverage for OBMC as part as an internship project. The draft of the current system's description will be available at the end of the email. Currently, I have been able to develop the required functions and am getting them checked by code reviews on Gerrit.

Draft of the Current System: https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/44409/1/code-coverage.md

Machine Per-Build Code Coverage

The following report details the code coverage system implemented for OpenBMC builds.

OpenBMC currently relies on a manual system for detailing code coverage of any repositories used for image building. With each machine, there is a repo combination unique to it. This means that currently, if one wants to view the status and changes of those repositories, they would have to manually clone it and run unit tests on the correct version. This system automates the process and allows the aggregation of code coverage for the selected build.

This process is done through a modified version of OBMC’s `get_unit_test_report.py` script, adding per-build capabilities and improving lcov data generation.

Running Files

Start by creating a repository for CI and CI scripts to be stored.

```
mkdir ci_test_area
cd ci_test_area
git clone [https://github.com/openbmc/openbmc-build-scripts.git](https://github.com/openbmc/openbmc-build-scripts.git)
```

To run with specified , image config files have to be modified to inherit the new file.
```
${BUILDDIR}/conf/local.conf
```

Add
```
INHERIT += “buildhistory”
BUILDHISTORY_COMMIT = “1”
```

Run the following command to run without a specified url_file or buildhistory path.

```
python3 openbmc-build-scripts/scripts/get_unit_test_report <target_dir>
```

Running with a buildhistory path

```
python3 openbmc-build-scripts/scripts/get_unit_test_report <target_dir> -build_history <build_history_path>
```

Source File and Revision Generation

The file `code_coverage.sh` generates a repositories.txt file, which stores the src_url and src_rev of repositories used during image creation. The format is stored in the following format:

```
<src_url> <src_rev>
```

The resulting file is intended to be used for any automated test suite runner, such as OBMC’s `get_unit_test_report.py`’s url_file parameter.

Automated Test Cases

Test case is automated through the use of `get_unit_test_report.py`, which takes care of cloning the specific repositories and running unit tests on it. This file was pre-existing in the codebase, but was slightly modified to work with the new files, as well as implementing optional source revision.

Optional parameters were modified to allow for more variables. The extra parameters added were

```
-build_history - Path to buildhistory that was created during image generation.
```

LCOV Aggregation

LCOV data is freely available after running the `get_unit_test_report.py` script. `code_coverage.py` works by taking all the available data and aggregating it into an easy to access file. This simplifies the process of lcov analysis, as the necessary data/files can be found from one simplified file.

Code_coverage.py works by taking in the path of directories leading to `<repo>/build/<coveragereport_path>/index.html.` It sifts through each file and reads the resulting lcov data from the index.html file.

[-- Attachment #2: Type: text/html, Size: 4643 bytes --]

Re: OBMC Internship Project: Per-Build Code Coverage

[Jose Iciano]

[-- Attachment #1: Type: text/plain, Size: 4010 bytes --]

Hello,

Following up on my previous email, I forgot to mention that I am currently an intern doing this as part of my summer project, with an end week of August 30th. With this in mind, I do have a limited amount of time to work with, so I would appreciate any help maintainers can provide to get code reviewed quickly. Thank you.

Jose Iciano

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Wednesday, June 23rd, 2021 at 7:11 PM, Jose Iciano <joseiciano@protonmail.com> wrote:

> Hello,
>
> My name is Jose Iciano and I currently working on an implementation of per-build code coverage for OBMC as part as an internship project. The draft of the current system's description will be available at the end of the email. Currently, I have been able to develop the required functions and am getting them checked by code reviews on Gerrit.
>
> Draft of the Current System: https://gerrit.openbmc-project.xyz/c/openbmc/docs/+/44409/1/code-coverage.md
>
> Machine Per-Build Code Coverage
>
> The following report details the code coverage system implemented for OpenBMC builds.
>
> OpenBMC currently relies on a manual system for detailing code coverage of any repositories used for image building. With each machine, there is a repo combination unique to it. This means that currently, if one wants to view the status and changes of those repositories, they would have to manually clone it and run unit tests on the correct version. This system automates the process and allows the aggregation of code coverage for the selected build.
>
> This process is done through a modified version of OBMC’s `get_unit_test_report.py` script, adding per-build capabilities and improving lcov data generation.
>
> Running Files
>
> Start by creating a repository for CI and CI scripts to be stored.
>
> ```
> mkdir ci_test_area
> cd ci_test_area
> git clone [https://github.com/openbmc/openbmc-build-scripts.git](https://github.com/openbmc/openbmc-build-scripts.git)
> ```
>
> To run with specified , image config files have to be modified to inherit the new file.
> ```
> ${BUILDDIR}/conf/local.conf
> ```
>
> Add
> ```
> INHERIT += “buildhistory”
> BUILDHISTORY_COMMIT = “1”
> ```
>
> Run the following command to run without a specified url_file or buildhistory path.
>
> ```
> python3 openbmc-build-scripts/scripts/get_unit_test_report <target_dir>
> ```
>
> Running with a buildhistory path
>
> ```
> python3 openbmc-build-scripts/scripts/get_unit_test_report <target_dir> -build_history <build_history_path>
> ```
>
> Source File and Revision Generation
>
> The file `code_coverage.sh` generates a repositories.txt file, which stores the src_url and src_rev of repositories used during image creation. The format is stored in the following format:
>
> ```
> <src_url> <src_rev>
> ```
>
> The resulting file is intended to be used for any automated test suite runner, such as OBMC’s `get_unit_test_report.py`’s url_file parameter.
>
> Automated Test Cases
>
> Test case is automated through the use of `get_unit_test_report.py`, which takes care of cloning the specific repositories and running unit tests on it. This file was pre-existing in the codebase, but was slightly modified to work with the new files, as well as implementing optional source revision.
>
> Optional parameters were modified to allow for more variables. The extra parameters added were
>
> ```
> -build_history - Path to buildhistory that was created during image generation.
> ```
>
> LCOV Aggregation
>
> LCOV data is freely available after running the `get_unit_test_report.py` script. `code_coverage.py` works by taking all the available data and aggregating it into an easy to access file. This simplifies the process of lcov analysis, as the necessary data/files can be found from one simplified file.
>
> Code_coverage.py works by taking in the path of directories leading to `<repo>/build/<coveragereport_path>/index.html.` It sifts through each file and reads the resulting lcov data from the index.html file.

[-- Attachment #2: Type: text/html, Size: 5578 bytes --]