From: Nagal, Amit <amit.nagal@amd.com>
Date: Monday, 16 December 2024 at 09:02
To: Gyorgy Szing <Gyorgy.Szing@arm.com>, Mark Dykes <Mark.Dykes@arm.com>, Thangaraj, Senthil Nathan <SenthilNathan.Thangaraj@amd.com>, Olivier Deprez <Olivier.Deprez@arm.com>, tf-a-tests@lists.trustedfirmware.org <tf-a-tests@lists.trustedfirmware.org>, Joanna Farley <Joanna.Farley@arm.com>, tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.org>
Cc: Jain, Ronak <ronak.jain@amd.com>, Simek, Michal <michal.simek@amd.com>
Subject: RE: Query regarding the test coverage thru tf-a-tests

Hi ,

I am referring to the code coverage as below :

https://ci-builds.trustedfirmware.org/static-files/htxSEy4sEOQXtuGetFtzHqAmpr9kguARVj1zPKr9ahIxNzM0MzM5MTcwMjg5Ojk6YW5vbnltb3VzOmpvYi90Zi1hLWNpLWNvdmVyYWdlLWdhdGV3YXkvMTIxL2FydGlmYWN0/merge/outdir/lcov/index.html

I see a lot of files from non plat code getting covered with code coverage .

can somebody help me with further information :

1. Where are cputests related unit test scripts hosted for non plat files like bl31_main.c ?
2. can we use the unit test & code coverage infra for other platforms say AMD-Xilinx?
3. How is the code coverage achieved for assembly .S files like bl31_entrypoint.S ?

Regards

Amit

Hi Everyone,

I see very limited number of unit tests cpp files hosted at https://git.trustedfirmware.org/plugins/gitiles/TF-A/tf-a-unit-tests/+/refs/heads/main/tests/

Do we have more unit tests implemented for TF-A and available publicly anywhere ?

Regards

Amit

Hi Amit,

“can this be utilized to perform unit testing for platforms other than fvp”

Unit-tests executables are running on the “build host” which often is the X86 based PC of the developer (or a CI worker). UT is favoring low cost and sacrificing some level of test correctness. It assumes the compiled C code is fully portable and works the same on all platforms. Moreover, it focuses on C function correctness, and checks if a specific function works as designed. Verifying higher level operation is out of scope (e.g. function implements any feature correctly or is behaving correctly when called by other functions in the system). In turn the complexity of embedded development is removed (managing the target, updating the firmware, using a JTAG interface for debugging, etc…), and the resource constraints of the embedded target is not present.

The code coverage executed on the fvp platform is a system test, it exercises the product on an emulated target. It verifies the correctness of the code as it executes like it will in a real-world scenario. This is a higher value test, but it is much more expensive in terms of time and complexity than UT.

“Is c-picker tool available inside arm still as of now ?”

No, it is public, although it is a bit hidden. It lives on the c-picker branch to the Trusted Services repository (see [1) Similarly, Firmware Test Builder lives on the fwtb branch (see 2).

This branch is independent of TS code and contains only the two tools. We have plans to move each to dedicated repositories, but I don’t know when I will have the time to do this.

(FWTB and c-picker was developed by my team (TS) and is used and maintained by both projects. Before the TS project was started, we were working on TF-A. I can help with the tools, but I am not familiar to TF-A specifics like, how the tools are integrated to TF-A or which parts of TF-A are covered by UT cases.)

1: https://review.trustedfirmware.org/plugins/gitiles/TS/trusted-services/+/refs/heads/topics/c-picker

2: https://review.trustedfirmware.org/plugins/gitiles/TS/trusted-services/+/refs/heads/topics/fwtb

/George