TF-A-Tests November 2024

tf-a-tests@lists.trustedfirmware.org

5 participants
4 discussions

Firmware-A v2.12 release code freeze notification

by Govindraj Raja

Hi All, The next release of the Firmware-A bundle of projects tagged v2.12 has an expected code freeze date of Nov, 8th 2024. Refer to the release cadence section from TF-A documentation (https://trustedfirmware-a.readthedocs.io/en/latest/about/release-informatio…). Closing out the release takes around 6-10 working days after the code freeze. v2.12 release preparation tasks start from now. We want to ensure that planned feature patches for the release are submitted in good time for the review process to conclude. As a kind recommendation and a matter of sharing CI resources, please launch CI jobs with care e.g.: -For simple platform, docs changes, or one liners, use Allow-CI+1 label (no need for a full Allow-CI+2 run). -For large patch stacks use Allow-CI+2 at top of the patch stack (and if required few individual Allow+CI+1 labels in the middle of the patch stack). -Carefully analyze results and fix the change if required, before launching new jobs on the same change. -If after issuing a Allow-CI+1 or Allow-CI+2 label a Build start notice is not added as a gerrit comment on the patch right away please be patient as under heavy load CI jobs can be queued and in extreme conditions it can be over an hour before the Build start notice is issued. Issuing another Allow-CI+1 or Allow-CI+2 label will just result in an additional job being queued. -- Thanks, Govindraj R

2 months, 2 weeks

Re: Query regarding the test coverage thru tf-a-tests

by Nagal, Amit

Hi Mark, From unit test prospective , I saw this document https://www.trustedfirmware.org/docs/TF-A-UnitLevelTesting.pdf. can this be utilized to perform unit testing for platforms other than fvp ? Is c-picker tool available inside arm still as of now ? Regards Amit From: Mark Dykes <Mark.Dykes(a)arm.com> Sent: Tuesday, October 29, 2024 9:20 PM To: Nagal, Amit <amit.nagal(a)amd.com>; Thangaraj, Senthil Nathan <SenthilNathan.Thangaraj(a)amd.com>; Olivier Deprez <Olivier.Deprez(a)arm.com>; tf-a-tests(a)lists.trustedfirmware.org; Joanna Farley <Joanna.Farley(a)arm.com> Cc: Jain, Ronak <ronak.jain(a)amd.com>; Simek, Michal <michal.simek(a)amd.com> Subject: Re: Query regarding the test coverage thru tf-a-tests Amit, It appears I was incorrect in getting coverage outside of the flows we have internally. I apologize for the confusion... Mark ________________________________ From: Nagal, Amit <amit.nagal(a)amd.com<mailto:amit.nagal@amd.com>> Sent: Friday, October 25, 2024 12:07 AM To: Thangaraj, Senthil Nathan <SenthilNathan.Thangaraj(a)amd.com<mailto:SenthilNathan.Thangaraj@amd.com>>; Mark Dykes <Mark.Dykes(a)arm.com<mailto:Mark.Dykes@arm.com>>; Olivier Deprez <Olivier.Deprez(a)arm.com<mailto:Olivier.Deprez@arm.com>>; tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org> <tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org>>; Joanna Farley <Joanna.Farley(a)arm.com<mailto:Joanna.Farley@arm.com>> Cc: Jain, Ronak <ronak.jain(a)amd.com<mailto:ronak.jain@amd.com>>; Simek, Michal <michal.simek(a)amd.com<mailto:michal.simek@amd.com>> Subject: RE: Query regarding the test coverage thru tf-a-tests Hi @Mark Dykes<mailto:Mark.Dykes@arm.com> , Can you please share insights about how to get coverage information using manual tests run for FVP platform . That will help us to atleast get the coverage information for non platform code ( we are interested for bl31) . Regards Amit From: Thangaraj, Senthil Nathan <SenthilNathan.Thangaraj(a)amd.com<mailto:SenthilNathan.Thangaraj@amd.com>> Sent: Tuesday, September 10, 2024 12:23 AM To: Mark Dykes <Mark.Dykes(a)arm.com<mailto:Mark.Dykes@arm.com>>; Olivier Deprez <Olivier.Deprez(a)arm.com<mailto:Olivier.Deprez@arm.com>>; tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org>; joanna.farley(a)arm.com<mailto:joanna.farley@arm.com> Cc: Jain, Ronak <ronak.jain(a)amd.com<mailto:ronak.jain@amd.com>>; Nagal, Amit <amit.nagal(a)amd.com<mailto:amit.nagal@amd.com>>; Simek, Michal <michal.simek(a)amd.com<mailto:michal.simek@amd.com>> Subject: RE: Query regarding the test coverage thru tf-a-tests Thank you Joanna and Mark, for your insights and response. I wasn’t aware that these tests are running on a Virtual Platform and that’s how the coverage is being measured. I’ll look into this further, decide how we want to integrate it into our project, and get back to you. I appreciate your help and support here. Best regards, Senthil From: Mark Dykes <Mark.Dykes(a)arm.com<mailto:Mark.Dykes@arm.com>> Sent: Wednesday, August 28, 2024 9:03 AM To: Olivier Deprez <Olivier.Deprez(a)arm.com<mailto:Olivier.Deprez@arm.com>>; tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org>; Thangaraj, Senthil Nathan <SenthilNathan.Thangaraj(a)amd.com<mailto:SenthilNathan.Thangaraj@amd.com>> Cc: Jain, Ronak <ronak.jain(a)amd.com<mailto:ronak.jain@amd.com>>; Nagal, Amit <amit.nagal(a)amd.com<mailto:amit.nagal@amd.com>>; Simek, Michal <michal.simek(a)amd.com<mailto:michal.simek@amd.com>> Subject: RE: Query regarding the test coverage thru tf-a-tests Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding. Senthil, I do have a way to get coverage for manual test runs using FVP if that helps. I can assist if needed…. Mark From: Olivier Deprez <Olivier.Deprez(a)arm.com<mailto:Olivier.Deprez@arm.com>> Sent: Wednesday, August 28, 2024 12:53 AM To: Mark Dykes <Mark.Dykes(a)arm.com<mailto:Mark.Dykes@arm.com>>; tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org>; Thangaraj, Senthil Nathan <SenthilNathan.Thangaraj(a)amd.com<mailto:SenthilNathan.Thangaraj@amd.com>> Cc: Jain, Ronak <ronak.jain(a)amd.com<mailto:ronak.jain@amd.com>>; Nagal, Amit <amit.nagal(a)amd.com<mailto:amit.nagal@amd.com>>; Simek, Michal <michal.simek(a)amd.com<mailto:michal.simek@amd.com>> Subject: Re: Query regarding the test coverage thru tf-a-tests Hi Senthil, * Is there a way to get code coverage for manual test runs? Sorry, I'm not aware of such capability. As you say code coverage implies use of CI scripting that isn't possible to reproduce locally out of the box (afaik). Regards, Olivier. ________________________________ From: Thangaraj, Senthil Nathan via TF-A-Tests <tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org>> Sent: 26 August 2024 23:58 To: Mark Dykes <Mark.Dykes(a)arm.com<mailto:Mark.Dykes@arm.com>>; Olivier Deprez <Olivier.Deprez(a)arm.com<mailto:Olivier.Deprez@arm.com>>; tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org> <tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org>> Cc: Jain, Ronak <ronak.jain(a)amd.com<mailto:ronak.jain@amd.com>>; Nagal, Amit <amit.nagal(a)amd.com<mailto:amit.nagal@amd.com>>; Simek, Michal <michal.simek(a)amd.com<mailto:michal.simek@amd.com>> Subject: [Tf-a-tests] Re: Query regarding the test coverage thru tf-a-tests Hi Mark and Oliver, A gentle reminder on the below request. Thanks, Senthil From: Thangaraj, Senthil Nathan Sent: Tuesday, August 20, 2024 3:44 PM To: Mark Dykes <Mark.Dykes(a)arm.com<mailto:Mark.Dykes@arm.com>>; Olivier Deprez <Olivier.Deprez(a)arm.com<mailto:Olivier.Deprez@arm.com>>; tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org> Cc: Jain, Ronak <ronak.jain(a)amd.com<mailto:ronak.jain@amd.com>>; Nagal, Amit <amit.nagal(a)amd.com<mailto:amit.nagal@amd.com>>; Simek, Michal <michal.simek(a)amd.com<mailto:michal.simek@amd.com>> Subject: RE: Query regarding the test coverage thru tf-a-tests Thanks a lot Mark and Oliver for your inputs and pointers. All of these seem to be more closely integrated with the CI/CD builds. Is there a way to get code coverage for manual test runs? Specifically, I'm looking to get coverage results after loading and executing all the tests from a developer build (e.g., using a binary like tftf.bin). Best regards, Senthil From: Mark Dykes <Mark.Dykes(a)arm.com<mailto:Mark.Dykes@arm.com<mailto:Mark.Dykes@arm.com%3cmailto:Mark.Dykes@arm.com>>> Sent: Tuesday, August 20, 2024 3:16 PM To: Olivier Deprez <Olivier.Deprez(a)arm.com<mailto:Olivier.Deprez@arm.com<mailto:Olivier.Deprez@arm.com%3cmailto:Olivier.Deprez@arm.com>>>; tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org%3cmailto:tf-a-tests@lists.trustedfirmware.org>>; Thangaraj, Senthil Nathan <SenthilNathan.Thangaraj(a)amd.com<mailto:SenthilNathan.Thangaraj@amd.com<mailto:SenthilNathan.Thangaraj@amd.com%3cmailto:SenthilNathan.Thangaraj@amd.com>>> Subject: RE: Query regarding the test coverage thru tf-a-tests Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding. All, This link is more current. The overall coverage numbers are not being reported due to an issue that is being worked on now. However you can see the individual coverage metrics if you click on a particular config where is says "success". The next page will have the link "Build Artifacts" at the top and so click that to the next where you will see the directory "trace_report". Click this and you will see a list of contents one of which should be index.html. If you click this you will see the individual coverage report for that config(test). tf-a-ci-coverage-gateway #72 [Jenkins] (trustedfirmware.org)<https://ci.trustedfirmware.org/job/tf-a-ci-coverage-gateway/72/> Let me know if you have questions... Mark From: Olivier Deprez <Olivier.Deprez(a)arm.com<mailto:Olivier.Deprez@arm.com<mailto:Olivier.Deprez@arm.com%3cmailto:Olivier.Deprez@arm.com>>> Sent: Tuesday, August 20, 2024 2:27 AM To: tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org%3cmailto:tf-a-tests@lists.trustedfirmware.org>>; Thangaraj, Senthil Nathan <SenthilNathan.Thangaraj(a)amd.com<mailto:SenthilNathan.Thangaraj@amd.com<mailto:SenthilNathan.Thangaraj@amd.com%3cmailto:SenthilNathan.Thangaraj@amd.com>>>; Mark Dykes <Mark.Dykes(a)arm.com<mailto:Mark.Dykes@arm.com<mailto:Mark.Dykes@arm.com%3cmailto:Mark.Dykes@arm.com>>> Subject: Re: Query regarding the test coverage thru tf-a-tests Hi, You can refer to this job showing code coverage results aggregation from multiple test configurations: https://ci.trustedfirmware.org/job/tf-a-ci-coverage-gateway/67/ https://ci-builds.trustedfirmware.org/static-files/IK1tucCNXFdhS4AAQth8N7zW… I don't think there is a test level granularity. @Mark Dykes<mailto:Mark.Dykes@arm.com> may know better. Regards, Olivier. ________________________________ From: Thangaraj, Senthil Nathan via TF-A-Tests <tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org%3cmailto:tf-a-tests@lists.trustedfirmware.org>>> Sent: 20 August 2024 08:40 To: tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org%3cmailto:tf-a-tests@lists.trustedfirmware.org>> <tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org%3cmailto:tf-a-tests@lists.trustedfirmware.org>>> Subject: [Tf-a-tests] Query regarding the test coverage thru tf-a-tests Dear TF-A team, I have a query regarding the TF-A test. Specifically, I would like to know if there is a method to find out the code coverage for a specific test or for all the tests in current run? Your direction in this regard will be really appreciated. Thank you, Senthil -- TF-A-Tests mailing list -- tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org%3cmailto:tf-a-tests@lists.trustedfirmware.org>> To unsubscribe send an email to tf-a-tests-leave(a)lists.trustedfirmware.org<mailto:tf-a-tests-leave@lists.trustedfirmware.org<mailto:tf-a-tests-leave@lists.trustedfirmware.org%3cmailto:tf-a-tests-leave@lists.trustedfirmware.org>> -- TF-A-Tests mailing list -- tf-a-tests(a)lists.trustedfirmware.org<mailto:tf-a-tests@lists.trustedfirmware.org> To unsubscribe send an email to tf-a-tests-leave(a)lists.trustedfirmware.org<mailto:tf-a-tests-leave@lists.trustedfirmware.org>

1 year, 1 month

Addition of mbedTLS as a submodule dependency to TF-A-Tests

by Soby Mathew

Hi Everyone, In order to facilitate development for Device Assignment tests for RME-DA, we have added MbedTLS repo as a submodule dependency to tf-a-tests. The merge commit can be found here : https://review.trustedfirmware.org/plugins/gitiles/TF-A/tf-a-tests/+/3e72cd… The patch is done in such a way that existing build of TF-A-Tests or Test run is not affected due to the additional dependency. Only tests which depend on MbedTLS will be affected in that they will either be skipped or fail at runtime due to the missing dependency. Also, the change allows to use the config `MBEDTLS_DIR` to point to a MbedTLS directory outside the tf-a-tests source tree. This aligns with the TF-A mechanism for MbedTLS dependancy in case the submodule mechanism is not preferred. We expect existing CI and testing infrastructure to be unaffected by this change. Please let us know if you have any comments. Best Regards Soby Mathew

1 year, 1 month

Trusted Firmware v2.12 release announcement

by Olivier Deprez

Hi All, We are pleased to announce the formal release of Trusted Firmware-A version 2.12 bundle of project deliverables. This includes Trusted Firmware-A, Trusted Firmware-A Tests, Hafnium, RMM, Trusted Services, and TF-A OpenCI scripts/jobs components. These went live on Nov, 21st 2024. Please find references to tags and change logs at the end of this email. Many thanks to the trustedfirmware.org community for the active engagement in delivering this release! Notable features of the release version 2.12 are as follows: TF-A/EL3 * New CPUs support: Cortex-A320 (Arcadia), Cortex-A720AE. * Arch extensions: : FEAT_THE, FEAT_LS64_ACCDATA, FEAT_Debugv8p9, FEAT_SCTLR2, FEAT_FGT2 ; Arm 9.4 : FEAT_D128 support. * Context management: Asymmetric CPU features, EL3 execution context, SVE S/R support. * RME: GPT contiguous descriptor (or GPT large mappings support). * Arm CCA: added attestation generic library, el3 attestation token signing support. Boot flow * Tools: * Introduced the Transfer List Compiler (TLC) Host Tool supporting the FW hand-off specification. * Introduced CoT Device Tree to C File (DT2C) tool. * mbedTLS Improvements: * Resolved random authentication failures with ECDSA. * Enhanced mbedTLS configuration selection. * Maximize usage of mbedTLS library APIs in TF-A. * Upgraded to mbedTLS v3.6.1. * Feature addition/support: * Support for the ECDSA P-384 curve with the PSA Crypto implementation. * Documentation: * Added the DPE Design and Threat Model document to TF-A readthedocs. Errata (CPU/GIC) * Cortex-A720, Cortex-A520, Cortex-X4 errata. TF-A platforms support * New platforms added: TC4, RD-1 AE, RK3566/RK3568. * Platform fixes: FPGA, FVP, Neoverse-RD, TC, Corstone-1000, Allwinner, AST2700, Poplar, Agilex, A3K, MT8188, iMX8M, S32G274A, qemu, Rpi3, Rockchip, STM32MP1, STM32MP2, Versal, ZynqMP. Hafnium/SPM (S-EL2) * FF-A features additions * FF-A v1.2 FFA_MSG_WAIT RX buffer ownership flag. * FF-A v1.2 FFA_VERSION endpoint restriction. * FF-A v1.1 VM availability messages. * Architectural support * vCPU IPI signaling. * Arch timer virtualization. * RME: GPF support and memcpy hardening. * Secure Interrupt Handling for UP S-EL1 partitions. TF-RMM (R-EL2) * REL0 RMM 1.0 support. * Support for alternative attestation token signing via EL3. * Various improvements and bugfixes as listed in the change log. TF-A Tests * Arm arch. extensions: AMU, FEAT_FGT2, LS64_ACCDATA, FEAT_Debugv8p9, LS64 64-byte load/store. * Asymmetric feature testing (FEAT_SPE, FEAT_TRBE, FEAT_TCR2). * EL1 and EL2 context switch tests. * FF-A: v1.2 support, S-EL2 arch timer virtualization, S-EL2+RME hardening, EL3 SPMC TFTF test suite, EL3 SVE test coverage. * RME: RMM v1.0 testing support, PCIe DOE support, RMI support for FEAT_LPA2, SIMD, PAuth. * Platforms: AMD Versal Gen 2 added, Neoverse-RD refactoring, FVP PCIe support. Release tags across repositories: https://git.trustedfirmware.org/plugins/gitiles/TF-A/trusted-firmware-a/+/r… https://git.trustedfirmware.org/plugins/gitiles/TF-A/tf-a-tests/+/refs/tags… https://git.trustedfirmware.org/plugins/gitiles/ci/tf-a-ci-scripts/+/refs/t… https://git.trustedfirmware.org/plugins/gitiles/ci/tf-a-job-configs/+/refs/… https://git.trustedfirmware.org/plugins/gitiles/hafnium/hafnium/+/refs/tags… https://git.trustedfirmware.org/plugins/gitiles/ci/hafnium-ci-scripts/+/ref… https://git.trustedfirmware.org/plugins/gitiles/ci/hafnium-job-configs/+/re… https://git.trustedfirmware.org/plugins/gitiles/TF-RMM/tf-rmm/+/refs/tags/t… https://git.trustedfirmware.org/plugins/gitiles/TS/trusted-services/+/refs/… Change logs: https://trustedfirmware-a.readthedocs.io/en/v2.12.0/change-log.html#id1 https://trustedfirmware-a-tests.readthedocs.io/en/latest/change-log.html#ve… https://hafnium.readthedocs.io/en/v2.12.0/change-log.html#v2-12 https://tf-rmm.readthedocs.io/en/latest/about/change-log.html#v0-6-0 Regards, Olivier.

1 year, 2 months

2026

2025

2024

2023

2022

2021

2020

2019

2018

TF-A-Tests November 2024