Hi All,
After the TF-A Tech-forum on 11th February 2021 (https://www.trustedfirmware.org/meetings/tf-a-technical-forum/) introducing the TF-A OpenCI I wanted to follow up with a posting on how we want to incorporate this new CI workflow into the TF-A project environment and who will have the ability to instigate OpenCI runs for the project.
Some guidelines we are looking to follow are:
* Be consistent across all Trustedfirmware.org hosted projects (TF-A, TF-M, Hafnium etc)
* Be as open as possible to allow project contributors access to the CI.
* Protect backend server resources from security attacks.
* Be open to tune the workflow process as needs demand.
As mentioned in the Tech-forum session the main day to day developer interface with the OpenCI is through Gerrit reviews with the CI+1/CI+2 gerrit labels for patches under review that start two levels of CI coverage. There is also a daily CI run on the integration branch that is started automatically. OpenCI runs for gerrit patch reviews are shown in the patch comments as links into https://ci.trustedfirmware.org/view/TF-A/ where all the Jenkin jobs including the daily build can be found.
In addition there is the occasional need to re-trigger CI jobs directly in the Jenkins UI shown above. This can occur if there is an intermittent failure in the CI infrastructure or due to some other cause however its hoped this is only needed to be used rarely and most contributors will not need to use this.
The initial plan is allow OpenCI invocation (through Gerrit and Jenkins) to all the TF-A maintainers and Code Owners listed in https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/docs/about… as a list of known project contributors. This list can be extended over time with other individuals nominated by anybody on the initial list.
The OpenCI will become the primary and only CI for setting the Gerrit verified label for patches under review. The existing ArmCI will become advisory and triggered separately and will not directly influence the Gerrit verified label for patches under review. The ArmCI will be maintained in parallel for those areas not yet migrated to the OpenCI and will be disabled once the next phases of the OpenCI development complete. Any CI results from the ArmCI will be communicated in patch review comments from Arm contributors as required.
Project documentation will be updated to incorporate OpenCI usage and a further Tech-forum will be held to go through the CI usage in more detail once the OpenCI is ready to take over as the primary TF-A CI. This is expected and hoping to take place in the next week or two as the final OpenCI deployment issues are resolved but exactly when will be communicated to this list once we have a firm date.
Thanks
Joanna
Hi Chris,
This seems like a good proposal to alleviate some of the pain around releases.
Some observations/questions.
1. Introducing additional tags will eat into precious real estate. This will be a problem for some changes and developers.
2. In the transition period, the proposal has the potential to "pollute" the history
3. The proposal will add more work for patches cherry-picked from other projects e.g. libfdt
4. How do we handle scenarios where platforms donot want to switch to this policy?
I can see how #1 might be of concern to many, but we will have to implement some policy to see how many commits really fall in this category.
-Varun
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Chris Kay via TF-A
Sent: Thursday, February 11, 2021 5:59 AM
To: tf-a(a)lists.trustedfirmware.org
Subject: [TF-A] Adoption of Conventional Commits
External email: Use caution opening links or attachments
Hi all,
Recently we had an internal discussion on the merits of introducing semantics to commit messages pushed to the main TF-A repository, the conclusion being that we would look to adopting the Conventional Commits<https://www.conventionalcommits.org/en/v1.0.0/> specification in the near future. There was one major reason for this, which was to help us in automating the changelog in future releases, but it might also help us to dramatically reduce the overall amount of work needed to make a formal release in the future.
This requires some buy-in (or buy-out, in this case) from maintainers because - even though it's to only a relatively minor extent - it does involve an adjustment to everybody's workflow. Notably, commit messages will be expected to adopt the structure defined by the specification, which will be enforced by the CI. Most commits that go upstream today adhere to "something that looks like Conventional Commits", so the change is not exactly sweeping, but any change has the potential be an inconvenience.
With that in mind, I propose the following:
* We collectively adopt the specification, enforced only for @arm.com contributors until such a time that the majority of maintainers are familiar with the new demands
* We suggest - in the prerequisites documentation - the installation of two helper tools:
* Commitizen<https://github.com/commitizen/cz-cli>
* Commitlint<https://github.com/conventional-changelog/commitlint>
Installation of these tools will be optional, but I believe they can help with the transition. In the patches currently in review, they are installed as Git hooks automatically upon execution of npm install, so it requires no manual installation or configuration (other than a relatively up-to-date Node.js installation).
You'll find the patches here<https://review.trustedfirmware.org/q/topic:%22ck%252Fconventional-commits%2…>, and specifically the changes to the prerequisites documentation here<https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/8224/1/docs/…>. Feel free to review these changes if you have comments specifically on their implementation.
Let me know if you have any questions or concerns. If everybody's on board, we can look to have this upstreamed shortly.
Chris
Hi Daniele,
TBBR specification, from where fip format was introduced, is not very clear about usage of serial number and it can be used in IMP DEFINED manner
"ToC Serial Number - 32 - The serial number of this ToC".
So, theoretically it's possible to use serial number for the purpose you described and it's a valid use of currently (un)used serial number.
Currently, at boot time serial number is checked against a non-zero value, which certainly will hold true if you put a valid timstamp instead of "0x12345678".
IMO you can go ahead and implement a mechanism to feed Build timestamp to be used as serial number.
Thanks
Manish P
________________________________
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of Manish Badarkhe via TF-A <tf-a(a)lists.trustedfirmware.org>
Sent: 11 February 2021 11:39
To: Daniele Alessandrelli <daniele.alessandrelli(a)linux.intel.com>; tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>
Subject: Re: [TF-A] Getting BUILD_STRING from FIP file
Hi Daniele
Please see my replies inline.
Thanks
Manish Badarkhe
From: Daniele Alessandrelli <daniele.alessandrelli(a)linux.intel.com>
Date: Thursday, 11 February 2021 at 11:22
To: Manish Badarkhe <Manish.Badarkhe(a)arm.com>, tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>
Subject: Re: [TF-A] Getting BUILD_STRING from FIP file
Hi Manish,
Thank you very much for the information.
16 bits are not enough to store an epoch, but I'll see if I can encode
some other unique build information into 16 bits.
Just a question, from your answer I assume that I shouldn't use the
'serial_number' field for what I want to do; so I wonder: what is that
field meant to be used for?
'serial_number' field is non-zero number provided by the fip creation tool.
It is hardcoded value i.e. TOC_HEADER_SERIAL_NUMBER=0x12345678 as of now.
Specifically used during the validation of the TOC header in the code.
Regards,
Daniele
On Thu, 2021-02-11 at 10:14 +0000, Manish Badarkhe wrote:
> Hi Daniele
>
> You can use the ‘flag’ field to mention the platform-specific data(in
> your case, a build number). Usage of the ‘flag’ field(64 bit) in the
> toc_header are as below:
> Bits 0-31 -> reserved
> Bits 32-47 -> platform defined data
> Bits 48-63 -> reserved
> You can make use of the flag[32:47] to put build information. I am
> not sure if you can accommodate epoch (converted timestamp) into this
> field but, you can encode any data to fit into this 16bit flag field
> to identify the FIP build.
>
> You can use a build command: fiptool update/create --plat-toc-flags
> <platform defined data> <your fip bin path> to put the platform
> defined data in the FIP image.
>
> Thanks
> Manish Badarkhe
> From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of
> Daniele Alessandrelli via TF-A <tf-a(a)lists.trustedfirmware.org>
> Date: Wednesday, 10 February 2021 at 17:04
> To: tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>
> Subject: [TF-A] Getting BUILD_STRING from FIP file
>
> Hi,
>
> Is there a way to get BUILD_STRING (or a similar string / number that
> uniquely identifies the TF-A build, e.g., BUILD_MESSAGE_TIMESTAMP)
> from
> the FIP file?
>
> Basically, I'm trying to find a way to know the build number of a FIP
> without flashing it.
>
> I've seen that the FIP TOC header has a 32-bit field named
> 'serial_number'. Can it be used to this end? I'm considering
> converting BUILD_MESSAGE_TIMESTAMP into an epoch and adding it as
> 'serial number', but I'm worried that might be an unintended usage of
> the 'serial_number' field.
>
> Regards,
> Daniele
>
>
>
Hi all,
Recently we had an internal discussion on the merits of introducing semantics to commit messages pushed to the main TF-A repository, the conclusion being that we would look to adopting the Conventional Commits<https://www.conventionalcommits.org/en/v1.0.0/> specification in the near future. There was one major reason for this, which was to help us in automating the changelog in future releases, but it might also help us to dramatically reduce the overall amount of work needed to make a formal release in the future.
This requires some buy-in (or buy-out, in this case) from maintainers because - even though it's to only a relatively minor extent - it does involve an adjustment to everybody's workflow. Notably, commit messages will be expected to adopt the structure defined by the specification, which will be enforced by the CI. Most commits that go upstream today adhere to "something that looks like Conventional Commits", so the change is not exactly sweeping, but any change has the potential be an inconvenience.
With that in mind, I propose the following:
* We collectively adopt the specification, enforced only for @arm.com contributors until such a time that the majority of maintainers are familiar with the new demands
* We suggest - in the prerequisites documentation - the installation of two helper tools:
* Commitizen<https://github.com/commitizen/cz-cli>
* Commitlint<https://github.com/conventional-changelog/commitlint>
Installation of these tools will be optional, but I believe they can help with the transition. In the patches currently in review, they are installed as Git hooks automatically upon execution of npm install, so it requires no manual installation or configuration (other than a relatively up-to-date Node.js installation).
You'll find the patches here<https://review.trustedfirmware.org/q/topic:%22ck%252Fconventional-commits%2…>, and specifically the changes to the prerequisites documentation here<https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/8224/1/docs/…>. Feel free to review these changes if you have comments specifically on their implementation.
Let me know if you have any questions or concerns. If everybody's on board, we can look to have this upstreamed shortly.
Chris
Hi Daniele
You can use the ‘flag’ field to mention the platform-specific data(in your case, a build number). Usage of the ‘flag’ field(64 bit) in the toc_header are as below:
1. Bits 0-31 -> reserved
2. Bits 32-47 -> platform defined data
3. Bits 48-63 -> reserved
You can make use of the flag[32:47] to put build information. I am not sure if you can accommodate epoch (converted timestamp) into this field but, you can encode any data to fit into this 16bit flag field to identify the FIP build.
You can use a build command: fiptool update/create --plat-toc-flags <platform defined data> <your fip bin path> to put the platform defined data in the FIP image.
Thanks
Manish Badarkhe
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of Daniele Alessandrelli via TF-A <tf-a(a)lists.trustedfirmware.org>
Date: Wednesday, 10 February 2021 at 17:04
To: tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>
Subject: [TF-A] Getting BUILD_STRING from FIP file
Hi,
Is there a way to get BUILD_STRING (or a similar string / number that
uniquely identifies the TF-A build, e.g., BUILD_MESSAGE_TIMESTAMP) from
the FIP file?
Basically, I'm trying to find a way to know the build number of a FIP
without flashing it.
I've seen that the FIP TOC header has a 32-bit field named
'serial_number'. Can it be used to this end? I'm considering
converting BUILD_MESSAGE_TIMESTAMP into an epoch and adding it as
'serial number', but I'm worried that might be an unintended usage of
the 'serial_number' field.
Regards,
Daniele
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Hi,
Is there a way to get BUILD_STRING (or a similar string / number that
uniquely identifies the TF-A build, e.g., BUILD_MESSAGE_TIMESTAMP) from
the FIP file?
Basically, I'm trying to find a way to know the build number of a FIP
without flashing it.
I've seen that the FIP TOC header has a 32-bit field named
'serial_number'. Can it be used to this end? I'm considering
converting BUILD_MESSAGE_TIMESTAMP into an epoch and adding it as
'serial number', but I'm worried that might be an unintended usage of
the 'serial_number' field.
Regards,
Daniele
Hi All,
The next TF-A Tech Forum is scheduled for Thu 11th February 2021 16:00 – 17:00 (GMT).
Agenda:
* TF-A: Open-CI Introduction & Status
* Presented by Joanna Farley with support from Linaro OpenCI Enablement Team
* Having a Public CI (Continuous Integration) extensible system has been a goal for a while and this presentation will give an introduction and a high level overview along with the current status. A brief walk through what CI jobs are available, when they are run and how results can be accessed will be shown/demoed. Deeper dives into the OpenCI results and how to analyse will be the subject of future Tech Forum sessions.
If TF-A contributors have anything they wish to present at any future TF-A tech forum please contact me to have that scheduled.
Previous sessions, both recording and presentation material can be found on the trustedfirmware.org TF-A Technical meeting webpage: https://www.trustedfirmware.org/meetings/tf-a-technical-forum/
A scheduling tracking page is also available to help track sessions suggested: https://developer.trustedfirmware.org/w/tf_a/tf-a-tech-forum-scheduling/ Final decisions on what will be presented will be shared a few days before the next meeting on the TF-A mailing list.
Join Zoom Meeting
https://zoom.us/j/9159704974<https://www.google.com/url?q=https%3A%2F%2Fzoom.us%2Fj%2F9159704974&sa=D&us…>
Meeting ID: 915 970 4974
One tap mobile
+16465588656,,9159704974# US (New York)
+16699009128,,9159704974# US (San Jose)
Dial by your location
+1 646 558 8656 US (New York)
+1 669 900 9128 US (San Jose)
877 853 5247 US Toll-free
888 788 0099 US Toll-free
Meeting ID: 915 970 4974
Find your local number: https://zoom.us/u/ad27hc6t7h<https://www.google.com/url?q=https%3A%2F%2Fzoom.us%2Fu%2Fad27hc6t7h&sa=D&us…>
Thanks
Joanna