Hi All,
Back in September Scott posted a query to the group related to a patch he has created relating to printf format specifiers and some of the maintainers from Arm have reservations about and we asked him to get opinions from the broader project community as his patch changes a number of different platforms as well as core code.
I’m trying to help him reinvigorate the discussion so reposting his request with patch link below that had stalled.
Joanna
________________________________
Scott Branden scott.branden at broadcom.com <mailto:tf-a%40lists.trustedfirmware.org?Subject=Re%3A%20%5BTF-A%5D%20ATF%20currently%20does%20not%20use%20proper%20printf%20format%20specifiers%0A%20for%20fixed%20width%20types&In-Reply-To=%3Cbd3b49f4-e8f9-9016-d11b-d08b81e6b43d%40broadcom.com%3E>
Mon Sep 14 18:34:45 UTC 2020
Hello,
ATF currently uses non-portable printf format specifiers for fixed width types defined in stdint.h
In addition, ATF redefines types defined in gcc for stdint.h with its own custom types causing additional issues.
This causes compilation issues when porting code to/from ATF.
AND, generates coverity parse errors as int64_t and uint64_t are incorrectly defined in ATF vs. gcc for aarch64.
The printf format specifiers in inttypes.h are to be used for the proper format specifiers.
And, uint64_t/int64_t should be defined the same as in gcc.
I tried fixing up all the instances of int64 printf format specifiers by introducing inttypes.h and redefined the stdint types correctly here:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/5437
We have checked the change into our local tree so that everything compiles and runs in our system. Please accept change upstream.
Regards,
Scott
Hi Varun,
I really like your proposal, but it would need significant investments on tooling. There is no tool which could help developers crafting the commit message in the right format, there is no tool, which can validate the format (and be used i.e. as a git-hook), and there is no tool, which can generate the change history document from git history.
Can you please extend the proposal and turn it to be an end-to-end solution? Can you contribute tooling for commit message editing and validation, and for change log document generation? Possibly the easiest would be to modify the javascript machinery available for Conventional Commits. Can you contribute the needed changes?
/George
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Varun Wadekar via TF-A
Sent: 26 February 2021 00:39
To: Chris Kay <Chris.Kay(a)arm.com>
Cc: tf-a(a)lists.trustedfirmware.org
Subject: Re: [TF-A] Adoption of Conventional Commits
Hi,
I really like the idea of using tags in the commit message, but the rigidity of the spec puts me off. Frankly, I believe we just need a way to identify commits and their intent. So, I would like to propose an approach that builds on the “Conventional Commits” spec.
The approach would be
1. Add an identifier (e.g. Tags: fix) to the commit message footer.
2. At the start of the release window run “git log”* to print a list of features, bug fixes, performance improvements, deprecations etc.
3. Either update the main changelog manually or use a script to append individual sections.
*git log v2.4...HEAD --no-merges --pretty='- %s (%C(auto)%h)' --grep "Tags: fix">
‘git log’ can be easily modified to look for other metadata as long as we agree to add it to the commit message.
Advantages
1. Light(er)
2. No impact to the subject header
3. Flexibility to define project specific tags
4. Training needs at par with “Conventional Commits” proposal
Thoughts?
-Varun
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org<mailto:tf-a-bounces@lists.trustedfirmware.org>> On Behalf Of Chris Kay via TF-A
Sent: Thursday, February 25, 2021 9:31 AM
To: tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>
Subject: Re: [TF-A] Adoption of Conventional Commits
External email: Use caution opening links or attachments
Thanks to all those who attended the Tech Forum today!
It’s become apparent that the initial 2 week deadline for alternative proposals or implementations is too short, so – as agreed – we’ll push the deadline for the investigation period to the end of March. This period is dedicated to evaluating the changelog automation proposal made, or to identifying alternative solutions. If you have an alternative proposal, any proof-of-concept tooling would be highly appreciated so we can get a clear idea of what sort of work and maintenance is going to be involved.
If you do find a solution you wish to propose, please give it just a short name (e.g. “Update it manually”) and make it obvious you want to propose it formally – I’ll collect up the proposals made on the mailing list thread at the end of March and set up a Wiki poll so we can get a clear picture of where the community wants to take this.
Chris
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org<mailto:tf-a-bounces@lists.trustedfirmware.org>> on behalf of Chris Kay via TF-A <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Reply to: Chris Kay <Chris.Kay(a)arm.com<mailto:Chris.Kay@arm.com>>
Date: Thursday, 11 February 2021 at 13:59
To: "tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>" <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Subject: [TF-A] Adoption of Conventional Commits
Hi all,
Recently we had an internal discussion on the merits of introducing semantics to commit messages pushed to the main TF-A repository, the conclusion being that we would look to adopting the Conventional Commits<https://www.conventionalcommits.org/en/v1.0.0/> specification in the near future. There was one major reason for this, which was to help us in automating the changelog in future releases, but it might also help us to dramatically reduce the overall amount of work needed to make a formal release in the future.
This requires some buy-in (or buy-out, in this case) from maintainers because - even though it’s to only a relatively minor extent - it does involve an adjustment to everybody’s workflow. Notably, commit messages will be expected to adopt the structure defined by the specification, which will be enforced by the CI. Most commits that go upstream today adhere to “something that looks like Conventional Commits”, so the change is not exactly sweeping, but any change has the potential be an inconvenience.
With that in mind, I propose the following:
* We collectively adopt the specification, enforced only for @arm.com contributors until such a time that the majority of maintainers are familiar with the new demands
* We suggest - in the prerequisites documentation - the installation of two helper tools:
* Commitizen<https://github.com/commitizen/cz-cli>
* Commitlint<https://github.com/conventional-changelog/commitlint>
Installation of these tools will be optional, but I believe they can help with the transition. In the patches currently in review, they are installed as Git hooks automatically upon execution of npm install, so it requires no manual installation or configuration (other than a relatively up-to-date Node.js installation).
You’ll find the patches here<https://review.trustedfirmware.org/q/topic:%22ck%252Fconventional-commits%2…>, and specifically the changes to the prerequisites documentation here<https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/8224/1/docs/…>. Feel free to review these changes if you have comments specifically on their implementation.
Let me know if you have any questions or concerns. If everybody’s on board, we can look to have this upstreamed shortly.
Chris
Hi,
I really like the idea of using tags in the commit message, but the rigidity of the spec puts me off. Frankly, I believe we just need a way to identify commits and their intent. So, I would like to propose an approach that builds on the “Conventional Commits” spec.
The approach would be
1. Add an identifier (e.g. Tags: fix) to the commit message footer.
2. At the start of the release window run “git log”* to print a list of features, bug fixes, performance improvements, deprecations etc.
3. Either update the main changelog manually or use a script to append individual sections.
*git log v2.4...HEAD --no-merges --pretty='- %s (%C(auto)%h)' --grep "Tags: fix">
‘git log’ can be easily modified to look for other metadata as long as we agree to add it to the commit message.
Advantages
1. Light(er)
2. No impact to the subject header
3. Flexibility to define project specific tags
4. Training needs at par with “Conventional Commits” proposal
Thoughts?
-Varun
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Chris Kay via TF-A
Sent: Thursday, February 25, 2021 9:31 AM
To: tf-a(a)lists.trustedfirmware.org
Subject: Re: [TF-A] Adoption of Conventional Commits
External email: Use caution opening links or attachments
Thanks to all those who attended the Tech Forum today!
It’s become apparent that the initial 2 week deadline for alternative proposals or implementations is too short, so – as agreed – we’ll push the deadline for the investigation period to the end of March. This period is dedicated to evaluating the changelog automation proposal made, or to identifying alternative solutions. If you have an alternative proposal, any proof-of-concept tooling would be highly appreciated so we can get a clear idea of what sort of work and maintenance is going to be involved.
If you do find a solution you wish to propose, please give it just a short name (e.g. “Update it manually”) and make it obvious you want to propose it formally – I’ll collect up the proposals made on the mailing list thread at the end of March and set up a Wiki poll so we can get a clear picture of where the community wants to take this.
Chris
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org<mailto:tf-a-bounces@lists.trustedfirmware.org>> on behalf of Chris Kay via TF-A <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Reply to: Chris Kay <Chris.Kay(a)arm.com<mailto:Chris.Kay@arm.com>>
Date: Thursday, 11 February 2021 at 13:59
To: "tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>" <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Subject: [TF-A] Adoption of Conventional Commits
Hi all,
Recently we had an internal discussion on the merits of introducing semantics to commit messages pushed to the main TF-A repository, the conclusion being that we would look to adopting the Conventional Commits<https://www.conventionalcommits.org/en/v1.0.0/> specification in the near future. There was one major reason for this, which was to help us in automating the changelog in future releases, but it might also help us to dramatically reduce the overall amount of work needed to make a formal release in the future.
This requires some buy-in (or buy-out, in this case) from maintainers because - even though it’s to only a relatively minor extent - it does involve an adjustment to everybody’s workflow. Notably, commit messages will be expected to adopt the structure defined by the specification, which will be enforced by the CI. Most commits that go upstream today adhere to “something that looks like Conventional Commits”, so the change is not exactly sweeping, but any change has the potential be an inconvenience.
With that in mind, I propose the following:
* We collectively adopt the specification, enforced only for @arm.com contributors until such a time that the majority of maintainers are familiar with the new demands
* We suggest - in the prerequisites documentation - the installation of two helper tools:
* Commitizen<https://github.com/commitizen/cz-cli>
* Commitlint<https://github.com/conventional-changelog/commitlint>
Installation of these tools will be optional, but I believe they can help with the transition. In the patches currently in review, they are installed as Git hooks automatically upon execution of npm install, so it requires no manual installation or configuration (other than a relatively up-to-date Node.js installation).
You’ll find the patches here<https://review.trustedfirmware.org/q/topic:%22ck%252Fconventional-commits%2…>, and specifically the changes to the prerequisites documentation here<https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/8224/1/docs/…>. Feel free to review these changes if you have comments specifically on their implementation.
Let me know if you have any questions or concerns. If everybody’s on board, we can look to have this upstreamed shortly.
Chris
Hi All,
After the TF-A Tech-forum on 11th February 2021 (https://www.trustedfirmware.org/meetings/tf-a-technical-forum/) introducing the TF-A OpenCI I wanted to follow up with a posting on how we want to incorporate this new CI workflow into the TF-A project environment and who will have the ability to instigate OpenCI runs for the project.
Some guidelines we are looking to follow are:
* Be consistent across all Trustedfirmware.org hosted projects (TF-A, TF-M, Hafnium etc)
* Be as open as possible to allow project contributors access to the CI.
* Protect backend server resources from security attacks.
* Be open to tune the workflow process as needs demand.
As mentioned in the Tech-forum session the main day to day developer interface with the OpenCI is through Gerrit reviews with the CI+1/CI+2 gerrit labels for patches under review that start two levels of CI coverage. There is also a daily CI run on the integration branch that is started automatically. OpenCI runs for gerrit patch reviews are shown in the patch comments as links into https://ci.trustedfirmware.org/view/TF-A/ where all the Jenkin jobs including the daily build can be found.
In addition there is the occasional need to re-trigger CI jobs directly in the Jenkins UI shown above. This can occur if there is an intermittent failure in the CI infrastructure or due to some other cause however its hoped this is only needed to be used rarely and most contributors will not need to use this.
The initial plan is allow OpenCI invocation (through Gerrit and Jenkins) to all the TF-A maintainers and Code Owners listed in https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/docs/about… as a list of known project contributors. This list can be extended over time with other individuals nominated by anybody on the initial list.
The OpenCI will become the primary and only CI for setting the Gerrit verified label for patches under review. The existing ArmCI will become advisory and triggered separately and will not directly influence the Gerrit verified label for patches under review. The ArmCI will be maintained in parallel for those areas not yet migrated to the OpenCI and will be disabled once the next phases of the OpenCI development complete. Any CI results from the ArmCI will be communicated in patch review comments from Arm contributors as required.
Project documentation will be updated to incorporate OpenCI usage and a further Tech-forum will be held to go through the CI usage in more detail once the OpenCI is ready to take over as the primary TF-A CI. This is expected and hoping to take place in the next week or two as the final OpenCI deployment issues are resolved but exactly when will be communicated to this list once we have a firm date.
Thanks
Joanna
Hi Chris,
This seems like a good proposal to alleviate some of the pain around releases.
Some observations/questions.
1. Introducing additional tags will eat into precious real estate. This will be a problem for some changes and developers.
2. In the transition period, the proposal has the potential to "pollute" the history
3. The proposal will add more work for patches cherry-picked from other projects e.g. libfdt
4. How do we handle scenarios where platforms donot want to switch to this policy?
I can see how #1 might be of concern to many, but we will have to implement some policy to see how many commits really fall in this category.
-Varun
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Chris Kay via TF-A
Sent: Thursday, February 11, 2021 5:59 AM
To: tf-a(a)lists.trustedfirmware.org
Subject: [TF-A] Adoption of Conventional Commits
External email: Use caution opening links or attachments
Hi all,
Recently we had an internal discussion on the merits of introducing semantics to commit messages pushed to the main TF-A repository, the conclusion being that we would look to adopting the Conventional Commits<https://www.conventionalcommits.org/en/v1.0.0/> specification in the near future. There was one major reason for this, which was to help us in automating the changelog in future releases, but it might also help us to dramatically reduce the overall amount of work needed to make a formal release in the future.
This requires some buy-in (or buy-out, in this case) from maintainers because - even though it's to only a relatively minor extent - it does involve an adjustment to everybody's workflow. Notably, commit messages will be expected to adopt the structure defined by the specification, which will be enforced by the CI. Most commits that go upstream today adhere to "something that looks like Conventional Commits", so the change is not exactly sweeping, but any change has the potential be an inconvenience.
With that in mind, I propose the following:
* We collectively adopt the specification, enforced only for @arm.com contributors until such a time that the majority of maintainers are familiar with the new demands
* We suggest - in the prerequisites documentation - the installation of two helper tools:
* Commitizen<https://github.com/commitizen/cz-cli>
* Commitlint<https://github.com/conventional-changelog/commitlint>
Installation of these tools will be optional, but I believe they can help with the transition. In the patches currently in review, they are installed as Git hooks automatically upon execution of npm install, so it requires no manual installation or configuration (other than a relatively up-to-date Node.js installation).
You'll find the patches here<https://review.trustedfirmware.org/q/topic:%22ck%252Fconventional-commits%2…>, and specifically the changes to the prerequisites documentation here<https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/8224/1/docs/…>. Feel free to review these changes if you have comments specifically on their implementation.
Let me know if you have any questions or concerns. If everybody's on board, we can look to have this upstreamed shortly.
Chris
Hi Daniele,
TBBR specification, from where fip format was introduced, is not very clear about usage of serial number and it can be used in IMP DEFINED manner
"ToC Serial Number - 32 - The serial number of this ToC".
So, theoretically it's possible to use serial number for the purpose you described and it's a valid use of currently (un)used serial number.
Currently, at boot time serial number is checked against a non-zero value, which certainly will hold true if you put a valid timstamp instead of "0x12345678".
IMO you can go ahead and implement a mechanism to feed Build timestamp to be used as serial number.
Thanks
Manish P
________________________________
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of Manish Badarkhe via TF-A <tf-a(a)lists.trustedfirmware.org>
Sent: 11 February 2021 11:39
To: Daniele Alessandrelli <daniele.alessandrelli(a)linux.intel.com>; tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>
Subject: Re: [TF-A] Getting BUILD_STRING from FIP file
Hi Daniele
Please see my replies inline.
Thanks
Manish Badarkhe
From: Daniele Alessandrelli <daniele.alessandrelli(a)linux.intel.com>
Date: Thursday, 11 February 2021 at 11:22
To: Manish Badarkhe <Manish.Badarkhe(a)arm.com>, tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>
Subject: Re: [TF-A] Getting BUILD_STRING from FIP file
Hi Manish,
Thank you very much for the information.
16 bits are not enough to store an epoch, but I'll see if I can encode
some other unique build information into 16 bits.
Just a question, from your answer I assume that I shouldn't use the
'serial_number' field for what I want to do; so I wonder: what is that
field meant to be used for?
'serial_number' field is non-zero number provided by the fip creation tool.
It is hardcoded value i.e. TOC_HEADER_SERIAL_NUMBER=0x12345678 as of now.
Specifically used during the validation of the TOC header in the code.
Regards,
Daniele
On Thu, 2021-02-11 at 10:14 +0000, Manish Badarkhe wrote:
> Hi Daniele
>
> You can use the ‘flag’ field to mention the platform-specific data(in
> your case, a build number). Usage of the ‘flag’ field(64 bit) in the
> toc_header are as below:
> Bits 0-31 -> reserved
> Bits 32-47 -> platform defined data
> Bits 48-63 -> reserved
> You can make use of the flag[32:47] to put build information. I am
> not sure if you can accommodate epoch (converted timestamp) into this
> field but, you can encode any data to fit into this 16bit flag field
> to identify the FIP build.
>
> You can use a build command: fiptool update/create --plat-toc-flags
> <platform defined data> <your fip bin path> to put the platform
> defined data in the FIP image.
>
> Thanks
> Manish Badarkhe
> From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of
> Daniele Alessandrelli via TF-A <tf-a(a)lists.trustedfirmware.org>
> Date: Wednesday, 10 February 2021 at 17:04
> To: tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>
> Subject: [TF-A] Getting BUILD_STRING from FIP file
>
> Hi,
>
> Is there a way to get BUILD_STRING (or a similar string / number that
> uniquely identifies the TF-A build, e.g., BUILD_MESSAGE_TIMESTAMP)
> from
> the FIP file?
>
> Basically, I'm trying to find a way to know the build number of a FIP
> without flashing it.
>
> I've seen that the FIP TOC header has a 32-bit field named
> 'serial_number'. Can it be used to this end? I'm considering
> converting BUILD_MESSAGE_TIMESTAMP into an epoch and adding it as
> 'serial number', but I'm worried that might be an unintended usage of
> the 'serial_number' field.
>
> Regards,
> Daniele
>
>
>
Hi all,
Recently we had an internal discussion on the merits of introducing semantics to commit messages pushed to the main TF-A repository, the conclusion being that we would look to adopting the Conventional Commits<https://www.conventionalcommits.org/en/v1.0.0/> specification in the near future. There was one major reason for this, which was to help us in automating the changelog in future releases, but it might also help us to dramatically reduce the overall amount of work needed to make a formal release in the future.
This requires some buy-in (or buy-out, in this case) from maintainers because - even though it's to only a relatively minor extent - it does involve an adjustment to everybody's workflow. Notably, commit messages will be expected to adopt the structure defined by the specification, which will be enforced by the CI. Most commits that go upstream today adhere to "something that looks like Conventional Commits", so the change is not exactly sweeping, but any change has the potential be an inconvenience.
With that in mind, I propose the following:
* We collectively adopt the specification, enforced only for @arm.com contributors until such a time that the majority of maintainers are familiar with the new demands
* We suggest - in the prerequisites documentation - the installation of two helper tools:
* Commitizen<https://github.com/commitizen/cz-cli>
* Commitlint<https://github.com/conventional-changelog/commitlint>
Installation of these tools will be optional, but I believe they can help with the transition. In the patches currently in review, they are installed as Git hooks automatically upon execution of npm install, so it requires no manual installation or configuration (other than a relatively up-to-date Node.js installation).
You'll find the patches here<https://review.trustedfirmware.org/q/topic:%22ck%252Fconventional-commits%2…>, and specifically the changes to the prerequisites documentation here<https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/8224/1/docs/…>. Feel free to review these changes if you have comments specifically on their implementation.
Let me know if you have any questions or concerns. If everybody's on board, we can look to have this upstreamed shortly.
Chris