Hi Harb,
This sounds like a useful abstraction to me. I can see it being useful
when we need to pass TPM logs from one stage to another, or to pass on
firmware update status. Things that /could/ be stuffed into a single
devicetree, but it is awkward to rewrite the devicetree for every piece
of dynamic data that gets generated and passed on. It would also be
helpful if a common approach can be used regardless of the normal-world
firmware (i.e., EDK2, U-Boot, or something else).
g.
On 22/03/2021 08:34, Harb Abdulhamid OS via TF-A wrote:
> Hello Folks,
>
> I'm emailing to start an open discussion about the adoption of a concept
> known as "hand-off blocks" or HOB to become a part of the TF-A Firmware
> Framework Architecture (FFA).� This is something that is a pretty major
> pain point when it comes to the adoption of TF-A in ARM Server SoC�s
> designed to enable a broad range of highly configurable datacenter
> platforms.
>
> What is a HOB (Background)?
>
> ---------------------------
>
> UEFI PI spec describes a particular definition for how HOB may be used
> for transitioning between the PEI and DXE boot phases, which is a good
> reference point for this discussion, but not necessarily the exact
> solution appropriate for TF-A.
>
> A HOB is simply a dynamically generated data structure passed in between
> two boot phases.� This is information that was obtained through
> discovery and needs to be passed forward to the next boot phase *once*,
> with no API needed to call back (e.g. no call back into previous
> firmware phase is needed to fetch this information at run-time - it is
> simply passed one time during boot).
>
> There may be one or more HOBs passed in between boot phases.� If there
> are more than one HOB that needs to be passed, this can be in a form of
> a "HOB table", which (for example) could be a UUID indexed array of
> pointers to HOB structures, used to locate a HOB of interest (based on
> UUID).� In such cases, instead of passing a single HOB, the boot phases
> may rely on passing the pointer to the HOB table.
>
> This has been extremely useful concept to employ on highly configurable
> systems that must rely on flexible discovery mechanisms to initialize
> and boot the system.� This is especially helpful when you have multiple
>
> Why do we need HOBs in TF-A?:
>
> -----------------------------
>
> It is desirable that EL3 firmware (e.g. TF-A) built for ARM Server SoC
> in a way that is SoC specific *but* platform agnostic.� This means that
> a single ARM SoC that a SiP may deliver to customers may provide a
> single TF-A binary (e.g. BL1, BL2, BL31) that could be used to support a
> broad range of platform designs and configurations in order to boot a
> platform specific firmware (e.g. BL33 and possibly even BL32 code).� In
> order to achieve this, the platform configuration must be *discovered*
> instead of statically compiled as it is today in TF-A via device tree
> based enumeration.� The mechanisms of discovery may differ broadly
> depending on the relevant industry standard, or in some cases may have
> rely on SiP specific discovery flows.
>
> For example:� On server systems that support a broad range DIMM memory
> population/topologies, all the necessary information required to boot is
> fully discovered via standard JEDEC Serial Presence Detect (SPD) over an
> I2C bus.� Leveraging the SPD bus, may platform variants could be
> supported with a single TF-A binary.� Not only is this information
> required to initialize memory in early boot phases (e.g. BL2), the
> subsequent boot phases will also need this SPD info to construct a
> system physical address map and properly initialize the MMU based on the
> memory present, and where the memory may be present.� Subsequent boot
> phases (e.g. BL33 / UEFI) may need to generate standard firmware tables
> to the operating systems, such as SMBIOS tables describing DIMM topology
> and various ACPI tables (e.g. SLIT, SRAT, even NFIT if NVDIMM's are
> present).
>
> In short, it all starts with a standardized or vendor specific discovery
> flow in an early boot stage (e.g. BL1/BL2), followed by the passing of
> information to the next boot stages (e.g. BL31/BL32/BL33).
>
> Today, every HOB may be a vendor specific structure, but in the future
> there may be benefit of defining standard HOBs.� This may be useful for
> memory discovery, passing the system physical address map, enabling TPM
> measured boot, and potentially many other common HOB use-cases.
>
> It would be extremely beneficial to the datacenter market segment if the
> TF-A community would adopt this concept of information passing between
> all boot phases as opposed to rely solely on device tree enumeration.
> This is not intended to replace device tree, rather intended as an
> alternative way to describe the info that must be discovered and
> dynamically generated.
>
> Conclusion:
>
> -----------
>
> We are proposing that the TF-A community begin pursuing the adoption of
> HOBs as a mechanism used for information exchange between each boot
> stage (e.g. BL1->BL2, BL2->BL31, BL31->BL32, and BL31->BL33)? Longer
> term we want to explore standardizing some HOB structures for the BL33
> phase (e.g. UEFI HOB structures), but initially would like to agree on
> this being a useful mechanism used to pass information between each boot
> stage.
>
> Thanks,
>
> --Harb
>
>
Hi Harb,
This sounds like a useful abstraction to me. I can see it being useful
when we need to pass TPM logs from one stage to another, or to pass on
firmware update status. Things that /could/ be stuffed into a single
devicetree, but it is awkward to rewrite the devicetree for every piece
of dynamic data that gets generated and passed on. It would also be
helpful if a common approach can be used regardless of the normal-world
firmware (i.e., EDK2, U-Boot, or something else).
g.
On 22/03/2021 08:34, Harb Abdulhamid OS via TF-A wrote:
> Hello Folks,
>
> I'm emailing to start an open discussion about the adoption of a concept
> known as "hand-off blocks" or HOB to become a part of the TF-A Firmware
> Framework Architecture (FFA).� This is something that is a pretty major
> pain point when it comes to the adoption of TF-A in ARM Server SoC�s
> designed to enable a broad range of highly configurable datacenter
> platforms.
>
> What is a HOB (Background)?
>
> ---------------------------
>
> UEFI PI spec describes a particular definition for how HOB may be used
> for transitioning between the PEI and DXE boot phases, which is a good
> reference point for this discussion, but not necessarily the exact
> solution appropriate for TF-A.
>
> A HOB is simply a dynamically generated data structure passed in between
> two boot phases.� This is information that was obtained through
> discovery and needs to be passed forward to the next boot phase *once*,
> with no API needed to call back (e.g. no call back into previous
> firmware phase is needed to fetch this information at run-time - it is
> simply passed one time during boot).
>
> There may be one or more HOBs passed in between boot phases.� If there
> are more than one HOB that needs to be passed, this can be in a form of
> a "HOB table", which (for example) could be a UUID indexed array of
> pointers to HOB structures, used to locate a HOB of interest (based on
> UUID).� In such cases, instead of passing a single HOB, the boot phases
> may rely on passing the pointer to the HOB table.
>
> This has been extremely useful concept to employ on highly configurable
> systems that must rely on flexible discovery mechanisms to initialize
> and boot the system.� This is especially helpful when you have multiple
>
> Why do we need HOBs in TF-A?:
>
> -----------------------------
>
> It is desirable that EL3 firmware (e.g. TF-A) built for ARM Server SoC
> in a way that is SoC specific *but* platform agnostic.� This means that
> a single ARM SoC that a SiP may deliver to customers may provide a
> single TF-A binary (e.g. BL1, BL2, BL31) that could be used to support a
> broad range of platform designs and configurations in order to boot a
> platform specific firmware (e.g. BL33 and possibly even BL32 code).� In
> order to achieve this, the platform configuration must be *discovered*
> instead of statically compiled as it is today in TF-A via device tree
> based enumeration.� The mechanisms of discovery may differ broadly
> depending on the relevant industry standard, or in some cases may have
> rely on SiP specific discovery flows.
>
> For example:� On server systems that support a broad range DIMM memory
> population/topologies, all the necessary information required to boot is
> fully discovered via standard JEDEC Serial Presence Detect (SPD) over an
> I2C bus.� Leveraging the SPD bus, may platform variants could be
> supported with a single TF-A binary.� Not only is this information
> required to initialize memory in early boot phases (e.g. BL2), the
> subsequent boot phases will also need this SPD info to construct a
> system physical address map and properly initialize the MMU based on the
> memory present, and where the memory may be present.� Subsequent boot
> phases (e.g. BL33 / UEFI) may need to generate standard firmware tables
> to the operating systems, such as SMBIOS tables describing DIMM topology
> and various ACPI tables (e.g. SLIT, SRAT, even NFIT if NVDIMM's are
> present).
>
> In short, it all starts with a standardized or vendor specific discovery
> flow in an early boot stage (e.g. BL1/BL2), followed by the passing of
> information to the next boot stages (e.g. BL31/BL32/BL33).
>
> Today, every HOB may be a vendor specific structure, but in the future
> there may be benefit of defining standard HOBs.� This may be useful for
> memory discovery, passing the system physical address map, enabling TPM
> measured boot, and potentially many other common HOB use-cases.
>
> It would be extremely beneficial to the datacenter market segment if the
> TF-A community would adopt this concept of information passing between
> all boot phases as opposed to rely solely on device tree enumeration.
> This is not intended to replace device tree, rather intended as an
> alternative way to describe the info that must be discovered and
> dynamically generated.
>
> Conclusion:
>
> -----------
>
> We are proposing that the TF-A community begin pursuing the adoption of
> HOBs as a mechanism used for information exchange between each boot
> stage (e.g. BL1->BL2, BL2->BL31, BL31->BL32, and BL31->BL33)? Longer
> term we want to explore standardizing some HOB structures for the BL33
> phase (e.g. UEFI HOB structures), but initially would like to agree on
> this being a useful mechanism used to pass information between each boot
> stage.
>
> Thanks,
>
> --Harb
>
>
Hi Peng,
+TF-A folk.
My 0.02$.
What is the problem you are trying to solve? Why do you need to run OP-TEE and EHF together? EHF was originally written to support a S-EL0 SP that is managed directly by TF-A in EL3 (TF-A folk can chime in).
The SP could perform RAS error handling for which it needs the EHF. The EHF triages asynchronous exceptions and hands RAS errors to the SP for further handling.
This is just one use case but there is no Trusted OS in these configurations.
So, it would help to understand the requirement.
cheers,
Achin
________________________________
From: OP-TEE <op-tee-bounces(a)lists.trustedfirmware.org> on behalf of Jens Wiklander via OP-TEE <op-tee(a)lists.trustedfirmware.org>
Sent: 17 March 2021 09:23
To: Peng Fan <peng.fan(a)nxp.com>
Cc: op-tee(a)lists.trustedfirmware.org <op-tee(a)lists.trustedfirmware.org>
Subject: Re: EHF + OPTEE on ARM64
On Wed, Mar 17, 2021 at 9:43 AM Peng Fan <peng.fan(a)nxp.com> wrote:
>
> > Subject: Re: EHF + OPTEE on ARM64
> >
> > On Wed, Mar 17, 2021 at 9:02 AM Peng Fan <peng.fan(a)nxp.com> wrote:
> > >
> > > > Subject: Re: EHF + OPTEE on ARM64
> > > >
> > > > On Wed, Mar 17, 2021 at 8:41 AM Peng Fan <peng.fan(a)nxp.com> wrote:
> > > > >
> > > > > > Subject: Re: EHF + OPTEE on ARM64
> > > > > >
> > > > > > On Tue, Mar 16, 2021 at 11:08 AM Peng Fan <peng.fan(a)nxp.com>
> > wrote:
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > In bl31/ehf.c, there are following two lines, per my
> > > > > > > understanding, when cpu is in secure world, the non-secure
> > > > > > > interrupt as FIQ(GICv3) will be directly catched by EL3, not S-EL1
> > > > > > > /* Route EL3 interrupts when in Secure and Non-secure.
> > */
> > > > > > > set_interrupt_rm_flag(flags, NON_SECURE);
> > > > > > > set_interrupt_rm_flag(flags, SECURE);
> > > > > > >
> > > > > > > So this will conflict with OP-TEE, because OP-TEE needs catch
> > > > > > > NS-interrupt as FIQ in S-EL1 world.
> > > > > >
> > > > > > In the case of GICv3, OP-TEE is configured to receive the
> > > > > > non-secure interrupts as FIQ and secure interrupts as IRQ. See
> > CFG_ARM_GICV3.
> > > > >
> > > > > But EHF needs NS-interrupt FIQ be catched by EL3 if I understand
> > > > > correct, per " set_interrupt_rm_flag(flags, SECURE);"
> > > > >
> > > > > So currently EHF could not work together with OP-TEE, right?
> > > >
> > > > To be honest, I'm not completely sure what EHF does. From OP-TEE
> > > > point of view we expect to receive the non-secure interrupts as a
> > > > way of doing a controlled exit. This allows OP-TEE to resume
> > > > execution with a different core on re-entry. If EL3 takes the
> > > > non-secure interrupts directly it will have to make sure to only re-enter
> > OP-TEE on this core as a return from exception.
> > >
> > > Is this easy to be achieved?
> >
> > I don't know, it depends on what you intend to do with this non-secure
> > interrupt. If it's handled at EL3 and then there's a return from exception back
> > to S-EL1 there's likely no harm done. But if there's a world switch involved
> > there might be trouble, OP-TEE might not be in a suitable state for a world
> > switch.
> >
> > >
> > > Or by using opteed_sel1_interrupt_handler, could we have similar
> > > behavior to allow the other core resume execution?
> >
> > Only OP-TEE itself can make a controlled exit as there's an internal state to
> > maintain. Currently that's signalled with a non-secure interrupt.
>
>
> Per EHF, https://trustedfirmware-a.readthedocs.io/en/latest/components/exception-han…
> On GICv3 systems, when executing in S-EL1, pending Non-secure interrupts of
> sufficient priority are signalled as FIQs, and therefore will be routed to EL3.
> As a result, S-EL1 software cannot expect to handle Non-secure interrupts at S-EL1.
> Essentially, this deprecates the routing mode described as CSS=0, TEL3=0.
>
> In order for S-EL1 software to handle Non-secure interrupts while having EHF enabled,
> the dispatcher must adopt a model where Non-secure interrupts are received at EL3,
> but are then synchronously handled over to S-EL1.
>
> The issue to me here how to synchronously handled over to S-EL1 and not break optee.
I understand. OP-TEE is masking interrupts in some critical sections,
while in such a state OP-TEE cannot handle any asynchronous interrupt.
Temporarily masking interrupts is normally a quick operation so we do
it in quite a few places.
So the crux of the problem is to make sure that OP-TEE is in a state
where it can make a controlled exit. I don't have any good ideas for
this right now.
Cheers,
Jens
Hello Folks,
I'm emailing to start an open discussion about the adoption of a concept known as "hand-off blocks" or HOB to become a part of the TF-A Firmware Framework Architecture (FFA). This is something that is a pretty major pain point when it comes to the adoption of TF-A in ARM Server SoC's designed to enable a broad range of highly configurable datacenter platforms.
What is a HOB (Background)?
---------------------------
UEFI PI spec describes a particular definition for how HOB may be used for transitioning between the PEI and DXE boot phases, which is a good reference point for this discussion, but not necessarily the exact solution appropriate for TF-A.
A HOB is simply a dynamically generated data structure passed in between two boot phases. This is information that was obtained through discovery and needs to be passed forward to the next boot phase *once*, with no API needed to call back (e.g. no call back into previous firmware phase is needed to fetch this information at run-time - it is simply passed one time during boot).
There may be one or more HOBs passed in between boot phases. If there are more than one HOB that needs to be passed, this can be in a form of a "HOB table", which (for example) could be a UUID indexed array of pointers to HOB structures, used to locate a HOB of interest (based on UUID). In such cases, instead of passing a single HOB, the boot phases may rely on passing the pointer to the HOB table.
This has been extremely useful concept to employ on highly configurable systems that must rely on flexible discovery mechanisms to initialize and boot the system. This is especially helpful when you have multiple
Why do we need HOBs in TF-A?:
-----------------------------
It is desirable that EL3 firmware (e.g. TF-A) built for ARM Server SoC in a way that is SoC specific *but* platform agnostic. This means that a single ARM SoC that a SiP may deliver to customers may provide a single TF-A binary (e.g. BL1, BL2, BL31) that could be used to support a broad range of platform designs and configurations in order to boot a platform specific firmware (e.g. BL33 and possibly even BL32 code). In order to achieve this, the platform configuration must be *discovered* instead of statically compiled as it is today in TF-A via device tree based enumeration. The mechanisms of discovery may differ broadly depending on the relevant industry standard, or in some cases may have rely on SiP specific discovery flows.
For example: On server systems that support a broad range DIMM memory population/topologies, all the necessary information required to boot is fully discovered via standard JEDEC Serial Presence Detect (SPD) over an I2C bus. Leveraging the SPD bus, may platform variants could be supported with a single TF-A binary. Not only is this information required to initialize memory in early boot phases (e.g. BL2), the subsequent boot phases will also need this SPD info to construct a system physical address map and properly initialize the MMU based on the memory present, and where the memory may be present. Subsequent boot phases (e.g. BL33 / UEFI) may need to generate standard firmware tables to the operating systems, such as SMBIOS tables describing DIMM topology and various ACPI tables (e.g. SLIT, SRAT, even NFIT if NVDIMM's are present).
In short, it all starts with a standardized or vendor specific discovery flow in an early boot stage (e.g. BL1/BL2), followed by the passing of information to the next boot stages (e.g. BL31/BL32/BL33).
Today, every HOB may be a vendor specific structure, but in the future there may be benefit of defining standard HOBs. This may be useful for memory discovery, passing the system physical address map, enabling TPM measured boot, and potentially many other common HOB use-cases.
It would be extremely beneficial to the datacenter market segment if the TF-A community would adopt this concept of information passing between all boot phases as opposed to rely solely on device tree enumeration. This is not intended to replace device tree, rather intended as an alternative way to describe the info that must be discovered and dynamically generated.
Conclusion:
-----------
We are proposing that the TF-A community begin pursuing the adoption of HOBs as a mechanism used for information exchange between each boot stage (e.g. BL1->BL2, BL2->BL31, BL31->BL32, and BL31->BL33)? Longer term we want to explore standardizing some HOB structures for the BL33 phase (e.g. UEFI HOB structures), but initially would like to agree on this being a useful mechanism used to pass information between each boot stage.
Thanks,
--Harb
We already have a Deprecated Interfaces removal policy for core interfaces: https://trustedfirmware-a.readthedocs.io/en/latest/about/release-informatio… although this is not quite the same for platforms the proposed purpose is similar of a managed removing of something that someone has a use case for in the short term.
The Arm platforms as mentioned by Manish are targeted as reference platforms for other project users to inspect which is perhaps a little different than other platform providers. So I think it is appropriate to have a transition period as those references platforms become outdated and normally replaced as it is with the sgm775 with the tc0 platform. I can understand for other platform providers they might want a more accelerated or removal approach and that’s fine I think.
I’m not suggesting we need to create a formal policy but leave it up to individual platform providers to decide how they manage their platforms as they know best how they are used.
Ensuring that the OpenCI is not left in a broken state from a build or test perspective during a deprecation period is of course necessary and the steps suggested by Manish look reasonable to me.
Hope this helps on the intent here.
Joanna
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of Varun Wadekar via TF-A <tf-a(a)lists.trustedfirmware.org>
Reply to: Varun Wadekar <vwadekar(a)nvidia.com>
Date: Thursday, 11 March 2021 at 22:12
To: Manish Pandey2 <Manish.Pandey2(a)arm.com>
Cc: "tf-a(a)lists.trustedfirmware.org" <tf-a(a)lists.trustedfirmware.org>
Subject: Re: [TF-A] Deprecating Arm's sgm775 platform
Hi,
I was wondering if there is any value in keeping non-compiling code in the tree, hence the question. For Tegra it is straight forward, so it would be easy to pull the plug.
-Varun
From: Manish Pandey2 <Manish.Pandey2(a)arm.com>
Sent: Thursday, March 11, 2021 2:44 AM
To: Varun Wadekar <vwadekar(a)nvidia.com>
Cc: tf-a(a)lists.trustedfirmware.org
Subject: Re: Deprecating Arm's sgm775 platform
External email: Use caution opening links or attachments
Hi Varun,
For Arm reference platforms, we are not sure who all are using it. That is why i proposed to keep in repo for around a year before deleting it.
But for NV platforms, if you are sure that nobody is going to require it, you can delete it earlier.
Also, instead of proposed step 2 earlier we can have an alternate
- 2. Don't allow it to be built by default. (introducing PLATFORM_DEPRECATED build macro)
+ 2. Instead of purposefully failing the build we can print a warning message (It's also quite possible that someday during cooling off period it stops to build naturally)
thanks
Manish
________________________________
From: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>
Sent: 11 March 2021 02:35
To: Manish Pandey2 <Manish.Pandey2(a)arm.com<mailto:Manish.Pandey2@arm.com>>
Cc: tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org> <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Subject: RE: Deprecating Arm's sgm775 platform
Hi Manish,
Just curious, what would be the reason to keep the platform alive for 2 release cycles? I have one Tegra platform that needs to be deprecated and so would like to understand the thought process.
-Varun
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org<mailto:tf-a-bounces@lists.trustedfirmware.org>> On Behalf Of Manish Pandey2 via TF-A
Sent: Tuesday, March 9, 2021 2:01 PM
To: tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>
Subject: [TF-A] Deprecating Arm's sgm775 platform
External email: Use caution opening links or attachments
Hi,
The purpose of the email is to notify about deprecation of sgm775 platform and proposed process for deprecating a platform.
Arm's System Guidance for Mobile(SGM-775) is an old platform and no longer maintained. It is superseded by Total Compute(tc0) platform.
Proposal for deprecating a platform:
1. Keep the code in repository. (at least for 2 release cycles)
2. Don't allow it to be built by default. (introducing PLATFORM_DEPRECATED build macro)
3. Disable CI testing.
4. Create appropriate documentation for deprecated platforms.
Let me know if you have any suggestions.
Thanks
Manish P
Hi,
Thank you once again for more of your comments.
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6122<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…> to
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6167
All the patches from the patch-set, are reviewed and all the comments are disposed-off till date.
Please share any further action, I need to work on.
Looking forward for your continued support.
Regards
Pankaj
From: Varun Wadekar <vwadekar(a)nvidia.com>
Sent: Tuesday, February 2, 2021 12:11 AM
To: Pankaj Gupta <pankaj.gupta(a)nxp.com>; Manish Pandey2 <Manish.Pandey2(a)arm.com>; Olivier Deprez <Olivier.Deprez(a)arm.com>; javier.almansasobrino(a)arm.com; jimmy.brisson(a)arm.com
Cc: Joanna Farley <Joanna.Farley(a)arm.com>; Alexei Fedorov <Alexei.Fedorov(a)arm.com>; Madhukar Pappireddy <Madhukar.Pappireddy(a)arm.com>
Subject: [EXT] RE: NXP Patch-Set for platform lx2160ardb/lx2160aqds/lx2162aqds
Caution: EXT Email
Thanks Pankaj. I don't have further comments.
From: Pankaj Gupta <pankaj.gupta(a)nxp.com<mailto:pankaj.gupta@nxp.com>>
Sent: Sunday, January 31, 2021 11:14 PM
To: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>; Manish Pandey2 <Manish.Pandey2(a)arm.com<mailto:Manish.Pandey2@arm.com>>; Olivier Deprez <Olivier.Deprez(a)arm.com<mailto:Olivier.Deprez@arm.com>>; javier.almansasobrino(a)arm.com<mailto:javier.almansasobrino@arm.com>; jimmy.brisson(a)arm.com<mailto:jimmy.brisson@arm.com>
Cc: Joanna Farley <Joanna.Farley(a)arm.com<mailto:Joanna.Farley@arm.com>>; Alexei Fedorov <Alexei.Fedorov(a)arm.com<mailto:Alexei.Fedorov@arm.com>>; Madhukar Pappireddy <Madhukar.Pappireddy(a)arm.com<mailto:Madhukar.Pappireddy@arm.com>>
Subject: NXP Patch-Set for platform lx2160ardb/lx2160aqds/lx2162aqds
External email: Use caution opening links or attachments
Hi all,
Thanks to all of you, for your efforts in reviewing the patches.
All the patches from the patch-set, are reviewed and all the comments are disposed-off till date.
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6122<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…> to https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6167<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…>
Please share any further action, I need to work on.
Looking forward for your continued support.
Thanks & regards
Pankaj
From: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>
Sent: Friday, December 4, 2020 10:37 PM
To: Pankaj Gupta <pankaj.gupta(a)nxp.com<mailto:pankaj.gupta@nxp.com>>; Manish Pandey2 <Manish.Pandey2(a)arm.com<mailto:Manish.Pandey2@arm.com>>
Cc: Joanna Farley <Joanna.Farley(a)arm.com<mailto:Joanna.Farley@arm.com>>; Alexei Fedorov <Alexei.Fedorov(a)arm.com<mailto:Alexei.Fedorov@arm.com>>; Madhukar Pappireddy <Madhukar.Pappireddy(a)arm.com<mailto:Madhukar.Pappireddy@arm.com>>
Subject: RE: [EXT] RE: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6122<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…>
Caution: EXT Email
Thanks Pankaj and Manish. Glad to see us agreeing to a path forward.
From: Pankaj Gupta <pankaj.gupta(a)nxp.com<mailto:pankaj.gupta@nxp.com>>
Sent: Thursday, December 3, 2020 11:39 PM
To: Manish Pandey2 <Manish.Pandey2(a)arm.com<mailto:Manish.Pandey2@arm.com>>; Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>
Cc: Joanna Farley <Joanna.Farley(a)arm.com<mailto:Joanna.Farley@arm.com>>; Alexei Fedorov <Alexei.Fedorov(a)arm.com<mailto:Alexei.Fedorov@arm.com>>; Madhukar Pappireddy <Madhukar.Pappireddy(a)arm.com<mailto:Madhukar.Pappireddy@arm.com>>
Subject: RE: [EXT] RE: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6122<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…>
External email: Use caution opening links or attachments
Hi,
Thanks for the email.
Your suggestion is good too. But with your suggestion, two lines are need for one line using the macro.
My concern is that, there are 6 SoC and 15+ platforms based on these 6 SoC, still pending to be added to this code base.
Keeping things simple in soc.mk and platform.mk is of key importance.
To achieve it:
* Complexity of source file addition is moved away from platform makefile to:
o drivers/nxp/driver.mk, and
o drivers/nxp/<ip>/<ip>.mk
* As a result, the soc.mk & platform.mk is less cluttered.
o With the single line addition based on this macro, it is easier to understand, which IP is part of BL2, BL31 or both.
o All the complexity about IP files to be included or not is moved to drivers/nxp/<ip>/<ip>.mk.
I got your point here.
Since it is very much specific to NXP platforms, it is to be moved to "plat/nxp".
I have tried this way. It is working and ready for review.
Regards
Pankaj
From: Manish Pandey2 <Manish.Pandey2(a)arm.com<mailto:Manish.Pandey2@arm.com>>
Sent: Friday, December 4, 2020 6:24 AM
To: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>; Pankaj Gupta <pankaj.gupta(a)nxp.com<mailto:pankaj.gupta@nxp.com>>
Cc: Joanna Farley <Joanna.Farley(a)arm.com<mailto:Joanna.Farley@arm.com>>; Alexei Fedorov <Alexei.Fedorov(a)arm.com<mailto:Alexei.Fedorov@arm.com>>; Madhukar Pappireddy <Madhukar.Pappireddy(a)arm.com<mailto:Madhukar.Pappireddy@arm.com>>
Subject: Re: [EXT] RE: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6122<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…>
Caution: EXT Email
Hi Pankaj,
I am still trying to understand the complete patchset, also trying to understand different level of makefiles e.g drivers/nxp/drivers.mk(is it really needed?)
I think the main concern is complexity added by intermediate mk files, this complexity can be reduced by declaring most of things in specific platform mk file as we know at compile time which all images need a particular driver.
For e.g XSPI driver, it has same source files in all the 3 scenarios (BL2/BL31 & COMMON) and there is no usage of IMAGE_BL31, IMAGE_BL2 in xspi driver suggesting that Image type does not alters functionality of driver.
So, what we can do is keeping driver mk file simpler and other details in platform mk file.
flexspi_xor.mk will be like:
XSPI_BOOT_SOURCES += $(FLEXSPI_DRIVERS_PATH)/flexspi_nor.c \
${FLEXSPI_DRIVERS_PATH}/fspi.c
ifeq ($(DEBUG),1)
XSPI_BOOT_SOURCES += ${FLEXSPI_DRIVERS_PATH}/test_fspi.c
endif
Platform mk file will be like: (say only bl2 needs xspi)
XSPI_NEEDED := yes
bl2_sources += ${XSPI_BOOT_SOURCES}
Regarding NEED_BL31/NEED_BL2, these flags tell if binary for given BL stage needs to be generated or not.
Finally, I am not saying that your approach is wrong but what i suggested is currently done by most of the platforms.
Also, see my comments on your SET_FLAG patch, if we indeed decide to go ahead with your current approach.
Thanks
Manish
________________________________
From: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>
Sent: 03 December 2020 17:35
To: Pankaj Gupta <pankaj.gupta(a)nxp.com<mailto:pankaj.gupta@nxp.com>>
Cc: Manish Pandey2 <Manish.Pandey2(a)arm.com<mailto:Manish.Pandey2@arm.com>>
Subject: RE: [EXT] RE: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6122<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…>
Hi,
Can you point me to a change that uses the newly introduced makefile macro? IIUC, you just need a way to differentiate between a BL2 v BL31 build.
Manish can you confirm if NEED_BL31 and NEED_BL2 can be helpful in this case?
>> I will replace the SET_FLAG macro with these two flags from entire source tree.
I suppose you are alluding to the NXP platform port only. Correct?
-Varun
From: Pankaj Gupta <pankaj.gupta(a)nxp.com<mailto:pankaj.gupta@nxp.com>>
Sent: Thursday, December 3, 2020 12:57 AM
To: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>
Cc: Manish Pandey2 <Manish.Pandey2(a)arm.com<mailto:Manish.Pandey2@arm.com>>
Subject: RE: [EXT] RE: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6122<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…>
External email: Use caution opening links or attachments
Hi Varun,
Thanks for your email.
NXP make files are implemented as :
1. Each IP driver has its own .mk file.
2. Each platform has its own "platform.mk".
* Every "platform.mk" includes "soc.mk" for the SoC on which this platform is based.
1. Based on the SoC, mandatory IP(s) are included(soc.mk) to BL2 or BL31 or both.
* This requires to pass the flag "<BL2 or BL31 or BL_COMM>_<IP>_NEEDED = yes" from soc.mk to mandatory IP(s) .mk.
1. But for certain IP(s), IP file sources inclusions to either BL2 or BL31, is based on optional features.
* This also requires to pass the flag to IP(s) .mk.
For instance:
* If flexspi_nor as a boot-source, This macro sets 2 flags.
i. XSPI_NEEDED = yes
* XSPI_NEEDED help identify if the xspi.mk needs to be included or not.
ii. BL2_ XSPI_NEEDED = yes
* XSPI_NEEDED needs to be included in BL2
* In optional feature WARM_RESET, I need to save the last reset_cause in flexspi_nor in BL31.
i. Again need two flags: XSPI_NEEDED = yes & BL31_ XSPI_NEEDED = yes
ii. XSPI_NEEDED = yes, is still required as it might happen the boot source is SD.
* In this case BL2_XSPI_NEEDED, is not set.
What I am gaining with this macros is:
* Without this macro, I need to add two lines:
* <IP>_NEEDED = yes.
* To include this driver.
* One of the flag to be added depending on the IP source file inclusion to BL2 or BL31 or BL_COMM:
* Flag as BL2_<IP>_NEEDED = yes
* Flag as BL31_<IP>_NEEDED = yes.
* Flag as BLCOMM_<IP>_NEEDED = yes.
* This macros helps me add one line $(eval $(call SET_FLAG, <IP>_NEEDED,BL2)), instead of above two lines.
If you suggest to remove the SET_FLAG macro, I will replace the SET_FLAG macro with these two flags from entire source tree.
Please share your view.
I reviewed the usage of NEED_BL31 and NEED_BL2.
They are set to 'yes' in ./Makefile, only when BL2_SOURCES & BL31_SOURCES is non-null;
They can be over-ridden, but cannot be used for each IP(s) source file.
Regards
Pankaj
From: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>
Sent: Thursday, December 3, 2020 4:17 AM
To: Pankaj Gupta <pankaj.gupta(a)nxp.com<mailto:pankaj.gupta@nxp.com>>
Cc: Manish Pandey2 <Manish.Pandey2(a)arm.com<mailto:Manish.Pandey2@arm.com>>
Subject: [EXT] RE: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6122<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…>
Caution: EXT Email
Hi,
>From your description, I think you are including different makefiles from NXP tree for BL31 versus BL2. Can you please help me understand why this decision needs to be taken at the top level makefile? Alternatively, why can't NXP platform decide which files to include?
Did you get a chance to review the NEED_BL31, NEED_BL2 makefile variables?
>From the gerrit change it is very difficult to understand how the newly introduced macro is used, so trying to suggest already available options to see if they work for you.
-Varun
From: Pankaj Gupta <pankaj.gupta(a)nxp.com<mailto:pankaj.gupta@nxp.com>>
Sent: Wednesday, December 2, 2020 5:21 AM
To: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>
Cc: Manish Pandey2 <Manish.Pandey2(a)arm.com<mailto:Manish.Pandey2@arm.com>>
Subject: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6122<https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…>
External email: Use caution opening links or attachments
Hi Varun,
As suggested by you, IMAGE_BL31, IMAGE_BL2 etc macros are useful for segregating the code within the same source file.
But this will not serve the purpose in our case.
Let me try to explain you with an example:
-- For one platform; and for a particulate IP driver's source file, if it is true to have in both, then :
#if defined(IMAGE_BL2) || #if defined(IMAGE_BL31)
-- But for another platform and for the same IP driver's source file, if it is true to have in BL2 only, then:
#if defined(IMAGE_BL2)
-- And for another SoC and for the same IP driver's source file, it if is true to have in BL31 only, then:
#if defined(IMAGE_BL31)
It will not be possible to write all the three varying inclusion of code for one IP used across multiple SoC and their platforms.
Now, I will explain how this macro helps me with above case:
* Taking an example of flexspi_nor as a boot-source:
* $(eval $(call SET_FLAG, XSPI_NEEDED,BL2))
* This macro sets 2 flags.
* XSPI_NEEDED = yes
* XSPI_NEEDED help identify if the xspi.mk needs to be included or not.
* BL2_ XSPI_NEEDED = yes
* XSPI_NEEDED needs to be included in BL2.
* For a conditional feature to enable in BL31, XSPI is to be included in BL31.
* BL31_XSPI_NEEDED = yes needs to be set.
* The correct solution should be if the feature is enabled, then:
$(eval $(call SET_FLAG, XSPI_NEEDED,BL31))
* In this case, I cannot set BL_COMM_XSPI_NEED = yes for all the platforms.
I hope, I am able to convey my thoughts to you.
This macro is very important for my code orientation. If you think it will not be required by other contributors, then lets rename this macro by prepending it with NXP or any name you suggests.
Thanks.
Regards
Pankaj
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Manish,
Just curious, what would be the reason to keep the platform alive for 2 release cycles? I have one Tegra platform that needs to be deprecated and so would like to understand the thought process.
-Varun
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Manish Pandey2 via TF-A
Sent: Tuesday, March 9, 2021 2:01 PM
To: tf-a(a)lists.trustedfirmware.org
Subject: [TF-A] Deprecating Arm's sgm775 platform
External email: Use caution opening links or attachments
Hi,
The purpose of the email is to notify about deprecation of sgm775 platform and proposed process for deprecating a platform.
Arm's System Guidance for Mobile(SGM-775) is an old platform and no longer maintained. It is superseded by Total Compute(tc0) platform.
Proposal for deprecating a platform:
1. Keep the code in repository. (at least for 2 release cycles)
2. Don't allow it to be built by default. (introducing PLATFORM_DEPRECATED build macro)
3. Disable CI testing.
4. Create appropriate documentation for deprecated platforms.
Let me know if you have any suggestions.
Thanks
Manish P
This event has been cancelled.
Title: TF-A Tech Forum
We run an open technical forum call for anyone to participate and it is not
restricted to Trusted Firmware project members. It will operate under the
guidance of the TF TSC. Feel free to forward this invite to
colleagues. Invites are via the TF-A mailing list and also published on the
Trusted Firmware website. Details are
here: https://www.trustedfirmware.org/meetings/tf-a-technical-forum/Tr…
Firmware is inviting you to a scheduled Zoom meeting.Join Zoom
Meetinghttps://zoom.us/j/9159704974Meeting ID: 915 970 4974One tap
mobile+16465588656,,9159704974# US (New York)+16699009128,,9159704974# US
(San Jose)Dial by your location +1 646 558
8656 US (New York) +1 669 900
9128 US (San Jose) 877 853 5247 US
Toll-free 888 788 0099 US Toll-freeMeeting ID:
915 970 4974Find your local
number: https://zoom.us/u/ad27hc6t7h
When: Thu 11 Mar 2021 16:00 – 17:00 United Kingdom Time
Calendar: tf-a(a)lists.trustedfirmware.org
Who:
* Bill Fletcher- creator
* marek.bykowski(a)gmail.com
* okash.khawaja(a)gmail.com
* tf-a(a)lists.trustedfirmware.org
Invitation from Google Calendar: https://calendar.google.com/calendar/
You are receiving this courtesy email at the account
tf-a(a)lists.trustedfirmware.org because you are an attendee of this event.
To stop receiving future updates for this event, decline this event.
Alternatively, you can sign up for a Google Account at
https://calendar.google.com/calendar/ and control your notification
settings for your entire calendar.
Forwarding this invitation could allow any recipient to send a response to
the organiser and be added to the guest list, invite others regardless of
their own invitation status or to modify your RSVP. Learn more at
https://support.google.com/calendar/answer/37135#forwarding
Hi Gyorgy,
I've added a new mail address in gerrit last month.
I can see both addresses in gerrit Email Addresses.
I've chosen the new one as my preferred address.
And I only receive gerrit mails on this preferred address.
Best regards,
Yann
On 3/10/21 4:26 PM, Gyorgy Szing via TF-A wrote:
> Hi,
>
> I just tried it and got the notification e-mail.
>
> Note: I filled the "New email address" text box and pressing "Send Verification" below. If that adds an additional email or replaces the current one is unknown to me.
>
> /George
>
> -----Original Message-----
> From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Igor Opaniuk via TF-A
> Sent: 10 March 2021 13:44
> To: tf-a(a)lists.trustedfirmware.org
> Subject: [TF-A] gerrit issues: not able to add additional email
>
> Hi,
>
> Gerrit for some reason doesn't send verification emails when adding additional email in account settings [1].
> I've tried to add two different emails, in both cases with no success.
> Anyone could quickly check if you can receive verification?
>
> Thanks
>
> [1] https://review.trustedfirmware.org/settings#EmailAddresses
>
> --
> Best regards - Freundliche Grüsse - Meilleures salutations
>
> Igor Opaniuk
>
> mailto: igor.opaniuk(a)gmail.com
> skype: igor.opanyuk
> +380 (93) 836 40 67
> http://ua.linkedin.com/in/iopaniuk
> --
> TF-A mailing list
> TF-A(a)lists.trustedfirmware.org
> https://lists.trustedfirmware.org/mailman/listinfo/tf-a
>
Hi,
I just tried it and got the notification e-mail.
Note: I filled the "New email address" text box and pressing "Send Verification" below. If that adds an additional email or replaces the current one is unknown to me.
/George
-----Original Message-----
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Igor Opaniuk via TF-A
Sent: 10 March 2021 13:44
To: tf-a(a)lists.trustedfirmware.org
Subject: [TF-A] gerrit issues: not able to add additional email
Hi,
Gerrit for some reason doesn't send verification emails when adding additional email in account settings [1].
I've tried to add two different emails, in both cases with no success.
Anyone could quickly check if you can receive verification?
Thanks
[1] https://review.trustedfirmware.org/settings#EmailAddresses
--
Best regards - Freundliche Grüsse - Meilleures salutations
Igor Opaniuk
mailto: igor.opaniuk(a)gmail.com
skype: igor.opanyuk
+380 (93) 836 40 67
http://ua.linkedin.com/in/iopaniuk
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Hi,
Gerrit for some reason doesn't send verification emails when adding
additional email in account settings [1].
I've tried to add two different emails, in both cases with no success.
Anyone could quickly check if you can receive verification?
Thanks
[1] https://review.trustedfirmware.org/settings#EmailAddresses
--
Best regards - Freundliche Grüsse - Meilleures salutations
Igor Opaniuk
mailto: igor.opaniuk(a)gmail.com
skype: igor.opanyuk
+380 (93) 836 40 67
http://ua.linkedin.com/in/iopaniuk
Apologies for the late notice I am cancelling this weeks TF-A Tech forum tomorrow as we don’t have any subjects ready to present this week.
We expect to have subjects for the next two sessions on 25th March and 8th April. Any subjects for future Tech-Forums from the contributor community always welcome so please reach out and I will help schedule. These can be more formal presentations or led discussions on subjects of interest to the TF-A project community.
Cancellation of the calendar invite will come from trustedformware.org as I don’t own the invite so it may not appear in your calendars until that is sent out.
Thanks
Joanna
Hi,
The purpose of the email is to notify about deprecation of sgm775 platform and proposed process for deprecating a platform.
Arm's System Guidance for Mobile(SGM-775) is an old platform and no longer maintained. It is superseded by Total Compute(tc0) platform.
Proposal for deprecating a platform:
1. Keep the code in repository. (at least for 2 release cycles)
2. Don't allow it to be built by default. (introducing PLATFORM_DEPRECATED build macro)
3. Disable CI testing.
4. Create appropriate documentation for deprecated platforms.
Let me know if you have any suggestions.
Thanks
Manish P
Hi,
Please find the latest report on new defect(s) introduced to ARM-software/arm-trusted-firmware found with Coverity Scan.
1 new defect(s) introduced to ARM-software/arm-trusted-firmware found with Coverity Scan.
New defect(s) Reported-by: Coverity Scan
Showing 1 of 1 defect(s)
** CID 367340: Control flow issues (MISSING_BREAK)
/plat/mediatek/mt8192/drivers/spm/mt_spm_vcorefs.c: 372 in spm_vcorefs_args()
________________________________________________________________________________________________________
*** CID 367340: Control flow issues (MISSING_BREAK)
/plat/mediatek/mt8192/drivers/spm/mt_spm_vcorefs.c: 372 in spm_vcorefs_args()
366 uint64_t spm_vcorefs_args(uint64_t x1, uint64_t x2, uint64_t x3, uint64_t *x4)
367 {
368 uint64_t cmd = x1;
369 uint64_t spm_flags;
370
371 switch (cmd) {
>>> CID 367340: Control flow issues (MISSING_BREAK)
>>> The case for value "VCOREFS_SMC_CMD_INIT" is not terminated by a "break" statement.
372 case VCOREFS_SMC_CMD_INIT:
373 /* vcore_dvfs init + kick */
374 mmio_write_32(DVFSRC_SW_REQ5, SW_REQ5_INIT_VAL);
375 spm_dvfsfw_init(0ULL, 0ULL);
376 spm_vcorefs_vcore_setting(x3 & 0xF);
377 spm_flags = SPM_FLAG_RUN_COMMON_SCENARIO;
________________________________________________________________________________________________________
To view the defects in Coverity Scan visit, https://u15810271.ct.sendgrid.net/ls/click?upn=HRESupC-2F2Czv4BOaCWWCy7my0P…
I was away last week for the tech forum and not had time to watch the recording which I need to do before joining some of this discussions but I can clarify the purpose of the Changelog and the difference with the git history.
The Changelog is part of the release documentation that summarises the main feature and changes that the release delivers. Its not expected every change in the git log history is included it is about recording the main themes of changes grouped together rather than chronologically. Most importantly it meant to be readable and understandable. I would expect as with any such documentation that some manual editing may be needed but the ask was for suggestions to make this task easier. As it stands today creating the Changelog can take days of manually reading the git history and re-writing, infact it’s the most labour intensive single task we do in a release.
So guidance, formatting and tooling in creating commit messages so we can reuse them in release documentation is the ask. If this helps consumption of the git log history for other purposes that’s a great bonus. Tooling and automation makes things more efficient for all. Making submitting patches from developers harder is defiantly not the goal, hopefully the tooling makes the effort easier or at least the same level of effort for the developer, if not the solution being sought needs improving.
Now we could do away with the release Changelog as it is today but the git log history is not a replacement for user facing release documentation. Before you know it we do away with all documentation and tell consumers to just read the source code 😉 Quote: “… documentation may mean different things to people in different roles.”
Joanna
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of Gyorgy Szing via TF-A <tf-a(a)lists.trustedfirmware.org>
Reply to: Gyorgy Szing <Gyorgy.Szing(a)arm.com>
Date: Wednesday, 3 March 2021 at 13:26
To: Chris Kay <Chris.Kay(a)arm.com>, Varun Wadekar <vwadekar(a)nvidia.com>, "tf-a(a)lists.trustedfirmware.org" <tf-a(a)lists.trustedfirmware.org>
Cc: nd <nd(a)arm.com>
Subject: Re: [TF-A] Adoption of Conventional Commits
Hi,
I think the basic question is: what is the difference between the change-log and the git history?
Depending on how we draw the line between the release notes and the change log the answer can be: not much. The change log mostly filters and extends the git history. And this filtering and extending needs a lot of manual work currently. But why we wanted to have two change-logs then? The real difference is the presentation format (reST/HTML vs git log), and the tooling you need to be able to read.
If the above is true, then the git log -> changelog transformation can be automated, but that needs the git history being machine readable. For developers this creates the requirement to properly format the commit message, and for reviewers adds extra work too. But that can be automated too right? And this is why we need tooling. Tooling on commit message authoring can be optional, but validation tools are mandatory. Otherwise we will end up with badly formatted commit messages (yes, manual validation is boring an error prone), failing automated translation, and the whole effort misses it’s main point.
(And as a side effect we also get a git hook framework, which is making a step forward with standardizing distributed automation.)
/George
From: Chris Kay <Chris.Kay(a)arm.com>
Sent: 03 March 2021 03:56
To: Varun Wadekar <vwadekar(a)nvidia.com>; Gyorgy Szing <Gyorgy.Szing(a)arm.com>; tf-a(a)lists.trustedfirmware.org
Cc: nd <nd(a)arm.com>
Subject: Re: [TF-A] Adoption of Conventional Commits
Hi Varun,
> I think you just increased the scope of the problem. We should add that as a new requirement – the commit message header should be pretty.
I don't think the scope has increased, but perhaps the requirement that we are able to generate the changelog was lacking clarity; it's not necessarily that the commit message headers should be pretty, but that the changelog should remain so to the extent that it can - it is still user-facing documentation, after all. By extension, we gain nothing from using the commit log to generate the changelog if they just mirror one another.
> Honestly, we should also check if in an effort to make the changelog “pretty”, are we losing the traditional git log formatting. Honestly, the git log gets used more than the changelog, so your proposal of changing the commit header has a greater impact. I would like to make this low impact to the developers that create patches on a daily basis.
The point I'm trying to emphasise is that there is no traditional Git log formatting - as it stands today, our commit guidelines make no mention of tags. As a result, the tags we do see vary drastically, from none at all to generic "TF-A:" tags, to platforms, drivers and sometimes to specific files. Everybody has their own status quo which they obviously want to maintain, but at some point we have to try to bring everybody onto the same page - commit style rules are not particularly rare for the same reasons code style rules aren't. I don't think the CC rules deviate all that much from the styles we most often see today.
> Introducing a completely new way of creating the commit message header or introducing more scripts to create that format is a no-no.
There are no mandatory scripts involved - you can continue to write your commits as you do today. The only tangible difference is that we are standardising the tag syntax.
> Personally, I feel that you are getting the required information from the git log by just adding tags, which to me, seems like a very low impact approach.
Incidentally, it was this very disagreement that brought on this investigation - you can see exactly what the v2.4 changelog looked like<https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/6654/1/docs/…> after basic categorisation, which is where it was decided that a straight dump of the commit log did not suffice for user documentation.
> Isnt that an easy fix? We just don’t add tags to such commits. I don’t see how “Conventional Commits” is better.
You can avoid tagging the revert commit, but you still need to detect whether the probably-tagged commit it reverts was merged before or after last release, and remove it from the log if the latter. I would suggest Conventional Commits is "better" because we don't even have to consider edge cases like these - we've done the configuration, we know it sorts this out for us, and there's nothing more we need to do to make it just work.
> As a maintainer, I feel that forcing developers to unlearn the standard way used by almost all other OSS projects, is disruptive. I am all for automating as long as the process does not get in my way every day.
But there is no "standard way" - some projects use "component: xyz" (e.g. Linux), others "[component] xyz" (e.g. LLVM), others yet don't use a tag at all (e.g. Mbed TLS), and I would argue most are like us: lax enough that it's largely down to individual contributors to determine their own. This just happens to be one style among many (and, as far as I know, the only one with an entire tooling ecosystem). I appreciate that you have a favoured variant, but I don't think it's any more useful to debate the most popular commit styles than it is to debate the most popular code styles.
As we can see today though, TF-A's existing commit guidelines go largely ignored, and it's our intention to rectify that in a way that allows us to do something useful with information that was previously inaccessible. I won't try to argue that enforcing something that wasn't previously enforced takes some initial getting used to, but I think the emphasis on the "extra work for committers" is severely overrepresented here - realistically, it's a minimal change to how we format the tags that we already write, and it's something some of us have already had to get used to (and have, honestly with very little effort).
> I think any proposal should be scalable and forward looking. I’m sure we will hit a scenario where someone needs custom tags and this proposal does not allow us that flexibility.
It does afford us that flexibility - we can extend the list of supported types, I'm just unsure of why we might. We would not have settled on this solution if we did not believe it to be scalable and, considering it does already see widespread usage, I would argue it's a relatively safe bet that it can handle most, if not all, of what we need now and in the future.
Chris
________________________________
From: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>
Sent: 03 March 2021 01:26
To: Chris Kay <Chris.Kay(a)arm.com<mailto:Chris.Kay@arm.com>>; Gyorgy Szing <Gyorgy.Szing(a)arm.com<mailto:Gyorgy.Szing@arm.com>>; tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org> <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Cc: nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: RE: [TF-A] Adoption of Conventional Commits
Hi Chris,
I think you just increased the scope of the problem. We should add that as a new requirement – the commit message header should be pretty.
Honestly, we should also check if in an effort to make the changelog “pretty”, are we losing the traditional git log formatting. Honestly, the git log gets used more than the changelog, so your proposal of changing the commit header has a greater impact. I would like to make this low impact to the developers that create patches on a daily basis. Introducing a completely new way of creating the commit message header or introducing more scripts to create that format is a no-no.
Personally, I feel that you are getting the required information from the git log by just adding tags, which to me, seems like a very low impact approach.
On the two examples, I don’t see a big difference in the supposedly human readable log you posted. But the proposal to get that is disruptive.
>> You can see here that it emphasises the scope for each change for human readability, and also omits both the revert commit and the commit it reverts because neither of them have been part of a release
Isnt that an easy fix? We just don’t add tags to such commits. I don’t see how “Conventional Commits” is better.
>> I think burdening reviewers with additional work is likely to prove unreliable, and certainly counter-productive if we can both largely automate the problem away and provide rapid feedback to developers before ever even having to push for review
As a maintainer, I feel that forcing developers to unlearn the standard way used by almost all other OSS projects, is disruptive. I am all for automating as long as the process does not get in my way every day.
>> The tooling I proposed does already offer some flexibility for defining our own types and scopes, though the default set is already pretty extensive
I think any proposal should be scalable and forward looking. I’m sure we will hit a scenario where someone needs custom tags and this proposal does not allow us that flexibility.
-Varun
From: Chris Kay <Chris.Kay(a)arm.com<mailto:Chris.Kay@arm.com>>
Sent: Tuesday, March 2, 2021 4:21 PM
To: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>; Gyorgy Szing <Gyorgy.Szing(a)arm.com<mailto:Gyorgy.Szing@arm.com>>; tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>
Cc: nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: Re: [TF-A] Adoption of Conventional Commits
External email: Use caution opening links or attachments
Hi guys,
Note: a lot of this email relies on HTML formatting to force monospace fonts and emphasis – it probably won’t show up correctly on the mailing lists archives.
One major point of contention with this model is that it’s not immediately clear what goes into the changelog. The obvious first answer is “the commit subject”, but let’s investigate that.
Here are the last 17 commits from upstream as of right now:
plat/marvell/armada: cleanup MSS SRAM if used for copy
plat/marvell: cn913x: allow CP1/CP2 mapping at BLE stage
plat/marvell/armada/common/mss: use MSS SRAM in secure mode
libc: memset: Fix MISRA issues
plat:xilinx:zynqmp: Remove the custom crash implementation
lib: cpus: aarch32: sanity check pointers before use
Revert "spmd: ensure SIMD context is saved/restored on SPMC entry/exit"
plat/arm/css: rename rd_n1e1_edge_scmi_plat_info array
docs: stm32mp1: correct formatting issues
marvell: uart: a3720: Increase TX FIFO EMPTY timeout from 2ms to 3ms
marvell: uart: a3720: Update delay code to be compatible with 1200 MHz CPU
marvell: uart: a3720: Fix comments in console_a3700_core_init() function
nxp: added the makefile helper macros
spmd: ensure SIMD context is saved/restored on SPMC entry/exit
nand: stm32_fmc_nand: remove dead code
plat/arm: juno: Refactor juno_getentropy()
bl32: Enable TRNG service build
Here it is if we applied Conventional Commits to it:
feat(marvell armada): cleanup MSS SRAM if used for copy
feat(marvell cn913x): allow CP1/CP2 mapping at BLE stage
feat(marvell armada): use MSS SRAM in secure mode
fix(libc): fix MISRA issues
refactor(xilinx zynqmp): remove the custom crash implementation
refactor(aarch32): add sanity check pointers before use
revert: fix(spmd): ensure SIMD context is saved/restored on SPMC entry/exit
refactor(arm css): rename rd_n1e1_edge_scmi_plat_info array
docs(stm32mp1): correct formatting issues
refactor(marvell a3720): increase TX FIFO EMPTY timeout from 2ms to 3ms
refactor(marvell a3720): update delay code to be compatible with 1200 MHz CPU
fix(marvell a3720): fix comments in console_a3700_core_init() function
build(nxp): add Makefile helper macros
fix(spmd): ensure SIMD context is saved/restored on SPMC entry/exit
refactor(stm32 fmc_nand): remove dead code
refactor(arm juno): Refactor juno_getentropy()
feat(bl32): Enable TRNG service build
Side note: the “screen real estate” concern some raised does not actually seem to manifest itself in any meaningful way – the longest line only increases by 3 characters, and the shortest line is actually reduced by 3 characters.
To me, immediately, the single subject style is much less mentally taxing to parse. Without it, there are at least four different schemes at play here that we need to interpret for every commit (and we’re only 17 commits deep!):
foo/bar/baz: xyz
foo: bar: baz: xyz
foo/bar: baz: xyz
Revert “xyz”
So, if we just forget about trying to read the history manually for a moment, without a standardised subject format we end up with a changelog that looks like this:
Features:
- plat/marvell/armada: cleanup MSS SRAM if used for copy
- plat/marvell: cn913x: allow CP1/CP2 mapping at BLE stage
- plat/marvell/armada/common/mss: use MSS SRAM in secure mode
- bl32: Enable TRNG service build
Bug Fixes:
- libc: memset: Fix MISRA issues
- marvell: uart: a3720: Fix comments in console_a3700_core_init() function
- spmd: ensure SIMD context is saved/restored on SPMC entry/exit
Build System:
- nxp: added the makefile helper macros
Code Refactoring:
- plat:xilinx:zynqmp: Remove the custom crash implementation
- lib: cpus: aarch32: sanity check pointers before use
- plat/arm/css: rename rd_n1e1_edge_scmi_plat_info array
- marvell: uart: a3720: Increase TX FIFO EMPTY timeout from 2ms to 3ms
- marvell: uart: a3720: Update delay code to be compatible with 1200 MHz CPU
- nand: stm32_fmc_nand: remove dead code
- plat/arm: juno: Refactor juno_getentropy()
Documentation:
- docs: stm32mp1: correct formatting issues
Reverts:
- Revert "spmd: ensure SIMD context is saved/restored on SPMC entry/exit"
This, in my opinion, still suffers from the same problem: as a human, it’s difficult to interpret.
Compare that to how we expect that to look with Conventional Commits:
Features:
- bl32: enable TRNG service build
- marvell armada: cleanup MSS SRAM if used for copy
- marvell armada: use MSS SRAM in secure mode
- marvell cn913x: allow CP1/CP2 mapping at BLE stage
Bug Fixes:
- libc: fix MISRA issues
- marvell a3720: fix comments in console_a3700_core_init() function
Build System:
- nxp: add Makefile helper macros
Code Refactoring:
- aarch32: add sanity check pointers before use
- arm css: rename rd_n1e1_edge_scmi_plat_info array
- arm juno: refactor juno_getentropy()
- marvell a3720: increase TX FIFO EMPTY timeout from 2ms to 3ms
- marvell a3720: update delay code to be compatible with 1200 MHz CPU
- stm32 fmc_nand: remove dead code
- xilinx zynqmp: remove the custom crash implementation
Documentation:
- stm32mp1: correct formatting issues
… and I feel like the value prop of using robust tooling becomes more obvious – this tooling is intended not just to categorise commits, but to understand them. You can see here that it emphasises the scope for each change for human readability, and also omits both the revert commit and the commit it reverts because neither of them have been part of a release.
Additionally, without a way to enforce this, we’re not necessarily solving one of the current fundamental problems: our changelogs do not accurately and reliably reflect the changes to the project. I think burdening reviewers with additional work is likely to prove unreliable, and certainly counter-productive if we can both largely automate the problem away and provide rapid feedback to developers before ever even having to push for review.
Just a quick note on one of your points:
3. Flexibility to define project specific tags
The tooling I proposed does already offer some flexibility for defining our own types and scopes, though the default set is already pretty extensive:
* build (Build System)
* ci (Continuous Integration)
* docs (Documentation)
* feat (Features)
* fix (Bug Fixes)
* perf (Performance Improvements)
* refactor (Code Refactoring)
* revert (Reverts)
* style (Styles)
* test (Tests)
Chris
From: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>
Date: Tuesday, 2 March 2021 at 22:31
To: Gyorgy Szing <Gyorgy.Szing(a)arm.com<mailto:Gyorgy.Szing@arm.com>>, Chris Kay <Chris.Kay(a)arm.com<mailto:Chris.Kay@arm.com>>, tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org> <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Cc: nd <nd(a)arm.com<mailto:nd@arm.com>>, nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: RE: [TF-A] Adoption of Conventional Commits
Hi George,
Few clarifications.
>> it would need significant investments on tooling
[VW] I am not sure why you say that. The only expectation form tooling perspective is to run the ‘git log’ command before the release.
>> There is no tool which could help developers crafting the commit message in the right format
[VW] I don’t think we need a tool to generate commit messages with the right tags. I expect developers to add the tag manually as the footer. Maintainers will have to check that tags exist as part of the reviews.
>> Possibly the easiest would be to modify the javascript machinery available for Conventional Commits
[VW] I don’t think we need any tools from the “Conventional Commits” toolbox for this to work.
My proposal was from the requirements I heard from Chris in the meeting. If there are any implicit or obvious requirements that I missed, I propose we freeze them first. A solution can only work if the requirements are frozen.
-Varun
From: Gyorgy Szing <Gyorgy.Szing(a)arm.com<mailto:Gyorgy.Szing@arm.com>>
Sent: Sunday, February 28, 2021 11:14 PM
To: Varun Wadekar <vwadekar(a)nvidia.com<mailto:vwadekar@nvidia.com>>; Chris Kay <Chris.Kay(a)arm.com<mailto:Chris.Kay@arm.com>>; tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>
Cc: nd <nd(a)arm.com<mailto:nd@arm.com>>; nd <nd(a)arm.com<mailto:nd@arm.com>>
Subject: RE: [TF-A] Adoption of Conventional Commits
External email: Use caution opening links or attachments
Hi Varun,
I really like your proposal, but it would need significant investments on tooling. There is no tool which could help developers crafting the commit message in the right format, there is no tool, which can validate the format (and be used i.e. as a git-hook), and there is no tool, which can generate the change history document from git history.
Can you please extend the proposal and turn it to be an end-to-end solution? Can you contribute tooling for commit message editing and validation, and for change log document generation? Possibly the easiest would be to modify the javascript machinery available for Conventional Commits. Can you contribute the needed changes?
/George
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org<mailto:tf-a-bounces@lists.trustedfirmware.org>> On Behalf Of Varun Wadekar via TF-A
Sent: 26 February 2021 00:39
To: Chris Kay <Chris.Kay(a)arm.com<mailto:Chris.Kay@arm.com>>
Cc: tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>
Subject: Re: [TF-A] Adoption of Conventional Commits
Hi,
I really like the idea of using tags in the commit message, but the rigidity of the spec puts me off. Frankly, I believe we just need a way to identify commits and their intent. So, I would like to propose an approach that builds on the “Conventional Commits” spec.
The approach would be
1. Add an identifier (e.g. Tags: fix) to the commit message footer.
2. At the start of the release window run “git log”* to print a list of features, bug fixes, performance improvements, deprecations etc.
3. Either update the main changelog manually or use a script to append individual sections.
*git log v2.4...HEAD --no-merges --pretty='- %s (%C(auto)%h)' --grep "Tags: fix">
‘git log’ can be easily modified to look for other metadata as long as we agree to add it to the commit message.
Advantages
1. Light(er)
2. No impact to the subject header
3. Flexibility to define project specific tags
4. Training needs at par with “Conventional Commits” proposal
Thoughts?
-Varun
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org<mailto:tf-a-bounces@lists.trustedfirmware.org>> On Behalf Of Chris Kay via TF-A
Sent: Thursday, February 25, 2021 9:31 AM
To: tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>
Subject: Re: [TF-A] Adoption of Conventional Commits
External email: Use caution opening links or attachments
Thanks to all those who attended the Tech Forum today!
It’s become apparent that the initial 2 week deadline for alternative proposals or implementations is too short, so – as agreed – we’ll push the deadline for the investigation period to the end of March. This period is dedicated to evaluating the changelog automation proposal made, or to identifying alternative solutions. If you have an alternative proposal, any proof-of-concept tooling would be highly appreciated so we can get a clear idea of what sort of work and maintenance is going to be involved.
If you do find a solution you wish to propose, please give it just a short name (e.g. “Update it manually”) and make it obvious you want to propose it formally – I’ll collect up the proposals made on the mailing list thread at the end of March and set up a Wiki poll so we can get a clear picture of where the community wants to take this.
Chris
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org<mailto:tf-a-bounces@lists.trustedfirmware.org>> on behalf of Chris Kay via TF-A <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Reply to: Chris Kay <Chris.Kay(a)arm.com<mailto:Chris.Kay@arm.com>>
Date: Thursday, 11 February 2021 at 13:59
To: "tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>" <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Subject: [TF-A] Adoption of Conventional Commits
Hi all,
Recently we had an internal discussion on the merits of introducing semantics to commit messages pushed to the main TF-A repository, the conclusion being that we would look to adopting the Conventional Commits<https://www.conventionalcommits.org/en/v1.0.0/> specification in the near future. There was one major reason for this, which was to help us in automating the changelog in future releases, but it might also help us to dramatically reduce the overall amount of work needed to make a formal release in the future.
This requires some buy-in (or buy-out, in this case) from maintainers because - even though it’s to only a relatively minor extent - it does involve an adjustment to everybody’s workflow. Notably, commit messages will be expected to adopt the structure defined by the specification, which will be enforced by the CI. Most commits that go upstream today adhere to “something that looks like Conventional Commits”, so the change is not exactly sweeping, but any change has the potential be an inconvenience.
With that in mind, I propose the following:
* We collectively adopt the specification, enforced only for @arm.com contributors until such a time that the majority of maintainers are familiar with the new demands
* We suggest - in the prerequisites documentation - the installation of two helper tools:
* Commitizen<https://github.com/commitizen/cz-cli>
* Commitlint<https://github.com/conventional-changelog/commitlint>
Installation of these tools will be optional, but I believe they can help with the transition. In the patches currently in review, they are installed as Git hooks automatically upon execution of npm install, so it requires no manual installation or configuration (other than a relatively up-to-date Node.js installation).
You’ll find the patches here<https://review.trustedfirmware.org/q/topic:%22ck%252Fconventional-commits%2…>, and specifically the changes to the prerequisites documentation here<https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/8224/1/docs/…>. Feel free to review these changes if you have comments specifically on their implementation.
Let me know if you have any questions or concerns. If everybody’s on board, we can look to have this upstreamed shortly.
Chris
Perhaps we ought to look at doing away with maintaining our own version of freestanding headers like stdint.h in the first place – they’re part of the freestanding portion of the C standard library for good reason (the implementations necessarily come directly from the compiler), and reimplementing them is really prone to portability errors like this (and can frequently confuse static analysers). If we are to continue using it, we should at least look into replacing the definitions with the builtin values provided by the compilers we use, e.g. typedef __UINT64_TYPE__ uint64_t;.
Using inttypes.h is the traditional wisdom for this particular specifier issue – `ll` for `long long`, `PRIu64` for `uint64_t. While it’s not particularly pleasant to read/write, it was the solution that the C standards committee came up with, so I approve of the principle of this change but I think a permanent solution would serve us better in the long run.
Chris
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of Joanna Farley via TF-A <tf-a(a)lists.trustedfirmware.org>
Date: Wednesday, 3 March 2021 at 15:43
To: Joanna Farley via TF-A <tf-a(a)lists.trustedfirmware.org>
Cc: Scott Branden <scott.branden(a)broadcom.com>
Subject: [TF-A] ATF currently does not use proper printf format specifiers for fixed width types
Hi All,
Back in September Scott posted a query to the group related to a patch he has created relating to printf format specifiers and some of the maintainers from Arm have reservations about and we asked him to get opinions from the broader project community as his patch changes a number of different platforms as well as core code.
I’m trying to help him reinvigorate the discussion so reposting his request with patch link below that had stalled.
Joanna
________________________________
Scott Branden scott.branden at broadcom.com <mailto:tf-a%40lists.trustedfirmware.org?Subject=Re%3A%20%5BTF-A%5D%20ATF%20currently%20does%20not%20use%20proper%20printf%20format%20specifiers%0A%20for%20fixed%20width%20types&In-Reply-To=%3Cbd3b49f4-e8f9-9016-d11b-d08b81e6b43d%40broadcom.com%3E>
Mon Sep 14 18:34:45 UTC 2020
Hello,
ATF currently uses non-portable printf format specifiers for fixed width types defined in stdint.h
In addition, ATF redefines types defined in gcc for stdint.h with its own custom types causing additional issues.
This causes compilation issues when porting code to/from ATF.
AND, generates coverity parse errors as int64_t and uint64_t are incorrectly defined in ATF vs. gcc for aarch64.
The printf format specifiers in inttypes.h are to be used for the proper format specifiers.
And, uint64_t/int64_t should be defined the same as in gcc.
I tried fixing up all the instances of int64 printf format specifiers by introducing inttypes.h and redefined the stdint types correctly here:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/5437
We have checked the change into our local tree so that everything compiles and runs in our system. Please accept change upstream.
Regards,
Scott
Hi All,
Back in September Scott posted a query to the group related to a patch he has created relating to printf format specifiers and some of the maintainers from Arm have reservations about and we asked him to get opinions from the broader project community as his patch changes a number of different platforms as well as core code.
I’m trying to help him reinvigorate the discussion so reposting his request with patch link below that had stalled.
Joanna
________________________________
Scott Branden scott.branden at broadcom.com <mailto:tf-a%40lists.trustedfirmware.org?Subject=Re%3A%20%5BTF-A%5D%20ATF%20currently%20does%20not%20use%20proper%20printf%20format%20specifiers%0A%20for%20fixed%20width%20types&In-Reply-To=%3Cbd3b49f4-e8f9-9016-d11b-d08b81e6b43d%40broadcom.com%3E>
Mon Sep 14 18:34:45 UTC 2020
Hello,
ATF currently uses non-portable printf format specifiers for fixed width types defined in stdint.h
In addition, ATF redefines types defined in gcc for stdint.h with its own custom types causing additional issues.
This causes compilation issues when porting code to/from ATF.
AND, generates coverity parse errors as int64_t and uint64_t are incorrectly defined in ATF vs. gcc for aarch64.
The printf format specifiers in inttypes.h are to be used for the proper format specifiers.
And, uint64_t/int64_t should be defined the same as in gcc.
I tried fixing up all the instances of int64 printf format specifiers by introducing inttypes.h and redefined the stdint types correctly here:
https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/5437
We have checked the change into our local tree so that everything compiles and runs in our system. Please accept change upstream.
Regards,
Scott
Hi Varun,
I really like your proposal, but it would need significant investments on tooling. There is no tool which could help developers crafting the commit message in the right format, there is no tool, which can validate the format (and be used i.e. as a git-hook), and there is no tool, which can generate the change history document from git history.
Can you please extend the proposal and turn it to be an end-to-end solution? Can you contribute tooling for commit message editing and validation, and for change log document generation? Possibly the easiest would be to modify the javascript machinery available for Conventional Commits. Can you contribute the needed changes?
/George
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Varun Wadekar via TF-A
Sent: 26 February 2021 00:39
To: Chris Kay <Chris.Kay(a)arm.com>
Cc: tf-a(a)lists.trustedfirmware.org
Subject: Re: [TF-A] Adoption of Conventional Commits
Hi,
I really like the idea of using tags in the commit message, but the rigidity of the spec puts me off. Frankly, I believe we just need a way to identify commits and their intent. So, I would like to propose an approach that builds on the “Conventional Commits” spec.
The approach would be
1. Add an identifier (e.g. Tags: fix) to the commit message footer.
2. At the start of the release window run “git log”* to print a list of features, bug fixes, performance improvements, deprecations etc.
3. Either update the main changelog manually or use a script to append individual sections.
*git log v2.4...HEAD --no-merges --pretty='- %s (%C(auto)%h)' --grep "Tags: fix">
‘git log’ can be easily modified to look for other metadata as long as we agree to add it to the commit message.
Advantages
1. Light(er)
2. No impact to the subject header
3. Flexibility to define project specific tags
4. Training needs at par with “Conventional Commits” proposal
Thoughts?
-Varun
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org<mailto:tf-a-bounces@lists.trustedfirmware.org>> On Behalf Of Chris Kay via TF-A
Sent: Thursday, February 25, 2021 9:31 AM
To: tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>
Subject: Re: [TF-A] Adoption of Conventional Commits
External email: Use caution opening links or attachments
Thanks to all those who attended the Tech Forum today!
It’s become apparent that the initial 2 week deadline for alternative proposals or implementations is too short, so – as agreed – we’ll push the deadline for the investigation period to the end of March. This period is dedicated to evaluating the changelog automation proposal made, or to identifying alternative solutions. If you have an alternative proposal, any proof-of-concept tooling would be highly appreciated so we can get a clear idea of what sort of work and maintenance is going to be involved.
If you do find a solution you wish to propose, please give it just a short name (e.g. “Update it manually”) and make it obvious you want to propose it formally – I’ll collect up the proposals made on the mailing list thread at the end of March and set up a Wiki poll so we can get a clear picture of where the community wants to take this.
Chris
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org<mailto:tf-a-bounces@lists.trustedfirmware.org>> on behalf of Chris Kay via TF-A <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Reply to: Chris Kay <Chris.Kay(a)arm.com<mailto:Chris.Kay@arm.com>>
Date: Thursday, 11 February 2021 at 13:59
To: "tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>" <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Subject: [TF-A] Adoption of Conventional Commits
Hi all,
Recently we had an internal discussion on the merits of introducing semantics to commit messages pushed to the main TF-A repository, the conclusion being that we would look to adopting the Conventional Commits<https://www.conventionalcommits.org/en/v1.0.0/> specification in the near future. There was one major reason for this, which was to help us in automating the changelog in future releases, but it might also help us to dramatically reduce the overall amount of work needed to make a formal release in the future.
This requires some buy-in (or buy-out, in this case) from maintainers because - even though it’s to only a relatively minor extent - it does involve an adjustment to everybody’s workflow. Notably, commit messages will be expected to adopt the structure defined by the specification, which will be enforced by the CI. Most commits that go upstream today adhere to “something that looks like Conventional Commits”, so the change is not exactly sweeping, but any change has the potential be an inconvenience.
With that in mind, I propose the following:
* We collectively adopt the specification, enforced only for @arm.com contributors until such a time that the majority of maintainers are familiar with the new demands
* We suggest - in the prerequisites documentation - the installation of two helper tools:
* Commitizen<https://github.com/commitizen/cz-cli>
* Commitlint<https://github.com/conventional-changelog/commitlint>
Installation of these tools will be optional, but I believe they can help with the transition. In the patches currently in review, they are installed as Git hooks automatically upon execution of npm install, so it requires no manual installation or configuration (other than a relatively up-to-date Node.js installation).
You’ll find the patches here<https://review.trustedfirmware.org/q/topic:%22ck%252Fconventional-commits%2…>, and specifically the changes to the prerequisites documentation here<https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/8224/1/docs/…>. Feel free to review these changes if you have comments specifically on their implementation.
Let me know if you have any questions or concerns. If everybody’s on board, we can look to have this upstreamed shortly.
Chris
Hi,
I really like the idea of using tags in the commit message, but the rigidity of the spec puts me off. Frankly, I believe we just need a way to identify commits and their intent. So, I would like to propose an approach that builds on the “Conventional Commits” spec.
The approach would be
1. Add an identifier (e.g. Tags: fix) to the commit message footer.
2. At the start of the release window run “git log”* to print a list of features, bug fixes, performance improvements, deprecations etc.
3. Either update the main changelog manually or use a script to append individual sections.
*git log v2.4...HEAD --no-merges --pretty='- %s (%C(auto)%h)' --grep "Tags: fix">
‘git log’ can be easily modified to look for other metadata as long as we agree to add it to the commit message.
Advantages
1. Light(er)
2. No impact to the subject header
3. Flexibility to define project specific tags
4. Training needs at par with “Conventional Commits” proposal
Thoughts?
-Varun
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Chris Kay via TF-A
Sent: Thursday, February 25, 2021 9:31 AM
To: tf-a(a)lists.trustedfirmware.org
Subject: Re: [TF-A] Adoption of Conventional Commits
External email: Use caution opening links or attachments
Thanks to all those who attended the Tech Forum today!
It’s become apparent that the initial 2 week deadline for alternative proposals or implementations is too short, so – as agreed – we’ll push the deadline for the investigation period to the end of March. This period is dedicated to evaluating the changelog automation proposal made, or to identifying alternative solutions. If you have an alternative proposal, any proof-of-concept tooling would be highly appreciated so we can get a clear idea of what sort of work and maintenance is going to be involved.
If you do find a solution you wish to propose, please give it just a short name (e.g. “Update it manually”) and make it obvious you want to propose it formally – I’ll collect up the proposals made on the mailing list thread at the end of March and set up a Wiki poll so we can get a clear picture of where the community wants to take this.
Chris
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org<mailto:tf-a-bounces@lists.trustedfirmware.org>> on behalf of Chris Kay via TF-A <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Reply to: Chris Kay <Chris.Kay(a)arm.com<mailto:Chris.Kay@arm.com>>
Date: Thursday, 11 February 2021 at 13:59
To: "tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>" <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Subject: [TF-A] Adoption of Conventional Commits
Hi all,
Recently we had an internal discussion on the merits of introducing semantics to commit messages pushed to the main TF-A repository, the conclusion being that we would look to adopting the Conventional Commits<https://www.conventionalcommits.org/en/v1.0.0/> specification in the near future. There was one major reason for this, which was to help us in automating the changelog in future releases, but it might also help us to dramatically reduce the overall amount of work needed to make a formal release in the future.
This requires some buy-in (or buy-out, in this case) from maintainers because - even though it’s to only a relatively minor extent - it does involve an adjustment to everybody’s workflow. Notably, commit messages will be expected to adopt the structure defined by the specification, which will be enforced by the CI. Most commits that go upstream today adhere to “something that looks like Conventional Commits”, so the change is not exactly sweeping, but any change has the potential be an inconvenience.
With that in mind, I propose the following:
* We collectively adopt the specification, enforced only for @arm.com contributors until such a time that the majority of maintainers are familiar with the new demands
* We suggest - in the prerequisites documentation - the installation of two helper tools:
* Commitizen<https://github.com/commitizen/cz-cli>
* Commitlint<https://github.com/conventional-changelog/commitlint>
Installation of these tools will be optional, but I believe they can help with the transition. In the patches currently in review, they are installed as Git hooks automatically upon execution of npm install, so it requires no manual installation or configuration (other than a relatively up-to-date Node.js installation).
You’ll find the patches here<https://review.trustedfirmware.org/q/topic:%22ck%252Fconventional-commits%2…>, and specifically the changes to the prerequisites documentation here<https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/8224/1/docs/…>. Feel free to review these changes if you have comments specifically on their implementation.
Let me know if you have any questions or concerns. If everybody’s on board, we can look to have this upstreamed shortly.
Chris
Hi All,
After the TF-A Tech-forum on 11th February 2021 (https://www.trustedfirmware.org/meetings/tf-a-technical-forum/) introducing the TF-A OpenCI I wanted to follow up with a posting on how we want to incorporate this new CI workflow into the TF-A project environment and who will have the ability to instigate OpenCI runs for the project.
Some guidelines we are looking to follow are:
* Be consistent across all Trustedfirmware.org hosted projects (TF-A, TF-M, Hafnium etc)
* Be as open as possible to allow project contributors access to the CI.
* Protect backend server resources from security attacks.
* Be open to tune the workflow process as needs demand.
As mentioned in the Tech-forum session the main day to day developer interface with the OpenCI is through Gerrit reviews with the CI+1/CI+2 gerrit labels for patches under review that start two levels of CI coverage. There is also a daily CI run on the integration branch that is started automatically. OpenCI runs for gerrit patch reviews are shown in the patch comments as links into https://ci.trustedfirmware.org/view/TF-A/ where all the Jenkin jobs including the daily build can be found.
In addition there is the occasional need to re-trigger CI jobs directly in the Jenkins UI shown above. This can occur if there is an intermittent failure in the CI infrastructure or due to some other cause however its hoped this is only needed to be used rarely and most contributors will not need to use this.
The initial plan is allow OpenCI invocation (through Gerrit and Jenkins) to all the TF-A maintainers and Code Owners listed in https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/tree/docs/about… as a list of known project contributors. This list can be extended over time with other individuals nominated by anybody on the initial list.
The OpenCI will become the primary and only CI for setting the Gerrit verified label for patches under review. The existing ArmCI will become advisory and triggered separately and will not directly influence the Gerrit verified label for patches under review. The ArmCI will be maintained in parallel for those areas not yet migrated to the OpenCI and will be disabled once the next phases of the OpenCI development complete. Any CI results from the ArmCI will be communicated in patch review comments from Arm contributors as required.
Project documentation will be updated to incorporate OpenCI usage and a further Tech-forum will be held to go through the CI usage in more detail once the OpenCI is ready to take over as the primary TF-A CI. This is expected and hoping to take place in the next week or two as the final OpenCI deployment issues are resolved but exactly when will be communicated to this list once we have a firm date.
Thanks
Joanna
Hi Chris,
This seems like a good proposal to alleviate some of the pain around releases.
Some observations/questions.
1. Introducing additional tags will eat into precious real estate. This will be a problem for some changes and developers.
2. In the transition period, the proposal has the potential to "pollute" the history
3. The proposal will add more work for patches cherry-picked from other projects e.g. libfdt
4. How do we handle scenarios where platforms donot want to switch to this policy?
I can see how #1 might be of concern to many, but we will have to implement some policy to see how many commits really fall in this category.
-Varun
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Chris Kay via TF-A
Sent: Thursday, February 11, 2021 5:59 AM
To: tf-a(a)lists.trustedfirmware.org
Subject: [TF-A] Adoption of Conventional Commits
External email: Use caution opening links or attachments
Hi all,
Recently we had an internal discussion on the merits of introducing semantics to commit messages pushed to the main TF-A repository, the conclusion being that we would look to adopting the Conventional Commits<https://www.conventionalcommits.org/en/v1.0.0/> specification in the near future. There was one major reason for this, which was to help us in automating the changelog in future releases, but it might also help us to dramatically reduce the overall amount of work needed to make a formal release in the future.
This requires some buy-in (or buy-out, in this case) from maintainers because - even though it's to only a relatively minor extent - it does involve an adjustment to everybody's workflow. Notably, commit messages will be expected to adopt the structure defined by the specification, which will be enforced by the CI. Most commits that go upstream today adhere to "something that looks like Conventional Commits", so the change is not exactly sweeping, but any change has the potential be an inconvenience.
With that in mind, I propose the following:
* We collectively adopt the specification, enforced only for @arm.com contributors until such a time that the majority of maintainers are familiar with the new demands
* We suggest - in the prerequisites documentation - the installation of two helper tools:
* Commitizen<https://github.com/commitizen/cz-cli>
* Commitlint<https://github.com/conventional-changelog/commitlint>
Installation of these tools will be optional, but I believe they can help with the transition. In the patches currently in review, they are installed as Git hooks automatically upon execution of npm install, so it requires no manual installation or configuration (other than a relatively up-to-date Node.js installation).
You'll find the patches here<https://review.trustedfirmware.org/q/topic:%22ck%252Fconventional-commits%2…>, and specifically the changes to the prerequisites documentation here<https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/8224/1/docs/…>. Feel free to review these changes if you have comments specifically on their implementation.
Let me know if you have any questions or concerns. If everybody's on board, we can look to have this upstreamed shortly.
Chris
Hi Daniele,
TBBR specification, from where fip format was introduced, is not very clear about usage of serial number and it can be used in IMP DEFINED manner
"ToC Serial Number - 32 - The serial number of this ToC".
So, theoretically it's possible to use serial number for the purpose you described and it's a valid use of currently (un)used serial number.
Currently, at boot time serial number is checked against a non-zero value, which certainly will hold true if you put a valid timstamp instead of "0x12345678".
IMO you can go ahead and implement a mechanism to feed Build timestamp to be used as serial number.
Thanks
Manish P
________________________________
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of Manish Badarkhe via TF-A <tf-a(a)lists.trustedfirmware.org>
Sent: 11 February 2021 11:39
To: Daniele Alessandrelli <daniele.alessandrelli(a)linux.intel.com>; tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>
Subject: Re: [TF-A] Getting BUILD_STRING from FIP file
Hi Daniele
Please see my replies inline.
Thanks
Manish Badarkhe
From: Daniele Alessandrelli <daniele.alessandrelli(a)linux.intel.com>
Date: Thursday, 11 February 2021 at 11:22
To: Manish Badarkhe <Manish.Badarkhe(a)arm.com>, tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>
Subject: Re: [TF-A] Getting BUILD_STRING from FIP file
Hi Manish,
Thank you very much for the information.
16 bits are not enough to store an epoch, but I'll see if I can encode
some other unique build information into 16 bits.
Just a question, from your answer I assume that I shouldn't use the
'serial_number' field for what I want to do; so I wonder: what is that
field meant to be used for?
'serial_number' field is non-zero number provided by the fip creation tool.
It is hardcoded value i.e. TOC_HEADER_SERIAL_NUMBER=0x12345678 as of now.
Specifically used during the validation of the TOC header in the code.
Regards,
Daniele
On Thu, 2021-02-11 at 10:14 +0000, Manish Badarkhe wrote:
> Hi Daniele
>
> You can use the ‘flag’ field to mention the platform-specific data(in
> your case, a build number). Usage of the ‘flag’ field(64 bit) in the
> toc_header are as below:
> Bits 0-31 -> reserved
> Bits 32-47 -> platform defined data
> Bits 48-63 -> reserved
> You can make use of the flag[32:47] to put build information. I am
> not sure if you can accommodate epoch (converted timestamp) into this
> field but, you can encode any data to fit into this 16bit flag field
> to identify the FIP build.
>
> You can use a build command: fiptool update/create --plat-toc-flags
> <platform defined data> <your fip bin path> to put the platform
> defined data in the FIP image.
>
> Thanks
> Manish Badarkhe
> From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> on behalf of
> Daniele Alessandrelli via TF-A <tf-a(a)lists.trustedfirmware.org>
> Date: Wednesday, 10 February 2021 at 17:04
> To: tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>
> Subject: [TF-A] Getting BUILD_STRING from FIP file
>
> Hi,
>
> Is there a way to get BUILD_STRING (or a similar string / number that
> uniquely identifies the TF-A build, e.g., BUILD_MESSAGE_TIMESTAMP)
> from
> the FIP file?
>
> Basically, I'm trying to find a way to know the build number of a FIP
> without flashing it.
>
> I've seen that the FIP TOC header has a 32-bit field named
> 'serial_number'. Can it be used to this end? I'm considering
> converting BUILD_MESSAGE_TIMESTAMP into an epoch and adding it as
> 'serial number', but I'm worried that might be an unintended usage of
> the 'serial_number' field.
>
> Regards,
> Daniele
>
>
>