Re: [TF-A] ATF LTS version
by Joanna Farley
I should also mention the Release processes we follow and the attempt to indicated the deprecation of interfaces in advance in the effort to maintain compatibility https://trustedfirmware-a.readthedocs.io/en/latest/about/release-informatio…
On 11/06/2020, 14:47, "TF-A on behalf of Joanna Farley via TF-A" <tf-a-bounces(a)lists.trustedfirmware.org on behalf of tf-a(a)lists.trustedfirmware.org> wrote:
Hi Varun,
I guess this suggestion came in response to last weeks Tech Forum discussion from a question about experiences people had from migrating to different TF-A tagged releases. In general we try and keep the tip of Master at tagged release quality through an extensive CI system ran on each patch. I appreciate this CI is currently a little opaque to many contributors as this is still in the process of being transitioned to the OpenCI hosted by Trustedfirmware.org servers which will be visible to all. As mentioned in the recent "Overview of the TF-A v2.3 Release" presentation on https://www.trustedfirmware.org/meetings/tf-a-technical-forum/ the additional testing done for a 6 monthly tagged release is quite minimal and the larger work is ensuring all documentation is up to date. Additionally all new features are generally behind their own build flags but I appreciate it is some work for a tagged release to be absorbed into product offerings.
I asked at the tech forum last week what more we could do to allow releases to be integrated more easily. On the call it was requested if we could disable weak bindings to symbols so it could be more easily seen where platform decisions may need to be made and we will look into this. If there are any more adjustments to the way tagged releases are produced please let us know.
One thing that had been considered briefly was the production of a security bug only branch that was maintained only between 6 month tagged releases before being replaced by the next security bug only branch based on the next 6 month release but that has not progressed very far as a proposal as until your email here it was perceived to not be in demand. A LTS branch is a larger endeavour as it sounds like something that includes more than security fixes and I look forward to you elaborating more as Matteo requests.
Thanks
Joanna
On 11/06/2020, 12:19, "Matteo Carlini via TF-A" <tf-a(a)lists.trustedfirmware.org> wrote:
Hi Varun,
Thanks for raising this topic (but please do embrace the official terminology “TF-A”…we never really promoted ATF and it's also absolutely outdated now 😉 ).
Arm has received different queries over time on supporting Trusted Firmware LTS releases, but the effort to sustain them is something that the Arm engineering team alone cannot really afford and commit to (either in the TF-A or TF-M space).
The idea has also been just raised to the Trusted Firmware project Board for initial consideration and we will be all very keen to understand how much interest there is from the wider TF-A community of adopters and external (non-Arm) maintainers, so to evaluate the possibility of a more concrete proposal to be carried on within the community Project.
I guess it will also be good to start by elaborating more concretely on the requirements that you would like to see in an hypothetical LTS versioning scheme.
Thanks
Matteo
> From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Varun Wadekar via TF-A
> Sent: 10 June 2020 22:47
> To: tf-a(a)lists.trustedfirmware.org
> Subject: [TF-A] ATF LTS version
>
> Hello team,
>
> To extend the discussion around version upgrades from our last call, I would like to understand if there is enough interest around generating a LTS version of the TF-A to alleviate the pain.
>
> For NVIDIA, this would be helpful as it streamlines the upgrade path for our devices in the field. The LTS version will guarantee security fixes, bug fixes, stability fixes for the longer term and we won’t have to upgrade the entire firmware to get these goodies.
>
> It would be interesting to see what OEMs and maintainers think about this? Has this been discussed at tf.org or Arm internally?
>
> -Varun
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
5 years, 4 months
Re: [TF-A] ATF LTS version
by Varun Wadekar
Hi Joanna,
I can follow up with a tech forum session with the ways it would help a platform owner. Most of it would be from NVIDIA's perspective, but it should apply to other platforms.
-Varun
-----Original Message-----
From: Joanna Farley <Joanna.Farley(a)arm.com>
Sent: Tuesday, June 16, 2020 5:51 AM
To: Varun Wadekar <vwadekar(a)nvidia.com>; Matteo Carlini <Matteo.Carlini(a)arm.com>
Cc: tf-a(a)lists.trustedfirmware.org
Subject: Re: [TF-A] ATF LTS version
External email: Use caution opening links or attachments
Hi Varun,
Its been a few days now and it would be good to have some other opinions from the broader project contributors but nothing posted yet beyond Francois.
As you suggested this do you think you could follow up with a tech forum session advocating the needs and benefits and how this could work? Having a more detailed proposal may help with the discussion. What I feel is that there would need to be a critical mass of contributors/consumers of the project wanting this to establish the need and they would need to also agree a common consumption model.
Thanks
Joanna
On 11/06/2020, 23:08, "Varun Wadekar" <vwadekar(a)nvidia.com> wrote:
Hello Joanna,
I had discussed the idea with Matteo in the past, but the discussion in the last tech forum prompted the email.
>> I appreciate this CI is currently a little opaque to many contributors as this is still in the process of being transitioned to the OpenCI hosted by Trustedfirmware.org servers which will be visible to all
I agree, it is hard to test all the use cases. The opaque nature of the CI is a bit annoying, but not a big issue.
>> the additional testing done for a 6 monthly tagged release is quite minimal and the larger work is ensuring all documentation is up to date. Additionally all new features are generally behind their own build flags but I appreciate it is some work for a tagged release to be absorbed into product offerings.
Interesting In one of our internal discussions we were exploring the possibility of using doxygen style comments and creating an API reference for a release without a lot of effort. We should try to explore this idea in the community.
>> One thing that had been considered briefly was the production of a security bug only branch
That is a good idea and can act as the base for the LTS version. But we should consider increasing the scope and include bug fixes, stability issues, performance issues, etc. I believe when the community widely adopts TFTF and starts upstreaming the test cases, we can expect more interest around a LTS release.
For platform owners (e.g. NVIDIA) it makes sense to plan our release strategy around LTS versions. Right now, our releases lack direction as we don’t know which version to use. And then there is additional pain of rebasing recent fixes/improvements on older releases.
-Varun
-----Original Message-----
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Joanna Farley via TF-A
Sent: Thursday, June 11, 2020 6:47 AM
To: Matteo Carlini <Matteo.Carlini(a)arm.com>; tf-a(a)lists.trustedfirmware.org
Subject: Re: [TF-A] ATF LTS version
External email: Use caution opening links or attachments
Hi Varun,
I guess this suggestion came in response to last weeks Tech Forum discussion from a question about experiences people had from migrating to different TF-A tagged releases. In general we try and keep the tip of Master at tagged release quality through an extensive CI system ran on each patch. I appreciate this CI is currently a little opaque to many contributors as this is still in the process of being transitioned to the OpenCI hosted by Trustedfirmware.org servers which will be visible to all. As mentioned in the recent "Overview of the TF-A v2.3 Release" presentation on https://www.trustedfirmware.org/meetings/tf-a-technical-forum/ the additional testing done for a 6 monthly tagged release is quite minimal and the larger work is ensuring all documentation is up to date. Additionally all new features are generally behind their own build flags but I appreciate it is some work for a tagged release to be absorbed into product offerings.
I asked at the tech forum last week what more we could do to allow releases to be integrated more easily. On the call it was requested if we could disable weak bindings to symbols so it could be more easily seen where platform decisions may need to be made and we will look into this. If there are any more adjustments to the way tagged releases are produced please let us know.
One thing that had been considered briefly was the production of a security bug only branch that was maintained only between 6 month tagged releases before being replaced by the next security bug only branch based on the next 6 month release but that has not progressed very far as a proposal as until your email here it was perceived to not be in demand. A LTS branch is a larger endeavour as it sounds like something that includes more than security fixes and I look forward to you elaborating more as Matteo requests.
Thanks
Joanna
On 11/06/2020, 12:19, "Matteo Carlini via TF-A" <tf-a(a)lists.trustedfirmware.org> wrote:
Hi Varun,
Thanks for raising this topic (but please do embrace the official terminology “TF-A”…we never really promoted ATF and it's also absolutely outdated now 😉 ).
Arm has received different queries over time on supporting Trusted Firmware LTS releases, but the effort to sustain them is something that the Arm engineering team alone cannot really afford and commit to (either in the TF-A or TF-M space).
The idea has also been just raised to the Trusted Firmware project Board for initial consideration and we will be all very keen to understand how much interest there is from the wider TF-A community of adopters and external (non-Arm) maintainers, so to evaluate the possibility of a more concrete proposal to be carried on within the community Project.
I guess it will also be good to start by elaborating more concretely on the requirements that you would like to see in an hypothetical LTS versioning scheme.
Thanks
Matteo
> From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Varun Wadekar via TF-A
> Sent: 10 June 2020 22:47
> To: tf-a(a)lists.trustedfirmware.org
> Subject: [TF-A] ATF LTS version
>
> Hello team,
>
> To extend the discussion around version upgrades from our last call, I would like to understand if there is enough interest around generating a LTS version of the TF-A to alleviate the pain.
>
> For NVIDIA, this would be helpful as it streamlines the upgrade path for our devices in the field. The LTS version will guarantee security fixes, bug fixes, stability fixes for the longer term and we won’t have to upgrade the entire firmware to get these goodies.
>
> It would be interesting to see what OEMs and maintainers think about this? Has this been discussed at tf.org or Arm internally?
>
> -Varun
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
5 years, 4 months
Re: [TF-A] ATF LTS version
by Joanna Farley
Thanks Varun, anything to help stimulate discussion is welcome so let me know when you have something and we can schedule it.
Cheers
Joanna
On 18/06/2020, 02:08, "Varun Wadekar" <vwadekar(a)nvidia.com> wrote:
Hi Joanna,
I can follow up with a tech forum session with the ways it would help a platform owner. Most of it would be from NVIDIA's perspective, but it should apply to other platforms.
-Varun
-----Original Message-----
From: Joanna Farley <Joanna.Farley(a)arm.com>
Sent: Tuesday, June 16, 2020 5:51 AM
To: Varun Wadekar <vwadekar(a)nvidia.com>; Matteo Carlini <Matteo.Carlini(a)arm.com>
Cc: tf-a(a)lists.trustedfirmware.org
Subject: Re: [TF-A] ATF LTS version
External email: Use caution opening links or attachments
Hi Varun,
Its been a few days now and it would be good to have some other opinions from the broader project contributors but nothing posted yet beyond Francois.
As you suggested this do you think you could follow up with a tech forum session advocating the needs and benefits and how this could work? Having a more detailed proposal may help with the discussion. What I feel is that there would need to be a critical mass of contributors/consumers of the project wanting this to establish the need and they would need to also agree a common consumption model.
Thanks
Joanna
On 11/06/2020, 23:08, "Varun Wadekar" <vwadekar(a)nvidia.com> wrote:
Hello Joanna,
I had discussed the idea with Matteo in the past, but the discussion in the last tech forum prompted the email.
>> I appreciate this CI is currently a little opaque to many contributors as this is still in the process of being transitioned to the OpenCI hosted by Trustedfirmware.org servers which will be visible to all
I agree, it is hard to test all the use cases. The opaque nature of the CI is a bit annoying, but not a big issue.
>> the additional testing done for a 6 monthly tagged release is quite minimal and the larger work is ensuring all documentation is up to date. Additionally all new features are generally behind their own build flags but I appreciate it is some work for a tagged release to be absorbed into product offerings.
Interesting In one of our internal discussions we were exploring the possibility of using doxygen style comments and creating an API reference for a release without a lot of effort. We should try to explore this idea in the community.
>> One thing that had been considered briefly was the production of a security bug only branch
That is a good idea and can act as the base for the LTS version. But we should consider increasing the scope and include bug fixes, stability issues, performance issues, etc. I believe when the community widely adopts TFTF and starts upstreaming the test cases, we can expect more interest around a LTS release.
For platform owners (e.g. NVIDIA) it makes sense to plan our release strategy around LTS versions. Right now, our releases lack direction as we don’t know which version to use. And then there is additional pain of rebasing recent fixes/improvements on older releases.
-Varun
-----Original Message-----
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Joanna Farley via TF-A
Sent: Thursday, June 11, 2020 6:47 AM
To: Matteo Carlini <Matteo.Carlini(a)arm.com>; tf-a(a)lists.trustedfirmware.org
Subject: Re: [TF-A] ATF LTS version
External email: Use caution opening links or attachments
Hi Varun,
I guess this suggestion came in response to last weeks Tech Forum discussion from a question about experiences people had from migrating to different TF-A tagged releases. In general we try and keep the tip of Master at tagged release quality through an extensive CI system ran on each patch. I appreciate this CI is currently a little opaque to many contributors as this is still in the process of being transitioned to the OpenCI hosted by Trustedfirmware.org servers which will be visible to all. As mentioned in the recent "Overview of the TF-A v2.3 Release" presentation on https://www.trustedfirmware.org/meetings/tf-a-technical-forum/ the additional testing done for a 6 monthly tagged release is quite minimal and the larger work is ensuring all documentation is up to date. Additionally all new features are generally behind their own build flags but I appreciate it is some work for a tagged release to be absorbed into product offerings.
I asked at the tech forum last week what more we could do to allow releases to be integrated more easily. On the call it was requested if we could disable weak bindings to symbols so it could be more easily seen where platform decisions may need to be made and we will look into this. If there are any more adjustments to the way tagged releases are produced please let us know.
One thing that had been considered briefly was the production of a security bug only branch that was maintained only between 6 month tagged releases before being replaced by the next security bug only branch based on the next 6 month release but that has not progressed very far as a proposal as until your email here it was perceived to not be in demand. A LTS branch is a larger endeavour as it sounds like something that includes more than security fixes and I look forward to you elaborating more as Matteo requests.
Thanks
Joanna
On 11/06/2020, 12:19, "Matteo Carlini via TF-A" <tf-a(a)lists.trustedfirmware.org> wrote:
Hi Varun,
Thanks for raising this topic (but please do embrace the official terminology “TF-A”…we never really promoted ATF and it's also absolutely outdated now 😉 ).
Arm has received different queries over time on supporting Trusted Firmware LTS releases, but the effort to sustain them is something that the Arm engineering team alone cannot really afford and commit to (either in the TF-A or TF-M space).
The idea has also been just raised to the Trusted Firmware project Board for initial consideration and we will be all very keen to understand how much interest there is from the wider TF-A community of adopters and external (non-Arm) maintainers, so to evaluate the possibility of a more concrete proposal to be carried on within the community Project.
I guess it will also be good to start by elaborating more concretely on the requirements that you would like to see in an hypothetical LTS versioning scheme.
Thanks
Matteo
> From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Varun Wadekar via TF-A
> Sent: 10 June 2020 22:47
> To: tf-a(a)lists.trustedfirmware.org
> Subject: [TF-A] ATF LTS version
>
> Hello team,
>
> To extend the discussion around version upgrades from our last call, I would like to understand if there is enough interest around generating a LTS version of the TF-A to alleviate the pain.
>
> For NVIDIA, this would be helpful as it streamlines the upgrade path for our devices in the field. The LTS version will guarantee security fixes, bug fixes, stability fixes for the longer term and we won’t have to upgrade the entire firmware to get these goodies.
>
> It would be interesting to see what OEMs and maintainers think about this? Has this been discussed at tf.org or Arm internally?
>
> -Varun
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
5 years, 4 months
Re: [TF-A] ATF LTS version
by Joanna Farley
Hi Varun,
Its been a few days now and it would be good to have some other opinions from the broader project contributors but nothing posted yet beyond Francois.
As you suggested this do you think you could follow up with a tech forum session advocating the needs and benefits and how this could work? Having a more detailed proposal may help with the discussion. What I feel is that there would need to be a critical mass of contributors/consumers of the project wanting this to establish the need and they would need to also agree a common consumption model.
Thanks
Joanna
On 11/06/2020, 23:08, "Varun Wadekar" <vwadekar(a)nvidia.com> wrote:
Hello Joanna,
I had discussed the idea with Matteo in the past, but the discussion in the last tech forum prompted the email.
>> I appreciate this CI is currently a little opaque to many contributors as this is still in the process of being transitioned to the OpenCI hosted by Trustedfirmware.org servers which will be visible to all
I agree, it is hard to test all the use cases. The opaque nature of the CI is a bit annoying, but not a big issue.
>> the additional testing done for a 6 monthly tagged release is quite minimal and the larger work is ensuring all documentation is up to date. Additionally all new features are generally behind their own build flags but I appreciate it is some work for a tagged release to be absorbed into product offerings.
Interesting In one of our internal discussions we were exploring the possibility of using doxygen style comments and creating an API reference for a release without a lot of effort. We should try to explore this idea in the community.
>> One thing that had been considered briefly was the production of a security bug only branch
That is a good idea and can act as the base for the LTS version. But we should consider increasing the scope and include bug fixes, stability issues, performance issues, etc. I believe when the community widely adopts TFTF and starts upstreaming the test cases, we can expect more interest around a LTS release.
For platform owners (e.g. NVIDIA) it makes sense to plan our release strategy around LTS versions. Right now, our releases lack direction as we don’t know which version to use. And then there is additional pain of rebasing recent fixes/improvements on older releases.
-Varun
-----Original Message-----
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Joanna Farley via TF-A
Sent: Thursday, June 11, 2020 6:47 AM
To: Matteo Carlini <Matteo.Carlini(a)arm.com>; tf-a(a)lists.trustedfirmware.org
Subject: Re: [TF-A] ATF LTS version
External email: Use caution opening links or attachments
Hi Varun,
I guess this suggestion came in response to last weeks Tech Forum discussion from a question about experiences people had from migrating to different TF-A tagged releases. In general we try and keep the tip of Master at tagged release quality through an extensive CI system ran on each patch. I appreciate this CI is currently a little opaque to many contributors as this is still in the process of being transitioned to the OpenCI hosted by Trustedfirmware.org servers which will be visible to all. As mentioned in the recent "Overview of the TF-A v2.3 Release" presentation on https://www.trustedfirmware.org/meetings/tf-a-technical-forum/ the additional testing done for a 6 monthly tagged release is quite minimal and the larger work is ensuring all documentation is up to date. Additionally all new features are generally behind their own build flags but I appreciate it is some work for a tagged release to be absorbed into product offerings.
I asked at the tech forum last week what more we could do to allow releases to be integrated more easily. On the call it was requested if we could disable weak bindings to symbols so it could be more easily seen where platform decisions may need to be made and we will look into this. If there are any more adjustments to the way tagged releases are produced please let us know.
One thing that had been considered briefly was the production of a security bug only branch that was maintained only between 6 month tagged releases before being replaced by the next security bug only branch based on the next 6 month release but that has not progressed very far as a proposal as until your email here it was perceived to not be in demand. A LTS branch is a larger endeavour as it sounds like something that includes more than security fixes and I look forward to you elaborating more as Matteo requests.
Thanks
Joanna
On 11/06/2020, 12:19, "Matteo Carlini via TF-A" <tf-a(a)lists.trustedfirmware.org> wrote:
Hi Varun,
Thanks for raising this topic (but please do embrace the official terminology “TF-A”…we never really promoted ATF and it's also absolutely outdated now 😉 ).
Arm has received different queries over time on supporting Trusted Firmware LTS releases, but the effort to sustain them is something that the Arm engineering team alone cannot really afford and commit to (either in the TF-A or TF-M space).
The idea has also been just raised to the Trusted Firmware project Board for initial consideration and we will be all very keen to understand how much interest there is from the wider TF-A community of adopters and external (non-Arm) maintainers, so to evaluate the possibility of a more concrete proposal to be carried on within the community Project.
I guess it will also be good to start by elaborating more concretely on the requirements that you would like to see in an hypothetical LTS versioning scheme.
Thanks
Matteo
> From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Varun Wadekar via TF-A
> Sent: 10 June 2020 22:47
> To: tf-a(a)lists.trustedfirmware.org
> Subject: [TF-A] ATF LTS version
>
> Hello team,
>
> To extend the discussion around version upgrades from our last call, I would like to understand if there is enough interest around generating a LTS version of the TF-A to alleviate the pain.
>
> For NVIDIA, this would be helpful as it streamlines the upgrade path for our devices in the field. The LTS version will guarantee security fixes, bug fixes, stability fixes for the longer term and we won’t have to upgrade the entire firmware to get these goodies.
>
> It would be interesting to see what OEMs and maintainers think about this? Has this been discussed at tf.org or Arm internally?
>
> -Varun
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
5 years, 4 months
Finally, a TF-A LTS Proposal!
by Okash Khawaja
TL;DR: Attached is the TF-A LTS proposal doc. Please review and share
your thoughts.
Hello everyone,
Long term support for TF-A has a long history. As a community we have
flirted with it for a while, and like any flirtation, it started with
gossip -- in our case, over mailing list [1] -- more than two years
ago. Then Varun Wadekar did a wonderful tech forum presentation [2]
which demonstrated interest in the topic and raised interesting
questions about LTS.
After some time, the affair lost headline status, only to be revived
again [3], earlier this year. This time however, there is a formal
proposal (no pun intended). After the mailing list discussion and
feedback from TF-A tech forum [4], we have put together a draft which
attempts to give a concrete idea about what the TF-A LTS will look
like.
Please spare some time to read through it and share your feedback. The
plan is to put LTS into action this November.
Cheers!
Okash
[1] https://lists.trustedfirmware.org/archives/search?mlist=tf-a%40lists.truste…
[2] https://www.trustedfirmware.org/docs/TF-A-LTS.pdf
[3] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.…
[4] https://www.trustedfirmware.org/docs/TForg_LTS_proposal.pdf
3 years, 3 months
Re: [TF-TSC] Finally, a TF-A LTS Proposal!
by Joanna Farley
One thing to consider for the LTS technical proposal is a release is made up of multiple repositories and other collateral (eg Readthedocs).
Take the v2.7 release https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.…
As can be seen its made up of more than TF-A and TF-A Tests (TFTF). It also incorporates Hafnium and TF-A OpenCI Scripts. Not tagged is the TF-A OpenCI Jobs (Jenkin Jobs). All generally tested and released together for the release and tagged. Other collateral includes the binaries used during testing (eg compiler versions and other binaries used during testing like FVP’s etc) that make up the supply chain of a release. You may want to consider if this proposal should incorporate all or some of these in the LTS proposal or just stick with the source code sub-components of a release.
Joanna
From: Okash Khawaja via TSC <tsc(a)lists.trustedfirmware.org>
Date: Monday, 8 August 2022 at 18:50
To: tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>
Cc: tsc(a)lists.trustedfirmware.org <tsc(a)lists.trustedfirmware.org>, Varun Wadekar <vwadekar(a)nvidia.com>
Subject: [TF-TSC] Finally, a TF-A LTS Proposal!
TL;DR: Attached is the TF-A LTS proposal doc. Please review and share
your thoughts.
Hello everyone,
Long term support for TF-A has a long history. As a community we have
flirted with it for a while, and like any flirtation, it started with
gossip -- in our case, over mailing list [1] -- more than two years
ago. Then Varun Wadekar did a wonderful tech forum presentation [2]
which demonstrated interest in the topic and raised interesting
questions about LTS.
After some time, the affair lost headline status, only to be revived
again [3], earlier this year. This time however, there is a formal
proposal (no pun intended). After the mailing list discussion and
feedback from TF-A tech forum [4], we have put together a draft which
attempts to give a concrete idea about what the TF-A LTS will look
like.
Please spare some time to read through it and share your feedback. The
plan is to put LTS into action this November.
Cheers!
Okash
[1] https://lists.trustedfirmware.org/archives/search?mlist=tf-a%40lists.truste…
[2] https://www.trustedfirmware.org/docs/TF-A-LTS.pdf
[3] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.…
[4] https://www.trustedfirmware.org/docs/TForg_LTS_proposal.pdf
3 years, 3 months
Rebooting LTS discussion
by Okash Khawaja
Hi,
There have been discussions about having long term support releases
for TF-A, e.g. the email thread [1] and a tech forum [2]. For partners
releasing TF-A in their production devices, LTS is very much needed.
From the previous discussions, it seems like there is an agreement
that LTS is a good idea but we need to build consensus on how to
support it. Any thoughts on this?
Thanks,
Okash
[1] https://lists.trustedfirmware.org/archives/search?mlist=tf-a%40lists.truste…
[2] https://www.trustedfirmware.org/docs/TF-A-LTS.pdf
3 years, 6 months
Re: [TF-TSC] Finally, a TF-A LTS Proposal!
by Dan Handley
Hi Okash/Varun/all
Thanks for this. Some comments from me:
* Criteria: I agree with Joakim that in some cases, a feature may need to be backported as a dependency, although I think this can be mainly decided by the LTS maintainer(s) on a case by case basis. TSC should only be needed where there's controversy/disagreement. There's no mention of normal (i.e. non-security) bugs in generic code. Presumably these don't meet the criteria (?), but it's better to be explicit.
* Frequency: Yearly in November makes sense but if they live for 5/7 years that obviously means 5/7 concurrent branches to maintain, which will make automation even more important.
* Regarding automation, clearly all the CI needs to be automated from the start. It may be possible to manually backport to start with but even this will need to be mostly automated if there are many concurrent branches. I know with the Linux LTS branches that patches are tagged for backporting in the mainline commit messages, and scripts automatically backport to the LTS branches. When patches fail to cleanly backport, it's up to the LTS community to address as it's not scalable for the LTS maintainer(s) to fix all problems. We could consider copying this model (even the scripts) eventually. You mention tagging patches on mainline instead, which could work but I guess could result in tag proliferation.
* Testing criteria: +1 on Joakim's comment about downstream testing not gating LTS patch merging. I’d also say that platforms can only be tested in the LTS CI if they are also tested in the mainline CI.
* Test-and-debug period: "it is expected that more testing and debugging will be done on the November release of TF-A". The testing criteria in the previous section suggest LTS tests are a subset of mainline testing, so in theory the LTS release could be made at the same time as main release. I guess you mean there needs to be more downstream testing of the mainline release before the first LTS release is made. That's fine but this should be made clear and there should be a fixed window so that downstream testing doesn't gate the LTS release.
* "[LTS Maintainer to] Monitor the main branch to identify candidate patches for the LTS branches". This might work to start with but I don't think is very scalable. I think it's reasonable to make all TF-A contributors responsible for identifying such patches at the time they created, with the LTS maintainer acting as backup when things are missed.
* "After reviews are complete, merge the patch and bump the minor version, if required". This suggest no downstream testing is required for minor LTS versions - is that right? I would have thought a small window is required. Or can we assume that further patches/releases are made if there are problems?
* "What happens when a bug fix applies just to a LTS branch and not to the master branch?". If a bug is only reproducible on the LTS branch, then it seems to me this is a reasonable exception to the "only backport from mainline" rule.
* "When testing a backported patch, what if one of the partners needs more time while the patch fix is time-critical and hence slowing other partners?". I would say there should be a fixed window. If a partner subsequently finds a problem, another minor release can be made.
* I suspect a new TF-A LTS mailing list will be required for this (TBD).
* Maintainership options: This is going to take more discussion, but my personal view is the current set of maintainers/contributors does not closely intersect with those interested in consuming an LTS, so it's wouldn't be fair for the current maintainers/contributors to take the whole burden of LTS maintainership.
Regards
Dan.
> -----Original Message-----
> From: Okash Khawaja via TSC <tsc(a)lists.trustedfirmware.org>
> Sent: 08 August 2022 18:49
> To: tf-a(a)lists.trustedfirmware.org
> Cc: tsc(a)lists.trustedfirmware.org; Varun Wadekar <vwadekar(a)nvidia.com>
> Subject: [TF-TSC] Finally, a TF-A LTS Proposal!
>
> TL;DR: Attached is the TF-A LTS proposal doc. Please review and share your
> thoughts.
>
> Hello everyone,
>
> Long term support for TF-A has a long history. As a community we have flirted
> with it for a while, and like any flirtation, it started with gossip -- in
> our case, over mailing list [1] -- more than two years ago. Then Varun
> Wadekar did a wonderful tech forum presentation [2] which demonstrated
> interest in the topic and raised interesting questions about LTS.
>
> After some time, the affair lost headline status, only to be revived again
> [3], earlier this year. This time however, there is a formal proposal (no pun
> intended). After the mailing list discussion and feedback from TF-A tech
> forum [4], we have put together a draft which attempts to give a concrete
> idea about what the TF-A LTS will look like.
>
> Please spare some time to read through it and share your feedback. The plan
> is to put LTS into action this November.
>
> Cheers!
> Okash
>
> [1] https://lists.trustedfirmware.org/archives/search?mlist=tf-
> a%40lists.trustedfirmware.org&q=%5BTF-A%5D+ATF+LTS+version
> [2] https://www.trustedfirmware.org/docs/TF-A-LTS.pdf
> [3] https://lists.trustedfirmware.org/archives/list/tf-
> a(a)lists.trustedfirmware.org/thread/ZZ2DDIW2DS4B7QYHYOL7XE4IPUW7LUNT/
> [4] https://www.trustedfirmware.org/docs/TForg_LTS_proposal.pdf
3 years, 2 months
Re: [TF-TSC] Finally, a TF-A LTS Proposal!
by Varun Wadekar
Hi,
Thanks for the feedback. My comments below.
> There's no mention of normal (i.e. non-security) bugs in generic code. Presumably these don't meet the criteria (?), but it's better to be explicit.
[VW] Makes sense. The intention is to focus on security issues, but I agree that we should look at such bugs on a case-by-case basis. We will try to include this in the guidance.
> I know with the Linux LTS branches that patches are tagged for backporting in the mainline commit messages, and scripts automatically backport to the LTS branches. When patches fail to cleanly backport, it's up to the LTS community to address as it's not scalable for the LTS maintainer(s) to fix all problems. We could consider copying this model (even the scripts) eventually.
[VW] That is a good idea. I agree we should try to replicate mechanisms that already work.
> I guess you mean there needs to be more downstream testing of the mainline release before the first LTS release is made. That's fine but this should be made clear and there should be a fixed window so that downstream testing doesn't gate the LTS release.
[VW] The intention was to allow platforms more testing time. We have tried to provide a deadline but will try to make it clear.
> I think it's reasonable to make all TF-A contributors responsible for identifying such patches at the time they created, with the LTS maintainer acting as backup when things are missed.
[VW] This is what I had in mind at the start and want to aim for. But, want to catch scenarios where contributors miss this. So, we want a reviewer list that never misses such a patch and then educate the contributors to get to the end goal.
> "After reviews are complete, merge the patch and bump the minor version, if required". This suggest no downstream testing is required for minor LTS versions - is that right? I would have thought a small window is required. Or can we assume that further patches/releases are made if there are problems?
[VW] The intent is to complete the testing _before_ the patch is merged to the branch. This testing again has a deadline to avoid slowing down the release process.
> "What happens when a bug fix applies just to a LTS branch and not to the master branch?". If a bug is only reproducible on the LTS branch, then it seems to me this is a reasonable exception to the "only backport from mainline" rule.
[VW] Agreed.
> "When testing a backported patch, what if one of the partners needs more time while the patch fix is time-critical and hence slowing other partners?". I would say there should be a fixed window. If a partner subsequently finds a problem, another minor release can be made.
[VW] Agreed.
> I suspect a new TF-A LTS mailing list will be required for this
[VW] Probably. Would like to address this in the Tech forum.
> This is going to take more discussion, but my personal view is the current set of maintainers/contributors does not closely intersect with those interested in consuming an LTS, so it's wouldn't be fair for the current maintainers/contributors to take the whole burden of LTS maintainership.
[VW] I understand. In the absence of a formal group, the easiest was forward was to request the current maintainers. We should discuss more in the Tech forum.
-Varun
-----Original Message-----
From: Dan Handley <Dan.Handley(a)arm.com>
Sent: Thursday, 11 August 2022 12:12 PM
To: Okash Khawaja <okash(a)google.com>; tf-a(a)lists.trustedfirmware.org
Cc: tsc(a)lists.trustedfirmware.org; Varun Wadekar <vwadekar(a)nvidia.com>
Subject: RE: [TF-TSC] Finally, a TF-A LTS Proposal!
External email: Use caution opening links or attachments
Hi Okash/Varun/all
Thanks for this. Some comments from me:
* Criteria: I agree with Joakim that in some cases, a feature may need to be backported as a dependency, although I think this can be mainly decided by the LTS maintainer(s) on a case by case basis. TSC should only be needed where there's controversy/disagreement. There's no mention of normal (i.e. non-security) bugs in generic code. Presumably these don't meet the criteria (?), but it's better to be explicit.
* Frequency: Yearly in November makes sense but if they live for 5/7 years that obviously means 5/7 concurrent branches to maintain, which will make automation even more important.
* Regarding automation, clearly all the CI needs to be automated from the start. It may be possible to manually backport to start with but even this will need to be mostly automated if there are many concurrent branches. I know with the Linux LTS branches that patches are tagged for backporting in the mainline commit messages, and scripts automatically backport to the LTS branches. When patches fail to cleanly backport, it's up to the LTS community to address as it's not scalable for the LTS maintainer(s) to fix all problems. We could consider copying this model (even the scripts) eventually. You mention tagging patches on mainline instead, which could work but I guess could result in tag proliferation.
* Testing criteria: +1 on Joakim's comment about downstream testing not gating LTS patch merging. I'd also say that platforms can only be tested in the LTS CI if they are also tested in the mainline CI.
* Test-and-debug period: "it is expected that more testing and debugging will be done on the November release of TF-A". The testing criteria in the previous section suggest LTS tests are a subset of mainline testing, so in theory the LTS release could be made at the same time as main release. I guess you mean there needs to be more downstream testing of the mainline release before the first LTS release is made. That's fine but this should be made clear and there should be a fixed window so that downstream testing doesn't gate the LTS release.
* "[LTS Maintainer to] Monitor the main branch to identify candidate patches for the LTS branches". This might work to start with but I don't think is very scalable. I think it's reasonable to make all TF-A contributors responsible for identifying such patches at the time they created, with the LTS maintainer acting as backup when things are missed.
* "After reviews are complete, merge the patch and bump the minor version, if required". This suggest no downstream testing is required for minor LTS versions - is that right? I would have thought a small window is required. Or can we assume that further patches/releases are made if there are problems?
* "What happens when a bug fix applies just to a LTS branch and not to the master branch?". If a bug is only reproducible on the LTS branch, then it seems to me this is a reasonable exception to the "only backport from mainline" rule.
* "When testing a backported patch, what if one of the partners needs more time while the patch fix is time-critical and hence slowing other partners?". I would say there should be a fixed window. If a partner subsequently finds a problem, another minor release can be made.
* I suspect a new TF-A LTS mailing list will be required for this (TBD).
* Maintainership options: This is going to take more discussion, but my personal view is the current set of maintainers/contributors does not closely intersect with those interested in consuming an LTS, so it's wouldn't be fair for the current maintainers/contributors to take the whole burden of LTS maintainership.
Regards
Dan.
> -----Original Message-----
> From: Okash Khawaja via TSC <tsc(a)lists.trustedfirmware.org>
> Sent: 08 August 2022 18:49
> To: tf-a(a)lists.trustedfirmware.org
> Cc: tsc(a)lists.trustedfirmware.org; Varun Wadekar <vwadekar(a)nvidia.com>
> Subject: [TF-TSC] Finally, a TF-A LTS Proposal!
>
> TL;DR: Attached is the TF-A LTS proposal doc. Please review and share
> your thoughts.
>
> Hello everyone,
>
> Long term support for TF-A has a long history. As a community we have
> flirted with it for a while, and like any flirtation, it started with
> gossip -- in our case, over mailing list [1] -- more than two years
> ago. Then Varun Wadekar did a wonderful tech forum presentation [2]
> which demonstrated interest in the topic and raised interesting questions about LTS.
>
> After some time, the affair lost headline status, only to be revived
> again [3], earlier this year. This time however, there is a formal
> proposal (no pun intended). After the mailing list discussion and
> feedback from TF-A tech forum [4], we have put together a draft which
> attempts to give a concrete idea about what the TF-A LTS will look like.
>
> Please spare some time to read through it and share your feedback. The
> plan is to put LTS into action this November.
>
> Cheers!
> Okash
>
> [1]
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
> s.trustedfirmware.org%2Farchives%2Fsearch%3Fmlist%3Dtf-&data=05%7C
> 01%7Cvwadekar%40nvidia.com%7Cdb167e872d95420c503408da7b8a5417%7C43083d
> 15727340c1b7db39efd9ccc17a%7C0%7C0%7C637958131793859497%7CUnknown%7CTW
> FpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6
> Mn0%3D%7C3000%7C%7C%7C&sdata=37mdLWDEDIRwAnsxo9c7bu2KwDq9oX%2B3Y4j
> %2BNiHzI%2Bc%3D&reserved=0
> a%40lists.trustedfirmware.org&q=%5BTF-A%5D+ATF+LTS+version
> [2]
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> trustedfirmware.org%2Fdocs%2FTF-A-LTS.pdf&data=05%7C01%7Cvwadekar%
> 40nvidia.com%7Cdb167e872d95420c503408da7b8a5417%7C43083d15727340c1b7db
> 39efd9ccc17a%7C0%7C0%7C637958131793859497%7CUnknown%7CTWFpbGZsb3d8eyJW
> IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%
> 7C%7C%7C&sdata=05dYPwDmH7OZMjMVb0AtN9eSJR8R4InaZ5ANo8qqqUA%3D&
> reserved=0 [3]
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
> s.trustedfirmware.org%2Farchives%2Flist%2Ftf-&data=05%7C01%7Cvwade
> kar%40nvidia.com%7Cdb167e872d95420c503408da7b8a5417%7C43083d15727340c1
> b7db39efd9ccc17a%7C0%7C0%7C637958131793859497%7CUnknown%7CTWFpbGZsb3d8
> eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3
> 000%7C%7C%7C&sdata=jjsY%2B9EEZQiTqg5o9RLucSB5WzwsheWOOUIzXz5PRUQ%3
> D&reserved=0
> a(a)lists.trustedfirmware.org/thread/ZZ2DDIW2DS4B7QYHYOL7XE4IPUW7LUNT/
> [4]
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> trustedfirmware.org%2Fdocs%2FTForg_LTS_proposal.pdf&data=05%7C01%7
> Cvwadekar%40nvidia.com%7Cdb167e872d95420c503408da7b8a5417%7C43083d1572
> 7340c1b7db39efd9ccc17a%7C0%7C0%7C637958131793859497%7CUnknown%7CTWFpbG
> Zsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%
> 3D%7C3000%7C%7C%7C&sdata=q1Pk6ucv8RA7g3Yj59CMkrb2ZRY1abNHSiCSHJBOn
> hg%3D&reserved=0
3 years, 2 months
Re: [TF-TSC] Finally, a TF-A LTS Proposal!
by Joakim Bech
Hi Okash, Varun, all,
On Mon, Aug 08, 2022 at 06:48:42PM +0100, Okash Khawaja via TSC wrote:
> TL;DR: Attached is the TF-A LTS proposal doc. Please review and share
> your thoughts.
>
> Hello everyone,
>
> Long term support for TF-A has a long history. As a community we have
> flirted with it for a while, and like any flirtation, it started with
> gossip -- in our case, over mailing list [1] -- more than two years
> ago. Then Varun Wadekar did a wonderful tech forum presentation [2]
> which demonstrated interest in the topic and raised interesting
> questions about LTS.
>
> After some time, the affair lost headline status, only to be revived
> again [3], earlier this year. This time however, there is a formal
> proposal (no pun intended). After the mailing list discussion and
> feedback from TF-A tech forum [4], we have put together a draft which
> attempts to give a concrete idea about what the TF-A LTS will look
> like.
>
> Please spare some time to read through it and share your feedback. The
> plan is to put LTS into action this November.
>
Thanks for putting this together, much appreciated. Some comments from
me/Linaro.
Criteria:
--------
- "large parts of the process can be automated"
I'm not sure I'm fully convinced about that, but if CI/CD/testing have
been configured to have good support for LTS-branches, then the actual
and manual work is mostly doing the actual backporting. Aiming for an
automated process is a good goal.
- "No features will be backported"
As for the criteria I believe your suggestion makes a lot of sense.
Even though I do agree with the "No features will be backported", I'm
pretty sure that from time to time there will be fixes that for some
reason need to pull in features as well, mainly as a dependency. Perhaps
we could change the sentence slightly to indicate that we generally
don't want to backport features, but if a fix depends or is dependant on
a feature, then in some cases we might consider backporting features as
well. Case by case? TF-TSC people could decide?
Lifetime and frequency:
----------------------
I support your suggestions and motivations.
Testing Criteria / 4.Platform tests:
-----------------------------------
Although I support this, I can see foresee that this most likely will be
challenging. I'm talking from my experience as an OP-TEE maintainer. As
the amount of platforms supported grew, it became harder and harder to
get the platform maintainers finding time and helping out with testing
release candidates on their own devices/platforms. We have tried a
couple of different things, like using stop lights in the docs, keeping
tested platforms "green", the ones that hasn't been tested recently
"orange" and the rest "red". However, we abandoned that, since it was a
lot of work to keep track of it. Instead we simply just keep track of
the tested platforms in the git release tag. I.e., we list all platforms
tested both by the core maintainers as well as external platform
maintainers. Sub-optimal, but keeps a track record and keeps things
moving. Here I'm mainly talking about platforms found upstream. In your
suggestion you're talking about platforms that are not maintained
upstream, which in my opinion makes things even more complicated.
In short, I do support something like this, but I think there needs to
be some kind of deadline for downstream testing. If not met, then that
platform shouldn't be considered as up-to-date for a LTS. I think this
is what you also had in mind with "a pre-defined window".
TFTF branching and others gits:
------------------------------
I agree with the need to "LTS tag" TFTF and potentially other projects
as well. I.e., as Joanna indicated, enabling LTS in TF-A might force us
to add LTS branches in the other TF.org gits as well. I've touched upon
it before, but maybe time for putting together TF.org build(s) that
builds (and tests) all projects under the TrustedFirmware umbrella?
Example timeline:
----------------
- "Note that here we don’t allow the extra testing and debugging time
that we had between Nov 2022 and early Feb 2023. This is because there
isn’t as much to test and debug as an annual LTS release has."
I think this is related to my comment further up. If the "large part of
the process can be automated" holds up, then I'd guess that the same
kind of testing could (and should) be done.
Maintainership:
--------------
In short LTS doesn't come for free, so in some way it needs to be
funded. Carving out LTS work from existing work as a maintainer can be
challenging and a lot of maintainers spend time on other "internal"
tasks and use cases as well.
- "A day in the life of a maintainer"
I guess this sections justify my doubts that "large parts of the process
can be automated".
Execution Plan:
--------------
+1
Future plans:
------------
Agree, it's hard to make a bulletproof strategy, but having a strategy
capturing most of the things is a lot better than having no strategy at
all.
Again, thanks for putting together the proposal, it's a good proposal.
> Cheers!
> Okash
>
> [1] https://lists.trustedfirmware.org/archives/search?mlist=tf-a%40lists.truste…
> [2] https://www.trustedfirmware.org/docs/TF-A-LTS.pdf
> [3] https://lists.trustedfirmware.org/archives/list/tf-a@lists.trustedfirmware.…
> [4] https://www.trustedfirmware.org/docs/TForg_LTS_proposal.pdf
> --
> TSC mailing list -- tsc(a)lists.trustedfirmware.org
> To unsubscribe send an email to tsc-leave(a)lists.trustedfirmware.org
// Regards,
Joakim
3 years, 2 months