Hello Erik,
Thanks for the feedback.
On 3/26/20 3:37 PM, Shreve, Erik wrote:
Sandrine,
Really glad to see this being pulled together. A couple of areas of feedback around the Platform Support Life Cycle.
As previously mentioned there are two orthogonal concerns captured in the current life cycle: Support and Functionality. I'd like to see these split out.
Yes, you are the second person to mention that and I agree with you both. Unless someone disagrees, I intend to update the proposal and separate these 2 concepts in the next version of the document.
For functionality, chip vendors may not have a business case for supporting all features on a given platform but they may provide full support for the features they have chosen to include. A simple example would be supporting PSA FF Isolation Level 1 only due to lack of HW isolation support needed to achieve Isolation Level 2 or greater.
I completely agree. It would not make sense to support all features on all platforms just for the sake of completeness. Each platform ought to implement what is relevant in its case.
That's what the current proposal tried to convey: a fully supported platform must "build all configurations *supported by this platform*" and "All *supported* configurations are tested in the CI". The key word here is supported and that would be defined by the platform itself. But I can see that maybe this wasn't clear enough. Your proposal below makes that a lot clearer.
Also, I'd like to see a stronger standard put forth for platform documentation. If a platform is "supported," I believe the documentation should be complete and accurate. A lack of complete and clear documentation leaves open a wide door for misuse/misconfiguration which could result in a vulnerable system.
Fair point.
But is it something we should include in this proposal or should we push it to a separate document setting expectations for the project's documentation, which the current general proposal could refer to (as in, "the platform should provide quality documentation up to the project's criteria defined in document XXX')?
This is definitely an important topic but I am wary of keeping the tf.org proposal concise and focused at this point. I am worried that if we put too much stuff in it discussions will diverge too much and we might never reach an agreement.
The same applies to testing standards for example, we could detail that in the proposal or simply leave it to projects to define it separately.
Here is a more concrete proposal:
Functional Support: Each project shall provide a standard feature or functionality list. Each platform shall include in its documentation a copy of this list with the supported functionality marked as supported. The platform documentation may reference a ticket if support is planned but not yet present. The platform documentation shall explicitly state if a feature or function has no plans for support.
Regarding the last item, this would require all platform maintainers to update their documentation every time a new feature is added to the project's global list of features. This seems too much of a constraint and unnecessary maintenance burden to me.
I think a better, more lightweight alternative might be to let platform maintainers list what's supported and if some feature is not listed, it implies that it is not supported. This does not prevent platform maintainers from indicating their future plans of supporting a feature if they want to.
The feature/functionality list shall be versioned, with the version tied to the release version(s) of the project. In this way, it will be clear if a platform was last officially updated for version X but the project is currently at version Y > X.
I can see that Joakim Bech proposed something similar, with more details about how this was implemented for OP-TEE.
Note: projects will need to adopt (if they have not already) a version scheme that distinguishes between feature updates and bug fixes.
Sorry I didn't get this, could you please elaborate?
Each project and platform shall use tags or similar functionality on tickets to associate tickets to features/functionality and platforms. If the names of tags can't match the name of the feature or platform exactly then a mapping shall be provided in the appropriate document(s).
If there's no appropriate tag in some cases, I guess we could always use a git SHA1 of a specific commit.
Life Cycle State
Fully Supported There is (at least) one active code owner for this platform. All supported features build and either all tests pass or failures are associated with tracked known issues. Other (not associated to a test) Known Issues are tracked Documentation is up to date
Note: Projects should document standards on how "active" code ownership is measured and further document standards on how code owners are warned about impending life cycle state changes.
Yes, good point, that is currently undefined in the proposal but I agree that it needs defining per project. I will add an item in the last section of the document.
I am starting to think that we need a list of items to be defined per project. This list would complement the general tf.org proposal. Things like code owners/maintainers activity, code review timelines, and so on.
Orphan There is no active code owner All supported features build and either all tests pass or failures are associated with tracked known issues. Other (not associated to a test) Known Issues may not have been maintained (as there is no active code owner) Documentation status is unclear since there is no active code owner. There has been no change to the feature/functionality list in the project since the platform was last "Fully Supported"
I am confused, you said earlier that you would like to see the concepts of support and functionality split out, but here you're listing 'orphan' as one of the possible states... Did I miss your point?
Out of date Same as orphan, but either: there have been changes to the feature/functionality list, or there are failing tests without tracked tickets, or there are known documentation issues.
Deprecated Same as Out of Date, but the build is broken. Platform may be removed from the project codebase in the future.
Erik Shreve, PSEM Software Security Engineer & Architect (CMCU Platform Development)