On Wed, 15 Nov 2023 17:15:35 +0000
Leif Lindholm <quic_llindhol(a)quicinc.com> wrote:
Hi,
> (I should remember to respond to this as well.)
>
> Hi André,
>
> On Mon, Nov 13, 2023 at 17:23:58 +0000, Andre Przywara wrote:
> > > We need to change make_helpers/arch_features.mk to stop enabling mandatory
> > > features as "present, no need to check" (value '1') but to use runtime
> > > check (value '2') instead. This way any platform which contains CPU not
> >
> > But using runtime checking *for flexible platforms* was the plan all
> > along. We did the groundwork for the FVP, feature by feature, check commit
> > 6503ff2910ae5ed for an example. For all flexible platforms, the idea is
> > that the default architecture revision stays at v8.0 (to cover "-cpu
> > cortex-a57", for instance), but all(?) features get promoted to a runtime
> > check (=2). Check the current version of plat/arm/board/fvp/platform.mk.
>
> Ah, that intended usage-pattern was not clear to me.
Yeah, sorry, I wanted to communicate this to people caring about the QEMU
port, but this was somehow missed.
> Would it be worth adding a sentence to the FEATURE_DETECTION section
> of build-options.rst?
Mmh, skimming over this makes me think this stanza deserves some rewrite
anyway. I will try to send a patch after the release.
But I am not sure this is the right place for the explanation, since
FEATURE_DETECTION is only a loosely related debug feature, and you get the
runtime detection when setting something to =2 regardless. Will try to
address this.
Thanks,
Andre
>
> Thanks for explaining!
>
> Regards,
>
> Leif
>
> > The same problem hits the Arm FPGA, which is a very simple platform
> > sporting the newest cores in development. Check commit b321c243423b93,
> > which enabled all relevant features to do a runtime check.
> >
> > So please just push a patch similar to that, and we should be good?
> >
> > Cheers,
> > Andre
> >
> > > yet fully developed can boot TF-A. Or platform where one firmware can boot
> > > different cpu models.
> > >
> > > I am sorry that I bring it so late in release cycle but was unable to use
> > > computer for a development for some time.
> >
(I should remember to respond to this as well.)
Hi André,
On Mon, Nov 13, 2023 at 17:23:58 +0000, Andre Przywara wrote:
> > We need to change make_helpers/arch_features.mk to stop enabling mandatory
> > features as "present, no need to check" (value '1') but to use runtime
> > check (value '2') instead. This way any platform which contains CPU not
>
> But using runtime checking *for flexible platforms* was the plan all
> along. We did the groundwork for the FVP, feature by feature, check commit
> 6503ff2910ae5ed for an example. For all flexible platforms, the idea is
> that the default architecture revision stays at v8.0 (to cover "-cpu
> cortex-a57", for instance), but all(?) features get promoted to a runtime
> check (=2). Check the current version of plat/arm/board/fvp/platform.mk.
Ah, that intended usage-pattern was not clear to me.
Would it be worth adding a sentence to the FEATURE_DETECTION section
of build-options.rst?
Thanks for explaining!
Regards,
Leif
> The same problem hits the Arm FPGA, which is a very simple platform
> sporting the newest cores in development. Check commit b321c243423b93,
> which enabled all relevant features to do a runtime check.
>
> So please just push a patch similar to that, and we should be good?
>
> Cheers,
> Andre
>
> > yet fully developed can boot TF-A. Or platform where one firmware can boot
> > different cpu models.
> >
> > I am sorry that I bring it so late in release cycle but was unable to use
> > computer for a development for some time.
>
Hello All,
We are sending this note, to notify you of all the implementation details related to the Architectural Features Detection Mechanism.
Summary:
This is a diagnostic tool, which is targeted to mitigate the runtime exceptions due to incorrect feature enablement. This is currently marked as experimental and disabled by default, but our target is to make it mandatory over the time.
All the platform owners are expected to read the details(and review the patch) and give it a try by enabling this mechanism and share your thoughts/feedback or any questions to @Jayanth Dodderi Chidanand<mailto:JAYANTHDODDERI.CHIDANAND@arm.com> or me.
Patches under Review: https://review.trustedfirmware.org/q/topic:jc/detect_feat
Details:
1. What is Feature Detection Mechanism?
* Feature Detection is a procedure/mechanism aimed at identifying the features which are enabled ( by software) and not detected/supported in the hardware.
* It could be considered as a diagnostic tool to quickly check and get assured which features are not supported by the hardware at an early stage of booting.
2. Why do we need it?
Currently, most of the feature-specific register's context management, save and restore routines are conditionally controlled with the ARM_ARCH_AT_LEAST macro, which is primarily causing exceptions under various scenarios like:
* For a given version of the architecture, the optional and mandatory features control the access to various registers. If the given version of implementation does not support both, unconditional access to such registers leads to undefined behaviour.
* Accessing registers without verifying their actual presence for the given implementation.
In general, the problem is broader than just this specific case. We should not rely on ARM_ARCH_AT_LEAST macro to associate with an architecture extension but rather supply the individual ENABLE_FEAT_xxx option for each feature.
Again, having individual build flags, won't resolve this completely. There is still room for error as users may unknowingly enable the flags. So, the build flags still need to be validated before performing any action guarded by them.
This mechanism helps in resolving this issue completely. It assists in detecting the features which are not present in the platform but are enabled by software unknowingly. It prevents the runtime exception, due to the consequences already mentioned.
3. How have we designed and implemented it?
We are introducing a tri-state approach for each feature build flag. From now on, the build flags take three values/states ( 0,1 or 2), and they imply as follows:
The 3 states are:
* ENABLE_FEAT_xxx = 0: The feature is disabled statically at compile time.
* ENABLE_FEAT_xxx = 1: The feature is enabled and must be present in hardware. There will be hard panic if the feature is not present at cold boot.
* ENABLE_FEAT_xxx = 2: The feature is enabled and detected at runtime
Based on the value defined for each feature flag, they get detected either at boot-time or at runtime, respectively.
For simplicity, let's take FEAT_HCX which is available in arch version 8.7. We provide a build option for enabling this feature, say "ENABLE_FEAT_HCX".
* ENABLE_FEAT_HCX=0; The feature is disabled statically at compile time.
* ENABLE_FEAT_HCX=1; The feature is enabled and must be present in hardware. There will be hard panic if the feature is not present at cold boot. i.e., we detect, whether the HCX feature is present in the PE, by reading its ID register and if not, panic will be called. Thereby at an early boot phase, we stop and report that FEAT_HCX is not supported by PE.
* ENABLE_FEAT_HCX=2; The feature is enabled but dynamically enabled at runtime depending on hardware capability. Here, a feature detection check will happen during runtime.
4. What is the status of this implementation? Is it completely implemented and tested?
We have divided the entire implementation into two phases. In phase-1 FEAT_STATES { 0, 1} are handled and FEAT_STATE{2} will be handled ahead. Currently, we are in phase-1 delivery, wherein we are introducing a procedure, which will read through all the enabled feature build flags, and if they are defined to state1, ENABLE_FEAT_XXX=1 the respective feature will be detected.
5. Which all features are considered here?
TF-A supports most arm architectural features from v8.0 and upwards. Some are mandatory and some are optional features as per the Arm ARM docs. So, both ( Mandatory and optional features from v8.0) are detected under this mechanism.
6. Does this mechanism modify or impact any existing implementation related to any of the architectural features supported in TF-A?
* Yes.
* Ideally, TF-A enables the architectural features which are mandatory by default from a particular arch version and upwards ( as per Arm-ARM docs ) and disables the optional features by default and allows the platforms to make the decision on enabling the optional feature based on their requirements.
* This pattern is followed for most of the features. However, there are some cases wherein optional features are enabled by default within TF-A ( Eg; FEAT_SPE, FEAT_SVE ), which shouldn't have been handled this way.
* With the feature detection mechanism in place, as stated earlier the procedure runs through all the enabled features(optional and mandatory) and identifies them.
* Now, FEAT_SPE and FEAT_SVE are optional features, which are enabled by default and when detected will not be identified by the PE, if it doesn't support it and panics during booting.
* So, since we have enabled these optional features within the TF-A build system, it would panic and stop booting.
* If we upstream this mechanism, all the partner's platforms will be impacted involuntarily.
This problem will not be seen in other cases like:
* Mandatory Feature: Let's say FEAT_FGT which is mandatory from the 8.6 version. So, if a platform is based on v8.6 it will implement this, and this feature will be detected. So no issue here. If the platform is based on v8.5, this FEAT_FGT is not enabled by the TF-A. It gets enabled from 8.6. So here, in this case, the feature is disabled so nothing to worry about.
* Optional Feature: Let's say FEAT_NV2 which is an optional feature from arch version 8.4. is supported by TF-A. Since it is an optional one as per Arm ARM, TF-A implements and disables it by default and allows the platforms to decide and enable them as per their requirements. So here, if the platform enables it, it implies they are sure this feature is implemented. If not, this mechanism will help them by detecting it, so that they disable it in future. In general, this would not break the boot flow in all scenarios.
But if the optional feature is enabled by TF-A itself, will stop the boot flow in most of scenarios.
So, to avoid breaking change, we have decided to overlook such optional features for now and update our partners and get their feedback. Based on that, in future, we will disable these optional features which were enabled by default and will send another email to enable it explicitly according to their requirements.
7. What should the platforms be aware of, with the upstreaming of this mechanism?
* Currently, we are introducing this entire implementation as an experimental mechanism, wherein we provide an explicit build flag (FEATURE_DETECTION) to enable the feature detection mechanism itself. We urge the platforms to enable this mechanism, test it and get used to its behaviour before it gets mandated.
* So, for now, it wouldn't cause any issue. But our plan is to make sure this mechanism runs by default, as we want to mitigate the runtime Exceptions. As part of the 2.7 release, we are targeting to upstream this implementation and later, have some time window, wherein our partners get used to it and provide feedback as well.
8. Will there be any breakdown during runtime, with respect to any of the platforms?
Yes. It's explained in detail above.
9. What is the long-term plan with this mechanism? When will this be completely implemented and tested end to end?
We target it to be implemented full-fledged by EoY 2022, but it depends on the feedback received from our partners and get this done.
Thanks
