Clock and power management in ATF, preemption issue.
I have seen that multiple Vendors are using arm scmi protocol to do clock management in bl31[0][1].
The problem with such implementations is any long running SCMI operations (like PLL locking for example) will hold the core in EL3 for extended period of time, adding to latency of NS EL1 interrupt handling as ATF is not preemptible.
This problem can be solved by making ATF preemptible, similar to how OP-TEE does it by by implementing a remote procedural call after it receives a interrupt. [2]How Linux does SMC call with YIELD flag enabled. [3]How OP-TEE handles the timer interrupt [4]How Linux receives an interrupt which came in EL3->SEL1
As quoted in exception Handling document in ATF docs[5]
"Receive exceptions, but handle part of the exception in EL3, and delegate processing of the error to dedicated software stack running at lower secure ELs (as above); additionally, the Normal world may also be required to participate in the handling, or be notified of such events (for example, as an event). In this scheme, exception handling potentially and maximally spans all ELs in both Secure and Normal worlds."
From this we can understand that we can delegate the exception to EL1 if we are already in EL3. (way to preempt)
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
We don't want to put it in TEE as it will constraint our devices to that specific TEE.
I am starting to work on a proposal to make ATF preemptible similar to how OP-TEE is doing it. a)Will the similar approach from OP-TEE if implemented and working be accepted by ATF upstream? The SMC call with YIELD option will only be preemtible so will not affect the normal flow.
b)As quoted in Trusted Firmware-A Document[6] Page 94, "Yielding calls are reserved exclusively for Trusted OS providers" "Yielding 0- 1 Reserved for existing Armv7-A calls Yielding 2-63 Trusted OS Standard Calls"
Can this range be consumed within ATF/bl31 firmware or is it necessary to forward all yielding calls to Trusted OS?
[0]https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/st/stm... [1]https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/21840 [2]https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/optee/smc_abi.c... [3]https://github.com/OP-TEE/optee_os/blob/fc57019cb35c8c1bad66fc6d814ace5debde... [4]https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/optee/smc_abi.c... [5]https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon... [6]https://trustedfirmware-a.readthedocs.io/_/downloads/en/latest/pdf/
Regards, Kamlesh
Hi Kamlesh,
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware
parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare-metal partition and does not require migrating the SCMI operations to a TEE.
cheers, Achin
________________________________ From: Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org Sent: 15 May 2024 10:33 To: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Cc: bb@ti.com bb@ti.com; vigneshr@ti.com vigneshr@ti.com; d-gole@ti.com d-gole@ti.com; afd@ti.com afd@ti.com; hnagalla@ti.com hnagalla@ti.com Subject: [TF-A] Make TF-A (bl31) (El3 firmware) preemtible
Clock and power management in ATF, preemption issue.
I have seen that multiple Vendors are using arm scmi protocol to do clock management in bl31[0][1].
The problem with such implementations is any long running SCMI operations (like PLL locking for example) will hold the core in EL3 for extended period of time, adding to latency of NS EL1 interrupt handling as ATF is not preemptible.
This problem can be solved by making ATF preemptible, similar to how OP-TEE does it by by implementing a remote procedural call after it receives a interrupt. [2]How Linux does SMC call with YIELD flag enabled. [3]How OP-TEE handles the timer interrupt [4]How Linux receives an interrupt which came in EL3->SEL1
As quoted in exception Handling document in ATF docs[5]
"Receive exceptions, but handle part of the exception in EL3, and delegate processing of the error to dedicated software stack running at lower secure ELs (as above); additionally, the Normal world may also be required to participate in the handling, or be notified of such events (for example, as an event). In this scheme, exception handling potentially and maximally spans all ELs in both Secure and Normal worlds."
From this we can understand that we can delegate the exception to EL1 if we are already in EL3. (way to preempt)
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
We don't want to put it in TEE as it will constraint our devices to that specific TEE.
I am starting to work on a proposal to make ATF preemptible similar to how OP-TEE is doing it. a)Will the similar approach from OP-TEE if implemented and working be accepted by ATF upstream? The SMC call with YIELD option will only be preemtible so will not affect the normal flow.
b)As quoted in Trusted Firmware-A Document[6] Page 94, "Yielding calls are reserved exclusively for Trusted OS providers" "Yielding 0- 1 Reserved for existing Armv7-A calls Yielding 2-63 Trusted OS Standard Calls"
Can this range be consumed within ATF/bl31 firmware or is it necessary to forward all yielding calls to Trusted OS?
[0]https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/st/stm... [1]https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/21840 [2]https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/optee/smc_abi.c... [3]https://github.com/OP-TEE/optee_os/blob/fc57019cb35c8c1bad66fc6d814ace5debde... [4]https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/optee/smc_abi.c... [5]https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon... [6]https://trustedfirmware-a.readthedocs.io/_/downloads/en/latest/pdf/
Regards, Kamlesh -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
Hi Achin,
Thanks for the suggestion,
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare-metal partition and does not require migrating the SCMI operations to a TEE.
As per [0], we can't have a SEL1 firmware at all(e.g. OP-TEE), if we run S-EL0 partition image.
"It is not currently possible for BL31 to integrate SPM support and a Secure Payload Dispatcher (SPD) at the same time; they are mutually exclusive. In the SPM bootflow, a Secure Partition image executing at S-EL0 replaces the Secure Payload image executing at S-EL1 (e.g. a Trusted OS). Both are referred to as BL32."
Let me know if you think otherwise.
What we want is, 1. Run clock and power management services as well as TEE services 2. We don't want to put clock and power and management services in TEE, so that we have liberty to use any TEE we want. Example, if we put our SCMI operations in OP-TEE, then we'll be constrained to use OP-TEE and only OP-TEE.
[0]https://trustedfirmware-a.readthedocs.io/en/latest/components/secure-partiti...
Regards, Kamlesh
Achin Gupta Achin.Gupta@arm.com writes:
Hi Kamlesh,
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware
parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare-metal partition and does not require migrating the SCMI operations to a TEE.
cheers, Achin
From: Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org Sent: 15 May 2024 10:33 To: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Cc: bb@ti.com bb@ti.com; vigneshr@ti.com vigneshr@ti.com; d-gole@ti.com d-gole@ti.com; afd@ti.com afd@ti.com; hnagalla@ti.com hnagalla@ti.com Subject: [TF-A] Make TF-A (bl31) (El3 firmware) preemtible
Clock and power management in ATF, preemption issue.
I have seen that multiple Vendors are using arm scmi protocol to do clock management in bl31[0][1].
The problem with such implementations is any long running SCMI operations (like PLL locking for example) will hold the core in EL3 for extended period of time, adding to latency of NS EL1 interrupt handling as ATF is not preemptible.
This problem can be solved by making ATF preemptible, similar to how OP-TEE does it by by implementing a remote procedural call after it receives a interrupt. [2]How Linux does SMC call with YIELD flag enabled. [3]How OP-TEE handles the timer interrupt [4]How Linux receives an interrupt which came in EL3->SEL1
As quoted in exception Handling document in ATF docs[5]
"Receive exceptions, but handle part of the exception in EL3, and delegate processing of the error to dedicated software stack running at lower secure ELs (as above); additionally, the Normal world may also be required to participate in the handling, or be notified of such events (for example, as an event). In this scheme, exception handling potentially and maximally spans all ELs in both Secure and Normal worlds."
From this we can understand that we can delegate the exception to EL1 if we are already in EL3. (way to preempt)
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
We don't want to put it in TEE as it will constraint our devices to that specific TEE.
I am starting to work on a proposal to make ATF preemptible similar to how OP-TEE is doing it. a)Will the similar approach from OP-TEE if implemented and working be accepted by ATF upstream? The SMC call with YIELD option will only be preemtible so will not affect the normal flow.
b)As quoted in Trusted Firmware-A Document[6] Page 94, "Yielding calls are reserved exclusively for Trusted OS providers" "Yielding 0- 1 Reserved for existing Armv7-A calls Yielding 2-63 Trusted OS Standard Calls"
Can this range be consumed within ATF/bl31 firmware or is it necessary to forward all yielding calls to Trusted OS?
[0]https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/st/stm... [1]https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/21840 [2]https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/optee/smc_abi.c... [3]https://github.com/OP-TEE/optee_os/blob/fc57019cb35c8c1bad66fc6d814ace5debde... [4]https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/optee/smc_abi.c... [5]https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon... [6]https://trustedfirmware-a.readthedocs.io/_/downloads/en/latest/pdf/
Regards, Kamlesh -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
Le 15 mai 2024 à 14:42, Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org a écrit :
Hi Achin,
Thanks for the suggestion,
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare-metal partition and does not require migrating the SCMI operations to a TEE.
As per [0], we can't have a SEL1 firmware at all(e.g. OP-TEE), if we run S-EL0 partition image.
"It is not currently possible for BL31 to integrate SPM support and a Secure Payload Dispatcher (SPD) at the same time; they are mutually exclusive.
It has been possible to run a TEE and UEFI crypto as a separate partition for UEFI SecureBoot with U-Boot. Adding a second partition may be complex but not impossible.
I added Ilias in the to: as he knows alls the details
In the SPM bootflow, a Secure Partition image executing at S-EL0 replaces the Secure Payload image executing at S-EL1 (e.g. a Trusted OS). Both are referred to as BL32."
Let me know if you think otherwise.
What we want is,
- Run clock and power management services as well as TEE services
- We don't want to put clock and power and management services in TEE,
so that we have liberty to use any TEE we want. Example, if we put our SCMI operations in OP-TEE, then we'll be constrained to use OP-TEE and only OP-TEE.
[0]https://trustedfirmware-a.readthedocs.io/en/latest/components/secure-partiti...
Regards, Kamlesh
Achin Gupta Achin.Gupta@arm.com writes:
Hi Kamlesh,
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware
parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare-metal partition and does not require migrating the SCMI operations to a TEE.
cheers, Achin
From: Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org Sent: 15 May 2024 10:33 To: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Cc: bb@ti.com bb@ti.com; vigneshr@ti.com vigneshr@ti.com; d-gole@ti.com d-gole@ti.com; afd@ti.com afd@ti.com; hnagalla@ti.com hnagalla@ti.com Subject: [TF-A] Make TF-A (bl31) (El3 firmware) preemtible
Clock and power management in ATF, preemption issue.
I have seen that multiple Vendors are using arm scmi protocol to do clock management in bl31[0][1].
The problem with such implementations is any long running SCMI operations (like PLL locking for example) will hold the core in EL3 for extended period of time, adding to latency of NS EL1 interrupt handling as ATF is not preemptible.
This problem can be solved by making ATF preemptible, similar to how OP-TEE does it by by implementing a remote procedural call after it receives a interrupt. [2]How Linux does SMC call with YIELD flag enabled. [3]How OP-TEE handles the timer interrupt [4]How Linux receives an interrupt which came in EL3->SEL1
As quoted in exception Handling document in ATF docs[5]
"Receive exceptions, but handle part of the exception in EL3, and delegate processing of the error to dedicated software stack running at lower secure ELs (as above); additionally, the Normal world may also be required to participate in the handling, or be notified of such events (for example, as an event). In this scheme, exception handling potentially and maximally spans all ELs in both Secure and Normal worlds."
From this we can understand that we can delegate the exception to EL1 if we are already in EL3. (way to preempt)
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
We don't want to put it in TEE as it will constraint our devices to that specific TEE.
I am starting to work on a proposal to make ATF preemptible similar to how OP-TEE is doing it. a)Will the similar approach from OP-TEE if implemented and working be accepted by ATF upstream? The SMC call with YIELD option will only be preemtible so will not affect the normal flow.
b)As quoted in Trusted Firmware-A Document[6] Page 94, "Yielding calls are reserved exclusively for Trusted OS providers" "Yielding 0- 1 Reserved for existing Armv7-A calls Yielding 2-63 Trusted OS Standard Calls"
Can this range be consumed within ATF/bl31 firmware or is it necessary to forward all yielding calls to Trusted OS?
[0]https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/st/stm... [1]https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/21840 [2]https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/optee/smc_abi.c... [3]https://github.com/OP-TEE/optee_os/blob/fc57019cb35c8c1bad66fc6d814ace5debde... [4]https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/optee/smc_abi.c... [5]https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon... [6]https://trustedfirmware-a.readthedocs.io/_/downloads/en/latest/pdf/
Regards, Kamlesh -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
Hi Kamlesh and Francois,
Please find my comments inline...
On Wed, 2024-05-15 at 13:16 +0000, ff via TF-A wrote:
Le 15 mai 2024 à 14:42, Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org a écrit :
Hi Achin,
Thanks for the suggestion,
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare- metal partition and does not require migrating the SCMI operations to a TEE.
As per [0], we can't have a SEL1 firmware at all(e.g. OP-TEE), if we run S-EL0 partition image.
"It is not currently possible for BL31 to integrate SPM support and a Secure Payload Dispatcher (SPD) at the same time; they are mutually exclusive.
It has been possible to run a TEE and UEFI crypto as a separate partition for UEFI SecureBoot with U-Boot. Adding a second partition may be complex but not impossible.
I really don't think we would want to support this upstream but I will let the TF-A team comment. It makes EL3 solve a problem that should be solved in the TEE. This goes against keeping EL3 simple and opens the door for more vendor specific firmware to creep into EL3.
The best bet is to get the TEE to support FF-A and migrate the SCMI operations to an FF-A S-EL0 SP. The SP would work with any TEE that has a compliant FF-A implementation. On a >= v8.4 system, the same SP would work with Hafnium. It will require some upfront investment but is the most scalable approach in the long term.
cheers, Achin
I added Ilias in the to: as he knows alls the details
In the SPM bootflow, a Secure Partition image executing at S-EL0 replaces the Secure Payload image executing at S-EL1 (e.g. a Trusted OS). Both are referred to as BL32."
Let me know if you think otherwise.
What we want is,
- Run clock and power management services as well as TEE services
- We don't want to put clock and power and management services in
TEE, so that we have liberty to use any TEE we want. Example, if we put our SCMI operations in OP-TEE, then we'll be constrained to use OP-TEE and only OP-TEE.
[0] https://trustedfirmware-a.readthedocs.io/en/latest/components/secur e-partition-manager-mm.html#secure-partition-manager-mm
Regards, Kamlesh
Achin Gupta Achin.Gupta@arm.com writes:
Hi Kamlesh,
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware
parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare-metal partition and does not require migrating the SCMI operations to a TEE.
cheers, Achin
From: Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org Sent: 15 May 2024 10:33 To: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Cc: bb@ti.com bb@ti.com; vigneshr@ti.com vigneshr@ti.com; d-gole@ti.com d-gole@ti.com; afd@ti.com afd@ti.com; hnagalla@ti.com hnagalla@ti.com Subject: [TF-A] Make TF-A (bl31) (El3 firmware) preemtible
Clock and power management in ATF, preemption issue.
I have seen that multiple Vendors are using arm scmi protocol to do clock management in bl31[0][1].
The problem with such implementations is any long running SCMI operations (like PLL locking for example) will hold the core in EL3 for extended period of time, adding to latency of NS EL1 interrupt handling as ATF is not preemptible.
This problem can be solved by making ATF preemptible, similar to how OP-TEE does it by by implementing a remote procedural call after it receives a interrupt. [2]How Linux does SMC call with YIELD flag enabled. [3]How OP-TEE handles the timer interrupt [4]How Linux receives an interrupt which came in EL3->SEL1
As quoted in exception Handling document in ATF docs[5]
"Receive exceptions, but handle part of the exception in EL3, and delegate processing of the error to dedicated software stack running at lower secure ELs (as above); additionally, the Normal world may also be required to participate in the handling, or be notified of such events (for example, as an event). In this scheme, exception handling potentially and maximally spans all ELs in both Secure and Normal worlds."
From this we can understand that we can delegate the exception to EL1 if we are already in EL3. (way to preempt)
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
We don't want to put it in TEE as it will constraint our devices to that specific TEE.
I am starting to work on a proposal to make ATF preemptible similar to how OP-TEE is doing it. a)Will the similar approach from OP-TEE if implemented and working be accepted by ATF upstream? The SMC call with YIELD option will only be preemtible so will not affect the normal flow.
b)As quoted in Trusted Firmware-A Document[6] Page 94, "Yielding calls are reserved exclusively for Trusted OS providers" "Yielding 0- 1 Reserved for existing Armv7-A calls Yielding 2-63 Trusted OS Standard Calls"
Can this range be consumed within ATF/bl31 firmware or is it necessary to forward all yielding calls to Trusted OS?
[0] https://github.com/ARM-software/arm-trusted-firmware/blob/master/ plat/st/stm32mp1/stm32mp1_scmi.c [1] https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/21 840 [2] https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/opte e/smc_abi.c#L930 [3] https://github.com/OP-TEE/optee_os/blob/fc57019cb35c8c1bad66fc6d8 14ace5debde170a/core/arch/arm/kernel/thread_optee_smc_a64.S#L246 [4] https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/opte e/smc_abi.c#L842 [5] https://github.com/ARM-software/arm-trusted-firmware/blob/master/ docs/components/exception-handling.rst [6] https://trustedfirmware-a.readthedocs.io/_/downloads/en/latest/pd f/
Regards, Kamlesh -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
Hi Kamlesh,
Maybe a simpler way out would be to implement cooperative yielding semantics in the SCMI logic in EL3 such that:
1. SCMI SMC comes in from Normal world. 2. SCMI command is sent to the SCP. 3. EL3 SCMI logic starts a counter and polls for a response. 4. When the counter hits 0, if the command has not completed and there is a pending interrupt, complete the SMC to return to the Normal world. 5. Normal world resumes the SMC. 6. Goto 3 unless the command has completed.
Yet another option would be to poll in the Normal world i.e. return immediately to Normal world after 2 above and then invoke an SMC repeatedly to check if the command has completed.
Just thinking aloud!
cheers, Achin
On Wed, 2024-05-15 at 13:48 +0000, Achin Gupta via TF-A wrote:
Hi Kamlesh and Francois,
Please find my comments inline...
On Wed, 2024-05-15 at 13:16 +0000, ff via TF-A wrote:
Le 15 mai 2024 à 14:42, Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org a écrit :
Hi Achin,
Thanks for the suggestion,
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare- metal partition and does not require migrating the SCMI operations to a TEE.
As per [0], we can't have a SEL1 firmware at all(e.g. OP-TEE), if we run S-EL0 partition image.
"It is not currently possible for BL31 to integrate SPM support and a Secure Payload Dispatcher (SPD) at the same time; they are mutually exclusive.
It has been possible to run a TEE and UEFI crypto as a separate partition for UEFI SecureBoot with U-Boot. Adding a second partition may be complex but not impossible.
I really don't think we would want to support this upstream but I will let the TF-A team comment. It makes EL3 solve a problem that should be solved in the TEE. This goes against keeping EL3 simple and opens the door for more vendor specific firmware to creep into EL3.
The best bet is to get the TEE to support FF-A and migrate the SCMI operations to an FF-A S-EL0 SP. The SP would work with any TEE that has a compliant FF-A implementation. On a >= v8.4 system, the same SP would work with Hafnium. It will require some upfront investment but is the most scalable approach in the long term.
cheers, Achin
I added Ilias in the to: as he knows alls the details
In the SPM bootflow, a Secure Partition image executing at S-EL0 replaces the Secure Payload image executing at S-EL1 (e.g. a Trusted OS). Both are referred to as BL32."
Let me know if you think otherwise.
What we want is,
- Run clock and power management services as well as TEE
services 2. We don't want to put clock and power and management services in TEE, so that we have liberty to use any TEE we want. Example, if we put our SCMI operations in OP-TEE, then we'll be constrained to use OP-TEE and only OP-TEE.
[0] https://trustedfirmware-a.readthedocs.io/en/latest/components/secur e-partition-manager-mm.html#secure-partition-manager-mm
Regards, Kamlesh
Achin Gupta Achin.Gupta@arm.com writes:
Hi Kamlesh,
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware
parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare-metal partition and does not require migrating the SCMI operations to a TEE.
cheers, Achin
From: Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org Sent: 15 May 2024 10:33 To: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Cc: bb@ti.com bb@ti.com; vigneshr@ti.com vigneshr@ti.com; d-gole@ti.com d-gole@ti.com; afd@ti.com afd@ti.com; hnagalla@ti.com hnagalla@ti.com Subject: [TF-A] Make TF-A (bl31) (El3 firmware) preemtible
Clock and power management in ATF, preemption issue.
I have seen that multiple Vendors are using arm scmi protocol to do clock management in bl31[0][1].
The problem with such implementations is any long running SCMI operations (like PLL locking for example) will hold the core in EL3 for extended period of time, adding to latency of NS EL1 interrupt handling as ATF is not preemptible.
This problem can be solved by making ATF preemptible, similar to how OP-TEE does it by by implementing a remote procedural call after it receives a interrupt. [2]How Linux does SMC call with YIELD flag enabled. [3]How OP-TEE handles the timer interrupt [4]How Linux receives an interrupt which came in EL3->SEL1
As quoted in exception Handling document in ATF docs[5]
"Receive exceptions, but handle part of the exception in EL3, and delegate processing of the error to dedicated software stack running at lower secure ELs (as above); additionally, the Normal world may also be required to participate in the handling, or be notified of such events (for example, as an event). In this scheme, exception handling potentially and maximally spans all ELs in both Secure and Normal worlds."
From this we can understand that we can delegate the exception to EL1 if we are already in EL3. (way to preempt)
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
We don't want to put it in TEE as it will constraint our devices to that specific TEE.
I am starting to work on a proposal to make ATF preemptible similar to how OP-TEE is doing it. a)Will the similar approach from OP-TEE if implemented and working be accepted by ATF upstream? The SMC call with YIELD option will only be preemtible so will not affect the normal flow.
b)As quoted in Trusted Firmware-A Document[6] Page 94, "Yielding calls are reserved exclusively for Trusted OS providers" "Yielding 0- 1 Reserved for existing Armv7-A calls Yielding 2-63 Trusted OS Standard Calls"
Can this range be consumed within ATF/bl31 firmware or is it necessary to forward all yielding calls to Trusted OS?
[0] https://github.com/ARM-software/arm-trusted-firmware/blob/master/ plat/st/stm32mp1/stm32mp1_scmi.c [1] https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/21 840 [2] https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/opte e/smc_abi.c#L930 [3] https://github.com/OP-TEE/optee_os/blob/fc57019cb35c8c1bad66fc6d8 14ace5debde170a/core/arch/arm/kernel/thread_optee_smc_a64.S#L24 6 [4] https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/opte e/smc_abi.c#L842 [5] https://github.com/ARM-software/arm-trusted-firmware/blob/master/ docs/components/exception-handling.rst [6] https://trustedfirmware-a.readthedocs.io/_/downloads/en/latest/pd f/
Regards, Kamlesh -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
On 5/15/24 10:09 AM, Achin Gupta wrote:
Hi Kamlesh,
Maybe a simpler way out would be to implement cooperative yielding semantics in the SCMI logic in EL3 such that:
We had also though about something like this, would require the addition of a some async versions of the standard SCMI calls. Or have the normal calls return some new return value like "retry" or "check back later" instead of pass/fail.
- SCMI SMC comes in from Normal world.
- SCMI command is sent to the SCP.
- EL3 SCMI logic starts a counter and polls for a response.
- When the counter hits 0, if the command has not completed and there
is a pending interrupt, complete the SMC to return to the Normal world.> 5. Normal world resumes the SMC. 6. Goto 3 unless the command has completed.
Yet another option would be to poll in the Normal world i.e. return immediately to Normal world after 2 above and then invoke an SMC repeatedly to check if the command has completed.
We would want to just return back to normal world immediately for anything other than the most simple calls for best RT performance. Otherwise we would need some policy on what the timeout would be, so that should be avoided.
Andrew
Just thinking aloud!
cheers, Achin
On Wed, 2024-05-15 at 13:48 +0000, Achin Gupta via TF-A wrote:
Hi Kamlesh and Francois,
Please find my comments inline...
On Wed, 2024-05-15 at 13:16 +0000, ff via TF-A wrote:
Le 15 mai 2024 à 14:42, Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org a écrit :
Hi Achin,
Thanks for the suggestion,
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare- metal partition and does not require migrating the SCMI operations to a TEE.
As per [0], we can't have a SEL1 firmware at all(e.g. OP-TEE), if we run S-EL0 partition image.
"It is not currently possible for BL31 to integrate SPM support and a Secure Payload Dispatcher (SPD) at the same time; they are mutually exclusive.
It has been possible to run a TEE and UEFI crypto as a separate partition for UEFI SecureBoot with U-Boot. Adding a second partition may be complex but not impossible.
I really don't think we would want to support this upstream but I will let the TF-A team comment. It makes EL3 solve a problem that should be solved in the TEE. This goes against keeping EL3 simple and opens the door for more vendor specific firmware to creep into EL3.
The best bet is to get the TEE to support FF-A and migrate the SCMI operations to an FF-A S-EL0 SP. The SP would work with any TEE that has a compliant FF-A implementation. On a >= v8.4 system, the same SP would work with Hafnium. It will require some upfront investment but is the most scalable approach in the long term.
cheers, Achin
I added Ilias in the to: as he knows alls the details
In the SPM bootflow, a Secure Partition image executing at S-EL0 replaces the Secure Payload image executing at S-EL1 (e.g. a Trusted OS). Both are referred to as BL32."
Let me know if you think otherwise.
What we want is,
- Run clock and power management services as well as TEE
services 2. We don't want to put clock and power and management services in TEE, so that we have liberty to use any TEE we want. Example, if we put our SCMI operations in OP-TEE, then we'll be constrained to use OP-TEE and only OP-TEE.
[0] https://trustedfirmware-a.readthedocs.io/en/latest/components/secur e-partition-manager-mm.html#secure-partition-manager-mm
Regards, Kamlesh
Achin Gupta Achin.Gupta@arm.com writes:
Hi Kamlesh,
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware
parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare-metal partition and does not require migrating the SCMI operations to a TEE.
cheers, Achin
From: Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org Sent: 15 May 2024 10:33 To: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Cc: bb@ti.com bb@ti.com; vigneshr@ti.com vigneshr@ti.com; d-gole@ti.com d-gole@ti.com; afd@ti.com afd@ti.com; hnagalla@ti.com hnagalla@ti.com Subject: [TF-A] Make TF-A (bl31) (El3 firmware) preemtible
Clock and power management in ATF, preemption issue.
I have seen that multiple Vendors are using arm scmi protocol to do clock management in bl31[0][1].
The problem with such implementations is any long running SCMI operations (like PLL locking for example) will hold the core in EL3 for extended period of time, adding to latency of NS EL1 interrupt handling as ATF is not preemptible.
This problem can be solved by making ATF preemptible, similar to how OP-TEE does it by by implementing a remote procedural call after it receives a interrupt. [2]How Linux does SMC call with YIELD flag enabled. [3]How OP-TEE handles the timer interrupt [4]How Linux receives an interrupt which came in EL3->SEL1
As quoted in exception Handling document in ATF docs[5]
"Receive exceptions, but handle part of the exception in EL3, and delegate processing of the error to dedicated software stack running at lower secure ELs (as above); additionally, the Normal world may also be required to participate in the handling, or be notified of such events (for example, as an event). In this scheme, exception handling potentially and maximally spans all ELs in both Secure and Normal worlds."
From this we can understand that we can delegate the exception to EL1 if we are already in EL3. (way to preempt)
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
We don't want to put it in TEE as it will constraint our devices to that specific TEE.
I am starting to work on a proposal to make ATF preemptible similar to how OP-TEE is doing it. a)Will the similar approach from OP-TEE if implemented and working be accepted by ATF upstream? The SMC call with YIELD option will only be preemtible so will not affect the normal flow.
b)As quoted in Trusted Firmware-A Document[6] Page 94, "Yielding calls are reserved exclusively for Trusted OS providers" "Yielding 0- 1 Reserved for existing Armv7-A calls Yielding 2-63 Trusted OS Standard Calls"
Can this range be consumed within ATF/bl31 firmware or is it necessary to forward all yielding calls to Trusted OS?
[0] https://github.com/ARM-software/arm-trusted-firmware/blob/master/ plat/st/stm32mp1/stm32mp1_scmi.c [1] https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/21 840 [2] https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/opte e/smc_abi.c#L930 [3] https://github.com/OP-TEE/optee_os/blob/fc57019cb35c8c1bad66fc6d8 14ace5debde170a/core/arch/arm/kernel/thread_optee_smc_a64.S#L24 6 [4] https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/opte e/smc_abi.c#L842 [5] https://github.com/ARM-software/arm-trusted-firmware/blob/master/ docs/components/exception-handling.rst [6] https://trustedfirmware-a.readthedocs.io/_/downloads/en/latest/pd f/
Regards, Kamlesh -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
Hi Everyone,
Generally making EL3 pre-emptible and returning to NS is a hard problem to solve. TF-A does allow SError (controlled via PSTATE.A) to be taken during EL3 execution, but the end result is a panic() or system shutdown with no prospects of resuming execution.
If EL3 firmware were to return back to NS world, it would have to be prepared to re-enter via a different entry point or a different SMC. This is unlike an OS handling an interrupt. AFAICS, To allow EL3 to be re-entrant via another entrypoint while having been pre-empted by a NS interrupt, EL3 would need to execute SCMI calls in separate C execution context (with its own stack and register context). Perhaps another useful way of thinking about this is that the SCMI SMC is run as a separate thread within EL3 firmware.
This separate C context can be pre-empted , stashed and restored appropriately as required.
This is doable, but given the tight memory constraints for EL3 firmware, it may not be possible to this is as a generic feature and not all SMCs can brought into this fold as would need special error codes to notify the caller about pre-emption , whether thread limit is hit etc.
What Achin suggested seems to be the straightforward way to implement this. The SCMI SMC calls would certainly need to be modified (as you mention) to cater for suspension and resumption . This would be needed even if EL3 were to implement pre-emption as once execution is returned back to NS, EL3 does not receive control back and NS caller would need to invoke the SMC again.
To mitigate concerns on RT performance, I can suggest another variation on Achin's proposal. Within the poll() loop, the code could either :
1. check for a pending interrupt by reading isr_el1 and return early if there is a NS interrupt pending. 2. A bit more risky, but could unmask PSTATE.I within the poll() loop and if interrupt handler is triggered, can signal via a flag, ensure that PSTATE.I is masked again and break out of the loop.
Best Regards Soby Mathew
-----Original Message----- From: Andrew Davis via TF-A tf-a@lists.trustedfirmware.org Sent: Wednesday, May 15, 2024 4:37 PM To: Achin Gupta Achin.Gupta@arm.com; ilias.apalodimas@linaro.org; kamlesh@ti.com; ff@shokubai.tech Cc: bb@ti.com; vigneshr@ti.com; tf-a@lists.trustedfirmware.org; d- gole@ti.com; hnagalla@ti.com Subject: [TF-A] Re: Make TF-A (bl31) (El3 firmware) preemtible
On 5/15/24 10:09 AM, Achin Gupta wrote:
Hi Kamlesh,
Maybe a simpler way out would be to implement cooperative yielding semantics in the SCMI logic in EL3 such that:
We had also though about something like this, would require the addition of a some async versions of the standard SCMI calls. Or have the normal calls return some new return value like "retry" or "check back later" instead of pass/fail.
- SCMI SMC comes in from Normal world.
- SCMI command is sent to the SCP.
- EL3 SCMI logic starts a counter and polls for a response.
- When the counter hits 0, if the command has not completed and there
is a pending interrupt, complete the SMC to return to the Normal world.>
- Normal world resumes the SMC.
- Goto 3 unless the command has completed.
Yet another option would be to poll in the Normal world i.e. return immediately to Normal world after 2 above and then invoke an SMC repeatedly to check if the command has completed.
We would want to just return back to normal world immediately for anything other than the most simple calls for best RT performance. Otherwise we would need some policy on what the timeout would be, so that should be avoided.
Andrew
Just thinking aloud!
cheers, Achin
On Wed, 2024-05-15 at 13:48 +0000, Achin Gupta via TF-A wrote:
Hi Kamlesh and Francois,
Please find my comments inline...
On Wed, 2024-05-15 at 13:16 +0000, ff via TF-A wrote:
Le 15 mai 2024 à 14:42, Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org a écrit :
Hi Achin,
Thanks for the suggestion,
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare- metal partition and does not require migrating the SCMI operations to a TEE.
As per [0], we can't have a SEL1 firmware at all(e.g. OP-TEE), if we run S-EL0 partition image.
"It is not currently possible for BL31 to integrate SPM support and a Secure Payload Dispatcher (SPD) at the same time; they are mutually exclusive.
It has been possible to run a TEE and UEFI crypto as a separate partition for UEFI SecureBoot with U-Boot. Adding a second partition may be complex but not impossible.
I really don't think we would want to support this upstream but I will let the TF-A team comment. It makes EL3 solve a problem that should be solved in the TEE. This goes against keeping EL3 simple and opens the door for more vendor specific firmware to creep into EL3.
The best bet is to get the TEE to support FF-A and migrate the SCMI operations to an FF-A S-EL0 SP. The SP would work with any TEE that has a compliant FF-A implementation. On a >= v8.4 system, the same SP would work with Hafnium. It will require some upfront investment but is the most scalable approach in the long term.
cheers, Achin
I added Ilias in the to: as he knows alls the details
In the SPM bootflow, a Secure Partition image executing at S-EL0 replaces the Secure Payload image executing at S-EL1 (e.g. a Trusted OS). Both are referred to as BL32."
Let me know if you think otherwise.
What we want is,
- Run clock and power management services as well as TEE services
- We don't want to put clock and power and management services in
TEE, so that we have liberty to use any TEE we want. Example, if we put our SCMI operations in OP-TEE, then we'll be constrained to use OP-TEE and only OP-TEE.
[0] https://trustedfirmware-a.readthedocs.io/en/latest/components/secur e-partition-manager-mm.html#secure-partition-manager-mm
Regards, Kamlesh
Achin Gupta Achin.Gupta@arm.com writes:
Hi Kamlesh,
> We are aware that ARM v8.4-A onwards arch are having a SEL2 level > with which we can run a vendor specific firmware parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare-metal partition and does not require migrating the SCMI operations to a TEE.
cheers, Achin
From: Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org Sent: 15 May 2024 10:33 To: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Cc: bb@ti.com bb@ti.com; vigneshr@ti.com vigneshr@ti.com; d-gole@ti.com d-gole@ti.com; afd@ti.com afd@ti.com; hnagalla@ti.com hnagalla@ti.com Subject: [TF-A] Make TF-A (bl31) (El3 firmware) preemtible
Clock and power management in ATF, preemption issue.
I have seen that multiple Vendors are using arm scmi protocol to do clock management in bl31[0][1].
The problem with such implementations is any long running SCMI operations (like PLL locking for example) will hold the core in EL3 for extended period of time, adding to latency of NS EL1 interrupt handling as ATF is not preemptible.
This problem can be solved by making ATF preemptible, similar to how OP-TEE does it by by implementing a remote procedural call after it receives a interrupt. [2]How Linux does SMC call with YIELD flag enabled. [3]How OP-TEE handles the timer interrupt [4]How Linux receives an interrupt which came in EL3->SEL1
As quoted in exception Handling document in ATF docs[5]
"Receive exceptions, but handle part of the exception in EL3, and delegate processing of the error to dedicated software stack running at lower secure ELs (as above); additionally, the Normal world may also be required to participate in the handling, or be notified of such events (for example, as an event). In this scheme, exception handling potentially and maximally spans all ELs in both Secure and Normal worlds."
From this we can understand that we can delegate the exception to EL1 if we are already in EL3. (way to preempt)
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
We don't want to put it in TEE as it will constraint our devices to that specific TEE.
I am starting to work on a proposal to make ATF preemptible similar to how OP-TEE is doing it. a)Will the similar approach from OP-TEE if implemented and working be accepted by ATF upstream? The SMC call with YIELD option will only be preemtible so will not affect the normal flow.
b)As quoted in Trusted Firmware-A Document[6] Page 94, "Yielding calls are reserved exclusively for Trusted OS providers" "Yielding 0- 1 Reserved for existing Armv7-A calls Yielding 2-63 Trusted OS Standard Calls"
Can this range be consumed within ATF/bl31 firmware or is it necessary to forward all yielding calls to Trusted OS?
[0] https://github.com/ARM-software/arm-trusted-firmware/blob/master/ plat/st/stm32mp1/stm32mp1_scmi.c [1] https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/21 840 [2] https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/opte e/smc_abi.c#L930 [3] https://github.com/OP-
TEE/optee_os/blob/fc57019cb35c8c1bad66fc6d8
14ace5debde170a/core/arch/arm/kernel/thread_optee_smc_a64.S#L24 6 [4] https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/opte e/smc_abi.c#L842 [5] https://github.com/ARM-software/arm-trusted-firmware/blob/master/ docs/components/exception-handling.rst [6] https://trustedfirmware-a.readthedocs.io/_/downloads/en/latest/pd f/
Regards, Kamlesh -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
Soby Mathew Soby.Mathew@arm.com writes:
++arm-scmi@vger.kernel.org list,
Thanks for the suggestions Achin And Soby,
Hi Everyone,
Generally making EL3 pre-emptible and returning to NS is a hard problem to solve. TF-A does allow SError (controlled via PSTATE.A) to be taken during EL3 execution, but the end result is a panic() or system shutdown with no prospects of resuming execution.
If EL3 firmware were to return back to NS world, it would have to be prepared to re-enter via a different entry point or a different SMC. This is unlike an OS handling an interrupt. AFAICS, To allow EL3 to be re-entrant via another entrypoint while having been pre-empted by a NS interrupt, EL3 would need to execute SCMI calls in separate C execution context (with its own stack and register context). Perhaps another useful way of thinking about this is that the SCMI SMC is run as a separate thread within EL3 firmware.
This separate C context can be pre-empted , stashed and restored appropriately as required.
This is doable, but given the tight memory constraints for EL3 firmware, it may not be possible to this is as a generic feature and not all SMCs can brought into this fold as would need special error codes to notify the caller about pre-emption , whether thread limit is hit etc.
What Achin suggested seems to be the straightforward way to implement this. The SCMI SMC calls would certainly need to be modified (as you mention) to cater for suspension and resumption . This would be needed even if EL3 were to implement pre-emption as once execution is returned back to NS, EL3 does not receive control back and NS caller would need to invoke the SMC again.
To mitigate concerns on RT performance, I can suggest another variation on Achin's proposal. Within the poll() loop, the code could either :
- check for a pending interrupt by reading isr_el1 and return early if there is a NS interrupt pending.
- A bit more risky, but could unmask PSTATE.I within the poll() loop and if interrupt handler is triggered, can signal via a flag, ensure that PSTATE.I is masked again and break out of the loop.
If I summarise this, 1. Problem is specific to SCMI, so keep it constrained to SCMI.
2. Make SCMI calls preemtible like following as per Achin's suggestion,
1. SCMI SMC comes in from Normal world. 2. SCMI command is sent to the SCP. 3. EL3 SCMI logic starts a counter and polls for a response. 4. When the counter hits 0, if the command has not completed and there is a pending interrupt, complete the SMC to return to the Normal world. 5. Normal world resumes the SMC. 6. Goto 3 unless the command has completed.
This is doable with addition of RETRY(preempted case) return value from SCMI calls along with SUCCESS and FAILURE return values.
Also, we'll have to make the SCMI calls re-entrant in a way that they are smart enough to pick up from where they got preempted last time.
will give us best result with what Soby suggested for polling
A bit more risky, but could unmask PSTATE.I within the poll() loop and if interrupt handler is triggered, can signal via a flag, ensure that PSTATE.I is masked again and break out of the loop.
Soby,
Could you please elaborate more on why you feel that it is "bit more risky", other than the specifics of implementation.
Kamlesh
Best Regards Soby Mathew
-----Original Message----- From: Andrew Davis via TF-A tf-a@lists.trustedfirmware.org Sent: Wednesday, May 15, 2024 4:37 PM To: Achin Gupta Achin.Gupta@arm.com; ilias.apalodimas@linaro.org; kamlesh@ti.com; ff@shokubai.tech Cc: bb@ti.com; vigneshr@ti.com; tf-a@lists.trustedfirmware.org; d- gole@ti.com; hnagalla@ti.com Subject: [TF-A] Re: Make TF-A (bl31) (El3 firmware) preemtible
On 5/15/24 10:09 AM, Achin Gupta wrote:
Hi Kamlesh,
Maybe a simpler way out would be to implement cooperative yielding semantics in the SCMI logic in EL3 such that:
We had also though about something like this, would require the addition of a some async versions of the standard SCMI calls. Or have the normal calls return some new return value like "retry" or "check back later" instead of pass/fail.
- SCMI SMC comes in from Normal world.
- SCMI command is sent to the SCP.
- EL3 SCMI logic starts a counter and polls for a response.
- When the counter hits 0, if the command has not completed and there
is a pending interrupt, complete the SMC to return to the Normal world.>
- Normal world resumes the SMC.
- Goto 3 unless the command has completed.
Yet another option would be to poll in the Normal world i.e. return immediately to Normal world after 2 above and then invoke an SMC repeatedly to check if the command has completed.
We would want to just return back to normal world immediately for anything other than the most simple calls for best RT performance. Otherwise we would need some policy on what the timeout would be, so that should be avoided.
Andrew
Just thinking aloud!
cheers, Achin
On Wed, 2024-05-15 at 13:48 +0000, Achin Gupta via TF-A wrote:
Hi Kamlesh and Francois,
Please find my comments inline...
On Wed, 2024-05-15 at 13:16 +0000, ff via TF-A wrote:
Le 15 mai 2024 à 14:42, Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org a écrit :
Hi Achin,
Thanks for the suggestion,
> Instead of adding the complexity of yielding semantics to BL31, > have you considered migrating the SCMI operations to a S-EL0 > partition that is managed by EL3 firmware? This would be a > preemptible bare- metal partition and does not require migrating > the SCMI operations to a TEE.
As per [0], we can't have a SEL1 firmware at all(e.g. OP-TEE), if we run S-EL0 partition image.
"It is not currently possible for BL31 to integrate SPM support and a Secure Payload Dispatcher (SPD) at the same time; they are mutually exclusive.
It has been possible to run a TEE and UEFI crypto as a separate partition for UEFI SecureBoot with U-Boot. Adding a second partition may be complex but not impossible.
I really don't think we would want to support this upstream but I will let the TF-A team comment. It makes EL3 solve a problem that should be solved in the TEE. This goes against keeping EL3 simple and opens the door for more vendor specific firmware to creep into EL3.
The best bet is to get the TEE to support FF-A and migrate the SCMI operations to an FF-A S-EL0 SP. The SP would work with any TEE that has a compliant FF-A implementation. On a >= v8.4 system, the same SP would work with Hafnium. It will require some upfront investment but is the most scalable approach in the long term.
cheers, Achin
I added Ilias in the to: as he knows alls the details
In the SPM bootflow, a Secure Partition image executing at S-EL0 replaces the Secure Payload image executing at S-EL1 (e.g. a Trusted OS). Both are referred to as BL32."
Let me know if you think otherwise.
What we want is,
- Run clock and power management services as well as TEE services
- We don't want to put clock and power and management services in
TEE, so that we have liberty to use any TEE we want. Example, if we put our SCMI operations in OP-TEE, then we'll be constrained to use OP-TEE and only OP-TEE.
[0] https://trustedfirmware-a.readthedocs.io/en/latest/components/secur e-partition-manager-mm.html#secure-partition-manager-mm
Regards, Kamlesh
Achin Gupta Achin.Gupta@arm.com writes:
> Hi Kamlesh, > >> We are aware that ARM v8.4-A onwards arch are having a SEL2 level >> with which we can run a vendor specific firmware > parallel to TEE at SEL1 in a secure partition. But how we do > handle clock and power management for version which doesn't > support SEL2? > > Instead of adding the complexity of yielding semantics to BL31, > have you considered migrating the SCMI operations to a S-EL0 > partition that is managed by EL3 firmware? This would be a > preemptible bare-metal partition and does not require migrating > the SCMI operations to a TEE. > > cheers, > Achin > > ________________________________ > From: Kamlesh Gurudasani via TF-A > tf-a@lists.trustedfirmware.org > Sent: 15 May 2024 10:33 > To: > tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org > Cc: bb@ti.com bb@ti.com; vigneshr@ti.com vigneshr@ti.com; > d-gole@ti.com d-gole@ti.com; afd@ti.com afd@ti.com; > hnagalla@ti.com hnagalla@ti.com > Subject: [TF-A] Make TF-A (bl31) (El3 firmware) preemtible > > Clock and power management in ATF, preemption issue. > > I have seen that multiple Vendors are using arm scmi protocol to > do clock management in bl31[0][1]. > > The problem with such implementations is any long running SCMI > operations (like PLL locking for example) will hold the core in > EL3 for extended period of time, adding to latency of NS EL1 > interrupt handling as ATF is not preemptible. > > This problem can be solved by making ATF preemptible, similar to > how OP-TEE does it by by implementing a remote procedural call > after it receives a interrupt. > [2]How Linux does SMC call with YIELD flag enabled. > [3]How OP-TEE handles the timer interrupt [4]How Linux receives an > interrupt which came in EL3->SEL1 > > As quoted in exception Handling document in ATF docs[5] > > "Receive exceptions, but handle part of the exception in EL3, and > delegate processing of the error to dedicated software stack > running at lower secure ELs (as above); additionally, the Normal > world may also be required to participate in the handling, or be > notified of such events (for example, as an event). > In this scheme, exception handling potentially and maximally spans > all ELs in both Secure and Normal worlds." > > From this we can understand that we can delegate the exception to > EL1 if we are already in EL3. > (way to preempt) > > We are aware that ARM v8.4-A onwards arch are having a SEL2 level > with which we can run a vendor specific firmware parallel to TEE > at SEL1 in a secure partition. But how we do handle clock and > power management for version which doesn't support SEL2? > > We don't want to put it in TEE as it will constraint our devices > to that specific TEE. > > I am starting to work on a proposal to make ATF preemptible > similar to how OP-TEE is doing it. > a)Will the similar approach from OP-TEE if implemented and working > be accepted by ATF upstream? > The SMC call with YIELD option will only be preemtible so will not > affect the normal flow. > > b)As quoted in Trusted Firmware-A Document[6] Page 94, "Yielding > calls are reserved exclusively for Trusted OS providers" > "Yielding 0- 1 Reserved for existing Armv7-A calls Yielding 2-63 > Trusted OS Standard Calls" > > Can this range be consumed within ATF/bl31 firmware or is it > necessary to forward all yielding calls to Trusted OS? > > [0] > https://github.com/ARM-software/arm-trusted-firmware/blob/master/ > plat/st/stm32mp1/stm32mp1_scmi.c > [1] > https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/21 > 840 > [2] > https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/opte > e/smc_abi.c#L930 > [3] > https://github.com/OP-
TEE/optee_os/blob/fc57019cb35c8c1bad66fc6d8
> 14ace5debde170a/core/arch/arm/kernel/thread_optee_smc_a64.S#L24 > 6 > [4] > https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/opte > e/smc_abi.c#L842 > [5] > https://github.com/ARM-software/arm-trusted-firmware/blob/master/ > docs/components/exception-handling.rst > [6] > https://trustedfirmware-a.readthedocs.io/_/downloads/en/latest/pd > f/ > > Regards, > Kamlesh > -- > TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe
> send an email to tf-a-leave@lists.trustedfirmware.org
TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
Hi Kamlesh
If I summarise this,
Problem is specific to SCMI, so keep it constrained to SCMI.
Make SCMI calls preemtible like following as per Achin's suggestion,
- SCMI SMC comes in from Normal world.
- SCMI command is sent to the SCP.
- EL3 SCMI logic starts a counter and polls for a response.
- When the counter hits 0, if the command has not completed and there
is a pending interrupt, complete the SMC to return to the Normal world. 5. Normal world resumes the SMC. 6. Goto 3 unless the command has completed.
This is doable with addition of RETRY(preempted case) return value from SCMI calls along with SUCCESS and FAILURE return values.
Also, we'll have to make the SCMI calls re-entrant in a way that they are smart enough to pick up from where they got preempted last time.
To yield immediately on a pending interrupt, my suggestion was to modify the sequence at step 3 :
3. EL3 SCMI reads isr_el1 sysreg for any pending interrupt while polling for response. 4. If interrupt pending, complete SMC and return to Normal world
This way the yield is more responsive on a pending interrupt. You could incorporate a poll counter within this pool loop for other reasons (like SCMI failing to respond in a timely manner thus giving the NS world an option to reschedule ).
Soby,
Could you please elaborate more on why you feel that it is "bit more risky", other than the specifics of implementation.
Kamlesh
I was referring to the implementation specifics wherein there are race conditions which need to be catered which includes PSTATE.I manipulation , the fact that the handler can be invoked multiple times if multiple NS interrupts are pended in increasing order of priorities etc. It gives the NS world a window to exploit in case there are any race conditions missed or not handled properly at EL3.
Also note that this handler must not ACK the interrupt to avoid altering the GIC Interrupt state for that pended interrupt.
Best Regards Soby Mathew
-----Original Message----- From: Kamlesh Gurudasani kamlesh@ti.com Sent: Monday, May 20, 2024 1:34 PM To: Soby Mathew Soby.Mathew@arm.com; Andrew Davis afd@ti.com; Achin Gupta Achin.Gupta@arm.com; ilias.apalodimas@linaro.org; ff@shokubai.tech; arm-scmi@vger.kernel.org Cc: bb@ti.com; vigneshr@ti.com; tf-a@lists.trustedfirmware.org; d- gole@ti.com; hnagalla@ti.com Subject: RE: [TF-A] Re: Make TF-A (bl31) (El3 firmware) preemtible
Soby Mathew Soby.Mathew@arm.com writes:
++arm-scmi@vger.kernel.org list,
Thanks for the suggestions Achin And Soby,
Hi Everyone,
Generally making EL3 pre-emptible and returning to NS is a hard problem to
solve. TF-A does allow SError (controlled via PSTATE.A) to be taken during EL3 execution, but the end result is a panic() or system shutdown with no prospects of resuming execution.
If EL3 firmware were to return back to NS world, it would have to be prepared
to re-enter via a different entry point or a different SMC. This is unlike an OS handling an interrupt. AFAICS, To allow EL3 to be re-entrant via another entrypoint while having been pre-empted by a NS interrupt, EL3 would need to execute SCMI calls in separate C execution context (with its own stack and register context). Perhaps another useful way of thinking about this is that the SCMI SMC is run as a separate thread within EL3 firmware.
This separate C context can be pre-empted , stashed and restored
appropriately as required.
This is doable, but given the tight memory constraints for EL3 firmware, it
may not be possible to this is as a generic feature and not all SMCs can brought into this fold as would need special error codes to notify the caller about pre- emption , whether thread limit is hit etc.
What Achin suggested seems to be the straightforward way to implement this.
The SCMI SMC calls would certainly need to be modified (as you mention) to cater for suspension and resumption . This would be needed even if EL3 were to implement pre-emption as once execution is returned back to NS, EL3 does not receive control back and NS caller would need to invoke the SMC again.
To mitigate concerns on RT performance, I can suggest another variation on
Achin's proposal. Within the poll() loop, the code could either :
- check for a pending interrupt by reading isr_el1 and return early if there is
a NS interrupt pending.
- A bit more risky, but could unmask PSTATE.I within the poll() loop and if
interrupt handler is triggered, can signal via a flag, ensure that PSTATE.I is masked again and break out of the loop.
If I summarise this,
Problem is specific to SCMI, so keep it constrained to SCMI.
Make SCMI calls preemtible like following as per Achin's suggestion,
- SCMI SMC comes in from Normal world.
- SCMI command is sent to the SCP.
- EL3 SCMI logic starts a counter and polls for a response.
- When the counter hits 0, if the command has not completed and there
is a pending interrupt, complete the SMC to return to the Normal world. 5. Normal world resumes the SMC. 6. Goto 3 unless the command has completed.
This is doable with addition of RETRY(preempted case) return value from SCMI calls along with SUCCESS and FAILURE return values.
Also, we'll have to make the SCMI calls re-entrant in a way that they are smart enough to pick up from where they got preempted last time.
will give us best result with what Soby suggested for polling
A bit more risky, but could unmask PSTATE.I within the poll() loop and if
interrupt handler is triggered, can signal
via a flag, ensure that PSTATE.I is masked again and break out of the loop.Soby,
Could you please elaborate more on why you feel that it is "bit more risky", other than the specifics of implementation.
Kamlesh
Best Regards Soby Mathew
-----Original Message----- From: Andrew Davis via TF-A tf-a@lists.trustedfirmware.org Sent: Wednesday, May 15, 2024 4:37 PM To: Achin Gupta Achin.Gupta@arm.com; ilias.apalodimas@linaro.org; kamlesh@ti.com; ff@shokubai.tech Cc: bb@ti.com; vigneshr@ti.com; tf-a@lists.trustedfirmware.org; d- gole@ti.com; hnagalla@ti.com Subject: [TF-A] Re: Make TF-A (bl31) (El3 firmware) preemtible
On 5/15/24 10:09 AM, Achin Gupta wrote:
Hi Kamlesh,
Maybe a simpler way out would be to implement cooperative yielding semantics in the SCMI logic in EL3 such that:
We had also though about something like this, would require the addition of a some async versions of the standard SCMI calls. Or have the normal calls return some new return value like "retry" or "check back later" instead of pass/fail.
- SCMI SMC comes in from Normal world.
- SCMI command is sent to the SCP.
- EL3 SCMI logic starts a counter and polls for a response.
- When the counter hits 0, if the command has not completed and
there is a pending interrupt, complete the SMC to return to the Normal world.>
- Normal world resumes the SMC.
- Goto 3 unless the command has completed.
Yet another option would be to poll in the Normal world i.e. return immediately to Normal world after 2 above and then invoke an SMC repeatedly to check if the command has completed.
We would want to just return back to normal world immediately for anything other than the most simple calls for best RT performance. Otherwise we would need some policy on what the timeout would be, so that should be avoided.
Andrew
Just thinking aloud!
cheers, Achin
On Wed, 2024-05-15 at 13:48 +0000, Achin Gupta via TF-A wrote:
Hi Kamlesh and Francois,
Please find my comments inline...
On Wed, 2024-05-15 at 13:16 +0000, ff via TF-A wrote:
> Le 15 mai 2024 à 14:42, Kamlesh Gurudasani via TF-A > tf-a@lists.trustedfirmware.org a écrit : > > Hi Achin, > > Thanks for the suggestion, > >> Instead of adding the complexity of yielding semantics to BL31, >> have you considered migrating the SCMI operations to a S-EL0 >> partition that is managed by EL3 firmware? This would be a >> preemptible bare- metal partition and does not require >> migrating the SCMI operations to a TEE. > > As per [0], we can't have a SEL1 firmware at all(e.g. OP-TEE), > if we run S-EL0 partition image. > > "It is not currently possible for BL31 to integrate SPM support > and a Secure Payload Dispatcher (SPD) at the same time; they are > mutually exclusive.
It has been possible to run a TEE and UEFI crypto as a separate partition for UEFI SecureBoot with U-Boot. Adding a second partition may be complex but not impossible.
I really don't think we would want to support this upstream but I will let the TF-A team comment. It makes EL3 solve a problem that should be solved in the TEE. This goes against keeping EL3 simple and opens the door for more vendor specific firmware to creep into EL3.
The best bet is to get the TEE to support FF-A and migrate the SCMI operations to an FF-A S-EL0 SP. The SP would work with any TEE that has a compliant FF-A implementation. On a >= v8.4 system, the same SP would work with Hafnium. It will require some upfront investment but is the most scalable approach in the long term.
cheers, Achin
I added Ilias in the to: as he knows alls the details
> In the SPM bootflow, a Secure Partition image executing at S-EL0 > replaces the Secure Payload image executing at S-EL1 (e.g. a > Trusted OS). Both are referred to as BL32." > > Let me know if you think otherwise. > > What we want is, > 1. Run clock and power management services as well as TEE > services 2. We don't want to put clock and power and management > services in TEE, so that we have liberty to use any TEE we want. > Example, if we put our SCMI operations in OP-TEE, then we'll be > constrained to use OP-TEE and only OP-TEE. > > [0] > https://trustedfirmware-a.readthedocs.io/en/latest/components/se > cur e-partition-manager-mm.html#secure-partition-manager-mm > > Regards, > Kamlesh > > Achin Gupta Achin.Gupta@arm.com writes: > >> Hi Kamlesh, >> >>> We are aware that ARM v8.4-A onwards arch are having a SEL2 >>> level with which we can run a vendor specific firmware >> parallel to TEE at SEL1 in a secure partition. But how we do >> handle clock and power management for version which doesn't >> support SEL2? >> >> Instead of adding the complexity of yielding semantics to BL31, >> have you considered migrating the SCMI operations to a S-EL0 >> partition that is managed by EL3 firmware? This would be a >> preemptible bare-metal partition and does not require migrating >> the SCMI operations to a TEE. >> >> cheers, >> Achin >> >> ________________________________ >> From: Kamlesh Gurudasani via TF-A >> tf-a@lists.trustedfirmware.org >> Sent: 15 May 2024 10:33 >> To: >> tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org >> Cc: bb@ti.com bb@ti.com; vigneshr@ti.com vigneshr@ti.com; >> d-gole@ti.com d-gole@ti.com; afd@ti.com afd@ti.com; >> hnagalla@ti.com hnagalla@ti.com >> Subject: [TF-A] Make TF-A (bl31) (El3 firmware) preemtible >> >> Clock and power management in ATF, preemption issue. >> >> I have seen that multiple Vendors are using arm scmi protocol >> to do clock management in bl31[0][1]. >> >> The problem with such implementations is any long running SCMI >> operations (like PLL locking for example) will hold the core in >> EL3 for extended period of time, adding to latency of NS EL1 >> interrupt handling as ATF is not preemptible. >> >> This problem can be solved by making ATF preemptible, similar >> to how OP-TEE does it by by implementing a remote procedural >> call after it receives a interrupt. >> [2]How Linux does SMC call with YIELD flag enabled. >> [3]How OP-TEE handles the timer interrupt [4]How Linux receives >> an interrupt which came in EL3->SEL1 >> >> As quoted in exception Handling document in ATF docs[5] >> >> "Receive exceptions, but handle part of the exception in EL3, >> and delegate processing of the error to dedicated software >> stack running at lower secure ELs (as above); additionally, the >> Normal world may also be required to participate in the >> handling, or be notified of such events (for example, as an event). >> In this scheme, exception handling potentially and maximally >> spans all ELs in both Secure and Normal worlds." >> >> From this we can understand that we can delegate the exception >> to >> EL1 if we are already in EL3. >> (way to preempt) >> >> We are aware that ARM v8.4-A onwards arch are having a SEL2 >> level with which we can run a vendor specific firmware parallel >> to TEE at SEL1 in a secure partition. But how we do handle >> clock and power management for version which doesn't support
SEL2?
>> >> We don't want to put it in TEE as it will constraint our >> devices to that specific TEE. >> >> I am starting to work on a proposal to make ATF preemptible >> similar to how OP-TEE is doing it. >> a)Will the similar approach from OP-TEE if implemented and >> working be accepted by ATF upstream? >> The SMC call with YIELD option will only be preemtible so will >> not affect the normal flow. >> >> b)As quoted in Trusted Firmware-A Document[6] Page 94, >> "Yielding calls are reserved exclusively for Trusted OS providers" >> "Yielding 0- 1 Reserved for existing Armv7-A calls Yielding >> 2-63 Trusted OS Standard Calls" >> >> Can this range be consumed within ATF/bl31 firmware or is it >> necessary to forward all yielding calls to Trusted OS? >> >> [0] >> https://github.com/ARM-software/arm-trusted-firmware/blob/maste >> r/ plat/st/stm32mp1/stm32mp1_scmi.c [1] >> https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/ >> 21 >> 840 >> [2] >> https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/op >> te >> e/smc_abi.c#L930 >> [3] >> https://github.com/OP-
TEE/optee_os/blob/fc57019cb35c8c1bad66fc6d8
>>
14ace5debde170a/core/arch/arm/kernel/thread_optee_smc_a64.S#L24
>> 6 >> [4] >> https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/op >> te >> e/smc_abi.c#L842 >> [5] >> https://github.com/ARM-software/arm-trusted-firmware/blob/maste >> r/ docs/components/exception-handling.rst >> [6] >> https://trustedfirmware-a.readthedocs.io/_/downloads/en/latest/ >> pd >> f/ >> >> Regards, >> Kamlesh >> -- >> TF-A mailing list -- tf-a@lists.trustedfirmware.org To >> unsubscribe send an email to >> tf-a-leave@lists.trustedfirmware.org > -- > TF-A mailing list -- tf-a@lists.trustedfirmware.org To > unsubscribe send an email to > tf-a-leave@lists.trustedfirmware.org
-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
Hi Francois,
On Wed, 15 May 2024 at 16:16, ff ff@shokubai.tech wrote:
Le 15 mai 2024 à 14:42, Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org a écrit :
Hi Achin,
Thanks for the suggestion,
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare-metal partition and does not require migrating the SCMI operations to a TEE.
As per [0], we can't have a SEL1 firmware at all(e.g. OP-TEE), if we run S-EL0 partition image.
"It is not currently possible for BL31 to integrate SPM support and a Secure Payload Dispatcher (SPD) at the same time; they are mutually exclusive.
It has been possible to run a TEE and UEFI crypto as a separate partition for UEFI SecureBoot with U-Boot. Adding a second partition may be complex but not impossible.
Thanks for cc'ing me and apologies for the late reply, I was travelling.
I added Ilias in the to: as he knows alls the details
It's technically possible, but we run StMM completely isolated from other TAs in the secure world. But we did this a few years ago when FF-A was just starting nad there was no code in u-boot. TF-A, EDK2 etc. FF-A is more mature nowadays so I'd rather use that and Trusted Services instead of adding another path in OP-TEE
Cheers /Ilias
In the SPM bootflow, a Secure Partition image executing at S-EL0 replaces the Secure Payload image executing at S-EL1 (e.g. a Trusted OS). Both are referred to as BL32."
Let me know if you think otherwise.
What we want is,
- Run clock and power management services as well as TEE services
- We don't want to put clock and power and management services in TEE,
so that we have liberty to use any TEE we want. Example, if we put our SCMI operations in OP-TEE, then we'll be constrained to use OP-TEE and only OP-TEE.
[0]https://trustedfirmware-a.readthedocs.io/en/latest/components/secure-partiti...
Regards, Kamlesh
Achin Gupta Achin.Gupta@arm.com writes:
Hi Kamlesh,
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware
parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
Instead of adding the complexity of yielding semantics to BL31, have you considered migrating the SCMI operations to a S-EL0 partition that is managed by EL3 firmware? This would be a preemptible bare-metal partition and does not require migrating the SCMI operations to a TEE.
cheers, Achin
From: Kamlesh Gurudasani via TF-A tf-a@lists.trustedfirmware.org Sent: 15 May 2024 10:33 To: tf-a@lists.trustedfirmware.org tf-a@lists.trustedfirmware.org Cc: bb@ti.com bb@ti.com; vigneshr@ti.com vigneshr@ti.com; d-gole@ti.com d-gole@ti.com; afd@ti.com afd@ti.com; hnagalla@ti.com hnagalla@ti.com Subject: [TF-A] Make TF-A (bl31) (El3 firmware) preemtible
Clock and power management in ATF, preemption issue.
I have seen that multiple Vendors are using arm scmi protocol to do clock management in bl31[0][1].
The problem with such implementations is any long running SCMI operations (like PLL locking for example) will hold the core in EL3 for extended period of time, adding to latency of NS EL1 interrupt handling as ATF is not preemptible.
This problem can be solved by making ATF preemptible, similar to how OP-TEE does it by by implementing a remote procedural call after it receives a interrupt. [2]How Linux does SMC call with YIELD flag enabled. [3]How OP-TEE handles the timer interrupt [4]How Linux receives an interrupt which came in EL3->SEL1
As quoted in exception Handling document in ATF docs[5]
"Receive exceptions, but handle part of the exception in EL3, and delegate processing of the error to dedicated software stack running at lower secure ELs (as above); additionally, the Normal world may also be required to participate in the handling, or be notified of such events (for example, as an event). In this scheme, exception handling potentially and maximally spans all ELs in both Secure and Normal worlds."
From this we can understand that we can delegate the exception to EL1 if we are already in EL3. (way to preempt)
We are aware that ARM v8.4-A onwards arch are having a SEL2 level with which we can run a vendor specific firmware parallel to TEE at SEL1 in a secure partition. But how we do handle clock and power management for version which doesn't support SEL2?
We don't want to put it in TEE as it will constraint our devices to that specific TEE.
I am starting to work on a proposal to make ATF preemptible similar to how OP-TEE is doing it. a)Will the similar approach from OP-TEE if implemented and working be accepted by ATF upstream? The SMC call with YIELD option will only be preemtible so will not affect the normal flow.
b)As quoted in Trusted Firmware-A Document[6] Page 94, "Yielding calls are reserved exclusively for Trusted OS providers" "Yielding 0- 1 Reserved for existing Armv7-A calls Yielding 2-63 Trusted OS Standard Calls"
Can this range be consumed within ATF/bl31 firmware or is it necessary to forward all yielding calls to Trusted OS?
[0]https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/st/stm... [1]https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/21840 [2]https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/optee/smc_abi.c... [3]https://github.com/OP-TEE/optee_os/blob/fc57019cb35c8c1bad66fc6d814ace5debde... [4]https://elixir.bootlin.com/linux/v6.9-rc3/source/drivers/tee/optee/smc_abi.c... [5]https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/compon... [6]https://trustedfirmware-a.readthedocs.io/_/downloads/en/latest/pdf/
Regards, Kamlesh -- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
-- TF-A mailing list -- tf-a@lists.trustedfirmware.org To unsubscribe send an email to tf-a-leave@lists.trustedfirmware.org
tf-a@lists.trustedfirmware.org
-
Achin Gupta -
Andrew Davis -
ff -
Ilias Apalodimas -
Kamlesh Gurudasani -
Soby Mathew