Thanks for the info Manish. Regarding testing, it seems like this should be possible with the standard FVP model? If I look at how system reset is implemented, it looks similar to our platform (platform-specific PSCI handler writes a register
to trigger the reset:
https://github.com/ARM-software/arm-trusted-firmware/blob/master/plat/arm/board/fvp/fvp_pm.c#L320). I think what would need to happen is to “open” the window in which a NS interrupt could interrupt OP-TEE while it executes its reset hook:
https://github.com/OP-TEE/optee_os/blob/master/core/arch/arm/kernel/thread.c#L1180. Possibly just putting an infinite loop in there would expose the issue. The NS interrupt that is received on our platform is a generic one, the kernel’s timer interrupt,
and therefore should also be active/enabled when running Linux on the FVP. Issuing a reboot command from Linux on the FVP should trigger this scenario.
Thanks,
Brian
From: Manish Pandey2 via TF-A <tf-a@lists.trustedfirmware.org>
Sent: Tuesday, November 14, 2023 6:17 AM
To: Olivier Deprez <Olivier.Deprez@arm.com>; tf-a@lists.trustedfirmware.org; Ethridge, Caleb <Caleb.Ethridge@analog.com>; Neely, Brian <Brian.Neely@analog.com>
Cc: op-tee@lists.trustedfirmware.org
Subject: [TF-A] Re: Handling of normal world interrupts with BL31 PSCI handler
|
[External] |
Hi Brian,
Your usecase is that you want to use SDEI framework to deliver some EL3 interrupts to NS and if your only concern is about system reset the easiest/temp solution would be to disable the G1NS interrupts
in OP-TEE dispatcher's system RESET hook.
The generic solution would take some time to implement as i need to analyse/discuss following scenarios before making a decision
1. Does secure world really needs EL3 interrupts or it can handle NS-interrupts as FIQ (pre-EHF, do not modify routing model for secure world). Notification to EL3 can be done using some standard
or IMPDEF mechanism. FFA has an ABI for this FFA_EL3_INTR_HANDLE. Current Hafnium implementation follows this for G0 interrupts.
2. The PSCI calls for CPU power down/up, migrate, system reset/off do invoke SPD hooks. It need to be carefully analysed whether we really need to disable G1NS interrupts in these paths or not
? For e.g in CPU power down path the PSCI spec mandates that its caller's responsbility to migrate interrupts from the core (so we don't need to do anything special in TF-A code). Similarly for reset/off and migrate we need to analyse what should be the best
way to avoid this situation.
Also, i need to work on test infrastructure to simulate these scenarios. It will be useful if your setup can be recreated using OP-TEE/TF-A ?
Thanks
Manish
From: Olivier Deprez <Olivier.Deprez@arm.com>
Sent: 14 November 2023 10:47
To: Manish Pandey2 <Manish.Pandey2@arm.com>;
tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.org>; Ethridge, Caleb <Caleb.Ethridge@analog.com>;
Neely, Brian <Brian.Neely@analog.com>
Cc: op-tee@lists.trustedfirmware.org <op-tee@lists.trustedfirmware.org>
Subject: Re: [TF-A] Re: Handling of normal world interrupts with BL31 PSCI handler
Hi,
Cc the OP-TEE mailing list, as I sense this might be more of an OP-TEE (/ OPTEED) design question than TF-A generic.
I agree with Manish/Achin the best way to eliminate the return by NS interrupt while OP-TEE kernel runs handling a PSCI request is to mask interrupts globally in
the GIC e.g. from within opteed_system_reset?
Additional questions:
The PSCI layer calls into OP-TEE SPD and then resumes OP-TEE to handle the PSCI call in a platform defined manner.
As suggested here
https://github.com/ARM-software/arm-trusted-firmware/blob/master/services/spd/opteed/opteed_pm.c#L233 ,
it looks this invocation is not supposed to return into TF-A?
Did you redefine the weak psci_system_reset function within OP-TEE for your platform?
Is this function designed to end up in a loop and never returns to TF-A?
As a side question, is this an issue that the TF-A's PSCI platform hook is possibly never called
https://github.com/ARM-software/arm-trusted-firmware/blob/master/lib/psci/psci_system_off.c#L50 ?
Regards,
Olivier.
From: Neely, Brian via TF-A <tf-a@lists.trustedfirmware.org>
Sent: 13 November 2023 19:01
To: Manish Pandey2 <Manish.Pandey2@arm.com>;
tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.org>; Ethridge, Caleb <Caleb.Ethridge@analog.com>
Subject: [TF-A] Re: Handling of normal world interrupts with BL31 PSCI handler
Hi Manish,
Just following up on your comment below. Are you planning to provide guidance as to which PSCI calls should have NS interrupts masked? Are these changes that should go in the mainline TF-A repo, or do you believe they are specific to our use case?
Thanks,
Brian
From: Manish Pandey2 via TF-A <tf-a@lists.trustedfirmware.org>
Sent: Friday, November 10, 2023 11:31 AM
To: tf-a@lists.trustedfirmware.org; Ethridge, Caleb <Caleb.Ethridge@analog.com>
Subject: [TF-A] Re: Handling of normal world interrupts with BL31 PSCI handler
|
[External] |
Hi Caleb,
The quick answer to your query is to mask the NS interrupts in BL31 (option a), You should not remove the callback to OPTEE as it may need to do its own state maintained before system
RESET.
There are other scenarios in PSCI
(e.g. CPU power down) path which need to consider disabling NS interrupts before invoking Secure world hooks, will provide further analysis on those later.
Thanks
Manish Pandey
From: Caleb Ethridge via TF-A <tf-a@lists.trustedfirmware.org>
Sent: 10 November 2023 15:12
To: tf-a@lists.trustedfirmware.org <tf-a@lists.trustedfirmware.org>
Subject: [TF-A] Handling of normal world interrupts with BL31 PSCI handler
Hello,
If normal world interrupts are received while invoking TF-A's PSCI reset handler, we have observed that the reset can be aborted.
In TF-A's PSCI reset handler, a call out to OP-TEE is made before performing the platform-specific reset:
https://github.com/ARM-software/arm-trusted-firmware/blob/master/lib/psci/psci_system_off.c#L44
https://github.com/ARM-software/arm-trusted-firmware/blob/master/services/spd/opteed/opteed_pm.c#L233
When OP-TEE is entered, it is possible to receive foreign (normal world) interrupts, which invokes the procedure described here:
https://optee.readthedocs.io/en/3.16.0/architecture/core.html#deliver-non-secure-interrupts-to-normal-world
When the SMC call is aborted as described above, this results in the reboot failing. Linux does not retry the PSCI reset (https://github.com/torvalds/linux/blob/master/drivers/firmware/psci/psci.c#L308).
This makes sense because it is not expecting the SMC call to fail (it expected to make an uninterruptable SMC call into the secure monitor, not a call into OP-TEE).
If OP-TEE itself is setup to handle PSCI reset calls, it also handles them in (uninterruptable) SMC context:
https://github.com/OP-TEE/optee_os/blob/master/core/arch/arm/sm/psci.c#L140
https://github.com/OP-TEE/optee_os/blob/master/core/arch/arm/sm/sm_a32.S#L96
Based on this, we see two possible solutions:
a) Masking the non-secure interrupts in BL31 while we are doing a reset
b) Removing the call to OPTEE in the reset handler so that we never leave the SMC context
Which option do you suggest? Or are we missing an important detail here?
Thanks,
Caleb
--
TF-A mailing list -- tf-a@lists.trustedfirmware.org
To unsubscribe send an email to
tf-a-leave@lists.trustedfirmware.org