Hi,
Cc the OP-TEE mailing list, as I sense this might be more of an OP-TEE (/ OPTEED) design question than TF-A generic.
I agree with Manish/Achin the best way to eliminate the return by NS interrupt while OP-TEE kernel runs handling a PSCI request is to mask interrupts globally in the GIC e.g. from within opteed_system_reset?
Additional questions:
On https://github.com/ARM-software/arm-trusted-firmware/blob/master/lib/psci/p…
The PSCI layer calls into OP-TEE SPD and then resumes OP-TEE to handle the PSCI call in a platform defined manner.
As suggested here https://github.com/ARM-software/arm-trusted-firmware/blob/master/services/s… , it looks this invocation is not supposed to return into TF-A?
Did you redefine the weak psci_system_reset function within OP-TEE for your platform?
Is this function designed to end up in a loop and never returns to TF-A?
As a side question, is this an issue that the TF-A's PSCI platform hook is possibly never called https://github.com/ARM-software/arm-trusted-firmware/blob/master/lib/psci/p… ?
Regards,
Olivier.
________________________________
From: Neely, Brian via TF-A <tf-a(a)lists.trustedfirmware.org>
Sent: 13 November 2023 19:01
To: Manish Pandey2 <Manish.Pandey2(a)arm.com>; tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>; Ethridge, Caleb <Caleb.Ethridge(a)analog.com>
Subject: [TF-A] Re: Handling of normal world interrupts with BL31 PSCI handler
Hi Manish,
Just following up on your comment below. Are you planning to provide guidance as to which PSCI calls should have NS interrupts masked? Are these changes that should go in the mainline TF-A repo, or do you believe they are specific to our use case?
Thanks,
Brian
From: Manish Pandey2 via TF-A <tf-a(a)lists.trustedfirmware.org>
Sent: Friday, November 10, 2023 11:31 AM
To: tf-a(a)lists.trustedfirmware.org; Ethridge, Caleb <Caleb.Ethridge(a)analog.com>
Subject: [TF-A] Re: Handling of normal world interrupts with BL31 PSCI handler
[External]
Hi Caleb,
The quick answer to your query is to mask the NS interrupts in BL31 (option a), You should not remove the callback to OPTEE as it may need to do its own state maintained before system RESET.
There are other scenarios in PSCI (e.g. CPU power down) path which need to consider disabling NS interrupts before invoking Secure world hooks, will provide further analysis on those later.
Thanks
Manish Pandey
________________________________
From: Caleb Ethridge via TF-A <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Sent: 10 November 2023 15:12
To: tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org> <tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>>
Subject: [TF-A] Handling of normal world interrupts with BL31 PSCI handler
Hello,
If normal world interrupts are received while invoking TF-A's PSCI reset handler, we have observed that the reset can be aborted.
In TF-A's PSCI reset handler, a call out to OP-TEE is made before performing the platform-specific reset:
https://github.com/ARM-software/arm-trusted-firmware/blob/master/lib/psci/p…<https://urldefense.com/v3/__https:/github.com/ARM-software/arm-trusted-firm…>
https://github.com/ARM-software/arm-trusted-firmware/blob/master/services/s…<https://urldefense.com/v3/__https:/github.com/ARM-software/arm-trusted-firm…>
When OP-TEE is entered, it is possible to receive foreign (normal world) interrupts, which invokes the procedure described here: https://optee.readthedocs.io/en/3.16.0/architecture/core.html#deliver-non-s…<https://urldefense.com/v3/__https:/optee.readthedocs.io/en/3.16.0/architect…>
When the SMC call is aborted as described above, this results in the reboot failing. Linux does not retry the PSCI reset (https://github.com/torvalds/linux/blob/master/drivers/firmware/psci/psci.c#…<https://urldefense.com/v3/__https:/github.com/torvalds/linux/blob/master/dr…>). This makes sense because it is not expecting the SMC call to fail (it expected to make an uninterruptable SMC call into the secure monitor, not a call into OP-TEE).
If OP-TEE itself is setup to handle PSCI reset calls, it also handles them in (uninterruptable) SMC context:
https://github.com/OP-TEE/optee_os/blob/master/core/arch/arm/sm/psci.c#L140<https://urldefense.com/v3/__https:/github.com/OP-TEE/optee_os/blob/master/c…>
https://github.com/OP-TEE/optee_os/blob/master/core/arch/arm/sm/sm_a32.S#L96<https://urldefense.com/v3/__https:/github.com/OP-TEE/optee_os/blob/master/c…>
Based on this, we see two possible solutions:
a) Masking the non-secure interrupts in BL31 while we are doing a reset
b) Removing the call to OPTEE in the reset handler so that we never leave the SMC context
Which option do you suggest? Or are we missing an important detail here?
Thanks,
Caleb
--
TF-A mailing list -- tf-a(a)lists.trustedfirmware.org<mailto:tf-a@lists.trustedfirmware.org>
To unsubscribe send an email to tf-a-leave(a)lists.trustedfirmware.org<mailto:tf-a-leave@lists.trustedfirmware.org>
Hi all,
I'm testing FF-A notifications with OP-TEE and Hafnium. I'm using
interrupts from the secure uart as a trigger to set a notification for
the normal world. Sometimes when testing I run into:
VERBOSE: Secure virtual interrupt not yet serviced by SP 8001.
FFA_MSG_SEND_DIRECT_RESP interrupted
Hafnium then returns an FFA_ERROR (code -5) as a response to the
FFA_MSG_SEND_DIRECT_RESP OP-TEE was just exiting with. After some
digging in the code I find a comment at the top of
plat_ffa_is_direct_response_interrupted()
https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/arch/aarch64/p…
/*
* A secure interrupt might trigger while the target SP is currently
* running to send a direct response. SPMC would then inject virtual
* interrupt to vCPU of target SP and resume it.
* However, it is possible that the S-EL1 SP could have its interrupts
* masked and hence might not handle the virtual interrupt before
* sending direct response message. In such a scenario, SPMC must
* return an error with code FFA_INTERRUPTED to inform the S-EL1 SP of
* a pending interrupt and allow it to be handled before sending the
* direct response.
*/
The specification doesn't mention this as a valid error code for
FFA_MSG_SEND_DIRECT_RESP. Is this something we can expect to be added
to the specification or at least something OP-TEE has to be prepared
to handle regardless?
As far as I can tell there's no way of guaranteeing that Hafnium will
not return this error for FFA_MSG_SEND_DIRECT_RESP. Even if we were
able to execute the smc instruction with secure interrupts unmasked,
what if the interrupt is raised just after the smc instruction has
been trapped in Hafnium?
It is a bit inconvenient as it means saving the registers passed to
the smc instruction to be able to restart the smc instruction with the
same arguments. It seems we may need to redesign the exit procedure.
It would be nice with an example of how an S-EL1 SP is supposed to
exit with FFA_MSG_SEND_DIRECT_RESP.
Thoughts?
Thanks,
Jens
Hello arm-soc maintainers,
Please pull this patch to add vmalloc support for shared buffer
registration with the secure world.
Thanks,
Jens
The following changes since commit b85ea95d086471afb4ad062012a4d73cd328fa86:
Linux 6.7-rc1 (2023-11-12 16:19:07 -0800)
are available in the Git repository at:
https://git.linaro.org/people/jens.wiklander/linux-tee.git tags/tee-iov-iter-for-v6.8
for you to fetch changes up to 7bdee41575919773818e525ea19e54eb817770af:
tee: Use iov_iter to better support shared buffer registration (2023-12-13 07:23:22 +0100)
----------------------------------------------------------------
TEE: improve shared buffer registration compatibility
----------------------------------------------------------------
Arnaud Pouliquen (1):
tee: Use iov_iter to better support shared buffer registration
drivers/tee/tee_shm.c | 78 +++++++++++++++++++++++++++------------------------
1 file changed, 42 insertions(+), 36 deletions(-)
Hello arm-soc maintainers,
Please pull these two small patches for the OP-TEE driver that remove a
redundant custom workqueue and add a description of an argument of a
optee_handle_rpc().
Note that this pull request is made on top of optee-supplicant-fix-for-v6.7
7269cba53d90 ("tee: optee: Fix supplicant based device enumeration")
which was merged into v6.7-rc5, but in turn was based on top of v6.6. I
did it like this because the Tested-bys where provided in this context.
Thanks,
Jens
The following changes since commit 7269cba53d906cf257c139d3b3a53ad272176bca:
tee: optee: Fix supplicant based device enumeration (2023-11-03 09:27:20 +0100)
are available in the Git repository at:
https://git.linaro.org/people/jens.wiklander/linux-tee.git tags/optee-cleanup-for-v6.8
for you to fetch changes up to b19773a1c6c02f5efc35e9f506aeddd2c7d2ac29:
optee: add missing description of RPC argument reference (2023-12-11 15:02:12 +0100)
----------------------------------------------------------------
OP-TEE cleanup
- Remove a redundant custom workqueue in the OP-TEE driver.
- Fix a missing description of an argument to optee_handle_rpc().
----------------------------------------------------------------
Etienne Carriere (1):
optee: add missing description of RPC argument reference
Sumit Garg (1):
tee: optee: Remove redundant custom workqueue
drivers/tee/optee/core.c | 13 ++-----------
drivers/tee/optee/optee_private.h | 2 --
drivers/tee/optee/smc_abi.c | 1 +
3 files changed, 3 insertions(+), 13 deletions(-)
This series introduces the tee based EFI Runtime Variable Service.
The eMMC device is typically owned by the non-secure world(linux in
this case). There is an existing solution utilizing eMMC RPMB partition
for EFI Variables, it is implemented by interacting with
OP-TEE, StandaloneMM(as EFI Variable Service Pseudo TA), eMMC driver
and tee-supplicant. The last piece is the tee-based variable access
driver to interact with OP-TEE and StandaloneMM.
Changelog:
v8 -> v9
- patch #6 "tee: optee: restore efivars ops when tee-supplicant stops"
is newly added
- remove !EFI_VARS_PSTORE Kconfig dependency, we have added a non-blocking
set_variable and it just returns EFI_UNSUPPORTED.
- remove obvious comments
v7 -> v8
Only patch #3 "efi: Add tee-based EFI variable driver" is updated.
- fix typos
- refactor error handling, direct return if applicable
- use devm_add_action_or_reset() for closing of tee context/session
- remove obvious comment
v6 -> v7
Patch #1-#4 are not updated.
Patch #5 is added into this series, original patch is here:
https://lore.kernel.org/all/20230609094532.562934-1-ilias.apalodimas@linaro…
There are two issues in the v6 series and v7 series addresses those.
1) efivar ops is not restored when the tee-supplicant daemon terminates.
-> As the following patch says, user must remove the device before
terminating tee-supplicant daemon.
https://lore.kernel.org/all/20230728134832.326467-1-sumit.garg@linaro.org/
2) cause panic when someone remounts the efivarfs as RW even if
SetVariable is not supported
-> The fifth patch addresses this issue.
"[PATCH v7 5/5] efivarfs: force RO when remounting if SetVariable is
not supported"
v5 -> v6
- new patch #4 is added in this series, #1-#3 patches are unchanged.
automatically update super block flag when the efivarops support
SetVariable runtime service, so that user does not need to manually
remount the efivarfs as RW.
v4 -> v5
- rebase to efi-next based on v6.4-rc1
- set generic_ops.query_variable_info, it works as expected as follows.
$ df -h /sys/firmware/efi/efivars/
Filesystem Size Used Avail Use% Mounted on
efivarfs 16K 1.3K 15K 8% /sys/firmware/efi/efivars
v3 -> v4:
- replace the reference from EDK2 to PI Specification
- remove EDK2 source code reference comments
- prepare nonblocking variant of set_variable, it just returns
EFI_UNSUPPORTED
- remove redundant buffer size check
- argument name change in mm_communicate
- function interface changes in setup_mm_hdr to remove (void **) cast
v2 -> v3:
- add CONFIG_EFI dependency to TEE_STMM_EFI
- add missing return code check for tee_client_invoke_func()
- directly call efivars_register/unregister from tee_stmm_efi.c
rfc v1 -> v2:
- split patch into three patches, one for drivers/tee,
one for include/linux/efi.h, and one for the driver/firmware/efi/stmm
- context/session management into probe() and remove() same as other tee
client driver
- StMM variable driver is moved from driver/tee/optee to driver/firmware/efi
- use "tee" prefix instead of "optee" in driver/firmware/efi/stmm/tee_stmm_efi.c,
this file does not contain op-tee specific code, abstracted by tee layer and
StMM variable driver will work on other tee implementation.
- PTA_STMM_CMD_COMMUNICATE -> PTA_STMM_CMD_COMMUNICATE
- implement query_variable_store() but currently not used
- no use of TEEC_SUCCESS, it is defined in driver/tee/optee/optee_private.h.
Other tee client drivers use 0 instead of using TEEC_SUCCESS
- remove TEEC_ERROR_EXCESS_DATA status, it is referred just to output
error message
Ilias Apalodimas (1):
efivarfs: force RO when remounting if SetVariable is not supported
Masahisa Kojima (5):
efi: expose efivar generic ops register function
efi: Add EFI_ACCESS_DENIED status code
efi: Add tee-based EFI variable driver
efivarfs: automatically update super block flag
tee: optee: restore efivars ops when tee-supplicant stops
drivers/firmware/efi/Kconfig | 15 +
drivers/firmware/efi/Makefile | 1 +
drivers/firmware/efi/efi.c | 18 +
drivers/firmware/efi/stmm/mm_communication.h | 236 +++++++
drivers/firmware/efi/stmm/tee_stmm_efi.c | 617 +++++++++++++++++++
drivers/firmware/efi/vars.c | 8 +
drivers/tee/optee/supp.c | 4 +
fs/efivarfs/super.c | 45 ++
include/linux/efi.h | 13 +
9 files changed, 957 insertions(+)
create mode 100644 drivers/firmware/efi/stmm/mm_communication.h
create mode 100644 drivers/firmware/efi/stmm/tee_stmm_efi.c
base-commit: b691118f2c44d16b84fc65b8147b33620eb18cac
--
2.30.2
Hello arm-soc maintainers,
Please pull these two patches that optmize how the OP-TEE driver private
shared memory allocated as dynamic shared memory.
Thanks,
Jens
The following changes since commit b85ea95d086471afb4ad062012a4d73cd328fa86:
Linux 6.7-rc1 (2023-11-12 16:19:07 -0800)
are available in the Git repository at:
https://git.linaro.org/people/jens.wiklander/linux-tee.git tags/kern-priv-shm-for-v6.8
for you to fetch changes up to 225a36b96359aceaa9a6399f2dff99627397e637:
optee: allocate shared memory with alloc_pages_exact() (2023-12-04 14:48:42 +0100)
----------------------------------------------------------------
OP-TEE kernel private shared memory optimizations
Optimize OP-TEE driver private shared memory allocated as dynamic shared
memory. Both to handle larger than one page allocations and for more
efficient memory usage.
----------------------------------------------------------------
Jens Wiklander (2):
optee: add page list to kernel private shared memory
optee: allocate shared memory with alloc_pages_exact()
drivers/tee/optee/core.c | 44 +++++++++++++++++++++++---------------------
drivers/tee/optee/smc_abi.c | 45 +++++++++++++++++++++------------------------
2 files changed, 44 insertions(+), 45 deletions(-)
Hello arm-soc maintainers,
Please pull these patches that enables asynchronous notifications in
OP-TEE FF-A driver. This pull request is on top of the
"system_thread_for_v6.8" pull request to avoid a merge conflict.
Thanks,
Jens
The following changes since commit 4b391c9c37646f25118355f414b9e6d9fefe782f:
firmware: arm_scmi: optee: use optee system invocation (2023-11-17 15:48:03 +0100)
are available in the Git repository at:
https://git.linaro.org/people/jens.wiklander/linux-tee.git tags/ffa-notif-for-v6.8
for you to fetch changes up to d0476a59de064205f4aaa8f7c6d6f32bc28a44d4:
optee: ffa_abi: add asynchronous notifications (2023-11-17 15:55:41 +0100)
----------------------------------------------------------------
OP-TEE: asynchronous notifications with FF-A
Add support for asynchronous notifications in the OP-TEE FF-A driver. This
is the FF-A counterpart to the asynchronous notifications already
available in the OP-TEE SMC ABI.
----------------------------------------------------------------
Jens Wiklander (2):
optee: provide optee_do_bottom_half() as a common function
optee: ffa_abi: add asynchronous notifications
drivers/tee/optee/call.c | 31 ++++++++++++-
drivers/tee/optee/ffa_abi.c | 93 +++++++++++++++++++++++++++++++++++++--
drivers/tee/optee/optee_ffa.h | 28 ++++++++++--
drivers/tee/optee/optee_private.h | 9 +++-
drivers/tee/optee/smc_abi.c | 36 ++-------------
5 files changed, 155 insertions(+), 42 deletions(-)
Hello arm-soc maintainers,
Please pull these patches that enables SCMI with OP-TEE transport to to
reserve a thread in the secure world to guarentee that requests will not
have to wait for a thread to become available.
Thanks,
Jens
The following changes since commit b85ea95d086471afb4ad062012a4d73cd328fa86:
Linux 6.7-rc1 (2023-11-12 16:19:07 -0800)
are available in the Git repository at:
https://git.linaro.org/people/jens.wiklander/linux-tee.git tags/system-thread-for-v6.8
for you to fetch changes up to 4b391c9c37646f25118355f414b9e6d9fefe782f:
firmware: arm_scmi: optee: use optee system invocation (2023-11-17 15:48:03 +0100)
----------------------------------------------------------------
OP-TEE add reserved system thread
Add support for a reserved system thread in the SMC-ABI of the OP-TEE driver.
SCMI with OP-TEE transport uses this to guarantee that it will always have
a thread available in the secure world.
----------------------------------------------------------------
Etienne Carriere (4):
tee: optee: system thread call property
tee: system session
tee: optee: support tracking system threads
firmware: arm_scmi: optee: use optee system invocation
drivers/firmware/arm_scmi/optee.c | 4 ++
drivers/tee/optee/call.c | 130 ++++++++++++++++++++++++++++++++++++--
drivers/tee/optee/core.c | 5 +-
drivers/tee/optee/ffa_abi.c | 14 ++--
drivers/tee/optee/optee_private.h | 29 ++++++++-
drivers/tee/optee/smc_abi.c | 32 +++++++---
drivers/tee/tee_core.c | 8 +++
include/linux/tee_drv.h | 16 +++++
8 files changed, 211 insertions(+), 27 deletions(-)