Hi all,
This patchset is a general cleanup of shared memory handling in the TEE
subsystem.
Until now has the in-kernel tee clients used tee_shm_alloc() and
tee_shm_register() to share memory with secure world. These two function
exposes via a flags parameter a bit more of the internals of the TEE
subsystem than one would like. So in order to make things easier are those
two functions replaced by few functions which should provide better
abstraction.
Two in-kernel tee clients are updated to use these new functions.
The shared memory pool handling is simplified, an internal matter for the
two TEE drivers OP-TEE and AMDTEE.
An OP-TEE driver internal tee_context is added to handle shared memory
allocations received via RPC, for instance the argument structure needed
to make more complex RPC requests. The tee_context used when doing such a
memory allocation must be kept until the memory is freed. With this we can
avoid keeping a tee_context of a client around for longer than necessary.
In the v1 review it was suggested [1] to allow physically non-contiguous
memory allocations by the drivers. It turned out to be harder than
anticipated so I'll save that for a separate patch.
This patchset is also available at [2].
Thanks,
Jens
[1] https://lore.kernel.org/linux-arm-kernel/20210609145811.GJ4910@sequoia/
[2] https://git.linaro.org/people/jens.wiklander/linux-tee.git/log/?h=tee_shm_v3
v2->v3:
* Make tee_shm_alloc_user_buf() and tee_shm_register_user_buf() internal
and don't export them to the drivers.
* Rename tee_shm_alloc_priv_kernel_buf() to tee_shm_alloc_priv_buf()
* Adressing comments on variable names and choice of types in "tee: replace
tee_shm_register()"
* Adding detailed explaination on alignment in "tee: simplify shm pool handling"
* Added Sumits R-B on a few of the patches
v1->v2:
* The commits three "tee: add tee_shm_alloc_kernel_buf()",
"tpm_ftpm_tee: use tee_shm_alloc_kernel_buf()" and
"firmware: tee_bnxt: use tee_shm_alloc_kernel_buf()" has been merged some
time ago as part of another patchset.
* Another in-kernel tee client is updated with the commit
"KEYS: trusted: tee: use tee_shm_register_kernel_buf()"
* tee_shm_alloc_anon_kernel_buf() is replaced with an easier to use function
tee_shm_alloc_priv_kernel_buf() and tee_shm_free_anon_kernel_buf() has
been dropped.
* A driver internal struct tee_context is used to when doing driver internal
calls to secure world.
* Adds patches to replace tee_shm_register() in a similar way as how
tee_shm_alloc() is replaced.
* A patch is added to clean up the TEE_SHM_* flags
* Fixed a warning reported by kernel test robot <lkp(a)intel.com>
Jens Wiklander (12):
hwrng: optee-rng: use tee_shm_alloc_kernel_buf()
tee: remove unused tee_shm_pool_alloc_res_mem()
tee: add tee_shm_alloc_user_buf()
tee: simplify shm pool handling
tee: replace tee_shm_alloc()
optee: add driver private tee_context
optee: use driver internal tee_contex for some rpc
optee: add optee_pool_op_free_helper()
tee: add tee_shm_register_{user,kernel}_buf()
KEYS: trusted: tee: use tee_shm_register_kernel_buf()
tee: replace tee_shm_register()
tee: refactor TEE_SHM_* flags
drivers/char/hw_random/optee-rng.c | 6 +-
drivers/tee/amdtee/shm_pool.c | 55 ++--
drivers/tee/optee/Kconfig | 8 -
drivers/tee/optee/call.c | 2 +-
drivers/tee/optee/core.c | 22 +-
drivers/tee/optee/device.c | 5 +-
drivers/tee/optee/ffa_abi.c | 136 ++++------
drivers/tee/optee/optee_private.h | 12 +-
drivers/tee/optee/smc_abi.c | 159 +++--------
drivers/tee/tee_core.c | 5 +-
drivers/tee/tee_private.h | 15 +-
drivers/tee/tee_shm.c | 320 +++++++++++++++--------
drivers/tee/tee_shm_pool.c | 162 +++---------
include/linux/tee_drv.h | 138 +++-------
security/keys/trusted-keys/trusted_tee.c | 23 +-
15 files changed, 438 insertions(+), 630 deletions(-)
--
2.31.1
The addition of a shutdown hook by commit f25889f93184 ("optee:
fix tee out of memory failure seen during kexec reboot") introduced a
kernel shutdown regression that can be triggered after running the
xtest suites.
Once the shutdown hook is called it is not possible to communicate any
more with the supplicant process because the system is not scheduling
task any longer. Thus if the optee driver shutdown path receives a
supplicant RPC request from the OP-TEE we will deadlock the kernel's
shutdown.
This unexpected event will in fact occur after the xtest suite has
been run. It seems some cached SHM kept alive a context object which
in turn kept alive a session towards a PTA or TA. Closing the session
results in a socket RPC command being sent back from OP-TEE.
This sequence of events is captured by a 5.15 kernel annotated with
extra prints:
Calling OPTEE_SMC_DISABLE_SHM_CACHE
OPTEE_SMC_DISABLE_SHM_CACHE returned 0
freeing SHM ptr 0xFFFFFF8001079380
Calling OPTEE_SMC_DISABLE_SHM_CACHE
OPTEE_SMC_DISABLE_SHM_CACHE returned 0
freeing SHM ptr 0xFFFFFF8001CC5580
Calling OPTEE_SMC_DISABLE_SHM_CACHE
OPTEE_SMC_DISABLE_SHM_CACHE returned 0
freeing SHM ptr 0xFFFFFF8006308A80
Calling OPTEE_SMC_DISABLE_SHM_CACHE
OPTEE_SMC_DISABLE_SHM_CACHE returned 0
freeing SHM ptr 0xFFFFFF8006308B00
optee: optee_handle_rpc: a0=0XFFFF0000 a1=0XA0 a2=0X0
optee: optee_handle_rpc: a0=0XFFFF0005 a1=0XFFFFFF80 a2=0X61E6500
optee: handle_rpc_func_cmd: cmd = 0XA
optee_supp_thrd_req: func=0XA
Introduce a shutdown state in the optee device object to return an
immediate error to all RPC requests in the shutdown path.
Fixes: f25889f93184 ("optee: fix tee out of memory failure seen during kexec reboot")
Signed-off-by: Lars Persson <larper(a)axis.com>
---
drivers/tee/optee/optee_private.h | 1 +
drivers/tee/optee/smc_abi.c | 5 ++++-
drivers/tee/optee/supp.c | 8 ++++++++
3 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/drivers/tee/optee/optee_private.h b/drivers/tee/optee/optee_private.h
index 46f74ab07c7e..9eb72931e11f 100644
--- a/drivers/tee/optee/optee_private.h
+++ b/drivers/tee/optee/optee_private.h
@@ -162,6 +162,7 @@ struct optee {
struct tee_shm_pool *pool;
unsigned int rpc_arg_count;
bool scan_bus_done;
+ bool shutting_down;
struct workqueue_struct *scan_bus_wq;
struct work_struct scan_bus_work;
};
diff --git a/drivers/tee/optee/smc_abi.c b/drivers/tee/optee/smc_abi.c
index 449d6a72d289..10af747da816 100644
--- a/drivers/tee/optee/smc_abi.c
+++ b/drivers/tee/optee/smc_abi.c
@@ -1356,7 +1356,10 @@ static int optee_smc_remove(struct platform_device *pdev)
*/
static void optee_shutdown(struct platform_device *pdev)
{
- optee_disable_shm_cache(platform_get_drvdata(pdev));
+ struct optee *optee = platform_get_drvdata(pdev);
+
+ optee->shutting_down = true;
+ optee_disable_shm_cache(optee);
}
static int optee_probe(struct platform_device *pdev)
diff --git a/drivers/tee/optee/supp.c b/drivers/tee/optee/supp.c
index 322a543b8c27..801b4ec659e8 100644
--- a/drivers/tee/optee/supp.c
+++ b/drivers/tee/optee/supp.c
@@ -83,6 +83,14 @@ u32 optee_supp_thrd_req(struct tee_context *ctx, u32 func, size_t num_params,
bool interruptable;
u32 ret;
+ /*
+ * When the system is shutting down we cannot talk
+ * to the supplicant anymore even if we seem to have
+ * an open session to it.
+ */
+ if (optee->shutting_down)
+ return TEEC_ERROR_COMMUNICATION;
+
/*
* Return in case there is no supplicant available and
* non-blocking request.
--
2.30.2
Hi,
OP-TEE Contributions (LOC) monthly meeting is planned for Thursday Jan 27
@16.00 (UTC).
If you have any topics you'd like to discuss, please let us know and we can
schedule them.
Meeting details:
---------------
Date/time: January 27(a)16.00 (UTC)
https://everytimezone.com/s/c3460919
Connection details: https://www.trustedfirmware.org/meetings/
Meeting notes: http://bit.ly/loc-notes
Regards,
Ruchika on behalf of the Linaro OP-TEE team
Hello arm-soc maintainers,
Please pull these OP-TEE driver fixes all concerning the recent changes
regarding FF-A and asynchronous notifications.
Thanks,
Jens
The following changes since commit e783362eb54cd99b2cac8b3a9aeac942e6f6ac07:
Linux 5.17-rc1 (2022-01-23 10:12:53 +0200)
are available in the Git repository at:
git://git.linaro.org/people/jens.wiklander/linux-tee.git tags/optee-fixes-for-v5.17
for you to fetch changes up to 4064c461148ab129dfe5eaeea129b4af6cf4b9b7:
optee: add error checks in optee_ffa_do_call_with_arg() (2022-01-24 13:00:59 +0100)
----------------------------------------------------------------
OP-TE fixes for v5.17
- Adds error checking in optee_ffa_do_call_with_arg()
- Reintroduces an accidentally lost fix for a memref size check
- Uses bitmap_free() to free memory obtained with bitmap_zalloc()
----------------------------------------------------------------
Christophe JAILLET (1):
optee: Use bitmap_free() to free bitmap
Jens Wiklander (1):
optee: add error checks in optee_ffa_do_call_with_arg()
Jerome Forissier (1):
tee: optee: do not check memref size on return from Secure World
drivers/tee/optee/ffa_abi.c | 15 ++++++++++++---
drivers/tee/optee/notif.c | 2 +-
drivers/tee/optee/smc_abi.c | 10 ----------
3 files changed, 13 insertions(+), 14 deletions(-)
Hi all,
This patchset is a general cleanup of shared memory handling in the TEE
subsystem.
Until now has the in-kernel tee clients used tee_shm_alloc() and
tee_shm_register() to share memory with secure world. These two function
exposes via a flags parameter a bit more of the internals of the TEE
subsystem than one would like. So in order to make things easier are those
two functions replaced by few functions which should provide better
abstraction.
Two in-kernel tee clients are updated to use these new functions.
The shared memory pool handling is simplified, an internal matter for the
two TEE drivers OP-TEE and AMDTEE.
An OP-TEE driver internal tee_context is added to handle shared memory
allocations received via RPC, for instance the argument structure needed
to make more complex RPC requests. The tee_context used when doing such a
memory allocation must be kept until the memory is freed. With this we can
avoid keeping a tee_context of a client around for longer than necessary.
In the v1 review it was suggested [1] to allow physically non-contiguous
memory allocations by the drivers. It turned out to be harder than
anticipated so I'll save that for a separate patch.
This patchset is also available at [2] and is based on the asynchronous
notification patches [3] which was just merged during this merge window.
Thanks,
Jens
[1] https://lore.kernel.org/linux-arm-kernel/20210609145811.GJ4910@sequoia/
[2] https://git.linaro.org/people/jens.wiklander/linux-tee.git/log/?h=tee_shm_v2
[3] https://git.linaro.org/people/jens.wiklander/linux-tee.git/log/?h=async_not…
v1->v2:
* The commits three "tee: add tee_shm_alloc_kernel_buf()",
"tpm_ftpm_tee: use tee_shm_alloc_kernel_buf()" and
"firmware: tee_bnxt: use tee_shm_alloc_kernel_buf()" has been merged some
time ago as part of another patchset.
* Another in-kernel tee client is updated with the commit
"KEYS: trusted: tee: use tee_shm_register_kernel_buf()"
* tee_shm_alloc_anon_kernel_buf() is replaced with an easier to use function
tee_shm_alloc_priv_kernel_buf() and tee_shm_free_anon_kernel_buf() has
been dropped.
* A driver internal struct tee_context is used to when doing driver internal
calls to secure world.
* Adds patches to replace tee_shm_register() in a similar way as how
tee_shm_alloc() is replaced.
* A patch is added to clean up the TEE_SHM_* flags
* Fixed a warning reported by kernel test robot <lkp(a)intel.com>
Jens Wiklander (12):
hwrng: optee-rng: use tee_shm_alloc_kernel_buf()
tee: remove unused tee_shm_pool_alloc_res_mem()
tee: add tee_shm_alloc_user_buf()
tee: simplify shm pool handling
tee: replace tee_shm_alloc()
optee: add driver private tee_context
optee: use driver internal tee_contex for some rpc
optee: add optee_pool_op_free_helper()
tee: add tee_shm_register_{user,kernel}_buf()
KEYS: trusted: tee: use tee_shm_register_kernel_buf()
tee: replace tee_shm_register()
tee: refactor TEE_SHM_* flags
drivers/char/hw_random/optee-rng.c | 6 +-
drivers/tee/amdtee/shm_pool.c | 55 ++--
drivers/tee/optee/Kconfig | 8 -
drivers/tee/optee/call.c | 2 +-
drivers/tee/optee/core.c | 22 +-
drivers/tee/optee/device.c | 5 +-
drivers/tee/optee/ffa_abi.c | 136 ++++------
drivers/tee/optee/optee_private.h | 12 +-
drivers/tee/optee/smc_abi.c | 155 +++--------
drivers/tee/tee_core.c | 5 +-
drivers/tee/tee_private.h | 11 -
drivers/tee/tee_shm.c | 322 +++++++++++++++--------
drivers/tee/tee_shm_pool.c | 162 +++---------
include/linux/tee_drv.h | 133 +++-------
security/keys/trusted-keys/trusted_tee.c | 23 +-
15 files changed, 434 insertions(+), 623 deletions(-)
--
2.31.1
[BCC all OP-TEE maintainers]
Hi OP-TEE maintainers & contributors,
OP-TEE v3.16.0 is scheduled to be released on 2022-01-28. So, now is a
good time to start testing the master branch on the various platforms
and report/fix any bugs.
The GitHub pull request for collecting Tested-by tags or any other
comments is https://github.com/OP-TEE/optee_os/pull/5094
As usual, we will create a release candidate tag one week before the
release date for final testing.
In addition to that you can find some additional information related
to releases here:
https://optee.readthedocs.io/en/latest/general/releases.html
Thanks,
Jens
The addition of a shutdown hook by commit f25889f93184 ("optee:
fix tee out of memory failure seen during kexec reboot") introduced a
kernel shutdown regression that can be triggered after running the
xtest suites.
Once the shutdown hook is called it is not possible to communicate any
more with the supplicant process because the system is not scheduling
task any longer. Thus if the optee driver shutdown path receives a
supplicant RPC request from the OP-TEE we will deadlock the kernel's
shutdown.
This unexpected event will in fact occur after the xtest suite has
been run. It seems some cached SHM kept alive a context object which
in turn kept alive a session towards a PTA or TA. Closing the session
results in a socket RPC command being sent back from OP-TEE.
This sequence of events is captured by a 5.15 kernel annotated with
extra prints:
Calling OPTEE_SMC_DISABLE_SHM_CACHE
OPTEE_SMC_DISABLE_SHM_CACHE returned 0
freeing SHM ptr 0xFFFFFF8001079380
Calling OPTEE_SMC_DISABLE_SHM_CACHE
OPTEE_SMC_DISABLE_SHM_CACHE returned 0
freeing SHM ptr 0xFFFFFF8001CC5580
Calling OPTEE_SMC_DISABLE_SHM_CACHE
OPTEE_SMC_DISABLE_SHM_CACHE returned 0
freeing SHM ptr 0xFFFFFF8006308A80
Calling OPTEE_SMC_DISABLE_SHM_CACHE
OPTEE_SMC_DISABLE_SHM_CACHE returned 0
freeing SHM ptr 0xFFFFFF8006308B00
optee: optee_handle_rpc: a0=0XFFFF0000 a1=0XA0 a2=0X0
optee: optee_handle_rpc: a0=0XFFFF0005 a1=0XFFFFFF80 a2=0X61E6500
optee: handle_rpc_func_cmd: cmd = 0XA
optee_supp_thrd_req: func=0XA
Introduce a shutdown state in the optee device object to return an
immediate error to all RPC requests in the shutdown path.
Fixes: f25889f93184 ("optee: fix tee out of memory failure seen during kexec reboot
Signed-off-by: Lars Persson <larper(a)axis.com>
---
drivers/tee/optee/optee_private.h | 1 +
drivers/tee/optee/smc_abi.c | 5 ++++-
drivers/tee/optee/supp.c | 8 ++++++++
3 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/drivers/tee/optee/optee_private.h b/drivers/tee/optee/optee_private.h
index 46f74ab07c7e..83380974ff44 100644
--- a/drivers/tee/optee/optee_private.h
+++ b/drivers/tee/optee/optee_private.h
@@ -164,6 +164,7 @@ struct optee {
bool scan_bus_done;
struct workqueue_struct *scan_bus_wq;
struct work_struct scan_bus_work;
+ bool shutting_down;
};
struct optee_session {
diff --git a/drivers/tee/optee/smc_abi.c b/drivers/tee/optee/smc_abi.c
index 449d6a72d289..10af747da816 100644
--- a/drivers/tee/optee/smc_abi.c
+++ b/drivers/tee/optee/smc_abi.c
@@ -1356,7 +1356,10 @@ static int optee_smc_remove(struct platform_device *pdev)
*/
static void optee_shutdown(struct platform_device *pdev)
{
- optee_disable_shm_cache(platform_get_drvdata(pdev));
+ struct optee *optee = platform_get_drvdata(pdev);
+
+ optee->shutting_down = true;
+ optee_disable_shm_cache(optee);
}
static int optee_probe(struct platform_device *pdev)
diff --git a/drivers/tee/optee/supp.c b/drivers/tee/optee/supp.c
index 322a543b8c27..801b4ec659e8 100644
--- a/drivers/tee/optee/supp.c
+++ b/drivers/tee/optee/supp.c
@@ -83,6 +83,14 @@ u32 optee_supp_thrd_req(struct tee_context *ctx, u32 func, size_t num_params,
bool interruptable;
u32 ret;
+ /*
+ * When the system is shutting down we cannot talk
+ * to the supplicant anymore even if we seem to have
+ * an open session to it.
+ */
+ if (optee->shutting_down)
+ return TEEC_ERROR_COMMUNICATION;
+
/*
* Return in case there is no supplicant available and
* non-blocking request.
--
2.30.2