Hi all,
This adds support for asynchronous notifications from OP-TEE in secure
world to the OP-TEE driver. This allows a design with a top half and bottom
half type of driver where the top half runs in secure interrupt context and
a notifications tells normal world to schedule a yielding call to do the
bottom half processing.
An interrupt is used to notify the driver that there are asynchronous
notifications pending.
v2->v3:
* Rebased on v5.14-rc2 which made the patch "dt-bindings: arm: Convert
optee binding to json-schema" from the V2 patch set obsolete.
* Applied Ard's Acked-by on "optee: add asynchronous notifications"
v1->v2:
* Added documentation
* Converted optee bindings to json-schema and added interrupt property
* Configure notification interrupt from DT instead of getting it
from secure world, suggested by Ard Biesheuvel <ardb(a)kernel.org>.
Thanks,
Jens
Jens Wiklander (6):
docs: staging/tee.rst: add a section on OP-TEE notifications
dt-bindings: arm: optee: add interrupt property
tee: fix put order in teedev_close_context()
tee: add tee_dev_open_helper() primitive
optee: separate notification functions
optee: add asynchronous notifications
.../arm/firmware/linaro,optee-tz.yaml | 4 +
Documentation/staging/tee.rst | 27 +++
drivers/tee/optee/Makefile | 1 +
drivers/tee/optee/call.c | 27 +++
drivers/tee/optee/core.c | 87 +++++--
drivers/tee/optee/notif.c | 226 ++++++++++++++++++
drivers/tee/optee/optee_msg.h | 9 +
drivers/tee/optee/optee_private.h | 23 +-
drivers/tee/optee/optee_rpc_cmd.h | 31 +--
drivers/tee/optee/optee_smc.h | 75 +++++-
drivers/tee/optee/rpc.c | 73 +-----
drivers/tee/tee_core.c | 37 ++-
include/linux/tee_drv.h | 27 +++
13 files changed, 523 insertions(+), 124 deletions(-)
create mode 100644 drivers/tee/optee/notif.c
--
2.31.1
Hi all,
This adds supports for the OP-TEE driver to communicate with secure world
using FF-A [1] as transport.
There is one change to the TEE subsystem with "tee: add sec_world_id to
struct tee_shm" to add support for holding globally unique handle assigned
by the FF-A. This is a field that I believe could useful for the AMDTEE
driver too.
For communication the OP-TEE message protocol is still used, but with a new
type of memory reference, struct optee_msg_param_fmem, to carry the
information needed by FF-A. The OP-TEE driver is refactored internally with
to sets of callbacks, one for the old SMC based communication and another
set with FF-A as transport. The functions relating to the SMC based ABI
are moved to smc_abi.c while the FF-A based ABI is added in a ffa_abi.c.
There is also a difference in how the drivers are instantiated. With the
SMC based transport we have a platform driver, module_platform_driver(),
today which we're keeping as is for this configuration. In a FF-A system we
have a FF-A driver, module_ffa_driver(), instead.
The OP-TEE driver can be compiled for both targets at the same time and
it's up to runtime configuration (device tree or ACPI) to decide how it's
initialized. Note that it's only the old SMC based driver instance that
need device tree or ACPI to initialize. The FF-A based driver relies on the
FF-A bus instead.
This can be tested QEMU
The repo for SPMC at S-EL1 retrieved by
repo init -u https://github.com/jenswi-linaro/manifest.git -m
qemu_v8.xml -b ffav4_spmc
repo sync
# Then checkout the branch optee_ffa_v5 from
# git://git.linaro.org/people/jens.wiklander/linux-tee.git
# in the linux directory
To build do:
cd build
make toolchains
make all
To boot:
make run-only
Test with xtest, perhaps only with the command "xtest 1004" in case you're
not interested in too many tests.
Thanks,
Jens
[1] https://developer.arm.com/documentation/den0077/latest
v4->v5:
- Rebased on v5.14, tricky conflicts primarily between "optee: isolate smc
abi" and mostly 376e4199e327 "tee: Correct inappropriate usage of
TEE_SHM_DMA_BUF flag" but also with the other kexec fixes that went into
v5.14-rc5.
- Addressing comments from Sumit and applying
Reviewed-by: Sumit Garg <sumit.garg(a)linaro.org> on "optee: isolate smc abi"
- Addressing comments from Sumit and applying
Acked-by: Sumit Garg <sumit.garg(a)linaro.org> on "optee: add FF-A support"
v3->v4:
- Made a bit more RPC code common between the SMC and FF-A ABIs as
requested by Sumit.
- Replaced module_platform_driver() with module_init()/module_exit() as
described in the commit "optee: isolate smc abi".
- Applied Sumit's R-B for the commits "tee: add sec_world_id to struct
tee_shm", "optee: simplify optee_release()", and "optee: refactor driver
with internal callbacks"
v2->v3:
- Rebased on 5.14-rc2 which now have the FF-A patches merged
- Fixed a couple bugs in optee_shm_register() and optee_shm_unregister()
which where introduced in "optee: refactor driver with internal callbacks"
in previous the version.
- Separated SMC ABI specifics into smc_abi.c to keep it separated from
the FF-A ABI functions as requested by Sumit.
- Added the FF-A specifics in ffa_abi.c
- Provided an implementation for optee_ffa_remove()
v1->v2:
- Rebased to the FF-A v7 patch
- Fixed a couple of reports from kernel test robot <lkp(a)intel.com>
Jens Wiklander (5):
tee: add sec_world_id to struct tee_shm
optee: simplify optee_release()
optee: refactor driver with internal callbacks
optee: isolate smc abi
optee: add FF-A support
drivers/tee/optee/Makefile | 7 +-
drivers/tee/optee/call.c | 445 ++--------
drivers/tee/optee/core.c | 719 ++-------------
drivers/tee/optee/ffa_abi.c | 907 +++++++++++++++++++
drivers/tee/optee/optee_ffa.h | 153 ++++
drivers/tee/optee/optee_msg.h | 27 +-
drivers/tee/optee/optee_private.h | 163 +++-
drivers/tee/optee/rpc.c | 237 +----
drivers/tee/optee/shm_pool.c | 101 ---
drivers/tee/optee/shm_pool.h | 14 -
drivers/tee/optee/smc_abi.c | 1360 +++++++++++++++++++++++++++++
include/linux/tee_drv.h | 7 +-
12 files changed, 2732 insertions(+), 1408 deletions(-)
create mode 100644 drivers/tee/optee/ffa_abi.c
create mode 100644 drivers/tee/optee/optee_ffa.h
delete mode 100644 drivers/tee/optee/shm_pool.c
delete mode 100644 drivers/tee/optee/shm_pool.h
create mode 100644 drivers/tee/optee/smc_abi.c
--
2.31.1
Hi all,
This adds support for asynchronous notifications from OP-TEE in secure
world to the OP-TEE driver. This allows a design with a top half and bottom
half type of driver where the top half runs in secure interrupt context and
a notifications tells normal world to schedule a yielding call to do the
bottom half processing.
An edge-triggered interrupt is used to notify the driver that there are
asynchronous notifications pending.
The documentation and DT bindings patches are now well reviewed, but
the patches with code would do with some more attention.
v4->v5:
* Rebased on v5.14-rc7
* Updated documentation to clarify that one interrupt may represent multiple
notifications as requested.
* Applied Marc's and Rob's tags
v3->v4:
* Clarfied the expected type of interrypt is edge-triggered, both in
the normal documentation and in the DT bindings as requested.
v2->v3:
* Rebased on v5.14-rc2 which made the patch "dt-bindings: arm: Convert
optee binding to json-schema" from the V2 patch set obsolete.
* Applied Ard's Acked-by on "optee: add asynchronous notifications"
v1->v2:
* Added documentation
* Converted optee bindings to json-schema and added interrupt property
* Configure notification interrupt from DT instead of getting it
from secure world, suggested by Ard Biesheuvel <ardb(a)kernel.org>.
Thanks,
Jens
Jens Wiklander (6):
docs: staging/tee.rst: add a section on OP-TEE notifications
dt-bindings: arm: optee: add interrupt property
tee: fix put order in teedev_close_context()
tee: add tee_dev_open_helper() primitive
optee: separate notification functions
optee: add asynchronous notifications
.../arm/firmware/linaro,optee-tz.yaml | 7 +
Documentation/staging/tee.rst | 30 +++
drivers/tee/optee/Makefile | 1 +
drivers/tee/optee/call.c | 27 +++
drivers/tee/optee/core.c | 87 +++++--
drivers/tee/optee/notif.c | 226 ++++++++++++++++++
drivers/tee/optee/optee_msg.h | 9 +
drivers/tee/optee/optee_private.h | 23 +-
drivers/tee/optee/optee_rpc_cmd.h | 31 +--
drivers/tee/optee/optee_smc.h | 75 +++++-
drivers/tee/optee/rpc.c | 73 +-----
drivers/tee/tee_core.c | 37 ++-
include/linux/tee_drv.h | 27 +++
13 files changed, 529 insertions(+), 124 deletions(-)
create mode 100644 drivers/tee/optee/notif.c
--
2.31.1
Hi all,
This adds supports for the OP-TEE driver to communicate with secure world
using FF-A [1] as transport.
There is one change to the TEE subsystem with "tee: add sec_world_id to
struct tee_shm" to add support for holding globally unique handle assigned
by the FF-A. This is a field that I believe could useful for the AMDTEE
driver too.
For communication the OP-TEE message protocol is still used, but with a new
type of memory reference, struct optee_msg_param_fmem, to carry the
information needed by FF-A. The OP-TEE driver is refactored internally with
to sets of callbacks, one for the old SMC based communication and another
set with FF-A as transport. The functions relating to the SMC based ABI
are moved to smc_abi.c while the FF-A based ABI is added in a ffa_abi.c.
There is also a difference in how the drivers are instantiated. With the
SMC based transport we have a platform driver, module_platform_driver(),
today which we're keeping as is for this configuration. In a FF-A system we
have a FF-A driver, module_ffa_driver(), instead.
The OP-TEE driver can be compiled for both targets at the same time and
it's up to runtime configuration (device tree or ACPI) to decide how it's
initialized. Note that it's only the old SMC based driver instance that
need device tree or ACPI to initialize. The FF-A based driver relies on the
FF-A bus instead.
This can be tested QEMU
The repo for SPMC at S-EL1 retrieved by
repo init -u https://github.com/jenswi-linaro/manifest.git -m
qemu_v8.xml -b ffav4_spmc
repo sync
# Then checkout the branch optee_ffa_v4 from
# git://git.linaro.org/people/jens.wiklander/linux-tee.git
# in the linux directory
To build do:
cd build
make toolchains
make all
To boot:
make run-only
Test with xtest, perhaps only with the command "xtest 1004" in case you're
not interested in too many tests.
Thanks,
Jens
[1] https://developer.arm.com/documentation/den0077/latest
v3->v4:
- Made a bit more RPC code common between the SMC and FF-A ABIs as
requested by Sumit.
- Replaced module_platform_driver() with module_init()/module_exit() as
described in the commit "optee: isolate smc abi".
- Applied Sumit's R-B for the commits "tee: add sec_world_id to struct
tee_shm", "optee: simplify optee_release()", and "optee: refactor driver
with internal callbacks"
v2->v3:
- Rebased on 5.14-rc2 which now have the FF-A patches merged
- Fixed a couple bugs in optee_shm_register() and optee_shm_unregister()
which where introduced in "optee: refactor driver with internal callbacks"
in previous the version.
- Separated SMC ABI specifics into smc_abi.c to keep it separated from
the FF-A ABI functions as requested by Sumit.
- Added the FF-A specifics in ffa_abi.c
- Provided an implementation for optee_ffa_remove()
v1->v2:
- Rebased to the FF-A v7 patch
- Fixed a couple of reports from kernel test robot <lkp(a)intel.com>
Jens Wiklander (5):
tee: add sec_world_id to struct tee_shm
optee: simplify optee_release()
optee: refactor driver with internal callbacks
optee: isolate smc abi
optee: add FF-A support
drivers/tee/optee/Makefile | 7 +-
drivers/tee/optee/call.c | 415 ++-------
drivers/tee/optee/core.c | 685 ++-------------
drivers/tee/optee/ffa_abi.c | 907 ++++++++++++++++++++
drivers/tee/optee/optee_ffa.h | 153 ++++
drivers/tee/optee/optee_msg.h | 27 +-
drivers/tee/optee/optee_private.h | 162 +++-
drivers/tee/optee/rpc.c | 236 +-----
drivers/tee/optee/shm_pool.c | 89 --
drivers/tee/optee/shm_pool.h | 14 -
drivers/tee/optee/smc_abi.c | 1299 +++++++++++++++++++++++++++++
include/linux/tee_drv.h | 7 +-
12 files changed, 2665 insertions(+), 1336 deletions(-)
create mode 100644 drivers/tee/optee/ffa_abi.c
create mode 100644 drivers/tee/optee/optee_ffa.h
delete mode 100644 drivers/tee/optee/shm_pool.c
delete mode 100644 drivers/tee/optee/shm_pool.h
create mode 100644 drivers/tee/optee/smc_abi.c
--
2.31.1
Hi,
The meeting was supposed to take place today, but it's still vacation time
and we have nothing on the agenda, hence I'm cancelling this month's
meeting. Next month things should be back to normal, so see you then.
Regards,
Joakim on behalf of the Linaro OP-TEE team
Hi all,
This adds support for asynchronous notifications from OP-TEE in secure
world to the OP-TEE driver. This allows a design with a top half and bottom
half type of driver where the top half runs in secure interrupt context and
a notifications tells normal world to schedule a yielding call to do the
bottom half processing.
An edge-triggered interrupt is used to notify the driver that there are
asynchronous notifications pending.
v3->v4:
* Clarfied the expected type of interrypt is edge-triggered, both in
the normal documentation and in the DT bindings as requested.
v2->v3:
* Rebased on v5.14-rc2 which made the patch "dt-bindings: arm: Convert
optee binding to json-schema" from the V2 patch set obsolete.
* Applied Ard's Acked-by on "optee: add asynchronous notifications"
v1->v2:
* Added documentation
* Converted optee bindings to json-schema and added interrupt property
* Configure notification interrupt from DT instead of getting it
from secure world, suggested by Ard Biesheuvel <ardb(a)kernel.org>.
Thanks,
Jens
Jens Wiklander (6):
docs: staging/tee.rst: add a section on OP-TEE notifications
dt-bindings: arm: optee: add interrupt property
tee: fix put order in teedev_close_context()
tee: add tee_dev_open_helper() primitive
optee: separate notification functions
optee: add asynchronous notifications
.../arm/firmware/linaro,optee-tz.yaml | 7 +
Documentation/staging/tee.rst | 29 +++
drivers/tee/optee/Makefile | 1 +
drivers/tee/optee/call.c | 27 +++
drivers/tee/optee/core.c | 87 +++++--
drivers/tee/optee/notif.c | 226 ++++++++++++++++++
drivers/tee/optee/optee_msg.h | 9 +
drivers/tee/optee/optee_private.h | 23 +-
drivers/tee/optee/optee_rpc_cmd.h | 31 +--
drivers/tee/optee/optee_smc.h | 75 +++++-
drivers/tee/optee/rpc.c | 73 +-----
drivers/tee/tee_core.c | 37 ++-
include/linux/tee_drv.h | 27 +++
13 files changed, 528 insertions(+), 124 deletions(-)
create mode 100644 drivers/tee/optee/notif.c
--
2.31.1
When the system is going to hibernate or suspend it might happen
that the tee-supplicant task is frozen first.
In this case a running OP-TEE task might get stuck in the loop using
wait_for_completion_interruptible to wait for response of tee-supplicant.
As a consequence other OP-TEE tasks waiting for the above or a
succeeding stuck OP-TEE task might get stuck as well
- waiting for call queue entry to be completed
- waiting for OPTEE_RPC_WAIT_QUEUE_WAKEUP
This will result in the tasks "refusing to freeze" and
the hibernate or suspend will fail.
OP-TEE issue: https://github.com/OP-TEE/optee_os/issues/4581
- Read back the object
PM: suspend entry (s2idle)
Filesystems sync: 0.000 seconds
Freezing user space processes ...
Freezing of tasks failed after 20.008 seconds (3 tasks refusing to freeze, wq_busy=0):
task:optee_example_s state:R running task stack: 0 pid: 124 ppid: 1 flags:0x00000001
[<807d3e24>] (__schedule) from [<841c4000>] (0x841c4000)
task:optee_example_s state:D stack: 0 pid: 126 ppid: 1 flags:0x00000001
[<807d3e24>] (__schedule) from [<807d41d0>] (schedule+0x60/0x120)
[<807d41d0>] (schedule) from [<807d7ffc>] (schedule_timeout+0x1f4/0x340)
[<807d7ffc>] (schedule_timeout) from [<807d56a0>] (wait_for_completion+0x94/0xfc)
[<807d56a0>] (wait_for_completion) from [<80692134>] (optee_cq_wait_for_completion+0x14/0x60)
[<80692134>] (optee_cq_wait_for_completion) from [<806924dc>] (optee_do_call_with_arg+0x14c/0x154)
[<806924dc>] (optee_do_call_with_arg) from [<80692edc>] (optee_shm_unregister+0x78/0xcc)
[<80692edc>] (optee_shm_unregister) from [<80690a9c>] (tee_shm_release+0x88/0x174)
[<80690a9c>] (tee_shm_release) from [<8057f89c>] (dma_buf_release+0x44/0xb0)
[<8057f89c>] (dma_buf_release) from [<8028e4e8>] (__dentry_kill+0x110/0x17c)
[<8028e4e8>] (__dentry_kill) from [<80276cfc>] (__fput+0xc0/0x234)
[<80276cfc>] (__fput) from [<80140b1c>] (task_work_run+0x90/0xbc)
[<80140b1c>] (task_work_run) from [<8010b1c8>] (do_work_pending+0x4a0/0x5a0)
[<8010b1c8>] (do_work_pending) from [<801000cc>] (slow_work_pending+0xc/0x20)
Exception stack(0x843f5fb0 to 0x843f5ff8)
5fa0: 00000000 7ef63448 fffffffe 00000000
5fc0: 7ef63448 76f163b0 7ef63448 00000006 7ef63448 7ef634e0 7ef63438 00000000
5fe0: 00000006 7ef63400 76e74833 76dff856 800e0130 00000004
task:optee_example_s state:D stack: 0 pid: 128 ppid: 1 flags:0x00000001
[<807d3e24>] (__schedule) from [<807d41d0>] (schedule+0x60/0x120)
[<807d41d0>] (schedule) from [<807d7ffc>] (schedule_timeout+0x1f4/0x340)
[<807d7ffc>] (schedule_timeout) from [<807d56a0>] (wait_for_completion+0x94/0xfc)
[<807d56a0>] (wait_for_completion) from [<8069359c>] (optee_handle_rpc+0x554/0x710)
[<8069359c>] (optee_handle_rpc) from [<806924cc>] (optee_do_call_with_arg+0x13c/0x154)
[<806924cc>] (optee_do_call_with_arg) from [<80692910>] (optee_invoke_func+0x110/0x190)
[<80692910>] (optee_invoke_func) from [<8068fe3c>] (tee_ioctl+0x113c/0x1244)
[<8068fe3c>] (tee_ioctl) from [<802892ec>] (sys_ioctl+0xe0/0xa24)
[<802892ec>] (sys_ioctl) from [<80100060>] (ret_fast_syscall+0x0/0x54)
Exception stack(0x8424ffa8 to 0x8424fff0)
ffa0: 00000000 7eb67584 00000003 8010a403 7eb67438 7eb675fc
ffc0: 00000000 7eb67584 7eb67604 00000036 7eb67448 7eb674e0 7eb67438 00000000
ffe0: 76ef7030 7eb6742c 76ee6469 76e83178
OOM killer enabled.
Restarting tasks ... done.
PM: suspend exit
sh: write error: Device or resource busy
The patch set will switch to interruptible waits and add try_to_freeze to allow the waiting
OP-TEE tasks to be frozen as well.
---
In my humble understanding without these patches OP-TEE tasks have only been frozen in user-space.
With these patches it is possible that OP-TEE tasks are frozen although the OP-TEE command
invocation didn't complete.
I'm unable to judge if there are any OP-TEE implementations relying on the fact that suspend won't
happen while the OP-TEE command invocation didn't complete.
The theoretical alternative would be to prevent that tee-supplicant is frozen first.
I was able to reproduce the issue in OP-TEE QEMU v7 using a modified version of
optee_example_secure_storage (loop around REE FS read, support multi-session).
See https://github.com/OP-TEE/optee_os/issues/4581 for details.
After applying these patches (minor adjustments of the includes) I was no longer able to
reproduce the issues.
In my tests OP-TEE QEMU v7 did suspend and resume without troubles.
I'm not able to test on other devices supporting OP-TEE.
I decided to handle each of the locations the OP-TEE task could get stuck as a separate commit.
The downside is that the above call stack doesn't really fit to any of the commits.
Christoph Gellner (3):
tee: optee: Allow to freeze the task waiting for tee-supplicant
tee: optee: Allow to freeze while waiting for call_queue
tee: optee: Allow to freeze while waiting in
OPTEE_RPC_WAIT_QUEUE_SLEEP
drivers/tee/optee/call.c | 8 +++++++-
drivers/tee/optee/rpc.c | 9 ++++++++-
drivers/tee/optee/supp.c | 3 +++
3 files changed, 18 insertions(+), 2 deletions(-)
base-commit: c4681547bcce777daf576925a966ffa824edd09d
--
2.32.0.rc0
Hello arm-soc maintainers,
Please pull these fixes relating to OP-TEE, ftpm (firmware TPM), and
tee_bnxt_fw (Broadcom BNXT firmware manager) drivers in kexec and kdump
(emergency kexec) based workflows.
The two patches "firmware: tee_bnxt: Release TEE shm, session, and context
during kexec" and "tpm_ftpm_tee: Free and unregister TEE shared memory
during kexec" are acked by their respective maintainers.
For more details please see the description of the last patch set
https://lore.kernel.org/lkml/20210614223317.999867-1-tyhicks@linux.microsof…
Thanks,
Jens
The following changes since commit 2734d6c1b1a089fb593ef6a23d4b70903526fe0c:
Linux 5.14-rc2 (2021-07-18 14:13:49 -0700)
are available in the Git repository at:
git://git.linaro.org:/people/jens.wiklander/linux-tee.git tags/tee-kexec-fixes-for-v5.14
for you to fetch changes up to 914ab19e471d8fb535ed50dff108b0a615f3c2d8:
firmware: tee_bnxt: Release TEE shm, session, and context during kexec (2021-07-21 07:55:50 +0200)
----------------------------------------------------------------
tee: Improve support for kexec and kdump
This fixes several bugs uncovered while exercising the OP-TEE, ftpm
(firmware TPM), and tee_bnxt_fw (Broadcom BNXT firmware manager) drivers
with kexec and kdump (emergency kexec) based workflows.
----------------------------------------------------------------
Allen Pais (2):
optee: fix tee out of memory failure seen during kexec reboot
firmware: tee_bnxt: Release TEE shm, session, and context during kexec
Jens Wiklander (1):
tee: add tee_shm_alloc_kernel_buf()
Sumit Garg (1):
tee: Correct inappropriate usage of TEE_SHM_DMA_BUF flag
Tyler Hicks (4):
optee: Fix memory leak when failing to register shm pages
optee: Refuse to load the driver under the kdump kernel
optee: Clear stale cache entries during initialization
tpm_ftpm_tee: Free and unregister TEE shared memory during kexec
drivers/char/tpm/tpm_ftpm_tee.c | 8 +++---
drivers/firmware/broadcom/tee_bnxt_fw.c | 14 ++++++++---
drivers/tee/optee/call.c | 38 ++++++++++++++++++++++++++---
drivers/tee/optee/core.c | 43 ++++++++++++++++++++++++++++++++-
drivers/tee/optee/optee_private.h | 1 +
drivers/tee/optee/rpc.c | 5 ++--
drivers/tee/optee/shm_pool.c | 20 ++++++++++++---
drivers/tee/tee_shm.c | 20 ++++++++++++++-
include/linux/tee_drv.h | 2 ++
9 files changed, 132 insertions(+), 19 deletions(-)