Hi, Jens, Olivier, Is it possible to solve this problem by increasing the buffer size such as 64k? We plan to adopt this temporary scheme first and then implement the standard retrieve process later. If this scheme is feasible, I have a few questions. 1. Should the RXTX buffer between tee driver and hafnium the same as the one between hafnium and optee? 2. Which FFA instance is currently allocating these buffers? 3. For Hafnium, I want to increase the HF_MAILBOX_SIZE in Hafnium, but there seem to be two restrictions: static_assert(HF_MAILBOX_SIZE == PAGE_SIZE, "Currently, a page is mapped for the send and receive buffers so " "the maximum request is the size of a page."); static_assert(MM_PPOOL_ENTRY_SIZE >= HF_MAILBOX_SIZE, "The page pool entry size must be at least as big as the mailbox " "size, so that memory region descriptors can be copied from the " "mailbox for memory sharing."); I tried to increase the MM_PPOOL_ENTRY_SIZE, but optee doesn't seem to be booted properly. Do you know how to avoid the impact of these restrictions? Regards, Yuye. ------------------------------------------------------------------ 发件人：Jens Wiklander via Hafnium hafnium@lists.trustedfirmware.org 发送时间：2023年4月14日(星期五) 14:23 收件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 抄 送：Olivier Deprez Olivier.Deprez@arm.com; hafnium hafnium@lists.trustedfirmware.org; op-tee op-tee@lists.trustedfirmware.org 主 题：[Hafnium] Re: fragment transmission while retrieving memory Hi Yuye, On Fri, Apr 14, 2023 at 4:29 AM 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com wrote:
Hi, Jens, Olivier,
If I understand properly, for this specific issue both optee and hafnium need to do further fix. The following discussion and questions are based on figure 18.4 in the FFA spec.
As I saw in the optee code, the retrieve mechanism would start with this code: retrieve_desc = spmc_retrieve_req(cookie); And in the function spmc_retrieve_req, optee sets the total length to the same size as fragment0 length before sending FFA_MEM_FRAG_REQ to hafnium, which means hafnium will only retrieve the fragment0 memory region.
In the example process, optee will determines the number of TX sized fragments of descriptor before sending FFA_MEM_FRAG_REQ to hafnium. I have a question here. How should optee calculate this quantity?
Hafnium then receives the FFA_MEM_FRAG_REQ. As I saw in the hafnium code, hafnium blocks the case that total length>fragment length. if (fragment_length != length) { dlog_verbose("Fragmentation not yet supported.\n"); return ffa_error(FFA_INVALID_PARAMETERS); } Obviously it is not what we expected. Then following the example process, hafnium should allocate handle and use it to associate fragments. I didn't find the corresponding implementation in hafnium code for this step. And I want to know how to implement the associate action here.
After that, optee lacks the implementation of FFA_MEM_FRAG_RX and FFA_MEM_FRAG_TX ABI in CFG_CORE_SEL2_SPMC enable case. Is there any support plan for the implementation of fragmented memory retrieve in the optee community? Does the hafnium community have a plan to implement it cooperatively?
If I've understood this correctly we have two different cases of eventual fragmentation of descriptors. The first is the descriptor passed with FFA_MEM_RETRIEVE_REQ_32, this one is not likely to need fragmentation support soon. The other is the descriptor received with FFA_MEM_RETRIEVE_RESP, I believe this is where you've run into trouble. From OP-TEE point of view, it would make sense if you took lead on implementing this. I can help with review etc. Thanks, Jens
Thanks a lot for your support.
Regards, Yuye.
发件人：Olivier Deprez Olivier.Deprez@arm.com 发送时间：2023年4月13日(星期四) 15:52 收件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 抄 送：hafnium hafnium@lists.trustedfirmware.org; Jens Wiklander jens.wiklander@linaro.org 主 题：Re: fragment transmission while retrieving memory
Hi Yuye, Jens,
For the record, and If I understand properly the last comment in the github issue: OP-TEE is missing the implementation for receipt of a fragmented retrieve response There is no further fix to be done in Hafnium for this specific issue at this moment, please let me know otherwise.
Thanks, Olivier.
From: 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com Sent: 12 April 2023 09:27 To: Olivier Deprez Olivier.Deprez@arm.com Cc: hafnium hafnium@lists.trustedfirmware.org; Jens Wiklander jens.wiklander@linaro.org Subject: Re: fragment transmission while retrieving memory
Hi, Olivier,
In our setup, Hafnium commit: 997476a74571aec4f1a23590d45edf516f3934f4 optee version: 3.20.0 Thanks.
Regards, Yuye.
发件人：Olivier Deprez Olivier.Deprez@arm.com 发送时间：2023年4月12日(星期三) 15:22 收件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 抄 送：hafnium hafnium@lists.trustedfirmware.org; Jens Wiklander jens.wiklander@linaro.org 主 题：Re: fragment transmission while retrieving memory
Hi Yuye,
AFAIK concerning Hafnium, fragmented mem sharing is supported for FFA_MEM_SHARE/LEND/DONATE and FFA_MEM_RETRIEVE_RESP. (For the sake of clarity, this isn't supported for FFA_MEM_RETRIEVE_REQ but this shouldn't be a concern as this limitation only exists in the case of mem sharing to multiple borrowers. In your case of a single borrower the mem retrieve req. shouldn't have to be fragmented).
Can you tell which hafnium commit hash is used in your setup?
At the moment, I cannot tell if the issue described concern a miss in Hafnium or OP-TEE. I need to dig a bit further into both implementations and I'll let you know.
Regards, Olivier.
From: 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com Sent: 12 April 2023 08:21 To: Olivier Deprez Olivier.Deprez@arm.com Cc: hafnium hafnium@lists.trustedfirmware.org; Jens Wiklander jens.wiklander@linaro.org Subject: fragment transmission while retrieving memory
Hi, Olivier,
Recently, I've been working on this issue. https://github.com/OP-TEE/optee_os/issues/5943 <https://github.com/OP-TEE/optee_os/issues/5943 > Do you know any differences between hafnium's current implementation of fragment transmission while retrieving memory and the example process described in FF-A 1.1 Figure 18.4?
Regards, Yuye.
Hi Yuye,
At first sight, I wouldn't recommend this path of increasing RX/TX buffer sizes (even though the spec permits it), as there are a lot of assumptions in many places for such buffers to be a 4K+4K pair. The would require adapting many components (kernel driver, hafnium, OP-TEE etc.)
The fastest and more reasonable path affecting only OP-TEE to my understanding, would be with implementing the missing support for fragmented mem retrieve response as Jens suggested.
Regards, Olivier.
________________________________ From: 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com Sent: 14 April 2023 11:06 To: Jens Wiklander jens.wiklander@linaro.org; Olivier Deprez Olivier.Deprez@arm.com Cc: op-tee op-tee@lists.trustedfirmware.org; hafnium hafnium@lists.trustedfirmware.org Subject: fragment transmission while retrieving memory
Hi, Jens, Olivier,
Is it possible to solve this problem by increasing the buffer size such as 64k? We plan to adopt this temporary scheme first and then implement the standard retrieve process later. If this scheme is feasible, I have a few questions. 1. Should the RXTX buffer between tee driver and hafnium the same as the one between hafnium and optee? 2. Which FFA instance is currently allocating these buffers? 3. For Hafnium, I want to increase the HF_MAILBOX_SIZE in Hafnium, but there seem to be two restrictions: static_assert(HF_MAILBOX_SIZE == PAGE_SIZE, "Currently, a page is mapped for the send and receive buffers so " "the maximum request is the size of a page."); static_assert(MM_PPOOL_ENTRY_SIZE >= HF_MAILBOX_SIZE, "The page pool entry size must be at least as big as the mailbox " "size, so that memory region descriptors can be copied from the " "mailbox for memory sharing."); I tried to increase the MM_PPOOL_ENTRY_SIZE, but optee doesn't seem to be booted properly. Do you know how to avoid the impact of these restrictions?
Regards, Yuye.
------------------------------------------------------------------ 发件人：Jens Wiklander via Hafnium hafnium@lists.trustedfirmware.org 发送时间：2023年4月14日(星期五) 14:23 收件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 抄 送：Olivier Deprez Olivier.Deprez@arm.com; hafnium hafnium@lists.trustedfirmware.org; op-tee op-tee@lists.trustedfirmware.org 主 题：[Hafnium] Re: fragment transmission while retrieving memory
Hi Yuye,
On Fri, Apr 14, 2023 at 4:29 AM 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com wrote:
Hi, Jens, Olivier,
If I understand properly, for this specific issue both optee and hafnium need to do further fix. The following discussion and questions are based on figure 18.4 in the FFA spec.
As I saw in the optee code, the retrieve mechanism would start with this code: retrieve_desc = spmc_retrieve_req(cookie); And in the function spmc_retrieve_req, optee sets the total length to the same size as fragment0 length before sending FFA_MEM_FRAG_REQ to hafnium, which means hafnium will only retrieve the fragment0 memory region.
In the example process, optee will determines the number of TX sized fragments of descriptor before sending FFA_MEM_FRAG_REQ to hafnium. I have a question here. How should optee calculate this quantity?
Hafnium then receives the FFA_MEM_FRAG_REQ. As I saw in the hafnium code, hafnium blocks the case that total length>fragment length. if (fragment_length != length) { dlog_verbose("Fragmentation not yet supported.\n"); return ffa_error(FFA_INVALID_PARAMETERS); } Obviously it is not what we expected. Then following the example process, hafnium should allocate handle and use it to associate fragments. I didn't find the corresponding implementation in hafnium code for this step. And I want to know how to implement the associate action here.
After that, optee lacks the implementation of FFA_MEM_FRAG_RX and FFA_MEM_FRAG_TX ABI in CFG_CORE_SEL2_SPMC enable case. Is there any support plan for the implementation of fragmented memory retrieve in the optee community? Does the hafnium community have a plan to implement it cooperatively?
If I've understood this correctly we have two different cases of eventual fragmentation of descriptors. The first is the descriptor passed with FFA_MEM_RETRIEVE_REQ_32, this one is not likely to need fragmentation support soon. The other is the descriptor received with FFA_MEM_RETRIEVE_RESP, I believe this is where you've run into trouble.
From OP-TEE point of view, it would make sense if you took lead on implementing this. I can help with review etc.
Thanks, Jens
Thanks a lot for your support.
Regards, Yuye.
发件人：Olivier Deprez Olivier.Deprez@arm.com 发送时间：2023年4月13日(星期四) 15:52 收件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 抄 送：hafnium hafnium@lists.trustedfirmware.org; Jens Wiklander jens.wiklander@linaro.org 主 题：Re: fragment transmission while retrieving memory
Hi Yuye, Jens,
For the record, and If I understand properly the last comment in the github issue: OP-TEE is missing the implementation for receipt of a fragmented retrieve response There is no further fix to be done in Hafnium for this specific issue at this moment, please let me know otherwise.
Thanks, Olivier.
From: 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com Sent: 12 April 2023 09:27 To: Olivier Deprez Olivier.Deprez@arm.com Cc: hafnium hafnium@lists.trustedfirmware.org; Jens Wiklander jens.wiklander@linaro.org Subject: Re: fragment transmission while retrieving memory
Hi, Olivier,
In our setup, Hafnium commit: 997476a74571aec4f1a23590d45edf516f3934f4 optee version: 3.20.0 Thanks.
Regards, Yuye.
发件人：Olivier Deprez Olivier.Deprez@arm.com 发送时间：2023年4月12日(星期三) 15:22 收件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 抄 送：hafnium hafnium@lists.trustedfirmware.org; Jens Wiklander jens.wiklander@linaro.org 主 题：Re: fragment transmission while retrieving memory
Hi Yuye,
AFAIK concerning Hafnium, fragmented mem sharing is supported for FFA_MEM_SHARE/LEND/DONATE and FFA_MEM_RETRIEVE_RESP. (For the sake of clarity, this isn't supported for FFA_MEM_RETRIEVE_REQ but this shouldn't be a concern as this limitation only exists in the case of mem sharing to multiple borrowers. In your case of a single borrower the mem retrieve req. shouldn't have to be fragmented).
Can you tell which hafnium commit hash is used in your setup?
At the moment, I cannot tell if the issue described concern a miss in Hafnium or OP-TEE. I need to dig a bit further into both implementations and I'll let you know.
Regards, Olivier.
From: 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com Sent: 12 April 2023 08:21 To: Olivier Deprez Olivier.Deprez@arm.com Cc: hafnium hafnium@lists.trustedfirmware.org; Jens Wiklander jens.wiklander@linaro.org Subject: fragment transmission while retrieving memory
Hi, Olivier,
Recently, I've been working on this issue. https://github.com/OP-TEE/optee_os/issues/5943 Do you know any differences between hafnium's current implementation of fragment transmission while retrieving memory and the example process described in FF-A 1.1 Figure 18.4?
Regards, Yuye.
-- Hafnium mailing list -- hafnium@lists.trustedfirmware.org To unsubscribe send an email to hafnium-leave@lists.trustedfirmware.org
op-tee@lists.trustedfirmware.org
-
Olivier Deprez -
梅建强(禹夜)