Hi, Olivier I have forgotten one thing, hafnium will invalidate the data cache for the whole image. adrp x0, ORIGIN_ADDRESS adrp x1, image_end sub x1, x1, x0 bl arch_cache_data_invalidate_range I solved this problem by using an address-fixed variable outside the data section instead of the global variabal malloced in data section. In addition, I would like to ask that if multiple optees run on the Hafnium, how to switch the vCPUs of different optees? If one of the optees is expected to be boot under the host OS (post-boot load), how to switch the vCPUs? Thanks for the support. Regards, Yuye. ------------------------------------------------------------------ 发件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 发送时间：2023年3月11日(星期六) 21:36 收件人：Olivier Deprez Olivier.Deprez@arm.com 抄 送：Jens Wiklander jens.wiklander@linaro.org; hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org 主 题：Re: Multi-OPTEE run with Hafnium Hi, Olivier, The framework message definition that I mentioned is refered in the FF-A Specification. I hope I didn't misinterpret it. The work flow is interpreted as follows: When the system run into the host OS, I want implemented the post-boot load optee as mentioned in the email with Hafnium as SPMC https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/18635 <https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/18635 > However, when the intialization about hafnium and optee has been finished, SMC_RET8 instruction failed reback from spmd to optee driver. I suspect that there may be some special handling of context needed here. Can you provide some suggestions or help? Regards, Yuye. ------------------------------------------------------------------ 发件人：Olivier Deprez Olivier.Deprez@arm.com 发送时间：2023年3月7日(星期二) 16:55 收件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 抄 送：Jens Wiklander jens.wiklander@linaro.org; hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org 主 题：Re: Multi-OPTEE run with Hafnium Hi Yuye, I sense those questions are related to downstream changes of yours, and I may not be able to provide accurate answers without seeing the code. Please clarify if the questions are related to the upstream version of Hafnium. See additional comments below [OD]. Regards, Olivier. From: 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com Sent: 06 March 2023 14:15 To: Olivier Deprez Olivier.Deprez@arm.com Cc: Jens Wiklander jens.wiklander@linaro.org Subject: Re: Multi-OPTEE run with Hafnium Hi, Olivier, On this question, I should add something based on the results of my tests. When I enter Hafnium through framwork message, the value of the global variable remains the same. [OD] please explain what is a framework message. When I re-enter the Hafnium image_entry via spmd_init, the global variable is restored to its original value. [OD] I'm not sure about the flow here, it depends about where the variable is declared and whether you have local changes to the hafnium boot flows. Do you know why? Regards, Yuye. ------------------------------------------------------------------ 发件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 发送时间：2023年3月6日(星期一) 17:00 收件人：Olivier Deprez olivier.deprez@arm.com 抄 送：Jens Wiklander jens.wiklander@linaro.org 主 题：Multi-OPTEE run with Hafnium Hi, Olivier, I want to ask a question about variable modification. Before this, let me explain the background about the question. Now, I have run into host OS with Hafnium as spmc and OPTEE as SP. Then the optee driver will send a framwork message to Hafnium to inform it that whether the current state of the host OS is booting or running, [OD] What is a 'framework message' in this context? Is this a custom direct request from driver to optee? so that Hafnium could know which vm node(SP) it should load and init at current state. [OD] Can you please explain why this is required? This current state is represented by a global variable. [OD] do you mean a variable in Hafnium's BSS space? Will the global variable change when message flow exit Hafnium and enter it again? [OD] This depends in which section the variable is declared (see above). Regards, Yuye.
Hi Yuye,
See comments inline [OD].
Regards, Olivier.
________________________________ From: 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com Sent: 13 March 2023 03:16 To: Olivier Deprez Olivier.Deprez@arm.com Cc: Jens Wiklander jens.wiklander@linaro.org; hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org Subject: Re: Multi-OPTEE run with Hafnium
Hi, Olivier
I have forgotten one thing, hafnium will invalidate the data cache for the whole image. adrp x0, ORIGIN_ADDRESS adrp x1, image_end sub x1, x1, x0 bl arch_cache_data_invalidate_range I solved this problem by using an address-fixed variable outside the data section instead of the global variabal malloced in data section.
[OD] Ok... I'm not too sure about the implications. This boot time invalidation is required otherwise you may end up with stale data in the bss/data sections when enabling MMU/turning D-cache on. There is only static allocation in Hafnium code base so I'm not sure what you mean by 'global variable malloced in data section'. If you're locating data outside of bss/data sections it's likely a text section (?) or some random space that is beyond Hafnium's control?...
In addition, I would like to ask that if multiple optees run on the Hafnium, how to switch the vCPUs of different optees? If one of the optees is expected to be boot under the host OS (post-boot load), how to switch the vCPUs?
[OD] This is currently unsupported AFAIK. I believe it would require adding support for multiple OP-TEE instances into the OP-TEE normal world driver (unless leveraging CFG_VIRTUALIZATION option which I don't believe is designed for this).
Thanks for the support.
Regards, Yuye.
------------------------------------------------------------------ 发件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 发送时间：2023年3月11日(星期六) 21:36 收件人：Olivier Deprez Olivier.Deprez@arm.com 抄 送：Jens Wiklander jens.wiklander@linaro.org; hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org 主 题：Re: Multi-OPTEE run with Hafnium
Hi, Olivier,
The framework message definition that I mentioned is refered in the FF-A Specification. I hope I didn't misinterpret it.
The work flow is interpreted as follows: When the system run into the host OS, I want implemented the post-boot load optee as mentioned in the email with Hafnium as SPMC https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/18635
[OD] AFAIU this option was designed in context of Chrome OS which has specific platform assumptions and threat model. At first sight, I don't believe this option is compatible with FF-A/Hafnium SW architecture without significant re-design. As indicated earlier, if the intent is to implement a FW update mechanism, it'd be prefereable to use an Arm standard e.g. https://developer.arm.com/documentation/den0118/latest
However, when the intialization about hafnium and optee has been finished, SMC_RET8 instruction failed reback from spmd to optee driver. I suspect that there may be some special handling of context needed here. Can you provide some suggestions or help?
Regards, Yuye.
------------------------------------------------------------------ 发件人：Olivier Deprez Olivier.Deprez@arm.com 发送时间：2023年3月7日(星期二) 16:55 收件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 抄 送：Jens Wiklander jens.wiklander@linaro.org; hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org 主 题：Re: Multi-OPTEE run with Hafnium
Hi Yuye,
I sense those questions are related to downstream changes of yours, and I may not be able to provide accurate answers without seeing the code. Please clarify if the questions are related to the upstream version of Hafnium. See additional comments below [OD].
Regards, Olivier.
________________________________ From: 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com Sent: 06 March 2023 14:15 To: Olivier Deprez Olivier.Deprez@arm.com Cc: Jens Wiklander jens.wiklander@linaro.org Subject: Re: Multi-OPTEE run with Hafnium
Hi, Olivier,
On this question, I should add something based on the results of my tests.
When I enter Hafnium through framwork message, the value of the global variable remains the same.
[OD] please explain what is a framework message.
When I re-enter the Hafnium image_entry via spmd_init, the global variable is restored to its original value.
[OD] I'm not sure about the flow here, it depends about where the variable is declared and whether you have local changes to the hafnium boot flows.
Do you know why?
Regards, Yuye.
------------------------------------------------------------------ 发件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 发送时间：2023年3月6日(星期一) 17:00 收件人：Olivier Deprez olivier.deprez@arm.com 抄 送：Jens Wiklander jens.wiklander@linaro.org 主 题：Multi-OPTEE run with Hafnium
Hi, Olivier,
I want to ask a question about variable modification. Before this, let me explain the background about the question.
Now, I have run into host OS with Hafnium as spmc and OPTEE as SP. Then the optee driver will send a framwork message to Hafnium to inform it that whether the current state of the host OS is booting or running, [OD] What is a 'framework message' in this context? Is this a custom direct request from driver to optee? so that Hafnium could know which vm node(SP) it should load and init at current state. [OD] Can you please explain why this is required? This current state is represented by a global variable. [OD] do you mean a variable in Hafnium's BSS space?
Will the global variable change when message flow exit Hafnium and enter it again? [OD] This depends in which section the variable is declared (see above).
Regards, Yuye.
Hi,
Comments below.
On Mon, Mar 13, 2023 at 6:17 PM Olivier Deprez Olivier.Deprez@arm.com wrote:
Hi Yuye,
See comments inline [OD].
Regards, Olivier.
From: 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com Sent: 13 March 2023 03:16 To: Olivier Deprez Olivier.Deprez@arm.com Cc: Jens Wiklander jens.wiklander@linaro.org; hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org Subject: Re: Multi-OPTEE run with Hafnium
Hi, Olivier
I have forgotten one thing, hafnium will invalidate the data cache for the whole image. adrp x0, ORIGIN_ADDRESS adrp x1, image_end sub x1, x1, x0 bl arch_cache_data_invalidate_range I solved this problem by using an address-fixed variable outside the data section instead of the global variabal malloced in data section.
[OD] Ok... I'm not too sure about the implications. This boot time invalidation is required otherwise you may end up with stale data in the bss/data sections when enabling MMU/turning D-cache on. There is only static allocation in Hafnium code base so I'm not sure what you mean by 'global variable malloced in data section'. If you're locating data outside of bss/data sections it's likely a text section (?) or some random space that is beyond Hafnium's control?...
In addition, I would like to ask that if multiple optees run on the Hafnium, how to switch the vCPUs of different optees? If one of the optees is expected to be boot under the host OS (post-boot load), how to switch the vCPUs?
[OD] This is currently unsupported AFAIK. I believe it would require adding support for multiple OP-TEE instances into the OP-TEE normal world driver (unless leveraging CFG_VIRTUALIZATION option which I don't believe is designed for this).
Another OP-TEE instance should have another UUID so some modifications in the normal world driver would be needed. CFG_VIRTUALIZATION is only intended to provide a clean instance for each guest in the normal world, but it's still a single SP from FF-A point of view.
Cheers, Jens
hafnium@lists.trustedfirmware.org
-
Jens Wiklander -
Olivier Deprez -
梅建强(禹夜)