Hi, Olivier,
This issue has not been resolved and further support is needed.
Then I'll try to update our code base to see if anything changes.
Regards,
Yuye.
------------------------------------------------------------------
发件人:Olivier Deprez <Olivier.Deprez(a)arm.com>
发送时间:2023年2月11日(星期六) 00:57
收件人:hafnium <hafnium(a)lists.trustedfirmware.org>; 梅建强(禹夜) <meijianqiang.mjq(a)alibaba-inc.com>
抄 送:常琳(特睿) <terui.cl(a)alibaba-inc.com>; 赵哲(为哲) <weizhe.zz(a)alibaba-inc.com>; 黄明(连一) <hm281385(a)alibaba-inc.com>
主 题:Re: Hafnium load-address
Hi Yuye,
Is this still an issue at your end? Just asking if we need to do further investigation.
Regards,
Olivier.
From: 梅建强(禹夜) via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 28 December 2022 10:09
To: hafnium <hafnium(a)lists.trustedfirmware.org>
Cc: 常琳(特睿) <terui.cl(a)alibaba-inc.com>; 赵哲(为哲) <weizhe.zz(a)alibaba-inc.com>; 黄明(连一) <hm281385(a)alibaba-inc.com>
Subject: [Hafnium] 回复:Re: Hafnium load-address
Hi,
Thanks for your efforts for this query. I have tried your solution. Although the Hafnium code could be built successfully, it does not work when the boot flow run into the entry of the Hafnium.
I tried to debug the error with trace32 tool. And I confirmed that the Hafnium code has been loaded over 34G of free address space. However, when PC register point to the start address of Hafnium, the code data is displayed as udf #0x0.
It may be a page table problem, and we are still locating the problem reason.
Regards,
Mei, Jianqiang.
------------------------------------------------------------------
发件人:Olivier Deprez via Hafnium <hafnium(a)lists.trustedfirmware.org>
发送时间:2022年12月21日(星期三) 18:02
收件人:梅建强(禹夜) <meijianqiang.mjq(a)alibaba-inc.com>
抄 送:hafnium <hafnium(a)lists.trustedfirmware.org>; 常琳(特睿) <terui.cl(a)alibaba-inc.com>; 赵哲(为哲) <weizhe.zz(a)alibaba-inc.com>; 黄明(连一) <hm281385(a)alibaba-inc.com>
主 题:[Hafnium] Re: Hafnium load-address
Hi,
Sorry for the delay in replying to this query.
After few trials I ended with this possible solution:
https://review.trustedfirmware.org/c/hafnium/hafnium/+/18510 <https://review.trustedfirmware.org/c/hafnium/hafnium/+/18510 > < < >https://review.trustedfirmware.org/c/hafnium/hafnium/+/18510 <https://review.trustedfirmware.org/c/hafnium/hafnium/+/18510 > >
This should permit loading the Hafnium image to an address up to 48 bits (and by updating project/reference/BUILD.gn origin_address per your suggestion).
It is possible to further extend beyond 48 bits (e.g. 52 bits which is the maximum permitted VA using FEAT_LPA2).
Let me know if this works for you, then I can proceed further with polishing this change.
Regards,
Olivier.
________________________________
From: 梅建强(禹夜) via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 11 December 2022 09:58
To: Olivier Deprez <Olivier.Deprez(a)arm.com>
Cc: hafnium <hafnium(a)lists.trustedfirmware.org>; 常琳(特睿) <terui.cl(a)alibaba-inc.com>; 赵哲(为哲) <weizhe.zz(a)alibaba-inc.com>; 黄明(连一) <hm281385(a)alibaba-inc.com>
Subject: [Hafnium] Hafnium load-address
Hello, expert
Firstly, thanks for your answer some time ago. That's useful to our project.We combined the Hafnium (run at S-EL2) and the optee (run at S-EL1), then modified the configuration to adapt to our own environment. Now we can start the os normally.
Now we want to optimize a technical point, which is to load and run hafnium over 34G of free address space. This requires hafnium to support 64-bit load-address.
I tried to modify the configuration in BUILD.gn as follows:
origin_address = "0x880000000"
And I encountered the build error:
FAILED: secure_aem_v8a_fvp_clang/hafnium.elf ld.lld -pie --gc-sections --defsym=ORIGIN_ADDRESS=0x880000000 -O2 --icf=all --fatal-warnings --color-diagnostics -T /home/yuye/hafnium/Hafnium/build/image/image.ld -o secure_aem_v8a_fvp_clang/hafnium.elf --start-group @secure_aem_v8a_fvp_clang/hafnium.elf.rsp --end-group ld.lld: error: secure_aem_v8a_fvp_clang/obj/src/arch/aarch64/entry.entry.o:(.init.entry+0xC8): relocation R_AARCH64_ABS32 out of range: 36507222016 is not in [-2147483648, 4294967295]; references ORIGIN_ADDRESS >>> defined in -defsym:1 ninja: build stopped: subcommand failed. Makefile:70: recipe for target 'all' failed make: *** [all] Error 1 + cp out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ../../V2/Ali_M1_Public/Nonosi/Platform/Ali/M1/Bin/ cp: cannot stat 'out/reference/secure_aem_v8a_fvp_clang/hafnium.bin': No such file or directory
In this case, it seems that the new address causes some relative addressing to reach too far. Is there any solutions or supports for this issue currently in the community?
Regards,
Mei, Jianqiang.
--
Hafnium mailing list -- hafnium(a)lists.trustedfirmware.org
To unsubscribe send an email to hafnium-leave(a)lists.trustedfirmware.org
--
Hafnium mailing list -- hafnium(a)lists.trustedfirmware.org
To unsubscribe send an email to hafnium-leave(a)lists.trustedfirmware.org
--
Hafnium mailing list -- hafnium(a)lists.trustedfirmware.org
To unsubscribe send an email to hafnium-leave(a)lists.trustedfirmware.org
Hi, Olivier,
Sorry for the delay in replying to this query.
As I've seen in the documentation, in the case of hafnium implemented as S-EL2,
NWd uses shared memory to communicate with SWd and there are two steps to register the page table.
Please correct me if there are any errors in my description.
Firstly, Linux sends FFA_MEM_SHARE to SPMC (Hafnium), which completes memory mapping for S-EL1's stage-2 transition.
Secondly, Linux sends FFA_MSG_SEND_DIRECT_REQ to OPTEE, and OPTEE then retrieves the IPA space from SPMC according to the cookie received
and then completes the memory mapping for S-EL1's stage-1 transition.
Our solution to the problem is: After the first step be completed, we add the invalidation of TLB,
which seems to greatly reduce the probability of problem occurrence when testing the optee_examples.
The root cause still needs to be further located.
It should be noted that our OPTEE version is based on 3.19-rc,
and Hafnium version is based on the commit:
https://git.trustedfirmware.org/hafnium/hafnium.git/commit/?id=dd883207ee9b… <https://git.trustedfirmware.org/hafnium/hafnium.git/commit/?id=dd883207ee9b… >
Anyway, thanks for the support.
Regards,
Yuye.
------------------------------------------------------------------
发件人:Olivier Deprez <Olivier.Deprez(a)arm.com>
发送时间:2023年2月11日(星期六) 00:56
收件人:hafnium <hafnium(a)lists.trustedfirmware.org>; 梅建强(禹夜) <meijianqiang.mjq(a)alibaba-inc.com>
主 题:Re: [Hafnium] hafnium page table configuration
Hi Yuye,
Quick feedback, we have a test case testing SP to SP mem sharing with a large physical address (for a NS memory region), so I expect this is covered:
https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/tftf/tests/runtime… <https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/tftf/tests/runtime… >
https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/cactus_… <https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/cactus_… >
https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/ar… <https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/ar… >
There may certainly be other reasons, but before investigating further, is this still an issue at your end?
Regards,
Olivier.
From: 梅建强(禹夜) via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 06 February 2023 07:41
To: hafnium <hafnium(a)lists.trustedfirmware.org>
Subject: [Hafnium] hafnium page table configuration
Hi,
At present, I suspect that this may be a hafnium problem,
secure_storage CA/TA may use over 34G address space before communication.
May I ask, does hafnium currently support the configuration of the page table of 0x8 80000000-xxxxxxxxx address range?
If yes, how to configure it?
The error log is as follows:
Current share states: SHARE 0x0 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 SHARE 0x1 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 SHARE 0x2 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 WARNING: Stage-2 page fault: pc=0x40032106, vmid=0x8001, vcpu=13, vaddr=0x40046ce0, ipaddr=0x8a9b28ce0, mode=0x81 0x63 NOTICE: Injecting Data Abort exception into VM 0x8001. D/TC:013 0 abort_handler:550 [abort] abort in User mode (TA will panic) E/TC:??? 0 E/TC:??? 0 User mode data-abort at address 0x40046ce0 (translation fault) E/TC:??? 0 esr 0x94020007 ttbr0 0x20000f03061a0 ttbr1 0x00000000 cidr 0x0 E/TC:??? 0 cpu #13 cpsr 0x00000130 E/TC:??? 0 x0 000000004003e348 x1 000000004003e349 E/TC:??? 0 x2 0000000040046ce0 x3 000000004003e348 E/TC:??? 0 x4 0000000040036088 x5 0000000000000000 E/TC:??? 0 x6 0000000000000000 x7 0000000040042e28 E/TC:??? 0 x8 0000000000000000 x9 0000000000000000 E/TC:??? 0 x10 0000000000000000 x11 0000000000000000 E/TC:??? 0 x12 0000000000000000 x13 0000000040042e28 E/TC:??? 0 x14 00000000400213cf x15 0000000000000000 E/TC:??? 0 x16 0000000000000000 x17 0000000000000000 E/TC:??? 0 x18 0000000000000000 x19 0000000000000000 E/TC:??? 0 x20 0000000000000000 x21 0000000000000000 E/TC:??? 0 x22 0000000000000000 x23 0000000000000000 E/TC:??? 0 x24 0000000000000000 x25 0000000000000000 E/TC:??? 0 x26 0000000000000000 x27 0000000000000000 E/TC:??? 0 x28 0000000000000000 x29 0000000000000000 E/TC:??? 0 x30 0000000000000000 elr 0000000040032106 E/TC:??? 0 sp_el0 0000000040042f80 E/LD: Status of TA f4e750bb-1437-4fbf-8785-8d3580c34994 E/LD: arch: arm E/LD: region 0: va 0x40006000 pa 0xf0404000 size 0x002000 flags rw-s (ldelf) E/LD: region 1: va 0x40008000 pa 0xf0406000 size 0x011000 flags r-xs (ldelf) E/LD: region 2: va 0x40019000 pa 0xf0417000 size 0x001000 flags rw-s (ldelf) E/LD: region 3: va 0x4001a000 pa 0xf0418000 size 0x004000 flags rw-s (ldelf) E/LD: region 4: va 0x4001e000 pa 0xf041c000 size 0x001000 flags r--s E/LD: region 5: va 0x4001f000 pa 0x00001000 size 0x017000 flags r-xs [0] E/LD: region 6: va 0x40036000 pa 0x00018000 size 0x00c000 flags rw-s [0] E/LD: region 7: va 0x40042000 pa 0xf0440000 size 0x001000 flags rw-s (stack) E/LD: region 8: va 0x40043000 pa 0x8b9101620 size 0x003000 flags rw-- (param) E/LD: region 9: va 0x40046000 pa 0x8a9b28ce0 size 0x001000 flags rw-- (param) E/LD: [0] f4e750bb-1437-4fbf-8785-8d3580c34994 @ 0x4001f000 E/LD: Call stack
Regards,
Yuye
------------------------------------------------------------------
发件人:梅建强(禹夜) <meijianqiang.mjq(a)alibaba-inc.com>
发送时间:2023年2月4日(星期六) 11:40
收件人:op-tee <op-tee(a)lists.trustedfirmware.org>
抄 送:赵哲(为哲) <weizhe.zz(a)alibaba-inc.com>; 高海源(码源) <haiyuan.ghy(a)alibaba-inc.com>; 王一蒙(北见) <wym389994(a)alibaba-inc.com>
主 题:OPTEE TA Crash
Hi,
Does anyone have a good solution to this problem?
https://github.com/OP-TEE/optee_os/issues/5803 <https://github.com/OP-TEE/optee_os/issues/5803 > <https://github.com/OP-TEE/optee_os/issues/5803 <https://github.com/OP-TEE/optee_os/issues/5803 > >
regards,
yuye
--
Hafnium mailing list -- hafnium(a)lists.trustedfirmware.org
To unsubscribe send an email to hafnium-leave(a)lists.trustedfirmware.org
Hi,
Thanks for your efforts for this query. I have tried your solution. Although the Hafnium code could be built successfully, it does not work when the boot flow run into the entry of the Hafnium.
I tried to debug the error with trace32 tool. And I confirmed that the Hafnium code has been loaded over 34G of free address space. However, when PC register point to the start address of Hafnium, the code data is displayed as udf #0x0.
It may be a page table problem, and we are still locating the problem reason.
Regards,
Mei, Jianqiang.
------------------------------------------------------------------
发件人:Olivier Deprez via Hafnium <hafnium(a)lists.trustedfirmware.org>
发送时间:2022年12月21日(星期三) 18:02
收件人:梅建强(禹夜) <meijianqiang.mjq(a)alibaba-inc.com>
抄 送:hafnium <hafnium(a)lists.trustedfirmware.org>; 常琳(特睿) <terui.cl(a)alibaba-inc.com>; 赵哲(为哲) <weizhe.zz(a)alibaba-inc.com>; 黄明(连一) <hm281385(a)alibaba-inc.com>
主 题:[Hafnium] Re: Hafnium load-address
Hi,
Sorry for the delay in replying to this query.
After few trials I ended with this possible solution:
https://review.trustedfirmware.org/c/hafnium/hafnium/+/18510 <https://review.trustedfirmware.org/c/hafnium/hafnium/+/18510 >
This should permit loading the Hafnium image to an address up to 48 bits (and by updating project/reference/BUILD.gn origin_address per your suggestion).
It is possible to further extend beyond 48 bits (e.g. 52 bits which is the maximum permitted VA using FEAT_LPA2).
Let me know if this works for you, then I can proceed further with polishing this change.
Regards,
Olivier.
________________________________
From: 梅建强(禹夜) via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 11 December 2022 09:58
To: Olivier Deprez <Olivier.Deprez(a)arm.com>
Cc: hafnium <hafnium(a)lists.trustedfirmware.org>; 常琳(特睿) <terui.cl(a)alibaba-inc.com>; 赵哲(为哲) <weizhe.zz(a)alibaba-inc.com>; 黄明(连一) <hm281385(a)alibaba-inc.com>
Subject: [Hafnium] Hafnium load-address
Hello, expert
Firstly, thanks for your answer some time ago. That's useful to our project.We combined the Hafnium (run at S-EL2) and the optee (run at S-EL1), then modified the configuration to adapt to our own environment. Now we can start the os normally.
Now we want to optimize a technical point, which is to load and run hafnium over 34G of free address space. This requires hafnium to support 64-bit load-address.
I tried to modify the configuration in BUILD.gn as follows:
origin_address = "0x880000000"
And I encountered the build error:
FAILED: secure_aem_v8a_fvp_clang/hafnium.elf ld.lld -pie --gc-sections --defsym=ORIGIN_ADDRESS=0x880000000 -O2 --icf=all --fatal-warnings --color-diagnostics -T /home/yuye/hafnium/Hafnium/build/image/image.ld -o secure_aem_v8a_fvp_clang/hafnium.elf --start-group @secure_aem_v8a_fvp_clang/hafnium.elf.rsp --end-group ld.lld: error: secure_aem_v8a_fvp_clang/obj/src/arch/aarch64/entry.entry.o:(.init.entry+0xC8): relocation R_AARCH64_ABS32 out of range: 36507222016 is not in [-2147483648, 4294967295]; references ORIGIN_ADDRESS >>> defined in -defsym:1 ninja: build stopped: subcommand failed. Makefile:70: recipe for target 'all' failed make: *** [all] Error 1 + cp out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ../../V2/Ali_M1_Public/Nonosi/Platform/Ali/M1/Bin/ cp: cannot stat 'out/reference/secure_aem_v8a_fvp_clang/hafnium.bin': No such file or directory
In this case, it seems that the new address causes some relative addressing to reach too far. Is there any solutions or supports for this issue currently in the community?
Regards,
Mei, Jianqiang.
--
Hafnium mailing list -- hafnium(a)lists.trustedfirmware.org
To unsubscribe send an email to hafnium-leave(a)lists.trustedfirmware.org
--
Hafnium mailing list -- hafnium(a)lists.trustedfirmware.org
To unsubscribe send an email to hafnium-leave(a)lists.trustedfirmware.org
Hi,
At present, I suspect that this may be a hafnium problem,
secure_storage CA/TA may use over 34G address space before communication.
May I ask, does hafnium currently support the configuration of the page table of 0x8 80000000-xxxxxxxxx address range?
If yes, how to configure it?
The error log is as follows:
Current share states: SHARE 0x0 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 SHARE 0x1 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 SHARE 0x2 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 WARNING: Stage-2 page fault: pc=0x40032106, vmid=0x8001, vcpu=13, vaddr=0x40046ce0, ipaddr=0x8a9b28ce0, mode=0x81 0x63 NOTICE: Injecting Data Abort exception into VM 0x8001. D/TC:013 0 abort_handler:550 [abort] abort in User mode (TA will panic) E/TC:??? 0 E/TC:??? 0 User mode data-abort at address 0x40046ce0 (translation fault) E/TC:??? 0 esr 0x94020007 ttbr0 0x20000f03061a0 ttbr1 0x00000000 cidr 0x0 E/TC:??? 0 cpu #13 cpsr 0x00000130 E/TC:??? 0 x0 000000004003e348 x1 000000004003e349 E/TC:??? 0 x2 0000000040046ce0 x3 000000004003e348 E/TC:??? 0 x4 0000000040036088 x5 0000000000000000 E/TC:??? 0 x6 0000000000000000 x7 0000000040042e28 E/TC:??? 0 x8 0000000000000000 x9 0000000000000000 E/TC:??? 0 x10 0000000000000000 x11 0000000000000000 E/TC:??? 0 x12 0000000000000000 x13 0000000040042e28 E/TC:??? 0 x14 00000000400213cf x15 0000000000000000 E/TC:??? 0 x16 0000000000000000 x17 0000000000000000 E/TC:??? 0 x18 0000000000000000 x19 0000000000000000 E/TC:??? 0 x20 0000000000000000 x21 0000000000000000 E/TC:??? 0 x22 0000000000000000 x23 0000000000000000 E/TC:??? 0 x24 0000000000000000 x25 0000000000000000 E/TC:??? 0 x26 0000000000000000 x27 0000000000000000 E/TC:??? 0 x28 0000000000000000 x29 0000000000000000 E/TC:??? 0 x30 0000000000000000 elr 0000000040032106 E/TC:??? 0 sp_el0 0000000040042f80 E/LD: Status of TA f4e750bb-1437-4fbf-8785-8d3580c34994 E/LD: arch: arm E/LD: region 0: va 0x40006000 pa 0xf0404000 size 0x002000 flags rw-s (ldelf) E/LD: region 1: va 0x40008000 pa 0xf0406000 size 0x011000 flags r-xs (ldelf) E/LD: region 2: va 0x40019000 pa 0xf0417000 size 0x001000 flags rw-s (ldelf) E/LD: region 3: va 0x4001a000 pa 0xf0418000 size 0x004000 flags rw-s (ldelf) E/LD: region 4: va 0x4001e000 pa 0xf041c000 size 0x001000 flags r--s E/LD: region 5: va 0x4001f000 pa 0x00001000 size 0x017000 flags r-xs [0] E/LD: region 6: va 0x40036000 pa 0x00018000 size 0x00c000 flags rw-s [0] E/LD: region 7: va 0x40042000 pa 0xf0440000 size 0x001000 flags rw-s (stack) E/LD: region 8: va 0x40043000 pa 0x8b9101620 size 0x003000 flags rw-- (param) E/LD: region 9: va 0x40046000 pa 0x8a9b28ce0 size 0x001000 flags rw-- (param) E/LD: [0] f4e750bb-1437-4fbf-8785-8d3580c34994 @ 0x4001f000 E/LD: Call stack
Regards,
Yuye
------------------------------------------------------------------
发件人:梅建强(禹夜) <meijianqiang.mjq(a)alibaba-inc.com>
发送时间:2023年2月4日(星期六) 11:40
收件人:op-tee <op-tee(a)lists.trustedfirmware.org>
抄 送:赵哲(为哲) <weizhe.zz(a)alibaba-inc.com>; 高海源(码源) <haiyuan.ghy(a)alibaba-inc.com>; 王一蒙(北见) <wym389994(a)alibaba-inc.com>
主 题:OPTEE TA Crash
Hi,
Does anyone have a good solution to this problem?
https://github.com/OP-TEE/optee_os/issues/5803 <https://github.com/OP-TEE/optee_os/issues/5803 >
regards,
yuye
Hi, expert
I want to confirm a problem about using hafnium to load spmc payload.
That is, Dose hafnium supports hyp_loaded for spmc payload or ffa_partion (such as optee_os) in the boot phase?
If the community confirms that it does not support it, I wonder, what might be the problem with doing so?
I see the following code in hanfnium:
if (manifest->vm[i].is_ffa_partition &&
!manifest->vm[i].is_hyp_loaded) {
TRY(parse_ffa_partition_package(stage1_locked, &vm_node,
&manifest->vm[i], vm_id,
ppool));
} else {
TRY(parse_vm(&vm_node, &manifest->vm[i], vm_id));
}
It seems that hafnium has no handling for both is_ffa_partition and is_hyp_loaded.
regards,
yuye
Hi,
Fyi we merged this gn tool prebuilt version update
https://review.trustedfirmware.org/q/topic:%22od%252Fgn-update-1901%22+(sta…
Essentially more recent sources were used to build the x86 host version (as the tool hasn't been updated since 2018), and this aligns to the version used on aarch64 host.
'make format' uses gn to format .gn build files, and as a consequence new formatting rules apply with the new tool version.
The side effect is that changes already submitted in the hafnium dashboards would have to be rebased, and run make format again to pass the CI.
Regards,
Olivier.