Hi, Olivier, 1. We solved the problem by update our Hafnium codebase on rebase of commit ca03054ba6a351534b93e6d64c12e671578eb340. The origin Hafnium codebase is on rebase of commit dd883207ee9b31c19169adf97c918d561dcb9a5c. OPTEE codebase stays with version 3.20.0. Which commit can solve this problem is still under investigation. I guess the reason may relate to that Hafnium has updated the implementation of FF-A mechanism in new commit. If you have already known the reason, please tell me. 2. In addition, although some cases run successfully, there are still many error messages generated during their running process, such as the following cases: * regression_1034 Test loading a large TA E/TC:011 0 mobj_ffa_get_by_cookie:384 Populating mobj from rx buffer, cookie 0x1 E/TC:011 0 mobj_ffa_get_by_cookie:384 Populating mobj from rx buffer, cookie 0x3 E/TC:??? 0 mobj_ffa_get_by_cookie:423 Failed to get cookie 0x3 internal_offs 0 E/LD: init_elf:486 sys_open_ta_bin(25497083-a58a-4fc5-8a72-1ad7b69b8562) E/TC:??? 0 ldelf_init_with_ldelf:131 ldelf failed with res: 0xffff000c TEEC_ERROR_OUT_OF_MEMORY - ignored regression_1034 OK * regression_1022 Test dlopen()/dlsym()/dlclose() API E/TC:011 0 mobj_ffa_get_by_cookie:384 Populating mobj from rx buffer, cookie 0x1 E/TC:011 0 mobj_ffa_get_by_cookie:384 Populating mobj from rx buffer, cookie 0x3 E/TC:011 0 mobj_ffa_get_by_cookie:384 Populating mobj from rx buffer, cookie 0x3 E/TC:011 0 mobj_ffa_get_by_cookie:384 Populating mobj from rx buffer, cookie 0x1 E/TC:011 0 mobj_ffa_get_by_cookie:384 Populating mobj from rx buffer, cookie 0x3 E/TC:??? 0 E/TC:??? 0 TA panicked with code 0x0 E/LD: Status of TA 5b9e0e40-2636-11e1-ad9e-0002a5d5c51b E/LD: arch: aarch64 E/LD: region 0: va 0x40006000 pa 0xf0404000 size 0x002000 flags rw-s (ldelf) E/LD: region 1: va 0x40008000 pa 0xf0406000 size 0x011000 flags r-xs (ldelf) E/LD: region 2: va 0x40019000 pa 0xf0417000 size 0x001000 flags rw-s (ldelf) E/LD: region 3: va 0x4001a000 pa 0xf0418000 size 0x004000 flags rw-s (ldelf) E/LD: region 4: va 0x4001e000 pa 0xf041c000 size 0x001000 flags r--s E/LD: region 5: va 0x4001f000 pa 0xf052e000 size 0x003000 flags rw-s (stack) E/LD: region 6: va 0x40038000 pa 0x00000000 size 0x00f000 flags r-xs [2] E/LD: region 7: va 0x40047000 pa 0x0000e000 size 0x002000 flags rw-s [2] E/LD: region 8: va 0x40054000 pa 0x00000000 size 0x00f000 flags r-xs [1] E/LD: region 9: va 0x40063000 pa 0x0000e000 size 0x002000 flags rw-s [1] E/LD: region 10: va 0x4008b000 pa 0x00001000 size 0x02a000 flags r-xs [0] E/LD: region 11: va 0x400b5000 pa 0x0002b000 size 0x0e7000 flags rw-s [0] E/LD: [0] 5b9e0e40-2636-11e1-ad9e-0002a5d5c51b @ 0x4008b000 E/LD: [1] ffd2bded-ab7d-4988-95ee-e4962fff7154 @ 0x40054000 E/LD: [2] b3091a65-9751-4784-abf7-0298a7cc35ba @ 0x40038000 E/LD: Call stack: E/LD: 0x4009e5a8 E/LD: 0x400a09b8 E/LD: 0x4009ed0c regression_1022 OK The E/TC message generated by mobj_ffa_get_by_cookie appears in almost all cases. Is this related to FFA_MEM_FRAG_RX/TX, or something else? And what is related to the E/LD message? I am currently studying the FFA specification to try to explain these problems. And some related questions raised in the other email(subject: Hafnium Dynamic Shared Memory). You can tell me if it's convenient for you. 3. Could memory managed by Hafnium be configured in any other way except by spmc manifest? Does Hafnium support the configuration of the location of running memory such as text, data, stacks, and heap? If yes, how? Thanks you for your help. Regards, Yuye. ------------------------------------------------------------------ 发件人：Olivier Deprez via Hafnium hafnium@lists.trustedfirmware.org 发送时间：2023年2月27日(星期一) 17:19 收件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 抄 送：hafnium hafnium@lists.trustedfirmware.org 主 题：[Hafnium] Re: xtest 1034 Hi Yuye, Can you make sure to use hafnium tip of master (if that's not the case)? We confirmed this specific test passes on models from this commit onwards: https://git.trustedfirmware.org/hafnium/hafnium.git/commit/?id=fdd29277caf20... <https://git.trustedfirmware.org/hafnium/hafnium.git/commit/?id=fdd29277caf20... > We may provide further guidance if the issue persists after rebasing. Thanks, Olivier. ________________________________ From: 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com Sent: 26 February 2023 06:49 To: hafnium hafnium@lists.trustedfirmware.org; Olivier Deprez Olivier.Deprez@arm.com; Jens Wiklander jens.wiklander@linaro.org Subject: xtest 1034 Hi, experts, For dynamic shared memory usage, I am not very clear. So we need to consult you about the related question. I failed to run xtest 1034 when I used Hafnium as SPMC and OPTEE as SP. Here are the debug logs： #./xtest 1034& [2] 10130 Test ID: 1034 Run test suite with level=0 TEE test application started over default TEE instance ###################################################### # # regression # ###################################################### regression_1034 Test loading a large TA WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000000 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f D/TC:055 0 mobj_ffa_get_by_cookie:381 cookie 0 resurrecting D/TC:??? 0 tee_ta_init_pseudo_ta_session:296 Lookup pseudo TA 25497083-a58a-4fc5-8a72-1ad7b69b8562 D/TC:??? 0 ldelf_load_ldelf:96 ldelf load address 0x40008000 D/LD: ldelf:134 Loading TS 25497083-a58a-4fc5-8a72-1ad7b69b8562 F/TC:??? 0 trace_syscall:151 syscall Add constant time memcmp_ct function #3https://github.com/OP-TEE/optee_os/pull/3 https://github.com/OP-TEE/optee_os/pull/3 > (syscall_get_property) F/TC:??? 0 trace_syscall:151 syscall GitHub usage documentation #5https://github.com/OP-TEE/optee_os/pull/5 https://github.com/OP-TEE/optee_os/pull/5 > (syscall_open_ta_session) #D/TC:??? 0 ldelf_syscall_open_bin:143 Lookup user TA ELF 25497083-a58a-4fc5-8a72-1ad7b69b8562 (early TA) D/TC:??? 0 ldelf_syscall_open_bin:146 res=0xffff0008 D/TC:??? 0 ldelf_syscall_open_bin:143 Lookup user TA ELF 25497083-a58a-4fc5-8a72-1ad7b69b8562 (Secure Storage TA) VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(60): 0x84000073 0x50 0x50 0x0 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x84000073 VERBOSE: Marked sending complete. Current share states: SHARE 0x0 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 SHARE 0x1 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 0 retrieved, sender's original mode: 0x7 WARNING: Atf_Debug(60): 0x84000061 0x0 0x1 0x0 0x0 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f E/TC:055 0 mobj_ffa_get_by_cookie:384 Populating mobj from rx buffer, cookie 0x1 VERBOSE: Hafnium_Debug ffa_handler func:0x84000074 Current share states: SHARE 0x0 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 SHARE 0x1 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 0 retrieved, sender's original mode: 0x7 Current share states: SHARE 0x0 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 SHARE 0x1 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 VERBOSE: Hafnium_Debug ffa_handler func:0x84000065 VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f D/TC:??? 0 ldelf_syscall_open_bin:146 res=0xffff0008 D/TC:??? 0 ldelf_syscall_open_bin:143 Lookup user TA ELF 25497083-a58a-4fc5-8a72-1ad7b69b8562 (REE) VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(55): 0x8400006f 0x8001 0x0 0x80000001 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400006f VERBOSE: Hafnium_Debug ffa_handler func:0x84000070 WARNING: Atf_Debug(55): 0x84000070 0x80010000 0x0 0x0 0x1 0x0 0x0 0x0 WARNING: Atf_Debug(58): 0x84000073 0x1190 0x1000 0x0 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x84000073 Current share states: SHARE 0x0 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 SHARE 0x1 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 SHARE 0x2 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): partially sent with 1 fragments, 0 retrieved, sender's original mode: 0x7 WARNING: Atf_Debug(58): 0x8400007a 0x2 0x0 0x1000 0x0 0x0 0x0 0x0 WARNING: Atf_Debug(58): 0x8400007b 0x2 0x0 0x190 0x0 0x0 0x0 0x0 VERBOSE: Hafnium_Debug ffa_handler func:0x8400007b VERBOSE: Hafnium_Debug fragment_length:0x190 VERBOSE: Hafnium_Debug fragment_copy:00000000ff30e000, from_msg:00000008bdd48000 ERROR: Data abort: pc=0xff21a688, esr=0x96000006, ec=0x25, far=0x9c Panic: EL2 exception the error occured when Hafnium run the code: api_ffa_mem_frag_tx memcpy_s(fragment_copy, MM_PPOOL_ENTRY_SIZE, from_msg, fragment_length); It seems that I did not add the page table at 0x00000000ff30e000 or 0x00000008bdd48000, but I am not quite clear about what fragment_copy and from_msg mean. Can someone help me see what the problem is? Regards, Yuye.