Hi Yuye,

See comments inline [OD].

Regards, Olivier.

________________________________ From: 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com Sent: 22 February 2023 04:17 To: Olivier Deprez Olivier.Deprez@arm.com; hafnium hafnium@lists.trustedfirmware.org Cc: 高海源(码源) haiyuan.ghy@alibaba-inc.com; 王一蒙(北见) wym389994@alibaba-inc.com Subject: 回复：[Hafnium] hafnium page table configuration

Hi, Olivier,

Sorry for the delay in replying to this query.

As I've seen in the documentation, in the case of hafnium implemented as S-EL2, NWd uses shared memory to communicate with SWd and there are two steps to register the page table. Please correct me if there are any errors in my description.

Firstly, Linux sends FFA_MEM_SHARE to SPMC (Hafnium), which completes memory mapping for S-EL1's stage-2 transition. Secondly, Linux sends FFA_MSG_SEND_DIRECT_REQ to OPTEE, and OPTEE then retrieves the IPA space from SPMC according to the cookie received and then completes the memory mapping for S-EL1's stage-1 transition.

[OD] That's quite correct, with the subtle difference that the region is mapped in SP's S2 page tables upon the receiver SP emitting the memory retrieve request. The SP maps the region in the S1 page tables after receiving the memory retrieve response from the SPMC.

Our solution to the problem is: After the first step be completed, we add the invalidation of TLB, which seems to greatly reduce the probability of problem occurrence when testing the optee_examples. The root cause still needs to be further located.

[OD] There is a probable miss with invalidating the S2 TLB entries for NS IPA space. Interestingly, this issue is not observed with models, but likely to happen on real silicon. Can you try the suggested temp. fix from https://github.com/OP-TEE/optee_os/issues/5803#issuecomment-1436084763 ? (while reverting your own TLB invalidation fix. ?) I'm working on a cleaner fix, and prioritize if this issue is confirmed at your end.

It should be noted that our OPTEE version is based on 3.19-rc, and Hafnium version is based on the commit: https://git.trustedfirmware.org/hafnium/hafnium.git/commit/?id=dd883207ee9b3...

Anyway, thanks for the support.

Regards, Yuye. ------------------------------------------------------------------ 发件人：Olivier Deprez Olivier.Deprez@arm.com 发送时间：2023年2月11日(星期六) 00:56 收件人：hafnium hafnium@lists.trustedfirmware.org; 梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 主　题：Re: [Hafnium] hafnium page table configuration

Hi Yuye,

Quick feedback, we have a test case testing SP to SP mem sharing with a large physical address (for a NS memory region), so I expect this is covered: https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/tftf/tests/runtime_... https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/cactus_t... https://git.trustedfirmware.org/TF-A/tf-a-tests.git/tree/spm/cactus/plat/arm...

There may certainly be other reasons, but before investigating further, is this still an issue at your end?

Regards, Olivier.

________________________________ From: 梅建强(禹夜) via Hafnium hafnium@lists.trustedfirmware.org Sent: 06 February 2023 07:41 To: hafnium hafnium@lists.trustedfirmware.org Subject: [Hafnium] hafnium page table configuration

Hi, At present, I suspect that this may be a hafnium problem, secure_storage CA/TA may use over 34G address space before communication. May I ask, does hafnium currently support the configuration of the page table of 0x8 80000000-xxxxxxxxx address range? If yes, how to configure it? The error log is as follows: Current share states: SHARE 0x0 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 SHARE 0x1 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 SHARE 0x2 (from VM 0x0, attributes 0x2f, flags 0x8, tag 0, to 1 recipients [VM 0x8001: 0x6 (offset 48)]): fully sent with 1 fragments, 1 retrieved, sender's original mode: 0x7 WARNING: Stage-2 page fault: pc=0x40032106, vmid=0x8001, vcpu=13, vaddr=0x40046ce0, ipaddr=0x8a9b28ce0, mode=0x81 0x63 NOTICE: Injecting Data Abort exception into VM 0x8001. D/TC:013 0 abort_handler:550 [abort] abort in User mode (TA will panic) E/TC:??? 0 E/TC:??? 0 User mode data-abort at address 0x40046ce0 (translation fault) E/TC:??? 0 esr 0x94020007 ttbr0 0x20000f03061a0 ttbr1 0x00000000 cidr 0x0 E/TC:??? 0 cpu #13 cpsr 0x00000130 E/TC:??? 0 x0 000000004003e348 x1 000000004003e349 E/TC:??? 0 x2 0000000040046ce0 x3 000000004003e348 E/TC:??? 0 x4 0000000040036088 x5 0000000000000000 E/TC:??? 0 x6 0000000000000000 x7 0000000040042e28 E/TC:??? 0 x8 0000000000000000 x9 0000000000000000 E/TC:??? 0 x10 0000000000000000 x11 0000000000000000 E/TC:??? 0 x12 0000000000000000 x13 0000000040042e28 E/TC:??? 0 x14 00000000400213cf x15 0000000000000000 E/TC:??? 0 x16 0000000000000000 x17 0000000000000000 E/TC:??? 0 x18 0000000000000000 x19 0000000000000000 E/TC:??? 0 x20 0000000000000000 x21 0000000000000000 E/TC:??? 0 x22 0000000000000000 x23 0000000000000000 E/TC:??? 0 x24 0000000000000000 x25 0000000000000000 E/TC:??? 0 x26 0000000000000000 x27 0000000000000000 E/TC:??? 0 x28 0000000000000000 x29 0000000000000000 E/TC:??? 0 x30 0000000000000000 elr 0000000040032106 E/TC:??? 0 sp_el0 0000000040042f80 E/LD: Status of TA f4e750bb-1437-4fbf-8785-8d3580c34994 E/LD: arch: arm E/LD: region 0: va 0x40006000 pa 0xf0404000 size 0x002000 flags rw-s (ldelf) E/LD: region 1: va 0x40008000 pa 0xf0406000 size 0x011000 flags r-xs (ldelf) E/LD: region 2: va 0x40019000 pa 0xf0417000 size 0x001000 flags rw-s (ldelf) E/LD: region 3: va 0x4001a000 pa 0xf0418000 size 0x004000 flags rw-s (ldelf) E/LD: region 4: va 0x4001e000 pa 0xf041c000 size 0x001000 flags r--s E/LD: region 5: va 0x4001f000 pa 0x00001000 size 0x017000 flags r-xs [0] E/LD: region 6: va 0x40036000 pa 0x00018000 size 0x00c000 flags rw-s [0] E/LD: region 7: va 0x40042000 pa 0xf0440000 size 0x001000 flags rw-s (stack) E/LD: region 8: va 0x40043000 pa 0x8b9101620 size 0x003000 flags rw-- (param) E/LD: region 9: va 0x40046000 pa 0x8a9b28ce0 size 0x001000 flags rw-- (param) E/LD: [0] f4e750bb-1437-4fbf-8785-8d3580c34994 @ 0x4001f000 E/LD: Call stack Regards, Yuye ------------------------------------------------------------------ 发件人：梅建强(禹夜) meijianqiang.mjq@alibaba-inc.com 发送时间：2023年2月4日(星期六) 11:40 收件人：op-tee op-tee@lists.trustedfirmware.org 抄　送：赵哲(为哲) weizhe.zz@alibaba-inc.com; 高海源(码源) haiyuan.ghy@alibaba-inc.com; 王一蒙(北见) wym389994@alibaba-inc.com 主　题：OPTEE TA Crash Hi, Does anyone have a good solution to this problem？ https://github.com/OP-TEE/optee_os/issues/5803 <https://github.com/OP-TEE/optee_os/issues/5803 > regards, yuye -- Hafnium mailing list -- hafnium@lists.trustedfirmware.org To unsubscribe send an email to hafnium-leave@lists.trustedfirmware.org