Hello,
We get the following error in the non-secure terminal in the QEMU, while booting the normal world with hafnium(SPMC) at S-EL2 and SP at S-EL1. NOTICE: Trapped access to system register write: op0=1, op1=0, crn=7, crm=14, op2=2, rt=11. NOTICE: Injecting Unknown Reason exception into VM 0x8001.
Above system register found as DC CISW.
It found that this happened with the recent hafnium change "feat(interrupts): preferred managed exit signal" Thanks.
Hi,
Thanks for reporting. Just for your information, until then, the SPMC has mostly (if not only) been tested on FVP at our end. Using qemu is new to us, but your finding is interesting.
Can you provide a bit more on the setup you use. Here are my guesses:
1. secure world runs hafnium at S-EL2, OP-TEE at S-EL1. Which OP-TEE version are you using? 2. normal world runs vanilla linux at NS-EL1 with FF-A kconfig option enabled. Is a hypervisor present at NS-EL2? 3. OP-TEE uses the optee build generated rootfs (incl. xtest, client lib. etc)?
About 1., can you provide build options used for building OP-TEE? In particular you need to make sure the CFG_CORE_SEL2_SPMC=y is specified. I'm also envisioning GIC support either needs changes or dedicated command line options.
Regards, Olivier.
________________________________ From: shiju.jose--- via Hafnium hafnium@lists.trustedfirmware.org Sent: 26 October 2022 12:52 To: hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org Subject: [Hafnium] Trapped access to system register and excpetion
Hello,
We get the following error in the non-secure terminal in the QEMU, while booting the normal world with hafnium(SPMC) at S-EL2 and SP at S-EL1. NOTICE: Trapped access to system register write: op0=1, op1=0, crn=7, crm=14, op2=2, rt=11. NOTICE: Injecting Unknown Reason exception into VM 0x8001.
Above system register found as DC CISW.
It found that this happened with the recent hafnium change "feat(interrupts): preferred managed exit signal" Thanks. -- Hafnium mailing list -- hafnium@lists.trustedfirmware.org To unsubscribe send an email to hafnium-leave@lists.trustedfirmware.org
Hi Olivier,
Please find the information.
1. secure world runs hafnium at S-EL2, OP-TEE at S-EL1 Answer: Yes. Which OP-TEE version are you using? Answer: 3.19.0 2. normal world runs vanilla linux at NS-EL1 with FF-A kconfig option enabled. Answer: linux-next with FF-A kconfig option enabled. Is a hypervisor present at NS-EL2? Answer: Hypervisor is not present at NS-EL2. 3. OP-TEE uses the optee build generated rootfs (incl. xtest, client lib. etc)? Answer: We use our proprietary rootsfs with the xtest added. Can you point where I can find the op-tee build generated rootfs? 4. About 1., can you provide build options used for building OP-TEE? Answer: The build command used for the OP-TEE OS. CROSS_COMPILE64=aarch64-none-elf- PLATFORM=vexpress PLATFORM_FLAVOR=qemu_armv8a CFG_ARM_GICV3=y CFG_CORE_SEL2_SPMC=y CFG_ARM64_core=y CFG_USER_TA_TARGETS=ta_arm64 CFG_TEE_CORE_DEBUG=y CFG_CORE_ASLR=n CFG_TEE_CORE_LOG_LEVEL=4 CFG_TEE_BENCHMARK=n CFG_WITH_STATS=y make
Thanks, Shiju
Hello, Please ignore my previous question about the op-tee build generated rootfs. I found it. https://optee.readthedocs.io/en/latest/building/gits/build.html
Hello,
I tested with the op-tee build generated rootfs, however this issue (Trapped access to system register and exception) still exists.
Thanks, Shiju
Hi,
Thanks for all the details, this is helpful.
Just to keep you posted, I get to boot TF-A/Hafnium/OP-TEE/linux down to console with ffa and optee drivers initializing properly. For now my attempts with xtest fail early for some reason (perhaps an issue with userspace), so I'm not (yet) facing the same issue with cache maintenance as you reported. I'll let know later when I'm there.
Meanwhile, I noted a number of fixes I had to do, and I wonder if you did the same e.g.: 1/ create an OP-TEE partition manifest with appropriate load address/entry point, declare the qemu secure uart address. I notice the UART console may trigger secure interrupts so that might require declaring as well in the manifest. 2/ In OP-TEE strip out any GIC related initialization as a SP doesn't have direct access to the physical GIC. 3/ enabled managed exit to handle the case where a NS interrupt triggers while OP-TEE/or TA runs.
For TF-A (BL1/BL2/BL31) do you actually use your own changes, or the earlier experimental changes of mine?
Regards, Olivier.
________________________________ From: shiju.jose--- via Hafnium hafnium@lists.trustedfirmware.org Sent: 27 October 2022 16:47 To: hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org Subject: [Hafnium] Re: Trapped access to system register and excpetion
Hello,
I tested with the op-tee build generated rootfs, however this issue (Trapped access to system register and exception) still exists.
Thanks, Shiju -- Hafnium mailing list -- hafnium@lists.trustedfirmware.org To unsubscribe send an email to hafnium-leave@lists.trustedfirmware.org
Hi Olivier,
Thanks for your efforts and update. Please find my reply inline,
Just to keep you posted, I get to boot TF-A/Hafnium/OP-TEE/linux down to console with ffa and optee drivers initializing properly. For now my attempts with xtest fail early for some reason (perhaps an issue with userspace), so I'm not (yet) facing the same issue with cache maintenance as you reported. I'll let know later when I'm there. Answer: Please note that the reported issue occurs in my setup while Linux is booting, not while running the xtest.
Meanwhile, I noted a number of fixes I had to do, and I wonder if you did the same e.g.: 1/ create an OP-TEE partition manifest with appropriate load address/entry point, declare the qemu secure uart address. I notice the UART console may trigger secure interrupts so that might require declaring as well in the manifest.
Answer: I got OP-TEE manifest file with load address and entry point etc. The uart address is specified as follows, but not sure it is qemu secure uart address. uart1 { base-address = <0x00000000 0x09040000>; pages-count = <1>; attributes = <0x3>; /* read-write */ };
2/ In OP-TEE strip out any GIC related initialization as a SP doesn't have direct access to the physical GIC. 3/ enabled managed exit to handle the case where a NS interrupt triggers while OP-TEE/or TA runs. Answer: I have not done this, can you please share more information, how to do this?
For TF-A (BL1/BL2/BL31) do you actually use your own changes, or the earlier experimental changes of mine? Answer: For TF-A, I use my own following changes, https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/17388 and https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/17389
Thanks, Shiju.
Hi,
Just a short follow up.
Answer: Please note that the reported issue occurs in my setup while Linux is booting, not while running the xtest.
I may have misunderstood the comment in this change: https://review.trustedfirmware.org/c/hafnium/project/reference/+/16322
"I built hafnium with your hafnium changes and booted good in my test setup for the QEMU S-EL2 booting to the normal world and able to ran few op-tee tests (xtest) successfully from the normal world. However please note the following which is unrelated to this change [...] in the non-secure terminal while booting linux in the normal world. Error Log: NOTICE: Trapped access to system register write: op0=1, op1=0, crn=7, crm=14, op2=2, rt=11. NOTICE: Injecting Unknown Reason exception into VM 0x8001. Above system register found as DC CISW."
Can you recall which exact setup(s) are passing/failing?
On how to reproduce, it requires few OP-TEE changes and some integration work. We didn't have plans to do this but I can try to explain what I have.
Regards, Olivier.
________________________________ From: shiju.jose--- via Hafnium hafnium@lists.trustedfirmware.org Sent: 28 October 2022 12:48 To: hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org Subject: [Hafnium] Re: Trapped access to system register and excpetion
Hi Olivier,
Thanks for your efforts and update. Please find my reply inline,
Just to keep you posted, I get to boot TF-A/Hafnium/OP-TEE/linux down to console with ffa and optee drivers initializing properly. For now my attempts with xtest fail early for some reason (perhaps an issue with userspace), so I'm not (yet) facing the same issue with cache maintenance as you reported. I'll let know later when I'm there. Answer: Please note that the reported issue occurs in my setup while Linux is booting, not while running the xtest.
Meanwhile, I noted a number of fixes I had to do, and I wonder if you did the same e.g.: 1/ create an OP-TEE partition manifest with appropriate load address/entry point, declare the qemu secure uart address. I notice the UART console may trigger secure interrupts so that might require declaring as well in the manifest.
Answer: I got OP-TEE manifest file with load address and entry point etc. The uart address is specified as follows, but not sure it is qemu secure uart address. uart1 { base-address = <0x00000000 0x09040000>; pages-count = <1>; attributes = <0x3>; /* read-write */ };
2/ In OP-TEE strip out any GIC related initialization as a SP doesn't have direct access to the physical GIC. 3/ enabled managed exit to handle the case where a NS interrupt triggers while OP-TEE/or TA runs. Answer: I have not done this, can you please share more information, how to do this?
For TF-A (BL1/BL2/BL31) do you actually use your own changes, or the earlier experimental changes of mine? Answer: For TF-A, I use my own following changes, https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/17388 and https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/17389
Thanks, Shiju. -- Hafnium mailing list -- hafnium@lists.trustedfirmware.org To unsubscribe send an email to hafnium-leave@lists.trustedfirmware.org
Hi Olivier,
Can you recall which exact setup(s) are passing/failing? Answer: Sorry for the confusion. The failure case: The following error occurred in the non-secure console while linux is booting in the normal world after switch from the secure world. Error Log: "NOTICE: Trapped access to system register write: op0=1, op1=0, crn=7, crm=14, op2=2, rt=11. NOTICE: Injecting Unknown Reason exception into VM 0x8001." This issue seen with the recent hafnium change "feat(interrupts): preferred managed exit signal".
We didn't have plans to do this but I can try to explain what I have. Answer: Presently I am trying to test and check is this issue present with your TF-A and hafnium changes. Secure side is booted(hafnium and op-tee), however facing some issue while switching to the normal world. It would be helpful if you could share the steps used in your setup to boot secure side and normal world.
Thanks, Shiju
hafnium@lists.trustedfirmware.org
-
Olivier Deprez -
shiju.jose＠huawei.com