Hi,

Can you provide the base TF-A commit hash you use when building?

Out of curiosity do you enable VHE in the SWd/SPM?

Regards, Olivier. ________________________________ From: Hafnium hafnium-bounces@lists.trustedfirmware.org on behalf of Chenxu Wang via Hafnium hafnium@lists.trustedfirmware.org Sent: 13 December 2021 17:34 To: hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org Subject: [Hafnium] A problem about assert failed in TF-A

Hi all, I am running Hafnium on FVP, with Cactus SP in SEL1 and KVM VHE enabled. Sometimes I send the "FFA_MSG_SEND_DIRECT_REQ" smc call in KVM (I fill 0x8400006f in x0, then VMID and SP ID in x1, let x2 as 0). It says assert failed, like this:

ASSERT: lib/el3_runtime/aarch64/context_mgmt.c:651 BACKTRACE: START: assert 0: EL3: 0x4005cac 1: EL3: 0x400323c 2: EL3: 0x400620c 3: EL3: 0x400e180 4: EL3: 0x4005a94 BACKTRACE: END: assert

I notice that when services/std_svc/spmd/spmd_main.c sends the FFA call (from NS to S) via "spmd_smc_forward(smc_fid, secure_origin,x1, x2, x3, x4, handle)", it will go to cm_el1_sysregs_context_restore(secure_state_out) and cm_el2_sysregs_context_restore(secure_state_out), then it will assert the cm_get_context(). it gets the NULL context, so assert failed.

Such assert is NOT ALWAYS failed, but I still want to solve this problem.

Since I have modified many lines of code in Hafnium and Cactus SP, I cannot show them here. Can you give me some advice on debugging? e.g., where should I check? -- Hafnium mailing list Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium

Hi Wang,

The mentioned assert hints a serious EL3 internal error, although I don't believe it comes from the reference implementation. It will be difficult to help without sharing changes made at least to the SPMD.

On a loosely related note, you're using a quite old code base, it'd be recommended rebasing to latest trees.

Regards, Olivier.

________________________________________ From: Chenxu Wang irakatz51@gmail.com Sent: 13 December 2021 19:18 To: Olivier Deprez Cc: Chenxu Wang via Hafnium Subject: Re: [Hafnium] A problem about assert failed in TF-A

Hi Olivier,

1. When I type git log in TF-A, here is the first info

commit 0aa70f4c4c023ca58dea2d093d3c08c69b652113 Merge: 8909fa9bb ffb07b043 Author: Madhukar Pappireddy madhukar.pappireddy@arm.com Date: Thu Feb 25 22:03:24 2021 +0000

Merge "plat/qemu: trigger reboot with secure pl061" into integration

2. When I type git log in Hafnium, here is the first info

commit aa336106d2feb7b9af3d7e3eec1294af27c35da0 Author: J-Alves joao.alves@arm.com Date: Mon Mar 1 13:02:45 2021 +0000

FF-A: FFA_MSG_SEND_DIRECT_REQ with invalid IDs

A call to FFA_MSG_SEND_DIRECT_REQ with invalid VM IDs should return error with code FFA_INVALID_PARAMETERS.

Change-Id: Iedfa173b37857f6f0b7297b6be012c51c51b169a Signed-off-by: J-Alves joao.alves@arm.com

3. I don't know how to check whether the Hafnium enable VHE in secure world. But I can see the current EL of Cactus SP (from TF-A-TESTS) is EL1. Should I provide other information?

Sincerely, Wang

Olivier Deprez Olivier.Deprez@arm.com 于2021年12月14日周二 01:29写道：

Hi,

Can you provide the base TF-A commit hash you use when building?

Out of curiosity do you enable VHE in the SWd/SPM?

Regards, Olivier. ________________________________ From: Hafnium hafnium-bounces@lists.trustedfirmware.org on behalf of Chenxu Wang via Hafnium hafnium@lists.trustedfirmware.org Sent: 13 December 2021 17:34 To: hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org Subject: [Hafnium] A problem about assert failed in TF-A

Hi all, I am running Hafnium on FVP, with Cactus SP in SEL1 and KVM VHE enabled. Sometimes I send the "FFA_MSG_SEND_DIRECT_REQ" smc call in KVM (I fill 0x8400006f in x0, then VMID and SP ID in x1, let x2 as 0). It says assert failed, like this:

ASSERT: lib/el3_runtime/aarch64/context_mgmt.c:651 BACKTRACE: START: assert 0: EL3: 0x4005cac 1: EL3: 0x400323c 2: EL3: 0x400620c 3: EL3: 0x400e180 4: EL3: 0x4005a94 BACKTRACE: END: assert

I notice that when services/std_svc/spmd/spmd_main.c sends the FFA call (from NS to S) via "spmd_smc_forward(smc_fid, secure_origin,x1, x2, x3, x4, handle)", it will go to cm_el1_sysregs_context_restore(secure_state_out) and cm_el2_sysregs_context_restore(secure_state_out), then it will assert the cm_get_context(). it gets the NULL context, so assert failed.

Such assert is NOT ALWAYS failed, but I still want to solve this problem.

Since I have modified many lines of code in Hafnium and Cactus SP, I cannot show them here. Can you give me some advice on debugging? e.g., where should I check? -- Hafnium mailing list Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium

Hi Wang,

With this level of details; this is difficult to say. You can extend to the TF-A ML if you wish. I'm hinting the SPMD because you are mentioning spmd_smc_forward and cm_el1/2_sysregs_context_restore which are within the SPMD/EL3 space. I wouldn't expect such assert to happen in any regular use case of the reference implementation (because this is a hard EL3 failure). But yes, the problem can be elsewhere in Hafnium or Cactus, but I'd say less likely to alter the EL3 state. Unless Hafnium has a bug leading to corrupting a secure memory region which doesn't belong to it. Beyond this, notice the assert is taken in cm_el1_sysregs_context_restore. It is called by cm_prepare_el3_exit which means it can be related to power management e.g. on a psci resume event. This can be a hint as you say this is occurring 'randomly'.

Regards, Olivier.

________________________________________ From: Chenxu Wang irakatz51@gmail.com Sent: 14 December 2021 14:38 To: Olivier Deprez Cc: Chenxu Wang via Hafnium Subject: Re: [Hafnium] A problem about assert failed in TF-A

Hi Olivier,

Do you mean that my "changes to SPMD" is the only reason for the EL3 internal error? Instead of the changes to Hafnium and Cactus SP?

Actually, I want to know, what is the possible reasons for suddenly losing the secure context. Should I ask TF-A mail list?

Sincerely, Wang

Olivier Deprez Olivier.Deprez@arm.com 于2021年12月14日周二 16:59写道：

Hi Wang,

The mentioned assert hints a serious EL3 internal error, although I don't believe it comes from the reference implementation. It will be difficult to help without sharing changes made at least to the SPMD.

On a loosely related note, you're using a quite old code base, it'd be recommended rebasing to latest trees.

Regards, Olivier.

---

From: Chenxu Wang irakatz51@gmail.com Sent: 13 December 2021 19:18 To: Olivier Deprez Cc: Chenxu Wang via Hafnium Subject: Re: [Hafnium] A problem about assert failed in TF-A

Hi Olivier,

1. When I type git log in TF-A, here is the first info

commit 0aa70f4c4c023ca58dea2d093d3c08c69b652113 Merge: 8909fa9bb ffb07b043 Author: Madhukar Pappireddy madhukar.pappireddy@arm.com Date: Thu Feb 25 22:03:24 2021 +0000

Merge "plat/qemu: trigger reboot with secure pl061" into integration

2. When I type git log in Hafnium, here is the first info

commit aa336106d2feb7b9af3d7e3eec1294af27c35da0 Author: J-Alves joao.alves@arm.com Date: Mon Mar 1 13:02:45 2021 +0000

FF-A: FFA_MSG_SEND_DIRECT_REQ with invalid IDs

A call to FFA_MSG_SEND_DIRECT_REQ with invalid VM IDs should return
error with code FFA_INVALID_PARAMETERS.

Change-Id: Iedfa173b37857f6f0b7297b6be012c51c51b169a
Signed-off-by: J-Alves <joao.alves@arm.com>

3. I don't know how to check whether the Hafnium enable VHE in secure

world. But I can see the current EL of Cactus SP (from TF-A-TESTS) is EL1. Should I provide other information?

Sincerely, Wang

Olivier Deprez Olivier.Deprez@arm.com 于2021年12月14日周二 01:29写道：

...

Hi,

Can you provide the base TF-A commit hash you use when building?

Out of curiosity do you enable VHE in the SWd/SPM?

Regards, Olivier. ________________________________ From: Hafnium hafnium-bounces@lists.trustedfirmware.org on behalf of Chenxu Wang via Hafnium hafnium@lists.trustedfirmware.org Sent: 13 December 2021 17:34 To: hafnium@lists.trustedfirmware.org hafnium@lists.trustedfirmware.org Subject: [Hafnium] A problem about assert failed in TF-A

Hi all, I am running Hafnium on FVP, with Cactus SP in SEL1 and KVM VHE enabled. Sometimes I send the "FFA_MSG_SEND_DIRECT_REQ" smc call in KVM (I fill 0x8400006f in x0, then VMID and SP ID in x1, let x2 as 0). It says assert failed, like this:

ASSERT: lib/el3_runtime/aarch64/context_mgmt.c:651 BACKTRACE: START: assert 0: EL3: 0x4005cac 1: EL3: 0x400323c 2: EL3: 0x400620c 3: EL3: 0x400e180 4: EL3: 0x4005a94 BACKTRACE: END: assert

I notice that when services/std_svc/spmd/spmd_main.c sends the FFA call (from NS to S) via "spmd_smc_forward(smc_fid, secure_origin,x1, x2, x3, x4, handle)", it will go to cm_el1_sysregs_context_restore(secure_state_out) and cm_el2_sysregs_context_restore(secure_state_out), then it will assert the cm_get_context(). it gets the NULL context, so assert failed.

Such assert is NOT ALWAYS failed, but I still want to solve this problem.

Since I have modified many lines of code in Hafnium and Cactus SP, I cannot show them here. Can you give me some advice on debugging? e.g., where should I check? -- Hafnium mailing list Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium