Hi,
A small heads up that from this change: https://review.trustedfirmware.org/c/hafnium/hafnium/+/11613
a developer needs to provide the LLVM/clang toolchain through the PATH environment variable.
See the documentation update:
https://review.trustedfirmware.org/plugins/gitiles/hafnium/hafnium/+/HEAD/d…
Until then the toolchain was hardcoded to use the version stored in the prebuilt submodule.
>From now, this weakens the dependency to the prebuilt toolchain and provides flexibility with providing an alternate toolchain on a x86 host. This also opens to building on an aarch64 host.
This has been tested with different combinations of hosts, ubuntu releases and toolchains downloaded from https://releases.llvm.org/download.html
x86_64 Ubuntu 18.04 / 20.04 clang/llvm 12.0.0 , 13.0.0
aarch64 Ubuntu 19.04 (Ampere eMAG) / 20.04, 21.04 (Rpi4) clang/llvm 12.0.0 , 12.0.1, 13.0.0
It is still possible to point PATH to the prebuilt toolchain version (Android llvm/clang 12.0.5) as indicated in the documentation.
If you have a live tree, please clean the out directory or run make clobber, once you update master. Builds run as before after the switch.
Limitations:
-the build breaks if using a native toolchain installed on the host (apt-get install clang..)
-the build breaks with Ubuntu 21.10/AArch64 (under investigation).
Regards,
Olivier.
TF-A List. This issue has also been discussed on Hafnium list before being posted here. Cross posting so we can have a single thread to track going forward.
See https://lists.trustedfirmware.org/pipermail/hafnium/2021-December/000209.ht…
with Olivier's last reply copied below. But see the archive above for full history of the thread.
> Hi Wang,
>
> With this level of details; this is difficult to say. You can extend to the TF-A ML if you wish. I'm hinting the SPMD because you are mentioning spmd_smc_forward and cm_el1/2_sysregs_context_restore which are within the SPMD/EL3 space. I wouldn't expect such assert to happen in any regular use case of the reference implementation (because this is a hard EL3 failure). But yes, the problem can be elsewhere in Hafnium or Cactus, but I'd say less likely to alter the EL3 state. Unless Hafnium has a bug leading to corrupting a secure memory region which doesn't belong to it.
> Beyond this, notice the assert is taken in cm_el1_sysregs_context_restore. It is called by cm_prepare_el3_exit which means it can be related to power management e.g. on a psci resume event. This can be a hint as you say this is occurring 'randomly'.
>
> Regards,
> Olivier.
Joanna
On 14/12/2021, 19:39, "TF-A on behalf of Chenxu Wang via TF-A" <tf-a-bounces(a)lists.trustedfirmware.org on behalf of tf-a(a)lists.trustedfirmware.org> wrote:
Hi all,
I am running FVP with 2CPUs, Cactus SP (SEL1), Hafnium (SEL2) and KVM VHE.
Sometimes I send the "FFA_MSG_SEND_DIRECT_REQ" smc call from KVM (I fill
0x8400006f in x0, then VMID and SP ID in x1, let x2 as 0). It says
assert failed, like this:
ASSERT: lib/el3_runtime/aarch64/context_mgmt.c:651
BACKTRACE: START: assert
0: EL3: 0x4005cac
1: EL3: 0x400323c
2: EL3: 0x400620c
3: EL3: 0x400e180
4: EL3: 0x4005a94
BACKTRACE: END: assert
After I check the bl31.dump, I notice that:
when services/std_svc/spmd/spmd_main.c sends the FFA
call (from NS to S) via "spmd_smc_forward(smc_fid, secure_origin,x1,
x2, x3, x4, handle)", it will go to
cm_el1_sysregs_context_restore(secure_state_out) and
cm_el2_sysregs_context_restore(secure_state_out), then it will assert
the cm_get_context(). it gets the NULL context, so assert failed.
Before the problem appeared, I have modified many codes on a dirty
TF-A v2.4 (commit hash is 0aa70f4c4c023ca58dea2d093d3c08c69b652113),
Hafnium and TF-A-TESTS. I also mail with Hafnium MailList, they
consider it can be a problem in EL3.
Such assert is NOT ALWAYS failed. I mean, maybe when I run FVP and
send "smc" now, it is failed. But when I shut down, run FVP, and send
the same instruction with the same parameter again, it is OK.
I want to know, what is the possible reasons for suddenly losing the
secure context. Can you give me some advice on debugging? e.g., where
should I check? Need I provide more info?
Sincerely,
Wang
--
TF-A mailing list
TF-A(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/tf-a
Hi all,
I am running Hafnium on FVP, with Cactus SP in SEL1 and KVM VHE enabled.
Sometimes I send the "FFA_MSG_SEND_DIRECT_REQ" smc call in KVM (I fill
0x8400006f in x0, then VMID and SP ID in x1, let x2 as 0). It says
assert failed, like this:
ASSERT: lib/el3_runtime/aarch64/context_mgmt.c:651
BACKTRACE: START: assert
0: EL3: 0x4005cac
1: EL3: 0x400323c
2: EL3: 0x400620c
3: EL3: 0x400e180
4: EL3: 0x4005a94
BACKTRACE: END: assert
I notice that when services/std_svc/spmd/spmd_main.c sends the FFA
call (from NS to S) via "spmd_smc_forward(smc_fid, secure_origin,x1,
x2, x3, x4, handle)", it will go to
cm_el1_sysregs_context_restore(secure_state_out) and
cm_el2_sysregs_context_restore(secure_state_out), then it will assert
the cm_get_context(). it gets the NULL context, so assert failed.
Such assert is NOT ALWAYS failed, but I still want to solve this problem.
Since I have modified many lines of code in Hafnium and Cactus SP, I
cannot show them here. Can you give me some advice on debugging? e.g.,
where should I check?
Hi Xiangyi Xu
For some reason your email was discarded by mailman, did you miss registering to the list?
See few comments inline [OD]
Regards,
Olivier.
________________________________________
From: xiangyi xu <xuxiangyi666(a)gmail.com>
Sent: 27 November 2021 14:13
To: hafnium(a)lists.trustedfirmware.org
Subject: Virtualising OP-TEE with Hafnium at S-EL2
HI ALL:
I am trying to setup Hafnium environment which loads OP-TEE in Security World and Linux in Normal World. I follow this instruction: https://lists.trustedfirmware.org/pipermail/hafnium/2021-January/000130.html. I can load the OP-TEE test successfully. But the BL33 payload is TF-A-tests example (BL33=../tf-a-tests/build/fvp/debug/tftf.bin). Could you share with us the tutorial to boot linux including the FF-A driver in the NS world while loading OP-TEE with Hafnium S-EL2? The presentation PPT is here: https://static.linaro.org/connect/lvc21/presentations/lvc21-305.pdf.
[OD] The mentioned presentation is in context of the Total Compute platform (as opposed to FVP).
It works great for what you want to do, and you can reproduce by following the user manual:
https://gitlab.arm.com/arm-reference-solutions/arm-reference-solutions-docs…
There is also work in progress at Linaro to provide a similar environment using the FVP.
@Jens, are there public instructions on how to build the end to end scenario targeting the FVP incl. Linux, FF-A driver, TF-A, Hafnium, OP-TEE ?
If not, I reckon this is in your mid term roadmap?
Secondly, I can boot Linux with QEMU and Hafnium. But it seems Hafnium only supports NS world partition while using QEMU. Is any instruction to boot OP_TEE in Security world with Hafnium S-EL2. Since, QEMU 6.0 already supports S-EL2. However, I could not compile ATF successfully with SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=qemu ...
[OD] qemu is not 'officially' supported for booting Hafnium at S-EL2. Although I have few changes in progress which may help. This is not a priority from Arm perspective, but let me come back on this.
By following https://teaclave.apache.org/trustzone-sdk-docs/getting-started-with-optee-f… , I can boot Linux in NS and OP-TEE in Trustzone with QEMU, but it doesn't work when I add Hafnium. Thanks!
Xiangyi Xu
Hi all,
I try to use Hafnium (deployed in SEL2) to access a DRAM address, e.g.
0xa000_0000, but failed.
I think the TTBR0_EL2 does not map this address space, or something wrong
with TZASC.
To solve this problem, I first add the DRAM region in TZASC and TTBR0_EL3
(by modifying TF-A), and EL3 can access this address. But Hafnium cannot
read or write it.
So, must I modify the .S files in Hafnium to add a page mapping to the DRAM
space? Can I add this page in some C files? Or should I do something else?
Sincerely,
Wang Chenxu
Hi Raghu,
Thanks for reporting.
This part of the test infrastructure (testing the SPMC) is still very fresh and requires improvement iterations so please bear with us. Also a reason it's not yet part of the automated non-regression with jenkins (as opposed to the legacy kokoro/test.sh). For the time being we still mostly rely on the TF-A CI for testing on the secure side.
IIUC this change was made to help with the test time as the FVP takes long to reload on every test.
But indeed it might have the side effect you describe.
So either we revert the FVP reloading on every test.
Or another (somewhat hackish) possibility is to clear the mentioned variables from within the test (or make them part of BSS)?
To be fair, the both worlds test scenario is not 100% stable on my machine (for some reason the connection is not always successful between the FVP and hftest) hence limiting confidence/robustness of my testing and investigations. So I wonder is the scripting is still somewhat a bit fragile.
Regards,
Olivier.
________________________________________
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of Raghu Krishnamurthy via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 03 August 2021 23:47
To: 'Raghu Krishnamurthy via Hafnium'
Subject: [Hafnium] Bug in hftest.py
Hi All,
Wanted to report to you that commit 18a25f9241f86ba2d637011ff465ce3869e8651b
in hafnium "appears" broken. The issue with the optimization in this patch
is that the partition images are not reloaded for each test run, which means
a previous test could have written data to say SRAM, and the following test
would use the old values from the previous test, when the same image is
executed again from SRAM for a following test. This would be a problem for
pretty much anything in the data section of a partition. In my case, I have
a counter in the data section of my partition, which does not get reset back
to its original value.
I've attached a patch to help repro the issue. Fix is to disable the
optimization or somehow reload the images for each run. This affects only
"both world" tests.
Let me know if I'm missing something here.
Apply patch and run timeout --foreground 300s ./test/hftest/hftest.py
--out_partitions out/reference/secure_aem_v8a_fvp_vm_clang --log
out/reference/kokoro_log --spmc
out/reference/secure_aem_v8a_fvp_clang/hafnium.bin --driver=fvp --hypervisor
out/reference/aem_v8a_fvp_clang/hafnium.bin --partitions_json
test/vmapi/ffa_secure_partitions/ffa_both_world_partitions_test.json
The command line is from kokoro/test_spmc.sh.
Thanks
Raghu
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/hafnium
Hi All,
Wanted to report to you that commit 18a25f9241f86ba2d637011ff465ce3869e8651b
in hafnium "appears" broken. The issue with the optimization in this patch
is that the partition images are not reloaded for each test run, which means
a previous test could have written data to say SRAM, and the following test
would use the old values from the previous test, when the same image is
executed again from SRAM for a following test. This would be a problem for
pretty much anything in the data section of a partition. In my case, I have
a counter in the data section of my partition, which does not get reset back
to its original value.
I've attached a patch to help repro the issue. Fix is to disable the
optimization or somehow reload the images for each run. This affects only
"both world" tests.
Let me know if I'm missing something here.
Apply patch and run timeout --foreground 300s ./test/hftest/hftest.py
--out_partitions out/reference/secure_aem_v8a_fvp_vm_clang --log
out/reference/kokoro_log --spmc
out/reference/secure_aem_v8a_fvp_clang/hafnium.bin --driver=fvp --hypervisor
out/reference/aem_v8a_fvp_clang/hafnium.bin --partitions_json
test/vmapi/ffa_secure_partitions/ffa_both_world_partitions_test.json
The command line is from kokoro/test_spmc.sh.
Thanks
Raghu
Hi Andrew,
I don't think Hafnium implements the different cacheability and shareability types for memory sharing at all does it?
[JA] No, it doesn't. At least that is my understanding as well. I noticed mostly due to lack of support in the mm library. Asking was a means to confirm.
The point (at least for now) was just about validation of the respective fields in the memory transaction descriptor.
Thank you for this! 🙂
Best regards,
João
________________________________
From: Andrew Walbran
Sent: Wednesday, July 14, 2021 5:29 PM
To: Joao Alves
Cc: hafnium(a)lists.trustedfirmware.org; Olivier Deprez; Achin Gupta; Mahesh Reddy Bireddy; Jaykumar Pitambarbhai Patel
Subject: Re: Hafnium - Memory attributes precedence checks for mem share
I don't think Hafnium implements the different cacheability and shareability types for memory sharing at all does it? We just didn't have a need for it, if you want to add support that should be fine.
This is mentioned in https://developer.trustedfirmware.org/T827, you can assign that to yourself if you want to take it on.
On Wed, 14 Jul 2021 at 13:50, Joao Alves <Joao.Alves(a)arm.com<mailto:Joao.Alves@arm.com>> wrote:
Hi Andrew,
We have been revising some aspects of the memory sharing implementation. The specification describes a set of precedence rules for the memory attributes specified in the memory transaction descriptor, including: Memory type, cacheability, shareability.
The sender would fill the memory attributes for the region to be shared. After memory send, the receiver should retrieve the regions, filling the memory attributes on its transaction descriptor that comply with the referred precedence rules.
The referred rules can be found in section 10.10.4 of the newly release FF-A v1.1 beta spec<https://developer.arm.com/documentation/den0077/c/?lang=en>, as follows.
Memory type precedence rules ( < reads as is less permissive than):
* Device-nGnRnE < Device-nGnRE < Device-nGRE < Device-GRE < Normal
Cacheability precedence rules:
* Non-cacheable < Write-Back Cacheable
Shareability precedence rules:
* Non-Shareable < Inner Shareable < Outer shareable
These checks are not part of the handling of FFA_MEMORY_RETRIEVE_REQ.
Was there an implementation defined reason for this? If so, could you please provide the rationale?
Thank you in advance for your help.
Best regards,
João Alves
Hi Andrew,
We have been revising some aspects of the memory sharing implementation. The specification describes a set of precedence rules for the memory attributes specified in the memory transaction descriptor, including: Memory type, cacheability, shareability.
The sender would fill the memory attributes for the region to be shared. After memory send, the receiver should retrieve the regions, filling the memory attributes on its transaction descriptor that comply with the referred precedence rules.
The referred rules can be found in section 10.10.4 of the newly release FF-A v1.1 beta spec<https://developer.arm.com/documentation/den0077/c/?lang=en>, as follows.
Memory type precedence rules ( < reads as is less permissive than):
* Device-nGnRnE < Device-nGnRE < Device-nGRE < Device-GRE < Normal
Cacheability precedence rules:
* Non-cacheable < Write-Back Cacheable
Shareability precedence rules:
* Non-Shareable < Inner Shareable < Outer shareable
These checks are not part of the handling of FFA_MEMORY_RETRIEVE_REQ.
Was there an implementation defined reason for this? If so, could you please provide the rationale?
Thank you in advance for your help.
Best regards,
João Alves