Hi,
I'm testing with Hafnium as SPMC at S-EL2 and OP-TEE as an SP at S-EL1 on
QEMU v7.0.0. I've run into a few problems and fixed most of them.
I believe the setup is similar to what Shiju is using in this mail thread
https://lists.trustedfirmware.org/archives/list/hafnium@lists.trustedfirmwa…
My setup can be duplicated with:
repo init -u https://github.com/jenswi-linaro/manifest.git -m qemu_v8.xml \
-b qemu_sel2
repo sync -j8
(cd hafnium && git submodule init && git submodule update)
cd build
make -j8 toolchains
make -j8 all
make run-only
With this xtest -x 1034 passes, xtest 1034 often causes
ERROR: Data abort: pc=0xe1198b8, esr=0x96000006, ec=0x25, far=0x9c
Panic: EL2 exception
Xtest runs dreadfully slow, I haven't investigated why yet, but at
least it works.
This is based on patches provided by Olivier at:
[1] https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/16412/2
[2] https://review.trustedfirmware.org/c/hafnium/hafnium/+/16323/7
I've also encountered the problem cache maintenance problem Shiju
described in the mail thread above:
NOTICE: Trapped access to system register write: op0=1, op1=0, crn=7,
crm=14, op2=2, rt=11.
It can be worked around by compiling OP-TEE with
CFG_CORE_WORKAROUND_NSITR_CACHE_PRIME=n. I'm pretty sure we do dcache
clean+inv elsewhere so I'm surprised it fails here. Is Hafnium expected
to block dcache clean+inv?
For Hafnium I've added two patches on top of [2], available at
https://github.com/jenswi-linaro/hafnium/tree/qemu_sel2:
- 79b4d2cbe06e SPMC: add missing ME initialization for secondary cores
- 659c79d5eacf feat(mm): fix FEAT_LPA workaround
For TF-A I've added a few patches on top of [1], available at
https://github.com/jenswi-linaro/arm-trusted-firmware/tree/qemu_sel2:
- a040396cae9e feat(qemu): add tos-fw-config for sel2 spmc
- 4f7d91723485 fix(qemu): change TOS_FW_CONFIG_NAME value
- fbfc9a222c7f spmd_smc_handler() add s/ns state to SMC traces
- ca65081b9cdc feat(sptool): add dependency to SP image
- b1e1b46a0680 fix(qemu): restore code to added needed psci nodes
For OP-TEE I've also added a few patches, available at
https://github.com/jenswi-linaro/optee_os/tree/qemu_sel2:
- 1057def23777 plat-vexpress: sel2 spmc: update for hafnium
- f18a54ed3524 core: ffa: use hvc instead of smc with S-EL2
- d18bbc92f7c1 core: mobj_ffa_add_pages_at() trust addresses from SPMC
There's also one patch for QEMU on top of v7.0.0, available at:
https://github.com/jenswi-linaro/qemu/tree/qemu_sel2
- 0c1e39672dcb Read PS bits from VTCR_EL2
The QEMU problem is fixed in v.7.1.0, but I can't get that version of
QEMU to work with TF-A. I guess it's because of yet another new CPU
feature since I'm running with "-cpu max".
I'll try to upstream the Hafnium and TF-A patches that make sense on
their own.
What's the plan with the interrupt controller?
How will OP-TEE be able to handle secure interrupts?
The hafnium git pulls in a few git submodules and even the source code
for a Linux kernel.
I guess this is useful in your internal CI setup, but when used
isolated as in my setup it makes no sense at all.
It would also be nice to be able to build with an external toolchain.
I hope this is a temporary situation, I don't see why Hafnium should
be pickier about toolchain than for instance TF-A.
Speaking of building, I haven't been able to figure out how to build
only for the QEMU variant I need so right now I'm building for
everything and that takes a bit longer than necessary.
I'm going to maintain the setup above as long as it's relevant to me. I may
add more patches on the branches or even rebase as needed. So if anyone is
using this, keep in mind that my branches may change without warning.
Thanks,
Jens
Hi,
Current Hafnium build and test configs cover both cases of non-VHE (HCR_EL2.E2H=0) and VHE (HCR_EL2.E2H=1).
From an Arm architecture perspective the latter is a superset of the former.
We get to know that non-VHE becomes a legacy, and other projects in general tend to always enable Armv8.1 VHE extension early at boot.
Our focus being S-EL2 (Armv8.4+) it looks reasonable to assume Armv8.1's VHE is present on chipsets loading and booting Hafnium.
That said we're exploring the possibility to abandon 'non-vhe' builds, and focus build and test on vhe-enabled builds.
The intent to simplify build configurations, and improve build and test time.
Joao made experiments in 2 steps, first on removing the said build configurations, and then runtime checks.
This gives good results at least in terms of build time , and simplification in build/test scripts.
https://review.trustedfirmware.org/c/hafnium/hafnium/+/18925/https://review.trustedfirmware.org/c/hafnium/project/reference/+/18926/
This message is to poll the community for feedback, as to whether this is foreseen as an issue (or not!) for on going deployments, before we engage further in this refactoring.
Thanks & Regards,
Olivier.
Hi,
Thanks for your efforts for this query. I have tried your solution. Although the Hafnium code could be built successfully, it does not work when the boot flow run into the entry of the Hafnium.
I tried to debug the error with trace32 tool. And I confirmed that the Hafnium code has been loaded over 34G of free address space. However, when PC register point to the start address of Hafnium, the code data is displayed as udf #0x0.
It may be a page table problem, and we are still locating the problem reason.
Regards,
Mei, Jianqiang.
------------------------------------------------------------------
发件人：Olivier Deprez via Hafnium <hafnium(a)lists.trustedfirmware.org>
发送时间：2022年12月21日(星期三) 18:02
收件人：梅建强(禹夜) <meijianqiang.mjq(a)alibaba-inc.com>
抄 送：hafnium <hafnium(a)lists.trustedfirmware.org>; 常琳(特睿) <terui.cl(a)alibaba-inc.com>; 赵哲(为哲) <weizhe.zz(a)alibaba-inc.com>; 黄明(连一) <hm281385(a)alibaba-inc.com>
主 题：[Hafnium] Re: Hafnium load-address
Hi,
Sorry for the delay in replying to this query.
After few trials I ended with this possible solution:
https://review.trustedfirmware.org/c/hafnium/hafnium/+/18510 <https://review.trustedfirmware.org/c/hafnium/hafnium/+/18510 >
This should permit loading the Hafnium image to an address up to 48 bits (and by updating project/reference/BUILD.gn origin_address per your suggestion).
It is possible to further extend beyond 48 bits (e.g. 52 bits which is the maximum permitted VA using FEAT_LPA2).
Let me know if this works for you, then I can proceed further with polishing this change.
Regards,
Olivier.
________________________________
From: 梅建强(禹夜) via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 11 December 2022 09:58
To: Olivier Deprez <Olivier.Deprez(a)arm.com>
Cc: hafnium <hafnium(a)lists.trustedfirmware.org>; 常琳(特睿) <terui.cl(a)alibaba-inc.com>; 赵哲(为哲) <weizhe.zz(a)alibaba-inc.com>; 黄明(连一) <hm281385(a)alibaba-inc.com>
Subject: [Hafnium] Hafnium load-address
Hello, expert
Firstly, thanks for your answer some time ago. That's useful to our project.We combined the Hafnium (run at S-EL2) and the optee (run at S-EL1), then modified the configuration to adapt to our own environment. Now we can start the os normally.
Now we want to optimize a technical point, which is to load and run hafnium over 34G of free address space. This requires hafnium to support 64-bit load-address.
I tried to modify the configuration in BUILD.gn as follows:
origin_address = "0x880000000"
And I encountered the build error:
FAILED: secure_aem_v8a_fvp_clang/hafnium.elf ld.lld -pie --gc-sections --defsym=ORIGIN_ADDRESS=0x880000000 -O2 --icf=all --fatal-warnings --color-diagnostics -T /home/yuye/hafnium/Hafnium/build/image/image.ld -o secure_aem_v8a_fvp_clang/hafnium.elf --start-group @secure_aem_v8a_fvp_clang/hafnium.elf.rsp --end-group ld.lld: error: secure_aem_v8a_fvp_clang/obj/src/arch/aarch64/entry.entry.o:(.init.entry+0xC8): relocation R_AARCH64_ABS32 out of range: 36507222016 is not in [-2147483648, 4294967295]; references ORIGIN_ADDRESS >>> defined in -defsym:1 ninja: build stopped: subcommand failed. Makefile:70: recipe for target 'all' failed make: *** [all] Error 1 + cp out/reference/secure_aem_v8a_fvp_clang/hafnium.bin ../../V2/Ali_M1_Public/Nonosi/Platform/Ali/M1/Bin/ cp: cannot stat 'out/reference/secure_aem_v8a_fvp_clang/hafnium.bin': No such file or directory
In this case, it seems that the new address causes some relative addressing to reach too far. Is there any solutions or supports for this issue currently in the community?
Regards,
Mei, Jianqiang.
--
Hafnium mailing list -- hafnium(a)lists.trustedfirmware.org
To unsubscribe send an email to hafnium-leave(a)lists.trustedfirmware.org
--
Hafnium mailing list -- hafnium(a)lists.trustedfirmware.org
To unsubscribe send an email to hafnium-leave(a)lists.trustedfirmware.org
Hi, expert
I want to confirm a problem about using hafnium to load spmc payload.
That is, Dose hafnium supports hyp_loaded for spmc payload or ffa_partion (such as optee_os) in the boot phase?
If the community confirms that it does not support it, I wonder, what might be the problem with doing so?
I see the following code in hanfnium：
if (manifest->vm[i].is_ffa_partition &&
!manifest->vm[i].is_hyp_loaded) {
TRY(parse_ffa_partition_package(stage1_locked, &vm_node,
&manifest->vm[i], vm_id,
ppool));
} else {
TRY(parse_vm(&vm_node, &manifest->vm[i], vm_id));
}
It seems that hafnium has no handling for both is_ffa_partition and is_hyp_loaded.
regards,
yuye
Hi,
Currently each patch submission automatically starts two jenkins jobs for cloning, building, running tests and static checks.
The longest of the two jobs can run for 30mn - 1h depending on server load.
A 10 patches stack queues as many of such jobs and it can be a long process to get all votes with a significant load applied to the servers.
Along with this the Allow-CI+1 label can be used by maintainers to re-trigger a job either because it failed, or the results/logs were flushed.
Per discussion with various stakeholders we come to a conclusion it would be preferable to only use the Allow-CI label and discard the automatic trigger for each and every patch.
Similarly to TF-A, a change submitted by a developer requires a maintainer to apply the Allow-CI+1 label to build and run tests.
For a large patch stack, the expectation is at least the top patch must pass the CI run before merging, but not necessarily all intermediate patches.
It's the maintainer discretion to apply the label at different places in the patch stack to get intermediate results as required.
I intend to submit a change shortly to adopt this new policy.
Let me know if any concern.
Regards,
Olivier.
