Hi all,
I am running Hafnium on FVP, with Cactus SP in SEL1 and KVM VHE enabled.
Sometimes I send the "FFA_MSG_SEND_DIRECT_REQ" smc call in KVM (I fill
0x8400006f in x0, then VMID and SP ID in x1, let x2 as 0). It says
assert failed, like this:
ASSERT: lib/el3_runtime/aarch64/context_mgmt.c:651
BACKTRACE: START: assert
0: EL3: 0x4005cac
1: EL3: 0x400323c
2: EL3: 0x400620c
3: EL3: 0x400e180
4: EL3: 0x4005a94
BACKTRACE: END: assert
I notice that when services/std_svc/spmd/spmd_main.c sends the FFA
call (from NS to S) via "spmd_smc_forward(smc_fid, secure_origin,x1,
x2, x3, x4, handle)", it will go to
cm_el1_sysregs_context_restore(secure_state_out) and
cm_el2_sysregs_context_restore(secure_state_out), then it will assert
the cm_get_context(). it gets the NULL context, so assert failed.
Such assert is NOT ALWAYS failed, but I still want to solve this problem.
Since I have modified many lines of code in Hafnium and Cactus SP, I
cannot show them here. Can you give me some advice on debugging? e.g.,
where should I check?
Hi Xiangyi Xu
For some reason your email was discarded by mailman, did you miss registering to the list?
See few comments inline [OD]
Regards,
Olivier.
________________________________________
From: xiangyi xu <xuxiangyi666(a)gmail.com>
Sent: 27 November 2021 14:13
To: hafnium(a)lists.trustedfirmware.org
Subject: Virtualising OP-TEE with Hafnium at S-EL2
HI ALL:
I am trying to setup Hafnium environment which loads OP-TEE in Security World and Linux in Normal World. I follow this instruction: https://lists.trustedfirmware.org/pipermail/hafnium/2021-January/000130.html. I can load the OP-TEE test successfully. But the BL33 payload is TF-A-tests example (BL33=../tf-a-tests/build/fvp/debug/tftf.bin). Could you share with us the tutorial to boot linux including the FF-A driver in the NS world while loading OP-TEE with Hafnium S-EL2? The presentation PPT is here: https://static.linaro.org/connect/lvc21/presentations/lvc21-305.pdf.
[OD] The mentioned presentation is in context of the Total Compute platform (as opposed to FVP).
It works great for what you want to do, and you can reproduce by following the user manual:
https://gitlab.arm.com/arm-reference-solutions/arm-reference-solutions-docs…
There is also work in progress at Linaro to provide a similar environment using the FVP.
@Jens, are there public instructions on how to build the end to end scenario targeting the FVP incl. Linux, FF-A driver, TF-A, Hafnium, OP-TEE ?
If not, I reckon this is in your mid term roadmap?
Secondly, I can boot Linux with QEMU and Hafnium. But it seems Hafnium only supports NS world partition while using QEMU. Is any instruction to boot OP_TEE in Security world with Hafnium S-EL2. Since, QEMU 6.0 already supports S-EL2. However, I could not compile ATF successfully with SPD=spmd CTX_INCLUDE_EL2_REGS=1 PLAT=qemu ...
[OD] qemu is not 'officially' supported for booting Hafnium at S-EL2. Although I have few changes in progress which may help. This is not a priority from Arm perspective, but let me come back on this.
By following https://teaclave.apache.org/trustzone-sdk-docs/getting-started-with-optee-f… , I can boot Linux in NS and OP-TEE in Trustzone with QEMU, but it doesn't work when I add Hafnium. Thanks!
Xiangyi Xu
Hi all,
I try to use Hafnium (deployed in SEL2) to access a DRAM address, e.g.
0xa000_0000, but failed.
I think the TTBR0_EL2 does not map this address space, or something wrong
with TZASC.
To solve this problem, I first add the DRAM region in TZASC and TTBR0_EL3
(by modifying TF-A), and EL3 can access this address. But Hafnium cannot
read or write it.
So, must I modify the .S files in Hafnium to add a page mapping to the DRAM
space? Can I add this page in some C files? Or should I do something else?
Sincerely,
Wang Chenxu
Hi Raghu,
Thanks for reporting.
This part of the test infrastructure (testing the SPMC) is still very fresh and requires improvement iterations so please bear with us. Also a reason it's not yet part of the automated non-regression with jenkins (as opposed to the legacy kokoro/test.sh). For the time being we still mostly rely on the TF-A CI for testing on the secure side.
IIUC this change was made to help with the test time as the FVP takes long to reload on every test.
But indeed it might have the side effect you describe.
So either we revert the FVP reloading on every test.
Or another (somewhat hackish) possibility is to clear the mentioned variables from within the test (or make them part of BSS)?
To be fair, the both worlds test scenario is not 100% stable on my machine (for some reason the connection is not always successful between the FVP and hftest) hence limiting confidence/robustness of my testing and investigations. So I wonder is the scripting is still somewhat a bit fragile.
Regards,
Olivier.
________________________________________
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of Raghu Krishnamurthy via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 03 August 2021 23:47
To: 'Raghu Krishnamurthy via Hafnium'
Subject: [Hafnium] Bug in hftest.py
Hi All,
Wanted to report to you that commit 18a25f9241f86ba2d637011ff465ce3869e8651b
in hafnium "appears" broken. The issue with the optimization in this patch
is that the partition images are not reloaded for each test run, which means
a previous test could have written data to say SRAM, and the following test
would use the old values from the previous test, when the same image is
executed again from SRAM for a following test. This would be a problem for
pretty much anything in the data section of a partition. In my case, I have
a counter in the data section of my partition, which does not get reset back
to its original value.
I've attached a patch to help repro the issue. Fix is to disable the
optimization or somehow reload the images for each run. This affects only
"both world" tests.
Let me know if I'm missing something here.
Apply patch and run timeout --foreground 300s ./test/hftest/hftest.py
--out_partitions out/reference/secure_aem_v8a_fvp_vm_clang --log
out/reference/kokoro_log --spmc
out/reference/secure_aem_v8a_fvp_clang/hafnium.bin --driver=fvp --hypervisor
out/reference/aem_v8a_fvp_clang/hafnium.bin --partitions_json
test/vmapi/ffa_secure_partitions/ffa_both_world_partitions_test.json
The command line is from kokoro/test_spmc.sh.
Thanks
Raghu
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/hafnium
Hi All,
Wanted to report to you that commit 18a25f9241f86ba2d637011ff465ce3869e8651b
in hafnium "appears" broken. The issue with the optimization in this patch
is that the partition images are not reloaded for each test run, which means
a previous test could have written data to say SRAM, and the following test
would use the old values from the previous test, when the same image is
executed again from SRAM for a following test. This would be a problem for
pretty much anything in the data section of a partition. In my case, I have
a counter in the data section of my partition, which does not get reset back
to its original value.
I've attached a patch to help repro the issue. Fix is to disable the
optimization or somehow reload the images for each run. This affects only
"both world" tests.
Let me know if I'm missing something here.
Apply patch and run timeout --foreground 300s ./test/hftest/hftest.py
--out_partitions out/reference/secure_aem_v8a_fvp_vm_clang --log
out/reference/kokoro_log --spmc
out/reference/secure_aem_v8a_fvp_clang/hafnium.bin --driver=fvp --hypervisor
out/reference/aem_v8a_fvp_clang/hafnium.bin --partitions_json
test/vmapi/ffa_secure_partitions/ffa_both_world_partitions_test.json
The command line is from kokoro/test_spmc.sh.
Thanks
Raghu
Hi Andrew,
I don't think Hafnium implements the different cacheability and shareability types for memory sharing at all does it?
[JA] No, it doesn't. At least that is my understanding as well. I noticed mostly due to lack of support in the mm library. Asking was a means to confirm.
The point (at least for now) was just about validation of the respective fields in the memory transaction descriptor.
Thank you for this! 🙂
Best regards,
João
________________________________
From: Andrew Walbran
Sent: Wednesday, July 14, 2021 5:29 PM
To: Joao Alves
Cc: hafnium(a)lists.trustedfirmware.org; Olivier Deprez; Achin Gupta; Mahesh Reddy Bireddy; Jaykumar Pitambarbhai Patel
Subject: Re: Hafnium - Memory attributes precedence checks for mem share
I don't think Hafnium implements the different cacheability and shareability types for memory sharing at all does it? We just didn't have a need for it, if you want to add support that should be fine.
This is mentioned in https://developer.trustedfirmware.org/T827, you can assign that to yourself if you want to take it on.
On Wed, 14 Jul 2021 at 13:50, Joao Alves <Joao.Alves(a)arm.com<mailto:Joao.Alves@arm.com>> wrote:
Hi Andrew,
We have been revising some aspects of the memory sharing implementation. The specification describes a set of precedence rules for the memory attributes specified in the memory transaction descriptor, including: Memory type, cacheability, shareability.
The sender would fill the memory attributes for the region to be shared. After memory send, the receiver should retrieve the regions, filling the memory attributes on its transaction descriptor that comply with the referred precedence rules.
The referred rules can be found in section 10.10.4 of the newly release FF-A v1.1 beta spec<https://developer.arm.com/documentation/den0077/c/?lang=en>, as follows.
Memory type precedence rules ( < reads as is less permissive than):
* Device-nGnRnE < Device-nGnRE < Device-nGRE < Device-GRE < Normal
Cacheability precedence rules:
* Non-cacheable < Write-Back Cacheable
Shareability precedence rules:
* Non-Shareable < Inner Shareable < Outer shareable
These checks are not part of the handling of FFA_MEMORY_RETRIEVE_REQ.
Was there an implementation defined reason for this? If so, could you please provide the rationale?
Thank you in advance for your help.
Best regards,
João Alves
Hi Andrew,
We have been revising some aspects of the memory sharing implementation. The specification describes a set of precedence rules for the memory attributes specified in the memory transaction descriptor, including: Memory type, cacheability, shareability.
The sender would fill the memory attributes for the region to be shared. After memory send, the receiver should retrieve the regions, filling the memory attributes on its transaction descriptor that comply with the referred precedence rules.
The referred rules can be found in section 10.10.4 of the newly release FF-A v1.1 beta spec<https://developer.arm.com/documentation/den0077/c/?lang=en>, as follows.
Memory type precedence rules ( < reads as is less permissive than):
* Device-nGnRnE < Device-nGnRE < Device-nGRE < Device-GRE < Normal
Cacheability precedence rules:
* Non-cacheable < Write-Back Cacheable
Shareability precedence rules:
* Non-Shareable < Inner Shareable < Outer shareable
These checks are not part of the handling of FFA_MEMORY_RETRIEVE_REQ.
Was there an implementation defined reason for this? If so, could you please provide the rationale?
Thank you in advance for your help.
Best regards,
João Alves
Hi,
>> @Arun, your view on how those two items were solved is beneficial to further elaborate our plans.
@Arunachalam Ganapathy your comments on this topic would be very helpful.
Thanks.
-----Original Message-----
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> On Behalf Of Varun Wadekar via Hafnium
Sent: Monday, May 31, 2021 1:49 PM
To: Olivier Deprez <Olivier.Deprez(a)arm.com>; hafnium(a)lists.trustedfirmware.org; Arunachalam Ganapathy <Arunachalam.Ganapathy(a)arm.com>
Cc: Bo Yan <byan(a)nvidia.com>
Subject: Re: [Hafnium] .git submodules increase hafnium code size
External email: Use caution opening links or attachments
Hi Olivier,
Thanks for answering my queries.
We are looking to deploy the following use case at NVIDIA.
<snip>
-ability to build only the SPMC (not all reference targets such as qemu, rpi4, fvp) -A distribution only requiring the Hypervisor/SPMC output binary ("out/reference/.../hafnium.bin") using any toolchain (be it arm64 or x86 host, and arbitrary clang version).
<snip>
>> As you noticed, the Hafnium Hypervisor/SPMC and test environment builds are closely coupled by the use of ninja/gn flow and scripts. We intend to approach those problems in the course of Q3 in Arm OSS roadmap.
[VW] Are there any local changes to decouple hafnium from its dependencies? We can evaluate Arm;s approach against what we use internally. Our changes moved the dependencies out of the tree and passed file locations to the build system with the help of command line arguments.
-Varun
-----Original Message-----
From: Olivier Deprez <Olivier.Deprez(a)arm.com>
Sent: Monday, May 31, 2021 11:03 AM
To: hafnium(a)lists.trustedfirmware.org; Varun Wadekar <vwadekar(a)nvidia.com>; Arunachalam Ganapathy <Arunachalam.Ganapathy(a)arm.com>
Cc: Bo Yan <byan(a)nvidia.com>
Subject: Re: .git submodules increase hafnium code size
External email: Use caution opening links or attachments
Hi Varun,
We had similar requests raised internally.
1- First in context of Total Compute delivery from Arm OSS platforms:
a. ability to build only the SPMC on TC0 platform (not all reference targets such as qemu, rpi4, fvp)
b. use a Yocto provided toolchain.
@Arun, your view on how those two items were solved is beneficial to further elaborate our plans.
2- A similar request as 1.b to build Hafnium as part of a distribution on arm64 host: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdeveloper…
In my view there are two consumers:
-A distribution only requiring the Hypervisor/SPMC output binary ("out/reference/.../hafnium.bin") using any toolchain (be it arm64 or x86 host, and arbitrary clang version).
-The Hf CI framework/automation needs the above, plus the test framework and tests (dependency on googletest, linux submodules etc). It's important to keep this item alive while trying to solve above item.
As you noticed, the Hafnium Hypervisor/SPMC and test environment builds are closely coupled by the use of ninja/gn flow and scripts.
They are using a fixed toolchain version through prebuilts to ensure builds are "reproducible", in particular with regards to the Hafnium CI.
We intend to approach those problems in the course of Q3 in Arm OSS roadmap.
As an early exploration we already have:
-clang 12 compiler upgrade. This is necessary if wiling to use any arbitrary clang version:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…
-Ability to build on arm64 host (done, internally).
-Identify the flow/script changes such that external dependencies can be used (on-going, internally).
I thought of localizing common dependencies to python/shell scripts by the use of definition files included in the mentioned scripts. This is only an early investigation, I will check how this intersects the changes you provided.
Regards,
Olivier.
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of Varun Wadekar via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 28 May 2021 16:47
To: hafnium(a)lists.trustedfirmware.org <hafnium(a)lists.trustedfirmware.org>
Cc: Bo Yan <byan(a)nvidia.com>
Subject: [Hafnium] .git submodules increase hafnium code size
Hi,
We at NVIDIA are evaluating Hafnium. During the initial investigation, we found out that the repository size (in terms of MB) is huge. This is mostly because of the "git submodules" used by the project. This is a great way to deliver Hafnium with its dependencies in one go.
But we think that the size can be trimmed by moving the toolchain, linux folder, googletest and dtc compiler out, leaving just the Hafnium code in the project. This way, companies like us can pick and choose instead of having to use everything. In a bid to ease the pain internally and only use the Hafnium code base we have crafted the following changes:
1. hafnium: support external projects (I10a07de3) * Gerrit Code Review (trustedfirmware.org)<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…>
2. hafnium: build with dtc and googletest out of tree (I057c9ad6) * Gerrit Code Review (trustedfirmware.org)<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…>
3. build: support external toolchain (Iafd029c1) * Gerrit Code Review (trustedfirmware.org)<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Freview.tr…>
This series does not have the patch to use an out of tree linux codebase. I assume these patches wont be acceptable in their current state, so would like to know how the community plans to handle this situation.
The code size is a real concern for us, as we already have copies of the dependencies in our codebase, so have no use for these duplicates.
Thanks.
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.tru…
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.tru…
Hi Varun,
We had similar requests raised internally.
1- First in context of Total Compute delivery from Arm OSS platforms:
a. ability to build only the SPMC on TC0 platform (not all reference targets such as qemu, rpi4, fvp)
b. use a Yocto provided toolchain.
@Arun, your view on how those two items were solved is beneficial to further elaborate our plans.
2- A similar request as 1.b to build Hafnium as part of a distribution on arm64 host: https://developer.trustedfirmware.org/T898
In my view there are two consumers:
-A distribution only requiring the Hypervisor/SPMC output binary ("out/reference/.../hafnium.bin") using any toolchain (be it arm64 or x86 host, and arbitrary clang version).
-The Hf CI framework/automation needs the above, plus the test framework and tests (dependency on googletest, linux submodules etc). It's important to keep this item alive while trying to solve above item.
As you noticed, the Hafnium Hypervisor/SPMC and test environment builds are closely coupled by the use of ninja/gn flow and scripts.
They are using a fixed toolchain version through prebuilts to ensure builds are "reproducible", in particular with regards to the Hafnium CI.
We intend to approach those problems in the course of Q3 in Arm OSS roadmap.
As an early exploration we already have:
-clang 12 compiler upgrade. This is necessary if wiling to use any arbitrary clang version:
https://review.trustedfirmware.org/q/topic:%22od%252Fhf-clang12%22+(status:…
-Ability to build on arm64 host (done, internally).
-Identify the flow/script changes such that external dependencies can be used (on-going, internally).
I thought of localizing common dependencies to python/shell scripts by the use of definition files included in the mentioned scripts. This is only an early investigation, I will check how this intersects the changes you provided.
Regards,
Olivier.
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of Varun Wadekar via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: 28 May 2021 16:47
To: hafnium(a)lists.trustedfirmware.org <hafnium(a)lists.trustedfirmware.org>
Cc: Bo Yan <byan(a)nvidia.com>
Subject: [Hafnium] .git submodules increase hafnium code size
Hi,
We at NVIDIA are evaluating Hafnium. During the initial investigation, we found out that the repository size (in terms of MB) is huge. This is mostly because of the "git submodules" used by the project. This is a great way to deliver Hafnium with its dependencies in one go.
But we think that the size can be trimmed by moving the toolchain, linux folder, googletest and dtc compiler out, leaving just the Hafnium code in the project. This way, companies like us can pick and choose instead of having to use everything. In a bid to ease the pain internally and only use the Hafnium code base we have crafted the following changes:
1. hafnium: support external projects (I10a07de3) * Gerrit Code Review (trustedfirmware.org)<https://review.trustedfirmware.org/c/hafnium/hafnium/+/10142>
2. hafnium: build with dtc and googletest out of tree (I057c9ad6) * Gerrit Code Review (trustedfirmware.org)<https://review.trustedfirmware.org/c/hafnium/hafnium/+/10144>
3. build: support external toolchain (Iafd029c1) * Gerrit Code Review (trustedfirmware.org)<https://review.trustedfirmware.org/c/hafnium/hafnium/+/10145>
This series does not have the patch to use an out of tree linux codebase. I assume these patches wont be acceptable in their current state, so would like to know how the community plans to handle this situation.
The code size is a real concern for us, as we already have copies of the dependencies in our codebase, so have no use for these duplicates.
Thanks.
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/hafnium