Hello All,
We have been working on enabling the use memory management interfaces between SWd and NWd, on Hafnium as SPMC. We identified a few points that would benefit from a general discussion/clarification through the Mailing List for this work to progress on time for the first release.
The whole memory has been mapped to a VM representative of the NWd FFA endpoints (in the code can be found as a global variable called 'other_world_vm') in this patch<https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
All the memory is configured as RW memory, to be permissive enough to ease the validation of the sender's original permissions vs the requested sharing permissions.
This approach seems valid given the following assumptions: the 'other_world_vm' is never scheduled, which means that there is no risk of memory leakage; the hypervisor should do the necessary validation of the permissions that a given NWd FFA endpoint has on a given memory region, before forwarding the handling of the operation to the SPMC.
The first point to bring to discussion is regarding the RW configuration.
Is this really the best configuration we would expect for memory to be shared from NWd to the SWd? Or is this excluding any use-case we are not aware of (e.g. at this point we are restricting instruction permissions to NX)?
Whilst testing the memory management interfaces, I noticed that if the instruction access on the memory region descriptor (in the spec has been described in section 5.12, and in the code defined as 'struct ffa_memory_region') is set as 'not specified' for the Lend and Donate operations, the receiver would acquire permission to execute the memory region. This is due to the implementation of the function 'ffa_memory_permissions_to_mode<https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…><https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>', which sets the memory mode to MM_MODE_X for case instruction access FFA_INSTRUCTION_ACCESS_NOT_SPECIFIED. My understanding is that in such case MM_MODE_X shouldn't be set. Please check this change<https://review.trustedfirmware.org/c/hafnium/hafnium/+/6008>.
@Andrew Walbran<mailto:qwandor@google.com>, is the change I provided correct? Or is there a reason for the current implementation of 'ffa_memory_permissions_to_mode'?
I also noticed that after a memory reclaim, the sender/owner of a memory region acquires RW and X permissions, even if these were not part of the original permissions when sending the memory region. This is due to definition of the variable 'memory_to_attributes' in the function 'ffa_memory_reclaim<https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>':
uint32_t memory_to_attributes = MM_MODE_R | MM_MODE_W | MM_MODE_X;
Down the call stack, the value of this variable is used to update the variable 'to_mode' in function 'ffa_retrieve_check_transition<https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…>'. The value of variable 'to_mode' is later passed to 'ffa_region_group_identity_map'<https://git.trustedfirmware.org/hafnium/hafnium.git/tree/src/ffa_memory.c#n…> to update the owner's memory mapping. My understanding is that the sender's memory mode should be reset to the mode it had prior to the original memory send operation, and that this is going to be a problem for memory management operations (even between between VMs and between SPs). I have seen this behavior on my set, however I would like to validate my analysis.
@Andrew Walbran<mailto:qwandor@google.com> would you be able to provide any comments on this?
Please let me know if anyone has any comments/questions.
Best regards,
João Alves
Hi,
This is to notify that we are planning to target the Trusted Firmware-A 2.4 release during the second week of November 2020 as part of the regular 6 month cadence. The aim is to consolidate all TF-A work since the 2.3 release. As part of this, a release candidate tag will be created and release activities will commence from Friday 30th Oct 2020 (code freeze date). Essentially we will not merge any major enhancements from this date until the release is made. Please ensure any Pull Requests (PR’s) desired to make the 2.4 release are submitted in good time, to be completed by Tuesday 27th Oct 2020. Any major enhancement PR’s still open after that date will not be merged until after the release
Please note, there is no guarantee that all patches under review will be merged before the code freeze date. Any patches that don’t make the code freeze date will complete their review and merge post the release date.
Thanks
Manish Badarkhe
Hi Varun,
Memory footprint will be a function of three build-time constants:
MAX_CPUS, MAX_VMS and HEAP_PAGES. They're defined in the build rule of each
target platform, e.g. for RPi4 it is here:
https://git.trustedfirmware.org/hafnium/project/reference.git/tree/BUILD.gn…
Hope this helps,
David
On Tue, Sep 15, 2020 at 1:08 AM Varun Wadekar via Hafnium <
hafnium(a)lists.trustedfirmware.org> wrote:
> Hi,
>
> We are trying to figure out the memory footprint (.text + .data) for the
> Hafnium image. I couldn't find a place where the number of CPUs are
> specified for a platform. I assume that the footprint will increase as we
> introduce more CPUs in the mix.
>
> Can someone please help me figure this out?
>
> Thanks.
> --
> Hafnium mailing list
> Hafnium(a)lists.trustedfirmware.org
> https://lists.trustedfirmware.org/mailman/listinfo/hafnium
>
Hi,
We are trying to figure out the memory footprint (.text + .data) for the Hafnium image. I couldn't find a place where the number of CPUs are specified for a platform. I assume that the footprint will increase as we introduce more CPUs in the mix.
Can someone please help me figure this out?
Thanks.
Hi Raghu,
Thank you very much for your feedback!
It seems like a good idea to automate it and the approach looks fine to me. Is my understanding correct that the generated *dtsi* files will need to be included in the *dts* files once and the hope is if there is a change, the dts files need not change since the dtsi files will be regenerated every time during build?
Yes, your understanding is correct.
The other way this could be done is that the tool can generate dts files which have nodes that change at build time, which can then be compiled and added as dtb files, added to the FIP. That way fvp_tb_fw_config.dts and fvp_spmc_manifest.dts will remain the same and not have any dependency on a changing dtsi file. I’d prefer that dts files that change frequently like in this case be autogenerated entirely and as separate files. That _seems_ cleaner but may not work out so in practice.
I agree with you that the approach you described above would make the whole process a bit cleaner.
One way this could be done is by passing the *.pre.dts files (generated after the initial processing from dtc) to the script that are part of the patch. Last time I worked on this, I added the possibility to generate the nodes structure to an already existing dts file. Thus, expanding the previous configuration with the values of the new nodes. However, this work needs further testing.
Alternatively, the python package I am using in the script can parse and generate dtb instead of dts. I was thinking to add the option to the dts_gen.py script (maybe rename it as well) to operate over the dtb format. So, making sure that "in-place" changes are working , and adding the option to operate over dtb files should allow for the cleaner approach that you referred. As is and making the referred changes, most of the script's logic should be preserved, so I don't think there will be a lot of changes to the current implementation, which is good. Please let me know if I was not very clear, or if you have any other comments.
Also thanks @Gyorgy Szing<mailto:Gyorgy.Szing@arm.com> for the feedback on the current state of the patch, I appreciate it.
Let me know if anyone has questions, or any other comments to the scripts and/or current discussion.
I will notify once the work progresses.
Best regards,
João Alves
________________________________
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> on behalf of Raghu Krishnamurthy via Hafnium <hafnium(a)lists.trustedfirmware.org>
Sent: Friday, August 7, 2020 5:54 PM
To: tf-a(a)lists.trustedfirmware.org <tf-a(a)lists.trustedfirmware.org>; hafnium(a)lists.trustedfirmware.org <hafnium(a)lists.trustedfirmware.org>
Subject: Re: [Hafnium] [TF-A] Fw: Automate generation of Partition's specific configuration
Hi,
It seems like a good idea to automate it and the approach looks fine to me. Is my understanding correct that the generated *dtsi* files will need to be included in the *dts* files once and the hope is if there is a change, the dts files need not change since the dtsi files will be regenerated every time during build?
The other way this could be done is that the tool can generate dts files which have nodes that change at build time, which can then be compiled and added as dtb files, added to the FIP. That way fvp_tb_fw_config.dts and fvp_spmc_manifest.dts will remain the same and not have any dependency on a changing dtsi file. I’d prefer that dts files that change frequently like in this case be autogenerated entirely and as separate files. That _seems_ cleaner but may not work out so in practice.
Thanks
Raghu
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Joao Alves via TF-A
Sent: Monday, August 3, 2020 10:08 AM
To: tf-a(a)lists.trustedfirmware.org
Subject: [TF-A] Fw: Automate generation of Partition's specific configuration
Hi all,
Forwarding email below as the referred work may be useful/relevant for other parts of the TF-A project as well.
Best regards,
João Alves
_____
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org <mailto:hafnium-bounces@lists.trustedfirmware.org> > on behalf of Joao Alves via Hafnium <hafnium(a)lists.trustedfirmware.org <mailto:hafnium@lists.trustedfirmware.org> >
Sent: Monday, August 3, 2020 5:59 PM
To: hafnium(a)lists.trustedfirmware.org <mailto:hafnium@lists.trustedfirmware.org> <hafnium(a)lists.trustedfirmware.org <mailto:hafnium@lists.trustedfirmware.org> >
Subject: [Hafnium] Automate generation of Partition's specific configuration
Hello all,
I have been trying to ease the process of adding a Secure Partition to a system using Secure Hafnium.
There is no way to automatically generate SP's specific configuration into TF-A's code-base. Considering FVP as the target platform, we need to manually add partition's specific configuration to files "fvp_tb_fw_config.dts" and "fvp_spmc_manifest.dts" (files held in FVP platform specific folder of TF-A codebase). The following snippet shows the hypervisor node from "fvp_spmc_manifest.dts", for the simple case of having in the system two Cactus Secure Partitions:
hypervisor {
compatible = "hafnium,hafnium";
vm1 {
is_ffa_partition;
debug_name = "cactus-primary";
load_address = <0x7000000>;
};
vm2 {
is_ffa_partition;
debug_name = "cactus-secondary";
load_address = <0x7100000>;
vcpu_count = <2>;
mem_size = <1048576>;
};
};
Some of the above properties are available in the partition's manifest, for example "debug_name" and "load_address".
If changing one of these values in the partition's manifest or adding another SP, we also need to update the referred files.
In order to avoid the burden of having to manually update partition's specific configuration and to make whole system more scalable, I started to write a script that is able to generate a specific node structure and fetch any property value from a any dts file. Then, applied it to fetch/generate SPs specific configuration and include it in aforementioned configuration files.
Although it is still a Work In Progress, the work can be found in the patch: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/5150.
The implementation is divided between two scripts:
* "dts_gen.py" - This is a generic solution for the problem. It can fetch/generate/alter any configuration using dts files.
* "sp_dts_gen.py" - Uses the previous command to solve the specific problem regarding SPs specific configuration.
Although is still Work In Progress, I am looking to obtain feedback/reviews from anyone that could be interested in using this implementation.
The above files contain a lot of comments on how to use them, and also describing the implementation.
If the obtained feedback is good, I can work on integrating this in TF-A's build-system.
Let me know if anyone has questions.
Best regards,
João Alves
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org <mailto:Hafnium@lists.trustedfirmware.org>
https://lists.trustedfirmware.org/mailman/listinfo/hafnium
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org
https://lists.trustedfirmware.org/mailman/listinfo/hafnium
On Mon, 3 Aug 2020 at 10:32, Olivier Deprez via Hafnium <
hafnium(a)lists.trustedfirmware.org> wrote:
> Hi,
>
> On a Stage-2 page fault, Hafnium does not report FAR_EL2 to FAR_EL1 in
> inject_el1_exception .
>
> As a consequence EL1 is resumed although it cannot determine the faulting
> address from FAR_EL1 (which likely holds zero or some other remnant value?).
>
> Is this intended?
>
That sounds like a bug.
Fuad, did you work on this? Any idea what's going on here?
(Note, Fuad is away this week but can hopefully reply next week.)
Hi,
It seems like a good idea to automate it and the approach looks fine to me. Is my understanding correct that the generated *dtsi* files will need to be included in the *dts* files once and the hope is if there is a change, the dts files need not change since the dtsi files will be regenerated every time during build?
The other way this could be done is that the tool can generate dts files which have nodes that change at build time, which can then be compiled and added as dtb files, added to the FIP. That way fvp_tb_fw_config.dts and fvp_spmc_manifest.dts will remain the same and not have any dependency on a changing dtsi file. I’d prefer that dts files that change frequently like in this case be autogenerated entirely and as separate files. That _seems_ cleaner but may not work out so in practice.
Thanks
Raghu
From: TF-A <tf-a-bounces(a)lists.trustedfirmware.org> On Behalf Of Joao Alves via TF-A
Sent: Monday, August 3, 2020 10:08 AM
To: tf-a(a)lists.trustedfirmware.org
Subject: [TF-A] Fw: Automate generation of Partition's specific configuration
Hi all,
Forwarding email below as the referred work may be useful/relevant for other parts of the TF-A project as well.
Best regards,
João Alves
_____
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org <mailto:hafnium-bounces@lists.trustedfirmware.org> > on behalf of Joao Alves via Hafnium <hafnium(a)lists.trustedfirmware.org <mailto:hafnium@lists.trustedfirmware.org> >
Sent: Monday, August 3, 2020 5:59 PM
To: hafnium(a)lists.trustedfirmware.org <mailto:hafnium@lists.trustedfirmware.org> <hafnium(a)lists.trustedfirmware.org <mailto:hafnium@lists.trustedfirmware.org> >
Subject: [Hafnium] Automate generation of Partition's specific configuration
Hello all,
I have been trying to ease the process of adding a Secure Partition to a system using Secure Hafnium.
There is no way to automatically generate SP's specific configuration into TF-A's code-base. Considering FVP as the target platform, we need to manually add partition's specific configuration to files "fvp_tb_fw_config.dts" and "fvp_spmc_manifest.dts" (files held in FVP platform specific folder of TF-A codebase). The following snippet shows the hypervisor node from "fvp_spmc_manifest.dts", for the simple case of having in the system two Cactus Secure Partitions:
hypervisor {
compatible = "hafnium,hafnium";
vm1 {
is_ffa_partition;
debug_name = "cactus-primary";
load_address = <0x7000000>;
};
vm2 {
is_ffa_partition;
debug_name = "cactus-secondary";
load_address = <0x7100000>;
vcpu_count = <2>;
mem_size = <1048576>;
};
};
Some of the above properties are available in the partition's manifest, for example "debug_name" and "load_address".
If changing one of these values in the partition's manifest or adding another SP, we also need to update the referred files.
In order to avoid the burden of having to manually update partition's specific configuration and to make whole system more scalable, I started to write a script that is able to generate a specific node structure and fetch any property value from a any dts file. Then, applied it to fetch/generate SPs specific configuration and include it in aforementioned configuration files.
Although it is still a Work In Progress, the work can be found in the patch: https://review.trustedfirmware.org/c/TF-A/trusted-firmware-a/+/5150.
The implementation is divided between two scripts:
* "dts_gen.py" - This is a generic solution for the problem. It can fetch/generate/alter any configuration using dts files.
* "sp_dts_gen.py" - Uses the previous command to solve the specific problem regarding SPs specific configuration.
Although is still Work In Progress, I am looking to obtain feedback/reviews from anyone that could be interested in using this implementation.
The above files contain a lot of comments on how to use them, and also describing the implementation.
If the obtained feedback is good, I can work on integrating this in TF-A's build-system.
Let me know if anyone has questions.
Best regards,
João Alves
--
Hafnium mailing list
Hafnium(a)lists.trustedfirmware.org <mailto:Hafnium@lists.trustedfirmware.org>
https://lists.trustedfirmware.org/mailman/listinfo/hafnium
The static partitioning scheme and option 1 looks fine to me. Option 2 is fine too but not sure I see why a platform would want to or need to customize the ID range.
-----Original Message-----
From: Hafnium <hafnium-bounces(a)lists.trustedfirmware.org> On Behalf Of Andrew Walbran via Hafnium
Sent: Tuesday, August 4, 2020 3:13 AM
To: Olivier Deprez <Olivier.Deprez(a)arm.com>
Cc: hafnium(a)lists.trustedfirmware.org
Subject: Re: [Hafnium] FF-A id ranges
On Mon, 3 Aug 2020 at 18:03, Olivier Deprez via Hafnium < hafnium(a)lists.trustedfirmware.org> wrote:
> Hi,
>
> We consider updating the range of FF-A IDs for Secure Partitions
> running in SWd on top of Hafnium.
>
> Bit 15 in FF-A IDs can serve to hint the world type into which the
> VM/SP resides.
> FF-A id bit 15 clear => id space [1 : 0x7fff] reserved for VMs on top
> of NWd Hafnium FF-A id bit 15 set => id space [0x8001 : 0xffff]
> reserved for SPs on top of SWd Hafnium
>
> 0 reserved to the Hypervisor itself.
> 0x8000 reserved for the TEE (or later for the SPMC).
>
> One way to achieve this is to define a static value for the base of
> FF-A ids, see:
>
> https://review.trustedfirmware.org/c/hafnium/hafnium/+/5165/1/inc/vmap
> i/hf/types.h#30
>
> The intent is to set 0 for the NWd Hypervisor build and 0x8000 for the
> SWd SPMC build.
> 1/ This can be done through the SECURE_WORLD macro (although this
> macro is only available to src/arch/aarch64/* files) 2/ This can be
> passed as a platform parameter in the build flow (e.g. same way as
> max_vms)
>
> There are others ways rather than using id ranges, e.g. setting the
> bit explicitly in FF-A IDs when doing the SPMC build.
>
> Any thoughts?
>
I guess Hafnium will need to know not just what IDs to use for the world it is operating in, but also how to interact with the other world. This seems like an arch-specific concept, so maybe some of the constants currently defined in hf/types should be moved into an aarch64 directory so they can be defined based on the SECURE_WORLD macro? This is essentially your option
1 above.
On Mon, 3 Aug 2020 at 18:03, Olivier Deprez via Hafnium <
hafnium(a)lists.trustedfirmware.org> wrote:
> Hi,
>
> We consider updating the range of FF-A IDs for Secure Partitions running
> in SWd on top of Hafnium.
>
> Bit 15 in FF-A IDs can serve to hint the world type into which the VM/SP
> resides.
> FF-A id bit 15 clear => id space [1 : 0x7fff] reserved for VMs on top of
> NWd Hafnium
> FF-A id bit 15 set => id space [0x8001 : 0xffff] reserved for SPs on top
> of SWd Hafnium
>
> 0 reserved to the Hypervisor itself.
> 0x8000 reserved for the TEE (or later for the SPMC).
>
> One way to achieve this is to define a static value for the base of FF-A
> ids, see:
>
> https://review.trustedfirmware.org/c/hafnium/hafnium/+/5165/1/inc/vmapi/hf/…
>
> The intent is to set 0 for the NWd Hypervisor build and 0x8000 for the SWd
> SPMC build.
> 1/ This can be done through the SECURE_WORLD macro (although this macro is
> only available to src/arch/aarch64/* files)
> 2/ This can be passed as a platform parameter in the build flow (e.g. same
> way as max_vms)
>
> There are others ways rather than using id ranges, e.g. setting the bit
> explicitly in FF-A IDs when doing the SPMC build.
>
> Any thoughts?
>
I guess Hafnium will need to know not just what IDs to use for the world it
is operating in, but also how to interact with the other world. This seems
like an arch-specific concept, so maybe some of the constants currently
defined in hf/types should be moved into an aarch64 directory so they can
be defined based on the SECURE_WORLD macro? This is essentially your option
1 above.
Hi,
We consider updating the range of FF-A IDs for Secure Partitions running in SWd on top of Hafnium.
Bit 15 in FF-A IDs can serve to hint the world type into which the VM/SP resides.
FF-A id bit 15 clear => id space [1 : 0x7fff] reserved for VMs on top of NWd Hafnium
FF-A id bit 15 set => id space [0x8001 : 0xffff] reserved for SPs on top of SWd Hafnium
0 reserved to the Hypervisor itself.
0x8000 reserved for the TEE (or later for the SPMC).
One way to achieve this is to define a static value for the base of FF-A ids, see:
https://review.trustedfirmware.org/c/hafnium/hafnium/+/5165/1/inc/vmapi/hf/…
The intent is to set 0 for the NWd Hypervisor build and 0x8000 for the SWd SPMC build.
1/ This can be done through the SECURE_WORLD macro (although this macro is only available to src/arch/aarch64/* files)
2/ This can be passed as a platform parameter in the build flow (e.g. same way as max_vms)
There are others ways rather than using id ranges, e.g. setting the bit explicitly in FF-A IDs when doing the SPMC build.
Any thoughts?
Regards,
Olivier.