Hi Olivier,
Sorry if my questions below sound obvious, I may miss bits of the VHE
architecture.
[RK] Thanks. Please feel free to push me on these questions. This was done more or less in isolation so I'm happy to be corrected and my understanding checked. Please feel free to suggest alternative paths to take as well.
I guess it requires an additional change in project/reference to
eventually enable the feature for a platform (enable_vhe=1 in the gn build flow)?
[RK] Correct. I had this change locally and did not push it since the default would be to have enable_vhe=0
The changes are effectively toggling HCR_EL2.E2H=1. Currently when
Hafnium ERETs resuming a SP, this happens with the secure EL1&0 translation regime. Though what's the next step forward? Does this require a host OS in secure world? (like linux does when booting under a Hypervisor with VHE enabled?) Would this be a VHE-enabled TOS?
[RK] Hanfium effectively becomes the host OS when we enable E2H, so we don't need anything special. Sorry if this is obvious or you already know - When linux boots with VHE enabled, it runs code written to work on EL1 in EL2 as the host OS and and EL1 register accesses redirect to the equivalent EL2 registers. Hafnium is effectively doing this, except hafnium's code is already written to access the EL2 system registers so I think of hafnium as the host OS. Let me know if that does not make sense, we can discuss further. The next steps answered below.
Is the later goal to enable the secure EL2&0 translation regime? 4/ Is there anything to do with HCR_EL2.TGE?
[RK] EL2&0 translation regime is enabled as soon as we set E2H is set, so the current patches already enable it. Note that I'm only using TTBR0_EL2 and not using TTBR1_EL2 since there is no need for it as of today. I don't expect there to be use for it in the near future either given that we want to use identity mapping everywhere. As for future steps, this is what I'm thinking - note that this needs more thought and exploration and are not fool proof thoughts: 1) Model S-EL0/EL0 partitions as "lightweight" VM's. By lightweight, what I mean is that we represent EL0 partitions using the existing VM and VCPU structures, except these VM's will be forced to only have a maximum of one VCPU, and the context switch for these partitions would be "light weight", ie only GPR's and tbbr0_el2. I have to explore some of the other settings/system registers in EL2 that may have to be set up appropriately and is open. The advantage of doing so is that we can reuse all the loading infrastructure for VM's on EL0 partitions too. 2) As for TGE, HCR_EL2.TGE will be set only for EL0 partitions so that these partitions would become the equivalent of "host" applications. This bit will not be set for S-EL1 partitions/VM's. Effectively, when E2H is set, this bit differentiates between a VM and a host application. 3) There needs to be more thought around how hafnium will handle EL0 partitions vs VM's w.r.t to interrupts since there is no injection of virtual interrupts in this case. Exception handling and TLB maintenance are other area's I need to explore. 4) Anything else that comes up?? I plan to start prototyping this and see where it takes me but at this point it seems achievable without having to break hafnium entirely.
Hi Achin,
Question for you. I interpreted the FF-A 1.0 spec as "requiring" VHE for S-EL0 partitions, based on options 1 and 2 in section 2.2.1, since option 2 does not mention S-EL0 partitions. Can you confirm this is the case? As I was thinking through this, it seems like we might be able to pull off EL0 partitions without using VHE, i.e by using HCR_EL2.E2H = 0 and HCR_EL2.TGE=1. HCR_EL2.TGE is present even on ARMv8.0 architecture and removes dependence on VHE, and effectively removes the necessity for this patch series. However, I'd like to understand your thoughts on why VHE is suggested in the spec and if hafnium should use VHE for S-EL0 partitions or if hafnium can use the approach suggested above.
Thanks Raghu
-----Original Message----- From: Olivier Deprez Olivier.Deprez@arm.com Sent: Tuesday, December 15, 2020 12:18 AM To: hafnium@lists.trustedfirmware.org; raghu.ncstate@icloud.com; Olivier Deprez Olivier.Deprez@arm.com Subject: Re: [Hafnium] VHE support
Hi Raghu,
one more
5/ maybe answer to 2/3/4 is that it requires an EL1-shim embedded into Hafnium which itself ERETs to a S-EL0 partition?
BTW notice my questions are obviously oriented towards the secure implementation.
Regards, Olivier.
________________________________________ From: Hafnium hafnium-bounces@lists.trustedfirmware.org on behalf of Olivier Deprez via Hafnium hafnium@lists.trustedfirmware.org Sent: 15 December 2020 09:10 To: hafnium@lists.trustedfirmware.org; raghu.ncstate@icloud.com Subject: Re: [Hafnium] VHE support
Hi Raghu,
Thanks for sharing this work.
Few thoughts...
1/ I guess it requires an additional change in project/reference to eventually enable the feature for a platform (enable_vhe=1 in the gn build flow)?
Sorry if my questions below sound obvious, I may miss bits of the VHE architecture.
2/ The changes are effectively toggling HCR_EL2.E2H=1. Currently when Hafnium ERETs resuming a SP, this happens with the secure EL1&0 translation regime. Though what's the next step forward? Does this require a host OS in secure world? (like linux does when booting under a Hypervisor with VHE enabled?) Would this be a VHE-enabled TOS?
3/ Is the later goal to enable the secure EL2&0 translation regime?
4/ Is there anything to do with HCR_EL2.TGE?
Regards, Olivier.
________________________________________ From: Hafnium hafnium-bounces@lists.trustedfirmware.org on behalf of Raghu Krishnamurthy via Hafnium hafnium@lists.trustedfirmware.org Sent: 15 December 2020 04:57 To: hafnium@lists.trustedfirmware.org Subject: [Hafnium] VHE support
Hi All,
I have a series of patches pushed to Gerrit at https://review.trustedfirmware.org/c/hafnium/hafnium/+/7599 with topic "vhe_enable". The goal of this patch series is to enable VM's in both secure and normal world to run with VHE enabled(hcr_el2.e2h=1), without breaking any existing functionality. This is expected to be the first step in the long term goal of enabling S-EL0 partitions(and optionally EL0 partitions), that require VHE support, per the FF-A 1.0 Spec. I'd appreciate feedback on the patches and approach taken to nominally enabling VHE. Note that the FF-A 1.0 spec(AFAIK) does not expect VHE support in the normal world but this patch series enables it anyway due to the wealth of available tests in the hafnium test suite to help with providing confidence in the implementation.
The patch series has been tested as follows:
Hafnium tests using QEMU(prebuilt in the hafnium repo) - Without VHE, since the prebuilt QEMU does not support VHE.
Hafnium tests using QEMU(5.2-RC4, built from source) - With and without VHE, this version of QEMU supports VHE.
Hafnium tests using FVP 11.12.28 - With and Without VHE.
TFTF tests for secure hafnium using FVP 11.12.28 - With and without VHE.
Thanks
Raghu
-- Hafnium mailing list Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium -- Hafnium mailing list Hafnium@lists.trustedfirmware.org https://lists.trustedfirmware.org/mailman/listinfo/hafnium